Verified errors in high-impact papers: write-ups
2026-10-23 22:23 UTC. 134 findings, each documented from the paper's own text and numbers (and, for trials, the registry's version history), with a runnable script and figures. Wording is factual and makes no claim about intent; none of these has yet been raised with the authors or journals.
How each one was checked. Numerical errors were found by a cheap full-paper screen (Claude Haiku 5.5 or GPT-6 Luna), confirmed by two cheap verifiers and then by Opus 5.5 and GPT-6.1 Sol independently. Registry discrepancies were flagged by GPT-6 Luna and Claude Haiku 5.5 comparing each trial report with its ClinicalTrials.gov record and confirmed by Opus. For this page each was checked again by agents with Python and web access: the publisher's version of record and supplements; Crossref, Europe PMC, PubPeer and Retraction Watch for corrections; for trials, the registry's version history (what was registered before enrollment and at publication), the paper's role and any disclosure; and a blind re-check by an agent that never saw the earlier verdicts. A separate agent then fact-checked each write-up, re-running the script, verifying every quote and date, and inspecting every figure; write-ups it rejected were revised up to twice.
Severity (from a separate consequence assessment): 5 = a headline conclusion or decision-relevant estimate is wrong (for trials: the conclusion rests on an outcome that was not the registered primary); 4 = a key result is wrong but the main conclusion survives; 3 = a secondary result; 2 = a misstatement a careful reader would catch; 1 = cosmetic.
Index
Numerical and internal-consistency errors (123)
Undisclosed differences from the trial registration (11)
2 verified finding(s) are held back because their write-up did not pass the fact-check after two revisions: PMC6878715, PMC12757054.
SIR and SMR confidence intervals are 34–42 times too narrow for the observed counts
J Hepatol (2020) · doi:10.1016/j.jhep.2020.07.040 · PMC7395653
Summary
The paper compares 111 COVID-19 cases and 20 deaths among 13,255 Spanish liver transplant (LT) recipients with the age- and gender-matched general population. It reports a standardised incidence ratio (SIR, ×100) of 191.22 (95% CI 190.28–192.16) and a standardised mortality ratio (SMR, ×100) of 95.55 (95% CI 94.25–96.85), with intervals of the same width for the over-60 subgroup. The Methods say the intervals treat the observed count as Poisson. With 111 cases and 20 deaths that method gives 157.3–230.3 for the SIR and 58.4–147.6 for the SMR. Every other standard method computed from the paper's own strata gives a similarly wide interval. The published widths instead correspond to Poisson counts of about 153,000–159,000 and 20,750 events. Supplementary Table S2 shows where the SMR width comes from: a projected count of 20,741 deaths, obtained by applying the LT stratum case-fatality to the general population's cases. The SIR still exceeds 100 once corrected. The SMR interval, however, includes 100. The paper's repeated statement that LT recipients' mortality is lower than in the matched general population, and its suggestion that immunosuppression may be protective, therefore lack statistical support in its own data.
What the paper reports
Patients and methods, "Data sources, calculation of standardised ratios, and statistical analysis":
Expected incidence and outcomes in LT patients were obtained from the numbers observed in the general population, after balancing for age and gender. Ninety-five percent confidence intervals (95% CI) were calculated under the assumption that the number of cases observed followed a Poisson distribution.
Abstract, Results:
A total of 111 liver transplant patients were diagnosed with COVID-19 (SIR = 191.2 [95% CI 190.3–192.2]).
The mortality rate was 18%, which was lower than in the matched general population (SMR = 95.5 [95% CI 94.2–96.8]).
Results, "Standardised incidence of COVID-19 in LT patients":
After adjusting for age and gender, the number of observed cases almost doubled the expected number of cases (Table S1) thus resulting in a standardised incidence ratio of 191.22 (95% CI 190.28–192.16). In the subset of patients older than 60 years, the standardised incidence ratio was even higher (206.48; 95% CI 205.45–207.52).
Results, "Standardised mortality ratio of COVID-19 in LT patients":
After adjusting for age and gender, the number of observed deaths among LT patients was slightly lower than the expected number of deaths (Table S2), thus resulting in a standardised mortality ratio of 95.55 (95% CI 94.25–96.85).
Adjusted mortality rates in patients older than 60 were similar in LT recipients and in the general population (standardised mortality ratio = 100.29; 95% CI 98.93–101.66).
Supplementary Tables S1 and S2 (mmc1.pdf) print the following at the foot of each table:
Observed 280,501; Expected 146,690 … 191.22 (95%CI 190.28-192.16)
Observed 20,741; Expected 21,707 … 95.55 (95%CI 94.25-96.85)
The conclusions that rest on the SMR interval. From the Abstract, Conclusions:
Being chronically immunosuppressed, liver transplant patients have an increased risk of acquiring COVID-19 but their mortality rates are lower than the matched general population.
From the Lay summary:
In liver transplant patients, chronic immunosuppression increases the risk of acquiring COVID-19 but it could reduce disease severity.
From the Highlights:
Mortality rates are lower than those observed in the matched general population.
From the Discussion:
we demonstrated that standardised mortality rates were actually lower in LT patients as compared with the general population
From the Discussion, concluding paragraph:
Mortality rates were lower than those observed in the age- and gender-matched general population, thereby suggesting that chronic immunosuppression could exert a certain protective effect against the most severe forms of COVID-19.
What the paper's own numbers show
Method. All ratios are written ×100, as in the paper. I recomputed each interval in three ways.
- The stated method. I took the observed count O (111 cases, 92 cases over 60 from Table S1, and 20 deaths, all over 60). The expected count E is implied by the reported ratio, E = O / (ratio/100). I then computed the exact (Garwood) Poisson 95% interval for O and divided it by E. Byar's approximation and the log-normal approximation agree to within about 3 units.
- Direct standardisation from the strata. I entered the 12 age × gender strata of Tables S1–S2 and checked how the point estimates were built. The published ratios are reproduced by direct standardisation: LT stratum rates are applied to the general-population structure, and the resulting projected count (the tables' "Observed" 280,501 and 20,741) is divided by the general-population count. For that estimator, the variance comes from the LT stratum counts, Σ(wᵢ²·xᵢ). I give log-scale intervals, plus Fay–Feuer gamma intervals as a sensitivity check.
- Indirect standardisation from the same strata, i.e. the conventional SIR/SMR (O/E, with E taken from the general-population stratum rates), with exact Poisson intervals.
Finally, I asked which Poisson count would produce each published width: N = (1.96 × ratio / half-width)².
Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Crude LT incidence | Results | 837.41 per 10⁵ (111 of 13,255) | 837.42 per 10⁵ |
| SIR point estimate | Abstract; Results; Table S1 | 191.22 ("Observed 280,501; Expected 146,690") | Direct standardisation from the strata: projected 280,556 cases / 146,690 = 191.26 |
| SIR, 95% CI | Abstract; Results; Table S1 | 190.28–192.16 (half-width 0.49% of the estimate) | Stated method (O = 111, E = 58.05): 157.3–230.3, 39 times as wide. Direct standardisation: 142.0–257.7 (Fay–Feuer 138.5–271.9). Indirect: 177.9 (146.3–214.2) |
| SIR >60, point and 95% CI | Results | 206.48 (205.45–207.52) | Stated method (O = 92, E = 44.56): 166.5–253.2, 42 times as wide. Direct: 206.61 (160.1–266.6; Fay–Feuer 157.3–269.7). Indirect: 194.9 (157.1–239.0) |
| SMR point estimate | Abstract; Results; Table S2 | 95.55 ("Observed 20,741; Expected 21,707") | Projected deaths 20,740.75 / 21,707 = 95.55 |
| SMR, 95% CI | Abstract; Results; Table S2 | 94.25–96.85 | Stated method (O = 20, E = 20.93): 58.4–147.6, 34 times as wide, includes 100. Direct: 57.8–158.0 (Fay–Feuer 53.7–310.3). Indirect: 97.3 (59.5–150.3) |
| SMR >60, point and 95% CI | Results | 100.29 (98.93–101.66) | 20,740.75 / 20,681 = 100.29. Stated method (O = 20, E = 19.94): 61.3–154.9. Direct: 60.7–165.8. Indirect: 99.3 (60.6–153.3) |
| Poisson count implied by each published width | derived | — | SIR 158,967; SIR >60 152,887; SMR 20,753; SMR >60 20,737 (the observed counts are 111, 92, 20, 20) |
| Published SMR intervals, reproduced | Table S2 | 94.25–96.85; 98.93–101.66 | Poisson on the projected 20,741 deaths: 94.25–96.85 and 98.93–101.65 |
| Published SIR >60 interval, reproduced | Results | 205.45–207.52 | Poisson on the projected >60 count (152,770): 205.44–207.52 |
| Published overall SIR interval | Table S1 | 190.28–192.16 | Not reproduced exactly. Poisson on 280,501 gives 190.51–191.93; on the 146,690 general-population cases, 190.24–192.20 |
Three of the four published intervals are reproduced, to within rounding, by treating a projected population-scale count as if it were the observed Poisson count. The fourth (overall SIR) has the same order of width. Its exact source could not be identified, but it corresponds to roughly 159,000 events. In every case the actual information is 111 cases or 20 deaths.
Two smaller inconsistencies in the supplement, neither of which changes any conclusion:
- In Table S1 the general-population strata sum to 48,640,775, while the printed overall population is 47,026,208. The printed crude incidence of 311.93 per 10⁵ uses the latter.
- In Table S2 the general-population deaths sum to 21,717, which matches the Results text. The SMR, however, uses "Expected 21,707": 20,741/21,707 = 95.55, while 20,741/21,717 = 95.50.
Figure
Figure 1. Reported and recomputed 95% confidence intervals for the four standardised ratios (×100), on one log-scale axis. The thin vertical line marks a ratio of 100, meaning no difference from the matched general population. Orange shows the values printed in the paper; their intervals are narrower than the marker. Filled blue shows the paper's stated method: an exact Poisson interval for the observed count (111 cases, 92 cases over 60, 20 deaths), scaled by the expected count implied by the reported ratio. Open blue squares show the interval for the directly standardised ratio, computed from the 12 strata in Supplementary Tables S1–S2 with the variance of the LT stratum counts. Both recomputations of the SIR stay above 100. Both recomputations of the SMR, overall and over 60, include 100.
Figure 2. Where the published widths come from. The gray line is the relation between a Poisson count N and the relative half-width of its 95% interval, 1.96/√N. Each reported interval (orange) is placed at the count its width implies: about 20,750 events for both SMR intervals and 152,887–158,967 events for the SIR intervals. Those values match the projected counts printed or implied in Tables S1–S2 (20,741 deaths; 152,770 cases over 60). The counts actually observed (blue) are 111 and 92 cases and 20 deaths. For these, the relative half-width is about 18–20% for the SIRs and about 44% for the SMR.
Why it matters
The incidence finding survives correction. The corrected SIR is about 1.9, with a 95% CI of about 1.57–2.30 under the stated method and about 1.42–2.58 for the directly standardised estimator. That still shows a higher incidence among LT recipients. The paper overstates its precision by a factor of about 40, but the direction and significance of the result do not change.
The mortality finding does not survive. Under every method computed here, the SMR interval runs from about 54–60 to 148–310, so it is consistent with mortality anywhere from roughly 40% lower to substantially higher than in the matched general population. The over-60 SMR, which the paper itself describes as "similar", has an interval of about 61–155. In short, 20 deaths against about 21 expected cannot distinguish LT recipients from the matched population.
The statement that mortality is "lower than in the matched general population" appears in five places: the Abstract Results, the Abstract Conclusions, a Highlight, the Discussion ("we demonstrated that standardised mortality rates were actually lower") and the concluding paragraph. The concluding paragraph uses it to suggest that "chronic immunosuppression could exert a certain protective effect", and the Lay summary says that immunosuppression "could reduce disease severity". These statements rest on an interval (94.25–96.85) that excludes 100 only because its variance comes from a projected count of 20,741 rather than from the 20 deaths observed. The paper is a widely cited early report on COVID-19 in transplant recipients (Europe PMC lists 260 citing records), so the unsupported claim about mortality and immunosuppression could have reached clinical readers.
Explanations considered
- Ratios written ×100. 191.22 means 1.9122. Scaling changes the point estimate and the interval in proportion, so it leaves the relative width (0.49%) unchanged. It does not explain the narrow intervals.
- A typographical or rounding slip. All four intervals are affected. The pattern is consistent across them, and three of the four are reproduced to within 0.01 by a single calculation (Poisson on a projected count). This is a systematic method choice, not a slip.
- Text conversion error. The same values appear in the PMC text, the Europe PMC XML and the publisher's typeset supplement PDF (mmc1.pdf), so a conversion error is ruled out.
- The interval reflects only the uncertainty of the general-population reference rates. That uncertainty is small, because the general population had 146,690 cases. But the Methods say the interval is based on the observed count being Poisson. Also, the published SMR width matches the projected LT count (20,741), not the general-population deaths (21,717). An interval that ignores the variability of the 111 or 20 LT events does not describe uncertainty in the LT ratio in any case.
- Direct rather than indirect standardisation. The point estimates are directly standardised, although the Methods' wording and the labels "Observed/Expected" suggest an indirect SIR/SMR. A directly standardised ratio has a valid variance, Σ(wᵢ²·xᵢ) over the LT strata. Computed from Tables S1–S2, that variance gives intervals wider than the stated-method intervals (Figure 1). The indirect version gives similar intervals (SIR 146.3–214.2, SMR 59.5–150.3). Neither choice yields intervals of ±0.5%–1.4%.
- A different confidence level. At these counts, the published widths correspond to confidence levels of about 4–5% (3.8%–4.9%), and the paper says 95% throughout.
None of these explanations accounts for the numbers.
Status
As of 2026-10-23:
- Version of record. The publisher's site (sciencedirect.com and journal-of-hepatology.eu) returned HTTP 403, and doi.org redirects only to a linkinghub stub. I therefore checked the publisher-deposited text in PubMed Central and Europe PMC. Both carry the values quoted above. Crossref lists the article as J Hepatol 2021;74(1):148–155; it was published online in 2020.
- Supplement. The publisher's supplement mmc1.pdf (md5 05e647ab36152f45688c49925b4052af, from the Europe PMC supplementary-files archive) prints the same SIR and SMR intervals under Tables S1 and S2. It also gives the stratum data used for every recomputation here.
- Corrections. Crossref update records, PMC notices and Europe PMC notices list no correction, erratum or retraction (
src/known.py, run 2026-10-23). - Published discussion. The 260 citing records listed in Europe PMC were screened by title. One letter in J Hepatol 2021 ("Age and comorbidity are central to the risk of death from COVID-19 in liver transplant recipients", PMC7863751) and the authors' reply (doi:10.1016/j.jhep.2021.04.005, PMC8056821) were found. An earlier check of both texts in this pipeline found that they discuss age, comorbidity and mycophenolate, not the SIR or SMR intervals.
- PubPeer. pubpeer.com returned HTTP 403 (Cloudflare) on 2026-10-23. Web searches for the DOI and title together with PubPeer, erratum or correction, run earlier in this pipeline, found no thread. A further web search could not be run today.
Sources: the PMC article (PMC7395653); the Europe PMC full-text XML and supplementary files for PMC7395653; Crossref metadata for doi:10.1016/j.jhep.2020.07.040; the Europe PMC citations list for PMID 32750442.
Suggested correction
- Recompute the four intervals from the observed counts, as the Methods describe: SIR 191.22 (95% CI about 157–230); SIR >60 206.48 (about 167–253); SMR 95.55 (about 58–148); SMR >60 100.29 (about 61–155). An alternative is to report the directly standardised ratios with intervals based on the LT stratum counts (about 142–258, 160–267, 58–158 and 61–166).
- State that LT recipients' COVID-19 mortality did not differ significantly from that of the age- and gender-matched general population. Revise the Abstract Results and Conclusions, the second Highlight, the Lay summary and the Discussion to match, and remove or clearly label as speculative the inference that chronic immunosuppression may be protective.
- Describe the standardisation actually used (direct, with LT stratum rates applied to the general-population structure) and relabel the "Observed" and "Expected" entries in Tables S1–S2 as projected and reference counts.
- Reconcile the Table S1 population total (strata sum 48,640,775 against 47,026,208 printed) and the Table S2 denominator (21,707 against 21,717 deaths).
Recomputation script (python recompute.py.txt)
# Recomputation for PMC7395653 (J Hepatol 2020, doi:10.1016/j.jhep.2020.07.040)
# Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
# Prints every recomputed number used in writeup.md and writes fig1/fig2 (.svg, .png)
# into the folder that holds this script.
#
# Inputs are copied from the paper:
# - Results text: 111 LT cases, SIR 191.22 (190.28-192.16); >60 SIR 206.48 (205.45-207.52);
# 20 deaths, SMR 95.55 (94.25-96.85); >60 SMR 100.29 (98.93-101.66).
# - Supplementary Tables S1 and S2 (mmc1.pdf): 12 age x gender strata.
# Ratios are written x100, as in the paper (191.22 means 1.9122).
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# ---------------------------------------------------------------- inputs
strata = ["0-19 M", "0-19 W", "20-39 M", "20-39 W", "40-49 M", "40-49 W",
"50-59 M", "50-59 W", "60-69 M", "60-69 W", "70+ M", "70+ W"]
# Table S1, general population (GP): population N and COVID-19 cases C
N_gp = np.array([4772317, 4481988, 5540648, 5474878, 3943490, 3869686,
3457353, 3516656, 2543236, 2738641, 3357068, 4944814])
C_gp = np.array([602, 625, 8497, 13307, 10105, 12173,
12818, 14620, 13398, 11387, 25693, 23465])
# Table S1, liver transplant (LT) population: patients n and COVID-19 cases c
n_lt = np.array([313, 285, 310, 280, 479, 333, 2373, 693, 3601, 1029, 2432, 1127])
c_lt = np.array([1, 0, 0, 1, 3, 4, 8, 2, 40, 13, 27, 12])
# Table S2: deaths among GP cases and among LT cases
D_gp = np.array([3, 5, 69, 55, 153, 84, 461, 206, 1355, 569, 10864, 7893])
D_lt = np.array([0, 0, 0, 0, 0, 0, 0, 0, 5, 3, 7, 5])
old = np.array([s.startswith(("60", "70")) for s in strata])
# Reported (point, lower, upper) as printed in Results
REP = {
"SIR overall": (191.22, 190.28, 192.16),
"SIR >60": (206.48, 205.45, 207.52),
"SMR overall": (95.55, 94.25, 96.85),
"SMR >60": (100.29, 98.93, 101.66),
}
# Abstract (rounded) versions
REP_ABS = {"SIR overall": (191.2, 190.3, 192.2), "SMR overall": (95.5, 94.2, 96.8)}
def garwood(o, a=0.05):
"""Exact Poisson CI for an observed count o."""
lo = stats.chi2.ppf(a / 2, 2 * o) / 2 if o > 0 else 0.0
hi = stats.chi2.ppf(1 - a / 2, 2 * (o + 1)) / 2
return lo, hi
def byar(o):
lo = o * (1 - 1 / (9 * o) - Z / (3 * np.sqrt(o))) ** 3
hi = (o + 1) * (1 - 1 / (9 * (o + 1)) + Z / (3 * np.sqrt(o + 1))) ** 3
return lo, hi
def fay_feuer(w, x, wmax):
"""Fay & Feuer (1997) gamma CI for a weighted sum of Poisson counts sum(w*x)."""
y, v = np.sum(w * x), np.sum(w ** 2 * x)
lo = stats.gamma.ppf(0.025, y ** 2 / v, scale=v / y)
hi = stats.gamma.ppf(0.975, (y + wmax) ** 2 / (v + wmax ** 2),
scale=(v + wmax ** 2) / (y + wmax))
return lo, hi
def log_delta(est, rel_se):
return est * np.exp(-Z * rel_se), est * np.exp(Z * rel_se)
def h(t):
print("\n" + "=" * 78 + "\n" + t + "\n" + "=" * 78)
# ---------------------------------------------------------------- 0. table checks
h("0. Totals of Supplementary Tables S1-S2")
print(f"GP cases (sum of strata) {C_gp.sum():,} (printed overall 146,690)")
print(f"GP population (sum of strata) {N_gp.sum():,} (printed overall 47,026,208)")
print(f"LT patients / LT cases {n_lt.sum():,} / {c_lt.sum()} (printed 13,255 / 111)")
print(f"GP deaths / LT deaths {D_gp.sum():,} / {D_lt.sum()} (printed 21,717 / 20)")
print(f"LT cases >60: {c_lt[old].sum()}; LT deaths >60: {D_lt[old].sum()}; "
f"GP cases >60: {C_gp[old].sum():,}; GP deaths >60: {D_gp[old].sum():,}")
print(f"Crude LT incidence 111/13,255 = {111 / 13255 * 1e5:.2f} per 10^5 (printed 837.41)")
print(f"Crude LT mortality 20/111 = {100 * 20 / 111:.2f}% (printed 18%, 18.01)")
# ---------------------------------------------------------------- 1. how the ratios were built
h("1. Reproducing the published point estimates from the strata")
proj_cases = np.sum(N_gp * c_lt / n_lt) # LT stratum rates applied to GP population
proj_cases_old = np.sum((N_gp * c_lt / n_lt)[old])
cfr_lt = np.divide(D_lt, c_lt, out=np.zeros(12), where=c_lt > 0)
proj_deaths = np.sum(C_gp * cfr_lt) # LT stratum case fatality applied to GP cases
print(f"Projected cases = sum(N_gp * c/n) = {proj_cases:,.1f} (Table S1 prints 'Observed 280,501')")
print(f" ratio to 146,690 GP cases = {100 * proj_cases / C_gp.sum():.2f} (reported 191.22)")
print(f" >60: projected {proj_cases_old:,.1f} / {C_gp[old].sum():,} = "
f"{100 * proj_cases_old / C_gp[old].sum():.2f} (reported 206.48)")
print(f"Projected deaths = sum(C_gp * d/c) = {proj_deaths:,.2f} (Table S2 prints 'Observed 20,741')")
print(f" ratio to 21,707 ('Expected' in S2) = {100 * proj_deaths / 21707:.2f} (reported 95.55)")
print(f" ratio to 21,717 (GP deaths in S2/text) = {100 * proj_deaths / 21717:.2f}")
print(f" >60: {proj_deaths:,.2f} / {D_gp[old].sum():,} = {100 * proj_deaths / D_gp[old].sum():.2f} (reported 100.29)")
print("=> the published ratios are directly standardised: LT stratum rates applied to the")
print(" GP structure. The numerator 'Observed' is a projected count, not an observed one.")
# ---------------------------------------------------------------- 2. stated method
h("2. The paper's stated method: CI from a Poisson observed count")
O = {"SIR overall": 111, "SIR >60": int(c_lt[old].sum()),
"SMR overall": 20, "SMR >60": int(D_lt[old].sum())}
stated = {}
for k, (r, lo, hi) in REP.items():
o = O[k]
E = o / (r / 100)
gl, gh = garwood(o)
bl, bh = byar(o)
stated[k] = (100 * gl / E, 100 * gh / E)
hw = (hi - lo) / 2
exact_hw = (stated[k][1] - stated[k][0]) / 2
print(f"{k}: O = {o}, E implied by reported ratio = {E:.2f}")
print(f" reported {r} ({lo}-{hi}); half-width {hw:.2f} = {100 * hw / r:.2f}% of estimate")
print(f" exact Poisson O CI {gl:.2f}-{gh:.2f} -> ratio CI {stated[k][0]:.1f}-{stated[k][1]:.1f}"
f" (Byar {100 * bl / E:.1f}-{100 * bh / E:.1f}; log-normal "
f"{r * np.exp(-Z / np.sqrt(o)):.1f}-{r * np.exp(Z / np.sqrt(o)):.1f})")
print(f" exact CI is {(stated[k][1] - stated[k][0]) / (hi - lo):.0f} times as wide as the reported CI")
for k, (r, lo, hi) in REP_ABS.items():
print(f"Abstract {k}: {r} ({lo}-{hi})")
# ---------------------------------------------------------------- 3. strata-based alternatives
h("3. Alternatives computed from the strata (both standardisation directions)")
# 3a indirect standardisation (conventional SIR/SMR)
E_ind = np.sum(n_lt * C_gp / N_gp)
E_ind_old = np.sum((n_lt * C_gp / N_gp)[old])
cfr_gp = D_gp / C_gp
Ed_ind = np.sum(c_lt * cfr_gp)
Ed_ind_old = np.sum((c_lt * cfr_gp)[old])
ind = {}
for k, o, E in [("SIR overall", 111, E_ind), ("SIR >60", O["SIR >60"], E_ind_old),
("SMR overall", 20, Ed_ind), ("SMR >60", 20, Ed_ind_old)]:
gl, gh = garwood(o)
ind[k] = (100 * o / E, 100 * gl / E, 100 * gh / E)
print(f"Indirect {k}: O = {o}, E = {E:.2f}, ratio {ind[k][0]:.1f} (exact 95% CI {ind[k][1]:.1f}-{ind[k][2]:.1f})")
# 3b direct standardisation with the variance of the LT stratum counts
dirci = {}
specs = [
("SIR overall", N_gp / n_lt / C_gp.sum(), c_lt, np.ones(12, bool)),
("SIR >60", N_gp / n_lt / C_gp[old].sum(), c_lt, old),
("SMR overall", np.where(c_lt > 0, C_gp / np.maximum(c_lt, 1), 0) / 21707, D_lt, c_lt > 0),
("SMR >60", np.where(c_lt > 0, C_gp / np.maximum(c_lt, 1), 0) / D_gp[old].sum(), D_lt, (c_lt > 0) & old),
]
for k, w, x, use in specs:
w = np.where(use, w, 0.0)
est = np.sum(w * x)
rel = np.sqrt(np.sum(w ** 2 * x)) / est
dl, dh = log_delta(100 * est, rel)
fl, fh = fay_feuer(w, x, w[use].max())
dirci[k] = (100 * est, dl, dh)
print(f"Direct {k}: ratio {100 * est:.2f}, relative SE {rel:.3f}; log-scale 95% CI {dl:.1f}-{dh:.1f};"
f" Fay-Feuer gamma 95% CI {100 * fl:.1f}-{100 * fh:.1f}")
# ---------------------------------------------------------------- 4. where the width comes from
h("4. Count implied by each reported interval width (Poisson, N = (z*ratio/half-width)^2)")
implied = {}
for k, (r, lo, hi) in REP.items():
hw = (hi - lo) / 2
implied[k] = (Z * r / hw) ** 2
print(f"{k}: implied N = {implied[k]:,.0f} (observed count {O[k]})")
zeq = hw / (r / np.sqrt(O[k])) # z that the observed-count SE would need
print(f" relative half-width for the observed count: {100 * Z / np.sqrt(O[k]):.1f}%;"
f" the reported width equals z = {zeq:.3f}, i.e. a {100 * (2 * stats.norm.cdf(zeq) - 1):.1f}% confidence level")
for lab, cnt, r in [("SIR overall, Poisson on 280,501 (S1 'Observed')", 280501, 191.22),
("SIR overall, Poisson on 146,690 (GP cases)", 146690, 191.22),
("SIR >60, Poisson on projected >60 count", proj_cases_old, 206.48),
("SMR overall, Poisson on 20,741 (S2 'Observed')", 20741, 95.55),
("SMR >60, Poisson on 20,741", 20741, 100.29)]:
se = r / np.sqrt(cnt)
print(f"{lab}: normal {r - Z * se:.2f}-{r + Z * se:.2f}")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "svg.fonttype": "none"})
def style(ax):
ax.set_facecolor(BG)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
ax.set_axisbelow(True)
# Figure 1: reported vs recomputed intervals on one log axis
groups = [("SIR, all LT patients", "SIR overall", "111 cases"),
("SIR, age >60", "SIR >60", "92 cases"),
("SMR, all LT patients", "SMR overall", "20 deaths"),
("SMR, age >60", "SMR >60", "20 deaths")]
fig, ax = plt.subplots(figsize=(7.2, 6.0), facecolor=BG)
style(ax)
y, yt, ylab = 0, [], []
for title, k, cnt in groups:
r, lo, hi = REP[k]
rows = [("Reported", r, lo, hi, ORANGE, "o", True),
("Exact Poisson, " + cnt, r, *stated[k], BLUE, "o", True),
("Strata, direct std.", *dirci[k], BLUE, "s", False)]
ax.text(30.5, y - 0.62, title, fontsize=11, fontweight="bold", color=INK, va="center")
for lab, p, a, b, col, mk, filled in rows:
ax.plot([a, b], [y, y], color=col, lw=2, solid_capstyle="butt")
ax.plot([a, a], [y - 0.12, y + 0.12], color=col, lw=2)
ax.plot([b, b], [y - 0.12, y + 0.12], color=col, lw=2)
ax.plot(p, y, marker=mk, ms=8, color=col, mfc=col if filled else BG, mew=2, zorder=3)
fmt = "{:.2f}" if lab == "Reported" else "{:.1f}"
ax.text(b * 1.04, y, f"{fmt.format(p)} ({fmt.format(a)}–{fmt.format(b)})",
va="center", fontsize=10, color=INK2)
yt.append(y)
ylab.append(lab)
y += 0.55
y += 0.75
ax.axvline(100, color=INK2, lw=0.9, zorder=1)
ax.text(100, -1.05, "ratio = 100\n(no difference)", ha="center", va="bottom", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(30, 720)
ax.set_xticks([40, 60, 100, 150, 200, 300])
ax.set_xticklabels(["40", "60", "100", "150", "200", "300"])
ax.minorticks_off()
ax.set_yticks(yt)
ax.set_yticklabels(ylab, fontsize=10)
for t, lab in zip(ax.get_yticklabels(), ylab):
t.set_color(INK)
ax.set_ylim(y - 0.5, -1.25)
ax.set_xlabel("Standardised ratio ×100 with 95% CI (log scale)")
handles = [plt.Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Reported in the paper"),
plt.Line2D([], [], color=BLUE, marker="o", ms=8, lw=2,
label="Recomputed: stated method, exact Poisson on the observed count"),
plt.Line2D([], [], color=BLUE, marker="s", ms=8, mfc=BG, mew=2, lw=2,
label="Recomputed: Tables S1–S2 strata, direct standardisation variance")]
ax.legend(handles=handles, loc="upper left", bbox_to_anchor=(-0.42, -0.11), frameon=False,
fontsize=10, ncol=1)
fig.tight_layout()
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 2: relative half-width vs the Poisson count it corresponds to
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
style(ax)
ax.grid(axis="y", which="major", color=GRID, lw=0.8)
xs = np.logspace(1, 6, 200)
ax.plot(xs, 100 * Z / np.sqrt(xs), color=INK2, lw=1.2, zorder=1)
ax.text(1500, 100 * Z / np.sqrt(1500) * 1.5, "Poisson: half-width = 1.96/√N", color=INK2, fontsize=10, ha="left")
pts_rep = [("SIR overall", "SIR, all"), ("SIR >60", "SIR, >60"), ("SMR overall", "SMR, all and >60")]
for k, lab in pts_rep:
r, lo, hi = REP[k]
ax.plot(implied[k], 100 * (hi - lo) / 2 / r, "o", ms=9, color=ORANGE, zorder=3)
ax.plot(implied["SMR >60"], 100 * (REP["SMR >60"][2] - REP["SMR >60"][1]) / 2 / REP["SMR >60"][0],
"o", ms=9, color=ORANGE, zorder=3)
ax.annotate(f"Reported SIR intervals: ≈ {implied['SIR >60']:,.0f}–{implied['SIR overall']:,.0f} events\n"
f"(projected >60 count {proj_cases_old:,.0f};\n Table S1 'Observed' 280,501)",
(implied["SIR overall"], 100 * 0.94 / 191.22), xytext=(250, 0.115), fontsize=10, color=INK,
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.annotate(f"Reported SMR intervals: ≈ {implied['SMR >60']:,.0f}–{implied['SMR overall']:,.0f} events\n"
f"(Table S2 'Observed' 20,741)",
(implied["SMR overall"], 100 * 1.30 / 95.55), xytext=(250, 0.55), fontsize=10, color=INK,
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
for o in (111, 92, 20):
ax.plot(o, 100 * Z / np.sqrt(o), "o", ms=9, color=BLUE, zorder=3)
ax.text(150, 100 * Z / np.sqrt(111) * 1.05, "111 cases (all) and 92 (>60)", color=INK, fontsize=10)
ax.text(27, 100 * Z / np.sqrt(20) * 1.05, "20 deaths", color=INK, fontsize=10)
ax.set_xscale("log")
ax.set_yscale("log")
ax.set_xlim(10, 1e6)
ax.set_ylim(0.1, 100)
ax.set_yticks([0.1, 0.2, 0.5, 1, 2, 5, 10, 20, 50])
ax.set_yticklabels(["0.1%", "0.2%", "0.5%", "1%", "2%", "5%", "10%", "20%", "50%"])
ax.set_xticks([10, 100, 1e3, 1e4, 1e5, 1e6])
ax.set_xticklabels(["10", "100", "1,000", "10,000", "100,000", "1,000,000"])
ax.minorticks_off()
ax.set_xlabel("Poisson count N that the interval corresponds to (log scale)")
ax.set_ylabel("95% CI half-width, % of estimate")
handles = [plt.Line2D([], [], color=ORANGE, marker="o", ms=9, lw=0,
label="Reported intervals, at the count their width implies"),
plt.Line2D([], [], color=BLUE, marker="o", ms=9, lw=0,
label="Recomputed: width for the observed counts")]
ax.legend(handles=handles, loc="upper right", frameon=False, fontsize=10)
fig.tight_layout()
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
print("\nfigures written to", HERE)
Reported risk ratios are ratios of mean 1/2 codes, understating relative risk reductions for perpetration 9- to 28-fold
JAMA Network Open (2018) · doi:10.1001/jamanetworkopen.2018.1213 · PMC6324287
Summary
This secondary analysis of a cluster randomized trial reports that a school-based HIV/STD risk-reduction intervention reduced self-reported perpetration of forced sex. The reported "risk ratios" for perpetration are 0.978, 0.964, 0.959, 0.967 and 0.964 at 3–54 months, and the paper sums up the effect as "about a 3% reduction in risk". The paper's own counts give risk ratios of 0.39 to 0.67, which are relative reductions of 33% to 61%. Every reported value instead matches (1 + p₁)/(1 + p₀), the ratio of group means of an outcome coded 1 (no) and 2 (yes), which is the coding the Methods describe. After approximate adjustment for the arms' different sex mix, all 10 cumulative-incidence ratios in Table 3 are reproduced to three decimals. The direction of the effect and the significance tests are not affected. What is wrong is the published effect size, its confidence intervals, the sex-comparison ratios in Table 4 and the "3%" statement, and these are the numbers a reader, meta-analyst or guideline would take from the paper.
What the paper reports
Methods, "Outcomes":
Binary variables were used to assess history of ever perpetrating and experiencing forced sexual intercourse, with responses coded 1 for respondents not reporting ever experiencing the event and 2 for those reporting ever experiencing the event.
Methods, "Statistical Analysis" (the superscript is a reference number):
The efficacy of the HIV/STD intervention compared with the control intervention on the forced sex outcomes was tested using generalized estimating equation Poisson regression models, adjusting for gender and students clustered within schools.
Risk ratios42 and corresponding 95% confidence intervals are reported.
Abstract, "Results" (repeated word for word in the Results text):
Fewer intervention than control participants reported forced sex perpetration postintervention compared with the control group at 3 months (9 of 561 [2%] vs 20 of 491 [4%]; risk ratio [RR], 0.978; 95% CI, 0.959-0.997), 6 months (17 of 561 [3%] vs 35 of 491 [7%]; RR, 0.964; 95% CI, 0.941-0.988), 12 months (21 of 561 [4%] vs 42 of 491 [9%]; RR, 0.959; 95% CI, 0.934-0.985), 42 months (41 of 561 [7%] vs 56 of 491 [11%]; RR, 0.967; 95% CI, 0.937-0.998), and 54 months (52 of 561 [9%] vs 68 of 491 [14%]; RR, 0.964; 95% CI, 0.932-0.997).
Table 3 (columns: perpetrating RR (95% CI) | P | experiencing RR (95% CI) | P), first and last rows:
3 | 0.978 (0.959-0.997) | .02 | 0.983 (0.963-1.004) | .11
3-54 | 0.990 (0.982-0.999) | .02 | 0.992 (0.982-1.001) | .09
Table 4 (first five columns: follow-up | boys | girls | "Boys vs Girls, RR (95% CI)" | P), perpetration at 12 months:
12 | 58/495 (12) | 5/557 (1) | 0.904 (0.881-0.928) | <.001
Results text:
Table 4 shows reports of perpetrating and experiencing forced sex were significantly higher among boys compared with girls at each of the postintervention assessments
Among boys, the effect on perpetration of forced sex was significant at 3-month follow-up (RR, 0.952; 95% CI, 0.917-0.988), 6-month follow-up (RR, 0.934; 95% CI, 0.892-0.979), and 12-month follow-up (RR, 0.924; 95% CI, 0.879-0.972)
Discussion, "Limitations":
The magnitude of the intervention effects in this trial was modest, about a 3% reduction in risk.
What the paper's own numbers show
Method. With a log-link Poisson model, the exponentiated arm coefficient estimates the ratio of the outcome's means in the two arms. For a 0/1 outcome that ratio is the risk ratio p₁/p₀. For the 1/2 coding the Methods describe it is (1 + p₁)/(1 + p₀), which stays close to 1 whatever the risks are. For each reported ratio I computed, from the counts printed next to it (Table 2 cumulative and time-specific incidence; Table 4 counts by sex): the risk ratio with an unadjusted Wald CI; the ratio of 1/2-code means with an unadjusted CI; and, as alternative readings, the inverted "no event" ratio (1 − p₀)/(1 − p₁), 1 + risk difference, and the odds ratio. The paper's model is adjusted for sex, and the arms differ in sex mix (Table 1: 306 of 561 vs 251 of 491 girls), while boys report forced sex several times as often as girls (Table 4). I therefore also applied an approximate sex standardization: each arm's expected value under no intervention effect, using the pooled sex-specific risks from Table 4, divides out the confounding. All CIs computed here ignore clustering. They show the scale of each quantity and do not replace the trial's own clustered analysis. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Perpetration, 3 mo (9/561 vs 20/491) | Abstract; Results; Table 3 | RR 0.978 (0.959–0.997) | risk ratio 0.394 (0.18–0.86); ratio of 1/2-code means 0.976 (0.957–0.996); sex-standardized 0.978 |
| Perpetration, 6 mo (17/561 vs 35/491) | Abstract; Results; Table 3 | RR 0.964 (0.941–0.988) | risk ratio 0.425 (0.24–0.75); code-mean ratio 0.962 (0.938–0.986); sex-standardized 0.964 |
| Perpetration, 12 mo (21/561 vs 42/491) | Abstract; Results; Table 3 | RR 0.959 (0.934–0.985) | risk ratio 0.438 (0.26–0.73); code-mean ratio 0.956 (0.930–0.982); sex-standardized 0.959 |
| Perpetration, 42 mo (41/561 vs 56/491) | Abstract; Results; Table 3 | RR 0.967 (0.937–0.998) | risk ratio 0.641 (0.44–0.94); code-mean ratio 0.963 (0.933–0.995); sex-standardized 0.967 |
| Perpetration, 54 mo (52/561 vs 68/491) | Abstract; Results; Table 3 | RR 0.964 (0.932–0.997) | risk ratio 0.669 (0.48–0.94); code-mean ratio 0.960 (0.927–0.994); sex-standardized 0.964 |
| Experience, 3/6/12/42/54 mo | Table 3 | 0.983 / 0.977 / 0.966 / 0.963 / 0.972 | risk ratios 0.544 / 0.586 / 0.561 / 0.655 / 0.771; code-mean ratios 0.981 / 0.974 / 0.962 / 0.959 / 0.967; sex-standardized 0.983 / 0.977 / 0.966 / 0.963 / 0.972 |
| Time-specific incidence pooled over visits, perpetration (52/2717 vs 68/2302) | Table 3, row 3-54 | 0.990 | risk ratio 0.648; code-mean ratio 0.990 |
| Time-specific incidence pooled over visits, experience (72/2680 vs 82/2287) | Table 3, row 3-54 | 0.992 | risk ratio 0.749; code-mean ratio 0.991 |
| Width of reported CIs | Table 3 | SE(log) 0.0099–0.0186 | matches the unadjusted SE of the code-mean ratio (ratio 0.96–0.99 at all 10 points); the unadjusted SE of the log risk ratio is 8 to 40 times larger |
| Boys vs girls, perpetration 3 mo (26/495 vs 3/557) | Table 4 | 0.956 | boys/girls risk ratio 9.75; girls/boys code-mean ratio 0.955 |
| Boys vs girls, perpetration 54 mo (95/495 vs 25/557) | Table 4 | 0.878 | boys/girls risk ratio 4.28; girls/boys code-mean ratio 0.877 |
| Boys vs girls, experience 12 mo (72/494 vs 5/556) | Table 4 | 0.882 | boys/girls risk ratio 16.21; girls/boys code-mean ratio 0.881 |
| The 12 Table 4 "Boys vs Girls" values that have counts beside them | Table 4 | 0.875–0.994 | girls/boys code-mean ratio within 0.0015 of each reported value; boys/girls risk ratios 2.80–18.01. The two pooled 3–54 month values in this column (0.968, 0.965) have no counts beside them and are also below 1 |
| Size of effect | Discussion, Limitations | "about a 3% reduction in risk" | mean of (1 − reported perpetration RR) = 3.4%; relative reductions from the counts 61%, 57%, 56%, 36%, 33% (perpetration) and 46%, 41%, 44%, 34%, 23% (experience) |
| Significance | Table 3 | P = .02, .004, .002, .04, .03 (perpetration) | unadjusted two-proportion tests P = 0.015, 0.002, 0.001, 0.022, 0.020; the reported P values are somewhat larger, as clustering would make them |
Across the 24 reported ratios with counts beside them (Table 3 cumulative and pooled rows, Table 4 sex comparison), the crude ratio of 1/2-code means differs from the reported value by at most 0.0047 (RMS 0.0024). The small remaining gap in Table 3 is in the direction sex adjustment predicts: the intervention arm has more girls, so the crude comparison slightly favors it. The approximate sex standardization closes the gap at all 10 cumulative time points (largest difference 0.0004).
Figure
Figure 1. Intervention vs control on one log-scale axis, with the line at 1 marking no effect. Orange circles: the "risk ratios" and 95% CIs reported in Table 3 (and, for perpetration, in the Abstract and Results). The reported CIs are so narrow that they barely extend past the markers. Blue diamonds: the risk ratio p₁/p₀ computed from the cumulative-incidence counts printed beside them (Table 2 and Abstract), with an unadjusted 95% CI that ignores clustering, so the true intervals would be somewhat wider. The reported values sit at 0.96–0.98 at every time point. The counts give 0.39–0.67 for perpetration and 0.54–0.77 for experience. The direction agrees, but on the relative-reduction scale the size of the effect differs by a factor of 9 to 28 for perpetration (8 to 27 for experience).
Figure 2. For each of the 24 reported ratios that have counts beside them, the difference between a candidate quantity computed from those counts and the reported value. The orange line at 0 is the reported value. Filled blue circles: ratio of mean 1/2 codes, (1 + p₁)/(1 + p₀) (for Table 4, girls over boys). Crosses: the same ratio after approximate sex standardization (Table 3 cumulative rows only, because the model was adjusted for sex). Hollow triangles: inverted "no event" ratio. Hollow squares: 1 + risk difference. The risk ratio and the odds ratio are off this scale (in the Table 3 cumulative rows the risk ratio is 0.20–0.58 below the reported values and the odds ratio 0.23–0.59 below). The two alternatives agree with the reported values only when events are rare, and they drift away as prevalence rises. The ratio of 1/2-code means stays within 0.005 everywhere, and within 0.0004 once sex is accounted for.
Why it matters
- The headline effect size. The Abstract and Results report perpetration "risk ratios" of 0.96–0.98, and the Limitations section turns these into "about a 3% reduction in risk". Read as risk ratios, these values say the intervention barely changed risk. The counts printed in the same sentence (9 of 561 vs 20 of 491 at 3 months) show the risk was cut by more than half at 3–12 months and by about a third at 42–54 months. The paper therefore understates its own finding. A reader who takes the numbers at face value, such as a meta-analyst pooling RRs or a program planner weighing cost against effect, would be misled by a factor of 9 to 28 on the relative-reduction scale (relative reduction from the counts divided by 1 − reported RR, at the five perpetration time points; 8 to 27 at the five experience time points).
- Precision. The reported intervals (for example 0.959–0.997) are intervals for the code-mean ratio. On the risk-ratio scale, the unadjusted intervals from the same counts are 0.18–0.86 at 3 months and 0.48–0.94 at 54 months. A meta-analysis would weight these estimates by the reported precision and would greatly overweight them.
- Table 4 and the sex comparisons. The column is labeled "Boys vs Girls, RR", but every value is below 1 although after baseline boys report forced sex 3 to 18 times as often as girls. The values match the girls/boys code-mean ratio. The Results text's qualitative statement that boys report more is correct, but the tabulated ratios point the other way. The intervention × sex interaction ratios (1.05–1.08 in the cumulative rows, 1.02 in the pooled 3–54 month rows) and the boys-only effects quoted in the Results (0.92–0.96) come from the same models. Their magnitudes cannot be read as multiplicative effects on risk, and the boys-only effects understate the effect for the same reason. The arm-by-sex counts needed to recompute them are not published.
- What still holds. For a 1/2-coded variable, a test of equal means is a test of equal proportions, so the coding does not invalidate the hypothesis tests. The unadjusted two-proportion P values from the counts are of the same order as the reported ones and slightly smaller, as expected before allowing for clustering. The Key Points, which give raw percentages (9% vs 14% at 54 months), are correct. The qualitative conclusions (the intervention reduced reported perpetration, and boys report more forced sex) are supported by the counts. What needs correcting is every effect estimate and CI, and the statement about the size of the effect.
Explanations considered
- The values are sex- and cluster-adjusted GEE estimates, so they need not equal crude ratios. This explains the 0.002–0.005 difference between the crude code-mean ratio and the reported values, and approximate sex standardization removes it. It cannot explain the gap to the risk ratio. Even in the extreme case where all events fell within one sex, the within-sex perpetration risk ratio at 3 months would be 0.37–0.42 (and 0.63–0.72 at 54 months), far from 0.978. Clustering mainly changes the standard errors. A working correlation structure could shift a point estimate somewhat, but the sex-standardized code-mean ratio already reproduces the reported values to three decimals, so there is no remaining gap for clustering to explain.
- The ratio is of "no event" probabilities, or inverted. This does not fit. (1 − p₀)/(1 − p₁) matches when events are rare but differs by up to 0.016 in Table 3 and 0.042 in Table 4 (Figure 2). It also would not be a risk ratio for the event the paper describes.
- The values are 1 + risk difference, and "3%" is an absolute reduction. This does not fit the tables. It deviates by up to 0.010 in Table 3 and 0.030 in Table 4 (Figure 2), and the paper labels the values as risk ratios from Poisson models. The perpetration absolute differences are 2.5–4.8 percentage points, so "3%" as an absolute difference would also be inaccurate at most time points, and the sentence says "reduction in risk".
- The values are odds ratios. No. The odds ratios from the counts are 0.38–0.74.
- A transcription or typesetting slip. No. The same pattern holds for all 24 printed ratios that have counts beside them, in the Abstract, Results, Table 3 and Table 4, and the CI widths match the code-mean scale. The PMC and Europe PMC copies of the publisher's JATS carry the same numbers.
- The ratio of 1/2-code means is the target quantity and only the label "risk ratio" is loose. Even read this way, the "about a 3% reduction in risk" statement and the "Boys vs Girls, RR" column describe risk, and the ratio of 1/2-code means is not a measure of risk. The Methods state that the 1/2 coding was used and that risk ratios are reported. The numbers show that these two statements describe different quantities.
- Table 4 orientation. Apart from the coding, the Table 4 values match girls/boys, not boys/girls as labeled. The direction of comparison in that column is therefore also mislabeled.
Status
- Version of record. The jamanetwork.com page could not be read: on 2026-10-23 the doi.org redirect target and the full-text page returned a bot-verification page (HTTP 403), as they had in an earlier check. The quoted passages and the reported values in Tables 2–4 were confirmed in the PMC-archived JATS, the live PMC page and the Europe PMC full-text XML (both retrieved 2026-10-23), which are identical at the quoted places. In the Europe PMC XML each Table 3 value sits in its own cell, so the columns are not misaligned. This is not a conversion artifact.
- Supplements. The only supplement is the trial protocol (jamanetwopen-1-e181213-s001.pdf). It contains no data or analysis output and does not bear on the computation.
- Corrections and comments. As of 2026-10-23, the only correction is "Error in Byline Order" (doi:10.1001/jamanetworkopen.2018.4304, PMC6324432; dated 2018-10-05 in Europe PMC, 2018-10-12 in Crossref). It changes only the author order. Crossref lists no update relations for the article DOI. Europe PMC (PMID 30646113) lists only that erratum and the invited commentary (doi:10.1001/jamanetworkopen.2018.1222). PubPeer could not be checked directly: its site and API returned Cloudflare blocks (HTTP 403) on 2026-10-23 and in an earlier check. Reader comments on the publisher page could not be checked, because that page could not be read. A general web search for the article title with "correction", "comment" and "letter" found the article, its university press releases, repository copies and the byline-order correction, and no letter, comment or blog post on the effect estimates.
- Sources. Paper as archived in PMC (PMC6324287; Tables 1–4, Methods, Results, Discussion); the live PMC page and the Europe PMC full-text XML (2026-10-23); Crossref and Europe PMC records; a general web search (2026-10-23). The publisher's page and PubPeer were attempted but could not be read. Calculations:
recompute.py.txtin this folder.
Suggested correction
- Refit the GEE models with the outcomes coded 0/1, for example as a log-link Poisson or log-binomial model with robust standard errors, adjusted for sex and clustering as described. Replace every risk ratio and 95% CI in the Abstract, the Results text (including the boys-only estimates) and Table 3, including the pooled 3–54 month rows.
- Recompute Table 4 on the 0/1 scale, state the direction of the comparison (boys vs girls) so that it matches the values, and re-estimate the intervention × sex interaction ratios.
- Revise "The magnitude of the intervention effects in this trial was modest, about a 3% reduction in risk" to reflect the corrected estimates. The crude risk ratios from the published counts suggest relative reductions of roughly a third to more than a half for perpetration.
Recomputation script (python recompute.py.txt)
"""Recompute the intervention-effect "risk ratios" of PMC6324287
(JAMA Netw Open 2018, doi:10.1001/jamanetworkopen.2018.1213).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
All inputs are copied from the paper as archived in PMC:
Table 1: arm sizes and number of girls per arm
Table 2: cumulative and time-specific incidence counts by arm
Table 3: reported RRs (95% CI) and P values for the intervention effect
Table 4: incidence counts by sex and reported "Boys vs Girls" RRs
Methods: outcomes "coded 1 for respondents not reporting ever experiencing the
event and 2 for those reporting ever experiencing the event", analysed with
GEE Poisson regression adjusted for gender and clustering.
If a log-link Poisson model is fitted to a variable Y coded 1/2 instead of 0/1, the
exponentiated coefficient estimates E[Y | arm 1] / E[Y | arm 0] = (1 + p1) / (1 + p0),
not the risk ratio p1 / p0.
All confidence intervals computed here are unadjusted (individuals treated as
independent, no clustering); they are for comparing scales, not substitutes for the
trial's own clustered analysis.
"""
import math
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# ------------------------------------------------------------------ inputs (from the paper)
MONTHS = [3, 6, 12, 42, 54]
N_INT, N_CTL = 561, 491 # Table 1
GIRLS = {"int": 306, "ctl": 251} # Table 1, "Female, No. (%)"
BOYS = {"int": N_INT - GIRLS["int"], "ctl": N_CTL - GIRLS["ctl"]}
# Table 2 cumulative incidence (a/n1 intervention, b/n0 control) and Table 3 reported RR, CI, P
PERP = {3: (9, 561, 20, 491, 0.978, 0.959, 0.997, .02),
6: (17, 561, 35, 491, 0.964, 0.941, 0.988, .004),
12: (21, 561, 42, 491, 0.959, 0.934, 0.985, .002),
42: (41, 561, 56, 491, 0.967, 0.937, 0.998, .04),
54: (52, 561, 68, 491, 0.964, 0.932, 0.997, .03)}
EXPE = {3: (13, 559, 21, 491, 0.983, 0.963, 1.004, .11),
6: (22, 559, 33, 491, 0.977, 0.953, 1.002, .07),
12: (30, 559, 47, 491, 0.966, 0.939, 0.994, .02),
42: (50, 559, 67, 491, 0.963, 0.931, 0.996, .03),
54: (72, 559, 82, 491, 0.972, 0.937, 1.008, .12)}
# Table 2 time-specific incidence (intervention list, control list) and Table 3 row "3-54"
TS = {"Perpetration": ([(9, 561), (8, 552), (4, 544), (20, 540), (11, 520)],
[(20, 491), (15, 471), (7, 456), (14, 449), (12, 435)], 0.990),
"Experience": ([(13, 559), (9, 546), (8, 537), (20, 529), (22, 509)],
[(21, 491), (12, 470), (14, 458), (20, 444), (15, 424)], 0.992)}
# Table 4: boys a/n1, girls b/n0, reported "Boys vs Girls" RR
T4 = {"Perpetration": {0: (4, 499, 1, 558, 0.994), 3: (26, 495, 3, 557, 0.956),
6: (47, 495, 5, 557, 0.923), 12: (58, 495, 5, 557, 0.904),
42: (78, 495, 19, 557, 0.894), 54: (95, 495, 25, 557, 0.878)},
"Experience": {0: (5, 499, 2, 558, 0.994), 3: (32, 494, 2, 556, 0.943),
6: (51, 494, 4, 556, 0.914), 12: (72, 494, 5, 556, 0.882),
42: (91, 494, 26, 556, 0.885), 54: (113, 494, 41, 556, 0.875)}}
# ------------------------------------------------------------------ helpers
def rr(a, n1, b, n0):
return (a / n1) / (b / n0)
def rr_ci(a, n1, b, n0):
r = rr(a, n1, b, n0)
se = math.sqrt(1 / a - 1 / n1 + 1 / b - 1 / n0)
return r, r * math.exp(-Z * se), r * math.exp(Z * se), se
def code_ratio(a, n1, b, n0):
"""Ratio of mean 1/2 codes: (1 + p1) / (1 + p0)."""
return (1 + a / n1) / (1 + b / n0)
def code_ratio_ci(a, n1, b, n0):
p1, p0 = a / n1, b / n0
m1, m0 = 1 + p1, 1 + p0
se = math.sqrt(p1 * (1 - p1) / (n1 * m1 ** 2) + p0 * (1 - p0) / (n0 * m0 ** 2))
r = m1 / m0
return r, r * math.exp(-Z * se), r * math.exp(Z * se), se
def noevent_inverted(a, n1, b, n0):
"""Control 'no event' proportion / intervention 'no event' proportion."""
return ((n0 - b) / n0) / ((n1 - a) / n1)
def one_plus_rd(a, n1, b, n0):
return 1 + a / n1 - b / n0
def odds_ratio(a, n1, b, n0):
return (a * (n0 - b)) / (b * (n1 - a))
def two_prop_p(a, n1, b, n0):
p = (a + b) / (n1 + n0)
se = math.sqrt(p * (1 - p) * (1 / n1 + 1 / n0))
zz = (a / n1 - b / n0) / se
return 2 * stats.norm.sf(abs(zz))
def sex_confounding(t, outcome, kind):
"""Approximate sex standardization. Expected arm-level quantity if each arm had the
pooled sex-specific risks of Table 4 and no intervention effect; the crude ratio
divided by this factor approximates a sex-adjusted ratio. kind='code' or 'binary'."""
a, nb, b, ng, _ = T4[outcome][t]
pb, pg = a / nb, b / ng
e_int = (BOYS["int"] * pb + GIRLS["int"] * pg) / N_INT
e_ctl = (BOYS["ctl"] * pb + GIRLS["ctl"] * pg) / N_CTL
if kind == "code":
return (1 + e_int) / (1 + e_ctl)
return e_int / e_ctl
def hr(title):
print("\n" + "=" * 100 + "\n" + title + "\n" + "=" * 100)
# ------------------------------------------------------------------ 1. Table 3 cumulative incidence
hr("1. Table 3 (and Abstract/Results): reported RR vs quantities computed from Table 2 counts")
rows_fig1 = []
rows_fig2 = []
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
print(f"\n{outcome}")
print(" mo | counts | reported RR (95% CI) | ratio of 1/2-code means (unadj. CI) "
"| sex-std code ratio | risk ratio p1/p0 (unadj. CI) | sex-std RR | rel. reduction | abs. diff (pp)")
for t, (a, n1, b, n0, R, L, U, P) in d.items():
cr, cl, cu, cse = code_ratio_ci(a, n1, b, n0)
r, rl, ru, rse = rr_ci(a, n1, b, n0)
cr_std = cr / sex_confounding(t, outcome, "code")
r_std = r / sex_confounding(t, outcome, "binary")
print(f" {t:>2} | {a:>2}/{n1} v {b:>2}/{n0} | {R:.3f} ({L:.3f}-{U:.3f}) "
f"| {cr:.3f} ({cl:.3f}-{cu:.3f}) | {cr_std:.3f} "
f"| {r:.3f} ({rl:.2f}-{ru:.2f}) | {r_std:.3f} "
f"| {100 * (1 - r):.0f}% | {100 * (a / n1 - b / n0):+.1f}")
rows_fig1.append((outcome, t, R, L, U, r, rl, ru))
rows_fig2.append((f"{outcome[:4]}. {t} mo", R, cr, noevent_inverted(a, n1, b, n0),
one_plus_rd(a, n1, b, n0), "T3"))
hr("2. Precision: width of reported CI vs unadjusted CI on each scale (log scale half-width / 1.96)")
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
for t, (a, n1, b, n0, R, L, U, P) in d.items():
rep_se = math.log(U / L) / (2 * Z)
_, _, _, cse = code_ratio_ci(a, n1, b, n0)
_, _, _, rse = rr_ci(a, n1, b, n0)
print(f" {outcome[:4]} {t:>2} mo: reported SE(log) {rep_se:.4f}; unadjusted SE(log) code-mean "
f"ratio {cse:.4f} (ratio {rep_se / cse:.2f}); unadjusted SE(log RR) {rse:.3f} "
f"(ratio of RR SE to reported SE: {rse / rep_se:.0f}x)")
hr("3. Significance: reported P vs unadjusted two-proportion test from the same counts")
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
for t, (a, n1, b, n0, R, L, U, P) in d.items():
print(f" {outcome[:4]} {t:>2} mo: reported P {P}; unadjusted two-proportion z-test P "
f"{two_prop_p(a, n1, b, n0):.4f}")
# ------------------------------------------------------------------ 2. Table 3 time-specific row
hr("4. Table 3 row '3-54' (time-specific incidence pooled over the 5 visits)")
for outcome, (iv, cv, R) in TS.items():
a = sum(x for x, _ in iv); n1 = sum(y for _, y in iv)
b = sum(x for x, _ in cv); n0 = sum(y for _, y in cv)
print(f" {outcome}: {a}/{n1} vs {b}/{n0}; reported {R:.3f}; ratio of code means "
f"{code_ratio(a, n1, b, n0):.3f}; risk ratio {rr(a, n1, b, n0):.3f}")
rows_fig2.append((f"{outcome[:4]}. 3-54 pooled", R, code_ratio(a, n1, b, n0),
noevent_inverted(a, n1, b, n0), one_plus_rd(a, n1, b, n0), "T3"))
# ------------------------------------------------------------------ 3. Table 4 sex comparison
hr("5. Table 4 'Boys vs Girls' RR vs quantities computed from the Table 4 counts")
for outcome, d in T4.items():
print(f"\n{outcome}")
for t, (a, n1, b, n0, R) in d.items():
print(f" {t:>2} mo: boys {a}/{n1} ({100 * a / n1:.1f}%) girls {b}/{n0} ({100 * b / n0:.1f}%); "
f"reported {R:.3f}; girls/boys code-mean ratio {code_ratio(b, n0, a, n1):.3f}; "
f"boys/girls code-mean ratio {code_ratio(a, n1, b, n0):.3f}; "
f"boys/girls risk ratio {rr(a, n1, b, n0):.2f}")
rows_fig2.append((f"{outcome[:4]}. " + ("baseline" if t == 0 else f"{t} mo"), R, code_ratio(b, n0, a, n1),
noevent_inverted(b, n0, a, n1), one_plus_rd(b, n0, a, n1), "T4"))
# Table 4 also prints pooled 3-54 month "Boys vs Girls" values, with no counts beside them
T4_POOLED = {"Perpetration": 0.968, "Experience": 0.965}
print("\n Table 4 rows '3-54' (no counts printed, so not recomputable): "
+ ", ".join(f"{k} {v:.3f}" for k, v in T4_POOLED.items()))
allt4 = [v[4] for d in T4.values() for v in d.values()] + list(T4_POOLED.values())
print(f" Table 4 'Boys vs Girls' column: {len(allt4)} values in all, "
f"{sum(len(d) for d in T4.values())} with counts; range {min(allt4):.3f}-{max(allt4):.3f}; "
f"all below 1: {all(v < 1 for v in allt4)}")
t4dev = [code_ratio(b, n0, a, n1) - R for d in T4.values() for (a, n1, b, n0, R) in d.values()]
t4rr = [rr(a, n1, b, n0) for d in T4.values() for (a, n1, b, n0, R) in d.values()]
print(f" the 12 with counts: max |girls/boys code-mean ratio - reported| {max(abs(v) for v in t4dev):.4f}; "
f"boys/girls risk ratios {min(t4rr):.2f}-{max(t4rr):.2f}")
# ------------------------------------------------------------------ 4. candidate definitions
hr("6. Which quantity do the 24 reported values track? (candidate minus reported)")
cands = {"ratio of mean 1/2 codes": 2, "inverted 'no event' ratio": 3, "1 + risk difference": 4}
for name, k in cands.items():
dev = np.array([row[k] - row[1] for row in rows_fig2])
print(f" {name:<26}: mean dev {dev.mean():+.4f}, max |dev| {np.abs(dev).max():.4f}, "
f"RMS {math.sqrt((dev ** 2).mean()):.4f}")
dev_rr = []
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
for t, (a, n1, b, n0, *_r) in d.items():
dev_rr.append(rr(a, n1, b, n0) - _r[0])
for tab in ("T3", "T4"):
sub = [r for r in rows_fig2 if r[5] == tab]
print(f" {tab} ({len(sub)} values): max |dev| code-mean {max(abs(r[2] - r[1]) for r in sub):.4f}, "
f"no-event {max(abs(r[3] - r[1]) for r in sub):.4f}, 1+RD {max(abs(r[4] - r[1]) for r in sub):.4f}")
dev_std = []
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
for t, (a, n1, b, n0, R, *_r) in d.items():
dev_std.append(code_ratio(a, n1, b, n0) / sex_confounding(t, outcome, "code") - R)
print(f" sex-standardized ratio of 1/2-code means (Table 3 cumulative, 10 values): max |dev| "
f"{max(abs(v) for v in dev_std):.4f}; equal to reported at 3 decimals in "
f"{sum(abs(v) < 0.0005 for v in dev_std)} of 10")
print(f" risk ratio p1/p0 (Table 3 cumulative only): deviations {min(dev_rr):+.3f} to {max(dev_rr):+.3f}")
print(" odds ratios, Table 3 cumulative: " + ", ".join(
f"{outcome[:4]} {t}: {odds_ratio(*v[:4]):.3f}" for outcome, d in (("Perpetration", PERP),
("Experience", EXPE))
for t, v in d.items()))
dev_or = [odds_ratio(*v[:4]) - v[4] for d in (PERP, EXPE) for v in d.values()]
print(f" odds ratio (Table 3 cumulative only): deviations {min(dev_or):+.4f} to {max(dev_or):+.4f}")
dev_code_cum = [r[2] - r[1] for r in rows_fig2[:10]]
print(f" crude ratio of 1/2-code means (Table 3 cumulative, 10 values): deviations "
f"{min(dev_code_cum):+.4f} to {max(dev_code_cum):+.4f}")
noev_t4 = [r[3] - r[1] for r in rows_fig2 if r[5] == "T4"]
print(f" inverted 'no event' ratio, Table 4: lowest deviation {min(noev_t4):+.4f} "
f"(figure 2 y-axis lower limit -0.046)")
# ------------------------------------------------------------------ 5. magnitude statement
hr("7. Discussion: 'about a 3% reduction in risk'")
rep = np.array([v[4] for v in PERP.values()])
print(f" mean(1 - reported perpetration RR) = {100 * np.mean(1 - rep):.1f}%")
rrs = np.array([rr(*v[:4]) for v in PERP.values()])
print(" perpetration relative reductions from counts: "
+ ", ".join(f"{100 * (1 - x):.0f}%" for x in rrs))
rre = np.array([rr(*v[:4]) for v in EXPE.values()])
print(" experience relative reductions from counts: "
+ ", ".join(f"{100 * (1 - x):.0f}%" for x in rre))
print(" ratio of relative reduction from counts to (1 - reported RR), perpetration: "
+ ", ".join(f"{(1 - x) / (1 - R):.1f}" for x, R in zip(rrs, rep)))
repe = np.array([v[4] for v in EXPE.values()])
print(" ratio of relative reduction from counts to (1 - reported RR), experience: "
+ ", ".join(f"{(1 - x) / (1 - R):.1f}" for x, R in zip(rre, repe)))
fold = [(1 - x) / (1 - R) for x, R in zip(np.r_[rrs, rre], np.r_[rep, repe])]
print(f" range of that ratio over all 10 points in Figure 1: {min(fold):.1f} to {max(fold):.1f}")
print(" perpetration absolute differences (pp): "
+ ", ".join(f"{100 * (v[0] / v[1] - v[2] / v[3]):.1f}" for v in PERP.values()))
hr("8. Can adjustment for sex move the risk ratio to ~0.97? Worst case: all events in one sex")
for t, (a, n1, b, n0, R, *_r) in PERP.items():
lo = (a / BOYS["int"]) / (b / BOYS["ctl"])
hi = (a / GIRLS["int"]) / (b / GIRLS["ctl"])
print(f" perpetration {t:>2} mo: within-boys RR if all events were boys {lo:.3f}; "
f"within-girls RR if all events were girls {hi:.3f}; reported {R:.3f}")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), bbox_inches="tight")
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, bbox_inches="tight")
plt.close(fig)
# Figure 1: reported RR vs risk ratio from the counts, log scale, one axis
fig, ax = plt.subplots(figsize=(7.2, 5.4))
labels, ys = [], []
y = 0
for i, (outcome, t, R, L, U, r, rl, ru) in enumerate(rows_fig1):
if i == 5:
y += 0.8
yy = -y
ax.plot([L, U], [yy + 0.17] * 2, color=ORANGE, lw=2, solid_capstyle="round")
ax.plot(R, yy + 0.17, "o", color=ORANGE, ms=8, mec=BG, mew=1.2, zorder=3)
ax.plot([rl, ru], [yy - 0.17] * 2, color=BLUE, lw=2, solid_capstyle="round")
ax.plot(r, yy - 0.17, "D", color=BLUE, ms=7.5, mec=BG, mew=1.2, zorder=3)
ax.text(U * 1.025, yy + 0.17, f"{R:.3f}", va="center", ha="left", fontsize=10, color=INK2)
ax.text(rl / 1.04, yy - 0.17, f"{r:.2f}", va="center", ha="right", fontsize=10, color=INK2)
labels.append(f"{t} mo")
ys.append(yy)
y += 1
ax.axvline(1.0, color=INK2, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.1, 1.6)
ticks = [0.1, 0.2, 0.3, 0.5, 0.7, 1.0, 1.5]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{x:g}" for x in ticks])
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_yticks(ys)
ax.set_yticklabels(labels)
ax.set_ylim(ys[-1] - 0.7, 0.9)
ax.grid(axis="x", color=GRID, lw=0.8, which="major")
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.text(0.103, ys[0] + 0.55, "Perpetrating forced sex", fontsize=10.5, fontweight="bold", va="bottom")
ax.text(0.103, ys[5] + 0.55, "Experiencing forced sex", fontsize=10.5, fontweight="bold", va="bottom")
ax.text(1.02, 0.75, "no effect", fontsize=10, color=INK2, va="center")
ax.set_xlabel("Intervention vs control, ratio (log scale)")
h1 = ax.plot([], [], "o-", color=ORANGE, ms=8, lw=2,
label="Reported \"risk ratio\" and 95% CI (Table 3)")[0]
h2 = ax.plot([], [], "D-", color=BLUE, ms=7.5, lw=2,
label="Risk ratio from the paper's counts (Table 2), unadjusted 95% CI")[0]
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.11), ncol=1,
frameon=False, fontsize=10)
save(fig, "fig1")
# Figure 2: which quantity the reported values track
fig, ax = plt.subplots(figsize=(7.2, 4.6))
x = np.arange(len(rows_fig2))
dev_code = np.array([r[2] - r[1] for r in rows_fig2])
dev_noev = np.array([r[3] - r[1] for r in rows_fig2])
dev_rd = np.array([r[4] - r[1] for r in rows_fig2])
ax.axhline(0, color=ORANGE, lw=2, zorder=1)
ax.plot(x, dev_rd, "s", ms=8, mfc=BG, mec=BLUE, mew=1.6, label="1 + risk difference")
ax.plot(x, dev_noev, "^", ms=8.5, mfc=BG, mec=BLUE, mew=1.6,
label="Inverted \"no event\" ratio, (1 − p₀)/(1 − p₁)")
ax.plot(x, dev_code, "o", ms=8, color=BLUE, mec=BG, mew=1.0,
label="Ratio of mean 1/2 codes, (1 + p₁)/(1 + p₀)")
std_x, std_dev = [], []
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
for t, (a, n1, b, n0, R, *_r) in d.items():
std_x.append(len(std_x)) # same order as the first 10 rows of rows_fig2
std_dev.append(code_ratio(a, n1, b, n0) / sex_confounding(t, outcome, "code") - R)
ax.plot(std_x, std_dev, "X", ms=9, color=BLUE, mec=BG, mew=0.8, zorder=4,
label="Ratio of mean 1/2 codes, approx. sex-standardized (Table 3 cumulative rows)")
ax.text(len(x) - 0.5, 0.0015, "reported value (difference = 0)", color=INK, fontsize=10,
ha="right", va="bottom")
n3 = sum(1 for r in rows_fig2 if r[5] == "T3")
ax.axvline(n3 - 0.5, color=INK2, lw=0.8, ls=(0, (2, 2)))
ax.text((n3 - 1) / 2, 0.012, "Table 3: intervention vs control", ha="center", fontsize=10, color=INK2)
ax.text(n3 + (len(x) - n3 - 1) / 2, 0.012, "Table 4: sex comparison", ha="center",
fontsize=10, color=INK2)
ax.set_xticks(x)
ax.set_xticklabels([r[0] for r in rows_fig2], rotation=60, ha="right", fontsize=10)
ax.set_ylabel("Candidate minus reported value")
ax.set_ylim(-0.046, 0.016)
ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.legend(loc="upper left", bbox_to_anchor=(-0.02, -0.33), frameon=False, fontsize=10)
save(fig, "fig2")
print("\nWrote fig1.svg/png and fig2.svg/png to", HERE)
Reported nationwide energy, cost and CO2 savings are 1,000 times what the paper's inputs give
Nat Commun (2025) · doi:10.1038/s41467-025-64962-2 · PMC12575868
Summary
The paper simulates an electrochromic smart window in office buildings in all 50 U.S. states and reports an average HVAC saving of 26.72 MJ/m² against low-e glass. It then applies that saving to the 97 billion ft² of U.S. commercial floor space and reports, in the Abstract, Results and Discussion, a nationwide saving of 66.87 billion MWh, worth $2.67–7.35 trillion and avoiding 0.87–66.94 billion t of CO2. The paper's own two inputs give 26.72 MJ/m² × 9.01 × 10⁹ m² = 2.41 × 10¹¹ MJ, which is 66.9 million MWh (66.9 TWh). The reported figure is 999.8 times larger, and the cost and CO2 totals, which Supplementary Eqs. S10–S11 compute from it, are 1,000 times too large as well: they should be about $2.7–8.4 billion and 0.87–67 million t. These totals are the abstract's headline impact claim. The reported energy figure is about 16 times all U.S. utility-scale electricity generation, so no reading of the inputs makes it possible.
What the paper reports
Abstract:
Simulations indicate that a large-scale deployment across the U.S. can save 66.87 billion MWh, amounting to $7.35 trillion, and reduce CO2 emissions by 66.94 billion tons.
With its rapid switching, long-term durability, and scalability, this device presents significant economic and environmental advantages for real-world applications.
Results, section "HVAC energy-saving capabilities as building smart windows", paragraph 4 (the "54" after "2018" is the citation to reference 54, the EIA Commercial Buildings Energy Consumption Survey):
As shown in Table S7 (Supplementary Information), the average energy savings across U.S. states is calculated to be 26.72 MJ m-2. The U.S. Energy Information Administration (EIA) reported that the total commercial building area in the U.S. was about 97 billion square feet in 201854, allowing for a calculation of total energy savings across the country to be as high as 66.87 billion MWh.
The economic costs of producing this large amount of electricity using nuclear, coal, gas, photovoltaic, onshore, and offshore wind power generation systems are estimated to range from $2.67 trillion to $7.35 trillion (Fig. 7d and Table S8, Eq. S10, Supplementary Information).
Even clean energy sources, like nuclear, photovoltaic, and wind, produce 0.87 billion tons of CO2, while coal, a highly polluting energy source, has an astounding 66.94 billion tons of CO2 emissions.
It holds significant economic and environmental value, aiding in energy conservation, emission reduction, and achieving net-zero energy buildings globally.
Discussion, paragraph 1:
The smart window consistently displayed higher energy efficiency across all states in the U.S., with an average value of 26.72 MJ m⁻². If implemented nationwide in office buildings, it is estimated to save 66.87 billion MWh of electricity. Depending on the power generation methods, the savings would translate into electricity costs ranging from $2.67 trillion to $7.35 trillion and a reduction in carbon emissions of between 0.87 billion and 66.94 billion tons.
Methods, building energy simulation:
A year-long simulation of monthly building energy consumption was performed for Phoenix, Arizona, along with an annual analysis of building energy consumption across various climate zones in all 50 U.S. states to investigate regional impacts.
Supplementary Information (MOESM1), Table S8 and Eqs. S10–S11. The equations are C = LCOE × E / 10¹² (trillion USD) and T = f × E / 10¹² (billion t CO2), "where … LCOE = levelized cost of electricity (USD/MWh), f = CO2 emission factor (kg CO2/MWh), and E = electricity consumption (MWh)". Table S8 lists the following LCOE (USD/MWh), cost (trillion USD), emission factor (kg/MWh) and CO2 (billion t): Nuclear 110 / 7.35 / 13 / 0.87; Coal 105 / 7.02 / 1001 / 66.94; Gas CCGT 60 / 4.01 / 486 / 32.50; Solar PV 55 / 3.68 / 43 / 2.88; Wind onshore 40 / 2.67 / 13 / 0.87; Wind offshore 125 / 8.36 / 13 / 0.87. The source data file (MOESM3, sheet "Fig. 7d") gives the same costs and CO2 totals.
All quotes from the article were checked verbatim against the PMC-archived text. The supplementary passages were read in the publisher's MOESM1 file.
What the paper's own numbers show
Method. I took the 50 per-state savings from Supplementary Table S7 and checked their mean and that each equals the low-e value minus the device value. I converted the stated floor area with the exact factor 1 ft² = 0.09290304 m², multiplied, and converted with 1 MWh = 3,600 MJ. I then applied the paper's own Eqs. S10–S11 and the Table S8 LCOE and emission factors to the recomputed energy. For scale, the recomputed and reported totals are compared with two EIA figures: U.S. utility-scale generation in 2023 and total energy use in U.S. commercial buildings in 2018. All calculations are in recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Mean annual HVAC saving, 50 states | Results; Discussion; Table S7 | 26.72 MJ/m² | 26.716 MJ/m² (mean of the 50 rows; each row = low-e − device) |
| U.S. commercial floor area | Results (ref. 54) | 97 billion ft² | 97 × 10⁹ × 0.09290304 = 9.0116 × 10⁹ m² |
| Nationwide annual saving, energy | — | — | 26.72 × 9.0116 × 10⁹ = 2.408 × 10¹¹ MJ |
| Nationwide annual saving, electricity | Abstract; Results; Discussion | 66.87 billion MWh (6.687 × 10¹⁰ MWh) | 6.689 × 10⁷ MWh = 66.89 million MWh (66.9 TWh) |
| Ratio reported / recomputed | — | — | 999.8 |
| Same total in kWh | — | — | 66.89 billion kWh |
| Per-area saving implied by 66.87 billion MWh | derived | — | 26,714 MJ/m², which is 489 times the largest single-state saving (54.60 MJ/m², Hawaii) and 12.8 times the largest total annual HVAC use in Table S7 (2,080.05 MJ/m², Alaska, low-e glass) |
| Cost of the saved electricity (Eq. S10) | Abstract; Results; Discussion; Table S8 | $2.67–7.35 trillion | $2.68–7.36 billion for the technologies in the stated range; $8.36 billion for offshore wind |
| CO2 associated with the saved electricity (Eq. S11) | Abstract; Results; Discussion; Table S8 | 0.87–66.94 billion t | 0.87–66.95 million t |
| Unit values implied by the text ranges | derived | $2.67–7.35 trillion; 0.87–66.94 billion t | $39.9–109.9 per MWh and 13.0–1,001 kg CO2/MWh, which match the Table S8 LCOE and emission factors. The cost and CO2 totals are therefore Eq. S10–S11 applied to 66.87 billion MWh, and carry the same factor of 1,000 |
| Offshore wind cost vs stated range (secondary) | Table S8 vs Results text | Text: "$2.67 trillion to $7.35 trillion", including offshore wind | Table S8 lists offshore wind at 8.36 (125 USD/MWh), above the stated upper end |
| Solar PV CO2 vs text (secondary) | Table S8 vs Results text | Text: "nuclear, photovoltaic, and wind, produce 0.87 billion tons" | Table S8 gives 2.88 for solar PV (43 kg/MWh) |
| Pre-revision values (peer review file, MOESM2, Table R1) | — | 24.439 MJ/m² → "61.1"; 26.72 MJ/m² → "66.87", in a row labelled "(MWh)" | 61.18 and 66.89 million MWh; the same ratio of about 1,000 (998.8 and 999.8) if the listed values are read as billion MWh, as in the paper |
| Reported total vs U.S. utility-scale generation, 2023 (EIA: 4,178 TWh) | external | 66,870 TWh | Reported = 16.0 times all U.S. generation; recomputed = 1.60% of it |
| Reported total vs energy used in U.S. commercial buildings, 2018 (EIA CBECS: 6.8 quadrillion Btu = 1,993 TWh) | external | 66,870 TWh | Reported = 33.6 times all commercial-building energy; recomputed = 3.36% of it |
The recomputed electricity figure (66.89 million MWh) and the reported one (66.87 billion MWh) agree in their leading digits and differ only in the multiplier. The small difference in the last digit is consistent with rounding of the mean saving or of the conversion factor (with the unrounded Table S7 mean, the recomputed value is 66.88 million MWh). Applying Eqs. S10–S11 to the reported E reproduces every Table S8 entry to within 0.01, so the supplement is internally consistent with the inflated E. The discrepancy enters at the conversion of the energy product to MWh.
Figure
Figure 1. The arithmetic chain behind the nationwide totals, using only numbers stated in the paper and its supplement. The mean per-area saving (Table S7) and the commercial floor area (ref. 54, converted to m²) give 2.408 × 10¹¹ MJ per year. The highlighted, dashed step is the conversion to MWh. At 3,600 MJ per MWh the total is 66.9 million MWh (blue, left). The paper reports 66.87 billion MWh (orange, right), which equals the same total expressed in kWh. The bottom row applies the paper's Eqs. S10–S11 and Table S8 factors. With the recomputed energy, the costs are $2.68–7.36 billion ($8.36 billion for offshore wind) and the CO2 is 0.87–66.95 million t. The paper reports the same digits in trillions of dollars and billions of tonnes.
Figure 2. Annual energy on a single logarithmic axis, in TWh (1 TWh = 1 million MWh). The orange dot is the paper's reported nationwide saving (66,870 TWh). The blue dot is the value recomputed from the paper's own inputs (66.9 TWh). The ratio between them is 999.8. The gray squares are external reference points from the U.S. EIA: all U.S. utility-scale electricity generation in 2023 (4,178 TWh) and all energy used in U.S. commercial buildings in 2018 (6.8 quadrillion Btu, 1,993 TWh). The recomputed saving is 3.4% of commercial-building energy use. The reported saving is 33.6 times that use and 16 times the country's entire electricity output.
Why it matters
The nationwide totals are the only quantitative statement of the device's real-world impact in the Abstract. The abstract's closing claim of "significant economic and environmental advantages" rests largely on them, and the Results ("substantial economic and environmental benefits") and Discussion repeat them. As printed, they overstate the energy, money and CO2 saved by three orders of magnitude. A reader of the abstract would take away savings of trillions of dollars and tens of billions of tonnes of CO2 per year. These figures are larger than the entire U.S. electricity system, and the CO2 figure is larger than recent estimates of total global annual emissions. The values the paper's inputs actually support are billions of dollars and tens of millions of tonnes. That is still a meaningful result, but it is a different claim. The correct values appear nowhere in the paper or its supplement, so a reader can recover them only by redoing the calculation. Table S8, Fig. 7d and the Fig. 7d source data carry the inflated values, so anyone reusing the supplement would inherit the error.
The following are not affected: the device chemistry and electrochromic measurements (switching times of 1.6 s and 0.8 s, 64.23% optical modulation, capacity, energy recovery of 44.4% and 31.0%, cycling stability), the outdoor temperature test, the SHGC values, and the per-area EnergyPlus results (Fig. 7a–c, Tables S6–S7, including the 26.72 MJ/m² average). The direction of the comparison with low-e glass is unchanged.
Explanations considered
- Multi-year or lifetime horizon. A factor of 1,000 would need 1,000 years of annual savings. The Methods describe annual simulations, and the paper states no lifetime, discounting or multi-year multiplier anywhere. This does not explain the numbers.
- Floor area read in m² instead of ft². 26.72 MJ/m² × 97 × 10⁹ m² = 720 million MWh, a factor of 10.8 rather than 1,000. This does not explain the numbers.
- Per window area rather than per floor area. In the simulated building (20 m × 10 m floor, twelve 3 m × 2 m windows), floor area is 200 m² and window area is 72 m². Whichever area the 26.72 MJ/m² refers to, re-basing changes the total by a factor of about 3 at most, and the national window area is smaller than the national floor area. This does not explain the numbers.
- Primary vs site energy, or thermal vs electrical energy. Source-to-site multipliers for electricity are about 3, not 1,000, and the paper treats the saving directly as electricity ("save 66.87 billion MWh of electricity"). This does not explain the numbers.
- A larger building stock. The paper uses commercial floor area (and the Discussion says "office buildings", a subset). Even adding all U.S. residential floor space would multiply the area by a small single-digit factor, not by 1,000. This does not explain the numbers.
- Per-area saving larger than stated. For 66.87 billion MWh the average saving would have to be 26,714 MJ/m². That is 12.8 times the largest total annual HVAC consumption of any state in Table S7, which would mean saving more energy than the buildings use. This does not explain the numbers.
- Text-conversion artifact in PMC. The figures are written out in words ("billion MWh", "trillion", "billion tons"). They are identical in the PMC text, the publisher's HTML and the publisher's PDF, and they are carried consistently into Table S8, Fig. 7d and the source data. This does not explain the numbers.
- A kWh value labelled as MWh. The reported 66.87 is numerically the recomputed total expressed in billions of kWh (66.89 billion kWh). The pre-revision values in the peer review file show nearly the same relationship (24.439 MJ/m² gives 61.18 billion kWh; the file lists "61.1", a ratio of 998.8). This matches the numbers exactly, but it is a description of where the factor of 1,000 enters, not a justification: the paper's Eqs. S10–S11 define E in MWh and use it as MWh.
Status
As of 2026-10-23:
- Version of record. I fetched the publisher's article page (https://www.nature.com/articles/s41467-025-64962-2, published 2025-10-30) and its 12-page PDF. Both carry "66.87 billion MWh", "$7.35 trillion", "66.94 billion tons", "26.72 MJ m−2", "97 billion square feet" and "$2.67 trillion to $7.35 trillion", identical to the PMC-archived text. No correction or amendment notice appears on the page.
- Supplements. All three electronic supplementary files from the publisher were read. MOESM1 (Supplementary Information): Table S7 (50 states; mean saving 26.716 MJ/m²), Table S8, and Eqs. S10–S11 with E defined in MWh. MOESM3 (source data): sheets "Fig. 7c" (the same 50 savings) and "Fig. 7d" (the same costs in trillions and CO2 in billions of tonnes as Table S8). MOESM2 (peer review file): the added formulas (R10–R11) and a table of pre- and post-revision values ("Total energy savings across U.S. states (MWh)": 61.1 and 66.87). None of these contains a multiplier or definition that reconciles the inputs with the reported total, and none contains the correctly scaled value.
- Corrections. No correction, erratum or retraction was found. Sources: Crossref metadata for the DOI (no update-to, updated-by or relation entries), the Crossref
filter=updates:query (0 results), PMC and Europe PMC notices (none;src/known.py), and the publisher's article page (none). - PubPeer. The PubPeer search API returned a human-verification challenge and could not be queried directly. A web search for the title together with "pubpeer", "correction" or "erratum" found no thread or notice about this article.
- External reference values (used only for scale): EIA, "What is U.S. electricity generation by energy source?" (https://www.eia.gov/tools/faqs/faq.php?id=427), "about 4,178 billion kilowatthours" of utility-scale generation in 2023 (preliminary); EIA CBECS (https://www.eia.gov/consumption/commercial/), U.S. commercial buildings "consumed 6.8 quadrillion British thermal units of energy" in 2018.
Suggested correction
Abstract. Replace "can save 66.87 billion MWh, amounting to $7.35 trillion, and reduce CO2 emissions by 66.94 billion tons" with:
can save 66.9 million MWh (66.9 TWh) per year, amounting to up to $7.36 billion, and reduce CO2 emissions by up to 66.95 million tons.
Results, paragraph 4. Replace "as high as 66.87 billion MWh" with "as high as 66.9 million MWh (66.9 TWh)". Replace "from $2.67 trillion to $7.35 trillion" with "from $2.68 billion to $8.36 billion". This upper end includes offshore wind, which the sentence lists; alternatively, offshore wind can be removed from the list and the range given as $2.68–7.36 billion. Replace "0.87 billion tons" and "66.94 billion tons" with "0.87 million tons" and "66.95 million tons". For solar PV, the Table S8 factor gives 2.88 million tons, so "photovoltaic" should be dropped from the 0.87 group or given its own value.
Discussion, paragraph 1. Make the same replacements: 66.9 million MWh; $2.68–8.36 billion (or $2.68–7.36 billion); 0.87–66.95 million tons.
Supplementary Table S8 and Fig. 7d (and source data sheet "Fig. 7d"). Relabel the cost column "billions, USD" and the CO2 column "million tons" (the digits are correct for E = 66.9 million MWh to within rounding). Alternatively, keep the trillion/billion headings and divide every entry by 1,000. In Eqs. S10–S11, the stated units then become C in billion USD = LCOE × E / 10⁹ and T in million t = f × E / 10⁹, with E = 6.69 × 10⁷ MWh.
Recomputation script (python recompute.py.txt)
"""Recompute the nationwide energy, cost and CO2 totals in PMC12575868
(Nat Commun 2025; doi:10.1038/s41467-025-64962-2) from the paper's own stated inputs,
and redraw fig1/fig2 in this file's folder.
Run: python recompute.py.txt (needs numpy and matplotlib)
Inputs are copied from the published article (Abstract, Results "HVAC energy-saving
capabilities as building smart windows", Discussion) and from Supplementary Information
Tables S7 and S8 and Eqs. S10-S11 (MOESM1_ESM.docx). Two external reference values, used
only for the plausibility comparison, are from the U.S. EIA and are labelled as such.
"""
import os
import numpy as np
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs from the paper
# Supplementary Table S7: annual HVAC energy, MJ per m2 (EESD, low-e glass, saving), 50 states
S7 = {
"Alabama": (274.50, 308.85, 34.35), "Alaska": (2074.00, 2080.05, 6.05),
"Arizona": (346.10, 389.50, 43.40), "Arkansas": (361.15, 395.80, 34.65),
"California": (70.80, 92.85, 22.05), "Colorado": (244.25, 267.50, 23.25),
"Connecticut": (367.25, 384.80, 17.55), "Delaware": (281.10, 303.15, 22.05),
"Florida": (448.30, 501.50, 53.20), "Georgia": (250.80, 280.20, 29.40),
"Hawaii": (398.85, 453.45, 54.60), "Idaho": (228.95, 252.00, 23.05),
"Illinois": (411.15, 434.10, 22.95), "Indiana": (373.80, 392.10, 18.30),
"Iowa": (502.75, 529.10, 26.35), "Kansas": (387.00, 420.15, 33.15),
"Kentucky": (304.25, 329.65, 25.40), "Louisiana": (370.35, 412.60, 42.25),
"Maine": (394.90, 410.05, 15.15), "Maryland": (292.85, 318.60, 25.75),
"Massachusetts": (260.90, 278.60, 17.70), "Michigan": (387.80, 409.40, 21.60),
"Minnesota": (595.05, 615.90, 20.85), "Mississippi": (330.80, 368.10, 37.30),
"Missouri": (413.10, 444.00, 30.90), "Montana": (293.85, 308.80, 14.95),
"Nebraska": (304.70, 328.10, 23.40), "Nevada": (304.20, 351.50, 47.30),
"New Hampshire": (413.55, 431.10, 17.55), "New Jersey": (244.40, 268.90, 24.50),
"New Mexico": (183.40, 212.00, 28.60), "New York": (258.55, 284.50, 25.95),
"North Carolina": (252.50, 283.75, 31.25), "North Dakota": (812.25, 828.90, 16.65),
"Ohio": (335.35, 359.35, 24.00), "Oklahoma": (360.75, 398.90, 38.15),
"Oregon": (150.15, 167.40, 17.25), "Pennsylvania": (289.55, 309.30, 19.75),
"Rhode Island": (185.30, 203.15, 17.85), "South Carolina": (323.55, 363.95, 40.40),
"South Dakota": (623.05, 645.75, 22.70), "Tennessee": (308.10, 345.40, 37.30),
"Texas": (358.35, 400.40, 42.05), "Utah": (215.55, 240.00, 24.45),
"Vermont": (469.00, 486.40, 17.40), "Virginia": (283.25, 315.90, 32.65),
"Washington": (124.90, 140.75, 15.85), "West Virginia": (307.15, 328.20, 21.05),
"Wisconsin": (467.85, 487.15, 19.30), "Wyoming": (381.80, 396.05, 14.25),
}
SAVING_REPORTED = 26.72 # MJ/m2, Results and Discussion
AREA_FT2 = 97e9 # "about 97 billion square feet in 2018" (EIA CBECS, ref 54)
E_REPORTED_MWH = 66.87e9 # "66.87 billion MWh" (Abstract, Results, Discussion)
# Supplementary Table S8: LCOE (USD/MWh), cost (trillion USD), f (kg CO2/MWh), CO2 (billion t)
S8 = {
"Nuclear": (110, 7.35, 13, 0.87),
"Coal": (105, 7.02, 1001, 66.94),
"Gas CCGT": (60, 4.01, 486, 32.50),
"Solar PV": (55, 3.68, 43, 2.88),
"Wind onshore": (40, 2.67, 13, 0.87),
"Wind offshore": (125, 8.36, 13, 0.87),
}
TEXT_COST_RANGE = (2.67, 7.35) # trillion USD, Abstract/Results/Discussion
TEXT_CO2_RANGE = (0.87, 66.94) # billion t, Results/Discussion
# Peer review file (MOESM2), pre-revision vs revised values (Table R1)
PRF = {"before": (24.439, 61.1), "after": (26.72, 66.87)} # (MJ/m2, "Total energy savings ... (MWh)")
# ---------------------------------------------------------------- unit constants
FT2_TO_M2 = 0.09290304 # exact
MJ_PER_MWH = 3600.0 # exact
MJ_PER_KWH = 3.6 # exact
J_PER_BTU = 1055.05585 # International Table Btu
# ---------------------------------------------------------------- external references (plausibility only)
US_GEN_2023_TWH = 4178.0 # EIA FAQ id=427: "about 4,178 billion kilowatthours" utility-scale, 2023 (preliminary)
CBECS_2018_QUADS = 6.8 # EIA CBECS 2018: commercial buildings "consumed 6.8 quadrillion British thermal units"
print("=== 1. Mean saving across 50 states (Table S7) ===")
names = list(S7)
eesd = np.array([S7[k][0] for k in names])
lowe = np.array([S7[k][1] for k in names])
sav = np.array([S7[k][2] for k in names])
print(f"n = {len(sav)}; mean saving = {sav.mean():.4f} MJ/m2 (reported {SAVING_REPORTED})")
print(f"every saving = low-e minus EESD: {np.allclose(lowe - eesd, sav, atol=0.006)}")
print(f"savings range {sav.min():.2f}-{sav.max():.2f} MJ/m2; "
f"total HVAC range: EESD {eesd.min():.2f}-{eesd.max():.2f}, low-e {lowe.min():.2f}-{lowe.max():.2f} MJ/m2")
print("\n=== 2. Nationwide energy saving from the paper's inputs ===")
area_m2 = AREA_FT2 * FT2_TO_M2
E_MJ = SAVING_REPORTED * area_m2
E_MWH = E_MJ / MJ_PER_MWH
E_KWH = E_MJ / MJ_PER_KWH
print(f"area = 97e9 ft2 x {FT2_TO_M2} = {area_m2:.4e} m2 ({area_m2/1e9:.2f} billion m2)")
print(f"E = {SAVING_REPORTED} MJ/m2 x {area_m2:.4e} m2 = {E_MJ:.4e} MJ")
print(f" = {E_MWH:.4e} MWh = {E_MWH/1e6:.2f} million MWh = {E_MWH/1e6:.2f} TWh")
print(f" = {E_KWH:.4e} kWh = {E_KWH/1e9:.2f} billion kWh")
E_MWH_mean = sav.mean() * area_m2 / MJ_PER_MWH
print(f"with the unrounded Table S7 mean ({sav.mean():.4f}): {E_MWH_mean/1e6:.2f} million MWh")
ratio = E_REPORTED_MWH / E_MWH
print(f"reported 66.87e9 MWh / recomputed = {ratio:.1f}")
implied = E_REPORTED_MWH * MJ_PER_MWH / area_m2
print(f"per-area saving implied by 66.87e9 MWh = {implied:,.0f} MJ/m2 ({implied/1000:.2f} GJ/m2)")
print(f" = {implied/lowe.max():.1f} x the largest total annual HVAC use in Table S7 "
f"({lowe.max():.2f} MJ/m2, low-e glass, Alaska)")
print(f" = {implied/sav.max():.0f} x the largest single-state saving ({sav.max():.2f} MJ/m2)")
print(f"if 97e9 were read as m2 rather than ft2: {SAVING_REPORTED*97e9/MJ_PER_MWH/1e6:.0f} million MWh "
f"(factor {SAVING_REPORTED*97e9/MJ_PER_MWH/E_MWH:.2f}, not 1000)")
print(f"years of savings needed to reach 66.87e9 MWh: {ratio:.0f}")
print("\n=== 3. Peer review file (pre-revision values) ===")
for k, (s, rep) in PRF.items():
e = s * area_m2 / MJ_PER_MWH
print(f"{k:6s}: {s} MJ/m2 -> {e/1e6:.2f} million MWh; table lists {rep} -> ratio "
f"{rep*1e9/e:.1f} if read as billion MWh")
print("\n=== 4. Cost and CO2 (Table S8; Eq. S10 C = LCOE*E/1e12, Eq. S11 T = f*E/1e12) ===")
print(f"{'Technology':14s} {'LCOE':>5s} {'S8 cost':>8s} {'LCOE*Erep':>10s} {'cost w/ E':>10s} "
f"{'f':>5s} {'S8 CO2':>7s} {'f*Erep':>7s} {'CO2 w/ E':>9s}")
print(f"{'':14s} {'$/MWh':>5s} {'T$':>8s} {'T$':>10s} {'B$':>10s} {'kg/MWh':>5s} {'Bt':>7s} {'Bt':>7s} {'Mt':>9s}")
costs_true, co2_true = {}, {}
for k, (l, c, f, t) in S8.items():
costs_true[k] = l * E_MWH / 1e9 # billion USD
co2_true[k] = f * E_MWH / 1e9 # kg -> million t
print(f"{k:14s} {l:5d} {c:8.2f} {l*E_REPORTED_MWH/1e12:10.2f} {costs_true[k]:10.2f} "
f"{f:5d} {t:7.2f} {f*E_REPORTED_MWH/1e12:7.2f} {co2_true[k]:9.2f}")
ct = sorted(costs_true.values())
print(f"recomputed cost range: ${ct[0]:.2f}-{ct[-1]:.2f} billion "
f"(without offshore wind: ${ct[0]:.2f}-{sorted(ct)[-2]:.2f} billion); reported "
f"${TEXT_COST_RANGE[0]}-{TEXT_COST_RANGE[1]} trillion")
co = sorted(co2_true.values())
print(f"recomputed CO2 range: {co[0]:.2f}-{co[-1]:.2f} million t; reported "
f"{TEXT_CO2_RANGE[0]}-{TEXT_CO2_RANGE[1]} billion t")
print(f"implied $/MWh from the text range: {TEXT_COST_RANGE[0]*1e12/E_REPORTED_MWH:.1f} to "
f"{TEXT_COST_RANGE[1]*1e12/E_REPORTED_MWH:.1f}")
print(f"implied kg CO2/MWh from the text range: {TEXT_CO2_RANGE[0]*1e12/E_REPORTED_MWH:.1f} to "
f"{TEXT_CO2_RANGE[1]*1e12/E_REPORTED_MWH:.1f}")
print(f"Table S8 offshore wind cost {S8['Wind offshore'][1]} T$ lies outside the text range "
f"{TEXT_COST_RANGE}: {S8['Wind offshore'][1] > TEXT_COST_RANGE[1]}")
print(f"Table S8 solar PV CO2 {S8['Solar PV'][3]} Bt vs text 'nuclear, photovoltaic, and wind ... 0.87'")
print("\n=== 5. Plausibility against U.S. totals (EIA) ===")
cbecs_twh = CBECS_2018_QUADS * 1e15 * J_PER_BTU / 3.6e15
rep_twh, true_twh = E_REPORTED_MWH / 1e6, E_MWH / 1e6
print(f"reported {rep_twh:,.0f} TWh; recomputed {true_twh:.1f} TWh")
print(f"US utility-scale generation 2023 = {US_GEN_2023_TWH:,.0f} TWh -> reported is "
f"{rep_twh/US_GEN_2023_TWH:.1f}x; recomputed is {100*true_twh/US_GEN_2023_TWH:.2f}%")
print(f"US commercial-building energy 2018 = {CBECS_2018_QUADS} quads = {cbecs_twh:,.0f} TWh -> reported is "
f"{rep_twh/cbecs_twh:.1f}x; recomputed is {100*true_twh/cbecs_twh:.2f}%")
# ================================================================ figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": SEC,
"svg.fonttype": "none", "font.family": "DejaVu Sans"})
def save(fig, name):
fig.savefig(os.path.join(HERE, f"{name}.svg"), facecolor=BG, bbox_inches="tight")
fig.savefig(os.path.join(HERE, f"{name}.png"), facecolor=BG, bbox_inches="tight", dpi=200)
plt.close(fig)
def box(ax, x, y, w, h, title, body, edge=SEC, fill="white", tcol=TXT, ls="-", lw=1.2):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc=fill, ec=edge, lw=lw, ls=ls))
ax.text(x + w / 2, y + h - 1.6, title, ha="center", va="top", fontsize=10, color=tcol, weight="bold")
ax.text(x + w / 2, y + h - 5.2, body, ha="center", va="top", fontsize=10, color=TXT, linespacing=1.35)
def arrow(ax, p, q, col=SEC):
ax.add_patch(FancyArrowPatch(p, q, arrowstyle="-|>", mutation_scale=13, lw=1.5, color=col,
shrinkA=0, shrinkB=0))
def sci(x):
e = int(np.floor(np.log10(x)))
return f"{x/10**e:.3f} × 10$^{{{e}}}$"
# ---------------------------------------------------------------- fig1: calculation chain
fig = plt.figure(figsize=(7.2, 6.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 100); ax.axis("off")
ax.set_facecolor(BG)
box(ax, 2, 85, 43, 12, "Mean saving per m² (Table S7)",
f"mean of 50 states = {sav.mean():.3f} MJ/m²\n(paper: {SAVING_REPORTED} MJ/m²)")
box(ax, 55, 85, 43, 12, "U.S. commercial floor area (ref. 54)",
f"97 × 10$^{{9}}$ ft² × {FT2_TO_M2:.4f}\n= {area_m2/1e9:.2f} × 10$^{{9}}$ m²")
arrow(ax, (23.5, 84.2), (40, 77.6)); arrow(ax, (76.5, 84.2), (60, 77.6))
box(ax, 22, 65, 56, 12, "Annual saving in energy units",
f"{SAVING_REPORTED} MJ/m² × {area_m2/1e9:.2f} × 10$^{{9}}$ m² = {sci(E_MJ)} MJ")
arrow(ax, (50, 64.2), (50, 58.4))
# highlighted divergence step
ax.add_patch(FancyBboxPatch((6, 45), 88, 13, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="#fdf0ea", ec=ORANGE, lw=1.6, ls=(0, (4, 2))))
ax.text(50, 56.4, "Step where the figures diverge: MJ → MWh", ha="center", va="top",
fontsize=10.5, weight="bold", color=TXT)
ax.text(50, 52.4, f"1 MWh = 3,600 MJ, so {sci(E_MJ)} MJ = {sci(E_MWH)} MWh.\n"
f"The paper's figure equals the same total in kWh ({E_KWH/1e9:.2f} billion kWh), stated as MWh.",
ha="center", va="top", fontsize=10, color=TXT, linespacing=1.35)
arrow(ax, (28, 44.2), (26, 37.6), BLUE); arrow(ax, (72, 44.2), (74, 37.6), ORANGE)
box(ax, 2, 25, 46, 12.5, "Recomputed from the paper's inputs",
f"{E_MWH/1e6:.1f} million MWh\n({E_MWH/1e6:.1f} TWh per year)", edge=BLUE, tcol=BLUE, lw=2)
box(ax, 52, 25, 46, 12.5, "Reported (Abstract, Results, Discussion)",
f"66.87 billion MWh\n(ratio to recomputed: {ratio:.1f})", edge=ORANGE, tcol=ORANGE, lw=2)
arrow(ax, (25, 24.2), (25, 18.2), BLUE); arrow(ax, (75, 24.2), (75, 18.2), ORANGE)
box(ax, 2, 3, 46, 15, "Eqs. S10–S11 with this E",
f"cost ${ct[0]:.2f}–{sorted(ct)[-2]:.2f} billion\n(offshore wind ${ct[-1]:.2f} billion)\n"
f"CO$_2$ {co[0]:.2f}–{co[-1]:.2f} million t", edge=BLUE, tcol=BLUE, lw=2)
box(ax, 52, 3, 46, 15, "Reported (Table S8 and text)",
f"cost ${TEXT_COST_RANGE[0]}–{TEXT_COST_RANGE[1]} trillion\n(Table S8 offshore wind 8.36 trillion)\n"
f"CO$_2$ {TEXT_CO2_RANGE[0]}–{TEXT_CO2_RANGE[1]} billion t", edge=ORANGE, tcol=ORANGE, lw=2)
save(fig, "fig1")
# ---------------------------------------------------------------- fig2: scale on one log axis
fig, ax = plt.subplots(figsize=(7.2, 3.3), facecolor=BG)
ax.set_facecolor(BG)
rows = [
("Paper: nationwide saving", rep_twh, ORANGE, "o", f"{rep_twh:,.0f} TWh (66.87 billion MWh)"),
("Recomputed from the\npaper's own inputs", true_twh, BLUE, "o", f"{true_twh:.1f} TWh (66.9 million MWh)"),
("All U.S. utility-scale\nelectricity, 2023 (EIA)", US_GEN_2023_TWH, SEC, "s", f"{US_GEN_2023_TWH:,.0f} TWh"),
("All energy used in U.S.\ncommercial buildings,\n2018 (EIA CBECS)", cbecs_twh, SEC, "s",
f"{cbecs_twh:,.0f} TWh (6.8 quadrillion Btu)"),
]
ys = np.arange(len(rows))[::-1]
for (lab, v, c, m, txt), y in zip(rows, ys):
ax.plot([v], [y], marker=m, ms=10, color=c, mec=c, ls="none", zorder=3)
if v > 2e4:
ax.text(v / 1.35, y, txt, ha="right", va="center", fontsize=10, color=c)
else:
ax.text(v * 1.35, y, txt, ha="left", va="center", fontsize=10, color=c)
ax.set_xscale("log")
ax.set_xlim(10, 3e5)
ax.set_ylim(-0.6, len(rows) - 0.4)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=TXT)
ax.set_xlabel("Annual energy, TWh (log scale; 1 TWh = 1 million MWh)", fontsize=10)
ax.grid(axis="x", which="major", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.annotate("", xy=(true_twh, ys[1] + 0.5), xytext=(rep_twh, ys[1] + 0.5),
arrowprops=dict(arrowstyle="<->", color=SEC, lw=1.2))
ax.text(np.sqrt(rep_twh * true_twh), ys[1] + 0.42, f"ratio {ratio:.1f}", ha="center", va="top",
fontsize=10, color=SEC)
save(fig, "fig2")
print("\nfigures written: fig1.svg/png, fig2.svg/png")
Abstract reports adverse-event rates of 17% and 10%; Results and Table 3 give 62% and 55%
Arthritis Rheumatol (2019) · doi:10.1002/art.41012 · PMC6900077
Summary
The abstract of this phase IIb/III trial of fasinumab in osteoarthritis pain states that treatment-emergent adverse event (TEAE) rates "were 17% with fasinumab and 10% with placebo". The Results text and Table 3 give the TEAE incidence for the 16-week treatment period as 62% and 55%, and Table 3's counts reproduce those figures exactly (209/337 = 62.0%; 45/82 = 54.9%). No period, organ class, population or definition in the paper yields 17% and 10% together, while the discontinuation figures in the same abstract sentence do match the body. The abstract's only overall tolerability figure therefore understates the absolute adverse-event burden by about 45 percentage points in each arm and overstates the relative excess (1.7-fold implied vs 1.13-fold in the data).
What the paper reports
Abstract, Results:
Treatment-emergent adverse event rates were 17% with fasinumab and 10% with placebo, and 4% and 1% of patients, respectively, discontinued treatment.
Patients and Methods, Safety end points:
Safety was evaluated based on the frequency of treatment-emergent adverse events (TEAEs), AEs of special interest (adjudicated arthropathy and sympathetic nervous system dysfunction), and laboratory tests.
Results, Safety, paragraph 2:
During the 16-week treatment period, the incidence of TEAEs was 62% with fasinumab and 55% with placebo (Table 3). Nervous system and musculoskeletal symptoms were more frequent following treatment with fasinumab (17% and 19%, respectively) than with placebo (9% and 17%, respectively).
Results, Safety, paragraph 3:
The incidence of serious TEAEs during treatment was low (2% with placebo versus 1% with fasinumab).
A low proportion of patients discontinued therapy due to TEAEs (4% [n = 14] in the fasinumab group and 1% [n = 1] in the placebo group).
Results, Safety, paragraph 4:
During the 20-week follow-up, the incidence of TEAEs was higher in the combined fasinumab group than in the placebo group (48% versus 38%) (Table 3)
Table 3 ("TEAEs reported in >3% of patients during the treatment and follow-up periods, by system organ class (safety analysis set)"; columns: placebo n = 82 | 1 mg n = 85 | 3 mg n = 84 | 6 mg n = 85 | 9 mg n = 83 | combined n = 337), treatment period and follow-up period summary rows:
≥1 TEAE | 45 (54.9) | 54 (63.5) | 52 (61.9) | 55 (64.7) | 48 (57.8) | 209 (62.0)
≥1 posttreatment AE | 31 (37.8) | 36 (42.4) | 38 (45.2) | 42 (49.4) | 44 (53.0) | 160 (47.5)
Results, Disposition of the patients:
During the study, 419 patients received ≥1 dose of study medication (1 patient randomized to receive placebo and 1 randomized to receive 9 mg fasinumab discontinued before drug administration).
Discussion, paragraph 5:
Fasinumab was generally well tolerated in the present study. Although the rate of TEAEs was higher with fasinumab than with placebo, there were few discontinuations attributable to TEAEs.
What the paper's own numbers show
Method. I took the counts and group sizes from Table 3 (safety analysis set), checked that the four dose columns add up to the combined column in every row of Table 3 (summary, organ-class and preferred-term rows, 42 in all), and recomputed every printed percentage in all six columns. I compared the abstract's 17%/10% with the treatment-period and follow-up-period rows and with every system-organ-class row in Table 3. I also listed which integer counts would print as 17% of 337 and 10% of 82, and checked the full-analysis-set denominators (338 and 83). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Any TEAE, fasinumab, treatment period | Abstract, Results | 17% | 209/337 = 62.0% |
| Any TEAE, placebo, treatment period | Abstract, Results | 10% | 45/82 = 54.9% |
| Any TEAE, fasinumab vs placebo | Results, Safety ¶2 | 62% vs 55% | 62.0% vs 54.9% (agrees) |
| ≥1 TEAE, combined column | Table 3 | 209 (62.0) | 54 + 52 + 55 + 48 = 209 of 85 + 84 + 85 + 83 = 337; 62.0% (agrees) |
| Any AE, follow-up period | Results, Safety ¶4; Table 3 | 48% vs 38%; 160 (47.5) vs 31 (37.8) | 160/337 = 47.5%; 31/82 = 37.8% (agrees; not 17%/10%) |
| Discontinuation due to TEAE | Abstract; Results, Safety ¶3 | 4% vs 1% (n = 14, n = 1) | 14/337 = 4.2%; 1/82 = 1.2% (agrees) |
| Nervous system disorders, treatment period | Results, Safety ¶2; Table 3 | 17% vs 9% | 57/337 = 16.9%; 7/82 = 8.5% |
| Vascular disorders, treatment period | Results, Safety ¶3; Table 3 | 6% vs 10% | 19/337 = 5.6%; 8/82 = 9.8% |
| Serious TEAEs, treatment period | Results, Safety ¶3 | 1% vs 2% | no counts given; not 17%/10% |
| Counts that would print as 17% and 10% | derived | — | 56–58 of 337 fasinumab patients and exactly 8 of 82 placebo patients |
| Table 3 rows giving 17% and 10% together | Table 3 (all 42 rows, both periods) | — | none |
| Any TEAE with full-analysis-set denominators | Disposition (338, 83) | — | 209/338 = 61.8%; 45/83 = 54.2% |
| Abstract minus Table 3, treatment period | derived | — | −45.0 pp (fasinumab), −44.9 pp (placebo) |
| Table 3 rate ÷ abstract rate | derived | — | 3.65 (fasinumab), 5.49 (placebo) |
| Risk ratio, fasinumab vs placebo | derived | 17/10 = 1.70 implied | (209/337)/(45/82) = 1.130 |
| Absolute difference | derived | 7 pp implied | 7.1 pp |
| Fisher exact test, 209/337 vs 45/82 | derived | — | OR 1.34, two-sided p = 0.26 |
Table 3 is internally consistent: in all 42 rows (summary, organ-class and preferred-term) the dose columns sum to the combined column, and all 252 printed percentages, in every dose column, match their counts. The Results text agrees with Table 3. Only the abstract's TEAE rates differ.
Figure
Figure 1. The conflicting statements, side by side. Left: the abstract's sentence, with the TEAE rates it prints in orange. Right: the Results (Safety, paragraph 2) sentence on the same quantity and the Table 3 "≥1 TEAE" counts, with the percentages recomputed from those counts in blue (209/337 = 62.0%, 45/82 = 54.9%). Both describe any TEAE in the 16-week treatment period. The discontinuation figures in the same abstract sentence (4% and 1%) agree with the body (14/337 = 4.2%, 1/82 = 1.2%).
Figure 2. Every summary row and system-organ-class row of Table 3, plotted as the placebo rate (x) against the combined fasinumab rate (y), both recomputed from the counts (blue; filled for the any-adverse-event rows, hollow for organ classes; circles for the treatment period, squares for follow-up; preferred-term rows are omitted, as none exceeds 11% in either arm). The orange cross is the abstract's pair, 17% vs 10%. No row lies at that point: the nervous-system row matches the fasinumab figure but not the placebo one (16.9% vs 8.5%), and the vascular row matches the placebo figure but not the fasinumab one (5.6% vs 9.8%). The gray diagonal marks equal rates in the two arms.
Why it matters
The abstract sentence is the abstract's only statement of overall tolerability, and readers who use the abstract alone (indexing services, evidence summaries, secondary coverage) receive the wrong figure.
- Absolute burden. The abstract implies that about 1 in 6 fasinumab-treated patients and 1 in 10 placebo patients had a TEAE. Table 3 shows about 3 in 5 and 1 in 2 (62.0% and 54.9%). The abstract understates both arms by about 45 percentage points.
- Relative excess. The abstract's figures imply a 1.7-fold risk with fasinumab (a 70% relative increase). The data give a 1.13-fold risk (OR 1.34; Fisher exact p = 0.26). The absolute difference is about 7 points either way (7 pp as stated, 7.1 pp recomputed). So the abstract makes the overall burden look smaller and the difference between arms look proportionally larger than the data show.
- What is unaffected. The direction is the same in both versions: TEAEs were more frequent with fasinumab. The discontinuation figures (4% vs 1%), the arthropathy figures, the serious-TEAE statement in the body and all efficacy results are not affected. The body's statements that fasinumab was "generally well tolerated" and that "the rate of TEAEs was higher with fasinumab than with placebo" are worded consistently with Table 3. The abstract's Conclusion ("The observed benefit-to-risk relationship favors further clinical development to explore the lowest doses") rests mainly on the efficacy and arthropathy results, which are correct, but a reader weighing that conclusion from the abstract alone would be weighing it against an adverse-event rate of 17% rather than 62%.
Explanations considered
- A different study period. This does not account for the numbers. The treatment period gives 62.0% vs 54.9% and the follow-up period gives 47.5% vs 37.8%.
- A single system organ class. This does not account for the numbers. Nervous system disorders give 16.9% (17%) for fasinumab but 8.5% (9%) for placebo, and the Results text itself prints them as 17% vs 9%. Vascular disorders give 9.8% (10%) for placebo but 5.6% for fasinumab. No Table 3 row in either period gives 17% and 10% together; that would need 56–58 of 337 fasinumab patients and 8 of 82 placebo patients in the same row (Figure 2).
- Serious TEAEs. This does not account for the numbers. The Results text gives 1% with fasinumab and 2% with placebo.
- Treatment-related (drug-related) TEAEs. This cannot be checked, and the wording argues against it. The paper reports no treatment-related TEAE rate anywhere, and the abstract uses the same term, "treatment-emergent adverse event", that the Methods define as the safety end point.
- A different analysis population. This does not account for the numbers. Table 3 uses the safety analysis set (82 and 337), and the abstract itself refers to "1 of 337 treated patients". Using the randomized numbers (83 and 338) gives 54.2% and 61.8%.
- A conversion or typesetting artifact. This is excluded as far as the available copies allow. The same abstract sentence appears in the PMC article, the Europe PMC full-text XML and the PubMed record (PMID 31207169).
- A correction elsewhere. None was found (see Status).
Status
- Version of record. The publisher's landing page and PDF (acrjournals.onlinelibrary.wiley.com, reached from the DOI) returned HTTP 403 to automated access and could not be read. The abstract sentence, the Results sentences and the Table 3 rows quoted above are identical in the PMC article (live page and archived text), the Europe PMC full-text XML, and the PubMed and Europe PMC abstract records. Every quote above was checked verbatim against the PMC-archived text.
- Supplements. The single supporting file (ART-71-1824-s001.docx, from the Europe PMC supplementary-files endpoint) holds Supplementary Methods, Supplementary Table 1 (WOMAC physical function) and Supplementary Figures 1–4. It contains no adverse-event tables and does not resolve the discrepancy. A ClinicalTrials.gov record that appears to correspond to this study (NCT02447276) has no posted results; the article gives no registration number, so the match is unconfirmed.
- Corrections and comments. As of 2026-10-23 no correction, erratum, retraction or expression of concern was found. Crossref lists no update relations for the DOI; PMC, Europe PMC and the PubMed record list no correction or comment links; the Crossref Labs record shows no Retraction Watch entry. Web searches for an erratum, correction or PubPeer discussion found none. PubPeer itself could not be queried directly (API 404 / access challenge).
- Sources. PMC article PMC6900077; Europe PMC full text and supplementary files; PubMed PMID 31207169; Crossref and Crossref Labs records for doi:10.1002/art.41012; ClinicalTrials.gov NCT02447276.
Suggested correction
- In the Abstract (Results), replace "Treatment-emergent adverse event rates were 17% with fasinumab and 10% with placebo" with "Treatment-emergent adverse event rates during the 16-week treatment period were 62% with fasinumab and 55% with placebo", matching the Results text and Table 3 (209/337 and 45/82). The discontinuation figures in the same sentence (4% and 1%) are correct and can stay.
- If the 17% and 10% were intended to describe a different quantity, name it in the abstract and report it, with counts, in the Results or Table 3, since no such quantity currently appears in the paper.
- Ask indexing services (PubMed, Europe PMC) to carry the corrected abstract.
Recomputation script (python recompute.py.txt)
"""Recompute the adverse-event rates of PMC6900077
(Arthritis Rheumatol 2019, doi:10.1002/art.41012).
Run: python recompute.py.txt (numbers + fig1/fig2)
python recompute.py.txt --candidates (also the five review candidates)
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes, next to this file:
fig1.svg/png, fig2.svg/png (the figures used in writeup.md)
candidates/cand1..cand5.svg/png (five candidate figure designs for internal review; written
only with --candidates; not part of the journal package)
Inputs are copied from the paper as archived in PMC:
Table 3 (safety analysis set): every row (summary, system organ class and preferred term),
all six columns, counts and printed percentages, as TABLE3_TEXT below
Results, Safety paragraph 3: discontinuations due to TEAEs (n = 14 fasinumab, n = 1 placebo)
Results, Disposition: randomized n = 338 fasinumab, 83 placebo (full analysis set)
Abstract, Results: "Treatment-emergent adverse event rates were 17% with fasinumab and
10% with placebo, and 4% and 1% of patients, respectively, discontinued treatment."
"""
import os
import re
import sys
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
from scipy.stats import fisher_exact # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
WRITE_CANDIDATES = "--candidates" in sys.argv[1:]
# ------------------------------------------------------------------ inputs (from the paper)
N_PBO, N_FAS = 82, 337 # Table 3 header, safety analysis set
DOSES = {"1 mg": 85, "3 mg": 84, "6 mg": 85, "9 mg": 83}
N_RAND_PBO, N_RAND_FAS = 83, 338 # Results, Disposition (randomized)
ABSTRACT = {"teae_fas": 17, "teae_pbo": 10, "disc_fas": 4, "disc_pbo": 1} # Abstract, %
TEXT = {"teae_fas": 62, "teae_pbo": 55, "fu_fas": 48, "fu_pbo": 38} # Results text, %
DISC = {"fas": 14, "pbo": 1} # Results, Safety paragraph 3
SERIOUS_TEXT = {"fas": 1, "pbo": 2} # Results, Safety paragraph 3, % (no counts given)
# Table 3, every row as printed: period (T = treatment, F = follow-up) | row | placebo |
# 1 mg | 3 mg | 6 mg | 9 mg | combined; each cell "n (%)", or "0".
TABLE3_TEXT = """\
T | ≥1 TEAE | 45 (54.9) | 54 (63.5) | 52 (61.9) | 55 (64.7) | 48 (57.8) | 209 (62.0)
T | Infections and infestations | 13 (15.9) | 18 (21.2) | 17 (20.2) | 16 (18.8) | 21 (25.3) | 72 (21.4)
T | Upper respiratory tract infection | 1 (1.2) | 5 (5.9) | 3 (3.6) | 3 (3.5) | 7 (8.4) | 18 (5.3)
T | Urinary tract infection | 3 (3.7) | 5 (5.9) | 1 (1.2) | 4 (4.7) | 3 (3.6) | 13 (3.9)
T | Sinusitis | 3 (3.7) | 1 (1.2) | 2 (2.4) | 2 (2.4) | 4 (4.8) | 9 (2.7)
T | Musculoskeletal and connective tissue disorders | 14 (17.1) | 16 (18.8) | 15 (17.9) | 18 (21.2) | 14 (16.9) | 63 (18.7)
T | Arthralgia | 2 (2.4) | 9 (10.6) | 5 (6.0) | 8 (9.4) | 5 (6.0) | 27 (8.0)
T | Back pain | 2 (2.4) | 4 (4.7) | 1 (1.2) | 2 (2.4) | 3 (3.6) | 10 (3.0)
T | Joint swelling | 0 | 1 (1.2) | 6 (7.1) | 3 (3.5) | 0 | 10 (3.0)
T | Pain in extremity | 3 (3.7) | 3 (3.5) | 1 (1.2) | 2 (2.4) | 4 (4.8) | 10 (3.0)
T | Musculoskeletal pain | 4 (4.9) | 0 | 1 (1.2) | 2 (2.4) | 1 (1.2) | 4 (1.2)
T | Myalgia | 0 | 1 (1.2) | 0 | 2 (2.4) | 0 | 3 (0.9)
T | Osteoarthritis | 0 | 0 | 0 | 0 | 3 (3.6) | 3 (0.9)
T | Nervous system disorders | 7 (8.5) | 15 (17.6) | 14 (16.7) | 15 (17.6) | 13 (15.7) | 57 (16.9)
T | Headache | 5 (6.1) | 7 (8.2) | 2 (2.4) | 4 (4.7) | 4 (4.8) | 17 (5.0)
T | Paresthesia | 0 | 2 (2.4) | 4 (4.8) | 0 | 4 (4.8) | 10 (3.0)
T | Dizziness | 2 (2.4) | 3 (3.5) | 2 (2.4) | 3 (3.5) | 1 (1.2) | 9 (2.7)
T | Hypoesthesia | 1 (1.2) | 2 (2.4) | 3 (3.6) | 2 (2.4) | 1 (1.2) | 8 (2.4)
T | Gastrointestinal disorders | 7 (8.5) | 9 (10.6) | 12 (14.3) | 9 (10.6) | 5 (6.0) | 35 (10.4)
T | Nausea | 3 (3.7) | 6 (7.1) | 1 (1.2) | 2 (2.4) | 1 (1.2) | 10 (3.0)
T | Diarrhea | 3 (3.7) | 1 (1.2) | 1 (1.2) | 4 (4.7) | 2 (2.4) | 8 (2.4)
T | Dry mouth | 1 (1.2) | 3 (3.5) | 4 (4.8) | 1 (1.2) | 0 | 8 (2.4)
T | Vomiting | 0 | 1 (1.2) | 3 (3.6) | 1 (1.2) | 2 (2.4) | 7 (2.1)
T | Vascular disorders | 8 (9.8) | 6 (7.1) | 4 (4.8) | 5 (5.9) | 4 (4.8) | 19 (5.6)
T | Orthostatic hypotension | 3 (3.7) | 3 (3.5) | 3 (3.6) | 3 (3.5) | 2 (2.4) | 11 (3.3)
T | Hypertension | 4 (4.9) | 1 (1.2) | 1 (1.2) | 2 (2.4) | 1 (1.2) | 5 (1.5)
T | Skin and subcutaneous tissue disorders | 1 (1.2) | 6 (7.1) | 4 (4.8) | 2 (2.4) | 5 (6.0) | 17 (5.0)
T | Rash | 0 | 1 (1.2) | 2 (2.4) | 1 (1.2) | 4 (4.8) | 8 (2.4)
F | ≥1 posttreatment AE | 31 (37.8) | 36 (42.4) | 38 (45.2) | 42 (49.4) | 44 (53.0) | 160 (47.5)
F | Musculoskeletal and connective tissue disorders | 10 (12.2) | 15 (17.6) | 20 (23.8) | 25 (29.4) | 23 (27.7) | 83 (24.6)
F | Arthralgia | 5 (6.1) | 3 (3.5) | 11 (13.1) | 12 (14.1) | 9 (10.8) | 35 (10.4)
F | Rapidly progressive OA | 0 | 2 (2.4) | 2 (2.4) | 5 (5.9) | 7 (8.4) | 16 (4.7)
F | OA | 0 | 3 (3.5) | 4 (4.8) | 3 (3.5) | 4 (4.8) | 14 (4.2)
F | Musculoskeletal pain | 2 (2.4) | 2 (2.4) | 5 (6.0) | 4 (4.7) | 1 (1.2) | 12 (3.6)
F | Joint swelling | 1 (1.2) | 1 (1.2) | 2 (2.4) | 4 (4.7) | 1 (1.2) | 8 (2.4)
F | Pain in extremity | 1 (1.2) | 1 (1.2) | 1 (1.2) | 3 (3.5) | 2 (2.4) | 7 (2.1)
F | Infections and infestations | 7 (8.5) | 11 (12.9) | 15 (17.9) | 8 (9.4) | 12 (14.5) | 46 (13.6)
F | Upper respiratory tract infection | 0 | 2 (2.4) | 7 (8.3) | 3 (3.5) | 0 | 12 (3.6)
F | Urinary tract infection | 2 (2.4) | 3 (3.5) | 2 (2.4) | 0 | 1 (1.2) | 6 (1.8)
F | Bronchitis | 1 (1.2) | 0 | 1 (1.2) | 0 | 3 (3.6) | 4 (1.2)
F | Vascular disorders | 4 (4.9) | 2 (2.4) | 5 (6.0) | 3 (3.5) | 5 (6.0) | 15 (4.5)
F | Orthostatic hypotension | 4 (4.9) | 2 (2.4) | 4 (4.8) | 0 | 4 (4.8) | 10 (3.0)
"""
def parse_table3(text):
out = []
for line in text.strip().splitlines():
c = [x.strip() for x in line.split("|")]
cells = []
for x in c[2:]:
m = re.fullmatch(r"(\d+)(?: \(([\d.]+)\))?", x)
n = int(m.group(1))
cells.append((n, float(m.group(2)) if m.group(2) else (0.0 if n == 0 else None)))
out.append((c[0], c[1], cells))
return out
TABLE3 = parse_table3(TABLE3_TEXT)
COL_N = [N_PBO] + list(DOSES.values()) + [N_FAS]
COL_NAMES = ["placebo", "1 mg", "3 mg", "6 mg", "9 mg", "combined"]
# Summary and system-organ-class rows used in the figures (subset of TABLE3, cross-checked below)
# (placebo n, [1, 3, 6, 9 mg n], combined n, printed placebo %, printed combined %)
TREAT = {
">=1 TEAE": (45, [54, 52, 55, 48], 209, 54.9, 62.0),
"Infections and infestations": (13, [18, 17, 16, 21], 72, 15.9, 21.4),
"Musculoskeletal and connective tissue": (14, [16, 15, 18, 14], 63, 17.1, 18.7),
"Nervous system": (7, [15, 14, 15, 13], 57, 8.5, 16.9),
"Gastrointestinal": (7, [9, 12, 9, 5], 35, 8.5, 10.4),
"Vascular": (8, [6, 4, 5, 4], 19, 9.8, 5.6),
"Skin and subcutaneous tissue": (1, [6, 4, 2, 5], 17, 1.2, 5.0),
}
FOLLOW = {
">=1 posttreatment AE": (31, [36, 38, 42, 44], 160, 37.8, 47.5),
"Musculoskeletal and connective tissue": (10, [15, 20, 25, 23], 83, 12.2, 24.6),
"Infections and infestations": (7, [11, 15, 8, 12], 46, 8.5, 13.6),
"Vascular": (4, [2, 5, 3, 5], 15, 4.9, 4.5),
}
def pct(a, b):
return 100.0 * a / b
def r1(x):
return round(x + 1e-9, 1)
# ------------------------------------------------------------------ 1. table arithmetic
print("=== 1. Table 3 arithmetic (safety analysis set) ===")
assert sum(DOSES.values()) == N_FAS
print(f"dose-group sizes {list(DOSES.values())} sum to {sum(DOSES.values())} = combined n {N_FAS}")
# 1a. every Table 3 row, every column
n_rows = len(TABLE3)
n_cells = sum(len(c) for _, _, c in TABLE3)
sum_bad = [(per, k) for per, k, c in TABLE3 if sum(n for n, _ in c[1:5]) != c[5][0]]
pct_bad = [(per, k, COL_NAMES[j], n, p, r1(pct(n, COL_N[j])))
for per, k, c in TABLE3 for j, (n, p) in enumerate(c) if p is None or r1(pct(n, COL_N[j])) != p]
print(f"all Table 3 rows: {n_rows} rows ({sum(1 for r in TABLE3 if r[0] == 'T')} treatment, "
f"{sum(1 for r in TABLE3 if r[0] == 'F')} follow-up), {n_cells} cells checked")
print(f" rows where 1 + 3 + 6 + 9 mg counts != combined count: {sum_bad if sum_bad else 'none'}")
print(f" cells (any column) where printed % != count/n rounded to 0.1: {pct_bad if pct_bad else 'none'}")
for label, rows in (("treatment period", TREAT), ("follow-up period", FOLLOW)):
print(f"\n[{label}]")
print(f" {'row':40s} {'pbo n':>5s} {'pbo %':>6s} {'fas n':>5s} {'fas %':>6s} doses sum printed % match")
for k, (p, ds, c, pp, pc) in rows.items():
ok_sum = sum(ds) == c
ok_pct = r1(pct(p, N_PBO)) == pp and r1(pct(c, N_FAS)) == pc
print(f" {k:40s} {p:5d} {pct(p, N_PBO):6.1f} {c:5d} {pct(c, N_FAS):6.1f} {str(ok_sum):9s} {ok_pct}")
# cross-check the figure subset against the full table
lookup = {(per, k): c for per, k, c in TABLE3}
for per, rows in (("T", TREAT), ("F", FOLLOW)):
for k, (p, ds, c, pp, pc) in rows.items():
full = [key for key in lookup if key[0] == per and key[1].replace("≥", ">=").startswith(k)]
assert len(full) == 1, (per, k, full)
cells = lookup[full[0]]
assert [n for n, _ in cells] == [p] + ds + [c] and cells[0][1] == pp and cells[5][1] == pc, k
print("\nsummary/organ-class subset used for the figures matches the full table")
teae_f, teae_p = pct(209, N_FAS), pct(45, N_PBO)
fu_f, fu_p = pct(160, N_FAS), pct(31, N_PBO)
print("\n=== 2. Any TEAE: abstract vs body ===")
print(f"treatment period, fasinumab: 209/337 = {teae_f:.1f}% (Results text {TEXT['teae_fas']}%, abstract {ABSTRACT['teae_fas']}%)")
print(f"treatment period, placebo: 45/82 = {teae_p:.1f}% (Results text {TEXT['teae_pbo']}%, abstract {ABSTRACT['teae_pbo']}%)")
print(f"follow-up period, fasinumab: 160/337 = {fu_f:.1f}% (Results text {TEXT['fu_fas']}%)")
print(f"follow-up period, placebo: 31/82 = {fu_p:.1f}% (Results text {TEXT['fu_pbo']}%)")
print(f"gap, abstract vs Table 3 (treatment): fasinumab {teae_f - 17:.1f} pp, placebo {teae_p - 10:.1f} pp")
print(f"Table 3 / abstract: fasinumab {teae_f / 17:.2f}x, placebo {teae_p / 10:.2f}x")
print(f"abstract implies risk ratio 17/10 = {17 / 10:.2f}; Table 3 gives ({209}/{N_FAS})/({45}/{N_PBO}) = {teae_f / teae_p:.3f}")
print(f"absolute difference: abstract {17 - 10} pp; Table 3 {teae_f - teae_p:.1f} pp")
odds, p = fisher_exact([[209, N_FAS - 209], [45, N_PBO - 45]])
print(f"Fisher exact test, 209/337 vs 45/82: OR = {odds:.2f}, two-sided p = {p:.2f}")
print(f"full analysis set denominators: 209/338 = {pct(209, N_RAND_FAS):.1f}%, 45/83 = {pct(45, N_RAND_PBO):.1f}%")
print("\n=== 3. Discontinuations due to TEAEs (same abstract sentence) ===")
print(f"fasinumab 14/337 = {pct(14, N_FAS):.1f}% (abstract 4%); placebo 1/82 = {pct(1, N_PBO):.1f}% (abstract 1%)")
print("\n=== 4. Which counts would print as 17% and 10%? ===")
f17 = [n for n in range(N_FAS + 1) if round(pct(n, N_FAS)) == 17]
p10 = [n for n in range(N_PBO + 1) if round(pct(n, N_PBO)) == 10]
print(f"fasinumab counts rounding to 17% of 337: {f17[0]}-{f17[-1]} "
f"({pct(f17[0], N_FAS):.1f}%-{pct(f17[-1], N_FAS):.1f}%)")
print(f"placebo counts rounding to 10% of 82: {p10} ({pct(p10[0], N_PBO):.1f}%)")
allrows = [("treatment", k, v) for k, v in TREAT.items()] + [("follow-up", k, v) for k, v in FOLLOW.items()]
hits = [(per, k) for per, k, (p_, _, c, _, _) in allrows
if round(pct(c, N_FAS)) == 17 and round(pct(p_, N_PBO)) == 10]
print(f"Table 3 summary/organ-class rows with fasinumab 17% AND placebo 10%: {hits if hits else 'none'}")
hits_all = [(per, k) for per, k, c in TABLE3
if round(pct(c[5][0], N_FAS)) == 17 and round(pct(c[0][0], N_PBO)) == 10]
print(f"any Table 3 row (incl. preferred terms) with fasinumab 17% AND placebo 10%: "
f"{hits_all if hits_all else 'none'}")
for per, k, (p_, _, c, _, _) in allrows:
if round(pct(c, N_FAS)) == 17 or round(pct(p_, N_PBO)) == 10:
print(f" partial match ({per}) {k}: fasinumab {pct(c, N_FAS):.1f}%, placebo {pct(p_, N_PBO):.1f}%")
print(f"serious TEAEs during treatment (Results text): fasinumab {SERIOUS_TEXT['fas']}%, placebo {SERIOUS_TEXT['pbo']}%")
# ------------------------------------------------------------------ figure style
BG, TXT, SEC, ORANGE, BLUE = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6"
GRID, PANEL = "#e4e3df", "#f3f2ef"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"axes.edgecolor": "#bdbcb8", "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def clean(ax, grid_axis="x"):
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.grid(axis=grid_axis, color=GRID, linewidth=0.8)
ax.set_axisbelow(True)
def panel(ax, x, y, w, h, title, lines):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc=PANEL, ec="#cfcdc8", lw=1))
ax.text(x + 0.02, y + h - 0.035, title, ha="left", va="top", fontsize=11, weight="bold")
yy = y + h - 0.13
for ln, col, size in lines:
ax.text(x + 0.02, yy, ln, ha="left", va="top", fontsize=size, color=col)
yy -= 0.068 * (ln.count("\n") + 1) + 0.02
# ---- candidate 1: two text panels joined by an annotated connector (abstract vs body)
def cand_text_panels():
fig = plt.figure(figsize=(W, 3.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
panel(ax, 0.02, 0.17, 0.40, 0.78, "Abstract, Results", [
("“Treatment-emergent adverse\nevent rates were", TXT, 10.5),
("17% with fasinumab and\n10% with placebo”", ORANGE, 11.5),
("Same sentence: discontinuations\n4% and 1%", SEC, 10),
])
panel(ax, 0.58, 0.17, 0.40, 0.78, "Results, Safety ¶2 and Table 3", [
("“the incidence of TEAEs was\n62% with fasinumab and\n55% with placebo (Table 3)”", TXT, 10.5),
("Table 3, ≥1 TEAE:\n209/337 = 62.0% (fasinumab)\n45/82 = 54.9% (placebo)", BLUE, 11),
("Discontinuations 14/337 = 4.2%\nand 1/82 = 1.2%", SEC, 10),
])
ax.annotate("", xy=(0.575, 0.62), xytext=(0.425, 0.62),
arrowprops=dict(arrowstyle="<->", color=TXT, lw=2))
ax.text(0.5, 0.66, "same quantity,\nsame period", ha="center", va="bottom", fontsize=10, color=SEC)
ax.text(0.5, 0.58, "17 vs 62\n10 vs 55", ha="center", va="top", fontsize=11, weight="bold")
ax.annotate("", xy=(0.575, 0.31), xytext=(0.425, 0.31),
arrowprops=dict(arrowstyle="<->", color=SEC, lw=1.2, ls="--"))
ax.text(0.5, 0.33, "agree", ha="center", va="bottom", fontsize=10, color=SEC)
ax.text(0.02, 0.07, "Orange: as printed in the abstract. Blue: recomputed from Table 3 counts\n"
"(any TEAE, 16-week treatment period, safety analysis set).", ha="left", va="center",
fontsize=10, color=SEC)
return fig
# ---- candidate 2: dot plot, abstract vs recomputed, on one 0-100% axis
def cand_dot_plot():
fig, ax = plt.subplots(figsize=(W, 3.3))
rows = [("Any TEAE, fasinumab", 17, teae_f), ("Any TEAE, placebo", 10, teae_p),
("Discontinued for TEAE, fasinumab", 4, pct(14, N_FAS)),
("Discontinued for TEAE, placebo", 1, pct(1, N_PBO))]
for i, (lab, rep, rec) in enumerate(rows):
y = len(rows) - 1 - i
ax.plot([rep, rec], [y, y], color="#bdbcb8", lw=2, zorder=1)
ax.scatter(rep, y, s=210, fc="none", ec=ORANGE, lw=2.5, zorder=4,
label="Abstract (as printed)" if i == 0 else None)
ax.scatter(rec, y, s=90, color=BLUE, zorder=3, ec=BG, lw=1.5,
label="Table 3 / Results counts, recomputed" if i == 0 else None)
if abs(rep - rec) > 5:
ax.text(rep, y + 0.22, f"{rep}%", ha="center", va="bottom", fontsize=10, color=TXT)
ax.text(rec, y + 0.22, f"{rec:.1f}%", ha="center", va="bottom", fontsize=10, color=TXT)
else:
ax.text(max(rep, rec) + 4, y, f"{rep}% vs {rec:.1f}% (agree)", ha="left", va="center",
fontsize=10, color=SEC)
ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows][::-1], color=TXT)
ax.set_xlim(0, 100); ax.set_ylim(-0.6, len(rows) - 0.25)
ax.set_xlabel("Patients (%), 16-week treatment period, safety analysis set")
clean(ax)
fig.legend(*ax.get_legend_handles_labels(), loc="upper center", ncol=2, frameon=False, fontsize=10)
fig.tight_layout(rect=(0, 0, 1, 0.9))
return fig
# ---- candidate 3: every Table 3 row as a point; the abstract's pair falls on none
def cand_search_scatter():
fig, ax = plt.subplots(figsize=(W, 5.4))
for per, rows, mk in (("treatment", TREAT, "o"), ("follow-up", FOLLOW, "s")):
for k, (p_, _, c, _, _) in rows.items():
overall = k.startswith(">=1")
ax.scatter(pct(p_, N_PBO), pct(c, N_FAS), s=70 if overall else 55, marker=mk,
fc=BLUE if overall else BG, ec=BLUE, lw=2, zorder=3)
ax.scatter(10, 17, s=90, marker="X", color=ORANGE, zorder=4, ec=BG, lw=1)
callouts = [("Nervous system", "nervous system, treatment:\n", (1.0, 34), (-0.2, 0.9)),
("Vascular", "vascular, treatment: ", (24, 5), (0.6, 0.0))]
for k, name, (tx, ty), (dx, dy) in callouts:
p_, _, c, _, _ = TREAT[k]
x0, y0 = pct(p_, N_PBO), pct(c, N_FAS)
ax.annotate(f"{name}{y0:.1f}% vs {x0:.1f}%", xy=(x0 + dx, y0 + dy), xytext=(tx, ty),
fontsize=10, color=SEC, va="center",
arrowprops=dict(arrowstyle="-", color=SEC, lw=0.8, shrinkA=2, shrinkB=4))
ax.text(teae_p - 1.5, teae_f, f"any TEAE, treatment\n{teae_f:.1f}% vs {teae_p:.1f}%",
fontsize=10, color=TXT, va="center", ha="right")
ax.text(fu_p - 1.5, fu_f, f"any AE, follow-up\n{fu_f:.1f}% vs {fu_p:.1f}%",
fontsize=10, color=TXT, va="center", ha="right")
ax.annotate("Abstract: 17% vs 10%\n(no Table 3 row is at this point)",
xy=(10.6, 17.6), xytext=(22, 31), fontsize=10, color=TXT, va="center",
bbox=dict(fc=BG, ec="none", pad=1.5),
arrowprops=dict(arrowstyle="-", color=SEC, lw=0.8, shrinkA=2, shrinkB=2))
ax.plot([0, 70], [0, 70], color="#bdbcb8", lw=1, zorder=0)
ax.text(60, 55, "equal rates", fontsize=10, color=SEC)
ax.set_xlim(0, 70); ax.set_ylim(0, 70)
ax.set_xlabel("Placebo, % of 82 patients"); ax.set_ylabel("Fasinumab combined, % of 337 patients")
clean(ax, "both")
from matplotlib.lines import Line2D
h = [Line2D([], [], marker="X", ls="", color=ORANGE, ms=10, label="Abstract (as printed)"),
Line2D([], [], marker="o", ls="", mfc=BLUE, mec=BLUE, ms=8, label="Any TEAE, treatment (recomputed)"),
Line2D([], [], marker="s", ls="", mfc=BLUE, mec=BLUE, ms=8, label="Any AE, follow-up (recomputed)"),
Line2D([], [], marker="o", ls="", mfc=BG, mec=BLUE, mew=2, ms=8, label="Organ-class rows, treatment"),
Line2D([], [], marker="s", ls="", mfc=BG, mec=BLUE, mew=2, ms=8, label="Organ-class rows, follow-up")]
fig.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, 0.005), ncol=2, frameon=False,
fontsize=10, columnspacing=2.0, handletextpad=0.4)
fig.subplots_adjust(left=0.10, right=0.97, top=0.97, bottom=0.28)
return fig
# ---- candidate 4: bars from zero, abstract vs treatment period vs follow-up
def cand_bars():
fig, ax = plt.subplots(figsize=(W, 3.4))
groups = [("Fasinumab (n = 337)", 17, teae_f, fu_f), ("Placebo (n = 82)", 10, teae_p, fu_p)]
h = 0.24
for i, (g, rep, tr, fu) in enumerate(groups):
y = 1 - i
ax.barh(y + h, rep, height=h - 0.03, color=ORANGE, label="Abstract: any TEAE (as printed)" if i == 0 else None)
ax.barh(y, tr, height=h - 0.03, color=BLUE, label="Table 3: ≥1 TEAE, treatment period" if i == 0 else None)
ax.barh(y - h, fu, height=h - 0.03, color=BG, ec=BLUE, lw=1.5, hatch="///",
label="Table 3: ≥1 AE, follow-up period" if i == 0 else None)
for yy, v, fmt in ((y + h, rep, f"{rep}%"), (y, tr, f"{tr:.1f}%"), (y - h, fu, f"{fu:.1f}%")):
ax.text(v + 1, yy, fmt, va="center", fontsize=10, color=TXT)
ax.set_yticks([1, 0]); ax.set_yticklabels([g[0] for g in groups], color=TXT)
ax.set_xlim(0, 100); ax.set_ylim(-0.55, 1.75)
ax.set_xlabel("Patients with at least one adverse event (%)")
clean(ax)
ax.legend(loc="upper right", frameon=False, fontsize=10)
fig.tight_layout()
return fig
# ---- candidate 5: calculation flow from Table 3 counts to the printed figures
def cand_flow():
fig = plt.figure(figsize=(W, 3.3))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
xs = [0.01, 0.29, 0.54, 0.79]; ws = [0.21, 0.17, 0.17, 0.20]
heads = ["Table 3 counts", "Recomputed", "Results text", "Abstract"]
for x, w, t in zip(xs, ws, heads):
ax.text(x + w / 2, 0.93, t, ha="center", va="center", fontsize=11, weight="bold")
rows = [(0.62, ["54+52+55+48\n= 209 of 337", f"209/337\n= {teae_f:.1f}%",
"“62% with\nfasinumab”", "“17% with\nfasinumab”"]),
(0.27, ["45 of 82\n(placebo)", f"45/82\n= {teae_p:.1f}%", "“55% with\nplacebo”",
"“10% with\nplacebo”"])]
for yc, cells in rows:
for j, (x, w, s) in enumerate(zip(xs, ws, cells)):
ec = ORANGE if j == 3 else (BLUE if j == 1 else "#cfcdc8")
ax.add_patch(FancyBboxPatch((x, yc - 0.13), w, 0.26, boxstyle="round,pad=0,rounding_size=0.012",
fc=PANEL, ec=ec, lw=2 if j in (1, 3) else 1))
ax.text(x + w / 2, yc, s, ha="center", va="center", fontsize=10.5,
color=TXT)
for j in range(3):
x0 = xs[j] + ws[j] + 0.005; x1 = xs[j + 1] - 0.005
ok = j < 2
ax.annotate("", xy=(x1, yc), xytext=(x0, yc),
arrowprops=dict(arrowstyle="->", color=SEC if ok else ORANGE, lw=2,
ls="-" if ok else "--"))
ax.text((x0 + x1) / 2, yc + 0.04, "=" if ok else "\u2260", ha="center", va="bottom",
fontsize=14, color=SEC if ok else TXT, weight="bold")
ax.text(0.01, 0.05, "Blue outline: recomputed from the paper's counts. Orange outline: abstract as printed.\n"
"= agrees after rounding; \u2260 disagrees. Any TEAE, 16-week treatment period, safety analysis set.",
fontsize=10, color=SEC, va="center")
return fig
CANDIDATES = [("cand1_text_panels", cand_text_panels), ("cand2_dot_plot", cand_dot_plot),
("cand3_search_scatter", cand_search_scatter), ("cand4_bars", cand_bars),
("cand5_calc_flow", cand_flow)]
if WRITE_CANDIDATES:
os.makedirs(CAND, exist_ok=True)
for name, fn in CANDIDATES:
save(fn(), os.path.join(CAND, name))
# Provisional choice for the write-up (to be swapped once a candidate is picked):
save(cand_text_panels(), os.path.join(HERE, "fig1"))
save(cand_search_scatter(), os.path.join(HERE, "fig2"))
print("\nwrote fig1, fig2" + (" and candidates/cand1..cand5" if WRITE_CANDIDATES else "") + " (.svg and .png)")
PIK3CA progression-free survival P = 0.016 cannot be derived from its reported HR and CI
BMC Med (2023) · doi:10.1186/s12916-023-03017-z · PMC10426213
Summary
This longitudinal ctDNA analysis of the phase Ib MONALEESASIA trial reports that patients with baseline PIK3CA alterations had shorter median progression-free survival (PFS) than patients without them, 12.7 vs 19.2 months, with P = 0.016. The Abstract gives this P value alongside the TP53 result and concludes that PIK3CA alterations "were associated with inferior outcomes". In the Results, the same P value is printed next to a hazard ratio of 0.93 (95% CI 0.52–1.69) in a sentence that calls the difference "not statistically significant". A Wald test back-calculated from that HR and CI gives z = −0.24 and P = 0.81, while the same check reproduces the paper's P values for TP53, bTMB and ctDNA detectability. The PIK3CA association in the Abstract therefore has no statistical support in the paper's own estimate. The paper's main findings on ctDNA monitoring do not depend on it.
What the paper reports
Abstract, Results:
Patients with alterations in PIK3CA and TP53 at baseline had shorter median progression-free survival compared with patients with wild-type PIK3CA and TP53, 12.7 and 7.3 months vs 19.2 and 19.4 months, respectively (P = .016 and P = .0001, respectively).
Abstract, Conclusions:
Higher ctDNA levels and PIK3CA and TP53 alterations detected at baseline were associated with inferior outcomes.
Results, "Gene alterations at baseline" (the PIK3CA and TP53 sentences, with the bTMB sentence that follows):
Median PFS of patients with TP53 mutations was significantly shorter than those without TP53 mutations (7.3 months vs 19.4 months; HR, 0.26; 95% CI, 0.12–0.53; P < 0.001; Fig. 7A). Median PFS of patients with PIK3CA mutations was 12.7 months vs 19.2 months for those without PIK3CA mutations; however, the difference was not statistically significant (HR, 0.93; 95% CI, 0.52–1.69; P = 0.016; Fig. 7B). Patients with undetectable ctDNA experienced a median PFS of 37.8 months (Fig. 7B).
In patients with detectable ctDNA, the median PFS of patients with high bTMB (> 10 mutations per megabase [mut/Mb]) was 12.7 months vs 19.4 months for those with low bTMB (< 10 mut/Mb); this numerically shorter PFS difference was not statistically significant (HR, 0.83; 95% CI, 0.46–1.5; P = 0.53; Fig. 7C).
Same paragraph, last sentence:
Therefore, in this particular cohort, neither bTMB nor PIK3CA status seems to provide an additional prognostic effect beyond ctDNA fraction.
Results, "ctDNA fraction by BOR and PFS" (used below as a control comparison):
More generally, patients with no detectable ctDNA at baseline had a PFS almost twice that observed in patients with detectable ctDNA at baseline: 38.0 months vs 19.2 months, respectively (hazard ratio [HR], 0.45; 95% CI, 0.24–0.84; P = 0.01; Fig. 1C).
Methods, "Statistical analysis":
A Cox proportional-hazards model was used to evaluate differences in PFS in the MONALEESASIA cohorts by ctDNA detectable vs undetectable status and mutation status in PIK3CA, TP53.
Methods, "Sample collection and ctDNA analysis":
All comparisons between mutated vs nonmutated (e.g., TP53 mutated vs nonmutated) were performed on patients with detectable ctDNA.
Figure 7 caption:
Progression-free survival for ribociclib plus ET in patients with or without mutations. A TP53. B PIK3CA. C High versus low TMB. Analysis by TMB (high versus low) only included patients with detectable ctDNA.
Figure 7 itself is an image. In the publisher's full-resolution file (12916_2023_3017_Fig7_HTML.png), panel B prints "P=. 016", panel A "P=.0001" and panel C "P=.53". Panels A and B each draw three Kaplan–Meier curves (mutated, nonmutated and "not detected/low power"); panel C draws two. The numbers at risk at time 0 are 12, 42 and 28 in panel A, 24, 30 and 28 in panel B, and 27 and 27 in panel C.
What the paper's own numbers show
Method. For a Cox model, a 95% Wald confidence interval is exp(b ± 1.96·SE), where b is the log hazard ratio. The standard error can therefore be recovered from the printed limits as SE = (ln upper − ln lower) / (2 × 1.96). The Wald statistic is z = ln(HR) / SE, and the two-sided P value is 2·(1 − Φ(|z|)). I applied this to all four PFS hazard ratios that the paper prints with a CI and a P value, using the three non-PIK3CA comparisons as controls for the method. I checked that each printed HR sits at the log-scale midpoint of its CI, that rounding of the printed values cannot move the result, and what HR or CI a P value of 0.016 would require. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| PIK3CA mutated vs nonmutated, P | Results; Abstract; Fig. 7B | HR 0.93 (95% CI 0.52–1.69); P = 0.016 | SE(ln HR) = 0.301, z = −0.24, Wald P = 0.81 |
| PIK3CA: rounding range of HR and CI limits | derived | HR 0.93, CI 0.52–1.69 | over HR 0.925–0.935 and limits ±0.005: P = 0.79–0.83 |
| PIK3CA: CI log-midpoint | derived | HR 0.93 | 0.937, so the HR and CI fit each other |
| PIK3CA: |z| that P = 0.016 implies | derived | P = 0.016 | |z| = 2.41, ten times the 0.24 that the CI gives |
| PIK3CA: CI that P = 0.016 would need at HR 0.93 | derived | CI 0.52–1.69 (SE 0.301) | SE 0.030, CI 0.877–0.987 (one tenth of the printed width) |
| PIK3CA: HR that P = 0.016 would need with the printed CI width | derived | HR 0.93 | 0.485 (CI 0.27–0.87) or 2.063 (CI 1.14–3.72) |
| TP53 mutated vs nonmutated, P (control) | Results; Abstract; Fig. 7A | HR 0.26 (0.12–0.53); P < 0.001 (Results), P = .0001 (Abstract, Fig. 7A) | z = −3.55, Wald P = 0.00038: consistent with < 0.001 |
| bTMB high vs low, P (control) | Results; Fig. 7C | HR 0.83 (0.46–1.5); P = 0.53 | z = −0.62, Wald P = 0.54: consistent |
| ctDNA undetectable vs detectable, P (control) | Results; Fig. 1C | HR 0.45 (0.24–0.84); P = 0.01 | z = −2.50, Wald P = 0.012: consistent |
| Numbers at risk at time 0 | Fig. 7A / 7B / 7C | 12 + 42 + 28; 24 + 30 + 28; 27 + 27 | 82, 82 and 54; the mutated vs nonmutated comparisons use the 54 patients with detectable ctDNA |
| Median PFS, undetectable ctDNA | Results (Fig. 1C text vs Fig. 7B text) | 38.0 months; 37.8 months | differ by 0.2 months (minor, separate from the P value issue) |
The method reproduces three of the four printed P values to within rounding. The PIK3CA value is the exception, by a factor of about 50 in P and 10 in z. The PIK3CA confidence interval contains 1, and an HR of 0.93 is close to no difference, so the Results sentence's own description ("not statistically significant") matches the HR and CI. It does not match the P value printed in the same parentheses.
Figure
Figure 1. For each of the four PFS comparisons that the paper reports with an HR, a 95% CI and a P value, the orange dot is the P value printed in the paper and the blue dot is the two-sided Wald P value recomputed from that HR and CI. The axis is logarithmic; the dashed line marks P = 0.05. For TP53 the orange dot is the Abstract and Figure 7A value (.0001); the Results give P < 0.001, which the recomputed 0.00038 satisfies. For bTMB and ctDNA detectability the two values coincide within rounding. For PIK3CA the printed P = 0.016 lies on the other side of 0.05 from the P = 0.81 that its own HR and CI imply.
Figure 2. The PIK3CA hazard ratio on a log scale, with HR = 1 as the reference line. Orange: the reported HR 0.93 with its 95% CI 0.52–1.69, which gives Wald P = 0.81. Blue: what P = 0.016 would require, either a CI one tenth as wide around the same HR (0.88–0.99), or, with the reported CI width, an HR of 0.48 or 2.06 whose interval excludes 1. Neither resembles the reported estimate, so a misprint in the HR alone or in one CI limit alone cannot produce the printed P value.
Why it matters
The PIK3CA result appears in the Abstract's Results with P = .016, paired with the TP53 result, and the Abstract's Conclusions list PIK3CA alterations among the baseline features "associated with inferior outcomes". A reader of the Abstract would take PIK3CA as a statistically supported adverse prognostic marker in this cohort. If the hazard ratio and confidence interval are correct, the comparison gives P ≈ 0.81 and an HR close to 1 with a wide interval (0.52–1.69). On that reading the data show no evidence of a PFS difference by PIK3CA status among patients with detectable ctDNA, which is also what the Results sentence and the paragraph's closing statement ("neither bTMB nor PIK3CA status seems to provide an additional prognostic effect beyond ctDNA fraction") say.
The effect is limited to this result. The full-text Conclusions do not rely on PIK3CA, and the Discussion describes the PIK3CA effect as "numerically shorter". The paper's main findings are unaffected: the decline of ctDNA on treatment, the relation between baseline ctDNA and PFS (HR 0.45, P = 0.01, reproduced as 0.012), the lead time of ctDNA increase before radiological progression, and the TP53 association (reproduced as P = 0.00038). The Discussion sentence "Patients with alterations in these genes experienced a shorter median PFS compared with patients with wild-type versions" remains true as a statement about medians but has no statistical support for PIK3CA.
A smaller point of interpretation: the printed HRs for TP53 (0.26), PIK3CA (0.93) and bTMB (0.83) are all below 1 although the mutated or high-bTMB groups have the shorter medians. They therefore appear to express the nonmutated or low-bTMB group relative to the other (for PIK3CA, about 1.08, 95% CI 0.59–1.92, mutated vs nonmutated). The direction does not change any P value in this write-up.
Explanations considered
- Typographical error in the HR or one CI limit. This does not account for the numbers. The printed HR lies at the log-scale midpoint of its CI (0.937 vs 0.93), so the three numbers fit each other. With the printed CI width, P = 0.016 would need an HR of 0.485 or 2.06; at HR 0.93 it would need a CI of 0.88–0.99. Changing any one printed number by a plausible typo does not get close.
- Rounding. This does not account for the numbers. Over the full rounding range of the HR and both limits, the Wald P stays between 0.79 and 0.83.
- A different test (for example a log-rank or likelihood-ratio test) for the same two groups. This is unlikely to account for the numbers. For a single two-group Cox comparison, Wald, score (log-rank) and likelihood-ratio tests usually agree closely; a gap from P = 0.81 to P = 0.016 would be extraordinary. The same Wald check reproduces the paper's P values for bTMB and for ctDNA detectability, which are two-group comparisons of similar size.
- The P value in Figure 7B is a three-group test that includes the ctDNA-undetectable patients. This is the most likely source of the number, but it is not confirmed. Panels 7A and 7B each draw three curves (mutated, nonmutated, and "not detected/low power", median PFS 37.8 months in the undetectable group), and the Methods name no test for the figures. The P values printed in those two panels fit a global test better than the pairwise one. A rough approximation that adds the 1-df detectable-vs-undetectable Wald component (χ² = 6.24, from Fig. 1C) to the pairwise component gives a 2-df P of about 0.04 for the PIK3CA panel (printed .016; 8.27 is needed for exactly 0.016) and about 0.00008 for the TP53 panel (printed .0001), whereas the pairwise TP53 Wald P is 0.00038. Panel 7C, which has only two curves, prints P = .53, matching its pairwise HR. If this is the explanation, the printed P = 0.016 tests whether PFS differs among the three groups, a difference driven largely by the long PFS of patients with undetectable ctDNA. It does not test PIK3CA mutated vs nonmutated, so it cannot support the Abstract's comparison "with wild-type PIK3CA", nor sit alongside an HR that the Methods say is computed only in patients with detectable ctDNA. The approximation ignores the correlation between components, so it shows only that the order of magnitude fits.
- A conversion or typesetting error in the archived text. This is excluded. The PMC text, the publisher-supplied JATS XML in Europe PMC, and the publisher's Figure 7 image all carry P = .016 for PIK3CA.
- The value is explained in a supplement. This is excluded. Neither supplementary file reports PIK3CA survival statistics (see Status).
Status
- Version of record. I compared the PMC-archived text with the publisher-supplied JATS XML in Europe PMC (PMC10426213/fullTextXML). Both carry the Abstract, Results and Methods sentences quoted above with identical values. I also read the publisher's full-resolution Figure 7 image (media.springernature.com, 12916_2023_3017_Fig7_HTML.png) for the printed P values, curves and numbers at risk. The Springer HTML and PDF pages could not be fetched automatically: they redirect to a cookie-authorization page. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py, except the Figure 7 panel contents, which were read from the image. - Supplements. Additional file 1 (12916_2023_3017_MOESM1_ESM.xlsx) is the 572-gene panel list (eTable 2). Additional file 2 (12916_2023_3017_MOESM2_ESM.docx) holds Supplementary Table 1 (cfDNA samples per cohort), Supplementary Table 2 (ctDNA change vs tumor change by response group) and Supplementary Fig. 1. Neither reports PIK3CA PFS, HRs or P values, and there is no patient-level survival data, so the correct P value cannot be computed from published material.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction notices. A web search for corrections or comments on the article found none. The PubPeer API and search page refused automated access (Cloudflare challenge and HTTP 403), so PubPeer could not be checked directly.
- Sources. PMC article PMC10426213; Europe PMC full-text XML for PMC10426213; publisher Figure 7 image and supplementary files from static-content.springer.com; Crossref; Europe PMC; trial registration NCT02333370 (cited in the paper).
Suggested correction
- Report the two-group Cox P value for PIK3CA mutated vs nonmutated among patients with detectable ctDNA, which the printed HR 0.93 (95% CI 0.52–1.69) implies is about 0.8, in the Results sentence and in the Abstract.
- Revise the Abstract's Results and Conclusions so that PIK3CA alterations are not presented as associated with inferior PFS on the basis of P = .016, for example by limiting the statistical association to TP53 and describing the PIK3CA difference in medians as not statistically significant.
- State in the Methods or the Figure 7 legend which test produced the P values printed on Figure 7A and 7B. If they are global tests across three groups including the ctDNA-undetectable patients, say so, and do not cite them as mutated vs wild-type comparisons.
- State the reference group of the hazard ratios (the HRs below 1 appear to compare nonmutated or low-bTMB patients with the others).
- Minor: reconcile the median PFS for patients with undetectable ctDNA (38.0 months in the Fig. 1C text, 37.8 months in the Fig. 7B text), or explain the difference.
Recomputation script (python recompute.py.txt)
"""Recompute the PFS P values of PMC10426213
(BMC Med 2023, doi:10.1186/s12916-023-03017-z, longitudinal ctDNA analysis of MONALEESASIA).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md,
writes fig1.svg/png and fig2.svg/png next to this file, and writes five candidate
figure designs to candidates/ (c1..c5, .svg and .png).
Inputs are copied from the paper as archived in PMC:
Results, "ctDNA fraction by BOR and PFS": ctDNA undetectable vs detectable, HR 0.45 (0.24-0.84), P = 0.01 (Fig. 1C)
Results, "Gene alterations at baseline": TP53 HR 0.26 (0.12-0.53), P < 0.001 (Fig. 7A)
PIK3CA HR 0.93 (0.52-1.69), P = 0.016 (Fig. 7B)
bTMB HR 0.83 (0.46-1.5), P = 0.53 (Fig. 7C)
Abstract: PIK3CA P = .016, TP53 P = .0001
Figure 7 (publisher image, numbers at risk at t = 0): A 12 mut / 42 nonmut / 28 not detected;
B 24 mut / 30 nonmut / 28 not detected; C 27 / 27
Method: for a Cox model, a 95% Wald CI is exp(b +/- 1.96 SE), so SE = (ln hi - ln lo) / (2 * 1.96),
z = ln(HR) / SE and the two-sided Wald P = 2 * (1 - Phi(|z|)).
"""
import math
import os
import numpy as np
from scipy import stats
import matplotlib
import matplotlib.ticker
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z975 = stats.norm.isf(0.025)
# ---------------------------------------------------------------- inputs (from the paper)
# key: (label, HR, lo, hi, reported P as a number for plotting, reported P as printed, location)
ROWS = [
("pik3ca", "PIK3CA mut vs nonmut", 0.93, 0.52, 1.69, 0.016, "P = 0.016 (Results); P = .016 (Abstract, Fig. 7B)", "Results; Abstract; Fig. 7B"),
("tp53", "TP53 mut vs nonmut", 0.26, 0.12, 0.53, 0.0001, "P < 0.001 (Results); P = .0001 (Abstract, Fig. 7A)", "Results; Abstract; Fig. 7A"),
("tmb", "bTMB high vs low", 0.83, 0.46, 1.50, 0.53, "P = 0.53 (Results); P = .53 (Fig. 7C)", "Results; Fig. 7C"),
("ctdna", "ctDNA undetectable vs detectable", 0.45, 0.24, 0.84, 0.01, "P = 0.01 (Results, Fig. 1C)", "Results; Fig. 1C"),
]
AT_RISK = {"Fig. 7A (TP53)": (12, 42, 28), "Fig. 7B (PIK3CA)": (24, 30, 28), "Fig. 7C (bTMB)": (27, 27)}
MEDIANS = {"PIK3CA mut": 12.7, "PIK3CA nonmut": 19.2, "TP53 mut": 7.3, "TP53 nonmut": 19.4,
"undetectable (Fig. 7B text)": 37.8, "undetectable (Fig. 1C text)": 38.0, "detectable (Fig. 1C text)": 19.2}
def wald(hr, lo, hi):
se = (math.log(hi) - math.log(lo)) / (2 * Z975)
z = math.log(hr) / se
return se, z, 2 * stats.norm.sf(abs(z))
R = {}
print("=== 1. Wald P back-calculated from each reported HR and 95% CI (two-sided) ===")
for key, lab, hr, lo, hi, prep, prep_txt, loc in ROWS:
se, z, p = wald(hr, lo, hi)
mid = math.exp((math.log(lo) + math.log(hi)) / 2)
R[key] = dict(lab=lab, hr=hr, lo=lo, hi=hi, se=se, z=z, p=p, prep=prep, prep_txt=prep_txt,
zrep=stats.norm.isf(prep / 2))
print(f" {lab:34s} HR {hr} ({lo}-{hi}): SE(lnHR) = {se:.3f}, z = {z:+.2f}, P = {p:.2g} ({p:.4f})"
f" reported: {prep_txt}")
print(f" {'':34s} CI log-midpoint = {mid:.3f} (HR printed {hr}); |z| implied by reported P = {R[key]['zrep']:.2f}")
pk = R["pik3ca"]
print("\n=== 2. What P = 0.016 would require for PIK3CA ===")
z016 = stats.norm.isf(0.016 / 2)
se_req = abs(math.log(pk["hr"])) / z016
lo_req, hi_req = math.exp(math.log(pk["hr"]) - Z975 * se_req), math.exp(math.log(pk["hr"]) + Z975 * se_req)
hr_lo_req, hr_hi_req = math.exp(-z016 * pk["se"]), math.exp(z016 * pk["se"])
R["req"] = dict(z016=z016, se_req=se_req, lo_req=lo_req, hi_req=hi_req, hr_lo_req=hr_lo_req, hr_hi_req=hr_hi_req)
print(f" |z| for two-sided P = 0.016: {z016:.2f}; reported CI gives |z| = {abs(pk['z']):.2f} "
f"({z016 / abs(pk['z']):.1f} times smaller)")
print(f" at HR 0.93, P = 0.016 needs SE(lnHR) = {se_req:.4f} (reported CI implies {pk['se']:.3f}, "
f"{pk['se'] / se_req:.1f} times larger), i.e. a 95% CI of {lo_req:.3f}-{hi_req:.3f}")
for h in (hr_lo_req, hr_hi_req):
print(f" with the reported CI width, P = 0.016 needs HR = {h:.3f}, CI "
f"{math.exp(math.log(h) - Z975 * pk['se']):.2f}-{math.exp(math.log(h) + Z975 * pk['se']):.2f}")
print(f" reported CI 0.52-1.69 contains HR = 1: {pk['lo'] < 1 < pk['hi']}")
print("\n=== 3. Sensitivity to rounding of the printed HR and CI limits (PIK3CA) ===")
ps = []
for hr in np.arange(0.925, 0.9351, 0.005):
for lo in np.arange(0.515, 0.5251, 0.005):
for hi in np.arange(1.685, 1.6951, 0.005):
ps.append(wald(hr, lo, hi)[2])
print(f" HR 0.925-0.935, lo 0.515-0.525, hi 1.685-1.695: Wald P ranges {min(ps):.3f}-{max(ps):.3f}")
print("\n=== 4. A possible source of the figure P values: a 3-group test (rough approximation) ===")
chi_det = R["ctdna"]["z"] ** 2
chi_pk = R["pik3ca"]["z"] ** 2
chi_tp = R["tp53"]["z"] ** 2
need016, need0001 = stats.chi2.isf(0.016, 2), stats.chi2.isf(0.0001, 2)
p3_pk = stats.chi2.sf(chi_det + chi_pk, 2)
p3_tp = stats.chi2.sf(chi_det + chi_tp, 2)
R["three"] = dict(chi_det=chi_det, chi_pk=chi_pk, chi_tp=chi_tp, p3_pk=p3_pk, p3_tp=p3_tp,
need016=need016, need0001=need0001)
print(f" Wald chi2 components: detectable vs undetectable {chi_det:.2f}; PIK3CA pairwise {chi_pk:.3f}; TP53 pairwise {chi_tp:.2f}")
print(f" 2-df chi2 needed for P = 0.016: {need016:.2f}; for P = 0.0001: {need0001:.2f}")
print(f" PIK3CA panel (mut / nonmut / not detected): approx chi2 {chi_det + chi_pk:.2f}, 2-df P ~ {p3_pk:.3f}"
f" (printed .016; pairwise Wald {R['pik3ca']['p']:.2f})")
print(f" TP53 panel (mut / nonmut / not detected): approx chi2 {chi_det + chi_tp:.2f}, 2-df P ~ {p3_tp:.1g}"
f" (printed .0001; pairwise Wald {R['tp53']['p']:.5f})")
print(" (Approximation only: adds a 1-df detectable-vs-undetectable component to the 1-df pairwise one.)")
print("\n=== 5. Numbers at risk at t = 0 in Figure 7 ===")
for k, v in AT_RISK.items():
print(f" {k}: {' + '.join(map(str, v))} = {sum(v)}")
print(f" detectable ctDNA in panels A/B: {24 + 30} = {12 + 42}; PIK3CA mut share of detectable {24 / 54:.1%}, of all 82 {24 / 82:.1%}")
print(f" TP53 mut share of detectable {12 / 54:.1%}, of all 82 {12 / 82:.1%}")
print("\n=== 6. Other small observations ===")
print(f" median PFS, undetectable ctDNA: {MEDIANS['undetectable (Fig. 1C text)']} months (Fig. 1C text) vs "
f"{MEDIANS['undetectable (Fig. 7B text)']} months (Fig. 7B text)")
print(" The TP53, PIK3CA and bTMB HRs are < 1 although the mutated / high-bTMB group has the shorter median,")
print(" so they appear to express the nonmutated / low group relative to the other; the Wald P is unaffected.")
print(f" e.g. 1/0.93 = {1 / 0.93:.2f} (1/1.69 = {1 / 1.69:.2f} to 1/0.52 = {1 / 0.52:.2f}); "
f"1/0.26 = {1 / 0.26:.2f} (1/0.53 = {1 / 0.53:.2f} to 1/0.12 = {1 / 0.12:.2f})")
# ---------------------------------------------------------------- figure style
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, NEUTRAL = "#eb6834", "#2a78d6", "#e4e3df", "#8a8984"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
"lines.linewidth": 1.5, "xtick.labelsize": 10, "ytick.labelsize": 10.5,
})
LW = 1.5 # 1.5 pt = 2 px at 96 dpi
MS = 9 # marker size in points (>= 8 px)
def save(fig, folder, stem):
fig.savefig(os.path.join(folder, stem + ".svg"))
fig.savefig(os.path.join(folder, stem + ".png"), dpi=200)
plt.close(fig)
def clean(ax, grid_axis="x"):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis=grid_axis, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
ORDER = ["pik3ca", "tp53", "tmb", "ctdna"]
YLAB = {"pik3ca": "PIK3CA mutated vs not\n(Fig. 7B)", "tp53": "TP53 mutated vs not\n(Fig. 7A)",
"tmb": "bTMB high vs low\n(Fig. 7C)", "ctdna": "ctDNA undetectable vs\ndetectable (Fig. 1C)"}
PREP_SHORT = {"pik3ca": "0.016", "tp53": ".0001", "tmb": "0.53", "ctdna": "0.01"}
def fmtp(p):
if p < 0.001:
return f"{p:.5f}".rstrip("0")
return f"{p:.2g}" if p < 0.1 else f"{p:.2f}"
# ---- C1: P-value ladder (reported P vs P recomputed from HR/CI, log axis) ----
REP_LABEL = {"pik3ca": "reported 0.016", "tp53": "reported .0001 (Abstract); < 0.001 (Results)",
"tmb": "reported 0.53", "ctdna": "reported 0.01"}
def draw_c1(folder, stem):
fig, ax = plt.subplots(figsize=(7.2, 4.2))
for i, k in enumerate(ORDER):
r = R[k]
y = len(ORDER) - 1 - i
ax.plot([r["prep"], r["p"]], [y, y], color=NEUTRAL, lw=LW, zorder=2)
ax.plot(r["prep"], y, "o", ms=MS, color=ORANGE, zorder=3)
ax.plot(r["p"], y, "o", ms=MS, color=BLUE, zorder=4, mfc=BLUE if k != "tmb" else "none", mew=2)
ax.annotate(REP_LABEL[k], (r["prep"], y), xytext=(-8 if k == "tp53" else 0, 8), textcoords="offset points",
ha={"tmb": "right", "tp53": "left"}.get(k, "center"), va="bottom", fontsize=10, color=TXT)
ax.annotate("recomputed " + fmtp(r["p"]), (r["p"], y), xytext=(0, -9), textcoords="offset points",
ha="center" if k != "tmb" else "right", va="top", fontsize=10, color=TXT)
ax.axvline(0.05, color=SEC, lw=0.8, ls=(0, (3, 2)), zorder=1)
ax.text(0.052, -0.85, "P = 0.05", ha="left", va="bottom", fontsize=10, color=SEC)
ax.set_xscale("log")
ax.set_xlim(2e-5, 2)
ax.set_ylim(-0.9, len(ORDER) - 0.45)
ax.set_yticks(range(len(ORDER)))
ax.set_yticklabels([YLAB[k] for k in reversed(ORDER)])
ax.set_xticks([1e-4, 1e-3, 1e-2, 0.1, 1])
ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.1", "1"])
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_xlabel("Two-sided P value (log scale)")
clean(ax)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="P as reported in the paper"),
Line2D([], [], marker="o", ls="", ms=MS, color=BLUE, label="Wald P from the paper's own HR and 95% CI")],
loc="upper center", bbox_to_anchor=(0.4, -0.17), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, folder, stem)
# ---- C2: the interval P = 0.016 would require (HR log axis) ----
def draw_c2(folder, stem):
q = R["req"]
se = pk["se"]
rows = [
("Reported\nHR 0.93 (0.52–1.69), P = 0.016", pk["hr"], pk["lo"], pk["hi"], ORANGE, "o"),
("95% CI that P = 0.016 would need\nat HR 0.93", pk["hr"], q["lo_req"], q["hi_req"], BLUE, "o"),
("HR that P = 0.016 would need\nwith the reported CI width", q["hr_lo_req"],
math.exp(math.log(q["hr_lo_req"]) - Z975 * se), math.exp(math.log(q["hr_lo_req"]) + Z975 * se), BLUE, "D"),
("", q["hr_hi_req"], math.exp(math.log(q["hr_hi_req"]) - Z975 * se),
math.exp(math.log(q["hr_hi_req"]) + Z975 * se), BLUE, "D"),
]
ys = [3, 2, 1, 0.55]
fig, ax = plt.subplots(figsize=(7.2, 3.8))
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
for (lab, hr, lo, hi, col, mk), y in zip(rows, ys):
ax.plot([lo, hi], [y, y], color=col, lw=LW * 1.4, solid_capstyle="butt", zorder=2)
ax.plot(hr, y, mk, ms=MS, color=col, zorder=3)
ax.annotate(f"{hr:.2f} ({lo:.2f}–{hi:.2f})", (hi, y), xytext=(6, 0), textcoords="offset points",
va="center", fontsize=10, color=TXT)
ax.text(pk["hr"], 2.32, f"Wald P from this interval: {pk['p']:.2f}", ha="center", va="bottom", fontsize=10, color=SEC)
ax.text(1.03, 0.2, "HR = 1 (no difference)", ha="left", va="bottom", fontsize=10, color=SEC)
ax.set_xscale("log")
ax.set_xlim(0.2, 10)
ax.set_ylim(0.15, 3.45)
ax.set_yticks([3, 2, 0.775])
ax.set_yticklabels([rows[0][0], rows[1][0], rows[2][0]], fontsize=10)
ax.set_xticks([0.25, 0.5, 1, 2, 4])
ax.set_xticklabels(["0.25", "0.5", "1", "2", "4"])
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_xlabel("Hazard ratio for PFS, PIK3CA comparison, with 95% CI (log scale)")
clean(ax)
ax.legend(handles=[Line2D([], [], marker="o", color=ORANGE, lw=LW, ms=MS, label="reported in the paper"),
Line2D([], [], marker="o", color=BLUE, lw=LW, ms=MS, label="what P = 0.016 would require")],
loc="upper center", bbox_to_anchor=(0.35, -0.2), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, folder, stem)
# ---- C3: z-score comparison (linear |z| axis, 1.96 reference) ----
def draw_c3(folder, stem):
fig, ax = plt.subplots(figsize=(7.2, 3.4))
for i, k in enumerate(ORDER):
r = R[k]
y = len(ORDER) - 1 - i
a, b = r["zrep"], abs(r["z"])
ax.plot([a, b], [y, y], color=NEUTRAL, lw=LW, zorder=2)
ax.plot(a, y, "o", ms=MS, color=ORANGE, zorder=3)
ax.plot(b, y, "o", ms=MS, color=BLUE, zorder=3)
ax.annotate(f"{a:.2f}", (a, y), xytext=(0, 9), textcoords="offset points", ha="center", fontsize=10, color=TXT)
ax.annotate(f"{b:.2f}", (b, y), xytext=(0, -17), textcoords="offset points", ha="center", fontsize=10, color=TXT)
ax.axvline(Z975, color=SEC, lw=0.8, ls=(0, (3, 2)))
ax.text(Z975, len(ORDER) - 0.45, "|z| = 1.96 (P = 0.05)", ha="center", va="bottom", fontsize=10, color=SEC)
ax.set_xlim(0, 4.4)
ax.set_ylim(-0.6, len(ORDER) - 0.2)
ax.set_yticks(range(len(ORDER)))
ax.set_yticklabels([YLAB[k] for k in reversed(ORDER)])
ax.set_xlabel("|z| (standard errors from no difference)")
clean(ax)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="|z| implied by the reported P"),
Line2D([], [], marker="o", ls="", ms=MS, color=BLUE, label="|z| from the reported HR and 95% CI")],
loc="upper center", bbox_to_anchor=(0.45, -0.22), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, folder, stem)
# ---- C4: forest plot with reported / recomputed P columns ----
def draw_c4(folder, stem):
fig = plt.figure(figsize=(7.2, 3.3))
ax = fig.add_axes([0.27, 0.19, 0.4, 0.62])
ax.set_xscale("log")
ax.set_xlim(0.1, 2.2)
ax.set_ylim(-0.6, len(ORDER) - 0.4)
for i, k in enumerate(ORDER):
r = R[k]
y = len(ORDER) - 1 - i
ax.plot([r["lo"], r["hi"]], [y, y], color=TXT, lw=LW, zorder=2)
ax.plot(r["hr"], y, "s", ms=MS - 1, color=TXT, zorder=3)
fig.text(0.70, ax.transData.transform((0, y))[1] / fig.dpi / 3.3, PREP_SHORT[k], color=ORANGE,
fontsize=10.5, va="center", ha="left", fontweight="bold" if k == "pik3ca" else "normal")
fig.text(0.83, ax.transData.transform((0, y))[1] / fig.dpi / 3.3, fmtp(r["p"]), color=BLUE,
fontsize=10.5, va="center", ha="left", fontweight="bold" if k == "pik3ca" else "normal")
ax.axvline(1, color=SEC, lw=0.8)
ax.set_yticks(range(len(ORDER)))
ax.set_yticklabels([YLAB[k] for k in reversed(ORDER)], fontsize=10)
ax.set_xticks([0.125, 0.25, 0.5, 1, 2])
ax.set_xticklabels(["0.125", "0.25", "0.5", "1", "2"])
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_xlabel("HR and 95% CI as printed (log scale)")
clean(ax)
fig.text(0.70, 0.84, "reported P", color=ORANGE, fontsize=10.5, ha="left", va="bottom")
fig.text(0.83, 0.84, "recomputed P\n(Wald, HR/CI)", color=BLUE, fontsize=10.5, ha="left", va="bottom")
save(fig, folder, stem)
# ---- C5: two text panels with an annotated connector ----
def draw_c5(folder, stem):
fig = plt.figure(figsize=(7.2, 4.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 10)
ax.set_ylim(0, 6.1)
ax.axis("off")
def panel(x, y, w, h, title, body, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.12",
fc="#ffffff", ec=edge, lw=LW))
ax.text(x + 0.2, y + h - 0.22, title, fontsize=10.5, fontweight="bold", va="top", color=TXT)
ax.text(x + 0.2, y + h - 0.68, body, fontsize=10, va="top", color=TXT, linespacing=1.4)
panel(0.2, 2.85, 4.55, 2.75, "Abstract (Results; Conclusions)",
'"Patients with alterations in PIK3CA\nand TP53 at baseline had shorter\nmedian progression-free survival\n'
'... (P = .016 and P = .0001,\nrespectively)."\n'
'"... PIK3CA and TP53 alterations ...\nwere associated with inferior outcomes."', ORANGE)
panel(5.25, 2.85, 4.55, 2.75, "Results, Gene alterations at baseline",
'"Median PFS of patients with PIK3CA\nmutations was 12.7 months vs 19.2\nmonths ...; however, the difference\n'
'was not statistically significant\n(HR, 0.93; 95% CI, 0.52–1.69;\nP = 0.016; Fig. 7B)."', ORANGE)
panel(2.55, 0.2, 4.9, 1.75, "Recomputed from the HR and 95% CI",
f"SE(ln HR) = {pk['se']:.3f}, z = {pk['z']:.2f}, Wald P = {pk['p']:.2f}\n"
f"P = 0.016 would need |z| = {R['req']['z016']:.2f}, i.e. a\n"
f"95% CI of {R['req']['lo_req']:.2f}–{R['req']['hi_req']:.2f} at HR 0.93", BLUE)
ax.add_patch(FancyArrowPatch((4.78, 4.25), (5.22, 4.25), arrowstyle="<->", mutation_scale=14, color=SEC, lw=LW))
ax.text(5.0, 5.85, "Same comparison and the same P value: cited as an association in the Abstract,"
"\ncalled not statistically significant in the Results",
ha="center", va="center", fontsize=10, color=SEC)
ax.add_patch(FancyArrowPatch((7.5, 2.85), (6.5, 1.98), arrowstyle="->", mutation_scale=14, color=SEC, lw=LW))
ax.text(7.25, 2.4, "CI 0.52–1.69 includes 1", fontsize=10, color=SEC, va="center", ha="left")
ax.add_patch(FancyArrowPatch((2.5, 2.85), (3.5, 1.98), arrowstyle="->", mutation_scale=14, color=SEC, lw=LW))
ax.text(2.75, 2.4, "P = .016 is not\nreproduced", fontsize=10, color=SEC, va="center", ha="right")
save(fig, folder, stem)
DRAW = {"c1_p_ladder": draw_c1, "c2_required_interval": draw_c2, "c3_z_scores": draw_c3,
"c4_forest_with_p_columns": draw_c4, "c5_text_panels": draw_c5}
for stem, fn in DRAW.items():
fn(CAND, stem)
# defaults used in writeup.md
draw_c1(HERE, "fig1")
draw_c2(HERE, "fig2")
print("\nFigures written: fig1 (= candidate c1), fig2 (= candidate c2), candidates/c1..c5")
Abstract reports all pocket-depth measures significant for CVD mortality; Table 2 shows none are
BMC Med (2023) · doi:10.1186/s12916-023-03139-4 · PMC10642059
Summary
This cohort study (SHIP-START, n = 3047, 125 CVD deaths) relates periodontal and inflammatory markers to mortality. Its abstract states that "For CVD mortality, all PPD related variables showed significant associations". The paper's own Table 2 shows the opposite: for CVD mortality, none of the three pocket probing depth (PPD) measures is significant. All six confidence intervals (per unit and per SD) include 1, the printed P values are 0.275, 0.302 and 0.337, and the paper's Results text says the same thing. The numbers in the table are internally consistent: Wald P values recomputed from the printed intervals are 0.274, 0.303 and 0.336. The abstract sentence is the only single-exposure CVD result it reports. Read alone, it reverses the finding for the periodontal marker that the paper describes as best reflecting current periodontal inflammation. The only published correction (January 2025) concerns funding and leaves the sentence unchanged.
What the paper reports
Abstract, Results:
For CVD mortality, all PPD related variables showed significant associations.
Results, "Inflammation and periodontitis as risk factors for mortality":
However, compared with the results for CAL and missing teeth, the PPD measures showed only weak associations with mortality, as indicated by smaller effect sizes. For CVD mortality, neither mean PPD nor extent PPD ≥ 3 mm/4 mm showed a relevant association (Table 2). Classical Cox proportional hazards models with fixed baseline exposures and covariates (Additional file 1: Table S4) confirmed these results.
Table 2, PPD rows, CVD-mortality columns. The columns are HR (95% CI) per 1-unit increase | HR (95% CI) per SD increase | P-value:
Mean PPD, mm | 1.111 (0.920; 1.341) | 1.078 (0.942; 1.233) | 0.275
Sites with PPD ≥ 3 mm, % | 1.004 (0.997; 1.011) | 1.096 (0.921; 1.305) | 0.302
Sites with PPD ≥ 4 mm, % | 1.004 (0.996; 1.012) | 1.070 (0.932; 1.228) | 0.337
Methods, "Statistical analyses", on the significance threshold:
Regarding the interaction terms, two-sided P values < 0.10 were considered statistically significant. Otherwise, 0.05 was set as the cut-off for significance.
A related sentence in the Abstract, Results, which comes just before the CVD sentence:
All markers of periodontitis and inflammation showed apparent associations with all-cause mortality (HRs per SD-increase: mean PPD: 1.068 (95% confidence interval (CI): 0.988–1.155) […]
The Abstract, Conclusions, and the main-text Conclusions:
Both periodontitis and inflammation were significantly associated with all-cause mortality and CVD mortality.
Periodontitis and systemic inflammation were both significantly associated with all considered types of mortality.
Discussion, on how PPD is interpreted:
measures of PPD, which best reflects current periodontal inflammation
What the paper's own numbers show
Method. Table 2 prints, for each exposure and outcome, an HR and 95% CI per unit, an HR and 95% CI per SD, and a single P value. I assumed each interval is a Wald interval, symmetric on the log-HR scale, as Cox model output is. On that basis I recomputed P from each printed interval: SE = (ln U − ln L)/(2 × 1.96), z = ln HR / SE, P = 2[1 − Φ(|z|)]. I also worked the other way and rebuilt each per-SD interval from the HR and the printed P. To confirm that the table's rows and columns are aligned, I ran the same check on the other CVD rows and on the PPD rows for the other two outcomes. I repeated it for Table S4 in Additional file 1, the time-invariant sensitivity models. recompute.py.txt prints every number below.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Mean PPD, CVD mortality, per SD | Table 2 | 1.078 (0.942; 1.233), P = 0.275 | CI includes 1; P = 0.274; CI rebuilt from P: 0.942–1.234 |
| Mean PPD, CVD mortality, per 1 mm | Table 2 | 1.111 (0.920; 1.341) | CI includes 1; P = 0.273 |
| Sites PPD ≥ 3 mm, CVD mortality, per SD | Table 2 | 1.096 (0.921; 1.305), P = 0.302 | CI includes 1; P = 0.303; CI rebuilt from P: 0.921–1.304 |
| Sites PPD ≥ 3 mm, CVD mortality, per 1 % | Table 2 | 1.004 (0.997; 1.011) | CI includes 1; P = 0.262 (anywhere from 0.174 to 0.359 within 3-decimal rounding of the HR and limits) |
| Sites PPD ≥ 4 mm, CVD mortality, per SD | Table 2 | 1.070 (0.932; 1.228), P = 0.337 | CI includes 1; P = 0.336; CI rebuilt from P: 0.932–1.228 |
| Sites PPD ≥ 4 mm, CVD mortality, per 1 % | Table 2 | 1.004 (0.996; 1.012) | CI includes 1; P = 0.326 (0.239–0.419 within rounding) |
| PPD measures significant for CVD mortality (P < 0.05) | Abstract vs Table 2 | Abstract: "all" (3 of 3) | 0 of 3 printed P; 0 of 6 recomputed P (both scalings); 0 of 6 CIs exclude 1 |
| One-sided P (printed P halved), the three PPD measures | derived | — | 0.138, 0.151, 0.169, all still > 0.05 |
| Control: mean CAL, CVD, per SD | Table 2 | 1.225 (1.042; 1.439), P = 0.014 | P = 0.014 |
| Control: missing teeth, CVD, per SD | Table 2 | 1.226 (1.025; 1.466), P = 0.026 | P = 0.026 |
| Control: periodontitis score, CVD, per SD | Table 2 | 1.399 (1.076; 1.820), P = 0.012 | P = 0.012 |
| Control: fibrinogen, CVD, per SD | Table 2 | 1.128 (0.981; 1.297), P = 0.090 | P = 0.091 |
| Control: sites CAL ≥ 3 mm, CVD, per SD | Table 2 | 1.288 (0.976; 1.702), P = 0.074 | P = 0.074 |
| Control: sites CAL ≥ 4 mm, CVD, per SD | Table 2 | 1.217 (1.002; 1.479), P = 0.047 | P = 0.048 |
| PPD measures, time-invariant models, CVD, per SD | Table S4 | 1.080 (0.927; 1.259) P = 0.321; 1.057 (0.883; 1.264) P = 0.547; 1.057 (0.917; 1.219) P = 0.444 | P = 0.324, 0.545, 0.445; 0 of 3 significant |
| PPD measures significant, other outcomes | Table 2 | all-cause P = 0.098, 0.025, 0.030; non-CVD P = 0.166, 0.030, 0.037 | 2 of 3 for each outcome; mean PPD not significant for either |
| PPD measures significant, other outcomes | Table S4 | all-cause P = 0.020, 0.031, 0.042; non-CVD P = 0.023, 0.018, 0.035 | 3 of 3 for each outcome |
| Mean PPD, all-cause mortality, per SD (abstract: "apparent association") | Abstract; Table 2 | 1.068 (0.988–1.155), P = 0.098 | CI includes 1; P = 0.099 |
Every recomputed P agrees with the printed P to within rounding, both for the PPD rows and for the control rows. The only exceptions are the per-unit rows for extent PPD, and those have little precision because the HR and its limits are printed to three decimals close to 1. The table is therefore internally consistent. What disagrees with it is the abstract sentence.
Figure
Figure 1. The left panel shows the abstract sentence on CVD mortality. The right panel shows the Results sentence and the three Table 2 rows it summarises, with the per-SD hazard ratio (95% CI), the P value printed in the paper (orange circle) and the P value recomputed from the printed interval (blue square). The connector names the conflict. The abstract implies 3 of 3 PPD measures are significant; the table has 0 of 3. All numbers are printed in the paper or by recompute.py.txt. The paper's own figures are not reproduced.
Figure 2. Hazard ratios per SD increase for CVD mortality from Table 2, on a log axis with a reference line at HR = 1. For each row, the orange interval is the 95% CI printed in the paper and the blue interval is the 95% CI rebuilt from the HR and the printed P. The printed P values are listed on the right. The two intervals coincide for every row, so the HR, CI and P columns are mutually consistent. All three PPD intervals cross 1. The three control rows from the same table (mean CAL, missing teeth, periodontitis score) do not, which shows that significant CVD associations in this table are reported as such.
Why it matters
Most readers see only the abstract, and in it this sentence is the only result given for a single exposure and CVD mortality. It tells them that every pocket-depth measure is significantly associated with cardiovascular death, but the paper's analysis finds no PPD measure significantly associated with CVD mortality as a main effect, in either the main models (Table 2) or the sensitivity models (Table S4). The difference matters for interpretation. The paper describes PPD as the measure that "best reflects current periodontal inflammation". In this paper, the CVD association is carried by markers of cumulative periodontal damage instead: mean CAL, extent CAL ≥ 4 mm, missing teeth and the periodontitis score. Someone relying on the abstract might cite this study as evidence that current periodontal inflammation, measured as pocket depth, predicts cardiovascular death, but the paper's main-effect analyses do not support that.
The error does not overturn the paper's other main results. The periodontitis score, mean CAL, missing teeth, leukocytes, CRP and the inflammation score are all significant for CVD mortality in Table 2. The additive-interaction (RERI) results are separate analyses and are not affected. Two related statements are broader than the table supports, though. First, the abstract lists mean PPD (P = 0.098) among markers with "apparent associations" with all-cause mortality. Second, both Conclusions say periodontitis and inflammation were significantly associated with CVD mortality, and that is not true for any PPD measure, for extent CAL ≥ 3 mm (P = 0.074) or for fibrinogen (P = 0.090).
Explanations considered
- Table 2 is wrong and the abstract is right. This does not fit. For every PPD row and every control row, the P values recomputed from the printed intervals match the printed P values. The Results text describes the same null result in words, and Table S4 shows it again in separately fitted time-invariant models on the same cohort. Three parts of the paper agree with one another and disagree with the abstract.
- Per-unit versus per-SD scaling. This is not the explanation. All six CVD intervals include 1, under both scalings.
- A different model (time-invariant exposures, competing risks). This is not the explanation. The Results say Table S4 "confirmed these results", and in Table S4 the CVD P values are 0.321, 0.547 and 0.444. The Methods mention competing-risk models (subdistribution hazards) for CVD mortality, but neither the paper nor Additional file 1 reports their results. No reported model makes PPD significant for CVD mortality as a main effect.
- One-sided testing or the 0.10 threshold. This is not the explanation. The Methods set 0.05 as the cut-off for main effects (and describe P values as two-sided for interaction terms). If the printed main-effect P values are two-sided, halving them for a one-sided test gives 0.138–0.169, still above 0.05. The 0.10 threshold applies only to interaction terms, and the main-effect P values (0.275–0.337) are above it anyway.
- The sentence refers to interaction terms. Under this reading the sentence could be defended. Table S5 in Additional file 1 flags, by symbol, multiplicative interactions significant at P < 0.10 for CVD mortality (the paper's threshold for interaction terms). At least one is flagged for every PPD measure: mean PPD with fibrinogen, CRP and the inflammation score; sites PPD ≥ 3 mm with fibrinogen; and sites PPD ≥ 4 mm with fibrinogen, CRP and the inflammation score. The placement and wording point to a main-effect reading, though. The sentence follows directly after the list of main-effect HRs per SD increase for all-cause mortality. It speaks of "PPD related variables" having "associations" and names no interaction or inflammation marker. The next sentence, "Interaction modelling revealed some variation with respect to mortality type and exposure combinations", is where the abstract turns to interactions. As printed, the sentence reads as a main-effect claim, which Table 2 and Table S4 contradict. At best it is ambiguous. The suggested correction below removes the ambiguity under either reading.
- The sentence describes another outcome column. I checked this. In Table 2, two of the three PPD measures are significant for all-cause mortality and two of three for non-CVD mortality (mean PPD is not, P = 0.098 and 0.166). In Table S4, all three are significant for all-cause and for non-CVD mortality. So "all PPD related variables showed significant associations" matches the non-CVD and all-cause columns of Table S4, not any column of the main Table 2. The paper does not say which outcome or model the sentence was meant to describe. As printed, it names CVD mortality, and for CVD mortality it is contradicted by both tables.
- Conversion or typesetting artefact. This is ruled out. The publisher's HTML abstract and Table 2 page, the PMC text and the Europe PMC full-text XML all carry the same sentence and the same cell values.
Status
As of 2026-10-23:
- Version of record. The publisher's article page (link.springer.com, which doi.org and bmcmedicine.biomedcentral.com both redirect to) sits behind a JavaScript challenge, so it was loaded in headless Chromium. It shows the same abstract sentence and the same Results sentence. Its Table 2 page has CVD-mortality cells identical to the PMC copy. The Europe PMC full-text XML and the PMC web page agree. The publisher PDF could not be retrieved (both the Springer and the Europe PMC PDF links returned a challenge page). Every quotation above was checked verbatim against the PMC-archived text.
- Supplement. The single supplementary file, Additional file 1 (12916_2023_3139_MOESM1_ESM.docx), was downloaded from the publisher's static-content server and its Table S4 read. Table S4 agrees with Table 2 that no PPD measure is significant for CVD mortality as a main effect (values in the table above), so it does not resolve the discrepancy. Its Table S5 was also read. It prints RERI values for CVD mortality and uses symbols to flag PPD × inflammation multiplicative interactions for CVD mortality as significant at P < 0.10, for each of the three PPD measures with at least one inflammation marker (see "Explanations considered"). The main text, Results, "Interaction between periodontitis and systemic inflammation", gives two such interactions for CVD mortality, "extent PPD and fibrinogen: HR = 1.147, P = 0.006; extent PPD and inflammation score: HR = 1.134, P = 0.042", and says that these HRs "ranged from 1.12 to 1.18". It does not say whether "extent PPD" there means ≥ 3 mm or ≥ 4 mm. These interaction results are the only PPD-related CVD results that the paper reports as significant.
- Corrections. One correction exists: doi:10.1186/s12916-025-03862-0 (BMC Med 2025, PMC11734466, dated 15 January 2025), found through Crossref update metadata and Europe PMC notices. Its full text adds a funding source that was "mistakenly omitted" and says nothing about the abstract or Table 2. The publisher's page still carries the abstract sentence, and the correction notice is shown on it. No retraction or expression of concern was found.
- PubPeer. The PubPeer search page and API refused automated access (a Cloudflare challenge, HTTP 403), including from headless Chromium. A web search for the title together with "PubPeer" turned up no PubPeer thread, only the article, the correction and repository copies. That there is no PubPeer thread is likely but not confirmed.
- Sources. PMC10642059 (article), PMC11734466 (correction), https://link.springer.com/article/10.1186/s12916-023-03139-4 (version of record), Additional file 1 (12916_2023_3139_MOESM1_ESM.docx).
Suggested correction
In the Abstract, Results, replace "For CVD mortality, all PPD related variables showed significant associations." with a sentence consistent with Table 2 and the Results text. For example: "For CVD mortality, none of the PPD-related variables showed a significant association (HRs per SD increase: mean PPD 1.078 (95% CI 0.942–1.233); extent PPD ≥ 3 mm 1.096 (0.921–1.305); extent PPD ≥ 4 mm 1.070 (0.932–1.228)), whereas mean CAL, extent CAL ≥ 4 mm, missing teeth, the periodontitis score and the inflammation markers other than fibrinogen did." The authors may also wish to review three related statements. First, the description of mean PPD (P = 0.098) as showing an "apparent association" with all-cause mortality. Second and third, the two Conclusions that periodontitis was significantly associated with CVD mortality, which hold for mean CAL, extent CAL ≥ 4 mm, missing teeth and the periodontitis score but not for the PPD measures (or for extent CAL ≥ 3 mm, P = 0.074).
Recomputation script (python recompute.py.txt)
"""Recompute the CVD-mortality results for the pocket probing depth (PPD) measures
in PMC10642059 (BMC Med 2023;21:430, doi:10.1186/s12916-023-03139-4).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes, next to this file:
fig1.svg/png (= candidates/cand_A) abstract vs Results/Table 2 text panels
fig2.svg/png (= candidates/cand_B) reported vs recomputed 95% CIs, log HR axis
candidates/cand_A..cand_E.svg/png five candidate figure designs
Inputs are copied from the paper as archived in PMC (Table 2) and from
Additional file 1 (12916_2023_3139_MOESM1_ESM.docx, Table S4).
Method. Each 95% CI is assumed symmetric on the log-HR scale (Wald), so
SE = (ln U - ln L) / (2 * 1.959964), z = ln(HR) / SE, P = 2 * (1 - Phi(|z|)).
In reverse, a CI is rebuilt from the HR and the reported two-sided P:
SE = |ln HR| / Phi^-1(1 - P/2), CI = HR * exp(+/- 1.959964 * SE).
"""
import os
import shutil
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z975 = norm.ppf(0.975)
# ------------------------------------------------------------------ inputs (paper)
# Table 2: (per-unit HR, lo, hi), (per-SD HR, lo, hi), reported P -- one P per row
T2 = {
"all-cause": {
"Mean PPD": ((1.097, 0.983, 1.224), (1.068, 0.988, 1.155), 0.098),
"PPD >= 3 mm": ((1.005, 1.000, 1.009), (1.125, 1.015, 1.247), 0.025),
"PPD >= 4 mm": ((1.005, 1.000, 1.010), (1.091, 1.008, 1.181), 0.030),
},
"CVD": {
"Mean PPD": ((1.111, 0.920, 1.341), (1.078, 0.942, 1.233), 0.275),
"PPD >= 3 mm": ((1.004, 0.997, 1.011), (1.096, 0.921, 1.305), 0.302),
"PPD >= 4 mm": ((1.004, 0.996, 1.012), (1.070, 0.932, 1.228), 0.337),
},
"non-CVD": {
"Mean PPD": ((1.100, 0.961, 1.258), (1.070, 0.972, 1.178), 0.166),
"PPD >= 3 mm": ((1.006, 1.001, 1.011), (1.151, 1.014, 1.308), 0.030),
"PPD >= 4 mm": ((1.006, 1.000, 1.012), (1.108, 1.006, 1.220), 0.037),
},
}
# Table 2, CVD mortality, other exposures (per-SD HR, lo, hi, reported P) -- controls
T2_CVD_OTHER = {
"Mean CAL": (1.225, 1.042, 1.439, 0.014),
"CAL >= 3 mm": (1.288, 0.976, 1.702, 0.074),
"CAL >= 4 mm": (1.217, 1.002, 1.479, 0.047),
"Missing teeth": (1.226, 1.025, 1.466, 0.026),
"Periodontitis score": (1.399, 1.076, 1.820, 0.012),
"Leukocytes": (1.316, 1.143, 1.514, None), # printed "< 0.001"
"Fibrinogen": (1.128, 0.981, 1.297, 0.090),
"CRP (log)": (1.236, 1.028, 1.488, 0.024),
"Inflammation score": (1.385, 1.141, 1.681, 0.001),
}
# Additional file 1, Table S4 (time-invariant models), per-SD HR, lo, hi, reported P
S4 = {
"all-cause": {
"Mean PPD": (1.109, 1.017, 1.209, 0.020),
"PPD >= 3 mm": (1.127, 1.011, 1.255, 0.031),
"PPD >= 4 mm": (1.091, 1.003, 1.186, 0.042),
},
"CVD": {
"Mean PPD": (1.080, 0.927, 1.259, 0.321),
"PPD >= 3 mm": (1.057, 0.883, 1.264, 0.547),
"PPD >= 4 mm": (1.057, 0.917, 1.219, 0.444),
},
"non-CVD": {
"Mean PPD": (1.128, 1.017, 1.253, 0.023),
"PPD >= 3 mm": (1.176, 1.028, 1.345, 0.018),
"PPD >= 4 mm": (1.116, 1.008, 1.237, 0.035),
},
}
PPD = ["Mean PPD", "PPD >= 3 mm", "PPD >= 4 mm"]
OUTCOMES = ["all-cause", "CVD", "non-CVD"]
def p_from_ci(hr, lo, hi):
se = (np.log(hi) - np.log(lo)) / (2 * Z975)
z = np.log(hr) / se
return 2 * norm.sf(abs(z)), se, z
def ci_from_p(hr, p):
se = abs(np.log(hr)) / norm.isf(p / 2)
return hr * np.exp(-Z975 * se), hr * np.exp(Z975 * se)
# ------------------------------------------------------------------ 1. CVD PPD rows
print("1. Table 2, CVD mortality, PPD measures (Wald P recomputed from the printed CI)")
print(f" {'measure':12s} {'scale':8s} {'HR (95% CI)':22s} {'SE(lnHR)':>8s} {'z':>6s} "
f"{'P recomp':>8s} {'P printed':>9s} CI includes 1")
REC = {}
for m in PPD:
unit, sd, prep = T2["CVD"][m]
for scale, (hr, lo, hi) in (("per unit", unit), ("per SD", sd)):
p, se, z = p_from_ci(hr, lo, hi)
REC[(m, scale)] = p
print(f" {m:12s} {scale:8s} {hr:.3f} ({lo:.3f}; {hi:.3f}) {se:8.4f} {z:6.2f} "
f"{p:8.3f} {prep:9.3f} {lo <= 1 <= hi}")
n_sig_sd = sum(T2["CVD"][m][2] < 0.05 for m in PPD)
n_sig_rec = sum(REC[(m, s)] < 0.05 for m in PPD for s in ("per unit", "per SD"))
n_ci_excl = sum(not (c[1] <= 1 <= c[2]) for m in PPD for c in T2["CVD"][m][:2])
print(f" PPD measures with printed P < 0.05 for CVD mortality: {n_sig_sd} of 3")
print(f" recomputed P < 0.05: {n_sig_rec} of 6 (both scalings); CIs excluding 1: {n_ci_excl} of 6")
print(" per-SD P range printed: %.3f-%.3f; recomputed: %.3f-%.3f" % (
min(T2["CVD"][m][2] for m in PPD), max(T2["CVD"][m][2] for m in PPD),
min(REC[(m, "per SD")] for m in PPD), max(REC[(m, "per SD")] for m in PPD)))
print(" one-sided P (printed P halved): " +
", ".join(f"{m} {T2['CVD'][m][2] / 2:.3f}" for m in PPD))
# per-unit rows with CI printed to 3 decimals near 1: how much can rounding move P?
for m in ("PPD >= 3 mm", "PPD >= 4 mm"):
hr, lo, hi = T2["CVD"][m][0]
ps = [p_from_ci(h, l, u)[0] for h in (hr - 5e-4, hr + 5e-4)
for l in (lo - 5e-4, lo + 5e-4) for u in (hi - 5e-4, hi + 5e-4)]
print(f" {m} per unit: P compatible with 3-decimal rounding of HR/CI: "
f"{min(ps):.3f}-{max(ps):.3f} (printed {T2['CVD'][m][2]:.3f})")
# ------------------------------------------------------------------ 2. CI rebuilt from P
print("\n2. 95% CI rebuilt from the per-SD HR and the printed P (CVD mortality)")
REBUILT = {}
for m in PPD:
hr, lo, hi = T2["CVD"][m][1]
rl, rh = ci_from_p(hr, T2["CVD"][m][2])
REBUILT[m] = (rl, rh)
print(f" {m:12s} printed ({lo:.3f}; {hi:.3f}) rebuilt ({rl:.3f}; {rh:.3f})")
# ------------------------------------------------------------------ 3. control rows
print("\n3. Control rows, Table 2 CVD mortality, per SD (table internal consistency)")
for k, (hr, lo, hi, p) in T2_CVD_OTHER.items():
pc = p_from_ci(hr, lo, hi)[0]
ptxt = "< 0.001" if p is None else f"{p:.3f}"
pctxt = "< 0.001" if pc < 0.001 else f"{pc:.3f}"
print(f" {k:20s} {hr:.3f} ({lo:.3f}; {hi:.3f}) P recomp {pctxt} printed {ptxt} "
f"{'sig' if (p is None or p < 0.05) else 'n.s.'}")
# ------------------------------------------------------------------ 4. other outcomes
print("\n4. Same PPD rows for the other outcomes (Table 2 per SD and Table S4 per SD)")
SIGCOUNT = {}
for src, data in (("Table 2", {o: {m: (*T2[o][m][1], T2[o][m][2]) for m in PPD} for o in OUTCOMES}),
("Table S4", S4)):
for o in OUTCOMES:
cells = []
n = 0
for m in PPD:
hr, lo, hi, p = data[o][m]
pc = p_from_ci(hr, lo, hi)[0]
n += p < 0.05
cells.append(f"{m} P {p:.3f} (recomp {pc:.3f})")
SIGCOUNT[(src, o)] = n
print(f" {src:8s} {o:9s} {n} of 3 significant: " + "; ".join(cells))
print(" all-cause mean PPD, Table 2 per SD: 1.068 (0.988; 1.155), P recomp "
f"{p_from_ci(1.068, 0.988, 1.155)[0]:.3f} (printed 0.098)")
# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "axes.edgecolor": INK2,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"text.color": INK, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
LW = 2 * 72 / 100 # 2 px at 100 px/in in points
MS = 8 # marker size in points (>= 8 px)
def save(fig, name, also=None):
for ext in ("svg", "png"):
p = os.path.join(CAND, f"{name}.{ext}")
fig.savefig(p, dpi=200, bbox_inches="tight", pad_inches=0.12)
if also:
shutil.copyfile(p, os.path.join(HERE, f"{also}.{ext}"))
plt.close(fig)
def box(ax, x, y, w, h, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc="white", ec=edge, lw=1.2, transform=ax.transAxes))
LBL = {"Mean PPD": "Mean PPD", "PPD >= 3 mm": "Sites PPD ≥ 3 mm",
"PPD >= 4 mm": "Sites PPD ≥ 4 mm"}
# ---- A: two text panels joined by an annotated connector -----------------------
fig = plt.figure(figsize=(7.2, 4.4))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1)
box(ax, 0.01, 0.36, 0.33, 0.60, INK2)
box(ax, 0.47, 0.02, 0.52, 0.94, INK2)
ax.text(0.03, 0.92, "Abstract, Results", fontsize=11, weight="bold", va="top")
ax.text(0.03, 0.83, "“For CVD mortality, all\nPPD related variables\nshowed significant\nassociations.”",
fontsize=10.5, va="top", style="italic", linespacing=1.4)
ax.text(0.03, 0.52, "Implies: 3 of 3 PPD\nmeasures significant\nfor CVD mortality",
fontsize=10, va="top", color=INK2, linespacing=1.35)
ax.text(0.49, 0.92, "Results text and Table 2", fontsize=11, weight="bold", va="top")
ax.text(0.49, 0.85, "“For CVD mortality, neither mean PPD nor\nextent PPD ≥ 3 mm/4 mm showed a relevant\n"
"association (Table 2).”", fontsize=10, va="top", style="italic", linespacing=1.4)
yh = 0.585
ax.text(0.49, yh, "CVD mortality,\nHR per SD (95% CI)", fontsize=10, color=INK2, va="center", linespacing=1.2)
ax.plot([0.775], [yh], "o", ms=MS, color=ORANGE)
ax.text(0.79, yh, "P\nprinted", fontsize=10, color=INK2, va="center", linespacing=1.1)
ax.plot([0.885], [yh], "s", ms=MS, color=BLUE)
ax.text(0.90, yh, "P\nrecomp.", fontsize=10, color=INK2, va="center", linespacing=1.1)
ax.plot([0.49, 0.975], [yh - 0.055, yh - 0.055], color=GRID, lw=1)
for i, m in enumerate(PPD):
y = yh - 0.13 - i * 0.115
hr, lo, hi = T2["CVD"][m][1]
ax.text(0.49, y + 0.025, LBL[m], fontsize=10, va="center")
ax.text(0.49, y - 0.028, f"{hr:.3f} ({lo:.3f}–{hi:.3f})", fontsize=10, va="center", color=INK2)
ax.text(0.79, y, f"{T2['CVD'][m][2]:.3f}", fontsize=10, va="center")
ax.text(0.90, y, f"{REC[(m, 'per SD')]:.3f}", fontsize=10, va="center")
ax.text(0.49, 0.055, "Every CI includes 1; every P > 0.05\n(the paper's cut-off for main effects).",
fontsize=10, color=INK2, va="bottom", linespacing=1.3)
ax.add_patch(FancyArrowPatch((0.345, 0.72), (0.465, 0.72), arrowstyle="<->", mutation_scale=14,
lw=LW, color=INK))
ax.text(0.405, 0.69, "Conflict", ha="center", va="top", fontsize=10.5, weight="bold")
ax.text(0.405, 0.63, "Abstract:\n3 of 3\n\nTable 2:\n0 of 3", ha="center", va="top", fontsize=10,
linespacing=1.3)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="printed in the paper"),
Line2D([], [], marker="s", ls="", ms=MS, color=BLUE,
label="recomputed from the\nprinted 95% CI (Wald,\nlog scale)")],
loc="lower left", bbox_to_anchor=(0.01, 0.03), frameon=False, fontsize=10, handletextpad=0.4)
save(fig, "cand_A", also="fig1")
# ---- B: reported vs rebuilt CI on one log axis, with control rows ---------------
rows = [(LBL[m], *T2["CVD"][m][1], REBUILT[m]) for m in PPD]
ctrl = []
for k in ("Mean CAL", "Missing teeth", "Periodontitis score"):
hr, lo, hi, p = T2_CVD_OTHER[k]
ctrl.append((k + " (control)", hr, lo, hi, ci_from_p(hr, p)))
allrows = rows + ctrl
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ys = np.array([6.5, 5.5, 4.5, 2.5, 1.5, 0.5])
for y, (lab, hr, lo, hi, (rl, rh)) in zip(ys, allrows):
ax.plot([lo, hi], [y + 0.15] * 2, color=ORANGE, lw=LW, solid_capstyle="round")
ax.plot(hr, y + 0.15, "o", ms=MS, color=ORANGE, mec=BG, mew=1)
ax.plot([rl, rh], [y - 0.15] * 2, color=BLUE, lw=LW, solid_capstyle="round")
ax.plot(hr, y - 0.15, "s", ms=MS - 1, color=BLUE, mec=BG, mew=1)
ax.axvline(1, color=INK, lw=1, zorder=1.5)
ax.set_xscale("log")
ticks = [0.9, 1.0, 1.2, 1.4, 1.6, 1.8]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:.1f}" for t in ticks]); ax.minorticks_off()
ax.set_xlim(0.86, 2.2)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in allrows], fontsize=10)
for t in ax.get_yticklabels()[3:]:
t.set_color(INK2)
ax.set_ylim(-0.1, 7.6)
ax.axhline(3.5, color=GRID, lw=1)
ax.text(0.87, 7.25, "PPD measures (abstract: “significant”)", fontsize=10, weight="bold", va="center",
bbox=dict(fc=BG, ec="none", pad=1), zorder=4)
ax.text(2.18, 7.25, "printed P", ha="right", fontsize=10, color=INK2, va="center")
ax.text(0.87, 3.15, "Other periodontal measures, same table", fontsize=10, color=INK2, va="center",
bbox=dict(fc=BG, ec="none", pad=1), zorder=4)
for y, m in zip(ys[:3], PPD):
ax.text(2.18, y, f"{T2['CVD'][m][2]:.3f}", ha="right", va="center", fontsize=10, color=INK2)
for y, k in zip(ys[3:], ("Mean CAL", "Missing teeth", "Periodontitis score")):
ax.text(2.18, y, f"{T2_CVD_OTHER[k][3]:.3f}", ha="right", va="center", fontsize=10, color=INK2)
ax.set_xlabel("Hazard ratio per SD increase, CVD mortality (log scale; line at HR = 1)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], marker="o", color=ORANGE, lw=LW, ms=MS, label="95% CI printed in Table 2"),
Line2D([], [], marker="s", color=BLUE, lw=LW, ms=MS - 1,
label="95% CI rebuilt from HR and printed P")],
loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_B", also="fig2")
# ---- C: significance grid, PPD x outcome x table --------------------------------
fig, ax = plt.subplots(figsize=(7.2, 3.4))
xs = {("Table 2", o): i for i, o in enumerate(OUTCOMES)}
xs.update({("Table S4", o): i + 3.6 for i, o in enumerate(OUTCOMES)})
for (src, o), x in xs.items():
if o == "CVD":
ax.add_patch(plt.Rectangle((x - 0.45, -0.5), 0.9, 3.0, fc="#f1f0ec", ec="none", zorder=0))
for j, m in enumerate(PPD):
y = 2 - j
if src == "Table 2":
hr, lo, hi = T2[o][m][1]; p = T2[o][m][2]
else:
hr, lo, hi, p = S4[o][m]
pc = p_from_ci(hr, lo, hi)[0]
filled = p < 0.05
ax.plot(x - 0.17, y + 0.12, "o", ms=11, mfc=ORANGE if filled else BG, mec=ORANGE, mew=2)
ax.plot(x + 0.17, y + 0.12, "s", ms=10, mfc=BLUE if pc < 0.05 else BG, mec=BLUE, mew=2)
ax.text(x, y - 0.25, f"{p:.3f}", ha="center", va="center", fontsize=10,
color=INK if filled else INK2)
ax.set_yticks([2, 1, 0]); ax.set_yticklabels([LBL[m] for m in PPD])
ax.set_xticks(list(xs.values())); ax.set_xticklabels([o for (_, o) in xs], fontsize=10)
for (src, x0) in (("Table 2 (time-varying, main)", 1), ("Table S4 (time-invariant)", 4.6)):
ax.text(x0, 3.55, src, ha="center", fontsize=10.5, weight="bold")
ax.text(1, -0.8, "CVD column: 0 of 3 significant", ha="center", fontsize=10, weight="bold")
ax.text(4.6, -0.8, "CVD column: 0 of 3 significant", ha="center", fontsize=10, weight="bold")
ax.set_xlim(-0.6, 6.2); ax.set_ylim(-1.0, 3.1)
ax.xaxis.tick_top()
for s in ax.spines.values():
s.set_visible(False)
ax.tick_params(length=0)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=10, mfc=ORANGE, mec=ORANGE, label="printed P < 0.05"),
Line2D([], [], marker="o", ls="", ms=10, mfc=BG, mec=ORANGE, mew=2, label="printed P ≥ 0.05"),
Line2D([], [], marker="s", ls="", ms=9, mfc=BLUE, mec=BLUE, label="recomputed P < 0.05"),
Line2D([], [], marker="s", ls="", ms=9, mfc=BG, mec=BLUE, mew=2, label="recomputed P ≥ 0.05")],
loc="upper center", bbox_to_anchor=(0.5, -0.12), ncol=4, frameon=False, fontsize=9.5,
handletextpad=0.3, columnspacing=1.0,
title="Number under each pair: printed P (HR per SD)", title_fontsize=10)
save(fig, "cand_C")
# ---- D: P values on one log axis with the 0.05 line -----------------------------
fig, ax = plt.subplots(figsize=(7.2, 3.6))
items = [(f"{LBL[m]} (Table 2)", T2["CVD"][m][2], REC[(m, 'per SD')]) for m in PPD]
items += [(f"{LBL[m]} (Table S4)", S4["CVD"][m][3], p_from_ci(*S4["CVD"][m][:3])[0]) for m in PPD]
items += [(f"{k} (Table 2, control)", T2_CVD_OTHER[k][3], p_from_ci(*T2_CVD_OTHER[k][:3])[0])
for k in ("Mean CAL", "Missing teeth")]
yy = np.arange(len(items))[::-1].astype(float)
yy[:6] += 0.5
for y, (lab, p, pc) in zip(yy, items):
ax.plot([min(p, pc), max(p, pc)], [y, y], color=GRID, lw=LW, zorder=1)
ax.plot(p, y + 0.13, "o", ms=MS, color=ORANGE, mec=BG, mew=1, zorder=3)
ax.plot(pc, y - 0.13, "s", ms=MS - 1, color=BLUE, mec=BG, mew=1, zorder=3)
ax.axvline(0.05, color=INK, lw=1)
ax.text(0.052, yy[-1] - 0.75, "P = 0.05", fontsize=10, va="bottom")
ax.axvspan(0.005, 0.05, color="#f1f0ec", zorder=0)
ax.text(0.0065, yy[0] + 0.15, "significant\nat α = 0.05", fontsize=10, color=INK2, va="top")
ax.set_xscale("log"); ax.set_xlim(0.005, 1)
ax.set_xticks([0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1]); ax.set_xticklabels(["0.01", "0.02", "0.05", "0.1", "0.2", "0.5", "1"])
ax.minorticks_off()
ax.set_yticks(yy); ax.set_yticklabels([i[0] for i in items], fontsize=10)
ax.set_ylim(yy[-1] - 0.8, yy[0] + 0.6)
ax.set_xlabel("P value, CVD mortality, HR per SD (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="P printed in the paper"),
Line2D([], [], marker="s", ls="", ms=MS - 1, color=BLUE, label="P recomputed from printed CI")],
loc="upper center", bbox_to_anchor=(0.35, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_D")
# ---- E: count comparison --------------------------------------------------------
fig, ax = plt.subplots(figsize=(7.2, 2.6))
bars = [("Abstract claim", 3, ORANGE, "o"),
("Table 2, printed P", SIGCOUNT[("Table 2", "CVD")], ORANGE, "o"),
("Table 2, P recomputed from CI", sum(REC[(m, 'per SD')] < 0.05 for m in PPD), BLUE, "s"),
("Table S4, printed P", SIGCOUNT[("Table S4", "CVD")], ORANGE, "o"),
("Table S4, P recomputed from CI", sum(p_from_ci(*S4["CVD"][m][:3])[0] < 0.05 for m in PPD), BLUE, "s")]
yb = np.arange(len(bars))[::-1]
for y, (lab, n, c, mk) in zip(yb, bars):
ax.plot([0, n], [y, y], color=c, lw=LW, solid_capstyle="round")
ax.plot(n, y, mk, ms=MS + 1, color=c, mec=BG, mew=1)
ax.text(n + 0.08, y, f"{n} of 3", va="center", fontsize=10)
ax.set_yticks(yb); ax.set_yticklabels([b[0] for b in bars], fontsize=10)
ax.set_xlim(-0.1, 3.5); ax.set_xticks([0, 1, 2, 3])
ax.set_xlabel("PPD measures significant for CVD mortality (P < 0.05), out of 3")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], marker="o", color=ORANGE, lw=LW, ms=MS, label="stated or printed in the paper"),
Line2D([], [], marker="s", color=BLUE, lw=LW, ms=MS, label="recomputed from the paper's CIs")],
loc="upper center", bbox_to_anchor=(0.35, -0.3), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_E")
print("\nwrote fig1/fig2 and candidates/cand_A..cand_E (svg, png) in", HERE)
All reported HPV prevalence confidence intervals are about 20 to 140 times narrower than the counts allow
BMC Med (2025) · doi:10.1186/s12916-025-04246-0 · PMC12211446
Summary
The Abstract, Results and Table 1 report a cervicovaginal HPV prevalence of 6.2% (51 of 822 women) with a 95% confidence interval of 6.18%–6.22%, and give intervals of ±0.01 to ±0.02 percentage points for every genotype-specific prevalence. From the paper's own counts, the exact binomial 95% interval for 51/822 is 4.65%–8.08%, 86 times as wide as the reported one, and the genotype intervals are 22 to 143 times as wide as reported; the point estimates themselves are correct. Four of the six reported intervals are reproduced to the printed decimal by one arithmetic step, a 95% half-width computed on the proportion scale (0.0165 for 51/822) added to a prevalence expressed in percent. Prevalence is the study's primary outcome, so its stated precision is overstated about 80-fold, which bears on the Discussion's comparison with 2009 estimates (the 2009 HPV16/18 value of 1.2% lies inside the corrected interval) and on any meta-analysis that uses the published interval.
What the paper reports
The prevalence of HPV infection was 6.2% (95% CI: 6.18%-6.22%), while the prevalence of high-risk genotypes 16 and 18 was 1.94% (95% CI: 1.93%-1.95%). The prevalence of 12 pooled high-risk HPV infection was 4.14% (95% CI:4.13%-4.15%).
Abstract, Results
The prevalence of cervicovaginal HPV infection among ever-married women of the 35-year-old age cohort was 6.2% (95% CI: 6.18%-6.22%), as 51, had tested positive for HPV infection out of 822 participants. The prevalence of high-risk carcinogenic HPV genotypes 16(n = 14) and 18(n = 2) was 1.94% (95% CI: 1.93%-1.95%), while the other 12 pooled high-risk HPV genotypes (n = 34) was 4.14% (95% CI: 4.13%-4.15%) (Table 1).
Results, paragraph 2
Table 1 ("Prevalence of cervico-vaginal HPV infection in the District") has a column headed "95% CI for the prevalence %". Its rows are: 12 pooled positive, 34, 4.14, 4.13-4.15; 16 positive, 14, 1.7, 1.69-1.71; 18 positive, 2, 0.24, 0.23-0.25; 16 & 12 pooled positive, 01, 0.12, 0.1-0.13; Negative, 771, 93.8; Total, 822.
On the design and analysis, the paper says:
In the district, there are 413 PHM areas.
Methods, paragraph 3 (each Public Health Midwife area is a cluster, and three women were selected from each)
the degree of accuracy desired specified was 0.016 (3.3/100 × 2). Therefore we needed 479 women.
Methods, paragraph 2 (the required sample size was then "multiplied by the design effect, which was taken as 1.1")
Data were entered by using SPSS version 20.
Methods, paragraph 8
Moreover, high-risk genotypes 16 and 18 prevalence were elevated from 1.2% to 1.94% within the same duration.
Discussion, paragraph 2
What the paper's own numbers show
Method. Each prevalence is a count out of n = 822. For each count I computed the 95% interval three ways: the Clopper–Pearson exact interval, the Wilson interval, and the Wald interval p ± 1.96·√(p(1−p)/n). I also computed the Wald interval with its variance inflated by the paper's stated design effect of 1.1. I then tested one candidate arithmetic route: take 1.96·SE on the proportion scale and add it, unconverted, to the prevalence in percent. All numbers come from recompute.py.txt.
The counts are internally consistent: 34 + 14 + 2 + 1 = 51 positives, 51 + 771 = 822, and 14 + 2 = 16 for HPV16 or 18. Every reported point estimate equals its count divided by 822 (for 16/822 = 1.946% the paper prints 1.94%, truncated rather than rounded, which does not matter here).
| Quantity | Location | Reported | Recomputed from the paper's counts |
|---|---|---|---|
| Any HPV, 51/822 | Abstract; Results p2 | 6.2% (95% CI 6.18–6.22) | 6.204%; exact 4.65–8.08; Wilson 4.75–8.07; Wald 4.56–7.85; Wald with design effect 1.1 4.47–7.93 |
| HPV16 or 18, 16/822 | Abstract; Results p2 | 1.94% (1.93–1.95) | 1.946%; exact 1.12–3.14; Wilson 1.20–3.14 |
| 12 pooled high-risk types, 34/822 | Abstract; Results p2; Table 1 | 4.14% (4.13–4.15) | 4.136%; exact 2.88–5.73; Wilson 2.97–5.72 |
| HPV16, 14/822 | Table 1 | 1.7% (1.69–1.71) | 1.703%; exact 0.93–2.84; Wilson 1.02–2.84 |
| HPV18, 2/822 | Table 1 | 0.24% (0.23–0.25) | 0.243%; exact 0.03–0.88; Wilson 0.07–0.88 |
| HPV16 & 12 pooled, 1/822 | Table 1 | 0.12% (0.1–0.13) | 0.122%; exact 0.00–0.68; Wilson 0.02–0.69 |
| Half-width of the overall interval | derived | 0.02 points | 1.65 points (Wald), 82 times the reported half-width; the exact interval is 86 times as wide as the reported one |
| How much too narrow, genotype rows (exact width ÷ reported width) | derived | — | HPV16 or 18: 101×; 12 pooled: 143×; HPV16: 95×; HPV18: 42×; HPV16 & 12 pooled: 22× |
| Sample size a ±0.02-point interval would need at 6.2% | derived | — | about 5.59 million |
| Population size at which a finite-population correction would give ±0.02 points | derived | — | 822.1, i.e. the sample itself |
| Percent prevalence ± 1.96·SE in proportion units, centred on the reported rounded prevalence | derived | 6.18–6.22; 1.93–1.95; 4.13–4.15; 1.69–1.71; 0.23–0.25; 0.1–0.13 | 6.18–6.22; 1.93–1.95; 4.13–4.15; 1.69–1.71 (all four exact matches); 0.24–0.24; 0.12–0.12 |
| Sample-size calculation | Methods p2 | 479; 915 | 1.96²·0.033·0.967/0.016² = 478.9; 479·1.1/(1−0.424) = 914.8 (consistent) |
| Planned precision vs achieved | Methods p2 (derived) | ±1.6 points (d = 0.016) at an assumed 3.3% | The 95% half-width at the observed 6.2% with n = 822 is 1.65 points, close to the planned ±1.6 points and about 80 times the reported 0.02 |
The paper's own sample-size calculation targeted a precision of ±1.6 percentage points (d = 0.016). The correct half-width at the observed prevalence, 1.65 points, is close to the planned ±1.6 points (the plan assumed 3.3% and n = 479, so the two need not coincide). The reported interval is roughly 80 times narrower than the precision the study was designed to achieve.
The arithmetic route above reproduces four of the six reported intervals exactly: 6.18–6.22, 1.93–1.95, 4.13–4.15 and 1.69–1.71. For the two smallest counts (2 and 1 positives), that route gives half-widths of 0.0034 and 0.0024 points. The paper reports 0.23–0.25 and 0.1–0.13, half-widths of 0.010 and 0.015 points, about 3 and 6 times what that route gives, so the route does not reproduce these two rows. They are still 42 and 22 times narrower than the exact intervals.
(An aside that does not affect the finding: the parenthetical "3.3/100 × 2" evaluates to 0.066, not 0.016. The stated 479 follows from d = 0.016, which is close to 3.3/100 ÷ 2 = 0.0165.)
Figure
Figure 1. Each row is one prevalence reported in the Abstract, the Results or Table 1, with its count out of 822. Orange diamonds show the paper's reported estimate and 95% CI; every reported interval is narrower than its marker, so its bar is hidden. Blue circles with capped bars show the exact (Clopper–Pearson) 95% CI computed from the same count. The labels to the right give each interval, marked "exact" or "reported". The dashed gray lines mark the 2009 values that the Discussion compares against (3.3%, 2.1% and 1.2%). The 1.2% value for HPV16 or 18 lies inside the exact interval for this study. The axis starts at zero and is linear.
Figure 2. Arithmetic for the overall interval. Start from 51/822, compute the standard error and the 95% half-width (0.0165 on the proportion scale), then express the interval in percent. Multiplying the half-width by 100 gives 6.20% ± 1.65 points (blue, 4.56%–7.85% by the Wald method). Adding 0.0165 unconverted to the printed 6.2% gives 6.1835%–6.2165%, which rounds to 6.18%–6.22% (orange), the reported interval to the second decimal place. The same step reproduces the reported intervals for HPV16 or 18, the 12 pooled types and HPV16 (Table above).
Why it matters
- Primary outcome. The study's stated objective is to determine prevalence. Its headline interval tells readers the prevalence is known to within ±0.02 percentage points. The data support only about ±1.7 points (exact 4.65%–8.08%).
- Comparisons in the Discussion. The Discussion says HPV16/18 prevalence "were elevated from 1.2% to 1.94%" since 2009. With the reported interval (1.93–1.95%), 1.2% looks far outside the estimate. With the exact interval (1.12–3.14%), 1.2% lies inside it, and a two-sided exact binomial test of 16/822 against a fixed 1.2% gives p = 0.054. That test also ignores the 2009 study's own sampling error and the differences in district and age range. The comparisons for overall prevalence (3.3% vs 6.2%, p = 3.3 × 10⁻⁵ against a fixed 3.3%) and for the other high-risk types (2.1% vs 4.14%, p = 0.0003) still point the same way once corrected. Those tests too treat the 2009 values as known exactly.
- The same intervals in an earlier article. An article from the same survey (BMC Public Health 2021, doi:10.1186/s12889-021-10190-4, PMC7805031) prints the same overall interval, "6.2% (95%CI: 6.18–6.22%)", in its Abstract and Results, and the same genotype intervals in its Table 4 (4.13–4.15, 1.69–1.71, 0.23–0.25, 0.1–0.13). Crossref, PMC and Europe PMC list no correction for it either. A correction to one article would leave the other unchanged.
- Use by others. In an inverse-variance meta-analysis, the reported interval implies a variance about 82.5² ≈ 6,800 times too small (the Wald half-width is 82.5 times the reported one). A pooled estimate that used it would be dominated by this one study.
- What is not affected. The point estimates, the counts, and the odds ratios from the logistic regression do not depend on the prevalence intervals. The abstract's conclusions are about risk factors and stand as they are on this point.
Explanations considered
- A different, valid interval method (Wilson, exact, Wald). No. All three give half-widths of 1.6–1.7 points for 51/822, and none comes within an order of magnitude of ±0.02.
- Design effect or cluster adjustment. No. The Methods state a design effect of 1.1 for the cluster sample of 413 PHM areas. A design effect of 1 or more widens the interval: 4.47–7.93% with 1.1.
- Finite-population correction. No. To shrink the half-width to 0.02 points, the target population would have to be about 822.1 women, which is the sample itself. The Methods describe selecting three women per cluster from eligible-couple registers across the district, so the sample is not a census.
- Units: proportion instead of percent. Not as a labelling issue. The text, the Abstract and the Table 1 header all state the intervals in percent. If the limits were proportions, 6.18–6.22 would mean 618%–622%. The units issue that does fit is internal to the calculation: a half-width in proportion units (0.0165) added to a prevalence in percent (6.2). This reproduces four of the six reported intervals exactly (Figure 2).
- An interval for a different quantity (such as a mean or a weighted estimate). Nothing in the Methods describes one. The Statistical analysis paragraph names only prevalence estimation, bivariate analysis and logistic regression, and the table column is headed "95% CI for the prevalence %".
- Rounding or transcription. No. A rounding slip could not shrink six independently computed intervals by a factor of 20 to 140.
- Conversion artefact in the archived text. No. The same intervals appear in three places in both the PMC text and the publisher-supplied JATS XML on Europe PMC (Abstract, Results paragraph 2, Table 1), and the abstract intervals appear identically in the PubMed record. The dash character in the ranges differs between sources (the Europe PMC JATS writes the Results range with a minus sign, U+2212, where the Abstract and the PMC text use a hyphen), but the digits and decimal points are identical everywhere.
- Misaligned table cells. No. Each Table 1 row's prevalence equals its own count divided by 822, and each row's interval sits on its own prevalence.
Status
As of 2026-10-23:
- Version of record. The publisher's page (doi.org → bmcmedicine.biomedcentral.com → link.springer.com) redirected to a login, and both publisher PDF links returned HTML rather than a PDF, so the publisher's page could not be read directly. Three other sources built from the publisher's files were checked and agree on every interval they contain: the publisher-supplied JATS full text on Europe PMC, the PMC article, and the PubMed abstract (PMID 40598371).
- Supplements. Both supplementary files were opened: Annexure 1, the questionnaire, and Annexure 2, the specimen-collection guideline. Neither contains analysis output or a description of the interval method. The data are available only from the corresponding author on request.
- Corrections and comments. None found.
src/known.py(Crossrefupdate-to, PMC and Europe PMC notices) returns nothing for this article or for the 2021 BMC Public Health article that carries the same intervals, and the Europe PMC record for this article has no comment or correction list. The PubMed record has no CommentsCorrections entries (erratum, comment or retraction). Web searches for the DOI and title with "correction", "erratum" and "PubPeer" found no notice and no PubPeer thread (rechecked on 2026-10-23); the PubPeer API returned HTTP 403 when queried directly. - Sources. PMC12211446 via
src/show.py(Abstract, Methods paragraphs 2, 3 and 8, Results paragraph 2, Table 1, Discussion paragraph 2);src/known.py 10.1186/s12916-025-04246-0; NCBI E-utilities efetch for PMID 40598371 (abstract text and CommentsCorrections); Europe PMC fullTextXML for PMC7805031 (2021 article);recompute.py.txtin this folder.
Suggested correction
Replace the prevalence intervals with intervals computed from the counts. Exact (Clopper–Pearson) 95% CIs, which a design effect of 1.1 would widen slightly, are:
- Any HPV: 6.2% (95% CI 4.65%–8.08%)
- HPV16 or 18: 1.94% (1.12%–3.14%)
- 12 pooled high-risk types: 4.14% (2.88%–5.73%)
- HPV16: 1.70% (0.93%–2.84%)
- HPV18: 0.24% (0.03%–0.88%)
- HPV16 and 12 pooled: 0.12% (0.00%–0.68%)
Apply these in the Abstract, Results paragraph 2 and Table 1. If the authors prefer a design-based interval, the Wald interval with the stated design effect for the overall prevalence is 4.47%–7.93%. The Discussion's statement that HPV16/18 prevalence rose from 1.2% to 1.94% should note that 1.2% lies within the corrected interval for this study. The same intervals in the Abstract, Results and Table 4 of the 2021 BMC Public Health article from this survey (doi:10.1186/s12889-021-10190-4) would need the same correction.
Recomputation script (python recompute.py.txt)
# Recomputation for PMC12211446 (BMC Med 2025, doi:10.1186/s12916-025-04246-0)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1/fig2 (SVG + PNG)
# in the folder that holds this script. Inputs are the counts printed in the paper
# (Results paragraph 2, Table 1); nothing else is assumed.
# Paragraph numbers count from 1 (show.py's [sec:X#pK] tags are 0-based, so #p1 = paragraph 2).
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
N = 822 # participants tested (Results p2, Table 1 total)
Z = stats.norm.ppf(0.975) # 1.95996
DEFF = 1.1 # design effect stated in Methods p2
# label, count, reported prevalence (%), reported 95% CI (%), where reported
ROWS = [
("Any HPV", 51, 6.2, (6.18, 6.22), "Abstract; Results p2"),
("HPV16 or 18", 16, 1.94, (1.93, 1.95), "Abstract; Results p2"),
("12 pooled hrHPV", 34, 4.14, (4.13, 4.15), "Abstract; Results p2; Table 1"),
("HPV16", 14, 1.7, (1.69, 1.71), "Table 1"),
("HPV18", 2, 0.24, (0.23, 0.25), "Table 1"),
("HPV16 & 12 pooled", 1, 0.12, (0.10, 0.13), "Table 1"),
]
def wald(x, n, deff=1.0):
p = x / n
se = np.sqrt(deff * p * (1 - p) / n)
return 100 * (p - Z * se), 100 * (p + Z * se)
def wilson(x, n):
p = x / n
den = 1 + Z**2 / n
c = (p + Z**2 / (2 * n)) / den
h = Z * np.sqrt(p * (1 - p) / n + Z**2 / (4 * n * n)) / den
return 100 * (c - h), 100 * (c + h)
def clopper_pearson(x, n):
lo = stats.beta.ppf(0.025, x, n - x + 1) if x > 0 else 0.0
hi = stats.beta.ppf(0.975, x + 1, n - x) if x < n else 1.0
return 100 * lo, 100 * hi
def unit_slip(x, n):
"""Percent prevalence +/- 1.96*SE where the SE is left on the proportion scale."""
p = x / n
hw = Z * np.sqrt(p * (1 - p) / n) # proportion units
return 100 * p - hw, 100 * p + hw, hw
print("=== 1. Counts")
print(f"34 + 14 + 2 + 1 = {34 + 14 + 2 + 1} positives; 51 + 771 = {51 + 771} total")
print(f"HPV16 or 18 = 14 + 2 = {14 + 2}")
print("\n=== 2. Point estimates and 95% CIs (percent), n = 822")
res = {}
for lab, x, prep, (lo, hi), loc in ROWS:
p = 100 * x / N
w, wd, wi, cp = wald(x, N), wald(x, N, DEFF), wilson(x, N), clopper_pearson(x, N)
s_lo, s_hi, s_hw = unit_slip(x, N)
rep_hw = (hi - lo) / 2
wald_hw = (w[1] - w[0]) / 2
res[lab] = dict(x=x, p=p, rep=(lo, hi), cp=cp, wil=wi, slip=(s_lo, s_hi))
print(f"{lab:18s} {x:2d}/822 = {p:.3f}% (reported {prep}) | reported CI {lo:.2f}-{hi:.2f} (half-width {rep_hw:.3f} pp)")
print(f"{'':18s} Wald {w[0]:.2f}-{w[1]:.2f} | Wald x DEFF 1.1 {wd[0]:.2f}-{wd[1]:.2f} | "
f"Wilson {wi[0]:.2f}-{wi[1]:.2f} | Clopper-Pearson {cp[0]:.2f}-{cp[1]:.2f}")
print(f"{'':18s} Wald half-width {wald_hw:.2f} pp = {wald_hw / rep_hw:.0f} x reported half-width")
print(f"{'':18s} percent p +/- 1.96*SE(proportion units, {s_hw:.4f}): {s_lo:.3f}-{s_hi:.3f}"
f" -> rounded {s_lo:.2f}-{s_hi:.2f}")
r_lo, r_hi = prep - s_hw, prep + s_hw
match = (round(r_lo, 2), round(r_hi, 2)) == (lo, hi)
res[lab]["slip_rep"] = (r_lo, r_hi)
print(f"{'':18s} same, centred on the reported (rounded) {prep}: {r_lo:.4f}-{r_hi:.4f}"
f" -> rounded {r_lo:.2f}-{r_hi:.2f}; equals reported CI: {match}")
print(f"{'':18s} reported half-width / route half-width (pp): {rep_hw:.3f} / {s_hw:.4f} = {rep_hw / s_hw:.1f}")
print(f"{'':18s} exact CI width {cp[1] - cp[0]:.2f} pp = {(cp[1] - cp[0]) / (hi - lo):.0f} x reported width")
print("\n=== 3. How the overall interval compares")
p = 51 / N
se = np.sqrt(p * (1 - p) / N)
print(f"SE of 51/822 = {se:.5f} as a proportion = {100 * se:.3f} percentage points")
print(f"95% half-width = {100 * Z * se:.3f} pp; reported half-width 0.02 pp; ratio {100 * Z * se / 0.02:.1f}")
print(f"SE alone / 0.02 = {100 * se / 0.02:.1f}")
cp51 = clopper_pearson(51, N)
print(f"Exact (Clopper-Pearson) half-width = {(cp51[1] - cp51[0]) / 2:.2f} pp")
n_req = p * (1 - p) * (Z / 0.0002) ** 2
print(f"Sample size a +/-0.02 pp Wald interval would need: {n_req:,.0f}")
f = (0.02 / (100 * Z * se)) ** 2 # required finite-population factor (N-n)/(N-1)
Npop = (N - f) / (1 - f)
print(f"Finite-population factor needed: (Npop-n)/(Npop-1) = {f:.5f} -> Npop = {Npop:.1f}")
print(f"Inverse-variance weight implied by reported CI vs Wald CI: {(100 * Z * se / 0.02) ** 2:,.0f} x")
print("\n=== 4. Sample-size calculation in Methods p2")
pp, d = 0.033, 0.016
n0 = 1.96**2 * pp * (1 - pp) / d**2
print(f"1.96^2*0.033*0.967/0.016^2 = {n0:.1f} (reported 479)")
print(f"479*1.1/(1-0.424) = {479 * 1.1 / (1 - 0.424):.1f} (reported 915)")
print(f"Planned 95% half-width at 3.3%: {100 * d:.1f} pp; at observed 6.2%, n=822: {100 * Z * se:.2f} pp")
print(f"(The parenthetical '3.3/100 x 2' evaluates to {3.3 / 100 * 2:.3f}; 0.016 is close to 3.3/100/2 = {3.3 / 100 / 2:.4f})")
print("\n=== 5. Discussion comparisons with 2009 values (one-sample exact binomial, two-sided;")
print(" treats the 2009 figure as fixed, so it ignores that study's own sampling error)")
for lab, x, ref in [("Any HPV vs 3.3%", 51, 0.033), ("HPV16 or 18 vs 1.2%", 16, 0.012),
("12 pooled vs 2.1%", 34, 0.021)]:
t = stats.binomtest(x, N, ref)
cp = clopper_pearson(x, N)
inside = cp[0] <= 100 * ref <= cp[1]
print(f"{lab:22s} {x}/822 = {100 * x / N:.2f}%; exact CI {cp[0]:.2f}-{cp[1]:.2f}; "
f"2009 value inside exact CI: {inside}; p = {t.pvalue:.4f}"
f"{' (< 1e-4: ' + format(t.pvalue, '.1e') + ')' if t.pvalue < 1e-4 else ''}")
# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"axes.edgecolor": INK2, "svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# Fig 1: dot-and-interval, reported vs exact CI, one linear axis in percent
refs2009 = {"Any HPV": 3.3, "HPV16 or 18": 1.2, "12 pooled hrHPV": 2.1}
order = ["Any HPV", "12 pooled hrHPV", "HPV16 or 18", "HPV16", "HPV18", "HPV16 & 12 pooled"]
fig, ax = plt.subplots(figsize=(7.2, 5.2))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
for i, lab in enumerate(order):
r = res[lab]
yb, yo = i - 0.17, i + 0.17
ax.plot(r["cp"], [yb, yb], color=BLUE, lw=2, solid_capstyle="butt")
for xv in r["cp"]:
ax.plot([xv, xv], [yb - 0.09, yb + 0.09], color=BLUE, lw=2)
ax.plot(r["p"], yb, "o", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=3)
ax.plot(r["rep"], [yo, yo], color=ORANGE, lw=2)
ax.plot(r["p"], yo, "D", ms=8, color=ORANGE, mec=BG, mew=1.2, zorder=3)
ax.text(r["cp"][1] + 0.15, yb, f"exact {r['cp'][0]:.2f}–{r['cp'][1]:.2f}", va="center",
fontsize=10, color=INK)
ax.text(r["cp"][1] + 0.15, yo, f"reported {r['rep'][0]:.2f}–{r['rep'][1]:.2f}", va="center",
fontsize=10, color=INK2)
if lab in refs2009:
v = refs2009[lab]
ax.plot([v, v], [i - 0.38, i + 0.38], color=INK2, lw=1, ls=(0, (2, 2)))
ax.text(v, i - 0.42, f"2009: {v}%", ha="center", va="bottom", fontsize=10, color=INK2)
ax.set_yticks(range(len(order)))
ax.set_yticklabels([f"{lab}\n({res[lab]['x']}/822)" for lab in order])
ax.invert_yaxis()
ax.set_xlim(-0.15, 11.2)
ax.set_xticks(range(0, 11, 2))
ax.spines["bottom"].set_bounds(0, 10)
ax.set_xlabel("Prevalence, % (95% CI)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h1 = ax.plot([], [], "-D", color=ORANGE, lw=2, ms=8, label="Reported in the paper (95% CI; bars hidden behind the markers)")[0]
h2 = ax.plot([], [], "-o", color=BLUE, lw=2, ms=8, label="Exact (Clopper–Pearson) 95% CI recomputed from the paper's counts")[0]
h3 = ax.plot([], [], color=INK2, lw=1, ls=(0, (2, 2)), label="2009 value cited in the Discussion")[0]
ax.legend(handles=[h1, h2, h3], loc="upper left", bbox_to_anchor=(-0.05, -0.15), ncol=1,
frameon=False, fontsize=10)
ax.set_title("Reported vs exact 95% intervals for the same counts (n = 822)",
loc="left", fontsize=11, color=INK)
fig.tight_layout()
save(fig, "fig1")
# Fig 2: arithmetic chain for the overall interval; the divergent step highlighted
hw_prop = Z * se
fig = plt.figure(figsize=(7.2, 3.6))
ax = fig.add_axes([0.01, 0.01, 0.98, 0.98])
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
ax.set_xlim(0, 100)
ax.set_ylim(0, 65)
ax.axis("off")
def box(x, y, w, h, title, body, edge=INK2, lw=1.0):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc=BG, ec=edge, lw=lw))
ax.text(x + w / 2, y + h - 2.6, title, ha="center", va="top", fontsize=10, color=INK2)
ax.text(x + w / 2, y + h / 2 - 2.2, body, ha="center", va="center", fontsize=10.5, color=INK)
def arrow(p0, p1, color=INK2, lw=1.2):
ax.add_patch(FancyArrowPatch(p0, p1, arrowstyle="-|>", mutation_scale=12, color=color, lw=lw))
box(1, 44, 29, 15, "Counts (Results p2)", f"51 positive / 822\np = {p:.5f}")
box(35.5, 44, 29, 15, "SE (proportion)", f"√(p(1−p)/n) = {se:.5f}")
box(70, 44, 29, 15, "Half-width (proportion)", f"1.96 × SE = {hw_prop:.4f}")
arrow((30.6, 51.5), (35, 51.5))
arrow((65.1, 51.5), (69.5, 51.5))
# branch
box(3, 6, 42, 24, "Same units: convert half-width to %",
f"{hw_prop:.4f} × 100 = {100 * hw_prop:.2f} pp\n6.20% ± {100 * hw_prop:.2f} pp\n→ {wald(51, N)[0]:.2f}% to {wald(51, N)[1]:.2f}%",
edge=BLUE, lw=2)
box(55, 6, 42, 24, "Mixed units: proportion added to %",
f"6.2% ± {hw_prop:.4f}\n→ {res['Any HPV']['slip_rep'][0]:.2f}% to {res['Any HPV']['slip_rep'][1]:.2f}%\n(paper: 6.18% to 6.22%)",
edge=ORANGE, lw=2)
arrow((84.5, 43.3), (76, 31), color=ORANGE, lw=2)
arrow((84.5, 43.3), (24, 31), color=BLUE, lw=2)
ax.text(83, 36, "factor of 100\nlost here", ha="left", va="center",
fontsize=10, color=INK)
ax.text(24, 2.0, "Recomputed (blue)", ha="center", fontsize=10, color=INK2)
ax.text(76, 2.0, "Reproduces the reported interval (orange)", ha="center", fontsize=10, color=INK2)
ax.text(1, 62.5, "How the overall 95% CI of 6.18%–6.22% is reproduced", ha="left", va="center",
fontsize=11, color=INK)
save(fig, "fig2")
print("\nFigures written to", HERE)
Abstract and conclusions call the 57.4% concordant trials discordant; discordance was 42.6%
BMC Med (2025) · doi:10.1186/s12916-025-04328-z · PMC12403310
Summary
The study compares the data sharing plans that 376 clinical trials stated in their registrations with those stated in their publications, and its Abstract (Results and Conclusions) and main Conclusions section say that more than half of the trials were discordant, citing "216/376, 57.4%". By the paper's own Methods definitions and Table 2 (155 Yes/Yes, 13 Yes/No, 147 No/Yes, 61 No/No), those 216 trials (57.4%) are the concordant ones, and 160/376 (42.6%, 95% CI 37.7–47.6%) are discordant, as the Results text, Table 3, Supplementary Table S6 and the Discussion ("over 40% were discordant") also state. Concordance is one of the study's two stated outcomes, so a reader of the abstract alone gets its headline figure the wrong way round: a minority of trials, not a majority, had discordant plans. No test result or odds ratio changes; the correction is to three sentences.
What the paper reports
Abstract, Results:
Over half (216/376, 57.4%) had discordance between registration and publications.
Abstract, Conclusions:
More than half of published trials have data sharing discordance between registration and publication.
Main text, Conclusions section:
More than half of published trials have discordance between data sharing plans that they submit in trial registration versus publications.
The rest of the paper uses the same 216 trials as the concordant group.
Methods, "Study outcomes":
Subsequently, data sharing concordance was assessed: (1) plan to share data in registration and publication (both Yes in registration and publication, i.e., “Yes/Yes”) and (2) no plan to share data in registration and publication (both No in registration and publication, i.e., “No/No”). Discordance between registered and published plans to share data included (1) plan to share data in registration but no plan to share data in the publication (Yes in registration but No in publication, i.e., “Yes/No”) and (2) no plan to share in registration but a plan to share in publication (No in registration but Yes in publication, i.e., “No/Yes”).
Results, "Concordance between registered and published plans to share data":
A total of 376 trials were included for assessing the concordance, among which 216 (57.4%) had concordance between registered and published plans to share data (n = 155 and 61 for Yes/Yes and No/No, respectively; Table 2). For the discordance, there were 13 (3.5%) and 147 (39.1%) trials as Yes/No and No/Yes, respectively.
Table 2 (registration rows × publication columns): Yes/Yes 155 (41.2%), Yes/No 13 (3.5%), No/Yes 147 (39.1%), No/No 61 (16.2%); total 376 (100%).
Table 3 column headers: "Concordance between registered and published plans to share data", with sub-columns "(n = 160)" and "Yes(n = 216)". The archived text has no label in front of the first sub-column; the counts identify it as the discordant group.
Discussion, paragraph 1:
It was found that more than half of trials published in high-profile journals did not plan to share data in registration, and over 40% were discordant between registration and publication plans.
Discussion, paragraph 4:
Despite the ICMJE requirement, we found discordant data sharing plans in over 40% of trials, a finding that requires attention to increase consistency and reliability.
All quotes above were checked verbatim against the PMC-archived article text.
What the paper's own numbers show
Method. I took the four cell counts from Table 2 and applied the Methods definitions: concordant = Yes/Yes + No/No, discordant = Yes/No + No/Yes. I gave each share a Wilson 95% confidence interval. I re-ran the McNemar test on the two discordant cells as a consistency check. I then applied the same definitions to the two sensitivity tables in Additional file 1 (Table S7a, 78 undecided/missing registrations removed; Table S7b, those registrations counted as "Yes"). All calculations are in recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Trials assessed for concordance | Results; Table 2 | 376 | 155 + 13 + 147 + 61 = 376 |
| Concordant (Yes/Yes + No/No) | Results; Table 2; Table 3 "Yes" | 216 (57.4%) "had concordance" | 155 + 61 = 216/376 = 57.4% (95% CI 52.4–62.3%) |
| Discordant (Yes/No + No/Yes) | Results; Table 2; Table 3 | 13 (3.5%) + 147 (39.1%); Table 3 n = 160 | 13 + 147 = 160/376 = 42.6% (95% CI 37.7–47.6%) |
| Share discordant | Abstract, Results | "Over half (216/376, 57.4%) had discordance" | 160/376 = 42.6%; 216/376 is the concordant share |
| Is discordance more than half? | Abstract Conclusions; main Conclusions | "More than half" | No: 160 < 188 (half of 376); the CI upper bound, 47.6%, is below 50% |
| Discordance | Discussion (twice) | "over 40%" | 42.6%, consistent |
| Overstatement in abstract | Abstract vs Table 2 | 57.4% | 57.4 − 42.6 = 14.8 percentage points |
| Concordance, undecided removed | Results; Table S7a | 64.7% | 193/298 = 64.8%; discordant 105/298 = 35.2% (30.0–40.8%) |
| Concordance, undecided as Yes | Results; Table S7b | 65.9% | 248/376 = 66.0%; discordant 128/376 = 34.0% (29.4–39.0%) |
| McNemar test, 13 vs 147 | Results | P < 0.001 | χ² = 112.2 (110.6 with continuity correction), P ≈ 3e-26; exact P ≈ 7e-30 |
| Registered plan to share | Abstract | 44.6% (171/383) | 171/383 = 44.6% |
Every Table 2 cell and margin percentage reproduces exactly. The sensitivity figures differ from the recomputed ones by 0.1 point because the paper sums its printed component percentages (52.0% + 12.7% and 55.8% + 10.1%), two of which are rounded down (38/298 = 12.75%, printed 12.7%; 210/376 = 55.85%, printed 55.8%). This difference does not affect the finding. Only the abstract and conclusion sentences disagree with the data.
Figure
Figure 1. On the left (orange) are the Abstract Results and Conclusions sentences, which call 216/376 (57.4%) of trials discordant; the main Conclusions section repeats this. On the right is the paper's Table 2, redrawn as a 2 × 2 grid (rows: plan to share in the registration; columns: plan to share in the publication). Cells with solid blue borders are concordant by the Methods definition: Yes/Yes 155 + No/No 61 = 216 (57.4%). Cells with dashed borders are discordant: Yes/No 13 + No/Yes 147 = 160 (42.6%). The Results text describes the same 216 trials as having "concordance". The arrow links the abstract's 216 to the Table 2 total it equals, the concordant sum.
Figure 2. Share of trials whose registered and published data sharing plans disagree, on a full 0–100% axis. Orange: the value given in the abstract and conclusions (57.4%, "over half"). Blue: values recomputed from the paper's own counts, with Wilson 95% confidence intervals. The rows are the main analysis (Table 2, 160/376) and the two sensitivity analyses in Additional file 1 (Table S7a, 105/298; Table S7b, 128/376). The thin vertical line marks half of the trials. Every recomputed estimate and confidence interval lies below that line. The only value above it is the reported one, and that figure is the concordant share.
Why it matters
Concordance between registered and published data sharing plans is one of the study's two stated outcomes, and the abstract and conclusions give its central figure the wrong way round. A reader of the abstract alone learns that a majority (57.4%) of trials in six high-profile journals changed their data sharing plans between registration and publication. By the paper's own counts, a majority (57.4%) were consistent and 42.6% were discordant. The 95% interval for discordance, 37.7–47.6%, excludes 50%, so the error is not a matter of sampling uncertainty. The size of the problem is overstated by about 15 percentage points and pushed across the majority threshold. Abstract and conclusion sentences are the parts most likely to be quoted in later literature and in policy discussion of ICMJE data sharing requirements.
The following results are not affected: the 44.6% (171/383) of registrations with a plan to share; the drug-intervention odds ratio (2.71); the COVID-19 odds ratio for concordance (0.59), which was modelled with the correct groups (Table 3, n = 216 vs n = 160); the McNemar test; the content-level discordance among the 155 Yes/Yes trials; and the sensitivity analyses. The broader message, that a substantial share of trials report inconsistent data sharing plans, still holds at 42.6%. The Discussion already states it correctly ("over 40%").
Explanations considered
- A different definition of discordance. The Methods give a single definition (Yes/No + No/Yes), and no other trial-level definition appears anywhere in the paper. This does not explain the numbers.
- A different population or denominator. The abstract's 216/376 uses the same 376 trial publications as Table 2 and the Results. These are the trials analysed after the later-starting registration ID was chosen for each of the 7 pooled publications. The only other denominator, 383 registration IDs, is used just for the 171/383 figure, which is consistent. This does not explain the numbers.
- Content-level discordance. The abstract also reports "significant discordance" in SAPs/protocols, code and IPD. That analysis covers only the 155 Yes/Yes trials. A count of 216 cannot be built from it, and 216 (57.4%) matches the concordant total (155 + 61) exactly. This does not explain the numbers.
- Sensitivity analyses. These report concordance of 64.7% and 65.9%, so discordance is about 35% and 34%. Discordance is below half in every analysis the paper reports. This does not explain the numbers.
- Rounding or presentation. The error is not a rounding or threshold effect: 160 versus 216 out of 376 is a swap of the two complementary groups.
- Text-conversion artifact in PMC. The same wording appears in the PMC page and in the publisher-supplied JATS full text served by Europe PMC. The error also appears in three separately written sentences in two sections. An archiving artifact is therefore unlikely.
- Correct elsewhere in the paper. The Methods, Results text, Table 2, Table 3, Supplementary Table S6 ("Concordance (n =216)" and "Discordant groups (n =160)") and the Discussion ("over 40% were discordant") all treat 216 as concordant and 160 as discordant. The conflict is confined to the abstract and the two conclusion statements, which is consistent with a wording error in those sentences.
Status
As of 2026-10-23:
- Version of record. The publisher's HTML and PDF could not be opened from this machine: doi.org redirects through bmcmedicine.biomedcentral.com to link.springer.com, which requires a login. The text checked was the PMC-archived article (pmc.ncbi.nlm.nih.gov/articles/PMC12403310/) and the publisher-supplied JATS full text served by Europe PMC. Both carry the three sentences quoted above, and Table 2 is identical in both.
- Supplements. Additional file 1 (12916_2025_4328_MOESM1_ESM.docx, from static-content.springer.com) was read. Table S6 labels the groups "Concordance (n =216)" and "Discordant groups (n =160)". Tables S7a and S7b give 155/13/92/38 (N = 298) and 210/36/92/38 (N = 376). Discordance is below half in both. The supplement supports the reading given here and does not resolve the conflict.
- Corrections. No correction, erratum or expression of concern was found. Crossref update metadata: none. Crossref
filter=updates:query: 0 results. PMC notices: none. Europe PMC (PMID 40890807) lists no comment or correction. Crossref record:update-to,updated-byandrelationempty (rechecked 2026-10-23). Web searches for the DOI or title together with "correction" or "erratum", run during verification, returned nothing relevant. - PubPeer. PubPeer could not be queried directly: its search page returned HTTP 403 and its API returned a Cloudflare human-verification page (rechecked 2026-10-23). Web searches run while this finding was verified, for the title together with "PubPeer", found no PubPeer thread; a further search on 2026-10-23 could not be run because the session's search quota was exhausted.
Suggested correction
Abstract, Results. Replace "Over half (216/376, 57.4%) had discordance between registration and publications." with:
Over half (216/376, 57.4%) had concordance between registration and publications, and 160/376 (42.6%) had discordance.
Abstract, Conclusions. Replace "More than half of published trials have data sharing discordance between registration and publication." with:
More than 40% of published trials have data sharing discordance between registration and publication.
Main Conclusions section. Replace "More than half of published trials have discordance between data sharing plans that they submit in trial registration versus publications." with:
More than 40% of published trials (160/376, 42.6%) have discordance between data sharing plans that they submit in trial registration versus publications.
The corrected wording matches the Discussion's own statement ("over 40% were discordant"). Optionally, the unlabeled first sub-column of Table 3 could be headed "No" to match "Yes(n = 216)".
Recomputation script (python recompute.py.txt)
"""Recompute the registration-vs-publication data sharing concordance in PMC12403310
(BMC Med 2025; doi:10.1186/s12916-025-04328-z) from the paper's own printed counts,
and redraw fig1/fig2 in this file's folder.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
Inputs are copied from the published article (Abstract, Methods, Results, Table 2,
Table 3, Discussion) and from Additional file 1 (Tables S6 and S7).
"""
import os
import numpy as np
from scipy import stats
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
# Table 2: rows = plan to share in registration, columns = plan to share in publication
YY, YN, NY, NN = 155, 13, 147, 61
N_REP = 376
ABSTRACT_K, ABSTRACT_PCT = 216, 57.4 # "Over half (216/376, 57.4%) had discordance"
# Additional file 1, Table S7 (same cell order: Yes/Yes, Yes/No, No/Yes, No/No)
S7A = (155, 13, 92, 38) # after removing 78 trials with undecided/missing registered plans
S7B = (210, 36, 92, 38) # undecided/missing registered plans treated as "Yes"
S7_REPORTED_CONC = {"S7a": 64.7, "S7b": 65.9} # Results, sensitivity paragraph
REG_YES, REG_N = 171, 383 # Abstract: 44.6% (171/383)
def pct(k, n):
return 100.0 * k / n
def wilson(k, n, z=1.959964):
p = k / n
d = 1 + z**2 / n
c = (p + z**2 / (2 * n)) / d
h = z * np.sqrt(p * (1 - p) / n + z**2 / (4 * n**2)) / d
return 100 * (c - h), 100 * (c + h)
print("=== 1. Table 2 cells and margins ===")
N = YY + YN + NY + NN
print(f"N = {N} (reported {N_REP})")
for lab, k in [("Yes/Yes", YY), ("Yes/No", YN), ("No/Yes", NY), ("No/No", NN)]:
print(f" {lab:8s} {k:4d} {pct(k, N):5.1f}%")
print(f" registration Yes {YY+YN} ({pct(YY+YN, N):.1f}%), No {NY+NN} ({pct(NY+NN, N):.1f}%)")
print(f" publication Yes {YY+NY} ({pct(YY+NY, N):.1f}%), No {YN+NN} ({pct(YN+NN, N):.1f}%)")
print("\n=== 2. Concordance and discordance (Methods definitions) ===")
conc, disc = YY + NN, YN + NY
lo_c, hi_c = wilson(conc, N)
lo_d, hi_d = wilson(disc, N)
print(f"concordant = Yes/Yes + No/No = {YY} + {NN} = {conc}/{N} = {pct(conc, N):.1f}% "
f"(Wilson 95% CI {lo_c:.1f} to {hi_c:.1f})")
print(f"discordant = Yes/No + No/Yes = {YN} + {NY} = {disc}/{N} = {pct(disc, N):.1f}% "
f"(Wilson 95% CI {lo_d:.1f} to {hi_d:.1f})")
print(f"Abstract figure {ABSTRACT_K}/{N_REP} = {pct(ABSTRACT_K, N_REP):.1f}% "
f"-> equals the concordant count: {ABSTRACT_K == conc}")
print(f"Is discordance more than half? {disc > N/2} (half of {N} = {N/2:.0f})")
print(f"Discordance overstated by {ABSTRACT_PCT - round(pct(disc, N), 1):.1f} percentage points "
f"(unrounded {pct(conc, N) - pct(disc, N):.2f})")
print(f"Discussion 'over 40% were discordant': {pct(disc, N):.1f}% > 40% -> {pct(disc, N) > 40}")
print("Table 3 column sizes printed: n = 160 and 'Yes' n = 216 "
f"-> discordant {disc}, concordant {conc}")
print("\n=== 3. McNemar test on the discordant cells (reported P < 0.001) ===")
x2 = (YN - NY) ** 2 / (YN + NY)
x2cc = (abs(YN - NY) - 1) ** 2 / (YN + NY)
p_exact = stats.binomtest(YN, YN + NY, 0.5).pvalue
print(f"chi2 = {x2:.1f}, P = {stats.chi2.sf(x2, 1):.1e}; continuity-corrected chi2 = {x2cc:.1f}, "
f"P = {stats.chi2.sf(x2cc, 1):.1e}; exact P = {p_exact:.1e}")
print("\n=== 4. Sensitivity analyses (Additional file 1, Table S7) ===")
sens = {}
for lab, cells in [("S7a", S7A), ("S7b", S7B)]:
a, b, c, d = cells
n = a + b + c + d
cc, dd = a + d, b + c
lo, hi = wilson(dd, n)
sens[lab] = (dd, n, pct(dd, n), lo, hi)
print(f"{lab}: N = {n}; concordant {cc} ({pct(cc, n):.1f}%; paper {S7_REPORTED_CONC[lab]}%"
f"); discordant {dd} ({pct(dd, n):.1f}%, "
f"Wilson 95% CI {lo:.1f} to {hi:.1f})")
print("\n=== 5. Registration-level figure (different denominator, consistent) ===")
print(f"{REG_YES}/{REG_N} = {pct(REG_YES, REG_N):.1f}% (reported 44.6%)")
# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, Rectangle, FancyArrowPatch
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": SEC,
"svg.fonttype": "none", "font.family": "DejaVu Sans"})
def save(fig, name):
fig.savefig(os.path.join(HERE, f"{name}.svg"), facecolor=BG, bbox_inches="tight")
fig.savefig(os.path.join(HERE, f"{name}.png"), facecolor=BG, bbox_inches="tight", dpi=200)
plt.close(fig)
# ---- fig1: abstract/conclusions vs Results/Table 2, with the 2x2 table drawn
fig = plt.figure(figsize=(7.2, 4.3), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100); ax.set_ylim(0, 60); ax.axis("off"); ax.set_facecolor(BG)
# left panel: what the abstract and conclusions say
ax.add_patch(FancyBboxPatch((1.5, 6), 38.5, 48, boxstyle="round,pad=0.4,rounding_size=1.5",
fc="white", ec=ORANGE, lw=1.5))
ax.text(3.5, 51.2, "Abstract (Results, Conclusions)\n+ main Conclusions section",
fontsize=10.5, weight="bold", va="top")
ax.text(3.5, 41.5, "As reported", color=ORANGE, fontsize=10, weight="bold", va="top")
ax.text(3.5, 37, "“Over half (216/376, 57.4%)\nhad discordance between\nregistration and publications.”",
fontsize=10, va="top", style="italic")
ax.text(3.5, 22.5, "“More than half of published\ntrials have data sharing\ndiscordance …”",
fontsize=10, va="top", style="italic")
ax.text(3.5, 9.5, "Reads 216 (57.4%) as discordant", fontsize=10, color=SEC, va="center")
# right panel: Table 2 as a 2x2 grid (rows = registration, columns = publication)
x0, y0, cw, ch = 60, 14, 16, 12
ax.text(59, 53.2, "Results text + Table 2 (same 376 trials)", fontsize=10.5, weight="bold", va="top")
ax.text(x0 + cw, 47.6, "Publication: plan to share?", ha="center", fontsize=10, color=SEC)
ax.text(x0 + cw / 2, 40.5, "Yes", ha="center", fontsize=10, color=SEC)
ax.text(x0 + 1.5 * cw, 40.5, "No", ha="center", fontsize=10, color=SEC)
ax.text(x0 + 2 * cw + 1.2, y0 + 1.5 * ch, "Yes", ha="left", va="center", fontsize=10, color=SEC)
ax.text(x0 + 2 * cw + 1.2, y0 + 0.5 * ch, "No", ha="left", va="center", fontsize=10, color=SEC)
ax.text(x0 + 2 * cw + 6.3, y0 + ch, "Registration: plan to share?", ha="center", va="center",
rotation=90, fontsize=10, color=SEC)
cells = [(0, 1, "Yes/Yes", YY, True), (1, 1, "Yes/No", YN, False),
(0, 0, "No/Yes", NY, False), (1, 0, "No/No", NN, True)]
for cx, cy, lab, k, is_conc in cells:
ax.add_patch(Rectangle((x0 + cx * cw, y0 + cy * ch), cw, ch, fc="#e6eef9" if is_conc else "white",
ec=BLUE if is_conc else SEC, lw=2 if is_conc else 1,
ls="-" if is_conc else (0, (3, 2))))
ax.text(x0 + cx * cw + cw / 2, y0 + cy * ch + ch * 0.62, f"{k}", ha="center", va="center",
fontsize=12, weight="bold", color=BLUE if is_conc else TXT)
ax.text(x0 + cx * cw + cw / 2, y0 + cy * ch + ch * 0.27, f"{lab} ({pct(k, N):.1f}%)",
ha="center", va="center", fontsize=10, color=SEC)
ax.text(x0 + cw, 10, f"Concordant (solid): {YY} + {NN} = {conc} ({pct(conc, N):.1f}%)",
ha="center", va="center", fontsize=10, color=BLUE, weight="bold")
ax.text(x0 + cw, 6.3, f"Discordant (dashed): {YN} + {NY} = {disc} ({pct(disc, N):.1f}%)",
ha="center", va="center", fontsize=10, color=TXT)
ax.text(x0 + cw, 2.4, "Results: \u201c216 (57.4%) had concordance\u201d",
ha="center", va="center", fontsize=10, color=SEC, style="italic")
# connector naming the conflict
arr = FancyArrowPatch((41.2, 33), (52.0, 10.6), connectionstyle="arc3,rad=-0.3",
arrowstyle="-|>", mutation_scale=14, lw=2, color=TXT)
ax.add_patch(arr)
ax.text(49.6, 41.5, "same 216 trials:\n\u201cdiscordance\u201d in\nthe abstract,\n\u201cconcordance\u201d in\nResults & Table 2",
ha="center", va="center", fontsize=10, color=TXT)
save(fig, "fig1")
# ---- fig2: share of trials discordant, reported vs recomputed, 0-100% axis
rows = [
("Abstract / Conclusions\n(as reported)", ABSTRACT_PCT, None, ORANGE, "o"),
(f"Table 2, main analysis\n({disc}/{N}, recomputed)", pct(disc, N), (lo_d, hi_d), BLUE, "o"),
(f"Table S7a, undecided removed\n({sens['S7a'][0]}/{sens['S7a'][1]}, recomputed)",
sens["S7a"][2], sens["S7a"][3:], BLUE, "s"),
(f"Table S7b, undecided as Yes\n({sens['S7b'][0]}/{sens['S7b'][1]}, recomputed)",
sens["S7b"][2], sens["S7b"][3:], BLUE, "s"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.4), facecolor=BG)
ax.set_facecolor(BG)
ys = np.arange(len(rows))[::-1]
for y, (lab, v, ci, col, mk) in zip(ys, rows):
if ci is not None:
ax.plot(ci, [y, y], color=col, lw=2, solid_capstyle="butt")
ax.plot(v, y, mk, color=col, ms=9, zorder=3)
txt = f"{v:.1f}%" if ci is None else f"{v:.1f}% ({ci[0]:.1f}–{ci[1]:.1f})"
ax.text((ci[1] if ci else v) + 2, y, txt, va="center", fontsize=10, color=col)
ax.axvline(50, color=SEC, lw=0.8)
ax.text(50.8, ys[0] + 0.5, "half of trials", fontsize=10, color=SEC, va="center")
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xlim(0, 100); ax.set_ylim(-0.6, len(rows) - 0.2)
ax.set_xlabel("Trials with discordant registered vs published data sharing plans (%)")
ax.xaxis.grid(True, color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.plot([], [], "o", color=ORANGE, ms=8, label="reported in paper")
ax.plot([], [], "o", color=BLUE, ms=8, label="recomputed from the paper's own counts (Wilson 95% CI)")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=2, fontsize=10, frameon=False)
save(fig, "fig2")
print("\nfigures written to", HERE)
Abstract labels the age-and-sex adjusted all-cancer hazard ratio as the multivariable adjusted estimate
BMJ (2017) · doi:10.1136/bmj.j108 · PMC5266623
Summary
The abstract and the Discussion give the hazard ratio for death from all cancers combined, for the most distressed group (GHQ-12 score 7-12) against the least distressed (0-6), as "multivariable adjusted hazard ratio 1.32, 95% confidence interval 1.18 to 1.48". In the paper's own Table 5, 1.32 (1.18 to 1.48) is the age and sex adjusted estimate (4353 deaths, n=163 363), and the Results text and Figures 2 and 3 describe it the same way. The multivariable estimate in Table 5 is 1.26 (1.11 to 1.42) (3800 deaths, n=146 008). The other six hazard ratios in the same abstract sentence all equal Table 5 multivariable rows exactly, so only the all-cancer figure comes from the other row. The association stays positive and statistically significant, but the fully adjusted excess risk is 26% rather than 32%. The abstract does not show that adjustment lowered the all-cancer estimate, and the Discussion uses the mislabelled figure to call an earlier meta-analysis estimate "of markedly lower magnitude".
What the paper reports
Abstract, Results. The all-cancer estimate is the first of seven, all under one "After adjustment for ..." clause:
After adjustment for age, sex, education, socioeconomic status, body mass index (BMI), and smoking and alcohol intake, and with reverse causality (by left censoring) and missing data (by imputation) taken into account, relative to people in the least distressed group (GHQ-12 score 0-6), death rates in the most distressed group (score 7-12) were consistently raised for cancer of all sites combined (multivariable adjusted hazard ratio 1.32, 95% confidence interval 1.18 to 1.48) and cancers not related to smoking (1.45, 1.23 to 1.71), as well as carcinoma of the colorectum (1.84, 1.21 to 2.78), prostate (2.42, 1.29 to 4.54), pancreas (2.76, 1.47 to 5.19), oesophagus (2.59, 1.34 to 5.00), and for leukaemia (3.86, 1.42 to 10.5).
Discussion, "Comparison with other studies":
the aggregated result for depression and cancer for all malignancies across the 13 studies included unity (relative risk 1.12, 95% confidence interval 0.99 to 1.26). This was of markedly lower magnitude than our estimate between psychological distress and overall cancer (multivariable adjusted hazard ratio 1.32, 95% confidence interval 1.18 to 1.48).
Results, paragraph beginning "During a mean (SD) follow-up":
In the 16 studies in aggregate, after adjustment for age and sex, higher levels of distress were associated with a 32% greater risk of total cancer mortality (hazard ratio 1.32, 95% confidence interval 1.18 to 1.48).
Table 5, "All cancers". The columns are deaths, number of people, distress score 0, 1-3, 4-6 and 7-12, P value for trend, and distress score 7-12 v 0-6:
Age and sex | 4353 | 163 363 | 1 (ref) | 0.99 (0.92 to 1.08) | 1.05 (0.91 to 1.20) | 1.31 (1.16 to 1.49) | <0.001 | 1.32 (1.18 to 1.48)
Multivariable† | 3800 | 146 008 | 1 | 0.95 (0.88 to 1.03) | 0.93 (0.78 to 1.10) | 1.22 (1.07 to 1.38) | 0.04 | 1.26 (1.11 to 1.42)
Table 5 footnote:
†Hazard ratio adjusted for age, sex (except in single sex analyses), BMI, educational attainment, smoking status, and frequency of alcohol consumption.
Methods, "Statistical analyses", which defines the two models:
Hazard ratios were minimally adjusted (age and sex only) and maximally adjusted (age, sex, BMI, educational attainment, smoking status, and frequency of alcohol consumption).
Figure 3 legend. In the typeset figure, the "All cancers combined" row reads 1.32 (1.18 to 1.48) under "adjusted for age and sex" and 1.26 (1.11 to 1.42) under "multivariable adjusted":
Hazard ratios are for psychological distress score of 7-12 (most distressed) relative to 0-6, and are age and—except in single sex analyses—sex adjusted (I2=15%), or multivariable adjusted (age, sex, BMI, educational attainment, smoking status, and alcohol consumption; I2=37%)
Figure 2 legend. The pooled "Overall" row of this forest plot is 4353 deaths, 1.32 (1.18 to 1.48):
Hazard ratios (adjusted for age and sex) are for psychological distress score of 7-12 (most distressed) relative to 0-6.
What the paper's own numbers show
Method. I copied the seven hazard ratios and 95% CIs from the abstract's Results sentence. For the same seven outcomes I copied both Table 5 rows, age (and sex) and multivariable, from the column "Distress score (7-12 v 0-6)". I then checked which row each abstract value equals, digit for digit. To show that the two all-cancer rows are separate estimates and not rounding variants of one number, I computed the geometric (log-scale) midpoint of each CI and the standard error of the log hazard ratio, (ln upper − ln lower)/(2 × 1.96). Finally, I repeated the Discussion's comparison with the earlier meta-analysis for each all-cancer estimate. That comparison is a z test on the difference of log ratios, treating the two estimates as independent. It is illustrative only: one estimate is a hazard ratio for distress and death, the other a relative risk for major depression and cancer occurrence. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| All cancers, labelled "multivariable adjusted" | Abstract, Results | 1.32 (1.18 to 1.48) | Equals the Table 5 age and sex row in all three numbers. The Table 5 multivariable row is 1.26 (1.11 to 1.42) |
| All cancers, labelled "multivariable adjusted" | Discussion, "Comparison with other studies" | 1.32 (1.18 to 1.48) | Same as above |
| All cancers, labelled "after adjustment for age and sex" | Results text | 1.32 (1.18 to 1.48) | Equals the Table 5 age and sex row |
| All cancers, age and sex | Table 5 | 1.32 (1.18 to 1.48); 4353 deaths; n=163 363; P for trend <0.001 | CI log-midpoint 1.322; SE(log HR) 0.0578; z = 4.80 |
| All cancers, multivariable | Table 5 | 1.26 (1.11 to 1.42); 3800 deaths; n=146 008; P for trend 0.04 | CI log-midpoint 1.255; SE(log HR) 0.0628; z = 3.68 |
| Other six abstract hazard ratios (not related to smoking, colorectum, prostate, pancreas, oesophagus, leukaemia) | Abstract vs Table 5 | 1.45, 1.84, 2.42, 2.76, 2.59, 3.86, each with its CI | 6 of 6 equal the Table 5 multivariable rows exactly. None equals the age and sex rows (1.38, 1.75, 2.29, 2.52, 2.42, 2.89) |
| Effect of adjustment | Table 5 | — | All cancers: 1.32 → 1.26 (down). The other six outcomes: all go up (for example leukaemia 2.89 → 3.86) |
| Excess all-cancer mortality, most v least distressed | Abstract (implied) vs Table 5 multivariable | 32% | 26%. The reported excess is 1.23 times the multivariable excess, and the multivariable excess is 19% smaller |
| Comparison with the earlier meta-analysis, RR 1.12 (0.99 to 1.26) | Discussion | 1.32 / 1.12 = 1.18; excess 32% v 12% | With 1.32: z = 1.95, P = 0.052. With 1.26: ratio 1.13, excess 26% v 12%, z = 1.34, P = 0.18 |
The two all-cancer estimates are each internally consistent: each printed hazard ratio sits at the log-midpoint of its own CI. They also come from different samples, with 553 deaths and 17 355 people fewer in the multivariable model. The paper states the reason. Results, paragraph beginning "Table 5 shows the four categories":
With different models being based on different analytical samples because of missing covariates, we recomputed these effects estimates in a non-missing dataset—that is, the same sample size in both models—and our results were unchanged.
So the paper attributes the different sample sizes to missing covariates, and it reports that re-estimating both models in the same (non-missing) sample left the results "unchanged". It gives no figures for that analysis, so the share of the 1.32 v 1.26 gap due to the extra covariates rather than the change of sample cannot be read from the paper.
Figure
Figure 1. Where the all-cancer estimate appears in the paper. Left: the abstract's wording, shortened at the ellipsis; the Discussion uses the same label and numbers. The estimate is in orange. Right: the two Table 5 rows for all cancers with their deaths and sample sizes. The solid orange arrow joins the abstract value to the Table 5 row that prints the identical numbers, which is the age and sex row. The dashed blue arrow joins the abstract's label, "multivariable adjusted", to the row that label names, which prints a different estimate, 1.26 (1.11 to 1.42). The quoted text and numbers are from the paper; the arrows and annotations are mine.
Figure 2. The seven hazard ratios (95% CI, log scale) in the abstract's Results sentence (orange filled circles), shown with the Table 5 multivariable estimate (blue diamonds) and the Table 5 age and sex estimate (open gray circles) for the same outcome. The thin vertical line marks a hazard ratio of 1. The label on the right of each row says which Table 5 row the abstract value equals. For the six site-specific or subgroup outcomes it is the multivariable row. For all cancers combined it is the age and sex row. All plotted values are from the paper's abstract and Table 5 as printed.
Why it matters
- The abstract gives the wrong model for its first and broadest estimate. The sentence says the all-cancer hazard ratio is adjusted for education, socioeconomic status, BMI, smoking and alcohol. Table 5 shows it is adjusted for age and sex only. A reader of the abstract alone gets a fully adjusted excess risk of 32% where the paper's multivariable model gives 26%, and a CI of 1.18 to 1.48 in place of 1.11 to 1.42. The abstract is also the text reproduced in PubMed and in secondary coverage.
- It hides the direction in which adjustment moved this estimate. Because every abstract value is presented as multivariable, a reader cannot tell that, for all cancers combined, adding covariates lowered the estimate from 1.32 to 1.26 and the P for trend from <0.001 to 0.04. For the other six outcomes in the sentence, the multivariable estimates are higher than the age and sex ones. So all cancers is the one outcome in the sentence where adjustment lowered the estimate, and it is the one where the age and sex value is shown. This matters for a reader weighing confounding by health behaviours, which the paper discusses.
- The Discussion uses the figure in an argument. The Discussion calls the earlier meta-analysis estimate, RR 1.12 (0.99 to 1.26), "of markedly lower magnitude" than "our estimate ... (multivariable adjusted hazard ratio 1.32 ...)". Against 1.32 the ratio is 1.18 and a simple test of the difference gives z = 1.95 (P = 0.052). Against the multivariable 1.26 the ratio is 1.13, with z = 1.34 (P = 0.18), and the two CIs overlap from 1.11 to 1.26. The contrast the Discussion draws is weaker than stated.
- Scope. The all-cancer association keeps its direction and its statistical significance: 1.26 (1.11 to 1.42) still excludes 1. The title, the abstract's Conclusion and the Discussion's "Principal findings" concern site-specific cancers, and the abstract reports those correctly. The error is a mislabelled headline estimate. It does not reach the paper's central site-specific findings.
Explanations considered
- The 1.32 comes from another multivariable analysis, such as the imputed-data or left-censored sensitivity analyses the abstract mentions. This does not account for the numbers. Those analyses are in the web appendices (appendix 1 table C; appendix 2 fig A), which are not deposited in PMC or Europe PMC and which I could not inspect. Four things in the paper argue against it. (a) The other six estimates in the same sentence, under the same "left censoring ... imputation" wording, equal the non-imputed Table 5 multivariable rows exactly. (b) The Results say the imputed meta-analysis "gave similar results to those seen with the non-missing dataset as reported in the present paper", which identifies the reported estimates as the non-imputed ones. (c) The Results say that in the left-censored analysis "most of the associations between distress and cancer were largely unaffected (fig A in appendix 2)", which presents it as a check rather than the source of the headline figure. (d) 1.32 (1.18 to 1.48) matches the Table 5 age and sex row in all three numbers, and the Results text and Figures 2 and 3 label it age and sex adjusted. Another model reproducing all three numbers to two decimals would be a coincidence. A related point: the abstract says all seven estimates take left censoring and imputation into account, but the Table 5 footnote, which defines the rows six of them come from, lists covariates only.
- A model with additional covariates (appendix 1 table B). I could not inspect that table. The Results say only that, in this analysis, "The strength of the relation between distress and total cancer was little changed", and give no figure for it. The abstract's covariate list is close to the main model (Table 5 footnote, Methods, Figure 3 legend) but not identical: the abstract names "socioeconomic status" as well as education, while those three places list educational attainment only. The Results describe the main model's covariates as including "socioeconomic position (education)", so the abstract's "socioeconomic status" corresponds to education in the main model. Table B's extra covariates do include one socioeconomic measure, area based deprivation, but the abstract does not mention the other three (physical activity, C reactive protein, quantity of alcohol consumed), and the six other estimates in the same sentence equal the main-model rows of Table 5.
- The same-sample re-analysis reported in the Results. The paper says it "recomputed these effects estimates in a non-missing dataset—that is, the same sample size in both models—and our results were unchanged". This does not account for 1.32 being labelled multivariable. In that analysis the multivariable model is fitted to the complete-case sample, and the multivariable estimate in the complete-case sample is the 1.26 (1.11 to 1.42) already shown in Table 5 (3800 deaths, n=146 008). Refitting in the complete-case sample changes the age and sex estimate, not the multivariable one, so the re-analysis gives no route by which the multivariable estimate becomes 1.32 (1.18 to 1.48), the numbers of the full-sample age and sex row.
- Rounding, or two presentations of one estimate. Ruled out. The two CIs have log-scale midpoints of 1.322 and 1.255, each matching its own printed hazard ratio, and they come from different samples (4353 deaths in 163 363 people v 3800 deaths in 146 008).
- The error is in Table 5, not in the abstract. Not supported. The Table 5 all-cancer rows agree with the Results text, with Figure 2 (pooled age and sex estimate 1.32 (1.18 to 1.48), 4353 deaths) and with Figure 3 (1.32 under age and sex, 1.26 under multivariable).
- An artefact of the PMC text conversion. Ruled out. The same abstract and Discussion wording and the same Table 5 cells appear in the publisher's typeset PDF (see Status). No minus signs, superscripts or special decimal marks are involved.
Status
- Version of record. I checked the PMC archived text (PMC5266623, via
src/show.py) and the publisher's typeset PDF ("the bmj | BMJ 2017;356:j108", 11 pages, created 19 January 2017), held in the UCL Discovery repository (eprint 1537638). Their abstract and Discussion wording and their Table 5 cells are identical. Text extracted from the PDF's Figure 2 and Figure 3 gives the values described above. Every quotation in this write-up was confirmed verbatim against the PMC text. The live bmj.com page returned HTTP 403 to automated access in an earlier check in this pipeline, which also found the same abstract sentence in Europe PMC full text and in PubMed (PMID 28122812). - Supplements. The web appendices (appendix 1 tables A-D; appendix 2 fig A) are not deposited in PMC or Europe PMC, and bmj.com was not reachable, so they were not inspected. The Europe PMC supplementary bundle holds only the four main figures.
- Corrections. As of 2026-10-23, I found no correction, erratum, retraction or expression of concern. Crossref shows no
update-to/updated-byrelation for 10.1136/bmj.j108, and a Crossref query for works updating this DOI returns 0 results.src/known.pylists no Crossref, PMC or Europe PMC notices. Europe PMC's comment/correction list for PMID 28122812 has two "Comment in" entries (Cancer 2017;123(13):2389 and MMW Fortschr Med 2017;159(15):38) and no correction. An earlier web search in this pipeline found no erratum. I could not repeat the web search in this session because the search budget was used up. - PubPeer. The PubPeer publications API, queried for this DOI on 2026-10-23, returned no feedback (
"feedbacks":[]). - Separate minor observation (not part of this finding). For liver cancer, the multivariable estimate is printed as 4.24 (0.66 to 27.0) in Table 5 but as 4.24 (0.66 to 8.52) in the Figure 3 text of the typeset PDF. The log-midpoint of 0.66 to 27.0 is 4.22, consistent with 4.24. The log-midpoint of 0.66 to 8.52 is 2.37, which is not. 8.52 is the upper limit of the liver age and sex CI.
Suggested correction
- In the abstract's Results, replace "cancer of all sites combined (multivariable adjusted hazard ratio 1.32, 95% confidence interval 1.18 to 1.48)" with "cancer of all sites combined (multivariable adjusted hazard ratio 1.26, 95% confidence interval 1.11 to 1.42)". Alternatively, keep 1.32 (1.18 to 1.48) and label it "age and sex adjusted", which would also need the sentence's opening clause to be reworded.
- Make the same change in the Discussion, "Comparison with other studies", and reconsider describing the earlier meta-analysis estimate, 1.12 (0.99 to 1.26), as "of markedly lower magnitude".
- Clarify in the abstract whether the reported estimates are from the main (complete-case, not left-censored) analysis, as Table 5 and the Results indicate, rather than "with reverse causality (by left censoring) and missing data (by imputation) taken into account".
- Separately, reconcile the liver multivariable upper confidence limit in Figure 3 (8.52) with Table 5 (27.0).
Recomputation script (python recompute.py.txt)
"""Recompute the all-cancer hazard ratios of PMC5266623
(BMJ 2017;356:j108, doi:10.1136/bmj.j108).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PubMed Central:
Abstract, Results: seven hazard ratios (GHQ-12 score 7-12 v 0-6), all introduced as
adjusted for age, sex, education, SES, BMI, smoking and alcohol
Table 5, last column "Distress score (7-12 v 0-6)": the "Age and sex" (or "Age") row and the
"Multivariable" row for each outcome, with deaths and number of people
Discussion, "Comparison with other studies": prior meta-analysis RR 1.12 (0.99 to 1.26)
"""
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from scipy.stats import norm # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)
# ---------------------------------------------------------------- inputs (from the paper)
# Table 5: outcome -> (minimal row (HR, lo, hi), multivariable row (HR, lo, hi))
# For prostate the minimal row is labelled "Age" (single-sex outcome).
T5 = {
"All cancers": ((1.32, 1.18, 1.48), (1.26, 1.11, 1.42)),
"Not related to smoking": ((1.38, 1.19, 1.60), (1.45, 1.23, 1.71)),
"Colorectal": ((1.75, 1.21, 2.53), (1.84, 1.21, 2.78)),
"Prostate": ((2.29, 1.36, 3.86), (2.42, 1.29, 4.54)),
"Pancreas": ((2.52, 1.47, 4.32), (2.76, 1.47, 5.19)),
"Oesophagus": ((2.42, 1.36, 4.32), (2.59, 1.34, 5.00)),
"Leukaemia": ((2.89, 1.32, 6.31), (3.86, 1.42, 10.5)),
}
# Table 5, All cancers: deaths and number of people in each model
T5_ALL_N = {"age_sex": (4353, 163363), "multivariable": (3800, 146008)}
# Abstract, Results (the first is explicitly "multivariable adjusted hazard ratio")
ABSTRACT = {
"All cancers": (1.32, 1.18, 1.48),
"Not related to smoking": (1.45, 1.23, 1.71),
"Colorectal": (1.84, 1.21, 2.78),
"Prostate": (2.42, 1.29, 4.54),
"Pancreas": (2.76, 1.47, 5.19),
"Oesophagus": (2.59, 1.34, 5.00),
"Leukaemia": (3.86, 1.42, 10.5),
}
# Discussion, "Comparison with other studies": the same all-cancer figure, "multivariable adjusted"
DISCUSSION_ALL = (1.32, 1.18, 1.48)
# Results p4: "after adjustment for age and sex ... (hazard ratio 1.32, 95% CI 1.18 to 1.48)"
RESULTS_P4_AGESEX = (1.32, 1.18, 1.48)
# Discussion: prior meta-analysis (depression and cancer incidence), "relative risk 1.12, 0.99 to 1.26"
PRIOR_META = (1.12, 0.99, 1.26)
def log_se(t):
_, lo, hi = t
return (math.log(hi) - math.log(lo)) / (2 * Z)
def midpoint(t):
_, lo, hi = t
return math.exp((math.log(lo) + math.log(hi)) / 2)
def num(x):
return f"{x:.1f}" if x >= 10 else f"{x:.2f}"
def fmt(t):
return f"{num(t[0])} ({num(t[1])} to {num(t[2])})"
print("=== 1. Which Table 5 row does each abstract hazard ratio come from? ===")
print(f"{'outcome':24s} {'abstract':20s} {'T5 age(/sex)':20s} {'T5 multivariable':20s} matches")
MATCH = {}
for k, v in ABSTRACT.items():
a, m = T5[k]
tag = "multivariable row" if v == m else ("AGE AND SEX ROW" if v == a else "neither")
MATCH[k] = tag
print(f"{k:24s} {fmt(v):20s} {fmt(a):20s} {fmt(m):20s} {tag}")
n_mv = sum(t == "multivariable row" for t in MATCH.values())
print(f"-> {n_mv} of {len(ABSTRACT)} abstract estimates equal the multivariable row; "
f"the all-cancer estimate equals the age and sex row.")
print("Change from age(/sex) to multivariable estimate, by outcome:")
for k in ABSTRACT:
a, m = T5[k]
print(f" {k:24s} {a[0]:.2f} -> {m[0]:.2f} ({'up' if m[0] > a[0] else 'down'})")
print("Discussion figure equals Table 5 age and sex row:", DISCUSSION_ALL == T5["All cancers"][0])
print("Results p4 (age and sex) figure equals Table 5 age and sex row:",
RESULTS_P4_AGESEX == T5["All cancers"][0])
print("\n=== 2. Are the two all-cancer rows separate, internally consistent estimates? ===")
AS, MV = T5["All cancers"]
for lab, t in (("age and sex", AS), ("multivariable", MV)):
se = log_se(t)
zval = math.log(t[0]) / se
p = 2 * norm.sf(abs(zval))
print(f"{lab:14s} HR {t[0]:.2f}: log SE {se:.4f}; geometric midpoint of CI {midpoint(t):.3f}; "
f"z = {zval:.2f}; two-sided P ~ {p:.1e}")
print(f"deaths/people: age and sex {T5_ALL_N['age_sex']}, multivariable {T5_ALL_N['multivariable']}; "
f"difference {T5_ALL_N['age_sex'][0] - T5_ALL_N['multivariable'][0]} deaths, "
f"{T5_ALL_N['age_sex'][1] - T5_ALL_N['multivariable'][1]} people")
print("\n=== 3. Size of the overstatement ===")
ex_rep, ex_mv = AS[0] - 1, MV[0] - 1
print(f"excess risk: reported {ex_rep*100:.0f}% v multivariable {ex_mv*100:.0f}%; "
f"reported excess is {ex_rep/ex_mv:.2f} x the multivariable excess; "
f"multivariable excess is {(1 - ex_mv/ex_rep)*100:.0f}% smaller")
print(f"ratio of hazard ratios 1.32/1.26 = {AS[0]/MV[0]:.3f}")
print(f"lower CI bound: {AS[1]:.2f} v {MV[1]:.2f}; upper: {AS[2]:.2f} v {MV[2]:.2f}")
print("\n=== 4. Discussion contrast with the prior meta-analysis (illustrative only) ===")
# Naive two-sample z test on the log scale, treating the estimates as independent. The two
# studies differ in exposure (distress v major depression) and outcome (mortality v incidence);
# this only shows how much the contrast depends on which all-cancer figure is used.
se_prior = log_se(PRIOR_META)
print(f"prior meta-analysis RR 1.12: log SE {se_prior:.4f}; excess {(PRIOR_META[0]-1)*100:.0f}%")
for lab, t in (("as reported (1.32)", AS), ("multivariable (1.26)", MV)):
d = math.log(t[0]) - math.log(PRIOR_META[0])
se = math.sqrt(log_se(t) ** 2 + se_prior ** 2)
zval = d / se
print(f"{lab:22s}: ratio to 1.12 = {t[0]/PRIOR_META[0]:.3f}; excess {(t[0]-1)*100:.0f}% v 12% "
f"({(t[0]-1)/(PRIOR_META[0]-1):.1f} x); z for difference = {zval:.2f}, "
f"P = {2*norm.sf(abs(zval)):.3f}; CIs overlap from {max(t[1], PRIOR_META[1]):.2f} "
f"to {min(t[2], PRIOR_META[2]):.2f}")
print("\n=== 5. Separate minor observation: liver, multivariable (Table 5 v typeset Figure 3) ===")
LIVER_T5_MV = (4.24, 0.66, 27.0) # Table 5 (PMC and typeset PDF)
LIVER_F3_MV = (4.24, 0.66, 8.52) # Figure 3 text in the typeset PDF
LIVER_T5_AS = (2.88, 0.97, 8.52) # Table 5 age and sex row
for lab, t in (("Table 5 multivariable", LIVER_T5_MV), ("Figure 3 multivariable", LIVER_F3_MV),
("Table 5 age and sex", LIVER_T5_AS)):
print(f"{lab:24s} {fmt(t):22s} geometric midpoint of CI {midpoint(t):.2f} (printed HR {t[0]})")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#8d8c88"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": "#c9c8c4",
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# ---- Figure 1: abstract/discussion label v Table 5 rows
fig = plt.figure(figsize=(7.2, 4.7))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.7)
ax.axis("off")
def panel(x, y, w, h, title, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.14, y + h - 0.18, title, fontsize=11, fontweight="bold", va="top", color=TXT)
# Left panel: abstract and discussion
panel(0.15, 1.2, 2.95, 3.35, "Abstract, Results\n(repeated in the Discussion)", ORANGE)
ax.text(0.29, 3.62,
"\u201cAfter adjustment for age, sex,\neducation, socioeconomic status,\nbody mass index (BMI), and\n"
"smoking and alcohol intake \u2026\ncancer of all sites combined\n(multivariable adjusted hazard\nratio",
fontsize=10, va="top", color=SEC, linespacing=1.3)
ax.text(0.29, 1.88, f"{AS[0]:.2f}, 95% CI {AS[1]:.2f} to {AS[2]:.2f})\u201d",
fontsize=11.5, fontweight="bold", color=ORANGE, va="center")
ax.text(0.29, 1.5, "label: multivariable adjusted", fontsize=10, color=SEC, va="center", style="italic")
# Right panel: Table 5 rows
panel(4.0, 1.2, 3.05, 3.35, "Table 5, All cancers\n(score 7-12 v 0-6)", "#9b9a96")
rows = [
(3.6, "Age and sex", f"{T5_ALL_N['age_sex'][0]} deaths, n = {T5_ALL_N['age_sex'][1]:,}".replace(",", " "),
fmt(AS), ORANGE),
(2.6, "Multivariable\u2020", f"{T5_ALL_N['multivariable'][0]} deaths, n = {T5_ALL_N['multivariable'][1]:,}"
.replace(",", " "), fmt(MV), BLUE),
]
for y, lab, nn, val, col in rows:
ax.text(4.14, y, lab, fontsize=10.5, fontweight="bold", va="center", color=TXT)
ax.text(4.14, y - 0.27, nn, fontsize=10, va="center", color=SEC)
ax.text(4.14, y - 0.57, val, fontsize=11.5, fontweight="bold", va="center", color=col)
ax.text(4.14, 1.5, "\u2020 age, sex, BMI, education,\n smoking, alcohol", fontsize=10, color=SEC, va="center",
linespacing=1.25)
# Connectors
ax.add_patch(FancyArrowPatch((3.14, 1.95), (3.98, 3.03), arrowstyle="-|>", mutation_scale=14,
color=ORANGE, lw=2, connectionstyle="arc3,rad=-0.15"))
ax.text(3.52, 3.1, "identical\nvalue", fontsize=10, color=ORANGE, ha="center", va="center")
ax.add_patch(FancyArrowPatch((3.14, 1.5), (3.98, 2.03), arrowstyle="-|>", mutation_scale=14,
color=BLUE, lw=2, linestyle=(0, (4, 3))))
ax.text(3.55, 1.38, "row the\nlabel names", fontsize=10, color=BLUE, ha="center", va="top")
ax.text(0.15, 0.55,
"The Results text gives the same 1.32 (1.18 to 1.48) \u201cafter adjustment for age and sex\u201d.\n"
f"The other {n_mv} hazard ratios in the abstract all equal Table 5 multivariable rows (fig 2).",
fontsize=10, color=SEC, va="center", linespacing=1.35)
save(fig, "fig1")
# ---- Figure 2: all seven abstract estimates against both Table 5 rows (log scale)
names = list(ABSTRACT)
fig, ax = plt.subplots(figsize=(7.2, 4.6))
fig.subplots_adjust(left=0.25, right=0.76, top=0.86, bottom=0.14)
off = {"abstract": 0.0, "mv": -0.24, "as": 0.24}
for i, k in enumerate(names):
y = -i
a, m = T5[k]
v = ABSTRACT[k]
for t, dy, col, mk, fc in ((a, off["as"], GRAY, "o", BG), (m, off["mv"], BLUE, "D", BLUE),
(v, off["abstract"], ORANGE, "o", ORANGE)):
ax.plot([t[1], t[2]], [y + dy] * 2, color=col, lw=2, solid_capstyle="butt", zorder=2)
ax.plot(t[0], y + dy, marker=mk, ms=8, mfc=fc, mec=col, mew=2, zorder=3, ls="none")
tag = "= age and sex row" if MATCH[k] != "multivariable row" else "= multivariable row"
ax.text(1.02, y, tag, fontsize=10, va="center", ha="left", transform=ax.get_yaxis_transform(),
color=ORANGE if MATCH[k] != "multivariable row" else SEC,
fontweight="bold" if MATCH[k] != "multivariable row" else "normal")
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.9, 12)
ticks = [1, 1.5, 2, 3, 5, 10]
ax.set_xticks(ticks)
ax.set_xticklabels([str(t) for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks([-i for i in range(len(names))])
ax.set_yticklabels(names)
ax.set_ylim(-len(names) + 0.5, 0.5)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Hazard ratio, distress score 7-12 v 0-6 (log scale; 95% CI)")
h = [plt.Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Abstract (“multivariable adjusted”)"),
plt.Line2D([], [], color=BLUE, marker="D", ms=8, lw=2, label="Table 5, multivariable row"),
plt.Line2D([], [], color=GRAY, marker="o", ms=8, mfc=BG, mew=2, lw=2, label="Table 5, age and sex row")]
fig.legend(handles=h, loc="upper center", ncol=2, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 1.0))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Primary-outcome 95% confidence interval is about 80 times too narrow for the reported failure counts
BMJ Glob Health (2024) · doi:10.1136/bmjgh-2023-014170 · PMC11331853
Summary
This non-inferiority trial compared spinal anaesthesia failure between trained medical officers (MOs) and consultant anaesthesiologists (CAs). It reports 7/231 (3.0%) versus 5/191 (2.6%) failures, a difference of 0.4 percentage points, with a 95% CI of 0.36–0.43% in the Abstract and Results and 0.36 to 0.44 in Table 3. From those counts, the 95% CI for the difference is −2.75 to 3.58 percentage points (Wald) or −3.29 to 3.85 (Newcombe). The printed interval is therefore about 79–90 times too narrow, and it excludes zero although the printed p = 0.7997, which the same counts reproduce exactly. The non-inferiority conclusion survives, because every recomputed upper bound stays below the 5-point margin. But the margin is cleared by about 1.2–1.4 points, not the 4.6 points the printed interval implies, and the emergent-case subgroup interval has the same defect.
What the paper reports
Abstract, Findings:
Spinal anaesthesia failure rate for MOs (7, 3.0%) was non-inferior to those of CA (5, 2.6%); difference in success rate of 0.4% (95% CI=0.36–0.43%; p=0.80).
Results, third paragraph (primary outcome):
There was no statistically significant difference in the primary outcome–delivery of spinal anaesthesia into the intrathecal space in three or fewer attempts (table 3). MOs were unsuccessful in 7 (3.0%) patients, while the CAs were unsuccessful in 5 (2.6%) patients, with a difference in success rate of 0.4% (CI=0.36–0.43%; p=0.7997).
Table 3, "Primary outcome measure: failure to deliver spinal anaesthesia". The columns are: outcome | N | Medical officers (n=231) | Consultant anaesthesiologists (n=191) | % between-group difference (95% CI) | P value. The footer reads "*Fisher’s exact test, otherwise χ2 test."
Failure to deliver spinal anaesthesia in all cases | 422 | 7 (3.0%) | 5 (2.6%) | 0.4 (0.36 to 0.44) | 0.7997
Failure to deliver spinal anaesthesia in emergent cases | 61 | 1 (3.6%) | 2 (6.1%) | -2.5 (-2.92 to -2.08) | 1.0000*
Failure due to inaccessibility of intrathecal space | 422 | 7 (3.0%) | 5 (2.6%) | 0.4 (0.36 to 0.44) | 0.7997
Table 2 gives the emergent-case denominators (columns: MO | CA | P value):
Emergency | 28 (12.2%) | 33 (17.3%) | 0.1435
Abstract, Methods, which defines the margin:
The primary outcome measure was spinal anaesthesia failure with a non-inferiority margin of 5%.
Methods, Statistical analysis, which names the analysis methods:
Patient characteristics, spinal anaesthesia failure rates and incidence of postoperative complications were analysed with T-test and Wilcoxon rank sum for continuous variables, and χ squared and Fisher’s exact tests for discrete variables.
The conclusions that rest on this result. Abstract, Interpretation:
This study demonstrates that failure rates of spinal anaesthesia care provided by trained MOs are non-inferior to care provided by CAs in low-risk surgical patients. This may support policy measures that use task-sharing as a means of expanding anaesthesia care capacity in rural Indian hospitals.
Discussion, fourth paragraph:
We believe this study may facilitate regional and national policy measures on the practical viability of task-sharing.
What the paper's own numbers show
Method. I took the failure counts and arm sizes from Table 3 (7/231 and 5/191) and the emergent-case denominators from Table 2 (28 and 33). Those emergent denominators are the only pair summing to 61 that reproduces the printed 3.6% and 6.1%. For each comparison I computed the difference in failure proportions (MO − CA) and its unpooled standard error, then the 95% CI by three standard methods for a difference of two independent proportions: Wald, Newcombe (hybrid Wilson score) and Agresti–Caffo. I also computed the uncorrected and Yates-corrected chi-square and the Fisher exact p-values, to check that the counts match the printed p-values. From the half-width of each printed interval I derived the standard error it implies under a normal approximation, and the p-value that standard error would imply. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| MO failure rate, all cases | Abstract; Results; Table 3 | 7 (3.0%), n = 231 | 3.03% |
| CA failure rate, all cases | Abstract; Results; Table 3 | 5 (2.6%), n = 191 | 2.62% |
| Difference, MO − CA | Abstract; Results; Table 3 | 0.4 | 0.413 percentage points (pp) |
| p-value, all cases | Table 3; Results (0.80 in Abstract) | 0.7997 (χ²) | 0.7997, uncorrected chi-square (Yates 1.0000; Fisher 1.0000) |
| 95% CI of difference, all cases | Abstract; Results | 0.36–0.43% | Wald −2.75 to 3.58 pp; Newcombe −3.29 to 3.85; Agresti–Caffo −3.06 to 3.71 |
| 95% CI of difference, all cases | Table 3, rows 1 and 3 | 0.36 to 0.44 | as above |
| Width of the 95% CI, all cases | Table 3; Abstract | 0.08 (Table 3); 0.07 (Abstract) | 6.33 pp (Wald): 79 times the Table 3 width and 90 times the Abstract width |
| Standard error, all cases | not reported | 0.0204 pp, implied by the Table 3 interval | 1.615 pp (Wald), 79 times larger |
| p-value implied by the printed interval, all cases | derived | z = 0.40 / 0.0204 = 19.6 | p ≈ 1.6 × 10⁻⁸⁵, against the printed 0.7997 |
| Emergent cases, rates and difference | Table 3, row 2; Table 2 | 1 (3.6%) vs 2 (6.1%); −2.5 | 1/28 = 3.57%, 2/33 = 6.06%; −2.49 pp |
| 95% CI, emergent cases | Table 3, row 2 | −2.92 to −2.08 | Wald −13.14 to 8.17 pp; Newcombe −16.35 to 12.32; width 21.31 pp, 25 times the printed 0.84 |
| p-value, emergent cases | Table 3, row 2 | 1.0000 (Fisher) | Fisher 1.0000; the printed interval would imply p ≈ 1.9 × 10⁻³¹ |
| Upper 95% bound vs the 5 pp margin, all cases | Abstract; Methods | 0.43 or 0.44, leaving about 4.6 pp | 3.58 (Wald), 3.71 (Agresti–Caffo), 3.85 (Newcombe), leaving 1.42, 1.29 and 1.15 pp; non-inferiority still holds |
| Upper bound of a two-sided 90% interval (one-sided 5% test) | derived | not reported | 3.07 pp, leaving 1.93 pp |
The counts and the p-value agree exactly, so the counts are not in doubt. Only the intervals are inconsistent with them. Both printed all-cases intervals lie entirely above zero, which would mean a significant difference in favour of CAs, while the stated p = 0.80 and the text's "no statistically significant difference" say the opposite. The emergent-case interval lies entirely below zero, which would mean MOs had significantly fewer failures, while its printed p is 1.0000.
Figure
Each row shows a point estimate and its 95% confidence interval for the difference in failure rate between medical officers and consultant anaesthesiologists (MO − CA, percentage points), on a single linear axis. Orange rows are the paper's printed values: 0.4 (0.36 to 0.43) in the Abstract and Results text, 0.4 (0.36 to 0.44) in Table 3, and −2.5 (−2.92 to −2.08) for emergent cases. All three printed intervals are narrower than the plotted dot. Blue rows are recomputed from the paper's own counts (7/231 vs 5/191; 1/28 vs 2/33) with the Wald and Newcombe methods. The thin solid line marks no difference (0), and the dashed line marks the 5-point non-inferiority margin from the Methods. The recomputed all-cases intervals span zero, as the printed p = 0.7997 requires, and still end below the margin. The recomputed emergent-case interval is roughly 21 points wide and crosses both zero and the margin.
Why it matters
The confidence interval's upper bound is the statistic on which a non-inferiority trial's conclusion rests, and this one appears in the Abstract, the Results text and Table 3. As printed, it suggests that the difference in failure rates is known to within about ±0.04 percentage points. The counts support only about ±3.2 points, so the data are compatible with MOs having a failure rate up to about 3.6–3.9 points higher than CAs. Non-inferiority at the 5-point margin still holds under every standard method I checked. But the evidence is much less decisive than the printed interval suggests, and a reader using the printed interval would overstate it. That matters because the Interpretation and Discussion present the trial as support for regional and national task-sharing policy.
The printed intervals also contradict the paper's own tests. An interval that excludes zero cannot accompany p = 0.80, so a reader cannot tell from the paper which of the two to believe without recomputing. In the emergent-case subgroup, the printed interval implies that MOs were reliably better in emergencies. Recomputed, the subgroup (61 patients, 3 failures) is uninformative in either direction. Any later meta-analysis or sample-size calculation that took the printed interval or its implied standard error at face value would understate the uncertainty about 80-fold.
Explanations considered
- Different population or denominator. This does not account for the numbers. The Abstract, the Results text and Table 3 rows 1 and 3 all use the same 422 patients and the same 7/231 versus 5/191. The emergent denominators (28 and 33) come from Table 2 and reproduce the printed percentages.
- Scale: proportion versus percentage points. This does not account for the numbers. The estimate 0.4 is in percentage points (0.004 as a proportion). On the proportion scale the Wald interval is −0.028 to 0.036, and on the percentage-point scale it is −2.75 to 3.58. Neither resembles 0.36 to 0.44.
- Ratio rather than difference. This does not account for the numbers. A risk-ratio interval would be centred near 1.16 and contain 1 (0.37 to 3.59), not sit around 0.4. The column heading reads "% between-group difference (95% CI)".
- One-sided or 90% interval. This does not account for the numbers. The upper bound of a two-sided 90% interval (equivalent to a one-sided 5% test) is 3.07 pp, still about seven times the printed 0.44.
- Choice of interval method. This does not account for the numbers. Wald, Newcombe and Agresti–Caffo intervals all span zero and run from about −3 to +4 pp. The paper does not name its interval method.
- Adjusted or stratified analysis. This does not account for the numbers. The Statistical analysis section lists only t-tests, Wilcoxon rank-sum, chi-square and Fisher's exact tests, with no regression, stratification or clustering, and none of these would shrink a standard error about 79-fold. The printed p = 0.7997 matches an unadjusted, uncorrected chi-square on these counts to four decimals, which confirms that the unadjusted 2 × 2 table was analysed.
- Rounding or typesetting. This does not account for the numbers. The interval appears three times in two forms (0.36–0.43% and 0.36 to 0.44), with ordinary decimal points, and the emergent row keeps its minus signs. A width error of this size cannot come from rounding.
- A possible arithmetic route (diagnostic only). If the estimate is kept in percentage points but the standard error is used as a proportion, 0.413 ± 1.96 × 0.0161 gives 0.38 to 0.44, and 0.4 ± 1.96 × 0.0161 gives 0.37 to 0.43. These are close to the printed all-cases intervals. The same route applied to the emergent row gives −2.60 to −2.38, not the printed −2.92 to −2.08. So this route does not explain every printed interval, and the actual computation cannot be identified without the analysis code. Either way, the printed intervals cannot be reconciled with the printed counts.
- Wording "difference in success rate". This is a separate, minor point. The Abstract and Results call 0.4 a "difference in success rate", but 3.0% − 2.6% is a difference in failure rates. A difference in success rates (MO − CA) would be −0.4. This affects the sign convention, not the width problem.
Status
- Version of record. The publisher's pages (gh.bmj.com, reached from the DOI, including the full-text, PDF and info pages) returned HTTP 403 to automated access, and the PMC HTML page showed only a CAPTCHA. I therefore checked the publisher-deposited JATS XML from Europe PMC against the PMC archive used here. The two agree word for word at every quoted place: the Abstract's "0.36–0.43%", the Results' "(CI=0.36–0.43%; p=0.7997)", and Table 3's three rows, each value in its own table cell. Every quote above was confirmed verbatim against the PMC-archived text.
- Supplements and data. The JATS XML lists no supplementary material, and the Europe PMC supplementary-files bundle holds only Figure 1. The data availability statement offers de-identified data on request, so no public dataset exists. The ClinicalTrials.gov record NCT04438811 is marked completed with no posted results. No source was available to reconstruct the authors' interval method, and none is needed, because the counts in Table 3 are enough to show that the printed intervals are impossible.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction or comment links (PMID 39153752). A web search for the title with "correction" returned only the article and index listings. The PubPeer API request for the DOI was blocked by a Cloudflare challenge, so I could not determine whether PubPeer has comments on this article.
- Sources. PMC article PMC11331853; Europe PMC full-text XML for PMC11331853; Europe PMC supplementary-files endpoint; ClinicalTrials.gov NCT04438811; Crossref; DOI landing page https://doi.org/10.1136/bmjgh-2023-014170 (publisher page not reachable).
Suggested correction
- Replace the 95% CI for the all-cases difference in the Abstract, the Results text and Table 3 (rows 1 and 3) with an interval computed from 7/231 versus 5/191. For example, the Wald interval is −2.75 to 3.58 percentage points and the Newcombe interval is −3.29 to 3.85. State which method is used.
- Replace the emergent-case interval in Table 3 (row 2), for example with the Wald interval −13.1 to 8.2 or the Newcombe interval −16.4 to 12.3 percentage points.
- Report the non-inferiority comparison explicitly, as the corrected upper bound against the 5-point margin. State whether the test is one-sided at 2.5% (95% CI) or at 5% (90% CI, upper bound 3.07).
- Make the Abstract, the Results text and Table 3 agree. They currently give 0.43 and 0.44 as the upper bound.
- Describe 0.4 as a difference in failure rates (MO − CA), or report −0.4 if it is meant as a difference in success rates.
- Review the wording of the Interpretation and Discussion in light of the wider interval: non-inferiority holds, but with about 1.2–1.4 points of headroom rather than about 4.6.
Recomputation script (python recompute.py.txt)
"""Recompute the primary-outcome confidence intervals of PMC11331853
(BMJ Glob Health 2024, doi:10.1136/bmjgh-2023-014170) from the paper's own counts,
and regenerate the figures in this script's folder.
Run: python recompute.py.txt
Needs numpy, scipy, matplotlib.
Inputs, all from the paper (Table 2, Table 3, abstract, Results):
all cases: MO 7/231 failures, CA 5/191 failures
emergent cases: MO 1/28, CA 2/33 (denominators: Table 2 row "Emergency", 28 | 33;
they also reproduce the printed 3.6% and 6.1%)
printed CIs: 0.36-0.43 (abstract, Results), 0.36-0.44 (Table 3), -2.92 to -2.08 (Table 3 emergent)
NI margin: 5 percentage points (Methods)
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z = stats.norm.ppf(0.975) # 1.959964
def wilson(x, n, z=Z):
p = x / n; den = 1 + z * z / n
c = (p + z * z / (2 * n)) / den
h = z * np.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / den
return c - h, c + h
def analyse(a, n1, b, n2, label):
p1, p2 = a / n1, b / n2; d = p1 - p2
se = np.sqrt(p1 * (1 - p1) / n1 + p2 * (1 - p2) / n2)
r = dict(p1=p1, p2=p2, d=d, se=se)
r["wald"] = (d - Z * se, d + Z * se)
r["wald90"] = (d - 1.645 * se, d + 1.645 * se)
l1, u1 = wilson(a, n1); l2, u2 = wilson(b, n2)
r["newcombe"] = (d - np.sqrt((p1 - l1) ** 2 + (u2 - p2) ** 2), d + np.sqrt((u1 - p1) ** 2 + (p2 - l2) ** 2))
q1, q2 = (a + 1) / (n1 + 2), (b + 1) / (n2 + 2)
seac = np.sqrt(q1 * (1 - q1) / (n1 + 2) + q2 * (1 - q2) / (n2 + 2))
r["agresti_caffo"] = (q1 - q2 - Z * seac, q1 - q2 + Z * seac)
t = np.array([[a, n1 - a], [b, n2 - b]])
r["p_chi2"] = stats.chi2_contingency(t, correction=False)[1]
r["p_yates"] = stats.chi2_contingency(t, correction=True)[1]
r["p_fisher"] = stats.fisher_exact(t)[1]
print(f"--- {label}")
print(f" MO {a}/{n1} = {100*p1:.2f}% CA {b}/{n2} = {100*p2:.2f}% difference MO-CA = {100*d:.3f} pp")
print(f" Wald SE = {100*se:.3f} pp")
for k in ("wald", "newcombe", "agresti_caffo", "wald90"):
lo, hi = r[k]; print(f" {k:14s} CI: {100*lo:6.2f} to {100*hi:5.2f} pp (width {100*(hi-lo):.2f} pp)")
print(f" p: uncorrected chi-square {r['p_chi2']:.4f}; Yates {r['p_yates']:.4f}; Fisher {r['p_fisher']:.4f}")
return r
print("Emergent denominators consistent with 3.6% and 6.1% and N=61:",
[(n1, 61 - n1) for n1 in range(10, 52) if round(100 / n1, 1) == 3.6 and round(200 / (61 - n1), 1) == 6.1])
A = analyse(7, 231, 5, 191, "All cases (Table 3 rows 1 and 3)")
E = analyse(1, 28, 2, 33, "Emergent cases (Table 3 row 2)")
print("\n--- Printed intervals")
printed = {"abstract": (0.40, 0.36, 0.43), "table": (0.40, 0.36, 0.44), "emergent": (-2.5, -2.92, -2.08)}
for k, (est, lo, hi) in printed.items():
se_imp = (hi - lo) / (2 * Z)
print(f" {k:9s}: {est} ({lo} to {hi}); width {hi-lo:.2f} pp; implied SE {se_imp:.4f} pp")
wa = 100 * (A["wald"][1] - A["wald"][0]); we = 100 * (E["wald"][1] - E["wald"][0])
print(f" all cases: Wald width {wa:.2f} pp = {wa/0.08:.0f} x Table 3 width, {wa/0.07:.0f} x abstract width")
print(f" all cases: Wald SE {100*A['se']:.3f} pp = {100*A['se']/((0.44-0.36)/(2*Z)):.0f} x SE implied by Table 3 interval")
print(f" emergent: Wald width {we:.2f} pp = {we/0.84:.0f} x printed width")
# p-value that the printed Table 3 interval would imply under a normal approximation
se_t = (0.44 - 0.36) / (2 * Z); z_t = 0.40 / se_t
p_implied = 2 * stats.norm.sf(z_t)
print(f" Table 3 interval implies z = {z_t:.1f}, two-sided p = {p_implied:.1e} (printed p = 0.7997)")
se_e = (-2.08 + 2.92) / (2 * Z); z_e = 2.5 / se_e
p_implied_e = 2 * stats.norm.sf(z_e)
print(f" emergent interval implies z = {z_e:.1f}, two-sided p = {p_implied_e:.1e} (printed p = 1.0000)")
print("\n--- Arithmetic route that reproduces the all-cases interval (diagnostic only)")
d_pp = 100 * A["d"]
print(f" {d_pp:.3f} pp +/- 1.96 x {A['se']:.4f} (SE left in proportion units) = {d_pp - Z*A['se']:.3f} to {d_pp + Z*A['se']:.3f}")
print(f" 0.4 pp +/- 1.96 x {A['se']:.4f} = {0.4 - Z*A['se']:.3f} to {0.4 + Z*A['se']:.3f}")
de_pp = 100 * E["d"]
print(f" emergent: {de_pp:.3f} +/- 1.96 x {E['se']:.4f} = {de_pp - Z*E['se']:.3f} to {de_pp + Z*E['se']:.3f} (printed -2.92 to -2.08: not reproduced)")
print("\n--- Other scales and summaries (Explanations considered)")
print(f" Wald CI on the proportion scale: {A['wald'][0]:.3f} to {A['wald'][1]:.3f}")
rr = A["p1"] / A["p2"]; se_lrr = np.sqrt(1/7 - 1/231 + 1/5 - 1/191)
print(f" risk ratio {rr:.2f}, 95% CI {rr*np.exp(-Z*se_lrr):.2f} to {rr*np.exp(Z*se_lrr):.2f}")
print(f" half-widths: printed Table 3 {(0.44-0.36)/2:.2f} pp; Wald {Z*100*A['se']:.2f} pp")
print(f" 90% upper bound {100*A['wald90'][1]:.2f} pp = {100*A['wald90'][1]/0.44:.1f} x the printed 0.44")
print("\n--- Non-inferiority margin (5 pp)")
for k in ("wald", "newcombe", "agresti_caffo", "wald90"):
hi = 100 * A[k][1]; print(f" {k:14s} upper bound {hi:.2f} pp -> headroom {5 - hi:.2f} pp; below margin: {hi < 5}")
for k in ("abstract", "table"):
print(f" printed {k:8s} upper bound {printed[k][2]} pp -> headroom {5 - printed[k][2]:.2f} pp")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "svg.fonttype": "none",
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG})
def save(fig, path):
fig.savefig(path + ".svg"); fig.savefig(path + ".png", dpi=200); plt.close(fig)
def style(ax):
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"): ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
def interval_rows(ax, rows):
for i, (lab, est, lo, hi, col, txt) in enumerate(rows):
y = len(rows) - 1 - i
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt")
ax.plot([lo, lo], [y - .12, y + .12], color=col, lw=2); ax.plot([hi, hi], [y - .12, y + .12], color=col, lw=2)
ax.plot(est, y, "o", ms=8, color=col, mec=BG, mew=1.5, zorder=3)
if txt:
ax.text(1.02, y, txt, va="center", fontsize=10, color=INK2, transform=ax.get_yaxis_transform())
ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows][::-1])
ax.set_ylim(-0.7, len(rows) - 0.3)
def pp(x): return 100 * x
# Candidate A (default fig1): every printed vs recomputed interval on one axis
def cand_a(path):
fig, ax = plt.subplots(figsize=(7.2, 4.6))
rows = [
("All cases, printed (Abstract, text)", 0.40, 0.36, 0.43, ORANGE, "0.4 (0.36 to 0.43)"),
("All cases, printed (Table 3)", 0.40, 0.36, 0.44, ORANGE, "0.4 (0.36 to 0.44)"),
("All cases, recomputed Wald", pp(A["d"]), pp(A["wald"][0]), pp(A["wald"][1]), BLUE, f"{pp(A['d']):.2f} ({pp(A['wald'][0]):.2f} to {pp(A['wald'][1]):.2f})"),
("All cases, recomputed Newcombe", pp(A["d"]), pp(A["newcombe"][0]), pp(A["newcombe"][1]), BLUE, f"{pp(A['d']):.2f} ({pp(A['newcombe'][0]):.2f} to {pp(A['newcombe'][1]):.2f})"),
("Emergent, printed (Table 3)", -2.5, -2.92, -2.08, ORANGE, "−2.5 (−2.92 to −2.08)"),
("Emergent, recomputed Wald", pp(E["d"]), pp(E["wald"][0]), pp(E["wald"][1]), BLUE, f"{pp(E['d']):.2f} ({pp(E['wald'][0]):.1f} to {pp(E['wald'][1]):.1f})"),
]
rows = [(a, b, c, d, e, f.replace("-", "−")) for a, b, c, d, e, f in rows]
style(ax); ax.axvline(0, color=INK2, lw=0.8)
ax.axvline(5, color=INK2, lw=1.2, ls=(0, (4, 3)))
ax.text(5.3, -0.55, "NI margin, 5 pp", fontsize=10, color=INK2, va="center")
interval_rows(ax, rows)
ax.axhline(1.5, color=GRID, lw=0.8)
ax.set_xlim(-15, 10); ax.set_xlabel("Failure-rate difference, MO − CA (pp), with 95% CI")
ax.text(1.02, len(rows) - 0.3, "estimate (95% CI)", fontsize=10, color=INK, transform=ax.get_yaxis_transform(), va="bottom")
ax.plot([], [], "o-", color=ORANGE, lw=2, ms=8, label="As printed")
ax.plot([], [], "o-", color=BLUE, lw=2, ms=8, label="Recomputed from the paper's counts (7/231 vs 5/191; 1/28 vs 2/33)")
fig.legend(*ax.get_legend_handles_labels(), loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=2, frameon=False)
fig.subplots_adjust(left=0.37, right=0.75, top=0.84, bottom=0.12); save(fig, path)
# Candidate B: primary outcome only, emphasising headroom to the margin
def cand_b(path):
fig, ax = plt.subplots(figsize=(7.2, 3.6))
rows = [
("Printed, Abstract and Results", 0.40, 0.36, 0.43, ORANGE, ""),
("Printed, Table 3", 0.40, 0.36, 0.44, ORANGE, ""),
("Recomputed, Wald 95%", pp(A["d"]), pp(A["wald"][0]), pp(A["wald"][1]), BLUE, ""),
("Recomputed, Newcombe 95%", pp(A["d"]), pp(A["newcombe"][0]), pp(A["newcombe"][1]), BLUE, ""),
("Recomputed, Agresti–Caffo 95%", pp(A["d"]), pp(A["agresti_caffo"][0]), pp(A["agresti_caffo"][1]), BLUE, ""),
]
style(ax); ax.axvline(0, color=INK2, lw=0.8); ax.axvline(5, color=INK2, lw=1.2, ls=(0, (4, 3)))
interval_rows(ax, rows)
for y, hi, col in ((4, 0.43, ORANGE), (2, pp(A["wald"][1]), BLUE)):
ax.annotate("", xy=(5, y + 0.3), xytext=(hi, y + 0.3), arrowprops=dict(arrowstyle="<->", color=INK2, lw=1))
ax.text(hi - 0.1, y + 0.3, f"{5 - hi:.2f} pp to margin", ha="right", va="center", fontsize=10, color=INK2)
ax.text(5.1, 0, "NI margin\n5 pp", fontsize=10, color=INK2, va="center")
ax.set_xlim(-4, 6.6); ax.set_xlabel("Failure-rate difference, MO − CA (pp), all cases, 95% CI")
ax.plot([], [], "o-", color=ORANGE, lw=2, ms=8, label="As printed")
ax.plot([], [], "o-", color=BLUE, lw=2, ms=8, label="Recomputed from 7/231 vs 5/191")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False)
fig.tight_layout(); save(fig, path)
# Candidate C: arithmetic flow, showing the step where units diverge
def cand_c(path):
fig, ax = plt.subplots(figsize=(7.2, 4.0)); ax.axis("off"); ax.set_xlim(0, 10); ax.set_ylim(0.4, 6.0)
def box(x, y, w, h, txt, ec, fc=BG, bold=False):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.04,rounding_size=0.12", fc=fc, ec=ec, lw=2))
ax.text(x + w / 2, y + h / 2, txt, ha="center", va="center", fontsize=10.5, fontweight="bold" if bold else "normal")
box(0.2, 4.7, 2.6, 1.1, "Counts\n7/231 vs 5/191", INK2)
box(3.4, 4.7, 3.0, 1.1, f"Difference\n{A['d']:.5f} = {pp(A['d']):.2f} pp", INK2)
box(7.0, 4.7, 2.8, 1.1, f"Wald SE\n{A['se']:.4f} = {pp(A['se']):.2f} pp", INK2)
for x0, x1 in ((2.8, 3.4), (6.4, 7.0)):
ax.annotate("", xy=(x1, 5.25), xytext=(x0, 5.25), arrowprops=dict(arrowstyle="->", color=INK2, lw=1.5))
box(0.3, 1.9, 4.4, 1.5, f"Same units (pp):\n{pp(A['d']):.2f} ± 1.96 × {pp(A['se']):.2f}\n= {pp(A['wald'][0]):.2f} to {pp(A['wald'][1]):.2f} pp".replace("-", "−"), BLUE)
box(5.3, 1.9, 4.4, 1.5, f"Mixed units:\n{pp(A['d']):.2f} pp ± 1.96 × {A['se']:.4f}\n= {pp(A['d'])-Z*A['se']:.2f} to {pp(A['d'])+Z*A['se']:.2f}", ORANGE, fc="#fdf0ea")
ax.annotate("", xy=(2.5, 3.4), xytext=(4.9, 4.7), arrowprops=dict(arrowstyle="->", color=BLUE, lw=1.5))
ax.annotate("", xy=(3.2, 3.4), xytext=(7.6, 4.7), arrowprops=dict(arrowstyle="->", color=BLUE, lw=1.5))
ax.annotate("", xy=(6.8, 3.4), xytext=(5.1, 4.7), arrowprops=dict(arrowstyle="->", color=ORANGE, lw=1.5))
ax.annotate("", xy=(7.5, 3.4), xytext=(8.4, 4.7), arrowprops=dict(arrowstyle="->", color=ORANGE, lw=1.5))
ax.text(2.5, 1.2, "Recomputed interval: spans 0,\nconsistent with the printed p = 0.7997", ha="center", va="center", fontsize=10, color=INK2)
ax.text(7.5, 1.2, "Printed: 0.36 to 0.43 (Abstract), 0.36 to 0.44 (Table 3).\nA reconstruction only: this route does not\nreproduce the emergent row (−2.92 to −2.08)", ha="center", va="center", fontsize=10, color=INK2)
ax.text(7.6, 4.05, "SE taken as a proportion\n(×100 omitted)", ha="left", fontsize=10, color=INK, style="italic", va="center")
fig.tight_layout(); save(fig, path)
# Candidate D: interval widths, printed vs recomputed
def cand_d(path):
fig, ax = plt.subplots(figsize=(7.2, 3.2))
labs = ["All cases", "Emergent cases"]
rep = [0.44 - 0.36, -2.08 + 2.92]; rec = [wa, we]
y = np.array([1, 0]); h = 0.34
ax.barh(y + h / 2 + 0.01, rep, height=h, color=ORANGE, label="Printed 95% CI width")
ax.barh(y - h / 2 - 0.01, rec, height=h, color=BLUE, label="Recomputed Wald 95% CI width")
for yy, r, c in zip(y, rep, rec):
ax.text(r + 0.3, yy + h / 2, f"{r:.2f} pp", va="center", fontsize=10, color=INK2)
ax.text(c + 0.3, yy - h / 2, f"{c:.2f} pp ({c/r:.0f}× wider)", va="center", fontsize=10, color=INK2)
style(ax); ax.set_yticks(y); ax.set_yticklabels(labs); ax.set_xlim(0, 28)
ax.set_xlabel("Width of the 95% CI for the difference in failure rate (percentage points)")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False)
fig.tight_layout(); save(fig, path)
# Candidate E: p-values, printed vs implied by the printed CI (log scale)
def cand_e(path):
fig, ax = plt.subplots(figsize=(7.2, 3.4))
rows = [("All cases", 0.7997, A["p_chi2"], p_implied), ("Emergent cases", 1.0, E["p_fisher"], p_implied_e)]
for i, (lab, prep, prec, pimp) in enumerate(rows):
y = 1.5 * (1 - i)
ax.plot([pimp, prep], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(prep, y + 0.08, "o", ms=9, color=ORANGE, zorder=3)
ax.plot(prec, y - 0.08, "D", ms=8, color=BLUE, zorder=3)
ax.plot(pimp, y, "o", ms=9, mfc=BG, mec=ORANGE, mew=2, zorder=3)
ax.text(pimp, y + 0.22, f"implied by printed CI: p ≈ {pimp:.0e}", ha="center", fontsize=10, color=INK2, va="bottom")
ax.text(prep * 0.3, y - 0.22, f"printed p = {prep:.4f}; recomputed {prec:.4f}", ha="right", fontsize=10, color=INK2, va="top")
ax.axvline(0.05, color=INK2, lw=0.8); ax.text(0.05, 2.15, "p = 0.05 ", fontsize=10, color=INK2, ha="right")
ax.set_xscale("log"); ax.set_xlim(1e-95, 30); style(ax)
ax.set_yticks([1.5, 0]); ax.set_yticklabels([r[0] for r in rows]); ax.set_ylim(-0.7, 2.3)
ax.set_xlabel("Two-sided p-value (log scale)")
ax.plot([], [], "o", color=ORANGE, ms=9, label="Printed p")
ax.plot([], [], "o", mfc=BG, mec=ORANGE, mew=2, ms=9, label="p implied by printed CI")
ax.plot([], [], "D", color=BLUE, ms=8, label="Recomputed from counts")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=3, frameon=False)
fig.tight_layout(); save(fig, path)
cand_a(os.path.join(HERE, "fig1"))
for f, n in ((cand_a, "a_all_intervals"), (cand_b, "b_margin_headroom"), (cand_c, "c_unit_flow"),
(cand_d, "d_ci_widths"), (cand_e, "e_pvalues")):
f(os.path.join(CAND, "cand_" + n))
print("\nFigures written to", HERE, "and", CAND)
Reported complete-abortion risk ratio of 2.55 should be about 1.02 by the paper's own conversion
BMJ Glob Health (2025) · doi:10.1136/bmjgh-2024-016024 · PMC11749663
Summary
For complete abortion in the review's only task-shifting trial (midlevel providers vs doctors, 1032 participants), the Abstract, the Results text and Table 2 all report a risk ratio (RR) of 2.55 (95% CI 0.82 to 4.27), described as "equivalent" or "little or no difference". The same Table 2 row gives risks of 96.1% and 97.3% and an adjusted risk difference (RD) of 1.49% (−0.17% to 3.14%), and its footnote § says the RR was converted from that RD with a 96% control risk; done in consistent units, that conversion gives RR 1.016 (0.998 to 1.033), whereas leaving the RD in percentage points, (0.96 + 1.49)/0.96, reproduces the published 2.55 (0.82 to 4.27) exactly. With a 96.1% control risk no RR can exceed 1/0.961 = 1.04, so the published value is impossible, and its interval is 100 times too wide on the linear scale. The verbal conclusion of no meaningful difference stands, but the first quantitative result in the Abstract misstates the size and precision of the effect, the correct value appears nowhere in the paper, and no correction has been found.
What the paper reports
Low certainty evidence indicates task shifting of medical abortion by doctors to midlevel providers may result in equivalent complete abortion (RR: 2.55, 95% CI: 0.82 to 4.27).
— Abstract, Results
Low certainty evidence indicates that compared with doctor-led medical abortion, nurse-led medical abortion may lead to little or no difference in abortion success (RR: 2.55, 95% CI: 0.82 to 4.27, one trial, 1032 participants) or the occurrence of an incomplete abortion (RR: 0.92, 95% CI: 0.45 to 1.89, one trial, 1032 participants, table 2). The certainty of evidence was downgraded by two levels due to serious imprecision (data from a single trial and CI including both benefit and harm) and serious risk of bias (no blinding of participants, personnel and outcome assessors).
— Results › Effects of interventions › Comparison 1. Who provides care (task shifting)
Patient health status: complete abortion | 96.1% | 97.3% | RR 2.55 (0.82 to 4.27)§ | 1032 (1 RCT) | Low†‡⊕⊕⊝⊝ | RD 1.49%¶ (–0.17% to 3.14%)
— Table 2 (Summary of findings 1), "complete abortion" row. Columns: outcome | assumed risk, control | corresponding risk, intervention | relative effect (95% CI) | participants | certainty | comments
§The effect in RR was converted from the adjusted risk difference provided in the report using control group risk of 96%.
— Table 2, footnote §
¶The risk difference calculated with intention-to-treat analysis estimated from a generalised estimating equation model with treatment as the fixed effect and service provider as a random effect and adjusted with woman’s age and length of gestation.
— Table 2, footnote ¶
the assumed risk in the comparison group and the relative effect of the intervention
— Table 2, footnote * (which says that the intervention-group risk is based on these two quantities)
What the paper's own numbers show
Method. Footnote § describes the conversion. If the control risk is p₀ and the risk difference is RD (both as proportions), the intervention risk is p₀ + RD, so RR = (p₀ + RD) / p₀. Applied to the point estimate and to each CI limit, this maps the RD interval linearly onto an RR interval. I applied it with p₀ = 0.96 (the footnote) and p₀ = 0.961 (Table 2), first with the RD as a proportion and then with the RD left as a percentage-point number. I also checked the result against the two risks in the same row of Table 2, and against the shape of the published interval. As an outside cross-check I used the counts in the source trial's abstract (Lancet 2011;377:1155-61, PMID 21458058). All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| RR, complete abortion | Abstract; Results Comparison 1; Table 2 | 2.55 (0.82 to 4.27) | 1.016 (0.998 to 1.033), from footnote § with RD as a proportion (same to 3 dp with p₀ 0.96 or 0.961) |
| Same formula, RD left in percentage points | Table 2 footnote § | — | (0.96 + 1.49)/0.96 = 2.552; (0.96 − 0.17)/0.96 = 0.823; (0.96 + 3.14)/0.96 = 4.271 → 2.55 (0.82 to 4.27), an exact match to the published values |
| Intervention risk implied by the conversion | Table 2 | 97.3% | 0.96 + 0.0149 = 0.975 (correct units); 0.96 + 1.49 = 2.45, i.e. 245% (published units) |
| Ratio of the two risks in Table 2 | Table 2 | 96.1% vs 97.3% | 97.3 / 96.1 = 1.012 |
| Largest RR possible with control risk 96.1% | Table 2 | — | 1 / 0.961 = 1.041 |
| Intervention risk implied by RR 2.55 (footnote *) | Table 2 | 97.3% | 0.961 × 2.55 = 2.451 (245%) |
| Shape of the reported RR interval | Table 2 | 0.82 to 4.27 around 2.55 | Linear gaps 1.73 below, 1.72 above (symmetric on the linear scale); log gaps 1.135 vs 0.516. RR width / RD width = 1.042 ≈ 1/0.96 = 1.042, i.e. a linear rescaling of the RD interval by 1/0.96 |
| Width of the published interval relative to the recomputed one | Table 2 | 0.82 to 4.27 | 100 times wider on the linear scale; 48.6 times wider on the log scale |
| Adjusted RD, complete abortion | Table 2 comments | 1.49% (−0.17% to 3.14%) | Not reproducible from counts (GEE-adjusted, footnote ¶); taken as given. Unadjusted from trial counts: 1.19% (−0.99% to 3.37%); trial abstract: 1.24% (−0.53 to 3.02) |
| Unadjusted RR, complete abortion (outside check) | Source trial abstract: 504/518 vs 494/514 | — | 1.012 (0.990 to 1.035) |
| RR, incomplete abortion (control check) | Results; Table 2 row 2 | 0.92 (0.45 to 1.89) | 14/518 vs 15/514: 0.926 (0.45 to 1.90), consistent |
The adjacent incomplete-abortion RR is reproduced from the trial's counts. This shows that the rest of the row's arithmetic holds, and that the discrepancy is confined to the RD-to-RR conversion.
Figure
Figure 1. The conversion described in Table 2 footnote §, from the inputs stated in Table 2 to the risk ratio. The two lanes differ only at the first step, which is shaded: whether the adjusted risk difference enters the formula as a proportion (0.0149, blue, recomputed) or as a percentage-point number (1.49, orange). The orange lane reproduces the published RR 2.55 (0.82 to 4.27) to two decimals. It does so by way of an intervention "risk" of 245%, and its result lies above 1/0.961 = 1.04, the largest RR possible with the 96.1% control risk in the same table row. The blue lane gives RR 1.016 (0.998 to 1.033), which agrees with the ratio of the tabulated risks (1.012). All values are computed in recompute.py.txt.
Figure 2. Risk ratio for complete abortion (midlevel providers vs doctors) on a log scale, with a thin reference line at RR = 1. Orange: the value published in the Abstract, Results and Table 2. Blue: the footnote § conversion recomputed with the RD as a proportion (its interval, 0.998 to 1.033, is narrower than the marker at this scale), and the plain ratio of the two Table 2 risks. The shaded region marks RRs above 1/0.961 = 1.04, which cannot occur when the control risk is 96.1%. The published point estimate and most of its interval lie in that region.
Why it matters
- It is the headline number for the outcome. The value appears three times: as the first quantitative result in the Abstract, in the Results text and in the summary-of-findings table. A reader who takes the RR at face value would read it as about 2.5 times the rate of complete abortion with a very wide interval. The trial data in the same row show nearly identical rates (97.3% vs 96.1%) and a tight interval (about 1.00 to 1.03).
- The number is internally contradictory. The text calls it "equivalent" / "little or no difference", which agrees with the RD but not with an RR of 2.55. Footnote * says the intervention risk is based on the control risk and the relative effect, and 96.1% × 2.55 is 245%, not 97.3%.
- It can propagate. Anyone who extracts the RR from the Abstract or Table 2 for an overview, a meta-analysis or a guideline would carry forward a ratio wrong by a factor of about 2.5, with an interval 100 times too wide on the linear scale (about 49 times on the log scale). The correct value is not stated anywhere in the paper, so a reader cannot pick it up from elsewhere in the text.
- The GRADE rationale cites the interval. The imprecision downgrade is justified as "CI including both benefit and harm" and, in footnote ‡, "wider CI". The correct interval (0.998 to 1.033) still just includes 1, so this wording is not strictly wrong, and a downgrade for a single trial may well stand. But the published interval of 0.82 to 4.27 makes the imprecision look far larger than it is.
- What does not change. The direction of the effect, the lack of statistical significance, the qualitative reading (equivalence), the low-certainty rating and the review's overall conclusions on task shifting are not affected.
Explanations considered
- A different effect measure (odds ratio, or the RR of a non-complete outcome). From the trial counts, the odds ratio for complete abortion is 1.457 (0.73 to 2.92). The RR of a non-complete abortion (doctor vs midlevel) is 1.440. Neither is 2.55 (0.82 to 4.27), and Table 2 labels the value as an RR.
- Covariate adjustment. Footnote ¶ says the RD came from a GEE model adjusted for age and gestation, which explains why the RD (1.49%) differs from the crude 1.19% to 1.24%. But no adjustment can produce a risk ratio above 1/0.961 = 1.04 when the control risk is 96.1%, and converting the adjusted RD gives 1.016.
- A different control risk. The footnote uses 96%, and Table 2 shows 96.1%. Both give RR 1.016 (0.998 to 1.033) with consistent units, and both give 2.55 (0.82 to 4.27) when the RD is not divided by 100. The choice of control risk is immaterial.
- The log-scale asymmetry is just a feature of the conversion. That is correct: an interval converted linearly from an RD interval is symmetric on the linear scale, as observed here. So the asymmetry on the log scale is not itself evidence of an error. The error is the factor of 100 in the RD, which the linear structure of the published interval confirms (its width is exactly 1/0.96 times the width of the RD interval in percentage points).
- A typesetting or XML conversion artefact. The same values appear in three separate places (Abstract, Results text, Table 2 cell) in the archived publisher XML. They all agree with each other and with an exact arithmetic reproduction, so a rendering artefact does not explain them.
- A rounding or transcription slip. A transposed digit or a rounding error would not produce three numbers (point estimate and both limits) that all match one systematic formula to two decimals.
- The supplementary material contains another calculation. Online supplemental file 1 (search strategy, study characteristics, extraction and risk-of-bias tables) and file 2 (reflexivity statement) give no further effect estimate or RR calculation for this outcome.
None of these accounts for the published numbers. The stated conversion, carried out with the RD in percentage points rather than as a proportion, reproduces them exactly.
Status
As of 2026-10-23:
- Version of record. The publisher's typeset PDF (PMC11749663.1.pdf from the PMC open-data bucket; 13 pages, "BMJ Glob Health 2025;10:e016024", created 2025-01-03) carries the same values in all three places: the Abstract, Results › Comparison 1, and the Table 2 complete-abortion row with footnote §. The minus signs in the RD interval are intact, and the Table 2 columns line up, so no cell is misplaced. The PMC and Europe PMC XML agree with the PDF. Every quote above was checked verbatim against the PMC text with
src/show.py. The publisher's HTML page (gh.bmj.com, reached from the DOI redirect) returned HTTP 403. - Supplements. I checked online supplemental file 1 (PDF, 24 pages, Tables S1–S7 and Figure S1) and file 2 (DOCX, reflexivity statement). Table S6 describes the trial (535 and 542 randomised) but gives no complete-abortion RR or RD. Neither file restates or corrects the figure.
- Corrections and notices.
src/known.pyreturned no Crossref updates, no PMC notices and no Europe PMC notices. The Crossref works record has noupdate-to,updated-byorrelationentries, and Crossrefrelation.objectandupdatesfilters on the DOI return 0 works. A Europe PMC search for the DOI returns only the original article. Earlier web searches for a correction, erratum or comment found none. The web-search quota was used up during this final check, so those searches were not repeated today. - PubPeer. The PubPeer search page and API return HTTP 403 (a Cloudflare challenge) from this machine, so PubPeer could not be checked directly. Earlier web searches found no PubPeer thread for this article.
- Sources. PMC11749663 (https://pmc.ncbi.nlm.nih.gov/articles/PMC11749663/); Crossref API (https://api.crossref.org/works/10.1136/bmjgh-2024-016024); Europe PMC REST search; source trial abstract, PubMed 21458058 (Lancet 2011;377:1155-61, doi:10.1016/S0140-6736(10)62229-5), retrieved through NCBI E-utilities.
Suggested correction
In the Abstract, in Results › Comparison 1 and in Table 2 (complete abortion row), replace "RR: 2.55, 95% CI: 0.82 to 4.27" with "RR: 1.02, 95% CI: 1.00 to 1.03". Unrounded, this is 1.016 (0.998 to 1.033), from (0.96 + RD)/0.96 with RD = 0.0149 (−0.0017 to 0.0314). If two decimals would hide the fact that the lower limit lies just below 1, report three decimals: 1.016 (0.998 to 1.033). The authors may also wish to:
- re-check the GRADE imprecision rationale for this outcome against the corrected interval, and
- check whether the same conversion step was applied elsewhere. For example, Table 3 footnote † describes an RR derived from an OR with an assumed control risk.
Recomputation script (python recompute.py.txt)
"""Recompute the complete-abortion risk ratio in PMC11749663 (BMJ Glob Health 2025,
doi:10.1136/bmjgh-2024-016024), Table 2, and regenerate fig1/fig2 in this folder.
Run: python recompute.py.txt
Needs numpy, scipy, matplotlib.
"""
from pathlib import Path
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, Rectangle
HERE = Path(__file__).resolve().parent
z = stats.norm.ppf(0.975)
# ---------------------------------------------------------------- inputs (paper)
RR_REP = (2.55, 0.82, 4.27) # Abstract; Results Comparison 1; Table 2 row "complete abortion"
RD_PP = (1.49, -0.17, 3.14) # Table 2 comments: adjusted RD, percentage points (footnote ¶)
P0_FOOT = 0.96 # footnote §: "control group risk of 96%"
CTRL, INTV = 0.961, 0.973 # Table 2 illustrative risks, control / intervention
print("=" * 72)
print("1. Reported values (paper)")
print(f" RR {RR_REP[0]} (95% CI {RR_REP[1]} to {RR_REP[2]})")
print(f" adjusted RD {RD_PP[0]}% ({RD_PP[1]}% to {RD_PP[2]}%); control 96.1%, intervention 97.3%")
def rr_from_rd(rd, p0):
"""Footnote §: RR = (p0 + RD) / p0, with RD and p0 on the same scale."""
return (p0 + rd) / p0
print("\n2. Footnote § conversion done in consistent units (RD / 100)")
correct = [rr_from_rd(x / 100, P0_FOOT) for x in RD_PP]
print(" p0 = 0.96 : RR = %.4f (%.4f to %.4f) -> %.3f (%.3f to %.3f)" % (*correct, *correct))
correct961 = [rr_from_rd(x / 100, CTRL) for x in RD_PP]
print(" p0 = 0.961: RR = %.4f (%.4f to %.4f)" % tuple(correct961))
print(" implied intervention risk 0.96 + 0.0149 = %.4f" % (P0_FOOT + RD_PP[0] / 100))
print("\n3. Same formula with RD left in percentage points (not divided by 100)")
slip = [rr_from_rd(x, P0_FOOT) for x in RD_PP]
print(" p0 = 0.96 : RR = %.3f (%.3f to %.3f) -> rounds to %.2f (%.2f to %.2f)" % (*slip, *slip))
slip961 = [rr_from_rd(x, CTRL) for x in RD_PP]
print(" p0 = 0.961: RR = %.3f (%.3f to %.3f)" % tuple(slip961))
print(" implied intervention 'risk' 0.96 + 1.49 = %.2f (i.e. %.0f%%)" % (P0_FOOT + RD_PP[0], 100 * (P0_FOOT + RD_PP[0])))
print("\n4. Consistency checks within Table 2")
ratio_tab = INTV / CTRL
print(f" ratio of Table 2 risks 97.3/96.1 = {ratio_tab:.4f}")
print(f" largest possible RR with control risk 96.1%: 1/0.961 = {1 / CTRL:.4f}")
print(f" intervention risk implied by RR 2.55 (footnote *): 0.961 x 2.55 = {CTRL * 2.55:.3f} ({100 * CTRL * 2.55:.0f}%)")
print(f" published / correct RR = {RR_REP[0] / correct[0]:.2f}; published / correct CI width = "
f"{(RR_REP[2] - RR_REP[1]) / (correct[2] - correct[1]):.1f} (exactly 100 before rounding: "
f"{(slip[2] - slip[1]) / (correct[2] - correct[1]):.1f})")
print(f" published / correct CI width on the log scale = "
f"{np.log(RR_REP[2] / RR_REP[1]) / np.log(correct[2] / correct[1]):.1f}")
print(f" intervention risk implied by RR {correct[0]:.4f}: 0.961 x {correct[0]:.4f} = {CTRL * correct[0]:.4f}")
print("\n5. Shape of the reported interval")
lo, hi = RR_REP[0] - RR_REP[1], RR_REP[2] - RR_REP[0]
print(f" linear gaps: {lo:.2f} below, {hi:.2f} above (midpoint {(RR_REP[1] + RR_REP[2]) / 2:.3f})")
llo, lhi = np.log(RR_REP[0]) - np.log(RR_REP[1]), np.log(RR_REP[2]) - np.log(RR_REP[0])
print(f" log gaps: {llo:.3f} below, {lhi:.3f} above")
print(f" RR width / RD width = {(RR_REP[2] - RR_REP[1]) / (RD_PP[2] - RD_PP[1]):.4f}; 1/0.96 = {1 / 0.96:.4f}")
print("\n6. Source trial (Lancet 2011;377:1155-61, PMID 21458058, abstract counts)")
a, n1 = 504, 518 # midlevel providers: complete abortions / analysed
b, n0 = 494, 514 # doctors
p1, p0 = a / n1, b / n0
rr = p1 / p0
se = np.sqrt(1 / a - 1 / n1 + 1 / b - 1 / n0)
print(f" complete: {a}/{n1} = {p1:.4f} vs {b}/{n0} = {p0:.4f}; n = {n1 + n0}")
print(f" unadjusted RR = {rr:.4f} ({np.exp(np.log(rr) - z * se):.4f} to {np.exp(np.log(rr) + z * se):.4f})"
f" -> {rr:.3f} ({np.exp(np.log(rr) - z * se):.3f} to {np.exp(np.log(rr) + z * se):.3f})")
rd = p1 - p0
serd = np.sqrt(p1 * (1 - p1) / n1 + p0 * (1 - p0) / n0)
print(f" unadjusted RD = {100 * rd:.2f}% ({100 * (rd - z * serd):.2f}% to {100 * (rd + z * serd):.2f}%);"
" trial abstract reports 1.24% (-0.53 to 3.02)")
print(f" footnote § conversion of the trial-abstract RD 1.24%: RR = {rr_from_rd(0.0124, P0_FOOT):.4f}")
inc1, inc0 = n1 - a, n0 - b - 5 # 5 failed abortions in doctor arm, none in midlevel arm
r = (inc1 / n1) / (inc0 / n0)
s = np.sqrt(1 / inc1 - 1 / n1 + 1 / inc0 - 1 / n0)
print(f" incomplete: {inc1}/{n1} vs {inc0}/{n0}: RR {r:.3f} ({np.exp(np.log(r) - z * s):.2f} to "
f"{np.exp(np.log(r) + z * s):.2f}); paper reports 0.92 (0.45 to 1.89)")
print("\n7. Other effect measures, to rule them out as the source of 2.55 (0.82 to 4.27)")
orr = (a / (n1 - a)) / (b / (n0 - b))
seo = np.sqrt(1 / a + 1 / (n1 - a) + 1 / b + 1 / (n0 - b))
print(f" odds ratio, complete abortion = {orr:.3f} ({np.exp(np.log(orr) - z * seo):.2f} to {np.exp(np.log(orr) + z * seo):.2f})")
print(f" RR of non-complete abortion, doctor vs midlevel = {((n0 - b) / n0) / ((n1 - a) / n1):.3f}")
# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, name):
fig.savefig(HERE / f"{name}.svg", facecolor=BG)
fig.savefig(HERE / f"{name}.png", dpi=200, facecolor=BG)
plt.close(fig)
# fig1: arithmetic chain, two lanes
fig = plt.figure(figsize=(7.2, 4.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0.5, 64)
ax.axis("off")
def box(x0, y0, w, h, text, edge, lw=2, face="white", bold_first=True, fs=10):
ax.add_patch(FancyBboxPatch((x0, y0), w, h, boxstyle="round,pad=0,rounding_size=1.2",
fc=face, ec=edge, lw=lw))
lines = text.split("\n")
n = len(lines)
for i, ln in enumerate(lines):
yy = y0 + h / 2 + (n - 1) / 2 * 3.3 - i * 3.3
ax.text(x0 + w / 2, yy, ln, ha="center", va="center", fontsize=fs, color=INK,
fontweight="bold" if (bold_first and i == 0) else "normal")
def arrow(x0, x1, y, color):
ax.annotate("", xy=(x1, y), xytext=(x0, y),
arrowprops=dict(arrowstyle="-|>", color=color, lw=2, mutation_scale=12))
box(2, 50.5, 96, 12,
"Inputs stated in Table 2\n"
"adjusted RD = 1.49% (−0.17% to 3.14%) · control risk 96% (footnote §)\n"
"conversion: RR = (0.96 + RD) / 0.96",
INK2, lw=1.2, face="#f3f2ef")
cols = [(2, 22), (27, 22), (52, 22), (77, 21)]
# highlight the diverging step (behind the first box of each lane only)
for yy in (29, 7):
ax.add_patch(FancyBboxPatch((0.9, yy - 1.1), 24.2, 15.7, boxstyle="round,pad=0,rounding_size=1.2",
fc="#e6e5e0", ec="none", zorder=0))
ax.text(13, 2.6, "units diverge here", ha="center", va="center", fontsize=10,
color=INK, fontweight="bold")
lanes = [
(29, BLUE, "Recomputed: RD divided by 100 before use (consistent units)",
["RD entered\n0.0149\n(−0.0017 to 0.0314)",
"Intervention risk\n0.96 + 0.0149\n= 0.975 (97.5%)",
f"RR = 0.975 / 0.96\n{correct[0]:.3f}\n({correct[1]:.3f} to {correct[2]:.3f})",
f"Check vs Table 2\n97.3 / 96.1 = {ratio_tab:.3f}\nconsistent"]),
(7, ORANGE, "Published: RD left in percentage points (not divided by 100)",
["RD entered\n1.49\n(−0.17 to 3.14)",
f"Intervention risk\n0.96 + 1.49\n= {P0_FOOT + RD_PP[0]:.2f} (245%)",
f"RR = 2.45 / 0.96\n{slip[0]:.2f}\n({slip[1]:.2f} to {slip[2]:.2f})",
f"Check vs Table 2\nabove max {1 / CTRL:.2f}\ninconsistent"]),
]
for y0, color, label, texts in lanes:
ax.add_patch(Rectangle((2.2, y0 + 15.2), 2.4, 2.4, fc=color, ec="none"))
ax.text(5.6, y0 + 16.4, label, ha="left", va="center", fontsize=10, color=INK)
for (x0, w), t in zip(cols, texts):
box(x0, y0, w, 13.5, t, color)
for (xa, wa), (xb, _) in zip(cols[:-1], cols[1:]):
arrow(xa + wa, xb, y0 + 6.75, color)
ax.text(62.0, 2.6, "= published 2.55 (0.82 to 4.27)", ha="center", va="center",
fontsize=10, color=INK, fontstyle="italic")
save(fig, "fig1")
# fig2: dot-and-interval on log scale
fig, ax = plt.subplots(figsize=(7.2, 2.9), facecolor=BG)
fig.subplots_adjust(left=0.035, right=0.97, top=0.97, bottom=0.18)
ax.set_facecolor(BG)
xmax_possible = 1 / CTRL
ax.axvspan(xmax_possible, 8, color="#efeeea", zorder=0, lw=0)
ax.text(1.08, -0.42, f"RR > 1/0.961 = {xmax_possible:.2f} is impossible with a 96.1% control risk",
fontsize=10, color=INK2, va="center", ha="left")
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
rows = [
(2.0, ORANGE, "o", RR_REP, "Published (Abstract, Results, Table 2): RR 2.55 (0.82 to 4.27)"),
(1.0, BLUE, "o", correct, f"Footnote § conversion, recomputed: RR {correct[0]:.3f} ({correct[1]:.3f} to {correct[2]:.3f})"),
(0.0, BLUE, "D", (ratio_tab, None, None), f"Ratio of Table 2 risks 97.3% / 96.1% = {ratio_tab:.3f}"),
]
for y, c, m, (pt, l, h), lab in rows:
if l is not None:
ax.plot([l, h], [y, y], color=c, lw=2, solid_capstyle="butt", zorder=3)
ax.plot(pt, y, marker=m, ms=9 if m == "o" else 8, color=c, mec=BG, mew=1.5, zorder=4, ls="none")
ax.text(0.52, y + 0.36, lab, fontsize=10, color=INK, va="center", ha="left")
ax.set_xscale("log")
ax.set_xlim(0.5, 8)
ax.set_ylim(-0.7, 2.6)
ax.set_xticks([0.5, 0.82, 1, 1.5, 2, 2.55, 4.27, 8])
ax.set_xticklabels(["0.5", "0.82", "1", "1.5", "2", "2.55", "4.27", "8"])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks([])
ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
for s in ("left", "right", "top"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.set_xlabel("Risk ratio, complete abortion, midlevel providers vs doctors (log scale; line at RR = 1)", fontsize=10)
save(fig, "fig2")
print("\nFigures written: fig1.svg/png, fig2.svg/png in", HERE)
Abstract's patient-handwashing confidence interval is impossible and conflicts with the paper's own non-significant clustered estimate
BMJ Glob Health (2016) · doi:10.1136/bmjgh-2016-000103 · PMC5321376
Summary
The abstract reports two behaviour changes after an infection prevention and control (IPC) workshop: maintenance of distance, RR 1.09 (95% CI 1.02 to 1.16), and patient handwashing, RR 1.19 (95% CI 1.3 to 1.25). The handwashing interval cannot be a confidence interval, because its lower limit (1.3) is above its upper limit (1.25) and the point estimate (1.19) lies outside it even if the limits are swapped. The paper's own clustered GEE analysis in Table 4, the analysis the Methods specify, gives handwashing RR 1.27 (0.95 to 1.71) and distance RR 1.11 (0.83 to 1.48). Neither is significant, and the Results text says handwashing "increased though not significantly". The abstract's distance figure equals, to two decimals, a crude risk ratio computed from the Table 4 counts that ignores clustering. As printed, the abstract presents as statistically significant two improvements that the body of the paper reports as non-significant.
What the paper reports
Abstract, Results:
The maintenance of distance (RR 1.09, 95% CI 1.02 to 1.16) and patient handwashing (RR 1.19, 95% CI 1.3 to 1.25) improved to >90%.
Results, "Adherence to IPC behaviours", first paragraph:
Patient handwashing, with or without HCW prompting, increased though not significantly from 82% to 99% (RR 1.21, 95% CI 0.95 to 1.71).
Results, "Adherence to IPC behaviours", second paragraph:
Virtually all HCWs stood 1.5 m from patients, increasing from 91% to 99% at follow-up (RR 1.11, 95% CI 0.83 to 1.48).
Table 4, screening block (columns: baseline correct, per cent, n = 90 | follow-up correct, per cent, n = 131 | RR | 95% CI):
Patient washed hands directly or washed on direction from HCW | 74 | 82 | 130 | 99 | 1.27 | 0.95 to 1.71†
Stood 1.5 m from patient | 82 | 91 | 130 | 99 | 1.11 | 0.83 to 1.48†
Table 4 footnote:
*Risk ratio using binomial regression (family: binomial, link: log) accounting for clustering at the health facility level (GEE).
†Indicates that a Poisson regression (family: Poisson, link: log) was used due to the failure of the binomial model to converge.
Methods, "Data analysis":
Generalised estimating equations (GEE) with robust SEs accounted for repeated measures among HCWs and clustering within PHUs.
For all statistical tests, a significance level of p<0.05 was chosen.
Discussion, first paragraph:
During workshops, HCWs focused on improving screening, maintaining physical distance and encouraging patient handwashing; changes in these domains were reflected in the improvements seen in these behaviours at follow-up. Significant improvements were not consistent across behaviours, partly due to several high baseline values (>80%).
What the paper's own numbers show
Method. I checked the abstract's handwashing interval for internal validity (lower limit no greater than upper limit, point estimate inside). From the Table 4 counts I recomputed each percentage and a crude follow-up versus baseline risk ratio with a log-scale Wald 95% CI, ignoring clustering. For each printed Table 4 interval I took the geometric centre, sqrt(lower × upper), which equals the point estimate of a symmetric log-scale interval, and the standard error it implies, ln(upper/lower)/3.92. I then compared the abstract's and the Results text's figures with both analyses. The GEE estimates themselves cannot be re-derived, because they need facility- and worker-level data. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Handwashing 95% CI, ordering | Abstract | 1.3 to 1.25 | lower limit 1.3 > upper limit 1.25: not a valid interval |
| Handwashing RR inside its CI? | Abstract | RR 1.19 | 1.19 lies outside 1.25–1.3, so it is outside the interval even with the limits swapped |
| Patient handwashing, % correct | Table 4; Results text | 82% → 99% | 74/90 = 82.2%, 130/131 = 99.2% |
| Handwashing RR, crude (ignoring clustering) | from Table 4 counts | — | 1.21 (1.10 to 1.33); log SE 0.050 |
| Handwashing RR, GEE Poisson | Table 4 | 1.27 (0.95 to 1.71) | centre of printed CI 1.275: internally consistent; implied robust log SE 0.150; includes 1 |
| Handwashing RR | Results text | 1.21 (0.95 to 1.71) | point estimate = crude 1.207; interval = the Table 4 GEE interval (centred on 1.275) |
| Stood 1.5 m at screening, % correct | Table 4; Results text | 91% → 99% | 82/90 = 91.1%, 130/131 = 99.2% |
| Distance RR, crude (ignoring clustering) | from Table 4 counts | — | 1.089 (1.019 to 1.164), i.e. 1.09 (1.02 to 1.16) |
| Distance RR | Abstract | 1.09 (1.02 to 1.16) | identical to the crude, unclustered value above |
| Distance RR, GEE Poisson | Table 4; Results text | 1.11 (0.83 to 1.48) | centre of printed CI 1.108: consistent; includes 1 |
| Stood 1.5 m at consultation, for contrast | Table 4 | 35/54 (65%) → 29/32 (91%); 1.18 (0.92 to 1.51) | crude 1.40 (1.12 to 1.75); neither matches the abstract |
| Hypothetical "1.13 to 1.25" for handwashing | — | — | geometric centre 1.188 (≈ 1.19), but it implies log SE 0.026: 0.52 × the crude SE and 0.17 × the GEE robust SE |
| Handwashing counts by district | Supplement, Annex 1 | Bo 34/45 → 58/58; Kenema 40/45 → 72/73 | sum 74/90 → 130/131, matching Table 4 |
Three things follow. First, the abstract's handwashing interval is not a confidence interval for any estimate. Second, the abstract's distance result reproduces exactly a crude, unclustered analysis of the Table 4 counts, while the Methods and Table 4 describe a clustered GEE analysis whose interval for the same comparison includes 1. Third, the abstract's handwashing estimate (1.19) matches neither the crude ratio (1.21) nor the GEE estimate (1.27). Its pattern resembles the distance figure: both of the abstract's intervals exclude 1, and both of the corresponding GEE intervals include 1.
Figure
Risk ratios for patient handwashing (top) and for standing 1.5 m from the patient at screening (bottom), on a log scale with a reference line at 1 (no change). Orange circles are values printed in the paper; blue squares are crude, unclustered Wald estimates recomputed from the Table 4 counts. The abstract's handwashing "interval" is drawn at its printed limits, with the lower limit (1.3) to the right of the upper limit (1.25) and the point estimate (1.19) outside both. The abstract's distance interval coincides with the crude recomputation. The paper's GEE intervals for both behaviours, in Table 4 and in the Results text, cross 1.
The abstract's two effect estimates (left) set against the Results text and Table 4 (right) and against the crude recomputation from the Table 4 counts (bottom). The abstract's intervals lie entirely above 1; the body's intervals for the same two behaviours include 1. The abstract's distance figure reproduces the unclustered calculation; its handwashing figure reproduces nothing in the paper.
Why it matters
These are the only two quantitative effect estimates in the abstract, and its sentence reads as a statistically significant improvement in both behaviours. The analysis described in the Methods (GEE with robust standard errors, significance at p<0.05) does not support that reading. Patient handwashing, RR 1.27 (0.95 to 1.71), and distance at screening, RR 1.11 (0.83 to 1.48), are both non-significant in Table 4, and the Results text says so for handwashing. A reader of the abstract alone, or a reviewer extracting effect sizes from it, would record two significant improvements and, for handwashing, an interval that cannot be used. Clustering matters here: the GEE robust standard errors implied by Table 4 are about three times (handwashing) and four times (distance) the unclustered ones, so the unclustered intervals are much too narrow for observations clustered within eight facilities.
The overstatement of significance is confined to the abstract. The Discussion describes the percentage improvements, which did occur (82% to 99% and 91% to 99%), and its next sentence states that "Significant improvements were not consistent across behaviours, partly due to several high baseline values (>80%)". That agrees with Table 4, where, of the handwashing items, only the prompted ones (screener asked patient to wash hands, RR 1.45, 1.16 to 1.80; patient washed hands on direction, RR 1.49, 1.19 to 1.86) are significant. The abstract carries no such qualification. Its conclusions are qualitative (favourable attitudes towards IPC, gaps in practice, the need for socio-behavioural improvement and a secure supply chain), and they do not depend on these two estimates.
Explanations considered
- Archiving or conversion error. Ruled out. The PMC XML, the Europe PMC record and the publisher-typeset PDF all print "RR 1.19, 95% CI 1.3 to 1.25". No minus signs, superscripts or decimal symbols are involved, and the Table 4 cells line up in both the XML and the PDF.
- A different row or population. The abstract says the behaviours "improved to >90%". For handwashing, only the combined row (82% → 99%) reaches 90%. The prompted rows reach 80%. For distance, both the screening row (91% → 99%) and the consultation row (65% → 91%) end above 90%, but only the screening row reproduces 1.09 (1.02 to 1.16). The consultation row gives 1.40 (1.12 to 1.75) crude and 1.18 (0.92 to 1.51) by GEE.
- A different model. Table 4's footnote explains that this row used a Poisson GEE because the binomial model did not converge. That model gives 1.27 (0.95 to 1.71), and the printed interval is centred on 1.275, so the table is internally consistent. The crude estimate is 1.21 (1.10 to 1.33). The abstract's 1.19 matches neither.
- Unadjusted figures in the abstract. This accounts exactly for the distance figure. It does not account for the handwashing figure: on the same crude basis handwashing would be 1.21 (1.10 to 1.33). Even if the abstract was meant to report crude estimates, those estimates ignore the clustering that the Methods say the analysis accounts for, and they disagree with Table 4 on significance.
- A typographical slip ("1.3" for "1.13"). An interval of 1.13 to 1.25 would be centred on 1.188 and would match 1.19, so a dropped digit would explain the impossible ordering. It would not explain where the estimate comes from. It is not the crude ratio or the GEE estimate, and such an interval would need a standard error about half the crude unclustered one and a sixth of the GEE one. Nor would it explain why the abstract implies significance when the Results text says the change "increased though not significantly".
- A one-sided interval or another stated convention. None is stated. The Methods give a significance level of p<0.05 for all tests, with GEE and an exchangeable working correlation.
- The Results text's "RR 1.21, 95% CI 0.95 to 1.71". Its point estimate is the crude ratio (1.207). Its interval is the Table 4 GEE interval, whose centre is 1.275. So the text pairs a crude point estimate with a model-based interval. This is a smaller inconsistency, but it does not affect the text's conclusion that the change was not significant, which agrees with Table 4.
Status
- Version of record. The DOI resolves to the publisher's page at gh.bmj.com (https://gh.bmj.com/lookup/doi/10.1136/bmjgh-2016-000103), which returned HTTP 403 to automated access. The version of record was read from a copy of the publisher-typeset PDF (12 pages, CC BY 4.0) on the RKI edoc repository and cross-checked against PMC and Europe PMC. All three carry the abstract sentence, the Results text and Table 4 exactly as quoted above.
- Supplement. The only supplement (bmjgh-2016-000103supp.pdf, "Annex 1. Proportions of correct IPC events before and after the workshop, by district") gives counts and percentages by district but no risk ratios or intervals. Its handwashing counts sum to Table 4's 74/90 and 130/131. It neither explains nor corrects the abstract's figures.
- Corrections. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no notices.
- PubPeer. The PubPeer API and search page refused automated access (HTTP 403; human verification required). A web search for the title together with "pubpeer" found no PubPeer thread. Search engines do not reliably index PubPeer, so a manual check at pubpeer.com is still advisable.
- Sources. PMC5321376 (read with
src/show.py);src/known.py 10.1136/bmjgh-2016-000103(Crossref, PMC and Europe PMC notices); https://doi.org/10.1136/bmjgh-2016-000103; https://edoc.rki.de/bitstream/handle/176904/2475/27svLccxFCudc.pdf (typeset PDF); Europe PMC fullTextXML and supplementaryFiles for PMC5321376.
Suggested correction
The abstract's Results sentence should report the paper's primary, clustered analysis. Using Table 4 it would read, for example: "Maintenance of distance during screening (91% to 99%; RR 1.11, 95% CI 0.83 to 1.48) and patient handwashing (82% to 99%; RR 1.27, 95% CI 0.95 to 1.71) reached >90%, although neither change was statistically significant; HCW prompting of patient handwashing (RR 1.45, 95% CI 1.16 to 1.80) and handwashing on prompting (RR 1.49, 95% CI 1.19 to 1.86) increased significantly." If the authors intended to report crude estimates in the abstract, that should be stated, the handwashing figure should read 1.21 (1.10 to 1.33), and the abstract should note that the clustered analysis is not significant. Separately, the Results text's handwashing estimate should read RR 1.27 to match its interval and Table 4. The Discussion already notes that significant improvements were not consistent across behaviours and needs no change.
Recomputation script (python recompute.py.txt)
"""Recompute the patient-handwashing and distance risk ratios of PMC5321376
(BMJ Glob Health 2016;1:e000103, doi:10.1136/bmjgh-2016-000103).
Run: python recompute.py.txt # numbers + fig1, fig2
python recompute.py.txt --candidates DIR # also write design1..5 to DIR
python recompute.py.txt --fig1 3 --fig2 0 # use design 3 as fig1, no fig2
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png (and fig2.svg/png unless --fig2 0) next to this file. Five candidate
figure designs exist; fig1/fig2 are design1/design2 unless chosen otherwise.
Inputs are copied from the paper as archived in PMC (identical in the typeset PDF):
Abstract, Results: distance RR 1.09 (1.02 to 1.16); handwashing RR 1.19 (1.3 to 1.25)
Results text: handwashing RR 1.21 (0.95 to 1.71); stood 1.5 m RR 1.11 (0.83 to 1.48)
Table 4: counts, percentages, GEE risk ratios and 95% CIs
Supplement (Annex 1): handwashing counts by district (Bo, Kenema)
"""
import argparse
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
ap = argparse.ArgumentParser()
ap.add_argument("--candidates", help="directory to write candidate designs 1-5 into")
ap.add_argument("--fig1", type=int, default=1, help="design number used as fig1")
ap.add_argument("--fig2", type=int, default=2, help="design number used as fig2 (0 = none)")
ARGS = ap.parse_args()
Z = 1.959964
# ---------------------------------------------------------------- inputs (from the paper)
# Table 4 rows: (baseline correct, baseline n, follow-up correct, follow-up n, RR, lo, hi)
# Screening block: baseline n = 90, follow-up n = 131. Consultation block: 54 and 32.
TABLE4 = {
"Screener asked patient to wash hands": (56, 90, 105, 131, 1.45, 1.16, 1.80),
"Patient washed hands on direction from HCW": (54, 90, 105, 131, 1.49, 1.19, 1.86),
"Patient washed hands directly or on direction (dagger)": (74, 90, 130, 131, 1.27, 0.95, 1.71),
"Stood 1.5 m from patient, screening (dagger)": (82, 90, 130, 131, 1.11, 0.83, 1.48),
"Stood 1.5 m from patient, consultation": (35, 54, 29, 32, 1.18, 0.92, 1.51),
}
TABLE4_PCT = { # printed per cent columns (baseline, follow-up)
"Screener asked patient to wash hands": (62, 80),
"Patient washed hands on direction from HCW": (60, 80),
"Patient washed hands directly or on direction (dagger)": (82, 99),
"Stood 1.5 m from patient, screening (dagger)": (91, 99),
"Stood 1.5 m from patient, consultation": (65, 91),
}
ABSTRACT = {"handwashing": (1.19, 1.3, 1.25), "distance": (1.09, 1.02, 1.16)}
RESULTS_TEXT = {"handwashing": (1.21, 0.95, 1.71), "distance": (1.11, 0.83, 1.48)}
ANNEX_HANDWASH = {"Bo": ((34, 45), (58, 58)), "Kenema": ((40, 45), (72, 73))}
HW = "Patient washed hands directly or on direction (dagger)"
DS = "Stood 1.5 m from patient, screening (dagger)"
DC = "Stood 1.5 m from patient, consultation"
def crude_rr(a1, n1, a0, n0):
"""Unclustered risk ratio, follow-up vs baseline, with a log-scale Wald 95% CI."""
rr = (a1 / n1) / (a0 / n0)
se = math.sqrt(1 / a1 - 1 / n1 + 1 / a0 - 1 / n0)
return rr, rr * math.exp(-Z * se), rr * math.exp(Z * se), se
def geo_centre(lo, hi):
return math.sqrt(lo * hi)
def log_se(lo, hi):
return math.log(hi / lo) / (2 * Z)
R = {} # recomputed numbers used by the figures
print("== 1. The abstract's handwashing interval ==")
rr, lo, hi = ABSTRACT["handwashing"]
print(f"abstract RR {rr}, 95% CI {lo} to {hi}")
print(f" lower limit <= upper limit? {lo <= hi} ({lo} > {hi})")
print(f" interval contains the RR? {min(lo, hi) <= rr <= max(lo, hi)}")
print(f" with the limits swapped (1.25 to 1.3), contains 1.19? {1.25 <= rr <= 1.3}")
print(f" hypothetical '1.13 to 1.25': geometric centre {geo_centre(1.13, 1.25):.3f}, "
f"implied log-scale SE {log_se(1.13, 1.25):.4f}")
print("\n== 2. Table 4 counts: percentages, crude RR, and the centre of each printed CI ==")
for k, (b, n0, f, n1, RR, L, H) in TABLE4.items():
r, l, h, se = crude_rr(f, n1, b, n0)
R[k] = (r, l, h, se)
pb, pf = TABLE4_PCT[k]
print(f"{k}:")
print(f" baseline {b}/{n0} = {100*b/n0:.1f}% (printed {pb}); follow-up {f}/{n1} = "
f"{100*f/n1:.1f}% (printed {pf})")
print(f" crude unclustered RR {r:.3f} (Wald 95% CI {l:.3f} to {h:.3f}); log SE {se:.4f}")
print(f" Table 4 GEE RR {RR} ({L} to {H}); geometric centre of the printed CI "
f"{geo_centre(L, H):.3f}; implied robust log SE {log_se(L, H):.4f}")
print("\n== 3. Where the abstract's numbers come from ==")
r, l, h, _ = R[DS]
print(f"distance: abstract 1.09 (1.02 to 1.16) vs crude screening distance "
f"{r:.2f} ({l:.2f} to {h:.2f}) -> exact match after rounding: "
f"{(round(r, 2), round(l, 2), round(h, 2)) == (1.09, 1.02, 1.16)}")
r2, l2, h2, _ = R[DC]
print(f" consultation distance, for contrast: crude {r2:.2f} ({l2:.2f} to {h2:.2f}); "
f"GEE 1.18 (0.92 to 1.51)")
r, l, h, se = R[HW]
print(f"handwashing: abstract 1.19 (1.3 to 1.25); crude {r:.2f} ({l:.2f} to {h:.2f}); "
f"Table 4 GEE 1.27 (0.95 to 1.71)")
print(f" a CI of 1.13 to 1.25 needs log SE {log_se(1.13, 1.25):.4f}, "
f"{log_se(1.13, 1.25)/se:.2f} x the crude unclustered SE ({se:.4f}) and "
f"{log_se(1.13, 1.25)/log_se(0.95, 1.71):.2f} x the GEE robust SE ({log_se(0.95, 1.71):.4f})")
for k, nm in ((HW, "handwashing"), (DS, "distance, screening")):
L, H = TABLE4[k][5], TABLE4[k][6]
print(f" {nm}: GEE robust log SE / crude log SE = {log_se(L, H)/R[k][3]:.2f}")
print("\n== 4. Results text: handwashing RR 1.21 (0.95 to 1.71) ==")
print(f" crude point estimate {(130/131)/(74/90):.3f}; geometric centre of 0.95-1.71 = "
f"{geo_centre(0.95, 1.71):.3f} (Table 4 prints 1.27)")
print(f" does 0.95-1.71 exclude 1? {0.95 > 1}; does the abstract's distance CI exclude 1? "
f"{1.02 > 1}; Table 4 distance CI 0.83-1.48 excludes 1? {0.83 > 1}")
print("\n== 5. Supplement Annex 1, handwashing by district ==")
bt = sum(v[0][0] for v in ANNEX_HANDWASH.values()); bn = sum(v[0][1] for v in ANNEX_HANDWASH.values())
ft = sum(v[1][0] for v in ANNEX_HANDWASH.values()); fn = sum(v[1][1] for v in ANNEX_HANDWASH.values())
print(f" baseline {bt}/{bn}, follow-up {ft}/{fn} (Table 4: 74/90, 130/131)")
# ---------------------------------------------------------------- figure style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
LW, MS = 2.0, 8
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg", bbox_inches="tight")
fig.savefig(p + ".png", dpi=200, bbox_inches="tight")
plt.close(fig)
def log_axis(ax, lo=0.8, hi=1.9, ticks=(0.8, 0.9, 1.0, 1.1, 1.2, 1.3, 1.5, 1.7)):
ax.set_xscale("log")
ax.set_xlim(lo, hi)
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.axvline(1.0, color=INK2, lw=0.9, zorder=1)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
def interval(ax, y, est, lo, hi, color, marker, label=None, dashed=False):
ax.plot([lo, hi], [y, y], color=color, lw=LW, ls="--" if dashed else "-",
solid_capstyle="butt", zorder=2)
ax.plot([lo, lo], [y - 0.12, y + 0.12], color=color, lw=LW, zorder=2)
ax.plot([hi, hi], [y - 0.12, y + 0.12], color=color, lw=LW, zorder=2)
ax.plot(est, y, marker, color=color, ms=MS, mec=BG, mew=1.5, zorder=3, label=label)
hw_c, ds_c = R[HW], R[DS]
# ---------------------------------------------------------------- design 1: forest plot
def design1():
fig, ax = plt.subplots(figsize=(4.72, 4.4))
rows = [
# y, label, (est, lo, hi), kind
(7, "Abstract", (1.19, 1.3, 1.25), "abs"),
(6, "Results text", RESULTS_TEXT["handwashing"], "rep"),
(5, "Table 4 (GEE)", (1.27, 0.95, 1.71), "rep"),
(4, "Crude, ignoring clustering", hw_c[:3], "rec"),
(2.5, "Abstract", ABSTRACT["distance"], "rep"),
(1.5, "Table 4 / text (GEE)", (1.11, 0.83, 1.48), "rep"),
(0.5, "Crude, ignoring clustering", ds_c[:3], "rec"),
]
for y, lab, (e, l, h), kind in rows:
if kind == "abs":
# printed limits drawn where they sit; the 'interval' runs backwards
ax.plot([1.25, 1.3], [y, y], color=ORANGE, lw=LW, ls=(0, (3, 2)), zorder=2)
ax.plot(1.3, y, marker="|", color=ORANGE, ms=14, mew=LW)
ax.plot(1.25, y, marker="|", color=ORANGE, ms=14, mew=LW)
ax.plot(e, y, "o", color=ORANGE, ms=MS, mec=BG, mew=1.5, zorder=3)
ax.annotate("lower 1.3", (1.3, y), (1.3, y + 0.42), ha="center", fontsize=10, color=INK2)
ax.annotate("upper 1.25", (1.25, y), (1.235, y - 0.55), ha="center", fontsize=10, color=INK2)
ax.text(1.02, y, "1.19 (1.3 to 1.25):\nlower limit > upper;\n1.19 lies outside", va="center",
ha="left", fontsize=10, color=INK,
transform=matplotlib.transforms.blended_transform_factory(ax.transAxes, ax.transData))
else:
c, m = (ORANGE, "o") if kind == "rep" else (BLUE, "s")
interval(ax, y, e, l, h, c, m)
txt = f"{e:.2f} ({l:.2f} to {h:.2f})"
ax.text(1.02, y, txt, va="center", ha="left", fontsize=10, color=INK,
transform=matplotlib.transforms.blended_transform_factory(ax.transAxes, ax.transData))
ax.set_yticks([r[0] for r in rows])
ax.set_yticklabels([r[1] for r in rows], fontsize=10, color=INK)
ax.text(0.0, 1.0, "Patient handwashing, 74/90 (82%) \u2192 130/131 (99%)", transform=ax.transAxes,
fontsize=10.5, weight="bold", ha="left", va="bottom")
ax.text(0.8, 3.15, "Stood 1.5 m at screening, 82/90 (91%) \u2192 130/131 (99%)", fontsize=10.5,
weight="bold", ha="left", va="bottom", transform=ax.transData,
bbox=dict(fc=BG, ec="none", pad=1.5), zorder=4)
log_axis(ax, 0.8, 1.8)
ax.set_ylim(-0.1, 7.7)
ax.set_xlabel("Risk ratio, follow-up vs baseline (log scale); line at 1 = no change")
h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="Reported in the paper")
h2, = ax.plot([], [], "s", color=BLUE, ms=MS,
label="Recomputed from Table 4 counts (unclustered Wald)")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.16), ncol=1,
frameon=False, fontsize=10)
return fig
# ---------------------------------------------------------------- design 2: two text panels
def box(ax, x, y, w, h, title, lines, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.01,rounding_size=0.015",
fc="#ffffff", ec=edge, lw=1.5))
ax.text(x + 0.02, y + h - 0.035, title, weight="bold", fontsize=10.5, va="top")
ax.text(x + 0.02, y + h - 0.10, "\n".join(lines), fontsize=10, va="top", color=INK,
linespacing=1.45)
def design2():
fig = plt.figure(figsize=(7.2, 3.0))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(-0.015, 1.015); ax.set_ylim(-0.025, 1.0); ax.axis("off")
box(ax, 0.0, 0.47, 0.41, 0.51, "Abstract, Results",
["\u201cmaintenance of distance", "(RR 1.09, 95% CI 1.02 to 1.16)",
"and patient handwashing", "(RR 1.19, 95% CI 1.3 to 1.25)",
"improved to >90%\u201d"], ORANGE)
box(ax, 0.57, 0.47, 0.43, 0.51, "Results text / Table 4",
["Handwashing \u201cincreased", "though not significantly\u201d:",
"RR 1.21 (text), 1.27 (Table 4),", "both 95% CI 0.95 to 1.71",
"Stood 1.5 m: RR 1.11 (0.83 to 1.48)"], ORANGE)
box(ax, 0.0, 0.0, 1.0, 0.29, "Recomputed from Table 4 counts, ignoring clustering",
[f"Stood 1.5 m, screening: {ds_c[0]:.2f} ({ds_c[1]:.2f} to {ds_c[2]:.2f}), same as abstract",
f"Patient handwashing: {hw_c[0]:.2f} ({hw_c[1]:.2f} to {hw_c[2]:.2f}), not the abstract's "
"1.19 (1.3 to 1.25)"],
BLUE)
ax.annotate("", xy=(0.565, 0.74), xytext=(0.415, 0.74),
arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
ax.text(0.49, 0.78, "conflict", ha="center", fontsize=10, weight="bold")
ax.text(0.49, 0.70, "CIs > 1\nvs\nCIs span 1", ha="center", va="top", fontsize=10,
color=INK2)
ax.annotate("", xy=(0.22, 0.30), xytext=(0.22, 0.46),
arrowprops=dict(arrowstyle="<-", color=BLUE, lw=2))
ax.text(0.235, 0.38, "distance matches crude", fontsize=10, color=INK2, va="center")
return fig
# ---------------------------------------------------------------- design 3: zoomed number line
def design3():
fig, ax = plt.subplots(figsize=(6.52, 2.9))
y_abs, y_rec, y_gee = 2.0, 1.0, 0.0
ax.plot([1.25, 1.3], [y_abs, y_abs], color=ORANGE, lw=LW, ls=(0, (3, 2)))
for x, lab, dy in [(1.3, "printed lower limit 1.3", 0.32), (1.25, "printed upper limit 1.25", -0.45)]:
ax.plot(x, y_abs, marker="|", color=ORANGE, ms=16, mew=LW)
ax.text(x, y_abs + dy, lab, ha="center", fontsize=10, color=INK2)
ax.plot(1.19, y_abs, "o", color=ORANGE, ms=MS, mec=BG, mew=1.5)
ax.text(1.175, y_abs, "RR 1.19", ha="right", va="center", fontsize=10, color=INK2)
interval(ax, y_rec, *hw_c[:3], BLUE, "s")
ax.text(hw_c[2] * 1.01, y_rec, f"{hw_c[0]:.2f} ({hw_c[1]:.2f} to {hw_c[2]:.2f})", va="center", fontsize=10)
interval(ax, y_gee, 1.27, 0.95, 1.71, ORANGE, "o")
ax.text(1.72, y_gee, "1.27 (0.95 to 1.71)", va="center", fontsize=10)
ax.set_yticks([y_abs, y_rec, y_gee])
ax.set_yticklabels(["Abstract (reported)", "Crude from Table 4 counts\n(recomputed)",
"Table 4 GEE (reported)"], color=INK)
log_axis(ax, 0.9, 2.1, (0.9, 1.0, 1.1, 1.2, 1.3, 1.5, 1.7, 2.0))
ax.set_ylim(-0.6, 2.7)
ax.set_xlabel("Patient handwashing risk ratio (log scale); line at 1 = no change")
return fig
# ---------------------------------------------------------------- design 4: provenance grid
def design4():
fig = plt.figure(figsize=(7.2, 3.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2); ax.set_ylim(0, 3.4); ax.axis("off")
cx = (1.9, 3.42, 4.94) # column centres, in inches
cols = ["Abstract\n(reported)", "Crude from counts\n(recomputed)", "Table 4 GEE\n(reported)"]
for j, c in enumerate(cols):
ax.text(cx[j], 3.1, c, ha="center", va="center", fontsize=10, weight="bold",
color=ORANGE if j != 1 else BLUE)
rows = [("Stood 1.5 m,\nscreening", "1.09 (1.02\u20131.16)",
f"{ds_c[0]:.2f} ({ds_c[1]:.2f}\u2013{ds_c[2]:.2f})", "1.11 (0.83\u20131.48)",
"same", "differs"),
("Patient\nhandwashing", "1.19 (1.3\u20131.25)",
f"{hw_c[0]:.2f} ({hw_c[1]:.2f}\u2013{hw_c[2]:.2f})", "1.27 (0.95\u20131.71)",
"differs", "differs")]
for i, (lab, a, c, g, m1, m2) in enumerate(rows):
y = 2.2 - i * 1.0
ax.text(0.05, y, lab, va="center", fontsize=10, weight="bold")
for j, v in enumerate((a, c, g)):
ax.add_patch(FancyBboxPatch((cx[j] - 0.725, y - 0.3), 1.45, 0.6,
boxstyle="round,pad=0.0,rounding_size=0.06",
fc="#ffffff", ec=BLUE if j == 1 else ORANGE, lw=1.5))
ax.text(cx[j], y, v, ha="center", va="center", fontsize=10)
ax.text(5.78, y, f"vs crude: {m1}\nvs GEE: {m2}", ha="left",
va="center", fontsize=10, color=INK2)
ax.text(0.05, 0.35, "Only the GEE intervals include 1 (no significant change).\n"
"The abstract's handwashing interval has its lower limit above its upper limit.",
fontsize=10, color=INK2, va="center")
return fig
# ---------------------------------------------------------------- design 5: crude vs GEE, all rows
def design5():
keys = [HW, DS, "Screener asked patient to wash hands",
"Patient washed hands on direction from HCW", DC]
labels = ["Patient washed hands\n(direct or prompted)", "Stood 1.5 m, screening",
"Screener asked patient\nto wash hands", "Patient washed hands\non direction",
"Stood 1.5 m, consultation"]
fig, ax = plt.subplots(figsize=(6.62, 4.4))
for i, k in enumerate(keys):
y = len(keys) - 1 - i
b, n0, f, n1, RR, L, H = TABLE4[k]
interval(ax, y + 0.17, RR, L, H, ORANGE, "o")
c = R[k]
interval(ax, y - 0.17, c[0], c[1], c[2], BLUE, "s")
# abstract values overlaid as open diamonds
ax.plot(1.19, 4.0, "D", mfc="none", mec=ORANGE, ms=MS, mew=LW)
ax.plot(1.09, 3.0, "D", mfc="none", mec=ORANGE, ms=MS, mew=LW)
ax.text(1.19, 4.48, "abstract 1.19\n(CI printed 1.3 to 1.25)", ha="center", fontsize=10, color=INK2)
ax.text(1.02, 2.62, "abstract 1.09 (1.02 to 1.16)", ha="left", fontsize=10, color=INK2, va="top")
ax.set_yticks(range(len(keys)))
ax.set_yticklabels(labels[::-1], color=INK)
log_axis(ax, 0.8, 2.0, (0.8, 0.9, 1.0, 1.2, 1.5, 2.0))
ax.set_ylim(-0.6, 5.15)
ax.set_xlabel("Risk ratio, follow-up vs baseline (log scale); line at 1 = no change")
h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="Table 4 GEE (reported)")
h2, = ax.plot([], [], "s", color=BLUE, ms=MS, label="Crude Wald from Table 4 counts (recomputed)")
h3, = ax.plot([], [], "D", mfc="none", mec=ORANGE, ms=MS, mew=LW, label="Abstract (reported)")
ax.legend(handles=[h1, h2, h3], loc="upper center", bbox_to_anchor=(0.4, -0.15), ncol=2,
frameon=False, fontsize=10)
return fig
designs = {1: design1, 2: design2, 3: design3, 4: design4, 5: design5}
if ARGS.candidates:
os.makedirs(ARGS.candidates, exist_ok=True)
for i, d in designs.items():
save(d(), os.path.join(ARGS.candidates, f"design{i}"))
print(f"\nwrote design1..5 (.svg, .png) to {ARGS.candidates}")
save(designs[ARGS.fig1](), os.path.join(HERE, "fig1"))
if ARGS.fig2:
save(designs[ARGS.fig2](), os.path.join(HERE, "fig2"))
print(f"wrote fig1 (design {ARGS.fig1})" + (f" and fig2 (design {ARGS.fig2})" if ARGS.fig2 else ""))
Ebola review calls case-fatality change non-significant while reporting OR 1.6, 95% CI excluding 1, p<0.001
BMJ Glob Health (2020) · doi:10.1136/bmjgh-2019-001955 · PMC7061886
Summary
This review of 34 Ebola virus disease outbreaks (1976–2019) states in its Abstract, Results and Abstract Conclusions that the case fatality rate (CFR) did not change significantly over four decades. It supports that statement with "OR in 2019 vs 1976=1.6 (95% CI 1.5 to 1.8), p<0.001". That interval excludes 1 and the P value is below 0.001, so the statistic contradicts the claim it is cited for. The Results text gives a different interval (1.5 to 1.6) for the same OR. The OR also points the wrong way for the paper's own CFRs (71% falling to 62% gives an OR of about 0.67, not 1.6). The Table 3 CFRs for three of the four periods and the overall CFR of "66% (62 to 71)" cannot be reproduced from the outbreak counts in Table 1. Those counts give a pooled CFR that falls from 71.3% to 41.5% (crude OR 0.29, 95% CI 0.24 to 0.34) and an overall CFR of 43.1%. The CFR trend is the review's only quantitative inference, and the Abstract Conclusions use it to support the claim that control challenges persist. As published, the reported statistics do not support the sentence they accompany. Whether the real trend is significant depends on an analysis the paper does not describe: a patient-level comparison finds a large decline, while analyses that allow for variation between outbreaks find no clear trend.
What the paper reports
Abstract, Methods:
Analyses of trends in case fatality were performed by calculating ORs between times.
Abstract, Results:
In the past four decades, a total of 34 EVD outbreaks affecting 34 356 cases and causing 14 823 deaths were reported in 11 countries in Sub-Saharan Africa. The overall case fatality rate (95% CI) was 66% (62 to 71) and did not change substantially over time (OR in 2019 vs 1976=1.6 (95% CI 1.5 to 1.8), p<0.001).
Abstract, Conclusions:
Sub-Saharan Africa continues to face considerable challenges in EVD control, whereby there are no significant changes in case fatality rates observed during the past four decades.
Methods, "Data extraction and analysis":
Additional analyses of trends in case fatality rates were performed by calculating ORs between periods using MedCalc statistical software (https://www.medcalc.org/calc/odds_ratio.php).
Results, "Context of EVD", fourth paragraph:
Case fatality rates varied from 71% during outbreaks in 1976–1986 to 62% during the later years. However, there was no statistically significant change over time (OR in 2019 vs 1976: 1.6 (95% CI 1.5 to 1.6)). The overall trend from 1976 to 2019 outbreaks shows a linear rate of change over time (table 3, figure 3).
Results, "Context of EVD", third paragraph:
During the past four decades, approximately 14 853 people have died from EVD outbreaks.
Table 3, "OR for case fatality rate of Ebola virus disease according to period" (columns: 1976–1986 | 1987–1997 | 1998–2008 | 2009–2019):
Death (n) | 454 | 348 | 733 | 13 288
Case fatality rate (%) | 71 | 67 | 64 | 62
OR (95% CI) | 1.6 (1.4 to 1.8) | 1.7 (1.5 to 1.9) | 1.4 (1.3 to 1.6) | 1.6 (1.5 to 1.6)
P value | <0.001 | <0.001 | <0.001 | <0.001
Table 1 lists each outbreak's year, country, virus, cases, deaths and CFR. Its total row:
Total | | | 34 356 | 14 823 |
Table 2, sex rows (columns: case | death | CFR (%)):
Gender | Male | 15 803 | 6373 | 46
Gender | Female | 18 553 | 8449 | 54
Table 3 does not state the comparison group or the direction of its ORs, and the paper does not say whether the analysis treated individual patients or outbreaks as the unit.
What the paper's own numbers show
Method. I entered the 34 outbreaks of Table 1 (year, cases, deaths) and assigned each multi-year outbreak (for example "2000–2001") to its first year. Grouping them into the four Table 3 periods reproduces Table 3's death counts exactly (454, 348, 733, 13 288), which confirms the grouping. For each period I computed the pooled CFR (deaths ÷ cases). I then computed crude odds ratios of death with Woolf 95% CIs, the CFR odds ratio implied by Table 3's own CFRs, and a chi-square test of equal CFR across periods. For each reported OR I also derived the z statistic its CI implies. These patient-level calculations treat every case as independent, as a crude OR from a 2×2 table does. The cases are concentrated in a few outbreaks: the 2014 outbreaks in Liberia and Sierra Leone alone account for 72.2% of all cases. So I also ran three outbreak-level analyses, which the paper does not report: a Spearman correlation of outbreak CFR with year, a quasi-binomial logistic model, and a DerSimonian–Laird random-effects meta-regression. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Does the cited OR show "no change"? | Abstract, Results | 1.6 (95% CI 1.5 to 1.8), p<0.001; "did not change substantially" | CI excludes 1; implied z = 10.1. By its own statistics the change is significant |
| Same OR in the body | Results text | 1.6 (95% CI 1.5 to 1.6); "no statistically significant change" | CI excludes 1 (implied z ≈ 28); also a different CI from the Abstract's for the same quantity |
| Table 3 ORs | Table 3 | 1.6, 1.7, 1.4, 1.6; all CIs above 1; all P < 0.001 | every CI excludes 1; none is a "no change" result |
| Direction of the OR, using Table 3's CFRs | Table 3; Results | 62% vs 71%, reported as "OR in 2019 vs 1976" = 1.6 | (0.62/0.38) ÷ (0.71/0.29) = 0.67 (0.56 to 0.79), p = 4.1 × 10⁻⁶; the inverse is 1.50 (1.26 to 1.78). A fall in CFR gives an OR below 1 |
| Deaths per period | Table 3 vs Table 1 | 454, 348, 733, 13 288 | 454, 348, 733, 13 288 (identical) |
| CFR, 1976–1986 | Table 3 | 71% | 454/637 = 71.3% |
| CFR, 1987–1997 | Table 3 | 67% | 348/467 = 74.5% |
| CFR, 1998–2008 | Table 3 | 64% | 733/1199 = 61.1% |
| CFR, 2009–2019 | Table 3; Results | 62% | 13 288/32 053 = 41.5% |
| OR of death, 2009–2019 vs 1976–1986 | from Table 1 | — | 0.285 (0.240 to 0.339), p ≈ 9 × 10⁻⁴⁶; the inverse is 3.50 |
| Equal CFR across the four periods | from Table 1 | "no statistically significant change" | χ² = 588, df = 3, p ≈ 4 × 10⁻¹²⁷ (patient level) |
| Overall CFR | Abstract | "66% (62 to 71)", labelled a 95% CI | pooled 14 823/34 356 = 43.1% (binomial 95% CI 42.6 to 43.7); unweighted mean of the 34 outbreak CFRs 61.8%. 66.0% is the mean of the four Table 3 CFRs and 62–71 is their range |
| Total deaths | Table 1; Abstract vs Results text | 14 823 vs "approximately 14 853" | Table 1 rows sum to 14 823; the text differs by 30 |
| CFR by sex | Table 2 | male 46, female 54 | male 6373/15 803 = 40.3%, female 8449/18 553 = 45.5%. 46.0% and 54.0% are each sex's share of all cases |
| Outbreak-level trend (not in the paper) | from Table 1 | — | Spearman ρ = −0.16 (p = 0.35); quasi-binomial OR 2009–2019 vs 1976–1986 0.29 (0.04 to 2.09), p = 0.21, dispersion 122; random-effects meta-regression OR per decade 0.81 (0.62 to 1.05), p = 0.11 |
The table supports three findings.
- The reported inference contradicts the reported wording. Every OR the paper prints (Abstract, Results, the four Table 3 columns) has a CI that excludes 1 and a P value below 0.001. The sentences that cite them say there was no significant change.
- The reported OR does not match the reported CFRs. For CFRs of 71% and then 62%, the OR of death for the later period is 0.67. Its inverse, 1.50 (1.26 to 1.78 with the Table 1 period sizes), is close to the reported 1.6 but describes the earlier period against the later. Neither the Abstract's CI (1.5 to 1.8) nor the Results' CI (1.5 to 1.6) is reproduced.
- Table 3's CFRs, apart from the first period, do not follow from Table 1. Table 3's death counts match Table 1 exactly. Its CFRs for 1987–1997, 1998–2008 and 2009–2019 differ from deaths ÷ cases by −7.5, +2.9 and +20.5 percentage points. The largest gap is in the period that holds 93% of all cases (32 053 of 34 356). They do not match the unweighted means of the outbreak CFRs either (76.5%, 53.5%, 64.9%, 58.9%). The Abstract's "66% (62 to 71)" is reproduced exactly by the mean and range of the four Table 3 CFRs, not by any pooled estimate or confidence interval.
Figure
Figure 1. Odds ratios of death with 95% CIs on a log scale. The thin vertical line is OR = 1 (no change). Orange: every OR the paper reports for the CFR trend (Abstract, Results text, and the four columns of Table 3). All lie to the right of 1 with intervals that exclude it, and all carry P < 0.001, yet the paper says there was no significant change. Blue diamonds are recomputed from the paper's own numbers. The upper one is the OR implied by Table 3's CFRs (62% vs 71%). The lower one is the crude OR for 2009–2019 vs 1976–1986 from the Table 1 counts. Both lie to the left of 1, the opposite direction from the reported 1.6. The hollow blue square is the same Table 1 comparison with a CI widened for variation between outbreaks (quasi-binomial logistic model, dispersion 122). It is not an analysis the paper reports. It shows that the data are compatible with no change once outbreaks rather than patients are treated as the unit.
Figure 2. Case fatality rate for each Table 3 period and overall, on an axis from 0 to 100%. Orange circles: Table 3's CFRs and the Abstract's overall 66%. Blue diamonds: deaths ÷ cases from the outbreak rows of Table 1, grouped so that the deaths equal Table 3's. The first period agrees (71% vs 71.3%). The others differ, most of all in 2009–2019 (62% vs 41.5%) and overall (66% vs 43.1%).
Why it matters
The CFR trend is the only quantitative inference in the review. It is stated in the Abstract Results, the Abstract Conclusions and the Results text, and the Abstract Conclusions present it as evidence for the review's framing ("continues to face considerable challenges in EVD control, whereby there are no significant changes in case fatality rates"). A reader who takes the Abstract at face value gets a claim (no significant change), a statistic that contradicts it (CI excluding 1, p<0.001), and an overall CFR (66%) that is about 23 percentage points above the pooled figure implied by the paper's own outbreak table (43.1%).
The paper's numbers do not settle what the correct conclusion is, and this write-up does not claim to. Two readings are defensible, and neither matches the paper.
- Patient-level (as a crude OR implies). The CFR fell substantially and significantly, from 71.3% to 41.5% (OR 0.29, 95% CI 0.24 to 0.34). Under this reading the conclusion "no significant changes" is reversed.
- Outbreak-level. The fall is driven largely by the very large 2014 West African outbreaks (Sierra Leone 26%, Liberia 45%), and CFR varies greatly between outbreaks (Pearson dispersion 122). Allowing for that variation, the decline is not statistically significant (quasi-binomial p = 0.21; random-effects meta-regression OR per decade 0.81, 95% CI 0.62 to 1.05; Spearman ρ = −0.16, p = 0.35). Under this reading "no clear trend" may be defensible. It would then rest on a different analysis, with intervals that include 1, not on the published OR of 1.6 with p<0.001.
Either way, the published statistics cannot support the published sentence, and the overall CFR and three of the four period CFRs are not what the paper's own table implies. The review's qualitative synthesis (epidemiological, health-system and sociocultural challenges) and its recommendations (one-health surveillance, community engagement, research on diagnostics, drugs and vaccines, health-system resilience) do not depend on these numbers.
Explanations considered
- "Did not change substantially" means not clinically important, not non-significant. This does not account for the text. The Results say "no statistically significant change" and the Abstract Conclusions say "no significant changes", and the pooled CFR difference from Table 1 (about 30 percentage points) is large on any clinical scale.
- The OR compares each period with a reference, and 1.6 is in the stated direction. This does not account for the numbers. Table 3 does not name a reference group. If "2019 vs 1976" means the later period against the earlier, any fall in CFR gives an OR below 1: 0.67 from Table 3's CFRs, 0.29 from Table 1's counts. An OR of 1.6 can describe only the earlier period against the later. Even then, a CI excluding 1 means a significant change.
- The OR compares single years, 2019 against 1976. This does not rescue it. The only 2019 outbreak in Table 1 (Uganda) had 4 deaths among 4 cases, so the OR is undefined. Using the 2018–2019 DRC outbreak (2150/3220 = 66.8%) against the 1976 outbreaks (431/602 = 71.6%) gives OR 0.80 (0.66 to 0.97), p = 0.021, which is still below 1 and still significant.
- The Table 3 ORs are some other ratio. I tried several standard quantities: pairwise ORs between periods; each period's OR, risk ratio and CFR ratio against all other periods; and their inverses. None reproduces all four ORs with their CIs. The risk ratios of the first three periods against all others (1.67, 1.74, 1.44) are close to the reported 1.6, 1.7 and 1.4. Their CIs differ (1.59–1.76, 1.65–1.84, 1.37–1.51), and the fourth period's risk ratio is 0.62, not 1.6. The paper's method cannot be identified from what is published.
- Table 3's CFRs are outbreak-averaged rather than pooled. This does not account for the numbers. The unweighted means of outbreak CFRs per period are 76.5%, 53.5%, 64.9% and 58.9%, not 71, 67, 64 and 62. The overall mean across 34 outbreaks is 61.8%, not 66%.
- "66% (62 to 71)" is a genuine 95% CI from a model. Possible, but the numbers coincide exactly with the arithmetic mean (66.0%) and range (62–71) of the four Table 3 CFRs. A pooled estimate from Table 1 would be 43.1%, with a patient-level 95% CI of 42.6 to 43.7, about 1 point wide rather than 9.
- The statement is correct under an outbreak-level analysis. This is partly possible, as shown above: the trend is not significant once between-outbreak variation is allowed for. But the paper reports no such analysis, and the statistics it does report (CIs excluding 1, P < 0.001) are the opposite of a non-significant result. The text and its supporting statistics cannot both be right as printed.
- A conversion or typesetting artefact in PMC. Excluded. The publisher's PDF carries the same Abstract, Results and Table 3 text (see Status).
- Different case counts in Table 3 than in Table 1. Table 3 prints no case counts, but its deaths equal Table 1's period sums exactly. A different set of cases would have to produce identical deaths in all four periods, which is implausible.
Status
- Version of record. The live publisher pages (doi.org → gh.bmj.com) refused automated access (HTTP 403), and the PMC HTML page returned a bot check. The publisher's PDF (gh.bmj.com/content/bmjgh/5/3/e001955.full.pdf, as archived by the Internet Archive on 2023-12-05) was text-extracted. It carries the same Abstract sentences, the same Results sentence with "1.6 (95% CI 1.5 to 1.6)", and the same Table 3 cells as the PMC and Europe PMC XML. Every quotation above was confirmed verbatim against the PMC-archived text.
- Supplements and data. The article has no supplementary data. The Europe PMC supplementary bundle contains only the three figure images, the JATS XML has no supplementary-material element, and the data availability statement reads "Data are available upon request." Figure 3 (cases by year with a linear CFR trendline) reports no OR or test and was not reproduced here.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations (
update-toabsent,relationempty). PMC and Europe PMC list no correction or comment links (PMID 32201623). The Crossref Labs record, which includes Retraction Watch data, lists no updates, and the archived publisher landing page shows no notice. The PubPeer API refused automated access in this session (Cloudflare challenge). A web search for the title with "correction", "erratum" or "PubPeer" found only unrelated errata. - Sources. PMC article PMC7061886; Europe PMC full-text XML; publisher PDF (Internet Archive copy); Crossref REST API; Europe PMC search; web search.
Suggested correction
- Replace "did not change substantially over time" (Abstract), "there was no statistically significant change over time" (Results) and "there are no significant changes in case fatality rates" (Abstract Conclusions) with wording that matches whatever analysis is reported. The current ORs, with CIs excluding 1 and P < 0.001, indicate a significant difference.
- State the comparison and direction of every OR in Table 3 (which period against which reference), and the unit of analysis. Report a single, consistent CI for the "2019 vs 1976" OR; the Abstract gives 1.5 to 1.8 and the Results 1.5 to 1.6.
- Recompute the Table 3 CFRs as deaths ÷ cases from Table 1 (71.3%, 74.5%, 61.1%, 41.5%) and add the case counts to the table, or explain how the printed 67%, 64% and 62% were derived.
- Replace the overall "66% (62 to 71)" with the pooled CFR from Table 1 (43.1%) and a valid interval, or label it as the mean and range of the period CFRs if that is what it is.
- Because CFR varies widely between outbreaks and one region contributes most of the cases, consider a trend analysis that accounts for clustering by outbreak (for example random-effects or quasi-binomial models). Base the conclusion on its estimate and interval.
- Minor: correct "approximately 14 853" deaths to 14 823 (Table 1 total), and change the Table 2 "CFR (%)" entries for sex (46, 54) to 40.3 and 45.5, or relabel the column as the share of cases. The Table 2 age groups sum to 33 456 cases, 900 fewer than the 34 356 total, and the table does not say how the remaining cases are classified.
Recomputation script (python recompute.py.txt)
"""Recompute the case-fatality trend statistics of PMC7061886
(BMJ Glob Health 2020, doi:10.1136/bmjgh-2019-001955).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Table 1: year, cases and deaths of each of the 34 outbreaks (printed CFR column also kept)
Table 2: cases and deaths by sex; cases by age group and source of infection
Table 3: deaths, CFR (%), OR (95% CI) and P value for four periods
Abstract and Results text: overall CFR, ORs "in 2019 vs 1976"
Multi-year outbreaks in Table 1 (e.g. "2000-2001") are assigned to their first year.
With that rule the period death counts equal Table 3's exactly.
"""
import itertools
import math
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# ------------------------------------------------------------------ inputs (from the paper)
# Table 1: (first year, country, cases, deaths, printed CFR %)
T1 = [
(1976, "DRC", 318, 280, 88), (1976, "South Sudan", 284, 151, 53), (1977, "DRC", 1, 1, 100),
(1979, "South Sudan", 34, 22, 65), (1994, "Gabon", 52, 31, 60), (1994, "Cote d'Ivoire", 1, 0, 0),
(1995, "DRC", 315, 250, 79), (1996, "Gabon", 37, 21, 57), (1996, "Gabon", 60, 45, 75),
(1996, "South Africa", 2, 1, 50), (2000, "Uganda", 425, 224, 53), (2001, "Gabon", 65, 53, 82),
(2001, "Rep. Congo", 57, 43, 75), (2002, "Rep. Congo", 143, 128, 89), (2003, "Rep. Congo", 35, 29, 83),
(2004, "South Sudan", 17, 7, 41), (2005, "Rep. Congo", 12, 10, 83), (2007, "DRC", 264, 187, 71),
(2007, "Uganda", 149, 37, 25), (2008, "DRC", 32, 15, 47), (2011, "Uganda", 1, 1, 100),
(2012, "Uganda", 11, 4, 36), (2012, "DRC", 36, 13, 36), (2012, "Uganda", 6, 3, 50),
(2014, "DRC", 69, 49, 71), (2014, "Guinea", 3814, 2544, 67), (2014, "Liberia", 10678, 4810, 45),
(2014, "Sierra Leone", 14124, 3659, 26), (2014, "Nigeria", 20, 8, 40), (2014, "Mali", 8, 6, 75),
(2017, "DRC", 8, 4, 50), (2018, "DRC", 54, 33, 61), (2018, "DRC", 3220, 2150, 67),
(2019, "Uganda", 4, 4, 100),
]
T1_TOTAL = (34356, 14823) # Table 1 "Total" row; also Abstract
TEXT_DEATHS = 14853 # Results, "Context of EVD": "approximately 14 853 people have died"
PERIODS = [(1976, 1986), (1987, 1997), (1998, 2008), (2009, 2019)]
LABELS = ["1976–1986", "1987–1997", "1998–2008", "2009–2019"]
T3_DEATHS = [454, 348, 733, 13288]
T3_CFR = [71, 67, 64, 62]
T3_OR = [(1.6, 1.4, 1.8), (1.7, 1.5, 1.9), (1.4, 1.3, 1.6), (1.6, 1.5, 1.6)] # all P < 0.001
ABS_OR = (1.6, 1.5, 1.8) # Abstract: "OR in 2019 vs 1976=1.6 (95% CI 1.5 to 1.8), p<0.001"
RES_OR = (1.6, 1.5, 1.6) # Results: "OR in 2019 vs 1976: 1.6 (95% CI 1.5 to 1.6)"
ABS_CFR = (66, 62, 71) # Abstract: "overall case fatality rate (95% CI) was 66% (62 to 71)"
T2_SEX = {"Male": (15803, 6373, 46), "Female": (18553, 8449, 54)} # cases, deaths, printed CFR
T2_AGE = {"0–15": 7215, "15–60": 24049, "60+": 2192}
T2_SOURCE = {"Health facility": 13742, "Household": 20614}
def orci(a, b, c, d):
"""OR of a/b vs c/d with Woolf (log) 95% CI and two-sided Wald P."""
o = (a * d) / (b * c)
se = math.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
lo, hi = math.exp(math.log(o) - Z * se), math.exp(math.log(o) + Z * se)
return o, lo, hi, 2 * stats.norm.sf(abs(math.log(o) / se))
yr = np.array([r[0] for r in T1], float)
n = np.array([r[2] for r in T1], float)
dth = np.array([r[3] for r in T1], float)
C, D = int(n.sum()), int(dth.sum())
print("=== 1. Totals (Table 1) ===")
print(f"sum of 34 outbreaks: cases {C} (Table 1 total {T1_TOTAL[0]}), deaths {D} (Table 1 total {T1_TOTAL[1]}; "
f"Results text {TEXT_DEATHS}, difference {TEXT_DEATHS - D})")
print(f"pooled overall CFR = {D}/{C} = {100 * D / C:.1f}%")
print("Table 1 printed CFR column vs deaths/cases, where the rounding differs:")
for r in T1:
if int(100 * r[3] / r[2] + 0.5) != r[4]:
print(f" {r[0]} {r[1]}: {r[3]}/{r[2]} = {100 * r[3] / r[2]:.2f}% printed {r[4]}")
print("\n=== 2. Periods of Table 3, rebuilt from Table 1 ===")
P = []
for (a, b), lab, rd, rc in zip(PERIODS, LABELS, T3_DEATHS, T3_CFR):
m = (yr >= a) & (yr <= b)
c, d = int(n[m].sum()), int(dth[m].sum())
P.append((c, d))
print(f"{lab}: {m.sum():2d} outbreaks, cases {c:5d}, deaths {d:5d} (Table 3: {rd}; equal: {d == rd}), "
f"pooled CFR {100 * d / c:.1f}% (Table 3: {rc}%), difference {rc - 100 * d / c:+.1f} points; "
f"unweighted mean of outbreak CFRs {np.mean(100 * dth[m] / n[m]):.1f}%")
pooled_cfr = [100 * d / c for c, d in P]
print("\n=== 3. Overall CFR, Abstract '66% (62 to 71)' ===")
se_p = math.sqrt((D / C) * (1 - D / C) / C)
print(f"pooled: {100 * D / C:.1f}% (binomial 95% CI {100 * (D / C - Z * se_p):.1f}–{100 * (D / C + Z * se_p):.1f})")
print(f"difference reported - pooled: {ABS_CFR[0] - 100 * D / C:.1f} points; "
f"share of all cases in 2009–2019: {100 * P[3][0] / C:.1f}%; "
f"pooled fall 1976–1986 to 2009–2019: {pooled_cfr[0] - pooled_cfr[3]:.1f} points")
print(f"unweighted mean of the 34 outbreak CFRs: {np.mean(100 * dth / n):.1f}%")
print(f"unweighted mean of the four Table 3 CFRs: {np.mean(T3_CFR):.1f}%; their range: {min(T3_CFR)}–{max(T3_CFR)}")
print("\n=== 4. What the reported ORs themselves imply ===")
for lab, (o, lo, hi) in [("Abstract", ABS_OR), ("Results", RES_OR)] + \
[(f"Table 3 {l}", t) for l, t in zip(LABELS, T3_OR)]:
se = (math.log(hi) - math.log(lo)) / (2 * Z)
zz = math.log(o) / se if se > 0 else float("inf")
print(f"{lab:22s} OR {o} ({lo} to {hi}): CI excludes 1: {lo > 1}; implied z = {zz:.1f}")
print("Abstract and Results print different CIs for the same 'OR in 2019 vs 1976':", ABS_OR[1:], RES_OR[1:])
print("\n=== 5. Direction: OR of death, later vs earlier ===")
c1, d1 = P[0]
c4, d4 = P[3]
o_rep = orci(0.62 * c4, c4 - 0.62 * c4, 0.71 * c1, c1 - 0.71 * c1)
print(f"from the reported CFRs 62% vs 71% (Table 1 period sizes): OR {o_rep[0]:.3f} "
f"({o_rep[1]:.3f}–{o_rep[2]:.3f}), p = {o_rep[3]:.1e}; inverse {1 / o_rep[0]:.2f} "
f"({1 / o_rep[2]:.2f}–{1 / o_rep[1]:.2f})")
print(f"odds at reported CFRs: {[round(p / (100 - p), 2) for p in T3_CFR]}; "
f"0.62/0.38 / (0.71/0.29) = {(0.62 / 0.38) / (0.71 / 0.29):.3f}")
o_crude = orci(d4, c4 - d4, d1, c1 - d1)
print(f"from Table 1 counts, 2009–2019 vs 1976–1986: OR {o_crude[0]:.3f} ({o_crude[1]:.3f}–{o_crude[2]:.3f}), "
f"p = {o_crude[3]:.1e}; inverse {1 / o_crude[0]:.2f}")
print("pairwise crude ORs (row period vs column period) from Table 1:")
for i, j in itertools.combinations(range(4), 2):
(ci, di), (cj, dj) = P[i], P[j]
o = orci(dj, cj - dj, di, ci - di)
print(f" {LABELS[j]} vs {LABELS[i]}: {o[0]:.2f} ({o[1]:.2f}–{o[2]:.2f}) p = {o[3]:.1e}")
tab = np.array([[d, c - d] for c, d in P])
chi2, pchi, dfchi, _ = stats.chi2_contingency(tab)
print(f"chi-square test of equal CFR across the 4 periods (patient-level): chi2 = {chi2:.0f}, df = {dfchi}, p = {pchi:.1e}")
print("\n=== 6. Allowing for between-outbreak variation (not in the paper) ===")
rho = stats.spearmanr(yr, dth / n)
print(f"Spearman correlation, outbreak CFR vs year (34 outbreaks): rho = {rho.statistic:.2f}, p = {rho.pvalue:.2f}")
def logit_fit(X):
b = np.zeros(X.shape[1])
for _ in range(100):
p = 1 / (1 + np.exp(-X @ b))
W = n * p * (1 - p)
b = np.linalg.solve(X.T @ (W[:, None] * X), X.T @ (W * (X @ b) + dth - n * p))
p = 1 / (1 + np.exp(-X @ b))
cov = np.linalg.inv(X.T @ ((n * p * (1 - p))[:, None] * X))
phi = np.sum((dth - n * p) ** 2 / (n * p * (1 - p))) / (len(n) - X.shape[1])
return b, cov, phi
Xp = np.column_stack([np.ones_like(yr)] + [((yr >= a) & (yr <= b)).astype(float) for a, b in PERIODS[1:]])
b, cov, phi = logit_fit(Xp)
se_b, se_q = math.sqrt(cov[3, 3]), math.sqrt(cov[3, 3] * phi)
dfq = len(n) - Xp.shape[1]
q_or = (math.exp(b[3]), math.exp(b[3] - stats.t.ppf(.975, dfq) * se_q), math.exp(b[3] + stats.t.ppf(.975, dfq) * se_q),
2 * stats.t.sf(abs(b[3] / se_q), dfq))
print(f"logistic model with period factors: OR 2009–2019 vs 1976–1986 = {math.exp(b[3]):.3f} "
f"(binomial 95% CI {math.exp(b[3] - Z * se_b):.3f}–{math.exp(b[3] + Z * se_b):.3f}); "
f"Pearson dispersion = {phi:.0f}")
print(f" quasi-binomial (dispersion-scaled, t on {dfq} df) 95% CI {q_or[1]:.3f}–{q_or[2]:.3f}, p = {q_or[3]:.3f}")
Xt = np.column_stack([np.ones_like(yr), (yr - 1976) / 10])
bt, covt, phit = logit_fit(Xt)
set_q = math.sqrt(covt[1, 1] * phit)
dft = len(n) - 2
print(f"linear trend: OR per decade {math.exp(bt[1]):.3f}; binomial p = {2 * stats.norm.sf(abs(bt[1] / math.sqrt(covt[1, 1]))):.1e}; "
f"quasi-binomial 95% CI {math.exp(bt[1] - stats.t.ppf(.975, dft) * set_q):.3f}–"
f"{math.exp(bt[1] + stats.t.ppf(.975, dft) * set_q):.3f}, p = {2 * stats.t.sf(abs(bt[1] / set_q), dft):.3f}")
# DerSimonian-Laird random-effects meta-regression of outbreak logit CFR on decade (0.5 continuity correction)
dd, nn = dth + 0.5, n + 1
lg, v = np.log(dd / (nn - dd)), 1 / dd + 1 / (nn - dd)
w = 1 / v
B = np.linalg.solve(Xt.T @ (w[:, None] * Xt), Xt.T @ (w * lg))
Q = np.sum(w * (lg - Xt @ B) ** 2)
Pm = np.diag(w) - (w[:, None] * Xt) @ np.linalg.inv(Xt.T @ (w[:, None] * Xt)) @ (Xt.T * w)
tau2 = max(0.0, (Q - dft) / np.trace(Pm))
w = 1 / (v + tau2)
cvr = np.linalg.inv(Xt.T @ (w[:, None] * Xt))
B = cvr @ (Xt.T @ (w * lg))
sr = math.sqrt(cvr[1, 1])
print(f"random-effects meta-regression: tau^2 = {tau2:.2f}; OR per decade {math.exp(B[1]):.3f} "
f"({math.exp(B[1] - Z * sr):.3f}–{math.exp(B[1] + Z * sr):.3f}), p = {2 * stats.norm.sf(abs(B[1] / sr)):.3f}")
share = {k: 100 * float(n[[r[1] == k and r[0] == 2014 for r in T1]].sum()) / C for k in ("Liberia", "Sierra Leone")}
print(f"share of all cases from the 2014 Liberia and Sierra Leone outbreaks: {share['Liberia'] + share['Sierra Leone']:.1f}%")
print("\n=== 7. Single-year reading of 'OR in 2019 vs 1976' ===")
m76, m19 = yr == 1976, yr == 2019
print(f"1976 outbreaks: {int(dth[m76].sum())}/{int(n[m76].sum())} = {100 * dth[m76].sum() / n[m76].sum():.1f}%; "
f"2019 outbreak (Uganda): {int(dth[m19].sum())}/{int(n[m19].sum())} = 100% (OR undefined: no survivors)")
o = orci(2150, 3220 - 2150, dth[m76].sum(), n[m76].sum() - dth[m76].sum())
print(f"2018–2019 DRC outbreak (2150/3220 = {100 * 2150 / 3220:.1f}%) vs 1976: OR {o[0]:.2f} ({o[1]:.2f}–{o[2]:.2f}), p = {o[3]:.1e}")
print("\n=== 8. Attempts to reproduce the Table 3 ORs ===")
for i, (c, d) in enumerate(P):
oc, od = C - c, D - d
o = orci(d, c - d, od, oc - od)
rr = (d / c) / (od / oc)
se = math.sqrt(1 / d - 1 / c + 1 / od - 1 / oc)
print(f"{LABELS[i]} vs all other periods: OR {o[0]:.2f} ({o[1]:.2f}–{o[2]:.2f}); risk ratio {rr:.2f} "
f"({rr * math.exp(-Z * se):.2f}–{rr * math.exp(Z * se):.2f}); inverse risk ratio {1 / rr:.2f}; "
f"period CFR / overall CFR {(d / c) / (D / C):.2f} [Table 3: {T3_OR[i]}]")
print("\n=== 9. Other quantities (Table 2, Results) ===")
for s, (c, d, pr) in T2_SEX.items():
print(f"{s}: CFR {d}/{c} = {100 * d / c:.1f}% (printed {pr}); share of cases {100 * c / C:.1f}%; share of deaths {100 * d / D:.1f}%")
print(f"male + female cases {sum(v[0] for v in T2_SEX.values())}, deaths {sum(v[1] for v in T2_SEX.values())}")
print(f"age 15–60 share of cases: {100 * T2_AGE['15–60'] / C:.1f}% (Results: 70%); age groups sum {sum(T2_AGE.values())}")
print(f"health facility + household cases: {sum(T2_SOURCE.values())}")
# ------------------------------------------------------------------ figures
BG, INK, MUTED, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK, "ytick.color": INK, "axes.edgecolor": MUTED,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# Figure 1: ORs on one log axis
rows = [
("Abstract: “OR in 2019 vs 1976”", ABS_OR, ORANGE, "o", "1.6 (1.5–1.8)"),
("Results text: “OR in 2019 vs 1976”", RES_OR, ORANGE, "o", "1.6 (1.5–1.6)"),
] + [(f"Table 3, column {l}", t, ORANGE, "o", f"{t[0]} ({t[1]}–{t[2]})") for l, t in zip(LABELS, T3_OR)] + [
("Table 3 CFRs 62% vs 71%", o_rep[:3], BLUE, "D", f"{o_rep[0]:.2f} ({o_rep[1]:.2f}–{o_rep[2]:.2f})"),
("Table 1 counts, 2009–19 vs 1976–86", o_crude[:3], BLUE, "D",
f"{o_crude[0]:.2f} ({o_crude[1]:.2f}–{o_crude[2]:.2f})"),
("Same, outbreak-level CI", q_or[:3], BLUE, "s",
f"{q_or[0]:.2f} ({q_or[1]:.2f}–{q_or[2]:.2f})"),
]
fig, ax = plt.subplots(figsize=(7.2, 5.4))
ypos = np.arange(len(rows))[::-1].astype(float)
ypos[-3:] -= 0.5
for y, (lab, (o, lo, hi), col, mk, txt) in zip(ypos, rows):
hollow = mk == "s"
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt")
ax.plot(o, y, mk, ms=8, color=col, mfc=BG if hollow else col, mew=2)
ax.text(4.3, y, txt, va="center", ha="left", fontsize=10, color=col if col == BLUE else "#b8481c")
ax.axvline(1, color=MUTED, lw=0.8, zorder=0)
ax.text(1.1, ypos[0] + 0.85, "OR = 1, no change", ha="left", va="bottom", fontsize=10, color=MUTED)
ax.set_xscale("log")
ax.set_xlim(0.03, 4.2)
ax.set_xticks([0.05, 0.1, 0.2, 0.5, 1, 2])
ax.set_xticklabels(["0.05", "0.1", "0.2", "0.5", "1", "2"])
ax.set_yticks(ypos)
ax.set_yticklabels([r[0] for r in rows])
ax.set_ylim(ypos[-1] - 0.7, ypos[0] + 1.5)
ax.grid(axis="x", color=GRID, lw=0.6)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Odds ratio of death (95% CI), log scale")
ax.text(0.9, ypos[0] + 0.85, "← lower CFR later", ha="right", va="bottom", fontsize=10, color=MUTED)
fig.legend(handles=[
Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Reported in the paper (all P < 0.001)"),
Line2D([], [], color=BLUE, marker="D", ms=8, lw=2, label="Recomputed from the paper’s numbers, patient-level CI"),
Line2D([], [], color=BLUE, marker="s", ms=8, mfc=BG, mew=2, lw=2,
label="Recomputed, CI allowing for variation between outbreaks (quasi-binomial)"),
], loc="lower left", ncol=1, frameon=False, fontsize=10, bbox_to_anchor=(0.02, 0.0))
fig.subplots_adjust(left=0.40, right=0.80, top=0.97, bottom=0.30)
save(fig, "fig1")
# Figure 2: period CFRs, reported vs pooled from Table 1
fig, ax = plt.subplots(figsize=(7.2, 3.6))
cats = LABELS + ["Overall"]
rep = T3_CFR + [ABS_CFR[0]]
rec = pooled_cfr + [100 * D / C]
x = np.arange(len(cats), dtype=float) * 1.4
x[-1] += 0.35
for xi, r, c in zip(x, rep, rec):
ax.plot([xi - 0.07, xi + 0.07], [r, c], color=GRID, lw=2, zorder=1)
ax.plot(xi - 0.07, r, "o", ms=9, color=ORANGE, zorder=3)
ax.plot(xi + 0.07, c, "D", ms=8, color=BLUE, zorder=3)
ax.text(xi - 0.19, r, f"{r}%", va="center", ha="right", fontsize=10, color="#b8481c")
ax.text(xi + 0.19, c, f"{c:.1f}%", va="center", ha="left", fontsize=10, color=BLUE)
ax.set_xticks(x)
ax.set_xticklabels([f"{l}\n{c:,} cases".replace(",", " ") for l, (c, _) in zip(LABELS, P)] + [f"Overall\n{C:,} cases".replace(",", " ")])
ax.set_ylim(0, 100)
ax.set_ylabel("Case fatality rate (%)")
ax.set_xlim(-0.6, x[-1] + 0.6)
ax.axvline((x[-2] + x[-1]) / 2, color=GRID, lw=0.8)
ax.grid(axis="y", color=GRID, lw=0.6)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
fig.legend(handles=[
Line2D([], [], color=ORANGE, marker="o", ms=9, lw=0, label="Reported (Table 3; Abstract for overall)"),
Line2D([], [], color=BLUE, marker="D", ms=8, lw=0, label="Deaths ÷ cases from Table 1"),
], loc="upper center", ncol=2, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 1.0))
fig.subplots_adjust(left=0.1, right=0.97, top=0.86, bottom=0.2)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Abstract reports p=0.002 for an odds ratio whose own 95% CI (0.86 to 7.36) includes 1
BMJ Glob Health (2022) · doi:10.1136/bmjgh-2022-009400 · PMC9494011
Summary
The abstract reports that identifying as "gay or queer" was associated with discrimination from healthcare providers since COVID-19 ("OR=2.51, 95% CI 0.86 to 7.36, p=0.002"), and Table 3 marks the same estimate, which belongs to the queer/asexual/pansexual/questioning group (aOR 2.52), as ** (p≤0.01). The interval includes 1: reconstructing the p-value from it on the log-odds scale gives p ≈ 0.09, and p=0.002 would need an interval of about 1.40 to 4.50; the bisexual healthcare estimate (aOR 2.74, CI 0.96 to 7.81, also **) likewise gives p ≈ 0.06. These two estimates are the paper's only basis for saying that queer and bisexual respondents had higher odds of discrimination from healthcare providers, so those statements in the abstract, Results and Discussion are not supported by the paper's own intervals. The gay respondents' healthcare estimate (aOR 3.45, CI 1.23 to 9.72) does exclude 1, at p ≈ 0.02 rather than the p≤0.01 its marker indicates, so the broader finding of elevated odds for some subgroups survives.
What the paper reports
Abstract, Results:
Identifying as gay or queer was associated with increased odds of experiencing the same or more discrimination from government representatives (OR=1.89, 95% CI 1.04 to 3.45, p=0.045) and healthcare providers (OR=2.51, 95% CI 0.86 to 7.36, p=0.002) due to COVID-19.
Results, paragraph on the final (healthcare provider) adjusted model:
Sexual orientation was associated with increased odds of experiencing discrimination or violence from healthcare providers since COVID-19, with gay (OR=3.45, 95% CI 1.22 to 9.71), bisexual (OR=2.74, 95% CI 0.96 to 7.81) or queer/asexual/pansexual (OR=2.51, 95% CI 0.86 to 7.36) orientation being significantly associated.
Table 3, multivariable logistic regression. Each outcome has five columns: OR | aOR | significance marker | 95% CI lower | 95% CI upper. The outcomes are, in order, discrimination based on sexual orientation and/or gender identity, from police and/or military, from government representatives, and from health providers. Heterosexual respondents are the reference category.
Gay | 1.83 | 1.82 | * | 1.05 | 3.17 | 0.98 | 1.27 | * | 1.05 | 1.80 | 1.73 | 1.89 | * | 1.04 | 3.45 | 3.07 | 3.45 | ** | 1.23 | 9.72
Bisexual | 1.36 | 1.30 | | 0.73 | 2.28 | 0.77 | 0.72 | | 0.39 | 1.32 | 1.07 | 1.13 | | 0.61 | 2.09 | 2.58 | 2.74 | ** | 0.96 | 7.81
Queer/asexual/pansexual/questioning | 1.93 | 2.00 | * | 1.07 | 3.72 | 0.78 | 0.62 | | 0.32 | 1.19 | 1.53 | 1.30 | * | 1.21 | 2.47 | 2.91 | 2.52 | ** | 0.86 | 7.36
Table 3 footer:
*P≤0.05; **p≤0.01; ***p=0.0.
Results, paragraph on the government-representatives model:
though only a gay (OR=1.89, 95% CI 1.04 to 3.45) and queer/asexual/pansexual (OR=1.30, 95% CI 0.68 to 2.47) orientation were significant when compared with their heterosexual counterparts.
Results, paragraph on the police/military model:
Sexual orientation nor gender identity was not significantly associated with a change in odds of experiencing violence or discrimination from police and/or military.
Discussion, second paragraph:
Specifically, gay and queer/asexual/pansexual/questioning individuals had higher odds of experiencing violence and discrimination from police, government representatives and healthcare providers when compared with their heterosexual peers. Bisexual individuals also had higher odds of experiencing violence and discrimination from healthcare providers.
Methods, Statistical analyses. No other inference method, test direction or significance threshold is described:
multivariable logistic regression analyses were conducted to examine the factors associated with experiences of stigma and discrimination
Table 1. The reference category for all sexual-orientation contrasts is small:
Heterosexual | 74 | 0.95
What the paper's own numbers show
Method. For a logistic-regression odds ratio with a Wald 95% interval, the interval is symmetric on the log scale, so SE(log OR) = [ln(upper) − ln(lower)] / (2 × 1.96). Then z = ln(OR) / SE, and the two-sided p = 2·[1 − Φ(|z|)]. The geometric midpoint √(lower × upper) should reproduce the point estimate, and this checks that the OR and CI belong together. To allow for rounding, I also recomputed p over every combination of the OR and both bounds shifted by ±0.005, which is the "p range" below. The conclusion does not depend on the reconstruction method in any case: a two-sided 95% interval that contains 1 cannot go with p < 0.05.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Healthcare providers, "gay or queer": OR 2.51 (0.86–7.36) | Abstract | p=0.002 | CI midpoint 2.52; SE 0.548; z 1.68; p = 0.093 (rounding range 0.091–0.095) |
| Same estimate, queer/asexual/pansexual/questioning: aOR 2.52 (0.86–7.36) | Table 3 | ** (p≤0.01) |
p = 0.091 (0.090–0.093) |
| Same estimate, 2.51 (0.86–7.36) | Results text | "significantly associated" | p = 0.093; CI includes 1 |
| 95% CI that OR 2.51 with p=0.002 would require | — | — | z 3.09; SE 0.298; 1.40 to 4.50 (the reported interval is 1.84 times wider on the log scale) |
| Same interval, read as a 90% / 99% CI | — | stated as 95% CI | p = 0.158 / 0.027 (neither is 0.002) |
| One-sided p for the same estimate | — | — | 0.046 |
| Healthcare providers, bisexual: aOR 2.74 (0.96–7.81) | Table 3; Results text | ** (p≤0.01); "significantly associated" |
p = 0.059 (0.058–0.061); CI includes 1 |
| Healthcare providers, gay: aOR 3.45 (1.23–9.72) | Table 3 | ** (p≤0.01) |
p = 0.019 (0.018–0.019): significant at 0.05, but not at the p≤0.01 the marker indicates |
| Healthcare providers, gay: OR 3.45 (1.22–9.71) | Results text | "significantly associated" | p = 0.019 (0.019–0.020) |
| Healthcare providers, lesbian: aOR 2.52 (0.50–12.87) | Table 3 | no marker | p = 0.265 (shown for comparison; consistent) |
| Government representatives, gay: aOR 1.89 (1.04–3.45) | Abstract; Table 3 | p=0.045; * |
p = 0.037 (0.036–0.039); broadly consistent |
| Government representatives, queer: aOR 1.30 (1.21–2.47) | Table 3 | * |
CI midpoint 1.73, not 1.30: the row's OR and CI cannot both be right |
| Government representatives, queer: OR 1.30 (0.68–2.47) | Results text | "significant" | CI midpoint 1.30 (coherent); p = 0.425 (0.415–0.435); CI includes 1 |
| Police/military, queer: aOR 0.62 (0.32–1.19) | Table 3 | no marker | p = 0.154; OR below 1 and not significant |
| Healthcare providers, disability: aOR 1.35 (1.07–1.71) | Abstract; Table 3 | p=0.009; ** |
p = 0.012 (0.010–0.015); consistent within rounding |
| Heterosexual reference group | Table 1 | n = 74 | 7758 − 6125 − 22 − 1158 − 379 = 74 (consistent) |
The healthcare-provider interval for the queer/asexual/pansexual/questioning group is internally coherent: its geometric midpoint (2.516) matches the reported point estimate (2.51 in the text, 2.52 in Table 3). What disagrees with the other numbers is the abstract's p-value and the table's ** marker. The same holds for the bisexual row (midpoint 2.738 against 2.74).
Figure
Figure 1. Adjusted odds ratios for discrimination from healthcare providers, by sexual orientation (reference: heterosexual, n = 74), on a log scale. The thin vertical line marks OR = 1. Orange dots and lines show each odds ratio and 95% CI as the paper reports it, and the orange text gives the significance the paper attaches to it (abstract p-value or Table 3 marker). Blue text gives the two-sided p recomputed from the same OR and CI with a log-scale Wald reconstruction. The dashed blue interval under the abstract row is the 95% CI (1.40–4.50) that an OR of 2.51 would need for p = 0.002 to be correct. The reported interval (0.86–7.36) is much wider and crosses 1. For the queer/asexual/pansexual/questioning and bisexual groups, the reported intervals include 1 (recomputed p = 0.09 and 0.06), although the abstract, Results text and Table 3 present them as significant at p = 0.002 or p ≤ 0.01. The gay-group interval excludes 1 (p = 0.019). All numbers come from the paper or from recompute.py.txt.
Why it matters
The abstract is the part of the paper most readers see, and it is where this estimate appears with p=0.002. At the conventional 0.05 level, the paper's own interval does not support a significant association between queer/asexual/pansexual/questioning identity and discrimination from healthcare providers. The same applies to bisexual identity (CI 0.96 to 7.81). The Discussion's statements that queer/asexual/pansexual/questioning and bisexual individuals "had higher odds" of discrimination from healthcare providers rest on these two estimates.
The abstract's wording, "gay or queer", also joins two estimates. The government figure (1.89, 1.04 to 3.45) is the gay estimate. The healthcare figure (2.51, 0.86 to 7.36) is the queer/asexual/pansexual/questioning estimate. The gay healthcare estimate is a different number (aOR 3.45, 1.23 to 9.72). A reader of the abstract cannot tell that the p=0.002 healthcare figure belongs to the 379-person queer/asexual/pansexual/questioning group rather than to gay respondents.
The same Discussion sentence also extends to police and government representatives, but the paper's other numbers do not support it for the queer group either. Its police/military aOR is 0.62 (0.32 to 1.19), and the Results state that sexual orientation was not significantly associated with police/military discrimination. Its government-representatives estimate appears as 1.30 with CI 1.21 to 2.47 in Table 3 (an interval whose midpoint is 1.73, not 1.30) and as 1.30 with CI 0.68 to 2.47 in the Results text. The text version is internally coherent and gives p ≈ 0.43.
What survives: the gay healthcare-provider estimate excludes 1 (p ≈ 0.02). So do the disability, immigrant and ethnic-minority estimates for healthcare providers in Table 3. The paper's broad finding that some marginalised subgroups reported higher odds of discrimination from healthcare providers therefore does not depend on the erroneous values. What fails is the claim of significance for the queer (and bisexual) healthcare-provider contrasts, which the abstract reports with a p-value that is over 40 times smaller than its own interval implies.
Explanations considered
- A correction already published.
known.py(Crossref, PMC and Europe PMC notices) returned nothing. Crossref shows noupdate-to,updated-byor relations. Europe PMC lists only the original article. Does not apply. - Text-conversion or extraction error. The same values appear in the abstract, the Results text and Table 3. They also appear identically in two separate renderings of the publisher's JATS full text (the PMC copy and the Europe PMC XML), where the Table 3 cells are separate elements (2.91 | 2.52 | ** | 0.86 | 7.36). Does not account for the numbers.
- The interval is not a 95% CI. Table 3 and the text both label it a 95% CI. Even read as a 99% CI, it implies p = 0.027. Read as a 90% CI, it implies p = 0.158. Neither gives 0.002. Does not account for it.
- One-sided test. The Methods never mention one-sided tests. A one-sided p from this interval is 0.046, still more than 20 times 0.002. Does not account for it.
- Unadjusted vs adjusted estimate. The abstract's 2.51 (0.86 to 7.36) matches the adjusted Table 3 row (aOR 2.52, the same interval) and not the crude OR (2.91). The p-value could belong to the crude model, but the paper reports no crude CI or p-value, and the abstract attaches p=0.002 to this OR and CI. This cannot be verified from the paper. Even if it were so, the Table 3
**on the adjusted row would still conflict with its own interval. - A global or joint test for sexual orientation. p=0.002 might be a likelihood-ratio test for the whole sexual-orientation variable. The paper does not report such a test, and the abstract presents the p-value as belonging to the single OR and CI. This would also not explain the
**on the queer and bisexual rows of Table 3. It does not reconcile the reported figures, though it cannot be excluded as the source of the number. - Wald vs profile-likelihood or exact intervals. With a reference group of only 74 heterosexual respondents, the log-OR may be somewhat skewed, and a likelihood-based interval could differ from a Wald one. But any valid two-sided 95% interval that contains 1 corresponds to p > 0.05 under the matching test. Moving p from about 0.09 to 0.002 would need an interval roughly half as wide on the log scale (1.40 to 4.50). Does not account for it.
- Multiplicity adjustment. Adjusting for multiple comparisons would make p-values larger, not smaller. Does not apply.
- The abstract's estimate is the gay estimate. The gay healthcare-provider estimate is OR 3.07 / aOR 3.45 (1.23 to 9.72; 1.22 to 9.71 in the text), with p ≈ 0.019. That is not p=0.002 either. In any case, 2.51 (0.86 to 7.36) is unambiguously the queer/asexual/pansexual/questioning row. Does not account for it.
- Rounding. Over every combination of the OR and the bounds shifted by ±0.005, p stays between 0.091 and 0.095. Does not account for it.
Status
As of 2026-10-23:
- Version of record. The publisher site (gh.bmj.com, reached through the doi.org redirect) returned HTTP 403 to automated access, and the PMC web page served only a browser check. I therefore checked the publisher-supplied JATS full text archived at Europe PMC (fullTextXML for PMC9494011). It matches the local PMC copy in the abstract sentence, the Results sentence and the Table 3 cells quoted above, so the error is in the version of record as archived.
- Supplementary material. None. The JATS has no supplementary-material elements, and Europe PMC reports hasSuppl = N. The data availability statement says data are available on reasonable request, at the discretion of the partner social networking company. No public data or output can settle whether the p-value, the marker or the interval is the wrong value.
- Corrections and notices. None found. I checked Crossref (
update-to,updated-by,relation: none), Crossref's Retraction Watch-derived update data (none), PMC and Europe PMC notices (none; Europe PMC search on the DOI returns only PMID 36130772), and a web search for a correction or erratum (no notice for this article). - PubPeer. The PubPeer API required human verification and could not be queried directly. Web searches found no PubPeer thread for this DOI. PubPeer is therefore not confirmed.
Suggested correction
- Abstract, Results: report the healthcare-provider estimate with a p-value consistent with its interval (about p=0.09 if the interval 0.86 to 7.36 is correct), and do not describe the association as significant. Alternatively, if the interval is the wrong value, give the correct one. Attribute each estimate to the right group: the government estimate (1.89, 1.04 to 3.45) is for gay respondents, and the healthcare estimate (2.51, 0.86 to 7.36) is for queer/asexual/pansexual/questioning respondents. If the abstract is meant to describe gay respondents' healthcare-provider estimate, that is aOR 3.45 (95% CI 1.23 to 9.72), p ≈ 0.02.
- Table 3, health-provider columns: remove the
**markers from the queer/asexual/pansexual/questioning row (CI 0.86 to 7.36) and the bisexual row (CI 0.96 to 7.81), and change the gay row's**to*(CI 1.23 to 9.72, p ≈ 0.02), unless the authors' output shows different intervals. - Table 3, government-representatives columns, queer/asexual/pansexual/questioning row: reconcile the lower bound (1.21 in the table vs 0.68 in the text) and the
*marker with the OR of 1.30. - Results (healthcare-provider and government-representatives paragraphs) and Discussion: restate which sexual-orientation groups had intervals excluding 1, specifically that the queer/asexual/pansexual/questioning and bisexual healthcare-provider estimates, and the queer police and government estimates, were not statistically significant at the 0.05 level.
- Methods: state how p-values and confidence intervals were computed (Wald or likelihood-based, two-sided) and the significance threshold used.
Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
"""Recompute the p-values implied by the odds ratios and 95% CIs reported in
PMC9494011 (BMJ Glob Health 2022, doi:10.1136/bmjgh-2022-009400), and redraw
the figure for the write-up. Run: python recompute.py.txt
Every input number below is copied from the article (abstract, Results text,
Table 1 or Table 3). Outputs fig1.svg / fig1.png next to this file."""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
Z95 = norm.ppf(0.975)
def wald(or_, lo, hi, level=0.95):
"""Log-scale Wald reconstruction: SE from the CI width, p from OR/SE."""
z = norm.ppf(1 - (1 - level) / 2)
se = (np.log(hi) - np.log(lo)) / (2 * z)
gm = np.sqrt(lo * hi) # point estimate the CI implies
p = 2 * norm.sf(abs(np.log(or_)) / se) # two-sided, from stated OR
p_gm = 2 * norm.sf(abs(np.log(gm)) / se) # two-sided, from CI midpoint
p_one = norm.sf(abs(np.log(or_)) / se) # one-sided
return gm, se, p, p_gm, p_one
def p_range(or_, lo, hi, d_or=0.005, d_ci=0.005):
"""Smallest and largest two-sided p compatible with 2-decimal rounding."""
ps = []
for o in (or_ - d_or, or_ + d_or):
for l in (lo - d_ci, lo + d_ci):
for h in (hi - d_ci, hi + d_ci):
ps.append(wald(o, l, h)[2])
return min(ps), max(ps)
# label, OR, lower, upper, what the paper reports about significance
ROWS = [
("Abstract: healthcare, 'gay or queer'", 2.51, 0.86, 7.36, "p=0.002"),
("Results text: healthcare, queer/asex./pans.", 2.51, 0.86, 7.36, "'significantly associated'"),
("Table 3: healthcare, queer/asex./pans./quest.", 2.52, 0.86, 7.36, "** (p<=0.01)"),
("Table 3: healthcare, bisexual", 2.74, 0.96, 7.81, "** (p<=0.01)"),
("Results text: healthcare, bisexual", 2.74, 0.96, 7.81, "'significantly associated'"),
("Table 3: healthcare, gay", 3.45, 1.23, 9.72, "** (p<=0.01)"),
("Results text: healthcare, gay", 3.45, 1.22, 9.71, "'significantly associated'"),
("Table 3: healthcare, lesbian", 2.52, 0.50, 12.87, "(no marker)"),
("Abstract/Table 3: government, gay", 1.89, 1.04, 3.45, "p=0.045 / *"),
("Table 3: government, queer/asex./pans./quest.", 1.30, 1.21, 2.47, "*"),
("Results text: government, queer/asex./pans.", 1.30, 0.68, 2.47, "'significant'"),
("Table 3: police, queer/asex./pans./quest.", 0.62, 0.32, 1.19, "(no marker)"),
("Abstract/Table 3: healthcare, disability", 1.35, 1.07, 1.71, "p=0.009 / **"),
]
print("Log-scale Wald reconstruction from each reported OR and 95% CI")
print(f"{'quantity':47s} {'OR':>5} {'95% CI':>12} {'CI mid':>6} {'SE':>6} "
f"{'p(2-s)':>7} {'p range (rounding)':>18} {'p(1-s)':>7} reported")
for lab, o, l, h, rep in ROWS:
gm, se, p, p_gm, p1 = wald(o, l, h)
lo_p, hi_p = p_range(o, l, h)
print(f"{lab:47s} {o:5.2f} {l:5.2f}-{h:<6.2f} {gm:6.2f} {se:6.3f} "
f"{p:7.3f} {lo_p:8.3f}-{hi_p:<9.3f} {p1:7.3f} {rep}")
# What would p=0.002 require for OR=2.51?
z_req = norm.isf(0.002 / 2)
se_req = np.log(2.51) / z_req
lo_req, hi_req = 2.51 * np.exp(-Z95 * se_req), 2.51 * np.exp(Z95 * se_req)
print(f"\nFor OR=2.51 with two-sided p=0.002: z={z_req:.2f}, SE={se_req:.3f}, "
f"required 95% CI {lo_req:.2f} to {hi_req:.2f}")
_, se_rep, _, _, _ = wald(2.51, 0.86, 7.36)
p_rep = wald(2.51, 0.86, 7.36)[2]
print(f"Reported CI implies SE={se_rep:.3f}, z={np.log(2.51)/se_rep:.2f}; "
f"ratio of SEs = {se_rep/se_req:.2f}")
print(f"Recomputed p / reported p = {p_rep/0.002:.0f}; "
f"one-sided p / 0.002 = {wald(2.51, 0.86, 7.36)[4]/0.002:.0f}")
for lab, l, h in (("queer healthcare", 0.86, 7.36), ("bisexual healthcare", 0.96, 7.81),
("queer government (Table 3)", 1.21, 2.47)):
print(f"CI geometric midpoint, {lab}: {np.sqrt(l*h):.3f}")
# Alternative readings of the interval level
print("\nIf the interval 0.86-7.36 were read at another confidence level:")
for lvl in (0.90, 0.95, 0.99):
print(f" {int(lvl*100)}% CI -> two-sided p = {wald(2.51, 0.86, 7.36, lvl)[2]:.3f}")
# Reference group size (Table 1) and the identity check
n, gay, les, bi, q, het = 7758, 6125, 22, 1158, 379, 74
print(f"\nTable 1: heterosexual reference n = {het}; "
f"7758 - (gay+lesbian+bisexual+queer) = {n - gay - les - bi - q}")
# ---------------------------------------------------------------- figure
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans",
"text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK})
# Healthcare-provider model, sexual-orientation rows (Table 3), plus the
# abstract's sentence; reported marker vs recomputed p.
fig_rows = [
("Abstract (healthcare,\n'gay or queer')", 2.51, 0.86, 7.36, "p = 0.002"),
("Table 3: queer/asexual/\npansexual/questioning", 2.52, 0.86, 7.36, "** (p ≤ 0.01)"),
("Table 3: bisexual", 2.74, 0.96, 7.81, "** (p ≤ 0.01)"),
("Table 3: gay", 3.45, 1.23, 9.72, "** (p ≤ 0.01)"),
]
fig, ax = plt.subplots(figsize=(7.2, 5.3), facecolor=BG)
fig.subplots_adjust(left=0.27, right=0.64, top=0.89, bottom=0.13)
ax.set_facecolor(BG)
ax.set_xscale("log")
ax.set_xlim(0.4, 16)
ticks = [0.5, 1, 2, 4, 8, 16]
ax.set_xticks(ticks)
ax.set_xticklabels([str(t) for t in ticks])
ax.minorticks_off()
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.axvline(1, color=INK2, lw=1)
ax.text(1.06, -0.75, "OR = 1 (no association)", color=INK2, fontsize=10,
ha="left", va="center")
ys = np.array([4.9, 3.1, 1.6, 0.1]) # extra room under the abstract row
tr = ax.get_yaxis_transform() # x in axes fraction, y in data units
for y, (lab, o, l, h, rep) in zip(ys, fig_rows):
p = wald(o, l, h)[2]
ax.plot([l, h], [y, y], color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot(o, y, "o", ms=9, color=ORANGE, mec=BG, mew=1)
ax.text(1.03, y + 0.30, f"{o:.2f} ({l:.2f}–{h:.2f})", transform=tr,
color=INK2, fontsize=10, va="center", ha="left")
ax.text(1.03, y, f"reported: {rep}", transform=tr, color=ORANGE,
fontsize=10, va="center", ha="left")
ax.text(1.03, y - 0.30, f"recomputed: p = {p:.3f}", transform=tr,
color=BLUE, fontsize=10, va="center", ha="left", fontweight="bold")
# interval that p = 0.002 would require (abstract row)
y0 = ys[0] - 0.75
ax.plot([lo_req, hi_req], [y0, y0], color=BLUE, lw=2, ls=(0, (4, 2)))
ax.plot(2.51, y0, "D", ms=8, color=BLUE, mec=BG, mew=1)
ax.text(1.03, y0, f"CI that p = 0.002 needs:\n{lo_req:.2f}–{hi_req:.2f}",
transform=tr, color=BLUE, fontsize=10, va="center", ha="left")
ax.set_yticks(list(ys) + [y0])
ax.set_yticklabels([r[0] for r in fig_rows] + ["abstract OR, if\np = 0.002 were right"])
for t in ax.get_yticklabels()[-1:]:
t.set_color(BLUE)
ax.set_ylim(-1.05, ys[0] + 0.6)
ax.set_xlabel("Adjusted OR, discrimination from healthcare\nproviders (log scale), with 95% CI")
for s_ in ("top", "right", "left"):
ax.spines[s_].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
handles = [Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8,
label="Reported OR and 95% CI (paper)"),
Line2D([], [], color=BLUE, lw=2, ls=(0, (4, 2)), marker="D", ms=7,
label="Recomputed from the paper's numbers")]
fig.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.5, 0.99),
ncol=2, frameon=False, fontsize=10)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
print("\nwrote fig1.svg and fig1.png")
Abstract reports whole-cohort estimates as sexual-minority results, making two non-significant subgroup findings appear significant
BMJ Glob Health (2022) · doi:10.1136/bmjgh-2022-010231 · PMC9791420
Summary
The paper's second aim is to compare the mental health of sexual minority adolescents with and without HIV, and the abstract reports that "Sexual minority adolescents living with HIV had significantly higher odds of high general anxiety (AOR: 2.42; 95% CI: 1.21 to 4.84; p=0.013)", along with three significant estimates for sexual minority adolescents with unknown HIV status. All four of these estimates, with both CI limits and the p-value, match the HIV-status rows of Table 1 exactly. Table 1 is the whole-cohort model (N=1247, heterosexual and sexual minority adolescents together), and the main text itself labels 2.42 as the result "In the overall cohort". The sexual-minority-only model in Table 2 (N=497) gives different values, and two of the four claims are not significant there: HIV-positive status and high anxiety, AOR 2.09 (0.93 to 4.69), p=0.073; unknown status and suicidal attempt/ideation, AOR 1.47 (0.98 to 2.22), p=0.064. The paper's headline comparison of sexual minority and heterosexual adolescents is unaffected, but the abstract's Results sentence on the second aim misstates which findings reached significance and contradicts the abstract's own Conclusion.
What the paper reports
Abstract, Background (the second aim):
and to determine associations between the mental health status of sexual minority adolescents living with and without HIV.
Abstract, Methods:
A second multivariate regression model was developed to establish associations between HIV status among sexual minority individuals and the dependent variables.
Abstract, Results:
Sexual minority adolescents living with HIV had significantly higher odds of high general anxiety (AOR: 2.42; 95% CI: 1.21 to 4.84; p=0.013), while those with unknown HIV status had significantly higher odds of depressive symptoms (AOR: 3.82, 95% CI: 2.78 to 5.27; p<0.001), high general anxiety (AOR: 3.09; 95% CI: 2.29 to 4.17; p<0.001) and suicidal attempt/ideation (AOR: 1.65; 95% CI: 1.22 to 2.24; p=0.001).
Abstract, Conclusion:
Although there was no significant difference in the mental health status of sexual minority adolescents living and not living with HIV, sexual minority adolescents with unknown HIV status reported worse mental health than their HIV-negative peers.
Methods, Data analysis:
The second set of multivariate regression analyses included data from only sexual minority individuals.
Significance was set at 5%.
Results, third paragraph (describing Table 1):
In the overall cohort, self-reported HIV-positive status among adolescents was associated with increased odds of high general anxiety (AOR: 2.42; 95% CI:1.21 to 4.84; p=0.013), but not depression or suicidal attempt/ideation.
Results, fourth paragraph (describing Table 2):
Having an unknown HIV status was associated with significantly higher odds of experiencing depressive symptoms (AOR: 3.11, 95% CI: 1.76 to 5.49, p<0.001) and high general anxiety (AOR: 2.32, 95% CI: 1.55 to 3.48, p<0.001).
Table 1, "Multivariate logistic regression analysis to determine association between sexual orientation, HIV status, mental health status of adolescents 13–19 years old in Nigeria (N=1247)", HIV-positive row (columns: total | depressive symptoms yes | no | AOR (95% CI) p | high general anxiety yes | no | AOR (95% CI) p):
Positive | 41(3.3) | 29(70.7) | 12(29.3) | 1.81(0.85 to 3.88)0.125 | 17(41.5) | 24(58.5) | 2.42(1.21 to 4.84)0.013
Table 2, "Multivariate logistic regression analysis to determine association between HIV status and mental health status among sexual minority adolescents 13–19 years old in Nigeria (N=497)", HIV-positive row (columns as above, then suicidal ideation yes | no | AOR (95% CI) p):
HIV positive | 29 (5.8) | 24(82.8) | 5(17.2) | 2.25(0.78 to 6.48)0.132 | 14(48.3) | 15(51.7) | 2.09(0.93 to 4.69)0.073 | 12(41.4) | 17(58.6) | 1.47(0.66 to 3.28)0.352
Table 2, unknown-status row:
Unknown HIV status | 222 (44.7) | 201(90.5) | 21(9.5) | 3.11(1.76 to 5.49)<0.001 | 130(58.6) | 92(41.4) | 2.32(1.55 to 3.48)<0.001 | 109(49.1) | 113(50.9) | 1.47(0.98 to 2.22)0.064
Results, fifth paragraph (exploratory analyses, describing supplementary Table S1, heterosexual adolescents only):
Exploratory analyses among heterosexual participants showed that, like sexual minority individuals, unknown HIV status was associated with an increased likelihood of reporting depressive and anxiety symptoms and self-harm/suicidal ideation in this study. The adjusted ORs were comparable in these latter multivariate models among heterosexual and sexual minority participants, though these were relatively larger in sexual minority individuals (see online supplemental table 1).
Supplementary Table S1 (N=750), unknown-status estimates for depressive symptoms, high general anxiety and suicidal ideation:
3.92 (2.66-5.78) <0.001 | 4.16 (2.63-6.56) <0.001 | 2.06 (1.28-3.30) 0.003
Discussion, first paragraph:
Among sexual minority adolescents, those with unknown HIV status had significantly higher odds of reporting depressive symptoms and generalised anxiety disorder. These findings partially support the hypotheses.
What the paper's own numbers show
Method. I matched each of the four estimates in the abstract's Results sentence on HIV status against the HIV-status rows of Table 1 (whole cohort) and Table 2 (sexual minorities only), comparing the AOR, both CI limits and the p-value. For every estimate I recomputed the Wald z and two-sided p implied by the printed 95% CI (SE = [ln(upper) − ln(lower)] / 3.92; z = ln(AOR)/SE), to confirm that each table is internally consistent and that the printed p-values are two-sided. As a model-free check on the key subgroup estimate, I computed crude odds ratios from the Table 2 counts. Finally, I checked that Table 2 and supplementary Table S1 (heterosexual adolescents only, N=750) partition the Table 1 cohort, which confirms which table covers which population. I also compared the unknown-status estimates in Table S1 with those in Table 2, as the Results' exploratory-analysis paragraph does. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| HIV-positive vs negative, high general anxiety, given for sexual minority adolescents | Abstract, Results | 2.42 (1.21 to 4.84), p=0.013 | Identical to Table 1 (whole cohort); Wald p from CI 0.0125 |
| Same estimate, whole cohort | Table 1; Results ¶3 ("In the overall cohort") | 2.42 (1.21 to 4.84), p=0.013 | SE 0.354, z 2.50, p 0.0125; CI excludes 1 |
| Same estimate, sexual minorities only | Table 2 | 2.09 (0.93 to 4.69), p=0.073 | SE 0.413, z 1.79, p 0.074; CI includes 1, not significant |
| Crude OR, same comparison, sexual minorities | Table 2 counts, 14/29 vs 79/246 | — | 1.97 (0.91 to 4.29), p=0.086 |
| Unknown vs negative, depressive symptoms, given for sexual minority adolescents | Abstract, Results | 3.82 (2.78 to 5.27), p<0.001 | Identical to Table 1; Table 2 value is 3.11 (1.76 to 5.49), p<0.001 |
| Unknown vs negative, high general anxiety, given for sexual minority adolescents | Abstract, Results | 3.09 (2.29 to 4.17), p<0.001 | Identical to Table 1; Table 2 value is 2.32 (1.55 to 3.48), p<0.001 |
| Unknown vs negative, suicidal attempt/ideation, given for sexual minority adolescents | Abstract, Results | 1.65 (1.22 to 2.24), p=0.001 | Identical to Table 1 (Wald p 0.0012); Table 2 value is 1.47 (0.98 to 2.22), p=0.064 (Wald p 0.065); not significant |
| Abstract estimates that match Table 1 / Table 2 exactly | Abstract vs Tables 1 and 2 | — | 4 of 4 / 0 of 4 |
| Abstract estimates with CI excluding 1 | Abstract vs Table 2 | 4 of 4 | 2 of 4 for the same estimates in Table 2 |
| Ratio of abstract AOR to Table 2 AOR | Abstract vs Table 2 | — | 1.16, 1.23, 1.33, 1.12 (the abstract's values are larger than the subgroup estimates for all four) |
| HIV-positive vs negative, depressive symptoms / suicidal attempt/ideation, sexual minorities | Table 2 | 2.25 (0.78 to 6.48), p=0.132 / 1.47 (0.66 to 3.28), p=0.352 | Wald p 0.133 / 0.346; neither significant (abstract does not report these) |
| Unknown vs negative, depressive symptoms / high general anxiety / suicidal attempt/ideation, heterosexuals vs sexual minorities | Table S1 vs Table 2; Results ¶5 ("like sexual minority individuals ... self-harm/suicidal ideation"; AORs "relatively larger in sexual minority individuals") | S1: 3.92 / 4.16 / 2.06 (p=0.003); Table 2: 3.11 / 2.32 / 1.47 (p=0.064) | The Table S1 AOR is larger than the Table 2 AOR for 3 of 3 outcomes; the Table 2 suicidal-ideation CI includes 1 |
| Sample sizes of the HIV-status rows | Tables 1, 2 and S1 | N=1247, 497, 750 | Rows sum to 1247, 497 and 750; Table 2 + Table S1 = Table 1 for each status (246+549=795; 29+12=41; 222+189=411) |
The Wald p-values recomputed from the printed CIs agree with the printed p-values for every estimate in both tables (largest difference 0.009, for a non-significant Table 1 estimate, 0.703 vs 0.712), so both tables are internally consistent, the printed p-values are two-sided, and the discrepancy lies only in the abstract's attribution of Table 1 values to the sexual-minority subgroup.
A second, smaller inconsistency in Table 1. In the depressive-symptom and high-anxiety columns of Table 1, the yes/no counts in the "Negative" and "Unknown" HIV rows do not add up to those rows' totals: "Negative" (n=795) shows 332/79 and 195/216, which sum to 411, and "Unknown" (n=411) shows 330/465 and 126/669, which sum to 795. Adding the Table 2 and Table S1 counts gives 330/465 and 126/669 for HIV-negative adolescents and 332/79 and 195/216 for those with unknown status, so these count cells (and their percentages, e.g. 332/411 = 80.8%) appear to have been placed in each other's rows. The suicidal-ideation column and the HIV-positive row add up correctly, and the column totals (691, 338, 309) match the Results text. This does not change any AOR, but it reverses the descriptive picture: as printed, Table 1 shows 80.8% of HIV-negative and 41.5% of unknown-status adolescents with depressive symptoms, when the subgroup tables imply the reverse.
Figure
Figure 1. Left: the abstract's Results sentence on sexual minority adolescents by HIV status, with two of its four estimates (orange). Top right: the same two rows of Table 1, the whole-cohort model (N=1247), which carry identical numbers; the other two abstract estimates (3.82 and 3.09) also match Table 1 exactly. Bottom right: the corresponding rows of Table 2, the model restricted to sexual minority adolescents (N=497), which is the population the abstract names (blue). Both Table 2 confidence intervals include 1, so neither association is significant at the paper's stated 5% level. All numbers are printed in the paper; recompute.py.txt checks each p-value against its CI.
Figure 2. Each of the four estimates the abstract reports for sexual minority adolescents (orange circles, identical to Table 1, whole cohort) next to the same estimate from the sexual-minority-only model in Table 2 (blue squares), on a log odds-ratio axis with the null value (OR = 1) as a thin vertical reference line. All four Table 2 estimates are smaller, and the intervals for HIV-positive status with high general anxiety and for unknown status with suicidal attempt/ideation cross 1.
Why it matters
The comparison of sexual minority adolescents living with and without HIV is one of the paper's two stated aims and has its own model (Table 2). The abstract's Results sentence on that aim reports four significant associations, but all four come from the whole-cohort model, where heterosexual adolescents make up 60% of the sample. In the subgroup the abstract names, HIV-positive status is not significantly associated with any of the three outcomes, and unknown status is not significantly associated with suicidal attempt/ideation; the two effects that do remain significant are 19% to 25% smaller than the abstract states (3.11 vs 3.82 and 2.32 vs 3.09).
The abstract is the part of the paper most often read, indexed and quoted. A reader who relies on it would conclude that HIV-positive sexual minority adolescents in this sample had significantly higher odds of high anxiety, and that unknown HIV status was associated with suicidal attempt/ideation in sexual minority adolescents. Neither is what the paper's subgroup analysis found. The abstract also contradicts itself: its Conclusion ("no significant difference in the mental health status of sexual minority adolescents living and not living with HIV") matches Table 2, while its Results sentence does not. Results paragraphs 3 and 4, the Discussion's summary of findings (its first paragraph) and the tables report the subgroup findings correctly; the Discussion's later paragraph on HIV status agrees with Table 2 that HIV-positive status had no significant effect, although it describes the comparison as one with "heterosexual peers" rather than with HIV-negative sexual minority peers, which is the Table 2 comparison. One sentence in the exploratory-analysis paragraph (Results, fifth paragraph) repeats the suicidal-ideation claim ("like sexual minority individuals, unknown HIV status was associated with ... self-harm/suicidal ideation"), and the same paragraph says the adjusted ORs were "relatively larger in sexual minority individuals", although the heterosexual unknown-status AORs in Table S1 (3.92, 4.16, 2.06) are larger than the sexual-minority ones in Table 2 (3.11, 2.32, 1.47) for all three outcomes. The headline comparison of sexual minority with heterosexual adolescents (AORs 5.54, 3.56 and 2.95) is not affected.
Explanations considered
- The abstract quotes a different sexual-minority model. This does not hold. All four abstract estimates match Table 1 to every printed digit (AOR, both CI limits and p), and none match Table 2. Table 1 reports sexual-orientation and HIV-status estimates side by side for N=1247; its HIV-status rows sum to 1247. The Results text describes 2.42 as the estimate "In the overall cohort". Table 2 is the only model the paper reports for sexual minorities alone.
- Tables 1 and 2 are mislabelled, and Table 1 is really the subgroup model. This does not hold. Table 1's HIV rows sum to 1247 and Table 2's to 497, and for each HIV status the Table 2 count plus the Table S1 (heterosexual) count equals the Table 1 count. Table 1 also contains the sexual-orientation rows that only a whole-cohort model can estimate.
- The abstract sentence was meant to describe the whole cohort. If so, the sentence names the wrong population: it begins "Sexual minority adolescents living with HIV" and refers to "those with unknown HIV status" in the same breath. The abstract's Methods do list HIV status among the variables of the first, whole-cohort model, but they then introduce the second model specifically "to establish associations between HIV status among sexual minority individuals and the dependent variables", and the Results sentence follows the sexual-orientation results as if reporting that model. Either way, the abstract presents whole-cohort estimates as sexual-minority results.
- A one-sided test. This does not account for the numbers. Halving the Table 2 p-values (0.073 and 0.064) would give values below 0.05, but the paper prints two-sided p-values throughout (the recomputed two-sided Wald p-values match the printed ones), its stated criterion is "Significance was set at 5%", and its Conclusion and Discussion treat these Table 2 results as non-significant. In any case, the abstract's numbers are not Table 2's numbers at all.
- Rounding or a conversion artefact. This does not hold. The differences (2.42 vs 2.09; 1.65 vs 1.47; lower CI limits 1.21 vs 0.93 and 1.22 vs 0.98) are far beyond rounding. The PMC text and the publisher-deposited JATS XML in Europe PMC carry identical values, with each AOR, CI and p in its own table cell.
- The supplement contains the abstract's figures. This does not hold. The only supplementary file (Table S1) is a model for heterosexual adolescents alone (N=750), in which HIV-positive status and high anxiety give AOR 3.12 (0.80 to 12.28), p=0.103. It contains no sexual-minority or whole-cohort model.
Status
- Version of record. The publisher's article page and PDF (gh.bmj.com/content/7/12/e010231) returned HTTP 403 to automated requests on 2026-10-23. The text was compared in the PMC article (PMC9791420) and in the publisher-deposited JATS XML via Europe PMC; both carry the same numbers at every quoted place.
- Supplement. The single supplementary file, bmjgh-2022-010231supp001.pdf (Table S1, heterosexual adolescents only, N=750), was obtained through the Europe PMC supplementary-files service and read in full. It does not resolve the discrepancy; its counts were used above to check Table 1.
- Corrections and comments. As of 2026-10-23, Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction, erratum, expression of concern or retraction (PMID 36564086). The publisher page could not be checked for notices or rapid responses (HTTP 403).
- PubPeer. The PubPeer API and search page required human verification and could not be queried directly on 2026-10-23. Web searches on 2026-10-23 for the title and DOI together with PubPeer and correction found no thread or notice.
- Sources. PMC article PMC9791420; Europe PMC full-text XML and supplementary files for PMC9791420; Crossref record for 10.1136/bmjgh-2022-010231; PubMed 36564086.
Suggested correction
Replace the abstract's Results sentence on HIV status with the estimates from the sexual-minority model (Table 2), for example:
Among sexual minority adolescents, HIV-positive status was not significantly associated with depressive symptoms, high general anxiety (AOR: 2.09; 95% CI: 0.93 to 4.69; p=0.073) or suicidal attempt/ideation, while those with unknown HIV status had significantly higher odds of depressive symptoms (AOR: 3.11; 95% CI: 1.76 to 5.49; p<0.001) and high general anxiety (AOR: 2.32; 95% CI: 1.55 to 3.48; p<0.001), but not of suicidal attempt/ideation (AOR: 1.47; 95% CI: 0.98 to 2.22; p=0.064).
If the authors prefer to keep the whole-cohort estimates in the abstract, the sentence should say that they come from the whole cohort (sexual minority and heterosexual adolescents together, the model reported in Table 1), and the Table 2 results should be added for the sexual-minority comparison that the second aim and the Conclusion refer to.
In the exploratory-analysis paragraph of the Results, the phrase "like sexual minority individuals" should not extend to self-harm/suicidal ideation, which was not significantly associated with unknown status among sexual minority adolescents (Table 2: AOR 1.47; 95% CI: 0.98 to 2.22; p=0.064), and the statement that the adjusted ORs were "relatively larger in sexual minority individuals" should be checked against Table S1, whose unknown-status AORs are the larger ones.
In Table 1, the depressive-symptom and high-general-anxiety counts and percentages in the "Negative" and "Unknown" HIV-status rows should be checked against the data; the subgroup tables imply that they belong in each other's rows. Two CIs in the education rows also cannot contain their point estimates and look like typographical errors: Table 1, primary education, suicidal ideation, "0.28(1.13 to 0.64)", and Table 2, secondary education, depressive symptoms, "0.58(1.19 to 1.76)".
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC9791420 write-up and regenerate the figures.
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Figures are written next to this file:
fig1.svg/png, fig2.svg/png figures used in writeup.md (provisional: fig1 = cand1, fig2 = cand2
until a design is chosen from the candidates)
candidates/cand1..cand5_*.svg/png five candidate figure designs, kept out of the deliverable files
All inputs are typed in from the article as archived in PubMed Central (PMC9791420):
Abstract, Results text, Table 1 (whole cohort, N=1247), Table 2 (sexual-minority
adolescents only, N=497), and the online supplement bmjgh-2022-010231supp001.pdf,
Table S1 (heterosexual adolescents only, N=750).
"""
import os
import shutil
import textwrap
import numpy as np
from scipy.stats import norm
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
Z975 = norm.ppf(0.975)
# ---------------------------------------------------------------------------
# 1. Inputs, as printed in the paper
# ---------------------------------------------------------------------------
# (AOR, lower, upper, printed p)
ABSTRACT_SM = { # Abstract, Results: attributed to "Sexual minority adolescents"
"HIV-positive: high anxiety": (2.42, 1.21, 4.84, "0.013"),
"Unknown status: depression": (3.82, 2.78, 5.27, "<0.001"),
"Unknown status: high anxiety": (3.09, 2.29, 4.17, "<0.001"),
"Unknown status: suicidal att./id.": (1.65, 1.22, 2.24, "0.001"),
}
TABLE1 = { # Table 1, whole cohort (N=1247), HIV status rows vs HIV-negative
"HIV-positive: depression": (1.81, 0.85, 3.88, "0.125"),
"HIV-positive: high anxiety": (2.42, 1.21, 4.84, "0.013"),
"HIV-positive: suicidal att./id.": (1.15, 0.56, 2.36, "0.712"),
"Unknown status: depression": (3.82, 2.78, 5.27, "<0.001"),
"Unknown status: high anxiety": (3.09, 2.29, 4.17, "<0.001"),
"Unknown status: suicidal att./id.": (1.65, 1.22, 2.24, "0.001"),
}
TABLE2 = { # Table 2, sexual-minority adolescents only (N=497)
"HIV-positive: depression": (2.25, 0.78, 6.48, "0.132"),
"HIV-positive: high anxiety": (2.09, 0.93, 4.69, "0.073"),
"HIV-positive: suicidal att./id.": (1.47, 0.66, 3.28, "0.352"),
"Unknown status: depression": (3.11, 1.76, 5.49, "<0.001"),
"Unknown status: high anxiety": (2.32, 1.55, 3.48, "<0.001"),
"Unknown status: suicidal att./id.": (1.47, 0.98, 2.22, "0.064"),
}
TABLES1_HIVPOS_ANX = (3.12, 0.80, 12.28, "0.103") # Table S1, heterosexuals only (N=750)
TABLES1_UNKNOWN = { # Table S1, heterosexuals only (N=750), unknown status vs HIV-negative
"Unknown status: depression": (3.92, 2.66, 5.78, "<0.001"),
"Unknown status: high anxiety": (4.16, 2.63, 6.56, "<0.001"),
"Unknown status: suicidal att./id.": (2.06, 1.28, 3.30, "0.003"),
}
# Outcome counts (yes, no) by HIV status
OUTCOMES = ["depression", "high anxiety", "suicidal att./id."]
T2_COUNTS = { # Table 2, sexual minorities (N=497)
"negative": {"n": 246, "depression": (174, 72), "high anxiety": (79, 167), "suicidal att./id.": (77, 169)},
"positive": {"n": 29, "depression": (24, 5), "high anxiety": (14, 15), "suicidal att./id.": (12, 17)},
"unknown": {"n": 222, "depression": (201, 21), "high anxiety": (130, 92), "suicidal att./id.": (109, 113)},
}
S1_COUNTS = { # Table S1, heterosexuals (N=750)
"negative": {"n": 549, "depression": (156, 393), "high anxiety": (47, 502), "suicidal att./id.": (72, 477)},
"positive": {"n": 12, "depression": (5, 7), "high anxiety": (3, 9), "suicidal att./id.": (0, 12)},
"unknown": {"n": 189, "depression": (131, 58), "high anxiety": (65, 124), "suicidal att./id.": (39, 150)},
}
T1_COUNTS = { # Table 1, whole cohort (N=1247), rows exactly as printed
"negative": {"n": 795, "depression": (332, 79), "high anxiety": (195, 216), "suicidal att./id.": (149, 646)},
"positive": {"n": 41, "depression": (29, 12), "high anxiety": (17, 24), "suicidal att./id.": (12, 29)},
"unknown": {"n": 411, "depression": (330, 465), "high anxiety": (126, 669), "suicidal att./id.": (148, 263)},
}
T1_TOTAL_YES = {"depression": 691, "high anxiety": 338, "suicidal att./id.": 309} # Results, paragraph 1
# ---------------------------------------------------------------------------
# 2. Calculations
# ---------------------------------------------------------------------------
def wald_from_ci(aor, lo, hi):
"""SE of log(AOR), Wald z and two-sided p implied by a symmetric 95% CI on the log scale."""
se = (np.log(hi) - np.log(lo)) / (2 * Z975)
z = np.log(aor) / se
return se, z, 2 * norm.sf(abs(z)), float(np.exp((np.log(lo) + np.log(hi)) / 2))
def crude_or(a, b, c, d):
"""Crude OR (exposed yes/no = a/b, reference yes/no = c/d), Woolf 95% CI and Wald p."""
orr = (a * d) / (b * c)
se = np.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
return orr, np.exp(np.log(orr) - Z975 * se), np.exp(np.log(orr) + Z975 * se), 2 * norm.sf(abs(np.log(orr) / se))
def fmt_p(p):
return "<0.0001" if p < 1e-4 else f"{p:.4f}"
WALD = {}
print("=" * 100)
print("A. Wald check of every estimate in the HIV-status rows: p implied by the printed 95% CI")
print("=" * 100)
print(f"{'source':10s} {'estimate':36s} {'AOR (95% CI)':>20s} {'CI mid':>7s} {'SE':>6s} {'z':>6s} "
f"{'p from CI':>10s} {'printed p':>9s} {'CI excl. 1':>10s}")
for src, table in [("Abstract", ABSTRACT_SM), ("Table 1", TABLE1), ("Table 2", TABLE2),
("Table S1", {"HIV-positive: high anxiety": TABLES1_HIVPOS_ANX, **TABLES1_UNKNOWN})]:
for k, (a, lo, hi, rep) in table.items():
se, z, p, mid = wald_from_ci(a, lo, hi)
WALD[(src, k)] = (se, z, p)
print(f"{src:10s} {k:36s} {a:5.2f} ({lo:4.2f} to {hi:5.2f}) {mid:7.2f} {se:6.3f} {z:6.2f} "
f"{fmt_p(p):>10s} {rep:>9s} {str(lo > 1 or hi < 1):>10s}")
print()
print("=" * 100)
print("B. Which model do the abstract's four 'sexual minority' figures come from?")
print("=" * 100)
for k, v in ABSTRACT_SM.items():
in_t1 = TABLE1[k] == v
in_t2 = TABLE2[k] == v
t2 = TABLE2[k]
print(f"{k:36s} abstract {v[0]} ({v[1]} to {v[2]}) p={v[3]:7s} | identical to Table 1: {in_t1} | "
f"identical to Table 2: {in_t2} | Table 2 value {t2[0]} ({t2[1]} to {t2[2]}) p={t2[3]}")
n_match_t1 = sum(TABLE1[k] == v for k, v in ABSTRACT_SM.items())
print(f"-> {n_match_t1} of {len(ABSTRACT_SM)} abstract estimates match Table 1 exactly (AOR, both CI limits, p); "
f"{sum(TABLE2[k] == v for k, v in ABSTRACT_SM.items())} match Table 2.")
for k in ["Unknown status: depression", "Unknown status: high anxiety"]:
print(f" Table 2 AOR is {100*(1-TABLE2[k][0]/ABSTRACT_SM[k][0]):.1f}% smaller than the abstract's, {k}")
print(f" heterosexual share of the whole cohort: 750/1247 = {100*750/1247:.1f}%")
diffs = [abs(WALD[(src, k)][2] - float(v[3])) for src, t in [("Table 1", TABLE1), ("Table 2", TABLE2)]
for k, v in t.items() if not v[3].startswith("<")]
print(f" largest |Wald p - printed p| over Tables 1 and 2: {max(diffs):.4f}")
sig_abs = sum(v[1] > 1 for v in ABSTRACT_SM.values())
sig_t2 = sum(TABLE2[k][1] > 1 for k in ABSTRACT_SM)
print(f"-> CI excludes 1: {sig_abs} of 4 in the abstract; {sig_t2} of 4 for the same estimates in Table 2.")
for k in ABSTRACT_SM:
print(f" ratio abstract/Table 2 AOR, {k:36s}: {ABSTRACT_SM[k][0] / TABLE2[k][0]:.2f}")
print()
print("=" * 100)
print("B2. Results paragraph 5 (exploratory analyses): unknown status, heterosexuals (Table S1) vs sexual minorities (Table 2)")
print("=" * 100)
for k, v in TABLES1_UNKNOWN.items():
t2 = TABLE2[k]
print(f"{k:36s} Table S1 {v[0]} ({v[1]} to {v[2]}) p={v[3]:7s} | Table 2 {t2[0]} ({t2[1]} to {t2[2]}) p={t2[3]:7s} "
f"| Table 2 CI excludes 1: {t2[1] > 1} | larger AOR: {'Table 2 (sexual minorities)' if t2[0] > v[0] else 'Table S1 (heterosexuals)'}")
n_s1_larger = sum(TABLES1_UNKNOWN[k][0] > TABLE2[k][0] for k in TABLES1_UNKNOWN)
print(f"-> Table S1 AOR larger than Table 2 AOR for {n_s1_larger} of 3 unknown-status outcomes.")
print()
print("=" * 100)
print("C. Crude ORs from the Table 2 counts (sexual minorities), vs HIV-negative")
print("=" * 100)
CRUDE_T2 = {}
for grp in ["positive", "unknown"]:
for o in OUTCOMES:
a, b = T2_COUNTS[grp][o]
c, d = T2_COUNTS["negative"][o]
orr, lo, hi, p = crude_or(a, b, c, d)
key = ("HIV-positive: " if grp == "positive" else "Unknown status: ") + o
CRUDE_T2[key] = (orr, lo, hi, p)
print(f"{key:36s} {a}/{a+b} vs {c}/{c+d}: crude OR {orr:.2f} (95% CI {lo:.2f} to {hi:.2f}), p={'<0.001' if p < 0.001 else format(p, '.3f')} "
f"| Table 2 AOR {TABLE2[key][0]}")
print()
print("=" * 100)
print("D. Arithmetic of the HIV-status rows")
print("=" * 100)
for name, cnt, N in [("Table 2", T2_COUNTS, 497), ("Table S1", S1_COUNTS, 750), ("Table 1", T1_COUNTS, 1247)]:
print(f"{name}: HIV rows sum to {sum(v['n'] for v in cnt.values())} (stated N={N})")
for g in ["negative", "positive", "unknown"]:
s = T2_COUNTS[g]["n"] + S1_COUNTS[g]["n"]
print(f" {g:8s}: Table 2 n + Table S1 n = {T2_COUNTS[g]['n']} + {S1_COUNTS[g]['n']} = {s}; "
f"Table 1 n = {T1_COUNTS[g]['n']} -> equal: {s == T1_COUNTS[g]['n']}")
print("Row check, Table 1 (yes + no should equal the row total):")
for g in ["negative", "positive", "unknown"]:
for o in OUTCOMES:
y, n = T1_COUNTS[g][o]
exp = (T2_COUNTS[g][o][0] + S1_COUNTS[g][o][0], T2_COUNTS[g][o][1] + S1_COUNTS[g][o][1])
print(f" {g:8s} {o:18s}: printed {y}/{n} (sum {y+n}, row total {T1_COUNTS[g]['n']}, "
f"{'ok' if y + n == T1_COUNTS[g]['n'] else 'MISMATCH'}); Table 2 + Table S1 give {exp[0]}/{exp[1]}")
for o in OUTCOMES:
s = sum(T1_COUNTS[g][o][0] for g in T1_COUNTS)
print(f" Table 1 'yes' column sum, {o:18s}: {s} (Results text: {T1_TOTAL_YES[o]})")
print(" Table 1 printed % for 'Negative' depression: 332/411 = "
f"{100*332/411:.1f}% (printed 80.8); 332/795 = {100*332/795:.1f}%")
# ---------------------------------------------------------------------------
# 3. Figures
# ---------------------------------------------------------------------------
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
from matplotlib.transforms import Bbox
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, PANEL = "#eb6834", "#2a78d6", "#e4e3df", "#f3f2ef"
plt.rcParams.update({
"font.family": ["Inter", "DejaVu Sans"], "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "path", # text as outlines: no dependence on installed fonts
"axes.linewidth": 0.8,
})
LW = 2.0 # 2 px lines
MS = 9 # marker size (pt), >= 8 px
W = 7.2 # figure width (in)
SHORT = { # row labels for the four abstract claims
"HIV-positive: high anxiety": "HIV-positive vs negative\nhigh general anxiety",
"Unknown status: depression": "Unknown vs negative\ndepressive symptoms",
"Unknown status: high anxiety": "Unknown vs negative\nhigh general anxiety",
"Unknown status: suicidal att./id.": "Unknown vs negative\nsuicidal attempt/ideation",
}
KEYS = list(ABSTRACT_SM)
def save(fig, stem):
"""Save at exactly W inches wide: crop height to the content, never widen past the figure."""
os.makedirs(CAND, exist_ok=True)
fig.canvas.draw()
bb = fig.get_tightbbox(fig.canvas.get_renderer())
if bb.x0 < -0.01 or bb.x1 > W + 0.01:
print(f"WARNING {stem}: content spans x={bb.x0:.2f}..{bb.x1:.2f} in, beyond the {W} in figure")
box = Bbox.from_extents(0, bb.y0 - 0.12, W, bb.y1 + 0.12)
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
fig.savefig(os.path.join(CAND, f"{stem}.{ext}"), bbox_inches=box, **kw)
plt.close(fig)
print(f"wrote {stem}.svg / {stem}.png")
def ci_str(v):
return f"{v[0]:.2f} ({v[1]:.2f} to {v[2]:.2f}), p={v[3]}"
def p_print(v): # printed p in "p=0.013" / "p<0.001" form
return f"p{v[3]}" if v[3].startswith("<") else f"p={v[3]}"
# --- Candidate 1: text-panel diagram (abstract vs Table 1 vs Table 2) --------------------------
def cand1():
fig = plt.figure(figsize=(W, 5.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 74); ax.axis("off")
def panel(x, y, w, h, title, sub):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=1.2",
fc=PANEL, ec="#c9c7c1", lw=1))
ax.text(x + 1.8, y + h - 2.6, title, fontsize=11.5, weight="bold", va="top")
ax.text(x + 1.8, y + h - 6.6, sub, fontsize=10, color=INK2, va="top")
# Abstract panel
panel(0.5, 6, 41, 60, "Abstract, Results", "sentence on sexual minority adolescents")
quote = ("“Sexual minority adolescents living with HIV had significantly higher odds of "
"high general anxiety (AOR: 2.42; 95% CI: 1.21 to 4.84; p=0.013), while those with unknown "
"HIV status had significantly higher odds of … suicidal attempt/ideation "
"(AOR: 1.65; 95% CI: 1.22 to 2.24; p=0.001).”")
ax.text(2.8, 52, textwrap.fill(quote, 41), fontsize=10, va="top", style="italic", linespacing=1.35)
a1, a4 = ABSTRACT_SM[KEYS[0]], ABSTRACT_SM[KEYS[3]]
ax.text(2.8, 21, "HIV-positive, high anxiety", fontsize=10, color=INK2)
ax.text(2.8, 17.2, ci_str(a1), fontsize=10.5, color=ORANGE, weight="bold")
ax.text(2.8, 12, "Unknown status, suicidal att./ideation", fontsize=10, color=INK2)
ax.text(2.8, 8.2, ci_str(a4), fontsize=10.5, color=ORANGE, weight="bold")
# Table 1 panel
t1a, t1d = TABLE1[KEYS[0]], TABLE1[KEYS[3]]
panel(58.5, 38, 41, 28, "Table 1: whole cohort", "N=1247, sexual minority and heterosexual")
ax.text(60.3, 52, "HIV-positive, high anxiety", fontsize=10, color=INK2)
ax.text(60.3, 48.2, ci_str(t1a), fontsize=10.5, color=INK, weight="bold")
ax.text(60.3, 44, "Unknown status, suicidal att./ideation", fontsize=10, color=INK2)
ax.text(60.3, 40.2, ci_str(t1d), fontsize=10.5, color=INK, weight="bold")
# Table 2 panel
t2a, t2d = TABLE2[KEYS[0]], TABLE2[KEYS[3]]
panel(58.5, 6, 41, 28, "Table 2: sexual minorities only", "N=497, the group the abstract names")
ax.text(60.3, 20, "HIV-positive, high anxiety", fontsize=10, color=INK2)
ax.text(60.3, 16.2, ci_str(t2a), fontsize=10.5, color=BLUE, weight="bold")
ax.text(60.3, 12, "Unknown status, suicidal att./ideation", fontsize=10, color=INK2)
ax.text(60.3, 8.2, ci_str(t2d), fontsize=10.5, color=BLUE, weight="bold")
# connectors
ax.add_patch(FancyArrowPatch((42, 44), (58.2, 50), arrowstyle="-|>", mutation_scale=14,
lw=LW, color=INK2, connectionstyle="arc3,rad=-0.15"))
ax.text(50.2, 53.5, "identical\nnumbers\n(all 4\nestimates)", ha="center", fontsize=10, color=INK, linespacing=1.15)
ax.add_patch(FancyArrowPatch((42, 28), (58.2, 22), arrowstyle="-|>", mutation_scale=14,
lw=LW, color=INK2, ls=(0, (4, 3)), connectionstyle="arc3,rad=0.15"))
ax.text(50.2, 10, "population\nnamed in\nthe abstract", ha="center", fontsize=10, color=INK, linespacing=1.15)
ax.text(99.5, 2.2, "Table 2: both CIs include 1, not significant at the paper's 5% level",
ha="right", fontsize=10, color=INK2)
# legend
h = [Line2D([], [], marker="s", ls="", ms=MS, color=ORANGE, label="value printed in the abstract"),
Line2D([], [], marker="s", ls="", ms=MS, color=BLUE, label="paper's own sexual-minority model (Table 2)")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False, fontsize=10)
save(fig, "cand1_text_panels")
# --- Candidate 2: dot-and-interval plot, log OR axis -------------------------------------------
def cand2():
fig, ax = plt.subplots(figsize=(W, 4.3))
fig.subplots_adjust(left=0.235, right=0.725, bottom=0.14, top=0.80)
tr = ax.get_yaxis_transform() # x in axes fraction, y in data
off = 0.17
for i, k in enumerate(KEYS):
y = len(KEYS) - 1 - i
for dy, v, col, mk in [(off, ABSTRACT_SM[k], ORANGE, "o"), (-off, TABLE2[k], BLUE, "s")]:
ax.plot([v[1], v[2]], [y + dy] * 2, color=col, lw=LW, solid_capstyle="butt")
ax.plot(v[0], y + dy, marker=mk, ms=MS, color=col, mec=BG, mew=1.5, ls="")
ax.text(1.03, y + dy, f"{v[0]:.2f} ({v[1]:.2f}–{v[2]:.2f}) {p_print(v)}",
va="center", fontsize=10, color=INK, transform=tr)
ax.axvline(1, color=INK, lw=1.0, zorder=2)
ax.text(1.03, len(KEYS) - 0.45, "no association (OR = 1)", fontsize=10, color=INK, va="bottom")
ax.set_xscale("log"); ax.set_xlim(0.5, 8)
ticks = [0.5, 1, 2, 4, 8]
ax.set_xticks(ticks); ax.set_xticklabels([str(t) for t in ticks]); ax.minorticks_off()
ax.set_yticks(range(len(KEYS))); ax.set_yticklabels([SHORT[k] for k in KEYS][::-1], fontsize=10)
ax.set_ylim(-0.6, len(KEYS) - 0.2)
ax.set_xlabel("Adjusted odds ratio (log scale) with 95% CI")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.text(1.03, len(KEYS) - 0.45, "AOR (95% CI), printed p", fontsize=10, color=INK2, va="bottom", transform=tr)
h = [Line2D([], [], marker="o", color=ORANGE, lw=LW, ms=MS,
label="Abstract, given for sexual minorities (= Table 1, whole cohort, N=1247)"),
Line2D([], [], marker="s", color=BLUE, lw=LW, ms=MS,
label="Table 2, sexual-minority model (N=497)")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, 1.0), bbox_transform=fig.transFigure,
ncol=1, frameon=False, fontsize=10)
save(fig, "cand2_forest")
# --- Candidate 3: Wald z recomputed from each printed CI, with the 5% threshold ---------------
def cand3():
fig, ax = plt.subplots(figsize=(W, 3.9))
fig.subplots_adjust(left=0.235, right=0.97, bottom=0.17, top=0.86)
for i, k in enumerate(KEYS):
y = len(KEYS) - 1 - i
za = WALD[("Abstract", k)][1]; zt = WALD[("Table 2", k)][1]
ax.plot([zt, za], [y, y], color="#b9b7b0", lw=LW, zorder=1)
ax.plot(za, y, "o", ms=MS + 1, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.plot(zt, y, "s", ms=MS + 1, color=BLUE, mec=BG, mew=1.5, zorder=3)
ax.text(za + 0.25, y + 0.16, f"z={za:.2f}", fontsize=10, color=INK, va="bottom")
ax.text(zt - 0.25, y + 0.16, f"z={zt:.2f}", fontsize=10, color=INK, va="bottom", ha="right")
ax.axvline(Z975, color=INK, lw=1.0, zorder=2)
ax.text(Z975 + 0.08, -0.62, "z = 1.96 (two-sided p = 0.05)", fontsize=10, color=INK, va="bottom")
ax.set_xlim(0, 9.5); ax.set_ylim(-0.7, len(KEYS) - 0.3)
ax.set_yticks(range(len(KEYS))); ax.set_yticklabels([SHORT[k] for k in KEYS][::-1], fontsize=10)
ax.set_xlabel("Wald z = ln(AOR) / SE, SE recomputed from the printed 95% CI")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h = [Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="from the abstract's CI (= Table 1)"),
Line2D([], [], marker="s", ls="", ms=MS, color=BLUE, label="from Table 2's CI (sexual minorities)")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, 1.0), bbox_transform=fig.transFigure,
ncol=2, frameon=False, fontsize=10)
save(fig, "cand3_wald_z")
# --- Candidate 4: comparison grid (abstract | Table 1 | Table 2) ------------------------------
def cand4():
fig = plt.figure(figsize=(W, 4.3))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(-2, 58); ax.axis("off")
cols = [(0.5, "Estimate"), (26, "Abstract\n(for sexual minorities)"),
(51.5, "Table 1\n(whole cohort,\nN=1247)"), (76, "Table 2\n(sexual minorities,\nN=497)")]
for x, t in cols:
ax.text(x, 53, t, fontsize=10, weight="bold", va="top")
ax.plot([0.5, 99.5], [45, 45], color=INK2, lw=0.8)
for i, k in enumerate(KEYS):
y = 40 - i * 10
if i % 2 == 0:
ax.add_patch(FancyBboxPatch((0.5, y - 4.6), 99, 9.2, boxstyle="square,pad=0", fc=PANEL, ec="none"))
ax.text(0.5, y, SHORT[k], fontsize=10, va="center", linespacing=1.2)
a, t1, t2 = ABSTRACT_SM[k], TABLE1[k], TABLE2[k]
for x, v, col in [(26, a, ORANGE), (51.5, t1, INK), (76, t2, BLUE)]:
ax.text(x, y + 1.6, f"{v[0]:.2f} ({v[1]:.2f}–{v[2]:.2f})", fontsize=10.5, color=col,
weight="bold", va="center")
sig = v[1] > 1 or v[2] < 1
ax.text(x, y - 2.0, f"{p_print(v)} · {'CI excludes 1' if sig else 'CI includes 1'}",
fontsize=10, color=INK2, va="center")
ax.text(48.9, y + 1.6, "=", fontsize=12, color=INK, ha="center", va="center", weight="bold")
ax.text(0.5, 2.0, "'=' marks an exact match of AOR, both CI limits and p.\nOrange: value printed in the abstract; "
"blue: paper's own sexual-minority model.", fontsize=10, color=INK2, va="center", linespacing=1.3)
save(fig, "cand4_grid")
# --- Candidate 5: one estimate, all the paper's versions of it --------------------------------
def cand5():
k = KEYS[0]
rows = [
("Abstract: “sexual minority\nadolescents”", ABSTRACT_SM[k], ORANGE, "o", True),
("Table 1: whole cohort\n(N=1247)", TABLE1[k], INK2, "D", False),
("Table 2: sexual minorities\n(N=497)", TABLE2[k], BLUE, "s", True),
("Crude OR, Table 2 counts\n(14/29 vs 79/246)", CRUDE_T2[k], BLUE, "s", False),
("Table S1: heterosexuals\nonly (N=750)", TABLES1_HIVPOS_ANX, INK2, "D", False),
]
fig, ax = plt.subplots(figsize=(W, 4.4))
fig.subplots_adjust(left=0.29, right=0.725, bottom=0.14, top=0.74)
tr = ax.get_yaxis_transform()
for i, (lab, v, col, mk, filled) in enumerate(rows):
y = len(rows) - 1 - i
ax.plot([v[1], v[2]], [y, y], color=col, lw=LW, solid_capstyle="butt")
ax.plot(v[0], y, marker=mk, ms=MS, color=col if filled else BG, mec=col, mew=2, ls="")
p = v[3] if isinstance(v[3], str) else f"{v[3]:.3f}"
ptxt = f"p{p}" if p.startswith("<") else f"p={p}"
ax.text(1.03, y, f"{v[0]:.2f} ({v[1]:.2f}–{v[2]:.2f}) {ptxt}", va="center", fontsize=10, transform=tr)
ax.axvline(1, color=INK, lw=1.0, zorder=2)
ax.text(1.03, len(rows) - 0.55, "OR = 1", fontsize=10, color=INK, va="bottom")
ax.set_xscale("log"); ax.set_xlim(0.5, 13)
ticks = [0.5, 1, 2, 4, 8]
ax.set_xticks(ticks); ax.set_xticklabels([str(t) for t in ticks]); ax.minorticks_off()
ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows][::-1], fontsize=10)
ax.set_ylim(-0.6, len(rows) - 0.4)
ax.set_xlabel("OR, high general anxiety, HIV-positive vs negative\n(log scale, 95% CI)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h = [Line2D([], [], marker="o", color=ORANGE, lw=LW, ms=MS, label="printed in the abstract"),
Line2D([], [], marker="s", color=BLUE, lw=LW, ms=MS, label="sexual-minority data (filled: Table 2 AOR; open: crude OR recomputed)"),
Line2D([], [], marker="D", color=INK2, lw=LW, ms=MS, mfc=BG, label="other populations in the paper")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, 1.0), bbox_transform=fig.transFigure,
ncol=1, frameon=False, fontsize=10)
save(fig, "cand5_one_estimate")
cand1(); cand2(); cand3(); cand4(); cand5()
for src, dst in [("cand1_text_panels", "fig1"), ("cand2_forest", "fig2")]:
for ext in ("svg", "png"):
shutil.copyfile(os.path.join(CAND, f"{src}.{ext}"), os.path.join(HERE, f"{dst}.{ext}"))
print("fig1 = cand1_text_panels, fig2 = cand2_forest (provisional, pending the choice of design)")
Abstract names G3244A where the body names G3242A as the second termination-abolishing pathogenic mutation
Cell (2010) · doi:10.1016/j.cell.2010.05.018 · PMC2887341
Summary
The Summary (abstract) of this crystal-structure paper on the human mitochondrial transcription terminator MTERF1 ends its results with a disease link: "the mitochondrial pathogenic G3249A and G3244A mutations interfere with key interactions for sequence recognition, eliminating termination." The body of the paper names a different second mutation, G3242A, in each of the three places where it states this result: the Introduction, the Results section on pathogenic mutations, and the Conclusions. The body also describes G3244A in terms that cannot fit the abstract sentence: its base pair makes "no specific interaction" with MTERF1 in the crystal structure, the mutation does not affect DNA binding, and its effect on termination is "moderate". A reader of the abstract alone is therefore told that the wrong variant loses a recognition contact and abolishes termination, and is never told about G3242A, the variant the paper's own data support.
What the paper reports
Abstract (Summary), final results sentence:
Finally, we have demonstrated that the mitochondrial pathogenic G3249A and G3244A mutations interfere with key interactions for sequence recognition, eliminating termination.
Introduction, last paragraph:
We have shown that two mutations, G3249A that causes a variant of Kearns-Sayre syndrome (Seneca et al., 2001) and G3242A that is associated with an uncharacterized mitochondrial disorder (Mimaki et al., 2009), interfere with protein-DNA interactions that are key for sequence recognition by MTERF1 and as a consequence eliminate its transcriptional termination activity.
Results and discussion, "Pathogenic mutations in the leu-tRNA", first paragraph (structure and binding):
All remaining mutations would not be expected to severely conflict with MTERF1 sequence recognition, except for G3242A and G3249A. G3242 interacts in the structure with Arg251, while G3249 forms a double hydrogen bond with Arg387 (Figure 6B). Both of these mutations would eliminate a guanine-arginine interaction: the hydrogen bond between O6 of the guanine and the amino group of the arginine could not be formed in the mutant DNA. Consistent with the mild decrease in binding observed in the R251A mutant, G3242A resulted in weak or no effect on binding. However, the G3249A mutation appears to completely eliminate specific DNA binding (Figure 6A and 3SE), consistent with the effect of the R387A mutation.
Same section, second paragraph (termination assays, Figure 6C/D):
As expected, only residual termination is observed in both orientations with the G3249A mutation. In addition, consistent with what was observed with the R251A mutant, the G3242A mutation, while not substantially altering binding, results nevertheless in a strong decrease in termination.
Unexpectedly, one additional mutation appears to moderately affect termination: G3244A, while not affecting DNA binding, results in a moderate but reproducible effect on termination. Interestingly, no specific interaction is observed in the crystal structure with any of the nucleotides in this base pair. However, it is adjacent to the extrahelical A:T base pair and displays a high base pair buckle (see Experimental Procedures).
Results and discussion, "Conclusions":
An important conclusion derived from the crystal structure is that two pathogenic mutations that interfere with arginine-guanine interactions involved in sequence recognition by MTERF1 result in a strong impairment in the ability of MTERF1 to terminate transcription. G3242A does not substantially affect binding, but very strongly reduces the ability of MTERF1 to terminate transcription, while G3249A appears to completely eliminate specific binding by MTERF1 and consequently severely impairs termination. Their strong effect on termination suggests that the pathogenic effects of the G3249A and G3242A MtDNA mutations might be at least partially mediated by interfering with the activity of MTERF1 at the leu-tRNA site.
Further experiments will be needed to address whether the G3249A and G3242A mutations result in transcriptional alterations in vivo and if this effect is sufficient to explain their clinical phenotype.
What the paper's own numbers show
Method. No arithmetic is involved. The error concerns which mutation is named, so I checked the paper's internal consistency. I parsed the PMC-archived full text and counted every mention of the five leu-tRNA variants the paper discusses (G3242A, A3243G, A3243T, G3244A, G3249A) in each section. I then extracted the pair of mutations that each section names as disrupting sequence recognition and eliminating or strongly impairing termination. Finally, I tabulated what the body says about each candidate's structural contact, its effect on binding and its effect on termination. recompute.py.txt holds the excerpts verbatim, checks all 33 excerpt sentences against the archived text (all found), and prints every value below.
| Quantity | Location | Reported | Recomputed from the paper's own text |
|---|---|---|---|
| Second mutation said to disrupt recognition and eliminate termination | Abstract (Summary) | G3244A (paired with G3249A) | Contradicted by all three body sections below |
| The same pair | Introduction, last paragraph | G3249A and G3242A | G3242A |
| Mutations expected to conflict with sequence recognition | Results, Pathogenic mutations, first paragraph | "except for G3242A and G3249A" | G3242A |
| The same pair | Conclusions (pair named twice; G3242A mentioned 4 times) | G3249A and G3242A | G3242A |
| Agreement of the three body sections with each other | Introduction, Results, Conclusions | — | All three name G3249A + G3242A |
| Agreement of the abstract with the body | Abstract vs body | — | No. The pairs differ in one mutation, G3244A vs G3242A, a single-character difference |
| Mentions of G3242A / G3244A / G3249A | Abstract | 0 / 1 / 1 | G3242A, the body's second mutation, never appears in the abstract |
| Mentions of G3242A / G3244A / G3249A | Introduction | 1 / 0 / 1 | — |
| Mentions of G3242A / G3244A / G3249A | Results, Pathogenic mutations | 3 / 2 / 3 | G3244A appears only here, as "one additional mutation" |
| Mentions of G3242A / G3244A / G3249A | Conclusions | 4 / 0 / 4 | — |
| Mentions in the whole article | All sections | G3242A 8, G3244A 3, G3249A 9 | — |
| Structural contact of the mutated base pair | Results, Pathogenic mutations | G3242 contacts Arg251; G3249 is double hydrogen-bonded to Arg387; G3244 base pair: "no specific interaction is observed" | Only G3242A and G3249A can remove a recognition contact |
| Effect on DNA binding | Results, Pathogenic mutations | G3242A "weak or no effect"; G3244A "not affecting DNA binding"; G3249A "completely eliminate specific DNA binding" | — |
| Effect on termination | Results, Pathogenic mutations; Conclusions | G3249A "only residual termination"; G3242A "strong decrease" / "very strongly reduces"; G3244A "moderate but reproducible effect" | G3244A's effect is moderate, not elimination |
By the paper's own account, G3244A meets neither condition in the abstract sentence. It removes no recognition contact, and it does not eliminate termination. G3242A meets the first condition and, in the body's words, comes close to meeting the second ("very strongly reduces").
Figure
The top panel holds the abstract's only sentence about the pathogenic mutations, with the second mutation it names (G3244A) in orange. The two lower panels hold verbatim excerpts from the body of the same article. The left panel collects what the Introduction, Results and Conclusions say about G3242A, the mutation they name alongside G3249A: it contacts Arg251, and it strongly decreases termination while leaving binding largely intact. G3242A does not appear in the abstract (dashed connector). The right panel collects everything the body says about G3244A. It appears only in the Results, as "one additional mutation" whose base pair makes no specific contact with MTERF1, which does not affect binding and which has a moderate effect on termination. The solid connector marks the conflict between the abstract's claim about G3244A and the body's description of it. All text is quoted from the PMC-archived article, and the section counts come from recompute.py.txt.
Why it matters
- It is the abstract's only disease-related result. The abstract's closing claim ("suggest a link between mitochondrial disease and the regulation of mitochondrial transcription") rests on this sentence. The version-of-record Highlights mention "Two mitochondrial pathogenic mutations" without naming them, so the abstract is the only summary that names the variants. In the abstract, the second disease variant tied to loss of MTERF1 function is the wrong one.
- Both the mechanism and the size of the effect are misattributed. For G3244A the abstract claims loss of a sequence-recognition contact and elimination of termination. The body reports no contact, normal binding and a moderate effect, which it tentatively explains by destabilization of a buckled G:C pair next to the everted A:T pair. A reader of the abstract would place G3244A in the wrong mechanistic category.
- The variant the data support is missing. G3242A, which the Conclusions call one of "two pathogenic mutations that interfere with arginine-guanine interactions", is not named in the abstract. Abstract-only readers would not learn that this variant was studied or what effect it had.
- The abstract is the version that circulates. The same sentence appears in PubMed/Europe PMC (PMID 20550934), in indexing services and in the OSTI record. It is the text most likely to be quoted, for example when variant pathogenicity evidence is assessed from the literature.
- What is not affected. The structure, the DNA unwinding and three-nucleotide eversion, the role of base flipping in stable binding and termination, and the body's account of the pathogenic mutations are all internally consistent. The data for G3242A and G3244A (Figure 6A, C, D) are described consistently in the body.
Explanations considered
- A single-character slip (G3242A typed as G3244A). The two names differ in one digit, and both variants are discussed in the same section, so this is plausible. It explains how the sentence could arise. It does not make the printed abstract correct, because the abstract still attributes a recognition contact and abolished termination to a variant that the paper shows has neither.
- A text-conversion or archiving artifact. Ruled out. The version of record (the Cell full-text page, read from a Wayback Machine snapshot because the live publisher pages refused automated access) has the same Summary sentence, and the same body sentences that name G3242A. The PMC author manuscript, PubMed, Europe PMC, Semantic Scholar and OSTI 1020213 all carry the same abstract wording.
- A different numbering convention, for example heavy-strand versus light-strand positions. Ruled out. mtDNA positions number base pairs, not strands, and the paper uses one numbering throughout. It discusses G3242, A3243 and G3244 as three distinct, adjacent positions with different structural roles (Arg251 contact, everted nucleotide, buckled base pair without contact).
- The abstract summarizing a different experiment, such as the forward-orientation assay or the supplement. Not supported. The Results describe the termination assays in both orientations together ("The results (Figure 6C/D) are mostly consistent with the DNA binding data") and give no orientation in which G3244A eliminates termination. The supplement (Figures S1–S3: alignment, conservation, ITC titrations) concerns binding, and the body states that G3244A does not affect binding. I could not download the supplement files or view the Figure 6 images (see Status), so this rests on the text's description of those data.
- "Eliminating termination" used loosely for "strongly impairing". Even on this reading the sentence fits G3242A ("very strongly reduces") but not G3244A ("moderate"), and the structural half of the sentence ("interfere with key interactions for sequence recognition") still excludes G3244A. The body itself uses "eliminate" for G3242A in the Introduction but "strong decrease" in the Results. That is a lesser wording difference within the body, separate from the abstract error.
- A deliberate claim that G3244A acts through sequence recognition. Not supported anywhere in the paper. The only mechanism the body proposes for G3244A is a slight destabilization of the bound protein, offered with "It is possible that".
Status
- Version of record. Checked. The live publisher pages (doi.org, cell.com full text and PDF, ScienceDirect) returned HTTP 403 or a redirect wall to automated access. The Elsevier API without a key gives metadata only (open archive, cover date 2010-06-11). I read the version of record from a Wayback Machine snapshot of the Cell full-text page (web.archive.org/web/20230111224332/https://www.cell.com/cell/fulltext/S0092-8674(10)00554-4). Its Summary sentence is identical to the PMC text. Its Introduction, Results and Conclusions name G3242A exactly as quoted above. Its Highlights say only "Two mitochondrial pathogenic mutations interfere with transcriptional termination" and do not name the mutations. All quotes in this write-up were confirmed verbatim against the PMC-archived text (author manuscript NIHMS208049).
- Supplements. PMC lists NIHMS208049-supplement-01.doc and -02.pdf. The version of record lists Figures S1–S3. Neither set could be downloaded (PMC download wall, no Europe PMC supplementary bundle, cell.com 403), and the Figure 6 images could not be fetched either. The conflict is between two passages of the main text, so the supplement cannot remove it.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI; its only relation is a Faculty Opinions review link. PMC and Europe PMC list no correction notices. Europe PMC links one "Comment in" item, a 2010 Cell Metabolism commentary (PMID 20620988), which is not a correction. The PMC record carries a related-article link to a retraction notice (PMC4698954); that notice concerns a different, cited paper on a related protein, not this article. PubPeer's API refused automated access (Cloudflare challenge), and a web search for the title on PubPeer returned no PubPeer entry. A web search for the abstract wording with "erratum" or "correction" found nothing relevant.
- Sources. PMC article PMC2887341 (author manuscript NIHMS208049); Wayback Machine snapshot of the Cell full-text page; PubMed/Europe PMC record PMID 20550934; Crossref metadata for 10.1016/j.cell.2010.05.018; OSTI 1020213; PubPeer (via API and web search).
Suggested correction
- In the Summary (abstract), replace "G3249A and G3244A" with "G3249A and G3242A", so that the sentence reads: "Finally, we have demonstrated that the mitochondrial pathogenic G3249A and G3242A mutations interfere with key interactions for sequence recognition, eliminating termination."
- Consider aligning the strength of the wording with the Results and Conclusions. G3249A abolishes specific binding and leaves only residual termination, while G3242A barely affects binding but strongly reduces termination. "Severely impairing termination" would match both. The same applies to "eliminate" for G3242A in the Introduction.
- Optionally, mention in the abstract that G3244A, which makes no contact with MTERF1, has a moderate effect on termination, because the body presents this as an unexpected finding.
- Because the abstract propagates through PubMed and indexing services, a published correction notice linked to the article record would be more effective than a silent change to the online text.
Recomputation script (python recompute.py.txt)
"""Recompute the checks behind the write-up for PMC2887341
(Cell 2010; doi:10.1016/j.cell.2010.05.018), and redraw fig1 in this file's folder.
Run: python recompute.py.txt (needs matplotlib)
There is no arithmetic to redo: the error is a mismatch between the Summary
(abstract) and the body text in which pathogenic mtDNA mutation is named as the
second one that disrupts sequence recognition and eliminates termination. This
script therefore
1. counts every mention of the leu-tRNA mutations (G3242A, A3243G, A3243T,
G3244A, G3249A) in each section,
2. extracts the pair of mutations each section names as eliminating or
strongly impairing termination,
3. tabulates what the body says about G3242A and G3244A (structural contact,
effect on binding, effect on termination), and
4. draws fig1.
The paragraphs used are copied verbatim below from the article as archived in
PubMed Central (author manuscript). If the repository tool src/show.py is
reachable, the script also re-reads the archived full text, checks that every
excerpt occurs in it verbatim, and repeats the counts on the whole article.
"""
import os
import re
import subprocess
import collections
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs
SECTIONS = collections.OrderedDict()
SECTIONS["Abstract (Summary)"] = (
"Defects in mitochondrial gene expression are associated with aging and disease. Mterf proteins "
"have been implicated in modulating transcription, replication and protein synthesis. We have solved "
"the structure of a member of this family, the human mitochondrial transcriptional terminator MTERF1, "
"bound to dsDNA containing the termination sequence. The structure indicates that upon sequence "
"recognition MTERF1 unwinds the DNA molecule, promoting eversion of three nucleotides. Base flipping "
"is critical for stable binding and transcriptional termination. Additional structural and biochemical "
"results provide insight into the DNA binding mechanism and explain how MTERF1 recognizes its target "
"sequence. Finally, we have demonstrated that the mitochondrial pathogenic G3249A and G3244A mutations "
"interfere with key interactions for sequence recognition, eliminating termination. Our results provide "
"insight into the role of mterf proteins and suggest a link between mitochondrial disease and the "
"regulation of mitochondrial transcription.")
SECTIONS["Introduction"] = (
"Interestingly, the pathogenic A3243G MELAS mutation was shown to decrease the ability of MTERF1 to "
"terminate transcription in vitro (Hess et al., 1991) and yet no obvious transcriptional alterations "
"are observed in cells carrying this MELAS mutation (Chomyn et al. 1992). "
"We have shown that two mutations, G3249A that causes a variant of Kearns-Sayre syndrome (Seneca et al., "
"2001) and G3242A that is associated with an uncharacterized mitochondrial disorder (Mimaki et al., 2009), "
"interfere with protein-DNA interactions that are key for sequence recognition by MTERF1 and as a "
"consequence eliminate its transcriptional termination activity.")
SECTIONS["Results: Pathogenic mutations (binding)"] = (
"It is important to note that the pathogenic effects of these mutations might be simply due to an "
"alteration on the leu-tRNA structure, and this appears indeed to be the case for the A3243G mutation "
"(Sasarman et al., 2008). A3243G is frequently identified in MELAS patients and has been previously "
"shown to slightly reduce MTERF1 binding (Hess et al., 1991) and proposed to interfere in vitro with "
"transcriptional termination (Hess et al., 1991; Chomyn et al., 1992). "
"Our measurements confirm that A3243G results in slightly decreased binding (Figure 6A and 3SE). "
"A similar binding affinity was observed for the A3243T mutation (Figure 6A). "
"All remaining mutations would not be expected to severely conflict with MTERF1 sequence recognition, "
"except for G3242A and G3249A. G3242 interacts in the structure with Arg251, while G3249 forms a double "
"hydrogen bond with Arg387 (Figure 6B). Both of these mutations would eliminate a guanine-arginine "
"interaction: the hydrogen bond between O6 of the guanine and the amino group of the arginine could "
"not be formed in the mutant DNA. Consistent with the mild decrease in binding observed in the R251A "
"mutant, G3242A resulted in weak or no effect on binding. However, the G3249A mutation appears to "
"completely eliminate specific DNA binding (Figure 6A and 3SE), consistent with the effect of the "
"R387A mutation.")
SECTIONS["Results: Pathogenic mutations (termination)"] = (
"As expected, only residual termination is observed in both orientations with the G3249A mutation. "
"In addition, consistent with what was observed with the R251A mutant, the G3242A mutation, while not "
"substantially altering binding, results nevertheless in a strong decrease in termination. The A3243G "
"mutation moderately reduces termination, while termination on A3243T is only marginally weaker than "
"for the wt sequence. The reduction in termination observed for A3243G is consistent with what was "
"previously reported (Hess et al., 1991). Unexpectedly, one additional mutation appears to moderately "
"affect termination: G3244A, while not affecting DNA binding, results in a moderate but reproducible "
"effect on termination. Interestingly, no specific interaction is observed in the crystal structure "
"with any of the nucleotides in this base pair. However, it is adjacent to the extrahelical A:T base "
"pair and displays a high base pair buckle (see Experimental Procedures). It is possible that the "
"strength of the G:C base pair in the wt sequence is important to maintain this conformation and that "
"the G3244A mutation thus leads to a slight destabilization of the bound MTERF1 that is sufficient to "
"make it more readily displaced by the RNA polymerase.")
SECTIONS["Conclusions"] = (
"An important conclusion derived from the crystal structure is that two pathogenic mutations that "
"interfere with arginine-guanine interactions involved in sequence recognition by MTERF1 result in a "
"strong impairment in the ability of MTERF1 to terminate transcription. G3242A does not substantially "
"affect binding, but very strongly reduces the ability of MTERF1 to terminate transcription, while "
"G3249A appears to completely eliminate specific binding by MTERF1 and consequently severely impairs "
"termination. Their strong effect on termination suggests that the pathogenic effects of the G3249A "
"and G3242A MtDNA mutations might be at least partially mediated by interfering with the activity of "
"MTERF1 at the leu-tRNA site. The G3249A mutation leads to the development of a variant of "
"Kearns-Sayre syndrome (Seneca et al., 2001), a mitochondrial myopathy, while the G3242A mutation has "
"been associated with an uncharacterized mitochondrial disorder (Mimaki et al., 2009). "
"Further experiments will be needed to address whether the G3249A and G3242A mutations result in "
"transcriptional alterations in vivo and if this effect is sufficient to explain their clinical phenotype.")
FIG6_LEGEND = ("The DNA residues involved in arginine guanine interactions with MTERF1 are indicated "
"by an asterisk.")
MUT = re.compile(r"\b[GA]32(?:42|43|44|49)[AGT]\b")
ORDER = ["G3242A", "A3243G", "A3243T", "G3244A", "G3249A"]
def counts(text):
return collections.Counter(MUT.findall(text))
def sentences(text):
return [s.strip() for s in re.split(r"(?<=\.)\s+(?=[A-Z])", text) if s.strip()]
# ------------------------------------------------- 1. mentions per section
print("=== 1. Mentions of each leu-tRNA mutation, by section (verbatim excerpts) ===")
print(f" {'section':45s} " + " ".join(f"{m:>7s}" for m in ORDER))
EXC = {}
for sec, txt in SECTIONS.items():
c = counts(txt)
EXC[sec] = c
print(f" {sec:45s} " + " ".join(f"{c.get(m, 0):7d}" for m in ORDER))
# ------------------------------------- 2. which pair each section names
print("\n=== 2. The pair each section names as disrupting recognition / eliminating termination ===")
PAIR_PATTERNS = [
("Abstract (Summary)", r"pathogenic (G\d{4}A) and (G\d{4}A) mutations interfere"),
("Introduction", r"two mutations, (G\d{4}A) that causes.*?and (G\d{4}A) that is associated"),
("Results: Pathogenic mutations (binding)", r"except for (G\d{4}A) and (G\d{4}A)\."),
("Conclusions", r"pathogenic effects of the (G\d{4}A) and (G\d{4}A) MtDNA mutations"),
]
PAIRS = {}
for sec, pat in PAIR_PATTERNS:
m = re.search(pat, SECTIONS[sec])
PAIRS[sec] = frozenset(m.groups())
print(f" {sec:45s} {' + '.join(sorted(PAIRS[sec], reverse=True))}")
body = {p for s, p in PAIRS.items() if s != "Abstract (Summary)"}
print(f" body sections agree with each other: {len(body) == 1}")
print(f" abstract pair equals body pair: {PAIRS['Abstract (Summary)'] in body}")
diff = PAIRS["Abstract (Summary)"] ^ next(iter(body))
print(f" mutations that differ: {sorted(diff)} (one digit apart: "
f"{sum(a != b for a, b in zip(*sorted(diff)))} character differs)")
print(f" G3242A mentions in the abstract: {EXC['Abstract (Summary)'].get('G3242A', 0)}; "
f"G3244A mentions in Introduction + Conclusions: "
f"{EXC['Introduction'].get('G3244A', 0) + EXC['Conclusions'].get('G3244A', 0)}")
# ------------------------------------- 3. what the body says about each
print("\n=== 3. What the body says about G3242A, G3244A and G3249A ===")
ALL = " ".join(v for k, v in SECTIONS.items() if k != "Abstract (Summary)")
KEYS = {
"G3242A": ["G3242 interacts", "G3242A resulted", "the G3242A mutation, while", "G3242A does not"],
"G3244A": ["G3244A, while", "no specific interaction", "the G3244A mutation thus"],
"G3249A": ["G3249 forms", "the G3249A mutation appears", "residual termination", "G3249A appears"],
}
for mut, keys in KEYS.items():
print(f" {mut}:")
for s in sentences(ALL):
if any(k in s for k in keys):
print(" -", textwrap.shorten(s, 230, placeholder=" ..."))
SUMMARY = { # attribute -> (G3242A, G3244A, G3249A), each a verbatim phrase from the body
"contact in the crystal structure": ("G3242 interacts in the structure with Arg251",
"no specific interaction is observed",
"G3249 forms a double hydrogen bond with Arg387"),
"effect on DNA binding": ("weak or no effect on binding", "not affecting DNA binding",
"completely eliminate specific DNA binding"),
"effect on termination": ("a strong decrease in termination",
"a moderate but reproducible effect on termination",
"only residual termination"),
}
print("\n attribute | G3242A | G3244A | G3249A")
for a, vals in SUMMARY.items():
for v in vals:
assert v in ALL, v
print(f" {a:33s} | " + " | ".join(vals))
# ------------------------- 4. optional cross-check against the archived full text
print("\n=== 4. Cross-check against the archived full text (src/show.py) ===")
FULLCOUNTS = None
show = os.path.join(HERE, "..", "..", "..", "src", "show.py")
py = "/home/ubuntu/papers-data/venv/bin/python"
if os.path.exists(show) and os.path.exists(py):
out = subprocess.run([py, show, "PMC2887341", "--full"], capture_output=True, text=True,
cwd=os.path.abspath(os.path.join(HERE, "..", "..", ".."))).stdout
flat = re.sub(r"\s+", " ", out)
excerpts = [s for t in SECTIONS.values() for s in sentences(t)] + [FIG6_LEGEND]
missing = [s for s in excerpts if s not in flat]
print(f" excerpt sentences found verbatim: {len(excerpts) - len(missing)} of {len(excerpts)}")
for s in missing:
print(" NOT FOUND:", s[:120])
sec = "header"
FULLCOUNTS = collections.defaultdict(collections.Counter)
for line in out.splitlines():
if line.startswith("ABSTRACT:"):
sec = "Abstract"
elif line.startswith("## "):
sec = line[3:].strip()
elif line.startswith("BACK MATTER"):
sec = "Back matter (figure legends)"
FULLCOUNTS[sec].update(MUT.findall(line))
print(" mentions by section in the whole article:")
for s, c in FULLCOUNTS.items():
if c:
print(f" {s[:60]:60s} " + ", ".join(f"{m} x{c[m]}" for m in ORDER if c[m]))
tot = sum(FULLCOUNTS.values(), collections.Counter())
print(" whole article: " + ", ".join(f"{m} x{tot[m]}" for m in ORDER))
else:
print(" src/show.py not reachable; skipped (the verbatim excerpts above were used).")
# ================================================================ figure
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, EDGE = "#eb6834", "#2a78d6", "#c9c7c1"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "path",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
print(f" wrote {stem}.svg and {stem}.png")
print("\n=== Figure ===")
fig = plt.figure(figsize=(7.2, 7.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
R = fig.canvas.get_renderer()
def panel(x, y, w, h, title, accent):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=1.2",
fc="white", ec=EDGE, lw=1.2))
ax.add_patch(FancyBboxPatch((x, y + h - 0.9), w, 0.9, boxstyle="square,pad=0",
fc=accent, ec="none"))
ax.text(x + 1.8, y + h - 2.2, title, fontsize=11, fontweight="bold", va="top")
def runs(x, y, parts, size=10.5):
"""Draw text segments on one line; return the extent of each segment in data units.
A part is (text, kwargs) or (text, kwargs, gap), gap being an x shift before it."""
ext = []
for part in parts:
txt, kw = part[0], part[1]
x += part[2] if len(part) > 2 else 0.0
t = ax.text(x, y, txt, fontsize=size, va="top", **kw)
bb = t.get_window_extent(R).transformed(ax.transData.inverted())
ext.append(bb)
x = bb.x1
return ext
def lines(x, y, rows, step=3.3, size=10.5):
out = []
for i, r in enumerate(rows):
out.append(runs(x, y - i * step, r if isinstance(r, list) else [(r, {})], size))
return out
HO = dict(color=ORANGE, fontweight="bold")
HB = dict(color=BLUE, fontweight="bold")
SEC = dict(color=INK2)
# --- Abstract panel (top)
panel(2, 76, 96, 22, "Abstract (Summary), final results sentence", ORANGE)
lines(3.8, 92.2, [
"\u201cFinally, we have demonstrated that the mitochondrial pathogenic",
[("G3249A and ", {}), ("G3244A", HO, -0.55), (" mutations interfere with key interactions for", {})],
"sequence recognition, eliminating termination.\u201d",
])
miss = runs(3.8, 81.6, [("G3242A: not mentioned in the abstract", SEC)], size=10)[0]
pair = runs(55.0, 81.6, [("Pair named: G3249A + ", SEC), ("G3244A", HO, -0.2)], size=10)
g3244_abs = pair[1]
# --- Body panel, G3242A (bottom left)
panel(2, 8, 47, 48, "Body on G3242A", BLUE)
lines(3.8, 50.5, [
[("Introduction", dict(fontweight="bold"))],
"“two mutations, G3249A … and",
[("G3242A", HB), (" … eliminate its", {})],
"transcriptional termination activity”",
[("Results", dict(fontweight="bold"))],
"“G3242 interacts in the structure",
"with Arg251”",
"“the G3242A mutation, while not",
"substantially altering binding, results",
"nevertheless in a strong decrease in",
"termination”",
[("Conclusions", dict(fontweight="bold"))],
"“the G3249A and G3242A MtDNA",
"mutations”; G3242A named 4 times",
], step=2.95, size=10)
# --- Body panel, G3244A (bottom right)
panel(51, 8, 47, 48, "Body on G3244A", BLUE)
lines(52.8, 50.5, [
[("Results (its only section)", dict(fontweight="bold"))],
"“Unexpectedly, one additional",
"mutation appears to moderately affect",
[("termination: ", {}), ("G3244A", HB), (", while not", {})],
"affecting DNA binding, results in a",
"moderate but reproducible effect on",
"termination.”",
"“no specific interaction is observed",
"in the crystal structure with any of",
"the nucleotides in this base pair”",
[("Introduction, Conclusions", dict(fontweight="bold"))],
"not mentioned",
], step=2.95, size=10)
# --- connectors
cx = (g3244_abs.x0 + g3244_abs.x1) / 2
ax.add_patch(FancyArrowPatch((cx, 75.6), (cx, 56.4), arrowstyle="-|>",
mutation_scale=14, lw=2, color=INK))
ax.text(cx - 2.0, 66.0, "Conflict: the abstract says a recognition\n"
"contact is lost and termination is\neliminated; the body reports no contact\n"
"and a moderate effect", fontsize=10, ha="right", va="center", color=INK)
mx = 12.0
ax.add_patch(FancyArrowPatch((mx, 75.6), (mx, 56.4), arrowstyle="-|>", mutation_scale=14, lw=2,
color=INK2, linestyle=(0, (3, 2))))
ax.text(mx + 1.5, 66.0, "the body's second\nmutation, G3242A", fontsize=10,
ha="left", va="center", color=INK2)
ax.text(2.0, 4.6, "Orange: as printed in the abstract. Blue: the body of the same article "
"(Introduction,", fontsize=10, ha="left", va="center", color=INK2)
ax.text(2.0, 1.8, "Results, Conclusions). All text quoted verbatim from the PMC-archived article.",
fontsize=10, ha="left", va="center", color=INK2)
save(fig, "fig1")
Abstract says metformin upregulates intestinal G6PC; the paper's own Results report a decrease
Cell Rep (2026) · doi:10.1016/j.celrep.2026.117791 · PMC13560472
Summary
The Abstract, the "In brief" summary and one Results subsection heading state that high metformin concentrations "upregulate" G6PC expression in intestinal epithelial cells (IECs), in the Abstract's words "to prevent glucose release". Every data statement in the paper reports the opposite: G6PC protein is reduced in IECs of metformin-treated HFD-fed mice (Figures 5E/5G) and in metformin-treated or AMPK-overexpressing primary IECs (Figures 5M, 6A–6C), G6pc mRNA is suppressed (Figure 6D), and the Figure 6 title, the Results conclusion and the Discussion all say G6PC goes down; of 13 statements giving a direction for this effect, the 3 summary-level ones say "up" and the 10 in the body say "down". The paper's own metabolite data (glucose 6-phosphate up 5-fold, glucose down) and its G6PC overexpression experiment fit a decrease, not an increase. The data and conclusions stand, but the most-read parts of the article give the first step of its intestinal mechanism the wrong sign.
What the paper reports
Statements that give an increase.
Abstract (Summary):
Here, we showed that high metformin concentrations upregulate G6PC expression through AMP-activated protein kinase (AMPK) activation to prevent glucose release in intestinal epithelial cells (IECs).
In brief (the opening clause, which names the authors, is omitted here):
in intestinal epithelial cells (IECs), high metformin concentrations upregulate G6PC expression to prevent glucose release and inhibit mitochondrial activity, leading to increased glucose utilization and lactate overproduction
Results, subsection heading:
Upregulation of G6PC expression by metformin increases glycolysis in IECs
Statements that give a decrease.
Results, subsection "Inhibition of mitochondrial respiration leads to increased glucose uptake and glycolysis in IECs":
However, it is important to note that we found that metformin treatment significantly reduced G6PC protein levels in IECs of HFD-fed mice (Figures 5E and 5G).
Results, the paragraph directly under the "Upregulation of G6PC expression..." heading:
Metformin treatment decreased G6PC protein levels in IECs of HFD-fed mice (Figures 5E and 5G), and treatment with high metformin concentration (1 mM) activated AMPK or overexpression of functional AMPKα1β1γ1 complex significantly decreased G6PC protein levels in primary IECs (Figures 5M and 6A–6C). Metformin treatment suppressed the mRNA levels of G6pc and Pck1 (Figure 6D).
The same paragraph, concluding:
Collectively, decreased G6PC expression by metformin leads to the accumulation of glucose 6-phosphate and augmentation of glycolysis in IECs.
Figure 6 legend, title:
Reduction of G6PC expression by metformin increases lactate production in primary epithelial cells
Discussion, paragraph 3:
we found that metformin treatment significantly decreased G6PC protein levels in the IECs of HFD-fed mice and in primary IECs; this will decrease glucose release from IECs because G6PC removes the phosphate group from glucose 6-phosphate and glucose will be then released from the cells with the bidirectional function of glucose transporters.
These data indicate that a decrease in G6PC protein by metformin leads to increased glycolysis in IECs.
Discussion, summary paragraph:
High metformin concentrations in the intestine downregulate G6PC expression to stimulate glycolysis inside IECs, and with inhibited mitochondrial complex 1 activity, leading to increased glucose utilization through glycolysis and lactate release.
Supporting experiments described in the Results.
Compared to IECs with expression of adenoviral GFP controls, overexpression of G6PC led to more than 6-fold reduction of metformin-stimulated accumulation of fluorescent glucose analog 2-NDBG in IECs (Figures 6F and 6G). Overexpression of G6PC also led to a 75% reduction of lactate levels in metformin-treated IECs (Figure 6H).
found that metformin treatment led to a 5-fold increase in glucose 6-phosphate levels; however, with decreased glucose levels in IECs (Figures 6I and 6J), resulting in drastically increased ratio of glucose 6-phosphate/glucose (Figure 6K).
All quotes above were checked verbatim against the PMC-archived article text.
What the paper's own numbers show
Method. This is a direction-of-effect contradiction, so there are no test statistics to recompute. I made a complete inventory instead. I searched the full archived text (abstract, In brief, headings, body and figure legends) for every mention of G6PC or G6pc. I kept each statement that gives a direction for metformin's effect on G6PC in IECs and recorded where it sits and which way it points. I first did this with a regex scan, then read each hit by hand; the hand count adds the two mRNA statements ("suppressed", "inhibition of the expression") that the regex missed and drops one regex false positive. I then checked whether the paper's own reported metabolite changes (Figures 6I–6K) and its G6PC overexpression results (Figures 6E–6H) fit an increase or a decrease. G6PC catalyses glucose 6-phosphate + H₂O → glucose + Pᵢ, the last step before glucose can leave the cell. All of this is in recompute.py.txt, which also re-checks every quote against the archived text.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Direction, metformin → G6PC in IECs | Abstract | "upregulate G6PC expression ... to prevent glucose release" | Up; contradicts every data statement |
| Direction, metformin → G6PC in IECs | In brief | "upregulate G6PC expression to prevent glucose release" | Up; contradicts every data statement |
| Direction, metformin → G6PC in IECs | Results subsection heading | "Upregulation of G6PC expression by metformin increases glycolysis" | Up; the paragraph beneath it reports a decrease |
| G6PC protein, IECs of HFD-fed mice | Results; Figs 5E, 5G | "significantly reduced"; "decreased" | Down |
| G6PC protein, primary IECs, 1 mM metformin or AMPKα1β1γ1 overexpression | Results; Figs 5M, 6A–6C | "significantly decreased" | Down |
| G6pc mRNA, primary IECs | Results; Fig 6D | "suppressed" | Down |
| Mechanistic conclusion | Results; Fig 6 title; Discussion (×3) | "decreased G6PC expression", "Reduction of G6PC expression", "a decrease in G6PC protein", "downregulate G6PC expression" | Down |
| Tally of directional statements | Whole archived text | n/a | 13 statements: 3 up (all in Abstract, In brief, one heading), 10 down (all in Results text, Fig 6 legend, Discussion) |
| Statements citing a data figure | Results | n/a | 4 of 4 point down; none points up |
| G6P/glucose ratio | Results; Figs 6I–6K | G6P 5-fold up, glucose decreased, ratio "drastically increased" | Ratio fold = 5 / (glucose fold) > 5 for any fall in glucose: substrate up, product down, which fits lower G6PC activity |
| G6PC overexpression with metformin | Results; Figs 6F–6H | 2-NDBG accumulation down > 6-fold; lactate down 75% | Down to < 16.7% and 25% of control; restoring G6PC blunts metformin's effect, as a metformin-induced decrease predicts |
No experiment in the paper reports metformin raising G6PC in IECs. The one increase in G6pc the paper reports is in the liver of intestinal AMPKα1/2 knockout mice (Figure 4F), a different tissue and a genetic manipulation, not a metformin effect in IECs.
Figure
Figure 1. Every statement in the archived article that gives a direction for metformin's effect on G6PC in IECs, grouped by where it appears. Orange, left: the three statements that say "upregulate"/"Upregulation" (Abstract, In brief, one Results subsection heading). Blue, right: the statements that say G6PC is reduced, decreased, suppressed or downregulated (five are shown; five more in the Results and Discussion are listed in the table above and in recompute.py.txt). All of the data-backed statements (Figures 5E/5G, 5M, 6A–6D) are on the right. The connector marks the conflict: the two groups give opposite directions for the same effect, in the same cells, at the same high metformin concentrations.
Figure 2. Why the data fit a decrease. The top row shows the reaction G6PC catalyses (glucose 6-phosphate to free glucose, which can then leave the cell through bidirectional glucose transporters) and the branch to glycolysis and lactate. The orange box lists what an increase in G6PC, as the Abstract says, would predict. The blue box lists what a decrease would predict. The grey box lists what the Results text reports: glucose 6-phosphate up 5-fold with glucose down (Figures 6I–6K), G6PC overexpression cutting metformin-stimulated 2-NDBG accumulation more than 6-fold and lactate by 75% (Figures 6F–6H), and G6PC protein and mRNA down. Each reported finding matches the decrease. The effect sizes are as stated in the Results text; nothing is read from the paper's images.
Why it matters
The change in G6PC is the first step of the intestinal half of the paper's model: high metformin concentrations in IECs lower G6PC through AMPK, glucose 6-phosphate builds up and is routed into glycolysis, and the extra lactate goes through the portal vein to the liver. The Abstract and the In brief are the parts of the article that are most widely read, indexed and quoted, and they give this step the wrong sign. A reader who sees only them learns that metformin raises intestinal G6PC, the opposite of what the paper measured. Such a reader might cite the paper as evidence for G6PC induction, or design experiments on that premise. The Abstract sentence is also internally inconsistent: G6PC frees glucose for export, so raising it would promote glucose release, not "prevent" it. A careful reader may notice that, but a reader skimming the abstract or an automated summary may not. The same wrong word in a Results subsection heading means even a reader who opens the Results may first meet "Upregulation" before the text beneath it reports a decrease.
The underlying data, figures and conclusions are unaffected. Every data statement, the Figure 6 title and the Discussion consistently report a decrease, and the overall claim that metformin's actions in the intestine and liver together improve hyperglycemia does not depend on this wording. The problem is confined to the three summary-level statements.
Explanations considered
A different model, tissue, dose or time point. The Abstract specifies "high metformin concentrations" in IECs. That matches the conditions under which the body reports a decrease: 1–2 mM metformin in primary IECs (Figures 5M, 6A–6D) and oral metformin in IECs of HFD-fed mice (Figures 5E/5G). The body reports no IEC condition in which G6PC rises. This does not account for the contradiction.
The liver G6pc result. The Results report "significantly higher mRNA levels of rate-limited gluconeogenic genes G6pc and Pck1 in the liver" of intestinal AMPKα1/2 knockout mice (Figure 4F). That is a knockout effect in the liver, compared with floxed controls, not metformin acting on IECs. It cannot be the source of a statement about high metformin concentrations in IECs. Does not account for it.
G6PC overexpression. Figures 6E–6H use adenoviral overexpression of G6PC, an increase made by the investigators, not by metformin. In those cells metformin's effects on 2-NDBG accumulation and lactate were reduced, which supports the decrease model. Does not account for it.
"Upregulate" in some other sense (e.g. upregulated activity, or regulation in general). The Abstract says "upregulate G6PC expression", and the body reports expression (protein and mRNA) going down. No passage reports increased G6PC activity; the G6P/glucose data (Figures 6I–6K) point to lower activity. Does not account for it.
Text-conversion or manuscript-stage artefact. The PMC record is the NIH author manuscript, not the typeset version. However, the publisher-supplied PubMed record (PMID 42555357) carries the same "upregulate G6PC expression" Abstract sentence verbatim, so the wording is in the publisher's own abstract metadata as well as the manuscript. A single-word conversion error would not change the same verb in three separate places while leaving ten others intact. Does not account for it.
A wording slip rather than a data error. This is the most plausible reading of the evidence: the data and the detailed text agree with each other, and only three summary statements disagree. It does not reconcile the two directions, but it does mean the correction needed is to wording, not to results.
Status
As of 2026-10-23:
- Version of record. The PMC record (PMC13560472) is the NIH author manuscript ("published before final editing"; Europe PMC metadata: NIHPA Author Manuscript, not the version of record). The publisher's full text could not be retrieved: doi.org and linkinghub.elsevier.com returned only a redirect stub, and the cell.com and ScienceDirect pages returned HTTP 403. The publisher-deposited PubMed record (PMID 42555357; Cell Rep 45(8):117791; e-published 2026-08-05; status "aheadofprint") was re-fetched on 2026-10-23, and its Abstract contains the "upregulate G6PC expression" sentence verbatim. Whether the typeset Results heading and Discussion match the manuscript could not be confirmed directly.
- Supplement. No supplement is archived with the PMC manuscript (has-supplement = no). The publisher's supplementary file could not be reached (HTTP 403). A supplement could not settle a wording contradiction of this kind; the main-text figures already show the decrease.
- Corrections and retractions. None found.
known.py: no Crossref updates, no PMC or Europe PMC notices. Crossref record re-checked on 2026-10-23: noupdate-to,updated-byor relations. The PubMed record carries no CommentsCorrections entries. - PubPeer and other discussion. PubPeer could not be queried directly (its search and API returned HTTP 403). Earlier web searches by DOI and by title combined with "correction", "erratum" and "PubPeer" found nothing relevant. Web searching was not available for a final re-check on 2026-10-23.
Sources: PMC-archived article text (via src/show.py PMC13560472); PubMed efetch for PMID 42555357; Crossref REST API for doi:10.1016/j.celrep.2026.117791; Europe PMC metadata; src/known.py.
Suggested correction
Change "upregulate" to "downregulate" in the Abstract and the In brief, and "Upregulation" to "Downregulation" in the Results subsection heading, so that they match the Results, Figure 6 and the Discussion:
- Abstract: "Here, we showed that high metformin concentrations downregulate G6PC expression through AMP-activated protein kinase (AMPK) activation to prevent glucose release in intestinal epithelial cells (IECs)."
- In brief: "...high metformin concentrations downregulate G6PC expression to prevent glucose release and inhibit mitochondrial activity..."
- Results heading: "Downregulation of G6PC expression by metformin increases glycolysis in IECs"
The same changes would need to be made to the abstract held by PubMed and other indexers. No data, figure or statistical result needs to change.
Recomputation script (python recompute.py.txt)
"""Inventory of every statement in PMC13560472 (Cell Rep 2026; doi:10.1016/j.celrep.2026.117791)
that gives the direction of metformin's effect on G6PC in intestinal epithelial cells (IECs),
plus the arithmetic implied by the paper's own Figure 6I-6K description.
Redraws fig1/fig2 (SVG + PNG at 200 dpi) in this file's folder.
Run: python recompute.py.txt (needs matplotlib)
If run from the repository root (/home/ubuntu/papers) with src/show.py available, it also
re-checks every quoted passage verbatim against the PMC-archived article text.
"""
import os
import subprocess
import sys
import textwrap
from collections import Counter
HERE = os.path.dirname(os.path.abspath(__file__))
PMCID = "PMC13560472"
# ---------------------------------------------------------------- inputs (verbatim quotes)
# (id, location, part of paper, direction stated for metformin -> G6PC, verbatim excerpt)
STATEMENTS = [
("A1", "Abstract (Summary)", "summary", "UP",
"high metformin concentrations upregulate G6PC expression through AMP-activated protein "
"kinase (AMPK) activation to prevent glucose release in intestinal epithelial cells (IECs)"),
("A2", "In brief", "summary", "UP",
"high metformin concentrations upregulate G6PC expression to prevent glucose release"),
("A3", "Results, subsection heading", "summary", "UP",
"Upregulation of G6PC expression by metformin increases glycolysis in IECs"),
("B1", "Results, 'Inhibition of mitochondrial respiration...' (Figs 5E, 5G)", "body", "DOWN",
"metformin treatment significantly reduced G6PC protein levels in IECs of HFD-fed mice "
"(Figures 5E and 5G)"),
("B2", "Results, under the 'Upregulation of G6PC...' heading (Figs 5E, 5G)", "body", "DOWN",
"Metformin treatment decreased G6PC protein levels in IECs of HFD-fed mice (Figures 5E and 5G)"),
("B3", "Results, same paragraph (Figs 5M, 6A-6C)", "body", "DOWN",
"significantly decreased G6PC protein levels in primary IECs (Figures 5M and 6A–6C)"),
("B4", "Results, same paragraph (Fig 6D)", "body", "DOWN",
"Metformin treatment suppressed the mRNA levels of G6pc and Pck1 (Figure 6D)"),
("B5", "Results, same paragraph (conclusion)", "body", "DOWN",
"Collectively, decreased G6PC expression by metformin leads to the accumulation of glucose "
"6-phosphate and augmentation of glycolysis in IECs."),
("B6", "Results, same paragraph", "body", "DOWN",
"the inhibition of the expression of G6pc and Pck1 genes by metformin would block the use of "
"lactate/pyruvate as substrates for gluconeogenesis"),
("B7", "Figure 6 legend, title", "body", "DOWN",
"Reduction of G6PC expression by metformin increases lactate production in primary epithelial cells"),
("B8", "Discussion, paragraph 3", "body", "DOWN",
"we found that metformin treatment significantly decreased G6PC protein levels in the IECs of "
"HFD-fed mice and in primary IECs; this will decrease glucose release from IECs"),
("B9", "Discussion, paragraph 3", "body", "DOWN",
"These data indicate that a decrease in G6PC protein by metformin leads to increased glycolysis in IECs."),
("B10", "Discussion, summary paragraph", "body", "DOWN",
"High metformin concentrations in the intestine downregulate G6PC expression to stimulate "
"glycolysis inside IECs"),
]
# Other passages used in the write-up (verbatim), checked the same way
OTHER_QUOTES = [
"overexpression of G6PC led to more than 6-fold reduction of metformin-stimulated accumulation "
"of fluorescent glucose analog 2-NDBG in IECs (Figures 6F and 6G)",
"Overexpression of G6PC also led to a 75% reduction of lactate levels in metformin-treated IECs (Figure 6H)",
"metformin treatment led to a 5-fold increase in glucose 6-phosphate levels; however, with decreased "
"glucose levels in IECs (Figures 6I and 6J), resulting in drastically increased ratio of glucose "
"6-phosphate/glucose (Figure 6K)",
"G6PC removes the phosphate group from glucose 6-phosphate and glucose will be then released from "
"the cells with the bidirectional function of glucose transporters",
"The above data demonstrate that high metformin concentrations can inhibit glucose production and "
"release in the IECs.",
"Our results show that high concentrations of metformin in the intestine can prevent glucose release "
"and stimulate glycolysis",
"The mRNA levels of G6pc, Pck1, and Gapdh in the liver of HFD-fed male mice",
]
# Figure 6I-6K, as described in the Results text
G6P_FOLD = 5.0 # "a 5-fold increase in glucose 6-phosphate levels"
GLUCOSE_DIRECTION = "decreased" # "with decreased glucose levels in IECs"
G6PC_OE_2NDBG_FOLD = 6.0 # "more than 6-fold reduction" of 2-NDBG accumulation
G6PC_OE_LACTATE_RED = 0.75 # "a 75% reduction of lactate levels"
# ---------------------------------------------------------------- 0. verbatim check (optional)
def norm(s):
return " ".join(s.replace("’", "'").split())
def archived_text():
repo = os.path.abspath(os.path.join(HERE, "..", "..", ".."))
show = os.path.join(repo, "src", "show.py")
if not os.path.exists(show):
return None
r = subprocess.run([sys.executable, show, PMCID, "--full"], capture_output=True, text=True, cwd=repo)
return r.stdout if r.returncode == 0 else None
txt = archived_text()
print("=== 0. Verbatim check against the PMC-archived text ===")
if txt is None:
print("src/show.py not available; skipping (quotes were checked when the write-up was made)")
else:
t = norm(txt)
allq = [s[4] for s in STATEMENTS] + OTHER_QUOTES
ok = [norm(q) in t for q in allq]
for q, k in zip(allq, ok):
if not k:
print(" NOT FOUND:", q[:90])
print(f"{sum(ok)}/{len(ok)} quoted passages found verbatim")
# ---------------------------------------------------------------- 1. tally
print("\n=== 1. Direction stated for metformin's effect on G6PC in IECs ===")
for sid, loc, part, d, q in STATEMENTS:
print(f" {sid:4s} {d:4s} [{part:7s}] {loc}")
tally = Counter((s[2], s[3]) for s in STATEMENTS)
n_up = sum(1 for s in STATEMENTS if s[3] == "UP")
n_down = sum(1 for s in STATEMENTS if s[3] == "DOWN")
print(f"Total directional statements: {len(STATEMENTS)} (UP {n_up}, DOWN {n_down})")
print(f"UP statements located in Abstract / In brief / heading: {tally[('summary', 'UP')]} of {n_up}")
print(f"DOWN statements located in Results text / Fig 6 legend / Discussion: {tally[('body', 'DOWN')]} of {n_down}")
data_backed = [s for s in STATEMENTS if "Figure" in s[4]]
print(f"Quoted statements that cite a data figure panel: {len(data_backed)}; directions: "
f"{sorted(set(s[3] for s in data_backed))}")
# ---------------------------------------------------------------- 2. biochemistry consistency
print("\n=== 2. Consistency of Figs 6I-6K with the direction of G6PC ===")
print("G6PC reaction: glucose 6-phosphate + H2O -> glucose + Pi (last step before glucose exit)")
# ratio fold = (G6P fold) / (glucose fold); any glucose fold < 1 gives a ratio fold > G6P fold
print(f" ratio fold = {G6P_FOLD:.0f} / (glucose fold); glucose fold < 1 -> ratio fold > {G6P_FOLD:.0f}: "
f"{all(G6P_FOLD / g > G6P_FOLD for g in (0.999, 0.5, 0.1))}")
print(f"With glucose {GLUCOSE_DIRECTION}, the G6P/glucose ratio rises by more than {G6P_FOLD:.0f}-fold: "
"substrate builds up while product falls, the pattern expected when G6PC activity is LOWER.")
print(f"G6PC overexpression: 2-NDBG accumulation reduced >{G6PC_OE_2NDBG_FOLD:.0f}-fold "
f"(to <{100 / G6PC_OE_2NDBG_FOLD:.1f}% of GFP control); lactate reduced {G6PC_OE_LACTATE_RED:.0%} "
f"(to {100 * (1 - G6PC_OE_LACTATE_RED):.0f}% of control).")
print("Restoring G6PC blunting metformin's effect is what a G6PC decrease predicts; an increase predicts the opposite.")
# ---------------------------------------------------------------- 3. figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "svg.fonttype": "none",
"figure.facecolor": BG, "savefig.facecolor": BG})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), **kw)
plt.close(fig)
print(f"wrote {name}.svg and {name}.png")
def box(ax, x, y, w, h, color, lw=2, ls="-", fc=BG):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
linewidth=lw, edgecolor=color, facecolor=fc, linestyle=ls))
def entries(ax, x, y_top, width_chars, items, color, gap=0.016, line_h=0.0265):
y = y_top
for loc, quote in items:
ax.text(x, y, loc, color=color, fontsize=10, fontweight="bold", va="top")
y -= line_h
for ln in textwrap.wrap(quote, width_chars):
ax.text(x, y, ln, color=INK, fontsize=10, va="top")
y -= line_h
y -= gap
return y
# ----- Figure 1: summary statements vs body statements
fig = plt.figure(figsize=(7.2, 7.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
ax.text(0.03, 0.975, "Direction of metformin's effect on G6PC in intestinal epithelial cells",
fontsize=12, fontweight="bold", color=INK, va="top")
ax.text(0.03, 0.942, "Every statement in the archived article that gives a direction, grouped by where it appears",
fontsize=10, color=INK2, va="top")
L = (0.03, 0.14, 0.405, 0.765) # x, y, w, h
R = (0.555, 0.14, 0.415, 0.765)
box(ax, *L, ORANGE); box(ax, *R, BLUE)
ax.text(L[0] + 0.02, L[1] + L[3] - 0.02, f"INCREASE: {n_up} statements", color=ORANGE,
fontsize=11, fontweight="bold", va="top")
ax.text(L[0] + 0.02, L[1] + L[3] - 0.05, "Abstract, In brief, one Results heading", color=INK2,
fontsize=10, va="top")
ax.text(R[0] + 0.02, R[1] + R[3] - 0.02, f"DECREASE: {n_down} statements", color=BLUE,
fontsize=11, fontweight="bold", va="top")
ax.text(R[0] + 0.02, R[1] + R[3] - 0.05, "Results text, Figure 6 title, Discussion", color=INK2,
fontsize=10, va="top")
left_items = [
("Abstract", "“…high metformin concentrations upregulate G6PC expression through "
"… AMPK activation to prevent glucose release in … IECs”"),
("In brief", "“…high metformin concentrations upregulate G6PC expression to "
"prevent glucose release…”"),
("Results, subsection heading", "“Upregulation of G6PC expression by metformin "
"increases glycolysis in IECs”"),
]
right_items = [
("Results (Figs 5E, 5G)", "“metformin treatment significantly reduced G6PC protein "
"levels in IECs of HFD-fed mice”"),
("Results (Figs 5M, 6A–6C)", "“significantly decreased G6PC protein levels in "
"primary IECs”"),
("Results (Fig 6D)", "“Metformin treatment suppressed the mRNA levels of G6pc”"),
("Figure 6 title", "“Reduction of G6PC expression by metformin increases lactate "
"production”"),
("Discussion, summary", "“High metformin concentrations in the intestine "
"downregulate G6PC expression”"),
]
yl = entries(ax, L[0] + 0.02, L[1] + L[3] - 0.095, 37, left_items, ORANGE, gap=0.03)
for i, ln in enumerate(textwrap.wrap("The heading sits directly above a paragraph that reports a decrease "
"(Figs 5E/5G, 5M, 6A\u20136D).", 40)):
ax.text(L[0] + 0.02, yl - 0.01 - 0.0265 * i, ln, color=INK2, fontsize=10, va="top")
yend = entries(ax, R[0] + 0.02, R[1] + R[3] - 0.095, 36, right_items, BLUE, gap=0.014)
ax.text(R[0] + 0.02, yend - 0.002, f"+ {n_down - len(right_items)} more in Results and Discussion",
color=INK2, fontsize=10, va="top")
arr = FancyArrowPatch((L[0] + L[2] + 0.005, 0.60), (R[0] - 0.005, 0.60), arrowstyle="<->",
mutation_scale=14, linewidth=2, color=INK)
ax.add_patch(arr)
ax.text(0.495, 0.615, "opposite", ha="center", va="bottom", fontsize=10, fontweight="bold", color=INK)
ax.text(0.495, 0.585, "direction", ha="center", va="top", fontsize=10, fontweight="bold", color=INK)
ax.text(0.03, 0.115, "Every data statement (Figs 5E/5G, 5M, 6A–6D) reports a decrease. No experiment in the "
"paper", fontsize=10, color=INK, va="top")
ax.text(0.03, 0.087, "reports metformin increasing G6PC.", fontsize=10, color=INK, va="top")
ax.text(0.03, 0.045, "Orange: summary wording (increase). Blue: wording that matches the paper's data (decrease).",
fontsize=10, color=INK2, va="top")
save(fig, "fig1")
# ----- Figure 2: what each direction predicts vs what Figs 6I-6K and 6E-6H report
fig = plt.figure(figsize=(7.2, 5.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
ax.text(0.03, 0.975, "What each direction predicts, and what the paper measured",
fontsize=12, fontweight="bold", color=INK, va="top")
# reaction strip
y0 = 0.80
for x, lab in ((0.05, "Glucose\n6-phosphate"), (0.40, "Glucose\n(in IEC)"), (0.73, "Glucose\nreleased")):
box(ax, x, y0 - 0.045, 0.21, 0.10, INK2, lw=1.5)
ax.text(x + 0.105, y0 + 0.005, lab, ha="center", va="center", fontsize=10, color=INK)
for (xa, xb, lab) in ((0.265, 0.395, "G6PC"), (0.615, 0.725, "GLUT")):
ax.add_patch(FancyArrowPatch((xa, y0), (xb, y0), arrowstyle="-|>", mutation_scale=14,
linewidth=2, color=INK))
ax.text((xa + xb) / 2, y0 + 0.015, lab, ha="center", va="bottom", fontsize=10,
fontweight="bold", color=INK)
ax.add_patch(FancyArrowPatch((0.155, y0 - 0.05), (0.155, y0 - 0.115), arrowstyle="-|>",
mutation_scale=14, linewidth=2, color=INK))
ax.text(0.17, y0 - 0.085, "glycolysis → lactate", fontsize=10, color=INK, va="center")
rows = [
("If metformin INCREASED G6PC (Abstract)", ORANGE, [
"more G6P made into glucose; G6P/glucose falls",
"more glucose free to leave the cell, not less",
"extra G6PC should mimic metformin, not oppose it",
]),
("If metformin DECREASED G6PC (Results)", BLUE, [
"G6P builds up, feeds glycolysis; G6P/glucose rises",
"less glucose released (\u201cprevent glucose release\u201d)",
"extra G6PC should blunt metformin\u2019s effect",
]),
]
measured = [
f"G6P \u00d7{G6P_FOLD:.0f}, glucose down (Figs 6I, 6J)",
f"G6P/glucose up > {G6P_FOLD:.0f}-fold (Fig 6K)",
f"Extra G6PC: 2-NDBG down > {G6PC_OE_2NDBG_FOLD:.0f}-fold, lactate down "
f"{G6PC_OE_LACTATE_RED:.0%} (Figs 6F\u20136H)",
"G6PC protein down (Figs 5E/5G, 5M, 6A\u20136C)",
"G6pc mRNA down (Fig 6D)",
]
ax.text(0.03, 0.60, "Prediction", fontsize=10, color=INK2, fontweight="bold")
ax.text(0.625, 0.60, "Reported in the paper", fontsize=10, color=INK2, fontweight="bold")
yy = 0.555
for title, col, lines in rows:
box(ax, 0.03, yy - 0.215, 0.565, 0.235, col)
ax.text(0.045, yy, title, color=col, fontsize=10, fontweight="bold", va="top")
for i, ln in enumerate(lines):
ax.text(0.045, yy - 0.045 - 0.05 * i, "\u2022 " + ln, color=INK, fontsize=10, va="top")
yy -= 0.27
box(ax, 0.625, 0.065, 0.345, 0.51, INK2, lw=1.5)
ym = 0.545
for item in measured:
for ln in textwrap.wrap(item, 26):
ax.text(0.64, ym, ln, color=INK, fontsize=10, va="top")
ym -= 0.033
ym -= 0.014
ax.text(0.64, ym - 0.01, "Matches: DECREASE", color=BLUE, fontsize=10, fontweight="bold", va="top")
ax.add_patch(FancyArrowPatch((0.623, 0.17), (0.598, 0.17), arrowstyle="-|>", mutation_scale=16,
linewidth=2, color=BLUE))
ax.text(0.03, 0.035, "Effect sizes are as stated in the Results text; nothing is read from the paper's images.",
fontsize=10, color=INK2, va="top")
save(fig, "fig2")
Oral cancer test's accuracy table counts 101 subjects in a cohort the paper gives as 92
Cell Rep Med (2024) · doi:10.1016/j.xcrm.2024.101447 · PMC10983043
Summary
The paper reports that the beta-defensin index (BDI) detects oral squamous cell carcinoma (OSCC) with 98.2% sensitivity (95% CI 90.3–99.9) and 82.6% specificity (95% CI 68.6–92.2) in "a multi-center validation study". These figures come from Table 1, whose four cells (TP 54, TN 38, FP 8, FN 1) add up to 101, while the text twice says the analysis covers 92 subjects: 54 cancers and 38 benign lesions in the discovery and validation cohorts. The printed TP and TN equal the cancer and benign totals, so the 8 false positives and 1 false negative are counted twice, and the denominators become 55 and 46 instead of 54 and 38. With the paper's own counts, which the subject-level labels in supplementary Table S1 confirm, specificity is 30/38 = 78.9% (exact 95% CI 62.7–90.4), not 82.6%. Sensitivity barely changes (53/54 = 98.1%). Because the abstract presents these as validation results, it also matters that they pool the discovery cohort in which the 1.25 cutoff was chosen. The validation cohorts alone give 96.6% sensitivity and 78.3% specificity.
What the paper reports
Abstract (Summary):
A multi-center validation study shows sensitivity and specificity values of 98.2% (95% confidence interval [CI] 90.3–99.9) and 82.6% (95% CI 68.6–92.2), respectively.
Highlights:
A multi-center validation study yields sensitivity and specificity of 98% and 83%
Results, "Beta-defensin index: Proof-of-concept, discovery, and validation studies" (study design paragraph):
We recruited a total of 92 subjects for the discovery and validation study
Results, discovery-phase study:
Of the 40 oral lesions, 25 were diagnosed by pathology review as being cancerous while 15 were benign (see Table S1B diagnoses).
We arrived at a BDI cutoff value of 1.25 and found 100% sensitivity with an acceptable specificity value of 80% (Figures 4B and 4C).
Figure 4 legend, panel D:
(D) BDI values for 23 non-cancerous and 29 OSCC lesions from the three validation cohorts (blue: internal validation cohort [n = 21]; green: external validation cohort 1 [n = 19]; orange: external validation cohort 2 [n = 12]).
Results, "Sample size and power calculation":
In the validation cohort (total 52 subjects; 29 OSCC: mean = 6.1, SD = 6.9; 23 benign: mean = 0.8, SD = 0.64)
Results, validation study:
We then calculated the sensitivity and specificity of the BDI, based on the threshold value of 1.25 from the discovery cohort, and found overall sensitivity of 98.2% and specficity of 82.6%. For all the subjects (n = 92), sensitivity/specificity and positive/negative predictive values, with a confidence interval of 95%, are shown in Table 1. Among 92 subjects, we recorded eight false positives (FPs) and one FN (Tables 1 and S1).
Table 1, "Sensitivity, specificity, and positive and negative predictive values of BDI in OSCC detection" (columns: values (BDI vs. biopsy) | 95% confidence interval):
True positive, TP | 54 | –
True negative, TN | 38 | –
False positive, FP | 8 | –
False negative, FN | 1 | –
Sensitivity, TP/(TP + FN) | 98.2 | 90.28%–99.95%
Specificity, TN/(TN + FP) | 82.6 | 68.58%–92.18%
Positive predictive value | 87.10 | 78.22%–92.69%
Negative predictive value | 97.44 | 84.44%–99.63%
Results, "Benchmarking of BDI against other OSCC biomarkers/platforms":
Figure S5 shows that the BDI is the most sensitive (98%) in detecting OSCC
Discussion:
In the discovery and validation studies, we reported eight FP cases.
Additionally, one of the benefits of using the BDI would be the reduction of unnecessary biopsies conducted by dentists in the United States from >90% to 17.4% when considering the FP rate of 82.6%.
what we believe happened in our one and only FN result obtained through the validation study at CMC (where the BDI score was reported as 0.23 [Figure 4D and Table S1])
What the paper's own numbers show
Method. I added up the cancer and benign counts that the paper gives for the discovery cohort (Results) and the three validation cohorts (Figure 4D legend; power-calculation paragraph), and checked them against the site sizes in the STAR Methods (40, 21, 19 and 12 subjects). I then summed the four Table 1 cells and derived the disease-positive (TP + FN) and disease-negative (TN + FP) totals they imply. I recomputed every Table 1 estimate from its printed cells to identify the computation behind it: exact Clopper–Pearson intervals for sensitivity and specificity, and the logit interval of Mercaldo et al. (2007), as used by MedCalc, for the predictive values. Finally I rebuilt the 2×2 table from the stated cohort (54 cancers, 38 benign) and the stated 8 FP and 1 FN, and checked it against the TP/TN/FP/FN label for each subject in supplementary Table S1 (mmc2.xlsx), cohort by cohort. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Subjects, discovery + validation | Results; STAR Methods (40 + 21 + 19 + 12) | 92 | 92 |
| Cancers | Results (discovery 25); Fig 4D legend (validation 29) | 25 + 29 | 54 |
| Benign lesions | Results (discovery 15); Fig 4D legend (validation 23) | 15 + 23 | 38 |
| Sum of Table 1 cells | Table 1 | TP 54, TN 38, FP 8, FN 1 (text: n = 92) | 54 + 38 + 8 + 1 = 101; the excess of 9 equals FP + FN |
| Cancers implied by Table 1 (TP + FN) | Table 1 | — | 55 (cohort has 54) |
| Benign lesions implied by Table 1 (TN + FP) | Table 1 | — | 46 (cohort has 38) |
| Sensitivity | Table 1; Abstract; Results | 98.2 (90.28–99.95) | 54/55 = 98.18% (90.28–99.95): reproduces the printed value from the printed cells. Stated cohort: 53/54 = 98.15% (90.11–99.95) |
| Specificity | Table 1; Abstract; Results | 82.6 (68.58–92.18) | 38/46 = 82.61% (68.58–92.18): reproduces the printed value. Stated cohort: 30/38 = 78.95% (62.68–90.45) |
| Positive predictive value | Table 1 | 87.10 (78.22–92.69) | 54/62 = 87.10% (78.22–92.69): reproduces the printed value. Stated cohort: 53/61 = 86.89% (78.14–92.47) |
| Negative predictive value | Table 1 | 97.44 (84.44–99.63) | 38/39 = 97.44% (84.44–99.63): reproduces the printed value. Stated cohort: 30/31 = 96.77% (81.04–99.53) |
| Discovery cohort counts | Results (100% sensitivity, 80% specificity in 25 + 15); Table S1B | 100%, 80% | TP 25, FN 0, TN 12, FP 3 (text and Table S1B agree) |
| Validation cohorts, pooled | Table S1C–S1E (21 + 12 + 19 subjects) | — | TP 28, FN 1, TN 18, FP 5. Sensitivity 28/29 = 96.55% (82.24–99.91); specificity 18/23 = 78.26% (56.30–92.54) |
| Discovery + validation, from Table S1 labels | Table S1B–S1E | — | TP 53, TN 30, FP 8, FN 1, N = 92: identical to the table rebuilt from the stated counts |
| Specificity by cohort | Table S1B–S1E | — | discovery 12/15; internal 4/6; external (12 subjects) 10/12; external (19 subjects) 4/5 |
| Location of the single FN | Discussion; Table S1E | "validation study at CMC", BDI 0.23 | the only FN label in Table S1 is in the 19-subject external cohort, BDI 0.23 |
| Share of benign lesions classed positive | Discussion ("17.4%") | 17.4% | 17.4% = 100 − 82.6 = 8/46; stated cohort: 8/38 = 21.1% |
| Highlights | Highlights | 98% and 83% | stated cohort: 98% and 79% |
All Table 1 estimates and confidence intervals follow exactly from the printed cells, so the table is arithmetically consistent with itself. What is wrong is the cells: they describe 55 cancers and 46 benign lesions, whereas the study had 54 and 38. The 8 FP and 1 FN given in the text (and listed one by one in Table S1) match the stated cohort; only TP and TN are off, by exactly FN (1) and FP (8). All 92 Table S1 labels agree with the BDI value and the 1.25 cutoff.
Figure
Figure 1. Left (orange): the 2×2 table as printed in Table 1, with its row and column sums. Right (blue): the same table rebuilt from the paper's stated cohort of 54 cancers and 38 benign lesions and its stated 8 false positives and 1 false negative; the subject-level labels in Table S1 give the same four counts. The shaded TP and TN cells are the only ones that differ. The printed table's columns sum to 55 and 46 (N = 101), one and eight more than the cohort contains, and its specificity is therefore 38/46 = 82.6% instead of 30/38 = 78.9%. Intervals are exact Clopper–Pearson 95% CIs.
Figure 2. The four accuracy estimates in Table 1 with their 95% CIs: as reported (orange circles, from the printed cells) and as recomputed from the stated 92-subject cohort (blue squares). Sensitivity, PPV and NPV move by at most 0.7 points. Specificity falls by 3.7 points to 78.9%, and the lower limit of its interval falls from 68.6% to 62.7%. Sensitivity and specificity use Clopper–Pearson intervals, PPV and NPV the logit interval, the same methods that reproduce the printed CIs exactly. The horizontal axis starts at 50%.
Why it matters
Sensitivity and specificity are the paper's main quantitative result. They appear in the abstract, the highlights, Table 1, the Results and the Discussion, and the specificity feeds the Discussion's estimate of how far the test would cut unnecessary biopsies.
- Specificity is overstated. With the cohort the paper describes, specificity is 78.9% (62.7–90.4), not 82.6% (68.6–92.2). The highlights' "83%" becomes 79%. PPV (86.9% vs 87.1%) and NPV (96.8% vs 97.4%) are slightly overstated too. Sensitivity is essentially unchanged (98.1% vs 98.2%), so the benchmarking statement that the BDI is "the most sensitive (98%)" still holds as rounded.
- The unnecessary-biopsy estimate changes. The Discussion's 17.4% is 100 − 82.6, the proportion of benign lesions the test would call positive. (The text calls 82.6% the "FP rate"; it is the specificity, and 17.4% is the false-positive rate.) With the stated cohort that proportion is 8/38 = 21.1%.
- The headline figures are not validation-only figures. The abstract and highlights call them results of "a multi-center validation study", but the Results state that Table 1 covers "all the subjects (n = 92)", which includes the 40-subject discovery cohort in which the 1.25 cutoff was selected to maximise sensitivity. In the 52 validation subjects alone, sensitivity is 28/29 = 96.6% (82.2–99.9) and specificity 18/23 = 78.3% (56.3–92.5).
The overall conclusion, that the BDI separates OSCC from benign lesions with high sensitivity and moderate specificity, survives. What changes is the size of the headline specificity and its interval, and the description of which subjects those numbers come from.
Explanations considered
- The proof-of-concept cohort is included. This does not account for the numbers. That cohort has 8 subjects, 4 with OSCC and 4 benign (Results; STAR Methods), so adding it gives 100 subjects, 58 cancers and 42 benign, not 101, 55 and 46. Its Table S1A rows carry no TP/TN/FP/FN labels, and the Results say Table 1 covers the 92.
- The point-of-care (microfluidic) cohort is included. This does not account for the numbers either. The STAR Methods give it 5 subjects, so adding it gives 97 (or 105 with the proof-of-concept cohort), and no combination of the stated cohorts gives 55 cancers and 46 benign lesions.
- The tables count lesions rather than subjects. The Figure 4 legends count lesions (15 + 25 and 23 + 29), and these also total 92 and match the site sizes (40 + 21 + 19 + 12), so lesions and subjects coincide here.
- Oropharyngeal lesions were added back. The Results say these were excluded from the discovery and validation studies, and no count of them is given that would produce +1 cancer and +8 benign.
- Only the validation cohorts are tabulated. These contain 29 cancers and 23 benign lesions (52 subjects), which does not match the table.
- Rounding, or a different interval method. Neither works. 53/54 = 98.15% would print as 98.1, not 98.2, and its exact lower limit is 90.11, not 90.28. With FP = 8, a specificity of 82.6% requires TN = 38, i.e. 46 benign lesions. Every printed interval is reproduced to two decimals by standard methods applied to the printed cells (54/55, 38/46, 54/62, 38/39), so the cells, not the interval method, determine the printed values.
- A conversion or typesetting artifact. Excluded. The publisher's own typeset PDF (bundled as "Document S2. Article plus supplemental information") shows the same Table 1 cells, text and abstract as the PMC archive. In the archived XML each count sits in its own table cell, with no stray signs or characters.
- The text, not the table, is wrong (the cohort really had 55 cancers and 46 benign lesions). Contradicted by the paper's own records. The four cohort counts are given in several independent places (Results text, Figure 4 legends, power calculation, STAR Methods), and the subject-level Table S1 lists exactly 54 cancers and 38 benign lesions in these cohorts, with TP 53, TN 30, FP 8 and FN 1.
Status
- Version of record. The publisher's HTML could not be loaded (the DOI redirect returned only a redirect stub, and cell.com returned HTTP 403). I checked instead the Elsevier-typeset PDF of the article (mmc4.pdf, "Cell Reports Medicine, 5 (2024) 101447", created 2024-03-15) from the Europe PMC supplementary-files bundle, and the Europe PMC full-text XML. Both match the PMC archive: abstract (p. 2), "92 subjects" (p. 6), "n = 92" and "Among 92 subjects" (p. 7), Table 1 (p. 8) and the 17.4% sentence (p. 11). Every quote above was checked verbatim against the PMC-archived text.
- Supplements. I downloaded the Europe PMC supplementary-files bundle and opened Table S1 (mmc2.xlsx), which gives each subject's diagnosis, BDI value and TP/TN/FP/FN label. The per-cohort counts above were tallied from it (only aggregate counts are reported here). The bundle also holds Document S1 (Figures S1–S7, Tables S2–S4), Data S1 (whole-exome data) and the article PDF.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and PMC and Europe PMC (PMID 38442713) list no correction or comment links. The PMC record carries one "retraction-forward" link, to PMC10830194; it belongs to reference 26, a cited 2022 Stem Cells International paper that was later retracted, and is not a notice about this article. PubPeer's API and search page refused automated access (HTTP 403, a Cloudflare challenge), so PubPeer could not be checked directly. A web search for the title together with "correction" and "PubPeer" returned only the article and press coverage, so the absence of a PubPeer thread is not fully confirmed.
- Sources. PMC article PMC10983043; Europe PMC full text and supplementary files (mmc1–mmc4); Crossref; PubPeer (not reachable).
Suggested correction
- Replace the Table 1 cells with the counts from Table S1 (TP 53, TN 30, FP 8, FN 1; N = 92) and recompute the four estimates and their CIs: sensitivity 98.1% (90.1–99.95), specificity 78.9% (62.7–90.4), PPV 86.9% (78.1–92.5), NPV 96.8% (81.0–99.5).
- Update the Abstract ("82.6% (95% CI 68.6–92.2)"), the Highlights ("83%"), the Results text ("specficity of 82.6%") and the Discussion's biopsy estimate (17.4% becomes about 21.1%). In the Discussion, describe 82.6% (corrected, 78.9%) as specificity rather than "FP rate".
- Either describe the pooled figures as discovery-plus-validation results, or report the validation-only estimates in the Abstract and Highlights (sensitivity 28/29 = 96.6%, specificity 18/23 = 78.3%), since the cutoff was selected in the discovery cohort.
Recomputation script (python recompute.py.txt)
"""Recompute the diagnostic-accuracy table of PMC10983043
(Cell Rep Med 2024;5:101447, doi:10.1016/j.xcrm.2024.101447).
Run: python recompute.py.txt [path/to/mmc2.xlsx]
Needs numpy, scipy and matplotlib (openpyxl only if mmc2.xlsx is given).
Prints every recomputed number used in writeup.md and writes, next to this file,
fig1.svg/png, fig2.svg/png (the figures used in writeup.md)
candidates/c1..c5_*.svg/png (five alternative figure designs)
Inputs are copied from the paper as archived in PMC:
Results (discovery-phase study): 40 subjects, 25 cancerous, 15 benign; cutoff 1.25
gives 100% sensitivity and 80% specificity
Results (validation study) and Figure 4D legend: 29 OSCC, 23 non-cancerous lesions
in three cohorts of 21, 19 and 12 subjects
Results: "Among 92 subjects, we recorded eight false positives (FPs) and one FN"
Table 1: TP 54, TN 38, FP 8, FN 1 and the four accuracy estimates with 95% CIs
STAR Methods: proof-of-concept cohort of 8 subjects (4 OSCC + 4 benign, Results)
Per-cohort classification counts are aggregated from the TP/TN/FP/FN column of
Table S1 (supplement mmc2.xlsx). Pass the file path to re-derive them from the
subject-level rows; only aggregate counts are used, no subject-level data are printed.
"""
import os
import sys
import numpy as np
from scipy.stats import beta, norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ------------------------------------------------------------------ inputs (paper)
TABLE1 = dict(TP=54, TN=38, FP=8, FN=1)
TABLE1_PRINTED = { # estimate, CI low, CI high (percent)
"Sensitivity": (98.2, 90.28, 99.95),
"Specificity": (82.6, 68.58, 92.18),
"PPV": (87.10, 78.22, 92.69),
"NPV": (97.44, 84.44, 99.63),
}
ABSTRACT_PRINTED = {"Sensitivity": (98.2, 90.3, 99.9), "Specificity": (82.6, 68.6, 92.2)}
DISC = dict(cancer=25, benign=15, sens=1.00, spec=0.80) # Results, discovery
VAL = dict(cancer=29, benign=23) # Results, Fig 4D legend
SITES = dict(discovery=40, internal=21, external_19=19, external_12=12) # Methods
STATED_N, STATED_FP, STATED_FN = 92, 8, 1 # Results, validation
POC = dict(n=8, cancer=4, benign=4) # proof-of-concept
CUTOFF = 1.25
# Table S1 (mmc2.xlsx), TP/TN/FP/FN column, aggregated per cohort
S1 = {
"Discovery (S1B, n=40)": dict(TP=25, TN=12, FP=3, FN=0),
"Internal validation (S1C, n=21)": dict(TP=15, TN=4, FP=2, FN=0),
"External, 12 subjects (S1D)": dict(TP=0, TN=10, FP=2, FN=0),
"External, 19 subjects (S1E)": dict(TP=13, TN=4, FP=1, FN=1),
}
def s1_from_xlsx(path):
"""Re-derive the per-cohort counts from Table S1; check labels against BDI > 1.25."""
import openpyxl
ws = openpyxl.load_workbook(path, data_only=True).worksheets[0]
out, sec, mism = {}, None, 0
for r in list(ws.iter_rows(values_only=True))[1:]:
if r[0] and str(r[0]).upper().startswith("TABLE"):
sec = str(r[0]).split("]")[0] + "]"
out[sec] = dict(TP=0, TN=0, FP=0, FN=0, cancer=0, benign=0)
continue
if sec is None or r[6] is None:
continue
canc = str(r[5]).strip().upper().startswith("Y")
out[sec]["cancer" if canc else "benign"] += 1
if r[11]:
lab = str(r[11]).strip()
out[sec][lab] += 1
pos = float(r[6]) > CUTOFF
exp = {(True, True): "TP", (False, False): "TN",
(True, False): "FP", (False, True): "FN"}[(pos, canc)]
mism += exp != lab
return out, mism
# ------------------------------------------------------------------ statistics
def cp(x, n, a=0.05):
"""Exact Clopper-Pearson interval."""
lo = 0.0 if x == 0 else beta.ppf(a / 2, x, n - x + 1)
hi = 1.0 if x == n else beta.ppf(1 - a / 2, x + 1, n - x)
return lo, hi
def logit_pv(TP, TN, FP, FN):
"""PPV/NPV with the Mercaldo et al. (2007) logit CI (as in MedCalc)."""
z = norm.ppf(0.975)
npos, nneg = TP + FN, TN + FP
se, sp = TP / npos, TN / nneg
ppv, npv = TP / (TP + FP), TN / (TN + FN)
v_ppv = ((1 - se) / se) / npos + (sp / (1 - sp)) / nneg
v_npv = (se / (1 - se)) / npos + ((1 - sp) / sp) / nneg if se < 1 else np.nan
f = lambda p, v: (1 / (1 + np.exp(-(np.log(p / (1 - p)) - z * np.sqrt(v)))),
1 / (1 + np.exp(-(np.log(p / (1 - p)) + z * np.sqrt(v)))))
return (ppv, *f(ppv, v_ppv)), (npv, *f(npv, v_npv))
def metrics(c):
TP, TN, FP, FN = c["TP"], c["TN"], c["FP"], c["FN"]
se, sp = TP / (TP + FN), TN / (TN + FP)
(ppv, pl, ph), (npv, nl, nh) = logit_pv(TP, TN, FP, FN)
return {
"Sensitivity": (100 * se, *(100 * v for v in cp(TP, TP + FN)), f"{TP}/{TP + FN}"),
"Specificity": (100 * sp, *(100 * v for v in cp(TN, TN + FP)), f"{TN}/{TN + FP}"),
"PPV": (100 * ppv, 100 * pl, 100 * ph, f"{TP}/{TP + FP}"),
"NPV": (100 * npv, 100 * nl, 100 * nh, f"{TN}/{TN + FN}"),
}
def show(label, c):
n = sum(c[k] for k in ("TP", "TN", "FP", "FN"))
print(f"\n--- {label}: TP={c['TP']} TN={c['TN']} FP={c['FP']} FN={c['FN']} "
f"N={n} cancers(TP+FN)={c['TP'] + c['FN']} benign(TN+FP)={c['TN'] + c['FP']}")
m = metrics(c)
for k, (est, lo, hi, frac) in m.items():
meth = "Clopper-Pearson" if k in ("Sensitivity", "Specificity") else "logit"
print(f" {k:<11} {frac:>6} = {est:6.2f}% 95% CI {lo:.2f}-{hi:.2f} ({meth})")
return m
def add(*cs):
return {k: sum(c[k] for c in cs) for k in ("TP", "TN", "FP", "FN")}
# ------------------------------------------------------------------ 1. head counts
print("=" * 78)
print("1. Head counts stated in the paper")
print(f" Subjects by site 40+21+19+12 = {sum(SITES.values())}")
print(f" Validation subjects 21+19+12 = {SITES['internal'] + SITES['external_19'] + SITES['external_12']}"
f" (= {VAL['cancer']} OSCC + {VAL['benign']} benign = {VAL['cancer'] + VAL['benign']})")
canc = DISC["cancer"] + VAL["cancer"]
ben = DISC["benign"] + VAL["benign"]
print(f" Cancers 25+29 = {canc}; benign 15+23 = {ben}; total = {canc + ben}")
t1n = sum(TABLE1.values())
print(f" Table 1 cells 54+38+8+1 = {t1n} (stated n = {STATED_N}; excess = {t1n - STATED_N}"
f" = FP+FN = {TABLE1['FP'] + TABLE1['FN']})")
print(f" Table 1 implies cancers TP+FN = {TABLE1['TP'] + TABLE1['FN']} (stated {canc}),"
f" benign TN+FP = {TABLE1['TN'] + TABLE1['FP']} (stated {ben})")
print(f" Table 1 TP = {TABLE1['TP']} equals the stated cancer total ({canc});"
f" TN = {TABLE1['TN']} equals the stated benign total ({ben})")
# ------------------------------------------------------------------ 2. accuracy
print("\n" + "=" * 78)
print("2. Accuracy estimates")
m_print = show("Table 1 cells as printed", TABLE1)
print(" Printed in Table 1:", {k: v for k, v in TABLE1_PRINTED.items()})
for k, (e, lo, hi) in TABLE1_PRINTED.items():
r = m_print[k]
print(f" {k:<11} printed {e} ({lo}-{hi}) vs recomputed {r[0]:.2f} ({r[1]:.2f}-{r[2]:.2f})"
f" -> {'match' if abs(round(r[1], 2) - lo) < 0.006 and abs(round(r[2], 2) - hi) < 0.006 else 'DIFFER'}")
implied = dict(TP=canc - STATED_FN, TN=ben - STATED_FP, FP=STATED_FP, FN=STATED_FN)
m_impl = show("Implied by stated cohort (54 cancers, 38 benign) with 8 FP and 1 FN", implied)
pooled_s1 = add(*S1.values())
m_s1 = show("Table S1 labels, discovery + validation pooled", pooled_s1)
print(f" identical to the cohort-implied table: {pooled_s1 == implied}")
disc = S1["Discovery (S1B, n=40)"]
val = add(*[v for k, v in S1.items() if not k.startswith("Discovery")])
print(f"\n Discovery check: sensitivity {disc['TP']}/{disc['TP'] + disc['FN']} ="
f" {disc['TP'] / (disc['TP'] + disc['FN']):.0%}; specificity {disc['TN']}/{disc['TN'] + disc['FP']}"
f" = {disc['TN'] / (disc['TN'] + disc['FP']):.0%} (paper: 100%, 80%)")
print(f" Discovery counts implied by the text alone: TP = {round(DISC['sens'] * DISC['cancer'])},"
f" TN = {round(DISC['spec'] * DISC['benign'])}, FP = {DISC['benign'] - round(DISC['spec'] * DISC['benign'])}")
m_val = show("Validation cohorts only (S1C+S1D+S1E)", val)
for k, c in S1.items():
nb = c["TN"] + c["FP"]
print(f" {k:<34} specificity {c['TN']}/{nb} = {100 * c['TN'] / nb:.1f}%"
+ (f"; sensitivity {c['TP']}/{c['TP'] + c['FN']}" if c['TP'] + c['FN'] else ""))
print("\n Abstract (rounded to 1 dp):", ABSTRACT_PRINTED)
for k in ("Sensitivity", "Specificity"):
print(f" {k}: Table 1 cells -> {m_print[k][0]:.1f} ({m_print[k][1]:.2f}-{m_print[k][2]:.2f});"
f" stated cohort -> {m_impl[k][0]:.1f} ({m_impl[k][1]:.2f}-{m_impl[k][2]:.2f})")
print(f" Highlights '98% and 83%' -> stated cohort {m_impl['Sensitivity'][0]:.0f}% and"
f" {m_impl['Specificity'][0]:.0f}%")
# ------------------------------------------------------------------ 3. derived claims
print("\n" + "=" * 78)
print("3. Derived claims")
print(f" Discussion '17.4%' = 100 - 82.6 = {100 - 82.6:.1f}% (i.e. 1 - specificity, 8/46 = {100 * 8 / 46:.1f}%)")
print(f" With the stated cohort: 1 - specificity = 8/38 = {100 * 8 / 38:.1f}%")
print(f" Point-estimate change in specificity: {m_print['Specificity'][0]:.2f} -> {m_impl['Specificity'][0]:.2f}"
f" ({m_impl['Specificity'][0] - m_print['Specificity'][0]:+.2f} points);"
f" lower CI bound {m_print['Specificity'][1]:.2f} -> {m_impl['Specificity'][1]:.2f}")
# ------------------------------------------------------------------ 4. alternatives
print("\n" + "=" * 78)
print("4. Innocent explanations: can any stated grouping give 55 cancers / 46 benign / 101?")
print(f" + proof-of-concept cohort: {STATED_N} + {POC['n']} = {STATED_N + POC['n']};"
f" cancers {canc + POC['cancer']}, benign {ben + POC['benign']}")
print(f" validation only: {VAL['cancer']} cancers, {VAL['benign']} benign, n = {VAL['cancer'] + VAL['benign']}")
print(f" discovery only: {DISC['cancer']} cancers, {DISC['benign']} benign, n = {DISC['cancer'] + DISC['benign']}")
print(f" rounding: 53/54 = {100 * 53 / 54:.3f}% -> {100 * 53 / 54:.1f}; CP lower bound {100 * cp(53, 54)[0]:.2f}"
f" vs printed 90.28; 54/55 CP lower bound {100 * cp(54, 55)[0]:.2f}")
print(f" 30/38 = {100 * 30 / 38:.2f}%; 38/46 = {100 * 38 / 46:.2f}%; with FP = 8, TN/(TN+8) = 0.826 needs TN = 38, i.e. 46 benign")
if len(sys.argv) > 1:
print("\n" + "=" * 78)
print("5. Re-derived from", os.path.basename(sys.argv[1]))
out, mism = s1_from_xlsx(sys.argv[1])
for k, v in out.items():
print(f" {k}: {v}")
print(f" labels disagreeing with BDI > {CUTOFF}: {mism}")
# ------------------------------------------------------------------ figures
ORANGE, BLUE = "#eb6834", "#2a78d6"
INK, INK2, BG, GRID = "#0b0b0b", "#52514e", "#fcfcfb", "#e4e3df"
TINT_O, TINT_B = "#fbe3d9", "#dde9f8"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
LW = 1.5 # 2 px at 96 px/in
def save(fig, path):
fig.savefig(path + ".svg")
fig.savefig(path + ".png", dpi=200)
plt.close(fig)
def box(ax, x, y, w, h, fc=BG, ec=INK2, lw=1.0, r=0.012):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle=f"round,pad=0,rounding_size={r}",
fc=fc, ec=ec, lw=lw, transform=ax.transAxes))
def f1(v):
"""One decimal, except keep two when one decimal would print 100.0."""
t = f"{v:.1f}"
return f"{v:.2f}" if t == "100.0" else t
# --- design 3: two 2x2 tables (default fig1)
def fig_two_tables(path):
fig = plt.figure(figsize=(7.2, 3.5))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1)
panels = [
(0.025, "Table 1 as printed", ORANGE, TINT_O, TABLE1, m_print,
"the paper's reported cells"),
(0.525, "Rebuilt from the stated cohort", BLUE, TINT_B, implied, m_impl,
"54 cancers, 38 benign; 8 FP, 1 FN"),
]
for x0, title, col, tint, c, m, sub in panels:
ax.add_patch(Rectangle((x0, 0.865), 0.011, 0.075, fc=col, ec="none", transform=ax.transAxes))
ax.text(x0 + 0.022, 0.915, title, fontsize=12, weight="bold", va="center")
ax.text(x0 + 0.022, 0.865, sub, fontsize=10, color=INK2, va="center")
cw, ch = 0.10, 0.14
gx, gy = x0 + 0.10, 0.44
for i, lab in enumerate(("Cancer", "Benign")):
ax.text(gx + (i + 0.5) * cw, gy + 2 * ch + 0.03, lab, ha="center", fontsize=10.5, color=INK2)
ax.text(gx + 2 * cw + 0.06, gy + 2 * ch + 0.03, "Total", ha="center", fontsize=10.5, color=INK2)
ax.text(gx - 0.012, gy + 1.5 * ch, "BDI +", ha="right", va="center", fontsize=10.5, color=INK2)
ax.text(gx - 0.012, gy + 0.5 * ch, "BDI −", ha="right", va="center", fontsize=10.5, color=INK2)
cells = [(0, 1, "TP", c["TP"]), (1, 1, "FP", c["FP"]), (0, 0, "FN", c["FN"]), (1, 0, "TN", c["TN"])]
for i, j, lab, v in cells:
hl = lab in ("TP", "TN")
ax.add_patch(Rectangle((gx + i * cw, gy + j * ch), cw, ch, fc=tint if hl else BG,
ec=INK2, lw=0.8, transform=ax.transAxes))
ax.text(gx + i * cw + 0.008, gy + j * ch + ch - 0.028, lab, fontsize=10, color=INK2, va="center")
ax.text(gx + (i + 0.5) * cw, gy + (j + 0.40) * ch, str(v), fontsize=14,
weight="bold" if hl else "normal", ha="center", va="center")
ncan, nben = c["TP"] + c["FN"], c["TN"] + c["FP"]
n = ncan + nben
for i, v in ((0, ncan), (1, nben)):
ax.text(gx + (i + 0.5) * cw, gy - 0.065, str(v), ha="center", fontsize=12, weight="bold")
ax.text(gx + 2 * cw + 0.06, gy - 0.065, f"N = {n}", ha="center", fontsize=12, weight="bold")
ax.plot([gx, gx + 2 * cw + 0.115], [gy - 0.018] * 2, color=INK2, lw=0.8, transform=ax.transAxes)
ok = (ncan, nben, n) == (canc, ben, STATED_N)
ax.text(gx + 2 * cw + 0.115, gy - 0.145,
"= stated 54 / 38 / 92" if ok else
f"stated 54 / 38 / 92: +{ncan - canc}, +{nben - ben}, +{n - STATED_N}",
ha="right", fontsize=10, color=INK2)
se, sp = m["Sensitivity"], m["Specificity"]
ax.text(x0 + 0.022, 0.155, f"Sensitivity {se[3]} = {f1(se[0])}% ({f1(se[1])}–{f1(se[2])})",
fontsize=10.5)
ax.text(x0 + 0.022, 0.085, f"Specificity {sp[3]} = {f1(sp[0])}% ({f1(sp[1])}–{f1(sp[2])})",
fontsize=10.5, weight="bold")
ax.plot([0.5, 0.5], [0.06, 0.95], color=GRID, lw=1.0, transform=ax.transAxes)
ax.text(0.5, 0.018, "Shaded cells differ between the two tables. 95% CIs: Clopper–Pearson.",
ha="center", fontsize=10, color=INK2)
save(fig, path)
# --- design 4: dot-and-interval (default fig2)
def fig_dots(path):
rows = ["Sensitivity", "Specificity", "PPV", "NPV"]
fig, ax = plt.subplots(figsize=(7.2, 3.6))
fig.subplots_adjust(left=0.16, right=0.67, top=0.86, bottom=0.17)
for i, k in enumerate(rows):
y = len(rows) - 1 - i
e, lo, hi = TABLE1_PRINTED[k]
ax.plot([lo, hi], [y + 0.14] * 2, color=ORANGE, lw=LW, solid_capstyle="butt")
ax.plot(e, y + 0.14, "o", color=ORANGE, ms=8, mec=BG, mew=1.2)
r = m_impl[k]
ax.plot([r[1], r[2]], [y - 0.14] * 2, color=BLUE, lw=LW, solid_capstyle="butt")
ax.plot(r[0], y - 0.14, "s", color=BLUE, ms=8, mec=BG, mew=1.2)
ax.text(101.5, y + 0.14, f"{f1(e)} ({f1(lo)}–{f1(hi)}) {m_print[k][3]}", va="center", fontsize=10, color=INK)
ax.text(101.5, y - 0.14, f"{f1(r[0])} ({f1(r[1])}–{f1(r[2])}) {r[3]}", va="center", fontsize=10, color=INK)
ax.set_yticks(range(len(rows))); ax.set_yticklabels(rows[::-1], fontsize=11, color=INK)
ax.set_xlim(50, 100); ax.set_ylim(-0.6, len(rows) - 0.4)
ax.set_xlabel("Estimate, % (95% CI); axis starts at 50%")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.plot([], [], "o-", color=ORANGE, ms=8, lw=LW, label="Reported (Table 1 cells, N = 101)")
ax.plot([], [], "s-", color=BLUE, ms=8, lw=LW, label="Recomputed (stated cohort, N = 92)")
ax.legend(loc="lower left", bbox_to_anchor=(-0.02, 1.0), ncol=2, frameon=False, fontsize=10,
handlelength=1.8, columnspacing=1.2)
save(fig, path)
# --- design 1: text vs table
def fig_text_vs_table(path):
fig = plt.figure(figsize=(7.2, 3.7))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
box(ax, 0.02, 0.30, 0.59, 0.62)
ax.text(0.04, 0.86, "Results text and Figure 4 legends", fontsize=11.5, weight="bold", va="center")
lines = [
("Discovery:", "“25 were diagnosed ... as being cancerous"),
("", "while 15 were benign”"),
("Validation:", "“23 non-cancerous and 29 OSCC lesions”"),
("Pooled:", "“For all the subjects (n = 92)”"),
("", "“eight false positives (FPs) and one FN”"),
]
for i, (a, b) in enumerate(lines):
y = 0.77 - i * 0.075
ax.text(0.04, y, a, fontsize=10, color=INK2, va="center")
ax.text(0.155, y, b, fontsize=10, va="center")
ax.add_patch(Rectangle((0.02, 0.30), 0.008, 0.62, fc=BLUE, ec="none", transform=ax.transAxes))
ax.text(0.04, 0.36, f"→ {canc} cancers + {ben} benign = {STATED_N}", fontsize=11, weight="bold", va="center")
box(ax, 0.69, 0.30, 0.29, 0.62)
ax.add_patch(Rectangle((0.69, 0.30), 0.008, 0.62, fc=ORANGE, ec="none", transform=ax.transAxes))
ax.text(0.71, 0.86, "Table 1", fontsize=11.5, weight="bold", va="center")
for i, (k, v) in enumerate(TABLE1.items()):
y = 0.77 - i * 0.075
ax.text(0.72, y, {"TP": "True positive", "TN": "True negative", "FP": "False positive",
"FN": "False negative"}[k], fontsize=10, va="center")
ax.text(0.955, y, str(v), fontsize=10.5, weight="bold", ha="right", va="center")
ax.text(0.72, 0.43, "Sensitivity 98.2", fontsize=10, va="center")
ax.text(0.72, 0.36, "Specificity 82.6", fontsize=10, va="center")
ax.annotate("", xy=(0.685, 0.61), xytext=(0.615, 0.61), xycoords="axes fraction",
arrowprops=dict(arrowstyle="<->", color=INK2, lw=LW))
ax.text(0.65, 0.655, "conflict", ha="center", fontsize=10, color=INK2)
ax.text(0.5, 0.17, f"Table 1 cells sum to 54 + 38 + 8 + 1 = {t1n}, not {STATED_N}: they imply "
f"{TABLE1['TP'] + TABLE1['FN']} cancers and {TABLE1['TN'] + TABLE1['FP']} benign lesions.",
ha="center", fontsize=10.5)
ax.text(0.5, 0.09, f"TP and TN equal the cohort totals ({canc}, {ben}), so the 8 FP and 1 FN are counted twice.",
ha="center", fontsize=10.5, color=INK2)
save(fig, path)
# --- design 2: calculation flow
def fig_flow(path):
fig = plt.figure(figsize=(7.2, 3.4))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
box(ax, 0.02, 0.33, 0.19, 0.32)
ax.text(0.115, 0.59, "Stated cohort", ha="center", fontsize=10.5, weight="bold")
ax.text(0.115, 0.51, f"{canc} cancers", ha="center", fontsize=10.5)
ax.text(0.115, 0.44, f"{ben} benign", ha="center", fontsize=10.5)
ax.text(0.115, 0.37, "8 FP, 1 FN", ha="center", fontsize=10, color=INK2)
paths = [
(0.60, ORANGE, "As reported in Table 1",
[f"TP = {canc}\nTN = {ben}", "FP 8, FN 1 added\non top: N = 101", f"Se 54/55 = {m_print['Sensitivity'][0]:.1f}%\nSp 38/46 = {m_print['Specificity'][0]:.1f}%"],
"cohort totals used\nas TP and TN"),
(0.08, BLUE, "Recomputed from the paper's counts",
[f"TP = {canc} − 1 = {implied['TP']}\nTN = {ben} − 8 = {implied['TN']}", "FP 8, FN 1 inside\nthe totals: N = 92",
f"Se 53/54 = {m_impl['Sensitivity'][0]:.1f}%\nSp 30/38 = {m_impl['Specificity'][0]:.1f}%"],
"errors taken out\nof the totals"),
]
xs = [0.28, 0.49, 0.70]
ws = [0.18, 0.18, 0.28]
h = 0.24
for y0, col, lab, steps, note in paths:
ax.add_patch(Rectangle((0.28, y0 + h + 0.035), 0.011, 0.055, fc=col, ec="none", transform=ax.transAxes))
ax.text(0.30, y0 + h + 0.062, lab, fontsize=10.5, weight="bold", va="center")
for j, (x, w, s) in enumerate(zip(xs, ws, steps)):
last = j == len(steps) - 1
box(ax, x, y0, w, h, ec=col if last else INK2, lw=LW if last else 0.8)
ax.text(x + w / 2, y0 + h / 2, s, ha="center", va="center", fontsize=10,
weight="bold" if last else "normal", linespacing=1.4)
if j:
ax.annotate("", xy=(x, y0 + h / 2), xytext=(xs[j - 1] + ws[j - 1], y0 + h / 2),
xycoords="axes fraction", arrowprops=dict(arrowstyle="->", color=INK2, lw=1.0))
ax.annotate("", xy=(xs[0], y0 + h / 2), xytext=(0.21, 0.49), xycoords="axes fraction",
arrowprops=dict(arrowstyle="->", color=INK2, lw=1.0, connectionstyle="arc3,rad=0"))
save(fig, path)
# --- design 5: icon array by cell
def fig_icons(path):
fig = plt.figure(figsize=(7.2, 3.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 72); ax.set_ylim(0, 30)
s, gap, per = 1.0, 0.25, 30
rows = [("TP", implied["TP"], TABLE1["TP"]), ("TN", implied["TN"], TABLE1["TN"]),
("FP", implied["FP"], TABLE1["FP"]), ("FN", implied["FN"], TABLE1["FN"])]
y = 24.0
for lab, n_ok, n_tab in rows:
ax.text(1.0, y - 0.6, lab, fontsize=11, weight="bold", va="center")
for k in range(n_tab):
cx, cy = 5 + (k % per) * (s + gap), y - (k // per) * (s + gap) - 1
extra = k >= n_ok
ax.add_patch(Rectangle((cx, cy), s, s, fc=BG if extra else BLUE,
ec=ORANGE if extra else BLUE, lw=1.2,
hatch="////" if extra else None))
ax.text(5 + per * (s + gap) + 1.0, y - 0.6,
f"Table 1: {n_tab} stated cohort: {n_ok}" + (f" (+{n_tab - n_ok})" if n_tab > n_ok else ""),
fontsize=10.5, va="center")
y -= 2.5 + 2.5 * ((n_tab - 1) // per)
ax.add_patch(Rectangle((5, 3.6), s, s, fc=BLUE, ec=BLUE))
ax.text(6.6, 4.1, "subject counted in both (Table S1 label; recomputed)", fontsize=10, va="center")
ax.add_patch(Rectangle((5, 1.4), s, s, fc=BG, ec=ORANGE, lw=1.2, hatch="////"))
ax.text(6.6, 1.9, "extra count present only in Table 1 as reported (9 in all: N = 101 vs 92)",
fontsize=10, va="center")
ax.text(1.0, 28.0, "Each square is one count in the 2×2 table", fontsize=11.5, weight="bold", va="center")
save(fig, path)
fig_two_tables(os.path.join(HERE, "fig1"))
fig_dots(os.path.join(HERE, "fig2"))
fig_text_vs_table(os.path.join(CAND, "c1_text_vs_table"))
fig_flow(os.path.join(CAND, "c2_calculation_flow"))
fig_two_tables(os.path.join(CAND, "c3_two_tables"))
fig_dots(os.path.join(CAND, "c4_dot_interval"))
fig_icons(os.path.join(CAND, "c5_icon_array"))
print("\nFigures written to", HERE, "and", CAND)
# ------------------------------------------------------------------ contact sheet of the five designs
def contact_sheet(path):
import matplotlib.image as mpimg
names = [("c1_text_vs_table", "1. Text vs table: conflicting excerpts joined by a connector"),
("c2_calculation_flow", "2. Calculation flow: how 54 / 38 became TP / TN"),
("c3_two_tables", "3. Two 2×2 tables side by side (used as fig1)"),
("c4_dot_interval", "4. Dot-and-interval plot of the four estimates (used as fig2)"),
("c5_icon_array", "5. Icon array: one square per count, extra counts hatched")]
imgs = [mpimg.imread(os.path.join(CAND, n + ".png")) for n, _ in names]
hs = [im.shape[0] / im.shape[1] * 7.2 for im in imgs]
fig = plt.figure(figsize=(7.2, sum(hs) + 0.45 * len(imgs)))
y = 1.0
H = sum(hs) + 0.45 * len(imgs)
for im, h, (_, lab) in zip(imgs, hs, names):
y -= 0.45 / H
fig.text(0.01, y + 0.12 / H, lab, fontsize=11, weight="bold", color=INK)
ax = fig.add_axes([0, y - h / H, 1, h / H]); ax.imshow(im); ax.axis("off")
y -= h / H
fig.savefig(path + ".png", dpi=110)
plt.close(fig)
contact_sheet(os.path.join(CAND, "all_five_designs"))
print("Contact sheet written to", os.path.join(CAND, "all_five_designs.png"))
Abstract reports all-grade adverse event rates as grade ≥3 treatment-related rates in a phase I trial
Clin Cancer Res (2025) · doi:10.1158/1078-0432.CCR-24-1983 · PMC12163594
Summary
The Abstract of this first-in-human phase I trial of intratumoral SAR441000, given alone or with cemiplimab, states that the most common grade ≥3 treatment-related adverse events (TRAEs) were fatigue in escalation (28.6% with monotherapy, 66.7% with the combination) and injection-site pain in expansion (31.7%). All three percentages equal the all-grade, all-causality treatment-emergent AE (TEAE) rates in Table 2B (6/21, 10/15 and 13/41). In the same table, grade ≥3 fatigue is 0/21 and 0/15, and grade ≥3 injection-site pain is 1/41 (2.4%). The paper's own Table 2A and Results text add that no monotherapy patient had a grade ≥3 TRAE of any kind and that only 1 of 15 escalation-combination patients did. Safety was a primary objective, and this sentence is the Abstract's only quantitative safety result. As printed, it overstates severe treatment-related toxicity by about 10-fold or more.
What the paper reports
The most common grade ≥3 treatment–related adverse events was fatigue in the escalation phase (monotherapy: 28.6% and combination therapy: 66.7%) and injection-site pain (31.7%) in the expansion phase.
— Abstract, Results
Of the 21 patients, 19 (90.5%) experienced any-grade AEs, of which the most common were vomiting and fatigue in 6 (28.6%) patients each, followed by hypertension, chills, and pain in extremities in 4 (19.0%) patients each. Nine (42.9%) patients experienced grade ≥3 AEs (Table 2). Furthermore, 13 (61.9%) patients reported TRAEs, but none were grade ≥3 or considered serious; grade ≥5 AEs were not reported.
— Results, Safety, "Escalation monotherapy"
All 15 patients experienced any-grade AEs, with the most common being fatigue (10/15; 66.7%), followed by hypertension and nausea in 5 (33.3%) patients each. Grade ≥3 AEs—hypertension, pyrexia, and anemia—were experienced by one (6.7%) patient each (Table 2). Ten (66.7%) patients experienced any-grade TRAEs, with the most common being fatigue (n = 7; 46.7%). One (6.7%) patient experienced grade ≥3 syncope (Supplementary Table S5).
— Results, Safety, "Escalation combination therapy"
All 41 patients experienced any-grade AEs, of which injection-site pain (n = 13; 31.7%) was the most common all-grade AE, followed by diarrhea, decreased appetite, and asthenia (n = 11 each; 26.8%; Table 2). [...] Thirty-three (80.5%) patients had any-grade TRAEs, of which six (14.6%) patients had TRAEs of grade ≥3 that were diarrhea, injection-site pain, and fatigue (Supplementary Table S5).
— Results, Safety, "Expansion combination"
Patients with any TRAE of grade ≥3 | 0 | 1 (6.7) | 6 (14.6)
— Table 2A ("AEs: (A) summary and (B) TEAEs occurring in ≥15% of patients"). The columns are escalation monotherapy (N = 21), escalation combination (N = 15) and expansion combination (N = 41).
Fatigue | 6 (28.6) | 0 | 10 (66.7) | 0 | 9 (22.0) | 1 (2.4)
Injection-site pain | 0 | 0 | 0 | 0 | 13 (31.7) | 1 (2.4)
— Table 2B, same three cohorts. Each cohort has an "All grades" column followed by a "Grade ≥3" column. The table footer defines the rows as TEAEs.
Intratumoral administration of SAR441000 in combination with cemiplimab was generally well tolerated with antitumor activity in the locoregional disease setting.
— Abstract, Conclusions
What the paper's own numbers show
Method. Each incidence is 100 × (patients with the event) / (patients in the cohort), with the cohort sizes from the Abstract and Table 2 (21, 15, 41). For each of the Abstract's three figures, the script tries every incidence the paper reports for the same AE and cohort: all-grade and grade ≥3 rates, all-causality TEAEs (Table 2B) and treatment-related AEs (Supplementary Table S5). It also checks the figure against the cohort's ceiling for grade ≥3 TRAEs of any kind (Table 2A). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Fatigue, escalation monotherapy, labelled "grade ≥3 treatment-related" | Abstract | 28.6% | equals all-grade TEAE 6/21 = 28.6% (Table 2B) |
| Grade ≥3 fatigue, escalation monotherapy | Table 2B; Suppl. Table S5 | 0; 0 | 0/21 = 0.0% (all causes and treatment-related) |
| Any grade ≥3 TRAE, escalation monotherapy | Table 2A; Results | 0; "none were grade ≥3" | 0/21 = 0.0%. Grade ≥3 TR fatigue of 28.6% would need 6.0 patients |
| Fatigue, escalation combination, labelled "grade ≥3 treatment-related" | Abstract | 66.7% | equals all-grade TEAE 10/15 = 66.7% (Table 2B) |
| Grade ≥3 fatigue, escalation combination | Table 2B; Suppl. Table S5 | 0; 0 | 0/15 = 0.0% |
| Any grade ≥3 TRAE, escalation combination | Table 2A; Results | 1 (6.7%) | 1/15 = 6.7%. Grade ≥3 TR fatigue of 66.7% would need 10.0 patients |
| Treatment-related fatigue, any grade, escalation combination | Results; Suppl. Table S5 | n = 7; 46.7% | 7/15 = 46.7%, so 66.7% is not a TRAE rate at any grade |
| Injection-site pain, expansion, labelled "grade ≥3 treatment-related" | Abstract | 31.7% | equals all-grade TEAE 13/41 = 31.7% (Table 2B) |
| Grade ≥3 injection-site pain, expansion | Table 2B; Suppl. Table S5 | 1 (2.4); 1 (2.4) | 1/41 = 2.4%; the Abstract's figure is 13.0 times this |
| Treatment-related injection-site pain, any grade, expansion | Suppl. Table S5 | 11 (26.8) | 11/41 = 26.8% |
| Any grade ≥3 TRAE, expansion | Table 2A; Results | 6 (14.6%) | 6/41 = 14.6% |
| Largest number of patients with any single grade ≥3 TRAE term, expansion | Suppl. Table S5 | 7 terms with 1 (2.4) each | 1/41 = 2.4%; no term is more common than any other |
| Ratio of Abstract figure to the cohort's any-grade ≥3 TRAE rate | derived | — | monotherapy: 28.6% vs 0% (no finite ratio); combination: 66.7 / 6.7 = 10.0; expansion: 31.7 / 14.6 = 2.2 |
All three Abstract figures match the all-grade, all-causality TEAE rate exactly. None matches any grade ≥3 rate or any treatment-related rate the paper reports. In the two escalation cohorts the stated figures cannot be grade ≥3 TRAE rates at all. Even if every patient with a grade ≥3 TRAE of any kind had had grade ≥3 fatigue, the maximum would be 0/21 and 1/15.
Figure
Figure 1. The left panel shows the Abstract sentence and its three percentages (orange). Each arrow joins a percentage to the Table 2B row it reproduces exactly. In all three rows the match is the "All grades" cell (orange), the count of patients with the event at any grade and from any cause. The blue cells show the grade ≥3 rate recomputed from the adjacent "Grade ≥3" count in the same row: 0/21, 0/15 and 1/41. Underneath, the right panel gives the cohort-level ceiling from Table 2A (any grade ≥3 TRAE: 0/21, 1/15, 6/41) and the Results text on monotherapy and on treatment-related fatigue. Only patient counts and percentages printed in the paper, or computed from them by recompute.py.txt, are shown.
Figure 2. The same comparison on one axis (percent of patients in each cohort, 0–100%). Orange dots are the Abstract's figures. Each sits inside an open ring marking the all-grade, all-causality rate from Table 2B. Blue diamonds are the grade ≥3 rates of the same AE (Table 2B), and open blue squares are the cohort's rate of grade ≥3 TRAEs of any kind (Table 2A). That second rate is an upper bound for any single grade ≥3 TRAE. In escalation monotherapy the diamond and the square coincide at 0%. In each cohort the Abstract's figure lies well above that upper bound.
Why it matters
Overall safety and tolerability are part of the trial's primary objective. The Abstract sentence quoted above is its only quantitative statement on safety. A reader of the Abstract alone would conclude that two-thirds of patients in escalation combination had severe treatment-related fatigue, and that nearly a third of expansion patients had severe treatment-related injection-site pain. The paper's tables say neither happened: no escalation patient had grade ≥3 fatigue, and 1 of 41 expansion patients had grade ≥3 injection-site pain. The sentence also misidentifies the "most common" grade ≥3 TRAE. Monotherapy had none. In escalation combination, the one patient with a grade ≥3 TRAE is described as having syncope. In expansion, Supplementary Table S5 lists seven grade ≥3 TRAE terms with one patient each, so no single term stands out.
The error runs against the paper's own conclusion. As printed, the Abstract's figures sit uneasily with "generally well tolerated"; the correct figures (grade ≥3 TRAEs of any kind in 0%, 6.7% and 14.6% of patients) support it. Abstract-level safety figures from first-in-human trials are often carried into reviews, meta-analyses of intratumoral immunotherapies and dose-selection discussions. A 10-fold or larger overstatement of severe toxicity could be passed on in those places.
The error does not affect the maximum administered dose (4,000 µg), the response counts, the pharmacokinetic and pharmacodynamic results, or the Discussion. The Discussion describes AE frequencies without the mislabelled percentages. It ends: "SAR441000 demonstrated a generally acceptable safety profile in both monotherapy and combination therapy cohorts, although a higher number of high-grade AEs were observed in the combination therapy cohort."
Explanations considered
- A different population or denominator. This does not account for the numbers. The Abstract gives the same cohort sizes (N = 21, 15, 41) as Table 2. The ratios 6/21, 10/15 and 13/41 reproduce 28.6%, 66.7% and 31.7% exactly.
- A treatment-related table rather than Table 2B. This does not account for the numbers. Supplementary Table S5 ("Treatment-related adverse events") gives fatigue as 2 (9.5) and 7 (46.7) at all grades and 0 at grade ≥3 in the two escalation cohorts. It gives injection-site pain in expansion as 11 (26.8) at all grades and 1 (2.4) at grade ≥3. None of these is 28.6%, 66.7% or 31.7%.
- A grade ≥3 rate that includes events not attributed to treatment. This does not account for the numbers. Grade ≥3 fatigue from any cause is also 0/21 and 0/15 (Table 2B), and grade ≥3 injection-site pain from any cause is 1/41.
- Patients counted once per event, or events rather than patients. This does not account for the numbers. Ten grade ≥3 fatigue events in escalation combination would still need grade ≥3 fatigue to appear in Table 2B or Supplementary Table S5, and both show 0. The cohort had only one patient with any grade ≥3 TRAE.
- A different time window or study phase. This does not account for the numbers. The Abstract sentence, Table 2 and the Results subsections all refer to the escalation and expansion phases over the treatment-emergent period. The table footnotes (timing of grade 5 events, definition of discontinuation) do not apply to these rows.
- A conversion or typesetting artefact in the archived copy. This was ruled out. The PMC HTML, the Europe PMC full-text XML and the PubMed abstract (PMID 40152791) give the same Abstract sentence. Table 2B lines up with its "All grades | Grade ≥3" header in each source checked. The numbers contain no signs or superscripts that could have been misread.
- The Abstract meant "all-grade" or "TEAE" and the qualifier is a wording slip. This is the most likely description of what went wrong, but it is not an innocent reading of the sentence as printed. "Grade ≥3 treatment-related" changes both the severity and the attribution of the event, and the figures are wrong under that label.
A smaller inconsistency sits in the paper's supporting material. The Results text names the single grade ≥3 TRAE in escalation combination as syncope, citing Supplementary Table S5. That table lists syncope as 1 (6.7) at all grades and 0 at grade ≥3, and lists dizziness and peripheral neuropathy as 1 (6.7) each at grade ≥3. This does not change the main finding: Table 2A gives one patient with any grade ≥3 TRAE in that cohort either way.
Status
As of 2026-10-23:
- Version of record: checked indirectly. The publisher's article page and the PDF link listed by Crossref (aacrjournals.org, /clincancerres/article/31/12/2358/762781/) returned HTTP 403 to automated requests. The PMC article page, the Europe PMC full-text XML and the PubMed record (PMID 40152791, built from the publisher's metadata) all give the Abstract sentence and the Table 2 rows word for word as quoted above.
- Supplement: checked. The Europe PMC supplementary package (Supplementary Figures S1–S4, Supplementary Tables S1–S9, Annexure 1) was downloaded. Supplementary Table S5 (treatment-related AEs, all grades and grade ≥3 by cohort) is cited above. It confirms grade ≥3 treatment-related fatigue of 0/21 and 0/15, and grade ≥3 treatment-related injection-site pain of 1/41. It does not resolve the discrepancy.
- Corrections: none found.
src/known.pyreturned no Crossref updates, PMC notices or Europe PMC notices. The Crossref record has no update-to or updated-by entries. The Europe PMC record has no comment or correction links. A web search for errata or comments on the article found none. - PubPeer: not reachable by automated query. The API returned "Human verification required" (HTTP 403). A web search found no PubPeer thread. A manual search at pubpeer.com is still advisable.
Sources: doi.org/10.1158/1078-0432.CCR-24-1983 · PMC12163594 · Europe PMC record · Europe PMC supplementary files · Crossref works record · ClinicalTrials.gov NCT03871348
Suggested correction
The Abstract sentence could be replaced with figures that match Table 2 and Supplementary Table S5. One option:
Grade ≥3 treatment-related adverse events occurred in 0 of 21 (0%), 1 of 15 (6.7%) and 6 of 41 (14.6%) patients in escalation monotherapy, escalation combination and expansion combination, respectively. The most common any-grade adverse events were fatigue in escalation (monotherapy: 28.6%; combination: 66.7%) and injection-site pain (31.7%) in expansion.
The authors could also check which preferred term was recorded for the single grade ≥3 TRAE in escalation combination. The Results text names syncope, while Supplementary Table S5 lists dizziness and peripheral neuropathy.
Recomputation script (python recompute.py.txt)
"""Recompute the abstract's grade >=3 treatment-related AE figures of PMC12163594
(Clin Cancer Res 2025, doi:10.1158/1078-0432.CCR-24-1983) and regenerate the figures.
Run: python recompute.py.txt
Needs: numpy, matplotlib.
Prints every recomputed number used in writeup.md. Writes fig1.svg/.png and
fig2.svg/.png into the folder that holds this file, plus five candidate figure
designs (cand1..cand5, .svg/.png) into ./candidates/.
All inputs are patient counts copied from the paper:
Table 2A (AE summary), Table 2B (TEAEs occurring in >=15% of patients, all causality),
the Results subsections "Escalation monotherapy", "Escalation combination therapy",
"Expansion combination", and Supplementary Table S5 (treatment-related AEs).
Incidence (%) = 100 * patients with the event / patients in the cohort.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, Rectangle
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ----------------------------------------------------------------------------- inputs
N = {"mono": 21, "comb": 15, "exp": 41}
LABEL = {"mono": "Escalation monotherapy", "comb": "Escalation combination",
"exp": "Expansion combination"}
# Abstract, Results: "The most common grade >=3 treatment-related adverse events was
# fatigue in the escalation phase (monotherapy: 28.6% and combination therapy: 66.7%)
# and injection-site pain (31.7%) in the expansion phase."
ABSTRACT = {"mono": ("Fatigue", 28.6), "comb": ("Fatigue", 66.7),
"exp": ("Injection-site pain", 31.7)}
# Table 2B (TEAEs, all causality): (all grades n, grade >=3 n)
TEAE = {
("mono", "Fatigue"): (6, 0),
("comb", "Fatigue"): (10, 0),
("exp", "Fatigue"): (9, 1),
("mono", "Injection-site pain"): (0, 0),
("comb", "Injection-site pain"): (0, 0),
("exp", "Injection-site pain"): (13, 1),
}
# Supplementary Table S5 (treatment-related AEs): (all grades n, grade >=3 n)
TRAE = {
("mono", "Fatigue"): (2, 0),
("comb", "Fatigue"): (7, 0),
("exp", "Fatigue"): (7, 1),
("mono", "Injection-site pain"): (0, 0),
("comb", "Injection-site pain"): (1, 0),
("exp", "Injection-site pain"): (11, 1),
}
# Table 2A: patients with any TRAE / any TRAE of grade >=3 (any preferred term)
ANY_TRAE = {"mono": 13, "comb": 10, "exp": 33}
ANY_TRAE_G3 = {"mono": 0, "comb": 1, "exp": 6}
# Table 2A: patients with grade >=3 TEAEs (any term, all causality)
ANY_TEAE_G3 = {"mono": 9, "comb": 5, "exp": 21}
# Supplementary Table S5: preferred terms with a grade >=3 TRAE, patients per term
S5_G3_TERMS = {
"mono": {},
"comb": {"Dizziness": 1, "Neuropathy peripheral": 1},
"exp": {"Diarrhoea": 1, "Muscular weakness": 1, "Fatigue": 1,
"Injection site pain": 1, "(term ending '...deterioration')": 1,
"Troponin increased": 1, "Hyperlipasaemia": 1},
}
def pct(n, d):
return round(100 * n / d, 1)
# ----------------------------------------------------------------------- recompute
print("=" * 78)
print("1. The abstract's three figures against the paper's own counts")
print("=" * 78)
rows = []
for k in ("mono", "comb", "exp"):
term, rep = ABSTRACT[k]
a, g3 = TEAE[(k, term)]
ta, tg3 = TRAE[(k, term)]
d = N[k]
r = dict(k=k, term=term, rep=rep, a=a, g3=g3, ta=ta, tg3=tg3, d=d,
p_all=pct(a, d), p_g3=pct(g3, d), p_tall=pct(ta, d), p_tg3=pct(tg3, d),
p_ceiling=pct(ANY_TRAE_G3[k], d), need=rep * d / 100)
rows.append(r)
print(f"\n{LABEL[k]} (N = {d}), {term}")
print(f" Abstract, labelled 'grade >=3 treatment-related': {rep}%")
print(f" All-grade TEAE, all causality (Table 2B): {a}/{d} = {r['p_all']}%"
f" -> matches abstract: {r['p_all'] == rep}")
print(f" Grade >=3 TEAE, all causality (Table 2B): {g3}/{d} = {r['p_g3']}%")
print(f" Any-grade treatment-related (Suppl. Table S5): {ta}/{d} = {r['p_tall']}%")
print(f" Grade >=3 treatment-related (Suppl. Table S5): {tg3}/{d} = {r['p_tg3']}%")
print(f" Any grade >=3 TRAE of any type (Table 2A): {ANY_TRAE_G3[k]}/{d} = {r['p_ceiling']}%")
print(f" Patients needed for {rep}%: {rep}*{d}/100 = {r['need']:.2f}"
f" (ceiling from Table 2A: {ANY_TRAE_G3[k]})")
print("\n" + "=" * 78)
print("2. Overstatement factors")
print("=" * 78)
for r in rows:
k = r["k"]
c = ANY_TRAE_G3[k]
if c == 0:
print(f"{LABEL[k]}: abstract {r['rep']}% vs any grade >=3 TRAE 0/{r['d']} = 0% (no finite ratio)")
else:
print(f"{LABEL[k]}: abstract {r['rep']}% / any grade >=3 TRAE {r['p_ceiling']}% = "
f"{(r['rep'] / 100) / (c / r['d']):.1f}x (in patients: {r['a']} vs {c})")
for r in rows:
if r["tg3"]:
print(f"{LABEL[r['k']]}: abstract {r['rep']}% vs grade >=3 TR {r['term'].lower()} "
f"{r['p_tg3']}% = {r['a'] / r['tg3']:.1f}x ({r['a']} vs {r['tg3']} patients)")
print("\n" + "=" * 78)
print("3. Other figures quoted in the write-up")
print("=" * 78)
for k in ("mono", "comb", "exp"):
print(f"{LABEL[k]}: any TRAE {ANY_TRAE[k]}/{N[k]} = {pct(ANY_TRAE[k], N[k])}%; "
f"grade >=3 TEAE (any) {ANY_TEAE_G3[k]}/{N[k]} = {pct(ANY_TEAE_G3[k], N[k])}%; "
f"grade >=3 TRAE (any) {ANY_TRAE_G3[k]}/{N[k]} = {pct(ANY_TRAE_G3[k], N[k])}%")
print(f"Expansion fatigue: all-grade TEAE 9/41 = {pct(9, 41)}%, grade >=3 1/41 = {pct(1, 41)}%")
print("Grade >=3 TRAE terms in Suppl. Table S5 (patients per term):")
for k in ("mono", "comb", "exp"):
terms = S5_G3_TERMS[k]
mx = max(terms.values()) if terms else 0
print(f" {LABEL[k]}: {len(terms)} terms, max patients per term = {mx}"
f"{' (' + str(round(100 * mx / N[k], 1)) + '%)' if mx else ''}: "
f"{', '.join(terms) if terms else 'none'}")
print(" Escalation combination: Results text names the one grade >=3 TRAE as 'syncope';"
" S5 lists syncope as 1 (6.7) all grades, 0 grade >=3.")
# --------------------------------------------------------------------------- style
BG, FG, FG2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
ORANGE_T, BLUE_T = "#fbe3d9", "#dde9f8"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": FG,
"axes.labelcolor": FG, "xtick.color": FG2, "ytick.color": FG,
"axes.edgecolor": GRID, "figure.facecolor": BG, "axes.facecolor": BG,
"svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg", facecolor=BG)
fig.savefig(p + ".png", dpi=200, facecolor=BG)
plt.close(fig)
def box(ax, x, y, w, h, fc, ec, lw=1.0, r=0.012):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle=f"round,pad=0,rounding_size={r}",
fc=fc, ec=ec, lw=lw, transform=ax.transAxes))
SHORT = {"mono": "Escalation mono (N = 21)", "comb": "Escalation combo (N = 15)",
"exp": "Expansion combo (N = 41)"}
# ------------------------------------------------- candidate 1 / fig1: text panels
def draw_text_panels():
fig = plt.figure(figsize=(W, 4.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1)
# left panel: abstract
L0, L1 = 0.025, 0.36
box(ax, L0, 0.20, L1 - L0, 0.71, "#ffffff", GRID)
ax.text(L0 + 0.015, 0.875, "Abstract, Results", fontsize=11, fontweight="bold", va="top")
ax.text(L0 + 0.015, 0.815,
"“The most common grade ≥3\ntreatment–related adverse\n"
"events was fatigue in the\nescalation phase\n(monotherapy: 28.6% and\n"
"combination therapy:\n66.7%) and injection-site\npain (31.7%) in the\n"
"expansion phase.”",
fontsize=10, va="top", color=FG, linespacing=1.3, style="italic")
ys = [0.395, 0.315, 0.235]
for y, r in zip(ys, rows):
box(ax, L0 + 0.012, y - 0.03, L1 - L0 - 0.024, 0.06, ORANGE_T, ORANGE, 1.2)
ax.text(L0 + 0.025, y, f"{r['rep']}%", color=FG, fontsize=10.5, fontweight="bold",
va="center")
ax.text(L0 + 0.105, y, f"{r['term'].replace('Injection-site pain', 'Inj.-site pain')},"
f" {('esc. mono', 'esc. combo', 'exp.')[ys.index(y)]}",
color=FG, fontsize=10, va="center")
# right panel: Table 2B + Table 2A + text
R0, R1 = 0.53, 0.985
box(ax, R0, 0.20, R1 - R0, 0.71, "#ffffff", GRID)
ax.text(R0 + 0.015, 0.875, "Table 2B (TEAEs, all causality)", fontsize=11,
fontweight="bold", va="top")
cA, cG = R0 + 0.27, R0 + 0.385
ax.text(cA, 0.765, "All\ngrades", ha="center", va="bottom", fontsize=10, color=FG2)
ax.text(cG, 0.765, "Grade\n≥3", ha="center", va="bottom", fontsize=10, color=FG2)
ax.plot([R0 + 0.015, R1 - 0.015], [0.755, 0.755], color=GRID, lw=1, transform=ax.transAxes)
names = ["Fatigue\nesc. mono, N=21", "Fatigue\nesc. combo, N=15",
"Inj.-site pain\nexpansion, N=41"]
yr = [0.68, 0.565, 0.45]
for y, nm, r in zip(yr, names, rows):
ax.text(R0 + 0.015, y, nm, fontsize=10, va="center", linespacing=1.15)
box(ax, cA - 0.06, y - 0.035, 0.12, 0.07, ORANGE_T, ORANGE, 1.2)
ax.text(cA, y, f"{r['a']} ({r['p_all']})", ha="center", va="center", fontsize=10.5,
fontweight="bold")
box(ax, cG - 0.045, y - 0.035, 0.09, 0.07, BLUE_T, BLUE, 1.2)
ax.text(cG, y, f"{r['p_g3']}%", ha="center", va="center", fontsize=10.5,
fontweight="bold")
ax.plot([R0 + 0.015, R1 - 0.015], [0.39, 0.39], color=GRID, lw=1, transform=ax.transAxes)
ax.text(R0 + 0.015, 0.37,
"Table 2A, any grade ≥3 TRAE:\n"
f" 0/21 · 1/15 ({rows[1]['p_ceiling']}%) · 6/41 ({rows[2]['p_ceiling']}%)\n"
"Results: monotherapy TRAEs “none\n were grade ≥3”; combo "
f"treatment-\n related fatigue 7/15 = {rows[1]['p_tall']}%",
fontsize=10, va="top", color=FG2, linespacing=1.3)
# connectors: abstract value -> all-grade cell
for y0, y1 in zip(ys, yr):
ax.annotate("", xy=(R0 - 0.004, y1), xytext=(L1 - 0.01, y0),
xycoords="axes fraction",
arrowprops=dict(arrowstyle="-|>", color=FG2, lw=1.4,
connectionstyle="arc3,rad=-0.15", shrinkA=0, shrinkB=2))
ax.text(0.445, 0.86, "same value,\ndifferent label:\n\u2018grade \u22653 TR\u2019\nvs \u2018all grades\u2019",
ha="center", va="top", fontsize=10, color=FG2, linespacing=1.3)
ax.text(0.5, 0.952, "Each abstract figure equals the all-grade, all-cause cell of Table 2B;\n"
"the grade \u22653 cells are 0, 0 and 1 patient.", ha="center", va="center",
fontsize=10, color=FG, fontweight="bold", linespacing=1.25)
# legend / key at bottom
box(ax, 0.025, 0.115, 0.03, 0.045, ORANGE_T, ORANGE, 1.2, r=0.006)
ax.text(0.065, 0.1375, "value printed in the paper (abstract figure and the Table 2B cell it equals)",
va="center", fontsize=10, color=FG2)
box(ax, 0.025, 0.045, 0.03, 0.045, BLUE_T, BLUE, 1.2, r=0.006)
ax.text(0.065, 0.0675, "grade ≥3 rate recomputed from Table 2B counts (n / N)",
va="center", fontsize=10, color=FG2)
return fig
save(draw_text_panels(), os.path.join(HERE, "fig1"), os.path.join(CAND, "cand1_text_panels"))
# ------------------------------------------------------ candidate 2 / fig2: dot plot
def draw_dots():
fig, ax = plt.subplots(figsize=(W, 3.5))
fig.subplots_adjust(left=0.30, right=0.97, top=0.80, bottom=0.17)
ylab = [f"{SHORT[r['k']]}\n{r['term']}" for r in rows]
yy = [2, 1, 0]
ax.set_xlim(-2, 100); ax.set_ylim(-0.6, 2.6)
ax.set_yticks(yy); ax.set_yticklabels(ylab, fontsize=10)
ax.set_xticks(range(0, 101, 20))
ax.set_xticklabels([f"{t}%" for t in range(0, 101, 20)])
ax.xaxis.grid(True, color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Patients in cohort (%)", color=FG2)
for y, r in zip(yy, rows):
ax.plot([r["p_g3"], r["rep"]], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(r["p_all"], y, "o", ms=15, mfc="none", mec=FG2, mew=1.5, zorder=2)
ax.plot(r["rep"], y, "o", ms=9, color=ORANGE, zorder=3)
ax.plot(r["p_ceiling"], y, "s", ms=9, mfc=BG, mec=BLUE, mew=2, zorder=3)
ax.plot(r["p_g3"], y, "D", ms=8, color=BLUE, zorder=4)
ax.text(r["rep"] + 2.2, y + 0.08, f"{r['rep']}%", color=FG, fontsize=10, va="bottom",
fontweight="bold")
g3txt = f"{r['p_g3']}%" if r["p_g3"] == r["p_ceiling"] else \
f"{r['p_g3']}% / {r['p_ceiling']}%"
ax.text(max(r["p_g3"], r["p_ceiling"]) + 2.2, y - 0.1, g3txt, color=FG, fontsize=10,
va="top")
h = [plt.Line2D([], [], ls="none", marker="o", ms=9, color=ORANGE),
plt.Line2D([], [], ls="none", marker="o", ms=13, mfc="none", mec=FG2, mew=1.5),
plt.Line2D([], [], ls="none", marker="D", ms=8, color=BLUE),
plt.Line2D([], [], ls="none", marker="s", ms=9, mfc=BG, mec=BLUE, mew=2)]
lab = ["Abstract (‘grade ≥3 TRAE’)",
"All grades, all causes (Table 2B)",
"Grade ≥3, this AE (Table 2B)",
"Any grade ≥3 TRAE (Table 2A)"]
fig.legend(h, lab, loc="upper center", ncol=2, frameon=False, fontsize=10,
bbox_to_anchor=(0.56, 1.0), handletextpad=0.4, columnspacing=1.2)
return fig
save(draw_dots(), os.path.join(HERE, "fig2"), os.path.join(CAND, "cand2_dot_plot"))
# ------------------------------------------------------ candidate 3: grouped bars
def draw_bars():
fig, ax = plt.subplots(figsize=(W, 4.6))
fig.subplots_adjust(left=0.36, right=0.95, top=0.95, bottom=0.11)
metrics = [("Abstract (‘grade ≥3 TR’)", "rep", ORANGE),
("All grades, all cause", "p_all", BLUE),
("All grades, treatment-related", "p_tall", BLUE),
("Grade ≥3, all cause", "p_g3", BLUE),
("Grade ≥3, treatment-related", "p_tg3", BLUE)]
ticks, labs = [], []
y = 0
for gi, r in enumerate(rows):
for name, key, col in metrics:
v = r[key]
ax.barh(y, v, height=0.72, color=col, zorder=2)
ax.text(v + 1.2, y, f"{v}%", va="center", fontsize=10, color=FG)
ticks.append(y); labs.append(name)
y -= 1
ax.text(-0.53, y + len(metrics) + 0.95, f"{SHORT[r['k']]}: {r['term'].lower()}",
transform=ax.get_yaxis_transform(), fontsize=10, fontweight="bold", va="center")
y -= 1.6
ax.set_yticks(ticks); ax.set_yticklabels(labs, fontsize=10)
ax.set_xlim(0, 100)
ax.set_xticks(range(0, 101, 20)); ax.set_xticklabels([f"{t}%" for t in range(0, 101, 20)])
ax.xaxis.grid(True, color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Patients in cohort (%). Orange: abstract figure; blue: Table 2B / Suppl. Table S5 counts ÷ N",
color=FG2, fontsize=10, loc="right")
return fig
save(draw_bars(), os.path.join(CAND, "cand3_grouped_bars"))
# ------------------------------------------------------- candidate 4: calc flow
def draw_flow():
fig = plt.figure(figsize=(W, 3.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
hdr = [(0.03, "Table 2B count"), (0.30, "÷ cohort N"), (0.53, "Result"),
(0.775, "Abstract label")]
for x, t in hdr:
ax.text(x, 0.94, t, fontsize=10, color=FG2, va="center", fontweight="bold")
ys = [0.74, 0.47, 0.20]
for y, r in zip(ys, rows):
term = r["term"].lower()
# all-grade path
box(ax, 0.03, y + 0.015, 0.24, 0.085, "#ffffff", GRID)
ax.text(0.04, y + 0.0575, f"{r['a']} pts, {term},\nany grade", fontsize=10, va="center")
ax.text(0.30, y + 0.0575, f"÷ {r['d']}", fontsize=10.5, va="center")
box(ax, 0.53, y + 0.015, 0.22, 0.085, ORANGE_T, ORANGE, 1.2)
ax.text(0.64, y + 0.0575, f"{r['p_all']}% (any grade)", fontsize=10, ha="center",
va="center")
ax.text(0.775, y + 0.0575, f"“grade ≥3 TR” {r['rep']}%\n= the any-grade rate",
fontsize=10, va="center", color=FG)
# grade >=3 path
box(ax, 0.03, y - 0.09, 0.24, 0.085, "#ffffff", GRID)
ax.text(0.04, y - 0.0475, f"{r['g3']} pt{'s' if r['g3'] != 1 else ''}, {term},\ngrade ≥3",
fontsize=10, va="center")
ax.text(0.30, y - 0.0475, f"÷ {r['d']}", fontsize=10.5, va="center")
box(ax, 0.53, y - 0.09, 0.22, 0.085, BLUE_T, BLUE, 1.2)
ax.text(0.64, y - 0.0475, f"{r['p_g3']}% (grade ≥3)", fontsize=10, ha="center",
va="center")
ax.text(0.775, y - 0.0475, "not reported\nin the abstract", fontsize=10, va="center",
color=FG2)
for yy in (y + 0.0575, y - 0.0475):
ax.annotate("", xy=(0.525, yy), xytext=(0.36, yy), xycoords="axes fraction",
arrowprops=dict(arrowstyle="-|>", color=FG2, lw=1.4))
ax.text(0.03, y + 0.125, f"{LABEL[r['k']]} (N = {r['d']})", fontsize=10,
fontweight="bold", va="center")
ax.text(0.03, 0.035, "Orange: rate printed in the paper. Blue: grade ≥3 rate recomputed "
"from the same table.", fontsize=10, color=FG2, va="center")
return fig
save(draw_flow(), os.path.join(CAND, "cand4_calc_flow"))
# ------------------------------------------------------- candidate 5: patient grid
def draw_grid():
fig = plt.figure(figsize=(W, 4.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 72); ax.set_ylim(0, 37)
s, gap = 1.25, 0.35
x0 = 32
y = 33.5
for r in rows:
ax.text(1, y + 0.6, f"{LABEL[r['k']]}, {r['term'].lower()} (N = {r['d']})",
fontsize=10, fontweight="bold", va="center")
y -= 3.2
for lab, cnt, col, colt in (
(f"Abstract (‘grade ≥3 TR’): {r['rep']}% → {round(r['need'])} pts", round(r["need"]), ORANGE,
ORANGE_T),
(f"Table 2B grade ≥3: {r['g3']} pt{'s' if r['g3'] != 1 else ''}", r["g3"], BLUE,
BLUE_T)):
ax.text(1, y + s / 2, lab, fontsize=10, va="center")
for i in range(r["d"]):
xi = x0 + i * (s + gap) if r["d"] <= 27 else x0 + (i % 21) * (s + gap)
yi = y if r["d"] <= 27 or i < 21 else y - (s + gap)
filled = i < cnt
ax.add_patch(Rectangle((xi, yi), s, s, fc=col if filled else BG,
ec=col if filled else "#bdbbb5", lw=0.8))
y -= 3.2 if r["d"] <= 27 else 4.8
y -= 0.6
ax.text(1, 1.2, "Each square is one patient. Filled: patients with the event at the rate "
"named on the left.", fontsize=10, color=FG2, va="center")
return fig
save(draw_grid(), os.path.join(CAND, "cand5_patient_grid"))
print("\nFigures written to", HERE, "and", CAND)
Table 1 crystallographic statistics do not match the AcrIIA19 structure described in the text and deposited
Commun Biol (2025) · doi:10.1038/s42003-025-09417-6 · PMC12855784
Summary
The paper's central result is a crystal structure of the anti-CRISPR protein AcrIIA19. The Results text describes it as a C222₁ crystal with four molecules in the asymmetric unit (ASU), solved at 1.98 Å, and refers the reader to Table 1 for the crystallographic statistics. Table 1 instead reports a tetragonal P4₁2₁2 cell (a = b = 64.43, c = 62.82 Å), data to 1.45 Å and one molecule in the ASU. These cannot describe the same crystal: four molecules per ASU in Table 1's cell would give a Matthews coefficient of 0.56 ų/Da, far below the physical minimum. The deposited entry the paper cites (PDB 9WA8) agrees with the text, not with Table 1, so the published data-collection and refinement statistics (resolution, R factors, completeness, B factors, atom counts, geometry) do not appear to belong to the structure the paper analyses.
What the paper reports
Results, "High-resolution structural analysis of AcrIIA19 reveals a previously uncharacterized structural fold", paragraph 1:
Following successful crystallization, the structure of AcrIIA19 was solved at a 1.98 Å resolution using MR, with the AlphaFold2 model used as a search template. Details on crystallographic and refinement statistics are listed in Table 1.
Same section, paragraph 2:
The crystal belonged to space group C2221 and contained four molecules (MolA to MolD) in the asymmetric unit (ASU; Fig. 1 C). The final structural model covered the complete AcrIIA19 (residues M1 to G115).
B-factor analysis showed relatively higher values at the N-terminus; however, the overall B-factor was low (average 48.8 Å), indicating that the structure was highly rigid (Fig. 1G).
The observation that all four molecules in the same ASU exhibited nearly identical structures served as additional indirect evidence supporting the high structural stability of AcrIIA19 (Fig. 1H).
Table 1, "Data collection and refinement statistics" (selected rows, as archived):
Space group | P 41 21 2
a, b, c (Å) | a = 64.43, b = 64.43, c = 62.82
Resolution range (Å) | 28.81–1.45
Total reflections | 616,092
Unique reflections | 24,054
Wilson B-factor (Å2) | 20.90
Rwork (%)a | 20.91 (27.03)
Rfree (%)a | 21.89 (25.83)
No. of molecules in the asymmetric unit | 1
No. of non-hydrogen atoms | 821
Macromolecules | 733
Average B-factor values (Åb) | 23.55
(Both resolution-range rows, under "Data collection" and under "Refinement", read 28.81–1.45. On the publisher's page the space group is typeset P 4₁ 2₁ 2.)
Figure 1 legend, panels C and H:
C Four molecules of AcrIIA19 in the same crystallographic asymmetry unit (ASU).
H Four superimposed AcrIIA19 structures in ASU.
Results, "AcrIIA19 forms a stable dimer in solution", paragraph 2:
The PISA complex formation score, which ranges from 0 to 1 with higher values indicating greater biological relevance, was 1.0 for both the AB and CD dimers.
Since the AB dimer and CD dimer represent essentially the same dimeric form (Fig. 2C, D), we selected the AB dimer as a representative structure to analyze in detail.
Data availability (back matter):
The coordinate and structural factors have been deposited in the Research Collaboratory for Structural Bioinformatics (RCSB) Protein Data Bank (PDB) under the PDB code 9WA8.
What the paper's own numbers show
Method. I compared the three crystallographic descriptors the text gives (space group, resolution, ASU content) with Table 1. I then tested whether each description is physically self-consistent. For each cell I computed the volume, and from it the Matthews coefficient Vm = V / (n_sym × n_mol × M), with n_sym = 8 general positions (true of both P4₁2₁2 and C222₁) and M = 14,545 Da, the chain mass of the deposited construct (residues 1–115 plus the LEHHHHHH tag). I counted the non-hydrogen atoms of a complete residue 1–115 chain from its sequence, and I enumerated the symmetry-unique reflections that Table 1's cell, space group and resolution range allow. Finally I compared both descriptions with the deposited entry PDB 9WA8 (RCSB metadata, coordinates and wwPDB validation report). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Space group | Results ¶2 vs Table 1 | C222₁ (text); P4₁2₁2 (Table 1) | Orthorhombic vs tetragonal: different crystal systems, not alternative settings. PDB 9WA8: C 2 2 2₁ |
| High-resolution limit | Results ¶1 vs Table 1 | 1.98 Å (text); 28.81–1.45 Å (Table 1, both rows) | Differ by 0.53 Å (26.8% of 1.98). PDB 9WA8: 28.95–1.98 Å |
| Molecules in the ASU | Results ¶2, Fig. 1C/H vs Table 1 | four (MolA to MolD); 1 | PDB 9WA8: 4 chains (A–D) |
| Cell | Table 1 | a = b = 64.43, c = 62.82 Å | Tetragonal metric (a = b), consistent with P4₁2₁2. V = 260,780 ų. PDB 9WA8 cell 63.41 × 98.98 × 196.66 Å, V = 1,234,301 ų (4.73× larger) |
| Matthews coefficient, Table 1 cell, 1 molecule per ASU | derived from Table 1 | — | 2.24 ų/Da (45% solvent): normal |
| Matthews coefficient, Table 1 cell, 4 molecules per ASU | derived from Table 1 + text | — | 0.56 ų/Da: impossible (protein crystals typically lie at about 1.7–3.5) |
| Matthews coefficient, deposited cell, 4 molecules per ASU | PDB 9WA8 | — | 2.65 ų/Da (54% solvent); the entry itself records 2.65 and 53.62% |
| Protein non-H atoms | Table 1 vs sequence | 733 | One complete chain M1–G115 has 946 (deposited chains C and D: 945 each), so 733 is 0.77 of one complete chain; four chains need about 3784. PDB 9WA8: 3845 |
| Unique reflections | Table 1 | 24,054 at 99.98% complete | Table 1's cell and space group allow 24,053 to 28.81–1.45 Å: consistent. The deposited cell would allow 109,549 over that range |
| Multiplicity | Table 1 | 25.6 | 616,092 / 24,054 = 25.6: consistent. PDB 9WA8 redundancy 13.2 |
| R-work / R-free | Table 1 vs PDB 9WA8 | 20.91 / 21.89% | PDB 9WA8: 23.88 / 27.5%; 42,609 reflections in refinement vs Table 1's 24,052 |
| Average B-factor | Results ¶2 vs Table 1 | 48.8 (text); 23.55, Wilson 20.90 (Table 1) | PDB 9WA8: mean atomic B 64.4, Wilson B 50.9 (validation report). The text's value is not reproduced by either source; it is far from Table 1's |
Table 1 is internally consistent: its cell is tetragonal as its space group requires, the reflection count matches the cell and resolution almost exactly, the multiplicity matches its totals, and the atom count fits a single chain. It therefore reads as a complete statistics table for a single-chain, 1.45 Å, P4₁2₁2 crystal. The Results text, the Figure 1 and 2 legends, the dimer analysis and the deposited entry all describe a different crystal: C222₁, 1.98 Å, four chains. Table 1's model is also not a complete 1–115 chain, which the text says the final model is.
Figure
Figure 1. The four descriptors on which the Results text and Table 1 disagree, as printed (orange). Each connector names the conflict. The blue lines are recomputed from Table 1's own cell: with the eight symmetry copies of P4₁2₁2 and a 14,545 Da chain, one molecule per ASU leaves a normal 2.24 ų/Da, while the text's four molecules would leave 0.56 ų/Da, which is impossible. The gray strip gives the deposited entry PDB 9WA8 that the paper's Data availability statement names; it matches the text on space group, resolution and ASU content.
Figure 2. Matthews coefficient (unit-cell volume per dalton of protein) for each combination of cell and ASU content, on a scale starting at zero. The shaded band is the typical range for protein crystals (about 1.7–3.5 ų/Da). Table 1's cell is plausible only with the one molecule Table 1 lists. The four molecules described in the text and in Figure 1C/H fit only the deposited C222₁ cell (2.65 ų/Da, the value recorded in the PDB entry).
Why it matters
The crystal structure is the paper's headline deliverable ("Here, we present the crystal structure of AcrIIA19" in the Abstract). Table 1 is where a reader, referee or re-user checks its quality: resolution, completeness, R-work/R-free, geometry, Ramachandran statistics and B factors. As published, those numbers appear to describe a different crystal. The quality indicators a reader would rely on for the 1.98 Å, four-chain structure are therefore not in the paper. Its deposited R-free (27.5%) is noticeably higher than Table 1's 21.89%, and the deposited 1.98 Å resolution is lower than Table 1's 1.45 Å.
The four-molecule ASU is not incidental. The near-identity of the four copies is offered as evidence of rigidity (Fig. 1H), and the dimer analysis (PISA scores for the AB and CD dimers, the AB interface, Fig. 2B–H) depends on four chains in the ASU. Those analyses are consistent with the deposited entry, so they are not themselves refuted; what is missing is the correct statistical description of the data behind them.
The main biological conclusions (a new fold, a homodimer in solution by SEC-MALS, WED-domain binding and inhibition of several Cas9 orthologs) rest largely on the deposited structure and on independent biochemical data, and are not contradicted by this problem.
Two smaller inconsistencies sit in the same section and do not change any conclusion. The Figure 2C legend says "The model shows two dimers formed by MolA/MolD and MolC/MolD", which names MolD twice, while the text analyses AB and CD dimers. The B-factor is given in Å in the text and "Åb" in Table 1, where Ų is meant.
Explanations considered
- Two crystal forms, one described in the text and one in the table. This does not account for the paper as written. The text says that the statistics for the structure it describes "are listed in Table 1"; Table 1 has a single data column and no label for a second form; and only one entry (9WA8) is deposited. A second crystal form would have to be described and deposited separately.
- Resolution range vs a single high-resolution limit. This does not account for the numbers. Table 1's high-resolution limit is 1.45 Å in both its data-collection and refinement rows, not 1.98 Å. The deposited entry's limit is 1.98 Å.
- "Molecules in the ASU" counted per unique chain or per protomer. This does not account for the numbers. The row is the literal ASU content, and Table 1's cell is too small to hold four chains per ASU (Vm 0.56 ų/Da). Its 733 protein atoms are fewer than one complete chain.
- Alternative space-group setting or notation. Not possible. C222₁ is orthorhombic and P4₁2₁2 is tetragonal; no change of setting converts one into the other, and the two cells differ in volume by a factor of 4.73.
- A typing error in one or two Table 1 fields. This is unlikely. The cell, space group, reflection count, multiplicity, atom count and ASU content agree with one another for a single-chain tetragonal crystal, so many fields would have to be wrong together, consistently.
- Conversion or typesetting artifact in PMC. Excluded. The publisher's article page and its separate Table 1 page carry the same values as the PMC archive. Each value sits in its own labelled cell, with no minus signs or decimal shifts involved.
- The text, not the table, is wrong. This does not fit the deposited entry. The PDB entry cited in Data availability (primary citation: this DOI) records C 2 2 2₁, 1.98 Å, four chains and a Matthews coefficient of 2.65, which agrees with the text and the figure legends.
The paper does not say what crystal Table 1 describes. The average B-factor in the text (48.8) matches neither Table 1 (23.55) nor the deposited coordinates (mean 64.4; Wilson estimate 50.9). That leaves its source unclear.
Status
- Version of record. The publisher's article page (nature.com/articles/s42003-025-09417-6) and its Table 1 page give the same values as the PMC archive: "space group C 222 1", "four molecules (MolA to MolD)", "1.98 Å", "average 48.8 Å" in the text; "P 4 1 2 1 2", a = b = 64.43, c = 62.82, 28.81–1.45, 1 molecule in the ASU, 733 macromolecule atoms in Table 1. Every quote above was checked verbatim against the PMC-archived text. The Data availability sentence is in the article's back matter.
- Deposited data. PDB 9WA8 (deposited 2025-08-11, released 2026-07-08; primary citation this DOI): space group C 2 2 21, cell 63.41 × 98.98 × 196.66 Å, 4 polymer chains, 28.95–1.98 Å, R-work 0.2388, R-free 0.275, 42,893 reflections, redundancy 13.2, 3904 atoms (3845 protein, 59 water), Matthews coefficient 2.65. Checked through the RCSB REST API, the mmCIF header and coordinates, and the wwPDB validation report (Wilson B 50.871).
- Supplements. All publisher supplementary files were checked: the Transparent Peer Review File (the reviewers do not raise the crystallographic statistics), the Supplementary Information (uncropped gels), the description of additional files and Supplementary Data 1 (assay data only). The Reporting Summary was listed but not opened. None contains crystallographic statistics.
- Corrections and comments. As of 2026-10-23, no correction, erratum, retraction or expression of concern was found. Crossref lists no update relations, PMC and Europe PMC list no notices, and the publisher's page shows none. The PubPeer API and search page both refused automated access (HTTP 403, Cloudflare), so PubPeer could not be checked directly. A web search found no PubPeer thread or correction.
- Sources. PMC article PMC12855784; https://www.nature.com/articles/s42003-025-09417-6 and its /tables/1 page; Europe PMC full-text XML; https://www.rcsb.org/structure/9wa8; https://data.rcsb.org/rest/v1/core/entry/9WA8; https://files.rcsb.org/download/9WA8.cif; the wwPDB validation report for 9WA8; Crossref.
Suggested correction
- Replace Table 1 with the data-collection and refinement statistics for the deposited 9WA8 dataset: space group C222₁, cell 63.41 × 98.98 × 196.66 Å, 28.95–1.98 Å, four molecules per ASU, and the matching reflection counts, completeness, multiplicity, I/σ(I), R-merge, R-work/R-free, atom counts, B factors and geometry. The values should match those in the PDB entry and its validation report.
- If the present Table 1 describes another AcrIIA19 crystal (or another protein) from the same work, identify it, state its relation to the structure analysed, and deposit it, or remove it.
- Reconcile the "average 48.8" B-factor in the Results with the corrected table, and give its unit as Ų.
- Minor: correct the Figure 2C legend ("MolA/MolD and MolC/MolD") to the AB and CD dimers analysed in the text.
Recomputation script (python recompute.py.txt)
"""Recompute the crystallographic consistency checks for PMC12855784
(Commun Biol 2025, doi:10.1038/s42003-025-09417-6).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file, plus five candidate
figure designs in ./candidates/ (cand1..cand5 .svg/.png).
Inputs are copied from the paper as archived in PMC (Results text and Table 1)
and from the deposited entry the paper's Data availability statement names,
PDB 9WA8 (RCSB entry metadata, mmCIF header, coordinates and wwPDB validation
report, retrieved 2026-10-23). No network access is needed: the deposited values
are written out below.
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
import matplotlib.ticker # noqa: E402,F401
from matplotlib.patches import FancyBboxPatch, Rectangle # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs: the paper
TEXT = { # Results, "High-resolution structural analysis of AcrIIA19 ...", paragraphs 1-2
"space_group": "C2221", "d_high": 1.98, "mol_asu": 4, "mean_B": 48.8,
"residues": (1, 115), # "The final structural model covered the complete AcrIIA19 (residues M1 to G115)"
}
TABLE1 = { # Table 1, "Data collection and refinement statistics"
"space_group": "P 41 21 2", "cell": (64.43, 64.43, 62.82), "angles": (90, 90, 90),
"d_low": 28.81, "d_high": 1.45, "total_refl": 616092, "unique_refl": 24054,
"multiplicity": 25.6, "completeness": 99.98, "wilson_B": 20.90,
"refl_refine": 24052, "Rwork": 20.91, "Rfree": 21.89, "mol_asu": 1,
"atoms_nonH": 821, "atoms_macro": 733, "atoms_ligand": 1, "atoms_solvent": 87,
"mean_B": 23.55, "mean_B_macro": 22.41,
}
# ---------------------------------------------------------------- inputs: deposited PDB 9WA8
PDB = {
"space_group": "C 2 2 21", "cell": (63.41, 98.98, 196.66), "Z_PDB": 32,
"d_low": 28.95, "d_high": 1.98, "unique_refl": 42893, "completeness": 97.92,
"redundancy": 13.2, "refl_refine": 42609, "Rwork": 23.88, "Rfree": 27.5,
"chains": 4, "atoms_total": 3904, "atoms_protein": 3845, "atoms_solvent": 59,
"matthews_deposited": 2.65, "solvent_pct_deposited": 53.62,
"chain_fw_Da": 14545.0, # entity 1 formula weight (residues 1-115 + LEHHHHHH tag)
"wilson_B_validation": 50.871,
"mean_B_coords": 64.36, # mean B_iso_or_equiv over all 3904 deposited atoms
"chain_atoms": {"A": 1002, "B": 953, "C": 945, "D": 945}, # protein atoms per chain
"chain_last_res": {"A": 121, "B": 116, "C": 115, "D": 115},
}
SEQ = ("MKLIINIEDKNYKYLTELAQQDNTNIGSIVNNLIQTHITDVNESYRSVDKKELDEFSRVMQHYFHEDLASMYDVIG"
"SDEELSTDKQMLKVYKKLYQDVALRNGIALELFNAYKKGLEHHHHHH") # PDB 9WA8 entity 1 sequence
# Number of general positions (symmetry operators incl. centring) per cell
NSYM = {"P41212": 8, "C2221": 8}
# Typical range of Matthews coefficients for protein crystals (about 1.7-3.5; Matthews 1968)
VM_RANGE = (1.7, 3.5)
def vol(c):
return c[0] * c[1] * c[2] # all angles are 90 degrees in both cells
def matthews(V, nsym, nmol, mw):
return V / (nsym * nmol * mw)
def solvent(vm):
return 100 * (1 - 1.230 / vm)
# ---------------------------------------------------------------- 1. direct comparison
print("=== 1. Results text vs Table 1 vs deposited PDB 9WA8 ===")
rows = [
("Space group", TEXT["space_group"], TABLE1["space_group"], PDB["space_group"]),
("High-resolution limit (A)", TEXT["d_high"], TABLE1["d_high"], PDB["d_high"]),
("Molecules in ASU", TEXT["mol_asu"], TABLE1["mol_asu"], PDB["chains"]),
("Average B (A^2)", TEXT["mean_B"], TABLE1["mean_B"], f"{PDB['mean_B_coords']} (coords), Wilson {PDB['wilson_B_validation']}"),
]
for r in rows:
print(f" {r[0]:28s} text={r[1]!s:8s} table1={r[2]!s:10s} PDB={r[3]}")
print(f" resolution difference text - table = {TEXT['d_high'] - TABLE1['d_high']:.2f} A "
f"({100 * (TEXT['d_high'] - TABLE1['d_high']) / TEXT['d_high']:.1f}% of 1.98)")
a, b, c = TABLE1["cell"]
print(f" Table 1 cell a=b ({a}={b}), c={c}: tetragonal metric, as P41212 requires; "
f"C2221 (orthorhombic, C-centred) needs no a=b, and PDB cell a/b = {PDB['cell'][0] / PDB['cell'][1]:.3f}")
# ---------------------------------------------------------------- 2. Matthews coefficients
print("\n=== 2. Cell volume and Matthews coefficient (chain MW = 14,545 Da, PDB entity 1) ===")
mw = PDB["chain_fw_Da"]
V1 = vol(TABLE1["cell"])
V2 = vol(PDB["cell"])
VM = {
"t1_1": matthews(V1, NSYM["P41212"], 1, mw),
"t1_4": matthews(V1, NSYM["P41212"], 4, mw),
"pdb_4": matthews(V2, NSYM["C2221"], 4, mw),
"pdb_1": matthews(V2, NSYM["C2221"], 1, mw),
}
print(f" Table 1 cell volume = {V1:,.0f} A^3")
print(f" PDB 9WA8 cell volume = {V2:,.0f} A^3 (ratio PDB/Table1 = {V2 / V1:.2f})")
print(f" Table 1 cell, P41212, 1 molecule/ASU (as Table 1) : Vm = {VM['t1_1']:.2f} A^3/Da, solvent {solvent(VM['t1_1']):.0f}%")
print(f" Table 1 cell, P41212, 4 molecules/ASU (as text) : Vm = {VM['t1_4']:.2f} A^3/Da, solvent {solvent(VM['t1_4']):.0f}%"
f" -> below the typical minimum {VM_RANGE[0]}")
print(f" PDB cell, C2221, 4 molecules/ASU (text, deposit): Vm = {VM['pdb_4']:.2f} A^3/Da, solvent {solvent(VM['pdb_4']):.0f}%"
f" (deposited value {PDB['matthews_deposited']}, {PDB['solvent_pct_deposited']}%)")
print(f" PDB cell, C2221, 1 molecule/ASU : Vm = {VM['pdb_1']:.2f} A^3/Da -> above the typical maximum {VM_RANGE[1]}")
print(f" molecules per cell: Table 1 = {NSYM['P41212'] * 1}, text = {NSYM['C2221'] * 4} (PDB Z = {PDB['Z_PDB']})")
# ---------------------------------------------------------------- 3. atom counts
print("\n=== 3. Non-hydrogen atom counts ===")
HEAVY = dict(G=4, A=5, S=6, C=6, V=7, T=7, P=7, I=8, L=8, D=8, N=8, M=8, E=9, Q=9, K=9,
H=10, F=11, R=11, Y=12, W=14)
seq115 = SEQ[:115]
assert seq115[0] == "M" and seq115[-1] == "G" # "residues M1 to G115"
full115 = sum(HEAVY[x] for x in seq115) + 1 # +1 for the C-terminal OXT
print(f" complete chain M1-G115 from sequence: {full115} non-H atoms (deposited chains C, D: "
f"{PDB['chain_atoms']['C']}, {PDB['chain_atoms']['D']})")
print(f" Table 1 macromolecule atoms = {TABLE1['atoms_macro']} = {TABLE1['atoms_macro'] / full115:.2f} x one complete chain; "
f"four complete chains = {4 * full115}")
print(f" PDB 9WA8 protein atoms = {PDB['atoms_protein']} = {PDB['atoms_protein'] / TABLE1['atoms_macro']:.2f} x Table 1")
print(f" Table 1 sum check: {TABLE1['atoms_macro']} + {TABLE1['atoms_ligand']} + {TABLE1['atoms_solvent']} = "
f"{TABLE1['atoms_macro'] + TABLE1['atoms_ligand'] + TABLE1['atoms_solvent']} (reported {TABLE1['atoms_nonH']})")
# ---------------------------------------------------------------- 4. reflection counts
print("\n=== 4. Unique reflections expected from each cell and resolution range ===")
def n_unique(cell, dlo, dhi, sg):
a, b, c = cell
H = int(a / dhi) + 1
K = int(b / dhi) + 1
L = int(c / dhi) + 1
h, k, l = np.meshgrid(np.arange(0, H + 1), np.arange(0, K + 1), np.arange(0, L + 1), indexing="ij")
s2 = (h / a) ** 2 + (k / b) ** 2 + (l / c) ** 2 # 1/d^2, orthogonal cell
ok = (s2 <= 1 / dhi ** 2) & (s2 >= 1 / dlo ** 2) & ~((h == 0) & (k == 0) & (l == 0))
if sg == "P41212": # Laue 4/mmm asymmetric unit h>=k>=0, l>=0; absences 00l l=4n, h00 h=2n
ok &= h >= k
ok &= ~((h == 0) & (k == 0) & (l % 4 != 0))
ok &= ~((k == 0) & (l == 0) & (h % 2 != 0))
elif sg == "C2221": # Laue mmm asymmetric unit h,k,l>=0; C-centring h+k=2n; 00l l=2n
ok &= (h + k) % 2 == 0
ok &= ~((h == 0) & (k == 0) & (l % 2 != 0))
return int(ok.sum())
N1 = n_unique(TABLE1["cell"], TABLE1["d_low"], TABLE1["d_high"], "P41212")
N2 = n_unique(PDB["cell"], PDB["d_low"], PDB["d_high"], "C2221")
N1x = n_unique(PDB["cell"], TABLE1["d_low"], TABLE1["d_high"], "C2221")
print(f" Table 1 cell, P41212, 28.81-1.45 A: {N1:,} possible; reported {TABLE1['unique_refl']:,} "
f"at {TABLE1['completeness']}% -> implies {TABLE1['unique_refl'] / (TABLE1['completeness'] / 100):,.0f}")
print(f" PDB cell, C2221, 28.95-1.98 A: {N2:,} possible; deposited {PDB['unique_refl']:,} "
f"at {PDB['completeness']}% -> implies {PDB['unique_refl'] / (PDB['completeness'] / 100):,.0f}")
print(f" PDB cell, C2221, if it had been 28.81-1.45 A: {N1x:,} possible (Table 1 reports {TABLE1['unique_refl']:,})")
print(f" multiplicity check Table 1: {TABLE1['total_refl']:,} / {TABLE1['unique_refl']:,} = "
f"{TABLE1['total_refl'] / TABLE1['unique_refl']:.1f} (reported {TABLE1['multiplicity']})")
# ---------------------------------------------------------------- 5. ratios to the deposit
print("\n=== 5. Table 1 and text values relative to the deposited entry ===")
RATIOS = [ # label, text value, Table 1 value, PDB value
("High-resolution limit", TEXT["d_high"], TABLE1["d_high"], PDB["d_high"]),
("Molecules in ASU", TEXT["mol_asu"], TABLE1["mol_asu"], PDB["chains"]),
("Unit-cell volume", None, V1, V2),
("Unique reflections", None, TABLE1["unique_refl"], PDB["unique_refl"]),
("Protein non-H atoms", None, TABLE1["atoms_macro"], PDB["atoms_protein"]),
("Multiplicity", None, TABLE1["multiplicity"], PDB["redundancy"]),
("R-free (%)", None, TABLE1["Rfree"], PDB["Rfree"]),
]
for lab, t, tb, p in RATIOS:
ts = f"text/PDB={t / p:.2f}" if t is not None else "text: not stated"
print(f" {lab:22s} Table1/PDB = {tb / p:.2f} {ts}")
print(f" Rwork Table 1 {TABLE1['Rwork']}% vs PDB {PDB['Rwork']}%; reflections in refinement {TABLE1['refl_refine']:,} vs {PDB['refl_refine']:,}")
print(f" Average B: text {TEXT['mean_B']}; Table 1 {TABLE1['mean_B']} (Wilson {TABLE1['wilson_B']}); "
f"PDB coordinates mean {PDB['mean_B_coords']}, validation Wilson {PDB['wilson_B_validation']}")
# ================================================================= figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 1.5,
})
def save(fig, path_stem):
fig.savefig(path_stem + ".svg")
fig.savefig(path_stem + ".png", dpi=200)
plt.close(fig)
SG_T = "C222₁" # C2221
SG_TB = "P4₁2₁2" # P41212
# ---------- design 1: two text panels joined by annotated connectors (+ deposit strip)
def design_contradiction(stem):
fig = plt.figure(figsize=(7.2, 5.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 78)
ax.axis("off")
PT, PB = 67, 25 # panel top / bottom
for x0, title, sub in [(2, "Results text", "High-resolution structural\nanalysis, paragraphs 1\u20132"),
(62, "Table 1", "Data collection and\nrefinement statistics")]:
ax.add_patch(FancyBboxPatch((x0, PB), 36, PT - PB, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec=SEC, lw=1.0))
ax.text(x0 + 1.5, PT - 3, title, fontsize=12, fontweight="bold", va="center")
ax.text(x0 + 1.5, PT - 8.3, sub, fontsize=10, color=SEC, va="center", linespacing=1.15)
left = [("space group", SG_T), ("solved at", "1.98 \u00c5 resolution"),
("asymmetric unit", "four molecules (MolA to MolD)"), ("overall B-factor", "average 48.8")]
right = [("Space group", SG_TB), ("Resolution range", "28.81\u20131.45 \u00c5"),
("Molecules in the ASU", "1"), ("Average B-factor", "23.55")]
conflict = ["orthorhombic vs\ntetragonal", "1.98 vs 1.45 \u00c5", "4 vs 1", "48.8 vs 23.55"]
ys = [51.5, 43.5, 35.5, 28.5]
for y, (l1, v1), (l2, v2), cf in zip(ys, left, right, conflict):
ax.text(3.5, y + 1.6, l1, fontsize=10, color=SEC, va="center")
ax.text(3.5, y - 1.3, v1, fontsize=10.5, color=ORANGE, fontweight="bold", va="center")
ax.text(63.5, y + 1.6, l2, fontsize=10, color=SEC, va="center")
ax.text(63.5, y - 1.3, v2, fontsize=10.5, color=ORANGE, fontweight="bold", va="center")
ax.annotate("", xy=(61.0, y), xytext=(39.0, y),
arrowprops=dict(arrowstyle="<->", color=TXT, lw=1.5, shrinkA=0, shrinkB=0))
ax.text(50, y + 0.9, cf, ha="center", va="bottom", fontsize=10, color=TXT, linespacing=1.05)
ax.text(50, PT - 3, "conflict", ha="center", va="center", fontsize=11, fontweight="bold", color=TXT)
# recomputed strip
ax.text(2.5, 20.5, "Recomputed from Table 1's own cell (8 symmetry copies per cell, 14,545 Da chain):",
fontsize=10, color=BLUE, va="center")
ax.text(2.5, 17.2, f"1 molecule per ASU (as Table 1 states): Vm = {VM['t1_1']:.2f} \u00c5\u00b3/Da, a normal value",
fontsize=10, color=BLUE, va="center", fontweight="bold")
ax.text(2.5, 13.9, f"4 molecules per ASU (as the text states): Vm = {VM['t1_4']:.2f} \u00c5\u00b3/Da, physically impossible",
fontsize=10, color=BLUE, va="center", fontweight="bold")
# deposit strip
ax.add_patch(FancyBboxPatch((2, 1.6), 96, 7, boxstyle="round,pad=0.3,rounding_size=1.0",
fc="#f1f0ec", ec="none"))
ax.text(3.5, 6.6, "Deposited entry PDB 9WA8 (named in Data availability) matches the text:",
fontsize=10, color=SEC, va="center")
ax.text(3.5, 3.5, f"{SG_T} \u00b7 1.98 \u00c5 \u00b7 4 chains \u00b7 cell 63.41 \u00d7 98.98 \u00d7 196.66 \u00c5"
f" \u00b7 Vm {VM['pdb_4']:.2f} \u00c5\u00b3/Da", fontsize=10, color=SEC, va="center")
# legend
for x, col, lab in [(2.5, ORANGE, "printed in the paper"), (36, BLUE, "recomputed from Table 1"),
(71, SEC, "deposited entry")]:
ax.add_patch(Rectangle((x, 73.2), 2.2, 2.6, fc=col, ec="none"))
ax.text(x + 3.2, 74.5, lab, fontsize=10, color=TXT, va="center")
save(fig, stem)
# ---------- design 2: scorecard matrix
def design_matrix(stem):
rows = [
("Space group", SG_T, SG_TB, SG_T),
("Crystal system", "orthorhombic", "tetragonal", "orthorhombic"),
("Cell a, b, c (Å)", "not stated", "64.43, 64.43, 62.82", "63.41, 98.98, 196.66"),
("Resolution limit (\u00c5)", "1.98", "1.45", "1.98"),
("Molecules in ASU", "4 (MolA–MolD)", "1", "4 chains"),
("Average B-factor", "48.8", "23.55", "64.4 (Wilson 50.9)"),
("Protein non-H atoms", "not stated", "733", "3845"),
("Unique reflections", "not stated", "24,054", "42,893"),
("R-work / R-free (%)", "not stated", "20.91 / 21.89", "23.9 / 27.5"),
]
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(-0.4, len(rows) + 2.0)
ax.axis("off")
xs = [2, 27, 44, 66.5]
heads = ["", "Results text", "Table 1", "PDB 9WA8"]
top = len(rows) + 1.2
for x, h in zip(xs, heads):
ax.text(x, top, h, fontsize=11, fontweight="bold", va="center")
ax.plot([2, 98], [top - 0.5, top - 0.5], color=SEC, lw=1)
for i, (lab, t, tb, p) in enumerate(rows):
y = len(rows) - i
if i % 2 == 0:
ax.add_patch(Rectangle((1, y - 0.48), 98, 0.96, fc="#f1f0ec", ec="none"))
ax.text(xs[0], y, lab, fontsize=10, va="center", color=TXT)
for x, v, col in [(xs[1], t, ORANGE), (xs[2], tb, ORANGE), (xs[3], p, SEC)]:
ns = v == "not stated"
ax.text(x, y, v, fontsize=10, va="center", color=SEC if ns else col,
fontweight="normal" if (ns or col == SEC) else "bold", style="italic" if ns else "normal")
diff_tb = tb.split(" ")[0] != p.split(" ")[0]
mark = "✕ differs" if diff_tb else "= same"
ax.text(98.5, y, mark, fontsize=10, va="center", ha="right", color=TXT if diff_tb else SEC)
ax.text(98.5, top, "Table 1 vs\ndeposit", fontsize=10, ha="right", va="center", color=SEC, linespacing=1.05)
ax.text(2, 0.1, "Orange: printed in the paper. Gray: deposited entry PDB 9WA8 (named in Data availability).",
fontsize=10, color=SEC, va="center")
save(fig, stem)
# ---------- design 3: unit cells to scale
def design_cells(stem):
fig, axs = plt.subplots(1, 2, figsize=(7.2, 5.0), gridspec_kw=dict(width_ratios=[1, 1], wspace=0.3))
a1, b1, c1 = TABLE1["cell"]
a2, b2, c2 = PDB["cell"]
for ax, (w1, h1, w2, h2, xl, yl, ttl) in zip(axs, [
(a1, b1, a2, b2, "a (Å)", "b (Å)", "a × b face"),
(a1, c1, a2, c2, "a (Å)", "c (Å)", "a × c face")]):
ax.add_patch(Rectangle((0, 0), w2, h2, fc="none", ec=SEC, lw=2))
ax.add_patch(Rectangle((0, 0), w1, h1, fc="none", ec=ORANGE, lw=2, ls=(0, (4, 2))))
ax.set_xlim(0, 105)
ax.set_ylim(0, 205 if "c face" in ttl else 105)
ax.set_aspect("equal")
ax.set_xlabel(xl)
ax.set_ylabel(yl)
ax.set_title(ttl, fontsize=11, loc="left")
ax.grid(color=GRID, lw=0.6)
ax.set_axisbelow(True)
ax.tick_params(labelsize=10)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
axs[0].text(4, 32, "Table 1\n64.43 \u00d7 64.43", color=TXT, fontsize=10, va="center")
axs[0].text(4, 82, "PDB 9WA8\n63.41 \u00d7 98.98", color=SEC, fontsize=10, va="center")
axs[1].text(4, 31, "Table 1\n64.43 \u00d7\n62.82", color=TXT, fontsize=10, va="center")
axs[1].text(4, 130, "PDB 9WA8\n63.41 \u00d7\n196.66", color=SEC, fontsize=10, va="center")
fig.legend(handles=[Line2D([], [], color=ORANGE, lw=2, ls=(0, (4, 2)), label=f"Table 1 cell ({SG_TB}), V = {V1/1e3:,.0f} × 10³ ų"),
Line2D([], [], color=SEC, lw=2, label=f"Deposited 9WA8 cell ({SG_T}), V = {V2/1e3:,.0f} × 10³ ų")],
loc="upper left", bbox_to_anchor=(0.06, 0.995), frameon=False, fontsize=10, ncol=1)
fig.text(0.06, 0.065, f"Both space groups have 8 symmetry copies per cell; the deposited cell is {V2 / V1:.2f}\u00d7 larger.\n"
f"Recomputed from Table 1's cell: Vm = {VM['t1_1']:.2f} \u00c5\u00b3/Da for 1 molecule per ASU (normal),\n"
f"{VM['t1_4']:.2f} \u00c5\u00b3/Da for 4 molecules per ASU (physically impossible).",
fontsize=10, color=BLUE, va="center", linespacing=1.2)
fig.subplots_adjust(left=0.1, right=0.98, top=0.82, bottom=0.24)
save(fig, stem)
# ---------- design 4: Matthews coefficient dot plot
def design_matthews(stem):
fig, ax = plt.subplots(figsize=(7.2, 3.6))
items = [
("Table 1 cell, P4\u20812\u20812, 1 molecule per ASU (as Table 1 states)", VM["t1_1"], BLUE, "o"),
("Table 1 cell, P4\u20812\u20812, 4 molecules per ASU (as the text states)", VM["t1_4"], BLUE, "o"),
("Deposited 9WA8 cell, C222\u2081, 4 molecules per ASU (text and deposit)", VM["pdb_4"], SEC, "D"),
]
ax.axvspan(*VM_RANGE, color="#eceae4", zorder=0)
ax.text(VM_RANGE[1] - 0.05, 3.05, f"typical range for protein\ncrystals, about {VM_RANGE[0]}\u2013{VM_RANGE[1]}",
ha="right", va="top", fontsize=10, color=SEC, linespacing=1.1)
for i, (lab, v, col, m) in enumerate(items):
y = 2 - i
ax.text(0.02, y + 0.3, lab, fontsize=10, color=TXT, va="center", ha="left")
ax.plot([0, v], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(v, y, marker=m, ms=10, color=col, mec=BG, mew=1.5, ls="none", zorder=3)
ax.text(v + 0.09, y, f"{v:.2f}", va="center", fontsize=10.5, color=TXT, fontweight="bold")
ax.set_yticks([])
ax.set_xlim(0, 4.6)
ax.set_ylim(-0.4, 3.1)
ax.set_xlabel("Matthews coefficient Vm (\u00c5\u00b3/Da): cell volume \u00f7 (8 \u00d7 molecules per ASU \u00d7 14,545 Da)")
ax.grid(axis="x", color=GRID, lw=0.6)
ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
fig.legend(handles=[Line2D([], [], marker="o", color=BLUE, ls="none", ms=9, label="recomputed from Table 1's cell"),
Line2D([], [], marker="D", color=SEC, ls="none", ms=9, label="computed from the deposited cell")],
loc="lower center", bbox_to_anchor=(0.5, 0.0), frameon=False, fontsize=10, ncol=2)
fig.tight_layout(rect=(0, 0.07, 1, 1))
save(fig, stem)
# ---------- design 5: ratio-to-deposit dot plot (log scale)
def design_ratios(stem):
fig, ax = plt.subplots(figsize=(7.2, 4.2))
n = len(RATIOS)
for i, (lab, t, tb, p) in enumerate(RATIOS):
y = n - 1 - i
ax.plot(tb / p, y, "o", ms=10, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.text(tb / p / 1.07, y, f"{tb / p:.2f}", fontsize=10, ha="right", va="center", color=TXT)
if t is not None:
ax.plot(t / p, y, "o", ms=10, mfc=BG, mec=ORANGE, mew=2, zorder=4)
ax.axvline(1, color=SEC, lw=1)
ax.text(1.03, -0.45, "same as deposit", fontsize=10, color=SEC, va="center")
ax.set_xscale("log")
ax.set_xlim(0.1, 2.5)
ax.set_xticks([0.1, 0.2, 0.5, 1, 2])
ax.set_xticklabels(["0.1", "0.2", "0.5", "1", "2"])
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_yticks(range(n))
ax.set_yticklabels([r[0] for r in RATIOS][::-1], fontsize=10)
ax.set_ylim(-0.6, n - 0.1)
ax.set_xlabel("Value in the paper ÷ value in deposited PDB 9WA8 (log scale)")
ax.grid(axis="x", color=GRID, lw=0.6)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], marker="o", color=ORANGE, ls="none", ms=9, label="Table 1 (reported)"),
Line2D([], [], marker="o", mfc=BG, mec=ORANGE, mew=2, ls="none", ms=9,
label="Results text (reported)")],
loc="lower left", bbox_to_anchor=(0.0, 1.0), frameon=False, fontsize=10, ncol=2)
fig.tight_layout()
save(fig, stem)
design_contradiction(os.path.join(HERE, "fig1"))
design_matthews(os.path.join(HERE, "fig2"))
design_contradiction(os.path.join(CAND, "cand1_contradiction_panels"))
design_matrix(os.path.join(CAND, "cand2_scorecard"))
design_cells(os.path.join(CAND, "cand3_unit_cells_to_scale"))
design_matthews(os.path.join(CAND, "cand4_matthews"))
design_ratios(os.path.join(CAND, "cand5_ratio_to_deposit"))
print("\nwrote fig1, fig2 and candidates/cand1..cand5 (.svg, .png)")
Sahel-wide foliage increases of 6% and 20% do not match the relative increases implied by the paper's own totals
Commun Biol (2019) · doi:10.1038/s42003-019-0383-9 · PMC6478729
Summary
The Abstract and Results report that satellite (VOD) estimates show Sahel-wide increases of 6% in total herbaceous mass (AGH) and 20% in total woody foliage mass (WPF) between 1992 and 2012. The same Results sentence gives the totals behind these percentages: AGH 0.28 Pg in 1992 and 0.31 Pg in 2012, WPF 0.16 Pg and 0.24 Pg. Those totals give relative increases of 10.7% and 50.0%. Allowing for rounding of the two-decimal totals, the possible ranges are 7.0–14.5% and 42.4–58.1%, so neither reported figure can be reached as an increase over each component's own 1992 value. Two denominators the paper does not state (1992 total vegetation mass, or the sum of the two years) can reach both figures within rounding of the printed totals, but the unrounded Supplementary Data does not reproduce the 20% WPF figure on either (see "Explanations considered"). The totals agree with the rest of the paragraph and with the paper's Supplementary Data, which points to the two percentages as the inconsistent figures. They are the only Sahel-wide figures in the Abstract and the paper's only Sahel-wide relative (percentage) increases. On the printed totals, the woody increase is understated about 2.5-fold (2.1–2.9-fold over rounding; 2.3-fold from the Supplementary Data endpoints) and the herbaceous increase about 1.8-fold (1.2–2.4-fold over rounding; 1.46-fold from the Supplementary Data endpoints). The direction of the main finding, that woody foliage rose faster than herbaceous mass, is unchanged.
What the paper reports
Abstract:
Satellite observations confirm these findings for the majority of the Sahel, with total herbaceous/woody foliage mass increases by 6%/20%.
Results, "Scaling field-site observations to the Sahel", paragraph 1 (the three sentences are consecutive):
For the entire western and central Sahel area, the summed AGH increased by 0.0025 Pg yr-1 over the 21-year period, and WPF by 0.004 Pg yr-1. Total AGH (WPF) was 0.28 Pg (0.16 Pg) in 1992 and 0.31 Pg (0.24 Pg) in 2012, which is a relative increase of 6% for AGH and 20% for WPF. The share of VOD estimated WPF (as related to the total vegetation mass) increased from 35% in 1992 to 43% in 2012, which is comparable to the percentage numbers observed from the Ferlo field samples.
The same paragraph, earlier:
According to fitted trends derived from VOD metrics from 1992 to 2012, AGH increased by + 5 ± 7 kg ha-1 yr-1 and WPF increased by +9 ± 8 kg ha-1 yr-1.
and later, giving the size of the significant-trend areas:
Areas with a significant (P < 0.05) increase in AGH cover a smaller region (289,375 km²; ~7%) than areas with a significant increase in WPF (2,711,875 km²; ~64%)
Discussion, paragraph 5, which builds on the composition of the increase:
However, a considerable portion of the observed increase in vegetation production consists of woody foliage mass, the fodder value of which is generally lower than that of herbaceous plants and would benefit primarily browsers (goats) over grazers (sheep and cattle).
What the paper's own numbers show
Method. The Results sentence defines the percentages as a "relative increase" computed from the stated 1992 and 2012 totals, so I took (2012 − 1992) / 1992 for each component. The totals are printed to two decimals, so I also computed the smallest and largest increase possible when each total varies by ±0.005 Pg. I then checked the totals against the WPF share given in the next sentence. Next I tried every other basis that could plausibly give 6% and 20%: the stated fitted trends, the per-hectare trends, other denominators, and the yearly Sahel series in the paper's Supplementary Data (file 42003_2019_383_MOESM4_ESM.csv, columns "Agh sahel" and "Wpf sahel", kg/ha, 1992–2012). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| AGH relative increase, 1992→2012 | Results p1; Abstract | 6% | (0.31 − 0.28) / 0.28 = 10.7%; 7.0–14.5% over rounding |
| WPF relative increase, 1992→2012 | Results p1; Abstract | 20% | (0.24 − 0.16) / 0.16 = 50.0%; 42.4–58.1% over rounding |
| Recomputed / reported | derived | — | Printed totals: AGH 1.79×, WPF 2.50×; over rounding AGH 1.17–2.42×, WPF 2.12–2.90×; Supplementary Data endpoints AGH 1.46×, WPF 2.33× |
| What the reported % would mean in Pg | derived | 6% of 0.28; 20% of 0.16 | 0.017 Pg and 0.032 Pg, against stated differences of 0.03 and 0.08 Pg |
| WPF share of total mass, 1992 | Results p1 | 35% | 0.16 / 0.44 = 36.4% (consistent) |
| WPF share of total mass, 2012 | Results p1 | 43% | 0.24 / 0.55 = 43.6% (consistent) |
| Summed trend × 20 years | Results p1 | AGH 0.0025, WPF 0.004 Pg/yr | AGH 0.050 Pg = 17.9% of 0.28; WPF 0.080 Pg = 50.0% of 0.16 (18.7% / 52.5% over 21 years) |
| Per-hectare trend × 20 years / 1992 per-hectare mass | Results p1 | 5 and 9 kg/ha/yr | AGH 14.9%; WPF 47.1% (study area of about 4.19 million km² implied by 289,375 km² ≈ 7% and 2,711,875 km² ≈ 64%) |
| Pg/yr trend converted to kg/ha/yr with that area | Results p1 | 5 and 9 kg/ha/yr | 5.97 and 9.56 kg/ha/yr (roughly consistent) |
| Supplementary Data, endpoints | MOESM4 csv | — | AGH 595.34 → 647.31 kg/ha (+8.7%); WPF 337.62 → 494.97 kg/ha (+46.6%) |
| Supplementary Data, fitted OLS line 1992→2012 | MOESM4 csv | slopes 5 and 9 kg/ha/yr | slopes 5.41 and 8.68 kg/ha/yr; AGH +17.5%, WPF +51.8% |
| Supplementary Data scaled to Pg (area implied by 0.44 Pg in 1992, 4.72 × 10⁸ ha) | MOESM4 csv | 0.28→0.31 and 0.16→0.24 Pg; 0.0025 and 0.004 Pg/yr | 0.281→0.305 and 0.159→0.233 Pg; 0.0026 and 0.0041 Pg/yr |
| Supplementary Data, WPF share | MOESM4 csv | 35% → 43% | 36.2% → 43.3% |
The Pg totals, the trends, the shares and the supplementary series agree with one another. Only the two percentages do not. On the paper's totals, total vegetation (AGH + WPF) rose from 0.44 to 0.55 Pg, or 25.0%. The woody-to-herbaceous contrast in relative increase is 50.0 / 10.7 = 4.7× on the printed totals (5.3× from the Supplementary Data endpoints), compared with 20 / 6 = 3.3× as reported.
Figure
Figure 1. Relative increase from 1992 to 2012 for each component, on one axis starting at zero. Orange dots are the percentages reported in the Abstract and Results (6% for AGH, 20% for WPF). Blue dots are the increases recomputed from the totals stated in the same sentence (10.7% and 50.0%). The blue bars show every value the increase could take if each two-decimal total is anywhere within ±0.005 Pg of its printed value. Both reported values lie outside these ranges, by 1.0 percentage point for AGH and 22.4 points for WPF.
Figure 2. Every basis tried for the two percentages. Each panel has its own scale, starting at zero. The orange line marks the reported value. Filled blue dots show each component's change measured against its own 1992 value: from the stated totals, from the Supplementary Data endpoints and fitted line, and from the stated Pg/yr and kg/ha/yr trends over 20 years. For the kg/ha/yr row the denominator is the 1992 per-hectare mass, 0.28 or 0.16 Pg spread over the study area of about 4.19 × 10⁸ ha implied by the significant-trend areas (669 and 382 kg/ha). The closest are 8.7% for AGH and 46.6% for WPF. Hollow dots use other denominators, none of which the paper states. Only two land near both reported values. Dividing by the sum of the two years gives 5.1% and 20.0%. Dividing by the 1992 total vegetation mass gives 6.8% and 18.2%. These are discussed under "Explanations considered".
Why it matters
The 6%/20% figures are the only Sahel-wide figures in the Abstract and the paper's only Sahel-wide relative (percentage) increases for the satellite analysis; the Results also give the Sahel-wide trends in kg/ha/yr and Pg/yr, which are not affected. The Abstract uses them as the quantitative evidence that "Satellite observations confirm these findings for the majority of the Sahel". A reader who cites the Abstract would take the woody foliage increase to be 20% when the paper's own totals give 50%, and the herbaceous increase to be 6% when they give about 11%.
The errors do not reverse anything. Both components still increase, and woody foliage still increases faster than herbaceous mass. With the recomputed values the contrast is larger (4.7× on the printed totals and 5.3× from the Supplementary Data endpoints, rather than 3.3×), so the paper's qualitative conclusion is, if anything, stronger. The per-hectare trends (field data: +17 vs +3 kg/ha/yr; VOD: +9 vs +5 kg/ha/yr), the 35%→43% WPF share, the rainfall decomposition and the significance tests do not depend on the two percentages and are not affected. The Discussion's statement that "a considerable portion of the observed increase in vegetation production consists of woody foliage mass" is not wrong. On the stated totals the woody share of the total increase is 0.08 of 0.11 Pg, about 73%, which the 6%/20% figures do not convey.
So this is a misstatement of magnitude in a headline number, not a change in direction. The correct values cannot be read off anywhere in the paper and have to be recomputed.
Explanations considered
- Rounding of the printed totals. This does not account for the numbers as stated, that is, as each component's increase over its own 1992 value. With every total free to vary by ±0.005 Pg, AGH can range only from 7.0% to 14.5% and WPF from 42.4% to 58.1% (Figure 1).
- Percentages from the fitted trend rather than the endpoints. This does not account for the numbers. The stated summed trends over 20 years give 17.9% and 50.0% (18.7% and 52.5% over 21 years). The per-hectare trends give 14.9% and 47.1%. The supplementary series' fitted line gives 17.5% and 51.8%. All are further from 6% and 20% than the endpoint values.
- The Pg totals, not the percentages, are misprinted. This is unlikely. The totals reproduce the 35% and 43% WPF shares stated in the next sentence (36.4% and 43.6%). Scaled from the Supplementary Data, they come to 0.281/0.305 Pg and 0.159/0.233 Pg, and the trend of that series matches the stated 0.0025 and 0.004 Pg/yr. The supplementary series itself gives increases of 8.7% and 46.6%.
- Increase relative to the 2012 value or to the mean of the two years. This does not account for the numbers: 9.7%/33.3% and 10.2%/40.0%.
- Each component's increase relative to total vegetation mass in 1992 (AGH + WPF = 0.44 Pg). This is the strongest alternative. From the printed totals it gives 6.8% and 18.2%, and over rounding of the totals the ranges are 4.4–9.3% for AGH and 15.6–20.9% for WPF, so on the printed totals both reported values can be reached on this basis. The case against it rests on three points. The paper calls the figures "a relative increase of 6% for AGH and 20% for WPF" and, in the Abstract, increases in "total herbaceous/woody foliage mass", not changes measured against total vegetation. The basis is not stated anywhere in the paper. And the unrounded Supplementary Data gives 5.6% for AGH, which rounds to 6%, but 16.9% for WPF, which rounds to 17%. The Supplementary Data therefore reproduces the AGH figure but not the WPF figure.
- Difference divided by the sum of the two years. From the printed totals this gives 5.1% and 20.0%, and over rounding the ranges are 3.4–6.8% for AGH and 17.5–22.5% for WPF, so both reported values can also be reached on this basis. This is not a relative increase in the usual sense: it equals half the change relative to the two-year mean. It is not stated in the paper. From the unrounded Supplementary Data it gives 4.2% and 18.9%, which would round to 4% and 19%, so the Supplementary Data reproduces neither figure on this basis.
- Change in each component's share of total vegetation. The WPF share rises from 36.4% to 43.6%, a relative change of 20.0%. The AGH share falls by 11.4%, though, so this basis cannot give a 6% increase for AGH.
- Conversion or typesetting artifact. Excluded. The publisher PDF, the publisher HTML, the PMC-archived text and the Europe PMC XML all print the same plain digits ("0.28", "0.31", "0.16", "0.24", "6%", "20%"). The passages contain no minus signs, superscripts or table cells that conversion could have altered.
No basis that the paper states, or that measures each component against its own baseline, reproduces 6% and 20%. Two unstated denominators (the 1992 total vegetation mass and the sum of the two years) can reach both values within rounding of the printed totals. Neither is described in the paper, and on the unrounded Supplementary Data the first reproduces the AGH figure but not the WPF figure, while the second reproduces neither. The published material does not show how the percentages were obtained.
Status
- Version of record. I checked the publisher PDF (nature.com/articles/s42003-019-0383-9.pdf, text extracted with pdftotext) for this write-up. The Abstract and the Results sentence appear there exactly as quoted. An earlier audit in this review found the same wording and numbers on the publisher HTML page and in the Europe PMC full-text XML. Every quote above was confirmed verbatim against the PMC-archived text (
src/show.py PMC6478729 --grep). PMC renders "kg ha−1 yr−1" as "kg ha-1 yr-1", and the quotes follow PMC. - Supplements checked. In the earlier audit: MOESM1 (Reporting Summary), MOESM2 (Supplementary Information, Supplementary Figs 1–8 and Table 1), MOESM3 (description of supplementary items) and MOESM4 (Supplementary Data .csv), all from static-content.springer.com. I downloaded and re-read MOESM4 for this write-up. None defines the percentages differently. MOESM4 was used for the recomputation above and supports the stated totals, not the percentages.
- Corrections and comments. As of 2026-10-23, I found no correction, erratum, author correction, retraction or expression of concern. The Crossref record for the DOI has no update-to, updated-by or relation entries, and a Crossref query for works that update this DOI returned 0 results. PMC and Europe PMC list no related notices (
src/known.py). The publisher's article page shows no correction or change history. For PubPeer, the extension API returned a Cloudflare challenge page to automated access, and an earlier DOI lookup returned "Not Found". A web search for the title together with PubPeer, correction or erratum found only copies of the article and repository records. - Sources. Publisher article page · PMC6478729 · Crossref record · Supplementary Data (MOESM4)
Suggested correction
- In the Results, "Scaling field-site observations to the Sahel", paragraph 1, replace "which is a relative increase of 6% for AGH and 20% for WPF" with the increases implied by the unrounded totals. The printed totals give about 11% for AGH and 50% for WPF, and the Supplementary Data series gives 8.7% and 46.6% from endpoints.
- Make the same change in the Abstract ("total herbaceous/woody foliage mass increases by 6%/20%").
- If a different basis was intended, for example the change relative to total vegetation mass, state that basis in both places and recompute both figures on it from the unrounded data.
- Optionally, report the increase in total vegetation (0.44 → 0.55 Pg, about 25%) and the woody share of that increase (about 73%), which make the fodder implications in the Discussion concrete.
Recomputation script (python recompute.py.txt)
"""Recompute the Sahel-wide relative increases reported in PMC6478729
(Commun Biol 2019, doi:10.1038/s42003-019-0383-9).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (Results, "Scaling field-site
observations to the Sahel", paragraph 1) and from the Supplementary Data file
(42003_2019_383_MOESM4_ESM.csv, columns "Agh sahel" and "Wpf sahel", kg/ha).
AGH = above-ground herbaceous mass; WPF = woody plant foliage mass.
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
TOT = {"AGH": (0.28, 0.31), "WPF": (0.16, 0.24)} # Pg, 1992 and 2012 (Results p1)
REPORTED = {"AGH": 6.0, "WPF": 20.0} # % "relative increase" (Results p1, Abstract)
TREND_PG = {"AGH": 0.0025, "WPF": 0.004} # Pg/yr, summed trend (Results p1)
TREND_HA = {"AGH": 5.0, "WPF": 9.0} # kg/ha/yr, fitted VOD trend (Results p1)
SHARE_REPORTED = (35.0, 43.0) # % WPF share of total, 1992 and 2012
SIG_AREA = {"AGH": (289375, 0.07), "WPF": (2711875, 0.64)} # km2 and "~%" of area (Results p1)
HALF = 0.005 # totals are printed to 2 decimals
# Supplementary Data (MOESM4), Sahel VOD estimates in kg/ha, 1992..2012
YEARS = np.arange(1992, 2013)
CSV = {
"AGH": [595.3383, 594.5558, 564.6489, 618.3494, 566.197, 449.3458, 814.1302, 983.0184,
661.9179, 710.5017, 506.1419, 829.6328, 539.4238, 802.3855, 687.8374, 726.8941,
722.1816, 575.7401, 921.7831, 587.7719, 647.3133],
"WPF": [337.616, 338.2679, 338.3046, 434.1801, 373.5995, 313.1169, 301.23, 391.4529,
495.0082, 439.4241, 429.4322, 403.0291, 458.2273, 429.9664, 464.0112, 453.4704,
493.9232, 486.6456, 452.9526, 531.8839, 494.9722],
}
CSV = {k: np.array(v) for k, v in CSV.items()}
def pct(x):
return f"{100 * x:.1f}%"
def grid(x):
return np.linspace(x - HALF, x + HALF, 101)
def rng(f, *vals):
"""Min and max of f over every combination of the rounded inputs."""
mesh = np.meshgrid(*[grid(v) for v in vals], indexing="ij")
out = f(*mesh)
return out.min(), out.max()
R = {} # results reused by the figures
print("=== 1. Relative increase from the stated totals: (2012 - 1992) / 1992 ===")
for k, (a, b) in TOT.items():
rel = (b - a) / a
lo, hi = rng(lambda x, y: (y - x) / x, a, b)
R[k] = dict(rel=100 * rel, lo=100 * lo, hi=100 * hi)
print(f" {k}: ({b} - {a}) / {a} = {pct(rel)} reported {REPORTED[k]:.0f}%")
print(f" range over rounding of both totals (+/-{HALF} Pg): {pct(lo)} to {pct(hi)}")
print(f" ratio recomputed / reported = {100 * rel / REPORTED[k]:.2f}")
print(f" reported % of the 1992 total = {REPORTED[k] / 100 * a:.3f} Pg "
f"(stated difference {b - a:.2f} Pg)")
print(f" WPF/AGH contrast: reported {REPORTED['WPF'] / REPORTED['AGH']:.1f}x, "
f"recomputed {R['WPF']['rel'] / R['AGH']['rel']:.1f}x")
print("\n=== 2. Consistency of the totals with the stated WPF share (35% -> 43%) ===")
s92 = TOT["WPF"][0] / (TOT["WPF"][0] + TOT["AGH"][0])
s12 = TOT["WPF"][1] / (TOT["WPF"][1] + TOT["AGH"][1])
print(f" 1992: 0.16 / 0.44 = {pct(s92)} (reported {SHARE_REPORTED[0]:.0f}%)")
print(f" 2012: 0.24 / 0.55 = {pct(s12)} (reported {SHARE_REPORTED[1]:.0f}%)")
print(f" total vegetation 0.44 -> 0.55 Pg: {pct(0.55 / 0.44 - 1)}")
dw, da = TOT["WPF"][1] - TOT["WPF"][0], TOT["AGH"][1] - TOT["AGH"][0]
print(f" woody share of the total increase: {dw:.2f} / {dw + da:.2f} Pg = {pct(dw / (dw + da))}")
print("\n=== 3. Alternative bases (none is stated in the paper) ===")
ALT = {}
for k, (a, b) in TOT.items():
o = "WPF" if k == "AGH" else "AGH"
oa, ob = TOT[o]
ALT.setdefault(k, {})
ALT[k]["Fitted trend x 20 yr / 1992 total"] = 100 * TREND_PG[k] * 20 / a
ALT[k]["Fitted trend x 21 yr / 1992 total"] = 100 * TREND_PG[k] * 21 / a
ALT[k]["Difference / 2012 total"] = 100 * (b - a) / b
ALT[k]["Difference / mean of both years"] = 100 * (b - a) / ((a + b) / 2)
ALT[k]["Difference / sum of both years"] = 100 * (b - a) / (a + b)
ALT[k]["Difference / 1992 AGH+WPF"] = 100 * (b - a) / (a + oa)
ALT[k]["Difference / 2012 AGH+WPF"] = 100 * (b - a) / (b + ob)
print(f" {k}: trend {TREND_PG[k]} Pg/yr x 20 = {TREND_PG[k] * 20:.3f} Pg "
f"(endpoint difference {b - a:.2f} Pg)")
for name, v in ALT[k].items():
print(f" {name:<36s} {v:5.1f}%")
lo, hi = rng(lambda x, y: (y - x) / (x + y), *TOT["AGH"])
lo2, hi2 = rng(lambda x, y: (y - x) / (x + y), *TOT["WPF"])
print(f" rounding range, difference / sum: AGH {pct(lo)} to {pct(hi)}; WPF {pct(lo2)} to {pct(hi2)}")
lo, hi = rng(lambda x, y, w: (y - x) / (x + w), TOT["AGH"][0], TOT["AGH"][1], TOT["WPF"][0])
lo2, hi2 = rng(lambda w, y, x: (y - w) / (x + w), TOT["WPF"][0], TOT["WPF"][1], TOT["AGH"][0])
print(f" rounding range, difference / 1992 AGH+WPF: AGH {pct(lo)} to {pct(hi)}; "
f"WPF {pct(lo2)} to {pct(hi2)}")
s_agh92, s_agh12 = 1 - s92, 1 - s12
print(f" relative change of each component's share of total: WPF {pct(s12 / s92 - 1)}, "
f"AGH {pct(s_agh12 / s_agh92 - 1)}")
print("\n=== 4. Per-hectare trends and the implied study area ===")
areas = {k: a / p for k, (a, p) in SIG_AREA.items()}
for k, v in areas.items():
print(f" area implied by {k} significant area: {SIG_AREA[k][0]} / {SIG_AREA[k][1]} = {v:,.0f} km2")
area_ha = np.mean(list(areas.values())) * 100
print(f" mean implied area = {area_ha:.3e} ha")
for k, (a, b) in TOT.items():
per_ha = TREND_PG[k] * 1e12 / area_ha
base = a * 1e12 / area_ha
v = 100 * TREND_HA[k] * 20 / base
ALT[k]["Per-ha trend x 20 yr / 1992 per-ha mass"] = v
print(f" {k}: {TREND_PG[k]} Pg/yr -> {per_ha:.2f} kg/ha/yr (reported {TREND_HA[k]:.0f}); "
f"1992 mass {base:.0f} kg/ha; {TREND_HA[k]:.0f} x 20 / {base:.0f} = {v:.1f}%")
print("\n=== 5. Supplementary Data (MOESM4), Sahel kg/ha series 1992-2012 ===")
tot_csv = CSV["AGH"] + CSV["WPF"]
area_csv = 0.44e12 / tot_csv[0] # ha implied by the paper's 0.44 Pg total for 1992
print(f" 1992 AGH+WPF = {tot_csv[0]:.2f} kg/ha; area implied by 0.44 Pg = {area_csv:.3e} ha")
CSVR = {}
for k, y in CSV.items():
o = "WPF" if k == "AGH" else "AGH"
slope, icpt = np.polyfit(YEARS, y, 1)
end = (y[-1] - y[0]) / y[0]
fit0, fit1 = icpt + slope * YEARS[0], icpt + slope * YEARS[-1]
CSVR[k] = dict(end=100 * end, trend=100 * (fit1 - fit0) / fit0)
print(f" {k}: {y[0]:.2f} -> {y[-1]:.2f} kg/ha; scaled {y[0] * area_csv / 1e12:.3f} -> "
f"{y[-1] * area_csv / 1e12:.3f} Pg")
print(f" OLS slope {slope:.2f} kg/ha/yr = {slope * area_csv / 1e12:.4f} Pg/yr")
print(f" endpoint increase {pct(end)}; fitted-line increase {CSVR[k]['trend']:.1f}%")
print(f" difference / 1992 AGH+WPF {pct((y[-1] - y[0]) / tot_csv[0])}; "
f"difference / sum of both years {pct((y[-1] - y[0]) / (y[-1] + y[0]))}")
print(f" WPF share: {pct(CSV['WPF'][0] / tot_csv[0])} (1992), {pct(CSV['WPF'][-1] / tot_csv[-1])} (2012)")
for k in CSVR:
print(f" {k}: closest filled-dot value to reported {REPORTED[k]:.0f}% is "
f"{min([R[k]['rel'], CSVR[k]['end'], CSVR[k]['trend'], ALT[k]['Fitted trend x 20 yr / 1992 total'], ALT[k]['Per-ha trend x 20 yr / 1992 per-ha mass']], key=lambda v: abs(v - REPORTED[k])):.1f}%")
for k in R:
print(f" {k}: gap between reported value and nearest end of rounding range: {R[k]['lo'] - REPORTED[k]:.1f} points")
print("\n=== 6. Understatement factor (recomputed / reported) ===")
for k in R:
print(f" {k}: printed totals {R[k]['rel'] / REPORTED[k]:.2f}x; over rounding "
f"{R[k]['lo'] / REPORTED[k]:.2f}x to {R[k]['hi'] / REPORTED[k]:.2f}x; "
f"Supplementary Data endpoints {CSVR[k]['end'] / REPORTED[k]:.2f}x")
print(f" WPF/AGH contrast from Supplementary Data endpoints: {CSVR['WPF']['end'] / CSVR['AGH']['end']:.1f}x")
# ---------------------------------------------------------------- figures
BG, TXT, SEC, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
"axes.spines.top": False, "axes.spines.right": False,
})
MS = 9 # marker size in points (>= 8 px)
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# Figure 1: reported vs recomputed relative increase, with rounding range
fig, ax = plt.subplots(figsize=(7.2, 3.2))
rows = [("WPF", "Woody foliage (WPF)\n0.16 → 0.24 Pg"), ("AGH", "Herbaceous (AGH)\n0.28 → 0.31 Pg")]
for i, (k, lab) in enumerate(rows):
y = i
r = R[k]
ax.plot([r["lo"], r["hi"]], [y, y], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
for xe in (r["lo"], r["hi"]):
ax.plot([xe, xe], [y - 0.09, y + 0.09], color=BLUE, lw=2, zorder=2)
ax.plot(r["rel"], y, "o", ms=MS + 1, color=BLUE, zorder=3)
ax.plot(REPORTED[k], y, "o", ms=MS + 1, color=ORANGE, zorder=3)
if r["lo"] - REPORTED[k] > 3: # connector only where the gap is wide enough to draw
ax.annotate("", xy=(r["lo"] - 0.6, y), xytext=(REPORTED[k] + 0.9, y),
arrowprops=dict(arrowstyle="-", color=SEC, lw=1, ls=(0, (2, 2))))
ax.text(REPORTED[k], y - 0.2, f"reported {REPORTED[k]:.0f}%", ha="center", va="top",
color=ORANGE, fontsize=10, fontweight="bold")
ax.text(r["rel"], y + 0.2, f"recomputed {r['rel']:.1f}%", ha="center", va="bottom",
color=BLUE, fontsize=10, fontweight="bold")
ax.text(r["hi"] + 1.0, y, f"{r['lo']:.1f}–{r['hi']:.1f}%\nwith rounding", ha="left",
va="center", color=SEC, fontsize=10, linespacing=1.1)
ax.set_yticks(range(len(rows)))
ax.set_yticklabels([lab for _, lab in rows])
ax.set_ylim(-0.6, 1.55)
ax.set_xlim(0, 70)
ax.set_xlabel("Relative increase 1992 → 2012 (%)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[
Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="Reported in Abstract and Results"),
Line2D([], [], marker="o", ls="-", lw=2, ms=MS, color=BLUE,
label="Recomputed from the stated Pg totals (bar: range over rounding)"),
], loc="upper center", bbox_to_anchor=(0.5, -0.27), frameon=False, ncol=1, fontsize=10)
fig.tight_layout()
save(fig, "fig1")
# Figure 2: every basis tried, for both components, against the reported values
order = [
("Stated totals: (2012 − 1992) / 1992", lambda k: R[k]["rel"], "main"),
("Supplementary Data: endpoints", lambda k: CSVR[k]["end"], "main"),
("Supplementary Data: fitted line", lambda k: CSVR[k]["trend"], "main"),
("Stated Pg/yr trend × 20 yr / 1992 total", lambda k: ALT[k]["Fitted trend x 20 yr / 1992 total"], "main"),
("Stated kg/ha/yr trend × 20 yr / 1992 kg/ha", lambda k: ALT[k]["Per-ha trend x 20 yr / 1992 per-ha mass"], "main"),
("Difference / 2012 total", lambda k: ALT[k]["Difference / 2012 total"], "alt"),
("Difference / mean of both years", lambda k: ALT[k]["Difference / mean of both years"], "alt"),
("Difference / 1992 AGH+WPF", lambda k: ALT[k]["Difference / 1992 AGH+WPF"], "alt"),
("Difference / sum of both years", lambda k: ALT[k]["Difference / sum of both years"], "alt"),
]
fig, axes = plt.subplots(1, 2, figsize=(7.2, 4.4), sharey=True)
for ax, k in zip(axes, ("AGH", "WPF")):
n = len(order)
for i, (lab, f, kind) in enumerate(order):
y = n - 1 - i
v = f(k)
face = BLUE if kind == "main" else BG
ax.plot(v, y, "o", ms=MS, color=BLUE, mfc=face, mew=2, zorder=3)
ax.text(v + (1.4 if k == "AGH" else 3.4), y, f"{v:.1f}", va="center", fontsize=10, color=TXT)
ax.axvline(REPORTED[k], color=ORANGE, lw=2, zorder=2)
ax.text(REPORTED[k] + (0.5 if k == "AGH" else 1.2), n - 0.35, f"reported {REPORTED[k]:.0f}%", color=ORANGE, ha="left",
va="bottom", fontsize=10, fontweight="bold")
ax.axhline(n - 5.5, color=SEC, lw=0.8, ls=(0, (3, 3)))
ax.set_xlim(0, 24 if k == "AGH" else 60)
ax.set_ylim(-0.6, n + 0.3)
ax.set_title("Herbaceous (AGH)" if k == "AGH" else "Woody foliage (WPF)", fontsize=11,
loc="left", pad=16)
ax.set_xlabel("Increase 1992 → 2012 (%)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
axes[0].set_yticks(range(len(order)))
axes[0].set_yticklabels([lab for lab, _, _ in order][::-1], fontsize=10)
fig.legend(handles=[
Line2D([], [], color=ORANGE, lw=2, label="Reported (6% / 20%)"),
Line2D([], [], marker="o", ls="", ms=MS, color=BLUE, mew=2,
label="Component's change over its own 1992 value (endpoints or fitted trend)"),
Line2D([], [], marker="o", ls="", ms=MS, color=BLUE, mfc=BG, mew=2,
label="Other denominators (none is stated in the paper)"),
], loc="lower center", ncol=1, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.0))
fig.tight_layout(rect=(0, 0.17, 1, 1))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png to", HERE)
Table 3 extrapolates VA costs with the 75% population share instead of the stated 5%
Diabetes Care (2009) · doi:10.2337/dc09-0670 · PMC2732170
Summary
The paper projects per-patient lifetime costs of Hurricane Katrina for three health care systems with the CORE Diabetes Model. It then scales them up to the diabetic adults of greater New Orleans (964,677 adults × 9.2%), with population shares stated as MCLNO 20%, VA 5% and TUHC 75%. In Table 3, the MCLNO and TUHC extrapolations reproduce exactly at their stated shares. All three VA extrapolations (direct, indirect, total), however, equal the per-patient effect times 0.75 × 964,677 × 9.2%, which is the TUHC share, not the stated 5%. The VA total is therefore 15 times too large ($260.1 million instead of $17.3 million). The population lifetime cost, the only dollar figure in the Abstract ("USD $504 million"), would be about $261.5 million (±10% range about $235–288 million) at the paper's own stated shares, not $504 million ($454–555 million).
What the paper reports
Methods, "Statistical analysis" (the extrapolation and its assumptions):
We further extrapolated the prevalence-based mean impact of an individual patient to the entire population in the greater New Orleans area using 2000 U.S. Census data.
MCLNO represented 20% of the population, VA represented 5%, and TUHC/privately insured represented 75%; the adult population affected in the greater New Orleans area was 964,677 adults; and an estimated diabetes prevalence of 9.2%
Table 3, footnotes to the "Extrapolated cost (US$)" column (they apply to every row):
Adult population (age ≥18 years): 964,677
Prevalence of diabetes in 2006 (Louisiana): 9.2%
Table 3, block headings: "MCLNO (20%)", "TUHC (75%)", "VA (5%)". Rows under the VA (5%) heading and the final row (columns: Effect | SD | P | Extrapolated cost | −10% variation | +10% variation):
Direct cost | 2,683 | 5,423 | <0.0001 | 178,587,759 | 160,728,983 | 196,446,535
Indirect cost | 1,224 | 2,850 | <0.0001 | 81,472,761 | 73,325,485 | 89,620,037
Total cost | 3,907 | 7,201 | <0.0001 | 260,060,520 | 234,054,468 | 286,066,572
Population cost | | | | 504,221,426 | 453,799,283 | 554,643,569
For comparison, the total-cost rows of the other two systems in Table 3:
Total cost | 5,243 | 8,664 | <0.0001 | 93,063,548 (MCLNO (20%) block)
Total cost | 2,270 | 6,927 | <0.0001 | 151,097,359 (TUHC (75%) block)
Abstract, "Results":
We estimate a lifetime cost of USD $504 million for the adult population affected, with the largest economic impact seen among MCLNO patients.
with the economic impact being quite substantial because of the large population size affected
Results, paragraph 3:
The largest economic impact was seen in MCLNO patients with $5,243 over their lifetime, followed by VA patients ($3,907), and privately insured TUHC patients ($2,270).
we estimated a lifetime cost of USD $504 million for the population affected by Hurricane Katrina. A 10% variation around the mean cost of each sample produced an estimate of lifetime costs of USD $454–$555 million due to the impact of Hurricane Katrina.
What the paper's own numbers show
Method. The diabetic adult population is N = 964,677 × 0.092 = 88,750.28. For each system and each cost row of Table 3, I divided the reported extrapolated cost by the per-patient effect. That gives the number of patients the effect was multiplied by, and dividing by N gives the population share actually used. I then recomputed each extrapolation at the share stated in the Methods and in the Table 3 heading, and summed the systems. I also checked that direct + indirect = total for every row, that the ±10% columns are 0.9× and 1.1× the extrapolated cost, and that the Population cost row is the sum of the three system totals. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Diabetic adults in the area | Methods; Table 3 footnotes | 964,677 adults; prevalence 9.2% | 88,750.28 |
| MCLNO patients implied (direct, indirect, total rows) | Table 3 | 62,160,699 / 3,502; 30,902,849 / 1,741; 93,063,548 / 5,243 | 17,750.06 each = 0.2000 × N (stated 20%) |
| TUHC patients implied (direct, indirect, total rows) | Table 3 | 122,475,392 / 1,840; 28,621,967 / 430; 151,097,359 / 2,270 | 66,562.71 each = 0.7500 × N (stated 75%) |
| VA patients implied (direct, indirect, total rows) | Table 3 | 178,587,759 / 2,683; 81,472,761 / 1,224; 260,060,520 / 3,907 | 66,562.71 each = 0.7500 × N (stated 5%: 4,437.51) |
| VA direct extrapolated cost | Table 3 | 178,587,759 | 11,905,851 at 5% (reported is 15.00×) |
| VA indirect extrapolated cost | Table 3 | 81,472,761 | 5,431,517 at 5% (15.00×) |
| VA total extrapolated cost | Table 3 | 260,060,520 | 17,337,368 at 5% (15.00×) |
| MCLNO and TUHC total extrapolated costs | Table 3 | 93,063,548; 151,097,359 | 93,063,548; 151,097,359 (reproduce to the dollar) |
| Population cost | Table 3; Abstract; Results | 504,221,426 ("USD $504 million") | Sum of the reported rows 504,221,427; unrounded with VA at 75% exactly 504,221,426.00. With VA at the stated 5%: 261,498,275 |
| ±10% range of the population cost | Table 3; Results | 453,799,283 – 554,643,569 ("USD $454–$555 million") | 0.9× and 1.1× of the reported total, as printed. With VA at 5%: 235,348,447 – 287,648,102 |
| Sum of population shares used | Methods vs Table 3 (implied) | 20% + 75% + 5% = 100% | 20% + 75% + 75% = 170%: 150,875 patients extrapolated vs 88,750 diabetic adults |
| Ranking of population-level costs by system | Table 3 | VA 260.1M > TUHC 151.1M > MCLNO 93.1M | With VA at 5%: TUHC 151.1M > MCLNO 93.1M > VA 17.3M |
| Overstatement of the population cost | derived | — | 504,221,426 − 261,498,275 = 242,723,151; reported / recomputed = 1.928 |
The internal arithmetic of Table 3 is otherwise consistent: per-patient direct + indirect equals total in every system (3,502 + 1,741 = 5,243; 1,840 + 430 = 2,270; 2,683 + 1,224 = 3,907), the extrapolated direct + indirect equals the extrapolated total, and every ±10% cell is 0.9× or 1.1× its extrapolated cost (to within $1). The only departure from the stated method is the multiplier in the three VA rows, and it carries through unchanged into the Population cost row, its ±10% range, the Abstract and the Results.
Figure
Figure 1. Extrapolated lifetime cost in US$ millions, drawn from zero on one axis. Orange bars are the values printed in Table 3. Blue bars are the per-patient effect from Table 3 multiplied by 964,677 × 9.2% × the share stated in the Methods and in the Table 3 heading (20%, 75%, 5%); the blue population bar is the sum of the three blue system bars. MCLNO and TUHC reproduce exactly. The VA row is 15 times the value at its stated 5% share, and the population cost falls from $504.2 million to $261.5 million.
Figure 2. The number of diabetic adults each system's per-patient cost is multiplied by. The orange bar is implied by the reported Table 3 costs (reported extrapolated cost ÷ per-patient effect, identical for the direct, indirect and total rows). The blue bar uses the stated shares. The vertical line is the whole diabetic adult population (964,677 × 9.2% = 88,750). The reported extrapolation counts 150,875 patients, 170% of the population, because the VA block uses the same 66,563 patients as the TUHC block rather than 4,438.
Why it matters
The population lifetime cost is the only dollar figure in the Abstract ("We estimate a lifetime cost of USD $504 million for the adult population affected") and is repeated with its sensitivity range in the Results. At the paper's own stated assumptions it is overstated about 1.93-fold: about $261.5 million (±10%: about $235–288 million) instead of $504 million ($454–555 million). The whole overstatement comes from the VA block, whose printed contribution ($260.1 million) would be $17.3 million at 5%. A reader citing the paper's economic burden estimate would be using a figure roughly twice what the described method produces.
The error also changes the relative picture at population level. As printed, Table 3 makes the VA the largest contributor to population cost (VA $260.1M > TUHC $151.1M > MCLNO $93.1M). At the stated shares the VA is the smallest (TUHC $151.1M > MCLNO $93.1M > VA $17.3M). The Abstract's phrase "with the largest economic impact seen among MCLNO patients" holds per patient (the Results give $5,243 vs $3,907 vs $2,270) and is unaffected.
What is not affected: the clinical results (changes in A1C, blood pressure and lipids, Table 2, and their P values), the per-patient CORE Diabetes Model estimates of life expectancy, QALE and costs in Table 3, the disparity conclusions, and the direction of the economic finding. A modelled population cost of about $261 million is still large, so the qualitative statement that the economic impact was substantial stands; its magnitude is roughly halved.
Explanations considered
- The VA costs were extrapolated to a different base population (for example, all veterans). This does not account for the numbers. The Methods describe one base for all three systems (964,677 adults × 9.2%) and one set of shares. The Table 3 footnotes apply to the whole column and name no veteran-specific population. And the VA multiplier is not some other population: it equals 0.75 × 964,677 × 0.092 to the cent (66,562.71), the TUHC multiplier.
- The VA and TUHC labels or shares are swapped. This does not account for the numbers. The TUHC rows reproduce exactly at 75%; TUHC at 5% would give $10,073,157, not $151,097,359. Only the VA block departs from its stated share.
- Rounding, units or a misread digit. This does not account for the numbers. The MCLNO and TUHC rows reproduce to the dollar, the gap in the VA rows is an exact factor of 15.00 (0.75 / 0.05) in all three rows, and the same factor runs through the ±10% columns and the Population cost row. A single transcription slip would not reproduce consistently across nine VA cells.
- The 5% is the error, and 75% was intended for the VA. This does not account for the numbers either. The 5% appears in the Methods, in the Table 3 heading "VA (5%)", and is the value that makes the shares sum to 100%. With 75% for the VA, the shares sum to 170% and the extrapolation counts 150,875 patients out of 88,750 diabetic adults, which cannot be a share of one population.
- A text-conversion artifact in the archived copy. This is ruled out. The same figures appear in the live PMC page and in the Europe PMC full-text XML; all are plain positive integers with thousands separators, so no minus signs, decimals, superscripts or shifted cells are involved.
- The population total was computed separately and only the VA rows are misprinted. This does not account for the numbers. The printed Population cost (504,221,426) equals the unrounded sum of the three system totals with the VA at 75% (504,221,426.00), so the total was built from the same 75% multiplier.
Status
- Version of record. The publisher's page (doi.org/10.2337/dc09-0670, which redirects to diabetesjournals.org, Diabetes Care 32(9):1632) and the publisher PDF returned HTTP 403 or a challenge page to automated access. I checked the publisher-deposited full text instead: the live PMC article page and the Europe PMC full-text XML. Both carry the Table 3 values, the "VA (5%)" heading, the Methods share statement and the Abstract figure exactly as quoted above. Every quote in this write-up was confirmed verbatim against the PMC-archived text.
- Supplements. The Europe PMC supplementary-files archive for the article holds only the images of Figures 1–3. The JATS XML has no supplementary-material element. The CORE Diabetes Model inputs and outputs and the population calculation are not published, so they could not be checked.
- Corrections and comments. As of 2026-10-23, no correction, erratum, retraction or expression of concern was found. Crossref lists no update relations for the DOI (including the updated-by field that carries Retraction Watch data). PMC and Europe PMC list no correction notices (PMID 19542210; no comment/correction links), and the PubMed record has no CommentsCorrections or Erratum entries. Europe PMC lists one 2011 letter to the editor (PMID 22190890) that cites the paper only in passing. Earlier web searches for an erratum or a PubPeer thread found none; the PubPeer search page and API returned HTTP 403 to automated access, so PubPeer could not be checked directly. A further web search on 2026-10-23 could not be run because the session's search quota was exhausted.
- Sources. PMC: https://pmc.ncbi.nlm.nih.gov/articles/PMC2732170/ ; Europe PMC full-text XML and REST records for PMID 19542210; Crossref REST API for 10.2337/dc09-0670; PubMed E-utilities record for PMID 19542210.
Suggested correction
- Recompute the VA extrapolated costs in Table 3 at the stated 5% share: direct about $11,905,851 (−10%/+10%: $10,715,266 / $13,096,436), indirect about $5,431,517 ($4,888,366 / $5,974,669), total about $17,337,368 ($15,603,631 / $19,071,105).
- Correct the Population cost row to about $261,498,275 (−10%/+10%: $235,348,447 / $287,648,102), and the Abstract and Results figures from "USD $504 million" and "USD $454–$555 million" to about $261 million and $235–$288 million.
- If the 75% multiplier for the VA was in fact intended, state the VA base population and its source, and explain how the shares relate to a single diabetic population of 88,750 adults; otherwise, the shares as stated (20/75/5) should govern all three blocks.
- While revising Table 3, the VA QALE entry (−1.208, larger in magnitude than the VA life-expectancy change of −0.264 and unlike the other two systems, where the QALE change is smaller than the life-expectancy change) may be worth confirming against the model output. This write-up does not establish it as an error.
Recomputation script (python recompute.py.txt)
"""Recompute the population-level cost extrapolation in Table 3 of PMC2732170
(Diabetes Care 2009;32(9):1632, doi:10.2337/dc09-0670).
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Methods (Statistical analysis): population shares MCLNO 20%, VA 5%, TUHC 75%;
964,677 adults; diabetes prevalence 9.2%
Table 3: per-patient effects (US$) and extrapolated costs with -10%/+10% columns
Table 3 footnotes: 964,677 adults (age >=18 y); prevalence 9.2%
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs (paper)
ADULTS = 964_677
PREV = 0.092
STATED_SHARE = {"MCLNO": 0.20, "TUHC": 0.75, "VA": 0.05} # Methods; Table 3 headings
# per-patient effect (US$): direct, indirect, total (Table 3, "Effect")
EFFECT = {"MCLNO": (3502, 1741, 5243), "TUHC": (1840, 430, 2270), "VA": (2683, 1224, 3907)}
# extrapolated cost (US$): direct, indirect, total (Table 3, "Extrapolated cost")
REPORTED = {
"MCLNO": (62_160_699, 30_902_849, 93_063_548),
"TUHC": (122_475_392, 28_621_967, 151_097_359),
"VA": (178_587_759, 81_472_761, 260_060_520),
}
# -10% and +10% columns (Table 3)
REPORTED_PM10 = {
"MCLNO": ((55_944_629, 68_376_769), (27_812_564, 33_993_134), (83_757_193, 102_369_903)),
"TUHC": ((110_227_853, 134_722_931), (25_759_770, 31_484_163), (135_987_623, 166_207_094)),
"VA": ((160_728_983, 196_446_535), (73_325_485, 89_620_037), (234_054_468, 286_066_572)),
}
REPORTED_POP = (504_221_426, 453_799_283, 554_643_569) # Table 3 "Population cost"
SYSTEMS = ("MCLNO", "TUHC", "VA")
PARTS = ("direct", "indirect", "total")
# ------------------------------------------------------------------ recomputation
N = ADULTS * PREV
print("=== 1. Base population ===")
print(f"diabetic adults = {ADULTS:,} x {PREV} = {N:,.2f}")
print(f"stated shares sum to {sum(STATED_SHARE.values()):.2f}")
print("\n=== 2. Patients implied by each reported extrapolated cost ===")
implied_share = {}
recomputed = {}
for s in SYSTEMS:
sh = STATED_SHARE[s]
print(f"\n[{s}] stated share {sh:.0%} -> {N * sh:,.1f} diabetic adults")
recomputed[s] = []
shares = []
for part, eff, rep in zip(PARTS, EFFECT[s], REPORTED[s]):
n_impl = rep / eff
shares.append(n_impl / N)
rec = eff * N * sh
recomputed[s].append(rec)
print(f" {part:8s}: reported {rep:>13,}; {rep:,} / {eff:,} = {n_impl:,.2f} patients"
f" = {n_impl / N:.4f} x N; at stated share: {eff:,} x {N * sh:,.2f} = {rec:,.0f}"
f" (reported / recomputed = {rep / rec:.2f})")
implied_share[s] = sum(shares) / 3
d, i, t = EFFECT[s]
rd, ri, rt = REPORTED[s]
print(f" checks: {d:,} + {i:,} = {d + i:,} (table {t:,}); "
f"{rd:,} + {ri:,} = {rd + ri:,} (table {rt:,})")
print("\n=== 3. The +/-10% columns are 0.9x and 1.1x the extrapolated cost ===")
for s in SYSTEMS:
for part, rep, (lo, hi) in zip(PARTS, REPORTED[s], REPORTED_PM10[s]):
print(f" {s:5s} {part:8s}: 0.9x = {0.9 * rep:,.0f} (table {lo:,}); "
f"1.1x = {1.1 * rep:,.0f} (table {hi:,})")
print("\n=== 4. Population totals ===")
sum_rep = sum(REPORTED[s][2] for s in SYSTEMS)
exact_rep = sum(EFFECT[s][2] * N * (0.75 if s == "VA" else STATED_SHARE[s]) for s in SYSTEMS)
print(f"sum of reported system totals = {sum_rep:,} (table {REPORTED_POP[0]:,}); "
f"unrounded with VA at 75% = {exact_rep:,.2f}")
print(f"reported +/-10%: 0.9x = {0.9 * REPORTED_POP[0]:,.0f}, 1.1x = {1.1 * REPORTED_POP[0]:,.0f} "
f"(table {REPORTED_POP[1]:,} - {REPORTED_POP[2]:,})")
va5 = recomputed["VA"][2]
corr = REPORTED["MCLNO"][2] + REPORTED["TUHC"][2] + va5
print(f"VA total at stated 5% = {va5:,.0f} (VA direct {recomputed['VA'][0]:,.0f}, "
f"indirect {recomputed['VA'][1]:,.0f})")
for part, rec in zip(PARTS, recomputed["VA"]):
print(f" VA {part:8s} at 5%: {rec:,.0f}; -10% {0.9 * rec:,.0f}; +10% {1.1 * rec:,.0f}")
print(f"population total with VA at 5% = 93,063,548 + 151,097,359 + {va5:,.0f} = {corr:,.0f}")
print(f"+/-10% with VA at 5%: {0.9 * corr:,.0f} - {1.1 * corr:,.0f}")
print(f"reported / corrected population total = {REPORTED_POP[0] / corr:.3f}")
print(f"overstatement = {REPORTED_POP[0] - corr:,.0f} "
f"(= VA reported total - VA at 5% = {REPORTED['VA'][2] - va5:,.0f})")
print("\n=== 5. Shares and patients actually used ===")
for s in SYSTEMS:
print(f" {s:5s}: implied share {implied_share[s]:.4f} (stated {STATED_SHARE[s]:.2f})")
used = sum(implied_share.values())
print(f"implied shares sum to {used:.4f}; patients extrapolated = {N * used:,.0f} "
f"vs {N:,.0f} diabetic adults ({used:.2f}x)")
print("\n=== 6. Alternative readings ===")
print(f"if the VA and TUHC labels were swapped, TUHC at 5% = 2,270 x {N * 0.05:,.2f} = "
f"{2270 * N * 0.05:,.0f} (table 151,097,359)")
print(f"VA multiplier / stated VA multiplier = 0.75 / 0.05 = {0.75 / 0.05:.0f}")
print("\n=== 7. Ranking of system totals ===")
rep_rank = sorted(SYSTEMS, key=lambda s: -REPORTED[s][2])
cor_tot = {"MCLNO": REPORTED["MCLNO"][2], "TUHC": REPORTED["TUHC"][2], "VA": va5}
cor_rank = sorted(SYSTEMS, key=lambda s: -cor_tot[s])
print("reported: " + " > ".join(f"{s} {REPORTED[s][2] / 1e6:.1f}M" for s in rep_rank))
print("VA at 5%: " + " > ".join(f"{s} {cor_tot[s] / 1e6:.1f}M" for s in cor_rank))
# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK, "text.color": INK, "svg.fonttype": "none"})
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# fig1: reported vs recomputed totals, US$ millions, one axis from zero
rows = [("MCLNO (20%)", REPORTED["MCLNO"][2], recomputed["MCLNO"][2]),
("TUHC (75%)", REPORTED["TUHC"][2], recomputed["TUHC"][2]),
("VA (5%)", REPORTED["VA"][2], recomputed["VA"][2]),
("Population cost", REPORTED_POP[0], corr)]
fig, ax = plt.subplots(figsize=(7.2, 4.4), facecolor=BG)
ax.set_facecolor(BG)
h = 0.34
for k, (lab, rep, rec) in enumerate(rows):
y = len(rows) - 1 - k
ax.barh(y + h / 2 + 0.01, rep / 1e6, height=h, color=ORANGE, edgecolor=BG, linewidth=2)
ax.barh(y - h / 2 - 0.01, rec / 1e6, height=h, color=BLUE, edgecolor=BG, linewidth=2)
ax.text(rep / 1e6 + 6, y + h / 2, f"{rep / 1e6:,.1f}", va="center", color=INK, fontsize=10)
ax.text(rec / 1e6 + 6, y - h / 2, f"{rec / 1e6:,.1f}", va="center", color=INK, fontsize=10)
ax.set_yticks(range(len(rows)))
ax.set_yticklabels([r[0] for r in rows][::-1])
ax.axhline(0.5, color=INK2, linewidth=0.8)
ax.set_xlim(0, 590)
ax.set_xlabel("Extrapolated lifetime cost, US$ millions (Table 3)")
ax.grid(axis="x", color=GRID, linewidth=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right"):
ax.spines[sp].set_visible(False)
from matplotlib.patches import Patch # noqa: E402
ax.legend(handles=[Patch(color=ORANGE, label="Reported in Table 3"),
Patch(color=BLUE, label="Recomputed at the stated share\n(effect x 964,677 x 9.2% x share)")],
loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False, fontsize=10)
ax.text(REPORTED["VA"][2] / 1e6 + 55, 1 + h / 2 + 0.01,
"<- computed with the 75% share,\n not the stated 5% (15x)", va="center", color=INK2, fontsize=10)
fig.tight_layout()
save(fig, "fig1")
# fig2: diabetic adults counted in the extrapolation vs the diabetic adults available
fig, ax = plt.subplots(figsize=(7.2, 3.2), facecolor=BG)
ax.set_facecolor(BG)
bars = [("Implied by the\nreported costs", ORANGE,
[("MCLNO", N * implied_share["MCLNO"]), ("TUHC", N * implied_share["TUHC"]),
("VA", N * implied_share["VA"])]),
("Stated shares\n20% / 75% / 5%", BLUE,
[("MCLNO", N * 0.20), ("TUHC", N * 0.75), ("VA", N * 0.05)])]
for k, (lab, col, segs) in enumerate(bars):
y = 1 - k
left = 0
for name, v in segs:
ax.barh(y, v / 1e3, left=left / 1e3, height=0.55, color=col, edgecolor=BG, linewidth=2)
txt = f"{name}\n{v:,.0f}"
if v > 10_000:
ax.text((left + v / 2) / 1e3, y, txt, ha="center", va="center", color="white", fontsize=10)
else:
ax.text((left + v) / 1e3 + 1.5, y + 0.43, f"{name} {v:,.0f}", ha="left", va="center",
color=INK, fontsize=10)
left += v
ax.text(left / 1e3 + 1.5, y, f"{left:,.0f}\n({left / N:.0%})", va="center", color=INK, fontsize=10)
ax.axvline(N / 1e3, color=INK2, linewidth=1)
ax.text(N / 1e3 + 1.5, 1.68, f"all diabetic adults in the area\n964,677 x 9.2% = {N:,.0f}",
color=INK2, fontsize=10, va="center")
ax.set_yticks([1, 0])
ax.set_yticklabels([b[0] for b in bars])
ax.set_ylim(-0.5, 1.95)
ax.set_xlim(0, 175)
ax.set_xlabel("Diabetic adults the costs are multiplied by (thousands)")
ax.grid(axis="x", color=GRID, linewidth=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right"):
ax.spines[sp].set_visible(False)
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png")
Reported p values for HbA1c and other secondary outcomes contradict their own confidence intervals
Diabetologia (2025) · doi:10.1007/s00125-025-06585-2 · PMC12779728
Summary
This 76-participant randomised trial compared video consultations with in-person care in adults with type 1 diabetes who use insulin pumps. Table 2 gives the between-group difference in HbA1c change as −2.01 mmol/mol with a 95% CI of −5.32 to 1.29 and p < 0.001, and the Abstract and the Discussion present a favourable, statistically significant HbA1c effect. That interval includes zero and implies a two-sided p of about 0.23. A p below 0.001 would need a standard error of at most 0.61, about a third (36%) of the 1.69 implied by the printed interval. The same mismatch between interval and p value appears in every other Table 2 row that has a p value, including the treatment-satisfaction and quality-of-life outcomes the Abstract calls superior and inferior. As printed, none of the secondary-outcome claims of statistical significance is supported by the paper's own intervals. The primary result for time in range, no clinically meaningful difference, is not affected.
What the paper reports
Table 2, "Primary and secondary endpoints" (columns: endpoint | video group | control group | difference (95% CI) | p value). The footer reads "Estimates are reported as least squares means ± SEs".
Change in HbA1c, mmol/mol | -4.13 ± 1.43 | -2.12 ± 1.38 | -2.01 (-5.32, 1.29) | <0.001
Change in HbA1c, % | -0.38 ± 0.13 | -0.19 ± 0.13 | -0.18 (-0.49, 0.12) |
The other Table 2 rows that have a p value:
TiR (3.9–10.0 mmol/l), % | 64.28 ± 2.91 | 63.49 ± 2.50 | 0.8 (-5.30, 6.89) | 0.251
DTQSc (range -18 to 18) | 10.23 ± 1.61 | 9.01 ± 1.42 | 1.22 (-2.51, 4.96) | 0.004
DTQSs (range 0–36) | 30.06 ± 1.25 | 28.09 ± 1.13 | 1.98 (-0.98, 4.93) | <0.001
ADDQoL19 (range -9 to +3) | -1.6 ± 0.33 | -1.33 ± 0.29 | -0.27 (-1.07, 0.53) | 0.004
Methods, "Statistical analysis":
All 95% CIs and p values are two-sided.
the key secondary outcomes were analysed in a prioritised order (i.e. ‘gatekeeping procedure’) [17].
The statistical tests were performed and interpreted in sequence until one of the analyses failed to show a statistically significant difference at a statistical significance level of 0.05
Missing data were imputed using retrieved participants from the same randomised treatment group, with results combined according to Rubin’s rule [18]. The main analyses are reported as least squares means and SEs, with differences between groups and corresponding 95% CIs adjusted for stratification and baseline outcome measurements.
Results, second paragraph. The text repeats the intervals but gives no p values for the secondary outcomes:
the observed difference of 0.8 percentage points (95% CI -5.3, 6.9; p=0.25)
the intervention group showed better treatment satisfaction compared with the controls, with a potential difference of 1.22 points for DTSQc (95% CI -2.51, 4.96) and 1.98 points for DTSQs (95% CI -0.98, 4.93)
Regarding glycaemic control, only HbA1c demonstrated a favourable difference, being lower in the intervention group than in the control group, with a mean difference of -2.01 mmol/mol (95% CI -5.32, 1.29) (-0.18%; -0.49%, 0.12%).
Results, "Sensitivity analyses and adherence to protocol":
No discrepancies were observed between the main findings and those from sensitivity analyses for primary or secondary endpoints
Abstract, Results:
For secondary outcomes, the video group was superior in terms of treatment satisfaction and reduction in HbA1c. However, the video group experienced an inferior impact on quality of life.
Abstract, Conclusions/interpretation:
However, compared with control, the intervention was associated with superior treatment satisfaction and a favourable effect on HbA1c, albeit with an inferior impact on quality of life.
Discussion, first paragraph:
However, although a statistically significant difference was found for several of the secondary outcomes
Discussion, second paragraph:
We did find a statistically significant improvement in HbA1c, albeit not necessarily clinically relevant, in the video consultation group.
Discussion, on quality of life:
Hence, one might speculate that although statistically different, the finding might not be clinically meaningful
Discussion, concluding paragraph:
However, better treatment satisfaction and a greater reduction in HbA1c were seen, although with a greater negative impact on quality of life.
What the paper's own numbers show
Method. The Methods describe a two-sided Wald-type comparison: an ANCOVA difference in least squares means with a 95% CI. For such a comparison the CI and the p value come from the same estimate and standard error, so each one fixes the other. For each Table 2 row I took the SE as the CI width divided by 2 × 1.96, computed z = difference / SE and the two-sided normal p value, and compared it with the printed p. As a second, independent check I computed the SE of the difference from the two arm SEs, assuming the arm estimates are uncorrelated, as they are in a randomised parallel-group trial. I also asked what SE and CI the printed p would need, and repeated the CI-to-p calculation for the sensitivity analyses in ESM Tables 1 and 2. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| HbA1c difference, video − control (mmol/mol) | Table 2; Results | −2.01 (95% CI −5.32, 1.29) | LS means −4.13 − (−2.12) = −2.01; CI midpoint −2.015, so the interval is centred on the estimate |
| SE of the HbA1c difference, from the CI | Table 2 | — | half-width 3.305; SE = 3.305 / 1.96 = 1.686 |
| HbA1c two-sided p, from the CI | Table 2 | <0.001 | z = −1.19; p = 0.233 (t with 60 or 70 df: 0.24) |
| HbA1c p, from the arm SEs | Table 2 | <0.001 | SE = √(1.43² + 1.38²) = 1.987; z = −1.01; p = 0.31; 95% CI (−5.91, 1.89) |
| HbA1c one-sided p | derived | — (Methods: all p values two-sided) | 0.117 |
| SE and CI that p = 0.001 would require | derived | — | SE ≤ 0.611, CI half-width ≤ 1.20, i.e. about (−3.21, −0.81), against a printed half-width of 3.305 |
| HbA1c difference, % scale | Table 2 | −0.18 (−0.49, 0.12), p cell blank | SE 0.156, p = 0.25; −2.01 / 10.929 = −0.184 (−0.487, 0.118), so the two scales agree with each other |
| TiR (primary) | Table 2; Abstract; Results | 0.8 (−5.30, 6.89), p = 0.251 | SE 3.110, z = 0.26, p = 0.80; p = 0.251 would need a CI half-width of 1.37, not 6.09 |
| DTSQc | Table 2 | 1.22 (−2.51, 4.96), p = 0.004 | SE 1.906, z = 0.64, p = 0.52 |
| DTSQs | Table 2 | 1.98 (−0.98, 4.93), p < 0.001 | SE 1.508, z = 1.31, p = 0.19 |
| ADDQoL19 | Table 2 | −0.27 (−1.07, 0.53), p = 0.004 | SE 0.408, z = −0.66, p = 0.51 |
| TbR < 3.0 mmol/l | Table 2 | 0.05 (−0.34, 0.44), p = 0.200 | p = 0.80 |
| Sensitivity analyses with a printed p | ESM Table 1 (per protocol); ESM Table 2 (baseline carried forward) | ESM T1 TiR 3.69 (−3.36, 10.75), p 0.294; ESM T2 TiR −0.56 (−5.49, 4.36), p 0.819; ESM T1 DTSQc 1.30 (−4.79, 7.39), p 0.658; ESM T2 DTSQc 0.94 (−3.57, 5.45), p 0.674 | in the same order: p = 0.305, 0.824, 0.676 and 0.683; these agree with their CIs to within 0.02 |
| HbA1c in the sensitivity analyses | ESM Table 1 (per protocol); ESM Table 2 (baseline carried forward) | −3.42 (−7.54, 0.69); −1.31 (−4.02, 1.4); p shown as "-" | p = 0.10 and 0.34; both intervals include zero |
All six Table 2 rows with both a printed p and a 95% CI are inconsistent with themselves, and in each case the printed p is smaller than the interval allows. Four of the six fall below 0.05 although every one of the six intervals includes zero. By contrast, the four p values printed in the ESM sensitivity tables match their intervals.
Side observation. In four Table 2 rows without a p value, the printed difference does not equal video minus control. TbR 3.0–3.8 mmol/l: 1.70 − 1.46 = 0.24, printed −0.20. TaR > 13.9 mmol/l: −1.54, printed 0.35. TaR 10.1–13.9 mmol/l: −0.88, printed −2.83. Glycaemic variability: 36.86 − 36.86 = 0.00, printed 2.24. In each case the printed CI is centred on the printed difference, not on the difference of the LS means. These rows do not feed the claims discussed here, but they suggest the Table 2 entries as a whole need checking.
Figure
Figure 1. For the primary outcome and the four secondary outcomes that the paper's conclusions rest on, the orange circle is the printed p (a left-pointing triangle marks "<0.001", which is an upper bound, drawn at 0.001), and the blue diamond is the two-sided p recomputed from the printed CI as described above. The axis is logarithmic, and the vertical line marks p = 0.05. Every recomputed value lies above 0.05, between 0.19 and 0.80, while four of the printed values lie below it. The gap is largest for HbA1c and DTSQs, where the printed p is at least 190 to 230 times smaller than the interval implies. The sixth row with a p value, TbR < 3.0 mmol/l (printed 0.200, recomputed 0.80), is not drawn. All values are from Table 2 or from recompute.py.txt.
Figure 2. The HbA1c difference on its own scale (mmol/mol, video − control). The orange interval is the printed 95% CI (−5.32, 1.29). The solid blue interval is the CI implied by the two arm SEs, 1.43 and 1.38 (−5.91, 1.89; p = 0.31). The dashed blue interval is hypothetical: the 95% CI that a p of 0.001 would require around the same estimate (−3.21, −0.81). Both intervals built from the paper's numbers cross zero (vertical line). Only an interval less than half as wide as the printed one, and entirely below zero, would be consistent with p < 0.001.
Why it matters
The trial's primary outcome, time in range, showed no meaningful difference, and that conclusion is unaffected: the CI (−5.30, 6.89) and the stated decision rule both point the same way, whichever p value is right. The paper's positive findings all come from the secondary outcomes. The Abstract reports that the video group "was superior in terms of treatment satisfaction and reduction in HbA1c" and had "an inferior impact on quality of life", and the Abstract conclusion and several Discussion paragraphs, including the concluding one, repeat these findings. The Discussion explicitly calls the HbA1c improvement "statistically significant" and the quality-of-life difference "statistically different". Those characterisations match the printed p values (<0.001, <0.001, 0.004, 0.004) but not the printed intervals, all of which include zero and imply p between 0.19 and 0.52. On the intervals, none of the secondary differences is statistically significant at 0.05. That includes the % HbA1c row, which has no p value but whose CI (−0.49, 0.12) gives p ≈ 0.25.
The paper's own gatekeeping rule points the same way. The Methods say the key secondary outcomes were analysed in a prioritised order, "listed in order under 'Outcomes and data collection'", and that the tests were interpreted in sequence until one failed at 0.05. The first outcome listed there is DTSQc, whose interval (−2.51, 4.96) includes zero and implies p = 0.52. On the intervals, the sequence therefore stops at its first step, and none of the later secondary outcomes (DTSQs, ADDQoL19, HbA1c) would be interpreted as confirmatory. If, as is usual, the sequence starts with the primary outcome, it would stop even earlier, since time in range was not significant either (printed p = 0.251; p from the CI 0.80).
Readers of the Abstract are therefore told of a favourable HbA1c effect and of superior treatment satisfaction that the reported data do not show to be different from no effect. A meta-analysis that took the HbA1c result from the text or CI would treat it as null. A reader or guideline that relied on the Abstract would treat it as positive.
Explanations considered
- Conversion or typesetting error in PMC. Ruled out. The publisher's version of record shows the same Table 2 values, including "<0.001" for HbA1c and a blank p cell for the % row, as do the PMC page and the Europe PMC XML.
- Rounding. Cannot account for the gap. The recomputed p values (0.19 to 0.80) differ from the printed ones (<0.001 to 0.251) by factors of 3 to more than 200, which rounding of the CI limits to two decimals cannot produce. The HbA1c interval is centred on its estimate (midpoint −2.015), and its % version converts exactly from the mmol/mol version, so the interval itself is not garbled.
- t rather than normal reference distribution. With 60 to 70 degrees of freedom the HbA1c p is 0.24, slightly larger, not smaller.
- One-sided test. The Methods say all p values are two-sided. A one-sided HbA1c p would still be 0.117.
- Multiplicity adjustment applied to CIs but not p values, or the reverse. The Methods say no multiplicity adjustment was applied. An adjustment would in any case raise p values, not lower them.
- Correlated arm estimates. Taking the arm SEs at face value makes the inconsistency worse (p = 0.31). For the SE of the difference to shrink to 0.61, the two arms' LS means would have to be correlated at about 0.91, which is not plausible for independently randomised parallel groups.
- Multiple imputation (Rubin's rules) affecting the CI but not the p value. For p < 0.001 to be right while the CI is also right, about 87% of the CI's variance would have to come from between-imputation variability. Only 13 of 76 participants (17%) did not complete the trial. The per-protocol analysis, which uses no imputation, also gives an HbA1c interval that includes zero (−7.54, 0.69; p ≈ 0.10).
- The p column reports a different test. For example, the p value could be for a within-group change from baseline, or for another term in the ANCOVA model. The video arm's within-group HbA1c change gives p = 0.004 (−4.13 / 1.43), which is still not <0.001. The other possibilities cannot be checked without the data. In any case the table pairs each p value with a between-group difference and CI, and the text interprets the p values as tests of that difference, so the inconsistency in the reported results would remain.
- The CI is wrong and the p value is right. This cannot be excluded without the data, but the paper's other numbers argue against it. The arm SEs independently imply an even wider interval. The interval is repeated in the Results text on two scales that agree with each other. Both sensitivity analyses give HbA1c intervals that include zero, and the Results state that they agree with the main analysis. The p values printed in the ESM tables also match their own CIs, so the mismatch is confined to the main Table 2.
None of these explanations reconciles the printed p values with the printed intervals.
Status
As of 2026-10-23:
- Version of record. On 2026-10-23 the publisher's Table 2 page (link.springer.com/article/10.1007/s00125-025-06585-2/tables/2) shows "Change in HbA1c, mmol/mol | −4.13 ± 1.43 | −2.12 ± 1.38 | −2.01 (−5.32, 1.29) | <0.001", the % row with a blank p cell, and the same values in the other rows, including the four differences that do not match their LS means. The publisher's article page carries the same Abstract and Discussion sentences. The article was published online on 7 November 2025 in Diabetologia 69(2):321–329 (February 2026). The PMC page and the Europe PMC full-text XML give identical values. Every quote above was checked verbatim against the PMC-archived text with
src/show.py, which renders minus signs as hyphens. - Supplements. The single ESM file (125_2025_6585_MOESM1_ESM.pdf, "ESM Tables", 5 pages) was downloaded from static-content.springer.com and read in full. ESM Tables 1 and 2 (sensitivity analyses) give HbA1c intervals that include zero, with no p value. ESM Table 4 (subgroups) gives HbA1c intervals that all include zero. No other HbA1c p value appears anywhere. Patient-level data are not public; the Data availability statement says they are available from the corresponding author on reasonable request.
- Corrections.
src/known.pylists no Crossref update relations and no PMC or Europe PMC correction, retraction or expression-of-concern notice. A Crossreffilter=updates:query for the DOI returns 0 results. - PubPeer. The PubPeer API and site returned HTTP 403 to automated requests on 2026-10-23, so PubPeer could not be checked directly. A web search for the title together with PubPeer found no thread.
Sources: PMC12779728 (PubMed Central archive, read with src/show.py); https://doi.org/10.1007/s00125-025-06585-2 and the publisher's Table 2 page; ESM file https://static-content.springer.com/esm/art%3A10.1007%2Fs00125-025-06585-2/MediaObjects/125_2025_6585_MOESM1_ESM.pdf; Europe PMC full-text XML; https://api.crossref.org/works?filter=updates:10.1007/s00125-025-06585-2; ClinicalTrials.gov NCT04612933 (registry entry, not used for any number here).
Suggested correction
- Re-derive the between-group p values for every Table 2 row from the same ANCOVA and multiple-imputation model that produced the differences and CIs, and correct the p column. Add the p value for the % HbA1c row, or state that it is the same test as the mmol/mol row.
- Revise the statements that depend on statistical significance of the secondary outcomes so that they match the corrected p values and the intervals. These are the Abstract Results and Conclusions/interpretation, and the first, second, quality-of-life and concluding paragraphs of the Discussion. If the intervals are correct, these differences should be described as not statistically significant, with the CI given.
- State how the gatekeeping procedure was applied: whether the sequence starts with the primary outcome, and how the secondary outcomes were interpreted given that the first listed key secondary outcome, DTSQc, has an interval that includes zero (−2.51, 4.96).
- Check the difference column for TbR 3.0–3.8 mmol/l, both TaR rows and glycaemic variability, where the printed difference does not equal video minus control.
Recomputation script (python recompute.py.txt)
"""Recompute the Table 2 p values of PMC12779728
(Diabetologia 2025, doi:10.1007/s00125-025-06585-2).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file, plus five candidate
figure designs (cand_A ... cand_E) and a contact sheet in ./candidates/.
Inputs are copied from the paper as archived in PubMed Central (identical in the
publisher's version of record) and from the single ESM file (ESM Tables 1 and 2).
Table 2 reports least squares means +- SE for each arm, the between-group difference
with its 95% CI, and a p value. Methods: "All 95% CIs and p values are two-sided."
"""
import os
import re
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z975 = stats.norm.ppf(0.975)
# ---------------------------------------------------------------- inputs (Table 2)
# name, LS mean video, SE, LS mean control, SE, difference, CI low, CI high, printed p
TABLE2 = [
("TiR (primary), %", 64.28, 2.91, 63.49, 2.50, 0.80, -5.30, 6.89, "0.251"),
("DTSQc", 10.23, 1.61, 9.01, 1.42, 1.22, -2.51, 4.96, "0.004"),
("DTSQs", 30.06, 1.25, 28.09, 1.13, 1.98, -0.98, 4.93, "<0.001"),
("ADDQoL19", -1.60, 0.33, -1.33, 0.29, -0.27, -1.07, 0.53, "0.004"),
("Change in HbA1c, mmol/mol", -4.13, 1.43, -2.12, 1.38, -2.01, -5.32, 1.29, "<0.001"),
("Change in HbA1c, %", -0.38, 0.13, -0.19, 0.13, -0.18, -0.49, 0.12, ""),
("TbR <3.0, %", 0.44, 0.18, 0.39, 0.16, 0.05, -0.34, 0.44, "0.200"),
("TbR 3.0-3.8, %", 1.70, 0.24, 1.46, 0.21, -0.20, -0.71, 0.31, "-"),
("TaR >13.9, %", 8.92, 2.29, 10.46, 1.87, 0.35, -3.04, 3.75, "-"),
("TaR 10.1-13.9, %", 24.30, 2.31, 25.18, 1.86, -2.83, -6.40, 0.74, "-"),
("Glycaemic variability, %", 36.86, 1.70, 36.86, 1.44, 2.24, -0.43, 4.90, "-"),
]
# ESM Tables 1 (per protocol) and 2 (ITT, baseline observation carried forward):
# rows that print a p value, plus the HbA1c rows (p printed as "-").
ESM = [
("ESM T1 per protocol, TiR", 3.69, -3.36, 10.75, "0.294"),
("ESM T1 per protocol, DTSQc", 1.30, -4.79, 7.39, "0.658"),
("ESM T2 BOCF, TiR", -0.56, -5.49, 4.36, "0.819"),
("ESM T2 BOCF, DTSQc", 0.94, -3.57, 5.45, "0.674"),
("ESM T1 per protocol, HbA1c mmol/mol", -3.42, -7.54, 0.69, "-"),
("ESM T2 BOCF, HbA1c mmol/mol", -1.31, -4.02, 1.40, "-"),
]
N_RAND, N_COMPLETE = 76, 32 + 31 # Abstract: 76 randomised, 32 + 31 completed
IFCC_PER_NGSP = 10.929 # mmol/mol per 1 percentage point of HbA1c
def se_from_ci(lo, hi):
return (hi - lo) / (2 * Z975)
def p_two(z):
return 2 * stats.norm.sf(abs(z))
def z_from_p(p):
return stats.norm.isf(p / 2)
def p_value_of(printed):
return float(printed.lstrip("<"))
# ---------------------------------------------------------------- 1. every Table 2 row
print("1. Table 2: p value implied by each row's own 95% CI and by its arm SEs (normal)")
print(f"{'endpoint':27s} {'V-C':>6s} {'diff':>6s} {'CImid':>6s} {'SE_CI':>6s} {'z_CI':>6s}"
f" {'p_CI':>6s} {'SE_arm':>6s} {'p_arm':>6s} {'p_printed':>9s}")
REC = {}
for n, mv, sv, mc, sc, d, lo, hi, p in TABLE2:
se_ci = se_from_ci(lo, hi)
z_ci = d / se_ci
se_arm = float(np.hypot(sv, sc))
p_arm = p_two((mv - mc) / se_arm)
REC[n] = dict(d=d, lo=lo, hi=hi, se=se_ci, z=z_ci, p=p_two(z_ci), se_arm=se_arm,
p_arm=p_arm, lsdiff=mv - mc, printed=p)
print(f"{n:27s} {mv - mc:6.2f} {d:6.2f} {(lo + hi) / 2:6.2f} {se_ci:6.3f} {z_ci:6.2f}"
f" {p_two(z_ci):6.3f} {se_arm:6.3f} {p_arm:6.3f} {p:>9s}")
TESTED = ["TiR (primary), %", "DTSQc", "DTSQs", "ADDQoL19", "Change in HbA1c, mmol/mol"]
print("\n |z| implied by the printed p (largest p allowed by '<'):")
for n in TESTED:
r = REC[n]
zp = z_from_p(p_value_of(r["printed"]))
print(f" {n:27s} printed p {r['printed']:>6s} -> |z| {'>=' if r['printed'].startswith('<') else '='}"
f" {zp:.2f}; CI gives |z| = {abs(r['z']):.2f}; SE the printed p needs = {abs(r['d']) / zp:.3f}"
f" (CI half-width {abs(r['d']) / zp * Z975:.2f}, printed {(r['hi'] - r['lo']) / 2:.2f})")
# ---------------------------------------------------------------- 2. HbA1c in detail
h = REC["Change in HbA1c, mmol/mol"]
d, lo, hi = h["d"], h["lo"], h["hi"]
print("\n2. HbA1c, mmol/mol")
print(f" LS means: -4.13 - (-2.12) = {-4.13 - -2.12:.2f}; CI midpoint {(lo + hi) / 2:.3f}")
print(f" CI half-width {(hi - lo) / 2:.3f}; SE {h['se']:.3f}; z {h['z']:.3f}; "
f"two-sided p {h['p']:.3f}; one-sided p {h['p'] / 2:.3f}")
for df in (60, 70):
print(f" t with df {df}: two-sided p {2 * stats.t.sf(abs(h['z']), df):.3f}")
print(f" SE from arm SEs sqrt(1.43^2+1.38^2) = {h['se_arm']:.3f}; z {d / h['se_arm']:.3f}; p {h['p_arm']:.3f};"
f" 95% CI ({d - Z975 * h['se_arm']:.2f}, {d + Z975 * h['se_arm']:.2f})")
z001 = z_from_p(0.001)
se_need = abs(d) / z001
HB_NEED = (d - Z975 * se_need, d + Z975 * se_need)
print(f" p = 0.001 needs |z| {z001:.3f}, SE {se_need:.3f}, CI half-width {se_need * Z975:.2f},"
f" i.e. 95% CI ({HB_NEED[0]:.2f}, {HB_NEED[1]:.2f})")
print(f" SE ratio printed CI / needed: {h['se'] / se_need:.2f}; needed / printed {se_need / h['se']:.3f}"
f" ({100 * se_need / h['se']:.0f}%); variance ratio {(h['se'] / se_need) ** 2:.1f}")
rho = (1.43 ** 2 + 1.38 ** 2 - se_need ** 2) / (2 * 1.43 * 1.38)
print(f" correlation between the two LS means needed to shrink the arm-SE difference SE to {se_need:.3f}: {rho:.2f}")
miss = 1 - N_COMPLETE / N_RAND
print(f" missing outcome share {N_RAND - N_COMPLETE}/{N_RAND} = {miss:.3f}; share of the CI's variance that "
f"would have to come from imputation alone for p<0.001: {1 - (se_need / h['se']) ** 2:.3f}")
print(f" within-group z (LS mean / SE): video {-4.13 / 1.43:.2f}, p {p_two(-4.13 / 1.43):.4f};"
f" control {-2.12 / 1.38:.2f}, p {p_two(-2.12 / 1.38):.3f}")
hp = REC["Change in HbA1c, %"]
print(f" % row: SE {hp['se']:.3f}, z {hp['z']:.2f}, p {hp['p']:.3f}")
print(f" mmol/mol -> % (/ {IFCC_PER_NGSP}): diff {d / IFCC_PER_NGSP:.3f}, CI ({lo / IFCC_PER_NGSP:.3f},"
f" {hi / IFCC_PER_NGSP:.3f}); arms {-4.13 / IFCC_PER_NGSP:.3f}, {-2.12 / IFCC_PER_NGSP:.3f}")
# ---------------------------------------------------------------- 3. ESM rows
print("\n3. ESM sensitivity tables: p implied by the printed CI")
for n, dd, l, u, p in ESM:
se = se_from_ci(l, u)
print(f" {n:38s} {dd:6.2f} ({l}, {u}) SE {se:.3f} p_CI {p_two(dd / se):.3f} printed {p}")
# ---------------------------------------------------------------- 4. side observation
print("\n4. Printed differences that do not equal video minus control LS means")
for n in ["TbR 3.0-3.8, %", "TaR >13.9, %", "TaR 10.1-13.9, %", "Glycaemic variability, %"]:
r = REC[n]
print(f" {n:27s} video - control = {r['lsdiff']:6.2f}; printed difference {r['d']:6.2f}"
f" (CI midpoint {(r['lo'] + r['hi']) / 2:.2f})")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"axes.spines.top": False, "axes.spines.right": False, "svg.fonttype": "none",
})
LABELS = {"TiR (primary), %": "TiR, % (primary)", "DTSQc": "DTSQc", "DTSQs": "DTSQs",
"ADDQoL19": "ADDQoL19", "Change in HbA1c, mmol/mol": "HbA1c change, mmol/mol"}
MS = 9 # marker size in points (>= 8 px)
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg", bbox_inches="tight")
fig.savefig(p + ".png", dpi=200, bbox_inches="tight")
plt.close(fig)
def fmt_p(p):
return f"{p:.2f}" if p >= 0.01 else f"{p:.3f}"
def mn(text):
"""Typographic minus signs for numbers drawn in figures."""
return re.sub(r"(?<![A-Za-z])-(?=\d)", "\u2212", text)
MASK = dict(boxstyle="square,pad=0.1", fc=BG, ec="none")
def design_A(paths):
"""Dot-and-interval (dumbbell) on one log p axis, five tested rows."""
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ys = np.arange(len(TESTED))[::-1]
for y, n in zip(ys, TESTED):
r = REC[n]
pr = p_value_of(r["printed"])
ax.plot([pr, r["p"]], [y, y], color=INK2, lw=1, alpha=0.5, zorder=1)
mk = "<" if r["printed"].startswith("<") else "o"
ax.plot(pr, y, marker=mk, ms=MS + 1, color=ORANGE, mec=BG, mew=1.5, ls="none", zorder=3)
ax.plot(r["p"], y, marker="D", ms=MS - 1, color=BLUE, mec=BG, mew=1.5, ls="none", zorder=3)
ax.annotate(f"printed {r['printed']}", (pr, y), xytext=(-8, 9), textcoords="offset points",
ha="right", fontsize=10, color=INK, bbox=MASK, zorder=4)
ax.annotate(f"from CI {fmt_p(r['p'])}", (r["p"], y), xytext=(8, 9), textcoords="offset points",
ha="left", fontsize=10, color=INK, bbox=MASK, zorder=4)
ax.axvline(0.05, color=INK2, lw=1, zorder=0)
ax.text(0.05, len(TESTED) - 0.35, "p = 0.05", ha="center", va="bottom", fontsize=10, color=INK2, bbox=MASK)
ax.set_xscale("log")
ax.set_xlim(3e-4, 3)
ax.set_xticks([0.001, 0.01, 0.05, 0.1, 0.5, 1])
ax.set_xticklabels(["0.001", "0.01", "0.05", "0.1", "0.5", "1"])
ax.set_yticks(ys)
ax.set_yticklabels([LABELS[n] for n in TESTED], color=INK)
ax.set_ylim(-0.6, len(TESTED) - 0.1)
ax.set_xlabel("Two-sided p value (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.tick_params(axis="y", length=0)
ax.spines["left"].set_visible(False)
h1 = plt.Line2D([], [], marker="o", color=ORANGE, ls="none", ms=MS, label="p printed in Table 2 ('<' = upper bound)")
h2 = plt.Line2D([], [], marker="D", color=BLUE, ls="none", ms=MS - 1, label="p recomputed from the row's own 95% CI")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
ax.set_title("Table 2: printed p values against the p values their 95% CIs imply",
loc="left", fontsize=11.5, color=INK)
save(fig, *paths)
def design_B(paths):
"""HbA1c only: printed CI vs CI from arm SEs vs the CI that p<0.001 would need."""
fig, ax = plt.subplots(figsize=(7.2, 3.2))
rows = [
("Printed in Table 2", d, lo, hi, ORANGE, "-", "o", f"printed p <0.001; this CI implies p = {fmt_p(h['p'])}"),
("From the arm SEs\n(1.43 and 1.38)", d, d - Z975 * h["se_arm"], d + Z975 * h["se_arm"], BLUE, "-", "D",
f"p = {fmt_p(h['p_arm'])}"),
("Needed for p = 0.001", d, HB_NEED[0], HB_NEED[1], BLUE, (0, (4, 2)), "D", "hypothetical: half-width 1.20"),
]
for i, (lab, c, l, u, col, ls, mk, note) in enumerate(rows):
y = len(rows) - 1 - i
ax.plot([l, u], [y, y], color=col, lw=2, ls=ls, solid_capstyle="round")
mfc = col if i < 2 else BG
ax.plot(c, y, marker=mk, ms=MS, color=col, mfc=mfc, mec=col, mew=2, ls="none")
ax.text(u + 0.15, y, mn(f"({l:.2f}, {u:.2f})"), va="center", fontsize=10, color=INK, bbox=MASK, zorder=4)
ax.text(l, y + 0.2, note, ha="left", va="bottom", fontsize=10, color=INK2, bbox=MASK, zorder=4)
ax.axvline(0, color=INK2, lw=0.8)
ax.text(0.05, -0.55, "no difference", fontsize=10, color=INK2, va="bottom")
ax.set_yticks(range(len(rows))[::-1])
ax.set_yticklabels([r[0] for r in rows], color=INK)
ax.set_ylim(-0.6, len(rows) - 0.2)
ax.set_xlim(-7, 4.2)
ax.set_xlabel("Difference in HbA1c change, video − control (mmol/mol), 95% CI")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.tick_params(axis="y", length=0)
ax.spines["left"].set_visible(False)
h1 = plt.Line2D([], [], marker="o", color=ORANGE, lw=2, ms=MS, label="printed in the paper")
h2 = plt.Line2D([], [], marker="D", color=BLUE, lw=2, ms=MS - 1, label="recomputed from the paper's numbers")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.25), ncol=2, frameon=False, fontsize=10)
ax.set_title("HbA1c difference −2.01 mmol/mol: the printed CI crosses zero", loc="left", fontsize=11.5)
save(fig, *paths)
def design_C(paths):
"""Same five rows on a linear |z| axis: z the printed p requires vs z the CI gives."""
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ys = np.arange(len(TESTED))[::-1]
for y, n in zip(ys, TESTED):
r = REC[n]
zp = z_from_p(p_value_of(r["printed"]))
zc = abs(r["z"])
ax.plot([zc, zp], [y, y], color=INK2, lw=1, alpha=0.5)
mk = ">" if r["printed"].startswith("<") else "o"
ax.plot(zp, y, marker=mk, ms=MS + 1, color=ORANGE, mec=BG, mew=1.5, ls="none", zorder=3)
ax.plot(zc, y, marker="D", ms=MS - 1, color=BLUE, mec=BG, mew=1.5, ls="none", zorder=3)
ax.annotate(f"{'≥' if mk == '>' else ''}{zp:.2f}", (zp, y), xytext=(0, 9), textcoords="offset points",
ha="center", fontsize=10)
ax.annotate(f"{zc:.2f}", (zc, y), xytext=(0, 9), textcoords="offset points", ha="center", fontsize=10)
ax.axvline(Z975, color=INK2, lw=1)
ax.text(Z975, len(TESTED) - 0.35, "|z| = 1.96 (p = 0.05)", ha="center", va="bottom", fontsize=10, color=INK2,
bbox=MASK)
ax.set_xlim(0, 4)
ax.set_yticks(ys)
ax.set_yticklabels([LABELS[n] for n in TESTED], color=INK)
ax.set_ylim(-0.6, len(TESTED) - 0.1)
ax.set_xlabel("|z| = |difference| / SE")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.tick_params(axis="y", length=0)
ax.spines["left"].set_visible(False)
h1 = plt.Line2D([], [], marker="o", color=ORANGE, ls="none", ms=MS, label="|z| the printed p requires ('>' = at least)")
h2 = plt.Line2D([], [], marker="D", color=BLUE, ls="none", ms=MS - 1, label="|z| the printed 95% CI gives")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
ax.set_title("Test statistic implied by each printed p value and by its CI", loc="left", fontsize=11.5)
save(fig, *paths)
def design_D(paths):
"""Graphic table: rows with the printed CI drawn in SE units, printed p and CI-implied p."""
fig, ax = plt.subplots(figsize=(7.2, 3.6))
ys = np.arange(len(TESTED))[::-1]
for y, n in zip(ys, TESTED):
r = REC[n]
zc = r["z"]
ax.plot([zc - Z975, zc + Z975], [y, y], color=ORANGE, lw=2, solid_capstyle="round")
ax.plot(zc, y, "o", ms=MS, color=ORANGE, mec=BG, mew=1.5)
ax.plot(5.2, y, "o", ms=MS, color=ORANGE, mec=BG)
ax.text(5.6, y, r["printed"], ha="left", va="center", fontsize=10.5, color=INK)
ax.plot(8.0, y, "D", ms=MS - 1, color=BLUE, mec=BG)
ax.text(8.4, y, fmt_p(r["p"]), ha="left", va="center", fontsize=10.5, color=INK, weight="bold")
ax.axvline(0, color=INK2, lw=1)
top = len(TESTED) - 0.45
for x, t in [(0, "95% CI in SE units"), (5.9, "Printed p"), (8.6, "p from CI")]:
ax.text(x, top, t, ha="center", va="bottom", fontsize=10, color=INK2, weight="bold", bbox=MASK)
ax.set_xlim(-3.6, 10.0)
ax.set_ylim(-0.6, len(TESTED) - 0.05)
ax.set_yticks(ys)
ax.set_yticklabels([mn(f"{LABELS[n]}\n{r['d']:.2f} ({r['lo']:.2f}, {r['hi']:.2f})")
for n, r in ((n, REC[n]) for n in TESTED)], color=INK, fontsize=10)
ax.set_xticks([-2, 0, 2])
ax.set_xticklabels(["−2", "0", "+2"])
ax.tick_params(axis="y", length=0)
for s in ("left", "bottom"):
ax.spines[s].set_visible(False)
ax.set_xlabel("CI drawn as (difference ÷ SE) ± 1.96; every interval crosses 0", fontsize=10, color=INK2)
h1 = plt.Line2D([], [], marker="o", color=ORANGE, lw=2, ms=MS, label="printed in Table 2")
h2 = plt.Line2D([], [], marker="D", color=BLUE, ls="none", ms=MS - 1, label="recomputed from the printed CI")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, *paths)
def design_E(paths):
"""Calculation flow from the printed HbA1c CI to its p value, set against the printed p and the claims."""
fig, ax = plt.subplots(figsize=(7.2, 4.4))
ax.set_xlim(0, 10)
ax.set_ylim(-0.1, 6.2)
ax.axis("off")
def box(x, y, w, hgt, text, edge, bold=False, fs=10.5):
ax.add_patch(FancyBboxPatch((x, y), w, hgt, boxstyle="round,pad=0.02,rounding_size=0.12",
fc=BG, ec=edge, lw=2))
ax.text(x + w / 2, y + hgt / 2, text, ha="center", va="center", fontsize=fs, color=INK,
weight="bold" if bold else "normal", wrap=True)
def arrow(x0, y0, x1, y1, col=INK2):
ax.annotate("", (x1, y1), (x0, y0), arrowprops=dict(arrowstyle="-|>", color=col, lw=1.5))
box(0.1, 4.7, 3.1, 1.1, "Table 2, printed\n−2.01 (−5.32, 1.29)", ORANGE, bold=True)
box(3.6, 4.7, 2.9, 1.1, f"CI half-width\n(1.29 + 5.32) / 2\n= {(hi - lo) / 2:.3f}", BLUE, fs=10)
box(6.9, 4.7, 3.0, 1.1, f"SE = {(hi - lo) / 2:.3f} / 1.96\n= {h['se']:.3f}", BLUE)
box(6.9, 3.0, 3.0, 1.1, mn(f"z = −2.01 / {h['se']:.3f}\n= {h['z']:.2f}"), BLUE)
box(3.6, 3.0, 2.9, 1.1, f"two-sided p\n= {h['p']:.2f}", BLUE, bold=True)
box(0.1, 3.0, 3.1, 1.1, "Table 2, printed\np < 0.001", ORANGE, bold=True)
arrow(3.2, 5.25, 3.6, 5.25)
arrow(6.5, 5.25, 6.9, 5.25)
arrow(8.4, 4.7, 8.4, 4.1)
arrow(6.9, 3.55, 6.5, 3.55)
ax.text(3.4, 3.55, "≠", ha="center", va="center", fontsize=20, color=INK, weight="bold")
ax.text(1.6, 2.8, f"conflict: p < 0.001 needs SE ≤ {se_need:.2f},\n"
f"i.e. CI half-width ≤ {se_need * Z975:.2f} (printed {(hi - lo) / 2:.3f})",
ha="left", va="top", fontsize=10, color=INK)
box(0.1, 0.0, 9.8, 1.75,
"Claims that rest on the printed p\n"
"Abstract: “the video group was superior in terms of treatment satisfaction\nand reduction in HbA1c”\n"
"Discussion: “We did find a statistically significant improvement in HbA1c”", INK2, fs=10)
arrow(1.0, 3.0, 1.0, 1.7, ORANGE)
h1 = plt.Line2D([], [], color=ORANGE, lw=2, label="printed in the paper")
h2 = plt.Line2D([], [], color=BLUE, lw=2, label="recomputed from the paper's numbers")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, 1.06), ncol=2, frameon=False, fontsize=10)
save(fig, *paths)
design_A([os.path.join(HERE, "fig1"), os.path.join(CAND, "cand_A_pvalue_dumbbell")])
design_B([os.path.join(HERE, "fig2"), os.path.join(CAND, "cand_B_hba1c_intervals")])
design_C([os.path.join(CAND, "cand_C_z_dumbbell")])
design_D([os.path.join(CAND, "cand_D_graphic_table")])
design_E([os.path.join(CAND, "cand_E_calculation_flow")])
def contact_sheet(path):
"""Five candidate designs side by side, labelled A-E, for choosing one."""
names = ["cand_A_pvalue_dumbbell", "cand_B_hba1c_intervals", "cand_C_z_dumbbell",
"cand_D_graphic_table", "cand_E_calculation_flow"]
titles = ["A p values, log axis (fig1)", "B HbA1c intervals (fig2)", "C |z| statistic, linear axis",
"D graphic table, CI in SE units", "E calculation flow, HbA1c"]
fig, axs = plt.subplots(3, 2, figsize=(14, 15))
for ax in axs.flat:
ax.axis("off")
for ax, n, t in zip(axs.flat, names, titles):
ax.imshow(plt.imread(os.path.join(CAND, n + ".png")))
ax.set_title(t, loc="left", fontsize=13, weight="bold", color=INK)
fig.tight_layout()
fig.savefig(path, dpi=110, facecolor=BG)
plt.close(fig)
contact_sheet(os.path.join(CAND, "contact_sheet.png"))
print("\nFigures written: fig1, fig2, candidates/cand_A ... cand_E (.svg and .png) and candidates/contact_sheet.png")
Abstract reports p<0.001 for the matched cystic fibrosis versus rheumatoid arthritis comparison; Table 4 gives p=0.055
Eur Respir J (2023) · doi:10.1183/13993003.00174-2023 · PMC10600351
Summary
The abstract reports the propensity-matched comparison of major adverse cardiac events (MACE) between people with cystic fibrosis (CF) and people with rheumatoid arthritis (RA) as HR 1.21 (95% CI 1.00–1.48) with p<0.001. Table 4 gives the same HR and CI for this comparison, with p=0.055. Back-calculating from the printed HR and CI gives p = 0.057, or 0.047–0.068 if the rounded values vary within their rounding ranges. That agrees with Table 4 and rules out p<0.001, which would need a CI of about 1.08–1.36. The only p<0.001 in the RA row of Table 4 belongs to the unadjusted analysis (HR 0.75), so the abstract presents a borderline, non-significant comparison as highly significant. The Results and Discussion also describe CF risk as "greater than" RA risk. The paper's main finding, a higher MACE risk in CF than in matched general-population comparators (HR 1.65, p<0.001), is not affected.
What the paper reports
Abstract, Results:
In propensity score-matched analyses, PwCF had higher risk of MACE than matched general population comparators (hazard ratio (HR) 1.65, 95% CI 1.40–1.95; p<0.001) and an equivalent or higher relative risk compared with other inflammatory conditions considered “high risk” for cardiovascular disease, including systemic lupus erythematosus (HR 0.95, 95% CI 0.82–1.09; p=0.44), rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p<0.001) and HIV (HR 0.93, 95% CI 0.82–1.06; p=0.29).
Table 4, "Rheumatoid arthritis" row. The columns are: unadjusted analysis (CF | events | non-CF | events | HR (95% CI) | p-value), then propensity score-matched analysis (the same six columns):
Rheumatoid arthritis | 6265 | 227 | 97 198 | 4547 | 0.75 (0.66–0.86) | <0.001 | 5901 | 223 | 5901 | 177 | 1.21 (1.00–1.48) | 0.055
Table 4 footer:
Propensity score matching based on known cardiac risk factors (age, sex, diabetes, hypertension, hypercholesterolaemia, family history of heart disease, smoking and chronic kidney disease).
Methods, "Statistical analysis":
After propensity score matching, a Cox's proportional hazards model was used to compare the matched cohorts.
Results, "Outcomes compared with other inflammatory diseases":
In CF, risk of MACE was equivalent to SLE and HIV (HR 0.95, 95% CI 0.82–1.09; p=0.44; and HR 0.93, 95% CI 0.82–1.06; p=0.29, respectively) and was greater than rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48).
Discussion, first paragraph:
Furthermore, cardiac risk in PwCF was equivalent to or higher than other chronic inflammatory diseases considered “high risk” for cardiac disease.
Discussion, fifth paragraph (the paragraph beginning "Chronic inflammation is a hallmark of CF"):
Here, we found that despite adjusting for the presence of classic cardiac risk factors, risk of cardiac disease in CF was equivalent to or greater than that of other inflammatory conditions.
What the paper's own numbers show
Method. Hazard ratios from a Cox model are normally reported with a Wald 95% CI on the log scale. For such a CI, SE(log HR) = (ln upper − ln lower) / (2 × 1.96), z = ln HR / SE, and the two-sided p = 2(1 − Φ(|z|)). This cannot reproduce the exact Cox p-value without the individual event times, but it is the standard check that a printed HR, CI and p-value are consistent. I applied it to the four matched comparisons the abstract reports. To allow for rounding, I also varied the RA HR and both bounds over every value that rounds to the printed two decimals. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| CF vs RA, matched: p-value | Abstract, Results | HR 1.21 (1.00–1.48); p<0.001 | SE(log HR) 0.100, z = 1.91, p = 0.057 |
| CF vs RA, matched: p-value | Table 4 | HR 1.21 (1.00–1.48); p = 0.055 | p = 0.057 (agrees) |
| CF vs RA, matched: p over rounding of HR and CI | Table 4 / Abstract | — | 0.047–0.068 |
| CF vs RA, matched: HR against CI midpoint | Table 4 / Abstract | 1.21 | geometric midpoint √(1.00 × 1.48) = 1.217 (consistent) |
| CF vs RA, matched: what p<0.001 would need | — | — | |z| > 3.29, SE ≤ 0.058, so a 95% CI no wider than 1.08–1.36 |
| CF vs RA, matched: one-sided p | — | — | 0.028 |
| CF vs RA, unadjusted | Table 4 | HR 0.75 (0.66–0.86); p<0.001 | z = −4.26, p = 2.0 × 10⁻⁵ (agrees); the only p<0.001 in the RA row |
| CF vs RA, matched: crude event counts | Table 4 | CF 223/5901 vs RA 177/5901 | 3.78% vs 3.00%; crude risk ratio 1.26 (1.04–1.53), p = 0.020 (ignores time to event and censoring, so it does not replace the Cox p) |
| CF vs SLE, matched (cross-check) | Abstract / Table 4 | HR 0.95 (0.82–1.09); p = 0.44 / 0.439 | p = 0.48 (rough agreement, within rounding of the CI) |
| CF vs HIV, matched (cross-check) | Abstract / Table 4 | HR 0.93 (0.82–1.06); p = 0.29 / 0.293 | p = 0.27 (rough agreement) |
| CF vs general population, matched (cross-check) | Abstract / Table 3 | HR 1.65 (1.40–1.95); p<0.001 | z = 5.92, p = 3 × 10⁻⁹ (agrees) |
The same back-calculation agrees with the printed p-values for the other three matched comparisons. For RA it agrees with Table 4 but not with the abstract. A CI whose lower bound is 1.00 is by construction close to p = 0.05, and a two-sided p<0.001 would require the lower bound to be at least about 1.08.
Figure
Figure 1. P-values for the four propensity-matched MACE comparisons named in the abstract, on one log-scale axis. Orange diamonds are the values printed in the abstract and orange squares the values printed in Table 3 (general population) or Table 4 (SLE, RA, HIV). A left-pointing arrow marks a value printed as "p<". Blue circles are the p-values recomputed from the printed HR and 95% CI. For RA, the blue bar spans the p-values obtained when the HR and both CI bounds vary within their rounding ranges (0.047–0.068). The dashed line marks p = 0.05 and the dotted line p = 0.001. For the general population, SLE and HIV the three values agree. For RA, Table 4 and the recomputed value sit on the 0.05 line, while the abstract places the result beyond 0.001. The recomputed general-population p (3 × 10⁻⁹) lies beyond the left edge of the axis.
Figure 2. Where the abstract's RA values occur in Table 4. The abstract's HR and CI (1.21, 1.00–1.48) are those of the propensity-matched columns, whose p-value is 0.055. The only p<0.001 in the RA row is in the unadjusted columns, for an HR of 0.75 (0.66–0.86) that points the other way. The figure shows where the printed values coincide. It does not show how the abstract's value arose.
Why it matters
- The abstract misstates the significance of a highlighted comparison. The abstract lists RA among the inflammatory conditions against which CF has "an equivalent or higher relative risk", and gives p<0.001 for it. The paper's own table, and the HR and CI themselves, put this comparison at p ≈ 0.055–0.057, just above the conventional 0.05 threshold. Readers who see only the abstract, or who index it, will take the RA comparison as strong evidence of excess risk in CF.
- The "greater than" claims rest on this comparison alone. The Results say CF risk "was greater than rheumatoid arthritis". The first and fifth Discussion paragraphs say CF risk was "equivalent to or higher than" or "equivalent to or greater than" that of other inflammatory conditions. SLE and HIV are both described as equivalent (p = 0.44 and 0.29), so the "higher" or "greater" part depends only on RA. With a CI whose lower bound is 1.00 and p = 0.055, the data are compatible with no difference, so "greater than" is not established at the 0.05 level. The point estimate still favours higher risk in CF.
- What is not affected. The comparison with matched general-population comparators (HR 1.65, 1.40–1.95, p<0.001), the SLE and HIV comparisons, the UK CF Registry risk-factor analyses, and the abstract's conclusion that "PwCF are at increased risk of adverse cardiac disease events" do not depend on this p-value. The argument in the Discussion that risk tools such as QRISK may need CF-specific adjustment, as they have for SLE, RA and HIV, holds equally well if CF risk is similar to RA risk.
Explanations considered
- A different analysis. Table 4 does contain a p<0.001 for RA, but in the unadjusted columns, with HR 0.75 (0.66–0.86). That HR points the other way (lower MACE risk in CF than in RA). The abstract pairs a p<0.001 with the matched HR and CI. The coincidence is shown in Figure 2. It would explain how the two values came to differ but would not make the abstract's p-value correct for the matched comparison it is attached to.
- Rounding. Letting the HR and both CI bounds take any value that rounds to 1.21, 1.00 and 1.48 gives p between 0.047 and 0.068. That is nowhere near 0.001, and the table's three-decimal p of 0.055 sits inside this range. Rounding does not account for the discrepancy.
- A one-sided test. The Methods mention no one-sided tests. A one-sided p from the same HR and CI would be 0.028, still not <0.001.
- Another model or population. The Methods describe a single approach: 1:1 propensity matching followed by a Cox proportional hazards model on the matched cohorts. The Table 4 footer lists the matching covariates and mentions no other model. Supplementary Table S4 (CF vs RA) gives the same matched MACE result, HR 1.21 (1.00–1.48), p = 0.055. No other analysis in the paper or supplement reports the matched RA comparison with p<0.001.
- Crude counts. Comparing 223/5901 with 177/5901 events without time to event gives p = 0.020. That is not <0.001 either, and it is not the analysis the paper reports.
- Conversion artifact. The publisher-typeset one-page summary PDF in the supplementary bundle and the Europe PMC full-text XML print the same abstract p<0.001 and the same Table 4 p = 0.055. The discrepancy is in the published article, not an artifact of the PMC conversion.
- A later correction. None was found (see Status).
None of these accounts for p<0.001. The Table 4 value of 0.055 agrees with the recomputation, so the abstract's value is the one in error.
A side observation, not part of this finding: the Table 4 SLE matched row lists the CF group as n = 223 with 385 events and the comparator as n = 5223. The Results text states that 5223 PwCF were matched with SLE comparators, so the "223" looks like a dropped digit. This was not examined further.
Status
As of 2026-10-23:
- Version of record. The publisher's pages (publications.ersnet.org, erj.ersjournals.com) returned HTTP 403 to automated access. The publisher-typeset one-page summary PDF (ERJ-00174-2023.Shareable.pdf, from the Europe PMC supplementary bundle) prints the same abstract text, "rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p<0.001)". The Europe PMC JATS XML and the live PMC page carry the same abstract and the same Table 4 matched RA p-value of 0.055.
- Supplement. ERJ-00174-2023.Supplement.pdf (17 pages). Supplementary Table S4 (CF vs RA) gives the matched MACE row as CF 5901 / 223 events vs RA 5901 / 177 events, HR 1.21 (1.00–1.48), p 0.055, and the unadjusted row as HR 0.75 (0.66–0.86), p<0.001. The supplement agrees with Table 4 and does not resolve the discrepancy.
- Corrections. Crossref lists no update-to or updated-by relations for the DOI. PMC and Europe PMC list no correction, erratum, retraction or expression of concern, and a Europe PMC search for errata returned none. The only linked item is an editorial, "Cystic fibrosis and the cardiovascular system: the unexpected heartache" (doi:10.1183/13993003.01253-2023; PMID 37884293), which is commentary rather than a correction.
- PubPeer and other commentary. PubPeer blocked automated access (403 and a Cloudflare challenge). A web search found no PubPeer thread or Retraction Watch entry for this article.
- Sources. The article as archived in PMC (PMC10600351: abstract, Methods, Results, Tables 3 and 4, Discussion); the Europe PMC full-text XML and supplementary files; Crossref, PMC and Europe PMC notice records for doi:10.1183/13993003.00174-2023.
Suggested correction
In the Abstract, Results, replace "rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p<0.001)" with "rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p=0.055)", in line with Table 4 and Supplementary Table S4.
Because this comparison does not reach p<0.05, the authors may also wish to revise the wording that reads it as an established difference. In the Results, "and was greater than rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48)" could read "and was numerically higher than, but not significantly different from, rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p=0.055)". The phrases "equivalent or higher" (Abstract), "equivalent to or higher than" (Discussion, first paragraph) and "equivalent to or greater than" (Discussion, fifth paragraph) could be qualified in the same way.
Recomputation script (python recompute.py.txt)
"""Recompute the p-values of the propensity-matched MACE comparisons in PMC10600351
(Eur Respir J 2023, doi:10.1183/13993003.00174-2023).
Run: python recompute.py.txt # prints all numbers, writes fig1/fig2 (.svg, .png)
python recompute.py.txt --candidates # also writes five candidate figure designs
# to candidates/ (A-E) for review
Needs numpy, scipy and matplotlib.
Inputs are copied from the paper as archived in PMC:
Abstract (Results), Results "Outcomes compared with other inflammatory diseases",
Table 3 (CF vs general population) and Table 4 (CF vs SLE, RA, HIV).
Method: for a ratio reported with a 95% Wald CI on the log scale,
SE(log HR) = (ln upper - ln lower) / (2 * 1.959964), z = ln HR / SE,
two-sided p = 2 * (1 - Phi(|z|)).
The exact Cox p-value cannot be rebuilt without individual event times; this
back-calculation is the standard check of internal consistency.
"""
import itertools
import os
import sys
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.ppf(0.975)
# ----------------------------------------------------------------- inputs (from the paper)
# name: (HR, lower, upper, p in Table 3/4, p in abstract)
MATCHED = {
"General population": (1.65, 1.40, 1.95, "<0.001", "<0.001"), # Table 3, abstract
"SLE": (0.95, 0.82, 1.09, "0.439", "0.44"), # Table 4, abstract
"Rheumatoid arthritis": (1.21, 1.00, 1.48, "0.055", "<0.001"), # Table 4, abstract
"HIV": (0.93, 0.82, 1.06, "0.293", "0.29"), # Table 4, abstract
}
RA_UNADJ = (0.75, 0.66, 0.86, "<0.001") # Table 4, RA row, unadjusted columns
RA_MATCHED_EVENTS = (223, 5901, 177, 5901) # Table 4: CF events/n, RA events/n
SLE_ROW_N = (223, 385, 5223, 389) # Table 4 SLE matched: CF n, events, non-CF n, events
def p_from_ci(hr, lo, hi):
se = (np.log(hi) - np.log(lo)) / (2 * Z975)
z = np.log(hr) / se
return se, z, 2 * norm.sf(abs(z))
def fmt_p(p):
return f"{p:.2g}" if p < 0.001 else f"{p:.3f}"
# ----------------------------------------------------------------- 1. back-calculated p
print("1. p-values back-calculated from the reported HR and 95% CI (matched analyses)")
RES = {}
for k, (h, lo, hi, p_tab, p_abs) in MATCHED.items():
se, z, p = p_from_ci(h, lo, hi)
RES[k] = (se, z, p)
print(f" {k:21s} HR {h:.2f} ({lo:.2f}-{hi:.2f}): SE(logHR)={se:.4f} z={z:+.3f} "
f"p={fmt_p(p)} table {p_tab}, abstract {p_abs}")
print(f" General population p = {RES['General population'][2]:.1e}")
# ----------------------------------------------------------------- 2. rounding range for RA
h0, l0, u0 = MATCHED["Rheumatoid arthritis"][:3]
grid = []
for h, l, u in itertools.product(np.linspace(h0 - .005, h0 + .005, 21),
np.linspace(l0 - .005, l0 + .005, 21),
np.linspace(u0 - .005, u0 + .005, 21)):
grid.append(p_from_ci(h, l, u)[2])
P_LO, P_HI = min(grid), max(grid)
print("\n2. RA matched: p over every HR, lower and upper bound consistent with rounding to 2 dp")
print(f" p range {P_LO:.4f} - {P_HI:.4f}")
print(f" geometric midpoint of CI sqrt(1.00*1.48) = {np.sqrt(l0 * u0):.3f} (reported HR 1.21)")
print(f" one-sided p = {RES['Rheumatoid arthritis'][2] / 2:.4f}")
# ----------------------------------------------------------------- 3. what p<0.001 would need
z001 = norm.isf(0.0005)
se_need = np.log(h0) / z001
CI_NEED = (h0 * np.exp(-Z975 * se_need), h0 * np.exp(Z975 * se_need))
print("\n3. RA matched: what a two-sided p<0.001 would require at HR 1.21")
print(f" |z| > {z001:.3f}; SE(logHR) <= {se_need:.4f} (reported CI implies "
f"{RES['Rheumatoid arthritis'][0]:.4f}); 95% CI would be at most "
f"{CI_NEED[0]:.2f}-{CI_NEED[1]:.2f}")
# ----------------------------------------------------------------- 4. the unadjusted RA row
se_u, z_u, p_u = p_from_ci(*RA_UNADJ[:3])
print("\n4. RA unadjusted (Table 4): HR 0.75 (0.66-0.86), reported p<0.001")
print(f" SE={se_u:.4f} z={z_u:+.3f} p={p_u:.1e} -> the only p<0.001 in the RA row")
# ----------------------------------------------------------------- 5. crude counts
a, n1, b, n2 = RA_MATCHED_EVENTS
rr = (a / n1) / (b / n2)
se_rr = np.sqrt(1 / a - 1 / n1 + 1 / b - 1 / n2)
z_rr = np.log(rr) / se_rr
CRUDE = (rr, rr * np.exp(-Z975 * se_rr), rr * np.exp(Z975 * se_rr), 2 * norm.sf(abs(z_rr)))
print("\n5. RA matched event counts 223/5901 vs 177/5901 (ignores time to event)")
print(f" risk {a / n1 * 100:.2f}% vs {b / n2 * 100:.2f}%; crude risk ratio {CRUDE[0]:.3f} "
f"(95% CI {CRUDE[1]:.3f}-{CRUDE[2]:.3f}), p={CRUDE[3]:.4f}")
# ----------------------------------------------------------------- 6. side note (not part of the claim)
print("\n6. Side note: Table 4 SLE matched row lists CF n=%d with %d events, against %d "
"non-CF; the Results text says 5223 PwCF were matched" % SLE_ROW_N[:3])
# ================================================================= figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, path):
fig.savefig(path + ".svg", bbox_inches="tight")
fig.savefig(path + ".png", dpi=200, bbox_inches="tight")
plt.close(fig)
def style_ax(ax, grid_axis="x"):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis=grid_axis, color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
def fig_pvalue_axis(path):
"""Design A: reported vs recomputed p-values on one log axis, with alpha 0.05."""
order = ["General population", "SLE", "Rheumatoid arthritis", "HIV"]
labels = {"General population": "CF vs general population\nHR 1.65 (1.40–1.95)",
"SLE": "CF vs SLE\nHR 0.95 (0.82–1.09)",
"Rheumatoid arthritis": "CF vs rheumatoid arthritis\nHR 1.21 (1.00–1.48)",
"HIV": "CF vs HIV\nHR 0.93 (0.82–1.06)"}
XMIN, XMAX = 1e-5, 1.0
fig, ax = plt.subplots(figsize=(7.2, 4.8))
ys = {k: len(order) - 1 - i for i, k in enumerate(order)}
ax.axvline(0.05, color=INK2, lw=1, ls=(0, (4, 3)))
ax.text(0.05 * 1.12, 3.5, "p = 0.05", color=INK2, fontsize=10, va="center", ha="left")
ax.axvline(0.001, color=INK2, lw=0.8, ls=":")
ax.text(0.001 * 1.12, 3.5, "p = 0.001", color=INK2, fontsize=10, va="center", ha="left")
D = 0.24
def lt_arrow(x, y, mk):
ax.annotate("", xy=(x / 6, y), xytext=(x, y),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2))
ax.plot(x, y, mk, ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=4)
for k in order:
y = ys[k]
p_tab, p_abs = MATCHED[k][3], MATCHED[k][4]
for val, mk, yy in ((p_abs, "D", y + D), (p_tab, "s", y)):
if val.startswith("<"):
lt_arrow(float(val[1:]), yy, mk)
else:
ax.plot(float(val), yy, mk, ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=4)
yb = y - D
p_rec = RES[k][2]
if p_rec < XMIN:
ax.annotate("", xy=(XMIN, yb), xytext=(XMIN * 8, yb),
arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=2))
ax.text(XMIN * 11, yb, f"recomputed: {p_rec:.0e} (beyond axis)", color=INK,
fontsize=10, va="center")
continue
if k == "Rheumatoid arthritis":
ax.plot([P_LO, P_HI], [yb, yb], color=BLUE, lw=2, solid_capstyle="butt")
for xv in (P_LO, P_HI):
ax.plot([xv, xv], [yb - .08, yb + .08], color=BLUE, lw=2)
ax.plot(p_rec, yb, "o", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=4)
y = ys["Rheumatoid arthritis"]
ax.text(0.001 / 9, y + D, "abstract: p<0.001 ", color=INK, fontsize=10,
ha="right", va="center", fontweight="bold")
ax.text(0.055 / 1.4, y, "Table 4: 0.055", color=INK, fontsize=10, va="center", ha="right")
ax.text(P_LO / 1.25, y - D, f"recomputed: {RES['Rheumatoid arthritis'][2]:.3f} "
f"(rounding range {P_LO:.3f}–{P_HI:.3f})", color=INK, fontsize=10, va="center",
ha="right")
ax.set_yticks([ys[k] for k in order], [labels[k] for k in order])
ax.set_xscale("log")
ax.set_xlim(XMIN, XMAX)
ax.set_xticks([1e-5, 1e-4, 1e-3, 1e-2, 0.05, 1e-1, 1],
["0.00001", "0.0001", "0.001", "0.01", "0.05", "0.1", "1"])
ax.minorticks_off()
ax.set_ylim(-0.5, 3.75)
ax.set_xlabel("Two-sided p-value (log scale)")
style_ax(ax)
h = [plt.Line2D([], [], marker="D", ls="", ms=8, color=ORANGE, label="Reported in abstract\n(arrow: “p<” value)"),
plt.Line2D([], [], marker="s", ls="", ms=8, color=ORANGE, label="Reported in Table 3 / 4"),
plt.Line2D([], [], marker="o", ls="-", lw=2, ms=8, color=BLUE,
label="Recomputed from HR and 95% CI\n(bar: range over rounding)")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.4, -0.15), ncol=3,
frameon=False, fontsize=9.5, handletextpad=0.4, columnspacing=1.2)
ax.set_title("Propensity-matched MACE comparisons: reported and recomputed p-values",
loc="left", fontsize=11.5, color=INK, x=-0.38, pad=18)
save(fig, path)
def box(ax, x, y, w, h, title, body, edge, fs=10.5):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.01,rounding_size=0.015",
fc="#ffffff", ec=edge, lw=2, transform=ax.transAxes,
clip_on=False))
ax.text(x + 0.02, y + h - 0.045, title, transform=ax.transAxes, fontsize=11,
fontweight="bold", va="top", color=INK)
ax.text(x + 0.02, y + h - 0.13, body, transform=ax.transAxes, fontsize=fs,
va="top", color=INK, linespacing=1.45)
def fig_text_panels(path):
"""Design B: abstract vs Table 4 excerpts with a connector naming the conflict."""
fig = plt.figure(figsize=(7.2, 3.3))
ax = fig.add_axes([0.02, 0.12, 0.96, 0.84])
ax.set_axis_off()
box(ax, 0.0, 0.36, 0.44, 0.60, "Abstract, Results",
"“rheumatoid arthritis\n(HR 1.21, 95% CI 1.00–1.48;\n p<0.001)”", ORANGE, fs=11)
box(ax, 0.56, 0.36, 0.44, 0.60, "Table 4, propensity-matched",
"Rheumatoid arthritis\nCF 223/5901 vs RA 177/5901\nHR 1.21 (1.00–1.48)\np-value 0.055",
ORANGE, fs=11)
ax.annotate("", xy=(0.555, 0.66), xytext=(0.445, 0.66), xycoords="axes fraction",
arrowprops=dict(arrowstyle="<|-|>", color=INK2, lw=2))
ax.text(0.5, 0.71, "same HR\nand CI", ha="center", fontsize=10, color=INK2,
transform=ax.transAxes)
ax.text(0.5, 0.61, "different\np-value", ha="center", fontsize=10, color=INK,
fontweight="bold", transform=ax.transAxes, va="top")
ax.text(0.0, 0.18, f"Recomputed from HR 1.21 and CI 1.00–1.48: p = "
f"{RES['Rheumatoid arthritis'][2]:.3f} (range {P_LO:.3f}–{P_HI:.3f} over rounding).",
transform=ax.transAxes, fontsize=10.5, color=BLUE, fontweight="bold")
ax.text(0.0, 0.06, "Orange boxes: text as printed in the paper. Blue: recomputed here.",
transform=ax.transAxes, fontsize=9.5, color=INK2)
save(fig, path)
def fig_hr_needed(path):
"""Design C: reported CI vs the widest CI compatible with p<0.001, on a log HR axis."""
fig, ax = plt.subplots(figsize=(7.2, 2.6))
rows = [("Reported (abstract and Table 4)\nHR 1.21, 95% CI 1.00–1.48", (l0, u0), ORANGE),
(f"Widest 95% CI that would give\np<0.001 at HR 1.21: "
f"{CI_NEED[0]:.2f}–{CI_NEED[1]:.2f}", CI_NEED, BLUE)]
ax.axvline(1, color=INK2, lw=1)
ax.text(1.005, 1.55, "HR = 1 (no difference)", color=INK2, fontsize=10, ha="left")
for i, (lab, (lo, hi), c) in enumerate(rows):
y = 1 - i
ax.plot([lo, hi], [y, y], color=c, lw=2)
ax.plot(h0, y, "o", ms=9, color=c, mec=BG, mew=1.5)
ax.text(hi * 1.02, y, f"{lo:.2f}–{hi:.2f}", color=INK, fontsize=10, va="center")
ax.set_yticks([1, 0], [r[0] for r in rows])
ax.set_xscale("log")
ax.set_xlim(0.9, 1.7)
ax.set_xticks([0.9, 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6], ["0.9", "1.0", "1.1", "1.2", "1.3",
"1.4", "1.5", "1.6"])
ax.minorticks_off()
ax.set_ylim(-0.5, 1.8)
ax.set_xlabel("Hazard ratio, CF vs rheumatoid arthritis (propensity-matched, log scale)")
style_ax(ax)
save(fig, path)
def fig_z(path):
"""Design D: |z| implied by each reported CI against the 0.05 and 0.001 thresholds."""
order = ["General population", "SLE", "Rheumatoid arthritis", "HIV"]
fig, ax = plt.subplots(figsize=(7.2, 3.0))
for i, k in enumerate(order):
y = len(order) - 1 - i
z = abs(RES[k][1])
ax.plot([0, z], [y, y], color=BLUE, lw=2)
ax.plot(z, y, "o", ms=8, color=BLUE, mec=BG, mew=1.5)
ax.text(z + 0.12, y, f"|z| = {z:.2f}", va="center", fontsize=10)
yra = 1
ax.plot(z001, yra + 0.28, "D", ms=8, color=ORANGE)
ax.text(z001 + 0.12, yra + 0.28, "abstract's p<0.001 needs |z| > 3.29", color=INK,
fontsize=10, va="center")
for zz, lab in ((Z975, "p = 0.05"), (z001, "p = 0.001")):
ax.axvline(zz, color=INK2, lw=1, ls=(0, (4, 3)))
ax.text(zz + 0.06, 3.55, lab, ha="left", fontsize=10, color=INK2)
ax.set_yticks(range(4)[::-1], ["vs general population", "vs SLE", "vs rheumatoid arthritis",
"vs HIV"])
ax.set_xlim(0, 6.8)
ax.set_ylim(-0.5, 3.8)
ax.set_xlabel("|z| implied by the reported HR and 95% CI (blue, recomputed)")
style_ax(ax)
save(fig, path)
def fig_row_origin(path):
"""Design E: the Table 4 RA row with both columns; the abstract's p matches the wrong one."""
fig = plt.figure(figsize=(7.2, 3.7))
ax = fig.add_axes([0.02, 0.1, 0.96, 0.8])
ax.set_axis_off()
box(ax, 0.01, 0.58, 0.46, 0.38, "Table 4, RA: unadjusted",
"HR 0.75 (0.66–0.86)\np-value <0.001", INK2, fs=11)
box(ax, 0.53, 0.58, 0.46, 0.38, "Table 4, RA: propensity-matched",
"HR 1.21 (1.00–1.48)\np-value 0.055", INK2, fs=11)
box(ax, 0.12, 0.04, 0.76, 0.30, "Abstract, Results",
"“rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p<0.001)”", ORANGE, fs=10.5)
ax.annotate("", xy=(0.66, 0.36), xytext=(0.76, 0.565), xycoords="axes fraction",
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=2))
ax.text(0.73, 0.46, "HR and CI as printed here", fontsize=10, color=INK2,
transform=ax.transAxes, va="center")
ax.annotate("", xy=(0.34, 0.36), xytext=(0.24, 0.565), xycoords="axes fraction",
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, ls="--"))
ax.text(0.27, 0.46, "the row's only p<0.001\n(a different analysis, HR < 1)",
fontsize=10, color=INK, transform=ax.transAxes, va="center", ha="right")
ax.text(0.01, -0.07, f"Recomputed from the matched HR and CI: p = "
f"{RES['Rheumatoid arthritis'][2]:.3f}, consistent with Table 4's 0.055 (blue = "
"recomputed here).", transform=ax.transAxes, fontsize=10, color=BLUE)
ax.text(0.01, 1.04, "Where the abstract's RA values match Table 4", transform=ax.transAxes,
fontsize=11.5, color=INK)
save(fig, path)
# write the chosen figures
fig_pvalue_axis(os.path.join(HERE, "fig1"))
fig_row_origin(os.path.join(HERE, "fig2"))
print("\nwrote fig1.svg/png (p-value axis) and fig2.svg/png (Table 4 row vs abstract)")
if "--candidates" in sys.argv:
cdir = os.path.join(HERE, "candidates")
os.makedirs(cdir, exist_ok=True)
for tag, fn in (("A_pvalue_axis", fig_pvalue_axis), ("B_text_panels", fig_text_panels),
("C_ci_needed", fig_hr_needed), ("D_z_thresholds", fig_z),
("E_table_row_origin", fig_row_origin)):
fn(os.path.join(cdir, tag))
print("wrote candidates/A-E (.svg, .png)")
Abstract transposes two of four unilateral and bilateral effusion mortality figures given correctly in Results
Eur Respir J (2015) · doi:10.1183/09031936.00217114 · PMC4857137
Summary
This cohort of 308 patients undergoing thoracentesis (247 unilateral, 61 bilateral) reports in its abstract that bilateral effusion, relative to unilateral, carried higher mortality, giving "17% versus 47%" with the 30-day hazard ratio and "36% versus 69%" with the 1-year hazard ratio. The Results instead give unilateral mortality as 17% (30 days) and 47% (1 year) and bilateral mortality as 36% and 69%, and only these figures reproduce the paper's reported totals of 65 and 158 deaths (43/247 + 22/61 and 116/247 + 42/61); no count out of 61 even rounds to 47%. The abstract, including the version of record indexed in PubMed, has therefore transposed 47% and 36%: it overstates bilateral 30-day mortality (47% instead of 36%), understates unilateral 1-year mortality (36% instead of 47%), and makes the absolute gap look 11 percentage points larger at both time points. The hazard ratios and the direction of the finding are unaffected, but this is the comparison the paper presents as its new contribution, and the incorrect 47% figure has since been cited.
What the paper reports
Abstract, Results (PMC author manuscript):
Patients with bilateral pleural effusion, relative to unilateral, were associated with higher risk of death at 30 days and 1 year (17% versus 47%; HR 2.58 CI [1.44–4.63] and 36% versus 69%; HR 2.32 CI [1.55–3.48])
Abstract, Results, as published in Eur Respir J 46(2):495–502 and indexed in PubMed (PMID 25837039):
Patients with bilateral, relative to unilateral, pleural effusion were associated with higher risk of death at 30 days and 1 year (17% versus 47% (hazard ratio (HR) 2.58, 95% CI 1.44-4.63) and 36% versus 69% (HR 2.32, 95% CI 1.55-3.48), respectively).
Results, first paragraph:
A total of 247 unilateral and 61 bilateral thoracenteses were performed. Overall mortality across all groups was 21% (65 of 308 patients) at 30-days and 51% (158 of 308 patients) at one year
Results, second paragraph:
As calculated from Figure 1, thirty-day and one-year mortality of patients with unilateral effusion was 17% and 47%, compared with 36% and 69% in patients with bilateral effusion.
Relative to patients with unilateral effusion, unadjusted mortality of patients with bilateral effusion was higher at thirty-days (p=0.0003) and at one-year (p=0.0001).
Results, fifth paragraph (Table 3, row "Bilateral", reads the same: 2.58 (1.44, 4.63), 0.002 | 2.32 (1.55, 3.48), <0.001):
was significantly associated with higher 30-day and 1-year mortality (HR 2.58 [1.44–4.63] p=0.002 and HR 2.32 [1.55–3.48] p<0.001, respectively)
Methods, Statistical Analysis, on what the percentages are:
Unadjusted 30-day and 1-year mortalities were calculated and diagrammed for subgroups according to the following hierarchy: laterality (bilateral versus unilateral) and malignancy (yes versus no).
Discussion, on the weight the paper gives the comparison:
The study is novel in evaluating mortality associated with bilateral effusions.
It is also the first study to report the association between bilateral pleural effusion and mortality.
Abstract, Conclusions:
Bilateral pleural effusion is distinctly associated with high mortality.
What the paper's own numbers show
Method. The group sizes (247 and 61) and the overall death counts (65 and 158 of 308) come from the Results. For each printed percentage I listed every integer number of deaths k for which k/n rounds to that percentage. I then asked which pairing of unilateral and bilateral percentages can reproduce the reported overall totals: the pairing in the Results (unilateral 17%/47%, bilateral 36%/69%), or the pairing in the abstract, where each "x% versus y%" is tied to one time point by the hazard ratio that follows it (30 days: 17% vs 47%; 1 year: 36% vs 69%). For each correct split I recomputed the risk difference and crude risk ratio. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Group sizes | Abstract; Results ¶1 | 247 unilateral, 61 bilateral | 247 + 61 = 308 |
| Overall deaths, 30 days | Results ¶1 | 65 of 308 (21%) | 21.1% |
| Overall deaths, 1 year | Results ¶1 | 158 of 308 (51%) | 51.3% |
| Unilateral, 30 days | Results ¶2; abstract | 17% | 43/247 = 17.4% (41–43 deaths round to 17%) |
| Bilateral, 30 days | Results ¶2 | 36% | 22/61 = 36.1% (22 is the only count giving 36%) |
| Bilateral, 30 days | Abstract (30-day pair, with HR 2.58) | 47% | not attainable: 28/61 = 45.9%, 29/61 = 47.5%; the value is the unilateral 1-year figure |
| Unilateral, 1 year | Results ¶2 | 47% | 116/247 = 47.0% |
| Unilateral, 1 year | Abstract (1-year pair, with HR 2.32) | 36% | would mean 88–90 deaths; the value is the bilateral 30-day figure |
| Bilateral, 1 year | Results ¶2; abstract | 69% | 42/61 = 68.9% |
| Total 30-day deaths implied, Results pairing | derived | 65 | 0.17 × 247 + 0.36 × 61 = 64.0 (63–65 with rounding); the only exact split is 43 + 22 = 65 |
| Total 30-day deaths implied, abstract pairing | derived | 65 | 0.17 × 247 + 0.47 × 61 = 70.7 (about 69–72); no integer split gives 65 |
| Total 1-year deaths implied, Results pairing | derived | 158 | 0.47 × 247 + 0.69 × 61 = 158.2 (157–159); the only exact split is 116 + 42 = 158 |
| Total 1-year deaths implied, abstract pairing | derived | 158 | 0.36 × 247 + 0.69 × 61 = 131.0 (130–132); no integer split gives 158 |
| Bilateral minus unilateral, 30 days | Abstract vs Results | 47 − 17 = 30 pp | 36.1 − 17.4 = 18.7 pp (19 pp from the rounded Results figures); crude RR 2.07 (2.76 as printed in the abstract) |
| Bilateral minus unilateral, 1 year | Abstract vs Results | 69 − 36 = 33 pp | 68.9 − 47.0 = 21.9 pp (22 pp from the rounded Results figures); crude RR 1.47 (1.92 as printed in the abstract) |
| Adjusted HR, bilateral vs unilateral | Abstract; Results ¶5; Table 3 | 2.58 (1.44–4.63) at 30 days; 2.32 (1.55–3.48) at 1 year | identical in all three places; not affected |
The four numbers in the abstract are the right four numbers, but 47% and 36% have swapped places. The Results figures, the death counts they imply and the reported overall totals all agree with one another. The abstract's pairing agrees with none of them.
One side observation does not form part of the finding. Pearson chi-square tests on the recomputed counts give unadjusted p = 0.0014 (30 days) and 0.0022 (1 year), and Fisher exact tests give 0.0026 at both time points. The Results report p = 0.0003 and 0.0001. The paper does not name the test, and a log-rank test on time to death could legitimately give different values, so I leave this unresolved.
Figure
Figure 1. The upper panel quotes the abstract's sentence and lays out its four percentages as the sentence assigns them: each "x% versus y%" pair is followed by its time-point hazard ratio (HR 2.58 is the 30-day HR and 2.32 the 1-year HR in Table 3), and in each pair bilateral is the higher value. The values as printed are shown in orange. The lower panel quotes the Results and shows, in blue, the death counts recomputed from the paper's own numbers: the only integer split that matches both the Results percentages and the reported totals of 65 and 158 deaths. The unilateral 30-day (17%) and bilateral 1-year (69%) cells agree. The bilateral 30-day and unilateral 1-year cells hold each other's values: the abstract's 47% is the unilateral 1-year mortality (116/247), and its 36% is the bilateral 30-day mortality (22/61).
Figure 2. Total deaths implied by applying each pairing's percentages to the 247 unilateral and 61 bilateral patients, on one axis of deaths that starts at zero. Filled blue circles use the Results pairing and open blue circles the abstract's pairing. The ranges in the labels allow for rounding of the printed percentages. Orange lines mark the reported totals (65/308 at 30 days, 158/308 at 1 year). The Results pairing reproduces both totals. The abstract's pairing implies about 71 deaths at 30 days and 131 at 1 year, and neither range contains the reported total.
Why it matters
- The comparison is the paper's stated novel contribution. The Discussion calls the bilateral analysis "novel" and "the first study to report" this association, and the abstract's last Results sentence is where most readers will take the absolute risks from.
- The absolute risks in the abstract are wrong in a way a reader cannot detect from the abstract alone. A bilateral 30-day mortality of 47% (the true value is 36%) and a unilateral 1-year mortality of 36% (the true value is 47%) both look plausible. Read as printed, the excess mortality with bilateral effusion is 30 pp instead of about 19 pp at 30 days, and 33 pp instead of about 22 pp at 1 year. The implied crude risk ratios are 2.76 instead of 2.07, and 1.92 instead of 1.47. The abstract also understates the 1-year mortality of the larger, unilateral group by 11 pp.
- The error has propagated. A 2022 article in Frontiers in Medicine (doi:10.3389/fmed.2022.828783) cites this study in its Background as reporting bilateral-effusion mortality of "47% at 30 days, 69% at 1 year". Its Discussion later gives the correct "17% for unilateral vs. 36% for bilateral" at 1 month.
- What is not affected. The direction of the association, the adjusted hazard ratios (2.58 and 2.32, consistent across the abstract, Results and Table 3), the reported significance, the etiology-specific mortality figures, the main-text Results, and the qualitative conclusion that bilateral effusion is associated with higher mortality all stand. The error lies in two absolute mortality figures in the abstract.
Explanations considered
- The pairs mean unilateral (30 days vs 1 year) and bilateral (30 days vs 1 year). Under this reading "17% versus 47%" would be the unilateral group over time and "36% versus 69%" the bilateral group, which would match the Results. This does not account for the sentence as printed. The sentence compares bilateral "relative to unilateral" and attaches HR 2.58 (the 30-day HR in Table 3) to the first pair and HR 2.32 (the 1-year HR) to the second. The version of record adds "respectively", which ties the first pair to 30 days and the second to 1 year. A reader therefore takes 17% vs 47% as the 30-day comparison and 36% vs 69% as the 1-year comparison. This reading may explain how the slip arose, but it does not make the printed text correct.
- The abstract quotes adjusted rather than crude mortality. This does not account for the numbers. The Methods describe the subgroup percentages as unadjusted, the Results say they were "calculated from Figure 1", the abstract presents no separate adjusted percentages, and the abstract uses exactly the same four values as the Results.
- A different subgroup, population or time point. This does not account for the numbers. The abstract and the Results use the same 247/61 split and the same two time points, and name no subgroup.
- Rounding. This does not account for the numbers. No number of deaths out of 61 rounds to 47% (28/61 = 45.9%, 29/61 = 47.5%), and neither of the abstract's pairings can reproduce the reported totals of 65 and 158 deaths under any rounding (Figure 2).
- A conversion artifact in the PMC author manuscript. This is ruled out. The publisher's copy-edited abstract in PubMed has the same pairing.
- The error was corrected later. No correction was found (see Status).
Status
- Version of record. The publisher's pages (doi.org redirects to publications.ersnet.org; erj.ersjournals.com/content/46/2/495 and its PDF) returned HTTP 403 to automated access, so I could not read the publisher's full-text Results directly. The final published abstract was checked through the PubMed record (PMID 25837039, Eur Respir J 2015;46(2):495–502, retrieved with NCBI E-utilities on 2026-10-23). It is the publisher's copy-edited text, and it contains the same pairing with "respectively" added. The PMC record (PMC4857137) is the NIH author manuscript. Every quote from it above was confirmed verbatim against the PMC-archived text.
- Supplements. PMC lists no supplementary material, and Europe PMC reports no open-access supplementary files. Supplementary material would not change the finding: the Results text and the overall totals already fix the correct values.
- Corrections and comments. As of 2026-10-23, I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI. PMC and Europe PMC list no correction notices (the Europe PMC commentCorrectionList is empty). The PubMed record (revised 2022-03-17) has no CommentsCorrections entry. Web searches for an erratum or a PubPeer thread found none. The PubPeer API returned HTTP 403 to automated access, so PubPeer could not be queried directly.
- Sources. PMC4857137 (author manuscript, as archived); PubMed 25837039 (E-utilities efetch); Crossref and Europe PMC REST records for doi:10.1183/09031936.00217114; Frontiers in Medicine 2022, doi:10.3389/fmed.2022.828783 (the citing article).
Suggested correction
In the Results sentence of the abstract, change
(17% versus 47%; HR 2.58 CI [1.44–4.63] and 36% versus 69%; HR 2.32 CI [1.55–3.48])
to
(36% versus 17%; HR 2.58 CI [1.44–4.63] and 69% versus 47%; HR 2.32 CI [1.55–3.48])
or, keeping the original order, "(17% versus 36% … and 47% versus 69% …)", stated explicitly as unilateral versus bilateral. The version of record needs the same change, with "respectively" kept. No change is needed to the main text, Table 3 or the conclusions. Because the incorrect figure has already been cited, a published correction notice linked to the article would be more useful than a silent update.
Recomputation script (python recompute.py.txt)
"""Recompute the unilateral vs bilateral mortality figures of PMC4857137
(Eur Respir J 2015;46(2):495-502, doi:10.1183/09031936.00217114).
Run: python recompute.py.txt
Needs scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (author manuscript):
Results p0: 247 unilateral and 61 bilateral thoracenteses; 65 of 308 deaths at
30 days, 158 of 308 at 1 year
Results p1: unilateral 17% (30 d) and 47% (1 y); bilateral 36% (30 d) and 69% (1 y);
unadjusted p = 0.0003 (30 d) and 0.0001 (1 y)
Abstract: "(17% versus 47%; HR 2.58 ...) and (36% versus 69%; HR 2.32 ...)"
Table 3: adjusted HR bilateral 2.58 (1.44, 4.63) at 30 d, 2.32 (1.55, 3.48) at 1 y
"""
import os
import textwrap
from scipy.stats import chi2_contingency, fisher_exact
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
N_U, N_B = 247, 61 # Results p0, Abstract
N_ALL = 308
TOTAL = {"30-day": 65, "1-year": 158} # Results p0
RESULTS_PCT = {"30-day": (17, 36), "1-year": (47, 69)} # (unilateral, bilateral), Results p1
ABSTRACT_PCT = {"30-day": (17, 47), "1-year": (36, 69)} # (unilateral, bilateral) as paired in the abstract
HR = {"30-day": (2.58, 1.44, 4.63), "1-year": (2.32, 1.55, 3.48)} # Abstract, Results p4, Table 3
P_REPORTED = {"30-day": "0.0003", "1-year": "0.0001"} # Results p1, test not named
def rnd(x):
"""Round half up to an integer percentage, as printed."""
return int(x + 0.5)
def counts_for(pct, n):
"""All integer counts k (0..n) whose k/n prints as pct when rounded to a whole percent."""
return [k for k in range(n + 1) if rnd(100 * k / n) == pct]
print("=== 1. Group sizes ===")
print(f"unilateral + bilateral = {N_U} + {N_B} = {N_U + N_B} (reported {N_ALL})")
for t in TOTAL:
print(f"overall {t} deaths: {TOTAL[t]}/{N_ALL} = {100 * TOTAL[t] / N_ALL:.1f}%")
print("\n=== 2. Which death counts are compatible with each printed percentage ===")
for t in ("30-day", "1-year"):
for label, (pu, pb) in (("Results", RESULTS_PCT[t]), ("Abstract", ABSTRACT_PCT[t])):
cu, cb = counts_for(pu, N_U), counts_for(pb, N_B)
print(f"{t} {label:8s}: unilateral {pu}% of {N_U} -> deaths {cu}; "
f"bilateral {pb}% of {N_B} -> deaths {cb if cb else 'NONE (no k/61 rounds to %d%%)' % pb}")
print("k/61 near 47%:", [(k, round(100 * k / N_B, 1)) for k in range(26, 31)])
print("\n=== 3. Implied total deaths under each pairing vs the reported total ===")
RANGES = {}
for t in ("30-day", "1-year"):
for label, (pu, pb) in (("Results", RESULTS_PCT[t]), ("Abstract", ABSTRACT_PCT[t])):
point = pu / 100 * N_U + pb / 100 * N_B
cu, cb = counts_for(pu, N_U), counts_for(pb, N_B)
if cb:
lo, hi = min(cu) + min(cb), max(cu) + max(cb)
else: # bilateral percentage not attainable: use the real-valued rounding interval
lo = (pu - 0.5) / 100 * N_U + (pb - 0.5) / 100 * N_B
hi = (pu + 0.5) / 100 * N_U + (pb + 0.5) / 100 * N_B
RANGES[(t, label)] = (point, lo, hi)
inside = lo <= TOTAL[t] <= hi
print(f"{t} {label:8s}: {pu/100:.2f}*{N_U} + {pb/100:.2f}*{N_B} = {point:.1f}; "
f"totals compatible with rounding {lo:.0f}-{hi:.0f}; reported {TOTAL[t]} -> "
f"{'consistent' if inside else 'INCONSISTENT'}")
print("\n=== 4. Integer splits that reproduce both the percentages and the reported totals ===")
SPLITS = {}
for t in ("30-day", "1-year"):
for label, (pu, pb) in (("Results", RESULTS_PCT[t]), ("Abstract", ABSTRACT_PCT[t])):
sols = [(u, TOTAL[t] - u) for u in counts_for(pu, N_U)
if (TOTAL[t] - u) in counts_for(pb, N_B)]
print(f"{t} {label:8s} pairing {pu}% vs {pb}%: solutions (unilateral, bilateral) = {sols}")
if label == "Results":
SPLITS[t] = sols[0]
for t, (u, b) in SPLITS.items():
print(f"{t}: unilateral {u}/{N_U} = {100*u/N_U:.1f}%, bilateral {b}/{N_B} = {100*b/N_B:.1f}%, "
f"sum {u + b} (reported {TOTAL[t]})")
print("\n=== 5. Size of the comparison: Results vs the abstract's pairing ===")
for t in ("30-day", "1-year"):
u, b = SPLITS[t]
ru, rb = 100 * u / N_U, 100 * b / N_B
au, ab = ABSTRACT_PCT[t]
print(f"{t}: from counts RD = {rb:.1f} - {ru:.1f} = {rb - ru:.1f} pp, crude RR = {rb / ru:.2f}")
print(f"{t}: Results text RD = {RESULTS_PCT[t][1]} - {RESULTS_PCT[t][0]} = "
f"{RESULTS_PCT[t][1] - RESULTS_PCT[t][0]} pp, RR = {RESULTS_PCT[t][1] / RESULTS_PCT[t][0]:.2f}")
print(f"{t}: abstract RD = {ab} - {au} = {ab - au} pp, RR = {ab / au:.2f}")
print(f"{t}: abstract overstates the gap by {(ab - au) - (RESULTS_PCT[t][1] - RESULTS_PCT[t][0])} pp")
print("\n=== 6. Hazard ratios (not affected) ===")
for t, (h, lo, hi) in HR.items():
print(f"{t}: HR {h} ({lo}-{hi}) - identical in Abstract, Results p4 and Table 3")
print("\n=== 7. Side note: unadjusted tests on the recomputed counts (test in paper not named) ===")
for t in ("30-day", "1-year"):
u, b = SPLITS[t]
tab = [[b, N_B - b], [u, N_U - u]]
_, p, _, _ = chi2_contingency(tab, correction=False)
_, py, _, _ = chi2_contingency(tab, correction=True)
pf = fisher_exact(tab)[1]
print(f"{t}: reported p = {P_REPORTED[t]}; Pearson chi-square p = {p:.4f}; "
f"Yates p = {py:.4f}; Fisher exact p = {pf:.4f}")
# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5,
"svg.fonttype": "none", "figure.facecolor": BG, "axes.facecolor": BG})
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# ---- fig1: abstract vs Results, two text panels; the swapped cells are joined by crossing connectors
FH = 5.5
fig = plt.figure(figsize=(7.2, FH))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, FH)
ax.axis("off")
COLS = [2.2, 3.45, 4.7, 5.95] # unilateral 30 d, bilateral 30 d, unilateral 1 y, bilateral 1 y
HEAD = ["Unilateral\n30 days", "Bilateral\n30 days", "Unilateral\n1 year", "Bilateral\n1 year"]
X0, W = 0.12, 6.96
def panel(y_bottom, h):
ax.add_patch(FancyBboxPatch((X0, y_bottom), W, h, boxstyle="round,pad=0.0,rounding_size=0.06",
fc="white", ec="#cfcdc7", lw=1))
# Panel A: abstract
panel(3.2, 2.2)
ax.text(X0 + 0.15, 5.18, "Abstract, Results sentence", fontsize=11.5, weight="bold", color=INK, va="center")
qa = ("\u201cPatients with bilateral pleural effusion, relative to unilateral, were associated with higher "
"risk of death at 30 days and 1 year (17% versus 47%; HR 2.58 CI [1.44\u20134.63] and 36% versus 69%; "
"HR 2.32 CI [1.55\u20133.48])\u201d")
ax.text(X0 + 0.15, 4.98, "\n".join(textwrap.wrap(qa, 86)), fontsize=10, color=INK2, style="italic",
va="top", linespacing=1.3)
for (c0, c1), lab in (((COLS[0], COLS[1]), "30-day pair, HR 2.58"), ((COLS[2], COLS[3]), "1-year pair, HR 2.32")):
ax.plot([c0 - 0.35, c1 + 0.35], [4.12, 4.12], color=INK2, lw=1)
ax.text((c0 + c1) / 2, 4.2, lab, ha="center", va="bottom", fontsize=10, color=INK2)
for x, hd in zip(COLS, HEAD):
ax.text(x, 3.88, hd, ha="center", va="center", fontsize=10, color=INK2, linespacing=1.1)
ax.text(X0 + 0.15, 3.45, "As printed", fontsize=10, color=ORANGE, weight="bold",
va="center", linespacing=1.1)
A = ["17%", "47%", "36%", "69%"]
for j, (x, v) in enumerate(zip(COLS, A)):
sw = j in (1, 2)
ax.text(x, 3.45, v, ha="center", va="center", fontsize=12, color=ORANGE if sw else INK,
weight="bold" if sw else "normal",
bbox=dict(boxstyle="round,pad=0.3", fc="white", ec=ORANGE if sw else "none", lw=1.6))
# Panel B: Results
panel(0.12, 1.98)
u30, b30 = SPLITS["30-day"]
u1, b1 = SPLITS["1-year"]
B = [(u30, N_U), (b30, N_B), (u1, N_U), (b1, N_B)]
ax.text(X0 + 0.15, 1.83, "Results,\nparagraph 2", fontsize=11.5, weight="bold", color=INK, va="center",
linespacing=1.1)
ax.text(X0 + 0.15, 1.33, "Recomputed\nfrom counts", fontsize=10, color=BLUE, weight="bold", va="center",
linespacing=1.1)
for j, (x, (k, n)) in enumerate(zip(COLS, B)):
sw = j in (1, 2)
ax.text(x, 1.62, f"{k}/{n}\n= {100 * k / n:.1f}%", ha="center", va="center", fontsize=11,
color=BLUE if sw else INK, weight="bold" if sw else "normal", linespacing=1.15,
bbox=dict(boxstyle="round,pad=0.3", fc="white", ec=BLUE if sw else "none", lw=1.6))
qb = ("\u201cthirty-day and one-year mortality of patients with unilateral effusion was 17% and 47%, "
"compared with 36% and 69% in patients with bilateral effusion\u201d. Counts: the only integer "
"split matching these percentages and the reported 65 and 158 total deaths.")
ax.text(X0 + 0.15, 0.98, "\n".join(textwrap.wrap(qb, 86)), fontsize=10, color=INK2, style="italic",
va="top", linespacing=1.3)
# connectors between the panels
ytop, ybot = 3.18, 2.02
for ja, jb in ((1, 2), (2, 1)): # abstract cell ja holds the value of Results cell jb
ax.add_patch(FancyArrowPatch((COLS[ja], ytop), (COLS[jb], ybot), arrowstyle="-|>",
mutation_scale=14, lw=2, color=INK, shrinkA=2, shrinkB=2))
ym = (ytop + ybot) / 2
XL = COLS[1] - 0.35
ax.text(XL, ym + 0.27, "47% and 36% transposed", fontsize=11, weight="bold", color=INK, va="center", ha="right")
ax.text(XL, ym - 0.02, "each value sits in the other's cell", fontsize=10, color=INK2, va="center", ha="right")
ax.text(XL, ym - 0.28, "17% and 69% agree", fontsize=10, color=INK2, va="center", ha="right")
save(fig, "fig1")
# ---- fig2: implied total deaths under each pairing vs the reported totals
fig, ax = plt.subplots(figsize=(7.2, 3.3))
fig.subplots_adjust(left=0.25, right=0.97, top=0.84, bottom=0.17)
rows = [("30-day", "Results"), ("30-day", "Abstract"), ("1-year", "Results"), ("1-year", "Abstract")]
ylab = {("30-day", "Results"): "30 days: Results\n(17% vs 36%)",
("30-day", "Abstract"): "30 days: Abstract\n(17% vs 47%)",
("1-year", "Results"): "1 year: Results\n(47% vs 69%)",
("1-year", "Abstract"): "1 year: Abstract\n(36% vs 69%)"}
ypos = {rows[0]: 4.0, rows[1]: 3.0, rows[2]: 1.4, rows[3]: 0.4}
for r in rows:
point, lo, hi = RANGES[r]
y = ypos[r]
filled = r[1] == "Results"
ax.plot([lo, hi], [y, y], color=BLUE, lw=2, solid_capstyle="butt")
ax.plot(point, y, "o", ms=9, mfc=BLUE if filled else BG, mec=BLUE, mew=2)
lab = f"{point:.0f} (range {lo:.0f}\u2013{hi:.0f})"
if hi < TOTAL[r[0]] - 5: # marker left of the reported line: label on the left
ax.text(lo - 4, y, lab, va="center", ha="right", fontsize=10, color=BLUE)
else:
ax.text(hi + 4, y, lab, va="center", fontsize=10, color=BLUE)
for t, (ya, yb) in (("30-day", (3.0, 4.0)), ("1-year", (0.4, 1.4))):
ax.plot([TOTAL[t]] * 2, [ya - 0.35, yb + 0.35], color=ORANGE, lw=2)
ax.text(TOTAL[t], yb + 0.5, f"reported {TOTAL[t]}/308", ha="center", fontsize=10, color=ORANGE)
ax.set_yticks([ypos[r] for r in rows], [ylab[r] for r in rows], fontsize=10)
ax.set_ylim(-0.3, 4.9)
ax.set_xlim(0, 230)
ax.set_xticks(range(0, 226, 25))
ax.set_xlabel("Total deaths implied by the group percentages (247 unilateral, 61 bilateral)",
fontsize=10, color=INK)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0, colors=INK)
ax.tick_params(axis="x", colors=INK2)
ax.plot([], [], "o", ms=9, mfc=BLUE, mec=BLUE, mew=2, label="Recomputed, Results pairing")
ax.plot([], [], "o", ms=9, mfc=BG, mec=BLUE, mew=2, label="Recomputed, abstract pairing")
ax.plot([], [], color=ORANGE, lw=2, label="Reported total deaths")
fig.legend(loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=3, frameon=False, fontsize=10,
handlelength=1.6, columnspacing=1.0)
save(fig, "fig2")
print("\nfigures written to", HERE)
Dorsal attention network mediation reports a negative indirect effect with an entirely positive confidence interval
Hum Brain Mapp (2023) · doi:10.1002/hbm.26474 · PMC10619388
Summary
The paper reports that the temporal variability of the dorsal attention network (DAN) mediates the link between anhedonia and attention in adolescents with major depressive disorder, with "indirect effect = -0.438, 95% confidence interval: [0.926, 1.079]". The estimate is negative but the interval is entirely positive, and the estimate lies 1.364 below the interval's lower bound, which is 17.8 interval half-widths. The paper's own Figure 7c(C) labels −0.438 as the direct path c′, not as the indirect effect. Its path coefficients give an indirect effect of a × b = −0.509 × 0.263 = −0.134, which also lies outside the printed interval. This result is the only inferential support for the fMRI half of the title claim and for an "Importantly" sentence in the Abstract, so as printed that claim has no valid confidence interval. The paper's other mediation result (EEG Microstate C-to-D transition) is internally consistent in the Results text.
What the paper reports
Results, "Network temporal properties associated with MDD", paragraph 4 (the DAN mediation):
Temporal variability in DAN mediated the relationship between anhedonia and attention (direct effect = -0.271; indirect effect = -0.438, 95% confidence interval: [0.926, 1.079]; Figure 7d).
Results, "Network temporal properties associated with MDD", paragraph 3 (the inputs to that mediation):
Correlation analyses revealed that temporal variability was significantly negatively correlated with anhedonia (r = -0.266, p = 0.049; Figure 7c(A)) and positively correlated with attention scores in adolescents with MDD (r = 0.419, p = 0.019; Figure 7c(B)).
Results, "Characteristics of EEG microstate in adolescents with MDD", paragraph 3 (the second mediation, given for comparison):
The transition rate from Microstates C to D mediated the relationship between anhedonia and attention (direct effect = -0.497; indirect effect = -0.389, 95% confidence interval: [-0.571, -0.032], Figure 3c(H)).
Figure 7, caption:
Correlation analyses indicated that the temporal variability was significantly positively correlated with anhedonia and negatively related with attention scores in MDD patients (c(A,B)). Mediation analysis revealed that the temporal variability of DAN mediated the relationship from anhedonia to attention (c(C)).
Figure 7, panel c(C). The panel is an image; these labels were read from the publisher's graphic (Europe PMC file HBM-44-5749-g009.jpg). The paths are a = −0.509 (anhedonia to DAN variability) and b = 0.263 (DAN variability to attention), and the anhedonia-to-attention arrows carry:
−0.438(c′)
−0.572(c)
Bootstrap estimate [95% CI] = −0.271* [0.926, 1.079]
Figure 3, panel c(H) (file HBM-44-5749-g001.jpg). Here a = 0.416 and b = −0.260, and:
-0.389 (c′)
-0.497 (c)
Bootstrap Estimate [95% CI] = 2.241* [-0.571,-0.032]
The finding is stated as a headline result in the Abstract:
Importantly, the temporal variability of DAN mediated the relationship between anhedonia and attention deficits in the patient group.
and in the Discussion, opening paragraph:
Importantly, the dynamic features of the attentional network (transition probability of C-D and temporal variability of the DAN) mediated the relationship between anhedonia and attentional deficits in the patient group.
Discussion, "Adolescent with MDD and network dynamic attributes":
The temporal variability of DAN mediates the relationship between anhedonia and attention.
The title reads "Attention network mediates the association between anhedonia and attentional deficit". The Methods ("Statistical analyses") say that anhedonia was the independent variable, attention the dependent variable and the C-to-D conversion rate the mediator. They do not describe how the mediation confidence intervals were obtained, and they do not mention the DAN mediation model.
What the paper's own numbers show
Method. For each mediation I checked whether the printed estimate lies inside its printed 95% CI. I computed the implied total effect (c = c′ + ab) and proportion mediated from the text, and the indirect effect a × b and c − c′ from the figure's path labels. I checked the sign that a × b must take given the reported correlations and paths. I tested whether any other DAN quantity the paper prints could be the one the interval describes, and whether the coefficients could be standardized. Finally I worked out the sample sizes that the reported r and p values imply. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| DAN indirect effect vs its CI | Results 3.6 p4 | −0.438, CI [0.926, 1.079] | Outside the CI. 1.364 below the lower bound (17.8 half-widths of 0.0765); CI midpoint 1.0025 |
| Sign-flipped DAN CI | Results 3.6 p4 | [0.926, 1.079] | [−1.079, −0.926] does not contain −0.438 either |
| DAN implied total effect, proportion mediated | Results 3.6 p4 | direct −0.271, indirect −0.438 | c = −0.709, proportion 0.618. Neither lies in the CI |
| DAN indirect effect from figure paths | Fig 7c(C) | a = −0.509, b = 0.263 | a × b = −0.134; c − c′ = −0.572 − (−0.438) = −0.134 (the paths agree with each other) |
| What −0.438 is | Results 3.6 p4 vs Fig 7c(C) | "indirect effect" in text | Labelled c′ (direct path) in the figure |
| What −0.271 is | Results 3.6 p4 vs Fig 7c(C) | "direct effect" in text | Labelled "Bootstrap estimate" in the figure; outside [0.926, 1.079] |
| Any printed DAN quantity inside the CI? | Text and Fig 7c(C) | c, a, b, c′, −0.271, a × b, c′/c (0.766), c/c′ (1.306) | None |
| Sign of the DAN indirect effect | Results p3; Fig 7 caption; Fig 7c(C) | r = −0.266 and +0.419; caption gives the opposite pair; a < 0, b > 0 | a × b < 0 under every source. The negative estimate is the plausible value and the positive CI is not |
| Standardized-coefficient check | derived | indirect −0.438 | If standardized, a × b would be −0.066; Fig 7 a = −0.509 ≠ r = −0.266. The coefficients are unstandardized, so the CI cannot be rebuilt from the text |
| C-D indirect effect vs its CI | Results 3.2 p3 | −0.389, CI [−0.571, −0.032] | Inside; CI negative and excludes 0. Total −0.886, proportion 0.439 |
| C-D indirect effect from figure paths | Fig 3c(H) | a = 0.416, b = −0.260 | a × b = −0.108 = c − c′; inside the CI. The figure's "Bootstrap Estimate" 2.241 lies outside its own CI |
| Sample size behind the DAN correlations | Results 3.6 p3 | r = −0.266, p = 0.049; r = 0.419, p = 0.019 | Consistent only with n = 55 and n = 31 respectively (with n = 188, p would be 2.2 × 10⁻⁴ and 2.2 × 10⁻⁹) |
Figure
Figure 1. The two mediation results on one axis of unstandardized indirect effect, with the null value 0 as a thin vertical line. Orange circles and bars are the indirect effects and 95% CIs printed in the Results text. Blue diamonds are a × b recomputed from the path coefficients in the paper's Figures 3c(H) and 7c(C); the paper reports no interval for these. For the EEG C-to-D mediation, the printed estimate (−0.389) and the recomputed a × b (−0.108) both lie inside the printed interval. For the DAN mediation, the printed estimate (−0.438) lies 1.364 below an interval that is entirely positive, and the recomputed a × b (−0.134) lies outside it too.
Figure 2. The DAN mediation numbers as printed in the Results text (left) and in the path-diagram labels of the paper's Figure 7c(C) (right), transcribed rather than reproduced. Connector 1: the text calls −0.438 the indirect effect, while the figure labels it c′, the direct path. Connector 2: the text calls −0.271 the direct effect, while the figure labels it the bootstrap estimate. The bottom panel recomputes the indirect effect from the figure's own paths (−0.134). None of the three candidate values lies inside the shared interval [0.926, 1.079].
Why it matters
The title's claim that the "attention network mediates the association between anhedonia and attentional deficit" rests on two analyses: an EEG microstate transition (C to D) and fMRI DAN temporal variability. The DAN result is the only statistical support for the fMRI half of that claim. It appears in the Abstract as an "Importantly" finding and is restated three times in the Discussion. As printed, its 95% CI contradicts its own estimate in sign and location. The paper's figure and text also disagree about which printed number is the indirect effect. A reader cannot tell what the indirect effect is (−0.438, −0.271 or −0.134), what its interval is, or whether that interval excludes zero. The interval is narrow and centred near 1. If it came from a ratio-type quantity, for which 1 is the null, it would include the null. If it belongs to the negative a × b, its true limits are not reported.
The inputs to the mediation are unclear too. The Figure 7 caption reverses the signs of both correlations given in the Results text. One Discussion sentence calls the DAN–attention correlation negative ("the temporal variability of the DAN was significantly negatively correlated with attention scores"), while the Results report r = 0.419. The two r/p pairs fit only n = 55 and n = 31, not the 188 patients enrolled, and the paper does not say which subsample each analysis used. A separate inconsistency: Results paragraph 3 says DAN variability was "significantly higher in adolescents with MDD than in HC (t = 2.582, p = .010)", while the Abstract, the Figure 7 caption and the Discussion say it was weaker.
The EEG C-to-D mediation is unaffected in the text: its estimate lies inside its interval, both negative. It still supports the broader statement that an attention-related dynamic measure mediates the anhedonia–attention link. Its figure, however, prints a "Bootstrap Estimate" of 2.241 outside its own interval, and the figure labels −0.389 as c′ rather than as the indirect effect.
Explanations considered
- The interval belongs to another DAN quantity. No. The direct effect (−0.271), the implied total effect (−0.709), the proportion mediated (0.618), the figure's a (−0.509), b (0.263), c (−0.572), c′ (−0.438) and a × b (−0.134) all lie outside [0.926, 1.079]. So do the ratios c′/c (0.766) and c/c′ (1.306).
- A lost minus sign. No. The negated interval [−1.079, −0.926] does not contain −0.438, −0.271 or −0.134. Also, the publisher's XML carries Unicode minus signs elsewhere in the same sentence but none in this interval, and the figure graphic prints the same positive interval. So this is not a text-conversion artefact.
- Standardized versus unstandardized coefficients. A change of scale cannot move a negative estimate into a positive interval. Standardized coefficients would give a × b ≈ −0.066 in any case, so the printed values are unstandardized.
- Bootstrap asymmetry. Percentile and bias-corrected bootstrap intervals can be asymmetric about the estimate. They still contain the estimate they are built around, and they cannot have the opposite sign to it.
- The interval is for a ratio or exponentiated effect. The interval's midpoint near 1 fits that reading. But the paper reports no such quantity, and exp(−0.134) = 0.875 and exp(−0.438) = 0.645 are both outside the interval. Under that reading the interval would include the null value 1, and the mediation would not be significant.
- A different sign convention in the figure caption. The caption's sign pair (positive with anhedonia, negative with attention) also gives a × b < 0. So no sign convention in the paper makes a positive indirect effect plausible.
- The interval is correct and the estimate is wrong. This cannot be ruled out from the paper alone. Every source in the paper, however, implies a negative indirect effect, so the positive interval is the less plausible number.
Status
- Version of record. The Wiley article page and PDF returned HTTP 403 to automated access. I checked the publisher-deposited full text as archived in PMC and the Europe PMC full-text XML (Wiley JATS). Both carry the quoted values. I read the mediation panels from the publisher's figure graphics in the Europe PMC package (HBM-44-5749-g001.jpg and -g009.jpg).
- Supplements. Neither PMC nor the Europe PMC XML lists supplementary material. The Europe PMC supplementary-files package contains only the article's figure graphics, with no data or supplementary tables.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction or comment links (PMID 37683097). The PubPeer API and search page refused automated access (HTTP 403), so PubPeer could not be checked. A general web search found no correction or discussion.
- Sources. PMC article PMC10619388; Europe PMC full-text XML and supplementary-files package for PMC10619388; Crossref record for 10.1002/hbm.26474; Europe PMC record PMID 37683097.
Suggested correction
- Re-run or re-extract the DAN mediation and report the indirect effect (a × b), its 95% CI, the direct effect c′ and the total effect c. Give the method used for the interval (for example the bootstrap type and number of resamples), the software, and whether the coefficients are unstandardized. Update the Results text, the Figure 7c(C) panel, and the Abstract and Discussion sentences that rest on this result.
- Make the labels in the Results text consistent with the path diagrams of Figures 3c(H) and 7c(C). The text currently calls the figures' c′ the "indirect effect", and calls the "bootstrap estimate" (DAN) or the total effect c (C-D) the "direct effect". Also correct the C-D panel's bootstrap estimate (2.241), which lies outside its own interval.
- Correct the Figure 7 caption's correlation signs, and the Discussion sentence calling the DAN–attention correlation negative, to match the Results (r = −0.266 with anhedonia, r = 0.419 with attention), or correct the Results if the caption is right.
- State the number of patients in each correlation and mediation analysis (the r/p pairs imply about 55 and 31). Reconcile Results paragraph 3 ("significantly higher in adolescents with MDD") with the Abstract and Discussion ("significantly weaker").
- If a valid interval for the DAN indirect effect includes zero, revise the title, the Abstract and the Discussion so that the mediation claim rests on the EEG result alone.
Recomputation script (python recompute.py.txt)
"""Recompute the mediation results of PMC10619388
(Hum Brain Mapp 2023, doi:10.1002/hbm.26474).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png, fig2.svg/png and five candidate figure designs
(candidates/cand_A ... cand_E) next to this file.
Inputs are copied from the paper as archived in PMC (Wiley JATS) and from the
publisher's figure graphics in the Europe PMC package:
Results 3.2, p3: C-D mediation, "direct effect = -0.497; indirect effect = -0.389,
95% confidence interval: [-0.571, -0.032]"
Results 3.6, p3: DAN correlations r = -0.266 (p = 0.049), r = 0.419 (p = 0.019)
Results 3.6, p4: DAN mediation, "direct effect = -0.271; indirect effect = -0.438,
95% confidence interval: [0.926, 1.079]"
Figure 3 c(H) path labels: a = 0.416, b = -0.260, -0.389 (c'), -0.497 (c),
"Bootstrap Estimate [95% CI] = 2.241* [-0.571,-0.032]"
Figure 7 c(C) path labels: a = -0.509, b = 0.263, -0.438 (c'), -0.572 (c),
"Bootstrap estimate [95% CI] = -0.271* [0.926, 1.079]"
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ------------------------------------------------------------------ inputs (from the paper)
TEXT = { # Results text
"C-D": dict(direct=-0.497, indirect=-0.389, lo=-0.571, hi=-0.032),
"DAN": dict(direct=-0.271, indirect=-0.438, lo=0.926, hi=1.079),
}
FIG = { # path-diagram labels in the publisher's figure graphics
"C-D": dict(a=0.416, b=-0.260, cprime=-0.389, c=-0.497, boot=2.241, lo=-0.571, hi=-0.032),
"DAN": dict(a=-0.509, b=0.263, cprime=-0.438, c=-0.572, boot=-0.271, lo=0.926, hi=1.079),
}
DAN_R = [(-0.266, 0.049, "anhedonia"), (0.419, 0.019, "attention")] # Results 3.6 p3
# ------------------------------------------------------------------ style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e3e2de"
LW, MS = 1.5, 8 # 1.5 pt = 2 px at 96 dpi; markers 8 pt
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "axes.edgecolor": INK2,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"text.color": INK, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
W = 7.2
MINUS = "−"
def f(x, d=3):
s = f"{x:.{d}f}"
return s.replace("-", MINUS)
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg", bbox_inches="tight")
fig.savefig(p + ".png", dpi=200, bbox_inches="tight")
plt.close(fig)
# ------------------------------------------------------------------ 1. containment checks
print("1. Estimate-versus-interval checks (Results text)")
for k, t in TEXT.items():
mid, hw = (t["lo"] + t["hi"]) / 2, (t["hi"] - t["lo"]) / 2
inside = t["lo"] <= t["indirect"] <= t["hi"]
total = t["direct"] + t["indirect"]
print(f" {k}: indirect {t['indirect']}, CI [{t['lo']}, {t['hi']}]; midpoint {mid:.4f}, "
f"half-width {hw:.4f}; inside={inside}; CI excludes 0={not (t['lo'] <= 0 <= t['hi'])}")
print(f" implied total c' + ab = {total:.3f}; proportion mediated = {t['indirect']/total:.3f}")
if not inside:
d = t["lo"] - t["indirect"]
print(f" distance to nearest bound = {d:.3f} = {d/hw:.1f} half-widths; "
f"to far bound = {t['hi'] - t['indirect']:.3f}")
print(f" sign-flipped CI [{-t['hi']}, {-t['lo']}] contains estimate: "
f"{-t['hi'] <= t['indirect'] <= -t['lo']}")
# ------------------------------------------------------------------ 2. figure path arithmetic
print("\n2. Path arithmetic from the figure labels")
AB = {}
for k, g in FIG.items():
ab, diff = g["a"] * g["b"], g["c"] - g["cprime"]
AB[k] = ab
print(f" {k}: a*b = {g['a']} x {g['b']} = {ab:.4f}; c - c' = {g['c']} - ({g['cprime']}) = {diff:.4f}; "
f"proportion mediated ab/c = {ab/g['c']:.3f}")
print(f" a*b inside printed CI [{g['lo']}, {g['hi']}]: {g['lo'] <= ab <= g['hi']}; "
f"bootstrap estimate {g['boot']} inside: {g['lo'] <= g['boot'] <= g['hi']}")
dan_vals = {"c (total, Fig 7)": FIG["DAN"]["c"], "a (Fig 7)": FIG["DAN"]["a"],
"'indirect' text = c' Fig 7": -0.438, "'direct' text = bootstrap est. Fig 7": -0.271,
"a*b": AB["DAN"], "b (Fig 7)": FIG["DAN"]["b"],
"c'/c": FIG["DAN"]["cprime"] / FIG["DAN"]["c"], "c/c'": FIG["DAN"]["c"] / FIG["DAN"]["cprime"],
"text total -0.709": -0.709, "proportion 0.618": 0.618}
print(" DAN candidate quantities and whether each lies in [0.926, 1.079]:")
for name, v in dan_vals.items():
print(f" {name:40s} {v:8.3f} {0.926 <= v <= 1.079}")
# ------------------------------------------------------------------ 3. sign logic
print(f" exp(a*b) = {np.exp(AB['DAN']):.3f}; exp(-0.438) = {np.exp(-0.438):.3f}; "
f"inside [0.926, 1.079]: {0.926 <= np.exp(AB['DAN']) <= 1.079}, {0.926 <= np.exp(-0.438) <= 1.079}")
print("\n3. Sign logic")
print(" DAN, Results text: r(anhedonia, var) = -0.266, r(var, attention) = +0.419 -> a*b < 0")
print(" DAN, Fig 7 caption: var positively with anhedonia, negatively with attention -> a*b < 0")
print(" DAN, Fig 7 paths: a = -0.509, b = +0.263 -> a*b < 0")
print(" C-D: a = +0.416, b = -0.260 -> a*b < 0; printed CI is negative (consistent)")
# ------------------------------------------------------------------ 4. standardized-coefficient check
rmx, rmy = -0.266, 0.419
c_tot = TEXT["DAN"]["direct"] + TEXT["DAN"]["indirect"]
b_std = (rmy - rmx * c_tot) / (1 - rmx ** 2)
print(f"\n4. If text coefficients were standardized: r_xy = {c_tot:.3f}, b = {b_std:.3f}, "
f"a*b = {rmx*b_std:.3f} (printed indirect -0.438) -> not standardized")
print(f" Fig 7 a = {FIG['DAN']['a']} differs from r = {rmx}: unstandardized path, so no CI can be rebuilt")
# ------------------------------------------------------------------ 5. implied n
def pval(r, n):
t = r * np.sqrt((n - 2) / (1 - r * r))
return 2 * stats.t.sf(abs(t), n - 2)
print("\n5. Sample sizes consistent with the reported r and p (r rounded to 3 dp, p to 3 dp)")
for r0, p0, lab in DAN_R:
ns = set()
for r in np.linspace(abs(r0) - 0.0005, abs(r0) + 0.0005, 21):
ns |= {n for n in range(10, 300) if round(pval(r, n), 3) == p0}
print(f" DAN variability vs {lab}: r = {r0}, p = {p0} -> n in {sorted(ns)} "
f"(p at n=188 would be {pval(r0, 188):.3e})")
# ================================================================== figures
NOTE = "Values from the paper's Results text (orange) and recomputed from its own figure path labels (blue)."
def cand_A(paths):
"""Dot-and-interval plot, one axis, both mediations."""
fig, ax = plt.subplots(figsize=(W, 3.3))
rows = [("C–D transition (EEG)", "C-D", 1.0), ("DAN temporal variability (fMRI)", "DAN", 0.0)]
for lab, k, y in rows:
t = TEXT[k]
ax.plot([t["lo"], t["hi"]], [y + 0.14] * 2, color=ORANGE, lw=LW * 2, solid_capstyle="butt")
for xb in (t["lo"], t["hi"]):
ax.plot([xb, xb], [y + 0.07, y + 0.21], color=ORANGE, lw=LW)
ax.plot(t["indirect"], y + 0.14, "o", color=ORANGE, ms=MS, mec=BG, mew=1.5, zorder=4)
ax.plot(AB[k], y - 0.14, "D", color=BLUE, ms=MS - 1, mec=BG, mew=1.5, zorder=4)
ax.text(t["indirect"], y + 0.30, f"{f(t['indirect'])}", ha="center", va="bottom", fontsize=10, color=INK)
ax.text(AB[k], y - 0.30, f"a×b = {f(AB[k])}", ha="center", va="top", fontsize=10, color=INK)
ax.text(-1.55, y, lab, ha="left", va="center", fontsize=10.5, color=INK, fontweight="bold")
t = TEXT["DAN"]
ax.text((t["lo"] + t["hi"]) / 2, 0.30, f"[{f(t['lo'])}, {f(t['hi'])}]", ha="center", va="bottom", fontsize=10)
ax.annotate("", xy=(t["lo"] - 0.02, 0.14), xytext=(t["indirect"] + 0.03, 0.14),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=1, shrinkA=0, shrinkB=0))
ax.text((t["lo"] + t["indirect"]) / 2 + 0.15, 0.04, "estimate lies 1.364 below\nthe interval's lower bound",
ha="center", va="top", fontsize=10, color=INK2)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
ax.text(0.01, 1.62, "null (0)", fontsize=10, color=INK2, ha="left", va="top")
ax.set_xlim(-1.6, 1.3)
ax.set_ylim(-0.55, 1.65)
ax.set_yticks([])
ax.set_xticks(np.arange(-1.0, 1.25, 0.25))
ax.set_xticklabels([f(v, 2) for v in np.arange(-1.0, 1.25, 0.25)])
ax.set_xlabel("Indirect effect (anhedonia → mediator → attention), unstandardized")
ax.grid(axis="x", color=GRID, lw=0.6)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
h1, = ax.plot([], [], "o-", color=ORANGE, ms=MS, lw=LW * 2, label="Reported in Results text: indirect effect and 95% CI")
h2, = ax.plot([], [], "D", color=BLUE, ms=MS - 1, label="Recomputed: a × b from the paper's figure path labels (no CI)")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.24), ncol=1, frameon=False, fontsize=10)
save(fig, *paths)
def cand_B(paths):
"""DAN-only number line: every printed DAN quantity versus the printed CI."""
fig, ax = plt.subplots(figsize=(W, 3.4))
t = TEXT["DAN"]
ax.axvspan(t["lo"], t["hi"], color=ORANGE, alpha=0.18, lw=0)
ax.plot([t["lo"], t["hi"]], [0, 0], color=ORANGE, lw=LW * 2)
ax.text((t["lo"] + t["hi"]) / 2, 1.05, "printed 95% CI\n[0.926, 1.079]", ha="center", va="bottom", fontsize=10)
pts = [ # (value, label, colour, marker, label x, label y)
(FIG["DAN"]["c"], "c (total), Fig 7", ORANGE, "o", -0.70, 0.55),
(FIG["DAN"]["a"], "a path, Fig 7", ORANGE, "o", -0.62, -0.55),
(-0.438, "'indirect' (text)\n= c\u2032 (figure)", ORANGE, "o", -0.40, 1.05),
(-0.271, "'direct' (text) =\n'bootstrap estimate' (figure)", ORANGE, "o", -0.20, -1.0),
(AB["DAN"], "a\u00d7b = c \u2212 c\u2032\n(recomputed)", BLUE, "D", 0.10, 0.55),
(FIG["DAN"]["b"], "b path, Fig 7", ORANGE, "o", 0.36, -0.55),
]
for v, lab, col, mk, tx, ty in pts:
ax.plot([v, tx], [0, ty * 0.85], color=INK2, lw=0.7, zorder=1)
ax.plot(v, 0, mk, color=col, ms=MS, mec=BG, mew=1.5, zorder=4)
ax.text(tx, ty, f"{f(v)} {lab}", ha="center", va="bottom" if ty > 0 else "top", fontsize=10, color=INK)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
ax.axhline(0, color=GRID, lw=0.8, zorder=0)
ax.set_xlim(-0.95, 1.25)
ax.set_ylim(-1.75, 1.85)
ax.set_yticks([])
ax.set_xlabel("DAN mediation: every coefficient the paper prints (unstandardized)")
ax.grid(axis="x", color=GRID, lw=0.6)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="Reported (text or figure labels)")
h2, = ax.plot([], [], "D", color=BLUE, ms=MS - 1, label="Recomputed from reported paths")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.3), ncol=2, frameon=False)
save(fig, *paths)
def box(ax, x, y, w, h, title, body, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.01,rounding_size=0.02",
fc="white", ec=edge, lw=LW, clip_on=False))
ax.text(x + 0.02, y + h - 0.035, title, fontsize=10.5, fontweight="bold", va="top", color=INK)
ax.text(x + 0.02, y + h - 0.11, body, fontsize=10, va="top", color=INK, linespacing=1.5)
def cand_C(paths):
"""Two text panels (Results text vs Figure 7c(C) labels) joined by annotated connectors."""
fig, ax = plt.subplots(figsize=(W, 4.2))
ax.set_xlim(-0.02, 1.02)
ax.set_ylim(0, 1)
ax.axis("off")
box(ax, 0.0, 0.52, 0.47, 0.46, "Results, section 3.6 (text)",
"direct effect = −0.271\nindirect effect = −0.438\n95% CI: [0.926, 1.079]", ORANGE)
box(ax, 0.53, 0.52, 0.47, 0.46, "Figure 7c(C) (path-diagram labels)",
"−0.438 (c′) −0.572 (c)\na = −0.509 b = 0.263\nBootstrap estimate [95% CI]\n= −0.271* [0.926, 1.079]",
ORANGE)
for (y0, y1), num in [((0.80, 0.865), "1"), ((0.86, 0.66), "2")]:
ax.add_patch(FancyArrowPatch((0.40, y0), (0.55, y1), arrowstyle="<->", mutation_scale=12,
color=INK2, lw=1.2))
ax.text(0.475, (y0 + y1) / 2 + (0.035 if num == "1" else -0.06), num, ha="center", va="center",
fontsize=10, fontweight="bold", color=INK,
bbox=dict(boxstyle="circle,pad=0.2", fc=BG, ec=INK2, lw=0.8))
ax.text(0.5, 0.465, "Conflict 1: −0.438 is called 'indirect' in the text but 'c′' (direct) in the figure.\n"
"Conflict 2: −0.271 is called 'direct' in the text but the 'bootstrap estimate' in the figure.",
ha="center", va="top", fontsize=10, color=INK)
box(ax, 0.0, 0.0, 1.0, 0.30, "Recomputed from the figure's own paths",
f"a × b = −0.509 × 0.263 = {f(AB['DAN'])} c − c′ = −0.572 − (−0.438) = "
f"{f(FIG['DAN']['c'] - FIG['DAN']['cprime'])}\n"
"Neither −0.438, −0.271 nor −0.134 lies inside [0.926, 1.079].", BLUE)
save(fig, *paths)
def cand_D(paths):
"""Calculation-flow diagram for the DAN indirect effect."""
fig, ax = plt.subplots(figsize=(W, 3.0))
ax.set_xlim(-0.02, 1.02)
ax.set_ylim(0, 1)
ax.axis("off")
def node(x, y, w, txt, col):
ax.add_patch(FancyBboxPatch((x, y), w, 0.2, boxstyle="round,pad=0.01,rounding_size=0.02",
fc="white", ec=col, lw=LW, clip_on=False))
ax.text(x + w / 2, y + 0.1, txt, ha="center", va="center", fontsize=10, color=INK)
def arrow(x0, y0, x1, y1):
ax.add_patch(FancyArrowPatch((x0, y0), (x1, y1), arrowstyle="-|>", mutation_scale=12, color=INK2, lw=1.2))
node(0.00, 0.72, 0.22, "a = −0.509\n(Fig 7c(C))", ORANGE)
node(0.00, 0.40, 0.22, "b = 0.263\n(Fig 7c(C))", ORANGE)
node(0.00, 0.08, 0.22, "c = −0.572\nc′ = −0.438", ORANGE)
node(0.32, 0.56, 0.22, f"a × b = {f(AB['DAN'])}", BLUE)
node(0.32, 0.08, 0.22, f"c − c′ = {f(FIG['DAN']['c'] - FIG['DAN']['cprime'])}", BLUE)
arrow(0.22, 0.82, 0.32, 0.68)
arrow(0.22, 0.50, 0.32, 0.64)
arrow(0.22, 0.18, 0.32, 0.18)
node(0.64, 0.56, 0.36, "Text: indirect = −0.438\n95% CI [0.926, 1.079]", ORANGE)
arrow(0.54, 0.66, 0.64, 0.66)
arrow(0.43, 0.28, 0.43, 0.56)
ax.text(0.82, 0.48, "Estimate ≠ a×b (−0.438 vs −0.134)\n"
"CI contains neither, and is\nentirely positive", ha="center", va="top", fontsize=10, color=INK)
ax.text(0.45, 0.40, "agree", fontsize=10, color=INK2, ha="left")
save(fig, *paths)
def cand_E(paths):
"""Sign-check grid: each mediation component's sign, by source."""
cols = ["X → M\n(a path)", "M → Y\n(b path)", "a×b", "printed\nindirect", "printed\n95% CI"]
rows = [
("DAN\nResults text", ["−\nr = −0.266", "+\nr = 0.419", "−", "−\n−0.438", "+\n[0.926, 1.079]"], [0, 0, 0, 0, 1]),
("DAN\nFig 7 caption", ["+\n'positively'", "−\n'negatively'", "−", "", ""], [1, 1, 0, 0, 0]),
("DAN\nFig 7 paths", ["−\na = −0.509", "+\nb = 0.263", f"−\n{f(AB['DAN'])}", "", ""], [0, 0, 0, 0, 0]),
("C–D\nResults, Fig 3", ["+\na = 0.416", "−\nb = −0.260", f"−\n{f(AB['C-D'])}", "−\n−0.389", "−\n[−0.571, −0.032]"], [0, 0, 0, 0, 0]),
]
fig, ax = plt.subplots(figsize=(W, 4.0))
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
x0, rh = 0.17, 0.165
widths = [0.16, 0.16, 0.13, 0.13, 0.25]
lefts = [x0 + sum(widths[:j]) for j in range(5)]
for j, c in enumerate(cols):
ax.text(lefts[j] + widths[j] / 2, 0.94, c, ha="center", va="center", fontsize=10, fontweight="bold")
ax.plot([0, 1], [0.865, 0.865], color=INK2, lw=0.8)
for i, (lab, vals, bad) in enumerate(rows):
y = 0.78 - i * rh
ax.text(0.0, y, lab, ha="left", va="center", fontsize=10, linespacing=1.3)
ax.plot([0, 1], [y - rh / 2, y - rh / 2], color=GRID, lw=0.8)
for j, (v, b) in enumerate(zip(vals, bad)):
if b:
ax.add_patch(FancyBboxPatch((lefts[j] + 0.008, y - rh / 2 + 0.012), widths[j] - 0.016, rh - 0.024,
boxstyle="round,pad=0,rounding_size=0.01", fc="none", ec=ORANGE, lw=LW))
ax.text(lefts[j] + widths[j] / 2, y, v, ha="center", va="center", fontsize=10, linespacing=1.3,
color=BLUE if j == 2 else INK)
ax.text(0.0, -0.02, "X = anhedonia, M = mediator, Y = attention.\nOrange outline: a reported sign that conflicts with the other sources.\n"
"Blue: a×b recomputed from the reported signs or path labels.", fontsize=10, color=INK2, va="bottom")
save(fig, *paths)
cand_A([os.path.join(HERE, "fig1"), os.path.join(CAND, "cand_A_dot_interval")])
cand_C([os.path.join(HERE, "fig2"), os.path.join(CAND, "cand_C_text_panels")])
cand_B([os.path.join(CAND, "cand_B_number_line")])
cand_D([os.path.join(CAND, "cand_D_calc_flow")])
cand_E([os.path.join(CAND, "cand_E_sign_grid")])
print("\nFigures written to", HERE, "and", CAND)
Paired t-test p-values for one-year cortical thickness change do not match the reported t statistics
Hum Brain Mapp (2009) · doi:10.1002/hbm.20740 · PMC2903209
Summary
The paper's only real-MRI validation (Section IV.B) tests one-year change in mean cortical thickness in 9 patients with Alzheimer's disease (AD) and 13 controls. It reports paired t statistics of −1.200 and −1.495 in AD (df = 8) and −0.274 and +0.175 in controls (df = 12), with p = 0.001, 0.001, 0.009 and 0.001. For those t statistics the two-sided p-values are 0.264, 0.173, 0.789 and 0.864. Table I independently gives t values of about the same size and p-values of 0.19 to 0.84. With the correct p-values none of the four within-group changes is statistically significant. That undoes the paper's statement that both methods "detect a systematic change" in both groups. It also undoes footnote 9's statement that FreeSurfer's increase in controls "is statistically significant", which the paper uses in its comparison with FreeSurfer.
What the paper reports
Section IV.B (Real MRI Data), paragraph 3, sample:
Our dataset includes pairs of scans over a one-year interval from 22 subjects (total of 44 scans), of whom 9 had been diagnosed with Alzheimer’s disease at their first scan, and 13 were age-matched normal subjects.
Section IV.B (Real MRI Data), paragraph 5, the tests in question:
Both techniques are able to detect a systematic change in cortical thickness over time, both in the AD (Alzheimer’s disease) group and in controls. Our method found thickness declined in AD (tpaired = -1.200; p=0.001), and in controls (tpaired = -0.274; p=0.009); FreeSurfer found thickness declined in AD (tpaired = -1.495; p=0.001), and increased in controls (tpaired = +0.175; p=0.001).9
Footnote 9, attached to that sentence (in the PMC text it sits in the back matter):
This increase, which is anatomically not expected, is statistically significant. No increase is found with our proposed method.
Section IV.B, paragraph 6, the baseline/follow-up correlations (the next paragraph):
Our technique gives measures that correlated highly over the 1 year time-interval in AD (p=0.001) and in controls (p=0.009), and so did FreeSurfer (p=0.001 both for AD and for normal subjects).
Section IV.B, paragraph 9, the interpretation:
Both our method and FreeSurfer report a few subjects that show increases (less pronounced with our approach), while on average our proposed method shows decline for both populations (much more significant for the AD subjects, as expected). Even so, FreeSurfer shows a net increase for the control subjects, not very large but still statistically significant. A number of reasons might explain this, including registration issues and segmentation errors, which are minimized in our proposed technique.
Table I ("Statistical Results on Real MRI Data"; relative change in mean thickness over one year), rows for the two groups (columns: proposed algorithm | FreeSurfer; spacing normalised):
Mean change for AD subjects | -1.42 % | -1.15 % Mean change for normal subjects | -0.19 % | +0.20 % SD of change for AD subjects | 3.52 | 2.40 SD of change for normal subjects | 3.39 | 2.82
The Abstract mentions this dataset without giving numbers ("demonstrate its performance on both artificial volumes and real 3D brain MRI data from subjects with Alzheimer’s disease and healthy individuals"). The Conclusion says the authors "presented reasonable results for longitudinal MRI scans of Alzheimer’s disease and normal subjects".
What the paper's own numbers show
Method. The tests compare each subject's baseline and follow-up thickness, so df = n − 1: 8 for AD and 12 for controls. I computed the two-sided p-value for each reported t from the t distribution, along with the one-sided p-value and the |t| that the reported p would require. As an independent check I rebuilt each t from Table I as mean % change / (SD / √n), with a rounding envelope of ±0.005 on the mean and SD. I also used Table I's "all subjects" rows to confirm the group sizes. All values come from recompute.py.txt (scipy.stats).
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Proposed method, AD: paired t, df = 8 | Sec IV.B para 5 | t = −1.200; p = 0.001 | two-sided p = 0.264 (one-sided 0.132); p = 0.001 needs |t| ≥ 5.04 |
| FreeSurfer, AD: paired t, df = 8 | Sec IV.B para 5 | t = −1.495; p = 0.001 | two-sided p = 0.173 (one-sided 0.087); p = 0.001 needs |t| ≥ 5.04 |
| Proposed method, controls: paired t, df = 12 | Sec IV.B para 5 | t = −0.274; p = 0.009 | two-sided p = 0.789 (one-sided 0.394); p = 0.009 needs |t| ≥ 3.11 |
| FreeSurfer, controls: paired t, df = 12 | Sec IV.B para 5; footnote 9 | t = +0.175; p = 0.001; "statistically significant" | two-sided p = 0.864 (one-sided 0.432); p = 0.001 needs |t| ≥ 4.32 |
| Same four t, with unlimited df (normal approximation) | — | — | p = 0.230, 0.135, 0.784, 0.861; no choice of df gives the reported p |
| t from Table I, proposed method, AD (−1.42 %, SD 3.52, n = 9) | Table I | (text: −1.200) | t = −1.210 (rounding range −1.216 to −1.204); p = 0.261 |
| t from Table I, FreeSurfer, AD (−1.15 %, SD 2.40, n = 9) | Table I | (text: −1.495) | t = −1.438 (−1.447 to −1.428); p = 0.189 |
| t from Table I, proposed method, controls (−0.19 %, SD 3.39, n = 13) | Table I | (text: −0.274) | t = −0.202 (−0.208 to −0.196); p = 0.843 |
| t from Table I, FreeSurfer, controls (+0.20 %, SD 2.82, n = 13) | Table I | (text: +0.175) | t = +0.256 (+0.249 to +0.263); p = 0.803 |
| Mean change, all subjects (weighted from group rows) | Table I | −0.69 % / −0.35 % | −0.693 % / −0.352 % |
| SD of change, all subjects (pooled from group rows) | Table I | 3.42 / 2.68 | 3.416 / 2.683 |
| Group separation, (mean normal − mean AD) / SD all | Table I | 0.36 / 0.50 | 0.360 / 0.504 |
| p-values of the t-tests vs p-values of the baseline/follow-up correlations | Sec IV.B paras 5 and 6 | 0.001, 0.009, 0.001, 0.001 (para 5); 0.001, 0.009, 0.001, 0.001 (para 6) | identical, in the same order; at n = 9 a correlation of r = 0.898 gives two-sided p = 0.001 |
The Table I checks confirm n = 9 and n = 13. All four printed t values are close to, but slightly outside, the range implied by Table I's rounded means and SDs. They differ from the Table I values by 0.010 and 0.058 (AD) and by 0.072 and 0.081 (controls). The paper does not say whether the tests used percent change or absolute thickness, which could account for those differences. They do not affect the conclusion: every t value the paper's numbers support gives a two-sided p between 0.17 and 0.86.
A separate check on the next paragraph found a smaller inconsistency of the same kind. It does not affect the main finding:
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Agreement of change between methods, all subjects (r = 0.572, n = 22) | Sec IV.B para 7 | p = 0.0028 | two-sided p = 0.0054 (one-sided 0.0027) |
| Agreement of change between methods, controls (r = 0.583, n = 13) | Sec IV.B para 7 | p = 0.0011 | two-sided p = 0.0365 (one-sided 0.0183) |
| Agreement of change between methods, AD (r = 0.558, n = 9) | Sec IV.B para 7 | p = 0.0047 | two-sided p = 0.1185 (one-sided 0.0592) |
Figure
Figure 1. The four paired t-tests of one-year change in mean cortical thickness in Section IV.B, on one log-scaled p-value axis. Orange dots are the p-values printed in the paper. Filled blue dots are the two-sided p-values for the printed t statistic and df (8 for 9 AD patients, 12 for 13 controls). Hollow blue rings are the two-sided p-values for t rebuilt from Table I's mean and SD of percent change. The thin vertical line marks p = 0.05. In every row the two recomputed values agree with each other (0.17 to 0.86) and lie well above the printed value, on the other side of the p = 0.05 line. All four changes are non-significant at the conventional level.
Figure 2. The left panel shows the four paired t-tests from Section IV.B paragraph 5, with the printed t, the printed p (orange) and the two-sided p for that t (blue). The right panel shows the p-values that paragraph 6 gives for the baseline/follow-up thickness correlations in the same four method-by-group cells. The printed p-values in the two paragraphs are identical, cell for cell. That identity is a fact about the text. It is consistent with, but does not prove, the explanation that the correlation p-values were entered in place of the t-test p-values (see Explanations considered).
Why it matters
The paper is mainly a methods paper. Its central contribution is defining thickness as a minimum line integral over a gray-matter probability map, together with the algorithm, the synthetic-data validation and the run time, and none of that depends on these p-values. The method-agreement correlation for all 22 subjects (r = 0.572) is also unaffected in substance.
The longitudinal AD/control dataset is the paper's only test on real MRI, and these four tests are its main quantitative result. With the corrected values:
- "Both techniques are able to detect a systematic change in cortical thickness over time, both in the AD … group and in controls" is not supported. Neither method detects a statistically significant one-year change in either group (p = 0.17 to 0.86 with n = 9 and n = 13). The mean changes are in the expected direction for AD. With these sample sizes and SDs of change, they do not differ significantly from zero.
- The comparative argument against FreeSurfer loses its statistical basis. Footnote 9 and paragraph 9 describe FreeSurfer's mean increase in controls as "statistically significant" and attribute it to "registration issues and segmentation errors, which are minimized in our proposed technique." The increase has p = 0.864 (t = +0.175), or 0.803 from Table I, and is consistent with no change. Neither method's control-group mean differs significantly from zero, so on these numbers the paper cannot conclude that one method is biased and the other is not.
- "Much more significant for the AD subjects" compares p-values (0.001 vs 0.009) that are not the p-values of these tests.
The descriptive content of Table I (means, SDs, group separation 0.36 vs 0.50) is internally consistent and not affected.
Explanations considered
- Different sample size or df. This does not account for the numbers. Section IV.B gives 9 AD patients and 13 controls, and Table I's pooled "all subjects" mean and SD reproduce exactly from those group sizes. No df rescues the values: even with unlimited df, t = −1.200 gives p = 0.230 and t = +0.175 gives p = 0.861.
- One-sided tests. This does not account for the numbers. The one-sided p-values are 0.132, 0.087, 0.394 and 0.432, all far above the reported values.
- The "t" values are something other than t statistics (for example mean differences or effect sizes). This does not account for the numbers. They match mean / (SD/√n) from Table I closely (within 0.06 for both AD tests), so they behave as one-sample t statistics on the paired change.
- A non-parametric test (Wilcoxon signed-rank) produced the p-values. This is not plausible for AD. With n = 9 the smallest attainable exact two-sided p is 2/2⁹ = 0.0039, so p = 0.001 cannot arise. It would also be inconsistent with the label "tpaired".
- Tests run across many vertices or voxels rather than subjects. Nothing in the paper indicates this. Table I summarises one percent change per subject, and as noted above, the reported t values cannot give these p-values at any df.
- Typographical error in the t values rather than the p-values. This does not account for the numbers. The t values agree with Table I, which is reported independently. Reaching p = 0.001 would need |t| ≥ 5.04 (AD) or ≥ 4.32 (controls), which is inconsistent with Table I's means and SDs.
- The p-values belong to a different analysis. This is consistent with the numbers. The four printed p-values are identical, in the same order, to those the next paragraph gives for the baseline/follow-up thickness correlations. Correlations of about r = 0.90 (n = 9) and r = 0.69 to 0.80 (n = 13) would produce them, which is plausible for test-retest thickness. Standard paired-samples output reports a correlation significance next to the paired-t significance. This would explain how the values could have been transposed, but it does not make them correct for the t-tests.
- PMC conversion or transcription artefact. This is ruled out. The publisher's typeset PDF (page 3193) carries the same numbers.
Status
- Version of record. The Wiley article pages (doi.org redirect to onlinelibrary.wiley.com, including /pdf, /pdfdirect and /full) returned HTTP 403. I checked the publisher's typeset PDF that PMC hosts instead (HBM-30-3188.pdf, 12 pages, "Human Brain Mapping 30:3188–3199 (2009)", © 2009 Wiley-Liss, created 2009-09-11). Page 3193 carries the paragraph 5 sentence, footnote 9 and the paragraph 6 correlation p-values exactly as quoted. Page 3194 carries Table I with the same values. Every quotation above was also confirmed verbatim against the PMC-archived text.
- Supplements. The article has no supporting information (Europe PMC hasSuppl = N), and no individual-subject data are deposited. Figure 7 shows per-subject bars only. Nothing available could resolve the discrepancy.
- Corrections and comments. As of 2026-10-23, no correction, erratum, expression of concern, retraction or published comment was found. Crossref lists no update relations (update-to / updated-by empty; filter=updates returns 0 items). PMC and Europe PMC list no notices (src/known.py). The PubMed record (PMID 19219850) has no CommentsCorrectionsList or ErratumIn entries. PubPeer's search page returned HTTP 403 to automated access and its publication API returned no record for the DOI. Web searches for the DOI with "correction", "erratum" or "PubPeer" found nothing.
Suggested correction
- Replace the four p-values in Section IV.B paragraph 5 with the p-values of the paired t-tests. For the printed t statistics these are, two-sided, 0.264 (proposed method, AD), 0.789 (proposed method, controls), 0.173 (FreeSurfer, AD) and 0.864 (FreeSurfer, controls). Also state whether the tests used percent change or absolute thickness, because the printed t values differ slightly from those implied by Table I.
- Revise the opening sentence of paragraph 5 ("Both techniques are able to detect a systematic change … both in the AD group and in controls"), footnote 9 ("is statistically significant"), and the paragraph 9 statements "much more significant for the AD subjects" and "not very large but still statistically significant", together with the inference about FreeSurfer's registration and segmentation errors that rests on them.
- Check the p-values for the method-agreement correlations in paragraph 7. For r = 0.583 (n = 13) and r = 0.558 (n = 9) the two-sided values are 0.037 and 0.119, not 0.0011 and 0.0047. Also state whether the tests are one- or two-sided (the all-subjects value of 0.0028 matches a one-sided test).
Recomputation script (python recompute.py.txt)
"""Recompute the paired t-test p-values in PMC2903209
(Hum Brain Mapp 2009;30:3188-3199, doi:10.1002/hbm.20740), Section IV.B "Real MRI Data".
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg / fig1.png and fig2.svg / fig2.png next to this file.
Inputs copied from the paper (as archived in PMC and in the publisher's PDF):
Sec IV.B para 3: 22 subjects scanned twice one year apart; 9 AD, 13 age-matched controls.
Sec IV.B para 5: four paired t statistics and their p-values.
Sec IV.B para 6: p-values of the baseline/follow-up thickness correlations.
Sec IV.B para 7: Pearson r (and p) for agreement of change between the two methods.
Table I: mean and SD of the one-year percent change in mean thickness, by group.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
import matplotlib.ticker
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs (from the paper)
N_AD, N_NC = 9, 13 # Sec IV.B para 3
# label: (reported t, n, reported p) # Sec IV.B para 5
REPORTED = {
"Proposed method, AD": (-1.200, N_AD, 0.001),
"Proposed method, controls": (-0.274, N_NC, 0.009),
"FreeSurfer, AD": (-1.495, N_AD, 0.001),
"FreeSurfer, controls": (+0.175, N_NC, 0.001),
}
# label: (mean % change, SD of % change, n) # Table I
TABLE_I = {
"Proposed method, AD": (-1.42, 3.52, N_AD),
"Proposed method, controls": (-0.19, 3.39, N_NC),
"FreeSurfer, AD": (-1.15, 2.40, N_AD),
"FreeSurfer, controls": (+0.20, 2.82, N_NC),
}
TABLE_I_ALL = { # method: (mean all, SD all, group separation)
"Proposed method": (-0.69, 3.42, 0.36),
"FreeSurfer": (-0.35, 2.68, 0.50),
}
# Baseline/follow-up correlation p-values, Sec IV.B para 6 (same order as REPORTED)
CORR_P_PARA5 = {
"Proposed method, AD": 0.001, "Proposed method, controls": 0.009,
"FreeSurfer, AD": 0.001, "FreeSurfer, controls": 0.001,
}
# Agreement of change between methods, Sec IV.B para 7: (r, n, reported p)
AGREEMENT = {
"All subjects": (0.572, N_AD + N_NC, 0.0028),
"Controls": (0.583, N_NC, 0.0011),
"AD": (0.558, N_AD, 0.0047),
}
def p_two(t, df):
return 2 * stats.t.sf(abs(t), df)
def p_one(t, df):
return stats.t.sf(abs(t), df)
def t_needed(p, df, sided=2):
return stats.t.isf(p / sided, df)
def r_pvalue(r, n, sided=2):
t = r * np.sqrt((n - 2) / (1 - r * r))
return sided * stats.t.sf(abs(t), n - 2)
def main():
rows = []
print("== 1. Reported paired t statistics vs reported p-values (Sec IV.B para 5) ==")
for lab, (t, n, prep) in REPORTED.items():
df = n - 1
p2, p1 = p_two(t, df), p_one(t, df)
print(f"{lab:27s} t={t:+.3f} df={df:2d} two-sided p={p2:.3f} one-sided p={p1:.3f} "
f"reported p={prep} |t| needed for reported p: two-sided {t_needed(prep, df):.2f}, "
f"one-sided {t_needed(prep, df, 1):.2f}")
rows.append([lab, t, df, prep, p2])
print("\n== 2. Even with unlimited df (normal approximation), the reported t cannot give the reported p ==")
for lab, (t, n, prep) in REPORTED.items():
print(f"{lab:27s} t={t:+.3f} two-sided p (df=inf) = {2 * stats.norm.sf(abs(t)):.3f}")
print("\n== 3. t rebuilt from Table I: mean % change / (SD / sqrt n) ==")
for i, (lab, (m, sd, n)) in enumerate(TABLE_I.items()):
t = m / (sd / np.sqrt(n))
p2 = p_two(t, n - 1)
# rounding envelope: mean +/- 0.005, SD +/- 0.005
ts = [(m + dm) / ((sd + ds) / np.sqrt(n)) for dm in (-0.005, 0.005) for ds in (-0.005, 0.005)]
print(f"{lab:27s} t={t:+.3f} (rounding range {min(ts):+.3f} to {max(ts):+.3f}) df={n - 1:2d} "
f"two-sided p={p2:.3f} reported t={REPORTED[lab][0]:+.3f} "
f"|reported - Table I| = {abs(REPORTED[lab][0] - t):.3f} "
f"inside rounding range: {min(ts) <= REPORTED[lab][0] <= max(ts)}")
rows[i].append(t)
rows[i].append(p2)
for lab, (t, n, _) in REPORTED.items():
m_impl = t * TABLE_I[lab][1] / np.sqrt(n)
print(f" mean % change implied by reported t and Table I SD, {lab}: {m_impl:+.3f} "
f"(Table I: {TABLE_I[lab][0]:+.2f})")
print("\n== 4. Table I internal consistency (confirms n = 9 and 13) ==")
for meth in TABLE_I_ALL:
mA, sA, _ = TABLE_I[f"{meth}, AD"]
mN, sN, _ = TABLE_I[f"{meth}, controls"]
mall_rep, sdall_rep, gs_rep = TABLE_I_ALL[meth]
mall = (N_AD * mA + N_NC * mN) / (N_AD + N_NC)
ss = (N_AD - 1) * sA**2 + (N_NC - 1) * sN**2 + N_AD * (mA - mall)**2 + N_NC * (mN - mall)**2
sdall = np.sqrt(ss / (N_AD + N_NC - 1))
gs = (mN - mA) / sdall_rep
print(f"{meth:16s} mean all={mall:+.3f} (rep {mall_rep:+.2f}) SD all={sdall:.3f} (rep {sdall_rep}) "
f"group separation={gs:.3f} (rep {gs_rep})")
print("\n== 5. Exact Wilcoxon signed-rank: smallest attainable two-sided p ==")
for n in (N_AD, N_NC):
print(f"n={n}: minimum two-sided p = 2/2^{n} = {2 / 2**n:.4f}")
print("\n== 6. Para 5 p-values vs para 6 baseline/follow-up correlation p-values ==")
for lab, (_, _, prep) in REPORTED.items():
print(f"{lab:27s} para 5 (t-test) p={prep} para 6 (correlation) p={CORR_P_PARA5[lab]} "
f"identical={prep == CORR_P_PARA5[lab]}")
for n, p in [(N_AD, 0.001), (N_NC, 0.009), (N_NC, 0.001)]:
tc = t_needed(p, n - 2)
print(f" correlation r giving two-sided p={p} at n={n}: r = {tc / np.sqrt(tc**2 + n - 2):.3f}")
print("\n== 7. Side check: Pearson r p-values for agreement of change (Sec IV.B para 7) ==")
for lab, (r, n, prep) in AGREEMENT.items():
print(f"{lab:13s} r={r} n={n} two-sided p={r_pvalue(r, n):.4f} one-sided p={r_pvalue(r, n, 1):.4f} "
f"reported p={prep}")
make_figure(rows)
make_figure2(rows)
def make_figure(rows):
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
"text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC})
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
ax.set_facecolor(BG)
y = np.arange(len(rows))[::-1]
for yi, (lab, t, df, prep, p2, t_tab, p_tab) in zip(y, rows):
lo, hi = min(prep, p2, p_tab), max(prep, p2, p_tab)
ax.plot([lo, hi], [yi, yi], color="#b9b7b1", lw=2, zorder=1)
ax.plot([min(p2, p_tab), max(p2, p_tab)], [yi, yi], color=BLUE, lw=2, zorder=2)
ax.scatter(prep, yi, s=80, color=ORANGE, zorder=3, edgecolor=ORANGE)
ax.scatter(p_tab, yi, s=190, facecolor=BG, edgecolor=BLUE, lw=2, zorder=4)
ax.scatter(p2, yi, s=70, color=BLUE, zorder=5, edgecolor=BLUE)
ax.annotate(f"reported\np = {prep:g}", (prep, yi), xytext=(0, 9), textcoords="offset points",
ha="center", va="bottom", fontsize=10, color=ORANGE, linespacing=0.95)
ax.annotate(f"{p2:.3f} from t\n{p_tab:.3f} from Table I", (max(p2, p_tab), yi), xytext=(11, 0),
textcoords="offset points", ha="left", va="center", fontsize=10, color=BLUE,
linespacing=1.0)
ax.set_yticks(y)
ax.set_yticklabels([f"{r[0]}\nt = {r[1]:+.3f}, df = {r[2]}" for r in rows], fontsize=10, color=TXT)
ax.set_xscale("log")
ax.set_xlim(3e-4, 25)
ax.set_ylim(-0.6, len(rows) - 0.25)
ax.axvline(0.05, color=SEC, lw=1.0, zorder=0)
ax.text(0.05, len(rows) - 0.35, "p = 0.05 ", ha="right", va="top", fontsize=10, color=SEC)
ax.set_xticks([0.001, 0.01, 0.05, 0.1, 1])
ax.set_xticklabels(["0.001", "0.01", "0.05", "0.1", "1"])
ax.spines["bottom"].set_bounds(3e-4, 1)
minor = [m * 10.0**e for e in (-4, -3, -2, -1) for m in range(2, 10) if 3e-4 <= m * 10.0**e < 1]
ax.xaxis.set_minor_locator(matplotlib.ticker.FixedLocator(minor))
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_xlabel("Two-sided p-value (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
handles = [
Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported p (Sec IV.B)"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE, label="Recomputed from reported t"),
Line2D([], [], marker="o", ls="", ms=9, mfc=BG, mec=BLUE, mew=2,
label="Recomputed from Table I mean, SD, n"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.42, -0.2), ncol=2,
frameon=False, fontsize=10, handletextpad=0.3, columnspacing=1.2)
fig.tight_layout()
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
print("\nwrote fig1.svg, fig1.png")
def make_figure2(rows):
"""Two text panels: para 5 t-tests and para 6 correlations share the same four p-values."""
from matplotlib.patches import FancyBboxPatch
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, EDGE = "#eb6834", "#2a78d6", "#cfcdc8"
fig = plt.figure(figsize=(7.2, 4.1), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.1)
ax.axis("off")
ax.text(3.6, 3.85, "The four p-values of the t-tests (paragraph 5) repeat, in the same order,\n"
"the four p-values of the baseline/follow-up correlations (paragraph 6)",
ha="center", va="center", fontsize=10.5, color=TXT, style="italic", linespacing=1.15)
for x0, w in ((0.1, 3.9), (4.4, 2.7)):
ax.add_patch(FancyBboxPatch((x0, 0.55), w, 2.85, boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec=EDGE, lw=1.2))
ax.text(0.25, 3.2, "Sec IV.B, paragraph 5", fontsize=12, weight="bold", color=TXT, va="center")
ax.text(0.25, 2.93, "paired t-tests of one-year change", fontsize=10.5, color=SEC, va="center")
ax.text(4.55, 3.2, "Sec IV.B, paragraph 6", fontsize=12, weight="bold", color=TXT, va="center")
ax.text(4.55, 2.93, "baseline vs follow-up correlation", fontsize=10.5, color=SEC, va="center")
for x, lab, ha in ((0.25, "group", "left"), (2.35, "t", "right"), (2.55, "reported p \u2192\nrecomputed p", "left"),
(4.55, "group", "left"), (6.15, "reported p", "left")):
ax.text(x, 2.6, lab, fontsize=10, color=SEC, ha=ha, va="center")
for i, (lab, t, df, prep, p2, *_) in enumerate(rows):
yy = 2.2 - i * 0.43
short = lab.replace("Proposed method", "Proposed")
ax.text(0.25, yy, short, fontsize=10.5, color=TXT, va="center")
ax.text(2.35, yy, f"{t:+.3f}", fontsize=10.5, color=TXT, ha="right", va="center")
ax.text(2.55, yy, f"{prep:g}", fontsize=10.5, color=ORANGE, weight="bold", va="center")
ax.text(3.08, yy, f"\u2192 {p2:.3f}", fontsize=10.5, color=BLUE, weight="bold", va="center")
ax.text(4.55, yy, short, fontsize=10.5, color=TXT, va="center")
ax.text(6.15, yy, f"{CORR_P_PARA5[lab]:g}", fontsize=10.5, color=ORANGE, weight="bold", va="center")
ax.plot([4.0, 4.4], [yy, yy], color=SEC, lw=1.2, ls=(0, (2, 2)))
ax.text(0.1, 0.3, "orange = p as printed in the paper", fontsize=10, color=ORANGE, va="center")
ax.text(0.1, 0.08, "blue = two-sided p for the printed t (df = 8 for AD, 12 for controls), recomputed (scipy)",
fontsize=10, color=BLUE, va="center")
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), facecolor=BG, **kw)
print("wrote fig2.svg, fig2.png")
if __name__ == "__main__":
main()
Table 4 brain-volume T values contradict their p values and the table's own means and SDs
Hum Brain Mapp (2022) · doi:10.1002/hbm.25798 · PMC9057088
Summary
Table 4 compares regional brain volumes between 49 healthy children and 53 children with ADHD by a two-independent-samples t test, and in 9 of its 10 rows the printed T value implies a p at least 50 times smaller than the printed p (for whole brain, T = 9.872 sits with p = .995, although no t of that size can have p above .064 at any degrees of freedom). Recomputed from the table's own means and SDs, only the hippocampus row matches its T (7.50 vs 7.401); every other row gives |t| between 0.41 and 1.42, none significant. As a result, lower frontal-lobe volume in ADHD, stated in the Abstract (table p = .008), recomputes to t = 1.42, p = .16, and lower total brain volume, stated in the Results (p < .05), has p = .995 in the table itself and p = .68 on recomputation. The paper's main finding, lower brain iron measured by QSM, rests on other tables and is not affected, but its volume findings and the Discussion built on them are.
What the paper reports
Methods, "Patient information" (group sizes):
53 children with ADHD were included in the study
49 children were included in the study
Methods, "Statistical analysis":
Two independent samples t test was used to compare the brain volume of children between the study group and the control group (p < .05 means the difference is statistically significant).
Table 4, caption and selected rows. The columns are Control group | Study group | T value | p value:
Volume values of brain regions in children [ x ¯ ± s, volume (cm3)]
Frontal | 189.77 ± 27.65 | 182.56 ± 23.53 | 12.871 | .008
Temporal | 138.32 ± 15.25 | 140.47 ± 18.34 | 10.132 | .213
Hippocampus | 12.37 ± 1.28 | 10.24 ± 1.56 | 7.401 | .002
TH | 14.97 ± 2.56 | 14.59 ± 2.39 | 8.601 | .804
Whole brain | 1452.89 ± 179.43 | 1437.96 ± 187.31 | 9.872 | .995
Abstract:
The volume of frontal lobe and hippocampus of children with ADHD was lower than that of healthy children (p < .05).
Results, "Brain volume comparison results":
The brain areas of the frontal lobe and hippocampus of children with ADHD are lower than healthy children (p < .05); the total brain volume of children with ADHD is lower than that of healthy children (p < .05) (Table 4).
Discussion, where the volume results are interpreted:
children with ADHD only have abnormal brain areas such as the frontal lobe and hippocampus (lower than healthy children)
The number of brain areas with abnormal iron content is more than the number of brain areas with abnormal volume
suggesting that the reduced volume of frontal and hippocampal brain regions may be due to decreased iron content in brain regions
the decrease in the volume of the frontal and temporal lobes and other brain regions may be due to
What the paper's own numbers show
Method. For every Table 4 row, I recomputed the two-sample t test from the printed means and SDs with n = 49 (control) and n = 53 (ADHD), using scipy.stats.ttest_ind_from_stats, both pooled-variance (Student) and Welch. I then made three checks. (i) The two-sided p each reported T implies on 100 df, the df of the stated test. (ii) The largest p any reported T could have at all, which occurs at df = 1. (iii) The |t| each reported p implies. As a control, the same procedure applied to Table 1 (age, weight, BMI and serum iron, which the Methods say were tested with the same test on the same groups) reproduces the published T and p values: t = 0.259, −0.904, −0.486 and 0.676 against 0.255, −0.904, −0.486 and 0.676. This confirms the group sizes and that "±" is the SD. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Frontal volume, T / p | Table 4 | 12.871 / .008 | t = 1.422, p = .158 (Welch 1.413, p = .161). T = 12.871 implies p = 6.1 × 10⁻²³ |
| Temporal volume, T / p | Table 4 | 10.132 / .213 | t = −0.641, p = .523 (ADHD mean is larger). T = 10.132 implies p = 5.1 × 10⁻¹⁷ |
| Hippocampus volume, T / p | Table 4 | 7.401 / .002 | t = 7.503, p = 2.6 × 10⁻¹¹. T matches; T = 7.401 implies p = 4.3 × 10⁻¹¹, not .002 |
| Thalamus (TH) volume, T / p | Table 4 | 8.601 / .804 | t = 0.775, p = .440. T = 8.601 implies p = 1.1 × 10⁻¹³ |
| Globus pallidus (GP) volume, T / p | Table 4 | 1.354 / .193 | t = −1.194, p = .235. The only row where T and p agree with each other (T implies p = .179) |
| Substantia nigra (SN) volume, T / p | Table 4 | 4.457 / .643 | t = −1.158, p = .249. T = 4.457 implies p = 2.2 × 10⁻⁵ |
| Putamen (PU) volume, T / p | Table 4 | 3.871 / .109 | t = 1.248, p = .215. T = 3.871 implies p = 1.9 × 10⁻⁴ |
| Red nucleus (RN) volume, T / p | Table 4 | 7.422 / .127 | t = 1.128, p = .262. T = 7.422 implies p = 3.9 × 10⁻¹¹ |
| Caudate nucleus (CN) volume, T / p | Table 4 | 3.214 / .566 | t = 0.977, p = .331. T = 3.214 implies p = .0018 |
| Whole-brain volume, T / p | Table 4 | 9.872 / .995 | t = 0.410, p = .682 (difference 14.93 cm³, SE 36.3). T = 9.872 implies p = 1.9 × 10⁻¹⁶ |
| Largest p possible for |t| = 9.872 | Table 4, whole brain | .995 | .064, even at df = 1 |
| |t| implied by p = .995 on 100 df | Table 4, whole brain | 9.872 | 0.006 |
| Whole-brain effect size | Table 4 | — | Cohen's d = 0.081. Reaching t = 9.872 at this d would need about 29 466 children per group |
| Frontal volume significance | Abstract; Results; Table 4 | lower in ADHD, p < .05 (table p = .008) | not significant, p = .158 |
| Total brain volume significance | Results | lower in ADHD, p < .05 | table p = .995; recomputed p = .682 |
| Hippocampus volume significance | Abstract; Results; Table 4 | lower in ADHD, p < .05 | confirmed, p = 2.6 × 10⁻¹¹ |
| Control check, Table 1 t values | Table 1 | 0.255 / −0.904 / −0.486 / 0.676 | 0.259 / −0.904 / −0.486 / 0.676 |
Table 4 therefore has two separate problems. First, the T column and the p column cannot both be correct in 9 of 10 rows. Second, the T column does not follow from the printed means and SDs. In 8 of 10 rows the printed T is 3.1 to 24 times the recomputed |t|. GP is close in size (1.354 vs 1.194) but, like temporal and SN, is printed as positive although the control mean is the smaller one. Only hippocampus agrees (ratio 0.99). The p column is closer to the summary data (all rows except frontal and hippocampus are non-significant in both), but it still does not reproduce them. Frontal is the decisive row: p = .008 is printed, and p = .158 follows from the table's own data.
A related observation. Table 3 (left vs right iron content within the ADHD group, n = 53) shows the same mismatch between its T and p columns. For example, SN has T = 4.872 with p = .993, and no t statistic of 4.872 can have a p above .129. For six of its nine rows, the reported T is also larger than a paired t test on those means and SDs could produce at any left–right correlation. Its conclusion of no left–right difference agrees with the printed p values and with an unpaired recomputation (all |t| ≤ 0.72), so this does not change a finding. It does show that the T columns in more than one table do not correspond to the stated tests. Table 2 (ANCOVA for iron content) is internally consistent: its F values reproduce its p values on F(1, 99), for example F = 1.013 gives p = .317 and F = 0.056 gives p = .813.
Figure
Figure 1. For each Table 4 row, the orange circle is the printed T value. Table 4 prints all T values without a sign, so they are drawn as positive. The blue diamond is the t statistic recomputed from the same row's means and SDs, with n = 49 controls and 53 children with ADHD, using a pooled-variance two-sample t test; positive means the control group is larger. The shaded band is |t| < 1.98, the region where p > .05 on 100 df. The two markers are offset vertically so that they stay visible where they coincide. Only the hippocampus row agrees. In every other row except GP, the printed T lies far outside the band while the recomputed t lies inside it. That includes frontal, which the Abstract reports as significantly smaller in ADHD.
Figure 2. Three two-sided p values per row on one log axis. The orange circle is the p printed in Table 4. The filled blue diamond is p recomputed from the row's means and SDs. The hollow blue square is the p that the printed T value implies on 100 df. The thin vertical line marks p = .05. If the table were internally consistent, the three markers would coincide in every row. They do so only for GP. For whole brain, the printed T implies p = 1.9 × 10⁻¹⁶, while the printed p is .995 and the recomputed p is .68. For frontal, the printed p (.008) lies on the significant side of .05 and the recomputed p (.158) on the other.
Why it matters
The paper's primary result is lower regional brain iron on quantitative susceptibility mapping in children with ADHD (Tables 2 and 3, and the ROC analysis in Table 5). Table 4 does not bear on it, and this write-up does not question it.
Table 4 is, however, the only source of the paper's volume findings, and two of them do not survive recomputation.
- Frontal-lobe volume. The Abstract, the Results and the Discussion all state that frontal volume is lower in ADHD (p < .05). The printed means differ by 7.21 cm³ against a pooled SD of 25.59 cm³ (d = 0.28). This gives t = 1.42, p = .16, which is not significant at the paper's own threshold.
- Total brain volume. The Results state that total brain volume is lower in ADHD (p < .05) and cite Table 4. The table itself prints p = .995 for whole brain, and recomputation gives p = .68 for a 1% difference.
- The Discussion's interpretation. The Discussion proposes that reduced iron delays development and so reduces the volume of the frontal (and temporal) lobes and hippocampus. It also contrasts the number of regions with abnormal iron against the number with abnormal volume. On the table's own data, only the hippocampal volume difference holds (t = 7.50). The frontal and whole-brain volume differences, and the temporal difference mentioned in the Discussion, have no support in Table 4.
Because the T column does not follow from the summary data in 9 of 10 rows, a reader also cannot use Table 4's T values for meta-analysis or to judge effect sizes. The means and SDs, which reproduce the Table 1 pattern, are the more usable part of the table, assuming they are correct.
Explanations considered
- The "±" values are standard errors, not SDs. This does not explain the table. The caption gives "x̄ ± s", and the Methods say data are shown as mean ± s. Treating them as SEMs would make every recomputed t smaller (whole brain t = 0.058), not larger. The Table 1 check reproduces only with "±" read as the SD.
- Different group sizes. This does not explain it. Table 1 reproduces exactly with n = 49 and 53. At the whole-brain effect size (d = 0.081), a t of 9.872 would need about 29 466 children per group.
- Welch (unequal-variance) t test. This does not explain it. Welch t values differ from the pooled values only in the second or third decimal (whole brain 0.411, frontal 1.413).
- An adjusted analysis, such as one for age, sex or intracranial volume. The Methods name covariate adjustment (ANCOVA for age) only for iron content, and they describe the volume comparison as a two-independent-samples t test. An adjusted model could in principle give a frontal p different from .16. It cannot make a test statistic of 9.872 sit with p = .995, or 10.132 with p = .213, under any t distribution: the largest p possible for |t| = 9.872 is .064, at df = 1. So an adjustment could explain at most one of the two columns, and the paper does not describe one.
- The T column holds a different statistic, for example Levene's F for equal variances, which SPSS prints next to the t test. Read as F(1, 100), the printed values give p = .0022 for whole brain, .0019 for temporal and .0042 for TH. These do not match the printed p values either, and a variance test on SDs as similar as 179.43 and 187.31 (variance ratio 1.09) would be unlikely to give F = 9.872. This reading does not account for the numbers.
- Rows or columns shifted during typesetting. No reordering of the printed values can match the recomputed ones. Nine recomputed |t| values lie between 0.41 and 1.42, but only one printed T (1.354) is below 3.2. Likewise, no printed p is near the recomputed hippocampus p of 2.6 × 10⁻¹¹ (the smallest printed p is .002). The PMC rendering and the Europe PMC XML have the same cells in the same order. The cells are plain text, with no lost minus signs, superscripts or decimal marks. So a conversion artifact is unlikely. The publisher's own HTML and PDF could not be opened (see Status).
- A typographical error in a single cell. This does not fit the pattern: 9 of 10 rows are affected, and the same T/p mismatch also appears in Table 3.
None of these explanations accounts for the table. The published material does not show how the T and p columns were produced.
Status
- Version of record. Wiley's article pages (onlinelibrary.wiley.com, /doi/full and /doi/pdfdirect) and the PMC PDF returned HTTP 403 or a bot challenge to automated access, so they were not read. I checked the publisher-deposited full text in two places, the PMC article (https://pmc.ncbi.nlm.nih.gov/articles/PMC9057088/) and the Europe PMC full-text XML (PMID 35107194). Table 4 is identical in both. Every quotation above was confirmed verbatim against the PMC-archived text.
- Supplements and data. The article has no supplementary-material files. The Data Availability Statement links to a private figshare item (https://figshare.com/s/1c22e32887bff32467eb). When it was opened in a browser, it was titled "ADHD DATA" and showed "No file added yet". No individual-level data were available to check Table 4 against.
- Corrections and comments. As of 2026-10-23, I found no correction, erratum, expression of concern or retraction. The Crossref work record has no update-to or updated-by entries. The Crossref filter
updates:10.1002/hbm.25798returns 0 items (re-run on 2026-10-23), and Crossref includes the Retraction Watch database. PMC lists no related notice. The Europe PMC record has no comment/correction list. Earlier web searches found no PubPeer thread or published comment on the article. On 2026-10-23, PubPeer's search page returned HTTP 403 to automated access and the web-search quota was exhausted, so a PubPeer thread could not be ruled out directly. - Sources. PMC full text (PMC9057088); Europe PMC REST API; Crossref REST API (api.crossref.org/works/10.1002/hbm.25798); the figshare link given in the article.
Suggested correction
- Re-run the volume comparisons from the source data with the test stated in the Methods, and replace the T and p columns of Table 4. If the analysis was adjusted (for example for age, sex or intracranial volume), say so in the Methods and the table footnote, and report the matching statistic and degrees of freedom.
- If the printed means and SDs are correct, revise the Abstract, the Results ("Brain volume comparison results") and the Discussion: lower frontal-lobe volume (recomputed p = .16) and lower total brain volume (table p = .995, recomputed p = .68) are not supported, and lower hippocampal volume is. Remove or qualify the Discussion passages that attribute reduced frontal and temporal volume to reduced iron.
- Check Table 3 the same way. Its T values are inconsistent with its p values, and in six rows with any paired test on the printed means and SDs. Report signed t statistics in Tables 3 and 4, as Table 1 does.
- Deposit the data in the figshare record named in the Data Availability Statement, which at the time of checking contained no files.
Recomputation script (python recompute.py.txt)
"""Recompute Table 4 (regional brain volumes) of PMC9057088
(Hum Brain Mapp 2022, doi:10.1002/hbm.25798).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the article as archived in PubMed Central:
Methods: 53 children with ADHD (study group), 49 healthy controls; brain volume
compared with a "Two independent samples t test"; data shown as mean ± s.
Table 1: patient information (used as a control check of n and the SD convention)
Table 2: iron content, ANCOVA F and p (checked for internal consistency only)
Table 3: left vs right iron content, ADHD group (T and p columns)
Table 4: volume of brain regions, mean ± s (cm3), T value, p value
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
N_C, N_S = 49, 53 # controls, ADHD
DF = N_C + N_S - 2 # 100
# ------------------------------------------------------------------ Table 4 (as printed)
# region, control mean, control s, study mean, study s, reported T, reported p
T4 = [
("Frontal", 189.77, 27.65, 182.56, 23.53, 12.871, .008),
("Temporal", 138.32, 15.25, 140.47, 18.34, 10.132, .213),
("Hippocampus", 12.37, 1.28, 10.24, 1.56, 7.401, .002),
("TH", 14.97, 2.56, 14.59, 2.39, 8.601, .804),
("GP", 3.62, 0.97, 3.87, 1.13, 1.354, .193),
("SN", 0.94, 0.16, 0.99, 0.26, 4.457, .643),
("PU", 16.97, 2.89, 16.18, 3.45, 3.871, .109),
("RN", 0.72, 0.22, 0.68, 0.13, 7.422, .127),
("CN", 9.26, 1.82, 8.96, 1.25, 3.214, .566),
("Whole brain", 1452.89, 179.43, 1437.96, 187.31, 9.872, .995),
]
# ------------------------------------------------------------------ Table 1 (control check)
T1 = [ # label, ctrl mean, ctrl s, study mean, study s, reported T, reported p
("Age", 8.56, 2.54, 8.44, 2.13, 0.255, .800),
("Weight", 28.15, 5.93, 29.23, 6.12, -0.904, .368),
("BMI", 17.19, 1.87, 17.38, 2.06, -0.486, .628),
("Trace iron", 7.38, 1.13, 7.24, 0.96, 0.676, .501),
]
# ------------------------------------------------------------------ Table 2 (F, p)
T2 = [("Frontal", 24.928, "<.001"), ("Hippocampus", 84.683, "<.001"), ("CN", 87.182, "<.001"),
("PU", 39.943, "<.001"), ("GP", 27.913, "<.001"), ("TH", 0.056, ".813"),
("Temporal", 1.013, ".317"), ("SN", 28.357, "<.001"), ("RN", 3.504, ".064")]
# ------------------------------------------------------------------ Table 3 (T, p)
T3 = [ # region, left mean, left s, right mean, right s, reported T, reported p
("Frontal", 60.18, 13.24, 61.93, 11.67, 1.823, .984),
("Temporal", 93.35, 16.41, 94.58, 14.27, 5.257, .092),
("Hippocampus", 55.27, 18.32, 56.89, 14.64, 2.874, .143),
("TH", 179.23, 23.43, 178.26, 32.54, 3.547, .752),
("GP", 393.56, 78.89, 392.47, 89.56, 1.168, .247),
("SN", 175.13, 38.03, 176.98, 45.21, 4.872, .993),
("PU", 57.65, 13.26, 56.18, 12.22, 2.146, .109),
("RN", 114.78, 26.67, 113.89, 33.78, 3.428, .136),
("CN", 321.89, 79.27, 322.96, 87.31, 1.224, .143),
]
def p2(t, df):
"""two-sided p for a t statistic"""
return 2 * stats.t.sf(abs(t), df)
print("=== 0. Control check: Table 1 reproduces with n = 49 / 53 and '±' = SD ===")
for lab, mc, sc, ms, ss, T, p in T1:
t, pp = stats.ttest_ind_from_stats(mc, sc, N_C, ms, ss, N_S, equal_var=True)
print(f" {lab:10s} recomputed t = {t:6.3f}, p = {pp:.3f} | reported T = {T:6.3f}, p = {p:.3f}")
print("\n=== 1. Table 4 recomputed from its own means and SDs (two independent samples t test) ===")
hdr = (f" {'region':12s} {'diff':>8s} {'SE':>7s} {'t_pool':>7s} {'p_pool':>8s} {'t_welch':>7s} "
f"{'p_welch':>8s} | {'T_rep':>7s} {'p_rep':>6s} {'p(T_rep|df=100)':>15s} {'p(T_rep|df=1)':>13s} "
f"{'|t| implied by p_rep':>20s}")
print(hdr)
R = []
for r, mc, sc, ms, ss, T, p in T4:
tp, pp = stats.ttest_ind_from_stats(mc, sc, N_C, ms, ss, N_S, equal_var=True)
tw, pw = stats.ttest_ind_from_stats(mc, sc, N_C, ms, ss, N_S, equal_var=False)
se = np.sqrt(sc**2 / N_C + ss**2 / N_S)
pT = p2(T, DF)
pT1 = p2(T, 1)
t_imp = stats.t.isf(p / 2, DF)
R.append(dict(region=r, diff=mc - ms, se=se, t=tp, p=pp, tw=tw, pw=pw, T=T, prep=p, pT=pT,
t_imp=t_imp))
print(f" {r:12s} {mc - ms:8.2f} {se:7.3f} {tp:7.3f} {pp:8.4f} {tw:7.3f} {pw:8.4f} | "
f"{T:7.3f} {p:6.3f} {pT:15.2e} {pT1:13.3f} {t_imp:20.3f}")
print(" (diff = control minus ADHD; SE = unpooled standard error of the difference)")
print(f" Hippocampus recomputed p = {R[2]['p']:.1e}")
print("\n=== 2. Rows where reported T and p contradict each other (df = 100) ===")
for x in R:
flag = "consistent" if abs(np.log10(x["pT"]) - np.log10(x["prep"])) < 0.3 else "INCONSISTENT"
print(f" {x['region']:12s} T={x['T']:6.3f} -> p={x['pT']:.2e}; reported p={x['prep']:.3f} {flag}")
print(" Largest two-sided p any |t| >= 9.872 can have, at the smallest possible df (1): "
f"{p2(9.872, 1):.3f}")
print(" Largest two-sided p for |t| = 4.457 (SN) at df = 1: "
f"{p2(4.457, 1):.3f}")
print("\n=== 3. Significance at alpha = .05: reported vs recomputed ===")
for x in R:
print(f" {x['region']:12s} reported p={x['prep']:.3f} ({'sig' if x['prep'] < .05 else 'ns'}) "
f"recomputed p={x['p']:.3g} ({'sig' if x['p'] < .05 else 'ns'})")
print("\n=== 4. Alternative readings ===")
wb = R[-1]
mc, sc, ms, ss = 1452.89, 179.43, 1437.96, 187.31
print(f" If '±' were SEM (whole brain): t = {(mc - ms) / np.hypot(sc, ss):.4f}")
sp = np.sqrt(((N_C - 1) * sc**2 + (N_S - 1) * ss**2) / DF)
d = (mc - ms) / sp
print(f" Whole brain Cohen's d = {d:.4f}; equal n per group needed for t = 9.872 at this d: "
f"~{2 * (9.872 / d) ** 2:.0f}")
fr = T4[0]
sp_f = np.sqrt(((N_C - 1) * fr[2]**2 + (N_S - 1) * fr[4]**2) / DF)
print(f" Frontal Cohen's d = {(fr[1] - fr[3]) / sp_f:.3f} (pooled SD {sp_f:.2f} cm3)")
print(f" Whole-brain difference as % of control mean: {100 * (mc - ms) / mc:.2f}%")
print(f" Variance-ratio F for whole-brain SDs: {(ss / sc) ** 2:.3f}")
print(" Ratio reported T / recomputed |t|:")
for x in R:
print(f" {x['region']:12s} {x['T'] / abs(x['t']):6.2f} (sign of recomputed t: {'+' if x['t'] > 0 else '-'})")
print(" Could any reordering of the printed columns match? For each recomputed value, "
"printed values within 15%:")
for x in R:
tm = [y["region"] for y in R if abs(y["T"] - abs(x["t"])) <= 0.15 * abs(x["t"])]
pm = [y["region"] for y in R if abs(y["prep"] - x["p"]) <= 0.15 * x["p"]]
print(f" {x['region']:12s} T from rows {tm or '-'}; p from rows {pm or '-'}")
print(" Reported T read as F(1,100) (e.g. a variance test), p:")
for x in R:
print(f" {x['region']:12s} F={x['T']:6.3f} -> p={stats.f.sf(x['T'], 1, DF):.4f} reported p={x['prep']:.3f}")
rec_p = [x["p"] for x in R]
rep_p = [x["prep"] for x in R]
rho, _ = stats.spearmanr(rec_p, rep_p)
print(f" Spearman correlation between recomputed and reported p across the 10 rows: {rho:.2f}")
print("\n=== 5. Table 2 (ANCOVA, iron): F vs p on F(1, 99) ===")
for r, F, p in T2:
print(f" {r:12s} F={F:7.3f} -> p={stats.f.sf(F, 1, 99):.3g} reported p={p}")
print("\n=== 6. Table 3 (left vs right iron, ADHD group n = 53): T vs p ===")
print(" r_needed = left-right correlation a paired t test would need to reach the reported T")
for r, ml, sl, mr, sr, T, p in T3:
t_ind, _ = stats.ttest_ind_from_stats(ml, sl, 53, mr, sr, 53)
sd_d = abs(ml - mr) * np.sqrt(53) / T
r_need = (sl**2 + sr**2 - sd_d**2) / (2 * sl * sr)
print(f" {r:12s} T={T:6.3f} reported p={p:.3f} p(T|df=52)={p2(T, 52):.2e} "
f"p(T|df=1)={p2(T, 1):.3f} independent-samples t={t_ind:6.3f} r_needed={r_need:.3f}")
# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"axes.edgecolor": INK2, "svg.fonttype": "none"})
labels = [x["region"] for x in R]
y = np.arange(len(R))[::-1]
def style(ax):
ax.set_facecolor(BG)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.tick_params(axis="y", length=0)
ax.set_yticks(y)
ax.set_yticklabels(labels)
# Figure 1: reported T vs recomputed t on one t axis
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
style(ax)
crit = stats.t.isf(0.025, DF)
ax.axvspan(-crit, crit, color="#efeeea", zorder=0, lw=0)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
for yi, x in zip(y, R):
ax.plot([x["t"], x["T"]], [yi - 0.14, yi + 0.14], color="#c9c8c3", lw=2, zorder=2)
ax.plot(x["T"], yi + 0.14, "o", ms=9, color=ORANGE, mec=BG, mew=1.5, zorder=4)
ax.plot(x["t"], yi - 0.14, "D", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=4)
ax.text(14.4, yi, f"{x['T']:.2f} vs {x['t']:.2f}", va="center", ha="left", fontsize=10,
color=INK2)
ax.text(14.4, len(R) - 0.35, "reported vs\nrecomputed", va="bottom", ha="left", fontsize=10,
color=INK, fontweight="bold")
ax.text(0, -1.05, f"|t| < {crit:.2f}: p > .05 (df = 100)", ha="center", va="center",
fontsize=10, color=INK2)
ax.set_xlim(-3, 14)
ax.set_ylim(-1.5, len(R) - 0.4)
ax.set_xlabel("t statistic, control minus ADHD volume")
leg = [Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported T value (Table 4)"),
Line2D([], [], marker="D", ls="", ms=8, color=BLUE,
label="t recomputed from Table 4 means and SDs (n = 49 / 53)")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 1.13), ncol=1, frameon=False,
fontsize=10)
fig.subplots_adjust(left=0.16, right=0.80, top=0.86, bottom=0.11)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# Figure 2: three p values per row on one log axis
fig, ax = plt.subplots(figsize=(7.2, 4.8), facecolor=BG)
style(ax)
ax.set_xscale("log")
ax.axvline(0.05, color=INK2, lw=0.8, zorder=1)
ax.text(0.03, -0.55, "p = .05", ha="right", va="bottom", fontsize=10, color=INK2)
for yi, x in zip(y, R):
lo = min(x["pT"], x["p"], x["prep"])
hi = max(x["pT"], x["p"], x["prep"])
ax.plot([lo, hi], [yi, yi], color="#c9c8c3", lw=2, zorder=2)
ax.plot(x["pT"], yi + 0.2, "s", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=4)
ax.plot(x["prep"], yi, "o", ms=9, color=ORANGE, mec=BG, mew=1.5, zorder=5)
ax.plot(x["p"], yi - 0.2, "D", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=4)
ax.set_xlim(1e-24, 3)
ax.set_ylim(-0.7, len(R) - 0.5)
ax.set_xticks([1e-24, 1e-20, 1e-16, 1e-12, 1e-8, 1e-4, 1])
ax.set_xlabel("two-sided p value (log scale)")
leg = [Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported p value (Table 4)"),
Line2D([], [], marker="D", ls="", ms=8, color=BLUE,
label="p recomputed from Table 4 means and SDs"),
Line2D([], [], marker="s", ls="", ms=8, mfc=BG, mec=BLUE, mew=2,
label="p implied by the reported T value (t, df = 100)")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.45, 1.2), ncol=1, frameon=False,
fontsize=10)
fig.subplots_adjust(left=0.16, right=0.97, top=0.82, bottom=0.11)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), facecolor=BG, **kw)
plt.close(fig)
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Abstract reports low birthweight rising from 9.0% to 10.1%; Table 1 shows 8.2% to 8.3%
Int J Epidemiol (2018) · doi:10.1093/ije/dyy106 · PMC6422062
Summary
The abstract of this four-cohort study from Pelotas, Brazil, states that the prevalence of low birthweight (LBW, < 2500 g) "increased from 9.0% to 10.1%" between 1982 and 2015, and that LBW was "higher for boys". The paper's own Table 1 counts give 478/5811 = 8.2% in 1982 and 345/4151 = 8.3% in 2015. No cohort reaches 10.1%, and the abstract's 9.0% matches the 2004 value. The Results, Discussion and Key Messages all describe LBW as stable (P for trend = 0.80, which reproduces exactly from the counts), and Table 2 shows LBW higher for girls in all four cohorts. LBW is one of the two outcomes named in the title, so the abstract reports a change in direction and size, and a sex difference, that the paper's tables and text do not support. The preterm figures in the same abstract sentence check out.
What the paper reports
Abstract, Results (the full sentence):
Preterm births increased from 5.8% (1982) to 13.8% (2015), and LBW prevalence increased from 9.0% to 10.1%, being higher for boys and for children born to mothers with low income and brown or black skin colour.
Abstract, Methods:
Four population-based birth cohort studies were carried out in the city of Pelotas in 1982, 1993, 2004 and 2015, with samples ranging from 4231 to 5914 liveborn children. Low birthweight (LBW) was defined as <2500 g
Methods, statistical analysis paragraph:
All analyses were restricted to live, singleton newborns.
Results, first paragraph:
The numbers of liveborn singletons in the four cohorts were 5816, 5168, 4147 and 4164. Birthweight information was missing for 5 (0.09%), 17 (0.33%), 1 (0.02%) and 13 (0.31%) children, respectively.
Table 1, "<2500" row (1982 | 1993 | 2004 | 2015):
<2500 | 8.2 | 9.1 | 9.0 | 8.3
Results, paragraph on Table 1:
The proportion of newborns with low birthweight remained stable at around 8% to 9% throughout the period (P for trend = 0.80).
Results, paragraph on Table 2:
As expected due to the use of the same absolute cutoff for both sexes, prevalence was higher among girls than boys in the four cohorts. There was no evidence of a time trend in low birthweight for either sex.
Table 2, sex rows (% LBW, 95% CI; 1982 | 1993 | 2004 | 2015 | P for linear trend):
Male | 7.2 (6.3; 8.1) | 8.0 (6.9; 9.0) | 8.1 (7.0; 9.3) | 7.6 (6.5; 8.7) | 0.476
Female | 9.3 (8.2; 10.4) | 10.2 (9.1; 11.4) | 9.9 (8.6; 11.2) | 9.1 (7.8; 10.3) | 0.784
Discussion:
One important finding of our study is that—for the whole population—mean birthweight and LBW prevalence remained stable.
Key Messages box (in the PMC archive copy this box follows the Funding statement):
At the population level, mean birthweight remained stable from 1982 to 2015, as did low birthweight.
What the paper's own numbers show
Method. Table 1 gives the number of live singletons in each 500 g birthweight band, plus the number not weighed. For each cohort I added the four bands below 2500 g and divided by the number weighed (band total, which equals the cohort total minus those not weighed). I recomputed the chi-square test for linear trend over the four cohorts (Cochran-Armitage; the Methods name "the chi-square statistic for linear trend"), the 1982–2015 difference with a Wald 95% CI, and Wilson 95% CIs per cohort. As a control I recomputed the preterm percentages from Table 1's gestational-age counts. I compared the abstract's sex statement with Table 2, and searched every LBW value in Tables 1 and 2 for a 9.0 (1982) to 10.1 (2015) series. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| LBW, 1982 | Abstract; Table 1 | 9.0% (abstract); 8.2 (Table 1) | (17+40+92+329)/5811 = 478/5811 = 8.23% (95% CI 7.55–8.96) |
| LBW, 1993 | Table 1 | 9.1 | 468/5151 = 9.09% (8.33–9.90) |
| LBW, 2004 | Table 1 | 9.0 | 372/4146 = 8.97% (8.14–9.88), the value the abstract gives for 1982 |
| LBW, 2015 | Abstract; Table 1 | 10.1% (abstract); 8.3 (Table 1) | (15+24+56+250)/4151 = 345/4151 = 8.31% (95% CI 7.51–9.19) |
| Weighed + not weighed = cohort total | Table 1 | 5816, 5168, 4147, 4164 | 5811+5, 5151+17, 4146+1, 4151+13: all match |
| LBW over all singletons, including the unweighed | Table 1 | — | 8.22%, 9.06%, 8.97%, 8.29% (no material change) |
| Highest LBW in any cohort | Table 1 | — | 9.09% (1993); no cohort rounds to 10.1% |
| Change 1982 → 2015 | Abstract vs Table 1 | +1.1 points (9.0 → 10.1), a 12.2% relative rise | +0.09 points (95% CI −1.01 to +1.18), a 1.0% relative rise |
| Linear trend, four cohorts | Results | P = 0.80 | χ² = 0.062, Z = 0.25, P = 0.803 (same with ordinal scores) |
| LBW by sex | Abstract vs Table 2 | "higher for boys" (abstract) | Table 2: girls higher in all four cohorts, by 2.1, 2.2, 1.8 and 1.5 points (Table 2 P = 0.003, 0.005, 0.044, 0.089) |
| Any Table 2 row running 9.0 → 10.1 | Table 2 | — | none (girls 9.3 → 9.1; poorest quintile 13.5 → 11.7) |
| Preterm (< 37 weeks), control | Abstract; Table 1 | 5.8% (1982), 13.8% (2015) | 265/4595 = 5.77%; 576/4164 = 13.83%: match |
Table 1, the trend P value in the Results, the Discussion and the Key Messages agree with one another. Only the abstract's LBW figures and its sex statement are out of line. The preterm half of the same abstract sentence reproduces exactly.
Figure
Figure 1. The abstract's Results sentence on LBW (left, orange: text as printed in the abstract) beside the paper's own tables and Results text (right, blue). The upper connector marks the conflict over direction: the abstract says LBW "increased" from 9.0% to 10.1% (+1.1 points), while Table 1 prints 8.2, 9.1, 9.0 and 8.3 and the Results call LBW stable (P for trend = 0.80). Recomputed from the Table 1 counts, LBW is 478/5811 = 8.2% in 1982 and 345/4151 = 8.3% in 2015 (+0.1 points), with trend P = 0.803. The lower connector marks the conflict over sex: the abstract says LBW was higher for boys, while Table 2 gives higher values for girls in every cohort (9.3 vs 7.2 in 1982, 10.2 vs 8.0, 9.9 vs 8.1, 9.1 vs 7.6 in 2015). All values come from the paper or from recompute.py.txt.
Figure 2. LBW prevalence by cohort, recomputed from the Table 1 counts (blue circles joined by a line, with Wilson 95% CIs), and the two values the abstract gives for 1982 and 2015 (orange diamonds). The abstract gives endpoints only, so they are not joined. The y-axis starts at zero. The recomputed series is flat at 8.2–9.1% (trend P = 0.80). The abstract's 1982 value (9.0%) lies just above the 1982 CI (7.55% to 8.96%), and its 2015 value (10.1%) lies above the whole 2015 CI (7.51% to 9.19%) and above every cohort.
Why it matters
LBW is one of the two outcomes in the title, and the abstract is the part of the paper most readers, indexers and evidence summaries use. The abstract tells them LBW rose by 1.1 percentage points (12% in relative terms) over 33 years. The paper's own data and its own conclusions say LBW did not change (+0.09 points, P = 0.80). A reader who takes the abstract at face value would cite a rising LBW trend in southern Brazil, which this paper does not show. The sex statement is reversed: the paper explains the female excess as expected "due to the use of the same absolute cutoff for both sexes", and the abstract says the opposite.
The paper's main conclusions are not affected. The preterm increase (5.8% to 13.8%), the stable mean birthweight, the income-group trends in mean birthweight, the adjusted 160 g decline and the caesarean-section interpretation are all stated correctly in the abstract and body. A flat LBW series is also what the abstract's own Background and Conclusions imply ("did not show reductions", "offset the expected improvements"), so the correct figures fit the paper's argument better than the printed ones. The LBW gradients by income and skin colour in the same abstract sentence are broadly consistent with Table 2 and were not examined further here.
Explanations considered
- A different denominator (excluding vs including unweighed babies). LBW over weighed newborns gives 8.23% and 8.31%; over all live singletons, 8.22% and 8.29%. Neither gives 9.0% or 10.1%. Does not explain it.
- A different population: all live births, including multiples. The abstract's Methods describe "samples ranging from 4231 to 5914 liveborn children", larger than the singleton counts, and multiples are mostly LBW. The paper does not report non-singleton counts or their birthweights, so this cannot be fully excluded from the paper's numbers alone. To reach 10.1% in 2015 would need at least 83 extra live births, every one of them LBW. If the 5914 maximum is the 1982 cohort (98 non-singletons), reaching 9.0% would need about 55% of them to be LBW. Against this, the Methods say "All analyses were restricted to live, singleton newborns", the paper reports no all-births LBW rates or trend test, and it states that LBW was stable. Even if the abstract figures came from an all-births tabulation, the abstract would still report a population and a trend that the paper's methods exclude and that its results contradict. This would also not explain the sex statement.
- The earlier three-cohort report. The paper cites its own 2008 report on the 1982, 1993 and 2004 cohorts (Cad Saude Publica 2008;24 Suppl 3:S390–S398), whose abstract says "We found a slight increase in the period in the low birth weight prevalence from 9% to 10%". The abstract's 9.0% → 10.1% resembles that earlier description, but this paper's tables and text use different figures. This may account for where the wording came from; it does not reconcile it with this paper's data.
- Gestational-age method (LMP vs best obstetric estimate). This changes only the preterm figures (the LMP-only reanalysis gives 5.8%, 11.2%, 16.2%, 17.7%). Birthweight does not depend on it. Does not explain it.
- A subgroup. No Table 2 row runs from 9.0 in 1982 to 10.1 in 2015 (girls 9.3 → 9.1, poorest quintile 13.5 → 11.7, black mothers 10.3 → 8.2). Does not explain it.
- A rounding or transcription issue in Table 1. The printed percentages reproduce from the counts to one decimal place, the band totals plus the unweighed equal the stated cohort sizes, and the trend P reproduces to 0.803. The table is internally consistent; the discrepancy is between the abstract and the table.
- A text-conversion artefact. The same abstract sentence appears in the publisher's version of record, Europe PMC full text, PubMed and the PMC archive copy. Not a conversion artefact.
Status
As of 2026-10-23:
- Version of record. The DOI resolves to the publisher's page (Int J Epidemiol 48 Suppl 1:i46–i53; online June 2018, print issue April 2019). Its abstract contains the same sentence ("LBW prevalence increased from 9.0% to 10.1%, being higher for boys ..."), and its Table 1 gives 8.2, 9.1, 9.0 and 8.3. The same text appears in Europe PMC full-text XML, the PubMed record (PMID 29939270) and the PMC archive copy.
- Supplements. The PMC/Europe PMC record has no supplementary material (Europe PMC hasSuppl = N). The publisher page lists a "Supplementary Data" entry with no files linked. No source data are available. The full PDF is behind a subscription and was not opened.
- Corrections. None found. Crossref shows no update-to, updated-by or relation entries; PMC and Europe PMC list no correction, erratum or retraction notices; the only linked item is a commentary on the supplement (doi:10.1093/ije/dyy214, "Commentary: A tale of many cities in one"), which is not an erratum. Web searches for a correction, erratum or corrigendum, and Retraction Watch, returned nothing relevant.
- PubPeer. Could not be checked: both the API and the search page returned HTTP 403.
- Sources. https://doi.org/10.1093/ije/dyy106; https://academic.oup.com/ije/article/48/Supplement_1/i46/5042986; Europe PMC REST full text for PMC6422062; PubMed efetch for PMID 29939270; Crossref API; Europe PMC record for doi:10.1590/s0102-311x2008001500004 (the 2008 report's abstract).
Suggested correction
In the abstract's Results, replace "and LBW prevalence increased from 9.0% to 10.1%, being higher for boys and for children born to mothers with low income and brown or black skin colour" with wording that matches Table 1, Table 2 and the Results, for example: "while LBW prevalence remained stable (8.2% in 1982 and 8.3% in 2015; P for trend = 0.80), being higher for girls and for children born to mothers with low income and brown or black skin colour." If the 9.0% and 10.1% figures come from a different population, such as all live births including multiples, the authors could say so, give that population's counts, and explain why it differs from the singleton population that the Methods specify for all analyses.
Recomputation script (python recompute.py.txt)
"""Recompute the low-birthweight (LBW) trend reported in PMC6422062
(Int J Epidemiol 2018, doi:10.1093/ije/dyy106).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png, fig2.svg/png and five candidate figure designs
(candidates/candidate1..5.svg/png) next to this file.
Inputs are copied from the paper as archived in PubMed Central:
Table 1: birthweight distribution (n per 500 g band, not weighed), <2500 % row,
gestational-age distribution, number of children
Table 2: LBW prevalence (95% CI) by sex of the newborn
Abstract (Results): "LBW prevalence increased from 9.0% to 10.1%, being higher for boys"
Results text: "remained stable at around 8% to 9% throughout the period (P for trend = 0.80)"
"""
import math
import os
import numpy as np
from scipy.stats import chi2
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs (from the paper)
YEARS = [1982, 1993, 2004, 2015]
BANDS = ["<1000", "1000-1499", "1500-1999", "2000-2499",
"2500-2999", "3000-3499", "3500-3999", ">=4000"]
BW = { # Table 1, n per birthweight band
1982: [17, 40, 92, 329, 1361, 2211, 1416, 345],
1993: [11, 24, 82, 351, 1283, 2040, 1080, 280],
2004: [24, 29, 70, 249, 1016, 1648, 912, 198],
2015: [15, 24, 56, 250, 960, 1713, 919, 214],
}
NOT_WEIGHED = {1982: 5, 1993: 17, 2004: 1, 2015: 13} # Table 1 "Not weighed"
N_CHILDREN = {1982: 5816, 1993: 5168, 2004: 4147, 2015: 4164} # Table 1 / Results text
T1_LBW = {1982: 8.2, 1993: 9.1, 2004: 9.0, 2015: 8.3} # Table 1 "<2500" row
GA = { # Table 1 gestational age: <37, 37-38, 39-41, 42+
1982: [265, 1007, 2854, 469], 1993: [517, 906, 2661, 518],
2004: [567, 1244, 2064, 267], 2015: [576, 1562, 1996, 30],
}
T1_PRETERM = {1982: 5.8, 1993: 11.2, 2004: 13.7, 2015: 13.8}
ABS_LBW = (9.0, 10.1) # Abstract: "increased from 9.0% to 10.1%"
ABS_PRETERM = (5.8, 13.8) # Abstract: "from 5.8% (1982) to 13.8% (2015)"
RESULTS_P_TREND = 0.80 # Results: "(P for trend = 0.80)"
T2_SEX = { # Table 2: % LBW (95% CI)
"Male": [(7.2, 6.3, 8.1), (8.0, 6.9, 9.0), (8.1, 7.0, 9.3), (7.6, 6.5, 8.7)],
"Female": [(9.3, 8.2, 10.4), (10.2, 9.1, 11.4), (9.9, 8.6, 11.2), (9.1, 7.8, 10.3)],
}
T2_SEX_P = [0.003, 0.005, 0.044, 0.089] # Table 2, P for sex difference per cohort
# Every other Table 2 row (1982 and 2015 values), to search for a 9.0 -> 10.1 pair
T2_OTHER = {"Q1": (13.5, 11.7), "Q2": (8.6, 8.0), "Q3": (6.9, 6.8), "Q4": (6.6, 8.4),
"Q5": (5.7, 6.7), "White": (7.8, 8.5), "Brown": (10.3, 7.7), "Black": (10.3, 8.2),
"Male": (7.2, 7.6), "Female": (9.3, 9.1)}
def wilson(k, n, z=1.959964):
p = k / n
d = 1 + z * z / n
c = (p + z * z / (2 * n)) / d
h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
return 100 * (c - h), 100 * (c + h)
# ---------------------------------------------------------------- 1. LBW per cohort
print("== 1. LBW (<2500 g) per cohort, from Table 1 counts ==")
lbw, wt, lbw_pct, lbw_ci = {}, {}, {}, {}
for y in YEARS:
b = BW[y]
wt[y] = sum(b)
lbw[y] = sum(b[:4])
lbw_pct[y] = 100 * lbw[y] / wt[y]
lbw_ci[y] = wilson(lbw[y], wt[y])
print(f"{y}: weighed {wt[y]} (+{NOT_WEIGHED[y]} not weighed = {wt[y] + NOT_WEIGHED[y]}, "
f"Table 1 N {N_CHILDREN[y]}); LBW n = {'+'.join(map(str, b[:4]))} = {lbw[y]}; "
f"{lbw[y]}/{wt[y]} = {lbw_pct[y]:.2f}% (Wilson 95% CI {lbw_ci[y][0]:.2f}-{lbw_ci[y][1]:.2f}); "
f"Table 1 prints {T1_LBW[y]}; over all {N_CHILDREN[y]} children = {100 * lbw[y] / N_CHILDREN[y]:.2f}%")
assert wt[y] + NOT_WEIGHED[y] == N_CHILDREN[y]
assert round(lbw_pct[y], 1) == T1_LBW[y]
print(f"Highest cohort value: {max(lbw_pct.values()):.2f}% ({max(lbw_pct, key=lbw_pct.get)})")
print(f"Abstract 9.0% equals the 2004 value ({lbw_pct[2004]:.2f}% -> {round(lbw_pct[2004], 1)}); "
f"1982 is {lbw_pct[1982]:.2f}%")
print("Cohorts whose LBW rounds to 10.1%:", [y for y in YEARS if round(lbw_pct[y], 1) == 10.1] or "none")
# ---------------------------------------------------------------- 2. change 1982 -> 2015
print("\n== 2. Change 1982 -> 2015 ==")
p1, n1 = lbw[1982] / wt[1982], wt[1982]
p2, n2 = lbw[2015] / wt[2015], wt[2015]
rd = 100 * (p2 - p1)
se = 100 * math.sqrt(p1 * (1 - p1) / n1 + p2 * (1 - p2) / n2)
rd_ci = (rd - 1.959964 * se, rd + 1.959964 * se)
print(f"Recomputed change: {rd:+.2f} percentage points (95% CI {rd_ci[0]:+.2f} to {rd_ci[1]:+.2f})")
print(f"Abstract change: {ABS_LBW[1] - ABS_LBW[0]:+.1f} percentage points (9.0 -> 10.1)")
print(f"Relative change recomputed {100 * (p2 / p1 - 1):+.1f}%; abstract {100 * (ABS_LBW[1] / ABS_LBW[0] - 1):+.1f}%")
print(f"The abstract's +1.1 lies {'inside' if rd_ci[0] <= ABS_LBW[1] - ABS_LBW[0] <= rd_ci[1] else 'outside'} "
"this interval, so the issue is the printed endpoint values, not sampling error in the difference")
# ---------------------------------------------------------------- 3. trend test
print("\n== 3. Chi-square for linear trend (Cochran-Armitage) over the four cohorts ==")
r = np.array([lbw[y] for y in YEARS], float)
n = np.array([wt[y] for y in YEARS], float)
pbar = r.sum() / n.sum()
for label, x in (("calendar-year scores", np.array(YEARS, float)), ("ordinal scores 0-3", np.arange(4.0))):
xb = (n * x).sum() / n.sum()
T = (r * (x - xb)).sum()
V = pbar * (1 - pbar) * (n * (x - xb) ** 2).sum()
print(f"{label}: chi2 = {T * T / V:.3f}, Z = {T / math.sqrt(V):.2f}, P = {chi2.sf(T * T / V, 1):.3f} "
f"(Results prints P = {RESULTS_P_TREND})")
print(f"Pooled LBW over all cohorts: {int(r.sum())}/{int(n.sum())} = {100 * pbar:.2f}%")
# ---------------------------------------------------------------- 4. preterm (control: abstract matches here)
print("\n== 4. Preterm (<37 weeks), from Table 1 gestational-age counts ==")
for y in YEARS:
g = GA[y]
print(f"{y}: {g[0]}/{sum(g)} = {100 * g[0] / sum(g):.2f}% (Table 1 prints {T1_PRETERM[y]}); "
f"gestational age unknown for {N_CHILDREN[y] - sum(g)} of {N_CHILDREN[y]}")
print(f"Abstract preterm endpoints {ABS_PRETERM}: match Table 1 -> "
f"{(round(100 * GA[1982][0] / sum(GA[1982]), 1), round(100 * GA[2015][0] / sum(GA[2015]), 1)) == ABS_PRETERM}")
# ---------------------------------------------------------------- 5. sex
print("\n== 5. LBW by sex (Table 2) vs abstract 'higher for boys' ==")
for i, y in enumerate(YEARS):
m, f = T2_SEX["Male"][i][0], T2_SEX["Female"][i][0]
print(f"{y}: male {m}%, female {f}% -> higher for {'girls' if f > m else 'boys'} "
f"(difference {f - m:+.1f} points; Table 2 P = {T2_SEX_P[i]})")
print("Girls higher in all four cohorts:",
all(T2_SEX["Female"][i][0] > T2_SEX["Male"][i][0] for i in range(4)))
# ---------------------------------------------------------------- 6. search for 9.0 -> 10.1
print("\n== 6. Does any Table 1 or Table 2 series run from 9.0 (1982) to 10.1 (2015)? ==")
hits = [k for k, (a, b) in T2_OTHER.items() if abs(a - 9.0) < 0.05 and abs(b - 10.1) < 0.05]
print("Table 2 rows with 1982 = 9.0 and 2015 = 10.1:", hits or "none")
allvals = [v for row in T2_SEX.values() for v, *_ in row] + list(T1_LBW.values()) + \
[v for pair in T2_OTHER.values() for v in pair]
print("Any Table 1/2 LBW value equal to 10.1:", [v for v in allvals if abs(v - 10.1) < 0.05] or "none")
# ---------------------------------------------------------------- 7. could non-singletons explain 9.0 / 10.1?
print("\n== 7. Abstract Methods: 'samples ranging from 4231 to 5914 liveborn children' (all live births) ==")
print("The paper does not give non-singleton counts per cohort. Minimum number e of extra (non-singleton)")
print("live births, ALL of them LBW, needed to lift each cohort to the abstract value:")
for y, target in ((1982, ABS_LBW[0]), (2015, ABS_LBW[1])):
t = target / 100
e_min = (t * wt[y] - lbw[y]) / (1 - t)
print(f"{y}: to reach {target}% needs e >= {math.ceil(e_min)} extra live births, all LBW")
e82 = 5914 - N_CHILDREN[1982]
need = 0.090 * (wt[1982] + e82) - lbw[1982]
print(f"If the 5914 maximum is the 1982 cohort: {e82} non-singletons; reaching 9.0% needs about "
f"{need:.0f} of them ({100 * need / e82:.0f}%) to be LBW; all {e82} LBW would give "
f"{100 * (lbw[1982] + e82) / (wt[1982] + e82):.2f}%")
print("So a different (all-births) denominator cannot be excluded from the paper's numbers alone,")
print("but the Methods restrict all analyses to singletons and no such rates are reported.")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
ORANGE_BG, BLUE_BG = "#fdf0ea", "#eaf2fb"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
"xtick.labelsize": 10, "ytick.labelsize": 10, "legend.fontsize": 10,
})
def save(fig, path_stem):
fig.savefig(path_stem + ".svg")
fig.savefig(path_stem + ".png", dpi=200)
plt.close(fig)
def clean(ax, grid_axis="y"):
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.spines["left"].set_color(GRID)
ax.spines["bottom"].set_color(SEC)
ax.grid(axis=grid_axis, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
ax.tick_params(length=0)
def box(ax, x, y, w, h, edge, face, title):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
lw=2, ec=edge, fc=face, transform=ax.transAxes))
ax.text(x + 0.022, y + h - 0.035, title, transform=ax.transAxes, fontsize=11.5,
fontweight="bold", va="top", color=TXT)
def txt(ax, x, y, s, **kw):
kw.setdefault("fontsize", 10.5)
ax.text(x, y, s, transform=ax.transAxes, va="top", color=kw.pop("color", TXT), linespacing=1.4, **kw)
# ---- candidate 1: two text panels joined by annotated connectors (abstract vs body)
def cand_text_panels(stem):
fig = plt.figure(figsize=(7.2, 5.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
txt(ax, 0.03, 0.97, "One abstract sentence, two conflicts with the paper's own table and text",
fontsize=12, fontweight="bold")
txt(ax, 0.03, 0.915, "LBW = birthweight < 2500 g; live singletons; Pelotas birth cohorts 1982-2015",
fontsize=10, color=SEC)
L, R, W, B, H = 0.03, 0.59, 0.38, 0.12, 0.74
box(ax, L, B, W, H, ORANGE, ORANGE_BG, "Abstract, Results")
box(ax, R, B, W, H, BLUE, BLUE_BG, "Table 1, Table 2, Results")
y1, y2 = 0.74, 0.40
txt(ax, L + 0.022, y1, "\"... and LBW prevalence\nincreased from 9.0%\nto 10.1%,")
txt(ax, L + 0.022, y2, "being higher for boys and\nfor children born to\nmothers with low income\n"
"and brown or black skin\ncolour.\"")
txt(ax, R + 0.022, y1, f"Table 1, '<2500' row (%):\n1982 {T1_LBW[1982]} 1993 {T1_LBW[1993]}\n"
f"2004 {T1_LBW[2004]} 2015 {T1_LBW[2015]}\n"
"Results: \"remained stable\nat around 8% to 9% ...\n(P for trend = 0.80)\"")
txt(ax, R + 0.022, y2, "Table 2, LBW % girls vs boys:\n"
+ "\n".join(f"{y} {T2_SEX['Female'][i][0]} vs {T2_SEX['Male'][i][0]}"
for i, y in enumerate(YEARS)))
for yy, lab in ((y1 - 0.06, "'increased'\n+1.1 pts\nvs stable\n+0.1 pts"),
(y2 - 0.06, "'boys'\nvs girls\nhigher in\nall four")):
ax.add_patch(FancyArrowPatch((L + W, yy), (R, yy), transform=ax.transAxes,
arrowstyle="<->", mutation_scale=14, lw=2, color=SEC))
ax.text((L + W + R) / 2, yy - 0.025, lab, transform=ax.transAxes, ha="center", va="top",
fontsize=10, color=TXT, linespacing=1.25)
txt(ax, 0.03, 0.085, "Orange: text as printed in the abstract. Blue: the paper's tables and Results text;\n"
f"Table 1 recomputes from its counts ({lbw[1982]}/{wt[1982]} = {lbw_pct[1982]:.1f}%, "
f"{lbw[2015]}/{wt[2015]} = {lbw_pct[2015]:.1f}%; trend P = 0.803).",
fontsize=10, color=SEC)
save(fig, stem)
# ---- candidate 2: LBW by cohort, recomputed with 95% CI, vs the abstract's two endpoints
def cand_trend(stem):
fig, ax = plt.subplots(figsize=(7.2, 4.2))
fig.subplots_adjust(left=0.10, right=0.76, top=0.84, bottom=0.12)
clean(ax)
xs = np.array(YEARS)
ys = np.array([lbw_pct[y] for y in YEARS])
lo = ys - np.array([lbw_ci[y][0] for y in YEARS]); hi = np.array([lbw_ci[y][1] for y in YEARS]) - ys
ax.errorbar(xs, ys, yerr=[lo, hi], fmt="o-", color=BLUE, lw=2, ms=8, capsize=0, elinewidth=2,
mec=BG, mew=1.5, zorder=3)
for x_, y_, l0 in zip(xs, ys, lo):
ax.text(x_, y_ - l0 - 0.2, f"{y_:.1f}", color=TXT, fontsize=10, va="top", ha="center")
ax.plot([1982, 2015], ABS_LBW, "D", color=ORANGE, ms=9, mec=BG, mew=1.5, zorder=4, ls="none")
for x_, v in zip((1982, 2015), ABS_LBW):
ax.text(x_ - 1.0, v, f"{v}", fontsize=10, va="center", ha="right")
ax.text(2016.8, ABS_LBW[1] + 0.3, "Abstract: \"increased\nfrom 9.0% to 10.1%\"\n(endpoints only)",
fontsize=10, va="center")
ax.text(2016.8, lbw_pct[2015] - 0.5, "Table 1 counts,\nrecomputed, 95% CI;\ntrend P = 0.80",
fontsize=10, va="top")
ax.set_ylim(0, 12); ax.set_xlim(1978, 2017)
ax.set_xticks(YEARS)
ax.set_ylabel("Low birthweight (%)")
fig.text(0.10, 0.955, "Low birthweight by cohort: the abstract vs the paper's own counts",
fontsize=12, fontweight="bold", va="top")
fig.text(0.10, 0.9, "Live singletons weighed at birth; the y-axis starts at zero", fontsize=10,
color=SEC, va="top")
ax.legend(handles=[Line2D([], [], color=ORANGE, marker="D", ls="none", ms=9, label="Reported in abstract"),
Line2D([], [], color=BLUE, marker="o", lw=2, ms=8, label="Recomputed from Table 1")],
loc="lower left", frameon=False, ncol=2)
save(fig, stem)
# ---- candidate 3: dumbbell of the 1982 -> 2015 change
def cand_dumbbell(stem):
fig, ax = plt.subplots(figsize=(7.2, 3.2))
fig.subplots_adjust(left=0.25, right=0.96, top=0.78, bottom=0.2)
clean(ax, "x")
rows = [("Abstract", ABS_LBW, ORANGE, "D", "change +1.1 points"),
("Table 1, recomputed", (lbw_pct[1982], lbw_pct[2015]), BLUE, "o",
f"change {rd:+.1f} points\n(95% CI {rd_ci[0]:+.1f} to {rd_ci[1]:+.1f})")]
for i, (lab, (a, b), c, m, ch) in enumerate(rows):
yy = 1 - i
ax.plot([a, b], [yy, yy], color=c, lw=2, zorder=2)
ax.plot([a], [yy], m, color=c, ms=9, mfc=BG, mew=2, zorder=3)
ax.plot([b], [yy], m, color=c, ms=9, mec=BG, mew=1.5, zorder=4)
ax.text(min(a, b) - 0.25, yy, f"1982 {a:.1f}", ha="right", va="center", fontsize=10)
ax.text(max(a, b) + 0.25, yy, f"2015 {b:.1f}", ha="left", va="center", fontsize=10)
ax.text(0.3, yy, ch, ha="left", va="center", fontsize=10, color=SEC)
ax.set_yticks([1, 0]); ax.set_yticklabels([r[0] for r in rows], fontsize=10.5, color=TXT)
ax.set_xlim(0, 12.5); ax.set_ylim(-0.6, 1.6)
ax.set_xlabel("Low birthweight (%); the axis starts at zero")
fig.text(0.03, 0.95, "Change in low birthweight, 1982 to 2015", fontsize=12, fontweight="bold", va="top")
fig.text(0.03, 0.88, "Open marker 1982, filled marker 2015", fontsize=10, color=SEC, va="top")
save(fig, stem)
# ---- candidate 4: sex panel (Table 2) vs the abstract's 'higher for boys'
def cand_sex(stem):
fig, ax = plt.subplots(figsize=(7.2, 4.0))
fig.subplots_adjust(left=0.10, right=0.78, top=0.78, bottom=0.12)
clean(ax)
off = {"Female": 0.8, "Male": -0.8}
style = {"Female": dict(color=TXT, marker="o"), "Male": dict(color=SEC, marker="s")}
for sx, rows in T2_SEX.items():
xs = np.array(YEARS) + off[sx]
v = np.array([r_[0] for r_ in rows]); l_ = np.array([r_[1] for r_ in rows]); h_ = np.array([r_[2] for r_ in rows])
ax.errorbar(xs, v, yerr=[v - l_, h_ - v], fmt=style[sx]["marker"], color=style[sx]["color"],
ms=8, elinewidth=2, capsize=0, mfc=style[sx]["color"] if sx == "Female" else BG, mew=2)
ax.text(2017.5, v[-1], f"{'Girls' if sx == 'Female' else 'Boys'} (Table 2)", va="center", fontsize=10)
ax.set_xticks(YEARS); ax.set_xlim(1978, 2017); ax.set_ylim(0, 12)
ax.set_ylabel("Low birthweight (%), 95% CI")
fig.text(0.10, 0.96, "Table 2: LBW is higher for girls in all four cohorts", fontsize=12,
fontweight="bold", va="top")
fig.text(0.105, 0.885, "Abstract: \"LBW prevalence ... being higher for boys\"", fontsize=10.5,
va="top", bbox=dict(boxstyle="round,pad=0.3", fc=ORANGE_BG, ec=ORANGE, lw=1.5))
ax.text(1978.3, 1.6, "P, girls vs boys:", fontsize=10, color=SEC, va="center")
for i, y in enumerate(YEARS):
ax.text(y, 0.8, f"{T2_SEX_P[i]}", ha="center", fontsize=10, color=SEC, va="center")
save(fig, stem)
# ---- candidate 5: calculation flow from Table 1 counts to the percentages
def cand_flow(stem):
fig = plt.figure(figsize=(7.2, 4.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.text(0.03, 0.955, "From Table 1 counts to the LBW percentage", fontsize=12, fontweight="bold", va="top")
cols = [(0.06, "Table 1 counts\n< 2500 g"), (0.32, "Weighed\nnewborns"),
(0.53, "LBW %\nrecomputed"), (0.75, "Abstract")]
for x, h in cols:
ax.text(x, 0.85, h, fontsize=10.5, fontweight="bold", va="top", color=TXT)
for j, y in enumerate((1982, 2015)):
yy = 0.58 - j * 0.27
ax.text(0.03, yy + 0.1, str(y), fontsize=10.5, fontweight="bold", color=SEC)
ax.text(0.06, yy, "+".join(map(str, BW[y][:4])) + f"\n= {lbw[y]}", fontsize=10.5, va="center")
ax.text(0.32, yy, f"{N_CHILDREN[y]} - {NOT_WEIGHED[y]}\n= {wt[y]}", fontsize=10.5, va="center")
ax.text(0.53, yy, f"{lbw[y]}/{wt[y]}\n= {lbw_pct[y]:.2f}%", fontsize=10.5, va="center",
bbox=dict(boxstyle="round,pad=0.35", fc=BLUE_BG, ec=BLUE, lw=2))
ax.text(0.75, yy, f"{ABS_LBW[j]}%", fontsize=12, va="center",
bbox=dict(boxstyle="round,pad=0.35", fc=ORANGE_BG, ec=ORANGE, lw=2))
ax.text(0.71, yy, "vs", fontsize=10.5, va="center", ha="center", color=SEC)
for x0, x1 in ((0.255, 0.305), (0.465, 0.515)):
ax.annotate("", xy=(x1, yy), xytext=(x0, yy), xycoords="axes fraction",
arrowprops=dict(arrowstyle="-|>", color=SEC, lw=1.5))
ax.text(0.86, yy, f"{ABS_LBW[j] - lbw_pct[y]:+.1f} pts\nvs Table 1", fontsize=10, va="center", color=SEC)
ax.text(0.03, 0.12, f"Table 1 itself prints {T1_LBW[1982]} and {T1_LBW[2015]}. The abstract's 9.0% "
f"equals the 2004 value\n({lbw[2004]}/{wt[2004]} = {lbw_pct[2004]:.2f}%); no cohort reaches 10.1% "
f"(highest: {max(lbw_pct.values()):.1f}% in 1993).", fontsize=10, color=SEC, va="top", linespacing=1.4)
save(fig, stem)
builders = [cand_text_panels, cand_trend, cand_dumbbell, cand_sex, cand_flow]
for i, fn in enumerate(builders, 1):
fn(os.path.join(CAND, f"candidate{i}"))
# Default selection for writeup.md until a design is chosen: fig1 = candidate 1, fig2 = candidate 2
cand_text_panels(os.path.join(HERE, "fig1"))
cand_trend(os.path.join(HERE, "fig2"))
print("\nWrote fig1, fig2 and candidates/candidate1-5 (.svg, .png) in", HERE)
Figure 6D calls a TRPV4 knockout effect significant at p = 0.06706, above the 0.05 threshold
J Neurosci (2018) · doi:10.1523/JNEUROSCI.0897-18.2018 · PMC6181316
Summary
The Figure 6D legend says that conventional TRPV4 knockout (TRPV4KO) mice "showed significantly less photoreceptor cell death than WT mice" 24 h after retinal detachment. It marks the comparison with an asterisk and prints p = 0.06706. The Methods state one decision rule for these TUNEL comparisons: "p < 0.05 was considered significant". Since 0.06706 > 0.05, the comparison is not significant under the paper's own rule. Nothing in the paper (a one-sided test, a different threshold, a correction notice) reconciles the two. Of the 16 exact p-values printed in the paper, this is the only one above 0.05. The same comparison is the first result stated in the Abstract (a ~50% reduction in apoptotic photoreceptors). The paper's main conclusion still has other support, including the Müller-glia conditional knockout in the same legend (p = 0.020863, which is significant and consistent).
What the paper reports
Methods, "Statistical analysis":
The Mann–Whitney U test was used to compare unpaired values of TUNEL-positive cell density and proinflammatory cytokine concentration. The data analyses were performed using Excel 2016 (Microsoft) with add-in software Statce14@. p < 0.05 was considered significant.
Figure 6 legend, panel D:
Conventional TRPV4KO mice showed significantly less photoreceptor cell death than WT mice 24 h after retinal detachments (*p = 0.06706, n = 7 or 8 eyes). The TRPV4CKO mice showed significantly less photoreceptor cell death than WT mice (**p = 0.020863, n = 6 eyes). The TRPV4CKO results were perfectly matched with those in TRPV4KO, indicating that Müller glial TRPV4 activation accelerates photoreceptor cell death in RD.
Abstract:
After RD, the number of apoptotic photoreceptors was reduced by ∼50% in TRPV4KO mice relative to wild-type mice, indicating the possible involvement of TRPV4 activation in RD-induced photoreceptor cell death.
Results, TUNEL paragraph. This passage gives the size of the effect but does not call the KO result significant:
The number of dead photoreceptors in TRPV4KO mice was 50% less than that for WT mice (Fig. 6D).
Consistent with those previous reports and our data (Figs. 1–5), the number of dead photoreceptors in TRPV4CKO mice was 50% less than that for WT mice, and the results were perfectly matched with the result in the conventional TRPV4KO (Fig. 6D).
Discussion, second paragraph:
Indeed, our Müller glia-specific conditional KO mice can significantly reduce the RD-induced photoreceptor cell death (Fig. 6), and the reduction ratio of dead photoreceptors was perfectly the same as the conventional TRPV4KO mice (Fig. 6).
What the paper's own numbers show
Method. The paper gives no per-eye TUNEL counts, group summaries or U statistics, so the test cannot be rerun. Two checks are possible. First, compare each printed p with the threshold stated in the Methods. Second, ask whether 0.06706 is a plausible two-sided Mann–Whitney p for the stated sample sizes, and so whether it could be a misprint of a smaller value. For the second check I enumerated the exact null distribution of U for n = 7 vs 8 (6435 rank arrangements, assuming no ties), and for 7 vs 7 and 8 vs 8 because the legend says "7 or 8 eyes". I also computed the normal approximation with and without continuity correction. For context I collected every exact p-value printed in the paper. All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Significance threshold | Methods, Statistical analysis | "p < 0.05 was considered significant" | α = 0.05 |
| WT vs conventional TRPV4KO, TUNEL+ photoreceptors at 24 h | Fig. 6D legend | "significantly less", *p = 0.06706, n = 7 or 8 eyes | 0.06706 > 0.05 by 0.01706 (1.34 × the threshold); not significant under the stated rule. Equivalent |z| = 1.831, against 1.960 needed |
| WT vs TRPV4CKO | Fig. 6D legend | "significantly less", **p = 0.020863, n = 6 eyes | 0.020863 < 0.05; significant, consistent |
| Other exact p-values in the paper | legends of Figs. 4, 5, 7, 8, 9, 10 | 14 values, from 0.000116 to 0.012851 | all < 0.05. Of the 16 printed exact p-values, 0.06706 is the only one ≥ 0.05; the next largest is 0.020863 |
| Hypothetical one-sided reading | not stated in the paper | — | 0.06706 / 2 = 0.03353 (would be < 0.05). If 0.06706 were already one-sided, the two-sided p would be 0.13412 |
| Exact two-sided Mann–Whitney p near the printed value, n = 7 vs 8, no ties | recomputation | 0.06706 | U = 11: 0.05408; U = 12: 0.07211. The printed value lies between them, and both are above 0.05 |
| Normal approximations closest to 0.06706, n = 7 vs 8 | recomputation | 0.06706 | U = 12: 0.06408 (no continuity correction), 0.07285 (with); U = 11 with correction: 0.05620 |
| Same check for n = 7 vs 7 and 8 vs 8 | recomputation | 0.06706 | 7 vs 7: exact 0.05303 / 0.07284 (U = 9 / 10); 8 vs 8: exact 0.06496 / 0.08298 (U = 14 / 15) |
| Exact p near a misprint value of 0.006706, n = 7 vs 8 | recomputation | — | U = 5: 0.00591; U = 6: 0.00932. A value near 0.0067 is also within the attainable range, so a misprint cannot be excluded |
The printed 0.06706 is not itself an exact no-ties value for any of these sample sizes. Ties in the TUNEL densities, or a normal approximation as statistics add-ins commonly use, would give values between the exact steps. As printed, 0.06706 falls where an ordinary two-sided Mann–Whitney result for about 7–8 eyes per group would fall when the rank separation is just short of significance (U = 11–12 of a possible 56 for 7 vs 8).
Figure
On a linear p-value axis starting at 0, the two orange dots are the p-values printed in the Figure 6D legend. Both comparisons are described there as "significantly less". The vertical line is the threshold stated in the Methods (p < 0.05), and the shaded band is the region the paper counts as significant. WT vs TRPV4CKO (p = 0.020863) lies inside it. WT vs conventional TRPV4KO (p = 0.06706) lies outside it. The blue open squares are recomputed: the two exact two-sided Mann–Whitney p-values on either side of the printed value for n = 7 vs 8 eyes with no ties (U = 11: 0.0541; U = 12: 0.0721). Both are above 0.05. Taken as printed, the knockout result is a two-sided value just short of significance.
All 16 exact p-values printed in the paper's figure legends, sorted and shown on a log scale. Every open dot is a comparison the legends call significant, and each lies below 0.05. The filled dot (Fig. 6D, WT vs TRPV4KO, p = 0.06706) is the only printed value above the threshold, and the legend calls it "significantly less" all the same. Values are copied from the legends; panel letters follow the legends' own labels.
Why it matters
The WT vs conventional TRPV4KO comparison is the first result in the Abstract ("reduced by ∼50% in TRPV4KO mice relative to wild-type mice"). It is also one half of an argument made twice in the paper: the CKO effect is "perfectly matched" with, or "perfectly the same as", the conventional KO effect, and this match is used to assign the effect to Müller glial TRPV4. Under the paper's criterion, the KO side of that comparison is a non-significant trend (p = 0.067, about 7–8 eyes per group), not an established reduction. The direction of the effect does not change. The 50% figure cannot be checked because no group values are given.
The paper's overall conclusion, that Müller glial TRPV4 contributes to photoreceptor death after retinal detachment, is weakened but not overturned. In the same legend, the Müller-glia/astrocyte conditional knockout reaches significance (p = 0.020863, n = 6). The agonist GSK increases photoreceptor death (Fig. 7A, p = 0.002622). The antagonist HC reduces it (Fig. 7B, p = 0.009823). The MCP-1 and macrophage experiments (Figs. 8–10) point the same way. Readers who rely on the global-knockout result as an independent replication, or quote the Abstract's ~50% reduction as statistically established, are relying on a comparison that the paper's own numbers do not support at α = 0.05.
Explanations considered
- A different or looser threshold for this figure. The Methods give a single rule, "p < 0.05 was considered significant", and name the Mann–Whitney test for TUNEL densities. The Figure 6 legend sets no other threshold. This does not explain the discrepancy.
- A one-sided test. Halving 0.06706 gives 0.0335, which would pass. The paper never says that any test was one-sided; searches for "one-tailed", "two-tailed" and "one-sided" return nothing. If 0.06706 were already one-sided, the two-sided value would be 0.134, which is further from significance. This does not explain it unless an unstated convention was used.
- The asterisk convention. The paper uses asterisks inconsistently. In Fig. 6D a single * marks p = 0.067 and ** marks p = 0.021. In Figs. 8A and 8C asterisks mean p < 0.01, and in Fig. 8B the asterisk means p < 0.05. No convention in the paper makes 0.067 count as significant, so this does not explain it. The published figure image, as described in the version-of-record check, marks WT vs KO with a single asterisk, which matches the legend.
- A misprint for 0.006706 (or similar). This cannot be excluded without the raw data. A value near 0.0067 is attainable for n = 7 vs 8 (exact values 0.0059 and 0.0093 at U = 5 and 6). But 0.06706 is also an ordinary two-sided value for these sample sizes, between the exact values 0.054 and 0.072 and close to the normal approximations at U = 11–12. Nothing else in the paper points to a smaller value. If it is a misprint, then the p-value is wrong rather than the significance label. Either way the published text is internally inconsistent.
- A different comparison, population or test. The legend gives the comparison (WT vs conventional TRPV4KO, 24 h after detachment, n = 7 or 8 eyes) and the p-value in the same sentence. The legend does not name the test. Some other legends in the paper use Student's t test, but the threshold the paper states is the same whichever test was used.
- Multiplicity adjustment. An adjustment could only raise a p-value, so it cannot turn 0.067 into a significant result.
- Rounding. 0.06706 is printed to five decimals. No rounding brings it below 0.05.
- Scope of the claim. The word "significantly" for the KO comparison appears only in the Figure 6D legend, and there it carries an asterisk. The Results text reports "50% less" without a significance claim. The Discussion's "significantly" refers to the CKO, which is valid. The Abstract states the ~50% reduction as a finding without a p-value. So the explicit inconsistency is confined to the Fig. 6D legend, and the Abstract states the same result as established.
- A later correction. No correction, erratum or retraction is listed (see Status).
Status
As of 2026-10-23:
- Version of record. The wording and numbers above are in the publisher's typeset PDF as hosted by PMC (14 pages; page 7 for the Figure 6 legend, page 6 for Statistical analysis). They are also in the live PMC page and the Europe PMC full-text XML. Relation symbols in the PDF were read from rendered page images, because its text layer drops them. The publisher's own site (jneurosci.org) refused automated access, so its HTML was not compared. The inconsistency is in the publisher's typeset text, not a conversion artifact.
- Supplements. The article has no supplementary data files. The Europe PMC supplementary bundle contains only the figure images. The Fig. 6D dot plot shows individual eyes but no values, so nothing in the paper can settle whether the p-value or the significance label is wrong.
- Corrections. Crossref lists no update-to or updated-by relations, and a Crossref filter for works that update this DOI returned none. PMC and Europe PMC list no correction, erratum, expression of concern or retraction (checked with
src/known.pyand Europe PMC search, PMID 30143574). - PubPeer. PubPeer's search refused automated access (Cloudflare verification), so it could not be checked directly. A web search for the title found no PubPeer entry.
- Sources. PMC6181316; doi:10.1523/JNEUROSCI.0897-18.2018; Crossref and Europe PMC REST APIs.
Suggested correction
The Figure 6D legend should be made consistent with the Methods. There are two possibilities:
- If p = 0.06706 is correct: change the legend to describe the KO vs WT difference as a non-significant trend, for example "TRPV4KO mice showed a trend toward less photoreceptor cell death than WT mice (p = 0.067, n = 7 or 8 eyes)". Remove the asterisk from the figure. Qualify the Abstract's "reduced by ∼50%" sentence and the "perfectly matched" / "perfectly the same" statements to match.
- If the printed value is a misprint: give the correct p-value.
In either case, it would help readers if the authors stated the group sizes for each genotype (which group had 7 eyes and which had 8), the test used for Fig. 6D, whether it was one- or two-sided and exact or approximate, and the group summaries (median and range, or the per-eye values) behind the "50% less" figure.
Recomputation script (python recompute.py.txt)
"""Recompute the numbers behind the Figure 6D significance claim in PMC6181316
(J Neurosci 2018, doi:10.1523/JNEUROSCI.0897-18.2018).
Run: python recompute.py.txt # prints every number, writes fig1/fig2 (.svg, .png)
python recompute.py.txt --candidates # also writes five candidate figure designs
# to candidates/ (A-E) for review
Needs numpy, scipy and matplotlib.
Inputs are copied from the paper as archived in PMC (PMC6181316):
Methods, "Statistical analysis": Mann-Whitney U test for TUNEL-positive cell density;
"p < 0.05 was considered significant."
Figure 6D legend: WT vs conventional TRPV4KO, "*p = 0.06706, n = 7 or 8 eyes";
WT vs TRPV4CKO, "**p = 0.020863, n = 6 eyes". Both called "significantly less".
Every other exact p-value printed in the figure legends (Figs. 4, 5, 7, 8, 9, 10).
The paper gives no per-eye values, so the test itself cannot be rerun. What can be
checked is (1) the printed p against the stated threshold and (2) whether the printed
p is a plausible Mann-Whitney value for the stated sample sizes. For (2) the exact
null distribution of U is enumerated (no ties assumed) and the normal approximation
is computed with and without continuity correction.
"""
import os
import sys
from itertools import combinations
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ----------------------------------------------------------------- inputs (from the paper)
ALPHA = 0.05 # Methods, Statistical analysis
P_KO = 0.06706 # Fig. 6D legend, WT vs conventional TRPV4KO, "n = 7 or 8 eyes"
P_CKO = 0.020863 # Fig. 6D legend, WT vs TRPV4CKO, "n = 6 eyes"
# every exact p-value printed in the paper (location, label, p, significance wording)
ALL_P = [
("Fig. 4C", "pressure-evoked currents", 0.00056),
("Fig. 5C", "TRPV4-evoked currents", 0.0058),
("Fig. 6D", "WT vs TRPV4KO, TUNEL", P_KO),
("Fig. 6D", "WT vs TRPV4CKO, TUNEL", P_CKO),
("Fig. 7A", "GSK vs control, TUNEL", 0.002622),
("Fig. 7B", "HC vs control, TUNEL", 0.009823),
("Fig. 8A", "MCP-1 mRNA, RD vs control", 0.000116),
("Fig. 8A", "MCP-1 mRNA, vs WT RD", 0.000232),
("Fig. 8B", "MCP-1 protein", 0.0010509),
("Fig. 8C", "GSK vs control", 0.000247),
("Fig. 8C", "Hypo vs control", 0.00944),
("Fig. 8C", "GSK+HC vs GSK", 0.001683),
("Fig. 8C", "Hypo+HC vs Hypo", 0.00092),
("Fig. 9A", "anti-MCP-1 vs PBS, TUNEL", 0.012611),
("Fig. 9A", "anti-MCP-1 vs IgG, TUNEL", 0.012851),
("Fig. 10", "CD11b+ cells", 0.00632),
]
# ----------------------------------------------------------------- helpers
def exact_U(n1, n2):
"""All C(n1+n2, n1) values of U for group 1 under H0, no ties."""
N = n1 + n2
return np.array([sum(i + 1 for i in c) - n1 * (n1 + 1) / 2
for c in combinations(range(N), n1)])
def exact_p2(Us, u):
return min(1.0, 2 * np.mean(Us <= u))
def normal_p2(u, n1, n2, cc=True):
mu = n1 * n2 / 2
sd = np.sqrt(n1 * n2 * (n1 + n2 + 1) / 12)
z = (abs(u - mu) - (0.5 if cc else 0.0)) / sd
return 2 * stats.norm.sf(z), z
# ----------------------------------------------------------------- 1. threshold check
print("1. Printed p-values against the stated threshold (p < 0.05)")
print(f" WT vs TRPV4KO : p = {P_KO} -> below 0.05? {P_KO < ALPHA}; "
f"exceeds 0.05 by {P_KO - ALPHA:.5f} ({P_KO / ALPHA:.3f} x the threshold)")
print(f" WT vs TRPV4CKO: p = {P_CKO} -> below 0.05? {P_CKO < ALPHA}")
print(f" z implied by a two-sided p of {P_KO}: {stats.norm.isf(P_KO / 2):.4f} "
f"(1.9600 needed for p = 0.05)")
# ----------------------------------------------------------------- 2. other p-values in the paper
print("\n2. All exact p-values printed in the paper")
above = [r for r in ALL_P if r[2] >= ALPHA]
for loc, lab, p in sorted(ALL_P, key=lambda r: r[2]):
print(f" {loc:8s} {lab:28s} p = {p:<10g} {'>= 0.05' if p >= ALPHA else '< 0.05'}")
print(f" {len(ALL_P)} printed p-values; {len(above)} at or above 0.05: "
f"{[(r[0], r[2]) for r in above]}")
print(f" largest p other than the KO comparison: "
f"{max(r[2] for r in ALL_P if r[2] != P_KO)}")
# ----------------------------------------------------------------- 3. hypothetical one-sided readings
print("\n3. One-sided readings (the paper states none)")
print(f" if 0.06706 were two-sided and halved: {P_KO / 2:.5f} (< 0.05: {P_KO / 2 < ALPHA})")
print(f" if 0.06706 were already one-sided, two-sided p = {min(1, 2 * P_KO):.5f}")
# ----------------------------------------------------------------- 4. attainable Mann-Whitney p-values
print("\n4. Mann-Whitney p-values attainable for the stated sample sizes (no ties)")
EXACT = {}
for n1, n2 in [(7, 8), (7, 7), (8, 8)]:
Us = exact_U(n1, n2)
EXACT[(n1, n2)] = Us
print(f" n = {n1} vs {n2}: {len(Us)} rank arrangements")
for u in range(8, 16):
pe = exact_p2(Us, u)
pc, _ = normal_p2(u, n1, n2, True)
pn, _ = normal_p2(u, n1, n2, False)
flag = " <- brackets 0.06706" if (exact_p2(Us, u - 1) < P_KO <= pe or
exact_p2(Us, u) < P_KO <= exact_p2(Us, u + 1)) else ""
print(f" U = {u:2d}: exact 2-sided {pe:.5f}; normal cc {pc:.5f}; "
f"normal no-cc {pn:.5f}{flag}")
U78 = EXACT[(7, 8)]
EX11, EX12 = exact_p2(U78, 11), exact_p2(U78, 12)
NORM = sorted([normal_p2(11, 7, 8, True)[0], normal_p2(12, 7, 8, True)[0],
normal_p2(12, 7, 8, False)[0]])
print(f" n = 7 vs 8: 0.06706 lies between exact values {EX11:.5f} (U = 11) and "
f"{EX12:.5f} (U = 12)")
print(f" n = 7 vs 8: normal approximations closest to 0.06706 (U = 11 with cc, U = 12 with and "
f"without cc) span {NORM[0]:.5f} to {NORM[-1]:.5f}")
print(f" gap between those two exact values: {EX12 - EX11:.5f}; 0.06706 is not itself an "
f"exact no-ties value")
P_TYPO = 0.006706
print(f" for reference, a value of {P_TYPO} would lie between exact values "
f"{exact_p2(U78, 5):.5f} (U = 5) and {exact_p2(U78, 6):.5f} (U = 6)")
U66 = exact_U(6, 6)
print(f" n = 6 vs 6 (CKO): exact U = 3 -> {exact_p2(U66, 3):.5f}, U = 4 -> "
f"{exact_p2(U66, 4):.5f}; normal cc U = 3 -> {normal_p2(3, 6, 6, True)[0]:.5f}, "
f"no-cc U = 4 -> {normal_p2(4, 6, 6, False)[0]:.5f}")
# ----------------------------------------------------------------- figure style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, GRAY = "#eb6834", "#2a78d6", "#8a8984"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "axes.facecolor": BG, "figure.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
LW, MS = 1.5, 8 # 1.5 pt = 2 px at 96 dpi; markers 8 pt
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
print(" wrote", os.path.relpath(path_noext, HERE) + ".svg/.png")
def clean(ax, grid_axis="x"):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis=grid_axis, color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
# ----------------------------------------------------------------- design A: p-value axis
def design_A(path):
from matplotlib.lines import Line2D
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ax.axvspan(0, ALPHA, color="#eef3fa", zorder=0)
ax.axvline(ALPHA, color=INK2, lw=1.0)
ax.text(ALPHA, 3.05, "stated threshold p = 0.05 (Methods)", ha="center", va="bottom",
fontsize=10, color=INK2)
ax.text(0.025, 2.85, "significant (p < 0.05)", ha="center", va="center",
fontsize=10, color=INK2)
ax.text(0.0775, 2.85, "not significant", ha="center", va="center",
fontsize=10, color=INK2)
y = {"cko": 0.4, "mw": 1.2, "ko": 2.1}
ax.plot(P_CKO, y["cko"], "o", ms=MS, color=ORANGE, zorder=3)
ax.annotate(f"p = {P_CKO} (**), \"significantly less\"", (P_CKO, y["cko"]),
xytext=(9, 0), textcoords="offset points", va="center", fontsize=10)
ax.plot(P_KO, y["ko"], "o", ms=MS, color=ORANGE, zorder=3)
ax.annotate(f"p = {P_KO} (*)\n\"significantly less\"", (P_KO, y["ko"]),
xytext=(9, 0), textcoords="offset points", va="center", fontsize=10)
ax.hlines(y["mw"], EX11, EX12, color=BLUE, lw=LW, zorder=2)
ax.plot([EX11, EX12], [y["mw"]] * 2, "s", ms=MS, color=BLUE, zorder=3, mfc=BG, mew=LW)
for u, pe, ha in ((11, EX11, "right"), (12, EX12, "left")):
ax.text(pe + (-0.0012 if ha == "right" else 0.0012), y["mw"] + 0.24,
f"U = {u}: {pe:.4f}", ha=ha, va="bottom", fontsize=10, color=INK2)
ax.plot([P_KO, P_KO], [y["mw"] + 0.12, y["ko"] - 0.12], color=GRAY, lw=1.0, ls=(0, (2, 2)))
ax.set_yticks([y["cko"], y["mw"], y["ko"]])
ax.set_yticklabels(["WT vs TRPV4CKO\n(n = 6 eyes)",
"neighbouring exact\nMann-Whitney p,\nn = 7 vs 8",
"WT vs TRPV4KO\n(n = 7 or 8 eyes)"], fontsize=10, color=INK)
ax.set_xlim(0, 0.10)
ax.set_ylim(0, 3.35)
ax.set_xlabel("two-sided p value (linear scale from 0)")
clean(ax)
h = [Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE,
label="reported in the Fig. 6D legend"),
Line2D([], [], marker="s", ls="-", lw=LW, ms=MS, color=BLUE, mfc=BG, mew=LW,
label="recomputed: exact two-sided p (no ties)")]
fig.legend(handles=h, loc="lower center", ncol=2, frameon=False, fontsize=10,
bbox_to_anchor=(0.5, 0.0))
fig.tight_layout(rect=(0, 0.07, 1, 1))
save(fig, path)
# ----------------------------------------------------------------- design B: all p-values in the paper
def design_B(path):
rows = sorted(ALL_P, key=lambda r: r[2])
fig, ax = plt.subplots(figsize=(7.2, 5.4))
ax.axvline(ALPHA, color=INK2, lw=1.0)
ax.text(ALPHA * 1.08, 7.5, "stated threshold p = 0.05", ha="left", rotation=90,
va="center", fontsize=10, color=INK2)
for i, (loc, lab, p) in enumerate(rows):
is_ko = p == P_KO
ax.hlines(i, 1e-4, p, color=GRID, lw=0.8, zorder=1)
ax.plot(p, i, "o", ms=MS if is_ko else MS - 1, color=ORANGE,
mfc=ORANGE if is_ko else BG, mew=LW, zorder=3)
if is_ko:
ax.text(0.0009, 1.3, "filled dot: p = 0.06706, the\nonly printed p above 0.05, yet\n"
"called \"significantly less\"", ha="left", va="center", fontsize=10,
color=INK)
ax.set_yticks(range(len(rows)))
ax.set_yticklabels([f"{r[0]} {r[1]}" for r in rows], fontsize=10)
for t, r in zip(ax.get_yticklabels(), rows):
t.set_color(INK if r[2] == P_KO else INK2)
if r[2] == P_KO:
t.set_fontweight("bold")
ax.set_xscale("log")
ax.set_xlim(8e-5, 0.3)
ax.set_ylim(-0.7, len(rows) - 0.3)
ax.set_xlabel("p value as printed in the figure legends (log scale)")
ax.set_xticks([1e-4, 1e-3, 1e-2, 0.05, 0.1])
ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.05", "0.1"])
clean(ax)
from matplotlib.lines import Line2D
h = [Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, mfc=BG, mew=LW,
label="reported p, called significant"),
Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE,
label="reported p = 0.06706 (Fig. 6D, KO)")]
fig.legend(handles=h, loc="lower center", ncol=2, frameon=False, fontsize=10,
bbox_to_anchor=(0.5, 0.0))
fig.tight_layout(rect=(0, 0.05, 1, 1))
save(fig, path)
# ----------------------------------------------------------------- design C: two text panels
def panel(ax, x, y, w, h, title, body, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.01,rounding_size=0.02",
fc="#ffffff", ec=edge, lw=LW))
ax.text(x + 0.02, y + h - 0.04, title, fontsize=10, fontweight="bold", va="top")
ax.text(x + 0.02, y + h - 0.13, body, fontsize=10, va="top", linespacing=1.4)
def design_C(path):
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
panel(ax, 0.01, 0.31, 0.36, 0.66, "Methods, \"Statistical analysis\"",
"\"The Mann-Whitney U test\nwas used to compare\nunpaired values of\nTUNEL-positive cell\n"
"density ... p < 0.05 was\nconsidered significant.\"", INK2)
panel(ax, 0.63, 0.31, 0.36, 0.66, "Figure 6D legend",
"\"Conventional TRPV4KO\nmice showed significantly\nless photoreceptor cell\n"
"death than WT mice 24 h\nafter retinal detachments\n(*p = 0.06706, n = 7 or 8\neyes).\"",
ORANGE)
ax.annotate("", xy=(0.62, 0.66), xytext=(0.38, 0.66),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=LW))
ax.text(0.50, 0.70, "conflict", ha="center", va="bottom", fontsize=10, color=INK,
fontweight="bold")
ax.text(0.50, 0.62, "0.06706 is not\n< 0.05, but is\ncalled significant", ha="center",
va="top", fontsize=10, color=INK2)
ax.add_patch(FancyBboxPatch((0.02, 0.03), 0.96, 0.22,
boxstyle="round,pad=0.01,rounding_size=0.02",
fc="#eef3fa", ec=BLUE, lw=LW))
ax.text(0.5, 0.14, f"Recomputed: {P_KO} > {ALPHA} (by {P_KO - ALPHA:.5f}), so not significant "
"under the paper's rule.\nThe CKO result in the same legend "
f"(p = {P_CKO}) is below 0.05 and is consistent.", ha="center",
va="center", fontsize=10)
ax.text(0.01, 1.0, "gray frame: stated rule; orange frame: reported claim; blue frame: recomputed check",
fontsize=10, color=INK2, va="bottom")
ax.set_ylim(0, 1.07)
save(fig, path)
# ----------------------------------------------------------------- design D: exact ladder of attainable p
def design_D(path):
fig, ax = plt.subplots(figsize=(7.2, 3.4))
us = np.arange(5, 15)
pe = [exact_p2(U78, u) for u in us]
pc = [normal_p2(u, 7, 8, True)[0] for u in us]
ax.axhline(ALPHA, color=INK2, lw=1.0)
ax.text(14.6, ALPHA + 0.002, "stated threshold p = 0.05", ha="right", va="bottom",
fontsize=10, color=INK2)
ax.plot(us, pe, "s-", color=BLUE, lw=LW, ms=MS, mfc=BG, mew=LW,
label="recomputed: exact two-sided p, n = 7 vs 8")
ax.plot(us, pc, "o--", color=BLUE, lw=LW, ms=MS - 2, alpha=0.75,
label="recomputed: normal approx. with continuity correction")
ax.axhline(P_KO, color=ORANGE, lw=LW, ls=(0, (1, 2)),
label=f"reported p = {P_KO} (Fig. 6D, called significant)")
ax.text(4.6, P_KO + 0.002, f"reported {P_KO}", ha="left", va="bottom", fontsize=10)
ax.set_xticks(us)
ax.set_xlabel("Mann-Whitney U statistic (smaller U = larger group difference)")
ax.set_ylabel("two-sided p value")
ax.set_ylim(0, 0.13)
ax.set_xlim(4.5, 14.8)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.legend(loc="upper left", frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
# ----------------------------------------------------------------- design E: ledger of the two comparisons
def design_E(path):
fig, ax = plt.subplots(figsize=(7.2, 2.8))
ax.set_xlim(0, 1)
ax.set_ylim(-0.12, 1)
ax.axis("off")
cols = [0.01, 0.30, 0.47, 0.71, 0.89]
heads = ["comparison", "printed p\n(reported)", "p < 0.05?\n(recomputed)",
"legend\nwording", "consistent?"]
for x, h in zip(cols, heads):
ax.text(x, 0.90, h, fontsize=10, color=INK2, va="top", fontweight="bold")
ax.plot([0.01, 0.99], [0.66, 0.66], color=GRID, lw=1.2)
rows = [("WT vs TRPV4KO\n(n = 7 or 8 eyes)", P_KO, 0.47),
("WT vs TRPV4CKO\n(n = 6 eyes)", P_CKO, 0.17)]
for lab, p, y in rows:
ok = p < ALPHA
ax.text(cols[0], y, lab, fontsize=10.5, va="center")
ax.text(cols[1], y, f"{p}", fontsize=11, va="center", color=INK)
ax.plot(cols[1] - 0.018, y, "o", ms=MS, color=ORANGE)
ax.text(cols[2], y, "yes" if ok else "no (> 0.05)", fontsize=10.5, va="center")
ax.plot(cols[2] - 0.018, y, "s", ms=MS, color=BLUE, mfc=BG, mew=LW)
ax.text(cols[3], y, "\"significantly\nless\"", fontsize=10.5, va="center")
ax.text(cols[4], y, "yes" if ok else "NO", fontsize=11, va="center",
fontweight="normal" if ok else "bold")
ax.plot([0.01, 0.99], [0.32, 0.32], color=GRID, lw=0.8)
ax.text(0.01, -0.08, "orange dot: printed in the paper; blue square: recomputed check", fontsize=10, color=INK2, va="bottom")
save(fig, path)
print("\n5. Figures")
design_A(os.path.join(HERE, "fig1"))
design_B(os.path.join(HERE, "fig2"))
if "--candidates" in sys.argv:
cd = os.path.join(HERE, "candidates")
os.makedirs(cd, exist_ok=True)
design_A(os.path.join(cd, "A_pvalue_axis"))
design_B(os.path.join(cd, "B_all_pvalues"))
design_C(os.path.join(cd, "C_text_panels"))
design_D(os.path.join(cd, "D_exact_ladder"))
design_E(os.path.join(cd, "E_ledger"))
Reported renal-cancer P=0.004 is P=0.24 under the paper's own stated Fisher test
JAMA (2010) · doi:10.1001/jama.2010.1877 · PMC3655326
Summary
The paper compares the prevalence of cancers in Cowden or Cowden-like (CS/CSL) patients with germline KILLIN promoter methylation against patients with germline PTEN mutations. The Abstract and the Results both report renal cell carcinoma in 4/45 methylation-positive patients and 6/155 PTEN-mutation-positive patients, with P=0.004. The Methods say these prevalences were compared with a two-tailed Fisher exact test. On those counts that test gives P=0.237 (odds ratio 2.42; exact 95% CI 0.48–10.72), and none of the other standard tests for a 2×2 table gives a value below P=0.09. The renal association is one of the two cancer risks named in the abstract Conclusions, and the Comment bases advice on kidney surveillance and test prioritization on it. The paper's primary finding (methylation in 45/123 patients) and its breast-cancer comparison are not affected.
What the paper reports
Abstract, Results:
Individuals with KILLIN-promoter methylation had 3-fold increased prevalence of breast cancer (35/42 vs 24/64, P<0.0001) and a >2-fold increase of kidney cancer (4/45 vs 6/155, P=0.004) over those with germline PTEN mutations.
Abstract, Conclusions:
Germline KILLIN- methylation is common among CS/CSL patients, and associated with increased risks of breast and renal cancer over PTEN mutation-positive patients. These observations need to be replicated.
Results, paragraph "Prevalent Cancers in Pathogenic PTEN Mutation Positive versus KILLIN Methylation":
Renal cell carcinoma was over-represented in the methylation positive subjects over PTEN mutation positive individuals (4/45 vs 6/155, P=0.004).
Methods, "Statistical Analysis":
The prevalence of component malignancies between KILLIN promoter methylation positive patients and germline pathogenic PTEN mutation positive patients was compared using the Fisher’s 2-tailed exact test with P<0.05 considered to be significant.
Comment, second paragraph:
Patients with CS-like features, especially where breast cancer and/or renal carcinomas are present in the individual and/or family, should be offered KILLIN methylation analysis first
Comment, third paragraph:
In contrast to germline PTEN mutations, germline methylation of the KILLIN promoter confers a significantly higher prevalence of female invasive breast cancer and renal cell carcinomas. The current national practice guidelines for those with PTEN germline mutations includes heightened surveillance of the female breasts and thyroid, but do not have awareness of renal cancer risk. If our observations of two- to three-fold increased risks of renal and/or breast cancer with KILLIN germline methylation over those of PTEN mutation holds, then extra vigilance for the organs-at-risk, breast and kidneys, is warranted.
Comment, "Conclusions":
Among CS/CSL patients, presence of germline KILLIN gene promoter hypermethylation was common and was associated with increased risk of breast and renal cancer compared with PTEN mutation-positive patients.
Every quote was checked verbatim against the article as archived in PubMed Central (the author manuscript). The publisher's version of record gives the same counts and P value in JAMA house style ("4/45 vs 6/155; P = .004").
What the paper's own numbers show
Method. The reported counts give the 2×2 table [[4, 41], [6, 149]]: renal cell carcinoma present or absent among 45 methylation-positive and 155 PTEN-mutation-positive patients. I applied the test the Methods name (two-tailed Fisher exact), then the other common tests for this table, to see whether any of them gives 0.004. I also worked out which counts would be needed to reach P=0.004. As a control, I recomputed the breast-cancer comparison from the same sentence and the methylation prevalence the same way. Every value below is printed by recompute.py.txt (scipy 1.18.1). The paper has no tables holding these counts; they appear only in the text.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Renal prevalence, methylated vs PTEN | Abstract Results; Results | 4/45 vs 6/155, ">2-fold" | 8.9% vs 3.9%; prevalence ratio 2.30 (consistent) |
| Expected renal cases among the 45 if prevalence were equal | derived | — | 2.25 (observed 4; 10 cases among 200 patients) |
| Fisher exact, two-tailed (the test the Methods name) | Abstract Results; Results | P=0.004 | P=0.237 |
| Fisher exact, one-tailed | same counts | — | P=0.164 |
| Fisher one-tailed mid-p | same counts | — | P=0.106 |
| Pearson chi-square, uncorrected / Yates-corrected | same counts | — | P=0.174 / 0.331 |
| Barnard / Boschloo exact, two-sided | same counts | — | P=0.212 / 0.208 |
| Exact binomial, 4/45 against a fixed rate of 6/155 (one-sided) | same counts | — | P=0.096 |
| Odds ratio (sample), Wald 95% CI, Wald P | same counts | not reported | 2.42 (0.65–8.99), P=0.186 |
| Odds ratio (conditional MLE), exact 95% CI | same counts | not reported | 2.41 (0.48–10.72) |
| Renal cases among the 45 needed for two-tailed Fisher P≤0.004 (PTEN group held at 6/155) | sensitivity | 4 | 8 (P=0.0038) |
| Renal cases among the 45 needed for P<0.05 (PTEN group held at 6/155) | sensitivity | 4 | 6 (P=0.029) |
| PTEN-group renal cases needed for P≤0.004 (methylated group held at 4/45) | sensitivity | 6/155 | 0/155 (P=0.0023); 1/155 gives P=0.0096 |
| Breast cancer, 35/42 vs 24/64 (control check) | Abstract Results; Results | P<0.0001 | Fisher two-tailed P=3.1×10⁻⁶ (consistent) |
| Methylation prevalence (control check) | Abstract Results | 45/123, 37% (95% CI 29–45%) | 36.6%; Wald 28.1–45.1%, exact 28.1–45.7% (consistent) |
The reported P value is about 1/59 of the value the stated test gives. The ">2-fold" prevalence ratio is correct. Only the P value, and the claim of statistical significance that rests on it, do not follow from the counts.
A smaller point, which does not change any conclusion: the Abstract calls the breast-cancer difference a "3-fold increased prevalence", but 35/42 vs 24/64 is 83.3% vs 37.5%, a prevalence ratio of 2.22 (odds ratio 8.33).
Figure
Figure 1. P values for renal cell carcinoma in 4/45 KILLIN-methylation-positive vs 6/155 PTEN-mutation-positive patients, on one logarithmic axis. The orange point is the value reported in the Abstract and Results (P=0.004). Blue points are recomputed from the same counts; the larger blue point is the two-tailed Fisher exact test named in the Methods (P=0.237), joined to the reported value by a dotted line. The dashed line marks the paper's significance threshold, P=0.05. Every recomputed value lies to the right of the threshold, between 0.096 and 0.331, so the conclusion does not depend on the choice of test.
Figure 2. Two-tailed Fisher exact P (blue, log scale) for x/45 methylation-positive patients with renal cancer against the reported 6/155 in the PTEN-mutation group, for x from 0 to 12. At the reported count of 4/45 the test gives P=0.237. The reported P=0.004 (orange dashed line) is first reached at 8/45 (P=0.0038), twice the reported count. A count of 6/45 would be needed to cross P=0.05.
Why it matters
- It changes significance. With the stated test, the renal comparison is far from the paper's own threshold of P<0.05 (P=0.24). The exact 95% CI for the odds ratio, 0.48–10.72, includes 1. With only 10 renal cancers among 200 patients, the data are compatible both with no difference and with a large one.
- It is a headline result. "Prevalence of component cancers in methylation-positive and PTEN mutation-positive patients" is a listed Main Outcome Measure. The renal association appears in the Abstract Results, the Abstract Conclusions ("associated with increased risks of breast and renal cancer") and the closing Conclusions.
- Clinical advice depends on it. The Comment says that current PTEN guidelines "do not have awareness of renal cancer risk", that "extra vigilance for the organs-at-risk, breast and kidneys, is warranted", and that patients with renal carcinomas in the individual or family should be offered KILLIN methylation analysis first. With the P value corrected, the paper's data no longer give statistically significant support for the kidney part of this advice.
- What is not affected. The primary finding (germline methylation in 45/123 PTEN-mutation-negative CS/CSL patients), the expression experiments, and the breast-cancer association (recomputed P=3.1×10⁻⁶) do not depend on this number. The paper itself calls the observations in need of replication and notes that the small sample "may result in type II error".
Explanations considered
- A different test was used. This does not account for the value. The Methods name the test exactly ("Fisher’s 2-tailed exact test"). I also tried one-tailed Fisher, mid-p, chi-square with and without continuity correction, Barnard, Boschloo, a binomial test against a fixed reference rate, and a Wald test on the odds ratio. The smallest result, 0.096, is 24 times larger than 0.004 (Figure 1).
- A slip between sections, or in conversion to PMC. This does not account for the value. The counts and the P value are identical in the Abstract and the Results, and the publisher's version of record matches the PMC archive. The values are plain text in prose, so no table cell, minus sign or decimal-point conversion is involved. A typographical error in the P value itself remains possible, but no standard analysis of the stated counts gives a value that rounds to 0.004.
- The counts, rather than the P value, are in error. Possible, but the paper gives no basis for it. The denominator 45 matches the number of methylation-positive patients reported elsewhere (45 of 123). With the PTEN group fixed at 6/155, reaching P≤0.004 requires 8/45 renal cancers. With the methylated group fixed at 4/45, it requires 0/155 in the PTEN group (P=0.0023). Keeping 6 PTEN-group cases, the group would have to be several times larger (6/500 gives P=0.0059; 6/1000 gives P=0.0005). The paper reports none of these figures. The 155-patient PTEN-mutation group is not described further in the text, whereas the breast comparison uses 64 PTEN-mutation-positive women.
- Other denominators in the paper. These do not account for the value. Substituting the breast-comparison denominators (4/42 vs 6/64), or 4/45 vs 6/64, gives P=1.0; 4/42 vs 6/155 gives P=0.225.
- Adjustment, stratification or person-time analysis. None is described. The paper reports crude prevalences, and the Methods name only the Fisher test for this comparison.
- The supplement holds a different analysis. Not verified. PMC lists one supplementary file. According to the text it holds the CS diagnostic criteria, primer sequences and a per-patient list of the methylation-positive individuals (eTable 1). It could not be retrieved (see Status). Nothing in the text says it holds a different renal analysis. eTable 1 might confirm or change the count of 4 renal cancers among the 45, but no table can make 4/45 vs 6/155 give P=0.004 under the stated test.
- Rounding. P=0.237 cannot round to 0.004.
None of these explanations accounts for the reported value.
Status
As of 2026-10-23:
- Version of record. The publisher's page (jamanetwork.com/journals/jama/fullarticle/10.1001/jama.2010.1877; JAMA 2010;304(24):2724) shows the same counts and "P = .004" in both the Abstract Results and the Results, and the same Methods wording on the two-tailed Fisher test. The PubMed abstract (PMID 21177507) and the PMC author manuscript (NIHMS467022) agree.
- Supplement. One file is listed in PMC ("KLB Supp_data_JAMA_revised.doc"). It could not be retrieved: the PMC download returned a proof-of-work/captcha page, Europe PMC reports the article as outside the open-access subset, and the JAMA supplemental-content tab showed no file link to an automated fetch. Its contents as described in the text are listed above.
- Corrections and notices. None found. The Crossref
updatesfilter for this DOI returns 0 records (re-queried 2026-10-23), and the work record has noupdate-toorupdated-by. PMC and Europe PMC list no correction or retraction notice (src/known.py). PubMed links only an accompanying editorial (JAMA 2010;304(24):2744-5) as "Comment in", and no erratum. The JAMA article page shows no correction notice. - PubPeer and other discussion. Earlier web searches found no PubPeer thread and no published comment on this point. The PubPeer API could not be queried directly (Cloudflare block, again on 2026-10-23), and a fresh web search on 2026-10-23 could not be run because the session's search budget was exhausted. Press coverage at the time repeated the >2-fold kidney-cancer finding.
- Sources. PMC3655326 (local archive via
src/show.py); the publisher's version of record; Crossref API; PubMed E-utilities; Europe PMC; web searches for correction, erratum and PubPeer. Recomputation:recompute.py.txtin this folder.
Suggested correction
In the Abstract Results and in the Results paragraph on prevalent cancers, replace "P=0.004" for renal cell carcinoma (4/45 vs 6/155) with the two-tailed Fisher exact value, P=0.24, and preferably give the odds ratio with its exact 95% CI (2.41, 0.48–10.72). Then revise the statements that rest on this result: the abstract Conclusions, the Comment ("confers a significantly higher prevalence of ... renal cell carcinomas", and the kidney-surveillance and testing-priority statements) and the closing Conclusions. They could say that renal cancer was more common in the methylation-positive group (8.9% vs 3.9%) but that the difference was not statistically significant in this sample. If the counts, rather than the P value, are in error, the correction should give the correct counts and describe the 155-patient PTEN-mutation comparison group. Separately, the breast-cancer difference described as "3-fold" corresponds to a prevalence ratio of 2.2 (83.3% vs 37.5%).
Recomputation script (python recompute.py.txt)
"""Recompute the renal-cancer comparison of PMC3655326
(JAMA 2010;304(24):2724-2731, doi:10.1001/jama.2010.1877).
Run: python recompute.py.txt
Needs numpy, scipy (>= 1.10 for odds_ratio / barnard / boschloo) and matplotlib.
Prints every recomputed number used in writeup.md and writes fig1.svg/png and
fig2.svg/png next to this file.
Inputs are copied from the paper's Abstract and Results ("Prevalent Cancers in
Pathogenic PTEN Mutation Positive versus KILLIN Methylation"). The paper has no
tables holding these counts; they appear only in the text.
Renal cell carcinoma: 4/45 KILLIN-methylated vs 6/155 PTEN-mutation-positive, reported P=0.004
Female invasive breast cancer: 35/42 vs 24/64, reported P<0.0001
Methylation prevalence: 45/123, reported 37% (95% CI 29-45%)
Methods: "compared using the Fisher's 2-tailed exact test with P<0.05 considered to be significant"
"""
import os
import numpy as np
from scipy import stats
from scipy.stats.contingency import odds_ratio
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs (from the paper)
RENAL = dict(a=4, n1=45, c=6, n2=155, reported_p=0.004)
BREAST = dict(a=35, n1=42, c=24, n2=64)
METH = dict(k=45, n=123)
ALPHA = 0.05
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, TXT, TXT2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
def table(a, n1, c, n2):
return [[a, n1 - a], [c, n2 - c]]
def fisher2(a, n1, c, n2):
return stats.fisher_exact(table(a, n1, c, n2), alternative="two-sided")[1]
def main():
r = RENAL
t = table(r["a"], r["n1"], r["c"], r["n2"])
p1, p2 = r["a"] / r["n1"], r["c"] / r["n2"]
print("=== Renal cell carcinoma: 4/45 (KILLIN-methylated) vs 6/155 (PTEN mutation+) ===")
print(f"prevalence {100*p1:.1f}% vs {100*p2:.1f}%; prevalence ratio {p1/p2:.2f}")
sample_or = stats.fisher_exact(t)[0]
cond = odds_ratio(t, kind="conditional")
ci = cond.confidence_interval(0.95)
print(f"sample odds ratio {sample_or:.2f}; conditional MLE OR {cond.statistic:.2f}, "
f"exact 95% CI {ci.low:.2f}-{ci.high:.2f}")
(a_, b_), (c_, d_) = t
se_log = np.sqrt(1 / a_ + 1 / b_ + 1 / c_ + 1 / d_)
lo, hi = np.exp(np.log(sample_or) + np.array([-1, 1]) * 1.959964 * se_log)
p_wald = 2 * stats.norm.sf(abs(np.log(sample_or)) / se_log)
print(f"sample OR Wald 95% CI {lo:.2f}-{hi:.2f}, Wald P {p_wald:.3f}")
chi = stats.chi2_contingency(t, correction=False)
chiy = stats.chi2_contingency(t, correction=True)
print(f"total events {r['a'] + r['c']} of {r['n1'] + r['n2']}; "
f"expected in methylated group under no difference {chi.expected_freq[0][0]:.2f} (observed {r['a']})")
results = {}
results["Fisher exact, two-sided\n(the test named in Methods)"] = fisher2(r["a"], r["n1"], r["c"], r["n2"])
results["Fisher exact, one-sided"] = stats.fisher_exact(t, alternative="greater")[1]
pmf = [stats.hypergeom.pmf(k, 200, 10, 45) for k in range(11)]
results["Fisher one-sided mid-p"] = sum(pmf[5:]) + 0.5 * pmf[4]
results["Pearson chi-square"] = chi.pvalue
results["Yates-corrected chi-square"] = chiy.pvalue
results["Barnard exact, two-sided"] = stats.barnard_exact(t).pvalue
results["Boschloo exact, two-sided"] = stats.boschloo_exact(t).pvalue
results["Binomial 4/45 vs fixed rate 6/155,\none-sided"] = stats.binomtest(4, 45, 6 / 155, alternative="greater").pvalue
for k, v in results.items():
print(f" {k.replace(chr(10), ' '):55s} P = {v:.4f}")
p_fisher = results["Fisher exact, two-sided\n(the test named in Methods)"]
print(f"reported P=0.004; recomputed two-sided Fisher P={p_fisher:.3f}, "
f"ratio {p_fisher / r['reported_p']:.0f}x")
print(f"smallest recomputed P across all tests: {min(results.values()):.4f}")
print("\n=== Counts that would give two-sided Fisher P near 0.004 ===")
sweep_a = {a: fisher2(a, 45, 6, 155) for a in range(0, 13)}
for a, p in sweep_a.items():
print(f" {a:2d}/45 vs 6/155: P = {p:.5f}")
need = min(a for a, p in sweep_a.items() if p <= r["reported_p"] and a > 4)
print(f"smallest methylated count (of 45) with P <= 0.004, holding 6/155: {need} (P = {sweep_a[need]:.4f})")
need05 = min(a for a, p in sweep_a.items() if p < ALPHA and a > 4)
print(f"smallest methylated count (of 45) with P < 0.05, holding 6/155: {need05} (P = {sweep_a[need05]:.4f})")
sweep_c = {c: fisher2(4, 45, c, 155) for c in range(0, 7)}
for c, p in sweep_c.items():
print(f" 4/45 vs {c}/155: P = {p:.5f}")
print(" other denominators (from the paper, then hypothetical larger PTEN groups):")
for n1, n2 in [(42, 64), (45, 64), (42, 155), (45, 500), (45, 1000)]:
print(f" 4/{n1} vs 6/{n2}: P = {fisher2(4, n1, 6, n2):.4f}")
print(f"scipy {__import__('scipy').__version__}")
print("\n=== Breast cancer 35/42 vs 24/64 (reported P<0.0001, '3-fold') ===")
b = BREAST
bt = table(b["a"], b["n1"], b["c"], b["n2"])
bp1, bp2 = b["a"] / b["n1"], b["c"] / b["n2"]
print(f"prevalence {100*bp1:.1f}% vs {100*bp2:.1f}%; prevalence ratio {bp1/bp2:.2f}; "
f"sample odds ratio {stats.fisher_exact(bt)[0]:.2f}")
print(f"two-sided Fisher P = {fisher2(**{k: b[k] for k in ('a','n1','c','n2')}):.1e}")
print("\n=== Methylation prevalence 45/123 (reported 37%, 95% CI 29-45%) ===")
ph = METH["k"] / METH["n"]
se = np.sqrt(ph * (1 - ph) / METH["n"])
cp = stats.binomtest(METH["k"], METH["n"]).proportion_ci(method="exact")
print(f"{100*ph:.1f}%; Wald 95% CI {100*(ph-1.96*se):.1f}-{100*(ph+1.96*se):.1f}%; "
f"Clopper-Pearson {100*cp.low:.1f}-{100*cp.high:.1f}%")
fig1(results, r["reported_p"])
fig2(sweep_a, r["reported_p"])
def style(ax):
ax.set_facecolor(BG)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(TXT2)
ax.tick_params(colors=TXT2, labelsize=10)
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
ax.set_axisbelow(True)
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
def fig1(results, reported):
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans"})
labels = ["Reported in the paper\n(Abstract and Results)"] + list(results.keys())
vals = [reported] + list(results.values())
y = np.arange(len(labels))[::-1]
fig, ax = plt.subplots(figsize=(7.2, 5.0), facecolor=BG)
style(ax)
ax.set_xscale("log")
ax.set_xlim(0.001, 1.0)
ax.axvline(ALPHA, color=TXT2, lw=1, ls=(0, (4, 3)))
ax.text(ALPHA * 1.08, y[0] + 0.55, "P = 0.05\n(paper's threshold)", color=TXT2, fontsize=10, va="center")
# gap between reported and recomputed Fisher value
ax.plot([vals[0], vals[1]], [y[0], y[1]], color=TXT2, lw=1, ls=":")
for i, (lab, v) in enumerate(zip(labels, vals)):
col = ORANGE if i == 0 else BLUE
ax.plot(v, y[i], "o", ms=10 if i < 2 else 8, color=col, zorder=3)
ax.text(v * (0.82 if i == 0 else 1.15), y[i], f"{v:.3f}", color=col, fontsize=10,
va="center", ha="right" if i == 0 else "left", fontweight="bold" if i < 2 else "normal")
ax.set_yticks(y)
ax.set_yticklabels(labels, fontsize=10, color=TXT)
ax.tick_params(axis="y", length=0)
ax.set_xticks([0.001, 0.004, 0.01, 0.05, 0.1, 0.25, 1.0])
ax.set_xticklabels(["0.001", "0.004", "0.01", "0.05", "0.1", "0.25", "1"])
ax.set_xlabel("P value for renal cancer, 4/45 vs 6/155 (log scale)", color=TXT, fontsize=10)
ax.set_ylim(y[-1] - 0.7, y[0] + 1.1)
h1, = ax.plot([], [], "o", ms=9, color=ORANGE, label="Reported by the paper")
h2, = ax.plot([], [], "o", ms=9, color=BLUE, label="Recomputed from the paper's counts")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.3, -0.16), ncol=2,
frameon=False, fontsize=10, labelcolor=TXT)
fig.tight_layout()
save(fig, "fig1")
def fig2(sweep, reported):
a = np.array(list(sweep.keys()))
p = np.array(list(sweep.values()))
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
style(ax)
ax.grid(axis="y", which="major", color=GRID, lw=0.8)
ax.set_yscale("log")
ax.plot(a, p, "-", color=BLUE, lw=2, zorder=2)
ax.plot(a, p, "o", ms=8, color=BLUE, zorder=3, label="Two-sided Fisher P, recomputed (6/155 held fixed)")
ax.axhline(reported, color=ORANGE, lw=2, ls="--", zorder=1)
ax.text(12.3, reported * 1.25, "Reported P = 0.004", color=ORANGE, ha="right", va="bottom", fontsize=10)
ax.axhline(ALPHA, color=TXT2, lw=1, ls=(0, (4, 3)))
ax.text(12.3, ALPHA * 1.15, "P = 0.05", color=TXT2, ha="right", va="bottom", fontsize=10)
ax.plot(4, sweep[4], "o", ms=13, mfc="none", mec=TXT, mew=1.5, zorder=4)
ax.annotate(f"Reported count 4/45\nP = {sweep[4]:.3f}", (4, sweep[4]), xytext=(5.6, 0.55),
fontsize=10, color=TXT, arrowprops=dict(arrowstyle="-", color=TXT2, lw=1))
ax.plot(8, sweep[8], "o", ms=13, mfc="none", mec=TXT, mew=1.5, zorder=4)
ax.annotate(f"8/45 would be needed\nP = {sweep[8]:.4f}", (8, sweep[8]), xytext=(8.6, 0.012),
fontsize=10, color=TXT, arrowprops=dict(arrowstyle="-", color=TXT2, lw=1))
ax.set_xticks(a)
ax.set_xlim(-0.5, 12.5)
ax.set_ylim(1e-5, 1.6)
ax.set_xlabel("Renal cancers among the 45 KILLIN-methylated patients (comparison group fixed at 6/155)",
color=TXT, fontsize=10)
ax.set_ylabel("Two-sided Fisher P (log scale)", color=TXT, fontsize=10)
ax.legend(loc="lower left", frameon=False, fontsize=10, labelcolor=TXT)
fig.tight_layout()
save(fig, "fig2")
if __name__ == "__main__":
main()
Key Points and abstract call the JAKi cancer risk significant though the paper's 95% CI includes 1
JAMA Netw Open (2024) · doi:10.1001/jamanetworkopen.2024.46336 · PMC11579790
Summary
The Key Points and the abstract's Conclusions and Relevance say that cancer risk was "statistically significantly" higher in people starting Janus kinase inhibitors (JAKis) than in people starting tumor necrosis factor inhibitors (TNFis). The paper's own estimate for that comparison is an adjusted hazard ratio (HR) of 1.36 with a 95% CI of 0.94-1.96. The interval includes 1. Working back from it gives a two-sided P of about 0.10, above the paper's stated threshold of 2-sided P < .05. The propensity-matched estimate (HR 1.79, 0.91-3.56; P about 0.09; shown as 1.8, 0.9-3.6 in eTable 10 of Supplement 1) and the crude rate ratio from Table 2 (1.21, 0.77-1.88) are not significant either, and the paper's Results, Discussion and main-text Conclusions all describe the JAKi result as not significant. The HR, its CI and its direction are reported correctly throughout. What is wrong is the significance label in the two most-read summaries, on the comparison of most safety interest (the JAKi cancer signal from the ORAL Surveillance trial).
What the paper reports
All passages are verbatim from the article as archived in PubMed Central.
The two summary statements:
"In this cohort study consisting of 25 305 individuals who initiated treatment with biologic or targeted synthetic disease-modifying antirheumatic drugs, a statistically significantly higher risk of incident cancer was seen with rituximab, abatacept, and JAKis compared with TNFis." (Key Points, Findings)
"In this cohort study of individuals with RA and new biologic or targeted synthetic DMARD exposures, individuals initiating rituximab, abatacept, and JAKis demonstrated higher incidence rates and statistically significantly increased risks of incident cancers compared with those initiating TNFis in the first 2 years after initiation of biologic or targeted synthetic DMARDs." (Abstract, Conclusions and Relevance)
The estimate those statements rest on, which appears in the abstract Results, the main Results and Table 2:
"Multivariable Cox proportional hazards regression analysis showed that rituximab was associated with a higher risk of incident cancer compared with TNFis (hazard ratio [HR], 1.91; 95% CI, 1.17-3.14), followed by abatacept (HR, 1.47; 95% CI, 1.03-2.11), and JAKis (HR, 1.36; 95% CI, 0.94-1.96)." (Abstract, Results)
"JAKi | 2570 (9) | 276 (169-351) | 22 (0.9) | 94 (59-143) | 1.36 (0.94-1.96)" (Table 2, JAKi row: exposures, median days to diagnosis, crude cancer diagnoses, diagnoses per 10 000 person-years, adjusted HR)
The significance convention:
"A 2-sided P < .05 was considered statistically significant." (Methods, Statistical Analysis)
The body of the paper describes the same result as not significant:
"Also like the primary analysis, we found that those treated with JAKis did not have a statistically significant increase in risk of cancer (HR, 1.79; 95% CI, 0.91-3.56) (eTable 10 and eFigure 4 in Supplement 1)." (Results, propensity-matched analysis paragraph)
"We observed that, among patients with RA, rituximab, abatacept, and JAKi initiations were associated with increased risks of incident cancer after 90 days and within 2 years of initiation, compared with TNFi initiation, although only rituximab and abatacept initiations had statistically significant associations." (Discussion, first paragraph)
"Our findings of increased incidence rate and risk of cancer with JAKi exposure compared with TNFi exposure are consistent with the results of ORAL Surveillance,6 although the results were not statistically significant." (Discussion, paragraph on prior JAKi studies)
"In this cohort study of individuals with RA and new biologic or targeted synthetic DMARD exposures, we found that new rituximab and abatacept initiators demonstrated higher incidence rates and statistically significantly increased risks of incident cancers compared with TNFi initiators in the first 2 years after biologic or targeted synthetic DMARD initiation." (Conclusions, main text)
What the paper's own numbers show
Method. Cox-model HRs come with Wald intervals on the log scale, so a reported HR and 95% CI fix the standard error: SE = [ln(upper) − ln(lower)] / (2 × 1.960). Then z = ln(HR) / SE and the two-sided P = 2 × [1 − Φ(|z|)]. This uses only the published numbers; the result is approximate because the bounds are rounded to two decimals. Before relying on it, I checked that the JAKi interval is symmetric about ln(1.36) on the log scale (0.369 below, 0.365 above), as a Wald interval should be. I applied the same calculation to every HR the paper reports, so the comparisons the paper correctly calls significant serve as a check. Separately, I recomputed the crude rates in Table 2 from their event counts. The person-years each rate implies give exact Poisson intervals that match the reported ones to within 1-2 units, so the inputs are internally consistent. From those I formed a crude JAKi/TNFi rate ratio and an exact conditional test. I did the same for the propensity-matched rates in eTable 10 of Supplement 1. All numbers come from recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| JAKi vs TNFi adjusted HR (95% CI) | Abstract Results; Results; Table 2 | 1.36 (0.94-1.96) | CI includes 1; geometric midpoint of the bounds 1.36 (consistent) |
| JAKi SE(ln HR), z | derived from Table 2 | not reported | SE 0.187; z 1.64 (z ≥ 1.96 needed for P < .05) |
| JAKi two-sided P | derived from Table 2 | not reported; Key Points Findings and abstract Conclusions and Relevance say "statistically significantly" | 0.10 (not < .05) |
| Smallest HR that would be significant at this SE | derived | n/a | 1.44 |
| JAKi one-sided P (not the stated convention) | derived | Methods: "2-sided P < .05" | 0.050 |
| JAKi propensity-matched HR (95% CI) | Results | 1.79 (0.91-3.56), "did not have a statistically significant increase" | P 0.094 (not significant; agrees with the text) |
| JAKi propensity-matched HR (95% CI) | Supplement 1, eTable 10 | 1.8 (0.9-3.6); 22 vs 14 cancers | same as the Results value rounded to one decimal; P about 0.10 from the rounded bounds |
| Propensity-matched crude rates per 10 000 PY, JAKi vs TNFi | Supplement 1, eTable 10 | 100 (63-151), 22 events; 60 (33-101), 14 events | imply about 2200 and 2333 PY; exact Poisson CIs 63-151 and 33-101 (exact match); rate ratio 1.67 (0.85-3.26), exact conditional P 0.14 |
| JAKi crude rate per 10 000 PY | Table 2 | 94 (59-143), 22 events | implies about 2340 PY; exact Poisson CI 59-142 |
| TNFi crude rate per 10 000 PY | Table 2 | 78 (66-91), 162 events | implies about 20 769 PY; exact Poisson CI 66-91 |
| Crude rate ratio JAKi/TNFi | derived from Table 2 | n/a | 1.21 (0.77-1.88); exact conditional P 0.39 |
| Rituximab vs TNFi HR | Table 2 | 1.91 (1.17-3.14) | P 0.010 (significant; agrees with the text) |
| Abatacept vs TNFi HR | Table 2 | 1.47 (1.03-2.11) | P 0.035 (significant; agrees with the text) |
| IL-6i vs TNFi HR | Table 2 | 1.04 (0.57-1.92) | P 0.90 |
| Propensity-matched rituximab, abatacept, IL-6i HRs | Results | 4.37 (1.48-12.93); 3.12 (1.52-6.44); 5.65 (1.11-28.79), all called significant | P 0.008; 0.002; 0.037 (agree with the text) |
Every HR the paper calls significant has a back-calculated P below .05 and a CI that excludes 1. The JAKi primary estimate is the only one the Key Points and abstract call significant that does not meet either criterion. No JAKi analysis reported anywhere in the paper or in Supplement 1 does.
Figure
Figure 1. The left panel (Key Points and Abstract) quotes the Key Points Findings and the abstract's Conclusions and Relevance, which group JAKis with rituximab and abatacept as having a "statistically significantly" increased cancer risk. The right panel gives the estimate those statements rest on (Table 2 and Results) and the body's own description of it: the Discussion says only rituximab and abatacept had significant associations, and the main-text Conclusions name only those two drugs. The strip below gives the reported HR and CI (orange) and the z and two-sided P recomputed from that CI (blue). All quotes are verbatim from the PMC version; the recomputed values come from recompute.py.txt.
Figure 2. Every adjusted HR versus TNFi that the paper reports, primary (Table 2) and propensity-matched (Results), drawn as the reported point estimate and 95% CI (orange) on one log-scale axis. The thin vertical line marks HR = 1. The right-hand columns give the reported values and the two-sided P recomputed from each CI (blue). Both JAKi rows (shaded) cross 1, with P = 0.101 and 0.094; the label on the primary JAKi row marks the estimate that the Key Points and abstract call "statistically significantly" higher. Every row the paper calls significant has P < .05.
Why it matters
The JAKi versus TNFi comparison is the question that motivates the study: the Importance section opens with the increased cancer risk with tofacitinib versus TNFis in ORAL Surveillance. The Key Points Findings sentence is the paper's headline result and is what many readers, abstracting services and news summaries carry forward. As written, it tells a reader that this US claims cohort found a statistically significant JAKi cancer signal. It did not. The point estimate (HR 1.36) is compatible with anything from a 6% lower hazard to a 96% higher hazard, and the paper's other JAKi analyses agree. A reader who takes only the Key Points or the abstract would count this study as independent statistical support for the ORAL Surveillance finding, when the paper's own Discussion calls the JAKi result "not statistically significant".
The rest of the paper is not affected. The HR, its CI and the direction of the JAKi estimate are reported correctly, the rituximab and abatacept results are significant as stated, and the authors' interpretation (channeling bias as a likely explanation) does not depend on the JAKi label. The error is confined to two sentences (one in the Key Points, one in the abstract), which makes it easy to correct.
Explanations considered
- Another JAKi analysis is being described. Not supported. The propensity-matched HR, 1.79 (0.91-3.56), has P about 0.09, and the Results call it not significant. Supplement 1, eTable 10, gives the same matched result as 1.8 (0.9-3.6), from 22 vs 14 cancers; the matched crude rate ratio it implies, 1.67 (0.85-3.26), has exact P 0.14. The crude rate ratio implied by Table 2, 1.21 (0.77-1.88), has P 0.39. No JAKi estimate anywhere in the paper or Supplement 1 has a CI that excludes 1.
- A one-sided test or a looser alpha. Not supported. The Methods state "A 2-sided P < .05 was considered statistically significant." The back-calculated two-sided P is 0.10. Only a one-sided test brings it to 0.050, which is still not below .05.
- "Significant" refers to the higher incidence rates, not the HR. Not supported. The wording is "statistically significantly higher risk" and "statistically significantly increased risks". The JAKi crude rate, 94 (59-143), overlaps the TNFi rate, 78 (66-91), and the crude rate ratio is not significant either.
- Transcription or rounding error in the HR or CI. Not supported. HR 1.36 is the geometric midpoint of 0.94-1.96. The interval is symmetric on the log scale, and the same numbers appear in the abstract, the Results and Table 2. The crude rates match exact Poisson intervals from their event counts to within 1-2 units.
- A conversion artifact in the archived text. Ruled out. Both bounds are positive decimals, so no minus sign can have been lost, and nothing depends on superscripts or table alignment. The conflicting statements are in prose, and the same wording appears on the live PMC page, in the Europe PMC full-text XML and in the PubMed abstract.
- Rounding at the threshold. Not applicable. A lower bound of 0.94 is not a rounded 1.00, and z = 1.64 is far from 1.96. With this SE, an HR of at least 1.44 would have been needed for significance.
None of these accounts for the wording of the Key Points and abstract. Since the body of the paper states the result correctly, those two sentences are inconsistent with the paper itself.
Status
As of 2026-10-23:
- Version of record. The publisher's full-text page (jamanetwork.com, article 2826510) returned an automated-traffic check and could not be read directly. The wording quoted above was confirmed in three publisher-supplied copies: the live PMC page, the Europe PMC full-text JATS XML, and the PubMed abstract (PMID 39565623). The 0.94-1.96 interval appears three times in the XML (abstract, Results, Table 2).
- Supplement. Supplement 1 (eTables 1-11, eFigures 1-4) was downloaded through Europe PMC and opened. eTable 9 gives the matched TNFi vs JAKi cohort (2421 per arm; 14 vs 22 cancer outcomes). eTable 10 (page 14, extractable text) gives the propensity-matched JAKi vs TNFi result: 22 vs 14 cancers, 100 (63-151) vs 60 (33-101) per 10 000 person-years, adjusted HR 1.8 (0.9-3.6), consistent with the HR 1.79 (0.91-3.56) in the Results. The supplement contains no alternative JAKi estimate or P value that would support the wording of the Key Points and abstract.
- Corrections. Crossref, PMC and Europe PMC list no correction, erratum, expression of concern or retraction notice. A web search found no erratum.
- PubPeer. A web search found no PubPeer thread for this article. PubPeer's own search returned a human-verification challenge and could not be queried directly.
- Sources: PMC11579790; PubMed 39565623; doi:10.1001/jamanetworkopen.2024.46336; Europe PMC full-text and supplementary-files endpoints for PMC11579790; Crossref metadata for the DOI.
Suggested correction
Bring the Key Points Findings sentence and the abstract's Conclusions and Relevance sentence into line with the Results, Discussion and main-text Conclusions. For example:
- Key Points, Findings: "…a statistically significantly higher risk of incident cancer was seen with rituximab and abatacept compared with TNFis; the higher risk seen with JAKis was not statistically significant (HR, 1.36; 95% CI, 0.94-1.96)."
- Abstract, Conclusions and Relevance: "…individuals initiating rituximab and abatacept demonstrated higher incidence rates and statistically significantly increased risks of incident cancers compared with those initiating TNFis…; individuals initiating JAKis had a higher incidence rate, but the increase in risk was not statistically significant."
The numerical results need no change.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC11579790 write-up and regenerate its figures.
Paper: "Use of Biologic or Targeted Synthetic Disease-Modifying Antirheumatic Drugs
and Cancer Risk", JAMA Netw Open 2024, doi:10.1001/jamanetworkopen.2024.46336.
All inputs are copied from the article (Table 2 and Results) and from Supplement 1
(eTable 10, page 14, text layer extracted with PyMuPDF). Run:
python recompute.py.txt
Figures are written next to this file: five candidate designs (cand_A ... cand_E,
SVG + PNG at 200 dpi), and fig1/fig2, which are copies of the currently chosen
candidates (see CHOSEN below). The candidates are for choosing a design only;
once a design is chosen, set CHOSEN, re-run, and send only writeup.md,
fig1/fig2 and this script to the journal (not the cand_* files).
"""
import os
import shutil
import numpy as np
from scipy import stats
from scipy.stats import binomtest
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.transforms import blended_transform_factory
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = stats.norm.ppf(0.975)
# ---------------------------------------------------------------------------
# Inputs reported in the paper
# ---------------------------------------------------------------------------
# Adjusted HR vs TNFi (95% CI). Primary: Table 2, Abstract Results, Results.
# Propensity-matched: Results paragraph beginning "In the propensity analysis".
HR = {
("Primary", "Rituximab"): (1.91, 1.17, 3.14),
("Primary", "Abatacept"): (1.47, 1.03, 2.11),
("Primary", "JAKi"): (1.36, 0.94, 1.96),
("Primary", "IL-6i"): (1.04, 0.57, 1.92),
("PS-matched", "Rituximab"): (4.37, 1.48, 12.93),
("PS-matched", "Abatacept"): (3.12, 1.52, 6.44),
("PS-matched", "JAKi"): (1.79, 0.91, 3.56),
("PS-matched", "IL-6i"): (5.65, 1.11, 28.79),
}
# Table 2: crude cancer diagnoses, rate per 10 000 person-years (95% CI)
EVENTS = {
"TNFi": (162, 78, 66, 91),
"Rituximab": (14, 171, 94, 285),
"IL-6i": (8, 88, 38, 173),
"Abatacept": (30, 142, 96, 201),
"JAKi": (22, 94, 59, 143),
}
# Supplement 1, eTable 10 (propensity-matched TNFi vs JAKi, 2421 per arm per
# eTable 9): crude cancer diagnoses, rate per 10 000 PY (95% CI), adjusted HR
# as printed there (rounded to one decimal).
ETAB10 = {
"TNFi": (14, 60, 33, 101),
"JAKi": (22, 100, 63, 151),
}
ETAB10_HR = (1.8, 0.9, 3.6)
def wald_from_ci(hr, lo, hi):
"""Back-calculate SE(log HR), z and P from a log-scale Wald 95% CI."""
se = (np.log(hi) - np.log(lo)) / (2 * Z975)
z = np.log(hr) / se
p2 = 2 * stats.norm.sf(abs(z))
p1 = stats.norm.sf(z)
gmid = np.exp((np.log(lo) + np.log(hi)) / 2)
return se, z, p2, p1, gmid
print("=" * 78)
print("1. Wald back-calculation from each reported HR and 95% CI (log scale)")
print("=" * 78)
RES = {}
for key, (hr, lo, hi) in HR.items():
se, z, p2, p1, gmid = wald_from_ci(hr, lo, hi)
RES[key] = dict(hr=hr, lo=lo, hi=hi, se=se, z=z, p2=p2, p1=p1, gmid=gmid)
print(f"{key[0]:10s} {key[1]:9s} HR {hr:5.2f} ({lo:.2f}-{hi:.2f}) "
f"geo-mid {gmid:5.2f} SE {se:.3f} z {z:5.2f} "
f"P two-sided {p2:.3f} P one-sided {p1:.3f} CI excludes 1: {lo > 1}")
j = RES[("Primary", "JAKi")]
print("\nJAKi primary, symmetry check on the log scale:")
print(f" ln(HR) - ln(lower) = {np.log(1.36) - np.log(0.94):.3f}")
print(f" ln(upper) - ln(HR) = {np.log(1.96) - np.log(1.36):.3f}")
print(f" SE(log HR) = {j['se']:.4f}; z = {j['z']:.3f}; two-sided P = {j['p2']:.3f}")
print(f" |z| needed for two-sided P < .05: {Z975:.3f}")
print(f" Smallest HR significant at this SE: {np.exp(Z975 * j['se']):.2f}")
print("\n" + "=" * 78)
print("2. Table 2 crude rates: implied person-years and exact Poisson 95% CI")
print("=" * 78)
PY = {}
for k, (n, r, lo, hi) in EVENTS.items():
py = n / r * 1e4
PY[k] = py
l = stats.chi2.ppf(0.025, 2 * n) / 2
u = stats.chi2.ppf(0.975, 2 * n + 2) / 2
print(f"{k:10s} events {n:3d} rate {r:3d} implied PY ~{py:6.0f} "
f"exact CI {l / py * 1e4:4.0f}-{u / py * 1e4:4.0f} (reported {lo}-{hi})")
n1, n0 = 22, 162
irr = 94 / 78
se_irr = np.sqrt(1 / n1 + 1 / n0)
irr_lo, irr_hi = irr * np.exp(-Z975 * se_irr), irr * np.exp(Z975 * se_irr)
pfrac = PY["JAKi"] / (PY["JAKi"] + PY["TNFi"])
p_exact = binomtest(n1, n1 + n0, pfrac).pvalue
print(f"\nCrude IRR JAKi/TNFi = 94/78 = {irr:.2f} "
f"(Wald 95% CI {irr_lo:.2f}-{irr_hi:.2f}); "
f"exact conditional (binomial) two-sided P = {p_exact:.2f}")
print("\n" + "=" * 78)
print("3. Supplement 1, eTable 10 (propensity-matched TNFi vs JAKi)")
print("=" * 78)
PY10 = {}
for k, (n, r, lo, hi) in ETAB10.items():
py = n / r * 1e4
PY10[k] = py
l = stats.chi2.ppf(0.025, 2 * n) / 2
u = stats.chi2.ppf(0.975, 2 * n + 2) / 2
print(f"{k:10s} events {n:3d} rate {r:3d} implied PY ~{py:6.0f} "
f"exact CI {l / py * 1e4:4.0f}-{u / py * 1e4:4.0f} (reported {lo}-{hi})")
e10 = wald_from_ci(*ETAB10_HR)
print(f"eTable 10 adjusted HR {ETAB10_HR[0]} ({ETAB10_HR[1]}-{ETAB10_HR[2]}): "
f"SE {e10[0]:.3f}, z {e10[1]:.2f}, two-sided P {e10[2]:.3f} "
f"(bounds rounded to 1 decimal, so approximate)")
pm_ = HR[("PS-matched", "JAKi")]
print(f" Results text gives {pm_[0]} ({pm_[1]}-{pm_[2]}); rounded to 1 decimal: "
f"{pm_[0]:.1f} ({pm_[1]:.1f}-{pm_[2]:.1f}) -> matches eTable 10: "
f"{(round(pm_[0], 1), round(pm_[1], 1), round(pm_[2], 1)) == ETAB10_HR}")
n1m, n0m = ETAB10["JAKi"][0], ETAB10["TNFi"][0]
irr10 = ETAB10["JAKi"][1] / ETAB10["TNFi"][1]
se10 = np.sqrt(1 / n1m + 1 / n0m)
pf10 = PY10["JAKi"] / (PY10["JAKi"] + PY10["TNFi"])
p10 = binomtest(n1m, n1m + n0m, pf10).pvalue
print(f"Crude matched rate ratio 100/60 = {irr10:.2f} (Wald 95% CI "
f"{irr10 * np.exp(-Z975 * se10):.2f}-{irr10 * np.exp(Z975 * se10):.2f}); "
f"exact conditional two-sided P = {p10:.2f}")
# ---------------------------------------------------------------------------
# Figures
# ---------------------------------------------------------------------------
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
LW = 2.0 # 2 px-ish lines
MS = 8.5 # marker size in points (>= 8)
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
def fmt_p(p):
return "<0.001" if p < 0.001 else f"{p:.3f}"
ORDER = ["Rituximab", "Abatacept", "JAKi", "IL-6i"]
# --- Candidate A: forest plot of reported HRs with recomputed P column -------
def cand_A():
rows = [("Primary", d) for d in ORDER] + [("PS-matched", d) for d in ORDER]
fig, ax = plt.subplots(figsize=(7.2, 5.2))
fig.subplots_adjust(left=0.14, right=0.645, top=0.91, bottom=0.11)
tr = blended_transform_factory(ax.transAxes, ax.transData)
ys = []
y = 0
for i, key in enumerate(rows):
if i == 4:
y += 0.8
ys.append(-y)
y += 1
for yy, key in zip(ys, rows):
r = RES[key]
is_j = key[1] == "JAKi"
if is_j:
ax.axhspan(yy - 0.45, yy + 0.45, color="#f3f1ec", zorder=0)
ax.plot([r["lo"], r["hi"]], [yy, yy], color=ORANGE, lw=LW, zorder=2,
solid_capstyle="butt")
ax.plot(r["hr"], yy, "o", color=ORANGE, ms=MS, zorder=3,
markeredgecolor=BG, markeredgewidth=1.2)
ax.text(1.02, yy, f"{r['hr']:.2f} ({r['lo']:.2f}-{r['hi']:.2f})",
va="center", ha="left", fontsize=10, color=INK, transform=tr)
ax.text(1.43, yy, fmt_p(r["p2"]), va="center", ha="left", fontsize=10,
color=BLUE, fontweight="bold" if is_j else "normal",
transform=tr)
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.4, 40)
ticks = [0.5, 1, 2, 5, 10, 20, 40]
ax.set_xticks(ticks)
ax.set_xticklabels([str(t) for t in ticks])
ax.minorticks_off()
ax.set_yticks(ys)
ax.set_yticklabels([k[1] for k in rows])
ax.set_ylim(ys[-1] - 0.8, 1.6)
ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Hazard ratio vs TNFi (log scale); vertical line = 1")
ax.text(0.405, ys[0] + 0.75, "Primary adjusted (Table 2)", fontsize=10,
color=INK2, fontweight="bold")
ax.text(0.405, ys[4] + 0.75, "Propensity-matched (Results)", fontsize=10,
color=INK2, fontweight="bold")
ax.text(1.02, 1.1, "Reported\nHR (95% CI)", fontsize=10, color=ORANGE,
fontweight="bold", ha="left", va="center", transform=tr)
ax.text(1.43, 1.1, "Recomputed\ntwo-sided P", fontsize=10, color=BLUE,
fontweight="bold", ha="left", va="center", transform=tr)
note = ax.text(2.15, ys[2], "Key Points/abstract call this\n"
"\"statistically significantly\"", fontsize=10,
color=INK, va="center", ha="left", linespacing=1.15)
# check that the label stays inside the plot area and the JAKi band
fig.canvas.draw()
bb = note.get_window_extent().transformed(ax.transData.inverted())
assert bb.x1 < ax.get_xlim()[1] and bb.y0 > ys[2] - 0.45 \
and bb.y1 < ys[2] + 0.45, bb
save(fig, "cand_A")
# --- Candidate B: back-calculated z statistics against the +/-1.96 line ------
def cand_B():
rows = [("Primary", d) for d in ORDER] + [("PS-matched", d) for d in ORDER]
labels = [f"{d}, {'primary' if a == 'Primary' else 'PS-matched'}"
for a, d in rows]
zs = [RES[k]["z"] for k in rows]
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ys = -np.arange(len(rows), dtype=float)
ys[4:] -= 0.6
ax.axvspan(-Z975, Z975, color="#efeee9", zorder=0)
for xv in (-Z975, Z975):
ax.axvline(xv, color=INK2, lw=0.9, ls=(0, (4, 3)), zorder=1)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
for yy, zz, k in zip(ys, zs, rows):
ax.plot([0, zz], [yy, yy], color=BLUE, lw=LW, zorder=2,
solid_capstyle="butt")
ax.plot(zz, yy, "o", color=BLUE, ms=MS, zorder=3,
markeredgecolor=BG, markeredgewidth=1.2)
tx = zz + 0.08 if (zz + 0.08 > Z975 + 0.05 or zz < 1.0) else Z975 + 0.06
ax.text(tx, yy, f"z = {zz:.2f}, P = {fmt_p(RES[k]['p2'])}",
va="center", fontsize=10, color=INK)
ax.set_yticks(ys)
ax.set_yticklabels(labels)
ax.set_xlim(-0.6, 4.6)
ax.set_ylim(ys[-1] - 0.8, 2.0)
ax.set_xlabel("z = ln(HR) / SE, back-calculated from the reported 95% CI")
ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.text(0.68, 1.15, "Not significant at the paper's\nstated 2-sided P < .05\n"
"(|z| < 1.96)", fontsize=10, color=INK2, ha="center", va="center")
ax.annotate("Key Points/abstract:\n\"statistically\nsignificantly\" higher",
xy=(j["z"], ys[2] - 0.12), xytext=(2.35, ys[3] - 0.05),
fontsize=10,
color=ORANGE, fontweight="bold", va="center",
arrowprops=dict(arrowstyle="-", color=ORANGE, lw=1.0))
fig.subplots_adjust(left=0.28, right=0.98, top=0.93, bottom=0.14)
save(fig, "cand_B")
# --- Candidate C: two text panels joined by an annotated connector ----------
def wrap_panel(ax, x, y, w, h, title, body):
ax.add_patch(FancyBboxPatch((x, y), w, h,
boxstyle="round,pad=0.008,rounding_size=0.015",
fc="#ffffff", ec=INK2, lw=1.0))
ax.text(x + 0.02, y + h - 0.04, title, fontsize=11, fontweight="bold",
va="top", color=INK)
ax.text(x + 0.02, y + h - 0.13, body, fontsize=10, va="top", color=INK,
linespacing=1.4)
def cand_C():
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
left = ("Key Points, Findings:\n"
"\"a statistically significantly higher\n"
"risk of incident cancer was seen with\n"
"rituximab, abatacept, and JAKis\n"
"compared with TNFis\"\n\n"
"Conclusions and Relevance:\n"
"\"...rituximab, abatacept, and JAKis\n"
"demonstrated ... statistically\n"
"significantly increased risks\"")
right = ("Table 2 and Results:\n"
"JAKis (HR, 1.36; 95% CI, 0.94-1.96)\n\n"
"Discussion:\n"
"\"only rituximab and abatacept\n"
"initiations had statistically\n"
"significant associations\"\n\n"
"Main-text Conclusions name only\n"
"rituximab and abatacept")
wrap_panel(ax, 0.03, 0.22, 0.40, 0.70, "Key Points and Abstract", left)
wrap_panel(ax, 0.57, 0.22, 0.40, 0.70, "Body of the paper", right)
arr = FancyArrowPatch((0.435, 0.57), (0.565, 0.57), arrowstyle="<->",
mutation_scale=14, color=INK2, lw=1.5)
ax.add_patch(arr)
ax.text(0.5, 0.64, "conflict", ha="center", fontsize=10, color=INK2,
fontweight="bold")
# bottom strip: the numbers that decide it
ax.text(0.03, 0.13, "Reported JAKi vs TNFi HR:", fontsize=10.5, color=INK2)
ax.text(0.30, 0.13, "1.36 (95% CI 0.94-1.96), CI includes 1", fontsize=10.5,
color=ORANGE, fontweight="bold")
ax.text(0.03, 0.06, "Recomputed from that CI:", fontsize=10.5, color=INK2)
ax.text(0.30, 0.06, f"z = {j['z']:.2f}, two-sided P = {j['p2']:.2f} "
"(paper's threshold: P < .05)", fontsize=10.5, color=BLUE,
fontweight="bold")
save(fig, "cand_C")
# --- Candidate D: every JAKi estimate the paper's numbers give, one axis -----
def cand_D():
pm = RES[("PS-matched", "JAKi")]
p_irr = 2 * stats.norm.sf(abs(np.log(irr) / se_irr))
items = [
("Adjusted HR, primary\n(Table 2)", j["hr"], j["lo"], j["hi"], ORANGE,
"reported", j["p2"]),
("Adjusted HR, propensity-\nmatched (Results)", pm["hr"], pm["lo"],
pm["hi"], ORANGE, "reported", pm["p2"]),
("Crude rate ratio\n94 / 78 per 10 000 PY", irr, irr_lo, irr_hi, BLUE,
"recomputed", p_exact),
]
fig, ax = plt.subplots(figsize=(7.2, 3.2))
tr = blended_transform_factory(ax.transAxes, ax.transData)
ys = [0, -1, -2]
for yy, (lab, est, lo, hi, col, kind, p) in zip(ys, items):
ax.plot([lo, hi], [yy, yy], color=col, lw=LW, solid_capstyle="butt",
zorder=2)
ax.plot(est, yy, "o" if kind == "reported" else "D", color=col, ms=MS,
markeredgecolor=BG, markeredgewidth=1.2, zorder=3)
ax.text(1.04, yy, f"{est:.2f} ({lo:.2f}-{hi:.2f})", va="center",
fontsize=10, color=INK, transform=tr)
ax.text(1.56, yy, f"{p:.2f}", va="center", fontsize=10, color=BLUE,
transform=tr)
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.5, 4)
ticks = [0.5, 0.75, 1, 1.5, 2, 3, 4]
ax.set_xticks(ticks)
ax.set_xticklabels([str(t) for t in ticks])
ax.minorticks_off()
ax.set_yticks(ys)
ax.set_yticklabels([it[0] for it in items])
ax.set_ylim(-2.7, 1.0)
ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("JAKi vs TNFi ratio (log scale); vertical line = 1")
ax.text(1.04, 0.62, "Estimate\n(95% CI)", fontsize=10, color=INK2,
fontweight="bold", transform=tr, va="center")
ax.text(1.56, 0.62, "Recomputed\nP", va="center", fontsize=10, color=BLUE,
fontweight="bold", transform=tr)
ax.plot([], [], "o", color=ORANGE, ms=MS, label="Reported in the paper")
ax.plot([], [], "D", color=BLUE, ms=MS,
label="Recomputed from Table 2 rates")
ax.legend(loc="lower center", bbox_to_anchor=(0.62, 1.02), ncol=2,
frameon=False, fontsize=10, handlelength=1.0)
fig.subplots_adjust(left=0.27, right=0.62, top=0.82, bottom=0.17)
save(fig, "cand_D")
# --- Candidate E: what each section says, drug by drug -----------------------
def cand_E():
cols = ["Key Points", "Abstract\nConclusions", "Discussion",
"Main-text\nConclusions", "Recomputed\nP (Table 2)"]
drugs = ["Rituximab", "Abatacept", "JAKi"]
S, N, O = "significant", "not significant", "not named"
says = {
"Rituximab": [S, S, S, S],
"Abatacept": [S, S, S, S],
"JAKi": [S, S, N, O],
}
fig, ax = plt.subplots(figsize=(7.2, 2.9))
ax.set_xlim(-0.5, len(cols) - 0.5)
ax.set_ylim(-len(drugs) + 0.5, 1.15)
ax.axis("off")
for ci, c in enumerate(cols):
ax.text(ci, 0.85, c, ha="center", va="center", fontsize=10,
fontweight="bold", color=BLUE if ci == 4 else INK)
ax.axhspan(-2.45, -1.55, xmin=0, xmax=1, color="#f3f1ec", zorder=0)
for ri, d in enumerate(drugs):
y = -ri
ax.text(-0.52, y, d, ha="right", va="center", fontsize=10.5,
fontweight="bold", clip_on=False)
for ci, v in enumerate(says[d]):
if v == S:
ax.plot(ci - 0.42, y, "o", color=ORANGE, ms=MS)
elif v == N:
ax.plot(ci - 0.42, y, "o", mfc=BG, mec=ORANGE, mew=2, ms=MS)
ax.text(ci - 0.33, y, v, va="center", fontsize=10,
color=INK if v != O else INK2)
p = RES[("Primary", d)]["p2"]
ax.text(4, y, f"{p:.3f}", ha="center", va="center", fontsize=10.5,
color=BLUE, fontweight="bold")
ax.plot([], [], "o", color=ORANGE, ms=MS,
label="paper says statistically significant")
ax.plot([], [], "o", mfc=BG, mec=ORANGE, mew=2, ms=MS,
label="paper says not significant")
ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.02), ncol=2,
frameon=False, fontsize=10)
fig.subplots_adjust(left=0.15, right=0.99, top=0.97, bottom=0.18)
save(fig, "cand_E")
cand_A()
cand_B()
cand_C()
cand_D()
cand_E()
# fig1 / fig2 are copies of the chosen candidates
CHOSEN = {"fig1": "cand_C", "fig2": "cand_A"}
for fig_name, cand in CHOSEN.items():
for ext in ("svg", "png"):
shutil.copyfile(os.path.join(HERE, f"{cand}.{ext}"),
os.path.join(HERE, f"{fig_name}.{ext}"))
print("\nFigures written to", HERE, "| chosen:", CHOSEN)
Week-24 HDRS-17 difference reported as P < .001; its t129 = 0.71 gives P = .48
JAMA Netw Open (2025) · doi:10.1001/jamanetworkopen.2025.29427 · PMC12406064
Summary
This randomized trial compared measurement-based care (MBC) with standard care for major depressive disorder. For the secondary outcome of change in HDRS-17 score at week 24, it reports a reduction of 18.1 points (95% CI, 16.4–19.6) with MBC and 17.0 points (95% CI, 15.6–18.5) with standard care, with "t129 = 0.71; P < .001". The same values appear in the Abstract, the Results and Table 3, and the Abstract calls the difference "significantly but modestly greater". The printed statistic gives a two-sided P of 0.48: on 129 degrees of freedom, P < .001 needs |t| ≥ 3.37. Rebuilding the test from the printed group CIs gives t = 1.02 and P = 0.31, and the 95% CI of the 1.1-point difference (−1.04 to 3.24) includes zero. As printed, the Abstract presents the trial's prespecified end-of-study outcome for change in symptom severity as a significant benefit of MBC, but the paper's own numbers show no significant difference. The other week-24 efficacy results were already nonsignificant. The primary time-to-event outcomes are not affected.
What the paper reports
Abstract, Results:
After week 24, reduction in mean HDRS-17 scores was significantly but modestly greater in the MBC group than in the standard care group (-18.1 [95% CI, 16.4-19.6] points vs -17.0 [95% CI, 15.6-18.5] points; t129 = 0.71; P < .001).
The sentence just before it in the Abstract reads:
At week 24, there were no significant differences in rates of response or remission between groups.
Results, fourth paragraph (beginning "After week 24, the response rates..."):
After week 24, the reduction in mean HDRS-17 scores was larger in the MBC group than in the standard care group (-18.1 [95% CI, 16.4-19.6] points vs -17.0 [95% CI, 15.6-18.5] points; t129 = 0.71; P < .001) (Table 3)
Table 3 (Efficacy Outcomes), last row (measure | MBC group | standard care group | P value):
Change in HDRS-17 score, mean (95% CI), points | 18.1 (16.4-19.6) | 17.0 (15.6-18.5) | <.001
Methods, Statistical Analysis:
Baseline HDRS-17 scores, antidepressant doses, number of clinic visits, changes in HDRS-17 at week 24, and number of adverse effects were compared with independent, 2-tailed t tests or Mann-Whitney tests, as appropriate. The significance threshold was 2-tailed P < .05 for all analyses.
Supplement 1 (trial protocol and statistical analysis plan), Statistical analysis, in both the protocol and the SAP section:
Analysis of covariance will be used to compare the changes in HDRS scores between the two groups.
(The protocol version reads "changes in HDRS and YMRS scores".) In the same documents, independent-sample t tests, Mann-Whitney U tests and χ² tests are planned for sociodemographic and clinical characteristics, discontinuation, response and remission rates, and side effects. The published Methods instead list changes in HDRS-17 at week 24 among the outcomes compared with independent t tests or Mann-Whitney tests.
Table 2, week 24 (MBC vs standard care): "No." 64 vs 67; "HDRS-17 score, mean (SD)" 5.6 (4.2) vs 6.7 (4.7), P value ".14".
In the archived text, "t129" is t with subscript 129 (the degrees of freedom). The publisher's HTML prints the two changes as −18.1 and −17.0 with a true minus sign. All quotes were checked verbatim against the PMC-archived article text.
What the paper's own numbers show
Method. I made three independent checks, all with scipy (see recompute.py.txt).
- I converted the printed statistic, t = 0.71 on 129 df, to a two-sided P value.
- I rebuilt the between-group test from the printed group means and 95% CIs. I took each group's standard error as (upper − lower) / (2 × t₀.₉₇₅, n−1), using the week-24 sample sizes of 64 and 67 from Table 2. These sizes also match Table 3's percentages (57/64 = 89.1%, 58/67 = 86.6%) and the printed df (64 + 67 − 2 = 129). From those SEs I ran a pooled-variance t test, and as a check a Welch test and a normal approximation. To allow for rounding, I shifted each of the six printed values by ±0.05 and recomputed t at every combination of shifts.
- I recomputed the Table 2 week-24 comparison of HDRS-17 levels from the printed means, SDs and sample sizes.
- I approximated the analysis the protocol planned, an analysis of covariance (ANCOVA) of the change with baseline HDRS-17 as covariate. From the identity SD(change)² = SD(baseline)² + SD(week 24)² − 2r·SD(baseline)·SD(week 24), the pooled SDs (baseline 4.66 from Table 2, week 24 4.46 from Table 2, change 6.17 from the Table 3 CIs) give the baseline-to-week-24 correlation r. The ANCOVA residual SD is then SD(week 24)·√(1 − r²). This is approximate: Table 2's baseline SDs (5.0 and 4.3) are for all 154 randomized participants, not only the 131 week-24 completers. The completers' baseline means implied by Table 2 and Table 3 (18.1 + 5.6 and 17.0 + 6.7) are both 23.7, so baseline adjustment leaves the 1.1-point difference unchanged.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Two-sided P for t = 0.71, df = 129 | Abstract; Results, 4th paragraph | P < .001 | 0.479 (one-sided 0.239); 0.476–0.482 for t from 0.705 to 0.715 |
| |t| needed for two-sided P < .001 at df = 129 | derived | — | 3.37 (for P < .05: 1.98) |
| Degrees of freedom | Abstract; Table 2 week-24 "No." | 129 | 64 + 67 − 2 = 129, consistent with a two-group comparison |
| Difference in mean change, MBC − standard care | Table 3 | 18.1 vs 17.0 | 1.1 points |
| Group SEs implied by the 95% CIs | Table 3 | CI widths 3.2 and 2.9 | 0.80 (SD 6.41) and 0.73 (SD 5.94) |
| Between-group t from the CIs (pooled) | Table 3 | t129 = 0.71; P < .001 | t = 1.02, df 129, P = 0.31 (Welch t = 1.02, df 127, P = 0.31; normal approximation z = 1.00, P = 0.32) |
| Same, with ±0.05 rounding on all printed values | Table 3 | — | t 0.90 to 1.15; P 0.25 to 0.37 |
| 95% CI of the 1.1-point difference | derived from Table 3 | not reported | −1.04 to 3.24 (includes 0) |
| SE of the difference that P < .001 would need | derived | — | ≤ 0.33; the CIs imply 1.08, about 3.3 times larger |
| SE and 95% CI implied by the printed t = 0.71 | derived | — | SE 1.55; CI −1.97 to 4.17 (includes 0) |
| ANCOVA adjusted for baseline (protocol's planned analysis) | Supplement 1; Tables 2–3 SDs | not reported | residual df 128, so F₁,₁₂₈, not t₁₂₉; implied r ≈ 0.08 (0.04 and 0.13 by group); F₁,₁₂₈ ≈ 2.0, P ≈ 0.16 |
| Correlation the ANCOVA would need for P < .001 | derived | — | residual SD ≤ 1.87, i.e. r ≥ 0.91 (for P < .05: r ≥ 0.70) |
| Week-24 HDRS-17 level, 5.6 (4.2) n = 64 vs 6.7 (4.7) n = 67 | Table 2 | P = .14 | t = −1.41, df 129, P = 0.16 (also nonsignificant) |
| Within-group change from baseline (one-sample t) | from Table 3 CIs | — | MBC t = 22.6, df 63; standard care t = 23.4, df 66; both P < 10⁻³⁰ |
| Week-4 HDRS-17 level, 8.3 (4.9) vs 12.6 (8.1) | Table 2 | P < .001 | t = −3.72, df 135, P = 0.0003 (consistent) |
| Weeks 2, 8, 12 HDRS-17 level | Table 2 | .05, .02, .04 | 0.056, 0.017, 0.034 (consistent) |
| Response at week 24, 57/64 vs 58/67 | Table 3; Results, 4th paragraph | χ² = 0.19; P = .79 | Pearson χ² = 0.19, P = 0.66; Fisher exact P = 0.79 |
| Remission at week 24, 47/64 vs 42/67 | Table 3; Results, 4th paragraph | χ² = 1.74; P = .20 | Pearson χ² = 1.74, P = 0.19; Fisher exact P = 0.20 |
| Response at week 12, 58/64 vs 46/68 | Table 3 | P < .001 | Pearson P = 0.0012; Fisher exact P = 0.0013 |
Every check of the week-24 change comparison gives a P value between about 0.16 and 0.48. None comes near .001. The other HDRS-17 comparisons in Table 2 and the response and remission comparisons in Table 3 reproduce closely; the dose and visit comparisons in Table 2 were not checked. Two of them have small discrepancies that do not change significance: the week-24 rate P values match Fisher's exact test rather than the Pearson χ² printed beside them, and the week-12 response P is 0.0012–0.0013, not < .001. Those two points are outside the scope of this report. The printed t of 0.71 is not exactly reproducible from the printed CIs either (they give t ≈ 1.02). Whichever of the two is used, the result is clearly nonsignificant.
Figure
Figure 1. Two-sided P values for MBC versus standard care at week 24, on one log-scaled axis. The thin vertical line marks the paper's significance threshold of P = .05. Orange marks values printed in the paper and blue marks values recomputed from the paper's own numbers. The top row is the printed P < .001 for the difference in HDRS-17 change; the arrow shows that the printed value is an upper bound. The next three rows recompute the same comparison: from the printed statistic t129 = 0.71 (P = 0.48); from the printed group 95% CIs (P = 0.31; the short bar is the range when each printed value is shifted by ±0.05 for rounding, 0.25–0.37); and as the ANCOVA planned in the protocol, approximated from the printed SDs (P ≈ 0.16 at the implied baseline-to-week-24 correlation r ≈ 0.08). Each row's value is printed to the left of the threshold line, joined to its marker by a dotted leader. The bottom two rows show the closely related week-24 comparison of HDRS-17 levels in Table 2, as printed (P = .14) and as recomputed (P = 0.16). Only the printed P for the change score lies below the threshold.
Figure 2. The between-group difference in week-24 HDRS-17 change (MBC minus standard care, 1.1 points), on a points scale with zero (no difference) marked. Blue: the 95% CI of the difference implied by the printed group CIs (−1.04 to 3.24) and by the printed t129 = 0.71 (−1.97 to 4.17). Both include zero. Orange, dashed: the widest 95% CI that a 1.1-point difference could have if the printed P < .001 were correct (0.45 to 1.75, SE ≤ 0.33). That interval is about one third as wide as the one the paper's own CIs imply.
Why it matters
Change in HDRS-17 score at week 24 is a prespecified secondary outcome: the trial's end-of-study outcome for change in symptom severity. The Abstract reports it as the one positive week-24 efficacy finding ("significantly but modestly greater ... P < .001"), immediately after stating that week-24 response and remission rates did not differ. With the P value corrected, no week-24 efficacy outcome favours MBC significantly: response P = .79, remission P = .20, HDRS-17 level P = .14 (Table 2), and HDRS-17 change P ≈ .48 (from the printed t), ≈ .31 (from the printed CIs) or about .16 (the protocol's ANCOVA, approximated from the printed SDs). The difference still favours MBC by 1.1 points, but its 95% CI runs from about −1 to +3 points. That interval is compatible with no difference.
As printed, a reader of the Abstract would conclude that MBC produced both faster improvement and a reliably greater reduction in severity by the end of the study. The paper's own numbers support only the first. The error is repeated in three places (Abstract, Results, Table 3), with no correct value given anywhere in the paper.
Not affected: the primary outcomes, time to response and time to remission (log-rank P = .001 for both; Cox HR 1.53 and 1.80); the week-12 rate differences; and the formal Conclusions, which rest on the time-to-event results. The Key Points do not cite this result.
Explanations considered
- A within-group (change-from-baseline) test. This is the only reading that yields P < .001: the one-sample t statistics are 22.6 and 23.4, with P < 10⁻³⁰. It does not fit the paper, though. Each within-group test has 63 or 66 df, not 129. The reported t is 0.71, not over 20. The P value sits in Table 3's between-group P value column. Both text passages describe a comparison ("greater in the MBC group than in the standard care group"). It does not account for the numbers.
- A one-sided test. One-sided P = 0.24. The Methods specify 2-tailed tests. It does not account for the numbers.
- A Mann-Whitney test instead of a t test. The Methods allow either "as appropriate". The printed statistic, however, is a t with 129 df, not a U or z statistic. The summary data show a 1.1-point difference against within-group SDs of about 6 (a standardized difference of about 0.18). A rank test with P < .001 would need a much larger separation between the groups than these summaries suggest. A rank-based result cannot be fully ruled out from summary data alone, but nothing in the paper reports one, and it would not match the t129 = 0.71 printed next to the P value.
- A covariate-adjusted analysis. The trial protocol and statistical analysis plan (Supplement 1) planned exactly this: "Analysis of covariance will be used to compare the changes in HDRS scores between the two groups." The published Methods instead name independent t tests or Mann-Whitney tests for the change at week 24. An ANCOVA still does not fit the printed result. With baseline as a covariate, the model has 64 + 67 − 3 = 128 residual df, and the group effect would be reported as F on 1 and 128 df, not as t129. On the numbers, P < .001 for a 1.1-point adjusted difference needs F ≥ 11.3, which means a residual SD of at most 1.87 and a baseline-to-week-24 correlation of at least 0.91. The printed SDs (baseline 5.0 and 4.3, week 24 4.2 and 4.7, change about 6.17 from the CIs) imply a correlation of about 0.08 (0.04 in the MBC group, 0.13 with standard care). That is approximate, because the baseline SDs are for all randomized participants, but it is far below 0.91. At that correlation the ANCOVA gives F₁,₁₂₈ ≈ 2.0 and P ≈ 0.16; even r = 0.7 would give only P ≈ .05. It does not account for the numbers.
- The t statistic is the misprint, not the P value. If P < .001 were correct, |t| would have to be at least 3.37 and the SE of the difference at most 0.33. The printed group CIs independently imply an SE of about 1.08 and t ≈ 1.02. Table 2's week-24 levels (P = .14) also show no significant difference. The P value is the number that conflicts with everything else, whichever of the two printed statistics is used.
- Rounding, or asymmetric CIs. Shifting all six printed values by ±0.05 keeps P between 0.25 and 0.37. The MBC interval is slightly asymmetric (1.7 below the mean, 1.5 above), which may mean it is not a plain t interval (for example, a bootstrap interval). Even if both group SEs were halved, t would be about 2 and P about .04, still far from .001. Rounding does not account for the numbers.
- Text-conversion artifact. The publisher's current HTML, the PMC page and the Europe PMC JATS XML all carry the same values. In the XML, the statistic is marked up as italic t with subscript 129, "0.71" uses a normal decimal point, and "<" encodes "<". The Table 3 row has four cells, with "<.001" in the P value column. A conversion artifact is ruled out.
Status
As of 2026-10-23:
- Version of record. The DOI resolves to the publisher's article page (jamanetwork.com/journals/jamanetworkopen/fullarticle/2838317). I fetched it on 2026-10-23. It is the current, corrected version and carries the notice "This article was corrected on July 14, 2026, to fix the values reported in the last 3 rows of Table 1." Its Abstract, Results and Table 3 still read "t129 = 0.71; P < .001" and "<.001", as quoted above. The PMC HTML page and the Europe PMC JATS XML match.
- Corrections. There is one correction: "Errors in Table 1" (doi:10.1001/jamanetworkopen.2026.26785; PMC13370302; July 2026). It concerns only the annual participant income, household income and time-since-diagnosis medians and IQRs in Table 1, which had been calculated with the SPSS "HAVERAGE" method. It does not address the week-24 HDRS-17 change result. Crossref update metadata and PMC notices list nothing else; Europe PMC lists only this correction.
- Supplements. I read all three supplementary PDFs from the Europe PMC supplementary-files archive. Supplement 1 is the trial protocol and statistical analysis plan. It plans independent-sample t tests, Mann-Whitney U tests and χ² tests for baseline characteristics, discontinuation, response and remission rates and side effects, and specifies analysis of covariance for the between-group comparison of HDRS changes; the published Methods specify independent t tests or Mann-Whitney tests for that comparison. Supplement 1 contains no outcome numbers. Supplement 2 contains eTables 1–3 (MBC schedule, adherence, adverse effects) and the eFigure (time to discontinuation); none of them analyses week-24 HDRS-17 change. Supplement 3 is the data sharing statement. Nothing in the supplements resolves the inconsistency.
- PubPeer and other discussion. An earlier audit step on the same day found no PubPeer thread for the DOI or title. The secondary news coverage that was read (2 Minute Medicine) does not mention the inconsistency. The full text of the linked invited commentary (doi:10.1001/jamanetworkopen.2025.29440) could not be accessed: the publisher page returned a bot-verification page, the PDF text was not extractable, and it is not in PMC or Europe PMC full text. Whether it mentions the inconsistency is therefore unknown. The PubPeer search page and API returned a human-verification challenge (HTTP 403). A repeat web search for this write-up was not possible because the session's search allowance was used up.
- Sources. https://doi.org/10.1001/jamanetworkopen.2025.29427; https://pmc.ncbi.nlm.nih.gov/articles/PMC12406064/; Europe PMC full text and supplementary files for PMC12406064; https://doi.org/10.1001/jamanetworkopen.2026.26785 (PMC13370302); Supplement 1 (jamanetwopen-e2529427-s001.pdf, Europe PMC supplementary files); https://www.2minutemedicine.com/measurement-based-care-to-enhance-antidepressant-treatment-outcomes-in-major-depressive-disorder-a-randomized-clinical-trial/.
Suggested correction
The authors should check the analysis output for the week-24 change comparison. The P value is 0.48 if the printed t129 = 0.71 is right, and about 0.31 if the printed group CIs are right; whichever value the analysis produces should be reported with the matching statistic. Adding the between-group difference and its 95% CI would let readers check the result directly. Using the printed t:
Abstract, Results. Replace "After week 24, reduction in mean HDRS-17 scores was significantly but modestly greater in the MBC group than in the standard care group (... t129 = 0.71; P < .001)." with:
After week 24, reduction in mean HDRS-17 scores did not differ significantly between the MBC group and the standard care group (−18.1 [95% CI, 16.4-19.6] points vs −17.0 [95% CI, 15.6-18.5] points; t129 = 0.71; P = .48).
Results, fourth paragraph. Replace "the reduction in mean HDRS-17 scores was larger in the MBC group than in the standard care group (... t129 = 0.71; P < .001)" with:
the reduction in mean HDRS-17 scores did not differ significantly between the MBC group and the standard care group (−18.1 [95% CI, 16.4-19.6] points vs −17.0 [95% CI, 15.6-18.5] points; t129 = 0.71; P = .48)
Table 3, last row. Replace the P value "<.001" with ".48", or with the value from the verified analysis.
With these changes, the Abstract sentence "No differences were observed in other secondary outcomes" would also cover HDRS-17 change. The Conclusions do not need to change.
Recomputation script (python recompute.py.txt)
"""Recompute the week-24 between-group test of HDRS-17 change in PMC12406064
(JAMA Netw Open 2025; doi:10.1001/jamanetworkopen.2025.29427) from the paper's own
printed numbers, run the cross-checks used in the write-up, and redraw fig1/fig2
in this file's folder.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
Every input below is copied from the published article (Abstract, Results,
Methods, Table 2, Table 3), except the analysis-plan quote in section 2b, which is
from Supplement 1 (trial protocol and statistical analysis plan).
"""
import os
import itertools
import numpy as np
from scipy import stats
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
# Abstract, Results (fourth paragraph) and Table 3 (last row): change in HDRS-17 score, mean (95% CI)
M1, LO1, HI1 = 18.1, 16.4, 19.6 # MBC
M2, LO2, HI2 = 17.0, 15.6, 18.5 # standard care
T_REP, DF_REP = 0.71, 129 # "t129 = 0.71"
P_REP = 0.001 # "P < .001"
# Table 2, week 24 "No." row (consistent with Table 3: 57/64 = 89.1%, 58/67 = 86.6%)
N1, N2 = 64, 67
# Table 2: HDRS-17 mean (SD), n, reported P (MBC first, then standard care)
TABLE2 = {
"Baseline": ((24.1, 5.0, 76), (23.4, 4.3, 78), None),
"Wk 2": ((12.2, 5.3, 68), (14.6, 8.9, 72), ".05"),
"Wk 4": ((8.3, 4.9, 66), (12.6, 8.1, 71), "<.001"),
"Wk 8": ((7.7, 5.2, 64), (10.4, 7.4, 68), ".02"),
"Wk 12": ((6.9, 5.3, 64), (9.2, 6.9, 68), ".04"),
"Wk 24": ((5.6, 4.2, 64), (6.7, 4.7, 67), ".14"),
}
# Table 3 / Results (fourth paragraph): response and remission counts, reported chi2 and P
TABLE3 = [
("Response wk 12", (58, 64), (46, 68), None, "<.001"),
("Response wk 24", (57, 64), (58, 67), 0.19, ".79"),
("Remission wk 12", (46, 64), (35, 68), None, ".02"),
("Remission wk 24", (47, 64), (42, 67), 1.74, ".20"),
]
R = {} # numbers reused by the figures
def p2(t, df):
return 2 * stats.t.sf(abs(t), df)
print("=== 1. The reported statistic: t129 = 0.71 ===")
R["p_from_t"] = p2(T_REP, DF_REP)
R["t_crit_001"] = stats.t.isf(0.0005, DF_REP)
R["t_crit_05"] = stats.t.isf(0.025, DF_REP)
print(f"df check: n1 + n2 - 2 = {N1} + {N2} - 2 = {N1 + N2 - 2} (reported {DF_REP})")
print(f"two-sided P for t = {T_REP}, df = {DF_REP}: {R['p_from_t']:.3f} (one-sided {R['p_from_t']/2:.3f})")
print(f"|t| needed for two-sided P < .001 at df = {DF_REP}: {R['t_crit_001']:.2f}")
print(f"|t| needed for two-sided P < .05 at df = {DF_REP}: {R['t_crit_05']:.2f}")
print(f"largest two-sided P for t in [0.705, 0.715): "
f"{p2(0.705, DF_REP):.3f}; smallest: {p2(0.715, DF_REP):.3f}")
print("\n=== 2. Between-group test rebuilt from the reported group 95% CIs ===")
q1, q2 = stats.t.ppf(0.975, N1 - 1), stats.t.ppf(0.975, N2 - 1)
se1, se2 = (HI1 - LO1) / (2 * q1), (HI2 - LO2) / (2 * q2)
sd1, sd2 = se1 * np.sqrt(N1), se2 * np.sqrt(N2)
diff = M1 - M2
print(f"MBC: CI width {HI1-LO1:.1f} -> SE {se1:.3f}, implied SD {sd1:.2f} "
f"(CI extends {M1-LO1:.1f} below, {HI1-M1:.1f} above the mean)")
print(f"SC : CI width {HI2-LO2:.1f} -> SE {se2:.3f}, implied SD {sd2:.2f} "
f"(CI extends {M2-LO2:.1f} below, {HI2-M2:.1f} above the mean)")
print(f"difference in mean change: {M1} - {M2} = {diff:.1f} points")
# pooled-variance (Student) t test, df = 129, as the reported subscript implies
sp = np.sqrt(((N1 - 1) * sd1**2 + (N2 - 1) * sd2**2) / (N1 + N2 - 2))
se_p = sp * np.sqrt(1 / N1 + 1 / N2)
t_p = diff / se_p
R["t_ci"], R["p_ci"], R["se_ci"] = t_p, p2(t_p, N1 + N2 - 2), se_p
qd = stats.t.ppf(0.975, N1 + N2 - 2)
R["ci_diff"] = (diff - qd * se_p, diff + qd * se_p)
print(f"Student: pooled SD {sp:.2f}, SE(diff) {se_p:.3f}, t = {t_p:.2f}, df = {N1+N2-2}, "
f"P = {R['p_ci']:.2f}")
print(f" 95% CI of the difference: {R['ci_diff'][0]:.2f} to {R['ci_diff'][1]:.2f}")
# Welch
se_w = np.sqrt(se1**2 + se2**2)
df_w = se_w**4 / (se1**4 / (N1 - 1) + se2**4 / (N2 - 1))
t_w = diff / se_w
print(f"Welch : SE(diff) {se_w:.3f}, t = {t_w:.2f}, df = {df_w:.0f}, P = {p2(t_w, df_w):.2f}")
# normal approximation (SE = width / 3.92)
se_z = np.sqrt(((HI1 - LO1) / 3.92) ** 2 + ((HI2 - LO2) / 3.92) ** 2)
z = diff / se_z
print(f"z-approx: SE(diff) {se_z:.3f}, z = {z:.2f}, P = {2*stats.norm.sf(z):.2f}")
# rounding: every printed value may be off by up to 0.05; take all corners
ts = []
for a, b, l1, h1, l2, h2 in itertools.product(*[(-0.05, 0.05)] * 6):
s1 = ((HI1 + h1) - (LO1 + l1)) / (2 * q1)
s2 = ((HI2 + h2) - (LO2 + l2)) / (2 * q2)
ts.append(((M1 + a) - (M2 + b)) / np.sqrt(s1**2 + s2**2))
R["t_rng"] = (min(ts), max(ts))
R["p_rng"] = (p2(max(ts), DF_REP), p2(min(ts), DF_REP))
print(f"rounding (+/-0.05 on all six printed values): t {R['t_rng'][0]:.2f} to {R['t_rng'][1]:.2f}, "
f"two-sided P {R['p_rng'][0]:.2f} to {R['p_rng'][1]:.2f}")
R["se_needed"] = diff / R["t_crit_001"]
print(f"SE(diff) a 1.1-point difference would need for P < .001: {R['se_needed']:.2f} "
f"(the CIs imply {se_p:.2f}, about {se_p / R['se_needed']:.1f} times larger)")
R["se_from_t"] = diff / T_REP
R["ci_from_t"] = (diff - qd * R["se_from_t"], diff + qd * R["se_from_t"])
print(f"SE(diff) implied by t = 0.71 and diff = 1.1: {R['se_from_t']:.2f} "
f"(pooled SD {R['se_from_t'] / np.sqrt(1/N1 + 1/N2):.2f}); "
f"95% CI {R['ci_from_t'][0]:.2f} to {R['ci_from_t'][1]:.2f}")
R["ci_needed"] = (diff - qd * R["se_needed"], diff + qd * R["se_needed"])
print(f"95% CI of the difference if P were exactly .001: "
f"{R['ci_needed'][0]:.2f} to {R['ci_needed'][1]:.2f}")
print(f"standardized difference: {diff:.1f} / pooled SD {sp:.2f} = {diff/sp:.2f}")
t_half = diff / (se_p / 2)
print(f"if both group SEs were halved: t = {t_half:.2f}, P = {p2(t_half, DF_REP):.3f}")
print("\n=== 2b. The protocol's planned analysis: ANCOVA adjusted for baseline ===")
# Supplement 1 (protocol and SAP): "Analysis of covariance will be used to compare the
# changes in HDRS scores between the two groups." The published Methods instead name
# independent t tests or Mann-Whitney tests. An ANCOVA with group + baseline HDRS-17 would
# report F on 1 and N - 3 df, not t on N - 2 df.
DF_ANC = N1 + N2 - 3
print(f"residual df of a two-group model adjusted for baseline: {N1} + {N2} - 3 = {DF_ANC} "
f"(printed subscript {DF_REP}); statistic would be F(1, {DF_ANC}), not t{DF_REP}")
(bx1, bsx1, bn1), (bx2, bsx2, bn2), _ = TABLE2["Baseline"]
(wy1, wsy1, _), (wy2, wsy2, _), _ = TABLE2["Wk 24"]
# baseline SDs are for all randomized participants (n 76/78), week-24 SDs for completers
sx = np.sqrt(((bn1 - 1) * bsx1**2 + (bn2 - 1) * bsx2**2) / (bn1 + bn2 - 2))
sy = np.sqrt(((N1 - 1) * wsy1**2 + (N2 - 1) * wsy2**2) / (N1 + N2 - 2))
sd_chg = sp # pooled change SD implied by Table 3 CIs
r_pool = (sx**2 + sy**2 - sd_chg**2) / (2 * sx * sy)
r_g1 = (bsx1**2 + wsy1**2 - sd1**2) / (2 * bsx1 * wsy1)
r_g2 = (bsx2**2 + wsy2**2 - sd2**2) / (2 * bsx2 * wsy2)
print(f"pooled SDs: baseline {sx:.2f} (Table 2, all randomized), week 24 {sy:.2f} (Table 2), "
f"change {sd_chg:.2f} (from Table 3 CIs)")
print(f"implied baseline-to-week-24 correlation r = (sx^2 + sy^2 - s_change^2) / (2 sx sy) "
f"= {r_pool:.2f} (per group: MBC {r_g1:.2f}, standard care {r_g2:.2f})")
# completers' baseline means implied by Table 3 change + Table 2 week-24 mean
cb1, cb2 = M1 + wy1, M2 + wy2
print(f"completers' baseline means implied: MBC {M1} + {wy1} = {cb1:.1f}, "
f"SC {M2} + {wy2} = {cb2:.1f} (so baseline adjustment shifts the 1.1-point difference "
f"by {r_pool * sy / sx * abs(cb1 - cb2):.2f}; adjusted difference = {wy2 - wy1:.1f})")
k = np.sqrt(1 / N1 + 1 / N2)
def ancova(r):
beta = r * sy / sx
# adjusted difference in reduction = -(Y1 - Y2) + beta * (X1 - X2), completers' baselines
adj = (wy2 - wy1) + beta * (cb1 - cb2)
s_res = sy * np.sqrt(1 - r**2) * np.sqrt((N1 + N2 - 2) / DF_ANC)
t = adj / (s_res * k)
return s_res, t, p2(t, DF_ANC)
s_res, t_anc, p_anc = ancova(r_pool)
R["p_ancova"], R["r_pool"] = p_anc, r_pool
print(f"ANCOVA at r = {r_pool:.2f}: residual SD {s_res:.2f}, SE(adjusted diff) {s_res*k:.3f}, "
f"t = {t_anc:.2f}, F(1, {DF_ANC}) = {t_anc**2:.2f}, P = {p_anc:.2f}")
t_need = stats.t.isf(0.0005, DF_ANC)
sres_need = diff / (t_need * k)
r_need = np.sqrt(1 - (sres_need / sy) ** 2 * DF_ANC / (N1 + N2 - 2))
t_need05 = stats.t.isf(0.025, DF_ANC)
r_need05 = np.sqrt(1 - (diff / (t_need05 * k) / sy) ** 2 * DF_ANC / (N1 + N2 - 2))
R["r_need"] = r_need
print(f"for ANCOVA P < .001 (|t| >= {t_need:.2f}, F >= {t_need**2:.1f}): residual SD <= "
f"{sres_need:.2f}, i.e. r >= {r_need:.2f}; for P < .05: r >= {r_need05:.2f}")
for rr in (0.0, 0.3, 0.5, 0.7, 0.9):
_, tt, pp = ancova(rr)
print(f" r = {rr:.1f}: t = {tt:.2f}, F = {tt**2:.2f}, P = {pp:.3f}")
print("\n=== 3. Alternative reading: within-group change from baseline ===")
for name, m, se, n in [("MBC", M1, se1, N1), ("SC", M2, se2, N2)]:
tt = m / se
print(f"{name}: one-sample t = {tt:.1f}, df = {n-1}, P = {p2(tt, n-1):.0e}")
print("\n=== 4. Table 2 cross-checks (Student t from mean, SD, n) ===")
for lbl, ((ma, sa, na), (mb, sb, nb), rep) in TABLE2.items():
s = np.sqrt(((na - 1) * sa**2 + (nb - 1) * sb**2) / (na + nb - 2))
tt = (ma - mb) / (s * np.sqrt(1 / na + 1 / nb))
pp = p2(tt, na + nb - 2)
tw = (ma - mb) / np.sqrt(sa**2 / na + sb**2 / nb)
dfw = (sa**2/na + sb**2/nb)**2 / ((sa**2/na)**2/(na-1) + (sb**2/nb)**2/(nb-1))
if lbl == "Wk 24":
R["p_wk24_level"] = pp
print(f"{lbl:8s}: diff {ma-mb:+.1f}, t = {tt:.2f}, df = {na+nb-2}, P = {pp:.4f}; "
f"Welch P = {p2(tw, dfw):.4f} (reported {rep})")
print(f"Table 2 means: MBC 24.1 - 5.6 = {24.1-5.6:.1f}, SC 23.4 - 6.7 = {23.4-6.7:.1f} "
f"(Table 3 change, completers: 18.1 and 17.0)")
print("\n=== 5. Table 3 rates (Pearson chi2, Yates chi2, Fisher exact) ===")
for lbl, (a, na), (b, nb), chi_rep, rep in TABLE3:
tab = np.array([[a, na - a], [b, nb - b]])
c0 = stats.chi2_contingency(tab, correction=False)
c1 = stats.chi2_contingency(tab, correction=True)
pf = stats.fisher_exact(tab)[1]
print(f"{lbl:16s}: {a}/{na} = {100*a/na:.1f}% vs {b}/{nb} = {100*b/nb:.1f}%; "
f"Pearson chi2 = {c0[0]:.2f}, P = {c0[1]:.4f}; Yates chi2 = {c1[0]:.2f}, P = {c1[1]:.4f}; "
f"Fisher P = {pf:.4f} [reported chi2 {chi_rep}, P {rep}]")
# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK, "axes.edgecolor": INK2, "svg.fonttype": "none"})
LW = 1.5 # 2 px at 96 px/in
MS = 9
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), bbox_inches="tight")
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, bbox_inches="tight")
plt.close(fig)
def style(ax):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
# Fig 1: two-sided P values on one log axis
rows = [
("Week-24 HDRS-17 change\nas printed (Abstract, Results, Table 3)", P_REP, None, "rep"),
("Same test: P from the printed\nstatistic $t_{129}$ = 0.71", R["p_from_t"], None, "rec"),
("Same test: rebuilt from the\nprinted group 95% CIs", R["p_ci"], R["p_rng"], "rec"),
("Protocol's ANCOVA, approximated\nfrom the printed SDs", R["p_ancova"], None, "rec"),
("Week-24 HDRS-17 level\n(Table 2), as printed", 0.14, None, "rep"),
("Week-24 HDRS-17 level\n(Table 2), recomputed", R["p_wk24_level"], None, "rec"),
]
labels = ["P < .001",
f"P = {R['p_from_t']:.2f}",
f"P = {R['p_ci']:.2f} (rounding range {R['p_rng'][0]:.2f}-{R['p_rng'][1]:.2f})",
f"P \u2248 {R['p_ancova']:.2f} ($F_{{1,128}}$ \u2248 {t_anc**2:.1f}, r \u2248 {R['r_pool']:.2f})",
"P = .14",
f"P = {R['p_wk24_level']:.2f}"]
fig, ax = plt.subplots(figsize=(7.2, 4.4))
style(ax)
ax.set_xscale("log")
ax.set_xlim(1e-4, 1.0)
ys = np.arange(len(rows))[::-1].astype(float)
ys[4:] -= 0.35
ax.axvline(0.05, color=INK2, lw=0.9, zorder=1)
ax.text(0.05, ys[0] + 0.62, "P = .05 (paper's threshold) ", color=INK2, fontsize=10,
ha="right", va="center")
XLAB = 0.04 # value labels end left of the threshold line
for y, (lab, p, rng, kind), txt in zip(ys, rows, labels):
c = ORANGE if kind == "rep" else BLUE
if rng is not None:
ax.plot(rng, [y, y], color=c, lw=LW, solid_capstyle="round", zorder=2)
if p == P_REP and kind == "rep":
ax.annotate("", xy=(1.6e-4, y), xytext=(P_REP, y),
arrowprops=dict(arrowstyle="-|>", color=c, lw=LW, mutation_scale=12), zorder=2)
ax.text(p * 1.35, y, txt, color=INK, fontsize=10, va="center")
else:
ax.plot([XLAB * 1.1, (rng[0] if rng else p) / 1.12], [y, y], color=GRID, lw=0.9,
ls=":", zorder=1)
ax.text(XLAB, y, txt, color=INK, fontsize=10, ha="right", va="center")
ax.plot([p], [y], marker="o", ms=MS, color=c, mec=BG, mew=1.5, zorder=3)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(ys[-1] - 0.6, ys[0] + 0.9)
ax.set_xticks([1e-4, 1e-3, 1e-2, 0.05, 0.1, 0.5, 1])
ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.05", "0.1", "0.5", "1"])
ax.set_xlabel("Two-sided P value, MBC vs standard care (log scale)")
handles = [Line2D([], [], ls="", marker="o", ms=MS, color=ORANGE, label="Reported in the paper"),
Line2D([], [], ls="", marker="o", ms=MS, color=BLUE,
label="Recomputed from the paper's own numbers")]
ax.legend(handles=handles, loc="lower left", bbox_to_anchor=(0, 1.0), ncol=2, frameon=False,
fontsize=10, handletextpad=0.3)
save(fig, "fig1")
# Fig 2: the 1.1-point difference and its 95% CI on one axis (HDRS-17 points)
rows2 = [
("95% CI implied by the\nprinted group CIs", R["ci_diff"], BLUE, "-"),
("95% CI implied by the\nprinted $t_{129}$ = 0.71", R["ci_from_t"], BLUE, "-"),
("Widest 95% CI compatible\nwith the printed P < .001", R["ci_needed"], ORANGE, "--"),
]
fig, ax = plt.subplots(figsize=(7.2, 2.9))
style(ax)
ax.axvline(0, color=INK2, lw=0.9, zorder=1)
ax.text(0, 2.75, "no difference ", color=INK2, fontsize=10, ha="right", va="center")
for y, (lab, (lo, hi), c, ls) in zip([2, 1, 0], rows2):
ax.plot([lo, hi], [y, y], color=c, lw=LW, ls=ls, solid_capstyle="round", zorder=2)
ax.plot([diff], [y], marker="o", ms=MS, color=c, mec=BG, mew=1.5, zorder=3)
ax.text(hi + 0.12, y, f"{diff:.1f} ({lo:.2f} to {hi:.2f})", color=INK, fontsize=10, va="center")
ax.set_yticks([2, 1, 0])
ax.set_yticklabels([r[0] for r in rows2], fontsize=10)
ax.set_ylim(-0.6, 3.0)
ax.set_xlim(-3, 7)
ax.set_xlabel("Difference in week-24 HDRS-17 change, MBC minus standard care (points)")
handles = [Line2D([], [], color=ORANGE, lw=LW, ls="--", marker="o", ms=MS,
label="What the reported P < .001 requires"),
Line2D([], [], color=BLUE, lw=LW, marker="o", ms=MS,
label="Recomputed from the paper's own numbers")]
ax.legend(handles=handles, loc="lower left", bbox_to_anchor=(0, 1.0), ncol=2, frameon=False,
fontsize=10, handletextpad=0.3)
save(fig, "fig2")
print("\nfigures written: fig1.svg/png, fig2.svg/png in", HERE)
Abstract reports overall-cohort length-of-stay estimates as the results for hypotensive patients
JAMA Netw Open (2025) · doi:10.1001/jamanetworkopen.2025.33266 · PMC12457976
Summary
The abstract states that, among patients with hypotension on admission, splenic angioembolization (SAE) and observation were each associated with a shorter hospital length of stay (HLOS) than open splenectomy (OS): SAE β = -1.44 (95% CI, -1.79 to -1.09; P < .001) and observation β = -1.41 (95% CI, -1.73 to -1.09; P < .001). These six numbers are identical to the paper's Table 2 rows for the total cohort (n = 7567). Table 2's rows for the hypotension subgroup (SBP <90 mm Hg, n = 651) are different: SAE β = -1.65 (-3.24 to -0.06; P = .04) and observation β = -0.13 (-1.58 to 1.32; P = .86). The Results text reports only the SAE figure for this subgroup. For observation, the abstract therefore reports a highly significant benefit of about 1.4 days where the paper's own subgroup analysis, and its inverse-probability-weighted (IPW) sensitivity analysis, show no significant difference. The claim is repeated in words in the main-text Conclusions, and it supports the headline statement that nonoperative management had "favorable outcomes ... even in patients with hypotension on admission".
What the paper reports
Abstract, Results:
Among patients with hypotension, there was no mortality difference, but shorter hospital length of stay was found in the SAE (β = -1.44; 95% CI, -1.79 to -1.09; P < .001) and observation (β = -1.41; 95% CI, -1.73 to -1.09; P < .001) groups.
Results, "Multivariable Analysis for Other Outcomes". This paragraph gives the same two estimates for the total cohort, then reports the hypotension subgroup separately and names only SAE:
Multivariable linear regression analysis of continuous variable outcomes is given in Table 2. SAE (β = -1.44; 95% CI, -1.79 to -1.09; P < .001) and observation (β = -1.41; 95% CI, -1.73 to -1.09; P < .001) showed a negative association with HLOS days.
The same results were observed in normal SBP subgroup. In the hypotension subgroup, SAE was also negatively associated with HLOS days (β = -1.65; 95% CI, -3.24 to -0.06; P = .04) and ICULOS days (β = -1.45; 95% CI, -2.62 to -0.29; P = .01).
Table 2, "SAE or Observation vs OS Outcomes Using Multivariable Linear Regression and IPW". Columns: outcome | No. | mean (SD) | regression β (95% CI) | P value | IPW β (95% CI) | P value. Total block, HLOS days:
SAE | 1547 | 6.35 (4.17) | -1.44 (-1.79 to -1.09) | <.001 | -1.77 (-2.24 to -1.30) | <.001
Observation | 4521 | 6.04 (5.00) | -1.41 (-1.73 to -1.09) | <.001 | -1.60 (-2.05 to -1.14) | <.001
Table 2, SBP <90 mm Hg block, HLOS days (block total 651; OS row 323 | 9.40 (8.31) | 0 [Reference]):
SAE | 124 | 7.48 (4.80) | -1.65 (-3.24 to -0.06) | .04 | -1.67 (-2.89 to -0.46) | .007
Observation | 204 | 8.74 (8.01) | -0.13 (-1.58 to 1.32) | .86 | -0.48 (-2.00 to 1.04) | .54
Main-text Conclusions:
Furthermore, in patients with hypotension on admission, nonoperative management (either with SAE or observation) was associated with no increased risk of complications or mortality and shorter LOS.
Key Points, "Findings" (the abstract's Conclusions and Relevance uses the same closing phrase):
In this cohort study of 7567 patients, nonoperative management (angioembolization or observation) was associated with favorable outcomes when compared with surgery in isolated severe blunt splenic injury, even in patients with hypotension on admission.
What the paper's own numbers show
Method. I compared the abstract's two "hypotension" estimate-and-CI triples with every HLOS row in Table 2 (total, SBP <90, SBP ≥90 and failure blocks, regression and IPW columns). I then checked each row's internal consistency with a Wald approximation from the printed interval: SE = (upper − lower)/(2 × 1.96), z = β/SE, two-sided P = 2Φ(−|z|). I checked that the block sizes add up, and I compared CI widths with the group sizes. As a further check I computed the unadjusted mean differences and Welch t tests from the Table 2 means and SDs. Patient-level data are not available, so the adjusted models cannot be refitted. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| HLOS β, SAE vs OS, "among patients with hypotension" | Abstract | -1.44 (-1.79 to -1.09), P < .001 | Identical to the Table 2 Total row; matches no SBP <90 row |
| HLOS β, observation vs OS, "among patients with hypotension" | Abstract | -1.41 (-1.73 to -1.09), P < .001 | Identical to the Table 2 Total row; matches no SBP <90 row |
| HLOS β, SAE, hypotension subgroup | Table 2, SBP <90; Results | -1.65 (-3.24 to -0.06), P = .04 | SE 0.811, z = -2.03, P = 0.042: consistent with itself |
| HLOS β, observation, hypotension subgroup | Table 2, SBP <90 | -0.13 (-1.58 to 1.32), P = .86 | SE 0.740, z = -0.18, P = 0.861; CI includes 0 |
| IPW HLOS β, SAE / observation, hypotension subgroup | Table 2, SBP <90, IPW | -1.67 (-2.89 to -0.46), P = .007 / -0.48 (-2.00 to 1.04), P = .54 | P = 0.007 / 0.536; observation CI includes 0 |
| HLOS β, total cohort, SAE / observation | Table 2, Total | -1.44 (-1.79 to -1.09) / -1.41 (-1.73 to -1.09), both P < .001 | z = -8.06 / -8.64, both P < 10⁻¹⁵ |
| HLOS β, normotension subgroup, SAE / observation | Table 2, SBP ≥90 | -1.44 (-1.80 to -1.09) / -1.53 (-1.86 to -1.20) | Close to, but not identical with, the abstract (observation -1.53 vs -1.41) |
| Abstract's failure-subgroup HLOS figures (control) | Abstract vs Table 2, Failure | 2.50 (1.27-3.73); 0.71 (0.07-1.35) | Identical to the Table 2 Failure rows: the abstract's other subgroup result is transcribed correctly |
| Group sizes, HLOS blocks | Table 2 | 7567; 651; 6916 | 1499 + 1547 + 4521 = 7567; 323 + 124 + 204 = 651; 1176 + 1423 + 4317 = 6916; 651 + 6916 = 7567 |
| CI width, abstract vs SBP <90 row | Abstract; Table 2 | 0.70 vs 3.18 (SAE); 0.64 vs 2.90 (observation) | The subgroup intervals are 4.5 times as wide as the abstract's |
| Residual SD implied by each CI, SE/√(1/n_arm + 1/n_OS) | Table 2 | HLOS SD 7.69 (SBP <90), 5.27 (total) | SBP <90 rows: 7.68 and 8.27; total rows: 4.93 and 5.48. The abstract's CIs placed at the SBP <90 group sizes would imply 1.69 and 1.83, so covariates would have to explain about 95% of the HLOS variance in that subgroup, while the table's own subgroup rows show almost no such reduction |
| ICULOS β, observation, hypotension subgroup | Table 2, SBP <90 | -0.49 (-1.59 to 0.61), P = .38 | P = 0.383; CI includes 0 |
| Unadjusted mean HLOS difference vs OS, hypotension subgroup | Table 2 means (SD) | OS 9.40 (8.31), SAE 7.48 (4.80), observation 8.74 (8.01) | SAE -1.92 days (Welch P = 0.003); observation -0.66 days (Welch P = 0.364) |
Each Table 2 row is internally consistent: the printed P values follow from the printed intervals, and the block sizes add to the cohort total. The table is not misaligned. The discrepancy lies between the abstract sentence and the subgroup rows it describes. The CI-width check points the same way. The abstract's intervals are as narrow as one would expect from about 3000-6000 patients per comparison, not from the 447 (SAE) and 527 (observation) patients in the hypotension comparisons.
Figure
Figure 1. The abstract's sentence on patients with hypotension (top, orange outline) quoted next to the two Table 2 blocks it could refer to. All six numbers, for both arms, match the Total-cohort block (n = 7567) exactly. The SBP <90 mm Hg block (n = 651), which is the subgroup the sentence names, gives a different SAE estimate with an interval 4.5 times as wide, and an observation estimate of -0.13 days whose 95% CI includes 0. The blue markers give the P values recomputed from each printed interval with a Wald approximation. They agree with the table's own P values (.04 and .86), so the table rows are internally consistent.
Figure 2. The same estimates on one axis, in days of hospital stay relative to open splenectomy. The orange squares are the abstract's "hypotension" values, the blue circles are the Table 2 SBP <90 rows, and the hollow gray circles are the Table 2 total-cohort rows. The orange and gray intervals coincide. For observation, the subgroup interval (blue) crosses the null line at 0, and the abstract's (orange) does not. The right-hand column gives each printed P value, with the Wald P recomputed from the printed CI for the subgroup rows.
Why it matters
Whether nonoperative management, and observation in particular, is reasonable for hemodynamically unstable patients with high-grade splenic injury is a clinically consequential question. The abstract is the part of the paper most readers rely on. In the hypotension subgroup, Table 2 and the Results show no difference in mortality (HR 0.95 for SAE, 1.08 for observation) or in any complication (OR 0.71, P = .30; OR 1.15, P = .62). That leaves length of stay as the only outcome on which the subgroup shows a benefit. For observation, that benefit is not present in the paper's own subgroup analysis: β = -0.13 days (95% CI, -1.58 to 1.32; P = .86), with IPW giving -0.48 days (P = .54). The unadjusted difference is also small and not significant (-0.66 days, P = .36). The main-text Conclusions ("nonoperative management (either with SAE or observation) was associated with ... shorter LOS") and the Key Points' "favorable outcomes ... even in patients with hypotension on admission" both rest partly on the abstract's version.
For SAE the direction is unchanged, but the abstract overstates the precision. The subgroup estimate is -1.65 days with a CI reaching almost to 0 (-0.06) and P = .04, not P < .001 with a 0.7-day-wide interval.
The paper's other main findings do not depend on this sentence. These are lower morbidity and shorter stay in the overall cohort, the same in the normotension subgroup, no mortality difference, and the failure-subgroup results. A null result does not show that observation has no effect on length of stay in hypotensive patients. It shows only that this analysis does not support the significant effect the abstract reports.
Explanations considered
- The abstract reports a different model or weighting from the table. The abstract's two triples match no other HLOS row in Table 2: not the SBP <90 regression or IPW rows, the SBP ≥90 rows, or the total-cohort IPW rows. They match only the total-cohort regression rows, digit for digit. This does not account for the numbers.
- "Patients with hypotension" means something other than SBP <90 mm Hg. The Methods define the subgroup as "patients presenting with hypotension (SBP <90 mm Hg)", and the total cohort is not a hypotension group. The total cohort is 91% normotensive (6916 of 7567). This does not account for the numbers.
- The table rows are misaligned or mislabeled. Each row's printed P value follows from its own interval (SBP <90 SAE P = 0.042 vs .04 printed; observation 0.861 vs .86; IPW 0.007 and 0.536). The block sizes sum correctly (323 + 124 + 204 = 651). The SBP <90 intervals are as wide as their group sizes and SDs imply. The abstract's intervals would require covariates to remove about 95% of the outcome variance in that subgroup, far more than the table's own subgroup rows show. The Results text also quotes -1.65 for the hypotension subgroup, which confirms the table's SBP <90 values. This does not account for the numbers.
- The supplement holds a different hypotension analysis. Supplement 1 (eAppendix and eTables 1-8) gives only unadjusted HLOS medians for the hypotension subgroup (eTable 4: OS 7, SAE 6, observation 6; three-group P = .003). It reports no pairwise regression coefficients and nothing matching -1.44 or -1.41 for that subgroup. This does not account for the numbers.
- A text-conversion artifact. The abstract sentence, the Results paragraph and Table 2 read the same in the publisher-supplied JATS XML. The values are plain text, not images. This does not account for the numbers.
- The abstract sentence is meant to combine two findings: no mortality difference in hypotension, and shorter stay overall. The sentence attaches the length-of-stay result to "Among patients with hypotension" grammatically. The main-text Conclusions repeat the subgroup claim explicitly for both SAE and observation. Under this reading the abstract would still say nothing correct about length of stay in the subgroup. It would at most reduce the error to ambiguous wording, and the Conclusions sentence would still be unsupported for observation.
Status
As of 2026-10-23:
- Version of record. The publisher page (jamanetwork.com, reached via doi.org) returned a bot check (HTTP 403) to automated access, so it was not read directly. The wording was confirmed in the publisher-supplied JATS full text in Europe PMC (PMC12457976) and in the PMC archive text (read with
src/show.py). Both have the abstract sentence, the Results paragraph and the Table 2 cells as quoted. - Supplements. Supplement 1 (eAppendix, eTables 1-8) was read. It does not report the hypotension-subgroup HLOS regression coefficients and does not change Table 2. Supplement 2 is the data sharing statement (ACS-TQIP registry data, no patient-level data).
- Corrections and notices. None found. Crossref update records, PMC notices and Europe PMC notices for doi:10.1001/jamanetworkopen.2025.33266 are all empty. A web search for a correction or erratum returned none.
- PubPeer. The PubPeer search and API returned a human-verification challenge to automated access, so they could not be queried directly. A web search for the title returned no PubPeer entry. This is not a definitive negative.
- Sources. Article (doi); PMC12457976; Europe PMC full-text XML and supplementary files for PMC12457976; Crossref and Europe PMC REST records for the DOI.
Suggested correction
In the Abstract, Results, replace the hypotension sentence with the Table 2 SBP <90 mm Hg values, for example: "Among patients with hypotension, there was no mortality difference; hospital length of stay was shorter in the SAE group (β = -1.65; 95% CI, -3.24 to -0.06; P = .04) but not significantly different in the observation group (β = -0.13; 95% CI, -1.58 to 1.32; P = .86)." The main-text Conclusions should then be revised: in patients with hypotension, shorter LOS was found for SAE only. The phrase "even in patients with hypotension on admission" in the Key Points and in the Abstract's Conclusions and Relevance should be checked against the subgroup results as corrected. In that subgroup, mortality and complications did not differ, and the LOS benefit applies to SAE only.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12457976 (JAMA Netw Open 2025; doi:10.1001/jamanetworkopen.2025.33266)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates the figures
# (fig1, fig2 and three alternative designs, candidate3-5) as SVG and 200-dpi PNG
# in this script's own folder. Every input below is copied from the article as
# archived in PubMed Central (abstract and Table 2).
import os
import math
import numpy as np
from scipy.stats import norm, ttest_ind_from_stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.ppf(0.975)
# ---------------------------------------------------------------- inputs
# Abstract, Results: "Among patients with hypotension ... SAE (...) and observation (...)"
ABSTRACT = {"SAE": (-1.44, -1.79, -1.09, "<.001"),
"Observation": (-1.41, -1.73, -1.09, "<.001")}
# Table 2, HLOS days rows: (n, mean, SD, beta, lo, hi, P, IPW beta, IPW lo, IPW hi, IPW P)
T2 = {
"Total": {"OS": (1499, 8.38, 6.53),
"SAE": (1547, 6.35, 4.17, -1.44, -1.79, -1.09, "<.001", -1.77, -2.24, -1.30, "<.001"),
"Observation": (4521, 6.04, 5.00, -1.41, -1.73, -1.09, "<.001", -1.60, -2.05, -1.14, "<.001")},
"SBP <90": {"OS": (323, 9.40, 8.31),
"SAE": (124, 7.48, 4.80, -1.65, -3.24, -0.06, ".04", -1.67, -2.89, -0.46, ".007"),
"Observation": (204, 8.74, 8.01, -0.13, -1.58, 1.32, ".86", -0.48, -2.00, 1.04, ".54")},
"SBP >=90": {"OS": (1176, 8.10, 5.93),
"SAE": (1423, 6.25, 4.10, -1.44, -1.80, -1.09, "<.001", -1.80, -2.31, -1.28, "<.001"),
"Observation": (4317, 5.91, 4.77, -1.53, -1.86, -1.20, "<.001", -1.83, -2.30, -1.35, "<.001")},
"Failure": {"OS": (1499, 8.38, 6.53),
"SAE": (105, 10.76, 6.07, 2.50, 1.27, 3.73, "<.001", 2.44, 1.14, 3.75, "<.001"),
"Observation": (608, 8.88, 6.96, 0.71, 0.07, 1.35, ".03", 1.20, 0.19, 2.21, ".02")},
}
# Table 2, SBP <90, ICULOS days (used for the Conclusions' "shorter LOS")
ICU_HYPO = {"SAE": (-1.45, -2.62, -0.29, ".01"), "Observation": (-0.49, -1.59, 0.61, ".38")}
SUBGROUP_HLOS_N = {"Total": 7567, "SBP <90": 651, "SBP >=90": 6916}
SUBGROUP_HLOS_SD = {"Total": 5.27, "SBP <90": 7.69, "SBP >=90": 4.93}
def wald(b, lo, hi):
se = (hi - lo) / (2 * Z975)
z = b / se
return se, z, 2 * norm.sf(abs(z))
def fmt_p(p):
return f"{p:.2g}" if p < 0.001 else f"{p:.3f}"
# ---------------------------------------------------------------- 1. Wald check of every row
print("1. Wald check, SE = (upper - lower) / (2 x 1.96), z = beta / SE, two-sided P")
rows = [("Abstract 'hypotension'", a, *ABSTRACT[a]) for a in ("SAE", "Observation")]
for g in T2:
for a in ("SAE", "Observation"):
r = T2[g][a]
rows.append((f"Table 2 {g} MLR", a, r[3], r[4], r[5], r[6]))
rows.append((f"Table 2 {g} IPW", a, r[7], r[8], r[9], r[10]))
for a in ("SAE", "Observation"):
rows.append(("Table 2 SBP <90 ICULOS MLR", a, *ICU_HYPO[a]))
WALD = {}
for src, arm, b, lo, hi, p in rows:
se, z, pc = wald(b, lo, hi)
WALD[(src, arm)] = (se, z, pc)
print(f" {src:28s} {arm:11s} beta={b:6.2f} CI=({lo:5.2f},{hi:5.2f}) printed P={p:6s} "
f"SE={se:.3f} z={z:6.2f} recomputed P={fmt_p(pc):8s} CI excludes 0: {lo * hi > 0}")
# ---------------------------------------------------------------- 2. which block do the abstract numbers match?
print("\n2. Does the abstract's 'hypotension' triple (beta, lower, upper) equal a Table 2 HLOS row?")
for g in T2:
for col, idx in (("MLR", (3, 4, 5)), ("IPW", (7, 8, 9))):
m = [tuple(T2[g][a][i] for i in idx) == ABSTRACT[a][:3] for a in ("SAE", "Observation")]
print(f" {g:9s} {col}: SAE match={m[0]!s:5s} Observation match={m[1]}")
print(" (control) Abstract failure-subgroup HLOS: SAE 2.50 (1.27-3.73), observation 0.71 (0.07-1.35)"
f" = Table 2 Failure rows: {(T2['Failure']['SAE'][3:6] == (2.50, 1.27, 3.73)) and (T2['Failure']['Observation'][3:6] == (0.71, 0.07, 1.35))}")
# ---------------------------------------------------------------- 3. sample sizes
print("\n3. Sample sizes of the HLOS blocks")
for g in ("Total", "SBP <90", "SBP >=90"):
s = sum(T2[g][a][0] for a in ("OS", "SAE", "Observation"))
print(f" {g:9s}: {T2[g]['OS'][0]} + {T2[g]['SAE'][0]} + {T2[g]['Observation'][0]} = {s} "
f"(printed {SUBGROUP_HLOS_N[g]})")
print(f" 651 + 6916 = {651 + 6916}")
for g in ("Total", "SBP <90"):
for a in ("SAE", "Observation"):
print(f" patients in the {g} {a} vs OS comparison: {T2[g][a][0]} + {T2[g]['OS'][0]} = {T2[g][a][0] + T2[g]['OS'][0]}")
print(f" normotensive share of the total cohort: 6916 / 7567 = {6916 / 7567:.1%}")
# ---------------------------------------------------------------- 4. CI widths
print("\n4. CI widths: abstract vs the hypotension rows")
CIW = {}
for a in ("SAE", "Observation"):
wa = ABSTRACT[a][2] - ABSTRACT[a][1]
r = T2["SBP <90"][a]
wh = r[5] - r[4]
CIW[a] = (wa, wh)
print(f" {a:11s}: abstract width {wa:.2f}; Table 2 SBP <90 width {wh:.2f}; ratio {wh / wa:.1f}")
# unadjusted SE of a difference in means and the residual SD each CI implies
print("\n5. Residual SD implied by each CI, SE / sqrt(1/n_arm + 1/n_OS) (rough: ignores covariates' effect on precision)")
IMPLIED = {}
for g in ("Total", "SBP <90", "SBP >=90"):
for a in ("SAE", "Observation"):
r, os_ = T2[g][a], T2[g]["OS"]
k = math.sqrt(1 / r[0] + 1 / os_[0])
se = wald(r[3], r[4], r[5])[0]
IMPLIED[(g, a)] = (k, se, se / k)
print(f" Table 2 {g:9s} {a:11s}: k={k:.4f} SE={se:.3f} implied residual SD={se / k:.2f} "
f"(block SD of HLOS {SUBGROUP_HLOS_SD[g]})")
for a in ("SAE", "Observation"):
r, os_ = T2["SBP <90"][a], T2["SBP <90"]["OS"]
k = math.sqrt(1 / r[0] + 1 / os_[0])
se = wald(*ABSTRACT[a][:3])[0]
sd_raw = math.sqrt(((r[0] - 1) * r[2] ** 2 + (os_[0] - 1) * os_[2] ** 2) / (r[0] + os_[0] - 2))
r2 = 1 - (se / k) ** 2 / sd_raw ** 2
IMPLIED[("Abstract", a)] = (k, se, se / k)
print(f" Abstract CI placed in the SBP <90 subgroup, {a:11s}: k={k:.4f} SE={se:.3f} implied residual "
f"SD={se / k:.2f}; pooled SD of the two arms {sd_raw:.2f}; covariates would need R^2={r2:.2f}")
# ---------------------------------------------------------------- 6. unadjusted differences
print("\n6. Unadjusted mean HLOS difference vs OS (Table 2 means and SDs), Welch t test")
for g in ("SBP <90", "Total"):
os_ = T2[g]["OS"]
for a in ("SAE", "Observation"):
r = T2[g][a]
t, p = ttest_ind_from_stats(r[1], r[2], r[0], os_[1], os_[2], os_[0], equal_var=False)
print(f" {g:7s} {a:11s}: {r[1]:.2f} - {os_[1]:.2f} = {r[1] - os_[1]:+.2f} days; Welch t={t:.2f}, P={fmt_p(p)}")
# ================================================================ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#8a8984"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "lines.linewidth": 2, "svg.fonttype": "none"})
M = "−" # minus sign
def n(x, d=2):
return f"{x:.{d}f}".replace("-", M)
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), bbox_inches="tight", pad_inches=0.12, **kw)
plt.close(fig)
print(f" wrote {name}.svg and {name}.png")
def box(ax, x, y, w, h, edge, lw=2):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc="#ffffff", ec=edge, lw=lw, transform=ax.transAxes))
print("\n7. Figures")
pS, pO = (T2["SBP <90"]["SAE"], T2["SBP <90"]["Observation"])
tS, tO = (T2["Total"]["SAE"], T2["Total"]["Observation"])
wS = WALD[("Table 2 SBP <90 MLR", "SAE")]
wO = WALD[("Table 2 SBP <90 MLR", "Observation")]
# ---------------- fig1: text panels (design 1)
fig = plt.figure(figsize=(7.2, 4.8))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.autoscale(False)
box(ax, 0.02, 0.72, 0.96, 0.26, ORANGE)
ax.text(0.04, 0.95, "Abstract, Results", fontsize=11, weight="bold", va="top")
ax.text(0.04, 0.89,
"\u201cAmong patients with hypotension, there was no mortality difference, but shorter\n"
"hospital length of stay was found in the SAE (\u03b2 = " + n(-1.44) + "; 95% CI, " + n(-1.79) + " to " + n(-1.09) +
"; P < .001)\nand observation (\u03b2 = " + n(-1.41) + "; 95% CI, " + n(-1.73) + " to " + n(-1.09) +
"; P < .001) groups.\u201d", fontsize=10.5, va="top", linespacing=1.45)
for x0, title, lines in (
(0.02, "Table 2, Total cohort, n = 7567",
[("SAE", f"{n(tS[3])} ({n(tS[4])} to {n(tS[5])}), P < .001", None),
("Observation", f"{n(tO[3])} ({n(tO[4])} to {n(tO[5])}), P < .001", None)]),
(0.52, "Table 2, SBP <90 mm Hg, n = 651",
[("SAE", f"{n(pS[3])} ({n(pS[4])} to {n(pS[5])}), P = .04", f"recomputed P = {wS[2]:.3f}"),
("Observation", f"{n(pO[3])} ({n(pO[4])} to {n(pO[5])}), P = .86",
f"recomputed P = {wO[2]:.2f}; CI includes 0")])):
box(ax, x0, 0.095, 0.46, 0.395, GRAY)
ax.text(x0 + 0.02, 0.465, title, fontsize=11, weight="bold", va="top")
ax.text(x0 + 0.02, 0.41, "HLOS \u03b2 vs open splenectomy, days (95% CI)", fontsize=10, color=INK2, va="top")
for i, (arm, est, rec) in enumerate(lines):
y = 0.355 - i * 0.13
ax.text(x0 + 0.02, y, arm, fontsize=10.5, va="top", weight="bold")
ax.text(x0 + 0.02, y - 0.042, est, fontsize=10.5, va="top")
if rec:
ax.plot([x0 + 0.03], [y - 0.1], "o", ms=8, color=BLUE, clip_on=False)
ax.text(x0 + 0.05, y - 0.1, rec, fontsize=10, color=INK2, va="center")
# connectors
ax.annotate("", xy=(0.25, 0.495), xytext=(0.25, 0.715), xycoords="data",
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=2))
ax.text(0.27, 0.605, "identical, digit for digit\n(all six numbers)", fontsize=10, va="center", color=INK)
ax.annotate("", xy=(0.75, 0.495), xytext=(0.75, 0.715), xycoords="data",
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, linestyle=(0, (4, 3))))
ax.text(0.77, 0.605, "the subgroup the sentence\nnames: numbers differ", fontsize=10, va="center", color=INK)
# legend
h = [Line2D([], [], color=ORANGE, lw=2, label="claim as reported in the abstract"),
Line2D([], [], marker="o", ms=8, color=BLUE, lw=0, label="P recomputed from the printed 95% CI")]
ax.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, -0.005), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# ---------------- fig2: forest plot on one axis (design 2)
def forest(ax, entries, xlim):
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for y, (lab, b, lo, hi, style, note) in entries.items():
col = {"abs": ORANGE, "sub": BLUE, "tot": GRAY}[style]
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="round", zorder=2)
ax.plot([b], [y], "s" if style == "abs" else "o", ms=9, color=col,
mfc=col if style != "tot" else BG, mew=2, zorder=3)
ax.text(xlim[1] + 0.08, y, note, va="center", fontsize=10, color=INK, clip_on=False)
ax.set_yticks(list(entries.keys()), [e[0] for e in entries.values()], fontsize=10.5, color=INK)
ax.tick_params(axis="y", length=0)
ax.set_xlim(*xlim)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
fig, ax = plt.subplots(figsize=(7.2, 4.3))
ent = {}
y = 7.6
for arm, sub, tot in (("SAE", pS, tS), ("Observation", pO, tO)):
a = ABSTRACT[arm]
w = WALD[("Table 2 SBP <90 MLR", arm)]
ent[y] = (f"{arm}: abstract, ‘hypotension’", a[0], a[1], a[2], "abs", "P < .001")
ent[y - 1] = (f"{arm}: Table 2, SBP <90 (n = 651)", sub[3], sub[4], sub[5], "sub",
f"P = {sub[6]} (recomputed {w[2]:.3f})")
ent[y - 2] = (f"{arm}: Table 2, total (n = 7567)", tot[3], tot[4], tot[5], "tot", "P < .001")
y -= 4
forest(ax, ent, (-3.6, 1.6))
ax.set_ylim(0.9, 8.3)
ax.set_xlabel("Adjusted difference in hospital length of stay vs open splenectomy, days (β, 95% CI)")
ax.text(0.05, 8.25, "longer stay →", fontsize=10, color=INK2, va="bottom")
ax.text(-0.05, 8.25, "← shorter stay", fontsize=10, color=INK2, va="bottom", ha="right")
h = [Line2D([], [], marker="s", ms=9, color=ORANGE, lw=2, label="abstract (reported)"),
Line2D([], [], marker="o", ms=9, color=BLUE, lw=2, label="Table 2 hypotension rows, P recomputed from CI"),
Line2D([], [], marker="o", ms=9, color=GRAY, mfc=BG, mew=2, lw=2, label="Table 2 total cohort")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.32, -0.16), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
# ---------------- candidate3: z statistics on one axis (design 3)
fig, ax = plt.subplots(figsize=(7.2, 3.3))
ax.axvspan(-1.96, 1.96, color="#efeeea", zorder=0)
ax.axvline(0, color=INK2, lw=0.8)
ax.text(0, 4.75, "|z| < 1.96: P > .05", ha="center", fontsize=10, color=INK2,
bbox=dict(fc="#efeeea", ec="none", pad=1))
yl = []
for i, arm in enumerate(("SAE", "Observation")):
za = WALD[("Abstract 'hypotension'", arm)][1]
zs = WALD[("Table 2 SBP <90 MLR", arm)][1]
y0 = 3.6 - i * 2.2
ax.plot([za], [y0], "s", ms=10, color=ORANGE)
ax.text(za + 0.25, y0, f"z = {n(za)}", va="center", fontsize=10)
ax.plot([zs], [y0 - 0.8], "o", ms=10, color=BLUE)
ax.text(zs + 0.3, y0 - 0.8, f"z = {n(zs)}, P = {WALD[('Table 2 SBP <90 MLR', arm)][2]:.2f}", va="center", fontsize=10,
bbox=dict(fc="#efeeea", ec="none", pad=1), zorder=4)
yl += [(y0, f"{arm}: abstract"), (y0 - 0.8, f"{arm}: Table 2, SBP <90")]
ax.set_yticks([a for a, _ in yl], [b for _, b in yl], fontsize=10.5, color=INK)
ax.tick_params(axis="y", length=0)
ax.set_xlim(-9.5, 4.5); ax.set_ylim(0.3, 5.1)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.set_xlabel("Wald z = β / SE, with SE = (upper − lower) / 3.92 from each printed 95% CI")
h = [Line2D([], [], marker="s", ms=10, color=ORANGE, lw=0, label="from the abstract's CI (reported)"),
Line2D([], [], marker="o", ms=10, color=BLUE, lw=0, label="from the Table 2 hypotension CI (recomputed)")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.45, -0.22), ncol=2, frameon=False, fontsize=10)
save(fig, "candidate3")
# ---------------- candidate4: lookup grid (design 4)
fig = plt.figure(figsize=(7.2, 3.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.autoscale(False)
cols = [0.03, 0.36, 0.68]
ax.text(cols[0], 0.93, "Table 2 HLOS row", weight="bold", fontsize=10.5)
ax.text(cols[1], 0.93, "SAE β (95% CI)", weight="bold", fontsize=10.5)
ax.text(cols[2], 0.93, "Observation β (95% CI)", weight="bold", fontsize=10.5)
ax.text(cols[0], 0.82, "Abstract, ‘hypotension’", fontsize=10.5, color=INK)
for c, arm in ((cols[1], "SAE"), (cols[2], "Observation")):
a = ABSTRACT[arm]
ax.plot([c - 0.012], [0.835], "s", ms=8, color=ORANGE, transform=ax.transAxes)
ax.text(c + 0.005, 0.82, f"{n(a[0])} ({n(a[1])} to {n(a[2])})", fontsize=10.5)
ax.plot([0.02, 0.98], [0.78, 0.78], color=INK2, lw=0.8, transform=ax.transAxes)
y = 0.70
for g, lab in (("Total", "Total, n = 7567"), ("SBP <90", "SBP <90, n = 651"), ("SBP >=90", "SBP ≥90, n = 6916")):
for col, idx, cl in (("MLR", (3, 4, 5), "regression"), ("IPW", (7, 8, 9), "IPW")):
ax.text(cols[0], y, f"{lab}, {cl}", fontsize=10.5, color=INK if g != "SBP <90" else INK, weight="bold" if g == "SBP <90" else "normal")
for c, arm in ((cols[1], "SAE"), (cols[2], "Observation")):
r = T2[g][arm]
tri = tuple(r[i] for i in idx)
same = tri == ABSTRACT[arm][:3]
ax.text(c + 0.005, y, f"{n(tri[0])} ({n(tri[1])} to {n(tri[2])})", fontsize=10.5)
if same:
ax.add_patch(FancyBboxPatch((c - 0.01, y - 0.025), 0.28, 0.075, boxstyle="round,pad=0,rounding_size=0.01",
fc="none", ec=ORANGE, lw=2, transform=ax.transAxes))
y -= 0.1
ax.text(0.03, 0.06, "Orange outline: identical to the abstract\u2019s \u2018hypotension\u2019 numbers. Only the Total-cohort\n"
"regression rows match; the SBP <90 rows (bold), which the sentence describes, do not.", fontsize=10, color=INK2)
save(fig, "candidate4")
# ---------------- candidate5: CI width vs sample size (design 5)
fig, ax = plt.subplots(figsize=(7.2, 4.0))
ax.grid(color=GRID, lw=0.8); ax.set_axisbelow(True)
labels = {("Total", "SAE"): "Total SAE", ("Total", "Observation"): "Total Obs",
("SBP >=90", "SAE"): "SBP≥90 SAE", ("SBP >=90", "Observation"): "SBP≥90 Obs",
("SBP <90", "SAE"): "SBP<90 SAE", ("SBP <90", "Observation"): "SBP<90 Obs"}
off = {("Total", "SAE"): (6, 6), ("Total", "Observation"): (6, -12), ("SBP >=90", "SAE"): (-70, 6),
("SBP >=90", "Observation"): (-70, -12), ("SBP <90", "SAE"): (8, 0), ("SBP <90", "Observation"): (8, -4)}
for key, lab in labels.items():
k, se, _ = IMPLIED[key]
ax.plot([k], [2 * Z975 * se], "o", ms=9, color=BLUE, zorder=3)
if key[0] == "SBP <90":
ax.annotate(lab, (k, 2 * Z975 * se), xytext=off[key], textcoords="offset points", fontsize=10, va="center")
ax.annotate("Total and SBP \u226590 rows\n(SAE and observation, 4 points)", (0.035, 0.72), xytext=(0.035, 1.35),
ha="center", fontsize=10, arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
for arm, dy in (("SAE", 8), ("Observation", -12)):
k, se, _ = IMPLIED[("Abstract", arm)]
ax.plot([k], [2 * Z975 * se], "s", ms=9, color=ORANGE, zorder=3)
ax.annotate(f"abstract ‘hypotension’ {arm}", (k, 2 * Z975 * se), xytext=(8, dy),
textcoords="offset points", fontsize=10, va="center")
ax.set_xlim(0, 0.13); ax.set_ylim(0, 3.5)
ax.set_xlabel(r"$\sqrt{1/n_{\rm arm} + 1/n_{\rm OS}}$, from the Table 2 group sizes")
ax.set_ylabel("Width of the printed 95% CI, days")
for s in ("top", "right"):
ax.spines[s].set_visible(False)
h = [Line2D([], [], marker="o", ms=9, color=BLUE, lw=0, label="Table 2 HLOS rows (width from the printed CI)"),
Line2D([], [], marker="s", ms=9, color=ORANGE, lw=0, label="abstract CI, placed at the hypotension group sizes")]
ax.legend(handles=h, loc="upper left", frameon=False, fontsize=10)
save(fig, "candidate5")
Risk-difference confidence intervals for the primary outcome are about 23 to 31 times too narrow
JAMA Netw Open (2026) · doi:10.1001/jamanetworkopen.2026.25965 · PMC13434869
Summary
The Abstract and the Results report the trial's primary-outcome effect sizes, the absolute differences in first-pass intubation success between direct laryngoscopy (DL), Macintosh-blade video laryngoscopy (VLM) and hyperangulated video laryngoscopy (VLH), with 95% confidence intervals 0.22 to 0.33 percentage points wide, for example 4.66 (95% CI, 4.45-4.78) for VLM vs DL (written "DL vs VLM" in the paper). The paper's own Table 2 counts (625/799, 673/812, 711/812) reproduce every point estimate and, to within 0.03, the paper's own z statistics (exactly with a pooled standard error), but with the method the paper names (an unpooled z approach), they give intervals 6.9 to 7.7 points wide: 0.80 to 8.52, 5.69 to 12.99 and 1.24 to 8.12. The numbers needed to treat in the Discussion carry the same narrow intervals (VLM 21.5, reported 20.9-22.2; from the counts, 11.7-125.2). The direction and statistical significance of every comparison are unchanged, but the published intervals overstate the precision of the trial's headline result 23- to 31-fold. That matters most for VLM, where the data are compatible with a gain of anywhere from 0.8 to 8.5 points.
What the paper reports
Differences between success rates were tested applying the z test for unpooled variance and reported as absolute differences with corresponding 95% CIs.
Abstract, Main Outcomes and Measure
Unadjusted absolute risk differences for first-pass intubation success were 4.66 (95% CI, 4.45-4.78) percentage points for DL vs VLM, 9.34 (95% CI, 9.22-9.46) percentage points for DL vs VLH, and 4.68 (95% CI, 4.57-4.79) percentage points for VLM vs VLH.
Abstract, Results
In 2-sided z tests, VLM and VLH were significantly more effective in facilitating first-pass success than the DL (VLM vs DL: z = 2.36, P = .02; VLH vs DL: z = 4.98, P < .001). VLH was in turn superior to VLM (z = 2.66; P = .008). Unadjusted absolute risk differences for a successful first attempt were 4.66 (95% CI, 4.45-4.78) percentage points for DL vs VLM, 9.34 (95% CI, 9.22-9.46) percentage points for DL vs VLH, and 4.68 (95% CI, 4.57-4.79) percentage points for VLM vs VLH.
Results, Primary End Point and Crossover Between Groups
Relative risks (RRs) for successful first-attempt intubation were significantly higher for both VLM and VLH over DL (VLM vs DL: RR, 1.06 [95% CI, 1.01-1.11]; VLH vs DL: RR, 1.12 [95% CI, 1.07-1.17]) and for VLM over VLH (RR, 1.06 [95% CI, 1.01-1.10]) (eTable 2 in Supplement 1).
Results, same paragraph
Numbers needed to treat were 21.5 (95% CI, 20.9-22.2) for VLM and 10.7 (95% CI, 10.6-10.8) for VLH.
Discussion
Table 2 ("First-Pass Success Rates of Tracheal Intubations Using Different Laryngoscopes"), mITT population, row "Yes": DL 625 (78.2), VLM 673 (82.9), VLH 711 (87.6); row "Total": 799, 812, 812.
What the paper's own numbers show
Method. For each pair of arms I took the Table 2 counts and computed the risk difference, its unpooled standard error √(p₁(1−p₁)/n₁ + p₂(1−p₂)/n₂), and the 95% Wald interval RD ± 1.96·SE, which is the interval that goes with the unpooled z test the paper says it used. As checks I also computed the Newcombe (hybrid Wilson score) and Agresti–Caffo intervals, the z statistics (unpooled and pooled), the log-scale Wald interval for the relative risk, and the NNT interval obtained by inverting the risk-difference limits. Finally, I worked backwards from the reported intervals: dividing a reported width by 2 × 1.96 gives the standard error that interval implies, and dividing the reported risk difference by the reported z gives a second implied standard error. Comparisons are labelled higher-success arm vs lower-success arm (VLM vs DL, VLH vs DL, VLH vs VLM); the paper writes them as DL vs VLM, DL vs VLH and VLM vs VLH, with the same positive values. All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| First-pass success, DL / VLM / VLH | Table 2 | 625/799 (78.2), 673/812 (82.9), 711/812 (87.6) | 78.22%, 82.88%, 87.56% |
| RD, VLM vs DL (pp) | Abstract; Results | 4.66 | 4.66 |
| 95% CI, VLM vs DL | Abstract; Results | 4.45 to 4.78 (width 0.33) | Wald 0.80 to 8.52 (width 7.72); Newcombe 0.79 to 8.52; Agresti–Caffo 0.79 to 8.51 |
| RD, VLH vs DL (pp) | Abstract; Results | 9.34 | 9.34 |
| 95% CI, VLH vs DL | Abstract; Results | 9.22 to 9.46 (width 0.24) | Wald 5.69 to 12.99 (width 7.31); Newcombe 5.68 to 12.99; Agresti–Caffo 5.66 to 12.97 |
| RD, VLH vs VLM (pp) | Abstract; Results | 4.68 | 4.68 |
| 95% CI, VLH vs VLM | Abstract; Results | 4.57 to 4.79 (width 0.22) | Wald 1.24 to 8.12 (width 6.89); Newcombe 1.23 to 8.13; Agresti–Caffo 1.22 to 8.12 |
| Recomputed width ÷ reported width | derived | — | 23.4×, 30.4×, 31.3× |
| z, VLM vs DL | Results | 2.36 (P = .02) | unpooled 2.37 (P = .018); pooled 2.36 |
| z, VLH vs DL | Results | 4.98 (P < .001) | unpooled 5.01 (P = 5.4 × 10⁻⁷); pooled 4.98 |
| z, VLH vs VLM | Results | 2.66 (P = .008) | unpooled 2.66 (P = .0077); pooled 2.66 |
| SE implied by reported RD ÷ reported z (pp) | derived | 1.975, 1.876, 1.759 | unpooled SE from counts 1.970, 1.864, 1.757 |
| SE implied by reported CI width ÷ 3.92 (pp) | derived | 0.084, 0.061, 0.056 | — |
| z that the reported CIs would imply | derived | — | 55.4, 152.6, 83.4 |
| Per-arm sample size at which a Wald CI would be as narrow as reported | derived | — | about 441,000, 746,000 and 796,000 (trial: 799–812) |
| Reported CI, distance below / above point | Abstract; Results | VLM vs DL −0.21 / +0.12; others −0.12 / +0.12 and −0.11 / +0.11 | Wald is symmetric (±3.86, ±3.65, ±3.44) |
| RR, VLM vs DL | Results; eTable 2 | 1.06 (1.01–1.11) | 1.060 (1.010–1.112) |
| RR, VLH vs DL | Results; eTable 2 | 1.12 (1.07–1.17) | 1.119 (1.070–1.171) |
| RR, VLH vs VLM | Results; eTable 2 | 1.06 (1.01–1.10) | 1.056 (1.014–1.100) |
| NNT, VLM vs DL | Discussion | 21.5 (20.9–22.2) | 21.5 (11.7–125.2) |
| NNT, VLH vs DL | Discussion | 10.7 (10.6–10.8) | 10.7 (7.7–17.6) |
| NNT limits from inverting the reported RD CIs | derived | — | VLM 20.9–22.5; VLH 10.6–10.8 |
Two independent routes give the same standard errors. The counts give 1.97, 1.86 and 1.76 points, and the paper's own risk differences divided by its own z statistics give 1.98, 1.88 and 1.76 points. The reported intervals imply standard errors of 0.06 to 0.08 points, about 23 to 31 times smaller. The relative-risk intervals in the same paragraph and in eTable 2 match the counts to the printed decimal, so the precision the paper reports on the ratio scale agrees with the wide intervals, not the narrow ones. The NNT intervals match the reciprocals of the reported risk-difference limits (exactly for VLH; for VLM the upper limit is 22.2 against 22.5), so they inherit the same error.
Figure
Figure 1. Each row is one of the three primary-outcome comparisons. Orange diamonds show the risk difference and 95% CI reported in the Abstract and Results. The reported intervals are narrower than the marker, so the bars are hidden behind it; the right-hand labels give their limits. Blue circles with capped bars show the 95% CI recomputed from the Table 2 counts with an unpooled Wald interval, the method the paper states. The solid gray line marks no difference and the dashed line marks the −5 point noninferiority margin. Every recomputed interval lies to the right of both, so significance and noninferiority are unchanged, but each is 23 to 31 times wider than reported. The axis is linear and includes zero.
Figure 2. Numbers needed to treat with VLM or VLH instead of DL to gain one additional first-pass success, as reported in the Discussion (orange diamonds) and as obtained by inverting the recomputed Wald risk-difference limits (blue circles). The axis is logarithmic because the NNT is a reciprocal. For VLM, the interval the counts support runs from about 12 to about 125 intubations, against a reported 20.9 to 22.2.
Why it matters
- The headline result. First-pass success is the primary outcome, and these intervals are the only interval estimates for it on the absolute scale in the Abstract. As printed, they say the VLM gain is known to lie between 4.45 and 4.78 points. The data support 0.8 to 8.5 points. The lower end is a small effect.
- Clinical translation. The NNT is the quantity a clinician or guideline panel would use. The published VLM interval (20.9–22.2) pins it near 21; the counts allow anything from about 12 to about 125. For VLH the corrected range is 7.7 to 17.6 instead of 10.6 to 10.8.
- Comparison between video blades. VLH vs VLM (the paper's "VLM vs VLH") is reported as 4.68 (4.57–4.79). The corrected interval, 1.24 to 8.12, still excludes zero, but the gap between the two blades is far less precisely determined than printed.
- Reuse. Meta-analyses and decision models that take the published interval, or back-calculate a standard error from it, would give this trial roughly 550 to 980 times its correct inverse-variance weight on the risk-difference scale (the square of the width ratios).
- What does not change. The point estimates, the ordering VLH > VLM > DL, the P values, the relative risks, the noninferiority conclusions and the superiority conclusions all stand. The Conclusions do not quote the intervals.
Explanations considered
- Different analysis population. The point estimates, the z statistics and the relative-risk intervals all match the mITT counts in Table 2, so the narrow intervals refer to the same 2423 patients. The per-protocol set (eTable 1: 590/758, 663/799, 704/804) is smaller and gives similarly wide intervals: 5.14 (1.20 to 9.08), 9.73 (5.99 to 13.46) and 4.58 (1.12 to 8.05). It does not account for the numbers.
- A different interval method. Wald, Newcombe score and Agresti–Caffo intervals all come out 6.9 to 7.7 points wide. A pooled standard error changes the z values only in the second decimal. No standard interval for about 800 patients per arm is 0.2 to 0.3 points wide; that width would need roughly 440,000 to 800,000 patients per arm.
- An adjusted or model-based estimate. The intervals are labelled "unadjusted" and sit next to unadjusted point estimates that equal the raw differences. Adjustment for site or covariates would not reduce the standard error 23- to 31-fold in a trial of this size, and the site-random-effect model is reported separately (eTable 3).
- A different confidence level. The text says "95% CI" each time. At the standard errors the counts imply, intervals of the reported widths would have confidence levels of about 5% to 7%, which no report uses.
- Units (proportion vs percentage points). All three quantities are labelled percentage points, the point estimates are correct in percentage points, and the reported limits are not the Wald limits divided or multiplied by 100. Unit confusion does not reproduce them.
- Conversion or typesetting artifact. The same intervals appear in the Abstract and, word for word, in the Results, as running text rather than table cells, in the archived JATS XML and, as checked earlier in this review, in the publisher's HTML. No dropped digit, sign or decimal mark turns 0.80–8.52 into 4.45–4.78.
- An arithmetic route that would reproduce the printed limits. I did not find one that reproduces all three intervals, and the first interval is asymmetric about its point estimate (−0.21 / +0.12), which a Wald interval cannot be. The write-up therefore states only that the printed intervals are inconsistent with the paper's own counts, z statistics and stated method; it does not say how they were produced.
Two smaller observations, which do not affect the main finding:
- The Results give the same z statistics (2.36 and 4.98) for the 1-sided noninferiority tests as for the 2-sided superiority tests. A noninferiority z against the stated −5 point margin, using the unpooled standard error, would be 4.90 (VLM) and 7.69 (VLH). Both conclusions hold either way. The reported z values match the pooled-variance statistic against zero to the printed decimal (2.36, 4.98, 2.66), while the unpooled values are 2.37, 5.01 and 2.66.
- The Results describe the third relative risk as "for VLM over VLH (RR, 1.06 ...)"; 1.06 is the ratio of VLH to VLM (87.56 / 82.88 = 1.056), and eTable 2 labels it "VLH vs. VLM".
Status
As of 2026-10-23:
- Version of record. The JAMA Network Open full text (via doi.org) carries the same intervals in the Abstract and Results, the same z statistics, the same Table 2 counts and the same NNT intervals as the PubMed Central archive (checked in an earlier step of this review; the publisher PDF was not retrievable). The quotes above were checked verbatim against the PMC-archived text.
- Supplements. Supplement 1 (eMethods, eFigure, eTables 1–8) was read; it gives no confidence intervals for the risk differences. Its eTable 2 relative-risk intervals match the counts (see above), and eTable 1 gives the per-protocol counts used above. Supplement 2 (protocol and statistical analysis plan) specifies one-sided and two-sided z tests with unpooled variance and a 95% interval for the difference judged against the −5 point margin; it does not describe a method that would give narrower intervals. Supplement 4 (data sharing statement) is covered under Data below. Supplement 3 (collaborators) was not relevant.
- Corrections and retractions. Crossref, PubMed Central and Europe PMC list no correction, erratum, expression of concern or retraction for this DOI.
- PubPeer. Web searches for the title and trial name found no PubPeer entry; PubPeer's own search endpoint was not reachable from this machine (HTTP 403), so its absence is not confirmed directly.
- Related content. PubMed links a commentary on the trial (PMID 42545704); it was not reviewed here.
- Data. The registration (NCT05228288) lists IPD sharing as No, but the article's Data Sharing Statement (Supplement 4) says deidentified participant data and a data dictionary are available from the corresponding author for non-commercial use. They were not requested for this check, so the Table 2 counts are its basis; with those data the journal or the authors could recompute the intervals directly.
Sources: the article in PubMed Central (PMC13434869) and at doi:10.1001/jamanetworkopen.2026.25965; Supplements 1, 2 and 4 from the Europe PMC supplementary files; ClinicalTrials.gov NCT05228288; Crossref and Europe PMC metadata.
Suggested correction
Replace the risk-difference intervals in the Abstract and in the Results (Primary End Point and Crossover Between Groups) with intervals computed from the Table 2 counts by the stated method. With the unpooled Wald interval these are:
- VLM vs DL: 4.66 (95% CI, 0.80–8.52) percentage points
- VLH vs DL: 9.34 (95% CI, 5.69–12.99) percentage points
- VLH vs VLM: 4.68 (95% CI, 1.24–8.12) percentage points
and replace the Discussion's NNT intervals with 21.5 (95% CI, 11.7–125.2) for VLM and 10.7 (95% CI, 7.7–17.6) for VLH. If a score-based interval is preferred, the Newcombe limits (0.79–8.52, 5.68–12.99, 1.23–8.13) differ only in the second decimal. Optionally, the authors could also report the noninferiority z statistics against the −5 point margin, state whether the reported z statistics used pooled or unpooled variance, and change "for VLM over VLH" to "for VLH over VLM".
Recomputation script (python recompute.py.txt)
# Recomputation for PMC13434869 (JAMA Netw Open 2026; doi:10.1001/jamanetworkopen.2026.25965)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1/fig2 (SVG + PNG)
# in the folder that holds this script. Inputs are the paper's own numbers only.
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975) # 1.959964
# ---- Inputs from the paper -------------------------------------------------
# Table 2 (mITT): first-pass success, No./total
mitt = {"DL": (625, 799), "VLM": (673, 812), "VLH": (711, 812)}
# eTable 1 (Supplement 1, per-protocol): first-pass success
pp = {"DL": (590, 758), "VLM": (663, 799), "VLH": (704, 804)}
# Abstract and Results: risk differences (pp) with 95% CI; Results: z statistics
# keys are (lower-success arm, higher-success arm)
reported = {
("DL", "VLM"): dict(rd=4.66, lo=4.45, hi=4.78, z=2.36, label="VLM vs DL"),
("DL", "VLH"): dict(rd=9.34, lo=9.22, hi=9.46, z=4.98, label="VLH vs DL"),
("VLM", "VLH"): dict(rd=4.68, lo=4.57, hi=4.79, z=2.66, label="VLH vs VLM"),
}
# Results / eTable 2: relative risks (mITT and per-protocol)
rr_rep = {("DL", "VLM"): (1.06, 1.01, 1.11), ("DL", "VLH"): (1.12, 1.07, 1.17),
("VLM", "VLH"): (1.06, 1.01, 1.10)}
rr_rep_pp = {("DL", "VLM"): (1.07, 1.01, 1.12), ("DL", "VLH"): (1.12, 1.07, 1.18),
("VLM", "VLH"): (1.06, 1.01, 1.10)}
# Discussion: numbers needed to treat (vs DL)
nnt_rep = {"VLM": (21.5, 20.9, 22.2), "VLH": (10.7, 10.6, 10.8)}
NI_MARGIN = -0.05 # Methods: noninferiority margin, 5 percentage points
def wilson(x, n, z=Z):
p = x / n
c = (p + z * z / (2 * n)) / (1 + z * z / n)
h = z * np.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / (1 + z * z / n)
return c - h, c + h
def newcombe(xb, nb, xa, na):
"""Newcombe hybrid score interval for pb - pa."""
pb, pa = xb / nb, xa / na
l1, u1 = wilson(xb, nb)
l2, u2 = wilson(xa, na)
d = pb - pa
return d - np.sqrt((pb - l1) ** 2 + (u2 - pa) ** 2), d + np.sqrt((u1 - pb) ** 2 + (pa - l2) ** 2)
def agresti_caffo(xb, nb, xa, na):
pb, pa = (xb + 1) / (nb + 2), (xa + 1) / (na + 2)
d = pb - pa
se = np.sqrt(pb * (1 - pb) / (nb + 2) + pa * (1 - pa) / (na + 2))
return d - Z * se, d + Z * se
def compare(data, a, b):
xa, na = data[a]
xb, nb = data[b]
pa, pb = xa / na, xb / nb
rd = pb - pa
se = np.sqrt(pa * (1 - pa) / na + pb * (1 - pb) / nb) # unpooled (stated method)
ppool = (xa + xb) / (na + nb)
se_pool = np.sqrt(ppool * (1 - ppool) * (1 / na + 1 / nb)) # pooled, for comparison
rr = pb / pa
se_lrr = np.sqrt(1 / xb - 1 / nb + 1 / xa - 1 / na)
return dict(pa=pa, pb=pb, rd=rd, se=se, se_pool=se_pool,
wald=(rd - Z * se, rd + Z * se), newc=newcombe(xb, nb, xa, na),
ac=agresti_caffo(xb, nb, xa, na),
z=rd / se, z_pool=rd / se_pool, z_ni=(rd - NI_MARGIN) / se,
rr=rr, rr_ci=(rr * np.exp(-Z * se_lrr), rr * np.exp(Z * se_lrr)))
print("=== Proportions (Table 2, mITT) ===")
for k, (x, n) in mitt.items():
print(f" {k}: {x}/{n} = {100 * x / n:.2f}%")
res = {}
print("\n=== Risk differences, mITT (percentage points) ===")
for (a, b), r in reported.items():
c = compare(mitt, a, b)
res[(a, b)] = c
rep_w = r["hi"] - r["lo"]
w = 100 * (c["wald"][1] - c["wald"][0])
se_rep_ci = rep_w / (2 * Z) # SE implied by reported CI width (pp)
se_rep_z = r["rd"] / r["z"] # SE implied by reported RD and z (pp)
print(f"{r['label']}: p = {100*c['pa']:.2f}% vs {100*c['pb']:.2f}%")
print(f" RD reported {r['rd']:.2f} recomputed {100*c['rd']:.2f}")
print(f" 95% CI reported {r['lo']:.2f} to {r['hi']:.2f} (width {rep_w:.2f})")
print(f" Wald (unpooled) {100*c['wald'][0]:.2f} to {100*c['wald'][1]:.2f} (width {w:.2f})")
print(f" Newcombe {100*c['newc'][0]:.2f} to {100*c['newc'][1]:.2f}")
print(f" Agresti-Caffo {100*c['ac'][0]:.2f} to {100*c['ac'][1]:.2f}")
print(f" Recomputed width / reported width = {w / rep_w:.1f}x")
print(f" Reported CI distances from point: -{r['rd']-r['lo']:.2f} / +{r['hi']-r['rd']:.2f}")
print(f" SE: recomputed {100*c['se']:.3f} pp; implied by reported z {se_rep_z:.3f} pp; "
f"implied by reported CI {se_rep_ci:.4f} pp")
print(f" z implied by reported CI = {r['rd']/se_rep_ci:.1f}")
print(f" z superiority: reported {r['z']:.2f}; unpooled {c['z']:.2f} "
f"(2-sided P = {2*stats.norm.sf(c['z']):.2g}, 1-sided P = {stats.norm.sf(c['z']):.2g}); "
f"pooled {c['z_pool']:.2f}")
print(f" z noninferiority vs -5 pp margin (unpooled) = {c['z_ni']:.2f}")
rr = rr_rep[(a, b)]
print(f" RR reported {rr[0]:.2f} ({rr[1]:.2f}-{rr[2]:.2f}); recomputed "
f"{c['rr']:.3f} ({c['rr_ci'][0]:.3f}-{c['rr_ci'][1]:.3f})")
# sample size per arm at which the Wald CI would be as narrow as reported
n_needed = (np.mean([mitt[a][1], mitt[b][1]])) * (100 * c['se'] / se_rep_ci) ** 2
print(f" Per-arm n for a CI this narrow at these rates: about {n_needed:,.0f}")
print("\n=== Further derived quantities ===")
for (a, b), r in reported.items():
c = res[(a, b)]
w_ratio = 100 * (c["wald"][1] - c["wald"][0]) / (r["hi"] - r["lo"])
half = (r["hi"] - r["lo"]) / 2
conf = 2 * stats.norm.cdf(half / (100 * c["se"])) - 1
print(f" {r['label']}: Wald half-width +/-{100*Z*c['se']:.2f} pp; inverse-variance weight ratio "
f"(reported/correct) = {w_ratio**2:.0f}x; confidence level of an interval of the reported "
f"width at the true SE = {100*conf:.1f}%")
print("\n=== Per-protocol (eTable 1): risk differences and RRs ===")
for (a, b), r in reported.items():
c = compare(pp, a, b)
rr = rr_rep_pp[(a, b)]
print(f"{r['label']}: RD {100*c['rd']:.2f} pp, Wald {100*c['wald'][0]:.2f} to {100*c['wald'][1]:.2f}; "
f"RR {c['rr']:.3f} ({c['rr_ci'][0]:.3f}-{c['rr_ci'][1]:.3f}) vs eTable 2 {rr[0]:.2f} ({rr[1]:.2f}-{rr[2]:.2f})")
print("\n=== Numbers needed to treat (vs DL), CI by inverting the Wald RD limits ===")
nnt = {}
for arm, key in (("VLM", ("DL", "VLM")), ("VLH", ("DL", "VLH"))):
c = res[key]
est, lo, hi = 1 / c["rd"], 1 / c["wald"][1], 1 / c["wald"][0]
nnt[arm] = (est, lo, hi)
r = nnt_rep[arm]
print(f" {arm}: reported {r[0]} ({r[1]}-{r[2]}); recomputed {est:.1f} ({lo:.1f}-{hi:.1f}); "
f"inverting the reported RD CI gives {100/reported[key]['hi']:.1f}-{100/reported[key]['lo']:.1f}")
# ---- Figures -----------------------------------------------------------------
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def style(ax):
ax.set_facecolor(BG)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: risk differences, reported vs recomputed 95% CI, one linear axis
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
style(ax)
keys = list(reported)
for i, k in enumerate(keys):
y = len(keys) - 1 - i
r, c = reported[k], res[k]
yr, yc = y + 0.17, y - 0.17
ax.errorbar(r["rd"], yr, xerr=[[r["rd"] - r["lo"]], [r["hi"] - r["rd"]]], fmt="D", ms=8,
color=ORANGE, mec=ORANGE, elinewidth=2, capsize=4, capthick=2, zorder=3)
ax.errorbar(100 * c["rd"], yc, xerr=[[100 * (c["rd"] - c["wald"][0])], [100 * (c["wald"][1] - c["rd"])]],
fmt="o", ms=8, color=BLUE, elinewidth=2, capsize=4, capthick=2, zorder=3)
ax.text(14.6, yr, f"reported {r['lo']:.2f} to {r['hi']:.2f}", va="center", ha="left",
fontsize=10, color=INK)
ax.text(14.6, yc, f"recomputed {100*c['wald'][0]:.2f} to {100*c['wald'][1]:.2f}", va="center",
ha="left", fontsize=10, color=INK)
ax.axvline(0, color=INK2, lw=1, zorder=1)
ax.axvline(-5, color=INK2, lw=1, ls=(0, (4, 3)), zorder=1)
ax.text(0.15, 2.62, "no difference", fontsize=10, color=INK2, va="bottom")
ax.text(-4.85, 2.62, "noninferiority\nmargin, −5", fontsize=10, color=INK2, va="bottom")
ax.set_yticks(range(len(keys)))
ax.set_yticklabels([reported[k]["label"] for k in reversed(keys)], fontsize=10.5, color=INK)
ax.set_xlim(-6, 14.3)
ax.set_ylim(-0.6, 3.15)
ax.set_xlabel("Difference in first-pass success, percentage points (95% CI)")
h1 = ax.errorbar([], [], xerr=[], fmt="D", ms=8, color=ORANGE, elinewidth=2, capsize=4,
label="Reported (Abstract and Results)")
h2 = ax.errorbar([], [], xerr=[], fmt="o", ms=8, color=BLUE, elinewidth=2, capsize=4,
label="Recomputed from Table 2 counts (unpooled Wald)")
ax.set_xticks([-5, 0, 5, 10])
fig.legend(handles=[h1, h2], loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False,
fontsize=10, handletextpad=0.4, columnspacing=1.2)
fig.subplots_adjust(left=0.14, right=0.71, top=0.97, bottom=0.27)
save(fig, "fig1")
# Figure 2: NNT vs DL, reported vs recomputed 95% CI, log axis
fig, ax = plt.subplots(figsize=(7.2, 2.7), facecolor=BG)
style(ax)
ax.set_xscale("log")
arms = ["VLM", "VLH"]
for i, arm in enumerate(arms):
y = len(arms) - 1 - i
r, c = nnt_rep[arm], nnt[arm]
yr, yc = y + 0.17, y - 0.17
ax.errorbar(r[0], yr, xerr=[[r[0] - r[1]], [r[2] - r[0]]], fmt="D", ms=8, color=ORANGE,
elinewidth=2, capsize=4, capthick=2, zorder=3)
ax.errorbar(c[0], yc, xerr=[[c[0] - c[1]], [c[2] - c[0]]], fmt="o", ms=8, color=BLUE,
elinewidth=2, capsize=4, capthick=2, zorder=3)
ax.text(195, yr, f"reported {r[0]} ({r[1]} to {r[2]})", va="center", fontsize=10)
ax.text(195, yc, f"recomputed {c[0]:.1f} ({c[1]:.1f} to {c[2]:.1f})", va="center", fontsize=10)
ax.set_xlim(5, 180)
ax.set_xticks([5, 10, 20, 50, 100])
ax.set_xticklabels(["5", "10", "20", "50", "100"])
ax.set_yticks(range(len(arms)))
ax.set_yticklabels([f"{a} vs DL" for a in reversed(arms)], fontsize=10.5, color=INK)
ax.set_ylim(-0.6, 1.6)
ax.set_xlabel("Number needed to treat (95% CI), log scale")
h1 = ax.errorbar([], [], xerr=[], fmt="D", ms=8, color=ORANGE, elinewidth=2, capsize=4,
label="Reported (Discussion)")
h2 = ax.errorbar([], [], xerr=[], fmt="o", ms=8, color=BLUE, elinewidth=2, capsize=4,
label="Recomputed: 1 / Wald RD limits")
fig.legend(handles=[h1, h2], loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False,
fontsize=10)
fig.subplots_adjust(left=0.14, right=0.63, top=0.96, bottom=0.38)
save(fig, "fig2")
print("\nFigures written to", HERE)
Reported HbA1c intervention effect of 0.02% lies outside its own 95% confidence interval of 0.03%–0.14%
JAMA Netw Open (2023) · doi:10.1001/jamanetworkopen.2022.50375 · PMC9856961
Summary
This secondary analysis of a cluster randomized trial (Texas Sprouts, 16 elementary schools) reports that, compared with control schools, the intervention produced a "0.02% reduction in mean hemoglobin A1c (95% CI, 0.03%-0.14%; P = .005)". The 0.02 figure appears in the Key Points, and the full estimate with its CI and P value appears in the Abstract, the Results text and Table 3. In each of those three places the estimate 0.02 lies below the lower limit of its own 95% CI. The four reported quantities (estimate 0.02, SE 0.01, CI 0.03 to 0.14, P = .005) cannot all be correct. The CI and the P value agree with each other and point to an effect of about 0.085 percentage points. The estimate and SE agree with neither, and the unadjusted difference in group changes in Table 3 (−0.03 vs +0.06, so 0.09) agrees with the CI. HbA1c is the only outcome behind the paper's conclusion that the intervention "improved glucose control". The Limitations say the improvement "may not be clinically relevant", and then add that "even small reductions in HbA1c levels reflect clinical improvements in glucose control". The improvement the paper reports, 0.02, is about one quarter of the size the paper's other numbers imply.
What the paper reports
Key Points, "Findings":
In this secondary analysis of a cluster randomized clinical trial of 695 children, a gardening, nutrition, and cooking intervention resulted in reductions of 0.02% for hemoglobin A1c and 6.40 mg/dL for low-density lipoprotein cholesterol levels.
Abstract, "Results":
Compared with control schools, children from Texas Sprouts schools had a 0.02% reduction in mean hemoglobin A1c (95% CI, 0.03%-0.14%; P = .005)
Results, paragraph on intervention effects:
Compared with children in the control schools, children in the Texas Sprouts intervention had a 0.02% reduction in mean HbA1c (95% CI, 0.03%-0.14%; P = .005)
Table 3, HbA1c row. Columns: intervention children, preintervention mean (SE), change mean (SE) | control children, preintervention mean (SE), change mean (SE) | differences in changes, mean (SE) [95% CI] | P value, intervention effect. (The minus sign is a Unicode minus in the publisher's XML.)
Hemoglobin A1c, %c | 236 | 5.27 (0.01) | -0.03 (0.02) | 221 | 5.22 (0.01) | 0.06 (0.02) | 0.02 (0.01) [0.03 to 0.14] | .005
Table 3, footnotes b and c:
Generalized linear mixed models with the identity link for continuous variables were used to test differences in metabolic parameters between the intervention and the control estimates, with schools as clusters fixed for their intervention effect and children nested within schools.
Children at 5 intervention schools and 5 control schools were analyzed for hemoglobin A1c.
Methods, "Statistical Analysis":
Data were analyzed using SPSS statistical software version 28 (IBM) and a 2-sided type I error level of α = .05 was used as the threshold for statistical significance.
The conclusions that rest on this result. Abstract, "Conclusions and Relevance":
In this cluster RCT, Texas Sprouts improved glucose control and reduced low-density lipoprotein cholesterol in high-risk youth.
Discussion, first paragraph:
To our knowledge, Texas Sprouts was the first cluster RCT to show that a school-based gardening, nutrition, and cooking intervention can improve glucose control and lower LDL cholesterol in elementary schoolchildren.
Discussion, paragraph on body composition:
The current intervention resulted in reductions in LDL cholesterol and improvements in HbA1c, independent of changes in body composition.
Discussion, "Limitations":
Another limitation is that the improvement in HbA1c was rather small, albeit significant, and may not be clinically relevant. However, health care professionals believe that even small reductions in HbA1c levels reflect clinical improvements in glucose control.
Conclusions:
In conclusion, the present RCT showed that a school-based gardening, nutrition, and cooking intervention resulted in a small, albeit significant, improvement in glycemic control and a reduction in LDL cholesterol in predominately low-income and racially diverse elementary schoolchildren.
What the paper's own numbers show
Method. The paper states an identity-link linear mixed model, so each 95% CI should be symmetric about its estimate, with half-width equal to a fixed multiplier times the SE. To find that multiplier I divided the CI half-width by the SE in the other rows of Table 3. In the seven rows without other problems it is 2.138 to 2.206 (median 2.148, not 1.96), which matches a t reference with about 14 degrees of freedom. I therefore used t(14) for the HbA1c checks, and give normal-reference values as well. For each pair of reported quantities (estimate, SE, CI, P) I worked out what one implies for the other. As an independent check I took the unadjusted difference between the two group changes in Table 3, with an SE from the two group SEs. That SE ignores clustering, so it is only a rough check. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Estimate inside its own 95% CI? | Abstract, Results, Table 3 | 0.02 with CI 0.03 to 0.14 | No: 0.02 is 0.01 below the lower limit |
| Centre of the reported CI | Table 3 | (estimate) 0.02 | (0.03 + 0.14) / 2 = 0.085 |
| SE implied by the CI width | Table 3 | 0.01 | 0.055 / 2.145 = 0.026 (t14); 0.028 (normal) |
| CI implied by estimate 0.02 and SE 0.01 | Table 3 | 0.03 to 0.14 | −0.001 to 0.041 (t14); 0.000 to 0.040 (normal) |
| P implied by estimate/SE (0.02/0.01 = 2.0) | Abstract, Results, Table 3 | .005 | .065 (t14); .046 (normal) |
| P implied by the reported CI | Abstract, Results, Table 3 | .005 | .0051 (t14); .0025 (normal); .004 to .007 for df 8 to 20 |
| Estimate needed for P = .005 if SE were 0.01 | Table 3 | 0.02 | 0.033 (t14); 0.028 (normal) |
| Unadjusted difference in changes, intervention − control | Table 3 change columns: −0.03 (0.02) vs 0.06 (0.02) | 0.02 | −0.09, i.e. a reduction of 0.09; SE 0.028, CI −0.145 to −0.035 (ignores clustering) |
| Sign of the difference column | Table 3, rows with clear effects (e.g. LDL changes −8.43 vs −3.53, difference −6.40) | HbA1c CI 0.03 to 0.14 (positive) | In those rows the column reads intervention − control, so a reduction should be negative: −0.14 to −0.03, which matches −0.145 to −0.035 |
| Reported effect vs CI centre | Key Points, Abstract | 0.02 | The CI centre is 4.25 times larger; the unadjusted difference is 4.5 times larger |
Three things follow. First, the estimate cannot belong to the reported interval, so at least one of them is wrong. Second, the CI and the P value fit together: under the multiplier the rest of Table 3 implies, the CI gives P = .0051, against the reported .005. The estimate 0.02 and SE 0.01 fit neither of them, and they are also much smaller than the unadjusted difference. Third, the CI's sign does not follow the convention the table uses in its rows with clear effects. In those rows (fasting glucose, total, LDL and non-HDL cholesterol) the column reads intervention − control, so an HbA1c reduction would be negative. With the sign restored, the HbA1c CI matches the unadjusted difference closely (−0.14 to −0.03 against −0.145 to −0.035). The most coherent reading of the paper's own numbers is an HbA1c reduction of about 0.085 to 0.09 percentage points, with the estimate and SE cells in error. The exact model estimate cannot be rebuilt from these rounded summaries, because elsewhere in Table 3 the model estimates differ from the unadjusted differences by up to about 2 SEs (for example, non-HDL cholesterol −6.13 against −3.83, about 2.2 SE). In fasting insulin and HDL the model estimate and the unadjusted difference differ in sign (+0.61 against −0.42; +0.43 against −0.31), which fits adjustment gaps of that size. So the sign convention is inferred from the rows with clear effects, not shown for every row.
Related observations in Table 3. These do not change the HbA1c finding, but they bear on how the table can be read:
- The P column does not follow from the displayed estimates and CIs in 6 of the other 8 rows. Fasting glucose has CI 1.60 to 6.84, which excludes zero (P ≈ .004 from the CI, t14), but P = .13 is printed. LDL has CI −8.97 to −3.82 (P ≈ .0001), but P = .048 is printed. The total cholesterol, HOMA-IR, HDL and non-HDL rows differ in the same way. So the "P value, intervention effect" column may come from a different test than the displayed difference; the paper does not say which. Among the rows whose CI excludes zero (fasting glucose, total, LDL and non-HDL cholesterol, HbA1c), HbA1c is the only one whose printed P matches its CI; the P values for fasting insulin (.474 from the CI, .46 printed) and triglycerides (.395 from the CI, .39 printed) also match their CIs.
- The triglycerides interval (−15.42 to 6.47) is not centred on its estimate 1.52 (centre −4.475), and its half-width is 4.3 times the SE, against about 2.15 in the other rows. A symmetric interval with upper limit 6.47 would have lower limit −3.43.
Figure
Figure 1. HbA1c effect in Table 3, on one axis in percentage points, with positive values meaning lower HbA1c in intervention schools than in control schools, as the paper's text reads "0.02% reduction (95% CI, 0.03%-0.14%)". The top row (orange) is what the paper reports: the dot (0.02) sits outside the bar (0.03 to 0.14). The blue rows are recomputed from the paper's own numbers. The second row is the interval that the reported estimate and SE would give (−0.001 to 0.041, P = .065), which does not match the reported CI or P. The third row is the centre and SE implied by the reported CI (0.085, SE 0.026, P = .0051), which matches the reported P. The fourth row is the unadjusted difference in the two group changes in Table 3 (−0.03 vs +0.06), shown as a reduction of 0.09 with CI 0.035 to 0.145. Its SE ignores clustering. The vertical line marks no effect. Intervals use the t(14) multiplier that the other rows of Table 3 imply.
Figure 2. Where each reported estimate sits within its own reported 95% CI, for all nine outcomes in Table 3, scaled so that the lower limit is 0, the centre 0.5 and the upper limit 1. Seven rows sit at the centre, as an identity-link model requires. Triglycerides sits off-centre (its lower limit, −15.42, is not symmetric about 1.52). HbA1c is the only estimate outside its interval.
Why it matters
- It is a headline result. The HbA1c effect is one of two results in the Key Points and the Abstract. The 0.02 figure appears in all four places the result is stated (Key Points, Abstract, Results, Table 3), and the estimate-outside-CI combination appears in three (Abstract, Results, Table 3).
- It carries the glucose conclusion. The Abstract says the intervention "improved glucose control", and the Discussion calls this a first for a cluster RCT. Fasting glucose, insulin and HOMA-IR were reported as showing no effect, so this claim rests on HbA1c alone.
- It changes the size judgement. The Limitations call the HbA1c improvement "rather small, albeit significant, and may not be clinically relevant", and then add that "health care professionals believe that even small reductions in HbA1c levels reflect clinical improvements in glucose control". The Conclusions call it "small". The improvement the paper reports, and the only HbA1c effect size it states, is 0.02 percentage points. The paper's own CI, P value and group changes point to roughly 0.085 to 0.09 percentage points, about 4 times larger. Whether that larger effect is clinically meaningful is a separate question, but a reader weighing these size statements against the reported 0.02 is weighing them against a figure the other numbers do not support.
- It does not, on the evidence shown, reverse the direction or the significance. The CI, the P value and the group changes all indicate lower HbA1c in intervention schools relative to control. So the qualitative conclusion is not overturned by these numbers; the reported magnitude and the internal consistency of the result are.
- Readers using the published numbers would be misled about the size. A meta-analyst taking 0.02 (SE 0.01) would enter an effect about one quarter the size, with an SE less than half of what the CI implies.
Explanations considered
- Model-adjusted estimate vs raw difference. Footnote b says the difference comes from a mixed model, so it need not equal the unadjusted −0.09. That can explain why 0.02 differs from 0.09. It cannot explain an estimate lying outside its own interval, and in the other rows the model estimate always sits at the centre of its CI.
- Asymmetric or transformed interval. The model uses an identity link, so the interval should be estimate ± multiplier × SE, as it is in seven other rows. Any valid interval, symmetric or not, must contain its point estimate.
- Different subsample, time point or units. The Abstract, Results and Table 3 all describe the same 457 children (236 and 221) at 5 intervention and 5 control schools (footnote c; Results, "Subsample B"), over the same 9-month intervention period (Abstract, "Interventions": "1 school year long (9 months)"; Abstract, "Main Outcomes and Measures" and "Results": "postintervention (9 months)" and "(9-month)" blood draws; Limitations: "only 9 months long"). The "n = 8 schools" in the Table 3 header is explained by footnote c. HbA1c is in % throughout; the SI footnote converts % to a fraction and does not affect the comparison.
- Sign convention. The text calls the effect a reduction and prints a positive CI, which would fit a convention in which positive means reduction. But in the Table 3 rows with clear effects the difference column reads intervention − control (fasting glucose +4.22 from changes 6.53 vs 2.34; LDL −6.40 from −8.43 vs −3.53; also total and non-HDL cholesterol). In fasting insulin and HDL the model estimate and the unadjusted difference differ in sign, which fits the adjustment gaps of up to about 2 SE noted above, so the convention rests on the rows with clear effects. Under that convention the HbA1c interval should be negative. Its sign appears dropped, and with the sign restored it matches the unadjusted difference. A sign convention cannot put 0.02 inside [0.03, 0.14].
- One-sided test. The Methods state a 2-sided α = .05.
- Rounding. All values are printed to two decimals. An estimate just under 0.025 (say 0.0249) with a lower limit just over it (0.0251) would round to 0.02 and 0.03 under any rounding rule, but the interval would then reach only 0.0002 below the estimate and about 0.115 above it, which an identity-link interval cannot do. It would also leave the SE (0.01) and P (.005) unexplained. Rounding cannot reconcile the four numbers.
- Conversion artifact. The publisher's version of record shows the same HbA1c values as PMC: 0.02 in the Key Points, and 0.02 with CI 0.03 to 0.14 and P = .005 in the Abstract, Results and Table 3. In the XML the minus signs of other rows (LDL, total cholesterol, non-HDL) are kept, so the positive HbA1c interval is not a lost character.
- A correct value in the supplements. Supplement 1 is the trial protocol, which predates the HbA1c measurements, and Supplement 2 is the data sharing statement. Neither contains model output, so neither resolves the discrepancy.
Status
- Version of record. Checked on the publisher's full text (jamanetwork.com/journals/jamanetworkopen/fullarticle/2800162, reached from doi.org), PMC (PMC9856961) and the Europe PMC full-text XML. All show the same HbA1c values in the Key Points, Abstract, Results and Table 3.
- Supplements. Supplement 1 (jamanetwopen-e2250375-s001.pdf, trial protocol, 15 pages) and Supplement 2 (jamanetwopen-e2250375-s002.pdf, data sharing statement, 1 page) were read. Neither contains HbA1c results. Participant data are stated to be available on request after an approved proposal and a data sharing agreement.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and a Crossref
updatesfilter returns no items. PMC and Europe PMC (PMID 36626172) list no correction or comment links. A web search found no erratum for this article. PubPeer refused automated access from this machine (HTTP 403 from both its API and its search page), so PubPeer could not be checked. - Sources. PMC article PMC9856961; publisher page https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2800162; Supplements 1 and 2; Crossref; Europe PMC; trial registration NCT02668744 (as cited by the paper, not used for any number here).
Suggested correction
- Report the HbA1c intervention effect from the fitted model with an estimate, SE, 95% CI and P value that agree with one another. Correct the Key Points, Abstract, Results text and Table 3 together. Use the sign convention of the Table 3 rows with clear effects (intervention − control, so a reduction is negative) in the table, and state the magnitude in the text.
- If, as the CI and P value suggest, the effect is about 0.08 to 0.09 percentage points, revisit the Limitations and Conclusions statements that the improvement was small and "may not be clinically relevant", which accompany the reported 0.02.
- State which test the "P value, intervention effect" column reports. For fasting glucose, HOMA-IR, total, LDL, HDL and non-HDL cholesterol the printed P values do not follow from the displayed differences and CIs (for example, fasting glucose has a CI of 1.60 to 6.84 but P = .13). If the P values are wrong, the statements that there were "no intervention effects" on glucose and other lipids need to be revisited too.
- Check the triglycerides interval (−15.42 to 6.47), which is not symmetric about its estimate 1.52, and remove the minus signs printed on SEs, as in "−6.40 (−1.20)".
Recomputation script (python recompute.py.txt)
"""Recompute the HbA1c intervention effect reported in PMC9856961
(JAMA Netw Open 2023, doi:10.1001/jamanetworkopen.2022.50375), Table 3.
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes, next to this file:
fig1.svg/png, fig2.svg/png (the figures used in writeup.md)
candidates/cand_A..E.svg/png (five alternative designs for the main figure)
All inputs are copied from Table 3 of the article as archived in PMC (identical in the
publisher's version of record). Columns: intervention change, control change,
"Differences in changes, mean (SE) [95% CI]", "P value, intervention effect".
In the rows with clear effects (fasting glucose, total, LDL and non-HDL cholesterol) the
table's difference column reads intervention minus control (e.g. LDL: changes -8.43 vs
-3.53, difference -6.40), so a reduction in HbA1c would be negative there. In fasting
insulin and HDL the model estimate and the unadjusted difference differ in sign (section
4b), so the convention is inferred from the rows with clear effects.
"""
import math
import os
import numpy as np
from scipy import stats
from scipy.optimize import brentq
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ------------------------------------------------------------------ inputs (Table 3)
# name: (intervention change, control change, estimate, SE, CI low, CI high, reported P)
# SEs printed with a minus sign in the table (e.g. "-6.40 (-1.20)") are entered as positive.
ROWS = {
"Fasting glucose": (6.53, 2.34, 4.22, 1.22, 1.60, 6.84, 0.13),
"Fasting insulin": (0.84, 1.26, 0.61, 0.80, -1.16, 2.37, 0.46),
"HOMA-IR": (0.68, 0.57, 0.39, 0.18, -0.001, 0.77, 0.84),
"Total cholesterol": (-7.35, -3.23, -5.72, 1.24, -8.39, -3.04, 0.12),
"LDL cholesterol": (-8.43, -3.53, -6.40, 1.20, -8.97, -3.82, 0.048),
"Triglycerides": (5.92, 1.42, 1.52, 2.54, -15.42, 6.47, 0.39),
"HDL cholesterol": (0.28, 0.59, 0.43, 0.34, -0.30, 1.17, 0.74),
"Non-HDL chol.": (-7.62, -3.79, -6.13, 1.05, -8.38, -3.89, 0.08),
"Hemoglobin A1c": (-0.03, 0.06, 0.02, 0.01, 0.03, 0.14, 0.005),
}
GROUP_SE_A1C = (0.02, 0.02) # SEs of the intervention and control HbA1c changes
N_A1C = (236, 221) # children with HbA1c, intervention / control
Z = stats.norm.ppf(0.975)
I_CH, C_CH, EST, SE, LO, HI, P_REP = ROWS["Hemoglobin A1c"]
# ------------------------------------------------------------------ colours / style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "axes.edgecolor": INK2,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"text.color": INK, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "axes.linewidth": 0.8,
})
def save(fig, path_noext):
fig.savefig(path_noext + ".svg", bbox_inches="tight")
fig.savefig(path_noext + ".png", dpi=200, bbox_inches="tight")
plt.close(fig)
def fp(p):
"""P value in the paper's style: no leading zero (.065, .0051)."""
d = 4 if p < 0.01 else 3
return f"{p:.{d}f}".lstrip("0")
def p2(z, df=None):
z = abs(z)
return 2 * (stats.t.sf(z, df) if df else stats.norm.sf(z))
def hdr(t):
print("\n" + "=" * 78 + "\n" + t + "\n" + "=" * 78)
# ================================================================== 1. HbA1c row
hdr("1. HbA1c row as reported (Table 3; same numbers in Abstract and Results)")
print(f"estimate {EST} SE {SE} 95% CI [{LO}, {HI}] P {P_REP}")
print(f"Is the estimate inside its own CI? {LO <= EST <= HI} "
f"(estimate is {LO - EST:.2f} below the lower limit)")
mid = (LO + HI) / 2
half = (HI - LO) / 2
print(f"CI midpoint = {mid:.3f}; CI half-width = {half:.3f}")
# ================================================================== 2. multiplier used elsewhere in Table 3
hdr("2. CI half-width / SE in the other rows of Table 3 (what multiplier the table uses)")
mults = {}
for k, (i, c, e, s, lo, hi, p) in ROWS.items():
m = (hi - lo) / 2 / s
sym = abs((hi - e) - (e - lo)) / ((hi - lo) / 2)
mults[k] = m
print(f"{k:18s} half-width/SE = {m:5.3f} midpoint {(lo + hi) / 2:8.3f} vs estimate {e:7.3f}")
reg = [m for k, m in mults.items() if k not in ("Triglycerides", "Hemoglobin A1c")]
m_med = float(np.median(reg))
print(f"\nThe 7 regular rows: multiplier {min(reg):.3f} to {max(reg):.3f}, median {m_med:.3f} "
f"(normal 1.960)")
df_med = brentq(lambda d: stats.t.ppf(0.975, d) - m_med, 2, 1000)
df_lo = brentq(lambda d: stats.t.ppf(0.975, d) - max(reg), 2, 1000)
df_hi = brentq(lambda d: stats.t.ppf(0.975, d) - min(reg), 2, 1000)
print(f"t quantile equal to the median multiplier: df = {df_med:.1f} "
f"(range of rows: df {df_lo:.1f} to {df_hi:.1f})")
DF = 14
TQ = stats.t.ppf(0.975, DF)
print(f"Used below: t reference with df = {DF}, t(0.975) = {TQ:.3f}")
# ================================================================== 3. what the reported numbers imply
hdr("3. HbA1c: reconstructing each reported quantity from the others")
ci_from_est_t = (EST - TQ * SE, EST + TQ * SE)
ci_from_est_z = (EST - Z * SE, EST + Z * SE)
print(f"CI from estimate 0.02 and SE 0.01, t({DF}): [{ci_from_est_t[0]:.4f}, {ci_from_est_t[1]:.4f}]")
print(f"CI from estimate 0.02 and SE 0.01, normal: [{ci_from_est_z[0]:.4f}, {ci_from_est_z[1]:.4f}]")
p_est_t = p2(EST / SE, DF)
p_est_z = p2(EST / SE)
print(f"P from 0.02/0.01 (ratio {EST / SE:.2f}): t({DF}) P = {p_est_t:.3f}; normal P = {p_est_z:.4f}"
f" (reported {P_REP})")
se_ci_t = half / TQ
se_ci_z = half / Z
print(f"SE implied by the CI width: t({DF}) {se_ci_t:.4f}; normal {se_ci_z:.4f} (reported {SE})")
p_ci_t = p2(mid / se_ci_t, DF)
p_ci_z = p2(mid / se_ci_z)
print(f"P implied by the CI (midpoint {mid:.3f}): t({DF}) ratio {mid / se_ci_t:.2f}, P = {p_ci_t:.4f}; "
f"normal ratio {mid / se_ci_z:.2f}, P = {p_ci_z:.4f} (reported {P_REP})")
for d in (8, 10, 12, 20):
q = stats.t.ppf(0.975, d)
print(f" sensitivity, df={d:2d}: P implied by the CI = {p2(mid / (half / q), d):.4f}")
z_rep = stats.norm.isf(P_REP / 2)
t_rep = stats.t.isf(P_REP / 2, DF)
# just under / just over 0.025, so the result does not depend on the rounding rule
print(f"Rounding check: estimate 0.0249 and lower limit 0.0251 (round to 0.02 / 0.03) would put "
f"the interval {0.0251 - 0.0249:.4f} below and {HI - 0.0249:.4f} above the estimate")
print(f"Estimate needed for P = .005 with SE 0.01: {t_rep * SE:.3f} (t{DF}) / {z_rep * SE:.3f} (normal)")
# ================================================================== 4. raw difference in group changes
hdr("4. Unadjusted difference in mean changes (Table 3 change columns)")
raw = I_CH - C_CH
se_raw = math.hypot(*GROUP_SE_A1C)
ci_raw = (raw - Z * se_raw, raw + Z * se_raw)
print(f"intervention change {I_CH:+.2f}, control change {C_CH:+.2f}: "
f"difference (I - C) = {raw:+.2f}")
print(f"SE from the two group SEs (ignores clustering) = {se_raw:.4f}; "
f"95% CI [{ci_raw[0]:.3f}, {ci_raw[1]:.3f}]; normal P = {p2(raw / se_raw):.4f}")
print(f"Reported CI with the table's sign convention restored: [{-HI:.2f}, {-LO:.2f}]")
print(f"Ratio of CI midpoint to the reported 0.02: {mid / EST:.2f}; of |raw difference|: "
f"{abs(raw) / EST:.2f}")
hdr("4b. Model estimate vs unadjusted difference in the other rows (how far they can differ)")
for k, (i, c, e, s, lo, hi, p) in ROWS.items():
print(f"{k:18s} unadjusted I-C {i - c:+7.2f} model estimate {e:+7.2f} "
f"gap {e - (i - c):+6.2f} ({(e - (i - c)) / s:+5.2f} SE)")
# ================================================================== 5. sign convention
hdr("5. Sign convention of the difference column")
print("Rows with clear effects (CI excludes zero), then the two rows where the model estimate and"
" the unadjusted difference differ in sign, then HbA1c:")
for k in ("Fasting glucose", "Total cholesterol", "LDL cholesterol", "Non-HDL chol.",
"Fasting insulin", "HDL cholesterol", "Hemoglobin A1c"):
i, c, e, s, lo, hi, p = ROWS[k]
print(f"{k:18s} I {i:+.2f} C {c:+.2f} I-C {i - c:+.2f} table estimate {e:+.2f} CI [{lo}, {hi}]")
# ================================================================== 6. P column across Table 3
hdr("6. Reported P vs P implied by each row's own estimate/SE and CI")
print(f"{'row':18s} {'est in CI':>9s} {'P est/SE t14':>12s} {'P CI t14':>10s} {'P est/SE z':>10s} "
f"{'P CI z':>9s} {'reported':>8s}")
P_ROWS = {}
for k, (i, c, e, s, lo, hi, p) in ROWS.items():
m = (lo + hi) / 2
sct, scz = (hi - lo) / 2 / TQ, (hi - lo) / 2 / Z
P_ROWS[k] = (p2(e / s, DF), p2(m / sct, DF), p)
print(f"{k:18s} {str(lo <= e <= hi):>9s} {p2(e / s, DF):12.4g} {p2(m / sct, DF):10.4g} "
f"{p2(e / s):10.4g} {p2(m / scz):9.4g} {p:8.3g}")
i, c, e, s, lo, hi, p = ROWS["Triglycerides"]
print(f"\nTriglycerides: lower limit that would be symmetric about {e} given the upper limit "
f"{hi}: {2 * e - hi:.2f} (printed {lo}); estimate +/- {TQ:.3f} x SE: "
f"[{e - TQ * s:.2f}, {e + TQ * s:.2f}]")
# ================================================================== figures
POS = dict(lw=2)
def axis_style(ax, xlabel):
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel(xlabel, color=INK)
# Reduction scale used in the HbA1c figures: positive = lower HbA1c in intervention
# schools relative to control, which is how the paper's text reads "0.02% reduction
# (95% CI, 0.03%-0.14%)". The unadjusted I-C difference is sign-flipped onto it.
RED_LABEL = ("Reduction in HbA1c vs control, percentage points\n"
"(positive = lower in intervention schools, as the paper's text reads it)")
forest_rows = [
("Reported in the paper\n(Abstract, Results, Table 3)", EST, LO, HI, ORANGE, "o",
f"0.02, CI 0.03 to 0.14, P = .005\nestimate lies outside its own interval"),
("Recomputed: interval from the\nreported estimate 0.02 and SE 0.01", EST,
ci_from_est_t[0], ci_from_est_t[1], BLUE, "o",
f"CI {ci_from_est_t[0]:.3f} to {ci_from_est_t[1]:.3f}, P = {fp(p_est_t)}"),
("Recomputed: centre and SE implied\nby the reported CI", mid, LO, HI, BLUE, "D",
f"{mid:.3f} (SE {se_ci_t:.3f}), P = {fp(p_ci_t)}"),
("Recomputed: unadjusted difference\nin group changes, -0.03 vs +0.06", -raw,
-ci_raw[1], -ci_raw[0], BLUE, "s",
f"{-raw:.2f}, CI {-ci_raw[1]:.3f} to {-ci_raw[0]:.3f}\n(ignores clustering)"),
]
def draw_forest(path):
fig, ax = plt.subplots(figsize=(7.2, 3.9))
n = len(forest_rows)
for j, (lab, e, lo, hi, col, mk, note) in enumerate(forest_rows):
y = n - 1 - j
ax.plot([lo, hi], [y, y], color=col, solid_capstyle="butt", zorder=3, **POS)
for xx in (lo, hi):
ax.plot([xx, xx], [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
ax.plot(e, y, mk, color=col, ms=9, mec=BG, mew=1.5, zorder=4)
ax.text(0.205, y, note, va="center", ha="left", fontsize=9.5, color=INK2)
# highlight the gap on the reported row
y0 = n - 1
ax.text(EST, y0 + 0.30, "0.02 < lower limit 0.03", ha="left", va="bottom",
fontsize=9.5, color=INK)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
ax.text(0.001, -0.75, "no effect", fontsize=9, color=INK2, ha="left")
ax.set_yticks(range(n))
ax.set_yticklabels([r[0] for r in forest_rows][::-1], fontsize=10)
for t, r in zip(ax.get_yticklabels(), forest_rows[::-1]):
t.set_color(INK)
ax.set_xlim(-0.02, 0.36)
ax.set_xticks([0, 0.05, 0.10, 0.15, 0.20])
ax.set_ylim(-0.9, n - 0.35)
axis_style(ax, RED_LABEL)
ax.xaxis.set_label_coords(0.29, -0.13)
from matplotlib.lines import Line2D
h = [Line2D([], [], color=ORANGE, marker="o", lw=2, ms=8, label="Reported by the paper"),
Line2D([], [], color=BLUE, marker="o", lw=2, ms=8,
label="Recomputed from the paper's own numbers")]
fig.legend(handles=h, loc="upper left", bbox_to_anchor=(0.0, 0.985), ncol=2,
frameon=False, fontsize=9.5)
fig.text(0.0, 1.04, f"HbA1c effect, Table 3 (n = {N_A1C[0]} vs {N_A1C[1]} children, "
f"10 schools); 95% intervals, t({DF})", fontsize=10.5, color=INK, ha="left")
save(fig, path)
draw_forest(os.path.join(HERE, "fig1"))
draw_forest(os.path.join(CAND, "cand_A_forest"))
# ---- Candidate B: one number line, reported row only, annotated
def draw_numberline(path):
fig, ax = plt.subplots(figsize=(7.2, 2.3))
ax.plot([LO, HI], [0, 0], color=ORANGE, lw=2, solid_capstyle="butt")
for xx in (LO, HI):
ax.plot([xx, xx], [-0.12, 0.12], color=ORANGE, lw=2)
ax.plot(EST, 0, "o", color=ORANGE, ms=10, mec=BG, mew=1.5)
ax.plot(mid, 0, "D", color=BLUE, ms=9, mec=BG, mew=1.5)
ax.text(EST, -0.32, "reported estimate\n0.02", ha="center", va="top", color=INK)
ax.text(LO, 0.25, "reported lower\nlimit 0.03", ha="center", va="bottom", color=INK)
ax.text(HI, 0.25, "reported upper\nlimit 0.14", ha="center", va="bottom", color=INK)
ax.text(mid, -0.32, f"recomputed centre\nof that CI, {mid:.3f}", ha="center", va="top",
color=INK)
ax.axvline(0, color=INK2, lw=0.8)
ax.text(0.002, -0.9, "no effect", color=INK2, fontsize=9)
ax.set_ylim(-1.0, 0.9)
ax.set_xlim(-0.01, 0.16)
ax.set_yticks([])
axis_style(ax, "Reduction in HbA1c vs control, percentage points")
ax.set_title("A 95% CI must contain its estimate: 0.02 lies outside [0.03, 0.14]\n"
"orange = reported by the paper, blue = recomputed", loc="left",
fontsize=10.5)
save(fig, path)
draw_numberline(os.path.join(CAND, "cand_B_numberline"))
# ---- Candidate C: calculation-flow diagram
def box(ax, x, y, w, h, text, edge, fs=9.8, bold=False):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.04",
fc=BG, ec=edge, lw=2))
ax.text(x + w / 2, y + h / 2, text, ha="center", va="center", fontsize=fs, color=INK,
fontweight="bold" if bold else "normal")
def arrow(ax, x0, y0, x1, y1, label=None, off=(0, 0.06)):
ax.annotate("", xy=(x1, y1), xytext=(x0, y0),
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.4))
if label:
ax.text((x0 + x1) / 2 + off[0], (y0 + y1) / 2 + off[1], label, ha="center",
fontsize=9, color=INK2)
def draw_flow(path):
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ax.set_xlim(-0.1, 10.1)
ax.set_ylim(0.3, 5.3)
ax.axis("off")
W, H, fs = 3.0, 1.45, 9.6
box(ax, 0.0, 3.4, W, H, "Table 3, change\nintervention \u22120.03\ncontrol +0.06", ORANGE, fs=fs)
box(ax, 3.5, 3.4, W, H, "Unadjusted difference\n\u22120.03 \u2212 0.06 = \u22120.09\n"
"(reduction of 0.09)", BLUE, fs=fs)
box(ax, 7.0, 3.4, W, H, f"SE {se_raw:.3f}\n(from the 2 group SEs)\nCI {-ci_raw[1]:.3f} to "
f"{-ci_raw[0]:.3f}\n(as a reduction)", BLUE, fs=fs)
box(ax, 0.0, 0.5, W, H, "Table 3, difference\n0.02 (SE 0.01)\nCI 0.03 to 0.14", ORANGE, fs=fs)
box(ax, 3.5, 0.5, W, H, f"Centre of that CI\n(0.03 + 0.14) / 2\n= {mid:.3f}", BLUE, fs=fs)
box(ax, 7.0, 0.5, W, H, f"SE implied by CI {se_ci_t:.3f}\nP implied by CI {fp(p_ci_t)}\n"
f"(reported P .005)", BLUE, fs=fs)
ym_top, ym_bot = 3.4 + H / 2, 0.5 + H / 2
for yy in (ym_top, ym_bot):
arrow(ax, 3.05, yy, 3.45, yy)
arrow(ax, 6.55, yy, 6.95, yy)
for xx in (1.5, 5.0):
ax.annotate("", xy=(xx, 0.5 + H + 0.08), xytext=(xx, 3.4 - 0.08),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.2, ls="--"))
yc = (0.5 + H + 3.4) / 2
ax.text(5.1, yc, f"agree: about 0.09 / {mid:.3f}", fontsize=9.5, color=INK, va="center")
ax.text(1.6, yc, "estimate 0.02 is outside\nits own CI [0.03, 0.14]", fontsize=9.5,
color=INK, va="center")
ax.text(0.0, 5.12, "HbA1c, Table 3. Orange boxes: reported by the paper. Blue boxes: "
"recomputed from those numbers.", fontsize=9.5, color=INK2)
save(fig, path)
draw_flow(os.path.join(CAND, "cand_C_flow"))
# ---- Candidate D: consistency graph of the four reported quantities
def draw_consistency(path):
fig, ax = plt.subplots(figsize=(7.2, 4.2))
ax.set_xlim(0, 10.4)
ax.set_ylim(0.1, 5.85)
ax.axis("off")
W, H = 2.2, 1.1
nodes = {"est": (0.1, 4.6, "Estimate\n0.02"), "se": (0.1, 0.3, "SE\n0.01"),
"ci": (8.1, 4.6, "95% CI\n0.03 to 0.14"), "p": (8.1, 0.3, "P value\n.005")}
for k, (x, y, t) in nodes.items():
box(ax, x, y, W, H, t, ORANGE, fs=10.5, bold=True)
def c(k):
x, y, _ = nodes[k]
return x + W / 2, y + H / 2
def edge(a, b, txt, ok, t=0.5, above=False):
(x0, y0), (x1, y1) = c(a), c(b)
ax.plot([x0, x1], [y0, y1], color=BLUE if ok else INK2, lw=2,
ls="-" if ok else (0, (4, 3)), zorder=0)
# a label placed above a horizontal link leaves the dashes visible
dy, va = (0.22, "bottom") if above else (0, "center")
ax.text(x0 + t * (x1 - x0), y0 + t * (y1 - y0) + dy, txt, ha="center", va=va,
fontsize=9.3, color=INK, zorder=5,
bbox=dict(boxstyle="round,pad=0.3", fc=BG, ec=BLUE if ok else INK2, lw=1))
edge("est", "ci", f"INCONSISTENT: estimate lies outside the CI\n"
f"(CI centre is {mid:.3f})", False, above=True)
edge("est", "p", f"INCONSISTENT:\n0.02 / 0.01 gives\nP = {fp(p_est_t)}, not .005", False,
t=0.3)
edge("se", "ci", f"INCONSISTENT:\nCI width implies\nSE {se_ci_t:.3f}, not 0.01", False,
t=0.7)
edge("ci", "p", f"consistent:\nCI implies\nP = {fp(p_ci_t)}", True)
edge("est", "se", f"INCONSISTENT:\ntogether give\nCI {ci_from_est_t[0]:.3f}\n"
f"to {ci_from_est_t[1]:.3f},\nnot 0.03 to 0.14", False)
fig.text(0.0, 1.0, "HbA1c, Table 3. Orange boxes: values reported by the paper. Each link is "
"a check recomputed\nfrom them (t reference, df = 14, as the other rows of "
"Table 3 imply).\nSolid blue link = consistent; dashed gray link = inconsistent.",
fontsize=9.5, color=INK2, va="bottom")
save(fig, path)
draw_consistency(os.path.join(CAND, "cand_D_consistency"))
# ---- Candidate E (also fig2): position of each estimate within its own CI, all of Table 3
def draw_position(path):
fig, ax = plt.subplots(figsize=(7.2, 3.8))
names = list(ROWS)[::-1]
ax.axvspan(0, 1, color="#efeeea", zorder=0, lw=0)
for x in (0, 0.5, 1):
ax.axvline(x, color=INK2, lw=0.8 if x != 0.5 else 0.6, ls="-" if x != 0.5 else ":",
zorder=1)
for y, k in enumerate(names):
i, c, e, s, lo, hi, p = ROWS[k]
pos = (e - lo) / (hi - lo)
out = not (lo <= e <= hi)
col = ORANGE
ax.plot(pos, y, "o", color=col, ms=9, mec=BG, mew=1.5, zorder=3)
if out or k == "Triglycerides":
ax.text(pos + (-0.04 if out else 0.04), y, f"{e} in [{lo}, {hi}]",
ha="right" if out else "left", va="center", fontsize=9.3, color=INK)
ax.set_yticks(range(len(names)))
ax.set_yticklabels(names, color=INK)
ax.set_xlim(-0.75, 1.35)
ax.set_xticks([0, 0.5, 1])
ax.set_xticklabels(["lower\nlimit", "centre", "upper\nlimit"])
axis_style(ax, "Where the reported estimate sits within its own reported 95% CI")
ax.grid(False)
ax.set_title("Table 3, all rows (orange = positions computed from reported values)",
loc="left", fontsize=10.5)
save(fig, path)
draw_position(os.path.join(HERE, "fig2"))
draw_position(os.path.join(CAND, "cand_E_table_overview"))
hdr("Figures written")
for f in sorted(os.listdir(HERE)) + ["candidates/" + g for g in sorted(os.listdir(CAND))]:
if f.endswith((".png", ".svg")):
print(" ", f)
Diabetes-only hyperglycemia primary outcome has denominators larger than the trial's possible diabetic population
JAMA Network Open (2023) · doi:10.1001/jamanetworkopen.2023.32517 · PMC10517374
Summary
This pilot randomized trial (ACTFAST-3) defines one of its two primary outcomes, avoidance of postoperative hyperglycemia, for patients with diabetes only, and states that "patients without diabetes were excluded from analysis". Yet the reported denominators (8676 intervention, 8815 control) exceed the most cases with diabetes that the paper's own Table 1 allows (5713 and 5887, even counting every case of unknown status as diabetic), and they equal exactly the analyzed cases with any glucose value in the Figure 1 flowchart. The published result (85.8% vs 85.8%; RR 1.00, 95% CI 0.99–1.01) therefore describes a population in which at least a third of cases, and probably about two-thirds, did not have diabetes. Its narrow confidence interval, which the Discussion describes as "tight confidence bounds", depends on that larger denominator, and simple calculations suggest an interval for the stated population would be roughly 2 to 3.5 times as wide.
What the paper reports
Methods, "Study Population":
For glycemic outcomes, patients without diabetes were excluded from analysis.
Methods, "Outcomes":
The primary outcomes were avoidance of hypothermia, defined as the proportion of patients who had a final recorded intraoperative core temperature greater than 36 °C, and avoidance of hyperglycemia, defined as the proportion of patients with diabetes who had a blood glucose level of 180 mg/dL or lower on arrival to the postanesthesia recovery area.
Abstract, "Main Outcomes and Measures":
(defined as the proportion of patients with diabetes who had a blood glucose level ≤180 mg/dL on arrival to the postanesthesia recovery area)
Abstract, "Results". The Results text (primary outcomes paragraph) repeats the same counts, percentages and RR:
There was no treatment effect on avoidance of hyperglycemia (7445 of 8676 patients [85.8%] in the intervention group vs 7559 of 8815 [85.8%] in the control group; rate ratio [RR], 1.00; 95% CI, 0.99-1.01)
Table 2, primary outcome row (columns: intervention, n = 12 980 | control, n = 13 274 | coefficient (95% CI) | P value):
No postoperative hyperglycemia | 7445/8676 (85.8) | 7559/8815 (85.8) | 1.00 (0.99 to 1.01) | >.99
Table 1, comorbidity row. Table 1 lists the control column first (control, n = 13 274 | intervention, n = 12 980):
Diabetes | 2909/10 296 (28.3) | 2824/10 091 (28.0)
Figure 1, the study flowchart. The flowchart is an image, so this text was read from the publisher's image file (jamanetwopen-e2332517-g001.jpg) rather than from a text conversion:
12980 Analyzed (1581 without temperature data, 4304 without glucose data)
13274 Analyzed (1652 without temperature data, 4459 without glucose data)
Figure 1 caption:
The analysis included all patients randomized during the study with outcomes present.
The findings and conclusions that rest on this outcome. Key Points, "Findings":
an intraoperative telemedicine decision support intervention did not significantly reduce postoperative hypothermia or hyperglycemia
Abstract, "Conclusions and Relevance":
anesthesia care quality measures did not differ between groups, with high confidence in the findings
Discussion, second paragraph:
There are several potential reasons for the lack of an intention-to-treat effect with tight confidence bounds on these quality of care outcomes.
Discussion, third paragraph:
Second, the overall adherence to each measure at baseline was high, making statistically significant improvements difficult to achieve.
What the paper's own numbers show
Method. Table 1, Table 2 and Figure 1 count cases in the same unit and with the same arm totals (12 980 and 13 274). For each arm I took the cases with diabetes from Table 1. I counted cases with unknown diabetes status as the arm total minus the Table 1 denominator. The sum of the two is an upper bound on the number of cases that could have diabetes. I compared that bound with the Table 2 hyperglycemia denominator, and compared the denominator with Figure 1's analyzed cases minus cases "without glucose data". I repeated the check for the two sensitivity analyses in Supplement 2 (eTables 4–7). Those tables print diabetes only as n (%), so there the bound uses the smallest known-status total consistent with the rounded percentage. For contrast I looked at the secondary outcome that is explicitly limited to patients with diabetes. Finally I checked the Table 2 arithmetic with crude, unclustered rate ratios. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Cases with diabetes, intervention | Table 1 | 2824/10 091 (28.0) | 28.0%; 12 980 − 10 091 = 2889 with unknown status |
| Cases with diabetes, control | Table 1 | 2909/10 296 (28.3) | 28.3%; 13 274 − 10 296 = 2978 with unknown status |
| Most cases that could have diabetes, intervention | Table 1 | — | 2824 + 2889 = 5713 |
| Most cases that could have diabetes, control | Table 1 | — | 2909 + 2978 = 5887 |
| Hyperglycemia denominator, intervention | Table 2; Abstract; Results | 8676 | 2963 more than the 5713 maximum; 3.07 × the documented diabetic cases |
| Hyperglycemia denominator, control | Table 2; Abstract; Results | 8815 | 2928 more than the 5887 maximum; 3.03 × the documented diabetic cases |
| Analyzed cases with glucose data, intervention | Figure 1 | 12 980 analyzed, 4304 without glucose data | 12 980 − 4304 = 8676, identical to the denominator |
| Analyzed cases with glucose data, control | Figure 1 | 13 274 analyzed, 4459 without glucose data | 13 274 − 4459 = 8815, identical to the denominator |
| Denominator as a share of all analyzed cases | Table 2 vs Figure 1 | — | 66.8% and 66.4%, against a diabetes prevalence of 28.0% and 28.3% |
| Share of the denominator that cannot have diabetes | derived | — | at least 34.2% (intervention) and 33.2% (control); 67.5% and 67.0% if only documented diabetes is counted |
| Sensitivity analysis, excluding repeat surgeries within 30 days | eTables 4–5 | denominators 8069 / 8169; diabetes 2614 (27.6%) / 2667 (27.6%) of 12 179 / 12 457 | most possible diabetic cases ≤ 5339 / 5478; denominators exceed this by 2730 / 2691 |
| Sensitivity analysis, ignoring prior assignment | eTables 6–7 | denominators 8693 / 8798; diabetes 2844 (28.2%) / 2889 (28.0%) of 12 965 / 13 289 | most possible diabetic cases ≤ 5741 / 5878; denominators exceed this by 2952 / 2920 |
| Diabetes-restricted secondary outcome (intraoperative glucose measurement), for contrast | Table 2 | denominators 1962 / 1996 | 69.5% and 68.6% of the documented diabetic cases, so this outcome does fit inside the diabetic population |
| No postoperative hyperglycemia, % and RR | Table 2 | 85.8 vs 85.8; RR 1.00 (0.99 to 1.01) | 85.81% vs 85.75%; crude RR 1.001 (unclustered 95% CI 0.989–1.013), consistent with the printed values for the printed denominators |
The Table 2 arithmetic is internally consistent. The problem is the population. The denominators equal every analyzed case with a glucose value, not the cases with diabetes. The same pattern appears in all three published versions of the analysis (the main analysis and both sensitivity analyses; Figure 2 below), so it is not a single transcription slip.
The comparison also turned up some smaller, separate inconsistencies. None of them changes a conclusion:
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Arm sizes | Abstract vs Methods, Table 1, Table 2, Figure 1 | Abstract: "control group; n = 12 980" and "intervention group; n = 13 274" | Every other location gives intervention 12 980 and control 13 274, so the abstract's labels are swapped |
| Hypothermia denominators | Table 2 vs Figure 1 | 11 447 / 11 672 | 12 980 − 1581 = 11 399 and 13 274 − 1652 = 11 622, which differ by 48 and 50 |
| Intraoperative glucose measurement, intervention % | Table 2 | 1346/1962 (68.7) | 68.60%, which rounds to 68.6 |
| Control hypothermia % | Abstract | "[66.7.%]" | typographical (66.7%) |
Figure
Figure 1. Counts for each arm on one axis of surgical cases, drawn from zero. The orange bar is the reported denominator of the hyperglycemia primary outcome (Table 2), which the paper defines for patients with diabetes only. The blue bars come from the paper's other numbers: the cases with documented diabetes (Table 1); the most cases that could have diabetes if every case with unknown status had it (Table 1); and the analyzed cases with any glucose value (Figure 1: analyzed minus "without glucose data"). The gray bar is all analyzed cases. In both arms the reported denominator exceeds the largest possible diabetic population, by 2963 and 2928 cases, and equals the number of cases with glucose data exactly.
Figure 2. The same comparison for the main analysis (Tables 1–2) and the two sensitivity analyses in Supplement 2 (eTables 4–5, excluding surgeries within 30 days of an index operation; eTables 6–7, ignoring prior treatment assignment). Hollow blue circles are cases with documented diabetes as printed. Blue diamonds are the most cases that could have diabetes; for the sensitivity analyses this is an upper bound derived from the rounded percentage. Orange circles are the reported hyperglycemia denominators. The number on each row is the reported denominator minus that maximum. In all six arm-analyses the denominator lies about 2700–3000 cases above the largest possible diabetic population.
Why it matters
Avoidance of hyperglycemia is one of the trial's two primary outcomes. Its null result is reported in the Key Points, the Abstract, the Results, Table 2 and eTables 5 and 7. The paper's stated population for it is patients with diabetes, and the published protocol that the paper cites for its sample size (reference 22, F1000Research 2018;7:623) planned it that way: its power calculation states that "we assumed that the prevalence of diabetes mellitus among Barnes-Jewish Hospital surgical patients is about 20%", and its sample-size table plans glucose clusters about a quarter the size of the temperature clusters (59 vs 239 cases, a ratio of 0.247). The reported denominators are instead about two-thirds of all analyzed cases (0.668 and 0.664).
Three consequences follow from the published numbers alone.
- The estimate does not describe the stated population. At least 34% of each denominator cannot have diabetes, and about 67% did not if only documented diabetes is counted. Glucose above 180 mg/dL on arrival in recovery is presumably much less common in patients without diabetes. Including them raises the avoidance rate (85.8%) and, if their rates are similar between arms, pulls any between-arm ratio toward 1. The diabetes-only rates and RR cannot be recovered from the published tables.
- The "tight confidence bounds" depend on the larger denominator. With the reported denominators, the crude unclustered 95% CI for the RR is 0.989–1.013, which reproduces the published 0.99–1.01. Two illustrations show the scale of the change; neither is an estimate of the true result. (a) With denominators the size of the documented diabetic population (2824 and 2909) and no difference between arms, the same calculation gives 0.979–1.021 at an 85.8% avoidance rate and 0.959–1.043 at 60%, about 1.7 to 3.5 times as wide. (b) If the cases without diabetes had the same avoidance rate q in both arms and every documented diabetic case were in the denominator, the implied diabetes-only avoidance would be 77.1% vs 77.1% at q = 0.90 and 58.5% vs 58.9% at q = 0.99. The RR would stay between 0.994 and 1.000, with unclustered 95% CIs from 0.972–1.029 to 0.951–1.038, 2.3 to 3.6 times as wide. All of these come before the clustering and Bonferroni-corrected α that the paper applies to its CIs, which widen them further.
- The explanation offered for the null result is affected. The Discussion attributes the null results partly to high baseline adherence. For this outcome, the 85.8% baseline describes the mixed population and has not been shown for patients with diabetes; in the scenario above it would be closer to 59–77%.
The trial's other primary outcome, hypothermia, does not depend on this issue. Most secondary outcomes are null and, apart from those limited to diabetes, are not affected either. The scenario in point 2 suggests the null result for hyperglycemia may well survive a correct analysis, but with a substantially wider interval. As published, the hyperglycemia result cannot be read as a finding about patients with diabetes, and the "high confidence" statement in the Abstract rests partly on it.
Explanations considered
- Diabetes is under-ascertained in Table 1 because of missing status. This does not account for the numbers. The bound already counts all 2889 and 2978 cases with unknown status as diabetic, and the denominators still exceed it by about 2950 per arm.
- The tables use different units (cases vs patients). This does not account for the numbers. Table 1, Table 2 and Figure 1 all use the same totals (12 980 and 13 274), and the sensitivity analysis that keeps only one operation per 30-day window shows the same excess.
- The outcome used a broader definition of diabetes than Table 1. This is not plausible. The denominators would require two-thirds of all surgical cases to have diabetes, against 28% in Table 1 and the roughly 20% the protocol assumed.
- Figure 1's "without glucose data" counts already exclude cases without diabetes. This does not account for the numbers. If they did, the 8676 and 8815 cases with glucose data would all have diabetes, which contradicts Table 1's maximum of 5713 and 5887.
- A transcription or typesetting error in Table 2. This is unlikely. The same plain integers appear in the Abstract, the Results text and Table 2, they match the flowchart arithmetic exactly, and the two separately computed sensitivity analyses show the same excess. The publisher's page carries the same values as the PMC archive, so a conversion artifact is excluded.
- The analysis followed the registered definition, and the text is what is wrong. This is possible, and it would account for the numbers. ClinicalTrials.gov (NCT02830126, last update posted 2020-01-22) registers the outcome as "Proportion of cases with blood glucose <=180 mg/dL upon arrival to the anesthesia recovery area", with no diabetes restriction. The protocol's outcome table (F1000Research 2018;7:623, Table 1) is worded similarly ("Proportion of cases with blood glucose ≥180 mg/dl upon arrival to the post-anesthesia recovery area"). Its power calculation, however, is framed around patients with diabetes, as quoted above. If every case with a glucose value was meant to be analyzed, then the Abstract and Methods misdescribe the outcome population, the sentence "patients without diabetes were excluded from analysis" is incorrect for this outcome, and the departure from the protocol's diabetic framing is not reported. Either way, the paper's stated population and its reported numbers cannot both be right, and the published material does not show which one is in error.
- Glucose is routinely measured in patients without diabetes. This is consistent with the numbers and would explain how two-thirds of all cases came to have a glucose value. It does not reconcile that set with the stated restriction.
Status
- Version of record. The publisher's full text (jamanetwork.com, article 2809789, reached from the DOI) shows the same Study Population sentence, the same Table 1 diabetes row (2909/10 296 and 2824/10 091) and the same Table 2 hyperglycemia row (7445/8676, 7559/8815; 1.00, 0.99 to 1.01) as the PMC archive. I also checked the Europe PMC full-text XML and the publisher's Figure 1 image. Every quote above was confirmed verbatim against the PMC-archived text, except the Figure 1 flowchart text, which was read from the image.
- Supplements. I opened and text-extracted all four supplements from the Europe PMC supplementary-files bundle. Supplement 1 is the trial protocol; it does not define the glycemic primary outcome numerically. Supplement 2 holds eTable 1 (outcome definitions, which cover the secondary and clinical outcomes but not the two primary outcomes) and the sensitivity analyses in eTables 4–7, used above. Supplements 3 (nonauthor collaborators) and 4 (data sharing statement) are not relevant.
- Protocol and registry. I checked the published protocol (reference 22 cites version 1, doi:10.12688/f1000research.14897.1; I read version 2 as archived in PMC, PMC6039946) and the ClinicalTrials.gov record NCT02830126 through its API.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction or comment links (PMID 37738052). The publisher's page shows no correction notice and no reader comments. The PubPeer API returned no feedback for the DOI; the PubPeer search page itself refused automated access (HTTP 403). A general web search could not be run in this session.
- Sources. PMC article PMC10517374; publisher page https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2809789; Supplement 2 (jamanetwopen-e2332517-s002.pdf); Figure 1 image (jamanetwopen-e2332517-g001.jpg); F1000Research protocol (PMC6039946); ClinicalTrials.gov NCT02830126; Crossref; Europe PMC; PubPeer.
Suggested correction
- Re-run the hyperglycemia primary outcome restricted to cases with diabetes, as the Methods state, and report the numerators, denominators, RR, Bonferroni-adjusted CI and Holm-adjusted P value. Update the Abstract, Key Points, Results text, Table 2, eTables 5 and 7, and Figure 2 if it uses the same population. Also report how many cases with diabetes had no glucose value.
- Alternatively, if the intended population was all cases with a postoperative glucose value, as in the registry wording, revise the outcome definition in the Abstract and Methods. Remove or qualify the sentence "For glycemic outcomes, patients without diabetes were excluded from analysis", report the change from the protocol's diabetic framing, and give the diabetes-only result as a secondary analysis.
- In either case, revisit the "high confidence" and "tight confidence bounds" statements as they apply to hyperglycemia.
- Minor: correct the swapped arm sizes in the Abstract; explain why the hypothermia denominators (11 447 / 11 672) differ from Figure 1 (11 399 / 11 622); change 68.7 to 68.6 for 1346/1962 if it is a crude proportion; and fix "[66.7.%]".
Recomputation script (python recompute.py.txt)
"""Recompute the hyperglycemia primary-outcome denominators of PMC10517374
(JAMA Netw Open 2023, doi:10.1001/jamanetworkopen.2023.32517).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC and from Supplement 2:
Figure 1 (flowchart image): analyzed cases and cases without temperature / glucose data
Table 1: cases with diabetes (No./total No. with known status)
Table 2: primary and secondary outcome counts
eTables 4-7 (Supplement 2): the two sensitivity analyses
Unit: the paper counts surgical cases (Table 1 and Table 2 use the same n per arm).
"""
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
ARMS = ("intervention", "control")
N = {"intervention": 12980, "control": 13274} # Figure 1, Table 1, Table 2
NO_GLUCOSE = {"intervention": 4304, "control": 4459} # Figure 1 "without glucose data"
NO_TEMP = {"intervention": 1581, "control": 1652} # Figure 1 "without temperature data"
DIABETES = {"intervention": (2824, 10091), "control": (2909, 10296)} # Table 1 "Diabetes"
HYPERGLY = {"intervention": (7445, 8676), "control": (7559, 8815)} # Table 2 primary
HYPOTHERM = {"intervention": (7602, 11447), "control": (7783, 11672)} # Table 2 primary
GLU_MEAS = {"intervention": (1346, 1962), "control": (1274, 1996)} # Table 2 secondary
GLU_MEAS_PRINTED_PCT = {"intervention": 68.7, "control": 63.8}
# Sensitivity analyses (Supplement 2). Diabetes is printed as n (%) without a denominator.
SENS = {
"Excluding repeat surgeries (eTables 4-5)": {
"N": {"intervention": 12179, "control": 12457},
"dm": {"intervention": (2614, 27.6), "control": (2667, 27.6)},
"hyper": {"intervention": (6923, 8069), "control": (6989, 8169)},
},
"Ignoring prior assignment (eTables 6-7)": {
"N": {"intervention": 12965, "control": 13289},
"dm": {"intervention": (2844, 28.2), "control": (2889, 28.0)},
"hyper": {"intervention": (7469, 8693), "control": (7535, 8798)},
},
}
# Published protocol (ref 22, F1000Res 2018;7:623), sample-size table: cluster sizes
PROTO_CLUSTER = {"temperature": 239, "glucose": 59}
PROTO_DM_PREV = 0.20
def pct(a, b):
return 100.0 * a / b
def rr_ci(a1, n1, a0, n0, z=1.959964):
rr = (a1 / n1) / (a0 / n0)
se = math.sqrt(1 / a1 - 1 / n1 + 1 / a0 - 1 / n0)
return rr, math.exp(math.log(rr) - z * se), math.exp(math.log(rr) + z * se)
R = {} # results reused by the figures
print("=== 1. Hyperglycemia denominator vs the population it is defined for ===")
for g in ARMS:
d, known = DIABETES[g]
unknown = N[g] - known
max_dm = d + unknown
den = HYPERGLY[g][1]
with_glu = N[g] - NO_GLUCOSE[g]
R[g] = dict(d=d, unknown=unknown, max_dm=max_dm, den=den, with_glu=with_glu)
print(f"\n[{g}] n = {N[g]}")
print(f" Table 1 diabetes: {d}/{known} = {pct(d, known):.1f}%; status unknown: {N[g]} - {known} = {unknown}")
print(f" most cases that could have diabetes (all unknown counted as diabetes): {d} + {unknown} = {max_dm}")
print(f" Table 2 hyperglycemia denominator: {den}")
print(f" denominator - maximum possible diabetic cases = {den - max_dm}")
print(f" denominator / documented diabetic cases = {den / d:.2f}")
print(f" Figure 1: {N[g]} - {NO_GLUCOSE[g]} without glucose data = {with_glu} -> equals denominator: {with_glu == den}")
print(f" denominator as share of all analyzed cases: {pct(den, N[g]):.1f}%")
print(f" share of denominator that cannot have diabetes (lower bound): ({den} - {max_dm}) / {den} = {pct(den - max_dm, den):.1f}%")
print(f" share of denominator without documented diabetes if all {d} diabetic cases had glucose data: "
f"({den} - {d}) / {den} = {pct(den - d, den):.1f}%")
print("\n=== 2. Same check in the two sensitivity analyses (Supplement 2) ===")
SENS_R = {}
for name, v in SENS.items():
for g in ARMS:
d, p = v["dm"][g]
n = v["N"][g]
den = v["hyper"][g][1]
# Diabetes % is rounded to 0.1; the smallest known-status total consistent with it
known_min = math.ceil(d / ((p + 0.05) / 100))
max_dm = d + (n - known_min)
SENS_R[(name, g)] = dict(d=d, max_dm=max_dm, den=den)
print(f"{name} [{g}]: n={n}; diabetes {d} ({p}%) -> known-status total >= {known_min}; "
f"max possible diabetic cases <= {max_dm}; hyperglycemia denominator {den}; "
f"denominator - documented diabetes = {den - d}; denominator - max possible = {den - max_dm}")
print("\n=== 3. A diabetes-restricted glucose outcome for comparison (Table 2, secondary) ===")
for g in ARMS:
a, n = GLU_MEAS[g]
d = DIABETES[g][0]
print(f"{g}: intraoperative glucose measurement denominator {n} = {pct(n, d):.1f}% of {d} diabetic cases "
f"(fits inside the diabetic population: {n <= d})")
print("\n=== 4. Published protocol sample size (ref 22) ===")
print(f"glucose cluster size / temperature cluster size = {PROTO_CLUSTER['glucose']}/{PROTO_CLUSTER['temperature']} "
f"= {PROTO_CLUSTER['glucose'] / PROTO_CLUSTER['temperature']:.3f} (protocol assumed diabetes prevalence {PROTO_DM_PREV:.0%})")
print(f"reported denominators / analyzed cases: {HYPERGLY['intervention'][1] / N['intervention']:.3f}, "
f"{HYPERGLY['control'][1] / N['control']:.3f}")
print("\n=== 5. Table 2 arithmetic (crude, unclustered; the paper's CIs are cluster-robust and Bonferroni-adjusted) ===")
for name, dct in (("No postoperative hyperglycemia", HYPERGLY), ("No postoperative hypothermia", HYPOTHERM),
("Intraoperative glucose measurement", GLU_MEAS)):
a1, n1 = dct["intervention"]
a0, n0 = dct["control"]
rr, lo, hi = rr_ci(a1, n1, a0, n0)
print(f"{name}: {pct(a1, n1):.2f}% vs {pct(a0, n0):.2f}%; crude RR {rr:.3f} (unclustered 95% CI {lo:.3f}-{hi:.3f})")
for g in ARMS:
a, n = GLU_MEAS[g]
print(f"glucose measurement {g}: {a}/{n} = {pct(a, n):.3f}% -> one decimal {pct(a, n):.1f} (printed {GLU_MEAS_PRINTED_PCT[g]})")
print("\n=== 6. Precision at diabetes-sized denominators (illustration only) ===")
a1, n1 = HYPERGLY["intervention"]
a0, n0 = HYPERGLY["control"]
rr, lo, hi = rr_ci(a1, n1, a0, n0)
print(f"reported denominators {n1}/{n0}: unclustered 95% CI half-width on log scale {(math.log(hi) - math.log(lo)) / 2:.4f}")
for p in (0.858, 0.70, 0.60):
m1, m0 = DIABETES["intervention"][0], DIABETES["control"][0]
se = math.sqrt((1 - p) / (p * m1) + (1 - p) / (p * m0))
print(f" {m1}/{m0} cases at avoidance rate {p:.3f} and RR 1: unclustered 95% CI {math.exp(-1.96 * se):.3f}-{math.exp(1.96 * se):.3f}")
print("\n=== 6b. Illustrative diabetes-only scenario (assumption, not an estimate) ===")
print("Assume every documented diabetic case (Table 1) is in the denominator and the cases without")
print("diabetes avoid hyperglycemia at the same rate q in both arms; back out the diabetic-only rates.")
_rr0, _lo0, _hi0 = rr_ci(*HYPERGLY["intervention"], *HYPERGLY["control"])
REF_W = _hi0 - _lo0
for q in (0.90, 0.95, 0.97, 0.99):
a = {}
for g in ARMS:
num, den = HYPERGLY[g]
d = DIABETES[g][0]
a[g] = num - q * (den - d)
m1, m0 = DIABETES["intervention"][0], DIABETES["control"][0]
rr, lo, hi = rr_ci(a["intervention"], m1, a["control"], m0)
print(f" q = {q:.2f}: diabetic avoidance {pct(a['intervention'], m1):.1f}% vs {pct(a['control'], m0):.1f}%; "
f"RR {rr:.3f} (unclustered 95% CI {lo:.3f}-{hi:.3f}); CI width {hi - lo:.3f}, "
f"{(hi - lo) / REF_W:.1f} x the unclustered width at the reported denominators")
print(f" reference: unclustered CI width at reported denominators = {REF_W:.3f} ({_lo0:.3f}-{_hi0:.3f})")
print("\n=== 7. Other, smaller inconsistencies ===")
for g in ARMS:
exp_t = N[g] - NO_TEMP[g]
print(f"hypothermia {g}: Figure 1 gives {N[g]} - {NO_TEMP[g]} = {exp_t}; Table 2 denominator {HYPOTHERM[g][1]}; "
f"difference {HYPOTHERM[g][1] - exp_t}")
print("Abstract: 'control group; n = 12 980' and 'intervention group; n = 13 274'; Methods, Table 1, Table 2 and "
"Figure 1 give intervention 12 980 and control 13 274 (labels swapped in the abstract).")
print("Abstract prints '[66.7.%]' for the control hypothermia percentage.")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, CONTEXT = "#eb6834", "#2a78d6", "#e4e3df", "#d6d5d0"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
def fmt(n):
return f"{n:,}"
# Figure 1: counts per arm, horizontal bars on one count axis
rows = [] # (label, value, color, group)
for g in ARMS:
r = R[g]
rows += [
(f"All analyzed cases (Figure 1)", N[g], CONTEXT, g),
(f"Cases with glucose data: {fmt(N[g])} − {fmt(NO_GLUCOSE[g])} (Figure 1)", r["with_glu"], BLUE, g),
("Hyperglycemia denominator (Table 2)", r["den"], ORANGE, g),
(f"Most cases that could have diabetes:\n{fmt(r['d'])} + {fmt(r['unknown'])} status unknown (Table 1)", r["max_dm"], BLUE, g),
("Cases with documented diabetes (Table 1)", r["d"], BLUE, g),
]
fig, ax = plt.subplots(figsize=(7.2, 7.0))
BAR_H = 0.52
y, ys = 0.0, []
heads = {}
for i, (lab, val, col, g) in enumerate(rows):
if i % 5 == 0:
heads[g] = y
y += 0.8
ys.append(y)
y += 1.0
if i % 5 == 4:
y += 0.5
ys = [-v for v in ys]
for (lab, val, col, g), yy in zip(rows, ys):
ax.barh(yy, val, height=BAR_H, color=col, edgecolor="none", zorder=2)
ax.text(val + 150, yy, fmt(val), va="center", ha="left", fontsize=10, color=TXT, zorder=3)
for g, yy in heads.items():
ax.text(-200, -yy, f"{g.capitalize()} arm (n = {fmt(N[g])})", fontsize=11, fontweight="bold",
ha="right", va="center", color=TXT)
for i, g in enumerate(ARMS):
r = R[g]
y_den, y_max = ys[5 * i + 2], ys[5 * i + 3]
y_arrow = (y_den + y_max) / 2
ax.plot([r["max_dm"], r["max_dm"]], [y_max - BAR_H / 2, y_den + BAR_H / 2], color=TXT, lw=1.2,
ls=(0, (3, 2)), zorder=4)
ax.annotate("", xy=(r["den"], y_arrow), xytext=(r["max_dm"], y_arrow),
arrowprops=dict(arrowstyle="<->", color=TXT, lw=1.4, shrinkA=0, shrinkB=0), zorder=5)
ax.text(r["den"] + 150, y_arrow, f"{fmt(r['den'] - r['max_dm'])} more than\nthe maximum",
ha="left", va="center", fontsize=10, color=TXT, zorder=5, linespacing=1.0)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xlim(0, 15800)
ax.set_xticks(range(0, 12001, 4000))
ax.set_xlabel("Surgical cases")
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, _: fmt(int(v))))
from matplotlib.patches import Patch # noqa: E402
fig.legend(handles=[Patch(color=ORANGE, label="Reported denominator of the diabetes-only primary outcome (Table 2)"),
Patch(color=BLUE, label="Implied by the paper's other numbers (Table 1, Figure 1)"),
Patch(color=CONTEXT, label="All analyzed cases (context)")],
loc="lower left", bbox_to_anchor=(0.02, 0.005), frameon=False, fontsize=10, ncol=1)
fig.tight_layout(rect=(0, 0.11, 1, 1))
save(fig, "fig1")
# Figure 2: the same gap in all three published analyses
items = [("Main analysis (Tables 1-2)", g, R[g]["d"], R[g]["max_dm"], R[g]["den"]) for g in ARMS]
for name in SENS:
for g in ARMS:
s = SENS_R[(name, g)]
items.append((name, g, s["d"], s["max_dm"], s["den"]))
fig, ax = plt.subplots(figsize=(7.2, 5.0))
yy = []
y = 0
last = None
for name, g, d, mx, den in items:
if last is not None and name != last:
y += 0.6
yy.append(-y)
last = name
y += 1
for (name, g, d, mx, den), yv in zip(items, yy):
ax.plot([d, den], [yv, yv], color=CONTEXT, lw=2, zorder=1)
ax.plot([d], [yv], "o", ms=9, mfc=BG, mec=BLUE, mew=2, zorder=3)
ax.plot([mx], [yv], "D", ms=8, color=BLUE, zorder=3)
ax.plot([den], [yv], "o", ms=9, color=ORANGE, zorder=3)
ax.text(den + 200, yv, f"+{fmt(den - mx)}", va="center", fontsize=10, color=TXT)
ax.set_yticks(yy)
ax.set_yticklabels([f"{n.split(' (')[0]}, {g}" for n, g, *_ in items], fontsize=10)
ax.set_xlim(0, 10600)
ax.set_xticks(range(0, 10001, 2000))
ax.set_xlabel("Surgical cases")
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, _: fmt(int(v))))
from matplotlib.lines import Line2D # noqa: E402
fig.legend(handles=[
Line2D([], [], marker="o", ls="none", ms=9, mfc=BG, mec=BLUE, mew=2, label="Cases with documented diabetes (Table 1, eTables 4, 6)"),
Line2D([], [], marker="D", ls="none", ms=8, color=BLUE, label="Most cases that could have diabetes (recomputed)"),
Line2D([], [], marker="o", ls="none", ms=9, color=ORANGE, label="Reported hyperglycemia denominator (Table 2, eTables 5, 7)"),
Line2D([], [], ls="none", label="+n: reported denominator minus the maximum possible"),
], loc="lower left", bbox_to_anchor=(0.02, 0.005), frameon=False, fontsize=10, ncol=1)
fig.tight_layout(rect=(0, 0.2, 1, 1))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Subgroup effects called significantly different although their 95% CIs overlap, contrary to the paper's criterion
JAMA Network Open (2024) · doi:10.1001/jamanetworkopen.2023.56619 · PMC10891466
Summary
The paper asks whether food insecurity modifies the association between SNAP receipt and nonadherence to antihypertensive medication. Its Methods say two subgroup estimates count as significantly different only if their 95% CIs do not overlap, and its Results then call the food-insecure estimate (−13.6 percentage points [pp]; 95% CI −25.0 to −2.3) and the food-secure estimate (5.0 pp; 95% CI −15.8 to 25.7) "statistically significantly different". Those intervals overlap from −15.8 to −2.3, and a Wald test on the same intervals gives a difference of 18.6 pp (95% CI −5.1 to 42.3; p = 0.12). The paper answers yes to its own effect-modification question, but neither its stated criterion nor a conventional test supports that answer. The food-insecure estimate on its own is not affected.
What the paper reports
The research question (Abstract, Key Points, Question):
"Is receipt of Supplemental Nutrition Assistance Program (SNAP) benefits associated with nonadherence to antihypertensive medications and, if so, does this association vary by food insecurity status?"
The criterion for effect modification (Methods, Statistical Analysis, second paragraph):
"The potential effect modification of food insecurity was tested by evaluating for statistical differences in the magnitude and direction of the PATEs for the food-secure and food-insecure subgroups; PATEs were considered statistically significantly different if their respective 95% CIs did not overlap."
The two subgroup estimates (Results, "Association Between SNAP Receipt and Nonadherence to Antihypertensive Medications"):
"Similarly, the rate of nonadherence to antihypertensive medication was similar (385 [56.7% weighted] vs 2671 [55.2% weighted]; PATE, 5.0 [95% CI, -15.8 to 25.7]) between SNAP recipients and nonrecipients among the 5354 patients who did not report experiencing food insecurity."
"In contrast, among the subgroup of 1338 antihypertensive medication users who experienced food insecurity, the rate of nonadherence to antihypertensive medications was 8.17 percentage points lower among SNAP recipients compared with nonrecipients (302 [56.0% weighted] vs 491 [60.0% weighted]; PATE, -13.6 [95% CI, -25.0 to -2.3])."
The conclusion drawn from them (Results, same paragraph, the sentence that follows):
"The observed PATE estimates among the food-secure vs food-insecure subgroups were statistically significantly different, suggesting that food insecurity modified the association between SNAP and nonadherence to antihypertensive medication therapy."
Table 2 gives the same values: food insecure, 302 (56.0) vs 491 (60.0), PATE −13.6 (−25.0 to −2.3); food secure, 385 (56.7) vs 2671 (55.2), PATE 5.0 (−15.8 to 25.7); overall, 687 (56.4) vs 3162 (55.8), PATE 1.7 (−21.2 to 24.6).
The finding is carried into the Discussion (first paragraph):
"Among a US sample of patients with hypertension identified from a linked, nationally representative dataset, we observed that SNAP had a differential association with nonadherence to antihypertensive medications between food-secure and food-insecure subgroups."
and into the Strengths (Discussion, Strengths and Limitations):
"Second, this study presented data suggesting that the potential consequences of SNAP receipt on adherence to antihypertensive medications are greatest among the food-insecure population. Thus, we have identified the food-insecure hypertension population as a target for food assistance intervention to address adherence to antihypertensive medications, especially when food assistance resources are limited."
(Quotes are from the PubMed Central archive, which prints minus signs as hyphens; the publisher's page prints the same values with true minus signs.)
What the paper's own numbers show
Method. All inputs come from Table 2 and the Results text. (1) The paper's criterion is applied as written: do the two reported 95% CIs overlap? (2) Each standard error (SE) is recovered from its CI as (upper − lower)/(2 × 1.96). Both CIs are symmetric about their point estimates, so this is valid. (3) The two subgroups are disjoint sets of participants, and the Methods describe a stratified analysis with subgroup-specific propensity scores. The difference is therefore tested with the usual Wald test for two independent estimates, z = (5.0 − (−13.6)) / √(SE₁² + SE₂²). (4) To check whether a different confidence level would rescue the claim, the confidence level at which the two intervals just stop overlapping is also computed. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Food-insecure PATE (95% CI) | Table 2; Results; Abstract | −13.6 (−25.0 to −2.3) | Symmetric (half-widths 11.4 / 11.3); SE 5.79 pp; single-group p = 0.019 |
| Food-secure PATE (95% CI) | Table 2; Results | 5.0 (−15.8 to 25.7) | Symmetric (half-widths 20.8 / 20.7); SE 10.59 pp |
| Do the two 95% CIs overlap? (paper's criterion) | Methods vs Results | "statistically significantly different" | Yes, they overlap, from −15.8 to −2.3 (13.5 pp). The criterion is not met |
| Food-insecure point estimate inside the food-secure 95% CI? | Table 2 | — | Yes (−13.6 lies within −15.8 to 25.7) |
| Difference, food secure − food insecure | derived from Table 2 | "statistically significantly different" | 18.6 pp, SE 12.07, 95% CI −5.1 to 42.3 (includes 0) |
| Wald test of the difference | derived from Table 2 | no test statistic given | z = 1.54; two-sided p = 0.123 (one-sided p = 0.062) |
| Confidence level at which the two CIs stop overlapping | derived | — | Only below 74.4%. The 90%, 85.1%, 83.4%, 80% and 75% intervals all overlap |
| Confidence level at which non-overlap would match p < 0.05 for these SEs | derived | — | 85.1%. The intervals overlap at that level too |
| Correlation between the subgroup estimates needed for p = 0.05 | derived | — | 0.45 |
| Nonadherent counts | Table 2; Results | 687; 3162; 3849 | 302 + 385 = 687; 491 + 2671 = 3162; 687 + 3162 = 3849 (consistent) |
| Subgroup sizes | Results | 5354 food secure; 1338 food insecure | 6692 − 1338 = 5354; 540 + 798 = 1338 (consistent) |
| Food-insecure gap described in the text | Results | "8.17 percentage points lower" | Weighted crude gap 56.0 − 60.0 = −4.0 pp; unweighted 55.9% − 61.5% = −5.6 pp; adjusted PATE −13.6 pp. 8.17 matches none of them (separate, minor) |
| Model behind the PATEs | Methods and Abstract vs Table 2 footnote b | Methods and Abstract: probit; footnote b: "from a logit model" | Internal inconsistency in the description (minor; does not change the CIs) |
The counts and sample sizes are mutually consistent, so the problem is not in the reported data. It lies only in the inference drawn from the two intervals.
Figure
Figure 1. All values are on one percentage-point axis (SNAP recipients minus nonrecipients; negative means less nonadherence with SNAP). Orange: the two subgroup PATEs and 95% CIs as reported in Table 2. The shaded band marks where the two intervals overlap (−15.8 to −2.3, 13.5 pp); under the paper's stated rule, any overlap means the estimates are not significantly different. Blue: the difference between subgroups (food secure − food insecure), recomputed from the same table, 18.6 pp (95% CI −5.1 to 42.3; z = 1.54; p = 0.12). The thin vertical line labelled "null (0)" marks no effect for the subgroup rows and no difference for the bottom row; the interval for the difference includes it.
Figure 2. Would a different confidence level change the conclusion? Blue curves: the food-insecure upper bound (solid) and the food-secure lower bound (dashed), recomputed from the SEs implied by Table 2 for confidence levels from 50% to 99%. Orange points: the reported 95% bounds (−2.3 and −15.8). The intervals overlap (shaded) at every level above 74.4%. That includes 85.1%, the level at which non-overlap of these two particular intervals would correspond to a two-sided Wald p < 0.05 (dotted line).
Why it matters
Effect modification by food insecurity is the paper's stated question. It is the second half of the Key Points question, it is the Objective ("To evaluate whether receipt of SNAP benefits can modify the consequences of food insecurity on nonadherence to antihypertensive medications."), and it is the purpose of the prespecified stratified analysis. The claim that the association differs by food-insecurity status underpins:
- the Key Points Findings and the Abstract Results ("among the food-insecure subgroup but not among their food-secure counterparts");
- the Key Points Meaning ("especially among food-insecure individuals") and the Abstract and main Conclusions ("especially if they are experiencing food insecurity");
- the opening of the Discussion ("a differential association ... between food-secure and food-insecure subgroups");
- the Strengths paragraph, which says the consequences of SNAP receipt are "greatest among the food-insecure population" and names that population "as a target for food assistance intervention".
On the paper's own numbers, the data are compatible with no difference between the subgroups: the difference has a 95% CI of −5.1 to 42.3 pp. A result that is significant in one subgroup and not in the other is not evidence that the subgroups differ, and the paper's non-overlap rule, conservative as it is, was the guard it set against that inference. What survives is the food-insecure estimate on its own (−13.6 pp, CI excluding 0, p ≈ 0.02 before any adjustment for multiple subgroups). The literal wording "associated ... among the food-insecure subgroup but not among their food-secure counterparts" therefore remains accurate as a description of the two subgroup estimates. The statement that food insecurity modified the association, the "differential association", and the targeting recommendations built on it are not supported. The hedged overall conclusion ("may be less likely ... especially if") is weakened rather than reversed.
Explanations considered
- The table and the text give different numbers. They do not. Table 2, the Results text and the Abstract all give −13.6 (−25.0 to −2.3); Table 2 and the Results give 5.0 (−15.8 to 25.7). Neither is a typo of the other.
- A sign was lost in conversion (for example, −2.3 was meant to be +2.3). No. Both intervals are symmetric about their point estimates (half-widths 11.4/11.3 and 20.8/20.7), and the publisher's own page prints these values with true minus signs. A positive upper bound would also make the food-insecure estimate itself non-significant, which contradicts the Abstract.
- Different estimands, scales or models. Both subgroup PATEs are average marginal effects on the percentage-point scale from the same weighted model specification. The probit/logit wording differs between the Methods and the Table 2 footnote, but that affects both subgroups alike and does not change the printed CIs.
- "Magnitude and direction" means opposite-signed point estimates are enough. The Methods attach that phrase to a statistical test with a specific rule (non-overlap of 95% CIs). Opposite signs of the point estimates do not satisfy that rule; the food-secure interval (−15.8 to 25.7) contains the food-insecure point estimate.
- A formal interaction test was run but not reported. The Methods name no such test, and Supplement 1 (eAppendix 2, the weighting and regression specification) contains no interaction term, heterogeneity test or other criterion. An unreported test cannot be ruled out, but the stated criterion is not met and a Wald test on the published intervals gives p = 0.12.
- A one-sided test, or a lower confidence level. Nothing in the paper states either. One-sided p = 0.062. The intervals separate only at confidence levels below 74.4% (Figure 2).
- The subgroup estimates are correlated, so the Wald SE is too large. For p = 0.05 the two estimates would need a correlation of about 0.45. They come from disjoint sets of participants with separately fitted weights; shared survey strata and clusters can induce some covariance, but one of that size is not plausible. In any case, the paper's own non-overlap criterion does not depend on the correlation.
- The error was corrected after publication. No correction, erratum or retraction was found (see Status).
One further observation does not change the conclusion. The overall-sample interval (1.7, −21.2 to 24.6; half-width 22.9) is about twice as wide as the food-insecure interval (half-width 11.4), although it covers five times as many participants. Without data or code this cannot be examined further; the analysis above uses the subgroup intervals exactly as published.
Status
As of 2026-10-23:
- Version of record. The publisher's page (doi.org redirects to jamanetwork.com, article 2815447) was read on 2026-10-23. It contains the criterion sentence ("PATEs were considered statistically significantly different if their respective 95% CIs did not overlap."), the "statistically significantly different, suggesting that food insecurity modified the association" sentence, the "8.17 percentage points lower" sentence, and the Table 2 rows food insecure −13.6 (−25.0 to −2.3) and food secure 5.0 (−15.8 to 25.7). The PMC archive (
src/show.py) and the Europe PMC full-text XML carry the same sentences and numbers, and the "from a logit model" footnote. The problem is in the version of record, not introduced by conversion. - Supplements. Supplement 1 (eAppendices 1 and 2, eFigure, eTable, eReferences) contains no subgroup comparison, interaction term or additional subgroup estimates. Supplement 2 is the Data Sharing Statement ("Data available: No"). Neither resolves the discrepancy, and no data or code are available to re-estimate the subgroup models.
- Corrections and comments. Crossref update metadata, PMC notices and Europe PMC notices show no correction, erratum, expression of concern or retraction; Europe PMC searches for an erratum in JAMA Network Open returned no hits. The only linked item is an invited commentary (PMID 38393734, "Food Security to Medication Adherence-Connecting Needs"), which is not a correction. The publisher page shows no correction notice. PubPeer could not be checked (its API and search page returned HTTP 403/404 to automated requests), and a general web search was not available in this session.
- Sources. https://doi.org/10.1001/jamanetworkopen.2023.56619 ; https://pmc.ncbi.nlm.nih.gov/articles/PMC10891466/ ; https://www.ebi.ac.uk/europepmc/webservices/rest/PMC10891466/fullTextXML ; Crossref REST API record for the DOI.
Suggested correction
- Replace "The observed PATE estimates among the food-secure vs food-insecure subgroups were statistically significantly different, suggesting that food insecurity modified the association …" with a statement that matches the stated criterion, for example: "The 95% CIs for the food-secure and food-insecure subgroups overlapped; by our prespecified criterion, the PATEs were therefore not statistically significantly different, and we did not find statistical evidence that food insecurity modified the association."
- Preferably also report a direct estimate of the difference between subgroups (an interaction model, or a test of the difference between the two PATEs), with its CI and p value. From the published intervals this is approximately 18.6 pp (95% CI −5.1 to 42.3; p ≈ 0.12).
- Revise the Key Points (Findings, Meaning), the Abstract Results and Conclusions, the opening of the Discussion, the Strengths paragraph and the Conclusions so that they report the food-insecure estimate without presenting a difference between subgroups as established.
- Correct "8.17 percentage points lower" to agree with Table 2 (adjusted PATE −13.6; weighted crude difference −4.0), and make footnote b of Table 2 ("from a logit model") agree with the probit model described in the Methods and the Abstract, or vice versa.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC10891466 write-up and regenerate its figures.
Paper: "Supplemental Nutrition Assistance Program and Adherence to
Antihypertensive Medications", JAMA Network Open 2024,
doi:10.1001/jamanetworkopen.2023.56619 (PMC10891466).
All inputs are copied from Table 2 and the Results text of the paper.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
Figures are written next to this file as fig1.svg/png and fig2.svg/png.
"""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
# Table 2: PATE, percentage-point difference (95% CI)
ROWS = {
"overall": (1.7, -21.2, 24.6),
"food_insecure": (-13.6, -25.0, -2.3),
"food_secure": (5.0, -15.8, 25.7),
}
# Table 2: nonadherent, No. (weighted %), SNAP vs no SNAP
COUNTS = {
"overall": ((687, 56.4), (3162, 55.8)),
"food_insecure": ((302, 56.0), (491, 60.0)),
"food_secure": ((385, 56.7), (2671, 55.2)),
}
N_TOTAL, N_FI, N_FS = 6692, 1338, 5354 # Results text
N_FI_SNAP, N_FI_NOSNAP = 540, 798 # Results text
NONADH_TOTAL = 3849 # Results text
TEXT_FI_GAP = 8.17 # Results text, "8.17 percentage points lower"
Z975 = norm.ppf(0.975)
def section(title):
print("\n" + "=" * 72 + "\n" + title + "\n" + "=" * 72)
# ------------------------------------------- 1. SEs implied by the CIs
section("1. Standard errors implied by the reported 95% CIs")
SE = {}
for k, (est, lo, hi) in ROWS.items():
SE[k] = (hi - lo) / (2 * Z975)
print(f"{k:14s} PATE {est:6.1f} CI [{lo:6.1f}, {hi:5.1f}] "
f"lower half-width {est - lo:5.1f} upper half-width {hi - est:5.1f} "
f"midpoint {(lo + hi) / 2:6.2f} SE {SE[k]:6.2f} "
f"single-group two-sided p {2 * norm.sf(abs(est) / SE[k]):.3f}")
print("-> Each CI is symmetric about its estimate (to rounding), so no minus sign is lost.")
fi, fs = ROWS["food_insecure"], ROWS["food_secure"]
# ------------------------------------------- 2. The paper's own criterion
section("2. The paper's own criterion: do the two 95% CIs overlap?")
ov_lo, ov_hi = max(fi[1], fs[1]), min(fi[2], fs[2])
print(f"Overlap interval: [{ov_lo}, {ov_hi}] overlap = {ov_lo < ov_hi} width = {ov_hi - ov_lo:.1f} pp")
print(f"Food-insecure estimate {fi[0]} inside food-secure CI [{fs[1]}, {fs[2]}]: {fs[1] <= fi[0] <= fs[2]}")
print(f"Food-secure estimate {fs[0]} inside food-insecure CI [{fi[1]}, {fi[2]}]: {fi[1] <= fs[0] <= fi[2]}")
print(f"Food-secure CI covers 0: {fs[1] <= 0 <= fs[2]}; food-insecure CI covers 0: {fi[1] <= 0 <= fi[2]}")
# ------------------------------------------- 3. Wald test of the difference
section("3. Wald test of the difference between the subgroup PATEs (independent strata)")
d = fs[0] - fi[0]
se_d = np.hypot(SE["food_insecure"], SE["food_secure"])
z = d / se_d
p2 = 2 * norm.sf(abs(z))
p1 = norm.sf(abs(z))
d_lo, d_hi = d - Z975 * se_d, d + Z975 * se_d
print(f"Difference (food secure - food insecure) = {d:.1f} pp")
print(f"SE = sqrt({SE['food_insecure']:.2f}^2 + {SE['food_secure']:.2f}^2) = {se_d:.2f} pp")
print(f"z = {z:.2f}; two-sided p = {p2:.3f}; one-sided p = {p1:.3f}")
print(f"95% CI for the difference: {d_lo:.1f} to {d_hi:.1f} pp (includes 0: {d_lo <= 0 <= d_hi})")
# correlation between the two estimates that would be needed for p = 0.05
se_needed = d / Z975
rho = (SE["food_insecure"] ** 2 + SE["food_secure"] ** 2 - se_needed ** 2) / (
2 * SE["food_insecure"] * SE["food_secure"])
print(f"For two-sided p = 0.05 the SE of the difference would have to be {se_needed:.2f}, "
f"i.e. a correlation of {rho:.2f} between the two subgroup estimates "
f"(they come from disjoint groups of participants, fitted separately).")
# ------------------------------------------- 4. Narrower CI levels
section("4. At which confidence level would the intervals stop overlapping?")
for lvl in (0.95, 0.90, 0.851, 0.834, 0.80, 0.75):
zz = norm.ppf(0.5 + lvl / 2)
fi_hi = fi[0] + zz * SE["food_insecure"]
fs_lo = fs[0] - zz * SE["food_secure"]
print(f"{lvl * 100:5.1f}% CIs: food-insecure upper {fi_hi:6.2f}, food-secure lower {fs_lo:6.2f} "
f"-> overlap = {fs_lo < fi_hi}")
z_touch = d / (SE["food_insecure"] + SE["food_secure"])
lvl_touch = 2 * norm.cdf(z_touch) - 1
print(f"The intervals just touch at the {100 * lvl_touch:.1f}% level (z = {z_touch:.2f}).")
# level at which 'non-overlap' would coincide with a two-sided p = 0.05 Wald test, for these SEs
z_eq = Z975 * se_d / (SE["food_insecure"] + SE["food_secure"])
lvl_eq = 2 * norm.cdf(z_eq) - 1
print(f"For these two SEs, non-overlap of {100 * lvl_eq:.1f}% CIs is equivalent to two-sided p < 0.05; "
f"the intervals still overlap at that level ({100 * lvl_touch:.1f}% < {100 * lvl_eq:.1f}%).")
# ------------------------------------------- 5. Other numbers in the paragraph
section("5. Other numbers in the same Results paragraph")
s_snap = COUNTS["food_insecure"][0][0] + COUNTS["food_secure"][0][0]
s_nos = COUNTS["food_insecure"][1][0] + COUNTS["food_secure"][1][0]
print(f"SNAP nonadherent 302 + 385 = {s_snap} (reported {COUNTS['overall'][0][0]})")
print(f"No-SNAP nonadherent 491 + 2671 = {s_nos} (reported {COUNTS['overall'][1][0]})")
print(f"Total nonadherent 687 + 3162 = {687 + 3162} (reported {NONADH_TOTAL})")
print(f"Food-secure n = 6692 - 1338 = {N_TOTAL - N_FI} (reported {N_FS})")
print(f"Food-insecure n = 540 + 798 = {N_FI_SNAP + N_FI_NOSNAP} (reported {N_FI})")
raw_fi = COUNTS["food_insecure"][0][1] - COUNTS["food_insecure"][1][1]
raw_fs = COUNTS["food_secure"][0][1] - COUNTS["food_secure"][1][1]
print(f"Food-insecure weighted raw difference 56.0 - 60.0 = {raw_fi:.1f} pp; "
f"adjusted PATE {fi[0]} pp; text says {TEXT_FI_GAP} pp lower "
f"(matches neither: {abs(raw_fi) != TEXT_FI_GAP and abs(fi[0]) != TEXT_FI_GAP})")
print(f"Food-secure weighted raw difference 56.7 - 55.2 = {raw_fs:.1f} pp; adjusted PATE {fs[0]} pp")
uw_snap = 100 * COUNTS["food_insecure"][0][0] / N_FI_SNAP
uw_nos = 100 * COUNTS["food_insecure"][1][0] / N_FI_NOSNAP
print(f"Food-insecure unweighted: 302/540 = {uw_snap:.1f}%, 491/798 = {uw_nos:.1f}%, "
f"difference {uw_snap - uw_nos:.1f} pp (also not 8.17)")
# ------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans",
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK, "text.color": INK, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG,
})
def fmt(v):
return f"{v:.1f}".replace("-", "\u2212")
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), bbox_inches="tight")
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, bbox_inches="tight")
plt.close(fig)
# Figure 1: the two reported subgroup CIs and the recomputed difference, one axis
fig, ax = plt.subplots(figsize=(7.2, 3.9))
rows = [
("Food insecure\n(Table 2, reported)", fi[0], fi[1], fi[2], ORANGE, "o"),
("Food secure\n(Table 2, reported)", fs[0], fs[1], fs[2], ORANGE, "o"),
("Difference, secure − insecure\n(recomputed from Table 2)", d, d_lo, d_hi, BLUE, "D"),
]
ys = [2.0, 1.0, -0.3]
ax.axvspan(ov_lo, ov_hi, ymin=0.38, ymax=0.97, color="#f1d9cf", alpha=0.6, lw=0, zorder=0)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
for (lab, est, lo, hi, col, mk), y in zip(rows, ys):
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
ax.plot([lo, lo], [y - 0.09, y + 0.09], color=col, lw=2, zorder=3)
ax.plot([hi, hi], [y - 0.09, y + 0.09], color=col, lw=2, zorder=3)
ax.plot(est, y, marker=mk, ms=9, color=col, zorder=4)
ax.text(hi + 1.2, y, f"{fmt(est)} ({fmt(lo)} to {fmt(hi)})", va="center", fontsize=10, color=col,
bbox=dict(boxstyle="square,pad=0.15", fc=BG, ec="none"), zorder=5)
ax.text(4.0, 2.6,
f"95% CIs overlap from {fmt(ov_lo)} to {fmt(ov_hi)} ({ov_hi - ov_lo:.1f} pp):\nthe paper's criterion for 'different' is not met",
ha="center", va="center", fontsize=10, color=INK, bbox=dict(boxstyle="square,pad=0.2", fc=BG, ec="none"))
ax.text(d, -0.75, f"Wald z = {z:.2f}, two-sided p = {p2:.2f}; CI includes 0",
ha="center", va="center", fontsize=10, color=BLUE, bbox=dict(boxstyle="square,pad=0.15", fc=BG, ec="none"), zorder=5)
ax.axhline(0.35, color=GRID, lw=1)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows])
ax.set_ylim(-1.0, 2.9)
ax.set_xlim(-30, 64)
ax.set_xlabel("Percentage-point difference in nonadherence, SNAP vs no SNAP (PATE)")
ax.xaxis.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.text(0.6, 0.45, "null (0)", fontsize=10, color=INK2, va="bottom")
save(fig, "fig1")
# Figure 2: interval bounds as a function of confidence level
lv = np.linspace(0.50, 0.99, 400)
zz = norm.ppf(0.5 + lv / 2)
fi_up = fi[0] + zz * SE["food_insecure"]
fs_lo = fs[0] - zz * SE["food_secure"]
fig, ax = plt.subplots(figsize=(7.2, 3.8))
ax.plot(lv * 100, fi_up, color=BLUE, lw=2, label="Food insecure: upper CI bound (recomputed)")
ax.plot(lv * 100, fs_lo, color=BLUE, lw=2, ls="--", label="Food secure: lower CI bound (recomputed)")
ax.plot([95], [fi[2]], "o", ms=9, color=ORANGE, zorder=5)
ax.plot([95], [fs[1]], "o", ms=9, color=ORANGE, zorder=5, label="Reported 95% CI bounds (Table 2)")
ax.annotate(fmt(fi[2]), (95, fi[2]), xytext=(-34, 4), textcoords="offset points", color=ORANGE, fontsize=10)
ax.annotate(fmt(fs[1]), (95, fs[1]), xytext=(-40, -4), textcoords="offset points", color=ORANGE, fontsize=10)
ax.fill_between(lv * 100, fs_lo, fi_up, where=fs_lo < fi_up, color="#f1d9cf", alpha=0.6, lw=0)
ax.axvline(lvl_touch * 100, color=INK2, lw=0.8)
ax.text(lvl_touch * 100 - 0.8, 4.0, f"intervals separate only at\nlevels below {lvl_touch * 100:.1f}%",
ha="right", va="center", fontsize=10, color=INK)
ax.axvline(lvl_eq * 100, color=INK2, lw=0.8, ls=":")
ax.text(lvl_eq * 100 + 0.5, 4.0, f"{lvl_eq * 100:.1f}%: non-overlap here\n\u21d4 Wald p < 0.05", ha="left", va="center",
fontsize=10, color=INK2)
ax.text(91.0, -9.5, "overlap", ha="center", fontsize=10, color=INK)
ax.set_xlim(50, 99)
ax.set_ylim(-27, 8)
ax.set_xlabel("Confidence level of both intervals (%)")
ax.set_ylabel("Percentage points")
ax.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.legend(loc="lower left", frameon=False, fontsize=10)
save(fig, "fig2")
print("\nFigures written to", HERE)
Printed stroke confidence interval excludes its own estimate; Table 2 shows the effect is non-significant
JAMA Network Open (2025) · doi:10.1001/jamanetworkopen.2025.39862 · PMC12569718
Summary
This UK Biobank cohort study used longitudinal targeted maximum likelihood estimation to estimate how 5-year risks of hospitalization for cardiovascular disease would change under hypothetical PM2.5 interventions at thresholds of 12, 10 and 9 µg/m³. The Abstract and the Results text give the stroke risk difference at the 9 µg/m³ threshold (the US EPA annual standard) as −2.26 per mille with a 95% CI of −8.97 to −20.64. That interval is impossible: its lower bound is above its upper bound, and the point estimate lies outside it. Table 2, Supplement 1 eTable 2, and a recomputation from the Table 2 risks all give an upper bound of about +20.6, so the interval includes zero and the stroke effect at 9 µg/m³ is not statistically significant. The Key Points nonetheless say that the 12 and 9 µg/m³ thresholds "would significantly reduce" stroke, heart failure and myocardial infarction. By Table 2, that holds for myocardial infarction, but not for stroke at 9 µg/m³ or for heart failure at 12 µg/m³. Two smaller sign discrepancies between the text and Table 2 point in the same direction.
What the paper reports
The PMC-archived text uses hyphen-minus signs, and the quotes below reproduce it. The publisher's version of record prints the same characters as true minus signs (U+2212).
Key Points, Findings:
implementing a hypothetical intervention for reducing PM2.5 exposure by 5% if it is above the thresholds of 12 and 9 µg/m3 would significantly reduce the 5-year risk of hospitalization for stroke, heart failure, and myocardial infarction, but not for arrhythmia.
Abstract, Results (the same sentence fragment also appears in the Results text, in the paragraph that describes Table 2):
the hypothetical PM2.5 intervention of reducing PM2.5 exposure by 5% if it is above the threshold of 9 µg/m3, the estimated 5-year risk difference of hospitalization for stroke was -2.26 per mille (95% CI, -8.97 to -20.64 per mille)
Results text, Table 2 paragraph:
For the threshold intervention of 9 µg/m3 compared with no intervention, the estimated 5-year risk difference of hospitalization for stroke was -2.26 per mille (95% CI, -8.97 to -20.64 per mille)
Table 2, stroke rows (columns: hypothetical intervention | 5-y risk (95% CI), per mille | risk difference (95% CI), per mille):
No intervention | 12.32 (11.92 to 12.72) | [Reference] ≤12 | 10.78 (10.07 to 11.67) | -1.54 (-2.21 to -0.73) ≤10 | 6.98 (3.76 to 19.60) | -5.34 (-8.59 to 7.24) ≤9 | 10.06 (3.37 to 32.97) | -2.26 (-8.97 to 20.64)
Supplement 1, eTable 2, stroke, main analysis (columns: no-intervention risk | ≤9 µg/m³ risk | risk difference):
Main analysis 12.32 (11.92, 12.72) 10.06 (3.37, 32.97) -2.26 (-8.97 to 20.64)
The two further discrepancies. Abstract, Results, stroke at 12 µg/m³:
the estimated 5-year risk difference of hospitalization for stroke was -1.54 per mille (95% CI, -2.21 to 0.73 per mille). However, the reduction in risk for arrhythmia was not statistically significant
Results text, stroke at 12 µg/m³, which agrees with Table 2:
the estimated 5-year risk difference of hospitalization for stroke was -1.54 per mille (95% CI, -2.21 to -0.73 per mille)
Results text, heart failure at 10 µg/m³:
for heart failure, -2.15 per mille (95% CI, -3.55 to -0.35 per mille)
Table 2, heart failure row ≤10:
≤10 | 4.35 (2.95 to 6.89) | -2.15 (-3.55 to 0.35)
Heart failure at 12 µg/m³, which agrees between the Results text and Table 2:
for heart failure, -0.51 per mille (95% CI, -1.07 to 0.35 per mille)
The significance criterion, from Methods, Statistical Analysis:
Two-sided P < .05 was considered statistically significant.
The statements that build on these results. Discussion, first paragraph:
Our findings indicate a decreased 5-year risk of hospitalization with a primary diagnosis of stroke, myocardial infarction, or heart failure for the hypothetical intervention of reducing PM2.5 exposure by 5% if it is above the annual average standards of 12 µg/m3, 10 µg/m3, or 9 µg/m3 compared with no intervention on ambient PM2.5. However, the 5-year risk difference was not statistically significant for the hospitalization with a primary diagnosis of arrhythmia.
Discussion, comparison with earlier studies:
While our study found decreased 5-year risks of stroke, myocardial infarction, and heart failure for the hypothetical PM2.5 intervention, the risk difference was not statistically significant for the outcome of arrhythmia.
Conclusions:
Our results indicate that a more stringent ambient PM2.5 standard in the United Kingdom may lower the risk of hospitalization for stroke, myocardial infarction, and heart failure.
What the paper's own numbers show
Method. Table 2 gives, for each of four outcomes, the 5-year risk with no intervention (the reference) and under each of the three threshold interventions, each with a 95% CI, plus the risk difference (RD) with its own 95% CI. I made two checks from Table 2 alone. (a) The RD point estimate should equal the intervention risk minus the reference risk. (b) The RD CI should be close to the intervention-risk CI minus the reference point estimate. Check (b) is an approximation: it ignores the uncertainty in the reference risk and its correlation with the intervention risk. It works here because the reference risks are estimated far more precisely (for stroke, 11.92 to 12.72) than the intervention risks (for stroke at 9 µg/m³, 3.37 to 32.97). I measured the approximation's accuracy against all 24 Table 2 RD bounds and used that accuracy as the yardstick for the text-versus-table discrepancies. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Accuracy of check (a), all 12 RD point estimates | Table 2 | — | every RD equals risk(intervention) − risk(no intervention) within 0.010 per mille (rounding) |
| Accuracy of check (b), all 24 RD CI bounds | Table 2 | — | every bound within 0.22 per mille of risk CI − reference risk (largest gap: arrhythmia ≤10 upper, 1.28 vs 1.06) |
| Stroke RD, ≤9 µg/m³ | Abstract; Results text | −2.26 (−8.97 to −20.64) | lower bound > upper bound; −2.26 is outside the interval, and stays outside if the bounds are reordered (−20.64 to −8.97) |
| Stroke RD, ≤9 µg/m³ | Table 2; eTable 2 | −2.26 (−8.97 to 20.64) | 10.06 − 12.32 = −2.26; (3.37 − 12.32, 32.97 − 12.32) = (−8.95 to +20.65). Includes 0 |
| Stroke RD upper bound, ≤9 µg/m³ | — | — | text vs recomputed: 41.29 apart; Table 2 vs recomputed: 0.01 apart. Using either end of the reference CI instead of 12.32 gives 20.25 to 21.05, so the bound is positive in every case |
| Stroke RD, ≤9 µg/m³, three robustness checks | eTable 2 | −3.39 (−9.25 to 18.47); −2.26 (−9.30 to 20.49); −4.02 (−9.26 to 14.91) | (−9.17 to 18.39); (−9.27 to 20.54); (−9.20 to 15.05). All include 0 |
| Stroke RD, ≤12 µg/m³ | Abstract | −1.54 (−2.21 to 0.73) | Table 2 and Results text: −2.21 to −0.73; recomputed (−2.25 to −0.65). The abstract's upper bound is 1.38 from the recomputed value, against 0.08 for Table 2 |
| Heart failure RD, ≤10 µg/m³ | Results text | −2.15 (−3.55 to −0.35) | Table 2: −3.55 to 0.35; recomputed (−3.55 to +0.39). The text's upper bound is 0.74 from the recomputed value, against 0.04 for Table 2. Includes 0 |
| Heart failure RD, ≤12 µg/m³ | Results text; Table 2 | −0.51 (−1.07 to 0.35) | recomputed (−1.13 to +0.41). Consistent, and includes 0 |
| Stroke RD, ≤10 µg/m³ | Results text; Table 2 | −5.34 (−8.59 to 7.24) | recomputed (−8.56 to +7.28). Consistent, and includes 0 |
| Myocardial infarction RD, ≤12 and ≤9 µg/m³ | Results text; Table 2 | −1.41 (−1.98 to −0.67); −8.64 (−9.16 to −6.38) | recomputed (−2.03 to −0.62); (−9.12 to −6.44). Consistent, and both exclude 0 |
| Heart failure RD, ≤9 µg/m³ | Abstract; Results text; Table 2 | −3.20 (−4.16 to −1.25) | recomputed (−4.10 to −1.30). Consistent, and excludes 0 |
| All other text values (abstract and Results text) | — | — | identical to Table 2 |
The paper does not report P values for the main risk differences. When a test and its 95% CI are derived consistently, a CI that includes zero corresponds to a two-sided P ≥ .05, which is the paper's own significance threshold. By Table 2, the significance of each risk difference is:
| Outcome | ≤12 µg/m³ | ≤10 µg/m³ | ≤9 µg/m³ |
|---|---|---|---|
| Stroke | significant | not significant | not significant |
| Myocardial infarction | significant | significant | significant |
| Heart failure | not significant | not significant (the Results text's −0.35 would make it appear significant) | significant |
| Arrhythmia | not significant | not significant | not significant |
The Key Points claim six significant reductions (stroke, heart failure and myocardial infarction, each at 12 and 9 µg/m³). Table 2 supports four of them. The two it does not support are stroke at 9 µg/m³, where only the impossible text interval excludes zero, and heart failure at 12 µg/m³, where the text and the table agree that the interval includes zero.
One further, minor observation. In Figure 3 of the paper (a forest plot of the 9 µg/m³ main analysis and robustness checks), the stroke main-analysis interval is drawn crossing the zero line, ending near 0.02 on the axis. That agrees with Table 2. The axis is labeled "per mille", but its tick values (−0.02 to 0.03) appear to be on the proportion scale, where 0.02 corresponds to 20 per mille.
Figure
Figure 1. The 5-year stroke risk difference for the 9 µg/m³ intervention versus no intervention, in per mille, on one linear axis. Orange marks are values printed in the paper; the hollow orange marker is the text value that disagrees with Table 2. Blue is the interval recomputed as the Table 2 risk CI under the intervention (3.37 to 32.97) minus the no-intervention risk (12.32). The thin vertical line is zero difference. The interval printed in the Abstract and Results text (−8.97 to −20.64) lies entirely to the left of its own point estimate. Table 2, eTable 2 and the recomputation agree on an interval of about −9 to +20.6, which includes zero. Plus signs in the value column are added for clarity; the paper prints positive bounds without a sign.
Figure 2. Three risk differences on one linear axis (per mille), using the same encoding as Figure 1. For stroke at 12 µg/m³, the Abstract prints an upper bound of +0.73. The Results text and Table 2 print −0.73, and the recomputed bound is −0.65. For heart failure at 10 µg/m³, the Results text prints −0.35, while Table 2 prints +0.35 and the recomputed bound is +0.39. In both cases the recomputation sides with Table 2, within the approximation's measured accuracy of 0.22 per mille. Heart failure at 12 µg/m³ is printed consistently, and its interval includes zero. Even so, the Key Points describe it as a significant reduction.
Why it matters
The 9 µg/m³ threshold is the one the paper ties to the US EPA primary annual standard, and the Abstract reports the 9 µg/m³ results first. The impossible stroke interval appears twice, in the Abstract and in the Results text. In both places all of the printed interval lies below zero, which makes the stroke reduction look statistically significant. The paper's own table says otherwise. Under the correct interval (−8.97 to +20.64 per mille) the data are compatible with anything from about a 9 per mille decrease to a 21 per mille increase in 5-year stroke hospitalization risk. Stroke is also not significant at 10 µg/m³, and it is not significant in any of the three robustness checks at 9 µg/m³. So the paper's data support a significant stroke reduction only at 12 µg/m³.
The Key Points sentence ("thresholds of 12 and 9 µg/m3 would significantly reduce ... stroke, heart failure, and myocardial infarction") is what most readers take from the article. Read naturally, it is not supported by Table 2 for stroke at 9 µg/m³ or for heart failure at 12 µg/m³. The Discussion's first paragraph lists decreased risk of stroke, myocardial infarction and heart failure at all three thresholds and contrasts them with arrhythmia, which "was not statistically significant". Read that way, the paragraph implies significance for stroke at 10 and 9 µg/m³ and for heart failure at 12 and 10 µg/m³, none of which Table 2 supports. The point estimates are all negative, so "decreased" is accurate as a description of direction only.
The scope of the problem is limited. Myocardial infarction is significant at every threshold, and heart failure at 9 µg/m³. The general conclusion that stricter PM2.5 standards may lower cardiovascular hospitalization is still supported by those results. What changes is the outcome-specific claim, which for stroke matters at the threshold most relevant to current policy.
Explanations considered
- A sign slip when the table values were transcribed into the text. This is the most plausible explanation, and it accounts for all three discrepancies. In each case one bound differs from Table 2 only by its sign: −20.64 for +20.64, +0.73 for −0.73, and −0.35 for +0.35. Table 2 agrees with the recomputation each time. It does not, however, rescue the Key Points or the Discussion, which match the mistyped text value for stroke at 9 µg/m³ rather than the table.
- A conversion artifact in PMC. Ruled out. The publisher's version of record, the Europe PMC full-text XML and the PMC page all print the same values.
- Table 2 is wrong and the text is right. Ruled out for stroke at 9 µg/m³. The text interval does not contain its own estimate, so it cannot be a valid confidence interval on any scale. Table 2's upper bound matches the risk CI (32.97 − 12.32 = 20.65). eTable 2 gives the same value, and the paper's Figure 3 draws the interval across zero. For the other two discrepancies, the recomputation sides with the table (0.08 and 0.04 per mille away) and not with the text (1.38 and 0.74 away). The approximation's largest error across all 24 table bounds is 0.22 per mille.
- The text uses a different interval method (for example a transformed scale or a different bootstrap). Ruled out for stroke at 9 µg/m³. Any valid interval contains its point estimate, and reordering the printed bounds (−20.64 to −8.97) still excludes −2.26. For the other two, nothing in the Methods describes a second interval method, and the remaining text values match Table 2 exactly.
- The recomputation itself is too crude. It cannot account for the numbers. Check (b) is only an approximation, but across all 24 Table 2 bounds it is accurate to 0.22 per mille. For stroke at 9 µg/m³ the discrepancy is 41 per mille. Using either end of the reference CI in place of its point estimate still gives an upper bound of 20.25 to 21.05.
- The Key Points use a looser reading of "12 and 9". This is possible. If the sentence means only that each outcome is significantly reduced at one or other threshold, Table 2 supports it: stroke at 12, heart failure at 9, myocardial infarction at both. But the Abstract's printed stroke interval at 9 µg/m³ lies wholly below zero, and the Discussion claims decreased risk at all three thresholds in contrast to the non-significant arrhythmia result. Both point to the stricter reading, under which two of the six claimed reductions are not significant.
- Significance was judged by P values rather than CIs. The paper reports no P values for the main risk differences, only CIs. The stated criterion is two-sided P < .05. With a consistently derived 95% CI that includes zero, that criterion is not met.
Status
As of 2026-10-23:
- Version of record. The publisher's full text (jamanetwork.com, article 2840619, reached through the doi.org redirect), the Europe PMC full-text XML and the PMC HTML page all carry the same numbers, with true minus signs (U+2212). In each, the Abstract and the Results text print the stroke interval at 9 µg/m³ as "−8.97 to −20.64", and Table 2 prints "−8.97 to 20.64". The Abstract's stroke interval at 12 µg/m³ (−2.21 to 0.73) and the Results text's heart failure interval at 10 µg/m³ (−3.55 to −0.35) also appear in the version of record. Every quote above was checked verbatim against the PMC-archived text.
- Supplements checked. Supplement 1 (eAppendices 1–2, eMethods, eTable 1 on subgroups, eTable 2 on the main analysis and robustness checks, eFigures 1–4). Its eTable 2 agrees with Table 2 for stroke at 9 µg/m³ (−2.26, −8.97 to 20.64), and all three stroke robustness checks include zero. Supplement 2 is the data sharing statement. Both were taken from the Europe PMC supplementary-files archive. The paper's Figure 3 image was also checked, and it draws the stroke interval at 9 µg/m³ across zero.
- Corrections and comments. I found no correction, erratum, retraction or expression of concern. Crossref lists no update-to or updated-by relations for the DOI. PMC has no linked notice. Europe PMC (PMID 41148138) has no comment/correction list, and a Europe PMC search for an erratum under the title returned nothing. Neither the publisher's page nor the PMC page shows a notice. PubPeer could not be checked, because its search page returned HTTP 403 to automated access and the API was blocked. A general web search was not available for this check.
Suggested correction
- In the Abstract (Results) and the Results text, change the stroke risk difference at 9 µg/m³ to "−2.26 per mille (95% CI, −8.97 to 20.64 per mille)", to match Table 2 and eTable 2.
- In the Abstract, change the stroke interval at 12 µg/m³ to "−2.21 to −0.73 per mille". In the Results text, change the heart failure interval at 10 µg/m³ to "−3.55 to 0.35 per mille".
- Revise the Key Points to match Table 2. For example: the 12 and 9 µg/m³ thresholds significantly reduced the risk for myocardial infarction. The reduction for stroke was significant at 12 µg/m³ but not at 10 or 9 µg/m³. The reduction for heart failure was significant at 9 µg/m³ but not at 12 or 10 µg/m³. No threshold significantly reduced the risk for arrhythmia.
- Revise the first Discussion paragraph and the comparison paragraph so that they do not imply significant stroke reductions at 10 and 9 µg/m³ or significant heart failure reductions at 12 and 10 µg/m³. Qualify the Conclusions sentence on stroke accordingly.
- Minor: check the Figure 3 axis, whose tick values appear to be proportions although the label says per mille.
Recomputation script (python recompute.py.txt)
"""Recompute the risk-difference confidence intervals of PMC12569718
(JAMA Netw Open 2025, doi:10.1001/jamanetworkopen.2025.39862).
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (Abstract, Results text, Table 2)
and from Supplement 1, eTable 2. Units: per mille (per 1000 participants), 5-year risk.
Method. Table 2 reports, for each outcome, the 5-year risk under no intervention
(the reference) and under each threshold intervention, each with a 95% CI, plus the
risk difference (RD) with its own 95% CI. Two checks follow from Table 2 alone:
(a) RD point estimate = risk(intervention) - risk(no intervention);
(b) RD 95% CI ~= risk(intervention) CI - risk(no intervention) point estimate.
(b) is an approximation: it ignores the uncertainty in the reference risk and its
correlation with the intervention risk. It works here because the reference risk is
estimated far more precisely than the intervention risks. Its accuracy is measured
below against the 24 Table 2 RD bounds, and that measured accuracy is the yardstick
for the discrepancies between the text and the table.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.transforms import blended_transform_factory # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
OUTCOMES = ("stroke", "MI", "HF", "arrhythmia")
NAMES = {"stroke": "Stroke", "MI": "Myocardial infarction", "HF": "Heart failure",
"arrhythmia": "Arrhythmia"}
THRESHOLDS = (12, 10, 9)
# Table 2, "No intervention" rows: 5-y risk (95% CI), per mille
REF = {"stroke": (12.32, 11.92, 12.72), "MI": (11.47, 11.08, 11.84),
"HF": (6.50, 6.23, 6.76), "arrhythmia": (18.86, 18.36, 19.35)}
# Table 2, threshold rows: (5-y risk (95% CI)), (risk difference (95% CI)), per mille
T2 = {
"stroke": {12: ((10.78, 10.07, 11.67), (-1.54, -2.21, -0.73)),
10: ((6.98, 3.76, 19.60), (-5.34, -8.59, 7.24)),
9: ((10.06, 3.37, 32.97), (-2.26, -8.97, 20.64))},
"MI": {12: ((10.07, 9.44, 10.85), (-1.41, -1.98, -0.67)),
10: ((4.15, 3.47, 5.54), (-7.32, -8.05, -5.86)),
9: ((2.83, 2.35, 5.03), (-8.64, -9.16, -6.38))},
"HF": {12: ((5.99, 5.37, 6.91), (-0.51, -1.07, 0.35)),
10: ((4.35, 2.95, 6.89), (-2.15, -3.55, 0.35)),
9: ((3.29, 2.40, 5.20), (-3.20, -4.16, -1.25))},
"arrhythmia": {12: ((16.80, 14.07, 21.79), (-2.06, -4.79, 3.12)),
10: ((9.81, 6.28, 19.92), (-9.05, -12.59, 1.28)),
9: ((14.70, 6.14, 31.74), (-4.16, -12.70, 12.93))},
}
# Abstract, Results section: RD (95% CI) as printed
ABSTRACT = {("stroke", 9): (-2.26, -8.97, -20.64), ("MI", 9): (-8.64, -9.16, -6.38),
("HF", 9): (-3.20, -4.16, -1.25), ("arrhythmia", 9): (-4.16, -12.70, 12.93),
("stroke", 12): (-1.54, -2.21, 0.73), ("arrhythmia", 12): (-2.06, -4.79, 3.12)}
# Results text, paragraph describing Table 2: RD (95% CI) as printed
RESULTS = {("stroke", 12): (-1.54, -2.21, -0.73), ("MI", 12): (-1.41, -1.98, -0.67),
("HF", 12): (-0.51, -1.07, 0.35), ("arrhythmia", 12): (-2.06, -4.79, 3.12),
("stroke", 10): (-5.34, -8.59, 7.24), ("MI", 10): (-7.32, -8.05, -5.86),
("HF", 10): (-2.15, -3.55, -0.35), ("arrhythmia", 10): (-9.05, -12.59, 1.28),
("stroke", 9): (-2.26, -8.97, -20.64), ("MI", 9): (-8.64, -9.16, -6.38),
("HF", 9): (-3.20, -4.16, -1.25), ("arrhythmia", 9): (-4.16, -12.70, 12.93)}
# Supplement 1, eTable 2, stroke at <=9 ug/m3: (5-y risk (95% CI)), (RD (95% CI))
# The no-intervention risk is 12.32 (11.92, 12.72) in every row.
ETABLE2_STROKE = {
"Main analysis": ((10.06, 3.37, 32.97), (-2.26, -8.97, 20.64)),
"Further adjustment for NO2 and NOx (2010)": ((8.93, 3.15, 30.71), (-3.39, -9.25, 18.47)),
"Further adjustment for family history of CVD": ((10.06, 3.05, 32.86), (-2.26, -9.30, 20.49)),
"Further adjustment for BMI": ((8.30, 3.12, 27.37), (-4.02, -9.26, 14.91)),
}
# Key Points, Findings: thresholds 12 and 9 "would significantly reduce" these outcomes
KEY_POINTS_CLAIM = [(o, th) for o in ("stroke", "HF", "MI") for th in (12, 9)]
def approx_ci(o, risk):
"""RD 95% CI approximated as the intervention-risk CI minus the reference point."""
r0 = REF[o][0]
return risk[1] - r0, risk[2] - r0
def sig(lo, hi):
return "significant" if (hi < 0 or lo > 0) else "NOT significant (CI includes 0)"
def f(x):
return f"{x:+.2f}".replace("+-", "-")
# ---------------------------------------------------------------- 1. Table 2 internal check
print("1) Table 2 internal check")
print(" RD = risk(int) - risk(no int); RD CI ~= risk(int) CI - risk(no int)")
APPROX = {}
max_pt, max_bd = 0.0, 0.0
for o in OUTCOMES:
for th in THRESHOLDS:
risk, (d, dl, dh) = T2[o][th]
pt = risk[0] - REF[o][0]
al, ah = approx_ci(o, risk)
APPROX[(o, th)] = (pt, al, ah)
max_pt = max(max_pt, abs(pt - d))
max_bd = max(max_bd, abs(al - dl), abs(ah - dh))
print(f" {o:10s} <={th:2d}: RD reported {d:6.2f}, recomputed {pt:6.2f} | "
f"CI reported ({dl:6.2f}, {dh:6.2f}), recomputed ({al:6.2f}, {ah:6.2f}) | "
f"Table 2: {sig(dl, dh)}")
print(f" Largest |reported - recomputed| RD point estimate over 12 cells: {max_pt:.3f}")
print(f" Largest |reported - recomputed| RD CI bound over 24 bounds: {max_bd:.3f}")
# ---------------------------------------------------------------- 2. text vs Table 2
def check_text(label, d):
print(f"\n2) {label} vs Table 2")
out = []
for (o, th), (e, lo, hi) in d.items():
te, tl, thi = T2[o][th][1]
issues = []
if lo > hi:
issues.append("lower bound > upper bound")
if not (min(lo, hi) <= e <= max(lo, hi)):
issues.append("estimate outside its own CI")
if (e, lo, hi) != (te, tl, thi):
diffs = [n for n, a, b in (("estimate", e, te), ("lower", lo, tl), ("upper", hi, thi))
if a != b]
issues.append(f"differs from Table 2 ({te}, {tl} to {thi}) in: {', '.join(diffs)}")
_, al, ah = APPROX[(o, th)]
print(f" {o:10s} <={th:2d}: printed {e} ({lo} to {hi}) | Table 2 {te} ({tl} to {thi}) | "
f"recomputed ({al:.2f} to {ah:.2f}) | {'; '.join(issues) or 'OK'}")
if issues:
out.append((o, th))
print(f" cells with a problem: {out}")
return out
check_text("Abstract (Results)", ABSTRACT)
check_text("Results text", RESULTS)
print("\n Size of each text-vs-table discrepancy, against the approximation's accuracy:")
for label, (o, th), which, printed in (
("Abstract & Results, stroke <=9 upper", ("stroke", 9), 2, -20.64),
("Abstract, stroke <=12 upper", ("stroke", 12), 2, 0.73),
("Results, HF <=10 upper", ("HF", 10), 2, -0.35)):
tab = T2[o][th][1][which]
rec = APPROX[(o, th)][which]
print(f" {label:38s}: printed {printed:7.2f}, Table 2 {tab:6.2f}, recomputed {rec:6.2f} | "
f"|printed - recomputed| = {abs(printed - rec):5.2f} vs |Table 2 - recomputed| = "
f"{abs(tab - rec):4.2f}")
# The printed stroke <=9 interval cannot be made valid by reordering its bounds
e, lo, hi = ABSTRACT[("stroke", 9)]
print(f"\n Stroke <=9 printed interval reordered: ({min(lo, hi)} to {max(lo, hi)}); "
f"contains estimate {e}? {min(lo, hi) <= e <= max(lo, hi)}")
r = T2["stroke"][9][0]
print(f" Stroke <=9 RD upper bound using either end of the reference CI instead of 12.32: "
f"{r[2] - REF['stroke'][2]:.2f} to {r[2] - REF['stroke'][1]:.2f} (all > 0)")
# ---------------------------------------------------------------- 3. Key Points vs Table 2
print("\n3) Key Points: thresholds 12 and 9 'would significantly reduce' stroke, HF and MI")
for o, th in KEY_POINTS_CLAIM:
d, lo, hi = T2[o][th][1]
_, al, ah = APPROX[(o, th)]
print(f" {o:6s} <={th:2d}: Table 2 {d} ({lo} to {hi}) -> {sig(lo, hi)}; "
f"recomputed CI ({al:.2f} to {ah:.2f}) -> {sig(al, ah)}")
print("\n Significance grid from Table 2 (S = CI excludes 0, ns = CI includes 0):")
print(" " + " " * 22 + " ".join(f"<={th:2d}" for th in THRESHOLDS))
for o in OUTCOMES:
cells = [" S " if (T2[o][th][1][2] < 0 or T2[o][th][1][1] > 0) else " ns " for th in THRESHOLDS]
print(f" {NAMES[o]:22s}" + " ".join(cells))
# ---------------------------------------------------------------- 4. Supplement eTable 2
print("\n4) Supplement 1, eTable 2, stroke at <=9 (main analysis and robustness checks)")
for k, (risk, (d, dl, dh)) in ETABLE2_STROKE.items():
pt = risk[0] - REF["stroke"][0]
al, ah = risk[1] - REF["stroke"][0], risk[2] - REF["stroke"][0]
print(f" {k:45s}: RD {d} ({dl} to {dh}), recomputed {pt:.2f} ({al:.2f} to {ah:.2f}) "
f"-> {sig(dl, dh)}")
# ---------------------------------------------------------------- figures
BG, TXT, SEC, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
print(f" wrote {stem}.svg and {stem}.png")
def interval(ax, y, est, lo, hi, color, marker="o", filled=True):
a, b = min(lo, hi), max(lo, hi)
ax.plot([a, b], [y, y], color=color, lw=2, solid_capstyle="butt", zorder=3)
for x in (a, b):
ax.plot([x, x], [y - 0.12, y + 0.12], color=color, lw=2, zorder=3)
ax.plot([est], [y], marker=marker, ms=9, color=color, mfc=color if filled else BG,
mew=2, zorder=4, ls="none")
def style(ax, xlo, xhi, xlabel):
ax.set_xlim(xlo, xhi)
ax.axvline(0, color=SEC, lw=0.8, zorder=1)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel(xlabel, fontsize=10.5, color=TXT)
print("\n5) Figures")
def value_col(ax, y, ee, ll, hh):
ax.text(1.03, y, f"{f(ee)} ({f(ll)} to {f(hh)})".replace("-", "\u2212"),
transform=ax.get_yaxis_transform(), va="center", ha="left", fontsize=10, color=TXT)
LEG_REPORTED_BAD = Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, mfc=BG, mew=2,
label="Reported in text, differs from Table 2")
LEG_REPORTED = Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, label="Reported in table")
LEG_RECOMP = Line2D([], [], color=BLUE, lw=2, marker="o", ms=8,
label="Recomputed from Table 2 risks")
# ---- fig1: stroke at <=9 ug/m3
fig, ax = plt.subplots(figsize=(7.2, 3.9))
e, lo, hi = ABSTRACT[("stroke", 9)]
t_e, t_lo, t_hi = T2["stroke"][9][1]
s_e, s_lo, s_hi = ETABLE2_STROKE["Main analysis"][1]
r_e, r_lo, r_hi = APPROX[("stroke", 9)]
rows = [
("Abstract and Results text", e, lo, hi, ORANGE, False),
("Table 2", t_e, t_lo, t_hi, ORANGE, True),
("Supplement 1, eTable 2", s_e, s_lo, s_hi, ORANGE, True),
("Recomputed: Table 2 risk CI\nminus no-intervention risk", r_e, r_lo, r_hi, BLUE, True),
]
ys = [3, 2, 1, 0]
for (lab, ee, ll, hh, c, filled), y in zip(rows, ys):
interval(ax, y, ee, ll, hh, c, filled=filled)
value_col(ax, y, ee, ll, hh)
ax.text(1.03, 3.6, "RD (95% CI)", transform=ax.get_yaxis_transform(),
va="center", ha="left", fontsize=10, color=SEC)
ax.annotate("estimate lies outside its printed interval", xy=(e - 0.3, 3.12),
xytext=(-23.5, 3.62), fontsize=10, color=SEC, ha="left", va="center", zorder=5,
bbox=dict(fc=BG, ec="none", pad=1.5),
arrowprops=dict(arrowstyle="-", color=SEC, lw=0.8))
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-0.6, 3.95)
style(ax, -24, 24, "5-year risk difference for stroke, \u22649 \u00b5g/m\u00b3\nvs no intervention (per mille)")
ax.text(0.5, -0.55, "no difference", fontsize=10, color=SEC, va="bottom")
fig.legend(handles=[LEG_REPORTED_BAD, LEG_REPORTED, LEG_RECOMP], loc="upper center",
bbox_to_anchor=(0.5, 0.995), frameon=False, fontsize=10, ncol=2,
handlelength=1.6, columnspacing=1.2)
fig.subplots_adjust(left=0.29, right=0.71, top=0.80, bottom=0.18)
save(fig, "fig1")
# ---- fig2: the two smaller sign discrepancies and the HF <=12 cell
fig, ax = plt.subplots(figsize=(7.2, 4.9))
groups = [
("Stroke, \u226412 \u00b5g/m\u00b3", [("Abstract", ABSTRACT[("stroke", 12)], ORANGE, False),
("Results text, Table 2", T2["stroke"][12][1], ORANGE, True),
("Recomputed", APPROX[("stroke", 12)], BLUE, True)]),
("Heart failure, \u226410 \u00b5g/m\u00b3", [("Results text", RESULTS[("HF", 10)], ORANGE, False),
("Table 2", T2["HF"][10][1], ORANGE, True),
("Recomputed", APPROX[("HF", 10)], BLUE, True)]),
("Heart failure, \u226412 \u00b5g/m\u00b3", [("Results text, Table 2", T2["HF"][12][1], ORANGE, True),
("Recomputed", APPROX[("HF", 12)], BLUE, True)]),
]
y = 0
yt, yl, heads = [], [], []
for gname, items in groups:
heads.append((gname, y))
y -= 0.75
for lab, (ee, ll, hh), c, filled in items:
interval(ax, y, ee, ll, hh, c, filled=filled)
value_col(ax, y, ee, ll, hh)
yt.append(y)
yl.append(lab)
y -= 0.75
y -= 0.35
for gname, yy in heads:
ax.text(0.015, yy, gname, transform=blended_transform_factory(fig.transFigure, ax.transData),
ha="left", va="center",
fontsize=10.5, fontweight="bold", color=TXT)
ax.text(1.03, 0, "RD (95% CI)", transform=ax.get_yaxis_transform(), va="center", ha="left",
fontsize=10, color=SEC)
ax.set_yticks(yt)
ax.set_yticklabels(yl, fontsize=10)
ax.set_ylim(y + 0.6, 0.45)
style(ax, -4, 1, "5-year risk difference vs no intervention (per mille)")
fig.legend(handles=[LEG_REPORTED_BAD, LEG_REPORTED, LEG_RECOMP], loc="upper center",
bbox_to_anchor=(0.5, 0.995), frameon=False, fontsize=10, ncol=2,
handlelength=1.6, columnspacing=1.2)
fig.subplots_adjust(left=0.29, right=0.71, top=0.86, bottom=0.11)
save(fig, "fig2")
Registry share of Asian or Pacific Islander patients is 2.98%, not 5.9%, reversing reported underrepresentation
JAMA Network Open (2022) · doi:10.1001/jamanetworkopen.2022.39884 · PMC9634497
Summary
The study compares the racial and ethnic makeup of 8309 participants in US phase 1 metastatic-cancer drug trials with incident cases in the Cancer in North America (CiNA) registry. For the whole 2000-2018 period, the Results report that Asian or Pacific Islander (API) participants were underrepresented: 4.5% of trial participants against 5.9% of registry cases, a difference of −1.5 percentage points (P < .001). The registry count and denominator printed in the same sentence, 145 568 of 4 891 486, give 2.98%, not 5.9%. That count is the exact sum of the paper's two period counts, whose shares (2.7% and 3.4%) are printed correctly. Measured against 2.98%, trial participation is higher than the registry share under either trial count the paper gives: +1.49 points (P ≈ 2 × 10⁻¹⁵) with the Table's 371 of 8309, or +0.30 points (P = .12, not significant) with the sentence's 272 of 8308. The reported overall API underrepresentation, which the Conclusions repeat, therefore does not follow from the paper's own data. Separately, the API trial counts in the Table do not add up (121 + 151 = 272, but the "All" row is 371), and this bears on the API time trend.
What the paper reports
Results, third paragraph, the overall (2000-2018) comparison. The PMC-archived text has a hyphen before 1.5; the publisher's HTML and the Europe PMC XML have a true minus sign (U+2212):
an underrepresentation of Asian or Pacific Islander participants (trials, 272 of 8308 [4.5%]; CiNA, 145 568 of 4 891 486 [5.9%]; difference, -1.5 percentage points; P < .001)
The same paragraph, period-specific comparison (repeated in the Abstract):
Asian or Pacific Islander patients (2000-2011: trials, 121 of 3245 [3.7%]; CiNA, 75 033 of 2 800 711 [2.7%]; difference, 1.1 percentage points; 2012-2018: trials, 151 of 5063 [3.0%]; CiNA 70 535 of 2 090 775 [3.4%]; difference, -0.75 percentage points)
Results, second paragraph, the overall trial count. The Abstract gives "371 Asian or Pacific Islander participants (4.5%)":
Of the 8309 total participants, 23 (0.3%) were American Indian or Alaska Native, 371 (4.5%) were Asian or Pacific Islander
The Table, "Phase 1 trials", Asian or Pacific Islander column:
All (N = 8309) | 371 (4.5) 2000-2011 (n = 3245) | 121 (3.7) 2012-2018 (n = 5063) | 151 (3.0)
The Table, "Industry trials", same column:
2012-2018 (n = 4628) | 226 (4.9)
Methods, "Statistical Analysis", the test, the pooling and the sign convention:
Proportions accounting for the sample size of each trial were calculated for each racial and ethnic group by combining the total number of participants for each racial and ethnic group and dividing by the total number of trial participants
differences with 95% CIs were estimated using a 2-sample test for equality of proportions with continuity correction
with positive percentage differences indicating overrepresentation and negative percentage differences indicating underrepresentation
Results, third paragraph, the ratios shown in Figure 3:
especially Asian and Pacific Islander patients (ratio of trial to CiNA participants, 1.44 in 2000-2011 to 0.87 in 2012-2018)
Key Points, "Findings":
found worsening underrepresentation of American Indian or Alaska Native, Asian or Pacific Islander, Black, and Hispanic or Latinx participants over time
Conclusions:
These include underenrollment of Asian or Pacific Islander patients, Black patients, and Hispanic or Latinx patients in clinical trials overall and underenrollment of American Indian or Alaska Native patients, Asian or Pacific Islander patients, Black patients, and Hispanic or Latinx patients in trials conducted at academic institutions
What the paper's own numbers show
Method. I took every count printed in the Abstract, Results, Table and Supplement eTable, recomputed each share, and checked whether overall counts equal the sums of the period counts. I then reran each comparison with the test the Methods name: a two-sample test for equality of proportions with continuity correction (R's prop.test, whose statistic is the Yates-corrected χ² on the 2 × 2 table), with its continuity-corrected 95% CI. As a check that this is the paper's test, it reproduces the reported White 2000-2011 result (+0.76 points, P = .235; reported 0.8, P = .23) and the overall White (+2.01; reported 2.0) and Black (−5.64; reported −5.6) differences. It also matches every significance marker in the paper's Figure 2: P < .001 for API 2000-2011, P = .048 for American Indian or Alaska Native 2012-2018 (marked P < .05), and no marker for API 2012-2018 (P = .13). Every number below is printed by recompute.py.txt.
Additivity. The overall CiNA counts equal the sums of the period counts, for API (75 033 + 70 535 = 145 568), for the denominator (2 800 711 + 2 090 775 = 4 891 486), and for White and Black. The overall API registry share is therefore the pooled share of the two periods, so it must lie between 2.68% and 3.37%. It cannot be 5.9%.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| CiNA API share, 2000-2011 | Abstract; Results ¶3 | 75 033 / 2 800 711 [2.7%] | 2.68% (consistent) |
| CiNA API share, 2012-2018 | Abstract; Results ¶3 | 70 535 / 2 090 775 [3.4%] | 3.37% (consistent) |
| CiNA API count, 2000-2018 | Results ¶3 | 145 568 | 75 033 + 70 535 = 145 568 (exact) |
| CiNA API share, 2000-2018 | Results ¶3 | 145 568 / 4 891 486 [5.9%] | 2.98%. 5.9% of 4 891 486 would be 288 598 cases (1.98 × the printed count) |
| CiNA groups summed, with API at 5.9% | derived | — | White + Black + AIAN + API = 5 000 409, which exceeds the 4 891 486 total by 108 923 |
| Trial API share, 2000-2018 (sentence) | Results ¶3 | 272 / 8308 [4.5%] | 3.27%. The bracketed 4.5% belongs to 371/8309 |
| Trial API share, 2000-2018 (Table) | Table "All"; Abstract; Results ¶2 | 371 / 8309 (4.5) | 4.47% (consistent) |
| Overall API difference, trials − CiNA, using 371/8309 | Results ¶3 | −1.5 pp, P < .001 | +1.49 pp (95% CI +1.04 to +1.94), P = 1.9 × 10⁻¹⁵ |
| Overall API difference, using 272/8308 | Results ¶3 | −1.5 pp, P < .001 | +0.30 pp (95% CI −0.09 to +0.69), P = .12 |
| Overall API difference from the printed shares | Results ¶3 | −1.5 pp | 4.5 − 5.9 = −1.4; 4.47 − 5.9 = −1.43. Neither gives −1.5 |
| Overall API trial/CiNA ratio | not reported | (underrepresentation, ratio < 1) | 1.50 (371/8309) or 1.10 (272/8308) |
| API difference, 2000-2011 | Abstract; Results ¶3 | +1.1 pp | +1.05 pp (95% CI +0.38 to +1.72), P = .0003 (rounding-level) |
| API difference, 2012-2018 | Abstract; Results ¶3 | −0.75 pp, no P value | −0.39 pp (95% CI −0.87 to +0.09), P = .13; printed shares give 3.0 − 3.4 = −0.4 |
| API trial/CiNA ratio, 2000-2011 → 2012-2018 | Results ¶3; Figure 3 | 1.44 → 0.87 | 1.39 → 0.88 |
| White overall difference (method check) | Abstract; Results ¶3 | +2.0 pp, P < .001 | +2.01 pp, P = 5.8 × 10⁻⁷ (consistent) |
| Black overall difference (method check) | Results ¶3 | −5.6 pp, P < .001 | −5.64 pp, P = 7.9 × 10⁻⁵⁷ (consistent) |
Against the registry share its own counts imply, the overall API difference has the reported size (1.49 points with the Table's count) but the opposite sign.
A second, separate inconsistency: the API trial counts. In the Table every other column adds up across the period rows (White 2780 + 4374 = 7154; Black 244 + 270 = 514; and so on). The API column does not.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| API, 2000-2011 + 2012-2018 vs "All" | Table, phase 1 trials | 121 + 151; All = 371 | 121 + 151 = 272, which is the count used in the Results sentence; 99 short of 371 |
| API, industry-funded subset vs all trials, 2012-2018 | Table | industry 226 (4.9); all trials 151 (3.0) | the subset exceeds the total by 75. The industry rows themselves add up (106 + 226 = 332) |
| "All" row minus the 2000-2011 row | Table | — | 371 − 121 = 250 (4.94% of 5063), which is ≥ the industry count of 226 |
| Supplement eTable (EGFR-positive trials added back) minus Table, overall vs by period | eTable; Table | — | additive for White (406 = 406), Black (17 = 17), AIAN (1 = 1) and Hispanic (4 = 4). For API, 870 − 371 = 499 overall but 598 by period. A 2012-2018 Table count of 250 would make API additive too |
| API difference, 2012-2018, if the count were 250 | derived | — | +1.56 pp (95% CI +0.96 to +2.17), P = 9.7 × 10⁻¹⁰; ratio 1.46 |
So the printed 151, the printed 371, or both cannot be right. The paper's other numbers (the industry subset and the eTable) are consistent with 371 overall and 250 in 2012-2018, but the paper alone does not establish which figure is wrong.
Smaller points, none of which changes a conclusion. The CiNA White share for 2012-2018, 1 735 077 / 2 090 775 = 82.99%, is printed as 82.9% (difference recomputes to +3.40 against the printed 3.5). The Table footnote gives 5076 participants in trials reporting ethnicity, but its period denominators sum to 1792 + 3295 = 5087. The overall Black comparison uses 8308 as the denominator against 8309 elsewhere. The text ratios for Black (0.64 → 0.42) and Hispanic or Latinx (1.55 → 0.98) recompute to 0.65 → 0.44 and 1.49 → 0.98. In the paper's Figure 3 the plotted API point estimates sit near 1.39 and 0.88, and the text values 1.44 and 0.87 lie close to the ends of the plotted intervals (read from the image, so approximate).
Figure
Figure 1. Asian or Pacific Islander shares on one percentage axis starting at zero. Orange rings are the values printed in the paper. Blue dots are the values recomputed from the numerator and denominator printed with them (listed under each row label). For each period the two agree (2.7 vs 2.68; 3.4 vs 3.37). For 2000-2018 the paper prints 5.9%, but its own count, 145 568 of 4 891 486, gives 2.98%. Because that count and denominator are the sums of the period counts, the pooled share has to fall in the shaded band between the two period shares. Both trial figures the paper gives for 2000-2018, 3.27% (272/8308, Results sentence, printed there as 4.5%) and 4.47% (371/8309, Table and Abstract), lie at or above 2.98%, not below it.
Figure 2. Trial share minus CiNA share, in percentage points, with 95% CIs from the test the paper names (two-sample test for equality of proportions with continuity correction). The thin vertical line marks no difference; by the paper's convention, negative values mean underrepresentation. Orange rings are printed values (the paper gives no CIs for these comparisons). For 2000-2018 the paper prints −1.5 (P < .001). Recomputed against the registry share its counts imply, the difference is +1.49 with the Table's count and +0.30 (P = .12) with the sentence's count. For 2000-2011 the printed and recomputed values agree. For 2012-2018 the printed −0.75 recomputes to −0.39 (P = .13). The hollow blue marker shows the 2012-2018 difference if the all-trial count were 371 − 121 = 250, the value the Table's "All" row and industry subset imply; it is a derived scenario, not a value printed in the paper.
Why it matters
- A reported finding is reversed. The Results state a significant overall underrepresentation of API participants over 2000-2018 (−1.5 points, P < .001). The paper's own counts show overrepresentation (+1.49 points, ratio 1.50, with the Table's 371) or no detectable difference (+0.30 points, P = .12, ratio 1.10, with the sentence's 272). Neither supports underrepresentation.
- The Conclusions repeat it. They list "underenrollment of Asian or Pacific Islander patients ... in clinical trials overall". The only overall API comparison in the paper is the erroneous one. If "overall" is instead read as all trials in the more recent period, the 2012-2018 difference recomputes to −0.39 points (P = .13), and the paper's own Figure 2 marks no significant difference there.
- The API part of the "worsening" claim is weaker than presented. The Key Points report worsening underrepresentation of API participants over time. With the printed period counts, the API difference moves from significant overrepresentation (+1.05 points) to a non-significant −0.39 points: a decline, but to parity rather than to underrepresentation. If the Table's "All" row is right and the 2012-2018 count is 250, API participants were overrepresented in both periods (+1.05 and +1.56 points) and there is no worsening for this group.
- Related statements in the Conclusions are also not supported by the paper's counts. These are derived comparisons the paper does not print, offered for the editor's attention rather than as the main finding. The Conclusions list API underenrollment at academic centers, but the Table's academic counts are above the CiNA share in both periods: 68/1806 (3.77%) vs 2.68%, +1.09 points, P = .005; 153/4145 (3.69%) vs 3.37%, +0.32 points, P = .28. The Conclusions also list Hispanic or Latinx underenrollment overall and at academic centers. Pooled from the paper's counts, the overall comparison is 401/5076 (7.90%) vs 325 415 / 4 891 486 (6.65%), +1.25 points (P = .0004). At academic centers it is 276/3523 (7.83%), +1.18 points (P = .005), and it is positive in both periods.
- What is not affected. The CiNA period shares and the comparisons for White and Black patients reproduce. The paper's central finding of strong and increasing underrepresentation of Black patients (−4.00 to −6.89 points) does not depend on these errors.
Explanations considered
- Text-conversion artifact (lost minus sign, decimal or digit). This does not account for the numbers. The publisher's HTML, the Europe PMC XML and the PMC archive carry the same sentence, the first two with a true minus sign. No dropped sign or decimal turns 2.98 into 5.9, and the sentence is running prose, not a table cell.
- A different registry population or denominator for the overall comparison. This does not account for the numbers. The sentence prints its own numerator and denominator (145 568 of 4 891 486), and that ratio is 2.98%. Both numbers are the exact sums of the period counts.
- A weighted rather than pooled share. This does not account for the numbers. The Methods describe pooled proportions. Any weighted average of the two period shares (2.68% and 3.37%) also lies between them.
- 5.9% is a typographical slip, and the difference and P value are right. This does not account for the numbers. Against 2.98%, the difference is positive with either trial count, so −1.5 and the stated underrepresentation cannot stand with a corrected share.
- The trial count, not the registry share, is the problem. This is only part of the picture. The sentence's trial figures are themselves inconsistent (272/8308 is 3.27%, yet it is printed as 4.5%), but neither trial count gives a negative difference against 2.98%. Only the registry share produces the reversed sign.
- The overall comparison comes from the sensitivity cohort that adds back the EGFR-positive trials. This does not account for the numbers. That cohort changes only the trial side; the registry is the same. The supplement's eTable gives trial counts only (API 870/9273 = 9.4% overall), and its eFigure lists CiNA API shares of 2.68% and 3.39% for the two periods, with no overall value and no 5.9%. Its higher trial share would make the difference even more positive.
None of these explains the printed 5.9%. The material published with the paper does not show how it arose.
Status
- Version of record. Checked. The DOI resolves to the publisher's full-text page (jamanetwork.com, article 2798149), which carries the sentence as quoted, with a true minus sign, and shows no correction notice. Its reader-comment section displays no comments. The Europe PMC full-text XML (publisher-supplied JATS) has the identical sentence. All quotes above were confirmed verbatim against the PMC-archived text.
- Supplement. Checked: jamanetwopen-e2239884-s001.pdf (eTable and eFigure, the sensitivity analysis including EGFR-positive trials), from the Europe PMC supplementary-files archive. The eTable gives trial counts only; the eFigure lists CiNA API shares of 2.68% (2000-2011) and 3.39% (2012-2018), consistent with the main text and never 5.9%. It does not resolve the discrepancy. No source data or code are deposited. The same archive holds the article's figures. Figure 2 shows CiNA API bars near 2.7% and 3.4% for the two periods and has no overall bar; Figure 3 is described above.
- Corrections and notices. None found as of 2026-10-23. Crossref lists no update-to or updated-by relations. There are no PMC or Europe PMC correction, retraction or expression-of-concern notices, and the Europe PMC record (PMID 36326764) has no comment or correction links.
- PubPeer. The PubPeer search page returned HTTP 403 and its API returned a human-verification page, so PubPeer could not be checked directly in this session. A web search could not be run in this session either. An earlier screening step in this review found no correction as of 2026-10-22.
Sources: doi.org/10.1001/jamanetworkopen.2022.39884; publisher full text; PMC9634497; Europe PMC full-text XML; Europe PMC supplementary files; Crossref record.
Suggested correction
- Results, overall API comparison. Recompute the registry share as 145 568 / 4 891 486 = 3.0% and settle the trial count (item 2). The sentence would then report an overrepresentation, "trials, 371 of 8309 [4.5%]; CiNA, 145 568 of 4 891 486 [3.0%]; difference, 1.5 percentage points; P < .001", or, if 272 of 8308 is correct, "trials, 272 of 8308 [3.3%]; CiNA, 145 568 of 4 891 486 [3.0%]; difference, 0.3 percentage points; P = .12", with no significant difference.
- Table, API column. Reconcile the "All" row (371) with the period rows (121 + 151 = 272), and the all-trial 2012-2018 count (151) with the industry-funded 2012-2018 count (226). Then update the 2012-2018 API comparison in the Abstract and Results (currently −0.75 points, which recomputes to −0.39 with 151), Figure 2, Figure 3 and the ratios in the text.
- Conclusions and Key Points. Revise the statement of API underenrollment "in clinical trials overall", and the API part of "worsening underrepresentation ... over time", in line with items 1 and 2. Check the statements on API and Hispanic or Latinx underenrollment overall and at academic centers against the Table's counts, which are above the CiNA shares.
- Minor. Change 82.9% to 83.0% for 1 735 077 / 2 090 775, reconcile the Hispanic or Latinx denominator (5076 vs 1792 + 3295 = 5087), use 8309 consistently, and state whether the ratios quoted in the text are point estimates.
Recomputation script (python recompute.py.txt)
"""Recompute the Asian or Pacific Islander (API) registry comparison of PMC9634497
(JAMA Netw Open 2022;5(11):e2239884, doi:10.1001/jamanetworkopen.2022.39884).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Results, paragraphs 2-3 (trial and CiNA counts, by period and overall)
Table (trial counts: all trials, industry-funded, academic centers)
Supplement eTable (the same Table with the EGFR+ NSCLC trials added back)
Test: the paper states "a 2-sample test for equality of proportions with continuity
correction" (R prop.test). prop.test's statistic equals the Yates-corrected chi-square on
the 2x2 table, and its CI is the Wald interval widened by the continuity correction.
"""
import math
import os
import numpy as np
from scipy.stats import chi2_contingency, norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
# CiNA registry counts, Results paragraph 3 (period) and paragraph 2 (overall)
CINA_N = {"2000-2011": 2800711, "2012-2018": 2090775, "overall": 4891486}
CINA = { # group: (2000-2011, 2012-2018, overall as printed or None if not printed)
"White": (2378019, 1735077, 4113096),
"Black": (322701, 255625, 578326),
"API": (75033, 70535, 145568),
"AIAN": (10905, 9484, None),
"Hispanic": (169297, 156118, None),
}
CINA_API_PCT_PRINTED_OVERALL = 5.9
# Trial counts, Table ("All" rows) and Results text
TRIAL_N = {"2000-2011": 3245, "2012-2018": 5063, "overall": 8309}
TRIAL = { # group: (2000-2011, 2012-2018, Table "All")
"White": (2780, 4374, 7154),
"Black": (244, 270, 514),
"API": (121, 151, 371),
"AIAN": (10, 13, 23),
"Hispanic": (161, 240, 401),
}
HISP_TRIAL_N = {"2000-2011": 1792, "2012-2018": 3295, "overall": 5076} # Table footnote
API_TEXT = (272, 8308) # Results paragraph 2, overall API comparison
BLACK_TEXT = (514, 8308) # Results paragraph 2, overall Black comparison
# Table subsets (API column): (2000-2011, 2012-2018, All), and subset sizes
INDUSTRY_API = (106, 226, 332)
INDUSTRY_N = (2737, 4628, 7365)
ACADEMIC_API = (68, 153, 221)
ACADEMIC_N = (1806, 4145, 5951)
ACADEMIC_HISP = (62, 214, 276)
ACADEMIC_HISP_N = (841, 2682, None)
ACADEMIC_AIAN_ALL = 16
# Supplement eTable, "All Phase I" rows: (2001-2011, 2012-2018, All)
ETABLE = {
"White": (2906, 4654, 7560),
"Black": (253, 278, 531),
"API": (126, 744, 870),
"AIAN": (10, 14, 24),
"Hispanic": (164, 241, 405),
}
# Reported values (text) for the comparisons
REPORTED = {
"API overall": dict(trial=4.5, cina=5.9, diff=-1.5, p="< .001"),
"API 2000-2011": dict(trial=3.7, cina=2.7, diff=1.1, p="not given"),
"API 2012-2018": dict(trial=3.0, cina=3.4, diff=-0.75, p="not given"),
"White overall": dict(trial=86.1, cina=84.1, diff=2.0, p="< .001"),
"Black overall": dict(trial=6.2, cina=11.8, diff=-5.6, p="< .001"),
"White 2000-2011": dict(trial=85.7, cina=84.9, diff=0.8, p=".23"),
"White 2012-2018": dict(trial=86.4, cina=82.9, diff=3.5, p="< .001"),
"Black 2000-2011": dict(trial=7.5, cina=11.5, diff=-4.0, p="not given"),
"Black 2012-2018": dict(trial=5.3, cina=12.2, diff=-6.9, p="not given"),
"AIAN 2000-2011": dict(trial=0.3, cina=0.4, diff=-0.08, p="not given"),
"AIAN 2012-2018": dict(trial=0.3, cina=0.5, diff=-0.20, p="not given"),
"Hispanic 2000-2011": dict(trial=9.0, cina=6.0, diff=3.0, p="not given"),
"Hispanic 2012-2018": dict(trial=7.3, cina=7.5, diff=-0.2, p="not given"),
}
REPORTED_RATIOS = { # Results paragraph 3: (2000-2011, 2012-2018)
"API": (1.44, 0.87), "Black": (0.64, 0.42), "Hispanic": (1.55, 0.98)}
# ---------------------------------------------------------------- helpers
def pct(a, b):
return 100.0 * a / b
def prop_test(x1, n1, x2, n2, conf=0.95):
"""R prop.test(c(x1,x2), c(n1,n2)) with continuity correction.
Returns (p1 %, p2 %, diff pp, ci_lo pp, ci_hi pp, P)."""
p1, p2 = x1 / n1, x2 / n2
delta = p1 - p2
yates = min(0.5, abs(delta) / (1 / n1 + 1 / n2))
z = norm.ppf(0.5 + conf / 2)
width = z * math.sqrt(p1 * (1 - p1) / n1 + p2 * (1 - p2) / n2) + yates * (1 / n1 + 1 / n2)
table = np.array([[x1, n1 - x1], [x2, n2 - x2]])
_, p, _, _ = chi2_contingency(table, correction=True)
return 100 * p1, 100 * p2, 100 * delta, 100 * (delta - width), 100 * (delta + width), p
def fmt_p(p):
return f"{p:.3f}" if p >= 0.001 else f"{p:.1e}"
def line(label, res, reported=None):
p1, p2, d, lo, hi, p = res
s = (f" {label:46s} trial {p1:6.2f}% CiNA {p2:6.2f}% diff {d:+6.2f} pp "
f"(95% CI {lo:+.2f} to {hi:+.2f}) P = {fmt_p(p)}")
if reported:
s += (f"\n {'':46s} reported: {reported['trial']}% vs {reported['cina']}%, "
f"diff {reported['diff']:+} pp, P {reported['p']}")
print(s)
results = {}
# ---------------------------------------------------------------- 1. sums and the CiNA share
print("1. Do the overall counts equal the sum of the period counts?")
print(f" CiNA N: {CINA_N['2000-2011']} + {CINA_N['2012-2018']} = "
f"{CINA_N['2000-2011'] + CINA_N['2012-2018']} (printed overall {CINA_N['overall']})")
for g, (a, b, o) in CINA.items():
print(f" CiNA {g:8s}: {a} + {b} = {a + b}" + (f" (printed overall {o})" if o else " (overall not printed)"))
print(f" Trial N: {TRIAL_N['2000-2011']} + {TRIAL_N['2012-2018']} = "
f"{TRIAL_N['2000-2011'] + TRIAL_N['2012-2018']} (Table 'All' N = {TRIAL_N['overall']})")
for g, (a, b, o) in TRIAL.items():
flag = "" if a + b == o else " <-- does not add up"
print(f" Trial {g:8s}: {a} + {b} = {a + b} (Table 'All' = {o}){flag}")
print(f" Hispanic trial N (footnote): {HISP_TRIAL_N['2000-2011']} + {HISP_TRIAL_N['2012-2018']} = "
f"{HISP_TRIAL_N['2000-2011'] + HISP_TRIAL_N['2012-2018']} (footnote overall {HISP_TRIAL_N['overall']})")
print("\n2. CiNA share of Asian or Pacific Islander patients")
api_c = CINA["API"]
cina_api_p00 = pct(api_c[0], CINA_N["2000-2011"])
cina_api_p12 = pct(api_c[1], CINA_N["2012-2018"])
cina_api_all = pct(api_c[2], CINA_N["overall"])
print(f" 2000-2011: {api_c[0]}/{CINA_N['2000-2011']} = {cina_api_p00:.2f}% (printed 2.7%)")
print(f" 2012-2018: {api_c[1]}/{CINA_N['2012-2018']} = {cina_api_p12:.2f}% (printed 3.4%)")
print(f" 2000-2018: {api_c[2]}/{CINA_N['overall']} = {cina_api_all:.2f}% (printed {CINA_API_PCT_PRINTED_OVERALL}%)")
print(f" A pooled share must lie between the period shares: {cina_api_p00:.2f}% to {cina_api_p12:.2f}%")
implied = round(CINA_API_PCT_PRINTED_OVERALL / 100 * CINA_N["overall"])
print(f" Count implied by 5.9% of {CINA_N['overall']}: {implied} "
f"({implied / api_c[2]:.2f} x the printed 145 568)")
w, b = CINA["White"][2], CINA["Black"][2]
aian = CINA["AIAN"][0] + CINA["AIAN"][1]
print(f" White + Black + AIAN + API (printed counts): {w} + {b} + {aian} + {api_c[2]} = "
f"{w + b + aian + api_c[2]} of {CINA_N['overall']}")
print(f" ... with API at 5.9%: {w + b + aian + implied} of {CINA_N['overall']} "
f"(exceeds the total by {w + b + aian + implied - CINA_N['overall']})")
print(f" White + Black + API(5.9%) alone: {w + b + implied} "
f"({'exceeds' if w + b + implied > CINA_N['overall'] else 'within'} the total)")
print("\n3. Trial share of Asian or Pacific Islander participants")
print(f" Results text: {API_TEXT[0]}/{API_TEXT[1]} = {pct(*API_TEXT):.2f}% (printed 4.5%)")
print(f" Table/Abstract: {TRIAL['API'][2]}/{TRIAL_N['overall']} = {pct(TRIAL['API'][2], TRIAL_N['overall']):.2f}% (printed 4.5%)")
print(f" {API_TEXT[0]} = {TRIAL['API'][0]} + {TRIAL['API'][1]} (sum of the Table's period rows): "
f"{API_TEXT[0] == TRIAL['API'][0] + TRIAL['API'][1]}")
# ---------------------------------------------------------------- 4. the tests
print("\n4. Two-sample tests with continuity correction (method check, then API)")
for key, (x1, n1, x2, n2) in {
"White overall": (7154, 8309, 4113096, 4891486),
"Black overall": (BLACK_TEXT[0], BLACK_TEXT[1], 578326, 4891486),
"White 2000-2011": (2780, 3245, 2378019, 2800711),
"White 2012-2018": (4374, 5063, 1735077, 2090775),
"Black 2000-2011": (244, 3245, 322701, 2800711),
"Black 2012-2018": (270, 5063, 255625, 2090775),
"AIAN 2000-2011": (10, 3245, 10905, 2800711),
"AIAN 2012-2018": (13, 5063, 9484, 2090775),
"Hispanic 2000-2011": (161, 1792, 169297, 2800711),
"Hispanic 2012-2018": (240, 3295, 156118, 2090775),
}.items():
results[key] = prop_test(x1, n1, x2, n2)
line(key, results[key], REPORTED[key])
print()
results["API overall, 371/8309"] = prop_test(371, 8309, 145568, 4891486)
results["API overall, 272/8308"] = prop_test(272, 8308, 145568, 4891486)
results["API 2000-2011"] = prop_test(121, 3245, 75033, 2800711)
results["API 2012-2018"] = prop_test(151, 5063, 70535, 2090775)
line("API overall, Table count 371/8309", results["API overall, 371/8309"], REPORTED["API overall"])
line("API overall, text count 272/8308", results["API overall, 272/8308"], REPORTED["API overall"])
line("API 2000-2011", results["API 2000-2011"], REPORTED["API 2000-2011"])
line("API 2012-2018", results["API 2012-2018"], REPORTED["API 2012-2018"])
d371 = results["API overall, 371/8309"][2]
print(f" Reported -1.5 vs recomputed {d371:+.2f} with 371/8309: same size, opposite sign")
print(f" Printed percentages: 4.5 - 5.9 = {4.5 - 5.9:+.1f}; 4.47 - 5.9 = "
f"{pct(371, 8309) - 5.9:+.2f}; 4.47 - 2.98 = {pct(371, 8309) - cina_api_all:+.2f}")
print(f" Printed percentages for 2012-2018: 3.0 - 3.4 = {3.0 - 3.4:+.1f} (printed -0.75)")
# ---------------------------------------------------------------- 5. ratios
print("\n5. Ratio of trial share to CiNA share (Results paragraph 3, Figure 3)")
ratio = {}
for g, (n_t, n_c) in {"API": (TRIAL_N, CINA_N), "Black": (TRIAL_N, CINA_N),
"Hispanic": (HISP_TRIAL_N, CINA_N)}.items():
r00 = (TRIAL[g][0] / n_t["2000-2011"]) / (CINA[g][0] / n_c["2000-2011"])
r12 = (TRIAL[g][1] / n_t["2012-2018"]) / (CINA[g][1] / n_c["2012-2018"])
ratio[g] = (r00, r12)
print(f" {g:8s}: {r00:.3f} -> {r12:.3f} (text: {REPORTED_RATIOS[g][0]} -> {REPORTED_RATIOS[g][1]})")
r_all_371 = (371 / 8309) / (145568 / 4891486)
r_all_272 = (272 / 8308) / (145568 / 4891486)
print(f" API overall: {r_all_371:.3f} (371/8309), {r_all_272:.3f} (272/8308)")
# ---------------------------------------------------------------- 6. Table consistency
print("\n6. Table: subsets vs all trials (API column)")
print(f" 2012-2018: industry {INDUSTRY_API[1]}, academic {ACADEMIC_API[1]}, all trials {TRIAL['API'][1]}"
f" -> industry subset exceeds all trials: {INDUSTRY_API[1] > TRIAL['API'][1]}")
print(f" Industry rows add up: {INDUSTRY_API[0]} + {INDUSTRY_API[1]} = {INDUSTRY_API[0] + INDUSTRY_API[1]} "
f"(All {INDUSTRY_API[2]}); academic: {ACADEMIC_API[0]} + {ACADEMIC_API[1]} = "
f"{ACADEMIC_API[0] + ACADEMIC_API[1]} (All {ACADEMIC_API[2]})")
print(f" Industry 2012-2018 API share: {INDUSTRY_API[1]}/{INDUSTRY_N[1]} = {pct(INDUSTRY_API[1], INDUSTRY_N[1]):.2f}% (printed 4.9)")
print(f" All-trials 2012-2018 API share: {TRIAL['API'][1]}/{TRIAL_N['2012-2018']} = "
f"{pct(TRIAL['API'][1], TRIAL_N['2012-2018']):.2f}% (printed 3.0)")
implied_12 = TRIAL["API"][2] - TRIAL["API"][0]
print(f" All-row minus 2000-2011 row: {TRIAL['API'][2]} - {TRIAL['API'][0]} = {implied_12} "
f"({pct(implied_12, TRIAL_N['2012-2018']):.2f}% of {TRIAL_N['2012-2018']}); >= industry {INDUSTRY_API[1]}: "
f"{implied_12 >= INDUSTRY_API[1]}")
print(" Supplement eTable minus Table, overall vs sum of periods (EGFR+ trials added back):")
for g, (e0, e1, eall) in ETABLE.items():
t0, t1, tall = TRIAL[g]
d_all = eall - tall
d_per = (e0 - t0) + (e1 - t1)
print(f" {g:8s}: eTable periods {e0}+{e1}={e0 + e1} (All {eall}); added overall {d_all}, "
f"added by period {d_per}{'' if d_all == d_per else ' <-- differs'}")
x = ETABLE["API"][2] - TRIAL["API"][2] - (ETABLE["API"][0] - TRIAL["API"][0])
x = ETABLE["API"][1] - x
print(f" 2012-2018 API count that makes the API column additive like the others: {x}")
results["API 2012-2018, count 250"] = prop_test(implied_12, 5063, 70535, 2090775)
line("API 2012-2018 if count were 371-121=250", results["API 2012-2018, count 250"])
print(f" ratio with 250: {(implied_12 / 5063) / (70535 / 2090775):.3f}")
# ---------------------------------------------------------------- 7. Conclusions sentence
print("\n7. Groups named in the Conclusions as under-enrolled 'overall' and 'at academic institutions'")
hisp_c = CINA["Hispanic"][0] + CINA["Hispanic"][1]
aian_c = CINA["AIAN"][0] + CINA["AIAN"][1]
results["Hispanic overall"] = prop_test(401, 5076, hisp_c, CINA_N["overall"])
results["AIAN overall"] = prop_test(23, 8309, aian_c, CINA_N["overall"])
line("Hispanic overall (401/5076 vs CiNA period sum)", results["Hispanic overall"])
line("AIAN overall (23/8309 vs CiNA period sum)", results["AIAN overall"])
line("Black overall", results["Black overall"])
line("API overall, Table count 371/8309", results["API overall, 371/8309"])
results["Academic API overall"] = prop_test(ACADEMIC_API[2], ACADEMIC_N[2], 145568, CINA_N["overall"])
results["Academic API 2000-2011"] = prop_test(ACADEMIC_API[0], ACADEMIC_N[0], 75033, CINA_N["2000-2011"])
results["Academic API 2012-2018"] = prop_test(ACADEMIC_API[1], ACADEMIC_N[1], 70535, CINA_N["2012-2018"])
ah_all_n = ACADEMIC_HISP_N[0] + ACADEMIC_HISP_N[1]
results["Academic Hispanic overall"] = prop_test(ACADEMIC_HISP[2], ah_all_n, hisp_c, CINA_N["overall"])
results["Academic Hispanic 2000-2011"] = prop_test(ACADEMIC_HISP[0], ACADEMIC_HISP_N[0], 169297, CINA_N["2000-2011"])
results["Academic Hispanic 2012-2018"] = prop_test(ACADEMIC_HISP[1], ACADEMIC_HISP_N[1], 156118, CINA_N["2012-2018"])
results["Academic AIAN overall"] = prop_test(ACADEMIC_AIAN_ALL, ACADEMIC_N[2], aian_c, CINA_N["overall"])
for k in ["Academic API overall", "Academic API 2000-2011", "Academic API 2012-2018",
"Academic Hispanic overall", "Academic Hispanic 2000-2011", "Academic Hispanic 2012-2018",
"Academic AIAN overall"]:
line(k, results[k])
print(f" (academic Hispanic denominator {ACADEMIC_HISP_N[0]} + {ACADEMIC_HISP_N[1]} = {ah_all_n}; "
f"276/{ah_all_n} = {pct(276, ah_all_n):.2f}%, printed 7.8)")
# ---------------------------------------------------------------- figures
INK, INK2 = "#0b0b0b", "#52514e"
BG, ORANGE, BLUE, GRID = "#fcfcfb", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "axes.edgecolor": INK2, "xtick.color": INK2, "ytick.color": INK,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "lines.linewidth": 2,
})
def style(ax):
ax.grid(axis="x", color=GRID, linewidth=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# Figure 1: shares, reported vs recomputed
rows1 = [ # label, reported %, recomputed %, recomputed label
("CiNA registry, 2000–2011\n75 033 / 2 800 711", 2.7, cina_api_p00),
("CiNA registry, 2012–2018\n70 535 / 2 090 775", 3.4, cina_api_p12),
("CiNA registry, 2000–2018\n145 568 / 4 891 486", 5.9, cina_api_all),
("Trials, 2000–2018 (Results text)\n272 / 8308", 4.5, pct(*API_TEXT)),
("Trials, 2000–2018 (Table, Abstract)\n371 / 8309", 4.5, pct(371, 8309)),
]
fig, ax = plt.subplots(figsize=(7.2, 4.7))
y = np.arange(len(rows1))[::-1].astype(float)
y[3:] -= 0.35 # gap between registry and trial rows
ax.axvspan(cina_api_p00, cina_api_p12, ymin=0.47, ymax=1.0, color=BLUE, alpha=0.10, linewidth=0)
ax.text((cina_api_p00 + cina_api_p12) / 2, y[0] + 0.55,
"a pooled 2000–2018 share\nmust fall in this band", ha="center", va="bottom",
fontsize=10, color=INK2)
for (lab, rep, rec), yy in zip(rows1, y):
ax.plot([min(rep, rec), max(rep, rec)], [yy, yy], color=GRID, linewidth=2, zorder=1)
ax.scatter([rep], [yy], s=150, facecolor="none", edgecolor=ORANGE, linewidth=2.2, zorder=3)
ax.scatter([rec], [yy], s=70, color=BLUE, zorder=4)
ax.text(5.9 + 0.15, y[2], "printed 5.9%", va="center", color=ORANGE, fontsize=10.5)
ax.text(cina_api_all - 0.15, y[2] - 0.32, f"{cina_api_all:.2f}%", ha="right", va="center",
color=BLUE, fontsize=10.5)
ax.text(4.5 + 0.15, y[3], "printed 4.5%", va="center", color=ORANGE, fontsize=10.5)
ax.text(pct(*API_TEXT) - 0.15, y[3] - 0.32, f"{pct(*API_TEXT):.2f}%", ha="right", va="center",
color=BLUE, fontsize=10.5)
ax.text(4.5 + 0.15, y[4], "printed 4.5%", va="center", color=ORANGE, fontsize=10.5)
ax.text(pct(371, 8309) - 0.15, y[4] - 0.32, f"{pct(371, 8309):.2f}%", ha="right", va="center",
color=BLUE, fontsize=10.5)
ax.set_yticks(y)
ax.set_yticklabels([r[0] for r in rows1], fontsize=10)
ax.set_xlim(0, 7.6)
ax.set_ylim(y[-1] - 0.7, y[0] + 1.35)
ax.set_xlabel("Asian or Pacific Islander share, %")
style(ax)
h1 = ax.scatter([], [], s=150, facecolor="none", edgecolor=ORANGE, linewidth=2.2,
label="printed in the paper")
h2 = ax.scatter([], [], s=70, color=BLUE, label="recomputed from the paper's own counts")
fig.legend(handles=[h1, h2], loc="lower center", frameon=False, fontsize=10, ncol=2,
bbox_to_anchor=(0.5, 0.0))
fig.tight_layout(rect=(0, 0.08, 1, 1))
save(fig, "fig1")
# Figure 2: differences trial - CiNA with 95% CIs
rows2 = [ # label, reported diff or None, result key or None, hollow
("2000–2018, as printed", REPORTED["API overall"]["diff"], None),
("2000–2018, Table count 371 / 8309", None, "API overall, 371/8309"),
("2000–2018, text count 272 / 8308", None, "API overall, 272/8308"),
("2000–2011, 121 / 3245", REPORTED["API 2000-2011"]["diff"], "API 2000-2011"),
("2012–2018, 151 / 5063", REPORTED["API 2012-2018"]["diff"], "API 2012-2018"),
("2012–2018, if 371 − 121 = 250", None, "API 2012-2018, count 250"),
]
fig, ax = plt.subplots(figsize=(7.2, 5.0))
y = np.array([5.6, 4.6, 3.6, 2.2, 1.0, 0.0])
ax.axvline(0, color=INK2, linewidth=0.9, zorder=1)
for (lab, rep, key), yy in zip(rows2, y):
off = 0.14 if (rep is not None and key is not None) else 0.0
if key is not None:
_, _, d, lo, hi, p = results[key]
hollow = key.endswith("250")
ax.plot([lo, hi], [yy - off, yy - off], color=BLUE, linewidth=2, zorder=2)
ax.scatter([d], [yy - off], s=70, zorder=3, linewidth=2,
facecolor=BG if hollow else BLUE, edgecolor=BLUE)
ptxt = "P < .001" if p < 0.001 else f"P = {p:.2f}".replace("0.", ".")
ax.text(hi + 0.08, yy - off, f"{d:+.2f} ({ptxt})".replace("-", "\u2212"), va="center",
color=BLUE, fontsize=10)
if rep is not None:
ax.scatter([rep], [yy + off], s=150, facecolor="none", edgecolor=ORANGE, linewidth=2.2, zorder=4)
ptxt = " (P < .001)" if key is None else ""
txt = f"{rep:+g}{ptxt}".replace("-", "\u2212")
if key is None:
ax.text(rep, yy + 0.24, txt.replace(" ", " "), va="bottom", ha="center",
color=ORANGE, fontsize=10)
else:
ax.text(rep + (0.14 if rep > 0 else -0.14), yy + off, txt, va="center",
ha="left" if rep > 0 else "right", color=ORANGE, fontsize=10)
ax.set_yticks(y)
ax.set_yticklabels([r[0] for r in rows2], fontsize=10)
ax.set_xlim(-2.6, 3.9)
ax.set_ylim(-0.6, 6.4)
ax.set_xlabel("Trial share minus CiNA share, percentage points\n"
"← under-representation over-representation →", fontsize=10.5)
style(ax)
h1 = ax.scatter([], [], s=150, facecolor="none", edgecolor=ORANGE, linewidth=2.2,
label="printed in the paper")
h2 = ax.errorbar([], [], xerr=[], fmt="o", color=BLUE, markersize=8, linewidth=2,
label="recomputed, 95% CI (prop.test)")
h3 = ax.scatter([], [], s=70, facecolor=BG, edgecolor=BLUE, linewidth=2,
label="recomputed for a count implied by the Table")
fig.legend(handles=[h1, h2, h3], loc="lower center", frameon=False, fontsize=10, ncol=2,
bbox_to_anchor=(0.5, 0.0))
fig.tight_layout(rect=(0, 0.12, 1, 1))
save(fig, "fig2")
print("\nFigures written to", HERE)
Right anterior insula odds ratio of 0.96 lies outside its own reported 95% CI (0.01–0.75)
JAMA Psychiatry (2015) · doi:10.1001/jamapsychiatry.2014.1604 · PMC5103694
Summary
The paper's main predictive result, given in the abstract and in the Results, is that right anterior insula (AI) volume predicted a diagnosis of major depressive disorder (MDD) after imaging, with "odds ratio, 0.96; 95% CI, 0.01–0.75; P = .03". The point estimate 0.96 lies above the upper bound of its own 95% CI, which is impossible for any standard interval. The CI and the P value agree with each other: on the log-odds scale the CI is centred on an OR of about 0.087 (0.061–0.106 when the printed bounds are allowed to vary within their rounding), and that implies a Wald P of 0.026, which matches the reported .03. An OR of 0.96 with the same standard error would give P ≈ 0.97. The direction and statistical significance of the finding therefore stand, but the printed effect size understates the association the paper's own CI and P value imply, roughly a 91% rather than a 4% lower odds of later MDD per unit of whole-brain-adjusted volume.
What the paper reports
The abstract:
"When covarying for children’s lifetime history of MDD episodes, their experience of pathological guilt during preschool, as well as their sex and age at the time of imaging, schoolchildren’s right-side AI volume was a significant predictor of being diagnosed as having an MDD episode after imaging (odds ratio, 0.96; 95% CI, 0.01–0.75; P = .03)." (Abstract, Results)
The same model in the Results, together with the other estimates printed in the same paragraph:
"Children with an MDD diagnosis up to the time of imaging were approximately 11 times as likely as same-age peers without a prior diagnosis of MDD to have an MDD diagnosis after their imaging (odds ratio [OR], 11.38; 95% CI, 2.88–44.94; P < .01). However, even when including the robust effect of MDD up to the time of imaging, as well as children’s age, sex, and PO guilt symptoms (OR, 0.86; 95% CI, 0.28–2.65; P = .79), children with larger right-side AI volumes were significantly less likely to be diagnosed as having full MDD after their imaging (diagnosed on average 14 months after the imaging date [OR, 0.96; 95% CI, 0.01–0.75; P = .03]) (Figure 3). Left-side AI volume was not significantly associated with children’s MDD diagnosis after the time of imaging (OR, 2.70; 95% CI, 0.24–29.88; P = .42)." (Results, "Do AI Volumes Predict a Subsequent Depression Diagnosis?", para 1)
"Results indicated that smaller right AI volume remained a significant predictor (OR, 0.12; 95% CI, 0.01–0.99; P = .04) of DSM-5 MDD diagnosis after imaging when covarying for MDD from baseline up until the time of imaging, age, sex, PO guilt, and children’s stressful or traumatic life events." (Results, same paragraph, follow-up model with life-event covariates)
The model and the predictor scale:
"Binary logistic regression analysis was used to test whether schoolchildren’s right or left AI volume predicted the likelihood of being diagnosed as having MDD any time after their imaging." (Methods, Statistical Analysis, "AI Volume as a Predictor of Full MDD After Imaging")
"AI volumes were adjusted by the total segmented whole-brain volume (structure divided by whole-brain volume, times 1000) before all analyses." (Methods, "Magnetic Resonance Imaging Acquisition and AI Volume Analysis", para 2)
"Fourth was MDD diagnosed after the time of imaging (yes or no) (MDD after imaging variable [n = 24]). This variable is used as the dependent variable in the final analysis." (Methods, Measures, "DSM Psychiatric Diagnoses")
What the paper's own numbers show
Method. A logistic-regression Wald interval is symmetric on the log-odds scale. From a printed OR, lower bound L and upper bound U, the estimate implied by the interval is the geometric midpoint exp((ln L + ln U)/2), the standard error is SE = (ln U − ln L)/(2 × 1.96), and the two-sided P value is 2Φ(−|ln OR|/SE). This was applied to all five estimates in the Results paragraph. For the right AI estimate the printed bounds were also varied over every value that rounds to them (0.005–0.015 and 0.745–0.755). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Right AI OR, main model: is the OR inside its CI? | Abstract; Results para 1 | OR 0.96; 95% CI 0.01–0.75 | No: 0.96 > 0.75 |
| Right AI, OR implied by the printed CI | derived from Abstract / Results | 0.96 | 0.087 (0.061–0.106 allowing for rounding of the bounds) |
| Right AI, SE(log OR) implied by the CI | derived | — | 1.10 (1.00–1.28 allowing for rounding) |
| Right AI, P implied by the CI | Abstract; Results para 1 | P = .03 | 0.026 (z = −2.22; 0.024–0.029 allowing for rounding) |
| Right AI, P that OR 0.96 would have at that SE | derived | P = .03 | 0.97 (0.967–0.975) |
| Right AI, 95% CI that OR 0.96 would need for P = .03 | derived | 0.01–0.75 | 0.925–0.996 (SE 0.019) |
| Prior MDD | Results para 1 | OR 11.38; 2.88–44.94; P < .01 | midpoint 11.377; P = 0.001 (consistent) |
| PO guilt | Results para 1 | OR 0.86; 0.28–2.65; P = .79 | midpoint 0.861; P = 0.795 (consistent) |
| Left AI volume | Results para 1 | OR 2.70; 0.24–29.88; P = .42 | midpoint 2.678; P = 0.423 (consistent) |
| Right AI, + life-event covariates | Results para 1 | OR 0.12; 0.01–0.99; P = .04 | midpoint 0.099 (0.070–0.122 allowing for rounding); P = 0.048–0.050 (OR inside its CI; P close) |
In the four other estimates the printed OR sits at the log-midpoint of its CI to within rounding: for prior MDD, for example, ln(OR) − ln(L) = 1.374 and ln(U) − ln(OR) = 1.373. Only the right AI main-model estimate departs from this pattern: ln(0.96) − ln(0.01) = 4.564, while ln(0.75) − ln(0.96) = −0.247. The CI implies an SE of 1.10, close to the left AI estimate's 1.23 and the follow-up model's 1.17, so the interval's width is in line with the other volume estimates from the same 24 events. It looks asymmetric only because the printed OR lies outside it.
Figure
Figure 1. Each row is one estimate from the Results paragraph on MDD after imaging, on a single log-scale odds-ratio axis, with the thin vertical line at OR = 1. Orange circles and lines are the ORs and 95% CIs as printed in the paper. Open blue diamonds mark the geometric midpoint of each printed CI; for the bottom four estimates they coincide with, or sit close to, the printed OR. The top row is the right AI main-model estimate as printed: the OR (0.96) is to the right of its own interval (0.01–0.75). The second row shows the estimate implied by that interval, 0.087 (blue diamond; the printed CI is shown faded); the short thick blue bar spans 0.061–0.106, the values obtained when the printed bounds vary within their rounding. The third row shows the 95% CI (0.925–0.996, an open blue square with a short line hidden mostly behind it) that an OR of 0.96 would need in order to have P = .03. All recomputed values come from recompute.py.txt.
Figure 2. The blue curve gives the two-sided Wald P value for any right AI odds ratio, at the standard error (SE of log OR = 1.10) implied by the printed CI of 0.01–0.75. The reported P = .03 falls on the curve at OR ≈ 0.087 (filled blue diamond, P ≈ 0.026). At the printed OR of 0.96 the curve gives P ≈ 0.97 (open blue diamond). The orange circle marks the reported pair (OR 0.96, P = .03), which lies far off the curve: the printed CI and P are consistent with each other, and the printed OR is not consistent with either.
Why it matters
The right AI result is the third aim of the study (the Objectives ask "whether AI volumes predict subsequent likelihood of having an episode of MDD") and the only predictive result reported in the abstract. The Discussion builds on it: it says that "right-side AI volume was a significant and robust predictor of DSM-5 MDD diagnosis after imaging", that "these study findings provide evidence for the first structural brain biomarker to date of risk for recurrent depression in childhood", and that reduced AI volume "served as a statistically significant biomarker of later depression recurrence".
The direction and significance of the association are not affected. The printed CI excludes 1, the printed P is .03, the two agree, and the follow-up model with life-event covariates (OR 0.12; 0.01–0.99; P = .04) points the same way. What is wrong is the effect size, and the difference is large. Read at face value, 0.96 describes a 4% reduction in the odds of later MDD per unit of whole-brain-adjusted AI volume (structure ÷ whole-brain volume × 1000), and an estimate that close to 1 could not reach P = .03 with an interval this wide. The interval and P value instead imply an OR near 0.09, a 91% reduction per unit, though one that is very imprecisely estimated: the printed CI spans a 75-fold range and the outcome has 24 events. A reader who uses the printed OR would conclude that the predictive effect is negligible, or would find the result self-contradictory. The correct point estimate cannot be read from the paper; only the authors' regression output can supply it. Studies that pool or cite this estimate would carry the error forward.
Explanations considered
Typesetting or archiving artifact. Ruled out. The same "0.96; 95% CI, 0.01–0.75; P = .03" appears in the abstract and in the Results of the PMC author manuscript and of the version of record on jamanetwork.com. The number sits in running prose, so a lost minus sign, a misread mid-dot or a shifted table cell cannot explain it.
Different scale or units for the predictor. Ruled out. Rescaling AI volume (per unit vs per SD, or a different multiplier on the whole-brain adjustment) changes the OR and both CI bounds together. It cannot place the estimate outside its own interval.
A different model behind the OR than behind the CI. Not supported. The Methods describe a single binary logistic regression with age, sex, prior MDD, PO symptoms and AI volume, and the abstract and the Results attach the same OR, CI and P to the same covariate set.
Rounding. Ruled out. Allowing each printed bound to take any value that rounds to it moves the implied OR only between 0.061 and 0.106.
Profile-likelihood or other asymmetric interval. Ruled out as an explanation for an estimate outside the interval. Such intervals can sit off-centre around the estimate, but they always contain it. The other four estimates in the paragraph are also almost exactly log-symmetric around their printed ORs, which indicates Wald-type intervals.
One-sided test. Not relevant. A one-sided P value would change the P value but not whether the estimate lies inside its own CI.
A dropped digit (e.g. 0.096 printed as 0.96). Possible but unverified. 0.096 lies within the rounding range of the CI's midpoint (0.061–0.106) and gives P = 0.033 at the implied SE. The paper offers no way to confirm the exact value.
The claim that the CI is "implausibly wide". Not an independent error. The implied SE of 1.10 on the log scale is similar to the left AI estimate's (1.23) and the follow-up model's (1.17). This width is plausible for a continuous predictor with 24 events.
Side observations, not part of this finding. In the follow-up model the printed OR (0.12) lies inside its CI and close to the midpoint (0.099); the CI implies P ≈ 0.049 against a printed .04, a small difference that could reflect a different test statistic. Separately, the Table's group-comparison column prints several "OR" values with negative CI bounds (for example "OR, -0.71; 95% CI, -1.10 to 0.37" for female sex), which looks like a labelling or scale issue in that column. Neither affects the result above.
Status
As of 2026-10-23:
- Version of record. The jamanetwork.com full text (https://jamanetwork.com/journals/jamapsychiatry/fullarticle/1935483, reached through the DOI) was fetched on 2026-10-23. It prints "(odds ratio, 0.96; 95% CI, 0.01-0.75; P = .03)" in the abstract and "[OR, 0.96; 95% CI, 0.01-0.75; P = .03]" in the Results. An earlier audit stage found the neighbouring estimates (11.38, 0.86, 2.70 and 0.12, with their CIs and P values) identical to the PMC copy. The PMC copy (PMC5103694) is the NIH author manuscript and carries the same numbers.
- Supplements. The article has one supplement (eFigures 1–2 and eTables 1–2). According to the publisher's listing, it covers the imaging schedule, subgroup AI volume comparisons, subgroup characteristics and covariate effects on AI volume. None of these is described as holding the logistic regression of MDD after imaging, and direct downloads of the PDF from the publisher and from PMC did not return the file. No source data or analysis code is available.
- Corrections.
src/known.py(Crossref updates, PMC notices, Europe PMC notices) returned none. A Crossreffilter=updates:query for the DOI returned 0 results on 2026-10-23, and the Europe PMC record has no comment/correction links. No correction notice appears on the publisher page. - PubPeer. The PubPeer API returned HTTP 403 on 2026-10-23, so PubPeer could not be checked directly. A web search for the title with "pubpeer", "erratum" and "correction" found no thread or notice.
Sources: PMC5103694 (PubMed Central archive, read with src/show.py); https://jamanetwork.com/journals/jamapsychiatry/fullarticle/1935483; https://api.crossref.org/works?filter=updates:10.1001/jamapsychiatry.2014.1604; Europe PMC REST search for DOI:10.1001/jamapsychiatry.2014.1604.
Suggested correction
Replace the right-side AI volume odds ratio in the abstract and in the Results paragraph "Do AI Volumes Predict a Subsequent Depression Diagnosis?" with the value from the fitted logistic regression, checking that it lies within the reported 95% CI (0.01–0.75) and fits P = .03. The paper's own numbers place it near 0.09 (0.061–0.106). If the CI or P value was the term misreported instead, correct that term and state the corrected triplet in both places. The authors could also state the unit of AI volume for which the OR is expressed (per unit of structure ÷ whole-brain volume × 1000) so readers can interpret the magnitude. In the same correction they might review the "OR" entries with negative confidence bounds in the Table.
Recomputation script (python recompute.py.txt)
"""Recompute the numbers in the PMC5103694 write-up and regenerate fig1/fig2.
Run: python recompute.py.txt
Inputs are the odds ratios, 95% CIs and P values printed in the paper
(Abstract; Results, "Do AI Volumes Predict a Subsequent Depression Diagnosis?", para 1).
Method: for a Wald interval on the log-odds scale, log(OR) is the midpoint of
log(lower) and log(upper), SE = (log(upper) - log(lower)) / (2 * 1.959964),
and the two-sided P = 2 * (1 - Phi(|log OR| / SE)).
"""
import os
import itertools
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.isf(0.025) # 1.959964
# label, reported OR, lower, upper, reported P (as printed)
ROWS = [
("Prior MDD (main model)", 11.38, 2.88, 44.94, "<.01"),
("PO guilt (main model)", 0.86, 0.28, 2.65, ".79"),
("Right AI volume (main model)", 0.96, 0.01, 0.75, ".03"),
("Left AI volume", 2.70, 0.24, 29.88, ".42"),
("Right AI volume (+ life events)", 0.12, 0.01, 0.99, ".04"),
]
def from_ci(lo, hi):
a, b = np.log(lo), np.log(hi)
se = (b - a) / (2 * Z)
mid = (a + b) / 2
return np.exp(mid), se, 2 * norm.sf(abs(mid) / se)
def wald_p(or_, se):
return 2 * norm.sf(abs(np.log(or_)) / se)
def half_unit(x):
"""Interval of values that round to x at the printed number of decimals (2 dp)."""
return x - 0.005, x + 0.005
print("== 1. Each reported OR against the CI it is printed with ==")
print(f"{'quantity':34s} {'OR':>6s} {'95% CI':>13s} {'OR in CI':>8s} {'geo-mid':>8s} "
f"{'SE(logOR)':>9s} {'P(mid)':>7s} {'P(rep OR)':>9s} {'rep P':>5s}")
res = {}
for lab, o, l, u, p in ROWS:
mid, se, pm = from_ci(l, u)
pr = wald_p(o, se)
res[lab] = dict(o=o, l=l, u=u, mid=mid, se=se, pm=pm, pr=pr, p=p)
print(f"{lab:34s} {o:6.2f} {l:5.2f}-{u:<7.2f} {str(l <= o <= u):>8s} {mid:8.3f} "
f"{se:9.3f} {pm:7.3f} {pr:9.3f} {p:>5s}")
print(f"{'':34s} log(OR) - log(lower) = {np.log(o)-np.log(l):6.3f}; "
f"log(upper) - log(OR) = {np.log(u)-np.log(o):6.3f}")
print("\n== 2. Right AI (main model): allow for rounding of the printed CI bounds ==")
# 0.01 could be anything in [0.005, 0.015); 0.75 in [0.745, 0.755)
lo_r, hi_r = half_unit(0.01), half_unit(0.75)
mids, ps, ses, p096 = [], [], [], []
for a, b in itertools.product(np.linspace(*lo_r, 201), np.linspace(*hi_r, 21)):
m, s, pm = from_ci(a, b)
mids.append(m); ps.append(pm); ses.append(s); p096.append(wald_p(0.96, s))
print(f"implied OR (geometric midpoint): {min(mids):.3f} to {max(mids):.3f}")
print(f"implied SE(log OR): {min(ses):.3f} to {max(ses):.3f}")
print(f"implied Wald P: {min(ps):.3f} to {max(ps):.3f} (reported P = .03)")
print(f"P for OR = 0.96 at those SEs: {min(p096):.3f} to {max(p096):.3f}")
print("\n== 3. CI that OR = 0.96 would need for a two-sided P = .03 ==")
z03 = norm.isf(0.03 / 2)
se_req = abs(np.log(0.96)) / z03
ci_req = np.exp(np.log(0.96) + np.array([-Z, Z]) * se_req)
print(f"z = {z03:.3f}; SE(log OR) = {se_req:.4f}; 95% CI = {ci_req[0]:.3f} to {ci_req[1]:.3f}")
# P range 0.025-0.035 (values that round to .03)
for pp in (0.025, 0.035):
s = abs(np.log(0.96)) / norm.isf(pp / 2)
c = np.exp(np.log(0.96) + np.array([-Z, Z]) * s)
print(f" if P = {pp:.3f}: 95% CI = {c[0]:.3f} to {c[1]:.3f}")
r = res["Right AI volume (main model)"]
print("\n== 4. Summary values used in the write-up ==")
print(f"Right AI main: geometric midpoint {r['mid']:.3f}, SE {r['se']:.2f}, z {np.log(r['mid'])/r['se']:.2f}, "
f"P {r['pm']:.3f}; P if OR were 0.96: {r['pr']:.2f}")
print(f"Implied odds reduction per unit volume: {100*(1-r['mid']):.0f}% (vs {100*(1-0.96):.0f}% for OR 0.96)")
print(f"Width of reported CI, upper/lower ratio: {0.75/0.01:.0f}-fold")
lr = res["Left AI volume"]
print(f"SE comparison: right AI {r['se']:.2f}, left AI {lr['se']:.2f}, right AI + life events "
f"{res['Right AI volume (+ life events)']['se']:.2f}")
print(f"Candidate 0.096 within rounding range of implied OR: {min(mids) <= 0.096 <= max(mids)}; "
f"P at OR 0.096, SE {r['se']:.3f}: {wald_p(0.096, r['se']):.3f}")
# life-events model rounding
mids2, ps2 = [], []
for a, b in itertools.product(np.linspace(*half_unit(0.01), 201), np.linspace(*half_unit(0.99), 21)):
m, s, pm = from_ci(a, b); mids2.append(m); ps2.append(pm)
print(f"Right AI + life events: implied OR {min(mids2):.3f}-{max(mids2):.3f}, implied P "
f"{min(ps2):.3f}-{max(ps2):.3f} (reported OR 0.12, P = .04)")
# ---------------------------------------------------------------- figures
BG, TXT, SEC, ORA, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e3e2de"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": SEC,
"svg.fonttype": "none", "font.family": "DejaVu Sans"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: dot-and-interval plot, one log axis
fig, ax = plt.subplots(figsize=(7.2, 5.0), facecolor=BG)
ax.set_facecolor(BG)
rows = [ # (y label, kind)
("Right AI volume, main model\n(as printed)", "rep_main"),
("Right AI volume, main model\n(OR implied by its printed CI)", "mid_main"),
("Right AI volume, main model\n(CI needed if OR = 0.96, P = .03)", "req"),
("Right AI volume + life events", "other:Right AI volume (+ life events)"),
("Left AI volume", "other:Left AI volume"),
("PO guilt", "other:PO guilt (main model)"),
("Prior MDD", "other:Prior MDD (main model)"),
]
n = len(rows)
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
ax.text(1.0, n - 0.35, " OR = 1 (no association)", color=SEC, fontsize=9.5, va="center", ha="left")
for i, (lab, kind) in enumerate(rows):
y = n - 1 - i
if kind == "rep_main":
ax.plot([0.01, 0.75], [y, y], color=ORA, lw=2, zorder=2)
ax.plot(0.96, y, "o", ms=9, color=ORA, zorder=3)
ax.annotate("OR 0.96 printed outside\nits own CI 0.01–0.75", (0.96, y), xytext=(2.2, y),
color=ORA, fontsize=9.5, va="center",
arrowprops=dict(arrowstyle="-", color=ORA, lw=1))
elif kind == "mid_main":
ax.plot([0.01, 0.75], [y, y], color=ORA, lw=2, alpha=0.35, zorder=2)
ax.plot([min(mids), max(mids)], [y, y], color=BLU, lw=5, solid_capstyle="butt", zorder=3)
ax.plot(r["mid"], y, "D", ms=9, color=BLU, mfc=BG, mew=2, zorder=4)
ax.annotate(f"≈{r['mid']:.3f} (range {min(mids):.3f}–{max(mids):.3f}\nwith bound rounding); P ≈ {r['pm']:.3f}",
(1.12, y), color=BLU, fontsize=9.5, va="center")
elif kind == "req":
ax.plot(ci_req, [y, y], color=BLU, lw=2, zorder=2)
ax.plot(0.96, y, "s", ms=9, color=BLU, mfc=BG, mew=2, zorder=3)
ax.annotate(f"{ci_req[0]:.3f}–{ci_req[1]:.3f}", (ci_req[1], y), xytext=(2.2, y),
color=BLU, fontsize=9.5, va="center",
arrowprops=dict(arrowstyle="-", color=BLU, lw=1))
else:
d = res[kind.split(":", 1)[1]]
ax.plot([d["l"], d["u"]], [y, y], color=ORA, lw=2, zorder=2)
ax.plot(d["o"], y, "o", ms=9, color=ORA, zorder=3)
ax.plot(d["mid"], y, "D", ms=9, color=BLU, mfc="none", mew=1.6, zorder=4)
ax.set_yticks(range(n))
ax.set_yticklabels([lab for lab, _ in rows][::-1], fontsize=10)
ax.set_xscale("log")
ax.set_xlim(0.004, 120)
ax.set_ylim(-0.6, n - 0.1)
ax.set_xticks([0.01, 0.1, 1, 10, 100])
ax.set_xticklabels(["0.01", "0.1", "1", "10", "100"])
ax.set_xlabel("Odds ratio for MDD diagnosis after imaging (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
h = [Line2D([], [], color=ORA, marker="o", ms=8, lw=2, label="Reported in the paper (OR and 95% CI)"),
Line2D([], [], color=BLU, marker="D", ms=8, mfc=BG, mew=2, lw=0,
label="Recomputed: geometric midpoint of the printed CI (thick bar: rounding range)"),
Line2D([], [], color=BLU, marker="s", ms=8, mfc=BG, mew=2, lw=2,
label="Recomputed: 95% CI implied by OR 0.96 with P = .03")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.42, -0.13), ncol=1, frameon=False, fontsize=9.5)
fig.tight_layout()
save(fig, "fig1")
# Figure 2: Wald P as a function of the OR, at the SE implied by the printed CI
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
ax.set_facecolor(BG)
grid = np.exp(np.linspace(np.log(0.003), np.log(10), 600))
ax.plot(grid, wald_p(grid, r["se"]), color=BLU, lw=2, zorder=2,
label=f"Recomputed: Wald P for any OR, SE(log OR) = {r['se']:.2f} from the printed CI")
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
ax.plot(0.96, 0.03, "o", ms=10, color=ORA, zorder=4, label="Reported pair: OR 0.96, P = .03")
ax.plot(r["mid"], r["pm"], "D", ms=9, color=BLU, zorder=4)
ax.plot(0.96, r["pr"], "D", ms=9, color=BLU, mfc=BG, mew=2, zorder=4)
ax.annotate(f"OR ≈ {r['mid']:.3f} gives P ≈ {r['pm']:.3f}", (r["mid"], r["pm"]), xytext=(0.006, 0.22),
color=BLU, fontsize=9.5, arrowprops=dict(arrowstyle="-", color=BLU, lw=1))
ax.annotate(f"OR 0.96 gives P ≈ {r['pr']:.2f}", (0.96, r["pr"]), xytext=(0.5, 0.90),
color=BLU, fontsize=9.5, ha="right", va="center",
arrowprops=dict(arrowstyle="-", color=BLU, lw=1))
ax.annotate("printed:\nOR 0.96, P = .03", (0.96, 0.03), xytext=(1.3, 0.10), color=ORA, fontsize=9.5,
arrowprops=dict(arrowstyle="-", color=ORA, lw=1))
ax.set_xscale("log")
ax.set_xlim(0.003, 10)
ax.set_ylim(0, 1.05)
ax.set_xticks([0.01, 0.1, 1, 10])
ax.set_xticklabels(["0.01", "0.1", "1", "10"])
ax.set_xlabel("Odds ratio for right AI volume (log scale)")
ax.set_ylabel("Two-sided P value")
ax.grid(color=GRID, lw=0.8, zorder=0)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.legend(loc="upper center", bbox_to_anchor=(0.45, -0.2), ncol=1, frameon=False, fontsize=9.5)
fig.tight_layout()
save(fig, "fig2")
print("\nFigures written to", HERE)
Abstract cites non-significant interaction odds ratios as evidence that correct fluid significantly reduced mortality
Lancet Child Adolesc Health (2018) · doi:10.1016/S2352-4642(18)30130-5 · PMC6004535
Summary
The abstract states that correct fluid prescription "significantly reduced the risk of early mortality (within 2 days) in all subgroups". It supports this with five adjusted odds ratios (AORs): 1·23, 0·95, 1·70, 1·50 and 0·86. These five values, with their CIs, are the "Multiplicative interaction" column of Table 2, which the table footnote defines as OR11/(OR10 × OR01), and they can be recomputed from the table's other cells. They measure how much the fluid association differs between children with and without a clinical sign. They are not the effect of correct fluid in either group. All five 95% CIs include 1 (approximate p = 0.50, 0.87, 0.07, 0.22, 0.58), and three of the five point estimates are above 1. So the statistics the abstract cites do not show a significant reduction in any subgroup. The same table, read for the correct-versus-wrong contrast within each stratum, does point to lower odds of early death with correct fluid in every subgroup. The headline direction may therefore hold, but the estimates, their label and the significance claim in the abstract's key fluid result are wrong as printed. Table 2 also contains two cell errors in its additive-interaction column.
What the paper reports
Abstract, Findings:
Correct fluid prescription significantly reduced the risk of early mortality (within 2 days) in all subgroups: abnormal respiratory signs (AOR 1·23, 0·68–2·24), abnormal circulatory signs (0·95, 0·53–1·73), pallor (1·70, 0·95–3·02), dehydration signs only (1·50, 0·79–2·88), and abnormal neurological signs (0·86, 0·51–1·48).
Abstract, Interpretation:
the prescription of recommended rehydration guidelines reduces risk of death
Research in context, Added value of this study:
Our findings also show that comorbidities are common and that correct fluid prescription is associated with reduced risk of death in these patients.
Methods, Data analysis (third paragraph, on effect modification):
We calculated the relative excess odds due to interaction, the attributable proportion, and the multiplicative interaction odds ratio (OR).
Results (effect-modification paragraph), where the same numbers appear under their correct name:
In fact, positive multiplicative interactions (OR>1) for association of incorrect fluid management with death in children with dehydration signs (OR 1·50, 95% CI 0·79–2·88), respiratory signs (OR 1·23, 0·68–2·24), and pallor (OR 1·70, 0·95–3·02) are evidence of interaction (figure 3).
Table 2, "Interactions of fluid management with risk factors for in-hospital early deaths" (columns: Multiplicative interaction | Fluid wrong and symptom present (OR10) | Fluid right and symptom present (OR11) | Fluid right and symptom absent (OR01) | RERI | Attributable proportion):
Pallor | 1·70 (0·95 to 3·02) | 2·16 (1·66 to 2·82) | 1·07 (0·69 to 1·67) | 0·29 (0·21 to 0·41) | -0·38 (-1·07 to 0·31) | -0·35 (-1·09 to 0·38)
Circulatory | 0·95 (0·53 to 1·73) | 1·92 (1·47 to 2·51) | 0·64 (0·40 to 1·03) | 0·35 (0·25 to 0·50) | -0·63 (-1·19 to -0·06) | -0·98 (-2·06 to 0·11)
Dehydration signs only | 1·50 (0·79 to 2·88) | 0·96 (0·70 to 1·33) | 0·38 (0·26 to 0·55) | 0·26 (0·15 to 0·45) | 0·25 (-0·18 to 0·48) | 0·40 (-0·53 to 1·33)
Neurological | 0·86 (0·51 to 1·48) | 3·57 (2·75 to 4·64) | 1·15 (0·77 to 1·72) | 0·37 (0·25 to 0·54) | -1·80 (-2·71 to -0·88) | -1·56 (-2·61 to -0·52)
Respiratory | 1·23 (0·68 to 2·24) | 3·18 (2·35 to 4·31) | 1·17 (0·80 to 1·73) | 0·30 (0·18 to 0·49) | -1·31 (-2·16 to -9·45) | -1·11 (-1·90 to -0·33)
Table 2 footnote (excerpts):
Interaction exists if RERI is not zero (RERI=OR11 – OR10 – OR01 + 1).
Multiplicative interaction=OR11/(OR10 × OR01).
The reference group was fluid wrong and symptom absent (OR00)=1·00.
Discussion, final paragraph:
Importantly, prescription of recommended rehydration guidelines reduces risk of death in this group of children.
What the paper's own numbers show
Method. I compared the five abstract values with each column of Table 2. Using the footnote formulas, I recomputed the multiplicative interaction OR, the RERI and the attributable proportion (AP) from the three cell ORs in each row (OR10, OR11, OR01; OR00 = 1). For each printed 95% CI, I took the standard error of log(OR) as (ln upper − ln lower)/(2 × 1.96) and derived an approximate two-sided Wald p value. Within the "symptom present" stratum, the effect of correct versus wrong fluid is OR11/OR10, because both cells share the reference OR00. Within the "symptom absent" stratum it is OR01. The paper does not report a CI for OR11/OR10. I approximated one by adding the two variances on the log scale and ignoring their covariance. Both estimates are contrasts against the same reference group, so their covariance is usually positive, and in that case the approximate interval is wider than the true one. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Respiratory "AOR" for correct fluid | Abstract; Table 2, multiplicative interaction | 1·23 (0·68–2·24) | Identical to Table 2 interaction OR; OR11/(OR10·OR01) = 1.17/(3.18 × 0.30) = 1.23; CI includes 1; p ≈ 0.50 |
| Circulatory "AOR" | Abstract; Table 2 | 0·95 (0·53–1·73) | Identical to interaction OR; 0.64/(1.92 × 0.35) = 0.95; CI includes 1; p ≈ 0.87 |
| Pallor "AOR" | Abstract; Table 2 | 1·70 (0·95–3·02) | Identical to interaction OR; 1.07/(2.16 × 0.29) = 1.71; CI includes 1; p ≈ 0.07 |
| Dehydration signs only "AOR" | Abstract; Table 2 | 1·50 (0·79–2·88) | Identical to interaction OR; 0.38/(0.96 × 0.26) = 1.52 (rounding of the inputs allows 1.47–1.58); CI includes 1; p ≈ 0.22 |
| Neurological "AOR" | Abstract; Table 2 | 0·86 (0·51–1·48) | Identical to interaction OR; 1.15/(3.57 × 0.37) = 0.87 (rounding allows 0.85–0.89); CI includes 1; p ≈ 0.58 |
| Cited CIs that exclude 1 | Abstract | "significantly reduced … in all subgroups" | 0 of 5; 3 of 5 point estimates are above 1 |
| Correct vs wrong fluid, symptom present (OR11/OR10) | Derived from Table 2 | not reported | Respiratory 0.37 (≈0.23–0.60); circulatory 0.33 (≈0.19–0.57); pallor 0.50 (≈0.30–0.83); dehydration only 0.40 (≈0.24–0.65); neurological 0.32 (≈0.20–0.52). Approximate CIs, covariance ignored |
| Correct vs wrong fluid, symptom absent (OR01) | Table 2 | 0·30, 0·35, 0·29, 0·26, 0·37 | Same values; all five printed CIs exclude 1 |
| Identity linking the two | Table 2 footnote | interaction = OR11/(OR10·OR01) | (OR11/OR10) / OR01 equals the interaction OR in every row: the interaction OR is the ratio of the two within-stratum fluid effects |
| Dehydration signs only, RERI | Table 2 | 0·25 (−0·18 to 0·48) | OR11 − OR10 − OR01 + 1 = 0.38 − 0.96 − 0.26 + 1 = 0.16; the printed CI's midpoint is 0.15 |
| Dehydration signs only, AP | Table 2 | 0·40 | RERI/OR11 = 0.16/0.38 = 0.42 with the recomputed RERI; 0.25/0.38 = 0.66 with the printed RERI |
| RERI and AP, other four rows | Table 2 | e.g. −0·38, −0·63, −1·80, −1·31 | Recompute to −0.38, −0.63, −1.79, −1.31; the AP CI midpoints in all four rows, and the RERI CI midpoints for circulatory, pallor and neurological, match the point estimates. The respiratory RERI CI midpoint (−5.81) does not (see next row) |
| Respiratory RERI CI | Table 2 | −1·31 (−2·16 to −9·45) | Upper limit (−9.45) is below both the point estimate and the lower limit, which is impossible; a symmetric interval around −1.31 would end near −0.46 |
So the abstract's numbers are the Table 2 interaction ORs. The cell ORs in each row reproduce them, so the column labels are internally consistent. None of the five is significant. The contrast the abstract's sentence describes (correct versus wrong fluid within a subgroup) is a different quantity. Table 2 points to it being below 1 in all five subgroups, in both strata, but the paper does not report it.
Figure
Figure 1. One log-scale odds-ratio axis; the vertical line marks OR = 1. Orange circles and lines are the five values the abstract cites as AORs for correct fluid prescription, with their printed 95% CIs. They are identical to Table 2's multiplicative-interaction column. Each crosses 1, and the approximate p value is printed beside it. Blue squares are the odds ratio for correct versus wrong fluid among children with the sign present, recomputed from the same Table 2 row as OR11/OR10. Their intervals are approximate because the covariance of the two estimates is not reported and is ignored here. All five blue estimates lie below 1. The abstract's sentence describes the blue quantity but cites the orange numbers.
Figure 2. Table 2's RERI column on a linear axis; the vertical line marks 0. Orange: RERI and 95% CI as printed. Blue squares: OR11 − OR10 − OR01 + 1 from the same row. Gray ticks: midpoint of the printed CI. In three rows (circulatory, pallor, neurological) the three agree. For dehydration signs only, the printed point (0.25) differs from both the formula (0.16) and the CI midpoint (0.15). For respiratory signs, the printed point (−1.31) matches the formula, but the printed interval runs from −2.16 to −9.45, so the point lies outside it and the CI midpoint (−5.81) is far from it.
Why it matters
The effect of correct fluid prescription is one of the paper's two stated aims and is named in its title. The abstract's Findings sentence is the only place in the summary that quantifies it, and the Interpretation ("reduces risk of death") and the closing Discussion sentence are the abstract-level and concluding statements of it. The body gives one other quantitative statement: "The predicted probability of early in-hospital death was reduced by 6% when derived from model II with correct fluid prescription compared with when fluid prescription is wrong" (Results, effect-modification paragraph). That is an overall model prediction, not a subgroup estimate, and it is printed without a CI or test, so it does not support "significantly … in all subgroups". A reader of the abstract alone would take the five AORs as subgroup treatment effects. Three are above 1 and none excludes 1, so the reader would see a contradiction or would misread interaction estimates as effects of fluid prescription. Meta-analysts and guideline reviewers who extract the abstract figures would record five non-significant ORs, some pointing to harm, for a result the paper calls a significant reduction. The Results paragraph also calls three non-significant interaction ORs "evidence of interaction". The data do not support that claim on the multiplicative scale. On the additive scale, the RERI CIs for pallor and dehydration signs only include 0, and the respiratory RERI CI is unusable as printed. The paper's own cells point to a consistent protective association of correct fluid. The defect is therefore in which estimates are reported and how they are described, not necessarily in the direction of the conclusion. The magnitude and precision of the subgroup effects cannot be checked from what is published.
Explanations considered
- Text-conversion or extraction artifact. Not supported. The version-of-record check found the same sentence and numbers in the publisher-deposited JATS XML in Europe PMC and in the separately supplied MEDLINE abstract (PMID 29971245). The publisher's typeset Figure 3 image was also inspected. Its interaction panel ("Adjusted odds ratios (model II-interactions)") shows the five abstract subgroups (plus airway signs) with CIs crossing 1. In the XML, Table 2's cells are aligned with their headers.
- The abstract values are the within-subgroup fluid effects and the Table 2 column is mislabelled. Not supported. Each value equals OR11/(OR10 × OR01) recomputed from the other cells in its row, within input rounding. The Results paragraph calls the same numbers "multiplicative interactions". Neither within-stratum effect (OR11/OR10 or OR01) is close to any of the five values.
- "Significantly" refers to something other than the cited ORs, such as the additive interaction. It does not account for the numbers. The sentence attaches the five ORs directly to the claim. RERI is a measure of interaction, not of the effect of fluid. Its printed CIs include 0 for pallor (−1.07 to 0.31) and dehydration signs only (−0.18 to 0.48), so it is not significant in all subgroups either.
- "Significantly" refers to the within-stratum correct-versus-wrong contrasts, and the wrong column was copied into the abstract. This would explain the wording. The approximate intervals for OR11/OR10 and the printed intervals for OR01 all exclude 1. But the paper never reports those subgroup estimates with CIs, so the claim remains unverifiable as published, and the numbers it does cite contradict it. The approximate intervals here ignore a covariance that only the authors' model output provides.
- Rounding. Rounding explains the small gaps between printed and recomputed interaction ORs (for example 1.50 against 1.52, inside the rounding range 1.47–1.58). It cannot move any CI to exclude 1. It also cannot explain the dehydration RERI (0.25 against 0.16, with a CI midpoint of 0.15 and an AP that matches 0.16) or the respiratory RERI interval, whose upper limit lies below its lower limit.
Status
As of 2026-10-23:
- Version of record. The publisher's HTML and PDF could not be opened directly. doi.org redirected to an Elsevier placeholder, and thelancet.com returned HTTP 403. The abstract sentence, Table 2 and the interaction values were confirmed in three publisher-derived copies: the publisher-deposited JATS XML in Europe PMC (CC BY), the MEDLINE abstract (PMID 29971245), and the publisher's Figure 3 image (gr3.jpg).
- Supplements. The Europe PMC supplementary bundle for PMC6004535 contains only the article figures (gr1–gr3). The JATS XML has no supplementary-material element or appendix reference. No source of subgroup-specific correct-versus-wrong fluid ORs with CIs was found. A web appendix listed on the journal site could not be ruled out, because that site returned HTTP 403.
- Corrections and retractions. Crossref update metadata, PMC notices and Europe PMC notices list no correction, erratum, expression of concern or retraction. A web search for the title with "correction" or "erratum" found no correction notice.
- Linked commentary. The PMC record links an accompanying Comment in the same journal: "Treating dehydrating diarrhoea at district hospital level in sub-Saharan Africa: from policy to reality", Lancet Child Adolesc Health 2018, pp 471–472, doi:10.1016/S2352-4642(18)30166-4, PMID 30169312. Its text was not read. It is not open access, Europe PMC holds no abstract or full text for it, and the journal and ScienceDirect pages returned HTTP 403. Whether it repeats or relies on the abstract's "significantly reduced … in all subgroups" claim is therefore not known.
- PubPeer. PubPeer's API returned a human-verification challenge (HTTP 403) to scripted access, so it could not be queried directly. A web search for the title together with "PubPeer" found no thread. Absence of a PubPeer discussion is therefore not fully confirmed.
- Sources. Article as archived in PMC (PMC6004535); Europe PMC JATS XML and supplementary files; MEDLINE record PMID 29971245; Crossref record for doi:10.1016/S2352-4642(18)30130-5; Europe PMC and Crossref records for the linked Comment (PMID 30169312, doi:10.1016/S2352-4642(18)30166-4), metadata only.
Suggested correction
- In the abstract Findings, replace the five interaction ORs with estimates of the quantity the sentence describes. One option is the adjusted OR for correct versus wrong fluid within each subgroup (OR11/OR10, with a CI from the fitted model including the covariance term). Another is the overall adjusted OR for correct fluid. Alternatively, keep the interaction ORs but describe them accurately, for example: "There was no evidence that the association of correct fluid prescription with early mortality differed by clinical sign on the multiplicative scale (interaction ORs 0·86–1·70, all 95% CIs including 1)." Remove "significantly" and "in all subgroups" unless subgroup-specific estimates with CIs excluding 1 are reported.
- In the Results effect-modification paragraph, revise "are evidence of interaction" for the three interaction ORs whose CIs include 1.
- In Table 2, correct the dehydration-signs-only RERI. The footnote formula gives 0.16, consistent with the printed CI midpoint (0.15) and the printed AP (0.40, against 0.42). Correct the respiratory RERI CI, whose printed upper limit (−9·45) cannot be right; a symmetric interval would end near −0.46.
- Consider adding a column to Table 2 with the within-stratum correct-versus-wrong fluid OR and its CI for each sign, so readers can check the abstract's claim directly.
Recomputation script (python recompute.py.txt)
"""Recompute the fluid-prescription subgroup results of PMC6004535
(Lancet Child Adolesc Health 2018, doi:10.1016/S2352-4642(18)30130-5).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes, next to this file:
fig1.svg/png the figure used in writeup.md (same design as candidate 1)
fig2.svg/png Table 2 additive-interaction (RERI) cells, reported vs recomputed
candidate1..5.svg/png five alternative designs for figure 1, for review
Inputs are copied from the paper as archived in PubMed Central:
Abstract, Findings: the five "AOR" values cited for correct fluid prescription
Table 2: multiplicative interaction OR, OR10, OR11, OR01, RERI and attributable
proportion (AP), each with its 95% CI. Reference group OR00 = 1 (fluid wrong,
symptom absent). Footnote: RERI = OR11 - OR10 - OR01 + 1; AP = RERI/OR11;
multiplicative interaction = OR11/(OR10 x OR01).
"""
import itertools
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)
# ---------------------------------------------------------------- inputs (from the paper)
ROWS = ["Respiratory", "Circulatory", "Pallor", "Dehydration signs only", "Neurological"] # abstract order
# Abstract, Findings: "(AOR 1.23, 0.68-2.24) ..." in this order
ABSTRACT = {
"Respiratory": (1.23, 0.68, 2.24),
"Circulatory": (0.95, 0.53, 1.73),
"Pallor": (1.70, 0.95, 3.02),
"Dehydration signs only": (1.50, 0.79, 2.88),
"Neurological": (0.86, 0.51, 1.48),
}
# Table 2 (row label: columns as printed)
T2 = {
"Pallor": dict(MI=(1.70, 0.95, 3.02), OR10=(2.16, 1.66, 2.82), OR11=(1.07, 0.69, 1.67),
OR01=(0.29, 0.21, 0.41), RERI=(-0.38, -1.07, 0.31), AP=(-0.35, -1.09, 0.38)),
"Circulatory": dict(MI=(0.95, 0.53, 1.73), OR10=(1.92, 1.47, 2.51), OR11=(0.64, 0.40, 1.03),
OR01=(0.35, 0.25, 0.50), RERI=(-0.63, -1.19, -0.06), AP=(-0.98, -2.06, 0.11)),
"Dehydration signs only": dict(MI=(1.50, 0.79, 2.88), OR10=(0.96, 0.70, 1.33), OR11=(0.38, 0.26, 0.55),
OR01=(0.26, 0.15, 0.45), RERI=(0.25, -0.18, 0.48), AP=(0.40, -0.53, 1.33)),
"Neurological": dict(MI=(0.86, 0.51, 1.48), OR10=(3.57, 2.75, 4.64), OR11=(1.15, 0.77, 1.72),
OR01=(0.37, 0.25, 0.54), RERI=(-1.80, -2.71, -0.88), AP=(-1.56, -2.61, -0.52)),
"Respiratory": dict(MI=(1.23, 0.68, 2.24), OR10=(3.18, 2.35, 4.31), OR11=(1.17, 0.80, 1.73),
OR01=(0.30, 0.18, 0.49), RERI=(-1.31, -2.16, -9.45), AP=(-1.11, -1.90, -0.33)),
}
SHORT = {"Respiratory": "Respiratory", "Circulatory": "Circulatory", "Pallor": "Pallor",
"Dehydration signs only": "Dehydration only", "Neurological": "Neurological"}
def se_log(c):
"""SE of log(OR) implied by a symmetric-on-log 95% CI."""
return (np.log(c[2]) - np.log(c[1])) / (2 * Z)
def p_from_ci(c):
return 2 * norm.sf(abs(np.log(c[0]) / se_log(c)))
def f2(x):
return f"{x:.2f}"
# ---------------------------------------------------------------- 1. abstract = Table 2 interaction column
print("1) Abstract 'AOR' values vs Table 2 'Multiplicative interaction' column")
P_ABS = {}
for k in ROWS:
a, t = ABSTRACT[k], T2[k]["MI"]
P_ABS[k] = p_from_ci(a)
print(f" {k:23s} abstract {a} Table2 MI {t} identical={a == t} "
f"CI includes 1={a[1] <= 1 <= a[2]} point>1={a[0] > 1} approx two-sided p={P_ABS[k]:.2f}")
n_excl = sum(not (ABSTRACT[k][1] <= 1 <= ABSTRACT[k][2]) for k in ROWS)
n_above = sum(ABSTRACT[k][0] > 1 for k in ROWS)
print(f" CIs excluding 1: {n_excl} of 5; point estimates above 1: {n_above} of 5")
print(" geometric midpoint of each CI (check that CIs are symmetric on the log scale):")
for k in ROWS:
a = ABSTRACT[k]
print(f" {k:23s} sqrt(lo*hi)={np.sqrt(a[1] * a[2]):.3f} vs point {a[0]}")
# ---------------------------------------------------------------- 2. recompute interaction measures
print("\n2) Interaction measures recomputed from Table 2 component ORs (footnote formulas)")
REC = {}
for k in ROWS:
d = T2[k]
o10, o11, o01 = d["OR10"][0], d["OR11"][0], d["OR01"][0]
mi = o11 / (o10 * o01)
# range allowed by 2-decimal rounding of the three inputs
grid = [(o11 + e1) / ((o10 + e2) * (o01 + e3)) for e1, e2, e3 in itertools.product((-0.005, 0.005), repeat=3)]
reri = o11 - o10 - o01 + 1
ap = reri / o11
REC[k] = dict(MI=mi, MI_lo=min(grid), MI_hi=max(grid), RERI=reri, AP=ap, AP_repRERI=d["RERI"][0] / o11)
print(f" {k:23s} MI reported {f2(d['MI'][0])} recomputed {mi:.3f} (rounding range {min(grid):.3f}-{max(grid):.3f})"
f" | RERI reported {f2(d['RERI'][0])} recomputed {reri:.2f}"
f" | AP reported {f2(d['AP'][0])}; RERI_recomputed/OR11={ap:.2f}; RERI_reported/OR11={REC[k]['AP_repRERI']:.2f}")
print("\n CI midpoints (RERI and AP CIs are symmetric on the linear scale):")
for k in ROWS:
d = T2[k]
mr = (d["RERI"][1] + d["RERI"][2]) / 2
ma = (d["AP"][1] + d["AP"][2]) / 2
print(f" {k:23s} RERI CI midpoint {mr:.3f} (reported point {f2(d['RERI'][0])}) | AP CI midpoint {ma:.3f} (reported {f2(d['AP'][0])})")
r = T2["Respiratory"]["RERI"]
sym_upper = 2 * r[0] - r[1]
print(f" Respiratory RERI CI printed ({r[1]} to {r[2]}): upper < lower = {r[2] < r[1]}; "
f"upper < point = {r[2] < r[0]}; a symmetric interval around {r[0]} would end near {sym_upper:.2f}")
# ---------------------------------------------------------------- 3. effect of correct vs wrong fluid within strata
print("\n3) Correct vs wrong fluid within each stratum (implied by Table 2; same model per row)")
print(" symptom present: OR11/OR10; approximate CI ignores the covariance of the two log-ORs")
print(" symptom absent : OR01 (contrast with OR00 = 1), CI as printed")
WITHIN = {}
for k in ROWS:
d = T2[k]
ratio = d["OR11"][0] / d["OR10"][0]
s = np.sqrt(se_log(d["OR11"]) ** 2 + se_log(d["OR10"]) ** 2)
lo, hi = ratio * np.exp(-Z * s), ratio * np.exp(Z * s)
p = 2 * norm.sf(abs(np.log(ratio) / s))
WITHIN[k] = (ratio, lo, hi, p)
o01 = d["OR01"]
print(f" {k:23s} present: {ratio:.2f} (approx {lo:.2f}-{hi:.2f}, p{"<0.001" if p < 0.001 else "~" + format(p, ".3f")})"
f" | absent: OR01 {f2(o01[0])} ({f2(o01[1])}-{f2(o01[2])}), CI excludes 1={o01[2] < 1}")
# identity: interaction OR = (present-stratum effect) / (absent-stratum effect)
print(f" {'':23s} check: (OR11/OR10)/OR01 = {ratio / o01[0]:.3f} = interaction OR {REC[k]['MI']:.3f}")
rng = [WITHIN[k][0] for k in ROWS]
print(f" range OR11/OR10: {min(rng):.2f}-{max(rng):.2f}; range OR01: "
f"{min(T2[k]['OR01'][0] for k in ROWS):.2f}-{max(T2[k]['OR01'][0] for k in ROWS):.2f}")
# ---------------------------------------------------------------- figure style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
W = 7.2
plt.rcParams.update({
"font.family": ["Inter", "DejaVu Sans"], "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
TICKS = [0.1, 0.2, 0.5, 1, 2, 4]
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), bbox_inches="tight", pad_inches=0.12, **kw)
plt.close(fig)
def log_axis(ax, lo=0.1, hi=4.6):
ax.set_xscale("log")
ax.set_xlim(lo, hi)
tk = [t for t in TICKS if lo <= t <= hi]
ax.set_xticks(tk)
ax.set_xticklabels([str(t) for t in tk])
ax.set_xlim(lo, hi)
ax.minorticks_off()
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.axvline(1, color=INK2, lw=0.9, zorder=1)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
# ---------------------------------------------------------------- candidate 1: two-series forest plot
def cand1(name):
fig, ax = plt.subplots(figsize=(W, 4.0))
y = np.arange(len(ROWS))[::-1].astype(float)
off = 0.17
for yi, k in zip(y, ROWS):
a = ABSTRACT[k]
ax.plot([a[1], a[2]], [yi + off] * 2, color=ORANGE, lw=2, solid_capstyle="round")
ax.plot(a[0], yi + off, "o", ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.text(4.5, yi + off, f"{f2(a[0])} ({f2(a[1])}–{f2(a[2])}), p≈{P_ABS[k]:.2f}",
va="center", ha="left", fontsize=10, color=INK)
w = WITHIN[k]
ax.plot([w[1], w[2]], [yi - off] * 2, color=BLUE, lw=2, solid_capstyle="round")
ax.plot(w[0], yi - off, "s", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=3)
ax.text(4.5, yi - off, f"{f2(w[0])} (≈{f2(w[1])}–{f2(w[2])})", va="center", ha="left",
fontsize=10, color=INK2)
log_axis(ax, 0.17, 4.4)
ax.set_yticks(y)
ax.set_yticklabels([SHORT[k] for k in ROWS], fontsize=10.5, color=INK)
ax.set_ylim(-0.7, len(ROWS) - 0.3)
ax.set_xlabel("Odds ratio for early death (log scale)", color=INK2)
leg = [Line2D([], [], color=ORANGE, marker="o", ms=8, mec=BG, label="Reported: 'AOR' cited in the abstract for correct fluid\n(= Table 2 multiplicative-interaction OR)"),
Line2D([], [], color=BLUE, marker="s", ms=8, mec=BG, label="Recomputed: correct vs wrong fluid, symptom present\n(OR11/OR10 from Table 2; approximate CI)")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, -0.17), ncol=2, frameon=False, fontsize=9.6 if False else 10,
handlelength=1.6, columnspacing=1.2)
ax.set_title("Abstract: “significantly reduced the risk of early mortality … in all subgroups”",
loc="left", fontsize=11, color=INK, pad=10)
save(fig, name)
# ---------------------------------------------------------------- candidate 2: abstract vs Table 2 text panels
def cand2(name):
fig = plt.figure(figsize=(W, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(-6, 54)
ax.axis("off")
def panel(x0, w, title, color):
ax.add_patch(FancyBboxPatch((x0, 4), w, 48, boxstyle="round,pad=0.4,rounding_size=1.5",
fc="#ffffff", ec=color, lw=2))
ax.text(x0 + 2, 49, title, fontsize=11, weight="bold", va="top", color=INK)
panel(1, 43, "Abstract, Findings", ORANGE)
abs_txt = ("“Correct fluid prescription significantly\n reduced the risk of early mortality\n"
" (within 2 days) in all subgroups:”")
ax.text(3, 43.5, abs_txt, fontsize=10, va="top", color=INK, style="italic")
yy = 31
for k in ROWS:
a = ABSTRACT[k]
ax.text(4, yy, SHORT[k], fontsize=10, va="center", color=INK2)
ax.text(42, yy, f"AOR {f2(a[0])} ({f2(a[1])}–{f2(a[2])})", fontsize=10, va="center", ha="right", color=INK)
yy -= 5
ax.text(3, 6.5, "values as printed (reported)", fontsize=10, color=INK2)
panel(56, 43, "Table 2, “Multiplicative interaction”", BLUE)
ax.text(58, 43.5, "Footnote: interaction = OR11/(OR10 × OR01)\nRecomputed from the table's own cells:",
fontsize=10, va="top", color=INK)
yy = 31
for k in ROWS:
a = T2[k]["MI"]
ax.text(58, yy, SHORT[k], fontsize=10, va="center", color=INK2)
ax.text(97, yy, f"{f2(a[0])} ({f2(a[1])}–{f2(a[2])}) → {REC[k]['MI']:.2f}",
fontsize=10, va="center", ha="right", color=INK)
yy -= 5
ax.text(58, 6.5, "printed (CI) → recomputed from cells", fontsize=10, color=INK2)
ax.add_patch(FancyArrowPatch((44.8, 21), (55.2, 21), arrowstyle="<|-|>", mutation_scale=14, color=INK2, lw=2))
ax.text(50, 23.5, "same\nfive\nnumbers", ha="center", va="bottom", fontsize=10, color=INK)
ax.text(50, -1.5, "The abstract cites the interaction ORs as “AOR” for correct fluid. "
"They are ratios of two effects, not effects of correct fluid; 0 of 5 CIs exclude 1.",
ha="center", va="center", fontsize=10, color=INK, wrap=True)
save(fig, name)
# ---------------------------------------------------------------- candidate 3: calculation flow grid
def cand3(name):
fig = plt.figure(figsize=(W, 3.9))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 52)
ax.axis("off")
cols = [(20, "OR10\nwrong fluid,\nsign present"), (33, "OR11\ncorrect fluid,\nsign present"),
(46, "OR01\ncorrect fluid,\nsign absent"), (66, "OR11/(OR10·OR01)\nrecomputed ≈\nabstract 'AOR'"),
(86, "OR11/OR10\ncorrect vs wrong,\nsign present")]
for x, lab in cols:
ax.text(x, 49, lab, ha="center", va="top", fontsize=10, color=INK, linespacing=1.15)
ax.text(33, 51.5, "Table 2, as printed (reported)", ha="center", fontsize=10, color=ORANGE, weight="bold")
ax.text(75, 51.5, "computed from those cells (recomputed)", ha="center", fontsize=10, color=BLUE, weight="bold")
ax.plot([13, 52], [37.5, 37.5], color=ORANGE, lw=2)
ax.plot([55, 97], [37.5, 37.5], color=BLUE, lw=2)
yy = 32
for k in ROWS:
d = T2[k]
ax.text(1, yy, SHORT[k], va="center", fontsize=10, color=INK)
for x, key in ((20, "OR10"), (33, "OR11"), (46, "OR01")):
ax.text(x, yy, f2(d[key][0]), ha="center", va="center", fontsize=10.5, color=INK)
ax.annotate("", xy=(54.5, yy), xytext=(50.5, yy), arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.2))
ax.text(66, yy, f"{REC[k]['MI']:.2f} (CI {f2(d['MI'][1])}–{f2(d['MI'][2])})", ha="center", va="center", fontsize=10.5, color=INK)
ax.text(86, yy, f"{WITHIN[k][0]:.2f}", ha="center", va="center", fontsize=10.5, color=INK, weight="bold")
yy -= 5.6
ax.text(65, 2.5, "all 5 CIs include 1", ha="center", fontsize=10, color=INK2)
ax.text(88, 2.5, "all 5 below 1", ha="center", fontsize=10, color=INK2)
save(fig, name)
# ---------------------------------------------------------------- candidate 4: interaction plot, small multiples
def cand4(name):
fig, axes = plt.subplots(1, 5, figsize=(W, 3.5), sharey=True)
for ax, k in zip(axes, ROWS):
d = T2[k]
ax.plot([0, 1], [1, d["OR01"][0]], color=ORANGE, lw=2, ls=(0, (2.5, 1.5)), marker="o", ms=8, mfc=BG, mec=ORANGE, mew=2)
ax.plot([0, 1], [d["OR10"][0], d["OR11"][0]], color=ORANGE, lw=2, marker="o", ms=8, mec=ORANGE, mew=1.2)
ax.set_yscale("log")
ax.set_ylim(0.13, 5)
ax.set_yticks([0.25, 0.5, 1, 2, 4])
ax.set_yticklabels(["0.25", "0.5", "1", "2", "4"])
ax.minorticks_off()
ax.axhline(1, color=INK2, lw=0.9, zorder=0)
ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_xlim(-0.35, 1.35)
ax.set_xticks([0, 1])
ax.set_xticklabels(["wrong", "correct"], fontsize=10)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.set_title(f"{SHORT[k].replace(' only', '')}\ninteraction {f2(d['MI'][0])}", fontsize=10, color=INK)
ax.text(0.5, 0.03, f"×{WITHIN[k][0]:.2f} | ×{d['OR01'][0]:.2f}", transform=ax.transAxes,
ha="center", fontsize=10, color=BLUE)
axes[0].set_ylabel("OR for early death vs\nwrong fluid, sign absent (log)", color=INK2)
fig.supxlabel("Fluid prescription at admission", fontsize=10, color=INK2, y=0.06)
leg = [Line2D([], [], color=ORANGE, lw=2.5, marker="o", ms=8, mec=ORANGE, label="Reported, sign present (OR10 → OR11): solid line, filled dots"),
Line2D([], [], color=ORANGE, lw=2.5, ls=(0, (2.5, 1.5)), marker="o", ms=8, mfc=BG, mec=ORANGE, mew=2, label="Reported, sign absent (1 → OR01): dashed line, open dots"),
Line2D([], [], color=BLUE, lw=0, marker="$×$", ms=9, label="Recomputed change with correct fluid: present | absent")]
fig.legend(handles=leg, loc="lower center", ncol=1, frameon=False, fontsize=10, bbox_to_anchor=(0.5, -0.2), handlelength=4)
fig.tight_layout(rect=(0, 0.04, 1, 1))
save(fig, name)
# ---------------------------------------------------------------- candidate 5: cited values only, with p
def cand5(name):
fig, ax = plt.subplots(figsize=(W, 3.2))
y = np.arange(len(ROWS))[::-1].astype(float)
ax.axvspan(0.3, 1, color="#f1f0ec", zorder=0)
ax.text(0.31, len(ROWS) - 0.3, "OR < 1 (shaded)",
ha="left", va="center", fontsize=10, color=INK2)
for yi, k in zip(y, ROWS):
a = ABSTRACT[k]
ax.plot([a[1], a[2]], [yi, yi], color=ORANGE, lw=2, solid_capstyle="round")
ax.plot(a[0], yi, "o", ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.text(3.6, yi, f"p ≈ {P_ABS[k]:.2f}", va="center", fontsize=10.5, color=BLUE)
log_axis(ax, 0.3, 3.4)
ax.set_xticks([0.3, 0.5, 1, 2, 3])
ax.set_xticklabels(["0.3", "0.5", "1", "2", "3"])
ax.set_yticks(y)
ax.set_yticklabels([SHORT[k] for k in ROWS], color=INK)
ax.set_ylim(-0.6, len(ROWS) + 0.05)
ax.set_xlabel("'AOR' (95% CI) cited in the abstract, log scale", color=INK2)
leg = [Line2D([], [], color=ORANGE, marker="o", ms=8, mec=BG, label="Reported in the abstract"),
Line2D([], [], color=BLUE, lw=0, marker="$p$", ms=9, label="Recomputed two-sided p from the printed CI")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.45, -0.22), ncol=2, frameon=False, fontsize=10)
save(fig, name)
# ---------------------------------------------------------------- figure 2: RERI cells
def fig2(name):
fig, ax = plt.subplots(figsize=(W, 3.4))
y = np.arange(len(ROWS))[::-1].astype(float)
off = 0.16
for yi, k in zip(y, ROWS):
d = T2[k]["RERI"]
lo, hi = sorted((d[1], d[2]))
ax.plot([lo, hi], [yi + off] * 2, color=ORANGE, lw=2, solid_capstyle="round")
ax.plot(d[0], yi + off, "o", ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.plot(REC[k]["RERI"], yi - off, "s", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=3)
mid = (d[1] + d[2]) / 2
ax.plot(mid, yi + off, "|", ms=12, mew=2, color=INK2, zorder=4)
k = "Dehydration signs only"
yd = y[ROWS.index(k)]
ax.annotate(f"printed {f2(T2[k]['RERI'][0])}; cells give {REC[k]['RERI']:.2f};\nCI midpoint {(T2[k]['RERI'][1] + T2[k]['RERI'][2]) / 2:.2f}",
xy=(0.5, yd), xytext=(1.2, yd), va="center", fontsize=10, color=INK,
arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
k = "Respiratory"
yr = y[ROWS.index(k)]
ax.annotate("printed CI “−2·16 to −9·45”:\nupper limit below point and lower limit",
xy=(-2.16, yr + off), xytext=(-7.5, yr - 0.62), va="center", fontsize=10, color=INK,
arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.axvline(0, color=INK2, lw=0.9)
ax.set_xlim(-10, 3.2)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_yticks(y)
ax.set_yticklabels([SHORT[k] for k in ROWS], color=INK)
ax.set_xlabel("RERI (relative excess risk due to interaction), linear scale; null = 0", color=INK2)
leg = [Line2D([], [], color=ORANGE, marker="o", ms=8, mec=BG, label="Reported RERI (95% CI as printed)"),
Line2D([], [], color=BLUE, lw=0, marker="s", ms=8, mec=BG, label="Recomputed OR11 − OR10 − OR01 + 1"),
Line2D([], [], color=INK2, lw=0, marker="|", ms=12, mew=2, label="Midpoint of printed CI")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.45, -0.2), ncol=3, frameon=False, fontsize=10,
columnspacing=1.0, handletextpad=0.4)
save(fig, name)
for fn, nm in ((cand1, "candidate1"), (cand2, "candidate2"), (cand3, "candidate3"),
(cand4, "candidate4"), (cand5, "candidate5"), (cand1, "fig1"), (fig2, "fig2")):
fn(nm)
print("\nFigures written to", HERE)
Several adjusted odds ratios in the abstract have confidence intervals narrower than the data allow
Lancet Global Health (2025) · doi:10.1016/S2214-109X(25)00061-0 · PMC12208783
Summary
The Abstract, Results and Table 2 report adjusted odds ratios (aORs) for catastrophic health expenditure (CHE; 76 cases among 296 participants) of 2·0 (95% CI 1·9–2·1, p<0·0001) for the second vs first poverty quintile (60 participants each) and 9·7 (9·6–9·8) for subcounty hospitals vs health centres (69 vs 164 participants; 9·8 (9·7–9·9) in Table 2). For a 0/1 covariate held by this many participants, no model-based maximum-likelihood logistic regression can give a standard error below a floor set by the group sizes, and the standard errors implied by the two reported intervals are 11 and 52 times smaller than that floor; three further abstract aORs (female sex, education above secondary school, county hospital) are also below it, by smaller margins. With the standard error implied by the Table 1 counts, the second-quintile aOR becomes 2.0 (0.86–4.67, p = 0.11), close to the paper's own five-facility sensitivity estimate of 2.1 (0.70–6.1) in Supplementary Table 4. The second quintile is the only poverty quintile reported with significantly raised adjusted odds, and the abstract's conclusion that CHE is "driven by socioeconomic … inequalities" leans on it; the facility-level associations keep their direction and significance, but their stated precision is overstated about 50-fold.
What the paper reports
CHE was associated with being aged 30–44 years old (aOR 2·6, 95% CI 1·3–5·3, p=0·010), being female (1·8, 1·3–2·7, p=0·0021), having higher than secondary school education (1·6, 1·1–2·3, p=0·0083), being accompanied during health-care-seeking visits (3·2, 1·7–5·9, p<0·0001), belonging to the second poverty quintile (2·0, 1·9–2·1, p<0·0001), and seeking care from subcounty hospitals (9·7, 9·6–9·8, p<0·0001) and county hospitals (25·1, 15·7–40·2, p<0·0001).
Abstract, Findings
…belonging to the second poverty quintile (2·0, 1·9–2·1, p<0·0001), and seeking care from subcounty (9·7, 9·6–9·8, p<0·0001) and county hospitals (25·1, 15·7–40·2, p<0·0001; table 2).
Results, paragraph on CHE incidence
Table 2 ("Univariable and multivariable logistic regression of health and social factors associated with catastrophic health expenditure"). Its columns are the unadjusted OR (n=296), p, the adjusted OR (n=296) and p. Two of its rows read:
Second quintile | 1·4 (0·8–2·4) | 0·30 | 2·0 (1·9–2·1) | <0·0001
Subcounty hospital | 9·7 (8·4–11·1) | <0·0001 | 9·8 (9·7–9·9) | <0·0001
Table 2
Table 1 gives the counts behind these rows. The first quintile has 13 CHE and 47 no CHE, the second quintile 15 and 45, health centres 11 and 153, subcounty hospitals 27 and 42, and the county hospital 38 and 25.
How the paper uses these results:
…driven by socioeconomic and sex inequalities and impaired access to health care and social protection.
Abstract, Interpretation
Independent drivers of CHE included being female, poverty, being accompanied by a carer during health-care seeking, and seeking care at subcounty or county hospitals.
Research in context, Added value of this study
Our findings showed that poorer quintiles were more likely to incur CHE than the richest quintile, although the relationship across poverty quintiles did not always reach significance and appeared non-linear.
Discussion, paragraph 3
On the analysis, the paper says "Univariable and multivariable logistic regression models generated unadjusted and adjusted odds ratios", that "for the 21 participants without household asset data, multiple imputation with chained equations was also used to impute poverty scores", and "Data were analysed in STATA (version 15) and R (version 4)." (Methods, Statistical analysis). The paper does not describe a clustered, robust or otherwise non-standard variance estimator.
What the paper's own numbers show
Method. For a logistic regression coefficient on a 0/1 indicator held by n₁ of N participants, the model-based variance is the corresponding diagonal element of the inverse Fisher information. For any positive-definite matrix, a diagonal element of the inverse is at least the corresponding element of the inverse of any principal submatrix that contains it, so [I⁻¹]ⱼⱼ ≥ [(I_sub)⁻¹]ⱼⱼ, where I_sub is the intercept-and-indicator block of the same information matrix, evaluated at the full model's fitted values p. That block's inverse gives 1/S₁ + 1/S₀, where S₁ and S₀ sum p(1 − p) over participants with and without the indicator. Since p(1 − p) ≤ 1/4 always, SE ≥ √(4/n₁ + 4/(N − n₁)) for any covariate set and any fitted values. Pooling multiply imputed fits by Rubin's rules can only add variance. Separately, I derived each reported interval's implied SE, ln(upper/lower)/(2 × 1.96), including the full range allowed by rounding the printed limits to one decimal place. I computed crude ORs with Wald intervals from the Table 1 counts. All numbers come from recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| aOR, second vs first poverty quintile | Abstract; Results; Table 2 | 2·0 (1·9–2·1), p<0·0001 | Implied SE 0.0255 (0.013–0.038 allowing for rounding), z = 27. Minimum possible SE with n₁ = 60: 0.289, which is 11× the implied SE and 7.5× the widest SE rounding allows. Narrowest possible 95% CI at aOR 2.0: 1.13–3.53 (p = 0.017) |
| Same, allowing for imputation of quintile in 21 participants | derived | — | If n₁ is anywhere from 39 to 81, the minimum SE is still 0.261, 10× the implied SE |
| Crude OR, second vs first quintile (15/60 vs 13/60) | Table 1 | Table 2 unadjusted: 1·4 (0·8–2·4), p=0·30 | 1.21 (0.52–2.81), SE 0.433, p = 0.67 (Fisher exact p = 0.83). Crude SE ÷ reported adjusted SE = 17 (11–34 allowing for rounding) |
| aOR 2.0 with the count-based SE | derived | — | 95% CI 0.86–4.67, p = 0.11 |
| Second quintile, same outcome and covariates with a five-level facility term | Supplementary Table 4 | 2.1 (0.70–6.1), p=0.099 | Implied SE 0.552, 22× the Table 2 SE. The interval includes 1 (the Wald p from this interval is 0.18) |
| aOR, subcounty hospital vs health centre | Abstract; Results | 9·7 (9·6–9·8), p<0·0001 | Implied SE 0.0053. Minimum possible SE with n₁ = 69: 0.275, 52× larger (35× the widest SE rounding allows). Narrowest possible CI at 9.7: 5.66–16.63 |
| aOR, subcounty hospital | Table 2 | 9·8 (9·7–9·9) | Differs from the Abstract and Results (9·7, 9·6–9·8). Same problem: minimum possible CI at 9.8 is 5.72–16.80 |
| Crude OR, subcounty vs health centre (27/69 vs 11/164) | Table 1 | Table 2 unadjusted: 9·7 (8·4–11·1) | 8.94 (4.10–19.50), SE 0.398, which is 76× the SE implied by the abstract's adjusted interval. aOR 9.7 with this SE: 4.4–21.2 |
| aOR, female | Abstract; Table 2 | 1·8 (1·3–2·7), p=0·0021 | Implied SE 0.186 (at most 0.201 allowing for rounding) vs minimum 0.236. Narrowest possible CI 1.13–2.86, p = 0.013 |
| aOR, above secondary education | Abstract; Table 2 | 1·6 (1·1–2·3), p=0·0083 | Implied SE 0.188 (at most 0.206) vs minimum 0.344. Narrowest possible CI 0.82–3.14, p = 0.17. Supplementary Table 4 gives 1.4 (0.50–4.2), p = 0.50 |
| aOR, county hospital | Abstract; Table 2 | 25·1 (15·7–40·2) | Implied SE 0.240 (at most 0.241) vs minimum 0.284. Narrowest possible CI 14.4–43.8. Still strongly significant |
| aOR, third quintile | Table 2 | 0·7 (0·6–0·9), p=0·0009 | Implied SE 0.103 (at most 0.139) vs minimum 0.293. Narrowest possible CI 0.39–1.24. Supplementary Table 4 gives 0.73 (0.22–2.4), p = 0.60 |
| aORs for age 30–44, accompanied, fourth and fifth quintile, NHIF | Abstract; Table 2 | e.g. 2·6 (1·3–5·3); 3·2 (1·7–5·9) | Consistent with the minimum (implied SE ≥ minimum) |
| Unadjusted secondary vs primary education (23/94 vs 42/163) | Table 1; Table 2 | 1·3 (1·2–1·5), p<0·0001 | Crude 0.93 (0.52–1.68), p = 0.82. Table 1 gives p = 0·90 for education |
| Unadjusted age 30–44 vs 18–29 (39/117 vs 14/103) | Table 1; Table 2 | 3·7 (2·9–4·6) | Crude 3.18 (1.61–6.29), SE 0.348 vs implied 0.118 |
| Unadjusted county vs health centre (38/63 vs 11/164) | Table 1; Table 2 | 24 (19·9–28·6) | Crude 21.1 (9.56–46.7), SE 0.405 vs implied 0.093 |
In the Abstract, 5 of the 7 adjusted associations have reported intervals narrower than any maximum-likelihood logistic fit to these data can produce. The violation is severe for the second quintile (11×) and subcounty hospital (52×). It is moderate for education above secondary school (1.8×). It is small for female sex (1.3×) and county hospital (1.2×). The bound is conservative: it assumes every fitted probability is 0.5, whereas the overall CHE rate is 26%. Realistic standard errors are larger, as the crude SEs and Supplementary Table 4 show.
The univariable column of Table 2 has the same pattern. Several of its intervals are much narrower than the crude counts give, and the secondary-education OR points the opposite way from the counts. That column appears unchanged in Supplementary Table 4, for example 3.7 (2.9–4.6), 1.3 (1.2–1.5) and 1.4 (0.76–2.4).
Figure
Figure 1. Odds ratios of catastrophic health expenditure with 95% CIs on a log scale. The thin vertical line marks OR = 1. Orange shows values the paper reports. For the second poverty quintile these are the main-model aOR 2.0 (1.9–2.1) from the Abstract, Results and Table 2 (diamond), and the same model with a five-level facility term from Supplementary Table 4 (square). For subcounty hospitals they are the Abstract and Results value 9.7 (9.6–9.8) and the Table 2 value 9.8 (9.7–9.9). Blue shows values recomputed from the paper's own numbers. The first blue row in each group is the narrowest interval any maximum-likelihood logistic fit could give at the reported point estimate. The second keeps the reported point estimate but uses the standard error of the crude log OR from the Table 1 counts. The third is the crude OR and Wald interval from those counts. The reported main-model intervals are hidden behind their markers. For the second quintile, the interval from the paper's own sensitivity model, the interval with the count-based SE (0.86–4.67) and the crude interval all include 1.
Figure 2. For each of the seven adjusted odds ratios in the Abstract (point estimate in brackets), the orange diamond is the standard error of the log OR implied by the reported 95% CI. Its bar is the range allowed by rounding the printed limits. The blue dot is the smallest standard error any maximum-likelihood logistic regression can give for that indicator, √(4/n₁ + 4/(296 − n₁)), with n₁ from Table 1. The shaded band to its right is the attainable range. Five of the seven reported intervals fall below this floor. The second-poverty-quintile and subcounty-hospital estimates are 11 and 52 times below it. The axis is logarithmic.
Why it matters
- The poverty finding. The second quintile is the only poverty quintile reported with significantly raised odds of CHE. The fourth and fifth quintile CIs include 1. The third quintile is reported as significantly lower, 0·7 (0·6–0·9), p=0·0009, but that interval is also narrower than the floor allows (narrowest possible CI 0.39–1.24), and Supplementary Table 4 gives 0.73 (0.22–2.4), p=0.60. With an attainable standard error, the second-quintile association is at most borderline: p = 0.017 at the theoretical minimum SE and p = 0.11 at the count-based SE. The paper's own five-facility sensitivity model gives 2.1 (0.70–6.1). The statements that CHE is "driven by socioeconomic … inequalities" (Abstract, Interpretation) and that poverty is an "independent driver" (Research in context) depend on this estimate. The Discussion already notes that the quintile relationship "did not always reach significance and appeared non-linear".
- Education and sex. At the minimum attainable SE, education above secondary school is no longer significant (narrowest CI 0.82–3.14). Supplementary Table 4 gives 1.4 (0.50–4.2), p = 0.50 for it, and 2.0 (0.95–4.0), p = 0.07 for female sex. The female aOR stays nominally significant at the bound (p = 0.013), but the bound is a floor, not an estimate.
- Facility level. The subcounty and county hospital associations keep their direction and remain highly significant with any plausible SE. Only their stated precision is wrong, by about 50-fold for subcounty hospitals.
- What stands. The CHE incidence (76/296, 26%), the cost breakdown, the NHIF coverage figures and the coping-strategy results do not depend on these intervals.
- Downstream use. A meta-analysis that takes 2.0 (1.9–2.1) or 9.7 (9.6–9.8) at face value would weight these estimates by inverse variance, that is, about 130 times (second quintile, 11.3²) and about 2,700 times (subcounty, 52.3²) more than even the theoretical floor on the standard error allows.
Explanations considered
- Rounding of the printed limits. Does not account for it. The widest interval consistent with "1·9–2·1" is 1.85–2.15, with SE 0.038, still 7.5 times below the floor. For "9·6–9·8" the factor is 35.
- A typo in one limit. Does not account for it. The second-quintile interval appears identically in the Abstract, Results and Table 2, and its p<0·0001 matches the interval (z ≈ 27). The subcounty value differs between Abstract (9·7, 9·6–9·8) and Table 2 (9·8, 9·7–9·9), but both are equally narrow. Four other adjusted intervals and several univariable intervals are also below the floor, so this is not one mistyped number.
- Covariate adjustment. Does not account for it. Adding covariates cannot push a model-based logistic-regression SE below √(4/n₁ + 4/(N − n₁)), as shown in the Method.
- Multiple imputation (21 participants without household asset data (Methods); 26 participants with missing income, expenditure or poverty data (Discussion)). Does not account for it. Rubin's total variance is at least the average within-imputation variance, and each of those obeys the bound. Even if imputation moved all 21 participants into or out of the second quintile, the bound would still be 0.261, 10 times the implied SE.
- Exact, penalised (Firth) or bootstrap intervals. None is mentioned in the paper, and for cells of this size (11 to 153 participants per cell in Table 1) such methods give intervals of similar width to the Wald interval, not 10–50 times narrower.
- A cluster-robust (sandwich) variance with facilities as clusters. This is the one standard estimator not bound by the inequality above. The paper does not mention it, and with five facilities, one of which forms the whole county-hospital level, such estimates are known to be unstable and too small. As a check, I computed the facility-clustered SE for the unadjusted three-level model from the facility counts in Supplementary Table 1b. These counts sum exactly to the Table 1 level totals. The result for county hospital (0.097, or 0.108 with the G/(G − 1) factor) is close to the 0.093 implied by Table 2. For subcounty hospital the result is 0.265–0.296, against 0.071 implied by Table 2. The Table 2 unadjusted ORs also differ from the crude ORs (9.7 vs 8.94; 24 vs 21.1). So this mechanism reproduces some of the reported intervals but not others. It cannot be confirmed without the data, and it would not make the reported intervals valid.
- A different sample or outcome in the model. Not supported. Table 2 states n=296 for both columns and the same 10% CHE outcome.
Status
- Version of record. The publisher's pages (thelancet.com, sciencedirect.com) and PDF were not reachable by automated access (HTTP 403/429). The open-access text deposited by the publisher was checked in the Europe PMC full-text XML for PMC12208783 (fetched 2026-10-23), the PMC article page and the local PMC archive copy. All carry 2·0 (1·9–2·1) and 9·7 (9·6–9·8) in the Abstract and Results, and 2·0 (1·9–2·1) and 9·8 (9·7–9·9) in Table 2. Every quotation above was checked verbatim against the archive copy. These are numbers in running text and table cells, not a rendering or conversion artefact.
- Supplements. Supplementary appendix 1 (mmc1.pdf, 2,538,411 bytes, md5 02ee6b399598cdef96ddbb2b8a1936e9, from the Europe PMC supplementary files) was read in full text. It contains Supplementary Table 4 (same 10% CHE outcome and covariates, with a five-level facility term), where the second quintile is 2.1 (0.70–6.1), p=0.099, and Supplementary Table 1b, which gives CHE by facility (5/85, 6/79, 12/38, 15/31, 38/63; these sum to the Table 1 level totals). It does not describe the variance estimator of the main model, and no individual-level data or code are provided. mmc2.pdf is a partnership declaration with no relevant data.
- Corrections. Crossref lists no update-to or updated-by relations, and PMC and Europe PMC list no correction, retraction or comment notice (checked 2026-10-23).
- PubPeer. pubpeer.com refused automated access (HTTP 403), and a web search could not be run in this session, so the absence of a PubPeer thread is not confirmed. Europe PMC's link services for PMID 40580995 list only a data citation and Altmetric, with no PubPeer link.
- As of 2026-10-23.
Suggested correction
- Re-estimate Table 2, both columns, with model-based maximum-likelihood standard errors, or state and justify the variance estimator used. With five facilities, a cluster-robust estimator is not appropriate without small-cluster corrections. Report the method in the Statistical analysis section.
- Replace the second-quintile, subcounty-hospital, female, above-secondary, county-hospital and third-quintile aOR intervals and p values in the Abstract, Results and Table 2. Also replace the univariable intervals that do not match the Table 1 counts.
- Reconcile the subcounty-hospital aOR between the Abstract and Results (9·7, 9·6–9·8) and Table 2 (9·8, 9·7–9·9).
- If the corrected second-quintile and above-secondary-education intervals include 1, revise the statements that poverty is an independent driver of CHE (Abstract Interpretation, Research in context, Discussion) and the abstract's list of associations accordingly.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC12208783 write-up and regenerate fig1/fig2.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
All inputs are copied from the article (Table 1, Table 2, Abstract) or from its
Supplementary appendix 1 (mmc1.pdf: Supplementary Tables 1b and 4).
"""
import os
import numpy as np
from scipy import stats
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
N, CHE = 296, 76
def crude(a, b, c, d):
"""a = CHE exposed, b = no CHE exposed, c = CHE reference, d = no CHE reference."""
OR = (a / b) / (c / d)
se = np.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
lo, hi = np.exp(np.log(OR) - Z * se), np.exp(np.log(OR) + Z * se)
p = 2 * stats.norm.sf(abs(np.log(OR)) / se)
return OR, lo, hi, se, p
def se_from_ci(lo, hi):
return np.log(hi / lo) / (2 * Z)
def se_rounding_range(lo, hi, h=0.05):
"""(narrowest, widest) SE consistent with limits printed to one decimal place."""
return se_from_ci(lo + h, hi - h), se_from_ci(lo - h, hi + h)
def se_min(n1, n=N):
"""Hard lower bound on the model-based (ML) SE of a logistic coefficient for a
0/1 indicator held by n1 of n participants, whatever the other covariates:
Var(b_j) = [I^-1]_jj >= [(I restricted to intercept and x_j)^-1]_jj = 1/S1 + 1/S0,
with S1 = sum over x_j = 1 of p(1-p) <= n1/4 and S0 <= (n - n1)/4."""
return np.sqrt(4 / n1 + 4 / (n - n1))
def ci(OR, se):
return OR * np.exp(-Z * se), OR * np.exp(Z * se)
def p_of(OR, se):
return 2 * stats.norm.sf(abs(np.log(OR)) / se)
print("Totals from Table 1: CHE", CHE, "of", N, f"({CHE / N:.1%})")
# ---------------------------------------------------------------------------
print("\n=== 1. Crude ORs from Table 1 counts vs Table 2 'unadjusted' column ===")
rows = [
("Age 30-44 vs 18-29", 39, 78, 14, 89, "3.7 (2.9-4.6), p=0.0010"),
("Female vs male", 42, 131, 34, 89, "0.7 (0.4-2.0), p=0.71"),
("Secondary vs primary", 23, 71, 42, 121, "1.3 (1.2-1.5), p<0.0001"),
("Above secondary vs primary", 11, 28, 42, 121, "1.7 (1.0-2.8), p=0.053"),
("Quintile 2 vs 1", 15, 45, 13, 47, "1.4 (0.8-2.4), p=0.30"),
("Quintile 3 vs 1", 10, 48, 13, 47, "1.0 (0.8-1.3), p=0.91"),
("Quintile 5 vs 1", 24, 35, 13, 47, "2.2 (1.1-4.7), p=0.035"),
("Subcounty vs health centre", 27, 42, 11, 153, "9.7 (8.4-11.1), p<0.0001"),
("County vs health centre", 38, 25, 11, 153, "24 (19.9-28.6), p<0.0001"),
]
CR = {}
for lab, a, b, c, d, rep in rows:
OR, lo, hi, se, p = crude(a, b, c, d)
CR[lab] = (OR, lo, hi, se, p)
print(f"{lab:28s} crude {OR:5.2f} ({lo:5.2f}-{hi:5.2f}) SE {se:.3f} p={p:.2g} | Table 2: {rep}")
print("Fisher exact p, quintile 2 vs 1:", f"{stats.fisher_exact([[15, 45], [13, 47]])[1]:.2f}")
print("Fisher exact p, secondary vs primary:", f"{stats.fisher_exact([[23, 71], [42, 121]])[1]:.2f}")
# ---------------------------------------------------------------------------
print("\n=== 2. Reported adjusted ORs vs the hard lower bound on any ML standard error ===")
# label, n1 (participants in the category, Table 1), aOR, lo, hi, where, in_abstract
adj = [
("Age 30-44", 117, 2.6, 1.3, 5.3, True),
("Female", 173, 1.8, 1.3, 2.7, True),
("Above secondary", 39, 1.6, 1.1, 2.3, True),
("Accompanied", 59, 3.2, 1.7, 5.9, True),
("Second quintile", 60, 2.0, 1.9, 2.1, True),
("Subcounty (abstract/text)", 69, 9.7, 9.6, 9.8, True),
("County", 63, 25.1, 15.7, 40.2, True),
("Subcounty (Table 2)", 69, 9.8, 9.7, 9.9, False),
("Third quintile", 58, 0.7, 0.6, 0.9, False),
("Fourth quintile", 59, 0.5, 0.2, 1.2, False),
("Fifth quintile", 59, 2.1, 0.6, 7.7, False),
("NHIF no", 212, 1.5, 0.7, 3.5, False),
]
print(f"{'term':26s} {'reported':>18s} {'SE_rep':>7s} {'SE range (rounding)':>20s} {'SE_min':>7s} {'SE_min/SE_rep':>13s} min CI at reported aOR verdict")
FIG2 = []
for lab, n1, OR, lo, hi, abst in adj:
se = se_from_ci(lo, hi)
s_lo, s_hi = se_rounding_range(lo, hi)
smin = se_min(n1)
verdict = "BELOW BOUND" if s_hi < smin else "consistent"
clo, chi = ci(OR, smin)
print(f"{lab:26s} {OR:5.1f} ({lo:4.1f}-{hi:4.1f}) {se:7.4f} {s_lo:9.4f}-{s_hi:.4f} {smin:7.3f} {smin / se:13.1f} {clo:6.2f}-{chi:6.2f} {verdict}")
if abst:
FIG2.append((lab, OR, lo, hi, se, s_lo, s_hi, smin))
n_abs_fail = sum(1 for r in FIG2 if r[6] < r[7])
print(f"Abstract aORs whose reported CI is narrower than any ML fit allows: {n_abs_fail} of {len(FIG2)}")
for _lab, _n1, _lo, _hi in [("Second quintile", 60, 1.9, 2.1), ("Subcounty (abstract/text)", 69, 9.6, 9.8)]:
_r = se_min(_n1) / se_from_ci(_lo, _hi)
print(f"Inverse-variance weight inflation vs the floor, {_lab}: ({_r:.1f})^2 = {_r ** 2:.0f}x")
print("p-value if each abstract aOR kept its point estimate but had the minimum possible SE:")
for lab, OR, lo, hi, se, s_lo, s_hi, smin in FIG2:
print(f" {lab:26s} reported-CI p {p_of(OR, se):.2g} p at minimum SE {p_of(OR, smin):.2g}")
# ---------------------------------------------------------------------------
print("\n=== 3. Second quintile: the reported aOR 2.0 (1.9-2.1) in context ===")
se_rep = se_from_ci(1.9, 2.1)
s_lo, s_hi = se_rounding_range(1.9, 2.1)
print(f"implied SE {se_rep:.4f} (rounding range {s_lo:.4f}-{s_hi:.4f}); implied z = {np.log(2.0) / se_rep:.1f}")
crude_q2 = CR["Quintile 2 vs 1"]
print(f"crude SE {crude_q2[3]:.3f}; ratio crude/reported {crude_q2[3] / se_rep:.0f} (rounding range {crude_q2[3] / s_hi:.0f}-{crude_q2[3] / s_lo:.0f})")
print(f"bound/reported {se_min(60) / se_rep:.1f}x; bound/widest rounding-consistent SE {se_min(60) / s_hi:.1f}x")
# Poverty quintile was imputed for 21 participants: allow n1 anywhere in 60 +/- 21
bmin = min(se_min(n1) for n1 in range(60 - 21, 60 + 22))
print(f"lower bound if n1 for Q2 is anywhere in 39..81 (imputation): {bmin:.3f} -> {bmin / se_rep:.0f}x the reported SE, {bmin / s_hi:.1f}x the widest rounding-consistent SE")
for name, s in [("hard minimum SE", se_min(60)), ("count-based crude SE", crude_q2[3])]:
lo2, hi2 = ci(2.0, s)
print(f"aOR 2.0 with {name} {s:.3f}: 95% CI {lo2:.2f}-{hi2:.2f}, p={p_of(2.0, s):.3f}")
se_s4 = se_from_ci(0.70, 6.1)
print(f"Supplementary Table 4 (same model, five-facility term): 2.1 (0.70-6.1), p=0.099; implied SE {se_s4:.3f} = {se_s4 / se_rep:.0f}x the Table 2 SE")
print(f" Wald p recomputed from that CI: {p_of(2.1, se_s4):.2f}")
se_q3 = se_from_ci(0.6, 0.9)
print(f"Third quintile: Table 2 aOR 0.7 (0.6-0.9), p=0.0009, implied SE {se_q3:.3f} vs bound {se_min(58):.3f}; "
f"narrowest CI at 0.7: {ci(0.7, se_min(58))[0]:.2f}-{ci(0.7, se_min(58))[1]:.2f}; "
f"Supplementary Table 4: 0.73 (0.22-2.4), p=0.60")
# ---------------------------------------------------------------------------
print("\n=== 4. Subcounty hospital: 9.7 (9.6-9.8) in the abstract and text, 9.8 (9.7-9.9) in Table 2 ===")
for lab, OR, lo, hi in [("abstract/text", 9.7, 9.6, 9.8), ("Table 2", 9.8, 9.7, 9.9)]:
se = se_from_ci(lo, hi)
s_lo, s_hi = se_rounding_range(lo, hi)
print(f"{lab:14s} implied SE {se:.4f} (rounding range {s_lo:.4f}-{s_hi:.4f}); bound {se_min(69):.3f} = {se_min(69) / se:.0f}x; bound/widest rounding-consistent SE {se_min(69) / s_hi:.0f}x")
cs = CR["Subcounty vs health centre"]
print(f"crude SE {cs[3]:.3f} = {cs[3] / se_from_ci(9.6, 9.8):.0f}x the abstract SE; aOR 9.7 with crude SE: CI {ci(9.7, cs[3])[0]:.1f}-{ci(9.7, cs[3])[1]:.1f}")
print(f"aOR 9.7 with hard minimum SE: CI {ci(9.7, se_min(69))[0]:.2f}-{ci(9.7, se_min(69))[1]:.2f}")
cc = CR["County vs health centre"]
print(f"County: crude SE {cc[3]:.3f}; reported adjusted SE {se_from_ci(15.7, 40.2):.3f}; bound {se_min(63):.3f}")
# ---------------------------------------------------------------------------
print("\n=== 5. Could a facility-clustered (sandwich) SE explain the narrow intervals? ===")
# Supplementary Table 1b ('Incurred catastrophic health expenditure' row): CHE / n by facility; level: 0 = health centre, 1 = subcounty, 2 = county
fac = [("Mitunguu HC", 0, 5, 85), ("Laare HC", 0, 6, 79), ("Mutuati SCH", 1, 12, 38),
("Kanyakine SCH", 1, 15, 31), ("Meru TRH", 2, 38, 63)]
lev_y = {L: sum(y for _, l, y, _ in fac if l == L) for L in range(3)}
lev_n = {L: sum(n for _, l, _, n in fac if l == L) for L in range(3)}
print("level CHE/n from facilities:", [f"{lev_y[L]}/{lev_n[L]}" for L in range(3)], "(Table 1: 11/164, 27/69, 38/63)")
# Univariable model with level dummies is saturated at the level, so fitted p = level proportion.
# Scores per facility for the level log-odds theta_L: u_g = y_g - n_g p_L. A_L = n_L p_L (1 - p_L).
G = len(fac)
var = {}
for L in range(3):
pL = lev_y[L] / lev_n[L]
A = lev_n[L] * pL * (1 - pL)
B = sum((y - n * pL) ** 2 for _, l, y, n in fac if l == L)
var[L] = B / A ** 2
for lab, k in [("without small-sample factor", 1.0), ("with G/(G-1)", G / (G - 1))]:
se_sub = np.sqrt(k * (var[0] + var[1]))
se_cty = np.sqrt(k * (var[0] + var[2]))
print(f"cluster-robust SE {lab}: subcounty {se_sub:.3f}, county {se_cty:.3f}")
print(f"Table 2 unadjusted implied SEs: subcounty {se_from_ci(8.4, 11.1):.3f}, county {se_from_ci(19.9, 28.6):.3f}")
# ---------------------------------------------------------------------------
# Figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
BG, INK, INK2, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def style(ax):
ax.set_facecolor(BG)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.grid(axis="x", color=GRID, lw=0.8, which="both")
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
# ---- Figure 1: second quintile and subcounty hospital
q2min = ci(2.0, se_min(60))
scmin = ci(9.7, se_min(69))
items = [
# group, label, OR, lo, hi, kind (rep / rec), marker
("Second vs first poverty quintile", "Reported aOR (Abstract, Results, Table 2)", 2.0, 1.9, 2.1, "rep", "D"),
("Second vs first poverty quintile", "Reported aOR, Supplementary Table 4 (five-facility model)", 2.1, 0.70, 6.1, "rep", "s"),
("Second vs first poverty quintile", "Narrowest CI any logistic ML fit allows at aOR 2.0", 2.0, *q2min, "rec", "o"),
("Second vs first poverty quintile", "aOR 2.0 with the count-based SE from Table 1", 2.0, *ci(2.0, crude_q2[3]), "rec", "o"),
("Second vs first poverty quintile", "Crude OR from Table 1 counts (15/60 vs 13/60)", crude_q2[0], crude_q2[1], crude_q2[2], "rec", "o"),
("Subcounty hospital vs health centre", "Reported aOR (Abstract, Results)", 9.7, 9.6, 9.8, "rep", "D"),
("Subcounty hospital vs health centre", "Reported aOR (Table 2)", 9.8, 9.7, 9.9, "rep", "D"),
("Subcounty hospital vs health centre", "Narrowest CI any logistic ML fit allows at aOR 9.7", 9.7, *scmin, "rec", "o"),
("Subcounty hospital vs health centre", "aOR 9.7 with the count-based SE from Table 1", 9.7, *ci(9.7, cs[3]), "rec", "o"),
("Subcounty hospital vs health centre", "Crude OR from Table 1 counts (27/69 vs 11/164)", cs[0], cs[1], cs[2], "rec", "o"),
]
fig, ax = plt.subplots(figsize=(7.2, 7.0), facecolor=BG)
style(ax)
STEP, GAP = 1.25, 1.1
ypos, y, last = [], 0.0, None
for g, *_ in items:
if g != last and last is not None:
y += GAP
ypos.append(y)
y += STEP
last = g
ypos = -np.array(ypos)
reptxt = {0: "2.0 (1.9–2.1)", 1: "2.1 (0.70–6.1)", 5: "9.7 (9.6–9.8)", 6: "9.8 (9.7–9.9)"}
for i, ((g, lab, OR, lo, hi, kind, mk), yy) in enumerate(zip(items, ypos)):
col = ORANGE if kind == "rep" else BLUE
ax.plot([lo, hi], [yy, yy], color=col, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([lo, lo], [yy - 0.13, yy + 0.13], color=col, lw=2)
ax.plot([hi, hi], [yy - 0.13, yy + 0.13], color=col, lw=2)
ax.plot(OR, yy, marker=mk, ms=8, color=col, mec=BG, mew=1.2, zorder=3)
txt = reptxt[i] if kind == "rep" else f"{OR:.2f} ({lo:.2f}–{hi:.2f})"
ax.text(1.02, yy, txt, transform=ax.get_yaxis_transform(), va="center", ha="left", fontsize=10, color=INK)
ax.text(0.34, yy + 0.2, lab, va="bottom", ha="left", fontsize=10, color=INK2)
# OR = 1 reference: short segment through each data row only, so it never crosses a label
ax.plot([1, 1], [yy - 0.3, yy + 0.18], color=INK2, lw=0.8, zorder=1)
ax.text(0.34, ypos[0] + 0.85, "Second vs first poverty quintile", fontsize=11, weight="bold", va="bottom")
ax.text(0.34, ypos[5] + 0.85, "Subcounty hospital vs health centre", fontsize=11, weight="bold", va="bottom")
ax.plot([1, 1], [ypos[-1] - 0.3, ypos[-1] - 0.6], color=INK2, lw=0.8, zorder=1)
ax.text(1.04, ypos[-1] - 0.75, "OR = 1 (no association)", ha="left", va="center", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(0.33, 30)
ax.set_xticks([0.5, 1, 2, 5, 10, 20])
ax.set_xticklabels(["0.5", "1", "2", "5", "10", "20"])
ax.set_ylim(ypos[-1] - 1.2, ypos[0] + 1.5)
ax.set_yticks([])
ax.set_xlabel("Odds ratio of catastrophic health expenditure, 95% CI (log scale)")
from matplotlib.lines import Line2D
leg = [Line2D([0], [0], color=ORANGE, lw=2, marker="D", ms=8, mec=BG, label="Reported in the paper"),
Line2D([0], [0], color=BLUE, lw=2, marker="o", ms=8, mec=BG, label="Recomputed from the paper's own numbers")]
ax.legend(handles=leg, loc="upper left", bbox_to_anchor=(0.0, -0.11), ncol=2, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.03, right=0.77, top=0.98, bottom=0.17)
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# ---- Figure 2: standard errors for the seven aORs in the abstract
order = ["Age 30-44", "Accompanied", "County", "Female", "Above secondary", "Second quintile", "Subcounty (abstract/text)"]
names = {"Age 30-44": "Aged 30–44 years (2.6)", "Accompanied": "Accompanied (3.2)", "County": "County hospital (25.1)",
"Female": "Female (1.8)", "Above secondary": "Above secondary education (1.6)",
"Second quintile": "Second poverty quintile (2.0)", "Subcounty (abstract/text)": "Subcounty hospital (9.7)"}
d = {r[0]: r for r in FIG2}
fig, ax = plt.subplots(figsize=(7.2, 4.3), facecolor=BG)
style(ax)
for i, k in enumerate(order):
lab, OR, lo, hi, se, s_lo, s_hi, smin = d[k]
yy = -i
ax.plot([s_lo, s_hi], [yy, yy], color=ORANGE, lw=2)
ax.plot(se, yy, marker="D", ms=8, color=ORANGE, mec=BG, mew=1.2, zorder=3)
ax.plot(smin, yy, marker="o", ms=8, color=BLUE, mec=BG, mew=1.2, zorder=3)
ax.plot([smin, 1.2], [yy, yy], color=BLUE, lw=6, alpha=0.18, solid_capstyle="butt", zorder=1)
status = "too small" if s_hi < smin else "possible"
ax.text(1.03, yy, f"{se:.3f} vs {smin:.3f} {status}", transform=ax.get_yaxis_transform(),
va="center", fontsize=10, color=INK)
ax.set_yticks(-np.arange(len(order)))
ax.set_yticklabels([names[k] for k in order], fontsize=10, color=INK)
ax.set_xscale("log")
ax.set_xlim(0.002, 1.2)
ax.set_xticks([0.003, 0.01, 0.03, 0.1, 0.3, 1])
ax.set_xticklabels(["0.003", "0.01", "0.03", "0.1", "0.3", "1"])
ax.text(1.03, 0.55, "reported vs minimum", transform=ax.get_yaxis_transform(), va="bottom", fontsize=10, color=INK2)
ax.set_ylim(-len(order) + 0.4, 0.9)
ax.set_xlabel("Standard error of log odds ratio (log scale)")
leg = [Line2D([0], [0], color=ORANGE, lw=2, marker="D", ms=8, mec=BG,
label="Implied by reported 95% CI (bar: range allowed by rounding)"),
Line2D([0], [0], color=BLUE, lw=0, marker="o", ms=8, mec=BG,
label="Smallest SE any logistic ML fit can give (shaded: attainable)")]
ax.legend(handles=leg, loc="upper left", bbox_to_anchor=(-0.55, -0.17), ncol=1, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.35, right=0.72, top=0.93, bottom=0.3)
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), facecolor=BG, **kw)
plt.close(fig)
print("\nFigures written to", HERE)
Abstract swaps the DALYs-averted estimates of the two dynamic choice HIV prevention policies
Lancet HIV (2025) · doi:10.1016/S2352-3018(25)00169-9 · PMC12854106
Summary
The abstract's Findings give 18 400 DALYs averted per year (95% CI 16 700–20 100) for dynamic choice HIV prevention (DCP) including long-acting cabotegravir and 56 400 (52 300–60 500) for DCP without cabotegravir. Table 5 prints the same two estimates and intervals with the labels reversed: 18 400 for DCP without cabotegravir and 56 400 for DCP including it. Every other number in the paper agrees with Table 5. Each policy's cost difference divided by its Table 5 DALYs gives the reported ICERs ($13.2M / 56 400 = $234; $8.6M / 18 400 = $467), and the reported net DALYs and the extended-dominance result also fit Table 5. With the abstract's labels, the same costs give $717 and $152, and DCP including cabotegravir would be dominated. As printed, the abstract reverses which policy delivers more health benefit, although its ICER and conclusion are right.
What the paper reports
Abstract, Findings:
Across setting-scenarios, both DCP policies led to DALYs being averted: 18 400 DALYs per year (95% CI 16 700–20 100) for DCP including cabotegravir and 56 400 DALYs per year (52 300–60 500) for DCP without cabotegravir in 10 million adults. Compared with no DCP, there was a mean increase in annual discounted costs over 50 years: $8·6 million (7·7–9·4) for DCP without cabotegravir and $13·2 million (11·6–14·8) for DCP including cabotegravir. Addition of long-acting cabotegravir PrEP to DCP was cost-effective (vs DCP without cabotegravir); the incremental cost-effectiveness ratio for DCP including cabotegravir (vs no DCP) was $234 per DALY averted.
Abstract, Findings (the incidence result, in the same paragraph):
These increases in PrEP use led to HIV incidence reductions, with incidence rate ratios of 0·89 (0·67–1·17) for DCP without cabotegravir and 0·64 (0·44–0·97) for DCP including cabotegravir, relative to no DCP.
Table 5, "HIV-related deaths, DALYs, and costs over 50 years and cost-effectiveness analysis compared with no DCP". Columns: No DCP | DCP without cabotegravir | DCP including cabotegravir. "Data are mean (95% CI); 90% range":
Difference in number of HIV-related deaths per year | ·· | -1300 (-1400 to -1200); -4500 to 800 | -3700 (-4000 to -3400); -12 300 to 200
Difference in DALYs per year over 50 years | ·· | -18 400 (-20 100 to -16 700); -68 600 to 19 000 | -56 400 (-60 500 to -52 300); -178 800 to 8200
Difference in annual cost over 50 years | ·· | $8·6 million (7·7 to 9·4); -6·1 to 32·3 | $13·2 million (11·6 to 14·8); -16·9 to 61·0
Difference in net DALYs* per year over 50 years | ·· | -1200 (-3600 to 1200); -65 900 to 55 800 | -29 900 (-35 400 to -24 400); -179 700 to 81 700
Incremental cost-effectiveness ratio (cost per DALY averted) | ·· | Dominated | $234
Incremental cost-effectiveness ratio (cost per DALY averted) for DCP compared with no DCP (if long-acting cabotegravir PrEP not available) | ·· | $467 | NA
Table 5 footnote: "*Given a cost-effectiveness threshold of $500 per DALY averted."
Results, paragraph 3:
Overall, as a mean across setting-scenarios, both DCP policies lead to DALYs being averted (table 5). There was, however, variability over setting-scenarios (appendix p 3); the policy with the least DALYs incurred was the status quo (no DCP or long-acting cabotegravir) in 6% of setting-scenarios, DCP without cabotegravir in 11% of setting-scenarios, and DCP including cabotegravir in 83% of setting-scenarios.
Results, paragraph 5:
The policy of DCP without cabotegravir was not a candidate for the cost-effective choice of intervention, as a combination of the policies of status quo and DCP including cabotegravir (represented by a point half way along the line between the two policies) was more effective and less costly than this policy (ie, the policy was extendedly dominated). The incremental cost-effectiveness ratio (ICER) for the policy of DCP including cabotegravir was $234 per DALY averted (figure). This policy incurred the lowest overall mean net DALYs at a cost-effectiveness threshold of $500 per DALY averted (table 5). If long-acting cabotegravir was not available, the ICER for DCP was $467.
Methods, "Cost-effectiveness analysis":
Net DALYs averted was calculated by DALYs averted plus the difference in costs divided by the cost-effectiveness threshold.
What the paper's own numbers show
Method. The abstract and Table 5 report the same quantity: mean DALYs per year over 50 years, discounted at 3%, for 10 million adults, compared with no DCP. Table 5 signs it as a difference, so a negative value means DALYs averted. I first compared the two pairs of estimates and confidence intervals. I then tested each possible assignment of the two DALY estimates to the two policies against three relationships the paper defines or reports. (1) ICER vs no DCP = difference in annual cost / DALYs averted, compared with the reported $234 and $467. (2) Difference in net DALYs = −DALYs averted + difference in cost / $500, compared with the Table 5 net-DALY row. (3) Extended dominance, which the Results state for DCP without cabotegravir. Finally I compared the ordering with the paper's other effectiveness outcomes: HIV deaths averted and incidence rate ratios. All inputs are the rounded values printed in the paper. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| DALYs averted/yr, DCP without cabotegravir | Table 5 | 18 400 (16 700–20 100) | consistent with every check below |
| DALYs averted/yr, DCP including cabotegravir | Table 5 | 56 400 (52 300–60 500) | consistent with every check below |
| DALYs averted/yr, DCP including cabotegravir | Abstract, Findings | 18 400 (16 700–20 100) | equals the Table 5 value for DCP without cabotegravir, CI included |
| DALYs averted/yr, DCP without cabotegravir | Abstract, Findings | 56 400 (52 300–60 500) | equals the Table 5 value for DCP including cabotegravir, CI included |
| ICER vs no DCP, DCP including cabotegravir | Abstract; Results p5; Table 5 | $234 | $13.2M / 56 400 = $234.04 (Table 5 labels); $13.2M / 18 400 = $717.39 (abstract labels) |
| ICER vs no DCP, DCP without cabotegravir | Results p5; Table 5 | $467 | $8.6M / 18 400 = $467.39 (Table 5 labels); $8.6M / 56 400 = $152.48 (abstract labels) |
| Net DALY difference, DCP without cabotegravir | Table 5 | −1200 | −18 400 + 17 200 = −1200 (Table 5 labels); −56 400 + 17 200 = −39 200 (abstract labels) |
| Net DALY difference, DCP including cabotegravir | Table 5 | −29 900 | −56 400 + 26 400 = −30 000 (Table 5 labels; a cost of $13.15M–$13.25M, which rounds to $13.2M, gives −30 100 to −29 900); −18 400 + 26 400 = +8000 (abstract labels) |
| Extended dominance of DCP without cabotegravir | Results p5; Table 5 "Dominated" | stated | Table 5 labels: $467 > $234, so it is extendedly dominated; the midpoint of no DCP and DCP including cabotegravir averts 28 200 DALYs at $6.6M, against 18 400 at $8.6M. Abstract labels: DCP including cabotegravir would avert fewer DALYs (18 400 < 56 400) at a higher cost, so it would be strongly dominated, which contradicts the paper |
| Incremental ICER, including vs without cabotegravir | Abstract ("cost-effective (vs DCP without cabotegravir)") | not printed | Table 5 labels: $4.6M / 38 000 = $121.05 per DALY, below the $500 threshold, consistent with the statement |
| HIV-related deaths averted per year | Table 5 | 1300 (without); 3700 (including) | ratio including/without 2.85; DALY ratio 3.07 with the Table 5 labels, 0.33 with the abstract labels |
| Incidence reduction vs no DCP | Abstract | IRR 0·89 (without); 0·64 (including) | 11% vs 36% reduction, so the policy including cabotegravir is the more effective one |
Figure
Figure 1. Left: the DALYs-averted sentence in the abstract's Findings, with its policy labels (orange) as printed. Right: the same estimates as printed in Table 5, where a negative difference means DALYs averted. The arrows join each abstract value to the identical value and interval in Table 5, which belongs to the other policy. Below, in blue: each policy's cost difference from Table 5 divided by its DALYs averted under each labelling. Only the Table 5 labelling reproduces the reported ICERs of $234 and $467. All values are from the paper or are printed by recompute.py.txt.
Figure 2. The ICER of each DCP policy against no DCP. Orange rings: the reported ICERs. Filled blue dots: cost difference / DALYs averted with the Table 5 labels. Open blue dots: the same calculation with the abstract's labels. The filled dots sit on the reported values. The open dots fall at $717, above the $500 threshold, for DCP including cabotegravir and at $152 for DCP without cabotegravir, which would reverse the paper's cost-effectiveness ranking. Dashed line: the $500 per DALY threshold used in the paper.
Why it matters
The sentence is the abstract's only statement of health benefit in DALYs. As printed, it says the policy without cabotegravir averts about three times as many DALYs as the policy including cabotegravir (56 400 vs 18 400). Table 5, the Results and the paper's conclusions all say the reverse. A reader who has only the abstract, or who takes the per-policy health gains from it for a review, a meta-analysis or a budget-impact estimate, would get the two policies' benefits the wrong way round. The abstract is then also inconsistent with itself. Its own costs ($8.6M and $13.2M) and its reported ICER of $234 cannot both hold with its DALY figures: with those figures, DCP including cabotegravir costs more and averts fewer DALYs than DCP without it, so it would be dominated rather than cost-effective.
The error does not affect the model results or the paper's conclusions. Table 5, the ICERs of $234 and $467, the net-DALY results, the extended-dominance finding and the Interpretation all use the correct assignment. The error is confined to the policy labels in one sentence of the abstract.
Explanations considered
- Different quantity, time horizon, population or comparator. Ruled out. The abstract and Table 5 both describe mean DALYs per year over 50 years for 10 million adults, compared with no DCP, and both print the same 95% CIs (16 700–20 100 and 52 300–60 500). The abstract states that "both DCP policies led to DALYs being averted" and gives the costs against no DCP, so it is not reporting an incremental comparison between the two DCP policies. Only the policy labels differ.
- Net DALYs rather than DALYs. Ruled out. The net-DALY differences in Table 5 are −1200 and −29 900, not 18 400 or 56 400.
- Table 5 is the version with the error, and the abstract is right. Ruled out by the paper's other results, all of which fit Table 5. These are both ICERs, both net-DALY values, the extended dominance of DCP without cabotegravir, the larger number of HIV deaths averted with cabotegravir (3700 vs 1300), the lower incidence rate ratio with cabotegravir (0·64 vs 0·89), and the statement that DCP including cabotegravir incurred the fewest DALYs in 83% of setting-scenarios. With the abstract's labels, none of the reported ICERs or net DALYs can be reproduced.
- Text conversion or column misalignment in the archived copy. Ruled out. The publisher PDF (Elsevier, "Published online September 11, 2025", obtained from an institutional repository deposit marked as the published version) has the same abstract sentence and the same Table 5 column order. The PubMed abstract (PMID 40946710) and the PMC/Europe PMC author manuscript text match as well. The values are whole numbers, so no decimal-point or minus-sign misreading is possible.
- The supplement gives a different picture. No. The appendix (126 pages) prints no aggregate figure of 18 400 or 56 400. Its plots of discounted annual DALYs and of DALYs averted against incremental cost show, qualitatively, more DALYs averted with DCP including cabotegravir. That agrees with Table 5, not with the abstract.
- Rounding. This accounts only for the 100-DALY gap between the recomputed (−30 000) and reported (−29 900) net DALYs for DCP including cabotegravir. It cannot account for a factor of three between the two labellings.
Status
As of 2026-10-23:
- Version of record: checked. The publisher PDF has the swapped labels in the abstract, and Table 5 is as described above. The PubMed abstract (PMID 40946710), the PMC archived text (PMC12854106, author manuscript NIHMS2140119) and Europe PMC are the same. The journal website (thelancet.com, sciencedirect.com) returned 403 or a bot check, so I could not view the online HTML version there.
- Supplement: the appendix (NIHMS2140119-supplement-Appendix.pdf) was checked as described above. It agrees with Table 5.
- Corrections: Crossref and PubMed list a publisher erratum, "Correction to Lancet HIV 2025; 12: e701–11" (Lancet HIV 2026;13(5):e296, doi:10.1016/S2352-3018(26)00080-9, PMID 42067279, May 2026). I could not get its text: the publisher page returned 403, and PubMed and Europe PMC list no abstract. I therefore cannot say whether this erratum corrects the swap described here. The PMC archived version still shows the swap. No PMC or Europe PMC correction notice was linked other than this erratum, and no retraction or expression of concern was found.
- PubPeer: searched for the DOI. The site returned a bot check to automated requests, so I could not confirm whether any comment exists.
- Sources: PMC12854106 (archived full text and Table 5); the publisher PDF from the repository deposit (eprints.whiterose.ac.uk/id/eprint/231798); PubMed records 40946710 and 42067279; Crossref metadata for the DOI; Europe PMC supplementary files.
Suggested correction
If the May 2026 erratum does not already address it, the abstract's Findings sentence should read:
Across setting-scenarios, both DCP policies led to DALYs being averted: 18 400 DALYs per year (95% CI 16 700–20 100) for DCP without cabotegravir and 56 400 DALYs per year (52 300–60 500) for DCP including cabotegravir in 10 million adults.
The same change should be made wherever the abstract is reproduced (PubMed, PMC, Europe PMC). No change to Table 5, the Results, the ICERs or the conclusions is needed.
Recomputation script (python recompute.py.txt)
"""Recompute the consistency checks for PMC12854106
(Lancet HIV 2025, doi:10.1016/S2352-3018(25)00169-9).
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.
All inputs are copied from the paper as archived in PMC (same values in the
publisher PDF):
Table 5 (columns: No DCP | DCP without cabotegravir | DCP including cabotegravir),
rows "Difference in DALYs per year over 50 years", "Difference in annual cost
over 50 years", "Difference in net DALYs* per year over 50 years",
"Difference in number of HIV-related deaths per year", and the two ICER rows.
Abstract, Findings: the DALYs-averted sentence, costs, ICER and incidence rate ratios.
Methods: threshold of US$500 per DALY averted.
All figures are means per year over 50 years, discounted at 3%, for 10 million adults,
compared with no DCP.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
THRESHOLD = 500.0 # Methods, US$ per DALY averted
POL = ("without", "including") # DCP without / including cabotegravir
NAME = {"without": "DCP without cabotegravir", "including": "DCP including cabotegravir"}
# Table 5: difference in DALYs per year vs no DCP (negative = averted), mean (95% CI)
T5_DALY = {"without": (-18400, -20100, -16700), "including": (-56400, -60500, -52300)}
# Abstract, Findings: DALYs averted per year, as labelled in the abstract
ABS_DALY = {"including": (18400, 16700, 20100), "without": (56400, 52300, 60500)}
# Table 5 and abstract: difference in annual discounted cost vs no DCP, US$
COST = {"without": 8.6e6, "including": 13.2e6}
# Table 5: difference in net DALYs per year (threshold $500)
T5_NET = {"without": -1200, "including": -29900}
# Table 5: difference in HIV-related deaths per year
T5_DEATHS = {"without": -1300, "including": -3700}
# Abstract: incidence rate ratios vs no DCP
IRR = {"without": 0.89, "including": 0.64}
# Reported ICERs vs no DCP: $234 (abstract, Results p5, Table 5) and $467
# (Results p5, Table 5 "if long-acting cabotegravir PrEP not available")
ICER_REPORTED = {"without": 467, "including": 234}
# ---------------------------------------------------------------- derived
averted_t5 = {p: -T5_DALY[p][0] for p in POL} # Table 5 attribution
averted_abs = {p: ABS_DALY[p][0] for p in POL} # abstract attribution
def line(s=""):
print(s)
line("1. Same estimates, same CIs, opposite labels")
for p in POL:
m, lo, hi = T5_DALY[p]
line(f" Table 5 {NAME[p]:28s}: averted {-m} (95% CI {-hi} to {-lo})")
for p in POL:
m, lo, hi = ABS_DALY[p]
line(f" Abstract {NAME[p]:28s}: averted {m} (95% CI {lo} to {hi})")
same_set = sorted(-T5_DALY[p][0] for p in POL) == sorted(ABS_DALY[p][0] for p in POL)
swapped = all(ABS_DALY[p][0] == -T5_DALY[q][0] for p, q in (("including", "without"), ("without", "including")))
line(f" same pair of values: {same_set}; abstract value for each policy = Table 5 value of the other: {swapped}")
line()
line("2. ICER vs no DCP = cost difference / DALYs averted")
icer_t5 = {p: COST[p] / averted_t5[p] for p in POL}
icer_abs = {p: COST[p] / averted_abs[p] for p in POL}
for p in POL:
line(f" {NAME[p]:28s}: reported ${ICER_REPORTED[p]}; "
f"Table 5 attribution ${COST[p]/1e6:.1f}M/{averted_t5[p]} = ${icer_t5[p]:.2f}; "
f"abstract attribution ${COST[p]/1e6:.1f}M/{averted_abs[p]} = ${icer_abs[p]:.2f}")
line()
line("3. Net DALY difference = -(DALYs averted) + cost difference / threshold")
net_t5 = {p: -averted_t5[p] + COST[p] / THRESHOLD for p in POL}
net_abs = {p: -averted_abs[p] + COST[p] / THRESHOLD for p in POL}
for p in POL:
line(f" {NAME[p]:28s}: reported {T5_NET[p]}; cost/threshold = {COST[p]/THRESHOLD:.0f}; "
f"Table 5 attribution {net_t5[p]:.0f}; abstract attribution {net_abs[p]:+.0f}")
line(" (the 100 gap for 'including' is consistent with the costs being printed to one decimal "
f"in $ millions: $13.15M-$13.25M gives {-56400+13.15e6/THRESHOLD:.0f} to {-56400+13.25e6/THRESHOLD:.0f})")
line()
line("4. Dominance")
mid_daly = averted_t5["including"] / 2
mid_cost = COST["including"] / 2
line(f" Table 5 attribution: ICER(without) {icer_t5['without']:.0f} > ICER(including) {icer_t5['including']:.0f} "
f"-> 'without' extendedly dominated: {icer_t5['without'] > icer_t5['including']}")
line(f" midpoint of no DCP and 'including': {mid_daly:.0f} DALYs averted at ${mid_cost/1e6:.1f}M, "
f"vs 'without' {averted_t5['without']} at ${COST['without']/1e6:.1f}M")
inc_d = averted_t5["including"] - averted_t5["without"]
inc_c = COST["including"] - COST["without"]
line(f" incremental, including vs without: {inc_d} DALYs, ${inc_c/1e6:.1f}M -> ${inc_c/inc_d:.2f} per DALY")
strong = averted_abs["including"] < averted_abs["without"] and COST["including"] > COST["without"]
line(f" abstract attribution: 'including' averts {averted_abs['including']} < {averted_abs['without']} "
f"at higher cost -> strongly dominated by 'without': {strong}")
line()
line("5. Other outcomes in the paper")
line(f" HIV deaths averted per year: without {-T5_DEATHS['without']}, "
f"including {-T5_DEATHS['including']}; ratio {T5_DEATHS['including']/T5_DEATHS['without']:.2f}")
line(f" DALYs averted ratio including/without, Table 5: {averted_t5['including']/averted_t5['without']:.2f}; "
f"abstract: {averted_abs['including']/averted_abs['without']:.2f}")
line(f" incidence reduction vs no DCP: without {100*(1-IRR['without']):.0f}%, including {100*(1-IRR['including']):.0f}%")
# ---------------------------------------------------------------- figure style
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
ORANGE = "#eb6834"
BLUE = "#2a78d6"
GRID = "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "text.parse_math": False,
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
def fmt(n):
return f"{n:,}".replace(",", " ")
# ---------------------------------------------------------------- fig1: abstract vs Table 5
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
for x0, title in ((2, "Abstract, Findings"), (54, "Table 5 and Results")):
ax.add_patch(FancyBboxPatch((x0, 30), 44, 62, boxstyle="round,pad=0.6,rounding_size=1.5",
fc="white", ec=INK2, lw=1))
ax.text(x0 + 2, 88, title, fontsize=11.5, fontweight="bold", va="top")
# Abstract panel (as printed)
a_inc = ABS_DALY["including"]
a_wo = ABS_DALY["without"]
ax.text(4, 78, f"{fmt(a_inc[0])} DALYs per year\n(95% CI {fmt(a_inc[1])}–{fmt(a_inc[2])})",
fontsize=10.5, va="top", color=INK)
ax.text(4, 67, "for DCP including cabotegravir", fontsize=10.5, va="top", color=ORANGE, fontweight="bold")
ax.text(4, 56, f"{fmt(a_wo[0])} DALYs per year\n(95% CI {fmt(a_wo[1])}–{fmt(a_wo[2])})",
fontsize=10.5, va="top", color=INK)
ax.text(4, 45, "for DCP without cabotegravir", fontsize=10.5, va="top", color=ORANGE, fontweight="bold")
ax.text(4, 37, "Reported ICER, including: $234 per DALY", fontsize=10, va="top", color=INK2)
# Table 5 panel (as printed)
t_wo = T5_DALY["without"]
t_inc = T5_DALY["including"]
ax.text(56, 78, "DCP without cabotegravir", fontsize=10.5, va="top", color=INK, fontweight="bold")
ax.text(56, 73, f"−{fmt(-t_wo[0])} DALYs per year\n(−{fmt(-t_wo[1])} to −{fmt(-t_wo[2])})",
fontsize=10.5, va="top")
ax.text(56, 56, "DCP including cabotegravir", fontsize=10.5, va="top", color=INK, fontweight="bold")
ax.text(56, 51, f"−{fmt(-t_inc[0])} DALYs per year\n(−{fmt(-t_inc[1])} to −{fmt(-t_inc[2])})",
fontsize=10.5, va="top")
ax.text(56, 39, "Reported ICERs: $467 (without),\n$234 (including)", fontsize=10, va="top", color=INK2)
# crossing connectors: each abstract number to the same number in Table 5
ax.add_patch(FancyArrowPatch((31, 75), (55, 69.5), arrowstyle="-|>", mutation_scale=12,
color=INK2, lw=1.5, connectionstyle="arc3,rad=0"))
ax.add_patch(FancyArrowPatch((31, 53), (55, 49.5), arrowstyle="-|>", mutation_scale=12,
color=INK2, lw=1.5, connectionstyle="arc3,rad=0"))
ax.text(50, 95.5, "Same estimates and CIs; the two policy labels are swapped",
ha="center", fontsize=11, fontweight="bold", va="center")
# recomputed check underneath
ax.text(2, 24, "Check with the paper's own costs (Table 5: +$8.6M without, +$13.2M including, vs no DCP):",
fontsize=10, va="top", color=INK)
ax.text(2, 18,
f"Table 5 labels: $13.2M / {fmt(averted_t5['including'])} = ${icer_t5['including']:.0f} and "
f"$8.6M / {fmt(averted_t5['without'])} = ${icer_t5['without']:.0f} (both match)",
fontsize=10, va="top", color=BLUE)
ax.text(2, 12,
f"Abstract labels: $13.2M / {fmt(averted_abs['including'])} = ${icer_abs['including']:.0f} and "
f"$8.6M / {fmt(averted_abs['without'])} = ${icer_abs['without']:.0f} (match neither)",
fontsize=10, va="top", color=BLUE)
ax.text(2, 2, "Orange: abstract labels as printed. Blue: recomputed from the paper's numbers.",
fontsize=10, ha="left", va="bottom", color=INK2)
save(fig, "fig1")
# ---------------------------------------------------------------- fig2: ICER dot plot
fig, ax = plt.subplots(figsize=(7.2, 3.0))
fig.subplots_adjust(left=0.30, right=0.97, top=0.80, bottom=0.20)
ypos = {"including": 1, "without": 0}
for p in POL:
y = ypos[p]
ax.plot([min(icer_t5[p], icer_abs[p]), max(icer_t5[p], icer_abs[p])], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(ICER_REPORTED[p], y, "o", ms=13, mfc="none", mec=ORANGE, mew=2, zorder=3)
ax.plot(icer_t5[p], y, "o", ms=8, color=BLUE, zorder=4)
ax.plot(icer_abs[p], y, "o", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=4)
right = icer_abs[p] > icer_t5[p]
ax.annotate(f"${icer_abs[p]:.0f}", (icer_abs[p], y), xytext=(10 if right else -10, 0),
textcoords="offset points", ha="left" if right else "right", va="center",
fontsize=10, color=BLUE)
ax.annotate(f"${ICER_REPORTED[p]} reported\n${icer_t5[p]:.0f} recomputed", (icer_t5[p], y),
xytext=(0, -12), textcoords="offset points", ha="center", va="top", fontsize=10, color=INK)
ax.axvline(THRESHOLD, color=INK2, lw=1, ls=(0, (3, 3)), zorder=2)
ax.text(THRESHOLD + 8, 1.55, "threshold $500", ha="left", fontsize=10, color=INK2)
ax.set_yticks([0, 1])
ax.set_yticklabels([NAME["without"], NAME["including"]])
ax.set_ylim(-0.75, 1.75)
ax.set_xlim(0, 800)
ax.set_xticks(range(0, 801, 100))
ax.set_xticklabels([f"${v}" for v in range(0, 801, 100)])
ax.set_xlabel("ICER vs no DCP (US$ per DALY averted)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h1, = ax.plot([], [], "o", ms=13, mfc="none", mec=ORANGE, mew=2, ls="")
h2, = ax.plot([], [], "o", ms=8, color=BLUE, ls="")
h3, = ax.plot([], [], "o", ms=8, mfc=BG, mec=BLUE, mew=2, ls="")
fig.legend([h1, h2, h3], ["Reported ICER", "Cost / DALYs with Table 5 labels", "Cost / DALYs with abstract labels"],
loc="upper center", ncol=3, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.99),
handletextpad=0.3, columnspacing=1.2)
save(fig, "fig2")
line()
line("Figures written: fig1.svg/png, fig2.svg/png")
95% upper bound for zero neural tube defects among 384 dolutegravir-exposed outcomes is tenfold too small
Lancet HIV (2021) · doi:10.1016/S2352-3018(20)30268-X · PMC7869891
Summary
The paper's primary safety outcome is neural tube defect (NTD) prevalence after periconceptional dolutegravir (DTG) exposure. It observes no NTDs among 384 DTG-exposed and 1068 efavirenz (EFV)-exposed birth outcomes (Table 3). In the Abstract and the Results it reports a 95% Wilson confidence interval of 0–0·0010 for the DTG group and 0–0·0036 for the EFV group. For 0 events in n, the Wilson 95% upper bound is z²/(n + z²). With n = 384 that is 0·0099, about ten times the printed value; a bound of 0·0010 would need about 3838 outcomes, and no DTG-exposed pregnancy count in the paper comes close. The same method reproduces every other Wilson (prevalence) interval in the paper, including the EFV bound. The paper cites five DTG-exposed NTD rates from other studies: 1·04% (the Botswana estimate used in the Methods power calculation), 0·94%, 0·66% and 0·30% (Botswana) and 0·4% (the Antiretroviral Pregnancy Registry). The printed DTG bound (0·10%) lies below all five, so it appears to exclude every one of them. The bound implied by the paper's own counts (0·99%) contains four of them, 0·94%, 0·66%, 0·4% and 0·30%. Only the 1·04% power-calculation rate lies above it, by 0·05 percentage points.
What the paper reports
Abstract, Methods:
NTD incidences were calculated with 95% Wilson confidence intervals (95%CI).
Abstract, Findings:
There were no NTD in either DTG-exposed (0 [95%CI 0, 0·0010]) or EFV-exposed groups (0 [95%CI 0, 0·0036]).
Methods, "Variable definitions and statistical analysis":
We computed Wilson 95% confidence intervals (95%CI) for prevalence estimates of the primary outcomes.
Results, third paragraph:
The estimated NTD prevalence for DTG-exposed infants was 0 (95%CI 0-0·0010) and 0 (95%CI 0-0·0036) in EFV-exposed infants. Our composite outcome was more frequent in DTG-exposed fetuses (n=25, 7% [95%CI 0·044-0·094]) compared to those EFV-exposed (n=43, 4% [95%CI 0·030-0·054])
Table 3 ("Birth outcomes of periconceptionally DTG- and EFV- exposed women, Brazil, 2015-2018"), excerpt (selected rows; the P-value column and the "Outcome" row that carries its P value are omitted):
| Only EFV regimen(N=1068) | Any DTG regimen(N=384) |
Neural tube defects, n (%) | 0 (0) | 0 (0)
Live birth | 1025 (96) | 359 (93)
Stillbirth | 15 (1) | 2 (1)
Abortion | 28 (3) | 23 (6)
Results, "Updated NTD incidence estimate":
2 / (0·78 * 900 + 382) = 0·0018 or 1·8 per 1,000 DTG-exposed pregnancies (95%CI 0·5-6·7 per 1,000 DTG-exposed pregnancies)
Research in context panel, "Evidence before this study" (the comparison rates):
out of 426 women exposed to DTG at conception, four gave birth to infants with NTD (0·94%) compared to 0·10% among those exposed to other antiretrovirals. Since that initial warning, updated data from the Botswana study showed a decreased prevalence compared to the initial evaluation though statistically increased compared to women on non-DTG regimens (0·30% vs 0·10%). Other recent smaller studies from Botswana demonstrated increased NTD prevalence among women with DTG exposure (1 among 152 women or 0·66%) but statistical analyses comparing risk to other groups included the null. Lastly, data from North America and Europe using the Antiretroviral Pregnancy registry observed 1 NTD event among 248 DTG-exposed women (0·4%).
Methods, "Study design and data sources" (the power calculation):
Assuming a NTD prevalence of 0·06% for births among DTG-unexposed women and a prevalence of 1·04% in the DTG-exposed women (the estimated prevalence from Botswana available at the time)
Abstract, Interpretation:
Neither DTG or EFV exposure was associated with NTD in our national cohort; incidence of NTD is likely well under 1% among DTG-exposed HIV-positive women
The version of record repeats the bound. The publisher's edited abstract, as indexed in PubMed (PMID 33387477), reads:
There were no NTDs in either dolutegravir-exposed (0, 95% CI 0-0·0010) or efavirenz-exposed groups (0, 95% CI 0-0·0036).
What the paper's own numbers show
Method. I computed the Wilson score interval at the stated 95% level (z = 1·959964) for 0 events, using every denominator the paper gives for each group: birth outcomes (Table 3 N), women, and live births. For x = 0 the upper bound has the closed form z²/(n + z²). I inverted that formula to find the n at which the bound equals 0·0010. To rule out a different method or level, I also computed Clopper–Pearson, the rule of three, and Wilson bounds at 90% (equal to a one-sided 95% bound) and 99%. To test whether the authors' method and denominators are as stated, I recomputed the paper's other Wilson intervals: the composite outcome in each group and the updated DTG estimate. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| DTG NTD, 0/384 birth outcomes, Wilson 95% upper bound | Abstract (Findings); Results ¶3; Table 3 N | 0·0010 | 0·0099 (0·99%), 9·9 times the reported value |
| DTG NTD, 0/382 women | Abstract; Results | (same 0·0010) | 0·0100 |
| DTG NTD, 0/359 live births | Table 3 | (same 0·0010) | 0·0106 |
| EFV NTD, 0/1068 birth outcomes, Wilson 95% upper bound | Abstract (Findings); Results ¶3; Table 3 N | 0·0036 | 0·0036 (0·00358), matches |
| EFV NTD, 0/1045 women; 0/1025 live births | Results; Table 3 | (same 0·0036) | 0·0037; 0·0037 |
| DTG 0/384, other methods and levels | derived | 0·0010 | Clopper–Pearson 95% 0·0096; rule of three 0·0078; Wilson 90% (one-sided 95%) 0·0070; Wilson 99% 0·0170 |
| n at which a 0/n Wilson 95% upper bound equals 0·0010 | derived | — | 3838 (the value rounds to 0·0010 only for n from 3655 to 4039) |
| Largest DTG-exposed pregnancy or outcome counts | Abstract; Results; Table 3 | 382 women; 384 birth outcomes; updated DTG denominator 1084 (whole analysed cohort: 1427 women, 1452 outcomes) | all below 3655 |
| Larger counts in the paper that are not DTG-exposed pregnancy denominators | Results ¶1; Introduction | 3390 possible pregnancies screened; 2507 selected for investigation; 22 624 Brazilian women on DTG aged 15–49 | 0/n Wilson 95% upper bounds 0·0011, 0·0015 and 0·0002; none falls in the 3655–4039 range that gives 0·0010 |
| Composite outcome DTG, 25/384 | Results ¶3; Table 3 (2 stillbirths + 23 abortions) | 7% (0·044–0·094) | 6·5% (0·0445–0·0943), matches |
| Composite outcome EFV, 43/1068 | Results ¶3; Table 3 (15 + 28) | 4% (0·030–0·054) | 4·0% (0·0300–0·0538), matches |
| Updated DTG estimate, 2/(0·78 × 900 + 382) = 2/1084 | Results, updated estimate; Abstract | 0·0018 (0·0005–0·0067) | 0·00185 (0·00051–0·00670), matches |
| Table 3 column totals | Table 3 | N = 384; N = 1068 | 359 + 2 + 23 = 384; 1025 + 15 + 28 = 1068 |
The DTG NTD bound is the only Wilson (prevalence) interval in the paper that does not reproduce. Every other Wilson interval reproduces with the stated method and the Table 3 denominators. The bootstrap 95% CIs for the propensity-weighted odds ratios (Methods; Results; Figure 2 of the paper) depend on individual-level data and were not recomputed.
Figure
Figure 1. NTD prevalence intervals on one axis, in percent. Each pair of rows shows the interval as reported in the paper (orange) and as recomputed from the paper's own counts with the Wilson 95% method the paper states (blue). Dots are point estimates and the end ticks are upper bounds, with values printed next to them. For EFV (0/1068) and for the updated DTG estimate (2/1084) the reported and recomputed intervals coincide. For DTG (0/384) the recomputed upper bound is 0·99%, 9·9 times the reported 0·10%. The lower rows show every NTD rate the paper cites for comparison, each with a thin dashed line running up through the intervals: five DTG-exposed rates (filled diamonds: Botswana 1·04% from the power calculation, 0·94%, 0·66% and 0·30%, and the Pregnancy Registry 0·4%) and two rates without DTG exposure (hollow diamonds: Botswana non-DTG 0·10%, Brazil general population 0·06%). As printed, the DTG interval ends at 0·10% and lies below all five DTG-exposed rates. As recomputed, it contains four of them; only the 1·04% power-calculation rate lies above it.
Figure 2. The Wilson 95% upper bound for 0 events in n, z²/(n + z²), on log–log axes (blue line). The recomputed DTG (n = 384, 0·99%) and EFV (n = 1068, 0·36%) bounds lie on the line, and the EFV value matches the paper. The reported DTG bound of 0·0010 at n = 384 (orange dot, with a dashed orange line at its level) lies far off the line. The line reaches 0·0010 only at n = 3838 (hollow diamond), about ten times the DTG sample and larger than the whole analysed cohort (1452 birth outcomes).
Why it matters
NTD prevalence among women exposed to DTG at conception is the outcome the study was set up to measure, after the 2018 safety signal from Botswana. Its confidence interval is the cohort's own statement of how high the DTG-associated NTD risk could plausibly be. The misprinted bound is in the Abstract Findings, which is also the version of record's abstract and is indexed in PubMed, and in the Results. The correct value is printed nowhere in the paper, so a reader cannot recover it without recomputing.
The error changes what the cohort appears to exclude.
- As printed (upper bound 0·10%), 0/384 would appear to rule out every DTG-exposed NTD rate the paper cites: 1·04%, 0·94%, 0·66%, 0·4% and 0·30%. It would also put the DTG upper bound at the Botswana non-DTG background of 0·10%, implying the cohort alone could exclude any material DTG excess.
- As recomputed (upper bound 0·99%), the interval contains the DTG-exposed rates 0·94%, 0·66%, 0·4% and 0·30%, and both rates without DTG exposure (0·10% and 0·06%). Only the 1·04% rate assumed in the power calculation lies above it, by 0·05 percentage points. Zero events in 384 has probability 0·32 if the true prevalence is 0·30%, 0·21 at 0·4% and 0·079 at 0·66%. At 0·94% it has probability 0·027, just above the 0·025 tail of a two-sided 95% interval, consistent with 0·94% lying just inside the interval (the exact Clopper–Pearson bound is 0·96%); at 1·04% it is 0·018. The cohort by itself therefore cannot distinguish the Botswana updated rate from background. The Discussion acknowledges this point in words ("it is likely that our study was underpowered to detect a difference in NTD risk"). The printed interval is far narrower than that statement implies.
Also unaffected: the point estimates (0 NTDs in each group), the Table 3 counts, the composite-outcome intervals and regressions, and the updated estimate (0·18%, 95% CI 0·05–0·67%). The Interpretation's "well under 1%" is supported by the updated estimate, which is computed correctly. On the cohort data alone, though, the upper bound is 0·99%, just under 1%. The overall conclusions survive. What is overstated, about tenfold, is the precision of the primary-outcome estimate in the Abstract and Results.
Explanations considered
- A different denominator. This does not account for the numbers. The paper gives 384 birth outcomes (Table 3), 382 women and 359 live births for the DTG group, and these give upper bounds of 0·0099, 0·0100 and 0·0106. A bound of 0·0010 needs at least about 3655 observations. The largest DTG-exposed pregnancy count in the paper is the updated estimate's 1084, and the whole analysed cohort has 1452 outcomes. The paper does print larger counts (3390 possible pregnancies screened and 2507 selected for investigation in the Results, and 22 624 Brazilian women on DTG aged 15–49 in the Introduction), but none is a DTG-exposed pregnancy denominator, and none would give 0·0010 under the stated method in any case: their 0/n Wilson 95% upper bounds are 0·0011, 0·0015 and 0·0002.
- A different interval method or level. This does not account for the numbers. For 0/384, Clopper–Pearson gives 0·0096, the rule of three 0·0078, a one-sided 95% (two-sided 90%) Wilson bound 0·0070 and a 99% Wilson bound 0·0170. None is near 0·0010. The paper states Wilson 95% intervals twice.
- The method was applied differently throughout. This does not account for the numbers. The same Wilson 95% calculation reproduces the EFV NTD bound (0·0036), both composite-outcome intervals (0·044–0·094 and 0·030–0·054) and the updated estimate (0·0005–0·0067) to the printed precision. Only the DTG NTD bound departs.
- Percent versus proportion. This does not account for the numbers. The NTD bounds are printed as proportions, like the EFV bound. Read as a percentage, 0·0010% would need about 384 000 outcomes.
- A conversion or typesetting artefact in the archived manuscript. This is ruled out for the abstract: the publisher's edited abstract in PubMed prints the same 0·0010. The value also appears identically in two separate places in the manuscript.
- A typographical slip. This is the simplest account consistent with the numbers. The correct value, 0·0099, rounds to 0·010 at three decimals, and for 382 women it is 0·0100 at four. An extra or misplaced zero in either gives 0·0010. Whatever the route, the printed value is wrong and about ten times too small.
Status
- Version of record. The publisher's full text on thelancet.com and sciencedirect.com could not be opened by automated access (HTTP 403 and Cloudflare challenge pages), and doi.org resolved only to a redirect stub. The publisher's edited abstract was checked through PubMed (PMID 33387477, record revised 2026-01-27) and Europe PMC. It repeats "0, 95% CI 0-0·0010" for DTG and "0, 95% CI 0-0·0036" for EFV, and confirms the denominators of 384 DTG-exposed and 1068 EFV-exposed fetuses. The Results paragraph of the version of record could not be seen. The PMC text (author manuscript, NIHMS1664153) was used for every other quote, and each quote above was checked verbatim against it. Separately, the edited abstract gives the stillbirth-or-miscarriage counts as 23 (6%) and 28 (3%) with p=0·0037, where the manuscript abstract gives 25 (6·5%) and 43 (4·0%) with p=0·05. This difference does not bear on the NTD bound and is not analysed here.
- Supplements. The appendix (Supplemental Tables 1–6, including results for all women in Supplemental Table 3) is listed on the PMC page but could not be downloaded: PMC returned a placeholder or HTTP 403, Europe PMC reported it as not open access, and the publisher copy was blocked. It is unlikely to change the finding, because the version-of-record abstract itself confirms the denominators and no denominator or standard method gives 0·0010.
- Corrections and comments. As of 2026-10-23, none was found. Crossref lists no update relations for the DOI, PMC and Europe PMC list no correction notices, and the PubMed record has no erratum or comment links. A web search for the title with "correction", "erratum" or "PubPeer" found no correction or comment thread. The PubPeer API was blocked by a Cloudflare challenge, so PubPeer itself could not be queried directly.
- Sources. PMC7869891 as archived (Abstract, Methods, Results, Table 3, Research in context panel, Discussion); PubMed efetch for PMID 33387477; Crossref and PMC/Europe PMC notice queries;
recompute.py.txtin this folder.
Suggested correction
- In the Abstract (Findings) and the Results (third paragraph), replace the DTG NTD interval "0–0·0010" with the Wilson 95% interval for 0/384, 0–0·0099 (0–0·0100 if the 382 women are the intended denominator). Leave the EFV interval (0–0·0036) as it is.
- State the denominator used for the NTD prevalence (birth outcomes, women or live births), since the paper uses all three elsewhere.
- Optionally, note that the corrected DTG interval contains the DTG-exposed prevalences from the Research in context panel (0·94%, 0·66%, 0·4% and 0·30%), though not the 1·04% assumed in the power calculation, which fits the Discussion's statement that the study was underpowered for NTDs. The updated estimate, 0·0018 (0·0005–0·0067), needs no change.
Recomputation script (python recompute.py.txt)
"""Recompute the 95% Wilson confidence intervals for neural tube defect (NTD)
prevalence in PMC7869891 (Lancet HIV 2021, doi:10.1016/S2352-3018(20)30268-X).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PubMed Central (author manuscript):
Abstract (Findings) and Results paragraph 3: reported NTD intervals
Table 3: N = 384 (any DTG regimen) and N = 1068 (only EFV regimen) birth outcomes,
0 NTDs in each; live births 359 / 1025; stillbirths 2 / 15; abortions 23 / 28
Results (Characteristics): 382 DTG-exposed and 1045 EFV-exposed women
Results (Updated NTD incidence estimate): 2 / (0.78 * 900 + 382)
Methods: "We computed Wilson 95% confidence intervals (95%CI) for prevalence
estimates of the primary outcomes."
Research in context panel / Discussion: comparison rates 0.94%, 0.30%, 0.66%
(1/152) and 0.10% (Botswana), 0.4% (1/248, Antiretroviral Pregnancy
Registry) and 0.06% (Brazil, general population)
Methods (power calculation): assumed DTG prevalence 1.04% ("the estimated
prevalence from Botswana available at the time")
"""
import math
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975) # 1.959964
# ------------------------------------------------------------------ inputs (paper)
N_DTG_OUT, N_EFV_OUT = 384, 1068 # Table 3 column Ns (birth outcomes)
N_DTG_WOMEN, N_EFV_WOMEN = 382, 1045 # Results / Abstract
N_DTG_LIVE, N_EFV_LIVE = 359, 1025 # Table 3 live births
STILL = {"DTG": 2, "EFV": 15} # Table 3
ABORT = {"DTG": 23, "EFV": 28} # Table 3
COMPOSITE = {"DTG": 25, "EFV": 43} # Abstract / Results (manuscript)
REPORTED = { # Abstract Findings and Results paragraph 3
"DTG NTD upper": 0.0010,
"EFV NTD upper": 0.0036,
"DTG composite": (0.044, 0.094),
"EFV composite": (0.030, 0.054),
"updated": (0.0018, 0.0005, 0.0067),
}
UPD_NEW, UPD_FRAC, UPD_EVENTS = 900, 0.78, 2 # Results, updated estimate
CHART_CONFIRMED, CHART_REVIEWED = 382, 490
POWER_ASSUMED_DTG = 0.0104 # Methods power calculation: assumed DTG prevalence 1.04%
COMPARATORS = [ # (rate, label, DTG-exposed?) rates cited in the paper
(POWER_ASSUMED_DTG, "Botswana DTG, power calc. 1.04%", True), # Methods
(0.0094, "Botswana DTG, initial 0.94%", True), # Research in context
(0.0066, "Botswana DTG, 1/152, 0.66%", True), # Research in context
(0.0040, "Pregnancy Registry DTG, 1/248, 0.4%", True), # Research in context
(0.0030, "Botswana DTG, updated 0.30%", True), # Research in context
(0.0010, "Botswana non-DTG 0.10%", False), # Research in context
(0.0006, "Brazil, general population 0.06%", False), # Methods / Discussion
]
def wilson(x, n, conf=0.95):
z = stats.norm.ppf(1 - (1 - conf) / 2)
p = x / n
den = 1 + z * z / n
c = (p + z * z / (2 * n)) / den
h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / den
return max(0.0, c - h), c + h
def cp_upper(x, n, conf=0.95):
return stats.beta.ppf(1 - (1 - conf) / 2, x + 1, n - x)
def n_for_zero_upper(u, z=Z):
"""n at which the Wilson upper bound for 0/n equals u: z^2/(n+z^2) = u."""
return z * z / u - z * z
print("=" * 78)
print("1. Wilson 95% upper bound for 0 NTDs, every denominator in the paper")
print(" closed form for x = 0: upper = z^2 / (n + z^2), z = %.6f" % Z)
rows = [("DTG birth outcomes (Table 3 N)", N_DTG_OUT), ("DTG women", N_DTG_WOMEN),
("DTG live births (Table 3)", N_DTG_LIVE), ("EFV birth outcomes (Table 3 N)", N_EFV_OUT),
("EFV women", N_EFV_WOMEN), ("EFV live births (Table 3)", N_EFV_LIVE)]
for lab, n in rows:
w = wilson(0, n)[1]
print(f" {lab:32s} n={n:5d} Wilson={w:.5f} ({100*w:.2f}%) "
f"Clopper-Pearson={cp_upper(0, n):.5f} rule of three={3/n:.5f}")
print(f" reported: DTG {REPORTED['DTG NTD upper']:.4f} EFV {REPORTED['EFV NTD upper']:.4f}")
dtg_w = wilson(0, N_DTG_OUT)[1]
efv_w = wilson(0, N_EFV_OUT)[1]
print(f" ratio recomputed/reported, DTG: {dtg_w / REPORTED['DTG NTD upper']:.2f}")
print(f" EFV recomputed {efv_w:.5f} rounds to {round(efv_w, 4):.4f} (reported 0.0036)")
print("\n2. Other levels / methods for 0/384")
for conf in (0.90, 0.95, 0.99):
print(f" Wilson {conf:.0%} upper = {wilson(0, N_DTG_OUT, conf)[1]:.5f}")
print(f" one-sided 95% Wilson (z=1.645) = {wilson(0, N_DTG_OUT, 0.90)[1]:.5f}")
print(f" Clopper-Pearson 95% = {cp_upper(0, N_DTG_OUT):.5f}; rule of three = {3/N_DTG_OUT:.5f}")
print(f" 0.0010 read as a percentage = {0.0010/100:.6f} as a proportion "
f"(would need n of about {n_for_zero_upper(0.0010/100):,.0f})")
print("\n3. Sample size at which a 0/n Wilson 95% upper bound is 0.0010")
print(f" exactly 0.0010: n = {n_for_zero_upper(0.0010):.1f}")
print(f" rounds to 0.0010 (0.00095 <= upper < 0.00105): "
f"{n_for_zero_upper(0.00105):.1f} < n <= {n_for_zero_upper(0.00095):.1f}")
print(f" largest DTG-exposed pregnancy/outcome counts: {N_DTG_WOMEN} women, {N_DTG_OUT} outcomes; "
f"updated DTG denominator {UPD_FRAC*UPD_NEW + N_DTG_WOMEN:.0f}; "
f"whole analysed cohort {N_DTG_WOMEN + N_EFV_WOMEN} women, {N_DTG_OUT + N_EFV_OUT} outcomes")
LARGER = [ # larger counts printed in the paper that are not DTG-exposed pregnancy denominators
("possible pregnancies screened (Results para 1)", 3390),
("selected for investigation (Results para 1)", 2507),
("Brazilian women on DTG aged 15-49, May 2018 (Introduction)", 22624),
]
print(" larger counts in the paper (not DTG-exposed pregnancy denominators), 0/n Wilson 95% upper:")
for lab, n in LARGER:
u = wilson(0, n)[1]
print(f" {lab:60s} n={n:6d} upper={u:.5f} rounds to {round(u, 4):.4f} "
f"in rounding range for 0.0010? {n_for_zero_upper(0.00105) < n <= n_for_zero_upper(0.00095)}")
print("\n4. Checks that the stated method and denominators reproduce the paper's other CIs")
for arm, n in (("DTG", N_DTG_OUT), ("EFV", N_EFV_OUT)):
x = COMPOSITE[arm]
lo, hi = wilson(x, n)
print(f" composite {arm} {x}/{n}: {100*x/n:.1f}% Wilson {lo:.4f}-{hi:.4f} "
f"(reported {REPORTED[arm + ' composite'][0]:.3f}-{REPORTED[arm + ' composite'][1]:.3f})")
print(f" Table 3 stillbirths + abortions {arm}: {STILL[arm]} + {ABORT[arm]} = {STILL[arm]+ABORT[arm]}")
print(f" Table 3 live + still + abortion {arm}: "
f"{(N_DTG_LIVE if arm == 'DTG' else N_EFV_LIVE) + STILL[arm] + ABORT[arm]} (N = {n})")
print(f" chart-review fraction 382/490 = {CHART_CONFIRMED/CHART_REVIEWED:.4f}")
n_upd = UPD_FRAC * UPD_NEW + N_DTG_WOMEN
lo, hi = wilson(UPD_EVENTS, n_upd)
print(f" updated estimate 2/(0.78*900+382) = 2/{n_upd:.0f} = {UPD_EVENTS/n_upd:.5f} "
f"Wilson {lo:.5f}-{hi:.5f} (reported 0.0018, 0.0005-0.0067)")
print("\n5. What 0/384 can exclude: comparator rates cited in the paper")
print(" (a rate on the bound itself counts as not excluded)")
for r, lab, _ in COMPARATORS:
p0 = (1 - r) ** N_DTG_OUT
print(f" {lab:36s} inside reported CI (<=0.0010)? {r <= 0.0010!s:5s} "
f"inside recomputed CI (<={dtg_w:.4f})? {r <= dtg_w!s:5s} P(0 events in 384) = {p0:.4f}")
print(f" gap between 1.04% and the recomputed bound: {100*(POWER_ASSUMED_DTG - dtg_w):.3f} "
f"percentage points")
print(" two-sided 95% interval: each tail has probability 0.025")
# ------------------------------------------------------------------ figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, REF = "#eb6834", "#2a78d6", "#e4e3df", "#9a9893"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.edgecolor": SEC, "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight")
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight")
plt.close(fig)
# Figure 1: reported vs recomputed intervals on one axis (percent), with every
# rate the paper cites drawn as its own row and a thin dashed line up through
# the interval rows
from matplotlib.lines import Line2D # noqa: E402
upd_lo, upd_hi = wilson(UPD_EVENTS, n_upd)
items = [ # (label, point, lo, hi, color)
("DTG, 0/384 — reported", 0, 0, REPORTED["DTG NTD upper"], ORANGE),
("DTG, 0/384 — recomputed (Wilson)", 0, 0, dtg_w, BLUE),
("EFV, 0/1068 — reported", 0, 0, REPORTED["EFV NTD upper"], ORANGE),
("EFV, 0/1068 — recomputed (Wilson)", 0, 0, efv_w, BLUE),
("Updated DTG, 2/1084 — reported", 0.0018, 0.0005, 0.0067, ORANGE),
("Updated DTG, 2/1084 — recomputed", UPD_EVENTS / n_upd, upd_lo, upd_hi, BLUE),
]
ys = [12.2, 11.6, 10.4, 9.8, 8.6, 8.0]
rate_ys = [6.6 - 0.6 * i for i in range(len(COMPARATORS))] # descending rates, top to bottom
TOP = 12.9
LBL_BOX = dict(boxstyle="square,pad=0.12", fc=BG, ec="none")
fig, ax = plt.subplots(figsize=(7.2, 6.6))
for (r, lab, is_dtg), y in zip(COMPARATORS, rate_ys):
ax.plot([100 * r, 100 * r], [y, TOP], color=REF, lw=0.9, ls=(0, (3, 3)), zorder=1)
ax.plot([100 * r], [y], marker="D", ms=8, color=SEC if is_dtg else BG, mec=SEC, mew=1.5,
zorder=4)
ax.text(100 * r + 0.02, y, f"{100*r:.2f}%", va="center", ha="left", fontsize=10,
color=SEC, zorder=5)
for (lab, p, lo, hi, col), y in zip(items, ys):
ax.plot([100 * lo, 100 * hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
ax.plot([100 * hi], [y], marker="|", ms=10, mew=2, color=col, zorder=3)
ax.plot([100 * p], [y], marker="o", ms=8, color=col, zorder=4, mfc=col, mec=BG, mew=1)
ax.text(100 * hi + 0.015, y, f"{100*hi:.2f}%", va="center", ha="left", fontsize=10,
color=col, bbox=LBL_BOX, zorder=5)
ax.set_yticks(ys + rate_ys)
ax.set_yticklabels([it[0] for it in items] + [c[1].rsplit(" ", 1)[0].rstrip(",").replace("power calc.", "power calculation")
for c in COMPARATORS],
fontsize=10)
for t, c in zip(ax.get_yticklabels(), [TXT] * len(items) + [SEC] * len(COMPARATORS)):
t.set_color(c)
ax.text(-0.03, 7.25, "Rates the paper cites (filled = DTG-exposed, hollow = not DTG-exposed)",
ha="left", va="center", fontsize=10, color=SEC, bbox=LBL_BOX, zorder=5)
ax.set_ylim(rate_ys[-1] - 0.5, TOP)
ax.set_xlim(-0.03, 1.2)
ax.set_xlabel("NTD prevalence, % (point estimate and 95% CI)")
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.annotate("", xy=(100 * dtg_w, 11.9), xytext=(100 * REPORTED["DTG NTD upper"], 11.9),
arrowprops=dict(arrowstyle="->", color=SEC, lw=1.2), zorder=5)
ax.text(100 * (dtg_w + REPORTED["DTG NTD upper"]) / 2, 11.93,
f"{dtg_w / REPORTED['DTG NTD upper']:.1f}× wider", ha="center", va="bottom",
fontsize=10, color=SEC, bbox=LBL_BOX, zorder=5)
ax.legend(handles=[Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, label="Reported in the paper"),
Line2D([], [], color=BLUE, lw=2, marker="o", ms=8,
label="Recomputed from the paper's counts (Wilson 95%)"),
Line2D([], [], color=SEC, lw=0.9, ls=(0, (3, 3)), marker="D", ms=8,
label="Rate cited in the paper")],
loc="upper center", bbox_to_anchor=(0.3, -0.1), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# Figure 2: Wilson 0/n upper bound as a function of n
n_grid = np.logspace(2, 4.2, 400)
fig, ax = plt.subplots(figsize=(7.2, 4.0))
ax.plot(n_grid, 100 * Z * Z / (n_grid + Z * Z), color=BLUE, lw=2)
n_req = n_for_zero_upper(REPORTED["DTG NTD upper"])
ax.axhline(100 * REPORTED["DTG NTD upper"], color=ORANGE, lw=1.2, ls=(0, (4, 3)))
ax.plot([N_DTG_OUT], [100 * REPORTED["DTG NTD upper"]], "o", ms=9, color=ORANGE)
ax.plot([N_DTG_OUT], [100 * dtg_w], "o", ms=9, color=BLUE, mec=BG)
ax.plot([N_EFV_OUT], [100 * efv_w], "s", ms=9, color=BLUE, mec=BG)
ax.plot([n_req], [100 * REPORTED["DTG NTD upper"]], "D", ms=8, mfc=BG, mec=BLUE, mew=2,
)
ax.annotate(f"DTG, n = 384: {100*dtg_w:.2f}%", (N_DTG_OUT, 100 * dtg_w), xytext=(10, 6),
textcoords="offset points", color=BLUE, fontsize=10)
ax.set_xscale("log")
ax.set_yscale("log")
ax.set_xlim(100, 15000)
ax.set_ylim(0.02, 5)
ax.set_xticks([100, 384, 1068, 3838, 10000])
ax.set_xticklabels(["100", "384", "1068", "3838", "10 000"])
ax.set_yticks([0.05, 0.1, 0.36, 1, 2])
ax.set_yticklabels(["0.05%", "0.10%", "0.36%", "1%", "2%"])
ax.minorticks_off()
ax.set_xlabel("Number of birth outcomes, n (log scale)")
ax.set_ylabel("Upper 95% bound, % (log scale)")
ax.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.annotate("Reported: 0.0010 at n = 384", (N_DTG_OUT, 100 * REPORTED["DTG NTD upper"]),
xytext=(10, -16), textcoords="offset points", color=ORANGE, fontsize=10)
ax.annotate(f"EFV, n = 1068: {100*efv_w:.2f}%\n(matches reported 0.0036)", (N_EFV_OUT, 100 * efv_w),
xytext=(12, 4), textcoords="offset points", color=BLUE, fontsize=10)
ax.annotate(f"n = {n_req:,.0f} needed for 0.0010", (n_req, 100 * REPORTED["DTG NTD upper"]),
xytext=(8, 8), textcoords="offset points", color=BLUE, fontsize=10)
ax.text(700, 2.3, "Line: Wilson 95% upper bound for 0 events in n", color=BLUE, fontsize=10, ha="left")
save(fig, "fig2")
print("\nfigures written: fig1.svg/png, fig2.svg/png")
Reported p<0.001 for mouse leak frequencies cannot be reproduced; the paper's counts give p=0.11–0.36
Lancet Neurology (2026) · doi:10.1016/S1474-4422(26)00140-7 · PMC13275636
Summary
The Results report that CSF leaks were significantly more frequent in all three Fbn2 mutant mouse strains than in wild-type mice: relative risks (RR) of 1.38, 1.33 and 1.27, 95% confidence intervals that exclude 1, and "all comparisons p<0·001". The legend of Figure 4A goes further, marking p<0.0001 for two of the strains. The paper names the χ² test for this comparison. Applied to the paper's own counts (15/19, 22/29 and 8/11 against 20/35 in wild-type), that test gives p = 0.109, 0.117 and 0.356. Fisher's exact test gives 0.14, 0.19 and 0.49, and the 95% CIs for the RRs are 0.96–2.00, 0.93–1.89 and 0.80–2.02, so all three include 1. The reported CIs are reproduced to two decimals only if each percentage is treated as a count out of 100 mice per group. The comparison is the main in-vivo support for the abstract's statement that the mutant mice "showed a predisposition for dural rupture". The paper's genetic burden results do not depend on it.
What the paper reports
Results, in-vivo paragraph (Marfan comparison, then Fbn2 strains):
There was no significant increase in CSF leak frequency in Marfan syndrome mice compared with wild-type mice (seven [54%] of 13 vs 20 [57%] of 35, respectively, p>0·05, relative risk [RR]=0·94, 95% CI 0·74–1·22, figure 4A). Contrastingly, these frequencies were significantly higher for Fbn2A1052T/+ mice (15 [79%] of 19), RR 1·38, 95% CI 1·14–1·69), for Fbn2D1581V/+ mice (22 [76%] of 29], RR 1·33, 95% CI 1·09–1·63), and for Fbn2M2387T/+ mice (eight [73%] of 11], RR 1·27, 95% CI 1·04–1·58), with all comparisons p<0·001 (figure 4A).
(The unmatched brackets are in the published text.)
Methods, Statistical analysis:
Proportions of mice with leaks were compared between mutant and wild-type mice by use of χ2 tests. p<0·05 was used as significance threshold.
Figure 4 legend:
(A) Proportion of CSF leak occurrence in mouse strains following intrathecal infusion.
*p<0·001. †p<0·0001.
The earlier check of the figure image (gr4.jpg from the publisher's supplementary package) found † on the A1052T and D1581V bars and * on the M2387T bar. The bar heights (about 57%, 54%, 79%, 76% and 73%) match the counts in the text.
Abstract, Findings:
Mice carrying variants equivalent to the three FBN2 variants in TB domains found in individuals with type 1b spontaneous spinal CSF leak (Fbn2A1052T/+, Fbn2D1581V/+, and Fbn2M2387T/+) showed a predisposition for dural rupture on controlled leak induction.
Research in context, Added value of this study:
Functional in-vitro studies and animal model data are supportive of a causal effect of the identified variants in FBN2.
Discussion, first paragraph:
We also showed increased CSF leak susceptibility and severity in fibrillin-2 mutant mice, and increased dural compliance in Marfan syndrome mice.
Discussion, mouse paragraph:
Our findings revealed that type 1b spontaneous spinal CSF leak-associated variants in the TB3, TB4, and TB7 domains of fibrillin-2 led to increased CSF leak frequency and severity in mice, a characteristic not attributable to increased pressure or volume at the time of leak.
Marfan syndrome mice also showed increased meningeal compliance without increased leak frequency or severity.
All quotes were checked verbatim against the PMC-archived text. Superscripts on the allele names (A1052T/+ etc.) are lost in plain text.
What the paper's own numbers show
Method. Each strain was compared with the wild-type group (20 of 35 mice with leaks) in a 2 × 2 table built from the counts printed in the Results. For each comparison I calculated the RR with its usual log-scale (Katz) 95% CI and the Pearson χ² test without continuity correction, which is the test named in the Methods. As checks I also ran the Yates-corrected χ², Fisher's exact test (two-sided and one-sided), Barnard's exact test, a pooled comparison of all Fbn2 mice, and a comparison against pooled controls (wild-type plus Marfan). To find where the reported intervals come from, I repeated the calculation with each printed percentage used as a count out of 100. All of this is in recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's counts |
|---|---|---|---|
| Leak frequencies | Results; Fig 4A | 79%, 76%, 73%, 54% vs 57% | 15/19 = 78.9%, 22/29 = 75.9%, 8/11 = 72.7%, 7/13 = 53.8%, 20/35 = 57.1% (consistent) |
| Fbn2 A1052T/+ vs WT, p | Results; Fig 4A (†) | p<0.001 (text); p<0.0001 (figure) | χ² p = 0.109 (Yates 0.192; Fisher 0.142; Barnard 0.123) |
| Fbn2 A1052T/+ RR (95% CI) | Results | 1.38 (1.14–1.69) | 1.38 (0.96–2.00) |
| Fbn2 D1581V/+ vs WT, p | Results; Fig 4A (†) | p<0.001 (text); p<0.0001 (figure) | χ² p = 0.117 (Yates 0.192; Fisher 0.186; Barnard 0.133) |
| Fbn2 D1581V/+ RR (95% CI) | Results | 1.33 (1.09–1.63) | 1.33 (0.93–1.89) |
| Fbn2 M2387T/+ vs WT, p | Results; Fig 4A (*) | p<0.001 | χ² p = 0.356 (Yates 0.569; Fisher 0.486; Barnard 0.449) |
| Fbn2 M2387T/+ RR (95% CI) | Results | 1.27 (1.04–1.58) | 1.27 (0.80–2.02) |
| Fbn1 C1039G/+ (Marfan) RR (95% CI), p | Results | 0.94 (0.74–1.22), p>0.05 | 0.94 (0.53–1.68), χ² p = 0.838. The conclusion (not significant) holds; the CI is too narrow |
| One-sided Fisher (mutant > WT) | check | n/a | 0.095, 0.095, 0.289 |
| SE(log RR): from the reported CI vs from the counts | check | 0.100, 0.103, 0.107, 0.128 | 0.188, 0.180, 0.236, 0.296 (1.75–2.32 times wider) |
| RR CI with percentages used as counts out of 100 | diagnostic | 1.14–1.69 / 1.09–1.63 / 1.04–1.58 / 0.74–1.22 | 79/100, 76/100, 73/100, 54/100 vs 57/100: 1.14–1.69 / 1.09–1.63 / 1.04–1.58 / 0.74–1.21 |
| χ² p with percentages used as counts out of 100 | diagnostic | p<0.001 (all three) | 0.0009 / 0.0044 / 0.018. Even this does not give p<0.001 for all three |
| All Fbn2 mice pooled (45/59) vs WT 20/35 | sensitivity | n/a | RR 1.33 (0.97–1.84); χ² p = 0.052; Fisher p = 0.066 |
| Each Fbn2 strain vs pooled controls (WT + Marfan, 27/48) | sensitivity | n/a | χ² p = 0.083, 0.083, 0.316 |
| Global 2 × 5 χ² across all five strains | sensitivity | n/a | χ² = 4.99, df = 4, p = 0.288 |
| Leaks needed for χ² p<0.001 vs WT 20/35 | sensitivity | observed 15/19, 22/29, 8/11 | ≥19/19, ≥28/29; not reachable with 11 mice (11/11 gives p = 0.008) |
Every printed percentage matches its count, so the counts are internally consistent. No standard test of these counts gives p<0.05 for any single strain, let alone p<0.001. For the M2387T group, p<0.001 cannot be reached against 20/35 even if all 11 mice had leaked.
Figure
Figure 1. Relative risk of CSF leak in each mutant strain compared with wild-type mice (20/35), on a log scale. Orange circles and bars are the RRs and 95% CIs printed in the Results. Blue squares and bars are the RRs and log-scale 95% CIs recomputed from the counts printed alongside them (shown under each strain name). The thin vertical line marks RR = 1. The point estimates agree. The recomputed intervals are 1.75 to 2.3 times wider on the log scale, and for all three Fbn2 strains they cross 1, whereas the reported intervals do not. The Marfan interval is also too narrow as printed, although both versions include 1.
Figure 2. p-values for each Fbn2 strain against wild-type, on a log scale. Orange triangles show the reported thresholds: filled for "p<0·001" in the Results text, open for the "p<0·0001" marker (†) in the Figure 4 legend. Each triangle marks an upper bound. Blue markers show the values recomputed from the paper's counts: filled squares for the Pearson χ² test named in the Methods, open diamonds for Fisher's exact test. The thin vertical line marks the paper's significance threshold, p = 0.05. The recomputed values lie two to three orders of magnitude above the reported bounds, and all lie above 0.05.
Why it matters
The in-vivo leak experiment is the paper's test of whether the FBN2 variants found in patients impair dural integrity in a living animal. The leak-frequency comparison is the only in-vivo measure the paper reports as showing a difference in leak occurrence between Fbn2 mice and wild-type mice. The abstract ("showed a predisposition for dural rupture on controlled leak induction"), the Research in context panel ("animal model data are supportive of a causal effect") and the Discussion ("led to increased CSF leak frequency") all rely on it. The contrast the Discussion draws with Marfan mice, which showed increased compliance "without increased leak frequency", also relies on it. Recomputed from the paper's own counts, none of the three strain-level differences is statistically significant at the paper's own threshold of 0.05. The pooled Fbn2 group is borderline (p = 0.052). The direction of effect is unchanged: every Fbn2 strain leaked more often than wild-type (73–79% vs 57%). With 11 to 29 mice per group, the data are compatible both with no difference and with a doubling of risk.
Several results are unaffected. The exome burden enrichment (p = 0.041, 0.004 and 0.0003 in the three control cohorts) rests on human data, and so do the domain-distribution and fibroblast-binding results. The separately reported larger pressure drop at leak onset in Fbn2 mice (Figure 4E–F) and the Marfan compliance result (Figure 4D) are different measurements; they could not be checked here because no per-mouse data are published. The genetic conclusion therefore still stands. What changes is that the leak-frequency data do not, by themselves, provide statistically significant in-vivo evidence for it.
Explanations considered
- A different test from the one named. The Methods name χ² tests. Pearson χ² (0.109, 0.117, 0.356), Yates-corrected χ², Fisher's exact (two-sided or one-sided), and Barnard's exact test all give p>0.09 for every strain. This does not account for the reported values.
- A different comparator or a global test. Comparing against pooled controls (wild-type plus Marfan) gives p = 0.083–0.316. A global 2 × 5 χ² gives p = 0.288. Pooling all Fbn2 mice gives p = 0.052. This does not account for the reported values.
- Different units (infusion trials rather than mice). The Figure 4A axis refers to trial outcome, but the Results report the counts as mice ("15 [79%] of 19"), the Methods describe the analysis as "Proportions of mice with leaks", and the bar heights match the mouse counts. The paper describes no repeated trials per mouse. If there were repeated trials per mouse, the mouse would remain the experimental unit, and treating trials as independent would understate the uncertainty rather than justify it. This does not account for the reported values on the evidence available.
- Percentages used as counts out of 100. Treating 79%, 76%, 73% and 54% as counts out of 100 per group (against 57/100) reproduces all four printed CIs to within 0.01. This is the only calculation found that does so. Even this calculation gives χ² p = 0.0009, 0.0044 and 0.018, so it does not reproduce "all comparisons p<0·001" or the p<0.0001 markers. The source of the p-values remains unidentified. The point estimates are unaffected, since an RR does not depend on the denominator.
- Correction for multiple comparisons. Any correction would make the p-values larger, not smaller.
- Rounding or typesetting. The discrepancies are a factor of 100 or more in p and roughly a factor of two in CI width. They appear in the text and in the figure markers, and they affect all three strains. A typesetting or rounding cause is not plausible.
- Text-conversion artifact in PMC. The same text appears in the publisher-supplied JATS in Europe PMC and on the live PMC page, and the figure markers come from the publisher's image. This does not account for the discrepancy.
- Data not in the paper. No per-mouse data or source data file is deposited, and the supplementary appendix has no contingency tables. If the printed counts were wrong, the recomputed values would change. However, the counts agree with the printed percentages and with the bar heights in Figure 4A, so they are what the paper presents as its data.
Status
As of 2026-10-23:
- Version of record. The publisher pages could not be opened from this machine: doi.org redirects to linkinghub.elsevier.com, which returned only a redirect stub, and thelancet.com and sciencedirect.com both returned HTTP 403. The text was checked in two archived copies, the publisher-supplied JATS full text served by Europe PMC and the PMC page (pmc.ncbi.nlm.nih.gov/articles/PMC13275636/). Both carry the Results passage, the Methods sentence and the Figure 4 legend quoted above.
- Supplements. The Europe PMC supplementary package was read in the earlier check. It holds mmc1.pdf (appendix: structural modelling, binding methods, co-IP blots, CRISPR sequences, uncropped blots) and the figure images gr1–gr4. It has no per-mouse leak data, no contingency tables and no further description of the leak-proportion test. It does not resolve the discrepancy. A re-download attempted for this write-up timed out.
- Corrections. None was found. Crossref update metadata (
src/known.py): none. PMC notices: none. Europe PMC (PMID 42309087) lists no comment or correction. The Crossref record has no update-to or relation entries. - PubPeer. The PubPeer search API returned a human-verification challenge, and the PubPeer search page returned HTTP 403, so PubPeer could not be queried. A web search could not be run for this write-up because the session's search budget was exhausted. No PubPeer discussion is known, but its absence is not confirmed.
Suggested correction
In the Results, in-vivo paragraph, the RR confidence intervals and p-values should be recomputed from the mouse-level counts. If the counts printed are correct, the passage would read approximately:
There was no significant increase in CSF leak frequency in Marfan syndrome mice compared with wild-type mice (seven [54%] of 13 vs 20 [57%] of 35, respectively, p=0·84, relative risk [RR]=0·94, 95% CI 0·53–1·68, figure 4A). Leak frequencies were numerically higher for Fbn2A1052T/+ mice (15 [79%] of 19; RR 1·38, 95% CI 0·96–2·00; p=0·11), for Fbn2D1581V/+ mice (22 [76%] of 29; RR 1·33, 95% CI 0·93–1·89; p=0·12), and for Fbn2M2387T/+ mice (eight [73%] of 11; RR 1·27, 95% CI 0·80–2·02; p=0·36), but none of these differences was statistically significant (χ² tests).
The significance markers on Figure 4A for the three Fbn2 strains would need to be removed or replaced with "NS". The Abstract ("showed a predisposition for dural rupture"), the Research in context panel and the two Discussion passages describing increased leak frequency should be revised to match. For example, they could state that leak frequency was higher in all three Fbn2 strains but not significantly so, and rest any in-vivo claim on the pressure-drop results, if those are confirmed. Depositing per-mouse leak data would let readers verify the corrected analysis.
Recomputation script (python recompute.py.txt)
"""Recompute the in-vivo CSF-leak frequency comparisons in PMC13275636
(Lancet Neurol 2026; doi:10.1016/S1474-4422(26)00140-7) from the paper's own printed
counts, and redraw fig1/fig2 in this file's folder.
Run: python recompute.py.txt (needs numpy, scipy >= 1.7, matplotlib)
Inputs are copied from the published article (Results, in-vivo paragraph; Methods,
Statistical analysis; Figure 4 legend).
"""
import os
import numpy as np
from scipy import stats
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# ---------------------------------------------------------------- inputs (Results)
WT = (20, 35) # "20 [57%] of 35" wild-type mice with leaks
GROUPS = [
# label, (leaks, mice), reported RR, CI low, CI high, reported p, printed %
("Fbn2 A1052T/+", (15, 19), 1.38, 1.14, 1.69, "p<0.001 (Fig 4A: p<0.0001)", 79),
("Fbn2 D1581V/+", (22, 29), 1.33, 1.09, 1.63, "p<0.001 (Fig 4A: p<0.0001)", 76),
("Fbn2 M2387T/+", (8, 11), 1.27, 1.04, 1.58, "p<0.001 (Fig 4A: p<0.001)", 73),
("Fbn1 C1039G/+ (Marfan)", (7, 13), 0.94, 0.74, 1.22, "p>0.05", 54),
]
WT_PCT = 57
REPORTED_P_BOUND = {"Fbn2 A1052T/+": 1e-4, "Fbn2 D1581V/+": 1e-4, "Fbn2 M2387T/+": 1e-3}
def katz(a, n1, c, n0):
"""Relative risk with the usual log-scale (Katz) 95% CI."""
rr = (a / n1) / (c / n0)
se = np.sqrt(1 / a - 1 / n1 + 1 / c - 1 / n0)
return rr, rr * np.exp(-Z * se), rr * np.exp(Z * se), se
def wald_linear(a, n1, c, n0):
"""Delta-method CI on the untransformed RR scale (an alternative some software uses)."""
rr, _, _, se = katz(a, n1, c, n0)
return rr - Z * rr * se, rr + Z * rr * se
def table(a, n1, c, n0):
return np.array([[a, n1 - a], [c, n0 - c]])
def pvals(a, n1, c, n0):
t = table(a, n1, c, n0)
chi2_stat, p_chi, _, _ = stats.chi2_contingency(t, correction=False)
p_yates = stats.chi2_contingency(t, correction=True)[1]
p_fisher = stats.fisher_exact(t)[1]
p_fisher_1s = stats.fisher_exact(t, alternative="greater")[1]
p_barnard = stats.barnard_exact(t).pvalue
return dict(chi2=chi2_stat, chi=p_chi, yates=p_yates, fisher=p_fisher,
fisher1=p_fisher_1s, barnard=p_barnard)
print("=== 1. Percentages printed in the Results ===")
for lab, (a, n), *_rest in GROUPS:
print(f"{lab:24s} {a}/{n} = {100*a/n:.1f}% (printed {_rest[-1]}%)")
print(f"{'Wild-type':24s} {WT[0]}/{WT[1]} = {100*WT[0]/WT[1]:.1f}% (printed {WT_PCT}%)")
print("\n=== 2. Each strain vs wild-type 20/35, with the paper's counts ===")
RES = {}
for lab, (a, n1), rr_r, lo_r, hi_r, p_r, _ in GROUPS:
rr, lo, hi, se = katz(a, n1, *WT)
wl, wh = wald_linear(a, n1, *WT)
p = pvals(a, n1, *WT)
RES[lab] = dict(rr=rr, lo=lo, hi=hi, se=se, **p)
print(f"{lab}: {a}/{n1} vs {WT[0]}/{WT[1]}")
print(f" RR {rr:.3f}; log-scale 95% CI {lo:.2f}-{hi:.2f}; linear-scale 95% CI {wl:.2f}-{wh:.2f}"
f" | reported RR {rr_r} (95% CI {lo_r}-{hi_r})")
print(f" Pearson chi2 = {p['chi2']:.2f}, p = {p['chi']:.3f}; Yates p = {p['yates']:.3f}; "
f"Fisher p = {p['fisher']:.3f}; one-sided Fisher p = {p['fisher1']:.3f}; "
f"Barnard p = {p['barnard']:.3f} | reported {p_r}")
print("\n=== 3. Width of the reported CIs vs the width the counts imply ===")
for lab, (a, n1), rr_r, lo_r, hi_r, _, _ in GROUPS:
se_rep = (np.log(hi_r) - np.log(lo_r)) / (2 * Z)
se_true = RES[lab]["se"]
print(f"{lab:24s} SE(log RR) implied by reported CI = {se_rep:.3f}; from counts = {se_true:.3f}; "
f"ratio {se_true/se_rep:.2f}; implied effective sample size inflation ~{(se_true/se_rep)**2:.1f}x")
print("\n=== 4. Diagnostic: percentages used as if they were counts out of 100 per arm ===")
for lab, (a, n1), rr_r, lo_r, hi_r, _, pct in GROUPS:
rr, lo, hi, _ = katz(pct, 100, WT_PCT, 100)
p = pvals(pct, 100, WT_PCT, 100)
print(f"{lab:24s} {pct}/100 vs {WT_PCT}/100: RR {rr:.3f}, 95% CI {lo:.2f}-{hi:.2f} "
f"(reported {lo_r}-{hi_r}); chi2 p = {p['chi']:.4f}, Fisher p = {p['fisher']:.4f}")
if lab in REPORTED_P_BOUND:
g = stats.chisquare([pct, 100 - pct], f_exp=[WT_PCT, 100 - WT_PCT])
print(f"{'':24s} goodness-of-fit chi2 of {pct}% vs expected {WT_PCT}%, n=100: p = {g.pvalue:.1e}")
print("\n=== 5. Sensitivity: other comparisons that do not rescue p<0.001 ===")
a = sum(g[1][0] for g in GROUPS[:3]); n1 = sum(g[1][1] for g in GROUPS[:3])
rr, lo, hi, _ = katz(a, n1, *WT); p = pvals(a, n1, *WT)
print(f"All Fbn2 strains pooled {a}/{n1} vs WT 20/35: RR {rr:.2f} (95% CI {lo:.2f}-{hi:.2f}); "
f"chi2 p = {p['chi']:.3f}; Fisher p = {p['fisher']:.3f}")
ctrl = (WT[0] + GROUPS[3][1][0], WT[1] + GROUPS[3][1][1])
print(f"Controls pooled (WT + Marfan) = {ctrl[0]}/{ctrl[1]}")
for lab, (a, n1), *_ in GROUPS[:3]:
p = pvals(a, n1, *ctrl)
print(f" {lab} vs pooled controls: chi2 p = {p['chi']:.3f}, Fisher p = {p['fisher']:.3f}")
t = np.array([[g[1][0] for g in GROUPS] + [WT[0]],
[g[1][1] - g[1][0] for g in GROUPS] + [WT[1] - WT[0]]])
c5 = stats.chi2_contingency(t)
print(f"Global 2x5 chi2 across all five strains: chi2 = {c5[0]:.2f}, df = {c5[2]}, p = {c5[1]:.3f}")
print("\n=== 6. Leak count needed for Pearson chi2 p<0.001 vs WT 20/35 ===")
for lab, (a, n1), *_ in GROUPS[:3]:
need = [k for k in range(n1 + 1)
if k / n1 > WT[0] / WT[1] and pvals(k, n1, *WT)["chi"] < 0.001]
req = f"needs >= {need[0]}/{n1} leaks" if need else "impossible at any leak count"
print(f"{lab:24s} n = {n1}: {req} (observed {a}); "
f"if all {n1} leaked: p = {pvals(n1, n1, *WT)['chi']:.4f}")
# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.ticker import FixedLocator, FixedFormatter, NullLocator
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": SEC,
"svg.fonttype": "none", "font.family": "DejaVu Sans"})
def save(fig, name):
fig.savefig(os.path.join(HERE, f"{name}.svg"), facecolor=BG, bbox_inches="tight")
fig.savefig(os.path.join(HERE, f"{name}.png"), facecolor=BG, bbox_inches="tight", dpi=200)
plt.close(fig)
def style(ax):
ax.set_facecolor(BG)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.xaxis.grid(True, color=GRID, lw=0.8, which="major")
ax.set_axisbelow(True)
# ---- fig1: relative risk, reported vs recomputed 95% CI, log scale
labels = [f"{g[0]}\n{g[1][0]}/{g[1][1]} vs WT {WT[0]}/{WT[1]}" for g in GROUPS]
fig, ax = plt.subplots(figsize=(7.2, 4.4), facecolor=BG)
style(ax)
ys = np.arange(len(GROUPS))[::-1].astype(float)
off = 0.17
for y, (lab, (a, n1), rr_r, lo_r, hi_r, _, _) in zip(ys, GROUPS):
r = RES[lab]
ax.plot([lo_r, hi_r], [y + off] * 2, color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot(rr_r, y + off, "o", color=ORANGE, ms=8, zorder=3)
ax.text(hi_r * 1.04, y + off, f"{lo_r:.2f}–{hi_r:.2f}", va="center", fontsize=10, color=ORANGE)
ax.plot([r["lo"], r["hi"]], [y - off] * 2, color=BLUE, lw=2, solid_capstyle="butt")
ax.plot(r["rr"], y - off, "s", color=BLUE, ms=8, zorder=3)
ax.text(r["hi"] * 1.04, y - off, f"{r['lo']:.2f}–{r['hi']:.2f}", va="center", fontsize=10,
color=BLUE)
ax.axvline(1, color=SEC, lw=0.8)
ax.text(0.98, ys[0] + 0.55, "RR = 1 (no difference)", ha="right", fontsize=10, color=SEC)
ax.set_xscale("log")
ticks = [0.5, 0.75, 1, 1.5, 2, 3]
ax.xaxis.set_major_locator(FixedLocator(ticks))
ax.xaxis.set_major_formatter(FixedFormatter([str(t) for t in ticks]))
ax.xaxis.set_minor_locator(NullLocator())
ax.set_xlim(0.45, 3.0)
ax.set_ylim(-0.6, len(GROUPS) - 0.2)
ax.set_yticks(ys); ax.set_yticklabels(labels, fontsize=10)
ax.set_xlabel("Relative risk of CSF leak vs wild-type (log scale), with 95% CI")
ax.plot([], [], "o-", color=ORANGE, ms=8, lw=2, label="reported in Results")
ax.plot([], [], "s-", color=BLUE, ms=8, lw=2, label="recomputed from the paper's counts")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.03), ncol=2, fontsize=10, frameon=False)
save(fig, "fig1")
# ---- fig2: p-values, reported bound vs recomputed, log scale
fig, ax = plt.subplots(figsize=(7.2, 3.5), facecolor=BG)
style(ax)
fb = GROUPS[:3]
ys2 = np.arange(len(fb))[::-1].astype(float)
for y, (lab, (a, n1), *_r) in zip(ys2, fb):
r = RES[lab]
b = REPORTED_P_BOUND[lab]
ax.plot(1e-3, y + off, "<", color=ORANGE, ms=9, zorder=3)
if b < 1e-3:
ax.plot(b, y + off, "<", color=ORANGE, ms=9, mfc=BG, mew=2, zorder=3)
ax.plot(r["chi"], y - off, "s", color=BLUE, ms=8, zorder=3)
ax.plot(r["fisher"], y - off, "D", color=BLUE, ms=8, mfc=BG, mew=2, zorder=3)
ax.plot([r["chi"], r["fisher"]], [y - off] * 2, color=BLUE, lw=2, zorder=2)
ax.text(max(r["chi"], r["fisher"]) * 1.25, y - off, f"{r['chi']:.2f} / {r['fisher']:.2f}",
va="center", fontsize=10, color=BLUE)
ax.axvline(0.05, color=SEC, lw=0.8)
ax.text(0.045, ys2[0] + 0.5, "p = 0.05", ha="right", fontsize=10, color=SEC)
ax.set_xscale("log")
pt = [1e-5, 1e-4, 1e-3, 1e-2, 0.05, 1]
ax.xaxis.set_major_locator(FixedLocator(pt))
ax.xaxis.set_major_formatter(FixedFormatter(["0.00001", "0.0001", "0.001", "0.01", "0.05", "1"]))
ax.xaxis.set_minor_locator(NullLocator())
ax.set_xlim(5e-6, 1.6)
ax.set_ylim(-0.55, len(fb) - 0.25)
ax.set_yticks(ys2)
ax.set_yticklabels([f"{g[0]}\n{g[1][0]}/{g[1][1]} vs WT {WT[0]}/{WT[1]}" for g in fb], fontsize=10)
ax.set_xlabel("p-value, mutant vs wild-type leak frequency (log scale)")
ax.plot([], [], "<", color=ORANGE, ms=9, label="reported: p<0.001 (Results text)")
ax.plot([], [], "<", color=ORANGE, ms=9, mfc=BG, mew=2, label="reported: p<0.0001 (Fig 4A legend)")
ax.plot([], [], "s", color=BLUE, ms=8, label="recomputed: Pearson χ² (named in Methods)")
ax.plot([], [], "D", color=BLUE, ms=8, mfc=BG, mew=2, label="recomputed: Fisher's exact")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.02), ncol=2, fontsize=10, frameon=False)
save(fig, "fig2")
print("\nfigures written to", HERE)
The firearm death-in-ED estimate is reported as 88.82, 83.82, 77.63 and 379.00
Lancet Reg Health Am (2026) · doi:10.1016/j.lana.2025.101343 · PMC12811418
Summary
The paper's headline mortality result is the adjusted relative risk (RR) of dying in the emergency department (ED) for women injured by firearms compared with women injured by other assault mechanisms. The same estimate appears with four different values: 88.82 ("89 times") in the abstract, 83.82 in Table 3, 77.63 in the Results text, and 379.00 in the Discussion. The paper's own numbers show where each value comes from. The abstract prints Table 3's confidence interval (72.38–97.06), and that interval is centred on 83.82, not 88.82. The Results paragraph that cites Table 3 reproduces the age 18–29 column of Table 4. The Discussion's 379.00 (266.15–539.7) is the odds ratio from the supplementary logistic regression, which the Methods say was set aside because it gave inflated estimates. All versions are far above 1, so the direction and significance of the association hold. But the size of the paper's key finding is misstated in the abstract, and the Discussion overstates it by a factor of about 4.5.
What the paper reports
The abstract:
Female patients who were injured by firearms had 89 times higher risk of dying in the ED (RR = 88.82; 95% CI 6 = 72.38–97.06) compared to female patients injured by non-firearm injury mechanisms.
— Abstract, Findings (the stray "6" is in the source, and in the PubMed abstract too)
Table 3, model (b), "firearm-related assault compared to all other adult female assault patients". The columns are all-assault RR, its CI, firearm RR, its CI:
In ED | 1.04 | 0.89, 1.21 | 83.82 | 72.38, 97.06
— Table 3, Death: In ED row
The Results paragraph on the firearm model, which ends by citing Table 3:
Female patients with firearm injuries had far greater risk of dying in the ED (RR = 77.63; 95% CI 62.60–96.27) or hospital (RR = 3.29; 95% CI 2.57–4.22). They also had substantially higher risk of being admitted to the hospital (RR = 19.94; 95% CI 18.04–22.02). Compared to White female patients with firearm injuries, Black (RR = 4.15; 95% CI 3.61–4.78), Hispanic (RR = 1.77; 95% CI 1.48–2.13) and Other (RR = 2.01; 95% CI 1.51–2.68) patients had greater risk of firearm injury (Table 3).
— Results, paragraph 4 ("In the firearm injury model for women…")
Table 4 gives the age-stratified models. In the In ED row, the second estimate is the 18–29 y firearm model:
In ED | 3.14 (2.45, 4.02) | 77.63 (62.60, 96.27)
— Table 4, Death: In ED row (first two of eight columns)
The same 18–29 y value restated:
There appeared to be a bimodal distribution of increased mortality for firearm injuries for 18–29 years (death in ED: RR = 77.65; 95% CI 62.60–96.27, death in hospital: RR = 3.29; 95% CI 2.57–4.22)
— Results, paragraph 5
The Discussion:
Female patients who were injured by firearms specifically had 379.00 (95% CI 266.15–539.7) times higher risk of dying in the ED compared to all-cause female assault patients.
— Discussion, paragraph 1
On the logistic model:
We initially employed a survey-weighted binary logistic regression model. However, several results were affected by sparse-data bias and quasi-complete separation, leading to inflated odds ratio estimates.
The results from the initial survey-weighted logistic regression are provided in the Supplement for comparison.
— Methods, statistical analysis
The initial logistic regression was affected by sparse-data bias and quasi-complete separation, which inflated the odds ratios for several variables.
— Discussion, final paragraph
Supplementary Table 1 ("Adjusted Logistic Regression Model…", file mmc1.docx), firearm model, Death row: "ED | 1.04 | 0.89 – 1.21 | 379.00 | 266.15 – 539.71".
Every quotation from the article was checked character for character against the text archived in PubMed Central (src/show.py PMC12811418). Each occurs exactly once, and recompute.py.txt reruns that check. The supplementary row is transcribed from the publisher's mmc1.docx. Paragraphs are numbered from 1 under the Results and Discussion headings.
What the paper's own numbers show
Method. The model is a survey-weighted Poisson GLM with a log link (Methods; Table 3 caption), so its 95% intervals are Wald intervals on the log scale: L = exp(b − 1.96·SE) and U = exp(b + 1.96·SE). The point estimate that belongs with a printed interval is therefore its geometric midpoint, exp(b) = √(L·U). Comparing each printed value with √(L·U), and then searching the paper's tables for the matching cell, shows which numbers belong together. No outside data are used. The underlying NEDS microdata are licensed and not public, so the model itself cannot be refitted. The check is limited to the internal consistency of the reported values.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Firearm vs other assault, death in ED, all ages | Table 3 (b) | 83.82 (72.38, 97.06) | √(72.38 × 97.06) = 83.816 → 83.82. Consistent (log distance 0.147 to each limit) |
| Same estimate | Abstract | "89 times", RR = 88.82 (72.38–97.06) | Same CI as Table 3, whose centre is 83.82. 88.82 is off-centre (log distance 0.205 to L, 0.089 to U), 0.77 SE above the centre, and 6.0% above Table 3. "89" is round(88.82); round(83.82) = 84 |
| Same estimate, "firearm injury model … (Table 3)" | Results para 4 | 77.63 (62.60–96.27) | √(62.60 × 96.27) = 77.631. Identical to Table 4, 18–29 y firearm column; 7.4% below Table 3 |
| Same paragraph: death in hospital, admitted, Black, Hispanic, Other | Results para 4 | 3.29 (2.57–4.22); 19.94 (18.04–22.02); 4.15 (3.61–4.78); 1.77 (1.48–2.13); 2.01 (1.51–2.68) | Table 3 gives 3.34 (2.79, 4.00); 16.50 (15.29, 17.80); 4.12 (3.75, 4.52); 1.87 (1.64, 2.12); 1.90 (1.55, 2.33); none of these 15 numbers agree. Table 4 18–29 y gives 3.29 (2.57, 4.22); 19.94 (18.05, 22.02); 4.16 (3.61, 4.78); 1.78 (1.48, 2.14); 2.01 (1.51, 2.68). Across all six outcome and race rows, 14 of 18 numbers are identical to Table 4 18–29 y and the other 4 are within 0.01 |
| Same paragraph: age | Results para 4 | 0.98 (0.97–0.98) | Matches Table 3 (0.98, 0.97–0.98), not Table 4 18–29 y (0.96, 0.95–0.98). This is the only row of the paragraph that comes from Table 3 |
| 18–29 y firearm, death in ED | Results para 5 | 77.65 (62.60–96.27) | Table 4 gives 77.63; √(L·U) = 77.631 |
| Firearm death in ED, "times higher risk" | Discussion para 1 | 379.00 (266.15–539.7) | √(266.15 × 539.7) = 379.00. Equal to Suppl. Table 1 logistic adjusted odds ratio 379.00 (266.15–539.71). It appears in no RR table and is 4.52 × Table 3's RR |
| Firearm model, Black vs White | Discussion para 1, "nearly five times" | — | Table 3 RR = 4.12 (3.75, 4.52), as in the abstract. Suppl. Table 1 OR = 4.74 (4.29–5.24). "Nearly five" fits the odds ratio |
| All-assault model, Native American vs White | Abstract | RR = 2.81 (2.67–2.97), no age group stated | Equal to Table 4, 18–29 y column (stated as such in Results para 7). The all-ages Table 3 value is 2.68 (2.60, 2.77) |
| All-assault model, 18–29 y, death in ED | Results para 5 | 3.14 (2.45–4.20) | Table 4 gives 3.14 (2.45, 4.02). √(2.45 × 4.02) = 3.138 fits 3.14, while √(2.45 × 4.20) = 3.208 does not, so 4.20 is a transposition of 4.02 |
So the one value that is internally consistent is Table 3's 83.82 (72.38–97.06). The abstract pairs a different point estimate with Table 3's interval. The two differ in a single digit (88.82 vs 83.82), which looks like a typographical slip. The Results paragraph on the firearm model reports the 18–29 y stratum of Table 4 instead of the all-ages model it cites. The Discussion reports the logistic odds ratio as a risk. The last three rows of the table are minor slips of the same kind and are listed for completeness.
Figure
Figure 1. The printed versions of one estimate, and the centres of their printed intervals. Panel A shows each printed value (orange dot) with its printed 95% CI (orange line), on a logarithmic axis that runs from the null value (thin line at 1) to 1000. Next to it is the geometric centre of that CI, √(L×U), recomputed (blue open diamond). The Results para 4 value equals the age 18–29 y value in Table 4. The Discussion value equals the logistic odds ratio in Supplementary Table S1 and lies about 4.5 times above the Poisson estimates. Panel B zooms in (log axis, 70–100) on the two rows that print the same interval, 72.38–97.06. Table 3's 83.82 sits at the centre (log distance 0.147 to each limit). The abstract's 88.82 does not (0.205 and 0.089). All intervals lie far from 1.
Figure 2. Results paragraph 4 against the two table columns it could come from. The left column (orange) holds the values printed in the Results paragraph that cites Table 3. The middle and right columns hold the firearm-model values printed in Table 3 (all ages) and Table 4 (age 18–29 y). A cell is blue and bold when all three of its numbers (estimate and both limits) equal the Results-text value to within 0.01, and grey otherwise. For death in the ED, death in hospital, admission and the three race contrasts, the Results text matches Table 4's 18–29 y column (14 of 18 numbers identical, the rest within 0.01) and none of Table 3's 18 numbers. Only the age coefficient matches Table 3.
Why it matters
- It is the headline quantitative finding. The abstract's only mortality estimate is this RR, and the abstract, Results and Discussion each lead with it. A reader, or an indexing service that shows only the abstract, gets 88.82. A reader of the Discussion gets 379. The value the paper's own tables support is 83.82: about 84 times, not 89. The Discussion overstates it by a factor of 4.52.
- The Discussion figure uses the model the paper itself rejects. The Methods and the closing Discussion paragraph explain that the logistic model was replaced because it "inflated the odds ratios". The 379.00 in the Discussion's summary of findings is that model's odds ratio, presented as "times higher risk". For an outcome this rare among non-firearm assaults, an odds ratio and a relative risk are not interchangeable. Here they differ by a factor of 4.5. The same paragraph's "nearly five times more likely to identify as Black" also matches the odds ratio (4.74), not the RR the abstract reports (4.12).
- The Results text attributes subgroup numbers to the overall model. The firearm-model paragraph says "(Table 3)" but gives the 18–29 y estimates for death in the ED, death in hospital, admission and race. For admission, the stated 19.94 is 21% higher than the all-ages 16.50. A reader who compares text with table finds that they disagree on most of the rows.
- What is not affected. Every version of the estimate excludes 1 by a wide margin, so the direction and statistical significance of the firearm–mortality association do not change. The title and the qualitative Conclusion ("significantly higher mortality rates") also stand. Table 3 and Table 4 are internally consistent: each of the 15 cells used here sits at the centre of its CI to within 0.006, which is rounding. The problem is therefore in how the tables were carried into the text, not in the models as tabulated.
Explanations considered
- The abstract reports a different model or population than Table 3. This does not work. The abstract's comparison ("compared to female patients injured by non-firearm injury mechanisms") is Table 3 model (b), and it prints exactly Table 3's interval, 72.38–97.06. No estimate of 88.82 appears in any table or in the supplement. A point estimate of 88.82 cannot come from a log-scale Wald interval of 72.38–97.06, because it lies 0.77 standard errors from the centre. The abstract's own interval therefore implies 83.82.
- A rounding or significant-figure difference. This does not work. 88.82 and 83.82 differ by 5.00 (6%), and "89 times" rounds 88.82, not 83.82 (84).
- Results paragraph 4 deliberately describes the youngest age group. This does not work. The paragraph opens "In the firearm injury model for women", gives no age group, and ends "(Table 3)". Its age coefficient (0.98, 0.97–0.98) is the all-ages Table 3 value; within a single age stratum the coefficient differs (0.96 for 18–29 y). The age-stratified results are introduced only in the next paragraph ("We re-ran 8 survey-weighted Poisson regression models by age category … in Table 4"). That paragraph restates the same 18–29 y firearm values, which confirms where they came from.
- The Discussion intentionally quotes the logistic model. The value is indeed the logistic odds ratio (Supplementary Table 1: 379.00, 266.15–539.71). But the sentence calls it "times higher risk" without saying it is an odds ratio from the supplementary model, and it sits in the summary of the paper's own findings. The Methods say that model gave inflated odds ratios and was replaced. The comparator in the sentence ("all-cause female assault patients") also differs from the model's comparator (assault by all other mechanisms). So even read as intentional, the sentence misdescribes the measure and does not report the paper's primary estimate.
- A conversion or typesetting artifact in PMC. This is ruled out. The same values appear in the Europe PMC full-text JATS XML (each of "RR = 88.82", "RR = 77.63", "RR = 77.65", "379.00" and "83.82" exactly once) and in the PubMed abstract (PMID 41551920), which also carries the stray "95% CI 6 =". The table cells line up with their headers, and no minus signs, superscripts or decimal commas are involved.
- A table rather than the text is wrong. This is unlikely. Each of the 15 Table 3 and Table 4 cells examined sits at the geometric centre of its own interval to within 0.006 (rounding), against 5.00 for the abstract's 88.82. The textual values match specific cells of other tables (Table 4 18–29 y; Supplementary Table 1). The only value with no source in any table is the abstract's 88.82, and its interval is Table 3's.
None of these explanations reconciles the four values. The consistent reading is that Table 3 holds the primary estimate, 83.82 (72.38–97.06), and that the abstract, the Results paragraph and the Discussion each misreport it in a different way.
Status
As of 2026-10-23:
- Version of record: checked through publisher-deposited copies. The publisher's own pages could not be opened: doi.org leads to linkinghub.elsevier.com, which returned a redirect stub, and thelancet.com and sciencedirect.com returned HTTP 403. The Europe PMC full-text XML for PMC12811418 and the PubMed record (PMID 41551920) both carry every quoted passage unchanged. The Elsevier article metadata lists The Lancet Regional Health – Americas, February 2026, open access, CC BY 4.0.
- Supplements: the article's only supplementary file, mmc1.docx ("Supplementary Tables", 31,729 bytes), was downloaded from the publisher CDN and read in full. It holds Supplementary Tables 1 and 2, both logistic regression (adjusted odds ratios), all ages and by age group. Supplementary Table 1 is the source of the Discussion's 379.00 (266.15–539.71) and of OR 4.74 for Black women. No RR of 88.82 appears in it.
- Corrections and notices:
src/known.py 10.1016/j.lana.2025.101343returns no Crossref update and no PMC or Europe PMC notice. The Europe PMC record lists no comment or correction links. No erratum, correction, expression of concern or retraction was found. - PubPeer: PubPeer's API returned "Human verification required" (HTTP 403), so it could not be queried directly. A web search for the title together with "pubpeer" returned no PubPeer page. A thread therefore cannot be ruled out with certainty, but none was found.
- Sources: PMC12811418 (
src/show.py); https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12811418/fullTextXML; PubMed 41551920; https://ars.els-cdn.com/content/image/1-s2.0-S2667193X25003540-mmc1.docx; https://doi.org/10.1016/j.lana.2025.101343.
Suggested correction
- Abstract, Findings: replace "89 times higher risk of dying in the ED (RR = 88.82; 95% CI 6 = 72.38–97.06)" with "84 times higher risk of dying in the ED (RR = 83.82; 95% CI 72.38–97.06)", matching Table 3.
- Results, paragraph 4: replace the age 18–29 y values with the all-ages Table 3 values: death in ED RR = 83.82 (72.38–97.06), death in hospital RR = 3.34 (2.79–4.00), admitted RR = 16.50 (15.29–17.80), Black RR = 4.12 (3.75–4.52), Hispanic RR = 1.87 (1.64–2.12), Other RR = 1.90 (1.55–2.33). Alternatively, label them explicitly as the 18–29 y estimates and cite Table 4.
- Results, paragraph 5: change "RR = 77.65" to "RR = 77.63", and change the upper limit of the 18–29 y all-assault death-in-ED interval from "4.20" to "4.02", matching Table 4.
- Discussion, paragraph 1: replace "379.00 (95% CI 266.15–539.7) times higher risk of dying in the ED compared to all-cause female assault patients" with the primary Poisson estimate: "an 84-fold higher risk of dying in the ED (RR = 83.82; 95% CI 72.38–97.06) compared to female patients injured by other assault mechanisms". If the logistic estimate is kept, label it as an odds ratio from the supplementary model. Similarly, "nearly five times more likely to identify as Black" should read about four times (RR = 4.12; 95% CI 3.75–4.52), matching the abstract and Table 3.
- Abstract, Native American estimate (optional): either state that RR = 2.81 (2.67–2.97) refers to women aged 18–29 y, or give the all-ages Table 3 value, RR = 2.68 (2.60–2.77).
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12811418 (Lancet Reg Health Am 2026, doi:10.1016/j.lana.2025.101343)
# One adjusted estimate (firearm vs other assault mechanisms, death in the ED) is printed
# with four different values in the abstract, Table 3, the Results text and the Discussion.
#
# Run: python recompute.py.txt
# Prints every number used in writeup.md and regenerates fig1.{svg,png} and fig2.{svg,png}
# in the folder that holds this file. Needs matplotlib (numpy not required).
#
# Inputs (all transcribed from the paper; nothing is estimated from outside data)
# * Text quotes and Tables 3 and 4: the article as archived in PubMed Central (PMC12811418).
# When run inside the /home/ubuntu/papers repo, the script re-checks that every quote
# occurs exactly once in the archived full text via src/show.py (skipped elsewhere).
# * Supplementary Tables S1/S2: the article's only supplementary file, mmc1.docx
# ("Supplementary Tables"), from
# https://ars.els-cdn.com/content/image/1-s2.0-S2667193X25003540-mmc1.docx
# These are the survey-weighted LOGISTIC regression results (adjusted odds ratios), which
# the Methods say were set aside as inflated by sparse-data bias / quasi-complete separation.
#
# Method. For a ratio estimate with a Wald interval on the log scale,
# L = exp(b - z*SE), U = exp(b + z*SE) => exp(b) = sqrt(L*U).
# So the point estimate that belongs with a printed interval is the interval's geometric
# midpoint. Comparing each printed point estimate with sqrt(L*U) identifies which printed
# values belong together, and the matching table cell identifies where each value came from.
import math, os, subprocess
import matplotlib
import matplotlib.ticker
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
PMCID = "PMC12811418"
# ------------------------------------------------------------------ quotes (verbatim)
QUOTES = [
("Abstract, Findings",
"Female patients who were injured by firearms had 89 times higher risk of dying in the ED "
"(RR = 88.82; 95% CI 6 = 72.38–97.06) compared to female patients injured by non-firearm "
"injury mechanisms."),
("Abstract, Findings",
"Racial disparities were prevalent, with Native American women experiencing the greatest risk "
"of being assaulted (RR = 2.81; 95% CI 2.67–2.97)."),
("Abstract, Findings",
"Victims of firearm related assaults had nearly 4.12 times the risk of identifying as Black "
"compared with those assaulted without firearms (95% CI 3.75–4.52)."),
("Methods, statistical analysis",
"We initially employed a survey-weighted binary logistic regression model. However, several "
"results were affected by sparse-data bias and quasi-complete separation, leading to inflated "
"odds ratio estimates."),
("Methods, statistical analysis",
"The results from the initial survey-weighted logistic regression are provided in the "
"Supplement for comparison."),
("Results, paragraph 4 (firearm injury model)",
"Female patients with firearm injuries had far greater risk of dying in the ED (RR = 77.63; "
"95% CI 62.60–96.27) or hospital (RR = 3.29; 95% CI 2.57–4.22). They also had substantially "
"higher risk of being admitted to the hospital (RR = 19.94; 95% CI 18.04–22.02). Compared to "
"White female patients with firearm injuries, Black (RR = 4.15; 95% CI 3.61–4.78), Hispanic "
"(RR = 1.77; 95% CI 1.48–2.13) and Other (RR = 2.01; 95% CI 1.51–2.68) patients had greater "
"risk of firearm injury (Table 3)."),
("Results, paragraph 5 (age strata, Table 4)",
"There appeared to be a bimodal distribution of increased mortality for firearm injuries for "
"18–29 years (death in ED: RR = 77.65; 95% CI 62.60–96.27, death in hospital: RR = 3.29; "
"95% CI 2.57–4.22)"),
("Results, paragraph 5 (age strata, Table 4)",
"Female patients 18–29 years old, had the greatest risk of death, whether in the ED "
"(RR = 3.14; 95% CI 2.45–4.20)"),
("Discussion, paragraph 1",
"Female patients who were injured by firearms specifically had 379.00 (95% CI 266.15–539.7) "
"times higher risk of dying in the ED compared to all-cause female assault patients."),
("Discussion, paragraph 1",
"victims of firearm-related assaults nearly five times more likely to identify as Black than "
"those assaulted without firearms"),
("Discussion, final paragraph",
"The initial logistic regression was affected by sparse-data bias and quasi-complete "
"separation, which inflated the odds ratios for several variables."),
("Table 3, Death: In ED row (all-assault RR, CI, firearm RR, CI)",
"In ED | 1.04 | 0.89, 1.21 | 83.82 | 72.38, 97.06"),
("Table 4, Death: In ED row (18-29 firearm = 2nd estimate)",
"In ED | 3.14 (2.45, 4.02) | 77.63 (62.60, 96.27)"),
]
def check_quotes():
repo = "/home/ubuntu/papers"
py = "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
print("Quote check: src/show.py not available here; skipped.\n")
return
full = subprocess.run([py, "src/show.py", PMCID, "--full"], cwd=repo,
capture_output=True, text=True).stdout
print("Quote check against the PMC-archived text (occurrences; must be 1):")
for loc, q in QUOTES:
n = full.count(q)
print(f" [{'ok ' if n == 1 else 'BAD'}] {n} {loc}: {q[:70]}...")
print()
# ------------------------------------------------------------------ the estimates
# (key, label, location, measure, point, lo, hi)
ED_DEATH = [
("abs", "Abstract", "Abstract, Findings", "RR", 88.82, 72.38, 97.06),
("t3", "Table 3 (b), all ages", "Table 3, Death: In ED", "RR", 83.82, 72.38, 97.06),
("rp3", "Results p4 (cites Table 3)", "Results para 4", "RR", 77.63, 62.60, 96.27),
("t4", "Table 4, firearm, 18-29 y", "Table 4, Death: In ED", "RR", 77.63, 62.60, 96.27),
("rp4", "Results p5, 18-29 y", "Results para 5", "RR", 77.65, 62.60, 96.27),
("disc", "Discussion", "Discussion para 1", "'risk'", 379.00, 266.15, 539.7),
("s1", "Suppl. Table S1 (logistic OR)", "Suppl. Table S1, Death: ED", "OR", 379.00, 266.15, 539.71),
]
# Other age strata, Table 4 firearm In ED (for context in the figure)
T4_OTHER = [("30-39 y", 78.40, 59.30, 103.65), ("40-49 y", 92.63, 66.98, 128.10),
("50+ y", 110.20, 70.04, 173.38)]
def gmid(lo, hi):
return math.sqrt(lo * hi)
def logasym(p, lo, hi):
# (log U - log p) - (log p - log L); 0 when p is the geometric midpoint
return (math.log(hi) - math.log(p)) - (math.log(p) - math.log(lo))
def part1():
print("1. Death in the ED, firearm vs other assault mechanisms: printed value vs geometric "
"midpoint of its printed 95% CI")
print(f" {'source':34s} {'meas':6s} {'printed':>8s} {'CI':>17s} {'sqrt(L*U)':>10s} "
f"{'log-asym':>9s}")
for k, lab, loc, m, p, lo, hi in ED_DEATH:
print(f" {lab:34s} {m:6s} {p:8.2f} {lo:8.2f}-{hi:<8.2f} {gmid(lo, hi):10.3f} "
f"{logasym(p, lo, hi):9.4f}")
p, lo, hi = 88.82, 72.38, 97.06
se = (math.log(hi) - math.log(lo)) / (2 * 1.959964)
print(f" Abstract CI: log-distance from 88.82 to L = {math.log(p/lo):.4f}, to U = "
f"{math.log(hi/p):.4f}; from 83.82: {math.log(83.82/lo):.4f} and {math.log(hi/83.82):.4f}")
print(f" Implied SE(log RR) = {se:.4f}; 88.82 lies {(math.log(p)-math.log(gmid(lo,hi)))/se:.2f} "
f"SE above the CI centre")
print(f" 88.82 / 83.82 = {88.82/83.82:.4f} (abstract {100*(88.82/83.82-1):.1f}% above Table 3)")
print(f" '89 times' = round(88.82) = {round(88.82)}; round(83.82) = {round(83.82)}")
print(f" 77.63 / 83.82 = {77.63/83.82:.4f} (Results p4 {100*(1-77.63/83.82):.1f}% below Table 3)")
print(f" 379.00 / 83.82 = {379.00/83.82:.2f} (Discussion vs Table 3)")
print(f" Abstract 88.82 and Table 3 83.82 differ in one digit (8 vs 3, second digit)")
print()
# Results p4 values vs Table 3 firearm column and Table 4 18-29 firearm column
P3_ROWS = [
# quantity, Results p4, Table 3 firearm (all ages), Table 4 firearm 18-29
("Death in ED", (77.63, 62.60, 96.27), (83.82, 72.38, 97.06), (77.63, 62.60, 96.27)),
("Death in hospital",(3.29, 2.57, 4.22), (3.34, 2.79, 4.00), (3.29, 2.57, 4.22)),
("Admitted", (19.94, 18.04, 22.02), (16.50, 15.29, 17.80), (19.94, 18.05, 22.02)),
("Black", (4.15, 3.61, 4.78), (4.12, 3.75, 4.52), (4.16, 3.61, 4.78)),
("Hispanic", (1.77, 1.48, 2.13), (1.87, 1.64, 2.12), (1.78, 1.48, 2.14)),
("Other race", (2.01, 1.51, 2.68), (1.90, 1.55, 2.33), (2.01, 1.51, 2.68)),
("Age (per year)", (0.98, 0.97, 0.98), (0.98, 0.97, 0.98), (0.96, 0.95, 0.98)),
]
def part2():
print("2. Results paragraph 4 ('firearm injury model ... (Table 3)'): which table column do its "
"numbers match?")
print(f" {'quantity':18s} {'Results p4':>20s} {'Table 3 firearm':>20s} {'Table 4 18-29':>20s} "
f"exact(T3) exact(T4) within0.01(T4)")
tot3 = tot4 = near4 = n = 0
for q, r, t3, t4 in P3_ROWS:
e3 = sum(a == b for a, b in zip(r, t3)); e4 = sum(a == b for a, b in zip(r, t4))
w4 = sum(abs(a - b) <= 0.0101 for a, b in zip(r, t4))
fmt = lambda t: f"{t[0]:.2f} ({t[1]:.2f}-{t[2]:.2f})"
print(f" {q:18s} {fmt(r):>20s} {fmt(t3):>20s} {fmt(t4):>20s} {e3:>8d}/3 {e4:>8d}/3 {w4:>8d}/3")
if q != "Age (per year)":
tot3 += e3; tot4 += e4; near4 += w4; n += 3
print(f" Six outcome/race rows (18 numbers): exact match Table 3 = {tot3}/{n}, "
f"Table 4 18-29 = {tot4}/{n}, within 0.01 of Table 4 18-29 = {near4}/{n}")
print(" Age row matches Table 3 (0.98, 0.97-0.98), not Table 4 18-29 (0.96, 0.95-0.98).")
print()
def part3():
print("3. Other values checked")
# Discussion 'nearly five times ... Black' vs RR and OR
print(f" Black, firearm model: Table 3 RR 4.12 (3.75-4.52), centre {gmid(3.75,4.52):.3f}; "
f"Suppl. S1 OR 4.74 (4.29-5.24), centre {gmid(4.29,5.24):.3f}; "
f"Discussion 'nearly five times' is closer to the OR (5 - 4.74 = {5-4.74:.2f}; 5 - 4.12 = {5-4.12:.2f})")
# Abstract Native American 2.81
print(f" Native American, all-assault model: abstract 2.81 (2.67-2.97), centre "
f"{gmid(2.67,2.97):.3f}; Table 4 18-29 = 2.81 (2.67, 2.97); Table 3 all ages = 2.68 (2.60, 2.77), "
f"centre {gmid(2.60,2.77):.3f}")
# Results p5: 18-29 all-assault death in ED, upper limit 4.20 vs 4.02
print(f" 18-29 all-assault death in ED: Results p5 3.14 (2.45-4.20), centre {gmid(2.45,4.20):.3f}; "
f"Table 4 3.14 (2.45, 4.02), centre {gmid(2.45,4.02):.3f}")
# Results p5 18-29 firearm in hospital matches Table 4
print(f" 18-29 firearm in hospital: Results p5 3.29 (2.57-4.22) = Table 4; centre {gmid(2.57,4.22):.3f}")
# admission: Results text vs Table 3
print(f" Admitted: Results 19.94 / Table 3 16.50 = {19.94/16.50:.4f} ({100*(19.94/16.50-1):.0f}% higher)")
# internal consistency of the table cells used here: printed estimate vs CI centre
cells = [("T3 In ED", 83.82, 72.38, 97.06), ("T3 In hospital", 3.34, 2.79, 4.00),
("T3 Admit", 16.50, 15.29, 17.80), ("T3 Black", 4.12, 3.75, 4.52),
("T3 Hispanic", 1.87, 1.64, 2.12), ("T3 Other", 1.90, 1.55, 2.33),
("T4 18-29 In ED", 77.63, 62.60, 96.27), ("T4 18-29 In hospital", 3.29, 2.57, 4.22),
("T4 18-29 Admit", 19.94, 18.05, 22.02), ("T4 18-29 Black", 4.16, 3.61, 4.78),
("T4 18-29 Hispanic", 1.78, 1.48, 2.14), ("T4 18-29 Other", 2.01, 1.51, 2.68),
("T4 30-39 In ED", 78.40, 59.30, 103.65), ("T4 40-49 In ED", 92.63, 66.98, 128.10),
("T4 50+ In ED", 110.20, 70.04, 173.38)]
worst = max(abs(p - gmid(lo, hi)) for _, p, lo, hi in cells)
print(" Table cells used here, printed vs sqrt(L*U): " +
"; ".join(f"{k} {p:.2f}/{gmid(lo, hi):.2f}" for k, p, lo, hi in cells))
print(f" Largest |printed - sqrt(L*U)| among these table cells = {worst:.3f} "
f"(abstract 88.82 vs 83.82: {88.82 - gmid(72.38, 97.06):.3f})")
# every version excludes 1
print(" Lower CI limit of every version of the firearm ED-death estimate > 1: "
+ str(all(lo > 1 for *_, lo, hi in ED_DEATH)))
print()
# ------------------------------------------------------------------ figures
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": TXT, "ytick.color": TXT,
"svg.fonttype": "none"})
def fig1():
rows = [ # label, sublabel, point, lo, hi
("Abstract", "“89 times”, RR = 88.82", 88.82, 72.38, 97.06),
("Table 3 (b), all ages", "Poisson RR = 83.82", 83.82, 72.38, 97.06),
("Results para 4", "“firearm injury model … (Table 3)”", 77.63, 62.60, 96.27),
("Table 4, age 18–29 y only", "Poisson RR = 77.63", 77.63, 62.60, 96.27),
("Discussion para 1", "“379.00 … times higher risk”", 379.00, 266.15, 539.7),
("Suppl. Table S1", "logistic odds ratio = 379.00", 379.00, 266.15, 539.71),
]
from matplotlib.lines import Line2D
fig, (ax, zx) = plt.subplots(2, 1, figsize=(7.2, 6.6), facecolor=BG,
gridspec_kw=dict(height_ratios=[6, 1.9], hspace=0.55))
ax.set_facecolor(BG); zx.set_facecolor(BG)
ys = [5 - i for i in range(len(rows))]
off = 0.24 # recomputed centre drawn just below the printed value
for y, (lab, sub, p, lo, hi) in zip(ys, rows):
ax.plot([lo, hi], [y + 0.08, y + 0.08], color=ORANGE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([p], [y + 0.08], "o", ms=8, color=ORANGE, zorder=4)
g = gmid(lo, hi)
ax.plot([g], [y - off], marker="D", ms=8, mfc="none", mec=BLUE, mew=2, zorder=5, ls="none")
his = "539.7" if hi == 539.7 else f"{hi:.2f}" # Discussion prints 539.7
ax.text(1.01, y + 0.08, f"{p:.2f} ({lo:.2f}–{his})", transform=ax.get_yaxis_transform(),
va="center", fontsize=10, color=ORANGE)
ax.text(1.01, y - off, f"{g:.2f}", transform=ax.get_yaxis_transform(),
va="center", fontsize=10, color=BLUE)
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
ax.text(1.07, -0.65, "null (ratio = 1)", fontsize=10, color=SEC, va="center")
ax.set_xscale("log")
ax.set_xlim(0.9, 1000)
ax.set_ylim(-0.95, 5.55)
ax.set_xticks([1, 10, 100, 1000]); ax.set_xticklabels(["1", "10", "100", "1000"])
ax.set_yticks(ys)
ax.set_yticklabels([f"{lab}\n{sub}" for lab, sub, *_ in rows], fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Firearm vs other assault mechanisms, death in the ED (ratio, log scale)")
ax.set_title("A. All printed versions of the estimate (full range, null at 1)", loc="left",
fontsize=10.5, color=TXT, x=-0.6)
# panel B: zoom on the two rows that print the same CI
for yy, p, lab in ((1, 88.82, "Abstract\nRR = 88.82"), (0, 83.82, "Table 3 (b)\nRR = 83.82")):
zx.plot([72.38, 97.06], [yy + 0.0, yy], color=ORANGE, lw=2, solid_capstyle="butt")
for e in (72.38, 97.06):
zx.plot([e, e], [yy - 0.15, yy + 0.15], color=ORANGE, lw=2)
zx.plot([p], [yy], "o", ms=8, color=ORANGE, zorder=4)
zx.plot([gmid(72.38, 97.06)], [yy], marker="D", ms=9, mfc="none", mec=BLUE, mew=2,
ls="none", zorder=5)
zx.set_yticks([1, 0]); zx.set_yticklabels(["Abstract\nRR = 88.82", "Table 3 (b)\nRR = 83.82"],
fontsize=10)
zx.set_xscale("log"); zx.set_xlim(70, 100); zx.set_ylim(-0.5, 1.5)
zx.set_xticks([72.38, 83.82, 88.82, 97.06])
zx.set_xticklabels(["72.38", "83.82", "88.82", "97.06"], fontsize=10)
zx.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
zx.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for sp in ("top", "right", "left"):
zx.spines[sp].set_visible(False)
zx.spines["bottom"].set_color(SEC); zx.tick_params(axis="y", length=0)
zx.set_title("B. Zoom (log scale, 70–100): one printed CI, two printed point estimates",
loc="left", fontsize=10.5, color=TXT, x=-0.6)
zx.text(1.01, 1, f"log distance\nto L {math.log(88.82/72.38):.3f}, to U {math.log(97.06/88.82):.3f}",
transform=zx.get_yaxis_transform(), va="center", fontsize=10, color=SEC)
zx.text(1.01, 0, f"log distance\nto L {math.log(83.82/72.38):.3f}, to U {math.log(97.06/83.82):.3f}",
transform=zx.get_yaxis_transform(), va="center", fontsize=10, color=SEC)
h = [Line2D([0], [0], color=ORANGE, lw=2, marker="o", ms=8,
label="value printed in the paper, with its 95% CI"),
Line2D([0], [0], ls="none", marker="D", ms=8, mfc="none", mec=BLUE, mew=2,
label="recomputed: geometric centre of that CI, √(L×U)")]
fig.legend(handles=h, loc="lower left", bbox_to_anchor=(0.2, 0.0), ncol=1, frameon=False,
fontsize=10, columnspacing=1.2, handletextpad=0.5)
fig.subplots_adjust(left=0.36, right=0.75, top=0.95, bottom=0.12)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
def fig2():
fig = plt.figure(figsize=(7.2, 5.0), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(-0.3, 4.7); ax.axis("off")
ax.text(0.15, 4.5, "Results paragraph 4 says it reports the all-ages firearm model (Table 3).",
fontsize=11, fontweight="bold", va="center")
ax.text(0.15, 4.2, "Its numbers are those of the age 18–29 y column of Table 4.",
fontsize=11, color=SEC, va="center")
cols = [(2.55, "Results para 4\n(text, “Table 3”)", ORANGE),
(4.15, "Table 3 (b)\nfirearm, all ages", TXT),
(5.75, "Table 4 (b)\nfirearm, 18–29 y", TXT)]
top = 3.62
for x, h, c in cols:
ax.text(x, top, h, ha="center", va="center", fontsize=10, fontweight="bold", color=c,
linespacing=1.25)
rows = [r for r in P3_ROWS]
dy = 0.39
y0 = top - 0.55
fmt = lambda t: f"{t[0]:.2f} ({t[1]:.2f}–{t[2]:.2f})"
for i, (q, r, t3, t4) in enumerate(rows):
y = y0 - i * dy
if i % 2 == 0:
ax.add_patch(FancyBboxPatch((0.12, y - dy / 2 + 0.02), 6.96, dy - 0.04,
boxstyle="round,pad=0,rounding_size=0.04",
fc="#f1f0ec", ec="none", zorder=0))
ax.text(0.2, y, q, va="center", fontsize=10)
ax.text(cols[0][0], y, fmt(r), ha="center", va="center", fontsize=10, color=ORANGE,
fontweight="bold")
for (x, _, _), t in ((cols[1], t3), (cols[2], t4)):
same = all(abs(a - b) <= 0.0101 for a, b in zip(r, t))
ax.text(x, y, fmt(t), ha="center", va="center", fontsize=10,
color=BLUE if same else SEC, fontweight="bold" if same else "normal")
yb = y0 - len(rows) * dy + 0.05
six = [r for r in P3_ROWS if r[0] != "Age (per year)"]
e4 = sum(a == b for _, r, t3, t4 in six for a, b in zip(r, t4))
e3 = sum(a == b for _, r, t3, t4 in six for a, b in zip(r, t3))
ax.text(0.2, yb - 0.08,
"Blue bold = equal to the Results-text value to within 0.01 on all three numbers; grey = differs.\n"
f"Six outcome and race rows: {e4} of 18 numbers equal Table 4 18–29 y, the other {18-e4} within 0.01;\n"
f"{e3} of 18 identical to Table 3. Only the age row (last) matches Table 3, not Table 4 18–29 y.",
fontsize=10, color=SEC, va="top", linespacing=1.35)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
if __name__ == "__main__":
check_quotes()
part1()
part2()
part3()
fig1()
fig2()
print("Figures written:", ", ".join(f"fig{i}.{e}" for i in (1, 2) for e in ("svg", "png")))
Abstract's adjusted odds ratios for education and prenatal care differ from the paper's own Results
Lancet Reg Health Am (2026) · doi:10.1016/j.lana.2026.101434 · PMC12995907
Summary
The Abstract reports that higher maternal education (aOR 2·41, 95% CI 2·35–2·48) and receipt of prenatal care (aOR 2·64, 95% CI 2·56–2·73) were associated with documented adequate partner treatment. The paper's national multivariable model, given in the Results text and in Fig. 2, has different values for the same two associations: 1·48 (1·39–1·59) for higher vs primary education and 3·05 (2·92–3·20) for prenatal care. In both cases the two intervals do not overlap. The race estimates in the same Abstract sentence match the Results exactly. Neither Abstract value appears anywhere else in the paper, and the Abstract's education interval is too narrow to be possible for a higher-vs-primary contrast given the Table 1 counts. Both associations keep their direction and statistical significance, but the Abstract overstates the education association, understates the prenatal-care association, and presents the two as about equal, while the Results show prenatal care as much the stronger of the two.
What the paper reports
Abstract, Findings:
Higher maternal education (aOR 2·41, 95% CI 2·35–2·48) and receipt of prenatal care (aOR 2·64, 95% CI 2·56–2·73) were associated with partner treatment, whereas odds were lower among Black (aOR 0·85, 95% CI 0·82–0·89) and Mixed women (aOR 0·81, 95% CI 0·79–0·84) and in the North region.
Abstract, Methods:
Multivariable logistic regression estimated adjusted odds ratios (aORs) and 95% CIs.
Results, paragraph on the national multivariable model:
Multivariable analysis included 214,259 participants, with 4069 exclusions due to missing maternal age. Prenatal care was the factor most strongly associated with adequate partner treatment (adjusted odds ratio [aOR] 3·05, 95% CI 2·92–3·20; p < 0·001).
Higher maternal education remained positively associated with partner treatment (higher vs primary: aOR 1·48, 1·39–1·59; secondary vs primary: aOR 1·23, 1·20–1·27; both p < 0·001)
Compared with White women, lower odds persisted among Black (aOR 0·85, 0·82–0·89; p < 0·001) and Mixed women (aOR 0·81, 0·79–0·84; p < 0·001)
Fig. 2 caption:
Unadjusted ORs are from separate univariable logistic regression models; aORs are from the multivariable logistic regression model including maternal age, race, education, prenatal care, and macro-region.
Fig. 2 itself is an image. Read from the figure file (gr2.jpg): pre-natal care "Yes", unadjusted 3·33 (3·22 – 3·46) and adjusted 3·05 (2·92 – 3·20); "Higher education", unadjusted 1·85 (1·73 – 1·97) and adjusted 1·48 (1·39 – 1·59). Fig. 3 (gr3.jpg), the region-stratified models, gives prenatal care "Yes" aORs of 2·11, 2·58, 3·70, 4·50 and 4·87 and higher-education aORs of 1·39, 1·72, 1·48, 1·38 and 1·41 (North, Northeast, Central-West, Southeast, South).
Discussion, second paragraph:
Prenatal care emerged as the strongest predictor of partner treatment, with approximately threefold higher odds compared with women without prenatal follow-up.
Table 1 (columns: all cases, n = 218,328 | adequate partner treatment, n = 50,609 | no adequate partner treatment, n = 167,719 | p-value), education and prenatal-care rows:
Primary | 94,267 (43·2%) | 19,367 (38·3%) | 74,900 (44·7%) | <0·001
Higher | 4427 (2·03%) | 1431 (2·83%) | 2996 (1·79%) | <0·001
No | 32,162 (14·7%) | 2627 (5·2%) | 29,535 (17·6%) | <0·001
Yes | 180,009 (82·4%) | 46,943 (92·8%) | 133,066 (79·3%) | <0·001
What the paper's own numbers show
Method. I compared each Abstract estimate with every other estimate of the same association in the paper: the Results text, Fig. 2 (adjusted and unadjusted) and Fig. 3 (five regional models). From the Table 1 counts I computed crude odds ratios with Woolf 95% CIs. From each printed 95% CI I recovered the implied standard error of the log odds ratio, SE = ln(upper/lower)/(2 × 1·96), together with the largest SE the two-decimal rounding allows. I then compared those SEs with a lower bound that holds for any maximum-likelihood logistic model fitted to these cases with model-based standard errors. When the model contains the levels of a categorical variable, the fitted probabilities within each level sum to the observed number of events in that level. As a result, the variance of the log odds ratio between two levels can be no smaller than the crude Woolf variance 1/a + 1/b + 1/c + 1/d computed from the level counts, whatever other covariates are in the model. Any subset of the cases, such as the n = 214,259 model sample or a single region, has smaller counts, so the bound only increases. Finally I searched every merge of levels of the model variables in Table 1 for a crude OR near 2·41 or 2·64. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Higher education, adjusted | Abstract | 2·41 (2·35–2·48) | Appears nowhere else in the paper |
| Higher vs primary education, adjusted | Results text; Fig. 2 | 1·48 (1·39–1·59) | Does not overlap the Abstract interval; the Abstract OR is 1·63 times larger (2·24 times on the log scale) |
| Higher vs primary education, crude | Fig. 2 (unadjusted); Table 1 counts | 1·85 (1·73–1·97) | 1·85 (1·73–1·97) from 1431/2996 vs 19,367/74,900, matching Fig. 2, so the Abstract value is not the crude OR either |
| SE(log OR) implied by the education interval | Abstract | 2·35–2·48 | 0·0137 (at most 0·0148 allowing for rounding) |
| SE(log OR) implied by the education interval | Results; Fig. 2 | 1·39–1·59 | 0·0343 |
| Smallest SE(log OR) possible for higher vs primary | Table 1 counts | — | 0·0331. The Abstract interval is 0·41 times this. Even counting only the 1431 treated women with higher education, SE ≥ 0·0264; the Abstract interval would need at least 4570 such women |
| Prenatal care, adjusted | Abstract | 2·64 (2·56–2·73) | Appears nowhere else in the paper |
| Prenatal care yes vs no, adjusted | Results text; Fig. 2 | 3·05 (2·92–3·20) | Does not overlap the Abstract interval; the Abstract OR is 0·87 times the Results value |
| Prenatal care yes vs no, crude | Table 1 counts | — | 3·97 (3·81–4·13) |
| SE(log OR) implied by the prenatal interval | Abstract | 2·56–2·73 | 0·0164 (at most 0·0174 allowing for rounding) |
| SE(log OR) implied by the prenatal interval | Results; Fig. 2 | 2·92–3·20 | 0·0234 |
| Smallest SE(log OR) possible for prenatal yes vs no | Table 1 counts | — | 0·0211. The Abstract interval is 0·78 times this. Even pooling "Ignored/Unknown" with "No", the bound is 0·0182 |
| Regional aORs, prenatal care | Fig. 3 | 2·11, 2·58, 3·70, 4·50, 4·87 | None is 2·64. The closest, Northeast 2·58 (2·37–2·80), has SE 0·0425, about 2.6 times the Abstract's |
| Regional aORs, higher education | Fig. 3 | 1·39, 1·72, 1·48, 1·38, 1·41 | None is near 2·41 |
| Black and Mixed vs White, adjusted | Abstract vs Results | 0·85 (0·82–0·89) and 0·81 (0·79–0·84) in both | Identical, so the Abstract sentence uses the national model for race |
| Relative strength, log OR of prenatal care ÷ log OR of higher education | Abstract vs Results | 2·64 vs 2·41; 3·05 vs 1·48 | 1·10 in the Abstract, 2·84 in the Results |
| Crude OR near 2·41 or 2·64 from any merge of levels | Table 1 (race, region, education, prenatal care) | — | None within 0·06 |
Table 1 is internally consistent: every variable sums to 50,609 treated and 167,719 untreated cases. The Results and Fig. 2 intervals are compatible with the bound (1·04 and 1·11 times it), as they should be. The Abstract intervals are not.
One smaller observation is separate from the claim. The Fig. 2 unadjusted OR for prenatal care "Yes", 3·33 (3·22–3·46), is not the Table 1 crude OR for yes vs no (3·97). It is reproduced exactly by yes vs "No" and "Ignored/Unknown" pooled: 3·33 (3·22–3·46). The "Ignored/Unknown" row in Fig. 2, 2·28 (2·11–2·47), is reproduced by ignored vs no alone. This suggests that the unadjusted "Yes" estimate may use a different reference group from the rest of the block. It does not affect the adjusted estimates.
Figure
Figure 1. Short excerpts from the Abstract (Findings, left) and from the Results text, whose values Fig. 2 repeats (right), for the four associations the Abstract quantifies. Orange arrows join two reported values of the same association that disagree: higher education 2·41 (2·35–2·48) vs 1·48 (1·39–1·59), and prenatal care 2·64 (2·56–2·73) vs 3·05 (2·92–3·20). In each case the intervals do not overlap. Dashed gray lines join values that are identical: Black women 0·85 (0·82–0·89) and Mixed women 0·81 (0·79–0·84). The excerpts are shortened; the full sentences are quoted above.
Figure 2. The standard error of the log odds ratio implied by each printed 95% CI, for higher vs primary education (top) and prenatal care yes vs no (bottom), on one linear axis that starts at zero. Orange diamonds are the Abstract intervals and hollow orange circles the Results / Fig. 2 intervals. The short orange bars show the range of SEs consistent with two-decimal rounding of the printed limits. The blue line in each group is the smallest SE any logistic model fitted to these cases can have, computed from the Table 1 counts (0·0331 and 0·0211). The shaded area to its left is unattainable. Both Results intervals lie just above the bound, and both Abstract intervals lie inside the unattainable area, the education interval at less than half the bound.
Why it matters
The Abstract is the most-read part of the paper and the only place, apart from the Results text and figures, where the paper puts numbers on its main associations. The title, the Research in context panel and the Conclusions describe these associations without numbers. In the Abstract:
- The education association is overstated. 2·41 is 63% larger than the Results' 1·48, and 2·24 times larger on the log-odds scale. The Abstract's interval (2·35–2·48) also suggests a precision that the 4427 women with higher education cannot provide.
- The prenatal-care association is understated. 2·64 is 13% smaller than 3·05. The Discussion's statement of "approximately threefold higher odds" matches the Results, not the Abstract.
- The relative ranking is altered. The Abstract presents education and prenatal care as roughly equal (log-OR ratio 1·10). In the Results, prenatal care is the strongest predictor, with a log OR 2·84 times that of higher education. The Discussion states that "public health prioritization should be guided primarily by effect size", so the size and ranking of these two associations are what a reader would act on.
The direction and statistical significance of both associations are the same in the Abstract and the Results. So are the paper's qualitative conclusions: partner treatment is low, it is more likely with prenatal care and higher education, and it is less likely among Black and Mixed women. The race estimates in the Abstract are correct. The error is in the reported size of two of the four quantified associations in the Abstract.
Explanations considered
- Rounding or typesetting of the same estimate. This does not account for the numbers. The intervals do not overlap for either association, and the differences (2·41 vs 1·48; 2·64 vs 3·05) are far larger than rounding. The publisher-supplied full text archived in PMC and Europe PMC has the same values, so a conversion artifact is excluded.
- The Abstract gives unadjusted odds ratios. This does not account for the numbers. The unadjusted ORs are 1·85 for higher education (Fig. 2 and Table 1) and 3·33 (Fig. 2) or 3·97 (Table 1) for prenatal care. The Abstract also labels its values "aOR".
- The Abstract gives a regional estimate. This does not account for the numbers. None of the five regional aORs in Fig. 3 equals 2·41 or 2·64, and every regional interval is far wider than the Abstract's. The SE bound also applies to any region, since each is a subset of the cases.
- The Abstract uses a different contrast, such as secondary-or-higher vs primary education. The paper does not describe one. An interval as narrow as 2·35–2·48 is possible only for a contrast between large groups, such as secondary vs primary (bound 0·0119). The paper reports that contrast as 1·23 (1·20–1·27), and the crude OR for secondary-or-higher vs primary is 1·47. For prenatal care, even pooling "Ignored/Unknown" with "No" gives a bound (0·0182) above the Abstract's largest possible SE (0·0174). Among the model variables in Table 1, no merge of levels gives a crude OR within 0·06 of either value. Arbitrary merges of the timing-of-diagnosis variable, which is not in the model, come within 0·06. All of them have SE of at least 0·0189, so they cannot produce the Abstract intervals either.
- The Abstract comes from a different model or an earlier version of the analysis. The paper gives no information on this, so it cannot be checked. Any logistic model fitted to these 218,328 cases, or to a subset of them, with model-based standard errors is excluded by the SE bound for the education interval, and also for the prenatal interval. A model fitted to a larger set of cases is not excluded by the bound. Such a model would have to count more women with documented partner treatment than the 50,609 in Table 1 (for example, at least 4570 treated women with higher education rather than 1431). The paper's outcome excludes the 78,734 records with missing partner-treatment status. The paper does not state how its CIs were computed. Robust (sandwich) standard errors are not mentioned, and the Results intervals are consistent with model-based ones.
- The Abstract values are correct and the Results and Fig. 2 are wrong. This is unlikely. Fig. 2, the Results text and the Discussion agree with each other. The Results and Fig. 2 intervals are compatible with the Table 1 counts, and the Abstract's education interval is not.
Status
- Version of record. The publisher's own pages could not be read: doi.org redirected to linkinghub.elsevier.com, which returned only a redirect stub, and thelancet.com and sciencedirect.com returned HTTP 403. The same text was checked in three archived copies of the publisher's full text: the PMC article page, the Europe PMC full-text XML (publisher-supplied JATS), and the local PMC archive. All three carry the Abstract values 2·41 (2·35–2·48) and 2·64 (2·56–2·73) and the Results values 3·05 (2·92–3·20) and 1·48 (1·39–1·59). In the text, 2·41 and 2·64 occur only in the Abstract. Every quote above was checked verbatim against the PMC-archived text, except the figure values, which were read from the figure images.
- Figures and supplements. I read the figure images Fig. 2 (gr2.jpg) and Fig. 3 (gr3.jpg) from the Europe PMC supplementary-files bundle. Neither contains 2·41 (2·35–2·48) or 2·64 (2·56–2·73). The only supplement, mmc1.pdf, holds Supplementary Table 1 (variable definitions and SINAN field codes) and Supplementary Table 2 (included vs excluded cases), with no regression output; it does not resolve the discrepancy. No code or analysis dataset is deposited. The data are public SINAN/SINASC records, but refitting the model was out of scope.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI (update-to and updated-by absent, relation empty). PMC and Europe PMC list no correction or comment links (PMID 41858869), and the PMC page carries only the standard editorial note on maps and affiliations. The PubPeer API was blocked by an automated-access challenge, so PubPeer could not be checked directly; a web search for the title together with correction, erratum or PubPeer found nothing.
- Sources. PMC article PMC12995907; Europe PMC full-text XML and supplementary files (gr2.jpg, gr3.jpg, mmc1.pdf); Crossref API; Europe PMC search; PubPeer; general web search.
Suggested correction
- Replace the Abstract's education and prenatal-care estimates with the values from the national multivariable model, stating the education contrast explicitly. For example: "Higher maternal education (higher vs primary: aOR 1·48, 95% CI 1·39–1·59) and receipt of prenatal care (aOR 3·05, 95% CI 2·92–3·20) were associated with partner treatment".
- If 2·41 (2·35–2·48) and 2·64 (2·56–2·73) come from another analysis, identify that analysis (model, sample, contrast and CI method) in the Methods and Results, and explain why the Abstract reports it instead of the main model.
- Minor, separate: check the reference group of the Fig. 2 unadjusted OR for prenatal care "Yes" (3·33), which matches yes vs "No" and "Ignored/Unknown" pooled rather than yes vs no (3·97), and state the reference in the figure if it differs from the adjusted estimate's.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12995907 (Lancet Reg Health Am 2026; doi:10.1016/j.lana.2026.101434)
# Abstract aORs for education and prenatal care vs the paper's own Results, Fig. 2, Fig. 3 and Table 1.
# Run: python recompute.py.txt (needs numpy + matplotlib)
# Prints every recomputed number used in writeup.md and writes fig1/fig2 (.svg, .png) and
# five candidate figure designs (candidates/cand1..cand5 .svg/.png) into this file's folder.
import math, itertools, os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
from matplotlib.patches import FancyBboxPatch
from matplotlib.transforms import blended_transform_factory
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z = 1.959964
# ---------------------------------------------------------------- inputs from the paper
# Table 1 counts: (adequate partner treatment, no adequate partner treatment)
T = {
"race": {"White": (14578, 39887), "Black": (4429, 16105), "Asian": (181, 584),
"Mixed": (27822, 98612), "Indigenous": (213, 544), "Ignored": (3386, 11987)},
"region": {"North": (4196, 15681), "Northeast": (14302, 50668), "Southeast": (20359, 71759),
"South": (7788, 20561), "Center-west": (3964, 9050)},
"edu": {"Primary": (19367, 74900), "Secondary": (17804, 47544), "Higher": (1431, 2996),
"Ignored": (12007, 42279)},
"pnc": {"No": (2627, 29535), "Yes": (46943, 133066), "Ignored": (1039, 5118)},
"timing": {"During prenatal care": (38593, 83773), "At delivery/curettage": (8290, 64508),
"After delivery": (2243, 13514), "Not performed": (217, 1286), "Ignored": (1266, 4638)},
}
N_TREATED, N_UNTREATED = 50609, 167719
# Reported estimates: (point, lo, hi)
ABS_EDU, ABS_PNC = (2.41, 2.35, 2.48), (2.64, 2.56, 2.73) # Abstract, Findings
RES_EDU, RES_PNC = (1.48, 1.39, 1.59), (3.05, 2.92, 3.20) # Results p5 and Fig. 2 (aOR)
RES_SEC = (1.23, 1.20, 1.27) # Results p5, secondary vs primary
FIG2_UNADJ_EDU, FIG2_UNADJ_PNC = (1.85, 1.73, 1.97), (3.33, 3.22, 3.46) # Fig. 2 unadjusted OR
FIG2_UNADJ_PNC_IGN = (2.28, 2.11, 2.47) # Fig. 2 unadjusted, PNC ignored vs no
ABS_RACE = {"Black": (0.85, 0.82, 0.89), "Mixed": (0.81, 0.79, 0.84)}
RES_RACE = {"Black": (0.85, 0.82, 0.89), "Mixed": (0.81, 0.79, 0.84)}
REGIONS = ["North", "Northeast", "Central-West", "Southeast", "South"] # Fig. 3 order
FIG3_PNC = [(2.11, 1.86, 2.40), (2.58, 2.37, 2.80), (3.70, 3.05, 4.48), (4.50, 4.11, 4.94), (4.87, 4.21, 5.62)]
FIG3_EDU = [(1.39, 1.12, 1.72), (1.72, 1.49, 1.99), (1.48, 1.16, 1.87), (1.38, 1.23, 1.55), (1.41, 1.21, 1.64)]
def orci(a, b, c, d):
"""Crude OR of (a treated, b untreated) vs reference (c, d), Woolf 95% CI and SE(log OR)."""
o = (a / b) / (c / d)
se = math.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
return o, o * math.exp(-Z * se), o * math.exp(Z * se), se
def se_ci(lo, hi):
return math.log(hi / lo) / (2 * Z)
def se_ci_maxround(lo, hi, dp=2):
"""Largest SE consistent with CI limits printed to dp decimals."""
h = 0.5 * 10 ** (-dp)
return math.log((hi + h) / (lo - h)) / (2 * Z)
def f(x): return f"{x:.2f}"
print("=" * 78)
print("1. Table 1 column totals")
for v, d in T.items():
print(f" {v:7s}: treated {sum(x[0] for x in d.values()):6d} (paper {N_TREATED}), "
f"untreated {sum(x[1] for x in d.values()):6d} (paper {N_UNTREATED})")
print("\n2. Crude ORs from Table 1 counts (Woolf 95% CI)")
crude = {}
pairs = {
"Higher vs Primary": (T["edu"]["Higher"], T["edu"]["Primary"]),
"Secondary vs Primary": (T["edu"]["Secondary"], T["edu"]["Primary"]),
"Edu Ignored vs Primary": (T["edu"]["Ignored"], T["edu"]["Primary"]),
"Secondary+Higher vs Primary": ((T["edu"]["Secondary"][0] + T["edu"]["Higher"][0],
T["edu"]["Secondary"][1] + T["edu"]["Higher"][1]), T["edu"]["Primary"]),
"PNC Yes vs No": (T["pnc"]["Yes"], T["pnc"]["No"]),
"PNC Ignored vs No": (T["pnc"]["Ignored"], T["pnc"]["No"]),
"PNC Yes vs (No+Ignored)": (T["pnc"]["Yes"], (T["pnc"]["No"][0] + T["pnc"]["Ignored"][0],
T["pnc"]["No"][1] + T["pnc"]["Ignored"][1])),
"Black vs White": (T["race"]["Black"], T["race"]["White"]),
"Mixed vs White": (T["race"]["Mixed"], T["race"]["White"]),
}
for k, (x, r) in pairs.items():
crude[k] = orci(*x, *r)
o, lo, hi, se = crude[k]
print(f" {k:28s} OR {o:.3f} ({lo:.2f}-{hi:.2f}) SE(log OR) {se:.4f}")
print(" Fig. 2 unadjusted: higher edu 1.85 (1.73-1.97); PNC yes 3.33 (3.22-3.46); PNC ignored 2.28 (2.11-2.47)")
print(" -> Fig. 2's unadjusted 'Yes' OR equals Yes vs (No+Ignored), not Yes vs No (side observation)")
print("\n3. SE(log OR) implied by each reported 95% CI (and largest SE allowed by 2-dp rounding)")
rep = [("Abstract, higher education", ABS_EDU), ("Results/Fig. 2, higher vs primary", RES_EDU),
("Abstract, prenatal care", ABS_PNC), ("Results/Fig. 2, prenatal yes vs no", RES_PNC),
("Results, secondary vs primary", RES_SEC), ("Fig. 2 unadjusted, higher edu", FIG2_UNADJ_EDU),
("Fig. 2 unadjusted, prenatal yes", FIG2_UNADJ_PNC)]
for lab, (p, lo, hi) in rep:
print(f" {lab:36s} {p:.2f} ({lo:.2f}-{hi:.2f}): SE {se_ci(lo, hi):.4f} "
f"(max with rounding {se_ci_maxround(lo, hi):.4f}); geometric midpoint {math.sqrt(lo*hi):.3f}")
print("\n4. Lower bounds on SE(log OR) for any logistic model fitted to these cases")
# For a logistic model that contains level indicators for the variable, the score equations make the
# fitted probabilities in each level sum to the observed events there, so the information carried by
# level A is at most n_A*pbar*(1-pbar) = 1/(1/a + 1/b). By Cauchy-Schwarz on the inverse information,
# Var(beta_A - beta_B) >= 1/I_A + 1/I_B >= (1/a + 1/b + 1/c + 1/d): the crude Woolf variance.
# Any subset of the 218,328 cases (the n = 214,259 model sample, a single region) has smaller
# counts, so the bound only grows. Weaker but more general: Var >= 1/a, a = treated cases in level.
b_edu = crude["Higher vs Primary"][3]
b_pnc = crude["PNC Yes vs No"][3]
b_edu_a = math.sqrt(1 / T["edu"]["Higher"][0])
b_pnc_a = math.sqrt(1 / T["pnc"]["No"][0])
print(f" Higher vs primary: SE >= {b_edu:.4f} (crude Woolf); SE >= {b_edu_a:.4f} from 1431 treated alone")
print(f" Prenatal yes vs no: SE >= {b_pnc:.4f} (crude Woolf); SE >= {b_pnc_a:.4f} from 2627 treated (No) alone")
se_ae, se_ap = se_ci(*ABS_EDU[1:]), se_ci(*ABS_PNC[1:])
se_re, se_rp = se_ci(*RES_EDU[1:]), se_ci(*RES_PNC[1:])
print(f" Abstract education SE {se_ae:.4f} = {se_ae/b_edu:.2f} x the bound; Results {se_re:.4f} = {se_re/b_edu:.2f} x")
print(f" Abstract prenatal SE {se_ap:.4f} = {se_ap/b_pnc:.2f} x the bound; Results {se_rp:.4f} = {se_rp/b_pnc:.2f} x")
need_a = 1 / se_ci_maxround(*ABS_EDU[1:]) ** 2
print(f" To allow the Abstract education CI, the higher-education level would need >= {need_a:.0f} "
f"treated cases (Table 1: 1431)")
need_p = 1 / se_ci_maxround(*ABS_PNC[1:]) ** 2
print(f" To allow the Abstract prenatal CI, the prenatal 'No' level would need >= {need_p:.0f} "
f"treated cases (Table 1: 2627)")
print(" Fig. 2 unadjusted prenatal SE "
f"{se_ci(*FIG2_UNADJ_PNC[1:]):.4f} vs bound for Yes vs (No+Ignored) {crude['PNC Yes vs (No+Ignored)'][3]:.4f}")
print("\n5. Size of the discrepancies")
for lab, a, r in [("Education", ABS_EDU, RES_EDU), ("Prenatal", ABS_PNC, RES_PNC)]:
print(f" {lab}: Abstract/Results OR ratio {a[0]/r[0]:.3f}; log OR {math.log(a[0]):.3f} vs "
f"{math.log(r[0]):.3f} (ratio {math.log(a[0])/math.log(r[0]):.2f}); CIs overlap: {a[1] <= r[2] and r[1] <= a[2]}")
print(f" Abstract: prenatal/education log-OR ratio {math.log(ABS_PNC[0])/math.log(ABS_EDU[0]):.2f}; "
f"Results: {math.log(RES_PNC[0])/math.log(RES_EDU[0]):.2f}")
for k in ABS_RACE:
print(f" {k}: Abstract {ABS_RACE[k]} vs Results {RES_RACE[k]} -> identical: {ABS_RACE[k] == RES_RACE[k]}")
print("\n6. Does 2.41 or 2.64 match any other estimate in the paper?")
for lab, vals in [("Fig. 3 prenatal", FIG3_PNC), ("Fig. 3 higher education", FIG3_EDU)]:
for reg, (p, lo, hi) in zip(REGIONS, vals):
print(f" {lab:24s} {reg:12s} {p:.2f} ({lo:.2f}-{hi:.2f}) SE {se_ci(lo, hi):.4f}")
def search(vars_):
hits = []
for v in vars_:
d = T[v]; keys = list(d)
for r1 in range(1, len(keys)):
for A in itertools.combinations(keys, r1):
rest = [k for k in keys if k not in A]
for r2 in range(1, len(rest) + 1):
for B in itertools.combinations(rest, r2):
a = sum(d[k][0] for k in A); b = sum(d[k][1] for k in A)
c = sum(d[k][0] for k in B); dd = sum(d[k][1] for k in B)
for (o, lo, hi, se), lab in ((orci(a, b, c, dd), (A, B)), (orci(c, dd, a, b), (B, A))):
for tgt in (2.41, 2.64):
if abs(o - tgt) < 0.06:
hits.append((v, " + ".join(lab[0]), "vs", " + ".join(lab[1]), round(o, 3), round(se, 4)))
return list(dict.fromkeys(hits))
h_model = search(["race", "region", "edu", "pnc"])
print(" Exhaustive crude-OR search over every merge of levels of the model variables in Table 1 (race, region,"
f" education, prenatal care), both directions, within 0.06 of 2.41 or 2.64: {h_model if h_model else 'none'}")
h_tim = search(["timing"])
print(f" Same search on timing of diagnosis (not a model covariate): {len(h_tim)} hits, all arbitrary merges, all with SE(log OR) >= 0.0189:")
for h in h_tim:
print(" ", h)
o, lo, hi, se = orci(*T["timing"]["During prenatal care"],
sum(v[0] for k, v in T["timing"].items() if k != "During prenatal care"),
sum(v[1] for k, v in T["timing"].items() if k != "During prenatal care"))
print(f" Timing, diagnosis during prenatal care vs all others: OR {o:.2f} ({lo:.2f}-{hi:.2f})")
# ---------------------------------------------------------------- figure style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
LW, MS = 1.5, 8
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "axes.edgecolor": INK2, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK, "text.color": INK, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg")
fig.savefig(p + ".png", dpi=200)
plt.close(fig)
def style_axis(ax):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
def fmt(t): # 2.41 (2.35–2.48) with the paper's middle-dot decimals
return f"{t[0]:.2f} ({t[1]:.2f}–{t[2]:.2f})".replace(".", "·")
# ---------------------------------------------------------------- candidate 1: dot-and-interval
def cand_dot_interval(paths):
ce, cp = crude["Higher vs Primary"], crude["PNC Yes vs No"]
groups = [
("Higher vs primary education", [
("Abstract aOR", ABS_EDU, "abs"), ("Results text and Fig. 2 aOR", RES_EDU, "body"),
("Fig. 2 unadjusted OR", FIG2_UNADJ_EDU, "body"),
("Crude OR from Table 1 counts", ce[:3], "rec")]),
("Prenatal care, yes vs no", [
("Abstract aOR", ABS_PNC, "abs"), ("Results text and Fig. 2 aOR", RES_PNC, "body"),
("Fig. 2 unadjusted OR", FIG2_UNADJ_PNC, "body"),
("Crude OR from Table 1 counts", cp[:3], "rec")]),
]
fig, ax = plt.subplots(figsize=(7.2, 4.4))
fig.subplots_adjust(left=0.31, right=0.98, top=0.87, bottom=0.12)
y, yt, yl = 0, [], []
for gname, rows in groups:
ax.text(0.01, y + 0.15, gname, transform=blended_transform_factory(fig.transFigure, ax.transData),
ha="left", va="center", fontsize=10.5, fontweight="bold")
y -= 1
for lab, (p, lo, hi), kind in rows:
col = BLUE if kind == "rec" else ORANGE
mfc = BG if kind == "body" else col
mk = "s" if kind == "rec" else ("D" if kind == "abs" else "o")
ax.plot([lo, hi], [y, y], color=col, lw=LW, solid_capstyle="butt")
ax.plot(p, y, marker=mk, ms=MS, mfc=mfc, mec=col, mew=1.5, color=col)
ax.text(hi * 1.03, y, fmt((p, lo, hi)), va="center", fontsize=10, color=INK)
yt.append(y); yl.append(lab)
y -= 1
y -= 0.6
ax.axvline(1, color=INK2, lw=0.8)
ax.set_xscale("log")
ax.set_xlim(0.9, 6.5)
ticks = [1, 1.5, 2, 3, 4, 5]
ax.set_xticks(ticks); ax.set_xticklabels([str(t).replace(".", "·") for t in ticks])
ax.minorticks_off()
ax.set_yticks(yt); ax.set_yticklabels(yl, fontsize=10)
ax.set_ylim(y + 0.6, 0.8)
ax.set_xlabel("Odds ratio (log scale); vertical line at 1 = no association")
style_axis(ax)
leg = [Line2D([], [], marker="D", color=ORANGE, mfc=ORANGE, ms=MS, lw=LW, label="Reported in the Abstract"),
Line2D([], [], marker="o", color=ORANGE, mfc=BG, mec=ORANGE, ms=MS, lw=LW, label="Reported in Results / Fig. 2"),
Line2D([], [], marker="s", color=BLUE, mfc=BLUE, ms=MS, lw=LW, label="Recomputed from Table 1")]
fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=3, frameon=False,
fontsize=10, handlelength=2.0, columnspacing=1.0)
save(fig, *paths)
# ---------------------------------------------------------------- candidate 2: two text panels
def cand_text_panels(paths):
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.6); ax.axis("off")
L0, L1, R0, R1 = 0.05, 2.6, 4.6, 7.15
top, bot = 4.05, 0.35
for x0, x1, title, sub in [(L0, L1, "Abstract", "Findings"),
(R0, R1, "Results text and Fig. 2", "national model, n = 214,259")]:
ax.add_patch(FancyBboxPatch((x0, bot), x1 - x0, top - bot, boxstyle="round,pad=0,rounding_size=0.06",
fc="#ffffff", ec=INK2, lw=1.0))
ax.text(x0 + 0.14, top + 0.33, title, fontsize=11.5, fontweight="bold", va="center")
ax.text(x0 + 0.14, top + 0.1, sub, fontsize=10, color=INK2, va="center")
rows = [
("Higher maternal education", "aOR 2·41, 95% CI 2·35–2·48",
"Higher vs primary education", "aOR 1·48, 1·39–1·59", "conflict"),
("Receipt of prenatal care", "aOR 2·64, 95% CI 2·56–2·73",
"Prenatal care (yes vs no)", "aOR 3·05, 95% CI 2·92–3·20", "conflict"),
("Black women", "aOR 0·85, 95% CI 0·82–0·89",
"Black vs White women", "aOR 0·85, 0·82–0·89", "same"),
("Mixed women", "aOR 0·81, 95% CI 0·79–0·84",
"Mixed vs White women", "aOR 0·81, 0·79–0·84", "same"),
]
ys = [3.55, 2.65, 1.65, 0.85]
notes = {0: ["Intervals do not overlap", "Abstract CI is narrower than", "any model on these cases allows"],
1: ["Intervals do not overlap", "Abstract CI is narrower than", "any model on these cases allows"]}
for i, ((ll, lv, rl, rv, kind), yy) in enumerate(zip(rows, ys)):
for x0, lab, val in [(L0, ll, lv), (R0, rl, rv)]:
ax.text(x0 + 0.14, yy + 0.15, lab, fontsize=10, color=INK2, va="center")
ax.text(x0 + 0.14, yy - 0.13, val, fontsize=10.5, fontweight="bold" if kind == "conflict" else "normal",
va="center")
if kind == "conflict":
ax.annotate("", xy=(R0 - 0.04, yy), xytext=(L1 + 0.04, yy),
arrowprops=dict(arrowstyle="<->", color=ORANGE, lw=LW))
ax.text((L1 + R0) / 2, yy + 0.2, "≠", ha="center", va="center", fontsize=16, color=INK)
ax.text((L1 + R0) / 2, yy - 0.2, "\n".join(notes[i][:1]), ha="center", va="top", fontsize=10, color=INK)
else:
ax.plot([L1 + 0.04, R0 - 0.04], [yy, yy], color=INK2, lw=1.0, ls=(0, (3, 2)))
ax.text((L1 + R0) / 2, yy + 0.14, "identical", ha="center", va="bottom", fontsize=10, color=INK2)
ax.plot([L0 + 0.14, R1 - 0.14], [2.12, 2.12], color=GRID, lw=1.0)
ax.text(3.6, 0.12, "Orange arrows: the two reported values of one estimate disagree. "
"Dashed gray: the values agree.", ha="center", va="center", fontsize=10, color=INK2)
save(fig, *paths)
# ---------------------------------------------------------------- candidate 3: interval width vs bound
def cand_se_bound(paths):
rows = [("Higher vs primary education", ABS_EDU, RES_EDU, b_edu),
("Prenatal care, yes vs no", ABS_PNC, RES_PNC, b_pnc)]
fig, ax = plt.subplots(figsize=(7.2, 4.0))
fig.subplots_adjust(left=0.33, right=0.97, top=0.86, bottom=0.13)
yt, yl = [], []
y = 0
for gname, a, r, bnd in rows:
ax.text(0.01, y + 0.75, gname, transform=blended_transform_factory(fig.transFigure, ax.transData),
ha="left", va="center", fontsize=10.5, fontweight="bold")
ax.fill_betweenx([y - 1.45, y + 0.35], 0, bnd, color="#eeedea", lw=0, zorder=0)
ax.plot([bnd, bnd], [y - 1.45, y + 0.35], color=BLUE, lw=LW, zorder=2)
ax.text(bnd, y + 0.42, f"smallest SE possible: {bnd:.4f}".replace(".", "·"), color=INK,
fontsize=10, va="bottom", ha="center")
for lab, t, filled, dy in [("Abstract", a, True, 0), ("Results / Fig. 2", r, False, -1)]:
s, smax = se_ci(*t[1:]), se_ci_maxround(*t[1:])
smin = math.log((t[2] - 0.005) / (t[1] + 0.005)) / (2 * Z)
ax.plot([smin, smax], [y + dy, y + dy], color=ORANGE, lw=LW)
ax.plot(s, y + dy, "D" if filled else "o", ms=MS, color=ORANGE, mfc=ORANGE if filled else BG, mew=1.5)
ax.text(smax + 0.0006, y + dy, f"{s:.4f}".replace(".", "·"), va="center", fontsize=10)
yt.append(y + dy); yl.append(f"{lab}, CI " + f"{t[1]:.2f}–{t[2]:.2f}".replace(".", "·"))
ax.text(0.0005, y - 1.42, "below the bound", fontsize=10, color=INK2, va="bottom")
y -= 2.6
ax.set_xlim(0, 0.05)
ax.set_ylim(y + 0.9, 1.25)
ax.set_yticks(yt); ax.set_yticklabels(yl)
xt = [0, 0.01, 0.02, 0.03, 0.04, 0.05]
ax.set_xticks(xt); ax.set_xticklabels([f"{v:.2f}".replace(".", "·") for v in xt])
ax.set_xlabel("Standard error of log(odds ratio) implied by the printed 95% CI")
style_axis(ax)
leg = [Line2D([], [], marker="D", color=ORANGE, mfc=ORANGE, ms=MS, lw=LW, label="Abstract (bar: rounding range)"),
Line2D([], [], marker="o", color=ORANGE, mfc=BG, ms=MS, lw=LW, label="Results / Fig. 2"),
Line2D([], [], color=BLUE, lw=LW, label="Bound from Table 1 counts")]
fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=3, frameon=False, fontsize=10,
handlelength=1.8, columnspacing=1.0)
save(fig, *paths)
# ---------------------------------------------------------------- candidate 4: every estimate in the paper
def cand_all_estimates(paths):
ce, cp = crude["Higher vs Primary"], crude["PNC Yes vs No"]
groups = [
("Higher vs primary education", ABS_EDU,
[("Results / Fig. 2 aOR", RES_EDU), ("Fig. 2 unadjusted OR", FIG2_UNADJ_EDU)] +
[(f"Fig. 3 aOR, {r}", t) for r, t in zip(REGIONS, FIG3_EDU)], ce[:3]),
("Prenatal care, yes vs no", ABS_PNC,
[("Results / Fig. 2 aOR", RES_PNC), ("Fig. 2 unadjusted OR", FIG2_UNADJ_PNC)] +
[(f"Fig. 3 aOR, {r}", t) for r, t in zip(REGIONS, FIG3_PNC)], cp[:3]),
]
fig, ax = plt.subplots(figsize=(7.2, 6.6))
fig.subplots_adjust(left=0.32, right=0.78, top=0.93, bottom=0.08)
y, yt, yl = 0, [], []
for gname, a, body, rc in groups:
ax.text(0.01, y + 0.1, gname, transform=blended_transform_factory(fig.transFigure, ax.transData),
ha="left", va="center", fontsize=10.5, fontweight="bold")
y -= 1
y0 = y
rows = [("Abstract aOR", a, "abs")] + [(l, t, "body") for l, t in body] + [("Crude OR, Table 1 counts", rc, "rec")]
for lab, (p, lo, hi), kind in rows:
col = BLUE if kind == "rec" else ORANGE
mk = {"abs": "D", "body": "o", "rec": "s"}[kind]
ax.plot([lo, hi], [y, y], color=col, lw=LW)
ax.plot(p, y, mk, ms=MS, color=col, mfc=BG if kind == "body" else col, mew=1.5)
ax.text(1.02, y, fmt((p, lo, hi)), va="center", ha="left", fontsize=10, transform=ax.get_yaxis_transform())
yt.append(y); yl.append(lab); y -= 1
ax.plot([a[0], a[0]], [y + 0.6, y0 + 0.4], color=ORANGE, lw=0.8, ls=(0, (3, 2)), zorder=0)
y -= 0.6
ax.axvline(1, color=INK2, lw=0.8)
ax.set_xscale("log"); ax.set_xlim(0.9, 6.0)
ticks = [1, 1.5, 2, 3, 4, 5]
ax.set_xticks(ticks); ax.set_xticklabels([str(t).replace(".", "·") for t in ticks]); ax.minorticks_off()
ax.set_yticks(yt); ax.set_yticklabels(yl)
ax.set_ylim(y + 0.6, 0.6)
ax.set_xlabel("Odds ratio (log scale); dashed line = Abstract value")
style_axis(ax)
leg = [Line2D([], [], marker="D", color=ORANGE, mfc=ORANGE, ms=MS, lw=LW, label="Abstract"),
Line2D([], [], marker="o", color=ORANGE, mfc=BG, ms=MS, lw=LW, label="Results, Fig. 2, Fig. 3"),
Line2D([], [], marker="s", color=BLUE, mfc=BLUE, ms=MS, lw=LW, label="Recomputed from Table 1")]
fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=3, frameon=False, fontsize=10)
save(fig, *paths)
# ---------------------------------------------------------------- candidate 5: slope chart Abstract -> Results
def cand_slope(paths):
items = [("Prenatal care", ABS_PNC, RES_PNC), ("Higher education", ABS_EDU, RES_EDU),
("Black women", ABS_RACE["Black"], RES_RACE["Black"]),
("Mixed women", ABS_RACE["Mixed"], RES_RACE["Mixed"])]
fig, ax = plt.subplots(figsize=(7.2, 4.6))
fig.subplots_adjust(left=0.1, right=0.98, top=0.88, bottom=0.1)
xa, xr = 0, 1
dy_a = {"Prenatal care": 1.035, "Higher education": 0.965, "Black women": 1.02, "Mixed women": 0.98}
dy_r = {"Prenatal care": 1.0, "Higher education": 1.0, "Black women": 1.02, "Mixed women": 0.98}
for lab, a, r in items:
changed = a != r
col = ORANGE if changed else INK2
ax.plot([xa, xr], [a[0], r[0]], color=col, lw=LW, ls="-" if changed else (0, (3, 2)))
for x, t in [(xa, a), (xr, r)]:
ax.plot([x, x], [t[1], t[2]], color=col, lw=LW + 1.5, solid_capstyle="butt")
ax.plot(x, t[0], "o", ms=MS, color=col, mfc=col if changed else BG, mew=1.5)
ax.text(xa - 0.06, a[0] * dy_a[lab], f"{lab} {a[0]:.2f}".replace(".", "·"), ha="right", va="center", fontsize=10)
ax.text(xr + 0.06, r[0] * dy_r[lab], f"{r[0]:.2f} {lab}".replace(".", "·"), ha="left", va="center", fontsize=10)
ax.set_yscale("log"); ax.set_ylim(0.7, 3.6)
yt = [0.75, 1, 1.5, 2, 2.5, 3, 3.5]
ax.set_yticks(yt); ax.set_yticklabels([str(v).replace(".", "·") for v in yt]); ax.minorticks_off()
ax.axhline(1, color=INK2, lw=0.8)
ax.set_xlim(-0.95, 1.95)
ax.set_xticks([xa, xr]); ax.set_xticklabels(["Abstract", "Results text / Fig. 2"], fontsize=10.5, fontweight="bold")
ax.set_ylabel("Adjusted odds ratio (log scale)")
for s in ("top", "right", "bottom"):
ax.spines[s].set_visible(False)
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.tick_params(axis="x", length=0)
leg = [Line2D([], [], color=ORANGE, lw=LW, marker="o", ms=MS, label="Reported values differ"),
Line2D([], [], color=INK2, lw=LW, ls=(0, (3, 2)), marker="o", mfc=BG, ms=MS, label="Reported values identical")]
fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=2, frameon=False, fontsize=10)
save(fig, *paths)
cand_dot_interval([os.path.join(CAND, "cand1_dot_interval")])
cand_text_panels([os.path.join(CAND, "cand2_text_panels"), os.path.join(HERE, "fig1")])
cand_se_bound([os.path.join(CAND, "cand3_se_bound"), os.path.join(HERE, "fig2")])
cand_all_estimates([os.path.join(CAND, "cand4_all_estimates")])
cand_slope([os.path.join(CAND, "cand5_slope")])
print("\nFigures written: fig1 (= candidates/cand2), fig2 (= candidates/cand3), candidates/cand1..cand5")
Abstract effect sizes for crowding, ageing and primary care coverage disagree with the paper's Table 2
Lancet Reg Health Am (2026) · doi:10.1016/j.lana.2026.101572 · PMC13382474
Summary
The abstract reports three effects from "the final spatial GAMLSS model", each per one percentage-point increase. It gives +3.25% expected tuberculosis (TB) mortality for household crowding, +5.92% for the population aged 60 or older, and −0.30% for primary health care (PHC) coverage. The paper's Table 2 reports that model's coefficients: 0.0211, 0.0432 and −0.0025. Converted with the formula the Results themselves use, 100·(exp(β)−1), these give +2.13%, +4.41% and −0.25%. The Results text prints exactly those three values. So the abstract states each effect larger than the paper's own estimate, by 52%, 34% and 20%. No coefficient in Table 2 or in the supplement's sensitivity models (Table S3) reproduces all three abstract figures. The direction and significance of each association are unchanged, but the abstract's headline magnitudes do not match the model they cite.
What the paper reports
The abstract:
In the final spatial GAMLSS model, proportions of household crowding (>2 residents/bedroom) and elderly population (>59 years) were associated with 3.25% and 5.92% increases in expected TB mortality for each one percentage-point increase, respectively. In contrast, each one percentage-point increase in primary health care coverage was associated with a 0.30% decrease in expected TB mortality.
— Abstract, Findings
The Results, describing the same model:
Table 2 presents the final spatial GAMLSS model associated with the outcome at p-value <0.01.
— Results, "Final association model", paragraph 1
Each additional month was associated with a 0.48% [ (exp (0.0048) -1) ∗ 100] increase in the expected number of municipal-monthly TB deaths.
— Results, "Final association model", paragraph 2 (this states how coefficients are converted to percentages)
Among the contextual variables, each one percentage-point increase in the proportion of households with more than two residents per bedroom was associated with a 2.13% increase in expected TB deaths. Similarly, each one percentage-point increase in the proportion of residents aged 60 years or older was associated with a 4.41% increase,
— Results, "Final association model", paragraph 3
Each one percentage-point increase in coverage was associated with a 0.25% decrease in expected TB deaths.
— Results, "Final association model", paragraph 4
Table 2 ("Parametric covariate estimates from the spatial GAMLSS model for monthly TB deaths in São Paulo, Brazil, 2020–2024"). The columns are variable, coded variable, estimate, standard error, exponential of the estimate and p-value:
Households with more than 2 residents per bedroom (%) | prop_mais2_dorm | 0.0211 | 0.0073 | 1.0213 | =0.0041
Population aged 60 years or older (%) | prop_mais59 | 0.0432 | 0.0077 | 1.0442 | <0.0001
Primary health care coverage (%) | cobertura_aps | -0.0025 | 0.0007 | 0.9975 | =0.0002
— Table 2
I checked each quotation against the article as archived in PubMed Central (src/show.py PMC13382474). Each occurs exactly once. recompute.py.txt repeats that check.
What the paper's own numbers show
Method. The model uses a negative binomial type I distribution with a log link and an offset of log municipal population. A one-unit increase in a covariate therefore multiplies expected deaths by exp(β), a change of 100·(exp(β)−1)%. The Results state this formula for the time trend. All three covariates are measured in percent, so a one-unit increase is one percentage point. I applied the formula to the printed Table 2 coefficients. To allow for rounding, I also used the full range each coefficient could have before it was rounded to four decimals (β ± 0.00005). I inverted each abstract figure, ln(1 + p/100), to find the coefficient it would require. I formed 95% intervals as β ± 1.96·SE from the printed standard errors; the paper reports SEs and p-values but no intervals. Finally, I read every coefficient in the supplement's Table S3, which has the final model and four models that each drop one covariate. I tested whether any of them gives an abstract figure, either exponentiated or as 100·β.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Crowding (>2 residents/bedroom), % change per 1 pp | Abstract | +3.25% | +2.13% = 100·(exp(0.0211)−1); rounding range +2.127 to +2.138% |
| Crowding, % change per 1 pp | Results text; Table 2 | +2.13%; β 0.0211, exp(β) 1.0213 | +2.13% (consistent) |
| Coefficient needed for the abstract's +3.25% | Abstract vs Table 2 | β = 0.0211 | ln(1.0325) = 0.0320 |
| Population aged 60+, % change per 1 pp | Abstract | +5.92% | +4.41% = 100·(exp(0.0432)−1); rounding range +4.409 to +4.420% |
| Population aged 60+, % change per 1 pp | Results text; Table 2 | +4.41%; β 0.0432, exp(β) 1.0442 | +4.41% (+4.42% from the rounded 1.0442; consistent) |
| Coefficient needed for the abstract's +5.92% | Abstract vs Table 2 | β = 0.0432 | ln(1.0592) = 0.0575 |
| PHC coverage, % change per 1 pp | Abstract | −0.30% | −0.25% = 100·(exp(−0.0025)−1); rounding range −0.255 to −0.245% |
| PHC coverage, % change per 1 pp | Results text; Table 2 | −0.25%; β −0.0025, exp(β) 0.9975 | −0.25% (consistent) |
| Coefficient needed for the abstract's −0.30% | Abstract vs Table 2 | β = −0.0025 | ln(0.9970) = −0.0030 |
| Abstract ÷ Table 2 effect (crowding / age 60+ / PHC) | Abstract vs Table 2 | — | 1.52 / 1.34 / 1.20 |
| 95% CI implied by Table 2, % per 1 pp (crowding / age 60+ / PHC) | Table 2 (β ± 1.96 SE) | not reported | +0.68 to +3.60 / +2.85 to +6.00 / −0.39 to −0.11 |
| Other Table 2 rows: time trend, TB incidence, education, Black or brown population | Results text | +0.48%, +2.13%, −3.18%, +1.43% | +0.48%, +2.13%, −3.19%, +1.43% (consistent within rounding) |
| Crowding coefficient in the Table S3 sensitivity models | Supplement, Table S3 | 0.0040 / 0.0344 / 0.0381 | +0.40% / +3.50% / +3.88%; none is +3.25% |
| Age 60+ coefficient in the Table S3 sensitivity models | Supplement, Table S3 | 0.0357 / 0.0313 / 0.0259 / 0.0592 | +3.63% / +3.18% / +2.62% / +6.10%; none is +5.92% after conversion |
| PHC coefficient in the Table S3 sensitivity models | Supplement, Table S3 | −0.0025 / −0.0037 / −0.0024 / −0.0033 | −0.25% / −0.37% / −0.24% / −0.33%; none is −0.30% |
Within the body of the paper the numbers agree with each other. Table 2's coefficients, its exponentiated column and the Results text all match, as do the four other covariates in the table. The abstract is the only place where the three values differ. Each abstract figure needs a coefficient roughly 20–50% larger in absolute value than the one printed, which is far outside any rounding range.
Figure
Figure 1. The left panel shows the three per-percentage-point effects in the abstract's Findings. The right panel shows, for the same covariates of the same model, the Table 2 coefficient and its printed exponential, the percentage given in the Results text, and 100·(exp(β)−1) recomputed from the coefficient. Orange marks values as printed in the paper; blue marks values recomputed in recompute.py.txt. Each connector gives the ratio of the abstract value to the recomputed value. The Results text and the recomputation agree to two decimals in all three rows. The abstract does not agree with either in any row.
Figure 2. This figure adds the uncertainty. For each covariate, the blue dot is 100·(exp(β)−1) from Table 2 and the blue line is the 95% interval from β ± 1.96·SE. The paper reports the SEs but not the intervals. The orange diamond is the value in the abstract, and the thin vertical line marks no change (0%). Each panel has its own horizontal scale because the effects differ by more than tenfold. All three abstract values lie inside the interval, so sampling uncertainty does not separate them from the estimate. Even so, none of them is the estimate the paper reports. The age 60+ value (+5.92%) sits near the upper limit (+6.00%), and the crowding value (+3.25%) sits in the upper quarter of its interval.
Why it matters
The abstract is where most readers, and most citing papers, get a study's effect sizes. Here every number in the abstract's Findings is overstated relative to the model the abstract names. Crowding is 1.52 times the Table 2 value, age 60+ is 1.34 times and PHC coverage is 1.20 times. The overstatement grows when the effects are applied to realistic differences between municipalities. Over a 10-percentage-point difference in crowding, Table 2 implies +23.5% expected TB deaths (100·(exp(10·0.0211)−1)), while the abstract's per-point figure compounds to +37.7%. For the population aged 60+ the corresponding values are +54.0% and +77.7%. The PHC effect is small either way: −2.5% versus −3.0% per 10 points. The abstract's Interpretation ends by calling to "strengthen primary health care", and that recommendation rests on this association.
The qualitative findings are not affected. Every association keeps its direction and its significance, and each abstract value lies inside the confidence interval implied by Table 2. The Interpretation, Discussion and conclusions describe the associations without these numbers, so they stand. The error is one of reported magnitude, confined to the abstract.
Explanations considered
- A different model. The abstract names "the final spatial GAMLSS model", and the Results identify Table 2 as that model. The supplement has four more models in Table S3, each dropping one covariate. None of them gives the abstract's three values. Crowding there converts to +0.40%, +3.50% or +3.88%, and PHC coverage to −0.25%, −0.37%, −0.24% or −0.33%. One figure matches by coincidence of digits: the age 60+ coefficient in the model without the Municipal Development Index is 0.0592, which equals the abstract's 5.92 only if the coefficient is read as a percentage without exponentiating. Exponentiated, it is +6.10%. That model's other two coefficients give +3.88% and −0.33% (or 3.81 and −0.33 as 100·β), not +3.25% and −0.30%. So no single reported model, under either conversion, accounts for the three abstract figures. The supplement's R script also contains a commented-out stepwise (stepGAIC) specification with a different covariate set. No coefficients from it are published, so it cannot be checked.
- A different conversion. Under the log link, the paper's stated conversion is 100·(exp(β)−1). Reading the coefficients directly as 100·β gives +2.11%, +4.32% and −0.25%, which are also not the abstract values. Converting the abstract figures back gives coefficients of 0.0320, 0.0575 and −0.0030, none of which appears in Table 2.
- Rounding. Before rounding to four decimals, the Table 2 coefficients could range only over +2.127 to +2.138%, +4.409 to +4.420% and −0.255 to −0.245%. The abstract values are far outside these ranges. The 4.41% versus 4.42% difference between the text and the rounded exp(β) in Table 2 is rounding, and it is unrelated to 5.92%.
- A different unit or scaling. All three covariates are percentages, and both the abstract and the Results express the effect "for each one percentage-point increase". No rescaling (per SD, per 10 points) is mentioned, and none gives the abstract values. Per 10 points, for example, the Table 2 crowding effect is +23.5%.
- Text conversion or archiving artifact. The abstract numbers are plain prose. The same strings appear in the live PMC page, the Europe PMC full-text XML and the local PMC archive. The Table 2 cells line up with their rows, the minus sign on −0.0025 is intact, and the cells agree with the Results text. An artifact would not account for three different numbers.
- A deliberate rounding or summary in the abstract. The abstract gives two-decimal values, just as the Results do. They are not rounded versions of the Results values: rounding 2.13 does not give 3.25.
No explanation considered accounts for the abstract's three figures. The values in the body of the paper are internally consistent and reproduce from the printed coefficients.
Status
- Version of record. The publisher pages could not be read: doi.org redirects to linkinghub.elsevier.com, which returned only a redirect stub; thelancet.com and ScienceDirect returned HTTP 403; the Elsevier text-mining API returned 429 and then 400. The publisher-supplied text was therefore checked in two archived copies. Both the live PMC page (https://pmc.ncbi.nlm.nih.gov/articles/PMC13382474/, re-fetched 2026-10-23) and the Europe PMC full-text XML contain the abstract string with "3.25% and 5.92% increases" and "a 0.30% decrease". The PMC page also has the Results values "2.13%" and "0.25%".
- Supplement. Supplementary file mmc1.pdf (14 pages, via Europe PMC) was read in full. It contains Tables S1–S3 and the R GAMLSS script. Table S3's final-model column repeats Table 2 (0.0211, 0.0432, −0.0025).
- Corrections and retractions. None were found in Crossref update metadata, PMC notices or Europe PMC (
src/known.py, 2026-10-23). - PubPeer. The PubPeer API requires browser verification and returned no results to the automated query. Web searches for the title together with "PubPeer", "erratum" or "correction" found no thread or notice. No PubPeer discussion was found as of 2026-10-23, but the search could not be done directly on PubPeer.
As of 2026-10-23.
Suggested correction
In the abstract's Findings, replace the three effect sizes with the values the paper reports in its Results and Table 2. Adding the confidence intervals would also help:
In the final spatial GAMLSS model, proportions of household crowding (>2 residents/bedroom) and elderly population (>59 years) were associated with 2.13% and 4.41% increases in expected TB mortality for each one percentage-point increase, respectively. In contrast, each one percentage-point increase in primary health care coverage was associated with a 0.25% decrease in expected TB mortality.
If the authors meant the abstract to report a different model, the abstract should name that model and its coefficients should be published. As shown above, though, none of the reported models gives all three current figures. The other numbers in the Results that we checked need no change.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC13382474 (Lancet Reg Health Am 2026, doi:10.1016/j.lana.2026.101572)
# Abstract vs Results / Table 2: per-percentage-point effects of household crowding,
# population aged 60+, and primary health care (PHC) coverage on expected TB mortality.
#
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1.{svg,png} and
# fig2.{svg,png} in the folder that holds this file. Needs matplotlib.
#
# Inputs
# * Table 2 (final spatial GAMLSS, negative binomial type I, log link, offset log population):
# coefficients, standard errors and exp(estimate) as printed in the article archived in
# PubMed Central (PMC13382474).
# * Table S3 (supplement mmc1.pdf): coefficients of the final model and of the four
# leave-one-covariate-out sensitivity models, as printed.
# * Quotes: copied verbatim from the PMC-archived article. When run inside the
# /home/ubuntu/papers repo, the script re-checks each quote with src/show.py (skipped elsewhere).
# * Percent change per unit = 100 * (exp(beta) - 1), the formula the Results state
# explicitly for the time trend: "0.48% [ (exp (0.0048) -1) * 100]".
import math, os, subprocess
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
HERE = os.path.dirname(os.path.abspath(__file__))
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
# ------------------------------------------------------------------ quotes
QUOTES = {
"abstract": ("Abstract, Findings",
"In the final spatial GAMLSS model, proportions of household crowding (>2 residents/bedroom) "
"and elderly population (>59 years) were associated with 3.25% and 5.92% increases in expected "
"TB mortality for each one percentage-point increase, respectively. In contrast, each one "
"percentage-point increase in primary health care coverage was associated with a 0.30% decrease "
"in expected TB mortality."),
"results_time": ("Results, Final association model, para 2",
"Each additional month was associated with a 0.48% [ (exp (0.0048) -1) ∗ 100] increase in the "
"expected number of municipal-monthly TB deaths."),
"results_pos": ("Results, Final association model, para 3",
"Among the contextual variables, each one percentage-point increase in the proportion of "
"households with more than two residents per bedroom was associated with a 2.13% increase in "
"expected TB deaths. Similarly, each one percentage-point increase in the proportion of "
"residents aged 60 years or older was associated with a 4.41% increase,"),
"results_phc": ("Results, Final association model, para 4",
"Each one percentage-point increase in coverage was associated with a 0.25% decrease in "
"expected TB deaths."),
"results_t2": ("Results, Final association model, para 1",
"Table 2 presents the final spatial GAMLSS model associated with the outcome at p-value <0.01."),
"t2_crowd": ("Table 2",
"Households with more than 2 residents per bedroom (%) | prop_mais2_dorm | 0.0211 | 0.0073 | 1.0213 | =0.0041"),
"t2_age": ("Table 2",
"Population aged 60 years or older (%) | prop_mais59 | 0.0432 | 0.0077 | 1.0442 | <0.0001"),
"t2_phc": ("Table 2",
"Primary health care coverage (%) | cobertura_aps | -0.0025 | 0.0007 | 0.9975 | =0.0002"),
"sens": ("Results, Diagnostic plots, para 3",
"In all reduced models, the direction of the associations for the remaining covariates was "
"preserved (Table S3)"),
}
def check_quotes():
repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
print("Quote check: src/show.py not available here; skipped.\n")
return
try:
full = subprocess.run([py, "src/show.py", "PMC13382474", "--full"], cwd=repo,
capture_output=True, text=True, timeout=300).stdout
except Exception as e:
print(f"Quote check: could not load archived text ({e}); skipped.\n")
return
norm = lambda s: " ".join(s.replace("−", "-").split())
full_n = norm(full)
print("Quote check against the PMC-archived full text (occurrences; 1 = found once):")
for key, (loc, q) in QUOTES.items():
print(f" {key:13s} {full_n.count(norm(q))} ({loc})")
print()
# ------------------------------------------------------------------ paper's numbers
# Table 2: name -> (estimate, SE, exp(estimate) as printed)
T2 = {
"time_id": (0.0048, 0.0009, 1.0048),
"crowding": (0.0211, 0.0073, 1.0213),
"incid_tb": (0.0211, 0.0014, 1.0213),
"age60": (0.0432, 0.0077, 1.0442),
"educ": (-0.0324, 0.0037, 0.9681),
"black_brown": (0.0142, 0.0032, 1.0143),
"phc": (-0.0025, 0.0007, 0.9975),
"ipdm": (-3.7159, 0.5513, 0.0243),
}
# Results text, % change per unit (printed; "1,43%" in the text is read as 1.43)
TEXT = {"time_id": 0.48, "crowding": 2.13, "incid_tb": 2.13, "age60": 4.41,
"educ": -3.18, "black_brown": 1.43, "phc": -0.25}
# Abstract, Findings (decrease written as a negative change)
ABSTRACT = {"crowding": 3.25, "age60": 5.92, "phc": -0.30}
LABEL = {"crowding": "Household crowding (>2 residents/bedroom)",
"age60": "Population aged 60+",
"phc": "Primary health care coverage"}
# Table S3: final model and four sensitivity models (None = covariate not included)
S3_MODELS = ["final", "without crowding", "without education", "without Black/Brown",
"without Municipal Development Index"]
S3 = {
"crowding": [0.0211, None, 0.0040, 0.0344, 0.0381],
"age60": [0.0432, 0.0357, 0.0313, 0.0259, 0.0592],
"phc": [-0.0025, -0.0025, -0.0037, -0.0024, -0.0033],
}
pct = lambda b: 100 * (math.exp(b) - 1)
Z = 1.959964
def main():
check_quotes()
print("1. Table 2: percent change per unit, 100*(exp(beta)-1), vs Results text and abstract")
print(f" {'variable':12s} {'beta':>8s} {'exp(b)':>8s} {'T2 exp':>7s} {'%(beta)':>8s} "
f"{'%(T2exp)':>8s} {'text':>6s} {'abstract':>8s}")
for k, (b, se, e) in T2.items():
print(f" {k:12s} {b:8.4f} {math.exp(b):8.4f} {e:7.4f} {pct(b):8.3f} "
f"{100*(e-1):8.2f} {str(TEXT.get(k, '')):>6s} {str(ABSTRACT.get(k, '')):>8s}")
print()
print("2. Rounding range: beta printed to 4 decimals, so true beta lies in [b-0.00005, b+0.00005)")
for k in ABSTRACT:
b = T2[k][0]
lo, hi = pct(b - 0.00005), pct(b + 0.00005)
print(f" {k:9s} beta {b:+.4f}: {lo:+.3f}% to {hi:+.3f}% (abstract {ABSTRACT[k]:+.2f}%)")
print()
print("3. Coefficient the abstract value would require, ln(1 + p/100), and abstract / recomputed")
for k, p in ABSTRACT.items():
b = T2[k][0]
print(f" {k:9s} abstract {p:+.2f}% needs beta {math.log(1 + p/100):+.4f} "
f"(Table 2 {b:+.4f}); abstract/recomputed = {p/pct(b):.3f} "
f"(overstated by {100*(p/pct(b)-1):.0f}%)")
print()
print("4. 95% CI implied by Table 2 (beta +/- 1.96 SE), as percent change per percentage point")
for k in ABSTRACT:
b, se, _ = T2[k]
lo, hi = pct(b - Z*se), pct(b + Z*se)
inside = lo <= ABSTRACT[k] <= hi
print(f" {k:9s} {pct(b):+.2f}% (95% CI {lo:+.2f} to {hi:+.2f}); abstract {ABSTRACT[k]:+.2f}% "
f"{'inside' if inside else 'OUTSIDE'} the interval")
print()
print("5. Table S3 sensitivity models: 100*(exp(beta)-1) and 100*beta")
for k, row in S3.items():
print(f" {k}:")
for m, b in zip(S3_MODELS, row):
if b is None:
print(f" {m:38s} not included")
else:
print(f" {m:38s} beta {b:+.4f} -> {pct(b):+.2f}% (100*beta = {100*b:+.2f})")
print()
print("6. Does any reported coefficient reproduce an abstract value (within 0.005 pp)?")
for k, p in ABSTRACT.items():
hits = []
for m, b in zip(S3_MODELS, S3[k]):
if b is None:
continue
if abs(round(pct(b), 2) - p) < 0.005:
hits.append(f"{m}: 100*(exp(b)-1)")
if abs(round(100*b, 2) - p) < 0.005:
hits.append(f"{m}: 100*beta (not exponentiated)")
print(f" {k:9s} {p:+.2f}%: {hits if hits else 'no match in any reported model'}")
nb = S3["crowding"][4], S3["age60"][4], S3["phc"][4]
print(f" Model without Municipal Development Index, read the same way for all three "
f"(100*(exp(b)-1)): {pct(nb[0]):+.2f}%, {pct(nb[1]):+.2f}%, {pct(nb[2]):+.2f}%; "
f"(100*beta): {100*nb[0]:+.2f}, {100*nb[1]:+.2f}, {100*nb[2]:+.2f}")
print(" Abstract trio: +3.25, +5.92, -0.30")
print()
print("7. Scaled to a 10-percentage-point difference, 100*(exp(10*beta)-1)")
for k, p in ABSTRACT.items():
b = T2[k][0]
print(f" {k:9s} Table 2: {pct(10*b):+.1f}% abstract value compounded: "
f"{100*((1+p/100)**10-1):+.1f}%")
print()
fig1()
fig2()
print("Figures written to", HERE)
# ------------------------------------------------------------------ figures
def _style():
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none",
})
def _save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
def fig1():
_style()
fig = plt.figure(figsize=(7.2, 4.8), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off()
ax.set_xlim(0, 1); ax.set_ylim(0, 1)
ax.text(0.02, 0.955, "Same model, two sets of effect sizes", fontsize=13, weight="bold", va="top")
ax.text(0.02, 0.900, "% change in expected TB mortality per 1 percentage-point increase, final spatial GAMLSS", fontsize=10, color=INK2, va="top")
L0, L1, R0, R1 = 0.02, 0.34, 0.49, 0.98
top, bot = 0.83, 0.17
for x0, x1, head in ((L0, L1, "Abstract (Findings)"), (R0, R1, "Results text / Table 2")):
ax.add_patch(FancyBboxPatch((x0, bot), x1 - x0, top - bot, boxstyle="round,pad=0,rounding_size=0.015",
fc="white", ec="#c9c8c3", lw=1.2))
ax.text(x0 + 0.015, top - 0.03, head, fontsize=11, weight="bold", va="top")
rows = [("crowding", 0.64), ("age60", 0.46), ("phc", 0.28)]
for k, y in rows:
b = T2[k][0]; e = T2[k][2]; p = ABSTRACT[k]; r = pct(b)
# left panel
ax.text(L0 + 0.015, y + 0.035, LABEL[k].replace(" (>2 residents/bedroom)", ""),
fontsize=10, color=INK2, va="center")
ax.text(L0 + 0.015, y - 0.025, f"{p:+.2f}%", fontsize=14, weight="bold", color=ORANGE, va="center")
# right panel
ax.text(R0 + 0.015, y + 0.035, f"Table 2: β = {b:+.4f}, exp(β) = {e:.4f}", fontsize=10,
color=INK2, va="center")
ax.text(R0 + 0.015, y - 0.025, f"text {TEXT[k]:+.2f}%", fontsize=12, weight="bold",
color=ORANGE, va="center")
ax.text(R0 + 0.205, y - 0.025, f"100·(e$^{{β}}$−1) = {r:+.2f}%", fontsize=12, weight="bold",
color=BLUE, va="center")
# connector
ax.annotate("", xy=(R0 - 0.005, y), xytext=(L1 + 0.005, y),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.5))
ax.text((L1 + R0) / 2, y + 0.022, "conflict", fontsize=10, color=INK, ha="center",
va="bottom", weight="bold")
ax.text((L1 + R0) / 2, y - 0.022, f"abstract {p / r:.2f}×", fontsize=10, color=INK2,
ha="center", va="top")
ax.text(0.02, 0.125, "Orange: as printed in the paper. Blue: recomputed from the Table 2 coefficient with",
fontsize=10, color=INK2, va="top")
ax.text(0.02, 0.080, "100·(exp(β)−1), the formula the Results apply to the time trend. No coefficient",
fontsize=10, color=INK2, va="top")
ax.text(0.02, 0.035, "reported in Table 2 or Table S3 gives all three abstract values.",
fontsize=10, color=INK2, va="top")
_save(fig, "fig1")
def fig2():
_style()
keys = ["crowding", "age60", "phc"]
fig, axes = plt.subplots(3, 1, figsize=(7.2, 4.8), facecolor=BG)
fig.subplots_adjust(left=0.27, right=0.96, top=0.86, bottom=0.19, hspace=1.05)
fig.text(0.02, 0.965, "Abstract values fall inside the 95% CIs, away from the Table 2 estimates",
fontsize=12, weight="bold", va="top")
fig.text(0.02, 0.915, "Percent change per 1 percentage-point increase; one panel per covariate, "
"each with its own scale", fontsize=10, color=INK2, va="top")
lims = {"crowding": (-0.5, 4.5), "age60": (-0.5, 7.0), "phc": (-0.5, 0.1)}
for ax, k in zip(axes, keys):
b, se, _ = T2[k]
est, lo, hi = pct(b), pct(b - Z*se), pct(b + Z*se)
ax.set_facecolor(BG)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
ax.plot([lo, hi], [0, 0], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([est], [0], "o", ms=9, color=BLUE, zorder=3,
label="Recomputed from Table 2: 100·(exp(β)−1), 95% CI β ± 1.96 SE")
ax.plot([ABSTRACT[k]], [0], "D", ms=9, color=ORANGE, mec="white", mew=0.8, zorder=4,
label="Reported in the abstract")
ax.annotate(f"{est:+.2f}", (est, 0), xytext=(0, 9), textcoords="offset points",
ha="center", fontsize=10, color=BLUE)
ax.annotate(f"{ABSTRACT[k]:+.2f}", (ABSTRACT[k], 0), xytext=(0, -17), textcoords="offset points",
ha="center", fontsize=10, color=ORANGE)
ax.set_xlim(*lims[k]); ax.set_ylim(-1, 1)
ax.set_yticks([0]); ax.set_yticklabels([LABEL[k].replace(" (>2 residents/bedroom)", "").replace("Primary health care", "Primary health\ncare")], fontsize=10.5,
color=INK)
ax.tick_params(axis="y", length=0)
ax.tick_params(axis="x", labelsize=10)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#bdbcb7")
ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, _: f"{v:g}%"))
h, l = axes[0].get_legend_handles_labels()
fig.legend(h, l, loc="lower left", bbox_to_anchor=(0.01, 0.0), ncol=1, frameon=False, fontsize=10,
handletextpad=0.4, columnspacing=1.2)
_save(fig, "fig2")
if __name__ == "__main__":
main()
Abstract reports 15.4% survival before 24 weeks; the paper's own counts give 24.1%
Lancet Reg Health Am (2022) · doi:10.1016/j.lana.2022.100330 · PMC9903864
Summary
The abstract gives survival to discharge or transfer for US infants born before 24 weeks' gestation in 2007–2018 (N = 77,050) as 15.4%. The Results and Discussion give the same quantity for the same group as 18,533 survivors (24.1%), and 18,533 / 77,050 = 24.05%. That count is supported three more ways inside the paper: the reported deaths, the Table 1 subgroup rows (by sex, race, payer and birth weight, each summing to about 18,533 survivors) and the yearly trend, whose values average about 24%. No count, denominator or subgroup in the paper produces 15.4%. The paper presents these rates as a national benchmark for prenatal counselling, so a reader of the abstract alone would understate survival in this group by 8.7 percentage points (36% in relative terms). None of the paper's trend or regression results depend on the abstract figure.
What the paper reports
Survival rates for infants <24 weeks and with completed 24 weeks were 15.4% and 71.6%, respectively
— Abstract, Findings
In the group of infants with GA <24 weeks (N = 77,050), only 18,533 (24.1%) survived
— Results, 2nd paragraph (the same sentence goes on: "in infants born with completed 24 weeks of gestation (n = 47,295), a total of 33,872 (71.6%) survived")
mortality among all periviable infants was 71,940 (57.86%)
— Results, 1st paragraph (periviable infants: 124,345, "of those, 77,050 infants were born <24 weeks, and 47,295 were born at 24 weeks of gestation")
The study presented the national survival of 24.1% and 71.6% for preterm infants with GA <24 weeks and completed 24 weeks, respectively
— Discussion, 1st paragraph
The majority (71.6%) of infants born with GA of 24 weeks survived, whereas only 24.1% of infants <24 weeks survived
— Discussion, 2nd paragraph
In infants with GA <24 weeks, survival started at 18.4% in 2007 and was 31.9% by the year 2018
— Results, trend paragraph (also in Research in context: "survival of infants born at GA <24 weeks increased from 18.4% to 31.9%")
Female | 34,939 | 9124 (26.1) Male | 41,768 | 9409 (22.5)
— Table 1, Gender rows, columns for GA <24 weeks (Total | Alive (%)). The footer says "All numbers represent weighted data."
This study provides national benchmarks that can be used during prenatal counselling
— Conclusions
What the paper's own numbers show
Method. I took the counts the paper reports for infants born before 24 weeks (total, survivors, deaths across all periviable infants, and every Table 1 subgroup row) and recomputed the survival percentage in each way the paper's numbers allow. I then looked for any count, denominator or subgroup that would produce 15.4%. As a check against the trend, I read the yearly values off the paper's Figure 1 (upper panel, dotted line) by pixel digitization. This was calibrated on the axis labels and matches the two stated endpoints within 0.2 points. All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Survival, GA <24 wk, 2007–2018 | Abstract, Findings | 15.4% | 18,533 / 77,050 = 24.05% |
| Survival, GA <24 wk, 2007–2018 | Results 2nd para; Discussion 1st and 2nd paras | 18,533 (24.1%) | 24.05% (agrees) |
| Survival, completed 24 wk | Abstract; Results; Discussion | 33,872 / 47,295 (71.6%) | 71.62% (agrees in all places) |
| Group sizes | Abstract; Results 1st para | 77,050 + 47,295; total 124,345 | 77,050 + 47,295 = 124,345 (agrees) |
| Mortality, all periviable | Results 1st para | 71,940 (57.86%) | 71,940 / 124,345 = 57.86%. Survivors 124,345 − 71,940 = 52,405 = 18,533 + 33,872 exactly, so 52,405 − 33,872 = 18,533 survivors <24 wk (24.05%) |
| Table 1 <24 wk survivors, rows summed | Table 1 | subgroup counts | sex 18,533 / 76,707 = 24.16%; race 18,535 / 77,050 = 24.06%; payer 18,533 / 77,051 = 24.05%; birth weight 18,534 / 77,051 = 24.05%; income 18,202 / 75,830 = 24.00% |
| Survivors needed for 15.4% | derived | — | 0.154 × 77,050 = 11,866 (11,827–11,904 round to 15.4%); no such count appears in the paper |
| Alternative denominator | derived | — | 18,533 / 124,345 (all ≤24 wk) = 14.90%, not 15.4%; 18,533 would need a denominator of 120,344 |
| Ratio of 24-wk to <24-wk survival | Abstract vs data | 71.6 / 15.4 = 4.65-fold | 71.62 / 24.05 = 2.98-fold |
| Yearly survival, <24 wk | Research in context; Results trend para; Discussion | 18.4% (2007) to 31.9% (2018) | Figure 1 reading (approx.): 18.2, 14.5, 21.0, 20.4, 21.0, 24.6, 24.6, 26.2, 28.7, 29.7, 30.3, 32.0; unweighted mean 24.27% |
| Pooled 15.4% vs yearly values | derived | — | Only 2008 (≈14.5%) lies below 15.4%. A pooled 15.4% would need at least (18.2 − 15.4) / (18.2 − 14.5) = 76% of the 12-year cohort to fall in 2008 |
Other inconsistencies noted in passing (not part of the main finding, and not needed for it):
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Survival by sex, completed 24 wk | Table 1 vs Results 2nd para, Abstract and Figure 1 lower panel | Female 17,635 / 25,480 (69.2); Male 16,213 / 21,766 (74.5); the text says "Survival among males was lower than females in infants <24 weeks (p < 0.001) and completed 24 weeks (p < 0.001)" | 69.2% and 74.5%: the counts and percentages agree, but they show higher crude survival in males, the opposite of the text |
| Survival by birth weight, completed 24 wk | Table 1 vs Results (four-group percentages 45.1% for ≤500 g and 73.8% for >500 g) | ≤500 g: 43,772 / 32,283 (45.1); >500 g: 3,523 / 1,590 (73.8) | 32,283 / 43,772 = 73.8% and 1,590 / 3,523 = 45.1%: each row's counts give the other row's printed percentage |
| Median length of stay, survivors | Results vs Table 2 | text 113 and 114 days; Table 2 118 and 110 days | not reconcilable from the reported numbers |
| Median charges, survivors | Results vs Table 2 (and Discussion) | text $1,050,953 and $1,016,751; Table 2 (and Discussion) $926,105 and $895,137 | not reconcilable from the reported numbers |
Figure
Figure 1. The contradiction in the paper's own text. Left: the abstract sentence, with the value at issue (15.4%, orange, as reported). Right: the same quantity elsewhere in the paper: the Results count of 18,533 survivors out of 77,050 (24.1%), the two Discussion statements of 24.1%, the Table 1 survivors by sex (9,124 + 9,409 = 18,533) and the stated trend endpoints. The connector marks that both sides describe the same group (GA <24 weeks, N = 77,050), the same period (2007–2018) and the same outcome (survival to discharge or transfer). Bottom (blue, recomputed): 18,533 / 77,050 = 24.05%. For comparison, 15.4% would require 11,866 survivors, and the nearest alternative denominator (all 124,345 periviable infants) gives 14.90%, which does not round to 15.4%. Values come from the paper or from recompute.py.txt.
Figure 2. Survival before 24 weeks on one percentage axis starting at 0. Orange circles are values reported in the paper. Blue diamonds are values recomputed from the paper's own counts: the direct ratio, the route through the reported deaths, and the Table 1 subgroup rows summed by sex, race/ethnicity, payer and birth weight. Every route drawn lands between 24.05% and 24.16% (the income rows, which cover 75,830 of the 77,050 infants and are not drawn, give 24.00%), matching the 24.1% in the Results and Discussion. Only the abstract sits at 15.4%. The bottom row shows that the one plausible alternative denominator gives 14.90%, not 15.4%. The shaded band runs from the reported 2007 value (18.4%) to the reported 2018 value (31.9%). It marks the stated endpoints, not the full yearly range (the paper's Figure 1 shows a lower value, about 14.5%, in 2008). The yearly values themselves are not drawn.
Why it matters
- It is the abstract's headline outcome. The paper's primary outcome is survival, and its stated purpose is to supply "national benchmarks that can be used during prenatal counselling". The abstract figure for infants before 24 weeks is 8.7 percentage points (36% relative) below what the paper's data show: about 1 survivor in 6.5 births instead of 1 in 4.2.
- The abstract is the most widely read and indexed version. The PubMed record (PMID 36777383) carries the same sentence with 15.4%, and so do other abstract indexes. Readers who do not open the full text cannot see the 24.1% in the body.
- It changes the contrast the abstract draws. The abstract implies 24-week infants survive 4.65 times as often as infants under 24 weeks (71.6 / 15.4). The data give 2.98 times (71.62 / 24.05).
- What does not change. The 24-week figure (71.6%), the upward trends (Z = 9.438 and Z = 3.30), the adjusted odds ratios, the IVH and NEC trends and the cost analysis do not use the abstract figure. The paper's conclusions are worded qualitatively and stand as written. The issue is the size of the headline estimate in the abstract.
Explanations considered
- A different denominator. The abstract names the group explicitly ("infants <24 weeks", following "77,050 infants <24 weeks"). The only other natural denominator, all 124,345 periviable infants, gives 18,533 / 124,345 = 14.90%, which does not round to 15.4%. To get 15.4% from 18,533 survivors, the denominator would have to be 120,344, and that number does not appear in the paper. Does not account for the numbers.
- A different count of survivors (for example unweighted or from an earlier extract). The Methods say "This study used weighted data for all analyses", and the Table 1 footer says all numbers are weighted. 15.4% of 77,050 is 11,866 survivors. No count near that figure appears in the text or tables, and every Table 1 subgroup row, plus the deaths reported in the Results, adds up to about 18,533. Nothing in the paper supports this explanation.
- A subgroup, a single year or a sub-period. No Table 1 row for infants before 24 weeks prints a value from 15.0% to 15.9%. The nearest are the "other payer" group (14.7%) and the ≤500 g group (7.4%), and the abstract sentence refers to all infants under 24 weeks, not to a subgroup. In the paper's Figure 1, one year (2008, about 14.5%) lies below 15.4%. So a pooled 15.4% is not strictly impossible from the trend alone, as an initial screening of this paper assumed. It would, however, require at least 76% of the twelve-year cohort to be born in 2008, against an even spread of about 6,400 per year. The unweighted mean of the yearly values (24.27%) agrees with 24.1%.
- Adjusted or model-based estimate. The survival percentages in the abstract, Results and Table 1 are crude weighted proportions. The paper reports odds ratios, not adjusted survival rates. Its companion figure in the same abstract sentence (71.6%) is the crude proportion, matching 33,872 / 47,295.
- Digit transposition. "15.4" is not a rearrangement of "24.1", so a simple transposition does not explain it. The figure has no evident arithmetic origin in the paper.
- A rendering or conversion artefact. The value reads 15.4% in the archived PMC text, in the publisher-typeset PDF deposited in PMC, in the Europe PMC full-text XML and in the PubMed abstract. The body text reads 24.1% in all of them. This is not an artefact of one rendering.
None of these accounts for 15.4%. The paper's counts, its tables and its trend all support 24.1%.
Status
As of 2026-10-23:
- Version of record. The publisher pages (ScienceDirect, thelancet.com) returned HTTP 403 or a bot challenge, and the DOI resolves only to a redirect page. I checked the publisher-typeset PDF instead: the 8-page PDF served from the PMC article page (
pmc.ncbi.nlm.nih.gov/articles/PMC9903864/pdf/main.pdf, document metadata "Creator: Elsevier", "The Lancet Regional Health - Americas, 14 (2022) 100330"). It carries 15.4% in the abstract, 18,533 (24.1%) in the Results, 24.1% twice in the Discussion, and the same Table 1 and Table 2 values used here. The Europe PMC full-text XML and the PubMed record (PMID 36777383, E-utilities efetch) agree: the indexed abstract reads 15.4%. Every quote above was checked verbatim against the PMC-archived text withsrc/show.py, so this is not a conversion artefact. - Supplements. The Europe PMC supplementary bundle contains the images of Figures 1 and 2 and
mmc1.docx(Supplementary Table S1: ICD-9/ICD-10 codes only). There are no survival counts or source data, so nothing in the supplement resolves the discrepancy. I used the paper's Figure 1 only for the approximate yearly reading described above (the 2008 dip below 15% is visible on the image); it is not reproduced here. - Corrections and notices. On 2026-10-23,
src/known.pyreturned no Crossref updates, PMC notices or Europe PMC notices. The Crossref REST record has noupdate-to,updated-byorrelationentries, and the Crossref filterupdates:10.1016/j.lana.2022.100330returns 0 works. The Europe PMC record has no comment/correction list, and the PubMed record has no CommentsCorrections links. An earlier web search in this verification found no erratum; I could not repeat it in this step because the session's web search quota was used up. - PubPeer. The PubPeer search page and its extension API returned HTTP 403 (Cloudflare challenge), so PubPeer could not be checked directly. An earlier web search for the DOI on PubPeer found no thread.
- Sources. https://pmc.ncbi.nlm.nih.gov/articles/PMC9903864/ ; https://pubmed.ncbi.nlm.nih.gov/36777383/ ; https://api.crossref.org/works/10.1016/j.lana.2022.100330 ; Europe PMC REST (core search for PMID 36777383, full-text XML and supplementaryFiles for PMC9903864).
Suggested correction
In the Abstract, Findings, replace "Survival rates for infants <24 weeks and with completed 24 weeks were 15.4% and 71.6%, respectively" with "Survival rates for infants <24 weeks and with completed 24 weeks were 24.1% and 71.6%, respectively", consistent with 18,533 / 77,050 in the Results, Discussion and Table 1. The same correction should be passed to PubMed and other indexes. The authors may also wish to check:
- the Table 1 rows for completed 24 weeks by sex (69.2% female vs 74.5% male), which run opposite to the text, the abstract odds ratio and Figure 1;
- the Table 1 rows for completed 24 weeks by birth weight, where each row's counts match the other row's percentage (for example, 1,590 / 3,523 = 45.1%, the value the text gives for ≤500 g); and
- the medians for length of stay and charges in the Results text (113 and 114 days; $1,050,953 and $1,016,751), which differ from Table 2 (118 and 110 days; $926,105 and $895,137).
Recomputation script (python recompute.py.txt)
"""Recompute the survival figures in PMC9903864 (Lancet Reg Health Am 2022,
doi:10.1016/j.lana.2022.100330) and regenerate fig1/fig2 in this folder.
Run: python recompute.py.txt
Needs numpy and matplotlib.
Every input below is copied from the article as archived in PubMed Central,
except YEARLY_LT24_APPROX, which is an approximate pixel reading of the
paper's Figure 1 (upper panel, dotted line). That reading is used only for the
pooling check in section 5 and is not drawn in either figure.
"""
from pathlib import Path
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
HERE = Path(__file__).resolve().parent
# ------------------------------------------------------------------ inputs (paper)
N_ALL = 124_345 # Abstract; Results, 1st paragraph: infants born at GA <=24 wk
N_LT24 = 77_050 # Abstract; Results, 1st paragraph; Table 1 header
N_24 = 47_295 # Abstract; Results, 1st paragraph; Table 1 header
ALIVE_LT24 = 18_533 # Results, 2nd paragraph: "only 18,533 (24.1%) survived"
ALIVE_24 = 33_872 # Results, 2nd paragraph: "a total of 33,872 (71.6%) survived"
DEATHS_ALL = 71_940 # Results, 1st paragraph: "mortality among all periviable infants was 71,940 (57.86%)"
ABSTRACT_LT24 = 15.4 # Abstract, Findings
ABSTRACT_24 = 71.6 # Abstract, Findings
RESULTS_LT24 = 24.1 # Results, 2nd paragraph; Discussion, 1st and 2nd paragraphs
TREND_2007, TREND_2018 = 18.4, 31.9 # Research in context; Results trend para; Discussion
# Table 1, GA <24 weeks: (total, alive) per row
T1_LT24 = {
"sex": [(34_939, 9_124), (41_768, 9_409)],
"race": [(22_834, 5_129), (21_632, 5_809), (11_801, 3_173), (20_783, 4_424)],
"payer": [(37_434, 10_607), (28_728, 6_326), (10_889, 1_600)],
"income": [(27_011, 6_706), (18_603, 4_577), (17_113, 4_017), (13_103, 2_902)],
"birth weight": [(36_392, 2_698), (40_659, 15_836)],
}
# Table 1, completed 24 weeks: (total, alive, printed %)
T1_24_SEX = {"Female": (25_480, 17_635, 69.2), "Male": (21_766, 16_213, 74.5)}
T1_24_BW = {"<=500 g": (43_772, 32_283, 45.1), ">500 g": (3_523, 1_590, 73.8)}
# Results trend text: four GA x BW groups
FOUR_GROUPS_TEXT = {"<24 wk, <=500 g": 7.4, "<24 wk, >500 g": 38.9,
"24 wk, <=500 g": 45.1, "24 wk, >500 g": 73.8}
# Results text vs Table 2 (surviving infants): medians
LOS_TEXT, LOS_T2 = (113, 114), (118, 110)
CHARGES_TEXT, CHARGES_T2 = (1_050_953, 1_016_751), (926_105, 895_137)
# Approximate pixel reading of Figure 1, upper panel, dotted line (<24 wk), 2007..2018.
# Calibrated on the 0%..80% axis labels; it reproduces the stated endpoints within 0.2 pt.
YEARS = list(range(2007, 2019))
YEARLY_LT24_APPROX = [18.2, 14.5, 21.0, 20.4, 21.0, 24.6, 24.6, 26.2, 28.7, 29.7, 30.3, 32.0]
def pct(a, b):
return 100.0 * a / b
out = {}
print("=" * 74)
print("1. Headline survival, GA <24 weeks and completed 24 weeks")
out["lt24"] = pct(ALIVE_LT24, N_LT24)
out["w24"] = pct(ALIVE_24, N_24)
print(f" N_LT24 + N_24 = {N_LT24 + N_24:,} (reported {N_ALL:,})")
print(f" <24 wk : 18,533 / 77,050 = {out['lt24']:.2f}% (Abstract {ABSTRACT_LT24}%, Results/Discussion {RESULTS_LT24}%)")
print(f" 24 wk : 33,872 / 47,295 = {out['w24']:.2f}% (Abstract {ABSTRACT_24}%, Results 71.6%)")
print(f" difference Abstract vs recomputed: {out['lt24'] - ABSTRACT_LT24:.2f} percentage points; "
f"relative understatement {100 * (1 - ABSTRACT_LT24 / out['lt24']):.0f}%")
print(f" 1 survivor per {100 / ABSTRACT_LT24:.1f} births (15.4%) vs 1 per {100 / out['lt24']:.1f} births (24.05%)")
print(f" 24 wk / <24 wk survival ratio: abstract {ABSTRACT_24 / ABSTRACT_LT24:.2f}x, data {out['w24'] / out['lt24']:.2f}x")
print("\n2. Route via the reported deaths (Results, 1st paragraph)")
surv_all = N_ALL - DEATHS_ALL
out["mort"] = pct(DEATHS_ALL, N_ALL)
out["via_deaths"] = pct(surv_all - ALIVE_24, N_LT24)
print(f" mortality 71,940 / 124,345 = {out['mort']:.2f}% (reported 57.86%)")
print(f" survivors 124,345 - 71,940 = {surv_all:,}; 18,533 + 33,872 = {ALIVE_LT24 + ALIVE_24:,}")
print(f" <24 wk survivors implied = {surv_all - ALIVE_24:,} -> {out['via_deaths']:.2f}% of 77,050")
print("\n3. Table 1 subgroup rows, GA <24 weeks, summed")
out["t1"] = {}
for name, rows in T1_LT24.items():
T = sum(r[0] for r in rows)
A = sum(r[1] for r in rows)
out["t1"][name] = pct(A, T)
print(f" by {name:<13}: total {T:>6,} alive {A:>6,} -> {pct(A, T):.2f}%")
print("\n4. What would be needed for the abstract value")
need = ABSTRACT_LT24 / 100 * N_LT24
print(f" survivors needed for 15.4% of 77,050: {need:,.0f} (no such count in the paper)")
lo, hi = (ABSTRACT_LT24 - 0.05) / 100 * N_LT24, (ABSTRACT_LT24 + 0.05) / 100 * N_LT24
print(f" range that rounds to 15.4%: {lo:,.0f} to {hi:,.0f} survivors")
out["alt_den"] = pct(ALIVE_LT24, N_ALL)
print(f" alternative denominator 18,533 / 124,345 (all periviable) = {out['alt_den']:.2f}%")
print(f" denominator that would turn 18,533 into 15.4%: {ALIVE_LT24 / (ABSTRACT_LT24 / 100):,.0f}")
print(f" all survivors / all periviable: {pct(ALIVE_LT24 + ALIVE_24, N_ALL):.2f}%")
print(f" ungrouped Table 1 rows (<24 wk) that print 15.x%: "
f"{[round(pct(a, t), 1) for rows in T1_LT24.values() for t, a in rows if 15 <= pct(a, t) < 16] or 'none'}")
print(f" digit check: '15.4' is a permutation of '24.1'? {sorted('15.4') == sorted('24.1')}")
print("\n5. Pooling check against the yearly trend")
eq = (ABSTRACT_LT24 * 12 - TREND_2007 - TREND_2018) / 10
eq2 = (out["lt24"] * 12 - TREND_2007 - TREND_2018) / 10
print(f" equal yearly weights, stated endpoints only: other 10 years must average {eq:.2f}% for 15.4%, "
f"{eq2:.2f}% for 24.05%")
y = np.array(YEARLY_LT24_APPROX)
print(f" Figure 1 reading (approx.): {', '.join(f'{v:.1f}' for v in y)}")
print(f" reading vs stated endpoints: 2007 {y[0]:.1f} vs {TREND_2007}; 2018 {y[-1]:.1f} vs {TREND_2018}")
print(f" unweighted mean of yearly values: {y.mean():.2f}%; minimum {y.min():.1f}% ({YEARS[int(y.argmin())]})")
others_min = np.delete(y, y.argmin()).min()
share = (others_min - ABSTRACT_LT24) / (others_min - y.min())
print(f" for a pooled 15.4%, the {YEARS[int(y.argmin())]} share of the cohort must be at least "
f"({others_min:.1f} - 15.4) / ({others_min:.1f} - {y.min():.1f}) = {100 * share:.0f}%")
print(f" average annual cohort if evenly spread: 77,050 / 12 = {N_LT24 / 12:,.0f}")
print("\n6. Side issues in Table 1 / Table 2 (not part of the main finding)")
for k, (t, a, p) in T1_24_SEX.items():
print(f" 24 wk {k:<6}: {a:,}/{t:,} = {pct(a, t):.1f}% (printed {p})")
print(" -> Table 1 shows higher crude survival for males at 24 wk; the text says males lower")
for k, (t, a, p) in T1_24_BW.items():
print(f" 24 wk {k:<7}: {a:,}/{t:,} = {pct(a, t):.1f}% (printed {p}; text gives {k} = "
f"{FOUR_GROUPS_TEXT['24 wk, ' + k]}%)")
print(" -> each 24-wk birth-weight row's counts give the other row's printed percentage")
print(f" 24 wk sex rows sum: total {sum(v[0] for v in T1_24_SEX.values()):,}, "
f"alive {sum(v[1] for v in T1_24_SEX.values()):,}")
print(f" LOS median, text {LOS_TEXT} vs Table 2 {LOS_T2} days")
print(f" charges median, text {CHARGES_TEXT} vs Table 2 {CHARGES_T2} USD")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(HERE / f"{stem}.svg")
fig.savefig(HERE / f"{stem}.png", dpi=200)
plt.close(fig)
# Figure 1: abstract vs body contradiction diagram
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.6)
ax.axis("off")
def panel(x, y, w, h, title, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.14, y + h - 0.2, title, fontsize=11, fontweight="bold", va="top")
# left: abstract
panel(0.2, 1.55, 2.85, 2.65, "Abstract, Findings", ORANGE)
ax.text(0.34, 3.72, "“Survival rates for infants\n<24 weeks and with completed\n24 weeks were",
fontsize=10, va="top", color=INK2, linespacing=1.35)
ax.text(0.34, 2.82, "15.4%", fontsize=22, fontweight="bold", color=ORANGE, va="top")
ax.text(1.48, 2.72, "and 71.6%,\nrespectively”", fontsize=10, va="top", color=INK2,
linespacing=1.35)
ax.text(0.34, 1.95, "N = 77,050 (<24 wk)", fontsize=10, color=INK2, va="top")
# right: results / discussion / table 1
panel(4.15, 1.55, 2.85, 2.65, "Results, Discussion, Table 1", INK2)
ax.text(4.29, 3.72, "Results: “(N = 77,050), only\n18,533 (24.1%) survived”",
fontsize=10, va="top", color=INK2, linespacing=1.35)
ax.text(4.29, 3.0, "Discussion (twice): 24.1%", fontsize=10, va="top", color=INK2)
ax.text(4.29, 2.6, "Table 1, <24 wk alive by sex:\n9,124 + 9,409 = 18,533",
fontsize=10, va="top", color=INK2, linespacing=1.35)
ax.text(4.29, 1.95, "Trend: 18.4% (2007) to 31.9% (2018)", fontsize=10, color=INK2, va="top")
# connector
ax.add_patch(FancyArrowPatch((3.12, 2.9), (4.08, 2.9), arrowstyle="<->", mutation_scale=14,
lw=2, color=INK))
ax.text(3.6, 3.2, "conflict", ha="center", fontsize=10, fontweight="bold")
ax.text(3.6, 2.7, "same group,\nperiod and\noutcome", ha="center", va="top",
fontsize=10, color=INK2, linespacing=1.3)
# recomputation strip
ax.add_patch(FancyBboxPatch((0.2, 0.3), 6.8, 0.95, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=BLUE, lw=1.5))
ax.text(0.34, 1.07, "Recomputed from the paper's own counts", fontsize=11, fontweight="bold",
va="top")
ax.text(0.34, 0.68, f"18,533 / 77,050 = {out['lt24']:.2f}%", fontsize=13, fontweight="bold",
color=BLUE, va="top")
ax.text(3.35, 0.73, f"15.4% would need {need:,.0f} survivors;\n"
f"18,533 / 124,345 (all ≤24 wk) = {out['alt_den']:.2f}%",
fontsize=10, color=INK2, va="top", linespacing=1.35)
ax.legend(handles=[Line2D([], [], marker="s", ls="", ms=9, color=ORANGE, label="value as reported (abstract)"),
Line2D([], [], marker="s", ls="", ms=9, color=BLUE, label="recomputed from paper's counts")],
loc="upper center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# Figure 2: dot comparison on one percentage axis starting at 0
rows = [
("Abstract, Findings", ABSTRACT_LT24, "rep"),
("Results; Discussion (×2)", RESULTS_LT24, "rep"),
("18,533 / 77,050", out["lt24"], "rec"),
("(124,345 − 71,940 deaths − 33,872) / 77,050", out["via_deaths"], "rec"),
("Table 1 rows summed: sex", out["t1"]["sex"], "rec"),
("Table 1 rows summed: race/ethnicity", out["t1"]["race"], "rec"),
("Table 1 rows summed: payer", out["t1"]["payer"], "rec"),
("Table 1 rows summed: birth weight", out["t1"]["birth weight"], "rec"),
("Alternative denominator: 18,533 / 124,345", out["alt_den"], "rec"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.4))
fig.subplots_adjust(left=0.47, right=0.97, top=0.86, bottom=0.13)
n = len(rows)
ypos = np.arange(n)[::-1]
ax.axvspan(TREND_2007, TREND_2018, color="#efeeea", zorder=0)
ax.text((TREND_2007 + TREND_2018) / 2, n - 0.35, "shaded: from the reported 2007\n(18.4%) to 2018 (31.9%) value",
ha="center", va="bottom", fontsize=10, color=INK2)
for yv, (lab, v, kind) in zip(ypos, rows):
c = ORANGE if kind == "rep" else BLUE
mk = "o" if kind == "rep" else "D"
ax.plot([0, v], [yv, yv], color=GRID, lw=1, zorder=1)
ax.plot(v, yv, marker=mk, ms=9, color=c, mec=BG, mew=1.5, zorder=3, ls="")
ax.text(v + 0.9, yv, f"{v:.2f}%" if kind == "rec" else f"{v:.1f}%", va="center",
fontsize=10, color=INK)
ax.set_yticks(ypos)
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_xlim(0, 40)
ax.set_ylim(-0.7, n + 0.6)
ax.set_xticks(range(0, 41, 5))
ax.set_xticklabels([f"{t}%" for t in range(0, 41, 5)])
ax.set_xlabel("Survival, GA <24 weeks, 2007–2018 pooled")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="reported in the paper"),
Line2D([], [], marker="D", ls="", ms=9, color=BLUE, label="recomputed from the paper's counts")],
loc="lower left", bbox_to_anchor=(-0.9, 1.08), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nfigures written:", ", ".join(sorted(p.name for p in HERE.glob("fig*.*"))))
Abstract reports low-birth-weight estimates, including a nominally significant ASD effect, as preterm-birth results
Lancet Reg Health Eur (2025) · doi:10.1016/j.lanepe.2025.101247 · PMC11910110
Summary
The abstract's Findings say that, compared with term birth, preterm birth was associated with higher ASD symptoms, 3·23 (95% CI 0·37, 6·08) percentiles, and give preterm effects of 4·15 for ADHD symptoms and −5·44 for non-verbal IQ. All three values, estimates and both CI limits, are the cells of the birth weight <2500 g row of Table 4, not the gestational age <37 wk row. Table 4's preterm row gives ASD 0·60 (−2·60, 3·79), which is null (p ≈ 0·71 recomputed from the CI), ADHD 2·51 and non-verbal IQ −3·41; the Results and Discussion agree with the table and state explicitly that preterm birth was not associated with ASD symptoms. A reader of the abstract alone would therefore take away a preterm–ASD association that the paper's own analysis did not find, and preterm effects on ADHD and IQ about 1·6 times larger than estimated.
What the paper reports
Abstract, Findings (second sentence):
As compared to term birth, preterm birth was associated with higher internalizing problems (difference 3·43 (95% CI: 2·52, 4·33)) and externalizing problems (difference 2·31 (95% CI: 1·16, 3·46)), ADHD symptoms (difference 4·15 (95% CI: 3·15, 5·16)), ASD symptoms (difference 3·23 (95% CI: 0·37, 6·08)), and lower non-verbal IQ (difference -5·44 (95% CI: -7·44, -3·44)).
Results, "Birth outcomes, emotional, behavior and cognitive outcomes", paragraph 2:
As compared to children born term, those born preterm had increased scores for ADHD symptoms in childhood (2·51 percentiles (95% CI: 1·61, 3·40)) and early adolescence (2·33 percentiles (95% CI: 1·16, 3·50)).
Not preterm birth, but low birth weight was associated with ASD symptoms (3·23 percentiles (95% CI: 0·37, 6·08)).
Preterm birth and low birth weight were also associated with a -3·41 percentiles (95% CI: -5·46, -1·36) and -5·44 percentiles (95% CI: -7·44, -3·44) lower Non-verbal IQ in childhood respectively.
Discussion, paragraph 4:
The lack of association for preterm birth might also be explained by lower numbers since ASD was only available in two cohorts, that have both a rather healthy population, without extremely premature children.
Table 4 (confounder model; childhood ADHD, ASD and non-verbal IQ columns), as printed:
<37 | 2·51 (1·61, 3·40)∗∗n = 4522 | … | 0·60 (-2·60, 3·79)n = 279 | -3·41 (-5·46, -1·36)∗∗n = 780
<2500 | 4·15 (3·15, 5·16)∗∗n = 3531 | … | 3·23 (0·37, 6·08)∗N = 353 | -5·44 (-7·44, -3·44)∗∗N = 821
Table 4 footer: "∗p value < 0·05, ∗∗p value < 0·01". Methods, Statistical analysis:
To correct for multiple testing, we used the Bonferroni correction method taking account for five outcomes and specified multiple-test corrected p-values as <0·01.
What the paper's own numbers show
Method. Each number in the abstract's "preterm birth" sentence was matched, estimate and both CI limits, against every relevant cell of Tables 3 and 4 (confounder model). For each cell a standard error was recovered from the printed 95% CI as SE = (upper − lower)/3·92 and a two-sided Wald p-value computed. The basic-model (Table S5) and complete-case (Table S9) results in the paper's Appendix A were checked the same way, to rule out that the abstract quotes a different model or analysis set. All arithmetic is in recompute.py.txt.
| Quantity | Location | Reported | Recomputed / located |
|---|---|---|---|
| ASD symptoms, "preterm birth" | Abstract, Findings | 3·23 (0·37, 6·08) | Identical to Table 4 <2500 g ASD cell (N = 353); not in the <37 wk row |
| ASD symptoms, preterm <37 wk | Table 4 | 0·60 (−2·60, 3·79), n = 279, no asterisk | SE 1·630, z 0·37, p = 0·71; CI includes 0 |
| ASD symptoms, birth weight <2500 g | Table 4; Results | 3·23 (0·37, 6·08)∗ | SE 1·457, z 2·22, p = 0·027 (matches ∗; does not meet the paper's Bonferroni p < 0·01) |
| ADHD symptoms (childhood), "preterm birth" | Abstract, Findings | 4·15 (3·15, 5·16) | Identical to Table 4 <2500 g cell; preterm cell is 2·51 (1·61, 3·40), p < 0·001; abstract/preterm ratio 1·65 |
| Non-verbal IQ, "preterm birth" | Abstract, Findings | −5·44 (−7·44, −3·44) | Identical to Table 4 <2500 g cell; preterm cell is −3·41 (−5·46, −1·36), p = 0·001; ratio 1·60 |
| Internalizing (childhood), "preterm birth" | Abstract, Findings | 3·43 (2·52, 4·33) | Identical to Table 3 <37 wk childhood cell (correct) |
| Externalizing, "preterm birth" | Abstract, Findings | 2·31 (1·16, 3·46) | Table 3 <37 wk early-adolescence cell; the childhood cell is 2·28 (1·43, 3·13). Correct exposure, different age window from the other four numbers (minor) |
| ASD, preterm, basic model | Appendix A, Table S5 | 1·35 (−1·89, 4·60) | z 0·82, p = 0·41; not the abstract value |
| ASD, <2500 g, basic model | Appendix A, Table S5 | 3·71 (0·82, 6·60)∗ | not the abstract value; no basic-model cell equals any abstract value |
| ASD, preterm, complete-case | Appendix A, Table S9 | −0·50 (−4·41, 3·41) | z −0·25, p = 0·80; no complete-case cell equals any abstract value |
| LGA vs AGA, ADHD (side finding) | Abstract vs Table 5 | Abstract −1·09 (−1·73, 0·45); Table 5 −1·09 (−1·73, −0·45)∗ | With +0·45 the CI midpoint is −0·64, not −1·09, and the interval would include 0; Table 5's interval is symmetric about −1·09 (p = 0·0008). The abstract upper limit has lost its minus sign |
Three of the five numbers in the "preterm birth" sentence (ADHD, ASD, non-verbal IQ) reproduce the confounder-model <2500 g row of Table 4 exactly, to two decimals in all three positions; none of them matches any preterm cell in the main, basic-model or complete-case tables, in any age window. In every one of these analyses the preterm ASD estimate is null. The first two numbers (internalizing, externalizing) do come from the preterm row of Table 3.
Figure
Figure 1. The abstract's ASD value (orange, as reported) is identical in estimate and both CI limits to the Table 4 row for birth weight <2500 g, not the row for gestational age <37 wk. The preterm row, 0·60 (−2·60, 3·79) with n = 279, has a recomputed p of 0·71; the low-birth-weight row has p = 0·027 (blue, recomputed from the printed 95% CIs). The Results and Discussion text (right) attribute the association to low birth weight and describe a lack of association for preterm birth.
Figure 2. For each outcome in the abstract's "preterm birth" sentence, the value stated in the abstract (orange circles) is plotted against the preterm (<37 wk vs 37–42 wk) childhood estimate from Tables 3 and 4 (blue squares), with 95% CIs on one axis of outcome-percentile differences and a reference line at no difference. The right-hand column names the table cell each abstract value reproduces and the recomputed p-value of the preterm cell. Internalizing and externalizing agree; ADHD and non-verbal IQ are overstated by about 60%; for ASD the abstract's interval excludes 0 while the paper's preterm interval is centred near 0.
Why it matters
- A null result is presented as a positive finding. The abstract states that preterm birth was associated with ASD symptoms, with a CI excluding zero. The paper's own preterm estimate is 0·60 (−2·60, 3·79), p ≈ 0·71, and the Results ("Not preterm birth, but low birth weight") and Discussion ("The lack of association for preterm birth") say the opposite of the abstract.
- Effect sizes are overstated. The preterm effects on ADHD symptoms (2·51) and non-verbal IQ (−3·41) appear in the abstract as 4·15 and −5·44, about 1·65× and 1·60× larger. Direction and statistical significance are unchanged for these two.
- The abstract is the part most often read, indexed and reused. The same sentence is in the PubMed record and in the STROBE checklist in the paper's appendix. Evidence summaries or meta-analyses that extract preterm effects from the abstract would take the low-birth-weight values.
- Even the relocated value is weak. The 3·23 ASD estimate, correctly attributed to low birth weight, has p ≈ 0·027 and does not meet the paper's own Bonferroni threshold (p < 0·01); it rests on two cohorts and 353 exposed children.
- What is not affected. The tables, Results text, Discussion and the Interpretation ("Both fetal and early childhood growth are associated with emotional, behavioral and cognitive outcomes…") are internally consistent and do not depend on these abstract numbers.
Explanations considered
- Preterm birth used loosely to mean "adverse birth characteristics". Does not account for it. The sentence opens "As compared to term birth, preterm birth", which names a specific contrast, and the paper itself distinguishes the two exposures for ASD ("Not preterm birth, but low birth weight").
- A different model (basic vs confounder-adjusted). Does not account for it. The abstract values equal the confounder-model cells of Table 4. The basic-model Table S5 gives preterm ASD 1·35 (−1·89, 4·60) and low-birth-weight ASD 3·71 (0·82, 6·60); neither equals 3·23 (0·37, 6·08).
- A different age window or population. Does not account for the ASD or non-verbal IQ values: both outcomes were measured only in childhood, in one set of cohorts (ASD: GENR and INMA). For ADHD the other preterm cells are 2·33 (early adolescence) and −1·74 (late adolescence), neither 4·15. Only the externalizing value is explained this way: 2·31 is the preterm early-adolescence cell, while the other numbers in the sentence are childhood values.
- A sensitivity analysis. Does not account for it. The complete-case Table S9 gives preterm ASD −0·50 (−4·41, 3·41), and no cell equals an abstract value. The two-stage results are shown only graphically (Figure S4) and described as "less precise"; the abstract values instead equal the one-stage confounder-model cells exactly. Apart from the abstract text repeated in the STROBE checklist, the triples 3·23 (0·37, 6·08), 4·15 (3·15, 5·16) and −5·44 (−7·44, −3·44) appear nowhere in the appendix.
- Coincidental agreement. Three triples (estimate, lower, upper) matching one specific table row exactly cannot plausibly be coincidence.
- A text-conversion artifact in PMC. Ruled out: the same sentence is in the Europe PMC JATS XML, in the PubMed abstract (from the publisher's deposit) and in the authors' own STROBE checklist in Appendix A.
- A typographical slip. Fits the LGA ADHD upper limit (0·45 for −0·45), a lost minus sign. It does not fit the main issue, where whole estimate–CI triples correspond to a different exposure row. The values coincide with the adjacent row of Table 4; the paper does not say how the abstract sentence was compiled, and nothing here depends on how it arose.
Status
As of 2026-10-23:
- Version of record. The publisher's pages (thelancet.com, sciencedirect.com, linkinghub) returned 403 or a redirect stub to automated access, so the publisher-hosted HTML/PDF could not be read directly. The same abstract wording and the same Table 4 cells were confirmed in four independent copies of the published text: the PMC article (show.py and the PMC web page), the Europe PMC JATS full-text XML (Table 4 parsed cell by cell, with U+2212 minus signs), the PubMed record PMID 40094119, and the STROBE checklist in the paper's own Appendix A (item 1(b) repeats the abstract sentence; item 16 repeats the Results sentence). A PDF-to-HTML conversion artifact is therefore excluded.
- Supplements. Appendix A (mmc1.docx, the only supplementary file) was read: Tables S4–S5 (basic models) and S8–S9 (complete-case analyses) do not contain the abstract values under preterm birth; preterm ASD is 1·35 (−1·89, 4·60) in Table S5 and −0·50 (−4·41, 3·41) in Table S9, both null.
- Corrections. No correction, erratum or retraction found in Crossref update metadata, PMC notices, or the Europe PMC record (no comment/correction links);
known.pywas re-run on 2026-10-23 and still returns no updates or notices. Web searches for the DOI and title with "correction"/"erratum", run earlier in this audit, found none; a further web search on 2026-10-23 could not be run (search quota exhausted). Because the publisher's own pages could not be read, a notice posted only on the publisher site cannot be fully excluded. - PubPeer. PubPeer's site and API refused automated access (403 / Cloudflare challenge); web searches for a PubPeer thread on the title or DOI found none. Absence of a thread is therefore not fully confirmed.
- Sources.
src/show.py PMC11910110(Abstract, Methods, Results, Discussion, Tables 3–5);src/known.py 10.1016/j.lanepe.2025.101247; Europe PMC REST full-text XML and supplementaryFiles for PMC11910110; PubMed efetch for PMID 40094119.
Suggested correction
In the Abstract, Findings, replace the second sentence so that each estimate is attributed to its own exposure and age window, for example:
As compared to term birth, preterm birth was associated with higher internalizing problems (difference 3·43 (95% CI: 2·52, 4·33)), externalizing problems (difference 2·28 (95% CI: 1·43, 3·13)) and ADHD symptoms (difference 2·51 (95% CI: 1·61, 3·40)) in childhood, and with lower non-verbal IQ (difference −3·41 (95% CI: −5·46, −1·36)), but not with ASD symptoms (difference 0·60 (95% CI: −2·60, 3·79)). Low birth weight (<2500 g) was associated with higher ADHD symptoms (difference 4·15 (95% CI: 3·15, 5·16)) and ASD symptoms (difference 3·23 (95% CI: 0·37, 6·08)) and lower non-verbal IQ (difference −5·44 (95% CI: −7·44, −3·44)).
If the early-adolescence externalizing value (2·31 (1·16, 3·46)) is preferred, label it as early adolescence. Consider noting that the low-birth-weight ASD association (p ≈ 0·03) does not meet the paper's Bonferroni threshold of p < 0·01. In the next sentence on large size for gestational age, correct the ADHD interval to (95% CI: −1·73, −0·45), as in Table 5. The same corrections apply to the abstract text repeated in the STROBE checklist (Appendix A, item 1(b)).
Recomputation script (python recompute.py.txt)
"""Recompute the checks behind the write-up for PMC11910110
(Lancet Reg Health Eur 2025; doi:10.1016/j.lanepe.2025.101247), and redraw
fig1/fig2 in this file's folder.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
Every estimate below is copied from the article as archived in PubMed Central
(Abstract Findings, Results text, Tables 3, 4 and 5; confounder model), plus
the basic-model Table S5 and complete-case Table S9 from the paper's Appendix A
(mmc1.docx) as cross-checks.
Values are differences in outcome percentiles with 95% CIs.
The only assumption is the usual normal approximation used to recover a
standard error from a printed 95% CI: SE = (upper - lower) / (2 * 1.959964).
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = stats.norm.isf(0.025)
# ------------------------------------------------------------------ inputs
# Table cells: key -> (estimate, lower, upper, n exposed)
T = {
# Table 3 (internalizing / externalizing), gestational age <37 wk vs 37-42 wk
("T3", "<37 wk", "Internalizing", "childhood"): (3.43, 2.52, 4.33, 4780),
("T3", "<37 wk", "Internalizing", "early adolescence"): (1.77, 0.57, 2.96, 2873),
("T3", "<37 wk", "Externalizing", "childhood"): (2.28, 1.43, 3.13, 4777),
("T3", "<37 wk", "Externalizing", "early adolescence"): (2.31, 1.16, 3.46, 2873),
# Table 3, birth weight <2500 g vs 2500-4500 g
("T3", "<2500 g", "Internalizing", "childhood"): (3.20, 2.20, 4.20, 3887),
("T3", "<2500 g", "Externalizing", "childhood"): (3.77, 2.83, 4.71, 3885),
# Table 4 (ADHD / ASD / non-verbal IQ), gestational age <37 wk
("T4", "<37 wk", "ADHD", "childhood"): (2.51, 1.61, 3.40, 4522),
("T4", "<37 wk", "ADHD", "early adolescence"): (2.33, 1.16, 3.50, 2878),
("T4", "<37 wk", "ADHD", "late adolescence"): (-1.74, -2.89, -0.58, 2668),
("T4", "<37 wk", "ASD", "childhood"): (0.60, -2.60, 3.79, 279),
("T4", "<37 wk", "Non-verbal IQ", "childhood"): (-3.41, -5.46, -1.36, 780),
# Table 4, birth weight <2500 g
("T4", "<2500 g", "ADHD", "childhood"): (4.15, 3.15, 5.16, 3531),
("T4", "<2500 g", "ADHD", "early adolescence"): (3.51, 2.15, 4.87, 2078),
("T4", "<2500 g", "ASD", "childhood"): (3.23, 0.37, 6.08, 353),
("T4", "<2500 g", "Non-verbal IQ", "childhood"): (-5.44, -7.44, -3.44, 821),
# Table 5, large for gestational age, ADHD
("T5", "LGA", "ADHD", "childhood"): (-1.09, -1.73, -0.45, 9718),
}
# Abstract, Findings: "As compared to term birth, preterm birth was associated
# with higher internalizing problems (...), externalizing problems (...), ADHD
# symptoms (...), ASD symptoms (...), and lower non-verbal IQ (...)."
ABSTRACT = [
("Internalizing", 3.43, 2.52, 4.33),
("Externalizing", 2.31, 1.16, 3.46),
("ADHD", 4.15, 3.15, 5.16),
("ASD", 3.23, 0.37, 6.08),
("Non-verbal IQ", -5.44, -7.44, -3.44),
]
# Abstract, LGA sentence: "lower ADHD symptoms (difference -1.09 (95% CI: -1.73, 0.45))"
ABSTRACT_LGA = (-1.09, -1.73, 0.45)
BONFERRONI_ALPHA = 0.01 # Methods: "specified multiple-test corrected p-values as <0.01"
def wald(est, lo, hi):
se = (hi - lo) / (2 * Z975)
z = est / se
return se, z, 2 * stats.norm.sf(abs(z))
def fmt(x):
return f"{x:+.2f}"
# ------------------------------------------------------------ 1. cell checks
print("=== 1. Table cells: SE from 95% CI, Wald z, two-sided p ===")
REC = {}
for k, (e, l, h, n) in T.items():
se, z, p = wald(e, l, h)
REC[k] = (se, z, p)
print(f" {k[0]} {k[1]:8s} {k[2]:14s} {k[3]:18s} est {fmt(e)} "
f"({fmt(l)}, {fmt(h)}) n={n:<5d} CI midpoint {(l+h)/2:+.3f} "
f"SE {se:.3f} z {z:+.2f} p {p:.4f}")
# -------------------------------------- 2. where the abstract numbers come from
print("\n=== 2. Abstract 'preterm birth' sentence: which table cell each number matches ===")
MATCH = {}
for name, e, l, h in ABSTRACT:
hits = [k for k, v in T.items() if v[:3] == (e, l, h)]
MATCH[name] = hits
pre = T[(("T3" if name in ("Internalizing", "Externalizing") else "T4"),
"<37 wk", name, "childhood")]
print(f" {name:14s} abstract {fmt(e)} ({fmt(l)}, {fmt(h)})")
print(f" matches: {[' / '.join(k) for k in hits]}")
print(f" Table preterm (<37 wk) childhood cell: {fmt(pre[0])} "
f"({fmt(pre[1])}, {fmt(pre[2])}); abstract / preterm ratio {e/pre[0]:.2f}")
# --------------------------------------------------------------- 3. ASD focus
print("\n=== 3. ASD symptoms in childhood (Table 4) ===")
se_pt, z_pt, p_pt = REC[("T4", "<37 wk", "ASD", "childhood")]
se_lb, z_lb, p_lb = REC[("T4", "<2500 g", "ASD", "childhood")]
print(f" preterm <37 wk : 0.60 (-2.60, 3.79), SE {se_pt:.3f}, z {z_pt:.2f}, p {p_pt:.3f}; "
f"CI contains 0: {-2.60 < 0 < 3.79}")
print(f" LBW <2500 g : 3.23 (0.37, 6.08), SE {se_lb:.3f}, z {z_lb:.2f}, p {p_lb:.3f}; "
f"passes Bonferroni p<{BONFERRONI_ALPHA}: {p_lb < BONFERRONI_ALPHA}")
print(f" Table 4 flags the LBW cell with a single asterisk (p<0.05), consistent with p {p_lb:.3f}")
# -------------------------------------------------- 4. size of the overstatement
print("\n=== 4. Overstatement of preterm effects in the abstract (ADHD, non-verbal IQ) ===")
for name in ("ADHD", "Non-verbal IQ"):
a = dict((x[0], x[1:]) for x in ABSTRACT)[name][0]
pre = T[("T4", "<37 wk", name, "childhood")]
p_pre = REC[("T4", "<37 wk", name, "childhood")][2]
print(f" {name:14s} abstract {fmt(a)} vs preterm {fmt(pre[0])}: "
f"difference {a-pre[0]:+.2f}, ratio {a/pre[0]:.2f}, preterm p {p_pre:.4f}")
# ------------------------------------------------ 5. externalizing age window
print("\n=== 5. Externalizing: age window ===")
for w in ("childhood", "early adolescence"):
e, l, h, n = T[("T3", "<37 wk", "Externalizing", w)]
print(f" preterm externalizing, {w:18s}: {fmt(e)} ({fmt(l)}, {fmt(h)})")
print(" abstract uses +2.31 (early adolescence) next to internalizing +3.43 (childhood)")
# --------------------------------------------- 6. LGA ADHD sign (side finding)
print("\n=== 6. LGA ADHD CI in abstract vs Table 5 ===")
e, l, h = ABSTRACT_LGA
print(f" abstract: {fmt(e)} ({fmt(l)}, {fmt(h)}): midpoint {(l+h)/2:+.2f}, CI contains 0: {l < 0 < h}")
e5, l5, h5, _ = T[("T5", "LGA", "ADHD", "childhood")]
se5, z5, p5 = REC[("T5", "LGA", "ADHD", "childhood")]
print(f" Table 5 : {fmt(e5)} ({fmt(l5)}, {fmt(h5)}): midpoint {(l5+h5)/2:+.2f}, p {p5:.4f}")
# ------------------------- 7. basic model (Appendix A, Table S5) as a cross-check
# Supplement Table S5 (basic model: sex, age at outcome, cohort), childhood columns.
S5 = {
("<37 wk", "ADHD"): (3.26, 2.36, 4.16),
("<37 wk", "ASD"): (1.35, -1.89, 4.60),
("<37 wk", "Non-verbal IQ"): (-3.58, -5.67, -1.49),
("<2500 g", "ADHD"): (5.15, 4.14, 6.16),
("<2500 g", "ASD"): (3.71, 0.82, 6.60),
("<2500 g", "Non-verbal IQ"): (-5.68, -7.71, -3.65),
}
print("\n=== 7. Basic model (Table S5): could the abstract values come from it? ===")
for (row, out), (e, l, h) in S5.items():
se, z, p = wald(e, l, h)
a = dict((x[0], x[1:]) for x in ABSTRACT)[out]
print(f" {row:8s} {out:14s} {fmt(e)} ({fmt(l)}, {fmt(h)}) z {z:+.2f} p {p:.4f} "
f"equals abstract value: {(e, l, h) == a}")
# --------------- 8. complete-case analysis (Appendix A, Table S9) as a cross-check
S9 = {
("<37 wk", "ADHD"): (2.40, 1.24, 3.56),
("<37 wk", "ASD"): (-0.50, -4.41, 3.41),
("<37 wk", "Non-verbal IQ"): (-2.87, -5.28, -0.45),
("<2500 g", "ADHD"): (4.37, 3.02, 5.72),
("<2500 g", "ASD"): (2.11, -1.28, 5.50),
("<2500 g", "Non-verbal IQ"): (-4.53, -6.90, -2.17),
}
print("\n=== 8. Complete-case analysis (Table S9): could the abstract values come from it? ===")
for (row, out), (e, l, h) in S9.items():
se, z, p = wald(e, l, h)
a = dict((x[0], x[1:]) for x in ABSTRACT)[out]
print(f" {row:8s} {out:14s} {fmt(e)} ({fmt(l)}, {fmt(h)}) z {z:+.2f} p {p:.4f} "
f"equals abstract value: {(e, l, h) == a}")
# ================================================================ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "path",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
print(f" wrote {stem}.svg and {stem}.png")
print("\n=== Figures ===")
# ---- fig1: abstract vs Table 4 / Results diagram (ASD)
fig = plt.figure(figsize=(7.2, 5.8))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 80)
ax.axis("off")
renderer = fig.canvas.get_renderer()
def panel(x, y, w, h, title):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=1.2",
fc="white", ec="#c9c7c1", lw=1.2))
ax.text(x + 1.8, y + h - 2.6, title, fontsize=11, fontweight="bold", va="top")
def runs(x, y, parts):
"""Draw text segments one after another on a line; return each segment's extent.
A part is (text, kwargs) or (text, kwargs, gap) where gap is extra space before it."""
ends = []
for part in parts:
txt, kw = part[0], part[1]
x += part[2] if len(part) > 2 else 0.0
t = ax.text(x, y, txt, fontsize=10.5, va="top", **kw)
bb = t.get_window_extent(renderer).transformed(ax.transData.inverted())
x = bb.x1
ends.append((bb.x0, bb.x1, bb.y0))
return ends
# Abstract panel (top)
panel(2, 55, 96, 23, "Abstract, Findings")
runs(3.8, 71.0, [("\u201cAs compared to term birth, preterm birth was associated with higher", {})])
runs(3.8, 67.2, [("internalizing problems (\u2026) and externalizing problems (\u2026), ADHD symptoms (\u2026),", {})])
seg = runs(3.8, 63.4, [("ASD symptoms (difference", {}),
("3\u00b723 (95% CI: 0\u00b737, 6\u00b708)", dict(color=ORANGE, fontweight="bold"), -0.65),
("), and lower", {}, -0.7)])
runs(3.8, 59.6, [("non-verbal IQ (\u2026).\u201d", {})])
val_x0, val_x1, val_y0 = seg[1]
# Table 4 panel (bottom left)
panel(2, 8, 56, 33, "Table 4, ASD symptoms (childhood)")
runs(3.8, 33.6, [("Gestational age <37 wk (preterm)", dict(fontweight="bold"))])
runs(3.8, 29.8, [("0\u00b760 (\u22122\u00b760, 3\u00b779), n = 279, no asterisk", {})])
runs(3.8, 26.0, [(f"recomputed: z = {z_pt:.2f}, p = {p_pt:.2f}", dict(color=BLUE, fontweight="bold"))])
runs(3.8, 20.0, [("Birth weight <2500 g (low birth weight)", dict(fontweight="bold"))])
lbw = runs(3.8, 16.2, [("3\u00b723 (0\u00b737, 6\u00b708)*, N = 353", {})])
runs(3.8, 12.4, [(f"recomputed: z = {z_lb:.2f}, p = {p_lb:.3f}", dict(color=BLUE, fontweight="bold"))])
# Results / Discussion panel (bottom right)
panel(61, 8, 37, 33, "Results and Discussion")
for i, line in enumerate(["\u201cNot preterm birth, but low birth", "weight was associated with ASD",
"symptoms (3\u00b723 percentiles", "(95% CI: 0\u00b737, 6\u00b708)).\u201d"]):
runs(62.8, 33.6 - 3.8 * i, [(line, {})])
runs(62.8, 16.2, [("\u201cThe lack of association for", {})])
runs(62.8, 12.4, [("preterm birth \u2026\u201d (Discussion)", {})])
# connector: abstract value -> low-birth-weight row, routed down the left panel's right margin
y_lbw = 16.2 - 1.6
vx = (val_x0 + val_x1) / 2
x_lane, y_jog = 59.5, 50.0
ax.plot([vx, vx, x_lane, x_lane], [val_y0 - 0.6, y_jog, y_jog, y_lbw], color=INK, lw=2,
solid_joinstyle="round", solid_capstyle="round")
ax.add_patch(FancyArrowPatch((x_lane, y_lbw), (lbw[0][1] + 1.0, y_lbw), arrowstyle="-|>",
mutation_scale=14, lw=2, color=INK))
ax.text(x_lane + 2.0, 46.5, "identical estimate and CI:\nTable 4 low-birth-weight row",
fontsize=10, ha="left", va="center", color=INK)
ax.text(3.0, 46.5, "Labelled \u2018preterm birth\u2019 in the abstract;\nthe preterm row is 0\u00b760, p = 0\u00b771",
fontsize=10, ha="left", va="center", color=INK2)
ax.text(2.0, 4.2, "Orange: as reported in the abstract. Blue: recomputed from the printed 95% CI "
"(SE = width/3.92).", fontsize=10, ha="left", va="center", color=INK2)
save(fig, "fig1")
# ---- fig2: dot-and-interval, abstract 'preterm' values vs Table preterm cells
fig, ax = plt.subplots(figsize=(7.2, 4.6))
fig.subplots_adjust(left=0.20, right=0.68, top=0.86, bottom=0.14)
names = [a[0] for a in ABSTRACT]
src = {
"Internalizing": "= <37 wk, childhood",
"Externalizing": "= <37 wk, early adol.",
"ADHD": "= <2500 g row",
"ASD": "= <2500 g row",
"Non-verbal IQ": "= <2500 g row",
}
ypos = np.arange(len(names))[::-1] * 1.0
off = 0.17
for y, (name, e, l, h) in zip(ypos, ABSTRACT):
tab = "T3" if name in ("Internalizing", "Externalizing") else "T4"
pe, pl, ph, pn = T[(tab, "<37 wk", name, "childhood")]
p_pre = REC[(tab, "<37 wk", name, "childhood")][2]
ax.plot([l, h], [y + off] * 2, color=ORANGE, lw=2, solid_capstyle="round")
ax.plot(e, y + off, "o", color=ORANGE, ms=8, mec=BG, mew=1.5)
ax.plot([pl, ph], [y - off] * 2, color=BLUE, lw=2, solid_capstyle="round")
ax.plot(pe, y - off, "s", color=BLUE, ms=8, mec=BG, mew=1.5)
ptxt = "p < 0.001" if p_pre < 0.001 else f"p = {p_pre:.3f}" if p_pre < 0.01 else f"p = {p_pre:.2f}"
ax.annotate(f"{e:+.2f} {src[name]}".replace("-", "\u2212", 1), xy=(1.02, y + off), xycoords=("axes fraction", "data"),
fontsize=10, va="center", color=INK, annotation_clip=False)
ax.annotate(f"{pe:+.2f} {ptxt}".replace("-", "\u2212", 1), xy=(1.02, y - off), xycoords=("axes fraction", "data"),
fontsize=10, va="center", color=INK2, annotation_clip=False)
ax.axvline(0, color=INK2, lw=0.8, zorder=0)
ax.set_yticks(ypos)
ax.set_yticklabels(names)
ax.set_ylim(ypos.min() - 0.6, ypos.max() + 0.6)
ax.set_xlim(-8.5, 7)
ax.set_xticks(np.arange(-8, 7, 2))
ax.set_xlabel("Difference vs term birth, outcome percentiles (95% CI)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#c9c7c1")
ax.tick_params(axis="y", length=0)
h1, = ax.plot([], [], "o-", color=ORANGE, ms=8, lw=2,
label="Abstract, stated as preterm birth (reported)")
h2, = ax.plot([], [], "s-", color=BLUE, ms=8, lw=2,
label="Table 3/4 preterm <37 wk, childhood; p recomputed from CI")
fig.legend(handles=[h1, h2], loc="upper left", bbox_to_anchor=(0.01, 0.995), frameon=False,
fontsize=10, ncol=1)
ax.annotate("source of abstract value\n/ preterm cell p", xy=(1.02, ypos.max() + 0.55),
xycoords=("axes fraction", "data"), fontsize=10, color=INK2, va="bottom",
annotation_clip=False)
save(fig, "fig2")
Reported overall 90-day mortality of 63.7% conflicts with the paper's own counts of 408 deaths
Lancet Reg Health Eur (2025) · doi:10.1016/j.lanepe.2025.101317 · PMC12266187
Summary
The abstract and the Results give the overall 90-day mortality of the 1109 included patients as "63.7% (n = 707)". Every breakdown of those same 1109 patients in the paper adds up to 408 deaths (36.8%) and 701 survivors (63.2%). The breakdowns are the three PROMISE risk groups, the two GRIm risk groups, and the PROMISE and GRIm death counts in the four cancer-type subgroups. The reported median overall survival of 4.78 months is longer than 90 days, which on its own requires 90-day mortality below 50%. The printed figure therefore overstates the cohort's primary-outcome event rate by 299 deaths, about 27 percentage points. It is also used in the Discussion to argue that mortality was higher than in the cohort in which PROMISE was developed.
What the paper reports
The headline figure, in the abstract:
PROMISE and GRIm scores were calculated for all 1109 patients, while CTI score was assessed in 333 patients with complete laboratory data. The 90-day mortality rate was 63.7% (n = 707).
— Abstract, Findings
The same figure opens the outcome paragraph of the Results, followed directly by the stratified counts:
The overall 90-day mortality rate was 63.7% (n = 707). For the PROMISE score, 271 patients (97.5%) in the low-risk group survived, with a 90-day mortality rate of 2.5% (n = 7). In the intermediate-risk group, 176 patients (81.9%) survived, and the 90-day mortality rate was 18.1% (n = 39). In the high-risk group, 254 patients (41.2%) survived, with a 90-day mortality rate of 58.8% (n = 362) (p < 0.0001). According to the PROMISE score, the median overall survival (OS) for the entire cohort was 4.78 months (95% CI: 4.29–5.28). For the GRIm score, the 90-day survival rate was 87.8% (n = 449) for low-risk patients and 42.2% (n = 252) for high-risk patients (p < 0.0001). The estimated median OS for the entire cohort was 4.63 months (95% CI: 4.14–5.12).
— Results, overall outcome paragraph
The risk-group sizes:
According to the PROMISE score, 278 patients (25.1%) were categorized as low risk, 215 patients (19.4%) as intermediate risk, and 616 patients (55.5%) as high risk. The GRIm score classified 511 patients (46.0%) as low risk and 598 patients (54.0%) as high risk.
— Results, patient characteristics paragraph (Table 1 column headers give the GRIm groups as "Low risk, n = 510 (46.0%) | High risk, n = 599 (54.0%)")
The cancer-type subgroups (PROMISE and GRIm sentences only; the CTI sentences refer to the 333-patient subset):
For lung cancer patients (n = 283), mortality rates based on the PROMISE score were 2.4% (n = 1) in the low-risk group, 3.6% (n = 2) in the intermediate-risk group, and 51.9% (n = 96) in the high-risk group (p < 0.0001). [...] The GRIm score showed a 7.9% (n = 9) mortality rate in the low-risk group and 53.3% (n = 90) in the high-risk group (p < 0.0001).
For gastrointestinal cancer patients (n = 350), the PROMISE score demonstrated mortality rates of 6.0% (n = 5) in the low-risk group, 23.7% (n = 18) in the intermediate-risk group, and 63.2% (n = 120) in the high-risk group (p < 0.0001). [...] The GRIm scores indicated a 13.6% (n = 20) mortality rate in the low-risk group compared to 60.6% (n = 123) in the high-risk group (p < 0.0001).
For breast cancer patients (n = 92), the PROMISE score revealed 90-day mortality rates of 2.8% (n = 1) in the low-risk group, 21.4% (n = 3) in the intermediate-risk group, and 59.5% (n = 25) in the high-risk group (p < 0.0001). [...] The GRIm score demonstrated mortality rates of 6.8% (n = 3) in the low-risk group compared to 54.2% (n = 26) in the high-risk group (p < 0.0001).
For patients with other cancers (n = 384), PROMISE score analysis showed no 90-day mortality in the low-risk group (n = 0), a mortality rate of 23.2% (n = 16) in the intermediate-risk group, and 60.8% (n = 121) in the high-risk group (p < 0.0001). [...] The GRIm scores revealed a 14.6% (n = 30) mortality rate in the low-risk group and 59.8% (n = 107) in the high-risk group (p < 0.0001).
— Results, section "Subgroup analysis by cancer type" ([...] marks omitted median-OS sentences)
The separately reported subset of 333 patients with a CTI score:
In this study, the 90-day mortality rate among 333 patients assessed using the CTI score was 35.1% (n = 117).
— Results, CTI subset paragraph
The Discussion:
According to the results of our study, the 90-day mortality rate was higher compared to the study by Mirallas et al., in which the PROMISE score was developed.
In the study by Mirallas et al., survival rates for low, intermediate, and high-risk groups were reported as 83.6%, 68.0%, and 33.4%, respectively.
— Discussion
Each quotation was checked against the article as archived in PubMed Central (src/show.py PMC12266187). Each one occurs exactly once (passages joined by [...] were checked sentence by sentence), and recompute.py.txt reruns that check.
What the paper's own numbers show
Method. The paper states that PROMISE and GRIm were calculated for all 1109 patients, and the PROMISE group sizes (278 + 215 + 616) and the cancer-type subgroup sizes (283 + 350 + 92 + 384) each sum to 1109. Each of these breakdowns is therefore exhaustive, and summing its death counts gives the cohort's total number of 90-day deaths. For PROMISE the paper gives deaths and survivors per group. For GRIm it gives survivors per group, so deaths are group size minus survivors, using either the text's or Table 1's group sizes. For the cancer types it gives deaths per risk group under each score. As an independent check, the Kaplan–Meier median OS is compared with the 90-day time point. The 333-patient CTI subset is recomputed the same way. No number was estimated; every input is a count or percentage printed in the text or Table 1. recompute.py.txt prints every value below and redraws both figures.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Overall 90-day mortality, 1109 patients | Abstract, Findings; Results, outcome paragraph | 63.7% (n = 707) | 408 deaths, 36.8% (every row below). Arithmetic check: 707/1109 = 63.75% |
| PROMISE deaths, low + intermediate + high | Results, outcome paragraph | 7, 39, 362 (2.5%, 18.1%, 58.8% of 278, 215, 616) | 7 + 39 + 362 = 408. Group percentages reproduce exactly |
| PROMISE survivors | Results, outcome paragraph | 271, 176, 254 | 271 + 176 + 254 = 701; 408 + 701 = 1109 |
| GRIm deaths (group size − survivors) | Results, outcome paragraph; Table 1 | survivors 449 (87.8%), 252 (42.2%); n 510/599 (Table 1) or 511/598 (text) | (510 − 449) + (599 − 252) = 61 + 347 = 408; with text n, 62 + 346 = 408. Survivors 449 + 252 = 701 |
| Deaths by cancer type, PROMISE | Results, "Subgroup analysis by cancer type" | lung 1+2+96, GI 5+18+120, breast 1+3+25, other 0+16+121 | 99 + 143 + 29 + 137 = 408 (n = 1109) |
| Deaths by cancer type, GRIm | Results, "Subgroup analysis by cancer type" | lung 9+90, GI 20+123, breast 3+26, other 30+107 | 99 + 143 + 29 + 137 = 408; per cancer type identical to PROMISE |
| Median OS vs 90-day mortality | Results, outcome paragraph | median OS 4.78 months ("According to the PROMISE score"); 4.63 months (stated after the GRIm results) | Both exceed 90 days (2.96 months), so Kaplan–Meier survival at 90 days is above 50% and 90-day mortality is below 50%. 707 deaths within 90 days would force Kaplan–Meier mortality at 90 days to at least 63.8% and the median below 90 days |
| CTI subset, 333 patients | Results, CTI subset paragraph | 35.1% (n = 117) | 117/333 = 35.1%. Also (166 − 147) + (167 − 69) = 117; PROMISE within subset 3 + 12 + 102 = 117; Combined score 0 + 1 + 116 = 117; CTI by cancer type 21 + 40 + 20 + 36 = 117. Consistent |
| Gap between stated and recomputed | — | 707 (63.7%) | 707 − 408 = 299 deaths; 63.75 − 36.79 = 27.0 percentage points |
Four exhaustive breakdowns, drawn from two scores and two different groupings of the same patients, agree on 408 deaths. They also agree on 701 survivors where survivors are given. The subset of 333 patients, reported the same way, is fully consistent, so the reporting of the stratified counts can be reproduced throughout. Only the single overall figure departs from them. Its percentage, 63.7%, is close to the share who survived (701/1109 = 63.2%), not the share who died.
Figure
Figure 1. Number of the 1109 included patients who died within 90 days. The orange bar is the overall figure printed in the abstract and Results (707, "63.7%"). Each blue bar sums the death counts the paper itself reports for one exhaustive breakdown of the same 1109 patients: the three PROMISE risk groups, the two GRIm risk groups (group size from Table 1 minus reported survivors), and the PROMISE and GRIm deaths in the four cancer-type subgroups. Segments are drawn in the order the formula on each row lists them; unlabeled thin segments are 7, 39 and 29. The dashed line marks the cohort size. All four breakdowns give 408 deaths (36.8%). Values come from the article text and Table 1 and are recomputed by recompute.py.txt.
Figure 2. The same comparison as percentages on one 0–100% axis. The printed overall mortality (orange, 63.7%) lies in the shaded region that the paper's own median OS of 4.78 months rules out. A Kaplan–Meier median longer than 90 days means fewer than half the patients had died by day 90. The mortality recomputed from the strata (36.8%) lies outside it. The printed value sits next to the recomputed survival share (blue diamond, 63.2%). The bottom row shows the separately reported CTI subset, where the reported value (orange ring) and the recomputed value (blue dot) coincide at 35.1%.
Why it matters
- The abstract misstates the event rate of the primary outcome. 90-day mortality is the outcome the scores are validated against. A reader taking the abstract at face value would conclude that nearly two in three patients with an unplanned oncology admission died within three months. The paper's own data indicate a little over one in three.
- The Discussion's comparison with the development cohort rests on it. The paper says 90-day mortality "was higher compared to the study by Mirallas et al." and attributes the difference to access to palliative care and to palliative patients being admitted to oncology wards. That comparison and its explanation should be re-examined with 36.8%. The stratum-level survival figures the Discussion quotes for the development study (83.6%, 68.0%, 33.4%) are each lower than this cohort's (97.5%, 81.9%, 41.2%). The paper does not report the development cohort's overall rate, so whether any difference remains cannot be judged from this article.
- Baseline risk used by readers. Anyone using the abstract as a baseline risk for this setting, for example to turn sensitivity and specificity into predictive values for local use, would start from the wrong figure.
- What is not affected by this error. The stratum-level mortality percentages, the Kaplan–Meier comparisons and the 333-patient CTI-subset counts do not use the 707 figure and are internally consistent. The direction of the main finding, that PROMISE and CTI separate low-risk from high-risk patients, does not depend on it.
- A separate observation, not part of this finding. Table 2 (n = 333) reports sensitivity, specificity, PPV, NPV and accuracy for each score. With the subset's own prevalence of 35.1% (117/333), the reported sensitivity and specificity imply different predictive values. For PROMISE, for example, they imply PPV 57.4%, NPV 91.3% and accuracy 72.9%, against the reported 78.0%, 83.0% and 79.9%. Taken one at a time, the reported PPV, NPV and accuracy correspond to prevalences of roughly 50–65%, not 35.1%. Table 2 gives no 2×2 counts, so the source of this cannot be determined. It is recorded here only so that Table 2 is not assumed to be unaffected;
recompute.py.txtprints the calculation (section 9).
Explanations considered
- A different denominator or population. Does not account for it. The 707 is attached to the full cohort in the abstract ("all 1109 patients") and the Results ("overall"). The PROMISE, GRIm and cancer-type breakdowns each cover exactly 1109 patients. The CTI subset has its own figure (117 of 333), which is consistent.
- Survivors labelled as deaths. Possibly related, but not an exact match. The paper's survivors total 701 (63.2%), not 707 (63.7%), so swapping the labels does not reproduce the printed numbers exactly. The closeness of 63.7% to 63.2% is consistent with a survival-type figure, or a mistyped count, having been entered in place of the mortality figure. The article does not allow this to be settled.
- A different time point (for example, deaths over the whole follow-up). Does not account for it. Both the abstract and the Results say "90-day mortality". The stratified counts that sum to 408 are explicitly 90-day counts. A median OS of 4.78 months (or 4.63 months) is incompatible with more than half the cohort dying by day 90.
- Patients excluded from the stratified analyses, for example censored before 90 days. Does not account for it. The stratified breakdowns account for all 1109 patients as either deaths or survivors (408 + 701 = 1109), leaving no patients unclassified. No patients are left over (1109 − 408 − 701 = 0) to supply 299 additional deaths.
- GRIm group sizes differ between text and Table 1 (511/598 vs 510/599). A minor separate inconsistency that does not affect the result: either version gives 408 deaths.
- Rounding. Does not account for it. Rounding only concerns the last digit: 707/1109 = 63.75%, which rounds to 63.8% rather than the printed 63.7%. The 27-point gap is unaffected.
- Conversion or typesetting artifact. Ruled out. The figures are plain prose and identical in the publisher-deposited JATS XML (Europe PMC) and the live PMC article. They involve no symbols, superscripts or table cells.
- A published correction. None found (see Status).
Status
- Version of record. The Elsevier and Lancet pages (linkinghub, ScienceDirect, thelancet.com HTML and PDF) returned HTTP 403 or a redirect page to scripted access, so the typeset page could not be read directly. The publisher-deposited JATS XML in Europe PMC and the live PMC article contain the same sentences in the abstract and the Results. The publisher's own Figure 2 image (from the Europe PMC supplementary bundle) gives numbers at risk at 3 months of 271/176/254 for PROMISE and 449/252 for GRIm, which sum to 701 in each case. These are consistent with 408 patients no longer at risk by about 90 days, not 707. That figure is cited here, not reproduced.
- Supplement. The single supplementary file (mmc1.docx, "Supplementary Figures S1–S6 and Table S1") was read in full. It contains univariable odds ratios, calibration statistics, Kaplan–Meier curves by cancer type, and baseline characteristics by CTI availability (333 vs 776). It reports no overall 90-day death count and does not resolve the discrepancy. No dataset is deposited.
- Corrections and notices. No correction, erratum, retraction or expression of concern was found. Crossref has no update-to or updated-by relations, and a Crossref
updatesfilter search returned none. PMC and Europe PMC list no notices; Europe PMC lists only the article (PMID 40672056) and a linked commentary in the same journal (PMC12173626). - PubPeer. PubPeer's search and API return a human-verification challenge to scripted access, so it could not be queried directly. A web search for the title and DOI found no PubPeer thread.
- Sources. Article as archived in PMC (PMC12266187); Europe PMC full-text XML and supplementary files; Crossref record for doi:10.1016/j.lanepe.2025.101317.
- As of 2026-10-23.
Suggested correction
In the abstract (Findings) and in the first sentence of the Results outcome paragraph, replace "63.7% (n = 707)" with the figure supported by the stratified data, which on the paper's counts is 36.8% (n = 408), after confirming it against the source dataset. In the Discussion, revisit the statement that 90-day mortality was higher than in the PROMISE development study, and the explanation offered for it, in light of the corrected rate. Two further points could be checked at the same time. One is the GRIm group sizes (511/598 in the text vs 510/599 in Table 1). The other is the predictive values in Table 2, which do not correspond to the 35.1% event rate of the 333-patient subset; giving the underlying 2×2 counts would let readers verify them.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12266187 (Lancet Reg Health Eur 2025, doi:10.1016/j.lanepe.2025.101317)
# Overall 90-day mortality "63.7% (n = 707)" versus the paper's own stratified counts.
#
# Run: python recompute.py.txt
# Prints every number used in writeup.md and regenerates fig1.{svg,png} and fig2.{svg,png}
# in the folder that holds this file. Needs matplotlib.
#
# Inputs: every count and percentage below is copied from the article text or Table 1
# as archived in PubMed Central (PMC12266187). When run inside the /home/ubuntu/papers
# repo, the script re-checks each quoted passage against the archived full text with
# src/show.py (skipped elsewhere). Nothing is estimated or read off an image.
import os, subprocess
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ quotes
QUOTES = [
("Abstract, Findings",
"PROMISE and GRIm scores were calculated for all 1109 patients, while CTI score was "
"assessed in 333 patients with complete laboratory data. The 90-day mortality rate was "
"63.7% (n = 707)."),
("Results, overall outcome paragraph",
"The overall 90-day mortality rate was 63.7% (n = 707). For the PROMISE score, 271 patients "
"(97.5%) in the low-risk group survived, with a 90-day mortality rate of 2.5% (n = 7). In the "
"intermediate-risk group, 176 patients (81.9%) survived, and the 90-day mortality rate was "
"18.1% (n = 39). In the high-risk group, 254 patients (41.2%) survived, with a 90-day "
"mortality rate of 58.8% (n = 362) (p < 0.0001). According to the PROMISE score, the median "
"overall survival (OS) for the entire cohort was 4.78 months (95% CI: 4.29–5.28). For the GRIm "
"score, the 90-day survival rate was 87.8% (n = 449) for low-risk patients and 42.2% (n = 252) "
"for high-risk patients (p < 0.0001). The estimated median OS for the entire cohort was 4.63 "
"months (95% CI: 4.14–5.12)."),
("Results, patient characteristics paragraph",
"According to the PROMISE score, 278 patients (25.1%) were categorized as low risk, 215 patients "
"(19.4%) as intermediate risk, and 616 patients (55.5%) as high risk. The GRIm score classified "
"511 patients (46.0%) as low risk and 598 patients (54.0%) as high risk."),
("Table 1, column headers",
"Low risk, n = 510 (46.0%) | High risk, n = 599 (54.0%)"),
("Results, Subgroup analysis by cancer type (lung)",
"For lung cancer patients (n = 283), mortality rates based on the PROMISE score were 2.4% "
"(n = 1) in the low-risk group, 3.6% (n = 2) in the intermediate-risk group, and 51.9% (n = 96) "
"in the high-risk group (p < 0.0001)."),
("Results, Subgroup analysis by cancer type (lung)",
"The GRIm score showed a 7.9% (n = 9) mortality rate in the low-risk group and 53.3% (n = 90) "
"in the high-risk group (p < 0.0001)."),
("Results, Subgroup analysis by cancer type (gastrointestinal)",
"For gastrointestinal cancer patients (n = 350), the PROMISE score demonstrated mortality rates "
"of 6.0% (n = 5) in the low-risk group, 23.7% (n = 18) in the intermediate-risk group, and 63.2% "
"(n = 120) in the high-risk group (p < 0.0001)."),
("Results, Subgroup analysis by cancer type (gastrointestinal)",
"The GRIm scores indicated a 13.6% (n = 20) mortality rate in the low-risk group compared to "
"60.6% (n = 123) in the high-risk group (p < 0.0001)."),
("Results, Subgroup analysis by cancer type (breast)",
"For breast cancer patients (n = 92), the PROMISE score revealed 90-day mortality rates of 2.8% "
"(n = 1) in the low-risk group, 21.4% (n = 3) in the intermediate-risk group, and 59.5% (n = 25) "
"in the high-risk group (p < 0.0001)."),
("Results, Subgroup analysis by cancer type (breast)",
"The GRIm score demonstrated mortality rates of 6.8% (n = 3) in the low-risk group compared to "
"54.2% (n = 26) in the high-risk group (p < 0.0001)."),
("Results, Subgroup analysis by cancer type (other cancers)",
"For patients with other cancers (n = 384), PROMISE score analysis showed no 90-day mortality "
"in the low-risk group (n = 0), a mortality rate of 23.2% (n = 16) in the intermediate-risk "
"group, and 60.8% (n = 121) in the high-risk group (p < 0.0001)."),
("Results, Subgroup analysis by cancer type (other cancers)",
"The GRIm scores revealed a 14.6% (n = 30) mortality rate in the low-risk group and 59.8% "
"(n = 107) in the high-risk group (p < 0.0001)."),
("Results, CTI subset paragraph",
"In this study, the 90-day mortality rate among 333 patients assessed using the CTI score was "
"35.1% (n = 117)."),
("Results, overall outcome paragraph (CTI)",
"Similarly, in CTI score assessments, low-risk patients demonstrated an 88.6% survival rate "
"(n = 147), while high-risk patients had a 41.3% survival rate (n = 69) (p < 0.0001)."),
("Results, patient characteristics paragraph (CTI)",
"Among the 333 patients evaluated using the CTI score, 166 patients (49.8%) were categorized as "
"low risk and 167 patients (50.2%) as high risk."),
("Discussion",
"According to the results of our study, the 90-day mortality rate was higher compared to the "
"study by Mirallas et al., in which the PROMISE score was developed."),
("Discussion",
"In the study by Mirallas et al., survival rates for low, intermediate, and high-risk groups "
"were reported as 83.6%, 68.0%, and 33.4%, respectively."),
]
def check_quotes():
repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
print("Quote check: src/show.py not available here; skipped.\n")
return
try:
full = subprocess.run([py, "src/show.py", "PMC12266187", "--full"], cwd=repo,
capture_output=True, text=True, timeout=300).stdout
except Exception as e:
print(f"Quote check: could not load archived text ({e}); skipped.\n")
return
norm = lambda s: " ".join(s.split())
full_n = norm(full)
print("Quote check against the PMC-archived full text (occurrences):")
for loc, q in QUOTES:
n = full_n.count(norm(q))
print(f" {n} {loc}: {q[:70]}...")
print()
# ------------------------------------------------------------------ data (from the paper)
N = 1109 # included patients (Abstract; Table 1)
STATED_DEATHS, STATED_PCT = 707, 63.7
# PROMISE strata: (group size, survivors, deaths, reported mortality %)
PROMISE = {"low": (278, 271, 7, 2.5), "intermediate": (215, 176, 39, 18.1),
"high": (616, 254, 362, 58.8)}
# GRIm strata: survivors and reported survival %; group sizes as given in text and in Table 1
GRIM_SURV = {"low": (449, 87.8), "high": (252, 42.2)}
GRIM_N_TEXT = {"low": 511, "high": 598}
GRIM_N_TABLE1 = {"low": 510, "high": 599}
# Cancer-type subgroups: n, PROMISE deaths (low, int, high), GRIm deaths (low, high),
# CTI subgroup n, CTI deaths (low, high)
CANCER = {
"Lung": (283, (1, 2, 96), (9, 90), 62, (3, 18)),
"Gastrointestinal": (350, (5, 18, 120), (20, 123), 95, (8, 32)),
"Breast": (92, (1, 3, 25), (3, 26), 54, (4, 16)),
"Other": (384, (0, 16, 121), (30, 107), 122, (4, 32)),
}
# CTI subset (333 patients)
CTI_N, CTI_DEATHS, CTI_PCT = 333, 117, 35.1
CTI_GROUPS = {"low": (166, 147), "high": (167, 69)} # (n, survivors)
CTI_PROMISE = {"low": (114, 111, 3), "intermediate": (64, 52, 12), "high": (155, 53, 102)}
COMBINED_DEATHS = (0, 1, 116)
# Median OS for the full cohort (months)
MEDIAN_OS = {"PROMISE analysis": 4.78, "GRIm analysis": 4.63}
# Table 2 (n = 333): sensitivity, specificity, PPV, NPV, accuracy (%)
TABLE2 = {"PROMISE Score": (88.7, 64.3, 78.0, 83.0, 79.9),
"CTI Score": (88.2, 67.5, 79.1, 81.4, 81.0),
"PROMISE-CTI Combined score": (92.4, 81.1, 85.3, 89.6, 86.7)}
# Development study survival by PROMISE stratum, as quoted in the Discussion
MIRALLAS_SURV = (83.6, 68.0, 33.4)
def pct(a, b):
return 100.0 * a / b
def main():
check_quotes()
print("1. Stated overall figure")
print(f" 707/1109 = {pct(707, N):.2f}% (reported 63.7%)")
print("\n2. PROMISE strata (cover all 1109 patients)")
nsum = sum(v[0] for v in PROMISE.values())
for k, (n, s, d, rp) in PROMISE.items():
print(f" {k:12s} n={n}: survivors+deaths={s + d}; mortality {pct(d, n):.1f}% "
f"(reported {rp}%); survival {pct(s, n):.1f}%")
P_D = sum(v[2] for v in PROMISE.values())
P_S = sum(v[1] for v in PROMISE.values())
print(f" group sizes sum to {nsum}")
print(f" deaths 7+39+362 = {P_D}; survivors 271+176+254 = {P_S}; total {P_D + P_S}")
print(f" mortality {P_D}/{N} = {pct(P_D, N):.2f}%; survival {P_S}/{N} = {pct(P_S, N):.2f}%")
print("\n3. GRIm strata (deaths = group size - survivors)")
G_S = sum(v[0] for v in GRIM_SURV.values())
for label, ns in (("text", GRIM_N_TEXT), ("Table 1", GRIM_N_TABLE1)):
dl, dh = ns["low"] - GRIM_SURV["low"][0], ns["high"] - GRIM_SURV["high"][0]
print(f" n from {label} {ns['low']}/{ns['high']}: survival "
f"{pct(449, ns['low']):.1f}%/{pct(252, ns['high']):.1f}% (reported 87.8%/42.2%); "
f"deaths {dl}+{dh} = {dl + dh}")
print(f" survivors 449+252 = {G_S}; deaths 1109-{G_S} = {N - G_S}")
GT_D = (GRIM_N_TABLE1["low"] - 449, GRIM_N_TABLE1["high"] - 252)
print("\n4. Cancer-type subgroups")
print(f" n: 283+350+92+384 = {sum(v[0] for v in CANCER.values())}")
for k, (n, pd, gd, cn, cd) in CANCER.items():
print(f" {k:16s} n={n}: PROMISE deaths {'+'.join(map(str, pd))} = {sum(pd)}; "
f"GRIm deaths {'+'.join(map(str, gd))} = {sum(gd)}; CTI deaths "
f"{'+'.join(map(str, cd))} = {sum(cd)} of {cn}")
CP = [sum(v[1]) for v in CANCER.values()]
CG = [sum(v[2]) for v in CANCER.values()]
print(f" PROMISE total {'+'.join(map(str, CP))} = {sum(CP)}; "
f"GRIm total {'+'.join(map(str, CG))} = {sum(CG)}")
print(f" CTI subgroup n sum = {sum(v[3] for v in CANCER.values())}; CTI deaths sum = "
f"{sum(sum(v[4]) for v in CANCER.values())}")
print("\n5. CTI subset (333 patients) - internally consistent")
cd = [n - s for n, s in CTI_GROUPS.values()]
print(f" 117/333 = {pct(117, 333):.2f}% (reported 35.1%)")
print(f" CTI groups: (166-147)+(167-69) = {cd[0]}+{cd[1]} = {sum(cd)}")
print(f" PROMISE within subset: 3+12+102 = {sum(v[2] for v in CTI_PROMISE.values())}; "
f"n 114+64+155 = {sum(v[0] for v in CTI_PROMISE.values())}")
print(f" Combined score: 0+1+116 = {sum(COMBINED_DEATHS)}")
print("\n6. Median OS versus 90-day mortality")
m90 = 90 / 30.4375
for k, v in MEDIAN_OS.items():
print(f" {k}: median OS {v} months > 90 days ({m90:.2f} months): {v > m90}")
print(" A Kaplan-Meier median above 90 days means S(90 d) > 0.5, i.e. 90-day mortality < 50%.")
print(" The KM estimate of mortality by day 90 is never below (deaths observed by day 90)/N,")
print(f" so 707 deaths would force KM mortality >= {pct(707, N):.1f}% and a median <= 90 days.")
print("\n7. Gap between stated and recomputed")
print(f" 707 - 408 = {707 - 408} deaths; {pct(707, N):.2f} - {pct(408, N):.2f} = "
f"{pct(707, N) - pct(408, N):.1f} percentage points")
print(f" survivors 701 ({pct(701, N):.1f}%) vs stated 707 ({pct(707, N):.1f}%): differ by 6")
print(f" 1109 - 707 = {N - 707}")
print("\n8. Discussion comparison with the development study (stratum survival, %)")
here = [round(pct(v[1], v[0]), 1) for v in PROMISE.values()]
for g, a, b in zip(("low", "intermediate", "high"), here, MIRALLAS_SURV):
print(f" {g:12s}: this cohort {a}% vs development study {b}%")
print("\n9. Table 2 (n = 333): prevalence implied by each reported value")
p = CTI_DEATHS / CTI_N
for k, (se, sp, ppv, npv, acc) in TABLE2.items():
se, sp, ppv, npv, acc = se / 100, sp / 100, ppv / 100, npv / 100, acc / 100
p_ppv = (1 - sp) * ppv / (se * (1 - ppv) + (1 - sp) * ppv)
p_npv = (1 - npv) * sp / ((1 - npv) * sp + npv * (1 - se))
p_acc = (acc - sp) / (se - sp)
e_ppv = se * p / (se * p + (1 - sp) * (1 - p))
e_npv = sp * (1 - p) / (sp * (1 - p) + (1 - se) * p)
e_acc = se * p + sp * (1 - p)
print(f" {k}: implied prevalence from PPV {100*p_ppv:.1f}%, NPV {100*p_npv:.1f}%, "
f"accuracy {100*p_acc:.1f}%")
print(f" expected at 35.1% prevalence: PPV {100*e_ppv:.1f}, NPV {100*e_npv:.1f}, "
f"accuracy {100*e_acc:.1f} (reported {ppv*100:.1f}, {npv*100:.1f}, {acc*100:.1f})")
make_figures(P_D, GT_D, CP, CG)
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE = "#eb6834", "#2a78d6"
GRID = "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"axes.edgecolor": "#b9b8b3", "svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, bbox_inches="tight", **kw)
plt.close(fig)
print(f" wrote {name}.svg, {name}.png")
def make_figures(P_D, GT_D, CP, CG):
print("\nFigures")
# ---- fig1: death counts, stated vs sums of the paper's breakdowns
rows = [
("Stated overall deaths\n(Abstract; Results)", [STATED_DEATHS], ORANGE, ["707"]),
("PROMISE risk groups\n7 + 39 + 362 (low, int., high)",
[v[2] for v in PROMISE.values()], BLUE, ["7", "39", "362"]),
("GRIm risk groups, n − survivors\n(510 − 449) + (599 − 252)", list(GT_D), BLUE,
[str(x) for x in GT_D]),
("Cancer types, PROMISE deaths\n99 + 143 + 29 + 137", CP, BLUE,
[str(x) for x in CP]),
("Cancer types, GRIm deaths\n99 + 143 + 29 + 137", CG, BLUE,
[str(x) for x in CG]),
]
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
ax.set_facecolor(BG)
shades = ["#2a78d6", "#5d9ae3", "#8fbbee", "#bcd6f5"]
for i, (lab, segs, col, txt) in enumerate(rows):
y = len(rows) - 1 - i
left = 0
for j, (s, t) in enumerate(zip(segs, txt)):
c = col if col == ORANGE else shades[j % 4]
ax.barh(y, s, left=left, height=0.56, color=c, edgecolor=BG, linewidth=2)
if s >= 60:
ax.text(left + s / 2, y, t, ha="center", va="center", fontsize=10,
color="white" if (col == ORANGE or j == 0) else INK)
left += s
tot = sum(segs)
ax.text(tot + 12, y, f"= {tot} ({pct(tot, N):.1f}% of {N})" if len(segs) > 1
else f"{tot} ({STATED_PCT}%)", va="center", fontsize=10.5,
color=INK, fontweight="bold")
ax.axvline(N, color=INK2, lw=1, ls=(0, (3, 3)))
ax.text(N, len(rows) - 0.35, f"all {N} patients", ha="right", va="bottom",
fontsize=10, color=INK2)
ax.set_yticks(range(len(rows)))
ax.set_yticklabels([r[0] for r in rows][::-1], fontsize=10)
ax.set_xlim(0, 1180)
ax.set_ylim(-0.6, len(rows) - 0.1)
ax.set_xlabel("Patients who died within 90 days (count)")
ax.xaxis.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
from matplotlib.patches import Patch
ax.legend(handles=[Patch(color=ORANGE, label="Reported in the paper"),
Patch(color=BLUE, label="Recomputed: sum of the paper's own subgroup counts")],
loc="upper center", bbox_to_anchor=(0.38, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# ---- fig2: percentages on one 0-100 axis with the median-OS bound
fig, ax = plt.subplots(figsize=(7.2, 2.9), facecolor=BG)
ax.set_facecolor(BG)
ax.axvspan(50, 100, color="#efeeea", zorder=0)
ax.axvline(50, color=INK2, lw=1, zorder=1)
ax.text(51.5, 0.25, "Ruled out by median OS 4.78 months (> 90 days):\n"
"90-day mortality must be below 50%", fontsize=10, color=INK2, va="center")
pts = [
(3, "Overall 90-day mortality,\nas printed (n = 707)", [(STATED_PCT, ORANGE, "o", "63.7%")]),
(2, "Overall 90-day mortality from\nthe paper's strata (408/1109)",
[(pct(408, N), BLUE, "o", f"{pct(408, N):.1f}%")]),
(1, "Share surviving 90 days from\nthe paper's strata (701/1109)",
[(pct(701, N), BLUE, "D", f"{pct(701, N):.1f}%")]),
(0, "CTI subset 90-day mortality\n(117/333): reported and recomputed",
[(CTI_PCT, ORANGE, "o", None), (pct(117, 333), BLUE, "o", f"{CTI_PCT}% (both)")]),
]
for y, lab, marks in pts:
for k, (x, c, m, t) in enumerate(marks):
ms = 14 if k == 0 else 8
ax.plot([x], [y], marker=m, ms=ms, color=c, mec=BG, mew=1.5, ls="none", zorder=3)
if t:
ax.text(x - 2.5, y + 0.02, t, ha="right", va="center", fontsize=10.5,
color=INK, fontweight="bold")
ax.set_yticks([p[0] for p in pts])
ax.set_yticklabels([p[1] for p in pts], fontsize=10)
ax.set_xlim(0, 100)
ax.set_ylim(-0.6, 3.7)
ax.set_xlabel("Percent of patients")
ax.xaxis.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
ax.legend(handles=[Line2D([], [], marker="o", ls="none", ms=9, color=ORANGE,
label="Reported in the paper"),
Line2D([], [], marker="o", ls="none", ms=9, color=BLUE,
label="Recomputed from the paper's counts")],
loc="upper center", bbox_to_anchor=(0.3, -0.25), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
if __name__ == "__main__":
main()
Abstract reports univariate hazard ratios as adjusted estimates and attaches them to the wrong treatments
Lancet Reg Health Eur (2025) · doi:10.1016/j.lanepe.2025.101402 · PMC12859599
Summary
The Abstract states that "in the adjusted multivariate model" immunotherapy and targeted therapy had hazard ratios (HRs) for second primary cancer (SPC), vs chemotherapy, of 0.470 (95% CI 0.338–0.654) and 0.701 (0.574–0.855). In the Results these same two HR/CI pairs are reported as the univariate Cox estimates, with the treatments the other way round: 0.470 for targeted therapy and 0.701 for immunotherapy. The adjusted multivariate estimates in the Results, repeated in the Discussion, are 0.768 (0.622–0.947) for immunotherapy and 0.556 (0.385–0.803) for targeted therapy. The paper's own event counts and incidence rates support the Results' assignment: the width of each interval matches the number of events in the group the Results name, and the crude rate ratios rank targeted therapy as the more protective treatment. As a result, the Abstract overstates the adjusted risk reduction for immunotherapy (53% vs 23%), understates it for targeted therapy (30% vs 44%), and reverses which treatment looks more protective after adjustment. Both treatments remain associated with lower SPC risk in every version.
What the paper reports
In the adjusted multivariate model, both immunotherapy and targeted therapy were associated with a reduced risk of SPCs: HR = 0.470 (95% CI 0.338–0.654) and HR = 0.701 (95% CI 0.574–0.855), respectively. Additional associated factors with SPCs included metastatic disease, HR = 0.456 (95% CI 0.377–0.552); history of tobacco use, HR = 1.599 (95% CI 1.127–2.269); and current smoking, HR = 1.319 (95% CI 0.923–1.884) p < 0.001.
Abstract, Findings
In the univariate Cox regression, the following factors were explored for their association with the development of SPCs: type of treatment received, age at diagnosis, sex, tumour stage, histological type, and smoking habits were explored. Both targeted therapy and immunotherapy, when compared to chemotherapy, were identified as protective factors with HR = 0.470 (95% CI 0.338–0.654) and HR = 0.701 (95% CI 0.574–0.855), respectively.
Results, "Factors associated with second primary cancer", paragraph 1
Other factors associated with the development of SPCs included metastatic disease HR = 0.456 (95% CI 0.377–0.552), history of tobacco use HR = 1.599 (95% CI 1.127–2.269), and active tobacco use HR = 1.319 (95% CI 0.923–1.884); p < 0.001.
Results, "Factors associated with second primary cancer", paragraph 3 (between the univariate paragraph and the adjusted-model paragraph)
In the adjusted multivariate model, the effect of treatment was confirmed with immunotherapy and targeted therapy, HR = 0.768 (95% CI 0.622–0.947) and HR = 0.556 (95% CI 0.385–0.803); p < 0.001, respectively. Other associated factors such as age HR = 1.028 (95% CI 1.018–1.039), small cell histology HR = 0.704 (95% CI 0.513–0.965), and metastatic disease HR = 0.518 (95% CI 0.428–0.626); p < 0.001, were also found.
Results, "Factors associated with second primary cancer", paragraph 4
Multivariate Cox regression analysis confirmed that both immunotherapy and targeted therapy were protective for SPC compared to chemotherapy with HR = 0.768 (95% CI 0.622–0.947) and HR = 0.556 (95% CI 0.385–0.803); p < 0.001, respectively.
Discussion, paragraph 1
The inputs used for the checks below:
A total of 480 patients (2.3%) developed a SPC. The incidence rate per 100 person-years were 1.059 for the immunotherapy group, 0.661 for the targeted therapy group, and 1.791 for the chemotherapy group. […] By treatment group, 298 patients (62.1%) had received chemotherapy, 142 (29.6%) immunotherapy, and 40 (8.3%) targeted therapy. The proportion of patients who developed an SPC was lowest in the targeted therapy group (1.5%) followed by the immunotherapy group (2.1%) and the chemotherapy group (2.9%) (Table 3).
Results, "Characteristics of the population with a second primary cancer by treatment type", paragraph 1
How the Abstract uses the result:
Patients treated with immunotherapy and targeted therapy exhibited a significantly lower incidence of SPCs. Tobacco use was a key associated factor.
Abstract, Interpretation
What the paper's own numbers show
Method. The Abstract, Results and Discussion values were compared as printed. Two checks show which treatment each HR/CI pair belongs to. (1) The standard error of a log HR implied by a printed 95% CI is ln(upper/lower)/(2 × 1.96). For a Cox HR comparing two groups, this SE is approximately √(1/d₁ + 1/d₀), where d₁ and d₀ are the event counts in the two groups (142 immunotherapy, 40 targeted therapy and 298 chemotherapy events; Results and Table 3). The approximation is close for an unadjusted comparison and somewhat larger after adjustment. (2) The crude incidence-rate ratio of each group vs chemotherapy was computed from the reported rates per 100 person-years. Each CI was also checked for internal consistency: its log-scale midpoint should equal the printed HR. All numbers come from recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Immunotherapy vs chemotherapy, "adjusted multivariate" | Abstract, Findings | 0.470 (0.338–0.654) | Results and Discussion give the adjusted HR as 0.768 (0.622–0.947). 0.470 (0.338–0.654) is the Results' univariate value for targeted therapy |
| Targeted therapy vs chemotherapy, "adjusted multivariate" | Abstract, Findings | 0.701 (0.574–0.855) | Results and Discussion give the adjusted HR as 0.556 (0.385–0.803). 0.701 (0.574–0.855) is the Results' univariate value for immunotherapy |
| SE(log HR) implied by 0.470 (0.338–0.654) | Abstract; Results ¶1 | — | 0.1684. Expected from events: targeted therapy (40 vs 298) 0.1684, immunotherapy (142 vs 298) 0.1020. Ratio implied/expected 1.000 for targeted therapy, 1.651 for immunotherapy |
| SE(log HR) implied by 0.701 (0.574–0.855) | Abstract; Results ¶1 | — | 0.1017. Matches immunotherapy (ratio 0.997), not targeted therapy (0.604) |
| SE(log HR) implied by adjusted 0.768 (0.622–0.947) | Results ¶4; Discussion | immunotherapy | 0.1072, 1.052 × the immunotherapy event-based value. Consistent |
| SE(log HR) implied by adjusted 0.556 (0.385–0.803) | Results ¶4; Discussion | targeted therapy | 0.1875, 1.114 × the targeted-therapy event-based value. Consistent |
| Log-scale midpoint of each CI | all four pairs | 0.470, 0.701, 0.768, 0.556 | 0.470, 0.701, 0.767, 0.556. Each pair is internally consistent, so no digit has been garbled |
| Crude SPC rate ratio vs chemotherapy | from rates per 100 person-years (1.059, 0.661, 1.791) | — | Immunotherapy 0.591 (0.484–0.722), targeted therapy 0.369 (0.265–0.513). Targeted therapy is the more protective group, as in the Results' univariate assignment |
| 12-month cumulative incidence ratio vs chemotherapy | Abstract; Results (1.22%, 0.85%, 0.67%) | — | Immunotherapy 0.697, targeted therapy 0.549. Same ordering |
| Hazard reduction, immunotherapy (100 × (1 − HR)) | Abstract vs Results ¶4 | 53.0% (abstract) | 23.2% (adjusted model). The abstract's figure is 2.28 times the adjusted one |
| Hazard reduction, targeted therapy | Abstract vs Results ¶4 | 29.9% (abstract) | 44.4% (adjusted model) |
| Which treatment is more protective after adjustment | Abstract vs Results ¶4 | immunotherapy (0.470 < 0.701) | targeted therapy (0.556 < 0.768) |
| Metastatic disease, listed in the abstract after the "adjusted" treatment HRs | Abstract vs Results ¶3, ¶4 | 0.456 (0.377–0.552) | 0.456 is the Results ¶3 value (between the univariate and adjusted paragraphs). The adjusted-model value is 0.518 (0.428–0.626) |
| Current smoking, "p < 0.001" | Abstract; Results ¶3 | 1.319 (0.923–1.884) | The CI includes 1. The Wald p implied by the CI is 0.13 (z = 1.52), so the "p < 0.001" cannot refer to this term on its own |
| Wald p implied by the adjusted CIs | Results ¶4; Discussion ("p < 0.001") | — | Immunotherapy z = −2.46, p = 0.014; targeted therapy z = −3.13, p = 0.0017. These are significant at 0.05, but neither is below 0.001; the printed p may refer to a joint test of the treatment variable |
| Crude SPC proportions | Abstract; Results; Table 3 | 2.9%, 2.1%, 1.5% | 298/10,137 = 2.94%, 142/6,820 = 2.08%, 40/2,628 = 1.52% with the followed-up denominators of Supplementary Figure S1 (sum 19,585). These are consistent and not part of the error |
So the Abstract's two "adjusted" treatment HRs are the univariate pair, with the treatment labels swapped. The event-count check fits the Results' labelling exactly (ratios 0.997 and 1.000) and misfits the Abstract's labelling by factors of 1.65 and 0.60. The crude rate ratios point the same way. The adjusted estimates in the body are consistent with the event counts and with the expected modest SE inflation after covariate adjustment.
Figure
Figure 1. Where each treatment HR appears in the paper and how it is labelled. Orange text gives the values as printed. The left panel is the Abstract's "adjusted multivariate model". The upper right panel is the Results' univariate Cox paragraph, which carries the same two HR/CI pairs with the treatment labels reversed. The lower right panel is the Results' adjusted multivariate paragraph, repeated in the Discussion, which carries different values under the same model label as the Abstract. The blue text below summarises the checks recomputed from the paper's own event counts and incidence rates (see recompute.py.txt). Both checks agree with the treatment labels in the Results.
Figure 2. For each of the four printed treatment HRs, the orange diamond is the standard error of the log HR implied by its 95% CI. The blue vertical lines are the approximate standard errors expected from the event counts, √(1/d₁ + 1/d₀): 0.102 for immunotherapy vs chemotherapy (142 vs 298 events, solid) and 0.168 for targeted therapy vs chemotherapy (40 vs 298 events, dashed). The interval of HR 0.470, which the Abstract assigns to immunotherapy, sits exactly on the targeted-therapy line. The interval of HR 0.701, which the Abstract assigns to targeted therapy, sits on the immunotherapy line. The two adjusted intervals from the Results lie just to the right of their own group's line, as expected after covariate adjustment. The axis starts at zero.
Why it matters
- The headline adjusted estimates. The Abstract is where readers, guideline authors and evidence syntheses usually take effect sizes from. Readers who take the Abstract's adjusted immunotherapy HR (0.470) would record a 53% lower SPC hazard, whereas the paper's adjusted model gives 23% (HR 0.768). For targeted therapy they would record 30% (HR 0.701) instead of 44% (HR 0.556).
- The ranking of treatments. After adjustment, the Abstract makes immunotherapy look more protective than targeted therapy. The body's adjusted model, the univariate model and the crude data (1.5% vs 2.1%; 0.661 vs 1.059 per 100 person-years) all rank targeted therapy first.
- Unadjusted shown as adjusted. The Discussion itself notes that the treatment associations "are likely influenced by the underlying clinical profiles of patients receiving each treatment". The Abstract presents unadjusted estimates as adjusted ones, so readers cannot see how much adjustment changed them. For immunotherapy the hazard reduction shrinks from 30% (univariate 0.701) to 23% (adjusted 0.768).
- Other factors in the Abstract. The Abstract's "additional associated factors" are also the values from Results ¶3, not the adjusted model (adjusted metastatic disease HR 0.518 vs 0.456 in the Abstract). The adjusted-model paragraph reports no tobacco HRs. Current smoking, given with "p < 0.001", has a CI that includes 1 (implied p = 0.13). The Interpretation ("Tobacco use was a key associated factor") rests on these values.
- What stands. All four treatment CIs exclude 1, so the qualitative conclusion holds in every version: both immunotherapy and targeted therapy are associated with lower SPC risk than chemotherapy. The crude incidence figures, cumulative incidence curves and SPC site distribution are not affected.
Explanations considered
- The abstract reports a different model (Fine–Gray rather than Cox). Does not account for it. The Methods describe "Univariate and multivariate Fine and Gray sub distribution hazard models … with sub hazard ratios (sHR)" as well as Cox models, but the paper reports no sHR for any covariate. In the body, 0.470 and 0.701 are called univariate Cox HRs, and the only values called adjusted multivariate are 0.768 and 0.556, both in the Results and in the Discussion. No passage gives 0.470 or 0.701 as an adjusted estimate from any model. If the Abstract came from another adjusted model, it would also be unlikely to reproduce the univariate CIs to three decimals.
- A subgroup analysis. Does not account for it. The within-subgroup multivariate HRs in Results ¶2 are 2.117 (0.989–4.534), 2.171 (1.160–4.063) and 1.411 (0.717–2.777), which are different numbers.
- The Results, not the Abstract, has the treatments swapped. Not supported. The CI widths match the event counts of the groups the Results names (implied/expected SE ratio 1.000 for 0.470 as targeted therapy, 0.997 for 0.701 as immunotherapy), and not the Abstract's assignment (1.65 and 0.60). The crude rate ratios (targeted 0.369, immunotherapy 0.591) and the 12-month cumulative incidences (0.67% vs 0.85%) also rank targeted therapy as more protective.
- A typo or digit transposition. Does not account for it. Each HR sits at the log-scale midpoint of its CI, so every pair is internally consistent. Four values (both HRs and both CIs) move together.
- A conversion or rendering artefact. Excluded. The values are in running text, not table cells. The publisher's typeset PDF carries the same wording and numbers (see Status).
- A supplementary table that reports 0.470/0.701 as adjusted. Not found. Supplementary Table S1 is a Cox model for overall survival, not for SPCs. Its treatment HRs (targeted 0.696/0.598, immunotherapy 0.719/0.564) match none of the disputed values.
Status
- Version of record. Checked in the publisher's typeset PDF (creator Elsevier, "The Lancet Regional Health - Europe, 56 (2025) 101402", 20 pages), obtained through the PMC /pdf/ endpoint. The Abstract, univariate-paragraph and adjusted-paragraph passages appear word for word. In the PDF text, 0.470 occurs only in the Abstract and in the univariate paragraph. The Europe PMC full-text XML, the live PMC page and the local PMC archive copy carry the same text. Every quotation above was checked verbatim against the archive copy. The ScienceDirect and thelancet.com pages refused automated access (HTTP 403 or a challenge page).
- Supplements. Both supplementary files were checked (Europe PMC supplementary files): mmc1.docx (a list of group members) and mmc2.pdf (5 pages: Figure S1 CONSORT diagram, Figure S2 overall survival curves, Table S1 Cox model for overall survival). Figure S1 confirms the SPC counts (immunotherapy 142/6,820, targeted therapy 40/2,628, chemotherapy 298/10,137). No supplement contains a Cox or Fine–Gray table for SPCs, so the treatment HRs appear only in the running text.
- Corrections. Crossref lists no update-to or updated-by relation for the DOI. PMC and Europe PMC list no correction, erratum, retraction or expression of concern (PMID 41624086; checked 2026-10-23).
- PubPeer. pubpeer.com refused automated access (HTTP 403). A web search for the title with "PubPeer" returned no PubPeer page. Europe PMC's link services for PMID 41624086 list only BioStudies, with no PubPeer link. The absence of a PubPeer thread is therefore likely but not confirmed.
- As of 2026-10-23.
Suggested correction
- In the Abstract, Findings, replace the treatment sentence with the adjusted estimates the Results and Discussion report: immunotherapy HR 0.768 (95% CI 0.622–0.947) and targeted therapy HR 0.556 (95% CI 0.385–0.803), vs chemotherapy. If the univariate values are kept as well, label them as univariate and attach them to the right treatments: targeted therapy 0.470 (0.338–0.654), immunotherapy 0.701 (0.574–0.855).
- Make the Abstract's "additional associated factors" match the model it describes. For the adjusted model, give metastatic disease 0.518 (0.428–0.626), age 1.028 (1.018–1.039) and small cell histology 0.704 (0.513–0.965). Report the adjusted tobacco HRs if the model included smoking, as the Methods list it among the covariates.
- Give a p value for each term, or say which test "p < 0.001" refers to. As printed, it is attached to current smoking, whose CI includes 1, and to two adjusted treatment HRs whose CIs imply p = 0.014 and p = 0.0017.
- Consider adding a table of the univariate and multivariate SPC models (Cox and, as the Methods describe, Fine–Gray) so that each estimate's model and covariate set is explicit. If needed, revise the Interpretation sentence on tobacco use to match the adjusted results.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC12859599 write-up and regenerate fig1/fig2.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
All inputs are copied from the article as archived in PubMed Central (Abstract,
Results "Characteristics of the population with a second primary cancer" and
"Factors associated with second primary cancer", Discussion) and from
Supplementary Figure S1 (CONSORT diagram, mmc2.pdf).
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# ---------------------------------------------------------------- inputs ----
EVENTS = {"chemo": 298, "IT": 142, "TT": 40} # Results; Table 3
N_TREATED = {"chemo": 10650, "IT": 7182, "TT": 2742} # Results, treatment groups
N_FOLLOWED = {"chemo": 10137, "IT": 6820, "TT": 2628} # Supplementary Figure S1
RATE = {"chemo": 1.791, "IT": 1.059, "TT": 0.661} # SPC per 100 person-years
CIF12 = {"chemo": 1.22, "IT": 0.85, "TT": 0.67} # 12-month cumulative incidence, %
# (HR, lower, upper) exactly as printed
ABSTRACT_ADJ = {"IT": (0.470, 0.338, 0.654), "TT": (0.701, 0.574, 0.855)}
RESULTS_UNIV = {"TT": (0.470, 0.338, 0.654), "IT": (0.701, 0.574, 0.855)}
RESULTS_ADJ = {"IT": (0.768, 0.622, 0.947), "TT": (0.556, 0.385, 0.803)} # also Discussion
OTHER = {
"metastatic, abstract / Results p2": (0.456, 0.377, 0.552),
"metastatic, Results adjusted": (0.518, 0.428, 0.626),
"history of tobacco use": (1.599, 1.127, 2.269),
"current (active) smoking": (1.319, 0.923, 1.884),
}
NAME = {"IT": "immunotherapy", "TT": "targeted therapy", "chemo": "chemotherapy"}
def ci_stats(h, lo, hi):
se = np.log(hi / lo) / (2 * Z)
mid = np.exp((np.log(lo) + np.log(hi)) / 2)
z = np.log(h) / se
p = 2 * stats.norm.sf(abs(z))
return se, mid, z, p
def line(s=""):
print(s)
# ------------------------------------------------------- 1. crude data ----
line("1. Crude SPC proportions")
line(f" total: {sum(EVENTS.values())}/{sum(N_TREATED.values())} = "
f"{100*sum(EVENTS.values())/sum(N_TREATED.values()):.2f}% "
f"(followed-up denominators sum to {sum(N_FOLLOWED.values())})")
for k in ["chemo", "IT", "TT"]:
line(f" {NAME[k]:17s} {EVENTS[k]}/{N_TREATED[k]} = {100*EVENTS[k]/N_TREATED[k]:.2f}% "
f"{EVENTS[k]}/{N_FOLLOWED[k]} (Fig S1) = {100*EVENTS[k]/N_FOLLOWED[k]:.2f}%")
line("\n2. Crude incidence-rate ratios vs chemotherapy (from the reported rates)")
PY = {k: 100 * EVENTS[k] / RATE[k] for k in EVENTS}
for k in ["chemo", "IT", "TT"]:
line(f" person-years implied, {NAME[k]}: {PY[k]:.0f}")
CRUDE_IRR = {}
for k in ["IT", "TT"]:
irr = RATE[k] / RATE["chemo"]
se = np.sqrt(1 / EVENTS[k] + 1 / EVENTS["chemo"])
CRUDE_IRR[k] = (irr, irr * np.exp(-Z * se), irr * np.exp(Z * se))
line(f" {NAME[k]:17s} IRR = {irr:.3f} (95% CI {CRUDE_IRR[k][1]:.3f}-{CRUDE_IRR[k][2]:.3f})")
line(f" 12-month cumulative incidence ratio vs chemo: IT {CIF12['IT']/CIF12['chemo']:.3f}, "
f"TT {CIF12['TT']/CIF12['chemo']:.3f}")
# ------------------------------------------- 3. SE expected from events ----
line("\n3. SE(log HR) expected from event counts, sqrt(1/d1 + 1/d0)")
SE_EXP = {k: np.sqrt(1 / EVENTS[k] + 1 / EVENTS["chemo"]) for k in ["IT", "TT"]}
for k in SE_EXP:
line(f" {NAME[k]:17s} ({EVENTS[k]} vs {EVENTS['chemo']} events): {SE_EXP[k]:.4f}")
# ------------------------------------------- 4. SE implied by each CI ----
line("\n4. Each reported HR: SE implied by its CI, log-midpoint, Wald z and p, nearest group")
ROWS = [
("0.470 (0.338-0.654)", (0.470, 0.338, 0.654), "Abstract: IT 'adjusted' | Results: TT univariate"),
("0.701 (0.574-0.855)", (0.701, 0.574, 0.855), "Abstract: TT 'adjusted' | Results: IT univariate"),
("0.768 (0.622-0.947)", (0.768, 0.622, 0.947), "Results/Discussion: IT adjusted"),
("0.556 (0.385-0.803)", (0.556, 0.385, 0.803), "Results/Discussion: TT adjusted"),
]
IMPLIED = {}
for lab, (h, lo, hi), where in ROWS:
se, mid, z, p = ci_stats(h, lo, hi)
near = min(SE_EXP, key=lambda k: abs(SE_EXP[k] - se))
IMPLIED[lab] = se
line(f" {lab}: SE = {se:.4f}, log-midpoint = {mid:.3f}, z = {z:.2f}, p = {p:.2g}, "
f"nearest event-based SE: {NAME[near]} [{where}]")
for k in SE_EXP:
line(f" ratio implied/expected: univariate {NAME[k]} "
f"{ci_stats(*RESULTS_UNIV[k])[0]/SE_EXP[k]:.3f}; adjusted {ci_stats(*RESULTS_ADJ[k])[0]/SE_EXP[k]:.3f}; "
f"abstract '{NAME[k]}' {ci_stats(*ABSTRACT_ADJ[k])[0]/SE_EXP[k]:.3f}")
line("\n5. Other factors (Wald p from the printed CI)")
for lab, v in OTHER.items():
se, mid, z, p = ci_stats(*v)
line(f" {lab:36s} HR {v[0]:.3f} ({v[1]:.3f}-{v[2]:.3f}): SE {se:.4f}, z {z:.2f}, p {p:.2g}")
# ------------------------------------------- 6. comparisons and sizes ----
line("\n6. Abstract vs Results")
line(f" abstract 'adjusted' == Results adjusted? {ABSTRACT_ADJ == RESULTS_ADJ}")
line(f" abstract values == Results univariate values (set)? "
f"{set(ABSTRACT_ADJ.values()) == set(RESULTS_UNIV.values())}")
line(f" abstract assignment == Results univariate assignment? {ABSTRACT_ADJ == RESULTS_UNIV}")
for k in ["IT", "TT"]:
a, r = ABSTRACT_ADJ[k][0], RESULTS_ADJ[k][0]
line(f" {NAME[k]:17s} hazard reduction: abstract {100*(1-a):.1f}% vs adjusted {100*(1-r):.1f}% "
f"(difference {100*abs(a-r):.1f} points; ratio of reductions {(1-a)/(1-r):.2f})")
line(f" ranking after adjustment: abstract says {'IT' if ABSTRACT_ADJ['IT'][0] < ABSTRACT_ADJ['TT'][0] else 'TT'} "
f"more protective; Results adjusted says {'IT' if RESULTS_ADJ['IT'][0] < RESULTS_ADJ['TT'][0] else 'TT'}")
line(" all four CIs exclude 1: "
f"{all(v[2] < 1 for d in (ABSTRACT_ADJ, RESULTS_ADJ) for v in d.values())}")
# --------------------------------------------------------------- figures ----
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: text-panel diagram
fig = plt.figure(figsize=(7.2, 6.4), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 92); ax.axis("off")
def panel(x, y, w, h, title, sub, rows):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec="#bdbcb7", lw=1))
ax.text(x + 1.5, y + h - 3, title, fontsize=11, weight="bold", va="center")
ax.text(x + 1.2, y + h - 6.6, sub, fontsize=10, color=INK2, va="center", style="italic")
for i, (lab, val) in enumerate(rows):
yy = y + h - 11.5 - i * 8
ax.text(x + 1.5, yy, lab, fontsize=10.5, va="center")
ax.text(x + 3.5, yy - 3.5, val, fontsize=10.5, va="center", color=ORANGE, weight="bold")
PW = 37
panel(2, 41, PW, 26, "Abstract, Findings", '"adjusted multivariate model"',
[("Immunotherapy", "HR 0.470 (0.338–0.654)"), ("Targeted therapy", "HR 0.701 (0.574–0.855)")])
panel(61, 60, PW, 26, "Results, univariate", '"univariate Cox regression"',
[("Targeted therapy", "HR 0.470 (0.338–0.654)"), ("Immunotherapy", "HR 0.701 (0.574–0.855)")])
panel(61, 26, PW, 26, "Results ¶4; Discussion", '"adjusted multivariate model"',
[("Immunotherapy", "HR 0.768 (0.622–0.947)"), ("Targeted therapy", "HR 0.556 (0.385–0.803)")])
for (y0, y1, txt, ty) in [(63.5, 63.5, "same numbers,\ncalled\nunivariate,\ntreatments\nswapped", 72.5),
(46.5, 46.5, "same model\nlabel,\ndifferent\nnumbers", 38.5)]:
ax.add_patch(FancyArrowPatch((39.9, y0), (60.1, y1), arrowstyle="<|-|>", mutation_scale=12,
lw=2, color=INK2, shrinkA=0, shrinkB=0))
ax.text(50, ty, txt, ha="center", va="center", fontsize=10, color=INK, weight="bold",
bbox=dict(fc=BG, ec="none", pad=1.5))
ax.text(2, 86, "Orange: values as printed in the paper.", fontsize=10, color=ORANGE, weight="bold")
ax.text(2, 82, "Blue: recomputed from the paper's own numbers.", fontsize=10, color=BLUE, weight="bold")
se470, se701 = IMPLIED["0.470 (0.338-0.654)"], IMPLIED["0.701 (0.574-0.855)"]
recomp = [
f"The CI of HR 0.470 implies SE(log HR) = {se470:.3f}. Event counts predict {SE_EXP['TT']:.3f} for targeted therapy",
f"(40 vs 298 events) and {SE_EXP['IT']:.3f} for immunotherapy (142 vs 298). The CI of HR 0.701 implies {se701:.3f}.",
f"Crude SPC rate ratios vs chemotherapy: targeted therapy {CRUDE_IRR['TT'][0]:.3f}, immunotherapy {CRUDE_IRR['IT'][0]:.3f}.",
"Both checks agree with the Results' treatment labels, not the Abstract's.",
]
for i, t in enumerate(recomp):
ax.text(2, 18 - i * 4.2, t, fontsize=10, color=BLUE, va="center")
save(fig, "fig1")
# Figure 2: SE implied by each reported CI vs SE expected from event counts
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
ax.set_facecolor(BG)
YLAB = ["HR 0.470 (0.338–0.654)\nAbstract: immunotherapy, 'adjusted'\nResults: targeted, univariate",
"HR 0.701 (0.574–0.855)\nAbstract: targeted, 'adjusted'\nResults: immunotherapy, univariate",
"HR 0.768 (0.622–0.947)\nResults, Discussion:\nimmunotherapy, adjusted",
"HR 0.556 (0.385–0.803)\nResults, Discussion:\ntargeted, adjusted"]
ys = np.arange(len(ROWS))[::-1]
top = len(ROWS) + 0.5
for k, ls, dx, ha in [("IT", "-", -0.004, "right"), ("TT", "--", 0.004, "left")]:
ax.axvline(SE_EXP[k], color=BLUE, lw=2, ls=ls, zorder=1)
ax.text(SE_EXP[k] + dx, top - 0.05,
f"expected from events,\n{NAME[k]}\n({EVENTS[k]} vs {EVENTS['chemo']}): {SE_EXP[k]:.3f}",
color=BLUE, fontsize=10, va="top", ha=ha)
ax.scatter([IMPLIED[r[0]] for r in ROWS], ys, marker="D", s=70, color=ORANGE, zorder=3,
label="SE(log HR) implied by the printed 95% CI (paper's values)")
for y, r in zip(ys, ROWS):
ax.text(IMPLIED[r[0]] + 0.006, y, f"{IMPLIED[r[0]]:.3f}", color=ORANGE, fontsize=10,
ha="left", va="center", weight="bold")
ax.plot([], [], color=BLUE, lw=2, label="SE expected from event counts, sqrt(1/d1 + 1/d0)")
ax.set_yticks(ys); ax.set_yticklabels(YLAB, fontsize=10)
ax.set_xlim(0, 0.26); ax.set_ylim(-0.5, top)
ax.set_xlabel("Standard error of log hazard ratio vs chemotherapy")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color("#bdbcb7"); ax.tick_params(axis="y", length=0)
fig.legend(loc="lower center", ncol=1, frameon=False, fontsize=10)
fig.tight_layout(rect=(0, 0.1, 1, 1))
save(fig, "fig2")
line("\nFigures written: fig1.svg/png, fig2.svg/png")
Abstract reports a 59.7% catalyst heat-duty reduction; the body reports 35.2% at the optimum
Nat Commun (2026) · doi:10.1038/s41467-026-69542-6 · PMC12966427
Summary
The abstract says that the optimal 1:1 iron-sulfated zirconia/silica (Fe-SZ/SiO2) catalyst "reduces relative heat duty by up to 59.7%" in the membrane vacuum regeneration (MVR) system. The Results and Discussion report something different for the same catalyst, solvent (3 M potassium glycinate, GlyK) and blank: a 35.2% reduction at 9 wt%, the loading the paper selects as optimal, and 40.3% at 12 wt%, the highest loading tested. Applying the paper's own Eq. 7 to its reported catalytic enhancements reproduces 35.2% and 40.3%. The same calculation shows that 59.7% is the relative heat duty that remains at 12 wt% (100/1.675 = 59.7%, i.e. 100 − 40.3), not a reduction. As written, the abstract's figure is 1.70 times the reduction at the stated optimum and 1.48 times the largest catalyst-only reduction in the paper. It is the abstract's only quantitative statement about the catalyst on its own.
What the paper reports
Abstract:
An optimal iron-sulfated zirconia to silica ratio of 1:1 reduces relative heat duty by up to 59.7% in the membrane vacuum system.
Finally, combining the optimized hybrid solvent, catalyst, and membrane system at 90 °C reduces thermal energy consumption by 66.8% relative to potassium glycinate, achieving an energy requirement of 2.6 GJ/tCO2.
Results, Composite solid acid catalyst › Catalytic support (SiO2 vs. Al2O3), Fig. 2g text (6 wt% 1:1 catalyst):
raising the total amount of desorbed CO2 from 244 to 307 mmol for 3 M GlyK
Results, Composite solid acid catalyst › Catalyst concentration and stability, paragraph 1 (the inputs):
the total desorbed CO2 increased from 244 to 377 mmol, corresponding to a 54.3% catalytic enhancement
the enhancement rising to 67.5%
Notably, the magnitude of improvement from 6 to 9 wt% (112%) was greater than that from 9 to 12 wt% (24.3%), suggesting a reduced catalytic effect beyond a certain concentration, indicating the presence of an optimum.
the use of 9 wt% catalyst resulted in the highest enhancement factor of 6.0 and was therefore selected as the optimal concentration for further evaluations.
Same section, paragraph 2 (text for Fig. 3g):
Accordingly, at 9 wt% and 12 wt%, the heat duty decreased by 35.2 and 40.3% as compared to the blank, respectively, reducing the thermal energy requirement from 7.8 to 5.1 and 4.7 GJ/tCO2.
Same section, paragraph 3 (stability at 9 wt%, Fig. S6b):
the RH% increased from 64.8% in cycle 1 to 68.8% in cycle 2
Results, Catalytic hybrid amino acid solvents in MVR (Fig. 5f text):
Considering GlyK as a benchmark, the catalyst reduced the RH of GlyK, SarK, and TauK to 64.8%, 90.1%, and 55.8%, respectively.
Notably, the combination of a hybrid solvent with the catalyst further reduced the RH to 33.2%, corresponding to a 66.8% reduction in energy consumption.
Discussion:
9 wt% fixed-bed catalyst was identified as the optimal concentration, achieving a 35.2% reduction in the system’s energy input.
Methods, MVR performance analysis (definition):
The relative heat duty, RH (%), was calculated using Eq. 7
where Eq. 7 is RH = Q_Cat / Q_Blank × 100.
All quotes were checked verbatim against the PMC-archived text (src/show.py; recompute.py.txt repeats the check). The string "59.7" occurs once in the archived article text, in the abstract.
What the paper's own numbers show
Method. The Methods define the catalytic enhancement CE = (M_cat − M_blank)/M_blank × 100 (Eq. 4), the thermal energy Q = m·cp·ΔT / (M_total_CO2·Mw) (Eq. 6) and RH = Q_cat/Q_blank × 100 (Eq. 7). In the catalyst-loading series the solvent (3 M GlyK), its mass and the temperature programme (90 °C) are the same with and without catalyst, so m, cp and ΔT cancel and RH = M_blank/M_cat × 100 = 100/(1 + CE/100). The heat-duty reduction is 100 − RH, and the thermal energy is 7.8 GJ/tCO2 (the blank) × RH/100. RH is computed from the reported CE values and, where the paper prints the desorbed amounts, also from the amounts in mmol. As a separate check, the reduction is recomputed from the reported, rounded energies, 1 − Q_cat/7.8. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| CE, 9 wt% 1:1 | Results, Catalyst concentration, para 1 | 54.3% (244 → 377 mmol) | 377/244 − 1 = 54.5% |
| CE, 6 wt% 1:1 | Results, Catalytic support | 25.6% (244 → 307 mmol) | 307/244 − 1 = 25.8% |
| Gain in CE, 6 → 9 wt% and 9 → 12 wt% | Results, para 1 | 112% and 24.3% | 112.1% and 24.3% |
| Enhancement factor CE/W | Results, para 1 | 6.0 at 9 wt% (highest) | 4.27 (6 wt%), 6.03 (9 wt%), 5.62 (12 wt%) |
| RH, 9 wt% (stated optimum) | Results, para 3; Fig. 5f text | 64.8% | 100/1.543 = 64.8% (from mmol: 64.7%) |
| Heat-duty reduction, 9 wt% | Results, para 2; Discussion | 35.2% | 100 − 64.8 = 35.2% (from mmol: 35.3%; from energies: 1 − 5.1/7.8 = 34.6%) |
| Thermal energy, 9 wt% | Results, para 2 | 5.1 GJ/tCO2 | 7.8 × 0.648 = 5.06 GJ/tCO2 |
| RH, 12 wt% (highest tested) | not stated in the text | — | 100/1.675 = 59.7% |
| Heat-duty reduction, 12 wt% | Results, para 2 | 40.3% | 100 − 59.7 = 40.3% (from energies: 1 − 4.7/7.8 = 39.7%) |
| Thermal energy, 12 wt% | Results, para 2 | 4.7 GJ/tCO2 | 7.8 × 0.597 = 4.66 GJ/tCO2 |
| Heat-duty reduction, 6 wt% 1:1 | not stated in the text | — | 100 − 100/1.256 = 20.4% (RH 79.6%, 6.21 GJ/tCO2) |
| Heat-duty reduction, "optimal" 1:1 catalyst | Abstract | 59.7% ("by up to") | 35.2% at the optimum; 40.3% at most (12 wt%) |
| What a 59.7% reduction would require | — | — | RH 40.3%, Q = 3.14 GJ/tCO2, CE = 148% (highest reported catalyst-only CE: 67.5%) |
| Abstract figure ÷ body figure | — | — | 59.7/35.2 = 1.70 (optimum); 59.7/40.3 = 1.48 (12 wt%) |
| Hybrid solvent + catalyst | Abstract; Results (Fig. 5f text) | RH 33.2%, 66.8% reduction, 2.6 GJ/tCO2 | 100 − 33.2 = 66.8%; 7.8 × 0.332 = 2.59 GJ/tCO2 |
The inputs in this section of the paper agree with one another to within rounding. The Results paragraph, the stability paragraph, the Fig. 5f text and the Discussion all agree on 35.2% (RH 64.8%) at 9 wt%, and the Results give 40.3% at 12 wt%. The abstract's catalyst figure is the only value that departs from them. It equals, to the printed decimal, the RH remaining at 12 wt%. The other two headline numbers in the abstract agree with the body: 66.8% and 2.6 GJ/tCO2 for the combined system, and the 69.1% desorption gain of the hybrid solvent (69.2% in the body; 589/348 − 1 = 69.3%).
Figure
Figure 1. Left: the abstract's sentence on the catalyst, which states 59.7% as a reduction and links it to the optimal configuration. Right: the reductions the Results (Fig. 3g text) and Discussion report for the same 1:1 Fe-SZ/SiO2 catalyst in 3 M GlyK against the same blank: 35.2% at the stated optimum (9 wt%) and 40.3% at 12 wt%. Orange text marks values reported in the paper. Bottom (blue): RH recomputed from the reported catalytic enhancements using the paper's Eq. 7, RH = 100/(1 + CE/100). The recomputation reproduces the body's reductions and shows that 59.7% is the share of the blank heat duty that remains at 12 wt%. A real 59.7% reduction would mean RH 40.3% and 3.14 GJ/tCO2. The paper reports neither value for the catalyst at any loading.
Figure 2. Each bar is the blank heat duty (3 M GlyK, no catalyst; 100%, 7.8 GJ/tCO2), split into the share that remains (light segment, RH) and the share removed (solid segment, reduction). Blue bars are recomputed from the paper's reported catalytic enhancements (25.6%, 54.3% and 67.5%) with Eq. 7. The column at the right gives the corresponding thermal energy, 7.8 × RH/100. The orange bar shows what the abstract's sentence states: a 59.7% reduction, which would leave 40.3% and imply about 3.14 GJ/tCO2 (marked *). The arrow links the abstract's 59.7 to the remaining-share segment at 12 wt%, which has the same value. The axis runs over the full 0 to 100% range.
Why it matters
The sentence is one of three quantitative claims in the abstract and the only one that isolates the catalyst. The catalyst is one of the paper's two named contributions, "catalytic solvent regeneration" and hybrid solvents. A reader of the abstract alone, or of a database or aggregator that reproduces it, would take the catalyst to cut heat duty by about 60% at its optimal configuration. The paper's own data show about 35% at that configuration (about 5.1 GJ/tCO2 instead of 7.8) and about 40% at the highest loading. A 60% reduction would mean about 3.1 GJ/tCO2. The abstract therefore overstates the catalyst's effect by a factor of 1.5 to 1.7. It also attaches the larger number to the word "optimal", although the paper selects 9 wt% as optimal on enhancement factor, which is the loading with the smaller reduction. The direction of the effect is unchanged: the catalyst still lowers the heat duty substantially. The body, figures and conclusions are not affected. These include the combined hybrid solvent plus catalyst result (RH 33.2%, a 66.8% reduction, 2.6 GJ/tCO2), which the abstract states correctly.
Explanations considered
- "Reduced to" written as "reduced by". 59.7 equals 100/1.675, the RH remaining at 12 wt% (equivalently 100 − 40.3). According to the version-of-record check, the publisher's full-resolution Fig. 3g, which is labelled RH (%), shows 59.7 for the 12 wt% bar and 64.8 for 9 wt%. This reading explains where the number comes from exactly. It does not make the sentence correct as published, because "reduces … by up to 59.7%" states a reduction. Even read as "to 59.7%", the value belongs to 12 wt%, not to the 9 wt% loading the paper calls optimal (RH 64.8%).
- "Up to" means the best case across loadings rather than the optimum. The best case for the 1:1 catalyst is 12 wt%, a 40.3% reduction. No loading or SZ:SiO2 ratio in the paper gives more: 20.4% at 6 wt% 1:1, 12.0% at 6 wt% 1:2 and 3.8% at 6 wt% 2:1. This reading changes which configuration is meant but not the size of the discrepancy.
- A different baseline or quantity. Measured against the 6 wt% catalyst instead of the blank, the reductions are 18.6% (9 wt%) and 25.0% (12 wt%). The gains in peak CO2 flux over the blank are 46.1% (9 wt%) and 56.5% (12 wt%), and the catalytic enhancement at 12 wt% is 67.5%. None of these equals 59.7. The closest other quantity in the paper is the hybrid solvent's reduction relative to GlyK without a catalyst: 1 − 3.2/7.8 = 59.0%, or 58.1% to 59.9% allowing for rounding of the two energies. That is a solvent effect, not the catalyst effect the sentence describes, and the paper does not state it as a percentage. The catalytic RH values for the other solvents (SarK 90.1%, TauK 55.8%) correspond to reductions of 9.9% and 44.2%, not 59.7%.
- Text conversion or archiving artefact. Ruled out. The PMC text, the Europe PMC full-text XML, the Nature Communications article page and the publisher PDF all carry the same abstract sentence and the same body values (see Status).
- Resolved in the supplement. No. The Supplementary Information does not contain 59.7. Its Fig. S6b gives RH 64.8% for cycle 1 at 9 wt%, which agrees with the body. The Transparent Peer Review file does not mention 59.7, 35.2 or 40.3. There is no separate Source Data file.
None of these makes the abstract's sentence consistent with the body. The arithmetic shows where the number most likely comes from. It says nothing about how the discrepancy arose.
Status
As of 2026-10-23:
- Version of record. The discrepancy is in the version of record. The abstract on the publisher's article page (reached through doi.org and re-checked on 2026-10-23) reads "reduces relative heat duty by up to 59.7% in the membrane vacuum system". The same page carries "the heat duty decreased by 35.2 and 40.3%" in the Results and "achieving a 35.2% reduction" in the Discussion. The publisher PDF (Nat Commun 2026, 17:2247) and the Europe PMC full-text XML carry the same text.
- Supplements. The Supplementary Information PDF (15 pages) and the Transparent Peer Review file were checked. Neither resolves the discrepancy (see above).
- Corrections. None found. Crossref lists no update-to or updated-by relations, and PMC and Europe PMC list no correction, erratum, retraction or expression of concern (
src/known.py). The publisher's article page shows no correction notice. - PubPeer and the web. Web searches for the title and DOI with "correction", "erratum" and "PubPeer", run in an earlier check on 2026-10-23, found no correction, PubPeer thread or published comment. PubPeer's own search could not be queried directly (it required human verification), and a repeat web search for this write-up was not possible because the session's search allowance was used up. A PubPeer thread therefore cannot be fully ruled out. Secondary listings, including a carbon-dioxide-removal literature aggregator and an institutional publication listing, repeat the abstract's 59.7% unchanged.
- Sources. PMC12966427 (
src/show.py); https://doi.org/10.1038/s41467-026-69542-6, which redirects to https://www.nature.com/articles/s41467-026-69542-6, and the publisher PDF; Europe PMC full-text XML; the article's supplementary files on the publisher's site; Crossref, PMC and Europe PMC notices (src/known.py); web searches as described above.
Suggested correction
In the abstract, replace
An optimal iron-sulfated zirconia to silica ratio of 1:1 reduces relative heat duty by up to 59.7% in the membrane vacuum system.
with a statement that matches the Results and Discussion, for example:
An optimal iron-sulfated zirconia to silica ratio of 1:1 at 9 wt% reduces the heat duty of the membrane vacuum system by 35.2% (by up to 40.3% at 12 wt%).
To keep the relative-heat-duty form instead, the sentence could say that the catalyst "reduces relative heat duty to 64.8% at the optimal 9 wt% loading (59.7% at 12 wt%)". No change to the body, figures or conclusions appears necessary.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12966427 (Nat Commun 2026, doi:10.1038/s41467-026-69542-6)
# "Catalytic hybrid solvent regeneration in membrane vacuum processes for direct air capture"
# Abstract vs body: size of the heat-duty reduction from the 1:1 Fe-SZ/SiO2 catalyst alone.
#
# Usage: python recompute.py.txt
# Needs: numpy, matplotlib.
# Output: prints every recomputed number used in writeup.md; writes fig1.svg/png and
# fig2.svg/png next to this script. If run from the papers repository (src/show.py
# present), it also checks each quoted statement verbatim against the PMC archive.
#
# Method. Eq. 6 of the paper: Q = m * cp * dT / (M_total_CO2 * Mw). In the catalyst-loading
# series the solvent (3 M GlyK), its mass and the temperature programme (90 C) are the same with
# and without catalyst, so Q is inversely proportional to the total CO2 desorbed in 30 min.
# Eq. 7: RH = Q_cat / Q_blank * 100, hence RH = M_blank / M_cat * 100 = 100 / (1 + CE/100),
# where CE (Eq. 4) is the catalyst enhancement. "Reduction in heat duty" = 100 - RH.
import os, sys, subprocess
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- 1. statements in the paper
QUOTES = [
("Abstract",
"An optimal iron-sulfated zirconia to silica ratio of 1:1 reduces relative heat duty by up to 59.7% in the membrane vacuum system."),
("Abstract",
"Finally, combining the optimized hybrid solvent, catalyst, and membrane system at 90 °C reduces thermal energy consumption by 66.8% relative to potassium glycinate, achieving an energy requirement of 2.6 GJ/tCO2."),
("Results, Catalytic support (SiO2 vs. Al2O3), Fig. 2g text",
"raising the total amount of desorbed CO2 from 244 to 307 mmol for 3 M GlyK"),
("Results, Catalyst concentration and stability, para 1",
"the total desorbed CO2 increased from 244 to 377 mmol, corresponding to a 54.3% catalytic enhancement"),
("Results, Catalyst concentration and stability, para 1",
"the enhancement rising to 67.5%"),
("Results, Catalyst concentration and stability, para 1",
"Notably, the magnitude of improvement from 6 to 9 wt% (112%) was greater than that from 9 to 12 wt% (24.3%), suggesting a reduced catalytic effect beyond a certain concentration, indicating the presence of an optimum."),
("Results, Catalyst concentration and stability, para 1",
"the use of 9 wt% catalyst resulted in the highest enhancement factor of 6.0 and was therefore selected as the optimal concentration for further evaluations."),
("Results, Catalyst concentration and stability, para 2 (Fig. 3g)",
"Accordingly, at 9 wt% and 12 wt%, the heat duty decreased by 35.2 and 40.3% as compared to the blank, respectively, reducing the thermal energy requirement from 7.8 to 5.1 and 4.7 GJ/tCO2."),
("Results, Catalyst concentration and stability, para 3",
"the RH% increased from 64.8% in cycle 1 to 68.8% in cycle 2"),
("Results, Catalytic hybrid amino acid solvents in MVR (Fig. 5f)",
"Considering GlyK as a benchmark, the catalyst reduced the RH of GlyK, SarK, and TauK to 64.8%, 90.1%, and 55.8%, respectively."),
("Results, Catalytic hybrid amino acid solvents in MVR (Fig. 5f)",
"Notably, the combination of a hybrid solvent with the catalyst further reduced the RH to 33.2%, corresponding to a 66.8% reduction in energy consumption."),
("Discussion",
"9 wt% fixed-bed catalyst was identified as the optimal concentration, achieving a 35.2% reduction in the system’s energy input."),
("Methods, MVR performance analysis, Eq. 7",
"The relative heat duty, RH (%), was calculated using Eq. 7"),
]
SHOW = os.path.join(HERE, "..", "..", "..", "src", "show.py")
if os.path.exists(SHOW):
print("== 1. Verbatim check against the PMC archive (src/show.py --full)")
full = subprocess.run([sys.executable, SHOW, "PMC12966427", "--full"], capture_output=True,
text=True, cwd=os.path.join(HERE, "..", "..", "..")).stdout
norm = lambda s: " ".join(s.split())
fulln = norm(full)
for loc, q in QUOTES:
print(f" [{'found' if norm(q) in fulln else 'NOT FOUND'}] {loc}: {q[:70]}...")
n597 = fulln.count("59.7")
print(f" occurrences of '59.7' in the archived text: {n597} (abstract only)")
else:
print("== 1. src/show.py not available; verbatim check skipped")
# ---------------------------------------------------------------- 2. the paper's own inputs
Q_BLANK = 7.8 # GJ/tCO2, 3 M GlyK blank (Results, Fig. 3g text)
M_BLANK = 244 # mmol CO2 desorbed in 30 min, blank
M_CAT = {6: 307, 9: 377} # mmol, 1:1 Fe-SZ/SiO2 at 6 and 9 wt%
CE_REP = {6: 25.6, 9: 54.3, 12: 67.5} # reported catalytic enhancement (%)
RED_REP = {9: 35.2, 12: 40.3} # reported heat-duty reduction (%)
Q_REP = {9: 5.1, 12: 4.7} # reported thermal energy (GJ/tCO2)
ABSTRACT_RED = 59.7 # abstract: "reduces relative heat duty by up to 59.7%"
print("\n== 2. Catalytic enhancement, Eq. 4, from the desorbed CO2 amounts")
print(f" 6 wt%: (307/244 - 1) = {(M_CAT[6]/M_BLANK - 1)*100:.1f}% (reported 25.6%)")
print(f" 9 wt%: (377/244 - 1) = {(M_CAT[9]/M_BLANK - 1)*100:.1f}% (reported 54.3%)")
print(f" 6 -> 9 wt% relative improvement: 54.3/25.6 - 1 = {(CE_REP[9]/CE_REP[6] - 1)*100:.1f}% (reported 112%)")
print(f" 9 -> 12 wt% relative improvement: 67.5/54.3 - 1 = {(CE_REP[12]/CE_REP[9] - 1)*100:.1f}% (reported 24.3%)")
print("\n== 3. Relative heat duty (Eq. 7) and reduction, two routes")
ROWS = {}
for w in (6, 9, 12):
rh_ce = 100.0 / (1 + CE_REP[w] / 100) # from reported CE
rh_m = 100.0 * M_BLANK / M_CAT[w] if w in M_CAT else None # from mmol, where printed
q = Q_BLANK * rh_ce / 100
ROWS[w] = dict(rh=rh_ce, red=100 - rh_ce, q=q, rh_m=rh_m)
s_m = f"; from mmol RH = {rh_m:.1f}%, reduction {100 - rh_m:.1f}%" if rh_m else ""
print(f" {w:>2} wt%: from CE, RH = 100/(1+{CE_REP[w]/100:.3f}) = {rh_ce:.2f}%, reduction {100 - rh_ce:.1f}%,"
f" Q = 7.8 x {rh_ce/100:.3f} = {q:.2f} GJ/t{s_m}")
print(f" reported: 9 wt% reduction 35.2% (RH 64.8%, Q 5.1); 12 wt% reduction 40.3% (Q 4.7)")
print(f" check from reported energies: 1 - 5.1/7.8 = {(1 - 5.1/7.8)*100:.1f}%; 1 - 4.7/7.8 = {(1 - 4.7/7.8)*100:.1f}%")
print(f" 100 - 35.2 = {100 - 35.2:.1f} (RH at 9 wt%, also stability cycle 1 and Fig. 5f GlyK value)")
print(f" 100 - 40.3 = {100 - 40.3:.1f} (RH at 12 wt%) <- equals the abstract's 59.7")
print("\n== 4. What the abstract's wording would imply")
rh_abs = 100 - ABSTRACT_RED
q_abs = Q_BLANK * rh_abs / 100
ce_abs = (100 / rh_abs - 1) * 100
print(f" 'reduces RH by 59.7%' -> RH = {rh_abs:.1f}%, Q = 7.8 x {rh_abs/100:.3f} = {q_abs:.2f} GJ/tCO2,"
f" requiring CE = {ce_abs:.0f}% (largest reported catalyst-only CE: 67.5%)")
print(f" ratio abstract / largest body reduction (12 wt%): 59.7/40.3 = {ABSTRACT_RED/RED_REP[12]:.2f}")
print(f" ratio abstract / optimum reduction (9 wt%): 59.7/35.2 = {ABSTRACT_RED/RED_REP[9]:.2f}")
print("\n== 5. Other headline numbers (consistency)")
print(f" hybrid + catalyst: RH 33.2% -> reduction {100 - 33.2:.1f}% (reported 66.8%);"
f" Q = 7.8 x 0.332 = {Q_BLANK*0.332:.2f} GJ/t (reported 2.6)")
print(f" SarK + catalyst RH 90.1% -> reduction {100-90.1:.1f}%; TauK + catalyst RH 55.8% -> reduction {100-55.8:.1f}%"
" (neither is a GlyK catalyst-only value, and neither equals 59.7% as a reduction)")
print(f" 6 wt% (1/2): (278/244 - 1) = {(278/244 - 1)*100:.1f}% (reported 13.7%)")
print(f" enhancement factor CE/W (Eq. 5): 6 wt% {CE_REP[6]/6:.2f}, 9 wt% {CE_REP[9]/9:.2f} (reported 6.0),"
f" 12 wt% {CE_REP[12]/12:.2f} -> 9 wt% is the maximum, as stated")
print(f" stability: 68.8/64.8 - 1 = {(68.8/64.8 - 1)*100:.1f}% (reported ~6.2%); 71.8/68.8 - 1 = {(71.8/68.8 - 1)*100:.1f}% (reported ~4.3%)")
print(f" hybrid vs TauK desorption: 589/348 - 1 = {(589/348 - 1)*100:.1f}% (abstract 69.1%, body 69.2%)")
print("\n== 6. Alternative readings of 59.7 (explanations considered)")
for lab, ce in (("6 wt% 1:1", 25.6), ("6 wt% 1:2", 13.7), ("6 wt% 2:1 (~4.0%)", 4.0)):
print(f" heat-duty reduction, {lab}: 100 - 100/(1+{ce/100:.3f}) = {100 - 100/(1 + ce/100):.1f}%")
print(f" reduction relative to 6 wt% instead of blank: 9 wt% {(1 - ROWS[9]['rh']/ROWS[6]['rh'])*100:.1f}%,"
f" 12 wt% {(1 - ROWS[12]['rh']/ROWS[6]['rh'])*100:.1f}%")
print(f" peak CO2 flux gain vs blank (19.3e-5): 9 wt% 28.2 -> {(28.2/19.3 - 1)*100:.1f}%, 12 wt% 30.2 -> {(30.2/19.3 - 1)*100:.1f}%")
lo, hi = (1 - 3.25/7.75)*100, (1 - 3.15/7.85)*100
print(f" hybrid solvent (3 M TauK + 1 M SarK) WITHOUT catalyst vs GlyK: 1 - 3.2/7.8 = {(1 - 3.2/7.8)*100:.1f}%"
f" (rounding range {lo:.1f}-{hi:.1f}%) -- a solvent effect, not the catalyst")
print(f" TauK alone vs GlyK (no catalyst): 1 - 5.5/7.8 = {(1 - 5.5/7.8)*100:.1f}%")
# ---------------------------------------------------------------- figures
BG, FG, FG2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
ORANGE_L, BLUE_L = "#f8d3c4", "#cfe0f5"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "svg.fonttype": "path",
"text.color": FG, "axes.labelcolor": FG, "xtick.color": FG2,
"ytick.color": FG, "axes.edgecolor": FG2})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# --- Figure 1: abstract vs body, two text panels joined by an annotated connector
fig = plt.figure(figsize=(7.2, 5.6))
fig.patch.set_facecolor(BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 78)
ax.axis("off")
def box(x, y, w, h, ec, lw=2, ls="-"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc=BG, ec=ec, lw=lw, ls=ls))
# left: abstract
box(2.5, 30, 40, 44, ORANGE)
ax.text(4.5, 71, "Abstract", fontsize=12, weight="bold", va="top")
ax.text(4.5, 65.5, "“An optimal iron-sulfated\nzirconia to silica ratio of\n1:1 reduces relative heat\n"
"duty by up to 59.7% in\nthe membrane vacuum\nsystem.”", fontsize=10.5, va="top", linespacing=1.35)
ax.text(4.5, 38.5, "Reduction stated:", fontsize=10, color=FG2, va="center")
ax.text(4.5, 33.8, "59.7% (reported)", fontsize=12, color=ORANGE, weight="bold", va="center")
# right: results + discussion
box(57.5, 30, 40, 44, FG2)
ax.text(59.5, 71, "Results (Fig. 3g text)\nand Discussion", fontsize=12, weight="bold", va="top")
ax.text(59.5, 62.5, "“at 9 wt% and 12 wt%, the heat\nduty decreased by 35.2 and\n40.3% as compared to the\n"
"blank” (7.8 → 5.1 and 4.7 GJ/tCO2)", fontsize=10, va="top", linespacing=1.3)
ax.text(59.5, 46.5, "“9 wt% ... optimal ... achieving\na 35.2% reduction”", fontsize=10, va="top",
linespacing=1.3)
ax.text(59.5, 38.5, "Reductions stated:", fontsize=10, color=FG2, va="center")
ax.text(59.5, 33.8, "35.2% (9 wt%), 40.3% (12 wt%)", fontsize=10.5, color=ORANGE, weight="bold", va="center")
# connector
arr = FancyArrowPatch((43.5, 52), (56.5, 52), arrowstyle="<->", mutation_scale=16, lw=2, color=FG)
ax.add_patch(arr)
ax.text(50, 55, "conflict", ha="center", va="bottom", fontsize=10, color=FG, weight="bold")
ax.text(50, 49, "59.7\nvs\n35.2 / 40.3", ha="center", va="top", fontsize=10,
color=FG2, linespacing=1.3)
# bottom: recomputation that names the conflict
box(2.5, 3, 95, 21, BLUE)
ax.text(4.5, 21.5, "Recomputed from the paper's own Eq. 7 (RH = Q_cat / Q_blank = 1 / (1 + CE))",
fontsize=10.5, va="top", color=BLUE, weight="bold")
r9, r12 = ROWS[9], ROWS[12]
ax.text(4.5, 16.2,
f"9 wt% (CE 54.3%): RH = {r9['rh']:.1f}% → reduction {r9['red']:.1f}% ({r9['q']:.2f} GJ/tCO2)\n"
f"12 wt% (CE 67.5%): RH = {r12['rh']:.1f}% → reduction {r12['red']:.1f}% ({r12['q']:.2f} GJ/tCO2)\n"
f"59.7% is the heat duty that remains at 12 wt%, not the reduction. A 59.7% reduction\n"
f"would mean RH = {rh_abs:.1f}% and {q_abs:.2f} GJ/tCO2; no such value appears in the paper.",
fontsize=10, va="top", color=FG, linespacing=1.4)
save(fig, "fig1")
# --- Figure 2: heat duty remaining vs removed, by catalyst loading, plus the abstract's reading
labels = ["Blank (3 M GlyK)", "6 wt%", "9 wt% (optimum)", "12 wt%",
"Abstract as written\n(“by up to 59.7%”)"]
rh = [100.0, ROWS[6]["rh"], ROWS[9]["rh"], ROWS[12]["rh"], rh_abs]
q = [Q_BLANK, ROWS[6]["q"], ROWS[9]["q"], ROWS[12]["q"], q_abs]
is_abs = [False, False, False, False, True]
fig, ax = plt.subplots(figsize=(7.2, 4.4))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
ys = np.arange(len(labels))[::-1].astype(float)
ys[-1] -= 0.35
for y, r, qq, ab in zip(ys, rh, q, is_abs):
c, cl = (ORANGE, ORANGE_L) if ab else (BLUE, BLUE_L)
ax.barh(y, r, height=0.6, color=cl, edgecolor=c, lw=1.5, zorder=2)
if r < 100:
ax.barh(y, 100 - r, left=r, height=0.6, color=c, edgecolor=c, lw=1.5, zorder=2)
ax.text(r + (100 - r) / 2, y, f"{100 - r:.1f}%", ha="center", va="center", color="white",
fontsize=10.5, weight="bold", zorder=3)
ax.text(r / 2, y, f"{r:.1f}%", ha="center", va="center", color=FG, fontsize=10.5, zorder=3)
ax.text(102, y, f"{qq:.2f}" if not ab else f"{qq:.2f}*", ha="left", va="center",
color=ORANGE if ab else BLUE, fontsize=10.5)
ax.text(102, ys[0] + 0.55, "GJ/tCO2", ha="left", va="bottom", color=FG2, fontsize=10)
ax.add_patch(FancyArrowPatch((rh_abs + (100 - rh_abs) / 2, ys[4] + 0.33), (ROWS[12]["rh"] / 2, ys[3] - 0.33),
arrowstyle="->", mutation_scale=14, lw=1.5, color=FG, zorder=4,
connectionstyle="arc3,rad=-0.15"))
ax.text(73, (ys[3] + ys[4]) / 2, "same number", ha="left", va="center",
fontsize=10, color=FG, zorder=4)
ax.set_xlim(0, 100)
ax.set_ylim(ys[-1] - 0.6, ys[0] + 0.85)
ax.set_yticks(ys)
ax.set_yticklabels(labels, fontsize=10)
ax.set_xlabel("Share of blank heat duty (%): light = remaining (RH), solid = reduction")
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
for s_ in ("top", "right", "left"):
ax.spines[s_].set_visible(False)
handles = [plt.Rectangle((0, 0), 1, 1, fc=BLUE, ec=BLUE,
label="Recomputed from the paper's enhancements (Eq. 7)"),
plt.Rectangle((0, 0), 1, 1, fc=ORANGE, ec=ORANGE,
label="Abstract's 59.7% read as a reduction (* implied energy)")]
ax.legend(handles=handles, loc="upper left", bbox_to_anchor=(-0.3, -0.16), ncol=1, frameon=False,
fontsize=10)
fig.subplots_adjust(left=0.27, right=0.88, top=0.93, bottom=0.27)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
The abstract reports a positive temperature association with H7 spread; the paper's own coefficients are negative
Nat Commun (2026) · doi:10.1038/s41467-026-72718-9 · PMC13351079
Summary
The abstract states that "temperature and poultry density show positive association" with the spread of H7 influenza viruses. The paper's Bayesian phylogeographic GLM reports the opposite sign for temperature: β = −0.484 (95% HPD −0.900 to −0.237) at origin and β = −0.227 (95% HPD −0.499 to −0.047) at destination. These are the estimates of the model without sample size (Fig. 9A). Both intervals lie entirely below zero, so by the paper's own rule both effects are significant and negative, and the Results and Discussion describe them that way ("a negative and supportive predictor"; "the negative correlation between temperature and transmission risk"). In the alternative models that add sample size as a predictor (Fig. 9B; Supplementary Fig. 24B), temperature is rarely included (activation rates of 2.7% or less) and its intervals span zero, so no model in the paper gives a supported positive temperature effect. The poultry-density half of the sentence agrees with the reported chicken-stock coefficient (+0.461); only the temperature direction is reversed. The error sits in the most-read part of the paper and concerns one of the named drivers of spread, but the underlying estimates and the body text are consistent with each other.
What the paper reports
In the abstract:
Geographic distance is negatively associated with the spread of the H7 virus, while temperature and poultry density show positive association.
— Abstract
The editor's summary printed with the abstract names temperature as a driver without giving a direction:
Authors show that spread of H7 avian influenza viruses is driven by geographic distance and temperature.
— Abstract (editor's summary)
In the Results, the GLM estimates are given as posterior mean and 95% HPD:
Geographic distance was strongly negatively associated with viral spread (coefficient = −0.873, 95% HPD: −0.995 to −0.753), with decisive support for its inclusion in the model (activation rate = 1) (Fig. 9A).
Chicken stock origin, defined as the number of annual chicken inventory at the source country, was consistently positively associated with virus spread (0.461, 0.111 to 0.845), while temperature origin (−0.484, −0.900 to −0.237) and temperature destination (−0.227, −0.499 to −0.047) was also considered as a negative and supportive predictor.
— Results, "Mechanisms of virus spatial spread"
In the Discussion:
Chicken stock origin consistently made a positive contribution to the spread of the virus, while temperature origin was considered as a supportively negative predictor.
Influenza viruses degrade more quickly in warm or dry conditions56, so high temperature may reduce viral persistence in the environment, which explains the negative correlation between temperature and transmission risk.
— Discussion, paragraph on the GLM
The concluding paragraph lists temperature without a direction:
Our findings highlight how geographic distance, temperature, GDP, and poultry density collectively shape viral dispersal across regions.
— Discussion, concluding paragraph
The Methods define the model and the significance rule:
This GLM approach was extended from the discrete diffusion model, which characterizes the logarithm of the transition rates within the CTMC matrix as a log-linear function dependent on multiple potential indicators of viral spread.
All predictors were log-transformed and standardized except bird migration before inclusion in the matrices.
HPD intervals (Highest Posterior Density intervals) from the posterior distribution were examined for each coefficient: if the HPD interval does not contain zero, the effect is considered significant; if it contains zero, the effect is uncertain.
— Methods, "Phylogeographic inference with epidemiological predictors"
Each quotation was checked against the article as archived in PubMed Central (src/show.py PMC13351079) and occurs exactly once; recompute.py.txt reruns that check. The archived PMC text renders minus signs as ASCII hyphens; the publisher's HTML uses the Unicode minus sign (U+2212), which is what is shown above. Footnote numerals (e.g. "56") are reproduced as they appear in the archived text.
What the paper's own numbers show
Method. The claim concerns direction only, so the check uses the paper's own estimates and its own decision rule. Under the stated model, log Λij = Σp βpδpxij,p, a negative β means that a higher (log-transformed, standardized) predictor value is associated with a lower transition rate, conditional on the predictor's inclusion. For each coefficient reported in the Results text, the sign of the 95% HPD interval was classified (entirely above zero: positive; entirely below: negative; straddling zero: uncertain) and compared with the direction the abstract states. As a descriptive aid, exp(β) gives the multiplicative change in the transition rate per one standard deviation of the predictor; its interval is the one-to-one transform of the reported HPD endpoints, which is not necessarily an HPD on that scale. No posterior samples or Fig. 9 source data are published, so the model itself cannot be re-fitted; the check is confined to the numbers the paper prints. recompute.py.txt prints every value below.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Direction of temperature association | Abstract | "positive association" | Contradicted by both temperature coefficients (rows below) |
| Temperature (origin), β and 95% HPD | Results, "Mechanisms of virus spatial spread" | −0.484 (−0.900 to −0.237); "a negative and supportive predictor" | HPD entirely below 0 (upper bound −0.237): negative, significant by the paper's rule; exp(β) = 0.616 (0.407 to 0.789), about 38% lower rate per SD |
| Temperature (destination), β and 95% HPD | Same paragraph | −0.227 (−0.499 to −0.047); same wording | HPD entirely below 0 (upper bound −0.047): negative; exp(β) = 0.797 (0.607 to 0.954), about 20% lower rate per SD |
| Direction of temperature association | Discussion | "supportively negative predictor"; "negative correlation between temperature and transmission risk" | Agrees with the coefficients |
| Chicken stock (origin), the abstract's "poultry density" | Results | +0.461 (0.111 to 0.845); abstract: positive | HPD entirely above 0: positive; exp(β) = 1.586 (1.117 to 2.328). Abstract agrees |
| Geographic distance | Results | −0.873 (−0.995 to −0.753); abstract: negative | HPD entirely below 0: negative; exp(β) = 0.418 (0.370 to 0.471). Abstract agrees |
| Abstract direction statements checked against the reported HPDs | Abstract vs Results | 4 directions (distance, poultry density, and temperature at origin and destination) | 2 of 4 agree (distance, poultry density); 2 of 4 contradicted (both temperature terms) |
Both temperature estimates printed in the text are negative, with intervals that exclude zero, and the paper's own text describes and explains a negative relationship. The abstract's "positive" cannot be derived from these numbers.
Figure
Figure 1. The conflicting statements, side by side. Left (orange): the abstract sentence, with the clause that assigns a positive association to temperature and poultry density highlighted. Right (blue): the Results sentence that reports both temperature coefficients with their 95% HPD intervals and calls temperature "a negative and supportive predictor," and the two Discussion sentences that call it "a supportively negative predictor" and explain "the negative correlation between temperature and transmission risk." The footer notes that the abstract's directions for distance and poultry density (chicken stock origin) agree with the reported coefficients; only the temperature direction conflicts.
Figure 2. Reported GLM coefficients and the direction each implies. Orange dots and lines are the posterior means and 95% HPD intervals as printed in the Results text (log scale of the transition rate; the vertical line at 0 marks no effect). At right, "Abstract says" gives the direction stated in the abstract (a dash where it states none), and the blue "Sign of reported HPD" column gives the direction implied by each interval under the paper's rule. The two temperature rows, in bold, are the only ones where the two columns disagree; both intervals lie entirely to the left of zero. The sample-size coefficients are shown for context only; according to the Fig. 9 legend they come from the model that includes sample size (Fig. 9B). The other four are from the Fig. 9A model (the distance value is cited to Fig. 9A in the text; the others match the Fig. 9A plot).
Why it matters
- The abstract reverses the direction of a named driver. Temperature is one of only two drivers named in the editor's summary ("driven by geographic distance and temperature"). A reader who sees only the abstract, as in indexing and alerting services, is told that warmer conditions go with more H7 spread, when the paper's estimates say warmer conditions go with less (about 38% lower transition rate per SD of log-transformed origin temperature, conditional on inclusion).
- The stated direction matters for the paper's own recommendations. The Discussion proposes "[I]ntegrating real-time environmental variables such as temperature monitoring into early warning systems" and explains the negative effect through faster viral degradation in warm conditions. Read with the abstract's positive sign, that mechanism would be contradicted and the direction of a temperature-based alert would be reversed.
- The main conclusions are unaffected. The coefficients themselves, the Results and the Discussion agree with one another. Nothing else in the paper depends on the abstract's sign: not the distance effect, the host-specific spread velocities, the roles of Anseriformes and Galliformes, Asia as an epicentre, or the One Health recommendations. The error is confined to one clause of one abstract sentence.
Explanations considered
- A different model reports a positive temperature effect. No model gives a supported positive effect. The paper fits a model without sample size (Fig. 9A) and one with it (Fig. 9B), and repeats both with GDP per capita added (Supplementary Fig. 24A and B, described as "similar"). In the two models without sample size, both temperature terms are negative: the text gives −0.484 and −0.227 for Fig. 9A, and Supplementary Fig. 24A plots means of about −0.45 and −0.2 with posterior draws below zero (activation rate 56.7% for Temperature (o)). In the two models with sample size, temperature has almost no inclusion support and intervals that span zero. In Fig. 9B the plotted means are about −0.33 (origin) and −0.02 (destination), with activation rates of 1.9% and 0.1%. In Supplementary Fig. 24B the Temperature (o) mean is plotted slightly above zero (about +0.15), but its median is about −0.35, its 95% HPD runs from roughly −0.8 to +2.4, and its activation rate is 2.7%; Temperature (d) sits near zero at 0.1%. Under the paper's rule, these are "uncertain" effects, not positive ones. The values for Fig. 9B and Supplementary Fig. 24 are read by eye from the publisher's figure images, as no source data are published for them. The model the text reports and the abstract summarizes is the one with significant negative temperature terms.
- "Positive association" means "supported for inclusion," not direction. This does not fit the sentence. The same sentence uses "negatively associated" for distance, which can only be a direction, and the Results keep the two ideas separate by calling temperature "a negative and supportive predictor." Even under this reading, the destination term would not qualify: the activation rate printed for Temperature (d) in Fig. 9A is 0.7%, which falls in the band the Methods describe as "<5% suggests potential unimportance." (Temperature (o) is printed at 68.0%.)
- Sign convention or transformation. The Methods specify a log-linear model of the transition rates with log-transformed, standardized predictors. Under that parameterization a negative β is a negative association, and the Discussion reads it that way. The transformation of the temperature variable does not change the sign of its association with spread.
- Confusion with a correlated predictor. The Results report a Spearman correlation of r = −0.65 between GDP per capita and temperature, and GDP was dropped from the final model for that reason. That is a correlation between two predictors, not the association of temperature with spread, and adding GDP does not produce a supported positive temperature effect (item 1): in Supplementary Fig. 24A both temperature terms remain negative, and in Supplementary Fig. 24B the origin term is included in 2.7% of samples with an interval spanning zero.
- "Poultry density" carries the positive sign and temperature was listed alongside it. The sentence does not allow that reading. "temperature and poultry density show positive association" makes the two nouns the joint subject of one plural verb, so as printed it assigns a positive direction to both. It also contrasts them ("while") with distance, which is "negatively associated". A reader cannot recover the negative temperature effect from the abstract.
- Rounding, a typo in the coefficients, or a text-conversion error. None applies. The sign is carried by the words "negative" and "positive" as well as by the minus signs, and the minus signs in the Results text agree with the Fig. 9A plot, where both temperature distributions lie below the zero line. The publisher's HTML and the PMC archive give the same abstract wording.
Limitations
- The check uses only the numbers the paper prints. No posterior samples, log files or Fig. 9 source data are published (the Source Data file covers Figs 1 and 8), so the GLM was not refitted and the HPD intervals were not recomputed.
- The values for Fig. 9B and Supplementary Fig. 24A and B are not given in the text and were read by eye from the publisher's figure images. They are approximate, but the finding does not depend on their exact values: none of them is a supported positive effect.
- exp(β) is shown only as a descriptive aid. It is conditional on the predictor's inclusion, and its interval is a transform of the reported HPD endpoints, not an HPD on that scale.
Status
As of 2026-10-23:
- Quotations. The 13 passages quoted in this write-up (listed in
recompute.py.txt) were each found exactly once in the PMC-archived text (src/show.py PMC13351079 --full);recompute.py.txtreruns that check, treating the PMC text's ASCII hyphens as the publisher's minus signs. - Numbers. Every number in the table and in Figure 2 is re-derived by
recompute.py.txtfrom the coefficients printed in the Results text. - Figures in the paper. Fig. 9 (publisher's image) and Supplementary Fig. 24 (supplementary PDF, p. 22) were inspected by eye; no values were extracted from them beyond the approximate readings in item 1.
- Refit. Not possible: no posterior samples or GLM source data are available.
- Corrections.
src/known.py(run 2026-10-23) returned empty lists: no Crossref updates and no PMC or Europe PMC correction or retraction notices.
Sources: PMC13351079 (PubMed Central archive, read with src/show.py); https://doi.org/10.1038/s41467-026-72718-9 (publisher's article page and Fig. 9 image); Supplementary Information PDF (41467_2026_72718_MOESM1_ESM.pdf), p. 22.
Suggested correction
In the Abstract, replace "while temperature and poultry density show positive association" with "while poultry density shows a positive association and temperature a negative association", so that the abstract matches the coefficients in the Results (temperature origin −0.484, 95% HPD −0.900 to −0.237; temperature destination −0.227, −0.499 to −0.047) and the Discussion. The authors could also state in the Results that, in the model that includes sample size (Fig. 9B), temperature has low inclusion support (activation rates 1.9% and 0.1%) and intervals that span zero.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC13351079 (Nat Commun 2026, doi:10.1038/s41467-026-72718-9)
# Abstract vs Results/Discussion: direction of the temperature association in the
# Bayesian phylogeographic GLM of H7 influenza spread.
#
# Run: python recompute.py.txt
# Prints every number used in writeup.md and regenerates fig1.{svg,png} and fig2.{svg,png}
# in the folder that holds this file. Needs matplotlib (numpy not required).
#
# Inputs
# * Quotes: copied verbatim from the article as archived in PubMed Central (PMC13351079).
# The PMC text served by src/show.py writes minus signs as ASCII '-', while the
# publisher's HTML uses U+2212; the check below treats the two as the same character.
# When run inside the /home/ubuntu/papers repo, each quote is re-checked against the
# archived full text (skipped elsewhere).
# * GLM coefficients: the posterior means and 95% HPD bounds printed in the Results
# paragraph "Mechanisms of virus spatial spread". No posterior samples or source data for
# Fig. 9 are published (the Source Data file covers Figs 1 and 8 only), so the model cannot
# be re-fitted; the check is on the paper's own reported numbers.
import math, os, subprocess
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
PMCID = "PMC13351079"
# ------------------------------------------------------------------ quotes
QUOTES = {
"abstract": ("Abstract",
"Geographic distance is negatively associated with the spread of the H7 virus, while "
"temperature and poultry density show positive association."),
"editor_summary": ("Abstract (editor's summary)",
"Authors show that spread of H7 avian influenza viruses is driven by geographic distance "
"and temperature."),
"results_distance": ("Results, 'Mechanisms of virus spatial spread'",
"Geographic distance was strongly negatively associated with viral spread (coefficient = "
"-0.873, 95% HPD: -0.995 to -0.753), with decisive support for its inclusion in the model "
"(activation rate = 1) (Fig. 9A)."),
"results_temp": ("Results, 'Mechanisms of virus spatial spread'",
"Chicken stock origin, defined as the number of annual chicken inventory at the source "
"country, was consistently positively associated with virus spread (0.461, 0.111 to 0.845), "
"while temperature origin (-0.484, -0.900 to -0.237) and temperature destination (-0.227, "
"-0.499 to -0.047) was also considered as a negative and supportive predictor."),
"results_gdp": ("Results, 'Mechanisms of virus spatial spread'",
"The results incorporating GDP were similar (Supplementary Fig. 24)."),
"disc_temp": ("Discussion, paragraph on the GLM",
"Chicken stock origin consistently made a positive contribution to the spread of the virus, "
"while temperature origin was considered as a supportively negative predictor."),
"disc_mech": ("Discussion, paragraph on the GLM",
"Influenza viruses degrade more quickly in warm or dry conditions56, so high temperature may "
"reduce viral persistence in the environment, which explains the negative correlation "
"between temperature and transmission risk."),
"conclusion": ("Discussion, concluding paragraph",
"Our findings highlight how geographic distance, temperature, GDP, and poultry density "
"collectively shape viral dispersal across regions."),
"disc_ews": ("Discussion, paragraph on prevention and control",
"Integrating real-time environmental variables such as temperature monitoring into early "
"warning systems"),
"methods_glm": ("Methods, 'Phylogeographic inference with epidemiological predictors'",
"This GLM approach was extended from the discrete diffusion model, which characterizes the "
"logarithm of the transition rates within the CTMC matrix as a log-linear function dependent "
"on multiple potential indicators of viral spread."),
"methods_std": ("Methods, 'Phylogeographic inference with epidemiological predictors'",
"All predictors were log-transformed and standardized except bird migration before "
"inclusion in the matrices."),
"methods_hpd": ("Methods, 'Phylogeographic inference with epidemiological predictors'",
"HPD intervals (Highest Posterior Density intervals) from the posterior distribution were "
"examined for each coefficient: if the HPD interval does not contain zero, the effect is "
"considered significant; if it contains zero, the effect is uncertain."),
"methods_act": ("Methods, 'Phylogeographic inference with epidemiological predictors'",
"Activation rate (the frequency with which a predictor is included in the model) was used "
"to gauge importance: an activation rate ≥50% indicates a highly important predictor, "
"10–50% indicates moderate importance, and <5% suggests potential unimportance."),
}
def norm(s):
return " ".join(s.replace("−", "-").split())
def check_quotes():
repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
print("Quote check: src/show.py not available here; skipped.\n")
return
try:
full = subprocess.run([py, "src/show.py", PMCID, "--full"], cwd=repo,
capture_output=True, text=True, timeout=300).stdout
except Exception as e: # pragma: no cover
print(f"Quote check: could not load archived text ({e}); skipped.\n")
return
full_n = norm(full)
print("Quote check against the PMC-archived full text (occurrences; >=1 means found verbatim):")
for key, (loc, q) in QUOTES.items():
print(f" {key:17s} {full_n.count(norm(q))} ({loc})")
print()
# ------------------------------------------------------------------ reported GLM numbers
# (label, mean, HPD low, HPD high, direction the abstract states or None)
# All from the Results paragraph quoted above (Fig. 9A model, except sample size, which the
# Fig. 9 legend places in the model of Fig. 9B).
COEFS = [
("Geographic distance", -0.873, -0.995, -0.753, "negative"),
("Chicken stock (origin)", 0.461, 0.111, 0.845, "positive"), # "poultry density"
("Temperature (origin)", -0.484, -0.900, -0.237, "positive"),
("Temperature (destination)", -0.227, -0.499, -0.047, "positive"),
("Sample size (origin)", 1.200, 0.777, 1.600, None),
("Sample size (destination)", 0.598, 0.402, 0.809, None),
]
def classify(lo, hi):
if lo > 0:
return "positive"
if hi < 0:
return "negative"
return "uncertain (HPD contains 0)"
def recompute():
print("Reported GLM coefficients (log scale) and the direction they imply under the paper's rule")
print("('if the HPD interval does not contain zero, the effect is considered significant'):\n")
print(f" {'predictor':27s} {'beta':>7s} {'95% HPD':>17s} {'implied':>9s} {'abstract says':>13s} agree?")
out = []
for lab, m, lo, hi, abs_dir in COEFS:
imp = classify(lo, hi)
agree = "-" if abs_dir is None else ("yes" if abs_dir == imp else "NO")
print(f" {lab:27s} {m:7.3f} [{lo:6.3f}, {hi:6.3f}] {imp:>9s} {abs_dir or '(not stated)':>13s} {agree}")
out.append((lab, m, lo, hi, abs_dir, imp, agree))
print()
print("Rate multipliers exp(beta) per 1 SD of the log-transformed, standardized predictor,")
print("conditional on inclusion (endpoints transformed one-to-one; not necessarily an HPD on")
print("the multiplier scale):")
for lab, m, lo, hi, *_ in out:
print(f" {lab:27s} exp(beta) = {math.exp(m):.3f} [{math.exp(lo):.3f}, {math.exp(hi):.3f}]")
print()
for lab, m, lo, hi, *_ in out:
if lab.startswith("Temperature"):
print(f" {lab}: upper HPD bound {hi:+.3f} < 0 -> {hi < 0}; "
f"multiplier {math.exp(m):.3f} -> {100 * (1 - math.exp(m)):.1f}% lower rate per SD")
n_temp_neg = sum(1 for r in out if r[0].startswith("Temperature") and r[5] == "negative")
print(f"\n Temperature coefficients whose HPD lies entirely below 0: {n_temp_neg} of 2")
print(f" Abstract direction statements contradicted by the reported HPDs: "
f"{sum(1 for r in out if r[6] == 'NO')} of {sum(1 for r in out if r[4])}\n")
return out
# ------------------------------------------------------------------ figures
BG, TXT, TXT2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "svg.fonttype": "none",
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"text.color": TXT, "axes.labelcolor": TXT, "xtick.color": TXT2,
"ytick.color": TXT})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
print(f"wrote {name}.svg and {name}.png")
def fig1():
W, H = 7.2, 5.0
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
def panel(x, w, title, sub, body, accent):
ax.add_patch(FancyBboxPatch((x, 0.95), w, 3.9, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=accent, lw=1.5))
ax.text(x + 0.14, 4.72, title, fontsize=11.5, fontweight="bold", va="top", color=TXT)
ax.text(x + 0.14, 4.42, sub, fontsize=10, va="top", color=accent)
yy = 4.08
for line, style in body:
ax.text(x + 0.14, yy, line, fontsize=10, va="top",
color={"n": TXT, "b": accent, "s": TXT2}[style],
fontweight="bold" if style == "b" else "normal",
fontstyle="italic" if style == "s" else "normal")
yy -= 0.235
left = [("“Geographic distance is negatively", "n"),
("associated with the spread of", "n"),
("the H7 virus, while", "n"),
("temperature and poultry", "b"),
("density show positive", "b"),
("association.”", "b"),
("", "n"), ("", "n"), ("", "n"), ("", "n"), ("", "n"),
("States: temperature > 0", "s"),
("(and poultry density > 0).", "s")]
right = [("Results, GLM paragraph:", "s"),
("“…temperature origin (−0.484, −0.900", "n"),
("to −0.237) and temperature destination", "n"),
("(−0.227, −0.499 to −0.047) was also", "n"),
("considered as a negative and", "b"),
("supportive predictor.”", "b"),
("Discussion:", "s"),
("“…temperature origin was considered", "n"),
("as a supportively negative predictor.”", "b"),
("“…which explains the negative", "b"),
("correlation between temperature and", "b"),
("transmission risk.”", "b"),
("Both temperature HPDs lie below 0.", "s")]
panel(0.15, 2.75, "Abstract", "reported claim", left, ORANGE)
panel(3.62, 3.43, "Results and Discussion", "the paper's own estimates and text", right, BLUE)
ax.add_patch(FancyArrowPatch((2.94, 3.55), (3.58, 3.55), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=TXT))
ax.text(3.26, 3.85, "conflict", ha="center", fontsize=10, fontweight="bold", color=TXT)
ax.text(3.26, 3.28, "sign of\ntemper-\nature", ha="center", va="top", fontsize=10, color=TXT2)
ax.text(0.15, 0.48,
"Distance (β = −0.873) and chicken stock origin (β = +0.461; the abstract's “poultry\n"
"density”) have the signs the abstract states. Only the temperature direction differs.",
fontsize=10, color=TXT2, va="center")
save(fig, "fig1")
def fig2(rows):
n = len(rows)
fig, ax = plt.subplots(figsize=(7.2, 3.9))
fig.subplots_adjust(left=0.29, right=0.66, top=0.86, bottom=0.17)
ax.set_xlim(-1.3, 1.8)
ax.set_ylim(n - 0.4, -0.7)
ax.axvline(0.0, color=TXT2, lw=1.0, zorder=1)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_xticks([-1.0, -0.5, 0.0, 0.5, 1.0, 1.5])
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(TXT2)
ax.tick_params(axis="y", length=0)
ax.set_yticks(range(n))
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xlabel("GLM coefficient β (log scale of transition rate)", fontsize=10, color=TXT)
for i, (lab, m, lo, hi, abs_dir, imp, agree) in enumerate(rows):
ax.plot([lo, hi], [i, i], color=ORANGE, lw=2, solid_capstyle="butt", zorder=3)
ax.plot([m], [i], "o", ms=8, color=ORANGE, zorder=4)
fw = "bold" if agree == "NO" else "normal"
ax.text(1.03, i, abs_dir or "—", transform=ax.get_yaxis_transform(), va="center",
fontsize=10, color=TXT, fontweight=fw)
ax.text(1.36, i, imp, transform=ax.get_yaxis_transform(), va="center",
fontsize=10, color=BLUE, fontweight=fw)
ax.text(1.03, -0.95, "Abstract\nsays", transform=ax.get_yaxis_transform(), va="bottom",
fontsize=10, color=TXT, fontweight="bold")
ax.text(1.36, -0.95, "Sign of\nreported HPD", transform=ax.get_yaxis_transform(), va="bottom",
fontsize=10, color=BLUE, fontweight="bold")
ax.text(0.0, -0.95, "no effect", ha="center", va="bottom", fontsize=10, color=TXT2)
ax.plot([], [], "-o", color=ORANGE, lw=2, ms=8, label="Reported mean and 95% HPD (Results)")
ax.legend(loc="upper left", bbox_to_anchor=(-0.45, -0.14), frameon=False, fontsize=10)
save(fig, "fig2")
if __name__ == "__main__":
check_quotes()
rows = recompute()
fig1()
fig2(rows)
Table 1 P values do not follow from its own mouse counts under the stated tests
Nat Commun (2017) · doi:10.1038/ncomms14382 · PMC5309857
Summary
Table 1 reports how many recipient mice were engrafted ("positive") after p190-B−/− LSK-SLAM cells were treated ex vivo: 9 of 16 untreated, 4 of 13 with rTGF-β1, and 13 of 20 with rTGF-β1 plus the p38MAPK inhibitor SB203580. It gives Fisher exact P values of 0.0006 and <0.0001 for the two pairwise comparisons, and a three-group χ² P<0.0001. With these counts, the stated tests give P = 0.264, P = 0.080 and χ² = 3.80 on 2 df (P = 0.149), so none of the three comparisons is significant at 0.05. All three reported values come out exactly if the rounded percentages (56%, 31%, 65%) are entered as counts out of 100 mice, as the footer's phrase "comparing per cent of negative and positive mice" suggests. Supplementary Table 3 shows the same pattern. Table 1 is the main-text in vivo test cited for the claim that rTGF-β1 "significantly reduced" long-term HSC activity through p38MAPK, so as printed that claim has no statistical support from these data.
What the paper reports
Table 1 ("Effect of TGFβ on p190-B−/− HSPC engraftment"). The columns are Chimera>1%, Chimera<1%, Total mice transplanted and P value:
p190-B-/- | 9 | 7 | 16 |
p190-B-/-+rTGFβ1 | 4 | 9 | 13 | 0.0006*
p190-B-/-+rTGFβ1+p38 inhi | 13 | 7 | 20 | < 0.0001†
Table 1
Table shows numbers of mice exhibiting more than 1% contribution to both myeloid and lymphoid lineages in PB (positive mice) and numbers of mice with less than 1% contribution in at least one lineage (negative mice) (2–3 independent experiments). P values were calculated by Fisher exact 2 × 2 contingency table by comparing per cent of negative and positive mice of the following groups: *p190-B-/- vs p190-B-/-+rTGF-β1, and †p190-B-/-+rTGF-β1 vs p190-B-/-+rTGF-β1+ SB203580. Difference between the three groups was analysed by χ2 test, P<0.0001.
Table 1, footer
How the Results use the table:
But, LT-HSC activity of pooled p190-B-/- LSK-SLAM cells cultured with rTGF-β1 was significantly reduced (30% of transplanted mice showing greater than 1% contribution to both myeloid and lymphoid lineages in PB versus 56% from non-treated cultures). This effect was prevented by addition of SB203580 (65% mice were engrafted (Table 1 and Supplementary Table 3).
Hence, rTGF-β1 can alter HSC activity via p38MAPK signalling pathway in vitro.
Results, section "p38MAPK pathway mediates TGF-β effect on HSPC activity"
Here, the authors show using both in vitro assays and in vivo transplantation that loss of the RhoGAP p190-B enhances self-renewal by inhibiting TGFβ/p38 signalling.
Abstract (editor's summary)
The statistics section names only the paired-daughter-cell and single-cell analyses ("In vitro paired daughter cell assay data were analysed by 2 × 2 Fisher exact test contingency table", Methods, Statistical analyses). It does not describe any other analysis of the transplantation counts.
Supplementary Table 3 (fetal-liver LSK-SLAM cells, from the Supplementary Information PDF) reports 8/4 (n = 12) untreated, 2/10 (n = 12) with rTGF-β1, P "<0.0001", and 4/6 (n = 10) with rTGF-β1 plus the p38 inhibitor, P "0.0005". Its footer reads: "P value was calculated by Fisher exact test ... P value by chi square test was < 0.0001 in both sets of transplant."
What the paper's own numbers show
Method. Each comparison named in the footer was set up as a 2 × 2 table of positive and negative mice and analysed with the two-sided Fisher exact test (scipy). As sensitivity checks I also ran the one-sided Fisher test, the χ² test with and without Yates correction, and Barnard's and Boschloo's unconditional exact tests. The three-group comparison is the Pearson χ² test on the 3 × 2 table. For each pairwise comparison I computed the difference in per cent positive with a Newcombe (hybrid Wilson score) 95% CI. To test the footer's wording, I then ran the same tests on the rounded percentages entered as if they were counts out of 100. Supplementary Table 3 was treated the same way. All numbers come from recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Per cent positive, p190-B−/− (9/16) | Table 1; Results | 56% | 56.2% |
| Per cent positive, +rTGF-β1 (4/13) | Table 1; Results | 30% | 30.8% |
| Per cent positive, +rTGF-β1+SB203580 (13/20) | Table 1; Results | 65% | 65.0% |
| Fisher, p190-B−/− vs +rTGF-β1 (9/7 vs 4/9) | Table 1, * | P = 0.0006 | P = 0.264 two-sided (one-sided 0.160; χ² 0.170, Yates 0.319; Barnard 0.184; Boschloo 0.192) |
| Difference in per cent positive, same comparison | derived | — | +25.5 points, 95% CI −9.9 to +52.9 |
| Fisher, +rTGF-β1 vs +rTGF-β1+SB203580 (4/9 vs 13/7) | Table 1, † | P < 0.0001 | P = 0.080 two-sided (one-sided 0.058; χ² 0.055, Yates 0.117; Barnard 0.063; Boschloo 0.069) |
| Difference in per cent positive, same comparison | derived | — | −34.2 points, 95% CI −59.0 to +0.3 |
| Three-group χ², 3 × 2 | Table 1 footer | P < 0.0001 | χ² = 3.80, df 2, P = 0.149 |
| Multiplier of the real counts needed to reach the reported P | derived | — | ×7 (first comparison), ×5 (second) |
| Fisher on % as counts, 56/44 vs 31/69 | diagnostic | 0.0006 | P = 5.8 × 10⁻⁴ (prints as 0.0006) |
| Fisher on % as counts, 31/69 vs 65/35 | diagnostic | < 0.0001 | P = 2.4 × 10⁻⁶ |
| 3 × 2 χ² on % as counts | diagnostic | < 0.0001 | χ² = 24.8, P = 4.1 × 10⁻⁶ |
| Fisher, Supp. Table 3: 8/4 vs 2/10 | Supp. Table 3 | P < 0.0001 | P = 0.036 (on % as counts, 67/33 vs 17/83: 6.2 × 10⁻¹³) |
| Fisher, Supp. Table 3: 2/10 vs 4/6 | Supp. Table 3 | P = 0.0005 | P = 0.348 (on % as counts, 17/83 vs 40/60: 5.06 × 10⁻⁴, prints as 0.0005) |
| 3 × 2 χ², Supp. Table 3 | Supp. Table 3 footer | P < 0.0001 | χ² = 6.20, df 2, P = 0.045 (on % as counts: 6.1 × 10⁻¹²) |
The cells, row totals (9 + 7 = 16, 4 + 9 = 13, 13 + 7 = 20), the percentages in the Results text and the footer all agree with each other. The discrepancy lies only in the P values. Each reported value sits exactly where the stated test lands when the per cent figures are treated as counts out of 100, which treats each group as about 5 to 8 times larger than it was (100 versus 13 to 20 mice). The match is specific. For the first comparison, 4/13 rounds to 31%, and 56/44 vs 31/69 gives 0.00058, which prints as the reported 0.0006. Using the 30% from the Results text instead gives 0.00033.
Figure
Figure 1. P values on a log scale for the three tests in Table 1 (top) and the three in Supplementary Table 3 (bottom). The thin vertical line marks p = 0.05. Orange diamonds are the values the paper prints; an arrow means the paper gives only an upper bound ("<0.0001"). Filled blue circles are the stated test (two-sided Fisher exact for pairs, Pearson χ² for three groups) applied to the mouse counts printed in the same table. Open blue squares are the same test applied to the rounded per cent positive and negative entered as if each group had 100 mice. In every row the reported value coincides with, or is a correct bound for, the open square, not the filled circle. On the actual counts, all three Table 1 tests are above 0.05.
Figure 2. Calculation flow for the first Table 1 comparison (p190-B−/− vs +rTGF-β1). Upper path: the two-sided Fisher exact test on the printed numbers of mice (9 positive / 7 negative vs 4 / 9) gives p = 0.264, which is 440 times the reported P = 0.0006 (orange box). Lower path: the same test on the rounded per cent positive and negative (56 / 44 vs 31 / 69), entered as if each group had 100 mice, gives p = 0.00058, which prints as the reported 0.0006. The line at the bottom gives the same pair of values for the other two Table 1 tests, both printed in the paper as P < 0.0001. Blue marks values recomputed from the paper's own numbers; orange marks the paper's reported value.
Why it matters
Table 1 is the paper's main-text in vivo functional test of the step from TGF-β through p38MAPK to long-term HSC activity. The Results cite it, together with Supplementary Table 3, for the statements that rTGF-β1 "significantly reduced" LT-HSC activity, that the effect "was prevented by addition of SB203580", and hence that "rTGF-β1 can alter HSC activity via p38MAPK signalling pathway". The editor's summary relies on "in vivo transplantation" for the TGFβ/p38 conclusion.
On the printed counts, neither pairwise comparison in Table 1 is significant, and the three-group test gives P = 0.149. The intervals show how little the data pin down. The rTGF-β1 reduction (+25.5 percentage points) is compatible with anything from a 10-point increase to a 53-point decrease, and the SB203580 rescue (−34.2 points) has an upper CI limit just above zero. The direction of the effects (56% → 31% → 65%) is as described. What the table does not support is the statement that the differences are significant, let alone at P ≤ 0.0006.
Supplementary Table 3 does not supply the missing support at the stated level. Its rTGF-β1 effect is nominally significant on the counts (Fisher P = 0.036, against the printed <0.0001), its p38-inhibitor rescue is not (P = 0.348, against the printed 0.0005), and its three-group χ² gives P = 0.045 (printed <0.0001). Pooling the two tables, which the paper does not describe, gives P = 0.012, 0.027 and a three-group P = 0.014. These values are shown for scale only and are not a reanalysis the paper reports.
The abstract itself does not quote these P values, and the paper's p190-B/TGF-β/p38 conclusions also rest on other experiments (paired-daughter-cell assays, p-p38MAPK and bioactive TGF-β measurements, and TGFBRI-inhibitor experiments). The same wording, "comparing per cent", appears in the Fisher-test legends of Figures 3, 7 and 9. I have not checked whether those tests have the same problem.
Explanations considered
- One-sided test. The footer does not say whether the tests were one- or two-sided. One-sided Fisher P values are 0.160 and 0.058, still far from 0.0006 and <0.0001. This does not account for the numbers.
- A different 2 × 2 test. Uncorrected χ², Yates-corrected χ², Barnard's and Boschloo's tests give P between 0.055 and 0.32 for the two Table 1 comparisons. This does not account for the numbers.
- Different or pooled data. The footer ties the P values to the Table 1 groups, and the Results percentages (56%, 30%, 65%) match the Table 1 counts. Supplementary Table 3 is a separate fetal-liver experiment with its own P values. Pooling it with Table 1 gives P = 0.012 and 0.027, nowhere near the reported values. Reaching the reported P values from these proportions would need each count multiplied by about 5 to 7. This does not account for the numbers.
- Misaligned cells, lost symbols or conversion errors. The PMC archive, the Europe PMC full text and the publisher's Table 1 page carry the same cells, totals, P values and footer. The row totals add up. The P values are plain decimals with no lost minus signs, superscripts or exponents. This does not account for the numbers.
- Correction already published. None was found (see Status).
- Test run on percentages rather than counts. The footer says the test compared "per cent of negative and positive mice". Entering the rounded percentages as counts out of 100 reproduces all three Table 1 values (5.8 × 10⁻⁴, 2.4 × 10⁻⁶, 4.1 × 10⁻⁶) and both numeric Supplementary Table 3 values (0.0005, and 6.2 × 10⁻¹³ under "<0.0001"). This very likely explains how the printed numbers arose. It does not make them valid: a Fisher or χ² test on percentages treats each group as 100 independent mice instead of the 10 to 20 actually transplanted, so it overstates the evidence. It explains the reported values but does not support them.
Status
As of 2026-10-23:
- Version of record. The publisher's Table 1 page (nature.com/articles/ncomms14382/tables/1, fetched 2026-10-23) shows the same counts, P values (0.0006*, < 0.0001†) and footer text as the PMC archive. The main article page carries the same Results sentence. The discrepancy is in the published text.
- Supplements. The Supplementary Information PDF (11 pages, from static-content.springer.com) was read. Supplementary Table 3 is a separate fetal-liver experiment, and it shows the same pattern as Table 1 (see above). Supplementary Data 1 and 2 are gene-expression lists and not relevant.
- Corrections and notices. Crossref has no
update-to,updated-byorrelationentries for the DOI. Neither PMC nor Europe PMC (PMID 28176763) lists a correction, retraction or expression of concern. The publisher's article page links to no correction. - PubPeer. A web search for the title and DOI found no PubPeer thread. The PubPeer API returned HTTP 403, so no direct query of its database was possible.
- Sources: PMC5309857 (Table 1, Results, Methods, editor's summary); https://www.nature.com/articles/ncomms14382 and its Table 1 page; Supplementary Information PDF (41467_2017_BFncomms14382_MOESM1632_ESM.pdf); api.crossref.org/works/10.1038/ncomms14382; Europe PMC REST search for DOI:10.1038/ncomms14382.
Suggested correction
Recompute the Table 1 P values on the numbers of mice (9/7, 4/9 and 13/7) rather than on percentages. With two-sided Fisher exact tests this gives P = 0.26 for p190-B−/− vs +rTGF-β1 and P = 0.080 for +rTGF-β1 vs +rTGF-β1+SB203580, and the three-group χ² gives χ² = 3.80, df 2, P = 0.15. Replace the footer's "comparing per cent of negative and positive mice" with "comparing numbers of negative and positive mice". Recompute Supplementary Table 3 the same way (P = 0.036, 0.35 and χ² P = 0.045). Revise the Results sentence "was significantly reduced" and the inference that the effect was "prevented" by SB203580 to match the recomputed values, for example by reporting the observed proportions with confidence intervals. The other Fisher tests whose legends use "per cent" (Figures 3, 7 and 9) should be checked to confirm they were run on counts.
Recomputation script (python recompute.py.txt)
# Recomputation for PMC5309857 (Nat Commun 2017, doi:10.1038/ncomms14382), Table 1
# and Supplementary Table 3. Run: python recompute.py.txt
# Prints every number used in writeup.md and writes fig1.svg / fig1.png next to this file.
import os
import numpy as np
from scipy.stats import fisher_exact, chi2_contingency, barnard_exact, boschloo_exact, norm
HERE = os.path.dirname(os.path.abspath(__file__))
# ---- Counts as printed: (positive mice "Chimera>1%", negative mice "Chimera<1%")
T1 = {"KO": (9, 7), "KO+TGF": (4, 9), "KO+TGF+p38i": (13, 7)} # Table 1
S3 = {"KO": (8, 4), "KO+TGF": (2, 10), "KO+TGF+p38i": (4, 6)} # Supplementary Table 3
PAIRS = [("KO", "KO+TGF"), ("KO+TGF", "KO+TGF+p38i")]
REPORTED = { # what the paper prints (value, is_upper_bound)
("T1", "KO", "KO+TGF"): (0.0006, False),
("T1", "KO+TGF", "KO+TGF+p38i"): (0.0001, True),
("T1", "chi2"): (0.0001, True),
("S3", "KO", "KO+TGF"): (0.0001, True),
("S3", "KO+TGF", "KO+TGF+p38i"): (0.0005, False),
("S3", "chi2"): (0.0001, True),
}
def wilson(x, n, z=1.959964):
p = x / n
c = (p + z*z/(2*n)) / (1 + z*z/n)
h = z*np.sqrt(p*(1-p)/n + z*z/(4*n*n)) / (1 + z*z/n)
return c - h, c + h
def newcombe(x1, n1, x2, n2):
"""Newcombe hybrid-score 95% CI for p1 - p2."""
p1, p2 = x1/n1, x2/n2
l1, u1 = wilson(x1, n1); l2, u2 = wilson(x2, n2)
d = p1 - p2
return d, d - np.sqrt((p1-l1)**2 + (u2-p2)**2), d + np.sqrt((u1-p1)**2 + (p2-l2)**2)
def as_pct_counts(tab):
"""Rounded per cent positive / negative, i.e. counts out of 100."""
out = {}
for k, (a, b) in tab.items():
p = round(100*a/(a+b))
out[k] = (p, 100-p)
return out
results = {}
for name, tab in [("T1", T1), ("S3", S3)]:
print("=" * 72)
print({"T1": "Table 1 (1T p190-B-/- LSK-SLAM)", "S3": "Supplementary Table 3 (fetal liver)"}[name])
for k, (a, b) in tab.items():
print(f" {k:12s} {a}/{a+b} positive = {100*a/(a+b):.1f}%")
for g1, g2 in PAIRS:
t = np.array([tab[g1], tab[g2]])
p2 = fisher_exact(t)[1]
p1 = min(fisher_exact(t, alternative="less")[1], fisher_exact(t, alternative="greater")[1])
c_u, p_u, _, _ = chi2_contingency(t, correction=False)
_, p_y, _, _ = chi2_contingency(t, correction=True)
p_b = barnard_exact(t).pvalue
p_bo = boschloo_exact(t).pvalue
a1, b1 = tab[g1]; a2, b2 = tab[g2]
d, lo, hi = newcombe(a1, a1+b1, a2, a2+b2)
pc = as_pct_counts(tab)
p_pct = fisher_exact(np.array([pc[g1], pc[g2]]))[1]
# smallest integer multiplier of the real counts that reaches the reported value
rep, _ = REPORTED[(name, g1, g2)]
mult = next(m for m in range(1, 50) if fisher_exact(t*m)[1] <= rep)
results[(name, g1, g2)] = dict(fisher=p2, pct=p_pct)
print(f"\n {g1} vs {g2}: table {t.tolist()}")
print(f" reported P : {'<' if REPORTED[(name,g1,g2)][1] else ''}{rep}")
print(f" Fisher two-sided : {p2:.4f}")
print(f" Fisher one-sided (smaller): {p1:.4f}")
print(f" chi2 uncorrected : chi2={c_u:.3f}, p={p_u:.4f}")
print(f" chi2 Yates : p={p_y:.4f}")
print(f" Barnard / Boschloo : {p_b:.4f} / {p_bo:.4f}")
print(f" difference in % positive : {100*d:+.1f} points, Newcombe 95% CI {100*lo:+.1f} to {100*hi:+.1f}")
print(f" percentages as counts/100 : {[pc[g1], pc[g2]]} -> Fisher two-sided p={p_pct:.2e}")
print(f" real counts must be multiplied by {mult} to reach p <= {rep}")
t3 = np.array(list(tab.values()))
c3, p3, df3, _ = chi2_contingency(t3)
pc = as_pct_counts(tab)
c3p, p3p, _, _ = chi2_contingency(np.array(list(pc.values())))
results[(name, "chi2")] = dict(fisher=p3, pct=p3p)
print(f"\n 3x2 chi-square on counts : chi2={c3:.2f}, df={df3}, p={p3:.3f}")
print(f" 3x2 chi-square on % as counts : {list(pc.values())} -> chi2={c3p:.1f}, p={p3p:.1e}")
# Rounding check: the Results text says 30% for 4/13 (30.8%); 30/70 instead of 31/69
print("\nRounding sensitivity, Table 1 comparison 1 with 56/44 vs 30/70:",
f"p={fisher_exact(np.array([[56,44],[30,70]]))[1]:.2e}")
# Pooled Table 1 + Supplementary Table 3 (an analysis the paper does not describe)
pool = {k: (T1[k][0]+S3[k][0], T1[k][1]+S3[k][1]) for k in T1}
print("\nPooled Table 1 + Supp. Table 3 (not an analysis the paper describes):", pool)
for g1, g2 in PAIRS:
print(f" {g1} vs {g2}: Fisher two-sided p={fisher_exact(np.array([pool[g1], pool[g2]]))[1]:.3f}")
c, p, d, _ = chi2_contingency(np.array(list(pool.values())))
print(f" 3x2 chi2={c:.2f}, df={d}, p={p:.3f}")
# ---------------------------------------------------------------- figure
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"svg.fonttype": "none"})
rows = [ # (key, label)
(("T1", "KO", "KO+TGF"), "Table 1: p190-B⁻/⁻ vs +rTGF-β1\n9/16 vs 4/13 positive mice"),
(("T1", "KO+TGF", "KO+TGF+p38i"), "Table 1: +rTGF-β1 vs +rTGF-β1+SB203580\n4/13 vs 13/20"),
(("T1", "chi2"), "Table 1: three groups, 3×2 χ²"),
(("S3", "KO", "KO+TGF"), "Supp. Table 3: p190-B⁻/⁻ vs +rTGF-β1\n8/12 vs 2/12"),
(("S3", "KO+TGF", "KO+TGF+p38i"), "Supp. Table 3: +rTGF-β1 vs +p38 inhibitor\n2/12 vs 4/10"),
(("S3", "chi2"), "Supp. Table 3: three groups, 3×2 χ²"),
]
fig, ax = plt.subplots(figsize=(7.2, 5.0), facecolor=BG)
ax.set_facecolor(BG)
ys = [5.6, 4.6, 3.6, 2.0, 1.0, 0.0]
for (key, lab), y in zip(rows, ys):
rep, ub = REPORTED[key]
r = results[key]
lo = min(rep, r["fisher"], r["pct"])
hi = max(rep, r["fisher"], r["pct"])
ax.plot([lo, hi], [y, y], color="#cfcdc8", lw=2, zorder=1)
ax.plot(r["pct"], y, marker="s", ms=12, mfc="none", mec=BLUE, mew=2, ls="none", zorder=3)
ax.plot(r["fisher"], y, marker="o", ms=9, color=BLUE, ls="none", zorder=4)
ax.plot(rep, y, marker="D", ms=8.5, color=ORANGE, ls="none", zorder=5)
if ub: # "<" bound: arrow pointing to smaller p
ax.annotate("", xy=(rep/12, y), xytext=(rep/1.6, y),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2), zorder=5)
ax.text(r["fisher"], y+0.24, f"{r['fisher']:.3f}", color=BLUE, fontsize=10, ha="center", va="bottom", bbox=dict(facecolor=BG, edgecolor="none", pad=0.6), zorder=6)
ax.text(rep, y+0.24, ("<" if ub else "") + f"{rep:g}", color=ORANGE, fontsize=10, ha="center", va="bottom")
ax.axvline(0.05, color=SEC, lw=0.8, zorder=0)
ax.text(0.04, -0.45, "p = 0.05", color=SEC, fontsize=10, ha="right", va="center", bbox=dict(facecolor=BG, edgecolor="none", pad=0.6))
ax.set_xscale("log")
ax.set_xlim(1e-14, 2)
ax.set_ylim(-0.6, 6.6)
ax.set_yticks(ys)
ax.set_yticklabels([lab for _, lab in rows], fontsize=10, color=TXT)
ax.axhline(2.8, color=GRID, lw=1)
ax.set_xlabel("P value (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
handles = [
Line2D([], [], marker="D", ms=8.5, color=ORANGE, ls="none", label="Reported in the paper (arrow = \"<\")"),
Line2D([], [], marker="o", ms=9, color=BLUE, ls="none", label="Recomputed: stated test on the mouse counts"),
Line2D([], [], marker="s", ms=12, mfc="none", mec=BLUE, mew=2, ls="none",
label="Recomputed: same test, rounded % entered as counts of 100"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.32, -0.13), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
fig.savefig(os.path.join(HERE, "fig1.svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, "fig1.png"), dpi=200, facecolor=BG)
print("\nwrote fig1.svg, fig1.png")
# ---------------------------------------------------------------- figure 2: calculation flow
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
r1 = results[("T1", "KO", "KO+TGF")]
r2 = results[("T1", "KO+TGF", "KO+TGF+p38i")]
r3 = results[("T1", "chi2")]
pc1 = as_pct_counts(T1)
fig, ax = plt.subplots(figsize=(7.2, 4.4), facecolor=BG)
ax.set_facecolor(BG); ax.set_xlim(0, 100); ax.set_ylim(0, 64); ax.axis("off")
def box(x, y, w, h, text, ec, bold=False, tc=TXT):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.3,rounding_size=1.2",
fc=BG, ec=ec, lw=2))
ax.text(x + w/2, y + h/2, text, ha="center", va="center", fontsize=10.5, color=tc,
fontweight="bold" if bold else "normal", linespacing=1.4)
def arrow(x0, y0, x1, y1, color, rad=0.0):
ax.add_patch(FancyArrowPatch((x0, y0), (x1, y1), arrowstyle="-|>", mutation_scale=14,
color=color, lw=2, connectionstyle=f"arc3,rad={rad}"))
box(0.5, 26, 13, 12, "Table 1\nmouse\ncounts", SEC)
ax.text(19, 57, "Stated test on the numbers of mice", color=BLUE, fontsize=10.5, fontweight="bold", va="center")
box(19, 41, 25, 11, f"[[{T1['KO'][0]}, {T1['KO'][1]}], [{T1['KO+TGF'][0]}, {T1['KO+TGF'][1]}]]\nn = 16 and 13", BLUE)
box(48, 41, 17, 11, "Fisher exact,\ntwo-sided", SEC)
box(69, 41, 21, 11, f"Recomputed\np = {r1['fisher']:.3f}", BLUE, bold=True, tc=BLUE)
ax.text(19, 24.5, "Same test, per cent as counts of 100", color=BLUE, fontsize=10.5,
fontweight="bold", va="center")
box(19, 8, 25, 11, f"[[{pc1['KO'][0]}, {pc1['KO'][1]}], [{pc1['KO+TGF'][0]}, {pc1['KO+TGF'][1]}]]\nn = 100 and 100", BLUE)
box(48, 8, 17, 11, "Fisher exact,\ntwo-sided", SEC)
box(69, 8, 21, 11, f"Recomputed\np = {r1['pct']:.5f}", BLUE, bold=True, tc=BLUE)
for y in (46.5, 13.5):
arrow(44.6, y, 47.5, y, SEC); arrow(65.6, y, 68.5, y, SEC)
arrow(14.2, 35, 18.4, 46.5, SEC, rad=-0.3)
arrow(14.2, 29, 18.4, 13.5, SEC, rad=0.3)
box(69, 27, 21, 9, "Reported\nP = 0.0006", ORANGE, bold=True, tc=ORANGE)
ax.plot([79.5, 79.5], [36.6, 40.4], color=SEC, lw=1, ls=(0, (2, 2)))
ax.plot([79.5, 79.5], [19.6, 26.4], color=SEC, lw=1, ls=(0, (2, 2)))
ax.text(81, 38.5, f"\u2260 {r1['fisher']/0.0006:.0f}\u00d7", ha="left", va="center", fontsize=10, color=SEC)
ax.text(81, 22.6, "= rounded", ha="left", va="center", fontsize=10, color=SEC)
ax.text(50, 1.0,
f"Other Table 1 tests, mouse counts vs per cent as counts: SB203580 comparison p = {r2['fisher']:.3f} vs {r2['pct']:.1e};"
f"\n3×2 χ² p = {r3['fisher']:.3f} vs {r3['pct']:.1e}. Both reported as P < 0.0001.",
ha="center", va="center", fontsize=10, color=SEC, linespacing=1.4)
fig.savefig(os.path.join(HERE, "fig2.svg"), facecolor=BG, bbox_inches="tight", pad_inches=0.1)
fig.savefig(os.path.join(HERE, "fig2.png"), dpi=200, facecolor=BG, bbox_inches="tight", pad_inches=0.1)
print(f"ratio reported/recomputed, first comparison: {r1['fisher']/0.0006:.0f}x")
print("wrote fig2.svg, fig2.png")
Abstract reports reduced PKN1 and PKN4, but the Results report increased PKN2 and no PKN4
Nat Commun (2018) · doi:10.1038/s41467-018-04411-5 · PMC5998138
Summary
For the isogenic cell-line comparison, the abstract states that PCQ "was used to identify reduced levels of serine/threonine protein kinases PKN1 and PKN4 in CFBE41o- cells compared to HBE41o- cells". The Results and Fig. 7 instead describe a PKN1/PKN2/PKN3 cluster in which PKN2 was "three-fold more abundant in CFBE41o- cells" and confirmed by western blot, the evidence for PKN1 is "conflicting" even about its presence, and PKN4 is never mentioned outside the abstract. The paper's own data files agree with the Results: PKN2-specific peptides have a CFBE41o-/HBE41o- ratio of 3.000, the only peptide on the PKN1 node is shared with PKN2 and PKN3 (ratio 0.500), and no file contains PKN4. Because the abstract is the indexed summary and this sentence is one of its two biological examples, it gets both the proteins and the direction of the one validated kinase change wrong.
What the paper reports
Abstract:
PCQ was used to identify reduced levels of serine/threonine protein kinases PKN1 and PKN4 in CFBE41o- cells compared to HBE41o- cells and to elucidate that shorter proteoforms of full-length caspase-4 and ephrin B receptor are differentially expressed.
Results, "Analysis of individual protein clusters", paragraph 2. This paragraph is the only passage in the Results about the PKN proteins:
Finally, a serine/threonine-protein kinase (PKN) cluster comprising PKN1 (Q16512), PKN2 (Q16513), and PKN3 (Q6P5Z2) was identified. PKN2 (PRK2) shares 59% sequence identity with PKN1 and was three-fold more abundant in CFBE41o- cells (Fig. 7b). Western blot confirmed that PKN2 was more abundant in CFBE41o- cells when normalized to the expression levels of Na+/K+-ATPase (Fig. 7c and 7d, Supplementary Fig. 12).
A difference in PKN2 protein quantity may contribute to the altered cell morphology between HBE41o- and CFBE41o– cells in cell culture because bronchial epithelial airway cells require PKN2 to establish apical junctions32.
PKN1 is part of two incomplete protein pairs with either PKN2 or PKN3, which provide conflicting evidence for a potential presence of PKN1. While the incomplete protein pair with PKN2 shows discrepant ratio values for the peptide nodes (∆ ≥ 2-fold), the incomplete protein pair with PKN3 shows similar ratio values (∆ < 2-fold).
Fig. 7 legend, panels b and d:
b The protein cluster of serine–threonine kinase-related protein kinases PKN1, PKN2, and PKN3 is displayed.
d Western blot analysis was used to verify the difference in PKN2 expression levels between HBE41o– and CFBE41o– cells.
Data in the western blot represent independent biological replicates, CFBE41o–: n = 2, HBE41o–: n = 2
Results, "Comparison of two isogenic cell lines with PCQ", on the labelling, which fixes the direction of the ratios:
CFBE41o– and HBE41o– cells were labeled with light and heavy isobaric isotopologues, respectively, and analyzed in a single experiment (n = 1)
Methods, "Cell culture and western blot". The only antibody against a PKN protein is anti-PKN2:
PKN2 and NaK-ATPase were detected by incubation with monoclonal anti-PKN2 antibody
Discussion. The summary of the cell-line comparison mentions no kinase:
When comparing two isogenic cell lines, we found that caspase-4 and ephrin type B receptor protein fragments were differentially regulated relative to their longer proteoforms, respectively.
What the paper's own numbers show
Method. I counted every mention of PKN1 to PKN4 in the PMC-archived text, separately for the abstract and for the rest of the article (Results, Discussion, Methods, figure legends). I searched every body sentence that names a PKN protein for wording of reduction ("reduc", "decreas", "lower", "less abundant", "down"). I then opened the two supplementary data files that hold the cell-line quantification. Supplementary Data 7 (MOESM9, the 180 classified complete protein pairs) gives log2 ratios for unique and shared peptide nodes. Supplementary Data 6 (MOESM8, "Network 5" in the Results) is the XGMML peptide-to-protein network for CFBE41o- vs HBE41o-, with ion counts for each peptide node. From it I extracted the connected component that holds the three PKN accessions. I converted log2 ratios to ratios. I checked the ratio direction against the normalized light and heavy ion counts, since the Results say light = CFBE41o- and heavy = HBE41o-. I also queried UniProt for a human gene named PKN4. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own text and data |
|---|---|---|---|
| Kinases named as changed in the cell-line comparison | Abstract | PKN1 and PKN4, "reduced levels" in CFBE41o- | Results name PKN2 as changed (increased); no reduction of any PKN protein is stated |
| Mentions of PKN1 / PKN2 / PKN3 / PKN4 | Abstract vs rest of article | 1 / 0 / 0 / 1 in the abstract | 5 / 13 / 4 / 0 in the rest of the article |
| Body sentences naming a PKN protein that use reduction wording | Results, Methods, legends | — | 0 of 11 |
| Members of the PKN cluster | Results; Fig. 7b legend | PKN1 (Q16512), PKN2 (Q16513), PKN3 (Q6P5Z2) | same three in Supplementary Data 6; no PKN4 node in the network file |
| PKN2-specific peptides (3 peptides), CFBE41o-/HBE41o- | Results ("three-fold more abundant in CFBE41o- cells"); Supp. Data 7 | log2 1.58496 | ratio 3.000 (= 2^1.585; normalized light/heavy ions 4.5/1.5 in Supp. Data 6), an increase |
| PKN2 western blot | Results; Fig. 7d (n = 2 vs n = 2); Supp. Fig. 12 | "more abundant in CFBE41o- cells" | an increase, the same direction as the mass-spectrometry ratio |
| PKN3-specific peptides (3 peptides) | Supp. Data 7 only | log2 −0.678 | ratio 0.625 (1.60-fold lower; light/heavy 2.5/4.0); not discussed in the text |
| Shared peptide IADFGLCK (PKN1, PKN2, PKN3) | Supp. Data 6, 7 | log2 −1.000 | ratio 0.500 (light/heavy 0.75/1.5) |
| Peptides mapped to the PKN1 node | Supp. Data 6 | — | only the shared IADFGLCK. The node groups Q16512 with two fragment entries (H0Y429, K7EL10), and its gene-name field reads "PKN1,PKN2" |
| PKN1–PKN2 incomplete pair | Results ("∆ ≥ 2-fold") | — | unique 3.000 vs shared 0.500, a 6.00-fold discrepancy |
| PKN1–PKN3 incomplete pair | Results ("∆ < 2-fold") | — | unique 0.625 vs shared 0.500, a 1.25-fold discrepancy |
| A PKN1-specific or PKN4 ratio | article and all supplementary files | — | none exists |
| Human gene named PKN4 | UniProt (query 2026-10-23) | — | none; human PKN genes are PKN1, PKN2 (alias PRK2, PRKCL2) and PKN3 |
The body's account is internally consistent, and it reproduces from the data files: both the "∆ ≥ 2-fold" and the "∆ < 2-fold" statements match the network ratios. The abstract sentence is the outlier. It names a protein (PKN4) with no data behind it in the paper. It describes as an identified reduction a PKN1 inference that the body calls conflicting. It leaves out the one PKN change that was quantified with protein-specific peptides and checked by western blot, which is an increase.
Figure
Figure 1. Short verbatim excerpts from the abstract (left, orange) and from the Results and Fig. 7 legend (right, blue), with the supplementary-data values printed by recompute.py.txt. The abstract names PKN1 and PKN4 and says "reduced". The body names a PKN1/PKN2/PKN3 cluster and reports PKN2 three-fold more abundant in CFBE41o- cells (ratio 3.0 in Supplementary Data 6 and 7), confirmed by western blot. It calls the evidence for PKN1's presence conflicting, and its only peptide is shared with PKN2 and PKN3 (ratio 0.5). PKN4 is mentioned nowhere outside the abstract.
Figure 2. CFBE41o-/HBE41o- abundance ratios on one log2 axis; the vertical line marks no change (ratio 1). The orange hatched bands mark the abstract's claim for PKN1 and PKN4. The abstract gives a direction ("reduced") and no value, so the bands span the whole "reduced" side and carry no magnitude. Filled blue circles are ratios from protein-specific peptides in Supplementary Data 6 and 7: PKN2 at 3.0, which is the Results' "three-fold more abundant", and PKN3 at 0.625, which the text does not discuss. The hollow blue circle is the PKN1 node's only peptide, which is shared with PKN2 and PKN3 (0.5), so it is not specific to PKN1. PKN4 has no data point because the paper and its supplements contain none.
Why it matters
- The abstract misstates one of its two biological examples. The cell-line comparison is the paper's demonstration on a human proteome, and the abstract gives two results from it: the kinases and the caspase-4 and ephrin B receptor proteoforms. For the kinases, a reader of the abstract would conclude that PKN1 and PKN4 are lower in ∆F508-CFTR CFBE41o- cells. The body supports neither claim as stated. The one kinase result it quantified with protein-specific peptides and checked by western blot goes the other way: PKN2 is higher in CFBE41o- cells. The Results then offer an interpretation of that increase (apical junctions and cell morphology). The abstract's reduction cannot carry that interpretation.
- A protein name with no counterpart. PKN4 is not a human gene symbol in UniProt and has no entry in the article or its supplementary files. A reader who searches for PKN4 data, or indexes the abstract, finds nothing to match it.
- Certainty is overstated for PKN1. The abstract says PCQ "was used to identify reduced levels" of PKN1. The body says the two incomplete pairs "provide conflicting evidence for a potential presence of PKN1". The only peptide on the PKN1 node is shared with PKN2 and PKN3, and the PKN1–PKN3 pair's ratios (0.625 vs 0.500) are consistent with PKN3 alone. The mass-spectrometry comparison is a single labelled experiment (n = 1), and there is no PKN1 blot.
- What is not affected. The PCQ method, the two-species proof of principle (58% vs 21% completion of incomplete protein pairs), the caspase-4 and ephrin B receptor proteoform results, the PRDX analysis, and the body's PKN2 result do not depend on this sentence. The Discussion and the conclusions do not repeat it. The error is limited to the abstract, but it misstates which proteins changed and in which direction.
Explanations considered
- The ratios are defined the other way round (HBE41o-/CFBE41o-). This does not account for the discrepancy. The abstract fixes the comparison itself ("in CFBE41o- cells compared to HBE41o- cells"), and the Results use the same direction ("more abundant in CFBE41o- cells"). In the network file, light/heavy ion counts reproduce each stored ratio (4.5/1.5 = 3.000 for PKN2), and the Results state that CFBE41o- was the light label. So the ratios are CFBE41o-/HBE41o-, and PKN2 is increased in CFBE41o-. Even if the direction were flipped, the protein would be PKN2, not PKN1 and PKN4.
- "PKN4" is an alias of a protein in the body. This does not account for it. The aliases the paper and UniProt give for PKN2 are PRK2 and PRKCL2. UniProt lists no human gene named PKN4, and the cluster's three accessions are PKN1, PKN2 and PKN3.
- "PKN4" stands for PKN3, which is lower in the data. This would partly fit the numbers but is not supported by the text. PKN3's specific peptides have a ratio of 0.625 in Supplementary Data 7, a 1.6-fold decrease, which is below the paper's own 2-fold threshold. The Results describe PKN3 only as a member of the cluster, never as reduced. Even on this reading, the abstract would still leave out the validated PKN2 increase and overstate PKN1. The published material cannot show whether the abstract meant PKN3.
- The PKN1 reduction is a PCQ inference from the shared peptide. This is consistent with the data and could explain the PKN1 half of the sentence. The shared peptide IADFGLCK is at 0.500 while PKN2's specific peptides are at 3.000. Under the paper's logic, this 6-fold discrepancy suggests that another protein, possibly PKN1 (or the PKN2 fragment H0Y429 grouped in the same node), is present at lower levels. The body itself declines that conclusion, though, because the PKN3 pair "shows similar ratio values". The abstract therefore states as a finding what the body presents as unresolved. This does not account for PKN4, or for the omission of PKN2.
- The result is in the supplementary material. It is not. All supplementary files were opened, some in an earlier pass of this pipeline (see Status). Supplementary Information (MOESM2) holds the full western blot exposures (Supp. Fig. 12, PKN2 only) and does not otherwise discuss PKN. The PKN entries in Supplementary Data 6, 7 and 8 match the Results as tabulated above. The other network files either contain no PKN accession or belong to a different experiment, not the CFBE41o-/HBE41o- comparison. No file reports a PKN1-specific ratio, and no file mentions PKN4.
- A conversion or typesetting artifact. This is ruled out. The publisher's HTML carries the same abstract sentence, in the page body and in its metadata, as the PMC and Europe PMC versions. The discrepancy is in protein names and in the direction of change, not in symbols or digits.
Status
- Version of record. The publisher's page (nature.com/articles/s41467-018-04411-5, published 13 June 2018) has the abstract sentence word for word: PKN4 appears in the abstract text and in the page's metadata and nowhere else. The Results paragraph and the Fig. 7 legend match the PMC archive. Europe PMC's full-text XML carries the same text (prior check in this pipeline). Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py --grep. - Supplements. I downloaded Supplementary Data 7 (MOESM9, XLSX) and the network files (MOESM4–8, MOESM10–14, zipped XGMML) from the publisher's server and searched them for Q16512, Q16513, Q6P5Z2 and PKN4. The values used above come from Supplementary Data 7 and Supplementary Data 6. Supplementary Data 8 (Network 6) contains the same PKN component with identical ratios. An earlier pass in this pipeline opened the remaining files (MOESM1 PDF, MOESM2 Supplementary Information DOCX, MOESM3 XLSX) and found no PKN content apart from Supp. Fig. 12.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI. PMC and Europe PMC list no correction or comment links (PMID 29899466). The publisher's page shows no correction notice, and a web search found none. I could not check PubPeer: its API and its search page both refused automated access (Cloudflare challenge; HTTP 403).
- Sources. PMC article PMC5998138; publisher page https://www.nature.com/articles/s41467-018-04411-5; supplementary files 41467_2018_4411_MOESM8_ESM.zip (Supplementary Data 6) and 41467_2018_4411_MOESM9_ESM.xlsx (Supplementary Data 7); UniProt REST (human PKN gene query); Crossref; Europe PMC.
Suggested correction
- Revise the abstract sentence so it matches the Results. For example: "PCQ was used to identify increased levels of the serine/threonine protein kinase PKN2 in CFBE41o- cells compared to HBE41o- cells, confirmed by western blot, and to elucidate that shorter proteoforms of full-length caspase-4 and ephrin B receptor are differentially expressed."
- If a statement about PKN1 is kept, word it as the body does: the evidence for a PKN1 proteoform is inconclusive and rests on a peptide it shares with PKN2 and PKN3.
- Remove "PKN4". If another protein was meant, name it and give the supporting data.
Recomputation script (python recompute.py.txt)
# Recomputation for PMC5998138 (Nat Commun 2018, doi:10.1038/s41467-018-04411-5)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates the figures
# (SVG + PNG at 200 dpi) in the folder that holds this script:
# fig1.*, fig2.* the figures used in writeup.md
# candidates/fig_A..fig_E.* five candidate designs for review
# Inputs: the article text as archived in PubMed Central (read through the repo's
# src/show.py when it is available) and the one PKN row of Supplementary Data 7
# (41467_2018_4411_MOESM9_ESM.xlsx), transcribed below and re-read from the
# publisher's server if it can be reached.
import io
import os
import re
import shutil
import subprocess
import textwrap
import urllib.request
import xml.etree.ElementTree as ET
import zipfile
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Rectangle
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
REPO = "/home/ubuntu/papers"
PY = "/home/ubuntu/papers-data/venv/bin/python"
SD7_URL = ("https://media.springernature.com/original/springer-static/esm/"
"art%3A10.1038%2Fs41467-018-04411-5/MediaObjects/41467_2018_4411_MOESM9_ESM.xlsx")
# ---------------------------------------------------------------- style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
ORANGE_T, BLUE_T = "#fbe3d9", "#dde9f8" # light tints for panel fills
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
# ---------------------------------------------------------------- 1. text
print("=== 1. Article text (PMC archive)")
txt = None
try:
txt = subprocess.run([PY, "src/show.py", "PMC5998138", "--full"], cwd=REPO,
capture_output=True, text=True, timeout=120).stdout
except Exception as e: # repo not present
print(" show.py not available:", e)
if txt:
lines = txt.splitlines()
abstract = next(l for l in lines if l.startswith("ABSTRACT:"))
body = "\n".join(l for l in lines if not l.startswith("ABSTRACT:"))
claim = re.search(r"PCQ was used to identify reduced[^.]*\.", abstract).group(0)
print(" Abstract sentence:", claim)
counts = {}
for k in ["PKN1", "PKN2", "PKN3", "PKN4"]:
counts[k] = (len(re.findall(k + r"\b", abstract)), len(re.findall(k + r"\b", body)))
print(f" {k}: abstract {counts[k][0]}, rest of article (Results, Methods, legends) {counts[k][1]}")
sents = list(dict.fromkeys(s.strip() for s in re.findall(r"[^.]*PKN[^.]*\.", body)))
red = [s for s in sents if re.search(r"reduc|decreas|lower|less abundant|down", s)]
print(f" body sentences naming a PKN protein: {len(sents)}; of these using reduced/decreased/"
f"lower/less abundant/down: {len(red)}")
m = re.search(r"PKN2 \(PRK2\)[^.]*\.", body).group(0)
print(" PKN2 sentence:", m)
print(" cluster accessions:", re.findall(r"PKN\d \(([A-Z0-9]{6})\)", body))
else:
counts = {"PKN1": (1, 5), "PKN2": (0, 13), "PKN3": (0, 4), "PKN4": (1, 0)}
print(" using counts recorded on 2026-10-23:", counts)
# ---------------------------------------------------------------- 2. SD7
# Supplementary Data 7, sheet "180 classified protein pairs": the only row naming a
# PKN accession (Q16512 PKN1, Q16513 PKN2, Q6P5Z2 PKN3). Columns: Unique Pep1 (U1),
# Ratio (U1), Protein 1, Shared Pep (S), Ratio (S), Protein 2, Unique Pep2 (U2),
# Ratio (U2), ClassificationCase. Ratios are log2.
SD7 = {"U1": ("LLGCEQLLTAVPGR_QGAENMTHTCASGTPK_SPAAALASSPSEGWLR", -0.678071905112637, "Q6P5Z2"),
"S": ("IADFGLCK", -1.0, "shared"),
"U2": ("FLSTEAISIMR_LDFDLEPEPPPAPPR_LDNLLLDTEGFVK", 1.58496250072115, "Q16513"),
"case": "CASE3"}
def read_xlsx_rows(blob):
"""Minimal stdlib reader for the first sheet of an .xlsx file."""
ns = {"m": "http://schemas.openxmlformats.org/spreadsheetml/2006/main"}
z = zipfile.ZipFile(io.BytesIO(blob))
shared = []
if "xl/sharedStrings.xml" in z.namelist():
for si in ET.fromstring(z.read("xl/sharedStrings.xml")).findall("m:si", ns):
shared.append("".join(t.text or "" for t in si.iter("{%s}t" % ns["m"])))
sheet = sorted(n for n in z.namelist() if n.startswith("xl/worksheets/sheet"))[0]
rows = []
for r in ET.fromstring(z.read(sheet)).iter("{%s}row" % ns["m"]):
row = []
for c in r.findall("m:c", ns):
v = c.find("m:v", ns)
val = "" if v is None else v.text
if c.get("t") == "s":
val = shared[int(val)]
elif c.get("t") == "inlineStr":
val = "".join(t.text or "" for t in c.iter("{%s}t" % ns["m"]))
row.append(val)
rows.append(row)
return rows
print("\n=== 2. Supplementary Data 7 (complete protein pairs)")
try:
req = urllib.request.Request(SD7_URL, headers={"User-Agent": "Mozilla/5.0"})
rows = read_xlsx_rows(urllib.request.urlopen(req, timeout=60).read())
hits = [r for r in rows if any(a in " ".join(r) for a in ("Q16512", "Q16513", "Q6P5Z2", "PKN"))]
print(f" downloaded; {len(rows) - 1} protein-pair rows; rows naming a PKN accession: {len(hits)}")
print(" PKN1 (Q16512) present in any row:", any("Q16512" in " ".join(r) for r in rows))
for h in hits:
print(" ", h)
h = hits[0]
assert abs(float(h[1]) - SD7["U1"][1]) < 1e-9 and abs(float(h[7]) - SD7["U2"][1]) < 1e-9
print(" matches the transcribed values")
except Exception as e:
print(" could not re-read the file (", type(e).__name__, "); using transcribed values")
u1, s, u2 = SD7["U1"][1], SD7["S"][1], SD7["U2"][1]
r_pkn3, r_sh, r_pkn2 = 2 ** u1, 2 ** s, 2 ** u2
print(f" PKN2 (Q16513) unique peptides: log2 {u2:.3f} -> ratio {r_pkn2:.3f} "
f"(log2(3) = {__import__('math').log2(3):.5f})")
print(f" PKN3 (Q6P5Z2) unique peptides: log2 {u1:.3f} -> ratio {r_pkn3:.3f} (= {1 / r_pkn3:.2f}-fold lower)")
print(f" shared peptide IADFGLCK: log2 {s:.3f} -> ratio {r_sh:.3f}")
print(f" PKN2-unique vs shared: {2 ** (u2 - s):.2f}-fold apart; PKN3-unique vs shared: {2 ** (u1 - s):.2f}-fold apart")
print(" ratio direction: the PKN2 value equals the Results' 'three-fold more abundant in "
"CFBE41o- cells', so the ratios are CFBE41o-/HBE41o-")
# ---------------------------------------------------------------- 3. network file
# Supplementary Data 6 (41467_2018_4411_MOESM8_ESM.zip, XGMML, "Network 5" in the
# Results; label CFBEvsHBE_protpepCollapsed_3ionsPerNode_Rc_IDs). The connected
# component holding the PKN proteins, transcribed (log2 finalRatio; normalized ion
# counts, light = CFBE41o-, heavy = HBE41o- per the Results):
NET = {
"proteins": {"Q16513 (PKN2)": ["U_PKN2", "IADFGLCK"],
"Q6P5Z2 (PKN3)": ["IADFGLCK", "U_PKN3"],
"H0Y429 K7EL10 Q16512 (PKN1 node; genes PKN1,PKN2)": ["IADFGLCK"]},
"peptides": {"U_PKN2": (1.5849625007211563, 4.5, 1.5, 0), # log2, normLight, normHeavy, significant
"IADFGLCK": (-1.0, 0.75, 1.5, 0),
"U_PKN3": (-0.6780719051126377, 2.5, 4.0, 0)},
}
SD6_URL = SD7_URL.replace("MOESM9_ESM.xlsx", "MOESM8_ESM.zip")
print("\n=== 3. Supplementary Data 6 (peptide-to-protein network 5, CFBE41o- vs HBE41o-)")
try:
req = urllib.request.Request(SD6_URL, headers={"User-Agent": "Mozilla/5.0"})
z = zipfile.ZipFile(io.BytesIO(urllib.request.urlopen(req, timeout=120).read()))
root = ET.fromstring(z.read(z.namelist()[0]))
tag = lambda e: e.tag.split("}")[-1]
nodes = {n.get("id"): n for n in root if tag(n) == "node"}
adj = {}
for e in root:
if tag(e) == "edge":
adj.setdefault(e.get("source"), set()).add(e.get("target"))
adj.setdefault(e.get("target"), set()).add(e.get("source"))
seed = [k for k in nodes if any(a in k for a in ("Q16512", "Q16513", "Q6P5Z2"))]
comp, fr = set(seed), list(seed)
while fr:
x = fr.pop()
for y in adj.get(x, ()):
if y not in comp:
comp.add(y); fr.append(y)
print(f" downloaded; PKN component has {len(comp)} nodes; any node naming PKN4:",
any("PKN4" in ET.tostring(nodes[k], encoding="unicode") for k in nodes))
for k in sorted(comp):
a = {x.get("name"): x.get("value") for x in nodes[k].iter() if tag(x) == "att"}
if a.get("isProtein") == "1":
print(f" protein {k} | genes {a.get('GeneName')} | peptide nodes {sorted(adj[k])}")
else:
print(f" peptide {k[:44]} | log2 {float(a['finalRatio']):.3f} | normLight {a['normLightIons']}"
f" | normHeavy {a['normHeavyIons']} | significant {a['significant']}")
except Exception as e:
print(" could not re-read the file (", type(e).__name__, "); using transcribed values")
for k, (l2, li, hv, sig) in NET["peptides"].items():
print(f" {k:9s} normLight/normHeavy = {li}/{hv} = {li / hv:.3f}; 2^log2 = {2 ** l2:.3f}; significant = {sig}")
print(" -> light/heavy reproduces each ratio, and light = CFBE41o-, so ratios are CFBE41o-/HBE41o-")
print(" PKN1 node: its only peptide is the shared IADFGLCK (ratio 0.500), which also maps to PKN2 and PKN3;")
print(f" pair with PKN2: unique {r_pkn2:.3f} vs shared {r_sh:.3f} -> {2 ** (u2 - s):.2f}-fold (>= 2, 'discrepant')")
print(f" pair with PKN3: unique {r_pkn3:.3f} vs shared {r_sh:.3f} -> {2 ** (u1 - s):.2f}-fold (< 2, 'similar')")
print(" No PKN1-specific or PKN4 ratio exists in the article or its supplements.")
# ---------------------------------------------------------------- helpers
def save(fig, path_noext):
os.makedirs(os.path.dirname(path_noext), exist_ok=True)
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
print(" wrote", os.path.relpath(path_noext, HERE) + ".svg/.png")
ABS_Q = ("PCQ was used to identify reduced levels of serine/threonine protein kinases "
"PKN1 and PKN4 in CFBE41o- cells compared to HBE41o- cells")
RES_Q = [("Results", "a serine/threonine-protein kinase (PKN) cluster comprising PKN1 (Q16512), "
"PKN2 (Q16513), and PKN3 (Q6P5Z2) was identified"),
("Results", "PKN2 (PRK2) ... was three-fold more abundant in CFBE41o- cells"),
("Results", "Western blot confirmed that PKN2 was more abundant in CFBE41o- cells"),
("Results", "conflicting evidence for a potential presence of PKN1")]
# ---------------------------------------------------------------- A: text panels
def quote_block(ax, x, y, q, wrap, step=0.19):
for ln in textwrap.wrap("\u201c" + q + "\u201d", wrap):
ax.text(x, y, ln, fontsize=10, color=INK, va="top")
y -= step
return y
def fig_A(path):
H = 5.0
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
# left: abstract
lx, lw_, ltop, lh = 0.12, 2.85, H - 0.12, 2.25
ax.add_patch(FancyBboxPatch((lx, ltop - lh), lw_, lh, boxstyle="round,pad=0,rounding_size=0.08",
fc=ORANGE_T, ec=ORANGE, lw=1.5))
ax.text(lx + 0.14, ltop - 0.16, "Abstract", fontsize=11, weight="bold", va="top")
ax.text(lx + 0.14, ltop - 0.40, "the claim as reported", fontsize=10, color=INK2, va="top")
quote_block(ax, lx + 0.14, ltop - 0.72, ABS_Q, 34)
ax.text(lx + 0.14, ltop - lh + 0.14, "Named: PKN1, PKN4\nDirection: reduced", fontsize=10,
weight="bold", color=INK, va="bottom", linespacing=1.4)
# right: body
rx, rw, rtop = 3.95, 3.13, H - 0.12
rh = H - 0.24
ax.add_patch(FancyBboxPatch((rx, rtop - rh), rw, rh, boxstyle="round,pad=0,rounding_size=0.08",
fc=BLUE_T, ec=BLUE, lw=1.5))
ax.text(rx + 0.14, rtop - 0.16, "Rest of the paper", fontsize=11, weight="bold", va="top")
ax.text(rx + 0.14, rtop - 0.40, "Results, Fig. 7, Supp. Data 6 and 7", fontsize=10, color=INK2,
va="top")
yy = rtop - 0.72
for _, q in RES_Q:
yy = quote_block(ax, rx + 0.14, yy, q, 40) - 0.10
ax.plot([rx + 0.14, rx + rw - 0.14], [yy + 0.02, yy + 0.02], color=BLUE, lw=0.8)
yy -= 0.10
ax.text(rx + 0.14, yy, f"Supp. Data 6, 7: PKN2 ratio {r_pkn2:.1f};\n"
f"PKN1's only peptide is shared with PKN2\nand PKN3 (ratio {r_sh:.1f})", fontsize=10,
color=INK, va="top", linespacing=1.3)
yy -= 0.72
ax.text(rx + 0.14, yy, f"PKN4: {counts['PKN4'][1]} mentions outside the abstract", fontsize=10,
color=INK, va="top")
ax.text(rx + 0.14, rtop - rh + 0.14, "Quantified and blotted: PKN2\nDirection: increased", fontsize=10,
weight="bold", color=INK, va="bottom", linespacing=1.4)
# connector
yc = ltop - 0.95
ax.add_patch(FancyArrowPatch((lx + lw_ + 0.04, yc), (rx - 0.04, yc), arrowstyle="<->",
mutation_scale=14, lw=2, color=INK2))
ax.text((lx + lw_ + rx) / 2, yc + 0.14, "conflict", ha="center", fontsize=10, weight="bold")
# mismatch summary under the abstract panel
sy = ltop - lh - 0.35
ax.text(lx + 0.02, sy, "The two statements differ in", fontsize=10, weight="bold", va="top")
ax.text(lx + 0.02, sy - 0.30, "1. direction: \u201creduced\u201d vs \u201cthree-fold\n"
" more abundant\u201d in CFBE41o- cells", fontsize=10, va="top")
ax.text(lx + 0.02, sy - 0.86, "2. proteins: PKN1 and PKN4 vs PKN2;\n"
" PKN4 appears nowhere else", fontsize=10, va="top")
save(fig, path)
# ---------------------------------------------------------------- B: evidence matrix
def fig_B(path):
cols = ["Abstract", "Results text", "Supp. Data 6, 7\n(CFBE/HBE ratio)", "Western blot\n(Fig. 7d)"]
rows = ["PKN1", "PKN2", "PKN3", "PKN4"]
cells = {
"PKN1": ["reduced", "presence\nuncertain", f"shared peptide only\n{r_sh:.1f}\u00d7 (also PKN2, 3)", "not tested"],
"PKN2": ["not named", "three-fold more\nabundant in CFBE", f"{r_pkn2:.1f}\u00d7\n(log2 {u2:.3f})", "more abundant\nin CFBE (n = 2 vs 2)"],
"PKN3": ["not named", "cluster member;\nno level stated", f"{r_pkn3:.3f}\u00d7\n(log2 {u1:.3f})", "not tested"],
"PKN4": ["reduced", "0 mentions", "not listed", "not tested"],
}
fig = plt.figure(figsize=(W, 3.7))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, 3.7); ax.axis("off")
x0, cw, rh, ytop = 0.85, 1.55, 0.62, 2.95
for j, c in enumerate(cols):
ax.text(x0 + j * cw + cw / 2, ytop + 0.30, c, ha="center", va="center", weight="bold", fontsize=10)
for i, r in enumerate(rows):
y = ytop - (i + 1) * rh
ax.text(x0 - 0.12, y + rh / 2, r, ha="right", va="center", weight="bold", fontsize=11)
for j, v in enumerate(cells[r]):
empty = v in ("not named", "not listed", "not tested", "0 mentions")
if j == 0:
fc, ec = (BG, GRID) if empty else (ORANGE_T, ORANGE)
else:
fc, ec = (BG, GRID) if empty else (BLUE_T, BLUE)
ax.add_patch(Rectangle((x0 + j * cw + 0.03, y + 0.03), cw - 0.06, rh - 0.06,
fc=fc, ec=ec, lw=1.5))
ax.text(x0 + j * cw + cw / 2, y + rh / 2, v, ha="center", va="center", fontsize=10,
color=INK2 if empty else INK, weight="bold" if (j == 0 and not empty) else "normal")
# legend
ly = 0.18
ax.add_patch(Rectangle((0.2, ly), 0.25, 0.18, fc=ORANGE_T, ec=ORANGE, lw=1.5))
ax.text(0.52, ly + 0.09, "claim as reported in the abstract", va="center", fontsize=10)
ax.add_patch(Rectangle((3.35, ly), 0.25, 0.18, fc=BLUE_T, ec=BLUE, lw=1.5))
ax.text(3.67, ly + 0.09, "what the body and supplements show", va="center", fontsize=10)
save(fig, path)
# ---------------------------------------------------------------- C: direction plot
def fig_C(path):
fig, ax = plt.subplots(figsize=(W, 3.4))
fig.subplots_adjust(left=0.10, right=0.97, top=0.80, bottom=0.17)
ys = {"PKN1": 3, "PKN2": 2, "PKN3": 1, "PKN4": 0}
ax.set_xscale("log", base=2)
ax.set_xlim(1 / 8, 8)
ax.set_ylim(-0.6, 3.75)
ticks = [1 / 8, 1 / 4, 1 / 2, 1, 2, 4, 8]
ax.set_xticks(ticks); ax.set_xticklabels(["1/8", "1/4", "1/2", "1", "2", "4", "8"])
ax.set_yticks(list(ys.values())); ax.set_yticklabels(list(ys.keys()), fontsize=11, color=INK)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.axvline(1, color=INK2, lw=1)
ax.text(1, 3.77, "no change", ha="center", va="bottom", fontsize=10, color=INK2)
ax.set_xlabel("Abundance ratio, CFBE41o- / HBE41o- (log scale)")
# abstract claim: direction only, no value -> shaded half-row on the 'reduced' side
for p in ("PKN1", "PKN4"):
y = ys[p] - (0.12 if p == "PKN1" else 0)
ax.add_patch(Rectangle((1 / 8, y - 0.22), 1 - 1 / 8, 0.44, fc=ORANGE_T, ec=ORANGE, lw=1.5,
hatch="///", alpha=1))
ax.text(0.2, y, "abstract: \u201creduced\u201d", ha="center", va="center",
fontsize=10, color=INK, bbox=dict(fc=BG, ec="none", pad=1.5))
ax.text(2.8, ys["PKN1"], "body: presence\nuncertain", ha="center", va="center", fontsize=10, color=INK2)
ax.plot([r_sh], [ys["PKN1"]], "o", ms=10, mfc=BG, mec=BLUE, mew=2, zorder=5)
ax.text(r_sh, ys["PKN1"] + 0.26, f"{r_sh:.1f} shared peptide only (also PKN2, PKN3)", ha="center",
va="bottom", fontsize=10, bbox=dict(fc=BG, ec="none", pad=1))
ax.text(2.8, ys["PKN4"], "body, supplements:\nnot mentioned", ha="center", va="center", fontsize=10, color=INK2)
# recomputed / reported in body
ax.plot([r_pkn2], [ys["PKN2"]], "o", ms=10, color=BLUE, mec=BG, mew=2)
ax.text(r_pkn2, ys["PKN2"] + 0.2, f"{r_pkn2:.1f} (Results; Supp. Data 6, 7)", va="bottom",
ha="center", fontsize=10, bbox=dict(fc=BG, ec="none", pad=1))
ax.plot([r_pkn3], [ys["PKN3"]], "o", ms=10, color=BLUE, mec=BG, mew=2)
ax.text(r_pkn3, ys["PKN3"] + 0.2, f"{r_pkn3:.3f} (Supp. Data 6, 7 only)", va="bottom", ha="center",
fontsize=10, bbox=dict(fc=BG, ec="none", pad=1))
# legend
from matplotlib.lines import Line2D
h = [Rectangle((0, 0), 1, 1, fc=ORANGE_T, ec=ORANGE, hatch="///", lw=1.5),
Line2D([], [], marker="o", ls="", ms=10, color=BLUE, mec=BG, mew=2),
Line2D([], [], marker="o", ls="", ms=10, mfc=BG, mec=BLUE, mew=2)]
ax.legend(h, ["Abstract: direction only", "Ratio, protein-specific peptides",
"Ratio, shared peptide"],
loc="lower center", bbox_to_anchor=(0.5, 1.07), ncol=3, frameon=False, fontsize=10,
handlelength=1.4, columnspacing=1.2)
save(fig, path)
# ---------------------------------------------------------------- D: mention counts
def fig_D(path):
fig, ax = plt.subplots(figsize=(W, 3.0))
fig.subplots_adjust(left=0.10, right=0.97, top=0.82, bottom=0.17)
names = ["PKN1", "PKN2", "PKN3", "PKN4"]
for i, n in enumerate(names):
y = len(names) - 1 - i
a, b = counts[n]
ax.barh(y + 0.19, a, height=0.34, color=ORANGE, edgecolor=BG, lw=2)
ax.barh(y - 0.19, b, height=0.34, color=BLUE, edgecolor=BG, lw=2)
ax.text(a + 0.2, y + 0.19, str(a), va="center", fontsize=10)
ax.text(b + 0.2, y - 0.19, str(b) + (" (named as reduced in abstract)" if n == "PKN4" else ""),
va="center", fontsize=10, color=INK)
ax.set_yticks(range(4)); ax.set_yticklabels(names[::-1], fontsize=11, color=INK)
ax.set_xlim(0, 15); ax.set_xlabel("Number of mentions in the archived article")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
h = [Rectangle((0, 0), 1, 1, fc=ORANGE), Rectangle((0, 0), 1, 1, fc=BLUE)]
ax.legend(h, ["Abstract (where the claim is made)", "Rest of article (Results, legends, Methods)"],
loc="lower center", bbox_to_anchor=(0.5, 1.02), ncol=2, frameon=False, fontsize=10)
save(fig, path)
# ---------------------------------------------------------------- E: SD7 peptide ratios
def fig_E(path):
fig, ax = plt.subplots(figsize=(W, 3.0))
fig.subplots_adjust(left=0.31, right=0.97, top=0.80, bottom=0.18)
rows = [("PKN2 (Q16513)\nunique peptides (3)", r_pkn2, u2),
("shared peptide IADFGLCK\n(PKN1 node's only peptide)", r_sh, s),
("PKN3 (Q6P5Z2)\nunique peptides (3)", r_pkn3, u1)]
ax.set_xscale("log", base=2); ax.set_xlim(1 / 8, 8); ax.set_ylim(-0.4, 2.75)
ax.set_xticks([1 / 8, 1 / 4, 1 / 2, 1, 2, 4, 8])
ax.set_xticklabels(["1/8", "1/4", "1/2", "1", "2", "4", "8"])
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.axvline(1, color=INK2, lw=1)
ax.text(1, 2.77, "no change", ha="center", va="bottom", fontsize=10, color=INK2)
for i, (lab, r, l2) in enumerate(rows):
y = 2 - i
ax.plot([1, r], [y, y], color=BLUE, lw=2)
ax.plot([r], [y], "o", ms=10, color=BLUE, mec=BG, mew=2)
ax.text(r, y + 0.2, f"{r:.3f} (log2 {l2:.3f})", va="bottom", ha="center", fontsize=10,
bbox=dict(fc=BG, ec="none", pad=1))
ax.set_yticks([2, 1, 0]); ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_xlabel("Abundance ratio, CFBE41o- / HBE41o- (log scale)")
ax.set_title("Every PKN peptide ratio in Supplementary Data 6 and 7",
fontsize=10, color=INK2, loc="left", pad=20)
save(fig, path)
print("\n=== 4. Figures")
fig_A(os.path.join(CAND, "fig_A_text_panels"))
fig_B(os.path.join(CAND, "fig_B_evidence_matrix"))
fig_C(os.path.join(CAND, "fig_C_direction_plot"))
fig_D(os.path.join(CAND, "fig_D_mention_counts"))
fig_E(os.path.join(CAND, "fig_E_peptide_ratios"))
# The figures used in writeup.md: fig1 = candidate A, fig2 = candidate C (until a design is chosen)
for src, dst in [("fig_A_text_panels", "fig1"), ("fig_C_direction_plot", "fig2")]:
for ext in (".svg", ".png"):
shutil.copyfile(os.path.join(CAND, src + ext), os.path.join(HERE, dst + ext))
print(" copied candidates A -> fig1, C -> fig2")
Abstract says host genetics explains almost half of meningitis susceptibility; the paper estimates 29%
Nat Commun (2019) · doi:10.1038/s41467-019-09976-3 · PMC6520353
Summary
The abstract states that human genetic variation "explains almost half of variation in susceptibility to pneumococcal meningitis and one-third of variation in severity". Table 4 gives 0.25 (GCTA) and 0.29 (LDAK) for susceptibility and 0.29 (GCTA) and 0.49 (LDAK) for severity; the Results and the Discussion quote only the LDAK values, as 29% ± 7% for susceptibility and 49% ± 14% for severity (the Discussion gives the severity error as ± 9%). No susceptibility estimate in the paper is near one half: 0.5 lies outside both recomputed susceptibility 95% confidence intervals. The abstract's two magnitudes match the body's headline values only if the phenotypes are swapped. The abstract also implies that host genetics matters more for susceptibility than for severity, while the paper's headline estimates rank them the other way. The significance of either estimate is unaffected, and so are the paper's other findings.
What the paper reports
Abstract:
In a genome-wide association study of human and pathogen we show that human variation explains almost half of variation in susceptibility to pneumococcal meningitis and one-third of variation in severity, identifying variants in CCDC33 associated with susceptibility.
Results, "Human genetics associates with susceptibility and severity", paragraph 3:
We then used two standard methods to calculate SNP heritability, which both showed that variation in host genetics explains 29% ± 7% of the observed variation in pneumococcal meningitis susceptibility and 49% ± 14% of the variation in meningitis severity (Table 4).
Discussion, paragraph 2:
Host genetics explained 29% ± 7% of variation in susceptibility to meningitis.
We found no evidence for pathogen genetics affecting the severity of disease, whereas human genetics explained 49% ± 9% of this variation.
Table 4, "Human SNP heritability (h2SNP) of meningitis in the MeninGene cohort" (columns: Phenotype | Method | Heritability | Error | p-value):
Susceptibility (pneumococcal) | GCTA | 0.25 | 0.05 | 2.4 × 10–6
Susceptibility (pneumococcal) | LDAK | 0.29 | 0.07 | 3.9 × 10–6
Severity (any species) | GCTA | 0.29 | 0.11 | 2.8 × 10–5
Severity (any species) | LDAK | 0.49 | 0.14 | 1.4 × 10–4
Table 4 footer:
Heritabilities are shown on the liability scale (adjusted for population prevalence and case ascertainment ratio).
What the paper's own numbers show
Method. The abstract gives words, not numbers, so I read "almost half" as about 0.50 and "one-third" as 0.333. I compared each with both Table 4 estimates for the same phenotype. I built 95% confidence intervals as estimate ± 1.96 × the Table 4 "Error" column, treating that column as a standard error, and checked whether each interval contains 0.5 or 1/3. I also checked that the Results and Discussion percentages match the Table 4 rows. Finally I tested whether the two LDAK estimates differ from each other, with a z-test that treats them as independent. That is an approximation: the two phenotypes share control samples, so the estimates are not strictly independent. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Host h² for susceptibility | Abstract | "almost half" (≈ 0.50) | Table 4 gives 0.25 (GCTA) and 0.29 (LDAK). The nearest is 0.21 below 0.50 |
| Host h² for severity | Abstract | "one-third" (≈ 0.33) | Table 4 gives 0.29 (GCTA, 0.04 away) and 0.49 (LDAK, 0.16 away). The text's headline value is 0.49 |
| Susceptibility, GCTA | Table 4 | 0.25 ± 0.05 | 95% CI 0.152–0.348, which excludes 0.5 |
| Susceptibility, LDAK | Table 4; Results p3; Discussion p2 | 0.29 ± 0.07; "29% ± 7%" | Text matches the table. 95% CI 0.153–0.427, which excludes 0.5 |
| Severity, GCTA | Table 4 | 0.29 ± 0.11 | 95% CI 0.074–0.506 |
| Severity, LDAK | Table 4; Results p3 | 0.49 ± 0.14; "49% ± 14%" | Text matches the table. 95% CI 0.216–0.764 |
| Abstract reading vs headline value, susceptibility | Abstract vs Results p3 | ≈ 0.50 vs 0.29 | 21.0 percentage points higher, 1.72 times the paper's estimate |
| Abstract reading vs headline value, severity | Abstract vs Results p3 | ≈ 0.33 vs 0.49 | 15.7 percentage points lower |
| Value nearest to "almost half" anywhere in Table 4 | Table 4 | — | Severity, LDAK, 0.49 (0.01 from 0.50) |
| Severity minus susceptibility, LDAK | Table 4 | 0.49 − 0.29 | Difference 0.20, SE 0.157, z = 1.28, two-sided p = 0.20 (GCTA: 0.04, z = 0.33, p = 0.74) |
| Severity error | Discussion p2 | "49% ± 9%" | Table 4 and the Results give ± 14%. With ± 9% the interval would be 0.314–0.666 |
The arithmetic in the body is internally consistent: the Results and Discussion quote the LDAK rows of Table 4. The inconsistency is between the abstract and everything else. For susceptibility no estimator in the paper gives a value near one half. For severity, "one-third" is close to the GCTA estimate (0.29), but the paper presents the LDAK estimate (0.49) as its headline severity figure in both the Results and the Discussion. Read together, the abstract's two magnitudes match the body's headline values (29% and 49%) with the phenotypes swapped.
Figure
Figure 1. On the left is the abstract sentence. On the right is the Results sentence that cites Table 4 (verbatim excerpts). Below each are the values each passage gives for the same two phenotypes. Orange diamonds mark the abstract's words read as fractions. Blue dots mark the Table 4 LDAK estimates quoted in the Results. For susceptibility, the abstract's "almost half" (≈ 0.50) conflicts with 0.29 ± 0.07, and 0.5 is outside both recomputed susceptibility 95% confidence intervals. For severity, the abstract's "one-third" (≈ 0.33) sits below the headline 0.49 ± 0.14, although it is close to the GCTA estimate of 0.29. All values are host SNP heritability on the liability scale.
Figure 2. All four Table 4 estimates (blue dots) with 95% confidence intervals recomputed as ± 1.96 × the printed error, on one heritability axis that starts at zero (no heritability). The orange vertical marks show where the abstract's words fall for each phenotype. The susceptibility mark at 0.5 lies to the right of both susceptibility intervals. The severity mark at 1/3 lies inside both severity intervals. The intervals are wide, so the severity wording is not ruled out statistically, but it does not match the value the paper reports as its headline severity estimate.
Why it matters
The abstract is the part of the paper most readers see, and the host-heritability sentence is its main quantitative claim about human genetics. As written it overstates the host contribution to susceptibility by about 20 percentage points (≈ 0.50 against 0.29) and understates the headline severity estimate (≈ 0.33 against 0.49). It also reverses the order of the two phenotypes. The abstract suggests that host genetics matters more for susceptibility, while the paper's headline estimates suggest the opposite. Because the two estimates are not significantly different (z = 1.28, p = 0.20 under an independence approximation), the reversal matters less for the substance than for how the result is described. Anyone who cites the abstract's figures, for example in a review comparing heritability across infectious diseases, would carry forward a susceptibility value the paper does not support.
What the discrepancy does not affect: both host estimates remain significantly above zero (Table 4 p-values 3.9 × 10–6 and 1.4 × 10–4 for LDAK). The pathogen heritability results (70% for invasiveness, 0% for severity) are reported separately and correctly. The CCDC33, UBE2U, pspC and zmpD findings and the vaccine-target discussion are not affected.
Explanations considered
- A different estimator. Table 4 reports GCTA and LDAK for each phenotype. For susceptibility both (0.25 and 0.29) are well below one half, so neither estimator supports "almost half". For severity, GCTA's 0.29 is close to "one-third", but the susceptibility half of the sentence still has no supporting estimate. The Results and Discussion use the LDAK values throughout. Does not account for the discrepancy.
- A different scale or population. All Table 4 values are on the liability scale (prevalence 1 × 10–3) in the Dutch MeninGene cohort, and the text quotes those same values. The paper reports no other host heritability, for example observed-scale or pooled invasive disease, that could be near 0.5 for susceptibility. Does not account for it.
- Confusion with a pathogen estimate. The abstract reports the pathogen figures (70% of invasive potential, no effect on severity) in a separate sentence. No pathogen estimate is presented as a host susceptibility value. Not applicable.
- The pooled severity phenotype. Severity heritability was estimated by combining severe pneumococcal cases with severe cases from any species. That changes how severity is defined, not which number belongs to which phenotype. Does not account for it.
- Rounding or uncertainty. "Almost half" cannot be read as a loose rounding of 0.29, and 0.5 lies outside both susceptibility 95% confidence intervals (upper limits 0.348 and 0.427). Does not account for it.
- A text-conversion artifact. The conflict is between words ("almost half", "one-third") and printed percentages and decimals. There are no minus signs, superscripts or column alignments that a conversion could have garbled, and the publisher's page has the same wording. Ruled out.
- Carried over from an earlier version. The bioRxiv preprint (doi:10.1101/386078, posted 2018-08-07) has the same abstract sentence ("human variation explains almost half of variation in susceptibility to pneumococcal meningitis and one-third of variation in severity"). That shows when the wording appeared, not why. It does not reconcile the abstract with the published Table 4.
- A second, minor inconsistency. The Discussion gives severity as "49% ± 9%", while Table 4 and the Results give ± 14%. This is separate from the main discrepancy and does not change the estimate itself.
Status
As of 2026-10-23:
- Version of record. The publisher's HTML (nature.com/articles/s41467-019-09976-3) has the same abstract, Results and Discussion wording as the PMC archive, including "49% ± 9%" in the Discussion. The Table 4 rows were checked in the PMC archive and in the Europe PMC full-text XML, which agree: 0.25 ± 0.05, 0.29 ± 0.07, 0.29 ± 0.11 and 0.49 ± 0.14.
- Supplements. Both electronic supplementary files on the publisher's site were searched: the Supplementary Information PDF (36 pages) and the Peer Review File (19 pages). Neither reports a host heritability value for susceptibility or severity, and neither mentions the abstract wording. The external data deposits (figshare, Zenodo and the analysis-code repository) hold summary statistics and metadata, not alternative heritability estimates.
- Corrections. Crossref shows no update relations (no correction, erratum or retraction). The only relation is to the preprint. PMC and Europe PMC list no correction or retraction notice; Europe PMC's only linked record is the preprint. The Crossref record that carries Retraction Watch data has no correction or retraction entry.
- PubPeer. PubPeer's search API returned a human-verification challenge and could not be queried automatically. A web search for the title on PubPeer found no PubPeer entry. Absence of comments there is therefore not confirmed.
- Preprint. bioRxiv doi:10.1101/386078, version 1 (2018-08-07), has the same abstract wording.
Sources: PMC6520353; publisher version; Crossref record; Europe PMC; bioRxiv preprint.
Suggested correction
Bring the abstract into line with Table 4 and the Results, for example:
… we show that human variation explains around 30% of variation in susceptibility to pneumococcal meningitis and almost half of variation in severity, …
If the authors prefer to describe severity by the GCTA estimate (0.29), the abstract should say so and give the same order of magnitude for both phenotypes ("around 30%"). In that case the Results and Discussion, which present 49% as the severity figure, would also need to change. Separately, the Discussion's "49% ± 9%" should read "49% ± 14%" to match Table 4 and the Results.
Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC6520353 and regenerate the figures.
Run: python recompute.py.txt
All inputs are copied from the article as archived in PubMed Central (PMC6520353):
Abstract, Results ("Human genetics associates with susceptibility and severity", paragraph 3),
Discussion paragraph 2 and Table 4 (paragraphs counted from 1). Outputs are written next to this file:
fig1.svg/png, fig2.svg/png and candidates/design_1..5.svg/png (five figure designs).
"""
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from scipy.stats import norm
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs
# Table 4: Human SNP heritability, liability scale (estimate, error, printed p)
T4 = {
("Susceptibility", "GCTA"): (0.25, 0.05, "2.4e-6"),
("Susceptibility", "LDAK"): (0.29, 0.07, "3.9e-6"),
("Severity", "GCTA"): (0.29, 0.11, "2.8e-5"),
("Severity", "LDAK"): (0.49, 0.14, "1.4e-4"),
}
# Abstract wording, read as the fractions the words denote
ABSTRACT = {"Susceptibility": ("almost half", 0.5), "Severity": ("one-third", 1 / 3)}
# Results paragraph 3 and Discussion paragraph 2 (percent, +/- percent)
RESULTS = {"Susceptibility": (29, 7), "Severity": (49, 14)}
DISCUSSION = {"Susceptibility": (29, 7), "Severity": (49, 9)}
Z = norm.ppf(0.975)
# ---------------------------------------------------------------- numbers
print("== Table 4 estimates with 95% CI (estimate +/- 1.96 x error)")
ci = {}
for (ph, m), (h, se, p) in T4.items():
lo, hi = h - Z * se, h + Z * se
ci[(ph, m)] = (lo, hi)
print(f"{ph:15s} {m}: {h:.2f} +/- {se:.2f} 95% CI {lo:.3f} to {hi:.3f} "
f"contains 0.5: {lo <= 0.5 <= hi} contains 1/3: {lo <= 1/3 <= hi}")
print("\n== Results and Discussion text vs Table 4 LDAK rows")
for ph in ("Susceptibility", "Severity"):
h, se, _ = T4[(ph, "LDAK")]
r, d = RESULTS[ph], DISCUSSION[ph]
print(f"{ph}: Results {r[0]}% +/- {r[1]}% Discussion {d[0]}% +/- {d[1]}% "
f"Table 4 LDAK {h*100:.0f}% +/- {se*100:.0f}% "
f"Results match: {r == (round(h*100), round(se*100))} "
f"Discussion match: {d == (round(h*100), round(se*100))}")
h, _, _ = T4[("Severity", "LDAK")]
print(f"Severity CI if the Discussion error 0.09 were used: {h-Z*0.09:.3f} to {h+Z*0.09:.3f}")
print("\n== Distance from each abstract word to each Table 4 estimate")
for ph, (word, val) in ABSTRACT.items():
print(f"Abstract '{word}' ({val:.3f}) for {ph}:")
for (ph2, m), (h, se, _) in T4.items():
print(f" {ph2:15s} {m}: |{h:.2f} - {val:.3f}| = {abs(h-val):.3f}"
f"{' <- same phenotype' if ph2 == ph else ''}")
best = {ph: min((abs(T4[(ph, m)][0] - v), m) for m in ("GCTA", "LDAK"))
for ph, (_, v) in ABSTRACT.items()}
print("Closest same-phenotype estimate:", {k: (m, round(dist, 3)) for k, (dist, m) in best.items()})
print("\n== Size of the discrepancy (abstract reading vs headline LDAK value)")
for ph, (word, val) in ABSTRACT.items():
h = T4[(ph, "LDAK")][0]
print(f"{ph}: abstract {val:.3f} vs {h:.2f}: difference {100*(val-h):+.1f} percentage points, "
f"ratio {val/h:.2f}")
print("\n== Is severity h2 different from susceptibility h2? (LDAK, treated as independent)")
(h1, s1, _), (h2, s2, _) = T4[("Susceptibility", "LDAK")], T4[("Severity", "LDAK")]
zd = (h2 - h1) / math.hypot(s1, s2)
print(f"difference {h2-h1:.2f}, SE {math.hypot(s1, s2):.4f}, z {zd:.2f}, two-sided p {2*norm.sf(abs(zd)):.2f}")
(g1, gs1, _), (g2, gs2, _) = T4[("Susceptibility", "GCTA")], T4[("Severity", "GCTA")]
zg = (g2 - g1) / math.hypot(gs1, gs2)
print(f"GCTA: difference {g2-g1:.2f}, z {zg:.2f}, two-sided p {2*norm.sf(abs(zg)):.2f}")
# ---------------------------------------------------------------- style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2
def save(fig, stem):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(f"{stem}.{ext}", bbox_inches="tight", pad_inches=0.15, **kw)
plt.close(fig)
def axis_clean(ax):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
# ---- design 1: two text panels joined by annotated connectors
def design_1(stem):
fig = plt.figure(figsize=(W, 4.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100); ax.set_ylim(0, 62); ax.axis("off")
panels = [
(1, "Abstract", ORANGE,
"\u201chuman variation explains almost\nhalf of variation in susceptibility\n"
"to pneumococcal meningitis and\none-third of variation in severity\u201d",
[("Susceptibility", "almost half (\u2248 0.50)"), ("Severity", "one-third (\u2248 0.33)")]),
(61, "Results and Table 4", BLUE,
"\u201cvariation in host genetics explains\n29% \u00b1 7% of the observed variation\n"
"in pneumococcal meningitis\nsusceptibility and 49% \u00b1 14% of the\n"
"variation in meningitis severity\u201d",
[("Susceptibility", "0.29 \u00b1 0.07"), ("Severity", "0.49 \u00b1 0.14")]),
]
rows_y = (18, 7.5)
for x0, title, col, quote, rows in panels:
ax.add_patch(FancyBboxPatch((x0, 2), 38, 57, boxstyle="round,pad=0.3,rounding_size=1.5",
fc="white", ec=col, lw=2))
ax.text(x0 + 2, 56, title, fontsize=11.5, weight="bold", va="top", color=INK)
ax.text(x0 + 2, 49, quote, fontsize=10, va="top", color=INK2, linespacing=1.35)
ax.plot([x0 + 2, x0 + 36], [23, 23], color=GRID, lw=1)
for (ph, val), y in zip(rows, rows_y):
ax.text(x0 + 4, y + 2.6, ph, fontsize=10.5, va="center", color=INK2)
ax.text(x0 + 4, y - 1.6, val, fontsize=11.5, va="center", weight="bold", color=INK)
ax.plot(x0 + 2.3, y - 1.6, "D" if col == ORANGE else "o", ms=8, color=col)
labels = ("0.50 vs 0.29\n(0.5 is outside\nboth 95% CIs)",
"0.33 vs 0.49\n(Table 4 GCTA: 0.29)")
for y, lab in zip(rows_y, labels):
ax.add_patch(FancyArrowPatch((40.5, y - 1.6), (60, y - 1.6), arrowstyle="<->", mutation_scale=12,
color=INK2, lw=2))
ax.text(50.2, y + 0.2, lab, ha="center", va="bottom", fontsize=10, color=INK, linespacing=1.2)
ax.text(50.2, 37, "Same\nphenotypes,\nmagnitudes\nin reverse\norder", ha="center", va="center",
fontsize=10.5, color=INK, weight="bold")
ax.text(50.2, -2.5, "Host SNP heritability, liability scale. The abstract gives words, read here as fractions. "
"LDAK values shown on the right.", ha="center", fontsize=10, color=INK2)
save(fig, stem)
# ---- design 2: dot-and-interval on one h2 axis
def design_2(stem):
fig, ax = plt.subplots(figsize=(W, 3.6))
order = [("Susceptibility", "GCTA"), ("Susceptibility", "LDAK"),
("Severity", "GCTA"), ("Severity", "LDAK")]
ys = [4.6, 3.8, 1.8, 1.0]
for (ph, m), y in zip(order, ys):
h, se, _ = T4[(ph, m)]
lo, hi = ci[(ph, m)]
ax.plot([lo, hi], [y, y], color=BLUE, lw=2, solid_capstyle="butt")
ax.plot(h, y, "o", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=3)
ax.text(0.81, y, f"{m} {h:.2f} ({lo:.2f}\u2013{hi:.2f})", va="center",
fontsize=10, color=INK2)
for ph, (yb, yt) in (("Susceptibility", (3.4, 5.0)), ("Severity", (0.6, 2.2))):
word, v = ABSTRACT[ph]
ax.plot([v, v], [yb, yt], color=ORANGE, lw=2.5)
ax.plot(v, yt, "D", ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.text(v, yt + 0.22, f"Abstract: \u201c{word}\u201d", ha="center", fontsize=10,
color=INK, weight="bold")
ax.axvline(0, color=INK2, lw=0.8)
ax.set_xlim(-0.02, 1.12); ax.set_ylim(0.2, 5.8)
ax.set_xticks([0, 0.2, 0.4, 0.6, 0.8])
ax.set_yticks([4.2, 1.4]); ax.set_yticklabels(["Susceptibility\n(pneumococcal)", "Severity\n(any species)"],
fontsize=10.5, color=INK)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Host SNP heritability, liability scale (h\u00b2 = 0 is no heritability)", x=0.36)
axis_clean(ax)
h1 = [plt.Line2D([], [], color=BLUE, marker="o", ms=8, lw=2,
label="Table 4 estimate, 95% CI recomputed as \u00b11.96 \u00d7 error"),
plt.Line2D([], [], color=ORANGE, marker="D", ms=8, lw=2.5,
label="Abstract wording, read as a fraction")]
ax.legend(handles=h1, loc="upper center", bbox_to_anchor=(0.45, -0.22), ncol=1,
frameon=False, fontsize=10)
save(fig, stem)
# ---- design 3: dumbbell, abstract vs headline LDAK value per phenotype
def design_3(stem):
fig, ax = plt.subplots(figsize=(W, 2.9))
for ph, y in (("Susceptibility", 2), ("Severity", 1)):
word, v = ABSTRACT[ph]
h, se, _ = T4[(ph, "LDAK")]
lo, hi = ci[(ph, "LDAK")]
ax.plot([lo, hi], [y - 0.12, y - 0.12], color=BLUE, lw=2, alpha=0.45)
ax.add_patch(FancyArrowPatch((v, y), (h, y), arrowstyle="-|>", mutation_scale=14,
color=INK2, lw=2, shrinkA=6, shrinkB=6))
ax.plot(v, y, "D", ms=9, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.plot(h, y, "o", ms=9, color=BLUE, mec=BG, mew=1.5, zorder=3)
ax.text(v, y + 0.2, f"“{word}”", ha="center", fontsize=10, color=INK)
ax.text(h, y + 0.2, f"{h:.2f} ± {se:.2f}", ha="center", fontsize=10, color=INK)
ax.text(0.985, y, f"{100*(v-h):+.0f} pp".replace("-", "\u2212"), ha="right", va="center", fontsize=10.5,
weight="bold", color=INK)
ax.set_xlim(0, 1.0); ax.set_ylim(0.5, 2.6)
ax.set_yticks([2, 1]); ax.set_yticklabels(["Susceptibility", "Severity"], fontsize=10.5, color=INK)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Host SNP heritability, liability scale")
axis_clean(ax)
hs = [plt.Line2D([], [], color=ORANGE, marker="D", ms=9, lw=0, label="Abstract (word read as fraction)"),
plt.Line2D([], [], color=BLUE, marker="o", ms=9, lw=0, label="Results / Table 4, LDAK"),
plt.Line2D([], [], color=BLUE, lw=2, alpha=0.45, label="its recomputed 95% CI")]
ax.legend(handles=hs, loc="upper center", bbox_to_anchor=(0.5, -0.3), ncol=3,
frameon=False, fontsize=10)
save(fig, stem)
# ---- design 4: slope chart, abstract -> body, lines cross
def design_4(stem):
fig, ax = plt.subplots(figsize=(W, 3.8))
for ph, ls in (("Susceptibility", "-"), ("Severity", "--")):
word, v = ABSTRACT[ph]
h, se, _ = T4[(ph, "LDAK")]
ax.plot([0, 1], [v, h], color=INK2, lw=2, ls=ls)
ax.plot(0, v, "D", ms=9, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.plot(1, h, "o", ms=9, color=BLUE, mec=BG, mew=1.5, zorder=3)
ax.text(-0.04, v, f"{ph}: “{word}”", ha="right", va="center", fontsize=10.5)
ax.text(1.04, h, f"{ph}: {h:.2f} ± {se:.2f}", ha="left", va="center", fontsize=10.5)
ax.set_xlim(-1.15, 1.85); ax.set_ylim(0, 0.6)
ax.set_xticks([0, 1]); ax.set_xticklabels(["Abstract\n(orange: words read as fractions)",
"Results / Table 4, LDAK\n(blue)"], fontsize=10.5, color=INK)
ax.set_ylabel("Host SNP heritability, liability scale")
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.text(0.5, 0.56, "Lines cross: the abstract ranks the phenotypes\nthe opposite way to the paper's own estimates",
ha="center", fontsize=10, color=INK2)
save(fig, stem)
# ---- design 5: one number line, distance from each abstract word to its own estimates
def design_5(stem):
fig, ax = plt.subplots(figsize=(W, 3.6))
ax.axhline(0, color=INK2, lw=1, zorder=1)
for h, lab, ha in ((0.25, "0.25\nSusc. GCTA", "right"),
(0.29, "0.29\nSusc. LDAK,\nSev. GCTA", "left"),
(0.49, "0.49\nSev. LDAK", "left")):
ax.plot(h, 0, "o", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=5) # drawn over the diamonds
ax.text(h + (0.006 if ha == "left" else -0.006), -0.12, lab, ha=ha, va="top",
fontsize=10, color=INK, linespacing=1.15)
for ph, (word, v) in ABSTRACT.items():
ax.plot(v, 0, "D", ms=13, color=ORANGE, mec=BG, mew=1.5, zorder=4)
ax.plot([v, v], [0.12, 0.42], color=ORANGE, lw=2)
ax.text(v + (0.006 if v > 0.4 else -0.006), 0.48, f"Abstract, {ph.lower()}:\n\u201c{word}\u201d",
ha="left" if v > 0.4 else "right", va="bottom",
fontsize=10, color=INK)
for ph, y in (("Susceptibility", -1.45), ("Severity", -2.35)):
word, v = ABSTRACT[ph]
dist, m = best[ph]
h = T4[(ph, m)][0]
ax.annotate("", xy=(h, y), xytext=(v, y),
arrowprops=dict(arrowstyle="|-|", color=INK2, lw=2, mutation_scale=4))
ax.text(min(v, h), y + 0.13, f"{ph}: \u201c{word}\u201d is {dist:.2f} from the\nnearest "
f"{ph.lower()} estimate ({m} {h:.2f})", va="bottom", fontsize=10, color=INK2)
ax.set_xlim(0, 0.72); ax.set_ylim(-2.5, 1.15)
ax.set_xticks([0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6])
ax.set_yticks([])
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.set_xlabel("Host SNP heritability, liability scale (blue dots: Table 4; orange diamonds: abstract)",
x=0.5)
save(fig, stem)
DESIGNS = [design_1, design_2, design_3, design_4, design_5]
for i, f in enumerate(DESIGNS, 1):
f(os.path.join(CAND, f"design_{i}"))
design_1(os.path.join(HERE, "fig1"))
design_2(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE)
Headline protection p-value cannot be reproduced from the paper's counts using its stated test
Nat Commun (2023) · doi:10.1038/s41467-023-36907-0 · PMC9990026
Summary
The paper reports that after nine repeated low-dose intrarectal SIVmac251 challenges, 4 of 7 vaccinated macaques and 6 of 6 controls were infected, giving a vaccine efficacy (VE) of 43%, and attaches "P < 0.0001" to that result. The paper says this comparison used a one-tailed Fisher's exact test. Applied to the paper's own counts (3/7 vs 0/6 uninfected), that test gives p = 35/286 = 0.122 (two-tailed 0.192), which is above the paper's significance threshold of 0.05. With 7 vs 6 animals, no Fisher (conditional) or permutation test can give a p-value below 1/1716 = 5.8 × 10⁻⁴. Unconditional exact tests (Barnard, Boschloo) cannot go below (7/13)⁷(6/13)⁶ = 1.3 × 10⁻⁴, and a large-sample two-sided log-rank test on any 7-vs-6 outcome cannot go below 1.7 × 10⁻⁴. Each of these floors is above 0.0001, so none of these tests can produce "P < 0.0001" with these group sizes, whatever the outcome. The 43% point estimate is arithmetically correct. The statistical significance claimed for the paper's primary protection result is not supported.
What the paper reports
Results, section "Tonsil vaccination with MVA prevented acquisition in 3/7 animals after 9 repeated low dose IR challenges with SIVmac251", first paragraph:
All 6/6 control animals were infected by the 9th low dose IR challenge whereas only 4/7 vaccinated animals were infected by the 9th challenge demonstrating a VE = 43% (Fig. 2a; P < 0.0001).
Methods, "Data analysis":
Rate of SIV acquisition after each challenge was determined using Kaplan–Meier plots and vaccine efficacy (VE) after the 9th IR challenge was determined using the formula VE = 1 – [(infection rate among vaccinated animals)/(infection rate among control animals)] × 100 and followed by Fisher’s exact test as described by Klasse et al.101.
A P < 0.05 was considered significant.
Figure 2 legend, panel (a):
(a) Study design and percentage of uninfected animals after repeated low dose intrarectal challenge with SIVmac251 in tonsil vaccinated (n = 6) and control (n = 6) groups. Statistical analysis was performed using one-tailed Fishers Exact Test as described in ref. 101 and a P < 0.05 (*) was considered significant.
Figure 2a, Kaplan–Meier panel (text printed inside the plot, read from the published figure image):
VE = 43% P = <0.0001
Methods, "Animals, vaccination, infection, and samples" (group sizes):
(unvaccinated control; n = 6 and vaccinated; n = 7)
Abstract:
43% (3/7) of vaccinated macaques remained uninfected after 9 challenges as compared to the unvaccinated control (0/6) animals.
Introduction, last paragraph:
Our results demonstrated a significant level of protection from acquisition
What the paper's own numbers show
Method. At the 9th challenge the paper's counts form a 2×2 table: vaccinated, 3 uninfected and 4 infected (n = 7); control, 0 uninfected and 6 infected (n = 6). Fisher's exact test conditions on the margins: 3 uninfected animals out of 13, 7 of them vaccinated. The one-tailed p-value in the direction of protection is the probability that all 3 uninfected animals fall in the vaccinated group if vaccination had no effect, which is C(7,3)/C(13,3) = 35/286. The full null distribution of the number of uninfected animals in the vaccinated group (0, 1, 2 or 3 of 3) is 20/286, 105/286, 126/286 and 35/286. The two-tailed p adds the 0-of-3 tail, giving (35 + 20)/286. For the lower bound, the most extreme possible result (all 7 vaccinated uninfected, all 6 controls infected) has one-tailed p = 1/C(13,7) = 1/1716. Its two-tailed p is also 1/1716, because the only opposite-tail table (just 1 of the 7 uninfected animals in the vaccinated group) has probability 7/1716 and is therefore not added. A search over every 7-vs-6 table gives the same minimum two-tailed Fisher p. Unconditional exact tests do not condition on the margins and have a lower floor here: for the most extreme table, Barnard's and Boschloo's tests both give (7/13)⁷(6/13)⁶ = 1.27 × 10⁻⁴. The Kaplan–Meier curves in Fig. 2a show when each animal became infected, which allows a log-rank (Mantel–Cox) test. Read from the published figure, approximately: controls step from 100% to 83%, 67%, 33%, 17% and 0% at challenges 1, 2, 4 (two animals), 7 and 9. Vaccinated animals step from 100% to 57% at challenge 2 (three animals) and to 43% at challenge 4, and three remain uninfected. The Fisher values were computed with scipy.stats.fisher_exact and checked against the closed-form binomial ratios, the unconditional values with scipy.stats.barnard_exact and boschloo_exact, and the log-rank values with the standard Mantel–Cox formula, all in recompute.py.txt. To find the log-rank floor, the script also enumerates all 57,257,200 possible outcomes, with each animal infected at one of challenges 1 to 9 or uninfected after the 9th.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Vaccine efficacy, 1 − (4/7)/(6/6) | Results; Abstract | 43% | 42.9% (consistent) |
| p-value for the 9th-challenge acquisition comparison | Results, first acquisition paragraph | P < 0.0001 | 0.122 with the stated test |
| One-tailed Fisher exact, 3/7 vs 0/6 uninfected | Test named in Methods and Fig. 2a legend | (not given separately) | 35/286 = 0.1224 |
| Two-tailed Fisher exact, 3/7 vs 0/6 | Same table | — | 55/286 = 0.1923 |
| One-tailed Fisher exact if n = 6 vaccinated, 3/6 vs 0/6 | Fig. 2a legend states n = 6 | — | 20/220 = 0.0909 (two-tailed 0.1818) |
| Smallest attainable Fisher or permutation p, 7 vs 6 animals | Lower bound for conditional exact tests of this design | — | one-tailed 1/1716 = 5.83 × 10⁻⁴; two-tailed also 1/1716 = 5.83 × 10⁻⁴ |
| Smallest attainable Barnard or Boschloo p, 7 vs 6 animals | Lower bound for unconditional exact tests of this design | — | (7/13)⁷(6/13)⁶ = 1.27 × 10⁻⁴ (still above 1 × 10⁻⁴) |
| Barnard exact, 3/7 vs 0/6 | Same table | — | one-sided 0.106 (two-sided 0.106) |
| Boschloo exact, 3/7 vs 0/6 | Same table | — | one-sided 0.054 (two-sided 0.108) |
| Log-rank (Mantel–Cox), Fig. 2a curves as read | Fig. 2a Kaplan–Meier panel (approximate readings) | — | χ² = 1.55, p = 0.21 (one-sided 0.11); χ² = 1.07, p = 0.30 if the first vaccinated step is read at challenge 1 |
| Largest log-rank χ² over every possible 7-vs-6 outcome | Lower bound for the large-sample log-rank test | — | χ² = 14.14, two-sided p = 1.7 × 10⁻⁴ |
| Fewest uninfected vaccinated animals (of 7, vs 0/6) needed for one-tailed p < 0.05 | Same test | — | 4 of 7 (p = 0.0490) |
| Ratio of recomputed p to reported bound | — | — | 0.1224 / 0.0001 ≈ 1,224 |
Figure
Figure 1. The p-value reported for the 9th-challenge acquisition comparison in the Results and the Fig. 2a panel (orange, "P < 0.0001", drawn as an upper bound with an arrow) and p-values recomputed from the paper's own counts and curves (blue), all on a single log-scaled axis. The dashed line marks the paper's significance threshold, α = 0.05. The one-tailed Fisher exact test named in the Methods and the Fig. 2a legend gives 0.122 for 3/7 vs 0/6 uninfected. The two-tailed version gives 0.192. If the vaccinated group had the n = 6 stated in the Fig. 2a legend, the one-tailed value is 0.091. The unconditional exact tests give one-sided 0.054 (Boschloo) and 0.106 (Barnard). A log-rank test on the Fig. 2a curves, as read approximately, gives 0.21. The shaded band marks values below 1/1716 = 5.8 × 10⁻⁴, the smallest p a Fisher or permutation test can give with 7 vs 6 animals. The dotted line marks 1.3 × 10⁻⁴, the smallest p a Barnard or Boschloo test can give. The reported bound lies below both.
Figure 2. One-tailed Fisher exact p-values for every possible outcome of the 9th-challenge comparison with 0/6 controls uninfected, as the number of uninfected vaccinated animals runs from 1/7 to 7/7 (gray). The observed 3/7 is shown in blue (p = 0.122). Even the most extreme outcome, all seven vaccinated animals uninfected, gives p = 1/1716 = 5.8 × 10⁻⁴. That value is above the reported "P < 0.0001" (orange line). The design would have needed at least 4 of 7 vaccinated animals uninfected to reach p < 0.05.
Why it matters
The 9th-challenge acquisition comparison is the paper's primary efficacy result. The paper's title says the vaccine "protects macaques", the Introduction describes "a significant level of protection from acquisition", and the Abstract presents the 3/7 vs 0/6 comparison as its main finding. "P < 0.0001" is the only p-value the paper gives for this comparison (Results text and Fig. 2a panel). Recomputed with the stated test, the comparison does not reach the paper's own α of 0.05 (one-tailed p = 0.122). A reader would therefore take a result in 13 animals that does not reach significance to be highly significant.
Some parts of the paper are not affected. The VE point estimate (42.9%, reported as 43%) and the descriptive counts are internally consistent, and so is the report that one vaccinated animal stayed uninfected through 22 challenges. The separate analyses of acute viremia, CD4 counts, cytokines and immune correlates use other tests and other data and are not addressed here. The point concerns only the inferential support for protection from acquisition.
Explanations considered
- A different test, such as a log-rank test on the Kaplan–Meier curves. The Methods say Kaplan–Meier plots were used for the rate of acquisition. However, the reported P is attached to the VE at the 9th challenge, and both the Methods and the Fig. 2a legend name Fisher's exact test for that comparison. The paper does not report a log-rank or Mantel–Cox test anywhere. The infection times can be read approximately from the Fig. 2a curves: controls infected at challenges 1, 2, 4, 4, 7 and 9; vaccinated animals infected at challenge 2 (three animals) and 4 (one animal), with three uninfected. A log-rank test on these readings gives χ² = 1.55, p = 0.21. If the first vaccinated step is read at challenge 1 instead, χ² = 1.07, p = 0.30. Neither reading comes close to 0.05. Across every possible outcome with 7 vs 6 animals, the large-sample log-rank χ² cannot exceed 14.14, so its two-sided p cannot go below 1.7 × 10⁻⁴. A one-sided version reaches below 1 × 10⁻⁴ only when all six controls are infected before any vaccinated animal (smallest value 8.5 × 10⁻⁵), which is far from the curves in Fig. 2a, where three vaccinated animals were infected when only two controls had been. An exact (permutation) version of the test cannot go below 1/1716 = 5.8 × 10⁻⁴. This explanation does not account for the reported number.
- An unconditional exact test (Barnard or Boschloo). These tests do not fix the table margins, and on 7 vs 6 animals they can reach lower values than Fisher's test, down to (7/13)⁷(6/13)⁶ = 1.27 × 10⁻⁴, which is still above 1 × 10⁻⁴. On the observed 3/7 vs 0/6 they give one-sided p = 0.106 (Barnard) and 0.054 (Boschloo). Neither is below 0.05. This explanation does not account for the number.
- Group size of 6 rather than 7. The Fig. 2a legend gives "tonsil vaccinated (n = 6)", while the Methods, Results and Abstract give 7. The post-infection panels and legends (Figs. 2b–e, 3–5, 7 and 8) also give "tonsil vaccinated (n = 6)", which matches the six vaccinated animals that eventually became infected. Before challenge, the ELISPOT legend (Fig. 6) gives n = 7, and the anti-MVA antibody panel (Fig. 1c) gives n = 6. Fig. 2a covers acquisition in all 7 vaccinated animals but carries the same n = 6 label as the post-infection panels. Even taken at face value, 3/6 vs 0/6 gives a one-tailed Fisher p of 0.091 (two-tailed 0.182). This is still not significant at 0.05 and is roughly 900 times the reported bound. This explanation does not account for the number.
- One-tailed vs two-tailed. Both were computed: 0.122 and 0.192. Neither is close to 0.0001. This explanation does not account for the number.
- Per-exposure analysis (Klasse et al.). The cited reference discusses estimating efficacy per exposure in repeated low-dose challenge studies. The paper, however, defines VE at the 9th challenge with a per-animal formula and reports 43%, which reproduces exactly from the per-animal counts. A per-exposure analysis would give a different VE than the one reported. The paper does not describe such an analysis or report its result, so it cannot be verified as the source of the p-value.
- Typographical slip, for example a misplaced decimal. No obvious single-character change links the recomputed 0.122 to "< 0.0001". The paper prints "P < 0.0001" as an inequality, which is the form statistical software uses for very small values. The source of the reported value cannot be determined from the paper.
- Conversion or rendering artifact. The sentence reads identically in the PMC archive, the Europe PMC full-text XML supplied by the publisher, and the PMC public page. This explanation is ruled out.
Status
- Version of record. The statement appears verbatim in the PubMed Central and Europe PMC archives of the published article. The DOI resolves to the publisher's article page, which redirected to an authentication page and could not be fetched directly. The archived full text is the publisher's version of record.
- Supplements. The Supplementary Information PDF (supplementary figures and captions) and the Reporting Summary are listed with the article. The Supplementary Information was read in full. It contains no statistical test for the acquisition comparison and no occurrence of "0.0001", "Fisher", "log-rank", "Mantel" or "Kaplan". No separate source-data file is linked on PMC. The supplement does not resolve the discrepancy.
- Figure 2a. The Kaplan–Meier panel was checked in the publisher's figure image. It prints "VE = 43%" and "P = <0.0001" and shows the step pattern used for the log-rank readings above.
- Corrections. No correction, erratum, retraction, expression of concern or editor's note was found in Crossref (no update-to or updated-by relations), PMC notices, or Europe PMC (no comment or correction links).
- PubPeer. PubPeer could not be queried directly (HTTP 403). A web search for the DOI and title on PubPeer and Retraction Watch found no thread or notice.
- Sources: PMC9990026; Europe PMC record PMID 36882405; api.crossref.org/works/10.1038/s41467-023-36907-0. Checked as of 2026-10-23.
Suggested correction
The authors should report the p-value for the 9th-challenge acquisition comparison that the stated method gives for the stated counts: a one-tailed Fisher's exact test of 3/7 vs 0/6 uninfected, p = 0.12 (two-tailed p = 0.19). They should state that this comparison is not significant at the paper's α of 0.05. If a different analysis produced the reported value, they should name that test, report the per-animal infection times it used, and confirm that the value is valid for 13 animals. Wording that rests on this comparison should be revised to match, specifically "significant level of protection from acquisition" in the Introduction and the claim that the vaccine "protects" in the title and Abstract. The group size in the Fig. 2a legend (n = 6 vaccinated) should be reconciled with n = 7 in the Methods, Results and Abstract. The post-infection panels use n = 6, which matches the six vaccinated animals eventually infected, but Fig. 2a covers acquisition in all 7.
Recomputation script (python recompute.py.txt)
# Recomputation for PMC9990026 (Nat Commun 2023, doi:10.1038/s41467-023-36907-0)
# Run: python recompute.py.txt
# Prints every recomputed number in writeup.md and regenerates fig1/fig2 (.svg, .png)
# plus five candidate figure designs (A-E) in ./candidates/ for design review.
# candidates/ is review material only and is not part of the deliverable.
import os, itertools
from math import comb
import numpy as np
from scipy.stats import fisher_exact, hypergeom, barnard_exact, boschloo_exact, chi2, norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------- numbers from the paper ----------------
N_VAC, N_CTL = 7, 6 # Methods: control n = 6, vaccinated n = 7
INF_VAC, INF_CTL = 4, 6 # Results: 4/7 vaccinated, 6/6 control infected by 9th challenge
UNINF_VAC, UNINF_CTL = N_VAC - INF_VAC, N_CTL - INF_CTL # 3 and 0
REPORTED_P = 1e-4 # Results and Fig. 2a panel: "P < 0.0001" (an upper bound)
ALPHA = 0.05 # Methods and Fig. 2a legend
# Approximate readings of the Fig. 2a Kaplan-Meier steps (challenge number at which
# each animal became infected; 10 = still uninfected after the 9th challenge).
# Controls: 100 -> 83 (ch 1) -> 67 (ch 2) -> 33 (ch 4, two animals) -> 17 (ch 7) -> 0 (ch 9).
# Vaccinated: 100 -> 57 (ch 2, three animals) -> 43 (ch 4); three animals uninfected.
KM_CTL = [1, 2, 4, 4, 7, 9]
KM_VAC = [2, 2, 2, 4, 10, 10, 10]
KM_VAC_ALT = [1, 1, 1, 4, 10, 10, 10] # alternative reading: first vaccinated step at challenge 1
# ---------------- recomputation ----------------
ve = 1 - (INF_VAC / N_VAC) / (INF_CTL / N_CTL)
print(f"VE = 1 - (4/7)/(6/6) = {ve:.4f} = {100*ve:.1f}% (paper: 43%)")
table = [[UNINF_VAC, INF_VAC], [UNINF_CTL, INF_CTL]] # rows vaccinated, control; cols uninfected, infected
_, p1 = fisher_exact(table, alternative="greater")
_, p2 = fisher_exact(table, alternative="two-sided")
p1_exact = comb(7, 3) / comb(13, 3)
print(f"2x2 table (uninfected, infected): vaccinated {table[0]}, control {table[1]}")
print(f"One-tailed Fisher exact p = C(7,3)/C(13,3) = {comb(7,3)}/{comb(13,3)} = {p1_exact:.4f} (scipy {p1:.4f})")
print(f"Two-tailed Fisher exact p = {p2:.4f} (= {round(p2*286)}/286)")
# Null distribution: number of the 3 uninfected animals that fall in the vaccinated group
null = {k: hypergeom.pmf(k, 13, 3, 7) for k in range(4)}
for k, v in null.items():
print(f" P(k={k} of 3 uninfected in vaccinated group) = {v:.4f} = {round(v*286)}/286")
# Fig. 2a legend gives n = 6 vaccinated: 3/6 vs 0/6
_, p1_6 = fisher_exact([[3, 3], [0, 6]], alternative="greater")
_, p2_6 = fisher_exact([[3, 3], [0, 6]], alternative="two-sided")
print(f"If n=6 vaccinated (3/6 vs 0/6): one-tailed = C(6,3)/C(12,3) = {comb(6,3)}/{comb(12,3)} = {p1_6:.4f}; two-tailed = {p2_6:.4f}")
# Floors of conditional (Fisher) / permutation tests: most extreme table [[7,0],[0,6]]
floor = 1 / comb(13, 7)
_, pf = fisher_exact([[7, 0], [0, 6]], alternative="greater")
_, pf2 = fisher_exact([[7, 0], [0, 6]], alternative="two-sided")
min2 = min(fisher_exact([[a, 7 - a], [b, 6 - b]], alternative="two-sided")[1]
for a in range(8) for b in range(7))
print(f"Smallest attainable one-tailed Fisher/permutation p, 7 vs 6 = 1/C(13,7) = 1/{comb(13,7)} = {floor:.3e} (scipy {pf:.3e})")
print(f"Two-tailed Fisher p for [[7,0],[0,6]] = {pf2:.3e} (opposite-tail table k=1 has probability "
f"{hypergeom.pmf(1, 13, 7, 7)*comb(13,7):.0f}/1716 > 1/1716, so not added); "
f"minimum two-tailed Fisher p over all 7-vs-6 tables = {min2:.3e}")
# Unconditional exact tests (Barnard, Boschloo)
bar_floor = barnard_exact([[7, 0], [0, 6]], alternative="greater").pvalue
bos_floor = boschloo_exact([[7, 0], [0, 6]], alternative="greater").pvalue
closed = (7 / 13) ** 7 * (6 / 13) ** 6
print(f"Barnard floor, one-sided, [[7,0],[0,6]] = {bar_floor:.3e}; Boschloo floor = {bos_floor:.3e}; "
f"(7/13)^7 (6/13)^6 = {closed:.3e}; above 1e-4? {closed > REPORTED_P}")
bar1 = barnard_exact(table, alternative="greater").pvalue
bar2 = barnard_exact(table, alternative="two-sided").pvalue
bos1 = boschloo_exact(table, alternative="greater").pvalue
bos2 = boschloo_exact(table, alternative="two-sided").pvalue
print(f"Observed 3/7 vs 0/6: Barnard one-sided {bar1:.4f} (two-sided {bar2:.4f}); "
f"Boschloo one-sided {bos1:.4f} (two-sided {bos2:.4f})")
# Log-rank (Mantel-Cox) test on the Fig. 2a readings
def logrank(tv, tc, n_times=9):
tv, tc = np.asarray(tv), np.asarray(tc)
oe = var = 0.0
for t in range(1, n_times + 1):
nv, nc = (tv >= t).sum(), (tc >= t).sum(); n = nv + nc
dv = (tv == t).sum(); d = dv + (tc == t).sum()
if d == 0 or n < 2:
continue
oe += dv - d * nv / n
var += d * (nv / n) * (nc / n) * (n - d) / (n - 1)
return oe ** 2 / var, oe / np.sqrt(var)
lr_c, lr_z = logrank(KM_VAC, KM_CTL)
lr_p = chi2.sf(lr_c, 1)
lr_c_alt, _ = logrank(KM_VAC_ALT, KM_CTL)
lr_p_alt = chi2.sf(lr_c_alt, 1)
print(f"Log-rank on Fig. 2a readings: chi2 = {lr_c:.2f}, p = {lr_p:.3f} (one-sided {norm.cdf(lr_z):.3f}); "
f"alternative reading (first vaccinated step at challenge 1): chi2 = {lr_c_alt:.2f}, p = {lr_p_alt:.3f}")
# Largest asymptotic log-rank chi2 over every possible 7-vs-6 outcome
# (each animal infected at challenge 1..9 or uninfected after 9).
def compositions(n, k):
rows = []
for c in itertools.combinations(range(n + k - 1), k - 1):
prev, row = -1, []
for x in c:
row.append(x - prev - 1); prev = x
row.append(n + k - 2 - prev); rows.append(row)
return np.array(rows)
SLOTS = 10
CV, CC = compositions(7, SLOTS), compositions(6, SLOTS)
riskV = np.cumsum(CV[:, ::-1], axis=1)[:, ::-1][:, :9].astype(float)
riskC = np.cumsum(CC[:, ::-1], axis=1)[:, ::-1][:, :9].astype(float)
Z1E4 = norm.isf(REPORTED_P) # one-sided normal z for p = 1e-4
first_vac = np.array([next((t for t in range(9) if r[t] > 0), 9) for r in CV]) # slot of first vaccinated infection
lr_max, n_below, n_below_other = 0.0, 0, 0
with np.errstate(divide="ignore", invalid="ignore"):
for i in range(len(CC)):
nv, nc = riskV, riskC[i][None, :]; n = nv + nc
dv = CV[:, :9]; d = dv + CC[i, :9][None, :]
e = np.where(n > 0, d * nv / n, 0.0)
v = np.where(n > 1, d * (nv / n) * (nc / n) * (n - d) / (n - 1), 0.0)
vv = v.sum(1)
oe = (dv - e).sum(1)
c2 = np.where(vv > 0, oe ** 2 / vv, 0.0)
lr_max = max(lr_max, c2.max())
z = np.where(vv > 0, oe / np.sqrt(vv), 0.0)
below = z < -Z1E4 # one-sided p < 1e-4 in the direction of protection
ctl_before = np.cumsum(CC[i])[np.maximum(first_vac - 1, 0)] * (first_vac > 0)
all_ctl_first = ctl_before == 6 # every control infected before any vaccinated animal
n_below += below.sum()
n_below_other += (below & ~all_ctl_first).sum()
lr_floor = chi2.sf(lr_max, 1)
print(f"Over all {len(CV)*len(CC):,} outcomes: max log-rank chi2 = {lr_max:.2f}, two-sided p = {lr_floor:.2e}, "
f"one-sided normal p = {norm.sf(np.sqrt(lr_max)):.2e}")
print(f" outcomes with one-sided log-rank p < 1e-4: {n_below:,}; of these, outcomes in which any vaccinated "
f"animal was infected before all 6 controls: {n_below_other}")
print(f"Reported bound 1e-4 below Fisher floor? {REPORTED_P < floor}; floor / 1e-4 = {floor/REPORTED_P:.1f}")
print(f"Recomputed one-tailed p / reported bound = {p1/REPORTED_P:.0f}x")
# Ladder: one-tailed p if k of 7 vaccinated stayed uninfected and 0/6 controls did
ladder = {}
for k in range(1, 8):
_, pk = fisher_exact([[k, 7 - k], [0, 6]], alternative="greater")
ladder[k] = pk
print(f" k={k}/7 vaccinated uninfected vs 0/6: one-tailed p = C(7,{k})/C(13,{k}) = {pk:.4g}")
k_needed = min(k for k, v in ladder.items() if v < ALPHA)
print(f"Fewest uninfected vaccinated animals (of 7, vs 0/6) giving one-tailed p < 0.05: {k_needed} (p = {ladder[k_needed]:.4f})")
# ---------------- style ----------------
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"axes.edgecolor": "#b9b8b3", "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def clean(ax):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
# ---------- Candidate A: p-values on one log axis ----------
def design_A(path):
import matplotlib.transforms as mtrans
fig, ax = plt.subplots(figsize=(7.2, 5.0))
rows = [
("Reported (Results; Fig. 2a)", REPORTED_P, ORANGE, "P < 0.0001"),
("Fisher one-tailed, 3/7 vs 0/6\n(test named in Methods, legend)", p1, BLUE, f"{p1:.3f}"),
("Fisher two-tailed, 3/7 vs 0/6", p2, BLUE, f"{p2:.3f}"),
("Fisher one-tailed, 3/6 vs 0/6\n(n = 6 as in Fig. 2a legend)", p1_6, BLUE, f"{p1_6:.3f}"),
("Boschloo one-sided, 3/7 vs 0/6", bos1, BLUE, f"{bos1:.3f}"),
("Barnard one-sided, 3/7 vs 0/6", bar1, BLUE, f"{bar1:.3f}"),
("Log-rank on Fig. 2a curves\n(approximate readings)", lr_p, BLUE, f"{lr_p:.2f}"),
]
ys = list(range(len(rows)))[::-1]
ax.axvspan(3e-5, floor, color="#efeeea", zorder=0)
ax.axvline(bar_floor, color=SEC, lw=1.2, ls=":", zorder=1)
ax.axvline(ALPHA, color=SEC, lw=1, ls="--", zorder=1)
ax.text(ALPHA / 1.15, len(rows) - 0.45, "α = 0.05", ha="right", va="center", fontsize=10, color=SEC)
tr = mtrans.blended_transform_factory(ax.transAxes, ax.transData)
ax.text(1.03, len(rows) - 0.45, "p", transform=tr, fontsize=10, color=SEC, ha="left", va="center")
for (lab, v, c, ann), y in zip(rows, ys):
if c == ORANGE:
ax.annotate("", xy=(4e-5, y), xytext=(v, y),
arrowprops=dict(arrowstyle="-|>", color=c, lw=2, mutation_scale=14))
ax.plot(v, y, marker="|", ms=14, mew=2.5, color=c)
else:
ax.plot(v, y, "o", ms=9, color=c, zorder=3)
ax.text(1.03, y, ann, transform=tr, fontsize=10, color=c, ha="left", va="center",
weight="bold" if c == ORANGE else "normal")
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
for t, r in zip(ax.get_yticklabels(), rows):
t.set_color(r[2] if r[2] == ORANGE else TXT)
ax.set_xscale("log")
ax.set_xlim(3e-5, 1)
ax.set_ylim(-0.6, len(rows) - 0.1)
ax.set_xlabel("p-value (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8, which="major")
clean(ax)
ax.tick_params(axis="y", length=0)
fig.tight_layout(rect=(0, 0.13, 0.93, 0.94))
fig.suptitle("9th-challenge acquisition p-value: reported (orange) vs recomputed (blue)",
x=0.015, ha="left", fontsize=11, color=TXT)
fig.text(0.015, 0.075, f"Shaded band: below 1/1716 = {floor:.1e}, the smallest p a Fisher or permutation test can give.",
fontsize=9.5, color=SEC)
fig.text(0.015, 0.03, f"Dotted line: {bar_floor:.1e}, the smallest p a Barnard or Boschloo test can give (7 vs 6 animals).",
fontsize=9.5, color=SEC)
save(fig, path)
# ---------- Candidate B: null distribution of the Fisher test ----------
def design_B(path):
fig, ax = plt.subplots(figsize=(7.2, 3.6))
ks = list(null.keys())
vals = [null[k] for k in ks]
cols = [BLUE if k == 3 else "#c9c8c3" for k in ks]
ax.bar(ks, vals, color=cols, width=0.6, zorder=2)
for k, v in zip(ks, vals):
ax.text(k, v + 0.012, f"{round(v*286)}/286\n= {v:.3f}", ha="center", fontsize=10,
color=BLUE if k == 3 else SEC)
ax.annotate(f"Observed (3/7 vs 0/6)\none-tailed p = {p1:.3f}", xy=(3, null[3] + 0.115),
xytext=(2.55, 0.31), fontsize=10, color=BLUE,
arrowprops=dict(arrowstyle="-", color=BLUE, lw=1))
# The bars are point probabilities, not p-values, so the reported p-value is not drawn on this
# axis. It is stated as text: the one-tailed p is the sum of the bars at least as extreme as
# the observed result, which here is the 3-of-3 bar alone.
ax.text(-0.42, 0.70, "Reported: P < 0.0001", fontsize=10, color=ORANGE, va="top", weight="bold")
ax.text(-0.42, 0.655, "The one-tailed p is the sum of bars at least as\nextreme as observed: here the 3-of-3 bar alone,\n"
f"35/286 = {p1:.3f}, about 1,200 times the reported bound.",
fontsize=9.5, color=SEC, va="top")
ax.set_xticks(ks)
ax.set_xticklabels([f"{k} of 3" for k in ks])
ax.set_xlabel("Uninfected animals (3 in total) that fall in the vaccinated group of 7")
ax.set_ylabel("Probability if vaccine had no effect")
ax.set_ylim(0, 0.75)
ax.set_yticks([0, 0.1, 0.2, 0.3, 0.4, 0.5])
ax.grid(axis="y", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.set_title("Fisher's exact test on the 9th-challenge counts", fontsize=11, loc="left")
fig.tight_layout()
save(fig, path)
# ---------- helpers for text diagrams ----------
def box(ax, x, y, w, h, title, body, edge, tcolor, size=10.5):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.02",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.02, y + h - 0.05, title, fontsize=10, color=SEC, va="top", weight="bold")
ax.text(x + 0.02, y + h - 0.17, body, fontsize=size, color=tcolor, va="top", linespacing=1.45)
# ---------- Candidate C: text panels, test named vs value reported ----------
def design_C(path):
fig, ax = plt.subplots(figsize=(7.2, 4.0))
fig.subplots_adjust(0.01, 0.01, 0.99, 0.99)
ax.set_axis_off(); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
box(ax, 0.01, 0.56, 0.46, 0.42, "Methods and Fig. 2a legend",
"VE after the 9th challenge\n\"followed by Fisher\u2019s exact test\"\n\"one-tailed Fishers Exact Test\"",
"#b9b8b3", TXT, 10)
box(ax, 0.53, 0.56, 0.46, 0.42, "Results",
"4/7 vaccinated vs 6/6 control\ninfected by the 9th challenge\n\"VE = 43% (Fig. 2a; P < 0.0001)\"",
ORANGE, ORANGE, 10)
box(ax, 0.12, 0.02, 0.76, 0.34, "Recomputed: the stated test on the stated counts",
f"One-tailed Fisher, 3/7 vs 0/6 uninfected:\np = C(7,3)/C(13,3) = 35/286 = {p1:.3f} (two-tailed {p2:.3f})",
BLUE, BLUE, 10)
for x0, x1 in ((0.24, 0.36), (0.76, 0.64)):
ax.add_patch(FancyArrowPatch((x0, 0.555), (x1, 0.365), arrowstyle="-|>",
mutation_scale=14, color=SEC, lw=1.5))
ax.text(0.5, 0.46, "Orange (reported) vs blue (recomputed):\nmore than 1,000-fold apart",
ha="center", va="center", fontsize=9.5, color=SEC)
save(fig, path)
# ---------- Candidate D: calculation flow ----------
def design_D(path):
fig, ax = plt.subplots(figsize=(7.2, 2.7))
fig.subplots_adjust(0.01, 0.01, 0.99, 0.99)
ax.set_axis_off(); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
steps = [
("Counts", "uninfected after\n9 challenges\n(Results):\nvaccinated 3/7\ncontrol 0/6", TXT, "#b9b8b3"),
("Stated test", "one-tailed\nFisher exact\n(Methods and\nFig. 2a legend)", TXT, "#b9b8b3"),
("Recomputed", f"C(7,3)/C(13,3)\n= 35/286\n= {p1:.3f}", BLUE, BLUE),
("Reported", "P < 0.0001", ORANGE, ORANGE),
]
w = 0.215; gap = (0.98 - 4 * w) / 3
for i, (t, b, c, e) in enumerate(steps):
x = 0.01 + i * (w + gap)
box(ax, x, 0.24, w, 0.72, t, b, e, c, 10)
if i < 2:
ax.add_patch(FancyArrowPatch((x + w, 0.62), (x + w + gap, 0.62), arrowstyle="-|>",
mutation_scale=12, color=SEC, lw=1.5))
xn = 0.01 + 3 * w + 2.5 * gap
ax.text(xn, 0.61, "\u2260", ha="center", va="center", fontsize=20, color=ORANGE)
ax.text(0.5, 0.08, f"Recomputed {p1:.3f} vs reported < 0.0001: more than 1,000-fold apart", ha="center",
fontsize=10, color=SEC)
save(fig, path)
# ---------- Candidate E: ladder of attainable p-values ----------
def design_E(path):
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ks = list(ladder.keys())
ax.axhspan(3e-5, floor, color="#efeeea", zorder=0)
ax.axhline(ALPHA, color=SEC, lw=1, ls="--")
ax.text(7.85, ALPHA * 1.25, "α = 0.05", ha="right", va="bottom", fontsize=10, color=SEC)
ax.plot(ks, [ladder[k] for k in ks], "-", color="#c9c8c3", lw=2, zorder=1)
ax.plot(ks, [ladder[k] for k in ks], "o", ms=8, color="#9a9994", zorder=2,
label="Possible outcomes: k of 7 vaccinated uninfected")
ax.plot(3, p1, "o", ms=11, color=BLUE, zorder=3, label=f"Observed 3/7, recomputed p = {p1:.3f}")
ax.axhline(REPORTED_P, color=ORANGE, lw=2, label="Reported: P < 0.0001")
ax.text(0.8, 2.2e-4, f"Shaded: below the smallest p a Fisher or permutation\ntest can give (all 7 uninfected: 1/1716 = {floor:.1e})",
fontsize=9.5, color=SEC, va="center")
ax.set_yscale("log")
ax.set_ylim(3e-5, 1)
ax.set_xlim(0.7, 7.9)
ax.set_xticks(ks)
ax.set_xticklabels([f"{k}/7" for k in ks])
ax.set_xlabel("Vaccinated animals uninfected after 9 challenges (controls 0/6)")
ax.set_ylabel("One-tailed Fisher p (log scale)")
ax.grid(axis="y", color=GRID, lw=0.8)
for s_ in ("top", "right"):
ax.spines[s_].set_visible(False)
ax.legend(loc="upper right", frameon=False, fontsize=9.5)
ax.set_title("Every possible outcome of a 7-vs-6 study gives p above 0.0001", fontsize=11, loc="left")
fig.tight_layout()
save(fig, path)
design_A(os.path.join(CAND, "A_pvalue_axis"))
design_B(os.path.join(CAND, "B_null_distribution"))
design_C(os.path.join(CAND, "C_text_panels"))
design_D(os.path.join(CAND, "D_calculation_flow"))
design_E(os.path.join(CAND, "E_attainable_ladder"))
# Provisional choice, pending the user's review of candidates A-E: fig1 = A, fig2 = E.
design_A(os.path.join(HERE, "fig1"))
design_E(os.path.join(HERE, "fig2"))
print("Figures written to", HERE)
Abstract's verified-vaccination percentages and effect do not match its own counts or the Results
Nat Med (2023) · doi:10.1038/s41591-023-02670-4 · PMC10719106
Summary
The abstract of this cluster-randomized trial in rural Ghana reports that 36.6% (355 of 1,058) of verified participants in the cash-incentive arms had at least one COVID-19 vaccine dose, compared with 30.3% (439 of 1,544) in the placebo arm, a difference of 6.3% (95% CI 2.4, 10.2; P = 0.001). The counts printed in that sentence give 33.6% and 28.4%. Those are the rates the Results text and Table 2 report for the same verified sample, where the difference is 5.2% (95% CI 1.5, 8.7; P = 0.005). The abstract's 36.6%, 6.3% and 2.4–10.2 match, figure for figure, a spillover estimate that the Results give for a different subgroup. Verified vaccination is the trial's only registry-confirmed outcome, so the abstract overstates the size and precision of its headline objective result, though the effect's direction and statistical significance are unchanged.
What the paper reports
Abstract:
We also verified vaccination status of participants: in the cash treatment arm, 36.6% (355 of 1,058) of verified participants had at least one dose of the COVID-19 vaccine compared to 30.3% (439 of 1,544) for those in the placebo—a difference of 6.3% (95% CI: 2.4, 10.2; P = 0.001).
Results, "Combined low and high cash treatment effect", third paragraph:
In the cash treatment arm (both low and high cash), we verified that 33.6% of individuals had at least one dose of the COVID-19 vaccine as of April 2023 compared to 28.4% of those in the placebo arm (difference from placebo: 5.2%; 95% CI: 1.5, 8.7; P = 0.005).
In relative terms, this represents about an 18% increase over the non-treated placebo vaccination rates.
Results, "Verified vaccination status":
For our six-district sample, we verified that 29.1% of the 3,075 participants received a COVID-19 vaccination
We verified that 28.4% of individuals in the placebo arm were vaccinated (at least one dose)
Table 2, phase IV panel ("Phase IV: verified vaccination status of treated participants (30 April 2022)"; columns: total sample | placebo | health message | low cash | high cash):
Number of participants | 3075 | 1544 | 473 | 548 | 510
Actual vaccination (%) | 29.1 (45.4) | 28.4 (45.1) | 21.6 (41.2) | 40.7 (49.2) | 25.9 (43.8)
Results, "Spillover", second paragraph, which describes Extended Data Table 5 (participants who saw the placebo video, grouped by the treatment assigned to their village):
Participants in placebo-treated village clusters had an average vaccination rate of 26.8%.
Individuals receiving the placebo video in low cash treatment villages had a verified vaccination rate of 38.3% (difference from placebo: 11.5%; 95% CI: 5.2, 17.8; P < 0.001).
Placebo participants in the combined cash treatment also had a verified vaccination rate, 36.6%, that is significantly larger than the placebo treatment village participants (difference from placebo: 6.3%; 95% CI: 2.4, 10.2; P = 0.002).
What the paper's own numbers show
Method. I divided the abstract's numerators by its denominators. I then rebuilt each arm's vaccinated count from the phase IV panel of Table 2 (n × printed %) and checked that the cash-arm count and size equal the abstract's 355 and 1,058. I computed the cash-minus-placebo difference with a naive two-proportion Wald 95% interval and normal-approximation P value. This interval ignores village clustering and serves only to check the printed arithmetic. Finally I set the abstract's figures beside the spillover paragraph's figures. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Cash-arm size | Abstract; Table 2 phase IV | 1,058; low cash 548, high cash 510 | 548 + 510 = 1,058, the same sample |
| Cash-arm vaccinated | Abstract; Table 2 phase IV | 355; 40.7% of 548, 25.9% of 510 | 223.0 + 132.1 → 223 + 132 = 355, the same count |
| Placebo size and vaccinated | Abstract; Table 2 phase IV | 439 of 1,544; 28.4% of 1,544 | 1,544 × 28.4% = 438.5 (438 or 439 after rounding), consistent |
| Total verified sample | Table 2 phase IV | 3,075; 29.1% | 895/3,075 = 29.1% from the rebuilt counts, consistent |
| Cash-arm rate | Abstract | 36.6% (355 of 1,058) | 355/1,058 = 33.55% → 33.6% |
| Cash-arm rate | Results, combined cash effect | 33.6% | 33.6%, agrees |
| Placebo rate | Abstract | 30.3% (439 of 1,544) | 439/1,544 = 28.43% → 28.4% |
| Placebo rate | Results; Table 2 | 28.4% | 28.4%, agrees |
| Counts the abstract's percentages would need | Abstract | 36.6% of 1,058; 30.3% of 1,544 | 387 (not 355); 468 (not 439) |
| Cash minus placebo | Abstract | 6.3% (95% CI 2.4, 10.2; P = 0.001) | 5.12 pp, naive 95% CI (1.5, 8.7), P = 0.0057 |
| Cash minus placebo | Results, combined cash effect | 5.2% (95% CI 1.5, 8.7; P = 0.005) | 5.12 pp (33.6 − 28.4 = 5.2 from the rounded rates); same interval |
| Relative increase | Results | "about 18%" | 5.12/28.43 = 18.0%; the abstract's figures imply 6.3/30.3 = 20.8% |
| Placebo-video participants in cash villages | Results, Spillover | 36.6%; difference 6.3 (95% CI 2.4, 10.2; P = 0.002) | Identical to the abstract's 36.6%, 6.3 and 2.4–10.2; the P values differ (0.002 vs 0.001) |
| Spillover reference rate | Results, Spillover | placebo villages 26.8% | 38.3 − 11.5 = 26.8 agrees for the low-cash line; 36.6 − 6.3 = 30.3, not 26.8 (36.6 − 26.8 = 9.8) |
The abstract's counts and denominators agree with every other part of the paper. Its percentages, difference, interval and P value do not agree with those counts. The abstract's placebo rate, 30.3%, is exactly 36.6 − 6.3, the spillover paragraph's rate minus its difference.
Figure
Figure 1. The left panel gives the abstract's verified-vaccination figures and the right panel the Results text and Table 2 (phase IV) figures, both as printed (orange). Both panels describe the same 1,058 cash-arm and 1,544 placebo participants. The blue strip shows the rates and difference recomputed from the abstract's own counts, 355/1,058 and 439/1,544. These agree with the Results and Table 2, not with the abstract. The note under the strip records where the abstract's 36.6%, 6.3% and 2.4–10.2 appear in the paper: in the spillover paragraph, for a different subgroup.
Figure 2. Cash-minus-placebo difference in verified vaccination (percentage points, 95% CI) on one axis, with zero as a thin reference line. Orange marks the values printed in the abstract and in the Results. Blue marks the value recomputed from the abstract's counts with a naive, unclustered two-proportion interval. Gray marks the spillover-subgroup estimate printed in the Results. The recomputed value and interval match the Results. The abstract's estimate and interval are the same as the spillover subgroup's.
Why it matters
- This is the headline objective outcome. Intentions and reported vaccinations are self-reports. Verified vaccination comes from District Health Office records and is the only registry-confirmed measure. The abstract presents it as the confirming result.
- Size of the effect. The abstract gives 6.3 percentage points. The paper's own counts give 5.1 (5.2 in the Results), so the abstract overstates the absolute effect by about 1.1 pp and implies a relative increase of about 21% rather than the 18% the Results report. It also gives a placebo baseline of 30.3% instead of 28.4%.
- Precision and significance. The abstract gives a lower confidence bound of 2.4 pp and P = 0.001. The analysis of the same sample gives 1.5 pp and P = 0.005. The effect remains positive and its interval still excludes zero, so the qualitative conclusion that cash incentives increased verified uptake does not change.
- Comparison with self-report. The Results give a 3.7-point cash effect on reported vaccination (the abstract gives 3.5). Against that, the abstract's verified effect looks 1.70 times as large (6.3/3.7). With the Results figure the ratio is 1.41 (5.2/3.7).
- Downstream use. Abstracts are what evidence summaries, policy briefs and meta-analyses usually extract. A reader who takes the abstract's percentages together with its counts gets an inconsistent 2×2 table.
A side note for a statistician: the Results interval (1.5, 8.7) is the same as a naive interval that ignores clustering, although the Results report an intracluster correlation of 0.64 for verified vaccination. This write-up compares like with like and makes no claim about which interval is appropriate. The inconsistency described here does not depend on that question.
Explanations considered
- A different population or subsample. Ruled out for the counts. 1,058 = 548 + 510, the two cash arms in Table 2 phase IV; 1,544 is the placebo arm there; and 223 + 132 = 355 rebuilds the cash count from Table 2's percentages. The numerators and denominators therefore belong to the main verified treated sample. No subsample is described that has those sizes and the abstract's percentages.
- Adjusted rather than crude estimates. Unlikely. The abstract gives the percentages with raw counts in parentheses ("36.6% (355 of 1,058)"), the form of a crude proportion. No adjustment turns 355/1,058 into 36.6%. The Results report the crude comparison for this sample as 33.6% vs 28.4%, and the model-based results (Table 3) are reported as odds ratios, not as percentages.
- A different time point or outcome definition. Not supported. Table 2 dates phase IV to 30 April 2022, and the Results "Verified vaccination status" section uses the end of April 2022. The combined-cash sentence says "as of April 2023", which does not match the 30 April 2022 cut-off in Table 2. Its figures, however, are the Table 2 figures. No other cut-off is given for the abstract.
- Rounding. Ruled out. 355/1,058 = 33.55% and 439/1,544 = 28.43%. Reaching 36.6% would take 387 vaccinated cash participants, and reaching 30.3% would take 468 vaccinated placebo participants.
- The abstract quoting the spillover analysis. This matches the abstract's percentage, difference and interval exactly (36.6%, 6.3, 2.4–10.2), but the P value differs (0.001 vs 0.002), and the spillover analysis concerns placebo-video participants in cash villages, not the cash arm. The spillover paragraph is itself not internally consistent: 36.6 − 6.3 = 30.3, not its stated reference rate of 26.8%. So the paper does not settle where the abstract's figures came from. Whatever their source, they do not describe the 355/1,058 vs 439/1,544 comparison they are attached to.
- A conversion or archiving artifact. Ruled out. The publisher's HTML version of record, the PMC web page, the Europe PMC full-text XML and the PMC archive all carry the same numbers.
- Already corrected. No. The only correction notice, an Author Correction (doi:10.1038/s41591-023-02772-z, 22 December 2023), replaces a wrong link to the protocol in the Methods and does not touch these figures.
Status
As of 2026-10-23:
- Version of record. The publisher's page (nature.com/articles/s41591-023-02670-4, marked "This article has been updated") has the same abstract, Results and spillover text quoted above. The PMC and Europe PMC full texts agree with it word for word.
- Corrections and notices. Crossref and Europe PMC list one Author Correction (doi:10.1038/s41591-023-02772-z; PMC10957464), dated 22 December 2023. Its full text concerns only the protocol link. No retraction or expression of concern is listed.
- Supplementary material. The Supplementary Information (98 pages: Supplementary Tables S1–S5 and questionnaires) gives no verified-vaccination figures of 30.3%, 36.6%, 33.6%, 1,544 or 1,058. Extended Data Table 5 is an image in the archived text and was not read from the image. Its contents are taken here from the Results paragraph that describes it.
- Replication data. A separate check tabulated the replication file the paper deposits on Harvard Dataverse (doi:10.7910/DVN/X5MBHP; the 3,075 rows with a non-missing verified-vaccination status, grouped by individual treatment). It found placebo 439/1,544 = 28.43%, low cash 223/548 = 40.69%, high cash 132/510 = 25.88% and cash combined 355/1,058 = 33.55%, which agrees with the Results and Table 2. No grouping tried gave 30.3% with n = 1,544 or 36.6% with n = 1,058. In the same file, placebo-video participants in cash villages were 196/599 = 32.7% vaccinated and those in placebo villages 184/686 = 26.8%. The first of these differs from the 36.6% in the spillover paragraph.
- PubPeer. The PubPeer search API and site returned a human-verification challenge to automated requests, so a direct search could not be completed. A web search for the article title on PubPeer returned no PubPeer thread.
- Sources. The article as archived in PMC (PMC10719106); the publisher's HTML; Crossref and Europe PMC notice records; the Author Correction (PMC10957464); the Supplementary Information; Harvard Dataverse doi:10.7910/DVN/X5MBHP.
Suggested correction
In the abstract, change the verified-vaccination sentence so that it matches its counts and the Results, for example: "in the cash treatment arm, 33.6% (355 of 1,058) of verified participants had at least one dose of the COVID-19 vaccine compared to 28.4% (439 of 1,544) for those in the placebo—a difference of 5.2% (95% CI: 1.5, 8.7; P = 0.005)." The authors may also wish to:
- Check the spillover paragraph's combined-cash figures (36.6%; difference 6.3%; 95% CI 2.4, 10.2; P = 0.002) against Extended Data Table 5 and the replication data, since 36.6 − 6.3 does not equal the stated reference rate of 26.8%.
- Change "as of April 2023" in the combined-cash sentence to the 30 April 2022 cut-off used in Table 2, if that is the intended date.
Recomputation script (python recompute.py.txt)
"""Recompute the verified-vaccination figures of PMC10719106
(Nat Med 2023, doi:10.1038/s41591-023-02670-4).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
All inputs are copied from the article as archived in PubMed Central:
Abstract: verified-vaccination sentence (percentages, counts, difference, CI, P)
Table 2, phase IV (verified vaccination status of treated participants, 30 April 2022):
number of participants and "Actual vaccination (%)" by arm
Results > Combined low and high cash treatment effect (33.6% vs 28.4%, 5.2, CI 1.5-8.7)
Results > Spillover, paragraph on Extended Data Table 5 (26.8%, 38.3%/11.5, 36.6%/6.3)
The naive confidence interval below is an unadjusted two-proportion Wald interval; it
ignores village clustering and is used only to check the arithmetic of the printed values.
"""
import math
import os
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
from scipy.stats import norm # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
# Abstract
ABS = dict(cash_pct=36.6, cash_k=355, cash_n=1058, plac_pct=30.3, plac_k=439, plac_n=1544,
diff=6.3, lo=2.4, hi=10.2, p="0.001")
# Results > Combined low and high cash treatment effect
RES = dict(cash_pct=33.6, plac_pct=28.4, diff=5.2, lo=1.5, hi=8.7, p="0.005", rel="about 18%")
# Table 2, phase IV
T2_N = dict(placebo=1544, health=473, low=548, high=510)
T2_PCT = dict(placebo=28.4, health=21.6, low=40.7, high=25.9)
T2_TOTAL_N, T2_TOTAL_PCT = 3075, 29.1
# Results > Spillover (Extended Data Table 5, panel B, as described in the text)
SPILL = dict(placebo_village=26.8, lowcash_placebo=38.3, lowcash_diff=11.5,
highcash_placebo=26.3, highcash_diff=0.5,
cash_placebo=36.6, cash_diff=6.3, lo=2.4, hi=10.2, p="0.002")
# Results > Combined cash effect on reported (self-reported) vaccination
SELF_DIFF = 3.7
def pct(k, n):
return 100.0 * k / n
def wald(k1, n1, k0, n0):
p1, p0 = k1 / n1, k0 / n0
d = p1 - p0
se = math.sqrt(p1 * (1 - p1) / n1 + p0 * (1 - p0) / n0)
z = norm.ppf(0.975)
return 100 * d, 100 * (d - z * se), 100 * (d + z * se), 2 * norm.sf(abs(d) / se)
def main():
print("== 1. Table 2 phase IV: rebuild the vaccinated counts from n and %")
k = {a: round(T2_N[a] * T2_PCT[a] / 100) for a in T2_N}
for a in T2_N:
print(f" {a:8s} n={T2_N[a]:5d} {T2_PCT[a]:.1f}% -> {T2_N[a] * T2_PCT[a] / 100:.1f} "
f"-> {k[a]} vaccinated")
tot_n, tot_k = sum(T2_N.values()), sum(k.values())
print(f" total n={tot_n} (Table 2: {T2_TOTAL_N}); vaccinated {tot_k} -> "
f"{pct(tot_k, tot_n):.1f}% (Table 2: {T2_TOTAL_PCT}%)")
cash_n = T2_N["low"] + T2_N["high"]
cash_k = k["low"] + k["high"]
print(f" cash arms combined: n = {T2_N['low']} + {T2_N['high']} = {cash_n} "
f"(abstract: {ABS['cash_n']}); vaccinated = {k['low']} + {k['high']} = {cash_k} "
f"(abstract: {ABS['cash_k']})")
print(f" placebo: n = {T2_N['placebo']} (abstract: {ABS['plac_n']}); "
f"vaccinated rebuilt = {k['placebo']} (abstract: {ABS['plac_k']}; "
f"{T2_N['placebo']}*28.35% = {T2_N['placebo'] * .2835:.1f} to "
f"{T2_N['placebo']}*28.45% = {T2_N['placebo'] * .2845:.1f}, so 438 or 439)")
print("\n== 2. Abstract: percentages implied by the abstract's own counts")
c = pct(ABS["cash_k"], ABS["cash_n"])
p = pct(ABS["plac_k"], ABS["plac_n"])
print(f" cash {ABS['cash_k']}/{ABS['cash_n']} = {c:.2f}% -> {c:.1f}% "
f"(abstract prints {ABS['cash_pct']}%; Results prints {RES['cash_pct']}%)")
print(f" placebo {ABS['plac_k']}/{ABS['plac_n']} = {p:.2f}% -> {p:.1f}% "
f"(abstract prints {ABS['plac_pct']}%; Results/Table 2 print {RES['plac_pct']}%)")
print(f" counts the printed percentages would need: {ABS['cash_pct']}% of {ABS['cash_n']} = "
f"{ABS['cash_pct'] / 100 * ABS['cash_n']:.1f}; {ABS['plac_pct']}% of {ABS['plac_n']} = "
f"{ABS['plac_pct'] / 100 * ABS['plac_n']:.1f}")
# interval of counts compatible with each printed percentage after rounding to 0.1
for lab, pc, n in (("cash", ABS["cash_pct"], ABS["cash_n"]),
("placebo", ABS["plac_pct"], ABS["plac_n"])):
lo_k = math.ceil((pc - 0.05) / 100 * n)
hi_k = math.floor((pc + 0.05) / 100 * n)
print(f" {lab}: counts that round to {pc}% of {n}: {lo_k} to {hi_k}")
print("\n== 3. Difference, naive two-proportion CI and P from the counts")
d, lo, hi, pv = wald(ABS["cash_k"], ABS["cash_n"], ABS["plac_k"], ABS["plac_n"])
print(f" difference = {d:.2f} pp, naive 95% CI ({lo:.1f}, {hi:.1f}), P = {pv:.4f}")
print(f" abstract: {ABS['diff']} ({ABS['lo']}, {ABS['hi']}), P = {ABS['p']}")
print(f" Results: {RES['diff']} ({RES['lo']}, {RES['hi']}), P = {RES['p']}")
print(f" difference of the Results' rounded rates: {RES['cash_pct']} - {RES['plac_pct']} = "
f"{RES['cash_pct'] - RES['plac_pct']:.1f}")
print(f" difference of the abstract's rates: {ABS['cash_pct']} - {ABS['plac_pct']} = "
f"{ABS['cash_pct'] - ABS['plac_pct']:.1f}")
print(f" overstatement in the abstract: {ABS['diff']} - {RES['diff']} = "
f"{ABS['diff'] - RES['diff']:.1f} pp (vs recomputed: {ABS['diff'] - d:.2f} pp)")
print("\n== 4. Relative effect")
print(f" from the counts: {d:.2f}/{p:.2f} = {100 * d / p:.1f}% "
f"(Results: {RES['rel']}; {RES['diff']}/{RES['plac_pct']} = "
f"{100 * RES['diff'] / RES['plac_pct']:.1f}%)")
print(f" implied by the abstract: {ABS['diff']}/{ABS['plac_pct']} = "
f"{100 * ABS['diff'] / ABS['plac_pct']:.1f}%")
print(f" ratio of verified to self-reported effect: abstract {ABS['diff']}/{SELF_DIFF} = "
f"{ABS['diff'] / SELF_DIFF:.2f}; Results {RES['diff']}/{SELF_DIFF} = "
f"{RES['diff'] / SELF_DIFF:.2f}")
print("\n== 5. Spillover paragraph (Extended Data Table 5 as described in Results)")
print(f" abstract figures vs spillover figures: 36.6/6.3/(2.4, 10.2) vs "
f"{SPILL['cash_placebo']}/{SPILL['cash_diff']}/({SPILL['lo']}, {SPILL['hi']}); "
f"P {ABS['p']} vs {SPILL['p']}")
print(f" low-cash placebo check: {SPILL['lowcash_placebo']} - {SPILL['lowcash_diff']} = "
f"{SPILL['lowcash_placebo'] - SPILL['lowcash_diff']:.1f} "
f"(stated placebo-village rate {SPILL['placebo_village']})")
print(f" combined-cash placebo check: {SPILL['cash_placebo']} - {SPILL['cash_diff']} = "
f"{SPILL['cash_placebo'] - SPILL['cash_diff']:.1f} "
f"(stated placebo-village rate {SPILL['placebo_village']}; "
f"{SPILL['cash_placebo']} - {SPILL['placebo_village']} = "
f"{SPILL['cash_placebo'] - SPILL['placebo_village']:.1f})")
print(f" abstract placebo rate {ABS['plac_pct']} equals "
f"{SPILL['cash_placebo']} - {SPILL['cash_diff']} = "
f"{SPILL['cash_placebo'] - SPILL['cash_diff']:.1f}")
print(f" combined-cash placebo rate must lie between the two cash-village placebo rates: "
f"{SPILL['highcash_placebo']} <= {SPILL['cash_placebo']} <= {SPILL['lowcash_placebo']}")
make_fig1(c, p, d, lo, hi)
make_fig2(d, lo, hi)
# ---------------------------------------------------------------- figures
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT, "xtick.color": TXT,
"ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "font.family": "DejaVu Sans",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
def panel(ax, x, y, w, h, title, lines, edge, wrap=36):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.008,rounding_size=0.015",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.015, y + h - 0.025, title, ha="left", va="top", fontsize=11, weight="bold",
color=TXT)
yy = y + h - 0.10
for txt, col in lines:
for ln in textwrap.wrap(txt, wrap):
ax.text(x + 0.015, yy, ln, ha="left", va="top", fontsize=10, color=col)
yy -= 0.055
yy -= 0.02
def make_fig1(c, p, d, lo, hi):
fig = plt.figure(figsize=(7.2, 4.9))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
# Left: abstract (reported, orange)
panel(ax, 0.015, 0.47, 0.42, 0.50, "Abstract", [
("Cash: 36.6% (355 of 1,058)", ORANGE),
("Placebo: 30.3% (439 of 1,544)", ORANGE),
("Difference 6.3% (95% CI 2.4, 10.2; P = 0.001)", ORANGE),
], ORANGE)
# Right: Results / Table 2 (same sample)
panel(ax, 0.565, 0.47, 0.42, 0.50, "Results text and Table 2", [
("Table 2, phase IV n: placebo 1,544; cash 548 + 510 = 1,058", SEC),
("Cash 33.6% vs placebo 28.4%", ORANGE),
("Difference 5.2% (95% CI 1.5, 8.7; P = 0.005)", ORANGE),
], SEC)
# Connector between the two panels
ax.annotate("", xy=(0.565, 0.70), xytext=(0.437, 0.70),
arrowprops=dict(arrowstyle="<->", color=TXT, lw=1.5))
ax.text(0.501, 0.73, "same n,\nsame\noutcome", ha="center", va="bottom", fontsize=10,
color=SEC)
ax.text(0.501, 0.67, "different\n% and\nestimate", ha="center", va="top", fontsize=10,
color=TXT, weight="bold")
# Recomputed strip (blue)
ax.add_patch(FancyBboxPatch((0.015, 0.20), 0.97, 0.20,
boxstyle="round,pad=0.008,rounding_size=0.015",
fc="white", ec=BLUE, lw=1.5))
ax.text(0.03, 0.375, "Recomputed from the abstract's own counts", ha="left", va="top",
fontsize=11, weight="bold")
ax.text(0.03, 0.305, f"355 / 1,058 = {c:.1f}% 439 / 1,544 = {p:.1f}%", ha="left",
va="top", fontsize=10.5, color=BLUE)
ax.text(0.03, 0.25, f"Difference {d:.1f} pp (naive unclustered 95% CI {lo:.1f}, {hi:.1f})",
ha="left", va="top", fontsize=10.5, color=BLUE)
note = ("The abstract's 36.6%, 6.3% and CI 2.4, 10.2 also appear in Results > Spillover, "
"for placebo-video participants in cash villages (a different subgroup; P = 0.002 "
"there). Orange: values as printed in the paper. Blue: recomputed.")
for i, ln in enumerate(textwrap.wrap(note, 92)):
ax.text(0.015, 0.15 - 0.05 * i, ln, ha="left", va="top", fontsize=10, color=SEC)
save(fig, "fig1")
def make_fig2(d, lo, hi):
rows = [
("Abstract", ABS["diff"], ABS["lo"], ABS["hi"], ORANGE, "o"),
("Results text", RES["diff"], RES["lo"], RES["hi"], ORANGE, "s"),
("Recomputed, 355/1,058 vs 439/1,544", d, lo, hi, BLUE, "D"),
("Spillover subgroup (Results)", SPILL["cash_diff"], SPILL["lo"],
SPILL["hi"], SEC, "^"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.4))
for i, (lab, m, l, h, col, mk) in enumerate(rows):
y = len(rows) - 1 - i
ax.plot([l, h], [y, y], color=col, lw=2, solid_capstyle="butt")
ax.plot([m], [y], marker=mk, color=col, ms=9, ls="none")
ax.text(h + 0.25, y, f"{m:.1f} ({l:.1f}, {h:.1f})", va="center", fontsize=10, color=col)
ax.axvline(0, color=SEC, lw=0.8)
ax.set_yticks(range(len(rows)))
ax.set_yticklabels([r[0] for r in rows][::-1])
ax.set_xlim(-0.5, 14)
ax.set_ylim(-0.6, len(rows) - 0.4)
ax.set_xlabel("Cash minus placebo, verified vaccination (pp, 95% CI)")
ax.set_title("Orange: as printed in the paper. Blue: recomputed. Gray: a different subgroup.",
loc="right", fontsize=10, color=SEC)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
fig.tight_layout()
save(fig, "fig2")
if __name__ == "__main__":
main()
Abstract says VEGFR1 is downregulated in hyperlipidemic mice; the Results and Discussion report an increase
Nat Med (2012) · doi:10.1038/nm.2759 · PMC3959651
Summary
The Abstract states that "we observed VEGFR1 downregulation in hyperlipidemic mice", but the paper's only mouse experiments (Apoe-null and Ldlr-null versus wild-type C57BL/6 mice, Fig. 3c–e) are reported in the Results as showing "increased endothelial-specific Vegfr1 expression", and the Discussion says Ldlr-null endothelial cells "express elevated amounts of Vegfr1". Of the eight main-text statements that give VEGFR1's direction when ApoB-lipoprotein levels are raised or lowered, seven have VEGFR1 moving with the lipoproteins, as the paper's model requires; the Abstract's mouse clause is the only exception. The data and the main conclusion are unaffected, but the Abstract gives the paper's only mammalian in vivo result the wrong sign, which, read alone, implies that hyperlipidemia lowers an anti-angiogenic decoy receptor. The same wording is in the version of record and the PubMed abstract, and no correction has been issued.
What the paper reports
The Abstract sentence:
Mechanistically, we identified downregulation of vascular endothelial growth factor receptor 1 (VEGFR1), which acts as a decoy receptor for VEGF, as a key mediator of the endothelial response to lipoproteins, and we observed VEGFR1 downregulation in hyperlipidemic mice.
Abstract, second-to-last sentence
The same mouse experiments as described in the body:
In addition, hyperlipidemic Apoe- or Ldlr-null mice (reviewed in refs. 27,28) also showed increased endothelial-specific Vegfr1 expression compared to control WT mice (Fig. 3c–e), confirming conservation of this pathway in higher vertebrates.
Results, VEGFR1 paragraph (last sentence)
The finding that endothelial cells of Ldlr-null mice express elevated amounts of Vegfr1 highlights the potential role of alternative lipoprotein receptors in mediating the endothelial response.
Discussion, paragraph 2
(e) Semiquantitative RT-PCR measurement of Vegfr1 mRNA expression in aortic arches dissected from WT (n = 5) and Apoe-null (n = 5) mice. *P = 4 × 10-14 by z test.
Fig. 3 legend, panel e
All analyses were performed using mice at 8–38 weeks of age fed with a normal chow diet.
Online Methods, "Mice and immunofluorescence"
The zebrafish and cell-culture results in the same Results paragraph:
VEGFR1 protein concentrations were also strongly downregulated in stl mutants but were substantially upregulated in apoCII morphants, in which endothelial cells face high concentrations of ApoB lipoproteins (Fig. 3b). We found similar effects on VEGFR1 levels in endothelial cells in vitro. Treatment of cultured human umbilical vein endothelial cells (HUVECs) with LDL elicited an increase in VEGFR1 but not VEGFR2 concentrations (Fig. 3b).
Results, VEGFR1 paragraph
Every quotation was checked character for character against the article as archived in PubMed Central (src/show.py PMC3959651 --grep; recompute.py.txt repeats the check). The Abstract, Results and Discussion sentences and the Fig. 3e P value were also checked against a text dump of the typeset Nature Medicine PDF (see Status).
What the paper's own numbers show
Method. There is nothing to recompute numerically, because the error is in the stated direction of a change. The Fig. 3e group means are shown only as plotted bars, and the text states their direction explicitly. recompute.py.txt therefore holds a hand-curated list of every main-text statement that gives a direction for VEGFR1 (protein, mRNA or reporter) under a condition in which ApoB-lipoprotein levels are raised or lowered relative to a control. To build it, every sentence of the Abstract, Results and Discussion was read. Each statement is copied verbatim and labelled with two signs: the lipoprotein direction of the condition (+1 higher, −1 lower) and the VEGFR1 direction stated (+1 up, −1 down). The product of the two signs is +1 when VEGFR1 moves with lipoproteins and −1 when it moves against them. The script checks each excerpt against the archived text, prints the table below and draws both figures.
| Quantity | Location | Reported | Recomputed from the paper's own text |
|---|---|---|---|
| VEGFR1 in hyperlipidemic mice | Abstract | "downregulation" | Lipoproteins up, VEGFR1 down: product −1, the only one |
| Endothelial Vegfr1, Apoe- or Ldlr-null vs WT mice | Results; Fig. 3c–e | "increased" | Lipoproteins up, VEGFR1 up: +1 |
| Vegfr1, Ldlr-null endothelium | Discussion, paragraph 2 | "elevated amounts" | Lipoproteins up, VEGFR1 up: +1 |
| Vegfr1 mRNA, aortic arch, WT (n = 5) vs Apoe-null (n = 5) | Fig. 3e legend | P = 4 × 10⁻¹⁴ by z test; direction stated in Results as an increase | Not re-tested (see note below) |
| vegfr1 mRNA, mtp morphants | Results; Fig. 3a | "lower" | Lipoproteins down, VEGFR1 down: +1 |
| VEGFR1 protein, stl mutants | Results; Fig. 3b | "strongly downregulated" | Lipoproteins down, VEGFR1 down: +1 |
| VEGFR1 protein, apoCII morphants | Results; Fig. 3b | "substantially upregulated" | Lipoproteins up, VEGFR1 up: +1 |
| VEGFR1, HUVECs + LDL | Results; Fig. 3b | "an increase" | Lipoproteins up, VEGFR1 up: +1 |
| yfp (vegfr1-promoter reporter) mRNA, mtp knockdown | Results; Fig. 3i,j | "decrease" (P = 0.0248) | Lipoproteins down, VEGFR1 down: +1 |
| Condition statements in which VEGFR1 moves with lipoproteins | Main text | — | 7 of 8 |
| Mouse statements: Abstract vs Results and Discussion | Main text | — | Abstract down; Results up; Discussion up. They disagree. |
The first clause of the same Abstract sentence ("we identified downregulation of ... VEGFR1 ... as a key mediator of the endothelial response to lipoproteins") is consistent with the body. It describes the lipoprotein-depleted stl and mtp-morphant zebrafish, in which VEGFR1 falls. The second clause makes a separate claim about the mice, and that claim has the opposite sign to the Results.
Related observation (not part of the main finding). One concluding phrase in the Results also states the relation with the opposite sign to the result it summarizes. After reporting that "mtp knockdown resulted in a significant (P=0.0248) decrease in YFP expression (Fig. 3i, j)", the sentence goes on: "providing evidence that ApoB-containing lipoproteins inhibit vegfr1 expression at the transcriptional level". Removing lipoproteins lowered the reporter, so the result shows lipoproteins promoting vegfr1 transcription, not inhibiting it. This wording is also in the typeset PDF. The rest of the paragraph makes clear what is meant, so this phrase is less prominent than the Abstract sentence, but a correction could fix both.
Not evaluated. The Fig. 3e statistics (P = 4 × 10⁻¹⁴ from a two-tailed z test of the ratio of two group means against 1, n = 5 per group, as described in the Online Methods) are not part of this finding and were not re-tested. The group means and standard errors are not given in the text.
Figure
Figure 1. The left panel (orange) shows the Abstract clause that states VEGFR1 goes down in hyperlipidemic mice. The right panel (blue) shows what the Results and Discussion report for the same mice: Apoe-null and Ldlr-null mice compared with WT C57BL/6 on a chow diet (Fig. 3c,d immunostaining; Fig. 3e aortic mRNA, n = 5 per group). The connector marks the conflict: one set of experiments, two opposite directions. All text is quoted from the paper. None of the paper's figures or data are reproduced.
Figure 2. Each marker is one of the eight main-text statements that give a direction for VEGFR1 under a raised or lowered ApoB-lipoprotein level (inclusion rule under Method). The horizontal position gives the lipoprotein level of the condition relative to its control. The vertical position gives the VEGFR1 direction stated. Squares are mouse statements; circles are zebrafish or HUVEC statements. The seven blue statements lie in the two "VEGFR1 with lipoproteins" quadrants, as the paper's model requires. The orange Abstract statement is the only one in a "VEGFR1 against lipoproteins" quadrant. The other two mouse statements, for the same animals, lie in the opposite quadrant. Positions show direction only, not magnitude. Directions and locations come from recompute.py.txt.
Why it matters
- The Abstract gives the mammalian result the wrong sign. The mouse experiment is the paper's only in vivo test of the pathway outside zebrafish, and the Results present it as "confirming conservation of this pathway in higher vertebrates". The Abstract states that result in reverse.
- Read alone, the Abstract implies the opposite biology. VEGFR1 is described as a decoy receptor that restrains angiogenesis. If hyperlipidemia lowered it, hyperlipidemia would favour VEGF signalling. The paper argues the reverse: excess ApoB lipoproteins raise VEGFR1 and inhibit angiogenesis (apoCII morphants, LDL-treated HUVECs, Discussion). Its closing link to pathological hyperlipidemia ("seems to be reactivated during pathological conditions of hyperlipidemia") depends on the increase.
- The error reaches readers of the abstract. The same sentence is in the PubMed and Europe PMC record (PMID 22581286), which indexing services, text-mining pipelines and readers without journal access rely on. Crossref lists 114 citing works.
- The data and the main conclusion are not affected. No reported value, test or P value changes. Fig. 3c–e, its legend, the Results and the Discussion all give the increase, and so does every zebrafish and HUVEC experiment. The error is in one clause of the summary, not in the data or analysis.
Explanations considered
- Archiving or conversion artifact. Ruled out. The publisher's abstract page, the typeset PDF and the PubMed/Europe PMC abstract all carry the same wording as the PMC author manuscript. The conflict is in a word ("downregulation" against "increased"), not in a symbol or sign that a conversion could alter.
- A different mouse model or treatment. Ruled out. The only mouse experiments are Apoe-null and Ldlr-null mice versus WT C57BL/6 on a normal chow diet (Online Methods; Fig. 3c–e). No mouse experiment lowers lipids (for example with a statin or loss of Mtp) in a way that could produce a decrease.
- "Downregulation" describes the mechanism, not the mice. Not an explanation. The first clause of the sentence does use "downregulation" for the lipoprotein-depleted state. But the second clause, "we observed VEGFR1 downregulation in hyperlipidemic mice", is a direct observational claim about the mice, and the mice were hyperlipidemic, not depleted.
- A different measure (protein vs mRNA, endothelium vs whole tissue). Ruled out. Immunostaining of aortic endothelium (Fig. 3c,d), aortic-arch mRNA (Fig. 3e), the Results ("increased endothelial-specific Vegfr1 expression") and the Discussion ("elevated amounts of Vegfr1") all give an increase. The paper reports no mouse measure that falls. In the publisher's Fig. 3e bar chart, read by eye, relative Vegfr1 is about 1.0 for WT and about 1.7 for Apoe-null. This approximate reading is not used in any calculation or figure here.
- The supplement resolves it. It does not. The Supplementary Text and Figures (Supplementary Figs. 1–7) contain no mouse data. The mouse data that settle the direction are in main Fig. 3c–e.
- The paper's model predicts a decrease. It does not. In every zebrafish and HUVEC condition VEGFR1 moves the same way as ApoB lipoproteins (7 of 8 condition statements; Fig. 2), so hyperlipidemia should raise VEGFR1, as the Results say.
None of these explanations reconciles the Abstract clause with the paper's mouse results.
Status
As of 2026-10-23:
- Version of record. The abstract on the publisher's page (doi.org redirects to nature.com/articles/nm.2759) contains "we observed VEGFR1 downregulation in hyperlipidemic mice" word for word. The page shows no change history and no correction notice. The full text is paywalled there, so a copy of the typeset Nature Medicine PDF (vol. 18, no. 6, June 2012, p. 967 onward), posted on a university lab website, was converted to text and searched. It contains the Abstract clause, the Results and Discussion sentences, the Fig. 3e P value and the "inhibit vegfr1 expression" phrase exactly as quoted. The PMC copy is the author manuscript (NIHMS475364, pmc-version 1) and has the same wording at all of these places.
- Supplements. Nature's "Supplementary Text and Figures" PDF (Supplementary Figs. 1–7, 8 pages) was searched. It has no matches for Apoe, Ldlr, hyperlip or mice, and VEGFR1 appears only in the title. The PMC manuscript's own supplement (a .pptx file) was not opened.
- Corrections.
src/known.pyreturns no Crossref updates and no PMC or Europe PMC notices. The Crossref record has no update-to, updated-by or relation entries. The Crossref labs API, which carries the Retraction Watch data, lists no updates. The Europe PMC record (PMID 22581286) has no comment/correction list. The Nature article page has no correction link. - PubPeer and the web. The PubPeer site and API could not be reached (HTTP 403 or empty response), so a PubPeer comment cannot be ruled out. Web searches for the title or DOI together with correction, erratum, PubPeer or Retraction Watch found only the article itself and catalogue listings.
- Sources: PMC3959651 (via
src/show.py);src/known.py 10.1038/nm.2759; https://doi.org/10.1038/nm.2759 (publisher page and abstract); the typeset article PDF and Nature's supplementary PDF (41591_2012_BFnm2759_MOESM12_ESM.pdf); https://api.crossref.org/works/10.1038/nm.2759; Europe PMC REST record for PMID 22581286.
Suggested correction
Change the direction in the Abstract so that it matches the Results, Fig. 3c–e and the Discussion:
- Abstract: "...as a key mediator of the endothelial response to lipoproteins, and we observed VEGFR1 upregulation in hyperlipidemic mice."
For consistency, the concluding phrase in the Results paragraph on Fig. 3i,j could also be changed, for example to "providing evidence that ApoB-containing lipoproteins promote vegfr1 expression at the transcriptional level" (or "regulate", as in the Discussion). No change to the data, figures, statistics or conclusions is needed.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC3959651 (Nat Med 2012, doi:10.1038/nm.2759)
# Abstract vs Results / Discussion: which way does VEGFR1 change in hyperlipidemic mice?
#
# Run: python recompute.py.txt
# Prints every number and direction used in writeup.md and regenerates fig1.{svg,png} and
# fig2.{svg,png} in the folder that holds this file. Needs matplotlib.
#
# The finding is about the direction of a change stated in words; there is no numeric value
# to recompute. The script therefore:
# 1. holds a hand-curated list of every statement in the main text (Abstract, Results,
# Discussion) that reports a direction for VEGFR1 under a stated change in ApoB-lipoprotein
# levels, each copied verbatim from the article as archived in PubMed Central and labelled
# with the lipoprotein direction of the condition and the VEGFR1 direction stated;
# 2. if run inside the /home/ubuntu/papers repo, re-checks each excerpt verbatim against the
# archived full text (src/show.py PMC3959651 --full) and confirms its section; skipped
# elsewhere;
# 3. optionally re-checks the same excerpts in a text dump of the publisher's typeset PDF
# (set VOR_TXT=/path/to/vor.txt, made with pdftotext); reported as "not run" otherwise;
# 4. multiplies the two directions for each statement (+1: VEGFR1 moves with lipoproteins,
# -1: against them), tallies them and draws the figures.
# Group sizes and P values are transcribed from the Fig. 3 legend and are not re-tested.
import os, re, subprocess
from collections import OrderedDict
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
# ------------------------------------------------------------------ statements
# key: (location, section tag in show.py --full, condition, lipoprotein direction of the
# condition vs its control (+1 higher, -1 lower), VEGFR1 direction stated, verbatim excerpt)
# Section tags: "ABSTRACT" = abstract line, "BODY" = untitled main-text block (Results and
# Discussion are not separately headed in the archived XML), "FIG3" = Fig. 3 legend.
CONDITIONS = OrderedDict([
("abstract_mice", ("Abstract, last mechanistic sentence", "ABSTRACT",
"Hyperlipidemic mice", +1, -1,
"we observed VEGFR1 downregulation in hyperlipidemic mice")),
("results_mice", ("Results, VEGFR1 paragraph (Fig. 3c-e)", "BODY",
"Apoe- or Ldlr-null mice vs WT", +1, +1,
"hyperlipidemic Apoe- or Ldlr-null mice (reviewed in refs. 27,28) also showed increased "
"endothelial-specific Vegfr1 expression compared to control WT mice")),
("discussion_ldlr", ("Discussion, paragraph 2", "BODY",
"Ldlr-null mice", +1, +1,
"endothelial cells of Ldlr-null mice express elevated amounts of Vegfr1")),
("results_mtp", ("Results, VEGFR1 paragraph (Fig. 3a)", "BODY",
"mtp morphant zebrafish", -1, -1,
"the vegfr1 mRNA levels were lower in mtp morpholino oligonucleotide-injected embryos "
"than in uninjected siblings")),
("results_stl", ("Results, VEGFR1 paragraph (Fig. 3b)", "BODY",
"stl mutant zebrafish", -1, -1,
"VEGFR1 protein concentrations were also strongly downregulated in stl mutants")),
("results_apocii", ("Results, VEGFR1 paragraph (Fig. 3b)", "BODY",
"apoCII morphant zebrafish", +1, +1,
"were substantially upregulated in apoCII morphants, in which endothelial cells face high "
"concentrations of ApoB lipoproteins")),
("results_huvec", ("Results, VEGFR1 paragraph (Fig. 3b)", "BODY",
"HUVECs + LDL", +1, +1,
"with LDL elicited an increase in VEGFR1 but not VEGFR2 concentrations")),
("results_yfp", ("Results, transcription paragraph (Fig. 3i,j)", "BODY",
"mtp knockdown, flt1:YFP reporter", -1, -1,
"mtp knockdown resulted in a significant (P=0.0248) decrease in YFP expression")),
])
# General statements of the lipoprotein -> VEGFR1 relation (no single condition).
# key: (location, section tag, implied sign of the relation, verbatim excerpt)
RELATIONS = OrderedDict([
("results_rescue_logic", ("Results, rescue paragraph (Fig. 3f-h)", "BODY", +1,
"if lipoprotein depletion promotes angiogenesis by downregulation of VEGFR1")),
("results_yfp_conclusion", ("Results, transcription paragraph (Fig. 3i,j)", "BODY", -1,
"providing evidence that ApoB-containing lipoproteins inhibit vegfr1 expression at the "
"transcriptional level")),
("discussion_regulated", ("Discussion, paragraph 2", "BODY", 0,
"expression of VEGFR1 is regulated in vivo by the availability of ApoB-containing "
"lipoproteins")),
("abstract_mediator", ("Abstract, same sentence, first clause", "ABSTRACT", 0,
"we identified downregulation of vascular endothelial growth factor receptor 1 (VEGFR1), "
"which acts as a decoy receptor for VEGF, as a key mediator of the endothelial response "
"to lipoproteins")),
])
# Fig. 3 legend, panel e, and Online Methods (transcribed; not re-tested)
FIG3E = {"n_WT": 5, "n_Apoe_null": 5, "P_reported": 4e-14, "test": "two-tailed z test on ratio vs 1"}
FIG3_SNIPPETS = [
"Semiquantitative RT-PCR measurement of Vegfr1 mRNA expression in aortic arches dissected "
"from WT (n = 5) and Apoe-null (n = 5) mice. *P = 4 × 10-14 by z test.",
"Confocal images of aortic roots in WT C57BL/6 (left, c) and Apoe-null (right, c) littermates "
"and of abdominal aortas in WT C57BL/6 (left, d) and Ldlr-null (right, d) littermates",
]
METHODS_SNIPPET = ("All analyses were performed using mice at 8–38 weeks of age fed with a normal "
"chow diet.")
WORD = {1: "up", -1: "down", 0: "neutral"}
# ------------------------------------------------------------------ verification
def load_archived_text():
repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
return None
try:
out = subprocess.run([py, "src/show.py", "PMC3959651", "--full"], cwd=repo,
capture_output=True, text=True, timeout=120).stdout
except Exception:
return None
return out or None
def split_sections(txt):
abstract, body, fig3, methods, cur = [], [], [], [], None
for line in txt.splitlines():
if line.startswith("ABSTRACT:"):
abstract.append(line); cur = None; continue
if line.startswith("## ONLINE METHODS"):
cur = "M"; continue
if line.startswith("## "):
cur = "B"; continue
if line.startswith("[Figure 3]"):
cur = "F3"
elif line.startswith("[Figure"):
cur = "OF"
if cur == "B":
body.append(line)
elif cur == "F3":
fig3.append(line)
elif cur == "M":
methods.append(line)
return {"ABSTRACT": "\n".join(abstract), "BODY": "\n".join(body),
"FIG3": "\n".join(fig3), "METHODS": "\n".join(methods)}
def verify(txt):
secs = split_sections(txt)
ok = True
print("Verbatim check against the archived PMC full text (src/show.py PMC3959651 --full):")
items = [(k, v[1], v[5]) for k, v in CONDITIONS.items()] + \
[(k, v[1], v[3]) for k, v in RELATIONS.items()] + \
[(f"fig3_legend_{i}", "FIG3", s) for i, s in enumerate(FIG3_SNIPPETS)] + \
[("methods_mice", "METHODS", METHODS_SNIPPET)]
for key, sec, s in items:
n_all, n_sec = txt.count(s), secs.get(sec, "").count(s)
good = n_sec >= 1
ok &= good
print(f" [{'ok' if good else 'MISSING'}] {key:24s} {n_all}x in full text, {n_sec}x in {sec}")
print(" all verbatim checks passed" if ok else " SOME CHECKS FAILED")
return ok
def verify_vor():
p = os.environ.get("VOR_TXT")
print("\nPublisher typeset PDF check:", end=" ")
if not (p and os.path.exists(p)):
print("not run (set VOR_TXT to a pdftotext dump of the typeset article)")
return
t = open(p, encoding="utf-8", errors="replace").read()
t = re.sub(r"\s+", " ", re.sub(r"-\n", "", t))
norm = lambda s: re.sub(r"\s+", "", s.replace("−", "-"))
tn = norm(t)
print()
for key in ("abstract_mice", "results_mice", "discussion_ldlr"):
print(f" {key:24s} found {tn.count(norm(CONDITIONS[key][5]))}x")
print(f" {'results_yfp_conclusion':24s} found {tn.count(norm(RELATIONS['results_yfp_conclusion'][3]))}x")
print(f" {'fig3e P = 4 x 10-14':24s} found {tn.count(norm('*P = 4 × 10-14 by z test'))}x")
# ------------------------------------------------------------------ tally
def tally():
print("\nCondition statements: lipoprotein direction x VEGFR1 direction")
print(f" {'key':18s} {'location':46s} {'condition':34s} lipo VEGFR1 product")
for k, (loc, _s, cond, l, v, _q) in CONDITIONS.items():
print(f" {k:18s} {loc:46s} {cond:34s} {WORD[l]:5s} {WORD[v]:7s} {l * v:+d}")
prods = {k: v[3] * v[4] for k, v in CONDITIONS.items()}
pos = [k for k, p in prods.items() if p > 0]
neg = [k for k, p in prods.items() if p < 0]
print(f"\n statements with VEGFR1 moving WITH lipoproteins (+1): {len(pos)} of {len(prods)}")
print(f" statements with VEGFR1 moving AGAINST lipoproteins (-1): {len(neg)} of {len(prods)} -> {neg}")
mice = [k for k, v in CONDITIONS.items() if "mice" in v[2]]
print(f" mouse statements: {len(mice)}; directions: "
+ ", ".join(f"{k}={WORD[CONDITIONS[k][4]]}" for k in mice))
print(f" Abstract mouse direction agrees with Results mouse direction: "
f"{CONDITIONS['abstract_mice'][4] == CONDITIONS['results_mice'][4]}")
print(f" all non-Abstract condition statements give +1: "
f"{all(p > 0 for k, p in prods.items() if k != 'abstract_mice')}")
print("\nGeneral relation statements (implied sign of ApoB lipoproteins -> VEGFR1):")
for k, (loc, _s, sgn, _q) in RELATIONS.items():
print(f" {k:24s} {loc:46s} {WORD[sgn] if sgn == 0 else f'{sgn:+d}'}")
print(" note: results_yfp_conclusion (-1) is drawn from the result in results_yfp, "
f"whose own product is {prods['results_yfp']:+d}")
print(f"\nFig. 3e (transcribed, not re-tested): n = {FIG3E['n_WT']} WT vs "
f"{FIG3E['n_Apoe_null']} Apoe-null; P = {FIG3E['P_reported']:.0e} by {FIG3E['test']}")
return prods
# ------------------------------------------------------------------ figures
def style():
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
"axes.edgecolor": SEC, "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"svg.fonttype": "path", "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG,
})
def panel(ax, x, y, w, h, color, title, sub, body):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.008,rounding_size=0.015",
lw=2, ec=color, fc="white"))
ax.text(x + 0.02, y + h - 0.035, title, fontsize=12, fontweight="bold", color=color, va="top")
ax.text(x + 0.02, y + h - 0.095, sub, fontsize=10, color=SEC, va="top", style="italic")
yy = y + h - 0.165
for txt, kw in body:
ax.text(x + 0.02, yy, txt, fontsize=kw.get("size", 10.5), color=kw.get("color", TXT),
fontweight=kw.get("weight", "normal"), va="top", linespacing=1.3)
yy -= kw.get("step", 0.12)
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), **kw)
plt.close(fig)
def fig1():
fig = plt.figure(figsize=(7.2, 5.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
ax.text(0.03, 0.98, "VEGFR1 in hyperlipidemic mice: two directions for one experiment",
fontsize=11.5, fontweight="bold", va="top")
panel(ax, 0.03, 0.27, 0.41, 0.64, ORANGE, "Abstract", "summary of the mouse result", [
("Abstract, last mechanistic sentence:", {"color": SEC, "size": 10, "step": 0.045}),
("“…and we observed VEGFR1\ndownregulation in hyperlipidemic\nmice.”", {"step": 0.26}),
("Stated: VEGFR1 goes down", {"color": ORANGE, "weight": "bold"}),
])
n_wt, n_ko = FIG3E["n_WT"], FIG3E["n_Apoe_null"]
panel(ax, 0.56, 0.27, 0.41, 0.64, BLUE, "Results and Discussion",
"Apoe-null and Ldlr-null vs WT mice", [
("Results (Fig. 3c–e):", {"color": SEC, "size": 10, "step": 0.045}),
("“hyperlipidemic Apoe- or Ldlr-null\nmice … showed increased\nendothelial-specific "
"Vegfr1\nexpression compared to control\nWT mice”", {"step": 0.20}),
("Discussion:", {"color": SEC, "size": 10, "step": 0.045}),
("“endothelial cells of Ldlr-null\nmice express elevated amounts\nof Vegfr1”",
{"step": 0.125}),
("Reported: Vegfr1 goes up", {"color": BLUE, "weight": "bold"}),
])
ax.add_patch(FancyArrowPatch((0.448, 0.59), (0.552, 0.59), arrowstyle="<|-|>",
mutation_scale=16, lw=2, color=TXT))
ax.text(0.5, 0.62, "opposite\ndirection", ha="center", va="bottom", fontsize=10,
fontweight="bold", linespacing=1.15)
ax.text(0.03, 0.215,
"These are the paper's only mouse experiments: Apoe-null and Ldlr-null mice vs WT\n"
f"C57BL/6 on chow (Fig. 3c,d staining; Fig. 3e aortic mRNA, n = {n_wt} vs {n_ko}).",
fontsize=10, va="top", linespacing=1.35)
ax.text(0.03, 0.11,
"In zebrafish and HUVECs the paper also reports VEGFR1 rising with lipoproteins\n"
"(apoCII morphants, LDL) and falling without them (mtp morphants, stl mutants).",
fontsize=10, color=SEC, va="top", linespacing=1.35)
save(fig, "fig1")
def fig2(prods):
fig, ax = plt.subplots(figsize=(7.2, 4.9))
fig.subplots_adjust(left=0.2, right=0.98, top=0.84, bottom=0.2)
ax.set_xlim(-1.75, 1.75); ax.set_ylim(-1.75, 1.75)
for v in (0,):
ax.axhline(v, color=GRID, lw=1.2, zorder=0); ax.axvline(v, color=GRID, lw=1.2, zorder=0)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.set_xticks([-0.9, 0.9]); ax.set_xticklabels(["lower than control", "higher than control"])
ax.set_yticks([-0.9, 0.9]); ax.set_yticklabels(["VEGFR1\nlower", "VEGFR1\nhigher"])
ax.tick_params(length=0, labelsize=10)
ax.set_xlabel("ApoB-lipoprotein level in the condition", fontsize=10.5)
# quadrant notes
ax.text(-1.7, 1.68, "VEGFR1 against lipoproteins", color=SEC, fontsize=10, va="top")
ax.text(1.7, 1.68, "VEGFR1 with lipoproteins", color=SEC, fontsize=10, va="top", ha="right")
ax.text(-1.7, -1.68, "VEGFR1 with lipoproteins", color=SEC, fontsize=10, va="bottom")
ax.text(1.7, -1.68, "VEGFR1 against lipoproteins", color=SEC, fontsize=10, va="bottom",
ha="right")
short = {"abstract_mice": "Abstract:\nhyperlipidemic mice",
"results_mice": "Results: Apoe-/Ldlr-null mice",
"discussion_ldlr": "Discussion: Ldlr-null mice",
"results_apocii": "Results: apoCII morphants",
"results_huvec": "Results: HUVECs + LDL",
"results_mtp": "Results: mtp morphants",
"results_stl": "Results: stl mutants",
"results_yfp": "Results: mtp KD, YFP reporter"}
slot = {}
for k, (_loc, _s, _c, l, v, _q) in CONDITIONS.items():
i = slot.get((l, v), 0); slot[(l, v)] = i + 1
x0, y0 = 0.9 * l, 0.9 * v + 0.45 - 0.26 * i if (l, v) != (1, -1) else -0.9
col = ORANGE if prods[k] < 0 else BLUE
mk = "s" if "mice" in CONDITIONS[k][2] else "o"
ax.plot(x0 - 0.72, y0, marker=mk, ms=9, color=col, mec="white", mew=1, zorder=3)
ax.text(x0 - 0.6, y0, short[k], fontsize=10, va="center",
color=ORANGE if prods[k] < 0 else TXT,
fontweight="bold" if prods[k] < 0 else "normal")
n_pos = sum(p > 0 for p in prods.values()); n = len(prods)
ax.set_title(f"{n_pos} of {n} condition statements: VEGFR1 moves with lipoproteins (blue);\n"
f"the Abstract's mouse statement alone goes against them (orange)",
fontsize=10.5, loc="left", color=TXT)
h = [Line2D([], [], marker="s", color=SEC, ls="none", ms=8, label="mouse statement"),
Line2D([], [], marker="o", color=SEC, ls="none", ms=8, label="zebrafish or HUVEC")]
ax.legend(handles=h, loc="upper left", bbox_to_anchor=(-0.27, -0.13), ncol=2, frameon=False,
fontsize=10)
save(fig, "fig2")
if __name__ == "__main__":
txt = load_archived_text()
if txt:
verify(txt)
else:
print("(archived text not available here; verbatim check skipped)")
verify_vor()
prods = tally()
style()
fig1()
fig2(prods)
print("\nWrote fig1.svg, fig1.png, fig2.svg, fig2.png to", HERE)
Abstract's baseline cognitive-component estimates differ from the Results and Table 2, changing one component's significance
Neurology (2026) · doi:10.1212/WNL.0000000000214804 · PMC13067375
Summary
The abstract reports cross-sectional estimates from the hurdle ("zero-inflation") model of CDR-SB, adjusted for EYO, for each of the three cognitive components (reserve CogR, demographic CogD, biomarker CogB). There are three estimates for asymptomatic carriers and two for symptomatic carriers. The Results text and Table 2 describe the same model and outcome and agree with each other, but all five of the abstract's estimates or intervals differ from them. The largest difference is for the demographic component in asymptomatic carriers: the abstract gives an odds ratio of 2.60 (95% CI 1.10–6.16), which excludes 1, while the Results and Table 2 give 1.24 (0.76–2.02), which includes 1. The abstract also understates the biomarker odds ratio (5.16 against 7.77). The headline reserve finding keeps its direction and significance in both versions. However, the abstract's statement that the demographic component is associated with being unimpaired at baseline is not supported by the paper's own Table 2.
What the paper reports
Abstract, "Result":
A total of 710 Dominantly Inherited Alzheimer Network (DIAN) participants were included in the analysis, comprising 271 non-DIAD carriers (nMC), 284 asymptomatic DIAD carriers (aMC), and 155 symptomatic DIAD carriers.
In asymptomatic carriers, using a zero-inflation model adjusted for EYO showed that a 1 SD increase in the reserve component (CogR) was associated with a 4.06-fold increase in the odds of being clinically unimpaired (CDR-SB = 0; 95% CI 1.84–8.95). Similarly, a 1 SD increase in the demographic (CogD) and biomarker (CogB) components increased the odds of being CDR-SB = 0 by 2.60 (95% CI 1.10–6.16) and 5.16 (95% CI 2.00–13.33), respectively.
Among symptomatic carriers, only the reserve and the biomarker components were significant. A 1 SD increase in CogR was associated with a 0.81-fold reduction in baseline CDR-SB score (95% CI 0.72–0.92), and a 1 SD increase in CogB was associated with a 0.60-fold reduction in CDR-SB (95% CI 0.50–0.71).
Results, "Cognitive Reserve, Clinical Status, and Disease Progression", first paragraph (the quotation keeps the original wording, including "multiple"):
For the asymptomatic DIAD carrier, using the zero-inflation model and after adjusting by EYO, 1 SD increase in the reserve component would multiply the odds of being CDR-SB = 0 by 3.66 (95% CI 2.11–6.38), while 1 SD increase in demographic component and biomarker component will multiple the odds by 1.24 (95% CI 0.76–2.02) and 7.77 (95% CI 3.40–17.80), respectively (Table 2).
Once symptomatic, only the reserve and the biomarker components were significant. One SD increase in reserve and in biomarker corresponds to a 0.83-fold (95% CI 0.75–0.92) and 0.60-fold (95% CI 0.52–0.69) decrease in baseline CDR-SB, respectively (Table 2).
Table 2 ("Relationships of Cognitive Components With Baseline Clinical Diagnosis in DIAD Carriers"), CDR-SB = 0 part (columns: characteristic | odds ratio | 95% CI):
Reserve component (cogR) | 3.66 | 2.11–6.38
Demographic component (cogD) | 1.24 | 0.76–2.02
Biomarker component (cogB) | 7.77 | 3.40–17.80
cogR:EYO | 1.07 | 1.02–1.13
cogD:EYO | 0.99 | 0.95–1.04
cogB:EYO | 1.06 | 0.98–1.14
Table 2, "Cognition decline" part:
Reserve component (cogR) | 0.83 | 0.75–0.92
Demographic component (cogD) | 0.94 | 0.86–1.04
Biomarker component (cogB) | 0.60 | 0.52–0.69
Methods, "Statistical Analysis":
Both cross-sectional and longitudinal models included fixed effects of EYO, the 3 cognitive components, and their interaction with EYO.
Results, same section, second paragraph. These are the longitudinal estimates in Table 3, a different model:
One SD increase in reserve, demographic, and biomarker components corresponds to a 3.29-fold (95% CI 2.06–5.26), 1.52-fold (95% CI 1.04–2.23), and 11.82-fold (95% CI 5.67–24.63) increase in odds, respectively.
Discussion, fourth paragraph:
Our study also indicates that demographic factors, for example., age, sex, education, and SES, play a significant role in maintaining asymptomatic status in individuals with DIAD.
However, after symptom onset, the demographic component's effect diminishes, with only the reserve (CogR) and biomarker (CogB) components remaining significantly associated with disease progression.
What the paper's own numbers show
Method. I set each estimate in the abstract beside the matching estimate in the Results text and Table 2. The two body sources print identical values. I checked each printed interval for internal consistency: a log-symmetric 95% CI should have its geometric midpoint, √(lower × upper), at the point estimate. From each interval I derived an approximate standard error on the log scale, SE = (ln upper − ln lower)/3.92, and a Wald z = ln(estimate)/SE with its two-sided p. These p-values are reconstructions from rounded printed values; the paper does not print p-values for these rows. I then tested three ways the two sets could both be right: rounding, evaluation of the component effects at a different EYO through the printed EYO interactions, and a mix-up with the longitudinal Table 3. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| CogR, odds of CDR-SB = 0, asymptomatic | Abstract | 4.06 (1.84–8.95) | geometric midpoint 4.06; SE 0.404; z = 3.47, p ≈ 0.0005 |
| CogR, odds of CDR-SB = 0, asymptomatic | Results text; Table 2 | 3.66 (2.11–6.38) | midpoint 3.67; SE 0.282; z = 4.60, p < 0.0001; abstract ÷ Table 2 = 1.11 |
| CogD, odds of CDR-SB = 0, asymptomatic | Abstract | 2.60 (1.10–6.16) | midpoint 2.60; SE 0.439; z = 2.17, p ≈ 0.030; CI excludes 1 |
| CogD, odds of CDR-SB = 0, asymptomatic | Results text; Table 2 | 1.24 (0.76–2.02) | midpoint 1.24; SE 0.249; z = 0.86, p ≈ 0.39; CI includes 1; abstract ÷ Table 2 = 2.10 |
| CogB, odds of CDR-SB = 0, asymptomatic | Abstract | 5.16 (2.00–13.33) | midpoint 5.16; SE 0.484; z = 3.39, p ≈ 0.0007 |
| CogB, odds of CDR-SB = 0, asymptomatic | Results text; Table 2 | 7.77 (3.40–17.80) | midpoint 7.78; SE 0.422; z = 4.85, p < 0.0001; abstract ÷ Table 2 = 0.66 |
| CogR, ratio on CDR-SB, symptomatic | Abstract vs Results text; Table 2 | 0.81 (0.72–0.92) vs 0.83 (0.75–0.92) | SE 0.063 vs 0.052; ratio 0.98; both p < 0.001 |
| CogB, ratio on CDR-SB, symptomatic | Abstract vs Results text; Table 2 | 0.60 (0.50–0.71) vs 0.60 (0.52–0.69) | same point estimate; SE 0.089 vs 0.072 (different intervals) |
| Size of the gaps against rounding | all five pairs | — | OR-scale differences 0.40, 1.36, 2.61, 0.02 and 0.00, with interval limits differing in every pair. Two-decimal rounding can move a value by at most 0.01 |
| EYO shift that would turn Table 2 into the abstract value, OR × interactionᵉ | Table 2 interaction rows (1.07, 0.99, 1.06) | — | CogR +1.5 y, CogD −73.7 y, CogB −7.0 y: no single shift fits |
| Abstract vs longitudinal estimates | Table 3; Results paragraph 2 | 3.29, 1.52, 11.82 (asymptomatic); 0.92, 0.73 (symptomatic) | none equals the abstract's 4.06, 2.60, 5.16, 0.81, 0.60 |
| Abstract SE ÷ Table 2 SE | derived | — | 1.43, 1.76, 1.15 (asymptomatic); 1.20, 1.24 (symptomatic): every abstract interval is wider |
Each set is internally consistent: every interval centres on its own point estimate. So the abstract values are not a body estimate printed with the wrong interval, or the reverse. The two sets are distinct estimates of what the paper presents as the same quantity.
Figure
The left panel lists the abstract's five estimates (95% CI) and the right panel the matching values in the Results text and Table 2, which print identical numbers. Both panels quote the model description each source gives. Each connector gives the ratio of the abstract estimate to the Table 2 estimate, computed in recompute.py.txt. Only the demographic component (CogD) in asymptomatic carriers changes significance: its interval excludes 1 in the abstract and includes 1 in Table 2. The other four pairs differ in point estimate, in interval, or in both, but keep the same direction and significance.
The same ten estimates on one logarithmic axis. Orange marks the abstract and blue the Results text and Table 2, each labelled with its printed value; the thin vertical line marks 1 (no association). The plot shows the sizes of the gaps that the text panels state as ratios. The biomarker odds ratio is a third lower in the abstract. The Table 2 interval for the demographic component crosses 1, while the abstract's does not.
Why it matters
The abstract is the part of the paper most readers see and the part that indexing services reproduce. In it, the demographic component (age, education, sex, race, socioeconomic status and vascular burden score, per the Methods) is presented as significantly associated with being clinically unimpaired at baseline: OR 2.60, 95% CI 1.10–6.16. The paper's own cross-sectional model gives 1.24 (0.76–2.02), an approximate p of 0.39, so the association the abstract reports is not supported by the paper's own analysis. The Discussion says that demographic factors "play a significant role in maintaining asymptomatic status". That sentence matches the abstract's figure. Among the paper's own results, it is supported only by the separate longitudinal estimate in Table 3 (1.52, 1.04–2.23; approximate p 0.031), not by the baseline Table 2 that the abstract describes.
The relative sizes of the components also change. In the abstract, the reserve odds ratio (4.06) is close to the biomarker odds ratio (5.16). In Table 2, the biomarker effect (7.77) is about twice the reserve effect (3.66). A reader comparing the protective weight of cognitive reserve with pathology burden would draw a different picture from each source.
What is not affected: the reserve component is significant in the same direction in every version (abstract 4.06, Table 2 3.66, Table 3 3.29), so the paper's central conclusion, that higher cognitive reserve goes with remaining unimpaired, does not depend on which set is correct. The two symptomatic-carrier estimates differ only slightly and stay significant.
Explanations considered
- Rounding or transcription of one digit. This does not explain the numbers. All five pairs differ, by up to 2.61 on the odds-ratio scale, and every interval in the abstract centres exactly on its own point estimate. A single mistyped value would leave the rest of the set unchanged and would usually break this internal consistency.
- A different reference EYO. Because the model contains component × EYO interactions, a component's odds ratio depends on the EYO at which it is evaluated. If the abstract used a different centring, one shift would have to convert all three Table 2 estimates to the abstract values. The printed interactions require +1.5 years for CogR, −73.7 years for CogD and −7.0 years for CogB. These shifts are incompatible with one another, and −73.7 years lies outside any plausible EYO range, so re-centring does not account for the gap.
- The abstract quotes the longitudinal model. It does not. Table 3 gives 3.29, 1.52 and 11.82 for asymptomatic carriers and 0.92 and 0.73 for symptomatic carriers; none equals an abstract value.
- Body and table disagree, so the abstract follows one of them. They do not disagree. The Results text cites Table 2 explicitly and prints the same five values, and the PMC HTML table shows the same rows. Misaligned cells, a missing minus sign or a lost superscript cannot apply: every value is a positive ratio, and the table and the prose agree.
- A conversion artefact in the archived text. This is ruled out. The quoted passages match the publisher-deposited JATS full text, read both as the live PMC page and as the Europe PMC XML.
- An earlier analysis. This is offered as context only and is not a conclusion about how the discrepancy arose. The abstract's estimates are identical, at the earlier abstract's printed precision, to those of a 2024 conference abstract on the same topic (Alzheimer's & Dementia 2024, doi:10.1002/alz.086295, PMC11710562). That abstract reports aMC n = 204 and sMC n = 115, against 284 and 155 here, and adjusts "by mutation mean AAO". Under simple 1/√n scaling, its standard errors would be about 1.18 and 1.16 times those of the larger sample. Every abstract interval is indeed wider than its Table 2 counterpart (1.15 to 1.76 times), which agrees with this, although it does not prove it. Whichever analysis produced them, the abstract's values are not the estimates that the paper's Results and Table 2 report for the 284/155-carrier sample.
Status
- Version of record. The publisher page and PDF on neurology.org returned HTTP 403 to automated access, and the PMC PDF could not be fetched headlessly. The quoted passages were therefore checked in the publisher-deposited JATS full text, read two ways: the live PMC HTML (pmc.ncbi.nlm.nih.gov/articles/PMC13067375/) and the Europe PMC fullTextXML. Both match the archived text used here exactly, and every quote above was re-checked against it by
recompute.py.txt. - Supplements. The Europe PMC supplementary-files bundle holds only the two figure images. The text cites eFigures 1–2 and eTable 1, but the publisher's supplement page returned 403. By their descriptions, these cover the tertile plots and the longitudinal CogR analyses, not the baseline zero-part estimates, so they would not decide which set is correct. Data are available on request; the paper states that the component-decomposition code is public.
- Corrections. Crossref lists no updates, and PMC and Europe PMC hold no correction, erratum, expression of concern or retraction notice for this DOI.
- PubPeer. Direct PubPeer search returned HTTP 403. A web search for the title together with PubPeer found no thread.
- Secondary coverage. A news summary of the paper (Neurology Advisor) quotes the body values (OR 3.66; 0.83).
- Sources: the PMC archive of PMC13067375; Crossref and PMC/Europe PMC notice checks for doi:10.1212/WNL.0000000000214804; Europe PMC full text of PMC11710562 (doi:10.1002/alz.086295). Status as of 2026-10-23.
Suggested correction
The authors should confirm which analysis is final and bring the abstract into line with it. If the Results text and Table 2 are correct, the abstract's Result paragraph should read:
… a 1 SD increase in the reserve component (CogR) was associated with a 3.66-fold increase in the odds of being clinically unimpaired (CDR-SB = 0; 95% CI 2.11–6.38). A 1 SD increase in the biomarker component (CogB) increased the odds of being CDR-SB = 0 by 7.77 (95% CI 3.40–17.80); the demographic component (CogD) was not significantly associated (1.24, 95% CI 0.76–2.02). … A 1 SD increase in CogR was associated with a 0.83-fold reduction in baseline CDR-SB score (95% CI 0.75–0.92), and a 1 SD increase in CogB was associated with a 0.60-fold reduction in CDR-SB (95% CI 0.52–0.69).
The Discussion sentence on demographic factors playing "a significant role in maintaining asymptomatic status" should then name the result that supports it, the longitudinal Table 3 estimate, or else be qualified. If instead the abstract's values are the intended ones, Table 2 and the Results text need correcting, along with an explanation of the sample and adjustment that produced them.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC13067375 (Neurology 2026, doi:10.1212/WNL.0000000000214804)
# Abstract vs Results text / Table 2: baseline estimates for the cognitive components.
#
# Run: python recompute.py.txt (prints every number in writeup.md,
# writes fig1.* and fig2.*)
# python recompute.py.txt --candidates (also writes five candidate figure designs,
# candidates/cand_A..E.{svg,png})
# Needs numpy, scipy and matplotlib. Figures are written next to this file.
#
# Inputs
# * Every estimate below is transcribed from the article as archived in PubMed Central
# (PMC13067375): Abstract (Results paragraph), Results section "Cognitive Reserve,
# Clinical Status, and Disease Progression", Table 2 and Table 3.
# * For context only (not part of the paper): the 2024 conference abstract
# "Cognitive reserve influences symptom onset and longitudinal decline in Dominantly
# Inherited Alzheimer's Disease", Alzheimer's & Dementia 2024, doi:10.1002/alz.086295,
# PMC11710562, as given in its Europe PMC full-text XML.
# * When run inside the /home/ubuntu/papers repo, the script re-checks each quote against the
# archived full text with src/show.py (skipped elsewhere).
# * Approximate Wald statistics assume each printed 95% CI is symmetric on the log scale:
# SE(log OR) = (ln hi - ln lo) / (2 x 1.959964), z = ln(OR) / SE, p = 2 x (1 - Phi(|z|)).
# These are reconstructions from rounded printed values, not values the paper reports.
import math, os, subprocess, sys
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Rectangle
from scipy.stats import norm
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.ppf(0.975)
# ------------------------------------------------------------------ quotes
QUOTES = {
"abs_n": ("Abstract, Result",
"A total of 710 Dominantly Inherited Alzheimer Network (DIAN) participants were included in "
"the analysis, comprising 271 non-DIAD carriers (nMC), 284 asymptomatic DIAD carriers (aMC), "
"and 155 symptomatic DIAD carriers."),
"abs_aMC": ("Abstract, Result",
"In asymptomatic carriers, using a zero-inflation model adjusted for EYO showed that a 1 SD "
"increase in the reserve component (CogR) was associated with a 4.06-fold increase in the odds "
"of being clinically unimpaired (CDR-SB = 0; 95% CI 1.84–8.95). Similarly, a 1 SD increase in "
"the demographic (CogD) and biomarker (CogB) components increased the odds of being CDR-SB = 0 "
"by 2.60 (95% CI 1.10–6.16) and 5.16 (95% CI 2.00–13.33), respectively."),
"abs_sMC": ("Abstract, Result",
"Among symptomatic carriers, only the reserve and the biomarker components were significant. "
"A 1 SD increase in CogR was associated with a 0.81-fold reduction in baseline CDR-SB score "
"(95% CI 0.72–0.92), and a 1 SD increase in CogB was associated with a 0.60-fold reduction in "
"CDR-SB (95% CI 0.50–0.71)."),
"res_aMC": ("Results, 'Cognitive Reserve, Clinical Status, and Disease Progression', para 1",
"For the asymptomatic DIAD carrier, using the zero-inflation model and after adjusting by EYO, "
"1 SD increase in the reserve component would multiply the odds of being CDR-SB = 0 by 3.66 "
"(95% CI 2.11–6.38), while 1 SD increase in demographic component and biomarker component will "
"multiple the odds by 1.24 (95% CI 0.76–2.02) and 7.77 (95% CI 3.40–17.80), respectively (Table 2)."),
"res_sMC": ("Results, 'Cognitive Reserve, Clinical Status, and Disease Progression', para 1",
"Once symptomatic, only the reserve and the biomarker components were significant. One SD "
"increase in reserve and in biomarker corresponds to a 0.83-fold (95% CI 0.75–0.92) and "
"0.60-fold (95% CI 0.52–0.69) decrease in baseline CDR-SB, respectively (Table 2)."),
"res_long": ("Results, 'Cognitive Reserve, Clinical Status, and Disease Progression', para 2",
"One SD increase in reserve, demographic, and biomarker components corresponds to a 3.29-fold "
"(95% CI 2.06–5.26), 1.52-fold (95% CI 1.04–2.23), and 11.82-fold (95% CI 5.67–24.63) increase "
"in odds, respectively."),
"methods": ("Methods, Statistical Analysis",
"Both cross-sectional and longitudinal models included fixed effects of EYO, the 3 cognitive "
"components, and their interaction with EYO."),
"disc_demo": ("Discussion, paragraph 4",
"Our study also indicates that demographic factors, for example., age, sex, education, and SES, "
"play a significant role in maintaining asymptomatic status in individuals with DIAD."),
"disc_after": ("Discussion, paragraph 4",
"However, after symptom onset, the demographic component's effect diminishes, with only the "
"reserve (CogR) and biomarker (CogB) components remaining significantly associated with disease "
"progression."),
"t2_cogR": ("Table 2, CDR-SB = 0 part", "Reserve component (cogR) | 3.66 | 2.11–6.38"),
"t2_cogD": ("Table 2, CDR-SB = 0 part", "Demographic component (cogD) | 1.24 | 0.76–2.02"),
"t2_cogB": ("Table 2, CDR-SB = 0 part", "Biomarker component (cogB) | 7.77 | 3.40–17.80"),
"t2_int": ("Table 2, CDR-SB = 0 part",
"cogR:EYO | 1.07 | 1.02–1.13\ncogD:EYO | 0.99 | 0.95–1.04\ncogB:EYO | 1.06 | 0.98–1.14"),
"t2_dec": ("Table 2, Cognition decline part",
"Reserve component (cogR) | 0.83 | 0.75–0.92\nDemographic component (cogD) | 0.94 | 0.86–1.04\n"
"Biomarker component (cogB) | 0.60 | 0.52–0.69"),
"t3_dec": ("Table 3, Longitudinal decline part",
"Reserve component (cogR) | 0.92 | 0.85–1.01\nDemographic component (cogD) | 0.94 | 0.87–1.03\n"
"Biomarker component (cogB) | 0.73 | 0.64–0.83"),
}
def check_quotes():
repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
print("Quote check: src/show.py not available here; skipped.")
return
try:
full = subprocess.run([py, "src/show.py", "PMC13067375", "--full"], cwd=repo,
capture_output=True, text=True, timeout=300).stdout
except Exception as e: # pragma: no cover
print(f"Quote check: could not load archived text ({e}); skipped.")
return
nrm = lambda s: " ".join(s.split())
full_n = nrm(full)
print("Quote check against the PMC-archived full text (occurrences; 0 = not found):")
for key, (loc, q) in QUOTES.items():
print(f" {key:10s} {full_n.count(nrm(q))} ({loc})")
# ------------------------------------------------------------------ estimates (OR, lo, hi)
ROWS = ["aMC CogR", "aMC CogD", "aMC CogB", "sMC CogR", "sMC CogB"]
ABSTRACT = {"aMC CogR": (4.06, 1.84, 8.95), "aMC CogD": (2.60, 1.10, 6.16),
"aMC CogB": (5.16, 2.00, 13.33), "sMC CogR": (0.81, 0.72, 0.92),
"sMC CogB": (0.60, 0.50, 0.71)}
TABLE2 = {"aMC CogR": (3.66, 2.11, 6.38), "aMC CogD": (1.24, 0.76, 2.02),
"aMC CogB": (7.77, 3.40, 17.80), "sMC CogR": (0.83, 0.75, 0.92),
"sMC CogD": (0.94, 0.86, 1.04), "sMC CogB": (0.60, 0.52, 0.69)}
RESULTS_TEXT = {k: TABLE2[k] for k in ROWS} # the Results prose prints the same values
TABLE3 = {"aMC CogR": (3.29, 2.06, 5.26), "aMC CogD": (1.52, 1.04, 2.23),
"aMC CogB": (11.82, 5.67, 24.63), "sMC CogR": (0.92, 0.85, 1.01),
"sMC CogD": (0.94, 0.87, 1.03), "sMC CogB": (0.73, 0.64, 0.83)}
T2_INTERACTION = {"CogR": 1.07, "CogD": 0.99, "CogB": 1.06} # cogX:EYO, CDR-SB = 0 part
# Context only: 2024 conference abstract (aMC n = 204, sMC n = 115), printed precision
CONF2024 = {"aMC CogR": (4.06, 1.8, 8.9), "aMC CogD": (2.6, 1.1, 6.2),
"aMC CogB": (5.16, 2.0, 13.3), "sMC CogR": (0.81, 0.72, 0.92),
"sMC CogB": (0.60, 0.50, 0.71)}
N_PAPER = {"aMC": 284, "sMC": 155}
N_CONF = {"aMC": 204, "sMC": 115}
def se(lo, hi):
return (math.log(hi) - math.log(lo)) / (2 * Z975)
def wald(o, lo, hi):
s = se(lo, hi)
z = math.log(o) / s
return math.sqrt(lo * hi), s, z, 2 * norm.sf(abs(z))
def report():
print("\n=== 1. Internal consistency and approximate Wald statistics ===")
print(f"{'source':14s} {'estimate':9s} {'OR':>6s} {'95% CI':>13s} {'geomean':>8s} "
f"{'SE':>6s} {'z':>6s} {'p':>8s} excl.1")
for name, src in [("Abstract", ABSTRACT), ("Table 2", TABLE2), ("Table 3", TABLE3)]:
for k, (o, lo, hi) in src.items():
gm, s, z, p = wald(o, lo, hi)
print(f"{name:14s} {k:9s} {o:6.2f} {lo:6.2f}-{hi:<6.2f} {gm:8.2f} {s:6.3f} "
f"{z:6.2f} {p:8.4f} {lo > 1 or hi < 1}")
print("\n=== 2. Abstract vs Results text / Table 2 ===")
for k in ROWS:
a, t = ABSTRACT[k], TABLE2[k]
print(f"{k:9s} abstract {a[0]:.2f} ({a[1]:.2f}-{a[2]:.2f}) | Table 2 {t[0]:.2f} "
f"({t[1]:.2f}-{t[2]:.2f}) | ratio abstract/Table2 = {a[0]/t[0]:.3f} | "
f"SE ratio = {se(*a[1:]) / se(*t[1:]):.2f} | identical? {a == t}")
print("Results text equals Table 2 for all five:", all(RESULTS_TEXT[k] == TABLE2[k] for k in ROWS))
print("\n=== 3. Rounding: largest change the printing precision could cause ===")
for k in ROWS:
a, t = ABSTRACT[k], TABLE2[k]
print(f"{k:9s} |abstract - Table 2| = {abs(a[0]-t[0]):.2f} on the OR scale "
f"(two-decimal rounding of both: at most 0.01)")
print("\n=== 4. Re-centring EYO: OR_at_e = OR_Table2 x interaction^e ===")
for c in ["CogR", "CogD", "CogB"]:
k = "aMC " + c
e = math.log(ABSTRACT[k][0] / TABLE2[k][0]) / math.log(T2_INTERACTION[c])
print(f"{c}: interaction {T2_INTERACTION[c]:.2f}; EYO shift needed to reach abstract value "
f"= {e:+.1f} y")
print("\n=== 5. Does the abstract match Table 3 (longitudinal)? ===")
for k in ROWS:
print(f"{k:9s} abstract {ABSTRACT[k][0]:.2f} vs Table 3 {TABLE3[k][0]:.2f}: "
f"equal? {ABSTRACT[k][0] == TABLE3[k][0]}")
print("\n=== 6. Context: abstract vs 2024 conference abstract, at the conference precision ===")
def rnd(x, ref):
d = len(f"{ref}".split(".")[1]) if "." in f"{ref}" else 0
return round(x + 1e-9, d)
for k in ROWS:
a, c = ABSTRACT[k], CONF2024[k]
same_pt = rnd(a[0], c[0]) == c[0]
ci_gap = max(abs(a[1] - c[1]), abs(a[2] - c[2]))
print(f"{k:9s} Neurology abstract {a[0]:.2f} ({a[1]:.2f}-{a[2]:.2f}) | conference "
f"{c[0]} ({c[1]}-{c[2]}) | point estimate same? {same_pt} | largest CI-limit "
f"difference {ci_gap:.2f} (within 1-decimal printing: {ci_gap <= 0.05 + 1e-9})")
for g in ["aMC", "sMC"]:
print(f"{g}: n = {N_CONF[g]} (conference) vs {N_PAPER[g]} (paper); "
f"sqrt(n ratio) = {math.sqrt(N_PAPER[g] / N_CONF[g]):.2f}")
# ------------------------------------------------------------------ style
BG, TXT, TXT2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#8a8984"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": TXT2, "ytick.color": TXT2,
"axes.edgecolor": "#b9b8b3",
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "lines.linewidth": 2,
})
LABEL = {"aMC CogR": "Reserve (CogR)", "aMC CogD": "Demographic (CogD)",
"aMC CogB": "Biomarker (CogB)", "sMC CogR": "Reserve (CogR)", "sMC CogB": "Biomarker (CogB)"}
def save(fig, stem, sub=""):
d = os.path.join(HERE, sub)
os.makedirs(d, exist_ok=True)
fig.savefig(os.path.join(d, stem + ".svg"))
fig.savefig(os.path.join(d, stem + ".png"), dpi=200)
plt.close(fig)
print(f"wrote {os.path.join(sub, stem)}.svg and .png")
def ci_txt(t):
return f"{t[0]:.2f} ({t[1]:.2f}–{t[2]:.2f})"
def log_axis(ax, lo, hi, ticks):
ax.set_xscale("log")
ax.set_xlim(lo, hi)
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
# ------------------------------------------------------------------ A: text panels
def cand_A(stem="cand_A", sub="candidates"):
W, H = 7.2, 5.8
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
px = {"L": 0.10, "R": 4.40}; pw = 2.70
top, bot = 5.68, 0.82
def panel(x, title, sub, accent, src, model):
ax.add_patch(FancyBboxPatch((x, bot), pw, top - bot, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=accent, lw=1.5))
ax.text(x + 0.12, top - 0.12, title, fontsize=11.5, fontweight="bold", va="top")
ax.text(x + 0.12, top - 0.42, sub, fontsize=10, va="top", color=accent)
ax.text(x + 0.12, top - 0.70, model, fontsize=10, va="top", color=TXT2, fontstyle="italic",
linespacing=1.15)
ys = {}
y = top - 1.62
for grp, keys in [("Asymptomatic: odds of CDR-SB = 0", ROWS[:3]),
("Symptomatic: ratio on CDR-SB", ROWS[3:])]:
ax.text(x + 0.12, y + 0.06, grp, fontsize=10, color=TXT2, va="bottom")
y -= 0.30
for k in keys:
ax.text(x + 0.12, y, LABEL[k].split(" (")[1][:-1], fontsize=10, va="center",
fontweight="bold")
ax.text(x + pw - 0.12, y, ci_txt(src[k]), fontsize=10, va="center", ha="right",
color=accent, fontweight="bold")
ys[k] = y
y -= 0.62
y -= 0.0
return ys
ysL = panel(px["L"], "Abstract", "Result paragraph", ORANGE, ABSTRACT,
"“…using a zero-inflation model\nadjusted for EYO…”")
panel(px["R"], "Results text and Table 2", "both print the same values", BLUE, TABLE2,
"“…using the zero-inflation model\nand after adjusting by EYO…”")
notes = {}
for k in ROWS:
a, t = ABSTRACT[k], TABLE2[k]
r = a[0] / t[0]
notes[k] = f"×{r:.2f}" if abs(r - 1) > 1e-9 else "×1.00, CI differs"
notes["aMC CogD"] = f"×{ABSTRACT['aMC CogD'][0]/TABLE2['aMC CogD'][0]:.2f}"
x0, x1 = px["L"] + pw + 0.04, px["R"] - 0.04
for k in ROWS:
y = ysL[k]
strong = k == "aMC CogD"
ax.add_patch(FancyArrowPatch((x0, y), (x1, y), arrowstyle="<|-|>", mutation_scale=11,
lw=2 if strong else 1.2, color=TXT if strong else GRAY))
ax.text((x0 + x1) / 2, y + 0.06, notes[k], ha="center", va="bottom", fontsize=10,
color=TXT if strong else TXT2, fontweight="bold" if strong else "normal",
linespacing=1.1)
if strong:
ax.text((x0 + x1) / 2, y - 0.07, "CI excludes 1\n→ includes 1", ha="center", va="top",
fontsize=10, color=TXT, fontweight="bold", linespacing=1.05)
ax.text((x0 + x1) / 2, top - 0.12, "conflict", ha="center", va="top", fontsize=11, fontweight="bold")
ax.text((x0 + x1) / 2, top - 0.45, "same model\nand outcome;\nall 5 estimates\ndiffer",
ha="center", va="top", fontsize=10, color=TXT2, linespacing=1.15)
ax.text(0.12, 0.42,
"Estimates (95% CI) per 1 SD of each component, as printed. Ratios abstract ÷ Table 2 are\n"
"computed in recompute.py.txt. Only the demographic component changes significance; the\n"
"symptomatic CogB estimate (0.60) is the same point value with a different interval.",
fontsize=10, color=TXT2, va="center", linespacing=1.25)
save(fig, stem, sub)
# ------------------------------------------------------------------ B: forest plot
def cand_B(stem="cand_B", sub="candidates"):
fig, ax = plt.subplots(figsize=(7.2, 4.8))
fig.subplots_adjust(left=0.27, right=0.78, top=0.90, bottom=0.13)
ypos, labels, y = {}, [], 0
heads = []
for grp, keys in [("Asymptomatic carriers\nodds of CDR-SB = 0", ROWS[:3]),
("Symptomatic carriers\nfold change in CDR-SB", ROWS[3:])]:
heads.append((y, grp)); y += 0.9
for k in keys:
ypos[k] = y; y += 1.25
y += 0.1
for k in ROWS:
for src, col, off, name in [(ABSTRACT, ORANGE, -0.17, "Abstract"),
(TABLE2, BLUE, 0.17, "Results text and Table 2")]:
o, lo, hi = src[k]
yy = ypos[k] + off
ax.plot([lo, hi], [yy, yy], color=col, lw=2, solid_capstyle="butt")
ax.plot(o, yy, "o", ms=8, color=col, label=name if k == ROWS[0] else None)
ax.text(1.03, yy, ci_txt(src[k]), transform=ax.get_yaxis_transform(), va="center",
fontsize=10, color=col, fontweight="bold")
ax.axvline(1, color=TXT2, lw=0.8)
log_axis(ax, 0.4, 25, [0.5, 1, 2, 5, 10, 20])
ax.set_yticks([ypos[k] for k in ROWS]); ax.set_yticklabels([LABEL[k] for k in ROWS], fontsize=10,
color=TXT)
for yh, g in heads:
ax.text(-0.36, yh + 0.1, g, transform=ax.get_yaxis_transform(), fontsize=10, color=TXT2,
va="center", ha="left", fontweight="bold", linespacing=1.1)
ax.set_ylim(y - 0.2, -0.6)
ax.set_xlabel("Odds ratio or fold change per 1 SD (log scale); thin line = 1 (no association)")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False, fontsize=10)
ax.text(0.42, ypos["aMC CogD"] + 0.62, "Table 2 CogD interval crosses 1", fontsize=10, color=TXT,
va="center", fontweight="bold")
save(fig, stem, sub)
# ------------------------------------------------------------------ C: dumbbell (shift)
def cand_C(stem="cand_C", sub="candidates"):
fig, ax = plt.subplots(figsize=(7.2, 3.6))
fig.subplots_adjust(left=0.34, right=0.97, top=0.86, bottom=0.17)
ys = {k: i + (0.4 if i >= 3 else 0) for i, k in enumerate(ROWS)}
for k in ROWS:
a, t = ABSTRACT[k][0], TABLE2[k][0]
y = ys[k]
if a != t:
ax.annotate("", xy=(t, y), xytext=(a, y),
arrowprops=dict(arrowstyle="-", color=GRAY, lw=2))
ax.plot(a, y, "o", ms=9, color=ORANGE, label="Abstract" if k == ROWS[0] else None, zorder=3)
ax.plot(t, y, "D", ms=8, color=BLUE, label="Results text and Table 2" if k == ROWS[0] else None,
zorder=3)
lbl = f"×{a/t:.2f}" if a != t else "same point value; CI differs"
ax.text(max(a, t) * 1.2, y, lbl, va="center", fontsize=10,
color=TXT if k == "aMC CogD" else TXT2,
fontweight="bold" if k == "aMC CogD" else "normal")
ax.axvline(1, color=TXT2, lw=0.8)
log_axis(ax, 0.4, 25, [0.5, 1, 2, 5, 10, 20])
ax.set_yticks([ys[k] for k in ROWS])
ax.set_yticklabels([("Asympt. " if k.startswith("a") else "Sympt. ") + LABEL[k] for k in ROWS],
color=TXT)
ax.set_ylim(max(ys.values()) + 0.6, -0.6)
ax.set_xlabel("Point estimate per 1 SD (log scale); thin line = 1")
ax.legend(loc="lower center", bbox_to_anchor=(0.45, 1.0), ncol=2, frameon=False)
save(fig, stem, sub)
# ------------------------------------------------------------------ D: provenance matrix
def cand_D(stem="cand_D", sub="candidates"):
W, H = 7.2, 4.3
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
cols = [("Neurology 2026\nabstract", ABSTRACT, "A"),
("2024 conference\nabstract*", CONF2024, "A"),
("Neurology 2026\nResults text\nand Table 2", TABLE2, "T"),
("Neurology 2026\nTable 3\n(longitudinal)", TABLE3, "O")]
x0, cw, rh, ytop = 1.27, 1.47, 0.42, 3.22
for j, (h, _, _) in enumerate(cols):
ax.text(x0 + j * cw + cw / 2, ytop + 0.08, h, ha="center", va="bottom", fontsize=10,
fontweight="bold", linespacing=1.1)
fill = {"A": "#fbe1d6", "T": "#d9e7f8", "O": "#f0efec"}
edge = {"A": ORANGE, "T": BLUE, "O": "#c9c8c3"}
for i, k in enumerate(ROWS):
y = ytop - (i + 1) * rh - (0.14 if i >= 3 else 0)
ax.text(0.10, y + rh / 2, ("Asympt. " if i < 3 else "Sympt. ") + k.split()[1], va="center",
fontsize=10, fontweight="bold")
for j, (_, src, grp) in enumerate(cols):
o, lo, hi = src[k]
ax.add_patch(Rectangle((x0 + j * cw + 0.03, y + 0.03), cw - 0.06, rh - 0.06,
fc=fill[grp], ec=edge[grp], lw=1))
txt = f"{o:g} ({lo:g}–{hi:g})" if src is CONF2024 else ci_txt(src[k])
ax.text(x0 + j * cw + cw / 2, y + rh / 2, txt, ha="center", va="center", fontsize=10)
yl = 0.74
for xx, grp, t in [(0.10, "A", "matches the Neurology abstract"),
(3.30, "T", "the paper's body"), (5.25, "O", "a different model")]:
ax.add_patch(Rectangle((xx, yl - 0.09), 0.22, 0.18, fc=fill[grp], ec=edge[grp], lw=1))
ax.text(xx + 0.30, yl, t, va="center", fontsize=10, color=TXT2)
ax.text(0.10, 0.30, "*Alzheimer's & Dementia 2024, doi:10.1002/alz.086295 (aMC n = 204, sMC n = 115;\n"
"adjusted by mutation mean AAO). The Neurology paper reports aMC n = 284, sMC n = 155.",
fontsize=10, color=TXT2, va="center", linespacing=1.2)
save(fig, stem, sub)
# ------------------------------------------------------------------ E: Wald z
def cand_E(stem="cand_E", sub="candidates"):
fig, ax = plt.subplots(figsize=(7.2, 3.6))
fig.subplots_adjust(left=0.34, right=0.97, top=0.86, bottom=0.17)
ys = {k: i + (0.4 if i >= 3 else 0) for i, k in enumerate(ROWS)}
ax.axvspan(-Z975, Z975, color="#efeeea", zorder=0)
ax.axvline(0, color=TXT2, lw=0.8)
for k in ROWS:
y = ys[k]
for src, col, mk, off, name in [(ABSTRACT, ORANGE, "o", -0.14, "Abstract"),
(TABLE2, BLUE, "D", 0.14, "Results text and Table 2")]:
_, _, z, p = wald(*src[k])
ax.plot(z, y + off, mk, ms=8, color=col, label=name if k == ROWS[0] else None)
ax.text(z + (0.25 if z > 0 else -0.25), y + off, f"p≈{p:.3f}" if p >= 0.001 else "p<0.001",
va="center", ha="left" if z > 0 else "right", fontsize=9.5, color=col)
ax.text(0, -0.45, "|z| < 1.96\n(p > 0.05)", ha="center", va="top", fontsize=10,
color=TXT2, linespacing=1.05)
ax.set_xlim(-10.5, 7.5)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_yticks([ys[k] for k in ROWS])
ax.set_yticklabels([("Asympt. " if k.startswith("a") else "Sympt. ") + LABEL[k] for k in ROWS],
color=TXT)
ax.set_ylim(max(ys.values()) + 0.6, -1.0)
ax.set_xlabel("Approximate Wald z = ln(estimate) / SE, SE from the printed 95% CI")
ax.legend(loc="lower center", bbox_to_anchor=(0.45, 1.0), ncol=2, frameon=False)
save(fig, stem, sub)
if __name__ == "__main__":
check_quotes()
report()
print()
cand_A("fig1", "")
cand_B("fig2", "")
if "--candidates" in sys.argv:
for f in (cand_A, cand_B, cand_C, cand_D, cand_E):
f()
Abstract reports 6-month BMI difference versus prednisone as 12-month result, and reuses it for height
Neurology (2026) · doi:10.1212/WNL.0000000000214756 · PMC13225239
Summary
The abstract gives the 12-month least-squares-mean (LSM) difference in BMI Z-score between vamorolone 6 mg/kg/d and prednisone as 0.57 [95% CI 0.24–0.90]. In the paper's Table 3, those three numbers are the 6-month value. The 12-month value is 0.24 (−0.04 to 0.52; p = 0.09), and its confidence interval includes zero. The abstract then gives the same triple, 0.57 [0.24–0.90], as the 12-month height Z-score difference versus prednisone, where Table 3 reports 0.44 (0.25 to 0.64). The deflazacort values in the same two sentences match Table 3 exactly, and the Results text and the Discussion agree with Table 3. So the error is confined to the two prednisone figures in the abstract. Those figures make the 12-month BMI excess on vamorolone look 2.4 times larger than the paper's own estimate, and statistically significant when it is not.
What the paper reports
Abstract, Results (BMI):
Body mass index (BMI) Z-scores increased in all groups, greatest with vamorolone (12-month LSM difference [95% CI]: vamorolone vs prednisone 0.57 [0.24–0.90]; vamorolone vs deflazacort 0.29 [0.03–0.56])
Abstract, Results (height), same sentence:
(12-month LSM difference in height Z-scores [95% CI]: vamorolone vs prednisone 0.57 [0.24–0.90]; vamorolone vs deflazacort 0.72 [0.53–0.91])
Table 3, "Comparisons of LSM Changes From Baseline Between Treatments at 6 and 12 Months for Anthropometric Outcomes Performed Using Weighted Data". Columns: n; LSM difference; 95% CI; p, at 6 mo and then at 12 mo:
BMI Z-score | Vam 6/Pred | 26/48 | 0.57; 0.24, 0.90 | 0.009 | 26/48 | 0.24; -0.04, 0.52 | 0.09
BMI Z-score | Vam 6/DFZ | 26/54 | 0.32; 0.00, 0.64 | 0.05 | 26/51 | 0.29; 0.03, 0.56 | 0.03
Height Z-score | Vam 6/Pred | 26/48 | 0.25; 0.14, 0.36 | <0.0001 | 26/48 | 0.44; 0.25, 0.64 | <0.0001
Height Z-score | Vam 6/DFZ | 26/51 | 0.50; 0.39, 0.61 | <0.0001 | 26/51 | 0.72; 0.53, 0.91 | <0.0001
Results, "Anthropometric Analyses" (labels 0.57 as the 6-month value):
LSM estimates of BMI Z-scores increased in all treatment groups at 12 months, with a decrease in BMI observed for prednisone at 6 months (prednisone vs vamorolone 6 mg/kg/d at 6 months, BMI Z-score LSM difference 0.57; 95% CI 0.24–0.90; p = 0.009) and a smaller increase for deflazacort compared with vamorolone at 12 months (LSM difference 0.29; 95% CI 0.03–0.56; p = 0.03) (Table 3; Figure 3A).
Discussion, paragraph 4:
To this end, we found that the increase in the BMI Z-score from baseline was greater on vamorolone than prednisone at 6 but not at 12 months.
What the paper's own numbers show
Method: each abstract triple (estimate, lower, upper) was compared for exact equality with all 12 cells of Table 3. Each cell was then checked for internal consistency. The standard error was recovered from the CI width as (upper − lower) / (2 × 1.96), and the two-sided p was recomputed from z = estimate / SE under a normal approximation. This is a check only: the paper's mixed model uses Kenward-Roger degrees of freedom, which give slightly larger p values. All calculations are in recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| BMI Z-score, vamorolone 6 vs prednisone, "12-month" | Abstract | 0.57 [0.24–0.90] | Table 3 at 12 mo: 0.24 (−0.04, 0.52). The triple equals only the 6-mo BMI cell |
| BMI Z-score, vamorolone 6 vs prednisone, 12 mo | Table 3 | 0.24; −0.04, 0.52; p = 0.09 | SE 0.143, z 1.68, p = 0.093 (consistent); CI includes 0 |
| BMI Z-score, vamorolone 6 vs prednisone, 6 mo | Table 3; Results text | 0.57; 0.24, 0.90; p = 0.009 | SE 0.168, z 3.39, p ≈ 0.0007 (see note) |
| Abstract minus Table 3, BMI vs prednisone, 12 mo | Abstract vs Table 3 | 0.57 vs 0.24 | Difference 0.33; ratio 2.38 |
| Height Z-score, vamorolone 6 vs prednisone, "12-month" | Abstract | 0.57 [0.24–0.90] | Table 3 at 12 mo: 0.44 (0.25, 0.64), and at 6 mo: 0.25. The triple equals the 6-mo BMI cell |
| Height Z-score, vamorolone 6 vs prednisone, 12 mo | Table 3 | 0.44; 0.25, 0.64; p < 0.0001 | SE 0.099, z 4.42, p ≈ 1e−5 (consistent); abstract overstates by 0.13 |
| BMI Z-score, vamorolone 6 vs deflazacort, 12 mo | Abstract; Table 3 | 0.29 [0.03–0.56]; p = 0.03 | Abstract equals Table 3 12-mo cell; p = 0.032 (consistent) |
| Height Z-score, vamorolone 6 vs deflazacort, 12 mo | Abstract; Table 3 | 0.72 [0.53–0.91]; p < 0.0001 | Abstract equals Table 3 12-mo cell; p ≈ 1e−13 (consistent) |
Every Table 3 CI midpoint equals its point estimate to rounding (for example 0.240 for 0.24, and 0.445 for 0.44). The table's columns are therefore not misaligned. Of the four comparisons the abstract labels "12-month", the two deflazacort ones match their 12-month cells and the two prednisone ones match neither 12-month cell.
Note on a separate, minor point: for the 6-month BMI cell, the CI 0.24–0.90 implies p ≈ 0.0007 under a normal approximation. With a t reference distribution, it implies p ≤ 0.003 for any df of 10 or more. The reported p = 0.009 would need about 4 degrees of freedom. This affects only the precision of a p value whose result is significant either way, and it is not the subject of this write-up.
Figure
Figure 1. The two abstract values labelled "12-month" for vamorolone 6 mg/kg/d versus prednisone (left, orange) set against Table 3 of the same paper (right, blue). Solid arrows join identical numbers: both abstract values (BMI and height) equal the 6-month BMI cell, 0.57; 0.24, 0.90. Dashed arrows lead to the 12-month cells the abstract label refers to: BMI 0.24 (−0.04 to 0.52, p = 0.09) and height 0.44 (0.25 to 0.64). All numbers are quoted from the article.
Figure 2. The same contrasts drawn on one LSM-difference axis, with a reference line at 0 (no difference). For BMI, the abstract's "12-month" interval (orange) lies exactly on the Table 3 6-month interval (blue diamond). The Table 3 12-month interval (blue dot) is shifted left and crosses 0. For height, the abstract interval is wider than and offset from both Table 3 height intervals. Values are from the paper; recomputed p values are given in the table above.
Why it matters
Steroid-related weight gain is a main safety concern in Duchenne muscular dystrophy, and comparing weight status with prednisone is one of the paper's key anthropometric results. Read alone, the abstract implies that at 12 months vamorolone raised BMI Z-score clearly more than prednisone did (0.57, CI excluding 0). The paper's own 12-month estimate is less than half that size (0.24) and is not statistically significant (CI −0.04 to 0.52, p = 0.09). That matches the Discussion's statement that the excess was present "at 6 but not at 12 months". Many readers see only the abstract, and abstract values are what get indexed and carried into reviews and meta-analyses, so the overstatement can spread.
The height error leaves the direction and significance of the growth finding intact: Table 3 gives 0.44 (0.25 to 0.64), p < 0.0001. Only its size is overstated, by 0.13 Z-score units. The paper's headline conclusions are unaffected: similar motor efficacy, growth protection with vamorolone, and BMI increases on all three treatments. Vamorolone's 12-month BMI excess over deflazacort (0.29, 0.03 to 0.56) is correctly reported.
Explanations considered
- A different model, population or weighting in the abstract. This does not account for the numbers. The abstract describes the same entropy-balanced, weighted mixed model for repeated measures as Table 3, and its two deflazacort values in the same sentences match the Table 3 12-month cells exactly. Only the prednisone values differ, and they equal another cell of the same table.
- Unweighted data (Figure 3B). This does not account for the numbers. Figure 3B shows unweighted arithmetic mean changes per group with t-based CIs, not between-treatment LSM differences, so it cannot be the source of a 0.57 [0.24–0.90] contrast. That triple is reproduced exactly by one Table 3 cell.
- Misaligned or mislabelled Table 3 columns. Ruled out. Each Table 3 CI is centred on its estimate, each cell's p value matches its CI apart from the 6-month BMI cell (see note above), and the Results text labels 0.57 (0.24–0.90; p = 0.009) explicitly as the 6-month BMI comparison. The Discussion's "at 6 but not at 12 months" fits the table's 12-month value of 0.24 with p = 0.09.
- Different time points across the two trials (weeks 24/48 vs months 6/12). This does not account for the numbers. The paper aligns these visits as 6 and 12 months for all comparisons, and the deflazacort contrasts, built the same way, match.
- PMC conversion artefact. Ruled out. The publisher's typeset PDF (Neurology 2026;107:e214756) has the same abstract and Table 3 values, and the Europe PMC JATS XML shows proper minus signs in "0.24; −0.04, 0.52".
- A height value that is genuinely 0.57. Not supported. Table 3 gives the vamorolone-vs-prednisone height Z-score difference as 0.25 at 6 months and 0.44 at 12 months. The abstract's height triple is identical, digit for digit, to the 6-month BMI cell.
Status
As of 2026-10-23:
- Version of record: checked. The publisher's typeset PDF (14 pp., CC BY 4.0) was obtained through the Newcastle University ePrints repository, record 312342, because neurology.org returned 403. Its abstract and Table 3 match the PMC text quoted above. The PMC archive copy was read with the project's
show.py. - Supplement: the PMC/Europe PMC archive includes no supplementary files. According to the text, eTable 1 and eFigures 1–4 cover cohort balancing and the vamorolone 2 mg/kg/d vs intermittent prednisone comparison, not these contrasts. The supplement could not be opened (neurology.org returned 403). It is not needed, because Table 3 of the main text holds the values in question.
- Corrections: a publisher correction exists for this article: doi:10.1212/WNL.0000000000218498, Neurology 2026;107(5):e218498, online 2026-07-30 (Crossref "update-to" this article; PubMed type "Published Erratum", PMID 42531535; PMC records a correction-forward link). Its text could not be retrieved: neurology.org returned 403, PubMed and Europe PMC list no abstract, and the notice is not in PMC. It is therefore unverified whether this correction addresses the abstract values described here. The PMC copy of the article still shows the values quoted above.
- PubPeer: a search for the DOI was attempted, but pubpeer.com required human verification and returned no results to this check. A web search found no PubPeer thread or other public comment on these values.
- Sources: PMC13225239 (PubMed Central JATS); Europe PMC JATS XML; publisher PDF via eprints.ncl.ac.uk/312342; Crossref metadata for 10.1212/WNL.0000000000214756 and 10.1212/WNL.0000000000218498.
Suggested correction
Unless the existing correction (e218498) already covers it, the abstract's Results sentence could be amended to the 12-month values in Table 3:
Body mass index (BMI) Z-scores increased in all groups, greatest with vamorolone (12-month LSM difference [95% CI]: vamorolone vs prednisone 0.24 [−0.04 to 0.52]; vamorolone vs deflazacort 0.29 [0.03–0.56]), whereas growth slowdown occurred in prednisone- and deflazacort-treated boys but not vamorolone (12-month LSM difference in height Z-scores [95% CI]: vamorolone vs prednisone 0.44 [0.25–0.64]; vamorolone vs deflazacort 0.72 [0.53–0.91]).
Because the 12-month BMI difference versus prednisone is not statistically significant, the phrase "greatest with vamorolone" could also be qualified, for example "greater with vamorolone than deflazacort at 12 months, and than prednisone at 6 months". That wording would match the paper's own Discussion. Separately, the authors could check the 6-month BMI p value (0.009) in Table 3 and the Results text against its CI (0.24–0.90).
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC13225239 write-up and regenerate its figures.
Paper: "Vamorolone for Duchenne Muscular Dystrophy", Neurology 2026,
doi:10.1212/WNL.0000000000214756 (PMC13225239).
All inputs are copied from the article as archived in PubMed Central: Table 3
(LSM differences, 95% CI and p values at 6 and 12 months) and the Abstract
(Results paragraph). Run:
python recompute.py.txt
Figures are written next to this file: five candidate designs (cand_A ... cand_E,
SVG + PNG at 200 dpi), and fig1/fig2, which are copies of the currently chosen
candidates (see CHOSEN below).
"""
import os
import shutil
import textwrap
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Rectangle
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = stats.norm.ppf(0.975)
# ---------------------------------------------------------------------------
# Inputs reported in the paper
# ---------------------------------------------------------------------------
# Table 3: (outcome, comparison, month) -> (LSM difference, CI low, CI high, reported p)
T3 = {
("BMI", "Pred", 6): (0.57, 0.24, 0.90, "0.009"),
("BMI", "Pred", 12): (0.24, -0.04, 0.52, "0.09"),
("BMI", "DFZ", 6): (0.32, 0.00, 0.64, "0.05"),
("BMI", "DFZ", 12): (0.29, 0.03, 0.56, "0.03"),
("Weight", "Pred", 6): (0.61, 0.34, 0.89, "<0.0001"),
("Weight", "Pred", 12): (0.58, 0.33, 0.82, "<0.0001"),
("Weight", "DFZ", 6): (0.60, 0.33, 0.87, "<0.0001"),
("Weight", "DFZ", 12): (0.78, 0.54, 1.02, "<0.0001"),
("Height", "Pred", 6): (0.25, 0.14, 0.36, "<0.0001"),
("Height", "Pred", 12): (0.44, 0.25, 0.64, "<0.0001"),
("Height", "DFZ", 6): (0.50, 0.39, 0.61, "<0.0001"),
("Height", "DFZ", 12): (0.72, 0.53, 0.91, "<0.0001"),
}
# Abstract, Results: values the abstract labels "12-month LSM difference [95% CI]"
ABSTRACT = {
("BMI", "Pred"): (0.57, 0.24, 0.90),
("BMI", "DFZ"): (0.29, 0.03, 0.56),
("Height", "Pred"): (0.57, 0.24, 0.90),
("Height", "DFZ"): (0.72, 0.53, 0.91),
}
NAME = {"Pred": "prednisone", "DFZ": "deflazacort"}
def wald(est, lo, hi):
"""SE from the CI width (normal approximation), z, two-sided p, CI midpoint."""
se = (hi - lo) / (2 * Z975)
z = est / se
return se, z, 2 * stats.norm.sf(abs(z)), (lo + hi) / 2
def fmt_p(p):
return f"{p:.2g}" if p >= 1e-4 else f"{p:.1e}"
# ---------------------------------------------------------------------------
# 1. Internal consistency of every Table 3 cell
# ---------------------------------------------------------------------------
print("1. Table 3 internal consistency (SE from CI width, normal approximation)")
print(f" {'cell':24s} {'est':>5s} {'mid':>6s} {'SE':>6s} {'z':>6s} {'p_recomp':>9s} reported p")
for (o, c, m), (e, lo, hi, p) in T3.items():
se, z, pr, mid = wald(e, lo, hi)
print(f" {o+' vam6/'+c+' '+str(m)+' mo':24s} {e:5.2f} {mid:6.3f} {se:6.3f} {z:6.2f} {fmt_p(pr):>9s} {p}")
# ---------------------------------------------------------------------------
# 2. Which Table 3 cell does each abstract "12-month" triple equal?
# ---------------------------------------------------------------------------
print("\n2. Abstract '12-month' values vs Table 3")
for (o, c), trip in ABSTRACT.items():
hits = [f"{k[0]} vs {k[1]} {k[2]} mo" for k, v in T3.items() if v[:3] == trip]
t12 = T3[(o, c, 12)][:3]
status = "matches 12-mo cell" if trip == t12 else "DOES NOT match 12-mo cell"
print(f" {o} vs {NAME[c]:11s}: abstract {trip}; Table 3 12-mo {t12} -> {status}; "
f"identical to Table 3 cell(s): {hits}")
a, t12, t6 = ABSTRACT[("BMI", "Pred")], T3[("BMI", "Pred", 12)], T3[("BMI", "Pred", 6)]
print("\n3. Size of the discrepancies")
print(f" BMI vs prednisone: abstract {a[0]} - Table 3 12-mo {t12[0]} = {a[0]-t12[0]:.2f}; "
f"ratio {a[0]/t12[0]:.2f}")
print(f" BMI vs prednisone, 12-mo CI ({t12[1]}, {t12[2]}) contains 0: {t12[1] < 0 < t12[2]}; "
f"abstract CI ({a[1]}, {a[2]}) contains 0: {a[1] < 0 < a[2]}")
ah, h12, h6 = ABSTRACT[("Height", "Pred")], T3[("Height", "Pred", 12)], T3[("Height", "Pred", 6)]
print(f" Height vs prednisone: abstract {ah[0]} - Table 3 12-mo {h12[0]} = {ah[0]-h12[0]:.2f}; "
f"Table 3 6-mo height value is {h6[0]}, so the abstract height triple equals neither height cell")
se12, z12, p12, _ = wald(*t12[:3])
print(f" BMI vs prednisone 12 mo: SE {se12:.3f}, z {z12:.2f}, recomputed p {p12:.3f} (reported {t12[3]})")
sea, za, pa, _ = wald(*a)
print(f" Abstract triple 0.57 [0.24-0.90] taken at face value: SE {sea:.3f}, z {za:.2f}, p {pa:.5f}")
# ---------------------------------------------------------------------------
# 4. Secondary observation: 6-month BMI vs prednisone, reported p = 0.009
# ---------------------------------------------------------------------------
print("\n4. 6-month BMI vs prednisone: what would reconcile p = 0.009 with 0.57 (0.24, 0.90)?")
e, lo, hi, _ = T3[("BMI", "Pred", 6)]
half = (hi - lo) / 2
for df in (3, 4, 5, 10, 20, 30, 50, 70):
tc = stats.t.ppf(0.975, df)
se = half / tc
t = e / se
print(f" t-based CI with df={df:3d}: SE {se:.3f}, t {t:.2f}, two-sided p {2*stats.t.sf(t, df):.4f}")
print(" (p = 0.009 needs roughly df = 4; with df of 10 or more (26 + 48 boys) p is at most"
" 0.003. This does not affect the abstract error.)")
# ---------------------------------------------------------------------------
# Figures
# ---------------------------------------------------------------------------
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
LW = 1.5 # points; 2 px at 96 px/in
MS = 8 # marker size in points (>= 8 px)
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def ci_str(v):
e, lo, hi = v[:3]
f = lambda x: f"{x:.2f}".replace("-", "−")
return f"{f(e)} [{f(lo)} to {f(hi)}]"
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), bbox_inches="tight", pad_inches=0.12, **kw)
plt.close(fig)
def recessive(ax, axis="x"):
ax.grid(axis=axis, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
# --- A: two text panels joined by annotated connectors ----------------------
def cand_A():
fig = plt.figure(figsize=(7.2, 5.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
def panel(x, y, w, h, title, sub, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.006,rounding_size=0.012",
fc="white", ec=edge, lw=LW))
ax.text(x + 0.018, y + h - 0.03, title, fontsize=11, weight="bold", va="top")
ax.text(x + 0.018, y + h - 0.085, sub, fontsize=10, color=INK2, va="top")
PX0, PX1, PW = 0.02, 0.58, 0.40
panel(PX0, 0.17, PW, 0.79, "Abstract (Results)", "Values labelled \u201c12-month\u201d", ORANGE)
q1 = ("\u201cBMI Z-scores increased in all groups, greatest with vamorolone "
"(12-month LSM difference [95% CI]: vamorolone vs prednisone")
q2 = "\u201c(12-month LSM difference in height Z-scores [95% CI]: vamorolone vs prednisone"
ax.text(0.038, 0.79, textwrap.fill(q1, 40), fontsize=10, va="top", style="italic")
ax.text(0.038, 0.615, "0.57 [0.24\u20130.90]\u201d", fontsize=11, va="center", color=ORANGE, weight="bold")
ax.text(0.038, 0.53, textwrap.fill(q2, 40), fontsize=10, va="top", style="italic")
ax.text(0.038, 0.405, "0.57 [0.24\u20130.90]\u201d", fontsize=11, va="center", color=ORANGE, weight="bold")
ax.text(0.038, 0.31, textwrap.fill("The paired deflazacort values (0.29 [0.03\u20130.56] and "
"0.72 [0.53\u20130.91]) match Table 3 at 12 months.", 40),
fontsize=10, color=INK2, va="top")
panel(PX1, 0.17, PW, 0.79, "Table 3 (weighted MMRM)", "Vamorolone 6 vs prednisone:\nLSM difference; 95% CI; p", BLUE)
rows = [("BMI Z-score, 6 mo", T3[("BMI", "Pred", 6)], 0.75),
("BMI Z-score, 12 mo", T3[("BMI", "Pred", 12)], 0.59),
("Height Z-score, 12 mo", T3[("Height", "Pred", 12)], 0.43)]
ycen = {}
for lab, v, y in rows:
ax.text(0.598, y, lab, fontsize=10, color=INK2, va="top")
p = "< " + v[3][1:] if v[3].startswith("<") else "= " + v[3]
ax.text(0.598, y - 0.065, f"{v[0]:.2f}; {v[1]:.2f}, {v[2]:.2f}".replace("-", "\u2212")
+ f" p {p}", fontsize=11, va="center", color=BLUE, weight="bold")
ycen[lab] = y - 0.065
ax.text(0.598, 0.31, textwrap.fill("The Results text and the Discussion agree with Table 3: "
"the BMI excess was present \u201cat 6 but not at 12 months\u201d.", 40),
fontsize=10, color=INK2, va="top")
def conn(y0, y1, rad, ls="-"):
ax.add_patch(FancyArrowPatch((PX0 + PW + 0.012, y0), (PX1 - 0.002, y1),
connectionstyle=f"arc3,rad={rad}", arrowstyle="-|>",
mutation_scale=13, lw=LW, color=INK, ls=ls, shrinkA=0, shrinkB=0))
ax.plot([0.25, PX0 + PW + 0.012], [y0, y0], color=INK, lw=LW, ls=ls)
conn(0.615, ycen["BMI Z-score, 6 mo"], -0.2)
conn(0.615, ycen["BMI Z-score, 12 mo"], 0.0, ls=(0, (4, 3)))
conn(0.405, ycen["BMI Z-score, 6 mo"], -0.45)
conn(0.405, ycen["Height Z-score, 12 mo"], 0.0, ls=(0, (4, 3)))
ax.text(0.02, 0.115, "Solid arrows: identical numbers. Both abstract values equal the 6-month BMI cell.",
fontsize=10)
ax.text(0.02, 0.07, "Dashed arrows: the 12-month cell each abstract label points to. BMI: 0.57 vs 0.24,",
fontsize=10)
ax.text(0.02, 0.025, "and the 12-month CI includes 0. Height: 0.57 vs 0.44.", fontsize=10)
save(fig, "cand_A")
# --- B: dot-and-interval, one axis, the two prednisone contrasts --------------
def cand_B():
fig, ax = plt.subplots(figsize=(7.2, 3.9))
items = [
("BMI", "Abstract, labelled 12 mo", ABSTRACT[("BMI", "Pred")], ORANGE, "o", None),
("BMI", "Table 3, 12 mo", T3[("BMI", "Pred", 12)], BLUE, "o", T3[("BMI", "Pred", 12)][3]),
("BMI", "Table 3, 6 mo", T3[("BMI", "Pred", 6)], BLUE, "D", T3[("BMI", "Pred", 6)][3]),
("Height", "Abstract, labelled 12 mo", ABSTRACT[("Height", "Pred")], ORANGE, "o", None),
("Height", "Table 3, 12 mo", T3[("Height", "Pred", 12)], BLUE, "o", T3[("Height", "Pred", 12)][3]),
("Height", "Table 3, 6 mo", T3[("Height", "Pred", 6)], BLUE, "D", T3[("Height", "Pred", 6)][3]),
]
ys = [6.2, 5.4, 4.6, 2.6, 1.8, 1.0]
for (o, lab, v, col, mk, p), y in zip(items, ys):
ax.plot([v[1], v[2]], [y, y], color=col, lw=LW, solid_capstyle="round")
ax.plot(v[0], y, marker=mk, ms=MS, color=col, mec=BG, mew=1.2,
mfc=col if mk == "o" else BG, zorder=3)
if mk == "D":
ax.plot(v[0], y, marker=mk, ms=MS - 1, color=col, mfc=BG, mew=LW, zorder=4)
txt = ci_str(v) + ((", p < " + p[1:] if p.startswith("<") else f", p = {p}") if p else "")
ax.text(1.02, y, txt, va="center", fontsize=10, color=INK)
ax.axvline(0, color=INK2, lw=0.8)
ax.set_yticks(ys)
ax.set_yticklabels([i[1] for i in items], fontsize=10)
for t, it in zip(ax.get_yticklabels(), items):
t.set_color(INK)
ax.text(-0.42, 6.85, "BMI Z-score, vamorolone 6 vs prednisone", fontsize=10.5, weight="bold",
transform=ax.transData, ha="left")
ax.text(-0.42, 3.25, "Height Z-score, vamorolone 6 vs prednisone", fontsize=10.5, weight="bold",
transform=ax.transData, ha="left")
ax.set_xlim(-0.42, 1.0); ax.set_ylim(0.4, 7.2)
ax.set_xlabel("LSM difference in Z-score from baseline (95% CI); 0 = no difference")
recessive(ax)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
h = [Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="Reported in Abstract as 12-month"),
Line2D([], [], color=BLUE, marker="o", ms=MS, lw=LW, label="Paper's Table 3, 12 months"),
Line2D([], [], color=BLUE, marker="D", ms=MS - 1, mfc=BG, lw=LW, label="Paper's Table 3, 6 months")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=3, frameon=False,
fontsize=10, handlelength=1.6, columnspacing=1.2)
save(fig, "cand_B")
# --- C: lookup map, each abstract triple wired to the Table 3 cell it equals --
def cand_C():
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
ax.text(0.03, 0.955, "Abstract: \u201c12-month LSM difference\u201d", weight="bold", fontsize=11)
ax.text(0.55, 0.955, "Table 3: LSM difference (95% CI)", weight="bold", fontsize=11)
cols = {12: 0.64, 6: 0.82} # 12-month column first, next to the abstract
W, H = 0.15, 0.12
rows = [("BMI", "Pred"), ("BMI", "DFZ"), ("Height", "Pred"), ("Height", "DFZ")]
ry = {r: 0.72 - i * 0.15 for i, r in enumerate(rows)}
for m, x in cols.items():
ax.text(x, 0.215 - 0.06, f"{m} months", ha="center", fontsize=10, color=INK2)
cell = {}
for r, y in ry.items():
for m, x in cols.items():
v = T3[(r[0], r[1], m)]
ax.add_patch(Rectangle((x - W / 2, y - H / 2), W, H, fc="white", ec=GRID, lw=1))
ax.text(x, y + 0.02, f"{v[0]:.2f}", ha="center", va="center", fontsize=10.5,
color=BLUE, weight="bold")
ax.text(x, y - 0.03, f"{v[1]:.2f}, {v[2]:.2f}".replace("-", "\u2212"), ha="center",
va="center", fontsize=10, color=BLUE)
cell[(r, m)] = (x, y)
for r in (("BMI", "Pred"), ("Height", "Pred")):
x, y = cell[(r, 12)]
ax.add_patch(Rectangle((x - W / 2, y - H / 2), W, H, fc="none", ec=INK, lw=LW, ls=(0, (3, 2))))
ax.text(0.64, 0.80, "expected", ha="center", fontsize=10, color=INK)
top = 0.905 # routing lane above the table
lane = {("BMI", "Pred"): 0.47, ("Height", "Pred"): 0.50}
for r, y in ry.items():
v = ABSTRACT[r]
ax.text(0.03, y + 0.022, f"{r[0]} vs {NAME[r[1]]}", fontsize=10, color=INK2, va="center")
ax.text(0.03, y - 0.028, ci_str(v), fontsize=10.5, color=ORANGE, weight="bold", va="center")
if T3[(r[0], r[1], 12)][:3] == v:
x, yy = cell[(r, 12)]
ax.add_patch(FancyArrowPatch((0.32, y), (x - W / 2 - 0.004, yy), arrowstyle="-|>",
mutation_scale=12, lw=LW, color=INK2))
ax.text(0.45, y + 0.018, "match", ha="center", fontsize=10, color=INK2)
else:
src = [k for k, t in T3.items() if t[:3] == v][0]
x, yy = cell[((src[0], src[1]), src[2])]
lx = lane[r]
ax.plot([0.32, lx, lx, x], [y, y, top, top], color=INK, lw=LW, solid_joinstyle="round")
ax.add_patch(FancyArrowPatch((x, top), (x, yy + H / 2 + 0.004), arrowstyle="-|>",
mutation_scale=12, lw=LW, color=INK, shrinkA=0))
ax.text(0.03, 0.09, textwrap.fill("Arrows lead to the Table 3 cell whose three numbers equal the "
"abstract value. Both prednisone values equal the 6-month BMI cell, "
"not the 12-month cells (dashed) that the label implies.", 80),
fontsize=10, va="center")
save(fig, "cand_C")
# --- D: all four abstract 12-month contrasts vs Table 3 12-month values ------
def cand_D():
fig, ax = plt.subplots(figsize=(7.2, 3.8))
order = [("BMI", "Pred"), ("BMI", "DFZ"), ("Height", "Pred"), ("Height", "DFZ")]
base = [4.0, 3.0, 2.0, 1.0]
for r, y in zip(order, base):
a, t = ABSTRACT[r], T3[(r[0], r[1], 12)]
for v, col, dy in ((a, ORANGE, 0.17), (t, BLUE, -0.17)):
ax.plot([v[1], v[2]], [y + dy] * 2, color=col, lw=LW, solid_capstyle="round")
ax.plot(v[0], y + dy, "o", ms=MS, color=col, mec=BG, mew=1.2, zorder=3)
d = a[0] - t[0]
note = "identical" if a == t[:3] else f"differ by {d:.2f}"
ax.text(1.12, y, note, va="center", fontsize=10, color=INK if d else INK2,
weight="bold" if d else "normal")
ax.axvline(0, color=INK2, lw=0.8)
ax.set_yticks(base)
ax.set_yticklabels([f"{r[0]} Z-score\nvs {NAME[r[1]]}" for r in order], fontsize=10, color=INK)
ax.set_xlim(-0.2, 1.1); ax.set_ylim(0.5, 4.6)
ax.set_xlabel("12-month LSM difference in Z-score, vamorolone 6 minus comparator (95% CI)")
recessive(ax); ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
h = [Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="Abstract (labelled 12-month)"),
Line2D([], [], color=BLUE, marker="o", ms=MS, lw=LW, label="Table 3, 12-month cell")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_D")
# --- E: trajectory 6 -> 12 months, with the abstract's 12-month claim overlaid -
def cand_E():
fig, ax = plt.subplots(figsize=(7.2, 4.0))
xs = {("BMI", 6): 0, ("BMI", 12): 1, ("Height", 6): 3.0, ("Height", 12): 4.0}
for o in ("BMI", "Height"):
v6, v12 = T3[(o, "Pred", 6)], T3[(o, "Pred", 12)]
ax.plot([xs[(o, 6)], xs[(o, 12)]], [v6[0], v12[0]], color=BLUE, lw=1.0, alpha=0.6)
for m, v in ((6, v6), (12, v12)):
x = xs[(o, m)]
ax.plot([x, x], [v[1], v[2]], color=BLUE, lw=LW)
ax.plot(x, v[0], "o", ms=MS, color=BLUE, mec=BG, mew=1.2, zorder=3)
ax.text(x - 0.07, v[0], f"{v[0]:.2f}", ha="right", va="center", fontsize=10, color=INK)
a = ABSTRACT[(o, "Pred")]
xa = xs[(o, 12)] + 0.75
ax.plot([xa, xa], [a[1], a[2]], color=ORANGE, lw=LW)
ax.plot(xa, a[0], "o", ms=MS, color=ORANGE, mec=BG, mew=1.2, zorder=3)
ax.text(xa + 0.07, a[0], f"{a[0]:.2f}", ha="left", va="center", fontsize=10, color=INK)
# the copied value: dotted guide from the 6-month BMI estimate
ax.plot([xs[("BMI", 6)], xs[("Height", 12)] + 0.75], [0.57, 0.57], color=INK2, lw=0.9, ls=(0, (2, 2)))
ax.text(2.4, 0.60, "0.57 [0.24–0.90] = 6-month BMI cell", ha="center", fontsize=10, color=INK2)
ax.axhline(0, color=INK2, lw=0.8)
ax.set_xticks([0, 1, 1.75, 3.0, 4.0, 4.75])
ax.set_xticklabels(["6 mo", "12 mo", "Abstract\n“12 mo”", "6 mo", "12 mo",
"Abstract\n“12 mo”"], fontsize=10)
ax.text(0.875, -0.36, "BMI Z-score", ha="center", fontsize=10.5, weight="bold", transform=ax.transData)
ax.text(3.875, -0.36, "Height Z-score", ha="center", fontsize=10.5, weight="bold", transform=ax.transData)
ax.set_xlim(-0.5, 5.3); ax.set_ylim(-0.12, 1.05)
ax.set_ylabel("LSM difference, vamorolone 6\nminus prednisone (95% CI)")
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="x", length=0, pad=4)
from matplotlib.lines import Line2D
h = [Line2D([], [], color=BLUE, marker="o", ms=MS, lw=LW, label="Paper's Table 3"),
Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="Reported in Abstract as 12-month")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.3), ncol=2, frameon=False, fontsize=10)
fig.subplots_adjust(bottom=0.2)
save(fig, "cand_E")
for f in (cand_A, cand_B, cand_C, cand_D, cand_E):
f()
CHOSEN = {"fig1": "cand_A", "fig2": "cand_B"}
for fig, cand in CHOSEN.items():
for ext in ("svg", "png"):
shutil.copyfile(os.path.join(HERE, f"{cand}.{ext}"), os.path.join(HERE, f"{fig}.{ext}"))
print("\nFigures written:", ", ".join(f"{c}.svg/.png" for c in "cand_A cand_B cand_C cand_D cand_E".split()),
"; fig1 =", CHOSEN["fig1"], "; fig2 =", CHOSEN["fig2"])
Abstract reports 73.5% of trials used biomarkers; the Results and phase counts give 56.3%
Neurology (2026) · doi:10.1212/WNL.0000000000218373 · PMC13445500
Summary
This systematic review of 238 phase II–IV Alzheimer disease drug trials states in its abstract that "73.5% (175/238) used at least one biomarker as efficacy measures". The Results section gives 56.3% (134/238) for the same quantity. The phase-stratified counts in the Results (79/135 + 55/103 = 134/238) and the Discussion's "more than half" agree with 134, not 175. The figure 175/238 (73.5%) appears in the Results as the count of trials that used clinical outcome measures as primary measures, so the abstract seems to carry that count in the biomarker position. The abstract overstates one of the review's headline descriptive results by 41 trials (17.2 percentage points). The paper's qualitative conclusion that biomarker use is widespread still holds at 56.3%.
What the paper reports
Abstract, Results:
Among 238 included trials, 95.4% (227/238) used at least one clinical outcome and 73.5% (175/238) used at least one biomarker as efficacy measures.
Abstract, Discussion:
while biomarker use is widespread
Results, "Efficacy Measures in AD Drug Trials" (two consecutive sentences):
Among 238 included trials, 95.4% (227/238) used at least one clinical outcome measure as an efficacy measure, of which 73.5% (175/238) used clinical outcome measures as primary measures and 84.5% (201/238) as secondary measures. In addition, 56.3% (134/238) of trials used at least one biomarker as an efficacy measure, of which 11.3% (27/238) used at least one biomarker as a primary measure and 41.2% (98/238) as a secondary measure.
The next sentence in the same paragraph:
Overall, 18.1% (43/238) of trials included neither a clinical outcome measure nor a biomarker as their primary measure.
Results, "Sample Characteristics":
Of 238 included trials, 56.7% (135/238) were phase II and 43.3% (103/238) were phase III or IV.
Results, "Clinical Relevance of Efficacy Measures", phase II and phase III–IV (domains assessed overall):
functional ability and dependency (66.7%, 90/135), and biomarkers (58.5%, 79/135).
functional ability and dependency (77.7%, 80/103), and biomarkers (53.4%, 55/103).
The same paragraph, restricted to primary measures (phase II, then phase III–IV):
followed by biomarkers (14.8%, 20/135)
and biomarkers (5.8%, 6/103)
Results, same section, and Abstract, Results:
Notably, 8.4% (20/238) used biomarkers as the only primary efficacy measures.
whereas 8.4% (20/238) used biomarkers as the sole primary efficacy measure
Discussion, first paragraph and third paragraph:
Moreover, more than half of the trials evaluated biomarkers, predominantly as secondary measures
more than half of trials used biomarkers as efficacy measures
Methods, "Data Extraction" and "Data Analysis":
We also extracted whether outcome measures were evaluated as primary and secondary outcome measures or any exploratory outcome measures.
We determined the proportion of trials that used clinical outcome measures or biomarkers as any efficacy measures, primary or secondary.
What the paper's own numbers show
Method. All counts share one denominator: the 238 included trials, split into 135 phase II and 103 phase III–IV trials. I recomputed every printed percentage from its printed count. I then rebuilt the number of trials with any biomarker use from the phase-stratified counts, compared it with the abstract and the Results, and checked which other Results quantity equals 175/238. As a further check I used the primary-measure counts: the trials with a clinical or biomarker primary measure (238 − 43), the clinical-primary count (175) and the biomarker-primary count (27). Together they imply an overlap, and from it a biomarker-only primary count that can be compared with the paper's 20/238. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Trials using ≥ 1 biomarker as an efficacy measure | Abstract, Results | 73.5% (175/238) | 175/238 = 73.53%: the arithmetic is right, but the count conflicts with the body |
| Trials using ≥ 1 biomarker as an efficacy measure | Results, Efficacy Measures | 56.3% (134/238) | 134/238 = 56.30% |
| Abstract minus Results | derived | — | 41 trials, 17.2 percentage points; the abstract count is 1.31 × the Results count |
| Trials using clinical outcomes as primary measures | Results, Efficacy Measures | 73.5% (175/238) | 175/238 = 73.53%, the same count and percentage the abstract gives for biomarkers |
| Phase II, any biomarker use | Results, Clinical Relevance | 58.5% (79/135) | 58.52% |
| Phase III–IV, any biomarker use | Results, Clinical Relevance | 53.4% (55/103) | 53.40% |
| Sum of the phase strata | derived | — | 135 + 103 = 238 trials; 79 + 55 = 134 biomarker trials (56.3%), equal to the Results and 41 below the abstract |
| Discussion: "more than half" used biomarkers | Discussion | — | half of 238 = 119; 134 fits "more than half". 175 is close to three-quarters (178.5) |
| Biomarker primary + secondary | Results, Efficacy Measures | 27/238 (11.3%) and 98/238 (41.2%) | 27 + 98 = 125, the most trials that could use a biomarker as a primary or secondary measure. 175 exceeds it by 50 |
| Trials with a clinical or biomarker primary measure | Results, Efficacy Measures | 43/238 (18.1%) with neither | 238 − 43 = 195 |
| Implied trials with both clinical and biomarker primary measures | derived | — | 175 + 27 − 195 = 7 |
| Implied biomarker-only primary measures | derived vs Results and Abstract | 8.4% (20/238) | 27 − 7 = 20, which matches. So 175 fits the paper's own figures as the clinical-primary count |
| Clinical outcome, any, primary and secondary use | Results, Efficacy Measures | 95.4% (227), 73.5% (175), 84.5% (201) | 95.38%, 73.53%, 84.45%; 175 + 201 − 227 = 149 trials with both |
Two smaller inconsistencies inside the Results came up. They do not bear on the choice between 134 and 175:
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Any biomarker use vs primary + secondary use | Results, Efficacy Measures | 134 vs 27 and 98 | 134 is 9 more than 27 + 98 = 125. The Methods say exploratory measures were also extracted, which could account for this, but the paper does not say so |
| Biomarker as primary measure, phase strata vs overall | Results, Clinical Relevance vs Efficacy Measures | 20/135 + 6/103 vs 27/238 | 20 + 6 = 26, one less than 27 |
Figure
Figure 1. The two conflicting passages, quoted from the paper. Left, the abstract's Results sentence. Right, the Results paragraph "Efficacy Measures in AD Drug Trials", shortened with ellipses. Reported values are in orange. The gray connector links the abstract's biomarker figure to the identical Results figure for clinical outcomes used as primary measures. The orange connector links it to the Results figure for biomarkers, which it contradicts. The blue panel gives the value recomputed from the paper's phase-stratified counts. It matches the Results (134/238, 56.3%), not the abstract.
Figure 2. The counts behind the conflict, on one axis of trials from 0 to 238. Orange circles are counts printed in the paper. Blue diamonds are counts recomputed from the paper's other figures: the sum of the phase II and phase III–IV biomarker counts (79 + 55 = 134), and the upper bound on primary or secondary biomarker use (27 + 98 = 125). The dashed line marks half of the 238 trials, the threshold of the Discussion's "more than half". The abstract's 175 equals the clinical-primary count and lies 41 trials above every other count of biomarker use in the paper.
Why it matters
The abstract's Results section opens with this figure, and for many readers the abstract is the only part of a review they read or quote. A secondary summary of the article, whose URL carries the date 25 July 2026, already repeats the abstract's figure ("73.5% of analyzed trials incorporated at least one biomarker as an efficacy endpoint").
- Size. The abstract overstates biomarker use by 41 trials, from just over half of trials to about three-quarters, a relative overstatement of 31%.
- Comparison inside the abstract. The abstract puts the two headline figures side by side: 95.4% of trials used clinical outcomes and 73.5% used biomarkers. Read together, they make the two kinds of efficacy measure look much closer in use than the body reports (95.4% vs 56.3%).
- Conclusion. The abstract's Discussion says "biomarker use is widespread". That remains a fair description at 56.3%, and the body and Discussion carry the correct value. The error changes the magnitude of a key descriptive result, not the direction of the paper's conclusions. None of the review's other headline figures (domains assessed, co-primary measures, 20/238 biomarker-only primaries, counts of distinct measures) depends on it.
Explanations considered
- The two figures measure different things. This does not account for the numbers. Both refer to the same 238 trials and use the same wording ("at least one biomarker as an efficacy measure"). Neither refers to a subgroup, a time point or primary-only use.
- The abstract counts exploratory measures and the Results do not. This does not account for the numbers. The Methods define the analysis as "any efficacy measures, primary or secondary", and both passages use the same phrase. The phase-stratified Results, which add up to 134, also cover all domains assessed. No passage gives a broader count than 134.
- The Results figure is the wrong one, and 175 is correct. This is contradicted by three independent figures in the paper. The phase counts sum to 134. The Discussion twice says "more than half", which would understate 73.5%. Primary plus secondary biomarker use is at most 125. Separately, 175 fits the clinical-primary count: with 195 trials having a clinical or biomarker primary measure, it implies 20 trials with biomarker-only primaries, exactly the 20/238 the paper reports.
- A conversion or typesetting artifact in the PMC copy. This is excluded as far as the available copies allow. The PMC article page, the Europe PMC full-text XML and the local PMC archive all carry the same wording and numbers. A secondary summary whose URL is dated 25 July 2026, a month before the 25 August 2026 issue date, also repeats 73.5%. This suggests the figure was already in the online-first version, although the page shows no explicit date.
- The supplement resolves it. This could not be checked. eTable 2 may list counts by type of efficacy measure, but the supplement was not available (see Status). The main text alone points to 134.
The smaller inconsistencies (134 vs 125; 26 vs 27) leave some uncertainty about the exact correct value, perhaps by a few trials. They do not bring it near 175.
Status
As of 2026-10-23:
- Version of record. The publisher's site (www.neurology.org, reached from the DOI) returned HTTP 403 for the article HTML, the PDF and the supplement, so I could not read the publisher's own rendering. The PMC article page (pmc.ncbi.nlm.nih.gov/articles/PMC13445500/), the Europe PMC full-text XML and the local PMC archive all carry the same abstract sentence (73.5%, 175/238) and the same Results sentence (56.3%, 134/238). The PMC-hosted publisher PDF (WNL-2026-300706.pdf) could not be downloaded, because the server returned an interstitial page. Every quotation above was checked verbatim against the PMC-archived text.
- Supplement. The article cites eTables and eFigures. Neither PMC nor Europe PMC includes the supplementary material, the publisher's supplement page returned 403, and a web search found no public copy. The Data Availability statement says aggregated data are available on request.
- Corrections and comments. I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction notices (PMID 42492033). PubMed links only to a companion editorial (PMID 42492030, doi:10.1212/WNL.0000000000218468), whose text I could not open. The PubPeer API returned no comments for the DOI, though its web search page refused automated access (HTTP 403). A web search for the title with "correction", "erratum" or "PubPeer" found nothing relevant.
- Sources. PMC article PMC13445500; Europe PMC full-text XML; Crossref; PubMed; PubPeer API; secondary summary at omnicuris.com (URL dated 25 July 2026).
Suggested correction
- In the abstract's Results, replace "73.5% (175/238) used at least one biomarker as efficacy measures" with "56.3% (134/238) used at least one biomarker as efficacy measures", if the authors' data confirm the Results figure.
- Reconcile the any-use count (134) with the primary and secondary counts (27 and 98, total 125). For example, state whether exploratory measures are included in "any" use. Also reconcile the phase-level biomarker-primary counts (20 + 6 = 26) with the overall 27.
- Check any figure or supplementary table that reports overall biomarker use (for example eTable 2) against the corrected value.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC13445500 write-up and regenerate its figures.
Paper: "Efficacy Measures Used in Alzheimer Disease Clinical Trials Between 2015
and 2025", Neurology 2026, doi:10.1212/WNL.0000000000218373 (PMC13445500).
Run: python recompute.py.txt
All inputs below are counts printed in the paper (location given for each).
Outputs: fig1.svg/png, fig2.svg/png and five candidate figure designs in
candidates/ (cand_A ... cand_E), all written next to this file.
"""
import os
import shutil
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs
N = 238 # included trials (Abstract; Results)
N_PH2, N_PH34 = 135, 103 # Results, Sample Characteristics
ABS_BIO_ANY = 175 # Abstract: "73.5% (175/238) used at least one biomarker"
ABS_BIO_PCT = 73.5
RES_BIO_ANY = 134 # Results: "56.3% (134/238) ... at least one biomarker"
RES_BIO_PCT = 56.3
RES_CLIN_ANY = 227 # Results: 95.4% (227/238) at least one clinical outcome
RES_CLIN_PRIM = 175 # Results: 73.5% (175/238) clinical outcomes as primary
RES_CLIN_SEC = 201 # Results: 84.5% (201/238) clinical outcomes as secondary
RES_BIO_PRIM = 27 # Results: 11.3% (27/238) biomarker as primary
RES_BIO_SEC = 98 # Results: 41.2% (98/238) biomarker as secondary
RES_NEITHER_PRIM = 43 # Results: 18.1% (43/238) neither as primary
RES_BIO_ONLY_PRIM = 20 # Results & Abstract: 8.4% (20/238) biomarkers only primary
PH2_BIO_ANY, PH34_BIO_ANY = 79, 55 # Results: 58.5% (79/135), 53.4% (55/103)
PH2_BIO_PRIM, PH34_BIO_PRIM = 20, 6 # Results: 14.8% (20/135), 5.8% (6/103)
def pct(k, n=N):
return 100.0 * k / n
def show(label, value, reported=None):
s = f"{label:<64s} {value}"
if reported is not None:
s += f" (paper: {reported})"
print(s)
print("=" * 100)
print("1. Percentages as printed")
show("Abstract biomarker any use 175/238", f"{pct(ABS_BIO_ANY):.2f}%", "73.5%")
show("Results biomarker any use 134/238", f"{pct(RES_BIO_ANY):.2f}%", "56.3%")
show("Results clinical outcome any use 227/238", f"{pct(RES_CLIN_ANY):.2f}%", "95.4%")
show("Results clinical outcome as primary 175/238", f"{pct(RES_CLIN_PRIM):.2f}%", "73.5%")
show("Results clinical outcome as secondary 201/238", f"{pct(RES_CLIN_SEC):.2f}%", "84.5%")
show("Results biomarker as primary 27/238", f"{pct(RES_BIO_PRIM):.2f}%", "11.3%")
show("Results biomarker as secondary 98/238", f"{pct(RES_BIO_SEC):.2f}%", "41.2%")
show("Results neither as primary 43/238", f"{pct(RES_NEITHER_PRIM):.2f}%", "18.1%")
show("Results biomarkers as only primary 20/238", f"{pct(RES_BIO_ONLY_PRIM):.2f}%", "8.4%")
show("Results phase II biomarker any use 79/135", f"{pct(PH2_BIO_ANY, N_PH2):.2f}%", "58.5%")
show("Results phase III-IV biomarker any use 55/103", f"{pct(PH34_BIO_ANY, N_PH34):.2f}%", "53.4%")
show("Results phase II biomarker primary 20/135", f"{pct(PH2_BIO_PRIM, N_PH2):.2f}%", "14.8%")
show("Results phase III-IV biomarker primary 6/103", f"{pct(PH34_BIO_PRIM, N_PH34):.2f}%", "5.8%")
print("\n2. Abstract vs Results")
gap = ABS_BIO_ANY - RES_BIO_ANY
show("Gap in trials (175 - 134)", gap)
show("Gap in percentage points", f"{pct(gap):.2f}")
show("Ratio abstract / Results (175/134)", f"{ABS_BIO_ANY / RES_BIO_ANY:.3f}")
show("Abstract biomarker count == Results clinical-primary count?",
ABS_BIO_ANY == RES_CLIN_PRIM)
print("\n3. Phase subtotals")
show("Phase denominators 135 + 103", N_PH2 + N_PH34)
ph_sum = PH2_BIO_ANY + PH34_BIO_ANY
show("Phase biomarker any-use 79 + 55", ph_sum)
show(" as % of 238", f"{pct(ph_sum):.2f}%")
show(" equals Results 134?", ph_sum == RES_BIO_ANY)
show(" equals Abstract 175?", ph_sum == ABS_BIO_ANY)
show("Phase biomarker primary 20 + 6", PH2_BIO_PRIM + PH34_BIO_PRIM)
show(" vs overall biomarker primary 27, difference",
RES_BIO_PRIM - (PH2_BIO_PRIM + PH34_BIO_PRIM))
print("\n4. Bounds from primary/secondary counts")
ub = RES_BIO_PRIM + RES_BIO_SEC
show("Biomarker primary + secondary (27 + 98), max primary-or-secondary use", ub)
show(" Abstract 175 exceeds this by", ABS_BIO_ANY - ub)
show(" Results 134 exceeds this by", RES_BIO_ANY - ub)
show("Clinical: primary + secondary - any (175 + 201 - 227) = both", RES_CLIN_PRIM + RES_CLIN_SEC - RES_CLIN_ANY)
any_prim = N - RES_NEITHER_PRIM
show("Trials with a clinical or biomarker primary measure (238 - 43)", any_prim)
both_prim = RES_CLIN_PRIM + RES_BIO_PRIM - any_prim
show(" implied both clinical and biomarker primary (175 + 27 - 195)", both_prim)
show(" implied biomarker-only primary (27 - 7)", RES_BIO_PRIM - both_prim, "20/238")
show("Half of 238 (Discussion: 'more than half')", N / 2)
show("Three-quarters of 238", 0.75 * N)
# ---------------------------------------------------------------- figure style
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
ORANGE = "#eb6834" # values as reported in the paper
BLUE = "#2a78d6" # values recomputed from the paper's own numbers
GRID = "#e4e3df"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans",
"font.size": 10.5,
"text.color": INK,
"axes.labelcolor": INK2,
"axes.edgecolor": INK2,
"xtick.color": INK2,
"ytick.color": INK2,
"figure.facecolor": BG,
"axes.facecolor": BG,
"savefig.facecolor": BG,
"lines.linewidth": 2,
"svg.hashsalt": "PMC13445500",
})
def save(fig, name, folder=CAND):
for ext in ("svg", "png"):
fig.savefig(os.path.join(folder, f"{name}.{ext}"), dpi=200,
bbox_inches="tight", pad_inches=0.12,
metadata={"Date": None} if ext == "svg" else None)
plt.close(fig)
def box(ax, x, y, w, h, edge=INK2, lw=1.2, fill="#ffffff"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc=fill, ec=edge, lw=lw, transform=ax.transAxes))
def arrow(ax, p0, p1, color=INK2, rad=0.0, style="-|>", lw=2, ls="-"):
ax.add_patch(FancyArrowPatch(p0, p1, transform=ax.transAxes, arrowstyle=style,
mutation_scale=14, color=color, lw=lw, ls=ls,
connectionstyle=f"arc3,rad={rad}", shrinkA=2, shrinkB=2))
def legend_handles():
return [Line2D([], [], marker="o", ls="", ms=9, mfc=ORANGE, mec=ORANGE,
label="Reported in the paper"),
Line2D([], [], marker="D", ls="", ms=8, mfc=BLUE, mec=BLUE,
label="Recomputed from the paper's own counts")]
# ======================================================= Candidate A: text panels
def cand_A():
fig = plt.figure(figsize=(W, 5.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.text(0.02, 0.975, "The abstract's biomarker count is the Results' clinical-primary count",
fontsize=12, weight="bold", va="top")
step = 0.052
def lines(x, y0, items):
"""items: (text, kind); kind 'h' header, 'q' quote, 'v' reported value."""
y = y0
pos = {}
for t, kind in items:
if kind == "h":
ax.text(x, y, t, fontsize=10.5, weight="bold", color=INK2, va="top")
elif kind == "v":
ax.text(x, y + 0.004, t, fontsize=12, weight="bold", color=ORANGE, va="top")
pos[t] = y - 0.022
else:
ax.text(x, y, t, fontsize=10.5, color=INK, va="top")
y -= step
return pos
PT, PB = 0.91, 0.30
box(ax, 0.02, PB, 0.37, PT - PB)
box(ax, 0.61, PB, 0.37, PT - PB)
pl = lines(0.04, 0.885, [("Abstract, Results", "h"), ("", "q"),
("\u201c95.4% (227/238) used at least", "q"),
("one clinical outcome and", "q"),
("73.5% (175/238)", "v"),
("used at least one biomarker", "q"),
("as efficacy measures.\u201d", "q")])
pr = lines(0.63, 0.885, [("Results, Efficacy Measures", "h"),
("in AD Drug Trials", "h"),
("\u201c... of which", "q"),
("73.5% (175/238)", "v"),
("used clinical outcome measures", "q"),
("as primary measures ...", "q"),
("In addition,", "q"),
("56.3% (134/238)", "v"),
("of trials used at least one bio-", "q"),
("marker as an efficacy measure\u201d", "q")])
yl = 0.885 - 4 * step - 0.022 # left orange value
yr1 = 0.885 - 3 * step - 0.022 # right 175
yr2 = 0.885 - 7 * step - 0.022 # right 134
arrow(ax, (0.30, yl + 0.012), (0.625, yr1), color=INK2, rad=-0.2)
ax.text(0.50, yr1 + 0.075, "same count,\ndifferent measure", fontsize=10, color=INK2,
ha="center", va="bottom", linespacing=1.2)
arrow(ax, (0.30, yl - 0.012), (0.625, yr2), color=ORANGE, rad=0.2, style="<|-|>")
ax.text(0.50, yr2 - 0.075, "conflict:\n175 vs 134", fontsize=10.5, weight="bold",
color=ORANGE, ha="center", va="top", linespacing=1.2)
# recomputation strip
box(ax, 0.02, 0.03, 0.96, 0.22, edge=BLUE, lw=1.6)
ax.text(0.04, 0.225, "Recomputed from the paper's own counts", fontsize=10.5, weight="bold",
color=BLUE, va="top")
ax.text(0.04, 0.165,
f"Phase II {PH2_BIO_ANY}/135 + phase III\u2013IV {PH34_BIO_ANY}/103 = "
f"{ph_sum}/238 = {pct(ph_sum):.1f}%, matching the Results",
fontsize=10.5, va="top")
ax.text(0.04, 0.105,
f"Abstract minus Results: {gap} trials, {pct(gap):.1f} percentage points",
fontsize=10.5, va="top")
save(fig, "cand_A_text_panels")
# ======================================================= Candidate B: dot comparison
def cand_B():
rows = [ # label, value, kind
("Abstract: ≥ 1 biomarker", ABS_BIO_ANY, "rep"),
("Results: ≥ 1 biomarker", RES_BIO_ANY, "rep"),
("Phase II + phase III–IV biomarker use (79 + 55)", ph_sum, "rec"),
("Biomarker primary + secondary (27 + 98), maximum", ub, "rec"),
("Results: clinical outcome as primary", RES_CLIN_PRIM, "rep"),
]
fig, ax = plt.subplots(figsize=(W, 3.5))
ys = list(range(len(rows)))[::-1]
for (lab, v, kind), y in zip(rows, ys):
c = ORANGE if kind == "rep" else BLUE
m = "o" if kind == "rep" else "D"
ax.plot([0, v], [y, y], color=GRID, lw=2, zorder=1, solid_capstyle="round")
ax.plot(v, y, marker=m, ms=10 if m == "o" else 9, color=c, mec=BG, mew=1.5, zorder=3)
ax.text(v + 4, y, f"{v} ({pct(v):.1f}%)", va="center", fontsize=10.5, color=INK)
ax.axvline(N / 2, color=INK2, lw=1, ls=(0, (3, 3)), zorder=0)
ax.text(N / 2 - 2, -0.42, "half of 238 (Discussion: “more than half”)",
fontsize=10, color=INK2, va="center", ha="right",
bbox=dict(fc=BG, ec="none", pad=1))
ax.set_yticks(ys, [r[0] for r in rows], fontsize=10.5)
ax.set_xlim(0, N); ax.set_ylim(-0.7, len(rows) - 0.4)
ax.set_xticks([0, 50, 100, 150, 200, 238])
ax.set_xlabel("Trials (of 238)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.annotate("", xy=(RES_BIO_ANY, ys[1] + 0.5), xytext=(ABS_BIO_ANY, ys[1] + 0.5),
arrowprops=dict(arrowstyle="<|-|>", color=ORANGE, lw=1.6))
ax.text((ABS_BIO_ANY + RES_BIO_ANY) / 2, ys[1] + 0.56, "41 trials", ha="center",
va="bottom", fontsize=10, color=INK, weight="bold")
ax.legend(handles=legend_handles(), loc="upper center", bbox_to_anchor=(0.35, -0.2),
ncol=2, frameon=False, fontsize=10)
save(fig, "cand_B_dot_comparison")
# ======================================================= Candidate C: calculation flow
def cand_C():
fig = plt.figure(figsize=(W, 3.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
def node(x, y, w, h, title, value, color, sub=None):
box(ax, x, y, w, h, edge=color, lw=1.8)
ax.text(x + w / 2, y + h - 0.045, title, ha="center", va="top", fontsize=10, color=INK2)
ax.text(x + w / 2, y + h / 2 - (0.02 if sub else 0.04), value, ha="center", va="center",
fontsize=12.5, weight="bold", color=INK)
if sub:
ax.text(x + w / 2, y + 0.05, sub, ha="center", va="bottom", fontsize=10, color=INK2)
node(0.02, 0.62, 0.22, 0.28, "Results, phase II", "79 / 135", ORANGE, "58.5%")
node(0.02, 0.22, 0.22, 0.28, "Results, phase III–IV", "55 / 103", ORANGE, "53.4%")
node(0.34, 0.42, 0.22, 0.28, "Recomputed sum", f"{ph_sum} / {N_PH2 + N_PH34}", BLUE,
f"{pct(ph_sum):.1f}%")
node(0.70, 0.62, 0.28, 0.28, "Results: ≥ 1 biomarker", "134 / 238", ORANGE, "56.3%")
node(0.70, 0.16, 0.28, 0.28, "Abstract: ≥ 1 biomarker", "175 / 238", ORANGE, "73.5%")
arrow(ax, (0.24, 0.76), (0.34, 0.60))
arrow(ax, (0.24, 0.36), (0.34, 0.52))
ax.text(0.29, 0.56, "+", fontsize=14, ha="center", va="center", color=INK2)
arrow(ax, (0.56, 0.60), (0.70, 0.74), color=BLUE)
ax.text(0.615, 0.715, "matches", fontsize=10.5, color=BLUE, weight="bold", ha="center")
arrow(ax, (0.56, 0.50), (0.70, 0.32), color=ORANGE, ls="--")
ax.text(0.555, 0.30, "does not match\n(41 trials fewer)", fontsize=10.5, color=ORANGE,
weight="bold", ha="center", va="top", linespacing=1.2)
ax.text(0.84, 0.035, "175/238 is the Results count of trials using\nclinical outcomes as primary measures",
fontsize=10, color=INK2, ha="center", va="bottom", linespacing=1.2)
ax.text(0.02, 0.975, "Blue: recomputed from the paper's counts. Orange: as printed in the paper.",
fontsize=10, color=INK2, va="top")
save(fig, "cand_C_calculation_flow")
# ======================================================= Candidate D: stacked bars
def cand_D():
fig, ax = plt.subplots(figsize=(W, 2.9))
rows = ["Abstract: ≥ 1 biomarker", "Results: ≥ 1 biomarker",
"Phase subtotals (recomputed)"]
y = [2, 1, 0]
h = 0.55
ax.barh(y[0], ABS_BIO_ANY, height=h, color=ORANGE)
ax.barh(y[1], RES_BIO_ANY, height=h, color=ORANGE)
ax.barh(y[2], PH2_BIO_ANY, height=h, color=BLUE, edgecolor=BG, linewidth=2)
ax.barh(y[2], PH34_BIO_ANY, left=PH2_BIO_ANY, height=h, color=BLUE, alpha=0.6,
edgecolor=BG, linewidth=2)
ax.text(PH2_BIO_ANY / 2, y[2], "phase II 79", ha="center", va="center", color="white",
fontsize=10, weight="bold")
ax.text(PH2_BIO_ANY + PH34_BIO_ANY / 2, y[2], "III–IV 55", ha="center", va="center",
color=INK, fontsize=10, weight="bold")
for yy, v in zip(y, [ABS_BIO_ANY, RES_BIO_ANY, ph_sum]):
ax.text(v + 3, yy, f"{v} ({pct(v):.1f}%)", va="center", fontsize=10.5)
ax.axvline(N / 2, color=INK2, lw=1, ls=(0, (3, 3)))
ax.text(N / 2 + 1.5, 2.55, "half of 238", fontsize=10, color=INK2, va="bottom")
ax.set_yticks(y, rows, fontsize=10.5)
ax.set_xlim(0, N); ax.set_ylim(-0.5, 2.9)
ax.set_xticks([0, 50, 100, 150, 200, 238]); ax.set_xlabel("Trials (of 238)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
hs = [plt.Rectangle((0, 0), 1, 1, color=ORANGE, label="Reported in the paper"),
plt.Rectangle((0, 0), 1, 1, color=BLUE, label="Recomputed from the paper's own counts")]
ax.legend(handles=hs, loc="upper center", bbox_to_anchor=(0.35, -0.28), ncol=2,
frameon=False, fontsize=10)
save(fig, "cand_D_stacked_bars")
# ======================================================= Candidate E: percentage strip
def cand_E():
fig, ax = plt.subplots(figsize=(W, 2.8))
yb, yc = 1, 0
ax.plot([0, 100], [yb, yb], color=GRID, lw=2, zorder=0)
ax.plot([0, 100], [yc, yc], color=GRID, lw=2, zorder=0)
ax.plot(ABS_BIO_PCT, yb, "o", ms=11, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.plot(RES_BIO_PCT, yb, "o", ms=11, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.plot(pct(ph_sum), yb - 0.17, "D", ms=9, color=BLUE, mec=BG, mew=1.5, zorder=3)
ax.plot(pct(RES_CLIN_PRIM), yc, "o", ms=11, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.text(ABS_BIO_PCT - 1, yb + 0.16, "Abstract 73.5%", ha="left", va="bottom", fontsize=10.5)
ax.text(RES_BIO_PCT + 1, yb + 0.16, "Results 56.3%", ha="right", va="bottom", fontsize=10.5,
bbox=dict(fc=BG, ec="none", pad=1), zorder=4)
ax.text(pct(ph_sum) - 1.8, yb - 0.17, f"phases 79 + 55 = {pct(ph_sum):.1f}%",
ha="right", va="center", fontsize=10.5, color=INK,
bbox=dict(fc=BG, ec="none", pad=1), zorder=4)
ax.text(pct(RES_CLIN_PRIM), yc - 0.18, "Results 73.5% (175/238)", ha="center", va="top",
fontsize=10.5)
ax.plot([ABS_BIO_PCT, ABS_BIO_PCT], [yc + 0.1, yb - 0.1], color=INK2, lw=1.2,
ls=(0, (2, 2)))
ax.text(ABS_BIO_PCT + 1.2, 0.5, "identical: 175/238", fontsize=10, color=INK2, va="center")
ax.axvline(50, color=INK2, lw=1, ls=(0, (3, 3)), zorder=0)
ax.text(49, -0.45, "50%: Discussion's “more than half”", ha="right", fontsize=10,
color=INK2, va="center")
ax.set_yticks([yb, yc], ["≥ 1 biomarker,\nany use", "Clinical outcome\nas primary"],
fontsize=10.5)
ax.set_xlim(0, 100); ax.set_ylim(-0.65, 1.55)
ax.set_xticks(range(0, 101, 25)); ax.set_xticklabels([f"{t}%" for t in range(0, 101, 25)])
ax.set_xlabel("Share of the 238 included trials")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=legend_handles(), loc="upper center", bbox_to_anchor=(0.4, -0.3),
ncol=2, frameon=False, fontsize=10)
save(fig, "cand_E_percentage_strip")
for f in (cand_A, cand_B, cand_C, cand_D, cand_E):
f()
# Provisional choice for the write-up (pending the reader's pick):
# fig1 = candidate A (text panels), fig2 = candidate B (dot comparison).
FIG1, FIG2 = "cand_A_text_panels", "cand_B_dot_comparison"
for src, dst in ((FIG1, "fig1"), (FIG2, "fig2")):
for ext in ("svg", "png"):
shutil.copyfile(os.path.join(CAND, f"{src}.{ext}"), os.path.join(HERE, f"{dst}.{ext}"))
print("\nFigures written to", HERE, "and", CAND)
Abstract pairs Ala510 with dA and Asn513 with dC; the paper's Results and Conclusion pair them the other way round
Nucleic Acids Res (2015) · doi:10.1093/nar/gkv760 · PMC4652769
Summary
The abstract states that "Ala510 and Asn513 facilitate incorporation of 8-oxo-dGMP opposite dA and dC, respectively". The body of the paper states the reverse pairing three times. The N513A paragraph of the Results concludes that "Asn513 plays an important role in facilitating 8-oxo-dGMP incorporation opposite dA". The Conclusion states that "Ala510 selectively enhances the incorporation of 8-oxo-dGMP opposite dC". The double-mutant paragraph describes "the opposite effects of Ala510 and Asn513, facilitating incorporation of the anti- and syn-conformations, respectively", and the paper ties the syn conformation to dA and the anti conformation to dC. The contradiction is between the abstract and the paper's own text, and that internal contradiction is the main basis of this report.
Table 1 supports the body text's assignment for Ala510 under both ways of summarising the data that I checked. For Asn513 the data support the body text (dA) on the absolute metric and are about even on the alternative metric; on neither metric do they support the abstract's dC assignment. The underlying kinetics and the Results text are internally consistent; the error is confined to one sentence of the abstract.
What the paper reports
Abstract (as archived in PMC):
Steady-state kinetics reveal that two residues lining the dNTP binding pocket, Ala510 and Asn513, play differential roles in dNTP selectivity. Specifically, Ala510 and Asn513 facilitate incorporation of 8-oxo-dGMP opposite dA and dC, respectively.
Results and Discussion, N513A paragraph:
However, the N513A substitution resulted in a much more drastic effect in the context of 8-oxo-dGMP incorporation opposite dA (25-fold, Figure 3D). Also, the catalytic efficiencies of dGMP and 8-oxo-dGMP incorporation opposite dC were reduced by 1.3 (Figure 3E) and 6.4-fold (Figure 3F) respectively. Taken together, these results suggest that Asn513 plays an important role in facilitating 8-oxo-dGMP incorporation opposite dA, implying that this residue selectively stabilizes the syn-conformation of 8-oxo-dGTP.
Results and Discussion, A510D paragraph:
Notably, the A510D substitution had the largest effect on the incorporation of 8-oxo-dGTP opposite dC (7.6-fold, Figure 3E) as opposed to dGTP (2.8-fold, Figure 3F). This suggests that Ala510 may preferentially stabilize the anti-conformation of 8-oxo-dGTP, promoting its incorporation opposite dC.
Results and Discussion, double-mutant paragraph:
Interestingly, the drop in catalytic efficiency was nearly identical for 8-oxo-dGMP incorporation opposite dA and dC. This is consistent with the opposite effects of Ala510 and Asn513, facilitating incorporation of the anti- and syn-conformations, respectively.
Conclusion:
As a consequence, Ala510 selectively enhances the incorporation of 8-oxo-dGMP opposite dC.
So the Results, the double-mutant interpretation and the Conclusion all assign Asn513 to the dA (syn) context and Ala510 to the dC (anti) context. The abstract's sentence assigns them the other way round.
What the paper's own numbers show
Method. Table 1 gives the catalytic efficiency kcat/KM (min⁻¹ µM⁻¹, mean ± SD of three independent experiments) for WT and each mutant, for the correct dNTP and for 8-oxo-dGTP, opposite both dA and dC. From those values I recomputed the fold loss of kcat/KM caused by each substitution (WT ÷ mutant) and compared it with the fold values printed in the Results text. I summarise each residue's template preference two ways, because the answer for Asn513 depends on which one is used:
- Metric 1, absolute fold loss. WT kcat/KM ÷ mutant kcat/KM for 8-oxo-dGTP, in each template context. This is the metric the abstract's wording ("facilitate incorporation ... opposite dA / dC") most directly implies, and the one on which the "reversed pairing" holds cleanly.
- Metric 2, loss relative to the correct dNTP. The 8-oxo-dGTP fold loss divided by the fold loss for the correct nucleotide (dTTP opposite dA, dGTP opposite dC) in the same context. This is the comparison the paper itself makes when it argues each residue's role, e.g. A510D 7.6-fold on 8-oxo-dGTP versus 2.8-fold on dGTP opposite dC.
Approximate SDs are first-order (delta-method) propagations of the Table 1 SDs on the log scale, treating WT and mutant as independent; with n = 3 per estimate they are a rough guide only. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from Table 1 |
|---|---|---|---|
| A510D fold loss, 8-oxo-dGTP opposite dA | Results | 5.1-fold | 5.11× (SD ~0.65) |
| A510D fold loss, 8-oxo-dGTP opposite dC | Results | 7.6-fold | 7.57× (SD ~1.67) |
| N513A fold loss, 8-oxo-dGTP opposite dA | Results | 25-fold | 24.74× (SD ~3.19) |
| N513A fold loss, 8-oxo-dGTP opposite dC | Results | 6.4-fold | 6.39× (SD ~0.71) |
| A510D fold loss, dTTP opposite dA | Results | 4.8-fold | 4.78× |
| A510D fold loss, dGTP opposite dC | Results | 2.8-fold | 2.76× |
| N513A fold loss, dTTP opposite dA | Results | 5.8-fold | 5.82× |
| N513A fold loss, dGTP opposite dC | Results | 1.3-fold | 1.33× |
The Results-text values and the values recomputed from Table 1 agree throughout, so the contradiction is not between the Results and the table; it is between the abstract and the rest of the paper.
Which context does each residue affect more?
| Residue | Metric 1 (absolute): dA vs dC | larger at | Metric 2 (÷ correct dNTP): dA vs dC | larger at |
|---|---|---|---|---|
| Ala510 (A510D) | 5.11× vs 7.57× | dC (ratio 1.48; |log diff|/SD ~1.5) | 1.07 vs 2.74 | dC (ratio 2.57; |log diff|/SD ~2.6) |
| Asn513 (N513A) | 24.74× vs 6.39× | dA (ratio 3.87; |log diff|/SD ~7.9) | 4.25 vs 4.79 | about equal (ratio 1.13; |log diff|/SD ~0.4) |
Reading. On the absolute metric the pairing is the clean reverse of the abstract: Ala510's larger effect is opposite dC, Asn513's is opposite dA (and the Asn513 gap is large, ~8 approximate SDs). On the relative metric Ala510 is again larger opposite dC, by a clear margin, while Asn513 is essentially even between the two contexts (4.25 vs 4.79, well under one approximate SD apart). So:
- The abstract's Ala510 → dA assignment is contradicted on both metrics.
- The abstract's Asn513 → dC assignment is contradicted on the absolute metric and is, at best, not supported on the relative metric (the point estimate there is slightly larger opposite dC, but the difference is within noise). It is the body text's Asn513 → dA assignment, not the abstract's, that the absolute metric supports.
A consistency check favours the body text. For the double mutant the fold loss is nearly identical opposite dA (55.8×) and dC (58.9×), which the paper reads as Ala510 and Asn513 having opposite template preferences that cancel. That reading requires Asn513 → dA and Ala510 → dC, i.e. the body text, not the abstract. The Results also note that removing Asn513 cripples 8-oxo-dGTP opposite dA (25-fold) far more than it does the correct dTTP in the same context (5.8-fold), which is the paper's own argument that Asn513 selectively helps the dA (syn) mispair.
Figure
Figure 1. The abstract's sentence (left, orange) pairs Ala510 with the dA context and Asn513 with the dC context. The Results, Conclusion and double-mutant paragraphs (right) pair them the other way: Asn513 with dA (syn), Ala510 with dC (anti). The strip below lists, recomputed from Table 1, the fold loss of kcat/KM for 8-oxo-dGTP in each context, both as the absolute loss and divided by the loss for the correct dNTP (blue). Blue values are recomputed by recompute.py.txt; the abstract gives no numbers. The Results text prints the same absolute values, rounded.
Design candidates for review.
recompute.py.txtalso writes five candidate layouts incandidates/(cand1–cand5).fig1is currently a copy ofcand1(the two-panel text diagram) andfig2a copy ofcand3(a one-axis log plot of the fold losses). The final figure choice is open until the reader picks from the five candidates.
Why it matters
- The error is in the abstract, and inverts a headline mechanistic claim. The abstract's differential-role sentence is the paper's one-line summary of its kinetics. Readers, indexers and citing authors who see only the abstract take away the reverse of what the Results, Conclusion and Table 1 support: that Ala510 helps 8-oxo-dGMP opposite dA and Asn513 opposite dC.
- The directions connect to different conformations. The paper links the dA mispair to the syn conformation (stabilised by the Asn513 hydrogen bond to the C8-carbonyl) and the dC pairing to the anti conformation (where the A510D paragraph invokes an unfavourable electrostatic contact). Swapping the residues swaps which conformation each is said to stabilise, which is a mechanistic claim, not just a labelling slip.
- The data and the analysis are not affected. Table 1, the Results text and the Conclusion are internally consistent. Only the abstract sentence is wrong, so the correction is a one-sentence fix.
Explanations considered
- A reader could pick the abstract's pairing out of the numbers on some metric. Not supported. On the absolute fold loss the pairing is the clean reverse of the abstract. On the loss relative to the correct dNTP, Ala510 is still clearly larger opposite dC, and Asn513 is about even (4.25 vs 4.79) rather than larger opposite dC. Neither metric gives Ala510 → dA, and neither gives a clear Asn513 → dC. The abstract's own body text states the reverse pairing explicitly in three places.
- "Respectively" was meant to map to "dC and dA". Not supported. English "A and B ... opposite dA and dC, respectively" maps A→dA, B→dC, and the sentence names Ala510 before Asn513. Reading it the other way would make it agree with the body, but that is not what the sentence says.
- A different measure of "facilitate". The abstract gives no numbers, so "facilitate ... opposite dA / dC" can only be read as the fold effect of each substitution in each context. Both the metrics I checked place Ala510's larger effect opposite dC; neither supports Ala510 → dA.
- A correction already issued. None found (see Status).
Status
- Checks performed.
- (1) Recompute: the 10 fold values printed in the Results text all match the values recomputed from Table 1 (see the table above);
recompute.py.txtprints them. - (2) Numbers in this write-up: every figure and number drawn here comes from Table 1 or from
recompute.py.txt; none is a new claim. - (4) Tone: the write-up reports the internal contradiction and the metric dependence without speculating about cause or naming any person.
- (5) This Status section records each check and its date.
- (6) Markdown and image links: the figure link points to
fig1.svg, which exists in this folder; the candidate layouts are incandidates/. - Version of record. On 2026-10-23 the abstract sentence and the 16 kcat/KM values of Table 1 were compared with the publisher's HTML page (academic.oup.com, via the DOI) and with the Europe PMC core record (PMID 26220180); the abstract sentence and all 16 values match. Every quotation above was also checked verbatim against the PMC-archived text with
src/show.py. - Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. A web search for the title together with "PubPeer", "correction" and "erratum" returned only the paper and its PDB entries. - Figure-panel citations. This write-up cites Table 1 for all numbers and does not rely on figure-panel letters. The paper's Results label some Figure 3 panels inconsistently between the A510D and N513A paragraphs (e.g. 8-oxo-dGMP opposite dC is cited as Figure 3E in one paragraph and the dGMP/8-oxo-dGMP panels are cited as 3E/3F in another); because this write-up cites Table 1 only, that inconsistency does not affect it.
- Sources. PMC article PMC4652769; publisher page academic.oup.com (doi:10.1093/nar/gkv760); Europe PMC core record (PMID 26220180); Crossref; PMC and Europe PMC notice feeds; https://doi.org/10.1093/nar/gkv760.
Suggested correction
In the abstract, replace
Specifically, Ala510 and Asn513 facilitate incorporation of 8-oxo-dGMP opposite dA and dC, respectively.
with
Specifically, Ala510 and Asn513 facilitate incorporation of 8-oxo-dGMP opposite dC and dA, respectively.
(or, equivalently, "Asn513 and Ala510 ... opposite dA and dC, respectively"), so that the abstract agrees with the Results, the Conclusion, the double-mutant interpretation and Table 1.
Recomputation script (python recompute.py.txt)
"""Recompute the Ala510 / Asn513 fold effects of PMC4652769
(Nucleic Acids Res 2015, doi:10.1093/nar/gkv760).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes, next to this file:
fig1.svg / fig1.png abstract vs body text (two text panels), with Table 1 numbers
fig2.svg / fig2.png fold reductions on one log axis, two metrics, approximate SDs
candidates/cand1..cand5.svg/png five candidate figure designs, for the reader to choose from
All inputs are copied from Table 1 of the article as archived in PubMed Central:
catalytic efficiency kcat/KM in min^-1 uM^-1, mean +/- SD of three independent
experiments. No raw data are used. (The check of these values against the publisher's
page is recorded in the Status section of writeup.md, not here.)
Colour code, the same in every figure:
orange = the pairing stated in the abstract (the abstract gives no numbers)
blue = recomputed from Table 1 by this script
grey = a fold value printed in the Results / Conclusion text (these agree with Table 1)
fig1 and fig2 are provisional copies of candidates 1 and 3 until a design is chosen.
"""
import os
import textwrap
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Rectangle # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ------------------------------------------------------------------ inputs (Table 1)
# (mean, SD) of kcat/KM
T1 = {
"A": { # template dA
"dTTP": {"WT": (32, 4), "A510D": (6.7, 0.6), "N513A": (5.5, 1.0), "A510D/N513A": (0.62, 0.07)},
"8oxo": {"WT": (2.4, 0.3), "A510D": (0.47, 0.01), "N513A": (0.097, 0.003), "A510D/N513A": (0.043, 0.005)},
},
"C": { # template dC
"dGTP": {"WT": (80, 10), "A510D": (29, 5), "N513A": (60, 4), "A510D/N513A": (11, 2)},
"8oxo": {"WT": (0.53, 0.05), "A510D": (0.070, 0.014), "N513A": (0.083, 0.005), "A510D/N513A": (0.009, 0.002)},
},
}
NORMAL = {"A": "dTTP", "C": "dGTP"}
# Fold values printed in the Results / Conclusion text
REPORTED = {
("A510D", "A", "8oxo"): 5.1, ("A510D", "C", "8oxo"): 7.6,
("N513A", "A", "8oxo"): 25, ("N513A", "C", "8oxo"): 6.4,
("A510D", "A", "dTTP"): 4.8, ("N513A", "A", "dTTP"): 5.8, ("A510D/N513A", "A", "dTTP"): 52,
("A510D", "C", "dGTP"): 2.8, ("N513A", "C", "dGTP"): 1.3, ("A510D/N513A", "C", "dGTP"): 7.3,
}
ABSTRACT = ("Specifically, Ala510 and Asn513 facilitate incorporation of 8-oxo-dGMP "
"opposite dA and dC, respectively.")
Q_N513 = ("Asn513 plays an important role in facilitating 8-oxo-dGMP incorporation opposite dA")
Q_A510 = ("Ala510 selectively enhances the incorporation of 8-oxo-dGMP opposite dC")
Q_DBL = ("the opposite effects of Ala510 and Asn513, facilitating incorporation of the anti- "
"and syn-conformations, respectively")
def fold(t, nt, mut):
"""Fold decrease in kcat/KM relative to WT, and an approximate SD.
The SD is a first-order (delta-method) propagation of the Table 1 SDs on the log scale,
treating WT and mutant as independent: sd(ratio) ~ ratio * sqrt(cv_wt^2 + cv_mut^2).
With n = 3 per estimate this is a rough guide only."""
(w, sw), (m, sm) = T1[t][nt]["WT"], T1[t][nt][mut]
r = w / m
return r, r * np.hypot(sw / w, sm / m)
def rel(t, mut):
"""Fold loss for 8-oxo-dGTP divided by fold loss for the correct dNTP in the same template
context (the comparison the paper itself makes, e.g. 7.6-fold vs 2.8-fold for A510D at dC).
Approximate SD by the same first-order propagation over the four Table 1 entries."""
r8, s8 = fold(t, "8oxo", mut)
rn, sn = fold(t, NORMAL[t], mut)
r = r8 / rn
return r, r * np.hypot(s8 / r8, sn / rn)
# ------------------------------------------------------------------ 1. fold reductions
print("1. Fold reduction in kcat/KM vs WT (Table 1), with text values")
print(f" {'mutant':12} {'template':8} {'dNTP':10} {'reported':>5} {'recomputed':>11} {'approx SD':>10}")
F = {}
for (mut, t, nt), rep in REPORTED.items():
r, s = fold(t, nt, mut)
F[(mut, t, nt)] = (r, s)
print(f" {mut:12} d{t:7} {('8-oxo-dGTP' if nt == '8oxo' else nt):10} {rep:>5} {r:11.2f} {s:10.2f}")
for t in "AC":
r, s = fold(t, "8oxo", "A510D/N513A")
F[("A510D/N513A", t, "8oxo")] = (r, s)
print(f" {'A510D/N513A':12} d{t:7} {'8-oxo-dGTP':10} {'-':>5} {r:11.2f} {s:10.2f}")
# ------------------------------------------------------------------ 2. WT comparisons
print("\n2. WT comparisons in the Results text")
wt = lambda t, nt: T1[t][nt]["WT"][0] # noqa: E731
print(f" dGMP vs 8-oxo-dGMP opposite dC: reported 150-fold, recomputed {wt('C','dGTP')/wt('C','8oxo'):.2f}")
print(f" dTMP vs 8-oxo-dGMP opposite dA: reported 13-fold, recomputed {wt('A','dTTP')/wt('A','8oxo'):.2f}")
print(f" dA/dC selectivity for 8-oxo-dGTP: reported 5:1, recomputed {wt('A','8oxo')/wt('C','8oxo'):.2f}:1")
# ------------------------------------------------------------------ 3. the pairing, absolute metric
print("\n3. Metric 1 (absolute): fold loss of kcat/KM for 8-oxo-dGTP vs WT, dA vs dC")
pair_abs = {}
for mut, res in (("A510D", "Ala510"), ("N513A", "Asn513")):
a, sa = F[(mut, "A", "8oxo")]
c, sc = F[(mut, "C", "8oxo")]
pair_abs[res] = "dA" if a > c else "dC"
# log-scale difference in units of its approximate SD
z = abs(np.log(a) - np.log(c)) / np.hypot(sa / a, sc / c)
print(f" {res} ({mut}): dA {a:.2f}x (SD ~{sa:.2f}) vs dC {c:.2f}x (SD ~{sc:.2f}); "
f"ratio {max(a, c)/min(a, c):.2f}; larger at {pair_abs[res]}; |log diff|/SD ~ {z:.1f}")
print(f" Larger effect: Ala510 -> {pair_abs['Ala510']}, Asn513 -> {pair_abs['Asn513']}")
# ------------------------------------------------------------------ 4. the pairing, relative metric
print("\n4. Metric 2 (relative): 8-oxo-dGTP fold loss / correct-dNTP fold loss in the same context")
print(" (dTTP opposite dA, dGTP opposite dC; the comparison the paper makes for each mutant)")
pair_rel = {}
REL = {}
for mut, res in (("A510D", "Ala510"), ("N513A", "Asn513")):
(a, sa), (c, sc) = rel("A", mut), rel("C", mut)
REL[(mut, "A")], REL[(mut, "C")] = (a, sa), (c, sc)
z = abs(np.log(a) - np.log(c)) / np.hypot(sa / a, sc / c)
pair_rel[res] = ("dA" if a > c else "dC") if z >= 1 else "about equal"
print(f" {res} ({mut}): dA {a:.2f} (SD ~{sa:.2f}) vs dC {c:.2f} (SD ~{sc:.2f}); "
f"ratio {max(a, c)/min(a, c):.2f}; larger: {pair_rel[res]}; |log diff|/SD ~ {z:.1f}")
print(f" Larger effect: Ala510 -> {pair_rel['Ala510']}, Asn513 -> {pair_rel['Asn513']}")
print("\n Abstract: Ala510 -> dA, Asn513 -> dC (\"... opposite dA and dC, respectively\")")
print(" Paper text: Ala510 -> dC (anti), Asn513 -> dA (syn)")
print(f" Ala510 abstract assignment supported by metric 1: {pair_abs['Ala510'] == 'dA'}, "
f"by metric 2: {pair_rel['Ala510'] == 'dA'}")
print(f" Asn513 abstract assignment supported by metric 1: {pair_abs['Asn513'] == 'dC'}, "
f"by metric 2: {pair_rel['Asn513']} (point estimate larger at dC, |log diff|/SD < 1)")
# ------------------------------------------------------------------ 5. double mutant
print("\n5. Double mutant, 8-oxo-dGTP (Results: 'nearly identical' for dA and dC)")
print(f" opposite dA {F[('A510D/N513A','A','8oxo')][0]:.1f}x, opposite dC {F[('A510D/N513A','C','8oxo')][0]:.1f}x")
# ================================================================== figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, BOX = "#eb6834", "#2a78d6", "#e4e3df", "#c9c8c3"
GREY = "#7a7975" # Results-printed values
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 11, "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"axes.edgecolor": BOX, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
W = 7.2
a510 = {t: F[("A510D", t, "8oxo")] for t in "AC"}
n513 = {t: F[("N513A", t, "8oxo")] for t in "AC"}
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def panel(ax, x, y, w, h, title):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc="white", ec=BOX, lw=1.2))
ax.text(x + 0.025, y + h - 0.035, title, fontsize=11.5, weight="bold", color=SEC, va="top")
def wrap(s, n):
return "\n".join(textwrap.wrap(s, n))
# ---------- candidate 1: two text panels joined by a connector (also fig1)
def cand1(path):
H = 7.4
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off(); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
rend = fig.canvas.get_renderer()
def height(t): # height of a drawn text object in axes fraction
bb = t.get_window_extent(renderer=rend)
return bb.height / (fig.dpi * H)
ax.text(0.03, 0.975, "Which residue facilitates 8-oxo-dGMP incorporation opposite dA, which opposite dC?",
fontsize=10.5, weight="bold", va="top")
top, bot = 0.93, 0.335
# left: abstract
panel(ax, 0.03, bot, 0.43, top - bot, "Abstract")
ax.text(0.055, 0.87, wrap("\u201c" + ABSTRACT + "\u201d", 36), fontsize=10.5, va="top", linespacing=1.4)
ax.text(0.055, 0.465, "Pairing as written (\u201crespectively\u201d):", fontsize=10, color=SEC, va="top")
ax.text(0.055, 0.425, "Ala510 \u2192 opposite dA", fontsize=12, weight="bold", color=ORANGE, va="top")
ax.text(0.055, 0.38, "Asn513 \u2192 opposite dC", fontsize=12, weight="bold", color=ORANGE, va="top")
# right: body text
panel(ax, 0.54, bot, 0.43, top - bot, "Results and Conclusion")
y = 0.87
for lab, q in (("Results, N513A paragraph:", Q_N513), ("Conclusion:", Q_A510),
("Results, double-mutant paragraph:", Q_DBL)):
t = ax.text(0.565, y, lab, fontsize=10, color=SEC, va="top")
y -= height(t) + 0.006
t = ax.text(0.565, y, wrap("\u201c\u2026" + q + "\u201d", 37), fontsize=10, va="top", linespacing=1.3)
y -= height(t) + 0.018
assert y > 0.48, y
ax.text(0.565, 0.465, "Pairing in the body text:", fontsize=10, color=SEC, va="top")
ax.text(0.565, 0.425, "Asn513 \u2192 opposite dA (syn)", fontsize=12, weight="bold", color=TXT, va="top")
ax.text(0.565, 0.38, "Ala510 \u2192 opposite dC (anti)", fontsize=12, weight="bold", color=TXT, va="top")
ax.add_patch(FancyArrowPatch((0.445, 0.40), (0.555, 0.40), arrowstyle="<->", mutation_scale=14,
lw=2, color=SEC))
ax.text(0.50, 0.385, "pairing\nreversed", ha="center", va="top", fontsize=10, style="italic", color=SEC)
# bottom strip: Table 1, both metrics
ax.text(0.03, 0.295, "Table 1, recomputed: loss of kcat/KM for 8-oxo-dGTP vs WT", fontsize=10.5,
weight="bold", va="top")
rows = [("absolute fold loss", a510, n513, "{:.1f}\u00d7"),
("\u00f7 loss for correct dNTP", {t: REL[("A510D", t)] for t in "AC"},
{t: REL[("N513A", t)] for t in "AC"}, "{:.2f}")]
ax.text(0.52, 0.250, "Ala510 (A510D)", fontsize=10, color=SEC, va="top", ha="center")
ax.text(0.82, 0.250, "Asn513 (N513A)", fontsize=10, color=SEC, va="top", ha="center")
for k, (lab, A, N, fmt) in enumerate(rows):
yy = 0.210 - k * 0.042
ax.text(0.03, yy, lab, fontsize=10, va="top")
ax.text(0.52, yy, f"dA {fmt.format(A['A'][0])} vs dC {fmt.format(A['C'][0])}", fontsize=10.5,
weight="bold", color=BLUE, va="top", ha="center")
ax.text(0.82, yy, f"dA {fmt.format(N['A'][0])} vs dC {fmt.format(N['C'][0])}", fontsize=10.5,
weight="bold", color=BLUE, va="top", ha="center")
ax.text(0.03, 0.112, "Correct dNTP: dTTP opposite dA, dGTP opposite dC. The Results text prints the same\n"
"absolute values, rounded: 5.1 and 7.6-fold (A510D), 25 and 6.4-fold (N513A).",
fontsize=10, color=SEC, va="top", linespacing=1.35)
ax.text(0.03, 0.040, "orange = pairing stated in the abstract; blue = recomputed from Table 1 by recompute.py.txt",
fontsize=10, color=SEC, va="top")
save(fig, path)
# ---------- candidate 2: two side-by-side bipartite mappings
def cand2(path):
fig = plt.figure(figsize=(W, 4.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off(); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
ax.text(0.03, 0.96, "Residue-to-template pairing for 8-oxo-dGMP incorporation", fontsize=12, weight="bold", va="top")
def block(x0, span, title, edges, color, ls):
ax.text(x0 + (span + 0.12) / 2, 0.85, title, ha="center", fontsize=11, weight="bold", color=SEC)
L = {"Ala510": 0.70, "Asn513": 0.34}
R = {"dA": 0.70, "dC": 0.34}
for k, y in L.items():
ax.text(x0 + 0.05, y, k, ha="center", va="center", fontsize=11.5, weight="bold",
bbox=dict(boxstyle="round,pad=0.35", fc="white", ec=BOX))
rl = {"dA": "opposite dA\n(syn)", "dC": "opposite dC\n(anti)"}
for k, y in R.items():
ax.text(x0 + span + 0.07, y, rl[k], ha="center", va="center", fontsize=10.5,
bbox=dict(boxstyle="round,pad=0.35", fc="white", ec=BOX))
for (l, r, lab, rep_lab) in edges:
x1, y1, x2, y2 = x0 + 0.105, L[l], x0 + span, R[r]
ax.plot([x1, x2], [y1, y2], color=color, lw=2.2, ls=ls)
if not lab:
continue
frac = 0.25
xm, ym = x1 + frac * (x2 - x1), y1 + frac * (y2 - y1)
ax.text(xm, ym, lab + "\n" + rep_lab, ha="center", va="center", fontsize=10.5, color=BG,
linespacing=1.3, bbox=dict(boxstyle="round,pad=0.3", fc=BG, ec=color, lw=1), zorder=5)
ax.text(xm, ym + 0.026, lab, ha="center", va="center", fontsize=10.5, color=color, weight="bold", zorder=6)
ax.text(xm, ym - 0.026, rep_lab, ha="center", va="center", fontsize=10, color=GREY, zorder=6)
block(0.03, 0.26, "Abstract (“respectively”)",
[("Ala510", "dA", "", ""), ("Asn513", "dC", "", "")], ORANGE, "--")
block(0.49, 0.36, "Table 1: larger absolute fold loss",
[("Ala510", "dC", f"{a510['C'][0]:.2f}×", f"Results: {REPORTED[('A510D', 'C', '8oxo')]}"),
("Asn513", "dA", f"{n513['A'][0]:.2f}×", f"Results: {REPORTED[('N513A', 'A', '8oxo')]}")],
BLUE, "-")
ax.plot([0.46, 0.46], [0.24, 0.82], color=BOX, lw=1)
ax.text(0.03, 0.185, f"Smaller effects: A510D {a510['A'][0]:.2f}× opposite dA; N513A {n513['C'][0]:.2f}× opposite dC "
f"(Results: 5.1 and 6.4).", fontsize=10, color=SEC)
ax.text(0.03, 0.135, f"Relative to the correct dNTP: A510D {REL[('A510D','A')][0]:.2f} (dA) vs "
f"{REL[('A510D','C')][0]:.2f} (dC); N513A {REL[('N513A','A')][0]:.2f} (dA) vs {REL[('N513A','C')][0]:.2f} (dC).",
fontsize=10, color=SEC)
ax.legend(handles=[Line2D([], [], color=ORANGE, ls="--", lw=2, label="pairing stated in the abstract"),
Line2D([], [], color=BLUE, lw=2, label="recomputed from Table 1"),
Line2D([], [], color=GREY, lw=0, marker="s", ms=8, label="printed in the Results")],
loc="lower center", bbox_to_anchor=(0.5, 0.01), ncol=3, frameon=False, fontsize=10,
columnspacing=1.0, handlelength=1.6, handletextpad=0.5)
save(fig, path)
# ---------- candidate 3: fold reductions on one log axis, two metrics, approximate SDs (also fig2)
def cand3(path):
fig, ax = plt.subplots(figsize=(W, 4.6))
fig.subplots_adjust(left=0.27, right=0.97, top=0.82, bottom=0.27)
rows = [("N513A", "C"), ("N513A", "A"), ("A510D", "C"), ("A510D", "A")]
labs = {("A510D", "A"): "A510D, opposite dA", ("A510D", "C"): "A510D, opposite dC",
("N513A", "A"): "N513A, opposite dA", ("N513A", "C"): "N513A, opposite dC"}
abstract_rows = {("A510D", "A"): "abstract: Ala510 → dA", ("N513A", "C"): "abstract: Asn513 → dC"}
off = 0.17
def bar(r, s, y, marker, filled):
lo, hi = np.exp(np.log(r) - s / r), np.exp(np.log(r) + s / r)
ax.plot([lo, hi], [y, y], color=BLUE, lw=2)
ax.plot(r, y, marker, ms=8, color=BLUE, mfc=BLUE if filled else BG, mew=2, zorder=3)
return hi
for i, (m, t) in enumerate(rows):
r, s = F[(m, t, "8oxo")]
hi = bar(r, s, i + off, "D", True)
ax.plot(REPORTED[(m, t, "8oxo")], i + off, "o", ms=14, mfc="none", mec=GREY, mew=1.6, zorder=4)
ax.text(hi * 1.08, i + off, f"{r:.1f}×", va="center", fontsize=10, color=BLUE)
r2, s2 = REL[(m, t)]
hi2 = bar(r2, s2, i - off, "s", False)
ax.text(hi2 * 1.08, i - off, f"{r2:.2f}", va="center", fontsize=10, color=BLUE)
if (m, t) in abstract_rows:
ax.text(58, i, abstract_rows[(m, t)], ha="right", va="center", fontsize=10, color=ORANGE,
weight="bold")
ax.set_yticks(range(4)); ax.set_yticklabels([labs[k] for k in rows], fontsize=10.5, color=TXT)
ax.set_xscale("log"); ax.set_xlim(0.8, 60)
ax.set_xticks([1, 2, 5, 10, 20, 50]); ax.set_xticklabels(["1", "2", "5", "10", "20", "50"])
ax.axvline(1, color=SEC, lw=0.8)
ax.axhline(1.5, color=BOX, lw=1)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ("top", "right"):
ax.spines[sp].set_visible(False)
ax.set_xlabel("Fold loss of kcat/KM vs WT (log scale; 1 = no effect)", fontsize=10.5)
ax.set_ylim(-0.55, 3.55)
fig.text(0.02, 0.97, "8-oxo-dGTP: effect of each substitution opposite dA and opposite dC", fontsize=11.5,
weight="bold", va="top")
fig.text(0.02, 0.915, wrap("Absolute loss is larger at dC for A510D and at dA for N513A. Relative to the "
"correct dNTP, A510D still favours dC; N513A is about equal.", 95),
fontsize=10, color=SEC, va="top")
ax.legend(handles=[Line2D([], [], marker="D", ls="-", lw=2, ms=8, color=BLUE,
label="absolute fold loss, Table 1 (±1 approx. SD)"),
Line2D([], [], marker="s", ls="-", lw=2, ms=8, color=BLUE, mfc=BG, mew=2,
label="÷ fold loss for the correct dNTP, Table 1"),
Line2D([], [], marker="o", ls="none", ms=12, mfc="none", mec=GREY, mew=1.6,
label="absolute value printed in the Results"),
Line2D([], [], ls="none", marker="$→$", ms=10, color=ORANGE,
label="pairing stated in the abstract")],
loc="upper center", bbox_to_anchor=(0.36, -0.17), ncol=2, frameon=False, fontsize=10,
columnspacing=1.2, handletextpad=0.5)
save(fig, path)
# ---------- candidate 4: 2 x 2 grid with outlined cells
def cand4(path):
fig = plt.figure(figsize=(W, 4.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off(); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
ax.text(0.03, 0.96, "Effect on 8-oxo-dGTP incorporation, by residue and template",
fontsize=12, weight="bold", va="top")
x0, y0, cw, ch = 0.30, 0.20, 0.30, 0.27
cols, rowsr = ["A", "C"], [("A510D", "Ala510 (A510D)"), ("N513A", "Asn513 (N513A)")]
abstract_cells = {("A510D", "A"), ("N513A", "C")}
for j, t in enumerate(cols):
ax.text(x0 + (j + 0.5) * cw, y0 + 2 * ch + 0.04, f"opposite d{t}", ha="center", fontsize=11, weight="bold", color=SEC)
for i, (m, lab) in enumerate(rowsr):
y = y0 + (1 - i) * ch
ax.text(x0 - 0.02, y + ch / 2, lab, ha="right", va="center", fontsize=11, weight="bold")
vals = {t: F[(m, t, "8oxo")][0] for t in cols}
for j, t in enumerate(cols):
x = x0 + j * cw
ax.add_patch(Rectangle((x, y), cw, ch, fc="white", ec=BOX, lw=1))
ax.text(x + cw / 2, y + ch * 0.68, f"{vals[t]:.1f}×", ha="center", va="center",
fontsize=16, weight="bold", color=BLUE)
ax.text(x + cw / 2, y + ch * 0.44, f"÷ correct dNTP: {REL[(m, t)][0]:.2f}",
ha="center", va="center", fontsize=10, color=BLUE)
ax.text(x + cw / 2, y + ch * 0.23, f"Results text: {REPORTED[(m, t, '8oxo')]}-fold",
ha="center", va="center", fontsize=10, color=GREY)
if vals[t] == max(vals.values()):
ax.add_patch(Rectangle((x + 0.008, y + 0.012), cw - 0.016, ch - 0.024, fc="none", ec=BLUE, lw=2.5))
if (m, t) in abstract_cells:
ax.add_patch(Rectangle((x + 0.022, y + 0.030), cw - 0.044, ch - 0.060, fc="none", ec=ORANGE,
lw=2, ls="--"))
ax.text(0.03, 0.135, "Large number: absolute fold loss of kcat/KM vs WT. Second line: that loss divided by the "
"loss\nfor the correct dNTP (dTTP at dA, dGTP at dC). Both recomputed from Table 1.",
fontsize=10, color=SEC, va="center")
ax.legend(handles=[Rectangle((0, 0), 1, 1, fc="none", ec=BLUE, lw=2.5, label="larger absolute loss per residue"),
Rectangle((0, 0), 1, 1, fc="none", ec=ORANGE, lw=2, ls="--", label="pairing stated in the abstract")],
loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False, fontsize=10)
save(fig, path)
# ---------- candidate 5: slope chart dA -> dC
def cand5(path):
fig, ax = plt.subplots(figsize=(W, 5.0))
fig.subplots_adjust(left=0.13, right=0.62, top=0.89, bottom=0.30)
xs = [0, 1]
for m, res, ls, mk in (("N513A", "Asn513 (N513A)", "-", "o"), ("A510D", "Ala510 (A510D)", "--", "s")):
v = [F[(m, t, "8oxo")][0] for t in "AC"]
ax.plot(xs, v, ls=ls, marker=mk, ms=8, lw=2, color=BLUE, zorder=3)
for x, t, val in zip(xs, "AC", v):
ax.plot(x, REPORTED[(m, t, "8oxo")], "o", ms=17, mfc="none", mec=GREY, mew=1.6, zorder=4)
dy = 1.0 if x == 0 else (1.13 if val == max(F[(mm, "C", "8oxo")][0] for mm in ("A510D", "N513A")) else 0.88)
ax.text(x + (0.09 if x else -0.09), val * dy, f"{val:.2f}×", ha="left" if x else "right",
va="center", fontsize=10, color=BLUE)
mid = np.sqrt(v[0] * v[1])
ax.text(0.5, mid * (1.75 if m == "N513A" else 0.68), res, ha="center", va="center",
fontsize=10.5, weight="bold")
ax.set_xticks(xs); ax.set_xticklabels(["opposite dA\n(syn)", "opposite dC\n(anti)"], fontsize=10.5, color=TXT)
ax.set_xlim(-0.35, 1.3)
ax.set_yscale("log"); ax.set_ylim(1, 40)
ax.set_yticks([1, 2, 5, 10, 20]); ax.set_yticklabels(["1", "2", "5", "10", "20"])
ax.set_ylabel("Absolute fold loss of kcat/KM,\n8-oxo-dGTP (log scale; 1 = no effect)", fontsize=10.5)
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ("top", "right"):
ax.spines[sp].set_visible(False)
ax.set_title("Each residue's effect in the two template contexts", fontsize=11.5, weight="bold", loc="left")
fig.text(0.67, 0.83, "Table 1, absolute loss:", fontsize=10.5, weight="bold", color=BLUE)
fig.text(0.67, 0.79, "Asn513 larger opposite dA\nAla510 larger opposite dC", fontsize=10.5, color=BLUE, va="top")
fig.text(0.67, 0.66, "÷ correct-dNTP loss:", fontsize=10.5, weight="bold", color=BLUE)
fig.text(0.67, 0.62, f"Ala510 {REL[('A510D','A')][0]:.2f} (dA), {REL[('A510D','C')][0]:.2f} (dC)\n"
f"Asn513 {REL[('N513A','A')][0]:.2f} (dA), {REL[('N513A','C')][0]:.2f} (dC)",
fontsize=10.5, color=BLUE, va="top")
fig.text(0.67, 0.47, "Abstract states:", fontsize=10.5, weight="bold", color=ORANGE)
fig.text(0.67, 0.43, "Ala510 → dA\nAsn513 → dC", fontsize=10.5, color=ORANGE, va="top")
ax.legend(handles=[Line2D([], [], marker="o", ls="-", lw=2, ms=8, color=BLUE,
label="Asn513 (N513A), recomputed from Table 1"),
Line2D([], [], marker="s", ls="--", lw=2, ms=8, color=BLUE,
label="Ala510 (A510D), recomputed from Table 1"),
Line2D([], [], marker="o", ls="none", ms=13, mfc="none", mec=GREY, mew=1.6,
label="value printed in the Results")],
loc="upper left", bbox_to_anchor=(-0.05, -0.17), ncol=1, frameon=False, fontsize=10)
save(fig, path)
for i, f in enumerate((cand1, cand2, cand3, cand4, cand5), 1):
f(os.path.join(CAND, f"cand{i}"))
cand1(os.path.join(HERE, "fig1"))
cand3(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE)
Abstract's age pattern for all STIs except HSV-2 is contradicted by the paper's syphilis estimates
PLOS Medicine (2018) · doi:10.1371/journal.pmed.1002511 · PMC5828349
Summary
This individual participant data meta-analysis of 18 HIV prevention studies reports STI prevalence among women in sub-Saharan Africa by age, region and population type. The abstract states that, in all three primary region/population groups, prevalence was higher among 15–24-year-old than 25–49-year-old women "for all STIs except HSV-2". The author summary repeats this, adding "regardless of region or population type". The paper's own syphilis estimates do not show this pattern. In South African clinic/community populations, overall syphilis prevalence was 4.3% (95% CI 2.5–7.2) at ages 25–49 and 1.5% (1.0–2.2) at 15–24. The intervals do not overlap, and the Results text itself says the older group's prevalence "was higher". In the other two groups, prevalence was higher or similar in older women. High-titer syphilis in South Africa (1.0% vs 0.4%) and trichomoniasis in South Africa (8.6% vs 7.9%) also have higher point estimates in older women. The age pattern does hold for chlamydia and gonorrhea. As written, though, the headline statement misdescribes syphilis, the STI most directly tied to antenatal screening and congenital-syphilis programs.
What the paper reports
Abstract, "Methods and findings". The sentence before it reads "Despite substantial study heterogeneity for some STIs/populations, several patterns emerged."
Across the three primary region/population groups (South Africa community-based, Southern/Eastern Africa community-based, and Eastern Africa higher-risk), prevalence was higher among 15–24-year-old than 25–49-year-old women for all STIs except HSV-2.
Author summary, "What did the researchers do and find?":
STI prevalence was highest among young women aged 15–24 years for all STIs except HSV-2, regardless of region or population type.
Discussion, opening paragraph:
We observed age-, region- and population-specific patterns in the prevalence of these infections, including higher prevalence of STIs among younger (15–24-year-old) than older (25–49-year-old) women, …
Discussion, the paragraph that links the findings to the WHO Global Strategy for STIs. Unlike the abstract, this sentence says "generally":
In terms of the “who,” across all populations and regions, STI prevalence was generally higher among younger than older women, except for the prevalence of antibodies to HSV-2, which increased with age.
Results, "Syphilis, overall (high, low, and unknown titer infection)", first paragraph:
Among 15–24-year-old women, the summary estimate of syphilis prevalence overall (i.e., proportion of participants with a positive RPR and positive treponemal test) was less than 2% in clinic/community populations in both South Africa (1.5% [95% CI: 1.0%, 2.2%]; I2 = 57.7%) and Southern/Eastern Africa (1.4% [95% CI: 1.0%, 1.9%]; I2 = 25.8%), and higher among higher-risk populations in Eastern Africa (8.4% [95% CI: 6.8%, 10.3%]; I2 = 46.3%) (Fig 5).
Results, same section, second paragraph:
Among 25–49-year-olds in clinic/community populations in South Africa, the summary estimate of syphilis prevalence overall among 25–49-year-old women (4.3% [95% CI: 2.5%, 7.2%]; I2 = 94.8%) was higher than that among younger women (1.5%) (Fig 6). In clinic/community populations in Southern/Eastern Africa, the summary estimate of syphilis prevalence overall (2.0% [95% CI: 1.4%, 2.6%]; I2 = 61.8%) was similar to the summary estimate among younger women (1.4%). In the higher-risk populations in Eastern Africa, the summary estimate among older women (7.5% [95% CI: 5.5%, 10.1%]; I2 = 85.8%) was similar to prevalence among younger women (8.4%).
Results, "Syphilis, high-titer infection":
Point estimates for women aged 25–49 years were similar to those in younger populations, but were more heterogeneous (Fig 8).
The "Overall prevalence" rows of Table 4 (15–24 years) and Table 5 (25–49 years), South African clinic/community populations, as archived in PMC:
Table 4: Syphilis, high-titer infection — Overall prevalence: 0.4% (0.3, 0.5); Trichomoniasis — Overall prevalence: 7.9% (5.9, 10.5)
Table 5: Syphilis, high-titer infection — Overall prevalence: 1.0% (0.5, 2.1); Trichomoniasis — Overall prevalence: 8.6% (6.5, 11.5)
Methods, "Statistical analysis". This describes how the strata were compared. No formal test between age groups is described.
We generated forest plots in R and inspected them visually to compare prevalence estimates across strata, and combined strata if estimates were similar.
What the paper's own numbers show
Method. I collected the paper's 15–24 and 25–49 summary prevalence estimates, with their 95% CIs, for each STI in each of the three primary groups. Overall syphilis comes from the Results text, which reports Figs 5 and 6 in numbers. Every other infection comes from the "Overall prevalence" row of Tables 4–9. Where the Results text also states a value, the table rows agree with it: chlamydia SA 15–24, trichomoniasis SA 25–49, trichomoniasis E Africa 15–24 and high-titer syphilis SA 15–24 match exactly, and HSV-2 SA 15–24 differs by 0.1 in one CI bound.
The paper pools study log odds with a random-effects model and back-transforms the pooled estimate and its CI. Its CIs are therefore symmetric on the logit scale, so the standard error of each pooled log odds can be recovered as (logit(upper) − logit(lower)) / 3.92. I then computed the older-versus-younger odds ratio, its 95% CI and an approximate z statistic, treating the two age groups' pooled estimates as independent, since they come from different women. This is a descriptive check of direction and size using only the published numbers. It is not a re-analysis of the participant data. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Direction of the age difference, all STIs except HSV-2, three primary groups | Abstract; Author summary | higher in 15–24 than 25–49 in every group | 9 of 12 infection × group cells (chlamydia, gonorrhea, overall syphilis, trichomoniasis) have the higher point estimate in 15–24, and 3 do not |
| Overall syphilis, SA clinic/community | Results text (Figs 5, 6) | 15–24: 1.5% (1.0, 2.2); 25–49: 4.3% (2.5, 7.2), "higher than that among younger women" | ratio 2.87; CIs do not overlap (2.5 > 2.2); OR 2.95 (1.49–5.84), z = +3.10, p ≈ 0.002, older higher |
| Overall syphilis, S/E Africa clinic/community | Results text | 15–24: 1.4% (1.0, 1.9); 25–49: 2.0% (1.4, 2.6), "similar" | OR 1.44 (0.91–2.26), z = +1.57, p ≈ 0.12; older point estimate higher, not significant |
| Overall syphilis, E Africa higher-risk | Results text | 15–24: 8.4% (6.8, 10.3); 25–49: 7.5% (5.5, 10.1), "similar" | OR 0.88 (0.59–1.32), z = −0.60, p ≈ 0.55; no clear difference |
| High-titer syphilis, SA clinic/community | Table 4 vs Table 5 | 15–24: 0.4% (0.3, 0.5); 25–49: 1.0% (0.5, 2.1); text: point estimates "similar" | ratio 2.50; CIs touch at 0.5; OR 2.52 (1.16–5.43), z = +2.35, p ≈ 0.02, older higher |
| High-titer syphilis, S/E Africa; E Africa higher-risk | Tables 6/7, 8/9 | 1.1% vs 0.9%; 4.5% vs 2.8% | OR 0.82 (0.20–3.39); OR 0.61 (0.38–0.97) |
| Trichomoniasis, SA clinic/community | Table 4 vs Table 5 | 15–24: 7.9% (5.9, 10.5); 25–49: 8.6% (6.5, 11.5) | OR 1.10 (0.70–1.71), z = +0.41, p ≈ 0.68; older point estimate slightly higher, no clear difference |
| Trichomoniasis, S/E Africa; E Africa higher-risk | Tables 6/7, 8/9 | 6.7% vs 5.8%; 12.7% vs 10.0% | OR 0.86 (0.47–1.57); OR 0.76 (0.33–1.78); younger point estimate higher, not significant |
| Chlamydia, three groups | Tables 4–9; Results | 15.1 vs 7.0; 2.7 vs 1.2; 10.3 vs 6.2 | younger higher in 3/3; OR 0.42, 0.44, 0.58, all with p < 0.05 |
| Gonorrhea, three groups | Tables 4–9; Results | 4.6 vs 2.5; 1.7 vs 0.9; 8.2 vs 5.2 | younger higher in 3/3; OR 0.53, 0.53, 0.61 (p ≈ 0.03, 0.07, 0.05) |
| HSV-2, three groups (excluded by the abstract) | Tables 4–9 | 39.3 vs 77.8; 46.8 vs 70.0; 56.3 vs 83.3 | older higher in 3/3, as the paper says |
Of the 12 non-HSV-2 cells, younger women's prevalence is higher with approximate p < 0.05 in 4 cells (all chlamydia, plus gonorrhea in South Africa). Older women's prevalence is higher with p < 0.05 in 1 cell, overall syphilis in South Africa. That cell has non-overlapping CIs, so the paper's own numbers support the reverse of the abstract statement there, as the paper's own Results text says. Across the three groups, syphilis does not show the pattern the abstract describes: the difference is reversed in South Africa, the point estimate is higher in older women in Southern/Eastern Africa, and the estimates are similar in Eastern Africa.
A separate, minor slip turned up in the Results trichomoniasis paragraph: "among 25–29-year-olds in the South African clinic/community-based populations … (summary estimate: 8.6% …)". The value matches the 25–49 row of Table 5, so "25–29" appears to be a typographical error for "25–49".
Figure
Figure 1. On the left, the abstract and author-summary statements, quoted verbatim. On the right, the paper's Results text on overall syphilis (Figs 5 and 6), with the 95% CIs as printed. The connector marks the conflict. In South African clinic/community women, the paper reports 4.3% (2.5–7.2) at 25–49 and 1.5% (1.0–2.2) at 15–24, and its Results text calls the older group's prevalence higher. The orange-marked line restates the reported interval bounds, which do not overlap. The blue-marked line is the odds ratio recomputed from those reported CIs on the logit scale: 2.95 (1.49–5.84), z = +3.10. The abstract statement implies an odds ratio below 1.
Figure 2. For each STI the abstract statement covers (all except HSV-2) and each of the three primary groups, the blue dot and line are the odds ratio, 25–49 versus 15–24 years, with its 95% CI. Each is recomputed from the paper's reported summary prevalence and 95% CI (overall syphilis from the Results text; other infections from the "Overall prevalence" rows of Tables 4–9). The axis is logarithmic, and the thin vertical line marks OR = 1 (no age difference). The shaded orange band is where the abstract statement says every point estimate should lie (younger higher). Chlamydia and gonorrhea lie inside it. Overall syphilis in South Africa lies entirely outside it, and so does high-titer syphilis in South Africa. Overall syphilis in Southern/Eastern Africa and trichomoniasis in South Africa have point estimates on the "older higher" side.
Why it matters
The age statement is one of the few cross-cutting patterns the abstract reports. It is the first finding listed after "several patterns emerged". The author summary repeats it in stronger form ("regardless of region or population type"), and the Discussion uses it to answer the "who" of the WHO Global Health Sector Strategy on STIs. Abstracts and author summaries are what most readers, guideline writers and secondary reviews take from a prevalence paper. A reader relying on them would conclude that syphilis, like chlamydia and gonorrhea, is concentrated in women under 25 in all three settings. The paper's own estimates show that it is not. In South African clinic/community populations, syphilis prevalence is roughly three times higher in women aged 25–49. Elsewhere it is higher or similar in older women.
For syphilis this difference has practical weight. Antenatal syphilis screening and congenital-syphilis elimination programs cover women of all reproductive ages, so where seroprevalence peaks matters for targeting. The underlying estimates, the Results text and the forest plots appear to be correct and internally consistent. The error is in how they are summarized. The paper's other conclusions, on chlamydia and gonorrhea age gradients, regional differences, higher-risk populations, and HSV-2 and BV burden, are not affected.
Explanations considered
- The abstract means "generally", not "in every case". This does not account for the numbers. The Discussion does say "generally", but the abstract and author summary do not hedge, and the author summary adds "regardless of region or population type". In any case the exception is not borderline. It is a full infection (syphilis) whose age difference is null or reversed in all three groups, and significantly reversed in one.
- "Higher" refers to statistically significant differences, and the syphilis differences are not significant. This does not account for the numbers. The paper did not test age differences; it compared forest plots visually (Methods). The one syphilis difference that the published CIs show clearly, in South Africa (non-overlapping CIs; recomputed p ≈ 0.002), runs in the opposite direction to the statement. Under this reading the trichomoniasis and gonorrhea cells would not support the statement either, since several are not significant.
- "Syphilis" in the abstract refers to high-titer (active) infection rather than overall seroprevalence. This does not account for the numbers. High-titer syphilis in South Africa is also higher in older women: 1.0% (0.5, 2.1) against 0.4% (0.3, 0.5), recomputed OR 2.52 (1.16–5.43). The Results describe the high-titer point estimates as "similar" across age groups, which is not "higher among 15–24-year-olds" either.
- The two age groups draw on different sets of studies, so the comparison is not like-for-like. This may be partly true for some strata (study inclusion was applied per infection, and some strata may lack a study). But the abstract statement compares exactly these published summary estimates, and the paper's own Results text makes the same comparison and finds the older group higher. It does not reconcile the abstract with the Results.
- High heterogeneity (I² = 94.8% for SA 25–49) makes the older-women estimate unreliable. This lowers the precision of that estimate, but the random-effects CI already reflects the heterogeneity, and it still excludes the younger estimate. The paper reports and interprets the estimate without reservation in its Results.
- The approximate test assumes independence of the two age-specific pooled estimates. The two estimates come from different women, so their sampling errors are independent. Many of the same studies contribute to both, and any positive correlation between a study's deviation at ages 15–24 and at 25–49 would make the difference more precise than assumed here, not less. For a few strata with zero-count studies, the paper used exact binomial limits rather than a logit standard error (Methods), so the back-calculated standard errors are approximate there (for example, high-titer syphilis in Southern/Eastern Africa). None of this affects the central fact: the reported SA overall-syphilis CIs do not overlap.
- Text conversion or archiving error. Ruled out. The abstract, author summary and Results sentences, including the 4.3% (2.5%, 7.2%) estimate, appear identically on the publisher's version of record. The table values were read from the publisher-deposited PMC XML.
Status
- Version of record. I checked the publisher's article page (journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002511). The abstract sentence, the author-summary sentence, the overall-syphilis Results paragraph, the high-titer sentence, the Discussion "who" sentence and the "25–29-year-olds" slip all appear there verbatim. Tables 4–9 are shown as images on that page, so their values were taken from the publisher-deposited PMC XML. Every quote in this write-up was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The supporting-information list includes S5 and S6 Figs (overall syphilis by initial region and population type, 15–24 and 25–49) and S9 and S10 Figs (overall syphilis restricted to the 17 studies with titer information). I did not open them. They are forest plots stratifying the same estimates and do not change what the main-text numbers show. S1–S4 Tables and S1–S3 Text (checklists and protocol) are not relevant to this point.
- Corrections. One correction exists, doi:10.1371/journal.pmed.1002608 (PMC6019393), dated 2018-06-26. It changes only the copyright license, from CC0 to CC BY 4.0, and does not address the age statement. Crossref, PMC and Europe PMC list no other notices, and none of them lists a retraction or expression of concern.
- Comments. The PubPeer API returned no feedback for the DOI. The PubPeer web search page blocked automated access. The publisher page shows "Reader Comments (0)". A web search for the title with "syphilis", "erratum" and "PubPeer" found only copies of the article and repository records, no comment or correction.
- As of 2026-10-23.
Suggested correction
- Abstract: replace "prevalence was higher among 15–24-year-old than 25–49-year-old women for all STIs except HSV-2" with a statement that matches the Results. For example: "chlamydia and gonorrhea prevalence was higher among 15–24-year-old than 25–49-year-old women in all three groups; syphilis prevalence was similar or higher among 25–49-year-olds (e.g., 4.3% vs 1.5% in South African clinic/community populations); HSV-2 prevalence increased with age."
- Author summary: revise "STI prevalence was highest among young women aged 15–24 years for all STIs except HSV-2, regardless of region or population type" in the same way, naming syphilis (and HSV-2) as exceptions.
- Discussion: qualify the opening summary and the "who" paragraph to state that syphilis did not follow the younger-higher pattern, notably in South African clinic/community populations.
- Results, "Syphilis, high-titer infection": consider stating the South African 25–49 estimate (1.0% [0.5, 2.1] vs 0.4% [0.3, 0.5]) rather than describing the point estimates as "similar".
- Minor: in the trichomoniasis paragraph, change "25–29-year-olds" to "25–49-year-olds".
Recomputation script (python recompute.py.txt)
"""Recompute the age comparisons behind the abstract's age statement in PMC5828349
(PLOS Medicine 2018, doi:10.1371/journal.pmed.1002511).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are the paper's own summary prevalence estimates, % (95% CI):
* overall syphilis: Results text, "Syphilis, overall" paragraphs (Figs 5 and 6);
* all other infections: the "Overall prevalence" row of the co-infection tables
(Table 4/5 South Africa clinic/community 15-24/25-49, Table 6/7 Southern/Eastern
Africa clinic/community, Table 8/9 Eastern Africa higher-risk).
The paper pools study log odds with a random-effects model and back-transforms the pooled
log odds and its 95% CI (Methods), so each CI is symmetric on the logit scale. The standard
error of each pooled log odds is therefore recovered as (logit(hi) - logit(lo)) / (2*1.96).
The 15-24 and 25-49 estimates come from different women, so the two pooled estimates are
treated as independent when comparing them (see writeup.md for the caveat).
"""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
import matplotlib.ticker
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.ppf(0.975)
GROUPS = ["South Africa clinic/community", "Southern/Eastern Africa clinic/community",
"Eastern Africa higher-risk"]
SHORT = {GROUPS[0]: "SA clinic/community", GROUPS[1]: "S/E Africa clinic/community",
GROUPS[2]: "E Africa higher-risk"}
# (infection, group): {age: (point, lo, hi)}, source
D = {
("Chlamydia", GROUPS[0]): ({"15-24": (15.1, 12.7, 17.8), "25-49": (7.0, 5.5, 8.8)}, "Tables 4/5"),
("Chlamydia", GROUPS[1]): ({"15-24": (2.7, 1.5, 3.9), "25-49": (1.2, 0.8, 1.7)}, "Tables 6/7"),
("Chlamydia", GROUPS[2]): ({"15-24": (10.3, 7.4, 14.1), "25-49": (6.2, 4.7, 8.2)}, "Tables 8/9"),
("Gonorrhea", GROUPS[0]): ({"15-24": (4.6, 3.3, 6.4), "25-49": (2.5, 1.6, 4.0)}, "Tables 4/5"),
("Gonorrhea", GROUPS[1]): ({"15-24": (1.7, 1.2, 2.6), "25-49": (0.9, 0.5, 1.6)}, "Tables 6/7"),
("Gonorrhea", GROUPS[2]): ({"15-24": (8.2, 5.8, 11.6), "25-49": (5.2, 3.9, 7.0)}, "Tables 8/9"),
("Syphilis, overall", GROUPS[0]): ({"15-24": (1.5, 1.0, 2.2), "25-49": (4.3, 2.5, 7.2)}, "Results text (Figs 5/6)"),
("Syphilis, overall", GROUPS[1]): ({"15-24": (1.4, 1.0, 1.9), "25-49": (2.0, 1.4, 2.6)}, "Results text (Figs 5/6)"),
("Syphilis, overall", GROUPS[2]): ({"15-24": (8.4, 6.8, 10.3), "25-49": (7.5, 5.5, 10.1)}, "Results text (Figs 5/6)"),
("Syphilis, high-titer", GROUPS[0]): ({"15-24": (0.4, 0.3, 0.5), "25-49": (1.0, 0.5, 2.1)}, "Tables 4/5"),
("Syphilis, high-titer", GROUPS[1]): ({"15-24": (1.1, 0.3, 1.9), "25-49": (0.9, 0.2, 1.7)}, "Tables 6/7"),
("Syphilis, high-titer", GROUPS[2]): ({"15-24": (4.5, 3.6, 5.5), "25-49": (2.8, 1.9, 4.2)}, "Tables 8/9"),
("Trichomoniasis", GROUPS[0]): ({"15-24": (7.9, 5.9, 10.5), "25-49": (8.6, 6.5, 11.5)}, "Tables 4/5"),
("Trichomoniasis", GROUPS[1]): ({"15-24": (6.7, 4.3, 10.1), "25-49": (5.8, 4.0, 8.4)}, "Tables 6/7"),
("Trichomoniasis", GROUPS[2]): ({"15-24": (12.7, 7.5, 20.6), "25-49": (10.0, 5.7, 17.1)}, "Tables 8/9"),
("HSV-2", GROUPS[0]): ({"15-24": (39.3, 34.4, 44.6), "25-49": (77.8, 75.6, 79.9)}, "Tables 4/5"),
("HSV-2", GROUPS[1]): ({"15-24": (46.8, 38.2, 55.6), "25-49": (70.0, 61.8, 77.1)}, "Tables 6/7"),
("HSV-2", GROUPS[2]): ({"15-24": (56.3, 49.2, 63.1), "25-49": (83.3, 79.3, 86.6)}, "Tables 8/9"),
}
def logit(p_pct):
p = p_pct / 100.0
return np.log(p / (1 - p))
def se_from_ci(lo, hi):
return (logit(hi) - logit(lo)) / (2 * Z975)
def compare(v):
a, b = v["15-24"], v["25-49"]
se = np.hypot(se_from_ci(a[1], a[2]), se_from_ci(b[1], b[2]))
d = logit(b[0]) - logit(a[0])
z = d / se
p = 2 * norm.sf(abs(z))
orr, lo, hi = np.exp(d), np.exp(d - Z975 * se), np.exp(d + Z975 * se)
overlap = not (a[2] < b[1] or b[2] < a[1])
return dict(ratio=b[0] / a[0], OR=orr, lo=lo, hi=hi, z=z, p=p, overlap=overlap,
younger_higher=a[0] > b[0])
# --------------------------------------------------------------- 1. all cells
print("=== 1. Older (25-49) vs younger (15-24) summary prevalence, every infection and group ===")
print("OR = odds ratio older/younger, recomputed from the reported estimates and CIs (logit scale)\n")
hdr = f"{'Infection':21s} {'Group':41s} {'15-24 %':>17s} {'25-49 %':>17s} {'ratio':>6s} {'OR (95% CI)':>20s} {'z':>6s} {'p':>7s} CIs overlap younger higher"
print(hdr)
R = {}
for (inf, grp), (v, src) in D.items():
c = compare(v)
R[(inf, grp)] = c
a, b = v["15-24"], v["25-49"]
print(f"{inf:21s} {grp:41s} {a[0]:5.1f} ({a[1]:.1f}, {a[2]:.1f}) {b[0]:5.1f} ({b[1]:.1f}, {b[2]:.1f}) "
f"{c['ratio']:6.2f} {c['OR']:6.2f} ({c['lo']:.2f}, {c['hi']:.2f}) {c['z']:+6.2f} {c['p']:7.2g} "
f"{str(c['overlap']):11s} {c['younger_higher']}")
# --------------------------------------------------------------- 2. the abstract statement
print("\n=== 2. Abstract: 'prevalence was higher among 15-24-year-old than 25-49-year-old women "
"for all STIs except HSV-2' (three primary groups) ===")
main_inf = ["Chlamydia", "Gonorrhea", "Syphilis, overall", "Trichomoniasis"]
cells = [(i, g) for i in main_inf for g in GROUPS]
yh = [k for k in cells if R[k]["younger_higher"]]
print(f"Non-HSV-2 STIs x groups (overall syphilis): {len(cells)} cells; younger point estimate higher in {len(yh)}")
sig_y = [k for k in cells if R[k]["younger_higher"] and R[k]["p"] < 0.05]
sig_o = [k for k in cells if (not R[k]["younger_higher"]) and R[k]["p"] < 0.05]
print(f" younger higher with approx. p < 0.05: {len(sig_y)}; older higher with approx. p < 0.05: {len(sig_o)}")
print("Cells where the 15-24 estimate is NOT higher (including high-titer syphilis):")
for k in cells + [("Syphilis, high-titer", g) for g in GROUPS]:
c = R[k]
if not c["younger_higher"]:
v = D[k][0]
print(f" {k[0]:21s} {SHORT[k[1]]:28s} 15-24 {v['15-24'][0]}% vs 25-49 {v['25-49'][0]}% "
f"OR {c['OR']:.2f} ({c['lo']:.2f}-{c['hi']:.2f}), z = {c['z']:+.2f}, p = {c['p']:.3f}, "
f"CIs overlap: {c['overlap']}")
for inf in ["Chlamydia", "Gonorrhea"]:
print(f"{inf}: younger higher in {sum(R[(inf, g)]['younger_higher'] for g in GROUPS)}/3 groups")
k = ("Syphilis, overall", GROUPS[0])
v = D[k][0]
print(f"\nKey cell, overall syphilis, SA clinic/community: 25-49 lower CI bound {v['25-49'][1]} > "
f"15-24 upper CI bound {v['15-24'][2]}: {v['25-49'][1] > v['15-24'][2]}")
print(f" ratio of prevalences 4.3/1.5 = {v['25-49'][0] / v['15-24'][0]:.2f}")
print(f" SE(logit) 15-24 = {se_from_ci(1.0, 2.2):.3f}, 25-49 = {se_from_ci(2.5, 7.2):.3f}")
k2 = ("Syphilis, high-titer", GROUPS[0])
v2 = D[k2][0]
print(f"High-titer syphilis, SA: 25-49 lower bound {v2['25-49'][1]} vs 15-24 upper bound {v2['15-24'][2]} "
f"(touch: {v2['25-49'][1] == v2['15-24'][2]}); ratio {v2['25-49'][0] / v2['15-24'][0]:.2f}")
# --------------------------------------------------------------- 3. consistency of table rows with text
print("\n=== 3. Table 'Overall prevalence' rows vs Results text (same estimates?) ===")
checks = [
("Chlamydia SA 15-24", "Table 4", (15.1, 12.7, 17.8), "Results", (15.1, 12.7, 17.8)),
("Trichomoniasis SA 25-49", "Table 5", (8.6, 6.5, 11.5), "Results", (8.6, 6.5, 11.5)),
("Trichomoniasis E Africa HR 15-24", "Table 8", (12.7, 7.5, 20.6), "Results", (12.7, 7.5, 20.6)),
("High-titer syphilis SA 15-24", "Table 4", (0.4, 0.3, 0.5), "Results", (0.4, 0.3, 0.5)),
("HSV-2 SA 15-24", "Table 4", (39.3, 34.4, 44.6), "Results", (39.3, 34.3, 44.6)),
]
for lab, s1, x1, s2, x2 in checks:
print(f" {lab:34s} {s1}: {x1} {s2}: {x2} identical: {x1 == x2}")
# --------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# ---- fig1: abstract vs results text panels
c = R[("Syphilis, overall", GROUPS[0])]
W, H = 7.2, 4.75
fig = plt.figure(figsize=(W, H), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
def panel(x, y, w, h, title, body):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=INK2, lw=1.2))
ax.text(x + 0.14, y + h - 0.16, title, fontsize=10.5, fontweight="bold", va="top", color=INK,
linespacing=1.3)
ax.text(x + 0.14, y + h - 0.78, body, fontsize=10, va="top", color=INK, linespacing=1.45)
PY, PH = 1.5, 3.15
panel(0.1, PY, 2.95, PH, "Abstract (Methods and findings)\nand Author summary",
"“… prevalence was higher\namong 15–24-year-old than\n25–49-year-old women for\n"
"all STIs except HSV-2.”\n(three primary groups)\n\n"
"“STI prevalence was highest\namong young women aged\n15–24 years for all STIs except\n"
"HSV-2, regardless of region\nor population type.”")
panel(4.15, PY, 2.95, PH, "Results, “Syphilis, overall”\n(Figs 5 and 6)",
"South Africa clinic/community\n 15–24 y: 1.5% (1.0, 2.2)\n 25–49 y: 4.3% (2.5, 7.2)\n"
" “… was higher than that\n among younger women”\n\n"
"S/E Africa clinic/community\n 25–49 vs 15–24: 2.0% vs 1.4%\n"
"E Africa higher-risk\n 25–49 vs 15–24: 7.5% vs 8.4%")
ym = PY + PH / 2 + 0.2
ax.annotate("", xy=(4.12, ym), xytext=(3.08, ym),
arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
ax.text(3.6, ym + 0.12, "contradicts", ha="center", va="bottom", fontsize=10, color=INK, fontweight="bold")
ax.text(3.6, ym - 0.12, "for syphilis\nin South\nAfrica", ha="center", va="top", fontsize=10, color=INK2)
# legend-style lines: reported (orange) and recomputed (blue)
ax.plot([0.12, 0.42], [1.08, 1.08], color=ORANGE, lw=2)
ax.plot([0.27], [1.08], "o", color=ORANGE, ms=8)
ax.text(0.55, 1.08, "Reported in the paper: SA 25–49 lower CI bound 2.5% > 15–24 upper CI\n"
"bound 2.2%, so the two intervals do not overlap.", fontsize=10, color=INK, va="center")
ax.plot([0.12, 0.42], [0.48, 0.48], color=BLUE, lw=2)
ax.plot([0.27], [0.48], "o", color=BLUE, ms=8)
ax.text(0.55, 0.48, f"Recomputed from those CIs (logit scale): odds ratio 25–49 vs 15–24\n"
f"= {c['OR']:.2f} (95% CI {c['lo']:.2f}–{c['hi']:.2f}), z = {c['z']:+.2f}, two-sided p ≈ "
f"{c['p']:.3f}.", fontsize=10, color=INK, va="center")
save(fig, "fig1")
# ---- fig2: OR older/younger for every non-HSV-2 cell, log axis
order = ["Chlamydia", "Gonorrhea", "Trichomoniasis", "Syphilis, overall", "Syphilis, high-titer"]
fig, ax = plt.subplots(figsize=(7.2, 7.4), facecolor=BG)
ax.set_facecolor(BG)
ys, labels, rows, heads = [], [], [], []
y = 0.0
for inf in order:
heads.append((y, inf))
y -= 0.85
for g in GROUPS:
ys.append(y); labels.append(SHORT[g]); rows.append((inf, g))
y -= 0.8
y -= 0.35
ax.axvspan(0.1, 1, color=ORANGE, alpha=0.08, lw=0)
ax.axvline(1, color=INK2, lw=0.8)
for yy, k in zip(ys, rows):
c = R[k]
ax.plot([c["lo"], c["hi"]], [yy, yy], color=BLUE, lw=2, solid_capstyle="butt")
ax.plot([c["OR"]], [yy], "o", color=BLUE, ms=8, mec=BG, mew=1.5)
ax.text(1.02, yy, f"{c['OR']:.2f} ({c['lo']:.2f}–{c['hi']:.2f})", va="center",
fontsize=10, color=INK2, transform=ax.get_yaxis_transform())
for yy, inf in heads:
ax.text(-0.02, yy, inf, ha="right", va="center", fontsize=10.5, fontweight="bold", color=INK,
transform=ax.get_yaxis_transform())
ax.set_xscale("log")
ax.set_xlim(0.15, 7)
ax.set_xticks([0.2, 0.5, 1, 2, 5])
ax.set_xticklabels(["0.2", "0.5", "1", "2", "5"])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks(ys); ax.set_yticklabels(labels, fontsize=10, color=INK)
top = 2.2
ax.set_ylim(min(ys) - 0.6, top)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s_ in ("top", "right", "left"):
ax.spines[s_].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Odds ratio, 25–49 vs 15–24 years (log scale)", color=INK)
ax.text(0.95, top - 0.05, "← younger higher\n(abstract: every row\nbelongs in this band)", ha="right",
va="top", fontsize=10, color=INK)
ax.text(1.05, top - 0.05, "older higher →", ha="left", va="top", fontsize=10, color=INK2)
ax.text(1.02, top - 0.05, "OR (95% CI)", ha="left", va="top", fontsize=10, color=INK, fontweight="bold",
transform=ax.get_yaxis_transform())
h1, = ax.plot([], [], "o-", color=BLUE, lw=2, ms=8)
h2 = plt.Rectangle((0, 0), 1, 1, color=ORANGE, alpha=0.25, lw=0)
ax.legend([h1, h2], ["Recomputed from the paper's estimates and 95% CIs",
"Region implied by the abstract statement"],
loc="upper left", bbox_to_anchor=(-0.55, -0.075), frameon=False, fontsize=10, ncol=1)
fig.subplots_adjust(left=0.36, right=0.77, top=0.98, bottom=0.15)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Abstract reverses the direction of the paper's exposed-versus-occluded skin connectivity result
PLoS Comput Biol (2018) · doi:10.1371/journal.pcbi.1006099 · PMC5927471
Summary
The abstract says that skin sites exposed to the external environment have less connected phage-bacteria networks and are "likely more susceptible to network degradation". The Results say the opposite: occluded sites were less connected than exposed and intermittently occluded sites (p < 0.001, Fig 4B). The Discussion also says exposed and intermittently exposed sites were more connected and predicted to be more resilient. Fig 4B as published agrees with the Results and Discussion: the median eigen centrality is about 0.176 for exposed sites and 0.138 for occluded sites, and the paper reads higher values as "more connected" throughout. The body of the paper is internally consistent, so the analysis is unaffected. But the abstract, which is what most readers and indexing services see, states one of the paper's three headline findings in the reverse direction, and so does its claim about vulnerability to degradation.
What the paper reports
Abstract:
Network structure differed between skin sites, with those exposed to the external environment being less connected and likely more susceptible to network degradation by microbial extinction events.
Results, "Network structures across the human skin landscape", second paragraph:
The average centrality of each sample was quantified using the weighted eigenvector centrality metric. Intermittently moist skin sites (dynamic sites that fluctuate between being moist and dry) were significantly more connected than the moist and sebaceous environments (p-value < 0.001, Fig 4A). Also, skin sites that were occluded from the environment were less connected than those that were constantly exposed to the environment or only intermittently occluded (p-value < 0.001, Fig 4B).
Discussion, third paragraph:
Network structure differed between environmentally exposed and occluded skin sites. The sites under greater environmental fluctuation and exposure (the exposed and intermittently exposed sites) were more connected and therefore were predicted to have a higher resilience against network degradation when random nodes are removed from the network.
Together these data suggested that body sites under greater degrees of fluctuation harbored more highly connected microbiomes that are potentially more robust to network disruption by extinction events.
Fig 4 legend:
(A) Notched box-plot depicting differences in average eigenvector centrality between moist, intermittently moist, and sebaceous skin sites and (B) occluded, intermittently occluded, and exposed sites.
Sample sizes for each group were: Moist = 81, Sebaceous = 56, IntMoist = 56, Occluded = 106, Exposed = 61, IntOccluded = 26.
What the paper's own numbers show
Method. The paper prints no group values for connectivity in text or tables. They appear only as box plots in Fig 4A and 4B. I took the Fig 4 raster (3800 × 3729 px) embedded on page 11 of the publisher's PDF. For each box I located the median line and the two hinges (quartiles) by pixel row, and converted rows to eigen-centrality units with a straight line fitted to the panel's own y-axis ticks. The fit is exact to within 1e-4, and one pixel is about 0.0003 units, so each value is good to about ±0.002. I then checked the group sizes in the legend. I also took the pairwise Wilcoxon p-values that the authors' public code repository writes for this figure (rtables/skinboxsig.tsv; R's pairwise.wilcox.test, Holm adjustment by default). Finally I compared the direction each passage states with the direction the figure shows. The paper's convention, which the Results apply to both panels, is that a higher eigen centrality means "more connected". Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's figure and data |
|---|---|---|---|
| Exposed vs occluded connectivity | Abstract | Exposed sites "less connected" and "more susceptible to network degradation" | Exposed median 0.176 > occluded median 0.138: exposed sites are the more connected |
| Occluded vs exposed and intermittently occluded | Results; Fig 4B | Occluded "less connected" (p < 0.001) | Occluded 0.138 < exposed 0.176 and IntOccluded 0.185, which agrees |
| Exposed and intermittently exposed | Discussion | "more connected", "higher resilience" | Agrees with Fig 4B |
| Fig 4B medians (Q1–Q3) | Fig 4B | Box plots only | Exposed 0.176 (0.147–0.196); IntOccluded 0.185 (0.172–0.194); Occluded 0.138 (0.094–0.173) |
| Median differences | Fig 4B | Not printed | Exposed − Occluded = +0.038; IntOccluded − Occluded = +0.047; IntOccluded − Exposed = +0.009 |
| Separation of the groups | Fig 4B | Not printed | The exposed median (0.176) lies above the occluded upper quartile (0.173) |
| Fig 4A medians (Q1–Q3), the convention check | Fig 4A; Results | IntMoist "more connected" than Moist and Sebaceous | IntMoist 0.187 (0.175–0.197) > Sebaceous 0.145 (0.122–0.167) and Moist 0.137 (0.078–0.176), so higher means "more connected" |
| Group sizes | Fig 4 legend | 81 + 56 + 56; 106 + 61 + 26 | Both groupings sum to 193 samples, which is consistent |
| Occluded vs exposed p | Results; repository table | p < 0.001 | Holm-adjusted Wilcoxon p = 1.38e-05, which agrees |
| Occluded vs IntOccluded p | Results; repository table | p < 0.001 | p = 1.38e-05, which agrees |
| IntOccluded vs exposed p | Repository table | Not stated in text | p = 0.277 (no difference) |
The Results, the Discussion, the figure and the repository's p-values agree with each other. The abstract is the only place in the paper where exposed sites are described as less connected.
Figure
Figure 1. The left panel gives the abstract's sentence on exposed skin sites. The right panel gives the Results sentence on Fig 4B and the Discussion sentence on the same comparison. All three are quoted as printed and framed in orange. The connector marks the conflict: the abstract says exposed sites are "less connected", while the Results and the Discussion say they are more connected than occluded sites. The blue strip gives the Fig 4B medians that recompute.py.txt reads from the published figure. In those medians, exposed (0.176) and intermittently occluded (0.185) sites are above occluded (0.138) sites, which agrees with the body of the paper.
Figure 2. Median eigen centrality (dot) and interquartile range (bar) for the six skin-site groups. All values are read by recompute.py.txt from the published Fig 4A and 4B and are shown on one axis that starts at zero. The moisture panel (top) fixes the paper's convention: the Results call IntMoist, the highest group, "more connected". The occlusion panel (bottom) shows exposed sites above occluded sites. The orange label gives the abstract's statement, which runs the other way. Group sizes are from the Fig 4 legend.
Why it matters
- It is a headline finding. The abstract gives three findings from this study: networks are person-specific, high-fat diets go with less connected networks, and skin networks depend on environmental exposure. The third is printed with the wrong sign.
- The ecological inference is reversed too. The abstract says exposed sites are "likely more susceptible to network degradation by microbial extinction events". The Discussion draws the opposite conclusion from the same result: exposed and intermittently exposed sites are "predicted to have a higher resilience against network degradation". It builds its broader thesis on that conclusion, that "body sites under greater degrees of fluctuation harbored more highly connected microbiomes". A reader of the abstract alone would take away the opposite of the paper's conclusion.
- Abstracts travel further than full texts. Indexing services, citing papers and reviews usually quote the abstract. A study citing this paper for "exposed skin sites have fragile phage-bacteria networks" would be citing the reverse of what the data show.
- What is not affected. The analysis, Fig 4, the p-values, the Results and the Discussion are consistent with one another. The abstract's general framing still holds: network structure differs between skin sites, and environment may shape phage-bacteria interactions. Only the direction of the skin example is wrong.
Explanations considered
- The abstract compares exposed sites with a different group. It cannot. The occlusion grouping has only three levels. Exposed sites are above occluded sites (0.176 vs 0.138, p = 1.38e-05) and do not differ from intermittently occluded sites (p = 0.277). No comparison in the paper makes exposed sites less connected. The moisture grouping is a separate classification, and the abstract's sentence is explicitly about exposure to the external environment.
- Lower centrality means more connected. The paper never reads its values that way. The Results call the highest group in Fig 4A (IntMoist) "more connected" and the lowest group in Fig 4B (Occluded) "less connected". In the gut analysis, "reduced degree centrality" and "decreased closeness centrality" go with "less connected" networks. The authors' script (
bin/CompareSkin.R) computes the plotted value as igraph'scentr_eigen(...)$centralization, a graph-level eigenvector centralization score. I did not re-run that analysis. However that metric is best interpreted, the paper reads it one way in the Results and the Discussion, and the abstract states the opposite under that same reading. - The abstract summarises the supplementary dissimilarity analysis (Fig 4D and 4F). That analysis says only that "Occluded sites were significantly different from exposed and intermittently occluded sites". It compares the spread of network dissimilarities (ANOVA and Tukey test on group dispersion) and has no direction of connectedness. It cannot support "less connected".
- A conversion or typesetting artifact. Ruled out. The same abstract wording appears in the publisher's HTML and PDF, in the PMC copy and in the Europe PMC record.
- An error in reading the figure. It cannot change the conclusion. The direction comes from the paper's own Results and Discussion text. The figure reading only corroborates it. The exposed and occluded medians sit about 120 pixel rows apart on the 3800-pixel raster, a gap of 0.038 units against a reading precision of about 0.002.
- A later correction fixed it. None was found (see Status). Crossref has one update, a publisher "new version" entry dated 2018-04-30. It matches the switch from the uncorrected proof to the final version and does not change the abstract.
Status
As of 2026-10-23:
- Version of record. The abstract wording quoted above is in the publisher's HTML and PDF (24 pages, dated April 18, 2018), the PMC copy (PMC5927471) and the Europe PMC record (PMID 29668682). For this write-up, Fig 4 was read from the publisher PDF as held in the PMC open-data collection (
pmc-oa-opendata/PMC5927471.1/PMC5927471.1.pdf). - Supplements. S1–S14 Fig and S1–S2 Table are listed on the publisher page. S14 Fig only tests whether the Fig 4A and 4B p-values survive added edge noise, and gives no group values. The authors' public code repository, linked from the article, contains the analysis script (
bin/CompareSkin.R) and the p-value table used above (rtables/skinboxsig.tsv). It has no per-sample centrality values. - Corrections. No correction, erratum, expression of concern or retraction was found. Crossref has only the 2018-04-30 "new version" entry and a link to the bioRxiv preprint (10.1101/144642). PMC and Europe PMC list no notices. Europe PMC's comment/correction list holds only the preprint link.
- PubPeer. The PubPeer search API and site returned a human-verification challenge to automated requests, so a direct search could not be done. A web search for the article title on PubPeer found no PubPeer thread.
Suggested correction
In the Abstract, replace
Network structure differed between skin sites, with those exposed to the external environment being less connected and likely more susceptible to network degradation by microbial extinction events.
with
Network structure differed between skin sites, with those occluded from the external environment being less connected and likely more susceptible to network degradation by microbial extinction events.
or, stated from the exposed side,
Network structure differed between skin sites, with those exposed to the external environment being more connected and likely more resilient to network degradation by microbial extinction events.
Either wording matches the Results, Fig 4B and the Discussion. No other part of the paper needs to change.
Recomputation script (python recompute.py.txt)
"""Recompute the skin-site connectivity check for PMC5927471
(PLoS Comput Biol 2018, doi:10.1371/journal.pcbi.1006099).
Run: python recompute.py.txt
Needs numpy, matplotlib and Pillow; PyMuPDF (pymupdf / fitz) to read the figure out of
the published PDF. Prints every recomputed number used in writeup.md and writes
fig1.svg/png, fig2.svg/png and candidates/cand1..cand5.svg/png next to this file.
Inputs
* Text quoted from the article as archived in PubMed Central (Abstract; Results,
"Network structures across the human skin landscape"; Discussion; Fig 4 legend).
* Fig 4 as printed in the publisher's PDF (page 11, embedded 3800 x 3729 px raster).
The PDF is fetched from the public PMC open-data bucket, which holds the same
publisher PDF (24 pages):
https://pmc-oa-opendata.s3.amazonaws.com/PMC5927471.1/PMC5927471.1.pdf
Box-plot medians and hinges (quartiles) of panels 4A and 4B are read off that
raster by pixel position against the panel's own y-axis ticks.
* The authors' public code repository (linked from the article's data-availability
statement), table rtables/skinboxsig.tsv: pairwise Wilcoxon p-values from R's
pairwise.wilcox.test (default Holm adjustment). Its six values are transcribed
below as REPO_P.
If the PDF download fails, the script falls back to the values it read on 2026-10-23
(stored below as FALLBACK_FIG) and says so.
Reading convention used by the paper itself: a higher eigen-centrality value is
described as "more connected" (Results: IntMoist, the highest box in Fig 4A, is "more
connected" than Moist and Sebaceous; Occluded, the lowest box in Fig 4B, is "less
connected").
"""
import io
import os
import tempfile
import textwrap
import urllib.request
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
PDF_URL = "https://pmc-oa-opendata.s3.amazonaws.com/PMC5927471.1/PMC5927471.1.pdf"
# ------------------------------------------------------------------ text (verbatim)
Q_ABSTRACT = ("Network structure differed between skin sites, with those exposed to the "
"external environment being less connected and likely more susceptible to "
"network degradation by microbial extinction events.")
Q_RESULTS = ("Also, skin sites that were occluded from the environment were less connected "
"than those that were constantly exposed to the environment or only "
"intermittently occluded (p-value < 0.001, Fig 4B).")
Q_RESULTS_A = ("Intermittently moist skin sites (dynamic sites that fluctuate between being "
"moist and dry) were significantly more connected than the moist and "
"sebaceous environments (p-value < 0.001, Fig 4A).")
Q_DISCUSSION = ("The sites under greater environmental fluctuation and exposure (the exposed "
"and intermittently exposed sites) were more connected and therefore were "
"predicted to have a higher resilience against network degradation when "
"random nodes are removed from the network.")
# Fig 4 legend
N = {"Moist": 81, "Sebaceous": 56, "IntMoist": 56,
"Occluded": 106, "Exposed": 61, "IntOccluded": 26}
# ------------------------------------------------------------------ fallbacks (2026-10-23)
FALLBACK_FIG = { # group: (Q1, median, Q3) in eigen-centrality units
"IntMoist": (0.175, 0.187, 0.197), "Moist": (0.078, 0.137, 0.176),
"Sebaceous": (0.122, 0.145, 0.167), "Exposed": (0.147, 0.176, 0.196),
"IntOccluded": (0.172, 0.185, 0.194), "Occluded": (0.094, 0.138, 0.173)}
REPO_P = { # rtables/skinboxsig.tsv, transcribed 2026-10-23
("IntOccluded", "Exposed"): 0.277498657068735,
("Occluded", "Exposed"): 1.38010591458678e-05,
("Occluded", "IntOccluded"): 1.38010591458678e-05,
("Sebaceous", "Moist"): 0.132875880147664,
("Moist", "IntMoist"): 2.66084000742917e-08,
("Sebaceous", "IntMoist"): 4.21495886720462e-08}
# ------------------------------------------------------------------ figure digitising
def fetch(url, timeout=60):
req = urllib.request.Request(url, headers={"User-Agent": "recompute.py"})
with urllib.request.urlopen(req, timeout=timeout) as r:
return r.read()
def load_fig4_raster():
"""Return Fig 4 as a greyscale numpy array, taken from the publisher PDF."""
try:
import pymupdf as fitz
except ImportError:
import fitz
from PIL import Image
pdf = fetch(PDF_URL)
with tempfile.TemporaryDirectory() as td:
path = os.path.join(td, "paper.pdf")
with open(path, "wb") as f:
f.write(pdf)
doc = fitz.open(path)
print(f"Publisher PDF: {doc.page_count} pages")
for pno in range(doc.page_count):
for img in doc[pno].get_images():
if (img[2], img[3]) == (3800, 3729):
raw = doc.extract_image(img[0])["image"]
print(f"Fig 4 raster found on PDF page {pno + 1}: 3800 x 3729 px")
return np.asarray(Image.open(io.BytesIO(raw)).convert("L")).astype(int)
raise RuntimeError("Fig 4 raster not found in PDF")
def axis_ticks(a, x0, x1):
"""y-axis column of a panel and the row centres of its tick marks (top to bottom)."""
dark = a < 80
sub = dark[:1150, x0:x1]
ax = int(sub.sum(0).argmax())
rows = [y for y in range(1150) if sub[y, ax - 12:ax - 2].all()]
groups = []
for y in rows:
if groups and y - groups[-1][-1] <= 1:
groups[-1].append(y)
else:
groups.append([y])
return ax + x0, [float(np.mean(g)) for g in groups]
def dark_runs(a, col, ylo=150, yhi=1100, thr=80):
out, cur = [], []
for y in range(ylo, yhi):
if a[y, col] < thr:
cur.append(y)
elif cur:
out.append(cur)
cur = []
if cur:
out.append(cur)
return out
def digitise(a):
res = {}
panels = [("A", (0, 1900), ["IntMoist", "Moist", "Sebaceous"], [158, 323, 488]),
("B", (1900, 3800), ["Exposed", "IntOccluded", "Occluded"], [763, 928, 1093])]
for panel, (x0, x1), labels, centres in panels:
axcol, tk = axis_ticks(a, x0, x1)
vals = [0.3, 0.2, 0.1, 0.0][:len(tk)]
slope, icpt = np.polyfit(tk, vals, 1)
resid = np.max(np.abs(np.polyval([slope, icpt], tk) - vals))
print(f"Fig 4{panel}: y-axis at x={axcol}px, ticks at rows "
f"{[round(t, 1) for t in tk]} = {vals}; 1 px = {abs(slope):.5f}; "
f"max tick misfit {resid:.1e}")
for lab, cx in zip(labels, centres):
xc = int(cx * 3800 / 1200) - 60 # just off the whisker, inside the notch
runs = dark_runs(a, xc)
imed = max(range(len(runs)), key=lambda i: len(runs[i])) # median: thickest line
r_med = float(np.mean(runs[imed]))
r_q3 = float(np.mean(runs[imed - 1])) # upper hinge line
r_q1 = float(np.mean(runs[imed + 1])) # lower hinge line
q1, med, q3 = (float(slope * r + icpt) for r in (r_q1, r_med, r_q3))
res[lab] = (q1, med, q3)
print(f" {lab:12s} n={N[lab]:3d} rows Q3/med/Q1 = {r_q3:6.1f}/{r_med:6.1f}/"
f"{r_q1:6.1f} (median line {len(runs[imed])} px thick) -> "
f"Q1 {q1:.3f} median {med:.3f} Q3 {q3:.3f}")
return res
try:
FIG = digitise(load_fig4_raster())
FIG_SRC = "read from the published Fig 4 (publisher PDF, p. 11)"
except Exception as e: # network or library problem
print(f"!! could not read the figure ({e}); using values read on 2026-10-23")
FIG, FIG_SRC = FALLBACK_FIG, "values read on 2026-10-23 (fallback)"
for k, (q1, m, q3) in FIG.items():
print(f" {k:12s} Q1 {q1:.3f} median {m:.3f} Q3 {q3:.3f}")
P, P_SRC = REPO_P, "authors' repository table rtables/skinboxsig.tsv"
med = {k: v[1] for k, v in FIG.items()}
print()
print("Group sizes (Fig 4 legend):")
print(f" moisture {N['Moist']} + {N['Sebaceous']} + {N['IntMoist']} = "
f"{N['Moist'] + N['Sebaceous'] + N['IntMoist']}")
print(f" occlusion {N['Occluded']} + {N['Exposed']} + {N['IntOccluded']} = "
f"{N['Occluded'] + N['Exposed'] + N['IntOccluded']}")
print()
print(f"Pairwise Wilcoxon p-values, Holm-adjusted ({P_SRC}):")
for (g1, g2), v in P.items():
print(f" {g1:11s} vs {g2:11s} p = {v:.3g} < 0.001: {v < 1e-3}")
d_eo = med["Exposed"] - med["Occluded"]
d_io = med["IntOccluded"] - med["Occluded"]
print()
print("Median differences read from Fig 4B (eigen-centrality units):")
print(f" Exposed - Occluded = {med['Exposed']:.3f} - {med['Occluded']:.3f} = {d_eo:+.3f}")
print(f" IntOccluded - Occluded = {med['IntOccluded']:.3f} - {med['Occluded']:.3f} = {d_io:+.3f}")
print(f" IntOccluded - Exposed = {med['IntOccluded'] - med['Exposed']:+.3f}")
print(f" Exposed median lies above the Occluded upper hinge (Q3 {FIG['Occluded'][2]:.3f}): "
f"{med['Exposed'] > FIG['Occluded'][2]}")
print()
print("Direction checks (paper's convention: higher centrality = more connected):")
print(" Fig 4A: IntMoist highest (Results: 'more connected') ->",
med["IntMoist"] > max(med["Moist"], med["Sebaceous"]))
print(" Fig 4B: Occluded lowest (Results: 'less connected') ->",
med["Occluded"] < min(med["Exposed"], med["IntOccluded"]))
print(" Discussion 'exposed ... more connected' agrees with Fig 4B ->",
med["Exposed"] > med["Occluded"])
print(" Abstract 'exposed ... less connected' agrees with Fig 4B ->",
med["Exposed"] < med["Occluded"])
# ================================================================== figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none"})
W = 7.2
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg", bbox_inches="tight", pad_inches=0.12)
fig.savefig(p + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
def wrap(s, n):
return "\n".join(textwrap.wrap(s, n))
def panel(ax, x, y, w, h, edge, title, body, n=46, body_size=10.5):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc="white", ec=edge, lw=2, transform=ax.transAxes))
ax.text(x + 0.02, y + h - 0.035, title, transform=ax.transAxes, fontsize=11,
fontweight="bold", va="top", color=INK)
ax.text(x + 0.02, y + h - 0.115, body, transform=ax.transAxes, fontsize=body_size,
va="top", color=INK, linespacing=1.35)
def blank(h):
fig = plt.figure(figsize=(W, h))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
return fig, ax
SHORT_ABS = ("“…those exposed to the external environment being less connected "
"and likely more susceptible to network degradation by microbial extinction "
"events.”")
SHORT_RES = ("“…skin sites that were occluded from the environment were less "
"connected than those that were constantly exposed to the environment or only "
"intermittently occluded (p-value < 0.001, Fig 4B).”")
SHORT_DIS = ("“…(the exposed and intermittently exposed sites) were more connected "
"and therefore were predicted to have a higher resilience against network "
"degradation…”")
def legend_row(ax, y, items):
x = 0.03
for color, label in items:
ax.add_patch(FancyBboxPatch((x, y - 0.012), 0.03, 0.024, boxstyle="round,pad=0,"
"rounding_size=0.004", fc="white", ec=color, lw=2, transform=ax.transAxes))
ax.text(x + 0.04, y, label, transform=ax.transAxes, va="center", fontsize=10,
color=INK2)
x += 0.06 + 0.0105 * len(label)
# ---------- candidate 1: two text panels + connector + figure strip (used as fig1)
def cand1():
fig, ax = blank(5.0)
panel(ax, 0.01, 0.35, 0.39, 0.62, ORANGE, "Abstract", wrap(SHORT_ABS, 33))
panel(ax, 0.60, 0.35, 0.39, 0.62, ORANGE, "Results and Discussion",
wrap(SHORT_RES, 35) + "\n\n" + wrap(SHORT_DIS, 35), body_size=10)
ax.add_patch(FancyArrowPatch((0.405, 0.665), (0.595, 0.665), arrowstyle="<|-|>",
mutation_scale=14, lw=2, color=INK2, transform=ax.transAxes))
ax.text(0.5, 0.70, "Opposite\ndirection", transform=ax.transAxes, ha="center",
va="bottom", fontsize=10.5, color=INK, fontweight="bold")
ax.text(0.5, 0.63, "exposed sites:\n“less connected”\nvs\n“more connected”",
transform=ax.transAxes, ha="center", va="top", fontsize=10, color=INK2)
ax.add_patch(FancyBboxPatch((0.01, 0.10), 0.98, 0.20, boxstyle="round,pad=0,"
"rounding_size=0.012", fc="white", ec=BLUE, lw=2, transform=ax.transAxes))
ax.text(0.03, 0.245, "Fig 4B medians read from the published figure (eigen centrality;"
" higher = more connected)", transform=ax.transAxes,
fontsize=10, color=INK2, va="center")
ax.text(0.03, 0.155, f"Exposed {med['Exposed']:.3f} · IntOccluded "
f"{med['IntOccluded']:.3f} · Occluded {med['Occluded']:.3f}"
" \u2192 exposed above occluded",
transform=ax.transAxes, fontsize=10.5, color=INK, va="center", fontweight="bold")
legend_row(ax, 0.04, [(ORANGE, "Text as printed in the paper"),
(BLUE, "Values read from the paper's Fig 4B")])
return fig
# ---------- candidate 2: direction ledger
def cand2():
fig, ax = blank(3.6)
rows = [("Abstract", "Exposed sites less connected; more\nsusceptible to network degradation",
"<", ORANGE),
("Results (Fig 4B)", "Occluded less connected than exposed\nand int. occluded (p < 0.001)",
">", ORANGE),
("Discussion", "Exposed and intermittently exposed\nmore connected; higher resilience",
">", ORANGE),
("Fig 4B medians", f"Exposed {med['Exposed']:.3f} vs Occluded {med['Occluded']:.3f}\n"
f"(IntOccluded {med['IntOccluded']:.3f})", ">", BLUE)]
ax.text(0.03, 0.93, "Source", transform=ax.transAxes, fontweight="bold", fontsize=10.5)
ax.text(0.25, 0.93, "What it says", transform=ax.transAxes, fontweight="bold",
fontsize=10.5)
ax.text(0.71, 0.93, "Exposed vs occluded", transform=ax.transAxes, fontweight="bold",
fontsize=10.5)
for i, (src, txt, sign, col) in enumerate(rows):
y = 0.76 - i * 0.19
ax.plot([0.02, 0.98], [y + 0.09, y + 0.09], color=GRID, lw=1, transform=ax.transAxes)
ax.plot([0.015, 0.015], [y - 0.07, y + 0.07], color=col, lw=4, transform=ax.transAxes,
solid_capstyle="butt")
ax.text(0.03, y, src, transform=ax.transAxes, va="center", fontsize=10.5)
ax.text(0.25, y, txt, transform=ax.transAxes, va="center", fontsize=10, color=INK2)
word = "▼ LESS connected" if sign == "<" else "▲ MORE connected"
ax.text(0.71, y, word, transform=ax.transAxes, va="center", fontsize=10.5,
fontweight="bold", color=INK)
legend_row(ax, 0.03, [(ORANGE, "Stated in the paper's text"),
(BLUE, "Read from the paper's Fig 4B")])
return fig
# ---------- candidate 3: dot + IQR plot of medians (used as fig2)
def cand3():
order = ["IntMoist", "Moist", "Sebaceous", "Exposed", "IntOccluded", "Occluded"]
fig, ax = plt.subplots(figsize=(W, 4.0))
ys = [6.4, 5.4, 4.4, 2.6, 1.6, 0.6]
for g, y in zip(order, ys):
q1, m, q3 = FIG[g]
ax.plot([q1, q3], [y, y], color=BLUE, lw=2, solid_capstyle="round", zorder=2)
ax.plot(m, y, "o", ms=9, color=BLUE, mec="white", mew=1.5, zorder=3)
ax.text(q3 + 0.004, y, f"{m:.3f}", va="center", fontsize=10, color=INK)
ax.set_yticks(ys)
ax.set_yticklabels([f"{g} (n={N[g]})" for g in order])
ax.set_xlim(0, 0.32)
ax.set_xticks([0, 0.05, 0.10, 0.15, 0.20, 0.25])
ax.set_ylim(0, 7.3)
ax.set_xlabel("Eigen centrality: median (dot) and interquartile range (bar)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.axhline(3.5, color=GRID, lw=1)
ax.text(0.003, 7.0, "Fig 4A, moisture", fontsize=10, color=INK2, fontweight="bold")
ax.text(0.003, 3.15, "Fig 4B, occlusion", fontsize=10, color=INK2, fontweight="bold")
ax.text(0.232, 5.4, "Results: IntMoist\n“more connected”", fontsize=10,
color=INK2, va="center")
ax.text(0.232, 2.15, "Abstract: exposed\nsites “less\nconnected”", fontsize=10,
color=ORANGE, fontweight="bold", va="center")
ax.text(0.232, 0.75, "Results, Discussion:\noccluded less,\nexposed more", fontsize=10,
color=INK2, va="center")
h1, = ax.plot([], [], "o-", color=BLUE, ms=9, lw=2)
fig.legend([h1], ["Values read from the paper's Fig 4 (publisher PDF)"],
loc="lower center", bbox_to_anchor=(0.5, -0.04), frameon=False, fontsize=10)
fig.tight_layout(rect=(0, 0.05, 1, 1))
return fig
# ---------- candidate 4: two-item slope chart, ordering implied vs ordering shown
def cand4():
fig, ax = plt.subplots(figsize=(W, 3.6))
left = {"Occluded": 2, "Exposed": 1}
right = {"Exposed": 2 if med["Exposed"] > med["Occluded"] else 1,
"Occluded": 1 if med["Exposed"] > med["Occluded"] else 2}
for g in ("Exposed", "Occluded"):
ax.plot([0, 1], [left[g], right[g]], color=INK2, lw=2, zorder=1)
ax.plot(0, left[g], "o", ms=11, color=ORANGE, mec="white", mew=1.5, zorder=3)
ax.plot(1, right[g], "o", ms=11, color=BLUE, mec="white", mew=1.5, zorder=3)
ax.text(-0.05, left[g], g, ha="right", va="center", fontsize=11)
ax.text(1.05, right[g], f"{g} (median {med[g]:.3f})", ha="left", va="center",
fontsize=11)
ax.text(0, 2.45, "Abstract\n(order it implies)", ha="center", fontsize=10.5,
fontweight="bold", color=INK)
ax.text(1, 2.45, "Fig 4B\n(order of medians)", ha="center", fontsize=10.5,
fontweight="bold", color=INK)
ax.text(0.5, 1.5, "lines cross:\nthe order is reversed", ha="center", va="center",
fontsize=10, color=INK2, bbox=dict(fc=BG, ec="none", pad=2))
ax.text(-0.62, 2.0, "more\nconnected", ha="center", va="center", fontsize=10, color=INK2)
ax.text(-0.62, 1.0, "less\nconnected", ha="center", va="center", fontsize=10, color=INK2)
ax.set_xlim(-0.8, 1.75)
ax.set_ylim(0.6, 2.9)
ax.axis("off")
ax.plot([], [], "o", color=ORANGE, ms=10, label="Abstract's statement (ranks only)")
ax.plot([], [], "o", color=BLUE, ms=10, label="Read from the paper's Fig 4B")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, -0.12), ncol=2, frameon=False,
fontsize=10)
return fig
# ---------- candidate 5: three stacked quotes with direction tags + compact value strip
def cand5():
fig = plt.figure(figsize=(W, 5.0))
axt = fig.add_axes([0.0, 0.115, 0.62, 0.885])
axt.set_xlim(0, 1)
axt.set_ylim(0, 1)
axt.axis("off")
items = [("Abstract", SHORT_ABS, "▼ exposed less connected"),
("Results (Fig 4B)", SHORT_RES, "▲ exposed more connected"),
("Discussion", SHORT_DIS, "▲ exposed more connected")]
for i, (t, q, tag) in enumerate(items):
y0 = 0.68 - i * 0.335
axt.add_patch(FancyBboxPatch((0.01, y0), 0.97, 0.31, boxstyle="round,pad=0,"
"rounding_size=0.015", fc="white", ec=ORANGE, lw=2,
transform=axt.transAxes))
axt.text(0.04, y0 + 0.28, f"{t}: {tag}", fontsize=10.5, fontweight="bold",
va="top", transform=axt.transAxes)
axt.text(0.04, y0 + 0.205, wrap(q, 54), fontsize=10, va="top", color=INK2,
transform=axt.transAxes, linespacing=1.3)
ax = fig.add_axes([0.76, 0.20, 0.23, 0.70])
order = ["Exposed", "IntOccluded", "Occluded"]
for i, g in enumerate(order):
q1, m, q3 = FIG[g]
ax.plot([i, i], [q1, q3], color=BLUE, lw=2)
ax.plot(i, m, "o", ms=9, color=BLUE, mec="white", mew=1.5)
ax.text(i + 0.13, m, f"{m:.3f}", va="center", fontsize=10)
ax.set_xticks(range(3))
ax.set_xticklabels(["Exp.", "Int.\nOcc.", "Occ."])
ax.set_xlim(-0.5, 2.9)
ax.set_ylim(0, 0.25)
ax.set_ylabel("Eigen centrality, Fig 4B")
ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.set_title("Median and IQR", fontsize=10.5, color=INK, loc="left")
fig.text(0.01, 0.0, "Orange frames: text as printed in the paper. Blue: values read "
"from the paper's Fig 4B.\nHigher eigen centrality = more connected, as the "
"paper uses it.", fontsize=10, color=INK2, va="bottom")
return fig
save(cand1(), os.path.join(HERE, "fig1"), os.path.join(CAND, "cand1_text_panels"))
save(cand2(), os.path.join(CAND, "cand2_direction_ledger"))
save(cand3(), os.path.join(HERE, "fig2"), os.path.join(CAND, "cand3_median_dotplot"))
save(cand4(), os.path.join(CAND, "cand4_rank_slope"))
save(cand5(), os.path.join(CAND, "cand5_quotes_plus_values"))
print()
print("Wrote fig1.svg/png, fig2.svg/png and candidates/cand1..cand5 (.svg, .png)")
Abstract says LabMol-149 has EC50 below 500 nM; Table 3 gives 1450 and 509 nM
PLoS Comput Biol (2020) · doi:10.1371/journal.pcbi.1007025 · PMC7048302
Summary
The Abstract reports that two of the five compounds chosen by the paper's deep-learning screen, LabMol-149 and LabMol-152, showed antiplasmodial activity "at low nanomolar concentrations (EC50 <500 nM)". The final paragraph of the Discussion says both compounds "completely satisfy" a criterion of EC50 <500 nM against sensitive and resistant strains. The paper's own Table 3 gives LabMol-149 EC50 values of 1.450 ± 0.524 μM against the sensitive 3D7 strain and 0.509 ± 0.177 μM against the resistant W2 strain. These are 1450 nM and 509 nM, so the compound is above the stated threshold on both strains, by a factor of 2.9 on 3D7. Only LabMol-152 (49 and 78 nM) meets the criterion. The headline count of qualifying compounds is therefore one, not two.
What the paper reports
Abstract:
Among them, two compounds, LabMol-149 and LabMol-152, showed potent antiplasmodial activity at low nanomolar concentrations (EC50 <500 nM) and low cytotoxicity in mammalian cells. Therefore, the computational approach employing deep learning developed here allowed us to discover two new families of potential next generation antimalarial agents, which are in compliance with the guidelines and criteria for antimalarial target candidates.
Author summary:
This allowed us to discover two new potential antimalarial candidates.
Discussion, final paragraph:
Moreover, two of these compounds (LabMol-149 and LabMol-152) completely satisfy the guidelines and criteria for discovery of new antimalarial drugs, i.e., activity at low nanomolar concentrations (EC50 <500nM) against sensitive and multiple resistant strains of Plasmodium spp. and SI greater than 10 folds [12,42].
Table 3, header and LabMol-149 row (columns: compound | EC50 (Pf3D7)(μM) | EC50 (PfW2)(μM) | CC50 (COS-7) (μM) | SIa | SIb):
LabMol-149 | 1.450 ± 0.524 | 0.509 ± 0.177 | >100 | >71.4 | >196
Table 3, the other two active candidate compounds:
LabMol-151 | 1.911 ± 0.292 | 1.616 ± 0.321 | 140.8 ± 11.2 | 73.7 | 87.1
LabMol-152 | 0.049 ± 0.029 | 0.078 ± 0.014 | 16.7 ± 10.3 | 340.8 | 214.1
Table 3 footer:
SIa: Selectivity index calculated by COS7 CC50/3D7 EC50; and SIb: Selectivity index calculated by COS7 CC50/W2 EC50; The data are expressed as mean ± SD of three independent assays.
Two related passages use different wording. Results, "Experimental validation":
The EC50 for each compound (Table 3) indicate that two compounds, 2-(4,6-diphenyl-1,2-dihydro-1,3,5-triazin-2-yl)phenol (LabMol-149), 4-{N-[3-(morpholin-4-yl)-1,4-dioxo-1,4-dihydronaphthalen-2-yl]acetamido}benzoic acid (LabMol-151) and N2-(3-fluorophenyl)-N4-[(oxolan-2-yl)methyl]quinazoline-2,4-diamine (LabMol-152), were potent at inhibiting the parasite growth, showing activities in submicromolar and low nanomolar range against both 3D7 and W2 strains.
Discussion, third paragraph:
Two compounds (LabMol-149 and LabMol-151) showed activity at submicromolar concentrations against asexual blood stages and low cytotoxicity in mammalian cells.
What the paper's own numbers show
Method. I converted every Table 3 EC50 from μM (the unit in the column header) to nM and compared each one with the 500 nM threshold stated in the Abstract and the Discussion. The Discussion applies that threshold to both sensitive and resistant strains, so a compound qualifies only if both its 3D7 and W2 values are below 500 nM. For LabMol-149 I also looked at mean − 1 SD, using the footer's statement that the values are mean ± SD of three assays. I recomputed every selectivity index from the footer's definitions (CC50/EC50), and checked the "submicromolar" (below 1 μM) wording against the same values. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| LabMol-149 EC50, 3D7 | Table 3; Abstract and Discussion say <500 nM | 1.450 ± 0.524 μM | 1450 nM, 2.90 × the 500 nM threshold; mean − 1 SD = 926 nM |
| LabMol-149 EC50, W2 | Table 3; Abstract and Discussion say <500 nM | 0.509 ± 0.177 μM | 509 nM, 1.018 × the threshold (9 nM above it) |
| LabMol-152 EC50, 3D7 / W2 | Table 3 | 0.049 / 0.078 μM | 49 / 78 nM, both below 500 nM |
| LabMol-151 EC50, 3D7 / W2 | Table 3 | 1.911 / 1.616 μM | 1911 / 1616 nM, both above 500 nM |
| Tested LabMol compounds with EC50 <500 nM on both strains | Abstract; Discussion, final paragraph | two (LabMol-149 and LabMol-152) | one (LabMol-152) |
| LabMol-149 SIa | Table 3 | >71.4 | 100 / 1.450 = >69.0; 71.4 corresponds to an EC50 of 1.401 μM |
| LabMol-149 SIb | Table 3 | >196 | 100 / 0.509 = >196.5 |
| LabMol-151 SIa / SIb | Table 3 | 73.7 / 87.1 | 73.7 / 87.1 |
| LabMol-152 SIa / SIb | Table 3 | 340.8 / 214.1 | 340.8 / 214.1 |
| Chloroquine SIa / SIb | Table 3 | 4,545 / 289 | >4545.5 / >289.0 |
| Pyrimethamine SIa / SIb | Table 3 | 2,702 / 5.5 | >2702.7 / >5.5 |
| LabMol-149 "submicromolar" | Discussion, third paragraph | submicromolar | 3D7 1450 nM, not below 1 μM; W2 509 nM, below 1 μM |
| LabMol-151 "submicromolar" | Discussion, third paragraph | submicromolar | 1911 / 1616 nM, below 1 μM on neither strain |
The table's arithmetic is internally consistent. Every selectivity index recomputes from the printed EC50 and CC50 values, except LabMol-149's SIa, which is slightly high (71.4 against 69.0). The problem is the text. It counts LabMol-149 among the compounds below 500 nM, and the table does not support that for either strain. The Results passage also says "two compounds" and then names three.
Figure
Figure 1. On the left (orange), the two passages that state the EC50 <500 nM criterion and name LabMol-149 and LabMol-152 as meeting it, quoted from the Abstract and the final paragraph of the Discussion. On the right (blue), the Table 3 values for the same two compounds, converted from μM to nM. LabMol-149 is 1450 nM against 3D7 (2.90 times the threshold) and 509 nM against W2 (1.02 times). LabMol-152 is 49 and 78 nM. Only LabMol-152 meets the criterion as the text states it.
Figure 2. Table 3 EC50 values (blue dots, mean; blue lines, ± 1 SD of three assays) for the three active candidate compounds on one log-scaled nM axis. The solid vertical line marks 500 nM and the dashed line 1 μM. The orange markers show what the text says about each compound: below 500 nM for LabMol-149 and LabMol-152 (Abstract and final paragraph of the Discussion), and "submicromolar" for LabMol-151 (Discussion, third paragraph; that sentence also calls LabMol-149 submicromolar). LabMol-149's 3D7 interval lies entirely above 500 nM, and both LabMol-151 values lie above 1 μM. The W2 value for LabMol-149 sits at the threshold, and its ± 1 SD interval spans it.
Why it matters
The count of compounds meeting the potency criterion is the paper's headline experimental result. It appears in the Abstract ("two compounds ... (EC50 <500 nM)"), in the Author summary ("two new potential antimalarial candidates") and in the final paragraph of the Discussion, which says the two compounds "completely satisfy" the criteria for new antimalarial drugs. The Abstract also concludes that the approach found "two new families of potential next generation antimalarial agents, which are in compliance with the guidelines and criteria".
By the paper's own Table 3, one compound (LabMol-152, a quinazoline) meets the stated criterion. LabMol-149 fails it on the sensitive strain by a wide margin. Its 3D7 EC50 of 1450 nM is also above the 1 μM cut-off that the paper itself uses to label training compounds "active" (Methods, "Datasets": "1,058 active compounds with EC50 ≤ 1 μM"). On the resistant strain it is essentially at the threshold, at 509 nM. LabMol-149 remains an active and selective hit (SI above about 69 on 3D7 and 196 on W2), and nothing here questions the measurements themselves. But it is a low-micromolar hit, not a compound that satisfies the sub-500 nM criterion.
Readers who rely on the Abstract, for example to compile hit lists or to judge how well deep-learning virtual screening performs in antimalarial discovery, would count two qualifying compounds from two scaffold families where the data show one. The rest of the paper's conclusions, such as the QSAR models, the screen and LabMol-152's potency, do not depend on this. These are in vitro preclinical results, so there is no direct clinical consequence.
Explanations considered
- A μM/nM unit mix-up. This does not account for the numbers. The Table 3 header gives μM in the PMC text, the publisher's XML, its PDF and Europe PMC. Read in μM, LabMol-152's values (0.049 and 0.078) agree with the Results text ("EC50 = 0.049 μM and 0.078 μM"), and the reference drugs' values (chloroquine 0.011 μM, pyrimethamine 0.037 μM) are plausible in that unit. Read as nM, they would not be.
- A decimal slip in the 3D7 value (0.145 μM printed as 1.450 μM). This does not account for the numbers. With CC50 >100 μM, an EC50 of 0.145 μM would give SIa >689.7. The printed SIa (>71.4) corresponds to about 1.40 μM, so the table's own selectivity column supports a micromolar EC50. The W2 value (0.509 μM) also agrees with its SIb (100/196 = 0.510 μM).
- Assay variation. This does not account for the 3D7 value. The footer gives mean ± SD of three assays, and the 3D7 mean − 1 SD is still 926 nM, nearly twice the threshold. For W2 the mean is 9 nM above 500 nM, and its ± 1 SD interval (332–686 nM) spans the threshold, so W2 alone could be called borderline. The stated criterion requires activity on both strains, however, and 3D7 is not borderline.
- The claim refers to only one strain. This does not rescue it. The Discussion applies the criterion "against sensitive and multiple resistant strains". Even the more potent of LabMol-149's two values (509 nM, W2) is above 500 nM.
- A different, looser criterion was intended. The Results passage uses vaguer wording ("submicromolar and low nanomolar range"). The Abstract and the Discussion, however, state the 500 nM cut-off explicitly, citing references 12 and 42, and attribute it to LabMol-149. A looser 1 μM cut-off would still not be met on 3D7 (1450 nM).
- The compound names were swapped in the text. Substituting LabMol-151 for LabMol-149 does not help: LabMol-151 is 1911 and 1616 nM. LabMol-152 is the only tested compound below 500 nM on both strains.
- A conversion artifact in the PMC copy. This is excluded. The publisher's article page (Version 2, the final version), its JATS XML and its printable PDF carry the same text and the same Table 3 values as the PMC archive.
- The EC50 lies outside the assay range. This is not the case. The antiplasmodial assay used a 12-point series from 0.019 to 40 μM (Methods, "Antiplasmodial assay"), and 1.450 μM lies within it.
Status
- Version of record. The publisher's article page (journals.plos.org, reached from the DOI; shown as Version 2, with Version 1 as the uncorrected proof), its JATS XML and its printable PDF (21 pages, dated February 18, 2020) show the same Abstract sentence, the same Discussion sentences and the same Table 3 rows, with μM in the column header, as the PMC archive. The Europe PMC full-text XML matches them as well. Every quote above was confirmed verbatim against the PMC-archived text.
- Supplements. The article lists seven supporting files: S1–S4 File (XLSX QSAR training datasets) and S1–S3 Fig (TIF images). S1 File (1163 rows, SMILES and Outcome) and S3 File (1247 rows, SMILES and pEC50) were opened; both contain only literature training data, with no LabMol entries. None of the supporting files holds assay-level dose-response data for the tested compounds, so none of them can change the Table 3 EC50 values.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, expression of concern or retraction. PMC and Europe PMC list no notices. Crossref's only update record is the article's own DOI with type "new_version", dated 2020-02-28, which is PLOS's routine record for replacing the uncorrected proof with the final version; a Crossref relation search returned only the article's own components. The PLOS comments page shows "Reader Comments (0)". The PubPeer API returned no feedback for the DOI (the PubPeer web page itself refused automated access), and a web search found no discussion of the issue.
- Sources. PMC article PMC7048302; publisher page https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1007025 and its comments page; PLOS JATS XML and printable PDF; Europe PMC full text; Crossref; PubPeer.
Suggested correction
- In the Abstract, change "two compounds, LabMol-149 and LabMol-152, showed potent antiplasmodial activity at low nanomolar concentrations (EC50 <500 nM)" so that it names LabMol-152 alone as meeting EC50 <500 nM. Describe LabMol-149 as active in the low-micromolar to high-nanomolar range (1.450 μM on 3D7, 0.509 μM on W2). Revise "two new families ... in compliance with the guidelines and criteria" and the Author summary's "two new potential antimalarial candidates" to match.
- In the final paragraph of the Discussion, restrict "completely satisfy the guidelines and criteria ... (EC50 <500nM)" to LabMol-152.
- In the third paragraph of the Discussion, replace "submicromolar" for LabMol-149 and LabMol-151, which are 1.450 / 0.509 μM and 1.911 / 1.616 μM, with "low micromolar" or the actual values.
- Minor: in the Results, "two compounds" is followed by three names; change it to "three compounds". Change LabMol-149's SIa from >71.4 to >69.0 (100/1.450), or state which EC50 value was used to compute it.
Recomputation script (python recompute.py.txt)
"""Recompute the potency claims of PMC7048302
(PLoS Comput Biol 2020, doi:10.1371/journal.pcbi.1007025) from the paper's own Table 3.
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from Table 3 of the paper as archived in PMC (identical in the
publisher's version of record). Table 3 header: EC50 and CC50 in uM; footer: "The data
are expressed as mean +/- SD of three independent assays"; SIa = CC50/3D7 EC50,
SIb = CC50/W2 EC50.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (Table 3, uM)
# cc_censored: CC50 printed as ">x", so the SI is a lower bound
T3 = {
"LabMol-149": dict(e3=1.450, sd3=0.524, ew=0.509, sdw=0.177, cc=100.0, cc_censored=True, SIa=71.4, SIb=196.0),
"LabMol-151": dict(e3=1.911, sd3=0.292, ew=1.616, sdw=0.321, cc=140.8, cc_censored=False, SIa=73.7, SIb=87.1),
"LabMol-152": dict(e3=0.049, sd3=0.029, ew=0.078, sdw=0.014, cc=16.7, cc_censored=False, SIa=340.8, SIb=214.1),
"Chloroquine": dict(e3=0.011, sd3=0.001, ew=0.173, sdw=0.020, cc=50.0, cc_censored=True, SIa=4545, SIb=289),
"Pyrimethamine": dict(e3=0.037, sd3=0.007, ew=18.240, sdw=4.537, cc=100.0, cc_censored=True, SIa=2702, SIb=5.5),
}
# LabMol-148 and LabMol-150: EC50 > 40 uM on both strains (inactive at the highest concentration)
THRESH_NM = 500.0 # Abstract and Discussion (final paragraph): "EC50 <500 nM"
SUBMICRO_NM = 1000.0 # "submicromolar" = below 1 uM
MODEL_ACTIVE_UM = 1.0 # Methods, Datasets: training compounds "active" if EC50 <= 1 uM
ASSAY_RANGE_UM = (0.019, 40.0) # Methods, Antiplasmodial assay: 12-point dilution series
# What the text says about each compound
CLAIM_LT500 = {"LabMol-149", "LabMol-152"} # Abstract; Discussion final paragraph
CLAIM_SUBMICRO = {"LabMol-149", "LabMol-151"} # Discussion, third paragraph
def nm(um):
return um * 1000.0
print("=== 1. Table 3 EC50 values in nM, against the stated <500 nM criterion ===")
for k, v in T3.items():
n3, nw = nm(v["e3"]), nm(v["ew"])
print(f"{k:14s} 3D7 {n3:7.0f} nM ({n3 / THRESH_NM:6.3f} x 500) W2 {nw:7.0f} nM ({nw / THRESH_NM:6.3f} x 500) "
f"<500 nM on both strains: {n3 < THRESH_NM and nw < THRESH_NM}")
print("\n=== 2. LabMol-149 in detail ===")
v = T3["LabMol-149"]
for strain, m, s in (("3D7", v["e3"], v["sd3"]), ("W2", v["ew"], v["sdw"])):
print(f"{strain}: mean {nm(m):.0f} nM, SD {nm(s):.0f} nM; mean - 1 SD = {nm(m - s):.0f} nM, "
f"mean + 1 SD = {nm(m + s):.0f} nM; mean exceeds 500 nM by {nm(m) - THRESH_NM:.0f} nM")
print(f"3D7 mean - 1 SD above 500 nM: {nm(v['e3'] - v['sd3']) > THRESH_NM}; "
f"above 1 uM (the paper's own model 'active' cut-off, EC50 <= {MODEL_ACTIVE_UM} uM): {v['e3'] > MODEL_ACTIVE_UM}")
print(f"3D7 EC50 {v['e3']} uM lies inside the tested range {ASSAY_RANGE_UM[0]}-{ASSAY_RANGE_UM[1]} uM: "
f"{ASSAY_RANGE_UM[0] < v['e3'] < ASSAY_RANGE_UM[1]}")
print("\n=== 3. Selectivity indices recomputed (CC50 / EC50) ===")
for k, val in T3.items():
sia, sib = val["cc"] / val["e3"], val["cc"] / val["ew"]
pre = ">" if val["cc_censored"] else ""
print(f"{k:14s} SIa reported {pre}{val['SIa']:<7} recomputed {pre}{sia:.1f} "
f"SIb reported {pre}{val['SIb']:<6} recomputed {pre}{sib:.1f}")
print(f"LabMol-149: 3D7 EC50 implied by SIa = 71.4 with CC50 = 100 uM: {100 / 71.4:.3f} uM")
print(f"LabMol-149: SIa if the 3D7 EC50 were a decimal slip for 0.145 uM: {100 / 0.145:.1f}")
print(f"LabMol-149: W2 EC50 implied by SIb = 196 with CC50 = 100 uM: {100 / 196:.3f} uM (Table 3: 0.509 uM)")
print("\n=== 4. Text descriptions checked against Table 3 ===")
for k in ("LabMol-149", "LabMol-151", "LabMol-152"):
val = T3[k]
n3, nw = nm(val["e3"]), nm(val["ew"])
print(f"{k}: called '<500 nM' in Abstract/Discussion: {k in CLAIM_LT500}; actually <500 nM on both: "
f"{n3 < THRESH_NM and nw < THRESH_NM} | called 'submicromolar' in Discussion: {k in CLAIM_SUBMICRO}; "
f"actually <1 uM on 3D7: {n3 < SUBMICRO_NM}, on W2: {nw < SUBMICRO_NM}")
n_meet = sum(nm(T3[k]["e3"]) < THRESH_NM and nm(T3[k]["ew"]) < THRESH_NM for k in ("LabMol-149", "LabMol-151", "LabMol-152"))
print(f"Tested LabMol compounds meeting EC50 <500 nM on both strains: {n_meet} (text claims 2)")
print("Results text: 'two compounds' followed by three names (LabMol-149, LabMol-151, LabMol-152).")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# Figure 1: text-vs-table diagram
v = T3["LabMol-149"]
w = T3["LabMol-152"]
fig = plt.figure(figsize=(7.2, 4.8))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.8)
ax.axis("off")
def panel(x, y, wd, ht, edge):
ax.add_patch(FancyBboxPatch((x, y), wd, ht, boxstyle="round,pad=0,rounding_size=0.08",
fc="#ffffff", ec=edge, lw=1.5))
# left panel: what the text says (reported, orange)
panel(0.15, 0.95, 3.15, 3.7, ORANGE)
ax.text(0.30, 4.50, "Abstract and Discussion", fontsize=10.5, fontweight="bold", va="top")
ax.text(0.30, 4.20, "Reported in the text", fontsize=10, color=SEC, va="top")
ax.add_line(Line2D([0.30, 0.30], [3.84, 2.62], color=ORANGE, lw=2))
ax.text(0.42, 3.84, "Abstract:\n\u201ctwo compounds, LabMol-149 and\nLabMol-152, showed potent\n"
"antiplasmodial activity at low\nnanomolar concentrations\n(EC50 <500 nM)\u201d",
fontsize=10, va="top", linespacing=1.25)
ax.add_line(Line2D([0.30, 0.30], [2.30, 1.08], color=ORANGE, lw=2))
ax.text(0.42, 2.30, "Discussion, final paragraph:\n\u201c(LabMol-149 and LabMol-152)\ncompletely satisfy [\u2026] (EC50\n"
"<500nM) against sensitive and\nmultiple resistant strains\u201d", fontsize=10, va="top", linespacing=1.25)
# right panel: Table 3 converted (recomputed, blue)
panel(3.95, 0.95, 3.1, 3.7, BLUE)
ax.text(4.10, 4.50, "Table 3 (\u03bcM), converted to nM", fontsize=10.5, fontweight="bold", va="top")
ax.text(4.10, 4.20, "Recomputed from the paper\u2019s table", fontsize=10, color=SEC, va="top")
rows = [
("LabMol-149", "3D7", v["e3"], v["sd3"]),
("LabMol-149", "W2", v["ew"], v["sdw"]),
("LabMol-152", "3D7", w["e3"], w["sd3"]),
("LabMol-152", "W2", w["ew"], w["sdw"]),
]
for (cpd, strain, m, s), yy in zip(rows, (3.74, 3.08, 2.22, 1.56)):
n = nm(m)
ok = n < THRESH_NM
ax.text(4.10, yy, f"{cpd}, {strain}: {m:.3f} \u00b1 {s:.3f} \u03bcM", fontsize=10, va="center",
fontweight="normal" if ok else "bold")
ax.text(4.10, yy - 0.25, f"= {n:.0f} nM", fontsize=10, va="center", color=BLUE, fontweight="bold")
verdict = f"{n / THRESH_NM:.2f} \u00d7 500 nM: fails" if not ok else "below 500 nM: meets"
ax.text(5.15, yy - 0.25, verdict, fontsize=10, va="center", color=TXT if not ok else SEC)
ax.add_line(Line2D([4.10, 6.90], [2.52, 2.52], color=GRID, lw=1))
# connector
ax.add_patch(FancyArrowPatch((3.33, 3.25), (3.92, 3.25), arrowstyle="<|-|>", mutation_scale=14, color=SEC, lw=2))
ax.text(3.625, 3.40, "conflict", fontsize=10, ha="center", va="bottom", color=SEC, fontweight="bold")
ax.text(3.6, 0.48, f"LabMol-149 is {nm(v['e3']):.0f} nM (3D7) and {nm(v['ew']):.0f} nM (W2): above 500 nM on both strains.\n"
f"Only LabMol-152 ({nm(w['e3']):.0f} and {nm(w['ew']):.0f} nM) meets the criterion the text states.",
fontsize=10, ha="center", va="center", linespacing=1.3)
save(fig, "fig1")
# Figure 2: EC50 mean +/- SD (nM, log scale) against the text's thresholds
cpds = ["LabMol-149", "LabMol-151", "LabMol-152"]
claims = {"LabMol-149": "Abstract/Discussion: <500 nM", "LabMol-151": "Discussion: “submicromolar”",
"LabMol-152": "Abstract/Discussion: <500 nM"}
fig, ax = plt.subplots(figsize=(7.2, 4.0))
fig.subplots_adjust(left=0.20, right=0.97, top=0.84, bottom=0.15)
ypos, ylabels = [], []
y = 0
for c in cpds:
val = T3[c]
limit = THRESH_NM if c in CLAIM_LT500 else SUBMICRO_NM
# claim: orange bracket from the claimed upper limit leftwards
ax.annotate("", xy=(limit * 0.55, y + 0.5), xytext=(limit, y + 0.5),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=12))
ax.plot([limit, limit], [y + 0.32, y + 0.68], color=ORANGE, lw=2)
ax.text(limit * 1.06, y + 0.5, claims[c], color=TXT, fontsize=10, va="center", ha="left")
for j, (strain, m, s) in enumerate((("3D7", val["e3"], val["sd3"]), ("W2", val["ew"], val["sdw"]))):
yy = y - 0.18 - 0.42 * j
ax.plot([nm(m - s), nm(m + s)], [yy, yy], color=BLUE, lw=2, solid_capstyle="butt")
ax.plot(nm(m), yy, "o", color=BLUE, ms=8, mec=BG, mew=1.5, zorder=3)
ax.text(nm(m + s) * 1.08, yy, f"{strain}: {nm(m):.0f} nM", fontsize=10, va="center", color=TXT)
ypos.append(yy)
ylabels.append(f"{c} {strain}")
y -= 1.75
ax.axvline(THRESH_NM, color=SEC, lw=0.9, zorder=0)
ax.axvline(SUBMICRO_NM, color=SEC, lw=0.9, ls=(0, (3, 3)), zorder=0)
ax.text(THRESH_NM * 0.96, 1.12, "500 nM", ha="right", va="bottom", fontsize=10, color=SEC)
ax.text(SUBMICRO_NM * 1.04, 1.12, "1 μM", ha="left", va="bottom", fontsize=10, color=SEC)
ax.set_xscale("log")
ax.set_xlim(10, 20000)
ax.set_ylim(y + 0.85, 1.05)
ax.set_yticks(ypos)
ax.set_yticklabels(ylabels, fontsize=10)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("EC50 (nM, log scale)")
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda x, _: f"{x:,.0f}"))
handles = [Line2D([], [], color=BLUE, lw=2, marker="o", ms=8, mec=BG,
label="Table 3 mean ± SD (n = 3), converted to nM"),
Line2D([], [], color=ORANGE, lw=2, marker=4, ms=8,
label="Text\u2019s claim: EC50 below this value")]
fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.995), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nWrote fig1.svg/png and fig2.svg/png to", HERE)
Abstract names March and October as best twice-yearly spraying months; body, table and data give September–November
PLoS Comput Biol (2020) · doi:10.1371/journal.pcbi.1007743 · PMC7200023
Summary
The abstract says the best ultra-low volume (ULV) strategy was spraying twice a year "in March and October", with a median of 172,000 infections [IQR 158,000–183,000], a 52% reduction from no spraying. The Results text, Table 2, the Discussion and the legends of Figs 4, 8 and 10 give these same figures, and the same 549 spray days, for a different month pair: September and November. One Results sentence repeats the abstract's "October and March". The authors' public source data reproduce the reported 172,000 [158,000–183,000] for September–November, which ranks 1st of the 66 month pairs tested. March–October has a median of 219,402 infections, ranks 16th, and gives a 39% reduction, not 52%. The paper's stated question is when to spray, so the abstract gives the wrong answer to it, although every number it reports is correct.
What the paper reports
The abstract's ULV headline:
"The ULV strategy with the fewest median infections was spraying twice yearly, in March and October, which led to a median of 172,000 infections [IQR: 158,000–183,000], a 52% reduction from baseline." (Abstract)
The abstract also states the paper's question:
"The optimal time to deploy these interventions and their relative epidemiological effects are, however, not well understood." (Abstract)
The Results give the same numbers to a different pair, and describe the general timing pattern:
"The best strategy for spraying twice yearly was spraying in September and November (172,000; IQR: 158,000–183,000) (Fig 4C). Generally, undertaking the first spray in August or September (typically, just before the dengue season) and the second in October or November (typically, near the start of the dengue season) led to the fewest infections (Fig 4C). In this case, the strategy that led to the fewest infections also led to the lowest average mosquito abundance: spraying in September and November (S4 Fig)." (Results, "Ultra-low volume spraying", paragraph 1)
The next paragraph uses the abstract's pair:
"Comparing the best strategies, spraying twice yearly (in October and March) averted the most infections, but required the most spraying campaigns: 22 campaigns of spraying, spending a median of 549 days spraying in total over 11 years." (Results, "Ultra-low volume spraying", paragraph 2)
Table 2 (ULV rows, columns: None | Adaptive threshold | Yearly | Twice yearly):
"Best strategy | N/A | When monthly incidence is 1σ above mean | September | September & November" "Number of infections, 2000–2010 (1,000s) [IQR] | 361 [347, 383] | 254 [210, 277] | 261 [250, 277] | 172 [158, 183]" "Days spent spraying [IQR] | 0 | 351 [341, 371] | 274 [272, 277] | 549 [545, 552]" (Table 2)
The Discussion and three figure legends agree with Table 2:
"The best strategy for ULV spraying was to spray twice per year, in September and November." (Discussion, "Summary of main findings")
"twice yearly corresponds to spraying in September and November." (Fig 4 legend, panel d; Fig 8 legend, panel c has "and twice yearly corresponds to spraying in September and November;")
"the bottom row the best twice yearly strategy (spraying in September and November)" (Fig 10 legend)
The Methods show that the two pairs are separate strategies, each with its own result:
"We tested yearly spraying starting in each month (12 strategies) and twice-yearly spraying in each pair of months (66 strategies) (Table 1)." (Methods, "Experiments")
"Comparison of the median predicted infections for twice yearly spraying strategies, beginning on the first days of the shown month." (Fig 4 legend, panel c)
What the paper's own numbers show
Method. I first listed every statement in the paper that names the best twice-yearly ULV pair, and checked each against the PMC archive. Seven statements name September and November (Results paragraph 1, twice; Table 2; the Discussion; the Fig 4, 8 and 10 legends). Two name March and October (the Abstract and Results paragraph 2). I then checked the arithmetic of the reported values. Finally, I used the source data from the GitHub repository given in the paper's Data Availability Statement. The file figures/scripts/spray_analysis_paired_all_years.RData holds 400 simulations × 3 outputs × 66 month pairs. I mapped pairs to columns with the pair-index formula from the authors' own Fig 4 script, and took infections from output column 3, the column that script uses for Fig 4. For each pair I computed the median, the IQR (R's default quantile, which numpy also uses) and the median number of spray days. These data reproduce the paper's no-spraying, yearly-September and best-threshold results, so they match the published analysis. All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Month pair with the fewest median infections (twice yearly, ULV) | Abstract; Results para 2 | March and October | September–November (rank 1 of 66) |
| Same | Results para 1; Table 2; Discussion; Fig 4d, 8c, 10 legends | September and November | September–November (rank 1 of 66) |
| Median infections [IQR], September–November | Results para 1; Table 2 | 172,000 [158,000–183,000] | 171,650 [158,131–183,442] (consistent) |
| Median infections [IQR], March–October | Abstract (as the best pair) | 172,000 [158,000–183,000] | 219,402 [208,280–230,152]; rank 16 of 66 |
| Reduction from no spraying, best twice-yearly pair | Abstract | 52% | 1 − 172/361 = 52.4%; from source data, Sep–Nov 52.4%, Mar–Oct 39.2% |
| No spraying, median [IQR] | Abstract; Table 2 | 361,000 [347,000–383,000] | 360,831 [346,748–383,264] (consistent) |
| Yearly September, median [IQR]; days | Results; Table 2 | 261,000 [250,000–277,000]; 274 | 260,708 [249,589–276,903]; 274 (consistent) |
| Spray days, twice yearly | Results para 2; Table 2 | 549 | 22 campaigns × 25 days = 550; median 549 for both Sep–Nov and Mar–Oct (median across all 66 pairs is 548–549, so this figure does not identify a pair) |
| Best ULV vs best TIRS | Results, TIRS | 17-fold | 172,000 / 9,900 = 17.4 (consistent) |
| Ranks of the four pairs with first spray Aug/Sep and second Oct/Nov | Results para 1 ("Generally, ...") | fewest infections | ranks 1–4 of 66 (consistent with the body) |
Two further checks need the RData file. Mar–Oct is not the best pair on the file's other outcome column either (rank 14 of 66, best is Sep–Nov), and not on mean infections (rank 15 of 66, best is Sep–Nov).
A secondary inconsistency, unrelated to the month pair. For the best TIRS strategy, the abstract gives "(9,900; [IQR: 8,720–11,400], a 94% reduction)". The Results give "9,900 infections (IQR: 8,720–11,300)", Table 2 gives "9.90 [8.72, 11.3]", and 1 − 9.9/361 = 97.3%. The Discussion's "97% reduction ... for repeated TIRS campaigns" is consistent with 97.3% (yearly TIRS gives 1 − 11.8/361 = 96.7%, also about 97%). The abstract's 94% and 11,400 do not match the body.
Figure
Figure 1. The two orange-bordered panels quote the paper. On the left, the abstract (and Results paragraph 2) name March and October as the best twice-yearly ULV pair. On the right, the Results, Table 2 and the Discussion (and the Fig 4d, 8c and 10 legends) name September and November. Both give the same 172,000 [158,000–183,000] infections and 549 spray days. The blue strip below gives values recomputed in recompute.py.txt from the authors' source data. September–November has median 171,650 [158,131–183,442], rank 1 of 66, a 52% reduction, and matches the reported result. March–October has median 219,402 [208,280–230,152], rank 16 of 66, a 39% reduction, and does not match.
Figure 2. Median and IQR of total infections over 2000–2010 (400 simulations each) for the 20 best of the 66 twice-yearly ULV month pairs, recomputed from the authors' source data. The top row, in orange, is the paper's reported best result, 172 [158, 183] thousand. The two pairs the paper names are in blue. September–November, named in Table 2 and most of the body, sits at the reported value. March–October, named in the abstract, ranks 16th, with an IQR that does not overlap the reported one. The thin vertical line marks the no-spraying median (361,000). The axis starts at zero.
Why it matters
The month pair is the result the paper sets out to find. The abstract describes timing as "not well understood", and the title is about optimizing deployment. A reader of the abstract alone would plan the first of two yearly ULV campaigns for March. The body puts the best first spray in August or September, "just before the dengue season", and the Author summary advises starting non-residual spraying "close to the start of the dengue transmission season". On the authors' own data, March–October gives a median of about 47,800 more infections over 11 years than September–November (219,402 vs 171,650). That is a 39% reduction instead of 52%, and fifteen pairs do better.
The error does not change the infection counts, the 52% figure (which belongs to September–November), the ranking of twice-yearly over yearly and threshold ULV strategies, the advantage of TIRS, or the recommendation of threshold-based TIRS. The paper's conclusions in the body stand. The problem is that the abstract, the part most often read, indexed and cited, gives a timing the paper's own analysis does not support. Results paragraph 2 repeats it, so it also appears in the main text.
Explanations considered
Typesetting or archiving artifact. Ruled out. The conflicting items are plain month names, not symbols, decimals or table alignment. An earlier audit stage found the same wording in the PLOS XML, the PLOS PDF and the PLOS article page, matching the PMC copy word for word. On 2026-10-23 I fetched the PLOS article page and confirmed that the abstract still reads "in March and October".
March–October and September–November as the same strategy under a different calendar. Ruled out. The Methods treat twice-yearly spraying as 66 unordered month pairs, each starting "on the first days of the shown month" every calendar year. March–October and September–November are separate strategies with separate results. In the source data they have different distributions (medians 219,402 vs 171,650, non-overlapping IQRs).
The abstract refers to a different outcome or summary statistic. Not supported. The abstract says "fewest median infections" and gives the same 172,000 [158,000–183,000] as Table 2. In the source data, March–October is not the best pair on the other output column (rank 14) or on mean infections (rank 15). The Results also say September–November gave the lowest mosquito abundance (S4 Fig).
The 549 spray days as evidence for March–October. Not informative. Every twice-yearly pair has 22 campaigns of 25 days. The median number of spray days is 548–549 for all 66 pairs, so "549 days" in Results paragraph 2 fits either wording.
Two statements against seven. The abstract is not the only place with March and October: Results paragraph 2 uses it too. But the paper reports one best twice-yearly strategy, attaches one set of numbers to it, and those numbers belong to September–November in both the paper's table and the source data. The paper does not explain where the March/October wording came from, and nothing here depends on how it arose.
Status
As of 2026-10-23:
- Version of record. The PLOS article page (journals.plos.org, reached via doi.org) was fetched on 2026-10-23. Its abstract still says "spraying twice yearly, in March and October", and no correction notice was found. An earlier audit stage compared the PLOS XML, the PLOS PDF (28 pages, abstract on p. 1) and the Europe PMC full text with the PMC archive and found them identical, including Results paragraph 2's "in October and March", Table 2 and the figure legends.
- Supplements and source data. The Data Availability Statement points to a public GitHub repository. Its
figures/scripts/spray_analysis_paired_all_years.RDataand the R script that draws Fig 4 were used above. The script picks the best strategy per category as the minimum median in output column 3 (max.col(-x)), which gives September–November. The Fig 5c panel code draws campaign start lines at September and November. The supporting-information figures (e.g. S4 Fig) were not re-read for this write-up. - Corrections. Crossref lists one update for the DOI, a "new version" record dated 2020-05-05 that points to the article's own DOI and is not a correction notice. PMC and Europe PMC list no correction, retraction or expression of concern (
src/known.py). - PubPeer and the wider web. An earlier audit stage's web searches for the DOI or title with "correction", "erratum" or "PubPeer" found no relevant notice or thread. The PubPeer API returned HTTP 403, and the PLOS reader-comments page showed 0 comments. I could not repeat the web search on 2026-10-23 because the session's search quota was used up. The medRxiv preprint (10.1101/19007971) returned HTTP 403, so it is unknown whether the preprint has the same wording. The published peer-review history was not examined.
Sources: PMC7200023 (PubMed Central archive); https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1007743; Crossref and PMC notice records for 10.1371/journal.pcbi.1007743; source data from the repository named in the paper's Data Availability Statement.
Suggested correction
Change the abstract sentence to: "The ULV strategy with the fewest median infections was spraying twice yearly, in September and November, which led to a median of 172,000 infections [IQR: 158,000–183,000], a 52% reduction from baseline." In Results ("Ultra-low volume spraying", paragraph 2), change "spraying twice yearly (in October and March)" to "spraying twice yearly (in September and November)". In the same notice, the authors could also reconcile the abstract's TIRS figures ("[IQR: 8,720–11,400], a 94% reduction") with the Results, Table 2 and the Discussion (IQR upper bound 11,300; 1 − 9.9/361 = 97%), or explain the basis of the 94%.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC7200023 (PLoS Comput Biol 2020, doi:10.1371/journal.pcbi.1007743)
# Abstract vs body: which month pair is the best twice-yearly ULV spraying strategy?
#
# Usage: python recompute.py.txt [path/to/spray_analysis_paired_all_years.RData]
# Needs: numpy, matplotlib. Optional: the 'rdata' package (pip install rdata) plus the
# source-data file figures/scripts/spray_analysis_paired_all_years.RData from the
# GitHub repository named in the paper's Data Availability Statement.
# Output: prints every recomputed number used in writeup.md; writes fig1.svg/png and
# fig2.svg/png next to this script.
#
# Sections
# 1. Every statement in the paper that names the best twice-yearly ULV month pair
# (checked verbatim against the PMC archive when src/show.py is available).
# 2. Arithmetic from the paper's own reported numbers.
# 3. Per-pair results from the authors' source data. PAIR_STATS below holds, for each of
# the 66 month pairs, the median, 25th and 75th percentile of total infections over
# 400 simulations (output column 3, as in the authors' Fig 4 script) and the median
# number of spray days (output column 1). The table was produced by this script's
# from_rdata() function; when the RData path is given, it is recomputed and checked.
# 4. Figures.
import os, sys, subprocess
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
MON = "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec".split()
# ---------------------------------------------------------------- 1. statements in the paper
STATEMENTS = [ # (location, verbatim text, month pair named)
("Abstract",
"The ULV strategy with the fewest median infections was spraying twice yearly, in March and October, "
"which led to a median of 172,000 infections [IQR: 158,000–183,000], a 52% reduction from baseline.",
"Mar-Oct"),
("Results, ULV para 1",
"The best strategy for spraying twice yearly was spraying in September and November (172,000; "
"IQR: 158,000–183,000) (Fig 4C).", "Sep-Nov"),
("Results, ULV para 1",
"Generally, undertaking the first spray in August or September (typically, just before the dengue "
"season) and the second in October or November (typically, near the start of the dengue season) "
"led to the fewest infections (Fig 4C).", "Aug/Sep then Oct/Nov"),
("Results, ULV para 2",
"Comparing the best strategies, spraying twice yearly (in October and March) averted the most "
"infections, but required the most spraying campaigns: 22 campaigns of spraying, spending a "
"median of 549 days spraying in total over 11 years.", "Mar-Oct"),
("Table 2, ULV best strategy row",
"ULV | Best strategy | N/A | When monthly incidence is 1σ above mean | September | September & November",
"Sep-Nov"),
("Table 2, ULV infections row",
"ULV | Number of infections, 2000–2010 (1,000s) [IQR] | 361 [347, 383] | 254 [210, 277] | "
"261 [250, 277] | 172 [158, 183]", "-"),
("Table 2, ULV days row",
"ULV | Days spent spraying [IQR] | 0 | 351 [341, 371] | 274 [272, 277] | 549 [545, 552]", "-"),
("Discussion, Summary of main findings",
"The best strategy for ULV spraying was to spray twice per year, in September and November.", "Sep-Nov"),
("Fig 4 legend, panel (d)",
"twice yearly corresponds to spraying in September and November.", "Sep-Nov"),
("Fig 8 legend, panel (c)",
"and twice yearly corresponds to spraying in September and November;", "Sep-Nov"),
("Fig 10 legend",
"the bottom row the best twice yearly strategy (spraying in September and November)", "Sep-Nov"),
("Methods, Experiments",
"We tested yearly spraying starting in each month (12 strategies) and twice-yearly spraying in "
"each pair of months (66 strategies) (Table 1).", "-"),
("Author summary",
"For non-residual insecticides, we find that it is best to begin spraying close to the start of "
"the dengue transmission season, as mosquito populations can rebound quickly and resume previous "
"levels of transmission.", "-"),
("Abstract (TIRS)",
"(9,900; [IQR: 8,720–11,400], a 94% reduction)", "-"),
("Results, TIRS para 1",
"resulting in 9,900 infections (IQR: 8,720–11,300)", "-"),
("Results, TIRS para 1",
"The result was many fewer infections (17-fold) than the best ULV strategy, which led to a median "
"of 172,000 infections.", "-"),
("Abstract (aim)",
"The optimal time to deploy these interventions and their relative epidemiological effects are, "
"however, not well understood.", "-"),
("Results, ULV para 1",
"In this case, the strategy that led to the fewest infections also led to the lowest average "
"mosquito abundance: spraying in September and November (S4 Fig).", "Sep-Nov"),
("Fig 4 legend, panel (c)",
"Comparison of the median predicted infections for twice yearly spraying strategies, beginning on "
"the first days of the shown month.", "-"),
("Discussion, Comparison to other studies",
"Our prediction of a 97% reduction in the number of infections for repeated TIRS campaigns", "-"),
]
def check_quotes():
show = os.path.join("/home/ubuntu/papers", "src", "show.py")
py = "/home/ubuntu/papers-data/venv/bin/python"
print("1) Statements in the paper (verbatim check against the PMC archive)")
full = None
if os.path.exists(show) and os.path.exists(py):
r = subprocess.run([py, show, "PMC7200023", "--full"], cwd="/home/ubuntu/papers",
capture_output=True, text=True)
full = r.stdout if r.returncode == 0 else None
for loc, q, pair in STATEMENTS:
tag = "not checked (archive tool unavailable)" if full is None else \
("found verbatim" if q in full else "NOT FOUND")
print(" %-40s pair named: %-22s %s" % (loc, pair, tag))
if full is not None:
assert all(q in full for _, q, _ in STATEMENTS), "a quote was not found verbatim"
# ---------------------------------------------------------------- 2. paper's own arithmetic
REP = dict(baseline=361000, ulv2=172000, ulv2_lo=158000, ulv2_hi=183000, ulv2_days=549,
tirs=9900, tirs_lo_abs=8720, tirs_hi_abs=11400, tirs_hi_res=11300,
tirs_yearly=11800, campaigns=22, days_per_campaign=25, years=11)
def arithmetic():
b = REP["baseline"]
out = {}
out["red_ulv2"] = 100 * (1 - REP["ulv2"] / b)
out["red_tirs"] = 100 * (1 - REP["tirs"] / b)
out["red_tirs_yearly"] = 100 * (1 - REP["tirs_yearly"] / b)
out["fold"] = REP["ulv2"] / REP["tirs"]
out["days"] = REP["campaigns"] * REP["days_per_campaign"]
print("\n2) Arithmetic from the paper's reported numbers")
print(" ULV twice-yearly reduction vs no spraying: 1 - 172/361 = %.1f%% (abstract: 52%%)" % out["red_ulv2"])
print(" 22 campaigns = 2 per year x %d years: %d" % (REP["years"], 2 * REP["years"]))
print(" 22 campaigns x 25 days = %d days (Table 2: median 549 [545, 552])" % out["days"])
print(" Best ULV / best TIRS: 172,000 / 9,900 = %.1f-fold (Results: 17-fold)" % out["fold"])
print(" TIRS adaptive threshold reduction: 1 - 9.9/361 = %.1f%% (abstract: 94%%; Discussion: 97%%)" % out["red_tirs"])
print(" TIRS yearly (Sep) reduction: 1 - 11.8/361 = %.1f%%" % out["red_tirs_yearly"])
print(" TIRS IQR upper bound: abstract 11,400 vs Results 11,300 and Table 2 11.3 (thousand)")
return out
# ---------------------------------------------------------------- 3. authors' source data
# (i, j) 1-based months, i < j : (median, q25, q75 infections; median spray days)
PAIR_STATS = {
(1, 2): (441708.00, 422530.50, 454094.25, 549.0), # Jan-Feb
(1, 3): (432794.00, 416724.00, 445201.75, 549.0), # Jan-Mar
(1, 4): (424317.50, 406502.25, 434176.75, 548.0), # Jan-Apr
(1, 5): (402984.50, 388331.75, 414365.25, 549.0), # Jan-May
(1, 6): (386051.00, 375338.75, 395504.25, 549.0), # Jan-Jun
(1, 7): (359459.50, 348058.50, 369248.25, 549.0), # Jan-Jul
(1, 8): (312679.00, 301817.75, 327553.00, 549.0), # Jan-Aug
(1, 9): (272079.00, 260225.00, 285449.25, 549.0), # Jan-Sep
(1, 10): (244997.50, 234358.25, 251953.75, 549.0), # Jan-Oct
(1, 11): (277347.00, 261847.25, 290349.25, 549.0), # Jan-Nov
(1, 12): (385329.00, 367194.00, 399237.00, 549.0), # Jan-Dec
(2, 3): (430668.50, 406035.50, 449033.00, 548.0), # Feb-Mar
(2, 4): (421545.00, 397439.75, 439812.00, 548.0), # Feb-Apr
(2, 5): (408950.50, 389504.75, 421645.25, 549.0), # Feb-May
(2, 6): (390143.00, 376047.50, 402808.00, 549.0), # Feb-Jun
(2, 7): (363220.50, 350213.75, 374916.50, 549.0), # Feb-Jul
(2, 8): (308439.00, 292948.00, 321977.75, 549.0), # Feb-Aug
(2, 9): (259513.00, 244273.75, 272738.75, 549.0), # Feb-Sep
(2, 10): (234932.00, 227319.50, 245102.25, 549.0), # Feb-Oct
(2, 11): (273375.50, 261919.75, 285630.00, 548.0), # Feb-Nov
(2, 12): (384562.50, 366859.00, 396050.25, 548.0), # Feb-Dec
(3, 4): (410443.00, 380868.50, 429001.00, 549.0), # Mar-Apr
(3, 5): (397934.50, 377044.50, 414733.25, 549.0), # Mar-May
(3, 6): (386533.00, 371343.00, 399381.25, 549.0), # Mar-Jun
(3, 7): (361814.50, 344152.50, 375015.50, 549.0), # Mar-Jul
(3, 8): (309186.00, 294559.75, 321067.00, 549.0), # Mar-Aug
(3, 9): (241259.00, 223740.00, 253670.50, 549.0), # Mar-Sep
(3, 10): (219402.50, 208280.50, 230152.00, 549.0), # Mar-Oct
(3, 11): (247274.00, 235556.50, 257797.25, 549.0), # Mar-Nov
(3, 12): (367927.00, 349634.25, 380767.75, 549.0), # Mar-Dec
(4, 5): (391103.00, 371128.25, 405921.00, 549.0), # Apr-May
(4, 6): (382233.00, 365624.50, 396880.50, 549.0), # Apr-Jun
(4, 7): (354168.50, 341128.75, 366589.25, 548.0), # Apr-Jul
(4, 8): (300416.50, 284996.50, 317564.00, 548.5), # Apr-Aug
(4, 9): (225956.00, 210163.00, 242784.25, 548.0), # Apr-Sep
(4, 10): (207010.00, 194652.50, 219858.50, 549.0), # Apr-Oct
(4, 11): (233842.50, 224488.50, 243371.25, 549.0), # Apr-Nov
(4, 12): (353418.00, 340148.00, 367024.50, 548.0), # Apr-Dec
(5, 6): (378949.00, 359401.25, 395366.00, 548.0), # May-Jun
(5, 7): (355846.00, 339660.00, 374676.00, 548.5), # May-Jul
(5, 8): (306893.50, 290227.25, 327889.75, 548.0), # May-Aug
(5, 9): (230192.50, 211397.25, 253140.75, 549.0), # May-Sep
(5, 10): (197582.00, 186184.00, 213469.75, 549.0), # May-Oct
(5, 11): (216630.00, 209058.00, 223355.25, 549.0), # May-Nov
(5, 12): (334639.00, 322462.75, 344777.25, 549.0), # May-Dec
(6, 7): (364683.00, 346838.50, 386625.75, 549.0), # Jun-Jul
(6, 8): (317711.00, 293881.75, 340245.75, 549.0), # Jun-Aug
(6, 9): (234496.00, 214429.00, 256260.50, 549.0), # Jun-Sep
(6, 10): (199118.00, 182987.75, 217445.00, 549.0), # Jun-Oct
(6, 11): (206853.50, 196577.75, 214035.00, 548.5), # Jun-Nov
(6, 12): (311791.00, 299312.75, 322937.25, 549.0), # Jun-Dec
(7, 8): (321920.00, 297495.75, 344139.50, 548.0), # Jul-Aug
(7, 9): (233935.50, 216610.50, 255196.75, 549.0), # Jul-Sep
(7, 10): (195357.50, 180265.75, 210540.25, 549.0), # Jul-Oct
(7, 11): (190111.00, 181210.75, 200435.50, 549.0), # Jul-Nov
(7, 12): (280071.00, 268760.75, 292520.75, 549.0), # Jul-Dec
(8, 9): (218566.50, 206763.50, 235713.50, 548.0), # Aug-Sep
(8, 10): (183435.00, 168294.25, 200184.25, 549.0), # Aug-Oct
(8, 11): (175365.50, 163921.25, 187042.25, 549.0), # Aug-Nov
(8, 12): (239777.00, 228854.50, 250519.25, 549.0), # Aug-Dec
(9, 10): (176823.50, 160305.75, 192265.75, 549.0), # Sep-Oct
(9, 11): (171650.50, 158131.00, 183441.50, 549.0), # Sep-Nov
(9, 12): (211443.50, 200702.00, 222217.50, 549.0), # Sep-Dec
(10, 11): (188972.50, 165478.00, 203092.00, 549.0), # Oct-Nov
(10, 12): (211302.00, 194398.25, 221180.25, 548.0), # Oct-Dec
(11, 12): (266319.00, 252818.25, 283819.75, 549.0), # Nov-Dec
}
NONE_STATS = (360831.0, 346748.0, 383264.5) # no spraying: median, q25, q75 (none.output col 3)
YEARLY_SEP = (260707.5, 249588.75, 276903.0, 274.0) # yearly September: median, q25, q75, days
THRESH_BEST = (254206.0, 351.0) # best adaptive threshold (1 sd, monthly): median, days
def pair_index(i, j):
"""Column of tot.output[['spray_twice_yearly_all']] for months i<j (1-based), from the authors'
Fig 4 script: pairs.by.month[i,j] <- -0.5*(i^2-25*i+24)+(j-i)."""
return int(-0.5 * (i * i - 25 * i + 24) + (j - i))
def from_rdata(path):
import rdata # pip install rdata
conv = rdata.conversion.convert(rdata.parser.parse_file(path), default_encoding="utf8")
tw = np.asarray(conv["tot.output"]["spray_twice_yearly_all"]) # 400 sims x 3 outputs x 66 pairs
yr = np.asarray(conv["tot.output"]["spray_yearly_all"])
th = np.asarray(conv["tot.output"]["spray_threshold_mean"])
no = np.asarray(conv["none.output"])
assert tw.shape == (400, 3, 66)
stats = {}
for i in range(1, 12):
for j in range(i + 1, 13):
k = pair_index(i, j) - 1
x = tw[:, 2, k]
q25, med, q75 = np.quantile(x, [0.25, 0.5, 0.75]) # = R quantile type 7
stats[(i, j)] = (round(med, 2), round(q25, 2), round(q75, 2), round(np.median(tw[:, 0, k]), 1))
none = tuple(np.quantile(no[:, 2], [0.5, 0.25, 0.75]))
ysep = tuple(np.quantile(yr[:, 2, 8], [0.5, 0.25, 0.75])) + (np.median(yr[:, 0, 8]),)
tbest = (np.median(th[:, 2, 0]), np.median(th[:, 0, 0]))
# Is Mar-Oct the best pair under any other summary in the file? (only with the RData file)
pairs = list(stats)
for label, f in [("output column 2, median", lambda k: np.median(tw[:, 1, k])),
("output column 3 (infections), mean", lambda k: np.mean(tw[:, 2, k]))]:
v = {p: f(pair_index(*p) - 1) for p in pairs}
order = sorted(pairs, key=v.get)
print(" [RData] %s: best pair %s-%s; Mar-Oct ranks %d of 66"
% (label, MON[order[0][0] - 1], MON[order[0][1] - 1], order.index((3, 10)) + 1))
return stats, none, ysep, tbest
def source_data(rpath):
print("\n3) Authors' source data (400 simulations per strategy)")
if rpath:
stats, none, ysep, tbest = from_rdata(rpath)
for k, v in PAIR_STATS.items():
assert np.allclose(v, stats[k], atol=0.01), (k, v, stats[k])
assert np.allclose(none, NONE_STATS) and np.allclose(ysep, YEARLY_SEP) and np.allclose(tbest, THRESH_BEST)
print(" recomputed from %s: matches the embedded table for all 66 pairs" % os.path.basename(rpath))
else:
print(" (no RData path given: using the embedded table produced by from_rdata())")
ranked = sorted(PAIR_STATS.items(), key=lambda kv: kv[1][0])
rank = {k: r + 1 for r, (k, _) in enumerate(ranked)}
b = NONE_STATS[0]
print(" No spraying: median %.0f [IQR %.0f-%.0f] (paper: 361 [347, 383] thousand)" % NONE_STATS)
print(" Yearly Sep: median %.0f [IQR %.0f-%.0f], %.0f days (paper: 261 [250, 277]; 274 days)" % YEARLY_SEP)
print(" Best threshold: median %.0f, %.0f days (paper: 254; 351 days)" % THRESH_BEST)
print(" Five best twice-yearly pairs:")
for (i, j), v in ranked[:5]:
print(" %2d. %s-%s median %.0f [IQR %.0f-%.0f], %.0f spray days" % ((rank[(i, j)], MON[i-1], MON[j-1]) + v))
out = {}
for (i, j), name in [((9, 11), "Sep-Nov"), ((3, 10), "Mar-Oct")]:
m, lo, hi, d = PAIR_STATS[(i, j)]
red = 100 * (1 - m / b)
out[name] = (m, lo, hi, d, rank[(i, j)], red)
print(" %s: median %.1f [IQR %.1f-%.1f], %.0f spray days, rank %d of 66, %.1f%% fewer than no spraying"
% (name, m, lo, hi, d, rank[(i, j)], red))
print(" Mar-Oct minus Sep-Nov median infections: %.0f; pairs better than Mar-Oct: %d"
% (out["Mar-Oct"][0] - out["Sep-Nov"][0], out["Mar-Oct"][4] - 1))
print(" IQRs overlap? %s" % (out["Mar-Oct"][1] <= out["Sep-Nov"][2]))
days = sorted(set(v[3] for v in PAIR_STATS.values()))
print(" Median spray days across all 66 pairs: %s (so 549 days does not identify a pair)" % days)
print(" Pairs with first spray in Aug/Sep and second in Oct/Nov, ranks: %s" %
sorted(rank[(i, j)] for i in (8, 9) for j in (10, 11)))
return ranked, rank, out
# ---------------------------------------------------------------- 4. figures
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#b9b7b1"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": FG,
"axes.labelcolor": FG, "xtick.color": FG2, "ytick.color": FG2,
"axes.edgecolor": FG2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
def fig1(src):
sn, mo = src["Sep-Nov"], src["Mar-Oct"]
fig = plt.figure(figsize=(7.2, 4.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.9); ax.axis("off")
def panel(x, y, w, h, edge, title, body, sub):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.14, y + h - 0.16, title, ha="left", va="top", fontsize=11, weight="bold", color=FG)
ax.text(x + 0.14, y + h - 0.50, body, ha="left", va="top", fontsize=10, color=FG, linespacing=1.4)
ax.text(x + 0.14, y + 0.14, sub, ha="left", va="bottom", fontsize=10, color=FG2, linespacing=1.3)
panel(0.12, 1.95, 3.05, 2.85, ORANGE, "Abstract",
"“spraying twice yearly, in\n"
"March and October, which led to\n"
"a median of 172,000 infections\n"
"[IQR: 158,000–183,000], a 52%\n"
"reduction from baseline”",
"Same pair in Results, ULV para 2:\n“in October and March” … 549 days")
panel(4.03, 1.95, 3.05, 2.85, ORANGE, "Results, Table 2, Discussion",
"“The best strategy for spraying\n"
"twice yearly was spraying in\n"
"September and November (172,000;\n"
"IQR: 158,000–183,000)”\n"
"Table 2: 172 [158, 183]; 549 days",
"Same pair in the legends of\nFig 4d, Fig 8c and Fig 10")
ax.add_patch(FancyArrowPatch((3.19, 3.45), (4.01, 3.45), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=FG))
ax.text(3.6, 3.68, "conflict", ha="center", va="bottom", fontsize=10, weight="bold", color=FG)
ax.text(3.6, 3.22, "same\nnumbers,\ndifferent\nmonths", ha="center", va="top", fontsize=10, color=FG2)
# recomputed strip
ax.add_patch(FancyBboxPatch((0.12, 0.12), 6.96, 1.5, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="#eef4fb", ec=BLUE, lw=1.5))
ax.text(0.26, 1.48, "Recomputed from the authors' source data (66 month pairs, 400 simulations each)",
ha="left", va="top", fontsize=10, weight="bold", color=BLUE)
ax.text(0.26, 1.16, "Mar–Oct: median %s [IQR %s–%s], rank %d of 66, %.0f%% reduction"
% (f"{mo[0]:,.0f}", f"{mo[1]:,.0f}", f"{mo[2]:,.0f}", mo[4], mo[5]),
ha="left", va="top", fontsize=10, color=FG)
ax.text(0.26, 0.86, "Sep–Nov: median %s [IQR %s–%s], rank %d of 66, %.0f%% reduction"
% (f"{sn[0]:,.0f}", f"{sn[1]:,.0f}", f"{sn[2]:,.0f}", sn[4], sn[5]),
ha="left", va="top", fontsize=10, color=FG)
ax.text(0.26, 0.56, "Both pairs: median %.0f spray days. The reported 172,000 [158,000–183,000]\n"
"and 52%% reduction fit Sep–Nov only." % sn[3], ha="left", va="top", fontsize=10, color=FG2,
linespacing=1.3)
# connectors from strip to the panels
ax.annotate("", xy=(1.65, 1.93), xytext=(1.65, 1.64),
arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=2, linestyle=(0, (3, 2))))
ax.text(1.75, 1.785, "does not match", ha="left", va="center", fontsize=10, color=BLUE)
ax.annotate("", xy=(5.55, 1.93), xytext=(5.55, 1.64),
arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=2))
ax.text(5.65, 1.785, "matches", ha="left", va="center", fontsize=10, color=BLUE)
save(fig, "fig1")
def fig2(ranked, rank, src, n_show=20):
rows = ranked[:n_show]
fig, ax = plt.subplots(figsize=(7.2, 6.4))
fig.subplots_adjust(left=0.22, right=0.97, top=0.86, bottom=0.09)
y_rep = 0
ax.errorbar(REP["ulv2"] / 1e3, y_rep,
xerr=[[(REP["ulv2"] - REP["ulv2_lo"]) / 1e3], [(REP["ulv2_hi"] - REP["ulv2"]) / 1e3]],
fmt="o", ms=8, color=ORANGE, lw=2, capsize=0, zorder=3)
labels = ["Reported best pair\n172 [158, 183]"]
for r, ((i, j), (m, lo, hi, d)) in enumerate(rows, start=1):
hl = (i, j) in [(9, 11), (3, 10)]
c = BLUE if hl else GRAY
ax.errorbar(m / 1e3, r + 0.5, xerr=[[(m - lo) / 1e3], [(hi - m) / 1e3]], fmt="o",
ms=8 if hl else 6, color=c, lw=2 if hl else 1.5, capsize=0, zorder=3)
labels.append("%d. %s–%s" % (r, MON[i - 1], MON[j - 1]))
if hl:
note = ("Pair named in the Abstract" if (i, j) == (3, 10) else "Pair named in Table 2")
left = (i, j) == (3, 10) # place the label where the row has free space
ax.text((lo / 1e3 - 6) if left else (hi / 1e3 + 6), r + 0.5, "%s: %.1f k" % (note, m / 1e3),
va="center", ha="right" if left else "left",
fontsize=10, color=BLUE)
ax.set_yticks([0] + [r + 0.5 for r in range(1, n_show + 1)])
ax.set_yticklabels(labels, fontsize=10)
for t in ax.get_yticklabels()[1:]:
t.set_color(FG2)
ax.get_yticklabels()[0].set_color(ORANGE)
ax.invert_yaxis()
ax.axvline(NONE_STATS[0] / 1e3, color=FG2, lw=1)
ax.text(NONE_STATS[0] / 1e3 - 4, n_show + 0.9, "no spraying\n%.0f k" % (NONE_STATS[0] / 1e3),
ha="right", va="bottom", fontsize=10, color=FG2)
ax.set_xlim(0, 400)
ax.set_ylim(n_show + 1.6, -0.8)
ax.set_xlabel("Total infections 2000–2010, thousands (median and IQR, 400 simulations)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
fig.text(0.02, 0.975, "Twice-yearly ULV month pairs ranked by median infections (best %d of 66)" % n_show,
fontsize=11, weight="bold", ha="left", va="top")
fig.text(0.02, 0.938, "Orange: the paper's reported best twice-yearly result (Abstract and Table 2).\n"
"Blue: the two named pairs, recomputed from the authors' source data. Gray: other pairs.",
fontsize=10, color=FG2, ha="left", va="top", linespacing=1.3)
save(fig, "fig2")
if __name__ == "__main__":
check_quotes()
arithmetic()
rpath = sys.argv[1] if len(sys.argv) > 1 else None
ranked, rank, src = source_data(rpath)
fig1(src)
fig2(ranked, rank, src)
print("\nFigures written to", HERE)
Abstract reports SF2/SRSF1 decrease in astrocytes; the paper's data show it only in PNG and SPG
PLoS Genet (2023) · doi:10.1371/journal.pgen.1010973 · PMC10553832
Summary
The abstract says that after glial TDP-43 induction "SF2/SRSF1 levels were found to be decreased in both PNG and in astrocytes." The paper's own expression data, S8 Table, report a significant decrease only in perineurial glia (PNG; log2 fold change −1.31, p<0.0001) and subperineurial glia (SPG; −0.85, p<0.001). For astrocyte-like glia (ALG), the cell type the paper calls astrocytes, the table gives "n/a", and the Results say outright that TDP-43 induction in astrocytes "does not on its own cause a decrease in SF2/SRSF1 expression." So the abstract names the one cell type in which the body says there is no decrease and leaves out SPG, where there is one. The lifespan rescue by SF2/SRSF1 knockdown (Fig 5) comes from separate experiments and is not affected, but the abstract misstates the only specific transcriptomic result it names.
What the paper reports
Abstract:
Notably, SF2/SRSF1 levels were found to be decreased in both PNG and in astrocytes. We found that further knockdown of SF2/SRSF1 in either PNG or astrocytes lessens the detrimental effects of TDP-43 pathology on lifespan, but extends survival of the glial cells.
Results, section "Knockdown of SF2/SRSF1 in astrocytes or PNG ameliorates the toxicity of TDP-43", first paragraph:
Our differential expression analysis identified SF2/SRSF1 as being down regulated in response to TDP-43 expression in PNG (-1.31 log2 fold change, p<0.0001) and SPG (-0.85 log2 fold change, p < .001), but not in CG, ALG, or EG (S8 Table).
Same section, second paragraph:
The fact that SF2/SRSF1 levels are reduced in SPG and PNG, but not other glial cell types, is intriguing given that pan-glial knockdown of SF2/SRSF1 can ameliorate the lifespan effects of pan-glial TDP-43 pathology (Fig 5A) [55]. We observe a substantial impact on lifespan from pathological TDP-43 induction in PNG and ALG (Fig 1), but only see down regulation of SF2/SRSF1 in PNG (S8 Table).
And although TDP-43 induction in astrocytes does not on its own cause a decrease in SF2/SRSF1 expression, knock-down in astrocytes of SF2/SRSF1 by RNAi does ameliorate the effects on lifespan from astrocytic induction of TDP-43 pathology.
Introduction, last paragraph:
We observe that in response to TDP-43 pathology, SF2/SRSF1 is down-regulated in both the PNG and SPG specialized BBB glia.
Discussion, paragraph on the expression profiling:
Notably, the SF2/SRSF1 gene exhibits decreased expression in PNG and SPG, but not other glial cell types (S8 Table).
How the paper identifies astrocytes. Introduction:
the cortex glia (CG), astrocyte-like glia (ALG), and ensheathing glia (EG) provide many of the functional characteristics of mammalian astrocytes and microglia
The strain table (Table 2) lists "R86E01-Gal4 (ALG)". The Fig 5C genotype in the Methods is "Tubulin-Gal80 ts / UAS-SF2-IR; UAS-TDP-43/ R86E01-Gal4", and the Fig 5 legend labels that panel "ALGts + TDP-43 + SF2-IR". The Results describe the same experiment as "the PNG or Astrocyte Gal4 lines" and "TDP-43 induction in either PNG or ALG (Fig 5B and 5C)". In this paper "astrocytes" and ALG are the same cells.
S8 Table (supplementary file pgen.1010973.s011), legend:
SF2 expression was significantly reduced at day 2 post induction of TDP-43 in SPG and PNG day 2.
The table's body, paraphrased from the PDF (row label "SF2 expression (MRN)"; values per group, given here as control → +TDP-43, then log2 fold change and p):
- PNG: 899 → 363; −1.31; p<0.0001
- SPG: 1090 → 594; −0.85; p<0.001
- CG: 780 → 727; n/a; n/a
- ALG: 591 → 453; n/a; n/a
- EG: 780 → 669; n/a; n/a
The column headers give n = 3 libraries per group, except n = 2 for the CG and ALG controls. The table does not say whether the per-group values are means, and it does not define "n/a"; the Results' "but not in CG, ALG, or EG" supports reading "n/a" as not significant. The Methods say the analysis used DESeq2 with a Benjamini–Hochberg FDR threshold of p < 0.05.
What the paper's own numbers show
Method. I took the five cell types' per-group SF2/SRSF1 expression values (control and TDP-43) and the reported log2 fold changes and p values from S8 Table. For each cell type I computed the raw log2 ratio of the two group values, treating them as group-level summaries. This is not the DESeq2 estimate, which is model-based and works on per-library counts, so it can only check that the table's group values and its reported fold changes agree; it is not a test. I then listed which cell types the Abstract, Introduction, Results, Discussion and S8 Table each say are decreased, and compared the lists. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| PNG, SF2/SRSF1 change | S8 Table; Results | 899 → 363 MRN; log2FC −1.31, p<0.0001 | log2(363/899) = −1.308; group values fall 59.6% (2^−1.31 = 0.403, a 59.7% fall) |
| SPG, SF2/SRSF1 change | S8 Table; Results | 1090 → 594 MRN; log2FC −0.85, p<0.001 | log2(594/1090) = −0.876; group values fall 45.5% (2^−0.85 = 0.555, a 44.5% fall). The 0.026 gap from −0.85 is consistent with model-based estimation |
| ALG (astrocyte-like glia), SF2/SRSF1 change | S8 Table; Results | 591 → 453 MRN; log2FC n/a, p n/a | log2(453/591) = −0.384 (−23.4%); not reported as significant ("n/a", read with the Results' "but not in ... ALG") |
| CG, SF2/SRSF1 change | S8 Table | 780 → 727 MRN; n/a | log2(727/780) = −0.102 (−6.8%) |
| EG, SF2/SRSF1 change | S8 Table | 780 → 669 MRN; n/a | log2(669/780) = −0.221 (−14.2%) |
| Cell types with a significant decrease | S8 Table and legend | PNG, SPG | PNG, SPG |
| Cell types with a decrease, per the text | Introduction; Results; Discussion | PNG and SPG (Results and Discussion add: not CG, ALG or EG) | agrees with S8 Table |
| Cell types with a decrease, per the abstract | Abstract | PNG and astrocytes (ALG) | ALG is not significant in S8 Table; SPG, which is significant, is not named |
Figure
Figure 1. The abstract sentence (left) and the Results sentences and S8 Table entry for the same cell type (right), quoted verbatim from the article as archived in PMC. The abstract names astrocytes, which the paper identifies as ALG, as a cell type in which SF2/SRSF1 decreased. The Results and S8 Table report no significant change in ALG (591 to 453 MRN, log2 fold change "n/a"), and the Results state that TDP-43 induction in astrocytes does not on its own decrease SF2/SRSF1. The Introduction and Discussion give the decrease as PNG and SPG.
Figure 2. SF2/SRSF1 log2 fold change after TDP-43 induction (day 2) in each glial cell type. Orange diamonds are the DESeq2 estimates reported in S8 Table and the Results, which exist only for PNG and SPG; the other three cell types are given as "n/a", which the Results describe as not down-regulated. Blue circles are the raw log2 ratio of the S8 Table group values, computed by recompute.py.txt as a consistency check. The vertical line marks no change. The ALG point, which the abstract describes as decreased, is a non-significant −0.38.
Why it matters
The abstract is what most readers and indexing services see, and this sentence is the only specific transcriptomic result the abstract names (it otherwise says only that the authors "identified numerous glial cell-type specific transcriptional changes"). As written, it tells readers that TDP-43 lowers SF2/SRSF1 in the two cell types with the strongest lifespan effects (PNG and astrocytes), and the next sentence's "further knockdown" builds on that. The body's account is different, and the Results call it "intriguing": the decrease is confined to the two blood-brain-barrier glia (PNG and SPG), it is absent in astrocytes, and knockdown in astrocytes still extends lifespan. The Results draw exactly this contrast ("although TDP-43 induction in astrocytes does not on its own cause a decrease..."). A reader who relies on the abstract is told there is a decrease in astrocytes, where the paper reports no significant change, and does not learn about the SPG decrease.
What is not affected: the lifespan extension from SF2/SRSF1 knockdown in pan-glial, PNG and ALG TDP-43 models (Fig 5A–C, log-rank tests), the delayed glial loss with knockdown (Fig 5D–E), and the reaper ablation result in PNG. These are separate experiments and do not depend on the expression data. The title's claim stands.
Explanations considered
"Astrocytes" in the abstract means a different cell type, such as SPG. This does not fit. The paper uses "astrocytes" for ALG throughout: the Results section title is "Knockdown of SF2/SRSF1 in astrocytes or PNG", the experiment uses "the PNG or Astrocyte Gal4 lines" and reports the outcome as "PNG or ALG (Fig 5B and 5C)", and the Fig 5C driver is R86E01-Gal4, listed in Table 2 as the ALG line. SPG is described as a blood-brain-barrier glial type, alongside PNG, and is never called astrocytes.
A different time point or dataset shows an ALG decrease. None found. S8 Table covers one time point (day 2 after induction), the same as the RNA-seq design described in the Results, and the text cites no other SF2/SRSF1 expression data.
The abstract describes a non-significant trend. The ALG group values do fall (raw log2 ratio −0.38, from n = 2 control libraries), but the paper reports this as n/a, and its own Results rule out this reading by saying ALG is not down-regulated and that astrocytic TDP-43 "does not on its own cause a decrease." The abstract's wording ("were found to be decreased", the same wording used for PNG) presents it as a finding.
Text conversion or archiving error. Not supported. The PLOS article page, the Europe PMC full text and the PMC archive have the same abstract sentence, and the S8 Table PDF from the publisher has the values above.
The small SPG discrepancy (−0.876 vs −0.85). A ratio of group values need not equal a DESeq2 model estimate. This gap is small and does not bear on the finding; the PNG value agrees to within 0.002.
A narrower reading of one Results sentence. "...only see down regulation of SF2/SRSF1 in PNG" agrees with the table: it compares only PNG and ALG, the two cell types with the strongest lifespan effects, and SPG is not part of that comparison.
Status
As of 2026-10-23:
- Version of record. The PLOS Genetics article page (the current "Version 2"; a "Version 1 Uncorrected Proof" is also linked) has the same abstract and Results wording as the PMC archive. The Europe PMC full text has the identical abstract sentence.
- Supplements. S8 Table (doi:10.1371/journal.pgen.1010973.s011), a one-page PDF, was downloaded from the publisher and its text extracted; the values match those above. It is the table the text cites for this result. The other supplementary items (S1–S3 Figs, S1–S7 Tables) were not needed for this check.
- Corrections. Crossref lists only a publisher "new version" relation dated 2023-10-05, ten days after online publication on 2023-09-25. This fits the replacement of the Version 1 Uncorrected Proof by Version 2. It has no correction DOI. PMC and Europe PMC list no correction, erratum, expression of concern or retraction, and Europe PMC's record links only the bioRxiv preprint (doi:10.1101/2023.05.04.539439). The PLOS comments page shows no reader comments.
- PubPeer. Web searches found no PubPeer thread or other public discussion of this point. PubPeer itself could not be searched directly (it returned a human-verification page), so a thread there cannot be ruled out.
Sources: the article as archived in PMC (PMC10553832); journals.plos.org/plosgenetics/article?id=10.1371/journal.pgen.1010973 and its supplementary file s011; Crossref and Europe PMC records for doi:10.1371/journal.pgen.1010973.
Suggested correction
In the abstract, replace
Notably, SF2/SRSF1 levels were found to be decreased in both PNG and in astrocytes. We found that further knockdown of SF2/SRSF1 in either PNG or astrocytes lessens...
with wording that matches the Results and S8 Table, for example:
Notably, SF2/SRSF1 levels were found to be decreased in PNG and SPG, but not in astrocytes. We found that knockdown of SF2/SRSF1 in either PNG or astrocytes lessens...
The word "further" would then apply only to PNG. If the authors prefer to keep the sentence about the two cell types tested in Fig 5, it could instead say that SF2/SRSF1 was decreased in PNG but not in astrocytes.
Recomputation script (python recompute.py.txt)
"""Recompute the SF2/SRSF1 cell-type result of PMC10553832
(PLoS Genet 2023, doi:10.1371/journal.pgen.1010973).
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file, plus five candidate figure designs
(design_A ... design_E, .svg and .png) in ./candidates/.
Inputs are copied from the paper:
S8 Table (supplementary file pgen.1010973.s011, "SF2 expression across glial cell
types"): the per-group SF2 expression value (row label "SF2 expression (MRN)") for
control and +TDP-43 libraries at day 2 after induction, the reported log2 fold change
and p value. The table does not define "n/a"; the Results' "but not in CG, ALG, or EG"
supports reading it as not significant. The table does not say whether the per-group
values are means; they are treated here as group-level summaries.
Text excerpts: Abstract, Introduction, Results ("Knockdown of SF2/SRSF1 in astrocytes
or PNG ameliorates the toxicity of TDP-43"), Discussion, as archived in PMC.
The raw log2 ratio of two group values is not the DESeq2 estimate the paper reports
(which is model-based and uses per-library normalized counts); it is shown only as a
check that the reported values and the table's group values agree.
"""
import math
import os
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs (from the paper)
# cell type: (n control, MRN control, n TDP-43, MRN TDP-43, reported log2FC, reported p)
S8 = {
"PNG": (3, 899, 3, 363, -1.31, "<0.0001"),
"SPG": (3, 1090, 3, 594, -0.85, "<0.001"),
"CG": (2, 780, 3, 727, None, "n/a"),
"ALG": (2, 591, 3, 453, None, "n/a"),
"EG": (3, 780, 3, 669, None, "n/a"),
}
LONG = {
"PNG": "PNG (perineurial glia)",
"SPG": "SPG (subperineurial glia)",
"CG": "CG (cortex glia)",
"ALG": "ALG (astrocyte-like glia)",
"EG": "EG (ensheathing glia)",
}
ORDER = ["PNG", "SPG", "CG", "ALG", "EG"]
ABSTRACT = ("Notably, SF2/SRSF1 levels were found to be decreased in both PNG "
"and in astrocytes.")
RESULTS_1 = ("Our differential expression analysis identified SF2/SRSF1 as being down "
"regulated in response to TDP-43 expression in PNG (-1.31 log2 fold change, "
"p<0.0001) and SPG (-0.85 log2 fold change, p < .001), but not in CG, ALG, "
"or EG (S8 Table).")
RESULTS_2 = ("And although TDP-43 induction in astrocytes does not on its own cause a "
"decrease in SF2/SRSF1 expression, knock-down in astrocytes of SF2/SRSF1 by "
"RNAi does ameliorate the effects on lifespan from astrocytic induction of "
"TDP-43 pathology.")
# What each part of the paper says about an SF2/SRSF1 decrease in each cell type.
# "dec" = says decreased, "not" = says not decreased, None = cell type not named.
SAYS = {
"Abstract": {"PNG": "dec", "SPG": None, "CG": None, "ALG": "dec", "EG": None},
"Introduction": {"PNG": "dec", "SPG": "dec", "CG": None, "ALG": None, "EG": None},
"Results": {"PNG": "dec", "SPG": "dec", "CG": "not", "ALG": "not", "EG": "not"},
"Discussion": {"PNG": "dec", "SPG": "dec", "CG": "not", "ALG": "not", "EG": "not"},
}
# ---------------------------------------------------------------- style
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
REP = "#eb6834" # values as reported by the paper
REC = "#2a78d6" # values recomputed from the paper's own numbers
GRID = "#e4e3df"
PANEL = "#f3f2ee"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans",
"font.size": 10,
"axes.edgecolor": INK2,
"axes.labelcolor": INK,
"xtick.color": INK2,
"ytick.color": INK2,
"text.color": INK,
"svg.fonttype": "none",
"lines.linewidth": 2,
})
def save(fig, path_noext):
fig.savefig(path_noext + ".svg", facecolor=BG)
fig.savefig(path_noext + ".png", dpi=200, facecolor=BG)
plt.close(fig)
def wrap(s, n):
return "\n".join(textwrap.wrap(s, n))
# ---------------------------------------------------------------- recomputation
print("S8 Table, SF2/SRSF1 expression (MRN) per group, day 2 after induction")
print(f"{'cell type':28s}{'ctrl':>6s}{'TDP43':>7s}{'ratio':>8s}{'raw log2':>10s}"
f"{'change':>9s}{'reported':>10s}{'p':>9s}")
RAW = {}
for k in ORDER:
nc, c, nt, t, r, p = S8[k]
ratio = t / c
RAW[k] = math.log2(ratio)
print(f"{LONG[k]:28s}{c:6d}{t:7d}{ratio:8.3f}{RAW[k]:10.3f}{100 * (ratio - 1):8.1f}%"
f"{('' if r is None else f'{r:.2f}'):>10s}{p:>9s}")
print("\nReported log2 fold changes converted to percent change")
for k in ("PNG", "SPG"):
r = S8[k][4]
print(f" {k}: 2^({r}) = {2 ** r:.3f} -> {100 * (2 ** r - 1):.1f}%")
print(f" SPG: reported -0.85 vs raw log2 ratio of group values {RAW['SPG']:.3f} "
f"(difference {RAW['SPG'] - (-0.85):.3f})")
print(f" PNG: reported -1.31 vs raw log2 ratio of group values {RAW['PNG']:.3f} "
f"(difference {RAW['PNG'] - (-1.31):.3f})")
sig = [k for k in ORDER if S8[k][5] != "n/a"]
print("\nCell types with a significant SF2/SRSF1 decrease in S8 Table:", ", ".join(sig))
abstract_types = ["PNG", "ALG"] # 'astrocytes' = ALG, see writeup
print("Cell types the abstract says are decreased: PNG, astrocytes (= ALG)")
print("Named in the abstract but not significant in S8 Table:",
", ".join(k for k in abstract_types if k not in sig))
print("Significant in S8 Table but not named in the abstract:",
", ".join(k for k in sig if k not in abstract_types))
print("\nWhat each section says (dec = decreased, not = not decreased, - = not named)")
print(f"{'':14s}" + "".join(f"{k:>6s}" for k in ORDER))
for sec, d in SAYS.items():
print(f"{sec:14s}" + "".join(f"{(d[k] or '-'):>6s}" for k in ORDER))
print(f"{'S8 Table':14s}" + "".join(f"{('dec' if k in sig else 'n/a'):>6s}" for k in ORDER))
# ================================================================ design A: two text panels
def design_a(path):
fig = plt.figure(figsize=(W, 4.9), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
L = (0.025, 0.385)
R = (0.585, 0.975)
top, bot = 0.875, 0.17
for (x0, x1) in (L, R):
ax.add_patch(FancyBboxPatch((x0, bot), x1 - x0, top - bot,
boxstyle="round,pad=0,rounding_size=0.015",
fc=PANEL, ec=GRID, lw=1))
ax.text(L[0] + 0.015, top - 0.04, "Abstract", fontsize=12, weight="bold", va="top")
ax.text(L[0] + 0.015, top - 0.12, wrap("“" + ABSTRACT + "”", 33),
fontsize=10, va="top", linespacing=1.35)
ax.text(L[0] + 0.015, bot + 0.16, "Cell types said to decrease:", fontsize=10,
color=INK2, va="top")
ax.text(L[0] + 0.015, bot + 0.095, "PNG, astrocytes (ALG)", fontsize=10.5,
weight="bold", color=INK, va="top")
ax.text(R[0] + 0.015, top - 0.04, "Results and S8 Table", fontsize=12,
weight="bold", va="top")
ax.text(R[0] + 0.015, top - 0.12,
wrap("“…down regulated in response to TDP-43 expression in PNG "
"(-1.31 log2 fold change, p<0.0001) and SPG (-0.85 log2 fold change, "
"p < .001), but not in CG, ALG, or EG (S8 Table).”", 35),
fontsize=10, va="top", linespacing=1.35)
ax.text(R[0] + 0.015, top - 0.405,
wrap("“…TDP-43 induction in astrocytes does not on its own cause a "
"decrease in SF2/SRSF1 expression…”", 35),
fontsize=10, va="top", linespacing=1.35)
ax.text(R[0] + 0.015, bot + 0.16, "S8 Table, ALG: 591 → 453 MRN,", fontsize=10,
color=INK2, va="top")
ax.text(R[0] + 0.015, bot + 0.095, "log2FC n/a (not significant)", fontsize=10.5,
weight="bold", va="top")
ymid = 0.52
ax.annotate("", xy=(R[0] - 0.008, ymid), xytext=(L[1] + 0.008, ymid),
arrowprops=dict(arrowstyle="<|-|>", color=INK, lw=2,
shrinkA=0, shrinkB=0, mutation_scale=14))
ax.text((L[1] + R[0]) / 2, ymid + 0.035, "Conflict:\nastrocytes (ALG)",
ha="center", va="bottom", fontsize=10, weight="bold", linespacing=1.3)
ax.text((L[1] + R[0]) / 2, ymid - 0.035, "“decreased”\nvs\nnot decreased",
ha="center", va="top", fontsize=10, color=INK2, linespacing=1.3)
ax.text(0.025, 0.95,
"SF2/SRSF1 after TDP-43 induction: what the abstract says vs the paper's data",
fontsize=11, weight="bold", va="top")
ax.text(0.025, 0.13,
"Introduction and Discussion agree with the Results: the decrease is in PNG "
"and SPG.\nSPG is not named in the abstract.",
fontsize=10, color=INK2, va="top")
ax.text(0.025, 0.05, "Quotes verbatim from the article as archived in PMC; "
"S8 Table = supplementary file s011.", fontsize=10, color=INK2, va="top")
save(fig, path)
# ================================================================ design B: section x cell-type matrix
def design_b(path):
secs = list(SAYS) + ["S8 Table"]
fig, ax = plt.subplots(figsize=(W, 3.9), facecolor=BG)
fig.subplots_adjust(left=0.27, right=0.985, top=0.81, bottom=0.17)
ax.set_facecolor(BG)
ax.set_xlim(0, len(secs))
ax.set_ylim(len(ORDER), 0)
for j, sec in enumerate(secs):
for i, k in enumerate(ORDER):
if sec == "S8 Table":
nc, c, nt, t, r, p = S8[k]
txt = f"{r:.2f}\np{p}" if r is not None else "n/a"
state = "dec" if r is not None else "not"
col = REP
else:
state = SAYS[sec][k]
txt = {"dec": "decreased", "not": "not\ndecreased", None: "—"}[state]
col = INK if state else INK2
if sec == "Abstract" and k == "ALG":
txt = "decreased\n(conflict)"
fc = "#e9e7e1" if state == "dec" else BG
ax.add_patch(Rectangle((j + 0.04, i + 0.06), 0.92, 0.88, fc=fc, ec=GRID, lw=1))
ax.text(j + 0.5, i + 0.5, txt, ha="center", va="center", fontsize=10,
color=col, linespacing=1.15,
weight="bold" if (sec == "S8 Table" and state == "dec")
or (sec == "Abstract" and k == "ALG") else "normal")
# conflict cell
i = ORDER.index("ALG")
ax.add_patch(Rectangle((0.04, i + 0.06), 0.92, 0.88, fc="none", ec=INK, lw=2.5))
ax.set_xticks([j + 0.5 for j in range(len(secs))])
ax.set_xticklabels(secs, fontsize=10)
ax.xaxis.tick_top()
ax.set_yticks([i + 0.5 for i in range(len(ORDER))])
ax.set_yticklabels([LONG[k] for k in ORDER], fontsize=10)
ax.tick_params(length=0)
for s in ax.spines.values():
s.set_visible(False)
fig.text(0.015, 0.965, "Where the paper says SF2/SRSF1 decreases after TDP-43 induction",
fontsize=11, weight="bold", va="top")
fig.text(0.015, 0.06, "Shaded = says decreased; orange = S8 Table's reported log2 fold "
"change and p; \u2014 = not named.", fontsize=10, color=INK2)
fig.text(0.015, 0.015, "Astrocytes in the abstract are the ALG row (the paper's "
"astrocyte Gal4 line, R86E01, drives ALG).", fontsize=10, color=INK2)
save(fig, path)
# ================================================================ design C: log2FC dot plot
def design_c(path):
fig, ax = plt.subplots(figsize=(W, 3.9), facecolor=BG)
fig.subplots_adjust(left=0.27, right=0.72, top=0.79, bottom=0.215)
ax.set_facecolor(BG)
ys = {k: i for i, k in enumerate(ORDER)}
ax.axvline(0, color=INK2, lw=1)
for k in ORDER:
y = ys[k]
ax.plot(RAW[k], y + 0.13, "o", ms=9, color=REC, mec=BG, mew=1.5, zorder=3)
r = S8[k][4]
if r is not None:
ax.plot(r, y - 0.13, "D", ms=8.5, color=REP, mec=BG, mew=1.5, zorder=3)
lab = (f"reported {r:.2f}, p{S8[k][5]}" if r is not None
else "reported n/a (not sig.)")
ax.text(1.03, y, lab, transform=ax.get_yaxis_transform(), va="center",
fontsize=10, color=INK if r is not None else INK2, clip_on=False)
ax.set_yticks(range(len(ORDER)))
ax.set_yticklabels([LONG[k] for k in ORDER])
ax.set_ylim(len(ORDER) - 0.5, -0.6)
ax.set_xlim(-1.5, 0.25)
ax.set_xticks([-1.5, -1.0, -0.5, 0.0])
ax.set_xlabel("SF2/SRSF1 log2 fold change, TDP-43 vs control (day 2)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(length=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.text(0.03, 0, "no change", rotation=90, fontsize=10, color=INK2,
transform=ax.get_xaxis_transform(), ha="left", va="bottom")
yalg = ys["ALG"]
ax.annotate("abstract: “decreased”\n(astrocytes)",
xy=(RAW["ALG"] - 0.04, yalg + 0.13), xytext=(-1.48, yalg + 0.2),
fontsize=10, weight="bold", va="center",
arrowprops=dict(arrowstyle="-|>", color=INK, lw=1.5, mutation_scale=10))
handles = [Line2D([], [], marker="D", ls="", color=REP, ms=8.5,
label="reported in S8 Table / Results (DESeq2)"),
Line2D([], [], marker="o", ls="", color=REC, ms=9,
label="recomputed: log2(TDP-43 / control), S8 Table group values")]
fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.92),
frameon=False, fontsize=10, ncol=1, handletextpad=0.4)
fig.text(0.015, 0.975, "SF2/SRSF1 change by glial cell type: only PNG and SPG are "
"significant", fontsize=11, weight="bold", va="top")
fig.text(0.015, 0.025, "The recomputed ratio of group values is a consistency check, "
"not a substitute for the DESeq2 test.", fontsize=10, color=INK2)
save(fig, path)
# ================================================================ design D: dumbbell of group MRN
def design_d(path):
fig, ax = plt.subplots(figsize=(W, 3.9), facecolor=BG)
fig.subplots_adjust(left=0.27, right=0.73, top=0.79, bottom=0.215)
ax.set_facecolor(BG)
for i, k in enumerate(ORDER):
nc, c, nt, t, r, p = S8[k]
ax.plot([t, c], [i, i], color=REP, lw=2, alpha=0.55, zorder=2)
ax.plot(c, i, "o", ms=9, mfc=BG, mec=REP, mew=2, zorder=3)
ax.plot(t, i, "o", ms=9, color=REP, mec=REP, zorder=3)
pctc = 100 * (t / c - 1)
sigtxt = f"S8: p{p}" if r is not None else "S8: n/a"
ax.text(1.03, i, f"{pctc:+.0f}% ", transform=ax.get_yaxis_transform(),
va="center", ha="left", fontsize=10, color=REC, weight="bold")
ax.text(1.17, i, sigtxt, transform=ax.get_yaxis_transform(), va="center",
ha="left", fontsize=10, color=INK if r is not None else INK2)
ax.set_yticks(range(len(ORDER)))
ax.set_yticklabels([LONG[k] for k in ORDER])
ax.set_ylim(len(ORDER) - 0.5, -0.6)
ax.set_xlim(0, 1200)
ax.set_xlabel("SF2/SRSF1 expression (MRN), S8 Table group values")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(length=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ia = ORDER.index("ALG")
ax.annotate("abstract: “decreased”", xy=(453, ia), xytext=(40, ia + 0.42),
fontsize=10, weight="bold", va="center",
arrowprops=dict(arrowstyle="-|>", color=INK, lw=1.5, mutation_scale=10))
handles = [Line2D([], [], marker="o", ls="", mfc=BG, mec=REP, mew=2, ms=9,
label="control (reported)"),
Line2D([], [], marker="o", ls="", color=REP, ms=9,
label="+TDP-43 (reported)"),
Line2D([], [], ls="", marker="$\\%$", color=REC, ms=10,
label="change (recomputed)")]
fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.92),
frameon=False, fontsize=10, ncol=3, handletextpad=0.3, columnspacing=1.2)
fig.text(0.015, 0.975, "SF2/SRSF1 expression, control vs TDP-43, by glial cell type",
fontsize=11, weight="bold", va="top")
fig.text(0.015, 0.025, "Axis starts at zero. Controls: n = 2 libraries for CG and ALG, "
"3 otherwise; TDP-43: n = 3.", fontsize=10, color=INK2)
save(fig, path)
# ================================================================ design E: sentence revision
def runs(ax, x, y, parts, fs=10.5, fig_w=W):
"""Draw one line of text runs in axes-fraction coordinates.
parts = [(text, dict(color=..., weight=..., strike=bool, under=bool))]."""
fig = ax.figure
fig.set_dpi(200) # measure at the PNG resolution (Agg glyph advances depend on dpi)
r = fig.canvas.get_renderer()
for txt, st in parts:
wt = st.get("weight", "normal")
t = ax.text(x, y, txt.replace(" ", "\u00a0"), fontsize=fs,
color=st.get("color", INK), weight=wt, va="baseline")
bb = t.get_window_extent(renderer=r)
x1 = x + bb.width / (fig.dpi * fig_w)
if st.get("strike"):
ax.plot([x, x1], [y + 0.010, y + 0.010], color=st.get("color", INK), lw=1.8)
if st.get("under"):
ax.plot([x, x1], [y - 0.014, y - 0.014], color=st.get("color", INK), lw=1.8)
x = x1
return x
def design_e(path):
fig = plt.figure(figsize=(W, 3.8), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
fig.canvas.draw()
ax.text(0.025, 0.95, "The abstract sentence, and what the paper's own results support",
fontsize=11, weight="bold", va="top")
ax.text(0.025, 0.80, "As published (Abstract)", fontsize=10, color=INK2)
runs(ax, 0.025, 0.72, [("“…SF2/SRSF1 levels were found to be decreased in both PNG and in ", {}),
("astrocytes", {"strike": True, "weight": "bold"}),
(".”", {})])
ax.text(0.025, 0.58, "Consistent with Results, Introduction, Discussion and S8 Table",
fontsize=10, color=INK2)
runs(ax, 0.025, 0.50, [("SF2/SRSF1 levels were found to be decreased in both PNG and in ", {}),
("SPG", {"under": True, "weight": "bold"}),
(", not in astrocytes.", {"under": True, "weight": "bold"})])
ax.plot([0.025, 0.975], [0.40, 0.40], color=GRID, lw=1)
ax.text(0.025, 0.375, "S8 Table. Orange: reported MRN per group (control → TDP-43), log2FC and p.\n"
"Blue: recomputed raw log2 ratio of the two group values.", fontsize=10, color=INK2,
va="top")
cols = [0.025, 0.215, 0.405, 0.575, 0.80]
for x, k in zip(cols, ORDER):
nc, c, nt, t, r, p = S8[k]
ax.text(x, 0.245, k + (" (astrocytes)" if k == "ALG" else ""), fontsize=10,
weight="bold", va="top")
ax.text(x, 0.18, f"{c} → {t}", fontsize=10, color=REP, va="top")
ax.text(x, 0.115, (f"{r:.2f}, p{p}" if r is not None else "n/a"), fontsize=10,
color=REP, va="top")
ax.text(x, 0.05, f"raw {RAW[k]:+.2f}", fontsize=10, color=REC, va="top")
save(fig, path)
design_a(os.path.join(CAND, "design_A_two_panels"))
design_b(os.path.join(CAND, "design_B_section_matrix"))
design_c(os.path.join(CAND, "design_C_log2fc_dots"))
design_d(os.path.join(CAND, "design_D_expression_dumbbell"))
design_e(os.path.join(CAND, "design_E_sentence_revision"))
# The write-up uses design A as fig1 and design C as fig2.
design_a(os.path.join(HERE, "fig1"))
design_c(os.path.join(HERE, "fig2"))
print("\nwrote fig1, fig2 and candidates/design_A..E (.svg, .png)")
Table 2 odds ratios for haplogroup B4 and other zero-cell rows cannot come from the counts
PLoS Genet (2013) · doi:10.1371/journal.pgen.1003696 · PMC3744426
Summary
The paper reports that mitochondrial haplogroup B4 is less common in biliary atresia (BA) than in controls, with an odds ratio (OR) of 0.71 (95% CI 0.66–0.77), and an age- and sex-adjusted OR of 0.82 (0.71–0.85) in the Abstract. Its own Table 2 shows that none of the 71 BA patients and 23 of the 200 controls carry B4. With an empty cell the sample odds ratio is 0. The exact 95% CI is 0 to 0.45, and adding 0.5 to each cell gives 0.053 (0.003–0.881). The printed 0.71 (0.66–0.77) instead equals, to two decimals, a different statistic: the proportion of B4-negative participants who are controls divided by the proportion of B4-positive participants who are controls, 0.714 (0.660–0.772). Three other zero-cell rows of Table 2 (M8&Z, Y and D) show the same pattern. The direction of the B4 association and its p-value (Fisher exact p = 0.0009) stand, but the effect size and its precision are misreported for the paper's headline genetic finding, and the adjusted estimate cannot be finite for these counts.
What the paper reports
Results, "Association of BA with specific mitochondrial haplogroups", second paragraph:
Evaluation of haplogroup distributions revealed that the haplogroup B4 was less prevalent in patients with BA than in the control group (odds ratio [OR], 0.71; 95% confidence interval [CI], 0.66–0.77; p = 0.001).
The same paragraph, adjusted analysis:
Multivariate logistic-regression analysis of haplogroups associated with BA with adjustment for age and sex, showing the risk for haplogroup B4 was p = 0.007, OR 0.82, 95%CI 0.71–0.85
Abstract:
The haplogroup B4 and E prevalence were significantly lower and higher respectively, in the patients with BA than in the controls (odds ratios, 0.82 [p = 0.007] and 7.36 [p = 0.032] respectively) in multivariate logistic-regression analysis.
Table 2, "Association between biliary atresia (BA) and the mitochondrial haplogroups". The columns are BA positive | BA negative | control positive | control negative | OR | 95% CI | p:
B4 | 0 | 71 | 23 | 177 | 0.71 | 0.66–0.77 | 0.001*
M8&Z | 0 | 71 | 5 | 195 | 1.36 | 1.27–1.47 | 0.331
Y | 0 | 71 | 1 | 199 | 1.36 | 1.26–1.46 | 1.000
D | 2 | 69 | 0 | 200 | 3.90 | 3.18–4.78 | 0.068
Z | 1 | 70 | 1 | 199 | 2.84 | 0.18–46.06 | 0.456
C | 1 | 70 | 1 | 199 | 2.84 | 0.18–46.07 | 1.456
Table 1, "Descriptive data of studied population". The columns are BA | control (children) | control (adult):
haplogroup B4 | 0 | 5 | 18
Materials and Methods, "Subjects", and "Statistical analysis":
The genetic background was similar in both control groups and was put together for statistical calculation.
Frequencies of all mitochondrial haplogroups in the BA patients and controls were tested for independency using Pearson chi-square statistics and Fisher's exact test as appropriate.
The statistical analyses were performed using the Statistical Package for Social Science (SPSS, version 12) software package.
The conclusions that rest on the B4 result. Author Summary:
The prevalence of haplogroup B4 and E was significantly lower and higher respectively, in the patients with BA than in the controls.
Discussion, first paragraph:
We found that whereas the haplogroup B4 was significantly less likely to occur in the patients with BA, the haplogroup E was more likely to occur in the patients with BA than in the healthy controls.
Conclusion:
The use of the cybrid models helped to establish the association of haplogroups B4 and E with BA
What the paper's own numbers show
Method. I first checked the B4 counts against Table 1: 5 child and 18 adult controls carry B4, so 23 of the 200 pooled controls, and 0 of 71 BA patients do. I then recomputed every row of Table 2 from its four counts: the sample odds ratio with a Woolf (log-Wald) CI, the odds ratio with 0.5 added to each cell (Haldane–Anscombe; for a single binary exposure this equals the Firth-penalised logistic estimate, which the script confirms), the exact conditional odds ratio and CI for B4, Fisher's exact test and Pearson's chi-square test. For the rows that did not reproduce, I tried simple ratios of proportions (with Katz log CIs) to find a statistic that does match the printed values. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| B4 carriers, BA vs controls | Table 1, Table 2 | 0/71 vs 23/200 (5 + 18) | consistent: 23 = 5 + 18; 177 = 200 − 23 |
| B4 odds ratio | Table 2; Results | 0.71 | sample OR (0 × 177)/(71 × 23) = 0; exact conditional MLE 0 |
| B4 95% CI | Table 2; Results | 0.66–0.77 | exact conditional CI 0–0.45; with 0.5 added to each cell, OR 0.053 (0.003–0.881) |
| B4 CI width on the log scale | Table 2 | 0.154 | 5.63 for the 0.5-corrected OR (36 times wider) |
| Statistic that does match 0.71 (0.66–0.77) | Table 2 | — | P(control | B4-negative) / P(control | B4-positive) = (177/248)/(23/23) = 0.714 (0.660–0.772) |
| B4 p | Table 2; Results | 0.001 | Fisher exact two-sided 0.0009 (matches); Pearson 0.0028; Yates 0.0062 |
| B4 adjusted OR | Abstract; Results | 0.82 (0.71–0.85), p = 0.007 | no finite maximum-likelihood estimate: all 23 B4 carriers are controls, so the B4 coefficient is quasi-completely separated with or without age and sex in the model. The printed CI is also off-centre on the log scale (geometric midpoint 0.777, not 0.82) |
| M8&Z | Table 2 | OR 1.36 (1.27–1.47) | sample OR 0 (0/71 vs 5/200); 0.5-corrected 0.249 (0.014–4.552). Matching statistic: P(control | positive)/P(control | negative) = 1.364 (1.269–1.467) |
| Y | Table 2 | OR 1.36 (1.26–1.46) | sample OR 0 (0/71 vs 1/200); 0.5-corrected 0.930 (0.037–23.092). Matching statistic: P(control | positive)/P(control | negative) = 1.357 (1.263–1.457) |
| D | Table 2 | OR 3.90 (3.18–4.78), p = 0.068 | sample OR infinite (2/71 vs 0/200); 0.5-corrected 14.424 (0.684–304.150). Matching statistic: P(BA | positive)/P(BA | negative) = 3.899 (3.181–4.778). The printed CI excludes 1 while the printed p (Fisher 0.0679) does not reach 0.05 |
| C p | Table 2 | 1.456 | Fisher 0.456; row Z, with identical counts, prints 0.456. A probability cannot exceed 1 |
| The 14 rows without a zero cell | Table 2 | ORs and CIs | all 14 ORs reproduce to two decimals and the CIs to within rounding, e.g. E 11.88 (1.30–108.16), Fisher p 0.0177 vs 0.018 |
| E adjusted OR (side note) | Abstract; Results | 7.36 (1.22–94.6) | CI geometric midpoint 10.74, not 7.36; cannot be checked further without individual data |
The ratios of proportions that match the zero-cell rows are not oriented the same way: for B4 the B4-negative group is the numerator, whereas for M8&Z, Y and D the carrier group is the numerator. In the opposite orientation M8&Z would give 0.733 (0.682–0.788). For the two rows whose p-values differ from Fisher's test, F (0.324) and D5 (0.372), the printed values equal Pearson's chi-square p (0.3240 and 0.3723), which the Methods allow ("as appropriate"), so I do not count them as errors.
Figure
Figure 1. Haplogroup B4, 0 of 71 BA patients vs 23 of 200 controls, on one logarithmic ratio axis with the null value 1 marked by a thin vertical line. Orange marks the values the paper prints: the "OR" 0.71 (0.66–0.77) of Table 2 and the Results, and the adjusted "OR" 0.82 (0.71–0.85) of the Abstract and Results. Blue marks values recomputed from the Table 2 counts. The sample odds ratio is 0, which a log axis cannot show, so its exact 95% CI is drawn as an arrow from 0.45 towards zero. The odds ratio with 0.5 added to each cell is 0.053, with a CI (0.003–0.881) spanning more than two orders of magnitude. The bottom row is the ratio of the proportion of controls among B4-negative participants to that among B4-positive participants, 0.714 (0.660–0.772), which reproduces the printed 0.71 (0.66–0.77). The printed intervals are narrow enough to sit almost inside their markers. All values come from Table 2 or from recompute.py.txt.
Figure 2. The four rows of Table 2 with an empty cell (B4, M8&Z, Y and D), on one logarithmic axis with the null value 1 marked. For each row, the top orange marker is the printed "OR" with its 95% CI, the blue diamond is the odds ratio with 0.5 added to each cell with its Woolf CI, and the hollow blue square is the ratio of proportions that matches the printed value. In every row the printed value and interval coincide with the ratio of proportions, not with any odds ratio, and the printed intervals are one to two orders of magnitude narrower than the counts allow. For M8&Z and Y the printed value lies above 1 although no BA patient carries the haplogroup.
Why it matters
B4 is the paper's headline genetic result. It appears in the title, the Abstract, the Author Summary, the Discussion and the Conclusion, and it is the reason the in vitro work compares a B4 cybrid with an E cybrid. The printed crude OR of 0.71 describes a 29% reduction in odds with a tight interval (0.66–0.77). The counts show something different in both size and certainty: no BA patient carried B4, so the data are compatible with anything from no carriers at all to an odds ratio of about 0.45 (exact CI), or 0.003 to 0.88 with a 0.5 correction. A reader using 0.71 (0.66–0.77) in a meta-analysis or a power calculation would carry over an effect measure of the wrong kind with a standard error about 36 times too small on the log scale.
The adjusted estimate in the Abstract, OR 0.82 (0.71–0.85), p = 0.007, is affected more seriously. With no B4 carrier among the cases, the logistic-regression likelihood keeps increasing as the B4 odds ratio goes to 0 (the script prints the profile log-likelihood at 0.1, 0.01 and 0.001), and adding age and sex as covariates cannot change that. Standard software then reports either no estimate or a very large negative coefficient with a very large standard error, not a finite OR with a narrow interval. Table 1 also shows that the BA patients are aged 0.1–11 years and the adult controls 18–60 years, so age alone separates the cases from three-quarters of the controls; any age-adjusted model of these data is fragile for that reason too.
What survives: the direction of the B4 association and its crude significance (Fisher exact p = 0.0009, matching the printed 0.001). The E association (crude OR 11.88) recomputes correctly, as do the other 13 rows without a zero cell. The cybrid experiments do not depend on Table 2's effect sizes. The finding is therefore a misreported key effect estimate and an adjusted estimate that cannot exist as stated, not an overturned conclusion. The errors in the M8&Z, Y and D rows matter less on their own, but the M8&Z and Y values point the wrong way (above 1 with no exposed cases), and the D interval contradicts its own p-value.
Explanations considered
- A different control group. If B4 had been compared against only one control group, the counts would differ. Table 1 gives 5 + 18 = 23 carriers among the 200 pooled controls, matching Table 2, and the Methods say the two control groups were "put together for statistical calculation". No subset of the controls gives an OR near 0.71 with 0 exposed cases, since any odds ratio with 0 exposed cases is 0.
- A zero-cell correction or an exact method. Adding 0.5 to each cell gives 0.053 (0.003–0.881); the exact conditional estimate is 0 with CI 0–0.45; the Firth estimate equals the 0.5-corrected one. None comes near 0.71, and none can give a CI as narrow as 0.66–0.77 from a table with an empty cell.
- A text-conversion or typesetting error. The publisher's PDF of the version of record prints the same Table 2 values as PMC and Europe PMC, cell for cell, and the Results text repeats 0.71 (0.66–0.77) independently of the table. The values are not a conversion artefact.
- A copying slip from the neighbouring row F, whose correct OR is also 0.71. Row F's CI is 0.36–1.41, so this would explain the B4 point estimate but not its interval. The ratio of proportions reproduces both the point estimate and both CI limits for B4, and it also reproduces M8&Z, Y and D, so it accounts for all four rows where a copying slip accounts for one number.
- A ratio of proportions reported under the heading OR. This fits every zero-cell row exactly. SPSS Crosstabs, the package named in the Methods, prints such ratios as "For cohort" risk estimates next to the odds ratio. This explains the numbers but does not make them odds ratios: they answer a different question (the relative probability of being a control, or a case, by carrier status), their orientation differs between rows, and they are not the measure the text and table headings name.
- The adjusted model used individual data that differ from Table 2. I cannot rule this out without the data. But Tables 1 and 2 agree that no BA patient carries B4, and with that the adjusted B4 odds ratio has no finite maximum-likelihood value whatever the other covariates are.
- Row C's p-value. A p-value of 1.456 is not a probability; row Z, with the same counts, prints 0.456, which matches Fisher's test. This reads as a typographical error and does not change any conclusion.
Status
- Version of record. Table 2 in the publisher's PDF (journals.plos.org, 11 pages, Table 2 on page 3) matches the PMC and Europe PMC full text for every row cited here. The PLOS HTML Abstract and Results give the same B4 and E numbers.
- Supplements. The article has two supplements: Figure S1 (characterisation of the ρ0 cells and the B4 and E cybrids) and Table S1 (full mitochondrial sequences of haplogroups B4 and E). Neither holds case-control counts or regression output, and no individual-level data are available.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI; PMC and Europe PMC list no correction or comment links (PMID 23966875); the PLOS article page shows no correction notice and no reader comments. The PubPeer API refused automated access (HTTP 403), so PubPeer could not be checked directly; a web search for the article with "correction", "erratum" and "PubPeer" found no thread or notice for this paper.
- Sources. PMC article PMC3744426; publisher page and PDF at journals.plos.org/plosgenetics (doi:10.1371/journal.pgen.1003696); Europe PMC record and full-text XML; Crossref; PubPeer (blocked); web search.
Suggested correction
- Replace the B4 effect estimate in Table 2 and the Results with an estimate that suits a zero cell, for example the exact conditional OR (0; 95% CI 0–0.45) or a 0.5-corrected or Firth OR (0.053; 95% CI 0.003–0.88), and state the method. Alternatively, report the carrier proportions (0/71 vs 23/200) with Fisher's exact p = 0.0009.
- Withdraw or replace the adjusted B4 estimate in the Abstract and Results (OR 0.82, 0.71–0.85, p = 0.007). If an adjusted analysis is kept, use a method that handles separation (Firth-penalised or exact logistic regression) and report its estimate and interval. Recheck the adjusted E estimate (7.36, 1.22–94.6), whose interval is not centred on it on the log scale.
- Correct the OR and CI columns of Table 2 for rows M8&Z, Y and D in the same way, and correct the p-value for row C from 1.456 to 0.456.
- State in the Methods which test produced each p-value in Table 2 (Fisher's exact test or Pearson's chi-square).
Recomputation script (python recompute.py.txt)
"""Recompute the Table 2 effect estimates of PMC3744426
(PLoS Genet 2013, doi:10.1371/journal.pgen.1003696).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png, fig2.svg/png and five candidate figure designs (candidates/)
next to this file.
Inputs are copied from the article as archived in PMC (identical to the publisher PDF):
Table 1: B4 carriers by group (BA 0; child controls 5; adult controls 18)
Table 2: BA positive / BA negative / control positive / control negative, OR, 95% CI, p
Results text and Abstract: adjusted (age, sex) ORs for B4 and E
"""
import os
import numpy as np
from scipy.stats import chi2_contingency, fisher_exact
from scipy.stats.contingency import odds_ratio
from scipy.special import expit
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z = 1.959964
# ------------------------------------------------------------------ inputs (Table 2)
# name: (BA+, BA-, Ctl+, Ctl-, reported OR, reported CI low, high, reported p)
T2 = {
"M10": (2, 69, 4, 196, 1.42, 0.25, 7.93, 0.654),
"M7": (11, 60, 32, 168, 0.96, 0.45, 2.03, 1.000),
"M8": (5, 66, 10, 190, 1.44, 0.48, 4.37, 0.549),
"E": (4, 67, 1, 199, 11.88, 1.31, 108.16, 0.018),
"G": (1, 70, 10, 190, 0.27, 0.03, 2.16, 0.298),
"M9": (4, 67, 4, 196, 2.93, 0.71, 12.02, 0.213),
"M8&Z": (0, 71, 5, 195, 1.36, 1.27, 1.47, 0.331),
"Z": (1, 70, 1, 199, 2.84, 0.18, 46.06, 0.456),
"C": (1, 70, 1, 199, 2.84, 0.18, 46.07, 1.456),
"B4": (0, 71, 23, 177, 0.71, 0.66, 0.77, 0.001),
"B5": (2, 69, 14, 186, 0.39, 0.09, 1.74, 0.253),
"F": (13, 58, 48, 152, 0.71, 0.36, 1.41, 0.324),
"A": (4, 67, 13, 187, 0.86, 0.27, 2.73, 1.000),
"Y": (0, 71, 1, 199, 1.36, 1.26, 1.46, 1.000),
"D": (2, 69, 0, 200, 3.90, 3.18, 4.78, 0.068),
"D4": (6, 65, 24, 176, 0.68, 0.27, 1.73, 0.512),
"D5": (4, 67, 18, 182, 0.60, 0.20, 1.85, 0.372),
"N9": (2, 69, 7, 193, 0.80, 0.16, 3.94, 1.000),
}
TABLE1_B4 = {"BA": 0, "control children": 5, "control adults": 18}
N_BA, N_CTL = 71, 200
ADJ = {"B4": (0.82, 0.71, 0.85, 0.007), "E": (7.36, 1.22, 94.6, 0.032)} # Abstract / Results
# ------------------------------------------------------------------ helpers
def woolf(a, b, c, d):
"""Sample OR ad/bc with Woolf (log-Wald) CI; None if a cell is zero."""
if 0 in (a, b, c, d):
return None
o = a * d / (b * c)
s = np.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
return o, o * np.exp(-Z * s), o * np.exp(Z * s)
def haldane(a, b, c, d):
return woolf(a + .5, b + .5, c + .5, d + .5)
def ratio_of_proportions(x1, n1, x2, n2):
"""(x1/n1)/(x2/n2) with Katz log CI."""
r = (x1 / n1) / (x2 / n2)
s = np.sqrt(1 / x1 - 1 / n1 + 1 / x2 - 1 / n2)
return r, r * np.exp(-Z * s), r * np.exp(Z * s)
def firth_or(a, b, c, d, iters=200):
"""Firth-penalised logistic regression of BA on carrier status (individual rows)."""
x = np.r_[np.ones(a), np.zeros(b), np.ones(c), np.zeros(d)]
y = np.r_[np.ones(a + b), np.zeros(c + d)]
X = np.c_[np.ones_like(x), x]
beta = np.zeros(2)
for _ in range(iters):
p = expit(X @ beta)
W = p * (1 - p)
info = X.T @ (X * W[:, None])
inv = np.linalg.inv(info)
h = W * np.einsum("ij,jk,ik->i", X, inv, X)
step = inv @ (X.T @ (y - p + h * (0.5 - p)))
beta = beta + step
if np.max(np.abs(step)) < 1e-12:
break
return float(np.exp(beta[1]))
def fmt(t, k=2):
return f"{t[0]:.{k}f} ({t[1]:.{k}f}–{t[2]:.{k}f})"
# ------------------------------------------------------------------ 1. B4 consistency with Table 1
print("== 1. B4 counts")
print("Table 1 B4 carriers:", TABLE1_B4, "-> controls", 5 + 18, "of", N_CTL,
"; BA", 0, "of", N_BA, "; B4-negative controls", N_CTL - 23)
# ------------------------------------------------------------------ 2. B4 odds ratio, recomputed
a, b, c, d = T2["B4"][:4]
print("\n== 2. B4 (BA 0/71 vs controls 23/200)")
print("sample OR (0*177)/(71*23) =", a * d / (b * c))
cond = odds_ratio([[a, b], [c, d]])
ci = cond.confidence_interval()
print(f"exact conditional MLE OR = {cond.statistic:.3f}, exact 95% CI {ci.low:.3f}–{ci.high:.3f}")
B4_HAL = haldane(a, b, c, d)
print("Haldane-Anscombe (+0.5) OR, Woolf CI =", fmt(B4_HAL, 3))
print(f"Firth-penalised logistic OR (no covariates) = {firth_or(a, b, c, d):.4f} (equals the +0.5 OR)")
print(f"Fisher exact two-sided p = {fisher_exact([[a, b], [c, d]])[1]:.4f} (reported 0.001)")
print(f"Pearson chi-square p (no correction) = {chi2_contingency([[a, b], [c, d]], correction=False)[1]:.4f}; "
f"Yates p = {chi2_contingency([[a, b], [c, d]], correction=True)[1]:.4f}")
B4_RR = ratio_of_proportions(d, b + d, c, a + c) # P(control | B4-neg) / P(control | B4-pos)
print("P(control | B4-negative) / P(control | B4-positive) = (177/248)/(23/23) =", fmt(B4_RR, 3),
" <- reported 'OR' 0.71 (0.66–0.77)")
print("width of reported CI on log scale:", round(np.log(0.77 / 0.66), 3),
"; width of +0.5 OR CI:", round(np.log(B4_HAL[2] / B4_HAL[1]), 3),
"; ratio:", round(np.log(B4_HAL[2] / B4_HAL[1]) / np.log(0.77 / 0.66), 1))
print(f"+0.5 OR to 4 dp = {B4_HAL[0]:.4f}")
# ------------------------------------------------------------------ 3. adjusted (age, sex) estimates
print("\n== 3. adjusted estimates (Abstract, Results)")
for k, (o, lo, hi, p) in ADJ.items():
print(f"{k}: reported OR {o} ({lo}–{hi}); geometric centre of CI = {np.sqrt(lo * hi):.3f}")
print("B4: every B4-positive participant is a control, so the B4 coefficient in any logistic model "
"of BA is quasi-completely separated; the likelihood keeps rising as OR -> 0, whatever "
"covariates are added. Profile of the log-likelihood (no covariates) for OR = 0.1, 0.01, 0.001:")
for orv in (0.1, 0.01, 0.001):
# intercept for B4-negative fixed at its MLE log(71/177); B4-positive group: 23 controls, 0 cases
p_pos = expit(np.log(71 / 177) + np.log(orv))
ll = 71 * np.log(71 / 248) + 177 * np.log(177 / 248) + 23 * np.log(1 - p_pos)
print(f" OR={orv:<6} loglik={ll:.4f}")
print("Table 1 age ranges: BA 0.1–11 y; child controls 0.3–16 y; adult controls 18–60 y")
print("adult controls (all older than every BA patient):", 148, "of", N_CTL, "=", f"{148 / N_CTL:.0%}")
# ------------------------------------------------------------------ 4. every Table 2 row
print("\n== 4. all Table 2 rows: reported vs recomputed")
print(f"{'row':5s} {'reported OR (CI)':22s} {'Woolf OR (CI)':22s} {'+0.5 OR (CI)':24s} "
f"{'rep p':>6s} {'Fisher':>7s} {'Pearson':>7s}")
REC = {}
for k, (a, b, c, d, ro, rl, rh, rp) in T2.items():
w = woolf(a, b, c, d)
h = haldane(a, b, c, d)
pf = fisher_exact([[a, b], [c, d]])[1]
pp = chi2_contingency([[a, b], [c, d]], correction=False)[1] if min(a, b, c, d) >= 0 else np.nan
REC[k] = dict(woolf=w, hal=h, fisher=pf, pearson=pp)
ws = fmt(w) if w else "0 or inf (zero cell)"
print(f"{k:5s} {ro:.2f} ({rl:.2f}–{rh:.2f}){'':5s} {ws:22s} {fmt(h, 3):24s} "
f"{rp:6.3f} {pf:7.4f} {pp:7.4f}")
print("\nzero-cell rows: the printed value equals a ratio of proportions")
ZERO = {
"B4": ("P(control | negative) / P(control | positive)", ratio_of_proportions(177, 248, 23, 23)),
"M8&Z": ("P(control | positive) / P(control | negative)", ratio_of_proportions(5, 5, 195, 266)),
"Y": ("P(control | positive) / P(control | negative)", ratio_of_proportions(1, 1, 199, 270)),
"D": ("P(BA | positive) / P(BA | negative)", ratio_of_proportions(2, 2, 69, 269)),
}
for k, (lab, r) in ZERO.items():
ro, rl, rh = T2[k][4:7]
print(f"{k:5s} reported {ro:.2f} ({rl:.2f}–{rh:.2f}) {lab} = {fmt(r, 3)} "
f"+0.5 OR = {fmt(REC[k]['hal'], 3)}")
print("M8&Z opposite orientation: P(control|neg)/P(control|pos) =",
fmt(ratio_of_proportions(195, 266, 5, 5), 3))
print("D: reported CI 3.18–4.78 excludes 1, reported p = 0.068; Fisher p =", round(REC["D"]["fisher"], 4))
print("C and Z have identical counts (1/70 vs 1/199): reported p 1.456 and 0.456; Fisher p =",
round(REC["C"]["fisher"], 4))
print("F and D5 reported p match Pearson chi-square without correction:",
round(REC["F"]["pearson"], 4), round(REC["D5"]["pearson"], 4))
print("E: Woolf OR", fmt(REC["E"]["woolf"]), "Fisher p", round(REC["E"]["fisher"], 4))
print("F row OR (13*152)/(58*48) =", round(13 * 152 / (58 * 48), 3), "with CI", fmt(REC["F"]["woolf"]))
nonzero_ok = [k for k in T2 if REC[k]["woolf"] and abs(REC[k]["woolf"][0] - T2[k][4]) < 0.006]
print("rows without a zero cell whose OR reproduces to 2 dp:", len(nonzero_ok), "of",
sum(1 for k in T2 if REC[k]["woolf"]))
# ================================================================== figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK, "axes.edgecolor": INK2, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def clean(ax):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
# ---- candidate A: B4 dot-and-interval, all estimates on one log axis
def design_a(path):
XMIN, XMAX = 0.001, 3
rows = [
("Table 2, Results:\n“OR” 0.71 (0.66–0.77)", 0.71, 0.66, 0.77, "rep"),
("Abstract, Results (adjusted):\n“OR” 0.82 (0.71–0.85)", 0.82, 0.71, 0.85, "rep"),
(f"Odds ratio (0×177)/(71×23) = 0,\nexact 95% CI 0–{ci.high:.2f}",
None, None, ci.high, "rec0"),
("Odds ratio, 0.5 added to cells:\n" + fmt(B4_HAL, 3), *B4_HAL, "rec"),
("Share of controls, B4− / B4+:\n" + fmt(B4_RR, 3),
*B4_RR, "rec"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ys = np.arange(len(rows))[::-1]
for y, (lab, o, lo, hi, kind) in zip(ys, rows):
col = ORANGE if kind == "rep" else BLUE
if kind == "rec0":
ax.annotate("", xy=(XMIN, y), xytext=(hi, y),
arrowprops=dict(arrowstyle="-|>", color=col, lw=2, mutation_scale=14))
ax.plot([hi, hi], [y - .12, y + .12], color=col, lw=2)
ax.text(XMIN * 1.15, y + 0.2, "estimate 0 (off the log axis)", color=INK2, fontsize=10,
va="bottom")
continue
ax.plot([lo, hi], [y, y], color=col, lw=2, zorder=3)
for e in (lo, hi):
ax.plot([e, e], [y - .14, y + .14], color=col, lw=2, zorder=3)
mk = "o" if kind == "rep" else "D"
ax.plot(o, y, mk, color=col, ms=9, mec=BG, mew=1.5, zorder=4)
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.text(0.93, ys[0] + 0.5, "no association", color=INK2, fontsize=10, va="center", ha="right")
ax.set_xscale("log")
ax.set_xlim(XMIN, XMAX)
ax.set_ylim(-0.6, len(rows) - 0.3)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xticks([0.001, 0.01, 0.1, 0.5, 1, 2])
ax.set_xticklabels(["0.001", "0.01", "0.1", "0.5", "1", "2"])
ax.set_xlabel("Ratio, log scale (B4: 0/71 BA vs 23/200 controls)")
clean(ax)
ax.plot([], [], "o-", color=ORANGE, ms=8, label="reported in the paper")
ax.plot([], [], "D-", color=BLUE, ms=8, label="recomputed from Table 2 counts")
fig.legend(loc="upper left", bbox_to_anchor=(0.01, 0.99), ncol=2, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.36, right=0.97, top=0.88, bottom=0.14)
save(fig, path)
# ---- candidate B: Cleveland dot plot over every Table 2 row
def design_b(path):
order = sorted(T2, key=lambda k: T2[k][4])
fig, ax = plt.subplots(figsize=(7.2, 5.6))
for i, k in enumerate(order):
ro = T2[k][4]
w, h = REC[k]["woolf"], REC[k]["hal"]
zero = w is None
rec = h[0] if zero else w[0]
ax.plot([min(ro, rec), max(ro, rec)], [i, i], color=GRID if not zero else INK2,
lw=2, zorder=2)
ax.plot(ro, i, "o", color=ORANGE, ms=11, mec=BG, mew=1.5, zorder=4)
if zero:
ax.plot(rec, i, "D", color=BLUE, ms=8, mfc=BG, mew=2, zorder=5)
ax.text(max(ro, rec) * 1.25, i, "zero cell", color=INK2, fontsize=10, va="center")
else:
ax.plot(rec, i, "D", color=BLUE, ms=5.5, zorder=6)
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.03, 40)
ax.set_yticks(range(len(order)))
ax.set_yticklabels(order)
ax.set_ylim(-0.7, len(order) - 0.3)
ax.set_xticks([0.05, 0.1, 0.5, 1, 2, 5, 10, 20])
ax.set_xticklabels(["0.05", "0.1", "0.5", "1", "2", "5", "10", "20"])
ax.set_xlabel("Odds ratio, BA vs controls (log scale; 1 = no association)")
clean(ax)
ax.plot([], [], "o", color=ORANGE, ms=11, label="“OR” printed in Table 2")
ax.plot([], [], "D", color=BLUE, ms=5.5, label="OR recomputed from the row's counts (inside the orange dot when they agree)")
ax.plot([], [], "D", color=BLUE, ms=8, mfc=BG, mew=2,
label="zero-cell row: OR with 0.5 added to each cell")
ax.legend(loc="lower left", bbox_to_anchor=(0, 1.0), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
# ---- shared box helpers for the diagram designs
def box(ax, x, y, w, h, text, edge, fs=10.5, ha="left", weight="normal", fill=BG):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.012,rounding_size=0.015",
fc=fill, ec=edge, lw=1.6))
tx = x + 0.02 if ha == "left" else x + w / 2
ax.text(tx, y + h / 2, text, ha=ha, va="center", fontsize=fs, color=INK, weight=weight,
linespacing=1.45, family="DejaVu Sans")
def arrow(ax, p, q, col=INK2, text=None, toff=(0, 0.03)):
ax.add_patch(FancyArrowPatch(p, q, arrowstyle="-|>", mutation_scale=14, color=col, lw=1.6))
if text:
ax.text((p[0] + q[0]) / 2 + toff[0], (p[1] + q[1]) / 2 + toff[1], text, ha="center",
va="bottom", fontsize=10, color=INK2)
def canvas(h):
fig = plt.figure(figsize=(7.2, h))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
return fig, ax
# ---- candidate C: calculation-flow diagram for the B4 row
def design_c(path):
fig, ax = canvas(4.4)
ax.text(0.03, 0.95, "Haplogroup B4, Table 2 counts", fontsize=11, weight="bold", va="center")
box(ax, 0.03, 0.60, 0.30, 0.28,
" B4+ B4−\nBA 0 71\ncontrols 23 177",
INK2, fs=10.5)
box(ax, 0.40, 0.66, 0.57, 0.22,
f"Odds ratio = (0 × 177) / (71 × 23) = 0\nexact 95% CI 0–{ci.high:.2f}\n"
f"with 0.5 added to each cell: {fmt(B4_HAL, 3)}", BLUE, fs=10.5)
box(ax, 0.40, 0.36, 0.57, 0.22,
"Proportion of controls, B4− / B4+\n"
"= (177/248) / (23/23) = " + fmt(B4_RR, 3), BLUE)
arrow(ax, (0.335, 0.77), (0.395, 0.77))
arrow(ax, (0.335, 0.66), (0.395, 0.47))
box(ax, 0.40, 0.05, 0.57, 0.20,
"Printed in Table 2 and Results as the odds ratio:\n“OR” 0.71, 95% CI 0.66–0.77, p = 0.001",
ORANGE)
ax.add_patch(FancyArrowPatch((0.695, 0.355), (0.695, 0.255), arrowstyle="<|-|>",
mutation_scale=14, color=INK2, lw=1.6))
ax.text(0.71, 0.305, "same value to two decimals", fontsize=10, color=INK2, va="center")
ax.text(0.03, 0.45, "blue = recomputed from\nthe paper's counts\n\norange = reported\nin the paper",
fontsize=10, color=INK2, va="top")
save(fig, path)
# ---- candidate D: text panels, Table 2 rows vs what the counts give
def design_d(path):
fig, ax = canvas(4.6)
ax.text(0.03, 0.955, "Table 2 (as published)", fontsize=11, weight="bold", va="center",
color=INK)
ax.text(0.46, 0.955, "Recomputed from the same counts", fontsize=11, weight="bold",
va="center", color=INK)
lines = [
("B4", "0/71 vs 23/200", "“OR” 0.71 (0.66–0.77)",
"OR 0; ratio of proportions 0.714 (0.66–0.77)"),
("M8&Z", "0/71 vs 5/200", "“OR” 1.36 (1.27–1.47)",
"OR 0; ratio of proportions 1.364 (1.27–1.47)"),
("Y", "0/71 vs 1/200", "“OR” 1.36 (1.26–1.46)",
"OR 0; ratio of proportions 1.357 (1.26–1.46)"),
("D", "2/71 vs 0/200", "“OR” 3.90 (3.18–4.78)",
"OR infinite; ratio of proportions 3.899 (3.18–4.78)"),
("C", "1/71 vs 1/200", "p = 1.456",
"Fisher p = %.3f (row Z, same counts: 0.456)" % REC["C"]["fisher"]),
]
top = 0.86
hgt = 0.135
for i, (k, cnt, rep, rec) in enumerate(lines):
y = top - i * (hgt + 0.03) - hgt
box(ax, 0.03, y, 0.33, hgt, f"{k} {cnt}\n{rep}", ORANGE, fs=10.5)
box(ax, 0.46, y, 0.51, hgt, rec.replace("; ", ";\n").replace(" (row Z", "\n(row Z"),
BLUE, fs=10.5)
arrow(ax, (0.365, y + hgt / 2), (0.455, y + hgt / 2))
ax.text(0.03, 0.025, "Counts: carriers among 71 BA patients vs among 200 controls.",
fontsize=10, color=INK2, ha="left")
save(fig, path)
# ---- candidate E: intervals for every zero-cell row on one log axis
def design_e(path):
rows = ["B4", "M8&Z", "Y", "D"]
fig, ax = plt.subplots(figsize=(7.2, 4.6))
yt, yl = [], []
for i, k in enumerate(rows):
base = (len(rows) - 1 - i) * 3.4
ro, rl, rh = T2[k][4:7]
h = REC[k]["hal"]
r = ZERO[k][1]
trio = [(base + 1.0, ro, rl, rh, ORANGE, "o", BLUE),
(base + 0.0, h[0], h[1], h[2], BLUE, "D", BLUE),
(base - 1.0, r[0], r[1], r[2], BLUE, "s", BLUE)]
for y, o, lo, hi, col, mk, _ in trio:
ax.plot([lo, hi], [y, y], color=col, lw=2, zorder=3)
ax.plot(o, y, mk, color=col, ms=8, mec=BG, mew=1.5, zorder=4,
mfc=col if mk != "s" else BG)
if mk == "s":
ax.plot(o, y, mk, color=col, ms=8, mfc=BG, mew=2, zorder=4)
a_, b_, c_, d_ = T2[k][:4]
yt.append(base)
yl.append(f"{k}\n{a_}/71 vs {c_}/200")
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.002, 400)
ax.set_yticks(yt)
ax.set_yticklabels(yl, fontsize=10)
ax.set_ylim(-1.8, (len(rows) - 1) * 3.4 + 1.8)
ax.set_xticks([0.01, 0.1, 1, 10, 100])
ax.set_xticklabels(["0.01", "0.1", "1", "10", "100"])
ax.set_xlabel("Ratio, BA vs controls (log scale; 1 = no association)")
clean(ax)
ax.plot([], [], "o-", color=ORANGE, ms=8, label="“OR” (95% CI) printed in Table 2")
ax.plot([], [], "D-", color=BLUE, ms=8, label="OR with 0.5 added to each cell")
ax.plot([], [], "s-", color=BLUE, ms=8, mfc=BG, mew=2,
label="ratio of proportions matching the printed value")
ax.legend(loc="lower left", bbox_to_anchor=(0, 1.0), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
design_a(os.path.join(CAND, "A_b4_intervals"))
design_b(os.path.join(CAND, "B_all_rows_dotplot"))
design_c(os.path.join(CAND, "C_b4_calculation_flow"))
design_d(os.path.join(CAND, "D_zero_cell_text_panels"))
design_e(os.path.join(CAND, "E_zero_cell_intervals"))
# provisional choice for the write-up: A as fig1, E as fig2
design_a(os.path.join(HERE, "fig1"))
design_e(os.path.join(HERE, "fig2"))
print("\nfigures written to", HERE)
Abstract gives the earlier cohort's 70% accuracy as this trial's result; recomputed value is 63%
PLoS Med (2019) · doi:10.1371/journal.pmed.1002813 · PMC6546206
Summary
The abstract of this multi-institutional validation trial (MULTI-IMPROD, 338 men analysed) reports that "the accuracy and NPV of IMPROD bpMRI for SPCa were 70% (113/161) and 95% (71/75)". The NPV of 71/75 belongs to this trial. The accuracy of 113/161, however, is the figure printed in Table 3 for the earlier single-centre IMPROD trial (161 men). From the paper's own Table 2 counts, accuracy in the 338-man trial is (142 + 71)/338 = 213/338 = 63.0% (Wilson 95% CI 57.8%–68.0%). Table 3 prints this as 63%, but with the count 217/338, which is 64.2%. Overall accuracy is part of the trial's stated primary outcome, so the abstract overstates a primary-outcome result by 7.2 percentage points, by a value outside its 95% CI. It also hides the fall in accuracy between the earlier trial and the validation. The paper's headline conclusion, a high NPV, does not depend on this figure and stands.
What the paper reports
Abstract, Methods and findings (the primary outcome and the cohort):
The primary outcome of the trial was diagnostic accuracy—including overall accuracy, sensitivity, specificity, negative predictive value (NPV), and positive predictive value—of IMPROD bpMRI for clinically significant PCa (SPCa), which was defined as a Gleason score ≥ 3 + 4 (Gleason grade group 2 or higher). In total, 338 (338/364, 93%) prospectively enrolled men completed the trial.
Abstract, Methods and findings, the next sentence:
The accuracy and NPV of IMPROD bpMRI for SPCa were 70% (113/161) and 95% (71/75) (95% CI 87%–98%), respectively.
Methods, "Study end points":
The primary outcome of the trial was diagnostic accuracy—including overall accuracy, sensitivity, specificity, NPV, and positive predictive value—of IMPROD bpMRI for SPCa, which was defined as a Gleason score ≥ 3 + 4.
Table 3, title and rows (columns: Measure | MULTI-IMPROD trial | IMPROD trial):
Comparison between the current trial (MULTI-IMPROD trial) and the pre-validation cohort (IMPROD trial).
Patient cohort in total | 338 | 161
Sensitivity* | 97% (142/146) [93%–99%] | 95% (79/83) [88%–98%]
Specificity* | 37% (71/192) [31%–44%] | 44% (34/78) [33%–55%]
NPV* | 95% (71/75) [87%–98%] | 90% (34/38) [76%–96%]
PPV* | 54% (141/263) [48%–60%] | 64% (79/123) [55%–72%]
Accuracy* | 63% (217/338) | 70% (113/161)
Table 3, footnote:
*Sensitivity, specificity, NPV, PPV, and accuracy values are based on binary classification (IMPROD bpMRI Likert score 1–2 versus 3–5) for predicting prostate cancer with Gleason score ≥ 3 + 4.
Table 2, title (the table gives men per Likert score 1–5 by biopsy outcome; its cells are used below):
Prevalence of prostate cancer and clinically significant prostate cancer in different IMPROD bpMRI Likert score groups based on a cohort of 338 men.
Results, "Diagnostic accuracy", first paragraph (relevant to a secondary finding below):
SPCa was diagnosed in 5% (2/34), 5% (2/41), 12% (8/66), 39% (23/59), and 88% (121/138) of men with IMPROD bpMRI Likert scores of 1, 2, 3, 4, and 5, respectively (Table 2).
What the paper's own numbers show
Method. Table 2 gives, for each Likert score from 1 to 5, the number of men with no cancer, Gleason 3+3, Gleason 3+4 and Gleason > 3+4. I checked that each column sums to its printed total (338 in all). Using the Table 3 footnote's definitions (test positive = Likert 3–5; reference positive = SPCa, Gleason ≥ 3+4), I built the 2 × 2 table for the 338 men. From it I computed sensitivity, specificity, NPV, PPV and overall accuracy, (TP + TN)/N, each with a Wilson 95% CI, and compared them with Table 3 and the abstract. I did the same for the earlier IMPROD trial from its Table 3 counts. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| 2 × 2 cells, this trial | Table 2 | (cell counts by Likert score) | TP = 8 + 23 + 111 = 142, FN = 2 + 2 = 4, FP = 263 − 142 = 121, TN = 75 − 4 = 71; N = 338 |
| Sensitivity, this trial | Table 3 | 97% (142/146) [93%–99%] | 142/146 = 97.3% [93.2–98.9]; matches |
| Specificity, this trial | Table 3 | 37% (71/192) [31%–44%] | 71/192 = 37.0% [30.5–44.0]; matches (lower bound differs by under one point) |
| NPV, this trial | Table 3; Abstract | 95% (71/75) [87%–98%] | 71/75 = 94.7% [87.1–97.9]; matches |
| PPV, this trial | Table 3 | 54% (141/263) [48%–60%] | 142/263 = 54.0% [48.0–59.9]; percentage matches, numerator printed one too low (141/263 = 53.6%) |
| Overall accuracy, this trial | Table 3 | 63% (217/338) | (142 + 71)/338 = 213/338 = 63.0% [57.8–68.0]; percentage matches, but 217/338 = 64.2% |
| Overall accuracy, as given in the abstract | Abstract | 70% (113/161) | 113/161 = 70.2%. Denominator 161 is not this trial's N (338); 70.2% lies outside the 57.8–68.0% CI of the recomputed value |
| Overall accuracy, earlier IMPROD trial | Table 3, IMPROD column | 70% (113/161) | (79 + 34)/161 = 113/161 = 70.2% [62.7–76.7]; identical to the abstract figure |
| NPV, earlier IMPROD trial | Table 3, IMPROD column | 90% (34/38) | 34/38 = 89.5% |
| Gap between abstract accuracy and this trial's accuracy | derived | — | 70.2% − 63.0% = 7.2 percentage points |
| SPCa in Likert 5 (secondary finding) | Results, "Diagnostic accuracy" | 88% (121/138) | Table 2: 37 + 74 = 111/138 = 80.4%. With 121 the per-score SPCa counts would total 156, not the 146 used throughout |
The abstract's single sentence therefore combines two cohorts: its NPV (71/75) comes from the 338 men of this trial, and its accuracy (113/161) from the 161 men of the earlier trial. Accuracy for this trial, from its own cells, is 63.0%, which is the percentage Table 3 prints.
Figure
Figure 1. Where each number in the abstract sentence comes from. Top: the two consecutive abstract sentences, quoted verbatim; the accuracy figure (orange outline, as reported) and the NPV figure (blue outline) are boxed. Bottom: the two columns of Table 3, with each printed value and the value recomputed from that column's own counts (TP + TN over the cohort N for accuracy; TN over test negatives for NPV). The abstract's accuracy, 70% (113/161), is the earlier single-centre cohort's value; its NPV, 95% (71/75), is this trial's. The recomputed accuracy for this trial's 338 men is 213/338 = 63.0%. All numbers are from the paper's Abstract, Table 2 and Table 3, or from recompute.py.txt.
Figure 2. Overall accuracy for SPCa on one 0–100% axis. The hollow orange dot is the abstract's reported value (70%; 113/161 = 70.2%). The blue dot and line are the accuracy recomputed for this trial's 338 men from Table 2 (213/338 = 63.0%, Wilson 95% CI 57.8–68.0%, also shaded). The gray dot and line are the earlier IMPROD trial (113/161 = 70.2%, 62.7–76.7%), shown for context: the abstract value coincides with it and falls outside the CI for this trial.
Why it matters
- It is a primary-outcome figure in the abstract. The abstract and Methods both name overall accuracy as part of the primary outcome. The abstract gives 70% for this trial; the trial's own data give 63.0%. The abstract is the part of the paper most readers, indexers and guideline reviewers see.
- It hides the change between cohorts. Read correctly, accuracy fell from 70.2% in the earlier single-centre trial to 63.0% in the multi-institutional validation, in line with the falls in specificity (43.6% to 37.0%) and PPV (64.2% to 54.0%) shown in Table 3. The aim stated in the abstract is to validate the protocol in a multi-institutional setting, so this change is what a reader of a validation study needs to see. Quoting the earlier figure as the result makes the two cohorts look identical on this measure.
- What is not affected. The abstract's conclusion rests on the NPV, "IMPROD bpMRI demonstrated a high NPV for SPCa", and that value (95%, 71/75, CI 87%–98%) is reported correctly. So are the 97% sensitivity and the 22% (75/338) reduction in biopsies with 4/146 SPCa missed. The direction of the findings does not change; the magnitude of overall accuracy does.
- Smaller count errors. Table 3's accuracy count (217/338) and PPV count (141/263) do not match the 2 × 2 cells (213 and 142); the printed percentages are right. The Results text gives SPCa in Likert 5 as 88% (121/138), whereas Table 2 gives 111/138 = 80.4%. This last figure enters the description of how SPCa prevalence rises with Likert score, which readers use when weighing whether to biopsy a Likert 5 finding.
Explanations considered
- The abstract deliberately quotes the earlier cohort. Does not account for it. The preceding sentence introduces "338 (338/364, 93%) prospectively enrolled men", and the same sentence pairs the accuracy with this trial's NPV (71/75) using "respectively". Nothing in the abstract tells the reader that 113/161 comes from the earlier IMPROD trial (NCT01864135).
- A different definition, threshold or subgroup. Does not account for it. The Table 3 footnote states that all five metrics use the same cut (Likert 1–2 versus 3–5) and the same reference (Gleason ≥ 3+4). No subgroup of the 338 men with 161 members is reported; 161 is the total of the earlier cohort in Table 3. The 2 × 2 cells rebuilt from Table 2 reproduce Table 3's sensitivity, specificity and NPV and their CIs, and give an accuracy of 213/338 = 63.0%.
- Alternative definitions of SPCa in the supplements. Does not account for it. S1 Table and S2 Table (two alternative SPCa definitions) give MULTI-IMPROD accuracies of 62% (210/338) and 48% (163/338). In both, accuracy equals TP + TN over 338, and neither is 70% or uses 161.
- Rounding. Does not account for it. 70% against 63% is 7.2 percentage points.
- A text-conversion error in the PMC copy. Ruled out. The publisher's HTML and PDF print the same abstract sentence and the same Table 3 values; the PMC and Europe PMC XML carry each value in its own correctly aligned table cell.
- Is Table 3's 217/338 the true count, with 63% the slip? No. 217 is inconsistent with the table's own sensitivity and specificity numerators (142 + 71 = 213) and with 63% (217/338 = 64.2%). The percentage is right and the numerator is off by 4. Likewise the PPV numerator 141 should be 142 (TP), with no effect on the printed 54%.
- Likert 5 count of 121. Table 2 gives 37 + 74 = 111 men with SPCa among 138 at Likert 5, and only 111 makes the per-score counts sum to the 146 SPCa cases used throughout. The figure 121 does appear elsewhere in the paper as the number of SPCa cases detected by targeted biopsy (121/338 and 121/263), but that is a different quantity with different denominators, so it does not reconcile 121/138.
Status
- Version of record. The publisher's HTML (journals.plos.org/plosmedicine, article 10.1371/journal.pmed.1002813) and the publisher's PDF (abstract on page 2, Table 3 on page 8) print the abstract sentence and Table 3 exactly as quoted. The PMC and Europe PMC full-text XML match. Every quote above was checked verbatim against the PMC-archived text.
- Supplements. The paper lists three supplementary files: S1 CONSORT checklist, S1 Table and S2 Table (alternative SPCa definitions). S1 and S2 Table were downloaded from PLOS and parsed; they are discussed under explanation 3 and do not reconcile the abstract figure. The protocols.io record linked from Crossref (doi:10.17504/protocols.io.ynifvce) is the imaging protocol and holds no outcome data; it was not opened.
- Corrections and comments. As of 2026-10-23 no correction, erratum, expression of concern or retraction was found. Crossref lists no update relations for the DOI; PMC and Europe PMC list no correction notices (the PMC record links three related articles of type "commentary", none a correction); a Crossref search for a "Correction:" item with this title under the PLOS prefix found none; the PLOS article page shows no correction notice; a web search found only the original article. PubPeer could not be checked: its API and search page both returned an HTTP 403 (Cloudflare challenge) to automated access.
- Sources. PMC article PMC6546206; publisher page and PDF at journals.plos.org; S1 and S2 Table (PLOS supplementary files); Crossref; Europe PMC; ClinicalTrials.gov registrations NCT02241122 and NCT01864135 as cited in the paper.
Suggested correction
- Abstract, Methods and findings: replace "The accuracy and NPV of IMPROD bpMRI for SPCa were 70% (113/161) and 95% (71/75) (95% CI 87%–98%), respectively." with "The accuracy and NPV of IMPROD bpMRI for SPCa were 63% (213/338) and 95% (71/75) (95% CI 87%–98%), respectively." If a CI for accuracy is wanted, the Wilson 95% CI is 58%–68%. If the authors wish to compare with the earlier trial, say so explicitly (for example, "compared with 70% (113/161) in the pre-validation IMPROD trial").
- Table 3, MULTI-IMPROD column: change "Accuracy 63% (217/338)" to "63% (213/338)" and "PPV 54% (141/263)" to "54% (142/263)".
- Results, "Diagnostic accuracy", first paragraph: change "88% (121/138)" to "80% (111/138)" for Likert score 5, as in Table 2.
Recomputation script (python recompute.py.txt)
"""Recompute the diagnostic-accuracy figures of PMC6546206
(PLoS Med 2019, doi:10.1371/journal.pmed.1002813, the MULTI-IMPROD trial).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes, next to this file:
fig1.svg/png provenance diagram: which cohort each number in the abstract sentence comes from
fig2.svg/png accuracy: abstract value vs the value recomputed for the 338-man cohort, with 95% CIs
candidates/cand1..cand5.svg/png five alternative figure designs for review
(cand1 = fig1, cand2 = fig2)
Inputs are copied from the paper as archived in PMC:
Table 2: men per IMPROD bpMRI Likert score (1..5) by biopsy outcome, MULTI-IMPROD cohort (n = 338)
Table 3: diagnostic metrics for MULTI-IMPROD (n = 338) and the earlier IMPROD trial (n = 161)
Test positive = Likert 3-5; reference positive = Gleason score >= 3+4 (SPCa), as the Table 3 footnote states.
"""
import os
from math import sqrt
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from scipy.stats import beta # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ------------------------------------------------------------------ inputs (from the paper)
# Table 2, columns = Likert 1, 2, 3, 4, 5
NO_CANCER = [31, 33, 42, 19, 6]
GG1 = [1, 6, 16, 17, 21] # Gleason 3+3
GG2 = [2, 2, 4, 13, 37] # Gleason 3+4
GG3P = [0, 0, 4, 10, 74] # Gleason > 3+4
TOTAL = [34, 41, 66, 59, 138]
# Table 3, as printed: (numerator, denominator, printed %)
T3_MULTI = {"Sensitivity": (142, 146, 97), "Specificity": (71, 192, 37), "NPV": (71, 75, 95),
"PPV": (141, 263, 54), "Accuracy": (217, 338, 63)}
T3_IMPROD = {"Sensitivity": (79, 83, 95), "Specificity": (34, 78, 44), "NPV": (34, 38, 90),
"PPV": (79, 123, 64), "Accuracy": (113, 161, 70)}
T3_MULTI_CI = {"Sensitivity": (93, 99), "Specificity": (31, 44), "NPV": (87, 98), "PPV": (48, 60)}
# Abstract, Methods and findings
ABS_ACC = (113, 161, 70)
ABS_NPV = (71, 75, 95)
ABS_NPV_CI = (87, 98)
# Results, "Diagnostic accuracy", paragraph 1 (Likert 5)
TEXT_L5 = (121, 138, 88)
TB_SPCA = 121 # Results, "Comparison of TB with SB": TB detected SPCa in 121/338
# ------------------------------------------------------------------ helpers
Z = 1.959964
def wilson(a, n):
p = a / n
c = (p + Z * Z / (2 * n)) / (1 + Z * Z / n)
h = Z * sqrt(p * (1 - p) / n + Z * Z / (4 * n * n)) / (1 + Z * Z / n)
return c - h, c + h
def exact(a, n):
lo = beta.ppf(0.025, a, n - a + 1) if a > 0 else 0.0
hi = beta.ppf(0.975, a + 1, n - a) if a < n else 1.0
return lo, hi
def pct(a, n):
return 100.0 * a / n
def line(label, a, n, ci=True):
s = f" {label:<40s} {a:>4d}/{n:<4d} = {pct(a, n):5.1f}%"
if ci:
w = wilson(a, n)
e = exact(a, n)
s += f" Wilson [{100*w[0]:.1f}-{100*w[1]:.1f}] Clopper-Pearson [{100*e[0]:.1f}-{100*e[1]:.1f}]"
print(s)
# ------------------------------------------------------------------ 1. Table 2 consistency
print("1. Table 2 (MULTI-IMPROD, n = 338)")
for i in range(5):
assert NO_CANCER[i] + GG1[i] + GG2[i] + GG3P[i] == TOTAL[i]
N = sum(TOTAL)
SPCA = [GG2[i] + GG3P[i] for i in range(5)]
print(f" rows sum to column totals; N = {N}; SPCa per Likert 1..5 = {SPCA}, total {sum(SPCA)}")
for i in range(5):
line(f"SPCa, Likert {i+1}", SPCA[i], TOTAL[i], ci=False)
# ------------------------------------------------------------------ 2. 2x2 table, MULTI-IMPROD
TP = sum(SPCA[2:])
FN = sum(SPCA[:2])
FP = sum(TOTAL[2:]) - TP
TN = sum(TOTAL[:2]) - FN
assert TP + FN + FP + TN == N == 338
print(f"\n2. MULTI-IMPROD 2x2 (Likert 3-5 positive, SPCa reference): TP={TP} FN={FN} FP={FP} TN={TN} N={N}")
line("Sensitivity TP/(TP+FN)", TP, TP + FN)
line("Specificity TN/(TN+FP)", TN, TN + FP)
line("NPV TN/(TN+FN)", TN, TN + FN)
line("PPV TP/(TP+FP)", TP, TP + FP)
line("Accuracy (TP+TN)/N", TP + TN, N)
ACC_M = (TP + TN, N)
print(" As printed in Table 3, MULTI-IMPROD column:")
for k, (a, n, p) in T3_MULTI.items():
print(f" {k:<12s} printed {p}% ({a}/{n}); {a}/{n} = {pct(a, n):.1f}%")
print(f" Accuracy numerator printed 217, recomputed {TP+TN}: difference {217-(TP+TN)}")
print(f" PPV numerator printed 141, recomputed {TP}: difference {TP-141}")
# ------------------------------------------------------------------ 3. IMPROD trial column
TP2, FN2 = 79, 83 - 79
TN2, FP2 = 34, 78 - 34
N2 = TP2 + FN2 + TN2 + FP2
assert N2 == 161 and TP2 + FP2 == 123 and TN2 + FN2 == 38
print(f"\n3. IMPROD trial (Table 3 right-hand column): TP={TP2} FN={FN2} FP={FP2} TN={TN2} N={N2}")
line("Accuracy (TP+TN)/N", TP2 + TN2, N2)
line("NPV", TN2, TN2 + FN2)
line("Specificity", TN2, TN2 + FP2)
line("PPV", TP2, TP2 + FP2)
# ------------------------------------------------------------------ 4. The abstract sentence
print("\n4. Abstract: 'accuracy and NPV ... were 70% (113/161) and 95% (71/75) (95% CI 87%-98%)'")
print(f" 113/161 = {pct(113,161):.1f}% = IMPROD trial accuracy ({TP2}+{TN2} = {TP2+TN2}; N = {N2})")
print(f" 71/75 = {pct(71,75):.1f}% = MULTI-IMPROD NPV (Wilson {100*wilson(71,75)[0]:.0f}-{100*wilson(71,75)[1]:.0f}%)")
lo, hi = wilson(*ACC_M)
print(f" MULTI-IMPROD accuracy {ACC_M[0]}/{ACC_M[1]} = {pct(*ACC_M):.1f}%, Wilson 95% CI {100*lo:.1f}-{100*hi:.1f}%;"
f" 70.2% inside CI? {lo <= 113/161 <= hi}")
print(f" Overstatement: {pct(113,161) - pct(*ACC_M):.1f} percentage points")
lo2, hi2 = wilson(113, 161)
print(f" IMPROD accuracy Wilson CI {100*lo2:.1f}-{100*hi2:.1f}%")
# ------------------------------------------------------------------ 5. Side finding (Likert 5)
print("\n5. Results text, Likert 5: printed 88% (121/138)")
print(f" 121/138 = {pct(121,138):.1f}%; Table 2 gives {GG2[4]}+{GG3P[4]} = {SPCA[4]}/138 = {pct(SPCA[4],138):.1f}%")
print(f" With 121 the Likert-group SPCa counts would sum to {sum(SPCA[:4]) + 121}, not {sum(SPCA)}")
print(f" 121 equals the TB-detected SPCa count ({TB_SPCA}/338 = {pct(TB_SPCA,338):.1f}%; {TB_SPCA}/263 = {pct(TB_SPCA,263):.1f}%)")
# ================================================================== figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRAY, GRID = "#eb6834", "#2a78d6", "#8a8984", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"lines.linewidth": 1.5, "svg.fonttype": "path",
})
W = 7.2
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg", bbox_inches="tight", pad_inches=0.12)
fig.savefig(p + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
def box(ax, x, y, w, h, edge=GRAY, lw=1.2, fill=BG):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc=fill, ec=edge, lw=lw, transform=ax.transAxes))
def arrow(ax, p0, p1, color, rad=0.0, lw=1.5):
ax.add_patch(FancyArrowPatch(p0, p1, transform=ax.transAxes, arrowstyle="-|>", mutation_scale=13,
color=color, lw=lw, connectionstyle=f"arc3,rad={rad}"))
def blank(h):
fig = plt.figure(figsize=(W, h))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_axis_off()
return fig, ax
m_acc = f"{ACC_M[0]}/{ACC_M[1]} = {pct(*ACC_M):.1f}%"
# ---------------------------------------------------------- candidate 1: provenance diagram (fig1)
def segments(fig, ax, x, y, parts, fs=10.5):
"""Place text pieces left to right; parts = [(text, edgecolor or None)]. Returns x-centres."""
r = fig.canvas.get_renderer()
inv = ax.transAxes.inverted()
centres = []
for txt, ec in parts:
kw = dict(fontsize=fs, va="top", transform=ax.transAxes)
if ec:
kw.update(weight="bold", bbox=dict(boxstyle="round,pad=0.22", fc=BG, ec=ec, lw=2))
t = ax.text(x, y, txt, **kw)
bb = t.get_window_extent(r)
(x0, _), (x1, _) = inv.transform([(bb.x0, bb.y0), (bb.x1, bb.y1)])
centres.append((x0 + x1) / 2)
x = x1 + (0.012 if ec else 0.0)
if ec:
x += 0.004
return centres
def cand1():
fig, ax = blank(4.7)
box(ax, 0.02, 0.68, 0.96, 0.29, edge=INK2)
ax.text(0.04, 0.945, "Abstract (Methods and findings)", fontsize=10.5, weight="bold", va="top")
ax.text(0.04, 0.875, "\u201cIn total, 338 (338/364, 93%) prospectively enrolled men completed the trial.",
fontsize=10.5, va="top")
ax.text(0.04, 0.815, "The accuracy and NPV of IMPROD bpMRI for SPCa were", fontsize=10.5, va="top")
c = segments(fig, ax, 0.04, 0.745, [("70% (113/161)", ORANGE), (" and ", None), ("95% (71/75)", BLUE),
(" (95% CI 87%\u201398%), respectively.\u201d", None)])
acc_x, npv_x = c[0], c[2]
panels = [
(0.52, "Table 3, MULTI-IMPROD column", "this trial: 4 centres, n = 338",
[("NPV", "95% (71/75)", f"recomputed {TN}/{TN+FN} = {pct(TN,TN+FN):.1f}%", True),
("Accuracy", "63% (217/338)", f"recomputed ({TP}+{TN})/{N} = {pct(*ACC_M):.1f}%", False)]),
(0.02, "Table 3, IMPROD trial column", "earlier single-centre cohort, n = 161",
[("NPV", "90% (34/38)", f"recomputed {TN2}/{TN2+FN2} = {pct(TN2,TN2+FN2):.1f}%", False),
("Accuracy", "70% (113/161)", f"recomputed ({TP2}+{TN2})/{N2} = {pct(TP2+TN2,N2):.1f}%", True)]),
]
for x0, title, sub, rows in panels:
box(ax, x0, 0.10, 0.46, 0.40, edge=INK2)
ax.text(x0 + 0.02, 0.47, title, fontsize=10.5, weight="bold", va="top")
ax.text(x0 + 0.02, 0.415, sub, fontsize=10, color=INK2, va="top")
for j, (k, pr, rc, hit) in enumerate(rows):
y = 0.335 - 0.115 * j
ax.text(x0 + 0.02, y, f"{k}: {pr}", fontsize=10.5, va="top", weight="bold" if hit else "normal")
ax.text(x0 + 0.02, y - 0.05, rc, fontsize=10, va="top", color=INK2)
arrow(ax, (acc_x, 0.69), (acc_x, 0.505), ORANGE)
arrow(ax, (npv_x, 0.69), (0.70, 0.505), BLUE, rad=0.1)
ax.text(acc_x + 0.02, 0.59, "accuracy: from the earlier\ntrial (n = 161)", fontsize=10, ha="left", va="center")
ax.text(0.97, 0.59, "NPV: from this trial\n(n = 338)", fontsize=10, ha="right", va="center")
ax.text(0.02, 0.045, f"Accuracy of this trial's 338 men, from its own cells: {m_acc}. "
f"(Table 3 prints 63%, but 217/338 = {pct(217,338):.1f}%.)", fontsize=10, color=INK2, va="center")
return fig
# ---------------------------------------------------------- candidate 2: dot-and-interval (fig2)
def cand2():
from matplotlib.lines import Line2D
rows = [ # label, a, n, kind
("Abstract, presented as\nthis trial: \u201c70% (113/161)\u201d", 113, 161, "rep"),
("This trial (n = 338),\nrecomputed (142 + 71)/338", ACC_M[0], ACC_M[1], "rec"),
("Earlier IMPROD trial\n(n = 161), Table 3", 113, 161, "ctx"),
]
col = {"rep": ORANGE, "rec": BLUE, "ctx": GRAY}
fig, ax = plt.subplots(figsize=(W, 2.9))
ys = list(range(len(rows)))[::-1]
lo_m, hi_m = wilson(*ACC_M)
ax.axvspan(100 * lo_m, 100 * hi_m, color=BLUE, alpha=0.08, lw=0)
for y, (lab, a, n, kind) in zip(ys, rows):
p = pct(a, n)
c = col[kind]
if kind == "rep":
ax.plot(p, y, "o", ms=10, mfc=BG, mec=c, mew=2.2)
ax.text(p + 2.5, y, f"70% as printed (113/161 = {p:.1f}%)", va="center", fontsize=10)
else:
lo, hi = wilson(a, n)
ax.plot([100 * lo, 100 * hi], [y, y], color=c, lw=2, solid_capstyle="round")
ax.plot(p, y, "o", ms=9, color=c)
ax.text(100 * hi + 1.5, y, f"{p:.1f}% [{100*lo:.1f}\u2013{100*hi:.1f}]", va="center", fontsize=10)
ax.text(100 * lo_m + 0.5, len(rows) - 0.45, "95% CI, this trial", fontsize=10, color=INK2, va="top", ha="right")
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_xlim(0, 100)
ax.set_ylim(-0.55, len(rows) - 0.4)
ax.set_xlabel("Overall accuracy for SPCa, % (Likert 3\u20135 vs 1\u20132; lines: Wilson 95% CI)", fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h = [Line2D([], [], marker="o", ls="", ms=9, mfc=BG, mec=ORANGE, mew=2, label="reported in the abstract"),
Line2D([], [], marker="o", ls="-", ms=9, color=BLUE, lw=2, label="recomputed, this trial"),
Line2D([], [], marker="o", ls="-", ms=9, color=GRAY, lw=2, label="earlier cohort (context)")]
ax.legend(handles=h, loc="lower left", bbox_to_anchor=(-0.02, 1.0), frameon=False, fontsize=10, ncol=3,
handletextpad=0.4, columnspacing=1.2)
return fig
# ---------------------------------------------------------- candidate 3: all metrics, both cohorts
def cand3():
metrics = [("Sensitivity", (TP, TP + FN), (TP2, TP2 + FN2)),
("Specificity", (TN, TN + FP), (TN2, TN2 + FP2)),
("NPV", (TN, TN + FN), (TN2, TN2 + FN2)),
("PPV", (TP, TP + FP), (TP2, TP2 + FP2)),
("Accuracy", ACC_M, (TP2 + TN2, N2))]
fig, ax = plt.subplots(figsize=(W, 3.4))
ys = list(range(len(metrics)))[::-1]
for y, (k, m, i) in zip(ys, metrics):
pm, pi = pct(*m), pct(*i)
ax.plot([pm, pi], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(pi, y, "s", ms=8, color=GRAY, zorder=2)
ax.plot(pm, y, "o", ms=9, color=BLUE, zorder=3)
# abstract values
ax.plot(pct(*ABS_NPV[:2]), ys[2], "o", ms=16, mfc="none", mec=ORANGE, mew=2, zorder=4)
ax.plot(pct(*ABS_ACC[:2]), ys[4], "o", ms=16, mfc="none", mec=ORANGE, mew=2, zorder=4)
ax.annotate("abstract NPV 95% (71/75)\n= this trial's value", (pct(71, 75), ys[2]), xytext=(72, ys[2] - 0.45),
fontsize=10, color=INK, ha="right", va="center",
arrowprops=dict(arrowstyle="-", color=ORANGE, lw=1.5))
ax.annotate("abstract accuracy 70% (113/161)\n= the earlier trial's value", (pct(113, 161), ys[4]),
xytext=(84, ys[4] + 0.55), fontsize=10, color=INK, ha="left", va="center",
arrowprops=dict(arrowstyle="-", color=ORANGE, lw=1.5))
ax.set_yticks(ys)
ax.set_yticklabels([m[0] for m in metrics], color=INK)
ax.set_xlim(0, 100)
ax.set_ylim(-0.6, len(metrics) - 0.3)
ax.set_xlabel("% (Likert 3–5 vs 1–2, SPCa = Gleason ≥ 3+4)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
h = [Line2D([], [], marker="o", ls="", ms=9, color=BLUE, label="MULTI-IMPROD (this trial, n = 338), recomputed"),
Line2D([], [], marker="s", ls="", ms=8, color=GRAY, label="IMPROD trial (earlier, n = 161), Table 3"),
Line2D([], [], marker="o", ls="", ms=12, mfc="none", mec=ORANGE, mew=2, label="value in the abstract")]
ax.legend(handles=h, loc="lower left", bbox_to_anchor=(0, 1.0), frameon=False, fontsize=10, ncol=2)
return fig
# ---------------------------------------------------------- candidate 4: calculation flow
def cand4():
fig, ax = blank(4.6)
top = [
(0.02, "1. Table 2 (n = 338)", ["SPCa, Likert 1\u20135:", " " + ", ".join(map(str, SPCA)),
"men, Likert 1\u20135:", " " + ", ".join(map(str, TOTAL))], INK2),
(0.355, "2. Cut at Likert \u2265 3", [f"TP {TP} FN {FN}", f"FP {FP} TN {TN}",
"reproduces Table 3 sens,", "spec, NPV and their CIs"], INK2),
(0.69, "3. Accuracy", [f"({TP} + {TN}) / {N}", f"= {TP+TN}/{N} = {pct(TP+TN,N):.1f}%",
"Wilson 95% CI", f"{100*wilson(TP+TN,N)[0]:.1f}\u2013{100*wilson(TP+TN,N)[1]:.1f}%"], BLUE),
]
for x0, t, body, c in top:
box(ax, x0, 0.56, 0.29, 0.41, edge=c, lw=2.2 if c == BLUE else 1.2)
ax.text(x0 + 0.02, 0.935, t, weight="bold", va="top", fontsize=10.5)
for j, b in enumerate(body):
ax.text(x0 + 0.02, 0.85 - 0.068 * j, b, va="top", fontsize=10,
color=INK if c == BLUE or j % 2 else INK2)
arrow(ax, (0.312, 0.765), (0.353, 0.765), INK2)
arrow(ax, (0.647, 0.765), (0.688, 0.765), INK2)
bottom = [
(0.355, "Table 3 prints", ["63% (217/338)", f"but 217/338 = {pct(217,338):.1f}%",
f"count {217-(TP+TN)} too high"]),
(0.69, "Abstract prints", ["70% (113/161)", f"= earlier trial's {TP2}+{TN2}",
f"{pct(113,161)-pct(TP+TN,N):.1f} points higher"]),
]
for x0, t, body in bottom:
box(ax, x0, 0.13, 0.29, 0.33, edge=ORANGE, lw=2.2)
ax.text(x0 + 0.02, 0.425, t, weight="bold", va="top", fontsize=10.5)
for j, b in enumerate(body):
ax.text(x0 + 0.02, 0.34 - 0.07 * j, b, va="top", fontsize=10, color=INK if j == 0 else INK2)
arrow(ax, (0.80, 0.56), (0.55, 0.465), INK2, rad=-0.1)
arrow(ax, (0.835, 0.56), (0.835, 0.465), INK2)
ax.text(0.02, 0.40, "compare step 3\nwith the printed\nvalues (orange)", fontsize=10, color=INK2, va="top")
ax.text(0.02, 0.04, "Blue: recomputed from the paper's own Table 2 counts. Orange: values as printed.",
fontsize=10, color=INK2, va="bottom")
return fig
# ---------------------------------------------------------- candidate 5: the two 2x2 tables
def cand5():
fig, axs = plt.subplots(1, 2, figsize=(W, 3.6))
for ax, (title, tp, fn, fp, tn, col, note) in zip(axs, [
("MULTI-IMPROD (this trial), n = 338", TP, FN, FP, TN, BLUE, f"accuracy {TP+TN}/{TP+FN+FP+TN} = {pct(TP+TN,TP+FN+FP+TN):.1f}%"),
("IMPROD trial (earlier), n = 161", TP2, FN2, FP2, TN2, GRAY, f"accuracy {TP2+TN2}/{N2} = {pct(TP2+TN2,N2):.1f}%")]):
ax.set_xlim(0, 2)
ax.set_ylim(0, 2.75)
ax.set_axis_off()
cells = {(0, 1): ("TP", tp), (1, 1): ("FP", fp), (0, 0): ("FN", fn), (1, 0): ("TN", tn)}
for (cx, cy), (k, v) in cells.items():
diag = k in ("TP", "TN")
ax.add_patch(FancyBboxPatch((cx + 0.05, cy + 0.05), 0.9, 0.9, boxstyle="round,pad=0,rounding_size=0.04",
fc=col if diag else BG, ec=col, lw=2, alpha=0.18 if diag else 1))
ax.add_patch(FancyBboxPatch((cx + 0.05, cy + 0.05), 0.9, 0.9, boxstyle="round,pad=0,rounding_size=0.04",
fc="none", ec=col, lw=2))
ax.text(cx + 0.5, cy + 0.6, k, ha="center", fontsize=10, color=INK2)
ax.text(cx + 0.5, cy + 0.3, str(v), ha="center", fontsize=13, weight="bold")
ax.text(0.5, 2.03, "SPCa", ha="center", fontsize=10, color=INK2)
ax.text(1.5, 2.03, "no SPCa", ha="center", fontsize=10, color=INK2)
ax.text(1.0, 2.35, title, ha="center", fontsize=10.5, weight="bold")
ax.text(1.0, -0.12, note, ha="center", va="top", fontsize=10.5)
axs[0].text(-0.05, 1.5, "Likert 3–5", ha="right", va="center", fontsize=10, color=INK2)
axs[0].text(-0.05, 0.5, "Likert 1–2", ha="right", va="center", fontsize=10, color=INK2)
fig.text(0.5, -0.12, "Abstract, for this trial: “accuracy … 70% (113/161)” — the right-hand "
"table's figure, not the left-hand table's.", ha="center", fontsize=10.5, color=INK,
bbox=dict(boxstyle="round,pad=0.35", fc=BG, ec=ORANGE, lw=2))
fig.subplots_adjust(wspace=0.25)
return fig
save(cand1(), os.path.join(HERE, "fig1"), os.path.join(CAND, "cand1_provenance"))
save(cand2(), os.path.join(HERE, "fig2"), os.path.join(CAND, "cand2_dot_interval"))
save(cand3(), os.path.join(CAND, "cand3_all_metrics"))
save(cand4(), os.path.join(CAND, "cand4_calculation_flow"))
save(cand5(), os.path.join(CAND, "cand5_two_by_two"))
print("\nWrote fig1, fig2 and candidates/cand1..cand5 (.svg, .png) in", HERE)
The abstract's univariate diabetes odds ratio is inverted, and its p-value disagrees with Fig 5
PLoS Med (2019) · doi:10.1371/journal.pmed.1002961 · PMC6822721
Summary
The abstract and the Results say that type 2 diabetes is a risk factor for being a tuberculosis transmitter. They report the univariate result as "odds ratio 0.19 [95% CI 0.02–1.10], p < 0.003". Fig 5 of the same paper gives the counts behind this comparison: 5 of 23 transmitters and 3 of 61 other clustered cases had diabetes. It prints OR 5.14 (0.90–36.45) and p = 0.03 for them. Recomputed from those counts, the OR for transmitters vs others is 5.24 (exact 95% CI 0.92–37.08) and the two-sided Fisher p is 0.032. The text's 0.19 (0.02–1.10) is that estimate with the groups swapped (0.191, 0.027–1.089). So the text gives an OR pointing the opposite way from its own word "enriched", and a p-value ten times smaller than the counts support. This is the only risk factor named in the abstract. At the univariate level, the counts support a borderline association (p = 0.032, exact CI just including 1), not one with p < 0.003.
What the paper reports
Abstract, "Methods and findings":
Our data suggest that type 2 diabetes mellitus is a risk factor associated with being a transmitter (odds ratio 0.19 [95% CI 0.02–1.10], p < 0.003).
Results, section "Identification of transmitters allows association of risk factors to transmission", second paragraph:
Transmitters were significantly enriched in diabetic patients in both univariate (Fisher’s exact test; OR 0.19 [95% CI 0.02–1.10], p < 0.003) and multivariate (logistic regression; OR 23.77 [95% CI 2.53–339.69], p < 0.009) statistical analyses.
The same section, first paragraph, defines the comparison:
We combined likely transmitter cases together with the index cases predicted by TransPhylo (n = 23) and compared them to the other clustered cases (n = 61).
Methods, "Statistical analysis":
We selected the index cases and the samples with higher than 0.6 posterior probability of being transmitters as predicted by TransPhylo (23 transmitters compared to the remaining 84 clustered cases), with sensitivity analysis of the later threshold in S1 Text and S1 Table. Then, we computed the odds ratio (OR) and 95% confidence intervals (Fisher’s exact test) to explore epidemiological variables associated with being a transmitter.
Fig 5, Diabetes row. Fig 5 is an image, so this row was read from the publisher's figure file (journal.pmed.1002961.g005). Its columns are All Patients (n=84), Transmitter (n=23), Other samples (n=61), Odd ratio (95% IC), p-value*:
Diabetes | 8 (9%) | 5 (22%) | 3 (5%) | 5.14 (0.90-36.45) | 0.03
Fig 5 caption:
The figure shows estimated odd ratios for each risk factor tested. *Fisher’s exact test. Comparisons were made between transmitter cases and the rest of the clustered samples.
Discussion, fourth paragraph:
Still, our data suggest that certain risk and epidemiological factors are enriched among the transmitters, while others are depleted.
What the paper's own numbers show
Method. I took the diabetes counts from Fig 5 and rebuilt the 2×2 table: 5 of 23 transmitters and 3 of 61 other clustered cases had diabetes, 8 of 84 in all. For transmitters vs others, and again with the groups swapped, I computed the conditional maximum-likelihood OR with its central exact (Cornfield) 95% CI. These are the quantities R's fisher.test prints. I also computed the two-sided Fisher exact p. As a separate check that does not use the counts, I back-calculated a log-Wald p from the text's own interval, 0.02–1.10. I searched every 23-vs-61 table with more diabetes among transmitters for tables that would give Fisher p < 0.003. I did the same Wald back-calculation for the multivariate result, which I cannot recompute without individual data. Every number below is printed by recompute.py.txt (scipy).
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Diabetic transmitters | Fig 5 | 5 (22%) of 23 | 5/23 = 21.7% |
| Diabetic other clustered cases | Fig 5 | 3 (5%) of 61 | 3/61 = 4.9% |
| OR, transmitters vs others | Fig 5 | 5.14 (0.90–36.45) | conditional MLE 5.24 (exact 95% CI 0.92–37.08); sample OR 5.37. The median-unbiased estimate is 5.15, closer to the printed 5.14 |
| OR as given in the text | Abstract; Results | 0.19 (0.02–1.10) | Groups swapped (others vs transmitters): 0.191 (0.027–1.089). Equivalently 1/5.14 = 0.195, 1/36.45 = 0.027, 1/0.90 = 1.11 |
| Direction implied by the text's OR | Abstract; Results | "risk factor", "significantly enriched" | With others at 4.9%, an OR of 0.19 would mean 0.97% of transmitters had diabetes (0.22 of 23 people), not 21.7% |
| Fisher exact p | Fig 5 | 0.03 | two-sided 0.0324; one-sided 0.0324 |
| Fisher exact p | Abstract; Results | p < 0.003 | 0.0324 from the Fig 5 counts, about ten times larger |
| p compatible with the text's own CI | Abstract; Results | p < 0.003 | log-Wald p from 0.19 (0.02–1.10) ≈ 0.10. A central exact 95% CI that includes 1 means each one-sided exact p is at least 0.025, so the two-sided Fisher p is at least 0.025 |
| Counts needed for Fisher p < 0.003 | derived | — | With 3 of 61 diabetic others, at least 8 of 23 transmitters (Fig 5: 5). With 5 diabetic transmitters, 0 of 61 others (Fig 5: 3) |
| Multivariate logistic OR | Results | 23.77 (2.53–339.69), p < 0.009 | Not recomputable. Wald p from the CI: 0.011 using 23.77, or 0.007 using the CI's geometric midpoint (29.32). Consistent with the reported p and with OR > 1 |
| Size of the comparison group | Methods vs Results, Fig 5 | "remaining 84" vs n = 61 | 23 + 61 = 84, so 84 is the total, not the comparison group |
Figure
Odds ratios for diabetes with 95% confidence intervals on a log scale; the thin vertical line marks OR = 1 (no association). Orange: values printed in the paper (abstract and Results text, Fig 5, and the multivariate model). Blue: values recomputed from Fig 5's counts (5/23 transmitters vs 3/61 other clustered cases), once with transmitters as the index group and once with the groups swapped. The text's univariate interval coincides with the swapped-group recomputation. Fig 5's interval coincides with the transmitters-vs-others recomputation. The p-value printed next to the text's interval (p < 0.003) matches neither.
The same comparison on a p-value scale (log scale); the thin vertical line marks 0.05. Orange: the text's "p < 0.003" (the arrow shows that the bound is an upper bound) and Fig 5's p = 0.03. Blue: the two-sided Fisher p recomputed from Fig 5's counts (0.0324), and the log-Wald p implied by the text's own interval 0.02–1.10 (≈ 0.10). The shaded region (p < 0.025) is excluded for a Fisher test whose exact 95% CI includes 1. The text's value lies inside it.
Why it matters
The diabetes result is the only risk factor for transmission named in the abstract. It also serves as the example for the abstract's wider claim that "factors associated with likely transmitters are different to those of simply being in a transmission cluster". As printed, the univariate statistics cannot all be right together:
- An OR of 0.19 means transmitters had lower odds of diabetes, which is the opposite of the claimed enrichment.
- Its interval includes 1.
- p < 0.003 is incompatible with that interval and with the counts in Fig 5.
A reader who takes the abstract at face value gets a wrong direction and a strength of evidence about ten times greater than the data support. Recomputed from the paper's own counts, the univariate result is OR 5.24, two-sided Fisher p = 0.032, with an exact CI (0.92–37.08) that just includes 1. That is a borderline association resting on 5 diabetic transmitters, not a result with p < 0.003. The text describes the univariate analysis as showing transmitters "significantly enriched". That description holds only at p = 0.032, and the matching exact interval does not exclude 1.
What is not affected:
- Direction. The direction of the association is supported by the Fig 5 counts (22% vs 5%) and by the separate multivariate model (OR 23.77, 2.53–339.69). That model's reported p < 0.009 is close to the 0.011 back-calculated from its interval.
- Main conclusions. The paper's main conclusions do not rest on this number: index cases are often misidentified (14 of 21 clusters), and some transmission occurs before symptom onset (5 of 14).
Explanations considered
- Reversed reference group. This fully accounts for the OR and its interval. Swapping the groups in the Fig 5 table gives 0.191 (0.027–1.089), which matches "0.19 (0.02–1.10)". So does taking the reciprocal of Fig 5's 5.14 (0.90–36.45). It does not make the sentence correct: the text pairs this inverted OR with "risk factor" and "significantly enriched", and Fig 5 itself reports the comparison in the transmitters-vs-others direction.
- A different two-sided or one-sided convention for p. This does not account for p < 0.003. From the Fig 5 counts, the two-sided Fisher p is 0.0324, the one-sided p is 0.0324, and doubling the one-sided p gives 0.065. None is below 0.003. Fig 5 prints 0.03. One possibility is that "0.003" is a misplaced decimal for 0.03, but the paper gives no explanation, so this is an inference.
- A different comparison group (the Methods' "remaining 84"). This does not account for it either. The Results and Fig 5 both use 61 others, and 23 + 61 = 84 is Fig 5's "All Patients" total, so "remaining 84" appears to be a wording slip in the Methods. More generally, any table whose exact 95% CI includes 1, as the printed 0.02–1.10 does, has a Fisher p of at least 0.025. No choice of comparison group can then produce p < 0.003 together with that interval.
- Fisher's p and the exact CI disagreeing near 0.05. This is real, but it does not rescue the text. Fisher's two-sided p (summing tables no more likely than the one observed) and the central exact CI can disagree near the 0.05 boundary. That explains why Fig 5 shows p = 0.03 alongside an interval that just includes 1. It cannot produce a p ten times smaller.
- Small differences between Fig 5 and the recomputation (5.14 vs 5.24; 0.90–36.45 vs 0.92–37.08). These are minor and not part of the claim. They may reflect different software or estimators: the median-unbiased estimate is 5.15. Either way, the interval includes 1 and the estimate is above 1.
- The multivariate result. This is a separate logistic model, consistent in direction and in its own p-value. I cannot recompute it from the paper, and it is not part of the claim.
- Conversion artefact. Ruled out. The same text appears in the publisher's HTML, the publisher's manuscript XML, PMC and Europe PMC, and the Fig 5 values come from the publisher's own image file.
Two further observations, which do not affect the claim:
- Fig 5's HIV row repeats the Diabetes row's counts, 5 (22%) and 3 (5%), but with OR 1.01, and its "All Patients" entry for HIV is 7 rather than 8.
- In the comparison with non-clustered cases, the Results give transmitters as 27% diabetic. Neither 5/23 (21.7%) nor 6/23 (26.1%) rounds to 27%.
Status
- Version of record. The errors are in the version of record. I checked the PLOS Medicine article page (journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002961), the publisher's manuscript XML, PMC (PMC6822721), Europe PMC full-text XML and the locally archived PMC text. All carry the same abstract and Results wording. The Fig 5 values were read from the publisher's figure file (journal.pmed.1002961.g005).
- Supplements. I listed all 19 supplementary files (S1–S13 Figs, S1–S4 Tables, S1–S2 Text) and opened S1–S4 Tables and S1 Text. None gives the diabetes 2×2 table or an odds ratio for diabetes. S3 Table lists diabetes in 13 of the 109 cases with epidemiological data. S1 Table is a meta-analysis of clock rates, although the Methods cite it for the threshold sensitivity analysis.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no notices. The PLOS article page shows "Reader Comments (0)". The PubPeer API asked for human verification, and the PubPeer search page refused automated access (HTTP 403), so I could not check PubPeer directly. A web search found no PubPeer page for the article.
- Sources. PMC6822721; https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002961 (HTML, manuscript XML, Fig 5 image g005, supplementary files); Europe PMC; Crossref; PubPeer.
Suggested correction
In the Abstract and in the Results ("Identification of transmitters..."), the univariate result should be reported in the same direction as the sentence and as Fig 5. Using Fig 5's own values, the replacement would be "odds ratio 5.14 [95% CI 0.90–36.45], p = 0.03". From the Fig 5 counts, the conditional maximum-likelihood estimate is "OR 5.24 [95% CI 0.92–37.08], p = 0.032".
Because that interval includes 1, the Results' "significantly enriched ... in both univariate ... and multivariate" analyses could be reworded. For example: diabetes was more frequent among transmitters (5/23 vs 3/61; Fisher's exact p = 0.03, OR 5.14, 95% CI 0.90–36.45) and was significantly associated in the multivariate model.
In the Methods, "the remaining 84 clustered cases" should read "the remaining 61 clustered cases (84 in total)".
The authors hold the individual data and could confirm these values, and the Fig 5 HIV row, against it.
Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
"""Recompute the diabetes risk-factor statistics in PMC6822721
(PLoS Med 2019, doi:10.1371/journal.pmed.1002961) from the paper's own Fig 5 counts,
and regenerate the figures in this folder.
Run: python recompute.py.txt
Needs numpy, scipy, matplotlib.
Inputs (all from the version of record):
Fig 5, Diabetes row: transmitters 5 (22%) of n = 23; other clustered samples 3 (5%) of n = 61;
odds ratio 5.14 (0.90-36.45); Fisher's exact p = 0.03
Abstract and Results text: OR 0.19 [95% CI 0.02-1.10], p < 0.003 (Fisher's exact test)
Results text, multivariate logistic regression: OR 23.77 [95% CI 2.53-339.69], p < 0.009
"""
import os
import numpy as np
from scipy.stats import fisher_exact, nchypergeom_fisher as NCH, norm
from scipy.optimize import brentq
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs
TX_DIAB, TX_N = 5, 23 # Fig 5, transmitters
OT_DIAB, OT_N = 3, 61 # Fig 5, other clustered samples
FIG5 = dict(or_=5.14, lo=0.90, hi=36.45, p=0.03)
TEXT = dict(or_=0.19, lo=0.02, hi=1.10, p=0.003) # p printed as "p < 0.003"
MULTI = dict(or_=23.77, lo=2.53, hi=339.69, p=0.009) # p printed as "p < 0.009"
# ---------------------------------------------------------------- helpers
def margins(a, b, c, d):
return a + b + c + d, a + b, a + c # total, row-1 total, column-1 total
def cond_mle(a, b, c, d):
"""Conditional maximum-likelihood OR (the estimate R's fisher.test prints)."""
M, n, N = margins(a, b, c, d)
return np.exp(brentq(lambda t: NCH.mean(M, n, N, np.exp(t)) - a, -20, 20))
def exact_ci(a, b, c, d, alpha=0.05):
"""Central exact (Cornfield) CI, as printed by R's fisher.test."""
M, n, N = margins(a, b, c, d)
lo = np.exp(brentq(lambda t: NCH.sf(a - 1, M, n, N, np.exp(t)) - alpha / 2, -20, 20))
hi = np.exp(brentq(lambda t: NCH.cdf(a, M, n, N, np.exp(t)) - alpha / 2, -20, 20))
return lo, hi
def median_unbiased(a, b, c, d):
M, n, N = margins(a, b, c, d)
f = lambda t: (NCH.cdf(a, M, n, N, np.exp(t)) - 0.5 * NCH.pmf(a, M, n, N, np.exp(t))) - 0.5
return np.exp(brentq(f, -20, 20))
def wald_p_from_ci(est, lo, hi):
se = (np.log(hi) - np.log(lo)) / (2 * norm.ppf(0.975))
z = np.log(est) / se
return se, z, 2 * norm.sf(abs(z))
def r(x, k=3):
return float(np.round(x, k))
# ---------------------------------------------------------------- numbers
print("=== Fig 5 diabetes counts ===")
a, b = TX_DIAB, TX_N - TX_DIAB
c, d = OT_DIAB, OT_N - OT_DIAB
print(f"transmitters diabetic {a}/{TX_N} = {a/TX_N:.1%}; other clustered {c}/{OT_N} = {c/OT_N:.1%}; "
f"all {a+c}/{TX_N+OT_N} = {(a+c)/(TX_N+OT_N):.1%}")
print(f"group sizes: {TX_N} + {OT_N} = {TX_N+OT_N} (Fig 5 'All Patients (n=84)')")
print("\n=== Transmitters vs others (direction stated in the text: 'enriched') ===")
sample_or = (a * d) / (b * c)
mle = cond_mle(a, b, c, d)
lo, hi = exact_ci(a, b, c, d)
mue = median_unbiased(a, b, c, d)
p2 = fisher_exact([[a, b], [c, d]])[1]
p_hi = fisher_exact([[a, b], [c, d]], alternative="greater")[1]
print(f"sample OR = {sample_or:.3f}")
print(f"conditional MLE OR = {mle:.3f}, exact 95% CI {lo:.3f}-{hi:.3f}")
print(f"median-unbiased OR = {mue:.3f} (Fig 5 prints 5.14)")
print(f"Fisher exact p: two-sided = {p2:.4f}; one-sided (greater) = {p_hi:.4f}; doubled one-sided = {min(1,2*p_hi):.4f}")
print(f"exact CI includes 1: {lo < 1 < hi}")
print("\n=== Reversed coding: others vs transmitters ===")
mle_r = cond_mle(c, d, a, b)
lo_r, hi_r = exact_ci(c, d, a, b)
print(f"conditional MLE OR = {mle_r:.3f}, exact 95% CI {lo_r:.3f}-{hi_r:.3f} (text: 0.19 [0.02-1.10])")
print(f"reciprocals of Fig 5: 1/{FIG5['or_']} = {1/FIG5['or_']:.3f}; 1/{FIG5['hi']} = {1/FIG5['hi']:.4f}; "
f"1/{FIG5['lo']} = {1/FIG5['lo']:.3f}")
print(f"reciprocals of text: 1/{TEXT['or_']} = {1/TEXT['or_']:.2f}; 1/{TEXT['hi']} = {1/TEXT['hi']:.3f}; "
f"1/{TEXT['lo']} = {1/TEXT['lo']:.0f}")
print("\n=== What the text's OR would imply ===")
odds_ot = c / d
odds_tx_implied = TEXT["or_"] * odds_ot
pct_implied = odds_tx_implied / (1 + odds_tx_implied)
print(f"with others at {c}/{OT_N} ({c/OT_N:.1%}), OR 0.19 implies {pct_implied:.2%} of transmitters diabetic "
f"= {pct_implied*TX_N:.2f} of {TX_N} people; Fig 5 reports {a}/{TX_N} = {a/TX_N:.1%}")
print("\n=== p-values compatible with the text's CI ===")
se_t, z_t, p_t = wald_p_from_ci(TEXT["or_"], TEXT["lo"], TEXT["hi"])
print(f"log-Wald back-calculation from 0.19 (0.02-1.10): se = {se_t:.3f}, z = {z_t:.3f}, p = {p_t:.3f}")
print("an exact central 95% CI that includes 1 means each one-sided exact p >= 0.025,")
print("so Fisher's two-sided p (which contains a full tail) is >= 0.025; text p < 0.003 cannot hold")
print("\n=== Which 23 vs 61 tables would give Fisher p < 0.003 with more diabetes in transmitters? ===")
hits = []
for x in range(TX_N + 1):
for y in range(OT_N + 1):
if x / TX_N > y / OT_N:
pp = fisher_exact([[x, TX_N - x], [y, OT_N - y]])[1]
if pp < 0.003:
hits.append((x, y, pp))
min_x_given_y3 = min(x for x, y, _ in hits if y == OT_DIAB)
print(f"with 3 diabetic others, transmitters would need at least {min_x_given_y3} of {TX_N} diabetic "
f"(Fig 5: {TX_DIAB}); with {TX_DIAB} diabetic transmitters, others would need "
f"{max(y for x, y, _ in hits if x == TX_DIAB)} of {OT_N}")
print("\n=== Multivariate result (reported only; not recomputable) ===")
se_m, z_m, p_m = wald_p_from_ci(MULTI["or_"], MULTI["lo"], MULTI["hi"])
gm = np.exp((np.log(MULTI["lo"]) + np.log(MULTI["hi"])) / 2)
_, _, p_m2 = wald_p_from_ci(gm, MULTI["lo"], MULTI["hi"])
print(f"Wald from CI with reported point 23.77: se = {se_m:.3f}, p = {p_m:.4f}; "
f"with CI geometric midpoint {gm:.2f}: p = {p_m2:.4f} (reported p < 0.009)")
print("\n=== Non-clustered comparison sentence (observation only) ===")
for k in (5, 6):
print(f"{k}/{TX_N} = {k/TX_N:.1%}", end="; ")
print("text says transmitters 27% diabetic in that comparison")
# ---------------------------------------------------------------- figure style
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
LW, MS = 2.0, 9
def save(fig, name, also=None):
for ext in ("svg", "png"):
kw = {"dpi": 200} if ext == "png" else {}
fig.savefig(os.path.join(CAND, f"{name}.{ext}"), bbox_inches="tight", **kw)
if also:
fig.savefig(os.path.join(HERE, f"{also}.{ext}"), bbox_inches="tight", **kw)
plt.close(fig)
def fmt(x):
return f"{x:.3g}" if x < 1 else (f"{x:.2f}" if x < 100 else f"{x:.0f}")
# ---------------------------------------------------------------- A: OR dot-and-interval
def cand_A():
rows = [ # label, est, lo, hi, color, kind, p-label
("Abstract & Results text\n(univariate, Fisher)", TEXT["or_"], TEXT["lo"], TEXT["hi"], ORANGE, "Reported",
"0.19 (0.02–1.10) p < 0.003"),
("Recomputed, others vs transmitters\n(reversed coding)", mle_r, lo_r, hi_r, BLUE, "Recomputed",
f"{mle_r:.3f} ({lo_r:.3f}–{hi_r:.3f}) p = {p2:.3f}"),
("Fig 5, Diabetes row\n(univariate, Fisher)", FIG5["or_"], FIG5["lo"], FIG5["hi"], ORANGE, "Reported",
"5.14 (0.90–36.45) p = 0.03"),
(f"Recomputed, transmitters vs others\n(Fig 5 counts {a}/{TX_N} vs {c}/{OT_N})", mle, lo, hi, BLUE, "Recomputed",
f"{mle:.2f} ({lo:.2f}–{hi:.2f}) p = {p2:.3f}"),
("Results text\n(multivariate logistic)", MULTI["or_"], MULTI["lo"], MULTI["hi"], ORANGE, "Reported",
"23.77 (2.53–339.69) p < 0.009"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.3))
ys = np.arange(len(rows))[::-1]
for y, (lab, e, l, h, col, kind, pl) in zip(ys, rows):
ax.plot([l, h], [y, y], color=col, lw=LW, solid_capstyle="butt")
ax.plot([e], [y], "o", color=col, ms=MS, mec=BG, mew=1.5,
label=kind if kind not in ax.get_legend_handles_labels()[1] else None)
ax.text(h * 1.25, y, pl, va="center", fontsize=10, color=INK)
ax.axvline(1, color=INK2, lw=0.8, zorder=0)
ax.set_xscale("log")
ax.set_xlim(0.01, 20000)
ax.set_xticks([0.01, 0.1, 1, 10, 100, 1000])
ax.set_xticklabels(["0.01", "0.1", "1", "10", "100", "1000"])
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-0.7, len(rows) - 0.3)
ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Odds ratio for diabetes, 95% CI (log scale)")
ax.text(0.85, len(rows) - 0.45, "← fewer diabetic transmitters", ha="right", fontsize=10, color=INK2)
ax.text(1.18, len(rows) - 0.45, "more diabetic transmitters →", ha="left", fontsize=10, color=INK2)
ax.legend(loc="lower left", frameon=False, fontsize=10, handletextpad=0.3)
return fig
# ---------------------------------------------------------------- B: p-value scale
def cand_B():
fig, ax = plt.subplots(figsize=(7.2, 3.6))
ax.axvspan(1e-4, 0.025, color="#f1f0ec", zorder=0)
ax.text(1.2e-4, -0.55, "Shaded: Fisher p below 0.025, impossible when the exact\n95% CI of the same test includes 1 (as 0.02–1.10 does)",
fontsize=10, color=INK2, va="center")
rows = [
("Abstract & Results text", TEXT["p"], ORANGE, "p < 0.003", True),
("Fig 5, Diabetes row", FIG5["p"], ORANGE, "p = 0.03", False),
(f"Recomputed Fisher, two-sided\n(Fig 5 counts {a}/{TX_N} vs {c}/{OT_N})", p2, BLUE, f"p = {p2:.4f}", False),
("Recomputed log-Wald p implied\nby the text's CI 0.02–1.10", p_t, BLUE, f"p ≈ {p_t:.2f}", False),
]
ys = np.arange(len(rows))[::-1]
seen = set()
for y, (lab, p, col, pl, less) in zip(ys, rows):
kind = "Reported" if col == ORANGE else "Recomputed"
ax.plot([p], [y], "o", color=col, ms=MS, mec=BG, mew=1.5, zorder=3,
label=None if kind in seen else kind)
seen.add(kind)
if less:
ax.annotate("", xy=(p / 3.5, y), xytext=(p / 1.15, y),
arrowprops=dict(arrowstyle="-|>", color=col, lw=LW))
ax.text(p * 1.35, y + 0.02, pl, va="center", fontsize=10)
ax.axvline(0.05, color=INK2, lw=0.8, zorder=1)
ax.text(0.053, 3.45, "0.05", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(1e-4, 1)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-1.0, len(rows) - 0.3)
ax.grid(axis="x", color=GRID, lw=0.6)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("p-value for diabetes, transmitters vs other clustered cases (log scale)")
ax.set_xticks([1e-4, 1e-3, 1e-2, 0.1, 1]); ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.1", "1"])
ax.legend(loc="lower left", bbox_to_anchor=(0.0, 1.0), ncol=2, frameon=False, fontsize=10)
return fig
# ---------------------------------------------------------------- C: text vs figure panels
def box(ax, x, y, w, h, title, body, edge, fc="white"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc=fc, ec=edge, lw=1.5))
ax.text(x + 0.02, y + h - 0.025, title, fontsize=10.5, weight="bold", va="top")
ax.text(x + 0.02, y + h - 0.085, body, fontsize=10, va="top", color=INK, linespacing=1.4)
def cand_C():
fig = plt.figure(figsize=(7.2, 5.0)); ax = fig.add_axes([0.01, 0.01, 0.98, 0.98])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
box(ax, 0.0, 0.545, 0.45, 0.445, "Abstract & Results text (printed)",
"\"type 2 diabetes mellitus is a risk\nfactor associated with being a\ntransmitter\"\n"
"\"Transmitters were significantly\nenriched in diabetic patients\"\n\n"
"OR 0.19 [95% CI 0.02–1.10]\np < 0.003", ORANGE)
box(ax, 0.55, 0.545, 0.45, 0.445, "Fig 5, Diabetes row (printed)",
"Transmitters (n = 23): 5 (22%)\nOther samples (n = 61): 3 (5%)\n\n"
"Odds ratio 5.14 (0.90–36.45)\np = 0.03", ORANGE)
ax.add_patch(FancyArrowPatch((0.45, 0.77), (0.55, 0.77), arrowstyle="<|-|>", mutation_scale=14,
color=INK2, lw=LW))
ax.text(0.5, 0.80, "conflict", ha="center", fontsize=10, color=INK2)
box(ax, 0.0, 0.27, 1.0, 0.245, "Recomputed: the direction is reversed",
f"0.19 (0.02–1.10) is the reciprocal of Fig 5: 1/5.14 = {1/FIG5['or_']:.3f},\n"
f"1/36.45 = {1/FIG5['hi']:.3f}, 1/0.90 = {1/FIG5['lo']:.2f}. Others vs transmitters: {mle_r:.3f} ({lo_r:.3f}–{hi_r:.3f}).\n"
"An OR below 1 means fewer diabetic transmitters, not \"enriched\".", BLUE, fc="#eef4fc")
box(ax, 0.0, 0.0, 1.0, 0.245, "Recomputed: the text's p-value is ten times too small",
f"Fisher exact two-sided p from the Fig 5 counts = {p2:.4f}\n"
"(Fig 5 prints 0.03; the text prints p < 0.003). An exact 95% CI that\n"
"includes 1, as 0.02–1.10 does, rules out a Fisher p below 0.025.", BLUE, fc="#eef4fc")
return fig
# ---------------------------------------------------------------- D: what OR 0.19 would imply
def cand_D():
fig, ax = plt.subplots(figsize=(7.2, 2.9))
rows = [
("Other clustered cases\n(Fig 5)", c / OT_N * 100, ORANGE, f"{c}/{OT_N} = {c/OT_N:.1%}"),
("Transmitters\n(Fig 5)", a / TX_N * 100, ORANGE, f"{a}/{TX_N} = {a/TX_N:.1%}"),
("Transmitters, if the text's\nOR 0.19 were correct", pct_implied * 100, BLUE,
f"{pct_implied:.1%} (≈ {pct_implied*TX_N:.1f} of {TX_N} people)"),
]
ys = np.arange(len(rows))[::-1]
seen = set()
for y, (lab, v, col, txt) in zip(ys, rows):
kind = "Reported" if col == ORANGE else "Recomputed (implied by OR 0.19 and the 4.9% in others)"
ax.plot([0, v], [y, y], color=col, lw=LW, alpha=0.5)
ax.plot([v], [y], "o", color=col, ms=MS, mec=BG, mew=1.5, label=None if kind in seen else kind)
seen.add(kind)
ax.text(v + 0.7, y, txt, va="center", fontsize=10)
ax.set_xlim(0, 30)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-0.6, len(rows) - 0.4)
ax.grid(axis="x", color=GRID, lw=0.6); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Share with diabetes (%)")
ax.legend(loc="lower left", bbox_to_anchor=(0.0, 1.0), ncol=2, frameon=False, fontsize=10)
return fig
# ---------------------------------------------------------------- E: calculation flow
def cand_E():
fig = plt.figure(figsize=(7.2, 4.8)); ax = fig.add_axes([0.01, 0.01, 0.98, 0.98])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
def b(x, y, w, h, title, body, edge, fc="white"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc=fc, ec=edge, lw=1.5))
ax.text(x + w / 2, y + h - 0.03, title, ha="center", va="top", fontsize=10.5, weight="bold")
ax.text(x + w / 2, y + h - 0.095, body, ha="center", va="top", fontsize=10, linespacing=1.35)
def arrow(p, q):
ax.add_patch(FancyArrowPatch(p, q, arrowstyle="-|>", mutation_scale=13, color=INK2, lw=1.6))
b(0.29, 0.79, 0.42, 0.20, "Fig 5 counts (as printed)",
f"Transmitters: {a} of {TX_N} diabetic\nOther clustered: {c} of {OT_N} diabetic", ORANGE)
b(0.0, 0.43, 0.44, 0.22, "Transmitters vs others",
f"recomputed OR {mle:.2f} ({lo:.2f}–{hi:.2f})\nFig 5 prints 5.14 (0.90–36.45)", BLUE, fc="#eef4fc")
b(0.56, 0.43, 0.44, 0.22, "Others vs transmitters",
f"recomputed OR {mle_r:.3f} ({lo_r:.3f}–{hi_r:.3f})\ntext prints 0.19 (0.02–1.10)", BLUE, fc="#eef4fc")
b(0.29, 0.04, 0.42, 0.25, "Fisher exact, two-sided",
f"recomputed p = {p2:.4f}\nFig 5 prints 0.03\ntext prints p < 0.003", BLUE, fc="#eef4fc")
arrow((0.36, 0.79), (0.22, 0.65)); arrow((0.64, 0.79), (0.78, 0.65)); arrow((0.5, 0.79), (0.5, 0.29))
ax.text(0.27, 0.73, "matches Fig 5", fontsize=10, color=INK2, ha="right", va="center")
ax.text(0.73, 0.73, "matches the text's OR, but\nthe text says \"enriched\"",
fontsize=10, color=INK2, ha="left", va="center")
ax.text(0.515, 0.36, "matches Fig 5, not the text", fontsize=10, color=INK2, ha="left")
ax.text(0.0, 0.0, "Orange outline: as printed. Blue: recomputed in this script.", fontsize=10, color=INK2)
return fig
save(cand_A(), "A_or_intervals", also="fig1")
save(cand_B(), "B_p_values", also="fig2")
save(cand_C(), "C_text_vs_fig5")
save(cand_D(), "D_implied_share")
save(cand_E(), "E_calculation_flow")
print("\nfigures written to", HERE, "and", CAND)
Abstract labels the 39.65 million prevented CVD events as deaths; Results report 13.86 million deaths
PLoS Med (2022) · doi:10.1371/journal.pmed.1004111 · PMC9725126
Summary
This microsimulation study compares hypertension screening scenarios in low- and middle-income countries. Its abstract states that almost 40 million (39,650,363) HTN-induced deaths over 10 years could be prevented by increasing screening coverage and completion (scenario S4). The Results give that same number, 39,650,363 (14.2%), as prevented CVD events, and give S4 prevented deaths as 13,856,507 (13.7%); the Author summary agrees ("almost 40 million HTN-induced CVD events and 14 million (13.7%) related deaths"). The same abstract sentence also places 13,856,507 beside the 0.6% result for improved test accuracy (S2). By the paper's own percentages, 0.6% corresponds to roughly 0.61 million deaths. So the abstract's mortality headline is about 2.9 times the Results figure, and its absolute death count for the accuracy scenario is about 23 times what the Results imply. The version of record carries this text, and no correction notice was found (as of 2026-10-23).
What the paper reports
Abstract, "Methods and findings":
The study showed that while improving test accuracy avoids 0.6% of HTN-induced deaths over 10 years (13,856,507 [9,382,742; 17,395,833]), almost 40 million (39,650,363 [31,34,233, 49,298,921], i.e., 12.7% [9.9, 15.8]) of the HTN-induced deaths could be prevented by increasing coverage and completion of a screening event in the same time frame.
Abstract, the next sentence:
Doubling the coverage only would still prevent 3,304,212 million ([2,274,664; 4,164,180], 12.1% [8.3, 15.2]) CVD events 10 years after the rollout of the program.
Author summary, "What did the researchers do and find?":
While improving test accuracy avoids only 0.6% of HTN-induced deaths over 10 years, scaling up test coverage and completion can avoid almost 40 million HTN-induced CVD events and 14 million (13.7%) related deaths.
Results, paragraph 3 (CVD events, Fig 3, scenario S4):
In fact, it leads to a reduction of over 7 million (7,031,120; 26%) CVD events in year 10 compared to current standard of care (scenario S0). In sum, this equals 39,650,363 (14.2%) prevented CVD events over the period of 10 years.
Results, paragraph 4 (deaths, Fig 4):
Whereas survival does not improve much by increasing the test accuracy with 0.6% of HTN-induced deaths avoided over 10 years, we observe a high impact by doubling the coverage of the first screen (S3) with 6.6%. In year 10 alone, more than 2 million more lives (2,392,109) can be saved by additionally increasing completing probability in scenario S4, equalling to a total of 13,856,507 (13.7%) of the HTN-induced deaths over the 10 years.
Discussion, paragraph 1:
Whereas the improvement of the test accuracy had only a small potential impact on CVD events and related mortality over 10 years (reduction of 0.5% and 0.6%, respectively), by doubling of the coverage of a first screen, 6.6% of HTN-induced deaths could be avoided over a period of 10 years, and by increasing screen completion in addition, even 13.7% of deaths would not occur.
Discussion, paragraph 2:
Moreover, according to a review on available data on HTN in the literature [30], in 2015, an estimated 7.5 million deaths were attributable to systolic blood pressure in LMICs. This number coincides with the number of deaths in our model for the baseline (S0).
Methods, "Simulation study design", which treats events and deaths as separate endpoints:
We investigated the effect of blood pressure screening on the total number of CVD events and the resulting deaths within the population.
What the paper's own numbers show
Method. Each prevented count in the Results comes with its relative reduction against the base case S0. Dividing the count by that share gives the 10-year S0 total the paper is working from: S0 CVD events = 39,650,363 / 0.142 and S0 deaths = 13,856,507 / 0.137. These implied totals let us check each abstract number. For each one we ask which outcome and which percentage it fits. We also ask what S0 total the abstract's own label and percentage would require. The printed percentages are rounded to 0.1 points, so the implied totals are accurate to within about 1%. That is far smaller than any discrepancy below. All arithmetic is in recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| S4 prevented CVD events, 10 y | Results para. 3; Author summary | 39,650,363 (14.2%); "almost 40 million" events | Implied S0 CVD events = 279,227,908 (27.9 M per year). Cross-check: year-10 figures imply 27.04 M (S4) and 27.31 M (S3) S0 events in year 10 |
| S4 prevented deaths, 10 y | Results para. 4; Author summary; Discussion | 13,856,507 (13.7%); "14 million (13.7%)" deaths | Implied S0 deaths = 101,142,387 (10.11 M per year); deaths/events in S0 = 0.362 |
| S4 prevented deaths, 10 y | Abstract | 39,650,363, "12.7% [9.9, 15.8]" of HTN-induced deaths | 2.86 times the Results figure; 25,793,856 more deaths than the Results report |
| S0 deaths implied by the abstract's S4 sentence | Abstract | (39,650,363 at 12.7%) | 312,207,583 deaths (31.2 M per year). That is 1.118 times the implied S0 CVD events, so more deaths than events; and 4.16 times the 7.5 M per year the Discussion says the baseline matches (Results reading: 1.35 times) |
| S4 relative reduction | Abstract vs Results | 12.7% vs 14.2% (events) and 13.7% (deaths) | 39,650,363 / 0.127 = 312.2 M matches neither implied denominator (279.2 M or 101.1 M); gaps of 1.5 and 1.0 points |
| S2 (improved accuracy) prevented deaths, 10 y | Abstract | 0.6% "(13,856,507 [9,382,742; 17,395,833])" | 0.6% × 101,142,387 = 606,854 deaths. 13,856,507 / 101,142,387 = 0.137, the S4 share. Read as 0.6%, 13,856,507 would need 2.31 billion S0 deaths (22.8 times the implied baseline) |
| Lower bound of the S4 interval | Abstract | "31,34,233" | 7 digits; a value near 31 million needs 8. In the publisher's JATS XML, "31" and "34" are tagged as citations of references 31 and 34 |
| S3 prevented CVD events | Abstract vs Results para. 3 | "3,304,212 million" vs 3,304,212 | Read literally, the abstract says 3.30 × 10^12; the Results value is 3,304,212 |
The abstract's mortality sentence takes the S4 CVD-event count and labels it as deaths. It then moves the S4 death count next to the S2 accuracy percentage. The Author summary, Results and Discussion all agree with one another; only the abstract departs from them.
Figure
Figure 1. Left, in orange: the two clauses of the abstract's "Methods and findings" sentence, quoted verbatim. Right: the Results paragraphs (counted from the first Results paragraph as 1) and Author summary that report the same numbers. The upper connector links 39,650,363. The abstract calls it HTN-induced deaths (12.7%), while Results paragraph 3 calls it prevented CVD events (14.2%). Recomputed (blue), it is 2.86 times the S4 death count given in Results paragraph 4. The lower connector links 13,856,507. The abstract places it beside the 0.6% improved-accuracy result (S2), while Results paragraph 4 gives it as the S4 death total (13.7%). Recomputed from the paper's implied baseline of 101.1 million S0 deaths over 10 years, the S2 figure is 0.61 million. The Author summary (bottom right) gives about 40 million CVD events and 14 million deaths for S4, which agrees with the Results.
Figure 2. Deaths prevented over 10 years relative to S0, in millions, on a linear axis starting at zero. Orange: the counts the abstract attaches to each scenario's death result. For S4 (top), the open blue marker is the count in Results paragraph 4 (13.86 M); the abstract's 39.65 M is 2.9 times larger. For S2 (bottom), the filled blue marker is 0.6% of the S0 deaths implied by the Results (0.61 M); the abstract's 13.86 M is 22.8 times larger. The interval the abstract prints with 13,856,507 (9.38–17.40 M) excludes the S2 value implied by the Results.
Why it matters
The abstract's "Methods and findings" sentence is the paper's headline mortality estimate, and it is the figure most likely to be quoted. It says almost 40 million deaths could be prevented under S4. The paper's own Results put this at about 13.9 million, so the abstract overstates deaths averted by about 25.8 million. The same sentence gives the improved-accuracy scenario about 13.9 million averted deaths in absolute terms, when its 0.6% corresponds to about 0.6 million. A reader of the abstract alone would therefore overestimate the absolute mortality benefit of both interventions. The ratio between them would also be off: the abstract's numbers give about 2.9 to 1 (39.65 M vs 13.86 M), while the Results imply about 23 to 1 (13.86 M vs 0.61 M).
The direction and ranking of the scenarios do not change. Coverage plus completion still clearly outperforms improved accuracy, and the Author summary, Results and Discussion state the percentages consistently. The paper's qualitative conclusion, that screening completion and coverage matter more than test accuracy, is therefore unaffected. The error concerns the absolute numbers in the abstract and, for S4, its percentage. The abstract's 12.7% [9.9, 15.8] cannot be matched to any figure in the main text, so it is not clear what that interval refers to.
Explanations considered
- "Deaths" in the abstract might mean the same as "CVD events" in the Results. This does not fit. The Methods treat CVD events and "the resulting deaths" as separate endpoints. The Results report them in separate paragraphs with separate figures (Fig 3 events, Fig 4 deaths). The Author summary gives both numbers side by side ("almost 40 million HTN-induced CVD events and 14 million (13.7%) related deaths"). If 39,650,363 were deaths at 12.7%, the implied baseline would be 312.2 million deaths over 10 years. That exceeds the 279.2 million baseline CVD events implied by the Results. Deaths that follow CVD events cannot outnumber the events.
- A different population, subgroup or time window. The abstract sentence says "over 10 years" and "in the same time frame", matching the 10-year totals in the Results. No income-group or other subset is named. The abstract's own S3 clause ("10 years after the rollout") matches the Results' year-10 count, 3,304,212 (12.1%). The implied year-10 S0 events (27.3 M) agree with the S4 year-10 figure (27.0 M), so the abstract draws on the same model output as the Results.
- The 13,856,507 might really be the S2 death count. This does not fit. The Results give 13,856,507 explicitly as the S4 total (13.7%). At the S0 baseline implied by the Results, the S2 share of 0.6% is about 0.61 million. For 13,856,507 to be 0.6%, the S0 baseline would have to be about 2.31 billion deaths, against a simulated population of 3.15 billion adults.
- A different run or a revised analysis. No sensitivity analysis or alternative run in the article or S1 Text reports 39,650,363 as deaths or 12.7% as the S4 reduction. S1 Text gives no 10-year absolute totals, so it can neither confirm nor reconcile the 12.7%.
- A text-conversion artefact in PMC. This is ruled out for the labelling. The publisher's HTML and the JATS XML (via Europe PMC) carry the same wording. The malformed lower bound "31,34,233" does come from typesetting: in the XML, "31" and "34" are marked up as reference citations. That markup is unrelated to the deaths-versus-events labelling and the misplaced 13,856,507.
- The Discussion's 7.5 million deaths per year. This is a loose calibration check, not a precise target. The Results reading implies about 10.1 million S0 deaths per year, the same order as 7.5 million. The abstract reading implies about 31.2 million per year, about four times larger. This check is supportive only; the conclusion rests on the explicit Results labels.
Status
- Version of record. I checked the publisher's article page at journals.plos.org (via the DOI), the PMC record and the publisher's JATS XML from the Europe PMC full-text service. All carry the quoted text unchanged, including the abstract sentence, the Author summary and Results paragraphs 3–4.
- Supplements. The three supporting files are the S1 STRESS Checklist, S1 Text and S1 HPACC Consortium. S1 Text covers model methods, population scaling (3.15 billion), Table A, Fig A (yearly percentage heatmaps for events and deaths, as images) and Fig B (income-class stratification). It gives no 10-year absolute totals or denominators and does not resolve the discrepancy. No source data or code is linked.
- Corrections.
src/known.pyreturned no Crossref updates, no PMC notices and no Europe PMC notices. The Crossref record has noupdate-to,updated-byor relation entries. I found no correction, erratum, retraction or expression of concern. - PubPeer. The PubPeer search page and API returned HTTP 403 from this machine, so PubPeer could not be checked directly. A web search for the article title with "pubpeer" found no PubPeer thread.
- Sources. https://doi.org/10.1371/journal.pmed.1004111; https://pmc.ncbi.nlm.nih.gov/articles/PMC9725126/; https://www.ebi.ac.uk/europepmc/webservices/rest/PMC9725126/fullTextXML; https://api.crossref.org/works/10.1371/journal.pmed.1004111; S1 Text (journal.pmed.1004111.s002.docx).
- As of 2026-10-23.
Suggested correction
In the abstract's "Methods and findings", align the mortality sentence with the Results. For example: "while improving test accuracy avoids 0.6% of HTN-induced deaths over 10 years, 13,856,507 [9,382,742; 17,395,833] (13.7%) of the HTN-induced deaths and almost 40 million CVD events (39,650,363 [lower bound; 49,298,921], i.e., 14.2%) could be prevented by increasing coverage and completion of a screening event in the same time frame." Here "lower bound" stands for the correct lower credible bound, which is not printed anywhere in the article. The authors should also confirm:
- The absolute number of deaths avoided under S2 (about 0.6 million by the paper's own percentages), with its credible interval.
- Which percentage and interval belong to the S4 CVD-event reduction (the Results give 14.2%; the abstract's 12.7% [9.9, 15.8] matches no reported figure).
- The S4 lower credible bound, garbled as "31,34,233".
- "3,304,212 million" corrected to "3,304,212" CVD events (S3, year 10).
Recomputation script (python recompute.py.txt)
"""Recompute the abstract-vs-Results figures of PMC9725126
(PLoS Med 2022, doi:10.1371/journal.pmed.1004111).
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.
All inputs are copied from the article as archived in PMC (identical in the
publisher's JATS XML):
Abstract (Methods and findings), Author summary, Results paragraphs 3-4 (Fig 3 events, Fig 4 deaths),
Discussion paragraphs 1-2.
Scenario labels: S0 base case, S2 improved accuracy, S3 increased coverage,
S4 increased coverage + improved completion. All totals are over 10 years.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs
# Results paragraph 3 (CVD events, Fig 3)
S4_EVENTS_10Y = 39_650_363 # "39,650,363 (14.2%) prevented CVD events over the period of 10 years"
S4_EVENTS_PCT = 0.142
S4_EVENTS_Y10 = 7_031_120 # "reduction of over 7 million (7,031,120; 26%) CVD events in year 10"
S4_EVENTS_Y10_PCT = 0.26
S3_EVENTS = 3_304_212 # "3,304,212 prevented CVD events (12.1%) ... 10 years after a rollout"
S3_EVENTS_PCT = 0.121
# Results paragraph 4 (deaths, Fig 4)
S4_DEATHS_10Y = 13_856_507 # "a total of 13,856,507 (13.7%) of the HTN-induced deaths over the 10 years"
S4_DEATHS_PCT = 0.137
S2_DEATHS_PCT = 0.006 # "0.6% of HTN-induced deaths avoided over 10 years"
S3_DEATHS_PCT = 0.066 # "(S3) with 6.6%"
# Abstract
ABS_S4_DEATHS = 39_650_363 # "almost 40 million (39,650,363 [...], i.e., 12.7% [9.9, 15.8]) of the HTN-induced deaths"
ABS_S4_PCT = 0.127
ABS_S4_LOWER_TEXT = "31,34,233" # printed lower bound of that interval
ABS_S4_UPPER = 49_298_921
ABS_S2_DEATHS = 13_856_507 # "improving test accuracy avoids 0.6% ... (13,856,507 [9,382,742; 17,395,833])"
ABS_S2_CI = (9_382_742, 17_395_833)
ABS_S3_TEXT_MILLION = 3_304_212 # "3,304,212 million ([2,274,664; 4,164,180], 12.1% [8.3, 15.2]) CVD events"
# Author summary: "almost 40 million HTN-induced CVD events and 14 million (13.7%) related deaths"
SUMMARY_EVENTS_M, SUMMARY_DEATHS_M, SUMMARY_DEATHS_PCT = 40, 14, 0.137
# Discussion paragraph 2: 7.5 million deaths attributable to SBP in LMICs (2015),
# which "coincides with the number of deaths in our model for the baseline (S0)"
DISC_ANNUAL_DEATHS = 7.5e6
M = 1e6
print("=== 1. Denominators implied by the Results (baseline S0, 10 years) ===")
base_events = S4_EVENTS_10Y / S4_EVENTS_PCT
base_deaths = S4_DEATHS_10Y / S4_DEATHS_PCT
print(f"S0 CVD events = 39,650,363 / 0.142 = {base_events:,.0f} ({base_events / 10 / M:.1f} M per year on average)")
print(f"S0 deaths = 13,856,507 / 0.137 = {base_deaths:,.0f} ({base_deaths / 10 / M:.2f} M per year on average)")
print(f"S0 deaths / S0 CVD events = {base_deaths / base_events:.3f}")
print(f"cross-check, year 10: S4 7,031,120 / 0.26 = {S4_EVENTS_Y10 / S4_EVENTS_Y10_PCT:,.0f} S0 events in year 10; "
f"S3 3,304,212 / 0.121 = {S3_EVENTS / S3_EVENTS_PCT:,.0f}")
print(f"Author summary rounding: 39,650,363 -> {S4_EVENTS_10Y / M:.1f} M ('almost {SUMMARY_EVENTS_M} million' events); "
f"13,856,507 -> {S4_DEATHS_10Y / M:.1f} M ('{SUMMARY_DEATHS_M} million', {SUMMARY_DEATHS_PCT:.1%} deaths)")
print("\n=== 2. Abstract S4 sentence: 39,650,363 labelled as deaths ===")
ratio = ABS_S4_DEATHS / S4_DEATHS_10Y
diff = ABS_S4_DEATHS - S4_DEATHS_10Y
print(f"abstract deaths / Results deaths = 39,650,363 / 13,856,507 = {ratio:.2f}")
print(f"difference = {diff:,} deaths")
implied_deaths_abs = ABS_S4_DEATHS / ABS_S4_PCT
print(f"if 39,650,363 were deaths at 12.7%: implied S0 deaths = {implied_deaths_abs:,.0f} "
f"({implied_deaths_abs / 10 / M:.1f} M per year)")
print(f" -> exceeds the implied S0 CVD events ({base_events:,.0f}) by a factor {implied_deaths_abs / base_events:.3f}")
print(f" -> per-year value vs Discussion's 7.5 M: x{implied_deaths_abs / 10 / DISC_ANNUAL_DEATHS:.2f}; "
f"Results reading: x{base_deaths / 10 / DISC_ANNUAL_DEATHS:.2f}")
print(f"abstract 12.7% vs Results 14.2% (events) and 13.7% (deaths): "
f"gaps {100 * (S4_EVENTS_PCT - ABS_S4_PCT):.1f} and {100 * (S4_DEATHS_PCT - ABS_S4_PCT):.1f} points")
print(f"39,650,363 / 0.127 = {S4_EVENTS_10Y / ABS_S4_PCT:,.0f} matches neither {base_events:,.0f} nor {base_deaths:,.0f}")
print("\n=== 3. Abstract S2 sentence: 13,856,507 beside 0.6% ===")
s2_deaths = S2_DEATHS_PCT * base_deaths
print(f"0.6% x {base_deaths:,.0f} = {s2_deaths:,.0f} deaths (S2, implied)")
print(f"13,856,507 / {base_deaths:,.0f} = {S4_DEATHS_10Y / base_deaths:.3f} (the S4 share, 13.7%)")
print(f"13,856,507 as 0.6% would imply S0 deaths = {ABS_S2_DEATHS / S2_DEATHS_PCT:,.0f} "
f"({ABS_S2_DEATHS / S2_DEATHS_PCT / base_deaths:.1f} x the Results-implied baseline)")
print(f"abstract S2 count / implied S2 count = {ABS_S2_DEATHS / s2_deaths:.1f}")
print(f"for reference, S3 deaths at 6.6% = {S3_DEATHS_PCT * base_deaths:,.0f}")
print(f"abstract interval printed with 13,856,507: {ABS_S2_CI[0]:,}-{ABS_S2_CI[1]:,}; "
f"its lower end is {ABS_S2_CI[0] / s2_deaths:.1f} x the implied S2 point value")
print("\n=== 4. Typesetting / unit slips in the same abstract paragraph ===")
digits = len(ABS_S4_LOWER_TEXT.replace(",", ""))
print(f"lower bound '{ABS_S4_LOWER_TEXT}': {digits} digits (a value near 31 million needs 8); "
f"upper bound {ABS_S4_UPPER:,}")
print(f"'3,304,212 million' read literally = {ABS_S3_TEXT_MILLION * M:,.0f}; Results value {S3_EVENTS:,}")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# fig1: abstract vs Results/Author summary, two text panels with connectors
fig = plt.figure(figsize=(7.2, 5.6))
fig.patch.set_facecolor(BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
def box(x, y, w, h, edge, title, body, title_col):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
facecolor="white", edgecolor=edge, linewidth=1.5))
ax.text(x + 1.5, y + h - 1.8, title, fontsize=10, fontweight="bold", color=title_col, va="top")
ax.text(x + 1.5, y + h - 7.0, body, fontsize=10, color=INK, va="top", linespacing=1.35)
ax.text(21, 96, "Abstract (Methods and findings)", ha="center", fontsize=11, fontweight="bold", color=ORANGE)
ax.text(79, 96, "Results and Author summary", ha="center", fontsize=11, fontweight="bold", color=INK)
LX, RX, W = 1.5, 59.5, 39
box(LX, 64, W, 27, ORANGE, "S4 sentence",
"“almost 40 million (39,650,363\n[31,34,233, 49,298,921], i.e.,\n12.7% [9.9, 15.8]) of the\n"
"HTN-induced deaths could be\nprevented”", ORANGE)
box(RX, 64, W, 27, INK2, "Results paragraph 3 (S4)",
"“In sum, this equals 39,650,363\n(14.2%) prevented CVD events\nover the period of 10 years.”", INK)
box(LX, 30, W, 27, ORANGE, "S2 sentence",
"“while improving test accuracy\navoids 0.6% of HTN-induced\ndeaths over 10 years\n"
"(13,856,507 [9,382,742;\n17,395,833])”", ORANGE)
box(RX, 30, W, 27, INK2, "Results paragraph 4 (S4)",
"“equalling to a total of\n13,856,507 (13.7%) of the\nHTN-induced deaths over the\n10 years.”", INK)
box(RX, 3, W, 20, INK2, "Author summary (S4)",
"“almost 40 million HTN-induced\nCVD events and 14 million\n(13.7%) related deaths”", INK)
def connector(y, line1, line2, line3):
ax.annotate("", xy=(RX - 0.6, y), xytext=(LX + W + 0.6, y),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.5))
ax.text(50, y + 1.5, line1, ha="center", va="bottom", fontsize=10, fontweight="bold", color=INK)
ax.text(50, y - 1.5, line2, ha="center", va="top", fontsize=10, color=INK2)
ax.text(50, y - 9.0, line3, ha="center", va="top", fontsize=10, color=BLUE)
connector(80, "same number", "deaths vs\nCVD events", f"recomputed:\n{ratio:.2f}× the\nS4 deaths")
connector(46, "same number", "S2 (0.6%) vs\nS4 (13.7%)", f"recomputed:\nS2 = {s2_deaths / M:.2f} M\ndeaths")
ax.text(21, 18, f"Abstract quotes in orange.\nBlue: recomputed from the\npaper's own Results numbers\n"
f"(S0 deaths = 13,856,507 / 0.137\n= {base_deaths / M:.1f} M over 10 years).",
ha="center", va="center", fontsize=10, color=INK2, linespacing=1.35)
save(fig, "fig1")
# fig2: deaths prevented over 10 years, abstract vs Results / recomputed
fig, ax = plt.subplots(figsize=(7.2, 3.5))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
rows = [("S2: improved test accuracy", ABS_S2_DEATHS, s2_deaths, "recomputed: 0.6% × S0 deaths"),
("S4: coverage + completion", ABS_S4_DEATHS, S4_DEATHS_10Y, "Results / Author summary")]
for i, (lab, rep, alt, altlab) in enumerate(rows):
y = i
ax.plot([alt / M, rep / M], [y, y], color=GRID, lw=2, zorder=1)
ax.scatter(rep / M, y, s=90, color=ORANGE, zorder=3, label="Abstract" if i == 0 else None)
if i == 0:
ax.scatter(alt / M, y, s=90, color=BLUE, zorder=3, label="Recomputed: 0.6% of S0 deaths")
else:
ax.scatter(alt / M, y, s=90, facecolor="white", edgecolor=BLUE, linewidth=2, zorder=3,
label="Results paragraph 4")
ax.text(rep / M, y + 0.2, f"{rep / M:.2f} M", ha="center", va="bottom", color=ORANGE, fontsize=10)
ax.text(alt / M, y + 0.2, f"{alt / M:.2f} M", ha="center", va="bottom", color=BLUE, fontsize=10)
ax.text((rep + alt) / 2 / M, y - 0.3, f"×{rep / alt:.1f}", ha="center", va="top", color=INK2, fontsize=10)
ax.set_yticks([0, 1])
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_ylim(-0.7, 1.55)
ax.set_xlim(0, 45)
ax.set_xlabel("HTN-induced deaths prevented over 10 years vs S0 (millions)", fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
fig.tight_layout(rect=(0, 0, 1, 0.88))
fig.legend(loc="upper center", ncol=3, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.99))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png to", HERE)
Abstract reverses the direction of the Alpha versus B.1.177 viral-burden difference reported in Table 1
PLoS Pathog (2023) · doi:10.1371/journal.ppat.1011461 · PMC10449197
Summary
The Abstract states that, among unvaccinated people with no known prior exposure, viral burden was "44% lower among Alpha variant infections" than among B.1.177 infections, and one Results sentence says the same. The paper's own coefficient points the other way. In Table 1, B.1.177 has a Beta score of +1.00 Ct against the Alpha reference, so B.1.177 samples had higher Ct values. Under the paper's stated rule (3 Ct = 10-fold change in viral load), Alpha burden is 10^(1/3) = 2.15 times that of B.1.177: 115% higher, or equivalently B.1.177 is 54% lower. The Results subsection heading, the sensitivity analysis and the Discussion ("115% higher among Alpha") all agree with this. The Abstract's first quantitative finding therefore has the wrong sign and the wrong size, and the publisher correction issued in October 2023 does not address it. A second sentence, in the sensitivity analysis, reverses the Delta versus Alpha direction among vaccinated people, and several smaller conversions in the text also disagree with Table 1.
What the paper reports
Abstract:
By analysing the adjusted Ct values using partial least squares regression, we found that among unvaccinated individuals with no known prior exposure, viral burden was 44% lower among Alpha variant infections, compared to those with the predecessor strain, B.1.177.
Results, subsection headed "Among unvaccinated individuals with no known prior exposure, viral burden was higher among Alpha compared to B.1.177 infections":
For samples sequenced at Oxford, Ct values in this group were higher for B.1.177 samples (Beta score = 1.00, VIP = 1.05) compared to Alpha. This corresponds to a 44% reduction in viral loads with Alpha.
Table 1, block "Variant amongst unvaccinated individuals with no known prior exposure" (columns: Oxford N | Beta score (viral load factor change*) | VIP):
Ref = Alpha (Oxford), Delta (Northumbria) | 3526
B.1.177 | 658 | 1.00 (2.15) | 1.05
Table 1 caption footnote:
*based upon a Ct value decrease of 3 being equivalent to a 10-fold increase in viral load [34].
Results, sensitivity analysis (subsection "The association between novel variants and higher viral burden is robust to assumptions about the shape of the within-host viral trajectory"):
It is noteworthy that the VIP value remained greater than unity across plausible parameter combinations, providing support for the observation that viral burden is higher in infections with the Alpha-variant compared to B.1.177, among unvaccinated individuals with no known prior exposure.
The second paragraph of the same subsection, on Delta versus Alpha among vaccinated individuals:
This analysis therefore provides support for the finding that samples with the Delta variant had a lower viral burden compared to samples with the Alpha variant among vaccinated individuals. In summary, our sensitivity analysis supports the finding that both the shift from B.1.177 to Alpha and the shift from Alpha to Delta was associated with an increase in viral burden amongst the population (mostly unvaccinated and mostly vaccinated, respectively) that prevailed at each stage.
Discussion, third paragraph:
Among individuals with no known prior immunity, viral burden was, on average, 115% higher among Alpha-variant compared to B.1.177 samples.
The same paragraph and the Abstract also give the other conversions checked below:
We found that viral burden decreased over time and was higher among older individuals (13% and 17% higher for every 10 year age increase, in the samples sequenced in Oxford and Northumbria, respectively).
Vaccination reduced viral burden, with the average reduction being 75% and 30% amongst the samples sequenced at Oxford and Northumbria, respectively.
Abstract:
Vaccination reduced viral burden by 67%, and among vaccinated individuals, viral burden was 286% higher among Delta variant, compared to Alpha variant, infections.
Results, subsection "Viral burden is higher among older individuals":
Based upon a 3-point decrease in Ct value being equivalent to a 10-fold increase in viral load (see [34]), these age effects equate to an approximate 14% and 17% increase in viral load for every 10 year age increase, respectively.
What the paper's own numbers show
Method. A Beta score is the difference in (epidemiologically adjusted) Ct between a category and its reference. The paper converts it to a viral-load factor with the rule in the Table 1 footnote and the Results: a Ct decrease of 3 is a 10-fold increase, so factor (category / reference) = 10^(−Beta/3). I applied this rule to all 27 Beta scores in Table 1 and compared the result with the factor printed beside each one. I then converted every percentage in the text back to the Beta it implies. The PLS model itself cannot be re-run: the authors' public repository holds MATLAB code but no Ct data. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Ct(B.1.177) − Ct(Alpha), unvaccinated, Oxford | Table 1; Results | Beta = 1.00 ("Ct values ... were higher for B.1.177") | B.1.177 has higher Ct, so lower viral burden |
| Viral load, Alpha / B.1.177 | Table 1 | factor 2.15 | 10^(1.00/3) = 2.154 |
| Alpha relative to B.1.177 | Discussion | "115% higher among Alpha" | +115.4% (matches) |
| Alpha relative to B.1.177 | Abstract; Results sentence | "44% lower among Alpha"; "44% reduction in viral loads with Alpha" | +115.4%: opposite direction |
| B.1.177 relative to Alpha | (the reverse reading) | — | −53.6%, not −44% |
| Beta implied by "44% lower among Alpha" | derived | — | 3·log10(0.56) = −0.76, opposite sign to the printed +1.00 |
| Table 1 factors that follow 10^(−Beta/3) | Table 1, all rows | 27 printed factors | 26 of 27 match to two decimals; the exception is this B.1.177 row, where 0.46 is expected and 2.15 = 1/0.464 is printed |
The other conversions in the text were checked the same way. These are smaller and do not change a conclusion:
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Vaccinated vs no known prior exposure, Oxford (Beta 1.44) | Table 1; Abstract; Results; Discussion | 0.33; "67%" reduction (Abstract, Results); "75%" (Discussion) | factor 0.331, a 66.9% reduction; the Discussion's 75% does not match |
| Vaccinated vs no known prior exposure, Northumbria (Beta 0.34) | Table 1; Discussion | 0.77; "30%" reduction | factor 0.770, a 23.0% reduction |
| Delta vs Alpha among vaccinated, Oxford (Beta −1.37) | Table 1; Abstract; Results; Discussion | 2.86; "286% higher" | factor 2.862, which is 186% higher (286% of Alpha's level) |
| Age per 10 years, Oxford (Beta −0.016 per year) | Table 1; Results; Discussion | 1.13; "14%" (Results); "13%" (Discussion) | factor 1.131, +13.1% |
| Age per 10 years, Northumbria (Beta −0.021 per year) | Table 1; Abstract; Results; Discussion | 1.17; "17%" | factor 1.175, +17.5% (consistent) |
| Direction of Delta vs Alpha among vaccinated, Oxford (Beta −1.37) | Results, sensitivity-analysis subsection, second paragraph | "samples with the Delta variant had a lower viral burden compared to samples with the Alpha variant among vaccinated individuals" | factor 2.862: Delta higher. Table 1, the Abstract ("286% higher among Delta"), the Results subsection on vaccinated individuals, the Discussion and the next sentence of the same paragraph ("the shift from Alpha to Delta was associated with an increase in viral burden") all say higher |
Figure
Why it matters
The Alpha versus B.1.177 comparison is the first quantitative finding in the Abstract, which is the part of the paper most readers and indexing services see. As printed, it tells readers that the Alpha variant was associated with lower within-host viral burden than its predecessor among unvaccinated people. The paper's coefficient, its Table 1 factor, its section heading, its sensitivity analysis and its Discussion all report the opposite: Alpha was associated with about 2.15-fold (115%) higher burden. A reader who relies on the Abstract, or who cites the Results sentence, would carry forward the reversed direction and a magnitude (44%) that no conversion of the published coefficient under the paper's rule produces.
The paper's broader narrative, that each new variant brought higher viral burden in the population then prevailing (B.1.177 to Alpha among the mostly unvaccinated, Alpha to Delta among the mostly vaccinated), is supported by its own analysis and survives once the Abstract is corrected. The Abstract's closing summary ("is influenced by the interplay of vaccination status and viral variant") states no direction and is not affected. The secondary discrepancies matter less. The sensitivity-analysis sentence that Delta had "a lower viral burden" than Alpha among vaccinated people contradicts the summary sentence that immediately follows it and every other statement of that result, so a careful reader can resolve it, but a reader quoting that sentence alone would carry the reversed direction. In addition, "286% higher" for a 2.86-fold factor overstates the Delta effect by 100 percentage points in the Abstract, and the Discussion's vaccination reductions (75% and 30%) are larger than the Table 1 values (67% and 23%).
Explanations considered
- The sentence names the wrong variant as the lower one, and 44% describes B.1.177 relative to Alpha. This accounts for the direction but not the size. Under the paper's rule B.1.177 is 53.6% lower than Alpha, not 44%. A 44% deficit for B.1.177 would need a Beta of +0.76, not +1.00.
- A different Ct-to-load rule. Only a 4-Ct-per-10-fold rule gives about 44% from a 1.00 Ct difference (1 − 10^(−1/4) = 43.8%), and even then it is B.1.177 that is lower. The paper states and uses the 3-Ct rule throughout: with 4 Ct the vaccination effect would be a 56.3% reduction (text: 67%), the Delta factor 2.20 (Table 1: 2.86) and the Northumbria age effect 12.8% (text: 17%). Other common rules (3.3 Ct per 10-fold, or 1 Ct per doubling) give 50.2% and 50.0%. None of them makes Alpha the lower one.
- The Beta sign convention is the reverse of what I assumed. This does not account for the numbers. The same Results sentence says "Ct values in this group were higher for B.1.177 samples (Beta score = 1.00 ...) compared to Alpha", and every other positive Beta in Table 1 (for example Vaccinated, 1.44) is printed with a factor below 1, meaning lower burden.
- The Abstract reports a different model or sensitivity setting. No such result appears in the paper. The sensitivity analysis states that the Beta for B.1.177 decreased under some trajectory assumptions, but that the evidence remained "support for the observation that viral burden is higher in infections with the Alpha-variant compared to B.1.177"; it reports no setting in which Alpha is lower.
- Carry-over from the preprint. The bioRxiv preprint abstract (v1, 2 December 2022, read through the bioRxiv API) states that "the average Ct value was 0.94 lower among Alpha variant infections, compared those with the predecessor strain, B.1.177", which is the same direction as Table 1 (Alpha 2.06-fold higher under the 3-Ct rule). The preprint abstract therefore does not contain the reversed statement. The preprint's full text could not be downloaded in this session, so the origin of the 44% figure could not be traced.
- A conversion or typesetting artifact in PMC. Excluded. The publisher's HTML and printable PDF and the Europe PMC XML carry the same Abstract, Results sentence and Table 1 row as the PMC archive.
Status
- Version of record. Checked on the publisher's site (journals.plos.org, HTML and the 24-page printable PDF) and in Europe PMC's full-text XML. All carry the Abstract sentence, the Results sentence ("44% reduction in viral loads with Alpha"), the Table 1 row "B.1.177 658 1.00 (2.15) 1.05" and the Discussion's "115% higher". Every quote above was confirmed verbatim against the PMC-archived text.
- Supplements and code. S1 Table (review of earlier studies), S2 Table (VIF values), S1 Fig (PLS error and loading plots), S2 Fig (distributions of the Fig 2 data), S1 Data (ENA accession numbers) and S1 Text (consortium members) contain no variant coefficients or viral-burden conversions. The code repository linked in the paper's Methods (Implementation of analysis) holds MATLAB scripts and a README, without Ct data or results.
- Corrections. One correction exists: doi:10.1371/journal.ppat.1011706 (PMC10558069, published 2023-10-06), listed by Crossref and Europe PMC. It corrects formatting errors in the sixth and ninth Methods equations, in-text expressions on page 20, errors in the Fig 4 caption, and a figure reference (Fig 4C/4F to 4E/4F) in the Results. It does not mention the Abstract, the Alpha versus B.1.177 percentage or Table 1. No retraction or expression of concern was found.
- Comments. PubPeer's API and search page both returned a Cloudflare block (HTTP 403) in this session, so PubPeer could not be checked. A web search for the DOI with "44%" and "B.1.177" returned only the article and repository mirrors repeating the Abstract.
- Sources. PMC10449197; https://journals.plos.org/plospathogens/article?id=10.1371/journal.ppat.1011461; correction PMC10558069; Crossref; Europe PMC; bioRxiv API record for doi:10.1101/2022.12.02.518847. As of 2026-10-23.
Suggested correction
- Abstract: replace "viral burden was 44% lower among Alpha variant infections, compared to those with the predecessor strain, B.1.177" with a statement consistent with Table 1, for example "viral burden was 115% higher (2.15-fold) among Alpha variant infections, compared to those with the predecessor strain, B.1.177".
- Results: change "This corresponds to a 44% reduction in viral loads with Alpha" to "This corresponds to a 115% increase in viral loads with Alpha (equivalently, a 54% reduction with B.1.177)".
- Table 1: print the B.1.177 factor in the same orientation as the other rows (0.46, B.1.177 relative to Alpha), or footnote that 2.15 is Alpha relative to B.1.177.
- Abstract, Results and Discussion: describe the Delta versus Alpha effect as "186% higher" or "2.86-fold", not "286% higher".
- Results, sensitivity-analysis subsection, second paragraph: change "samples with the Delta variant had a lower viral burden compared to samples with the Alpha variant among vaccinated individuals" to "... had a higher viral burden ...".
- Discussion: change the vaccination reductions from 75% and 30% to 67% and 23%, and align the Oxford age effect (14% in the Results, 13% in the Discussion; 13.1% from Table 1).
Recomputation script (python recompute.py.txt)
"""Recompute the Alpha vs B.1.177 viral-burden statements of PMC10449197
(PLoS Pathog 2023, doi:10.1371/journal.ppat.1011461).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and writes
five candidate figure designs (candidates/cand_A..E.svg/png) next to this file, for the user
to choose from. No design is embedded until one is chosen:
python recompute.py.txt --pick A,E # writes fig1 = cand_A, fig2 = cand_E and prints captions
The candidates/ folder is review material only and is not part of what goes to the journal.
Inputs are copied from the paper as archived in PMC (Table 1 and the text).
Conversion rule stated in the paper (Table 1 footnote; Results, "Viral burden is higher
among older individuals"): a Ct decrease of 3 is a 10-fold increase in viral load, so a
Beta score b (Ct of the category minus Ct of the reference) corresponds to a viral-load
factor (category / reference) of 10 ** (-b / 3).
"""
import math
import os
import sys
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2
def factor(beta):
"""Viral-load factor (category / reference) for a Ct difference beta."""
return 10 ** (-beta / 3)
# ------------------------------------------------------------------ inputs (Table 1)
# (label, site, beta as printed, multiplier to the printed unit, printed factor)
TABLE1 = [
("Sample date (per year)", "Oxford", 0.008, 1, 0.99),
("Sample date (per year)", "Northumbria", 0.013, 1, 0.99),
("Age (per 10 years)", "Oxford", -0.016, 10, 1.13),
("Age (per 10 years)", "Northumbria", -0.021, 10, 1.17),
("Known prior exposure", "Oxford", 1.56, 1, 0.30),
("Known prior exposure", "Northumbria", -0.12, 1, 1.10),
("Vaccinated", "Oxford", 1.44, 1, 0.33),
("Vaccinated", "Northumbria", 0.34, 1, 0.77),
("B.1.177 vs Alpha, unvaccinated", "Oxford", 1.00, 1, 2.15),
("Delta vs Alpha, unvaccinated", "Oxford", -0.01, 1, 1.01),
("BA.1 vs Delta, unvaccinated", "Northumbria", -0.95, 1, 2.07),
("B.1.177 vs Alpha, vaccinated", "Oxford", 1.82, 1, 0.25),
("Delta vs Alpha, vaccinated", "Oxford", -1.37, 1, 2.86),
("BA.1 vs Delta, vaccinated", "Northumbria", -0.59, 1, 1.57),
("B.1.177 vs Alpha, prior exposure", "Oxford", 5.17, 1, 0.02),
("Delta vs Alpha, prior exposure", "Oxford", -3.05, 1, 10.39),
("BA.1 vs Delta, prior exposure", "Northumbria", -0.35, 1, 1.31),
("AstraZeneca vs Pfizer", "Oxford", -0.19, 1, 1.16),
("AstraZeneca vs Pfizer", "Northumbria", -0.10, 1, 1.08),
(">=2 doses vs 1", "Oxford", -0.09, 1, 1.07),
(">=2 doses vs 1", "Northumbria", 0.07, 1, 0.95),
("Other ethnicities vs White", "Oxford", 0.24, 1, 0.83),
("Other ethnicities vs White", "Northumbria", 0.46, 1, 0.70),
("Male vs female", "Oxford", -0.11, 1, 1.09),
("Male vs female", "Northumbria", 0.50, 1, 0.68),
("Health care worker", "Oxford", 0.44, 1, 0.71),
("Health care worker", "Northumbria", 0.28, 1, 0.81),
]
BETA_B177 = 1.00 # Table 1: B.1.177 vs Alpha (ref), unvaccinated, no known prior exposure, Oxford
ABSTRACT_PCT_LOWER = 44 # Abstract and Results: "44% lower among Alpha" / "44% reduction ... with Alpha"
DISCUSSION_PCT_HIGHER = 115
# ------------------------------------------------------------------ 1. main comparison
print("== 1. B.1.177 vs Alpha, unvaccinated, no known prior exposure (Oxford) ==")
b177_over_alpha = factor(BETA_B177)
alpha_over_b177 = 1 / b177_over_alpha
print(f"Beta (Ct B.1.177 - Ct Alpha) = {BETA_B177:+.2f}")
print(f"B.1.177 / Alpha viral load = 10^(-1/3) = {b177_over_alpha:.4f}")
print(f"Alpha / B.1.177 viral load = 10^(+1/3) = {alpha_over_b177:.4f} (Table 1 prints 2.15)")
print(f"Alpha relative to B.1.177 = {100 * (alpha_over_b177 - 1):+.1f}% (Discussion: +115%)")
print(f"B.1.177 relative to Alpha = {100 * (b177_over_alpha - 1):+.1f}%")
abs_ratio = 1 - ABSTRACT_PCT_LOWER / 100
# Alpha/B.1.177 = 0.56 -> Ct(Alpha) - Ct(B.1.177) = -3*log10(0.56) -> beta = 3*log10(0.56)
beta_implied_abs = 3 * math.log10(abs_ratio)
print(f"Abstract: Alpha / B.1.177 = {abs_ratio:.2f}")
print(f" implied Beta (Ct B.1.177 - Ct Alpha) = {beta_implied_abs:+.3f} (Table 1: +1.00)")
beta_implied_disc = -3 * math.log10(1 / (1 + DISCUSSION_PCT_HIGHER / 100))
print(f"Discussion: Alpha / B.1.177 = {1 + DISCUSSION_PCT_HIGHER / 100:.2f}; implied Beta = {beta_implied_disc:+.3f}")
beta_b177_44_lower = -3 * math.log10(abs_ratio)
print(f"Beta needed for 'B.1.177 44% lower than Alpha' = {beta_b177_44_lower:+.3f}")
print("\n-- Conversions that might give 44% from Beta = 1.00 --")
cands = [
("3 Ct per 10-fold, B.1.177 lower than Alpha (paper's rule)", 1 - 10 ** (-1 / 3)),
("3.3 Ct per 10-fold", 1 - 10 ** (-1 / 3.3)),
("1 Ct = 2-fold", 1 - 2 ** -1),
("4 Ct per 10-fold", 1 - 10 ** (-1 / 4)),
]
for lab, v in cands:
print(f" {lab:58s} -> {100 * v:.1f}% lower")
print(" (none of these makes Alpha the lower one; only the 4-Ct rule gives ~44%)")
print(" Other text percentages under a 4-Ct rule (check whether the paper used it):")
print(f" Vaccinated Oxford, Beta 1.44 -> {100 * (1 - 10 ** (-1.44 / 4)):.1f}% reduction (text: 67%)")
print(f" Delta vs Alpha vacc, Beta -1.37 -> factor {10 ** (1.37 / 4):.2f} (Table 1: 2.86)")
print(f" Age Northumbria, -0.21 per 10y -> {100 * (10 ** (0.21 / 4) - 1):.1f}% (text: 17%)")
# preprint (bioRxiv v1, 2022-12-02) abstract: Ct "0.94 lower among Alpha ... compared [to] B.1.177"
pre = 0.94
print(f"\nPreprint abstract: Ct 0.94 lower with Alpha -> Alpha/B.1.177 = {10 ** (pre / 3):.2f} "
f"(+{100 * (10 ** (pre / 3) - 1):.0f}%); B.1.177 {100 * (1 - 10 ** (-pre / 3)):.1f}% lower")
# ------------------------------------------------------------------ 2. Table 1 consistency
print("\n== 2. Table 1: printed factor vs 10^(-Beta/3) for every entry ==")
rows = []
for lab, site, b, mult, printed in TABLE1:
calc = factor(b * mult)
recip = abs(printed - round(1 / calc, 2)) <= 0.011
ok = abs(printed - round(calc, 2)) <= 0.011
rows.append((lab, site, b * mult, calc, printed, ok))
flag = "matches" if ok else ("RECIPROCAL" if recip else "MISMATCH")
print(f" {lab:34s} {site:11s} beta={b * mult:+.3f} 10^(-b/3)={calc:7.3f} printed={printed:6.2f} {flag}")
n_ok = sum(r[5] for r in rows)
print(f" {n_ok} of {len(rows)} entries match 10^(-Beta/3) to 2 decimals; the exception is "
f"B.1.177 vs Alpha, unvaccinated (expected {factor(BETA_B177):.2f}, printed 2.15 = 1/{factor(BETA_B177):.3f})")
# ------------------------------------------------------------------ 3. other text percentages
print("\n== 3. Other percentages in the text ==")
pcts = [
("Vaccination, Oxford (reduction)", 1.44, "Abstract/Results 67%; Discussion 75%", "red"),
("Vaccination, Northumbria (reduction)", 0.34, "Discussion 30%", "red"),
("Delta vs Alpha, vaccinated, Oxford", -1.37, "Abstract/Results/Discussion '286% higher'", "inc"),
("Age per 10 y, Oxford", -0.16, "Results 14%; Discussion 13%", "inc"),
("Age per 10 y, Northumbria", -0.21, "Abstract/Results/Discussion 17%", "inc"),
]
for lab, b, txt, kind in pcts:
f = factor(b)
if kind == "red":
print(f" {lab:38s} factor {f:.3f} -> {100 * (1 - f):.1f}% reduction [{txt}]")
else:
print(f" {lab:38s} factor {f:.3f} -> {100 * (f - 1):.1f}% higher [{txt}]")
print("\n== 4. Direction of the Delta vs Alpha effect among vaccinated (Oxford) ==")
f_delta = factor(-1.37)
print(f" Beta (Ct Delta - Ct Alpha) = -1.37 -> Delta / Alpha viral load = {f_delta:.3f} (Table 1: 2.86)")
print(f" Delta is {'HIGHER' if f_delta > 1 else 'LOWER'} than Alpha by {100 * (f_delta - 1):.0f}%; "
"the sensitivity-analysis sentence says Delta had a 'lower viral burden'")
# ================================================================== figures
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def wrap(s, n):
return "\n".join(textwrap.wrap(s, n))
def box(ax, x, y, w, h, edge, lw=1.4, fc="white"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc=fc, ec=edge, lw=lw, transform=ax.transAxes))
def fig_A():
"""Two text panels joined by an annotated connector."""
fig = plt.figure(figsize=(W, 5.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.text(0.03, 0.965, "Alpha vs B.1.177: unvaccinated, no known prior exposure (Oxford)",
fontsize=11.5, weight="bold", va="top")
LQ, RQ = "\u201c", "\u201d"
def panel(x, title, items, verdict):
box(ax, x, 0.29, 0.41, 0.6, ORANGE)
t = ax.text(x + 0.02, 0.865, title, fontsize=10, weight="bold", va="top")
fig.canvas.draw() # keep every panel title inside its box
bb = t.get_window_extent().transformed(ax.transAxes.inverted())
assert bb.x1 < x + 0.41 - 0.01, f"panel title overflows: {title}"
y = 0.79
for where, txt in items:
ax.text(x + 0.02, y, where, fontsize=10, va="top", color=INK2)
body = wrap(txt, 36)
ax.text(x + 0.02, y - 0.045, body, fontsize=10, va="top", color=INK)
y -= 0.045 + 0.042 * (body.count("\n") + 1) + 0.025
ax.text(x + 0.02, 0.325, verdict, fontsize=10.5, weight="bold", color=ORANGE)
panel(0.03, "Abstract and a Results sentence", [
("Abstract", LQ + "viral burden was 44% lower among Alpha variant infections, compared to those "
"with the predecessor strain, B.1.177" + RQ),
("Results", LQ + "This corresponds to a 44% reduction in viral loads with Alpha." + RQ),
], "States: Alpha LOWER, by 44%")
panel(0.56, "Table 1, heading, Discussion", [
("Table 1, B.1.177 row (ref = Alpha)", "Beta score 1.00 (2.15)"),
("Results subsection heading", LQ + "viral burden was higher among Alpha compared to B.1.177 "
"infections" + RQ),
("Discussion", LQ + "viral burden was, on average, 115% higher among Alpha-variant compared to "
"B.1.177 samples" + RQ),
], "States: Alpha HIGHER, by 115%")
ax.add_patch(FancyArrowPatch((0.44, 0.55), (0.56, 0.55), arrowstyle="<|-|>", mutation_scale=14,
color=INK, lw=2, transform=ax.transAxes))
ax.text(0.5, 0.58, "opposite\ndirection", ha="center", va="bottom", fontsize=10, weight="bold")
# recomputation strip
box(ax, 0.03, 0.03, 0.94, 0.21, BLUE, fc="#f3f7fd")
ax.text(0.05, 0.215, "Recomputed with the paper's rule (3 Ct = 10-fold change):", fontsize=10.5,
weight="bold", va="top", color=INK)
ax.text(0.05, 0.15, f"Beta = +1.00 Ct \u2192 Alpha / B.1.177 = 10^(1/3) = {alpha_over_b177:.2f} \u2192 "
f"Alpha {100 * (alpha_over_b177 - 1):.0f}% higher\n(equivalently B.1.177 "
f"{100 * (1 - b177_over_alpha):.0f}% lower). '44% lower with Alpha' would need Beta = "
f"{beta_implied_abs:+.2f}.", fontsize=10, va="top", color=BLUE, linespacing=1.5)
return fig
def fig_B():
"""One log axis: the Alpha/B.1.177 ratio implied by each statement."""
fig, ax = plt.subplots(figsize=(W, 3.3))
items = [
("Abstract: “44% lower among Alpha”", abs_ratio, ORANGE, "o"),
("Results: “44% reduction ... with Alpha”", abs_ratio, ORANGE, "o"),
("Discussion: “115% higher among Alpha”", 1 + DISCUSSION_PCT_HIGHER / 100, ORANGE, "o"),
("Table 1: Beta = +1.00 Ct, via 10^(Beta/3)", alpha_over_b177, BLUE, "D"),
]
ys = list(range(len(items)))[::-1]
for (lab, v, c, m), y in zip(items, ys):
ax.plot([v], [y], marker=m, ms=10, color=c, mec=BG, mew=1.5, ls="none", zorder=3)
ax.text(v * (1.06 if v > 1 else 0.94), y, f"{v:.2f}", va="center",
ha="left" if v > 1 else "right", fontsize=10, color=INK)
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.set_xscale("log"); ax.set_xlim(0.35, 3.5)
ax.minorticks_off()
ax.set_xticks([0.4, 0.5, 0.7, 1, 1.5, 2, 3]); ax.set_xticklabels(["0.4", "0.5", "0.7", "1", "1.5", "2", "3"])
ax.set_yticks(ys); ax.set_yticklabels([i[0] for i in items]); ax.set_ylim(-0.6, len(items) - 0.4)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0, labelcolor=INK)
ax.set_xlabel("Viral load, Alpha / B.1.177 (log scale)")
ax.text(0.97, len(items) - 0.45, "← Alpha lower", ha="right", fontsize=10, color=INK2, va="top")
ax.text(1.03, len(items) - 0.45, "Alpha higher →", ha="left", fontsize=10, color=INK2, va="top")
ax.plot([], [], "o", color=ORANGE, ms=9, label="stated in the paper")
ax.plot([], [], "D", color=BLUE, ms=9, label="recomputed from Table 1 Beta")
ax.legend(loc="lower center", bbox_to_anchor=(0.35, 1.0), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
return fig
def fig_C():
"""Calculation-flow diagram."""
fig = plt.figure(figsize=(W, 3.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
def node(x, y, w, h, txt, edge, color=INK, bold=False, fc="white"):
box(ax, x, y, w, h, edge, fc=fc)
ax.text(x + w / 2, y + h / 2, txt, ha="center", va="center", fontsize=10, color=color,
weight="bold" if bold else "normal")
def arrow(p, q):
ax.add_patch(FancyArrowPatch(p, q, arrowstyle="-|>", mutation_scale=12, color=INK2, lw=1.6,
transform=ax.transAxes))
node(0.02, 0.62, 0.24, 0.26, "Table 1\nB.1.177 vs Alpha (ref)\nBeta = +1.00 Ct", ORANGE)
node(0.37, 0.62, 0.26, 0.26, "Paper's rule:\n3 Ct = 10-fold\nfactor = 10^(-Beta/3)", INK2)
node(0.73, 0.62, 0.25, 0.26, f"Alpha / B.1.177\n= 10^(1/3) = {alpha_over_b177:.2f}", BLUE, color=BLUE, bold=True)
arrow((0.26, 0.75), (0.37, 0.75)); arrow((0.63, 0.75), (0.73, 0.75))
node(0.55, 0.12, 0.2, 0.3, f"Alpha\n{100 * (alpha_over_b177 - 1):.0f}% higher\n(Discussion: 115%,\nmatches)",
BLUE, color=BLUE)
node(0.78, 0.12, 0.2, 0.3, f"B.1.177\n{100 * (1 - b177_over_alpha):.0f}% lower\nthan Alpha", BLUE, color=BLUE)
arrow((0.80, 0.62), (0.68, 0.42)); arrow((0.88, 0.62), (0.88, 0.42))
node(0.02, 0.12, 0.42, 0.3, f"Abstract and Results: Alpha 44% lower\n= Alpha / B.1.177 of {abs_ratio:.2f}\n"
f"would need Beta = {beta_implied_abs:+.2f} Ct\n(opposite sign to Table 1)", ORANGE, color=INK)
ax.text(0.495, 0.27, "≠", fontsize=22, ha="center", va="center", color=INK)
ax.text(0.02, 0.965, "From the Table 1 coefficient to a percentage", fontsize=11.5, weight="bold", va="top")
return fig
def fig_D():
"""Ct-difference number line: the Beta each statement implies."""
fig, ax = plt.subplots(figsize=(W, 2.9))
items = [
("Abstract and Results: “44% lower among Alpha”", beta_implied_abs, BLUE, "D",
"implied by the text"),
("Discussion: “115% higher among Alpha”", beta_implied_disc, BLUE, "D", "implied by the text"),
("Table 1: Beta score as printed", BETA_B177, ORANGE, "o", "printed"),
]
ys = [2, 1, 0]
for (lab, v, c, m, _), y in zip(items, ys):
ax.plot([0, v], [y, y], color=c, lw=2, zorder=2)
ax.plot([v], [y], marker=m, ms=10, color=c, mec=BG, mew=1.5, zorder=3)
ax.text(v + (0.06 if v > 0 else -0.06), y + 0.22, f"{v:+.2f}", ha="left" if v > 0 else "right",
fontsize=10, color=INK)
ax.axvline(0, color=INK2, lw=0.8)
ax.set_xlim(-1.5, 1.5); ax.set_ylim(-0.6, 2.7)
ax.set_yticks(ys); ax.set_yticklabels([i[0] for i in items])
ax.tick_params(axis="y", length=0, labelcolor=INK)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.set_xlabel("Ct(B.1.177) − Ct(Alpha), Ct units")
ax.text(-0.05, -0.5, "← Alpha lower burden", ha="right", fontsize=10, color=INK2)
ax.text(0.05, -0.5, "Alpha higher burden →", ha="left", fontsize=10, color=INK2)
ax.plot([], [], "o", color=ORANGE, ms=9, label="printed in Table 1")
ax.plot([], [], "D", color=BLUE, ms=9, label="Beta implied by the text")
ax.legend(loc="lower center", bbox_to_anchor=(0.3, 1.0), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
return fig
def fig_E():
"""Every Table 1 entry: printed factor vs 10^(-Beta/3)."""
fig, ax = plt.subplots(figsize=(W, 4.6))
lo, hi = 0.015, 15
ax.plot([lo, hi], [lo, hi], color=BLUE, lw=2, zorder=1, label="printed = 10^(−Beta/3) (paper's rule)")
xs = [r[3] for r in rows]; ys = [r[4] for r in rows]
okx = [x for x, r in zip(xs, rows) if r[5]]; oky = [y for y, r in zip(ys, rows) if r[5]]
ax.plot(okx, oky, "o", ms=8, color=ORANGE, mec=BG, mew=1.2, zorder=3,
label=f"Table 1 entries ({n_ok} of {len(rows)} on the line)")
bad = [r for r in rows if not r[5]]
for r in bad:
ax.plot([r[3]], [r[4]], "o", ms=11, mfc="none", mec=ORANGE, mew=2.2, zorder=4)
ax.plot([r[3]], [r[3]], "D", ms=9, color=BLUE, mec=BG, mew=1.2, zorder=4)
ax.annotate("", xy=(r[3], r[4] * 0.93), xytext=(r[3], r[3] * 1.08),
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.4))
ax.text(r[3] * 0.8, r[4], f"B.1.177 vs Alpha, unvaccinated\nprinted {r[4]:.2f}", ha="right", va="center",
fontsize=10)
ax.text(r[3] * 1.2, r[3] * 0.9, f"expected {r[3]:.2f}", ha="left", va="top", fontsize=10, color=INK)
ax.set_xscale("log"); ax.set_yscale("log")
ax.set_xlim(lo, hi); ax.set_ylim(lo, hi)
ticks = [0.02, 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.set_yticks(ticks); ax.set_yticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.grid(color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.set_xlabel("Viral-load factor recomputed from the printed Beta, 10^(−Beta/3)")
ax.set_ylabel("Viral-load factor printed in Table 1")
ax.legend(loc="upper left", frameon=False, fontsize=10)
fig.tight_layout()
return fig
CAPTIONS = {
"A": "The two sets of statements the paper makes about the same comparison (B.1.177 vs Alpha, "
"unvaccinated with no known prior exposure, Oxford samples), quoted verbatim. The orange-outlined "
"panels hold the paper's text and Table 1 entry. On the left, the Abstract and one Results sentence "
"say viral burden was 44% lower with Alpha. On the right, the Table 1 coefficient, the heading of the "
"Results subsection that reports it, and the Discussion say Alpha was higher, by 115%. The blue strip "
"at the bottom is the recomputation from the printed Beta score with the paper's own 3-Ct-per-10-fold "
"rule; it agrees with the right-hand panel. A 44% reduction with Alpha would need a Beta of -0.76.",
"B": "The Alpha / B.1.177 viral-load ratio implied by each statement in the paper (orange dots), and the "
"ratio recomputed from the Table 1 Beta score of +1.00 with the paper's 3-Ct-per-10-fold rule (blue "
"diamond), on one log axis. The vertical line marks no difference. The Abstract and the Results "
"sentence imply 0.56 (Alpha lower); the Discussion and the recomputation give 2.15 (Alpha higher).",
"C": "From the Table 1 coefficient to a percentage. The printed Beta of +1.00 Ct (orange), converted with "
"the paper's rule, gives Alpha / B.1.177 = 2.15: Alpha 115% higher, or B.1.177 54% lower (blue). The "
"Abstract and Results statement that Alpha was 44% lower would need a Beta of -0.76 Ct, the opposite "
"sign.",
"D": "The Ct difference, Ct(B.1.177) - Ct(Alpha), printed in Table 1 (orange) and the difference each text "
"statement implies under the paper's 3-Ct-per-10-fold rule (blue). The Discussion implies +1.00, "
"matching Table 1; the Abstract and Results sentence imply -0.76, the opposite sign.",
"E": "Each orange dot is one of the 27 viral-load factors printed in Table 1, plotted against the factor "
"recomputed from the Beta score printed beside it, 10^(-Beta/3) (both axes log scale; age is per 10 "
"years, sample date per year). The blue line is where the two agree. Twenty-six entries lie on it. "
"The exception (hollow orange circle) is B.1.177 vs Alpha among unvaccinated people with no known "
"prior exposure: its Beta of +1.00 implies 0.46 (blue diamond), but 2.15, the reciprocal, is printed. "
"The printed 2.15 is the Alpha / B.1.177 ratio, consistent with the Discussion; the Abstract's "
"\"44% lower among Alpha\" matches neither orientation.",
}
FIGS = {"A": fig_A, "B": fig_B, "C": fig_C, "D": fig_D, "E": fig_E}
for name, fn in FIGS.items():
save(fn(), os.path.join(CAND, f"cand_{name}"))
print("\nwrote candidates/cand_A..E (.svg, .png)")
if "--pick" in sys.argv:
picks = sys.argv[sys.argv.index("--pick") + 1].upper().split(",")
for i, name in enumerate(picks, 1):
save(FIGS[name](), os.path.join(HERE, f"fig{i}"))
print(f"\nfig{i} = cand_{name}\nFigure {i}. {CAPTIONS[name]}")
The abstract and the body disagree on nitric oxide after arg1 knockdown; which wording is right depends on the comparison
PLoS Pathog (2012) · doi:10.1371/journal.ppat.1002417 · PMC3261917
Summary
The abstract reports that miRNAi-mediated knockdown of host arginase 1 (arg1) in BHK cells "led to increased generation of nitric oxide and reduced parasite burden (p<0.005)." The other three places in the body that describe the nitric oxide (NO) result of the same Fig. 6I experiment use the opposite wording: the Results subsection heading reads "promotes parasite killing without enhanced NO production," the Results prose says the enhanced killing "was not accompanied by greater parasite-induced NO production (Fig. 6I)," the Fig. 6I legend says "Arg1 knockdown did not induce NO production," and the Discussion says the effect "did not appear to be driven by enhanced production of NO." This is a wording inconsistency about the direction of a single result.
It can be resolved only by naming the comparison, because the two descriptions refer to different control groups and both can be literally true. The paper prints no NO numbers — they appear only inside the Fig. 6I bar chart — so we digitised the four bars from the publisher's figure image. Relative to infected non-targeted control cells, knockdown NO is about 1.37 times higher (≈5.5 vs ≈4.0 µM); the figure draws no significance marker for that difference, and this is the comparison the abstract's "increased generation of nitric oxide" fits. Relative to uninfected cells, knockdown NO is essentially unchanged (≈5.5 vs ≈5.4 µM); that is the comparison the heading, prose, legend and Discussion fit when they say NO was not induced or not enhanced. So the inconsistency is in how the finding is phrased and against which reference, not a contradiction of the underlying measurement. It matters because the abstract and the body read as opposite directional claims about the same experiment, and the paper's stated mechanism is NO-independent (polyamine-mediated), so a reader of the abstract alone could take the opposite of what the body concludes.
What the paper reports
The abstract sentence (the two claims are "increased NO" and "reduced parasite burden"):
"miRNAi-mediated selective knockdown of hamster arginase 1 (arg1) in BHK cells led to increased generation of nitric oxide and reduced parasite burden (p<0.005)." (Abstract)
The Results subsection heading for the same experiment:
"Knockdown of host arg1 promotes parasite killing without enhanced NO production" (Results, subsection heading)
The Results prose, citing Fig. 6I directly:
"This enhancement of parasite killing by knockdown of arg1 in BHK cells was not accompanied by greater parasite-induced NO production (Fig. 6I), even though BHK cells were fully capable of generating NO (see Fig. 5B). Thus it would appear that parasite-induced macrophage arg1 contributes to L. donovani replication through mechanisms other than reduction of NO production." (Results, same subsection)
The Fig. 6I legend:
"Arg1 knockdown did not induce NO production in BHK cells infected with L. donovani." (Figure 6, legend, panel I)
The Discussion:
"This increased control of infection mediated by targeted arg1 knockdown did not appear to be driven by enhanced production of NO by the isolated infected macrophage, suggesting that the parasite-induced arginase has a pathological effect through the increase in polyamines that promote parasite growth." (Discussion)
What the paper's own figure shows
Method. No per-cell NO values are archived in text; the Fig. 6I bars are the only source. We downloaded the publisher's Figure 6 image (journals.plos.org, image g006, size=large) and measured the four bars in pixels against the panel's y-axis ticks (0 to 8 µM), using python. The extraction — axis calibration, bar tops, SD caps, and the position of the one significance bracket — is in recompute.py.txt, section 2, and reruns from the downloaded image. Reading precision is about ±0.05 µM (3 px), so the values below are estimates, not the authors' numbers.
Digitised Fig. 6I values (nitrites + nitrates, mean ± SD, µM):
| Uninfected | Infected (48 h) | |
|---|---|---|
| Arg1 knockdown | ≈5.4 ± 0.3 | ≈5.5 ± 0.4 |
| Non-targeted control | ≈6.3 ± 0.5 | ≈4.0 ± 0.5 |
The single significance bracket (**) runs between the two non-targeted bars (uninfected vs infected): infection lowered NO in control cells (≈6.3 → ≈4.0 µM). Knockdown cells held their baseline (≈5.4 → ≈5.5 µM). So:
- Relative to infected non-targeted control cells, knockdown NO is higher: ≈5.5 vs ≈4.0 µM, a ratio of about 1.37. The figure shows no significance marker for this difference.
- Relative to uninfected cells, infection did not induce NO in knockdown cells: ≈5.5 vs ≈5.4 µM (about +0.1 µM). Knockdown NO is also not above the uninfected non-targeted level (≈5.5 vs ≈6.3 µM).
Both the abstract and the body can therefore be read as literally true, each against a different control. The body's heading, prose and legend speak of "induction" and of NO relative to the uninfected or baseline state; the abstract's "increased generation of nitric oxide" fits a comparison of knockdown with the infected non-targeted control.
The second claim in the abstract sentence (reduced parasite burden) is supported by the Results, which give p<0.01 at 48 h and p<0.001 at 72 h (Fig. 6H); the abstract's bound p<0.005 is satisfied by the 72 h value. The knockdown efficiency (>90% reduction in arg1 mRNA, p=0.002; Fig. 6G) is not in dispute.
| Quantity | Location | Reported | Digitised / cross-check |
|---|---|---|---|
| NO after arg1 knockdown (wording) | Abstract | "increased generation of nitric oxide" | Fits KD vs infected non-targeted control: ≈5.5 vs ≈4.0 µM (×1.37; no significance marker drawn) |
| NO after arg1 knockdown (wording) | Results heading | "without enhanced NO production" | Fits KD infected vs uninfected: ≈5.5 vs ≈5.4 µM (not induced) |
| NO after arg1 knockdown (wording) | Results prose (Fig. 6I) | "not accompanied by greater parasite-induced NO production" | Infection did not raise NO in KD cells above its uninfected level |
| NO after arg1 knockdown (wording) | Fig. 6I legend | "did not induce NO production" | KD infected ≈5.5 µM vs KD uninfected ≈5.4 µM (no induction) |
| NO after arg1 knockdown (wording) | Discussion | "did not appear to be driven by enhanced production of NO" | Hedged; consistent with the no-induction reading |
| Only significance marker in Fig. 6I (**) | Fig. 6I | between non-targeted bars | Non-targeted uninfected ≈6.3 vs infected ≈4.0 µM (infection lowered control NO) |
| Reduced parasite burden p-value | Abstract | p<0.005 | Consistent: Results give p<0.01 (48 h), p<0.001 (72 h); the 72 h value satisfies p<0.005 |
| arg1 mRNA knockdown efficiency | Results / Fig. 6G | >90% reduction, p=0.002 | Stated once; not in dispute |
Figure
Figure 1. The abstract's wording (top left) and the body's wording (top right) describe the same Fig. 6I knockdown experiment but fit different reference groups. Below, the four Fig. 6I bars are shown as means ± SD on one axis, digitised from the publisher's figure image by pixel measurement (filled = Arg1 knockdown, open = non-targeted; the paper prints no NO numbers). Comparison (a): knockdown vs non-targeted among infected cells, ≈5.5 vs ≈4.0 µM (×1.37), which has no significance marker in the figure and matches the abstract's "increased" wording. Comparison (b): knockdown infected vs uninfected, ≈5.5 vs ≈5.4 µM, essentially unchanged, which matches the body's "not induced" wording. The paper's only significance mark (**) is between the two non-targeted bars (uninfected vs infected). All quotes are verbatim from the PMC-archived text; the blue values are digitised estimates, not the authors' numbers, and the paper's own figure is not reproduced.
Why it matters
The two statements sit in different parts of the paper — one in the abstract, which databases index, and the rest in the Results, legend and Discussion — and they read as opposite directional claims about the same measurement. The abstract's "increased generation of nitric oxide," taken on its own, suggests the canonical arginase-versus-NOS2 story in which inhibiting host arginase restores NO and that NO kills the parasite. The body instead concludes that enhanced parasite control was not driven by induction of NO and attributes it to reduced polyamine synthesis, explicitly NO-independent. Read against the Fig. 6I values, neither statement is simply wrong: the abstract's wording fits a comparison with infected control cells (for which the figure shows no significance marker), while the body's wording fits a comparison with uninfected or baseline cells. If a reader reconciles the abstract and the body by the wrong comparison, they could take away a mechanism the paper does not claim. The paper's principal conclusions are unaffected: arg1 knockdown reduces parasite burden, STAT6 knockdown reduces arg1 and enhances parasite control, and parasite-induced STAT6/arg1 drives progressive disease; these depend on the NO-independent reading. A secondary, cosmetic point: the abstract gives p<0.005 for the burden effect while the Results give p<0.01 at 48 h and p<0.001 at 72 h; the bound is satisfied by the 72 h value and changes no conclusion.
Explanations considered
- The two statements compare with different control groups. This is the resolution supported by the figure. "Increased generation of nitric oxide" is true relative to the infected non-targeted control (≈5.5 vs ≈4.0 µM; no significance marker shown). "Without enhanced NO production" / "did not induce NO" is true relative to the uninfected or baseline state (≈5.5 vs ≈5.4 µM). The inconsistency is which reference each text names, not the measurement. The appropriate fix is to make the comparison explicit in both places (see Suggested correction); intent is not assessed.
- Two separate NO measurements (different cells, time points, or stimuli). No. All five locations name the same model: miRNAi arg1 knockdown in BHK cells infected with L. donovani, NO measured in Fig. 6I by Griess assay at 48 h. The Results prose and the Fig. 6I legend describe exactly the experiment the abstract summarises.
- A word slip in the abstract (e.g. "increased" was meant to be "without increased"). Possible, but not required: the digitised values show that "increased" is defensible against the infected control, so a simple reversal of the abstract is not the only reconciliation and could itself misdescribe the data relative to that control. The write-up does not assert which phrasing the authors intended.
- A typesetting or PMC conversion artifact. No. The publisher's version of record carries the identical abstract sentence and the identical body statements; the wording difference is present in both.
- The supplements resolve it. No. The four supporting figures cover polyamine content, BHK infection, the nor-NOHA arginase inhibitor, and STAT6 activation. None reports NO after arg1 knockdown, which appears only in main Fig. 6I. There is no source-data file.
Two smaller, cosmetic inconsistencies, noted for completeness; neither changes any conclusion:
- The Fig. 6I legend ends "(p<0.001)" for the one significance bracket, but the same legend's key defines ", p<0.01" (and "*, p<0.001"). The two uses of "" disagree.
- The Results prose cites "see Fig. 5B" for BHK cells being able to make NO, but Fig. 5B is the arg1/NOS2 mRNA ratio in splenic macrophages; the BHK NO data are in Fig. 6B.
Status
As of 2026-10-23. Version of record checked at the publisher (journals.plos.org, PLoS Pathogens): the abstract sentence, the Results heading, the Results prose, and the Fig. 6I legend match the PMC archive word for word, and the Discussion sentence matches. The Fig. 6I values were digitised from the publisher's Figure 6 image (image g006, size=large); the SHA-256 of the downloaded image is printed by recompute.py.txt. Supplements S1-S4 were identified from the publisher page and none covers NO after arg1 knockdown; there is no source-data file. Corrections: none found. Crossref lists no update/correction relation; src/known.py returned no PMC or Europe PMC notices; the Europe PMC record for PMID 22275864 has no comment/correction list and no erratum citing the DOI; the publisher page shows no correction notice and zero reader comments. PubPeer could not be checked (API behind Cloudflare, search page returned HTTP 403); web searches found no PubPeer thread or Retraction Watch coverage. Sources: src/show.py PMC3261917 --full (PMC-archived abstract, Results, Fig. 5/6 legends, Discussion, Methods); publisher HTML and Figure 6 image; Crossref and Europe PMC via src/known.py.
Suggested correction
Reconcile the abstract and the body by stating, in both, the comparison being made. For example, the abstract and the Fig. 6I-related text could say that after infection, arg1-knockdown cells retained their baseline NO while non-targeted control cells lost NO, so that knockdown NO was higher than in infected controls but was not induced above uninfected levels. We do not recommend simply reversing the abstract to "without increased generation of nitric oxide," because against the infected non-targeted control that wording could itself be inaccurate; the correct phrasing depends on which comparison the authors intend. The two cosmetic points above (the "**" p-value in the Fig. 6I legend, and the "see Fig. 5B" vs Fig. 6B citation) could be corrected in the same notice. A published author correction is the appropriate mechanism; no data or main conclusions need revision.
Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
"""
Checks for PMC3261917
"Progressive Visceral Leishmaniasis Is Driven by Dominant Parasite-induced
STAT6 Activation and STAT6-dependent Host Arginase 1 Expression"
PLoS Pathog 2012, doi:10.1371/journal.ppat.1002417
The issue is a wording inconsistency about nitric oxide (NO) after miRNAi
knockdown of host arginase 1 (arg1) in BHK cells. The abstract says knockdown
"led to increased generation of nitric oxide"; the Results heading, Results
prose, Fig. 6I legend and Discussion say NO was not enhanced / not induced.
Whether each statement is true depends on which control the knockdown is
compared with, so this script also reads the Fig. 6I bars from the
publisher's published figure image.
Sections
1. An ENCODING (not a computation) of what each quoted location says.
2. Digitisation of Fig. 6I from the publisher's Figure 6 image
(downloaded at run time to a temporary folder; the image is not stored
or redrawn here).
3. Computed comparisons on the digitised values and on stated p-values.
4. Figures: fig1.svg/png (the layout used in writeup.md) and five
alternative layouts in ./candidates/ (cand1..cand5).
Run: python recompute.py.txt
Needs numpy and matplotlib, and network access to journals.plos.org.
"""
import os, sys, hashlib, tempfile, textwrap, urllib.request
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
from matplotlib.patches import FancyBboxPatch, ConnectionPatch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
# ----------------------------------------------------------------------
# 1. ENCODING of quoted statements (hand-coded, not computed)
# ----------------------------------------------------------------------
# Each statement is quoted verbatim from the PMC-archived text
# (src/show.py PMC3261917 --full). The second field is the comparison the
# wording implies, as read by the write-up; it is an encoding, not a result.
statements = [
("Abstract",
"led to increased generation of nitric oxide and reduced parasite burden (p<0.005)",
"NO increased (reference not stated)"),
("Results heading",
"Knockdown of host arg1 promotes parasite killing without enhanced NO production",
"NO not enhanced"),
("Results text",
"was not accompanied by greater parasite-induced NO production (Fig. 6I)",
"no greater parasite-induced NO (i.e. no induction by infection)"),
("Fig. 6I legend",
"Arg1 knockdown did not induce NO production in BHK cells infected with L. donovani.",
"NO not induced"),
("Discussion",
"did not appear to be driven by enhanced production of NO",
"NO not enhanced (hedged)"),
]
print("=" * 72)
print("1. ENCODING OF QUOTED STATEMENTS (hand-coded reading, not a computation)")
print("=" * 72)
for loc, q, reading in statements:
print(" %-16s %s" % (loc, reading))
n_up = sum(1 for s in statements if s[2].startswith("NO increased"))
print(" Locations whose wording says NO increased: %d of %d" % (n_up, len(statements)))
# ----------------------------------------------------------------------
# 2. DIGITISE Fig. 6I from the publisher's Figure 6 image
# ----------------------------------------------------------------------
FIG_URL = ("https://journals.plos.org/plospathogens/article/figure/image"
"?size=large&id=10.1371/journal.ppat.1002417.g006")
tmpdir = tempfile.mkdtemp(prefix="pmc3261917_")
img_path = os.path.join(tmpdir, "g006.png")
try:
req = urllib.request.Request(FIG_URL, headers={"User-Agent": "Mozilla/5.0"})
with urllib.request.urlopen(req, timeout=60) as r, open(img_path, "wb") as f:
f.write(r.read())
except Exception as e:
sys.exit("Could not download Figure 6 image (%s); digitisation needs it." % e)
sha = hashlib.sha256(open(img_path, "rb").read()).hexdigest()
img = mpimg.imread(img_path)[..., :3].mean(axis=2)
dark = img < 0.5
print("\n" + "=" * 72)
print("2. DIGITISATION OF Fig. 6I (publisher image g006, size=large)")
print("=" * 72)
print(" image %d x %d px, sha256 %s" % (img.shape[1], img.shape[0], sha))
assert img.shape[:2] == (2556, 2042), "image size changed; re-check pixel ranges"
def runs(v, off=0):
out, s = [], None
for i, b in enumerate(v):
if b and s is None: s = i
if not b and s is not None: out.append((s + off, i - 1 + off)); s = None
if s is not None: out.append((s + off, len(v) - 1 + off))
return out
# Panel I sits at x 1520-2042, y 1930-2440 of the image. The y axis line is at
# x = 1541-1542; tick marks extend left of it. Read the tick rows at x = 1535.
tick_runs = runs(dark[1930:2440, 1535], 1930)
tick_y = np.array([(a + b) / 2 for a, b in tick_runs])
tick_val = np.array([8, 6, 4, 2, 0], float) # axis labels, top to bottom
assert len(tick_y) == 5, tick_runs
slope, icpt = np.polyfit(tick_y, tick_val, 1) # uM per pixel row
resid = tick_val - (slope * tick_y + icpt)
px_per_uM = -1 / slope
print(" tick rows (px):", ", ".join("%.1f" % y for y in tick_y), "-> 0..8 uM")
print(" scale %.2f px per uM; max tick residual %.4f uM" % (px_per_uM, abs(resid).max()))
to_uM = lambda y: slope * y + icpt
# Bar column ranges (px), read from the image by eye; tops and caps are
# then found automatically. Black bars = Arg1 knockdown, white = non-targeted.
bars = {
("uninfected", "knockdown"): (1578, 1652),
("uninfected", "non-targeted"): (1652, 1727),
("infected", "knockdown"): (1836, 1910),
("infected", "non-targeted"): (1910, 1984),
}
Y_MIN = 2035 # below the significance bracket (row ~2031) and the legend key
D = {}
for key, (x0, x1) in bars.items():
tops = []
for x in (x0 + 6, x1 - 6): # near the bar edges, away from the SD stem
r = [a for a, b in runs(dark[Y_MIN:2424, x], Y_MIN)]
tops.append(r[0] + 0.5) # centre of the 2-px top outline
top = float(np.mean(tops))
xc = (x0 + x1) / 2
caps = []
for x in range(int(xc) - 8, int(xc) + 9):
if abs(x - xc) <= 2: continue # skip the vertical SD stem
short = [a for a, b in runs(dark[2008:int(top) - 3, x], 2008) if b - a <= 4]
if short: caps.append(max(short) + 0.5) # lowest short run above the bar = SD cap
cap = float(np.median(caps))
D[key] = dict(mean=to_uM(top), sd=to_uM(cap) - to_uM(top), xc=xc)
print(" %-10s %-13s mean %.2f uM SD %.2f uM (top row %.1f, cap row %.1f)"
% (key[0], key[1], D[key]["mean"], D[key]["sd"], top, cap))
# The one significance bracket: a horizontal 2-px line at row ~2031 above the bars.
br = [r for r in runs(dark[2031, 1560:2020], 1560) if r[1] - r[0] > 40]
assert len(br) == 1, br
bx0, bx1 = br[0]
def bar_at(x):
for k, (a, b) in bars.items():
if a <= x <= b: return k
print(" significance bracket (**) spans x %d-%d: from %s %s to %s %s"
% (bx0, bx1, *bar_at(bx0), *bar_at(bx1)))
print(" reading precision: about +/-0.05 uM (3 px); values are estimates.")
# ----------------------------------------------------------------------
# 3. COMPUTED COMPARISONS
# ----------------------------------------------------------------------
print("\n" + "=" * 72)
print("3. COMPARISONS ON THE DIGITISED Fig. 6I MEANS")
print("=" * 72)
ku = D[("uninfected", "knockdown")]["mean"]; nu = D[("uninfected", "non-targeted")]["mean"]
ki = D[("infected", "knockdown")]["mean"]; ni = D[("infected", "non-targeted")]["mean"]
cmp = {
"KD infected vs non-targeted infected (abstract-type comparison)": (ki, ni),
"KD infected vs KD uninfected (induction by infection)": (ki, ku),
"KD infected vs non-targeted uninfected": (ki, nu),
"non-targeted infected vs non-targeted uninfected (** bracket)": (ni, nu),
}
R = {}
for name, (a, b) in cmp.items():
R[name] = (a - b, a / b)
print(" %-66s diff %+.2f uM ratio %.2f" % (name, a - b, a / b))
ratio_kd_nt_inf = ki / ni
print(" -> Relative to infected non-targeted cells, knockdown NO is higher (x%.2f);"
% ratio_kd_nt_inf)
print(" no significance marker is drawn for this difference.")
print(" -> Relative to uninfected knockdown cells, infection changed NO by %+.2f uM"
% (ki - ku))
print(" (not induced).")
print("\n Stated p-value checks")
abs_bound = 0.005
results_burden = {"48 h": 0.01, "72 h": 0.001} # Results: p<0.01, p<0.001
for t, p in results_burden.items():
print(" parasite burden %s: stated p<%g; within the abstract's p<%g? %s"
% (t, p, abs_bound, p <= abs_bound))
legend_key = {"*": 0.05, "**": 0.01, "***": 0.001} # Fig. 6 legend key
panel_I_note = ("**", 0.001) # panel I text: "(**p<0.001)"
print(" Fig. 6I legend: '%s' given as p<%g; key defines '%s' as p<%g; same? %s"
% (panel_I_note[0], panel_I_note[1], panel_I_note[0],
legend_key[panel_I_note[0]], panel_I_note[1] == legend_key[panel_I_note[0]]))
print(" arg1 mRNA knockdown efficiency: >90% reduction, p = 0.002 (Fig. 6G); stated once")
# ----------------------------------------------------------------------
# 4. FIGURES
# ----------------------------------------------------------------------
BG = "#fcfcfb"; INK = "#0b0b0b"; SEC = "#52514e"; ORANGE = "#eb6834"; BLUE = "#2a78d6"
GRID = "#e4e3e0"; OR_BG = "#fdf0ea"; BL_BG = "#edf3fb"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
"axes.edgecolor": SEC, "axes.labelcolor": INK, "xtick.color": SEC, "ytick.color": SEC,
"figure.facecolor": BG, "savefig.facecolor": BG, "axes.facecolor": BG,
"xtick.labelsize": 10, "ytick.labelsize": 10, "lines.linewidth": 1.5})
W = 7.2
os.makedirs(os.path.join(HERE, "candidates"), exist_ok=True)
def wrap(s, n): return "\n".join(textwrap.wrap(s, n))
def save(fig, path_stem):
fig.savefig(path_stem + ".svg")
fig.savefig(path_stem + ".png", dpi=200)
plt.close(fig)
def panel(ov, x, y, w, h, fc, ec):
ov.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.008",
fc=fc, ec=ec, lw=1.5, transform=ov.transAxes, zorder=1))
def overlay(fig):
ov = fig.add_axes([0, 0, 1, 1]); ov.axis("off"); ov.set_xlim(0, 1); ov.set_ylim(0, 1)
ov.set_zorder(-1); return ov
Q_ABS = '"...knockdown of hamster arginase 1 (arg1) in BHK cells led to increased generation of nitric oxide and reduced parasite burden (p<0.005)."'
Q_BODY = [("Results heading", '"...promotes parasite killing without enhanced NO production"'),
("Results text", '"...was not accompanied by greater parasite-induced NO production (Fig. 6I)"'),
("Fig. 6I legend", '"Arg1 knockdown did not induce NO production..."'),
("Discussion", '"...did not appear to be driven by enhanced production of NO..."')]
CONDS = [("uninfected", "knockdown"), ("uninfected", "non-targeted"),
("infected", "knockdown"), ("infected", "non-targeted")]
LAB_ABS = "true if compared with infected\nnon-targeted cells: %.1f vs %.1f µM\n(×%.2f; no significance marker)" % (ki, ni, ratio_kd_nt_inf)
LAB_BODY = "true if compared with uninfected\ncells: %.1f vs %.1f µM (not induced)" % (ki, ku)
SRC = ("Quotes verbatim from the PMC-archived text. Blue values digitised by pixel measurement from the "
"publisher's Fig. 6I image (mean ± SD, ±0.05 µM reading error); the paper prints no NO numbers.")
def dot_axis(ax, xpos, show_legend=True, ylabel=True):
"""Digitised Fig. 6I means ± SD as dots on one 0-8 uM axis."""
ax.set_ylim(0, 8); ax.set_yticks([0, 2, 4, 6, 8])
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"): ax.spines[s].set_visible(False)
for k in CONDS:
m, sd = D[k]["mean"], D[k]["sd"]; x = xpos[k]
filled = k[1] == "knockdown"
ax.errorbar(x, m, yerr=sd, fmt="o", ms=9, color=BLUE, mfc=BLUE if filled else BG,
mec=BLUE, mew=2, elinewidth=2, capsize=4, zorder=3)
if ylabel: ax.set_ylabel("Nitrites + nitrates (µM)")
if show_legend:
h = [Line2D([], [], marker="o", ls="", ms=9, color=BLUE, mfc=BLUE, mew=2, label="Arg1 knockdown"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE, mfc=BG, mew=2, label="Non-targeted")]
ax.legend(handles=h, title="Digitised from Fig. 6I", frameon=False, loc="lower left",
fontsize=10, title_fontsize=10)
def bracket(ax, x0, x1, y, text, color=INK, dy=0.15, va="bottom", fs=10):
ax.plot([x0, x0, x1, x1], [y - dy, y, y, y - dy], color=color, lw=1.5, zorder=4)
ax.text((x0 + x1) / 2, y + 0.08, text, ha="center", va=va, color=color, fontsize=fs)
# ---- Layout A: text panels on top, dot plot below with the comparisons ----
def layout_A(stem):
fig = plt.figure(figsize=(W, 8.2)); ov = overlay(fig)
fig.text(0.02, 0.982, "Arg1 knockdown and NO (Fig. 6I): the abstract and the body fit different reference groups",
fontsize=10, color=SEC, va="top")
panel(ov, 0.02, 0.60, 0.40, 0.35, OR_BG, ORANGE)
fig.text(0.04, 0.932, "ABSTRACT (paper's text)", color=ORANGE, fontweight="bold", va="top")
fig.text(0.04, 0.895, wrap(Q_ABS, 36), va="top")
fig.text(0.04, 0.615, "says NO increased: fits (a)", color=ORANGE, fontweight="bold")
panel(ov, 0.44, 0.60, 0.54, 0.35, OR_BG, ORANGE)
fig.text(0.46, 0.932, "BODY, 4 locations (paper's text)", color=ORANGE, fontweight="bold", va="top")
y = 0.895
for loc, q in Q_BODY:
t = wrap(loc + ": " + q, 50)
fig.text(0.46, y, t, va="top"); y -= 0.0265 * (t.count("\n") + 1) + 0.010
fig.text(0.46, 0.615, "says NO not enhanced / not induced: fits (b)", color=ORANGE, fontweight="bold")
fig.add_artist(ConnectionPatch(xyA=(0.22, 0.598), xyB=(0.71, 0.598), coordsA="figure fraction",
coordsB="figure fraction", arrowstyle="<->", color=INK, lw=2, connectionstyle="arc3,rad=0.12"))
fig.text(0.465, 0.548, "opposite wording, same experiment", ha="center", va="top", fontweight="bold")
ax = fig.add_axes([0.10, 0.12, 0.88, 0.37])
xpos = {CONDS[0]: 0.8, CONDS[1]: 1.2, CONDS[2]: 2.3, CONDS[3]: 2.7}
dot_axis(ax, xpos)
ax.set_xlim(0.4, 4.1); ax.set_xticks([1.0, 2.5]); ax.set_xticklabels(["Uninfected", "Infected (48 h)"])
# (a) knockdown vs non-targeted, infected
ax.plot([2.85, 2.95, 2.95, 2.85], [ki, ki, ni, ni], color=INK, lw=1.5)
ax.text(3.02, (ki + ni) / 2, "(a) ×%.2f\nno significance\nmarker" % ratio_kd_nt_inf, va="center")
# (b) knockdown infected vs uninfected
ax.plot([0.8, 2.3], [ku, ki], color=SEC, lw=1.5, ls=":")
ax.text(1.55, 4.95, "(b) %+.1f µM:\nnot induced" % (ki - ku), ha="center", va="top", color=INK)
# the paper's only significance bracket
bracket(ax, 1.2, 2.7, 7.25, "** (the paper's only mark): non-targeted, uninfected vs infected",
color=SEC, dy=0.2)
fig.text(0.02, 0.012, wrap(SRC, 98), fontsize=10, color=SEC, va="bottom")
save(fig, stem)
# ---- Layout B: slope chart, quotes as side callouts ----
def layout_B(stem):
fig = plt.figure(figsize=(W, 5.8)); ov = overlay(fig)
ax = fig.add_axes([0.10, 0.22, 0.40, 0.68])
for grp, mfc, ls in (("knockdown", BLUE, "-"), ("non-targeted", BG, "--")):
m = [D[(c, grp)]["mean"] for c in ("uninfected", "infected")]
s = [D[(c, grp)]["sd"] for c in ("uninfected", "infected")]
ax.errorbar([0, 1], m, yerr=s, color=BLUE, ls=ls, lw=2, marker="o", ms=9, mfc=mfc, mew=2,
capsize=4, elinewidth=1.5, zorder=3)
ax.text(1.08, m[1], ("Arg1 knockdown" if grp == "knockdown" else "Non-targeted") + "\n%.1f µM" % m[1],
color=BLUE, va="center")
ax.text(-0.08, m[0], "%.1f" % m[0], color=BLUE, va="center", ha="right")
ax.set_ylim(0, 8); ax.set_yticks([0, 2, 4, 6, 8]); ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_xlim(-0.35, 1.55); ax.set_xticks([0, 1]); ax.set_xticklabels(["Uninfected", "Infected"])
for s in ("top", "right"): ax.spines[s].set_visible(False)
ax.set_ylabel("Nitrites + nitrates (µM)")
ax.set_title("Fig. 6I, digitised (mean ± SD)", fontsize=10, color=BLUE, loc="left")
panel(ov, 0.60, 0.57, 0.38, 0.36, OR_BG, ORANGE)
fig.text(0.62, 0.91, "ABSTRACT", color=ORANGE, fontweight="bold", va="top")
fig.text(0.62, 0.87, wrap('"...led to increased generation of nitric oxide..."', 34), va="top")
fig.text(0.62, 0.78, wrap("Fits knockdown vs non-targeted, infected: %.1f vs %.1f µM (×%.2f), no significance marker."
% (ki, ni, ratio_kd_nt_inf), 34), va="top", color=SEC)
panel(ov, 0.60, 0.14, 0.38, 0.40, OR_BG, ORANGE)
fig.text(0.62, 0.52, "RESULTS, LEGEND, DISCUSSION", color=ORANGE, fontweight="bold", va="top")
fig.text(0.62, 0.48, wrap('"without enhanced NO production" · "...not accompanied by greater parasite-induced NO production" · '
'"did not induce NO production"', 34), va="top")
fig.text(0.62, 0.27, wrap("Fits knockdown, infected vs uninfected: %.1f vs %.1f µM, not induced." % (ki, ku), 34),
va="top", color=SEC)
fig.text(0.02, 0.975, "Same experiment, two references: which comparison each text describes", fontsize=10, color=SEC, va="top")
fig.text(0.02, 0.012, wrap(SRC, 98), fontsize=10, color=SEC)
save(fig, stem)
# ---- Layout C: three columns, Abstract | digitised values | Body ----
def layout_C(stem):
fig = plt.figure(figsize=(W, 5.0)); ov = overlay(fig)
fig.text(0.02, 0.975, "Abstract vs body on NO after arg1 knockdown, read against the Fig. 6I values",
fontsize=10, color=SEC, va="top")
panel(ov, 0.02, 0.14, 0.26, 0.77, OR_BG, ORANGE)
fig.text(0.035, 0.89, "ABSTRACT", color=ORANGE, fontweight="bold", va="top")
fig.text(0.035, 0.845, wrap(Q_ABS, 23), va="top")
fig.text(0.035, 0.165, "NO increased", color=ORANGE, fontweight="bold")
panel(ov, 0.72, 0.14, 0.26, 0.77, OR_BG, ORANGE)
fig.text(0.735, 0.89, "BODY (4 places)", color=ORANGE, fontweight="bold", va="top")
fig.text(0.735, 0.845, wrap('"without enhanced NO production"; "...not accompanied by greater parasite-induced NO production"; '
'"did not induce NO production"; "did not appear to be driven by enhanced production of NO"', 23), va="top")
fig.text(0.735, 0.165, wrap("NO not enhanced / not induced", 18), color=ORANGE, fontweight="bold")
panel(ov, 0.30, 0.14, 0.40, 0.77, BL_BG, BLUE)
fig.text(0.50, 0.89, "Fig. 6I, digitised", color=BLUE, fontweight="bold", va="top", ha="center")
fig.text(0.50, 0.845, "mean ± SD, µM", color=SEC, va="top", ha="center")
rows = [("", "Uninfected", "Infected"),
("Knockdown", "%.1f ± %.1f" % (ku, D[CONDS[0]]["sd"]), "%.1f ± %.1f" % (ki, D[CONDS[2]]["sd"])),
("Non-targeted", "%.1f ± %.1f" % (nu, D[CONDS[1]]["sd"]), "%.1f ± %.1f" % (ni, D[CONDS[3]]["sd"]))]
for i, r in enumerate(rows):
yy = 0.77 - i * 0.075
fig.text(0.315, yy, r[0], va="top")
fig.text(0.515, yy, r[1], va="top", ha="center", color=BLUE if i else SEC)
fig.text(0.625, yy, r[2], va="top", ha="center", color=BLUE if i else SEC)
fig.text(0.315, 0.53, wrap("(a) knockdown vs non-targeted, infected: %.1f vs %.1f µM, ×%.2f, no significance marker"
% (ki, ni, ratio_kd_nt_inf), 36), va="top")
fig.text(0.315, 0.39, wrap("(b) knockdown, infected vs uninfected: %.1f vs %.1f µM, not induced" % (ki, ku), 36), va="top")
fig.text(0.315, 0.27, wrap("The paper's only mark (**): non-targeted, uninfected vs infected", 40), va="top", color=SEC)
fig.add_artist(ConnectionPatch(xyA=(0.28, 0.50), xyB=(0.31, 0.50), coordsA="figure fraction",
coordsB="figure fraction", arrowstyle="->", color=INK, lw=2))
fig.add_artist(ConnectionPatch(xyA=(0.72, 0.36), xyB=(0.69, 0.36), coordsA="figure fraction",
coordsB="figure fraction", arrowstyle="->", color=INK, lw=2))
fig.text(0.02, 0.012, wrap("Quotes verbatim (PMC text). Abstract fits (a); body fits (b). Values digitised from the "
"publisher's Fig. 6I image (±0.05 µM reading error); the paper prints no NO numbers.", 98),
fontsize=10, color=SEC)
save(fig, stem)
# ---- Layout D: horizontal dot-interval rows on one uM axis, quotes above ----
def layout_D(stem):
fig = plt.figure(figsize=(W, 5.8)); ov = overlay(fig)
panel(ov, 0.02, 0.75, 0.47, 0.21, OR_BG, ORANGE)
fig.text(0.035, 0.94, "ABSTRACT", color=ORANGE, fontweight="bold", va="top")
fig.text(0.035, 0.895, wrap('"...knockdown... led to increased generation of nitric oxide..."', 42), va="top")
fig.text(0.035, 0.77, "NO increased: fits (a)", color=ORANGE, fontweight="bold")
panel(ov, 0.51, 0.75, 0.47, 0.21, OR_BG, ORANGE)
fig.text(0.525, 0.94, "RESULTS, Fig. 6I LEGEND, DISCUSSION", color=ORANGE, fontweight="bold", va="top")
fig.text(0.525, 0.895, wrap('"without enhanced NO production"; "did not induce NO production"', 42), va="top")
fig.text(0.525, 0.77, "NO not induced: fits (b)", color=ORANGE, fontweight="bold")
ax = fig.add_axes([0.30, 0.25, 0.60, 0.44])
order = [CONDS[0], CONDS[2], CONDS[3], CONDS[1]] # top to bottom
names = {CONDS[0]: "Knockdown, uninfected", CONDS[2]: "Knockdown, infected",
CONDS[3]: "Non-targeted, infected", CONDS[1]: "Non-targeted, uninfected"}
yv = {k: 3 - i for i, k in enumerate(order)}
for k in order:
m, sd = D[k]["mean"], D[k]["sd"]; f = k[1] == "knockdown"
ax.errorbar(m, yv[k], xerr=sd, fmt="o", ms=9, color=BLUE, mfc=BLUE if f else BG, mew=2, elinewidth=2, capsize=4)
ax.text(m, yv[k] + 0.22, "%.1f" % m, color=BLUE, ha="center", va="bottom")
ax.set_yticks([yv[k] for k in order]); ax.set_yticklabels([names[k] for k in order])
ax.set_xlim(0, 8); ax.set_xticks([0, 2, 4, 6, 8]); ax.set_ylim(-0.5, 3.7)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"): ax.spines[s].set_visible(False)
ax.set_xlabel("Nitrites + nitrates (µM), digitised from Fig. 6I (mean ± SD)", color=BLUE)
def vbr(y0, y1, txt, col):
ax.plot([7.35, 7.5, 7.5, 7.35], [y0 - 0.05, y0 - 0.05, y1 + 0.05, y1 + 0.05], color=col, lw=1.5, clip_on=False)
ax.text(7.6, (y0 + y1) / 2, txt, va="center", color=col, clip_on=False)
vbr(3, 2, "(b) %+.1f" % (ki - ku), INK)
vbr(2 - 0.1, 1 + 0.1, "(a) ×%.2f" % ratio_kd_nt_inf, INK)
vbr(1 - 0.1, 0, "** paper", SEC)
fig.text(0.02, 0.015, wrap("(a) knockdown vs non-targeted, infected: higher, no significance marker. (b) knockdown, "
"infected vs uninfected: not induced. ** is the paper's only mark. Quotes verbatim (PMC text); "
"values digitised from the publisher's Fig. 6I image (±0.05 µM reading error).", 98),
fontsize=10, color=SEC)
save(fig, stem)
# ---- Layout E: stacked text panels left, dot plot right, connectors to the comparisons ----
def layout_E(stem):
fig = plt.figure(figsize=(W, 5.9)); ov = overlay(fig)
panel(ov, 0.02, 0.55, 0.40, 0.38, OR_BG, ORANGE)
fig.text(0.035, 0.91, "ABSTRACT", color=ORANGE, fontweight="bold", va="top")
fig.text(0.035, 0.865, wrap(Q_ABS, 36), va="top")
fig.text(0.035, 0.575, "NO increased", color=ORANGE, fontweight="bold")
panel(ov, 0.02, 0.12, 0.40, 0.40, OR_BG, ORANGE)
fig.text(0.035, 0.50, "BODY (4 locations)", color=ORANGE, fontweight="bold", va="top")
fig.text(0.035, 0.455, wrap('"without enhanced NO production" · "...not accompanied by greater parasite-induced NO production" · '
'"did not induce NO production" · "did not appear to be driven by enhanced production of NO"', 36), va="top")
fig.text(0.035, 0.145, "NO not enhanced / not induced", color=ORANGE, fontweight="bold")
ax = fig.add_axes([0.53, 0.12, 0.45, 0.78])
xpos = {CONDS[0]: 0.8, CONDS[1]: 1.2, CONDS[2]: 2.0, CONDS[3]: 2.4}
dot_axis(ax, xpos, ylabel=False)
ax.set_xlim(0.45, 2.75); ax.set_xticks([1.0, 2.2]); ax.set_xticklabels(["Uninfected", "Infected"])
ax.set_title("Fig. 6I digitised, µM (mean ± SD)", fontsize=10, color=BLUE, loc="left")
bracket(ax, 2.0, 2.4, 6.5, "×%.2f\nno mark" % ratio_kd_nt_inf)
ax.plot([0.8, 0.8, 2.0, 2.0], [4.85, 4.65, 4.65, 4.85], color=INK, lw=1.5)
ax.text(1.4, 4.5, "%+.1f µM\nnot induced" % (ki - ku), ha="center", va="top")
fig.add_artist(ConnectionPatch(xyA=(0.42, 0.74), coordsA="figure fraction", xyB=(1.98, 7.2), coordsB=ax.transData,
arrowstyle="->", color=ORANGE, lw=2, connectionstyle="arc3,rad=-0.15"))
fig.add_artist(ConnectionPatch(xyA=(0.42, 0.32), coordsA="figure fraction", xyB=(1.0, 3.95), coordsB=ax.transData,
arrowstyle="->", color=ORANGE, lw=2, connectionstyle="arc3,rad=0.15"))
fig.text(0.02, 0.975, "Two texts, two reference groups (Fig. 6I knockdown experiment)", fontsize=10, color=SEC, va="top")
fig.text(0.02, 0.012, wrap(SRC, 98), fontsize=10, color=SEC)
save(fig, stem)
LAYOUTS = [layout_A, layout_B, layout_C, layout_D, layout_E]
for i, f in enumerate(LAYOUTS, 1):
f(os.path.join(HERE, "candidates", "cand%d" % i))
FIG1 = layout_A # layout used in writeup.md
FIG1(os.path.join(HERE, "fig1"))
print("\nWrote fig1.svg/png (layout %s) and candidates/cand1-cand5 in %s"
% (FIG1.__name__[-1], HERE))
Abstract says vitamin D3 rose in Scribble knockout mice; Results and Fig. 3E show a fall
Proc Natl Acad Sci U S A (2023) · doi:10.1073/pnas.2220851120 · PMC10266016
Summary
The abstract states that knocking out Scribble in renal proximal tubules "caused significant plasma phosphate elevation and increased aggregate vitamin D3 levels". The paper defines a single aggregate vitamin D3 measure: three metabolites, each normalized to littermate controls and then combined. Fig. 3E is the only place it is plotted. The Results text and the Fig. 3 legend both describe that measure as decreased, and the points plotted in Fig. 3E agree: digitized, the six tamoxifen-induced knockout values have a mean of 63.2% of control, and 5 of the 6 lie below 100%. Vitamin D is one of the two in vivo findings the abstract reports, so a reader of the abstract alone gets the direction of that finding reversed.
What the paper reports
Abstract (as archived in PMC, after the Significance statement):
The loss of Scribble impacted serum phosphate and vitamin D levels and caused significant plasma phosphate elevation and increased aggregate vitamin D3 levels, whereas blood glucose levels remained unchanged.
Results, section "Loss of Scribble Impacts Renal Phosphate Homeostasis and Serum Vitamin D Levels", second paragraph:
Combined 1,25(OH)2D3 + 24,25(OH)2D3 + 24(OH)D3 levels, as indicated in Methods, decreased only in Tam-injected animals and not in corresponding Cre littermates (Fig. 3E).
Fig. 3 legend:
(B) Targeted Scribble KO mice display increased serum phosphate (C), normal levels of calcium and glucose (D), decreased 1,25[OH]2-vitamin D (1,25[OH]2D) levels (E), and normal levels of PTH and FGF23 (F and G).
Materials and Methods, "Mice.", where the aggregate is defined:
Vitamin D3 was measured as described (52) on littermate pairs. 1,25 vitamin D3, 24,25 vitamin D3, and 25 vitamin D3 forms were normalized to control levels and combined.
Fig. 3E (an image; the axis title and group labels were read from the archived figure file pnas.2220851120fig03.jpg): the y-axis is "Plasma Vitamin D % control", the groups are Veh, Tam and Cre, and a bracket marks Veh vs Tam with "*" and Veh vs Cre with "ns".
Statements elsewhere that give no direction. Significance statement:
The loss of this interaction impacts serum phosphate and vitamin D levels in Scribble KO mice.
Discussion, second paragraph:
Targeted loss of Scribble in proximal tubules resulted in pronounced alterations to serum phosphate and vitamin D levels, whereas glucose, calcium, PTH and FGF23 levels remained unaffected.
Discussion, third paragraph, on mechanism:
PTHR activates the 25-hydroxyvitamin D3-1α-hydroxylase catalyzing the formation of biologically active 1,25(OH)2-vitamin D3.
What the paper's own numbers show
Method. The text gives no vitamin D values, means or test statistics, so the direction can be checked in two ways: by comparing what the text says in each place, and by reading the plotted points in Fig. 3E. For the second, I took the figure as archived in PMC and Europe PMC (708 × 927 px). I calibrated the y-axis from its five tick marks (0, 50, 100, 150 and 200%; largest residual 0.5 percentage points) and located each dot as the centroid of its connected block of dark pixels. As a check, the six Veh (control) dots, which are fixed at 100% by the normalization, come out at 99.8%. The mean computed from the digitized dots matches the mean bar drawn in the figure to within 0.7 percentage points for Tam and exactly for Cre. The paper does not say which test produced the "*". Because each knockout value is already a ratio to its own littermate control, a one-sample test against 100% is the natural analogue, and I report it only to show that the plotted points are consistent with the asterisk. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own figure |
|---|---|---|---|
| Direction of the aggregate vitamin D3 measure | Abstract | "increased" | decrease (opposite sign) |
| Direction of the same measure | Results, second paragraph | "decreased only in Tam-injected animals" | decrease (agrees) |
| Direction of panel E | Fig. 3 legend | "decreased … levels (E)" | decrease (agrees) |
| Control (Veh) dots | Fig. 3E | fixed at 100% | 99.8% (calibration check) |
| Tam (knockout) dots, % of control | Fig. 3E | plotted only | 109.2, 78.0, 55.8, 49.2, 44.0, 42.7 |
| Tam mean (median) | Fig. 3E | mean bar drawn, no value given | 63.2% (52.5%); drawn bar reads 62.5% |
| Change vs control | Fig. 3E | — | −36.8 percentage points |
| Tam dots below 100% | Fig. 3E | — | 5 of 6 |
| Tam 95% CI of the mean (t, df = 5) | Fig. 3E | — | 35.9% to 90.4%; excludes 100% |
| Test of Tam vs control | Fig. 3E | "*" | one-sample t = −3.47, two-sided p = 0.018; exact Wilcoxon p = 0.0625 |
| Cre-only littermates, % of control | Fig. 3E | "ns" | 150.8, 133.4, 126.0, 109.9, 78.5, 69.2; mean 111.3% (95% CI 77.8 to 144.9; p = 0.43) |
| Digitizing error needed to move the Tam mean to 100% | — | — | 21.9 px for every dot; a dot is about 5 px across |
The direction does not depend on how precisely the dots were located: every Tam dot would have to be misplaced by about four dot diameters for the mean to reach 100%. The only group whose mean lies above 100% is Cre, and that difference is marked "ns" and is not attributed to loss of Scribble.
Figure
Figure 1. Top: the two text statements about the same aggregate vitamin D measure. The abstract (orange outline) says "increased"; the Results text and Fig. 3 legend (grey outline) say "decreased". Bottom: the six knockout (Tam) values read from Fig. 3E, as % of littermate control (open blue circles), with their mean of 63.2% (filled blue circle) and its t-based 95% interval of 35.9% to 90.4% (blue line); the vertical line marks the control level of 100%. Blue values are recomputed by recompute.py.txt from the paper's own figure; they are not values printed in the paper. The axis starts at zero.
Design candidates for review.
recompute.py.txtalso writes five candidate layouts of this figure,fig1_Atofig1_E(fig1 is currently design D). Only the chosen design will be kept as fig1.
Why it matters
- The abstract reverses one of its two in vivo findings. Plasma phosphate and vitamin D are the only in vivo results in the abstract. Many readers, indexers and citing authors see only the abstract, and from it they will take an increase in vitamin D3 that the paper's data do not show.
- The direction matters physiologically. High plasma phosphate with lower vitamin D metabolites and unchanged PTH and FGF23, which is what the Results and Fig. 3 report, fits reduced PTHR signaling in the proximal tubule. The paper's own Discussion notes that PTHR activates the 1α-hydroxylase that forms 1,25(OH)2D3. Higher vitamin D, as the abstract states, would point the other way.
- The overall conclusion is not affected. The paper's broader claim, that Scribble loss alters phosphate and vitamin D homeostasis, holds whichever way vitamin D moved. The error is in how the abstract describes the result, not in the data or the analysis.
Explanations considered
- A different vitamin D measure. Not supported. Methods define one aggregate ("normalized to control levels and combined"), Fig. 3E is the only vitamin D panel, and the SI Appendix (4 pages: Figs. S1–S2 and two movie legends) contains no vitamin D data. "Aggregate vitamin D3" in the abstract can only refer to the Fig. 3E measure, which the Results call "Combined" levels.
- A different comparison group. Not supported. The Cre-only littermates have a mean slightly above 100% (111.3%), but the figure marks that difference "ns", and the abstract attributes the change to "the loss of Scribble", which is the Tam group. The Results say the change occurred "only in Tam-injected animals and not in corresponding Cre littermates".
- An inverted scale, such as control divided by knockout. Not supported. The axis is "% control" with the controls fixed at 100, and both the Results and the legend use "decreased" for the same panel.
- A conversion or typesetting artifact. Not possible here. The conflict is between the words "increased" and "decreased", not a sign, superscript or digit. The Europe PMC full-text XML deposited for this article carries the same abstract, Results and legend wording.
- Digitizing error. Not able to explain it. The calibration reproduces the control row (99.8%) and both drawn mean bars, and a reversal would need every Tam dot to be misplaced by about 22 px.
- A correction already issued. None found (see Status).
- Side inconsistencies that do not change the direction. The Results name the third metabolite "24(OH)D3", while Methods list "25 vitamin D3". The legend calls panel E "1,25[OH]2-vitamin D" although the panel shows the combined measure. The legend's panel letters follow the items rather than preceding them ("(B) … increased serum phosphate (C)"), while the Results cite phosphate as Fig. 3B. Panel E is labelled as vitamin D in the figure itself, so none of these affects the direction.
Status
- Version of record. The publisher's page (pnas.org, reached through the DOI) returned HTTP 403 to automated requests. I checked the Europe PMC full-text XML deposited for the article, and its abstract, Results paragraph, Fig. 3 legend and Methods sentence match the PMC archive word for word. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. The Fig. 3E axis title, group labels and significance marks were read from the published figure image. - Supplements. I downloaded the Europe PMC supplementary bundle. It holds the SI Appendix (
pnas.2220851120.sapp.pdf, 4 pages), two molecular dynamics movies and the three main figure images. A text search of the SI Appendix finds "vitamin" only in the title and no vitamin D data. The Data Availability statement places all data in the article or SI, and there is no source-data file. - Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Europe PMC record (PMID 37252981) lists no comments or corrections, and a Europe PMC search for an erratum on this title returned none. PubPeer's search page returned HTTP 403 and its API returned no record to automated requests. A web search for the title together with "PubPeer" found no thread for this article. - Sources. PMC article PMC10266016; Europe PMC full-text XML and supplementary files for PMC10266016; figure image
pnas.2220851120fig03.jpg; SI Appendixpnas.2220851120.sapp.pdf; Crossref; Europe PMC; PubPeer; https://doi.org/10.1073/pnas.2220851120.
Suggested correction
In the abstract, replace
caused significant plasma phosphate elevation and increased aggregate vitamin D3 levels
with
caused significant plasma phosphate elevation and decreased aggregate vitamin D3 levels
so that the abstract agrees with the Results text, the Fig. 3 legend and the data in Fig. 3E. The authors may also wish to make the third metabolite the same in Results ("24(OH)D3") and Methods ("25 vitamin D3"), to describe panel E in the legend as the combined measure, and to give the test used for Fig. 3E with its n and P value.
Recomputation script (python recompute.py.txt)
"""Recompute the direction of the vitamin D result in PMC10266016
(Proc Natl Acad Sci U S A 2023, doi:10.1073/pnas.2220851120).
Run: python recompute.py.txt (uses the pixel positions recorded below)
python recompute.py.txt <fig03.jpg> (also re-measures them from the image)
Needs numpy, scipy and matplotlib (plus Pillow for the optional re-measurement).
Prints every recomputed number used in writeup.md and writes fig1.svg/png and the
five candidate designs fig1_A ... fig1_E (.svg/.png) next to this file.
Input: Fig. 3E of the paper ("Plasma Vitamin D % control"), as archived in PMC /
Europe PMC (pnas.2220851120fig03.jpg, 708 x 927 px). Panel E is the paper's only
vitamin D data. The plotted values are not printed anywhere in the text, so they were
read off the image: dark pixels (grey < 110) in the panel were grouped into connected
blobs and each dot's centre taken as the blob centroid; two Cre dots that touch were
split by their row profile, and one Cre dot drawn on top of its mean bar was read from
the row profile too (the re-measurement prints the blobs it can separate automatically).
The y-axis was calibrated from the five tick marks.
"""
import os
import sys
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (Fig. 3E pixels)
TICK_ROW = np.array([448.0, 478.0, 508.0, 537.5, 567.0]) # tick marks on the y-axis (x = 71-75)
TICK_VAL = np.array([200.0, 150.0, 100.0, 50.0, 0.0]) # tick labels, % control
VEH_ROW = 507.8 # six Veh dots, one merged row
TAM_ROWS = np.array([502.2, 520.8, 534.0, 537.9, 541.0, 541.8]) # six Tam dots
TAM_BAR_ROW = 530.0 # Tam mean bar as drawn
CRE_ROWS = np.array([477.5, 487.8, 492.2, 501.8, 520.5, 526.0]) # six Cre dots
CRE_BAR_ROW = 501.0 # Cre mean bar as drawn
# ---------------------------------------------------------------- inputs (text, verbatim)
Q_ABSTRACT = ("caused significant plasma phosphate elevation and increased "
"aggregate vitamin D3 levels")
Q_RESULTS = ("Combined 1,25(OH)2D3 + 24,25(OH)2D3 + 24(OH)D3 levels ... "
"decreased only in Tam-injected animals")
Q_LEGEND = "decreased 1,25[OH]2-vitamin D (1,25[OH]2D) levels (E)"
Q_METHODS = ("1,25 vitamin D3, 24,25 vitamin D3, and 25 vitamin D3 forms were "
"normalized to control levels and combined.")
def remeasure(path):
"""Re-detect the Fig. 3E dots from the archived JPEG and print their rows."""
from PIL import Image
from scipy import ndimage
im = np.asarray(Image.open(path).convert("L")).astype(float)
y0, x0 = 440, 60
sub = im[y0:580, x0:220] < 110
lab, n = ndimage.label(sub)
print(f"re-measured from {os.path.basename(path)} ({im.shape[1]}x{im.shape[0]} px):")
for i, sl in enumerate(ndimage.find_objects(lab)):
area = int((lab[sl] == i + 1).sum())
cy, cx = ndimage.center_of_mass(lab == i + 1)
w = sl[1].stop - sl[1].start
h = sl[0].stop - sl[0].start
if 15 <= area <= 50 and w <= 7 and 135 <= cx + x0 <= 205 and cy + y0 > 470:
grp = "Tam" if cx + x0 < 170 else "Cre"
note = " (two touching dots)" if h > 8 else ""
print(f" {grp} dot x={cx + x0:6.1f} row={cy + y0:6.1f} area={area}{note}")
print()
# ---------------------------------------------------------------- calculation
slope, icpt = np.polyfit(TICK_ROW, TICK_VAL, 1)
resid = TICK_VAL - (slope * TICK_ROW + icpt)
def pct(row):
return slope * np.asarray(row, dtype=float) + icpt
veh = float(pct(VEH_ROW))
tam = pct(TAM_ROWS)
cre = pct(CRE_ROWS)
tam_bar = float(pct(TAM_BAR_ROW))
cre_bar = float(pct(CRE_BAR_ROW))
n = len(tam)
tam_mean = tam.mean()
tam_sd = tam.std(ddof=1)
tam_se = tam_sd / np.sqrt(n)
tcrit = stats.t.ppf(0.975, n - 1)
tam_ci = (tam_mean - tcrit * tam_se, tam_mean + tcrit * tam_se)
tt = stats.ttest_1samp(tam, 100.0)
wx = stats.wilcoxon(tam - 100.0)
cre_tt = stats.ttest_1samp(cre, 100.0)
cre_ci = (cre.mean() - tcrit * cre.std(ddof=1) / np.sqrt(n),
cre.mean() + tcrit * cre.std(ddof=1) / np.sqrt(n))
# how far the digitizing would have to be off for the sign to flip
px_per_pct = 1.0 / abs(slope)
shift_to_100 = (100.0 - tam_mean) * px_per_pct
def report():
print("Fig. 3E, plasma vitamin D (% control), digitized")
print(f" axis calibration: {slope:.4f} % per px (max tick residual {abs(resid).max():.2f} %)")
print(f" Veh row -> {veh:.1f} % (should be 100: the controls are the normalizing base)")
print(" Tam points:", ", ".join(f"{v:.1f}" for v in tam))
print(f" Tam mean {tam_mean:.1f} %, median {np.median(tam):.1f} %, SD {tam_sd:.1f}")
print(f" Tam mean bar as drawn: {tam_bar:.1f} %")
print(f" Tam change vs control: {tam_mean - 100:+.1f} percentage points")
print(f" Tam points below 100 %: {(tam < 100).sum()} of {n}")
print(f" Tam 95% CI of mean (t, df={n - 1}): {tam_ci[0]:.1f} to {tam_ci[1]:.1f} %")
print(f" one-sample t vs 100: t = {tt.statistic:.2f}, two-sided p = {tt.pvalue:.4f}")
print(f" Wilcoxon signed-rank vs 100: p = {wx.pvalue:.4f} (exact, n = 6)")
print(" Cre points:", ", ".join(f"{v:.1f}" for v in cre))
print(f" Cre mean {cre.mean():.1f} %, mean bar as drawn {cre_bar:.1f} %, "
f"95% CI {cre_ci[0]:.1f} to {cre_ci[1]:.1f} %, t vs 100 p = {cre_tt.pvalue:.3f}")
print(f" digitizing error needed to move the Tam mean up to 100 %: "
f"{shift_to_100:.1f} px for every dot (dot diameter about 5 px)")
print()
print("Direction of the same aggregate measure, as stated in the paper")
rows = [("Abstract", "increased", "+"),
("Results, 'Loss of Scribble Impacts ...' p1", "decreased", "-"),
("Fig. 3 legend, panel E", "decreased", "-"),
("Fig. 3E plotted data (digitized)",
f"mean {tam_mean:.0f} % of control", "-" if tam_mean < 100 else "+")]
for loc, word, sign in rows:
print(f" {sign} {loc:44s} {word}")
print(" Abstract agrees with the plotted data:", rows[0][2] == rows[3][2])
# ---------------------------------------------------------------- figure style
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
ORANGE = "#eb6834" # the paper's reported statement / value
BLUE = "#2a78d6" # recomputed from the paper's own figure
GRID = "#e4e3df"
PANEL = "#f3f2ef"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "axes.facecolor": BG, "figure.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
SUB = str.maketrans("23", "₂₃")
def vd(s):
"""Typeset vitamin D names with subscripts."""
return (s.replace("(OH)2D3", "(OH)₂D₃").replace("(OH)D3", "(OH)D₃")
.replace("[OH]2", "[OH]₂").replace("D3 ", "D₃ ").replace("D3\n", "D₃\n"))
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200)
plt.close(fig)
def textbox(ax, x, y, w, h, title, body, hl=None, hl_color=ORANGE, edge=INK2, hl2=None):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc=PANEL, ec=edge, lw=1.0, transform=ax.transAxes))
ax.text(x + 0.02, y + h - 0.05, title, transform=ax.transAxes, fontsize=10.5,
fontweight="bold", va="top", color=INK)
ax.text(x + 0.02, y + h - 0.16, body, transform=ax.transAxes, fontsize=10.5,
va="top", color=INK, linespacing=1.45)
if hl2:
ax.text(x + 0.02, y + 0.035, hl2, transform=ax.transAxes, fontsize=10,
fontweight="bold", va="bottom", color=BLUE)
if hl:
ax.text(x + 0.02, y + (0.11 if hl2 else 0.05), hl, transform=ax.transAxes, fontsize=11,
fontweight="bold", va="bottom", color=hl_color)
def blank(figsize):
fig = plt.figure(figsize=figsize)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_axis_off()
return fig, ax
# ---------------------------------------------------------------- A: two text panels
def design_A():
fig, ax = blank((W, 3.7))
ax.text(0.03, 0.95, "The same aggregate vitamin D measure, described in opposite directions",
fontsize=11.5, fontweight="bold", va="top", transform=ax.transAxes)
textbox(ax, 0.03, 0.12, 0.40, 0.70, "Abstract",
vd("“…caused significant plasma\nphosphate elevation and\n"
"increased aggregate vitamin D3 \nlevels…”"),
hl="▲ increased", hl_color=ORANGE, edge=ORANGE)
textbox(ax, 0.57, 0.12, 0.40, 0.70, "Results text and Fig. 3 legend",
vd("Results: “Combined 1,25(OH)2D3 +\n24,25(OH)2D3 + 24(OH)D3 levels …\n"
"decreased only in Tam-injected\nanimals”\n"
"Legend: “decreased 1,25[OH]2-\nvitamin D … levels (E)”"),
hl="▼ decreased", hl_color=INK2,
hl2=f"Fig. 3E: Tam mean {tam_mean:.0f}% of control")
ax.add_patch(FancyArrowPatch((0.435, 0.47), (0.565, 0.47), transform=ax.transAxes,
arrowstyle="<->", mutation_scale=14, lw=2, color=INK2))
ax.text(0.5, 0.55, "opposite\nsign", ha="center", va="bottom", fontsize=10.5,
color=INK, transform=ax.transAxes, linespacing=1.2)
ax.text(0.5, 0.40, "same\nmeasure\n(Methods)", ha="center", va="top", fontsize=10,
color=INK2, transform=ax.transAxes, linespacing=1.2)
ax.text(0.03, 0.03, "Orange: the abstract's statement. Blue: recomputed from the "
"points plotted in Fig. 3E.", fontsize=10, color=INK2, transform=ax.transAxes)
return fig
# ---------------------------------------------------------------- B: direction ledger
def design_B():
fig, ax = blank((W, 3.2))
ax.text(0.03, 0.95, "Every place the paper gives the direction of the vitamin D change",
fontsize=11.5, fontweight="bold", va="top", transform=ax.transAxes)
rows = [("Abstract", vd("“increased aggregate vitamin D3 levels”"), "up", ORANGE,
"stated in the paper"),
("Results, p1", vd("“…decreased only in Tam-injected animals”"), "down",
INK2, "stated in the paper"),
("Fig. 3 legend", vd("“decreased 1,25[OH]2-vitamin D … levels (E)”"),
"down", INK2, "stated in the paper"),
("Fig. 3E data", f"Tam mean {tam_mean:.1f}% of control; "
f"{(tam < 100).sum()} of {n} dots below 100%", "down", BLUE,
"recomputed (digitized)")]
ax.text(0.80, 0.81, "increase", ha="center", fontsize=10, color=INK2, transform=ax.transAxes)
ax.text(0.92, 0.81, "decrease", ha="center", fontsize=10, color=INK2, transform=ax.transAxes)
for i, (loc, txt, d, col, src) in enumerate(rows):
y = 0.68 - i * 0.165
ax.plot([0.03, 0.97], [y - 0.075, y - 0.075], color=GRID, lw=1, transform=ax.transAxes)
ax.text(0.03, y + 0.02, loc, fontweight="bold", fontsize=10.5, va="center",
transform=ax.transAxes)
ax.text(0.03, y - 0.04, src, fontsize=10, color=INK2, va="center",
transform=ax.transAxes)
ax.text(0.22, y, txt, fontsize=10.5, va="center", transform=ax.transAxes)
xm = 0.80 if d == "up" else 0.92
ax.text(xm, y, "▲" if d == "up" else "▼", fontsize=15, ha="center",
va="center", color=col, transform=ax.transAxes)
ax.text(0.03, 0.02, "Orange: the abstract. Blue: recomputed from the plotted points. "
"Grey: the other text.", fontsize=10, color=INK2,
transform=ax.transAxes)
return fig
# ---------------------------------------------------------------- C: one-axis direction scale
def design_C():
fig, ax = plt.subplots(figsize=(W, 2.9))
fig.subplots_adjust(left=0.04, right=0.97, top=0.80, bottom=0.25)
ax.set_xlim(0, 160)
ax.set_ylim(-1, 1.55)
ax.set_yticks([])
for s in ("left", "right", "top"):
ax.spines[s].set_visible(False)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.axvline(100, color=INK2, lw=1)
ax.text(100, 1.50, "control = 100%", ha="center", va="bottom", fontsize=10, color=INK2)
# abstract: direction only (no number given) -> arrow into the >100 region
ax.annotate("", xy=(150, 0.65), xytext=(101, 0.65),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=14))
ax.text(103, 0.78, "Abstract: “increased”\n(direction only, no value)",
fontsize=10.5, color=INK, va="bottom", linespacing=1.25)
# recomputed
ax.scatter(tam, np.full(n, -0.35), s=42, facecolor=BG, edgecolor=BLUE, lw=1.5, zorder=3)
ax.plot(tam_ci, [-0.35, -0.35], color=BLUE, lw=2, zorder=2)
ax.scatter([tam_mean], [-0.35], s=80, color=BLUE, zorder=4)
ax.text(tam_mean, -0.05, f"Fig. 3E Tam mean {tam_mean:.1f}% (95% CI {tam_ci[0]:.0f}–"
f"{tam_ci[1]:.0f}), recomputed", ha="center", fontsize=10.5, va="bottom")
ax.text(2, -0.85, "Results and legend: “decreased”", fontsize=10, color=INK2)
ax.set_xlabel("Combined plasma vitamin D in Scribble KO (Tam) mice, % of littermate control",
fontsize=10.5)
fig.text(0.04, 0.93, "The abstract's direction lies on the wrong side of the control line",
fontsize=11.5, fontweight="bold")
return fig
# ---------------------------------------------------------------- D: text + strip
def design_D():
fig = plt.figure(figsize=(W, 3.9))
ax = fig.add_axes([0, 0.34, 1, 0.66])
ax.set_axis_off()
ax.text(0.03, 0.95, "Abstract versus the paper's own text and data on vitamin D",
fontsize=11.5, fontweight="bold", va="top", transform=ax.transAxes)
textbox(ax, 0.03, 0.04, 0.40, 0.74, "Abstract",
vd("“…plasma phosphate elevation\nand increased aggregate\nvitamin D3 "
"levels…”"), hl="▲ increased", hl_color=ORANGE, edge=ORANGE)
textbox(ax, 0.57, 0.04, 0.40, 0.74, "Results text and Fig. 3 legend",
vd("“…decreased only in Tam-\ninjected animals” (Results)\n"
"“decreased 1,25[OH]2-vitamin D\n… levels (E)” (legend)"),
hl="▼ decreased", hl_color=INK2)
ax.add_patch(FancyArrowPatch((0.435, 0.41), (0.565, 0.41), transform=ax.transAxes,
arrowstyle="<->", mutation_scale=14, lw=2, color=INK2))
ax.text(0.5, 0.47, "opposite\nsign", ha="center", va="bottom", fontsize=10.5,
transform=ax.transAxes, linespacing=1.2)
s = fig.add_axes([0.08, 0.16, 0.86, 0.12])
s.set_xlim(0, 160)
s.set_ylim(-1, 1)
s.set_yticks([])
for sp in ("left", "right", "top"):
s.spines[sp].set_visible(False)
s.grid(axis="x", color=GRID, lw=0.8)
s.set_axisbelow(True)
s.axvline(100, color=INK2, lw=1)
s.scatter(tam, np.zeros(n), s=40, facecolor=BG, edgecolor=BLUE, lw=1.5, zorder=3)
s.plot(tam_ci, [0, 0], color=BLUE, lw=2)
s.scatter([tam_mean], [0], s=80, color=BLUE, zorder=4)
s.text(103, 0.15, "control = 100%", fontsize=10, color=INK2, va="bottom")
s.set_xlabel(f"Fig. 3E, Tam mice: digitized points, mean {tam_mean:.1f}% and 95% CI "
f"({tam_ci[0]:.0f}–{tam_ci[1]:.0f}), % of control (recomputed)",
fontsize=10, color=INK)
return fig
# ---------------------------------------------------------------- E: flow of the measure
def design_E():
fig, ax = blank((W, 3.0))
ax.text(0.03, 0.95, "One measure, traced from its definition to the abstract",
fontsize=11.5, fontweight="bold", va="top", transform=ax.transAxes)
boxes = [(0.03, "Methods", "1,25 + 24,25 +\n25 vitamin D₃,\nnormalized to\ncontrol, combined", INK2, None),
(0.27, "Fig. 3E data", f"Tam mean\n{tam_mean:.1f}% of control\n"
f"{(tam < 100).sum()} of {n} below 100", BLUE, "▼"),
(0.51, "Results, legend", "“decreased”\n“decreased”", INK2,
"▼"),
(0.75, "Abstract", "“increased\naggregate\nvitamin D₃\nlevels”", ORANGE,
"▲")]
for x, t, b, c, mark in boxes:
ax.add_patch(FancyBboxPatch((x, 0.22), 0.21, 0.58,
boxstyle="round,pad=0,rounding_size=0.015", fc=PANEL,
ec=c, lw=1.4 if c != INK2 else 1.0, transform=ax.transAxes))
ax.text(x + 0.015, 0.75, t, fontweight="bold", fontsize=10.5, va="top",
transform=ax.transAxes)
ax.text(x + 0.015, 0.63, b, fontsize=10.5, va="top", transform=ax.transAxes,
linespacing=1.35)
if mark:
ax.text(x + 0.015, 0.26, mark, fontsize=16, ha="left", va="bottom", color=c,
transform=ax.transAxes)
for x0 in (0.24, 0.48, 0.72):
ok = x0 < 0.7
ax.add_patch(FancyArrowPatch((x0, 0.51), (x0 + 0.03, 0.51), transform=ax.transAxes,
arrowstyle="-|>", mutation_scale=12, lw=2,
color=INK2 if ok else ORANGE))
ax.text(0.735, 0.14, "sign flips here", ha="center", fontsize=10.5, color=INK,
fontweight="bold", transform=ax.transAxes)
ax.text(0.03, 0.03, "Blue: recomputed from the plotted points. Orange: the abstract's "
"statement.", fontsize=10, color=INK2, transform=ax.transAxes)
return fig
DESIGNS = {"A": design_A, "B": design_B, "C": design_C, "D": design_D, "E": design_E}
CHOSEN = "D" # the design used as fig1 in writeup.md
if __name__ == "__main__":
if len(sys.argv) > 1:
remeasure(sys.argv[1])
report()
for k, f in DESIGNS.items():
save(f(), f"fig1_{k}")
save(DESIGNS[CHOSEN](), "fig1")
print(f"\nwrote fig1_A..fig1_E (.svg/.png) and fig1 = design {CHOSEN} to {HERE}")
Abstract's tissue-culture counts (1/6, 6/6) conflict with the paper's Results and design (2/8, 8/8)
Proc Natl Acad Sci U S A (2024) · doi:10.1073/pnas.2318627121 · PMC11331133
Summary
The abstract reports the study's postmortem tissue-culture result as MAP-positive tissue in "1 out of 6" phage-treated calves and "6 out of 6" infected control calves. The Methods and the Fig. 1 legend assign 8 calves to each infected group, and the Results, using the same 49-day tissue culture and duplex qPCR at the single necropsy (16 weeks post-inoculation), report 8 MAP-positive calves in the MAP-only group and 2 in the phage-treated group, which is 8/8 and 2/8, with no calf reported as excluded. The direction and significance of the protective effect are unchanged (recomputed Fisher exact p = 0.015 for the abstract's counts, 0.0070 for the Results'), but the abstract understates the infected calves in each group as 6 rather than 8, and puts residual infection in the phage-treated group at 16.7% (1 calf) rather than 25.0% (2 calves).
What the paper reports
Abstract, the postmortem result:
Postmortem tissue culture analysis further supported the effectiveness of MPs, with only 1 out of 6 animals in the phage-treated group testing positive for MAP colonized tissues compared to 6 out of 6 animals in the infected control group.
Results, "Oral Administration of Phages Prevented Establishment of MAP Infection in Intestinal Tissues", first paragraph:
To quantify the prevention of infection, MAP infection status was measured in intestinal tissues with high sensitivity (20) via duplex qPCR after samples were cultured for 49 d.
Within the MAP-only group, there were eight MAP-positive calves, each with 3 to 6 positive tissue types (Fig. 4A). However, in the phage-treated group, only two calves were MAP positive, with only one positive tissue type each.
Therefore, MP administration strongly reduced MAP-positive tissues (P value > 0.01) and provided almost complete protection against MAP infection.
Materials and Methods, "Study Design":
The first eight calves were designated the MAP-inoculated control group (MAP-only group). The next eight calves to arrive were designated the phage-treated MAP infection group (phage-treated group). The last three calves were designated the uninfected control (control).
Fig. 1 legend:
Newborn dairy calves (n = 19) were collected within 12 h of birth and randomly assigned into three groups: MAP-only (n = 8), phage-treated (n = 8), and control (n = 3).
Materials and Methods, "Inoculum":
Pellets were prepared per group (n = 8) per day.
Materials and Methods, "Necropsies and Tissue Cultures":
Calves were killed 16 wk postinoculation
Six tissue samples per calf were obtained: ileum, distal jejunum, ileocecal valve, and their associated lymph nodes.
The only "6 out of 8" in the paper concerns phage recovery, not MAP. Results, "Phages Were Persistent in the Small Intestine...":
MPs were detected in both the ileum and jejunum of 6 out of 8 of the phage-treated calves. The inability to recover MPs from the tissues of two calves was attributed to a documented sampling error with these specific samples
The only exclusions reported are for the blood assay. Results, first section:
Two samples were excluded from our results based on cell viability assessments (19).
The conclusion that rests on the tissue result, Discussion, first sentence:
This study demonstrated the ability of MPs to provide almost complete protection against MAP infection and the ability to eliminate fecal shedding by calves.
What the paper's own numbers show
Method. I took the group sizes from the Methods and the Fig. 1 legend and the numbers of MAP-positive calves from the Results text. I computed animal-level positivity for each group, with exact (Clopper–Pearson) 95% intervals, and did the same for the abstract's counts. For both versions I computed the risk difference, the relative reduction and a two-sided Fisher exact test on the 2×2 table. The paper itself reports only per-tissue chi-square tests, so the Fisher test is my addition and serves only to compare the two versions. I worked out which calves would have to be left out to turn the Results' counts into the abstract's. As a cross-check on the per-calf counts, I read the bar heights of the paper's Fig. 4A, which gives the number of positive animals for each tissue type. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Calves per group | Methods; Fig. 1 legend; Inoculum | MAP-only 8, phage-treated 8, control 3; total 19 | 8 + 8 + 3 = 19, consistent; 16 infected calves |
| Phage-treated calves with MAP-positive tissue | Abstract | 1 out of 6 | 16.7% (exact 95% CI 0.4–64.1%) |
| Phage-treated calves with MAP-positive tissue | Results, tissue paragraph | "only two calves were MAP positive" | 2/8 = 25.0% (exact 95% CI 3.2–65.1%) |
| Infected-control calves with MAP-positive tissue | Abstract | 6 out of 6 | 100% (exact 95% CI 54.1–100%) |
| MAP-only calves with MAP-positive tissue | Results, tissue paragraph | "eight MAP-positive calves" | 8/8 = 100% (exact 95% CI 63.1–100%) |
| Risk difference, phage minus control | derived | — | abstract −83.3 points; Results −75.0 points |
| Relative reduction in positive calves | derived | — | abstract 83.3%; Results 75.0% |
| Fisher exact test, two-sided | derived (not in the paper) | — | abstract [[1, 5], [6, 0]] p = 0.0152; Results [[2, 6], [8, 0]] p = 0.0070 |
| Calves that would have to be left out to get the abstract's counts | derived | — | phage-treated: 1 positive and 1 negative; MAP-only: 2 positives |
| Most MAP-only animals positive in any one tissue type | Fig. 4A (read from image) | 8 (jejunum LN) | consistent with 8 positive calves of 8 |
| MAP-only positive tissues, summed over the six types | Fig. 4A (read from image) | 7, 7, 6, 7, 8, 3 | 38 over 8 calves = 4.75 per calf, consistent with "3 to 6 positive tissue types" each |
| Phage-treated positive tissues | Fig. 4A (read from image) | jejunum 1, ileocecal valve 1 | 2 tissues in 2 different tissue types, consistent with "two calves ... one positive tissue type each" |
The Results text, the Methods group sizes and Fig. 4A agree with each other: 8 of 8 MAP-only calves and 2 of 8 phage-treated calves had at least one MAP-positive tissue. The abstract is the only place where the result appears as 1/6 and 6/6.
Three smaller inconsistencies turned up during the check. None affects the claim above or any conclusion.
| Quantity | Location | Reported | Recomputed / note |
|---|---|---|---|
| MAP-only fecal shedding "increased from 60 to 100%" | Results, fecal shedding | 60% | 60% of 8 = 4.8 calves, not a whole number. Fig. 3 shows 3/8 = 37.5% (F57) or 5/8 = 62.5% (IS900) at 10 wk PI, and 8/8 at 16 wk |
| Significance of the tissue reduction | Results, tissue paragraph | "(P value > 0.01)" | inconsistent with "strongly reduced"; Fig. 4A marks the significant tissue types with asterisks, so "<" was probably meant |
| qPCR positivity cutoff | Methods "qPCR Procedure"; Fig. 4 legend; Fig. 3 legend | "Ct values ≤ 37 considered positive"; "Ct < 37"; "Ct ... >37.0" | the Fig. 3 legend reverses the direction of the cutoff |
Figure
Figure 1. The abstract's sentence (left, orange) and the corresponding statements in the Results and Methods (right, blue), with the proportions each implies. The two panels refer to the same assay (49-day tissue culture followed by duplex F57/IS900 qPCR) and the same single necropsy at 16 weeks post-inoculation, and the paper reports no calf excluded from it. The numerators disagree (1 vs 2 positive phage-treated calves; 6 vs 8 positive infected controls), and so do the denominators (6 vs the 8 calves enrolled per group). The Fisher exact p-values at the bottom were recomputed from each pair of counts; the paper does not report them.
Figure 2. One square per calf. Filled squares are calves with MAP-positive tissue, hollow squares are calves without, and dashed outlines are calves enrolled in the group (n = 8, Methods and Fig. 1 legend) that the abstract's count leaves out. Orange rows are the abstract's counts. Blue rows are the Results' counts set against the Methods group size. Going from the Results to the abstract takes away one positive and one negative calf in the phage-treated group and two positive calves in the infected control group.
Why it matters
The postmortem tissue result is the study's direct measure of infection, and the abstract is where most readers, and abstracting services, will take it from. As published, the abstract:
- Misstates the sample size. It describes two groups of 6 animals (12 in total) where the study enrolled two infected groups of 8 (16 in total). A reader using the abstract would take the study to be smaller than it was.
- Overstates the protective effect somewhat. It puts residual infection in the phage-treated group at 1 calf (16.7%) instead of 2 (25.0%), and the relative reduction at 83.3% instead of 75.0%. With groups this small, each calf moves the proportion by 12.5 points.
- Does not change the qualitative conclusion. Both versions show a large reduction in the same direction, and both are significant by Fisher's exact test (p = 0.015 and p = 0.0070). The exact 95% intervals for the phage-treated proportion are wide either way (0.4–64.1% and 3.2–65.1%). The title, the Significance statement, the Discussion and the Conclusion state the effect qualitatively and are not affected. The fecal-shedding result is separate and is also not affected.
So this is an error in a headline number, not in the study's main finding. It matters for anyone who quotes the abstract's figures, pools them in a meta-analysis or uses them to plan a follow-up study's sample size.
Explanations considered
- Some calves were excluded from the tissue analysis. This does not account for the numbers. The paper reports no death, removal or exclusion of any calf. All calves were killed at 16 weeks post-inoculation, and six tissues were taken from each. The only exclusions mentioned are two or three blood samples for the IFN-γ assay. Even if exclusions had happened, getting from 2/8 to 1/6 means leaving out exactly one positive and one negative phage-treated calf, and getting from 8/8 to 6/6 means leaving out two positive controls. Nothing in the paper describes such a selection, and the Results text itself counts 8 and 2 positive calves.
- The abstract refers to the 6 phage-treated calves from which phages were recovered. This does not account for the numbers. That "6 out of 8" is phage recovery from tissue, with two calves lost to a sampling error. It cannot apply to the infected control group, which received no phage, and the paper does not report MAP status for that subset.
- The abstract uses a different assay. This does not account for the numbers. The abstract says "postmortem tissue culture analysis", which is the 49-day culture plus duplex qPCR behind the Results' 8 and 2 positive calves (Fig. 4A). The direct-extraction F57 data (Fig. 4B and 4C) show a similar per-tissue pattern and give no per-calf count that would give 1/6 or 6/6.
- The abstract uses a different time point. This does not account for the numbers. There is a single necropsy, at 16 weeks post-inoculation.
- The 6 refers to the six tissue types, not to animals. This partly matches the numbers. Fig. 4A shows positive tissue in all 6 tissue types in the MAP-only group, which would give "6 out of 6". In the phage-treated group, however, the two positive tissues were in two different tissue types (jejunum and ileocecal valve), so the same reading gives 2/6, not 1/6. The abstract also says "animals" explicitly. This reading therefore reconciles neither figure as written.
- A conversion or typesetting artifact. This is excluded. The numbers are plain inline integers and words ("1 out of 6", "eight", "two"). They appear identically in the JATS XML from Europe PMC, the PMC HTML page and the PMC archive used here.
- The error was corrected later. As of 2026-10-23 no correction has been issued (see Status).
Status
- Version of record. The abstract sentence and the Results sentences quoted above appear word for word in the Europe PMC full-text XML (the JATS version of record) and on the PMC HTML page, and they match the PMC archive used for this check. The publisher's page on pnas.org returned HTTP 403 to automated access. Every quote above was checked verbatim against the PMC-archived text. The Fig. 3A/3B and 4A counts were read from the PMC copies of the figure images (pnas.2318627121fig03.jpg and pnas.2318627121fig04.jpg).
- Supplements. The SI Appendix (pnas.2318627121.sapp.pdf, 2 pages, from the Europe PMC supplementary-files bundle) holds Supplementary Table 1, which lists phage presence in fecal and tissue samples for each animal. It numbers the calves 1–8 (MAP-only), 9–16 (phage-treated) and 17–19 (uninfected control), which confirms the group sizes of 8. It gives no MAP tissue results, so it does not reconcile the abstract with the Results.
- Corrections and comments. Crossref lists no update relations for the DOI. PMC and Europe PMC (PMID 39102547) list no correction, erratum, expression of concern or retraction. The PubPeer API and search page refused automated access (HTTP 403 / Cloudflare challenge), so PubPeer could not be checked directly. A web search for the title with "pubpeer" returned no PubPeer page.
- Sources. PMC article PMC11331133; Europe PMC full-text XML and supplementary files; Crossref; Europe PMC record PMID 39102547; web search. Checked as of 2026-10-23.
Suggested correction
- Change the abstract sentence to match the Results and the design, for example: "Postmortem tissue culture analysis further supported the effectiveness of MPs, with only 2 out of 8 animals in the phage-treated group testing positive for MAP colonized tissues compared to 8 out of 8 animals in the infected control group." If the abstract's figures came from a different analysis or subset, state that analysis and its denominators in the Methods and Results instead.
- Minor: correct "(P value > 0.01)" in the tissue paragraph, presumably to "<"; give the MAP-only fecal-shedding proportions as counts out of 8 (for example 5/8 at 10 weeks by IS900, or 3/8 by F57), since 60% is not a possible proportion of 8 calves; and change "Ct value of >37.0" in the Fig. 3 legend to match the Methods ("≤ 37").
Recomputation script (python recompute.py.txt)
"""Recompute the animal-level tissue-culture result of PMC11331133
(Proc Natl Acad Sci U S A 2024, doi:10.1073/pnas.2318627121).
Run: python recompute.py.txt
Needs scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract: "only 1 out of 6 animals in the phage-treated group testing positive
for MAP colonized tissues compared to 6 out of 6 animals in the
infected control group"
Results (tissues): "Within the MAP-only group, there were eight MAP-positive calves ...
in the phage-treated group, only two calves were MAP positive"
Methods, Fig. 1: MAP-only n = 8, phage-treated n = 8, control n = 3 (n = 19)
Results (feces): MAP-only shedding "increased from 60 to 100%"
"""
import os
import warnings
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle # noqa: E402
from scipy.stats import beta, fisher_exact # noqa: E402
warnings.filterwarnings("ignore", message=".*findfont.*")
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
GROUP_N = {"MAP-only": 8, "phage-treated": 8, "control": 3} # Methods; Fig. 1 legend
TOTAL_STATED = 19 # Methods; Fig. 1 legend
ABSTRACT = {"phage-treated": (1, 6), "MAP-only": (6, 6)} # (positive, total)
RESULTS_POS = {"phage-treated": 2, "MAP-only": 8} # positive calves
RESULTS = {g: (RESULTS_POS[g], GROUP_N[g]) for g in RESULTS_POS} # total = group size
SHEDDING_START_PCT = 60 # Results, fecal shedding
# Read from the bar heights of the paper's Fig. 4A image (pnas.2318627121fig04.jpg, PMC):
# number of MAP-positive animals per tissue type after 49-d culture + duplex qPCR.
TISSUES = ("ileum", "jejunum", "ileocecal valve", "ileum LN", "jejunum LN", "ileocecal valve LN")
FIG4A = {"MAP-only": (7, 7, 6, 7, 8, 3), "phage-treated": (0, 1, 1, 0, 0, 0)}
# Read from the paper's Fig. 3A/3B image (pnas.2318627121fig03.jpg): MAP-only calves with
# positive fecal culture at 10 wk PI (first week of active shedding) and 16 wk PI.
FIG3_MAP_ONLY = {"F57": {10: 3, 16: 8}, "IS900": {10: 5, 16: 8}}
def cp_ci(k, n, a=0.05):
"""Clopper-Pearson exact 95% interval for k/n."""
lo = 0.0 if k == 0 else beta.ppf(a / 2, k, n - k + 1)
hi = 1.0 if k == n else beta.ppf(1 - a / 2, k + 1, n - k)
return lo, hi
def summarise(label, d):
kp, np_ = d["phage-treated"]
kc, nc = d["MAP-only"]
pp, pc = kp / np_, kc / nc
table = [[kp, np_ - kp], [kc, nc - kc]]
p = fisher_exact(table)[1]
print(f"\n{label}")
for g, (k, n) in (("phage-treated", (kp, np_)), ("MAP-only / infected control", (kc, nc))):
lo, hi = cp_ci(k, n)
print(f" {g:28s} {k}/{n} = {100 * k / n:5.1f}% exact 95% CI {100 * lo:4.1f}-{100 * hi:5.1f}%")
print(f" risk difference (phage - control) = {100 * (pp - pc):.1f} percentage points")
print(f" relative reduction 1 - p_phage/p_control = {100 * (1 - pp / pc):.1f}%")
print(f" 2x2 table {table}: Fisher exact two-sided p = {p:.4f}")
return p
print("=== Design (Methods, Fig. 1 legend)")
s = sum(GROUP_N.values())
print(f"Group sizes {GROUP_N}; sum = {s} (stated {TOTAL_STATED}); consistent = {s == TOTAL_STATED}")
print(f"Infected calves (MAP-only + phage-treated) = {GROUP_N['MAP-only'] + GROUP_N['phage-treated']}")
print("\n=== Animal-level tissue-culture positivity")
p_abs = summarise("Abstract", ABSTRACT)
p_res = summarise("Results, with the Methods group sizes", RESULTS)
print("\n=== What the abstract's figures would require")
for g in ("phage-treated", "MAP-only"):
ka, na = ABSTRACT[g]
kr, nr = RESULTS[g]
drop_pos, drop_neg = kr - ka, (nr - kr) - (na - ka)
print(f"{g}: from {kr}/{nr} to {ka}/{na} requires leaving out {drop_pos} positive "
f"and {drop_neg} negative calves ({nr - na} calves in total)")
print("Calves in the abstract's two groups: 6 + 6 = 12, versus 8 + 8 = 16 infected calves enrolled")
print("\n=== Tissue-type reading (Fig. 4A, read from the image)")
for g in ("MAP-only", "phage-treated"):
c = FIG4A[g]
types = sum(1 for x in c if x > 0)
print(f"{g}: positive tissue types {types}/6; most animals positive in one tissue type = "
f"{max(c)}; positive tissues summed over types = {sum(c)}")
print(f"MAP-only: {sum(FIG4A['MAP-only'])} positive tissues over 8 calves = "
f"{sum(FIG4A['MAP-only']) / 8:.2f} per calf (Results: 3 to 6 each)")
print("Phage-treated: 2 positive tissues in 2 different tissue types (jejunum, ileocecal valve),"
" so a tissue-type count would be 2/6, not 1/6")
print("\n=== Side check: fecal shedding 'from 60 to 100%' in a group of 8")
print("Possible percentages of 8 calves:", ", ".join(f"{k}/8 = {100 * k / 8:.1f}%" for k in range(9)))
print(f"{SHEDDING_START_PCT}% of 8 = {SHEDDING_START_PCT / 100 * 8:.1f} calves (not a whole number); "
f"nearest are 5/8 = 62.5% and 4/8 = 50.0%")
for gene, d in FIG3_MAP_ONLY.items():
print(f"Fig. 3 {gene}: MAP-only calves shedding at 10 wk PI = {d[10]}/8 = {100 * d[10] / 8:.1f}%, "
f"at 16 wk PI = {d[16]}/8 = {100 * d[16] / 8:.1f}%")
# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": ["Inter", "DejaVu Sans"], "font.size": 10.5,
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
print("wrote", name + ".svg and", name + ".png")
# Figure 1: abstract vs Results/Methods, two text panels joined by a connector
fig = plt.figure(figsize=(7.2, 4.9))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.9)
ax.axis("off")
def panel(x, w, colour, title, sub, lines):
y0, h = 1.2, 3.25
ax.add_patch(FancyBboxPatch((x, y0), w, h, boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec=colour, lw=1.5))
ax.add_patch(Rectangle((x, y0 + h - 0.08), w, 0.08, fc=colour, ec="none"))
ax.text(x + 0.15, y0 + h - 0.32, title, fontsize=12, fontweight="bold", va="center")
ax.text(x + 0.15, y0 + h - 0.58, sub, fontsize=10, color=INK2, va="center")
y = y0 + h - 0.9
for kind, txt in lines:
if kind == "q":
ax.text(x + 0.15, y, txt, fontsize=10, color=INK, va="top", style="italic",
linespacing=1.35)
y -= 0.2 * (txt.count("\n") + 1) + 0.2
else:
ax.text(x + 0.15, y, txt, fontsize=11.5, color=colour, va="top", fontweight="bold")
y -= 0.36
panel(0.2, 3.0, ORANGE, "Abstract", "as reported",
[("q", "“only 1 out of 6 animals in the\nphage-treated group testing positive\n"
"for MAP colonized tissues compared\nto 6 out of 6 animals in the\n"
"infected control group”"),
("n", "phage-treated: 1/6 = 16.7%"),
("n", "infected control: 6/6 = 100%")])
panel(4.0, 3.0, BLUE, "Results + Methods", "paper's own counts",
[("q", "“there were eight MAP-positive\ncalves” (MAP-only group)\n"
"“only two calves were MAP positive”\n(phage-treated group)\n"
"Group sizes (Methods, Fig. 1):\nMAP-only n = 8, phage-treated n = 8"),
("n", "phage-treated: 2/8 = 25.0%"),
("n", "MAP-only: 8/8 = 100%")])
ax.annotate("", xy=(4.0, 2.85), xytext=(3.2, 2.85),
arrowprops=dict(arrowstyle="<->", color=INK, lw=2, shrinkA=0, shrinkB=0))
ax.text(3.6, 3.05, "conflict", ha="center", fontsize=10, fontweight="bold")
ax.text(3.6, 0.6, "Same assay (49-d tissue culture + duplex qPCR), same single necropsy at 16 wk PI,\n"
"no exclusions reported. Numerators differ (1 vs 2; 6 vs 8) and denominators differ (6 vs 8).\n"
f"Both versions are significant: Fisher exact p = {p_abs:.3f} (abstract) vs {p_res:.4f} (Results).",
ha="center", va="center", fontsize=10, color=INK2, linespacing=1.4)
ax.text(0.2, 4.68, "Calves with MAP-positive tissue at necropsy: abstract vs body of the paper",
fontsize=12, fontweight="bold", va="center")
save(fig, "fig1")
# Figure 2: one square per calf, so numerator and denominator are both visible
rows = [
("Phage-treated, abstract", ABSTRACT["phage-treated"], ORANGE),
("Phage-treated, Results", RESULTS["phage-treated"], BLUE),
("Infected control, abstract", ABSTRACT["MAP-only"], ORANGE),
("MAP-only, Results", RESULTS["MAP-only"], BLUE),
]
fig, ax = plt.subplots(figsize=(7.2, 3.2))
fig.subplots_adjust(left=0.30, right=0.80, top=0.80, bottom=0.17)
size, gap = 0.8, 0.2
for i, (lab, (k, n), col) in enumerate(rows):
y = [4.0, 3.0, 1.4, 0.4][i]
for j in range(8):
if j < n:
filled = j < k
ax.add_patch(Rectangle((j, y - size / 2), size, size, fc=col if filled else "white",
ec=col, lw=2))
else:
ax.add_patch(Rectangle((j, y - size / 2), size, size, fc="none", ec="#b9b8b3",
lw=1.2, ls=(0, (2, 2))))
ax.text(-0.3, y, lab, ha="right", va="center", fontsize=10.5)
ax.text(8.2, y, f"{k}/{n} = {100 * k / n:.1f}%", ha="left", va="center", fontsize=10.5,
fontweight="bold", color=INK)
ax.set_xlim(-0.1, 8.0)
ax.set_ylim(-0.2, 4.6)
ax.axis("off")
fig.text(0.02, 0.94, "One square per calf: MAP-positive tissue at necropsy", fontsize=12,
fontweight="bold")
fig.text(0.02, 0.87, "Filled = positive, hollow = negative, dashed = calf enrolled (n = 8) "
"but absent from the abstract's count", fontsize=10, color=INK2)
fig.legend(handles=[Rectangle((0, 0), 1, 1, fc=ORANGE, ec=ORANGE),
Rectangle((0, 0), 1, 1, fc=BLUE, ec=BLUE)],
labels=["Reported in the abstract", "Paper's own Results counts, Methods group size"],
loc="lower center", ncol=2, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.0))
save(fig, "fig2")
Title says the 2025–2030 guidelines raise water use; the paper's own results show lower water use
Proc Natl Acad Sci U S A (2026) · doi:10.1073/pnas.2604814123 · PMC13273247
Summary
This PNAS brief report compares diets that follow the 2025–2030 Dietary Guidelines for Americans (no ultraprocessed foods, protein raised to 1.2–1.6 g/kg) with the current Mean American Diet (MAD) on greenhouse gas (GHG) emissions, freshwater use, land use and nitrogen fertilizer use. Its title says the guidelines are "associated with higher land, water and nitrogen use, and greenhouse gas emissions", but its abstract, Results and Discussion all report that water use is "lower by 7 to 19% than the MAD". The paper's own Dataset S1 gives −6.6% to −19.5% for those scenarios (the text's range, to the nearest percent) and lower water use than the MAD in every scenario modelled, while the other three outcomes rise by up to 31.8%. The title thus states the wrong direction for one of the four outcomes it names, and a reader who sees only the title, as indexes, citations and news coverage present it, gets the water result backwards.
What the paper reports
Title:
The 2025–2030 Dietary Guidelines for Americans are associated with higher land, water and nitrogen use, and greenhouse gas emissions
Abstract, which states the comparator and the water result:
Here, we quantified the greenhouse gas (GHG) emissions, freshwater use, land use, and nitrogen fertilizer use of adherence to these new guidelines in comparison to the current Mean American Diet (MAD).
We find that although eliminating ultraprocessed foods from the MAD could yield substantial environmental benefits—given their large share of total intake and animal-based makeup—the recommended increase in animal-based proteins, will more than offset this environmental dividend in GHG, land use, and nitrogen fertilizer use by as much as 32%, but not in water use (i.e., water use is lower by 7 to 19% than the MAD).
Overall, adherence to the 2025–2030 Dietary Guidelines for Americans would confer net environmental harm across most key metrics relative to the prevailing U.S. diet.
Introduction, last paragraph:
We then quantified greenhouse gas (GHG) emissions, freshwater use, land use, and nitrogen fertilizer use for each diet and compared them against the MAD.
Results, second paragraph:
Our results indicate that GHG, land use, and nitrogen fertilizer dividends of eliminating UPFs from the MAD are outweighed by the impacts associated with increased protein intake (1.2 to 1.6 g/kg) by as much as 32%. In contrast, eliminating UPF and increasing protein levels to 2025–2030 DGAs reduced the water footprint compared to MAD by 7 to 19%.
Discussion, first paragraph:
For land use, GHG emissions, and nitrogen fertilizer use the net effects are adverse. For water use, the net effect is beneficial, but would be even more so were the avoidance of UPFs combined with a lower protein threshold, and the sourcing of protein preferentially from plants.
The paper has no tables. Fig. 1 panel B ("freshwater use (panel B)") plots the water results. All quotes above were checked verbatim against the PMC-archived article text.
What the paper's own numbers show
Method. Dataset S1 (pnas.2604814123.sd01.xlsx) has a "summary" sheet. Its rows 5–11 give annual per-capita GHG (kg CO2e), water (litres), land (m²·yr) and nitrogen fertilizer (g Nr), plus daily kcal and protein, for the MAD, the ultraprocessed-food (UPF) component of the MAD, and five UPF-free scenarios: 0.8, 1.2 and 1.6 g/kg, and 1.6 g/kg weighted toward plant or animal sources. I took the cached cell values and computed each scenario's percentage change against the MAD. I then compared the results with the ranges stated in the text. The sheet is identical to the one in the data file that the paper links under Data Availability. The sheet's own "Range" cells (K15:L15) pool GHG, land and nitrogen for the 1.2, 1.6 and 1.6 g/kg animal scenarios, and leave water out. All calculations are in recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Direction of water use, guideline diets vs MAD | Title | "higher … water … use" | Lower in all 5 scenarios (−6.6% to −27.7%) |
| Water, 1.2 g/kg vs MAD | Abstract; Results ¶2; Fig. 1B | "lower by 7 to 19%" (upper end) | −19.45% |
| Water, 1.6 g/kg vs MAD | Abstract; Results ¶2; Fig. 1B | "lower by 7 to 19%" (lower end) | −6.58% |
| Water, 1.6 g/kg animal vs MAD | Fig. 1B | within 7–19% lower | −13.14% |
| Water, 1.6 g/kg plant vs MAD | Fig. 1B | lower (not quantified) | −24.92% |
| Water, 0.8 g/kg vs MAD | Fig. 1B | lower (not quantified) | −27.73% |
| Net water effect | Discussion ¶1 | "beneficial" | Consistent: every scenario is below the MAD |
| GHG, land, N fertilizer vs MAD, 1.2–1.6 g/kg scenarios | Abstract; Results ¶2 | "by as much as 32%" | −2.5% (GHG, 1.2 g/kg) to +31.8% (land, 1.6 g/kg) |
| 1.6 g/kg vs MAD: GHG / land / N | Fig. 1A, C, D | higher | +21.5% / +31.8% / +7.7% |
| UPF share of MAD impacts (GHG, water, land, N) | Results ¶2 | "40 to 58%" | 46.1%, 52.4%, 40.3%, 57.6% |
| Protein, 0.8 / 1.2 / 1.6 g/kg × 84 kg | Results ¶1; Methods | 67 / – / 134 g per day | 67.2 / 100.8 / 134.4 g (modelled 67.3 / 100.8 / 134.3) |
Every number in the abstract, Results and Discussion reproduces from Dataset S1. The title is the only statement that disagrees with the data, and it disagrees only on the direction of water use.
Figure
Figure 1. The title clause (left, orange) lists water among the outcomes that are "higher" under the 2025–2030 guidelines. On the right are the three places where the text reports the water result, each against the Mean American Diet (MAD) (gray): the abstract ("lower by 7 to 19% than the MAD"), the second paragraph of Results ("reduced the water footprint compared to MAD by 7 to 19%") and the first paragraph of Discussion ("the net effect is beneficial"). Below them (blue) are the water changes recomputed from the summary sheet of Dataset S1 for the three scenarios in the guidelines' protein range that the text's 7–19% covers. The 1.6 g/kg plant scenario (−24.9%) and the 0.8 g/kg scenario (−27.7%) are also below the MAD and are not shown in this panel.
Figure 2. Change against the MAD for each of the four outcomes named in the title, in the four UPF-free scenarios at 1.2–1.6 g/kg protein (blue markers, recomputed from Dataset S1; the hollow marker is the plant-weighted 1.6 g/kg diet). The thin vertical line marks no change. Orange marks show what the paper states: the "lower by 7 to 19%" interval for water, the "as much as 32%" maximum for GHG, land and nitrogen (reached for land), and the direction that the title gives for water (arrow pointing to higher values). Land, GHG and nitrogen fertilizer rise in the 1.6 g/kg scenarios. Every water value is below zero, so none matches the direction the title gives.
Why it matters
The title is the paper's headline claim. It names four outcomes and says all four are higher under the guidelines. Against the comparator the paper states throughout (the Mean American Diet), one of those four, freshwater use, goes the other way in every scenario the paper models: it falls by 6.6–19.5% at the guidelines' protein levels and by up to 27.7% overall. The body of the paper reports this correctly and calls the net water effect "beneficial". The abstract's closing sentence is qualified accordingly ("net environmental harm across most key metrics").
The title is what bibliographic databases index, what citing papers and reference lists reproduce, and what headlines and social media posts pick up. A reader who relies on the title, which is the usual case for a widely covered policy paper, comes away with the water result reversed. One news item that links to this PMC record says the guidelines are linked to "16 percent more water consumption", alongside 33% higher emissions and 28% greater land use, and gives no comparator (Species Unite). In Dataset S1 those three figures coincide with the 1.6 g/kg animal versus 1.6 g/kg plant contrast (+32.6% GHG, +27.7% land, +15.7% water). That is a comparison between two guideline-range diets, not between the guidelines and the current diet.
The following results are not affected: the GHG, land-use and nitrogen-fertilizer increases (up to +31.8% against the MAD), the finding that plant-sourced protein does better than animal-sourced protein on every metric, the UPF shares, and the paper's main conclusion of net environmental harm on most metrics. The paper reports scenario point estimates without inferential statistics, so no test or interval changes.
Explanations considered
- A different comparator. The title names no comparator. Within the UPF-free scenarios, water use does rise with protein: the 1.2 g/kg and 1.6 g/kg diets use 11.4% and 29.3% more water than the 0.8 g/kg diet. Read against a UPF-free 0.8 g/kg diet, "higher water" would be true. The paper, however, states its comparator explicitly in the abstract ("in comparison to the current Mean American Diet (MAD)"), the Introduction ("compared them against the MAD") and the abstract's conclusion ("relative to the prevailing U.S. diet"). In the title, "higher" applies to land, nitrogen and GHG on the MAD comparison. The abstract singles water out as the exception to that comparison ("but not in water use"). This does not account for the title.
- The marginal effect of the added protein. The Discussion notes that water use would be lower still "were the avoidance of UPFs combined with a lower protein threshold". This describes the protein increase as eating into a water saving, not as producing higher water use than the current diet. The same paragraph calls the net effect "beneficial". This does not account for the title.
- Animal-weighted versus plant-weighted sourcing. The 1.6 g/kg animal diet uses 15.7% more water than the 1.6 g/kg plant diet. That is a comparison between two guideline-range diets. The title describes the guidelines as a whole, and even the animal-weighted diet uses 13.1% less water than the MAD. This does not account for the title.
- A scenario with higher water use. None exists. All five modelled scenarios are below the MAD on water (−6.6% to −27.7%). This does not account for the title.
- Normalisation. All diets are isocaloric (about 2,500 kcal/day), so water per 1,000 kcal has the same sign as the totals. Per gram of protein, every guideline-range diet is also lower than the MAD (−15.4% to −40.8%). This does not account for the title.
- Uncertainty. The paper gives no intervals, so it cannot be judged whether the smallest reduction (−6.6%) is distinguishable from zero. The paper itself, though, states the direction as lower in three places and never as higher or as uncertain. This does not account for the title.
- Text-conversion artifact. The conflict is between words ("higher" against "lower", "reduced", "beneficial"). It does not depend on a sign, symbol or table cell. The same title is in the Crossref metadata deposited by the publisher, the Europe PMC record and the live PMC page. An archiving artifact is ruled out.
Status
As of 2026-10-23:
- Version of record. The publisher's site (pnas.org, the DOI target) returned HTTP 403 from this machine, so its HTML and PDF could not be read. The copies checked were the PMC-archived text (via
src/show.py), the live PMC page, the Europe PMC full-text XML and record (PMID 42224503), and the Crossref metadata title. All carry the same title and the same abstract, Results and Discussion sentences. - Supplements. Dataset S1 (pnas.2604814123.sd01.xlsx, downloaded via Europe PMC) was read. Its "summary" sheet gives the scenario totals used above, and the same sheet appears unchanged in the authors' linked data file. It confirms the text's 7–19% water reduction and shows no scenario with higher water use. The supplement supports the body text and does not resolve the conflict with the title.
- Corrections. No correction, erratum, expression of concern or retraction was found.
src/known.py: no Crossref updates, no PMC notices, no Europe PMC notices. In the Crossref REST record, update-to and updated-by are absent. The Crossreffilter=updates:query returned 0 items in an earlier check today, and a repeat query was rate-limited (HTTP 429). Europe PMC lists one linked item, a "Comment in" PNAS commentary (PMID 42475589, doi:10.1073/pnas.2617208123). It is not a correction, and only its metadata was available. - PubPeer. The PubPeer search API returned HTTP 403. A web search for the title together with "PubPeer", "correction" or "erratum" found no PubPeer thread and no correction, only news coverage.
Suggested correction
Change the title to give the direction that the paper's results report for water, for example:
The 2025–2030 Dietary Guidelines for Americans are associated with higher land and nitrogen use and greenhouse gas emissions, but lower water use
or, if the title should keep to the adverse outcomes only:
The 2025–2030 Dietary Guidelines for Americans are associated with higher land and nitrogen use, and greenhouse gas emissions
Either wording matches the abstract ("but not in water use (i.e., water use is lower by 7 to 19% than the MAD)"), the Results and the Discussion. No numbers, figures or conclusions in the body need to change.
Recomputation script (python recompute.py.txt)
"""Recompute the direction of each environmental outcome in PMC13273247
(Proc Natl Acad Sci U S A 2026; doi:10.1073/pnas.2604814123) from the paper's own
data, and redraw fig1/fig2 in this file's folder.
Run: python recompute.py.txt (needs numpy and matplotlib)
Inputs are the cached cell values of the "summary" sheet (rows 5-11, columns F-K)
of Dataset S1 (pnas.2604814123.sd01.xlsx), the supporting-information workbook of
the article. The same sheet, with identical values, is in the data file the paper
links under Data Availability (NOVA+PROTEIN_DGA25-30_ver5.xlsx). Units are annual
per-capita totals, as in the paper's Fig. 1 (GHG kg CO2e, water litres, land m2-yr,
nitrogen fertilizer g Nr), plus daily kcal and protein (g).
Quoted text is copied verbatim from the PMC-archived article.
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
METRICS = ["GHG", "Water", "Land", "N fertilizer"]
# Dataset S1, sheet "summary": GHG (F), water (G), land (H), N fertilizer (I), kcal (J), protein (K)
S1 = {
"MAD": [1620.3738784771374, 1884982.1709933979, 16238.053625377528, 48733.303782843548, 2500.3896000000004, 105.88118154726627],
"UPF (in MAD)": [747.61481329437038, 987787.62738541316, 6537.7528699819459, 28078.039550923815, 1636.7655999999999, 50.996616966708991],
"0.8 g/kg": [1253.3556489623356, 1362294.9058782607, 12576.133868282266, 47734.201604786555, 2499.6447058832632, 67.285004777165909],
"1.2 g/kg": [1579.418981096685, 1518270.9459631008, 16646.023128729423, 49705.242645896185, 2499.6957642603857, 100.84518830094257],
"1.6 g/kg": [1969.1244734977586, 1760953.6181536382, 21407.659406569564, 52485.30446813177, 2499.516060583017, 134.26708806627164],
"1.6 g/kg PLANT": [1497.7793450200566, 1415155.5062714668, 16597.693788935609, 43883.534702103032, 2499.6400527792202, 134.37530603776761],
"1.6 g/kg ANIMAL": [1985.3613800760761, 1637370.2722960121, 21199.247774193638, 52751.029784245409, 2500.187619501251, 134.21656241614866],
}
SCEN = ["0.8 g/kg", "1.2 g/kg", "1.6 g/kg", "1.6 g/kg PLANT", "1.6 g/kg ANIMAL"]
# Scenarios the authors' own "Range" cells (summary!K15:L15) pool: 1.2, 1.6 and 1.6 ANIMAL
DGA = ["1.2 g/kg", "1.6 g/kg", "1.6 g/kg ANIMAL"]
# What the paper states
REPORTED_WATER = (-19.0, -7.0) # Abstract: "water use is lower by 7 to 19% than the MAD"
REPORTED_MAX_OTHER = 32.0 # Abstract/Results: "by as much as 32%" (GHG, land, N fertilizer)
REPORTED_UPF_SHARE = (40.0, 58.0) # Results: "(40 to 58%)"
BODY_WEIGHT = 84.0 # Methods: average adult weight, kg
TITLE = ("The 2025–2030 Dietary Guidelines for Americans are associated with higher land, "
"water and nitrogen use, and greenhouse gas emissions")
ABSTRACT_Q = "but not in water use (i.e., water use is lower by 7 to 19% than the MAD)"
RESULTS_Q = "reduced the water footprint compared to MAD by 7 to 19%"
DISCUSSION_Q = "For water use, the net effect is beneficial"
def pc(a, b):
return 100.0 * (a - b) / b
mad = np.array(S1["MAD"])
ch = {k: pc(np.array(S1[k][:4]), mad[:4]) for k in SCEN}
print("=== 1. Percent change versus the Mean American Diet (MAD), Dataset S1 summary sheet ===")
print(f"{'scenario':18s}" + "".join(f"{m:>14s}" for m in METRICS) + f"{'kcal/d':>9s}{'protein g/d':>13s}")
for k in ["MAD"] + SCEN:
v = S1[k]
row = "".join(f"{x:+13.2f}%" for x in ch[k]) if k != "MAD" else "".join(f"{'(baseline)':>14s}" for _ in METRICS)
print(f"{k:18s}{row}{v[4]:9.1f}{v[5]:13.1f}")
print("\n=== 2. Water: the paper's stated range and the title's direction ===")
w_dga = [ch[k][1] for k in DGA]
print("Water change vs MAD, DGA-range scenarios (1.2, 1.6, 1.6 ANIMAL):", [f"{x:+.2f}%" for x in w_dga])
print(f" reductions run from {-max(w_dga):.1f}% to {-min(w_dga):.1f}% "
f"(paper: 'lower by {-REPORTED_WATER[1]:.0f} to {-REPORTED_WATER[0]:.0f}%') -> "
f"matches after rounding: {round(-max(w_dga)) == -REPORTED_WATER[1] and round(-min(w_dga)) == -REPORTED_WATER[0]}")
w_all = {k: ch[k][1] for k in SCEN}
print("Water change vs MAD, every modelled scenario:", {k: f"{v:+.1f}%" for k, v in w_all.items()})
print(" any scenario with HIGHER water use than the MAD?", any(v > 0 for v in w_all.values()))
print(" smallest reduction:", f"{max(w_all.values()):+.1f}%", " largest reduction:", f"{min(w_all.values()):+.1f}%")
print("\n=== 3. GHG, land and N fertilizer (the other three title outcomes) ===")
other = [(k, METRICS[j], ch[k][j]) for k in DGA for j in (0, 2, 3)]
lo = min(other, key=lambda t: t[2]); hi = max(other, key=lambda t: t[2])
print(f" range over 1.2, 1.6, 1.6 ANIMAL: {lo[2]:+.1f}% ({lo[1]}, {lo[0]}) to {hi[2]:+.1f}% ({hi[1]}, {hi[0]})")
print(f" paper: 'as much as {REPORTED_MAX_OTHER:.0f}%' -> max rounds to {round(hi[2])}: {round(hi[2]) == REPORTED_MAX_OTHER}")
for k in DGA:
print(f" {k:16s} GHG {ch[k][0]:+.1f}% land {ch[k][2]:+.1f}% N {ch[k][3]:+.1f}% | water {ch[k][1]:+.1f}%")
print("\n=== 4. Other numbers in the text ===")
share = [100 * S1["UPF (in MAD)"][j] / mad[j] for j in range(4)]
print("UPF share of MAD impacts (GHG, water, land, N):", [f"{x:.1f}%" for x in share],
f" range {min(share):.1f}-{max(share):.1f}% (paper: 40 to 58%)")
print(f"UPF share of MAD calories: {100*S1['UPF (in MAD)'][4]/mad[4]:.1f}% (paper: 'over 50%')")
print("Protein targets 0.8/1.2/1.6 g/kg x 84 kg =", [round(p * BODY_WEIGHT, 1) for p in (0.8, 1.2, 1.6)],
" modelled:", [round(S1[k][5], 1) for k in ("0.8 g/kg", "1.2 g/kg", "1.6 g/kg")])
print(f"MAD protein: {mad[5]:.1f} g/d = {mad[5]/BODY_WEIGHT:.2f} g/kg (paper: 'MAD protein intake level is 1.2 g/kg')")
print("\n=== 5. Alternative comparators examined as explanations for the title ===")
for k in ["1.2 g/kg", "1.6 g/kg", "1.6 g/kg ANIMAL"]:
print(f" water, {k:16s} vs 0.8 g/kg UPF-free diet: {pc(S1[k][1], S1['0.8 g/kg'][1]):+.1f}%")
print(f" water, 1.6 g/kg ANIMAL vs 1.6 g/kg PLANT: {pc(S1['1.6 g/kg ANIMAL'][1], S1['1.6 g/kg PLANT'][1]):+.1f}%")
print(" 1.6 g/kg ANIMAL vs 1.6 g/kg PLANT, (GHG, water, land, N):",
[f"{pc(S1['1.6 g/kg ANIMAL'][j], S1['1.6 g/kg PLANT'][j]):+.1f}%" for j in range(4)])
print(" water per gram of daily protein vs MAD:",
{k: f"{pc(S1[k][1]/S1[k][5], mad[1]/mad[5]):+.1f}%" for k in SCEN})
print(" water per 1,000 kcal vs MAD (all diets ~2,500 kcal):",
{k: f"{pc(S1[k][1]/S1[k][4], mad[1]/mad[4]):+.1f}%" for k in SCEN})
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none", "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# Figure 1: title vs body text (text-vs-text contradiction)
fig = plt.figure(figsize=(7.2, 4.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.9); ax.axis("off")
def panel(x, y, w, h, head, body, edge, lw=2, ls="-", body_color=INK, size=10):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.06",
fc="white", ec=edge, lw=lw, ls=ls))
ax.text(x + 0.12, y + h - 0.12, head, ha="left", va="top", fontsize=10, color=INK2, weight="bold")
ax.text(x + 0.12, y + h - 0.42, body, ha="left", va="top", fontsize=size, color=body_color,
linespacing=1.35)
panel(0.15, 2.55, 2.5, 1.6, "Title",
"\u201c\u2026are associated with\nhigher land, water and\nnitrogen use, and\ngreenhouse gas emissions\u201d",
ORANGE, lw=2.5)
ax.text(0.27, 2.65, "Water: higher", color=ORANGE, fontsize=11, weight="bold", ha="left", va="bottom")
wd = {k: ch[k][1] for k in DGA}
fmt = lambda x: f"{x:+.1f}%".replace("-", "\u2212")
rx, rw = 4.0, 3.05
panel(rx, 3.62, rw, 0.98, "Abstract", "\u201c\u2026but not in water use (i.e., water\nuse is lower by 7 to 19% than the MAD)\u201d",
INK2, lw=1.2, size=9.5)
panel(rx, 2.66, rw, 0.86, "Results, 2nd paragraph", "\u201c\u2026reduced the water footprint\ncompared to MAD by 7 to 19%\u201d",
INK2, lw=1.2, size=9.5)
panel(rx, 1.90, rw, 0.66, "Discussion, 1st paragraph", "\u201cFor water use, the net effect is beneficial\u201d",
INK2, lw=1.2, size=9.5)
panel(rx, 0.55, rw, 1.25, "Recomputed from Dataset S1", "", BLUE, lw=2.5)
for i, k in enumerate(DGA):
yy = 1.38 - 0.19 * i
ax.text(rx + 0.12, yy, k.replace("ANIMAL", "animal"), color=BLUE, fontsize=10, ha="left", va="top")
ax.text(rx + 2.0, yy, fmt(wd[k]), color=BLUE, fontsize=10, ha="right", va="top")
ax.text(rx + 0.12, 0.64, "Water: lower in all scenarios", color=BLUE, fontsize=10, weight="bold",
ha="left", va="bottom")
# bracket grouping the right-hand panels
bx = rx - 0.12
ax.plot([bx, bx], [0.55, 4.60], color=INK2, lw=1.2)
for yy in (0.55, 4.60):
ax.plot([bx, bx + 0.07], [yy, yy], color=INK2, lw=1.2)
ax.add_patch(FancyArrowPatch((2.7, 3.35), (bx - 0.05, 2.6), arrowstyle="<|-|>", mutation_scale=14,
color=INK, lw=2, connectionstyle="arc3,rad=0.0"))
ax.text(3.27, 3.55, "Opposite\ndirection", ha="center", va="bottom", fontsize=10.5, weight="bold", color=INK)
ax.text(3.27, 2.55, "Paper's stated\ncomparator:\nMean American\nDiet (MAD)", ha="center", va="top", fontsize=9.5,
color=INK2, linespacing=1.3)
ax.text(0.15, 4.72, "Freshwater use under the 2025\u20132030 guideline diets, relative to the MAD",
fontsize=11, weight="bold", ha="left", va="center")
ax.text(0.15, 2.2, "Orange: the paper's headline\nclaim (title).\nGray: the paper's own text.\n"
"Blue: recomputed from the\npaper's Dataset S1\n(summary sheet).", fontsize=9.5, color=INK2,
ha="left", va="top", linespacing=1.35)
save(fig, "fig1")
# Figure 2: percent change vs MAD for all four title outcomes
fig, ax = plt.subplots(figsize=(7.2, 4.1))
fig.subplots_adjust(left=0.17, right=0.97, top=0.80, bottom=0.14)
order = ["GHG", "Water", "Land", "N fertilizer"]
ypos = {m: i for i, m in enumerate(reversed(order))}
marks = {"1.2 g/kg": "o", "1.6 g/kg": "s", "1.6 g/kg ANIMAL": "^", "1.6 g/kg PLANT": "v"}
offs = {"1.2 g/kg": 0.12, "1.6 g/kg": 0.04, "1.6 g/kg ANIMAL": -0.04, "1.6 g/kg PLANT": -0.12}
ax.axvline(0, color=INK2, lw=1)
for m in order:
j = METRICS.index(m); y = ypos[m]
vals = [ch[k][j] for k in marks]
ax.plot([min(vals), max(vals)], [y, y], color=BLUE, lw=2, alpha=0.35, zorder=1)
for k, mk in marks.items():
filled = k != "1.6 g/kg PLANT"
ax.plot(ch[k][j], y + offs[k], mk, ms=8.5, mec=BLUE, mfc=BLUE if filled else BG, mew=2, zorder=3)
# reported values
yw = ypos["Water"] + 0.33
ax.plot(REPORTED_WATER, [yw, yw], color=ORANGE, lw=2)
for x in REPORTED_WATER:
ax.plot([x, x], [yw - 0.07, yw + 0.07], color=ORANGE, lw=2)
ax.text(np.mean(REPORTED_WATER), yw + 0.1, "Abstract/Results: “lower by 7 to 19%”", color=INK,
fontsize=9.5, ha="center", va="bottom")
ax.annotate("", xy=(13, yw - 0.02), xytext=(0.6, yw - 0.02),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=13))
ax.text(13.8, yw - 0.02, "Title: “higher … water”", color=INK, fontsize=9.5, ha="left", va="center")
yl = ypos["Land"] + 0.33
ax.plot(REPORTED_MAX_OTHER, yl, "D", ms=8.5, color=ORANGE, zorder=3)
ax.text(REPORTED_MAX_OTHER - 1.2, yl + 0.02, "Text: “as much as 32%”", color=INK, fontsize=9.5,
ha="right", va="center")
ax.set_yticks([ypos[m] for m in order]); ax.set_yticklabels(order, fontsize=10.5, color=INK)
ax.set_ylim(-0.55, 3.62)
ax.set_xlim(-32, 40)
ax.set_xlabel("Change versus the Mean American Diet (%)", fontsize=10.5)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
h = [Line2D([], [], ls="", marker=mk, ms=8, mec=BLUE, mew=2, mfc=BLUE if k != "1.6 g/kg PLANT" else BG,
label=k.replace("ANIMAL", "animal").replace("PLANT", "plant")) for k, mk in marks.items()]
h += [Line2D([], [], color=ORANGE, lw=2, marker="D", ms=7, label="reported in the paper")]
leg = ax.legend(handles=h, ncol=5, loc="lower left", bbox_to_anchor=(-0.2, 1.01), frameon=False,
fontsize=9, handletextpad=0.3, columnspacing=0.9)
fig.text(0.02, 0.965, "Blue: recomputed from Dataset S1. Orange: values and direction stated in the paper.",
fontsize=10, color=INK2, ha="left", va="top")
save(fig, "fig2")
print("\nFigures written:", [os.path.join(HERE, f"fig{i}.{e}") for i in (1, 2) for e in ("svg", "png")])
The abstract names the inhibited RNA strand as negative, but the paper's own data show positive
Proc Natl Acad Sci U S A (2018) · doi:10.1073/pnas.1805195115 · PMC6176606
Summary
The paper studies a dengue virus genome carrying an NS3-G459L mutation that blocks an internal self-cleavage and acts as a trans-dominant inhibitor of wild-type virus in the same cell. The abstract states that this variant "dominantly inhibited negative-strand RNA synthesis." The Results section, using strand-specific RT-qPCR (Fig. 5 F and G), reports the opposite direction: positive-strand synthesis was strongly inhibited, while negative-strand synthesis was allowed (negative-strand RNA was significantly greater than the polymerase-dead control). The Results conclusion and the Discussion both state that positive-strand synthesis is the inhibited step. The single word "negative-strand" in the abstract therefore reverses the direction of the paper's key mechanistic result, and should read "positive-strand."
What the paper reports
an NS3-G459L variant that is incapable of cleavage at the internal NS3 junction dominantly inhibited negative-strand RNA synthesis of wild-type virus present in the same cell
— Abstract
To identify whether the dominant NS3-G459L mutant genome inhibited negative-strand synthesis (and therefore positive-strand synthesis as well) or just positive-strand synthesis, we performed strand-specific RT-qPCR to measure both strands of WT and GDD replicons (Fig. 5 F and G). ... The amount of positive-strand replicon RNA detected when cotransfected with the NS3-G459L dominant mutant was indistinguishable from that detected for the GDD replicon alone, indicating that positive-strand synthesis was strongly inhibited (Fig. 5F). By contrast, the amount of negative-strand replicon RNA was significantly greater than that for the GDD replicon alone (Fig. 5G), indicating that the NS3-G459L dominant mutant allows negative-strand synthesis and therefore specifically inhibits positive-strand synthesis.
— Results, "NS3-G459L Is a Dominant Inhibitor of WT Viral Growth"
Most importantly, viral genomes that abrogate this self-processing event trans-dominantly inhibit positive-strand RNA synthesis of WT genomes present in the same cell
— Discussion, "NS3 Internal Cleavage"
(F) Positive-strand replicon RNA was detected by strand-specific RT-qPCR. (G) Negative-strand replicon RNA was detected by strand-specific RT-qPCR (n = 2).
— Legend to Fig. 5
What the paper's own numbers show
There is no arithmetic to redo. The discrepancy is a reversed qualitative direction between two parts of the same paper describing one experiment: a strand-specific RT-qPCR of the luciferase replicon cotransfected with the NS3-G459L variant, measured at a single time point and normalized to the wild-type replicon alone, with the polymerase-dead GDD replicon as the no-synthesis reference. The two panels of that one experiment (Fig. 5F, positive strand; Fig. 5G, negative strand) are each reported relative to the GDD control. Taking those two relations as stated fixes which strand is inhibited.
| quantity | location | reported | recomputed (from the paper's own numbers) |
|---|---|---|---|
| Strand whose synthesis G459L inhibits | Abstract | negative-strand | positive-strand |
| Positive-strand replicon RNA, G459L vs GDD-dead | Results / Fig. 5F | = GDD → positive synthesis strongly inhibited | positive-strand inhibited (consistent with body) |
| Negative-strand replicon RNA, G459L vs GDD-dead (n = 2) | Results / Fig. 5G | > GDD → negative synthesis allowed | negative-strand allowed, not inhibited |
| Strand inhibited (stated) | Results conclusion | positive-strand | positive-strand |
| Strand inhibited (stated) | Discussion | positive-strand | positive-strand |
Four statements in the body (Fig. 5F, Fig. 5G, the Results conclusion, and the Discussion) place the inhibition on the positive strand; one statement in the abstract places it on the negative strand. recompute.py.txt encodes the two reported relations and prints VERDICT: claim CONFIRMED.
Figure
The abstract (left, orange, as reported) states that negative-strand synthesis is inhibited. The Results and Discussion (right, blue, from the paper's own numbers) state that positive-strand synthesis is inhibited while negative-strand synthesis is allowed, based on the strand-specific RT-qPCR in Fig. 5F/G of the same cotransfection experiment. The connector names the conflict: the two descriptions assign the inhibition to opposite strands of one assay. Colors are reinforced by the panel labels and legend so the distinction does not rely on color alone.
Why it matters
Which RNA strand's synthesis is blocked is the mechanistic heart of this result. A reader using the abstract alone would conclude that the dominant-negative genome blocks negative-strand (template) synthesis, an early step of replication. The paper's data support the reverse: negative-strand synthesis proceeds and positive-strand (progeny genome) synthesis is specifically inhibited. Flavivirus and antiviral-mechanism researchers who cite the abstract without reading the Results could adopt an inverted picture of where in the replication cycle the trans-dominant block acts. The paper's broader conclusion — that abrogating the internal NS3 self-cleavage yields trans-dominant inhibition of viral replication, supporting the proposed antiviral strategy — is unaffected in direction, magnitude, or significance, because it does not depend on which strand is affected. The error is confined to one word of one abstract sentence and is contradicted correctly elsewhere in the same paper.
Explanations considered
- A different experiment or population in the abstract vs. the body. No. Both refer to the same strand-specific RT-qPCR of the replicon cotransfected with NS3-G459L; the abstract summarizes exactly the experiment reported in Fig. 5 F and G.
- A different time point, normalization, or unit. No. It is one assay (single time point, normalized to the wild-type replicon alone); Fig. 5F and 5G are simply the positive- and negative-strand measurements of that one experiment.
- Loose reading: inhibiting positive-strand synthesis indirectly lowers negative-strand RNA, so "inhibited negative" is approximately true. No. Fig. 5G reports negative-strand RNA as significantly greater than the polymerase-dead GDD control, and the text explicitly calls negative-strand synthesis "allowed," not reduced.
- A logical-framing rescue in which "inhibited negative" is a defensible summary. No. The Results pose an explicit dichotomy — the mutant either "inhibited negative-strand synthesis (and therefore positive-strand synthesis as well) or just positive-strand synthesis" — and answer "just positive-strand." The abstract asserts the alternative the experiment rejected.
- A one-sided test, an adjusted-vs-unadjusted value, or a stated convention. Not applicable. This is a reversed qualitative direction, not a statistical-threshold or reporting-convention issue.
- A published correction or erratum that resolves it. None.
known.pyreturns empty Crossref, PMC, and EuropePMC notices, and no PubPeer thread was found.
Status
- Version of record checked. The error is present in the version of record, not an archiving artifact. The abstract sentence reading "negative-strand RNA synthesis" appears identically in the Europe PMC publisher-deposited full-text XML and in the PMC HTML; the PNAS publisher page could not be read directly (HTTP 403), but a third-party citation page independently quotes the abstract with "negative-strand RNA synthesis." The Results and Discussion text quoted above comes from the PMC archive via
src/show.py. - Supplements. The SI Appendix (primer sequences for the strand-specific RT-qPCR and supporting figures) does not bear on the discrepancy, which is a single word choice in the abstract rather than a numeric or table value. The main-text Results, the Fig. 5F/G legend, and the Discussion already give the correct, consistent direction.
- Corrections and PubPeer. No correction, erratum, or retraction notice found via Crossref, PMC, or EuropePMC (
known.py); no PubPeer thread located by search. - Sources. Abstract, Results, Discussion, and Fig. 5 legend as archived in PubMed Central (PMC6176606); Crossref/PMC/EuropePMC correction records via
known.py. As of 2026-10-23.
Suggested correction
In the abstract, change "dominantly inhibited negative-strand RNA synthesis of wild-type virus present in the same cell" to "dominantly inhibited positive-strand RNA synthesis of wild-type virus present in the same cell," making the abstract agree with the Results (Fig. 5 F and G), the Results conclusion, and the Discussion. No figure, table, or numeric value needs to change.
Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
"""
Recompute / re-derivation for the documented error in PMC6176606
(Proc Natl Acad Sci U S A 2018, doi:10.1073/pnas.1805195115).
The error is a reversed qualitative direction, not an arithmetic slip:
the Abstract names one RNA strand (negative) as the one whose synthesis the
NS3-G459L trans-dominant variant inhibits, while the Results (strand-specific
RT-qPCR, Fig. 5 F and G) and the Discussion both report the opposite --
positive-strand synthesis is inhibited and negative-strand synthesis is allowed.
Running this file:
python recompute.py.txt
prints every value used in the write-up and regenerates fig1.svg / fig1.png
in the folder that holds this script.
No data are taken from outside the paper. The RT-qPCR panels (Fig. 5F, 5G) are
reported in the paper only qualitatively relative to the polymerase-dead GDD
control ("indistinguishable from" / "significantly greater than"); those two
relations are the inputs, encoded verbatim below.
"""
import os
# ----------------------------------------------------------------------
# Inputs: the paper's own statements (strand-specific RT-qPCR, one assay,
# 26 h, normalized to the WT replicon alone). GDD = polymerase-dead control.
# ----------------------------------------------------------------------
statements = {
"abstract": {
"text": ("an NS3-G459L variant that is incapable of cleavage at the "
"internal NS3 junction dominantly inhibited NEGATIVE-strand "
"RNA synthesis of wild-type virus present in the same cell"),
"inhibited_strand": "negative",
},
"results_Fig5F_positive": {
"observation": "positive-strand RNA (G459L) == GDD-dead control",
"meaning": "positive-strand synthesis strongly INHIBITED",
"inhibited": "positive",
},
"results_Fig5G_negative": {
"observation": "negative-strand RNA (G459L) > GDD-dead control",
"meaning": "negative-strand synthesis ALLOWED (not inhibited)",
"allowed": "negative",
"n": 2,
},
"results_conclusion": {
"text": ("the NS3-G459L dominant mutant allows negative-strand "
"synthesis and therefore specifically inhibits "
"positive-strand synthesis"),
"inhibited_strand": "positive",
"allowed_strand": "negative",
},
"discussion": {
"text": ("viral genomes that abrogate this self-processing event "
"trans-dominantly inhibit positive-strand RNA synthesis of "
"WT genomes present in the same cell"),
"inhibited_strand": "positive",
},
}
# ----------------------------------------------------------------------
# Re-derivation
# ----------------------------------------------------------------------
body_inhibited = "positive" # Fig. 5F + Results conclusion + Discussion
body_allowed = "negative" # Fig. 5G + Results conclusion
abstract_inhibited = statements["abstract"]["inhibited_strand"] # "negative"
contradiction = (abstract_inhibited != body_inhibited)
reversed_direction = (abstract_inhibited == body_allowed)
def main():
print("Source: PMC6176606, doi:10.1073/pnas.1805195115")
print("Error type: abstract-vs-body contradiction (reversed strand direction)")
print("-" * 68)
print("Fig. 5F positive strand : G459L == GDD(dead) -> positive synthesis INHIBITED")
print("Fig. 5G negative strand : G459L > GDD(dead) -> negative synthesis ALLOWED (n=2)")
print()
print(f"BODY: strand inhibited = {body_inhibited!r}")
print(f"BODY: strand allowed = {body_allowed!r}")
print(f"ABSTRACT: strand stated inhibited = {abstract_inhibited!r}")
print()
print(f"Abstract contradicts body? {contradiction}")
print(f"Direction reversed (abstract names ALLOWED strand as {reversed_direction}")
print(f" the inhibited one)?")
print("-" * 68)
# Table mirrored in the write-up
rows = [
("Strand G459L inhibits", "Abstract",
"negative-strand", "positive-strand"),
("Positive-strand RNA, G459L vs GDD-dead", "Results / Fig. 5F",
"= GDD => positive INHIBITED", "positive INHIBITED (consistent)"),
("Negative-strand RNA, G459L vs GDD-dead (n=2)", "Results / Fig. 5G",
"> GDD => negative ALLOWED", "negative ALLOWED, not inhibited"),
("Strand inhibited (stated)", "Results conclusion",
"positive-strand", "positive-strand"),
("Strand inhibited (stated)", "Discussion",
"positive-strand", "positive-strand"),
]
w = [44, 20, 30, 32]
hdr = ("quantity", "location", "reported", "recomputed")
print(" | ".join(h.ljust(wi) for h, wi in zip(hdr, w)))
print("-+-".join("-" * wi for wi in w))
for r in rows:
print(" | ".join(str(c).ljust(wi) for c, wi in zip(r, w)))
print("-" * 68)
# Innocent explanations checked
checks = {
"Different experiment/population?":
"No - same strand-specific RT-qPCR, same G459L cotransfection.",
"Different time/normalization/units?":
"No - one assay (26 h, normalized to WT-alone); 5F & 5G are the two strands.",
"Inhibiting + strand loosely lowers - strand?":
"No - 5G shows negative SIGNIFICANTLY GREATER than GDD; text says ALLOWED.",
"Logical framing rescues abstract?":
"No - Results pose 'inhibit negative (hence both) OR just positive' and answer 'just positive'.",
"One-sided/adjusted test / stated convention?":
"N/A - reversed qualitative direction, not a statistical threshold.",
"Correction/erratum?":
"No Crossref/PMC/EuropePMC notice; no PubPeer thread.",
}
for q, a in checks.items():
print(f"- {q} {a}")
print("-" * 68)
verdict = "CONFIRMED" if (contradiction and reversed_direction) else "NOT CONFIRMED"
print(f"VERDICT: claim {verdict}")
make_figure()
# ----------------------------------------------------------------------
# Figure: two text panels (Abstract vs Results/Discussion) joined by an
# annotated connector naming the conflict.
# ----------------------------------------------------------------------
def make_figure():
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
BG = "#fcfcfb"
INK = "#0b0b0b"
SUB = "#52514e"
ORANGE = "#eb6834" # the paper's reported value (abstract)
BLUE = "#2a78d6" # value recomputed from the paper's own numbers
GRID = "#d8d7d4"
here = os.path.dirname(os.path.abspath(__file__))
fig, ax = plt.subplots(figsize=(7.2, 4.5))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
# Title
ax.text(50, 96, "Abstract names the allowed strand as the inhibited one",
ha="center", va="top", fontsize=13, fontweight="bold", color=INK)
ax.text(50, 90.5,
"PMC6176606 - strand-specific RT-qPCR (Fig. 5 F and G)",
ha="center", va="top", fontsize=10, color=SUB)
# ---- Left panel: Abstract (reported, orange) ----
lx, lw = 2.5, 38
ly, lh = 28, 50
ax.add_patch(FancyBboxPatch((lx, ly), lw, lh,
boxstyle="round,pad=0.6,rounding_size=2.2",
linewidth=2, edgecolor=ORANGE, facecolor="#fdece4"))
ax.text(lx + lw / 2, ly + lh - 4, "ABSTRACT", ha="center", va="top",
fontsize=11, fontweight="bold", color=ORANGE)
ax.text(lx + lw / 2, ly + lh - 11,
"\"...dominantly inhibited\n"
"NEGATIVE-strand RNA\n"
"synthesis of wild-type virus\n"
"present in the same cell\"",
ha="center", va="top", fontsize=9.5, color=INK, style="italic")
ax.text(lx + lw / 2, ly + 8,
"Reads: NEGATIVE-strand\nsynthesis is inhibited",
ha="center", va="center", fontsize=10, color=ORANGE,
fontweight="bold")
# ---- Right panel: Results / Discussion (recomputed, blue) ----
rx, rw = 59.5, 38
ax.add_patch(FancyBboxPatch((rx, ly), rw, lh,
boxstyle="round,pad=0.6,rounding_size=2.2",
linewidth=2, edgecolor=BLUE, facecolor="#e7f0fb"))
ax.text(rx + rw / 2, ly + lh - 4, "RESULTS / DISCUSSION",
ha="center", va="top", fontsize=11, fontweight="bold", color=BLUE)
ax.text(rx + rw / 2, ly + lh - 11,
"Fig. 5F positive = GDD-dead\n"
" -> positive INHIBITED\n"
"Fig. 5G negative > GDD-dead (n=2)\n"
" -> negative ALLOWED",
ha="center", va="top", fontsize=9, color=INK)
ax.text(rx + rw / 2, ly + 8,
"States: POSITIVE-strand\nsynthesis is inhibited",
ha="center", va="center", fontsize=10, color=BLUE,
fontweight="bold")
# ---- Connector naming the conflict (in the central gap only) ----
gap_l, gap_r = lx + lw, rx # 40.5 -> 59.5
cy = ly + lh / 2
con = FancyArrowPatch((gap_l + 0.5, cy), (gap_r - 0.5, cy),
arrowstyle="<|-|>", mutation_scale=14,
linewidth=2, color=INK,
connectionstyle="arc3,rad=0.0")
ax.add_patch(con)
mid = (gap_l + gap_r) / 2
ax.text(mid, cy + 6, "DIRECTION", ha="center", va="bottom",
fontsize=9.5, fontweight="bold", color=INK)
ax.text(mid, cy + 2.2, "REVERSED", ha="center", va="bottom",
fontsize=9.5, fontweight="bold", color=INK)
ax.text(mid, cy - 4, "same assay,", ha="center", va="top",
fontsize=8.5, color=SUB)
ax.text(mid, cy - 7.5, "opposite strand", ha="center", va="top",
fontsize=8.5, color=SUB)
# ---- Footer legend (color never the sole signal) ----
ax.plot([8, 12], [12, 12], color=ORANGE, linewidth=2)
ax.text(13, 12, "as reported in the paper (abstract)",
ha="left", va="center", fontsize=9.5, color=SUB)
ax.plot([8, 12], [6, 6], color=BLUE, linewidth=2)
ax.text(13, 6, "from the paper's own numbers (Results / Discussion)",
ha="left", va="center", fontsize=9.5, color=SUB)
# faint recessive grid baseline (single horizontal)
for gy in (22,):
ax.axhline(gy, color=GRID, linewidth=0.8, zorder=0)
fig.tight_layout()
svg = os.path.join(here, "fig1.svg")
png = os.path.join(here, "fig1.png")
fig.savefig(svg, facecolor=BG)
fig.savefig(png, dpi=200, facecolor=BG)
plt.close(fig)
print(f"wrote {svg}")
print(f"wrote {png}")
if __name__ == "__main__":
main()
Significance statement says tracer chromosome goes "exclusively" with X; Results report 28 of 42 cells
Proc Natl Acad Sci U S A (2018) · doi:10.1073/pnas.1809583115 · PMC6310853
Summary
The paper's Significance statement says that male Ts65Dn translocation mice carrying a maternally derived supernumerary chromosome place this "tracer" chromosome "exclusively" in the haploid set that contains the X chromosome. The only experiment behind that sentence is in Results (Fig. 4 H–L). It reports the tracer in the X-containing hemisphere in "the majority of cases": n = 28/42 cells, P = 0.022. That is 66.7% (exact 95% CI 50.5% to 80.4%), with 14 of 42 cells (33.3%) not counted on the X side. The reported P reproduces as a one-sided binomial test against 0.5 (0.0218; two-sided 0.0436). The paper's own Fig. 4L also shows some tracer markers below the chromosome 16 midpoint, which the legend defines as the Y-containing hemisphere. The Significance statement thus describes as all-or-none a modest majority that only just reaches significance, in the experiment that the Results present as the direct test of the parental-origin model. The paper's main finding, that homologs are kept apart in mitosis, rests on other experiments and is not affected.
What the paper reports
Male translocation mice with a maternal-derived supernumerary chromosome display the tracer chromosome exclusively to the haploid set containing the X chromosome.
— Abstract, Significance statement
Using 3D chromosome imaging of a translocation mouse with a supernumerary chromosome, we found that this maternally derived chromosome is positioned by parental origin. These data, taken together, support the identity of haploid sets by parental origin.
— Abstract (main)
In the majority of cases we examined, the hemisphere containing the translocated chromosome 1716 also contained the maternally derived X chromosome (n = 28/42 cells, P = 0.022; Fig. 4 H–L and Movie S7).
— Results › In Vitro and in Vivo Segregation of Mouse Chromosomes, paragraph 2 (the plain-text archive renders 1716 as "1716")
To identify which X-, or Y-containing haploid set was most likely associated with the translocation chromosome, we first determined the approximate boundary between the haploid sets by finding the midpoint of the pair of homologous chromosome 16. We then mapped the position of the translocated chromosome (Fig. 4L).
— Same paragraph, immediately before
A maternally derived supernumerary chromosome segregates with the X chromosome.
— Fig. 4 legend, title sentence
The X- or Y-containing hemisphere domains are defined as being above/below the plus sign (+), respectively.
— Fig. 4 legend, panel L (n = 42)
Movie S7. The translocated chromosome in the Ts65Dn MEFs was consistently detected in the same hemisphere as the maternal-derived X chromosome.
— SI Appendix, movie legends (the Results cite Movie S7 alongside n = 28/42)
Center of mass calculations were used for hemisphere localization; however, chromosomes that were positioned with the majority of their staining along the boundary of the hemispheres, due to the limits of resolution, were scored to support the null hypothesis, which goes against our model, in the calculation of the one-tailed P value which gives rise to an underestimation of statistical significance.
— Materials and Methods › Statistical Analysis
Although our data support the paradigm (13, 14, 47) that haploid sets are derived by parental origin, there are limitations to our analysis.
— Discussion, paragraph 3
The same paragraph adds that in the reciprocal (paternal) cross the tracer "localized to the boundary between two nuclear hemispheres, making it difficult to determine its association to either X- or Y-containing hemispheres (SI Appendix, Fig. S5 C–I)."
What the paper's own numbers show
Method. The Methods state that hemisphere assignments were tested with a binomial distribution against p = 0.5, using a one-tailed P value. I recomputed every count that the paper reports with a P value, as one-sided P(X ≥ x | n, 0.5), as two-sided exact P, and as the single-term binomial probability b(x; n, 0.5) that the Methods formula writes out. For the tracer result I computed the exact (Clopper–Pearson) 95% CI for the proportion. As a separate, approximate check, I read the positions of the tracer circles and the chromosome 16 midpoint markers ("+") off the published Fig. 4 image (757 × 449 px) by pixel colour. Each circle was classed as above, within ±3 axis units of, or below its local midpoint. All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Tracer 1716 in the maternal-X hemisphere | Significance statement | "exclusively", i.e. 42/42 (100%) | — |
| Same result | Results, mouse section para 2; Fig. 4 H–L | n = 28/42 cells, P = 0.022, "the majority of cases" | 66.7%; exact 95% CI 50.5% to 80.4% (excludes 100%); one-sided P = 0.0218, two-sided P = 0.0436 |
| Cells not counted in the X hemisphere | Same | — | 14/42 (33.3%) |
| Margin over the significance threshold | Same | — | smallest count out of 42 with one-sided P < 0.05 is 27; the reported 28 is one cell above it |
| Compatibility with a near-exclusive effect | Same | — | if the true share were 90%, P(≤ 28 of 42) = 3.5 × 10⁻⁵; if 95%, 8.5 × 10⁻⁹ |
| Tracer circles vs midpoint, read from the image (approximate) | Fig. 4L (n = 42) | no count given in the legend | 41 of 42 circles resolved: 35 above (X side), 2 within ±3 units, 4 below (Y side), all 4 among the nuclei at the right of the plot |
| XY one per hemisphere, WT male MEFs | Results, mouse section para 1; Fig. 4 E, F | 28/42, P = 0.022 | one-sided P = 0.0218 (match) |
| Chr 16 homologs in separate hemispheres, Ts65Dn MEFs | Results, mouse section para 2; SI Fig. S5 A, B | 28/42, P = 0.022 | one-sided P = 0.0218 (match) |
| Chr 17 homologs, WT MEF anaphase | Results, mouse section para 1 | 28/36, P = 0.001 | one-sided 0.0006 (match) |
| Chr 4, HUVEC metaphase / anaphase | Results, Homologous and XY section para 1 | 15/20, P = 0.021; 28/38, P = 0.003 | 0.0207; 0.0025 (match) |
| XX, female HUVEC metaphase / anaphase | Same section, para 2 | 18/24, P = 0.011; 32/42, P = 0.001 | 0.0113; 0.0005 (match) |
| XY, male HUVEC metaphase | Same section, para 2 | 19/28, P = 0.044 | 0.0436 (match) |
| XY, male HUVEC anaphase (side finding) | Same section, para 2; Fig. 2 D–F | 20/28, P = 0.001 | one-sided 0.0178, two-sided 0.0357, b(x) 0.0116; none gives 0.001 (that would need about 23/28) |
Two things follow. First, every reported P except one reproduces as a one-sided binomial tail probability, so the 28/42 count is the paper's own figure for this result, not a typo. Second, a count of 28/42 is a majority, not an exclusive pattern, and the CI rules out anything near 100%.
The image read-out is resolution-limited and is not meant as a recount. It suggests more circles above the midpoint than the 28 counted in the text. One possible reason is the Methods' rule that boundary chromosomes were scored against the model. I do not treat this difference as a second error. The read-out matters here only because it shows tracer markers on the Y side of the boundary in the paper's own figure.
Figure
Figure 1. The same experiment described in two places. Left: the Significance statement, whose word "exclusively" implies the tracer is in the X-containing set in all cells (orange, the paper's wording). Right: the Results sentence that reports the experiment, n = 28/42 cells, which is 66.7% (blue, recomputed from the paper's own count). The connector marks that both sentences describe the same 42 cells. Below: the 14 cells not counted on the X side, and the exact 95% CI, which excludes 100%.
Figure 2. Share of the 42 male Ts65Dn MEF cells with the maternal-origin tracer chromosome in the X-containing hemisphere, on one 0–100% axis. Orange: the share implied by "exclusively" in the Significance statement (100%). Blue: the share reported in Results, 28/42 = 66.7%, with its exact 95% CI (50.5% to 80.4%), computed in recompute.py.txt. The dashed line marks the chance value of 50% used as the null in the paper's binomial test. The lower CI limit lies just above that line, and the upper limit lies well below 100%.
Why it matters
- It is the summary sentence for the paper's only direct test of parental origin. The Results open this experiment with "To directly test whether each haploid set is derived by parental origin". The main abstract then draws the conclusion that the data "support the identity of haploid sets by parental origin". The Significance statement is the version most readers see, and it gives an all-or-none result.
- The size of the effect is overstated. The data show a tracer on the X side in two of three cells. The evidence against chance is modest: one-sided P = 0.022, two-sided P = 0.044, and one cell fewer on the X side (27/42, P = 0.044) would still be significant, but 26/42 (P = 0.082) would not. The CI lower limit (50.5%) sits just above the null.
- The figure title and the movie legend repeat the categorical form. The Fig. 4 legend title ("segregates with the X chromosome") gives no qualifier, and the SI legend for Movie S7 says "consistently detected in the same hemisphere as the maternal-derived X chromosome". Meanwhile the panel the Fig. 4 title describes (Fig. 4L) shows tracer markers below the midpoint.
- The supporting evidence is otherwise limited. The authors' Discussion notes that the reciprocal paternal cross could not assign the tracer to either hemisphere. The Results qualify the interpretation ("If the 1716 chromosome behaves similarly to other wild-type autosomes"). Neither qualification reaches the Significance statement.
- Repeated count. The identical "n = 28/42 cells, P = 0.022" is given for three different measurements in the mouse section: XY separation in WT MEFs, chromosome 16 homolog separation in Ts65Dn MEFs, and tracer-with-X co-localization. The n = 42 given for the XY and tracer measurements matches the Fig. 4 legend, and identical counts can occur by chance. The per-cell data are not in the paper or the SI Appendix, so the three counts cannot be checked independently.
- What does not change. The direction of the tracer result (more often with the X than chance), the antipairing of homologs, and the haploid-set separation in human and mouse cells all rest on the other counts above, which reproduce. One exception: the XY HUVEC anaphase P (reported 0.001, recomputed 0.018 one-sided) is misreported but is still below 0.05.
Explanations considered
- "Exclusively" means the tracer is in one set rather than split across both. The sentence names the set ("the haploid set containing the X chromosome"), so it is a claim about which set, and the Results give that as 28/42.
- Loose wording in a lay summary. The main abstract avoids the word, saying only that the chromosome "is positioned by parental origin". But the Significance statement is part of the published abstract, and the word has a clear meaning that the reported count does not support. Whatever the reason for the wording, it does not change what the numbers show.
- The 14 remaining cells may include boundary cells rather than Y-side cells. The Methods say that boundary chromosomes were "scored to support the null hypothesis". So some of the 14 may be ambiguous rather than clearly on the Y side. This could explain why the image read-out shows more than 28 circles above the midpoint. It does not rescue "exclusively": the paper's own count is 28/42, and Fig. 4L, read by the legend's definition, has about 4 tracer markers below the midpoint, that is, on the Y side.
- A different population or experiment behind the Significance sentence. The Significance statement refers to "translocation mice with a maternal-derived supernumerary chromosome" (male). That is the Ts65Dn maternal × WT paternal cross in Fig. 4 G–L (n = 42). The in vivo embryo panels (Fig. 4 M–P) are painted for chromosomes 16 and 17 and report no tracer-with-X count. The Results cite Movie S7 for the same n = 28/42. No other tracer count appears in the paper.
- A conversion or typesetting artefact. The wording is the same in the publisher's XML archived in PMC, in Europe PMC's full text and on the live PMC page. The companion "PNAS Plus Significance Statements" record (PMC6310825) repeats the clause with "exclusively". The disputed numbers (28/42, 0.022) are plain text, with no superscripts or table cells involved. The only conversion artefact is the chromosome label 1716, which the plain-text archive renders as "1716".
- A typo in the count. The count 28/42 and its P value are mutually consistent (one-sided 0.0218), and the text describes it as "the majority of cases", so the body is internally coherent. The discrepancy lies between the body and the Significance statement.
None of these accounts for the gap between "exclusively" and 28 of 42 cells.
Status
As of 2026-10-23:
- Version of record. The publisher pages (pnas.org, reached through the DOI redirect, and the PDF and SI URLs) returned HTTP 403 to automated access. I therefore checked the publisher-deposited text: the JATS XML archived in PubMed Central (PMC6310853, read with
src/show.py), the Europe PMC full-text XML (title and DOI checked; 115(52):E12235–E12244, PMID 30530674) and the live PMC article page. Every quote from the article above was checked verbatim against the PMC-archived text, and all three sources carry the same wording and numbers. The companion "PNAS Plus Significance Statements" record (PMC6310825) was read on the live PMC page; its Europe PMC full text returned HTTP 500. - Supplements. The SI Appendix (pnas.1809583115.sapp.pdf, from the Europe PMC supplementary-files archive; its md5, 629446949a72863dfbeb0748dcfc62b9, matches the PMC listing) was read in full as text. It contains Figs. S1–S5 and legends for Movies S1–S7, but no per-cell table and no separate count for the tracer-with-X assignment. Fig. S5 A–B covers the maternal cross (n = 42) and Fig. S5 C–I the reciprocal paternal cross (n = 34). The Movie S7 legend is quoted above. The movies were not viewed; Movie S7 was not in the archive.
- Figure image. Fig. 4 was read from the published image file (link below) for the approximate Fig. 4L read-out. No part of it is reproduced here.
- Corrections and notices.
src/known.pyreturned no Crossref update relations and no PMC or Europe PMC notices. A Europe PMC search for the DOI returns only the article and the companion record. - PubPeer and the wider web. The PubPeer API and search page both returned HTTP 403, so PubPeer could not be queried directly. Web searches made earlier in this verification, for the title together with correction, erratum or PubPeer, and for PubPeer together with the DOI, found no correction and no PubPeer thread. They could not be repeated for this write-up because the session's web-search quota was used up. A PubPeer thread therefore cannot be fully ruled out.
- Sources. https://pmc.ncbi.nlm.nih.gov/articles/PMC6310853/ ; https://pmc.ncbi.nlm.nih.gov/articles/PMC6310825/ ; Europe PMC REST (search, full text and supplementary files for PMC6310853); https://api.crossref.org/works/10.1073/pnas.1809583115 ; published Fig. 4 image, https://cdn.ncbi.nlm.nih.gov/pmc/blobs/c3f5/6310853/c1c7e28b198f/pnas.1809583115fig04.jpg .
Suggested correction
In the Significance statement, replace "display the tracer chromosome exclusively to the haploid set containing the X chromosome" with wording that matches the Results, for example: "display the tracer chromosome more often in the haploid set containing the X chromosome (28 of 42 cells, 67%; one-sided binomial P = 0.022)". Correspondingly, the Fig. 4 legend title could read "A maternally derived supernumerary chromosome segregates preferentially with the X chromosome", and "consistently" in the Movie S7 legend could be replaced by "in most cells (28/42)". The authors may also wish to:
- state the boundary-scoring rule next to the 28/42 count, and give the numbers of cells on the X side, on the Y side and on the boundary in Fig. 4L;
- confirm the three separate "n = 28/42 cells, P = 0.022" counts in the mouse section against the source data; and
- correct the XY HUVEC anaphase P value (20/28 gives one-sided P = 0.018, not 0.001).
Recomputation script (python recompute.py.txt)
"""Recompute the numbers behind the parental-origin (Ts65Dn tracer) result in PMC6310853
(Proc Natl Acad Sci U S A 2018;115(52):E12235-E12244, doi:10.1073/pnas.1809583115),
and regenerate fig1/fig2 in this folder.
Run: python recompute.py.txt [path/to/pnas.1809583115fig04.jpg]
Needs numpy, scipy, matplotlib; Pillow only for the optional Fig. 4L read-out (part 5).
If no image path is given, part 5 downloads the published Fig. 4 image from the PMC CDN
into a temporary directory that is deleted afterwards; if that fails, part 5 is skipped and
the figures are still produced. The image is only read, never copied into this folder.
"""
import os
import shutil
import sys
import tempfile
import urllib.request
from pathlib import Path
import numpy as np
from scipy.stats import binom, binomtest
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = Path(__file__).resolve().parent
# ------------------------------------------------------------------ inputs (paper)
# (label, x, n, reported P, location)
ROWS = [
("Chr 4, HUVEC metaphase", 15, 20, "0.021", "Results, Homologous and XY ..., para 1"),
("Chr 4, HUVEC anaphase", 28, 38, "0.003", "Results, Homologous and XY ..., para 1"),
("XX, female HUVEC metaphase", 18, 24, "0.011", "Results, Homologous and XY ..., para 2"),
("XX, female HUVEC anaphase", 32, 42, "0.001", "Results, Homologous and XY ..., para 2"),
("XY, male HUVEC metaphase", 19, 28, "0.044", "Results, Homologous and XY ..., para 2"),
("XY, male HUVEC anaphase", 20, 28, "0.001", "Results, Homologous and XY ..., para 2"),
("Chr 17, WT MEF anaphase", 28, 36, "0.001", "Results, Mouse section, para 1"),
("XY, WT male MEF", 28, 42, "0.022", "Results, Mouse section, para 1"),
("Chr 16 homologs apart, Ts65Dn MEF", 28, 42, "0.022", "Results, Mouse section, para 2"),
("Tracer 17(16) with maternal X, Ts65Dn MEF", 28, 42, "0.022", "Results, Mouse section, para 2"),
]
X_KEY, N_KEY = 28, 42
print("=" * 78)
print("1. Binomial tests against p = 0.5 for every count reported with a P value")
print(" Methods: binomial null, 'one-tailed P value'.")
print(f" {'comparison':42s} {'x/n':>7s} {'%':>6s} {'P1(>=x)':>9s} {'P2':>8s} {'b(x)':>8s} {'reported':>8s}")
results = {}
for lab, x, n, rep, loc in ROWS:
p1 = binom.sf(x - 1, n, 0.5)
p2 = binomtest(x, n, 0.5).pvalue
pmf = binom.pmf(x, n, 0.5)
results[lab] = (p1, p2, pmf)
print(f" {lab:42s} {x:>3d}/{n:<3d} {100*x/n:6.1f} {p1:9.4f} {p2:8.4f} {pmf:8.4f} {rep:>8s}")
print("\n2. The parental-origin result (tracer chromosome 17(16) in the maternal-X hemisphere)")
frac = X_KEY / N_KEY
ci = binomtest(X_KEY, N_KEY, 0.5).proportion_ci(confidence_level=0.95, method="exact")
p1, p2, _ = results["Tracer 17(16) with maternal X, Ts65Dn MEF"]
print(f" reported: n = {X_KEY}/{N_KEY} cells, P = 0.022")
print(f" fraction in X hemisphere = {X_KEY}/{N_KEY} = {frac:.4f} ({100*frac:.1f}%)")
print(f" cells not counted as X hemisphere = {N_KEY - X_KEY}/{N_KEY} ({100*(N_KEY - X_KEY)/N_KEY:.1f}%)")
print(f" exact (Clopper-Pearson) 95% CI for the fraction = {ci.low:.3f} to {ci.high:.3f} "
f"({100*ci.low:.1f}% to {100*ci.high:.1f}%)")
print(f" one-sided binomial P(X >= {X_KEY} | n = {N_KEY}, p = 0.5) = {p1:.4f}; two-sided = {p2:.4f}")
print(f" 'exclusively' corresponds to {N_KEY}/{N_KEY} = 100%; 100% lies outside the CI "
f"(upper limit {100*ci.high:.1f}%)")
for ptrue in (0.90, 0.95):
print(f" if the true fraction were {ptrue:.2f}: P(X <= {X_KEY} | n = {N_KEY}) = {binom.cdf(X_KEY, N_KEY, ptrue):.1e}")
xmin = int(min(k for k in range(N_KEY + 1) if binom.sf(k - 1, N_KEY, 0.5) < 0.05))
print(f" smallest count out of {N_KEY} with one-sided P < 0.05: {xmin} "
f"(P = {binom.sf(xmin - 1, N_KEY, 0.5):.4f}); the reported 28 is {X_KEY - xmin} above it")
print(f" one-sided P for 26/42 = {binom.sf(25, N_KEY, 0.5):.4f} (not < 0.05)")
print("\n3. Identical count reported for three different measurements")
for lab in ("XY, WT male MEF", "Chr 16 homologs apart, Ts65Dn MEF", "Tracer 17(16) with maternal X, Ts65Dn MEF"):
print(f" {lab:42s} 28/42, P = 0.022 reported; one-sided P recomputed {results[lab][0]:.4f}")
print("\n4. Side finding: XY male HUVEC anaphase")
p1, p2, pmf = results["XY, male HUVEC anaphase"]
print(f" reported 20/28, P = 0.001; recomputed one-sided {p1:.4f}, two-sided {p2:.4f}, pmf {pmf:.4f}")
for x in range(20, 29):
if binom.sf(x - 1, 28, 0.5) < 0.0015:
print(f" smallest x out of 28 whose one-sided P rounds to <= 0.001: {x} (P = {binom.sf(x-1, 28, 0.5):.4f})")
break
# ------------------------------------------------------------------ part 5: Fig. 4L read-out
FIG_URL = ("https://cdn.ncbi.nlm.nih.gov/pmc/blobs/c3f5/6310853/c1c7e28b198f/"
"pnas.1809583115fig04.jpg")
TMPDIR = None
def get_fig4():
global TMPDIR
if len(sys.argv) > 1 and os.path.exists(sys.argv[1]):
return sys.argv[1]
try:
d = TMPDIR = tempfile.mkdtemp(prefix="pmc6310853_")
path = os.path.join(d, "fig4.jpg")
req = urllib.request.Request(FIG_URL, headers={"User-Agent": "Mozilla/5.0"})
with urllib.request.urlopen(req, timeout=60) as r, open(path, "wb") as f:
f.write(r.read())
return path
except Exception as e: # network not available
print(f" (could not download Fig. 4: {e})")
return None
print("\n5. Fig. 4L read from the published image (757 x 449 px; tracer circles vs chr16 midpoint '+')")
fig4 = get_fig4()
if fig4 is None:
print(" skipped")
else:
from PIL import Image
from scipy import ndimage as ndi
im = np.asarray(Image.open(fig4).convert("RGB")).astype(int)
if im.shape[:2] != (449, 757):
print(f" unexpected image size {im.shape[:2]}; skipped")
else:
y0, x0 = 336, 64 # inside the panel L frame (frame rows 334-424, cols 62-182)
sub = im[y0:423, x0:181]
r, g, b = sub[..., 0], sub[..., 1], sub[..., 2]
TOP, BOT = 334, 424 # frame lines = +50 / -50 on the y axis
pct = lambda y: 50 - 100 * (y - TOP) / (BOT - TOP)
yel = (r > 170) & (g > 170) & (b < 160)
circ = ndi.binary_dilation(yel, iterations=3)
gry = (abs(r - g) < 25) & (abs(g - b) < 25) & (r > 100) & (r < 225) & ~circ
gry[:15, :] = False # drop the 'n=42' label
ys, xs = np.nonzero(gry)
p = np.polyfit(xs + x0, ys + y0, 1)
for _ in range(3):
keep = abs(ys + y0 - np.polyval(p, xs + x0)) < 3
p = np.polyfit(xs[keep] + x0, ys[keep] + y0, 1)
def mid(X):
m = keep & (abs(xs + x0 - X) <= 5)
if m.sum() < 3:
return pct(np.polyval(p, X))
return pct(np.median(ys[m] + y0))
lab, nb = ndi.label(yel)
cs = ndi.center_of_mass(yel, lab, range(1, nb + 1))
sz = ndi.sum(yel, lab, range(1, nb + 1))
cnt = {"above": 0, "near": 0, "below": 0}
tot = 0
for (cy, cx), s in sorted(zip(cs, sz), key=lambda t: t[0][1]):
if s < 4:
continue
k = 2 if s > 20 else 1 # two overlapping circles merged into one blob
tot += k
d = pct(cy + y0) - mid(cx + x0)
t = "above" if d >= 3 else ("below" if d <= -3 else "near")
cnt[t] += k
if t != "above":
print(f" x = {cx + x0:5.1f} px: tracer {pct(cy + y0):5.1f}, midpoint {mid(cx + x0):5.1f}, "
f"difference {d:5.1f} -> {t}")
print(f" circles resolved: {tot} of 42; above midpoint (X side) {cnt['above']}, "
f"within +/-3 units {cnt['near']}, below (Y side) {cnt['below']}")
print(" (approximate: low resolution, overlapping circles; boundary threshold 3 axis units)")
if TMPDIR:
shutil.rmtree(TMPDIR, ignore_errors=True)
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, name):
fig.savefig(HERE / f"{name}.svg")
fig.savefig(HERE / f"{name}.png", dpi=200)
plt.close(fig)
print(f" wrote {name}.svg / {name}.png")
print("\n6. Figures")
# ---- fig1: Significance statement vs Results (abstract-vs-body contradiction)
fig = plt.figure(figsize=(7.2, 4.1))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.1); ax.axis("off")
def panel(x, y, w, h, title, body, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.15, y + h - 0.18, title, fontsize=10.5, fontweight="bold", color=INK, va="top")
ax.text(x + 0.15, y + h - 0.55, body, fontsize=10, color=INK, va="top", linespacing=1.45)
panel(0.12, 1.35, 2.85, 2.55, "Significance statement",
"“Male translocation mice with a\nmaternal-derived supernumerary\n"
"chromosome display the tracer\nchromosome exclusively to the\n"
"haploid set containing the X\nchromosome.”", INK2)
panel(4.23, 1.35, 2.85, 2.55, "Results, mouse section, para 2",
"“In the majority of cases we\nexamined, the hemisphere\ncontaining "
"the translocated\nchromosome 17¹⁶ also contained\nthe maternally derived X\n"
"chromosome (n = 28/42 cells,\nP = 0.022 ...)”", INK2)
ax.text(1.545, 1.55, "implies 42/42 = 100%", ha="center", fontsize=10.5, color=ORANGE, fontweight="bold")
ax.text(5.655, 1.55, f"{X_KEY}/{N_KEY} = {100*frac:.1f}%", ha="center", fontsize=10.5, color=BLUE, fontweight="bold")
arr = FancyArrowPatch((3.0, 2.62), (4.2, 2.62), arrowstyle="<->", mutation_scale=14, lw=2, color=INK2)
ax.add_patch(arr)
ax.text(3.6, 2.80, "same 42\ncells", ha="center", va="bottom", fontsize=10, color=INK2)
ax.text(3.6, 2.45, "“exclusively”\nvs\n“majority”", ha="center", va="top", fontsize=10, color=INK)
ax.text(3.6, 1.02, "Both sentences describe one experiment: Fig. 4 H–L, male Ts65Dn MEFs, n = 42.",
ha="center", fontsize=10, color=INK2)
ax.text(3.6, 0.62,
f"{N_KEY - X_KEY} of {N_KEY} cells ({100*(N_KEY - X_KEY)/N_KEY:.1f}%) not counted on the X side; "
f"exact 95% CI {100*ci.low:.1f}–{100*ci.high:.1f}% excludes 100%",
ha="center", fontsize=10, color=INK)
ax.text(3.6, 0.25, "orange = paper's wording (Significance); blue = recomputed from the paper's own count",
ha="center", fontsize=9.5, color=INK2, style="italic")
save(fig, "fig1")
# ---- fig2: proportion on one axis, claim vs data with exact CI, null reference
fig, ax = plt.subplots(figsize=(7.2, 2.9))
fig.subplots_adjust(left=0.30, right=0.97, top=0.86, bottom=0.22)
ax.axvline(50, color=INK2, lw=1, ls=(0, (4, 3)), zorder=1)
ax.text(49, 0.45, "chance (p = 0.5)", ha="right", va="bottom", fontsize=9.5, color=INK2)
ax.plot([100], [2], "o", ms=10, color=ORANGE, zorder=3)
ax.annotate("100%", (100, 2), xytext=(-10, 0), textcoords="offset points", ha="right", va="center",
fontsize=10, color=INK)
ax.plot([100 * ci.low, 100 * ci.high], [1, 1], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
for v in (ci.low, ci.high):
ax.plot([100 * v, 100 * v], [0.88, 1.12], color=BLUE, lw=2, zorder=2)
ax.plot([100 * frac], [1], "o", ms=10, color=BLUE, zorder=3)
ax.annotate(f"{100*frac:.1f}% (95% CI {100*ci.low:.1f}–{100*ci.high:.1f}%)", (52, 1),
xytext=(0, -14), textcoords="offset points", ha="left", va="top", fontsize=10, color=INK)
ax.set_yticks([2, 1])
ax.set_yticklabels(["Significance statement:\n“exclusively” (paper)",
"Results: 28/42 cells\n(recomputed, exact CI)"], fontsize=10, color=INK)
ax.set_ylim(0.35, 2.6)
ax.set_xlim(0, 105)
ax.set_xticks([0, 25, 50, 75, 100])
ax.set_xticklabels(["0%", "25%", "50%", "75%", "100%"])
ax.set_xlabel("Cells with the tracer chromosome in the X-containing hemisphere", fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_title("Share of 42 male Ts65Dn MEF cells, maternal-origin tracer (Fig. 4 H–L)",
fontsize=10.5, color=INK, loc="left", x=-0.39)
save(fig, "fig2")
Figure 3 legend calls a transduction P value of 0.096 statistically significant
Proc Natl Acad Sci U S A (2022) · doi:10.1073/pnas.1910844119 · PMC8812682
Summary
The Figure 3 legend reports a Friedman/exact test of CAR transduction across three activation conditions (CD3, CD3/CD28, CD3/CD28/CD81) and states that the data "reject the null hypothesis, indicating a statistically significant difference in transduction (P = 0.096)". At the conventional 0.05 level, which the paper's parallel Figure 2 statement also uses (P = 0.038, called significant), a P value of 0.096 does not reject the null hypothesis. No other significance level, one-sided test or adjustment is stated anywhere in the article. This P value is the only inferential statistic the article gives for any transduction comparison, and transduction is the outcome named in the title. The observed direction of the effect is unaffected, but its stated statistical significance is not supported by the number printed.
What the paper reports
Figure 3 legend ("CD81 costimulation improves CAR transduction efficiency"):
These data are representative of three experiments, with transduction efficiency across experiments illustrated. For CAR transduction, we employed the nonparametric Friedman test and exact test for repeated-measures experiments. Our null hypothesis (H0) is that there is no difference in median transduction between groups. Our data reject the null hypothesis, indicating a statistically significant difference in transduction (P = 0.096) for cells treated with additional CD81 costimulation.
Figure 2 legend ("CD81 costimulation enhances activation of isolated naive T cells"), the parallel statement for activation:
These data are representative of four experiments with average CD69 expression prior to CAR transduction across experiments illustrated. For activation, we employed the nonparametric Friedman test and exact test for repeated-measures experiments. Our null hypothesis (H0) is that there is no difference in median activation between groups. Our data reject the null hypothesis, indicating a statistically significant difference in activation (P = 0.038) for cells treated with additional CD81 costimulation.
Results, "CD81 Costimulation Enhances CAR Transduction in Naive T Cells":
In specific, activating naive T cells with additional CD81 costimulation achieved greater transduction efficiency as compared with CD3 or CD3/CD28 activation conditions (Fig. 3A and SI Appendix, Fig. S2A).
Abstract (Significance statement):
We show that CD81 costimulation of naive T cells prior to CAR transduction can lead to enhanced CAR expression in this T cell subset.
Discussion, second paragraph:
Importantly, the T cell product derived from these CD81-costimulated naive T cells achieved greater transduction efficiency, with greater levels of CD19 CAR expression.
The article has no statistics subsection in its Methods and states no significance level. The only P values in the article are the two above and the three in the Figure 1C legend (CD4 "<0.0001", CD8 "<0.0002", naive "<0.0001"; seven experiments; no test named).
What the paper's own numbers show
Method. I compared each reported P value with the conventional significance level of 0.05, checked whether rounding, halving (a one-sided reading) or a 0.10 level would change the comparison, and confirmed that the parallel Figure 2 statement follows the same template with a P value below 0.05. To see whether a P value below 0.05 was reachable with the stated design, I enumerated the exact null distribution of the Friedman statistic for three conditions and three, four and seven experiments (all (3!)^n equally likely within-experiment rankings, assuming no ties). The per-experiment transduction values are not in the article, so the test itself cannot be recomputed. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Transduction P value | Fig. 3 legend | P = 0.096, "reject the null hypothesis", "statistically significant" | 0.096 > 0.05, so H0 is not rejected at 0.05; P is 1.92 times the threshold |
| Unrounded range of the transduction P value | Fig. 3 legend | 0.096 | any value in [0.0955, 0.0965) rounds to 0.096; all are above 0.05 |
| One-sided reading | derived | not stated | 0.096 / 2 = 0.048 (< 0.05), but the legend prints 0.096 as the P it calls significant |
| 0.10 level | derived | not stated | 0.096 ≤ 0.10, but no 0.10 level is stated anywhere in the article |
| Activation P value | Fig. 2 legend | P = 0.038, "statistically significant" | 0.038 < 0.05, so H0 is rejected at 0.05; consistent |
| Smallest attainable exact Friedman P, 3 experiments × 3 conditions | Fig. 3 design | not stated | 0.0278 (6/216), reached only when all three experiments rank the conditions identically; the next value is 0.1944 |
| Is 0.096 an attainable exact P for 3 × 3? | Fig. 3 design | 0.096 | no; the nearest chi-square-approximation P is 0.0970 (Q = 4.667), whose exact P is 0.1944 |
| Smallest attainable exact Friedman P, 4 experiments × 3 conditions | Fig. 2 design | not stated | 0.0046; 0.038 is not attainable exactly (nearest: exact 0.0417, chi-square 0.0388 at Q = 6.5) |
| Smallest attainable exact Friedman P, 7 experiments × 3 conditions | Fig. 1C design | "<0.0001", "<0.0002", "<0.0001" | 0.000021, so values below 0.0001 are attainable; consistent if the same test was used |
The central finding needs no reconstruction: the legend's own P value is above 0.05, and the legend's own conclusion is that the null hypothesis is rejected. The attainability rows are a secondary observation. Neither 0.096 nor 0.038 matches the exact Friedman distribution for the simplest reading of the stated designs, which suggests that the printed values come from a different computation (for example the chi-square approximation, ties, or a different number of blocks) or contain a transcription slip. The article does not give enough detail to tell which.
Figure
P values on a log scale, with the 0.05 level as a vertical reference line. Filled orange points are the P values printed in the Figure 2 (activation, four experiments) and Figure 3 (transduction, three experiments) legends; both legends describe their result as statistically significant. Open blue points are the exact Friedman P values that three conditions and the stated number of experiments can produce (no ties), computed by recompute.py.txt. The Figure 2 value falls to the left of the line, where H0 is rejected. The Figure 3 value falls to the right of it, where H0 is not rejected. For the Figure 3 design an exact P below 0.05 (0.028) was attainable, and 0.096 itself is not one of the attainable values.
Why it matters
The article's title states that CD81 costimulation "skews CAR transduction toward naive T cells", and the Significance statement and Discussion describe enhanced transduction and CAR expression. The Figure 3 legend contains the only inferential test the article reports for any transduction comparison: the transduction results in Figures 4 and 5 carry no P values. The legend presents that test as a rejection of the null hypothesis. Read at the conventional 0.05 level, the test does not reject it. With three experiments the result is better described as a non-significant difference in the stated direction. The observed direction and size of the transduction difference are unchanged, and the activation results (Fig. 1C; Fig. 2, P = 0.038) are not affected. The paper calls itself a proof of principle, and its qualitative message survives. What does not survive is the claim that the transduction difference in Figure 3 is statistically significant. A reader or a meta-analyst taking the legend at its word would count as significant a result that is not significant by the paper's own figure.
Explanations considered
- Rounding. Any unrounded value that prints as 0.096 lies between 0.0955 and 0.0965, all above 0.05. Rounding does not account for it.
- A one-sided test. Halving gives 0.048, which is below 0.05. But the legend prints 0.096 as the value it calls significant, and it states a non-directional null ("no difference in median transduction between groups"). The Friedman statistic is an omnibus test across three groups whose P is already an upper-tail probability, so it has no natural one-sided version. A one-sided convention would have been reported as 0.048. This does not account for the sentence as written.
- A significance level of 0.10. P = 0.096 would meet a 0.10 level. No such level is stated anywhere in the article, and the parallel Figure 2 sentence uses the same wording for P = 0.038. A 0.10 level would make the sentence consistent only if it had been declared, and it was not.
- Multiplicity adjustment. Adjustment can only raise a P value, so an adjusted or unadjusted reading cannot bring 0.096 below 0.05.
- A different quantity or population. A P value has no units. The 0.038 in Figure 2 belongs to a separate test of activation in four experiments, not to an alternative reading of the transduction result.
- Significance at 0.05 unreachable with three experiments. This is ruled out by enumeration. With three conditions and three experiments, the exact Friedman test can produce P = 0.028 when every experiment orders the conditions the same way, so a significant result was possible with this design.
- A misprinted P value. This cannot be excluded. Because 0.096 is not an attainable exact P for three experiments and three conditions, the printed value may not be the value the stated test produced. If the experiments all ranked the conditions identically, the exact P would be 0.028 and the significance statement would be correct with a misprinted number. If they did not, the exact P would be 0.194 or larger, and the result would be further from significance. The per-experiment values (cited as SI Appendix Fig. S2A) are not reproduced in the retrievable text, so this cannot be resolved from the article. Either way, the published sentence pairs a conclusion with a number that does not support it, and only the authors can say which of the two is right.
Status
- Version of record. As of 2026-10-23, the sentence appears verbatim in the Figure 3 legend of the PubMed Central version (PMC8812682), in the Europe PMC full-text XML, and in the local PMC archive used here. The value 0.096 occurs only in the Figure 3 legend. The legend is published as text, not as part of an image, and all three copies agree, so a conversion artifact is not plausible.
- Supplement. The SI Appendix is cited for the transduction data (Fig. S2A). Per-experiment transduction measurements or ranks are not available in the retrievable text. They would settle the misprint question above, but they would not change the inconsistency between the printed P value and the printed conclusion.
- Corrections and comments. The Crossref record for the DOI has no update-to, updated-by or relation entries. PMC and Europe PMC list no correction, erratum, expression of concern or retraction, and the PMC page has no notice. Searches of PubPeer and Retraction Watch found no entry.
- Sources checked. The PMC article page and archive (
src/show.py PMC8812682),src/known.py 10.1073/pnas.1910844119(Crossref, PMC and Europe PMC notices: none), the Crossref works API, the Europe PMC full-text XML and core search, PubPeer and Retraction Watch.
Suggested correction
Either correct the conclusion or correct the number, whichever the underlying data support:
- If P = 0.096 is the correct result of the stated test, the Figure 3 legend could read: "Our data do not reject the null hypothesis at the 0.05 level (P = 0.096); the difference in transduction with additional CD81 costimulation was not statistically significant." Any statement elsewhere implying a statistically demonstrated transduction advantage in Figure 3 should be worded descriptively to match.
- If 0.096 is a misprint, the corrected P value should be given, together with the number of experiments (blocks) and conditions entering the test, whether the exact or the chi-square P is reported, and the per-experiment transduction values (or ranks) so that readers can verify it.
In either case, stating the significance level used for all tests (presumably 0.05, as in Figure 2) would remove the ambiguity.
Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC8812682 and regenerate fig1.svg / fig1.png.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
Inputs are only numbers printed in the article:
Fig. 3 legend: transduction, Friedman test and exact test, "three experiments", P = 0.096
Fig. 2 legend: activation, Friedman test and exact test, "four experiments", P = 0.038
Fig. 1C legend: activation, "seven experiments", P < 0.0001 / < 0.0002 / < 0.0001
Three activation conditions throughout: CD3, CD3/CD28, CD3/CD28/CD81 (k = 3).
"""
import itertools
import os
from collections import Counter
from fractions import Fraction
from scipy.stats import chi2
HERE = os.path.dirname(os.path.abspath(__file__))
ALPHA = 0.05
P_FIG3 = 0.096 # transduction, called "statistically significant"
P_FIG2 = 0.038 # activation, called "statistically significant"
K = 3 # conditions: CD3, CD3/CD28, CD3/CD28/CD81
def friedman_exact(n, k):
"""Exact null distribution of the Friedman statistic Q (no ties).
Returns list of (Q, exact upper-tail P, chi-square approximation P), Q ascending.
Every within-block ranking is equally likely under H0: (k!)^n configurations.
"""
perms = list(itertools.permutations(range(1, k + 1)))
sums = Counter({tuple([0] * k): 1})
for _ in range(n):
nxt = Counter()
for s, m in sums.items():
for p in perms:
nxt[tuple(a + b for a, b in zip(s, p))] += m
sums = nxt
total = len(perms) ** n
qc = Counter()
for s, m in sums.items():
q = Fraction(12, n * k * (k + 1)) * sum(r * r for r in s) - 3 * n * (k + 1)
qc[q] += m
qs = sorted(qc)
out = []
for q in qs:
tail = sum(qc[x] for x in qs if x >= q)
out.append((float(q), tail / total, float(chi2.sf(float(q), k - 1))))
return out, total
print("=== 1. The significance statement in the Fig. 3 legend ===")
print(f"Reported P (Fig. 3, transduction) = {P_FIG3}")
print(f"Conventional alpha = {ALPHA}")
print(f"P <= alpha ? = {P_FIG3 <= ALPHA} -> H0 not rejected at 0.05")
print(f"P / alpha = {P_FIG3 / ALPHA:.2f}")
print(f"P - alpha = {P_FIG3 - ALPHA:.3f}")
lo, hi = P_FIG3 - 0.0005, P_FIG3 + 0.0005
print(f"Range of unrounded P that rounds to 0.096 = [{lo:.4f}, {hi:.4f}); lower end > 0.05: {lo > ALPHA}")
print(f"Half of reported P (one-sided reading) = {P_FIG3 / 2:.3f} (< 0.05: {P_FIG3 / 2 < ALPHA})")
print(f"Reported P <= 0.10 ? = {P_FIG3 <= 0.10}")
print()
print("=== 2. The parallel statement in the Fig. 2 legend ===")
print(f"Reported P (Fig. 2, activation) = {P_FIG2}")
print(f"P <= alpha ? = {P_FIG2 <= ALPHA} -> H0 rejected at 0.05 (consistent)")
print()
print("=== 3. Exact Friedman P values attainable with k = 3 conditions (no ties) ===")
designs = {}
for n, note in [(3, "Fig. 3: three experiments"), (4, "Fig. 2: four experiments"),
(7, "Fig. 1C: seven experiments")]:
dist, total = friedman_exact(n, K)
designs[n] = dist
smallest = min(p for _, p, _ in dist)
print(f"n = {n} blocks x k = {K} ({note}); {total} equally likely rank configurations")
print(f" smallest attainable exact P = {smallest:.6f} (< 0.05: {smallest < ALPHA})")
if n <= 4:
print(" attainable values (Q: exact P | chi-square approx P):")
for q, p, a in sorted(dist, key=lambda t: -t[0]):
print(f" Q = {q:6.3f}: exact {p:.4f} | chi2 {a:.4f}")
else:
print(f" smallest attainable exact P < 0.0001: {smallest < 0.0001}")
print()
d3 = designs[3]
print("Fig. 3 design (n = 3): is 0.096 an attainable exact P?",
any(abs(p - P_FIG3) < 0.0005 for _, p, _ in d3))
q_near = min(d3, key=lambda t: abs(t[2] - P_FIG3))
print(f" nearest chi-square-approximation P: Q = {q_near[0]:.3f}, chi2 P = {q_near[2]:.4f}, "
f"exact P for the same Q = {q_near[1]:.4f}")
d4 = designs[4]
print("Fig. 2 design (n = 4): is 0.038 an attainable exact P?",
any(abs(p - P_FIG2) < 0.0005 for _, p, _ in d4))
q_near4 = min(d4, key=lambda t: abs(t[2] - P_FIG2))
print(f" nearest chi-square-approximation P: Q = {q_near4[0]:.3f}, chi2 P = {q_near4[2]:.4f}, "
f"exact P for the same Q = {q_near4[1]:.4f}")
print()
# ---------------------------------------------------------------- figure
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e3e2de"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
"svg.fonttype": "none"})
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
ax.set_facecolor(BG)
ax.set_xscale("log")
XMIN, XMAX = 0.003, 1.0
ax.set_xlim(XMIN, XMAX)
rows = [ # (y, label)
(3.0, "Fig. 2 (activation)\nreported P"),
(2.3, "Fig. 2 design\nattainable exact P"),
(1.0, "Fig. 3 (transduction)\nreported P"),
(0.3, "Fig. 3 design\nattainable exact P"),
]
ax.set_ylim(-0.25, 3.55)
ax.set_yticks([y for y, _ in rows])
ax.set_yticklabels([l for _, l in rows], fontsize=10)
# alpha reference line and regions
ax.axvline(ALPHA, color=INK2, lw=1, zorder=1)
ax.text(ALPHA * 0.92, -0.2, "\u2190 P \u2264 \u03b1 = 0.05: reject H\u2080", ha="right", va="bottom",
color=INK2, fontsize=10)
ax.text(ALPHA * 1.08, -0.2, "P > 0.05: do not reject H\u2080 \u2192", ha="left", va="bottom",
color=INK2, fontsize=10)
# attainable exact P values (blue, recomputed); label the ones that bracket 0.05
labels = {2.3: {0.0046: -1, 0.0417: -1, 0.0694: 1}, 0.3: {0.0278: 1, 0.1944: 1}}
for y, dist in [(2.3, designs[4]), (0.3, designs[3])]:
ps = sorted({round(p, 4) for _, p, _ in dist if p >= XMIN})
ax.plot(ps, [y] * len(ps), ls="none", marker="o", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=3)
for p, side in labels[y].items():
ax.annotate(f"{p:.3f}", (p, y), xytext=(0, 11 * side), textcoords="offset points",
ha="center", va="bottom" if side > 0 else "top", color=INK2, fontsize=10,
bbox=dict(facecolor=BG, edgecolor="none", pad=1.0), zorder=5)
# reported P values (orange, from the paper)
for y, p, verdict in [(3.0, P_FIG2, "described as significant"),
(1.0, P_FIG3, "described as significant")]:
ax.plot([p], [y], ls="none", marker="o", ms=10, color=ORANGE, mec=BG, mew=1.5, zorder=4)
ax.annotate(f"P = {p:.3f}, {verdict}", (p, y), xytext=(0, 11),
textcoords="offset points", ha="center", va="bottom", color=INK, fontsize=10,
bbox=dict(facecolor=BG, edgecolor="none", pad=1.0), zorder=5)
ax.set_xticks([0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1.0])
ax.set_xticklabels(["0.005", "0.01", "0.02", "0.05", "0.1", "0.2", "0.5", "1"])
ax.set_xlabel("P value (log scale)", color=INK)
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0, labelcolor=INK)
ax.tick_params(colors=INK2, axis="x")
handles = [
Line2D([], [], ls="none", marker="o", ms=10, color=ORANGE, label="Reported in the paper"),
Line2D([], [], ls="none", marker="o", ms=8, mfc=BG, mec=BLUE, mew=2,
label="Recomputed: exact Friedman P values the stated design can produce"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.4, -0.2), ncol=1,
frameon=False, fontsize=10, labelcolor=INK)
fig.tight_layout()
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
out = os.path.join(HERE, f"fig1.{ext}")
fig.savefig(out, facecolor=BG, bbox_inches="tight", **kw)
print("wrote", out)
Chi-square P = 0.0379 for casein B-cell responders does not follow from 19/45 vs 10/35
Proc Natl Acad Sci U S A (2022) · doi:10.1073/pnas.2117034119 · PMC8916005
Summary
The paper reports that more patients with multiple sclerosis (MS) than patients with other neurologic diseases (OND) had casein-specific B cells in an ELISPOT assay: 19/45 (42%) vs 10/35 (28%), "P = 0.0379; χ2 test". A chi-square test on those counts gives χ² = 1.59, P = 0.208. Fisher's exact test gives P = 0.246, and none of the eight other standard two-group tests tried comes below P = 0.10. The reported P is reproduced exactly (P = 0.03794) only when the printed percentages are entered as if they were counts out of 100 patients per group. This comparison is the only between-group support the paper gives for the B-cell half of the abstract's statement that patients with MS had "significantly higher B cell and antibody reactivity to bovine casein". The mouse experiments and the separate serum IgG ELISA comparison are not affected.
What the paper reports
The result itself:
"While the number of casein-specific B cell spots/patient was similar in the two patient groups, the number of patients positive for casein-specific B cells was higher in the MS (19/45; 42%) than in the OND group (10/35; 28%) (P = 0.0379; χ2 test)." (Results, "Patients with MS Harbor Bovine Casein-Specific Antibodies and B Cells", paragraph 3)
The group sizes are stated in the same paragraph, in the Methods and in Table 2:
"we tested the CNS- and casein-specific B cell response in patients with MS (n = 45) and those with ONDs (n = 35)" (Results, same paragraph)
"PBMCs from 45 patients with MS and 35 patients with ONDs were tested by ELISPOT assay." (Materials and Methods, "Patient Sample ELISPOT")
"ELISPOT | 35 | 45" and "ELISPOT casein responders, %† | 28 | 42" (Table 2, columns "Patients with ONDs" | "Patients with MS")
The claims that rest on this comparison:
"The results obtained from the mouse model were complemented by clinical data showing that serum samples from patients with MS contained significantly higher B cell and antibody reactivity to bovine casein than those from patients with other neurologic diseases." (Abstract)
"Patients with MS show elevated numbers of B cell responses and IgG titers to bovine casein in the blood." (Fig. 4 title)
The Statistical Analysis section lists the tests used and the significance level; it does not mention a chi-square test:
"differences between two parametric groups were assessed using an unpaired t test or paired t test while ANOVA was used for greater than two groups. For nonparametric datasets, a Mann–Whitney U test was used. For both parametric and nonparametric datasets, a significance level of 5% was chosen." (Materials and Methods, "Statistical Analysis")
What the paper's own numbers show
Method. The stated counts give the 2×2 table MS 19 positive / 26 negative and OND 10 positive / 25 negative (80 patients). recompute.py.txt runs the usual two-group tests for a difference in proportions on this table with scipy: Pearson chi-square with and without Yates' continuity correction, its one-sided version, the likelihood-ratio G-test, the N−1 chi-square, and Fisher's, Barnard's and Boschloo's exact tests. It then searches every 2×2 table with row totals 45 and 35 for one whose chi-square P (uncorrected or Yates) equals 0.0379. Finally it tries tests that are not two-group comparisons and a chi-square on the printed percentages entered as counts.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| MS responders | Results para 3; Table 2 | 19/45; 42% | 42.22% |
| OND responders | Results para 3; Table 2 | 10/35; 28% | 28.57% (29% with conventional rounding) |
| Difference in responder proportion | derived | 42% vs 28% | 13.7 percentage points; Newcombe 95% CI −7.5 to 32.6 |
| Pearson χ², no correction | Results para 3 | P = 0.0379 (χ2 test) | χ² = 1.588, df 1, P = 0.2077 |
| Pearson χ², Yates correction | Results para 3 | P = 0.0379 | χ² = 1.052, P = 0.3051 |
| Pearson χ², one-sided | Results para 3 | P = 0.0379 | P = 0.1038 |
| G-test / N−1 χ² | Results para 3 | P = 0.0379 | P = 0.2050 / 0.2105 |
| Fisher exact, two-sided / one-sided | Results para 3 | P = 0.0379 | P = 0.2463 / 0.1525 |
| Barnard / Boschloo exact, two-sided | Results para 3 | P = 0.0379 | P = 0.2326 / 0.2175 |
| χ² needed for P = 0.0379 (df 1) | derived | — | 4.31; observed 1.59 |
| Any table with row totals 45 and 35 giving P = 0.0379 | derived | — | none (closest: Yates P = 0.0373 for 17/45 vs 5/35, and 0.0384 for 23/45 vs 9/35) |
| Percentages entered as counts, 42/100 vs 28/100, Pearson χ² | derived | P = 0.0379 | χ² = 4.308, P = 0.037940 (exact match) |
| Same, 42/100 vs 29/100 (conventional rounding) | derived | — | P = 0.0547 |
| Same, with Yates correction | derived | — | P = 0.0539 |
| MS rate against a fixed rate of 10/35 or 0.28 (goodness of fit; binomial) | derived | — | P = 0.043 or 0.034; binomial P = 0.048 or 0.045 |
| ELISA OD > 1.0 rates, 16/39 vs 5/23 | Table 2 (no P given) | 41.02% vs 21.7% | χ² P = 0.1211; Fisher P = 0.1673 |
| Group sizes needed at the same proportions for P < 0.05 | derived | — | about 3 times larger (135/105: P = 0.029; 90/70: P = 0.075) |
The expected numbers of responders under no group difference are 16.3 (MS) and 12.7 (OND), against 19 and 10 observed. The four cell contributions to Pearson's statistic sum to 1.59, well short of the 4.31 needed for P = 0.0379. The 2×2 table is fully determined by the text and Table 2, and the table's row totals equal the totals in the Methods, so there is no unstated count that could be varied.
Figure
Figure 1. P values for the comparison of casein-specific B-cell responders, 19/45 (MS) vs 10/35 (OND), on one log-scaled axis. The orange point is the value reported in the Results (P = 0.0379, "χ2 test"). The filled blue points are nine standard two-group tests recomputed from the stated counts. All lie between 0.10 and 0.31, on the far side of the dashed line at the paper's stated 5% significance level. The open blue point in the shaded row is a Pearson chi-square run on the printed percentages as if each group had 100 patients (42/100 vs 28/100). It gives P = 0.0379, the reported value, to four decimals. All values come from recompute.py.txt.
Why it matters
The comparison changes from significant (as reported) to not significant (as recomputed) at the 5% level the paper sets. The recomputed 95% CI for the difference in responder proportions runs from −7.5 to +32.6 percentage points and includes zero. The same paragraph says that "the number of casein-specific B cell spots/patient was similar in the two patient groups", so the paper's ELISPOT data provide no between-group difference in B-cell reactivity that is statistically significant.
This affects the following statements:
- Abstract. "significantly higher B cell and antibody reactivity to bovine casein than those from patients with other neurologic diseases". The antibody half rests on the serum IgG ELISA (39 MS vs 23 OND, "P = 0.032; Mann–Whitney U test"), which this check does not touch. The B-cell half rests on the chi-square comparison alone.
- Fig. 4 title. "Patients with MS show elevated numbers of B cell responses". Fig. 4 itself shows the ELISA (panels A and B) and an MS-only correlation (panel C); the B-cell group contrast appears only in the text and Table 2.
- Table 2. The "ELISPOT casein responders, %" row (28 vs 42) is presented as a group contrast, without a P value.
- Translational conclusions. The paragraph closes with "these findings corroborated the link between casein- and MAG-specific antibodies and MS", and the Discussion calls for "personalized dietary restrictions". Without a significant B-cell difference, the human evidence that patients with MS differ from OND controls rests on the single ELISA comparison.
The direction of the observed difference is unchanged (42% vs 29%), and the data are compatible with a modest difference. They are equally compatible with none. At the observed proportions, about three times as many patients per group would have been needed for P < 0.05.
The main mouse findings do not depend on this test: casein immunization with demyelination and IgG deposition, MAG cross-reactivity shown with antibodies and adsorption, and complement dependence. Neither do the ELISA comparison (P = 0.032) or the MS-only correlation between CNS- and casein-specific spot counts (r = 0.605).
Explanations considered
- Continuity correction, or a one-sided test. Yates' correction raises P to 0.305. A one-sided Pearson test gives 0.104 and a one-sided Fisher test 0.153. None is near 0.0379.
- An exact or likelihood-based test labelled "χ2". The G-test, N−1 chi-square and Fisher's, Barnard's and Boschloo's tests give P = 0.205 to 0.246.
- Different or misprinted counts. The denominators 45 and 35 appear in the Results, the Methods and Table 2. The percentages match the counts (19/45 = 42.2%; 10/35 = 28.6%, printed as 28), so a misprint in the counts would also have to appear in the percentages. Across all 2×2 tables with row totals 45 and 35, no table gives P = 0.0379 to four decimals with or without Yates' correction. The nearest values (0.0373, 0.0384) need responder counts far from 19 and 10, such as 17/45 vs 5/35 with Yates' correction.
- The P value belongs to a different comparison. The other proportion contrast in Table 2, ELISA OD > 1.0 (41.02% of 39 vs 21.7% of 23, i.e. 16 vs 5), gives P = 0.121 (Fisher 0.167). The ELISA titer comparison has its own reported P (0.032), and the spot counts per patient are described as "similar". None of these accounts for 0.0379.
- A one-sample test against the OND rate. Comparing the MS count with a fixed rate of 10/35 or 0.28 gives P = 0.034 to 0.048 (goodness of fit or binomial), never 0.0379. Such a test also treats the OND estimate as known without error, so it is not a valid two-group comparison.
- Percentages entered as counts. A Pearson chi-square without correction on 42/100 vs 28/100 gives χ² = 4.308, P = 0.037940, which matches the reported value to all four printed decimals. This also fits the printed 28% (with 29%, P = 0.0547). The paper does not describe how the test was run, so this is a numerical match, not a documented procedure. Even so, the calculation treats 80 patients as 200 and is not a valid analysis of the stated data. It explains where the number could come from; it does not make the number correct.
- Text-conversion artifact. The counts and P value are plain text with ordinary decimal points. They read identically in the PMC archive and the Europe PMC full-text XML, with no minus signs, superscripts or table cells involved.
Status
As of 2026-10-23:
- Version of record. The publisher's page (pnas.org, reached through doi.org) returned HTTP 403 to automated access and could not be read. The archived full text in PubMed Central (PMC8916005) and the Europe PMC full-text XML carry the identical sentence, counts and P value, and Table 2 matches.
- Supplement. The SI Appendix (pnas.2117034119.sapp.pdf, from the Europe PMC supplementary-files archive) was converted to text and searched. Tables S3 and S4, which the paragraph cites, list patient characteristics (diagnosis, sex, age, disease duration, EDSS, dairy intake) but not per-patient ELISPOT results or responder status. The SI contains no "ELISPOT", "chi" or "χ". The data availability statement points to the article and SI; no source-data file was found. The supplement does not resolve the discrepancy.
- Corrections. None found. Crossref update and relation metadata are empty, and a Crossref
updates:filter on the DOI returns 0 items. Crossref carries Retraction Watch entries, and none appear. The PMC and Europe PMC notice lists are empty, Europe PMC has no comment/correction list, and PubMed (PMID 35235454) has no CommentsCorrections entries. - PubPeer. PubPeer's site and API refused automated access (HTTP 403 and 429), so it could not be searched directly. An earlier web search found no PubPeer thread for the article. A PubPeer comment therefore cannot be fully ruled out.
Sources: PMC full text via src/show.py PMC8916005; src/known.py 10.1073/pnas.2117034119; api.crossref.org; Europe PMC REST (search, fullTextXML, supplementaryFiles); PubMed E-utilities.
Suggested correction
In Results, "Patients with MS Harbor Bovine Casein-Specific Antibodies and B Cells", paragraph 3, the sentence could read: "the proportion of patients positive for casein-specific B cells was 42% (19/45) in the MS group and 29% (10/35) in the OND group; this difference was not statistically significant (P = 0.21, χ2 test; Fisher's exact test P = 0.25)." Reporting the difference with its 95% CI (13.7 percentage points, −7.5 to 32.6) would show the uncertainty.
Consequential changes:
- Abstract. Limit "significantly higher" to the antibody (ELISA) result, e.g. "...contained significantly higher antibody reactivity to bovine casein..., and a numerically, but not significantly, higher proportion had casein-specific B cells."
- Fig. 4 title. Remove "elevated numbers of B cell responses" or qualify it.
- Table 2. Optionally add counts to the "ELISPOT casein responders" row (10/35, 29%; 19/45, 42%).
- Statistical Analysis. Add the chi-square (or Fisher) test, since the section currently lists only t tests, ANOVA, Mann–Whitney and Pearson correlation.
Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC8916005 and regenerate fig1.
Run: python recompute.py.txt (writes fig1.svg and fig1.png next to this file)
Inputs come from the article as archived in PubMed Central (PMC8916005):
Results, "Patients with MS Harbor Bovine Casein-Specific Antibodies and B Cells",
paragraph 3: "the number of patients positive for casein-specific B cells was higher
in the MS (19/45; 42%) than in the OND group (10/35; 28%) (P = 0.0379; chi2 test)".
Table 2: ELISPOT n = 35 (OND), 45 (MS); "ELISPOT casein responders, %" 28 (OND), 42 (MS);
ELISA n = 23 (OND), 39 (MS); "ELISA OD value > 1.0, %" 21.7 (OND), 41.02 (MS).
Requires numpy, scipy (>= 1.7 for barnard_exact/boschloo_exact) and matplotlib.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
REPORTED_P = 0.0379
MS_POS, MS_N = 19, 45 # MS patients positive for casein-specific B cells
OND_POS, OND_N = 10, 35 # OND patients positive
REPORTED_PCT = (42, 28) # percentages as printed in the text and in Table 2
tab = np.array([[MS_POS, MS_N - MS_POS], [OND_POS, OND_N - OND_POS]])
print("== Inputs ==")
print("2x2 table [[pos, neg] MS; [pos, neg] OND]:", tab.tolist())
print(f"MS responders: {MS_POS}/{MS_N} = {100*MS_POS/MS_N:.2f}% (reported 42%)")
print(f"OND responders: {OND_POS}/{OND_N} = {100*OND_POS/OND_N:.2f}% (reported 28%; "
f"conventional rounding gives {round(100*OND_POS/OND_N)}%)")
diff = MS_POS/MS_N - OND_POS/OND_N
print(f"Difference in proportions: {100*diff:.2f} percentage points")
exp = stats.contingency.expected_freq(tab)
print("Expected counts under no difference:", np.round(exp, 2).tolist())
contrib = (tab - exp) ** 2 / exp
print("Pearson contributions:", np.round(contrib.ravel(), 3).tolist())
print("\n== Two-group tests on the stated counts 19/45 vs 10/35 ==")
res = {}
c, p, _, _ = stats.chi2_contingency(tab, correction=False)
res["Pearson chi2, no correction"] = (c, p)
c, p, _, _ = stats.chi2_contingency(tab, correction=True)
res["Pearson chi2, Yates correction"] = (c, p)
res["Pearson chi2, one-sided"] = (res["Pearson chi2, no correction"][0],
res["Pearson chi2, no correction"][1] / 2)
g, p, _, _ = stats.chi2_contingency(tab, correction=False, lambda_="log-likelihood")
res["G-test (likelihood ratio)"] = (g, p)
N = tab.sum()
cN1 = res["Pearson chi2, no correction"][0] * (N - 1) / N
res["N-1 chi2"] = (cN1, stats.chi2.sf(cN1, 1))
_, p = stats.fisher_exact(tab)
res["Fisher exact, two-sided"] = (np.nan, p)
_, p = stats.fisher_exact(tab, alternative="greater")
res["Fisher exact, one-sided"] = (np.nan, p)
r = stats.barnard_exact(tab)
res["Barnard exact, two-sided"] = (r.statistic, r.pvalue)
r = stats.boschloo_exact(tab)
res["Boschloo exact, two-sided"] = (r.statistic, r.pvalue)
for k, (s, p) in res.items():
s_txt = "" if np.isnan(s) else f"stat = {s:.4f}"
print(f" {k:32s} {s_txt:16s} P = {p:.4f}")
chi_needed = stats.chi2.isf(REPORTED_P, 1)
print(f"\nChi-square (df 1) needed for P = {REPORTED_P}: {chi_needed:.4f}; "
f"observed {res['Pearson chi2, no correction'][0]:.4f}")
# Newcombe (Wilson score) 95% CI for the difference in proportions
def wilson(x, n, z=1.959964):
ph = x / n
centre = (ph + z*z/(2*n)) / (1 + z*z/n)
half = z*np.sqrt(ph*(1-ph)/n + z*z/(4*n*n)) / (1 + z*z/n)
return centre - half, centre + half
l1, u1 = wilson(MS_POS, MS_N)
l2, u2 = wilson(OND_POS, OND_N)
p1, p2 = MS_POS/MS_N, OND_POS/OND_N
lo = diff - np.sqrt((p1-l1)**2 + (u2-p2)**2)
hi = diff + np.sqrt((u1-p1)**2 + (p2-l2)**2)
print(f"Newcombe 95% CI for the difference: {100*lo:.1f} to {100*hi:.1f} percentage points")
print("\n== Exhaustive search: all 2x2 tables with row totals 45 and 35 ==")
best = []
for a in range(MS_N + 1):
for b in range(OND_N + 1):
t = np.array([[a, MS_N - a], [b, OND_N - b]])
if (t.sum(0) == 0).any():
continue
for corr in (False, True):
_, p, _, _ = stats.chi2_contingency(t, correction=corr)
best.append((abs(p - REPORTED_P), a, b, corr, p))
best.sort()
print("Closest five (uncorrected or Yates) to P = 0.0379:")
for d, a, b, corr, p in best[:5]:
print(f" {a}/45 vs {b}/35, Yates={corr}: P = {p:.4f}")
n_match = sum(1 for d, *_ in best if d < 0.00005)
print(f"Tables giving P = 0.0379 to 4 decimals: {n_match}; "
f"among tables within 4 responders of 19 and 10: "
f"{sum(1 for d,a,b,c,p in best if d < 0.00005 and abs(a-19)<=4 and abs(b-10)<=4)}")
print("\n== Percentages entered as counts out of 100 ==")
pct_res = {}
for a, b in [(42, 28), (42.22, 28.57), (42, 29)]:
t = np.array([[a, 100 - a], [b, 100 - b]])
c, p, _, _ = stats.chi2_contingency(t, correction=False)
pct_res[(a, b)] = (c, p)
print(f" {a}/100 vs {b}/100, uncorrected: chi2 = {c:.4f}, P = {p:.6f}")
c, p, _, _ = stats.chi2_contingency(np.array([[42, 58], [28, 72]]), correction=True)
print(f" 42/100 vs 28/100, Yates: chi2 = {c:.4f}, P = {p:.6f}")
print("\n== Other tests that do not compare two groups ==")
for lab, p0 in [("MS vs fixed p0 = 10/35", 10/35), ("MS vs fixed p0 = 0.28", 0.28)]:
o = np.array([MS_POS, MS_N - MS_POS]); e = np.array([p0*MS_N, (1-p0)*MS_N])
c, p = stats.chisquare(o, e)
pb = stats.binomtest(MS_POS, MS_N, p0, alternative="two-sided").pvalue
print(f" {lab}: goodness-of-fit chi2 = {c:.4f}, P = {p:.4f}; binomial two-sided P = {pb:.4f}")
c, p = stats.chisquare([MS_POS, OND_POS])
print(f" 19 vs 10 responders, equal expected: chi2 = {c:.4f}, P = {p:.4f}")
print("\n== ELISA OD > 1.0 rates in Table 2 (no P value given there) ==")
ms_e, ond_e = 0.4102*39, 0.217*23
print(f" 41.02% of 39 = {ms_e:.2f}; 21.7% of 23 = {ond_e:.2f} -> 16/39 vs 5/23")
te = np.array([[16, 23], [5, 18]])
c, p, _, _ = stats.chi2_contingency(te, correction=False)
print(f" Pearson chi2 = {c:.4f}, P = {p:.4f}")
_, p = stats.fisher_exact(te)
print(f" Fisher exact two-sided P = {p:.4f}")
print("\n== Sample size needed at the same proportions (19/45 vs 10/35) ==")
for k in range(1, 5):
t = tab * k
c, p, _, _ = stats.chi2_contingency(t, correction=False)
print(f" x{k}: n = {MS_N*k}/{OND_N*k}, P = {p:.4f}")
# ------------------------------------------------------------------ figure 1
BG, TXT, SUB = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans",
"text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SUB, "ytick.color": TXT,
"svg.fonttype": "none"})
rows = [
("Reported in the paper (χ² test)", REPORTED_P, "rep"),
("Pearson χ², no correction", res["Pearson chi2, no correction"][1], "re"),
("Pearson χ², Yates correction", res["Pearson chi2, Yates correction"][1], "re"),
("Pearson χ², one-sided", res["Pearson chi2, one-sided"][1], "re"),
("G-test (likelihood ratio)", res["G-test (likelihood ratio)"][1], "re"),
("N−1 χ²", res["N-1 chi2"][1], "re"),
("Fisher exact, two-sided", res["Fisher exact, two-sided"][1], "re"),
("Fisher exact, one-sided", res["Fisher exact, one-sided"][1], "re"),
("Barnard exact, two-sided", res["Barnard exact, two-sided"][1], "re"),
("Boschloo exact, two-sided", res["Boschloo exact, two-sided"][1], "re"),
("Percentages entered as counts\n(42/100 vs 28/100), Pearson χ²", pct_res[(42, 28)][1], "pct"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.9), facecolor=BG)
ax.set_facecolor(BG)
ys = np.arange(len(rows))[::-1].astype(float)
ys[-1] -= 0.5 # separate the "percentages as counts" row
ys[1:-1] -= 0.25 # separate the reported row from the recomputed block
for (lab, p, kind), y in zip(rows, ys):
if kind == "rep":
ax.plot(p, y, "o", ms=10, color=ORANGE, zorder=3)
ax.annotate(f"P = {p:.4f}", (p, y), xytext=(-10, 0), textcoords="offset points",
va="center", ha="right", color=ORANGE, fontsize=10, fontweight="bold")
elif kind == "re":
ax.plot(p, y, "o", ms=9, color=BLUE, zorder=3)
ax.annotate(f"{p:.3f}", (p, y), xytext=(9, 0), textcoords="offset points",
va="center", color=BLUE, fontsize=10)
else:
ax.plot(p, y, "o", ms=9, mfc=BG, mec=BLUE, mew=2, zorder=3)
ax.annotate(f"P = {p:.4f}", (p, y), xytext=(-10, 0), textcoords="offset points",
va="center", ha="right", color=BLUE, fontsize=10)
ax.text(0.058, y, "matches the reported value,\nbut assumes 100 patients per group",
va="center", ha="left", color=BLUE, fontsize=10)
ax.axvline(0.05, color=SUB, lw=1, ls="--", zorder=1)
ax.text(0.05, ys[0] + 0.75, "P = 0.05", color=SUB, fontsize=10, ha="center", va="bottom",
bbox=dict(facecolor=BG, edgecolor="none", pad=1.5), zorder=2)
ax.axhspan(ys[-1] - 0.5, ys[-1] + 0.6, color="#efeeea", zorder=0)
ax.set_xscale("log")
ax.set_xlim(0.01, 1)
ax.set_xticks([0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1])
ax.set_xticklabels(["0.01", "0.02", "0.05", "0.1", "0.2", "0.5", "1"])
ax.set_ylim(ys[-1] - 0.7, ys[0] + 1.3)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
for t, (_, _, kind) in zip(ax.get_yticklabels(), rows):
t.set_color(ORANGE if kind == "rep" else TXT)
ax.set_xlabel("P value (log scale)", fontsize=10)
ax.grid(axis="x", color="#e4e3df", lw=0.8, zorder=0)
ax.tick_params(axis="y", length=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SUB)
from matplotlib.lines import Line2D
handles = [Line2D([], [], marker="o", ls="", ms=10, color=ORANGE, label="Reported in the paper"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE,
label="Recomputed from the stated counts 19/45 vs 10/35"),
Line2D([], [], marker="o", ls="", ms=9, mfc=BG, mec=BLUE, mew=2,
label="Recomputed from the printed percentages as counts")]
fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.995), ncol=1,
frameon=False, fontsize=10)
fig.text(0.01, 0.80, "MS vs OND casein-specific B-cell responders (Results, ELISPOT)",
fontsize=10, color=SUB)
fig.subplots_adjust(left=0.37, right=0.97, top=0.78, bottom=0.10)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
print("\nwrote fig1.svg and fig1.png to", HERE)
Abstract reports PGS-ADHD odds ratios above 1 as decreased odds; Results report them below 1
Psychol Med (2024) · doi:10.1017/S0033291724002186 · PMC11578915
Summary
The study fits multinomial regressions of four treatment trajectories on polygenic scores (PGS) in 10,577 people with early-onset depression, with the brief contact trajectory as the reference. The Abstract says higher PGS-ADHD was linked to "decreased odds" of the prolonged initial contact and persistent contact trajectories, but it prints odds ratios above 1 for both: 1.06 (1.01–1.11) and 1.12 (1.03–1.21). For the same comparisons the Results text and Supplementary Table 2 give 0.91 (0.87–0.96), p = 0.0002, and 0.90 (0.83–0.97), p = 0.007. All four intervals exclude 1, so the Abstract's numbers disagree with the body in direction and value, and the two Abstract triples are identical to the PGS-AN rows of Supplementary Table 2. The Abstract is the paper's most widely read summary, and its printed figures reverse the direction of the result the Discussion calls the strongest.
What the paper reports
Relative to the brief contact trajectory, higher PGS for ADHD was associated with a decreased odds of membership in the prolonged initial contact (odds ratio = 1.06, 95% confidence interval = 1.01–1.11) and persistent contact (1.12, 1.03–1.21) trajectories, while PGS-AN was associated with increased odds of membership in the persistent contact trajectory (1.12, 1.03–1.21).
— Abstract, Results
PGS-ADHD was associated with decreased odds of membership in the prolonged initial contact (OR = 0.91 [0.87–0.96], p = 0.0002) and persistent contact classes (OR = 0.90 [0.83–0.97], p = 0.007). PGS-AN was associated with increased odds of membership in the persistent contact class (OR = 1.12 [1.03–1.21], p = 0.005).
— Results, "PGS and trajectory patterns in secondary care", second paragraph
For each PGS and family history variable, we calculated the odds ratio (OR), 95% confidence intervals (CIs), and p-value for trajectory class membership relative to the brief contact class using multinomial regression models adjusted for sex, age-at-onset, and calendar year-at-diagnosis in three-year bands.
— Methods, "Statistical analysis"
Statistical significance was assessed at the Bonferroni-adjusted alpha of 0.0083 for the PGS (0.05/6 PGSs), and 0.017 for parental history (0.05/3 parental history variables).
— Methods, "Statistical analysis"
The brief contact class (class 1) was used as the reference category in the multinomial regressions.
— Figure 2 legend
The strongest results were for PGS-ADHD, which was associated with increased likelihood of a brief contact pattern relative to the prolonged initial contact and persistent contact patterns, particularly for younger individuals.
— Discussion
Supplementary Table 2 ("Associations between continuous polygenic scores (PGSs) and trajectory classes"; brief contact is the reference) lists, as OR, SE, p-value, lower CI, upper CI:
Prolonged initial contact | PGS-ADHD | 0.91 | 0.02 | 2.12x10-04 | 0.87 | 0.96 Persistent contact | PGS-ADHD | 0.90 | 0.04 | 6.99x10-03 | 0.83 | 0.97 Prolonged initial contact | PGS-AN | 1.06 | 0.02 | 3.06x10-02 | 1.01 | 1.11 Persistent contact | PGS-AN | 1.12 | 0.04 | 5.00x10-03 | 1.03 | 1.21
— Supplementary material (S0033291724002186sup001.docx), STable 2, rows for PGS-ADHD and PGS-AN
What the paper's own numbers show
Method. In a multinomial logistic model with brief contact as the reference, the odds ratio for class k is exp(β), where β is the change in log[P(k)/P(brief)] per SD of PGS. An OR above 1 therefore means increased odds of class k relative to brief contact, and an OR below 1 means decreased odds. A Wald 95% CI is exp(β ± 1.96 SE), so SE = (ln hi − ln lo)/3.92, z = ln(OR)/SE and the two-sided p follows. I applied this to every Abstract and Results triple. I also took reciprocals of the Results values, to test whether a swapped reference category could produce the Abstract's numbers, and searched all 18 rows of Supplementary Table 2, and the other trajectory-class tables of the supplement (STables 3, 4, 5 and 18), for the Abstract's exact OR and CI bounds and for any PGS-ADHD estimate above 1 for these two classes. Every number below is printed by recompute.py.txt (Python, numpy/scipy).
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| PGS-ADHD, prolonged initial contact | Abstract | 1.06 (1.01–1.11), "decreased odds" | OR > 1, so increased odds (+6% per SD); z = +2.42, p = 0.0155 |
| PGS-ADHD, prolonged initial contact | Results text; STable 2 | 0.91 [0.87–0.96], p = 0.0002 | OR < 1, decreased odds (−9% per SD); z = −3.76, p = 0.0002 |
| PGS-ADHD, persistent contact | Abstract | 1.12 (1.03–1.21), "decreased odds" | OR > 1, so increased odds (+12% per SD); z = +2.76, p = 0.0058 |
| PGS-ADHD, persistent contact | Results text; STable 2 | 0.90 [0.83–0.97], p = 0.007 | OR < 1, decreased odds (−10% per SD); z = −2.65, p = 0.0081 |
| PGS-AN, persistent contact | Abstract; Results text; STable 2 | 1.12 [1.03–1.21] (p = 0.005 in Results text and STable 2) | z = +2.76, p = 0.0058; all three places agree |
| Reciprocal of Results PGS-ADHD, prolonged | derived | Abstract: 1.06 (1.01–1.11) | 1.10 [1.04–1.15]; does not match |
| Reciprocal of Results PGS-ADHD, persistent | derived | Abstract: 1.12 (1.03–1.21) | 1.11 [1.03–1.20]; does not match |
| Supplement rows equal to the Abstract's "ADHD, prolonged" triple | STable 2; STable 5 | 1.06 (1.01–1.11) | PGS-AN, prolonged initial contact, in STable 2 (printed p = 0.0306) and in STable 5 (printed p = 0.0281); no PGS-ADHD row |
| STable 2 rows equal to the Abstract's "ADHD, persistent" triple | STable 2 | 1.12 (1.03–1.21) | one row: PGS-AN, persistent contact |
| Abstract prolonged-contact value against the Bonferroni α | Abstract; Methods | α = 0.05/6 = 0.0083 | p = 0.0155 from the CI (0.0306 as printed in STable 2 for the matching PGS-AN row); neither passes |
| PGS-ADHD in the mutually adjusted model | STable 5 | 0.90 [0.86–0.95] and 0.91 [0.84–0.99] | both below 1 |
| PGS-ADHD for these two classes in the age strata and top-decile/5% contrasts | STable 18; STables 3–4 | 18 estimates | 16 below 1; 2 point estimates above 1 (age 21–25, persistent contact, 1.13 [0.98–1.31], p = 0.0968; top 5% vs bottom 5%, persistent contact, 1.04 [0.62–1.74]), both with CIs that include 1; none has a CI wholly above 1 and none matches an Abstract triple |
The Results-text and STable 2 values are internally consistent: the p values recovered from their CIs (0.0002 and 0.0081) match the printed 0.0002 and 0.007 to within rounding of the two-decimal bounds. The Abstract's two "ADHD" triples are not consistent with the word "decreased" in the same sentence, with the Results text, or with any reciprocal of it. Each matches, digit for digit, a PGS-AN row of Supplementary Table 2.
Figure
Figure 1. The two PGS-ADHD comparisons against the brief contact trajectory, as printed in the Abstract (left) and in the Results text (right). All numbers are copied from the paper (orange: Abstract and Results text as printed); no recomputed values are drawn. The Abstract prints OR 1.06 (1.01–1.11) for prolonged initial contact and 1.12 (1.03–1.21) for persistent contact, both above 1. The Results text, which agrees with Supplementary Table 2, prints 0.91 [0.87–0.96] (p = 0.0002) and 0.90 [0.83–0.97] (p = 0.007), both below 1. Each pair of intervals excludes 1 on opposite sides, so the two sources disagree in direction, not only in value. Both sources describe the association in words as "decreased odds", which fits only the Results values. The Abstract's two triples are identical to the PGS-AN rows of Supplementary Table 2 for the same classes.
Why it matters
The PGS-ADHD association is the result the Discussion singles out as "the strongest". The Abstract is what most readers, indexers and citing authors see, and it is the only part of the paper whose numbers point the other way. A reader who takes the Abstract's numbers at face value would conclude that higher PGS-ADHD raises the odds of the longer-treatment trajectories, by about 6% and 12% per SD. The body says the opposite: it lowers them, by about 9% and 10% per SD. The Abstract's prolonged-contact value would also fail the paper's own multiple-testing threshold (p = 0.0155 from its CI, against α = 0.0083), whereas the body's value passes (p = 0.0002). Anyone pooling these estimates in a meta-analysis or citing them from the Abstract would carry the wrong sign.
The error is confined to the Abstract's numbers. The Abstract's wording, the Results text, Figure 2 (brief contact as reference), Supplementary Tables 2 and 5, the Discussion and the Conclusions all agree with ORs below 1, and the age-stratified results describe "the negative association between PGS-ADHD and the persistent contact class" as driven by ages 10–20 (STable 18: 0.81 [0.71–0.94] at 10–15 and 0.84 [0.74–0.95] at 16–20). The paper's main conclusions are therefore not affected.
Explanations considered
- A different reference category. The Abstract states the comparison as "relative to the brief contact trajectory", the same as the Methods, the Results text and the Figure 2 legend. If the reference were swapped, the Results values would become 1.10 [1.04–1.15] and 1.11 [1.03–1.20]. Neither matches the Abstract's 1.06 (1.01–1.11), and a swap would also make the Abstract's word "decreased" wrong. This does not account for the numbers.
- A different model or subgroup. The mutually adjusted model (STable 5) gives PGS-ADHD ORs of 0.90 [0.86–0.95] and 0.91 [0.84–0.99]. The age-stratified analyses are described as a "negative association". No model or subgroup in the paper gives a PGS-ADHD OR for these two classes whose CI lies entirely above 1, or that matches the Abstract's triples. The only point estimates above 1 (age 21–25, persistent contact, 1.13 [0.98–1.31], STable 18; top 5% vs bottom 5%, persistent contact, 1.04 [0.62–1.74], STable 4) have CIs that include 1. This does not account for the numbers.
- A text-conversion or rendering artifact. The same values appear in the PMC archive, the Europe PMC full text, the publisher's abstract deposited with Crossref, and the Cambridge Core version of record. They are complete OR–CI triples in running text, with no minus signs, superscripts or table cells involved. This does not account for the numbers.
- The Abstract being correct and the body wrong. The Results text, Supplementary Table 2 (with p values that match its CIs), Supplementary Table 5, the Discussion, and the Abstract's own wording all agree on ORs below 1. Only the Abstract's two numeric triples disagree. The body values are the ones supported throughout the paper.
- Values carried from another row. Both Abstract triples equal, OR and both CI bounds, the PGS-AN rows of Supplementary Table 2 for the same two classes (1.06 [1.01–1.11], p = 0.0306; 1.12 [1.03–1.21], p = 0.005). The same 1.06 (1.01–1.11) triple is also the PGS-AN, prolonged initial contact row of the mutually adjusted model (STable 5, p = 0.0281), and it appears nowhere in the main text. This match is consistent with the Abstract reporting the PGS-AN rows under the PGS-ADHD label. It describes what the numbers show, not how they came to be there.
No explanation found reconciles the Abstract's numbers with the rest of the paper.
Status
- Version of record. Checked on Cambridge Core (doi.org resolves to cambridge.org/core/product/identifier/S0033291724002186/type/journal_article). The Abstract and Results text carry the same values as the PMC archive. The publisher's abstract in the Crossref record and the Europe PMC full text also carry 1.06 (1.01–1.11) and 1.12 (1.03–1.21).
- Supplement. S0033291724002186sup001.docx, downloaded through Europe PMC. STable 2 and STable 5 support the Results text, as shown above; STables 3, 4 and 18 contain no PGS-ADHD estimate for these two classes that matches the Abstract or has a CI wholly above 1. A side observation: the STable 2 and STable 5 footnotes give the Bonferroni alpha as "0.00083", while the Methods give 0.0083 (0.05/6). This does not bear on the Abstract discrepancy.
- Corrections. As of 2026-10-23 no correction, erratum, corrigendum, expression of concern or retraction was found. Crossref lists no update relations for the DOI, and a Crossref
updatesfilter search returned none. PMC and Europe PMC list no notices. - PubPeer. The PubPeer API and search page refused automated access (Cloudflare challenge). A web search for the title with "pubpeer" returned no PubPeer thread.
- Sources. PMC11578915 (local archive via
src/show.py); Europe PMC full text and supplementary files; Crossref works record for 10.1017/S0033291724002186; Cambridge Core version of record; PubPeer;src/known.py.
Suggested correction
In the Abstract, Results paragraph, replace "(odds ratio = 1.06, 95% confidence interval = 1.01–1.11)" with "(odds ratio = 0.91, 95% confidence interval = 0.87–0.96)" and "persistent contact (1.12, 1.03–1.21) trajectories" with "persistent contact (0.90, 0.83–0.97) trajectories", matching the Results text and Supplementary Table 2. The PGS-AN estimate in the same sentence (1.12, 1.03–1.21) is correct and needs no change. Separately, the Bonferroni alpha in the footnotes of Supplementary Tables 2 and 5 could be corrected from 0.00083 to 0.0083 to match the Methods.
Recomputation script (python recompute.py.txt)
"""Recompute the PGS-ADHD abstract-versus-Results discrepancy in PMC11578915
(Psychol Med 2024, doi:10.1017/S0033291724002186) and regenerate the figures.
Run: python recompute.py.txt
Needs: numpy, scipy, matplotlib.
Prints every recomputed number used in writeup.md. Writes, into the folder that
holds this file:
candidates/cand_A..cand_E (.svg/.png) five alternative designs for Figure 1
fig1.svg / fig1.png a copy of the design named in FIG1 below
All inputs are copied from the paper: the Abstract, the Results section
("PGS and trajectory patterns in secondary care"), the Methods (Bonferroni
alpha) and Supplementary Table 2 (STable 2; continuous PGS vs trajectory class,
brief contact = reference). STable 5 (mutually adjusted model), STable 18
(age-stratified models) and STables 3-4 (top 10% / top 5% contrasts) are used only as
further checks.
Method: Wald inference on the log-odds scale. A 95% CI for an odds ratio is
exp(ln OR +/- 1.96 SE), so SE = (ln hi - ln lo) / (2 * 1.959964),
z = ln(OR) / SE and two-sided p = 2 * (1 - Phi(|z|)).
"""
import os
import shutil
import numpy as np
from scipy.stats import norm
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
FIG1 = "A" # which candidate design is copied to fig1.*
Z = norm.ppf(0.975)
ALPHA_BONF = 0.05 / 6 # Methods: "0.0083 for the PGS (0.05/6 PGSs)"
def wald(o, lo, hi):
se = (np.log(hi) - np.log(lo)) / (2 * Z)
z = np.log(o) / se
return se, z, 2 * norm.sf(abs(z))
line = "-" * 78
# ------------------------------------------------------------------ inputs
ABSTRACT = { # label: (OR, lo, hi) -- Abstract, Results paragraph
"ADHD, prolonged initial contact": (1.06, 1.01, 1.11),
"ADHD, persistent contact": (1.12, 1.03, 1.21),
"AN, persistent contact": (1.12, 1.03, 1.21),
}
RESULTS = { # label: (OR, lo, hi, printed p) -- Results text
"ADHD, prolonged initial contact": (0.91, 0.87, 0.96, 0.0002),
"ADHD, persistent contact": (0.90, 0.83, 0.97, 0.007),
"AN, persistent contact": (1.12, 1.03, 1.21, 0.005),
}
# STable 2: (class, PGS): (OR, SE, p, lo, hi)
ST2 = {
("Prolonged initial contact", "MDD"): (0.96, 0.02, 6.60e-02, 0.91, 1.00),
("Later re-entry", "MDD"): (1.09, 0.04, 1.47e-02, 1.02, 1.17),
("Persistent contact", "MDD"): (0.95, 0.04, 1.89e-01, 0.88, 1.03),
("Prolonged initial contact", "ADHD"): (0.91, 0.02, 2.12e-04, 0.87, 0.96),
("Later re-entry", "ADHD"): (1.00, 0.04, 9.32e-01, 0.93, 1.07),
("Persistent contact", "ADHD"): (0.90, 0.04, 6.99e-03, 0.83, 0.97),
("Prolonged initial contact", "ASD"): (1.01, 0.02, 5.79e-01, 0.97, 1.06),
("Later re-entry", "ASD"): (1.08, 0.04, 3.15e-02, 1.01, 1.16),
("Persistent contact", "ASD"): (0.95, 0.04, 2.20e-01, 0.88, 1.03),
("Prolonged initial contact", "BD"): (1.03, 0.02, 2.00e-01, 0.98, 1.08),
("Later re-entry", "BD"): (0.99, 0.04, 8.68e-01, 0.92, 1.07),
("Persistent contact", "BD"): (0.99, 0.04, 7.15e-01, 0.91, 1.07),
("Prolonged initial contact", "SCZ"): (0.99, 0.02, 6.37e-01, 0.94, 1.04),
("Later re-entry", "SCZ"): (1.05, 0.04, 2.31e-01, 0.97, 1.12),
("Persistent contact", "SCZ"): (0.99, 0.04, 7.83e-01, 0.91, 1.07),
("Prolonged initial contact", "AN"): (1.06, 0.02, 3.06e-02, 1.01, 1.11),
("Later re-entry", "AN"): (1.02, 0.04, 5.99e-01, 0.95, 1.10),
("Persistent contact", "AN"): (1.12, 0.04, 5.00e-03, 1.03, 1.21),
}
ST5_ADHD = { # mutually adjusted model, STable 5
"prolonged initial contact": (0.90, 0.86, 0.95, 1.93e-04),
"persistent contact": (0.91, 0.84, 0.99, 3.28e-02),
}
ST5_AN_PRO = (1.06, 1.01, 1.11, 2.81e-02) # STable 5, PGS-AN, prolonged initial contact
# Every other PGS-ADHD estimate in the supplement for the two classes at issue
# (source table, model/subgroup, class): (OR, lo, hi, printed p)
OTHER_ADHD = {
("STable 18", "age 10-15", "prolonged initial contact"): (0.85, 0.77, 0.95, 2.83e-03),
("STable 18", "age 10-15", "persistent contact"): (0.81, 0.71, 0.94, 3.78e-03),
("STable 18", "age 16-20", "prolonged initial contact"): (0.95, 0.88, 1.01, 1.17e-01),
("STable 18", "age 16-20", "persistent contact"): (0.84, 0.74, 0.95, 5.03e-03),
("STable 18", "age 21-25", "prolonged initial contact"): (0.90, 0.82, 0.99, 3.01e-02),
("STable 18", "age 21-25", "persistent contact"): (1.13, 0.98, 1.31, 9.68e-02),
("STable 3", "top 10% vs bottom 10%", "prolonged initial contact"): (0.70, 0.56, 0.87, 1.60e-03),
("STable 3", "top 10% vs bottom 10%", "persistent contact"): (0.71, 0.50, 1.00, 5.17e-02),
("STable 3", "top 10% vs bottom 90%", "prolonged initial contact"): (0.83, 0.70, 0.99, 3.49e-02),
("STable 3", "top 10% vs bottom 90%", "persistent contact"): (0.88, 0.67, 1.15, 3.35e-01),
("STable 3", "top 10% vs bottom 50%", "prolonged initial contact"): (0.79, 0.66, 0.94, 8.95e-03),
("STable 3", "top 10% vs bottom 50%", "persistent contact"): (0.80, 0.61, 1.06, 1.22e-01),
("STable 4", "top 5% vs bottom 5%", "prolonged initial contact"): (0.70, 0.51, 0.96, 2.80e-02),
("STable 4", "top 5% vs bottom 5%", "persistent contact"): (1.04, 0.62, 1.74, 8.77e-01),
("STable 4", "top 5% vs bottom 95%", "prolonged initial contact"): (0.82, 0.65, 1.04, 1.04e-01),
("STable 4", "top 5% vs bottom 95%", "persistent contact"): (0.97, 0.68, 1.39, 8.82e-01),
("STable 4", "top 5% vs bottom 50%", "prolonged initial contact"): (0.77, 0.61, 0.99, 3.80e-02),
("STable 4", "top 5% vs bottom 50%", "persistent contact"): (0.88, 0.61, 1.28, 5.12e-01),
}
# ------------------------------------------------------------------ 1. abstract values
print(line)
print(f"Bonferroni alpha for the PGS = 0.05/6 = {ALPHA_BONF:.5f}")
print(line)
print("1. Abstract values: direction and p implied by each printed CI")
abs_calc = {}
for k, (o, lo, hi) in ABSTRACT.items():
se, z, p = wald(o, lo, hi)
abs_calc[k] = (se, z, p)
print(f" {k:32s} OR {o:.2f} [{lo:.2f}-{hi:.2f}] direction "
f"{'increased' if o > 1 else 'decreased'}; SE {se:.4f}; z {z:+.2f}; "
f"p {p:.4f}; passes Bonferroni: {p < ALPHA_BONF}; "
f"change in odds per SD {100 * (o - 1):+.0f}%")
# ------------------------------------------------------------------ 2. Results values
print(line)
print("2. Results-text values: p implied by each printed CI vs printed p")
res_calc = {}
for k, (o, lo, hi, pr) in RESULTS.items():
se, z, p = wald(o, lo, hi)
res_calc[k] = (se, z, p)
print(f" {k:32s} OR {o:.2f} [{lo:.2f}-{hi:.2f}] printed p {pr} | "
f"SE {se:.4f}; z {z:+.2f}; implied p {p:.4f}; direction "
f"{'increased' if o > 1 else 'decreased'}; change in odds per SD {100 * (o - 1):+.0f}%")
# ------------------------------------------------------------------ 3. reference swap
print(line)
print("3. Reciprocals of the Results ADHD values (what a swapped reference category would give)")
recip = {}
for k in ("ADHD, prolonged initial contact", "ADHD, persistent contact"):
o, lo, hi, _ = RESULTS[k]
r = (1 / o, 1 / hi, 1 / lo)
recip[k] = r
a = ABSTRACT[k]
print(f" {k:32s} 1/OR = {r[0]:.2f} [{r[1]:.2f}-{r[2]:.2f}] abstract {a[0]:.2f} "
f"[{a[1]:.2f}-{a[2]:.2f}] matches to 2 dp: "
f"{tuple(round(x, 2) for x in r) == a}")
# ------------------------------------------------------------------ 4. matching to STable 2
print(line)
print("4. Which STable 2 rows carry exactly the abstract's OR and CI (2 dp)?")
matches = {}
for k, (o, lo, hi) in ABSTRACT.items():
m = [f"{c} / PGS-{g}" for (c, g), v in ST2.items() if (v[0], v[3], v[4]) == (o, lo, hi)]
matches[k] = m
print(f" abstract {k:32s} {o:.2f} [{lo:.2f}-{hi:.2f}] -> {m}")
print(" STable 2 rows labelled PGS-ADHD for the same classes:")
for c in ("Prolonged initial contact", "Persistent contact"):
v = ST2[(c, "ADHD")]
print(f" {c:28s} OR {v[0]:.2f} [{v[3]:.2f}-{v[4]:.2f}] p {v[2]:.2e}")
print(" STable 2 printed SE vs SE recovered from its CI, and printed p vs implied p:")
for (c, g), (o, se_p, p_p, lo, hi) in ST2.items():
if g in ("ADHD", "AN") and c != "Later re-entry":
se, z, p = wald(o, lo, hi)
print(f" {c:28s} PGS-{g:4s} printed SE {se_p:.2f} vs {se:.3f}; printed p {p_p:.2e} vs {p:.2e}")
n_any = sum(1 for v in ST2.values() if (v[0], v[3], v[4]) == ABSTRACT["ADHD, prolonged initial contact"])
print(f" rows in STable 2 equal to 1.06 [1.01-1.11]: {n_any}")
# ------------------------------------------------------------------ 5. mutually adjusted model
print(line)
print("5. STable 5 (mutually adjusted) PGS-ADHD rows")
for k, (o, lo, hi, pr) in ST5_ADHD.items():
se, z, p = wald(o, lo, hi)
print(f" {k:28s} OR {o:.2f} [{lo:.2f}-{hi:.2f}] printed p {pr:.2e}; implied p {p:.2e}; below 1: {hi < 1}")
o, lo, hi, pr = ST5_AN_PRO
print(f" STable 5 PGS-AN, prolonged initial contact: OR {o:.2f} [{lo:.2f}-{hi:.2f}] printed p {pr:.2e}; "
f"equals the Abstract's 'ADHD, prolonged' triple: {(o, lo, hi) == ABSTRACT['ADHD, prolonged initial contact']}")
# ------------------------------------------------------------------ 6. other PGS-ADHD estimates
print(line)
print("6. Other PGS-ADHD estimates for the two classes (STable 18 age strata, STable 3 and 4 top-10%/5% contrasts)")
abs_triples = {ABSTRACT["ADHD, prolonged initial contact"], ABSTRACT["ADHD, persistent contact"]}
for (tab, sub, c), (o, lo, hi, pr) in OTHER_ADHD.items():
se, z, p = wald(o, lo, hi)
print(f" {tab:9s} {sub:22s} {c:26s} OR {o:.2f} [{lo:.2f}-{hi:.2f}] printed p {pr:.2e}; implied p {p:.3f}; "
f"OR>1 {o > 1}; CI wholly above 1 {lo > 1}; equals an Abstract triple {(o, lo, hi) in abs_triples}")
n_gt1 = sum(v[0] > 1 for v in OTHER_ADHD.values())
n_ci_gt1 = sum(v[1] > 1 for v in OTHER_ADHD.values())
n_match = sum(v[:3] in abs_triples for v in OTHER_ADHD.values())
print(f" point estimates above 1: {n_gt1}; CIs wholly above 1: {n_ci_gt1}; equal to an Abstract triple: {n_match}")
print(line)
# ================================================================== figures
import matplotlib # noqa: E402
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
LW, MS = 1.5, 8 # 1.5 pt = 2 px; marker 8 pt
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
W = 7.2
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(CAND, f"{name}.{ext}"), bbox_inches="tight", **kw)
plt.close(fig)
def fmt(t):
return f"{t[0]:.2f} [{t[1]:.2f}–{t[2]:.2f}]"
def log_axis(ax, lo=0.78, hi=1.30, ticks=(0.8, 0.85, 0.9, 0.95, 1.0, 1.05, 1.1, 1.15, 1.2)):
ax.set_xscale("log")
ax.set_xlim(lo, hi)
ticks = [t for t in ticks if lo <= t <= hi]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.grid(axis="x", color=GRID, lw=0.6)
ax.set_axisbelow(True)
ax.axvline(1.0, color=INK2, lw=0.8, zorder=1)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
A_PRO, A_PER = ABSTRACT["ADHD, prolonged initial contact"], ABSTRACT["ADHD, persistent contact"]
R_PRO, R_PER = RESULTS["ADHD, prolonged initial contact"], RESULTS["ADHD, persistent contact"]
AN_PRO = ST2[("Prolonged initial contact", "AN")]
AN_PER = ST2[("Persistent contact", "AN")]
# ------------------------------------------------------------------ A: two text panels
def cand_A():
fig = plt.figure(figsize=(W, 4.3))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off()
ax.set_xlim(0, 100); ax.set_ylim(0, 60)
def panel(x, w, title, sub):
ax.add_patch(FancyBboxPatch((x, 4), w, 50, boxstyle="round,pad=0,rounding_size=1.2",
fc="white", ec=INK2, lw=0.8))
ax.text(x + 2, 50.5, title, fontsize=11.5, weight="bold", va="top")
ax.text(x + 2, 46.3, sub, fontsize=10, color=INK2, va="top")
panel(1, 38, "Abstract", "Results paragraph")
panel(57, 42, "Results text", "“PGS and trajectory patterns”")
ax.text(50, 58.5, "PGS-ADHD vs the brief contact trajectory (reference): the same two comparisons",
ha="center", fontsize=10.5, va="top")
rows = [("Prolonged initial contact", A_PRO, (R_PRO[0], R_PRO[1], R_PRO[2]), f"p = {R_PRO[3]}", 34),
("Persistent contact", A_PER, (R_PER[0], R_PER[1], R_PER[2]), f"p = {R_PER[3]}", 19)]
for lab, a, r, ptxt, y in rows:
ax.text(3, y + 4.2, lab, fontsize=10, color=INK2)
ax.text(3, y, f"OR {fmt(a)}", fontsize=12.5, color=ORANGE, weight="bold")
ax.text(3, y - 4, "OR > 1: increased odds", fontsize=10, color=INK2)
ax.text(59.3, y + 4.2, lab, fontsize=10, color=INK2)
ax.text(59.3, y, f"OR {fmt(r)}", fontsize=12.5, color=ORANGE, weight="bold")
ax.text(59.3, y - 4, f"{ptxt}; OR < 1: decreased odds", fontsize=10, color=INK2)
ax.annotate("", xy=(56.6, y + 1), xytext=(39.4, y + 1),
arrowprops=dict(arrowstyle="<->", color=INK, lw=LW))
ax.text(48, y + 2.6, "opposite\ndirection", ha="center", va="bottom", fontsize=10, weight="bold", linespacing=1.1)
ax.text(48, y - 1.0, "both CIs\nexclude 1", ha="center", va="top", fontsize=10, color=INK2, linespacing=1.1)
ax.text(3, 9.8, "Wording: “decreased odds”", fontsize=10)
ax.text(3, 6.0, "Values equal the PGS-AN rows", fontsize=10, color=INK2)
ax.text(59.3, 9.8, "Wording: “decreased odds”", fontsize=10)
ax.text(59.3, 6.0, "Matches PGS-ADHD rows of STable 2", fontsize=10, color=INK2)
ax.text(1, 1.0, "Orange: values as printed in the paper.", fontsize=10, color=INK2)
save(fig, "cand_A")
# ------------------------------------------------------------------ B: dot-and-interval, 2 rows
def cand_B():
fig, ax = plt.subplots(figsize=(W, 3.4))
log_axis(ax, 0.80, 1.36)
items = [ # y, (o, lo, hi), style
(3.25, A_PRO, "abs"), (2.95, (R_PRO[0], R_PRO[1], R_PRO[2]), "res"), (2.65, recip["ADHD, prolonged initial contact"], "rec"),
(1.35, A_PER, "abs"), (1.05, (R_PER[0], R_PER[1], R_PER[2]), "res"), (0.75, recip["ADHD, persistent contact"], "rec"),
]
for y, (o, lo, hi), st in items:
col = BLUE if st == "rec" else ORANGE
mk = {"abs": "D", "res": "o", "rec": "s"}[st]
fc = "white" if st == "abs" else col
ax.plot([lo, hi], [y, y], color=col, lw=LW, solid_capstyle="round", zorder=2)
ax.plot(o, y, mk, ms=MS, mfc=fc, mec=col, mew=LW, zorder=3)
ax.text(1.232, y, fmt((o, lo, hi)), va="center", fontsize=10, color=INK)
ax.set_yticks([2.95, 1.05])
ax.set_yticklabels(["Prolonged\ninitial contact", "Persistent\ncontact"], fontsize=10.5, color=INK)
ax.set_ylim(0.4, 3.75)
ax.set_xlabel("Odds ratio per SD of PGS-ADHD vs brief contact (log scale)")
ax.text(0.995, 3.68, "← decreased odds", ha="right", va="top", fontsize=10, color=INK2)
ax.text(1.005, 3.68, "increased odds →", ha="left", va="top", fontsize=10, color=INK2)
h = [Line2D([], [], color=ORANGE, marker="D", mfc="white", ms=MS, lw=LW, mew=LW, label="Abstract (printed)"),
Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="Results text and STable 2 (printed)"),
Line2D([], [], color=BLUE, marker="s", ms=MS, lw=LW, label="1/OR of Results values (recomputed)")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.22), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_B")
# ------------------------------------------------------------------ C: source-matching diagram
def cand_C():
fig = plt.figure(figsize=(W, 4.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off()
ax.set_xlim(0, 100); ax.set_ylim(0, 56)
ax.text(2, 53, "Abstract, as printed", fontsize=11.5, weight="bold", va="top")
ax.text(58, 53, "Supplementary Table 2, as printed", fontsize=11.5, weight="bold", va="top")
left = [("PGS-ADHD, prolonged", A_PRO, 40), ("PGS-ADHD, persistent", A_PER, 28), ("PGS-AN, persistent", ABSTRACT["AN, persistent contact"], 16)]
right = {("Prolonged initial contact", "ADHD"): 44, ("Persistent contact", "ADHD"): 35,
("Prolonged initial contact", "AN"): 23, ("Persistent contact", "AN"): 14}
for (c, g), y in right.items():
v = ST2[(c, g)]
ax.add_patch(FancyBboxPatch((58, y - 3.2), 40, 6.4, boxstyle="round,pad=0,rounding_size=0.8",
fc="white", ec=INK2, lw=0.8))
ax.text(60, y + 0.9, f"PGS-{g}, {c.split()[0].lower()}", fontsize=10, color=INK2, va="center")
ax.text(60, y - 1.5, f"OR {v[0]:.2f} [{v[3]:.2f}–{v[4]:.2f}]", fontsize=10.5, color=ORANGE, weight="bold", va="center")
for lab, t, y in left:
ax.add_patch(FancyBboxPatch((2, y - 3.2), 36, 6.4, boxstyle="round,pad=0,rounding_size=0.8",
fc="white", ec=INK2, lw=0.8))
ax.text(4, y + 0.9, lab, fontsize=10, color=INK2, va="center")
ax.text(4, y - 1.5, f"OR {fmt(t)}", fontsize=10.5, color=ORANGE, weight="bold", va="center")
# identical-value links
for yl, yr in ((40, 23), (28, 14), (16, 14)):
ax.annotate("", xy=(57.6, yr), xytext=(38.4, yl),
arrowprops=dict(arrowstyle="-|>", color=INK, lw=LW, shrinkA=0, shrinkB=0))
# rows with the same label
for yl, yr in ((40, 44), (28, 35)):
ax.annotate("", xy=(57.6, yr), xytext=(38.4, yl),
arrowprops=dict(arrowstyle="-", color=INK2, lw=1.0, ls=(0, (3, 2)), shrinkA=0, shrinkB=0))
h = [Line2D([], [], color=INK, lw=LW, label="identical OR and both CI bounds"),
Line2D([], [], color=INK2, lw=1.0, ls=(0, (3, 2)), label="same PGS and class label")]
ax.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, -0.01), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_C")
# ------------------------------------------------------------------ D: forest of STable 2 with abstract overlay
def cand_D():
order = ["MDD", "ADHD", "ASD", "BD", "SCZ", "AN"]
classes = ["Prolonged initial contact", "Persistent contact"]
fig, axs = plt.subplots(1, 2, figsize=(W, 3.9), sharey=True)
for ax, c in zip(axs, classes):
log_axis(ax, 0.80, 1.26, ticks=(0.8, 0.9, 1.0, 1.1, 1.2))
for i, g in enumerate(order):
y = len(order) - i
o, _, p, lo, hi = ST2[(c, g)]
ax.plot([lo, hi], [y, y], color=ORANGE, lw=LW, solid_capstyle="round", zorder=2)
ax.plot(o, y, "o", ms=MS, color=ORANGE, zorder=3)
a = A_PRO if c.startswith("Prolonged") else A_PER
yA = len(order) - order.index("ADHD") + 0.38
ax.plot([a[1], a[2]], [yA, yA], color=ORANGE, lw=LW, ls=(0, (2, 1.5)), zorder=2)
ax.plot(a[0], yA, "D", ms=MS, mfc="white", mec=ORANGE, mew=LW, zorder=3)
yAN = len(order) - order.index("AN")
ax.annotate("", xy=(a[0], yAN + 0.22), xytext=(a[0], yA - 0.2),
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.0, ls=(0, (3, 2))))
ax.set_title(c, fontsize=10.5, color=INK)
ax.set_xlabel("OR per SD vs brief contact (log)")
axs[0].set_yticks(range(len(order), 0, -1))
axs[0].set_yticklabels([f"PGS-{g}" for g in order], fontsize=10, color=INK)
axs[0].set_ylim(0.4, 6.9)
h = [Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="STable 2 (printed)"),
Line2D([], [], color=ORANGE, marker="D", mfc="white", ms=MS, lw=LW, ls=(0, (2, 1.5)), mew=LW,
label="Abstract's PGS-ADHD value (printed)"),
Line2D([], [], color=INK2, lw=1.0, ls=(0, (3, 2)), label="lands on the PGS-AN row")]
fig.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, -0.07), ncol=3, frameon=False, fontsize=10)
fig.tight_layout(rect=(0, 0.06, 1, 1))
save(fig, "cand_D")
# ------------------------------------------------------------------ E: p-value strip
def cand_E():
fig, ax = plt.subplots(figsize=(W, 3.2))
rows = [
("Prolonged, Abstract\n1.06 [1.01–1.11]", None, abs_calc["ADHD, prolonged initial contact"][2]),
("Prolonged, Results\n0.91 [0.87–0.96]", R_PRO[3], res_calc["ADHD, prolonged initial contact"][2]),
("Persistent, Abstract\n1.12 [1.03–1.21]", None, abs_calc["ADHD, persistent contact"][2]),
("Persistent, Results\n0.90 [0.83–0.97]", R_PER[3], res_calc["ADHD, persistent contact"][2]),
]
ax.set_xscale("log"); ax.set_xlim(5e-5, 0.2)
ax.set_xticks([1e-4, 1e-3, 1e-2, 1e-1]); ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.1"]); ax.minorticks_off()
ax.grid(axis="x", color=GRID, lw=0.6); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.axvline(ALPHA_BONF, color=INK2, lw=0.8, ls=(0, (4, 2)))
ax.axvline(0.05, color=INK2, lw=0.8, ls=(0, (1, 2)))
ax.text(ALPHA_BONF / 1.08, 4.62, "Bonferroni α 0.0083", fontsize=10, color=INK2, va="top", ha="right")
ax.text(0.05 * 1.08, 4.62, "0.05", fontsize=10, color=INK2, va="top")
for i, (lab, pr, pc) in enumerate(rows):
y = len(rows) - i
if pr is not None:
ax.plot(pr, y, "o", ms=MS, color=ORANGE, zorder=3)
ax.plot(pc, y, "D", ms=MS, color=BLUE, zorder=4, mec="white", mew=1.0)
ax.text(max(pc, pr or 0) * 1.35, y - 0.05, f"{pc:.4f}", fontsize=10, va="center", color=INK)
ax.set_yticks(range(len(rows), 0, -1)); ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_ylim(0.4, 4.7)
ax.set_xlabel("Two-sided p for PGS-ADHD vs brief contact (log scale)")
h = [Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=0, label="p printed in Results"),
Line2D([], [], color=BLUE, marker="D", ms=MS, lw=0, label="p recomputed from the printed CI")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.25), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_E")
# Which values each design draws, so the Figure 1 caption can name them correctly
CAPTION_KEY = {
"A": "All numbers are copied from the paper (orange: Abstract and Results text as printed); no recomputed values are drawn.",
"B": "Orange: odds ratios and 95% CIs as printed (Abstract; Results text and STable 2). Blue: reciprocals of the Results values, recomputed here to show what a swapped reference category would give.",
"C": "All numbers are copied from the paper (orange: Abstract and STable 2 as printed); connectors mark identical OR-CI triples and shared labels.",
"D": "All numbers are copied from the paper (orange: STable 2 rows, and the Abstract's PGS-ADHD values as open diamonds).",
"E": "Orange: p values as printed in the Results text (the Abstract prints none). Blue: two-sided p recomputed from each printed 95% CI.",
}
print(f"Figure 1 caption key for candidate {FIG1}: {CAPTION_KEY[FIG1]}")
for f in (cand_A, cand_B, cand_C, cand_D, cand_E):
f()
for ext in ("svg", "png"):
shutil.copyfile(os.path.join(CAND, f"cand_{FIG1}.{ext}"), os.path.join(HERE, f"fig1.{ext}"))
print(f"figures written to {CAND}; fig1 = candidate {FIG1}")
Cohort 2 is reported as 361,258 participants in Table 2 but 32,220 in the flowchart
Psychol Med (2026) · doi:10.1017/S0033291726103328 · PMC12902173
Summary
The paper analyses two matched cohorts from Taiwan's one-million-person Longitudinal Health Insurance Database (LHID): psychological distress (PD, meaning depression or anxiety) followed for incident IBS (Cohort 1), and IBS followed for incident PD (Cohort 2). For Cohort 2, the Results text and Table 2 describe 361,258 participants (180,629 per arm), while the sample-selection flowchart (Figure 1) gives a final follow-up N of 32,220 (16,110 per arm), 11.2 times smaller. The paper's own follow-up event percentages (2,031 = 12.6% and 1,308 = 8.1%) fit only the flowchart's 16,110 per arm, and the width of the headline hazard-ratio interval fits the roughly 3,300 events in that smaller sample. Table 2 therefore cannot describe the cohort that was analysed. It also reports 45.77% of the IBS arm "With PD" at baseline in a cohort defined by excluding prior PD. Cohort 1 shows the same pattern (Table 1: 234,361 per arm; flowchart: 18,402 per arm).
What the paper reports
Results, "Baseline characteristics: Cohort 2 (IBS predicting psychological distress)":
Table 2 summarizes the baseline characteristics of 361,258 participants, divided evenly into IBS (n = 180,629) and non-IBS controls (n = 180,629).
Table 2, column headers and the first block (rows "Without PD" and "With PD"):
Variable | IBS(n = 180,629) | Without IBS(n = 180,629) | p-value
Without PD | 97,948 (54.23%) | 148,971 (82.47%) |
With PD | 82,681 (45.77%) | 31,658 (17.53%) |
Results, the same section, "Psychological distress outcomes":
The prevalence of psychological distress was substantially higher among individuals with IBS (45.77%) compared with controls (17.53%, p < .001).
Methods, "Cohort 2: IBS predicting psychological distress", on the definition of the cohort:
Those with any depressive or anxiety disorder before the index date were excluded to ensure incident outcome assessment.
A consolidated flowchart including all exclusions (prior IBS, prior psychological distress, and missing data) and final analytic sample sizes is shown in Figure 1.
Figure 1 flowchart, Cohort 2 (list items, rendered as text in PMC and in the publisher's HTML):
Newly diagnosed IBS cases (2002–2012): 16,110
Excluded prior depressive or anxiety disorders: 3,231 excluded
IBS cohort: 16,110
Matched controls: 16,110 Final follow-up N for PD outcomes: 32,220
Results, "Cohort 2: IBS predicting incident depression or anxiety":
In the reverse analysis, during a mean follow-up of 6.9 years, 2,031 IBS patients (12.6%) developed either depression or anxiety, compared to 1,308 (8.1%) in the control group.
Cohort 1 has the same kind of discrepancy. Results, "Baseline characteristics: Cohort 1":
A total of 468,722 participants were included, with 234,361 individuals in each group after matching.
Figure 1 flowchart, Cohort 1:
Newly diagnosed depressive/anxiety disorders (2002–2012): 18,402
Psychological distress group: 18,402
Matched controls: 18,402 Final follow-up N for IBS outcome: 36,804
Results, Cohort 1 follow-up, and the Cohort 1 baseline IBS prevalence:
During a mean follow-up period of 7.3 years, 1,498 individuals (8.1%) in the psychological distress group developed IBS, compared to 921 (5.0%) in the control group.
The prevalence of IBS at baseline was significantly higher in the psychological distress group compared to controls (32.11% vs. 12.04%, p < .001)
The Cohort 1 definition (Methods) and the Discussion:
Individuals with a prior diagnosis of IBS (ICD-9-CM: 564.1) before the index date were excluded to ensure incident outcome assessment.
Our findings apply only to incident cases, as individuals with any history of IBS or prior depressive/anxiety disorders were excluded from both cohorts.
The large sample enabled robust subgroup and interaction analyses.
What the paper's own numbers show
Method. I took every sample size, count and percentage for both cohorts from the Results text, the Figure 1 flowchart text, Tables 1–4 and the Methods description of the LHID. I checked (i) each stated total against twice the per-arm N; (ii) that each block of Table 2 sums to its column header; (iii) which denominators reproduce the printed follow-up event percentages, by finding every per-arm N for which the printed event count divided by N rounds to the printed percentage; (iv) the flowchart's own subtraction; (v) the number of events implied by the width of each overall adjusted hazard ratio's 95% CI. For (v) I used the standard approximation for a 1:1 comparison, var(log HR) ≈ 4/D, and also computed the interval expected from the reported event counts, SE ≈ √(1/d₁ + 1/d₂), and the interval expected if the Table N had produced events at the printed rates. Finally, I compared the cohort sizes with the size of the LHID. Patient-level data are not available, so the models cannot be refitted. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Cohort 2 total N | Results text; Table 2 headers | 361,258 (180,629 per arm) | 180,629 × 2 = 361,258, internally consistent |
| Cohort 2 final follow-up N | Figure 1 flowchart | 32,220 (16,110 per arm) | 16,110 × 2 = 32,220, internally consistent |
| Ratio of the two Cohort 2 totals | — | — | 361,258 / 32,220 = 11.21; difference 329,038 (164,519 per arm) |
| Table 2 block sums (PD status, sex, age group, urbanization, income) | Table 2 | n = 180,629 per column | every block sums to 180,629 in both columns |
| Depression + anxiety = "With PD" | Table 2 | 82,681 and 31,658 | 15,806 + 66,875 = 82,681; 7,339 + 24,319 = 31,658 |
| IBS-arm follow-up events | Results | 2,031 (12.6%) | 2,031 / 16,110 = 12.61%; 2,031 / 180,629 = 1.12% |
| Control-arm follow-up events | Results | 1,308 (8.1%) | 1,308 / 16,110 = 8.12%; 1,308 / 180,629 = 0.72% |
| Per-arm N that reproduces both printed percentages | derived | — | 16,056 to 16,183: contains 16,110, excludes 180,629 |
| IBS cohort after excluding prior PD | Figure 1 flowchart | 16,110 new cases, 3,231 excluded, cohort 16,110 | 16,110 − 3,231 = 12,879, not 16,110 |
| IBS arm "With PD" at baseline | Table 2; Results | 82,681 (45.77%) | 82,681 / 180,629 = 45.77%, in a cohort that excluded prior PD; 25.6 times the flowchart's 3,231 exclusions and 40.7 times the 2,031 follow-up events |
| Cohort 2 aHR 1.46 (1.37–1.56): events implied by CI width | Results; Table 4 | — | SE(log HR) = 0.0331; 4/SE² ≈ 3,644 events (3,288–4,061 allowing for rounding of the limits); reported events 2,031 + 1,308 = 3,339, inside that range |
| CI expected from 3,339 events | derived | 1.37–1.56 | 1.36–1.57 (matches) |
| CI expected if 180,629 per arm had the printed rates | derived | — | 22,759 + 14,631 = 37,390 events; CI 1.43–1.49 (does not match; the reported interval is 3.1 times as wide) |
| Cohort 1 total N | Results text; Table 1 headers | 468,722 (234,361 per arm) | 234,361 × 2 = 468,722 |
| Cohort 1 final follow-up N | Figure 1 flowchart | 36,804 (18,402 per arm) | 18,402 × 2 = 36,804; ratio 468,722 / 36,804 = 12.74 |
| Cohort 1 follow-up events | Results | 1,498 (8.1%) and 921 (5.0%) | /18,402 = 8.14% and 5.00%; /234,361 = 0.64% and 0.39%; N reproducing both: 18,381–18,606 |
| Cohort 1 aHR 1.55 (1.42–1.69): events implied by CI width | Results; Table 3 | — | ≈ 2,028 (1,885–2,188), slightly fewer than the 2,419 reported (1,498 + 921), i.e. the reported interval is a little wider than the events alone imply; both are far from the 30,701 implied by the Table N. CI from 2,419 events 1.43–1.68; from Table N at printed rates (30,701 events) 1.51–1.59, 3.8 times narrower than reported |
| PD arm with IBS at baseline | Table 1; Results | 75,244 (32.11%) | 75,244 / 234,361 = 32.11%, in a cohort that excluded prior IBS |
| Adults in the LHID | Methods | 1,000,000 − 1,982 − 2,614 | 995,404 |
| Cohort 1 eligible population | Figure 1 flowchart | "990,0xx" | 995,404 − 5,412 = 989,992, which is not of the form 990,0xx either |
| Exposed arms of both cohorts | Tables 1 and 2 | 234,361 + 180,629 | 414,990 = 41.7% of LHID adults |
| Exposed arms of both cohorts | Figure 1 flowchart | 18,402 + 16,110 | 34,512 = 3.5% of LHID adults |
The two exposed groups essentially cannot share people. Cohort 1 excludes anyone with IBS before the PD diagnosis, and Cohort 2 excludes anyone with PD before the IBS diagnosis, so each person can be exposed in at most one cohort (apart from anyone first diagnosed with both on the same date). Taken at face value, Tables 1 and 2 would mean that 41.7% of all adults in the database received a first diagnosis of PD or of IBS between 2002 and 2012, with the other condition absent beforehand. The flowchart's figures correspond to 3.5%.
Both totals for each cohort are internally consistent. Table 2's blocks all sum to 180,629, and the flowchart's numbers multiply out to 32,220. The two descriptions cannot both refer to the same matched sample. The follow-up percentages and the CI widths fit the flowchart, so the survival analyses appear to have been run on about 16,110 patients per arm in Cohort 2 (and about 18,402 in Cohort 1). It is not stated what population Tables 1 and 2 describe.
Figure
Figure 1. Participants per arm for each cohort on one linear axis starting at zero. Orange markers are printed in the paper: circles are the Table and Results-text values (234,361 for Cohort 1, 180,629 for Cohort 2), and squares are the Figure 1 flowchart values (18,402 and 16,110). The grey bar and the label on it give the ratio between the two printed values (12.7 and 11.2). The blue diamond marks the range of per-arm N (narrower than the marker at this scale) for which both printed follow-up percentages are reproduced from the printed event counts (Cohort 2: 2,031 at 12.6% and 1,308 at 8.1% give 16,056–16,183; Cohort 1: 1,498 at 8.1% and 921 at 5.0% give 18,381–18,606), computed by recompute.py.txt. In both cohorts this range contains the flowchart value and excludes the Table value.
Figure 2. Overall adjusted hazard ratios on one log-scale axis; the vertical line marks HR = 1. Orange circles are the reported aHRs and 95% CIs (Table 4: 1.46, 1.37–1.56; Table 3: 1.55, 1.42–1.69). Filled blue diamonds are the intervals expected from the reported numbers of events (3,339 and 2,419), with SE(log HR) = √(1/d₁ + 1/d₂); they agree closely with the reported intervals (for Cohort 1 the reported interval is slightly wider than 2,419 events alone imply, which corresponds to about 2,028 events). Open blue diamonds are the intervals expected if the Table sample sizes had produced events at the printed rates (37,390 and 30,701 events); they are about three to four times narrower than reported (3.1x for Cohort 2, 3.8x for Cohort 1, measured as SE(log HR) and identical for the linear width). The reported precision is what the flowchart-sized cohorts would give, not what the Table-sized cohorts would give.
Why it matters
Cohort 2 is one of the paper's two primary analyses. Its sample size is given in the Results text and in Table 2 as 361,258, about eleven times the 32,220 in the flowchart that the Methods identify as showing the "final analytic sample sizes". The follow-up events and the precision of the hazard ratio fit the smaller number, so the hazard ratios were most likely estimated on about 32,000 people. Their direction and statistical significance are probably not affected by the misstated N.
What is affected is the description of the study population. Table 2, the Results paragraphs that summarize it ("well matched", the income and comorbidity contrasts, the 45.77% vs 17.53% PD prevalence) and the corresponding Table 1 material for Cohort 1 cannot describe the analysed cohorts. They report N values 11–13 times too large and a large prevalence of the very condition that each cohort was defined to exclude: 82,681 IBS patients "With PD" in Cohort 2 and 75,244 PD patients "With IBS" in Cohort 1. A reader cannot use these tables to judge baseline balance, comorbidity burden or generalizability, and the Discussion's "The large sample enabled robust subgroup and interaction analyses" rests in part on the larger figures. Anyone extracting the sample size for a systematic review or meta-analysis would take 361,258 (or 468,722) from the text, overstating the size of these cohorts eleven- to thirteen-fold.
The flowchart has its own problem. It lists 16,110 newly diagnosed IBS cases, then 3,231 exclusions for prior PD, and then an IBS cohort of 16,110 again, where 12,879 would be expected. So it is not clear whether 16,110 is the number before or after the exclusion.
A separate inconsistency adds to the uncertainty. The abstract gives "IBS was associated with increased risks of depression (adjusted hazard ratio [aHR] = 1.55) and anxiety (aHR = 1.68)", but 1.55 and 1.68 are the overall and female HRs for the opposite direction (PD → IBS) in Table 3. The body reports no depression- or anxiety-specific HRs, and the abstract's 1.45 and 1.51 for "depression and anxiety ... developing IBS" do not appear as point estimates anywhere in the body. Table 4's overall HR for IBS → PD is 1.46.
Explanations considered
- Table 2 describes the population before exclusions or before matching. This does not account for the numbers. Table 2 shows identical sex, age, age-group and urbanization counts in both columns, each with p = 1.000, so it describes a 1:1 matched sample, the same stage as the flowchart's "After matching 1:1 with controls". The Results text also says the participants were "divided evenly", and the Cohort 1 text says "234,361 individuals in each group after matching".
- Table 2 comes from a larger database (for example the full NHIRD rather than the 1-million-person LHID). Nothing in the paper says so. The Methods state that the cohorts were drawn from the LHID, a one-million-person subset of the NHIRD (as does Supplementary Table S1), and no table note gives another source. Even so, the event percentages in the Results fit only the flowchart denominators, so the paper's own analyses use the smaller cohorts.
- The rows count visits, claims or person-years rather than people. This does not account for the numbers. The rows are counts of people by sex, age group, urbanization level and income, each block sums exactly to the column N, and the percentages are of that N.
- The follow-up percentages use a different denominator from Table 2 on purpose. Nothing in the paper says so. The text gives "2,031 IBS patients (12.6%)" with no other denominator, and 12.6% is 2,031/16,110. A different denominator would mean that the analysed cohort is not the one in Table 2, which is the inconsistency itself.
- The "With PD" row in Table 2 (and "With IBS" in Table 1) reports outcomes during follow-up, not baseline status. This does not resolve the problem either. The Results place these figures under "Baseline characteristics" and call the Table 1 figure "The prevalence of IBS at baseline". Read as outcomes, 82,681 would contradict the 2,031 events reported for the same arm (40.7 times larger).
- The flowchart is wrong and the Tables are right. This is not consistent with the rest of the paper. The event percentages and the width of the hazard-ratio intervals both fit the flowchart sizes. With 180,629 per arm and events at the printed rates, the interval would be about 1.43–1.49, not 1.37–1.56. The Table sizes would also mean that 41.7% of the database's adults were incident cases of one condition without the other.
- A conversion or typesetting artifact in PMC. This is excluded. The publisher's HTML on cambridge.org and the Europe PMC JATS XML carry the same numbers: 361,258 and 180,629 in the text and Table 2 headers, and 16,110 / 3,231 / 16,110 / 16,110 / 32,220 in the flowchart.
- The supplement clarifies the cohorts. It does not (see Status). It gives no cohort sizes, and the "complete flow diagram (Figure S1)" that the supplementary text refers to is a cumulative-incidence plot.
Status
- Version of record. The publisher's HTML article page (doi.org redirects to cambridge.org/core) carries the same Results text ("361,258 participants, divided evenly into IBS (n = 180,629) and non-IBS controls (n = 180,629)"), the same flowchart text (16,110 / 3,231 excluded / 16,110 / 16,110 / 32,220) and the same follow-up sentence (2,031, 12.6%; 1,308, 8.1%). On that page Table 2 is an image. The Europe PMC full-text XML has Table 2 headers "IBS(n = 180,629)" and "Without IBS(n = 180,629)" and the same flowchart items. Every quote above was confirmed verbatim against the PMC-archived text.
- Supplements. One supplementary file (doi:10.1017/S0033291726103328.sm001, a .docx) was downloaded and read. It holds Supplementary Table S1 (LHID sampling and attrition: 1,000,000 sampled, 1,982 excluded for missing demographics, 2,614 under 18), Tables S2–S3 (interaction aHRs, no Ns) and Figures S1–S5 (cumulative-incidence curves without numbers at risk, and forest plots). It gives no cohort sizes. Its text refers to a "complete flow diagram (Figure S1)", but Figure S1 is a cumulative-incidence plot. Data are available from the authors on request.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI. PMC and Europe PMC (PMID 41667938) list no correction or comment links. The Cambridge article page shows no correction notice, and the article page's metadata lists no corrections, retractions or addenda. A web search for the title with "correction", "erratum" or "PubPeer" returned nothing relevant. PubPeer could not be checked directly: its API returned HTTP 403 to automated access.
- Sources. PMC article PMC12902173; publisher page at cambridge.org (doi:10.1017/S0033291726103328); supplementary file S0033291726103328sup001.docx; Europe PMC full-text XML; Crossref; PubPeer (not reachable).
Suggested correction
- State which population Tables 1 and 2 describe. If the analysed cohorts are those in Figure 1 (18,402 and 16,110 per arm, as the event percentages and hazard-ratio precision indicate), replace Tables 1 and 2 and the Results paragraphs that summarize them with the baseline characteristics of those cohorts, and correct "468,722" / "234,361" and "361,258" / "180,629" in the Results text.
- Explain why Tables 1 and 2 show 75,244 PD patients with IBS and 82,681 IBS patients with PD at baseline in cohorts defined by excluding those conditions, or remove these rows if they do not describe baseline status.
- Reconcile the Cohort 2 flowchart: 16,110 newly diagnosed IBS cases minus 3,231 exclusions should not leave an IBS cohort of 16,110. Give the counts before and after exclusion, and give the Cohort 1 eligible population as a number instead of "990,0xx" (995,404 − 5,412 = 989,992, which does not fit "990,0xx" either).
- Correct the abstract's hazard ratios so that each one matches a reported estimate in the body for the stated direction and outcome (for IBS → PD, Table 4 gives 1.46 overall), or add the depression- and anxiety-specific estimates to the Results.
- Reword statements that rest on the larger sample sizes, such as "The large sample enabled robust subgroup and interaction analyses", if the analysed cohorts are the smaller ones.
Recomputation script (python recompute.py.txt)
"""Recompute the cohort sizes of PMC12902173
(Psychol Med 2026, doi:10.1017/S0033291726103328).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file, plus five design candidates in candidates/.
All inputs are copied from the article as archived in PubMed Central (the same numbers are
in the publisher's HTML and the Europe PMC XML): Methods (data source and the Figure 1
flowchart text), Results, the Abstract and Tables 1-4. No patient-level data are used.
"""
import math
import os
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ------------------------------------------------------------------ inputs (from the paper)
LHID = 1_000_000 # Methods, data source
MISSING_DEMO = 1_982 # Methods / flowchart
UNDER_18 = 2_614 # Methods / flowchart
PRIOR_IBS_C1 = 5_412 # flowchart, Cohort 1
# Cohort 2: IBS -> psychological distress (PD)
C2_TABLE_ARM = 180_629 # Table 2 headers and Results text ("divided evenly")
C2_TEXT_TOTAL = 361_258 # Results text
C2_FC_NEW = 16_110 # flowchart: newly diagnosed IBS cases
C2_FC_EXCL = 3_231 # flowchart: excluded prior depressive or anxiety disorders
C2_FC_ARM = 16_110 # flowchart: IBS cohort / matched controls
C2_FC_TOTAL = 32_220 # flowchart: final follow-up N for PD outcomes
C2_EVENTS = (2_031, 1_308) # Results: IBS group, control group
C2_EVENT_PCT = (12.6, 8.1) # Results
C2_AHR = (1.46, 1.37, 1.56) # Results / Table 4, overall
TABLE2 = { # (IBS, without IBS) counts per block of Table 2
"Psychological distress (without PD, with PD)": ((97_948, 82_681), (148_971, 31_658)),
"Gender (male, female)": ((91_509, 89_120), (91_509, 89_120)),
"Age group (18-24, 25-44, 45-64, >=65)": ((17_211, 67_837, 66_504, 29_077),) * 2,
"Urbanization (levels 1-4)": ((55_995, 68_639, 28_900, 27_095),) * 2,
"Monthly income (3 bands)": ((43_194, 85_106, 52_329), (99_860, 55_902, 24_867)),
}
T2_WITH_PD = (82_681, 31_658)
T2_PD_PCT = (45.77, 17.53)
T2_DEP_ANX = ((15_806, 66_875), (7_339, 24_319))
# Cohort 1: PD -> IBS
C1_TABLE_ARM = 234_361 # Table 1 headers and Results text
C1_TEXT_TOTAL = 468_722
C1_FC_ARM = 18_402 # flowchart
C1_FC_TOTAL = 36_804
C1_EVENTS = (1_498, 921)
C1_EVENT_PCT = (8.1, 5.0)
C1_AHR = (1.55, 1.42, 1.69) # Results / Table 3, overall
T1_WITH_IBS = (75_244, 28_210)
T1_IBS_PCT = (32.11, 12.04)
TABLE3 = {"Overall": 1.55, "Female": 1.68, "Male": 1.39}
TABLE4 = {"Overall": 1.46, "Female": 1.52, "Male": 1.38}
ABSTRACT_HR = {"IBS -> depression": 1.55, "IBS -> anxiety": 1.68,
"depression -> IBS": 1.45, "anxiety -> IBS": 1.51}
Z = 1.959964
def pct(a, b, d=2):
return round(100 * a / b, d)
def n_range(events, pct_printed, dec=1):
"""Denominators N for which 100*events/N rounds to pct_printed at `dec` decimals."""
h = 0.5 * 10 ** (-dec)
return math.floor(events / ((pct_printed + h) / 100)) + 1, math.floor(events / ((pct_printed - h) / 100))
def se_from_ci(lo, hi):
return (math.log(hi) - math.log(lo)) / (2 * Z)
# ------------------------------------------------------------------ 1. Cohort 2 sizes
print("1. Cohort 2 (IBS -> psychological distress)")
print(f" Table 2 / text: {C2_TABLE_ARM:,} x 2 = {2 * C2_TABLE_ARM:,} (text says {C2_TEXT_TOTAL:,})")
print(f" Flowchart: {C2_FC_ARM:,} x 2 = {2 * C2_FC_ARM:,} (flowchart says {C2_FC_TOTAL:,})")
c2_ratio = C2_TEXT_TOTAL / C2_FC_TOTAL
print(f" Ratio text/flowchart = {c2_ratio:.2f}; difference = {C2_TEXT_TOTAL - C2_FC_TOTAL:,} "
f"({C2_TABLE_ARM - C2_FC_ARM:,} per arm)")
c2_after_excl = C2_FC_NEW - C2_FC_EXCL
print(f" Flowchart internal: {C2_FC_NEW:,} new IBS - {C2_FC_EXCL:,} excluded = {c2_after_excl:,}, "
f"but IBS cohort printed as {C2_FC_ARM:,}")
print(" Table 2 block sums (IBS column, control column):")
for k, (a, b) in TABLE2.items():
print(f" {k}: {sum(a):,}, {sum(b):,}")
print(f" With PD: {T2_WITH_PD[0]:,}/{C2_TABLE_ARM:,} = {pct(T2_WITH_PD[0], C2_TABLE_ARM)}%, "
f"{T2_WITH_PD[1]:,}/{C2_TABLE_ARM:,} = {pct(T2_WITH_PD[1], C2_TABLE_ARM)}% "
f"(printed {T2_PD_PCT[0]}% and {T2_PD_PCT[1]}%)")
for lab, (d, a), wp in zip(("IBS", "control"), T2_DEP_ANX, T2_WITH_PD):
print(f" {lab}: depression + anxiety = {d:,} + {a:,} = {d + a:,} (= 'With PD' {wp:,}); "
f"depression share {pct(d, wp)}%")
print(" Follow-up events:")
c2_pct_fc, c2_pct_tab = [], []
for lab, e, rp in zip(("IBS", "control"), C2_EVENTS, C2_EVENT_PCT):
a, b = 100 * e / C2_FC_ARM, 100 * e / C2_TABLE_ARM
c2_pct_fc.append(a); c2_pct_tab.append(b)
lo, hi = n_range(e, rp)
print(f" {lab}: {e:,}/{C2_FC_ARM:,} = {a:.2f}%; {e:,}/{C2_TABLE_ARM:,} = {b:.2f}% "
f"(printed {rp}%); N giving {rp}% after rounding: {lo:,}-{hi:,}")
c2_lo = max(n_range(e, p)[0] for e, p in zip(C2_EVENTS, C2_EVENT_PCT))
c2_hi = min(n_range(e, p)[1] for e, p in zip(C2_EVENTS, C2_EVENT_PCT))
print(f" Both percentages hold only for N per arm in {c2_lo:,}-{c2_hi:,}: contains "
f"{C2_FC_ARM:,}, excludes {C2_TABLE_ARM:,}")
print(f" Prevalent PD at baseline in a cohort that excluded prior PD: {T2_WITH_PD[0]:,} "
f"vs {C2_FC_EXCL:,} excluded in the flowchart ({T2_WITH_PD[0] / C2_FC_EXCL:.1f}x); "
f"vs {C2_EVENTS[0]:,} follow-up events ({T2_WITH_PD[0] / C2_EVENTS[0]:.1f}x)")
# ------------------------------------------------------------------ 2. Cohort 1 sizes
print("\n2. Cohort 1 (psychological distress -> IBS)")
print(f" Table 1 / text: {C1_TABLE_ARM:,} x 2 = {2 * C1_TABLE_ARM:,} (text says {C1_TEXT_TOTAL:,})")
print(f" Flowchart: {C1_FC_ARM:,} x 2 = {2 * C1_FC_ARM:,} (flowchart says {C1_FC_TOTAL:,})")
c1_ratio = C1_TEXT_TOTAL / C1_FC_TOTAL
print(f" Ratio text/flowchart = {c1_ratio:.2f}")
c1_pct_fc, c1_pct_tab = [], []
for lab, e, rp in zip(("PD", "control"), C1_EVENTS, C1_EVENT_PCT):
a, b = 100 * e / C1_FC_ARM, 100 * e / C1_TABLE_ARM
c1_pct_fc.append(a); c1_pct_tab.append(b)
lo, hi = n_range(e, rp)
print(f" {lab}: {e:,}/{C1_FC_ARM:,} = {a:.2f}%; {e:,}/{C1_TABLE_ARM:,} = {b:.2f}% "
f"(printed {rp}%); N giving {rp}% after rounding: {lo:,}-{hi:,}")
c1_lo = max(n_range(e, p)[0] for e, p in zip(C1_EVENTS, C1_EVENT_PCT))
c1_hi = min(n_range(e, p)[1] for e, p in zip(C1_EVENTS, C1_EVENT_PCT))
print(f" Both percentages hold only for N per arm in {c1_lo:,}-{c1_hi:,}: contains "
f"{C1_FC_ARM:,}, excludes {C1_TABLE_ARM:,}")
print(f" With IBS at baseline (Table 1): {T1_WITH_IBS[0]:,}/{C1_TABLE_ARM:,} = "
f"{pct(T1_WITH_IBS[0], C1_TABLE_ARM)}%, {T1_WITH_IBS[1]:,}/{C1_TABLE_ARM:,} = "
f"{pct(T1_WITH_IBS[1], C1_TABLE_ARM)}% (printed {T1_IBS_PCT[0]}%, {T1_IBS_PCT[1]}%), in a "
f"cohort that excluded prior IBS (flowchart: {PRIOR_IBS_C1:,} excluded)")
# ------------------------------------------------------------------ 3. source population
print("\n3. Source population (LHID)")
adults = LHID - MISSING_DEMO - UNDER_18
elig_c1 = adults - PRIOR_IBS_C1
print(f" {LHID:,} - {MISSING_DEMO:,} - {UNDER_18:,} = {adults:,} adults with complete data")
print(f" minus {PRIOR_IBS_C1:,} prior IBS = {elig_c1:,} (flowchart prints '990,0xx'; "
f"{elig_c1:,} is {'' if 990_000 <= elig_c1 < 990_100 else 'not '}of that form)")
exp_tab = C1_TABLE_ARM + C2_TABLE_ARM
exp_fc = C1_FC_ARM + C2_FC_ARM
print(f" Exposed arms, Tables 1+2: {C1_TABLE_ARM:,} + {C2_TABLE_ARM:,} = {exp_tab:,} = "
f"{pct(exp_tab, adults, 1)}% of adults")
print(f" Exposed arms, flowchart: {C1_FC_ARM:,} + {C2_FC_ARM:,} = {exp_fc:,} = "
f"{pct(exp_fc, adults, 1)}% of adults")
all_tab = C1_TEXT_TOTAL + C2_TEXT_TOTAL
print(f" All four Table arms: {C1_TEXT_TOTAL:,} + {C2_TEXT_TOTAL:,} = {all_tab:,} = "
f"{pct(all_tab, adults, 1)}% of adults (controls may overlap between cohorts)")
print(" The two exposed groups essentially cannot overlap: Cohort 1 excludes prior IBS and"
" Cohort 2 excludes prior PD, so a person is exposed in at most one of them (apart from"
" anyone first diagnosed with both on the same date).")
# ------------------------------------------------------------------ 4. CI width vs events
print("\n4. Number of events implied by the width of the overall aHR interval")
print(" Schoenfeld approximation for 1:1 allocation: var(log HR) = 4 / D")
ci_rows = []
for lab, (hr, lo, hi), ev, ev_pct, arm_tab in (
("Cohort 2 (Table 4)", C2_AHR, C2_EVENTS, C2_EVENT_PCT, C2_TABLE_ARM),
("Cohort 1 (Table 3)", C1_AHR, C1_EVENTS, C1_EVENT_PCT, C1_TABLE_ARM)):
se = se_from_ci(lo, hi)
se_min = se_from_ci(lo - 0.005, hi + 0.005)
se_max = se_from_ci(lo + 0.005, hi - 0.005)
d_impl = 4 / se ** 2
d_rep = sum(ev)
se_rep = math.sqrt(1 / ev[0] + 1 / ev[1])
ev_big = [p / 100 * arm_tab for p in ev_pct]
se_big = math.sqrt(1 / ev_big[0] + 1 / ev_big[1])
ci_rep = (hr * math.exp(-Z * se_rep), hr * math.exp(Z * se_rep))
ci_big = (hr * math.exp(-Z * se_big), hr * math.exp(Z * se_big))
print(f" {lab}: aHR {hr} ({lo}-{hi}); SE(log HR) = {se:.4f} "
f"(rounding range {se_max:.4f}-{se_min:.4f})")
print(f" implied events 4/SE^2 = {d_impl:,.0f} (range {4 / se_min ** 2:,.0f}-"
f"{4 / se_max ** 2:,.0f}); reported events {ev[0]:,} + {ev[1]:,} = {d_rep:,}")
print(f" 95% CI expected from the reported events (SE = sqrt(1/d1 + 1/d2) = {se_rep:.4f}): "
f"{ci_rep[0]:.2f}-{ci_rep[1]:.2f}")
print(f" if {arm_tab:,} per arm had the printed event rates: events "
f"{ev_big[0]:,.0f} + {ev_big[1]:,.0f} = {sum(ev_big):,.0f}; SE {se_big:.4f}; "
f"95% CI {ci_big[0]:.2f}-{ci_big[1]:.2f}")
d_lo, d_hi = 4 / se_min ** 2, 4 / se_max ** 2
inside = d_lo <= d_rep <= d_hi
print(f" reported events {d_rep:,} {'lie inside' if inside else 'lie outside'} the implied "
f"range {d_lo:,.0f}-{d_hi:,.0f}")
w_rep = hi - lo
w_big = ci_big[1] - ci_big[0]
print(f" reported interval vs interval expected from Table N: SE {se:.4f} / {se_big:.4f} "
f"= {se / se_big:.2f}x; linear width {w_rep:.2f} / {w_big:.4f} = {w_rep / w_big:.2f}x")
ci_rows.append((lab, hr, (lo, hi), ci_rep, ci_big, d_rep, sum(ev_big)))
# ------------------------------------------------------------------ 5. abstract HRs
print("\n5. Abstract hazard ratios vs Tables 3 and 4")
for k, v in ABSTRACT_HR.items():
t3 = [s for s, x in TABLE3.items() if x == v]
t4 = [s for s, x in TABLE4.items() if x == v]
print(f" Abstract {k} = {v}: Table 3 (PD -> IBS) rows {t3 or 'none'}; "
f"Table 4 (IBS -> PD) rows {t4 or 'none'}")
print(f" Table 4 overall (IBS -> PD) = {TABLE4['Overall']}; Table 3 overall (PD -> IBS) = "
f"{TABLE3['Overall']}. No depression- or anxiety-specific HRs are reported in the body.")
# ================================================================== figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e3e2de"
W = 7.2
LW = 1.5 # points; 1.5 pt = 2 px at 96 dpi
plt.rcParams.update({
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"font.size": 10, "axes.edgecolor": INK2, "axes.labelcolor": INK, "text.color": INK,
"xtick.color": INK2, "ytick.color": INK2, "axes.spines.top": False,
"axes.spines.right": False, "font.family": "DejaVu Sans", "svg.fonttype": "none",
})
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg", bbox_inches="tight", pad_inches=0.12)
fig.savefig(p + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
def wrap(s, w):
return "\n".join(textwrap.wrap(s, w))
def fmt(n):
return f"{n:,.0f}"
# ---------------------------------------------------------------- candidate A: N per arm dot plot
def cand_A():
"""Per-arm N: Tables/text vs flowchart (both printed), with the N range implied by the
printed event counts and percentages (recomputed)."""
fig, ax = plt.subplots(figsize=(W, 3.3))
rows = [("Cohort 1\nPD \u2192 IBS", C1_TABLE_ARM, C1_FC_ARM, (c1_lo, c1_hi), C1_EVENTS,
C1_EVENT_PCT, "Table 1"),
("Cohort 2\nIBS \u2192 PD", C2_TABLE_ARM, C2_FC_ARM, (c2_lo, c2_hi), C2_EVENTS,
C2_EVENT_PCT, "Table 2")]
ys = [1, 0]
for y, (lab, tab, fc, (lo, hi), ev, ep, tname) in zip(ys, rows):
ax.plot([fc, tab], [y, y], color=GRID, lw=4, zorder=1, solid_capstyle="round")
ax.scatter(tab, y, s=150, color=ORANGE, zorder=3, edgecolor="white", lw=0.8)
ax.scatter(fc, y, s=150, color=ORANGE, marker="s", zorder=3, edgecolor="white", lw=0.8)
ax.plot([lo, hi], [y - 0.2, y - 0.2], color=BLUE, lw=LW * 2, zorder=2)
ax.scatter((lo + hi) / 2, y - 0.2, s=70, color=BLUE, marker="D", zorder=3,
edgecolor="white", lw=0.6)
ax.text(tab, y + 0.17, f"{tname} / text: {fmt(tab)}", ha="center", va="bottom",
color=ORANGE, fontsize=10)
ax.text(fc, y + 0.17, f"Figure 1 flowchart: {fmt(fc)}", ha="left", va="bottom",
color=ORANGE, fontsize=10)
ax.text(hi + 4000, y - 0.2,
f"N implied by {fmt(ev[0])} ({ep[0]}%) and {fmt(ev[1])} ({ep[1]}%) events: "
f"{fmt(lo)}\u2013{fmt(hi)}", ha="left", va="center", color=BLUE, fontsize=10)
ax.text((fc + tab) / 2, y, f"\u00d7{tab / fc:.1f}", ha="center", va="center",
color=INK2, fontsize=10, zorder=4,
bbox=dict(boxstyle="square,pad=0.25", fc=BG, ec="none"))
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-0.55, 1.55); ax.set_xlim(0, 255_000)
ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, _: f"{v / 1000:.0f}k"))
ax.set_xlabel("Participants per arm (exposed group; controls are matched 1:1)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.spines["left"].set_visible(False); ax.tick_params(axis="y", length=0)
h = [Line2D([], [], marker="o", ls="", color=ORANGE, ms=9, label="reported: Table / Results text"),
Line2D([], [], marker="s", ls="", color=ORANGE, ms=9, label="reported: flowchart"),
Line2D([], [], marker="D", ls="", color=BLUE, ms=7,
label="recomputed: N consistent with the printed event %")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.25), ncol=2,
frameon=False, fontsize=10)
return fig
# ---------------------------------------------------------------- candidate B: text panels
def cand_B():
fig = plt.figure(figsize=(W, 5.4))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 80); ax.axis("off")
def panel(x, y, w, h, title, lines):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec=ORANGE, lw=LW))
ax.text(x + 1.5, y + h - 1.5, title, fontsize=11, fontweight="bold", va="top")
ax.text(x + 1.5, y + h - 6.5, lines, fontsize=10, va="top", linespacing=1.4)
panel(2, 30, 43, 46, "Results text and Table 2",
wrap("\u201cTable 2 summarizes the baseline characteristics of 361,258 participants, "
"divided evenly into IBS (n = 180,629) and non-IBS controls (n = 180,629).\u201d", 40)
+ "\n\nTable 2 column headers:\n\u201cIBS(n = 180,629)\u201d\n\u201cWithout IBS(n = 180,629)\u201d"
+ "\n\nTable 2, \u201cWith PD\u201d:\n\u201c82,681 (45.77%) | 31,658 (17.53%)\u201d")
panel(55, 30, 43, 46, "Figure 1 flowchart, Cohort 2",
"\u201cNewly diagnosed IBS cases\n(2002\u20132012): 16,110\u201d\n"
"\u201cExcluded prior depressive or\nanxiety disorders: 3,231 excluded\u201d\n"
"\u201cIBS cohort: 16,110\u201d\n\u201cMatched controls: 16,110\u201d\n"
"\u201cFinal follow-up N for PD\noutcomes: 32,220\u201d\n\nResults, follow-up:\n"
+ wrap("\u201c2,031 IBS patients (12.6%) \u2026 compared to 1,308 (8.1%)\u201d", 40))
ax.annotate("", xy=(54.2, 53), xytext=(45.8, 53),
arrowprops=dict(arrowstyle="<->", color=BLUE, lw=LW))
ax.text(50, 56, "\u00d711.2", ha="center", color=BLUE, fontsize=11, fontweight="bold")
ax.text(2, 23, "Recomputed from the paper\u2019s own numbers", fontsize=10.5,
fontweight="bold", color=BLUE)
notes = [f"180,629 \u00d7 2 = {2 * C2_TABLE_ARM:,}, but 16,110 \u00d7 2 = {2 * C2_FC_ARM:,}"
f" (ratio {c2_ratio:.2f})",
f"2,031 / 16,110 = {c2_pct_fc[0]:.1f}% and 1,308 / 16,110 = {c2_pct_fc[1]:.1f}%"
f" (as printed); over 180,629: {c2_pct_tab[0]:.2f}% and {c2_pct_tab[1]:.2f}%",
f"16,110 \u2212 3,231 = {c2_after_excl:,}, not 16,110",
f"82,681 IBS patients \u201cWith PD\u201d in a cohort that excluded prior PD"]
for i, s in enumerate(notes):
ax.text(3, 18 - i * 4.4, "\u2022 " + s, fontsize=10, color=BLUE, va="center")
return fig
# ---------------------------------------------------------------- candidate C: calculation flow
def cand_C():
fig = plt.figure(figsize=(W, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 64); ax.axis("off")
def box(x, y, w, h, s, ec, fs=10, bold=False):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.3,rounding_size=1",
fc="white", ec=ec, lw=LW))
ax.text(x + w / 2, y + h / 2, s, ha="center", va="center", fontsize=fs,
fontweight="bold" if bold else "normal")
def arrow(x0, y0, x1, y1, c=INK2):
ax.annotate("", xy=(x1, y1), xytext=(x0, y0),
arrowprops=dict(arrowstyle="-|>", color=c, lw=LW))
ax.text(1, 61, "Cohort 2 as printed in the Methods and the Figure 1 flowchart", fontsize=10.5,
fontweight="bold")
box(1, 44, 21, 12, "Methods: LHID\n1,000,000 \u2212 1,982\n\u2212 2,614", ORANGE)
box(27, 44, 20, 12, "new IBS cases\n2002\u20132012\n16,110", ORANGE)
box(52, 44, 20, 12, "excluded prior\nPD: 3,231\nIBS cohort: 16,110", ORANGE)
box(77, 44, 22, 12, "matched 1:1\n16,110 + 16,110\n= 32,220", ORANGE)
arrow(22.6, 50, 26.4, 50); arrow(47.6, 50, 51.4, 50); arrow(72.6, 50, 76.4, 50)
ax.text(24, 38.5, "Recomputed from the paper\u2019s own numbers", fontsize=10.5,
fontweight="bold", color=BLUE, va="center")
box(1, 22, 21, 11, f"{adults:,}\nadults", BLUE)
box(52, 22, 20, 11, f"16,110 \u2212 3,231\n= {c2_after_excl:,}", BLUE)
box(77, 22, 22, 11, f"2,031 (12.6%), 1,308\n(8.1%) events imply\nN = {c2_lo:,}\u2013{c2_hi:,}", BLUE)
ax.plot([11.5, 11.5], [43.4, 33.6], color=BLUE, lw=LW, ls=(0, (3, 2)))
ax.plot([62, 62], [43.4, 41.0], color=BLUE, lw=LW, ls=(0, (3, 2)))
ax.plot([62, 62], [36.0, 33.6], color=BLUE, lw=LW, ls=(0, (3, 2)))
ax.plot([88, 88], [43.4, 33.6], color=BLUE, lw=LW, ls=(0, (3, 2)))
ax.text(1, 15, "Cohort 2 as printed in the Results text and Table 2", fontsize=10.5,
fontweight="bold")
box(52, 1, 47, 11, f"180,629 + 180,629 = 361,258\n({c2_ratio:.1f} times the flowchart\u2019s 32,220)",
ORANGE, bold=True)
box(1, 1, 46, 11, "82,681 (45.77%) of the IBS arm \u201cWith PD\u201d\nalthough prior PD was "
"an exclusion", ORANGE)
return fig
# ---------------------------------------------------------------- candidate D: event % dot plot
def cand_D():
fig, ax = plt.subplots(figsize=(W, 3.5))
rows = [("Cohort 1, PD arm\n1,498 events", C1_EVENT_PCT[0], c1_pct_fc[0], c1_pct_tab[0]),
("Cohort 1, controls\n921 events", C1_EVENT_PCT[1], c1_pct_fc[1], c1_pct_tab[1]),
("Cohort 2, IBS arm\n2,031 events", C2_EVENT_PCT[0], c2_pct_fc[0], c2_pct_tab[0]),
("Cohort 2, controls\n1,308 events", C2_EVENT_PCT[1], c2_pct_fc[1], c2_pct_tab[1])]
ys = [3, 2, 1, 0]
for y, (lab, rep, fc, tab) in zip(ys, rows):
ax.plot([tab, rep], [y, y], color=GRID, lw=4, zorder=1)
ax.scatter(rep, y, s=170, color=ORANGE, zorder=2, edgecolor="white", lw=0.8)
ax.scatter(fc, y, s=60, color=BLUE, marker="D", zorder=3, edgecolor="white", lw=0.6)
ax.scatter(tab, y, s=110, facecolor=BG, edgecolor=BLUE, marker="D", lw=LW, zorder=3)
ax.text(rep + 0.3, y, f"{rep}%", va="center", color=ORANGE, fontsize=10)
ax.text(tab + 0.3, y + 0.22, f"{tab:.2f}%", va="bottom", ha="left", color=BLUE,
fontsize=10)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xlim(0, 14.5); ax.set_ylim(-0.6, 3.6)
ax.set_xlabel("Share of the arm with an outcome event during follow-up (%)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.spines["left"].set_visible(False); ax.tick_params(axis="y", length=0)
h = [Line2D([], [], marker="o", ls="", color=ORANGE, ms=10, label="reported %"),
Line2D([], [], marker="D", ls="", color=BLUE, ms=7,
label="recomputed: events / flowchart N"),
Line2D([], [], marker="D", ls="", mfc=BG, mec=BLUE, mew=LW, ms=9,
label="recomputed: events / Table N")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.22), ncol=3,
frameon=False, fontsize=10)
return fig
# ---------------------------------------------------------------- candidate E: CI width check
def cand_E():
fig, ax = plt.subplots(figsize=(3.4, 3.8))
y = 0
yt, yl = [], []
for lab, hr, rep_ci, ci_rep, ci_big, d_rep, d_big in ci_rows:
tname = "Table 2" if "Cohort 2" in lab else "Table 1"
items = [("reported aHR, 95% CI", rep_ci, ORANGE, "o", 0.28),
(f"recomputed: {d_rep:,} reported events", ci_rep, BLUE, "D", 0.0),
(f"recomputed: {tname} N at printed rates\n({d_big:,.0f} events)",
ci_big, BLUE, "D", -0.28)]
for txt, (lo, hi), c, m, dy in items:
mfc = c if dy >= 0 else BG
ax.plot([lo, hi], [y + dy, y + dy], color=c, lw=LW)
ax.plot([hr], [y + dy], marker=m, color=c, mfc=mfc, mec=c, mew=LW, ms=8, ls="")
ax.text(1.02, y + dy, f"{lo:.2f}\u2013{hi:.2f} {txt}", va="center", fontsize=10,
color=c, transform=ax.get_yaxis_transform(), clip_on=False)
yt.append(y); yl.append(lab.replace(" (", "\n("))
y -= 1.2
ax.axvline(1, color=INK2, lw=0.8)
ax.set_xscale("log")
ax.set_xticks([1, 1.2, 1.4, 1.6, 1.8])
ax.get_xaxis().set_major_formatter(matplotlib.ticker.FormatStrFormatter("%.1f"))
ax.get_xaxis().set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_xlim(0.95, 1.8)
ax.minorticks_off()
ax.set_yticks(yt); ax.set_yticklabels(yl, fontsize=10)
ax.set_ylim(y + 0.75, 0.55)
ax.set_xlabel("Overall adjusted hazard ratio\n(log scale; line at 1 = no association)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.spines["left"].set_visible(False); ax.tick_params(axis="y", length=0)
return fig
for name, fn in (("A_n_per_arm", cand_A), ("B_text_panels", cand_B),
("C_calc_flow", cand_C), ("D_event_pct", cand_D), ("E_ci_width", cand_E)):
save(fn(), os.path.join(CAND, name))
save(cand_A(), os.path.join(HERE, "fig1"))
save(cand_E(), os.path.join(HERE, "fig2"))
print("\nFigures written: fig1 (= candidate A), fig2 (= candidate E), candidates/A-E")
The logistic-regression "odds ratios", including those in the abstract, appear to be log-odds coefficients
Psychol Med 2022;52(14):3289-3296 (published online 18 March 2021) · doi:10.1017/S0033291721000088 · PMC9693673
Summary
The paper reports two multivariable logistic regressions for clinically elevated hsCRP (⩾3 mg/L) and labels every estimate an odds ratio (OR). One of them is negative ("OR −4.39, 95% CI −7.04 to −1.97"), and six more have negative lower confidence limits (four distinct intervals; the age and gender intervals appear in both models). An odds ratio cannot be negative. Read as log-odds coefficients, the printed values hang together: every printed CI agrees with its printed p-value on significance at 0.05 (the interval excludes 0 exactly when p < 0.05), and the paper's own Table 1 counts reproduce the estimates closely (unadjusted log-odds 2.39 against the printed 2.44). Read as ORs, the two main-effect intervals in model 2 contradict their p-values. On the coefficient reading, the abstract's headline odds ratios are understated about 2.7- to 4.7-fold: "OR 2.44" corresponds to an OR of about 11.5 (95% CI 3.6 to 51.4), "OR 1.52" to about 4.6 and "OR 1.09" to about 3.0. The direction and significance of each finding are unchanged, but the effect sizes as printed are on the wrong scale, and read literally as ORs, the CI given for depression (0.27 to 2.01) includes 1 and would contradict the stated significance.
What the paper reports
Methods, "Statistical analysis":
To address the second hypothesis, two models of multivariable logistic regression were used. The first assessed how much risk comorbid depression and overweight contributed to the hsCRP levels ⩾3 mg/L relative to other participant groups, with age and gender as covariates. The second assessed how much risk depression and overweight status (BMI ⩾ 25) separately contributed to hsCRP levels ⩾3 mg/L, with age and gender as covariates.
Results, "High sensitivity C-Reactive Protein", first logistic paragraph (model 1):
Overweight patients with depression were at a significantly increased risk of having hsCRP ⩾3 mg/L [OR 2.44, 95% confidence interval (CI) 1.28 to 3.94, p < 0.001], overweight controls did not have a significantly increased risk of hsCRP ⩾3 mg/L (OR 1.51, 95% CI −0.33 to 2.87, p = 0.13), normal-weight patients with depression did not have a significantly increased risk of hsCRP ⩾3 mg/L (OR 1.08, 95% CI −0.6 to 2.4, p = 0.28), while normal-weight controls were at a significantly decreased risk of hsCRP ⩾3 mg/L (OR −4.39, 95% CI −7.04 to −1.97, p < 0.001). Age and gender were not significantly associated with hsCRP ⩾3 mg/L [OR 0.01, 95% CI −0.04 to 0.06, p = 0.621, OR 0.61, 95% CI −0.19 to 1.47, p = 0.145, respectively].
Results, same section, second logistic paragraph (model 2):
Depression status was significantly associated with hsCRP ⩾3 mg/L (OR 1.09, 95% CI 0.27 to 2.01, p = 0.013) and overweight status was significantly associated with hsCRP ⩾3 mg/L (OR 1.52, 95% CI 0.7 to 2.41, p < 0.001). Age and gender were not significantly associated with hsCRP ⩾3 mg/L [OR 0.62, 95% CI −0.04 to 0.06, p = 0.647, OR 0.01, 95% CI −0.19 to 1.47, p = 0.137, respectively].
Abstract, "Results":
Multivariable logistic regression showed that comorbid depression and overweight significantly increased the risk of clinically elevated hsCRP levels ⩾3 mg/L (OR 2.44, 1.28–3.94). In a separate multivariable logistic regression model, overweight status contributed most to the risk of having hsCRP levels ⩾3 mg/L (OR 1.52, 0.7–2.41), while depression also contributed a significant risk (OR 1.09, 0.27–2).
Table 1, row "hsCRP ⩾3 mg/L (%)". The columns are overweight & depressed (N = 69), overweight control (N = 35), normal weight & depressed (N = 55), normal weight control (N = 57), statistic, p:
hsCRP ⩾3 mg/L (%) | 26 (37.7%) | 5 (14.3%) | 6 (10.9%) | 3 (5.3%) | x2 = 25.96 | <0.001
The conclusions that rest on these models. Abstract, "Conclusion":
Comorbid depression and overweight status are associated with increased hsCRP, and the coexistence of these conditions amplified the risk of clinically elevated hsCRP levels. Overweight status contributed most to the risk of clinically elevated hsCRP levels, but depression also contributed to a significant risk.
Discussion, first paragraph:
Of clinical relevance, overweight patients with depression had the highest risk for hsCRP levels ⩾3 mg/L, indicating clinically elevated peripheral inflammation.
What the paper's own numbers show
Method. In logistic regression a coefficient β is a log odds ratio, and the odds ratio is exp(β). An OR must be positive and its null is 1, while a coefficient can take any sign and its null is 0. I made three checks:
- Sign. I checked each printed estimate and CI limit for values ⩽ 0, which no OR can take.
- CI against p-value. I took each printed CI as a 95% Wald interval and back-calculated the two-sided p-value it implies under two readings: as a coefficient (null 0) and as an OR (null 1, computed on the log scale). I also checked, more simply, whether each interval excludes the null exactly when the printed p is below 0.05. The printed intervals are not symmetric about their estimates, and most look like profile-likelihood intervals (see check 3), so a Wald back-calculation from them is only approximate. The significance check does not depend on that assumption.
- Refit from Table 1. I refitted both models without the age and gender covariates, using only the grouped counts in Table 1 (40 cases with hsCRP ⩾3 mg/L among 216 participants). The paper's covariate-adjusted models need individual data, so the refits show the scale and approximate size of the estimates, not their exact values. I report both Wald and profile-likelihood intervals for the refits, because most printed intervals are asymmetric about their estimates in the way profile-likelihood intervals are (2.44 lies 1.16 above its lower limit of 1.28 and 1.50 below its upper limit of 3.94).
Every number below is printed by recompute.py.txt. Quoted passages keep the paper's minus signs (U+2212); elsewhere minus signs are typed as hyphens.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| hsCRP ⩾3 mg/L by group | Table 1 | 26/69, 5/35, 6/55, 3/57; χ² = 25.96, p < 0.001 | 37.7%, 14.3%, 10.9%, 5.3%; Pearson χ² = 25.96, df 3, p = 9.7 × 10⁻⁶ (matches) |
| Overweight + depressed v. reference | Abstract; Results, model 1 | "OR 2.44 (1.28 to 3.94)", p < 0.001 | As a coefficient, the CI implies p = 0.0003 (matches p < 0.001); as an OR, it implies p = 0.002. Unadjusted refit: log-odds 2.39 (profile 1.26 to 3.87; Wald 1.13 to 3.65), p = 0.0002. Crude OR = (26/43)/(3/54) = 10.9. If the printed value is a coefficient, OR = exp(2.44) = 11.5 (3.6 to 51.4) |
| Overweight controls v. reference | Results, model 1 | "OR 1.51 (−0.33 to 2.87)", p = 0.13 | Negative lower limit is impossible for an OR. As a coefficient, the CI includes 0 (not significant, agreeing with p = 0.13), but a Wald back-calculation gives p = 0.064, about half the printed 0.13. Refit: log-odds 1.10 (profile -0.37 to 2.74; Wald -0.40 to 2.60), Wald p = 0.15. exp(printed) = 4.5 (0.72 to 17.6) |
| Normal-weight depressed v. reference | Results, model 1 | "OR 1.08 (−0.6 to 2.4)", p = 0.28 | Negative lower limit is impossible for an OR. As a coefficient, the CI includes 0 (not significant, agreeing with p = 0.28), but a Wald back-calculation gives p = 0.16, about half the printed 0.28. Refit: log-odds 0.79 (profile -0.60 to 2.39; Wald -0.65 to 2.23), Wald p = 0.28. exp(printed) = 2.9 (0.55 to 11.0) |
| "Normal-weight controls" | Results, model 1 | "OR −4.39 (−7.04 to −1.97)", p < 0.001 | Impossible as an OR. With four groups and an intercept, one group must be the reference and gets no coefficient of its own. Normal-weight controls are the reference implied by the other three contrasts. Their observed log-odds is ln(3/54) = -2.89, and the refit intercept is -2.89 (profile -4.30 to -1.89), so −4.39 is most consistent with the model intercept, which in the adjusted model refers to age 0 |
| Depression | Abstract; Results, model 2 | "OR 1.09 (0.27 to 2.01)", p = 0.013 | Read as an OR, the CI contains 1 and implies p = 0.87, contradicting p = 0.013. Read as a coefficient, it implies p = 0.014 (matches). Refit: log-odds 1.12 (profile 0.31 to 2.03), p = 0.010. exp(printed) = 3.0 (1.31 to 7.46) |
| Overweight | Abstract; Results, model 2 | "OR 1.52 (0.7 to 2.41)", p < 0.001 | Read as an OR, the CI contains 1 and implies p = 0.18, contradicting p < 0.001. Read as a coefficient, it implies p = 0.0005 (matches). Refit: log-odds 1.46 (profile 0.68 to 2.32), p = 0.00045. exp(printed) = 4.6 (2.01 to 11.1) |
| Age, gender | Results, model 1 | "OR 0.01 (−0.04 to 0.06)", p = 0.621; "OR 0.61 (−0.19 to 1.47)", p = 0.145 | Negative lower limits are impossible for ORs. As coefficients, implied p = 0.70 and 0.15. exp(printed) = 1.01 (0.96 to 1.06) per year and 1.84 (0.83 to 4.35) |
| Age, gender | Results, model 2 | "OR 0.62 (−0.04 to 0.06)", p = 0.647; "OR 0.01 (−0.19 to 1.47)", p = 0.137 | The age estimate 0.62 lies outside its own CI (-0.04 to 0.06). The two point estimates appear to be transposed relative to their CIs and to model 1 (0.01 with -0.04 to 0.06; 0.61 with -0.19 to 1.47). Both CIs include 0 and both p-values are above 0.05, so significance still agrees on the coefficient reading |
| Size of understatement | Abstract | 2.44, 1.52, 1.09 | exp(printed) / printed = 4.7×, 3.0×, 2.7× |
In the refit of model 1, the printed intervals and the profile-likelihood intervals line up closely: 1.28 to 3.94 against 1.26 to 3.87, -0.33 to 2.87 against -0.37 to 2.74, and -0.6 to 2.4 against -0.60 to 2.39. For overweight + depressed, depression and overweight, the printed intervals also share the refit's upward skew (for example 1.16 below and 1.50 above the estimate, against 1.13 and 1.48 in the refit). That pattern is what R's confint() gives for a glm fit on the coefficient scale. The Methods state that all analyses used R 4.0.0.
The Wald back-calculation is less exact. On the coefficient reading it matches the printed p for overweight + depressed (0.0003 v. < 0.001), "normal-weight controls" (0.0007 v. < 0.001), depression (0.014 v. 0.013), overweight (0.0005 v. < 0.001) and model 1 gender (0.15 v. 0.145), and is close to it for model 1 age (0.70 v. 0.621). For overweight controls (0.064 v. 0.13) and normal-weight depressed (0.16 v. 0.28) it gives about half the printed value, a gap a reader rerunning the script will see. In these two rows the printed limits are close to the refit's limits, but the printed estimates (1.51, 1.08) are higher than the refit estimates (1.10, 0.79). The printed estimates therefore sit nearer the upper limit (1.84 below and 1.36 above; 1.68 below and 1.32 above). The back-calculation divides the estimate by an SE taken from the interval width, so a larger estimate with the same width gives a larger z and a smaller p-value. The printed p-values for these two rows are close to the unadjusted refit Wald p-values (0.15 and 0.28). Because the printed intervals look like asymmetric profile intervals, a Wald back-calculation from them is only approximate, and these two rows cannot be resolved further without the model output. The gap is in size only: on the coefficient reading, no printed interval contradicts its printed p-value on significance at 0.05.
Figure
Each row is one group or main-effect estimate the paper prints with the label "OR"; the age and gender rows are not shown. Orange circles and lines are the printed estimate and 95% CI, on the linear scale on which they are printed. Blue squares and lines are log-odds coefficients refitted, without the age and gender covariates, from the hsCRP ⩾3 mg/L counts in Table 1, with profile-likelihood 95% intervals. The shaded region holds values below 0, which no odds ratio can take. The solid line at 0 is the null for a log-odds coefficient, and the dashed line at 1 is the null for an odds ratio. Whether each printed p-value (right) is below 0.05 matches whether its interval crosses 0, not whether it crosses 1. That is clearest for depression and overweight in model 2, whose intervals cross 1 but are reported as significant. The row labelled "normal-weight controls" has no blue refit because the reference group of a categorical predictor has no contrast of its own.
The same estimates on a log odds-ratio axis. Orange is the printed value taken literally as an OR. Lower limits below 0 cannot be placed on this axis and are drawn as arrows. Blue is exp(printed value), the odds ratio implied if the printed numbers are coefficients, labelled with its point estimate. The intercept-like "normal-weight controls" row is omitted because its printed value is negative. The age and gender rows of both models are not shown either.
Why it matters
The three figures in the abstract are the paper's quantitative answer to its second aim, "what extent overweight status and depression status contributed to clinically elevated hsCRP levels". As printed, they suggest modest associations: odds roughly 2.4, 1.5 and 1.1 times higher. If they are coefficients, the implied odds ratios are about 11.5, 4.6 and 3.0, a 2.7- to 4.7-fold difference. A reader citing these ORs, or pooling them in a meta-analysis, would carry the wrong magnitude. Read literally as ORs, the printed intervals also contradict the paper's own significance statements: the depression interval (0.27 to 2.01) and the overweight interval (0.7 to 2.41) both include 1. A reader who takes the label at face value could therefore conclude that neither main effect is significant, which is the opposite of what the abstract states.
The qualitative conclusions do not depend on the scale. Overweight-and-depressed participants have the highest odds, overweight contributes more than depression, and both are significant in model 2. Those orderings and significance calls are the same on either scale, and the Table 1 refits support them. The second-paragraph age and gender estimates are a separate, smaller transcription problem.
Explanations considered
- Conversion or typesetting artefact (lost or added minus signs). This does not account for the numbers. The evidence check found the same values, with true minus signs (U+2212), in the publisher's HTML and PDF on Cambridge Core and in the PMC and Europe PMC XML. Removing the signs would not help either: an OR of 4.39 would mean increased risk, while the text says normal-weight controls were at "significantly decreased risk".
- The printed values are ORs and only the negative ones are typos. This does not account for them. The positive-only intervals for depression (0.27 to 2.01) and overweight (0.7 to 2.41) would then contain 1 and imply p = 0.87 and p = 0.18, not the printed p = 0.013 and p < 0.001. Separately, the unadjusted crude OR for overweight-depressed v. normal-weight controls is 10.9, far from 2.44.
- The values are coefficients, correctly computed, and labelled "OR" in error. This accounts for everything checked. On the coefficient reading, every printed CI agrees with its printed p-value on significance at 0.05. Of the eight rows that are not transposed, a Wald back-calculation also matches the printed p-value closely for five and is close to it for a sixth, model 1 age (0.70 v. 0.621). It is about a factor of 2 low for overweight controls (0.064 v. 0.13) and normal-weight depressed (0.16 v. 0.28), whose printed p-values are instead close to the unadjusted refit Wald p-values (0.15 and 0.28). Since the printed intervals look like asymmetric profile intervals, the back-calculation is only approximate, and this gap does not point to a different scale: on the OR reading those two rows have impossible negative limits. The unadjusted refits from Table 1 give 2.39, 1.10, 0.79, 1.12 and 1.46 against the printed 2.44, 1.51, 1.08, 1.09 and 1.52, and their profile intervals closely match the printed intervals. The covariates would be expected to shift the estimates somewhat (age and gender differ between groups, Table 1), so small differences are expected.
- The "−4.39" is a genuine group contrast. Unlikely. With an intercept, only three of the four groups can have coefficients, and the three positive coefficients are the contrasts against normal-weight controls. −4.39 fits an intercept: the unadjusted intercept is -2.89, and an adjusted intercept refers to age 0, so a more negative value is expected. Its exact meaning cannot be confirmed without the model output.
- The supplement holds the correct output. It does not. The only supplementary file contains a non-smoker version of Table 1 and a consortium member list, with no regression output.
Status
As of 2026-10-23:
- Version of record. Checked on Cambridge Core (HTML and PDF, S0033291721000088a.pdf) and in the PMC and Europe PMC full-text XML. All four sources carry the same values, including the negative "OR" and the negative limits, so the problem is in the version of record and is not introduced by conversion.
- Supplement. The only supplementary file is S0033291721000088sup001.docx. It holds Supplementary Table 1 (non-smokers: hsCRP ⩾3 mg/L in 23/58, 4/32, 5/44 and 3/51; χ² = 24.405) and the consortium list. It has no logistic-regression output, so it cannot settle whether the values are coefficients or ORs.
- Corrections and notices. No correction, erratum, expression of concern, retraction or comment was found. Crossref shows no
update-to/updated-byrelations. The Crossref Labs updates feed, which carries the Retraction Watch data, is empty. PMC and Europe PMC show no linked notices (Europe PMC record PMID 33731235, no comment/correction list), and the Cambridge Core page shows none either. - Repository copy. The Cardiff ORCA repository has a record for the paper (orca.cardiff.ac.uk/id/eprint/138378). The site refused automated access (HTTP 403), so I did not check its full text or which version it holds. It could show whether the "OR" labels were already in the accepted manuscript.
- PubPeer. PubPeer refused direct access (HTTP 403), and a web search for the title with "PubPeer" found no PubPeer entry. So PubPeer was searched only indirectly.
- Sources. doi.org/10.1017/S0033291721000088; PMC9693673; Crossref REST API
api.crossref.org/works/10.1017/S0033291721000088; Europe PMC REST search.
Suggested correction
- State that the values reported in the logistic-regression paragraphs of the Results and in the abstract are log-odds coefficients (β), or replace them with odds ratios, exp(β), and their exponentiated CIs. If the printed values are the model coefficients, the abstract would read approximately: - "OR 11.5, 95% CI 3.6–51.4" for comorbid depression and overweight; - "OR 4.6, 2.0–11.1" for overweight; - "OR 3.0, 1.3–7.5" for depression.
These values should be confirmed from the original model output. 2. In model 1, report the reference group explicitly (apparently normal-weight controls), and either drop the "−4.39" or label it as the intercept rather than as a risk estimate for normal-weight controls. 3. In the model 2 paragraph, check the age and gender estimates, which appear transposed (0.62 is printed with the interval -0.04 to 0.06). 4. Consider giving the full model output (coefficients, SEs, intercept and coding) in a table or supplement.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC9693673 (Psychol Med 2022;52(14):3289-3296, online 18 March 2021;
# doi:10.1017/S0033291721000088)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1/fig2
# (SVG + PNG) plus five candidate figure designs in ./candidates/.
# Inputs are only numbers printed in the paper (Results text and Table 1).
import os
import numpy as np
from scipy import stats, optimize
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z = stats.norm.ppf(0.975)
# ---------------------------------------------------------------- inputs
# Table 1, row "hsCRP >=3 mg/L (%)": events / group N
groups = {"OW_dep": (26, 69), "OW_ctl": (5, 35), "NW_dep": (6, 55), "NW_ctl": (3, 57)}
# Results, hsCRP paragraphs 1 and 2: (estimate, CI low, CI high, reported p) as printed with label "OR"
rep1 = {
"OW_dep": (2.44, 1.28, 3.94, "<0.001"),
"OW_ctl": (1.51, -0.33, 2.87, "0.13"),
"NW_dep": (1.08, -0.6, 2.4, "0.28"),
"NW_ctl": (-4.39, -7.04, -1.97, "<0.001"),
"age": (0.01, -0.04, 0.06, "0.621"),
"gender": (0.61, -0.19, 1.47, "0.145"),
}
rep2 = {
"depression": (1.09, 0.27, 2.01, "0.013"),
"overweight": (1.52, 0.7, 2.41, "<0.001"),
"age (as printed)": (0.62, -0.04, 0.06, "0.647"),
"gender (as printed)": (0.01, -0.19, 1.47, "0.137"),
}
def pnum(s):
return 0.001 if s.startswith("<") else float(s)
def fmt(x, d=2):
return f"{x:.{d}f}"
# ---------------------------------------------------------------- 1. Table 1 arithmetic
print("1. Table 1 hsCRP >=3 mg/L row")
for g, (k, n) in groups.items():
print(f" {g}: {k}/{n} = {100*k/n:.1f}%")
obs = np.array([[k, n - k] for k, n in groups.values()])
chi2, p_chi, dof, _ = stats.chi2_contingency(obs, correction=False)
print(f" Pearson chi2 = {chi2:.2f}, df = {dof}, p = {p_chi:.2g} (reported x2 = 25.96, p < 0.001)")
print(f" total events = {obs[:,0].sum()}, non-events = {obs[:,1].sum()}, N = {obs.sum()}")
# ---------------------------------------------------------------- 2. reading the printed values two ways
def wald_from_ci(b, lo, hi):
se = (hi - lo) / (2 * Z)
return se, 2 * stats.norm.sf(abs(b / se))
def as_or(b, lo, hi):
"""p implied if (b, lo, hi) were an OR and its CI (Wald on log scale)."""
if b <= 0 or lo <= 0:
return None
se = (np.log(hi) - np.log(lo)) / (2 * Z)
return 2 * stats.norm.sf(abs(np.log(b) / se))
readings = {}
for title, rep in (("Model 1 (group status + age + gender)", rep1), ("Model 2 (depression + overweight + age + gender)", rep2)):
print(f"\n2. {title}")
print(" name | printed 'OR' (CI), p | impossible as OR? | p if log-odds | p if OR | exp(printed) = OR if log-odds")
for k, (b, lo, hi, pr) in rep.items():
_, pl = wald_from_ci(b, lo, hi)
po = as_or(b, lo, hi)
imp = (b <= 0) or (lo <= 0)
ci_has_1 = (lo < 1 < hi)
readings[k] = dict(b=b, lo=lo, hi=hi, pr=pr, p_logodds=pl, p_or=po, exp=(np.exp(b), np.exp(lo), np.exp(hi)))
e = f"{np.exp(b):.3g} ({np.exp(lo):.3g} to {np.exp(hi):.3g})"
pos = "impossible" if po is None else f"{po:.3f} (CI contains 1: {ci_has_1})"
print(f" {k}: {b} ({lo} to {hi}), p {pr} | {'YES' if imp else 'no'} | {pl:.3g} | {pos} | {e}")
print(" Significance at 0.05 on the coefficient reading: does 'CI excludes 0' agree with 'printed p < 0.05'?")
for k, (b, lo, hi, pr) in rep.items():
excl0 = not (lo < 0 < hi)
sig = pnum(pr) < 0.05
print(f" {k}: CI excludes 0 = {excl0}; printed p < 0.05 = {sig}; agree = {excl0 == sig}")
print(" Point estimate inside its own printed CI?",
{k: bool(v[1] <= v[0] <= v[2]) for k, v in rep.items()})
# CI asymmetry of the printed headline interval (Wald would be symmetric)
b, lo, hi, _ = rep1["OW_dep"]
print(f"\n OW_dep printed interval: distance below = {b-lo:.2f}, above = {hi-b:.2f} (asymmetric; midpoint {(lo+hi)/2:.2f})")
# ---------------------------------------------------------------- 3. unadjusted logistic fits from Table 1 counts
names = list(groups)
y = np.array([groups[g][0] for g in names], float)
n = np.array([groups[g][1] for g in names], float)
def nll(beta, X):
eta = X @ beta
return -np.sum(y * eta - n * np.logaddexp(0, eta))
def fit(X, fixed=None):
if fixed is None:
r = optimize.minimize(nll, np.zeros(X.shape[1]), args=(X,), method="BFGS", options={"gtol": 1e-10})
return r.x, r.fun
j, val = fixed
free = [i for i in range(X.shape[1]) if i != j]
def f(bf):
bb = np.empty(X.shape[1]); bb[free] = bf; bb[j] = val
return nll(bb, X)
r = optimize.minimize(f, np.zeros(len(free)), method="BFGS", options={"gtol": 1e-10})
return None, r.fun
def summarize(X, labels):
beta, f0 = fit(X)
p = 1 / (1 + np.exp(-X @ beta))
W = n * p * (1 - p)
se = np.sqrt(np.diag(np.linalg.inv(X.T @ (W[:, None] * X))))
out = {}
crit = stats.chi2.ppf(0.95, 1) / 2
for j, lab in enumerate(labels):
g = lambda t: fit(X, (j, t))[1] - f0 - crit
plo = optimize.brentq(g, beta[j] - 12 * se[j], beta[j])
phi = optimize.brentq(g, beta[j], beta[j] + 12 * se[j])
pw = 2 * stats.norm.sf(abs(beta[j] / se[j]))
out[lab] = dict(b=beta[j], se=se[j], wlo=beta[j] - Z * se[j], whi=beta[j] + Z * se[j], plo=plo, phi=phi, p=pw)
print(f" {lab}: log-odds {beta[j]:.2f} (Wald {beta[j]-Z*se[j]:.2f} to {beta[j]+Z*se[j]:.2f}; "
f"profile {plo:.2f} to {phi:.2f}), p = {pw:.2g}; OR = {np.exp(beta[j]):.2f} "
f"(profile {np.exp(plo):.2f} to {np.exp(phi):.2f})")
return out
print("\n3a. Unadjusted model 1 from Table 1 counts (reference = normal-weight controls)")
X1 = np.array([[1, 1, 0, 0], [1, 0, 1, 0], [1, 0, 0, 1], [1, 0, 0, 0]], float)
m1 = summarize(X1, ["intercept", "OW_dep", "OW_ctl", "NW_dep"])
print("\n3b. Unadjusted model 2 from Table 1 counts (main effects, no interaction)")
X2 = np.array([[1, 1, 1], [1, 0, 1], [1, 1, 0], [1, 0, 0]], float) # intercept, depression, overweight
m2 = summarize(X2, ["intercept", "depression", "overweight"])
print("\n3b2. Interval skew: distance from estimate to lower / upper limit, printed v. unadjusted refit (profile)")
for k_rep, k_fit, mm in (("OW_dep", "OW_dep", m1), ("OW_ctl", "OW_ctl", m1), ("NW_dep", "NW_dep", m1),
("depression", "depression", m2), ("overweight", "overweight", m2)):
b_, lo_, hi_, _ = {**rep1, **rep2}[k_rep]
f_ = mm[k_fit]
print(f" {k_rep}: printed {b_-lo_:.2f} below / {hi_-b_:.2f} above; "
f"refit {f_['b']-f_['plo']:.2f} below / {f_['phi']-f_['b']:.2f} above")
print("\n3c. Crude ORs v. normal-weight controls, (a/b)/(c/d)")
crude = {}
for g in ["OW_dep", "OW_ctl", "NW_dep"]:
k, N = groups[g]
crude[g] = (k / (N - k)) / (3 / 54)
print(f" {g}: OR = {crude[g]:.2f}, log OR = {np.log(crude[g]):.2f}")
# intercept plausibility: NW controls' observed log-odds and a rough age-0 shift
lo_nw = np.log(3 / 54)
shift = lo_nw - 0.01 * 30 - 0.61 * 44 / 57
print(f"\n3d. Normal-weight controls: observed log-odds ln(3/54) = {lo_nw:.2f}; "
f"shifted to age 0 and the other gender using printed age 0.01/yr (median age 30) "
f"and gender 0.61 (77.2% female): about {shift:.2f} (printed '-4.39', CI -7.04 to -1.97)")
print("\n3e. Headline ratios, exp(printed) / printed")
for k, r in (("OW_dep", rep1["OW_dep"]), ("overweight", rep2["overweight"]), ("depression", rep2["depression"])):
print(f" {k}: exp({r[0]}) = {np.exp(r[0]):.2f}; ratio to printed = {np.exp(r[0])/r[0]:.1f}x")
# ================================================================ figures
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "axes.edgecolor": FG2, "axes.labelcolor": FG,
"xtick.color": FG2, "ytick.color": FG, "text.color": FG, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
def style(ax, grid_axis="x"):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis=grid_axis, color=GRID, lw=0.8, zorder=0)
ax.tick_params(axis="y", length=0)
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
ROWS = [ # (label, printed tuple, Table-1 refit dict or None)
("Overweight + depressed\n(model 1)", rep1["OW_dep"], m1["OW_dep"]),
("Overweight controls\n(model 1)", rep1["OW_ctl"], m1["OW_ctl"]),
("Normal-weight depressed\n(model 1)", rep1["NW_dep"], m1["NW_dep"]),
("'Normal-weight controls'\n(model 1)", rep1["NW_ctl"], None),
("Depression\n(model 2)", rep2["depression"], m2["depression"]),
("Overweight\n(model 2)", rep2["overweight"], m2["overweight"]),
]
# ---- design A: printed values on their own (linear) scale, both candidate nulls
def design_A(path):
fig, ax = plt.subplots(figsize=(7.2, 4.9))
ypos = np.arange(len(ROWS))[::-1].astype(float)
ax.axvspan(-7.6, 0, color="#f1efea", zorder=0.5, lw=0)
ax.text(-7.45, len(ROWS) - 0.35, "values < 0: impossible for an OR", color=FG2, fontsize=10, va="center")
ax.axvline(0, color=FG, lw=1, zorder=1)
ax.axvline(1, color=FG2, lw=1, ls=(0, (4, 3)), zorder=1)
ax.text(-0.12, -0.75, "0 = null for a\nlog-odds coefficient", fontsize=10, color=FG, ha="right", va="center")
ax.text(1.12, -0.75, "1 = null for\nan odds ratio", fontsize=10, color=FG2, ha="left", va="center")
for yy, (lab, (b, lo, hi, pr), m) in zip(ypos, ROWS):
ax.plot([lo, hi], [yy + 0.12] * 2, color=ORANGE, lw=2, zorder=3, solid_capstyle="butt")
ax.plot(b, yy + 0.12, "o", ms=8, color=ORANGE, zorder=4)
if m is None:
ax.text(-7.45, yy - 0.42, "no refit: a reference group\nhas no contrast of its own",
fontsize=10, color=BLUE, va="center", ha="left", linespacing=1.1,
bbox=dict(fc="#f1efea", ec="none", pad=1.5), zorder=2)
if m is not None:
ax.plot([m["plo"], m["phi"]], [yy - 0.14] * 2, color=BLUE, lw=2, zorder=3, solid_capstyle="butt")
ax.plot(m["b"], yy - 0.14, "s", ms=7.5, color=BLUE, zorder=4)
ax.text(4.75, yy, f"p {pr}" if pr.startswith("<") else f"p = {pr}", fontsize=10, color=FG2, va="center")
ax.set_yticks(ypos)
ax.set_yticklabels([r[0] for r in ROWS], fontsize=10)
ax.set_xlim(-7.6, 5.6)
ax.set_ylim(-1.6, len(ROWS) - 0.1)
ax.set_xticks(range(-7, 5))
ax.set_xlabel("Value on the scale the paper prints (labelled 'OR', 95% CI)")
style(ax)
ax.text(4.75, len(ROWS) - 0.35, "printed p", fontsize=10, color=FG2, va="center", weight="bold")
h1 = ax.errorbar([], [], xerr=[], fmt="o", color=ORANGE, ms=8, label="Printed in the paper as 'OR (95% CI)'")
h2 = ax.errorbar([], [], xerr=[], fmt="s", color=BLUE, ms=7.5, label="Log-odds refitted from Table 1 counts (unadjusted)")
ax.legend(handles=[h1, h2], loc="lower left", bbox_to_anchor=(-0.36, 1.0), frameon=False, ncol=1, fontsize=10)
fig.tight_layout()
save(fig, path)
# ---- design B: odds-ratio (log) axis, printed-as-OR v. exp(printed)
def design_B(path):
rows = [r for r in ROWS if r[1][0] > 0]
fig, ax = plt.subplots(figsize=(7.2, 4.4))
ypos = np.arange(len(rows))[::-1].astype(float)
left = 0.1
ax.axvline(1, color=FG2, lw=1, zorder=1)
for yy, (lab, (b, lo, hi, pr), m) in zip(ypos, rows):
y0 = yy + 0.15
if lo > 0:
ax.plot([lo, hi], [y0] * 2, color=ORANGE, lw=2, zorder=3)
else:
ax.annotate("", xy=(left * 1.03, y0), xytext=(hi, y0),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=12), zorder=3)
ax.text(left * 1.12, y0 + 0.2, f"lower limit {str(lo).replace('-', '\u2212')} < 0", fontsize=10, color=ORANGE, va="center")
ax.plot(b, y0, "o", ms=8, color=ORANGE, zorder=4)
e, elo, ehi = np.exp(b), np.exp(lo), np.exp(hi)
y1 = yy - 0.15
ax.plot([elo, ehi], [y1] * 2, color=BLUE, lw=2, zorder=3)
ax.plot(e, y1, "s", ms=7.5, color=BLUE, zorder=4)
ax.text(ehi * 1.08, y1, f"{e:.1f}", fontsize=10, color=BLUE, va="center")
ax.set_xscale("log")
ax.set_xlim(left, 120)
ax.set_xticks([0.1, 0.3, 1, 3, 10, 30, 100])
ax.set_xticklabels(["0.1", "0.3", "1", "3", "10", "30", "100"])
ax.set_yticks(ypos)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-0.7, len(rows) - 0.3)
ax.set_xlabel("Odds ratio for hsCRP ⩾3 mg/L (log scale, 95% CI)")
style(ax)
h1 = ax.errorbar([], [], xerr=[], fmt="o", color=ORANGE, ms=8, label="Printed value read as an OR")
h2 = ax.errorbar([], [], xerr=[], fmt="s", color=BLUE, ms=7.5, label="exp(printed value): the OR if it is a log-odds coefficient")
ax.legend(handles=[h1, h2], loc="lower left", bbox_to_anchor=(-0.33, 1.0), frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
# ---- design C: p-value consistency under the two readings
def design_C(path):
keys = [("Overweight + depressed (m1)", "OW_dep"), ("Overweight controls (m1)", "OW_ctl"),
("Normal-weight depressed (m1)", "NW_dep"), ("'Normal-weight controls' (m1)", "NW_ctl"),
("Depression (m2)", "depression"), ("Overweight (m2)", "overweight")]
fig, ax = plt.subplots(figsize=(7.2, 4.3))
ypos = np.arange(len(keys))[::-1].astype(float)
ax.axvline(0.05, color=FG2, lw=1, ls=(0, (4, 3)))
ax.text(0.052, len(keys) - 0.45, "p = 0.05", fontsize=10, color=FG2)
for yy, (lab, k) in zip(ypos, keys):
r = readings[k]
pr = pnum(r["pr"])
ax.plot(pr, yy + 0.18, "o", ms=9, color=ORANGE, zorder=4)
if r["pr"].startswith("<"):
ax.annotate("", xy=(pr / 3, yy + 0.18), xytext=(pr, yy + 0.18),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=10))
ax.plot(r["p_logodds"], yy, "s", ms=8, color=BLUE, zorder=4)
if r["p_or"] is None:
ax.text(1.5e-5, yy - 0.3, "OR reading impossible", fontsize=10, color=FG2, va="center", ha="left")
else:
ax.plot(r["p_or"], yy - 0.2, "D", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=4)
ax.set_xscale("log")
ax.set_xlim(1e-5, 1.5)
ax.set_yticks(ypos)
ax.set_yticklabels([k[0] for k in keys], fontsize=10)
ax.set_xlabel("Two-sided p-value (log scale)")
style(ax)
h = [plt.Line2D([], [], marker="o", ls="", color=ORANGE, ms=9, label="p printed in the paper"),
plt.Line2D([], [], marker="s", ls="", color=BLUE, ms=8, label="p implied by the printed CI read as log-odds"),
plt.Line2D([], [], marker="D", ls="", mfc=BG, mec=BLUE, mew=2, ms=8, label="p implied by the printed CI read as an OR")]
ax.legend(handles=h, loc="lower left", bbox_to_anchor=(-0.4, 1.0), frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
# ---- design D: calculation flow for the headline abstract figure
def box(ax, x, y, w, h, txt, ec, fs=10.5, weight="normal"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.02",
fc=BG, ec=ec, lw=2))
ax.text(x + w / 2, y + h / 2, txt, ha="center", va="center", fontsize=fs, color=FG, weight=weight)
def arrow(ax, a, b, txt=None, off=(0, 0.03)):
ax.annotate("", xy=b, xytext=a, arrowprops=dict(arrowstyle="-|>", color=FG2, lw=1.6, mutation_scale=14))
if txt:
ax.text((a[0] + b[0]) / 2 + off[0], (a[1] + b[1]) / 2 + off[1], txt, ha="center", fontsize=10, color=FG2)
def design_D(path):
fig = plt.figure(figsize=(7.2, 4.0))
ax = fig.add_axes([0.02, 0.02, 0.96, 0.96])
ax.set_xlim(-0.02, 1.02); ax.set_ylim(0, 1); ax.axis("off")
lr = np.log(crude["OW_dep"])
b, lo, hi, _ = rep1["OW_dep"]
X = [(0.0, 0.34), (0.40, 0.25), (0.71, 0.29)]
top = ["Table 1, hsCRP ⩾3 mg/L\noverweight + depressed: 26/69\nnormal-weight controls: 3/57",
f"log odds ratio\nln[(26/43)/(3/54)]\n= {lr:.2f}",
f"crude odds ratio\n= {crude['OW_dep']:.1f}"]
bot = ["Abstract and Results print\n'OR 2.44 (1.28–3.94)'",
f"read as a log-odds\ncoefficient: {b:.2f}\n({lo:.2f} to {hi:.2f})",
f"odds ratio = exp(2.44)\n= {np.exp(b):.1f}\n(95% CI {np.exp(lo):.1f} to {np.exp(hi):.1f})"]
for (x, w), t in zip(X, top):
box(ax, x, 0.62, w, 0.27, t, BLUE, fs=10)
for (x, w), t, ec in zip(X, bot, [ORANGE, ORANGE, BLUE]):
box(ax, x, 0.08, w, 0.27, t, ec, fs=10)
for yy in (0.755, 0.215):
arrow(ax, (0.355, yy), (0.385, yy))
arrow(ax, (0.665, yy), (0.695, yy), "exp", off=(0, 0.035))
ax.annotate("", xy=(0.525, 0.365), xytext=(0.525, 0.605),
arrowprops=dict(arrowstyle="<|-|>", color=FG2, lw=1.6, mutation_scale=14))
ax.text(0.545, 0.485, f"{lr:.2f} v. {b:.2f}: same (log-odds) scale;\nthe printed value is also adjusted\nfor age and gender",
fontsize=10, color=FG2, va="center")
ax.text(0.0, 0.985, "Blue boxes: computed from the paper's own numbers. Orange boxes: as printed.", fontsize=10, color=FG2, va="top")
save(fig, path)
# ---- design E: dumbbell, printed headline 'ORs' v. exp(printed)
def design_E(path):
rows = [("Overweight + depressed v. normal-weight\ncontrols (abstract, model 1)", rep1["OW_dep"]),
("Overweight (abstract, model 2)", rep2["overweight"]),
("Depression (abstract, model 2)", rep2["depression"])]
fig, ax = plt.subplots(figsize=(7.2, 3.3))
ypos = np.arange(len(rows))[::-1].astype(float)
ax.axvline(1, color=FG2, lw=1)
for yy, (lab, (b, lo, hi, pr)) in zip(ypos, rows):
e = np.exp(b)
ax.annotate("", xy=(e / 1.08, yy), xytext=(b * 1.06, yy),
arrowprops=dict(arrowstyle="-|>", color=FG2, lw=1.4, mutation_scale=12))
ax.plot(b, yy, "o", ms=10, color=ORANGE, zorder=4)
ax.plot(e, yy, "s", ms=9.5, color=BLUE, zorder=4)
ax.text(b, yy + 0.27, f"{b:.2f}", ha="center", fontsize=10, color=ORANGE)
ax.text(e, yy + 0.27, f"{e:.1f}", ha="center", fontsize=10, color=BLUE)
ax.text(np.sqrt(b * e), yy - 0.33, f"x{e/b:.1f}", ha="center", fontsize=10, color=FG2)
ax.set_xscale("log")
ax.set_xlim(0.8, 20)
ax.set_xticks([1, 2, 5, 10, 20]); ax.set_xticklabels(["1", "2", "5", "10", "20"])
ax.set_yticks(ypos); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-0.7, len(rows) - 0.3)
ax.set_xlabel("Odds ratio (log scale)")
style(ax)
h = [plt.Line2D([], [], marker="o", ls="", color=ORANGE, ms=10, label="'OR' printed in the abstract"),
plt.Line2D([], [], marker="s", ls="", color=BLUE, ms=9.5, label="exp(printed value): the OR if it is a log-odds coefficient")]
ax.legend(handles=h, loc="lower left", bbox_to_anchor=(-0.45, 1.0), frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
designs = [("A_printed_scale_two_nulls", design_A), ("B_odds_ratio_axis", design_B),
("C_pvalue_consistency", design_C), ("D_calculation_flow", design_D),
("E_headline_dumbbell", design_E)]
for i, (nm, f) in enumerate(designs, 1):
f(os.path.join(CAND, f"cand{i}_{nm}"))
design_A(os.path.join(HERE, "fig1"))
design_B(os.path.join(HERE, "fig2"))
# contact sheet of the five candidates
imgs = [plt.imread(os.path.join(CAND, f"cand{i}_{nm}.png")) for i, (nm, _) in enumerate(designs, 1)]
fig, axs = plt.subplots(3, 2, figsize=(14.4, 15))
for i, ax in enumerate(axs.flat):
ax.axis("off")
if i < len(imgs):
ax.imshow(imgs[i])
ax.set_title(f"Candidate {i+1}: {designs[i][0][2:].replace('_', ' ')}", fontsize=13, loc="left")
fig.tight_layout()
fig.savefig(os.path.join(CAND, "contact_sheet.png"), dpi=110)
plt.close(fig)
print("\nFigures written: fig1.svg/png, fig2.svg/png, candidates/cand1..5 (.svg/.png), candidates/contact_sheet.png")
Reported positive predictive value of 55% conflicts with the paper's own sensitivity, specificity and prevalence
Sci Adv (2023) · doi:10.1126/sciadv.adg7545 · PMC10732528
Summary
The paper reports that its serial panel of three bivariate miRNA ratios separates preeclampsia (PE) cases from controls with sensitivity 93%, specificity 79%, positive predictive value (PPV) 55% and negative predictive value (NPV) 89%. It states these figures in the Abstract, the Results and the Discussion, and the Results and Discussion add "given a 57.7% prevalence of PE in our study" (71 of 123 samples). At that prevalence, a sensitivity of 93% and a specificity of 79% give a PPV of 85.8%. The per-sample classifications in the paper's data file S5 give 66 true positives and 11 false positives, so PPV = 66/77 = 85.7%. The reported NPV and both likelihood ratios agree with the same inputs; only the PPV does not. The error understates the panel's performance in a headline figure, and it affects the paper's comparison with the sFlt1:PlGF ratio, whose main limitation the paper gives as a low PPV.
What the paper reports
Abstract:
Applying an iterative machine learning method identified three bivariate miRNA biomarkers (miR-522-3p/miR-4732-5p, miR-516a-5p/miR-144-3p, and miR-27b-3p/let-7b-5p) that, when applied serially, distinguished between PE cases of different severity and differentiated cases from controls with a sensitivity of 93%, specificity of 79%, positive predictive value (PPV) of 55%, and negative predictive value (NPV) of 89%.
Results, "Identification of a small panel of candidate bivariate biomarkers":
We then re-ran the algorithm an additional two times, correctly identifying all but five PE cases, three of which were diagnosed as superimposed PE and one mild PE (Fig. 7A).
This iterative panel of three bivariate biomarkers achieved a PPV of 55% at a sensitivity of 93%, specificity of 79%, a positive likelihood ratio (+LR) of 4.43, and a negative likelihood ratio (-LR) of 0.09, given a 57.7% prevalence of PE in our study.
Discussion:
Overall, this iterative process was able to identify over 90% of cases with a false-positive rate of less than 10% at each iteration and achieved a PPV of 55% at a sensitivity of 93% and specificity of 79% (+LR = 4.43, -LR = 0.09) given a prevalence of PE of 57.7% in our study.
Results, "Characteristics of the study population" (the cohort the panel was applied to):
Cases were defined as pregnancies with an adjudicated PE diagnosis (n = 71), and controls (n = 52) were from participants with either hypertension that did not meet the criteria for PE or normal (nonhypertensive) pregnancy outcomes. The pass-filter 71 cases and 52 controls were subjected to further analysis.
Materials and Methods, "Candidate bivariate marker selection" (the definition used):
PPV was calculated as PPV = True Positives/(True Positives + False Positives), and NPV was calculated as NPV = True Negatives/(True Negatives + False Negatives).
Introduction (the comparison the PPV feeds into):
The primary limitation of the sFlt1:PlGF ratio is its relatively low positive predictive value (PPV), which numerous studies have reported to be between 30 and 65% (9–12).
What the paper's own numbers show
Method. I recomputed the PPV in two independent ways. (1) From the paper's rounded rates, using Bayes' rule, PPV = Se·p / (Se·p + (1 − Sp)(1 − p)), with Se = 0.93, Sp = 0.79 and p = 0.577. The same formula is stated in the medRxiv preprint of this paper. (2) From counts. Data file S5 ("Refers to Figure 7") lists all 123 samples with the iteration (1, 2 or 3) at which each one was flagged as PE, or "Not Removed". Flagged cases are true positives, unflagged cases false negatives, flagged controls false positives and unflagged controls true negatives. I applied the Methods definitions to those counts. I also computed what a PPV of 55% would require, both in prevalence and in false positives. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Prevalence | Results; Discussion | 57.7% | 71/123 = 57.72% |
| Classification counts | Data file S5 | cases flagged at iterations 1/2/3: 40/15/11, not flagged 5; controls flagged 3/4/4, not flagged 41 | TP = 66, FN = 5, FP = 11, TN = 41 |
| Sensitivity | Abstract; Results; Discussion | 93% | 66/71 = 92.96% |
| Specificity | Abstract; Results; Discussion | 79% | 41/52 = 78.85% |
| PPV | Abstract; Results; Discussion | 55% | 85.8% from the rounded rates; 66/77 = 85.71% from the counts |
| NPV | Abstract | 89% | 89.2% from the rounded rates; 41/46 = 89.13% from the counts |
| +LR | Results; Discussion | 4.43 | 0.93/0.21 = 4.43; 4.39 from the counts |
| −LR | Results; Discussion | 0.09 | 0.07/0.79 = 0.089; 0.089 from the counts |
| PPV if specificity were anywhere from 38/52 to 44/52 (TP fixed at 66) | derived | — | 82.5% to 89.2% |
| Prevalence at which Se 93% and Sp 79% give PPV 55% | derived | — | 21.6%; the NPV there would be 97.6%, not 89% |
| False positives needed for PPV 55% with 66 true positives | derived | — | 66/0.55 − 66 = 54, more than the 52 controls |
| PPV of the miRNA panel alone in the 70-patient sFlt1:PlGF subset (42 TP, 6 FP) | Results | not stated | 42/48 = 87.5% |
| Combined miRNA then sFlt1:PlGF (42 TP, 2 FP, 21 TN, 5 FN) | Abstract; Results | Se 89.4%, Sp 91.3%, PPV 95.5%, NPV 80.8% | 89.36%, 91.3%, 95.5%, 80.8% (consistent) |
| Preprint inputs (prevalence 57.8%, Sp 78.8%) | medRxiv preprint, Methods | PPV 55% | 85.7% with the preprint's own formula |
Every other metric of the panel agrees with its inputs to the printed precision. The PPV is the only one that does not, and it is about 31 percentage points below the value the paper's own numbers give.
Figure
Figure 1. The blue curve is the PPV that a test with the paper's reported sensitivity (93%) and specificity (79%) has at each prevalence (Bayes' rule). The thin vertical line marks the study prevalence the paper states, 71/123 = 57.7%. At that prevalence the curve gives 85.8% (blue diamond), and the counts in data file S5 give 66/77 = 85.7%. The paper's reported 55% (orange circle) lies well below the curve. The curve reaches 55% only at a prevalence of 21.6% (open circle), and at that prevalence the NPV would be 97.6%, not the 89% the paper reports. All values come from the paper or from recompute.py.txt.
Figure 2. The count reconstruction from data file S5. Summing the cases and controls flagged at each of the three iterations gives TP 66, FN 5, FP 11 and TN 41. These counts reproduce the reported sensitivity, specificity and NPV, but give a PPV of 85.7% against the reported 55%. The lower panel shows that a 55% PPV would need 54 false positives, more than the 52 controls in the cohort.
Why it matters
- Headline figure. The PPV is one of four performance figures in the Abstract for the paper's main diagnostic result, and it is repeated in the Results and the Discussion. Readers who cite the panel's performance will cite 55%.
- Comparison with sFlt1:PlGF. The Introduction gives the sFlt1:PlGF ratio's "relatively low positive predictive value", 30 to 65%, as its main limitation. At 55% the miRNA panel appears no better on this measure. At about 86% it appears better, although PPVs from cohorts with different prevalence cannot be compared directly, and this cohort's prevalence (57.7%) is high for a triage population.
- Gain from combining assays. The paper reports that applying sFlt1:PlGF after the miRNA panel gives a PPV of 95.5%. Against a stated baseline of 55% this looks like a large gain. The actual baseline is about 86% in the full cohort and 87.5% in the 70-patient subset that has sFlt1:PlGF data, so the gain is about 8 to 10 percentage points.
- What is not affected. The reported sensitivity, specificity, NPV and likelihood ratios are consistent with the data. So are the combined-assay counts and metrics, the identities of the three biomarkers and the call for larger validation. The error makes the panel look worse than the paper's own data show, not better.
Explanations considered
- A different prevalence. A PPV is sometimes recalculated for an external, lower prevalence. This would explain 55% if the paper had used a prevalence of about 21.6%. But the paper explicitly attaches the 55% to "a 57.7% prevalence of PE in our study", which is 71/123, the full cohort the panel was applied to. At 21.6% the NPV would be 97.6%, not the reported 89%, so the reported NPV and PPV cannot both come from the same adjusted prevalence. This does not account for the numbers.
- A different definition of PPV. The Methods define PPV as TP/(TP + FP), the standard formula. I tried other ratios that could be mislabeled: TP/(TP + FP + FN) = 80.5%, accuracy = 87.0%, TN/(TN + FP + FN) = 71.9%, TP/N = 53.7%, 1 − PPV = 14.3% and the false-positive rate = 21.2%. None gives 55%. TP/N (53.7%) is the closest, but it rounds to 54% and is not a predictive value. This does not account for the numbers.
- Different integer counts. The sensitivity is pinned by the text ("all but five PE cases", so 66 of 71). Any specificity count from 38/52 to 44/52 gives a PPV between 82.5% and 89.2%. A PPV of 55% with 66 true positives would need 54 false positives, more than the 52 controls. Data file S5 gives the exact counts (FP = 11). This does not account for the numbers.
- A subset or a different stage of the iteration. In the 70-patient subset with sFlt1:PlGF data, the miRNA panel alone has 42 true and 6 false positives (PPV 87.5%). The three iterations in data file S5, taken one at a time, give PPVs of 40/43 = 93.0%, 15/19 = 78.9% and 11/15 = 73.3%. None of these is near 55%. This does not account for the numbers.
- A conversion or typesetting artifact. The 55% appears in running text in three places, in the PMC archive, the Europe PMC full-text XML and the medRxiv preprint. It is not in a table, so a misaligned column is ruled out, and it is not a lost minus sign or decimal. The preprint states the Bayes formula and its inputs (prevalence 57.8%, specificity 78.8%). Those inputs give 85.7% with that formula, so the 55% was already present in the preprint and was carried into the published version unchanged. This does not account for the numbers.
None of these explanations reconciles 55% with the paper's other figures. I have not found what calculation produced it.
A related observation (not part of the main claim). For the 18-patient independent validation cohort (11 cases, 7 controls), the Results report "a PPV of 85% at a sensitivity of 91% and specificity of 57%". A sensitivity of 91% with 11 cases is 10/11, which agrees with the Discussion's "we found all but one PE case". A specificity of 57% with 7 controls is 4/7, which leaves 3 false positives. That gives PPV = 10/13 = 76.9%. A PPV of 85% (11/13 = 84.6%) would require all 11 cases to be detected. Data file S6 gives only the cohort's demographics, not per-sample calls, so I could not check this from counts.
Status
As of 2026-10-23:
- Version of record. science.org returned HTTP 403 to automated requests, so I checked the publisher-deposited text instead. The PMC article page, the Europe PMC full-text XML and the local PMC archive all contain the three sentences quoted above word for word.
- Supplements. I checked the supplementary PDF (Figs. S1 to S3) and data files S1 to S11 (Excel). Data file S5 gives the per-sample classifications used above. It does not mention a PPV.
- Preprint. The medRxiv preprint (doi:10.1101/2023.01.26.23285072, posted 2023-01-29) also reports a PPV of 55% at sensitivity 93% and prevalence 57.7%. Its Methods give the Bayes formula, which yields about 86% with its stated inputs.
- Corrections and retractions. Crossref update metadata, PMC notices and Europe PMC notices list no correction, expression of concern or retraction.
- PubPeer. PubPeer's publication lookup for the DOI returned no comments. Its web search page refused automated access (HTTP 403). A general web search found no discussion of this error.
Sources: article (DOI); PMC10732528; medRxiv preprint; supplementary files via the Europe PMC supplementaryFiles service.
Suggested correction
In the Abstract, the Results ("Identification of a small panel of candidate bivariate biomarkers") and the Discussion, the PPV of the three-bivariate panel in the 123-sample cohort should read 86% (66/77 = 85.7% from data file S5; 85.8% from the rounded sensitivity, specificity and prevalence) instead of 55%. Sensitivity, specificity, NPV, +LR and −LR need no change. If the 55% was meant to describe a different population or prevalence, the text should say which one, and the NPV should then be recomputed for it. The authors may also wish to check the validation-cohort PPV of 85% in the Results. With the stated sensitivity (10/11) and specificity (4/7) it computes to 10/13 = 77%.
Recomputation script (python recompute.py.txt)
# Recompute the diagnostic metrics of the three-bivariate miRNA panel in
# Sci Adv 2023, doi:10.1126/sciadv.adg7545 (PMC10732528), and draw the figures.
#
# Run: python recompute.py.txt
# Every input below is a number printed in the article or in its data file S5
# (per-sample iteration at which each of the 123 samples was flagged as PE).
# Outputs: fig1.svg/png, fig2.svg/png and five candidate designs in candidates/.
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
import numpy as np
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs
CASES, CONTROLS = 71, 52 # Results, "Characteristics of the study population"
N = CASES + CONTROLS
REP = dict(se=0.93, sp=0.79, ppv=0.55, npv=0.89, plr=4.43, nlr=0.09, prev=0.577) # Abstract/Results/Discussion
# Data file S5: iteration at which each sample was flagged (1, 2, 3) or "Not Removed"
S5 = {"case": {1: 40, 2: 15, 3: 11, "not": 5}, "control": {1: 3, 2: 4, 3: 4, "not": 41}}
TP = sum(v for k, v in S5["case"].items() if k != "not")
FN = S5["case"]["not"]
FP = sum(v for k, v in S5["control"].items() if k != "not")
TN = S5["control"]["not"]
assert TP + FN == CASES and FP + TN == CONTROLS
def ppv_f(se, sp, p):
return se * p / (se * p + (1 - sp) * (1 - p))
def npv_f(se, sp, p):
return sp * (1 - p) / (sp * (1 - p) + (1 - se) * p)
def pct(x, d=1):
return f"{100 * x:.{d}f}%"
print("=== 1. Prevalence")
prev = CASES / N
print(f"prevalence = {CASES}/{N} = {pct(prev, 2)} (reported 57.7%)")
print("\n=== 2. From the paper's rounded rates (Se 93%, Sp 79%, prevalence 57.7%)")
se, sp, p = REP["se"], REP["sp"], REP["prev"]
ppv_r, npv_r = ppv_f(se, sp, p), npv_f(se, sp, p)
plr_r, nlr_r = se / (1 - sp), (1 - se) / sp
print(f"PPV = {se}*{p} / ({se}*{p} + {1-sp:.2f}*{1-p:.3f}) = {pct(ppv_r)} (reported 55%)")
print(f"NPV = {pct(npv_r)} (reported 89%) +LR = {plr_r:.2f} (reported 4.43) -LR = {nlr_r:.3f} (reported 0.09)")
print("\n=== 3. From the counts in data file S5")
se_c, sp_c = TP / CASES, TN / CONTROLS
ppv_c, npv_c = TP / (TP + FP), TN / (TN + FN)
plr_c, nlr_c = se_c / (1 - sp_c), (1 - se_c) / sp_c
print(f"cases flagged at iterations 1/2/3 = {S5['case'][1]}/{S5['case'][2]}/{S5['case'][3]}, not flagged = {FN}")
print(f"controls flagged at iterations 1/2/3 = {S5['control'][1]}/{S5['control'][2]}/{S5['control'][3]}, not flagged = {TN}")
print(f"TP={TP} FN={FN} FP={FP} TN={TN}")
print(f"Se = {TP}/{CASES} = {pct(se_c, 2)} Sp = {TN}/{CONTROLS} = {pct(sp_c, 2)}")
print(f"PPV = {TP}/{TP+FP} = {pct(ppv_c, 2)} NPV = {TN}/{TN+FN} = {pct(npv_c, 2)}")
print(f"+LR = {plr_c:.2f} -LR = {nlr_c:.3f}")
print(f"gap between recomputed and reported PPV = {100*(ppv_c-REP['ppv']):.1f} percentage points")
for it in (1, 2, 3):
t, f_ = S5["case"][it], S5["control"][it]
print(f" iteration {it} alone: PPV = {t}/{t+f_} = {pct(t/(t+f_))}")
print("\n=== 4. Sensitivity of the PPV to the exact specificity count (TP fixed at 66)")
sweep = []
for tn in range(38, 45):
fp = CONTROLS - tn
sweep.append(TP / (TP + fp))
print(f" TN={tn} FP={fp}: Sp={pct(tn/CONTROLS)} PPV={pct(TP/(TP+fp))}")
print(f"lowest PPV in this range = {pct(min(sweep))}")
print("\n=== 5. What would a PPV of 55% require?")
target = REP["ppv"]
p55 = target * (1 - sp) / (se * (1 - target) + target * (1 - sp))
print(f"prevalence giving PPV 55% at Se 93%/Sp 79% = {pct(p55)}; NPV at that prevalence = {pct(npv_f(se, sp, p55))} (reported 89%)")
fp55 = TP / target - TP
print(f"false positives needed with TP={TP}: {fp55:.1f} (only {CONTROLS} controls exist)")
print("\n=== 6. Other ratios tried, none gives 55%")
print(f"TP/(TP+FP+FN) = {pct(TP/(TP+FP+FN))} accuracy = {pct((TP+TN)/N)} TN/(TN+FP+FN) = {pct(TN/(TN+FP+FN))}")
print(f"1 - PPV = {pct(1-ppv_c)} FP/(FP+TN) = {pct(FP/CONTROLS)} TP/N = {pct(TP/N)}")
print("\n=== 7. Preprint inputs (medRxiv 10.1101/2023.01.26.23285072: prevalence 57.8%, Sp 78.8%)")
print(f"PPV formula stated in the preprint, applied to Se 0.93, Sp 0.788, p 0.578 = {pct(ppv_f(0.93, 0.788, 0.578))}")
print("\n=== 8. sFlt1:PlGF subset (n = 70), Results")
subset = {"miRNA panel alone": (42, 6, 17, 5),
"miRNA then sFlt1:PlGF": (42, 2, 21, 5),
"sFlt1:PlGF only on miRNA positives": (39, 0, 23, 8)}
for lab, (tp, fp, tn, fn) in subset.items():
print(f" {lab}: Se={pct(tp/(tp+fn), 2)} Sp={pct(tn/(tn+fp))} PPV={pct(tp/(tp+fp))} NPV={pct(tn/(tn+fn))}")
print(f" PPV gain from adding sFlt1:PlGF: {100*(0.955-ppv_c):.1f} points vs full-cohort {pct(ppv_c)}, "
f"{100*(42/44-42/48):.1f} points vs subset {pct(42/48)}")
print(" (reported for 'then': Se 89.4%, Sp 91.3%, PPV 95.5%, NPV 80.8%; for 'only on positives': Se 82.9%, Sp 100%)")
print("\n=== 9. Side observation: independent validation cohort (11 cases, 7 controls)")
vtp, vfn, vtn, vfp = 10, 1, 4, 3 # Se 91% = 10/11 ('all but one PE case'), Sp 57% = 4/7
print(f"Se={pct(vtp/11)} Sp={pct(vtn/7)} PPV={vtp}/{vtp+vfp} = {pct(vtp/(vtp+vfp))} (reported 85%); 11/13 = {pct(11/13)}")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "axes.edgecolor": INK2, "xtick.color": INK2, "ytick.color": INK2,
"axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
"axes.titlesize": 11.5, "axes.titleweight": "bold", "axes.titlelocation": "left",
"svg.fonttype": "none", "lines.linewidth": 2,
})
def style_axes(ax, grid_axis="both"):
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.grid(True, axis=grid_axis, color=GRID, linewidth=0.8)
ax.set_axisbelow(True)
def save(fig, path):
fig.savefig(path + ".svg")
fig.savefig(path + ".png", dpi=200)
plt.close(fig)
print("wrote", os.path.relpath(path, HERE) + ".svg/.png")
print("\n=== figures")
# A. Dot comparison: reported vs recomputed for the four percentage metrics
def fig_A(path):
fig, ax = plt.subplots(figsize=(W, 3.8))
rows = [("Sensitivity", REP["se"], se_c, f"{TP}/{CASES}"),
("Specificity", REP["sp"], sp_c, f"{TN}/{CONTROLS}"),
("NPV", REP["npv"], npv_c, f"{TN}/{TN+FN}"),
("PPV", REP["ppv"], ppv_c, f"{TP}/{TP+FP}")]
d = 0.14
for i, (lab, r, c, frac) in enumerate(rows):
y = len(rows) - 1 - i
ax.scatter(r * 100, y + d, s=80, color=ORANGE, marker="o", zorder=3, edgecolor=BG, linewidth=1.2)
ax.scatter(c * 100, y - d, s=80, color=BLUE, marker="D", zorder=4, edgecolor=BG, linewidth=1.2)
if lab == "PPV":
ax.text(r * 100 - 2.5, y + d, f"reported {r*100:.0f}%", ha="right", va="center", fontsize=10)
ax.text(c * 100 - 2.5, y - d, f"{frac} = {c*100:.1f}%", ha="right", va="center", fontsize=10)
else:
ax.text(min(r, c) * 100 - 2.5, y, f"reported {r*100:.0f}%; {frac} = {c*100:.1f}%",
ha="right", va="center", fontsize=10, color=INK2)
ax.set_yticks(range(len(rows)))
ax.set_yticklabels([r[0] for r in rows][::-1])
ax.set_xlim(0, 100)
ax.set_ylim(-0.5, len(rows) - 0.5)
ax.set_xlabel("Percent (whole cohort: 71 cases, 52 controls)")
style_axes(ax, "x")
ax.scatter([], [], s=70, color=ORANGE, marker="o", label="Reported in the paper")
ax.scatter([], [], s=70, color=BLUE, marker="D", label="Recomputed from data file S5 counts")
ax.legend(loc="upper center", bbox_to_anchor=(0.45, -0.2), ncol=2, frameon=False, fontsize=10)
ax.set_title("Only the PPV disagrees with the panel's own classification counts")
fig.tight_layout()
save(fig, path)
# B. PPV as a function of prevalence at the reported Se/Sp
def fig_B(path):
fig, ax = plt.subplots(figsize=(W, 4.3))
ps = np.linspace(0.005, 0.995, 400)
ax.plot(ps * 100, ppv_f(se, sp, ps) * 100, color=BLUE, lw=2, zorder=2,
label="PPV implied by Se 93% and Sp 79%")
ax.axvline(p * 100, color=INK2, lw=0.8, zorder=1)
ax.text(p * 100 + 1, 3, f"study prevalence\n71/123 = {prev*100:.1f}%", color=INK2, fontsize=10, va="bottom")
ax.plot([p55 * 100, p * 100], [55, 55], color=INK2, lw=1, ls=(0, (3, 3)), zorder=1)
ax.scatter(p55 * 100, 55, s=64, facecolor=BG, edgecolor=BLUE, linewidth=2, zorder=4)
ax.annotate(f"the curve reaches 55% only at a prevalence\nof {p55*100:.1f}%, where NPV would be "
f"{npv_f(se, sp, p55)*100:.1f}%, not 89%",
(p55 * 100, 55), xytext=(8, -10), textcoords="offset points", ha="left", va="top",
fontsize=10, color=INK2)
ax.scatter(p * 100, ppv_r * 100, s=110, color=BLUE, marker="D", zorder=5, edgecolor=BG, linewidth=1.5,
label="Recomputed PPV at 57.7%")
ax.annotate(f"recomputed {ppv_r*100:.1f}%\n(counts: {TP}/{TP+FP} = {ppv_c*100:.1f}%)", (p * 100, ppv_r * 100),
xytext=(12, -8), textcoords="offset points", ha="left", va="top", fontsize=10)
ax.scatter(p * 100, 55, s=110, color=ORANGE, marker="o", zorder=5, edgecolor=BG, linewidth=1.5,
label="Reported PPV at 57.7%")
ax.annotate("reported 55%", (p * 100, 55), xytext=(10, 0), textcoords="offset points",
ha="left", va="center", fontsize=10)
ax.set_xlim(0, 100); ax.set_ylim(0, 100)
ax.set_xlabel("Prevalence of preeclampsia (%)")
ax.set_ylabel("Positive predictive value (%)")
style_axes(ax)
ax.legend(loc="upper left", frameon=False, fontsize=10)
ax.set_title("At the study's prevalence, Se 93% and Sp 79% give a PPV of 86%")
fig.tight_layout()
save(fig, path)
def box(ax, x, y, w, h, text, edge, fs=10.5, weight="normal", fc=BG, color=INK):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.006,rounding_size=0.012",
linewidth=2, edgecolor=edge, facecolor=fc))
if text:
ax.text(x + w / 2, y + h / 2, text, ha="center", va="center", fontsize=fs, fontweight=weight,
color=color, linespacing=1.3)
def arrow(ax, a, b, color=INK2):
ax.add_patch(FancyArrowPatch(a, b, arrowstyle="-|>", mutation_scale=13, lw=1.5, color=color))
# C. Calculation flow from data file S5 counts
def fig_C(path):
fig = plt.figure(figsize=(W, 4.2))
ax = fig.add_axes([0.02, 0.02, 0.96, 0.96])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
box(ax, 0.005, 0.56, 0.19, 0.34, "Data file S5\n123 samples,\neach with the\niteration that\nflagged it as PE", INK2, fs=10)
box(ax, 0.235, 0.75, 0.30, 0.19,
f"71 cases\nflagged {S5['case'][1]}+{S5['case'][2]}+{S5['case'][3]} = {TP} (TP)\nnot flagged: {FN} (FN)", BLUE, fs=10)
box(ax, 0.235, 0.52, 0.30, 0.19,
f"52 controls\nflagged {S5['control'][1]}+{S5['control'][2]}+{S5['control'][3]} = {FP} (FP)\nnot flagged: {TN} (TN)", BLUE, fs=10)
arrow(ax, (0.20, 0.78), (0.23, 0.83)); arrow(ax, (0.20, 0.68), (0.23, 0.62))
arrow(ax, (0.545, 0.73), (0.575, 0.73))
ax.text(0.59, 0.96, "Recomputed", fontsize=10, color=BLUE, fontweight="bold", va="center")
ax.text(0.995, 0.96, "Reported", fontsize=10, color=ORANGE, fontweight="bold", va="center", ha="right")
outs = [(0.87, f"Se {TP}/{CASES} = {se_c*100:.1f}%", "93% (matches)", True),
(0.77, f"Sp {TN}/{CONTROLS} = {sp_c*100:.1f}%", "79% (matches)", True),
(0.67, f"NPV {TN}/{TN+FN} = {npv_c*100:.1f}%", "89% (matches)", True),
(0.57, f"PPV {TP}/{TP+FP} = {ppv_c*100:.1f}%", "55% (no match)", False)]
for y, t, r, ok in outs:
ax.text(0.59, y, t, fontsize=10, color=INK, va="center")
ax.text(0.995, y, r, fontsize=10, ha="right", va="center", color=INK2 if ok else INK)
ax.add_patch(FancyBboxPatch((0.585, 0.53), 0.405, 0.08, boxstyle="round,pad=0.004,rounding_size=0.01",
linewidth=1.5, edgecolor=ORANGE, facecolor="none"))
ax.plot([0.005, 0.995], [0.42, 0.42], color=GRID, lw=1)
ax.text(0.005, 0.34, "What a PPV of 55% would require", fontsize=10.5, fontweight="bold", color=INK, va="center")
ax.text(0.005, 0.23, f"With the same {TP} true positives: {TP}/0.55 - {TP} = {fp55:.0f} false positives, "
f"but there are only {CONTROLS} controls.", fontsize=10, color=INK, va="center")
ax.text(0.005, 0.13, f"With Se 93% and Sp 79%: a prevalence of {p55*100:.1f}% (the paper states "
f"{REP['prev']*100:.1f}%),", fontsize=10, color=INK, va="center")
ax.text(0.005, 0.05, f"at which the NPV would be {npv_f(se, sp, p55)*100:.1f}% rather than the reported 89%.",
fontsize=10, color=INK, va="center")
save(fig, path)
# D. False positives observed vs required
def fig_D(path):
fig, ax = plt.subplots(figsize=(W, 2.8))
vals = [("Observed\n(data file S5)", FP, BLUE, None, f"{FP} PPV {TP}/{TP+FP} = {ppv_c*100:.1f}%"),
("Needed for the\nreported PPV of 55%", fp55, ORANGE, "//", f"{fp55:.0f} PPV {TP}/{TP+fp55:.0f} = 55%")]
for i, (lab, v, c, h, txt) in enumerate(vals):
y = 1 - i
ax.barh(y, v, height=0.46, color=c if h is None else BG, edgecolor=c, hatch=h, linewidth=2)
ax.text(v + 1.5, y, txt, va="center", ha="left", fontsize=10)
ax.axvline(CONTROLS, color=INK, lw=1)
ax.text(CONTROLS - 1, 1.48, f"all {CONTROLS} controls", ha="right", va="center", fontsize=10, color=INK2)
ax.set_yticks([1, 0]); ax.set_yticklabels([v[0] for v in vals])
ax.set_xlim(0, 100); ax.set_ylim(-0.45, 1.7)
ax.set_xlabel(f"Controls classified as preeclampsia (true positives fixed at {TP})")
style_axes(ax, "x")
fig.tight_layout(rect=(0, 0, 1, 0.9))
fig.suptitle(f"A 55% PPV would need {fp55:.0f} false positives; the cohort has only {CONTROLS} controls",
x=0.01, ha="left", fontsize=11.5, fontweight="bold")
save(fig, path)
# E. Text panels: the three reported locations vs the paper's own inputs
def fig_E(path):
fig = plt.figure(figsize=(W, 4.6))
ax = fig.add_axes([0.02, 0.02, 0.96, 0.96])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
quotes = [("Abstract", "“...sensitivity of 93%, specificity of 79%,\npositive predictive value (PPV) of 55%,\nand negative predictive value (NPV) of 89%.”"),
("Results (Fig. 7 paragraph)", "“...achieved a PPV of 55% at a sensitivity of\n93%, specificity of 79%, ... (+LR) of 4.43, ...\n(-LR) of 0.09, given a 57.7% prevalence...”"),
("Discussion", "“...achieved a PPV of 55% at a sensitivity of\n93% and specificity of 79% (+LR = 4.43,\n-LR = 0.09) given a prevalence of PE of 57.7%...”")]
ys = [0.65, 0.345, 0.04]
for (h, q), y in zip(quotes, ys):
box(ax, 0.005, y, 0.53, 0.26, "", ORANGE)
ax.text(0.02, y + 0.215, h, fontsize=10.5, fontweight="bold", va="center", color=INK)
ax.text(0.02, y + 0.095, q, fontsize=10, va="center", color=INK, linespacing=1.3)
bx, by, bw, bh = 0.60, 0.22, 0.385, 0.56
box(ax, bx, by, bw, bh, "", BLUE)
ax.text(bx + 0.02, by + bh - 0.05, "The paper's own inputs", fontsize=10.5, fontweight="bold", va="center")
lines = [("Se 0.93, Sp 0.79,", False), ("prevalence 71/123 = 0.577", False),
(f"PPV = {ppv_r*100:.1f}% (reported 55%)", True),
(f"NPV = {npv_r*100:.1f}% (reported 89%)", False),
(f"+LR = {plr_r:.2f} (reported 4.43)", False),
(f"-LR = {nlr_r:.3f} (reported 0.09)", False),
(f"S5 counts: PPV {TP}/{TP+FP} = {ppv_c*100:.1f}%", True)]
for k, (t, b) in enumerate(lines):
ax.text(bx + 0.02, by + bh - 0.12 - k * 0.062, t, fontsize=10, va="center",
fontweight="bold" if b else "normal", color=INK)
for y in ys:
ax.add_patch(FancyArrowPatch((0.54, y + 0.13), (bx - 0.005, 0.50), arrowstyle="-|>", mutation_scale=12,
lw=1.3, color=INK2))
ax.text(bx + bw / 2, 0.86, "same 55% in all three places;\nonly the PPV fails the check", fontsize=10,
color=INK2, ha="center", va="center")
ax.text(0.005, 0.975, "Orange boxes: the paper's text", fontsize=10, color=INK2, va="center")
ax.text(0.995, 0.975, "Blue box: recomputed", fontsize=10, color=INK2, va="center", ha="right")
save(fig, path)
fig_A(os.path.join(CAND, "A_metric_dots"))
fig_B(os.path.join(CAND, "B_ppv_vs_prevalence"))
fig_C(os.path.join(CAND, "C_count_flow"))
fig_D(os.path.join(CAND, "D_false_positives_needed"))
fig_E(os.path.join(CAND, "E_text_panels"))
# provisional choice for the write-up, pending a pick among candidates/A-E
fig_B(os.path.join(HERE, "fig1"))
fig_C(os.path.join(HERE, "fig2"))
Reported CO2 storage capacities per km³ are about 920 times what the paper's inputs give
Sci Adv (2024) · doi:10.1126/sciadv.adq0515 · PMC11566992
Summary
The paper's only quantitative estimate of reservoir storage capacity is a first-order calculation. It fills the 5.7% effective porosity of a basalt reservoir with a fluid carrying 0.65 M CO2 (6.5 M once the chelating agent GLDA has fully degraded) and reports capacities of 1.5 and 15 Gt CO2 per km³. Multiplying the paper's own inputs (10⁹ m³ × 0.057 × 650 mol/m³ × 44.01 g/mol) gives 0.0016 and 0.016 Gt per km³ (1.6 and 16 Mt). The reported values are about 920 times larger, which fits a megatonne-to-gigatonne unit slip, and they would require a pore volume 52 times the volume of the rock. The same inflated figure produces the claim that "up to 20 Gt of CO2 annually" fits in 1.3 km³ (1/25,000 of Iceland's active rift zone): with the paper's inputs, 1.3 km³ holds about 0.021 Gt, and the stated "two orders of magnitude higher" comparison with the cited 0.18 Gt/km³ becomes about 11 times lower.
What the paper reports
All passages are from the article as archived in PubMed Central. The archive text writes km3 and CO2 for km³ and CO₂. Where the paper names the cited study (its reference 17) by author, the name is replaced here by "[the cited study]"; everything else is verbatim.
The inputs:
"Specifically, the total porosity ranged from 10.99 to 15.23%, of which only 18.7 to 57.5% were connected porosity, resulting in a mean effective porosity of 4.8%. Assuming an enhancement in pore properties comparable to that observed in the experiment with injection of the acidic GLDA solution, the effective porosity was anticipated to increase from 4.8 to 5.7%." (Results, "CO2 mineralization with GLDA biodegradation and overall evaluation", para 2)
"if a solution containing 20 wt % GLDA-Na4 (C9H9NNa4O8) was used for injection, then CO2 concentration is expected to increase 10-fold, from 0.65 M to 0.65 + 0.65 × 9 (number of carbon atoms in GLDA) = 6.5 M." (Same subsection, para 1)
The capacity calculation and the claims built on it:
"When filling all effective porosity with a CO2-charged 20 wt % GLDA-Na4 seawater solution (CO2 concentration estimated as 0.65 M), the reservoir exhibited a CO2 storage capacity of 1.5 Gt/km3." (Same subsection, para 2)
"With complete degradation of GLDA, the concentration of CO2 in the fluid would attain a value of 6.5 M, resulting in a reservoir capable of storing 15 Gt of CO2/km3, two orders of magnitude higher than the value reported by [the cited study] (17) based on CO2 capture and storage using fresh water (a mean value of 0.18 Gt of CO2/km3)." (Same subsection, para 2)
"Up to 20 Gt of CO2 annually could potentially be sequestered in a 1.3 km3 reservoir volume, which is 1/25,000 of the active rift zone in Iceland (area of 33,000 km2, thickness of 1 km)." (Same subsection, para 2)
The abstract's closing claim:
"Promising CO2 storage and mineralization capacities two orders higher than current techniques, this approach reduces required reservoir volume while enhancing efficiency." (Abstract)
The paper uses M for mol/L throughout. For example, the Methods give "~0.0325, 0.065, 0.162, 0.325, and 0.65 M" for 1 to 20 wt % GLDA solutions (Materials and Methods, "Powder experiments", para 2).
What the paper's own numbers show
Method. The paper's stated rule is to fill all effective porosity of the rock with fluid at the stated CO2 concentration. For 1 km³ of rock (10⁹ m³) this gives:
capacity = 10⁹ m³ × effective porosity × concentration (mol/L × 1,000 L/m³) × 44.01 g/mol ÷ 10¹⁵ g/Gt.
The only constant in the capacity calculation not taken from the paper is the molar mass of CO2. One ceiling uses an assumed dense-CO2 density of 700 kg/m³, and one comparison uses an assumed basalt density of 2,950 kg/m³. recompute.py.txt evaluates the expression, inverts it to find the porosity the reported values would need, and recomputes the 1.3 km³ and 1/25,000 figures. It also takes the cited 0.18 Gt/km³ back to the cited study's own tables.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Pore fluid in 1 km³ at 5.7% effective porosity | Results, "CO2 mineralization …", para 2 | not stated | 5.7 × 10⁷ m³ |
| CO2 in that fluid at 0.65 M | same | not stated | 3.7 × 10¹⁰ mol = 1.63 × 10¹² g |
| Storage capacity at 0.65 M | same | 1.5 Gt/km³ | 0.0016 Gt/km³ (1.6 Mt/km³); reported is 920× larger |
| Concentration after full GLDA degradation | same, para 1 | 0.65 + 0.65 × 9 = 6.5 M | 6.5 M (consistent) |
| Storage capacity at 6.5 M | same, para 2 | 15 Gt/km³ | 0.016 Gt/km³ (16 Mt/km³); reported is 920× larger |
| Effective porosity the reported values would need | derived | 5.7% stated | 5,244% (52 m³ of fluid per m³ of rock), at either concentration |
| CO2 mass per m³ of rock implied by the reported values | derived | 1.5 / 15 Gt/km³ | 1,500 / 15,000 kg per m³ of rock |
| Ceiling: 6.5 M fluid filling 100% of the rock | derived | – | 0.29 Gt/km³ |
| Ceiling: dense CO2 (700 kg/m³, assumed) filling all 5.7% pores | derived | – | 0.040 Gt/km³ |
| Reservoir volume for 20 Gt | same, para 2 | 1.3 km³ = 1/25,000 of 33,000 km³ | 20 / 15 = 1.33 km³ and 33,000 / 25,000 = 1.32 km³ (the reported chain is self-consistent with 15 Gt/km³) |
| CO2 held in 1.3 km³ at 6.5 M | same | "up to 20 Gt" | 0.021 Gt (21 Mt) |
| Reservoir volume for 20 Gt at the recomputed capacity | derived | 1.3 km³ | 1,227 km³ = 1/27 of the 33,000 km³ rift zone |
| Cited comparator, from the cited study's own tables | cited study, Tables 7 and 8 | 0.18 Gt/km³ | 0.33 Gt / 1.8 km³ = 0.183; 6,100 Gt / 33,000 km³ = 0.185 Gt/km³ |
| Ratio of the 6.5 M capacity to the comparator | same, para 2; Abstract | "two orders of magnitude higher" (15 / 0.18 = 83×) | 0.016 / 0.18 = 0.09, about 11× lower |
Each later number follows arithmetically from the 15 Gt/km³ figure: 1.3 km³ × 15 Gt/km³ = 19.5 Gt ("up to 20 Gt"), and 1.3 km³ is 1/25,000 of the 33,000 km³ rift zone. The error therefore enters once, at the step that converts grams of CO2 to gigatonnes, and is carried through the rest of the paragraph.
Figure
Figure 1. The top row gives the paper's stated inputs, multiplied in sequence for 1 km³ of rock: 5.7% effective porosity gives 5.7 × 10⁷ m³ of pore fluid, 0.65 M (650 mol/m³) gives 3.7 × 10¹⁰ mol of CO2, and 44.01 g/mol gives 1.6 × 10¹² g, or 1.63 Mt. The dashed box marks the conversion from grams to gigatonnes, where the reported values become about 920 times larger than the product of the inputs. The table below gives each downstream result twice: recomputed from the paper's inputs (blue marker, middle column) and as reported in the paper (orange marker, right column). All recomputed numbers come from recompute.py.txt.
Figure 2. Storage capacity in Gt CO2 per km³ of rock, on a log scale. For each scenario, the orange dot is the value reported in the paper and the blue dot is the value recomputed from the paper's own porosity and concentration; the labelled factor is the ratio of the two. The dotted lines are two ceilings, also computed in recompute.py.txt. Ceiling A (0.040 Gt/km³) is pure dense CO2 at an assumed 700 kg/m³ filling all of the 5.7% effective porosity. Ceiling B (0.29 Gt/km³) is the 6.5 M fluid filling 100% of the rock volume. Both reported values lie above both ceilings. The grey diamond is the 0.18 Gt/km³ comparator the paper cites, a mineral-carbonation estimate that the cited study derives as 0.33 Gt for a 1.8 km³ formation. The recomputed 6.5 M value lies below it, not two orders of magnitude above it.
Why it matters
The two capacities are the paper's only quantitative estimate of storage at reservoir scale, and they support the paper's scale-up claims: that 20 Gt of CO2 per year (the 2100 removal target the Introduction cites) could be stored in 1.3 km³, a small fraction of Iceland's rift zone, and, in the abstract, that the approach offers capacities "two orders higher than current techniques" and "reduces required reservoir volume". With the paper's own inputs, the reservoir volume needed for 20 Gt is about 1,230 km³ rather than 1.3 km³, roughly 1/27 of the rift zone rather than 1/25,000. The comparison with the cited value is not two orders of magnitude higher: in the same units it is about 11 times lower.
The abstract gives no numbers and does not say which comparison its "two orders higher" rests on. The only quantitative comparison in the body is the one recomputed here, so with the paper's inputs the abstract claim is not supported by the body.
The experimental results do not depend on this calculation. These are the measured changes in basalt cores after acidic GLDA flooding (16% gain in effective porosity, 26-fold gain in permeability), the CO2 absorption of GLDA seawater solutions, and the batch and flooding chemistry. The stoichiometric step from 0.65 M to 6.5 M after GLDA degradation is arithmetically consistent and is not in question here.
Explanations considered
A different meaning of "M". The paper uses M as mol/L, and 1 mol/L equals 1 kmol/m³, which is what the recomputation uses. The value 0.65 M for 20 wt % GLDA-Na4 (molar mass 351.1 g/mol, so 0.570 mol per kg of solution) implies a solution density of 1.14 kg/L, which is plausible. No reading of M makes the concentration 1,000 times larger.
A different porosity. To give 1.5 Gt/km³ at 0.65 M, or 15 Gt/km³ at 6.5 M, the pore fluid would have to be 52 times the volume of the rock (an effective porosity of about 5,200%). The paper's stated values are 4.8% and 5.7%. In its own experiments, the flooding core's effective porosity rose from 12.4 to 14.4%, and its basalt samples' total porosity is 14–20%. The cited study gives 10.99–15.23% total. None of these comes close. Even if the whole rock volume were 6.5 M fluid, the capacity would be 0.29 Gt/km³.
A megatonne/gigatonne slip. The correct results are 1.63 and 16.3 Mt/km³. Reporting them as 1.5 and 15 Gt/km³ fits a factor-of-1,000 unit slip combined with rounding or slightly different inputs (the remaining factor is 0.92). This is the most economical arithmetic explanation of how the numbers arose. It does not make the Gt figures correct, and every downstream statement in the paragraph inherits the factor.
A different basis (per km³ of pore space, or including mineralized CO2). The text says the calculation fills "all effective porosity" with the solution at the stated concentration. Even 1 km³ of pure pore fluid at 6.5 M holds only 0.29 Gt, not 15. Mineralization converts CO2 already in the fluid into carbonate and adds no CO2. Filling all 5.7% of the pore space with dense CO2 is capped at about 0.04 Gt/km³. The reported 15 Gt/km³ means 15,000 kg of CO2 per m³ of rock, about five times (5.1×) the density of basalt itself (taking a typical 2,950 kg/m³, an assumed value).
"Annually" meaning repeated injection. The paper gives no injection rate, pore-fluid turnover or number of cycles per year. The 20 Gt figure equals the static capacity times the volume (1.3 km³ × 15 Gt/km³ = 19.5 Gt), so "annually" adds no factor to the calculation.
The comparator is in Mt as well. If the cited 0.18 Gt/km³ were also inflated by a factor of 1,000, the paper's ratio would survive. The cited study's own tables rule this out. Its Table 7 gives a mean CO2 sequestration potential of 0.33 Gt for a CarbFix formation of 1.8 km³ (0.4 km × 2.5 km × 1.8 km), which is 0.18 Gt/km³. Its Table 8 gives 6,100 Gt for the 33,000 km³ rift zone, which is 0.185 Gt/km³. The paper takes its 33,000 km² × 1 km rift-zone dimensions from the same table. The value is computed from reactive surface area and a mineral-carbonation fixation rate per unit area, not from CO2 dissolved in pore fluid. It is in gigatonnes in the source, and it rests on a different basis from the paper's fluid-filling calculation. With consistent units, the paper's 0.016 Gt/km³ is about 11 times below it.
Conversion or archiving artifact. Ruled out. The publisher-typeset PDF (Science Advances, deposited in the PMC Open Access collection) carries the same numbers and units as the PMC text and the Europe PMC XML, with km³ correctly marked up. The quantities are not affected by formatting.
A derivation in the supplement. The only supplementary file (Tables S1–S2, Figs. S1–S11) contains rock compositions, the seawater recipe and experimental figures. It contains no capacity calculation, no Gt or km³ values, and no alternative units.
Status
As of 2026-10-23:
- Version of record. science.org, reached through doi.org, returned HTTP 403 to automated fetches. An earlier audit stage therefore checked the publisher-typeset PDF from the PMC Open Access bucket (
s3://pmc-oa-opendata/PMC11566992.1/, a Science Advances InDesign PDF created 2024-11-14). Its Results subsection "CO2 mineralization with GLDA biodegradation and overall evaluation" carries every quoted number unchanged (4.8 to 5.7%; 0.65 M, 1.5 Gt/km³; 6.5 M, 15 Gt/km³; 0.18 Gt/km³; 20 Gt, 1.3 km³, 1/25,000). The abstract sentence is also unchanged. The quotes above were re-checked verbatim against the PMC archive text on 2026-10-23. - Supplements.
sciadv.adq0515_sm.pdf(13 pages) was read in full at an earlier audit stage. It has no capacity derivation. No source-data files are deposited. - Cited comparator. The cited study's accepted manuscript (OSTI 1538337), Tables 7 and 8, was read on 2026-10-23 to establish the basis and units of 0.18 Gt/km³.
- Corrections.
src/known.py(run 2026-10-23) lists no Crossref updates and no PMC or Europe PMC correction or retraction notices. A Crossreffilter=updates:query for the DOI returns 0 results. The PMC record lists no related notices. Crossref Labs (Retraction Watch data) and a Europe PMC search for the DOI showed no correction at an earlier audit stage. - PubPeer. The PubPeer API returned HTTP 403 on 2026-10-23, so PubPeer could not be checked directly. A web search on 2026-10-23 for the DOI or title together with PubPeer, correction and erratum found no relevant thread or notice.
Sources: PMC11566992 (PubMed Central archive, read with src/show.py); https://doi.org/10.1126/sciadv.adq0515; publisher PDF and supplement from s3://pmc-oa-opendata/PMC11566992.1/; cited study, doi:10.1016/j.ijggc.2017.12.008 (accepted manuscript https://www.osti.gov/servlets/purl/1538337); https://api.crossref.org/works?filter=updates:10.1126/sciadv.adq0515.
Suggested correction
In the Results paragraph:
- Replace "a CO2 storage capacity of 1.5 Gt/km3" with "a CO2 storage capacity of about 1.6 Mt/km3 (0.0016 Gt/km3)".
- Replace "15 Gt of CO2/km3" with "about 16 Mt of CO2/km3 (0.016 Gt/km3)".
- Rewrite the comparison with the cited value of 0.18 Gt/km3. In the same units the corrected figure is about one-eleventh of it, not two orders of magnitude higher, and the cited value is a mineral-carbonation estimate rather than a dissolved-CO2 capacity.
- Restate the 20 Gt sentence. Under the stated assumptions, 1.3 km³ holds about 21 Mt of CO2, and 20 Gt would need about 1,230 km³, roughly 1/27 of the 33,000 km³ rift zone. If "annually" is meant to reflect repeated injection, state the injection rate or turnover that supports it.
In the abstract, revise "capacities two orders higher than current techniques" and "reduces required reservoir volume" to match the corrected calculation, or name the comparison and basis that support them.
Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC11566992 and regenerate fig1/fig2.
Run: python recompute.py.txt (writes fig1.svg/png and fig2.svg/png next to this file)
Inputs are taken from the article as archived in PubMed Central (PMC11566992),
Results subsection "CO2 mineralization with GLDA biodegradation and overall
evaluation", paragraphs 1-2:
- effective porosity after GLDA treatment: 5.7 % (from 4.8 %)
- CO2 concentration of the injected fluid: 0.65 M; 6.5 M after complete GLDA degradation
- reported capacities: 1.5 and 15 Gt CO2 per km3; up to 20 Gt in 1.3 km3
- active rift zone in Iceland: 33,000 km2 x 1 km; comparator 0.18 Gt CO2/km3
The comparator's own basis is taken from the cited study (ref. 17 of the paper,
Int. J. Greenh. Gas Control 70:146-156 (2018), doi:10.1016/j.ijggc.2017.12.008, accepted manuscript, Tables 7 and 8).
Three constants are not from either paper: the molar mass of CO2 (used in the
capacity calculation); for one physical ceiling, an assumed dense-CO2 density of
700 kg/m3; and, for one comparison only, a typical basalt density of 2,950 kg/m3
(both assumed values are labelled as such).
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (paper)
PHI0, PHI = 0.048, 0.057 # effective porosity before / after GLDA treatment
C1, C2 = 0.65, 6.5 # mol/L CO2 (GLDA-loaded; after full degradation)
REP1, REP2 = 1.5, 15.0 # reported Gt CO2 per km3
REP_ANNUAL, REP_VOL = 20.0, 1.3 # "Up to 20 Gt ... in a 1.3 km3 reservoir volume"
RIFT_KM3 = 33000 * 1 # 33,000 km2 x 1 km
RIFT_FRACTION_REP = 25000 # "1/25,000 of the active rift zone"
CITED = 0.18 # Gt CO2 per km3, as quoted from the cited study
# Cited study (ref. 17), Table 7 (mean row) and Table 8, as printed in that study
CITED_SITE_GT, CITED_SITE_KM3 = 0.33, 1.8 # CarbFix site: 0.4 km x 2.5 km x 1.8 km
CITED_RIFT_GT = 6100 # active rift zone, 33,000 km3
# ---------------------------------------------------------------- constants
M_CO2 = 44.01 # g/mol
KM3 = 1e9 # m3
G_PER_GT = 1e15 # g per Gt
RHO_DENSE_CO2 = 700 # kg/m3, assumed (generous) dense/supercritical CO2 density
RHO_BASALT = 2950 # kg/m3, assumed typical basalt grain/bulk density (~2,900-3,000)
def gt_per_km3(conc_M, phi):
"""CO2 (Gt) held in 1 km3 of rock whose effective pores are filled with fluid at conc_M mol/L."""
mol = conc_M * 1000 * phi * KM3 # mol/L -> mol/m3, times pore volume
return mol * M_CO2 / G_PER_GT
pore_m3 = PHI * KM3
mol1 = C1 * 1000 * pore_m3
g1 = mol1 * M_CO2
cap1, cap2 = gt_per_km3(C1, PHI), gt_per_km3(C2, PHI)
ratio1, ratio2 = REP1 / cap1, REP2 / cap2
print("== Capacity per km3 of rock, paper's own inputs ==")
print(f"pore fluid in 1 km3 at {PHI:.1%} effective porosity: {pore_m3:.3g} m3")
print(f"0.65 M: {mol1:.4g} mol CO2 = {g1:.4g} g = {g1/1e12:.3f} Mt = {cap1:.5f} Gt/km3; "
f"reported {REP1} Gt/km3; reported/recomputed = {ratio1:.0f}x")
print(f"6.5 M: {cap2*1e3:.2f} Mt = {cap2:.4f} Gt/km3; reported {REP2} Gt/km3; "
f"reported/recomputed = {ratio2:.0f}x")
print(f"10x step from 0.65 M to 6.5 M (0.65 + 0.65 x 9): {0.65 + 0.65*9:.2f} M (paper's arithmetic is consistent)")
print()
print("== What the reported values would require ==")
for c, rep in [(C1, REP1), (C2, REP2)]:
phi_needed = rep * G_PER_GT / (c * 1000 * M_CO2 * KM3)
print(f"{rep} Gt/km3 at {c} M needs effective porosity {phi_needed:.1%}; "
f"implied CO2 mass per m3 of rock = {rep*1e12/KM3:,.0f} kg/m3")
print(f"15 Gt/km3 = {REP2*1e12/KM3:,.0f} kg CO2 per m3 of rock = "
f"{REP2*1e12/KM3/RHO_BASALT:.1f}x a basalt density of {RHO_BASALT:,} kg/m3 (assumed)")
print()
print("== Physical ceilings (per km3 of rock) ==")
ceil_100 = gt_per_km3(C2, 1.0)
ceil_dense = PHI * KM3 * RHO_DENSE_CO2 / 1e12
print(f"6.5 M fluid filling 100% of the rock volume: {ceil_100:.3f} Gt/km3")
print(f"pure dense CO2 ({RHO_DENSE_CO2} kg/m3, assumed) filling all 5.7% pores: {ceil_dense:.4f} Gt/km3")
print()
print("== 20 Gt in 1.3 km3 ==")
print(f"reported chain: 20 / 15 = {REP_ANNUAL/REP2:.2f} km3; 33,000 / 25,000 = {RIFT_KM3/RIFT_FRACTION_REP:.2f} km3; "
f"1.3 x 15 = {REP_VOL*REP2:.1f} Gt")
held_13 = REP_VOL * cap2
vol_20 = REP_ANNUAL / cap2
print(f"CO2 held in 1.3 km3 at 6.5 M (recomputed): {held_13:.4f} Gt = {held_13*1e3:.1f} Mt")
print(f"volume needed for 20 Gt at the recomputed 6.5 M capacity: {vol_20:,.0f} km3 "
f"= 1/{RIFT_KM3/vol_20:.0f} of the {RIFT_KM3:,} km3 rift zone")
print()
print("== Comparison with the cited value (0.18 Gt/km3) ==")
print(f"cited study, CarbFix site: {CITED_SITE_GT} Gt / {CITED_SITE_KM3} km3 = {CITED_SITE_GT/CITED_SITE_KM3:.3f} Gt/km3")
print(f"cited study, rift zone: {CITED_RIFT_GT} Gt / {RIFT_KM3:,} km3 = {CITED_RIFT_GT/RIFT_KM3:.3f} Gt/km3")
print(f"reported 15 / 0.18 = {REP2/CITED:.0f}x ('two orders of magnitude higher')")
print(f"recomputed {cap2:.4f} / 0.18 = {cap2/CITED:.3f} (about {CITED/cap2:.0f}x lower)")
print()
print("== Side check: 0.65 M for 20 wt % GLDA-Na4 ==")
MW_GLDA_NA4 = 9*12.011 + 9*1.008 + 14.007 + 4*22.990 + 8*15.999 # C9H9NNa4O8
molal = 200 / MW_GLDA_NA4
print(f"MW C9H9NNa4O8 = {MW_GLDA_NA4:.1f} g/mol; 20 wt % = {molal:.3f} mol/kg solution; "
f"0.65 M implies a solution density of {C1/molal:.2f} kg/L (plausible; M is mol/L)")
# ---------------------------------------------------------------- figure style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, HL = "#eb6834", "#2a78d6", "#e4e3df", "#fbe9e1"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"text.color": INK, "svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
def fmt_sci(x):
e = int(f"{x:e}".split("e")[1])
m = x / 10**e
return f"{m:.2g}×10$^{{{e}}}$"
# ---------------------------------------------------------------- fig1: arithmetic chain
W, H = 7.2, 6.65
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
def box(x, y, w, h, text, ec=INK2, fc="white", color=INK, weight="normal", lw=1.5, ls="-", size=10):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.06",
fc=fc, ec=ec, lw=lw, ls=ls))
ax.text(x + w/2, y + h/2, text, ha="center", va="center", fontsize=size, color=color,
weight=weight, linespacing=1.35)
def arrow(x0, y0, x1, y1, color=INK2):
ax.add_patch(FancyArrowPatch((x0, y0), (x1, y1), arrowstyle="-|>", mutation_scale=12,
color=color, lw=1.5, shrinkA=0, shrinkB=0))
ax.text(0.15, H - 0.18, "Capacity of 1 km³ of rock, using only the paper's inputs",
ha="left", va="top", fontsize=11, weight="bold")
ax.text(0.15, H - 0.45, "Shared steps (the paper's stated inputs)", ha="left", va="top",
fontsize=10, color=INK2)
bw, bh, gap, y0 = 1.55, 0.95, 0.22, H - 1.65
xs = [0.15 + i*(bw + gap) for i in range(4)]
steps = ["Rock volume\n1 km³ = 10$^{9}$ m³",
f"× 5.7% effective\nporosity\n= {fmt_sci(pore_m3)} m³ fluid",
f"× 0.65 M\n(650 mol/m³)\n= {fmt_sci(mol1)} mol",
f"× 44.01 g/mol\n= {fmt_sci(g1)} g CO$_2$\n= {g1/1e12:.2f} Mt"]
for x, s in zip(xs, steps):
box(x, y0, bw, bh, s)
for i in range(3):
arrow(xs[i] + bw, y0 + bh/2, xs[i+1], y0 + bh/2)
# divergence step
dx, dy, dw, dh = xs[2] - 0.9, y0 - 1.05, W - 0.15 - (xs[2] - 0.9), 0.62
ax.add_patch(FancyBboxPatch((dx, dy), dw, dh, boxstyle="round,pad=0.02,rounding_size=0.06",
fc=HL, ec=ORANGE, lw=2, ls="--"))
ax.text(dx + dw/2, dy + dh/2 + 0.11, "Unit step: grams → gigatonnes (÷10$^{15}$)",
ha="center", va="center", fontsize=10, weight="bold")
ax.text(dx + dw/2, dy + dh/2 - 0.15, f"the reported values diverge here by ≈{ratio1:.0f}×",
ha="center", va="center", fontsize=10, color=INK2)
arrow(xs[3] + bw/2, y0, xs[3] + bw/2, dy + dh)
# outcome table
ty = dy - 0.42
cx_lab, cx_b, cx_o = 0.15, 3.75, 5.95
ax.text(cx_lab, ty, "Result", ha="left", va="center", fontsize=10, color=INK2, weight="bold")
ax.plot([cx_b - 1.05], [ty], "o", ms=8, color=BLUE)
ax.plot([cx_o - 0.85], [ty], "o", ms=8, color=ORANGE)
ax.text(cx_b, ty, "Recomputed from\nthe paper's inputs", ha="center", va="center", fontsize=10,
color=BLUE, weight="bold")
ax.text(cx_o, ty, "Reported in\nthe paper", ha="center", va="center", fontsize=10,
color=ORANGE, weight="bold")
rows = [("Capacity per km³, 0.65 M", f"{cap1:.4f} Gt ({cap1*1e3:.1f} Mt)", "1.5 Gt"),
("Capacity per km³, 6.5 M\n(after GLDA degradation, ×10)", f"{cap2:.3f} Gt ({cap2*1e3:.0f} Mt)", "15 Gt"),
("CO$_2$ held in 1.3 km³ at 6.5 M", f"{held_13:.3f} Gt ({held_13*1e3:.0f} Mt)", "up to 20 Gt"),
("Rock volume needed for 20 Gt", f"{vol_20:,.0f} km³\n(1/{RIFT_KM3/vol_20:.0f} of the rift zone)",
"1.3 km³\n(1/25,000 of the rift zone)"),
("Ratio to the cited\n0.18 Gt/km³ (ref. 17)", f"{cap2/CITED:.2f}×\n(about {CITED/cap2:.0f}× lower)",
"“two orders of\nmagnitude higher”")]
rh = 0.62
for i, (lab, b, o) in enumerate(rows):
yy = ty - 0.55 - i*rh
ax.plot([0.15, W - 0.15], [yy + rh/2, yy + rh/2], color=GRID, lw=1)
ax.text(cx_lab, yy, lab, ha="left", va="center", fontsize=10, linespacing=1.2)
ax.text(cx_b, yy, b, ha="center", va="center", fontsize=10, color=INK, linespacing=1.2)
ax.text(cx_o, yy, o, ha="center", va="center", fontsize=10, color=INK, linespacing=1.2)
save(fig, "fig1")
# ---------------------------------------------------------------- fig2: log-scale comparison
fig, ax = plt.subplots(figsize=(7.2, 3.7))
fig.subplots_adjust(left=0.30, right=0.97, top=0.80, bottom=0.17)
rows2 = [("0.65 M fluid in\n5.7% porosity", REP1, cap1),
("6.5 M fluid in 5.7% porosity\n(after GLDA degradation)", REP2, cap2)]
yp = [2, 1]
for y, (lab, rep, rec) in zip(yp, rows2):
ax.plot([rec, rep], [y, y], color=INK2, lw=1, zorder=1)
ax.plot(rep, y, "o", ms=9, color=ORANGE, zorder=3)
ax.plot(rec, y, "o", ms=9, color=BLUE, zorder=3)
gm = 0.11 if y == 2 else 1.6
ax.text(gm, y + 0.13, f"×{rep/rec:.0f}", ha="center", va="bottom", fontsize=10, color=INK2)
ax.text(rep * 1.25, y, f"{rep:g}", ha="left", va="center", fontsize=10, color=INK)
ax.text(rec / 1.25, y, f"{rec:.4f}" if rec < 0.01 else f"{rec:.3f}", ha="right", va="center",
fontsize=10, color=INK)
ax.plot(CITED, 0, "D", ms=8, color=INK2, zorder=3)
ax.text(CITED * 1.3, 0, "0.18 (cited mineral-carbonation\nestimate: 0.33 Gt / 1.8 km³)",
ha="left", va="center", fontsize=10, color=INK)
ax.set_yticks([2, 1, 0])
ax.set_yticklabels([r[0] for r in rows2] + ["Comparator cited in\nthe paper (ref. 17)"])
ax.set_ylim(-0.6, 2.6)
ax.set_xscale("log")
ax.set_xlim(2e-4, 2e2)
for xv, lab, ha in [(ceil_dense, f"A: {ceil_dense:.3f} ", "right"), (ceil_100, f" B: {ceil_100:.2f}", "left")]:
ax.axvline(xv, color=INK2, lw=1, ls=":", zorder=0)
ax.text(xv, 2.62, lab, ha=ha, va="bottom", fontsize=10, color=INK2)
ax.set_xlabel("Gt CO$_2$ per km³ of rock (log scale)", fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8, which="major")
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0, labelsize=10)
ax.tick_params(axis="x", labelsize=10, colors=INK2)
handles = [Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported in the paper"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE, label="Recomputed from the paper's inputs"),
Line2D([], [], ls=":", color=INK2, lw=1, label="Ceilings (see caption)")]
fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.995), ncol=3, frameon=False,
fontsize=10, handletextpad=0.4, columnspacing=1.2)
save(fig, "fig2")
print("\nfigures written:", ", ".join(f"fig{i}.{e}" for i in (1, 2) for e in ("svg", "png")))
The abstract gives the top tolerated monkey dose in mg/kg; the body gives μg/kg
Sci Adv (2021) · doi:10.1126/sciadv.abi8193 · PMC8357232
Summary
The abstract states that the T cell–engaging bispecific CCW702 "was well tolerated up to 34.1 mg/kg per dose" in cynomolgus monkeys. The Results, the Methods and the Fig. 6 legend all describe a single repeat-dose monkey study with doses of 2, 9.8 and 34.1 μg/kg, and they put the no-observed-adverse-effect level (NOAEL) at 34.1 μg/kg. Read literally, the abstract's 34.1 mg/kg is 34,100 μg/kg, 1,000 times the highest dose tested. The paper's own exposure value for that group (day-1 AUC 1.02 h·μg/mL) gives an apparent clearance of 33 mL/h/kg for a μg/kg dose. For a mg/kg dose it gives about 33,400 mL/h/kg, which is 2.6 times total cardiac output and physically impossible. This is the abstract's only primate safety number. Read as printed, it puts the tolerated monkey dose level with the highest mouse efficacy dose (35.2 mg/kg), when the tested dose was 33 to 1,000 times lower than the mouse efficacy doses.
What the paper reports
Abstract:
In cynomolgus monkeys, CCW702 was well tolerated up to 34.1 mg/kg per dose, with near-complete subcutaneous bioavailability and a PK profile supporting testing of a weekly dosing regimen in patients.
Results, "Pharmacokinetics and pharmacodynamics in nonhuman primate", second paragraph:
Doses of 2, 9.8, and 34.1 μg/kg were administered to animals QAD for 10 days (five total doses).
CCW702 administration was generally well tolerated at these dose levels with no changes in animal body weight, food consumption, or body temperature and the no-observed-adverse-effect level (NOAEL) determined to be the highest dose in the study of 34.1 μg/kg.
This occurred despite similar exposure of CCW702 on day 1 and on day 9 (1.02 hours·μg/ml versus 0.705 hours·μg/ml for the 34.1 μg/kg group) (fig. S17).
Same section, first paragraph (single-dose PK studies):
CCW702 exposure was detectable by electrochemiluminescence bioanalytical assay at doses of ≥9.8 μg/kg.
Subcutaneous dosing had near complete bioavailability (F = 108 ± 20.2%) with a maximum concentration (Cmax) and half-life (t1/2) of 1.5 (ng/ml)/(mg/kg) and 11.4 hours compared with the intravenous route of administration Cmax of 28.3 (ng/ml)/(mg/kg) and t1/2 of 1.6 hours, respectively (fig. S17).
Fig. 6 legend, panel A (panel B gives the same doses):
Heatmaps of cytokines measured from peripheral blood of animals administrated CCW702 at 2, 9.8, and 34.1 μg/kg, every other day for 10 days (a total of five total doses).
Methods, "Pharmacokinetic and pharmacodynamic 10-day study in cyno":
Nine naïve male cynomolgus monkeys (Macaca fascicularis, Chinese origin), age 2 to 4 years, weighing 2.5 to 3.2 kg were used for the study.
The NOAEL was determined to be 34.1 μg/kg under the study conditions.
For context, Results, "In vivo efficacy in xenograft and patient-derived xenograft models":
every 7 days (Q7D) at four-dose levels (1.13, 3.5, 11.1, and 35.2 mg/kg; groups G to J)
What the paper's own numbers show
Method. I converted the abstract's figure to μg/kg and compared it with the doses tested in the body. To find which unit the data support, I divided the top dose by the paper's day-1 AUC for that group. That gives the apparent clearance CL/F, and since F ≈ 1 (108%) it is close to systemic clearance. I compared CL/F under each reading with standard physiological values for the monkey (Davies & Morris, Pharm Res 1993;10:1093–5): cardiac output 217 mL/min/kg (13,020 mL/h/kg), hepatic blood flow 43.6 mL/min/kg (2,616 mL/h/kg) and plasma volume 44.8 mL/kg. Plasma clearance cannot exceed the blood flow that carries the drug to the organs that eliminate it, so it cannot exceed cardiac output. The AUC interval is given in fig. S17, which I could not access. A partial AUC would make CL/F an overestimate, but by much less than the factor of 1,000 at issue here. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Highest tolerated monkey dose | Abstract | 34.1 mg/kg per dose | = 34,100 μg/kg |
| Monkey repeat-dose levels | Results NHP para 2; Fig. 6 legend | 2, 9.8, 34.1 μg/kg, QAD × 5 | highest = 34.1 μg/kg = 0.0341 mg/kg |
| NOAEL | Results NHP para 2; Methods | 34.1 μg/kg | equals the top dose tested |
| Abstract value / top dose tested | derived | (implied equal) | 1,000 |
| Cumulative dose, top group | derived | — | 5 × 34.1 = 170.5 μg/kg = 0.171 mg/kg |
| Amount per dose, 2.5–3.2 kg animal | Methods (weights) | — | 0.085–0.109 mg if μg/kg; 85–109 mg if mg/kg |
| CL/F if top dose = 34.1 μg/kg | Results NHP para 2 (AUC 1.02 h·μg/mL) | — | 33.4 mL/h/kg, 0.26% of cardiac output (day 9: 48.4 mL/h/kg) |
| CL/F if top dose = 34.1 mg/kg | same | — | 33,431 mL/h/kg, 2.6 × total cardiac output |
| SC Cmax and AUC at top dose | Results NHP para 1 (1.5 per unit dose, t1/2 11.4 h) | AUC 1.02 h·μg/mL | Cmax 51.2 ng/mL; crude AUC ≈ Cmax/k = 841 ng·h/mL, close to the reported 1,020. This check holds under either reading, so it does not settle the unit on its own |
| Mouse Q7D efficacy doses / top monkey dose | Results (mouse efficacy) | 1.13–35.2 mg/kg | 33× to 1,032× the top monkey dose tested |
| Abstract value / top mouse dose | derived | — | 34.1 / 35.2 = 0.97 |
Figure
Figure 1. The two statements of the top cynomolgus dose. The left panel (orange, the value as printed in the abstract) quotes the abstract's "34.1 mg/kg per dose", which equals 34,100 μg/kg. The right panel (blue) shows the doses used in the Results, Methods and Fig. 6 legend, with the NOAEL at 34.1 μg/kg. The connector marks the factor of 1,000 between the two. The bottom line of the right panel gives the apparent clearance recomputed from the paper's own day-1 AUC, which is plausible only for the μg/kg reading. All values come from the paper or from recompute.py.txt.
Figure 2. Where the factor of 1,000 enters. The dose label "34.1" for the top group is read in the abstract's unit (orange, mg/kg) and in the body's unit (blue, μg/kg). Each reading is divided by the paper's own day-1 AUC for that group (1,020 ng·h/mL). The mg/kg reading gives an apparent clearance of 33,431 mL/h/kg, 2.6 times the total cardiac output of a monkey, which is not possible. The μg/kg reading gives 33.4 mL/h/kg, which is plausible. The cardiac output value is an external reference (Davies & Morris 1993); all other numbers come from the paper or from recompute.py.txt.
Why it matters
The abstract sentence is the paper's only summary of primate safety. It is also the evidence the abstract offers just before it says the drug is in a first-in-human trial (NCT04077021). The data in the body are reported consistently, and the qualitative finding holds: CCW702 was well tolerated at every dose tested, and the NOAEL was the top dose. The error is in magnitude. Read as printed, the abstract suggests that monkeys tolerated about 34 mg/kg per dose, roughly the highest mouse efficacy dose (35.2 mg/kg Q7D), which implies a wide tolerability margin over the efficacious range. In fact the highest dose tested in monkeys was 0.0341 mg/kg per dose, 33 to 1,000 times below the mouse Q7D efficacy doses. Mouse and monkey doses in mg/kg are not directly interchangeable across species, but that is the comparison a reader of the abstract would make. No dose above 34.1 μg/kg was tested in monkeys, so the study established a NOAEL but not a maximum tolerated dose. The Methods also report clinical pathology changes and microscopic findings, judged not adverse, at doses above 2 μg/kg. A reader who relies on the abstract, or a database or review that cites it, will carry a tolerated dose that is 1,000 times too high.
Explanations considered
- A different, higher-dose monkey study. This does not account for the numbers. The only monkey tolerability study described is the 10-day repeat-dose study (nine animals, three groups), and its doses are 2, 9.8 and 34.1 μg/kg. The single-dose PK studies report no dose in mg/kg; their only dose statement is detectability "at doses of ≥9.8 μg/kg". The numeral 34.1 in the abstract matches the body's top dose exactly, unit aside.
- The body is wrong and the abstract is right. This is ruled out by the paper's own data. With the reported day-1 AUC, a 34.1 mg/kg dose implies a clearance of 33,431 mL/h/kg, 2.6 times total cardiac output. The μg/kg reading gives 33.4 mL/h/kg. The body uses μg/kg in six separate places (two Results paragraphs, both Fig. 6 legend panels, the Methods), and the version-of-record check found the same unit in the panel labels of the Fig. 6 image itself ("34.1 μg/kg/dose").
- A text-conversion artifact (μ rendered as m). This is ruled out. The same "34.1 mg/kg" appears in the publisher's JATS XML at Europe PMC, on the live PMC page and in the PubMed abstract, which the publisher supplies separately. The same files render μ correctly everywhere else.
- Mixing up mouse and monkey units. The mouse doses really are in mg/kg (0.15 to 35.2 mg/kg), and this may be how mg/kg came to appear. It does not change the result, because the abstract sentence names cynomolgus monkeys explicitly.
- A related unit question (side note, not part of this claim). The dose-normalized Cmax values are labeled "(ng/ml)/(mg/kg)". An IV Cmax of 28.3 ng/mL per mg/kg would require an initial distribution volume of about 35 L/kg for a roughly 50 kDa protein. Per μg/kg it gives 35 mL/kg, close to plasma volume (44.8 mL/kg, which predicts about 22 ng/mL per μg/kg). So these labels may also be 1,000-fold off. I list this only as an observation, because I could not check it against fig. S17.
Status
As of 2026-10-23. Version of record: the publisher's site (science.org, the target of the DOI redirect) returned HTTP 403, so I could not read its HTML or PDF directly. Four publisher-derived sources carry the same text: the live PMC page, the Europe PMC full-text XML (publisher JATS), the PubMed record (PMID 34380625), and the publisher's Fig. 6 image file, whose panel labels read "2 μg/kg/dose", "9.8 μg/kg/dose" and "34.1 μg/kg/dose". The quotations above were checked verbatim against the PMC-archived text. Supplement: the supplementary PDF (abi8193_sm.pdf, figs. S1 to S18, including fig. S17 with the monkey PK) is hosted only on science.org, which returned 403. Europe PMC's supplementary bundle holds only the main figures, so I could not check figs. S17 and S18. Corrections: I found none. Crossref has no update-to or updated-by relation, and the updates: filter returns 0 results. Crossref Labs (which includes Retraction Watch data) lists no updates. Europe PMC and PubMed show no errata or comment/correction links, and two web searches for an erratum or correction found none. PubPeer: the search page and the API returned 403 or not found, so I could not check PubPeer directly. A general web search found no PubPeer thread.
Suggested correction
In the abstract, replace "In cynomolgus monkeys, CCW702 was well tolerated up to 34.1 mg/kg per dose" with "In cynomolgus monkeys, CCW702 was well tolerated up to 34.1 μg/kg per dose (the highest dose tested)". The authors may also want to check the unit labels of the dose-normalized Cmax values in the first NHP Results paragraph and in fig. S17 ("(ng/ml)/(mg/kg)"), which the paper's own values suggest should be per μg/kg.
Recomputation script (python recompute.py.txt)
"""Recompute the cynomolgus dose-unit check for PMC8357232
(Sci Adv 2021, doi:10.1126/sciadv.abi8193).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes, next to this file, five candidate figure designs (candidate1..5.svg/.png)
plus fig1 (= candidate 1) and fig2 (= candidate 2).
Inputs are copied from the article as archived in PubMed Central:
Abstract: "well tolerated up to 34.1 mg/kg per dose"
Results (NHP para 2), Methods (10-day cyno study), Fig. 6 legend: doses 2, 9.8, 34.1 ug/kg; NOAEL 34.1 ug/kg
Results (NHP para 2): day-1 exposure 1.02 h*ug/mL, day-9 0.705 h*ug/mL, 34.1 ug/kg group
Results (NHP para 1): SC F = 108%, SC Cmax 1.5 and IV Cmax 28.3 "(ng/ml)/(mg/kg)", SC t1/2 11.4 h
Methods: monkeys weighed 2.5 to 3.2 kg
Results (mouse efficacy): Q7D doses 1.13, 3.5, 11.1, 35.2 mg/kg
External reference values (not from the paper), cynomolgus/rhesus monkey, Davies & Morris,
Pharm Res 1993;10:1093-5: plasma volume 44.8 mL/kg, cardiac output 217 mL/min/kg,
hepatic blood flow 43.6 mL/min/kg.
"""
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs (paper)
ABSTRACT_TOP_MG_KG = 34.1
CYNO_DOSES_UG_KG = [2.0, 9.8, 34.1]
NOAEL_UG_KG = 34.1
N_DOSES = 5
AUC_DAY1_H_UG_ML = 1.02
AUC_DAY9_H_UG_ML = 0.705
F_SC = 1.08
CMAX_SC_PER_UNIT = 1.5 # printed as (ng/ml)/(mg/kg)
CMAX_IV_PER_UNIT = 28.3 # printed as (ng/ml)/(mg/kg)
T_HALF_SC_H = 11.4
WEIGHT_KG = (2.5, 3.2)
MOUSE_Q7D_MG_KG = [1.13, 3.5, 11.1, 35.2]
# ------------------------------------------------------------------ external reference
PLASMA_VOL_ML_KG = 44.8
CARDIAC_OUTPUT_ML_MIN_KG = 217.0
HEPATIC_FLOW_ML_MIN_KG = 43.6
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none"})
def section(t):
print("\n== " + t)
# ------------------------------------------------------------------ 1. unit conversion
section("1. Abstract dose vs. dose tested")
abstract_ug = ABSTRACT_TOP_MG_KG * 1000
top_tested = max(CYNO_DOSES_UG_KG)
ratio = abstract_ug / top_tested
print(f"abstract 34.1 mg/kg = {abstract_ug:,.0f} ug/kg")
print(f"highest dose tested (Results/Methods/Fig. 6) = {top_tested} ug/kg = {top_tested/1000:.4f} mg/kg; NOAEL = {NOAEL_UG_KG} ug/kg")
print(f"ratio abstract / tested = {ratio:,.0f}")
print(f"cumulative over {N_DOSES} doses at top dose = {N_DOSES*top_tested:.1f} ug/kg = {N_DOSES*top_tested/1000:.3f} mg/kg")
for w in WEIGHT_KG:
print(f" per-dose amount for a {w} kg monkey: {top_tested*w/1000:.3f} mg if ug/kg, {ABSTRACT_TOP_MG_KG*w:.1f} mg if mg/kg")
# ------------------------------------------------------------------ 2. clearance check
section("2. Apparent clearance from the paper's own day-1 AUC (34.1 group)")
auc_ng_h_ml = AUC_DAY1_H_UG_ML * 1000
cl = {}
for label, dose_ng_kg in [("34.1 ug/kg", 34.1e3), ("34.1 mg/kg", 34.1e6)]:
cl[label] = dose_ng_kg / auc_ng_h_ml # mL/h/kg = CL/F; F ~ 1 (108%)
print(f"CL/F if dose = {label}: {cl[label]:,.1f} mL/h/kg = {cl[label]/1000:,.2f} L/h/kg")
co_ml_h_kg = CARDIAC_OUTPUT_ML_MIN_KG * 60
hep_ml_h_kg = HEPATIC_FLOW_ML_MIN_KG * 60
print(f"reference: cardiac output {co_ml_h_kg:,.0f} mL/h/kg, hepatic blood flow {hep_ml_h_kg:,.0f} mL/h/kg")
print(f"mg/kg reading: CL/F = {cl['34.1 mg/kg']/co_ml_h_kg:.1f} x total cardiac output")
print(f"ug/kg reading: CL/F = {cl['34.1 ug/kg']/co_ml_h_kg*100:.2f}% of cardiac output")
print(f"day-9 CL/F (ug/kg reading): {34.1e3/(AUC_DAY9_H_UG_ML*1000):.1f} mL/h/kg")
# ------------------------------------------------------------------ 3. side note: Cmax labels
section("3. Side note: the '(ng/ml)/(mg/kg)' Cmax labels")
c0_per_ug = 1000 / PLASMA_VOL_ML_KG # ng/mL per ug/kg if dose stays in plasma volume
print(f"IV dose of 1 ug/kg in {PLASMA_VOL_ML_KG} mL/kg plasma -> C0 about {c0_per_ug:.1f} ng/mL "
f"(1 mg/kg -> {c0_per_ug*1000:,.0f} ng/mL); reported IV Cmax {CMAX_IV_PER_UNIT} per unit dose")
print(f"implied initial volume if per mg/kg: {1e6/CMAX_IV_PER_UNIT/1000:,.1f} L/kg; if per ug/kg: {1e3/CMAX_IV_PER_UNIT:.1f} mL/kg")
cmax_sc = CMAX_SC_PER_UNIT * 34.1
k = math.log(2) / T_HALF_SC_H
print(f"SC Cmax at top dose = {cmax_sc:.1f} ng/mL; crude AUC ~ Cmax/k = {cmax_sc/k:.0f} ng*h/mL vs reported {auc_ng_h_ml:.0f}")
print(" (this pairing is consistent under either reading, so it does not by itself fix the unit)")
# ------------------------------------------------------------------ 4. context
section("4. Context: mouse Q7D efficacy doses vs. highest cyno dose tested")
for d in MOUSE_Q7D_MG_KG:
print(f" mouse {d} mg/kg = {d*1000:,.0f} ug/kg = {d*1000/top_tested:,.0f} x the top cyno dose")
print(f"abstract figure 34.1 mg/kg vs top mouse dose 35.2 mg/kg: ratio {ABSTRACT_TOP_MG_KG/35.2:.2f}")
# ================================================================== figures
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), bbox_inches="tight", pad_inches=0.12, **kw)
def box(ax, x, y, w, h, edge, lw=2.0, fill=BG):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
lw=lw, ec=edge, fc=fill, transform=ax.transData, clip_on=False))
def cand1():
"""Two text panels: Abstract vs body, joined by an annotated connector."""
fig, ax = plt.subplots(figsize=(7.2, 3.6)); ax.axis("off")
ax.set_xlim(-0.01, 1.01); ax.set_ylim(0, 1); ax.set_position([0.01, 0.01, 0.98, 0.98])
box(ax, 0.0, 0.04, 0.38, 0.90, ORANGE)
ax.text(0.02, 0.89, "Abstract (as printed)", fontsize=11.5, weight="bold", va="top")
ax.text(0.02, 0.77, "\u201cIn cynomolgus monkeys,\nCCW702 was well\ntolerated up to", va="top", fontsize=10.5, linespacing=1.35)
ax.text(0.02, 0.48, "34.1 mg/kg", va="top", fontsize=15, weight="bold")
ax.text(0.02, 0.36, "per dose\u201d", va="top", fontsize=10.5)
ax.text(0.02, 0.08, "34.1 mg/kg = 34,100 \u03bcg/kg", va="bottom", fontsize=10, color=INK2)
box(ax, 0.58, 0.04, 0.42, 0.90, BLUE)
ax.text(0.60, 0.89, "Results, Methods, Fig. 6", fontsize=11.5, weight="bold", va="top")
ax.text(0.60, 0.77, "Doses tested: 2, 9.8, 34.1 \u03bcg/kg\n(every other day, five doses)", va="top", fontsize=10, linespacing=1.35)
ax.text(0.60, 0.59, "\u201c\u2026 NOAEL \u2026 the highest dose\nin the study of", va="top", fontsize=10, linespacing=1.35)
ax.text(0.60, 0.43, "34.1 \u03bcg/kg\u201d", va="top", fontsize=15, weight="bold")
ax.text(0.60, 0.08, "Day-1 AUC 1.02 h\u00b7\u03bcg/mL gives\nCL/F 33 mL/h/kg only for \u03bcg/kg", va="bottom", fontsize=9.8, color=INK2)
ax.add_patch(FancyArrowPatch((0.385, 0.42), (0.575, 0.42), arrowstyle="<->", mutation_scale=14,
lw=2, color=INK2, transform=ax.transData))
ax.text(0.48, 0.47, "same dose,\nunit differs", ha="center", va="bottom", fontsize=10.5)
ax.text(0.48, 0.37, f"\u00d7{ratio:,.0f}", ha="center", va="top", fontsize=15, weight="bold")
save(fig, "candidate1"); save(fig, "fig1"); plt.close(fig)
def cand2():
"""Arithmetic flow: dose / AUC -> clearance; divergence at the unit step."""
fig, ax = plt.subplots(figsize=(7.2, 3.6)); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1)
box(ax, 0.00, 0.38, 0.18, 0.24, INK2)
ax.text(0.09, 0.50, "Dose label\n\u201c34.1\u201d\n(top group)", ha="center", va="center", fontsize=10.5)
rows = [(0.68, ORANGE, "Abstract: mg/kg", "34,100,000 ng/kg", f"{cl['34.1 mg/kg']:,.0f} mL/h/kg",
f"{cl['34.1 mg/kg']/co_ml_h_kg:.1f}\u00d7 total cardiac\noutput: not possible"),
(0.08, BLUE, "Body: \u03bcg/kg", "34,100 ng/kg", f"{cl['34.1 ug/kg']:,.1f} mL/h/kg",
f"{cl['34.1 ug/kg']/co_ml_h_kg*100:.1f}% of cardiac\noutput: plausible")]
for y, col, unit, ng, clv, verdict in rows:
box(ax, 0.235, y, 0.25, 0.24, col, lw=3)
ax.text(0.36, y + 0.12, f"{unit}\n= {ng}", ha="center", va="center", fontsize=9.8, linespacing=1.15)
box(ax, 0.53, y, 0.18, 0.24, INK2)
ax.text(0.62, y + 0.12, "\u00f7 day-1 AUC\n1,020\nng\u00b7h/mL", ha="center", va="center", fontsize=9.8)
ax.text(0.74, y + 0.17, "CL/F = " + clv, fontsize=10.5, weight="bold", va="center")
ax.text(0.74, y + 0.07, verdict, fontsize=9.8, color=INK2, va="center", linespacing=1.2)
ax.add_patch(FancyArrowPatch((0.485, y + 0.12), (0.53, y + 0.12), arrowstyle="-|>", mutation_scale=12, lw=2, color=INK2, transform=ax.transData))
ax.add_patch(FancyArrowPatch((0.18, 0.50), (0.235, y + 0.12), arrowstyle="-|>", mutation_scale=12, lw=2, color=col, transform=ax.transData))
ax.text(0.36, 0.50, "the factor of 1,000\nenters at this step", ha="center", va="center", fontsize=10.5, weight="bold")
ax.text(0.0, -0.04, "AUC: Results, 34.1 group, day 1. F \u2248 1 (108%), so CL/F \u2248 CL. "
"Cardiac output 13,020 mL/h/kg (monkey reference value).", fontsize=9.5, color=INK2, va="top")
save(fig, "candidate2"); save(fig, "fig2"); plt.close(fig)
def cand3():
"""Dose ladder on one log axis (ug/kg)."""
fig, ax = plt.subplots(figsize=(7.2, 3.0))
rows = ["Mouse efficacy, Q7D\n(Results)", "Cyno, as stated\nin Abstract", "Cyno, doses tested\n(Results, Methods, Fig. 6)"]
ax.scatter([d * 1000 for d in MOUSE_Q7D_MG_KG], [0] * 4, s=70, marker="s", facecolor="white", edgecolor=INK2, lw=2, zorder=3, label="Mouse doses (context)")
ax.scatter([abstract_ug], [1], s=90, color=ORANGE, zorder=3, label="Abstract: 34.1 mg/kg")
ax.scatter(CYNO_DOSES_UG_KG, [2] * 3, s=90, color=BLUE, zorder=3, label="Body: 2, 9.8, 34.1 μg/kg")
ax.annotate("", xy=(abstract_ug, 1.5), xytext=(34.1, 1.5), arrowprops=dict(arrowstyle="<->", color=INK2, lw=2))
ax.text(math.sqrt(abstract_ug * 34.1), 1.62, "×1,000", ha="center", va="bottom", fontsize=11, weight="bold")
ax.text(abstract_ug, 0.75, "34,100", ha="center", va="top", fontsize=10)
for d in CYNO_DOSES_UG_KG:
ax.text(d, 2.25, f"{d:g}", ha="center", va="bottom", fontsize=10)
for d in MOUSE_Q7D_MG_KG:
ax.text(d * 1000, -0.25, f"{d*1000:,.0f}", ha="center", va="top", fontsize=10)
ax.set_xscale("log"); ax.set_xlim(1, 1e5); ax.set_ylim(-0.7, 2.7)
ax.set_yticks([0, 1, 2]); ax.set_yticklabels(rows); ax.tick_params(axis="y", length=0, labelcolor=INK)
ax.set_xlabel("Dose per administration (μg/kg, log scale)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"): ax.spines[s].set_visible(False)
ax.legend(loc="upper left", frameon=False, fontsize=9.5, bbox_to_anchor=(1.0, 1.0))
save(fig, "candidate3"); plt.close(fig)
def cand4():
"""Implied clearance under each reading vs physiological ceilings, one log axis."""
fig, ax = plt.subplots(figsize=(7.2, 2.6))
ax.axvline(hep_ml_h_kg, color=INK2, lw=1, ls=":")
ax.axvline(co_ml_h_kg, color=INK2, lw=1, ls="--")
ax.text(hep_ml_h_kg, 1.55, f"hepatic blood flow\n{hep_ml_h_kg:,.0f}", ha="right", va="top", fontsize=9.5, color=INK2)
ax.text(co_ml_h_kg * 1.1, 1.55, f"total cardiac\noutput {co_ml_h_kg:,.0f}", ha="left", va="top", fontsize=9.5, color=INK2)
ax.scatter([cl["34.1 mg/kg"]], [1], s=90, color=ORANGE, zorder=3)
ax.scatter([cl["34.1 ug/kg"]], [0], s=90, color=BLUE, zorder=3)
ax.text(cl["34.1 mg/kg"] * 1.25, 1, f"{cl['34.1 mg/kg']:,.0f}", ha="left", va="center", fontsize=10.5, weight="bold")
ax.text(cl["34.1 ug/kg"] * 1.25, 0, f"{cl['34.1 ug/kg']:.1f}", ha="left", va="center", fontsize=10.5, weight="bold")
ax.set_yticks([0, 1]); ax.set_yticklabels(["Dose read as 34.1 μg/kg\n(Results, Methods)", "Dose read as 34.1 mg/kg\n(Abstract)"])
ax.tick_params(axis="y", length=0, labelcolor=INK)
ax.set_xscale("log"); ax.set_xlim(1, 1e6); ax.set_ylim(-0.6, 1.7)
ax.set_xlabel("Apparent clearance CL/F = dose / day-1 AUC (mL/h/kg, log scale)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"): ax.spines[s].set_visible(False)
save(fig, "candidate4"); plt.close(fig)
def cand5():
"""Where the dose appears, and in which unit: a location-by-unit dot table."""
locs = ["Abstract", "Results, NHP para 1 (detectable ≥ 9.8)", "Results, NHP para 2 (dose groups)",
"Results, NHP para 2 (NOAEL)", "Results, NHP para 2 (cytokines, AUC)",
"Fig. 6 legend, panels A and B", "Methods, 10-day cyno study (NOAEL)"]
unit = ["mg", "ug", "ug", "ug", "ug", "ug", "ug"]
fig, ax = plt.subplots(figsize=(7.2, 3.2))
for i, (l, u) in enumerate(zip(locs, unit)):
y = len(locs) - 1 - i
x = 0 if u == "mg" else 1
ax.scatter([x], [y], s=110, color=ORANGE if u == "mg" else BLUE, zorder=3)
ax.scatter([1 - x], [y], s=60, facecolor="none", edgecolor=GRID, lw=1.5, zorder=2)
ax.set_yticks(range(len(locs))); ax.set_yticklabels(locs[::-1]); ax.tick_params(axis="y", length=0, labelcolor=INK)
ax.set_xticks([0, 1]); ax.set_xticklabels(["mg/kg", "μg/kg"], fontsize=11, color=INK)
ax.xaxis.tick_top(); ax.tick_params(axis="x", length=0)
ax.set_xlim(-0.6, 2.6); ax.set_ylim(-0.6, len(locs) - 0.4)
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ax.spines.values(): s.set_visible(False)
ax.text(1.35, len(locs) - 1, "34.1 mg/kg (as printed)\n= 34,100 μg/kg", va="center", fontsize=10, color=INK2)
ax.text(1.35, 2.5, "All six body locations:\n2, 9.8, 34.1 μg/kg\n(filled dot = unit used;\nopen = not used)", va="center", fontsize=10, color=INK2)
save(fig, "candidate5"); plt.close(fig)
for f in (cand1, cand2, cand3, cand4, cand5):
f()
print("\nfigures written to", HERE)
The abstract's anxiety odds ratio (5.80) lies outside its own 95% CI (1.19 to 4.22)
Ann Rheum Dis (2017) · doi:10.1136/annrheumdis-2017-212000 · PMC5867422
Summary
The abstract reports that high fear scores on the FAIR questionnaire were associated with high HADS anxiety, "OR 5.80, 95% CI (1.19 to 4.22)". The point estimate is above the upper limit of its own interval, which a Wald interval cannot produce. The geometric centre of the printed interval is 2.24, while the abstract's other four odds ratios sit at the centres of their intervals to within rounding. Figure 3, the paper's forest plot, splits HADS anxiety into two categories. Read by pixel, its 8–10 vs <8 row has an interval of about 1.20 to 4.18, which matches the abstract's interval, and its >10 vs <8 row has an OR of about 5.97, which matches the abstract's 5.80. The abstract therefore appears to pair the estimate of one category with the interval of the other, without naming either. Under either pairing the association remains positive and its interval excludes 1, but the size of the effect given in the abstract is not a valid estimate-interval pair.
What the paper reports
Abstract, Results (identical in the PMC archive and in the publisher's typeset PDF):
High fear scores were associated with high Arthritis Helplessness Index scores (OR 6.85, 95% CI (3.95 to 11.87)); high Hospital Anxiety and Depression Scale anxiety (OR 5.80, 95% CI (1.19 to 4.22)) and depression (OR 2.37, 95% CI (1.29 to 4.37)) scores; low education level (OR 3.48, 95% CI (1.37 to 8.83)); and high perceived disease activity (OR 2.36, 95% CI (1.10 to 5.04)).
Methods, statistical analysis:
Univariate, then multivariate logistic regression was used to identify patient variables (including demographic, social and economic characteristics; disease status, and anxiety/depression and helplessness levels) independently associated with the highest compared with the lowest fear score cluster.
Results, "Factors associated with fear scores" (the text gives no numbers and refers to Figure 3):
Cluster 1 (high fear scores) was associated with higher global rating of disease activity by the patient, high AHI helplessness scores and high HADS anxiety and depression scores.
Figure 3 caption:
High scores indicate those >75 (shown in red); moderate scores, 50–75 (shown in blue); and low scores, <50. Data are presented as ORs (95% CIs).
Figure 3 itself is an image. Its HADS anxiety block has two rows, labeled "8-10 vs <8" and ">10 vs <8". It shows markers on a log axis from 0.1 to 10 and prints no numbers.
Discussion, second and third paragraphs:
In contrast, a strong association was observed between FAIR scores and scores on the AHI (≥20) or HADS (≥10 for anxiety and ≥8 for depression), all of which are non-specific markers of psychological distress.
In agreement with this, we observed a robust association between fears and non-specific measures of psychological distress, such as the AHI, the HADS anxiety score and, to a lesser extent, the HADS depression score.
What the paper's own numbers show
Method. An odds ratio from logistic regression with a Wald interval satisfies CI = exp(b ∓ 1.96·SE) with OR = exp(b). So the OR must lie inside its interval and equal the interval's geometric centre, √(lower × upper), up to rounding. Each arm of the interval implies an SE(log OR): ln(OR/lower)/1.96 for the lower arm and ln(upper/OR)/1.96 for the upper arm. The two must agree and both must be positive. I applied this to all five odds ratios in the abstract.
Figure 3 gives the HADS anxiety association in two categories, so I also measured its red markers (high vs low fear cluster) by pixel position. I calibrated the log axis on its tick marks at 0.1, 1 and 10, which are exactly 194 px per decade apart, so 1 px is about 1.2% of an OR. As a check on the method, the four abstract ORs that correspond to a single Figure 3 row read 0.4% to 2.1% high, and their lower limits match to within 0.02. Upper ends beyond 10 are cut off at the edge of the plot, as the AHI row shows: its printed upper limit is 11.87, but the line stops at about 9.8. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| HADS anxiety OR and 95% CI | Abstract | 5.80 (1.19 to 4.22) | OR lies outside its CI: 1.58 above the upper limit (a factor of 1.37) |
| Geometric centre of the HADS anxiety CI | Abstract | — | √(1.19 × 4.22) = 2.24; reported OR / centre = 2.59 |
| SE(log OR) implied by each arm, HADS anxiety | derived | — | lower arm 0.808, upper arm −0.162 (a negative SE is impossible); symmetric SE 0.323 |
| Interval of the printed width centred on 5.80 | derived | — | 3.08 to 10.92 |
| AHI OR and 95% CI | Abstract | 6.85 (3.95 to 11.87) | centre 6.85; arms 0.281 / 0.280 (consistent) |
| HADS depression OR and 95% CI | Abstract | 2.37 (1.29 to 4.37) | centre 2.37; arms 0.310 / 0.312 (consistent) |
| Low education OR and 95% CI | Abstract | 3.48 (1.37 to 8.83) | centre 3.48; arms 0.476 / 0.475 (consistent) |
| High perceived activity OR and 95% CI | Abstract | 2.36 (1.10 to 5.04) | centre 2.35; arms 0.389 / 0.387 (consistent) |
| HADS anxiety 8–10 vs <8 | Figure 3 (image, no numbers) | marker and line | OR ≈ 2.28, line ≈ 1.20 to ≈ 4.18: matches the abstract's interval and its centre (2.24) |
| HADS anxiety >10 vs <8 | Figure 3 (image, no numbers) | marker and line | OR ≈ 5.97, line ≈ 3.03 to ≥ 9.94 (cut off at 10): matches the abstract's point estimate (5.80) |
| Figure-reading check, AHI / depression / education / activity | Figure 3 vs Abstract | 6.85 / 2.37 / 3.48 / 2.36 | read as 6.88 / 2.42 / 3.52 / 2.41 (ratio to printed 1.004 to 1.021) |
| Significance, if the OR is 2.24 with the printed CI | derived | — | z = 2.50, p ≈ 0.013 |
| Significance, if the OR is 5.80 with an SE like the printed CI's | derived (illustration only) | — | 3.08 to 10.92; z = 5.44 |
The lower end of the Figure 3 >10 row (≈ 3.03) lies close to the lower limit of an interval of the printed width centred on 5.80 (3.08). Its lower arm implies an SE of 0.346, against 0.323 for the printed interval. That fits the reading that 5.80 is the >10 estimate and that its own interval was replaced by the 8–10 interval.
A separate, smaller inconsistency turned up in the same abstract. It does not affect the odds ratios:
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Mean FAIR score, RA | Abstract vs Results | Abstract: "51.2 (±25.4) in RA"; Results: "51.8±25.4" | Overall mean 54.9 with axSpA 60.5 (n = 432 and 240) implies an RA mean of 51.79, so 51.8 fits. With 51.2, the overall mean would be 54.52, not 54.9 |
Figure
Figure 1. HADS anxiety odds ratios for the high vs low fear cluster, on one log axis with the null value (OR = 1) as a thin vertical line. Orange marks are numbers printed in the abstract. Blue marks are recomputed from them: the geometric centre of the printed interval (2.24) and an interval of the printed width centred on the printed OR (3.08 to 10.92). Gray hollow diamonds are approximate readings of the two HADS anxiety rows of the paper's Figure 3, measured by pixel position; the >10 row's line runs to the edge of that plot at 10, so its upper end is a lower bound. The printed pair (top row) is impossible because 5.80 lies above 4.22. Each of the two consistent pairings agrees with one Figure 3 row, and both lie wholly above 1.
Figure 2. The abstract's estimate and interval trace to different rows of Figure 3. The printed interval, 1.19 to 4.22, matches the 8–10 vs <8 row (≈ 1.20 to ≈ 4.18). The printed point estimate, 5.80, matches the >10 vs <8 row (≈ 5.97). The abstract names neither category. Figure 3 values are approximate readings, because the figure prints no numbers.
Why it matters
The HADS anxiety estimate is one of five headline associations in the abstract, and for most readers the abstract is the only place it appears as a number. The Results text gives no odds ratios, and Figure 3 shows them only graphically.
- The printed pair cannot be used as published. A reader or meta-analyst who takes the SE from the printed interval and the estimate from the printed OR gets a log OR that is about one SE above the interval's upper limit. A reader who back-calculates from either arm alone gets an SE of 0.808 or a negative one.
- The size of the anxiety effect depends on which number is right. With the >10 category the OR is about 5.8, close to the AHI estimate (6.85) and well above depression (2.37). With the 8–10 category it is about 2.2–2.3, about the same as depression. The Discussion's statement that the association is with "the HADS anxiety score and, to a lesser extent, the HADS depression score" holds for the >10 category but not for the 8–10 category. The abstract does not say which category its number refers to, and readers have no way to tell.
- Direction and significance are not affected. Both consistent pairings lie wholly above 1 (z = 2.50 for 2.24 with the printed interval). Figure 3, which appears to be drawn correctly, shows the same. The abstract's conclusion that "High fear scores were associated with psychological distress" is unaffected.
Explanations considered
- Conversion or typesetting artifact in PMC. This is excluded. The publisher's typeset PDF (InDesign file, created 2018-01-11) has the same numbers, as do the PMC HTML, the Europe PMC XML and the PubMed record. The numbers involve no minus signs, superscripts or table cells that a conversion could garble.
- An asymmetric (profile-likelihood or exact) interval. This does not account for the numbers. An asymmetric interval still contains its own estimate, and 5.80 lies outside 1.19 to 4.22. The other four abstract intervals are symmetric on the log scale (their two arms differ by at most 0.002 in SE units), which indicates that Wald intervals were used.
- A different confidence level (90% or 99%). This does not account for the numbers. Any interval for an estimate contains that estimate, whatever its level.
- A single-digit misprint in the OR. This does not fit. With the printed interval, an OR of 2.80, 3.80, 4.80 or 5.30 gives arms of 0.437/0.209, 0.592/0.053, 0.712/−0.066 and 0.762/−0.116. Only a value near 2.24 makes the arms agree, and 2.24 is not one keystroke away from 5.80.
- A misprint in the interval. This is possible but less likely than the pairing explanation. The printed interval is not arbitrary: it matches the Figure 3 8–10 row to within the precision of the reading at both ends.
- The abstract's estimate and interval come from the two HADS anxiety categories of Figure 3. This accounts for all the numbers. The interval matches the 8–10 vs <8 row, and the point estimate matches the >10 vs <8 row. Which category the abstract meant to report cannot be confirmed without the regression output. The words "high ... anxiety" and the Discussion's cut-off of "≥10 for anxiety" suggest the upper category. Even so, ≥10 is not the same as the figure's >10, and a score of 10 falls in the figure's 8–10 row.
- Imprecise figure reading. This does not change the conclusion. The impossibility of the printed pair follows from the abstract's numbers alone. The figure readings agree with the four unambiguous abstract ORs to within about 2%, while the two HADS anxiety rows differ from each other by a factor of about 2.6.
Status
- Version of record. I checked the publisher's typeset PDF (6 pages, Adobe InDesign CS6, created 2018-01-11, from the PMC open-access bucket,
PMC5867422.1.pdf). Its abstract reads "Scale anxiety (or 5.80, 95% CI (1.19 to 4.22))" and "51.2 (±25.4) in rA", where "or" and "rA" are the PDF's small-caps text, and its Results read "(51.8±25.4; 52 (33–71))". The publisher's web pages (ard.bmj.com, ard.eular.org) returned HTTP 403 to automated access. The PMC HTML, Europe PMC full-text XML and the PubMed record (PMID 29102955) carry the same values. The article appeared online in 2017 and in print in 2018 (77(2):258–263). Every quote above was confirmed verbatim against the PMC-archived text. - Figure 3. I measured the publisher's image
annrheumdis-2017-212000f03.jpg(703 × 1015 px) from Europe PMC.recompute.py.txt --figure <file>repeats the measurement, and the script records the file's SHA-256 hash. - Supplements. The four supplementary files (a flow chart, a correlation figure, ROC curves and tables of the questionnaire items) contain no odds ratios and do not resolve the inconsistency.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, PMC and Europe PMC list no correction or comment links, PubMed has no CommentsCorrections entry, and the typeset PDF carries no notice. PubPeer's API and search page refused automated access (Cloudflare challenge and HTTP 403). A web search found no PubPeer entry and no discussion of these numbers.
- Sources. PMC article PMC5867422; publisher PDF via the PMC open-access bucket; Figure 3 and supplementary files via Europe PMC; Crossref; Europe PMC; PubMed; PubPeer (attempted).
Suggested correction
- In the abstract, replace "OR 5.80, 95% CI (1.19 to 4.22)" with an estimate and interval from the same row of the regression output, and name the category. For example: "HADS anxiety >10 vs <8 (OR 5.80, 95% CI (x to y))", or the 8–10 vs <8 estimate with 1.19 to 4.22, whichever was intended. Reporting both categories would remove the ambiguity.
- Give the Figure 3 odds ratios and intervals as numbers, in the figure or in a table (online if necessary), so that readers do not have to read them off the plot. Extending the axis past 10 would show the full AHI and HADS >10 intervals.
- Make the HADS anxiety cut-off consistent between the Discussion ("≥10 for anxiety") and Figure 3 (">10").
- Minor: change the abstract's RA mean FAIR score from 51.2 to 51.8, which agrees with the Results and with the overall mean of 54.9.
Recomputation script (python recompute.py.txt)
"""Recompute the internal consistency of the odds ratios in the abstract of PMC5867422
(Ann Rheum Dis 2017, doi:10.1136/annrheumdis-2017-212000, the FAIR questionnaire paper).
Run: python recompute.py.txt # prints all numbers, writes fig1/fig2 (.svg, .png)
python recompute.py.txt --candidates # also writes five candidate figure designs
# to candidates/ (A-E) for review
python recompute.py.txt --figure f03.jpg # re-measures Figure 3 from the publisher's image
Needs numpy and matplotlib.
Inputs are copied from the paper as archived in PMC (identical in the publisher's typeset PDF):
Abstract (Results): the five odds ratios with 95% CIs, the RA and axSpA mean scores;
Results ("Distribution of scores"): overall mean 54.9, RA 51.8, axSpA 60.5; n = 432 RA, 240 axSpA.
Method: an odds ratio from logistic regression with a Wald interval satisfies
CI = exp(b -/+ 1.959964 * SE), OR = exp(b),
so the OR must lie inside its CI and equal the geometric centre sqrt(lower * upper)
(up to rounding), and SE(log OR) = (ln upper - ln lower) / (2 * 1.959964).
Figure 3 (the paper's forest plot) gives the HADS anxiety OR in two categories (8-10 vs <8 and
>10 vs <8) but prints no numbers. Its red markers (high vs low fear cluster) were read by pixel
position on the log axis, calibrated on the tick marks at 0.1, 1 and 10 (x = 145.5, 339.5, 533.5 px;
194 px per decade, so 1 px is about 1.2% of an OR). The readings stored below were produced by
read_figure3() on the image file annrheumdis-2017-212000f03.jpg (703 x 1015 px, SHA-256
903d9a0e66f16d94a68154971c8494fc339ad26b9623da0327d18fab51888df0), downloaded from
https://europepmc.org/api/fulltextRepo?pprId=PMC5867422&type=FILE&fileName=annrheumdis-2017-212000f03.jpg&mimeType=image/jpeg
Pass --figure <path> to repeat the measurement. Interval ends drawn past OR = 10 are cut off at
the edge of the plot, so an upper end read near 10 is a lower bound for the true limit.
"""
import os
import sys
from math import exp, log, sqrt, erf
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.ticker import FixedLocator, NullLocator, FuncFormatter # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = 1.959964
# ---------------------------------------------------------------- inputs from the paper
ABSTRACT = { # label: (OR, lower, upper), Abstract (Results)
"Helplessness (AHI)": (6.85, 3.95, 11.87),
"HADS anxiety": (5.80, 1.19, 4.22),
"HADS depression": (2.37, 1.29, 4.37),
"Low education": (3.48, 1.37, 8.83),
"High perceived activity": (2.36, 1.10, 5.04),
}
# Figure 3 rows matching each abstract OR (red markers, cluster 1 vs cluster 3)
FIG_ROW_FOR = {
"Helplessness (AHI)": "AHI >=20 vs <20",
"HADS depression": "HADS depression >=8 vs <8",
"Low education": "Primary vs tertiary education",
"High perceived activity": "Patient-rated activity >60 vs <=30",
}
FIG_LABELS = ["AHI >=20 vs <20", "HADS anxiety 8-10 vs <8", "HADS anxiety >10 vs <8",
"HADS depression >=8 vs <8", "Patient-rated activity 30-60 vs <=30",
"Patient-rated activity >60 vs <=30", "Foreign vs France born",
"Primary vs tertiary education", "Secondary vs tertiary education",
"Employed vs retired", "Unemployed vs retired", "Living alone vs with others"]
# Stored readings (point, left end, right end), produced by read_figure3() on the file above
FIG3_STORED = {
"AHI >=20 vs <20": (6.88, 3.94, 9.82),
"HADS anxiety 8-10 vs <8": (2.28, 1.20, 4.18),
"HADS anxiety >10 vs <8": (5.97, 3.03, 9.94),
"HADS depression >=8 vs <8": (2.42, 1.31, 4.23),
"Patient-rated activity 30-60 vs <=30": (1.85, 1.02, 3.30),
"Patient-rated activity >60 vs <=30": (2.41, 1.11, 4.99),
"Foreign vs France born": (2.30, 1.09, 4.65),
"Primary vs tertiary education": (3.52, 1.37, 8.72),
"Secondary vs tertiary education": (1.67, 0.90, 2.89),
"Employed vs retired": (3.54, 1.78, 6.72),
"Unemployed vs retired": (3.02, 1.31, 6.64),
"Living alone vs with others": (2.34, 1.20, 4.49),
}
N_RA, N_AXSPA = 432, 240
MEAN_ALL, MEAN_AXSPA = 54.9, 60.5
MEAN_RA_ABSTRACT, MEAN_RA_RESULTS = 51.2, 51.8
def read_figure3(path):
"""Measure the red markers of Figure 3 by pixel. Returns {label: (OR, left, right)}."""
import matplotlib.image as mimg
img = mimg.imread(path).astype(int)
R, G, B = img[..., 0], img[..., 1], img[..., 2]
red = (R > 170) & (G < 120) & (B < 120)
red[:, 560:] = False # drop the legend
x1, ppd = 339.5, 194.0 # OR = 1 tick, pixels per decade (ticks 145.5/339.5/533.5)
to_or = lambda x: 10 ** ((x - x1) / ppd) # noqa: E731
rows = np.where(red.sum(1) > 0)[0]
groups, cur = [], [rows[0]]
for r in rows[1:]:
if r - cur[-1] <= 3:
cur.append(r)
else:
groups.append(cur)
cur = [r]
groups.append(cur)
out = {}
for lab, g in zip(FIG_LABELS, groups):
cc = red[g[0]:g[-1] + 1].sum(0)
xs = np.where(cc > 0)[0]
dot = np.where(cc >= 0.8 * cc.max())[0]
out[lab] = (round(to_or(dot.mean()), 2), round(to_or(xs.min()), 2),
round(to_or(xs.max() + 1), 2))
return out
def p_two_sided(z):
return 2 * (1 - 0.5 * (1 + erf(abs(z) / sqrt(2))))
# ---------------------------------------------------------------- 1. abstract ORs
print("=== 1. Abstract ORs: does each OR sit at the geometric centre of its 95% CI? ===")
RES = {}
for k, (o, lo, hi) in ABSTRACT.items():
gm = sqrt(lo * hi)
se_lo = (log(o) - log(lo)) / Z
se_hi = (log(hi) - log(o)) / Z
se_sym = (log(hi) - log(lo)) / (2 * Z)
RES[k] = dict(gm=gm, se_lo=se_lo, se_hi=se_hi, se=se_sym, inside=lo <= o <= hi)
print(f"{k:25s} OR {o:5.2f} CI ({lo:.2f}, {hi:5.2f}) inside={str(lo <= o <= hi):5s} "
f"centre sqrt(lo*hi) = {gm:5.2f} ratio OR/centre = {o / gm:5.3f} "
f"SE from lower arm = {se_lo:6.3f}, upper arm = {se_hi:6.3f}, symmetric = {se_sym:.3f}")
o, lo, hi = ABSTRACT["HADS anxiety"]
a = RES["HADS anxiety"]
hw = (log(hi) - log(lo)) / 2
same_width = (exp(log(o) - hw), exp(log(o) + hw))
print("\n=== 2. HADS anxiety pair ===")
print(f"5.80 exceeds the upper limit 4.22 by {o - hi:.2f} (a factor of {o / hi:.3f}); "
f"on the log scale by {(log(o) - log(hi)) / a['se']:.2f} SE (SE = {a['se']:.3f} from the printed CI)")
print(f"OR / centre of printed CI = {o / a['gm']:.2f}")
print(f"Interval of the printed width centred on 5.80: ({same_width[0]:.2f}, {same_width[1]:.2f})")
zc = log(a["gm"]) / a["se"]
zo = log(o) / a["se"]
print(f"Pairing 1, OR {a['gm']:.2f} (1.19 to 4.22): z = {zc:.2f}, p = {p_two_sided(zc):.4f}")
print(f"Pairing 2, OR 5.80 ({same_width[0]:.2f} to {same_width[1]:.2f}) if SE were the same: "
f"z = {zo:.2f}, p = {p_two_sided(zo):.1e}")
print("Single-digit misprints of the OR, checked against the printed CI (arms must agree):")
for cand in (2.80, 3.80, 4.80, 5.30):
print(f" OR {cand:.2f}: SE from lower arm {(log(cand) - log(lo)) / Z:.3f}, "
f"upper arm {(log(hi) - log(cand)) / Z:.3f}")
# ---------------------------------------------------------------- 3. Figure 3
fig_path = sys.argv[sys.argv.index("--figure") + 1] if "--figure" in sys.argv else None
FIG3 = read_figure3(fig_path) if fig_path else FIG3_STORED
print(f"\n=== 3. Figure 3 red markers ({'measured from ' + fig_path if fig_path else 'stored readings'}) ===")
for lab in FIG_LABELS:
p, l_, r_ = FIG3[lab]
print(f"{lab:38s} OR ~{p:5.2f} line from ~{l_:4.2f} to ~{r_:5.2f}")
print("\nCalibration: abstract value vs Figure 3 reading for the four unambiguous ORs")
ratios = []
for k, lab in FIG_ROW_FOR.items():
o_, l_, h_ = ABSTRACT[k]
p, fl, fr = FIG3[lab]
ratios.append(p / o_)
clip = " (true upper limit beyond the plot edge at 10)" if h_ > 10 else ""
print(f"{k:25s} abstract {o_:5.2f} ({l_:.2f}, {h_:5.2f}) figure {p:5.2f} ({fl:.2f}, {fr:5.2f}) "
f"point ratio {p / o_:.3f}{clip}")
print(f"mean point ratio figure/abstract = {np.mean(ratios):.3f} (range {min(ratios):.3f}-{max(ratios):.3f})")
f810, f10 = FIG3["HADS anxiety 8-10 vs <8"], FIG3["HADS anxiety >10 vs <8"]
print("\nHADS anxiety: which Figure 3 row does each abstract number match?")
print(f"abstract CI (1.19, 4.22) vs 8-10 row line ({f810[1]:.2f}, {f810[2]:.2f}); "
f"vs >10 row line ({f10[1]:.2f}, {f10[2]:.2f})")
print(f"abstract OR 5.80 vs >10 row point {f10[0]:.2f} (ratio {f10[0] / 5.80:.3f}); "
f"vs 8-10 row point {f810[0]:.2f}")
print(f"centre of abstract CI {a['gm']:.2f} vs 8-10 row point {f810[0]:.2f} (ratio {f810[0] / a['gm']:.3f})")
print(f"interval of printed width centred on 5.80 ({same_width[0]:.2f}, {same_width[1]:.2f}) vs "
f">10 row ({f10[1]:.2f}, >= {f10[2]:.2f}, cut off at the plot edge)")
se_fig10 = (log(f10[0]) - log(f10[1])) / Z
print(f">10 row: SE(log OR) from the figure's lower arm = {se_fig10:.3f}; implied upper limit "
f"{exp(log(f10[0]) + Z * se_fig10):.2f}")
# ---------------------------------------------------------------- 4. side check
print("\n=== 4. Side check: RA mean FAIR score (abstract 51.2 vs Results 51.8) ===")
ra_implied = (MEAN_ALL * (N_RA + N_AXSPA) - N_AXSPA * MEAN_AXSPA) / N_RA
print(f"RA mean implied by overall 54.9 and axSpA 60.5: {ra_implied:.2f}")
for m in (MEAN_RA_ABSTRACT, MEAN_RA_RESULTS):
print(f"overall mean if RA = {m}: {(N_RA * m + N_AXSPA * MEAN_AXSPA) / (N_RA + N_AXSPA):.2f}")
# ================================================================= figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, GRAY = "#eb6834", "#2a78d6", "#52514e"
plt.rcParams.update({
"font.family": ["DejaVu Sans"], "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "path", "lines.linewidth": 2,
})
MS = 9 # marker size in points (>= 8 px)
def save(fig, stem):
fig.savefig(stem + ".svg", bbox_inches="tight", pad_inches=0.12)
fig.savefig(stem + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
def log_axis(ax, lo=0.7, hi=16, ticks=(1, 2, 3, 5, 10)):
ax.set_xscale("log")
ax.set_xlim(lo, hi)
ax.xaxis.set_major_locator(FixedLocator(list(ticks)))
ax.xaxis.set_minor_locator(NullLocator())
ax.xaxis.set_major_formatter(FuncFormatter(lambda v, _: f"{v:g}"))
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.axvline(1, color=INK2, lw=0.9, zorder=1)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Odds ratio, high vs low fear cluster (log scale; thin line = no association)")
def interval(ax, y, pt, l_, r_, ptc, linec, marker="o", hollow=False, arrow=False):
ax.plot([l_, r_], [y, y], color=linec, lw=2, solid_capstyle="butt", zorder=2)
for x in (l_, r_):
ax.plot([x, x], [y - 0.12, y + 0.12], color=linec, lw=2, zorder=2)
if arrow:
ax.annotate("", xy=(r_ * 1.16, y), xytext=(r_, y),
arrowprops=dict(arrowstyle="-|>", color=linec, lw=2, shrinkA=0, shrinkB=0))
ax.plot([pt], [y], marker=marker, ms=MS + (1 if marker == "D" else 2), zorder=3,
mfc=BG if hollow else ptc, mec=ptc, mew=2, ls="none")
def legend_handles(items):
hs = []
for lab, kw in items:
hs.append(plt.Line2D([], [], label=lab, **kw))
return hs
# ---- Candidate A: the printed pair and the centre of its own interval
def fig_core(stem):
fig, ax = plt.subplots(figsize=(7.2, 2.4))
interval(ax, 1, 5.80, 1.19, 4.22, ORANGE, ORANGE)
interval(ax, 0, a["gm"], 1.19, 4.22, BLUE, ORANGE)
ax.set_yticks([1, 0])
ax.set_yticklabels(["Abstract as printed:\nOR 5.80 (1.19 to 4.22)",
f"Centre of the printed CI:\nsqrt(1.19 x 4.22) = {a['gm']:.2f}"], color=INK)
ax.set_ylim(-0.6, 1.7)
log_axis(ax)
ax.annotate("5.80 lies above its own\nupper limit (4.22)", xy=(5.80, 1.12), xytext=(7.4, 1.45),
fontsize=10, color=INK, va="center",
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.9))
ax.legend(handles=legend_handles([
("printed in the abstract", dict(color=ORANGE, marker="o", ms=MS, lw=2)),
("recomputed from the printed CI", dict(color=BLUE, marker="o", ms=MS, ls="none"))]),
loc="upper center", bbox_to_anchor=(0.45, -0.32), ncol=2, frameon=False, fontsize=10)
save(fig, stem)
# ---- Candidate B: all five abstract ORs, reported point vs geometric centre
def fig_all_ors(stem):
keys = list(ABSTRACT)
fig, ax = plt.subplots(figsize=(7.2, 3.3))
for i, k in enumerate(keys):
y = len(keys) - 1 - i
o_, l_, h_ = ABSTRACT[k]
ax.plot([l_, h_], [y, y], color=ORANGE, lw=2, zorder=2)
for x in (l_, h_):
ax.plot([x, x], [y - 0.12, y + 0.12], color=ORANGE, lw=2)
ax.plot([RES[k]["gm"]], [y], marker="o", ms=MS + 5, mfc="none", mec=BLUE, mew=2, zorder=4)
ax.plot([o_], [y], marker="o", ms=MS, color=ORANGE, zorder=3)
txt = (f"OR {o_:.2f}, centre {RES[k]['gm']:.2f}" if abs(o_ / RES[k]["gm"] - 1) < 0.01
else f"OR {o_:.2f}, centre {RES[k]['gm']:.2f}: OR outside its CI")
ax.text(17.5, y, txt, va="center", fontsize=10,
color=INK if abs(o_ / RES[k]["gm"] - 1) < 0.01 else INK, weight="normal"
if abs(o_ / RES[k]["gm"] - 1) < 0.01 else "bold")
ax.set_yticks(range(len(keys)))
ax.set_yticklabels(keys[::-1], color=INK)
ax.set_ylim(-0.7, len(keys) - 0.3)
log_axis(ax, 0.7, 16)
ax.legend(handles=legend_handles([
("OR and 95% CI as printed in the abstract", dict(color=ORANGE, marker="o", ms=MS, lw=2)),
("geometric centre of the printed CI (recomputed)",
dict(color=BLUE, marker="o", ms=MS + 5, mfc="none", mew=2, ls="none"))]),
loc="upper center", bbox_to_anchor=(0.5, -0.28), ncol=1, frameon=False, fontsize=10)
save(fig, stem)
# ---- Candidate C: the two arms of each interval, expressed as an implied SE
def fig_arms(stem):
keys = list(ABSTRACT)
fig, ax = plt.subplots(figsize=(7.2, 3.2))
for i, k in enumerate(keys):
y = len(keys) - 1 - i
r = RES[k]
ax.plot([r["se_lo"], r["se_hi"]], [y + 0.13, y - 0.13], color=GRID, lw=2, zorder=1)
ax.plot([r["se_lo"]], [y + 0.13], marker="o", ms=MS, color=BLUE, ls="none", zorder=3)
ax.plot([r["se_hi"]], [y - 0.13], marker="s", ms=MS, mfc=BG, mec=BLUE, mew=2, ls="none", zorder=3)
ax.text(0.95, y, f"{r['se_lo']:.3f} / {r['se_hi']:.3f}", va="center", fontsize=10, color=INK)
ax.axvline(0, color=INK2, lw=0.9)
ax.set_xlim(-0.3, 1.2)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_yticks(range(len(keys)))
ax.set_yticklabels(keys[::-1], color=INK)
ax.set_ylim(-0.7, len(keys) - 0.3)
ax.set_xlabel("SE(log OR) implied by each arm of the printed CI\n"
"(lower arm = ln(OR/lower)/1.96; upper arm = ln(upper/OR)/1.96; must agree and be > 0)")
ax.legend(handles=legend_handles([
("from the lower arm", dict(color=BLUE, marker="o", ms=MS, ls="none")),
("from the upper arm", dict(color=BLUE, marker="s", ms=MS, mfc=BG, mew=2, ls="none"))]),
loc="lower right", bbox_to_anchor=(1.0, 1.0), ncol=2, frameon=False, fontsize=10)
save(fig, stem)
# ---- Candidate D: text panels, abstract excerpt vs the two Figure 3 rows
def fig_text_panels(stem):
from matplotlib.patches import FancyBboxPatch
fig = plt.figure(figsize=(7.2, 3.0))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
for x0, w, title in ((0.01, 0.38, "Abstract (Results), as printed"),
(0.60, 0.39, "Figure 3, high vs low fear\n(red markers), read by pixel")):
ax.add_patch(FancyBboxPatch((x0, 0.13), w, 0.84, boxstyle="round,pad=0.0,rounding_size=0.015",
fc="#f3f2ee", ec=GRID, lw=1))
ax.text(x0 + 0.02, 0.88, title, fontsize=10.5, weight="bold", va="center", color=INK)
ax.text(0.03, 0.70, "\u201chigh Hospital Anxiety and\nDepression Scale anxiety", fontsize=10.5,
va="center", color=INK)
ax.text(0.03, 0.53, "(OR 5.80,", fontsize=12, va="center", color=ORANGE, weight="bold")
ax.text(0.03, 0.38, "95% CI (1.19 to 4.22))\u201d", fontsize=12, va="center", color=ORANGE,
weight="bold")
ax.text(0.03, 0.22, "no HADS category is named", fontsize=10, va="center", color=INK2, style="italic")
p1, p2 = FIG3["HADS anxiety 8-10 vs <8"], FIG3["HADS anxiety >10 vs <8"]
ax.text(0.62, 0.66, "HADS anxiety >10 vs <8", fontsize=10.5, va="center", color=INK, weight="bold")
ax.text(0.62, 0.58, f"OR \u2248{p2[0]:.2f} (\u2248{p2[1]:.2f} to \u226510;", fontsize=10.5,
va="center", color=GRAY)
ax.text(0.62, 0.50, "line cut off at the plot edge)", fontsize=10.5, va="center", color=GRAY)
ax.text(0.62, 0.34, "HADS anxiety 8\u201310 vs <8", fontsize=10.5, va="center", color=INK, weight="bold")
ax.text(0.62, 0.26, f"OR \u2248{p1[0]:.2f} (\u2248{p1[1]:.2f} to \u2248{p1[2]:.2f})", fontsize=10.5,
va="center", color=GRAY)
kw = dict(arrowstyle="-|>", color=INK2, lw=1.4, shrinkA=4, shrinkB=4)
ax.annotate("", xy=(0.605, 0.62), xytext=(0.175, 0.53), arrowprops=kw)
ax.annotate("", xy=(0.605, 0.30), xytext=(0.355, 0.38), arrowprops=kw)
ax.text(0.495, 0.60, "point estimate\nmatches", fontsize=10, ha="center", va="center", color=INK,
bbox=dict(fc=BG, ec="none", pad=1.5))
ax.text(0.495, 0.27, "interval\nmatches", fontsize=10, ha="center", va="center", color=INK,
bbox=dict(fc=BG, ec="none", pad=1.5))
ax.text(0.01, 0.05, "Orange: numbers printed in the abstract. Gray: approximate values read from "
"Figure 3, which prints no numbers.", fontsize=10, color=INK2, va="center")
save(fig, stem)
# ---- Candidate E: printed pair, the two internally consistent pairings, and Figure 3
def fig_pairings(stem):
fig, ax = plt.subplots(figsize=(7.2, 4.1))
p1, p2 = FIG3["HADS anxiety 8-10 vs <8"], FIG3["HADS anxiety >10 vs <8"]
rows = [
(6.0, "Abstract as printed", lambda y: interval(ax, y, 5.80, 1.19, 4.22, ORANGE, ORANGE)),
(4.3, f"Centre of printed CI ({a['gm']:.2f})\nwith the printed CI",
lambda y: interval(ax, y, a["gm"], 1.19, 4.22, BLUE, ORANGE)),
(3.3, "Figure 3: 8–10 vs <8",
lambda y: interval(ax, y, p1[0], p1[1], p1[2], GRAY, GRAY, marker="D", hollow=True)),
(1.6, "Printed OR 5.80 with\nthe printed CI’s width",
lambda y: interval(ax, y, 5.80, *same_width, ORANGE, BLUE)),
(0.6, "Figure 3: >10 vs <8",
lambda y: interval(ax, y, p2[0], p2[1], p2[2], GRAY, GRAY, marker="D", hollow=True, arrow=True)),
]
for y, _, draw in rows:
draw(y)
ax.set_yticks([r[0] for r in rows])
ax.set_yticklabels([r[1] for r in rows], color=INK)
ax.text(0.72, 5.15, "If the interval is right: HADS anxiety 8–10 vs <8", fontsize=10,
weight="bold", color=INK, va="center", bbox=dict(fc=BG, ec="none", pad=1.5))
ax.text(0.72, 2.45, "If the point estimate is right: HADS anxiety >10 vs <8", fontsize=10,
weight="bold", color=INK, va="center", bbox=dict(fc=BG, ec="none", pad=1.5))
ax.annotate("5.80 lies above its own\nupper limit (4.22)", xy=(5.80, 6.15), xytext=(7.3, 6.45),
fontsize=10, color=INK, va="center", arrowprops=dict(arrowstyle="-", color=INK2, lw=0.9))
ax.text(11.6, 1.6, f"{same_width[0]:.2f} to\n{same_width[1]:.2f}", fontsize=10, va="center", color=INK)
ax.text(12.2, 0.6, "cut off\nat 10", fontsize=10, va="center", color=INK2)
ax.text(4.6, 4.3, "1.19 to 4.22", fontsize=10, va="center", color=INK)
ax.set_ylim(0.0, 6.9)
log_axis(ax, 0.7, 20, ticks=(1, 2, 3, 5, 10, 20))
ax.legend(handles=legend_handles([
("printed in the abstract", dict(color=ORANGE, marker="o", ms=MS, lw=2)),
("recomputed from the printed numbers", dict(color=BLUE, marker="o", ms=MS, lw=2)),
("read from Figure 3 (approximate)",
dict(color=GRAY, marker="D", ms=MS, mfc=BG, mew=2, lw=2))]),
loc="upper center", bbox_to_anchor=(0.42, -0.2), ncol=3, frameon=False, fontsize=10,
handlelength=2.2, columnspacing=1.2)
save(fig, stem)
fig_pairings(os.path.join(HERE, "fig1"))
fig_text_panels(os.path.join(HERE, "fig2"))
print("\nwrote fig1.svg/png (printed pair vs consistent pairings) and fig2.svg/png (abstract vs Figure 3 rows)")
if "--candidates" in sys.argv:
cdir = os.path.join(HERE, "candidates")
os.makedirs(cdir, exist_ok=True)
for tag, fn in (("A_core_interval", fig_core), ("B_all_abstract_ors", fig_all_ors),
("C_interval_arms", fig_arms), ("D_text_panels", fig_text_panels),
("E_two_pairings", fig_pairings)):
fn(os.path.join(cdir, tag))
print("wrote candidates/A-E (.svg, .png)")
Printed 95% CI for a mortality predictor implies P ≈ 0.98, not the reported 0.006
BMC Med (2024) · doi:10.1186/s12916-024-03812-2 · PMC11656579
Summary
The paper fits a multivariable logistic model of mortality in patients who developed immune checkpoint inhibitor-related endocrinopathies (IREs; N = 173). It reports the pre-treatment platelet-to-lymphocyte ratio (PLR) as OR 1.004, 95% CI 0.7–1.4, P = 0.006, and names PLR as one of two significant predictors. The same three figures appear in the Abstract, in the Results text and in Table 5, and they cannot all be correct. A 95% CI of 0.7–1.4 contains 1 and implies a two-sided P of 0.98 (no lower than 0.73 under any rounding of the printed values). An OR of 1.004 with P = 0.006 would need a 95% CI of about 1.0011–1.0069. A reader therefore cannot tell from the paper whether PLR independently predicts mortality in this subgroup.
What the paper reports
After MVR, factors influencing mortality in the subgroup with IREs included a higher age (OR 1.1, 95% CI 1.04–1.2, P = 0.001) and platelet-to-lymphocyte ratio (OR 1.004, 95% CI 0.7–1.4, P = 0.006).
— Abstract, Results
According to the MVR analysis, only age at ICI initiation (OR 1.1, 95% CI: 1.04–1.2; P = 0.001) and PLR (OR 1.004, 95% CI: 0.7–1.4; P = 0.006) were found to be statistically significant factors for predicting mortality.
— Results, "Factors influencing mortality among patients who developed endocrinopathy post-ICI"
PLR | 1.001 (1.0001-1.003) | 0.03 | 1.004 (0.7-1.4) | 0.006
— Table 5, "Univariate and multivariate logistic regression analysis of factors influencing mortality following endocrinopathies post immune checkpoint inhibitor therapy", PLR row. The columns are univariate OR (95% CI), univariate P, multivariate OR (95% CI) and multivariate P. In the version of record the multivariate P of 0.006 is set in bold. The table footer says "P values in bold indicate statistical significance".
Similarly, mortality was predicted by age at ICI initiation and a higher pre-ICI PLR.
— Discussion, first paragraph
A P value of < 0.05 indicated statistical significance.
— Methods, "Statistical analysis"
Two related passages bear on how PLR is scaled. The first gives the definition (the archived text renders the superscript in "× 10³" as "103"):
Platelet-to-lymphocyte ratio (PLR) was calculated for the included patients using the formula PLR = Lymphocyte Count (× 103/μL) ÷ Platelet Count (× 103/μL).
— Methods, "Data sources and measures"
PLR | 145.3 (64.2-229.2) | 143.9 (27-259.6) | 0.7
— Table 3, PLR row (median and IQR, IRE vs no IRE)
What the paper's own numbers show
Method. A Wald 95% CI for a logistic-regression odds ratio is exp(ln OR ± 1.96 × SE), so it is symmetric on the log scale. The standard error can therefore be recovered from the printed bounds as SE = (ln hi − ln lo)/3.92, and the two-sided P follows from z = ln(OR)/SE. Run backwards, the 95% CI that must go with a given OR and P is exp(ln OR ± 1.96 × ln OR / z_P). To allow for rounding, each printed number was varied across its whole rounding interval (0.7 means 0.65–0.75, 1.004 means 1.0035–1.0045), and the estimate was also allowed to sit at the geometric centre of the CI. As a check on the method, the same calculation was run on nine other rows from Tables 4 and 5. Every number below is printed by recompute.py.txt (Python, numpy/scipy).
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Does the 95% CI contain 1? | Abstract; Results; Table 5 | 0.7–1.4 | Yes |
| SE of ln(OR) implied by the CI | Table 5 | 0.7–1.4 | (ln 1.4 − ln 0.7)/3.92 = 0.177 |
| Geometric centre of the CI | Table 5 | OR 1.004 | 0.990 |
| Two-sided P, multivariable PLR | Abstract; Results; Table 5 | 0.006 | 0.98 with OR 1.004 (z = 0.023); 0.95 with the CI centre; one-sided 0.49 |
| Smallest P under any rounding of the printed values | Table 5 | 0.006 | 0.976 with the estimate rounding to 1.004; 0.73 if the estimate is taken as the CI centre |
| P if 0.7–1.4 were a 90%, 99% or 99.9% CI | Table 5 | 0.006 | 0.985, 0.976, 0.970 |
| 95% CI required by OR 1.004 with P = 0.006 | Abstract; Results; Table 5 | 0.7–1.4 | 1.0011–1.0069 (SE 0.00145, 122 times smaller than the SE implied by the printed CI) |
| Required CI over rounding of OR and P | derived | 0.7–1.4 | lower bound 1.0010–1.0013, upper bound 1.0060–1.0078 |
| OR per 100 PLR units, if 1.004 is per unit | derived | 1.004 | 1.004^100 = 1.49, which lies outside 0.7–1.4 |
| Printed CI read as per 100 units, converted to per unit | derived | 0.7–1.4 | 0.9964–1.0034, which still contains 1 |
| Control: univariable PLR, 1.001 (1.0001–1.003) | Table 5 | P = 0.03 | 0.036 (0.027–0.044 over rounding of the bounds) |
| Control: multivariable age, 1.1 (1.04–1.2) | Abstract; Results; Table 5 | P = 0.001 | 0.0024 (0.0002–0.0075) |
| Control: univariable HbA1c, 1.3 (1.01–1.6) | Table 5 | P = 0.03 | 0.041 (0.036–0.046) |
| Control: IRE model, pembrolizumab, 2.6 (1.05–6.3) | Abstract; Table 4 | P = 0.04 | 0.039 (0.038–0.040) |
| PLR scale: Methods formula, lymphocytes ÷ platelets | Methods; Table 3 | medians 145.3 and 143.9 | Using the reference ranges in the Table 3 footer, lymphocytes ÷ platelets falls between 0.0024 and 0.020, and platelets ÷ lymphocytes between 50 and 410. The Table 3 values fit platelets ÷ lymphocytes |
Figure 2 and the script's output list the other control rows: univariable age, univariable haemoglobin, and IRE-model age, WBC and nivolumab. In those nine control rows the recomputed P agrees with the printed P to within rounding, or within about a factor of two where the printed estimate is coarsely rounded. In the multivariable PLR row, the printed P and the P implied by the printed CI differ by more than two orders of magnitude and fall on opposite sides of the paper's 0.05 threshold.
Figure
Figure 1. Odds ratio for mortality per unit PLR in the multivariable model of the IRE subgroup, on one log-scale axis. The thin vertical line marks OR = 1. The orange interval and dot show the paper's printed result: OR 1.004, 95% CI 0.7–1.4, stated P = 0.006. The interval straddles OR = 1. Its geometric centre is 0.990 (open blue diamond), and recomputed from its width it gives a two-sided P of 0.98 for an OR of 1.004. The filled blue diamond marks the 95% CI that an OR of 1.004 with P = 0.006 would actually require, 1.0011–1.0069. At this scale that interval is narrower than the marker. The printed interval and the printed P value cannot describe the same estimate.
Figure 2. Two-sided P values printed in Tables 4 and 5 (orange circles; the arrow marks a value printed as "<0.001") compared with P values recomputed from each row's printed 95% CI (blue diamonds, with the estimate at the CI's geometric centre). The light-blue bars show how far the recomputed P moves when each printed CI bound varies across its rounding interval. The dashed line marks P = 0.05. In nine rows the printed and recomputed values are close, which shows that the method recovers the paper's own reporting. The multivariable PLR row (top) is the exception. Its printed P is 0.006, while its printed CI gives 0.73–1.0.
Why it matters
Mortality in the IRE subgroup is one of the paper's stated secondary outcomes. PLR is one of only two variables the Abstract lists as multivariable predictors of it. The Results text and Table 5 repeat the result, and the Discussion's opening summary restates it ("mortality was predicted by age at ICI initiation and a higher pre-ICI PLR"). The printed figures support two incompatible readings:
- If the CI is correct, PLR is not a significant independent predictor (P ≈ 0.98). The claim in the Abstract, Results and Discussion would then be reversed.
- If the OR and the P value are correct, the CI is misprinted and would be about 1.001–1.007 per unit, roughly 1.12–1.98 per 100 units of PLR. The significance claim would stand, but the printed interval misstates its precision by a factor of about 120 on the log scale.
Nothing in the paper shows which reading is correct. The supplement contains no regression output that could settle it (see Status).
How PLR is defined adds a second difficulty with this variable. As written in Methods, the formula (lymphocytes ÷ platelets) would give values of about 0.002–0.02, not the medians near 145 shown in Table 3. If the values in Table 3 are platelets ÷ lymphocytes, as their size suggests, the formula in Methods is inverted. Either way, a reader cannot be sure what unit the per-unit OR refers to.
The paper's main conclusions do not depend on this row. These are the IRE incidence of 26.7%, the commonest IRE types, the predictors of IRE development, and the statement that "The development of IREs did not seem to influence mortality" (Abstract, Conclusions). The error affects a secondary, prognostic result, but that result is printed in the Abstract.
Explanations considered
- Rounding. This does not account for the numbers. With every printed value varied across its rounding interval, the smallest two-sided P that can be reached is 0.976 if the estimate rounds to 1.004, or 0.73 if the estimate is taken as the CI centre. In the other direction, the CI that would fit P = 0.006 (1.0011–1.0069) rounds to "1.0–1.0" at one decimal place, not to "0.7–1.4".
- A one-sided test. This does not account for the numbers. The one-sided P from the printed CI is 0.49. Methods mention no one-sided tests and set significance at P < 0.05.
- A confidence level other than 95%. This does not account for the numbers. Read as a 90%, 99% or 99.9% interval, 0.7–1.4 gives P between 0.970 and 0.985.
- A rescaled PLR unit (for example, OR per 10 or per 100 units). This does not account for the numbers. Neither Table 5 nor Methods states any rescaling. The OR and its CI are printed together, so they should be on the same scale, and on that scale the CI is centred at 0.990, not 1.004. If only the OR had been rescaled, 1.004 per unit would be 1.49 per 100 units, which lies outside 0.7–1.4. If the CI were per 100 units, it would be 0.9964–1.0034 per unit, which still contains 1.
- A non-Wald interval (profile likelihood or bootstrap). This does not account for the numbers. Such intervals can be somewhat asymmetric, but a 95% interval that runs from 0.7 to 1.4 around an estimate of 1.004 cannot accompany a test that rejects OR = 1 at P = 0.006. An interval obtained by inverting a test with P = 0.006 excludes 1 at the 95% level. An unstable fit with a very large SE would also give a large P, not a small one.
- Adjusted and unadjusted figures mixed together. This does not account for the numbers. All three figures sit in the multivariate columns of Table 5, and the Abstract and Results text label them as multivariable (MVR) results. The univariable PLR row (1.001, 1.0001–1.003, P = 0.03) is internally consistent (recomputed P 0.036).
- A conversion or typesetting artefact in the archived copy. This was ruled out. The publisher's HTML (link.springer.com, including the standalone Table 5 page), the PMC text and the Europe PMC full-text XML all give "1.004 (0.7-1.4)" and "0.006" in separate table cells. The Abstract and the Results text repeat the same figures in prose.
- A general reporting pattern. This does not account for the numbers. In nine other rows of Tables 4 and 5, the recomputed P values agree with the printed ones (Figure 2). The discrepancy is specific to this row.
From the paper's numbers alone, it is not possible to tell which of the three printed values is in error.
Status
As of 2026-10-23:
- Version of record: checked. The publisher's HTML at link.springer.com (published 18 December 2024) was rendered in headless Chromium. It shows the same Abstract sentence and the same Results sentence. The standalone Table 5 page gives the PLR row as "1.004 (0.7-1.4)" with "0.006" in bold. The PMC HTML and the Europe PMC full-text XML match. The publisher PDF could not be retrieved: the request returned an HTML interstitial page.
- Supplement: checked. Additional file 1 (12916_2024_3812_MOESM1_ESM.docx, Tables S1–S5) was downloaded, and its MD5 checksum matches the one listed in the PMC XML. It gives laboratory values (including platelets, lymphocytes and the platelet/lymphocyte ratio), follow-up endocrine profiles and subgroup characteristics. It contains no regression coefficients, standard errors or model output, so it does not resolve the discrepancy.
- Corrections: none found. Crossref lists no updates (no update-to or updated-by entries) and PMC lists no notices, both checked with
src/known.py. Europe PMC (PMID 39695571) shows no comment or correction links and no erratum records. The Retraction Watch database has no entry for the DOI. The publisher page shows no correction notice. - PubPeer: not reachable by automated query. The API returned HTTP 403, and the search page served a bot-verification challenge. Earlier web searches during this audit found no PubPeer thread, erratum or published comment. Fresh web searches could not be run today because the session's search quota was used up. A manual search at pubpeer.com is still advisable.
Sources: doi.org/10.1186/s12916-024-03812-2 · link.springer.com article · Table 5 page · PMC11656579 · Europe PMC record · Crossref works record
Suggested correction
The authors could re-check the multivariable logistic model of mortality in the IRE subgroup against the original software output. They could then report the PLR odds ratio, its 95% CI and its P value exactly as produced, so that the three values agree, and state the PLR unit that the OR refers to.
- If the CI is the misprinted value, the corrected interval should replace "0.7–1.4" in the Abstract, the Results text and Table 5.
- If the P value or the estimate is in error, the statement that PLR is a significant independent predictor of mortality should be revised in the Abstract, the Results and the Discussion.
Separately, the PLR formula in Methods ("Lymphocyte Count ÷ Platelet Count") should be checked against the values in Table 3, which fit platelets ÷ lymphocytes.
Recomputation script (python recompute.py.txt)
"""Recompute the multivariable PLR mortality result of PMC11656579
(BMC Med 2024, doi:10.1186/s12916-024-03812-2) and regenerate the figures.
Run: python recompute.py.txt
Needs: numpy, scipy, matplotlib.
Prints every recomputed number used in writeup.md and writes fig1.svg/.png and
fig2.svg/.png into the folder that holds this file.
All inputs are copied from the paper (Abstract, Results, Methods, Tables 3, 4, 5).
Method: Wald test on the log-odds scale. A 95% CI for an odds ratio is
exp(ln OR +/- 1.96 SE), so SE = (ln hi - ln lo) / (2 * 1.959964) and
z = ln(OR) / SE, two-sided P = 2 * (1 - Phi(|z|)).
"""
import itertools
import os
import numpy as np
from scipy.stats import norm
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975) # 1.959964
def se_from_ci(lo, hi, z=Z):
return (np.log(hi) - np.log(lo)) / (2 * z)
def p_two_sided(est, lo, hi, z=Z):
se = se_from_ci(lo, hi, z)
zz = np.log(est) / se
return se, zz, 2 * norm.sf(abs(zz))
def half_unit(s):
"""Half of the last printed digit ('0.7' -> 0.05; '1.0001' -> 0.00005).
A bound printed without decimals ('1') is treated as one decimal (1.0 +/- 0.05)."""
d = len(s.split(".")[1]) if "." in s else 1
return 0.5 * 10 ** (-d)
def interval(s, n=41):
v, h = float(s), half_unit(s)
return np.linspace(v - h, v + h - 1e-12, n)
def p_range(est_s, lo_s, hi_s, use_printed_est=True):
"""Min and max two-sided P over every value that rounds to the printed numbers.
The estimate is taken as the geometric midpoint of the (unrounded) CI and,
if use_printed_est, also anywhere inside its own rounding interval."""
ps = []
for lo, hi in itertools.product(interval(lo_s), interval(hi_s)):
if not (0 < lo < hi):
continue
mid = np.sqrt(lo * hi)
ps.append(p_two_sided(mid, lo, hi)[2])
if use_printed_est:
for est in interval(est_s, 11):
ps.append(p_two_sided(est, lo, hi)[2])
return min(ps), max(ps)
line = "-" * 78
# ---------------------------------------------------------------- 1. the PLR row
print(line)
print("1. Multivariable PLR, mortality model in the IRE subgroup (Abstract; Results; Table 5)")
print(" printed: OR 1.004, 95% CI 0.7-1.4, P = 0.006")
OR, LO, HI, P_REP = 1.004, 0.7, 1.4, 0.006
se, zz, p = p_two_sided(OR, LO, HI)
mid = np.sqrt(LO * HI)
print(f" CI contains 1: {LO < 1 < HI}")
print(f" SE(ln OR) from CI = {se:.4f}")
print(f" z = ln(1.004)/SE = {zz:.4f}")
print(f" two-sided P = {p:.4f} one-sided P = {p / 2:.4f}")
print(f" geometric midpoint of CI = {mid:.4f}")
_, zm, pm = p_two_sided(mid, LO, HI)
print(f" P using the CI midpoint = {pm:.4f} (z = {zm:.4f})")
# rounding: estimate held to values that round to 1.004
pmin_est = min(p_two_sided(e, lo, hi)[2]
for lo in interval("0.7") for hi in interval("1.4") for e in interval("1.004", 11))
pmin_all, pmax_all = p_range("1.004", "0.7", "1.4")
print(f" min P, estimate rounding to 1.004, bounds rounding to 0.7 / 1.4 = {pmin_est:.4f}")
print(f" min P, also allowing the estimate to be the CI midpoint = {pmin_all:.4f}")
for level in (0.90, 0.99, 0.999):
zl = norm.ppf(0.5 + level / 2)
print(f" if 0.7-1.4 were a {level * 100:g}% CI: P = {p_two_sided(OR, LO, HI, zl)[2]:.4f}")
# ---------------------------------------------------------------- 2. CI required by OR and P
print(line)
print("2. 95% CI that OR 1.004 with P = 0.006 would require")
z_req = norm.isf(P_REP / 2)
se_req = np.log(OR) / z_req
req_lo, req_hi = np.exp(np.log(OR) - Z * se_req), np.exp(np.log(OR) + Z * se_req)
print(f" z = {z_req:.4f}; SE = {se_req:.6f}; CI = {req_lo:.4f}-{req_hi:.4f}")
print(f" ratio of SE implied by printed CI to SE required: {se / se_req:.0f}")
los, his = [], []
for e in interval("1.004", 11):
for pp in np.linspace(0.0055, 0.0065, 11):
s = np.log(e) / norm.isf(pp / 2)
los.append(np.exp(np.log(e) - Z * s)); his.append(np.exp(np.log(e) + Z * s))
print(f" over OR 1.0035-1.0045 and P 0.0055-0.0065: lower {min(los):.4f}-{max(los):.4f}, "
f"upper {min(his):.4f}-{max(his):.4f}")
print(f" required CI at 1 decimal: {req_lo:.1f}-{req_hi:.1f}")
# ---------------------------------------------------------------- 3. scaling checks
print(line)
print("3. Rescaling checks")
print(f" 1.004^10 = {OR ** 10:.4f}; 1.004^100 = {OR ** 100:.4f}")
print(f" 1.004^100 inside 0.7-1.4: {LO < OR ** 100 < HI}")
print(f" 0.7-1.4 read as per 100 units -> per unit {LO ** 0.01:.5f}-{HI ** 0.01:.5f} (contains 1)")
print(f" per-100-unit OR implied by the required CI: {req_lo ** 100:.3f}-{req_hi ** 100:.3f}")
# ---------------------------------------------------------------- 4. PLR definition vs Table 3
print(line)
print("4. PLR definition (Methods) vs values (Table 3)")
print(" Methods: PLR = lymphocyte count / platelet count")
print(" Table 3 median PLR: 145.3 (IRE), 143.9 (no IRE)")
print(" Table 3 footer reference ranges: platelets 150-410, lymphocytes 1-3 (x10^3/uL)")
print(f" lymphocytes/platelets at the reference limits: {1 / 410:.4f}-{3 / 150:.4f}")
print(f" platelets/lymphocytes at the reference limits: {150 / 3:.0f}-{410 / 1:.0f}")
# ---------------------------------------------------------------- 5. control rows
print(line)
print("5. Same calculation on other printed rows. P from the CI midpoint; 'range' lets the CI bounds")
print(" vary over their rounding intervals (estimate = CI midpoint); 'wide' also lets the")
print(" estimate vary over its own rounding interval")
CONTROLS = [
# label, source, est, lo, hi, printed P (string), printed P value for plotting
("Mortality MVR: PLR", "Table 5", "1.004", "0.7", "1.4", "0.006", 0.006),
("Mortality UV: PLR", "Table 5", "1.001", "1.0001", "1.003", "0.03", 0.03),
("Mortality MVR: age", "Table 5", "1.1", "1.04", "1.2", "0.001", 0.001),
("Mortality UV: age", "Table 5", "1.04", "1.02", "1.07", "<0.001", 0.001),
("Mortality UV: HbA1c", "Table 5", "1.3", "1.01", "1.6", "0.03", 0.03),
("Mortality UV: hemoglobin", "Table 5", "0.8", "0.7", "1", "0.03", 0.03),
("IRE MVR: age", "Table 4", "1.02", "1.003", "1.03", "0.02", 0.02),
("IRE MVR: WBC", "Table 4", "0.94", "0.89", "0.99", "0.04", 0.04),
("IRE MVR: pembrolizumab", "Table 4", "2.6", "1.05", "6.3", "0.04", 0.04),
("IRE MVR: nivolumab", "Table 4", "2.6", "1.04", "6.6", "0.04", 0.04),
]
results = []
for lab, src, e, lo, hi, prs, prv in CONTROLS:
lo_f, hi_f = float(lo), float(hi)
_, zc, pc = p_two_sided(np.sqrt(lo_f * hi_f), lo_f, hi_f)
a, b = p_range(e, lo, hi, use_printed_est=False)
wa, wb = p_range(e, lo, hi)
results.append((lab, src, prs, prv, pc, a, b))
print(f" {lab:24s} {src}: {e} ({lo}-{hi}) printed P {prs:>6s} | "
f"recomputed {pc:.4f}; range {a:.4f}-{b:.4f}; wide {wa:.4f}-{wb:.4f}")
# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": FG,
"axes.edgecolor": FG2, "axes.labelcolor": FG, "xtick.color": FG2, "ytick.color": FG2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def style(ax):
ax.grid(axis="x", which="both", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
# Figure 1: printed interval vs required interval, one log axis
fig, ax = plt.subplots(figsize=(7.2, 3.7))
style(ax)
ax.set_xscale("log")
ax.axvline(1, color=FG2, lw=0.9, zorder=1)
ax.text(1, 2.62, "OR = 1 (no association)", ha="center", va="bottom", color=FG2, fontsize=10)
y_p, y_r = 1.75, 0.75
ax.plot([LO, HI], [y_p, y_p], color=ORANGE, lw=2, solid_capstyle="butt", zorder=3)
for x in (LO, HI):
ax.plot([x, x], [y_p - 0.08, y_p + 0.08], color=ORANGE, lw=2, zorder=3)
ax.plot(OR, y_p, "o", ms=9, color=ORANGE, zorder=4)
ax.plot(mid, y_p, "D", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=5)
ax.text(LO, y_p + 0.17, "Printed: OR 1.004, 95% CI 0.7–1.4, stated P = 0.006",
ha="left", va="bottom", color=ORANGE, fontsize=10.5, fontweight="bold")
ax.text(HI, y_p - 0.17, f"Recomputed from this CI: centre {mid:.3f}; P = {p:.2f} for OR 1.004",
ha="right", va="top", color=BLUE, fontsize=10)
ax.plot([req_lo, req_hi], [y_r, y_r], color=BLUE, lw=2, zorder=3)
ax.plot(OR, y_r, "D", ms=8, color=BLUE, zorder=4)
ax.text(LO, y_r + 0.17,
f"Recomputed: 95% CI needed for OR 1.004 with P = 0.006: {req_lo:.4f}–{req_hi:.4f}",
ha="left", va="bottom", color=BLUE, fontsize=10.5, fontweight="bold")
ax.text(1.03, y_r, "(narrower than the marker at this scale)",
ha="left", va="center", color=FG2, fontsize=10)
ax.set_xlim(0.62, 1.6)
ax.set_ylim(0.25, 2.9)
ticks = [0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.4]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks([])
ax.set_xlabel("Odds ratio for mortality per unit PLR (log scale)")
handles = [Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Reported in the paper"),
Line2D([], [], color=BLUE, marker="D", ms=8, lw=2, label="Recomputed from the paper's numbers")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.5, -0.24), ncol=2,
frameon=False, fontsize=10)
fig.tight_layout()
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), **kw)
plt.close(fig)
# Figure 2: printed vs recomputed P values across rows, one log axis
fig, ax = plt.subplots(figsize=(7.2, 5.0))
style(ax)
ax.set_xscale("log")
n = len(results)
ys = np.arange(n)[::-1]
ax.axvline(0.05, color=FG2, lw=0.9, ls=(0, (4, 3)), zorder=1)
ax.text(0.05, n - 0.35, "P = 0.05", ha="center", va="bottom", color=FG2, fontsize=10)
for y, (lab, src, prs, prv, pc, a, b) in zip(ys, results):
ax.plot([a, b], [y, y], color=BLUE, alpha=0.35, lw=7, solid_capstyle="butt", zorder=2)
ax.plot(pc, y, "D", ms=8, color=BLUE, zorder=4)
ax.plot(prv, y, "o", ms=9, mfc=ORANGE, mec=ORANGE, zorder=3)
if prs.startswith("<"):
ax.annotate("", xy=(prv * 0.55, y), xytext=(prv, y),
arrowprops=dict(arrowstyle="->", color=ORANGE, lw=2))
ax.set_yticks(ys)
ax.set_yticklabels([f"{r[0]} ({r[1]})" for r in results])
for t, r in zip(ax.get_yticklabels(), results):
if r[0] == "Mortality MVR: PLR":
t.set_fontweight("bold"); t.set_color(FG)
ax.set_xlim(1e-4, 1.5)
ax.set_ylim(-0.7, n - 0.1)
ax.set_xticks([1e-4, 1e-3, 1e-2, 0.05, 1])
ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.05", "1"])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_xlabel("Two-sided P value (log scale)")
handles = [Line2D([], [], color=ORANGE, marker="o", ms=8, lw=0, label="P printed in the paper"),
Line2D([], [], color=BLUE, marker="D", ms=8, lw=0, label="P recomputed from the printed 95% CI"),
Line2D([], [], color=BLUE, alpha=0.35, lw=7, label="Range over rounding of the printed CI bounds")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.3, -0.14), ncol=1,
frameon=False, fontsize=10)
fig.tight_layout()
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), **kw)
plt.close(fig)
print(line)
print("Figures written to", HERE)
Abstract reverses the direction of both age effects on cortical thickness reported in Table 3
Brain (2023) · doi:10.1093/brain/awad100 · PMC10473572
Summary
In its brain-resilience results, the abstract says that "older age was associated with higher baseline cortical thickness and slower rate of cortical thinning independent of tau". The paper's own Table 3 reports a negative age coefficient on AD-signature cortical thickness (standardized β = −0.49, 95% CI −0.613 to −0.366, P < 0.001) and a negative Time × Age coefficient (standardized β = −0.051, 95% CI −0.083 to −0.020, P = 0.002). Both mean that older age goes with thinner cortex and faster thinning. The Results, both relevant Discussion passages and Supplementary Table 5 all give that direction, and the printed intervals reproduce the printed t values, so the signs in the table are not typesetting slips. Only the abstract sentence states the reverse. Because the abstract is the part most often read and indexed, readers who rely on it get the direction of a secondary finding backwards; the estimates themselves and the paper's main conclusions are unaffected.
What the paper reports
Abstract, results paragraph:
Higher education also modified the negative impact of tau-PET on cortical thinning, while older age was associated with higher baseline cortical thickness and slower rate of cortical thinning independent of tau.
For contrast, the same abstract paragraph gives the cognitive-resilience result with the favourable direction attached to younger age:
Younger age, higher education and greater cortical thickness were associated with better cognitive performance at baseline.
Results, "Brain resilience", first paragraph:
In models that estimated main effects (i.e. after removing the three-way interaction term), older age was related to thinner cross-sectional AD-signature cortex [stβ (95%CI) = -0.49 (-0.613,-0.366), P < 0.001] and to accelerated cortical thinning [stβ (95%CI) = -0.051 (-0.083,-0.02), P < 0.01] independent of temporal meta-ROI tau.
Results of analyses with global tau burden were consistent with these findings (Supplementary Table 5).
Table 3 ("Results of linear mixed-effect models investigating determinants of brain resilience to tau burden in the temporal meta-ROI"), Age row. Columns: Time × Tau × Variable β (unstandardized | standardized | t | P), Time × Variable β (same four), Variable β (same four):
Agea | 0.001 (-0.001,0.002) | 0.009 (-0.021,0.038) | 0.566 | 0.572 | -0.001 (-0.002,0) | -0.051 (-0.083,-0.02) | -3.182 | 0.002 | -0.012 (-0.015,-0.009) | -0.49 (-0.613,-0.366) | -7.774 | <0.001
Table 3 footer, on coding of the outcome and predictors:
Time (years), temporal meta-ROI tau (SUVR), age at baseline (years), educational level (years of education) and ICV (dm3) were centred and used in continuous form
The ‘Time × Variable β’ and ‘Variable β’ coefficients are main effects from models where the non-significant interaction term was removed.
Methods, "Determinants":
For the current study, age (in years) was defined as the age at the time of the tau-PET scan
Discussion, first paragraph:
While there was no evidence for age as a moderator in brain resilience models, older age was associated with lower cortical thickness at the time of the tau-PET scan, and with faster cortical thinning over time.
Discussion, "Age and cortical thickness":
we found that younger age was associated with both higher baseline cortical thickness and slower rate of cortical thinning at similar levels of tau burden.
Conclusion:
we found that age, education, ICV and cortical thickness play a role in cognitive resilience, while age and education contribute to brain resilience.
What the paper's own numbers show
Method. The outcome of the Table 3 models is AD-signature cortical thickness in mm (a higher value is a thicker cortex), and age is continuous, centred, in years. A negative Age coefficient therefore means a thinner cortex at the tau-PET scan for older participants, and a negative Time × Age coefficient means a more negative thickness slope, i.e. faster thinning, for older participants. The abstract's sentence requires both coefficients to be positive. To check that the printed signs belong to the printed statistics, I recomputed each standard error from its 95% CI (SE = (upper − lower)/(2 × 1.96)), the t value and P value that follow, and conversely the CI implied by the printed β and t. I also checked that the unstandardized and standardized rows agree, and repeated the sign check on Supplementary Table 5 (global tau, global thickness). Raw data are not available, so the models themselves cannot be refitted. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Age → baseline thickness, standardized β | Table 3; Results | −0.49 (−0.613, −0.366), t = −7.774, P < 0.001 | CI midpoint −0.4895; SE 0.0630; t −7.78; P ≈ 7.5 × 10⁻¹⁵; negative, CI excludes 0 |
| Time × Age → rate of thinning, standardized β | Table 3; Results | −0.051 (−0.083, −0.02), t = −3.182, P = 0.002 (Results: P < 0.01) | CI midpoint −0.0515; SE 0.0161; t −3.17; P ≈ 0.0015; negative, CI excludes 0 |
| CI rebuilt from printed β and t | Table 3 | as above | Age −0.614 to −0.366; Time × Age −0.082 to −0.020 (match the printed CIs) |
| Age, unstandardized (mm per year of age) | Table 3 | −0.012 (−0.015, −0.009) | t −7.84 from the CI; negative; 10 years older ≈ 0.12 mm thinner at the tau-PET scan |
| Time × Age, unstandardized | Table 3 | −0.001 (−0.002, 0) | negative; t from the CI is only −1.96 because the values are rounded to 3 decimals |
| Agreement of standardized and unstandardized rows | Table 3 | — | −0.49 / −0.012 = 40.8; a printed −0.001 (−0.0015 to −0.0005) × 40.8 = −0.061 to −0.020, which contains the printed −0.051 |
| Expected thickness gap, 1 SD older vs average age, average tau | Table 3 model without the three-way term | — | −0.49 SD at the tau-PET scan; −0.567 SD at the median MRI follow-up (18 months); −0.758 SD at the longest (63 months): the gap widens |
| Age as moderator of the tau effect (Time × Tau × Age) | Table 3 | 0.009 (−0.021, 0.038), P = 0.572 | t 0.60 from the CI; not significant, consistent with "independent of tau" |
| Age → baseline thickness, global analysis | Supplementary Table 5 | −0.374 (−0.522 – −0.227), t = −5.001, P < 0.001 | t −4.97 from the CI; negative |
| Time × Age → rate of thinning, global analysis | Supplementary Table 5 | −0.044 (−0.084 – −0.003), t = −2.112, P = 0.036 | t −2.13 from the CI; negative |
| Direction stated in the abstract | Abstract | older → higher baseline thickness, slower thinning | requires Age β > 0 and Time × Age β > 0; both printed coefficients are < 0 |
Figure
Figure 1. The left panel holds the abstract's sentence on age and brain resilience and the direction it states. The right panel holds the Results sentence and the Table 3 Age row (standardized β with 95% CI and t), all as printed (orange). The connector marks the conflict: the two panels give opposite directions for both the baseline (cross-sectional) effect and the longitudinal (Time × Age) effect. The blue lines give the t and P values recomputed from the printed confidence intervals; they match the printed t values, so the negative signs are internally consistent within the table.
Figure 2. Each panel shows one Table 3 coefficient on its own symmetric axis, with zero as a thin reference line; the shaded half is the direction the abstract states (older age with thicker cortex, or with slower thinning). Orange circles and bars are the printed estimates and 95% CIs; blue squares and bars are the intervals rebuilt from the printed β and t (SE = β/t). Both intervals lie entirely on the negative side, away from the direction stated in the abstract.
Why it matters
- Direction, not magnitude. The error reverses the stated direction of two associations. The size and significance are printed correctly in the body: about 0.49 SD thinner AD-signature cortex per SD of age (about 0.012 mm per year of age), and a thickness slope about 0.051 SD per year more negative per SD of age. Both intervals exclude zero.
- It changes the reading of a resilience finding. As written, the abstract says older age is associated with preserved brain structure, which a reader would take as older age conferring brain resilience. The paper's actual finding is the conventional one, which its Discussion states: younger age goes with thicker cortex and slower thinning at similar tau burden, as an additive effect rather than a moderator.
- Where it sits. The abstract is the part of the paper that most readers, databases and evidence syntheses use. Someone who takes the age-resilience association from the abstract alone gets the sign wrong, and nothing in the abstract signals the conflict. The Conclusion's "age and education contribute to brain resilience" gives no direction, so read with the abstract it carries the reversed direction too.
- What it does not affect. The headline result (education as the most robust determinant), the education moderation of tau-related thinning, the cognitive-resilience results and the age results in the body are not affected. Age did not moderate the tau–thinning relationship (P = 0.572), and the abstract's "independent of tau" is consistent with that.
Explanations considered
- The outcome is an atrophy score, so a negative coefficient means less atrophy. Ruled out. The Table 3 header names the outcome "AD-signature cortical thickness", the Methods compute cortical thickness with FreeSurfer as "the distance from the grey matter–white matter boundary to the corresponding pial surface", the Table 2 footer gives AD-signature cortical thickness in mm, and the Results describe tau uptake as "significantly negatively associated with cortical thinning". A negative coefficient is a thinner cortex.
- Age is reverse-coded or entered as a "younger" indicator. Ruled out. The Methods define age as age in years at the tau-PET scan, and the Table 3 footer says it was centred and used in continuous form.
- The abstract describes a different model (full model, global tau, sensitivity analysis). Not supported. The Table 3 footer and the Results say the age main effects come from the model without the non-significant three-way term, which matches the abstract's "independent of tau". The global-tau analysis (Supplementary Table 5) gives negative age coefficients as well (−0.374 and −0.044). No analysis in the paper or the supplement reports a positive age effect on thickness or a slower thinning rate with age.
- The table signs are typos and the abstract is right. Ruled out. The printed CIs reproduce the printed t values (−7.78 vs −7.774; −3.17 vs −3.182), the unstandardized and standardized rows agree in sign and scale, and the Results and two Discussion passages describe the same negative direction in words.
- The sentence means "younger age" was associated with thicker cortex and slower thinning. That is what the body reports, and it is also how the Discussion phrases it ("younger age was associated with both higher baseline cortical thickness and slower rate of cortical thinning"). The abstract, however, says "older age". As printed, the abstract sentence states the opposite of the paper's results.
- A conversion or archiving artifact. Ruled out. The publisher's version of record, the PMC page, the Europe PMC full-text XML and the PMC archive used here all carry the same abstract wording.
- Already corrected. No correction, erratum or other notice was found (see Status).
Status
As of 2026-10-23:
- Version of record. The DOI resolves to the publisher's version of record (Brain 146(9):3719). Its abstract carries the same sentence ("older age was associated with higher baseline cortical thickness and slower rate of cortical thinning independent of tau"), and its Results and Table 3 carry the same negative coefficients quoted above. The PMC page, the Europe PMC full-text XML and the PMC archive agree word for word.
- Supplementary material. The single supplementary file (awad100_supplementary_data.pdf, retrieved from Europe PMC) was read. Supplementary Table 5 (global tau, global cortical thickness) reports negative age coefficients for both baseline thickness and thinning rate, and nothing in the supplement supports the abstract's direction.
- Corrections and notices. Crossref (no update-to or updated-by relations; Crossref now includes Retraction Watch data), PMC and Europe PMC notice records, and PubMed CommentsCorrections list no correction, erratum, expression of concern or retraction. The publisher's article page shows none; its "Corrected and typeset" date is a production date. None of the 51 citing records in Europe PMC is a correction, comment or reply.
- PubPeer. The PubPeer API returned 403 to automated requests, so a direct search could not be completed. Web searches for a correction, erratum or PubPeer thread on this article found none.
- Sources. The article as archived in PMC (PMC10473572); the publisher's version of record; the supplementary data file; Crossref, PMC, Europe PMC and PubMed records.
Suggested correction
In the abstract, change the sentence so that it states the direction reported in Table 3, the Results and the Discussion, for example:
Higher education also modified the negative impact of tau-PET on cortical thinning, while older age was associated with lower baseline cortical thickness and faster rate of cortical thinning independent of tau.
Equivalently, "younger age was associated with higher baseline cortical thickness and slower rate of cortical thinning independent of tau", which matches the wording of the Discussion. No change to the estimates, tables or conclusions appears to be needed.
Recomputation script (python recompute.py.txt)
"""Recompute the age effects on brain resilience in PMC10473572
(Brain 2023, doi:10.1093/brain/awad100).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file, plus five candidate figure
designs (cand1..cand5, .svg and .png) in the candidates/ subfolder.
All inputs are copied from the article as archived in PubMed Central:
Table 3, Age row (brain resilience, outcome = AD-signature cortical thickness, mm):
Time x Tau x Age, Time x Age and Age coefficients, unstandardized and standardized,
with 95% CIs, t and P.
Results > Brain resilience (the same standardized values, quoted in the text).
Methods / Results: median MRI follow-up 18 months (range 9-63 months).
Supplementary Table 5, Age row (global tau, global cortical thickness as outcome).
No raw data are available, so the model itself cannot be refitted. The checks below test
(i) that the printed CIs, t and P belong together, so the printed signs are not typos, and
(ii) what direction the printed coefficients imply.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from scipy.stats import norm # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
Z = norm.ppf(0.975)
# ---------------------------------------------------------------- inputs (from the paper)
# Table 3, Age row: (beta, lo, hi, t, P as printed)
T3 = {
"age_std": (-0.49, -0.613, -0.366, -7.774, "<0.001"),
"timexage_std": (-0.051, -0.083, -0.020, -3.182, "0.002"),
"age_unstd": (-0.012, -0.015, -0.009, -7.774, "<0.001"),
"timexage_unstd": (-0.001, -0.002, 0.0, -3.182, "0.002"),
"3way_std": (0.009, -0.021, 0.038, 0.566, "0.572"),
}
# Supplementary Table 5, Age row (global tau, global cortical thickness)
S5 = {
"age_std": (-0.374, -0.522, -0.227, -5.001, "<0.001"),
"timexage_std": (-0.044, -0.084, -0.003, -2.112, "0.036"),
}
FU_MEDIAN_Y, FU_MAX_Y = 18 / 12, 63 / 12 # brain resilience subsample, MRI follow-up
LABELS = {
"age_std": "Age, standardized (baseline thickness)",
"timexage_std": "Time x Age, standardized (rate of thinning)",
"age_unstd": "Age, unstandardized (mm per year of age)",
"timexage_unstd": "Time x Age, unstandardized (mm/yr per year of age)",
"3way_std": "Time x Tau x Age, standardized (moderation)",
}
def from_ci(b, lo, hi):
se = (hi - lo) / (2 * Z)
t = b / se
return (lo + hi) / 2, se, t, 2 * norm.sf(abs(t))
def main():
R = {}
print("== 1. Table 3, Age row: SE, t and P recomputed from the printed 95% CIs")
print(" (normal approximation; SE = (upper - lower) / (2 x 1.96))")
for k, (b, lo, hi, t, p) in T3.items():
mid, se, tr, pr = from_ci(b, lo, hi)
R[k] = (mid, se, tr, pr)
print(f" {LABELS[k]}")
print(f" printed beta {b} ({lo}, {hi}), t = {t}, P = {p}")
print(f" CI midpoint {mid:.4f}; SE {se:.4f}; t from CI {tr:.2f}; P from CI {pr:.2g}; "
f"sign {'negative' if b < 0 else 'positive'}; CI excludes 0: {hi < 0 or lo > 0}")
print("\n== 2. 95% CI rebuilt from the printed beta and t (SE = beta / t)")
CI_T = {}
for k in ("age_std", "timexage_std"):
b, lo, hi, t, p = T3[k]
se = b / t
CI_T[k] = (b - Z * se, b + Z * se)
print(f" {LABELS[k]}: SE = {se:.4f}; CI {b - Z * se:.3f} to {b + Z * se:.3f} "
f"(printed {lo} to {hi})")
print("\n== 3. Do the unstandardized and standardized rows agree in sign and scale?")
ratio = T3["age_std"][0] / T3["age_unstd"][0]
print(f" Age: standardized / unstandardized = {T3['age_std'][0]} / {T3['age_unstd'][0]} "
f"= {ratio:.1f} (= SD of age / SD of thickness, in years per mm)")
lo_u, hi_u = -0.0015, -0.0005 # values that round to -0.001
print(f" Time x Age unstandardized printed as -0.001, i.e. between {lo_u} and {hi_u};")
print(f" times {ratio:.1f} gives a standardized value between {lo_u * ratio:.3f} and "
f"{hi_u * ratio:.3f}; printed standardized value {T3['timexage_std'][0]} lies inside")
print(" (time is in years in both, so the same factor applies)")
print(f" Unstandardized Age: -0.012 mm per year of age -> 10 years older = "
f"{10 * T3['age_unstd'][0]:.2f} mm thinner AD-signature cortex at the tau-PET scan")
print("\n== 4. What the Table 3 coefficients imply (model without the 3-way term, average tau)")
b0, b1 = T3["age_std"][0], T3["timexage_std"][0]
for yrs, lab in ((0, "baseline (tau-PET)"), (FU_MEDIAN_Y, "median MRI follow-up, 18 months"),
(FU_MAX_Y, "longest MRI follow-up, 63 months")):
print(f" 1 SD older, at {lab}: expected thickness difference = {b0} + ({b1} x {yrs:.2f}) "
f"= {b0 + b1 * yrs:.3f} SD")
print(" Both terms are negative: the older participant starts thinner and the gap widens,")
print(" i.e. older age -> lower baseline thickness and faster thinning.")
print(" The abstract ('older age ... higher baseline cortical thickness and slower rate of")
print(" cortical thinning') would need Age beta > 0 and Time x Age beta > 0.")
b, lo, hi, t, p = T3["3way_std"]
print(f" Age as moderator of the tau effect: {b} ({lo}, {hi}), P = {p}: not significant,")
print(" consistent with the abstract's 'independent of tau'.")
print("\n== 5. Supplementary Table 5 (global tau, global cortical thickness), Age row")
for k, (b, lo, hi, t, p) in S5.items():
mid, se, tr, pr = from_ci(b, lo, hi)
print(f" {LABELS[k]}: printed {b} ({lo}, {hi}), t = {t}, P = {p}; "
f"t from CI {tr:.2f}; P from CI {pr:.2g}; sign {'negative' if b < 0 else 'positive'}")
os.makedirs(CAND, exist_ok=True)
designs = [cand1_text_panels, cand2_intervals, cand3_grid, cand4_trajectory, cand5_combined]
for i, f in enumerate(designs, 1):
fig = f(R, CI_T)
save(fig, os.path.join(CAND, f"cand{i}"))
plt.close(fig)
# chosen designs for the write-up
save(cand1_text_panels(R, CI_T), os.path.join(HERE, "fig1"))
save(cand2_intervals(R, CI_T), os.path.join(HERE, "fig2"))
print("\nFigures written: fig1, fig2 and candidates/cand1..cand5 (.svg, .png)")
# ---------------------------------------------------------------- figures
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
PANEL = "#f3f2ef"
plt.rcParams.update({
"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT, "xtick.color": TXT,
"ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "font.family": "DejaVu Sans",
})
W = 7.2
def save(fig, stem):
fig.savefig(stem + ".svg")
fig.savefig(stem + ".png", dpi=200)
def box(ax, x, y, w, h, edge=SEC, face=PANEL, lw=1.0):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc=face, ec=edge, lw=lw, transform=ax.transAxes))
def lines(ax, x, y, items, dy=0.058):
"""items: list of (text, color, weight, size)."""
for txt, col, wt, sz in items:
ax.text(x, y, txt, color=col, fontweight=wt, fontsize=sz, va="top", ha="left",
transform=ax.transAxes)
y -= dy * sz / 10.5
return y
def fmt(v, nd=3):
return f"{v:.{nd}f}".replace("-", "−")
# 1 ---- two text panels joined by an annotated connector
def cand1_text_panels(R, CI_T):
fig = plt.figure(figsize=(W, 4.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1)
ax.text(0.03, 0.965, "The abstract and the body give opposite directions for both age effects",
fontsize=12, fontweight="bold", va="top")
ax.text(0.03, 0.91, "Brain resilience models; outcome = AD-signature cortical thickness (mm)",
fontsize=10, color=SEC, va="top")
# left panel
box(ax, 0.03, 0.255, 0.385, 0.605)
lines(ax, 0.05, 0.835, [
("ABSTRACT", SEC, "bold", 10),
("\u201c\u2026older age was associated", TXT, "normal", 10.5),
("with higher baseline cortical", TXT, "bold", 10.5),
("thickness and slower rate of", TXT, "bold", 10.5),
("cortical thinning independent", TXT, "bold", 10.5),
("of tau.\u201d", TXT, "normal", 10.5),
], dy=0.052)
lines(ax, 0.05, 0.43, [
("Direction stated:", SEC, "normal", 10),
("older \u2192 thicker at baseline", ORANGE, "bold", 10.5),
("older \u2192 slower thinning", ORANGE, "bold", 10.5),
], dy=0.052)
# right panel
box(ax, 0.55, 0.255, 0.425, 0.605)
lines(ax, 0.57, 0.835, [
("RESULTS AND TABLE 3", SEC, "bold", 10),
("\u201colder age was related to", TXT, "normal", 10.5),
("thinner cross-sectional", TXT, "bold", 10.5),
("AD-signature cortex [\u2026] and", TXT, "normal", 10.5),
("to accelerated cortical", TXT, "bold", 10.5),
("thinning\u201d", TXT, "bold", 10.5),
], dy=0.052)
lines(ax, 0.57, 0.50, [
("Table 3, Age row, std. \u03b2 (95% CI):", SEC, "normal", 10),
("Age \u22120.49 (\u22120.613, \u22120.366)", ORANGE, "bold", 10),
("Time\u00d7Age \u22120.051 (\u22120.083, \u22120.02)", ORANGE, "bold", 10),
("t = \u22127.774 and \u22123.182", ORANGE, "bold", 10),
("older \u2192 thinner, faster thinning", TXT, "bold", 10.5),
], dy=0.052)
# connector
ar = FancyArrowPatch((0.42, 0.56), (0.545, 0.56), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=TXT, transform=ax.transAxes)
ax.add_patch(ar)
ax.text(0.4825, 0.605, "opposite", ha="center", fontsize=10, fontweight="bold")
ax.text(0.4825, 0.535, "sign for\nboth", ha="center", va="top", fontsize=10)
# recomputed strip
_, se0, t0, p0 = R["age_std"]; _, se1, t1, p1 = R["timexage_std"]
ax.text(0.03, 0.215, "Recomputed from the printed 95% CIs:", fontsize=10, color=SEC, va="top")
ax.text(0.03, 0.165, f"Age t \u2248 {fmt(t0, 2)} (P \u2248 {p0:.0e}); Time\u00d7Age t \u2248 "
f"{fmt(t1, 2)} (P \u2248 {p1:.4f})", fontsize=10, color=BLUE, fontweight="bold", va="top")
ax.text(0.03, 0.115, "Both intervals exclude 0, and the t values match the printed ones,",
fontsize=10, color=BLUE, va="top")
ax.text(0.03, 0.078, "so the negative signs in Table 3 are not typesetting slips.",
fontsize=10, color=BLUE, va="top")
ax.text(0.97, 0.008, "orange = printed in the paper blue = recomputed from its numbers",
fontsize=10, color=SEC, ha="right", va="bottom")
return fig
# 2 ---- dot-and-interval plots, one panel per coefficient, zero line, abstract direction shaded
def cand2_intervals(R, CI_T):
fig, axs = plt.subplots(2, 1, figsize=(W, 4.2), gridspec_kw=dict(hspace=0.75))
spec = [("age_std", "Age → baseline thickness (standardized β)", 0.7,
"older → thinner at baseline", "abstract: older → thicker at baseline"),
("timexage_std", "Time × Age → rate of thinning (standardized β per year)", 0.1,
"older → faster thinning", "abstract: older → slower thinning")]
for ax, (k, title, lim, lneg, lpos) in zip(axs, spec):
b, lo, hi, t, p = T3[k]
ax.axvspan(0, lim, color=ORANGE, alpha=0.08, lw=0)
ax.axvline(0, color=SEC, lw=0.8)
ax.errorbar([b], [1], xerr=[[b - lo], [hi - b]], fmt="o", color=ORANGE, ms=8, lw=2,
capsize=0)
cl, ch = CI_T[k]
ax.errorbar([b], [0], xerr=[[b - cl], [ch - b]], fmt="s", color=BLUE, ms=8, lw=2,
capsize=0)
ax.set_yticks([1, 0])
ax.set_yticklabels(["Table 3, printed", "rebuilt from β and t"], fontsize=10)
ax.set_ylim(-0.7, 2.3)
ax.set_xlim(-lim, lim)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_title(title, loc="left", fontsize=10.5, fontweight="bold", pad=6)
ax.text(lim * 0.97, 2.15, lpos, ha="right", va="top", fontsize=10, color=SEC)
ax.text(-lim * 0.97, 2.15, lneg, ha="left", va="top", fontsize=10, color=SEC)
ax.annotate(f"{fmt(b, 3 if abs(b) < 0.1 else 2)} ({fmt(lo)}, {fmt(hi)})", (hi, 1),
xytext=(8, 0), textcoords="offset points", va="center", fontsize=10,
color=ORANGE, fontweight="bold")
ax.annotate(f"({fmt(cl)}, {fmt(ch)})", (ch, 0), xytext=(8, 0),
textcoords="offset points", va="center", fontsize=10, color=BLUE,
fontweight="bold")
fig.subplots_adjust(left=0.22, right=0.97, top=0.9, bottom=0.08)
return fig
# 3 ---- consistency grid: every place the paper states the direction
def cand3_grid(R, CI_T):
rows = [
("Abstract", "thicker", "slower", True),
("Results › Brain resilience", "thinner", "faster (accelerated)", False),
("Table 3, Age row", "−0.49 (thinner)", "−0.051 (faster)", False),
("Discussion, 1st paragraph", "lower", "faster", False),
("Discussion › Age and cortical thickness*", "lower", "faster", False),
("Supplementary Table 5 (global)", "−0.374 (thinner)", "−0.044 (faster)", False),
]
fig = plt.figure(figsize=(W, 3.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off"); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
ax.text(0.03, 0.95, "Where the paper states the direction of the age effects on thickness",
fontsize=11.5, fontweight="bold", va="top")
x0, x1, x2 = 0.03, 0.46, 0.73
yh = 0.83
ax.text(x1, yh, "Older age → baseline\nthickness", fontsize=10, color=SEC, va="center",
fontweight="bold")
ax.text(x2, yh, "Older age → rate\nof thinning", fontsize=10, color=SEC, va="center",
fontweight="bold")
dy = 0.095
for i, (src, a, b, bad) in enumerate(rows):
y = yh - 0.12 - i * dy
if bad:
ax.add_patch(FancyBboxPatch((0.02, y - dy / 2 + 0.008), 0.96, dy - 0.016,
boxstyle="round,pad=0,rounding_size=0.01",
fc="#fbe6dd", ec=ORANGE, lw=1.5, transform=ax.transAxes))
else:
ax.plot([0.02, 0.98], [y - dy / 2, y - dy / 2], color=GRID, lw=0.8,
transform=ax.transAxes)
col = ORANGE if bad else TXT
ax.text(x0 + 0.01, y, src, fontsize=10.5, va="center", fontweight="bold" if bad else "normal")
ax.text(x1, y, a, fontsize=10.5, va="center", color=col, fontweight="bold")
ax.text(x2, y, b, fontsize=10.5, va="center", color=col, fontweight="bold")
ax.text(0.03, 0.025, "Highlighted: the only statement with the opposite direction.\n"
"*Phrased as 'younger age → higher baseline thickness, slower thinning'.",
fontsize=10, color=SEC, va="bottom")
return fig
# 4 ---- implied expected difference over follow-up, from the Table 3 coefficients
def cand4_trajectory(R, CI_T):
import numpy as np
b0, b1 = T3["age_std"][0], T3["timexage_std"][0]
t = np.linspace(0, FU_MAX_Y, 50)
fig, ax = plt.subplots(figsize=(W, 3.6))
ax.axhspan(0, 0.9, color=ORANGE, alpha=0.08, lw=0)
ax.axhline(0, color=SEC, lw=0.8)
ax.plot(t, b0 + b1 * t, color=BLUE, lw=2)
ax.plot([0], [b0], "o", color=ORANGE, ms=8)
ax.annotate(f"Table 3 Age β = {fmt(b0, 2)}\n(95% CI {fmt(T3['age_std'][1])}, "
f"{fmt(T3['age_std'][2])})", (0, b0), xytext=(8, -20), textcoords="offset points",
va="top", fontsize=10, color=ORANGE, fontweight="bold")
for yrs, lab in ((FU_MEDIAN_Y, "median follow-up"), (FU_MAX_Y, "longest follow-up")):
v = b0 + b1 * yrs
ax.plot([yrs], [v], "s", color=BLUE, ms=8)
ax.annotate(f"{lab}: {fmt(v, 2)}", (yrs, v), xytext=(0, 10), textcoords="offset points",
ha="center" if yrs < 4 else "right", fontsize=10, color=BLUE,
fontweight="bold")
ax.text(0.1, 0.8, "region the abstract implies (older participant thicker, gap closing or "
"reversing)", fontsize=10, color=SEC, va="top")
ax.text(FU_MAX_Y * 0.98, -0.2, f"slope = Time×Age β = {fmt(b1)} SD/year\n(gap widens: "
"faster thinning)", fontsize=10, color=BLUE, ha="right", va="top")
ax.set_xlim(-0.1, FU_MAX_Y + 0.1); ax.set_ylim(-0.9, 0.9)
ax.set_xlabel("Years from tau-PET")
ax.set_ylabel("Thickness difference, 1 SD older\nvs average age (SD units)")
ax.grid(color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.set_title("Expected AD-signature thickness gap implied by Table 3 (average tau)",
loc="left", fontsize=11, fontweight="bold")
fig.tight_layout()
return fig
# 5 ---- compact: abstract claim on the left, the two printed intervals on the right
def cand5_combined(R, CI_T):
fig = plt.figure(figsize=(W, 3.4))
axt = fig.add_axes([0, 0, 0.36, 1]); axt.axis("off")
box(axt, 0.06, 0.34, 0.9, 0.5)
lines(axt, 0.11, 0.80, [
("ABSTRACT", SEC, "bold", 10),
("“older age was associated", TXT, "normal", 10.5),
("with higher baseline", TXT, "bold", 10.5),
("cortical thickness and", TXT, "bold", 10.5),
("slower rate of cortical", TXT, "bold", 10.5),
("thinning”", TXT, "bold", 10.5),
("implies both β > 0", ORANGE, "bold", 10.5),
])
axt.text(0.06, 0.95, "Claim vs Table 3", fontsize=11.5, fontweight="bold", va="top")
ax = fig.add_axes([0.47, 0.2, 0.5, 0.62])
ks = [("age_std", "Age\n(baseline)"), ("timexage_std", "Time×Age\n(slope)")]
ax.axvspan(0, 0.7, color=ORANGE, alpha=0.08, lw=0)
ax.axvline(0, color=SEC, lw=0.8)
for y, (k, lab) in zip((1, 0), ks):
b, lo, hi, t, p = T3[k]
ax.errorbar([b], [y], xerr=[[b - lo], [hi - b]], fmt="o", color=ORANGE, ms=8, lw=2)
ax.annotate(f"{fmt(b, 3 if abs(b) < 0.1 else 2)}", (b, y), xytext=(0, 9),
textcoords="offset points", ha="center", fontsize=10, color=ORANGE,
fontweight="bold")
ax.set_yticks([1, 0]); ax.set_yticklabels([k[1] for k in ks], fontsize=10)
ax.set_ylim(-0.6, 1.6); ax.set_xlim(-0.7, 0.7)
ax.set_xlabel("Standardized β, Table 3 (95% CI)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.text(0.68, 1.5, "abstract's\ndirection", ha="right", va="top", fontsize=10, color=SEC)
ax.set_title("Table 3 (orange, printed): both < 0", loc="left", fontsize=10.5,
fontweight="bold")
_, _, t0, _ = R["age_std"]; _, _, t1, _ = R["timexage_std"]
fig.text(0.47, 0.03, f"Recomputed from CIs: t ≈ {fmt(t0, 2)} and {fmt(t1, 2)}",
fontsize=10, color=BLUE, fontweight="bold")
return fig
if __name__ == "__main__":
main()
A Discussion passage reverses the lesion–donation directions stated in the Abstract and Results
Brain (2018) · doi:10.1093/brain/awy064 · PMC7341482
Summary
The paper maps penetrating brain injuries in 94 veterans against how much money each gave away on an altruistic decision task, and its Abstract, Results, Discussion summary and Figure 3 legend agree that lesions of the dorsomedial parietal cortex (precuneus, paracentral lobule) increased donations while lesions of the right posterior middle temporal gyrus (MTG) and superior temporal sulcus decreased them. The Discussion subsection that interprets this result says the opposite for both regions ("Damage to the left medial parietal lobe decreased donations, whereas damage to the right middle temporal gyrus increased donations"), and calls left-sided a parietal effect that the Abstract calls "bilateral" and Table 3 lists for both hemispheres (12 right, 11 left subjects). The paragraph's closing interpretation follows the reversed direction, so the paper's functional reading of one of its two main findings conflicts with that finding as stated, and the figures' colour bars do not independently settle which statement is correct. No statistic depends on the wording, but no correction has been published.
What the paper reports
The direction given in the Abstract (Abstract, main finding (ii)):
altruistic donations were increased by bilateral lesions of the dorsomedial parietal cortex, whereas lesions of the right posterior superior temporal sulcus and middle temporal gyri decreased donations
The same direction in the Results (Results > Donation, paragraph 1, sentences 1–2):
Lesions that led to increased donations were restricted to the paracentral lobule and precuneus, as well as to small areas in the right dorsomedial prefrontal cortex (Fig. 3).
Injuries leading to a decrease in donations were restricted to small sectors of the right ventrolateral occipital cortex and posterior middle temporal gyrus (Fig. 3).
The same direction in the Discussion's summary of main findings (Discussion, paragraph 1):
altruistic (costly) donations were increased by bilateral lesions of the dorsomedial parietal cortex, whereas lesions of the right posterior superior temporal sulcus and middle temporal gyri had the opposite effect
The colour code of the donation map (Figure 3 legend):
Statistical β-maps of lesions that decreased (red) or increased (blue) donations on the Altruistic Decision Task.
The reversed statement (Discussion > Brain regions implicated in donations, paragraph 1, sentence 3):
Damage to the left medial parietal lobe decreased donations, whereas damage to the right middle temporal gyrus increased donations.
The interpretation that closes the same paragraph:
Overall, our findings suggest that the superomedial parietal cortex is somehow involved in the facilitation of donations, whereas the ventrolateral occipitotemporal cortex is necessary for decisions that reduce the probability of donating.
The definition of the sign of the lesion maps (Materials and methods, lesion analysis paragraph):
each with a positive or a negative sign (i.e. the variable of interest is positively or negatively correlated with the presence of a brain lesion at a specific location
What the paper's own numbers show
Method. The error concerns the direction of an effect, not a computed value. For each passage that describes the lesion–donation result, I recorded which way it says a lesion moves donations in the two regions involved: medial parietal (precuneus/paracentral) and right posterior temporal (posterior MTG, pSTS, ventrolateral occipital). I then compared each passage with Results > Donation. The closing sentence of the Discussion paragraph speaks of the function of intact regions, so I translated it with the usual lesion-deficit reading: if a region "facilitates" donations, damage to it should lower them. I checked the laterality claim against Table 3, and checked that the Table 2 subgroups add up and that their means fit their labels. To see whether the figures follow the colour code their legends give, I counted strongly red and strongly blue pixels in the relevant panels of the paper's Figs 2 and 3 as archived in PMC (the figures are not reproduced here). As a side check, I tested whether each Table 2 column sums to $60, which the task design requires (see below). Every quote and number is checked or computed in recompute.py.txt.
| Quantity | Location | Reported | Recomputed / checked |
|---|---|---|---|
| Effect of medial parietal (precuneus/paracentral) lesions on donations | Abstract (ii); Results > Donation; Discussion summary (ii) | increased | Same direction in all three passages |
| Same effect | Discussion > Brain regions implicated in donations, sentence 3 | "decreased" | Opposite to the three passages above |
| Effect of right posterior MTG/pSTS lesions on donations | Abstract (ii); Results > Donation; Discussion summary (ii) | decreased | Same direction in all three passages |
| Same effect | Discussion > Brain regions implicated in donations, sentence 3 | "increased" | Opposite to the three passages above |
| Lesion effects implied by the closing interpretation (parietal "facilitation"; occipitotemporal cortex needed to "reduce" donating) | Same paragraph, last sentence | — | Parietal lesion → fewer donations, occipitotemporal lesion → more: the same as the reversed sentence |
| Passages agreeing with Results / reversing both regions | All six passages above, Fig. 3 legend included | — | 4 agree (Results included) / 2 reverse |
| Laterality of the medial parietal effect | Abstract and Discussion summary: "bilateral"; Discussion subsection: "left" | left (subsection) | Table 3, "Dorsomedial parietal cortex (precuneus)": side R/L, 12 right / 11 left subjects (23 in all) |
| Fig. 3, medial parietal panels A, B, G: red / blue pixels | Fig. 3 | blue = increased (legend) | 0 / 562, 0 / 664, 0 / 148: blue only, so "increased" under the legend |
| Fig. 3, lateral occipitotemporal panels E, F: red / blue pixels | Fig. 3 | red = decreased (legend) | 313 / 50 and 419 / 40: mostly red, so "decreased" under the legend |
| Fig. 2 legend check: right perisylvian (A) vs medial dorsomedial prefrontal (E–G), red / blue pixels | Fig. 2; Results > Punishment (perisylvian decreased, dorsomedial prefrontal increased punishment) | red = decreased, blue = increased | 3817 / 312 and 241 / 3092: the legend code matches the Results text for punishment as well |
| Fig. 3 colour bar | Fig. 3 (labelled "Beta") | — | The end labelled 10 is dark red, RGB (140, 20, 29); the end labelled −10 is dark blue, RGB (0, 49, 96) |
| Lesion subgroup sizes (P / D / S) | Table 2 | 18+34+22+20; 43+23+28(+0); 32+25+32+5 | 94 / 94 / 94, equal to the 94 patients |
| Mean donations, D+ (n = 23) / D− (n = 28) / controls (n = 28) | Table 2 | $20.0 / $9.7 / $12.9 | D+ is 7.1 above controls and D− 3.2 below: the means fit the subgroup labels. Table 2 does not say which regions define D+ and D− |
| Side check: mean donations + punishments + savings | Table 2, P− column | 13.5 + 2.6 + 39.4 | 55.5 (55.35–55.65 under rounding), not 60 |
| Side check: same sum | Table 2, D− column | 9.7 + 4.5 + 43.8 | 58.0 (57.85–58.15), not 60 |
| Side check: same sum | Table 2, D0 and S+ columns (savings printed without decimals) | 14.7 + 4.6 + 40; 11.4 + 2.6 + 45 | 59.3 (58.70–59.90) and 59.0 (58.40–59.60): just short of 60 even allowing ±0.5 for the whole-dollar savings |
| Side check: same sum | Table 2, the other 8 columns, controls included | — | 59.6–60.0, all consistent with 60 under rounding |
Four sources agree on the direction: the Abstract, the Results, the Discussion's summary, and the Figure 3 legend read against the map colours. The Discussion subsection reverses both regions, and its closing interpretation follows that reversal.
The Table 2 sums are a separate matter and do not bear on which direction is correct. The task gives each participant $60, with 30 opportunities to donate and 30 to punish at $1 each, and pays out "$60 minus the amount you spent on donations and punishments" (supplementary instructions). Donations + punishments + savings therefore equal 60 for every participant, and so do subgroup means taken over the same participants. The P− and D− columns are well outside what rounding allows.
Figure
Figure 1. Short excerpts quoted from the paper. The blue panel holds the direction stated in the Abstract (finding ii) and in Results > Donation; the Discussion's opening summary and the Figure 3 legend give the same direction. The orange panel holds the Discussion subsection "Brain regions implicated in donations": its third sentence, and the closing sentence that interprets the finding. The connector names the conflict. In the subsection, lesions in both regions have the opposite effect on donations, and the parietal effect, "bilateral" in the Abstract, becomes "left".
Figure 2. The direction each passage gives for the effect of a lesion on donations, by region. Blue marks the direction stated in the Abstract and Results; orange marks the reverse. The first four rows agree. The Discussion sentence reverses both regions, and so does the paragraph's closing interpretation when read in the usual lesion-deficit way (italic, because the direction is inferred rather than stated). The grey bottom row is conditional: it shows what Figure 3 would say if its colour bar (red = +10, blue = −10, labelled "Beta") were read with the Methods' definition, under which a positive sign means that a lesion goes with more donations. That reading would agree with the Discussion sentence and not with the legend. The paper does not say how β was signed for display (see Explanations considered).
Why it matters
- It concerns one of the paper's two main findings. The donation map is finding (ii) in the Abstract and in the Discussion summary. The subsection "Brain regions implicated in donations" is the only place where the paper interprets this finding, and there the reader is told the opposite of the Abstract and Results for both regions.
- The functional interpretation follows the reversed version. The closing statement, that the superomedial parietal cortex is "involved in the facilitation of donations" and the ventrolateral occipitotemporal cortex is "necessary for decisions that reduce the probability of donating", matches the reversed sentence, not the Results. If parietal lesions increase donations, as the Abstract says, the standard lesion-deficit inference is that the intact parietal region restrains giving, the opposite of facilitating it. As printed, the paper's conclusion about what these regions do and its headline result point in opposite directions, and a reader citing either one cannot tell from the text which is intended.
- The laterality also changes. The subsection calls the parietal effect left-sided. The Abstract and Discussion summary say "bilateral", and Table 3 lists the precuneus region with 12 right-hemisphere and 11 left-hemisphere subjects.
- What does not change. No statistic, p-value or effect size depends on these sentences. The paper's broader conclusion, that "altruistic punishment and donation are determined by largely non-overlapping cerebral regions", holds whichever direction is correct. The punishment findings are stated consistently in the Abstract, the Results and the Fig. 2 legend.
Explanations considered
- A different sign convention in the subsection. The reversed sentence uses the same wording as the Results ("Damage to ... decreased donations"), with the same outcome (number of donations), the same patients and the same analysis. No passage introduces a convention under which the words would change meaning, such as describing intact tissue or a deficit score. Does not account for it.
- An isolated slip in one sentence. If only one sentence were wrong, the closing interpretation in the same paragraph would be expected to follow the Results. It follows the reversed sentence instead: "facilitation of donations" by the superomedial parietal cortex implies that parietal damage lowers donations. Two statements in the subsection therefore share the reversed direction. Does not account for it as a one-word typo, although the problem is confined to this subsection.
- The figures settle which version is right. Read with their legends (red = decreased, blue = increased), Figs 2 and 3 agree with the Abstract and Results. For punishment the legend code also matches the Results text: right perisylvian clusters are red ("decreased punishment") and dorsomedial prefrontal clusters blue ("increased"). The legends of Supplementary Figs 1 and 2 use the same code ("red and blue indicate less and more, respectively"; "increased donations (blue) and decreased donations (red)"). However, the colour bars of Figs 2 and 3 are labelled "Beta" and run from red at +10 to blue at −10, and the Methods say the sign of the maps reflects whether the score "is positively or negatively correlated with the presence of a brain lesion". Read literally, that makes red mean "lesion associated with more donations", which is the opposite of the legend. On that reading the Discussion sentence would match the maps, and the Abstract, Results and legends would be reversed for donations and punishment alike. The paper does not say whether the displayed β was sign-flipped. The figures do not independently settle the question. All the text and every legend point one way; only the colour-bar label, combined with the Methods' sign definition, points the other.
- Table 2 settles it. The subgroup means fit their labels (D+ $20.0, D− $9.7, controls $12.9), but Table 2 does not say which regions define D+ and D−, and Table 3 lists the donation regions without a direction. Neither table resolves it.
- A conversion artefact in the PMC copy. The publisher's HTML and typeset PDF and the Europe PMC full-text XML all contain the same sentences. Only words are involved, so minus signs, superscripts and table alignment play no part. Does not account for it.
- For the Table 2 side observation: rounding or differing denominators. One-decimal rounding can move a three-term sum by at most 0.15, or by up to 0.6 when a savings mean is printed as a whole number. The P− (55.5) and D− (58.0) columns are outside this range. The sums could differ from 60 if some participants were missing one of the three measures, so that the means were taken over different sets of people, but the paper reports a single n per subgroup and does not mention missing data. A transcription error in one cell per affected column would also explain it; the paper gives no way to tell which cell. Not resolved by the published material.
Status
As of 2026-10-23:
- Version of record. The publisher's page (doi:10.1093/brain/awy064, which resolves to academic.oup.com/brain/article/141/5/1558/4953518) and the publisher-typeset PDF ("BRAIN 2018: 141; 1558–1569") contain the same Abstract finding (ii), Results > Donation text, Discussion summary, reversed Discussion sentence and closing interpretation as the PMC archive. Table 2 in the typeset PDF has the same values. The Europe PMC full-text XML has identical wording. The discrepancy is in the version of record.
- Supplements. The single supplementary file (task instructions, list of organisations, Supplementary Tables 1–2, legends of Supplementary Figs 1–2) does not describe the lesion–donation directions in words beyond the subgroup labels ("D + : lesions increased donations"; "D ‒ : lesions decreased donations"). Its figure legends use the same colour code as the main figures. The instructions confirm that payout equals $60 minus donations and punishments.
- Corrections. Crossref lists no update relations. PubMed Central and Europe PMC list no correction, erratum or retraction notices. The publisher's page shows none.
- PubPeer and the web. The PubPeer search page returned HTTP 403 (checked again on 2026-10-23), so PubPeer could not be checked directly. Web searches made during this audit for the title or DOI combined with "PubPeer", "correction", "erratum" or "corrigendum" returned only the article itself, institutional listings and press coverage. A further search could not be run on 2026-10-23 because the search quota was exhausted.
- Sources. PMC7341482 (archived full text, figure images); publisher HTML and typeset PDF (version of record); Europe PMC full-text XML; Crossref metadata; the supplementary PDF (awy064_brain-2017-01334-file008.pdf).
Suggested correction
Ask the authors to state the sign convention used to display the β-maps in Figs 2 and 3, check it against the underlying maps, and correct whichever passages disagree with it.
If the Abstract, Results and figure legends are correct, the Discussion subsection "Brain regions implicated in donations" could read, for example: "Damage to the medial parietal lobe (bilaterally) increased donations, whereas damage to the right posterior middle temporal gyrus decreased donations." The closing interpretation would then need to be revised to match: on the usual lesion-deficit reading, the intact superomedial parietal cortex would be associated with restraining donations, and the right ventrolateral occipitotemporal cortex with supporting them.
If instead the colour bars are correct and the legends reversed, the Abstract (findings i and ii), the Results, the Discussion summary and the figure legends would all need correcting.
Separately, the authors could check the P−, D−, D0 and S+ columns of Table 2, whose mean donations, punishments and savings should sum to $60.
Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC7341482 and regenerate the figures.
Run: python recompute.py.txt
(writes fig1.svg/png and fig2.svg/png next to this file)
Paper: "Altruistic decisions following penetrating traumatic brain injury",
Brain 2018;141(5):1558-1569, doi:10.1093/brain/awy064, PMC7341482.
All inputs are transcribed from the article (Abstract, Results > Donation,
Discussion, Tables 2 and 3, legends of Figs 2 and 3) and from its single
supplementary file (task instructions, Supplementary Table 2 and figure legends).
Optional parts, skipped with a message if unavailable:
part 0 checks every quote used in the write-up verbatim against the PMC archive
(needs the papers repository and its src/show.py);
part 4 downloads the paper's Fig. 2 and Fig. 3 images from the PMC CDN into a
temporary directory, checks their MD5, and counts strongly red and strongly
blue pixels in a few panels. Nothing from the images is saved or redrawn.
"""
import hashlib
import os
import subprocess
import tempfile
import textwrap
import urllib.request
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------------------
# 0. Quotes used in writeup.md, checked verbatim against the PMC archive
# ---------------------------------------------------------------------------
QUOTES = {
"Abstract (ii)":
"altruistic donations were increased by bilateral lesions of the dorsomedial "
"parietal cortex, whereas lesions of the right posterior superior temporal "
"sulcus and middle temporal gyri decreased donations",
"Results > Donation (1)":
"Lesions that led to increased donations were restricted to the paracentral "
"lobule and precuneus, as well as to small areas in the right dorsomedial "
"prefrontal cortex (Fig. 3).",
"Results > Donation (2)":
"Injuries leading to a decrease in donations were restricted to small sectors "
"of the right ventrolateral occipital cortex and posterior middle temporal "
"gyrus (Fig. 3).",
"Discussion summary (ii)":
"altruistic (costly) donations were increased by bilateral lesions of the "
"dorsomedial parietal cortex, whereas lesions of the right posterior superior "
"temporal sulcus and middle temporal gyri had the opposite effect",
"Fig. 3 legend":
"Statistical β-maps of lesions that decreased (red) or increased (blue) "
"donations on the Altruistic Decision Task.",
"Discussion > donations, sentence 3":
"Damage to the left medial parietal lobe decreased donations, whereas damage "
"to the right middle temporal gyrus increased donations.",
"Discussion > donations, closing sentence":
"Overall, our findings suggest that the superomedial parietal cortex is "
"somehow involved in the facilitation of donations, whereas the ventrolateral "
"occipitotemporal cortex is necessary for decisions that reduce the "
"probability of donating.",
"Methods, sign of maps":
"each with a positive or a negative sign (i.e. the variable of interest is "
"positively or negatively correlated with the presence of a brain lesion at a "
"specific location",
"Methods, task":
"each participant had the opportunity to keep a maximum of $60, donate a "
"maximum of $30, or spend a maximum of $30 in punishments",
"Results > Punishment":
"In the right hemisphere, the lesions leading to decreased punishment followed "
"a perisylvian distribution",
"Results > Punishment (increase)":
"Lesions that increased punishments were found (i) bilaterally in the "
"dorsomedial prefrontal cortex",
}
REPO = "/home/ubuntu/papers"
PY = "/home/ubuntu/papers-data/venv/bin/python"
print("== 0. Verbatim check of quotes against the PMC archive ==")
try:
full = subprocess.run([PY, "src/show.py", "PMC7341482", "--full"], cwd=REPO,
capture_output=True, text=True, timeout=120).stdout
if not full:
raise RuntimeError("show.py returned nothing")
for k, q in QUOTES.items():
print(f" {k:42s} {'found verbatim' if q in full else 'NOT FOUND'}")
except Exception as e: # repository not available
print(f" skipped ({e})")
# ---------------------------------------------------------------------------
# 1. Direction of the lesion effect on donations, passage by passage
# "+" = lesion increased donations, "-" = lesion decreased donations
# ---------------------------------------------------------------------------
PAR = "medial parietal"
TMP = "right posterior temporal"
passages = [
# (location, kind, {region: sign}, parietal laterality)
("Abstract, finding (ii)", "stated", {PAR: "+", TMP: "-"}, "bilateral"),
("Results > Donation", "stated", {PAR: "+", TMP: "-"}, "not stated"),
("Discussion, opening summary (ii)", "stated", {PAR: "+", TMP: "-"}, "bilateral"),
("Fig. 3 legend read with map colours", "stated", {PAR: "+", TMP: "-"}, "n/a"),
("Discussion > donations, sentence 3", "stated", {PAR: "-", TMP: "+"}, "left"),
("Discussion > donations, closing sentence", "implied (lesion-deficit reading)",
{PAR: "-", TMP: "+"}, "n/a"),
]
print("\n== 1. Direction of lesion effect on donations, by passage ==")
ref = passages[1][2] # Results > Donation
n_agree = n_rev = 0
for loc, kind, d, lat in passages:
flips = [r for r in d if d[r] != ref[r]]
tag = ("REVERSES Results for both regions" if len(flips) == 2 else
"agrees with Results" if not flips else f"reverses {flips}")
n_agree += not flips
n_rev += len(flips) == 2
print(f" {loc:42s} parietal {d[PAR]} temporal {d[TMP]} "
f"parietal side: {lat:10s} [{kind}] -> {tag}")
print(f" passages agreeing with Results: {n_agree} (including Results itself); "
f"reversing both regions: {n_rev}")
# ---------------------------------------------------------------------------
# 2. Table 2 (lesion subgroups)
# Methods and supplementary instructions: $60 to start, 30 donate and 30 punish
# opportunities at $1 each; payout = $60 minus donations minus punishments.
# So for each participant D + P + S = 60, and the same holds for subgroup means
# computed over the same participants.
# ---------------------------------------------------------------------------
T2 = { # (donations, punishments, savings, n) as printed
"P0": ("13.2", "3.8", "42.9", 18), "P+": ("16.1", "6.6", "37.3", 34),
"P-": ("13.5", "2.6", "39.4", 22), "P±": ("14.1", "4.0", "41.8", 20),
"D0": ("14.7", "4.6", "40", 43), "D+": ("20.0", "4.7", "35.2", 23),
"D-": ("9.7", "4.5", "43.8", 28),
"S0": ("14.3", "4.4", "41", 32), "S+": ("11.4", "2.6", "45", 25),
"S-": ("17.9", "6.7", "35", 32), "S±": ("14.2", "2.4", "43.4", 5),
"Controls": ("12.9", "5.8", "41.2", 28),
}
N_PATIENTS = 94
def halfwidth(s):
"""Largest rounding error of a printed value (0.05 for one decimal, 0.5 for none)."""
return 0.5 * 10 ** (-(len(s.split(".")[1]) if "." in s else 0))
print("\n== 2. Table 2 ==")
for g in "PDS":
ks = [k for k in T2 if k[0] == g]
print(f" {g} subgroups: n = " + " + ".join(str(T2[k][3]) for k in ks)
+ f" = {sum(T2[k][3] for k in ks)} (patients: {N_PATIENTS})")
d_plus, d_minus, ctl = (float(T2[k][0]) for k in ("D+", "D-", "Controls"))
print(f" mean donations D+ {d_plus:.1f} D- {d_minus:.1f} controls {ctl:.1f}; "
f"D+ minus controls = {d_plus - ctl:+.1f}, D- minus controls = {d_minus - ctl:+.1f}")
print(" mean donations + punishments + savings (each participant's total is 60):")
offenders = []
for k, (d, p, s, n) in T2.items():
tot = float(d) + float(p) + float(s)
hw = halfwidth(d) + halfwidth(p) + halfwidth(s)
ok = tot - hw <= 60 + 1e-9 <= tot + hw + 2e-9
if not ok:
offenders.append((k, tot, tot - hw, tot + hw))
print(f" {k:9s} n={n:3d} {d:>5s} + {p:>4s} + {s:>5s} = {tot:5.1f} "
f"(range under rounding {tot - hw:.2f} to {tot + hw:.2f}) "
f"{'consistent with 60' if ok else 'NOT consistent with 60'}")
print(" columns whose sum cannot be 60 under rounding: "
+ "; ".join(f"{k} {t:.1f} ({lo:.2f}-{hi:.2f})" for k, t, lo, hi in offenders))
# ---------------------------------------------------------------------------
# 3. Table 3 (donation rows), as printed; laterality of the parietal effect
# ---------------------------------------------------------------------------
T3_DON = [("Middle temporal sulcus (posterior half)", "R", "37", "6"),
("Dorsomedial parietal cortex (precuneus)", "R/L", "5/7m", "12/11"),
("Medial orbitofrontal/gyrus rectus", "R/L", "11", "6/7"),
("Inferior frontal gyrus", "L", "45/46/47", "17")]
print("\n== 3. Table 3, donation rows ==")
for r in T3_DON:
print(f" {r[0]:42s} side {r[1]:4s} BA {r[2]:9s} subjects (R/L) {r[3]}")
prec_r, prec_l = map(int, T3_DON[1][3].split("/"))
print(f" precuneus: {prec_r} right + {prec_l} left subjects = {prec_r + prec_l}; "
"listed for both hemispheres, not left only")
print(" Table 3 gives no direction (increase/decrease) for any donation region")
# ---------------------------------------------------------------------------
# 4. Optional: colour check of the paper's Fig. 2 and Fig. 3 (not redrawn)
# ---------------------------------------------------------------------------
FIGS = {
"f2": ("https://cdn.ncbi.nlm.nih.gov/pmc/blobs/e84c/7341482/bd9cea49707e/awy064f2.jpg",
"bd9cea49707e56866080625dd2ce2009"),
"f3": ("https://cdn.ncbi.nlm.nih.gov/pmc/blobs/e84c/7341482/af8bc4065be1/awy064f3.jpg",
"af8bc4065be1d710719d4406426873f8"),
}
# panel boxes (x0, x1, y0, y1) in pixels of the 764 x 512 PMC images
BOX3 = {"A medial parietal (precuneus/SPL)": (40, 260, 60, 270),
"B medial (precuneus/dmPFC)": (255, 480, 0, 160),
"G medial": (490, 690, 300, 512),
"E lateral (occipital/pMTG)": (0, 255, 300, 512),
"F lateral (occipital/pMTG)": (255, 490, 300, 512)}
BOX2 = {"A right lateral perisylvian": (0, 240, 20, 215),
"E/F/G medial dmPFC": (250, 700, 215, 512)}
def counts(img, box):
x0, x1, y0, y1 = box
a = img[y0:y1, x0:x1, :3].astype(float)
if a.max() <= 1:
a = a * 255
R, G, B = a[..., 0], a[..., 1], a[..., 2]
red = ((R - B) > 60) & ((R - G) > 50)
blue = ((B - R) > 60) & (B > 110)
return int(red.sum()), int(blue.sum())
print("\n== 4. Colour check of the paper's figures "
"(legends: red = decreased, blue = increased) ==")
try:
from matplotlib.image import imread
imgs = {}
with tempfile.TemporaryDirectory() as td:
for k, (url, md5) in FIGS.items():
req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0"})
data = urllib.request.urlopen(req, timeout=30).read()
if hashlib.md5(data).hexdigest() != md5:
raise RuntimeError(f"{k}: image differs from the one checked")
path = os.path.join(td, k + ".jpg")
with open(path, "wb") as fh:
fh.write(data)
imgs[k] = imread(path)
print(" Fig. 3 (donations):")
for k, b in BOX3.items():
r, bl = counts(imgs["f3"], b)
print(f" {k:36s} red {r:5d} blue {bl:5d}")
print(" Fig. 2 (punishment), legend convention checked against Results > Punishment:")
for k, b in BOX2.items():
r, bl = counts(imgs["f2"], b)
print(f" {k:36s} red {r:5d} blue {bl:5d}")
col = imgs["f3"][:, 722, :3].astype(int) # colour bar column of Fig. 3
top, bot = tuple(int(v) for v in col[60]), tuple(int(v) for v in col[460])
print(f" Fig. 3 colour bar (labelled 'Beta'): end labelled 10 has RGB {top} (red); "
f"end labelled -10 has RGB {bot} (blue)")
except Exception as e: # network or image problems: skip this optional part
print(f" skipped ({e})")
# ---------------------------------------------------------------------------
# Figures
# ---------------------------------------------------------------------------
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#d9d8d4"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
"text.color": TXT, "svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# ---- fig1: two text panels joined by an annotated connector ---------------
W, H = 7.2, 5.0
fig = plt.figure(figsize=(W, H), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, W)
ax.set_ylim(0, H)
ax.axis("off")
PX, PW, PY0, PH = (0.12, 3.78), 3.3, 1.55, 3.33
def panel(x, color, title, sub, blocks):
ax.add_patch(FancyBboxPatch((x, PY0), PW, PH,
boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=color, lw=2))
ax.text(x + 0.15, PY0 + PH - 0.14, title, fontsize=11, fontweight="bold",
color=color, va="top")
ax.text(x + 0.15, PY0 + PH - 0.42, sub, fontsize=10, color=SEC, va="top")
yy = PY0 + PH - 0.80
for head, body in blocks:
ax.text(x + 0.15, yy, head, fontsize=10, color=SEC, va="top", style="italic")
yy -= 0.23
wrapped = textwrap.fill(body, 40)
ax.text(x + 0.15, yy, wrapped, fontsize=10, color=TXT, va="top",
linespacing=1.3)
yy -= 0.205 * (wrapped.count("\n") + 1) + 0.2
left = [("Abstract, finding (ii)",
"“...donations were increased by bilateral lesions of the dorsomedial "
"parietal cortex, whereas lesions of the right posterior superior temporal "
"sulcus and middle temporal gyri decreased donations”"),
("Results > Donation",
"“Lesions that led to increased donations were restricted to the "
"paracentral lobule and precuneus ...”")]
right = [("Paragraph 1, third sentence",
"“Damage to the left medial parietal lobe decreased donations, whereas "
"damage to the right middle temporal gyrus increased donations.”"),
("Same paragraph, closing sentence",
"“...the superomedial parietal cortex is somehow involved in the "
"facilitation of donations ...”")]
panel(PX[0], BLUE, "Abstract and Results",
"(also Discussion summary, Fig. 3 legend)", left)
panel(PX[1], ORANGE, "Discussion subsection",
"“Brain regions implicated in donations”", right)
# connector: a bracket from the bottom of each panel to a labelled box
lx, rx = PX[0] + PW / 2, PX[1] + PW / 2
ybot, ymid = PY0 - 0.03, 1.12
ax.plot([lx, lx, rx, rx], [ybot, ymid, ymid, ybot], color=TXT, lw=2,
solid_joinstyle="miter", zorder=1)
ax.add_patch(FancyBboxPatch((W / 2 - 1.85, 0.86), 3.7, 0.52,
boxstyle="round,pad=0.02,rounding_size=0.06",
fc=BG, ec=TXT, lw=2, zorder=2))
ax.text(W / 2, 1.12, "Conflict: opposite direction for both regions",
ha="center", va="center", fontsize=10.5, fontweight="bold", color=TXT, zorder=3)
rows = [("medial parietal lesions:", "increased", "decreased"),
("right MTG lesions:", "decreased", "increased"),
("parietal laterality:", "“bilateral”", "“left”")]
for i, (lab, a, b) in enumerate(rows):
y = 0.55 - 0.21 * i
ax.text(2.85, y, lab, ha="right", va="center", fontsize=10, color=SEC)
ax.text(2.95, y, a, ha="left", va="center", fontsize=10, color=BLUE,
fontweight="bold")
ax.text(4.0, y, "→", ha="center", va="center", fontsize=10, color=SEC)
ax.text(4.15, y, b, ha="left", va="center", fontsize=10, color=ORANGE,
fontweight="bold")
save(fig, "fig1")
# ---- fig2: direction grid, passage x region --------------------------------
rows = [
("Abstract, finding (ii)", "+", "-", "stated"),
("Results > Donation", "+", "-", "stated"),
("Discussion, opening summary (ii)", "+", "-", "stated"),
("Fig. 3 legend + map colours", "+", "-", "stated"),
("Discussion > donations, sentence 3", "-", "+", "stated"),
("Same paragraph, closing sentence*", "-", "+", "implied"),
("Fig. 3 colour bar, if +β = more†", "-", "+", "conditional"),
]
nr = len(rows)
fig, ax = plt.subplots(figsize=(7.2, 5.4), facecolor=BG)
ax.set_facecolor(BG)
ax.set_xlim(0, 7.2)
ax.set_ylim(-3.95, nr + 0.85)
ax.axis("off")
cols = {1: (3.85, "Medial parietal\n(precuneus/paracentral)"),
2: (5.9, "Right posterior MTG/\npSTS/occipital")}
ax.text(0.08, nr + 0.35, "Passage", va="center", fontsize=10, fontweight="bold")
for _, (cx, lab) in cols.items():
ax.text(cx, nr + 0.35, lab, ha="center", va="center", fontsize=10,
fontweight="bold")
ax.plot([0.05, 7.15], [nr - 0.5] * 2, color=SEC, lw=1)
for i, (lab, a, b, kind) in enumerate(rows):
y = nr - 1 - i
ax.plot([0.05, 7.15], [y - 0.5] * 2, color=GRID, lw=0.8)
ax.text(0.08, y, lab, va="center", fontsize=10,
color=TXT if kind == "stated" else SEC,
style="normal" if kind == "stated" else "italic")
for j, s in ((1, a), (2, b)):
cx = cols[j][0]
color = BLUE if s == rows[1][j] else ORANGE
if kind == "conditional":
color = SEC
word = "▲ increased" if s == "+" else "▼ decreased"
ax.text(cx, y, word, ha="center", va="center", fontsize=10.5,
color=color, fontweight="bold" if kind == "stated" else "normal")
leg = [(BLUE, "same direction as Abstract and Results"),
(ORANGE, "reverse direction"),
(SEC, "depends on a sign convention the paper does not state")]
for k, (c, t) in enumerate(leg):
yy = -0.95 - 0.42 * k
ax.plot([0.15], [yy], marker="s", ms=9, color=c, ls="none")
ax.text(0.32, yy, t, va="center", fontsize=10, color=TXT)
foot = ("* Lesion-deficit reading: a region that “facilitates” donations "
"should lower them when damaged.\n"
"† The colour bar, labelled “Beta”, runs from red (+10) to blue "
"(−10). The Methods\n define the sign as a positive or negative "
"correlation with lesion presence;\n the legend says red = decreased.")
ax.text(0.08, -2.25, foot, fontsize=10, color=SEC, va="top", linespacing=1.35)
fig.subplots_adjust(left=0.01, right=0.99, top=0.99, bottom=0.01)
save(fig, "fig2")
print("\nfigures written: fig1.svg/png, fig2.svg/png in", HERE)
Abstract reports 18 PDX models for the co-clinical trial; the paper's data contain 15
Cancer Res (2022) · doi:10.1158/0008-5472.CAN-22-0198 · PMC9478530
Summary
The abstract says the combination of MEK and CDK4/6 inhibition was tested "across 18 PDX models" and that 60% of tumors regressed, meeting the co-clinical trial's primary endpoint. Every other count in the paper gives 15 models. The Results subgroups add up to 6 + 2 + 3 + 4 = 15, the Results report "9 of 15 models (60%)", the Fig. 1 legends say 15 models, and Supplementary Tables S1 and S3 each list the same 15 model IDs. Every model used in the later experiments is one of those 15. Of the two counts, only 15 gives exactly 60% (9/15); with 18 the nearest values are 55.6% (10/18) and 61.1% (11/18), 60% of 18 is 10.8 models, and the 9 responders reported in the Results give 50%. The response rate and the "endpoint met" conclusion hold at 9 of 15. What the abstract gets wrong is the size of the evidence base: it overstates it by 3 models (20%), and it sits in the place readers and meta-analysts most often take sample sizes from.
What the paper reports
Abstract:
Across 18 PDX models undergoing dual inhibition of MEK and CDK4/6, 60% of tumors regressed, meeting the co-clinical trial primary endpoint.
Results, "Combined inhibition of MEK and CDK4/6 in colorectal cancer PDX models", first paragraph:
we evaluated typical RAS models that include a mixture of exon 2 KRASG12/13 (n = 6) P-loop domain mutated PDX models, extended RAS models that include NRASQ61 switch II domain mutated tumors (n = 2) along with KRASA146T nucleotide exchange promoting domain mutated tumors (n = 3; refs. 23, 24), and wild-type RAS tumors (n = 4).
Results, same section, second paragraph:
The PDX co-clinical trial of MEK (trametinib) and CDK4/6 (palbociclib) inhibition met its prespecified primary endpoint of tumor regression of ≥30% in more than 30% of the models. As shown in Fig. 1B and C, the combination of MEK and CDK4/6 inhibition produced ≥30% regression in 9 of 15 models (60%) with significantly greater tumor growth inhibition when compared with single agent–treated PDXs.
Figure 1 legend, panels A to D:
Six PDX models harboring codon 12 or 13 (“typical” RAS), 5 with codons 61 or 149 mutations (“extended” RAS), and 4 wild-type RAS PDX models were analyzed by RPPA
Trametinib (0.25 mpk, daily; green), palbociclib (75 mpk, daily; blue), and their combination (red) were evaluated in 15 mCRC PDX models.
C, Waterfall representation of 15 PDX models’ responses, per PDX model results.
D, Effect of treatments on mice body weight. Right, Tukey plot, after 21 days of treatment. Summarized data for 15 PDX models.
Results, "Mechanisms of acquired resistance":
Intriguingly, elevated phospho-SHP2 level was detected in 11 of 15 PDX models treated with the combination
Materials and Methods, "In vivo studies and PDXs":
PDX model details are provided in Supplementary Table S1.
Discussion, first paragraph:
The co-clinical trial met its primary endpoint of demonstrating tumor regressions in greater than 30% of models
What the paper's own numbers show
Method. I added up the RAS subgroup counts given in the Results text and in the Fig. 1A legend. I then built the list of PDX model IDs from Supplementary Table S1 (sheet "PDX details"), Supplementary Table S3 (the ranking of combination responses, grouped by RAS status), and the Fig. 1B panel titles. I added every model named for the follow-on experiments: the dose study (Fig. 2, Supp. Table S6), the durability cohort (Fig. 4, Supp. Table S7), the SHP2 triple combination (Fig. 5) and Supp. Fig. S2. Last, I checked which denominators can produce the abstract's 60%. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| PDX models given MEK + CDK4/6 inhibition | Abstract | 18 | 15; no set of 18 models appears anywhere in the paper or supplement |
| RAS subgroups, text | Results, paragraph 1 | n = 6, 2, 3, 4 | 6 + 2 + 3 + 4 = 15 |
| RAS subgroups, figure legend | Fig. 1A legend | 6 typical, 5 extended, 4 wild type | 6 + 5 + 4 = 15; the text's 2 + 3 = 5 extended models match |
| Models in the co-clinical trial | Results, paragraph 2; Fig. 1B–D legends | "9 of 15 models"; "15 mCRC PDX models" | 15 |
| Model panels | Fig. 1B | – | 15 |
| Model rows | Supp. Table S1 ("PDX details") | – | 15 unique IDs |
| Model rows by RAS group | Supp. Table S3 | – | 15 (6 typical, 5 extended, 4 wild type); same IDs as S1 and Fig. 1B |
| Models in follow-on experiments | Figs. 2, 4, 5; Supp. Fig. S2; Supp. Tables S6, S7 | 3, 8, 2 and 2 models | 9 distinct models, 0 outside the 15 |
| All distinct models named anywhere | Paper and supplement | – | 15, so 18 − 15 = 3 models are unaccounted for |
| Durability cohort | Results; Fig. 4 | "eight PDX models (4 of each RASmut subtype)" | 8, all among the 15 (4 typical, 4 extended) |
| Regression rate, 15 models | Abstract; Results | 60%; "9 of 15 models (60%)" | 9/15 = 60.0% |
| Same 9 responders over 18 models | – | – | 9/18 = 50.0% |
| Responders implied by 60% of 18 | Abstract | 60% | 0.6 × 18 = 10.8, not a whole number; 10/18 = 55.6%, 11/18 = 61.1% |
| Values of k for which k/n rounds to 60% | – | – | n = 15: k = 9 only; n = 18: none |
| Primary endpoint (more than 30% of models with ≥30% regression) | Results | met | 9/15 = 60% > 30%. Exact one-sided binomial test against 30% (my calculation; the paper reports no test): P = 0.015. Clopper–Pearson 95% CI 32.3% to 83.7%, one-sided 95% lower bound 36.0% |
Figure
Figure 1. Two text panels hold short excerpts from the article as archived in PMC. The left panel (orange, the value reported in the abstract) gives the abstract's sentence with its count of 18 models and its 60% response rate. The right panel (blue, counts found in or recomputed from the paper's own body and supplement) gives the Results' "9 of 15 models (60%)", the Results' subgroup counts, which add up to 15, the Fig. 1B legend's "15 mCRC PDX models", and the 15 rows in each of Supplementary Tables S1 and S3. The connector names the conflict: no set of 18 models is described anywhere in the paper, and of the two counts only 15 gives exactly 60% (9/15). The arithmetic is in recompute.py.txt.
Figure 2. Each gray dot is a response rate that can occur with n models (k/n, k = 0 to n), on a full 0–100% axis. The orange vertical line marks the 60% reported in both the abstract and the Results. The dotted line marks the primary endpoint's 30% threshold. With n = 15, 60% falls exactly on 9/15 (filled blue). With n = 18, no attainable value equals 60%: the same 9 responders give 9/18 = 50.0% (filled blue), and the two nearest values are 10/18 = 55.6% and 11/18 = 61.1% (open blue). Of the two counts in the paper, the reported 60% therefore matches the 15-model denominator exactly, and not the 18 stated in the abstract.
Why it matters
The abstract is the only place in the paper that says 18, and the abstract is where readers, indexers and meta-analysts usually take the number of models. Anyone counting PDX models across preclinical studies of MEK plus CDK4/6 inhibition, or citing this study as an example of a co-clinical trial design, would record 3 models (20%) that the paper does not document.
The efficacy conclusion is unaffected. The correct figure is 9 of 15 models with at least 30% regression (60%), well above the prespecified 30% threshold. The exact 95% interval for 9/15 runs from 32.3% to 83.7%. A reader who combined the abstract's 18 with the 9 responders given in the Results would get 50%, which still exceeds the 30% threshold but does not match the abstract's own 60%. The subgroup comparisons (typical vs. extended RAS, TP53, CMS) already rest on small groups, and the 15-model count is the right basis for judging them.
Explanations considered
- A wider set of models, such as the co-clinical trial plus the dose, durability or SHP2 experiments. This does not explain the 18. The models in Fig. 2 and Supp. Table S6 (C1138, B1011, C1174), Fig. 4 and Supp. Table S7 (8 models: C1138, C1047, B8093, B1011, B1008, F3008, C1012, C0997), Fig. 5 (B1008, F3008) and Supp. Fig. S2 (F3008, B1011) are all among the 15. The union of every model named in the paper and supplement is 15.
- A different unit, such as tumors or mice instead of models. This does not fit. The abstract says "18 PDX models" outright. Fig. 1B reports n = 6–9 tumors per arm for each model, so a count of tumors would be roughly 90 to 135 in the combination arm alone (15 models × 6–9 tumors), not 18.
- Another denominator that gives 60%. Not with 18. 0.6 × 18 = 10.8 is not a whole number, and no k/18 rounds to a whole-percent 60% (10/18 = 55.6%, 11/18 = 61.1%). Other denominators such as 5, 10 or 20 could give exactly 60%, but no such model count appears in the paper. Of the two counts the paper gives, only 15 yields exactly 60% (9/15), which is what the Results report.
- Models in the supplement that the main text omits. Supplementary Table S1 (PDX details) and Supplementary Table S3 (response ranking) each list exactly 15 models, with the same IDs as the 15 panels of Fig. 1B, split 6 typical, 5 extended and 4 wild type. The Supplementary Materials and Methods describe no other cohort.
- A text-conversion or archiving artifact. Ruled out. "18" is a spelled-out integer in running text, so it cannot be a lost sign, a misread superscript or a shifted table cell. The same wording appears on the live PMC page, in the Europe PMC full-text XML, in the PubMed abstract (PMID 35913398), and in the abstract on the publisher's figshare collection for the article.
- A later correction. None found (see Status).
- Side observation, not part of the claim. The Fig. 1A legend describes the extended group as "codons 61 or 149", while the Results text names KRAS A146T. The Fig. 1B panel titles label the three KRAS codon-146 models as B1011 A146T, C1151 A146T and C1012 A146V, so the text's "KRASA146T (n = 3)" is also slightly off. None of this changes any count: the 5 extended models equal the text's 2 NRAS Q61 plus 3 KRAS codon-146 models.
Status
As of 2026-10-23:
- Version of record. The publisher page (aacrjournals.org, Cancer Res 82(18):3335) and its PDF returned HTTP 403 to automated access and could not be read directly. The abstract sentence with "18" and the Results' "9 of 15 models (60%)" are identical on the live PMC page (PMC9478530), in the Europe PMC full-text XML, in the PubMed record (PMID 35913398) and in the abstract on AACR's figshare collection for the article (doi:10.1158/0008-5472.c.6513959). The figshare collection is published by the journal, so the error is very likely in the version of record too.
- Supplements. I took the full supplementary package (Figures S1–S5, Supplementary Materials and Methods, Tables S1–S9) from Europe PMC's supplementaryFiles endpoint and read Table S1, Table S3, Table S6 ("Results of alternative trametinib/palbociclib regimens study"; models C1138, B1011 and C1174), Table S7 ("TP53 mutational status in mCRC PDX models") and the Supplementary Materials and Methods. All are consistent with 15 models, as described above.
- Corrections. Crossref has no update-to or updated-by relations for the DOI. PMC and Europe PMC list no erratum, correction, expression of concern or retraction.
- PubPeer. The PubPeer publications API, queried with the DOI, returned no comments, and a web search found no PubPeer thread for the article.
Suggested correction
In the abstract, replace "Across 18 PDX models undergoing dual inhibition of MEK and CDK4/6, 60% of tumors regressed" with "Across 15 PDX models undergoing dual inhibition of MEK and CDK4/6, 9 (60%) regressed". This brings the abstract in line with the Results, the Fig. 1 legends and Supplementary Tables S1 and S3. The journal could also check the Fig. 1A legend's "codons 61 or 149" and the Results' "KRASA146T (n = 3)" against the Fig. 1B panel labels, which show two A146T models and one A146V model.
Recomputation script (python recompute.py.txt)
"""Recompute the PDX model count behind the abstract of PMC9478530
(Cancer Res 2022;82(18):3335, doi:10.1158/0008-5472.CAN-22-0198).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md,
writes fig1.svg/png and fig2.svg/png next to this file, and writes five candidate
figure designs (A-E) to ./candidates/.
Inputs are copied from the article as archived in PubMed Central and from its
supplementary files (Europe PMC supplementaryFiles package):
Abstract: "Across 18 PDX models ... 60% of tumors regressed"
Results, first section: subgroup counts n = 6 (KRAS G12/13), 2 (NRAS Q61),
3 (KRAS A146T), 4 (RAS wild type); "9 of 15 models (60%)"
Fig. 1 legend: 6 typical + 5 extended + 4 wild type; "15 mCRC PDX models"
Supplementary Table S1: sheet "PDX details", one row per model (model IDs only used here)
Supplementary Table S3: combination-response ranking, one row per model, by RAS group
Figs. 2, 4, 5, Supp. Fig. S2, Supp. Tables S6 and S7: models named for the follow-on experiments
Model IDs are laboratory PDX codes, not patient identifiers.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle # noqa: E402
from scipy import stats # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs (from the paper)
ABSTRACT_N = 18 # Abstract: "Across 18 PDX models"
ABSTRACT_PCT = 60 # Abstract: "60% of tumors regressed"
RESULTS_RESPONDERS = 9 # Results: "≥30% regression in 9 of 15 models (60%)"
RESULTS_N = 15
RESULTS_PCT = 60
ENDPOINT_THRESHOLD = 0.30 # Results: primary endpoint "≥30% in more than 30% of the models"
SUBGROUPS_TEXT = {"KRAS G12/13 (typical)": 6, "NRAS Q61 (extended)": 2,
"KRAS A146T (extended)": 3, "RAS wild type": 4} # Results paragraph 1
SUBGROUPS_FIG1A = {"typical": 6, "extended": 5, "wild type": 4} # Fig. 1A legend
FIG1_LEGEND_N = 15 # Fig. 1B, 1C and 1D legends: "15 mCRC PDX models" / "15 PDX models"
SHP2_OF = 15 # Results: "elevated phospho-SHP2 ... in 11 of 15 PDX models treated with the combination"
S3 = { # Supplementary Table S3, model -> RAS group
"B1008": "typical", "C1035": "typical", "C1047": "typical",
"C1138": "typical", "C1177": "typical", "F3008": "typical",
"B1011": "extended", "B8093": "extended", "C0997": "extended",
"C1012": "extended", "C1151": "extended",
"B1025": "wild type", "B8178": "wild type", "C1023": "wild type", "C1174": "wild type",
}
S1 = ["C1035", "C1047", "B1008", "F3008", "C1138", "C1177", "B8093", "C0997",
"C1151", "C1012", "C1023", "B1025", "C1174", "B8178", "B1011"] # Supp. Table S1, "PDX details"
FIG1B = ["C1035", "C1047", "B1008", "F3008", "C1138", "C1177", "B8093", "C0997",
"C1151", "B1011", "C1012", "C1023", "B1025", "C1174", "B8178"] # Fig. 1B panel titles
FIG2 = ["C1138", "B1011", "C1174"] # Fig. 2 / Supp. Table S6 (dose study)
FIG4 = ["C1138", "C1047", "B8093", "B1011", "B1008", "F3008", "C1012", "C0997"] # Fig. 4 / Supp. Table S7 (TP53 status, durability cohort)
FIG5 = ["B1008", "F3008"] # Fig. 5 (SHP2 triple)
SUPP_S2 = ["F3008", "B1011"] # Supp. Fig. S2
# ---------------------------------------------------------------- recomputation
def line(label, value):
print(f"{label:<62} {value}")
print("== Model counts")
sum_text = sum(SUBGROUPS_TEXT.values())
sum_fig1a = sum(SUBGROUPS_FIG1A.values())
line("Results subgroups 6 + 2 + 3 + 4", sum_text)
line("Fig. 1A legend 6 + 5 + 4", sum_fig1a)
line("Extended group in text (NRAS Q61 + KRAS A146T) 2 + 3", 2 + 3)
s3_groups = {g: sum(v == g for v in S3.values()) for g in ("typical", "extended", "wild type")}
line("Supp. Table S3 models", f"{len(S3)} {s3_groups}")
line("Supp. Table S1 models (unique)", len(set(S1)))
line("Fig. 1B model panels (unique)", len(set(FIG1B)))
line("S1, S3 and Fig. 1B list the same models", set(S1) == set(S3) == set(FIG1B))
followon = set(FIG2) | set(FIG4) | set(FIG5) | set(SUPP_S2)
line("Models in Figs. 2, 4, 5, Supp. Fig. S2, Supp. Tables S6, S7 (unique)", len(followon))
line("... of which outside the 15", len(followon - set(S3)))
union = set(S3) | set(S1) | set(FIG1B) | followon
line("Union of all models named in paper and supplement", len(union))
line("Abstract count minus union", ABSTRACT_N - len(union))
line("Abstract count relative to 15", f"{ABSTRACT_N / len(union):.2f} x (+{ABSTRACT_N / len(union) - 1:.0%})")
fig4_groups = {g: sum(S3[m] == g for m in FIG4) for g in ("typical", "extended", "wild type")}
line("Fig. 4 durability cohort by group", fig4_groups)
print("\n== Response rate")
r15 = RESULTS_RESPONDERS / RESULTS_N
r18 = RESULTS_RESPONDERS / ABSTRACT_N
line("9/15", f"{r15:.1%}")
line("9/18", f"{r18:.1%}")
line("60% of 15", f"{0.60 * RESULTS_N:.1f}")
line("60% of 18", f"{0.60 * ABSTRACT_N:.1f} (not a whole number)")
for k in range(10, 13):
line(f"{k}/18", f"{k / 18:.1%}")
hits15 = [k for k in range(16) if round(100 * k / 15) == 60]
hits18 = [k for k in range(19) if round(100 * k / 18) == 60]
line("k with k/15 rounding to 60%", hits15)
line("k with k/18 rounding to 60%", hits18 if hits18 else "none")
line("(for contrast) n <= 20 with some k/n exactly 60%",
[n for n in range(1, 21) if any(100 * k == 60 * n for k in range(n + 1))])
print("\n== Primary endpoint (>30% of models with >=30% regression)")
p15 = stats.binom.sf(RESULTS_RESPONDERS - 1, RESULTS_N, ENDPOINT_THRESHOLD)
lb15 = stats.beta.ppf(0.05, RESULTS_RESPONDERS, RESULTS_N - RESULTS_RESPONDERS + 1)
lo15 = stats.beta.ppf(0.025, RESULTS_RESPONDERS, RESULTS_N - RESULTS_RESPONDERS + 1)
hi15 = stats.beta.ppf(0.975, RESULTS_RESPONDERS + 1, RESULTS_N - RESULTS_RESPONDERS)
line("Exact one-sided binomial P(X >= 9 | n=15, p=0.30)", f"{p15:.4f}")
line("Clopper-Pearson one-sided 95% lower bound, 9/15", f"{lb15:.1%}")
line("Clopper-Pearson two-sided 95% CI, 9/15", f"{lo15:.1%} to {hi15:.1%}")
p18 = stats.binom.sf(RESULTS_RESPONDERS - 1, ABSTRACT_N, ENDPOINT_THRESHOLD)
line("(for contrast) P(X >= 9 | n=18, p=0.30)", f"{p18:.4f}")
for k in (10, 11):
lo = stats.beta.ppf(0.025, k, ABSTRACT_N - k + 1)
hi = stats.beta.ppf(0.975, k + 1, ABSTRACT_N - k)
line(f"(for contrast) Clopper-Pearson two-sided 95% CI, {k}/18", f"{lo:.1%} to {hi:.1%}")
# ---------------------------------------------------------------- figure style
ORANGE, BLUE = "#eb6834", "#2a78d6"
INK, INK2, BG, GRID = "#0b0b0b", "#52514e", "#fcfcfb", "#e6e5e1"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "path", "lines.linewidth": 1.5,
"figure.dpi": 200, "text.hinting": "none", "text.hinting_factor": 1,
})
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def runs(ax, x, y, parts, size=10.5, ha_box=None):
"""Draw text runs [(text, color, weight)] left to right starting at axes coords (x, y)."""
fig = ax.figure
r = fig.canvas.get_renderer()
axw = ax.get_window_extent(renderer=r).width
def width(s, weight):
t = ax.text(0, 0, s, fontsize=size, fontweight=weight, transform=ax.transAxes)
w = t.get_window_extent(renderer=r).width
t.remove()
return w / axw
for text, color, weight in parts:
core = text.lstrip(" ")
x += width("|" + text[:len(text) - len(core)] + "|", "normal") - width("||", "normal")
ax.text(x, y, core, color=color, fontsize=size, fontweight=weight,
transform=ax.transAxes, va="baseline", ha="left")
# advance by the run's width, with the bounding-box padding cancelled out
x += width("|" + core + "|", weight) - width("||", weight)
return x
def box(ax, x, y, w, h, edge, lw=1.5, ls="-", fill=BG):
p = FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
transform=ax.transAxes, fc=fill, ec=edge, lw=lw, ls=ls)
ax.add_patch(p)
def blank(h):
fig = plt.figure(figsize=(W, h))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_axis_off()
return fig, ax
# ---------------------------------------------------------------- A: two text panels
def cand_A(path):
fig, ax = blank(3.9)
ax.text(0.03, 0.93, "The abstract's model count disagrees with every count in the body",
fontsize=12, fontweight="bold", transform=ax.transAxes)
# left panel: abstract
box(ax, 0.03, 0.12, 0.38, 0.70, ORANGE)
ax.text(0.05, 0.755, "ABSTRACT", fontsize=10, color=ORANGE, fontweight="bold",
transform=ax.transAxes)
ax.text(0.05, 0.695, "(reported)", fontsize=10, color=ORANGE, transform=ax.transAxes)
runs(ax, 0.05, 0.62, [("“Across ", INK, "normal"), ("18", ORANGE, "bold"),
(" PDX models", INK, "normal")])
runs(ax, 0.05, 0.54, [("undergoing dual inhibition of", INK, "normal")])
runs(ax, 0.05, 0.46, [("MEK and CDK4/6, ", INK, "normal"), ("60%", ORANGE, "bold"),
(" of tumors", INK, "normal")])
runs(ax, 0.05, 0.38, [("regressed…”", INK, "normal")])
ax.text(0.05, 0.20, f"With 18 models: 9/18 = {r18:.0%};\n60% of 18 = {0.6 * ABSTRACT_N:.1f} models",
fontsize=10, color=INK2, transform=ax.transAxes, linespacing=1.4)
# right panel: body
box(ax, 0.59, 0.12, 0.38, 0.70, BLUE)
ax.text(0.61, 0.755, "BODY & SUPPLEMENT", fontsize=10, color=BLUE, fontweight="bold",
transform=ax.transAxes)
ax.text(0.61, 0.695, "(reported / recomputed)", fontsize=10, color=BLUE, transform=ax.transAxes)
runs(ax, 0.61, 0.62, [("“≥30% regression in ", INK, "normal"), ("9 of 15", BLUE, "bold")])
runs(ax, 0.61, 0.54, [("models (60%)”", INK, "normal"), (" Results", INK2, "normal")])
runs(ax, 0.61, 0.45, [("n = 6 + 2 + 3 + 4 = ", INK, "normal"), (f"{sum_text}", BLUE, "bold"),
(" Results", INK2, "normal")])
runs(ax, 0.61, 0.36, [("“evaluated in ", INK, "normal"), ("15", BLUE, "bold"),
(" mCRC PDX", INK, "normal")])
runs(ax, 0.61, 0.28, [("models”", INK, "normal"), (" Fig. 1B legend", INK2, "normal")])
runs(ax, 0.61, 0.19, [("Supp. Tables S1, S3: ", INK, "normal"), (f"{len(S3)}", BLUE, "bold"),
(" rows each", INK, "normal")])
# connector
ax.annotate("", xy=(0.585, 0.47), xytext=(0.415, 0.47), xycoords="axes fraction",
arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.5))
ax.text(0.50, 0.55, "18 ≠ 15", ha="center", fontsize=12, fontweight="bold",
transform=ax.transAxes)
ax.text(0.50, 0.33, "no set of 18\nmodels is\ndescribed; of\nthe two counts,\nonly 9/15 = 60%", ha="center",
va="center", fontsize=10, color=INK2, transform=ax.transAxes, linespacing=1.3)
ax.text(0.03, 0.04, "Excerpts from the PMC-archived article and its supplement; "
"arithmetic from recompute.py.txt.", fontsize=10, color=INK2, transform=ax.transAxes)
save(fig, path)
# ---------------------------------------------------------------- B: count by location dot plot
def cand_B(path):
rows = [
("Abstract: “Across 18 PDX models”", ABSTRACT_N, "abs"),
("Results: subgroups 6 + 2 + 3 + 4", sum_text, "body"),
("Results: “9 of 15 models (60%)”", RESULTS_N, "body"),
("Fig. 1A legend: 6 + 5 + 4", sum_fig1a, "body"),
("Fig. 1B–D legends: “15 … PDX models”", FIG1_LEGEND_N, "body"),
("Fig. 1B: model panels", len(set(FIG1B)), "body"),
("Results: phospho-SHP2 “in 11 of 15”", SHP2_OF, "body"),
("Supp. Table S1: model rows", len(set(S1)), "body"),
("Supp. Table S3: model rows", len(S3), "body"),
("All models named anywhere", len(union), "body"),
]
fig = plt.figure(figsize=(W, 4.6))
ax = fig.add_axes([0.42, 0.12, 0.54, 0.70])
ys = list(range(len(rows)))[::-1]
for (lab, v, kind), y in zip(rows, ys):
c = ORANGE if kind == "abs" else BLUE
ax.plot([0, v], [y, y], color=GRID, lw=1.5, zorder=1)
ax.plot(v, y, "o", ms=9, color=c, mec=BG, mew=1.5, zorder=3)
ax.text(v + 0.5, y, f"{v}", va="center", fontsize=10.5, color=INK)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10.5, color=INK)
ax.set_xlim(0, 20.5)
ax.set_xticks(range(0, 21, 5))
ax.set_xlabel("Number of PDX models")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.axvspan(15, 18, color=ORANGE, alpha=0.08, lw=0)
ax.set_ylim(-0.6, len(rows) - 1 + 1.3)
ax.text(16.5, ys[0] + 0.75, "3 models\nnot found", ha="center", va="center", fontsize=10,
color=INK2, linespacing=1.1)
h1, = ax.plot([], [], "o", ms=9, color=ORANGE, label="Abstract (reported)")
h2, = ax.plot([], [], "o", ms=9, color=BLUE, label="Body and supplement (stated or counted)")
fig.legend(handles=[h1, h2], loc="upper left", bbox_to_anchor=(0.005, 0.93), ncol=2,
frameon=False, fontsize=10)
fig.text(0.01, 0.945, "Model count for the dual MEK + CDK4/6 co-clinical trial, by location",
fontsize=11.5, fontweight="bold")
save(fig, path)
# ---------------------------------------------------------------- C: percentage lattice
def cand_C(path):
fig = plt.figure(figsize=(W, 3.3))
ax = fig.add_axes([0.12, 0.2, 0.84, 0.56])
for n, y in ((15, 1), (18, 0)):
xs = [100 * k / n for k in range(n + 1)]
ax.plot(xs, [y] * len(xs), "o", ms=5, color="#b9b8b3", zorder=2)
ax.plot([0, 100], [y, y], color=GRID, lw=1.5, zorder=1)
ax.axvline(100 * ENDPOINT_THRESHOLD, color=INK2, lw=0.8, ls=(0, (2, 2)), zorder=0)
ax.text(100 * ENDPOINT_THRESHOLD + 0.8, 1.55, "endpoint threshold 30%", fontsize=10, color=INK2)
ax.axvline(ABSTRACT_PCT, color=ORANGE, lw=1.5, zorder=1)
ax.text(ABSTRACT_PCT + 0.8, 1.55, "Abstract and Results: 60%", fontsize=10, color=ORANGE,
fontweight="bold")
ax.plot(100 * r15, 1, "o", ms=10, color=BLUE, mec=BG, mew=1.5, zorder=4)
ax.annotate("9/15 = 60.0%", (100 * r15, 1), xytext=(100 * r15 + 4, 1.28), fontsize=10.5,
color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.plot(100 * r18, 0, "o", ms=10, color=BLUE, mec=BG, mew=1.5, zorder=4)
ax.annotate(f"9/18 = {r18:.1%}\n(same 9 responders)", (100 * r18, 0), xytext=(100 * r18 - 24, -0.62),
fontsize=10, color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
for k in (10, 11):
ax.plot(100 * k / 18, 0, "o", ms=9, mfc=BG, mec=BLUE, mew=1.5, zorder=4)
ax.annotate(f"10/18 = {10 / 18:.1%}", (100 * 10 / 18, 0), xytext=(38, 0.42), fontsize=10,
color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.annotate(f"11/18 = {11 / 18:.1%}\n60% of 18 = 10.8 models",
(100 * 11 / 18, 0), xytext=(66, -0.62), fontsize=10, color=INK,
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.set_yticks([1, 0])
ax.set_yticklabels(["n = 15", "n = 18"], fontsize=10.5, color=INK)
ax.set_ylim(-0.9, 1.85)
ax.set_xlim(0, 100)
ax.set_xticks(range(0, 101, 10))
ax.set_xticklabels([f"{t}%" for t in range(0, 101, 10)])
ax.set_xlabel("Share of models regressing ≥30% (every attainable value k/n shown in gray)")
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
fig.text(0.01, 0.92, "Exactly 60% is attainable with 15 models (9/15), not with 18",
fontsize=11.5, fontweight="bold")
save(fig, path)
# ---------------------------------------------------------------- D: unit chart of models
def cand_D(path):
fig, ax = blank(4.0)
ax.text(0.02, 0.93, "The abstract says 18 models; the paper names 15", fontsize=12,
fontweight="bold", transform=ax.transAxes)
groups = [("Typical RAS\n(KRAS G12/13)", [m for m in S3 if S3[m] == "typical"]),
("Extended RAS\n(NRAS Q61, KRAS A146T)", [m for m in S3 if S3[m] == "extended"]),
("RAS wild type", [m for m in S3 if S3[m] == "wild type"]),
("Abstract's extra\nmodels", [None, None, None])]
bw, bh, gap, x0 = 0.092, 0.13, 0.011, 0.29
for i, (lab, models) in enumerate(groups):
y = 0.70 - i * 0.175
ax.text(0.02, y + bh / 2, lab, va="center", fontsize=10, color=INK if models[0] else ORANGE,
transform=ax.transAxes, linespacing=1.2)
for j, m in enumerate(models):
x = x0 + j * (bw + gap)
if m:
box(ax, x, y, bw, bh, BLUE, fill="#e9f1fb")
ax.text(x + bw / 2, y + bh / 2, m, ha="center", va="center", fontsize=10,
transform=ax.transAxes)
else:
box(ax, x, y, bw, bh, ORANGE, ls=(0, (3, 2)))
ax.text(x + bw / 2, y + bh / 2, "?", ha="center", va="center", fontsize=11,
color=ORANGE, transform=ax.transAxes)
ax.text(x0 + len(models) * (bw + gap) + 0.005, y + bh / 2,
f"{len(models)}" if models[0] else "3 (none named)", va="center", fontsize=10.5,
fontweight="bold", color=BLUE if models[0] else ORANGE, transform=ax.transAxes)
ax.text(0.02, 0.025, f"Blue: models listed in Supp. Table S3 (6 + 5 + 4 = {len(S3)}); "
"the same 15 appear in Supp. Table S1\nand Fig. 1B. Orange dashed: the 3 further models "
"the abstract's “18” implies;\nnone is described in the paper or supplement.",
fontsize=10, color=INK2, transform=ax.transAxes, linespacing=1.35)
save(fig, path)
# ---------------------------------------------------------------- E: calculation flow
def cand_E(path):
fig, ax = blank(3.6)
ax.text(0.02, 0.93, "Following the paper's own counts to the abstract's 60%", fontsize=12,
fontweight="bold", transform=ax.transAxes)
def node(x, y, w, h, txt, edge, bold=False, fill=BG, ls="-"):
box(ax, x, y, w, h, edge, fill=fill, ls=ls)
ax.text(x + w / 2, y + h / 2, txt, ha="center", va="center", fontsize=10,
fontweight="bold" if bold else "normal", transform=ax.transAxes, linespacing=1.3)
def arrow(x1, y1, x2, y2):
ax.annotate("", xy=(x2, y2), xytext=(x1, y1), xycoords="axes fraction",
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.5, mutation_scale=12))
ax.text(0.02, 0.80, "From the Results and Fig. 1 (recomputed)", fontsize=10, color=BLUE,
fontweight="bold", transform=ax.transAxes)
box(ax, 0.02, 0.48, 0.25, 0.26, BLUE)
sub = "\n".join(f"{k:<12}{v:>2}" for k, v in
zip(("KRAS G12/13", "NRAS Q61", "KRAS A146T", "RAS WT"), SUBGROUPS_TEXT.values()))
ax.text(0.145, 0.61, sub, ha="center", va="center", fontsize=10, family="DejaVu Sans Mono",
transform=ax.transAxes, linespacing=1.3)
arrow(0.27, 0.61, 0.32, 0.61)
node(0.32, 0.52, 0.15, 0.18, f"{sum_text} models", BLUE, bold=True)
arrow(0.47, 0.61, 0.52, 0.61)
node(0.52, 0.52, 0.17, 0.18, f"{RESULTS_RESPONDERS} regressed\n≥30%", BLUE)
arrow(0.69, 0.61, 0.74, 0.61)
node(0.74, 0.52, 0.23, 0.18, f"9/15 = {r15:.1%}\nmatches 60%", BLUE, bold=True)
ax.text(0.02, 0.38, "From the abstract (reported)", fontsize=10, color=ORANGE, fontweight="bold",
transform=ax.transAxes)
node(0.32, 0.10, 0.15, 0.18, f"{ABSTRACT_N} models", ORANGE, bold=True)
arrow(0.47, 0.22, 0.52, 0.255)
arrow(0.47, 0.15, 0.52, 0.10)
node(0.52, 0.20, 0.45, 0.11, f"same 9 responders: 9/18 = {r18:.1%}, not 60%", ORANGE)
node(0.52, 0.02, 0.45, 0.15, f"keep 60%: 0.6 × 18 = {0.6 * ABSTRACT_N:.1f} models\n(not a whole number)",
ORANGE)
save(fig, path)
cand_A(os.path.join(CAND, "A_text_panels"))
cand_B(os.path.join(CAND, "B_count_by_location"))
cand_C(os.path.join(CAND, "C_percentage_lattice"))
cand_D(os.path.join(CAND, "D_model_grid"))
cand_E(os.path.join(CAND, "E_calculation_flow"))
# Provisional choice for the write-up, pending review of the candidates:
cand_A(os.path.join(HERE, "fig1"))
cand_C(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE, "and", CAND)
Fisher exact P reported as 0.0005 for 1/6 versus 7/18 remissions; the counts give 0.62
Clin Cancer Res (2012) · doi:10.1158/1078-0432.CCR-12-2442 · PMC3596113
Summary
The Discussion of this phase I trial compares the CR/CRi rate (complete remission plus complete remission with incomplete recovery) between dose groups of SCH 900776: 1 of 6 patients (17%) below 40 mg/m² and 7 of 18 (39%) at 40 mg/m² or above, reported as "P = 0.0005, Fisher exact test". The counts agree with Table 3, Table 4, the Results text and the abstract. A Fisher exact test on them gives a two-sided P of 0.62 (one-sided 0.32), and with 6 and 18 patients and 8 responders no table in the reported direction can reach a P below 0.066. This is the only P value in the paper, and the Discussion uses it as statistical support for a dose-response that would point to a contribution of SCH 900776 itself; the trial's toxicity-based dose-finding conclusions do not depend on it.
What the paper reports
The comparison and its P value (Discussion, paragraph 1):
Indeed, the CR/CRi rate was 1 of 6 (17%) for patients who received <40 mg/m2 SCH 900776 versus 7 of 18 (39%) for those receiving ≥ 40 mg/m2 (P = 0.0005, Fisher exact test).
The sentence immediately before it, which states the claim the P value supports (Discussion, paragraph 1; the PMC text prints "doseresponse", the publisher's HTML "dose-response"):
The notion that part of the responses might relate to adding SCH 900776 to the timed sequential cytarabine infusions is supported by the presence of a doseresponse curve, with clinical responses to SCH 900776 beginning at 40 mg/m2, in conjunction with Cmax exceeding 1,000 ng/mL.
The same counts, without a P value (Results, "Clinical outcome"):
Similarly, the overall response rate (CR plus CRi) was 8 of 24 (33%), with 1 CRi (17%) of 6 patients treated with SCH 900776 doses of ≤ 20 mg/m2 and 7 (39%; 5 CR, 2 CRi) of 18 patients treated at dose level 3 (40 mg/m2) or higher, with at least 2 of 6 (33%) in each of dose levels 3 through 5.
Abstract, Results:
Complete remissions occurred in 8 of 24 (33%) patients, with 7 of 8 at 40 mg/m2 or higher.
Table 3 ("Dose escalation and clinical outcome for 24 adults with relapsed or refractory acute leukemia"), dose-level labels and the "Response (CR/CRi)" column, condensed to one line with the archived typography kept:
- 10 mg/m2 (n = 3): 1 · 2. 20 mg/m2 (n = 3): 0 · 3. 40 (mg/m2 (n = 6): 3 · 4. 56 (mg/m2 (n = 6): 2 · 5. 140 mg (n = 6): 2 · Total: 8
Table 4 lists the 8 responders individually, at dose levels 1, 3, 3, 3, 4, 4, 5 and 5.
The authors' caveats, later in the same paragraph (Discussion, paragraph 1):
we cannot exclude the possibility that some or all of the responders might have responded to timed sequential cytarabine without SCH 900776
Moreover, the small sample size and heterogeneous nature of this patient population preclude any ability to draw accurate conclusions about additive or synergistic clinical effects of adding SCH 900776 to cytarabine
The statistics subsection of the Methods names no hypothesis test (Methods, "Statistical considerations"):
The primary objective was to determine the combination-recommended phase II dose (RP2D) by assessing the safety, tolerability, DLT, and antileukemic activity (CR rate) of the combination of cytarabine and SCH 900776 in adult patients with relapsed and/or refractory acute leukemias.
What the paper's own numbers show
Method. I rebuilt the 2×2 table from Table 3. Dose levels 1–2 (10 and 20 mg/m², 3 patients each) form the group below 40 mg/m²: 1 + 0 = 1 responder of 6. Dose levels 3–5 (40 mg/m², 56 mg/m² and a flat 140 mg that the paper states is equivalent to 80 mg/m²; 6 patients each) form the group at or above 40 mg/m²: 3 + 2 + 2 = 7 responders of 18. Table 4's responder dose levels and the abstract's "7 of 8" give the same split. The table is therefore [[1 responder, 5 non-responders], [7, 11]]. I ran Fisher's exact test with scipy and cross-checked it by enumerating the hypergeometric distribution directly. I also ran the usual alternatives for a 2×2 table, two tests across all five dose levels, and enumerated every table with the same margins (6 vs 18 patients, 8 responders). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| CR/CRi below 40 mg/m² | Discussion ¶1; Results; Table 3 | 1 of 6 (17%) | 1/6 = 16.7% (Table 3: 1 + 0 of 3 + 3) |
| CR/CRi at or above 40 mg/m² | Discussion ¶1; Results; Table 3; abstract | 7 of 18 (39%); "7 of 8" | 7/18 = 38.9% (Table 3: 3 + 2 + 2 of 6 + 6 + 6); Table 4: 7 of 8 responders at level ≥ 3 |
| Fisher exact test, below vs at or above 40 mg/m² | Discussion ¶1 | P = 0.0005 | two-sided P = 0.6214, 1,243 times larger |
| Fisher exact, one-sided (fewer responses below 40 mg/m²) | – | – | P = 0.3191 |
| Odds ratio, ≥ 40 vs < 40 mg/m² | – | – | sample 3.18; conditional MLE 3.05, exact 95% CI 0.25 to 171.6 |
| Exact 95% CIs for the two response rates | – | – | 16.7% (0.4% to 64.1%) and 38.9% (17.3% to 64.3%) |
| Pearson chi-square / Yates-corrected chi-square | – | – | P = 0.3173 / 0.6171 |
| Barnard / Boschloo exact unconditional tests | – | – | P = 0.4090 / 0.3761 |
| Cochran-Armitage trend over the 5 dose levels | – | – | P = 0.7423 (scores 1–5); 0.7159 (mg/m²); 0.6541 (log dose) |
| Fisher-Freeman-Halton exact test, 2×5 dose levels | – | – | P = 0.8265 |
| Smallest two-sided Fisher P in the reported direction, same margins | – | – | P = 0.0664, at 0/6 vs 8/18 |
| Tables with these margins giving P ≤ 0.0005 | – | – | only 6/6 vs 2/18 (P = 0.00021), the reverse of the reported pattern |
| Responders needed at ≥ 40 mg/m² for P ≤ 0.0005 if the low-dose group stays at 1/6 | – | – | 18 of 18 (P = 0.00014) |
The arithmetic of the counts and percentages is correct; only the P value is not. It is 1,243 times smaller than the two-sided Fisher P for the stated counts and lies below the smallest value that any table with these margins can produce in the reported direction (0.066). With 8 responders among 24 patients, 2.0 responders would be expected in the low-dose group if response did not depend on dose; 1 was observed.
Figure
Figure 1. P values for CR/CRi below vs at or above 40 mg/m² (1/6 vs 7/18, Table 3) on one logarithmic axis. The orange point is the value printed in the Discussion (P = 0.0005, Fisher exact test). The blue points are recomputed from the paper's own counts: the stated test (Fisher exact, two-sided, and one-sided in the reported direction), common alternatives for a 2×2 table (Pearson and Yates chi-square, Barnard and Boschloo exact tests), and two tests across the five dose levels (Cochran-Armitage trend, Fisher-Freeman-Halton). The dashed line marks P = 0.05 for reference. The shaded region (P < 0.066) contains P values that no 2×2 table with the paper's margins can produce in the reported direction. All recomputed values lie between 0.32 and 0.83.
Figure 2. With the margins fixed by the paper (6 patients below 40 mg/m², 18 at or above, 8 responders), only seven 2×2 tables are possible. Each is labelled on the horizontal axis by its responder counts, below 40 mg/m² on the top line and at or above 40 mg/m² on the bottom line. Blue markers give the two-sided Fisher P for each table; the filled marker is the observed table (P = 0.62). The orange line is the reported P = 0.0005. Only the table in which all 6 low-dose patients respond, the opposite of the reported pattern, falls below it. The most extreme table in the reported direction (0/6 vs 8/18) gives P = 0.066.
Why it matters
The P value is the only formal statistical result in the paper. The Discussion presents it as evidence for a dose-response, and uses the dose-response to support "the notion that part of the responses might relate to adding SCH 900776" to cytarabine. As printed, it reads as strong evidence that response rates differ between the dose groups. Computed from the paper's own counts, the two-group comparison gives P = 0.62, the trend test across all five dose levels gives P = 0.65 to 0.74 depending on scoring, and the exact 95% CI for the odds ratio runs from 0.25 to 172. With 6 patients in the lower-dose group, the data are consistent both with no dose effect and with a large one; they do not establish a dose-response either way.
The error is limited in scope. It is not in the title, the abstract or any table. The abstract gives only correct counts ("8 of 24 (33%)", "7 of 8 at 40 mg/m2 or higher"), although its emphasis on the 7 of 8 carries the same dose-response reading, now without statistical support. The phase I conclusions (feasibility and tolerability, the dose-limiting toxicities at 140 mg, the recommended phase II dose of 100 mg, about 56 mg/m², the pharmacokinetics and the H2Ax observations) rest on other data and are unaffected. The raw counts are printed next to the P value, and the authors state in the same paragraph that the small, heterogeneous sample precludes firm conclusions about added effect. Even so, a reader who takes the P value at face value, for example when judging whether the dose-response signal justifies a randomized trial, or when citing the paper as evidence of activity attributable to the Chk1 inhibitor, would overestimate that signal considerably.
Three smaller inconsistencies, separate from the P value and not affecting any conclusion, turned up during the check:
- Results, "Clinical outcome": day-14 complete tumor clearance at the two lowest doses is given as "2 of 6 (17%)"; 2/6 is 33% (Table 3: 1 + 1 of 3 + 3).
- The CR/CRi split differs between tables. Table 3 and the Results text give 5 CR and 3 CRi, with "3 (2/1)" at dose level 3. Table 4 lists three CRs at dose level 3 and 6 CR and 2 CRi overall. The combined CR/CRi counts, which the P value concerns, are the same in both.
- Discussion, paragraph 5: "2 of 10 (20%) patients with a complex karyotype achieved CR/CRi versus 5 of 14 (36%)" covers all 24 patients but sums to 7 of the 8 responders.
Explanations considered
- A different test, or a one-sided test. Fisher one-sided gives 0.32 and two-sided 0.62; Pearson chi-square 0.32 and Yates-corrected 0.62; Barnard 0.41 and Boschloo 0.38; the Cochran-Armitage trend test 0.65 to 0.74 and the 2×5 Freeman-Halton test 0.83. None comes within two orders of magnitude of 0.0005, and the Methods describe no other analysis. Does not account for the value.
- Different counts, grouping or cut-point. The 1/6 vs 7/18 split is confirmed independently by Table 3, Table 4, the Results text and the abstract, so the 2×2 table is fixed. Placing the 140 mg flat dose anywhere at or above 40 mg/m² does not change it. Other cut-points on the same data (below 20, 56 or 80 mg/m²) each give a Fisher P of 1.00. Keeping 1/6 in the low-dose group, the high-dose group would need 18 of 18 responders to reach P ≤ 0.0005. Does not account for the value.
- The P value belongs to another comparison. The nearest is CR/CRi in patients with day-14 tumor clearance against those with residual leukemia (8 of 12 vs 0 of 12, from the Results text): two-sided P = 0.0013, one-sided 0.00067. That is the same order of magnitude but not the same number, the paper attaches no P value to that comparison, and the sentence clearly refers to the dose groups. Other comparisons in the paper give larger values: day-14 clearance by dose group (2/6 vs 10/18) 0.64, AML vs other leukemias (8/21 vs 0/3) 0.53, complex karyotype vs not (2/10 vs 5/14) 0.65. A one-sample test of 7/18 against an assumed historical response rate gives 0.00002 (against 5%) or 0.0012 (against 10%); neither matches, and the sentence describes a two-group Fisher test. Does not account for the value; the source of 0.0005 cannot be identified from the paper.
- A typographical slip, such as misplaced digits. Cannot be excluded, but neither "0.05" nor "0.5" matches the correct two-sided value of 0.62, so a simple digit shift does not reconcile the numbers. Does not reconcile the numbers.
- An error introduced in the archived manuscript. The PMC copy is the NIH author manuscript. An archived copy of the publisher's full-text page (see Status) contains the identical sentence, "(P = 0.0005, Fisher exact test)", together with the same Results counts and abstract. The value is plain text with no superscripts or special characters that a conversion could alter. Ruled out for the sentence.
- Patient-level or supplementary data that would change the table. The paper has no supplementary material. Table 3's per-level counts and Table 4's per-patient dose levels fully determine the 2×2 table. Ruled out.
Status
As of 2026-10-23:
- Version of record. The live publisher site (aacrjournals.org, reached through the doi.org redirect) returned HTTP 403 and a browser-verification page to automated access, as did the publisher PDF link listed in Crossref. I therefore checked an Internet Archive capture of the publisher's full-text HTML (clincancerres.aacrjournals.org/content/18/24/6723.full, captured 2021-10-19; citation metadata Clin Cancer Res 18(24):6723–6731). It contains the Discussion sentence word for word, including "(P = 0.0005, Fisher exact test)", the same Results paragraph and the same abstract. Table 3 is not rendered inline in that capture, so the table itself was checked only in the PMC copy, where it agrees with the Results text and Table 4. The PMC copy (NIHMS author manuscript) matches the publisher text in every passage compared, apart from the hyphen in "dose-response".
- Supplements. None. The PMC record has no supplementary material, Europe PMC lists none, and the archived publisher page links none.
- Corrections, retractions, expressions of concern. None found. The repository's
known.pycheck returned no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to, updated-by or relation entries, and the PubMed record has no comment/correction links. - PubPeer. Could not be checked directly: both the PubPeer search page and its browser-extension API returned HTTP 403 / Cloudflare verification to automated access. Earlier web searches for the DOI and title with "PubPeer", "erratum" and "correction" found no thread or notice; a fresh web search could not be run for this write-up.
- Sources. PMC3596113 (as archived in PubMed Central); doi:10.1158/1078-0432.CCR-12-2442; Internet Archive capture of the publisher's full-text page dated 2021-10-19; Crossref and Europe PMC records; ClinicalTrials.gov NCT00907517 (no per-patient results that bear on these counts).
Suggested correction
The sentence in Discussion paragraph 1 could read: "Indeed, the CR/CRi rate was 1 of 6 (17%) for patients who received <40 mg/m2 SCH 900776 versus 7 of 18 (39%) for those receiving ≥ 40 mg/m2 (P = 0.62, two-sided Fisher exact test)." Because neither this comparison nor a trend test across the five dose levels (P ≈ 0.7) supports a dose-response at this sample size, the preceding sentence's statement that the responses are "supported by the presence of a dose-response curve" could be qualified as a descriptive observation. If the reported value came from a different comparison, the correction should say which one. Separately, "2 of 6 (17%)" for day-14 tumor clearance at ≤ 20 mg/m² should read "2 of 6 (33%)", and the CR/CRi breakdown at dose level 3 and the karyotype counts could be reconciled between Table 3, Table 4 and the Discussion.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC3596113 write-up and regenerate its figures.
Paper: "Phase I and Pharmacologic Trial of Cytosine Arabinoside with the Selective
Checkpoint 1 Inhibitor Sch 900776 in Refractory Acute Leukemias",
Clin Cancer Res 2012;18(24):6723-31, doi:10.1158/1078-0432.CCR-12-2442 (PMC3596113).
All inputs are copied from Table 3, Table 4 and the Results/Discussion text.
Run: python recompute.py.txt (needs numpy, scipy >= 1.7, matplotlib)
Writes fig1.svg/png and fig2.svg/png next to this file.
"""
import os
from math import comb
from itertools import product
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
# Table 3, per dose level: (level, dose in mg/m2, n, day-14 CTC, CR/CRi, CR, CRi).
# Level 5 was a flat 140 mg, which the paper states is "equivalent to 80 mg/m2".
TABLE3 = [
(1, 10, 3, 1, 1, 0, 1),
(2, 20, 3, 1, 0, 0, 0),
(3, 40, 6, 4, 3, 2, 1),
(4, 56, 6, 3, 2, 2, 0),
(5, 80, 6, 3, 2, 1, 1),
]
# Table 4, one row per responder: (dose level, response)
TABLE4 = [(1, "CRi"), (3, "CR"), (3, "CR"), (3, "CR"), (4, "CR"), (4, "CR"), (5, "CRi"), (5, "CR")]
REPORTED_P = 0.0005 # Discussion, paragraph 1: "(P = 0.0005, Fisher exact test)"
lev = np.array([r[0] for r in TABLE3]); dose = np.array([r[1] for r in TABLE3], float)
ns = np.array([r[2] for r in TABLE3]); ctc = np.array([r[3] for r in TABLE3])
rs = np.array([r[4] for r in TABLE3])
cr3 = sum(r[5] for r in TABLE3); cri3 = sum(r[6] for r in TABLE3)
def fmt(p):
return f"{p:.4f}" if p >= 1e-3 else f"{p:.5f}"
def section(t):
print("\n" + "=" * 72 + "\n" + t + "\n" + "=" * 72)
# ------------------------------------------------- 1. counts
section("1. Counts from Table 3 (checked against Table 4 and the text)")
N, R = int(ns.sum()), int(rs.sum())
lo = dose < 40
lo_n, lo_r = int(ns[lo].sum()), int(rs[lo].sum())
hi_n, hi_r = int(ns[~lo].sum()), int(rs[~lo].sum())
print(f"Total CR/CRi: {R}/{N} = {R/N:.1%}")
print(f"<40 mg/m2 (levels 1-2): {lo_r}/{lo_n} = {lo_r/lo_n:.1%} (paper: '1 of 6 (17%)')")
print(f">=40 mg/m2 (levels 3-5): {hi_r}/{hi_n} = {hi_r/hi_n:.1%} (paper: '7 of 18 (39%)')")
t4_levels = [lv for lv, _ in TABLE4]
print(f"Table 4 responder dose levels: {t4_levels}; below level 3: {sum(l < 3 for l in t4_levels)}, "
f"level 3 or above: {sum(l >= 3 for l in t4_levels)} (abstract: '7 of 8 at 40 mg/m2 or higher')")
tab = np.array([[lo_r, lo_n - lo_r], [hi_r, hi_n - hi_r]])
print("2x2 table [responders, non-responders], rows <40 / >=40:", tab.tolist())
# ------------------------------------------------- 2. the stated test
section("2. Fisher exact test of <40 vs >=40 mg/m2 (the test the paper names)")
or_lo_vs_hi, p2 = stats.fisher_exact(tab, alternative="two-sided")
p_less = stats.fisher_exact(tab, alternative="less")[1]
p_greater = stats.fisher_exact(tab, alternative="greater")[1]
print(f"Sample odds ratio (<40 vs >=40) = {or_lo_vs_hi:.3f}; (>=40 vs <40) = {1/or_lo_vs_hi:.2f}")
print(f"Two-sided P = {fmt(p2)}")
print(f"One-sided P, fewer responses below 40 mg/m2 (the reported direction) = {fmt(p_less)}")
print(f"One-sided P, opposite direction = {fmt(p_greater)}")
print(f"Reported P / recomputed two-sided P: {REPORTED_P} vs {p2:.4f} -> recomputed is {p2/REPORTED_P:,.0f} times larger")
# manual hypergeometric enumeration as a cross-check of scipy
pmf = {x: comb(R, x) * comb(N - R, lo_n - x) / comb(N, lo_n) for x in range(0, min(R, lo_n) + 1)}
p2_manual = sum(v for v in pmf.values() if v <= pmf[lo_r] * (1 + 1e-7))
print("Hypergeometric P(X = x responders below 40 mg/m2):", {k: round(v, 4) for k, v in pmf.items()})
print(f"Manual two-sided P = {fmt(p2_manual)}; manual P(X <= {lo_r}) = {fmt(sum(pmf[x] for x in range(lo_r + 1)))}")
ci_or = stats.contingency.odds_ratio(tab[::-1], kind="conditional")
lo_ci, hi_ci = ci_or.confidence_interval(0.95)
print(f"Conditional MLE odds ratio (>=40 vs <40) = {ci_or.statistic:.2f}, exact 95% CI {lo_ci:.2f} to {hi_ci:.1f}")
for lab, r, n in (("<40", lo_r, lo_n), (">=40", hi_r, hi_n)):
ci = stats.binomtest(r, n).proportion_ci(0.95, method="exact")
print(f"Clopper-Pearson 95% CI for CR/CRi rate {lab}: {r}/{n} = {r/n:.1%} ({ci.low:.1%} to {ci.high:.1%})")
# ------------------------------------------------- 3. other tests
section("3. Other common tests on the same 2x2 table")
results = {
"Fisher exact, two-sided\n(the stated test)": p2,
"Fisher exact, one-sided\n(reported direction)": p_less,
"Pearson chi-square": stats.chi2_contingency(tab, correction=False)[1],
"Chi-square, Yates": stats.chi2_contingency(tab, correction=True)[1],
"Barnard exact": stats.barnard_exact(tab).pvalue,
"Boschloo exact": stats.boschloo_exact(tab).pvalue,
}
for k in ("Pearson chi-square", "Chi-square, Yates", "Barnard exact", "Boschloo exact"):
print(f"{k}: P = {fmt(results[k])}")
# ------------------------------------------------- 4. trend / all levels
section("4. Tests across the five dose levels")
def ca_trend(ns, rs, s):
s = np.asarray(s, float); n_ = ns.sum(); pbar = rs.sum() / n_
T = np.sum(s * (rs - ns * pbar))
V = pbar * (1 - pbar) * (np.sum(ns * s**2) - np.sum(ns * s) ** 2 / n_)
z = T / np.sqrt(V)
return z, 2 * stats.norm.sf(abs(z))
ca = {}
for name, sc in (("level 1-5", lev), ("dose mg/m2", dose), ("log dose", np.log(dose))):
z, p = ca_trend(ns, rs, sc); ca[name] = p
print(f"Cochran-Armitage, scores = {name:10s}: z = {z:.3f}, two-sided P = {fmt(p)}")
results["Cochran-Armitage trend,\n5 dose levels"] = ca["level 1-5"]
def prob(rr):
return np.prod([comb(int(n_), int(r_)) for r_, n_ in zip(rr, ns)]) / comb(N, R)
p_obs = prob(rs)
ffh = sum(prob(rr) for rr in product(*[range(int(n_) + 1) for n_ in ns])
if sum(rr) == R and prob(rr) <= p_obs * (1 + 1e-9))
print(f"Fisher-Freeman-Halton exact test, 2x5 table: P = {fmt(ffh)}")
results["Freeman-Halton exact,\n2x5 dose levels"] = ffh
# ------------------------------------------------- 5. attainable P values
section("5. Every 2x2 table with the paper's margins (6 vs 18 patients, 8 responders)")
fisher_by_x = {}
for x in range(0, lo_n + 1):
t = [[x, lo_n - x], [R - x, hi_n - (R - x)]]
fisher_by_x[x] = stats.fisher_exact(t)[1]
print(f" {x}/6 below 40 vs {R-x}/18 at or above: two-sided Fisher P = {fmt(fisher_by_x[x])}")
expected_lo = R * lo_n / N
print(f"Expected responders below 40 mg/m2 under no difference: {expected_lo:.1f}")
p_min_dir = min(fisher_by_x[x] for x in range(0, lo_r + 1))
print(f"Smallest P in the reported direction (fewer responders below 40 mg/m2 than expected): {fmt(p_min_dir)} (at 0/6 vs 8/18)")
print(f"Smallest P for any table: {fmt(min(fisher_by_x.values()))} at 6/6 vs 2/18 (the reverse pattern)")
print(f"Tables with P <= 0.0005: {[f'{x}/6 vs {R-x}/18' for x, p in fisher_by_x.items() if p <= REPORTED_P]}")
print("\nKeeping 1/6 below 40 mg/m2 and 18 patients at or above (margins free):")
for k in range(hi_r, hi_n + 1):
p = stats.fisher_exact([[1, 5], [k, hi_n - k]])[1]
if p <= REPORTED_P:
print(f" first k with P <= 0.0005: {k}/18 ({k/18:.0%}), P = {fmt(p)}")
break
print("Keeping 7/18 at or above 40 mg/m2: P for 0/6 below 40 =", fmt(stats.fisher_exact([[0, 6], [7, 11]])[1]))
# ------------------------------------------------- 6. alternative cut-points
section("6. Other dose cut-points (two-sided Fisher)")
for cut in (20, 56, 80):
m = dose < cut
t = [[int(rs[m].sum()), int(ns[m].sum() - rs[m].sum())], [int(rs[~m].sum()), int(ns[~m].sum() - rs[~m].sum())]]
print(f" <{cut} vs >={cut} mg/m2: {t[0][0]}/{sum(t[0])} vs {t[1][0]}/{sum(t[1])}: P = {fmt(stats.fisher_exact(t)[1])}")
# ------------------------------------------------- 7. other comparisons in the paper
section("7. Other comparisons in the paper (could 0.0005 belong to one of them?)")
other = {
"CR/CRi with day-14 CTC (8/12) vs residual leukemia (0/12)": [[8, 4], [0, 12]],
"Day-14 CTC <40 (2/6) vs >=40 (10/18)": [[2, 4], [10, 8]],
"CR/CRi AML (8/21) vs other leukemias (0/3)": [[8, 13], [0, 3]],
"CR/CRi complex karyotype (2/10) vs not (5/14)": [[2, 8], [5, 9]],
}
for k, t in other.items():
p_two = stats.fisher_exact(t)[1]
p_one = min(stats.fisher_exact(t, alternative=a)[1] for a in ("less", "greater"))
print(f" {k}: two-sided P = {fmt(p_two)}, smaller one-sided P = {fmt(p_one)}")
for p0 in (0.05, 0.10):
print(f" Binomial test of 7/18 against a {p0:.0%} response rate: one-sided P = "
f"{fmt(stats.binomtest(7, 18, p0, alternative='greater').pvalue)}")
# ------------------------------------------------- 8. side observations
section("8. Side observations (minor, separate from the P value)")
c = int(ctc[lo].sum())
print(f"Day-14 CTC at <=20 mg/m2: {c}/{lo_n} = {c/lo_n:.1%} (Results text: '2 of 6 (17%)')")
cr4 = sum(r == "CR" for _, r in TABLE4); cri4 = sum(r == "CRi" for _, r in TABLE4)
cr4_l3 = sum(1 for lv, r in TABLE4 if lv == 3 and r == "CR")
print(f"CR / CRi split: Table 3 total {cr3}/{cri3} (Results text: 1 CRi + 5 CR, 2 CRi = 5/3); "
f"Table 4 rows {cr4}/{cri4}; level 3: Table 3 '3 (2/1)', Table 4 {cr4_l3} CR")
print(f"Complex karyotype: 2 of 10 + 5 of 14 = {2+5} responders among {10+14} patients (total responders {R}); "
f"2/10 = {2/10:.0%}, 5/14 = {5/14:.0%}")
print(f"Total day-14 CTC: {ctc.sum()}/{N} = {ctc.sum()/N:.0%}")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e6e5e1"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"axes.edgecolor": "#bdbcb7", "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 1.5,
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
def plabel(p):
return f"P = {p:.4f}" if p < 0.01 else (f"P = {p:.3f}" if p < 0.1 else f"P = {p:.2f}")
# Figure 1: reported P vs P values recomputed from the paper's own counts, one log axis
rows = [("Reported in the paper\n(Fisher exact test)", REPORTED_P, ORANGE)]
rows += [(k, v, BLUE) for k, v in results.items()]
fig, ax = plt.subplots(figsize=(7.2, 5.0))
ys = np.arange(len(rows))[::-1].astype(float)
ax.axvspan(1e-4, p_min_dir, color="#efeeea", zorder=0, lw=0)
ax.text(1.2e-4, ys[0] + 0.95, f"Shaded: P values no table with these margins\ncan reach in the reported direction (P < {p_min_dir:.3f})",
fontsize=10, color=INK2, va="center", ha="left")
ax.axvline(0.05, color=INK2, lw=1, ls=(0, (3, 3)), zorder=1)
ax.text(0.05 * 0.93, ys[-1] - 0.62, "P = 0.05", ha="right", va="center", color=INK2, fontsize=10)
for y, (lab, p, col) in zip(ys, rows):
ax.plot([p], [y], "o", ms=9, color=col, mec=BG, mew=1.2, zorder=3)
if p < 0.01:
ax.text(p * 1.4, y, plabel(p), va="center", ha="left", fontsize=10)
else:
ax.text(p * 0.8, y, plabel(p), va="center", ha="right", fontsize=10)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xscale("log"); ax.set_xlim(1e-4, 1.6)
ax.set_ylim(ys[-1] - 1.0, ys[0] + 1.6)
ax.grid(axis="x", which="major", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("P value (log scale)")
ax.set_title("CR/CRi below vs at or above 40 mg/m²: 1/6 vs 7/18 (Table 3)", loc="left", fontsize=11.5)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported (Discussion)"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE, label="Recomputed from Table 3 counts"),
],
loc="upper center", bbox_to_anchor=(0.3, -0.13), ncol=2, frameon=False, fontsize=10,
handletextpad=0.4, columnspacing=1.2)
fig.tight_layout()
save(fig, "fig1")
# Figure 2: two-sided Fisher P for every table with the paper's margins
fig, ax = plt.subplots(figsize=(7.2, 4.3))
xs = np.array(sorted(fisher_by_x)); ps = np.array([fisher_by_x[x] for x in xs])
for x, p in zip(xs, ps):
obs = (x == lo_r)
ax.plot([x], [p], "o", ms=11 if obs else 8, color=BLUE if obs else BG, mec=BLUE, mew=2, zorder=3)
ax.axhline(REPORTED_P, color=ORANGE, lw=2, zorder=2)
ax.text(2.0, REPORTED_P * 1.3, "Reported P = 0.0005", color=INK, ha="left", va="bottom", fontsize=10)
ax.annotate(f"Observed: 1/6 vs 7/18\n{plabel(fisher_by_x[lo_r])}", xy=(lo_r, fisher_by_x[lo_r]),
xytext=(1.45, 0.07), fontsize=10, arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.annotate(f"Most extreme table in the\nreported direction: 0/6 vs 8/18\n{plabel(fisher_by_x[0])}",
xy=(0, fisher_by_x[0]), xytext=(0.1, 0.0028), fontsize=10,
arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.annotate(f"All 6 low-dose patients respond\n(reverse pattern), {plabel(fisher_by_x[6])}",
xy=(6, fisher_by_x[6]), xytext=(2.9, 0.00018), fontsize=10, va="center",
arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.set_yscale("log"); ax.set_ylim(1e-4, 2)
ax.set_xlim(-0.4, 6.4); ax.set_xticks(xs)
ax.set_xticklabels([f"{x}/6\n{R-x}/18" for x in xs], fontsize=10)
ax.set_xlabel("CR/CRi below 40 mg/m² (top line) and at or above 40 mg/m² (bottom line)")
ax.set_ylabel("Two-sided Fisher P (log scale)")
ax.grid(axis="y", which="major", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.set_title("Every possible split of 8 responders between 6 and 18 patients", loc="left", fontsize=11.5)
ax.legend(handles=[Line2D([], [], color=ORANGE, lw=2, label="Reported in the paper"),
Line2D([], [], marker="o", ls="", ms=10, color=BLUE, label="Recomputed, observed table"),
Line2D([], [], marker="o", ls="", ms=8, mfc=BG, mec=BLUE, mew=2, label="Recomputed, other tables")],
loc="upper right", frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")
print("\nFigures written:", ", ".join(f"fig{i}.{e}" for i in (1, 2) for e in ("svg", "png")))
Hypertension–knowledge p = 0.007 cannot be reproduced from Table 1; its summary statistics give p ≈ 0.59
Diabetologia (2024) · doi:10.1007/s00125-024-06304-3 · PMC11732930
Summary
The paper surveys 1050 Jordanian adults. In its Abstract, Results and Table 1 it reports that diabetes knowledge scores differ significantly by hypertension status: 46.46 ± 13.11 with hypertension (n = 72) against 47.31 ± 12.87 without (n = 978), p = 0.007. The paper's own means, SDs and group sizes do not give that result. Student's t test, which the Table 1 footer names, gives t = −0.540 and p = 0.589 (Welch p = 0.596), with a 95% CI for the difference of −3.94 to 2.24 points. The companion behaviour comparison (p = 0.017) recomputes to p = 0.794. Both are printed as significant associations. The recomputed values are clearly non-significant, so the unadjusted link between hypertension and diabetes knowledge that the Abstract lists has no support in the paper's own numbers.
What the paper reports
Significant associations were found between diabetes knowledge and the following factors: […] hypertension (46.46 ± 13.11 for present vs 47.31 ± 12.87 for absent; p=0.007)
— Abstract, Results. The bracket marks four omitted factors (age, sex, marital status, smoking status). The sentence ends with diabetes status.
Behaviour scores correlated similarly with these factors except for diabetes and smoking status.
— Abstract, Results (this sentence presents the hypertension–behaviour association as significant as well)
Non-hypertensive individuals had higher knowledge scores (47.31 ± 12.87) than those with hypertension (46.46 ± 13.11, p=0.007). Behaviour scores were also higher in non-hypertensive individuals (85.10 ± 19.90) compared with hypertensive individuals (84.46 ± 21.75, p=0.017).
— Results, second paragraph
Yes | 72 (6.86) | 46.46 ± 13.11 | 0.007 | 84.46 ± 21.75 | 0.017 No | 978 (93.14) | 47.31 ± 12.87 | | 85.10 ± 19.90 |
— Table 1, "Distribution of diabetes knowledge and behaviour scores across demographic and clinical categories", Hypertension rows. The columns are n (%), knowledge score, p value, behaviour score, p value.
based on Student’s t tests for variables with two categories and one-way ANOVA for variables with three or more categories
— Table 1 footer. The footer also says the scores are "presented as means ± SD".
Differences in knowledge and behaviour scores across categories were tested using unpaired t test for binary factors (e.g. sex or marital status) and one-way ANOVA for multi-level factors (e.g. age or BMI), with statistical significance set at p<0.05.
— Methods, "Statistical analysis"
Variables that demonstrated significant associations (p<0.05) in preliminary t tests and ANOVA were included in the regression models.
— Methods, "Statistical analysis"
Two later passages describe hypertension's adjusted effect. It points the opposite way from the unadjusted means:
Hypertension status had a significant positive coefficient (4.5346, p<0.0001), indicating higher adjusted knowledge scores among individuals with hypertension.
— Results, third paragraph (Table 2: "Hypertension | 4.5346 | 0.558 | <0.0001")
Participants with hypertension exhibited higher diabetes knowledge and behaviour scores, possibly reflecting a broader understanding of chronic disease management.
— Discussion, third paragraph
What the paper's own numbers show
Method. An unpaired two-sample t test needs only the group sizes, means and SDs, and all of them are printed in Table 1. Student's pooled-variance test (df = n₁ + n₂ − 2 = 1048) is the test that Table 1's footer names. Welch's unequal-variance test and a z test serve as sensitivity checks. For each printed p, the t statistic it requires is found from the inverse t distribution. From that follow the mean difference, the standard error and the 95% interval that the p value implies. To test whether the method recovers the paper's own reporting, the same calculation was applied to every comparison in Table 1: 20 two-group t tests, and 6 one-way ANOVAs computed from group n, mean and SD. Every number below is printed by recompute.py.txt (Python, numpy/scipy).
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Knowledge, mean difference (hypertension yes − no) | Abstract; Results; Table 1 | 46.46 vs 47.31 | −0.85 points |
| Knowledge, Student's t test (pooled SD 12.89, SE 1.574, df 1048) | Abstract; Results; Table 1 | p = 0.007 | t = −0.540, two-sided p = 0.589 |
| Knowledge, Welch t test (SE √(2.387 + 0.169) = 1.599, df 81.4) | sensitivity | — | t = −0.532, p = 0.596 |
| Knowledge, z test / one-sided Student p | sensitivity | — | 0.595 / 0.295 |
| Knowledge, 95% CI for the difference | derived | — | −3.94 to 2.24 points |
| Knowledge, |t| needed for p = 0.007 (df 1048) | derived | p = 0.007 | 2.702 |
| Knowledge, difference needed for p = 0.007 / p = 0.05 with the printed SDs and n | derived | 0.85 | 4.25 / 3.09 points |
| Knowledge, SE implied by a 0.85 difference with p = 0.007 | derived | — | 0.315, one-fifth of the SE the printed SDs give |
| Knowledge, common SD that p = 0.007 would need with n = 72 and 978 | derived | 13.11 and 12.87 | 2.58 |
| Knowledge, smallest p with each mean and SD moved anywhere within its rounding interval | derived | p = 0.007 | 0.585 |
| Knowledge, largest |t| reachable with 978 controls, however large the hypertensive group | derived | — | 2.07 (p = 0.039), still short of 2.70 |
| Knowledge, equal group size needed for p ≤ 0.007 | derived | 72 and 978 | 3400 per group (study total 1050) |
| Behaviour, mean difference (yes − no) | Results; Table 1 | 84.46 vs 85.10 | −0.64 points |
| Behaviour, Student's t test (SE 2.446) | Results; Table 1 | p = 0.017 | t = −0.262, p = 0.794 |
| Behaviour, Welch t test (SE 2.641, df 80.0) | sensitivity | — | t = −0.242, p = 0.809 |
| Behaviour, 95% CI for the difference | derived | — | −5.44 to 4.16 points |
| Behaviour, difference needed for p = 0.017 | derived | 0.64 | 5.85 points |
| Control: smoking, knowledge (45.78 vs 48.22, n = 452 / 598) | Abstract; Results; Table 1 | p = 0.003 | t = −3.10, p = 0.0020 |
| Control: sex, marital status (knowledge), age (both scores) | Abstract; Results; Table 1 | p < 0.0001 | 2.0 × 10⁻⁵ to 8.5 × 10⁻¹² |
| Control: education, employment and diabetes status (behaviour) | Table 1 | 0.913, 0.429, 0.294 | 0.91, 0.43, 0.29 |
| All 26 comparisons in Table 1 | Table 1 | — | 14 reproduce; 12 do not; 3 of the 12 cross p = 0.05: hypertension knowledge, hypertension behaviour, physical-activity behaviour |
| Hypertension coefficient, adjusted model | Table 2 / Table 3 | +4.5346 (SE 0.558) / +0.5410 (SE 0.133), p < 0.0001 | Opposite sign to the unadjusted −0.85 / −0.64; a separate analysis, not a source of 0.007 or 0.017 |
"Reproduce" means that a printed "<" bound holds, or that the recomputed p lies within 10% of the printed value or within one unit of its last printed digit. Many comparisons reproduce closely, among them the knowledge comparisons for four of the five other factors that the Abstract lists next to hypertension (age, sex, marital status, smoking). The fifth, diabetes status, does not reproduce: its knowledge comparison recomputes to p = 0.0052 (Welch 0.0084) against a printed "<0.001", although it remains significant at p < 0.05. So the paper's summary statistics and the stated test do recover many of its own p values. The hypertension rows are among those that do not, and they are two of the three where the recomputed value lands on the other side of the paper's p < 0.05 threshold. The other non-reproducing rows are listed in the script's output and shown in Figure 2.
Figures
Figure 1. Mean score difference between participants with and without hypertension (yes minus no), in points on the 0–100 scale, on one axis. The thin vertical line marks no difference. Orange circles and intervals show the printed differences (−0.85 for knowledge, −0.64 for behaviour), each with the 95% interval its printed p value implies (p = 0.007: −1.47 to −0.23; p = 0.017: −1.17 to −0.11). Neither orange interval includes zero. Blue diamonds and intervals are the 95% CIs recomputed from the Table 1 SDs and group sizes with Student's t test: −3.94 to 2.24 (p = 0.59) and −5.44 to 4.16 (p = 0.79). Both blue intervals contain zero. For the printed p values to hold, the standard errors would have to be about one-fifth (knowledge) to one-ninth (behaviour) of what the printed SDs and group sizes give.
Figure 2. Two-sided p values printed in Table 1 (orange circles; an arrow marks a value printed as a "<" bound) against p values recomputed from each row's n, mean and SD (blue diamonds; Student's t for two-category variables, one-way ANOVA for three-category variables, as the Table 1 footer specifies). The dashed line marks p = 0.05. Where a printed value is reproduced, the two markers sit together (often the diamond covers the circle) and the row label is in grey. For rows printed as a "<" bound, a reproduced value can lie far to the left of the printed bound; the thin grey line then marks agreement, not a gap. Row labels in bold mark comparisons whose printed p is not reproduced. The shaded rows are the hypertension comparisons, two of the three rows whose recomputed p falls on the other side of p = 0.05 (the third is physical-activity behaviour). Many rows reproduce, including age, sex, marital status and smoking (knowledge), four of the five other factors listed in the Abstract; the fifth, diabetes status, does not reproduce (0.0052 vs printed <0.001). The method therefore recovers the paper's reporting in general. Gaps also occur in rows other than hypertension (for example dyslipidaemia and physical-activity behaviour).
Why it matters
The hypertension result is printed in the Abstract, as one of six factors with "Significant associations" with diabetes knowledge, and the Results text repeats it. The Abstract's next sentence ("Behaviour scores correlated similarly with these factors except for diabetes and smoking status") presents the behaviour association as significant too. Recomputed from the paper's own Table 1, neither comparison is significant: p = 0.59 for knowledge and p = 0.79 for behaviour. The 95% CI for the knowledge difference, −3.94 to 2.24 points, is compatible with no difference. The direction of the printed means is unchanged, with hypertensive participants about 0.85 points lower on a 0–100 scale. What changes is the conclusion that the difference is statistically detectable.
The error has two further consequences.
- Selection into the regression models. Methods state that variables significant at p < 0.05 in the preliminary t tests and ANOVA were entered into the regression models. Hypertension was also included on the strength of prior literature ("covariate adjustment was guided by previous research … hypertension [36]"), so its inclusion does not rest on the unadjusted test alone.
- The Discussion's account of hypertension. The Discussion says participants with hypertension "exhibited higher diabetes knowledge and behaviour scores". That rests on the adjusted coefficients (+4.5346 and +0.5410, both p < 0.0001). The unadjusted means point slightly the other way, and their difference is not significant. So the unadjusted comparison gives no support to the adjusted finding. A reader cannot tell from the paper how an unadjusted difference of −0.85 points becomes an adjusted +4.53 points.
The paper's central conclusions do not depend on these rows. These are the association between knowledge and behaviour (the knowledge–attitude–behaviour framing) and the differences by age and sex, which do reproduce. The error affects a secondary result that is printed in the Abstract.
A related observation, separate from the claim above: the Table 1 rows for different variables imply different overall knowledge means for the same 1050 participants. About 47.2 follows from most variables (for example sex 47.23, hypertension 47.25, smoking 47.17), but 41.33 from age, 63.09 from diabetes status and 75.98 from location. This does not change the hypertension recomputation, which uses only the hypertension rows. It does show that Table 1 has other internal inconsistencies.
Explanations considered
- Rounding of the printed means and SDs. This does not account for the numbers. Moving every mean and SD to the edge of its rounding interval gives a smallest p of 0.585. The difference would have to be 4.25 points, not 0.85, to give p = 0.007.
- A different version of the t test (Welch, z, one-sided). This does not account for the numbers. Welch gives p = 0.596, a z test 0.595 and a one-sided Student test 0.295. Methods set significance at p < 0.05 with no mention of one-sided tests.
- The ± values being standard errors rather than SDs. This does not account for the numbers, and the paper rules it out. The Table 1 footer says "means ± SD". Read as SEs, the values would make the test weaker (p = 0.96) and would imply an SD of about 111 in the hypertensive group on a 0–100 score.
- The p value coming from the adjusted model. This does not account for the numbers. Tables 2 and 3 give the hypertension coefficient as +4.5346 and +0.5410 with p < 0.0001. Those values have the opposite sign to the unadjusted differences, and their p values do not match 0.007 or 0.017. Table 1 is presented as the unadjusted comparison.
- A rank-based or other non-parametric test on the raw data. This cannot be fully excluded from summary statistics alone. However, the Table 1 footer and Methods both name the t test, and the paper gives no other test for these comparisons. With near-identical means and SDs, a distribution-free test would need markedly different distribution shapes in the two groups to reach p = 0.007. Nothing in the paper describes such shapes.
- Values moved between rows. This is not established. The recomputed p for the diabetes-status knowledge comparison is 0.0052 (Student) or 0.0084 (Welch), which is close to 0.007, but that row is printed as "<0.001". No row recomputes to 0.017. The coincidence alone cannot show that p values were shifted between rows.
- A conversion or typesetting artefact in the archived copy. This is very unlikely: the publisher page gives the same Abstract and Results text, and PMC and Europe PMC give identical Table 1 cells; the publisher's PDF table could not be retrieved automatically. The PMC HTML and the Europe PMC full-text XML give the Table 1 hypertension cells as separate
<td>elements with "0.007" and "0.017". The neighbouring smoking row reproduces from the same layout (p = 0.0020 against a printed 0.003).
From the paper's numbers alone it is not possible to tell whether the p values, the means or the SDs in the hypertension rows are in error. Taken together, the printed means, SDs and group sizes are consistent with each other and with the Table 1 overall mean of about 47.2, and they give p ≈ 0.59 and p ≈ 0.79.
Status
As of 2026-10-23:
- Version of record: checked. On the publisher's article page (link.springer.com, reached after the cookie redirect), the Abstract reads "hypertension (46.46 ± 13.11 for present vs 47.31 ± 12.87 for absent; p=0.007)" and the Results sentence matches the one quoted above. The Table 1 cells did not come through in the publisher-page capture, and the standalone table page and the PDF returned a bot check. The PMC HTML page and the Europe PMC full-text XML give identical Table 1 cells, with the Student's t footer. The figures agree across the Abstract, Results and Table 1 in every source.
- Supplement: none exists. No electronic supplementary material is listed on the publisher page or on PMC, and Europe PMC records no supplementary files (hasSuppl = N, no
<supplementary-material>element). The data availability statement says the data are available from the corresponding author on request. No public data could settle the discrepancy. - Corrections: none found.
src/known.pyfound no Crossref updates and no PMC or Europe PMC notices. The Crossref works record has no update-to or updated-by entries and no relations, and a Crossref relation search for the DOI returned nothing. The Europe PMC record (PMID 39505766) has an empty comment/correction list. A Europe PMC search for a correction or erratum to this article in Diabetologia found none, and the captured publisher page shows no correction notice. - PubPeer: not reachable by automated query, because the API returned a human-verification check. Web searches (including one today for the title with "pubpeer", "correction" and "erratum") found no PubPeer thread, erratum or published comment. A manual search at pubpeer.com is still advisable.
Sources: doi.org/10.1007/s00125-024-06304-3 · link.springer.com article · PMC11732930 · Europe PMC record · Crossref works record
Suggested correction
The authors could re-run the unadjusted comparisons by hypertension status from the original data and report the group sizes, means, SDs and p values exactly as produced, so that they agree with each other.
- If the printed means, SDs and group sizes are correct, the hypertension p values should read about 0.59 (knowledge) and 0.79 (behaviour). Hypertension should then be removed from the Abstract's list of factors significantly associated with knowledge, and from the behaviour statement that follows it. The Results sentences should say the differences were not significant.
- If the p values are correct, the corrected means or SDs should replace those in the Abstract, the Results and Table 1.
It would also help to reconcile the other Table 1 p values that do not reproduce from their rows (Figure 2, script output section 3), and the differing overall knowledge means implied by the diabetes-status, location and age rows. The text could explain how the near-zero unadjusted difference relates to the adjusted hypertension coefficients in Tables 2 and 3.
Recomputation script (python recompute.py.txt)
"""Recompute the unadjusted hypertension comparisons of PMC11732930
(Diabetologia 2024, doi:10.1007/s00125-024-06304-3) and regenerate the figures.
Run: python recompute.py.txt
Needs: numpy, scipy, matplotlib.
Prints every recomputed number used in writeup.md. Writes fig1.svg/.png and
fig2.svg/.png into the folder that holds this file, and five candidate figure
designs (A-E) into its candidates/ subfolder (fig1 = design A, fig2 = design B).
All inputs are copied from Table 1 of the paper (n, mean +/- SD, printed p).
The Table 1 footer states that two-category variables were compared with
Student's t tests and variables with three or more categories with one-way
ANOVA. Student's t is computed with pooled variance (df = n1 + n2 - 2); Welch's
t is shown as a sensitivity check.
"""
import os
import shutil
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------------------
# Table 1 inputs: label, (n, mean, SD) group 1, (n, mean, SD) group 2, printed p
# Differences are always group 1 minus group 2, in the order of Table 1.
# ---------------------------------------------------------------------------
TWO_GROUP = [
("Sex (male vs female)", "knowledge", (474, 45.09, 13.27), (576, 49.00, 12.35), "<0.0001"),
("Sex (male vs female)", "behaviour", (474, 81.81, 22.00), (576, 88.00, 18.14), "<0.0001"),
("Marital status", "knowledge", (652, 45.39, 13.10), (398, 50.92, 11.69), "<0.0001"),
("Marital status", "behaviour", (652, 83.47, 20.91), (398, 88.53, 18.11), "0.0003"),
("Education", "knowledge", (176, 47.04, 14.58), (874, 47.27, 12.56), "0.457"),
("Education", "behaviour", (176, 85.06, 21.33), (874, 85.24, 19.98), "0.913"),
("Employment", "knowledge", (431, 47.70, 12.35), (619, 46.91, 13.29), "0.745"),
("Employment", "behaviour", (431, 85.80, 20.42), (619, 84.80, 20.05), "0.429"),
("Hypertension (yes vs no)", "knowledge", (72, 46.46, 13.11), (978, 47.31, 12.87), "0.007"),
("Hypertension (yes vs no)", "behaviour", (72, 84.46, 21.75), (978, 85.10, 19.90), "0.017"),
("Dyslipidaemia", "knowledge", (16, 47.12, 11.34), (1034, 47.25, 12.98), "0.076"),
("Dyslipidaemia", "behaviour", (16, 86.25, 22.45), (1034, 85.00, 19.98), "0.228"),
("Coronary artery disease", "knowledge", (11, 47.45, 10.58), (1039, 47.19, 12.77), "0.975"),
("Coronary artery disease", "behaviour", (11, 82.73, 20.58), (1039, 85.20, 20.00), "0.849"),
("Smoking", "knowledge", (452, 45.78, 13.22), (598, 48.22, 12.15), "0.003"),
("Smoking", "behaviour", (452, 84.09, 21.32), (598, 86.15, 19.43), "0.076"),
("Diabetes status", "knowledge", (52, 69.49, 17.35), (998, 62.76, 16.88), "<0.001"),
("Diabetes status", "behaviour", (52, 88.08, 20.39), (998, 85.06, 20.19), "0.294"),
("Location", "knowledge", (921, 75.60, 20.90), (129, 78.71, 19.35), "0.1108"),
("Location", "behaviour", (921, 84.99, 20.29), (129, 86.74, 19.53), "0.3571"),
]
MULTI_GROUP = [
("Age (3 groups)", "knowledge", [(542, 39.73, 9.95), (382, 42.61, 9.18), (126, 44.32, 9.53)], "<0.0001"),
("Age (3 groups)", "behaviour", [(542, 82.42, 21.84), (382, 88.06, 17.72), (126, 88.57, 18.27)], "<0.0001"),
("Physical activity (3 groups)", "knowledge", [(851, 46.37, 12.73), (91, 47.90, 11.88), (108, 48.35, 13.07)], "0.200"),
("Physical activity (3 groups)", "behaviour", [(851, 83.40, 20.56), (91, 87.77, 18.22), (108, 90.08, 17.95)], "0.507"),
("BMI (3 groups)", "knowledge", [(535, 46.52, 12.99), (345, 48.60, 11.82), (170, 47.13, 12.20)], "0.089"),
("BMI (3 groups)", "behaviour", [(535, 84.52, 20.12), (345, 86.70, 18.98), (170, 83.05, 21.64)], "0.108"),
]
def pooled(a, b):
(n1, m1, s1), (n2, m2, s2) = a, b
df = n1 + n2 - 2
sp = np.sqrt(((n1 - 1) * s1 ** 2 + (n2 - 1) * s2 ** 2) / df)
se = sp * np.sqrt(1 / n1 + 1 / n2)
t = (m1 - m2) / se
return dict(diff=m1 - m2, sp=sp, se=se, t=t, df=df, p=2 * stats.t.sf(abs(t), df))
def welch(a, b):
(n1, m1, s1), (n2, m2, s2) = a, b
v1, v2 = s1 ** 2 / n1, s2 ** 2 / n2
se = np.sqrt(v1 + v2)
df = (v1 + v2) ** 2 / (v1 ** 2 / (n1 - 1) + v2 ** 2 / (n2 - 1))
t = (m1 - m2) / se
return dict(diff=m1 - m2, v1=v1, v2=v2, se=se, t=t, df=df, p=2 * stats.t.sf(abs(t), df))
def anova(groups):
N = sum(g[0] for g in groups)
k = len(groups)
gm = sum(g[0] * g[1] for g in groups) / N
ssb = sum(g[0] * (g[1] - gm) ** 2 for g in groups)
ssw = sum((g[0] - 1) * g[2] ** 2 for g in groups)
F = (ssb / (k - 1)) / (ssw / (N - k))
return F, k - 1, N - k, stats.f.sf(F, k - 1, N - k)
def parse_p(s):
"""Return (value, is_upper_bound, half of last printed digit)."""
ub = s.startswith("<")
v = s.lstrip("<")
dec = len(v.split(".")[1])
return float(v), ub, 10 ** (-dec)
def agrees(printed, rec):
"""Printed p reproduces if a '<x' bound holds, or if the recomputed value,
rounded to the printed precision, is within one unit of the last digit,
or if it lies within 10% of the printed value."""
v, ub, unit = parse_p(printed)
if ub:
return rec < v
return abs(round(rec / unit) - round(v / unit)) <= 1 or abs(rec - v) <= 0.10 * v
def ci95(res):
q = stats.t.ppf(0.975, res["df"])
return res["diff"] - q * res["se"], res["diff"] + q * res["se"]
def fmt_p(p):
return f"{p:.2g}" if p >= 0.001 else f"{p:.1e}"
print("=" * 78)
print("1. Hypertension, diabetes KNOWLEDGE score (Table 1; Abstract; Results)")
print("=" * 78)
HY_K = ((72, 46.46, 13.11), (978, 47.31, 12.87))
HY_B = ((72, 84.46, 21.75), (978, 85.10, 19.90))
sk, wk = pooled(*HY_K), welch(*HY_K)
lo_k, hi_k = ci95(sk)
print(f"mean difference (yes - no) {sk['diff']:.2f}")
print(f"Welch variance components {wk['v1']:.3f} + {wk['v2']:.3f} = {wk['v1'] + wk['v2']:.3f}")
print(f"Welch SE {wk['se']:.3f}")
print(f"Welch t, df, two-sided p t = {wk['t']:.3f}, df = {wk['df']:.1f}, p = {wk['p']:.3f}")
print(f"pooled SD {sk['sp']:.2f}")
print(f"Student SE {sk['se']:.4f}")
print(f"Student t, df, two-sided p t = {sk['t']:.3f}, df = {sk['df']}, p = {sk['p']:.3f}")
print(f"Student one-sided p {sk['p'] / 2:.3f}")
print(f"z-test (Welch SE) two-sided p {2 * stats.norm.sf(abs(wk['t'])):.3f}")
print(f"95% CI for the difference (Student) {lo_k:.2f} to {hi_k:.2f}")
t_007 = stats.t.ppf(1 - 0.007 / 2, sk["df"])
t_05 = stats.t.ppf(1 - 0.05 / 2, sk["df"])
print(f"|t| needed for p = 0.007 (df 1048) {t_007:.3f}")
print(f"difference needed for p = 0.007 {t_007 * sk['se']:.2f} points")
print(f"difference needed for p = 0.05 {t_05 * sk['se']:.2f} points")
se_imp_k = abs(sk["diff"]) / t_007
print(f"SE implied by diff 0.85 and p = 0.007 {se_imp_k:.3f} (ratio to Student SE {sk['se'] / se_imp_k:.1f})")
imp_lo_k, imp_hi_k = sk["diff"] - t_05 * se_imp_k, sk["diff"] + t_05 * se_imp_k
print(f"95% CI implied by diff and p = 0.007 {imp_lo_k:.2f} to {imp_hi_k:.2f}")
# Common SD (if both groups shared one SD) needed for p = 0.007
sd_need = abs(sk["diff"]) / (t_007 * np.sqrt(1 / 72 + 1 / 978))
print(f"common SD that p = 0.007 would need {sd_need:.2f} (printed SDs 13.11 and 12.87)")
# Even an unlimited hypertensive group cannot reach p = 0.007 with 978 controls:
t_lim = abs(sk["diff"]) / (12.87 / np.sqrt(978))
print(f"largest |t| with 978 non-hypertensive and infinitely many hypertensive: {t_lim:.2f}"
f" (p = {2 * stats.norm.sf(t_lim):.3f})")
# Equal group sizes needed
for n in range(10, 200000):
if pooled((n, 46.46, 13.11), (n, 47.31, 12.87))["p"] <= 0.007:
print(f"equal n per group needed for p <= 0.007: {n} (total {2 * n}; study total 1050)")
break
# Rounding: move every printed mean and SD to the edge of its rounding interval
pmin = 1.0
for d1 in (-0.005, 0.005):
for d2 in (-0.005, 0.005):
for e1 in (-0.005, 0.005):
for e2 in (-0.005, 0.005):
r = pooled((72, 46.46 + d1, 13.11 + e1), (978, 47.31 + d2, 12.87 + e2))
pmin = min(pmin, r["p"])
print(f"smallest Student p over rounding of means and SDs {pmin:.3f}")
# If the +/- values were SEs rather than SDs
se_alt = np.sqrt(13.11 ** 2 + 12.87 ** 2)
print(f"if +/- were SEs: t = {0.85 / se_alt:.3f}, z-test p = {2 * stats.norm.sf(0.85 / se_alt):.2f};"
f" implied SD in the hypertensive group = {13.11 * np.sqrt(72):.0f} on a 0-100 score")
print()
print("=" * 78)
print("2. Hypertension, diabetes BEHAVIOUR score (Table 1; Results)")
print("=" * 78)
sb, wb = pooled(*HY_B), welch(*HY_B)
lo_b, hi_b = ci95(sb)
print(f"mean difference (yes - no) {sb['diff']:.2f}")
print(f"Student SE, t, p SE = {sb['se']:.3f}, t = {sb['t']:.3f}, p = {sb['p']:.3f}")
print(f"Welch SE, t, df, p SE = {wb['se']:.3f}, t = {wb['t']:.3f}, df = {wb['df']:.1f}, p = {wb['p']:.3f}")
print(f"95% CI for the difference (Student) {lo_b:.2f} to {hi_b:.2f}")
t_017 = stats.t.ppf(1 - 0.017 / 2, sb["df"])
print(f"difference needed for p = 0.017 {t_017 * sb['se']:.2f} points")
se_imp_b = abs(sb["diff"]) / t_017
imp_lo_b, imp_hi_b = sb["diff"] - t_05 * se_imp_b, sb["diff"] + t_05 * se_imp_b
print(f"SE implied by diff 0.64 and p = 0.017 {se_imp_b:.3f} (ratio to Student SE {sb['se'] / se_imp_b:.1f})")
print(f"95% CI implied by diff and p = 0.017 {imp_lo_b:.2f} to {imp_hi_b:.2f}")
print()
print("=" * 78)
print("3. Every Table 1 comparison: printed p vs p recomputed from Table 1 summaries")
print("=" * 78)
print(f"{'variable':30s} {'score':9s} {'diff':>6s} {'Student p':>10s} {'Welch p':>9s} {'printed':>8s} reproduces?")
ALL = []
for lab, sc, a, b, rp in TWO_GROUP:
s, w = pooled(a, b), welch(a, b)
ok = agrees(rp, s["p"])
ALL.append((lab, sc, rp, s["p"], ok))
print(f"{lab:30s} {sc:9s} {s['diff']:6.2f} {fmt_p(s['p']):>10s} {fmt_p(w['p']):>9s} {rp:>8s} {'yes' if ok else 'NO'}")
for lab, sc, gs, rp in MULTI_GROUP:
F, d1, d2, p = anova(gs)
ok = agrees(rp, p)
ALL.append((lab, sc, rp, p, ok))
print(f"{lab:30s} {sc:9s} {'F=' + format(F, '.2f'):>6s} {fmt_p(p):>10s} {'':>9s} {rp:>8s} {'yes' if ok else 'NO'}")
n_ok = sum(r[4] for r in ALL)
print(f"\nreproduce: {n_ok} of {len(ALL)}; do not reproduce: {len(ALL) - n_ok}")
flip = [r for r in ALL if (parse_p(r[2])[0] < 0.05) != (r[3] < 0.05)]
print("comparisons whose significance at 0.05 changes:", [f"{r[0]} / {r[1]}" for r in flip])
print("smoking knowledge (control): Student t = "
f"{pooled((452, 45.78, 13.22), (598, 48.22, 12.15))['t']:.2f}, p = {pooled((452, 45.78, 13.22), (598, 48.22, 12.15))['p']:.4f}")
dm = pooled((52, 69.49, 17.35), (998, 62.76, 16.88))
dmw = welch((52, 69.49, 17.35), (998, 62.76, 16.88))
print(f"diabetes-status knowledge: Student p = {dm['p']:.4f}, Welch p = {dmw['p']:.4f} (printed <0.001)")
print()
print("=" * 78)
print("4. Sample-wide knowledge mean implied by each variable's Table 1 rows")
print("=" * 78)
groups_k = {lab: [a, b] for lab, sc, a, b, rp in TWO_GROUP if sc == "knowledge"}
groups_k.update({lab: gs for lab, sc, gs, rp in MULTI_GROUP if sc == "knowledge"})
for lab, gs in groups_k.items():
N = sum(g[0] for g in gs)
print(f" {lab:30s} N = {N:5d} implied mean = {sum(g[0] * g[1] for g in gs) / N:6.2f}")
print()
print("=" * 78)
print("5. Adjusted regression rows (Tables 2 and 3), for comparison only")
print("=" * 78)
for lab, b_, se_ in [("knowledge (Table 2)", 4.5346, 0.558), ("behaviour (Table 3)", 0.5410, 0.133)]:
z = b_ / se_
print(f" hypertension, {lab}: coefficient {b_:+.4f}, SE {se_}, z = {z:.2f}, p = {2 * stats.norm.sf(z):.1e};"
f" sign {'+' if b_ > 0 else '-'} vs unadjusted difference sign -")
# ---------------------------------------------------------------------------
# Figures
# ---------------------------------------------------------------------------
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, BLUE_L = "#eb6834", "#2a78d6", "#a9c8ee"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
"axes.spines.top": False, "axes.spines.right": False, "axes.linewidth": 0.8,
"svg.fonttype": "none",
})
LW, MS = 2.0, 9 # 2 px lines (at 72 pt/in) and markers >= 8 px
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(CAND, f"{name}.{ext}"), dpi=200, bbox_inches="tight")
plt.close(fig)
def grid_x(ax):
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
# Design A: dot-and-interval, mean difference with 95% CI, reported vs recomputed
def design_a():
fig, ax = plt.subplots(figsize=(7.2, 3.4))
rows = [
("Knowledge score", sk["diff"], (imp_lo_k, imp_hi_k), (lo_k, hi_k), "0.007", sk["p"]),
("Behaviour score", sb["diff"], (imp_lo_b, imp_hi_b), (lo_b, hi_b), "0.017", sb["p"]),
]
y = 3.0
yt, yl = [], []
for lab, d, imp, rec, rp, pp in rows:
ax.plot(imp, [y + 0.28] * 2, color=ORANGE, lw=LW, solid_capstyle="round", zorder=3)
ax.plot(d, y + 0.28, "o", color=ORANGE, ms=MS, mec=BG, mew=1.5, zorder=4)
ax.text(imp[1] + 0.25, y + 0.28, f"interval implied by printed p = {rp}", va="center", fontsize=10, color=INK)
ax.plot(rec, [y - 0.28] * 2, color=BLUE, lw=LW, solid_capstyle="round", zorder=3)
ax.plot(d, y - 0.28, "D", color=BLUE, ms=MS - 1, mec=BG, mew=1.5, zorder=4)
ax.text(rec[1] + 0.25, y - 0.28, f"recomputed 95% CI, p = {pp:.2f}", va="center", fontsize=10, color=INK)
yt.append(y)
yl.append(lab)
y -= 1.6
ax.plot([0, 0], [0.6, 3.6], color=INK2, lw=0.8, zorder=1)
ax.text(0, 3.65, "no difference", ha="center", va="bottom", fontsize=10, color=INK2)
ax.set_yticks(yt, yl, fontsize=10.5, color=INK)
ax.tick_params(axis="y", length=0)
ax.spines["left"].set_visible(False)
ax.set_ylim(0.6, 3.9)
ax.set_xlim(-6.5, 10.5)
ax.set_xlabel("Mean score difference, hypertension yes minus no (points on the 0-100 scale)")
grid_x(ax)
h1, = ax.plot([], [], "o-", color=ORANGE, lw=LW, ms=MS, label="Paper: printed difference with the 95% interval its p value implies")
h2, = ax.plot([], [], "D-", color=BLUE, lw=LW, ms=MS - 1, label="Recomputed from Table 1 means, SDs and n (Student's t)")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.22), frameon=False, fontsize=10, ncol=1)
save(fig, "A_difference_intervals")
# Design B: printed vs recomputed p for every Table 1 comparison (log scale)
def design_b():
order = list(range(len(ALL)))
fig, ax = plt.subplots(figsize=(7.2, 7.6))
labels = []
for i, idx in enumerate(order):
lab, sc, rp, rec, ok = ALL[idx]
y = len(order) - 1 - i
v, ub, _ = parse_p(rp)
is_h = lab.startswith("Hypertension")
if is_h:
ax.axhspan(y - 0.5, y + 0.5, color="#f3f2ee", zorder=0)
ax.plot([min(v, rec), max(v, rec)], [y, y], color=INK2 if ok else INK, lw=0.8 if ok else 1.4, zorder=2)
if ub:
ax.annotate("", xy=(v / 3, y), xytext=(v, y),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=1.5, mutation_scale=10), zorder=3)
ax.plot(v, y, "o", color=ORANGE, ms=MS, mec=BG, mew=1.2, zorder=4)
ax.plot(max(rec, 1e-12), y, "D", color=BLUE, ms=MS - 1, mec=BG, mew=1.2, zorder=5)
labels.append((y, f"{lab.split(' (')[0]}: {sc}", ok))
ax.set_xscale("log")
ax.set_xlim(1e-12, 1.6)
ax.axvline(0.05, color=INK2, lw=0.8, ls=(0, (4, 3)), zorder=1)
ax.text(0.05 * 1.25, len(order) - 0.45, "p = 0.05", ha="left", va="center", fontsize=10, color=INK2)
ax.set_yticks([l[0] for l in labels], [l[1] for l in labels], fontsize=10)
for tick, (_, _, ok) in zip(ax.get_yticklabels(), labels):
tick.set_color(INK if not ok else INK2)
tick.set_fontweight("bold" if not ok else "normal")
ax.tick_params(axis="y", length=0)
ax.spines["left"].set_visible(False)
ax.set_ylim(-0.7, len(order) + 0.2)
ax.set_xlabel("Two-sided p value (log scale)")
grid_x(ax)
h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="Printed in Table 1 (arrow: printed as '<' bound)")
h2, = ax.plot([], [], "D", color=BLUE, ms=MS - 1, label="Recomputed from Table 1 summaries (t test or ANOVA)")
ax.legend(handles=[h1, h2], loc="upper left", bbox_to_anchor=(-0.02, -0.07), frameon=False, fontsize=10, ncol=1)
ax.text(-0.02, -0.165, "Bold row label: printed p not reproduced (differs by more than 10% and by more than one unit "
"in its last printed digit).\nShaded rows: hypertension.",
transform=ax.transAxes, ha="left", va="top", fontsize=10, color=INK2)
save(fig, "B_all_table1_pvalues")
# Design C: observed difference vs the difference each p value would need
def design_c():
fig, ax = plt.subplots(figsize=(7.2, 2.9))
items = [
(abs(sk["diff"]), ORANGE, "o", "Printed difference\n|46.46 - 47.31| = 0.85", +1),
(t_05 * sk["se"], BLUE, "D", f"Needed for p = 0.05:\n{t_05 * sk['se']:.2f}", -1),
(t_007 * sk["se"], BLUE, "D", f"Needed for the printed\np = 0.007: {t_007 * sk['se']:.2f}\n"
f"({t_007 * sk['se'] / abs(sk['diff']):.1f} times 0.85)", +1),
]
ax.plot([0, 6], [1, 1], color=GRID, lw=LW, zorder=1)
ax.plot([abs(sk["diff"]), t_007 * sk["se"]], [1, 1], color=INK2, lw=1.0, zorder=2)
for x, c, m, txt, side in items:
ax.plot(x, 1, m, color=c, ms=MS + 1, mec=BG, mew=1.5, zorder=4)
ax.text(x, 1 + 0.1 * side, txt, ha="center", va="bottom" if side > 0 else "top", fontsize=10, color=INK)
ax.set_xlim(0, 6)
ax.set_ylim(0.55, 1.75)
ax.set_yticks([])
ax.spines["left"].set_visible(False)
ax.set_xlabel("Absolute mean difference in knowledge score, hypertension yes vs no (points, 0-100 scale)\n"
"with the printed SDs (13.11, 12.87) and group sizes (72, 978), Student's t")
grid_x(ax)
h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="From the paper (Table 1)")
h2, = ax.plot([], [], "D", color=BLUE, ms=MS - 1, label="Recomputed from the paper's SDs and n")
ax.legend(handles=[h1, h2], loc="upper left", frameon=False, fontsize=10, ncol=2, bbox_to_anchor=(0, 1.08))
save(fig, "C_difference_needed")
# Design D: each group's mean with its own 95% CI
def design_d():
fig, ax = plt.subplots(figsize=(7.2, 3.0))
gs = [("Hypertension: yes (n = 72)", 72, 46.46, 13.11), ("Hypertension: no (n = 978)", 978, 47.31, 12.87)]
for i, (lab, n, m, s) in enumerate(gs):
y = 1 - i
h = stats.t.ppf(0.975, n - 1) * s / np.sqrt(n)
ax.plot([m - h, m + h], [y, y], color=BLUE, lw=LW, solid_capstyle="round", zorder=3)
ax.plot(m, y, "o", color=ORANGE, ms=MS, mec=BG, mew=1.5, zorder=4)
ax.text(m + h + 0.3, y, f"{m:.2f} (95% CI {m - h:.2f} to {m + h:.2f})", va="center", fontsize=10)
ax.set_yticks([1, 0], [g[0] for g in gs], fontsize=10.5, color=INK)
ax.tick_params(axis="y", length=0)
ax.spines["left"].set_visible(False)
ax.set_ylim(-0.6, 1.9)
ax.set_xlim(40, 60)
ax.text(40.6, 1.6, f"Printed: p = 0.007", color=INK, fontsize=10.5, va="center")
ax.plot(40.25, 1.6, "o", color=ORANGE, ms=8, clip_on=False)
ax.text(48.6, 1.6, f"Recomputed (Student's t): p = {sk['p']:.2f}", color=INK, fontsize=10.5, va="center")
ax.plot(48.25, 1.6, "D", color=BLUE, ms=7, clip_on=False)
ax.set_xlabel("Mean diabetes knowledge score (0-100 scale; axis shows 40-60)")
grid_x(ax)
h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="Group mean printed in Table 1")
h2, = ax.plot([], [], "-", color=BLUE, lw=LW, label="95% CI recomputed from the printed SD and n")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.3), frameon=False, fontsize=10, ncol=2)
save(fig, "D_group_means")
# Design E: where the test statistic sits on the null t distribution
def design_e():
fig, ax = plt.subplots(figsize=(7.2, 3.3))
x = np.linspace(-4.2, 4.2, 801)
yv = stats.t.pdf(x, sk["df"])
ax.plot(x, yv, color=INK2, lw=1.2, zorder=2)
ta = abs(sk["t"])
m = np.abs(x) >= ta
ax.fill_between(x, 0, yv, where=m & (x < 0), color=BLUE_L, lw=0, zorder=1)
ax.fill_between(x, 0, yv, where=m & (x > 0), color=BLUE_L, lw=0, zorder=1)
ax.axvline(sk["t"], color=BLUE, lw=LW, zorder=3)
ax.plot(sk["t"], stats.t.pdf(sk["t"], sk["df"]), "D", color=BLUE, ms=MS - 1, mec=BG, zorder=4)
for s in (-1, 1):
ax.axvline(s * t_007, color=ORANGE, lw=LW, ls=(0, (5, 3)), zorder=3)
ax.text(sk["t"] - 0.08, 0.43, f"recomputed t = {sk['t']:.2f}\nshaded tails: p = {sk['p']:.2f}",
ha="right", va="top", fontsize=10, color=INK)
ax.text(-t_007 - 0.08, 0.2, f"|t| needed for\nprinted p = 0.007:\n{t_007:.2f}", ha="right", va="center",
fontsize=10, color=INK)
ax.set_ylim(0, 0.45)
ax.set_xlim(-4.2, 4.2)
ax.set_yticks([])
ax.spines["left"].set_visible(False)
ax.set_xlabel("Student's t statistic under no difference (df = 1048), hypertension knowledge comparison")
grid_x(ax)
h1, = ax.plot([], [], "-", color=BLUE, lw=LW, label="Recomputed from Table 1 summaries")
h2, = ax.plot([], [], "--", color=ORANGE, lw=LW, label="Implied by the printed p = 0.007")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.2), frameon=False, fontsize=10, ncol=2)
save(fig, "E_null_distribution")
design_a()
design_b()
design_c()
design_d()
design_e()
for src, dst in [("A_difference_intervals", "fig1"), ("B_all_table1_pvalues", "fig2")]:
for ext in ("svg", "png"):
shutil.copyfile(os.path.join(CAND, f"{src}.{ext}"), os.path.join(HERE, f"{dst}.{ext}"))
print("\nfigures written: fig1 (design A), fig2 (design B), candidates/A-E (.svg, .png)")
Abstract calls a non-significant subgroup difference at seroconversion significant; Results and Table 3 report none
Diabetologia (2025) · doi:10.1007/s00125-025-06573-6 · PMC12686032
Summary
This DIPP cohort analysis compares 1334 children with islet autoimmunity in three groups: an affected first-degree relative (FDR) at birth (FDR+, n = 145), an FDR diagnosed during follow-up (FDR− FDR+, n = 87), and no affected FDR (FDR−, n = 1102). The abstract's Results state that at seroconversion both FDR+ and FDR− FDR+ children were "significantly more often positive for GADA and multiple autoantibodies" than FDR− children. The paper's own Table 3 gives non-significant results for the FDR− FDR+ group: GADA OR 1.3 (95% CI 0.9, 2.1), p = 0.201, and multipositivity OR 1.5 (0.9, 2.4), p = 0.106. The Results text and the Discussion both say this group did not differ from FDR− children at seroconversion, and a recomputation from the Table 2 counts agrees (p = 0.19 and 0.11). The abstract is the only place where the claim appears. A reader who sees only the abstract could reasonably conclude that a family history arising after birth already marks a distinct autoantibody profile at seroconversion, which the body of the paper does not show.
What the paper reports
Text is quoted from the PubMed Central archive, which writes the group label as "FDR- FDR+" with a hyphen; the version of record typesets it with a minus sign (FDR− FDR+).
Abstract, Results:
At seroconversion, both FDR+ and FDR- FDR+ children were significantly more often positive for GADA and multiple autoantibodies than children without affected FDRs (FDR-).
Results, fourth paragraph (beginning “We compared the islet autoantibody profiles”):
At seroconversion, there was a statistically significant difference between the three groups for GADA positivity and multipositivity (Table 2). In pairwise comparisons, at seroconversion, FDR+ children had a higher risk for GADA positivity and multipositivity than FDR- children, but no difference was observed between FDR- FDR+ and FDR- children (Table 3).
Table 3, "At seroconversion" rows (columns: FDR+ vs FDR− | p value | FDR− FDR+ vs FDR− | p value; footnote: "Values are OR (95% CI) by logistic regression analysis"):
GADA+ | 1.5 (1.1, 2.2) | 0.020 | 1.3 (0.9, 2.1) | 0.201
Multipositive | 1.8 (1.2, 2.6) | 0.002 | 1.5 (0.9, 2.4) | 0.106
Table 2, "At seroconversion" rows (columns: FDR+ (n=145) | FDR− FDR+ (n=87) | FDR− (n=1102) | p value; data are % (n); footnote: "The exact χ2 test was used for categorical variables"):
GADA+ | 62.8 (91) | 59.8 (52) | 52.5 (578) | 0.036
Multipositive | 35.2 (51) | 31.0 (27) | 23.3 (257) | 0.004
Discussion, seventh paragraph (beginning “In terms of the timing of type 1 diabetes in the FDRs”):
FDR- FDR+ children did not differ from FDR- children at seroconversion, but showed a significantly higher risk of being positive for IAA, GADA, IA-2A or multiple autoantibodies during the follow-up period than FDR- children (Table 3).
Methods, "Statistical analyses":
The standard normal distribution (SND) test was used for pairwise comparisons of categorical variables. Logistic regression analysis was used to estimate the risk of various islet autoantibody profiles and progression to type 1 diabetes in children with and without an FDR with the disease.
Abstract, Conclusions/interpretation (for comparison; this sentence does not mention the FDR− FDR+ group):
At seroconversion, genetically susceptible children who had one or more FDRs with type 1 diabetes, especially an affected father, were more often positive for GADA and multiple islet autoantibodies.
What the paper's own numbers show
Method. From the Table 2 counts I rebuilt each 2×2 table (group vs FDR−; positive vs not positive) at seroconversion. For each I computed the unadjusted odds ratio with a Woolf 95% CI and Wald p value. These are exactly what a logistic regression of the outcome on a single group indicator gives, and the Methods name no covariates. I also computed the pooled two-proportion z test (the "SND test" named in the Methods), Fisher's exact test, the likelihood-ratio test and the Yates-corrected χ² test, and repeated the three-group χ² tests from Table 2. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Group sizes | Abstract; Table 2 | 145 / 87 / 1102; total 1334 | 145 + 87 + 1102 = 1334 |
| GADA+ at seroconversion, FDR− FDR+ vs FDR− | Table 2; Table 3 | 59.8% (52) vs 52.5% (578); OR 1.3 (0.9, 2.1), p = 0.201 | 52/87 vs 578/1102; OR 1.35 (0.86, 2.10); Wald p = 0.189; SND p = 0.188; Fisher p = 0.220; LR p = 0.186 |
| Multipositive at seroconversion, FDR− FDR+ vs FDR− | Table 2; Table 3 | 31.0% (27) vs 23.3% (257); OR 1.5 (0.9, 2.4), p = 0.106 | 27/87 vs 257/1102; OR 1.48 (0.92, 2.38); Wald p = 0.106; SND p = 0.104; Fisher p = 0.117; LR p = 0.114 |
| GADA+ at seroconversion, FDR+ vs FDR− | Table 3 | OR 1.5 (1.1, 2.2), p = 0.020 | 91/145 vs 578/1102; OR 1.53 (1.07, 2.18); p = 0.020 |
| Multipositive at seroconversion, FDR+ vs FDR− | Table 3 | OR 1.8 (1.2, 2.6), p = 0.002 | 51/145 vs 257/1102; OR 1.78 (1.23, 2.58); p = 0.002 |
| Three-group test, GADA+ at seroconversion | Table 2 | p = 0.036 (exact χ²) | Pearson χ² = 6.71, df 2, p = 0.035 |
| Three-group test, multipositive at seroconversion | Table 2 | p = 0.004 (exact χ²) | Pearson χ² = 11.31, df 2, p = 0.004 |
| Abstract statement on FDR− FDR+ vs FDR− at seroconversion | Abstract, Results | "significantly more often positive for GADA and multiple autoantibodies" | not significant on either measure under any of the tests above; both 95% CIs include 1 |
The two disputed cells of Table 3 reproduce from Table 2: the odds ratios and intervals match to the printed precision, and the p values are close (printed 0.201, recomputed 0.189 for GADA; printed 0.106, recomputed 0.106 for multipositivity). The body of the paper is internally consistent. Only the abstract sentence departs from it.
Side observation, not part of the error. When every Table 3 cell is recomputed in the same way from the Table 2 counts, six other cells do not reproduce (rule applied in recompute.py.txt, section 7: the recomputed OR, rounded to one decimal, differs from the print by more than 0.1, or the p value differs by more than 0.05). Five are in the FDR− FDR+ vs FDR− column: IAA+ at seroconversion (printed 1.3 (0.8, 2.1), p = 0.181; recomputed 1.52 (0.97, 2.39), p = 0.070), IA-2A+ at seroconversion (printed 1.0 (0.5, 1.6), p = 0.937; recomputed 0.95 (0.52, 1.75), p = 0.869), ZnT8A+ at seroconversion (printed 0.8 (0.3, 1.5), p = 0.484; recomputed 0.89 (0.43, 1.81), p = 0.739), GADA+ during follow-up (printed 2.8 (1.1, 3.7), p = 0.021; recomputed 2.05 (1.12, 3.76), p = 0.020) and ZnT8A+ during follow-up (printed 1.2 (0.8, 1.8), p = 0.361; recomputed 1.48 (0.96, 2.30), p = 0.077). One is in the FDR+ vs FDR− column: ZnT8A+ during follow-up (printed 1.2 (0.8, 1.7), p = 0.227; recomputed 1.21 (0.86, 1.71), p = 0.278). For GADA+ during follow-up, the printed estimate 2.8 is far from the geometric midpoint of its interval (1.1, 3.7), which is 2.0, while the recomputed OR is 2.05; this suggests a typographical slip in the point estimate. None of these six changes which side of p = 0.05 a result falls on, and none bears on the abstract sentence.
Figure
Figure 1. The left panel quotes the abstract's claim that FDR− FDR+ children were significantly more often GADA-positive and multipositive at seroconversion than FDR− children. The right panel quotes the Results sentence and the Table 3 estimates for the same comparison, which are non-significant. Values the paper reports are in orange. The blue panel gives the estimates recomputed from the Table 2 counts (unadjusted logistic regression, equivalent to the 2×2 odds ratio), with Fisher exact p values for comparison. Both recomputed intervals include OR = 1.
Figure 2. Odds ratios with 95% confidence intervals for the four pairwise comparisons at seroconversion, on a log scale with a thin reference line at OR = 1 (no difference). Orange circles show Table 3 as printed; blue diamonds show the values recomputed from the Table 2 counts, with the p value for each printed at the right. The two shaded rows are the FDR− FDR+ comparisons, which the abstract calls significant but Table 3 and the Results text do not. In both, the printed and the recomputed intervals cross 1. The two unshaded FDR+ rows, which the abstract also mentions, are significant and reproduce.
Why it matters
- The error is in the most-read part of the paper. The abstract is what most readers, indexers and citing authors see. Nothing in the abstract signals that the FDR− FDR+ comparison was non-significant, so a reader of the abstract alone cannot detect the problem.
- It changes a substantive interpretation. The FDR− FDR+ group is the paper's way of asking whether a family history that appears only after birth goes with a different autoimmune process. The abstract's sentence implies that these children already differ at the very start of islet autoimmunity. The paper's own analysis, and its Discussion, say instead that they differ from FDR− children only later, during follow-up.
- The headline conclusions are not affected. The FDR+ vs FDR− differences at seroconversion (p = 0.020 and 0.002) reproduce. So do the follow-up differences, the higher progression rates (55.9% vs 57.5% vs 38.9%), the shorter time to diabetes in FDR+ children, and the paternal vs maternal comparison. The abstract's Conclusions sentence does not name the FDR− FDR+ group; its wording (“had one or more FDRs”, “especially an affected father”) matches the paper's definition of the at-birth FDR+ group and is consistent with the body under that reading.
- Direction is unchanged, significance is not. The FDR− FDR+ point estimates are above 1 (GADA 59.8% vs 52.5%; multipositive 31.0% vs 23.3%), and they are close to the FDR+ estimates (FDR− FDR+ vs FDR+: Fisher p = 0.68 and 0.57). The data are therefore compatible with a modest difference, but the study cannot show one for this group of 87 children: the intervals run from about 0.9 to over 2. "Significantly more often" overstates what the analysis supports.
Explanations considered
- An adjusted model that would make the comparison significant. The Methods describe logistic regression with no covariates, and Table 3's footnote says only "by logistic regression analysis". The unadjusted odds ratios from the Table 2 counts reproduce the two disputed Table 3 cells (printed 1.3 and 1.5, recomputed 1.35 and 1.48; printed p = 0.201 and 0.106, recomputed 0.189 and 0.106). No adjusted analysis is reported anywhere in the paper or supplement. This does not account for the claim.
- A different pairwise test. The Methods name the SND test for pairwise comparisons of categorical variables. It gives p = 0.188 and 0.104. Fisher's exact test (0.220, 0.117), the likelihood-ratio test (0.186, 0.114) and the Yates-corrected χ² test (0.228, 0.135) are all non-significant too. Even a one-sided SND test, which the paper does not use, gives p = 0.094 and 0.052. This does not account for the claim.
- The three-group tests in Table 2. The exact χ² tests across all three groups are significant (p = 0.036 and 0.004; recomputed 0.035 and 0.004). An omnibus test shows only that the three groups are not all equal. It does not establish that each family-history group differs from FDR−, and the Results text itself moves from the omnibus result to pairwise comparisons that find "no difference" for FDR− FDR+. This does not account for the sentence, which makes a claim about each group ("both … were").
- A pooled reading ("children with any affected FDR"). If the 145 FDR+ and 87 FDR− FDR+ children are pooled (n = 232), the comparison with FDR− is significant (GADA OR 1.46 (1.09, 1.95), p = 0.011; multipositive OR 1.67 (1.23, 2.26), p = 0.001). The paper reports no pooled analysis, and the sentence names both groups separately. This could explain how a looser summary arose, but it does not make the sentence as written correct.
- A text-conversion artifact. The sentence was checked against the publisher's typeset PDF (Diabetologia (2026) 69:93–102) and the Europe PMC full-text XML. It reads the same in both, and the Table 3 values and Results sentence also match. Not an artifact.
- A supplement that reports a different analysis. The electronic supplementary material (ESM Methods and ESM Tables 1–4) covers HLA genotypes, father/mother/sibling splits and the multipositive subset. It contains no FDR− FDR+ vs FDR− comparison at seroconversion for the full cohort. It does not resolve the conflict.
Status
- Version of record. The abstract sentence, the Results sentence, Table 2 and Table 3 were compared with the publisher's typeset PDF, which is headed "Diabetologia (2026) 69:93–102", carries the same DOI, and is hosted by the University of Oulu repository (https://oulurepo.oulu.fi/handle/10024/59105), and with the Europe PMC full-text XML. All quoted passages are identical to the PMC archive. The publisher's own article page could not be loaded by automated access, which met a login or verification step.
- Supplement. 125_2025_6573_MOESM1_ESM.pdf (9 pages: ESM Methods, ESM Tables 1–4) was reviewed and does not change the comparison.
- Corrections and notices. No correction, erratum, retraction or expression of concern was found. Crossref lists no updates for the DOI, PubMed Central and Europe PMC list no notices, and the Europe PMC record (PMID 41204978) has no linked comments or corrections.
- PubPeer and the web. A web search for the title and DOI with "correction", "erratum" and "PubPeer" found no correction, PubPeer thread or published comment. PubPeer's own search could not be queried directly, because it required human verification, so a PubPeer thread cannot be fully ruled out.
- Checked as of 2026-10-23.
Suggested correction
In the abstract's Results, replace
At seroconversion, both FDR+ and FDR− FDR+ children were significantly more often positive for GADA and multiple autoantibodies than children without affected FDRs (FDR−).
with
At seroconversion, FDR+ children were significantly more often positive for GADA and multiple autoantibodies than children without affected FDRs (FDR−), whereas FDR− FDR+ children did not differ significantly from FDR− children (GADA OR 1.3 [95% CI 0.9, 2.1]; multipositivity OR 1.5 [0.9, 2.4]).
No change to the Results, Tables or Discussion is needed for this point. Separately, the authors may wish to check the six cells of Table 3 listed above (five in the FDR− FDR+ column, one in the FDR+ column) that do not reproduce from the Table 2 counts. One example is GADA+ during follow-up, printed as 2.8 (1.1, 3.7), where the counts give 2.05 (1.12, 3.76).
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12686032 (Diabetologia 2025, doi:10.1007/s00125-025-06573-6)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1 and fig2
# (SVG and 200 dpi PNG) in this script's folder.
# Inputs are the counts and estimates printed in the paper's Table 2 and Table 3.
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
N = {"FDR+": 145, "FDR- FDR+": 87, "FDR-": 1102}
# Table 2, % (n): (FDR+, FDR- FDR+, FDR-)
T2 = {
("seroconversion", "IAA+"): (83, 55, 585),
("seroconversion", "GADA+"): (91, 52, 578),
("seroconversion", "IA-2A+"): (25, 13, 172),
("seroconversion", "ZnT8A+"): (17, 9, 127),
("seroconversion", "Multipositive"): (51, 27, 257),
("follow-up", "IAA+"): (104, 71, 756),
("follow-up", "GADA+"): (124, 74, 810),
("follow-up", "IA-2A+"): (98, 56, 587),
("follow-up", "ZnT8A+"): (71, 47, 487),
("follow-up", "Multipositive"): (107, 64, 622),
("follow-up", "T1D diagnosis"): (81, 50, 429),
}
T2_pct = { # printed percentages
("seroconversion", "GADA+"): (62.8, 59.8, 52.5),
("seroconversion", "Multipositive"): (35.2, 31.0, 23.3),
}
T2_p = {("seroconversion", "GADA+"): 0.036, ("seroconversion", "Multipositive"): 0.004,
("seroconversion", "IAA+"): 0.143, ("seroconversion", "IA-2A+"): 0.862,
("seroconversion", "ZnT8A+"): 0.940}
# Table 3, OR (95% CI), p: FDR+ vs FDR-, then FDR- FDR+ vs FDR-
T3 = {
("seroconversion", "IAA+"): ((1.2, 0.8, 1.7, 0.356), (1.3, 0.8, 2.1, 0.181)),
("seroconversion", "GADA+"): ((1.5, 1.1, 2.2, 0.020), (1.3, 0.9, 2.1, 0.201)),
("seroconversion", "IA-2A+"): ((1.1, 0.7, 1.7, 0.612), (1.0, 0.5, 1.6, 0.937)),
("seroconversion", "ZnT8A+"): ((1.0, 0.5, 1.7, 0.944), (0.8, 0.3, 1.5, 0.484)),
("seroconversion", "Multipositive"): ((1.8, 1.2, 2.6, 0.002), (1.5, 0.9, 2.4, 0.106)),
("follow-up", "IAA+"): ((1.2, 0.8, 1.7, 0.472), (2.0, 1.2, 3.5, 0.014)),
("follow-up", "GADA+"): ((2.1, 1.3, 3.4, 0.002), (2.8, 1.1, 3.7, 0.021)),
("follow-up", "IA-2A+"): ((1.8, 1.3, 2.7, 0.001), (1.6, 1.0, 2.5, 0.049)),
("follow-up", "ZnT8A+"): ((1.2, 0.8, 1.7, 0.227), (1.2, 0.8, 1.8, 0.361)),
("follow-up", "Multipositive"): ((2.2, 1.5, 3.2, 0.0005), (2.1, 1.3, 3.5, 0.002)), # "<0.001" drawn as 0.0005
("follow-up", "T1D diagnosis"): ((2.0, 1.4, 2.8, 0.0005), (2.1, 1.4, 3.3, 0.0005)),
}
# ---------------------------------------------------------------- methods
def woolf(a, n1, c, n0):
"""Unadjusted OR, Woolf 95% CI, Wald p. Identical to a logistic regression
of the outcome on a single group indicator (no covariates)."""
b, d = n1 - a, n0 - c
lor = np.log(a * d / (b * c))
se = np.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
z = lor / se
return np.exp(lor), np.exp(lor - 1.96 * se), np.exp(lor + 1.96 * se), 2 * stats.norm.sf(abs(z)), z
def snd(a, n1, c, n0):
"""Pooled two-proportion z test (the 'SND test' named in the Methods)."""
p1, p0, pp = a / n1, c / n0, (a + c) / (n1 + n0)
z = (p1 - p0) / np.sqrt(pp * (1 - pp) * (1 / n1 + 1 / n0))
return z, 2 * stats.norm.sf(abs(z)), stats.norm.sf(z)
def wilson(x, n, z=1.96):
p = x / n
den = 1 + z * z / n
c = (p + z * z / (2 * n)) / den
h = z * np.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / den
return 100 * p, 100 * (c - h), 100 * (c + h)
def table22(a, n1, c, n0):
return np.array([[a, n1 - a], [c, n0 - c]])
R = {} # recomputed results reused by the figures
print("=" * 78)
print("1. Group sizes")
print(f" 145 + 87 + 1102 = {sum(N.values())} (paper: 1334); 145/1334 = {100*145/1334:.1f}% (paper: 10.9%)")
print("=" * 78)
print("2. Table 2 percentages at seroconversion (recomputed from the printed n)")
for key in [("seroconversion", "GADA+"), ("seroconversion", "Multipositive")]:
x = T2[key]
pct = [100 * xi / ni for xi, ni in zip(x, N.values())]
print(f" {key[1]:14s} {pct[0]:.1f} / {pct[1]:.1f} / {pct[2]:.1f} (paper: {T2_pct[key]})")
print("=" * 78)
print("3. Three-group tests at seroconversion (Table 2 p column)")
for key in [("seroconversion", k) for k in ["IAA+", "GADA+", "IA-2A+", "ZnT8A+", "Multipositive"]]:
x = T2[key]
tab = np.array([[xi, ni - xi] for xi, ni in zip(x, N.values())])
chi2, p3, _, _ = stats.chi2_contingency(tab, correction=False)
print(f" {key[1]:14s} Pearson chi2 = {chi2:.2f}, df 2, p = {p3:.3f} (paper, exact chi2: {T2_p[key]})")
print("=" * 78)
print("4. The disputed comparison: FDR- FDR+ (n=87) vs FDR- (n=1102), at seroconversion")
for k in ["GADA+", "Multipositive"]:
x1, x2, x0 = T2[("seroconversion", k)]
o, lo, hi, p, zw = woolf(x2, 87, x0, 1102)
zs, ps, p1s = snd(x2, 87, x0, 1102)
pf = stats.fisher_exact(table22(x2, 87, x0, 1102))[1]
pyates = stats.chi2_contingency(table22(x2, 87, x0, 1102), correction=True)[1]
plr = stats.chi2_contingency(table22(x2, 87, x0, 1102), correction=False, lambda_="log-likelihood")[1]
rep = T3[("seroconversion", k)][1]
R[("FDR- FDR+", k)] = dict(or_=o, lo=lo, hi=hi, p=p, pf=pf, ps=ps)
print(f" {k}: {x2}/87 ({100*x2/87:.1f}%) vs {x0}/1102 ({100*x0/1102:.1f}%), "
f"difference {100*(x2/87 - x0/1102):.1f} percentage points")
print(f" unadjusted OR {o:.2f} (95% CI {lo:.2f}, {hi:.2f}); Wald z = {zw:.2f}, p = {p:.3f}")
print(f" SND / two-proportion z = {zs:.2f}, two-sided p = {ps:.3f}, one-sided p = {p1s:.3f}")
print(f" Fisher exact p = {pf:.3f}; Yates chi2 p = {pyates:.3f}; likelihood-ratio p = {plr:.3f}")
print(f" Table 3 prints: OR {rep[0]} ({rep[1]}, {rep[2]}), p = {rep[3]}")
print("=" * 78)
print("5. The comparison the abstract gets right: FDR+ (n=145) vs FDR- (n=1102), at seroconversion")
for k in ["GADA+", "Multipositive"]:
x1, x2, x0 = T2[("seroconversion", k)]
o, lo, hi, p, zw = woolf(x1, 145, x0, 1102)
pf = stats.fisher_exact(table22(x1, 145, x0, 1102))[1]
zs, ps, _ = snd(x1, 145, x0, 1102)
rep = T3[("seroconversion", k)][0]
R[("FDR+", k)] = dict(or_=o, lo=lo, hi=hi, p=p, pf=pf, ps=ps)
print(f" {k}: {x1}/145 vs {x0}/1102; OR {o:.2f} ({lo:.2f}, {hi:.2f}), Wald p = {p:.3f}, "
f"SND p = {ps:.3f}, Fisher p = {pf:.3f} (Table 3: OR {rep[0]} ({rep[1]}, {rep[2]}), p = {rep[3]})")
print("=" * 78)
print("6. Alternative readings of the abstract sentence")
print(" (a) pooled 'ever FDR+' (145 + 87 = 232) vs FDR- -- not what the sentence says ('both ... were')")
for k in ["GADA+", "Multipositive"]:
x1, x2, x0 = T2[("seroconversion", k)]
o, lo, hi, p, _ = woolf(x1 + x2, 232, x0, 1102)
print(f" {k}: {x1+x2}/232 vs {x0}/1102, OR {o:.2f} ({lo:.2f}, {hi:.2f}), p = {p:.4f}")
print(" (b) FDR- FDR+ vs FDR+ (are the two family-history groups different from each other?)")
for k in ["GADA+", "Multipositive"]:
x1, x2, x0 = T2[("seroconversion", k)]
print(f" {k}: {x2}/87 vs {x1}/145, Fisher p = {stats.fisher_exact(table22(x2, 87, x1, 145))[1]:.3f}")
print(" (c) three-group test without the FDR- FDR+ group's contribution is not needed:"
" the omnibus test only says the three groups are not all equal.")
print(" (d) smallest one-sided p for FDR- FDR+ vs FDR- (GADA, multipositive):",
f"{snd(52, 87, 578, 1102)[2]:.3f}, {snd(27, 87, 257, 1102)[2]:.3f}")
print("=" * 78)
print("7. Side observations: every Table 3 cell recomputed as an unadjusted OR from Table 2 counts")
print(" (flag '<--': rounded OR differs from the print by more than 0.1, or p differs by more than 0.05;"
" none bears on the abstract claim)")
FLAGGED = []
for key, (r1, r2) in T3.items():
x1, x2, x0 = T2[key]
for lab, a, n1, r in [("FDR+ vs FDR-", x1, 145, r1), ("FDR- FDR+ vs FDR-", x2, 87, r2)]:
o, lo, hi, p, _ = woolf(a, n1, x0, 1102)
flag = " <--" if abs(round(o, 1) - r[0]) > 0.1 + 1e-9 or (r[3] > 0.001 and abs(p - r[3]) > 0.05) else ""
if flag:
FLAGGED.append((key, lab))
rp = "<0.001" if r[3] == 0.0005 else r[3]
ps_ = "<0.001" if p < 0.001 else f"{p:.3f}"
print(f" {key[0]:14s} {key[1]:14s} {lab:18s} OR {o:.2f} ({lo:.2f}, {hi:.2f}) p={ps_}"
f" printed {r[0]} ({r[1]}, {r[2]}) p={rp}{flag}")
print(f" flagged cells: {len(FLAGGED)} "
f"({sum(l == 'FDR- FDR+ vs FDR-' for _, l in FLAGGED)} FDR- FDR+ vs FDR-, "
f"{sum(l == 'FDR+ vs FDR-' for _, l in FLAGGED)} FDR+ vs FDR-)")
r = T3[("follow-up", "GADA+")][1]
print(f" GADA+ during follow-up, FDR- FDR+ vs FDR-: geometric midpoint of printed CI ({r[1]}, {r[2]}) = "
f"{np.sqrt(r[1] * r[2]):.1f}; printed OR {r[0]}")
# ================================================================= figures
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAYBOX = "#eb6834", "#2a78d6", "#e4e3df", "#52514e"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": FG,
"axes.labelcolor": FG, "xtick.color": FG2, "ytick.color": FG2,
"axes.edgecolor": "#b9b8b3", "svg.fonttype": "none",
"figure.facecolor": BG, "axes.facecolor": BG,
})
W = 7.2
LW = 2.0 # 2 px-equivalent line weight in points
MS = 8.5 # marker size (pt)
def save(fig, path):
fig.savefig(path + ".svg", facecolor=BG)
fig.savefig(path + ".png", dpi=200, facecolor=BG)
plt.close(fig)
def box(ax, x, y, w, h, edge, lw=2.0):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=1.2",
fc=BG, ec=edge, lw=lw, transform=ax.transData))
def fmt_or(o, lo, hi, nd=1):
return f"{o:.{nd}f} ({lo:.{nd}f}, {hi:.{nd}f})"
G = R[("FDR- FDR+", "GADA+")]; M = R[("FDR- FDR+", "Multipositive")]
GP = R[("FDR+", "GADA+")]; MP = R[("FDR+", "Multipositive")]
# ---------------------------------------------- Figure 1: two text panels
def fig_text_panels(path):
fig = plt.figure(figsize=(W, 5.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 78); ax.axis("off")
# left: abstract
box(ax, 2, 30, 41, 45, ORANGE)
ax.text(5, 71, "Abstract, Results", weight="bold", fontsize=12.5, va="top")
ax.text(5, 65.5, "“At seroconversion, both FDR+ and\n"
"FDR− FDR+ children were\n"
"significantly more often positive for\n"
"GADA and multiple autoantibodies\n"
"than children without affected FDRs\n"
"(FDR−).”",
fontsize=10.5, va="top", linespacing=1.45)
ax.text(5, 39.5, "Claim for FDR− FDR+ vs FDR−:", color=FG2, fontsize=10.5, va="top")
ax.text(5, 35.5, "significant (as reported)", color=ORANGE, weight="bold", fontsize=11.5, va="top")
# right: body
box(ax, 57, 30, 41, 45, GRAYBOX)
ax.text(60, 71, "Results text and Table 3", weight="bold", fontsize=12.5, va="top")
ax.text(60, 65.5, "“…but no difference was observed\n"
"between FDR− FDR+ and FDR−\n"
"children (Table 3).”", fontsize=10.5, va="top", linespacing=1.45)
ax.text(60, 52, "Table 3, FDR− FDR+ vs FDR−,\nOR (95% CI), p:", color=FG2, fontsize=10.5, va="top",
linespacing=1.3)
ax.text(60, 44.5, "GADA+: 1.3 (0.9, 2.1), p = 0.201\n"
"Multipositive: 1.5 (0.9, 2.4), p = 0.106",
fontsize=10, va="top", color=ORANGE, linespacing=1.5)
ax.text(60, 35.5, "not significant (as reported)", color=ORANGE, weight="bold", fontsize=11.5, va="top")
# connector
ax.add_patch(FancyArrowPatch((44, 52), (56, 52), arrowstyle="<|-|>", mutation_scale=16, lw=LW, color=FG))
ax.text(50, 56, "conflict", ha="center", weight="bold", fontsize=11)
ax.text(50, 47.5, "same\ncomparison,\nopposite\nverdict", ha="center", va="top", color=FG2, fontsize=9.5,
linespacing=1.3)
# bottom: recomputed
box(ax, 2, 3, 96, 23, BLUE)
ax.text(5, 22.5, "Recomputed from the Table 2 counts (unadjusted logistic regression)",
color=BLUE, weight="bold", fontsize=10.5, va="top")
ax.text(5, 16.5,
f"GADA+: 52/87 vs 578/1102, OR {fmt_or(G['or_'], G['lo'], G['hi'], 2)}, "
f"p = {G['p']:.2f} (Fisher exact {G['pf']:.2f})\n"
f"Multipositive: 27/87 vs 257/1102, OR {fmt_or(M['or_'], M['lo'], M['hi'], 2)}, "
f"p = {M['p']:.2f} (Fisher exact {M['pf']:.2f})\n"
"Both intervals include OR = 1, as Table 3 and the Results text state.",
fontsize=10, va="top", linespacing=1.6)
save(fig, path)
# ---------------------------------------------- Figure 2: forest plot (reported vs recomputed)
ROWS = [ # label, reported tuple, recomputed dict, disputed?
("GADA+\nFDR+ vs FDR−", T3[("seroconversion", "GADA+")][0], GP, False),
("GADA+\nFDR− FDR+ vs FDR−", T3[("seroconversion", "GADA+")][1], G, True),
("Multipositive\nFDR+ vs FDR−", T3[("seroconversion", "Multipositive")][0], MP, False),
("Multipositive\nFDR− FDR+ vs FDR−", T3[("seroconversion", "Multipositive")][1], M, True),
]
def forest(ax, rows, ptext=True):
ax.set_xscale("log")
ax.set_xlim(0.6, 4.0)
ticks = [0.6, 0.8, 1, 1.5, 2, 3, 4]
ax.set_xticks(ticks); ax.set_xticklabels([str(t) for t in ticks]); ax.minorticks_off()
ax.axvline(1, color=FG2, lw=0.9, zorder=1)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
n = len(rows)
for i, (lab, rep, rc, disp) in enumerate(rows):
y = n - 1 - i
if disp:
ax.axhspan(y - 0.45, y + 0.45, color="#f3f2ee", zorder=0)
ax.plot([rep[1], rep[2]], [y + 0.14] * 2, color=ORANGE, lw=LW, zorder=3, solid_capstyle="butt")
ax.plot(rep[0], y + 0.14, "o", color=ORANGE, ms=MS, zorder=4)
ax.plot([rc["lo"], rc["hi"]], [y - 0.14] * 2, color=BLUE, lw=LW, zorder=3, solid_capstyle="butt")
ax.plot(rc["or_"], y - 0.14, "D", color=BLUE, ms=MS - 1, zorder=4)
if ptext:
ax.text(4.25, y + 0.14, f"p = {rep[3]:.3f}", color=ORANGE, va="center", fontsize=10)
ax.text(4.25, y - 0.14, f"p = {rc['p']:.3f}", color=BLUE, va="center", fontsize=10)
ax.set_yticks(range(n)); ax.set_yticklabels([r[0] for r in rows][::-1], fontsize=10, color=FG)
ax.set_ylim(-0.6, n - 0.4)
ax.set_xlabel("Odds ratio at seroconversion (log scale); line at OR = 1 means no difference")
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
def legend_handles():
from matplotlib.lines import Line2D
return [Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="Reported (Table 3), 95% CI"),
Line2D([], [], color=BLUE, marker="D", ms=MS - 1, lw=LW,
label="Recomputed from Table 2 counts, 95% CI")]
def fig_forest(path):
fig, ax = plt.subplots(figsize=(W, 4.4))
fig.subplots_adjust(left=0.27, right=0.83, top=0.80, bottom=0.14)
forest(ax, ROWS)
ax.legend(handles=legend_handles(), loc="lower left", bbox_to_anchor=(-0.36, 1.08), ncol=2,
frameon=False, fontsize=9.6, handlelength=2.2)
for y in [2, 0]:
ax.text(0.62, y + 0.36, "abstract: “significantly more often”", fontsize=9.3, color=FG2,
va="center", style="italic")
save(fig, path)
if __name__ == "__main__":
fig_text_panels(os.path.join(HERE, "fig1"))
fig_forest(os.path.join(HERE, "fig2"))
print("=" * 78)
print("figures written: fig1.svg/.png, fig2.svg/.png")
The 2013 estimate of MSM living with HIV lies above its own 90% upper limit
Epidemiology (2016) · doi:10.1097/EDE.0000000000000423 · PMC4733816
Summary
The paper's main applied result is the number of men who have sex with men (MSM) living with HIV in the UK in 2013, given in the Abstract and Results as 48,310 (90% plausibility range 39,900–45,560). The Methods define the point estimate as the median and the range limits as the 5th and 95th percentiles of one distribution of model outputs, so the median cannot exceed the upper limit. Here it does, by 2,750. The point estimate is consistent with the paper's other numbers (its four CD4 strata sum to exactly 48,310), and the paper's own Figure 3A shows the 2013 range running from about 40,000 to about 60,400. The printed upper limit is therefore wrong, and as printed it understates the upward uncertainty of the headline estimate by roughly 15,000 people, in a paper whose stated contribution is a method that produces such ranges.
What the paper reports
Abstract:
In 2013, 48,310 (90% plausibility range: 39,900–45,560) men who have sex with men were estimated to be living with HIV in the UK, of whom 10,400 (6,160–17,350) were undiagnosed.
Results, third paragraph of "Calibration to Data on MSM in the UK and Resulting Estimates for 2013":
By 2013, an estimated 48,310 (90% plausibility range: 39,900–45,560) MSM were living with HIV in the UK (Figure 3), and 2,300, 5,500, 11,830, and 28,680 individuals had CD4 count ≤200, 201–350, 351–500, and >500 cells/mm3 respectively. It is thought that 10,400 (6,160–17,350) MSM were living with undiagnosed HIV, which means the undiagnosed proportion is 22% (13%–36%, Figure 3).
Methods, third paragraph of "How the Model Is Calibrated to Country Data", which defines the estimate and the range:
The median, 5th and 95th percentiles of the distribution of these model outputs at each calendar year are considered the point estimate and plausibility range limits, respectively.
Results, first paragraph of the same subsection, which says which runs the results come from:
The following results presented are those generated using the 742 parameter sets where the calibration score was <0.2 (median: 0.178, min: 0.107, max: 0.199) among the 1,000 resampled simulations.
Figure 3 legend:
Estimates of the (A) total number of MSM living with HIV in the UK and (B) total number of MSM living with undiagnosed HIV, by calendar year. Columns and error bars: Modeled median and 90% plausibility range.
Discussion, second paragraph (point estimate only):
The number of MSM living with HIV in the UK is now estimated to be 48,310 for 2013.
Discussion, final (tenth) paragraph, on what the method contributes:
This novel method allows us to also naturally produce plausibility ranges for these estimates, which describe the uncertainty surrounding the estimates, based on the data to which the model was calibrated.
What the paper's own numbers show
Method. I checked the reported estimate against its range using the paper's definition (median between the 5th and 95th percentiles; this holds for any distribution, and recompute.py.txt confirms it on four very different simulated ones). To see which of the three numbers is at fault, I summed the CD4 strata and the undiagnosed CD4 strata given in the same paragraph and recomputed the undiagnosed proportion. I checked every other estimate-with-range in the Results the same way, and expressed each as the position of the estimate within its range, (estimate − lower)/(upper − lower), which must lie between 0 and 1. Finally I read the 2013 column of the paper's Figure 3A, which the Results cite for this estimate. I fitted a straight line to the pixel rows of the eight labelled y-axis tick marks (0 to 70,000) on the PMC image ede-27-247-g004.jpg; one pixel row is 165 people, and the largest tick-mark residual is 154. I then converted the measured rows of the grey bar and the diamond. Every number below is printed by recompute.py.txt.
In the table, Results paragraphs are counted from 1 within the subsection "Calibration to Data on MSM in the UK and Resulting Estimates for 2013".
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| MSM living with HIV, 2013: is the estimate inside its range? | Abstract; Results, 3rd paragraph | 48,310 (39,900–45,560) | No. 48,310 − 45,560 = 2,750 above the upper limit |
| Lower limit to estimate | Abstract; Results, 3rd paragraph | 39,900 | 8,410 below the estimate; possible |
| Required upper limit (95th percentile ≥ median) | Methods, 3rd paragraph of "How the Model Is Calibrated to Country Data" | 45,560 | must be ≥ 48,310 |
| Sum of CD4 strata, 2013 | Results, 3rd paragraph | 2,300 + 5,500 + 11,830 + 28,680 | 48,310, equal to the point estimate |
| Sum of undiagnosed CD4 strata, 2013 | Results, 3rd paragraph | 710 + 1,930 + 3,040 + 4,720 | 10,400, equal to the undiagnosed estimate |
| Undiagnosed proportion | Results, 3rd paragraph | 22% (13%–36%) | 10,400 / 48,310 = 21.5%, rounds to 22% |
| Undiagnosed MSM, 2013 | Abstract; Results, 3rd paragraph | 10,400 (6,160–17,350) | inside; position 0.38 |
| Infections per year, 2010–2013 | Abstract; Results, 2nd paragraph | 3,210 (1,730–5,350) | inside; position 0.41 |
| Cumulative infections by end of 2012 | Results, 2nd paragraph | 67,720 (56,470–78,800) | inside; position 0.50 |
| Asymptomatic diagnosis rate per 3 months | Results, 2nd paragraph | 0.042 (0.025–0.069) | inside; position 0.39 |
| MSM living with HIV, 2013, as printed | Abstract; Results, 3rd paragraph | 48,310 (39,900–45,560) | position 1.49 |
| Figure 3A, 2013 column, read off | Figure 3A | median and 90% range drawn, no numbers | median about 48,500; range about 40,000–60,400 (position 0.42) |
| Figure 3A, 2012 and 2011 columns, read off | Figure 3A | as above | 2012: about 46,100 (38,200–56,400); 2011: about 43,800 (36,200–52,300) |
| Figure 3A minus text, 2013 | Figure 3A vs Abstract | — | median +190, lower limit +100, upper limit +14,840 |
| Check of the reading method: Figure 3B, undiagnosed, 2013 | Figure 3B vs Results, 3rd paragraph | 10,400 (6,160–17,350) | read off as about 10,400 (6,100–17,400) |
| eAppendix row described as "the same as the main result" | eAppendix eTable 6 (threshold 0.20, weights A) | 121 parameter sets; 51,000 (41,400–61,000) | upper limit above the median; but the set count and median differ from the main text (742 sets, 48,310) |
| Upper limits in all nine sensitivity scenarios | eAppendix eTable 6 | 59,700 to 61,500 | none is below 48,310 |
The point estimate fits every other number in the Results, and the lower limit agrees with Figure 3A to within reading precision. Only the upper limit fails, and Figure 3A, which plots the same quantity from the same model runs, puts it at about 60,400, not 45,560. The figure reading is approximate (about ±200 people), but the difference from the text is about 74 times larger than that. The same reading method applied to the 2013 column of Figure 3B (undiagnosed) gives about 10,400 (6,100–17,400), against the printed 10,400 (6,160–17,350), so the method recovers a range the text prints correctly.
Figure
Figure 1. MSM living with HIV in the UK in 2013, on one axis of people (the axis starts at 30,000; no bar lengths are compared). Orange: the estimate and 90% plausibility range printed in the Abstract and Results, 48,310 (39,900–45,560). The printed upper limit lies 2,750 to the left of the estimate, which the Methods' definition (median ≤ 95th percentile) does not allow. Blue: the same quantity read from the 2013 column of the paper's Figure 3A, about 48,500 (40,000–60,400). Grey: the eAppendix eTable 6 row that the supplement describes as the main result, 51,000 (41,400–61,000), shown for context; it comes from 121 parameter sets, not the 742 the main text uses. The dashed line marks the reported estimate. The figure reading and the supplement agree that the upper limit is near 60,000.
Figure 2. For each estimate with a 90% range in the Results, the position of the point estimate within its range, (estimate − lower)/(upper − lower). A median must fall between 0 (the lower limit) and 1 (the upper limit). The five other ranges in the Results (blue) put the estimate at 0.38 to 0.50, as does the Figure 3A reading for the 2013 total (0.42). The printed 2013 total (orange) is at 1.49, outside its own range.
Why it matters
- It is the paper's headline applied number. The range appears in the Abstract and in the Results paragraph that reports the main estimate for the UK. Readers who quote the Abstract will carry an impossible interval, and the error is not obvious unless the reader checks the ordering.
- As printed, it understates uncertainty, all on one side. The printed range spans 5,660 people and ends below the estimate. The paper's own figure suggests a span of about 20,400 people, about 12,090 of them above the reported estimate. A reader who used the printed interval, for example to plan services or to compare with other UK estimates, would wrongly conclude that values above 45,560 are implausible under the model.
- The method's stated contribution is uncertainty ranges. The Discussion says the method naturally produces "plausibility ranges for these estimates, which describe the uncertainty surrounding the estimates". The main applied range is the paper's showcase of that feature.
- What is not affected. The point estimate of 48,310, the CD4 breakdown, the undiagnosed estimate of 10,400 (6,160–17,350), the incidence and diagnosis-rate estimates and their ranges, the pseudo-data validation (Figure 5), and the conclusions about data availability do not depend on the printed upper limit. Figure 3A itself appears correct.
Explanations considered
- A different definition of the estimate or the range. This does not account for the numbers. The Methods name the median and the 5th and 95th percentiles of one distribution of model outputs. A median cannot exceed its own 95th percentile under any distribution, so no reading of the definition allows the printed range. A mean outside a percentile interval would be possible in principle, but the paper calls the estimate a median, and Figure 3's legend says "Modeled median".
- The two numbers refer to different quantities, years or runs. This does not account for the numbers. Both sentences give the estimate and range together, for the same quantity (total MSM living with HIV in the UK, 2013), and both cite Figure 3, whose 2013 median matches 48,310. The "based on prior distributions" outputs and the pseudo-epidemic results (Figure 5) are separate and are not the source.
- The point estimate is the wrong number, not the upper limit. This is ruled out by the paper's other numbers. 48,310 is the exact sum of the four CD4 strata, it gives the stated 22% undiagnosed, it is repeated in the Discussion, and it matches the Figure 3A median to within reading precision. The lower limit matches Figure 3A as well. Only the upper limit disagrees.
- A conversion artifact in the PMC text. This is excluded. The same "48,310 (90% plausibility range: 39,900–45,560)" appears in the PMC-archived text, on the PMC article page and in the Europe PMC full-text XML (the last two fetched on 2026-10-23), and "48,310 (90% plausibility range:39,900-45,560)" appears in the Abstract and Results of the manuscript file ede-27-247-s001.docx, which is independent of the PMC text conversion. The publisher's typeset PDF was not re-verified for this write-up (see Status).
- A typographical slip in one place. The same range appears in the Abstract, the Results and both places in the manuscript version supplied as supplementary file ede-27-247-s001.docx. So the error was present before typesetting and was carried into both places. The published material does not show how it arose. Possible correct values include a limit near 60,400 (Figure 3A) or 61,000 (the eTable 6 row described as the main result), but the exact figure cannot be recovered from the paper.
- A side observation, not needed for the main finding. The eAppendix states that its eTable 6 row for threshold 0.20 and weights A "is the same as the main result presented in the main manuscript", yet that row reports 121 parameter sets and 51,000 (41,400–61,000), while the main text reports 742 sets and 48,310. The supplement's own description of the calibration (page 20) says "The results in the manuscript are based on the 742 runs", so eTable 6 may report parameter sets rather than resampled runs, or a separate run. Either way, its upper limit, like those of all nine sensitivity scenarios (59,700 to 61,500), is well above 48,310.
Status
- Version of record. The archived PMC text, the PMC article page and the Europe PMC full-text XML (the last two fetched on 2026-10-23) all print 48,310 (90% plausibility range: 39,900–45,560). The publisher's typeset PDF was not re-verified: the Europe PMC PDF links and journals.lww.com (where the doi.org link resolves) refused automated access on 2026-10-23. Every quote above was confirmed verbatim against the PMC-archived text with
src/show.py, except the eAppendix sentences, which I read from the supplement PDF. - Figure. I read Figure 3 from the image ede-27-247-g004.jpg (800 × 1244 px; the same file is in the Europe PMC supplementaryFiles zip) and measured the 2011–2013 columns of panel A pixel by pixel. The pixel rows are recorded in
recompute.py.txt. The image is not reproduced here. Read the same way, panel B's 2013 column gives about 10,400 (6,100–17,400), which matches the printed undiagnosed estimate 10,400 (6,160–17,350). - Supplements. (a) The eAppendix (the article's Supplemental Digital Content 1, links.lww.com/EDE/A994, 30 pages), downloaded on 2026-10-23: I read page 20 (742 runs) and eTable 6 (pages 23–24), including the sentence on page 24 that the threshold 0.20, weights A row "is the same as the main result". The eAppendix does not print 45,560 anywhere, and it does not resolve the discrepancy. (b) The PMC supplementary file ede-27-247-s001.docx is a manuscript version of the article. I checked it directly on 2026-10-23, from the zip returned by the Europe PMC supplementaryFiles service (www.ebi.ac.uk/europepmc/webservices/rest/PMC4733816/supplementaryFiles). It prints "48,310 (90% plausibility range:39,900-45,560)" in both its Abstract and its Results, and 48,310 again in its Discussion.
- Corrections. One erratum exists: Epidemiology 2016;27(3):e24, doi:10.1097/EDE.0000000000000474, PMC6855333, PMID 27043876. I read its PMC page on 2026-10-23. It corrects only the Open Access footnote, to a CC BY 4.0 licence statement, and does not mention the estimate or its range. Crossref lists no update-to, updated-by or relation entries for the article. PMC and Europe PMC list no other notice, and there is no retraction or expression of concern.
- Comments. Not re-verified for this write-up: a PubPeer search on 2026-10-23 was refused by a human-verification check, and no Retraction Watch search was recorded.
- As of 2026-10-23, the range 39,900–45,560 is uncorrected.
- Sources. PMC4733816 (archived XML and article page); Europe PMC full-text XML and supplementaryFiles zip (manuscript docx and Figure 3 image ede-27-247-g004.jpg); eAppendix (links.lww.com/EDE/A994); erratum PMC6855333; Crossref.
Suggested correction
- Replace the upper limit of the 2013 estimate in both the Abstract and the Results ("48,310 (90% plausibility range: 39,900–45,560)") with the 95th percentile of the 742 accepted runs. The paper's Figure 3A puts it at about 60,400, so the corrected text would read about "48,310 (90% plausibility range: 39,900–60,400)" if the figure is right. The authors' model output is needed for the exact value.
- Confirm that the lower limit (39,900) and the point estimate (48,310) were taken from the same output. Both match Figure 3A, so no change is expected.
- Optionally, clarify in the eAppendix how the eTable 6 row described as "the same as the main result" (121 parameter sets; 51,000, 41,400–61,000) relates to the main text's 742 runs and 48,310.
Recomputation script (python recompute.py.txt)
"""Recompute the 2013 estimate of MSM living with HIV in the UK and its 90% plausibility
range, PMC4733816 (Epidemiology 2016;27(2):247-256, doi:10.1097/EDE.0000000000000423).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png, fig2.svg/png and five candidate designs in candidates/ next to this file.
Inputs are copied from the paper as archived in PubMed Central:
Abstract and Results ("Calibration to Data on MSM in the UK ...", first to third paragraphs)
Methods ("How the Model Is Calibrated to Country Data", third paragraph): the point estimate is
the median, and the range limits are the 5th and 95th percentiles of one distribution
Figure 3A (image ede-27-247-g004.jpg, 800 x 1244 px): pixel rows measured once from the
image and entered below; the image itself is not redrawn
eAppendix (links.lww.com/EDE/A994), eTable 6: sensitivity analysis
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ------------------------------------------------------------------ inputs from the paper
EST, LO, HI = 48310, 39900, 45560 # Abstract; Results, 3rd paragraph
CD4_TOTAL = [2300, 5500, 11830, 28680] # Results, 3rd paragraph: <=200, 201-350, 351-500, >500
CD4_UNDIAG = [710, 1930, 3040, 4720] # Results, 3rd paragraph, undiagnosed
CD4_LABELS = ["CD4 ≤200", "201–350", "351–500", ">500"]
OTHER = [ # (label, location, estimate, lower, upper)
("Undiagnosed MSM, 2013", "Abstract; Results, 3rd paragraph", 10400, 6160, 17350),
("Undiagnosed proportion, 2013 (%)", "Results, 3rd paragraph", 22, 13, 36),
("Infections per year, 2010–2013", "Abstract; Results, 2nd paragraph", 3210, 1730, 5350),
("Cumulative infections by end 2012", "Results, 2nd paragraph", 67720, 56470, 78800),
("Asymptomatic diagnosis rate per 3 months", "Results, 2nd paragraph", 0.042, 0.025, 0.069),
]
# eAppendix eTable 6, row "threshold 0.20, weight A", which the eAppendix says "is the same as
# the main result presented in the main manuscript"
ETAB6_MAIN = dict(sets=121, est=51000, lo=41400, hi=61000)
ETAB6_ALL_HI = [60000, 60700, 60400, 61000, 61200, 60400, 61500, 59700, 61200] # 9 scenarios
MAIN_SETS = 742 # Results, 1st paragraph
# Figure 3A: pixel rows of the labelled y-axis tick marks (y, value) and the 2011-2013 columns, measured on the image
GRID = [(46, 70000), (107.5, 60000), (167.5, 50000), (228.5, 40000),
(288.5, 30000), (349.5, 20000), (411.5, 10000), (471, 0)]
FIG3A = { # year: (top row of grey bar, bottom row of grey bar, diamond top row, diamond bottom row)
2011: (154, 251, 201, 210),
2012: (129, 239, 187, 196),
2013: (105, 228, 173, 181),
}
# Figure 3B (undiagnosed), same image: y-axis tick-mark rows and the 2013 column, used as a check of the
# reading method against a range the text prints (10,400; 6,160-17,350)
GRID_B = [(696.5, 20000), (801.5, 15000), (908.5, 10000), (1015.5, 5000), (1121, 0)]
FIG3B_2013 = (752, 991, 895, 905)
# ------------------------------------------------------------------ calculations
print("=== 2013 MSM living with HIV: reported value and range ===")
print(f"Reported: {EST:,} (90% range {LO:,}-{HI:,})")
print(f"Estimate inside its own range? {LO <= EST <= HI}")
print(f"Estimate minus upper limit: {EST - HI:,}")
print(f"Estimate minus lower limit: {EST - LO:,}")
print(f"Reported range width: {HI - LO:,}")
print("\n=== Order constraint implied by the Methods (median <= 95th percentile) ===")
rng = np.random.default_rng(0)
ok = True
for name, draw in [("normal", rng.normal(48310, 5000, 10**5)),
("lognormal", rng.lognormal(np.log(48310), 0.2, 10**5)),
("right-skewed exponential", rng.exponential(1, 10**5)),
("discrete mixture", rng.choice([1, 2, 50], size=10**5, p=[.6, .3, .1]))]:
p5, p50, p95 = np.percentile(draw, [5, 50, 95])
ok &= p5 <= p50 <= p95
print(f" {name:25s} P5={p5:10.2f} P50={p50:10.2f} P95={p95:10.2f} ordered={p5 <= p50 <= p95}")
print(f"Ordering holds in every case: {ok} (it holds for any distribution by definition)")
print(f"So the 95th percentile must be >= {EST:,}; the printed {HI:,} is not.")
print("\n=== Is 48,310 the sound number? ===")
s_tot, s_und = sum(CD4_TOTAL), sum(CD4_UNDIAG)
print(f"Sum of CD4 strata: {' + '.join(f'{v:,}' for v in CD4_TOTAL)} = {s_tot:,} (reported {EST:,})")
print(f"Sum of undiagnosed CD4 strata: {' + '.join(f'{v:,}' for v in CD4_UNDIAG)} = {s_und:,} (reported 10,400)")
prop = 100 * 10400 / EST
print(f"Undiagnosed proportion 10,400 / 48,310 = {prop:.1f}% (reported 22%)")
print(f"Mirror of the lower gap above the estimate: {EST:,} + {EST - LO:,} = {EST + (EST - LO):,} (indicative only)")
print("\n=== Every other estimate-with-range in the Results ===")
pos = {}
for lab, loc, e, lo, hi in OTHER:
p = (e - lo) / (hi - lo)
pos[lab] = p
print(f" {lab:42s} {e} ({lo}-{hi}) inside={lo <= e <= hi} position in range={p:.2f}")
p_main = (EST - LO) / (HI - LO)
print(f" {'MSM living with HIV, 2013':42s} {EST} ({LO}-{HI}) inside={LO <= EST <= HI} position in range={p_main:.2f}")
print("\n=== Figure 3A read-off (paper's own chart of the same quantity) ===")
gy, gv = np.array(GRID).T
slope, icpt = np.polyfit(gy, gv, 1)
resid = gv - (slope * gy + icpt)
print(f"Calibration: {slope:.1f} people per pixel row; max tick-mark residual {np.abs(resid).max():.0f}")
fig3 = {}
for yr, (top, bot, d0, d1) in FIG3A.items():
hi_v = slope * (top - 0.5) + icpt
lo_v = slope * (bot + 0.5) + icpt
med_v = slope * ((d0 + d1) / 2) + icpt
fig3[yr] = (round(med_v, -2), round(lo_v, -2), round(hi_v, -2))
print(f" {yr}: median about {fig3[yr][0]:,.0f}; 90% range about {fig3[yr][1]:,.0f}-{fig3[yr][2]:,.0f}")
f_med, f_lo, f_hi = fig3[2013]
print(f"Figure 3A 2013 upper limit minus text upper limit: about {f_hi - HI:,.0f}")
print(f"Figure 3A 2013 lower limit minus text lower limit: about {f_lo - LO:,.0f}")
print(f"Figure 3A 2013 median minus text estimate: about {f_med - EST:,.0f}")
print(f"Reading precision: one pixel row = {abs(slope):.0f} people")
print(f"Figure 3A 2013 range width: about {f_hi - f_lo:,.0f}; above the reported estimate: about {f_hi - EST:,.0f}")
print(f"Printed range width: {HI - LO:,}; text-vs-figure upper-limit gap / 200-person reading error = {(f_hi - HI) / 200:.0f}")
gyb, gvb = np.array(GRID_B).T
sb, ib = np.polyfit(gyb, gvb, 1)
t, bt, d0, d1 = FIG3B_2013
print(f"Check on Figure 3B (undiagnosed, 2013): median about {round(sb * (d0 + d1) / 2 + ib, -2):,.0f}; "
f"range about {round(sb * (bt + 0.5) + ib, -2):,.0f}-{round(sb * (t - 0.5) + ib, -2):,.0f} "
f"(text: 10,400; 6,160-17,350); one pixel row = {abs(sb):.0f} people")
print("\n=== eAppendix eTable 6 ===")
print(f"Row described as the main result: {ETAB6_MAIN['sets']} parameter sets, "
f"{ETAB6_MAIN['est']:,} ({ETAB6_MAIN['lo']:,}-{ETAB6_MAIN['hi']:,}); main text: {MAIN_SETS} sets, {EST:,} ({LO:,}-{HI:,})")
print(f"Upper limits across all 9 sensitivity scenarios: {min(ETAB6_ALL_HI):,}-{max(ETAB6_ALL_HI):,}")
print(f"Number of scenarios with upper limit below {EST:,}: {sum(h < EST for h in ETAB6_ALL_HI)}")
# ------------------------------------------------------------------ figure style
BG, INK, INK2, GRIDC = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, GREY = "#eb6834", "#2a78d6", "#9a9893"
plt.rcParams.update({
"font.family": ["Inter", "DejaVu Sans"], "font.size": 10.5,
"axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"text.color": INK, "axes.spines.top": False, "axes.spines.right": False,
"svg.fonttype": "none",
})
W = 7.2
def save(fig, path_noext):
fig.savefig(path_noext + ".svg", bbox_inches="tight")
fig.savefig(path_noext + ".png", dpi=200, bbox_inches="tight")
plt.close(fig)
def kfmt(ax):
ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, _: f"{v:,.0f}"))
# ---- Candidate A: dot-and-interval rows on one axis --------------------------------------
def cand_a(path):
fig, ax = plt.subplots(figsize=(W, 3.9))
rows = [
("Reported in Abstract and Results", ORANGE, EST, LO, HI, "o"),
("Paper's Figure 3A, 2013 column (read off)", BLUE, f_med, f_lo, f_hi, "D"),
("eAppendix eTable 6, row called the main result", GREY, ETAB6_MAIN["est"],
ETAB6_MAIN["lo"], ETAB6_MAIN["hi"], "s"),
]
ys = [3, 1.7, 0.4]
ax.axvline(EST, color=INK2, lw=0.8, ls=(0, (3, 3)), zorder=1)
ax.text(EST, 3.72, f"reported estimate {EST:,}", ha="center", va="bottom", color=INK2, fontsize=10,
bbox=dict(fc=BG, ec="none", pad=1))
for (lab, col, e, lo, hi, m), y in zip(rows, ys):
ax.plot([lo, hi], [y, y], color=col, lw=2.5, solid_capstyle="butt", zorder=2)
for x in (lo, hi):
ax.plot([x, x], [y - 0.12, y + 0.12], color=col, lw=2, zorder=2)
ax.plot(e, y, m, color=col, ms=9, mec=BG, mew=1.5, zorder=3)
ax.text(30500, y + 0.36, lab, ha="left", va="center", fontsize=10.5, color=INK, zorder=4,
bbox=dict(fc=BG, ec="none", pad=1.5))
ax.text(max(hi, e) + 900, y, f"{e:,.0f} ({lo:,.0f}\u2013{hi:,.0f})", ha="left", va="center",
fontsize=10, color=INK2)
# gap between the printed upper limit and the estimate
ax.annotate("", xy=(EST, 3 - 0.36), xytext=(HI, 3 - 0.36),
arrowprops=dict(arrowstyle="<->", color=ORANGE, lw=1.5, shrinkA=0, shrinkB=0))
ax.text(EST + 900, 3 - 0.36, f"upper limit is {EST - HI:,} below the estimate",
ha="left", va="center", fontsize=10, color=INK)
ax.set_ylim(-0.05, 4.05)
ax.set_xlim(30000, 72000)
ax.set_yticks([])
ax.spines["left"].set_visible(False)
ax.grid(axis="x", color=GRIDC, lw=0.8)
ax.set_axisbelow(True)
kfmt(ax)
ax.set_xlabel("MSM living with HIV in the UK, 2013 (median and 90% plausibility range)")
from matplotlib.lines import Line2D
h = [Line2D([], [], color=ORANGE, marker="o", lw=2.5, ms=8, label="Reported in the text"),
Line2D([], [], color=BLUE, marker="D", lw=2.5, ms=8, label="Read from the paper's own Figure 3A"),
Line2D([], [], color=GREY, marker="s", lw=2.5, ms=8, label="Supplement, eTable 6 (121 sets)")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.22), ncol=3, frameon=False,
fontsize=10, handlelength=2.2, columnspacing=1.2)
save(fig, path)
# ---- Candidate B: one number line with the order constraint -----------------------------
def cand_b(path):
fig, ax = plt.subplots(figsize=(W, 2.6))
ax.set_xlim(36000, 52000)
ax.set_ylim(-1.3, 1.75)
ax.axhline(0, color=INK2, lw=1)
marks = [(LO, "lower limit\n(5th percentile)", 39900), (HI, "upper limit\n(95th percentile)", 45560),
(EST, "point estimate\n(median)", 48310)]
for x, lab, _ in marks:
ax.plot(x, 0, "o", color=ORANGE, ms=10, mec=BG, mew=1.5, zorder=3)
ax.text(x, 0.28, f"{x:,}", ha="center", va="bottom", fontsize=11, fontweight="bold")
ax.text(x, -0.25, lab, ha="center", va="top", fontsize=10, color=INK2)
ax.annotate("", xy=(HI, 0.95), xytext=(EST, 0.95), arrowprops=dict(arrowstyle="->", color=ORANGE, lw=2))
ax.text((HI + EST) / 2, 1.03, f"the 95th percentile is {EST - HI:,} below the median",
ha="center", va="bottom", fontsize=10.5)
ax.text(36100, -1.2, "Required by the Methods' definition, for any distribution: "
"5th percentile ≤ median ≤ 95th percentile", ha="left", va="bottom", fontsize=10, color=BLUE)
ax.text(36100, 1.6, "Reported values (Abstract; Results), in order along one axis", ha="left", va="bottom", fontsize=10, color=INK2)
ax.set_yticks([])
for s in ("left", "bottom"):
ax.spines[s].set_visible(False)
ax.set_xticks([])
save(fig, path)
# ---- Candidate C: text panels joined by an annotated connector --------------------------
def cand_c(path):
fig, ax = plt.subplots(figsize=(W, 5.0))
ax.set_xlim(0, 10)
ax.set_ylim(0, 6.85)
ax.axis("off")
def panel(x, y, w, h, title, body, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.12",
fc="#ffffff", ec=edge, lw=1.6))
ax.text(x + 0.18, y + h - 0.2, title, ha="left", va="top", fontsize=10, color=INK2, fontweight="bold")
ax.text(x + 0.18, y + h - 0.62, body, ha="left", va="top", fontsize=10.5, color=INK, linespacing=1.4)
panel(0.1, 3.6, 4.7, 3.15, "Abstract; Results, 3rd paragraph",
"\u201cIn 2013, 48,310 (90% plausibility\nrange: 39,900\u201345,560) men who\n"
"have sex with men were estimated\nto be living with HIV in the UK\u201d", ORANGE)
panel(5.2, 3.6, 4.7, 3.15, "Methods, model calibration",
"\u201cThe median, 5th and 95th\npercentiles of the distribution of\n"
"these model outputs ... are\nconsidered the point estimate and\nplausibility range limits\u201d", INK2)
panel(0.1, 0.1, 4.7, 2.5, "Results, 3rd paragraph: CD4 strata",
"2,300 + 5,500 + 11,830 + 28,680\n= 48,310, the point estimate\n"
"10,400 / 48,310 = 21.5%,\nthe stated 22% undiagnosed", BLUE)
panel(5.2, 0.1, 4.7, 2.5, "What the definition requires",
"median \u2264 95th percentile, so the\nupper limit must be \u2265 48,310\n"
"printed upper limit 45,560\nfalls 2,750 short", BLUE)
for xc in (2.45, 7.55):
ax.plot([xc, xc], [3.6, 3.32], color=ORANGE, lw=1.6)
ax.plot([xc, xc], [2.78, 2.6], color=BLUE, lw=1.6, ls=(0, (2, 2)))
ax.text(5.0, 3.05, "conflict: median 48,310 > 95th percentile 45,560",
ha="center", va="center", fontsize=10.5, color=INK, fontweight="bold",
bbox=dict(boxstyle="round,pad=0.35", fc=BG, ec=ORANGE, lw=1.5))
save(fig, path)
# ---- Candidate D: where each estimate sits inside its own range -------------------------
def cand_d(path):
labs = [o[0] for o in OTHER] + ["MSM living with HIV, 2013 (Fig. 3A read-off)",
"MSM living with HIV, 2013 (as printed)"]
vals = [pos[o[0]] for o in OTHER] + [(f_med - f_lo) / (f_hi - f_lo), p_main]
cols = [BLUE] * (len(OTHER) + 1) + [ORANGE]
fig, ax = plt.subplots(figsize=(W, 3.6))
y = np.arange(len(labs))[::-1]
ax.axvspan(0, 1, color="#f0efeb", zorder=0)
for xv, t in ((0, "lower limit"), (1, "upper limit")):
ax.axvline(xv, color=INK2, lw=1, zorder=1)
ax.text(xv, len(labs) - 0.35, t, ha="center", va="bottom", fontsize=10, color=INK2,
bbox=dict(fc=BG, ec="none", pad=1.5))
for yi, v, c, lab in zip(y, vals, cols, labs):
ax.plot([0, 1], [yi, yi], color=GRIDC, lw=1, zorder=1)
ax.plot(v, yi, "o", color=c, ms=9, mec=BG, mew=1.5, zorder=3)
ax.text(v + 0.04, yi, f"{v:.2f}", va="center", ha="left", fontsize=10, color=INK2)
ax.set_yticks(y)
ax.set_yticklabels(labs, fontsize=10)
ax.tick_params(axis="y", length=0)
ax.spines["left"].set_visible(False)
ax.set_xlim(-0.1, 1.7)
ax.set_ylim(-0.6, len(labs) - 0.1)
ax.set_xticks([0, 0.5, 1, 1.5])
ax.set_xlabel("Position of the point estimate within its own 90% range\n"
"(estimate − lower) / (upper − lower); must lie between 0 and 1")
from matplotlib.lines import Line2D
h = [Line2D([], [], color=ORANGE, marker="o", lw=0, ms=8, label="Reported 2013 total, as printed"),
Line2D([], [], color=BLUE, marker="o", lw=0, ms=8, label="Other ranges in the paper, and the Fig. 3A read-off")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.3, -0.3), ncol=2, frameon=False, fontsize=10)
save(fig, path)
# ---- Candidate E: calculation flow, CD4 strata summed against the range -----------------
def cand_e(path):
fig, ax = plt.subplots(figsize=(W, 2.9))
left = 0
shades = ["#9cc0ea", "#6fa3e2", "#4a8cdc", BLUE]
for v, lab, c in zip(CD4_TOTAL, CD4_LABELS, shades):
ax.barh(2, v, left=left, height=0.5, color=c, ec=BG, lw=2)
if v > 10000:
ax.text(left + v / 2, 2, f"{lab}\n{v:,}", ha="center", va="center", fontsize=10,
color="#ffffff" if c in (BLUE, "#4a8cdc") else INK)
left += v
ax.text(0, 2.42, f"Sum of the four reported CD4 strata = {s_tot:,}", ha="left", va="bottom", fontsize=10.5)
ax.text(0, 1.68, f"{CD4_LABELS[0]}: {CD4_TOTAL[0]:,} {CD4_LABELS[1]}: {CD4_TOTAL[1]:,}",
ha="left", va="top", fontsize=10, color=INK2)
ax.plot([LO, HI], [1, 1], color=ORANGE, lw=2.5)
for x in (LO, HI):
ax.plot([x, x], [0.85, 1.15], color=ORANGE, lw=2)
ax.plot(EST, 1, "o", color=ORANGE, ms=9, mec=BG, mew=1.5)
ax.text(0, 1, "Reported 90% range\n39,900–45,560 and estimate", ha="left", va="center", fontsize=10.5)
ax.axvline(EST, color=INK2, lw=0.8, ls=(0, (3, 3)))
ax.text(EST + 400, 0.5, f"{EST:,}", ha="left", va="center", fontsize=10, color=INK2)
ax.set_xlim(0, 60000)
ax.set_ylim(0.3, 2.85)
ax.set_yticks([])
ax.spines["left"].set_visible(False)
ax.grid(axis="x", color=GRIDC, lw=0.8)
ax.set_axisbelow(True)
kfmt(ax)
ax.set_xlabel("MSM living with HIV in the UK, 2013")
ax.text(HI - 200, 1.22, "upper limit ends 2,750 short of the estimate", ha="right", va="bottom",
fontsize=10, color=INK)
save(fig, path)
cand_a(os.path.join(CAND, "cand_A_interval_rows"))
cand_b(os.path.join(CAND, "cand_B_order_line"))
cand_c(os.path.join(CAND, "cand_C_text_panels"))
cand_d(os.path.join(CAND, "cand_D_position_in_range"))
cand_e(os.path.join(CAND, "cand_E_cd4_flow"))
cand_a(os.path.join(HERE, "fig1"))
cand_d(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE, "and", CAND)
Abstract reports a 31% fall in endotoxin activity on the prudent diet; Results report 38%
Gastroenterology (2012) · doi:10.1053/j.gastro.2012.01.034 · PMC3978718
Summary
This short crossover study fed 8 healthy inpatients a Western-style diet and a prudent-style diet for 1 month each and measured plasma endotoxin activity before and after each diet. The abstract says the prudent-style diet reduced endotoxin activity by 31%. The Results paragraph that cites Figure 1 says the decrease was 38%. Both passages describe the same subjects, assay and before/after comparison, and both give the same +71% for the Western-style diet. Rounding cannot produce a 7-point gap. The paper prints no group means for endotoxin activity, only the individual-subject plots in Figure 1. Reading those plots by eye gives a 37.5% fall in the group mean, which supports the Results figure. The abstract's 31% matches no single summary method that also gives +71% for the Western diet. The abstract is the number most likely to be quoted, so the size of the prudent-diet effect is now ambiguous in the literature. The direction of the effect, the Western-diet result and the paper's conclusions are unaffected.
What the paper reports
Abstract:
Placing 8 healthy subjects on a Western-style diet for 1 month induced a 71% increase in plasma levels of endotoxin activity (endotoxemia), whereas a prudent-style diet reduced levels by 31%.
Main text, Results paragraph (the paragraph that cites Figure 1 and Table 1):
Our study shows that 4 weeks of feeding a Western-style diet induced highly significant increases (by 71%) in plasma endotoxin activity levels, in contrast to a major decrease (by 38%) after the prudent-style diet (Figure 1) (P < .001).
Main text, same paragraph:
Baseline endotoxin activity levels were about identical
Main text, study description. This shows that both passages concern one group of subjects and one design:
Briefly, 8 healthy subjects were fed a Western-style and a “prudent-style” diet for 1 month as inpatients hospitalized in a metabolic ward in a crossover study separated by a 1-month washout period.
Figure 1 legend:
Data from 8 individual subjects shown as endotoxin activity units for the Western-style diet (WD) and the prudent-style diet (PD). Paired t tests were used to compare effects of the diets on endotoxin activity levels. The Mann–Whitney sum test was used to compare effects of the 2 diets on endotoxin activity levels.
In the Figure 1 image, panels A (Western diet, before vs after) and B (prudent diet, before vs after) are labelled "P≤.05". Panel D, which compares the two diets after the intervention, is labelled "P≤.001". Neither the text nor the figure gives a group mean, SD or percentage for endotoxin activity. Table 1 gives cytokine concentrations only, as before and after means with a "Change (%)" column.
What the paper's own numbers show
Method. The paper reports the endotoxin results only as two percentages and as individual-subject lines in Figure 1, panels A and B (8 subjects each). I read each subject's before and after values off the enlarged figure image by eye. Each reading is uncertain by about ±0.005 activity units. These readings are approximations, not the authors' data. From them I computed the percentage change in four standard ways: - the change in the group mean (mean after / mean before − 1); - the mean of each subject's own percentage change; - the geometric mean of the after/before ratios; - the change in the median.
To carry the reading error through, I added independent uniform ±0.005 noise to every point, repeated this 20,000 times and took the 2.5–97.5 percentile band of each statistic. As a check on which method the authors used elsewhere, I recomputed Table 1's cytokine percentages from its rounded means. All calculations are in recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Prudent diet, change in endotoxin activity | Abstract | −31% | — (same quantity as the next row) |
| Prudent diet, change in endotoxin activity | Results paragraph citing Fig. 1 | −38% | — |
| Gap between the two printed values | Abstract vs Results | — | 7 percentage points (23% of the abstract value); post/baseline ratio 0.69 vs 0.62 |
| Western diet, change in endotoxin activity | Abstract and Results | +71% (both) | — |
| Prudent diet, group mean before → after | Fig. 1B (read by eye) | not reported | 0.115 → 0.072 units |
| Western diet, group mean before → after | Fig. 1A (read by eye) | not reported | 0.120 → 0.207 units |
| Prudent diet, change of group means | Fig. 1B | −31% / −38% | −37.5% (band −39.5 to −35.4) |
| Western diet, change of group means | Fig. 1A | +71% | +72.2% (band +69.0 to +75.5) |
| Prudent diet, mean of per-subject % changes | Fig. 1B | — | −33.3% (band −35.9 to −30.4) |
| Western diet, mean of per-subject % changes | Fig. 1A | — | +80.2% (band +75.6 to +85.3) |
| Prudent / Western, geometric mean ratio | Fig. 1A–B | — | −35.7% (−38.4 to −33.1) / +71.3% (+66.9 to +75.9) |
| Prudent / Western, change of medians | Fig. 1A–B | — | −28.7% (−33.5 to −24.1) / +78.3% (+71.8 to +85.5) |
| Within-diet paired t test, PD / WD | Results text; Fig. 1A–B labels | "P < .001" (text); "P≤.05" (figure) | p ≈ 0.020 / 0.017 (from figure readings) |
| Table 1, PD cytokines: TNF-α, MCP-1, IL-6, IL-8 | Table 1 | −17, −9, −11, −28% | −16.0, −8.7, −12.5, −26.3% (change of rounded means) |
| Table 1, WD cytokines: TNF-α, MCP-1, IL-6, IL-8 | Table 1 | −1, −9, −2, −13% | 0.0, −9.1, 0.0, −11.8% (change of rounded means) |
Three findings stand out: - The change in the group mean, read off Figure 1, reproduces both the Results' −38% and the shared +71%. - The geometric mean ratio also gives values close to −38% and +71%. - The only methods whose bands reach −31% are the mean of per-subject changes and the change of medians. Both give +78% to +80% for the Western diet, not the +71% printed beside the 31% in the abstract.
Table 1's printed percentages agree, within the rounding of its means, with the change-of-means method. This fits the 38% value, although rounded means cannot rule out other methods.
Figure
The two passages that report the prudent-diet result, as printed in the archived article. In orange are the paper's values: the abstract gives a 31% decrease and the Results paragraph citing Figure 1 gives a 38% decrease. Both concern the same 8 subjects, the same neutrophil-priming assay and the same before/after comparison, and both pair the value with an identical +71% for the Western-style diet. The footer gives the change of group means computed in recompute.py.txt from Figure 1 values read by eye.
Percentage change in endotoxin activity recomputed from the individual values in Figure 1, read by eye (blue dots), under four summary methods. The bars show the 2.5–97.5 percentile range when each reading is perturbed by ±0.005 units. The dashed orange lines are the printed values: −31% (abstract), −38% (Results) and +71% (both). The left cluster is the prudent diet and the right cluster the Western diet, on one shared axis. The change of group means and the geometric mean ratio match −38% and +71% together. No method matches −31% and +71% together.
Why it matters
The abstract is where readers, reviews and meta-analyses usually take effect sizes from. Here it gives a prudent-diet reduction (31%, ratio 0.69) that differs from the body (38%, ratio 0.62) by 7 percentage points, or about 23% of the abstract value. A reader who compares the two diets gets a different contrast depending on the source: a post-diet ratio of 1.71/0.69 ≈ 2.48 between diets if the abstract is right, against 1.71/0.62 ≈ 2.76 if the body is right. Because the paper prints no group means or SDs for endotoxin activity, a reader cannot tell from the article which value is correct. Anyone sizing a follow-up study or pooling this result has to choose between two figures the paper gives for the same quantity.
The error is limited in scope: - The Western-diet +71%, the result named in the title, is the same in both places. - The prudent diet lowers endotoxin activity in both statements. - The cytokine results in Table 1 and the paper's qualitative interpretation do not depend on which value is correct.
A smaller, related point concerns the P value. The Results sentence attaches "P < .001" to the within-diet changes. The Figure 1 panels for those changes (A and B) are labelled "P≤.05", and the figure readings give paired-t p-values of about 0.02. The "P≤.001" label appears only on panel D, the between-diet comparison after the intervention. The Results sentence may be citing the panel D value. As written, though, it reads as applying to the within-diet changes.
Explanations considered
- Different populations, time points, assays or units. Ruled out. The paper describes one cohort of 8 subjects, one crossover with 4 weeks on each diet, and one assay, with endotoxin activity measured before and after each diet. Both passages give the same +71% for the Western diet, which would be unlikely if they described different analyses.
- Rounding. Ruled out. A value that prints as 31% must lie between 30.5% and 31.5%, and one that prints as 38% must lie between 37.5% and 38.5%. The two ranges do not overlap.
- Different summary methods (change of means, mean of individual changes, geometric mean, medians). These do not reconcile the values. Under the readings of Figure 1, a method that gives about 31% for the prudent diet gives about 78–80% for the Western diet. A method that gives about 71% for the Western diet gives about 36–38% for the prudent diet. No single method produces the abstract's pair (−31%, +71%). It remains possible that the abstract used one method for the prudent diet and another for the Western diet, but the paper does not say so.
- Decrease expressed relative to the post-diet level, or as a ratio. This does not fit. Taking the change of means relative to the post-diet level gives (0.115 − 0.072)/0.072 ≈ 60%, not 31%. A post/baseline ratio of 0.69 is what a 31% decrease means, which is the abstract's claim, not a different quantity.
- Reading error in the figure. This affects only which of the two printed values the figure supports, not the existence of the inconsistency. The two printed values contradict each other whatever the figure shows. The reading-error bands (about ±2 points) are much narrower than the 7-point gap.
- Text conversion artifact in the archived copy. Unlikely. Both values appear as plain text with a % sign, not in tables or superscripts. Both sentences describe the change in words ("reduced", "decrease"), so a dropped sign would not explain the gap. The abstract's 31% also appears in the PubMed record, which derives from the publisher.
- Correction already published. None found (see Status).
Status
As of 2026-10-23:
- Version of record. I could not open the typeset publisher full text. ScienceDirect (pii S0016508512001588) and gastrojournal.org returned HTTP 403 or a bot challenge to automated access. The abstract's "reduced levels by 31%" is confirmed in the PubMed record (PMID 22326433), which derives from the publisher's abstract. The Results sentence with "by 38%" was checked in the PubMed Central copy, which is the NIH author manuscript (NIHMS561466, PMC3978718), using both the archived JATS XML and the rendered PMC page. I could not confirm whether the typeset Results text also reads 38%.
- Supplement. The only file online is one publisher page, NIHMS561466-supplement-supplement_1.pdf. It holds Supplementary Table 1 (diet composition) and Supplementary Table 2 (end-of-diet glucose, insulin and lipids). It contains no endotoxin values, so it cannot settle 31% vs 38%. The "Supplementary Patients and Methods" text cited in the article was not among the files available to me.
- Corrections and retractions. None in Crossref update metadata, PubMed Central notices or Europe PMC notices.
- PubPeer. The PubPeer search page and API require human verification and could not be queried automatically. A web search found no PubPeer thread for this article. I cannot rule out a thread that is not indexed.
- Sources. The PMC author manuscript PMC3978718 (text, Table 1, Figure 1 image nihms561466f1.jpg); PubMed 22326433; Crossref record for doi:10.1053/j.gastro.2012.01.034. All numbers are recomputed in
recompute.py.txt.
Suggested correction
The authors could state which value is correct for the change in plasma endotoxin activity after the prudent-style diet. They could also give the group mean (SD) endotoxin activity before and after each diet, and say how the percentage change was calculated (for example, change of group means or mean of per-subject changes). If the Results value is correct, the abstract would read: "…whereas a prudent-style diet reduced levels by 38%." If the abstract value is correct, the Results sentence would need the same correction. The authors could also say which comparison the "P < .001" in the Results sentence refers to, since Figure 1 labels the within-diet changes "P≤.05" and only the between-diet comparison after the intervention "P≤.001".
Recomputation script (python recompute.py.txt)
"""Recompute the numbers in the PMC3978718 write-up and regenerate fig1/fig2.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
Inputs
------
* Reported values (paper): abstract -31% (prudent diet, PD), +71% (Western
diet, WD); Results paragraph citing Figure 1: -38% (PD), +71% (WD);
Table 1 cytokine means before/after and printed % changes.
* Individual endotoxin activity values: the paper gives these only as plotted
points in Figure 1 (panels A and B, 8 subjects each). The values below were
read by eye from the enlarged figure image (PMC file nihms561466f1.jpg);
reading error is roughly +/-0.005 units per point. They are approximations,
not the authors' data.
"""
import os
import numpy as np
from scipy import stats
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- reported
REP = {"abstract_PD": -31, "results_PD": -38, "abstract_WD": 71, "results_WD": 71}
# ------------------------------------------------- Figure 1, read by eye
# subject order (by plotting symbol): light-blue '+', purple 'x', blue diamond,
# orange circle, green triangle, brown, cyan, red square
pd_before = np.array([0.246, 0.138, 0.132, 0.120, 0.080, 0.077, 0.067, 0.062])
pd_after = np.array([0.120, 0.089, 0.080, 0.0625, 0.094, 0.047, 0.050, 0.034])
wd_before = np.array([0.235, 0.130, 0.172, 0.125, 0.105, 0.078, 0.050, 0.068])
wd_after = np.array([0.290, 0.375, 0.210, 0.200, 0.275, 0.105, 0.100, 0.103])
CONV = {
"Change of group means": lambda b, a: (a.mean() / b.mean() - 1) * 100,
"Mean of per-subject % changes": lambda b, a: np.mean(a / b - 1) * 100,
"Geometric mean ratio": lambda b, a: (np.exp(np.mean(np.log(a / b))) - 1) * 100,
"Change of medians": lambda b, a: (np.median(a) / np.median(b) - 1) * 100,
}
def jitter_band(b, a, f, n=20000, half=0.005, seed=0):
rng = np.random.default_rng(seed)
vals = np.empty(n)
for i in range(n):
bb = b + rng.uniform(-half, half, b.size)
aa = a + rng.uniform(-half, half, a.size)
vals[i] = f(bb, aa)
return np.percentile(vals, [2.5, 97.5])
print("=== Figure 1 (read by eye), endotoxin activity units ===")
for name, b, a in [("Prudent (PD)", pd_before, pd_after), ("Western (WD)", wd_before, wd_after)]:
t = stats.ttest_rel(a, b)
print(f"{name}: mean before {b.mean():.3f}, after {a.mean():.3f}; paired t p = {t.pvalue:.3f}")
print("Paper reports: PD -31% (abstract), -38% (Results); WD +71% (both)")
RES = {}
print("\n=== % change under each summary convention (2.5-97.5% band from +/-0.005 reading error) ===")
for cname, f in CONV.items():
pdv, wdv = f(pd_before, pd_after), f(wd_before, wd_after)
pdb, wdb = jitter_band(pd_before, pd_after, f), jitter_band(wd_before, wd_after, f)
RES[cname] = (pdv, pdb, wdv, wdb)
print(f"{cname:32s} PD {pdv:+6.1f} [{pdb[0]:+.1f}, {pdb[1]:+.1f}] "
f"WD {wdv:+6.1f} [{wdb[0]:+.1f}, {wdb[1]:+.1f}]")
print("\n=== Which conventions reproduce the printed pairs? (|recomputed - reported| <= band) ===")
for cname, (pdv, pdb, wdv, wdb) in RES.items():
ok31 = pdb[0] - 0.5 <= -31 <= pdb[1] + 0.5
ok38 = pdb[0] - 0.5 <= -38 <= pdb[1] + 0.5
ok71 = wdb[0] - 0.5 <= 71 <= wdb[1] + 0.5
print(f"{cname:32s} PD=-31? {ok31!s:5s} PD=-38? {ok38!s:5s} WD=+71? {ok71!s:5s}")
print("\n=== Gap between the two printed PD values ===")
print(f"38 - 31 = {38-31} percentage points; relative to abstract value: {(38-31)/31*100:.0f}%")
print(f"post/baseline ratio: abstract {1-0.31:.2f}, Results {1-0.38:.2f}")
print(f"Diet contrast (post/baseline WD / PD): abstract {1.71/0.69:.2f}, Results {1.71/0.62:.2f}")
print(f"PD decrease expressed relative to post-diet mean (change of means): "
f"{(pd_before.mean()-pd_after.mean())/pd_after.mean()*100:.0f}%")
print(f"Rounding: any value in [-31.5, -30.5) prints as 31%, any in [-38.5, -37.5) as 38%; the intervals do not overlap.")
print("\n=== Table 1: convention check (change of rounded means vs printed % change) ===")
T1 = [("PD TNF-a", 10, 8.4, -17), ("PD MCP-1", 195, 178, -9), ("PD IL-6", 1.6, 1.4, -11), ("PD IL-8", 19, 14, -28),
("WD TNF-a", 9, 9, -1), ("WD MCP-1", 198, 180, -9), ("WD IL-6", 1.8, 1.8, -2), ("WD IL-8", 17, 15, -13)]
for n, b, a, rep in T1:
print(f"{n:9s} before {b:>5} after {a:>5}: change of means {(a/b-1)*100:+6.1f}% printed {rep:+d}%")
# ================================================================ figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
BG, INK, INK2, ORG, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "svg.fonttype": "none",
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2})
def fmt(v):
return f"{v:+.1f}".replace("-", "\u2212")
# ---------- fig1: abstract vs Results text panels with connector
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 64); ax.axis("off")
def panel(x, title, lines, pdtxt):
ax.add_patch(FancyBboxPatch((x, 15), 42, 42, boxstyle="round,pad=0.4,rounding_size=1.5",
fc="white", ec="#c9c8c3", lw=1.2))
ax.text(x + 2, 54.5, title, fontsize=11, fontweight="bold", va="top", color=INK)
y = 48.5
for ln in lines:
ax.text(x + 2, y, ln, fontsize=10, va="top", color=INK2)
y -= 4.3
ax.text(x + 21, 23.5, pdtxt, fontsize=17, fontweight="bold", ha="center", va="center", color=ORG)
ax.text(x + 21, 18.3, "Western diet: +71%", fontsize=10.5, ha="center", va="center", color=ORG)
panel(3, "Abstract", ["\u201cPlacing 8 healthy subjects on a",
"Western-style diet for 1 month",
"induced a 71% increase \u2026",
"whereas a prudent-style diet",
"reduced levels by 31%.\u201d"],
"Prudent diet: \u221231%")
panel(55, "Results (paragraph citing Fig. 1)", ["\u201c\u2026 increases (by 71%) in plasma",
"endotoxin activity levels, in contrast",
"to a major decrease (by 38%) after",
"the prudent-style diet (Figure 1)",
"(P < .001).\u201d"],
"Prudent diet: \u221238%")
ax.annotate("", xy=(54.2, 23.5), xytext=(45.8, 23.5),
arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
ax.text(50, 26.5, "conflict", ha="center", va="bottom", fontsize=10, fontweight="bold", color=INK)
ax.text(50, 6.8, "Same 8 subjects, same assay, same before/after comparison: the prudent-diet decrease is printed\n"
f"as 31% in the abstract and 38% in the Results (gap {38-31} percentage points); the Western-diet value agrees.\n"
f"Figure 1, read by eye, change of group means: prudent {fmt(RES['Change of group means'][0])}%, "
f"Western {fmt(RES['Change of group means'][2])}% (shown in fig. 2).",
ha="center", va="center", fontsize=9.5, color=INK, linespacing=1.5)
ax.text(50, 61, "Values in orange are as printed in the paper", ha="center", va="center", fontsize=10, color=ORG)
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), **kw)
plt.close(fig)
# ---------- fig2: conventions vs printed values, one axis
fig, ax = plt.subplots(figsize=(7.2, 4.0))
names = list(RES)
ys = np.arange(len(names))[::-1].astype(float)
top = len(names) - 0.5
ax.axvline(0, color=INK2, lw=0.8)
for v, lab, ha in [(-31, "abstract \u221231%", "left"), (-38, "Results \u221238%", "right"),
(71, "abstract and\nResults +71%", "right")]:
ax.plot([v, v], [-0.5, top], color=ORG, lw=2, ls=(0, (4, 2)))
ax.text(v + (0.8 if ha == "left" else -0.8), top + 0.05, lab, color=ORG, fontsize=9.5, ha=ha, va="bottom")
ax.text(-35, top + 0.75, "Prudent diet", ha="center", va="bottom", fontsize=10.5, fontweight="bold")
ax.text(75, top + 0.75, "Western diet", ha="center", va="bottom", fontsize=10.5, fontweight="bold")
for y, n in zip(ys, names):
pdv, pdb, wdv, wdb = RES[n]
for v, b in [(pdv, pdb), (wdv, wdb)]:
ax.plot(b, [y, y], color=BLU, lw=2, solid_capstyle="butt")
ax.plot(v, y, "o", ms=8, color=BLU)
ax.text(-58, y, fmt(pdv), color=BLU, fontsize=10, ha="left", va="center")
ax.text(110, y, fmt(wdv), color=BLU, fontsize=10, ha="right", va="center")
ax.set_yticks(ys); ax.set_yticklabels(names, color=INK)
ax.set_xlim(-60, 112); ax.set_ylim(-0.6, top + 1.35)
ax.set_xlabel("% change in endotoxin activity over 4 weeks on each diet")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
ax.plot([], [], "o-", color=BLU, ms=8, lw=2, label="recomputed from Figure 1 values read by eye (bar: 95% reading-error band)")
ax.plot([], [], color=ORG, lw=2, ls=(0, (4, 2)), label="value printed in the paper")
ax.legend(loc="upper center", bbox_to_anchor=(0.4, -0.2), ncol=1, frameon=False, fontsize=9.5)
fig.tight_layout()
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), **kw)
plt.close(fig)
print("\nfigures written to", HERE)
Abstract reverses the direction of the paper's own DDIT4 and gastric cancer survival finding
Gastroenterology (2020) · doi:10.1053/j.gastro.2020.09.016 · PMC7857017
Summary
The abstract states that "Lower DDIT4 expression correlated with longer survival" in gastric cancer. The rest of the paper reports the opposite for the only gastric cancer survival cohort it analyses, a tissue microarray (TMA) of 509 patients. The Introduction, Results, Figure 4 title and Discussion all say that high DDIT4 goes with longer survival, and the Kaplan-Meier curves in Figure 4B agree: the DDIT4-positive group (n = 200) lies above the DDIT4-negative group (n = 309), log-rank P = .021. The reversed sentence is also in the copyedited abstract indexed in PubMed, and no correction has been issued, so readers who see only the abstract get the direction of the paper's main human result backwards.
What the paper reports
Abstract, Results (PMC author manuscript):
Lower DDIT4 expression correlated with longer survival of gastric cancer patients.
Abstract, Results, as deposited in PubMed (PMID 32956680) and Europe PMC, in the journal's copyedited wording:
Lower DDIT4 expression correlated with longer survival of patients with gastric cancer.
Abstract, Conclusions:
During injury-induced metaplastic proliferation, failure of licensing mTORC1 reactivation correlates with increased proliferation of cells harboring DNA damage, as well as increased tumor formation and growth in mice and humans.
Introduction, final (5th) paragraph:
Patients with low expression have poorer survival.
Results, section "Loss of DDIT4 causes cell-autonomous increased growth of human gastric…", paragraph 1:
We examined another annotated tissue microarray on which we had compiled 509 patients with gastric cancer. We saw statistically significant correlation of high DDIT4 expression with increased patient survival and more differentiated tumors, especially in patients who had received chemotherapy, consistent with DDIT4 slowing proliferation in response to DNA damage (Figure 4A–C; Table S3).
Figure 4 title:
In humans, loss of DDIT4 correlates with worse gastric cancer survival and causes cell-autonomous increased mTORC1 and proliferation despite chemotherapy
Figure 4 legend, panel B:
Overall survival determined via Kaplan-Meier curves separated by DDIT4 expression, with an event defined as a cancer-related death.
Discussion, paragraph 3:
Our results showing DDIT4 expression correlating with less proliferation in precancerous lesions and better survival of patients with gastric cancer confirm and extend on those of a previous report43.
Discussion, paragraph 4:
Paradoxically, however, studies across multiple organs have shown disparate results for DDIT4 in tumorigenesis with some studies showing higher DDIT4 decreases cancer survival44, 47–50.
Figure 4B and 4C, legend boxes printed on the image (read from the PMC image file nihms-1661019-f0004.jpg, not from the text):
4B, Overall Patients: DDIT4 negative (n=309); DDIT4 positive (n=200); P = .021
4C, Patients receiving chemotherapy: DDIT4 negative (n=263); DDIT4 positive (n=170); P = .016
What the paper's own numbers show
Method. The paper reports no hazard ratio or survival estimate in its text, so there is no estimate to recompute. Instead I recorded the direction of the DDIT4 and survival association in every place the paper describes it, and checked the group sizes printed on Figure 4B and 4C against the cohort size in the Results. All quotes from the paper's text were matched verbatim against the PMC-archived text, and recompute.py.txt repeats this check for every one of them; the PubMed wording was checked against the PubMed efetch record, and the Figure 4B and 4C counts were read from the figure image. In Figure 4B and 4C the DDIT4-positive curve lies above the DDIT4-negative curve over most of follow-up. In this paper, "positive" is the high-expression group (Figure 4A shows cores scored "negative" or "positive").
| Quantity | Location | Reported | Recomputed / what the paper's own material shows |
|---|---|---|---|
| Direction of DDIT4 and survival | Abstract, Results (PMC and PubMed) | Low DDIT4 → longer survival | Opposite to every statement below and to Figure 4B |
| Direction of DDIT4 and survival | Introduction, final (5th) paragraph | Low DDIT4 → poorer survival | High DDIT4 → longer survival |
| Direction of DDIT4 and survival | Results, 509-patient TMA | High DDIT4 → increased survival | High DDIT4 → longer survival |
| Direction of DDIT4 and survival | Figure 4 title | Loss of DDIT4 → worse survival | High DDIT4 → longer survival |
| Direction of DDIT4 and survival | Discussion, paragraph 3 | DDIT4 expression → better survival | High DDIT4 → longer survival |
| Direction of DDIT4 and survival | Figure 4B and 4C curves | Positive curve above negative curve | High DDIT4 → longer survival |
| Body statements that agree | Intro, Results, Fig 4 title, Discussion, Fig 4B | n/a | 5 of 5 agree with each other; the abstract contradicts all 5 |
| Cohort size | Results text vs Figure 4B | 509 patients; 309 + 200 | 309 + 200 = 509 (matches) |
| Chemotherapy subgroup | Figure 4C | 263 + 170 | 433 patients, 85.1% of the 509 |
| DDIT4-positive share | Figure 4B / 4C | 200 of 509; 170 of 433 | 39.3% in both |
| Log-rank P | Figure 4B / 4C | .021 / .016 | Not recomputable from published material (no individual data) |
Figure
Figure 1. On the left (orange outline) is the abstract's Results sentence, quoted from the PMC author manuscript; the same sentence, in copyedited wording ("…of patients with gastric cancer"), is in the publisher-deposited abstract in PubMed. On the right (blue outline) are short excerpts from the Introduction, the Results, the Figure 4 title and the Discussion, plus the group sizes and log-rank P value printed on Figure 4B. They describe the same 509-patient gastric cancer tissue microarray (the Introduction sentence does not name the cohort, but the 509-patient TMA is the paper's only survival analysis) and the same endpoint (overall survival, with cancer-related death as the event). The abstract states that low DDIT4 goes with longer survival. Every body statement, and the Kaplan-Meier data, state or show that high DDIT4 goes with longer survival. The counts come from recompute.py.txt, which transcribes them from the Figure 4 image. The paper's own figure is not reproduced.
Why it matters
- The human survival association is the paper's only patient-outcome evidence linking DDIT4 loss to gastric cancer. The abstract's Conclusions draw on it in part ("increased tumor formation and growth in mice and humans", which also covers the human-cell xenograft and in-vitro data), and so does the paper's mechanism (DDIT4 restrains proliferation of DNA-damaged cells, so losing it should worsen outcomes). The abstract's Results sentence contradicts both the abstract's own Conclusions and the evidence behind them.
- The abstract is what most readers, indexing services, systematic reviewers and automated literature tools see. A reader of the abstract alone would conclude that DDIT4 is a marker of poor prognosis in gastric cancer, which is the reverse of what this study found.
- The confusion is easy to compound. The Discussion itself notes that other studies found "higher DDIT4 decreases cancer survival". A reader comparing studies by their abstracts could wrongly count this paper among them.
- This write-up does not question the survival analysis itself (Kaplan-Meier curves with a log-rank test) or the numbers printed on Figure 4B and 4C. Other estimates in the paper, and Supplementary Table 3, were not checked. The error is a reversed statement in the abstract, the most widely read part of the paper.
Explanations considered
- A different population or subgroup. This does not explain it. The paper has one gastric cancer survival cohort, the 509-patient TMA in Figure 4B. The only subgroup shown, patients who received chemotherapy (Figure 4C, n = 433), has the same direction (positive curve above negative, P = .016), and the Results call the association stronger there ("especially in patients who had received chemotherapy"). The second TMA, of 218 patients with normal and precancerous lesions, has no survival analysis.
- Results from other studies. This does not explain it. Discussion paragraph 4 cites external studies (references 44 and 47–50) in which higher DDIT4 lowers survival, but it presents them as disagreeing with this study ("Paradoxically, however…"), not as part of its own findings.
- Inverted coding or scale. No evidence of this. DDIT4 is scored "negative" or "positive" by immunohistochemistry (Figure 4A), the figure legends label the groups explicitly, and the text uses "loss of DDIT4" and "low expression" in agreement with the "negative" group.
- A different endpoint. This does not explain it. The only endpoint is overall survival, with cancer-related death as the event (Figure 4B legend), and the same endpoint is used throughout.
- A conversion or manuscript-copy artifact. Ruled out. The conflict is in words ("Lower … longer" against "high … increased"), not in symbols, signs or table cells. The same sentence, in the journal's copyedited wording, appears in the abstract deposited in PubMed and Europe PMC.
- A correction already issued. None was found (see Status).
- What remains. Every part of the paper except one abstract sentence agrees on the direction, and so do the Kaplan-Meier curves. The simplest reading is that the abstract sentence was worded backwards.
Status
- Version of record. The publisher's pages could not be reached by automated access: doi.org redirects to linkinghub.elsevier.com, which returned only a redirect stub, and sciencedirect.com and gastrojournal.org returned HTTP 403. The copyedited abstract deposited in PubMed (PMID 32956680, efetch XML) and in the Europe PMC core record reads "Lower DDIT4 expression correlated with longer survival of patients with gastric cancer." Its wording differs slightly from the manuscript ("patients with gastric cancer" rather than "gastric cancer patients"), which indicates it is the journal's edited version, so the reversed sentence very likely also appears in the version of record. PMC7857017 is the NIH author manuscript (NIHMS1661019). The publisher's full-text body could not be checked; the body quotes above come from the PMC author manuscript.
- Figures and supplements. I opened Figure 4 itself (nihms-1661019-f0004.jpg, from the PMC CDN) and read the group sizes and P values in panels B and C. The PMC page lists Supplementary Tables 1–3 (.docx), Supplementary Figures 1–6, Supplementary Figure Legends and Supplementary Materials and Methods. Supplementary Table 3, the clinicopathological table cited for the survival result, could not be downloaded: PMC served a download-challenge page, and Europe PMC reported the supplementary files as not open access. Figure 4B settles the direction without it.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI. PMC and Europe PMC list no notices, and the PubMed record has no CommentsCorrections entries. Crossref Labs (which includes Retraction Watch data) shows no updates. PubPeer could not be checked directly: its API and search page were blocked by a bot-protection challenge (HTTP 403). A web search for the title with "pubpeer" returned no PubPeer page.
- Sources. PMC article PMC7857017 (author manuscript NIHMS1661019) and its Figure 4 image; PubMed 32956680 (efetch XML); Europe PMC core record; Crossref REST API; Crossref Labs; PubPeer (attempted); web search.
Suggested correction
- Change the abstract's Results sentence so that it matches the body of the paper and Figure 4B, for example: "Higher DDIT4 expression correlated with longer survival of patients with gastric cancer." The equivalent "Lower DDIT4 expression correlated with shorter survival…" would also work.
- Ideally, add the effect size to the abstract and Results (for example a hazard ratio with a 95% CI from a Cox model, alongside the log-rank P = .021 for the 509 patients), so the size and direction of the association can be read without the figure.
- Ask PubMed and Europe PMC to update their abstract records when the correction is published, since those are the copies most readers see.
Recomputation script (python recompute.py.txt)
"""Check the direction of the DDIT4-survival association stated in PMC7857017
(Gastroenterology 2020, doi:10.1053/j.gastro.2020.09.016).
Run: python recompute.py.txt
Needs matplotlib. Prints every number used in writeup.md and writes fig1.svg and
fig1.png next to this file.
There is no estimate to recompute: the paper reports no hazard ratio, only Kaplan-Meier
curves with log-rank P values (Figure 4B-C). The check is therefore (1) the direction
stated in each place the paper describes the 509-patient gastric cancer survival result,
and (2) the group sizes and P values printed on Figure 4B-C, read from the PMC image
nihms-1661019-f0004.jpg (they are not in the text, so they are typed in below).
If the repository's src/show.py is available (run from /home/ubuntu/papers), every quote
from the paper's text is also confirmed verbatim against the PMC-archived text.
"""
import os
import subprocess
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Patch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs (from the paper)
# direction: +1 = high DDIT4 -> longer survival ; -1 = high (i.e. not low) DDIT4 -> shorter survival
STATEMENTS = [
("Abstract, Results",
"Lower DDIT4 expression correlated with longer survival of gastric cancer patients.", -1),
("Introduction, final (5th) paragraph",
"Patients with low expression have poorer survival.", +1),
("Results, human gastric cancer TMA",
"We saw statistically significant correlation of high DDIT4 expression with increased "
"patient survival and more differentiated tumors, especially in patients who had received "
"chemotherapy", +1),
("Figure 4 title",
"In humans, loss of DDIT4 correlates with worse gastric cancer survival", +1),
("Discussion, paragraph 3",
"Our results showing DDIT4 expression correlating with less proliferation in precancerous "
"lesions and better survival of patients with gastric cancer", +1),
]
# Other quotes from the PMC text used in writeup.md (checked verbatim, no direction coded)
OTHER_QUOTES = [
("Abstract, Conclusions",
"During injury-induced metaplastic proliferation, failure of licensing mTORC1 reactivation "
"correlates with increased proliferation of cells harboring DNA damage, as well as increased "
"tumor formation and growth in mice and humans."),
("Results, human gastric cancer TMA (full passage)",
"We examined another annotated tissue microarray on which we had compiled 509 patients with "
"gastric cancer. We saw statistically significant correlation of high DDIT4 expression with "
"increased patient survival and more differentiated tumors, especially in patients who had "
"received chemotherapy, consistent with DDIT4 slowing proliferation in response to DNA damage "
"(Figure 4A–C; Table S3)."),
("Figure 4 legend, panel B",
"Overall survival determined via Kaplan-Meier curves separated by DDIT4 expression, with an "
"event defined as a cancer-related death."),
("Discussion, paragraph 3 (full sentence)",
"Our results showing DDIT4 expression correlating with less proliferation in precancerous "
"lesions and better survival of patients with gastric cancer confirm and extend on those of "
"a previous report43."),
("Discussion, paragraph 4",
"Paradoxically, however, studies across multiple organs have shown disparate results for "
"DDIT4 in tumorigenesis with some studies showing higher DDIT4 decreases cancer survival44, "
"47–50."),
]
# The copyedited abstract deposited in PubMed (PMID 32956680) and Europe PMC:
PUBMED_ABSTRACT = "Lower DDIT4 expression correlated with longer survival of patients with gastric cancer."
TMA_N_TEXT = 509 # Results: "compiled 509 patients with gastric cancer"
# Figure 4B (overall) and 4C (patients receiving chemotherapy), legend boxes on the image
FIG4 = {
"4B overall": {"negative": 309, "positive": 200, "P": 0.021},
"4C chemotherapy": {"negative": 263, "positive": 170, "P": 0.016},
}
# On both panels the DDIT4-positive (green) curve lies above the DDIT4-negative (blue) curve
# over most of follow-up, i.e. the data show high DDIT4 -> longer survival.
FIG4_DIRECTION = +1
def check_quotes():
show = os.path.join(os.getcwd(), "src", "show.py")
if not os.path.exists(show):
print("src/show.py not found in the working directory; quote check skipped")
return
py = "/home/ubuntu/papers-data/venv/bin/python"
py = py if os.path.exists(py) else "python3"
txt = subprocess.run([py, show, "PMC7857017", "--full"], capture_output=True,
text=True).stdout
quotes = [(loc, s) for loc, s, _ in STATEMENTS] + OTHER_QUOTES
for loc, s in quotes:
print(f" verbatim in PMC text: {s in txt!s:5} ({loc})")
print(f" PMC-text quotes found: {sum(s in txt for _, s in quotes)} of {len(quotes)}"
" (the PubMed wording and the Figure 4B-C counts are not in the PMC text)")
def main():
print("== Direction of the DDIT4-survival association, by location")
word = {+1: "high DDIT4 -> longer survival", -1: "low DDIT4 -> longer survival"}
for loc, _, d in STATEMENTS:
print(f" {loc:36s} {word[d]}")
print(f" {'Figure 4B-C Kaplan-Meier curves':36s} {word[FIG4_DIRECTION]}")
body = [d for _, _, d in STATEMENTS[1:]] + [FIG4_DIRECTION]
abstract = STATEMENTS[0][2]
print(f" body statements agreeing with each other: {sum(d == body[0] for d in body)} of {len(body)}")
print(f" abstract opposite to every body statement: {all(d == -abstract for d in body)}")
print(f" PubMed/Europe PMC abstract wording: {PUBMED_ABSTRACT!r}")
check_quotes()
print("\n== Figure 4B-C group sizes")
b, c = FIG4["4B overall"], FIG4["4C chemotherapy"]
nb = b["negative"] + b["positive"]
nc = c["negative"] + c["positive"]
print(f" 4B: {b['negative']} negative + {b['positive']} positive = {nb}"
f" (Results text: {TMA_N_TEXT}; match = {nb == TMA_N_TEXT}); log-rank P = {b['P']}")
print(f" 4C: {c['negative']} negative + {c['positive']} positive = {nc}"
f" = {100 * nc / nb:.1f}% of the 4B cohort; log-rank P = {c['P']}")
print(f" DDIT4-positive share: 4B {100 * b['positive'] / nb:.1f}%, 4C {100 * c['positive'] / nc:.1f}%")
make_figure(b)
# ------------------------------------------------------------------ figure
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
def box(ax, x, y, w, h, color):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
linewidth=2, edgecolor=color, facecolor="white"))
def make_figure(b):
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "svg.fonttype": "none"})
fig = plt.figure(figsize=(7.2, 5.9), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off"); ax.set_facecolor(BG)
# left panel: abstract
lx, lw, ly, lh = 0.03, 0.30, 0.36, 0.48
box(ax, lx, ly, lw, lh, ORANGE)
ax.text(lx + 0.02, ly + lh - 0.035, "Abstract (Results)", fontsize=11, weight="bold",
color=INK, va="top")
ax.text(lx + 0.02, ly + lh - 0.085, "PMC author manuscript", fontsize=10,
color=INK2, va="top")
q = textwrap.fill("“Lower DDIT4 expression correlated with longer survival of gastric "
"cancer patients.”", 24)
ax.text(lx + 0.02, ly + lh - 0.135, q, fontsize=10.5, color=INK, va="top", style="italic")
ax.text(lx + 0.02, ly + 0.11, "Same sentence, in\ncopyedited wording, is\nin the PubMed abstract", fontsize=10,
color=INK2, va="bottom")
ax.text(lx + 0.02, ly + 0.03, "Stated: LOW DDIT4 →\nlonger survival", fontsize=10.5,
weight="bold", color=INK, va="bottom")
# right panel: body + figure data
rx, rw, ry, rh = 0.44, 0.53, 0.17, 0.79
box(ax, rx, ry, rw, rh, BLUE)
ax.text(rx + 0.02, ry + rh - 0.03, "Body of the paper (509-patient TMA)", fontsize=11,
weight="bold", color=INK, va="top")
items = [
("Introduction", "“Patients with low expression have poorer survival.”"),
("Results", "“high DDIT4 expression with increased patient survival”"),
("Figure 4 title", "“loss of DDIT4 correlates with worse gastric cancer survival”"),
("Discussion", "“DDIT4 expression correlating with … better survival”"),
("Figure 4B data", f"DDIT4-positive curve (n={b['positive']}) above DDIT4-negative "
f"(n={b['negative']}); log-rank P = {str(b['P'])[1:]}"),
]
y = ry + rh - 0.095
for head, body in items:
ax.text(rx + 0.02, y, head, fontsize=10, weight="bold", color=INK2, va="top")
lines = textwrap.fill(body, 47)
ax.text(rx + 0.02, y - 0.036, lines, fontsize=10, color=INK, va="top")
y -= 0.036 + 0.031 * (lines.count("\n") + 1) + 0.025
ax.text(rx + 0.02, ry + 0.025, "Stated and shown: HIGH DDIT4 →\nlonger survival",
fontsize=10.5, weight="bold", color=INK, va="bottom")
# connector
arr = FancyArrowPatch((lx + lw + 0.005, ly + lh / 2), (rx - 0.005, ly + lh / 2),
arrowstyle="<->", mutation_scale=14, linewidth=2, color=INK2)
ax.add_patch(arr)
ax.text((lx + lw + rx) / 2, ly + lh / 2 + 0.03, "opposite\ndirection", ha="center",
va="bottom", fontsize=10, color=INK, weight="bold")
ax.text((lx + lw + rx) / 2, ly + lh / 2 - 0.03, "same\ncohort,\nsame\nendpoint", ha="center",
va="top", fontsize=10, color=INK2)
ax.legend(handles=[Patch(facecolor="white", edgecolor=ORANGE, linewidth=2,
label="As reported in the abstract"),
Patch(facecolor="white", edgecolor=BLUE, linewidth=2,
label="Paper's own body text and Figure 4B data")],
loc="lower left", bbox_to_anchor=(0.03, 0.02), frameon=False, fontsize=10,
ncol=1, handlelength=1.6)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
plt.close(fig)
print("\nwrote fig1.svg, fig1.png")
if __name__ == "__main__":
main()
Abstract hazard ratio for SLiM-CRAB status lies outside its own confidence interval and contradicts the Results
J Clin Oncol (2024) · doi:10.1200/JCO.23.02771 · PMC11404760
Summary
This single-arm phase II trial in high-risk smoldering myeloma (90 patients) reports in its abstract that having any SLiM-CRAB criterion predicted progression to multiple myeloma (MM), with "hazard ratio, 0.12; 95% CI, 0.14 to 1.13; P = .03". Those three numbers cannot belong together. The point estimate lies below the lower confidence limit, and the interval contains 1, so a Wald test based on it gives P ≈ 0.084, not .03. The Results section reports the same comparison as HR 0.10 (95% CI 0.01 to 0.96; P = .046). Those values are internally consistent (the interval's centre is 0.098, Wald P 0.046) and roughly match the paper's raw counts. The correct result keeps its direction and nominal significance. But the abstract figures, the version most readers see, cannot be read as printed, and the body's figures show that the estimate rests on 5 events and has an interval spanning about two orders of magnitude.
What the paper reports
Abstract, Results:
The presence of any SLiM CRAB criteria predicted progression to MM (four of the five patients; hazard ratio, 0.12; 95% CI, 0.14 to 1.13; P = .03).
Results, "Time-To-Event End Points", paragraph 4 of that section (same comparison, same outcome, same figure):
As mentioned above, 28 patients presented with any one SLiM biomarker. Although their response rate was not different from that observed in the standard high-risk–SMM population (Appendix Table A2), the presence of any SLiM biomarker predicted progression to MM (HR, 0.10 [95% CI, 0.01 to 0.96]; P = .046; Fig 3B).
The subgroup sizes. Abstract, Results:
90 patients were included, and 31% met at least one SixtyLightchain MRI (SLiM)-hypercalcemia, renal impairment, anemia, bone disease (CRAB) criterion.
Table 1:
SLiM-CRAB criteria, No. (%) | 28 (31)
Appendix Table A2 gives the columns "High Risk (n = 62)" and "Ultra High Risk (n = 28)". Its footnote b reads:
Four patients biochemically progressed and one of them progressed to myeloma.
The model. Methods, "Statistical Analysis":
The Kaplan-Meier method was used to characterize event time distributions, and the corresponding treatment hazard ratio was estimated using a stratified Cox regression model to compare outcomes based on baseline characteristics and MRD status.
The conclusion that rests on this result. Discussion, paragraph 6:
In our study, the only feature predicting progression to MM was the presence of any SLiM biomarker, now incorporated into the definition of MM, which would reinforce this hypothesis.
What the paper's own numbers show
Method. A Cox-model 95% CI is symmetric on the log scale, so the reported HR should lie close to the interval's geometric centre, sqrt(lower × upper). The standard error of log HR is ln(upper/lower)/(2 × 1.96), and the two-sided Wald P is 2Φ(−|ln centre|/SE). I applied this check to every hazard ratio printed in the paper (five in all), which shows whether the paper follows one consistent convention. I then checked whether rounding or a single misplaced digit could reconcile the abstract. As a rough check on direction and size that is independent of the Cox model, I computed a crude risk ratio from the counts (1 of 62 vs 4 of 28 progressed to MM). It ignores follow-up time and is not the paper's estimate. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Is the HR inside its 95% CI? | Abstract, Results | HR 0.12; CI 0.14 to 1.13 | No: 0.12 < 0.14 |
| Centre of the CI, sqrt(lo × hi) | Abstract, Results | HR 0.12 | 0.398 |
| Does the CI exclude 1? | Abstract, Results | P = .03 | No: 1.13 > 1, so any test matching this interval gives P ≥ .050 |
| Wald P from the CI | Abstract, Results | P = .03 | 0.084 (SE of log HR 0.533) |
| Can rounding explain HR < lower limit? | Abstract, Results | 0.12 vs 0.14 | No: values rounding to 0.12 are < 0.125; values rounding to 0.14 are ≥ 0.135 |
| Centre of the CI | Results, Time-To-Event End Points; Fig 3B | HR 0.10; CI 0.01 to 0.96 | 0.098 |
| Wald P from the CI | Results, Time-To-Event End Points; Fig 3B | P = .046 | 0.046 from the CI centre; 0.048 from HR 0.10 (SE 1.164) |
| The paper's other three HRs: CI centre and Wald P | Results, Time-To-Event End Points (Fig 2B, Appendix Fig A1A, A1B) | 0.29 (0.13 to 0.66), P = .003; 0.42 (0.22 to 0.81), P = .01; 0.40 (0.18 to 0.85), P = .018 | 0.293, P 0.003; 0.422, P 0.009; 0.391, P 0.018: all consistent |
| Abstract with the lower limit read as 0.014 | hypothetical single typo | n/a | Centre 0.126 (fits 0.12), but the CI still contains 1; Wald P 0.058 to 0.064 |
| CI that HR 0.12 with P = .03 would imply | hypothetical | n/a | 0.018 to 0.814 |
| Progression to MM by SLiM status | Abstract ("four of the five"); Table 1; Table A2 | 4 of 5 events in the SLiM group; n = 28 vs 62 | 4/28 = 14.3% vs 1/62 = 1.6% |
| Crude risk ratio, no SLiM vs SLiM | from the counts above | n/a | 0.113 (95% CI 0.013 to 0.965); Wald P 0.046; Fisher exact P 0.031 |
| Share of patients with any SLiM-CRAB criterion | Abstract; Table 1 | 31%; 28 (31) | 28/90 = 31.1% (consistent) |
Of the five hazard ratios printed in the paper, the abstract's is the only one whose point estimate, interval and P value do not fit together. The other four, including the Results-section statement of this same comparison, agree with the Wald method to within rounding.
Further observations, not part of the main finding. The following are reported here so that a correction can address them together. I have not investigated them further.
| Quantity | Location | Reported | Check |
|---|---|---|---|
| 70-month TTP | Abstract; Results, Time-To-Event End Points | 94% (95% CI, 84 to 89) | Estimate lies outside its own CI |
| 70-month OS | Abstract; Results, Time-To-Event End Points | 92% (95% CI, 82 to 89) | Estimate lies outside its own CI |
| 70-month TTBP | Results, Time-To-Event End Points | 59% (95% CI, 66 to 76) | Estimate lies outside its own CI |
| Patients with any SLiM criterion | Results, Time-To-Event End Points, paragraph 2 | "12 [38.7%] of 31 patients" vs "18 [30.5%] of 59 patients" | 31 here vs 28 in Table 1, Table A2 and the Results paragraph quoted above; 12 + 18 = 30, against 36 biochemical progressions reported in the same section |
| Sustained uMRD 4 years after ASCT | Abstract / Discussion and Table A1 / CONTEXT box | "29 patients (31%)" / "28 patients (31%)" and "28 (31)" / "40 (44%)" | 29/90 = 32.2%; 28/90 = 31.1%; 40/90 = 44.4%. The three counts differ |
Figure
Figure 1. One comparison, any SLiM-CRAB criterion vs none and progression to MM, as printed in two places and as recomputed. Orange circles and lines show values printed in the paper; blue diamonds show values recomputed from the paper's own numbers. The abstract's point estimate (0.12) lies to the left of its own interval (0.14 to 1.13), and that interval crosses the reference line at 1. The interval's own centre is 0.40, with a Wald P of 0.084. The Results-section estimate (0.10, 0.01 to 0.96) sits at the centre of its interval and implies P = 0.046, as printed. A crude risk ratio from the counts (1/62 vs 4/28), which ignores follow-up time, lands near the Results value. The horizontal axis is logarithmic, and the vertical line marks a ratio of 1.
Figure 2. Every hazard ratio in the paper, comparing the printed P value (orange circle) with the Wald P implied by its printed 95% CI (blue diamond). Within each row the circle and diamond are offset slightly up and down so that both stay visible. For four of the five, the two agree to within rounding. Only the abstract's statement crosses the P = .05 line, at .03 printed against 0.084 implied by its interval. This suggests that the paper's intervals and P values otherwise follow one convention, and that the abstract figures are an exception rather than a different method.
Why it matters
The error does not touch the trial's primary end point (uMRD 3 months after ASCT, 56 of 90) or its sustained-uMRD secondary end point. Nor, on the evidence of the body text and Fig 3B, does it reverse the reported finding: the internally consistent value (HR 0.10, 95% CI 0.01 to 0.96, P = .046) has the same direction and is nominally significant.
It matters for three reasons:
- The abstract is uninterpretable as printed. A reader who has only the abstract sees an interval (0.14 to 1.13) that includes no effect, next to a claim that SLiM status "predicted" progression and a P value of .03. Read at face value, the interval says the association is not statistically significant, which contradicts the sentence it supports.
- The body values show how fragile the result is. The finding rests on 5 progression events, 4 of them in the SLiM group. Its interval spans roughly two orders of magnitude, and its P value (.046) sits just under the threshold. The Discussion uses this result, "the only feature predicting progression to MM", to support the hypothesis that early treatment of true high-risk smoldering myeloma raises the chance of operational cure. A reader should weigh it as a hypothesis-generating signal.
- The direction convention is not stated. "Presence ... predicted progression" corresponds to an HR above 1 for SLiM present vs absent. Both printed HRs are below 1, so the reference group must be the SLiM group (absent vs present). The crude proportions (1.6% vs 14.3%) support that reading, but the paper does not state it.
Explanations considered
- Rounding. This cannot explain the result. A value that rounds to 0.12 is below 0.125, and a lower limit that rounds to 0.14 is at least 0.135, so no rounding puts 0.12 inside 0.14 to 1.13. Rounding also cannot turn an interval that crosses 1 into one compatible with P < .05.
- A different test for the P value (for example log-rank, score or exact test rather than Wald). With only 5 events these tests can differ noticeably. For instance, a Fisher exact test on the crude counts gives P = 0.031. This could account for P = .03 differing from a Wald P on its own. It cannot account for a point estimate outside its own interval, or for a different HR and CI from the Results section, which also reports P = .046 for the same comparison.
- Different model, adjustment or population. The Methods name a single model for comparisons by baseline characteristics (a stratified Cox regression). The abstract and the Results both describe any SLiM criterion vs none, progression to MM, in the ITT population, and the Results cite Fig 3B. Neither text marks one value as adjusted, landmarked or from a different time point. According to the version-of-record check, the Fig 3B panel itself prints HR 0.10 (95% CI 0.01 to 0.96), P = .046, which matches the Results.
- Direction or reference-group convention. An inverted HR (1/0.12 ≈ 8.3) does not produce the interval 0.14 to 1.13 either. Both statements share the same convention (HR < 1), so the convention does not explain why they differ.
- One-sided test. No one-sided convention puts a point estimate outside its own two-sided interval. And an interval that includes 1 at the 95% level is not compatible with a two-sided P of .03.
- A single typo in the interval. If the lower limit should read 0.014, the point estimate fits (centre 0.126). But the interval would still include 1, with a Wald P of about 0.06, so the abstract would still conflict with its own P = .03. Reconciling HR 0.12 with P = .03 would need an interval of about 0.018 to 0.81, which differs from the printed one in both limits. No single-character change makes the abstract consistent.
- Text-conversion artefact. The publisher's typeset PDF, the PMC article, the Europe PMC full-text XML and the PubMed abstract all carry the same numbers, so the discrepancy is in the version of record.
Status
As of 2026-10-23:
- Version of record. ascopubs.org returned HTTP 403. The publisher's typeset PDF (JCO 2024;42:3247-3256, 14 pages including the Appendix) was read from a copy held in an institutional repository. Its abstract and Results carry the same wording and numbers as quoted above. The "P 5" in its extracted text is the journal's font glyph for "=". The PMC article (pmc.ncbi.nlm.nih.gov/articles/PMC11404760), the Europe PMC full-text XML and the PubMed record (PMID 39038268) agree.
- Supplements. There is no separate data supplement. The Appendix (Tables A1 and A2, Fig A1) is part of the article. It gives SLiM subgroup sizes and response rates but does not restate the HR for SLiM status and progression to MM. Individual patient data needed to refit the Cox model are not published.
- Corrections and notices. The project's
known.pyreturned no Crossref update, PMC notice or Europe PMC notice. The Crossref record has noupdate-toorupdated-byrelation, the PubMed record has no CommentsCorrections entries, and the PMC page shows no correction notice. A web search for an erratum or correction citing the DOI found none. - PubPeer. It could not be checked: pubpeer.com returned HTTP 403 (Cloudflare) to both the search page and the API.
- Sources: doi:10.1200/JCO.23.02771; PMC11404760; PubMed 39038268; Crossref; Europe PMC.
Suggested correction
- In the abstract, replace "hazard ratio, 0.12; 95% CI, 0.14 to 1.13; P = .03" with the values from the fitted model. If the model is the one reported in the Results and Fig 3B, these are "hazard ratio, 0.10; 95% CI, 0.01 to 0.96; P = .046".
- State the reference group for the SLiM hazard ratio (it appears to be SLiM present as reference), so that an HR below 1 reads correctly alongside "presence ... predicted progression". Consider reporting it as SLiM present vs absent (HR ≈ 10) instead.
- Given 5 events, consider stating the number of events with the HR, and describing the association as exploratory in the abstract and Discussion.
- Separately, check the 70-month TTP, OS and TTBP intervals (each excludes its own estimate). Also check the SLiM subgroup count (31 vs 28) in the biochemical-progression paragraph, and the sustained-uMRD count at 4 years (29 in the abstract, 28 in the Discussion and Table A1, 40 in the CONTEXT box).
Recomputation script (python recompute.py.txt)
"""Recompute the SLiM-CRAB hazard ratio statements of PMC11404760
(J Clin Oncol 2024, doi:10.1200/JCO.23.02771).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes, next to this file:
fig1.svg / fig1.png (= candidate A, dot-and-interval plot)
fig2.svg / fig2.png (= candidate C, reported vs CI-implied P for every HR)
candidates/cand_A..E.svg / .png (five candidate designs for review)
All inputs are copied from the article as archived in PubMed Central:
Abstract, Results: HR 0.12; 95% CI 0.14 to 1.13; P = .03 (any SLiM-CRAB -> MM)
Results, Time-To-Event End Points p3: HR 0.10 [95% CI 0.01 to 0.96]; P = .046 (Fig 3B)
Results, Time-To-Event End Points p0-p1: the three other HRs printed in the paper
Table 1 / Appendix Table A2: 28 of 90 patients with any SLiM-CRAB criterion (62 without)
Abstract / Appendix Table A2 footnotes: 4 of the 5 progressions to MM were in the SLiM group
Wald method: for a ratio with 95% CI (lo, hi) on the log scale,
SE = ln(hi/lo) / (2 * 1.959964), centre = sqrt(lo * hi), z = ln(centre) / SE.
"""
import math
import os
import numpy as np
from scipy.stats import norm, fisher_exact
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z = norm.ppf(0.975)
# ---------------------------------------------------------------- inputs (from the paper)
# (label, location, HR, lo, hi, printed P)
HRS = [
("Any SLiM-CRAB vs none -> MM (Abstract)", "Abstract, Results", 0.12, 0.14, 1.13, 0.03),
("Any SLiM vs none -> MM (Results, Fig 3B)", "Results, TTE p3", 0.10, 0.01, 0.96, 0.046),
("uMRD vs detectable MRD -> biochem. progression", "Results, TTE p0 (Fig 2B)", 0.29, 0.13, 0.66, 0.003),
("HRCA vs none -> biochem. progression", "Results, TTE p1 (Fig A1A)", 0.42, 0.22, 0.81, 0.01),
("HRCA incl. 1q vs none -> biochem. progression", "Results, TTE p1 (Fig A1B)", 0.40, 0.18, 0.85, 0.018),
]
N_TOTAL = 90
N_SLIM = 28 # Table 1 "SLiM-CRAB criteria, No. (%) | 28 (31)"; Table A2 n = 28
N_NOSLIM = 62 # Table A2 n = 62
MM_SLIM, MM_NOSLIM = 4, 1 # abstract: "four of the five patients"; five progressed in total
# Side observations: (label, location, estimate %, lo %, hi %)
KM = [
("70-month TTP", "Abstract; Results TTE p3", 94, 84, 89),
("70-month OS", "Abstract; Results TTE p4", 92, 82, 89),
("70-month TTBP", "Results TTE p0", 59, 66, 76),
]
SUSTAINED = [("Abstract", 29), ("Discussion p1 / Table A1", 28), ("CONTEXT box", 40)]
# ---------------------------------------------------------------- helpers
def wald(hr, lo, hi):
se = math.log(hi / lo) / (2 * Z)
centre = math.sqrt(lo * hi)
p_centre = 2 * norm.sf(abs(math.log(centre)) / se)
p_hr = 2 * norm.sf(abs(math.log(hr)) / se)
return se, centre, p_centre, p_hr
def fmt_p(p):
return f"{p:.3f}" if p >= 0.001 else f"{p:.1e}"
R = []
print("=== 1. Every hazard ratio printed in the paper, checked against its own 95% CI ===")
for lab, loc, hr, lo, hi, p in HRS:
se, c, pc, ph = wald(hr, lo, hi)
R.append(dict(lab=lab, loc=loc, hr=hr, lo=lo, hi=hi, p=p, se=se, c=c, pc=pc, ph=ph))
print(f"\n{lab} [{loc}]")
print(f" printed: HR {hr}; 95% CI {lo} to {hi}; P = {p}")
print(f" HR inside its CI? {lo <= hr <= hi}; CI includes 1? {lo <= 1 <= hi}")
print(f" CI centre sqrt(lo*hi) = {c:.3f}; SE(log HR) = {se:.3f}")
print(f" Wald P from CI centre = {fmt_p(pc)}; Wald P from printed HR with that SE = {fmt_p(ph)}")
A, B = R[0], R[1]
print("\n=== 2. Can rounding explain the abstract? ===")
print(" values that round to HR 0.12 lie in [0.115, 0.125); values that round to a lower limit of 0.14 lie in [0.135, 0.145)")
print(f" the two ranges overlap: {0.125 > 0.135}")
print(f" smallest z compatible with a 95% CI that includes 1 is 1.96, i.e. P >= {2 * norm.sf(Z):.3f}")
print("\n=== 3. Single-typo hypothesis: lower limit 0.014 instead of 0.14 ===")
se, c, pc, ph = wald(0.12, 0.014, 1.13)
print(f" CI 0.014 to 1.13: centre {c:.3f}; SE {se:.3f}; Wald P from centre {pc:.3f}; from HR 0.12 {ph:.3f}; includes 1: True")
z03 = norm.isf(0.03 / 2)
se03 = abs(math.log(0.12)) / z03
print(f" for HR 0.12 and P = .03 the Wald CI would be {math.exp(math.log(0.12) - Z * se03):.3f} to {math.exp(math.log(0.12) + Z * se03):.3f}")
TYPO = dict(c=c, se=se, pc=pc, ph=ph)
print("\n=== 4. Crude check from counts (ignores follow-up time; not the Cox model) ===")
r1, r0 = MM_NOSLIM / N_NOSLIM, MM_SLIM / N_SLIM
rr = r1 / r0
se_rr = math.sqrt(1 / MM_NOSLIM - 1 / N_NOSLIM + 1 / MM_SLIM - 1 / N_SLIM)
rr_lo, rr_hi = math.exp(math.log(rr) - Z * se_rr), math.exp(math.log(rr) + Z * se_rr)
p_rr = 2 * norm.sf(abs(math.log(rr)) / se_rr)
_, p_fisher = fisher_exact([[MM_NOSLIM, N_NOSLIM - MM_NOSLIM], [MM_SLIM, N_SLIM - MM_SLIM]])
print(f" progression to MM: no SLiM {MM_NOSLIM}/{N_NOSLIM} = {100 * r1:.1f}%; SLiM {MM_SLIM}/{N_SLIM} = {100 * r0:.1f}%")
print(f" crude risk ratio (no SLiM / SLiM) = {rr:.3f}; 95% CI {rr_lo:.3f} to {rr_hi:.3f}; Wald P = {p_rr:.3f}; Fisher exact P = {p_fisher:.3f}")
print(f" N_SLIM + N_NOSLIM = {N_SLIM + N_NOSLIM} (= {N_TOTAL}); 28/90 = {100 * N_SLIM / N_TOTAL:.1f}%")
CRUDE = dict(rr=rr, lo=rr_lo, hi=rr_hi, p=p_rr)
print("\n=== 5. Side observations ===")
for lab, loc, e, lo, hi in KM:
print(f" {lab} [{loc}]: {e}% (95% CI {lo} to {hi}) -> estimate inside its CI? {lo <= e <= hi}")
print(" biochemical progression by SLiM (Results TTE p1): 12/31 = %.1f%%, 18/59 = %.1f%%; 31 + 59 = %d; 12 + 18 = %d (36 biochemical progressions reported)"
% (1200 / 31, 1800 / 59, 31 + 59, 12 + 18))
for loc, n in SUSTAINED:
print(f" sustained uMRD at 4 years [{loc}]: {n}/90 = {100 * n / 90:.1f}%")
# ---------------------------------------------------------------- figure style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
W = 7.2
MS = 9 # marker size in points (>= 8 px)
LW = 1.5 # 1.5 pt = 2 px at 96 dpi
def save(fig, name, folder=HERE):
for ext in ("svg", "png"):
fig.savefig(os.path.join(folder, f"{name}.{ext}"), dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
def log_axis(ax, lo, hi, ticks, label):
ax.set_xscale("log")
ax.set_xlim(lo, hi)
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel(label, color=INK2)
LEGEND = [
Line2D([0], [0], color=ORANGE, lw=LW, marker="o", ms=MS, label="Reported in the paper"),
Line2D([0], [0], color=BLUE, lw=LW, marker="D", ms=MS - 1, label="Recomputed from the paper's own numbers"),
]
# ---------------------------------------------------------------- candidate A: dot-and-interval
def cand_A():
rows = [
("Abstract, as printed", ORANGE, "o", A["hr"], A["lo"], A["hi"],
f"HR 0.12 (0.14 to 1.13); P = .03"),
("Abstract interval: its centre\nand Wald P", BLUE, "D", A["c"], A["lo"], A["hi"],
f"{A['c']:.2f} (0.14 to 1.13); P = {A['pc']:.3f}"),
("Results text and Fig 3B,\nas printed", ORANGE, "o", B["hr"], B["lo"], B["hi"],
"HR 0.10 (0.01 to 0.96); P = .046"),
("Results interval: its centre\nand Wald P", BLUE, "D", B["c"], B["lo"], B["hi"],
f"{B['c']:.3f} (0.01 to 0.96); P = {B['pc']:.3f}"),
("Crude risk ratio from counts\n1/62 vs 4/28 (no Cox model)", BLUE, "D", CRUDE["rr"], CRUDE["lo"], CRUDE["hi"],
f"{CRUDE['rr']:.3f} ({CRUDE['lo']:.3f} to {CRUDE['hi']:.2f}); P = {CRUDE['p']:.3f}"),
]
fig, ax = plt.subplots(figsize=(W, 3.9))
y = np.arange(len(rows))[::-1].astype(float)
y[2:] -= 0.35 # gap between abstract pair and body pair
for yi, (lab, col, mk, est, lo, hi, txt) in zip(y, rows):
ax.plot([lo, hi], [yi, yi], color=col, lw=LW, solid_capstyle="butt", zorder=2)
ax.plot(est, yi, marker=mk, ms=MS if mk == "o" else MS - 1, color=col,
mec=BG, mew=1.2, zorder=3)
ax.text(3.4, yi, txt, va="center", ha="left", fontsize=10, color=INK)
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.text(0.93, y[0] + 0.75, "ratio = 1 (no association)", ha="right", va="bottom", fontsize=10, color=INK2)
ax.annotate("0.12 lies below its own\nlower limit (0.14)", xy=(0.12, y[0]), xytext=(0.0125, y[0] + 0.05),
fontsize=10, color=INK, va="center",
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=0.9, shrinkB=6))
ax.set_yticks(y)
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
log_axis(ax, 0.004, 3.0, [0.01, 0.03, 0.1, 0.3, 1, 3], "Ratio, SLiM-CRAB absent vs present (log scale)")
ax.set_ylim(y[-1] - 0.6, y[0] + 1.25)
ax.text(3.4, y[0] + 0.75, "Estimate (95% CI); P", fontsize=10, color=INK2, va="bottom")
ax.legend(handles=LEGEND, loc="upper left", bbox_to_anchor=(0.0, -0.2), ncol=2, frameon=False, fontsize=10)
ax.set_title("Same comparison, two incompatible results: any SLiM-CRAB criterion and progression to MM",
fontsize=11, loc="left", color=INK, pad=10)
return fig
# ---------------------------------------------------------------- candidate B: two text panels
def cand_B():
fig = plt.figure(figsize=(W, 3.3))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(8, 50)
ax.axis("off")
def panel(x, y, w, h, title, quote, check):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec=INK2, lw=1.0))
ax.text(x + 1.5, y + h - 2.0, title, fontsize=11, fontweight="bold", va="top")
ax.add_patch(plt.Rectangle((x + 1.5, y + h - 15.8), 0.6, 10.6, color=ORANGE, lw=0))
ax.text(x + 3.2, y + h - 5.4, quote, fontsize=10, va="top", color=INK)
ax.text(x + 1.5, y + h - 17.6, "Recomputed check:", fontsize=10, va="top", color=INK2)
ax.plot(x + 2.3, y + h - 22.0, marker="D", ms=MS - 1, color=BLUE, mec=BG)
ax.text(x + 4.2, y + h - 20.2, check, fontsize=10, va="top", color=INK)
panel(1.5, 19, 42, 29, "Abstract, Results",
"“...predicted progression to MM\n(four of the five patients; hazard\nratio, 0.12; 95% CI, 0.14 to 1.13;\nP = .03).”",
f"HR 0.12 is below the lower limit 0.14\nInterval contains 1\nCI centre {A['c']:.2f}; Wald P {A['pc']:.3f}")
panel(56.5, 19, 42, 29, "Results text (Fig 3B)",
"“...predicted progression to MM\n(HR, 0.10 [95% CI, 0.01 to 0.96];\nP = .046; Fig 3B).”",
f"HR 0.10 is inside its interval\nInterval excludes 1\nCI centre {B['c']:.3f}; Wald P {B['pc']:.3f}")
ax.annotate("", xy=(55.6, 34), xytext=(44.4, 34),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.5))
ax.text(50, 35.5, "same\nanalysis", ha="center", va="bottom", fontsize=10, color=INK2)
ax.text(50, 30.5, "different\nHR, CI, P", ha="center", va="top", fontsize=10, color=INK)
ax.text(50, 10.5, "Conflict: the two statements describe the same analysis (any SLiM criterion vs none, progression to MM,\n"
"ITT population) but give different HR, CI and P. Only the Results values are internally consistent.",
ha="center", va="bottom", fontsize=10, color=INK)
return fig
# ---------------------------------------------------------------- candidate C: reported vs CI-implied P
def cand_C():
fig, ax = plt.subplots(figsize=(W, 3.4))
y = np.arange(len(R))[::-1]
for yi, r in zip(y, R):
ax.plot([r["p"], r["pc"]], [yi + 0.13, yi - 0.13], color=GRID, lw=2, zorder=1)
ax.plot(r["p"], yi + 0.13, "o", ms=MS, color=ORANGE, mec=BG, mew=1.2, zorder=3)
ax.plot(r["pc"], yi - 0.13, "D", ms=MS - 1, color=BLUE, mec=BG, mew=1.2, zorder=3)
ax.text(1.15, yi, f"{r['p']:g} vs {r['pc']:.3f}", va="center", fontsize=10)
ax.axvline(0.05, color=INK2, lw=0.8)
ax.text(0.047, y[0] + 0.6, "P = .05", ha="right", va="bottom", fontsize=10, color=INK2)
ax.text(1.15, y[0] + 0.6, "printed vs from CI", fontsize=10, color=INK2, va="bottom")
ax.set_yticks(y)
ax.set_yticklabels([r["lab"].replace(" -> ", " → ") for r in R], fontsize=10, color=INK)
log_axis(ax, 0.001, 1.0, [0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1], "Two-sided P value (log scale)")
ax.set_ylim(-0.6, y[0] + 1.1)
leg = [Line2D([0], [0], color=ORANGE, lw=0, marker="o", ms=MS, label="P printed in the paper"),
Line2D([0], [0], color=BLUE, lw=0, marker="D", ms=MS - 1, label="Wald P implied by the printed 95% CI")]
ax.legend(handles=leg, loc="upper left", bbox_to_anchor=(0.0, -0.22), ncol=2, frameon=False, fontsize=10)
ax.set_title("Every hazard ratio in the paper: printed P vs the P its own interval implies",
fontsize=11, loc="left", pad=10)
return fig
# ---------------------------------------------------------------- candidate D: printed HR vs CI centre, all HRs
def cand_D():
fig, ax = plt.subplots(figsize=(W, 3.4))
y = np.arange(len(R))[::-1]
for yi, r in zip(y, R):
ax.plot([r["lo"], r["hi"]], [yi, yi], color=INK2, lw=LW, alpha=0.55, zorder=1, solid_capstyle="butt")
ax.plot(r["c"], yi - 0.2, "D", ms=MS - 1, color=BLUE, mec=BG, mew=1.2, zorder=3)
ax.plot(r["hr"], yi + 0.2, "o", ms=MS, color=ORANGE, mec=BG, mew=1.2, zorder=4)
ax.text(2.4, yi, f"{r['hr']:.2f} vs {r['c']:.3f}", va="center", fontsize=10)
ax.axvline(1, color=INK2, lw=0.8)
ax.text(2.4, y[0] + 0.6, "printed vs centre", fontsize=10, color=INK2, va="bottom")
ax.set_yticks(y)
ax.set_yticklabels([r["lab"].replace(" -> ", " → ") for r in R], fontsize=10, color=INK)
log_axis(ax, 0.006, 2.0, [0.01, 0.03, 0.1, 0.3, 1], "Hazard ratio (log scale); gray line = printed 95% CI")
ax.set_ylim(-0.6, y[0] + 1.1)
leg = [Line2D([0], [0], color=ORANGE, lw=0, marker="o", ms=MS, label="HR printed in the paper"),
Line2D([0], [0], color=BLUE, lw=0, marker="D", ms=MS - 1, label="Centre of the printed CI, sqrt(lo × hi)"),
Line2D([0], [0], color=INK2, alpha=0.55, lw=LW, label="Printed 95% CI")]
ax.legend(handles=leg, loc="upper left", bbox_to_anchor=(0.0, -0.22), ncol=3, frameon=False, fontsize=10)
ax.set_title("Every hazard ratio in the paper: only the abstract's sits off its interval's centre",
fontsize=11, loc="left", pad=10)
return fig
# ---------------------------------------------------------------- candidate E: consistency checklist
def cand_E():
checks = [
("Printed HR lies inside its 95% CI", ("No: 0.12 < 0.14", False), ("Yes: 0.01 ≤ 0.10 ≤ 0.96", True)),
("CI excludes 1, as P < .05 requires", ("No: interval contains 1", False), ("Yes: upper limit 0.96", True)),
("CI centre sqrt(lo × hi) ≈ printed HR", (f"No: centre {A['c']:.2f}", False), (f"Yes: centre {B['c']:.3f}", True)),
("Wald P from CI ≈ printed P", (f"No: {A['pc']:.3f} vs .03", False), (f"Yes: {B['pc']:.3f} vs .046", True)),
]
fig = plt.figure(figsize=(W, 3.3))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 50)
ax.axis("off")
cx = [2, 38, 72]
for x, t1, t2 in ((cx[1], "Abstract", "HR 0.12 (0.14 to 1.13); P = .03"),
(cx[2], "Results text / Fig 3B", "HR 0.10 (0.01 to 0.96); P = .046")):
ax.text(x, 46, t1, fontsize=10, va="top", fontweight="bold")
ax.text(x, 42.6, t2, fontsize=10, va="top")
ax.add_patch(plt.Rectangle((x - 1.2, 38.6), 0.6, 7.4, color=ORANGE, lw=0))
ax.text(cx[0], 46, "Check, recomputed from the\npaper's own numbers", fontsize=10, va="top", color=INK2)
for i, (lab, a, b) in enumerate(checks):
yy = 32 - i * 8
ax.plot([1, 99], [yy + 4.4, yy + 4.4], color=GRID, lw=1)
ax.text(cx[0], yy, lab, fontsize=10, va="center")
for x, (txt, ok) in ((cx[1], a), (cx[2], b)):
ax.plot(x + 0.6, yy, marker="D", ms=MS - 2, color=BLUE, mec=BG)
ax.text(x + 2.2, yy, ("✓ " if ok else "✗ ") + txt, fontsize=10, va="center",
fontweight="normal" if ok else "bold")
ax.text(cx[0], 1.5, "Orange bar: values printed in the paper. Blue diamond: check recomputed from them (Wald method, log scale).",
fontsize=10, color=INK2, va="center")
return fig
for name, fn in zip("ABCDE", (cand_A, cand_B, cand_C, cand_D, cand_E)):
save(fn(), f"cand_{name}", CAND)
save(cand_A(), "fig1")
save(cand_C(), "fig2")
print("\nwrote fig1.*, fig2.* and candidates/cand_A..E.* in", HERE)
Abstract reports two obesity-group odds ratios as the overweight and obesity estimates for EPT use
JAMA Netw Open (2025) · doi:10.1001/jamanetworkopen.2025.19236 · PMC12235495
Summary
The study pools 31 case-control studies to estimate associations between exogenous hormone use and intrinsic-like breast cancer subtypes, stratified by body mass index (BMI). The abstract says the associations of current estrogen-progestin therapy (EPT) use with luminal A–like and luminal B–ERBB2-like tumors "were attenuated but remained" for women with overweight (OR 1.40, 95% CI 1.02–1.92) or obesity (OR 1.68, 95% CI 1.01–2.78). In the paper's own Results text and in Supplement 1 eTable 3, both of these values come from the obesity stratum: 1.40 is the luminal A–like OR and 1.68 is the luminal B–ERBB2-like OR. The overweight estimates, 2.18 (1.83–2.61) for luminal A–like and 1.83 (1.32–2.55) for luminal B–ERBB2-like, do not appear in the abstract. No direction or significance changes, but an abstract reader would take the overweight luminal A–like excess odds to be about a third of what the paper's own data show, and would see a BMI pattern (1.40 for overweight, then 1.68 for obesity) that the full results do not show.
What the paper reports
Abstract, Results:
Current EPT users with healthy weight were more likely to be diagnosed with luminal A–like (odds ratio [OR], 2.51 [95% CI, 2.26-2.80]) or luminal B–ERBB2-like (OR, 1.95 [95% CI, 1.61-2.37]) subtypes. These associations were attenuated but remained for individuals with overweight (OR, 1.40 [95% CI, 1.02-1.92]) or obesity (OR, 1.68 [95% CI, 1.01-2.78]).
Results, "Combined EPT Use and Tumor Subtypes", paragraph 2:
In women with healthy weight, current EPT users were more likely to be diagnosed with luminal A–like (OR, 2.51 [95% CI, 2.26-2.80]), luminal B–like (OR, 1.47 [95% CI, 1.17-1.86]), or luminal B–ERBB2–like (OR, 1.95 [95% CI, 1.61-2.37]) breast tumors, with significant heterogeneity across subtypes (P < .001 for heterogeneity).
Results, "Combined EPT Use and Tumor Subtypes", paragraph 3:
In women with overweight, current EPT use was positively associated with luminal A–like (OR, 2.18 [95% CI, 1.83-2.61]) and luminal B–ERBB2-like (OR, 1.83 [95% CI, 1.32-2.55]) tumors but not with other subtypes (P < .001 for heterogeneity). In women with obesity, associations for luminal A–like (OR, 1.40 [95% CI, 1.02-1.92]) and luminal B–ERBB2-like (OR, 1.68 [95% CI, 1.01-2.78]) tumors were seen, with smaller effect sizes and nonsignificant heterogeneity (P < .11 for heterogeneity).
Methods, Statistical Analysis (the only BMI definition in the paper):
To evaluate heterogeneity in associations by BMI, associations were stratified by BMI categories as follows: lean or healthy weight (hereinafter, healthy weight) (18.5-<25 [calculated as weight in kilograms divided by height in meters squared]), overweight (25-<30), or obesity (≥30).
Supplement 1, eTable 3 (EPT use by intrinsic-like subtype, stratified by BMI), "Current use" rows: Lean/normal, luminal A-like 2.51 (2.26,2.80) and luminal B-ERBB2-like 1.95 (1.61,2.37). Overweight, 2.18 (1.83,2.61) and 1.83 (1.32,2.55). Obese, 1.40 (1.02,1.92) and 1.68 (1.01,2.78).
All quotes from the article were checked verbatim against the PMC-archived text with src/show.py PMC12235495 --grep. The eTable values were read from the supplement PDF.
What the paper's own numbers show
Method. I took every current-EPT odds ratio and 95% CI for luminal A–like and luminal B–ERBB2-like tumors from the Results text, and checked that each CI is symmetric about its OR on the log scale. I checked the text values against Supplement 1 eTable 3. I then matched each (OR, lower, upper) triple in the abstract against every BMI-by-subtype cell in the Results. To size the discrepancy, I compared the abstract's "overweight" value with the paper's overweight estimates as a ratio of ORs and as excess odds (OR − 1). I computed a z statistic for the difference of log ORs, using standard errors taken from the CI widths. That z treats the overweight and obesity strata as independent, which holds because they are disjoint groups of women. All calculations are in recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Healthy weight, luminal A–like | Abstract; Results para. 2; eTable 3 | 2.51 (2.26–2.80) | Abstract and Results identical; geometric CI midpoint 2.516 |
| Healthy weight, luminal B–ERBB2-like | Abstract; Results para. 2; eTable 3 | 1.95 (1.61–2.37) | Abstract and Results identical; midpoint 1.953 |
| Overweight, luminal A–like | Results para. 3; eTable 3 | 2.18 (1.83–2.61) | Midpoint 2.185, consistent; not in the abstract |
| Overweight, luminal B–ERBB2-like | Results para. 3; eTable 3 | 1.83 (1.32–2.55) | Midpoint 1.835, consistent; not in the abstract |
| Obesity, luminal A–like | Results para. 3; eTable 3 | 1.40 (1.02–1.92) | Midpoint 1.399, consistent |
| Obesity, luminal B–ERBB2-like | Results para. 3; eTable 3 | 1.68 (1.01–2.78) | Midpoint 1.676, consistent |
| Abstract value labelled "overweight" | Abstract, Results | 1.40 (1.02–1.92) | Equals only the obesity, luminal A–like cell; matches no overweight cell |
| Abstract value labelled "obesity" | Abstract, Results | 1.68 (1.01–2.78) | Equals only the obesity, luminal B–ERBB2-like cell |
| Overweight luminal A–like vs abstract "overweight" | Results vs Abstract | 2.18 vs 1.40 | Ratio of ORs 1.56; excess odds 1.18 vs 0.40, so the abstract shows 34% of it; z = 2.39 for the difference |
| Overweight luminal B–ERBB2-like vs abstract "overweight" | Results vs Abstract | 1.83 vs 1.40 | Ratio of ORs 1.31; excess odds 0.83 vs 0.40 (48%); z = 1.15 |
| BMI gradient, luminal A–like | Results paras. 2–3 | 2.51 → 2.18 → 1.40 | Falls steadily with BMI |
| BMI gradient, luminal B–ERBB2-like | Results paras. 2–3 | 1.95 → 1.83 → 1.68 | Falls steadily with BMI |
| BMI gradient as the abstract presents it | Abstract | overweight 1.40, obesity 1.68 | Rises from overweight to obesity, which no subtype in the Results does |
| Significance, overweight and obesity cells | Results para. 3 | All four CIs exclude 1 | z (log OR / SE) = 8.60, 3.60, 2.09, 2.01; unchanged |
| Fully adjusted model (sensitivity) | eTable 7 | Overweight 2.18 (1.82–2.61), 1.81 (1.30–2.52); obese 1.41 (1.02–1.93), 1.65 (0.99–2.75) | No overweight cell equals 1.40 (1.02–1.92) |
Every value in the Results text agrees with eTable 3, and every CI is consistent with its point estimate. The other four current-EPT values in the abstract (2.51, 1.95 and 1.47 for healthy weight, 1.68 for obesity) match Results values for the BMI group they are given under. The only mismatch is the BMI label on the first of the two higher-BMI values.
Figure
Figure 1. On the left (orange) is the abstract's sentence on attenuation, with the value attached to each BMI group as printed. On the right (blue) are the current-EPT odds ratios (95% CI) for women with overweight and obesity from Results paragraph 3, which are identical to the Supplement 1 eTable 3 "Current use" rows. Arrows link each abstract value to the only Results cell that has the same OR and CI. Both arrows end in the obesity row. The abstract's "overweight" value is the obesity, luminal A–like estimate. The shaded overweight row (2.18 and 1.83) is not reported anywhere in the abstract.
Figure 2. Odds ratios with 95% CIs for current EPT use vs never use, on a log axis with a thin reference line at OR = 1. Blue: values from the Results text and eTable 3 (circles, luminal A–like; squares, luminal B–ERBB2-like). Orange diamonds: the abstract's values, drawn under the BMI group the abstract names. In the healthy-weight and obesity groups each orange interval repeats a blue one. In the overweight group the abstract's interval (1.02–1.92) matches neither overweight estimate. It reproduces the obesity, luminal A–like interval one group lower. The blue luminal A–like estimates fall steadily from 2.51 to 2.18 to 1.40.
Why it matters
The BMI stratification is central to how the paper frames its EPT findings: the Key Points and Conclusions stress associations "particularly in participants with healthy weight", and the Discussion describes the luminal A–like and luminal B–ERBB2-like associations as "consistent across BMI groups, albeit attenuated with greater adiposity". The abstract is the only part of the paper that most readers, and most citing articles, see. A reader who relies on it would conclude that among women with overweight, current EPT use is associated with an OR of about 1.40. By the paper's own estimates the overweight luminal A–like OR is 2.18. That is roughly three times the excess odds (1.18 vs 0.40), and the difference is larger than sampling error alone would plausibly produce (z ≈ 2.4). The mislabeling also distorts the dose-response picture by BMI: the abstract's numbers rise from overweight (1.40) to obesity (1.68), while the full results fall steadily with BMI for both subtypes. Because neither value in the sentence is tied to a subtype, a reader also cannot tell that the two numbers refer to different tumor subtypes.
The consequences are limited. All four overweight and obesity estimates are above 1 with CIs that exclude 1, so the abstract's qualitative statement ("attenuated but remained") is true for both BMI groups. The title, Key Points, abstract Conclusions, body Results, Discussion, Figure 1 and the supplement are correct. The estrogen-only therapy and oral contraceptive findings, and the heterogeneity tests, are not affected. The error is a misreported secondary estimate in the abstract, not a change to any headline conclusion.
Explanations considered
- A different model, population or adjustment set in the abstract. The abstract and the Results both describe current EPT use vs nonuse in postmenopausal women, from the same polytomous logistic model stratified by BMI. No footnote, legend or Methods statement points to a different model for the abstract. The fully adjusted sensitivity model (eTable 7) gives overweight estimates of 2.18 and 1.81, not 1.40. This does not explain the numbers.
- A different BMI definition. The Methods and the abstract's own Design section use one set of cut points (overweight 25–<30, obesity ≥30), and eTable 3 uses the same. This does not explain the numbers.
- The value belongs to another overweight estimate. I checked whether 1.40 (1.02–1.92) is an overweight value for some other subtype or exposure. The overweight current-EPT estimates in the text are 2.18 and 1.83. The overweight past-EPT values mentioned in the Results are 1.50 and 1.57, both with different CIs. In the whole of Supplement 1, the exact triple 1.40 (1.02, 1.92) occurs once, in the Obese, current-use, luminal A-like cell of eTable 3. The four other ORs of 1.40 in the supplement all have different CIs. This does not explain the numbers.
- "Overweight or obesity" meant as one combined group. On this reading the two values would be the luminal A–like and luminal B–ERBB2-like ORs for women with higher BMI. The sentence's grammar does not support that reading: each value sits directly after a different BMI category, and no subtype is named. Even on this reading, both values come from the obesity stratum alone, and the overweight stratum, which has the larger estimates, is left out. This explanation reduces the problem to ambiguous wording, but the numbers would still be attributed to women with overweight when they come only from women with obesity.
- Rounding or transcription within the same cell. The abstract triples match the obesity cells to every printed digit and differ from the overweight cells by 0.78 and 0.43 on the OR scale. This is not a rounding effect.
- Text-conversion artifact in PMC. The same wording appears on the publisher's full-text page, on the PMC HTML page and in the Europe PMC full-text XML. This does not explain the numbers.
The pattern fits a mislabeled sentence in the abstract: the two obesity-row values were placed after "overweight" and "obesity" in place of an overweight value and an obesity value for the same subtype. Nothing in the paper shows how this happened.
Status
As of 2026-10-23:
- Version of record. doi.org resolves to the publisher's full-text page (jamanetwork.com/journals/jamanetworkopen/fullarticle/2836044), which was read in this workflow. Its abstract has the same "overweight (OR, 1.40 [95% CI, 1.02-1.92]) or obesity (OR, 1.68 [95% CI, 1.01-2.78])" wording. Its body Results give the same overweight (2.18, 1.83) and obesity (1.40, 1.68) values as PMC. The PMC page and the Europe PMC full-text XML carry identical text.
- Supplements. Supplement 1 (jamanetwopen-e2519236-s001.pdf: eMethods, eTables 1–29, eFigures 1–3) was downloaded via Europe PMC and read. eTable 3 agrees with the Results text in every cell used here. eTable 7, the fully adjusted model, gives similar values. Neither table has 1.40 (1.02–1.92) or 1.68 (1.01–2.78) in an overweight row. The supplement confirms the Results text and resolves which stratum the abstract values belong to.
- Corrections. None found.
src/known.pyreturned no Crossref updates and no PMC or Europe PMC notices. The Crossref works record has no update-to or updated-by relation. A Crossreffilter=updates:query returned 0 results. The Crossref Labs record, which carries the Retraction Watch data, lists no updates. Europe PMC (PMID 40622713) lists no comment or correction. The publisher page shows no correction notice. Earlier web searches in this workflow for the title together with "correction" or "erratum" found nothing relevant. - PubPeer. The PubPeer search API and search page both returned HTTP 403 and could not be queried directly. An earlier web search in this workflow for the DOI together with "PubPeer" found no thread. A repeat web search could not be run for this write-up because the session's search quota was exhausted.
Sources: publisher full text; PMC12235495; Europe PMC REST full-text XML and supplementary files for PMC12235495; Crossref REST API for doi:10.1001/jamanetworkopen.2025.19236.
Suggested correction
Abstract, Results. Replace "These associations were attenuated but remained for individuals with overweight (OR, 1.40 [95% CI, 1.02-1.92]) or obesity (OR, 1.68 [95% CI, 1.01-2.78])." with:
These associations were attenuated but remained for individuals with overweight (luminal A–like: OR, 2.18 [95% CI, 1.83-2.61]; luminal B–ERBB2-like: OR, 1.83 [95% CI, 1.32-2.55]) or obesity (luminal A–like: OR, 1.40 [95% CI, 1.02-1.92]; luminal B–ERBB2-like: OR, 1.68 [95% CI, 1.01-2.78]).
If word limits require a shorter sentence, the luminal A–like values alone (overweight 2.18 [1.83-2.61]; obesity 1.40 [1.02-1.92]) would give the BMI gradient correctly. All values are those already given in the Results text and in Supplement 1 eTable 3. No other part of the paper needs to change.
Recomputation script (python recompute.py.txt)
"""Check the BMI-stratified current-EPT odds ratios in the abstract of PMC12235495
(JAMA Netw Open 2025; doi:10.1001/jamanetworkopen.2025.19236) against the paper's own
Results text and Supplement 1 eTable 3, and redraw fig1/fig2 in this file's folder.
Run: python recompute.py.txt (needs numpy, matplotlib)
All inputs are copied from the published article (Abstract; Results, "Combined EPT Use
and Tumor Subtypes", paragraphs 2 and 3) and from Supplement 1 (eTables 3 and 7).
"""
import os
import math
import numpy as np
HERE = os.path.dirname(os.path.abspath(__file__))
Z = 1.959964
# ---------------------------------------------------------------- inputs
# Results text: current EPT use vs never use, by BMI stratum and intrinsic-like subtype.
# (OR, lower 95% CI, upper 95% CI). Identical values in Supplement 1 eTable 3, "Current use" rows.
RES = {
("healthy weight", "luminal A-like"): (2.51, 2.26, 2.80), # Results para 2
("healthy weight", "luminal B-like"): (1.47, 1.17, 1.86), # Results para 2
("healthy weight", "luminal B-ERBB2-like"): (1.95, 1.61, 2.37), # Results para 2
("overweight", "luminal A-like"): (2.18, 1.83, 2.61), # Results para 3
("overweight", "luminal B-ERBB2-like"): (1.83, 1.32, 2.55), # Results para 3
("obesity", "luminal A-like"): (1.40, 1.02, 1.92), # Results para 3
("obesity", "luminal B-ERBB2-like"): (1.68, 1.01, 2.78), # Results para 3
}
# Supplement 1, eTable 3 (base model), "Current use" rows, as printed.
ETABLE3 = {
("healthy weight", "luminal A-like"): (2.51, 2.26, 2.80),
("healthy weight", "luminal B-ERBB2-like"): (1.95, 1.61, 2.37),
("overweight", "luminal A-like"): (2.18, 1.83, 2.61),
("overweight", "luminal B-ERBB2-like"): (1.83, 1.32, 2.55),
("obesity", "luminal A-like"): (1.40, 1.02, 1.92),
("obesity", "luminal B-ERBB2-like"): (1.68, 1.01, 2.78),
}
# Supplement 1, eTable 7 (fully adjusted model), "Current use" rows.
ETABLE7 = {
("overweight", "luminal A-like"): (2.18, 1.82, 2.61),
("overweight", "luminal B-ERBB2-like"): (1.81, 1.30, 2.52),
("obesity", "luminal A-like"): (1.41, 1.02, 1.93),
("obesity", "luminal B-ERBB2-like"): (1.65, 0.99, 2.75),
}
# Abstract, Results: the BMI group each value is attached to (subtype as stated, if any).
ABS = [
("healthy weight", "luminal A-like", (2.51, 2.26, 2.80)),
("healthy weight", "luminal B-ERBB2-like", (1.95, 1.61, 2.37)),
("overweight", "(no subtype named)", (1.40, 1.02, 1.92)),
("obesity", "(no subtype named)", (1.68, 1.01, 2.78)),
("healthy weight", "luminal B-like", (1.47, 1.17, 1.86)),
]
def se_log(lo, hi):
return (math.log(hi) - math.log(lo)) / (2 * Z)
def gmid(lo, hi):
return math.exp((math.log(lo) + math.log(hi)) / 2)
def fmt(t):
return f"{t[0]:.2f} ({t[1]:.2f}-{t[2]:.2f})"
print("=== 1. Internal consistency of the Results values (CI symmetric on the log scale) ===")
for k, (o, lo, hi) in RES.items():
print(f" {k[0]:15s} {k[1]:22s} OR {o:.2f} geometric midpoint of CI {gmid(lo, hi):.3f} "
f"SE(log OR) {se_log(lo, hi):.3f}")
print("\n=== 2. Results text vs Supplement 1 eTable 3 ===")
for k, v in ETABLE3.items():
print(f" {k[0]:15s} {k[1]:22s} text {fmt(RES[k])} eTable 3 {fmt(v)} identical: {RES[k] == v}")
print("\n=== 3. Which Results estimate does each abstract value equal? (exact OR + CI match) ===")
for bmi, sub, v in ABS:
hits = [f"{b} / {s}" for (b, s), rv in RES.items() if rv == v]
same = any(b == bmi for (b, s), rv in RES.items() if rv == v)
print(f" abstract '{bmi}' {sub:22s} {fmt(v)} -> Results: {hits}; same BMI group: {same}")
ow = {s: v for (b, s), v in RES.items() if b == "overweight"}
print(f" Results overweight estimates: " + "; ".join(f"{s} {fmt(v)}" for s, v in ow.items()))
print(f" Does 1.40 (1.02-1.92) appear among any overweight estimate? "
f"{any(v == (1.40, 1.02, 1.92) for v in ow.values())}")
print(" Fully adjusted eTable 7 values for the same cells: " +
"; ".join(f"{b}/{s} {fmt(v)}" for (b, s), v in ETABLE7.items()))
print(f" Does the abstract's 'overweight' triple equal any eTable 7 overweight cell? "
f"{any(v == (1.40, 1.02, 1.92) for (b, s), v in ETABLE7.items() if b == 'overweight')}")
print("\n=== 4. Size of the discrepancy for women with overweight ===")
a = ABS[2][2]
for s, v in ow.items():
ex_rep, ex_true = a[0] - 1, v[0] - 1
d = math.log(v[0]) - math.log(a[0])
se_d = math.sqrt(se_log(v[1], v[2]) ** 2 + se_log(a[1], a[2]) ** 2)
print(f" {s:22s} abstract 1.40 vs Results {v[0]:.2f}: ratio of ORs {v[0]/a[0]:.2f}; "
f"excess odds {ex_rep:.2f} vs {ex_true:.2f} -> abstract shows {100*ex_rep/ex_true:.0f}% "
f"of the excess (Results value is {ex_true/ex_rep:.1f}x)")
print(f" {'':22s} difference of log ORs {d:.3f}, SE {se_d:.3f}, z {d/se_d:.2f} "
f"(treating the overweight and obesity strata as independent samples)")
print(f" Overweight CI lower bounds vs abstract value: luminal A-like lower bound 1.83 > 1.40: {1.83 > 1.40}; "
f"the two CIs overlap only between 1.83 and 1.92")
print("\n=== 5. BMI gradient as shown by the abstract and by the Results ===")
for s in ["luminal A-like", "luminal B-ERBB2-like"]:
seq = [RES[(b, s)][0] for b in ["healthy weight", "overweight", "obesity"]]
print(f" Results {s:22s} healthy -> overweight -> obesity: {seq}; monotone decreasing: "
f"{all(x > y for x, y in zip(seq, seq[1:]))}")
print(f" Abstract 'overweight' 1.40 < 'obesity' 1.68: looks non-monotone: {1.40 < 1.68}")
print("\n=== 6. Significance unchanged ===")
for k in [("overweight", "luminal A-like"), ("overweight", "luminal B-ERBB2-like"),
("obesity", "luminal A-like"), ("obesity", "luminal B-ERBB2-like")]:
o, lo, hi = RES[k]
print(f" {k[0]:11s} {k[1]:22s} CI excludes 1: {lo > 1} z = {math.log(o)/se_log(lo, hi):.2f}")
# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, Rectangle, FancyArrowPatch
from matplotlib.lines import Line2D
BG, TXT, SEC, ORA, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "font.family": "DejaVu Sans",
"svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, f"{name}.svg"), facecolor=BG, bbox_inches="tight")
fig.savefig(os.path.join(HERE, f"{name}.png"), facecolor=BG, bbox_inches="tight", dpi=200)
plt.close(fig)
# ---- Figure 1: abstract vs Results text diagram
fig = plt.figure(figsize=(7.2, 4.9), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(-7, 62); ax.axis("off")
ax.add_patch(FancyBboxPatch((1.5, 6), 34.5, 50, boxstyle="round,pad=0.4,rounding_size=1.5",
fc="white", ec=ORA, lw=2))
ax.text(4, 52.5, "Abstract, Results", fontsize=11.5, weight="bold", va="top")
ax.text(4, 47.5, "As labelled in the abstract", fontsize=10.5, color=ORA, weight="bold", va="top")
ax.text(4, 42, "\u201cThese associations were\nattenuated but remained for\nindividuals with \u2026\u201d",
fontsize=10.5, style="italic", va="top", linespacing=1.35)
ax.text(4, 27.6, "overweight", fontsize=10.5, color=SEC, va="center")
ax.text(4, 23.6, "OR 1.40 (1.02\u20131.92)", fontsize=11, color=ORA, weight="bold", va="center")
ax.text(4, 16.6, "obesity", fontsize=10.5, color=SEC, va="center")
ax.text(4, 12.6, "OR 1.68 (1.01\u20132.78)", fontsize=11, color=ORA, weight="bold", va="center")
ax.text(4, 8.4, "No subtype named for either", fontsize=10, color=SEC, va="center")
# Results grid (row labels on the right)
x0, cw, ch = 57, 15.5, 11
ax.text(x0 + cw, 58, "Results text, para. 3", fontsize=11.5, weight="bold", ha="center", va="top")
ax.text(x0 + cw, 53.5, "(identical in eTable 3)", fontsize=10.5, color=BLU, ha="center", va="top")
ax.text(x0 + cw / 2, 45, "Luminal A\u2013like", ha="center", fontsize=10.5, color=SEC)
ax.text(x0 + 1.5 * cw, 45, "Luminal B\u2013\nERBB2-like", ha="center", fontsize=10.5, color=SEC,
linespacing=1.1)
rows = [("overweight", 32), ("obesity", 21)]
for b, y in rows:
ax.text(x0 + 2 * cw + 1.2, y + ch / 2, b.capitalize(), ha="left", va="center", fontsize=10.5, color=SEC)
for j, s in enumerate(["luminal A-like", "luminal B-ERBB2-like"]):
o, lo, hi = RES[(b, s)]
ax.add_patch(Rectangle((x0 + j * cw, y), cw, ch, fc="#e8f0fb" if b == "overweight" else "white",
ec=BLU, lw=2 if b == "obesity" else 1.2))
ax.text(x0 + j * cw + cw / 2, y + ch * 0.64, f"{o:.2f}", ha="center", va="center",
fontsize=12, weight="bold", color=BLU)
ax.text(x0 + j * cw + cw / 2, y + ch * 0.28, f"({lo:.2f}\u2013{hi:.2f})", ha="center",
va="center", fontsize=10, color=SEC)
ax.text(x0 - 1.5, 37.5, "Overweight row\n(shaded): not in\nthe abstract", ha="right", va="center",
fontsize=10, color=SEC, linespacing=1.2)
ax.text(x0 + 2 * cw + 1.2, 16, "Current EPT\nvs never use", ha="left", va="top", fontsize=10,
color=SEC, linespacing=1.2)
# connectors
ax.add_patch(FancyArrowPatch((32, 23.6), (x0 - 0.3, 26.5), arrowstyle="-|>", mutation_scale=13,
lw=2, color=TXT, shrinkA=2, shrinkB=2))
ax.add_patch(FancyArrowPatch((32, 12.6), (x0 + 1.5 * cw, 21 - 0.3), arrowstyle="-|>", mutation_scale=13,
lw=2, color=TXT, connectionstyle="arc3,rad=0.3", shrinkA=2, shrinkB=2))
ax.text(50, -1.8, "Conflict: both abstract values are the obesity-row estimates. The value labelled\n"
"\u201coverweight\u201d is the obesity, luminal A\u2013like OR; the overweight luminal A\u2013like\n"
"OR is 2.18 (1.83\u20132.61) and the overweight luminal B\u2013ERBB2-like OR is 1.83 (1.32\u20132.55).",
ha="center", va="center", fontsize=10, linespacing=1.3)
save(fig, "fig1")
# ---- Figure 2: dot-and-interval plot, abstract labels vs Results values, by BMI
fig, ax = plt.subplots(figsize=(7.2, 4.8), facecolor=BG)
ax.set_facecolor(BG)
rows = [] # (label for y group, kind, value, text)
plan = [
("healthy weight", [("res", "luminal A-like"), ("res", "luminal B-ERBB2-like"),
("abs", 0, "abstract, luminal A\u2013like"), ("abs", 1, "abstract, luminal B\u2013ERBB2-like")]),
("overweight", [("res", "luminal A-like"), ("res", "luminal B-ERBB2-like"),
("abs", 2, "abstract, \u201coverweight\u201d")]),
("obesity", [("res", "luminal A-like"), ("res", "luminal B-ERBB2-like"),
("abs", 3, "abstract, \u201cobesity\u201d")]),
]
mk = {"luminal A-like": "o", "luminal B-ERBB2-like": "s"}
y, step, gap, centers = 0.0, 0.34, 0.55, {}
for b, items in plan:
ys = []
for it in items:
if it[0] == "res":
o, lo, hi = RES[(b, it[1])]
col, m, lab = BLU, mk[it[1]], f"{o:.2f} {it[1].replace('-like', '')}"
else:
o, lo, hi = ABS[it[1]][2]
col, m, lab = ORA, "D", f"{o:.2f} {it[2]}"
ax.plot([lo, hi], [-y, -y], color=col, lw=2, solid_capstyle="butt")
ax.plot(o, -y, marker=m, ms=8, color=col, ls="none")
ax.text(hi * 1.03, -y, lab, va="center", fontsize=10, color=col)
ys.append(-y); y += step
centers[b] = np.mean(ys); y += gap
ax.axvline(1, color=SEC, lw=0.8)
ax.set_xscale("log")
ax.set_xlim(0.9, 7.0)
ticks = [1, 1.5, 2, 3, 4]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:g}" for t in ticks]); ax.minorticks_off()
ax.set_yticks(list(centers.values()))
ax.set_yticklabels(["Healthy\nweight", "Overweight", "Obesity"], fontsize=10.5)
ax.set_ylim(-y + gap - 0.3, 0.3)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Odds ratio, current EPT use vs never use (log scale; 95% CI)", fontsize=10.5)
h = [Line2D([], [], color=BLU, marker="o", ms=8, lw=2, label="Results text / eTable 3, luminal A\u2013like"),
Line2D([], [], color=BLU, marker="s", ms=8, lw=2, label="Results text / eTable 3, luminal B\u2013ERBB2-like"),
Line2D([], [], color=ORA, marker="D", ms=8, lw=2, label="Abstract, under the BMI group it names")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.45, -0.15), ncol=1, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nFigures written: fig1.svg/png, fig2.svg/png in", HERE)
Abstract reports a diastolic blood pressure difference that the trial's own analyses do not show
JAMA Network Open (2021) · doi:10.1001/jamanetworkopen.2021.12417 · PMC8176331
Summary
This randomized trial of a smartphone lifestyle app (204 adults with type 2 diabetes in Singapore) lists diastolic blood pressure in its abstract among the outcomes with "intergroup differences favoring the intervention". The paper's Table 2 gives the diastolic between-group differences as −2.6 mm Hg (95% CI −6.2 to 0.9; P = .15) at 3 months and −2.4 mm Hg (95% CI −5.7 to 1.0; P = .17) at 6 months, and the complete-case analysis in Supplement 1 gives P = .13 and .08. All four intervals include zero, recomputation from the printed means and SDs agrees with them, and the paper's own Results text says that between-group differences in blood pressure were not observed at 6 months. The primary outcome (weight) and the glycemic results are unaffected, but the abstract credits the app with a blood pressure benefit that none of the paper's reported analyses supports at its stated threshold of 2-sided P < .05.
What the paper reports
Abstract, Results (the sentence follows the 6-month weight, HbA1c and medication results):
Intergroup differences favoring the intervention were also noted for fasting blood glucose, diastolic blood pressure, and dietary changes.
Results, "Metabolic Outcomes":
Between-group differences in blood pressure and lipids were not observed at 6 months.
Table 2, "Change in diastolic blood pressure, mm Hg" (blood pressure block: control group, 72 participants; intervention group, 67 participants). Columns: participants, control mean (SD) change, intervention mean (SD) change, mean difference (95% CI), P value, Cohen d. Superscript b is reproduced as "b":
3 mo | 139 | -1.8 (11.0) | -4.4 (9.9)b | -2.6 (-6.2 to 0.9) | .15 | 0.25
6 mo | 139 | -3.1 (9.2)b | -5.4 (11.1)b | -2.4 (-5.7 to 1.0) | .17 | 0.23
Table 2, footnote b (it marks within-group change from baseline, not between-group differences):
Statistically significant change from baseline to postintervention at P < .05, after Benjamini-Hochberg correction with false discovery rate at 0.20 and with 76 participants.
Supplement 1, eTable 1 (complete-case analysis), "∆ Diastolic blood pressure (mmHg)", as extracted from the PDF:
3 months 132 -1.6 (10.6) -4.1 (8.8)* -2.6 (-5.9 to 0.8) .13 0.26
6 months 131 -2.5 (7.9)* -5.4 (10.9)* -2.9 (-6.2 to 0.4) .08 0.30
Methods, "Statistical Analysis":
Type I error for multiple comparisons was adjusted using the Benjamini-Hochberg procedure with false discovery rate at 0.20.
Statistical significance was set at 2-sided P < .05.
Results, "Participants":
Baseline characteristics were similar between study groups, except for a significantly higher diastolic blood pressure in the control group.
What the paper's own numbers show
Method. The paper fitted generalized linear mixed models to change from baseline on 5 multiply imputed data sets, with recruitment source as a random effect. That model cannot be refitted from published summaries, so I checked the printed numbers against each other in two ways. (a) From the printed mean (SD) changes and group sizes (72 and 67 for blood pressure), I computed the intervention-minus-control difference with a pooled-variance t 95% CI and 2-sided P, a Welch P, and Cohen d. (b) I back-calculated the 2-sided P implied by each printed 95% CI (normal approximation, SE = CI width / 3.92). eTable 1 does not give the group split behind its n = 132 and 131, so its rows assume 66/66 and 66/65; across every split consistent with at most 72 control and 67 intervention participants, the recomputed P ranges only from .142 to .148 (3 mo) and .080 to .085 (6 mo). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Diastolic BP difference, 3 mo (multiple imputation) | Table 2 | −2.6 (−6.2 to 0.9); P = .15; d = 0.25 | −2.60 (−6.12 to 0.92); P = .146 (Welch .145; from printed CI .151); d = 0.25 |
| Diastolic BP difference, 6 mo (multiple imputation) | Table 2 | −2.4 (−5.7 to 1.0); P = .17; d = 0.23 | −2.30 from the rounded arm means (−5.71 to 1.11); P = .185 (Welch .188; from printed CI .160); d = 0.23 |
| Diastolic BP difference, 3 mo (complete case) | Supplement 1, eTable 1 | −2.6 (−5.9 to 0.8); P = .13 | −2.50 (−5.85 to 0.85); P = .143 (from printed CI .128) |
| Diastolic BP difference, 6 mo (complete case) | Supplement 1, eTable 1 | −2.9 (−6.2 to 0.4); P = .08 | −2.90 (−6.19 to 0.39); P = .083 (from printed CI .085) |
| One-sided P implied by the printed CIs (not the paper's convention) | Table 2; eTable 1 | — | .076 and .080 (multiple imputation, 3 and 6 mo); .064 and .042 (complete case) |
| Systolic BP difference, 3 mo / 6 mo (for comparison) | Table 2 | −4.2 (−8.8 to 0.3), P = .07 / −2.8 (−7.7 to 2.0), P = .25 | −4.30 (−8.98 to 0.38), P = .072 / −2.80 (−7.66 to 2.06), P = .257 |
| Systolic BP difference, 3 mo (complete case) | Supplement 1, eTable 1 | −4.7 (−9.3 to −0.2); P = .04 | −4.70 (−9.25 to −0.15); P = .043, the only blood pressure comparison in the paper below .05 |
| Fasting blood glucose difference, 6 mo (named in the same abstract sentence) | Table 2 | −12.6 (−23.4 to −3.6); P = .01 | −12.6 (−21.4 to −3.8); P = .005 |
| Calorie intake difference, 6 mo (named in the same abstract sentence, "dietary changes") | Table 2 | −305.8 (−441.0 to −170.7); P < .001 | −305.5 (−441.1 to −169.9); P < .001 |
| Baseline diastolic BP, control vs intervention | Table 1; Results | 85.7 (9.8) vs 82.7 (8.8); "significantly higher" in control | difference 3.0 mm Hg; t = 2.30, P = .023 |
| Participants with blood pressure data | Table 2 | 139 of 204 randomized | 68.1% |
Both multiple-imputation analyses and both complete-case analyses of diastolic blood pressure have 95% CIs that include zero, and recomputation reproduces each printed P value to within rounding. Of the outcomes named in the abstract sentence, fasting blood glucose and the dietary outcomes have between-group P values below .05 in Table 2. Diastolic blood pressure does not, in any analysis the paper reports.
Figure
Figure 1. The conflicting statements, quoted from the article. Left: the abstract lists diastolic blood pressure among the outcomes with intergroup differences favoring the intervention. Right: the Results text says between-group differences in blood pressure were not observed at 6 months, and Table 2 prints diastolic between-group differences whose 95% CIs include zero at both time points (orange: values printed in the paper). Below (blue): the 2-sided P values recomputed from Table 2's own mean (SD) changes and group sizes (.146 and .185), and those implied by the printed CIs (.151 and .160). The paper sets significance at 2-sided P < .05.
Figure 2. Between-group difference in change in diastolic blood pressure (intervention minus control, mm Hg) with 95% CIs, for every analysis the paper reports: multiple imputation (Table 2) and complete case (Supplement 1, eTable 1). Orange circles and lines are the printed values. Blue squares and lines are recomputed from the printed mean (SD) changes with a pooled-variance t interval; the complete-case rows assume a 66/66 and 66/65 group split, which eTable 1 does not print. The vertical line marks no difference. Every interval, printed or recomputed, crosses it, and the P values at right are all above .05.
Why it matters
Blood pressure is a prespecified secondary outcome (Methods, "Outcomes"). The abstract's Results paragraph is where a reader learns which secondary outcomes differed between arms, and as written it tells that reader that the app improved diastolic blood pressure relative to usual care. The paper's analyses instead show point estimates of about −2.4 to −2.9 mm Hg with intervals running from about −6 to about +1 mm Hg. That result is compatible with no effect, and the paper's own Results text describes it as no difference. Abstracts are what is indexed, screened for systematic reviews and quoted in secondary literature, so a secondary outcome reported there as a benefit can be carried forward as one. The paper has been cited about 88 times. Meta-analysts who extract from Table 2 would get the correct estimate; readers who rely on the abstract would not.
Two further features of the data argue for caution even about the point estimate. Blood pressure was analysed in 139 of 204 randomized participants (68.1%). Baseline diastolic pressure was 3.0 mm Hg higher in the control group (P = .023), an imbalance the paper reports but does not adjust for in any presented analysis.
The error is limited in scope. The title, Key Points and abstract Conclusions mention only weight, glycemic and medication outcomes. The primary outcome (weight change, −2.4 kg at 6 months, P < .001), HbA1c (−0.4%, P = .01) and fasting glucose stand on Table 2. The main-text Conclusions refer more broadly to "significant improvements in weight and multiple metabolic profiles"; of the metabolic outcomes in Table 2, only HbA1c and fasting glucose have between-group P < .05 at 6 months.
Explanations considered
-
A different time point. Table 2 and eTable 1 give diastolic results at 3 and 6 months only, with P from .08 to .17. No time point reaches P < .05. This does not account for the claim.
-
Within-group rather than between-group change. Footnote b marks the intervention arm's diastolic decrease as significant at both times, so "favoring the intervention" could describe a within-arm improvement. But the abstract says "Intergroup differences", and the control arm's 6-month decrease is also marked significant, so a within-group reading does not distinguish the arms. This does not account for the wording.
-
"Favoring" meaning only the direction of the point estimate. The other outcomes in the sentence (fasting glucose, dietary changes) are significant in Table 2. If direction alone were the criterion, systolic blood pressure (−4.2 and −2.8 mm Hg) and total and LDL cholesterol at 6 months would also qualify, and they are not listed. The paper's own Results sentence also treats the blood pressure result as no difference. This does not account for the claim.
-
A one-sided test. The Methods state 2-sided tests. One-sided P values implied by the printed CIs are .076 and .080 for the main analysis. Only the 6-month complete-case result would fall below .05 one-sided (.042), and the paper presents the complete-case analysis as a supplementary check. This does not account for the claim.
-
The paper's Benjamini-Hochberg procedure. BH adjustment can only raise a P value, so it cannot turn P = .15 or .17 into P < .05. For completeness: if all 21 between-group P values in Table 2 at a time point are treated as one family and q ≤ 0.20 is used as the criterion (the paper applies FDR 0.20 to within-group changes only), diastolic BP has q = .175 at 3 months and would pass, and q = .255 at 6 months and would not. On the same rule, systolic BP (q = .092), total cholesterol (q = .056) and LDL cholesterol (q = .173) would also pass at 3 months, and none of them appears in the abstract. The paper reports no between-group FDR analysis and sets significance at P < .05. This does not plausibly account for the claim.
-
A systolic/diastolic slip. The only blood pressure comparison in the paper with P < .05 is complete-case systolic BP at 3 months (P = .04, eTable 1). If the abstract meant systolic, it would still rest on a supplementary analysis at the earlier time point, with P = .07 in the main analysis. This would explain the word but not support the claim.
-
A baseline-adjusted analysis. Because baseline diastolic pressure was higher in the control group, an ANCOVA could give a larger estimate than the change-score analysis. As an illustration only, using the full-sample baseline gap from Table 1 (the 139-person subgroup's gap is not reported) and an assumed baseline coefficient between 0.3 and 1.0, the 6-month adjusted estimate would lie between −4.5 and −2.4 mm Hg. Its standard error cannot be derived from the published summaries. No such analysis is reported in the paper or supplement, so it cannot be the basis of the abstract statement.
-
Conversion or transcription artifact. The abstract sentence, the Results sentence and the Table 2 rows read identically in the publisher's page, the live PMC page and the PMC archive. The printed P values agree with their own CIs and with recomputation. No artifact is indicated.
Status
- Version of record. The publisher's article page (reached through doi.org, jamanetwork.com article 2780570) carries the same abstract sentence (the fetched text was cut off after "diastolic blood pressure"), the same Results sentence and the same Table 2 diastolic rows. The live PMC page and the archived PMC text match. All quotes from the article above were checked verbatim against the PMC-archived text; the eTable 1 rows were read from the supplement PDF.
- Supplements. Supplement 1 (eAppendix, eFigures 1–2, eTables 1–3), Supplement 2 (trial protocol) and Supplement 3 (data-sharing statement) were obtained from the Europe PMC supplementary-files archive. eTable 1, the complete-case analysis, does not resolve the discrepancy (diastolic P = .13 and .08). No individual-level data are posted, so the mixed model could not be rerun.
- Corrections. As of 2026-10-23 no correction, erratum, retraction or expression of concern was found: Crossref lists no update relations for the DOI, the Crossref/Retraction Watch data have no entry, Europe PMC (PMID 34081137) lists no comment or correction links, PubMed lists no linked notices, and the PMC page shows none.
- PubPeer. PubPeer's search page and API returned an access challenge (HTTP 403) to automated queries, and a web search could not be run in this session, so PubPeer comments could not be checked.
- Related observations (not the subject of this note). (i) The abstract gives P = .001 for the 6-month HbA1c difference in the subgroup with baseline HbA1c ≥ 8%; the Results text and Table 3 give P = .02 for the same comparison (−0.9%, 95% CI −1.6 to −0.1), whose CI implies P = .019 (Welch from the Table 3 summaries: .039). (ii) In Table 2 the 6-month fasting glucose CI (−23.4 to −3.6) is identical to the 3-month CI; its midpoint is −13.5 rather than the printed −12.6, and recomputation gives −21.4 to −3.8 (P = .005). Neither changes a conclusion.
Suggested correction
In the Abstract, Results, replace
Intergroup differences favoring the intervention were also noted for fasting blood glucose, diastolic blood pressure, and dietary changes.
with, for example,
Intergroup differences favoring the intervention were also noted for fasting blood glucose and dietary changes; between-group differences in blood pressure were not statistically significant (diastolic blood pressure at 6 months, −2.4 mm Hg [95% CI, −5.7 to 1.0]; P = .17).
If the authors intended a baseline-adjusted or other analysis, that analysis and its result should be reported in the Results and the supplement before it is summarized in the abstract. The editors may also wish to check the abstract's P = .001 for the HbA1c ≥ 8% subgroup against the P = .02 in the Results and Table 3, and the 6-month fasting glucose CI in Table 2.
Recomputation script (python recompute.py.txt)
"""Recompute the diastolic blood pressure (DBP) between-group results of PMC8176331
(JAMA Netw Open 2021, doi:10.1001/jamanetworkopen.2021.12417).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC and from Supplement 1:
Table 1 baseline blood pressure, control n = 105, intervention n = 99
Table 2 multiple-imputation (MI) results; blood pressure block: control 72, intervention 67
eTable 1 (Supplement 1) complete-case (CC) results; total n = 132 (3 mo) and 131 (6 mo),
split between groups not printed
The paper fitted generalized linear mixed models on 5 imputed data sets. Those models
cannot be refitted from published summaries, so two approximations are used:
(a) a two-sample t test (pooled and Welch) from the printed mean (SD) changes and group n;
(b) the 2-sided P value implied by the printed difference and its 95% CI
(normal approximation, SE = CI width / (2 x 1.96)).
Both only check the printed numbers against each other; neither is a reanalysis.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# ------------------------------------------------------------------ inputs (from the paper)
# label: (control mean, SD, n, intervention mean, SD, n, reported diff, reported CI, reported P, reported d, location)
ROWS = {
"DBP 3 mo (MI)": (-1.8, 11.0, 72, -4.4, 9.9, 67, -2.6, (-6.2, 0.9), ".15", 0.25, "Table 2"),
"DBP 6 mo (MI)": (-3.1, 9.2, 72, -5.4, 11.1, 67, -2.4, (-5.7, 1.0), ".17", 0.23, "Table 2"),
"SBP 3 mo (MI)": (-2.2, 14.7, 72, -6.5, 13.1, 67, -4.2, (-8.8, 0.3), ".07", 0.31, "Table 2"),
"SBP 6 mo (MI)": (-5.0, 13.8, 72, -7.8, 15.2, 67, -2.8, (-7.7, 2.0), ".25", 0.19, "Table 2"),
# complete case: group split not printed; 66/66 and 66/65 assumed (sensitivity below)
"DBP 3 mo (CC)": (-1.6, 10.6, 66, -4.1, 8.8, 66, -2.6, (-5.9, 0.8), ".13", 0.26, "eTable 1"),
"DBP 6 mo (CC)": (-2.5, 7.9, 66, -5.4, 10.9, 65, -2.9, (-6.2, 0.4), ".08", 0.30, "eTable 1"),
"SBP 3 mo (CC)": (-1.4, 13.8, 66, -6.1, 12.6, 66, -4.7, (-9.3, -0.2), ".04", 0.36, "eTable 1"),
"SBP 6 mo (CC)": (-4.4, 13.2, 66, -7.9, 14.2, 65, -3.5, (-8.2, 1.3), ".14", 0.26, "eTable 1"),
# comparators named in the same abstract sentence (Table 2, n = 105 / 99)
"FBG 3 mo (MI)": (-3.6, 32.4, 105, -18.0, 37.8, 99, -14.4, (-23.4, -3.6), ".005", 0.41, "Table 2"),
"FBG 6 mo (MI)": (-1.8, 25.2, 105, -14.4, 37.8, 99, -12.6, (-23.4, -3.6), ".01", 0.39, "Table 2"),
"Calories 6 mo (MI)": (-245.8, 466.7, 105, -551.3, 515.4, 99, -305.8, (-441.0, -170.7), "<.001", 0.62, "Table 2"),
}
BASELINE_DBP = (85.7, 9.8, 105, 82.7, 8.8, 99) # Table 1, control vs intervention
BASELINE_SBP = (135.3, 13.0, 105, 134.7, 13.5, 99)
BP_N, TOTAL_N = 139, 204 # Table 2 blood pressure block vs randomized
def recompute(mc, sc, nc, mi, si, ni, rep_diff, rep_ci):
d = mi - mc
sp = np.sqrt(((nc - 1) * sc**2 + (ni - 1) * si**2) / (nc + ni - 2))
se_p = sp * np.sqrt(1 / nc + 1 / ni)
df = nc + ni - 2
q = stats.t.ppf(0.975, df)
p_pool = 2 * stats.t.sf(abs(d / se_p), df)
se_w = np.sqrt(sc**2 / nc + si**2 / ni)
dfw = se_w**4 / ((sc**2 / nc) ** 2 / (nc - 1) + (si**2 / ni) ** 2 / (ni - 1))
p_welch = 2 * stats.t.sf(abs(d / se_w), dfw)
se_ci = (rep_ci[1] - rep_ci[0]) / (2 * Z)
p_ci = 2 * stats.norm.sf(abs(rep_diff) / se_ci)
return dict(diff=d, lo=d - q * se_p, hi=d + q * se_p, p_pool=p_pool, p_welch=p_welch,
p_ci=p_ci, d=abs(d) / sp, p_one=p_pool / 2, mid_ci=(rep_ci[0] + rep_ci[1]) / 2)
R = {}
print("=== 1. Between-group differences (intervention minus control) ===")
print(" diff/CI from rounded mean (SD) changes, pooled-variance t; P also by Welch and from the printed CI\n")
for k, v in ROWS.items():
mc, sc, nc, mi, si, ni, rd, rci, rp, rdd, loc = v
r = recompute(mc, sc, nc, mi, si, ni, rd, rci)
R[k] = r
print(f"{k:20s} [{loc}] reported {rd:+.1f} ({rci[0]} to {rci[1]}), P={rp}, d={rdd}")
print(f"{'':20s} recomputed {r['diff']:+.2f} ({r['lo']:+.2f} to {r['hi']:+.2f}); "
f"P pooled={r['p_pool']:.3f}, Welch={r['p_welch']:.3f}, from printed CI={r['p_ci']:.3f}; "
f"d={r['d']:.2f}; one-sided P={r['p_one']:.3f}; midpoint of printed CI={r['mid_ci']:+.2f}")
print("\n=== 2. Complete-case rows: sensitivity to the unknown group split ===")
for k, tot in (("DBP 3 mo (CC)", 132), ("DBP 6 mo (CC)", 131)):
mc, sc, _, mi, si, _, rd, rci, *_ = ROWS[k]
ps = []
for nc in range(tot - 72, 73): # control at most 72, intervention at most 67
ni = tot - nc
if ni > 67 or ni < 1:
continue
ps.append(recompute(mc, sc, nc, mi, si, ni, rd, rci)["p_pool"])
print(f"{k}: over every split with control <= 72 and intervention <= 67, "
f"2-sided P ranges {min(ps):.3f} to {max(ps):.3f}")
print("\n=== 3. Does any DBP between-group test reach 2-sided P < .05? ===")
dbp = [k for k in ROWS if k.startswith("DBP")]
for k in dbp:
r = R[k]
print(f" {k:16s} printed CI includes 0: {ROWS[k][7][0] < 0 < ROWS[k][7][1]}; "
f"recomputed CI includes 0: {r['lo'] < 0 < r['hi']}; max P (any method) "
f"{max(r['p_pool'], r['p_welch'], r['p_ci']):.3f}, min P {min(r['p_pool'], r['p_welch'], r['p_ci']):.3f}")
print(f" any DBP row with recomputed 2-sided P < .05: {any(min(R[k]['p_pool'], R[k]['p_welch'], R[k]['p_ci']) < .05 for k in dbp)}")
print("\n=== 4. Baseline imbalance and coverage ===")
for lab, b in (("DBP", BASELINE_DBP), ("SBP", BASELINE_SBP)):
t, p = stats.ttest_ind_from_stats(*b)
print(f" baseline {lab}: control {b[0]} ({b[1]}) vs intervention {b[3]} ({b[4]}); "
f"difference {b[0] - b[3]:.1f} mm Hg; t={t:.2f}, P={p:.3f}")
print(f" participants in the blood pressure analysis: {BP_N}/{TOTAL_N} = {100 * BP_N / TOTAL_N:.1f}%")
print("\n=== 5. One-sided P implied by the printed CIs (the paper states 2-sided tests) ===")
for k in dbp:
print(f" {k:16s} one-sided P from printed CI = {R[k]['p_ci'] / 2:.3f}")
print("\n=== 6. Benjamini-Hochberg q over all 21 between-group P values printed in Table 2 ===")
# printed P values, Table 2, in row order: weight kg, weight %, BMI, HbA1c, FBG, SBP, DBP, cost,
# total chol, LDL, HDL, TG, creatinine, calories, carbohydrate, sugar, protein, total fat, sat fat, fiber, PA.
# "<.001" is entered as .001 (conservative: larger than the true value).
NAMES = ["weight kg", "weight %", "BMI", "HbA1c", "FBG", "SBP", "DBP", "cost", "total chol", "LDL",
"HDL", "TG", "creatinine", "calories", "carbohydrate", "sugar", "protein", "total fat",
"sat fat", "fiber", "physical activity"]
P_T2 = {
"3 mo": [.001, .001, .001, .006, .005, .07, .15, .04, .04, .14, .90, .48, .91, .001, .001, .009, .006, .001, .001, .02, .007],
"6 mo": [.001, .001, .001, .01, .01, .25, .17, .01, .65, .53, .76, .49, .63, .001, .001, .01, .09, .001, .001, .54, .009],
}
def bh(p):
p = np.asarray(p, float); m = len(p); o = np.argsort(p)
q = np.empty(m); run = 1.0
for rank in range(m, 0, -1):
i = o[rank - 1]; run = min(run, p[i] * m / rank); q[i] = run
return q
for tp, ps in P_T2.items():
q = bh(ps)
print(f" {tp}: q(DBP)={q[NAMES.index('DBP')]:.3f} q(SBP)={q[NAMES.index('SBP')]:.3f} "
f"q(LDL)={q[NAMES.index('LDL')]:.3f} q(total chol)={q[NAMES.index('total chol')]:.3f}")
print(f" outcomes with q <= 0.20: {[n for n, qq in zip(NAMES, q) if qq <= 0.20]}")
print("\n=== 7. Illustration only: what a baseline-adjusted (ANCOVA) estimate could be ===")
# change-score difference = (F_I - F_C) - (B_I - B_C); ANCOVA = (F_I - F_C) - b (B_I - B_C)
# => ANCOVA = change difference + (1 - b)(B_I - B_C). Baseline gap from Table 1 (all 204, not the 139).
gap = BASELINE_DBP[3] - BASELINE_DBP[0]
for b in (0.3, 0.5, 0.7, 1.0):
print(f" 6 mo, assumed b={b:.1f}: {-2.4 + (1 - b) * gap:+.1f} mm Hg (standard error not derivable)")
print("\n=== 8. Side observations (not the subject of the note) ===")
r = R["FBG 6 mo (MI)"]
print(f" FBG 6 mo: printed CI (-23.4 to -3.6) is identical to the 3-mo row; its midpoint is {r['mid_ci']:+.1f}, "
f"not the printed -12.6; recomputed {r['diff']:+.1f} ({r['lo']:.1f} to {r['hi']:.1f}), P={r['p_pool']:.3f}")
# HbA1c >= 8% subgroup, 6 mo, Table 3: control -1.0 (1.4) n=29, intervention -1.8 (1.4) n=26; -0.9 (-1.6 to -0.1), P=.02
hb = recompute(-1.0, 1.4, 29, -1.8, 1.4, 26, -0.9, (-1.6, -0.1))
print(f" HbA1c>=8% subgroup 6 mo: abstract P=.001; Results text and Table 3 P=.02; "
f"P from Table 3 CI={hb['p_ci']:.3f}; Welch from Table 3 summaries={hb['p_welch']:.3f}")
# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "svg.fonttype": "none"})
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# ---- Figure 1: abstract vs Results / Table 2
fig = plt.figure(figsize=(7.2, 4.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.6); ax.axis("off")
def panel(x, y, w, h, title, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.14, y + h - 0.2, title, fontsize=11, weight="bold", va="top", color=INK)
panel(0.15, 1.05, 2.85, 3.25, "Abstract, Results", INK2)
ax.text(0.29, 3.68,
"“Intergroup differences favoring\nthe intervention were also noted\n"
"for fasting blood glucose,\ndiastolic blood pressure, and\ndietary changes.”",
fontsize=10.5, va="top", color=INK, linespacing=1.45)
ax.text(0.29, 1.62, "Claim: a between-group\ndifference in diastolic BP", fontsize=10,
va="top", color=INK2, style="italic", linespacing=1.35)
panel(3.75, 1.05, 3.3, 3.25, "Results text and Table 2", INK2)
ax.text(3.89, 3.68, "“Between-group differences in blood\npressure and lipids were not\nobserved at 6 months.”",
fontsize=10.5, va="top", color=INK, linespacing=1.45)
ax.text(3.89, 2.55, "Table 2, change in diastolic BP\n(intervention − control, mm Hg)",
fontsize=10, va="top", color=INK2, linespacing=1.35)
for i, (k, lab) in enumerate((("DBP 3 mo (MI)", "3 mo"), ("DBP 6 mo (MI)", "6 mo"))):
v = ROWS[k]
yy = 1.88 - i * 0.28
ax.text(3.89, yy, f"{lab}: −{abs(v[6]):.1f} ({v[7][0]:.1f} to {v[7][1]:.1f}), P = {v[8]}".replace("-", "−"),
fontsize=10.5, va="top", color=ORANGE, weight="bold")
ax.text(3.89, 1.27, "printed in the paper", fontsize=10, va="top", color=ORANGE)
ax.annotate("", xy=(3.72, 2.7), xytext=(3.03, 2.7),
arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
ax.text(3.375, 2.82, "conflict", ha="center", fontsize=10, color=INK, weight="bold")
r3, r6 = R["DBP 3 mo (MI)"], R["DBP 6 mo (MI)"]
ax.text(0.15, 0.78,
f"Recomputed from Table 2's own mean (SD) changes: P = {r3['p_pool']:.3f} (3 mo), {r6['p_pool']:.3f} (6 mo);",
fontsize=10, color=BLUE, va="top")
ax.text(0.15, 0.50,
f"P implied by the printed 95% CIs: {r3['p_ci']:.3f} and {r6['p_ci']:.3f}. Every CI, printed or recomputed, includes 0.",
fontsize=10, color=BLUE, va="top")
ax.text(0.15, 0.22, "Orange: printed in the paper. Blue: recomputed from the paper's numbers.",
fontsize=10, color=INK2, va="top")
save(fig, "fig1")
# ---- Figure 2: dot-and-interval, DBP between-group differences, reported vs recomputed
order = ["DBP 3 mo (MI)", "DBP 6 mo (MI)", "DBP 3 mo (CC)", "DBP 6 mo (CC)"]
names = ["3 mo, multiple imputation\n(Table 2)", "6 mo, multiple imputation\n(Table 2)",
"3 mo, complete case\n(eTable 1)", "6 mo, complete case\n(eTable 1)"]
fig, ax = plt.subplots(figsize=(7.2, 4.3), facecolor=BG)
ax.set_facecolor(BG)
off = 0.13
for i, k in enumerate(order):
y = len(order) - 1 - i
v, r = ROWS[k], R[k]
ax.plot(v[7], [y + off] * 2, color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot(v[6], y + off, "o", ms=8, color=ORANGE, mec=BG, mew=1.5,
label="Reported (paper)" if i == 0 else None)
ax.plot([r["lo"], r["hi"]], [y - off] * 2, color=BLUE, lw=2, solid_capstyle="butt")
ax.plot(r["diff"], y - off, "s", ms=8, color=BLUE, mec=BG, mew=1.5,
label="Recomputed from the paper's mean (SD) changes" if i == 0 else None)
ax.text(3.0, y + off, f"P = {v[8]}", va="center", fontsize=10, color=INK)
ax.text(3.0, y - off, f"P = {r['p_pool']:.3f}".replace("0.", ".", 1), va="center", fontsize=10, color=INK)
ax.axvline(0, color=INK2, lw=0.8)
ax.text(0.08, 3.55, "no difference", fontsize=10, color=INK2, va="center")
ax.set_yticks(range(len(order)))
ax.set_yticklabels(names[::-1], fontsize=10, color=INK)
ax.set_xlim(-7, 4.2); ax.set_ylim(-0.6, 3.75)
ax.set_xticks(range(-7, 3))
ax.set_xlabel("Diastolic BP change, intervention − control (mm Hg), 95% CI")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
fig.tight_layout(rect=(0, 0, 1, 0.9))
fig.legend(loc="upper center", ncol=2, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.99))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Hospital-stay estimate of 8.4 days lies outside its own reported 95% CI of 4.4–6.7
Kidney Int (2021) · doi:10.1016/j.kint.2021.02.026 · PMC7927648
Summary
This retrospective cohort of 152 children hospitalized with acute COVID-19 or MIS-C reports, in the Abstract, the Results and Table 4, that acute kidney injury (AKI, stages 1–3) was associated with an 8.4-day longer hospital stay, with a 95% CI of 4.4 to 6.7 and P < 0.0001, from unadjusted simple linear regression. The point estimate lies 1.7 days above the upper bound of its own interval. A regression CI is centred on its coefficient, and the stated interval is centred on 5.55. Every other row of Table 4 except one is centred on its estimate. At least one of the three printed numbers is therefore wrong, and the paper does not contain enough information to say which. The direction and statistical significance of the association survive every reconstruction considered, but the precision is overstated or the size is misstated. A second Table 4 row, for length of mechanical ventilation, is also not centred on its estimate.
What the paper reports
Abstract:
In unadjusted models, patients with acute kidney injury had 8.4 days longer hospitalizations compared to patients without acute kidney injury (95% confidence interval, 4.4-6.7).
Results, "Clinical course and outcomes", third paragraph:
In addition, patients with AKI had a significantly greater length of hospital stay of 8.4 days compared with patients without AKI (95% CI, 4.4–6.7; P < 0.0001).
Table 4, "Association of AKI with hospital outcomes", under the column header "β (95% CI)" (columns: outcome by AKI stage | β (95% CI) | P value):
Length of hospital stay, d AKI stage 1 + 2 + 3 | 8.4 (4.4 to 6.7) | <0.0001 AKI stage 2 + 3 | 5.2 (0.07 to 10.2) | 0.047 Length of PICU stay, d AKI stage 1 + 2 + 3 | 5 (0.86 to 9.1) | 0.02 AKI stage 2 + 3 | 2.3 (–3.7 to 8.3) | 0.45 Length of mechanical ventilation, d AKI stage 1 + 2 + 3 | –3.9 (–9.4 to 12.7) | 0.15 AKI stage 2 + 3 | –1.08 (–7.4 to 5.3) | 0.71
The same table gives mortality as a hazard ratio: "AKI stage 1 + 2 + 3 | 0.57 (0.03 to 11.8) | 0.72".
Methods, "Statistical analysis":
Multiple imputation was used for missing values in regression analysis. Subsequently, simple linear regression analysis was performed to assess the relationship between AKI and continuous outcomes, including PICU and hospital length of stay and length of time on mechanical ventilation.
The conclusions that draw on this result. Discussion, first paragraph:
AKI, in unadjusted models, was significantly associated with longer PICU and hospital length of stay.
Discussion, "Conclusions":
Pediatric COVID-19–related AKI, similar to reports in larger AKI studies, was associated with poor outcomes, such as increased PICU and hospital length of stay.
What the paper's own numbers show
Method. For simple linear regression the 95% CI of a coefficient is β ± t(0.975, df) × SE. It always contains β and is centred on it. A CI pooled across multiply imputed data sets with Rubin's rules has the same form, so the same holds. For each Table 4 row I checked whether the estimate lies inside its interval and computed the centre of the interval, the SE implied by its width (df = 150, from N = 152), and the P value implied by β/SE. For the hazard ratio I did the same on the log scale with a normal critical value. I then rebuilt the hospital-stay row under each assumption about which number is correct. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Hospital stay, AKI 1–3: is β inside its CI? | Abstract; Results; Table 4 | 8.4 (4.4 to 6.7) | No. 8.4 is 1.7 d above the upper bound 6.7 |
| Hospital stay, AKI 1–3: centre of the CI | Table 4 | 8.4 | 5.55 (half-width 1.15; SE from width 0.582) |
| Same row, if 8.4 and the lower bound 4.4 are right | derived | upper bound 6.7; P < 0.0001 | upper bound 12.4; SE 2.024; t = 4.15; P = 5.6 × 10⁻⁵ (consistent with P < 0.0001) |
| Same row, if 8.4 and the upper bound 6.7 are right | derived | — | lower bound 2 × 8.4 − 6.7 = 10.1, above the estimate: impossible |
| Same row, if the CI 4.4–6.7 is right | derived | β = 8.4 | β = 5.55; t = 9.54; P = 3.8 × 10⁻¹⁷ (also < 0.0001) |
| SE under the 4.4–12.4 reconstruction vs SE implied by the printed width | derived | — | 2.024 / 0.582 = 3.48 times larger |
| Hospital stay, AKI 2–3 | Table 4 | 5.2 (0.07 to 10.2), P = 0.047 | centre 5.135; P from β/SE 0.044: consistent |
| PICU stay, AKI 1–3 | Table 4 | 5 (0.86 to 9.1), P = 0.02 | centre 4.98; P 0.018: consistent |
| PICU stay, AKI 2–3 | Table 4 | 2.3 (–3.7 to 8.3), P = 0.45 | centre 2.30; P 0.45: consistent |
| Mechanical ventilation, AKI 1–3 | Table 4 | –3.9 (–9.4 to 12.7), P = 0.15 | inside, but centre +1.65; arms 5.5 below and 16.6 above; P from β/SE 0.49 (0.45 if only the 11 ventilated patients in Table 3 were analysed, df = 9): inconsistent |
| Mechanical ventilation, AKI 2–3 | Table 4 | –1.08 (–7.4 to 5.3), P = 0.71 | centre –1.05; P 0.74: consistent |
| Mortality hazard ratio, AKI 1–3 | Table 4 | 0.57 (0.03 to 11.8), P = 0.72 | geometric centre 0.595; P 0.71: consistent |
| Median hospital stay, without vs with AKI | Table 3 | acute COVID-19 2.8 vs 22.2 d; MIS-C 4.6 vs 8.2 d | differences 19.4 and 3.6 d (medians by subgroup, so they cannot reproduce the pooled β) |
Five of the seven Table 4 rows are centred on their estimates to within rounding, and their printed P values agree with the P implied by their intervals. The hospital-stay row for AKI stages 1–3 is the only one whose estimate falls outside its interval. The mechanical-ventilation row for AKI stages 1–3 is the only other row that is not centred on its estimate, and its printed P = 0.15 does not match the P implied by its interval.
A few smaller inconsistencies turned up during the comparison. None of them affects a conclusion:
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| PICU stay, AKI 1–3 | Results text vs Table 4 | text "95% CI, 0.86–9; P = 0.025"; table "0.86 to 9.1", 0.02 | same estimate, different rounding of the bound and P |
| Acute COVID-19 group size | Table 3 vs Table 1 and Abstract | Table 3 "No AKI (N = 88)", "AKI (N = 8)" | 88 + 8 = 96; Table 1 gives 97 overall and No AKI N = 89 |
| Mechanical ventilation, acute COVID-19 with AKI | Table 3 | 2 (33.3) of N = 8 | 2/8 = 25.0% |
| Vasopressors, acute COVID-19 with AKI | Table 3 | 5 (32) of N = 8 | 5/8 = 62.5% |
Figure
Figure 1. The six day-scale rows of Table 4 on one axis, with no effect (0 days) as the thin vertical line. Orange dots and bars are the reported β and 95% CI. Blue rings mark the centre of each reported interval, computed from its printed bounds; where the orange dot sits inside the blue ring the estimate is centred, as it must be for a linear-regression CI. For hospital stay with AKI stages 1–3 the estimate 8.4 lies outside the interval, 1.7 days above its upper bound, and the interval's centre is 5.55. The dashed blue row below it is one possible reconstruction, not a correction: the interval centred on 8.4 that keeps the printed lower bound 4.4 (4.4 to 12.4, P = 6 × 10⁻⁵). The mechanical-ventilation row for AKI stages 1–3 contains its estimate, but its centre (+1.65) is 5.55 days from the estimate.
Why it matters
The result appears in the Abstract as one of the study's main findings, in the Results text and in Table 4, and it supports the Discussion and Conclusions statements that AKI was associated with longer hospital stays. As printed, the interval is impossible. A reader cannot tell whether the effect is about 8.4 days with a wide interval (for example 4.4 to 12.4) or about 5.6 days with the narrow interval printed. The two readings differ in effect size by almost 3 days and in standard error by a factor of about 3.5.
What does not change: every reconstruction leaves the lower bound well above zero and gives P < 0.0001, so the qualitative statement that AKI was associated with a significantly longer unadjusted hospital stay is not overturned. Table 3 medians (2.8 vs 22.2 days in acute COVID-19, 4.6 vs 8.2 days in MIS-C) and the PICU-stay result (5 days, 0.86 to 9.1) point the same way. Length of stay is a secondary outcome. The paper states that "The primary outcome of this study was the incidence rate of AKI" (11.8% overall), and that result, the albumin and white-cell odds ratios, the MIS-C systolic-dysfunction comparison and the mortality result do not depend on this row.
The mechanical-ventilation row for AKI stages 1–3 is a second, smaller problem: the printed estimate, interval and P value cannot all come from one symmetric interval. It is reported as non-significant, and every value implied by its interval is non-significant too, so the paper's statement that there was no significant association with length of mechanical ventilation is not affected.
Explanations considered
- The interval is on a transformed scale (log days) and was back-transformed. This does not account for the numbers. The Methods describe simple linear regression of length of stay, Table 4 heads the column "β (95% CI)" under "Length of hospital stay, d", and no transformation is mentioned. A back-transformed interval would also still contain its back-transformed estimate. The other day-scale rows are symmetric on the raw scale.
- A bootstrap or other asymmetric interval. No such method is described, and even a percentile bootstrap interval would almost never exclude the point estimate. The other rows are symmetric, which points to Wald-type intervals.
- Multiple imputation. The Methods state that multiple imputation was used for missing values. Rubin's rules give a pooled estimate and a t-based interval centred on it, so pooling cannot place the estimate outside the interval.
- The estimate and interval were taken from different rows or models. No other row in Table 4 has the interval 4.4 to 6.7 or the estimate 8.4. The same pair appears together in the Abstract, the Results text and Table 4, so they were not misaligned in one place only.
- A conversion or typesetting artifact. This is excluded. The PMC page and the Europe PMC full-text XML, which comes from the publisher's deposit, give the same plain decimals in all three places, with no minus signs, superscripts or mid-dots that could have been lost. The table cell holds the estimate and the interval together.
- A mistyped upper bound. This is plausible but cannot be proved. If 8.4 and 4.4 are correct, the upper bound would be about 12.4, the implied SE (about 2.0) is similar to those of the PICU-stay rows (2.1 and 3.0), and the implied P (5.6 × 10⁻⁵) fits the printed P < 0.0001. The alternative reading (estimate 5.55, interval as printed) is also compatible with P < 0.0001. The published material does not distinguish them.
- For the mechanical-ventilation row, a small analysed sample. Table 3 lists only 11 ventilated patients, so the regression may have used far fewer than 152 observations. That changes the critical value but not the centre of a symmetric interval: with df = 9 the interval still centres on +1.65 and implies P = 0.45, not 0.15.
Status
- Version of record. The PMC article page and the Europe PMC full-text XML (publisher-supplied JATS) give the same figures in the Abstract, the Results and Table 4. The publisher's own pages could not be opened: doi.org redirects to linkinghub.elsevier.com, which served only a redirect stub, and kidney-international.org returned HTTP 403. Every quote above was confirmed verbatim against the PMC-archived text.
- Supplement. The only supplement, mmc1.pdf (4 pages, retrieved through the Europe PMC supplementary-files endpoint; its md5 matches the one in the PMC XML), contains Tables S1–S4: outcomes by AKI stage as medians, echocardiography in MIS-C, admission characteristics by unit, and logistic regression of baseline factors. It has no linear-regression output for length of stay and no individual-level data, so it does not resolve the error.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, the Crossref record that includes Retraction Watch data shows no updates, and PMC and Europe PMC list no correction notices. The only linked item is a commentary in Kidney Int 2021;100(1):16–19 (PMID 34154708, PMC8221810), which does not mention the hospital-stay estimate or its interval. A web search found no erratum and no PubPeer thread; PubPeer itself could not be queried directly (HTTP 403).
- Sources. PMC article PMC7927648; Europe PMC full text and supplementary files; Crossref REST API; PubMed; DOI resolver and publisher site (not reachable).
Suggested correction
- Re-run or re-extract the unadjusted linear regression of hospital length of stay on AKI (stages 1–3) and correct the estimate, the 95% CI or both in the Abstract, the Results text ("Clinical course and outcomes") and Table 4, together with the P value if it changes.
- Check the mechanical-ventilation row for AKI stages 1–3 in Table 4, whose estimate (–3.9), interval (–9.4 to 12.7) and P value (0.15) are mutually inconsistent, and state how many patients each length-of-ventilation model included.
- Minor: reconcile the PICU-stay interval and P value between the Results text and Table 4; correct the acute COVID-19 No AKI column size in Table 3 (88 vs 89); and check the percentages for mechanical ventilation (2 of 8) and vasopressors (5 of 8) in the acute COVID-19 AKI column of Table 3.
Recomputation script (python recompute.py.txt)
"""Recompute the Table 4 regression intervals of PMC7927648
(Kidney Int 2021, doi:10.1016/j.kint.2021.02.026).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md,
writes fig1.svg/png next to this file, and writes five candidate figure designs to
candidates/ (c1..c5 .svg/.png plus contact_sheet.png). fig1 is a copy of the candidate
named in CHOSEN below.
Inputs are copied from the paper as archived in PMC:
Table 4 (beta or hazard ratio, 95% CI, P), also quoted in the Abstract and Results
Methods: unadjusted simple linear regression (SPSS 26), multiple imputation; cohort N = 152
Table 3: medians (IQR) by AKI status, and counts used in the minor checks
"""
import math
import os
import shutil
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from scipy import stats # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
CHOSEN = "c1"
# ---------------------------------------------------------------- inputs (from the paper)
N = 152 # Abstract / Results: "The cohort included 152 patients"
DF = N - 2 # simple linear regression, one predictor
TCRIT = stats.t.ppf(0.975, DF)
# label: (estimate, lower, upper, printed P, scale) -- Table 4
ROWS = [
("Hospital stay, AKI 1-3", 8.4, 4.4, 6.7, "<0.0001", "lin"),
("Hospital stay, AKI 2-3", 5.2, 0.07, 10.2, "0.047", "lin"),
("PICU stay, AKI 1-3", 5.0, 0.86, 9.1, "0.02", "lin"),
("PICU stay, AKI 2-3", 2.3, -3.7, 8.3, "0.45", "lin"),
("Mechanical ventilation, AKI 1-3", -3.9, -9.4, 12.7, "0.15", "lin"),
("Mechanical ventilation, AKI 2-3", -1.08, -7.4, 5.3, "0.71", "lin"),
("Mortality (hazard ratio), AKI 1-3", 0.57, 0.03, 11.8, "0.72", "log"),
]
N_VENTILATED = 4 + 2 + 1 + 4 # Table 3, "Mechanical ventilation, n": 4, 2, 1, 4
R = {}
print("=== 1. Is each Table 4 estimate centred in its own 95% CI? ===")
print(f"t critical value, df = {DF}: {TCRIT:.4f}")
for lab, b, lo, hi, p, sc in ROWS:
if sc == "log":
mid = math.exp((math.log(lo) + math.log(hi)) / 2)
half = (math.log(hi) - math.log(lo)) / 2
se = half / 1.959964
pos = (math.log(b) - math.log(mid)) / half
pp = 2 * stats.norm.sf(abs(math.log(b) / se))
else:
mid = (lo + hi) / 2
half = (hi - lo) / 2
se = half / TCRIT
pos = (b - mid) / half
pp = 2 * stats.t.sf(abs(b / se), DF)
inside = lo <= b <= hi
R[lab] = dict(b=b, lo=lo, hi=hi, p=p, sc=sc, mid=mid, half=half, se=se, pos=pos, pp=pp, inside=inside)
print(f"{lab:36s} est {b:6.2f} CI {lo:6.2f} to {hi:6.2f} inside={inside!s:5s} "
f"CI centre {mid:6.3f} est - centre {b - mid if sc == 'lin' else b / mid:+.3f}"
f"{'' if sc == 'lin' else ' (ratio)'} position in CI {pos:+.2f} "
f"SE from CI {se:.3f} P from est/SE {pp:.2g} (printed {p})")
print("\n=== 2. The hospital-stay row, AKI stages 1-3 ===")
b, lo, hi = 8.4, 4.4, 6.7
print(f"8.4 within [4.4, 6.7]: {lo <= b <= hi}; 8.4 - 6.7 = {b - hi:.1f} days above the upper bound")
print(f"centre of printed CI = {(lo + hi) / 2:.2f}; half-width = {(hi - lo) / 2:.2f}")
se_ci = (hi - lo) / 2 / TCRIT
print(f"SE implied by the printed CI width = {se_ci:.3f}")
up_rec = 2 * b - lo
se_rec = (b - lo) / TCRIT
p_rec = 2 * stats.t.sf(b / se_rec, DF)
print(f"(a) if 8.4 and the lower bound 4.4 are right: upper bound = 2*8.4 - 4.4 = {up_rec:.1f}; "
f"SE = {se_rec:.3f}; t = {b / se_rec:.2f}; P = {p_rec:.1e} (printed P < 0.0001: {p_rec < 1e-4})")
print(f" SE under (a) / SE implied by printed width = {se_rec / se_ci:.2f}")
lo_b = 2 * b - hi
print(f"(b) if 8.4 and the upper bound 6.7 are right: lower bound = 2*8.4 - 6.7 = {lo_b:.1f} > 8.4, impossible")
mid = (lo + hi) / 2
p_c = 2 * stats.t.sf(mid / se_ci, DF)
print(f"(c) if the CI 4.4-6.7 is right: estimate = {mid:.2f}; t = {mid / se_ci:.2f}; P = {p_c:.1e} "
f"(also < 0.0001: {p_c < 1e-4})")
print("Every reconstruction keeps the lower bound well above 0, so the direction and significance survive.")
print("\n=== 3. Side finding: mechanical-ventilation row, AKI stages 1-3 ===")
b, lo, hi = -3.9, -9.4, 12.7
for df, lab in ((DF, f"df = {DF} (whole cohort)"), (N_VENTILATED - 2, f"df = {N_VENTILATED - 2} (only the {N_VENTILATED} ventilated patients, Table 3)")):
tc = stats.t.ppf(0.975, df)
se = (hi - lo) / 2 / tc
print(f" {lab}: CI centre {(lo + hi) / 2:+.2f}; SE {se:.3f}; P from -3.9/SE {2 * stats.t.sf(abs(b) / se, df):.2f} (printed 0.15)")
print(f" estimate - lower = {b - lo:.1f}; upper - estimate = {hi - b:.1f}")
print("\n=== 4. Plausibility of an ~8-day difference (Table 3 medians, not a regression) ===")
print(f"acute COVID-19: 22.2 - 2.8 = {22.2 - 2.8:.1f} d; MIS-C: 8.2 - 4.6 = {8.2 - 4.6:.1f} d")
print("\n=== 5. Smaller inconsistencies noted in passing ===")
print("PICU stay, AKI 1-3: Results text '95% CI, 0.86-9; P = 0.025'; Table 4 '0.86 to 9.1', P 0.02")
print(f"Table 3 acute COVID-19 columns: 88 + 8 = {88 + 8}; Table 1 and Abstract: 97 (No AKI N = 89 in Table 1)")
print(f"Table 3 mechanical ventilation, acute COVID-19 with AKI: 2/8 = {200 / 8:.1f}% (printed 33.3)")
print(f"Table 3 vasopressors, acute COVID-19 with AKI: 5/8 = {500 / 8:.1f}% (printed 32)")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
LW = 1.5 # 2 px at 96 px/in is 1.5 pt
MS = 8.5
def save(fig, stem, folder=CAND):
fig.savefig(os.path.join(folder, stem + ".svg"))
fig.savefig(os.path.join(folder, stem + ".png"), dpi=200)
plt.close(fig)
def tidy(ax, xgrid=True, ygrid=False):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
if xgrid:
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
if ygrid:
ax.yaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
LIN = [r for r in ROWS if r[5] == "lin"]
FLAG = "Hospital stay, AKI 1-3"
# --- c1: all day-scale Table 4 rows, reported intervals with CI centres, plus the reconstruction
fig, ax = plt.subplots(figsize=(7.2, 4.9))
labels, ys = [], []
y = 0
for lab, b, lo, hi, p, sc in LIN:
r = R[lab]
ax.plot([lo, hi], [y, y], color=ORANGE, lw=LW, solid_capstyle="butt", zorder=2)
for e in (lo, hi):
ax.plot([e, e], [y - 0.13, y + 0.13], color=ORANGE, lw=LW, zorder=2)
ax.plot([b], [y], "o", ms=MS, color=ORANGE, zorder=4)
ax.plot([r["mid"]], [y], "o", ms=MS + 5, mfc="none", mec=BLUE, mew=2, zorder=5)
labels.append(f"{lab}\n{b:g} ({lo:g} to {hi:g}), P {p if p.startswith('<') else '= ' + p}")
ys.append(y)
if lab == FLAG:
y += 0.9
ax.plot([4.4, up_rec], [y, y], color=BLUE, lw=LW, ls=(0, (4, 2)), zorder=2)
for e in (4.4, up_rec):
ax.plot([e, e], [y - 0.13, y + 0.13], color=BLUE, lw=LW, zorder=2)
ax.plot([8.4], [y], "D", ms=MS - 1, color=BLUE, zorder=4)
labels.append(f"Same row, rebuilt from 8.4 and 4.4\n8.4 (4.4 to {up_rec:.1f}), P = {p_rec:.0e}".replace("e-0", "e-"))
ys.append(y)
ax.annotate("8.4 lies 1.7 d\nabove its own\nupper bound", xy=(8.4, 0), xytext=(10.6, -0.05),
fontsize=10, color=TXT, va="center", ha="left",
arrowprops=dict(arrowstyle="-", color=SEC, lw=0.8))
y += 1.15
ax.axvline(0, color=SEC, lw=0.8, zorder=1)
ax.set_yticks(ys)
ax.set_yticklabels(labels, fontsize=10)
ax.invert_yaxis()
ax.set_xlim(-11, 16.5)
ax.set_xticks(range(-10, 16, 5))
ax.set_xlabel("β: difference with vs without AKI, days")
tidy(ax)
fig.legend(handles=[
Line2D([], [], marker="o", ls="-", lw=LW, ms=MS, color=ORANGE, label="Reported β and 95% CI (Table 4)"),
Line2D([], [], marker="o", ls="none", ms=MS + 5, mfc="none", mec=BLUE, mew=2,
label="Centre of the reported CI (recomputed); a ring around a dot means the estimate is centred"),
Line2D([], [], marker="D", ls=(0, (4, 2)), lw=LW, ms=MS - 1, color=BLUE,
label="Interval centred on 8.4 with lower bound 4.4 (recomputed)")],
loc="lower left", bbox_to_anchor=(0.01, 0.0), frameon=False, fontsize=10, ncol=1)
fig.tight_layout(rect=(0, 0.17, 1, 1))
save(fig, "c1")
# --- c2: the flagged row alone, three readings on one axis
fig, ax = plt.subplots(figsize=(7.2, 3.3))
items = [
("Reported (Abstract, Results, Table 4)\n8.4 (4.4 to 6.7), P < 0.0001", 8.4, 4.4, 6.7, ORANGE, "o", "-"),
(f"If the CI is right: estimate = its centre\n{(4.4 + 6.7) / 2:.2f} (4.4 to 6.7), P = {p_c:.0e}".replace("e-", "e−"),
(4.4 + 6.7) / 2, 4.4, 6.7, BLUE, "D", (0, (4, 2))),
(f"If 8.4 and 4.4 are right: symmetric CI\n8.4 (4.4 to {up_rec:.1f}), P = {p_rec:.0e}".replace("e-0", "e−"),
8.4, 4.4, up_rec, BLUE, "D", (0, (4, 2))),
]
for i, (lab, b, lo, hi, col, mk, ls) in enumerate(items):
ax.plot([lo, hi], [i, i], color=col, lw=LW, ls=ls, zorder=2)
for e in (lo, hi):
ax.plot([e, e], [i - 0.12, i + 0.12], color=col, lw=LW, zorder=2)
ax.plot([b], [i], mk, ms=MS, color=col, zorder=4)
ax.annotate("", xy=(6.7, -0.32), xytext=(8.4, -0.32),
arrowprops=dict(arrowstyle="<->", color=TXT, lw=1.0, shrinkA=0, shrinkB=0))
ax.text(7.55, -0.42, "1.7 d outside", ha="center", va="bottom", fontsize=10, color=TXT)
ax.axvline(0, color=SEC, lw=0.8, zorder=1)
ax.set_yticks(range(len(items)))
ax.set_yticklabels([it[0] for it in items], fontsize=10)
ax.set_ylim(len(items) - 0.5, -0.8)
ax.set_xlim(-0.5, 14)
ax.set_xticks(range(0, 15, 2))
ax.set_xlabel("Longer hospital stay with AKI (stages 1–3), days")
tidy(ax)
fig.legend(handles=[
Line2D([], [], marker="o", ls="-", lw=LW, ms=MS, color=ORANGE, label="Reported"),
Line2D([], [], marker="D", ls=(0, (4, 2)), lw=LW, ms=MS, color=BLUE, label="Recomputed from the paper's numbers")],
loc="lower left", bbox_to_anchor=(0.01, 0.0), frameon=False, fontsize=10, ncol=2)
fig.tight_layout(rect=(0, 0.09, 1, 1))
save(fig, "c2")
# --- c3: where each estimate sits inside its own CI (-1 = lower bound, +1 = upper bound)
fig, ax = plt.subplots(figsize=(7.2, 3.9))
order = [r[0] for r in ROWS]
ax.axvspan(-1, 1, color="#efeeea", zorder=0, lw=0)
ax.text(0, -0.85, "inside the CI", ha="center", va="center", fontsize=10, color=SEC)
for x in (-1, 1):
ax.axvline(x, color=SEC, lw=0.8, ls=(0, (2, 2)), zorder=1)
ax.axvline(0, color=SEC, lw=0.8, zorder=1)
for i, lab in enumerate(order):
r = R[lab]
ax.plot([0, r["pos"]], [i, i], color=BLUE, lw=LW, zorder=2)
ax.plot([r["pos"]], [i], "o", ms=MS, color=BLUE, zorder=3)
ax.text(r["pos"] + (0.1 if r["pos"] >= 0 else -0.1), i - 0.3, f"{r['pos'] + 0.0:+.2f}".replace("-0.00", "+0.00").replace("-", "−"),
ha="left" if r["pos"] >= 0 else "right", va="center", fontsize=10, color=TXT)
ax.set_yticks(range(len(order)))
ax.set_yticklabels([f"{lab}\n{R[lab]['b']:g} ({R[lab]['lo']:g} to {R[lab]['hi']:g})" for lab in order], fontsize=10)
ax.set_ylim(len(order) - 0.5, -1.2)
ax.set_xlim(-1.6, 3.0)
ax.set_xticks([-1, 0, 1, 2, 3])
ax.set_xticklabels(["−1\nlower bound", "0\ncentre", "+1\nupper bound", "+2", "+3"])
ax.set_xlabel("Where the reported estimate sits in its reported CI (recomputed;\nhazard ratio on the log scale)")
tidy(ax)
fig.tight_layout()
save(fig, "c3")
# --- c4: symmetry check, lower arm vs upper arm of each interval
fig, ax = plt.subplots(figsize=(7.2, 5.0))
lim = (-3, 18)
ax.plot(lim, lim, color=SEC, lw=0.8, zorder=1)
ax.text(17.6, 13.6, "symmetric CI\n(arms equal)", fontsize=10, color=SEC, ha="right", va="top")
ax.axhline(0, color=SEC, lw=0.8, ls=(0, (2, 2)), zorder=1)
ax.text(17.8, 0.25, "below 0: estimate above its own upper bound", fontsize=10, color=SEC,
ha="right", va="bottom")
TPOS = {"Hospital stay, AKI 1-3": (5.2, -2.2), "Hospital stay, AKI 2-3": (9.5, 4.4),
"PICU stay, AKI 1-3": (9.5, 1.9), "PICU stay, AKI 2-3": (-2.5, 9.6),
"Mechanical ventilation, AKI 1-3": (6.4, 16.6), "Mechanical ventilation, AKI 2-3": (-2.5, 12.6)}
for lab, b, lo, hi, p, sc in LIN:
la, ua = b - lo, hi - b
ax.plot([la], [ua], "o", ms=MS, color=BLUE, zorder=3)
tx, ty = TPOS[lab]
ax.annotate(f"{lab}\n({la:.1f}, {ua:.1f})".replace("-", "−").replace("AKI 1−3", "AKI 1-3").replace("AKI 2−3", "AKI 2-3"),
xy=(la, ua), xytext=(tx, ty), fontsize=10, ha="left", va="center", color=TXT,
arrowprops=dict(arrowstyle="-", color=SEC, lw=0.7, shrinkA=2, shrinkB=5))
ax.set_xlim(*lim)
ax.set_ylim(*lim)
ax.set_aspect("equal")
ax.set_xticks(range(0, 19, 5))
ax.set_yticks(range(0, 19, 5))
ax.set_xlabel("Estimate − lower bound, days (recomputed from Table 4)")
ax.set_ylabel("Upper bound − estimate, days")
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
fig.tight_layout()
save(fig, "c4")
# --- c5: the three printed locations as text, above one number line
fig = plt.figure(figsize=(7.2, 5.0))
quotes = [
("Abstract", "“… 8.4 days longer hospitalizations compared to patients without acute kidney injury\n(95% confidence interval, 4.4-6.7).”"),
("Results, Clinical course and outcomes", "“… greater length of hospital stay of 8.4 days compared with patients without AKI\n(95% CI, 4.4–6.7; P < 0.0001).”"),
("Table 4, Length of hospital stay, AKI stage 1 + 2 + 3", "β (95% CI) “8.4 (4.4 to 6.7)” P value “<0.0001”"),
]
for i, (where, q) in enumerate(quotes):
y0 = 0.80 - i * 0.165
fig.patches.append(matplotlib.patches.FancyBboxPatch(
(0.02, y0), 0.96, 0.15, boxstyle="round,pad=0.004,rounding_size=0.01", transform=fig.transFigure,
facecolor="#ffffff", edgecolor="#d6d5d0", lw=1))
fig.text(0.035, y0 + 0.13, where, fontsize=10, fontweight="bold", va="top", color=TXT)
fig.text(0.035, y0 + 0.088, q, fontsize=10, va="top", color=TXT, linespacing=1.3)
fig.text(0.5, 0.425, "the same estimate and interval in all three places, drawn on one axis below",
ha="center", fontsize=10, color=SEC)
ax = fig.add_axes((0.05, 0.13, 0.9, 0.25))
ax.plot([4.4, 6.7], [0, 0], color=ORANGE, lw=LW * 2, solid_capstyle="butt", zorder=2)
for e in (4.4, 6.7):
ax.plot([e, e], [-0.18, 0.18], color=ORANGE, lw=LW, zorder=2)
ax.plot([8.4], [0], "o", ms=MS + 1, color=ORANGE, zorder=4)
ax.plot([5.55], [0], "o", ms=MS, mfc=BG, mec=BLUE, mew=2, zorder=5)
ax.text(5.55, 0.32, "reported 95% CI 4.4 to 6.7", ha="center", fontsize=10, color=TXT)
ax.text(8.4, 0.32, "reported β = 8.4", ha="center", fontsize=10, color=TXT)
ax.text(5.55, -0.42, "CI centre 5.55\n(recomputed)", ha="center", va="top", fontsize=10, color=TXT)
ax.annotate("", xy=(6.7, -0.25), xytext=(8.4, -0.25),
arrowprops=dict(arrowstyle="<->", color=TXT, lw=1.0, shrinkA=0, shrinkB=0))
ax.text(7.55, -0.42, "1.7 d", ha="center", va="top", fontsize=10, color=TXT)
ax.axvline(0, color=SEC, lw=0.8)
ax.set_xlim(-0.3, 10)
ax.set_ylim(-1, 0.7)
ax.set_yticks([])
ax.set_xticks(range(0, 11, 2))
ax.set_xlabel("Longer hospital stay with AKI, days")
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
save(fig, "c5")
# contact sheet (PNG only, for choosing a design)
import matplotlib.image as mpimg # noqa: E402
fig, axs = plt.subplots(3, 2, figsize=(14, 16))
for ax, stem in zip(axs.flat, ("c1", "c2", "c3", "c4", "c5")):
ax.imshow(mpimg.imread(os.path.join(CAND, stem + ".png")))
ax.set_title(stem, fontsize=16, fontweight="bold", loc="left")
ax.axis("off")
axs.flat[-1].axis("off")
fig.tight_layout()
fig.savefig(os.path.join(CAND, "contact_sheet.png"), dpi=100)
plt.close(fig)
for ext in ("svg", "png"):
shutil.copyfile(os.path.join(CAND, f"{CHOSEN}.{ext}"), os.path.join(HERE, f"fig1.{ext}"))
print(f"\nwrote candidates c1-c5 and contact_sheet.png in {CAND}; fig1 = {CHOSEN}")
Abstract reports 58 (53%) of 113 resistant sites; Results' own counts give 57 (51%) of 112
Lancet Infect Dis (2014) · doi:10.1016/S1473-3099(14)70855-2 · PMC4178238
Summary
This systematic review reports its headline result in the abstract as chloroquine resistance in "58 (53%) of 113 assessable study sites". The Results instead give 57 (51%) of 112 site estimates, and the body's own sub-counts support that figure: 177 site estimates minus 65 that could not be categorised (37 + 15 + 4 + 4 + 5) leaves 112, and resistance categories 1, 2 and 3 (35 + 8 + 14) add up to 57. The abstract figure is also wrong on its own terms, because 58/113 = 51.3%, which rounds to 51%, not 53%. The qualitative finding, that about half of assessable sites showed resistance, is unchanged. But the abstract's one headline prevalence figure, the number most likely to be quoted, is about 2 percentage points too high, and its count and denominator match nothing in the paper.
What the paper reports
Abstract, Findings:
We identified 129 eligible clinical trials involving 21 694 patients at 179 study sites and 26 case reports describing 54 patients. Chloroquine resistance was present in 58 (53%) of 113 assessable study sites, spread across most countries that are endemic for P vivax.
Results, fifth paragraph:
Estimates of chloroquine efficacy could be derived for 177 sites from 144 treatment groups. In 65 (37%) of these sites chloroquine resistance could not be categorised for the following reasons: early administration of primaquine (37 sites); no documentation of parasitological data at day 28 (15 sites); sample size of fewer than ten patients (four sites); good response documented in asymptomatic patients (four sites); and administration of only a single dose of chloroquine (five sites). Overall P vivax was defined as chloroquine resistant for 57 (51%) of 112 site estimates, 35 (61%) of these sites fulfilled the predefined criteria for category 1 resistance, eight (14%) for category 2 resistance, and 14 (25%) for category 3 resistance.
Table 1, "Site estimates of chloroquine efficacy" column, "Overall" row:
Overall | 129 | 230 | 177
Discussion, ninth paragraph:
Our definitions were conservative, with data excluded from almost 40% of studies that we thought could be confounded. Almost half of the informative studies suggested evidence of reduced chloroquine efficacy.
What the paper's own numbers show
Method. I checked each printed percentage against its own fraction. I rebuilt the assessable denominator from the Table 1 site total and the five listed reasons for exclusion, and rebuilt the resistant count from the three resistance categories. I checked the Table 1 site total by each of its three breakdowns (location, year, origin of patients). Finally, I searched for fractions that would round to 53%. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Resistant sites, abstract | Abstract, Findings | 58 (53%) of 113 | 58/113 = 51.3%, which rounds to 51%, not 53% |
| Resistant sites, Results | Results, para 5 | 57 (51%) of 112 | 57/112 = 50.9% (51%); consistent |
| Chloroquine site estimates | Table 1; Results, para 5 | 177 | 134 + 13 + 22 + 8 = 177 by location; 8 + 41 + 80 + 48 = 177 by year; 22 + 21 + 134 = 177 by origin |
| Uncategorisable sites | Results, para 5 | 65 (37%) | 37 + 15 + 4 + 4 + 5 = 65; 65/177 = 36.7% (37%) |
| Assessable sites | Results 112; Abstract 113 | 112 / 113 | 177 − 65 = 112 |
| Resistant sites = categories 1 + 2 + 3 | Results 57; Abstract 58 | 57 / 58 | 35 + 8 + 14 = 57 |
| Category shares | Results, para 5 | 61% / 14% / 25% | of 57: 61.4% / 14.0% / 24.6% (match); of 58: 60.3% / 13.8% / 24.1% (35/58 would print as 60%, not 61%) |
| Implied sensitive sites | derived | — | 112 − 57 = 55 (Results); 113 − 58 = 55 (Abstract) |
| Fractions that round to 53% | derived | 53% | 58/109, 58/110, 60/113 or 59/112; none of these counts appears in the paper |
| Size of the discrepancy | derived | — | printed 53% is 1.67 points above 58/113 and 2.11 points above 57/112; the two fractions differ by 0.43 points |
Figure
Figure 1. The two passages that report the overall prevalence of chloroquine-resistant P vivax sites. Values printed in the abstract are in orange; values printed in the Results, which agree with the body's sub-counts, and values recomputed from the paper's numbers are in blue. The abstract's count (58) and denominator (113) are each one site higher than the Results' (57 and 112), and its percentage (53%) does not follow from its own fraction (58/113 = 51.3%). The lower panel rebuilds the Results' figures from the Table 1 site total, the five listed reasons for exclusion and the three resistance categories. All numbers are from the paper or from recompute.py.txt.
Why it matters
The abstract's sentence is the paper's single summary figure for how widespread chloroquine-resistant P vivax is, and it is the version most likely to be quoted in later reviews, guideline background documents and resistance summaries. The error is small: the figure the body supports is 51% (57/112), and the abstract's own fraction also gives 51%. Its direction and the qualitative reading ("almost half of the informative studies", in the Discussion's words) are unchanged, and so are the conclusions that resistance is geographically widespread and that monitoring needs to improve. The practical consequences are that readers who quote the abstract carry a percentage about 2 points too high, and that the counts 58 and 113 cannot be reconciled with the paper's own tables and text. The abstract and the body disagree in three places at once (count, denominator and percentage), which will puzzle anyone who tries to check or update the analysis.
Explanations considered
- A different unit of analysis. The abstract says "assessable study sites" and the Results say "site estimates". The paper gives only two site totals: 179 study sites for all drugs (Abstract) and 177 site estimates of chloroquine efficacy (Results; Table 1, whose three breakdowns each sum to 177). The only assessable denominator the paper derives is 177 − 65 = 112. No figure in the text, tables or legends gives 113. Applying the 65 exclusions to the 179 study sites would give 114, not 113. This does not account for the numbers.
- A different set of resistance categories. Resistance is defined as categories 1, 2 or 3, and these sum to 35 + 8 + 14 = 57. No further resistant subgroup is described that could add a 58th site. Both versions imply 55 sensitive sites (112 − 57 = 55 and 113 − 58 = 55). The abstract therefore has one extra site that is counted as resistant. The printed category shares also fit a base of 57 rather than 58 (35/58 = 60.3% would print as 60%, not 61%). This does not account for the numbers.
- Rounding convention. 58/113 = 51.33%. No rounding rule turns that into 53%. A printed 53% would need 58/109, 58/110 or 60/113 (or 59/112), and none of these counts appears in the paper. This does not account for the percentage.
- A conversion artefact in the archived text. The same sentences appear in the Lancet's own typeset PDF (the online-first version), in the PMC HTML and in the Europe PMC XML. This is not a conversion artefact.
- A data update between versions, such as an earlier count left in the abstract. The body is internally consistent throughout (65 = 37 + 15 + 4 + 4 + 5; 65/177 = 37%; 57/112 = 51%; the category shares of 57). That points to the abstract's figure as the inconsistent one, but the paper gives no information on how it arose, and even such a history would not explain why 53% does not equal 58/113.
- The supplement. The single supplementary appendix contains reference lists and a reference table only. It has no site-level categories or counts, so it cannot reconcile 58/113 with 57/112.
Status
- Version of record. I checked the Lancet's typeset PDF ("Published online September 9, 2014", www.thelancet.com/infection headers), held in the Oxford ORA repository. It contains both sentences word for word: "Chloroquine resistance was present in 58 (53%) of 113 assessable study sites" in the abstract, and "Overall P vivax was defined as chloroquine resistant for 57 (51%) of 112 site estimates" in the Results. The PMC copy is the publisher-deposited open-access (CC BY) version and shows the same text, as do the PMC HTML page and the Europe PMC full-text XML. The publisher's site (thelancet.com) refused automated access (HTTP 403), and the DOI resolver stopped at a redirect page, so I could not compare against the final issue version (14(10):982–91) there. Every quote above was confirmed verbatim against the PMC-archived text.
- Supplements. I read the one supplementary appendix (mmc1.pdf, 21 pages, from the Europe PMC supplementary-files bundle). It holds the reference lists for the clinical trials and case reports and a reference table, with no site-level resistance data. The authors' 2015 reply to letters (PMC4629410; Lancet Infect Dis 2015;15:630–1) says the full dataset is available through the Worldwide Antimalarial Resistance Network. I did not find a downloadable copy: the network's publication page (iddo.org/node/5315) lists the paper but offers no data files. The reply addresses other points (literature from China, follow-up duration) and does not mention the site counts.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and a Crossref query for works that update it returned none. PMC and Europe PMC list no correction notices. The Europe PMC record (PMID 25213732) links only comments: an editorial comment and two 2015 letters with the authors' reply. The reply, which answers both letters (on literature from China and on follow-up duration), does not mention the site counts. The PubPeer API was blocked by a Cloudflare challenge, and a web search found no PubPeer thread for the paper.
- Sources. PMC article PMC4178238; Lancet typeset PDF via ORA (ora.ox.ac.uk, uuid:db45d120-9689-40f3-8e61-1006e9886bc0); Europe PMC full text and supplementary files; authors' reply PMC4629410; Crossref; Europe PMC; iddo.org/node/5315; PubPeer.
Suggested correction
- In the abstract's Findings, replace "Chloroquine resistance was present in 58 (53%) of 113 assessable study sites" with "Chloroquine resistance was present in 57 (51%) of 112 assessable study sites". This matches the Results, the five listed exclusions from the 177 site estimates in Table 1, and the 35 + 8 + 14 resistance categories.
- If the authors' dataset shows that 58 of 113 is the correct count, then the Results paragraph (the 112 denominator, the 65 exclusions and their breakdown, the 57 total and the category counts and shares) needs correcting instead. The abstract percentage would then still need to change from 53% to 51%, because 58/113 = 51.3%.
Recomputation script (python recompute.py.txt)
# Recompute the site counts behind the chloroquine-resistance prevalence in
# PMC4178238 (Lancet Infect Dis 2014; doi:10.1016/S1473-3099(14)70855-2)
# and regenerate fig1.svg / fig1.png in this file's folder.
# Run: python recompute.py.txt
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
pct = lambda a, b: 100.0 * a / b
# ---- Numbers printed in the paper -------------------------------------------
abs_res, abs_den, abs_pct = 58, 113, 53 # Abstract, Findings
res_res, res_den, res_pct = 57, 112, 51 # Results, para 5
sites_all_drugs = 179 # Abstract: "179 study sites"
cq_sites = 177 # Results / Table 1
uncat = {"early primaquine": 37, "no day-28 data": 15, "n < 10": 4,
"asymptomatic, good response": 4, "single CQ dose": 5}
uncat_reported, uncat_pct_reported = 65, 37
cats = {1: 35, 2: 8, 3: 14}
cats_pct_reported = {1: 61, 2: 14, 3: 25}
table1 = {"location": [134, 13, 22, 8], "year": [8, 41, 80, 48],
"origin": [22, 21, 134]}
print("== Abstract figure checked against its own fraction")
print("58/113 = %.2f%% -> rounds to %d%% (printed %d%%)"
% (pct(abs_res, abs_den), round(pct(abs_res, abs_den)), abs_pct))
print("== Results figure checked against its own fraction")
print("57/112 = %.2f%% -> rounds to %d%% (printed %d%%)"
% (pct(res_res, res_den), round(pct(res_res, res_den)), res_pct))
print("== Table 1 site-estimate totals")
for k, v in table1.items():
print(" by %-8s: %s = %d" % (k, "+".join(map(str, v)), sum(v)))
print("== Uncategorisable sites (Results para 5)")
u = sum(uncat.values())
print(" 37+15+4+4+5 = %d (printed %d); %d/177 = %.1f%% (printed %d%%)"
% (u, uncat_reported, u, pct(u, cq_sites), uncat_pct_reported))
assess = cq_sites - u
print(" assessable = 177 - %d = %d (Results %d; Abstract %d)"
% (u, assess, res_den, abs_den))
print("== Resistant sites = categories 1+2+3")
r = sum(cats.values())
print(" 35+8+14 = %d (Results %d; Abstract %d)" % (r, res_res, abs_res))
for c, n in cats.items():
print(" category %d: %d/57 = %.1f%% (printed %d%%)"
% (c, n, pct(n, r), cats_pct_reported[c]))
print(" same shares with 58 as base: " + ", ".join(
"%d/58 = %.1f%%" % (n, pct(n, 58)) for n in cats.values()))
print(" 179 study sites - 65 = %d (not 113)" % (sites_all_drugs - u))
print("== Implied sensitive (category 4) sites")
print(" Results: 112-57 = %d; Abstract: 113-58 = %d"
% (res_den - res_res, abs_den - abs_res))
print("== Differences")
print(" printed 53%% minus 58/113: %.2f points" % (abs_pct - pct(58, 113)))
print(" printed 53%% minus 57/112: %.2f points" % (abs_pct - pct(57, 112)))
print(" 58/113 minus 57/112: %.2f points" % (pct(58, 113) - pct(57, 112)))
print("== Fractions that would round to 53%")
print(" d with round(58/d) = 53%%: %s"
% [d for d in range(90, 131) if round(pct(58, d)) == 53])
print(" n with round(n/113) = 53%%: %s"
% [n for n in range(40, 81) if round(pct(n, 113)) == 53])
print(" n with round(n/112) = 53%%: %s"
% [n for n in range(40, 81) if round(pct(n, 112)) == 53])
print(" n with round(n/177) = 53%%: %s"
% [n for n in range(60, 120) if round(pct(n, 177)) == 53])
print(" n with round(n/179) = 53%%: %s"
% [n for n in range(60, 120) if round(pct(n, 179)) == 53])
print(" 58/179 = %.1f%%, 58/177 = %.1f%%, 57/177 = %.1f%%"
% (pct(58, 179), pct(58, 177), pct(57, 177)))
# ---- Figure 1: abstract vs Results, with the body's arithmetic -------------
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#d9d8d4"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
"svg.fonttype": "none"})
fig = plt.figure(figsize=(7.2, 5.0), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 72); ax.set_ylim(0, 50)
ax.axis("off"); ax.set_facecolor(BG)
def panel(x, y, w, h, edge, title):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.3,rounding_size=1",
fc="white", ec=edge, lw=1.5))
ax.text(x + 1, y + h - 1.6, title, fontsize=11, fontweight="bold",
color=TXT, va="top")
# Left: Abstract
panel(1.5, 21, 28.5, 25, ORANGE, "Abstract, Findings")
ax.text(2.5, 40.5, "“Chloroquine resistance was present\nin 58 (53%) of 113 "
"assessable\nstudy sites”", fontsize=10, color=TXT, va="top",
style="italic", linespacing=1.35)
ax.text(2.5, 31.0, "Count 58", fontsize=11, color=ORANGE, fontweight="bold")
ax.text(2.5, 28.0, "Denominator 113", fontsize=11, color=ORANGE, fontweight="bold")
ax.text(2.5, 25.0, "Percentage 53%", fontsize=11, color=ORANGE, fontweight="bold")
ax.text(2.5, 22.0, "but 58/113 = %.1f%% → 51%%" % pct(58, 113),
fontsize=10, color=BLUE, fontweight="bold")
# Right: Results
panel(42, 21, 28.5, 25, BLUE, "Results, paragraph 5")
ax.text(43, 40.5, "“Overall P vivax was defined as\nchloroquine resistant "
"for 57 (51%)\nof 112 site estimates”", fontsize=10, color=TXT,
va="top", style="italic", linespacing=1.35)
ax.text(43, 31.0, "Count 57", fontsize=11, color=BLUE, fontweight="bold")
ax.text(43, 28.0, "Denominator 112", fontsize=11, color=BLUE, fontweight="bold")
ax.text(43, 25.0, "Percentage 51%", fontsize=11, color=BLUE, fontweight="bold")
ax.text(43, 22.0, "57/112 = %.1f%% → 51%%" % pct(57, 112),
fontsize=10, color=BLUE, fontweight="bold")
# Connector
ax.add_patch(FancyArrowPatch((31.0, 33.5), (41.3, 33.5), arrowstyle="<->",
mutation_scale=14, lw=2, color=SEC))
ax.text(36, 35.2, "conflict", ha="center", fontsize=10, color=SEC, fontweight="bold")
ax.text(36, 31.6, "one site\nhigher in\nboth count\nand\ndenominator", ha="center",
va="top", fontsize=10, color=SEC, linespacing=1.2)
# Bottom: the body's own arithmetic
panel(1.5, 1.5, 69, 15.5, GRID, "What the Results' own sub-counts give (recomputed)")
ax.text(2.5, 11.6, "Site estimates (Table 1) 177", fontsize=10, color=TXT)
ax.text(2.5, 8.6, "− uncategorisable 37+15+4+4+5 = %d" % u, fontsize=10, color=TXT)
ax.text(2.5, 5.6, "= assessable %d" % assess, fontsize=11, color=BLUE, fontweight="bold")
ax.text(37, 11.6, "Resistant = category 1 + 2 + 3", fontsize=10, color=TXT)
ax.text(37, 8.6, "35 + 8 + 14 = %d" % r, fontsize=10, color=TXT)
ax.text(37, 5.6, "= resistant %d → %d/%d = %.1f%%" % (r, r, assess, pct(r, assess)),
fontsize=11, color=BLUE, fontweight="bold")
ax.text(2.5, 2.6, "No count or subgroup in the paper gives 113 or 58; 53% needs 58/109, 58/110 or 60/113.",
fontsize=10, color=SEC)
# Legend (names the colours)
ax.text(1.8, 48.3, "■", color=ORANGE, fontsize=12, va="center")
ax.text(3.6, 48.3, "printed in the abstract", color=TXT, fontsize=10, va="center")
ax.text(26, 48.3, "■", color=BLUE, fontsize=12, va="center")
ax.text(27.8, 48.3, "recomputed from / consistent with the body's counts",
color=TXT, fontsize=10, va="center")
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, "fig1." + ext), facecolor=BG, **kw)
print("wrote fig1.svg and fig1.png in", HERE)
Abstract reports lower Ct values in vaccinated participants; the paper's data and text show higher
Lancet Reg Health Am (2025) · doi:10.1016/j.lana.2025.101309 · PMC12703870
Summary
The Findings section of the Abstract states that vaccinated participants had "significantly lower RT-qPCR Ct values" than placebo recipients. The Results report the opposite: mean Ct was higher in the vaccinated group for both serotypes, by 5.0 cycles for DENV-1 (27.2 vs 22.2) and 2.9 cycles for DENV-2 (28.2 vs 25.3), and the Research in context panel, the Discussion, Fig. 2a/2i and Supplementary Table 6 all agree with the Results. Because the paper defines Ct as inversely related to the amount of viral nucleic acid, the Abstract's wording implies more viral RNA in vaccinated participants, which contradicts both the data and the conclusion drawn in the same sentence ("the vaccine could be reducing viral replication"). The error is confined to one word in the Abstract, but it reverses the direction of the paper's main virological finding in the part of the paper most often read on its own.
What the paper reports
Abstract, Findings:
Despite the limited statistical power due to the low number of samples, we observed significantly lower RT-qPCR Ct values in the vaccinated group, particularly for the DENV-1 cases, suggesting that the vaccine could be reducing viral replication.
Research in context, "Added value of this study":
Additionally, we observed higher RT-qPCR Ct values in the vaccinated group compared to the placebo group, indicating a lower viral load among vaccinated participants and suggesting that the Butantan-DV is initially effective in reducing viral loads.
Results, "Characterization of RT-qPCR cycle threshold values":
RT-qPCR cycle threshold (Ct) values indirectly measure DENV genomic load in samples, with an inverse relationship to the input target nucleic acid. Our analysis included a total of 365 DENV-positive samples, with an overall mean Ct of 23.7 (SD: 4.85) for DENV-1 (n = 161) and 26.5 (SD: 4.67) for DENV-2 (n = 205). When comparing RT-qPCR Ct values between the Pb and Vx groups for each serotype, a statistically significant difference was observed (p < 0.001). The Vx group exhibited a higher mean Ct value than the Pb group. Specifically, the mean Ct for DENV-2 was 28.2 (SD: 4.18) in the Vx group compared to 25.3 (SD: 4.65) in the Pb group. This trend was even stronger for DENV-1, with the mean Ct being 27.2 (SD: 3.98) in the Vx group compared to 22.2 (SD: 4.40) in the Pb group (Fig. 2a and i) (Supplementary Table S6).
Discussion, fourth paragraph:
The higher Ct values observed in the vaccinated group, therefore, suggest that Butantan-DV-induced immunity may confer protection not only by preventing infection, but also by mitigating disease severity through the suppression of viral replication.
Results, "Sample characterization":
There were 234 (64.1%) participants included in the Pb group and 131 (35.8%) participants in the Vx group confirmed for DENV.
Supplementary Table 6 ("Comparative analysis of cycle threshold (Ct) values for DENV-1 and DENV-2 between vaccinated and placebo groups"; columns: N positive Placebo | Butantan-DV | mean Ct Placebo | Butantan-DV | Δ mean | t-statistic | p-value | Cohen's d):
DENV-1 | 113 | 48 | 22.2 | 27.2 | 5.00 | 10.87 | < 0.001 | 1.16
DENV-2 | 122 | 83 | 25.3 | 28.2 | 2.90 | 6.21 | < 0.001 | 0.86
Its footnote reads:
The p-value and t-statistic were calculated using the Welch's t-test for unequal variances.
What the paper's own numbers show
Method. I took the group means and SDs from the Results text and the group sizes from Supplementary Table 6. I computed the vaccinated-minus-placebo difference in mean Ct for each serotype and checked whether rounding of the printed means (to 0.1) could change its sign. To rule out swapped group labels in the body, I checked that the n-weighted combination of the two group means reproduces the overall mean the Results report for each serotype, and compared that with the combination obtained if the labels were swapped. I then recomputed a Welch t-test, its 95% CI and Cohen's d from the summary statistics. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Direction, all serotypes | Abstract, Findings | "significantly lower RT-qPCR Ct values in the vaccinated group" | Opposite to every number below |
| DENV-1 mean Ct, Vx vs Pb | Results; Suppl. Table 6 | 27.2 (SD 3.98) vs 22.2 (SD 4.40); Δ 5.00 | Vx − Pb = +5.0 cycles (+4.9 to +5.1 allowing for rounding): vaccinated higher |
| DENV-2 mean Ct, Vx vs Pb | Results; Suppl. Table 6 | 28.2 (SD 4.18) vs 25.3 (SD 4.65); Δ 2.90 | Vx − Pb = +2.9 cycles (+2.8 to +3.0): vaccinated higher |
| DENV-1 overall mean and SD | Results | 23.7 (SD 4.85), n = 161 | With Table S6 n (48 Vx, 113 Pb): 23.69 (SD 4.84). Labels swapped: 25.71 |
| DENV-2 overall mean and SD | Results | 26.5 (SD 4.67), n = 205 | With Table S6 n (83 Vx, 122 Pb): 26.47 (SD 4.68). Labels swapped: 27.03 |
| Vaccinated share implied by overall means | Results | 131 Vx of 365 | 0.300 × 161 + 0.414 × 205 ≈ 133.1 Vx, close to the 131 in Table S6 and the Results |
| DENV-1 difference, 95% CI and test | Results "p < 0.001"; Table S6 | p < 0.001 | Welch 95% CI +3.6 to +6.4; t = 7.06 (df 97.4), p = 2.4 × 10⁻¹⁰ |
| DENV-2 difference, 95% CI and test | Results "p < 0.001"; Table S6 | p < 0.001 | Welch 95% CI +1.7 to +4.1; t = 4.66 (df 187.9), p = 6.0 × 10⁻⁶ |
| Meaning of the direction | Results | Ct has "an inverse relationship to the input target nucleic acid" | Higher Ct in Vx = less viral RNA in Vx, as stated in Research in context and the Discussion |
The body of the paper is internally consistent: the group means, overall means, group sizes, Fig. 2a and 2i (boxplots in which the vaccinated group sits above placebo for both serotypes) and Supplementary Table 6 all describe higher Ct in vaccinated participants. Neither CI comes near zero, so the reversed direction in the Abstract cannot be a matter of rounding or of a borderline result.
Four smaller inconsistencies turned up during the check. None of them affects the direction or the significance of the difference:
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Welch t-statistic | Suppl. Table 6 | 10.87 (DENV-1), 6.21 (DENV-2) | 7.06 and 4.66 from the printed means, SDs and n; p stays below 0.001 |
| Cohen's d, DENV-2 | Suppl. Table 6 | 0.86 | 0.65 with the pooled SD (4.47); DENV-1 reproduces (1.17 vs 1.16) |
| Group counts | Suppl. Table 6 vs Results | 113 + 122 = 235 placebo; 161 + 205 = 366 serotype results | Results give 234 placebo and 365 samples (difference of one) |
| Test used for the Ct comparison | Methods vs Suppl. Table 6 | Methods and Fig. 2 legend: Wilcoxon rank-sum; Table S6: Welch t-test | Both tests are reported as p < 0.001 |
Figure
Figure 1. The two statements of the vaccinated-vs-placebo Ct comparison, quoted from the Abstract (Findings) and from the Results ("Characterization of RT-qPCR cycle threshold values"). Orange outlines mark text and values as reported in the paper; blue text is the difference recomputed from the Results means. The Abstract states that Ct was lower in the vaccinated group; the Results, which describe the same 365 samples, state and show that it was higher, by 5.0 cycles for DENV-1 and 2.9 cycles for DENV-2. The other places in the paper that give a direction (Research in context, Discussion, Fig. 2a/2i, Supplementary Table 6) agree with the Results.
Figure 2. Difference in mean RT-qPCR Ct, vaccinated minus placebo, on one axis of Ct cycles. Orange diamonds are the differences printed in Supplementary Table 6 (5.00 and 2.90). Blue circles and lines are the differences recomputed from the Results means and SDs with the Table S6 group sizes, with Welch 95% confidence intervals (+3.6 to +6.4 and +1.7 to +4.1). The vertical line marks no difference. The shaded half is the direction the Abstract states (vaccinated Ct lower); both intervals lie entirely in the other half.
Why it matters
The Ct comparison is the only viral-load finding in the Abstract; the other findings there describe the sample, lineages and selection. Many readers, indexing services and secondary summaries rely on the Abstract alone. A reader who takes the Abstract's "significantly lower RT-qPCR Ct values in the vaccinated group" at face value would conclude that vaccinated participants with breakthrough infections carried more viral RNA than unvaccinated ones, which is the opposite of what the paper found and the opposite of the protective interpretation the paper draws.
The sentence also contradicts itself. Under the paper's own definition, lower Ct means more viral nucleic acid, so "lower Ct" cannot support "the vaccine could be reducing viral replication". A reader who knows the Ct scale will notice the conflict; a reader who does not may repeat either half. Press coverage of the study (for example Agência FAPESP) reported reduced viral load in vaccinated participants, which matches the body of the paper, so I found no sign that the reversed wording has been repeated, but the Abstract is the text that remains attached to the article in PubMed, Europe PMC and the publisher's site.
The error does not affect the size, significance or interpretation of the result in the body of the paper, nor its main phylogenetic conclusions (no lineage linked to breakthrough infections, no positive selection, similar intra-host mutation rates), which do not depend on Ct.
Explanations considered
- The Abstract refers to a different metric, such as viral load or ΔCt. This does not account for the text. The Abstract names "RT-qPCR Ct values" explicitly, and "lower viral load" is the phrase the paper uses elsewhere for the same finding (Research in context).
- The Abstract summarizes medians rather than means. This does not account for it. The Methods compare medians with Wilcoxon and Kruskal-Wallis tests, and the boxplots in Fig. 2a and 2i show the vaccinated group's box and median above the placebo group's for both serotypes, so medians and means point the same way.
- The Abstract summarizes a different subgroup or time window. This does not account for it. The Abstract and the Results both describe the same 365 DENV-positive samples. The analyses by days after vaccination (Fig. 2e–h and 2m–p) report no correlation and are not what the Abstract summarizes; in Fig. 2e and 2m the vaccinated curve also lies above the placebo curve over most of the range.
- The group labels are swapped in the body, and the Abstract is right. This is excluded by the paper's own numbers. With the Table S6 group sizes, the n-weighted group means reproduce the reported overall means (23.69 vs 23.7; 26.47 vs 26.5), whereas swapped labels would give 25.71 and 27.03. Fig. 2a/2i, Table S6, the Research in context panel and the Discussion all carry the same labelling.
- The Abstract uses an inverted convention for the Ct scale. This does not account for it. The paper states that Ct has "an inverse relationship to the input target nucleic acid", and the Abstract's own conclusion (reduced viral replication) only follows from higher Ct.
- A conversion or typesetting artifact in the archived text. This is excluded. The publisher's typeset PDF (Elsevier, 18 pages) carries the same word "lower" in the Summary on page 1 and "higher" on pages 2, 9 and 15, as do the PMC and Europe PMC full texts.
The remaining explanation consistent with all the evidence is that the word "lower" in the Abstract should read "higher".
Status
- Version of record. I checked the publisher's typeset PDF from the PMC Open Access dataset (PMC12703870.1; PDF metadata: creator Elsevier, "The Lancet Regional Health - Americas, 53 (2026) 101309", created 2025-11-29). It reads the same as the PMC-archived text in every passage quoted above. The publisher's web pages (thelancet.com, sciencedirect.com) refused automated access. Every quote above was confirmed verbatim against the PMC-archived text with
src/show.py, except the Supplementary Table 6 rows, which were read from the supplement file. - Supplements. The only supplement, mmc1.docx (from the Europe PMC supplementary-files bundle), was opened and its Supplementary Table 6 extracted; it is quoted above. I also viewed the publisher's Fig. 2 image (gr2.jpg) to confirm the direction of panels a, e, i and m; the figure is not reproduced here.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, expression of concern or retraction: Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction or comment links (PMID 41403717). The PubPeer API was blocked by a Cloudflare challenge, and a web search found no PubPeer entry for the DOI. A web search for the article found only press coverage.
- Sources. PMC article PMC12703870; PMC Open Access PDF (s3://pmc-oa-opendata/PMC12703870.1/PMC12703870.1.pdf); Europe PMC REST API (full text, supplementary files, metadata); Crossref; PubPeer; Agência FAPESP coverage.
Suggested correction
- In the Abstract (Findings), change "we observed significantly lower RT-qPCR Ct values in the vaccinated group" to "we observed significantly higher RT-qPCR Ct values (indicating lower viral load) in the vaccinated group". No other text, table or figure needs to change for this point.
- Minor, for Supplementary Table 6: check the Welch t-statistics (7.06 and 4.66 follow from the printed means, SDs and n, against 10.87 and 6.21 printed) and the DENV-2 Cohen's d (0.65 vs 0.86); reconcile the placebo count (113 + 122 = 235) with the 234 placebo participants and 365 samples in the Results; and state which test (Wilcoxon rank-sum, per the Methods, or Welch t, per the table) gives the reported p < 0.001.
Recomputation script (python recompute.py.txt)
"""Recompute the vaccinated-vs-placebo RT-qPCR Ct comparison of PMC12703870
(Lancet Reg Health Am, doi:10.1016/j.lana.2025.101309).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes, next to this file:
fig1.svg/png, fig2.svg/png (the figures used in writeup.md)
candidates/cand_A..E.svg/png (five alternative figure designs for review)
Inputs are copied from the paper as archived in PMC and from its only supplement
(mmc1.docx, Supplementary Table 6):
Results, "Characterization of RT-qPCR cycle threshold values": group means/SDs,
overall means/SDs and serotype n
Results, "Sample characterization": 234 Pb and 131 Vx participants (365 in all)
Supplementary Table 6: group n, mean difference, Welch t, Cohen's d
"""
import math
import os
import textwrap
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, Patch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ------------------------------------------------------------------ inputs (from the paper)
# Results text: mean (SD)
REP = {
"DENV-1": dict(n=161, overall=(23.7, 4.85), vx=(27.2, 3.98), pb=(22.2, 4.40)),
"DENV-2": dict(n=205, overall=(26.5, 4.67), vx=(28.2, 4.18), pb=(25.3, 4.65)),
}
# Supplementary Table 6: N positive (Pb, Vx), delta mean, t (Welch), Cohen's d
S6 = {
"DENV-1": dict(n_pb=113, n_vx=48, delta=5.00, t=10.87, d=1.16),
"DENV-2": dict(n_pb=122, n_vx=83, delta=2.90, t=6.21, d=0.86),
}
N_PB_TOTAL, N_VX_TOTAL, N_TOTAL = 234, 131, 365 # Results, "Sample characterization"
# ------------------------------------------------------------------ style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE = "#eb6834", "#2a78d6"
GRID = "#e4e3df"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5,
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, path_noext):
fig.savefig(path_noext + ".svg", bbox_inches="tight", pad_inches=0.12)
fig.savefig(path_noext + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
# ------------------------------------------------------------------ 1. direction and size
print("1. Direction of the group difference (Results text)")
R = {}
for s, r in REP.items():
mv, sv = r["vx"]
mp, sp = r["pb"]
diff = mv - mp
# rounding to one decimal moves each mean by at most 0.05
lo, hi = diff - 0.1, diff + 0.1
R[s] = dict(diff=diff)
print(f" {s}: Vx {mv} - Pb {mp} = {diff:+.1f} Ct cycles "
f"(range allowed by rounding {lo:+.1f} to {hi:+.1f}) -> vaccinated "
f"{'HIGHER' if lo > 0 else 'LOWER/unclear'}")
# ------------------------------------------------------------------ 2. group sizes and consistency
print("\n2. Group sizes: implied by the overall mean vs Supplementary Table 6")
implied_vx_total = 0.0
for s, r in REP.items():
mo, so = r["overall"]
mv, sv = r["vx"]
mp, sp = r["pb"]
w = (mo - mp) / (mv - mp)
implied_vx_total += w * r["n"]
nv, npb = S6[s]["n_vx"], S6[s]["n_pb"]
n = nv + npb
m = (nv * mv + npb * mp) / n
ss = (nv - 1) * sv ** 2 + (npb - 1) * sp ** 2 + nv * (mv - m) ** 2 + npb * (mp - m) ** 2
sd = math.sqrt(ss / (n - 1))
R[s].update(n_vx=nv, n_pb=npb, pooled_mean=m, pooled_sd=sd)
print(f" {s}: overall mean {mo} implies Vx share {w:.3f} -> n_Vx ~ {w * r['n']:.1f}, "
f"n_Pb ~ {r['n'] - w * r['n']:.1f}")
print(f" Table S6 n: Vx {nv}, Pb {npb} (sum {n}; Results n = {r['n']}); "
f"n-weighted mean {m:.2f} (reported {mo}), pooled SD {sd:.2f} (reported {so})")
print(f" if the group labels were swapped: n-weighted mean "
f"{(nv * mp + npb * mv) / n:.2f} (reported {mo})")
vx6 = sum(S6[s]["n_vx"] for s in S6)
pb6 = sum(S6[s]["n_pb"] for s in S6)
print(f" Vx implied by overall means ~ {implied_vx_total:.1f}; Table S6 Vx total {vx6} "
f"(Results {N_VX_TOTAL}); Table S6 Pb total {pb6} (Results {N_PB_TOTAL}); "
f"serotype n {REP['DENV-1']['n']} + {REP['DENV-2']['n']} = "
f"{REP['DENV-1']['n'] + REP['DENV-2']['n']} (Results {N_TOTAL} samples)")
# ------------------------------------------------------------------ 3. tests from summary stats
print("\n3. Welch t-test from the reported means, SDs and Table S6 n (two-sided)")
for s, r in REP.items():
mv, sv = r["vx"]
mp, sp = r["pb"]
nv, npb = R[s]["n_vx"], R[s]["n_pb"]
t, p = stats.ttest_ind_from_stats(mv, sv, nv, mp, sp, npb, equal_var=False)
se = math.sqrt(sv ** 2 / nv + sp ** 2 / npb)
df = se ** 4 / ((sv ** 2 / nv) ** 2 / (nv - 1) + (sp ** 2 / npb) ** 2 / (npb - 1))
q = stats.t.ppf(0.975, df)
ci = (R[s]["diff"] - q * se, R[s]["diff"] + q * se)
sp_pool = math.sqrt(((nv - 1) * sv ** 2 + (npb - 1) * sp ** 2) / (nv + npb - 2))
d = R[s]["diff"] / sp_pool
R[s].update(t=t, p=p, df=df, ci=ci, d=d)
print(f" {s}: diff {R[s]['diff']:+.2f}, SE {se:.3f}, Welch df {df:.1f}, "
f"95% CI {ci[0]:+.2f} to {ci[1]:+.2f}; t = {t:.2f}, p = {p:.1e}")
print(f" Table S6: delta {S6[s]['delta']:.2f}, t {S6[s]['t']:.2f}, d {S6[s]['d']:.2f}; "
f"recomputed Cohen's d (pooled SD {sp_pool:.2f}) = {d:.2f}")
print(f" t needed to reproduce Table S6 t with this difference: SE would be "
f"{R[s]['diff'] / S6[s]['t']:.3f} (vs {se:.3f})")
print("\n4. What the direction means on the paper's own scale")
print(" Results: Ct values have 'an inverse relationship to the input target nucleic acid'.")
print(" Higher Ct in Vx -> less viral RNA in Vx -> consistent with 'reducing viral replication'.")
print(" 'Lower Ct in Vx' (Abstract) would mean more viral RNA in Vx, contradicting the same")
print(" sentence's conclusion, the Results numbers, Research in context and the Discussion.")
# ------------------------------------------------------------------ shared text (verbatim)
Q_ABS = ("we observed significantly lower RT-qPCR Ct values in the vaccinated group, "
"particularly for the DENV-1 cases, suggesting that the vaccine could be "
"reducing viral replication")
Q_ADD = ("we observed higher RT-qPCR Ct values in the vaccinated group compared to the "
"placebo group, indicating a lower viral load among vaccinated participants")
Q_RES = "The Vx group exhibited a higher mean Ct value than the Pb group."
Q_RES1 = ("the mean Ct being 27.2 (SD: 3.98) in the Vx group compared to 22.2 (SD: 4.40) "
"in the Pb group")
Q_RES2 = "the mean Ct for DENV-2 was 28.2 (SD: 4.18) in the Vx group compared to 25.3 (SD: 4.65) in the Pb group"
Q_DIS = "The higher Ct values observed in the vaccinated group"
Q_INV = "with an inverse relationship to the input target nucleic acid"
d1, d2 = R["DENV-1"]["diff"], R["DENV-2"]["diff"]
RECOMP = f"Vx − Pb mean Ct: {d1:+.1f} cycles (DENV-1), {d2:+.1f} cycles (DENV-2)"
def wrap(s, n):
return "\n".join(textwrap.wrap(s, n, break_on_hyphens=False))
def panel(ax, x, y, w, h, edge, lw=1.6, face="white"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
lw=lw, ec=edge, fc=face, transform=ax.transAxes))
def legend_text_kinds(ax, loc="lower center", y=-0.02):
h = [Patch(fc="white", ec=ORANGE, lw=2, label="Text or values as reported in the paper"),
Patch(fc="white", ec=BLUE, lw=2, label="Recomputed from the paper's own numbers")]
ax.legend(handles=h, loc=loc, bbox_to_anchor=(0.5, y), ncol=2, frameon=False, fontsize=10)
# ------------------------------------------------------------------ design A: two text panels
def design_a(path):
fig = plt.figure(figsize=(W, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.axis("off")
# left: Abstract
panel(ax, 0.02, 0.25, 0.42, 0.67, ORANGE)
ax.text(0.04, 0.88, "Abstract, Findings", fontsize=11.5, weight="bold", va="top")
ax.text(0.04, 0.80, "States: vaccinated Ct LOWER", fontsize=10.5, color=INK2, va="top")
ax.text(0.04, 0.72, "\u201c\u2026" + wrap(Q_ABS, 38) + "\u201d", fontsize=10, va="top",
linespacing=1.35)
ax.text(0.04, 0.30, "(The Abstract gives no Ct values.)", fontsize=10, color=INK2)
# right: Results
panel(ax, 0.56, 0.25, 0.42, 0.67, ORANGE)
ax.text(0.58, 0.88, "Results, Ct paragraph", fontsize=11.5, weight="bold", va="top")
ax.text(0.58, 0.80, "States: vaccinated Ct HIGHER", fontsize=10.5, color=INK2, va="top")
ax.text(0.58, 0.72, "\u201c" + wrap(Q_RES, 38) + "\u201d", fontsize=10, va="top",
linespacing=1.35)
ax.text(0.58, 0.60, "Mean Ct, Vx vs Pb:\nDENV-1 27.2 vs 22.2\nDENV-2 28.2 vs 25.3\n"
"p < 0.001 for each serotype", fontsize=10, va="top", linespacing=1.45)
ax.text(0.58, 0.395, f"Recomputed Vx \u2212 Pb:\n{d1:+.1f} cycles (DENV-1)\n"
f"{d2:+.1f} cycles (DENV-2)", fontsize=10, color=BLUE, weight="bold", va="top",
linespacing=1.35)
# connector
ax.annotate("", xy=(0.555, 0.61), xytext=(0.445, 0.61),
arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
ax.text(0.50, 0.66, "opposite\ndirection", ha="center", va="bottom", fontsize=10,
weight="bold")
ax.text(0.50, 0.56, "same 365\nsamples", ha="center", va="top", fontsize=10, color=INK2)
ax.text(0.02, 0.205, "Research in context (Added value), Discussion, Fig. 2a/2i and "
"Supplementary Table 6 also give the higher-Ct\ndirection. Results: Ct has "
"\u201can inverse relationship to the input target nucleic acid\u201d, so higher "
"Ct = less viral RNA.", fontsize=10, color=INK2, va="top", linespacing=1.4)
legend_text_kinds(ax, y=-0.01)
save(fig, path)
# ------------------------------------------------------------------ design B: four-location tally
def design_b(path):
rows = [
("Abstract, Findings", "“…significantly lower RT-qPCR Ct values in the vaccinated group…”", "LOWER", ORANGE),
("Research in context,\nAdded value", "“…higher RT-qPCR Ct values in the vaccinated group compared to the placebo group…”", "HIGHER", ORANGE),
("Results, Ct paragraph", "“" + Q_RES + "”", "HIGHER", ORANGE),
("Discussion, para. 4", "“" + Q_DIS + "…”", "HIGHER", ORANGE),
("Paper's means,\nrecomputed", RECOMP, "HIGHER", BLUE),
]
fig = plt.figure(figsize=(W, 4.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.axis("off")
ax.text(0.02, 0.965, "Where the paper states the direction", fontsize=10.5, color=INK2, va="top")
ax.text(0.97, 0.965, "Vaccinated Ct vs placebo", fontsize=10.5, color=INK2, va="top", ha="right")
top, hgt = 0.89, 0.148
for i, (loc, txt, dirn, col) in enumerate(rows):
y = top - i * hgt
is_abs = i == 0
panel(ax, 0.02, y - hgt + 0.02, 0.96, hgt - 0.025, col, lw=2.6 if is_abs else 1.4)
ax.text(0.04, y - hgt / 2 + 0.007, loc, fontsize=10.5, weight="bold", va="center",
linespacing=1.2)
ax.text(0.305, y - hgt / 2 + 0.007, wrap(txt, 46), fontsize=10, va="center",
color=BLUE if col == BLUE else INK, linespacing=1.3)
arrow = "▼" if dirn == "LOWER" else "▲"
ax.text(0.955, y - hgt / 2 + 0.007, f"{arrow} {dirn}", fontsize=11, weight="bold",
ha="right", va="center")
ax.text(0.02, 0.135, "Only the Abstract gives “lower”; every other location, "
"and the numbers, give “higher” (higher Ct = less viral RNA).",
fontsize=10, color=INK2, va="top")
legend_text_kinds(ax, y=-0.005)
save(fig, path)
# ------------------------------------------------------------------ design C: means +/- SD on the Ct scale
def design_c(path):
fig, ax = plt.subplots(figsize=(W, 4.2))
xs = {"DENV-1": 0, "DENV-2": 1}
off = 0.16
for s, r in REP.items():
x0 = xs[s]
for k, dx, mk in (("pb", -off, "o"), ("vx", off, "s")):
m, sd = r[k]
ax.errorbar(x0 + dx, m, yerr=sd, fmt=mk, ms=9, color=ORANGE, mfc=ORANGE,
mec=BG, mew=1.5, elinewidth=2, capsize=0, zorder=3)
ax.text(x0 + dx + 0.05, m, f"{'Pb' if k == 'pb' else 'Vx'} {m}", va="center",
fontsize=10, color=INK)
top = r["vx"][0] + r["vx"][1] + 1.2
ax.plot([x0 - off, x0 - off, x0 + off, x0 + off],
[r["pb"][0] + r["pb"][1] + 0.6, top, top, r["vx"][0] + r["vx"][1] + 0.6],
color=BLUE, lw=2)
ax.text(x0, top + 0.5, f"Vx − Pb = {R[s]['diff']:+.1f}", ha="center",
va="bottom", color=BLUE, weight="bold", fontsize=10.5)
ax.set_xticks([0, 1], ["DENV-1 (n = 161)", "DENV-2 (n = 205)"], fontsize=10.5, color=INK)
ax.set_xlim(-0.6, 1.6)
ax.set_ylim(0, 40)
ax.set_ylabel("RT-qPCR Ct (mean ± SD)\nhigher Ct = less viral RNA")
ax.yaxis.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right"):
ax.spines[sp].set_visible(False)
ax.text(-0.55, 4.2, "Abstract: “significantly lower RT-qPCR Ct values in the "
"vaccinated group”\n— the reported means show the opposite",
fontsize=10, color=INK, va="center")
h = [Line2D([], [], marker="o", ls="none", color=ORANGE, ms=9, label="Placebo (Pb), reported"),
Line2D([], [], marker="s", ls="none", color=ORANGE, ms=9, label="Vaccinated (Vx), reported"),
Line2D([], [], color=BLUE, lw=2, label="Difference recomputed")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.1), ncol=3,
frameon=False, fontsize=10)
save(fig, path)
# ------------------------------------------------------------------ design D: difference with CI, null at 0
def design_d(path):
fig, ax = plt.subplots(figsize=(W, 3.3))
ys = {"DENV-1": 1, "DENV-2": 0}
ax.axvspan(-7, 0, color="#f1f0ec", zorder=0)
ax.axvline(0, color=INK2, lw=1, zorder=1)
ax.text(-3.5, 1.62, "Direction stated in the Abstract:\nvaccinated Ct lower",
ha="center", va="top", fontsize=10, color=INK2)
ax.text(3.5, 1.62, "Direction in Results, Added value,\nDiscussion: vaccinated Ct higher",
ha="center", va="top", fontsize=10, color=INK2)
for s in REP:
y = ys[s]
lo, hi = R[s]["ci"]
ax.plot([lo, hi], [y - 0.1] * 2, color=BLUE, lw=2, zorder=2)
ax.plot(R[s]["diff"], y - 0.1, "o", color=BLUE, ms=9, mec=BG, mew=1.5, zorder=3)
ax.text(hi + 0.15, y - 0.1, f"{R[s]['diff']:+.1f} ({lo:+.1f} to {hi:+.1f})",
va="center", fontsize=10, color=INK)
ax.plot(S6[s]["delta"], y + 0.12, "D", color=ORANGE, ms=8, mec=BG, mew=1.5, zorder=3)
ax.text(S6[s]["delta"] + 0.25, y + 0.12, f"{S6[s]['delta']:+.2f}", va="center",
fontsize=10, color=INK)
ax.set_yticks([1, 0], ["DENV-1\n(Vx 48, Pb 113)", "DENV-2\n(Vx 83, Pb 122)"],
fontsize=10.5, color=INK)
ax.set_xlim(-7, 7)
ax.set_ylim(-0.5, 1.7)
ax.set_xlabel("Vaccinated − placebo mean RT-qPCR Ct (cycles)")
ax.xaxis.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
h = [Line2D([], [], marker="D", ls="none", color=ORANGE, ms=8,
label="Reported difference (Suppl. Table 6)"),
Line2D([], [], marker="o", color=BLUE, lw=2, ms=9,
label="Recomputed: Results means, Welch 95% CI")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.22), ncol=2,
frameon=False, fontsize=10)
save(fig, path)
# ------------------------------------------------------------------ design E: logic chain
def design_e(path):
fig = plt.figure(figsize=(W, 3.8))
ax = fig.add_axes([0, 0, 1, 1])
ax.axis("off")
xs = [0.02, 0.355, 0.69]
bw = 0.29
def row(y, label, boxes, cols, links):
ax.text(0.02, y + 0.255, label, fontsize=11, weight="bold", va="bottom")
for x, txt, c in zip(xs, boxes, cols):
panel(ax, x, y, bw, 0.24, c)
ax.text(x + 0.012, y + 0.12, txt, fontsize=10, va="center", linespacing=1.3,
color=BLUE if c == BLUE else INK)
for i, (sym, ok) in enumerate(links):
xa, xb = xs[i] + bw, xs[i + 1]
ax.annotate("", xy=(xb - 0.004, y + 0.12), xytext=(xa + 0.004, y + 0.12),
arrowprops=dict(arrowstyle="->", color=INK, lw=2))
ax.text((xa + xb) / 2, y + 0.155, sym, ha="center", va="bottom", fontsize=12,
weight="bold")
row(0.60, "Abstract, Findings",
["Premise:\n“significantly lower\nRT-qPCR Ct values in\nthe vaccinated group”",
"Paper's definition\n(Results): Ct inverse to\nnucleic acid, so lower\nCt = more viral RNA",
"Conclusion:\n“the vaccine could be\nreducing viral\nreplication”"],
[ORANGE, ORANGE, ORANGE], [("", True), ("≠", False)])
ax.text(0.6675, 0.585, "does not follow", ha="center", va="top", fontsize=10,
color=INK2)
row(0.16, "Results, Added value, Discussion; recomputed from the Results means",
["Recomputed Vx \u2212 Pb\nmean Ct: " + f"{d1:+.1f} (DENV-1),\n{d2:+.1f} (DENV-2), so\nvaccinated Ct higher",
"Same definition:\nhigher Ct =\nless viral RNA",
"Same conclusion:\nsuppression of viral\nreplication"],
[BLUE, ORANGE, ORANGE], [("", True), ("=", True)])
ax.text(0.6675, 0.145, "follows", ha="center", va="top", fontsize=10, color=INK2)
legend_text_kinds(ax, y=-0.05)
save(fig, path)
design_a(os.path.join(CAND, "cand_A_text_panels"))
design_b(os.path.join(CAND, "cand_B_location_tally"))
design_c(os.path.join(CAND, "cand_C_means_sd"))
design_d(os.path.join(CAND, "cand_D_difference_ci"))
design_e(os.path.join(CAND, "cand_E_logic_chain"))
# provisional choice for writeup.md, pending review of the candidates
design_a(os.path.join(HERE, "fig1"))
design_d(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE)
Abstract reverses the direction of the cerebellar IDH3A protein difference reported in Table 3B
Mol Psychiatry (2016) · doi:10.1038/mp.2015.217 · PMC5078854
Summary
The abstract states that protein expression of IDH3A in the cerebellum of patients with bipolar disorder (BD) "was higher than that of controls". The paper's own Table 3B gives the opposite: 0.804 ± 0.390 in BD against 1.227 ± 0.543 in controls (n = 15 per group). That is 0.423 units, or 34.5%, lower in BD, and the Results and Discussion both describe it as lower. Recomputing from the table's summary statistics reproduces its reported tests to rounding (ANOVA F(3, 56) = 5.73 against the printed 5.74, P = 0.0017; Dunnett P = 0.0172 for BD vs controls, printed 0.017), so the table and Results are internally consistent and only the abstract's direction word is wrong. The abstract is the most widely read and indexed part of the paper, and as printed it makes the protein result appear to contradict the lower IDH3A and IDH3B mRNA reported in the preceding sentence of the same abstract.
What the paper reports
Abstract:
Moreover, protein expression of IDH3A in the cerebellum from BD patients was higher than that of controls.
Abstract, preceding sentence (the mRNA result the protein finding is presented alongside):
Furthermore, gene expression of two subtypes (IDH3A and IDH3B) of isocitrate dehydrogenase (IDH) in the dorsolateral prefrontal cortex from BD patients was significantly lower than that of controls
Results, "Protein expression of IDH3A and IDH3B in the cerebellum and parietal cortex":
Results showed that protein levels of IDH3A in the cerebellum from BD (P=0.017), major depressive disorder (P=0.006) and schizophrenia (P=0.001) were significantly lower than in controls (Table 3B).
Table 3B, cerebellum row (columns: Controls n=15 | BD n=15 | MDD n=15 | Schizophrenia n=15 | One-way ANOVA):
IDH3A | 1.227±0.543 | 0.804±0.390* | 0.746±0.356** | 0.660±0.303** | F(3, 56)=5.74, P=0.002
Table 3B footer:
The data are the mean±s.d.*P<0.05, **P<0.01 (vs control, Dunnett's test).
Discussion, first paragraph:
Furthermore, protein levels of IDH3A in the cerebellar tissue from BD patients were lower than those in controls.
Discussion, second paragraph:
We did find reduced expression of IDH3A protein in the cerebellum in BD and other psychiatric disorders such as major depressive disorder and schizophrenia.
Taken all together, it is likely that decreased activity of IDH3A in the brain may play a crucial role in the pathogenesis of BD
The conclusions that draw on the finding. Abstract:
These findings suggest that abnormality in the metabolism of isocitrate by IDH3A in the mitochondria plays a key role in the pathogenesis of BD, supporting the mitochondrial dysfunction hypothesis of BD. Therefore, IDH3 in the citric acid cycle could potentially be a novel therapeutic target for BD.
Conclusions:
Our study provides evidence for abnormality in the metabolism of isocitrate by IDH3A in the pathogenesis of BD.
What the paper's own numbers show
Method. I took the group means and standard deviations printed in Table 3B (n = 15 per group, four groups) and Table 3A (dorsolateral prefrontal cortex, DLPFC, mRNA; controls n = 34, BD n = 35). For each Table 3B row I rebuilt the one-way ANOVA from the summary statistics (between-group and pooled within-group sums of squares, df 3 and 56). For the cerebellar IDH3A row I computed each patient group's difference from controls, the pooled-variance t statistic on 56 df, and the exact two-sided Dunnett P for three comparisons against a common control with equal n (correlation 0.5), by numerical integration rather than simulation. For all seven rows of Table 3A I ran Student and Welch two-sample t tests from the summary statistics, and found the range of P values each test can give when every printed mean and SD is allowed to be off by up to 0.0005 (their 3-decimal rounding). I also applied the Benjamini–Hochberg adjustment over the seven genes to the printed P values, to check the table's false discovery rate (FDR) column. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Direction of cerebellar IDH3A protein, BD vs controls | Abstract | "higher than that of controls" | lower: BD − controls = 0.804 − 1.227 = −0.423 (−34.5%; ratio 0.655) |
| Direction of cerebellar IDH3A protein, BD vs controls | Results; Discussion ¶1 and ¶2 | "significantly lower"; "lower"; "reduced" | lower (agrees with Table 3B) |
| Cerebellar IDH3A protein, controls vs BD | Table 3B | 1.227 ± 0.543 vs 0.804 ± 0.390* | as printed; pooled t(56) = −2.84 |
| One-way ANOVA, cerebellar IDH3A | Table 3B | F(3, 56) = 5.74, P = 0.002 | F(3, 56) = 5.73, P = 0.0017 (rounding of the printed means and SDs allows F = 5.70 to 5.77) |
| Dunnett, BD vs controls, cerebellar IDH3A | Results; Table 3B (*) | P = 0.017 | t = −2.840, P = 0.0172 |
| Dunnett, MDD vs controls, cerebellar IDH3A | Results; Table 3B (**) | P = 0.006 | t = −3.229, P = 0.0058 |
| Dunnett, schizophrenia vs controls, cerebellar IDH3A | Results; Table 3B (**) | P = 0.001 | t = −3.807, P = 0.0010 |
| One-way ANOVA, cerebellar IDH3B | Table 3B | F(3, 56) = 2.30, P = 0.087 | F = 2.30, P = 0.0869; BD − controls = −0.181 |
| One-way ANOVA, parietal IDH3A | Table 3B | F(3, 56) = 2.58, P = 0.063 | F = 2.58, P = 0.0628; BD − controls = −0.279 |
| One-way ANOVA, parietal IDH3B | Table 3B | F(3, 56) = 0.63, P = 0.597 | F = 0.63, P = 0.5959; BD − controls = −0.117 |
| DLPFC IDH3A mRNA, controls vs BD | Table 3A; Abstract ("lower") | 1.466 ± 0.832 vs 1.017 ± 0.416, P = 0.007 | BD − controls = −0.449 (−30.6%); Student P = 0.0058, Welch P = 0.0069 |
| DLPFC IDH3B mRNA, controls vs BD | Table 3A; Abstract ("lower") | 1.198 ± 0.420 vs 0.957 ± 0.287, P = 0.007 | BD − controls = −0.241 (−20.1%); Student P = 0.0069, Welch P = 0.0074 |
| DLPFC mRNA, other Table 3A genes (unpaired t-test) | Table 3A | ACO1 0.178; ACO2 0.487; IDH1 0.755; IDH2 0.279; IDH3G 0.422 | Student P: ACO1 0.175; ACO2 0.478; IDH1 0.758; IDH2 0.277; IDH3G 0.419. Each printed P lies inside the range that 3-decimal rounding of the means and SDs allows (for example ACO2 0.459 to 0.497) |
| FDR column, Table 3A | Table 3A | 0.414; 0.568; 0.755; 0.488; 0.026; 0.026; 0.568 (ACO1 … IDH3G) | Benjamini–Hochberg of the printed P values: 0.415; 0.568; 0.755; 0.488; 0.025; 0.025; 0.568 (printed P ± 0.0005 gives 0.023 to 0.026 for IDH3A and IDH3B) |
| Cerebellar IDH3A protein, BD vs the other patient groups | Table 3B | — | BD − MDD = +0.058; BD − schizophrenia = +0.144 (BD is higher only relative to these groups, not to controls) |
All four one-way ANOVAs and the three Dunnett P values in Table 3B are reproduced to rounding from the printed summary statistics. In Table 3A, the printed t-test P values for IDH3A and IDH3B match the Welch test (0.0069 and 0.0074); the Student test gives 0.0058 for IDH3A, which does not round to the printed 0.007 even allowing for rounding of the inputs. For the other five genes the recomputed P values differ from the printed ones by up to 0.009 (ACO2: 0.478 against 0.487), but each printed value lies within the range that the 3-decimal rounding of the printed means and SDs allows, so these differences say nothing either way. The FDR column matches a Benjamini–Hochberg adjustment of the printed P values to rounding. None of this bears on the error: the only statement that disagrees with the numbers is the direction word in the abstract.
Figure
Figure 1. The two places in the paper that describe the same quantity: cerebellar IDH3A protein, BD vs controls, Stanley post-mortem samples, n = 15 per group. Left: the abstract sentence, with its direction word in orange (reported). Right: the Results sentence (black) and the Table 3B means (orange, reported), with the difference recomputed from those means in blue (BD − controls = −0.423, 34.5% lower). The connector marks the conflict. The Discussion, not shown in a panel, agrees with the Results. All text is quoted verbatim; all numbers come from Table 3B or recompute.py.txt.
Figure 2. BD/control ratio of means for every IDH3A and IDH3B measurement in Tables 3A and 3B, on a log scale with 1 (no difference) as the reference line. Points and intervals (blue) are recomputed from the printed means and standard deviations; intervals are approximate 95% confidence intervals for the ratio by the delta method on the log scale and are shown only to convey precision, not as the paper's own analysis. The right-hand labels give the direction the abstract states for the three measurements it mentions; only the cerebellar IDH3A protein label (orange) conflicts with its point, which lies at 0.655 (95% CI 0.470 to 0.914). The figure shows that, read correctly, the protein result points the same way as the two mRNA results the abstract cites beside it.
Why it matters
- The error sits in the abstract. Abstracts are what indexing services, citation screens and most readers see. A reader of the abstract alone is told that IDH3A protein goes up in BD while IDH3A and IDH3B mRNA go down, which reads as mixed or contradictory evidence. The body of the paper reports a consistent deficit.
- It reverses the direction of a supporting result, not its size or significance. The size (−34.5%) and the significance (Dunnett P = 0.017) are correct in the body. The paper's headline findings (higher CSF isocitrate in BD; lower IDH3A/IDH3B mRNA in DLPFC) do not depend on this sentence, and the corrected direction fits the paper's argument better than the printed one.
- Direction matters for the paper's stated implications. The Discussion's inference that "decreased activity of IDH3A in the brain may play a crucial role", and the suggestion that IDH3 is "a novel therapeutic target", both rest on a deficit. An abstract that reports an excess points any reader who uses it, for example in a review or meta-analysis of post-mortem mitochondrial markers, the wrong way.
- Scope. The finding is not specific to BD: the same comparison shows lower cerebellar IDH3A in major depressive disorder and schizophrenia too (−39.2% and −46.2%), as the Results and Discussion say. That is a matter of interpretation and is not affected by the error.
Explanations considered
- A different tissue, measure or comparison. The abstract sentence names the same quantity as Table 3B and the Results: IDH3A protein, cerebellum, BD vs controls. No other cerebellar IDH3A measurement appears in the paper. Does not account for it.
- A comparison with the other patient groups rather than with controls. BD is higher than MDD (+0.058) and schizophrenia (+0.144) in Table 3B, but the abstract explicitly says "than that of controls", and these differences were not tested or reported. Does not account for it.
- An inverted ratio or normalization convention (for example, values expressed as control/BD). The table, the Results and the Discussion all read the same values as lower in BD, so the paper itself uses the convention under which BD is lower. Does not account for it.
- A text-conversion artifact in the archived copy. The publisher's version of record, the PMC page and the Europe PMC XML all carry the word "higher" in the abstract and the same Table 3B values. Does not account for it.
- A supplement that changes the result. The Supplementary Information describes the western blot and qPCR methods and gives no direction of effect; Supplementary Table S4 holds sample demographics only. Nothing in the supplements could make "higher" correct, since the main text and Table 3B give the direction plainly. Does not account for it.
- An error in the table rather than the abstract. The Results, both Discussion paragraphs and the significance markers all agree with the table, and the table's ANOVA and Dunnett P values are reproduced to rounding from its own means and SDs. The table is internally consistent; the abstract is the outlier.
- A published correction. None was found (see Status).
Status
As of 2026-10-23:
- Version of record checked. The publisher's article page (reached from doi.org) carries the same abstract sentence ("was higher than that of controls"), the same Results and Discussion wording ("significantly lower", "lower"), and the same Table 3B cerebellum IDH3A row and Dunnett footnote. The PMC and Europe PMC full texts match. The discrepancy is in the published article, not introduced by conversion.
- Supplements checked. Supplementary Information (DOC) and Supplementary Table S4 were examined; they report methods and demographics and do not resolve the conflict.
- Corrections. No correction, erratum or retraction was found: Crossref has no update relations (its relation field lists only an F1000 recommendation), and PMC and Europe PMC list no correction notices. Europe PMC links one "Comment in" item (PMID 29155801, Mol Psychiatry 2018;23:792–793, doi:10.1038/mp.2017.149), a letter rather than a correction; according to the version-of-record check it describes the paper as reporting reduced cerebellar IDH3A protein, which matches the body of the paper.
- PubPeer. The PubPeer API could not be queried (it required human verification), so PubPeer was not checked directly. Web searches found no PubPeer thread and no Retraction Watch coverage.
- Sources.
src/show.py PMC5078854(PMC archived text, tables and abstract);src/known.py 10.1038/mp.2015.217(Crossref, PMC and Europe PMC notices: none); https://doi.org/10.1038/mp.2015.217 and the publisher's Table 3B page; Europe PMC full-text XML; the Supplementary Information file.
Suggested correction
In the abstract, replace
Moreover, protein expression of IDH3A in the cerebellum from BD patients was higher than that of controls.
with
Moreover, protein expression of IDH3A in the cerebellum from BD patients was lower than that of controls.
No change is needed to Table 3B, the Results, the Discussion or the Conclusions, which already report the direction correctly. Optionally, the corrected sentence could add the effect and test (0.804 ± 0.390 vs 1.227 ± 0.543; Dunnett P = 0.017) and note that the reduction was also seen in major depressive disorder and schizophrenia, as the Results state.
Recomputation script (python recompute.py.txt)
# Recomputation for PMC5078854 (Mol Psychiatry 2016, doi:10.1038/mp.2015.217)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1/fig2
# (SVG + PNG at 200 dpi) in this file's own folder. Inputs are the summary
# statistics printed in Table 3A and Table 3B of the paper; nothing else is
# assumed.
# python recompute.py.txt --candidates DIR
# additionally renders the five candidate figure designs (A to E) into DIR, for
# choosing a design; they are not part of the write-up.
import os
import sys
import itertools
import numpy as np
from scipy import stats, integrate
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = sys.argv[sys.argv.index("--candidates") + 1] if "--candidates" in sys.argv else None
# ----------------------------------------------------------------------------
# 1. Inputs from the paper
# ----------------------------------------------------------------------------
GROUPS = ["Controls", "BD", "MDD", "Schizophrenia"]
N_PER_GROUP = 15
TABLE3B = { # (mean, s.d.) per group, n=15 each; reported one-way ANOVA (F, P)
"Cerebellum IDH3A": ([(1.227, 0.543), (0.804, 0.390), (0.746, 0.356), (0.660, 0.303)], (5.74, 0.002)),
"Cerebellum IDH3B": ([(1.020, 0.883), (0.839, 0.373), (0.605, 0.243), (0.610, 0.255)], (2.30, 0.087)),
"Parietal IDH3A": ([(0.951, 0.527), (0.672, 0.310), (0.648, 0.435), (0.564, 0.307)], (2.58, 0.063)),
"Parietal IDH3B": ([(0.738, 0.311), (0.621, 0.330), (0.760, 0.560), (0.602, 0.300)], (0.63, 0.597)),
}
DUNNETT_REPORTED = {"BD": 0.017, "MDD": 0.006, "Schizophrenia": 0.001} # Results text
TABLE3A = { # DLPFC mRNA: control mean, sd, n, BD mean, sd, n, reported unpaired t-test P
"DLPFC IDH3A mRNA": (1.466, 0.832, 34, 1.017, 0.416, 35, 0.007),
"DLPFC IDH3B mRNA": (1.198, 0.420, 34, 0.957, 0.287, 35, 0.007),
}
# All seven rows of Table 3A: (control mean, sd, BD mean, sd, reported t-test P, reported FDR P)
TABLE3A_ALL = {
"ACO1": (1.083, 0.205, 1.171, 0.315, 0.178, 0.414),
"ACO2": (0.922, 0.149, 0.897, 0.142, 0.487, 0.568),
"IDH1": (1.208, 0.337, 1.233, 0.333, 0.755, 0.755),
"IDH2": (1.316, 0.544, 1.186, 0.437, 0.279, 0.488),
"IDH3A": (1.466, 0.832, 1.017, 0.416, 0.007, 0.026),
"IDH3B": (1.198, 0.420, 0.957, 0.287, 0.007, 0.026),
"IDH3G": (1.084, 0.174, 1.049, 0.183, 0.422, 0.568),
}
def dunnett_two_sided_p(t, k, df):
"""Exact two-sided Dunnett P for k treatment-vs-control contrasts with equal n
(correlation 0.5). With U0, Ui iid N(0,1) and S^2 ~ chi2_df/df, the contrast
statistics are (Ui-U0)/(sqrt2*S); conditioning on U0=u and S=s makes them
independent. P(max|T| < c) = E[ prod_i (Phi(u + sqrt2*c*s) - Phi(u - sqrt2*c*s)) ]."""
c = abs(t)
def inner(s):
f = lambda u: stats.norm.pdf(u) * (stats.norm.cdf(u + np.sqrt(2) * c * s)
- stats.norm.cdf(u - np.sqrt(2) * c * s)) ** k
return integrate.quad(f, -10, 10, epsabs=1e-12)[0]
# density of S = sqrt(chi2_df / df)
dens = lambda s: stats.chi.pdf(s * np.sqrt(df), df) * np.sqrt(df)
lo, hi = stats.chi.ppf([1e-12, 1 - 1e-12], df) / np.sqrt(df)
cover = integrate.quad(lambda s: dens(s) * inner(s), lo, hi, epsabs=1e-12, limit=200)[0]
return 1 - cover
print("=" * 78)
print("Table 3B: one-way ANOVA from summary statistics (n=15 per group, df 3, 56)")
print("=" * 78)
RES = {}
for name, (rows, (F_rep, P_rep)) in TABLE3B.items():
m = np.array([a for a, b in rows]); s = np.array([b for a, b in rows])
K = len(m); N = N_PER_GROUP * K
gm = m.mean()
ssb = N_PER_GROUP * ((m - gm) ** 2).sum()
ssw = ((N_PER_GROUP - 1) * s ** 2).sum()
msw = ssw / (N - K)
F = (ssb / (K - 1)) / msw
P = stats.f.sf(F, K - 1, N - K)
RES[name] = dict(m=m, s=s, msw=msw, F=F, P=P)
print(f"{name:18s} F(3,56) = {F:.2f}, P = {P:.4f} [reported F(3, 56)={F_rep}, P={P_rep}]")
Fs = []
for dm in itertools.product([-5e-4, 5e-4], repeat=K):
for ds in itertools.product([-5e-4, 5e-4], repeat=K):
mm = m + np.array(dm); ss = s + np.array(ds)
Fs.append((N_PER_GROUP * ((mm - mm.mean()) ** 2).sum() / (K - 1))
/ (((N_PER_GROUP - 1) * ss ** 2).sum() / (N - K)))
print(f" F range under 3-decimal rounding of means and SDs: {min(Fs):.3f} to {max(Fs):.3f}")
for i in range(1, K):
d = m[i] - m[0]
t = d / np.sqrt(msw * 2 / N_PER_GROUP)
p_unadj = 2 * stats.t.sf(abs(t), N - K)
print(f" {GROUPS[i]:13s} - Controls = {d:+.3f} ({100 * (m[i] / m[0] - 1):+.1f}%; "
f"ratio {m[i] / m[0]:.3f}), pooled t(56) = {t:+.2f}, unadjusted two-sided P = {p_unadj:.4f}")
print()
print("Cerebellar IDH3A: Dunnett two-sided P (3 comparisons vs controls, df 56)")
r = RES["Cerebellum IDH3A"]
DUN = {}
for i, g in enumerate(GROUPS[1:], start=1):
t = (r["m"][i] - r["m"][0]) / np.sqrt(r["msw"] * 2 / N_PER_GROUP)
p = dunnett_two_sided_p(t, 3, 56)
DUN[g] = (t, p)
print(f" {g:13s} t = {t:+.3f}, Dunnett P = {p:.4f} [reported P={DUNNETT_REPORTED[g]}]")
bd_ctrl = r["m"][1] - r["m"][0]
bd_ratio = r["m"][1] / r["m"][0]
print()
print(f"Direction, cerebellar IDH3A protein, BD vs controls: BD - controls = {bd_ctrl:+.3f} "
f"({100 * (bd_ratio - 1):+.1f}%), ratio {bd_ratio:.3f} -> BD is "
f"{'LOWER' if bd_ctrl < 0 else 'HIGHER'} than controls")
print(f"For contrast, BD vs the other patient groups: BD - MDD = {r['m'][1] - r['m'][2]:+.3f}, "
f"BD - schizophrenia = {r['m'][1] - r['m'][3]:+.3f}")
print()
print("=" * 78)
print("Table 3A: DLPFC mRNA, controls (n=34) vs BD (n=35)")
print("=" * 78)
MRNA = {}
for name, (m1, s1, n1, m2, s2, n2, prep) in TABLE3A.items():
st = stats.ttest_ind_from_stats(m1, s1, n1, m2, s2, n2)
we = stats.ttest_ind_from_stats(m1, s1, n1, m2, s2, n2, equal_var=False)
MRNA[name] = (m1, s1, n1, m2, s2, n2)
print(f"{name:18s} BD - controls = {m2 - m1:+.3f} ({100 * (m2 / m1 - 1):+.1f}%; ratio {m2 / m1:.3f}); "
f"Student P = {st.pvalue:.4f}, Welch P = {we.pvalue:.4f} [reported P={prep}]")
print()
print("All seven Table 3A rows (n=34 vs 35). P recomputed from the printed means/SDs;")
print("'range' = smallest to largest P obtainable when each printed mean and SD may be")
print("off by up to 0.0005 from 3-decimal rounding (extremes over the 16 corners);")
print("'fits' = the range overlaps the interval the printed 3-decimal P stands for.")
def p_t(m1, s1, m2, s2, welch):
return stats.ttest_ind_from_stats(m1, s1, 34, m2, s2, 35, equal_var=not welch).pvalue
def p_range(m1, s1, m2, s2, welch):
ps = [p_t(m1 + a, s1 + b, m2 + c, s2 + d, welch)
for a, b, c, d in itertools.product([-5e-4, 5e-4], repeat=4)]
return min(ps), max(ps)
P_REP = []
for g, (m1, s1, m2, s2, prep, frep) in TABLE3A_ALL.items():
ps, pw = p_t(m1, s1, m2, s2, False), p_t(m1, s1, m2, s2, True)
(sl, sh), (wl, wh) = p_range(m1, s1, m2, s2, False), p_range(m1, s1, m2, s2, True)
fits = lambda lo, hi: "fits" if (lo < prep + 5e-4 and prep - 5e-4 <= hi) else "does NOT fit"
P_REP.append(prep)
print(f" {g:6s} Student P = {ps:.4f} (range {sl:.4f}-{sh:.4f}, {fits(sl, sh)}); "
f"Welch P = {pw:.4f} (range {wl:.4f}-{wh:.4f}, {fits(wl, wh)}) [reported P={prep}]")
def bh(p):
p = np.asarray(p, float); n = len(p); o = np.argsort(p)
adj = np.minimum.accumulate((p[o] * n / np.arange(1, n + 1))[::-1])[::-1]
out = np.empty(n); out[o] = np.minimum(adj, 1); return out
print()
print("Table 3A FDR column: Benjamini-Hochberg over the 7 genes")
fdr_rep = bh(P_REP)
fdr_lo = bh(np.maximum(np.array(P_REP) - 5e-4, 0)); fdr_hi = bh(np.array(P_REP) + 5e-4)
for i, (g, row) in enumerate(TABLE3A_ALL.items()):
print(f" {g:6s} BH of printed P = {fdr_rep[i]:.3f} (printed P +/- 0.0005 gives {fdr_lo[i]:.4f} to "
f"{fdr_hi[i]:.4f}) [reported FDR P={row[5]}]")
# BD/control ratios with approximate 95% CIs (delta method on the log ratio),
# used only for the ratio figure (candidate D / fig2).
print()
print("BD/control ratios, approximate 95% CI (delta method on log ratio, normal approx.)")
RATIOS = []
def ratio_ci(m1, s1, n1, m2, s2, n2):
lr = np.log(m2 / m1)
se = np.sqrt(s1 ** 2 / (n1 * m1 ** 2) + s2 ** 2 / (n2 * m2 ** 2))
return m2 / m1, np.exp(lr - 1.96 * se), np.exp(lr + 1.96 * se)
for name, (m1, s1, n1, m2, s2, n2) in MRNA.items():
RATIOS.append((name, *ratio_ci(m1, s1, n1, m2, s2, n2)))
for name in ["Cerebellum IDH3A", "Cerebellum IDH3B", "Parietal IDH3A", "Parietal IDH3B"]:
m = RES[name]["m"]; s = RES[name]["s"]
RATIOS.append((name + " protein", *ratio_ci(m[0], s[0], 15, m[1], s[1], 15)))
for name, est, lo, hi in RATIOS:
print(f" {name:26s} ratio {est:.3f} (95% CI {lo:.3f} to {hi:.3f})")
# ----------------------------------------------------------------------------
# 2. Figures
# ----------------------------------------------------------------------------
BG = "#fcfcfb"; INK = "#0b0b0b"; INK2 = "#52514e"; ORANGE = "#eb6834"; BLUE = "#2a78d6"
GRID = "#e4e3df"; PANEL = "#f3f2ef"; EDGE = "#c9c7c1"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
W = 7.2
ABSTRACT_Q = ("Moreover, protein expression of IDH3A in the cerebellum from BD patients "
"was higher than that of controls.")
RESULTS_Q = ("Results showed that protein levels of IDH3A in the cerebellum from BD (P=0.017), "
"major depressive disorder (P=0.006) and schizophrenia (P=0.001) were "
"significantly lower than in controls (Table 3B).")
DISC_Q = ("Furthermore, protein levels of IDH3A in the cerebellar tissue from BD patients "
"were lower than those in controls.")
TABLE_ROW = "IDH3A | 1.227±0.543 | 0.804±0.390* | 0.746±0.356** | 0.660±0.303** | F(3, 56)=5.74, P=0.002"
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def rich_text(ax, x, y, runs, width, size=10.5, line=1.45):
"""Draw word-wrapped text made of (text, color, weight) runs in axes coords,
starting at the top-left (x, y). Returns the y of the next line."""
fig = ax.figure
rend = fig.canvas.get_renderer()
inv = ax.transAxes.inverted()
probe = ax.text(0, 0, "Ag", fontsize=size)
bb = probe.get_window_extent(rend); probe.remove()
lh = (inv.transform((0, bb.height)) - inv.transform((0, 0)))[1] * line
sp_t = ax.text(0, 0, "a a", fontsize=size); sp_a = ax.text(0, 0, "aa", fontsize=size)
sw = sp_t.get_window_extent(rend).width - sp_a.get_window_extent(rend).width
sp_t.remove(); sp_a.remove()
sw = 0.8 * (inv.transform((sw, 0)) - inv.transform((0, 0)))[0]
cx, cy = x, y
for text, color, weight in runs:
for word in text.split(" "):
if word == "":
continue
t = ax.text(cx, cy, word, color=color, fontweight=weight, fontsize=size,
va="top", ha="left", transform=ax.transAxes)
wb = t.get_window_extent(rend)
ww = (inv.transform((wb.width, 0)) - inv.transform((0, 0)))[0]
if cx + ww > x + width and cx > x:
cx, cy = x, cy - lh
t.set_position((cx, cy))
cx += ww + sw
return cy - lh
def panel(ax, x, y, w, h, fc=PANEL, ec=EDGE, lw=1.0):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
transform=ax.transAxes, fc=fc, ec=ec, lw=lw))
def tag(ax, x, y, text, color):
ax.text(x, y, text, transform=ax.transAxes, fontsize=10, color="white", fontweight="bold",
va="center", ha="left",
bbox=dict(boxstyle="round,pad=0.3", fc=color, ec="none"))
# ---- Candidate A (fig1): two text panels joined by a connector -------------
def design_A(path):
fig = plt.figure(figsize=(W, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.text(0.03, 0.955, "Same quantity, opposite direction:\ncerebellar IDH3A protein, BD vs controls (n=15 per group)",
fontsize=11, fontweight="bold", transform=ax.transAxes, va="top")
# left panel: abstract
panel(ax, 0.03, 0.30, 0.38, 0.55)
ax.text(0.05, 0.81, "Abstract", fontsize=11, fontweight="bold", transform=ax.transAxes, va="top")
rich_text(ax, 0.05, 0.73, [("“Moreover, protein expression of IDH3A in the cerebellum from BD patients was", INK, "normal"),
("higher", ORANGE, "bold"), ("than that of controls.”", INK, "normal")], 0.34)
tag(ax, 0.05, 0.36, "reported: BD higher", ORANGE)
# right panel: results + table
panel(ax, 0.52, 0.30, 0.45, 0.55)
ax.text(0.54, 0.81, "Results and Table 3B", fontsize=11, fontweight="bold", transform=ax.transAxes, va="top")
yy = rich_text(ax, 0.54, 0.73, [("“protein levels of IDH3A in the cerebellum from BD (P=0.017) [...] were significantly", INK, "normal"),
("lower", INK, "bold"), ("than in controls (Table 3B).”", INK, "normal")], 0.41)
ax.text(0.54, yy - 0.015, "Table 3B, cerebellum IDH3A (mean ± s.d.):", fontsize=10, color=INK2,
transform=ax.transAxes, va="top")
ax.text(0.54, yy - 0.075, "controls 1.227 ± 0.543", fontsize=10, color=ORANGE, fontweight="normal",
transform=ax.transAxes, va="top")
ax.text(0.775, yy - 0.075, "BD 0.804 ± 0.390*", fontsize=10, color=ORANGE, fontweight="normal",
transform=ax.transAxes, va="top")
tag(ax, 0.54, 0.36, f"recomputed: BD − controls = {bd_ctrl:+.3f}", BLUE)
# connector
arr = FancyArrowPatch((0.415, 0.575), (0.515, 0.575), transform=ax.transAxes,
arrowstyle="<|-|>", mutation_scale=14, lw=2, color=INK2)
ax.add_patch(arr)
ax.text(0.465, 0.61, "conflict", ha="center", fontsize=10, color=INK2, transform=ax.transAxes,
fontweight="bold")
# footer
rich_text(ax, 0.03, 0.22, [("The Discussion also states the protein levels “were lower than those in controls”. "
f"Recomputed from Table 3B: BD is {100 * (1 - bd_ratio):.1f}% below controls, pooled t(56) = {DUN['BD'][0]:.2f}, Dunnett P = {DUN['BD'][1]:.3f} "
"(reported P=0.017). Only the abstract gives the direction as higher.", INK2, "normal")],
0.94, size=10)
save(fig, path)
# ---- Candidate B: statement map across the paper ---------------------------
def design_B(path):
rows = [
("Abstract", "“...was higher than that of controls.”", "higher", ORANGE),
("Results", "“...BD (P=0.017) ... significantly lower than in controls”", "lower", ORANGE),
("Table 3B", "controls 1.227±0.543 BD 0.804±0.390*", "lower", ORANGE),
("Discussion ¶1", "“...from BD patients were lower than those in controls.”", "lower", ORANGE),
("Discussion ¶2", "“We did find reduced expression of IDH3A protein ...”", "lower", ORANGE),
("Recomputed", f"BD − controls = {bd_ctrl:+.3f} ({100 * (bd_ratio - 1):+.1f}%), Dunnett P = {DUN['BD'][1]:.3f}", "lower", BLUE),
]
fig = plt.figure(figsize=(W, 4.3))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.text(0.03, 0.955, "Direction of cerebellar IDH3A protein, BD vs controls, in each part of the paper",
fontsize=11, fontweight="bold", transform=ax.transAxes, va="top")
ax.text(0.03, 0.885, "orange = stated in the paper; blue = recomputed from Table 3B", fontsize=10,
color=INK2, transform=ax.transAxes, va="top")
top = 0.80; step = 0.118
for i, (loc, txt, d, col) in enumerate(rows):
y = top - i * step
if loc == "Abstract":
panel(ax, 0.02, y - 0.05, 0.96, 0.10, fc="#fbe9e1", ec=ORANGE, lw=1.5)
elif loc == "Recomputed":
ax.plot([0.03, 0.97], [y + 0.062, y + 0.062], color=EDGE, lw=1, transform=ax.transAxes)
ax.text(0.04, y, loc, fontsize=10.5, fontweight="bold", va="center", transform=ax.transAxes)
ax.text(0.20, y, txt, fontsize=10, va="center", transform=ax.transAxes, color=INK)
mk = "^" if d == "higher" else "v"
ax.plot([0.85], [y], marker=mk, ms=11, color=col, transform=ax.transAxes, ls="none")
ax.text(0.875, y, d, fontsize=10.5, va="center", transform=ax.transAxes, color=INK, fontweight="bold")
ax.text(0.04, 0.06, "The abstract is the only place that states the direction as higher.", fontsize=10,
color=INK2, transform=ax.transAxes)
save(fig, path)
# ---- Candidate C: group means ± s.d. with the abstract claim annotated ------
def design_C(path):
m = RES["Cerebellum IDH3A"]["m"]; s = RES["Cerebellum IDH3A"]["s"]
fig, ax = plt.subplots(figsize=(W, 4.4))
fig.subplots_adjust(left=0.11, right=0.97, top=0.86, bottom=0.12)
x = np.arange(4)
ax.errorbar(x, m, yerr=s, fmt="o", ms=9, color=ORANGE, ecolor=ORANGE, elinewidth=2, capsize=0,
label="Table 3B: mean ± s.d. (reported)", zorder=3)
ax.axhline(m[0], color=INK2, lw=1, ls=(0, (4, 3)), zorder=1)
ax.text(3.45, m[0] + 0.03, "control mean", fontsize=10, color=INK2, ha="right")
# recomputed difference bracket
ax.annotate("", xy=(1.18, m[1]), xytext=(1.18, m[0]),
arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=2, mutation_scale=14))
ax.text(1.25, (m[0] + m[1]) / 2, f"recomputed:\n{bd_ctrl:+.3f} ({100 * (bd_ratio - 1):+.1f}%)\n"
f"Dunnett P = {DUN['BD'][1]:.3f}", color=INK, fontsize=10, va="center")
ax.plot([], [], color=BLUE, lw=2, label="BD − controls, recomputed")
# abstract claim
ax.annotate("", xy=(0.82, m[0] + 0.42), xytext=(0.82, m[0] + 0.05),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, ls="--", mutation_scale=14))
ax.text(0.90, m[0] + 0.30, "abstract (reported):\nBD “higher than\nthat of controls”", color=INK,
fontsize=10, ha="left", va="center")
ax.set_xticks(x, ["Controls", "BD", "MDD", "Schizophrenia"])
ax.set_xlim(-0.5, 3.5); ax.set_ylim(0, 2.0)
ax.set_ylabel("IDH3A protein, cerebellum\n(relative expression)")
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right"]:
ax.spines[sp].set_visible(False)
ax.legend(loc="upper right", bbox_to_anchor=(1.0, 1.02), frameon=False, fontsize=10)
fig.suptitle("Cerebellar IDH3A protein is lower in BD than in controls (n=15 per group)",
x=0.03, ha="left", fontsize=11, fontweight="bold")
save(fig, path)
# ---- Candidate D (fig2): BD/control ratios on a log axis --------------------
def design_D(path):
order = ["DLPFC IDH3A mRNA", "DLPFC IDH3B mRNA", "Cerebellum IDH3A protein",
"Cerebellum IDH3B protein", "Parietal IDH3A protein", "Parietal IDH3B protein"]
abstract_note = {"DLPFC IDH3A mRNA": "abstract: lower", "DLPFC IDH3B mRNA": "abstract: lower",
"Cerebellum IDH3A protein": "abstract: higher"}
d = {n: (e, l, h) for n, e, l, h in RATIOS}
fig, ax = plt.subplots(figsize=(W, 4.3))
fig.subplots_adjust(left=0.30, right=0.76, top=0.80, bottom=0.14)
ys = np.arange(len(order))[::-1]
ax.axvline(1, color=INK2, lw=1)
for y, n in zip(ys, order):
e, l, h = d[n]
ax.plot([l, h], [y, y], color=BLUE, lw=2, solid_capstyle="round")
ax.plot([e], [y], "o", ms=9, color=BLUE, mec=BG, mew=1.5, zorder=3)
if n in abstract_note:
note = abstract_note[n]
ax.text(1.62, y, note, va="center", fontsize=10,
color=ORANGE if "higher" in note else INK2, fontweight="bold" if "higher" in note else "normal",
transform=ax.get_yaxis_transform() if False else ax.transData, clip_on=False)
ax.set_xscale("log")
ticks = [0.4, 0.5, 0.6, 0.8, 1.0, 1.25, 1.5]
ax.set_xticks(ticks, [f"{t:g}" for t in ticks]); ax.minorticks_off()
ax.set_xlim(0.38, 1.55)
ax.set_yticks(ys, order)
ax.set_xlabel("BD / control ratio of means (log scale); 1 = no difference")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.text(0.40, len(order) - 0.35, "← lower in BD", fontsize=10, color=INK2)
ax.text(1.04, len(order) - 0.35, "higher in BD →", fontsize=10, color=INK2)
ax.plot([], [], "o-", color=BLUE, lw=2, ms=8, label="recomputed from\nTables 3A/3B\n(approx. 95% CI)")
ax.legend(loc="lower left", bbox_to_anchor=(1.0, -0.02), frameon=False, fontsize=10,
handlelength=1.6, borderaxespad=0)
fig.suptitle("BD/control ratios for IDH3A and IDH3B, Tables 3A and 3B:\nall point estimates below 1", x=0.03, ha="left", y=0.985,
fontsize=11, fontweight="bold")
save(fig, path)
# ---- Candidate E: printed vs supported sentence + number line --------------
def design_E(path):
m = RES["Cerebellum IDH3A"]["m"]; s = RES["Cerebellum IDH3A"]["s"]
fig = plt.figure(figsize=(W, 4.3))
ax = fig.add_axes([0, 0.50, 1, 0.50]); ax.axis("off")
ax.text(0.03, 0.92, "The abstract sentence as printed, and as Table 3B supports it",
fontsize=11, fontweight="bold", transform=ax.transAxes, va="top")
ax.text(0.03, 0.66, "Printed", fontsize=10.5, color=INK2, transform=ax.transAxes, va="top")
rich_text(ax, 0.17, 0.66, [("“protein expression of IDH3A in the cerebellum from BD patients was", INK, "normal"),
("higher", ORANGE, "bold"), ("than that of controls.”", INK, "normal")], 0.80)
ax.text(0.03, 0.28, "Supported", fontsize=10.5, color=INK2, transform=ax.transAxes, va="top")
rich_text(ax, 0.17, 0.28, [("protein expression of IDH3A in the cerebellum from BD patients was", INK, "normal"),
("lower", BLUE, "bold"), ("than that of controls.", INK, "normal")], 0.80)
bx = fig.add_axes([0.17, 0.20, 0.55, 0.25])
for y, (mm, ss, lab) in zip([2, 0], [(m[0], s[0], "Controls"), (m[1], s[1], "BD")]):
bx.plot([mm - ss, mm + ss], [y, y], color=ORANGE, lw=2)
bx.plot([mm], [y], "o", ms=9, color=ORANGE)
bx.text(1.02, y, f"{lab} {mm:.3f} ± {ss:.3f}", va="center", fontsize=10,
transform=bx.get_yaxis_transform(), clip_on=False)
bx.annotate("", xy=(m[1], 1), xytext=(m[0], 1),
arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=2, mutation_scale=14))
bx.text(m[0] + 0.04, 1, f"recomputed {bd_ctrl:+.3f} ({100 * (bd_ratio - 1):+.1f}%)", va="center",
fontsize=10, color=INK)
bx.set_ylim(-0.6, 2.6); bx.set_xlim(0, 2.0); bx.set_yticks([])
bx.set_xlabel("Table 3B cerebellar IDH3A protein,\nmean ± s.d. (reported), n=15 per group", fontsize=10)
bx.grid(axis="x", color=GRID, lw=0.8); bx.set_axisbelow(True)
for sp in ["top", "right", "left"]:
bx.spines[sp].set_visible(False)
save(fig, path)
design_A(os.path.join(HERE, "fig1"))
design_D(os.path.join(HERE, "fig2"))
print()
print("Figures written: fig1.svg/png (design A), fig2.svg/png (design D)")
if CAND:
os.makedirs(CAND, exist_ok=True)
for nm, fn in zip("ABCDE", [design_A, design_B, design_C, design_D, design_E]):
fn(os.path.join(CAND, f"candidate_{nm}"))
print(f"Candidate designs written to {CAND}: candidate_A..E.svg/png")
Re-positive rate's 95% confidence interval excludes its own point estimate in abstract and Results
Nat Commun (2020) · doi:10.1038/s41467-020-19802-w · PMC7679396
Summary
The paper reports that 107 of 34,424 recovered COVID-19 patients tested positive again in Wuhan's city-wide screening, a re-positive rate of 0.31%, with a 95% confidence interval of 0.423–0.574%. The interval lies entirely above the point estimate, which none of the standard two-sided intervals (exact, score, Wald, Agresti–Coull, Jeffreys) does for 107/34,424. Applied to 107/34,424, the paper's stated method (Clopper–Pearson exact) gives 0.255–0.375%. The same method reproduces exactly all 27 other intervals in Tables 1–2 and the antibody text, and also the two single- and mixed-testing intervals once their unprinted counts are inferred. The point estimate and the paper's conclusions are unaffected, but the wrong interval is printed in the abstract. Anyone who extracts it, for example for a meta-analysis of re-positivity after recovery, gets a range whose lower bound lies above the correct upper bound, centred near 0.50% instead of 0.31%.
What the paper reports
Abstract:
107 of 34,424 previously recovered COVID-19 patients tested positive again (re-positive rate 0.31%, 95% CI 0.423–0.574%).
Results, second paragraph (screening results):
Of the 34,424 participants with a history of COVID-19, 107 tested positive again, giving a repositive rate of 0.310% (95% CI 0.423–0.574%).
Results, first paragraph (the denominators):
Of the 9899,828 participants, 9,865,404 had no previous diagnosis of COVID-19, and 34,424 were recovered COVID-19 patients.
Methods, "Statistical analysis" (the stated method, which covers both the asymptomatic and the re-positive rates):
Detection rate of asymptomatic positive or repositive cases was calculated by dividing the number of individuals with a positive result of nucleic acid testing by the number of participants tested. Because of extremely low detection rates, we calculated 95% confidence intervals of estimated proportions using Pearson–Klopper exact method, implemented through R package “binom” version 1.1-1…
Discussion, third paragraph (the statement that rests on this estimate):
There was a low repositive rate in recovered COVID-19 patients in Wuhan.
What the paper's own numbers show
Method. I took the count (107) and denominator (34,424) printed in the abstract and the Results and computed the proportion and its two-sided 95% Clopper–Pearson interval from beta quantiles. This is the interval that R's binom package returns for method = "exact". For comparison I also computed the Wilson, Wald, Agresti–Coull, Jeffreys and Poisson-exact intervals. To confirm that I was applying the paper's method the way the authors did, I applied the same calculation to every other interval in the paper whose count and denominator are printed: the 22 rows of Table 1 (asymptomatic positives per 10,000 by sex, age and district), the four rows of Table 2, the IgG-positive proportion in the Results (antibody results, % of 300) and the asymptomatic rate in the second paragraph of the Results. The paper has two further intervals, the asymptomatic rates for single and mixed testing in the fourth paragraph of the Results, for which only the share of samples (76.7% and 23.3%) is printed, not the counts. For these I took the denominators as 76.7% and 23.3% of 9,899,828 and searched for the counts whose exact interval matches the printed one. Finally I searched for the count, denominator and confidence level that would produce the published interval, and compared the counts in the publisher's Source Data workbook with the printed Tables 1 and 2. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Re-positive rate, 107/34,424 | Abstract; Results para 2 | 0.31% (abstract), 0.310% (Results) | 0.3108%, which rounds to 0.311%; the Results' 0.310% is the truncated value, a negligible difference |
| 95% CI, Clopper–Pearson (stated method) | Abstract; Results para 2 | 0.423–0.574% | 0.255–0.375% (0.2548–0.3755%) |
| 95% CI, other standard methods | — | — | Wilson 0.257–0.375%; Wald 0.252–0.370%; Agresti–Coull 0.257–0.376%; Jeffreys 0.256–0.374%; Poisson exact 0.255–0.376% |
| Does the reported CI contain the point estimate? | — | — | No; 0.311% is below the reported lower bound of 0.423% |
| Gap between the intervals | — | — | Reported lower bound 0.423% exceeds the recomputed upper bound 0.375% by 0.048 points; the intervals do not overlap |
| Midpoint of the interval | — | 0.4985% (midpoint of 0.423–0.574%) | 0.311%; the reported midpoint is 1.60 times the point estimate |
| Width / implied standard error | — | 0.151 points; implied SE 0.0385 points | 0.121 points; binomial SE 0.0300 points |
| Reported bound ÷ recomputed bound | — | — | lower 1.66, upper 1.53 |
| Count x out of 34,424 whose exact CI rounds to 0.423–0.574% | reverse search | — | only x = 170 (0.4225–0.5737%; rate 0.494%) |
| Denominator n for which 107/n gives 0.423–0.574% | reverse search, n = 10,000–60,000 | — | none; the closest (n = 22,514) gives 0.390–0.574% |
| Other confidence levels for 107/34,424 | reverse search | — | 90%: 0.263–0.365%; 99%: 0.239–0.397%; 99.9%: 0.221–0.422%. Even the 99.9% upper bound is below 0.423% |
| Table 1, 22 rates per 10,000 with CIs | Table 1 | e.g. Total 0.303 (0.270–0.339); Male 0.256 (0.214–0.303); Wuchang 0.807 (0.633–1.015); East Lake Scenic 0.000 (0.000–0.430) | all 22 reproduce exactly with Clopper–Pearson |
| Antibody proportions, % of 300 | Table 2; Results para 9 | 53.7 (47.8–59.4); 36.7 (31.2–42.4); 9.7 (6.6–13.6); 0.0 (0.0–1.2); IgG+ 63.3 (57.6–68.8) | all 5 reproduce exactly from the printed counts (161, 110, 29, 0; IgG+ 190) |
| Asymptomatic rates, single and mixed testing (counts not printed) | Results para 4 | single 0.321 (0.282–0.364)/10,000; mixed 0.243 (0.183–0.315)/10,000 | with denominators of about 7,593,168 (76.7% of 9,899,828) and 2,306,660 (23.3%), only the counts 244 and 56 reproduce the printed rate and interval, and 244 + 56 = 300, the total of asymptomatic positives. Both intervals are consistent with Clopper–Pearson |
| Asymptomatic rate, denominator 9,865,404 | Results para 2 ("participants without a history of COVID-19") | 0.303 (0.270–0.339)/10,000 | 0.304 (0.271–0.341)/10,000; the printed figures match 300/9,899,828 (all participants), as in Table 1 |
| Denominators | Results para 1; Table 1; Table 2 | 9,865,404 + 34,424; total 9,899,828 | sum 9,899,828; sex, age and district totals also sum to 9,899,828 and district cases to 300; printed antibody counts sum to 300, with 161 + 29 = 190 IgG-positive |
| Antibody counts in the publisher's Source Data workbook | Source Data, sheet "Raw data from Table 2" | printed Table 2: 161, 110, 29, 0; Results: 190 IgG-positive | the sheet gives 157, 110, 28, 5 (also summing to 300), hence 157 + 28 = 185 IgG-positive, 61.7% (55.9–67.2%) instead of 63.3% (57.6–68.8%). The sheet "Raw data from Table 1" matches the printed Table 1 in all 21 rows |
The stated method reproduces every other interval in Tables 1–2 and the antibody text to the last printed digit, and the two single/mixed intervals are reproduced by inferred counts that add up to the paper's 300 asymptomatic positives, so the method itself is not in doubt. The count, the denominator and the point estimate are mutually consistent. Only the re-positive interval cannot be obtained from the printed inputs, under the stated method or any other standard one.
There is one minor, separate observation. The Results attach the asymptomatic rate to the 9,865,404 participants without a COVID-19 history, but the printed rate and interval are those for all 9,899,828 participants (Table 1, Total). The difference is in the third decimal and changes nothing. A second separate observation concerns the antibody counts: the publisher's Source Data workbook gives different Table 2 counts from the printed table (row above). This does not bear on the re-positive interval.
Figure
Figure 1. The re-positive rate among recovered patients on one percentage axis, starting at zero. The orange row is the reported point estimate and 95% CI (identical in the Abstract and the Results). The blue rows are intervals recomputed from the paper's own count and denominator, 107/34,424: Clopper–Pearson (the method the Methods name), Wilson score and Wald. The dashed vertical line marks the point estimate, 0.311%. Every recomputed interval contains it and spans about 0.25–0.38%. The reported interval, 0.423–0.574%, lies wholly to the right of its own point estimate and does not overlap any of the recomputed intervals.
Figure 2. Each reported confidence bound in the paper, divided by the Clopper–Pearson bound recomputed from that row's count and denominator and rounded to the same number of decimals (log scale; 1 means identical). The counts are printed in the paper except for the single- and mixed-testing rates, where they are inferred (244 and 56; see the table). Within each row, hollow circles (lower bounds) are drawn just above the row and filled diamonds (upper bounds) just below it, so both stay visible when their ratios are equal; lower bounds of zero are omitted. All 22 Table 1 intervals, all 5 antibody intervals (from the printed Table 2 counts) and both single/mixed intervals give a ratio of exactly 1.000. The Results-text asymptomatic rate, checked against the 9,865,404 denominator the text names, gives 0.996 and 0.994 (it matches the 9,899,828 total exactly). The re-positive interval (orange) gives 1.66 for the lower bound and 1.53 for the upper bound. It is the only interval that the stated method does not reproduce with any of the paper's denominators.
Why it matters
- The interval is in the abstract. The abstract is the part of the paper most often read, indexed and quoted. As printed, it puts the plausible range of re-positivity at 0.42–0.57%. The correct range is 0.26–0.38%, and the two do not overlap.
- Evidence synthesis. Systematic reviews and meta-analyses of SARS-CoV-2 re-positivity or re-detection after recovery commonly extract a proportion with its CI, or back-calculate a standard error from the CI. Using the published interval would place this study's estimate near 0.50% (the interval midpoint) instead of 0.31%, with a standard error about 1.28 times too large (0.0385 vs 0.0300 percentage points). The study's large denominator would still give it notable weight, so the shifted centre (about 0.50%) would pull a pooled estimate upward, while its inverse-variance weight would be about 0.61 of the correct weight, that is, understated by about 40%.
- Internal coherence. A reader who sees an interval that excludes its own estimate cannot tell from the paper which number is wrong: the count, the denominator, the point estimate or the interval. The arithmetic shows that the count, denominator and point estimate agree with one another and that the interval is the outlier.
- What is not affected. The headline result, the asymptomatic detection rate of 0.303 per 10,000 (0.270–0.339), reproduces exactly, as do all of Table 1 and, with inferred counts, the single- and mixed-testing rates. The Table 2 intervals and the 63.3% IgG-positive figure reproduce exactly from the printed counts; the Source Data workbook's different counts (157, 110, 28, 5) would give 185 IgG-positive (61.7%), which does not change the paper's reading that most asymptomatic positives had antibodies. The Discussion's statement that the re-positive rate was low holds with the correct interval, whose upper bound (0.375%) is in fact lower than the published lower bound. The conclusion that post-lockdown prevalence in Wuhan was very low is unchanged.
Explanations considered
- A different interval method. This does not account for the numbers. Clopper–Pearson, Wilson, Wald, Agresti–Coull, Jeffreys and Poisson-exact intervals for 107/34,424 all lie within 0.252–0.376%. For these data the standard two-sided intervals (exact, score, Wald, Agresti–Coull, Jeffreys, Poisson exact) all contain the sample proportion, so none of the standard methods gives an interval wholly above 0.311%.
- A different confidence level. This does not account for the numbers. Changing the level changes the width, not the centre: the 90%, 99% and 99.9% exact intervals are 0.263–0.365%, 0.239–0.397% and 0.221–0.422%, all of which contain 0.311%.
- A different denominator. This does not account for the numbers. No denominator between 10,000 and 60,000 gives 107/n an exact interval that rounds to 0.423–0.574%. A smaller denominator would also change the point estimate, which the paper prints consistently as 0.31%/0.310%.
- A different count. Arithmetically, the exact 95% interval for 170/34,424 (0.494%) is 0.4225–0.5737%, which rounds to the published bounds, and no other count does. The number 170 does not appear in the archived text, and the published material does not show where the interval came from. The match is an arithmetic observation only.
- A conversion or typesetting error. This is excluded. The publisher's HTML (published 20 November 2020), the PMC archive and the Europe PMC full-text XML all carry the same interval in both the Abstract and the Results. The medRxiv preprint (10.1101/2020.06.29.20142554, version 1, 30 June 2020) already reported a relapse rate of 0.310% with the same 95% CI of 0.423–0.574%, so the interval was present before peer review.
- A different definition of "re-positive" or a subgroup. This does not account for the numbers. The Methods define re-positive cases as recovered patients who tested positive again in the screening. The abstract and Results tie the interval directly to "107 of 34,424", and the paper reports no other re-positive count or subgroup.
Status
- Version of record. The publisher's page (nature.com/articles/s41467-020-19802-w, published 20 November 2020) has the same Abstract and Results text as the PMC archive. Its change history has one entry, dated 17 December 2020: the Peer Review file had been omitted and was added. That update did not change the interval. Every quote above was confirmed verbatim against the PMC-archived text with
src/show.py. - Supplements. All four electronic supplementary files were checked. The Supplementary Information covers protocols, case definitions and the Ct values of asymptomatic cases, and has nothing on re-positive cases. The Peer Review File does not mention the re-positive interval. The Source Data workbook covers Tables 1–2 and Figures 1–3 but has no counts of recovered or re-positive patients. Its Table 1 sheet matches the printed Table 1, but its Table 2 sheet gives antibody counts of 157 (IgM−/IgG+), 110 (IgM−/IgG−), 28 (IgM+/IgG+) and 5 (IgM+/IgG−), which differ from the printed 161, 110, 29 and 0 and imply 185 IgG-positive cases rather than the 190 in the Results, although both sets sum to 300. The re-positive interval is unaffected. The Reporting Summary is a standard form. None of the supplements resolves the discrepancy.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI (its relation field links only the medRxiv preprint), and PMC and Europe PMC list no correction notices. The publisher's page shows no correction notice. A reader comment on the publisher's page dated 29 December 2020 pointed out that the point estimate lies outside the confidence interval; commenting is now closed, and no reply or correction followed in the record. The PubPeer API refused automated access (HTTP 403), and a web search found no PubPeer thread for the article.
- Sources. PMC article PMC7679396; publisher page https://www.nature.com/articles/s41467-020-19802-w and its electronic supplementary material; Europe PMC full-text XML; medRxiv 10.1101/2020.06.29.20142554 v1; Crossref; Europe PMC; PubPeer.
Suggested correction
- Replace the re-positive interval in the Abstract and in the second paragraph of the Results. If 107 of 34,424 is correct, the text would read "re-positive rate 0.31%, 95% CI 0.25–0.38%" in the Abstract and "repositive rate of 0.311% (95% CI 0.255–0.375%)" in the Results, using the Clopper–Pearson method stated in the Methods. (107/34,424 = 0.3108%; the printed 0.310% is the truncated value, and 0.311% is the rounded one.) If the count or denominator is the figure in error instead, correct it and recompute both the rate and the interval.
- Minor: in the second paragraph of the Results, either give the asymptomatic rate for the 9,865,404 participants without a COVID-19 history (0.304 per 10,000, 95% CI 0.271–0.341), or state that 0.303 (0.270–0.339) refers to all 9,899,828 participants, as in Table 1. Separately, "9899,828" in the first paragraph of the Results should read "9,899,828".
- Minor and separate: the antibody counts in the Source Data workbook (157, 110, 28, 5) differ from the printed Table 2 (161, 110, 29, 0) and from the 190 IgG-positive cases in the Results. Whichever set is in error should be corrected so that the Source Data and the printed table agree; if the Source Data counts are the correct ones, Table 2 and the 63.3% (57.6–68.8%) IgG-positive figure in the Results would change, the latter to 61.7% (55.9–67.2%).
Recomputation script (python recompute.py.txt)
"""Recompute the confidence intervals of PMC7679396
(Nat Commun 2020, doi:10.1038/s41467-020-19802-w).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract and Results paragraph 2: 107 re-positive of 34,424 recovered patients,
"0.31%" / "0.310% (95% CI 0.423-0.574%)"; 300 asymptomatic positives,
"0.303 (95% CI 0.270-0.339)/10,000"
Results paragraph 1: 9,899,828 tested = 9,865,404 without and 34,424 with prior COVID-19
Table 1: counts, cases and detection rate per 10,000 (95% CI) by sex, age and district
Table 2 and Results paragraph 9: antibody results in the 300 asymptomatic cases
Results paragraph 4: single and mixed testing for 76.7% and 23.3% of samples, asymptomatic
rates "0.321 (95% CI 0.282-0.364)/10,000 and 0.243 (95% CI 0.183-0.315)/10,000".
The counts behind these two rates are not printed; they are inferred below.
Methods: 95% CIs by the "Pearson-Klopper exact method" (R package binom), i.e. the
Clopper-Pearson interval, computed here with the beta quantile function.
"""
import math
import os
import numpy as np
from scipy.stats import beta, chi2, norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)
# ---------------------------------------------------------------- inputs (from the paper)
X_RE, N_RE = 107, 34424 # re-positive cases / recovered patients
REP_RE_PT_ABS, REP_RE_PT_RES = 0.31, 0.310 # % (abstract, Results)
REP_RE_CI = (0.423, 0.574) # % (abstract and Results)
N_TOTAL, N_NOHIST, N_HIST = 9899828, 9865404, 34424
X_ASYM = 300
REP_ASYM = (0.303, 0.270, 0.339) # per 10,000 (abstract, Results, Table 1 Total)
# Table 1: label, participants, asymptomatic positives, reported rate (lo, hi) per 10,000
TABLE1 = [
("Total", 9899828, 300, (0.303, 0.270, 0.339)),
("Male", 5162960, 132, (0.256, 0.214, 0.303)),
("Female", 4736868, 168, (0.355, 0.303, 0.413)),
("Age <=17", 969014, 12, (0.124, 0.064, 0.216)),
("Age 18-44", 4448230, 104, (0.234, 0.191, 0.283)),
("Age 45-59", 2492943, 96, (0.385, 0.312, 0.470)),
("Age >=60", 1989641, 88, (0.442, 0.355, 0.545)),
("Wuchang", 904636, 73, (0.807, 0.633, 1.015)),
("Qingshan", 414312, 23, (0.555, 0.352, 0.833)),
("Qiaokou", 583440, 32, (0.548, 0.375, 0.774)),
("Hanyang", 717429, 29, (0.404, 0.271, 0.581)),
("Jianghan", 524224, 19, (0.362, 0.218, 0.566)),
("Hongshan", 1103079, 35, (0.317, 0.221, 0.441)),
("East Lake High-tech", 782987, 19, (0.243, 0.146, 0.379)),
("Jiangan", 800440, 19, (0.237, 0.143, 0.371)),
("Caidian", 503595, 11, (0.218, 0.109, 0.391)),
("Jiangxia", 671248, 14, (0.209, 0.114, 0.350)),
("Huangpi", 979920, 14, (0.143, 0.078, 0.240)),
("Hannan", 417022, 4, (0.096, 0.026, 0.246)),
("Dongxihu", 777204, 5, (0.064, 0.021, 0.150)),
("Xinzhou", 634408, 3, (0.047, 0.010, 0.138)),
("East Lake Scenic", 85884, 0, (0.000, 0.000, 0.430)),
]
# Table 2 and Results paragraph 9: label, count of 300, reported % (lo, hi)
TABLE2 = [
("IgM- IgG+", 161, (53.7, 47.8, 59.4)),
("IgM- IgG-", 110, (36.7, 31.2, 42.4)),
("IgM+ IgG+", 29, (9.7, 6.6, 13.6)),
("IgM+ IgG-", 0, (0.0, 0.0, 1.2)),
("IgG+ (text)", 190, (63.3, 57.6, 68.8)),
]
# Results paragraph 4: label, share of samples, reported rate (lo, hi) per 10,000
SINGLE_MIXED = [
("Single testing", 0.767, (0.321, 0.282, 0.364)),
("Mixed testing", 0.233, (0.243, 0.183, 0.315)),
]
# ---------------------------------------------------------------- interval methods
def clopper_pearson(x, n, a=0.05):
lo = 0.0 if x == 0 else beta.ppf(a / 2, x, n - x + 1)
hi = 1.0 if x == n else beta.ppf(1 - a / 2, x + 1, n - x)
return lo, hi
def wilson(x, n, z=Z):
p = x / n
d = 1 + z * z / n
c = (p + z * z / (2 * n)) / d
h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
return c - h, c + h
def wald(x, n, z=Z):
p = x / n
se = math.sqrt(p * (1 - p) / n)
return p - z * se, p + z * se
def agresti_coull(x, n, z=Z):
nt = n + z * z
pt = (x + z * z / 2) / nt
se = math.sqrt(pt * (1 - pt) / nt)
return pt - z * se, pt + z * se
def jeffreys(x, n, a=0.05):
return beta.ppf(a / 2, x + 0.5, n - x + 0.5), beta.ppf(1 - a / 2, x + 0.5, n - x + 0.5)
def poisson_exact(x, n, a=0.05):
lo = 0.0 if x == 0 else chi2.ppf(a / 2, 2 * x) / 2
hi = chi2.ppf(1 - a / 2, 2 * x + 2) / 2
return lo / n, hi / n
def pct(v):
return 100.0 * v
# ---------------------------------------------------------------- 1. the re-positive rate
print("=" * 78)
print("1. Re-positive rate among recovered patients (Abstract; Results paragraph 2)")
print("=" * 78)
p_re = X_RE / N_RE
print(f"Point estimate 107/34,424 = {pct(p_re):.4f}% (reported {REP_RE_PT_ABS}% abstract, "
f"{REP_RE_PT_RES:.3f}% Results)")
print(f"Reported 95% CI: {REP_RE_CI[0]:.3f}-{REP_RE_CI[1]:.3f}%")
print(f"Does the reported CI contain the point estimate? {REP_RE_CI[0] <= pct(p_re) <= REP_RE_CI[1]}")
methods = [
("Clopper-Pearson exact (stated method)", clopper_pearson),
("Wilson score", wilson),
("Wald (normal approximation)", wald),
("Agresti-Coull", agresti_coull),
("Jeffreys", jeffreys),
("Poisson exact (count 107 as Poisson)", poisson_exact),
]
RE_CI = {}
for name, f in methods:
lo, hi = f(X_RE, N_RE)
RE_CI[name] = (lo, hi)
print(f" {name:40s} {pct(lo):.4f}-{pct(hi):.4f}% -> {pct(lo):.3f}-{pct(hi):.3f}%")
cp_lo, cp_hi = RE_CI["Clopper-Pearson exact (stated method)"]
print(f"Width: reported {REP_RE_CI[1] - REP_RE_CI[0]:.3f} points; Clopper-Pearson "
f"{pct(cp_hi - cp_lo):.3f} points")
print(f"Reported lower bound {REP_RE_CI[0]}% minus recomputed upper bound {pct(cp_hi):.3f}% = "
f"{REP_RE_CI[0] - pct(cp_hi):.3f} points (no overlap)")
mid = (REP_RE_CI[0] + REP_RE_CI[1]) / 2
print(f"Midpoint of reported CI: {mid:.4f}%; ratio to point estimate {mid / pct(p_re):.2f}")
print(f"Reported bounds / Clopper-Pearson bounds: lower {REP_RE_CI[0] / pct(cp_lo):.2f}, "
f"upper {REP_RE_CI[1] / pct(cp_hi):.2f}")
se_from_reported = (REP_RE_CI[1] - REP_RE_CI[0]) / (2 * Z)
se_true = pct(math.sqrt(p_re * (1 - p_re) / N_RE))
print(f"SE back-calculated from reported CI: {se_from_reported:.4f} points; binomial SE "
f"{se_true:.4f} points (ratio {se_from_reported / se_true:.2f})")
print(f"Inverse-variance weight with the back-calculated SE, relative to the correct weight: "
f"{(se_true / se_from_reported) ** 2:.2f}")
print("\nReverse search 1: which count x out of 34,424 gives a Clopper-Pearson CI that rounds "
"to the reported one?")
match_x = []
for x in range(1, 400):
lo, hi = clopper_pearson(x, N_RE)
if round(pct(lo), 3) == REP_RE_CI[0] and round(pct(hi), 3) == REP_RE_CI[1]:
match_x.append(x)
print(f" x = {x}: {pct(lo):.4f}-{pct(hi):.4f}% (rate {pct(x / N_RE):.3f}%)")
X_MATCH = match_x[0] if match_x else None
for x in (169, 171):
lo, hi = clopper_pearson(x, N_RE)
print(f" neighbour x = {x}: {pct(lo):.3f}-{pct(hi):.3f}%")
print("\nReverse search 2: which denominator n gives 107/n a Clopper-Pearson CI that rounds "
"to the reported one?")
hits = []
for n in range(10000, 60001):
lo, hi = clopper_pearson(X_RE, n)
if round(pct(lo), 3) == REP_RE_CI[0] and round(pct(hi), 3) == REP_RE_CI[1]:
hits.append(n)
print(f" denominators matching both bounds: {len(hits)}")
best = min(range(10000, 60001),
key=lambda n: sum(abs(pct(b) - r) for b, r in zip(clopper_pearson(X_RE, n), REP_RE_CI)))
lo, hi = clopper_pearson(X_RE, best)
print(f" closest: n = {best}: {pct(lo):.3f}-{pct(hi):.3f}% (lower bound cannot be matched)")
print("\nReverse search 3: other alpha levels for 107/34,424 (Clopper-Pearson)")
for conf in (0.90, 0.99, 0.999):
lo, hi = clopper_pearson(X_RE, N_RE, a=1 - conf)
print(f" {int(conf * 1000) / 10:g}% CI: {pct(lo):.3f}-{pct(hi):.3f}%")
print(" Every standard two-sided interval for 107/34,424 contains 0.311%; none lies above it.")
# ---------------------------------------------------------------- 2. the method elsewhere
print()
print("=" * 78)
print("2. The stated method applied to the paper's other intervals")
print("=" * 78)
ROWS = [] # (group, label, rep_lo, rep_hi, cp_lo, cp_hi) on the paper's own scale
def show(group, label, x, n, rep, scale, nd):
lo, hi = clopper_pearson(x, n)
est, lo, hi = scale * x / n, scale * lo, scale * hi
ok = (round(est, nd), round(lo, nd), round(hi, nd)) == tuple(round(v, nd) for v in rep)
fmt = f"{{:.{nd}f}}"
s = f"{fmt.format(est)} ({fmt.format(lo)}-{fmt.format(hi)})"
r = f"{fmt.format(rep[0])} ({fmt.format(rep[1])}-{fmt.format(rep[2])})"
print(f" {label:22s} {x:>4d}/{n:<9,d} recomputed {s:24s} reported {r:24s} "
f"{'match' if ok else 'DIFFERS'}")
ROWS.append((group, label, rep[1], rep[2], lo, hi))
return ok
print("Table 1, per 10,000:")
n_ok1 = sum(show("Table 1", lab, x, n, rep, 1e4, 3) for lab, n, x, rep in TABLE1)
print(f" -> {n_ok1} of {len(TABLE1)} Table 1 rows reproduce exactly")
print("Table 2 and Results paragraph 9, % of 300:")
n_ok2 = sum(show("Table 2", lab, x, 300, rep, 100, 1) for lab, x, rep in TABLE2)
print(f" -> {n_ok2} of {len(TABLE2)} antibody intervals reproduce exactly")
print("Results paragraph 2, asymptomatic rate with the 'no history' denominator 9,865,404:")
show("Results", "300/9,865,404", X_ASYM, N_NOHIST, REP_ASYM, 1e4, 3)
print("Results paragraph 4, single and mixed testing, per 10,000 (counts not printed; inferred):")
print(" Denominator taken as share x 9,899,828; the count is the one whose exact CI matches.")
SM_X = []
for lab, share, rp in SINGLE_MIXED:
n_sm = round(share * N_TOTAL)
xs = [x for x in range(0, X_ASYM + 1)
if (round(1e4 * x / n_sm, 3), *(round(1e4 * v, 3) for v in clopper_pearson(x, n_sm)))
== rp]
print(f" {lab}: n ~ {n_sm:,}; counts reproducing {rp[0]:.3f} ({rp[1]:.3f}-{rp[2]:.3f}): {xs}")
# robustness: shares that round to the printed percentage
for sh in (share - 0.0003, share + 0.0002):
n2 = round(sh * N_TOTAL)
xs2 = [x for x in range(0, X_ASYM + 1)
if (round(1e4 * x / n2, 3), *(round(1e4 * v, 3) for v in clopper_pearson(x, n2)))
== rp]
print(f" share {100 * sh:.2f}% (n = {n2:,}): matching counts {xs2}")
SM_X.append(xs[0])
show("Single/mixed", f"{lab} {xs[0]}/~{n_sm:,}", xs[0], n_sm, rp, 1e4, 3)
print(f" Inferred counts {SM_X[0]} + {SM_X[1]} = {sum(SM_X)} (total asymptomatic positives "
f"{X_ASYM})")
print("Re-positive rate, % (for comparison):")
show("Re-positive", "107/34,424", X_RE, N_RE, (REP_RE_PT_RES, *REP_RE_CI), 100, 3)
print("\nDenominator checks:")
print(f" 9,865,404 + 34,424 = {N_NOHIST + N_HIST:,} (reported total 9,899,828)")
print(f" Male + Female = {5162960 + 4736868:,}")
print(f" Sum of age groups = {969014 + 4448230 + 2492943 + 1989641:,}")
print(f" Sum of districts = {sum(r[1] for r in TABLE1[7:]):,}; "
f"cases = {sum(r[2] for r in TABLE1[7:])}")
print(f" Antibody counts 161+110+29+0 = {161 + 110 + 29 + 0}; IgG+ 161+29 = {161 + 29}")
# ---------------------------------------------------------------- 3. Source Data workbook
# Counts copied from the publisher's Source Data workbook (41467_2020_19802_MOESM4_ESM.xlsx),
# sheets "Raw data from Table 1" and "Raw data from Table 2".
SD_TABLE1 = {
"Male": (5162960, 132), "Female": (4736868, 168),
"Age <=17": (969014, 12), "Age 18-44": (4448230, 104), "Age 45-59": (2492943, 96),
"Age >=60": (1989641, 88), "Wuchang": (904636, 73), "Qingshan": (414312, 23),
"Qiaokou": (583440, 32), "Hanyang": (717429, 29), "Jianghan": (524224, 19),
"Hongshan": (1103079, 35), "East Lake High-tech": (782987, 19), "Jiangan": (800440, 19),
"Caidian": (503595, 11), "Jiangxia": (671248, 14), "Huangpi": (979920, 14),
"Hannan": (417022, 4), "Dongxihu": (777204, 5), "Xinzhou": (634408, 3),
"East Lake Scenic": (85884, 0),
}
SD_TABLE2 = [("IgM- IgG+", 157), ("IgM- IgG-", 110), ("IgM+ IgG+", 28), ("IgM+ IgG-", 5)]
print()
print("=" * 78)
print("3. Publisher's Source Data workbook compared with the printed tables")
print("=" * 78)
t1 = {lab: (n, x) for lab, n, x, _ in TABLE1}
same1 = sum(t1[k] == v for k, v in SD_TABLE1.items())
print(f"Table 1 sheet: {same1} of {len(SD_TABLE1)} rows identical to the printed Table 1 "
f"(the sheet has no Total row)")
print("Table 2 sheet vs printed Table 2 (counts of 300; % with Clopper-Pearson 95% CI):")
for (lab, xsd), (_, xp, rep) in zip(SD_TABLE2, TABLE2):
lo, hi = clopper_pearson(xsd, 300)
print(f" {lab:10s} Source Data {xsd:>3d} -> {100 * xsd / 300:.1f} ({100 * lo:.1f}-{100 * hi:.1f})"
f" printed {xp:>3d} -> {rep[0]:.1f} ({rep[1]:.1f}-{rep[2]:.1f})"
f" {'same' if xsd == xp else 'DIFFERS'}")
sd_sum = sum(x for _, x in SD_TABLE2)
sd_igg = SD_TABLE2[0][1] + SD_TABLE2[2][1]
lo, hi = clopper_pearson(sd_igg, 300)
print(f" Source Data sum = {sd_sum}; IgG+ = {SD_TABLE2[0][1]}+{SD_TABLE2[2][1]} = {sd_igg} -> "
f"{100 * sd_igg / 300:.1f}% ({100 * lo:.1f}-{100 * hi:.1f}); printed IgG+ 190 -> 63.3% "
f"(57.6-68.8)")
sd_igm = SD_TABLE2[2][1] + SD_TABLE2[3][1]
print(f" IgM+ = {sd_igm} in the Source Data vs {29 + 0} in the printed table")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#8a8984"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.edgecolor": SEC, "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": TXT,
"axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "axes.spines.top": False, "axes.spines.right": False,
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# Figure 1: reported vs recomputed interval for 107/34,424 on one % axis
rows = [
("Reported\n(Abstract; Results)", REP_RE_PT_RES, REP_RE_CI, ORANGE, "o"),
("Recomputed: Clopper–Pearson\n(the stated method)", pct(p_re),
tuple(pct(v) for v in RE_CI["Clopper-Pearson exact (stated method)"]), BLUE, "o"),
("Recomputed: Wilson score", pct(p_re), tuple(pct(v) for v in RE_CI["Wilson score"]), BLUE, "s"),
("Recomputed: Wald", pct(p_re),
tuple(pct(v) for v in RE_CI["Wald (normal approximation)"]), BLUE, "D"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.6))
ys = np.arange(len(rows))[::-1].astype(float)
for (lab, est, (lo, hi), col, mk), y in zip(rows, ys):
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
ax.plot([lo, lo], [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
ax.plot([hi, hi], [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
ax.plot(est, y, marker=mk, ms=9, color=col, mec=col, zorder=4)
ax.text(hi + 0.012, y, f"{lo:.3f}–{hi:.3f}%", va="center", color=col, fontsize=10.5)
if col == ORANGE:
ax.annotate(f"reported point estimate {est:.3f}%\nlies outside its own interval",
xy=(est + 0.008, y - 0.06), xytext=(0.395, y - 0.47), va="center",
color=ORANGE, fontsize=10, arrowprops=dict(arrowstyle="-", color=ORANGE, lw=1))
ax.axvline(pct(p_re), color=SEC, lw=0.8, ls=(0, (3, 3)), zorder=1)
ax.text(pct(p_re), len(rows) - 0.45, f"107/34,424 = {pct(p_re):.3f}%", ha="center",
va="bottom", color=SEC, fontsize=10)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows])
for t, r in zip(ax.get_yticklabels(), rows):
t.set_color(r[3])
ax.set_xlim(0, 0.70)
ax.set_ylim(-0.6, len(rows) - 0.1)
ax.set_xlabel("Re-positive rate, % (point estimate and 95% CI)")
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.tick_params(axis="y", length=0)
fig.tight_layout()
save(fig, "fig1")
# Figure 2: reported bound / recomputed Clopper-Pearson bound for every interval in the paper
# (the two single/mixed-testing rates use the inferred counts above)
fig, ax = plt.subplots(figsize=(7.2, 4.2))
groups = [("Table 1 (22 rates per 10,000)", "Table 1"),
("Table 2 and Results\n(5 antibody %, printed counts)", "Table 2"),
("Results: asymptomatic rate,\ndenominator 9,865,404", "Results"),
("Results: single / mixed testing\n(2 rates, counts inferred)", "Single/mixed"),
("Re-positive rate, 107/34,424", "Re-positive")]
ys = np.arange(len(groups))[::-1].astype(float)
rng = np.random.default_rng(1)
for (glab, gkey), y in zip(groups, ys):
sub = [r for r in ROWS if r[0] == gkey]
col = ORANGE if gkey == "Re-positive" else BLUE
for _, _, rlo, rhi, clo, chi in sub:
jit = 0.0 # identical ratios within a group would overlap anyway
if clo > 0: # lower bounds drawn above the row centre, upper bounds below it
ax.plot(rlo / round(clo, 3 if gkey != "Table 2" else 1), y + 0.14 + jit, "o", ms=8,
mfc="none", mec=col, mew=1.6, zorder=3)
ax.plot(rhi / round(chi, 3 if gkey != "Table 2" else 1), y - 0.14 + jit, "D", ms=8,
color=col, alpha=0.85, zorder=3)
re = [r for r in ROWS if r[0] == "Re-positive"][0]
ax.text(re[3] / round(re[5], 3), ys[-1] - 0.3,
f"upper {REP_RE_CI[1] / round(pct(cp_hi), 3):.2f}", ha="center", va="top",
color=ORANGE, fontsize=10)
ax.text(re[2] / round(re[4], 3), ys[-1] + 0.3,
f"lower {REP_RE_CI[0] / round(pct(cp_lo), 3):.2f}", ha="center", va="bottom",
color=ORANGE, fontsize=10)
ax.axvline(1.0, color=SEC, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.85, 2.0)
ax.set_xticks([0.9, 1.0, 1.25, 1.5, 2.0])
ax.set_xticklabels(["0.9", "1", "1.25", "1.5", "2"])
for y, txt in zip(ys[:4], ["all 22 match (ratio 1.000)",
"all 5 match (ratio 1.000)",
"0.996 and 0.994",
"both match (ratio 1.000)"]):
ax.text(1.06, y, txt, va="center", color=BLUE, fontsize=10)
ax.minorticks_off()
ax.set_yticks(ys)
ax.set_yticklabels([g[0] for g in groups])
ax.set_ylim(-0.6, len(groups) - 0.4)
ax.set_xlabel("Reported ÷ recomputed Clopper–Pearson bound (log scale)")
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.tick_params(axis="y", length=0)
h1 = ax.plot([], [], "o", ms=8, mfc="none", mec=SEC, mew=1.6, label="lower bound")[0]
h2 = ax.plot([], [], "D", ms=8, color=SEC, label="upper bound")[0]
ax.legend(handles=[h1, h2], loc="lower right", bbox_to_anchor=(1.0, 1.0), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")
print("\nRatios plotted in Figure 2 (reported / recomputed, both rounded as printed):")
for g, lab, rlo, rhi, clo, chi in ROWS:
nd = 1 if g == "Table 2" else 3
rl = "n/a" if round(clo, nd) == 0 else f"{rlo / round(clo, nd):.3f}"
print(f" {g:12s} {lab:30s} lower {rl:>6s} upper {rhi / round(chi, nd):.3f}")
print("\nWrote fig1.svg, fig1.png, fig2.svg, fig2.png to", HERE)
Abstract says serum-treated human islets came from T2DM donors; the donor table lists none
Nat Metab (2024) · doi:10.1038/s42255-024-01130-8 · PMC12005094
Summary
The abstract's only human result says that "human islets from male and female donors with T2DM" showed fewer senescence markers after treatment with serum collected after an exercise training programme. The paper's own donor table (Extended Data Table 1) lists 17 islet donors, and all 17 are marked T2D = "NO", including the 11 used for serum treatment. In the Results, the Methods and Extended Data Table 2, T2DM status belongs to the people who gave the serum, so the experiment exposed islets from non-diabetic cadaveric donors to serum from exercise-trained people with T2DM. The abstract, the Results subsection heading and the Fig. 8 title instead describe an experiment on T2DM islets; no measured value changes, but the design behind the paper's main translational claim is misstated.
What the paper reports
The abstract (the only human result it states):
"Remarkably, human islets from male and female donors with T2DM treated with serum collected after a 10-week endurance exercise training programme showed a significant decrease in the levels of senescence markers." (Abstract)
The same framing appears in a heading and a figure title:
"Postexercise serum reduces senescence markers in T2DM human islets" (Results, subsection heading)
"Postexercise serum reduces senescence markers in T2DM islets." (Fig. 8 title)
The body text describes the design differently. Diabetes is attributed to the serum donors, and the islets come from separate cadaveric donors:
"To test whether exercise training affects islet senescence in humans, we obtained serum from five individuals with T2DM who participated in a moderate-intensity exercise training programme. [...] Serum was matched by sex, age and body mass index (BMI) (Extended Data Table 2) to pancreatic islets from cadaveric donors, received from the Integrated Islet Distribution Program (IIDP) (Extended Data Table 1)." (Results, "Postexercise serum reduces senescence markers in T2DM human islets", para 1)
"Human islet donors were matched by sex, age and BMI to serum donors (individuals with or without T2DM who underwent a moderate-intensity aerobic training programme)." (Methods, "Serum treatment in vitro")
"Notably, serum obtained from exercise-trained people with T2DM decreased the expression of senescence markers in human cadaveric islets, translating these findings to human physiology." (Introduction, last paragraph)
"Details of donors are given in Extended Data Table 1." (Methods, "Human islets")
Extended Data Table 1, "Donor information for human pancreatic samples" (columns: Age (years) | T2D | BMI | Applications), has 17 rows. Every T2D cell reads "NO". Eleven rows list "Serum treatment" among the applications. Two examples:
"54 | NO | 24.5 | Serum treatment, WB, FACS, qPCR"
"39 | NO | 32.4 | Serum treatment, WB"
(Extended Data Table 1, first and eleventh rows)
Elsewhere, the paper describes its human islets as non-diabetic:
"we exposed cadaveric human islets from donors without T2DM to glucagon in vitro for 4 days" (Results, "Glucagon activates AMPK in mouse and human islets", para 4)
"human (nondiabetic, 37–62 years old, male and female, n = 8 for qPCR and n = 11 for β-gal expression) islets" (Fig. 3 legend, panels i–k)
What the paper's own numbers show
Method. Both Extended Data tables were transcribed from the PMC archive into recompute.py.txt (the same values appear in the version-of-record table images). The script counts the T2D column for all islet donors and for the subset whose Applications column includes "Serum treatment". It also checks the table sizes against the counts given in the text.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| T2D status of the serum-treated islet donors | Abstract | "donors with T2DM" | 0 of 11 serum-treatment donors have T2D = YES (Extended Data Table 1) |
| Islet donors listed | Extended Data Table 1 | 17 rows | 17 |
| Islet donors with T2D = YES | Extended Data Table 1, T2D column | all "NO" | 0 of 17 |
| Islet donors used for serum treatment | Extended Data Table 1, Applications column | 11 rows list "Serum treatment" | 11 (ages 22–62); 0 with T2D |
| Serum volunteers listed | Extended Data Table 2 vs Methods ("seven female and six male participants") | 13 | 13 (consistent) |
| Serum volunteers with T2D = YES | Extended Data Table 2 vs Results ("serum from five individuals with T2DM") | 5 used, per the text | 10 of 13 listed; the paper does not say which 5 of the 10 supplied serum for Fig. 8 |
| Islet preparations | Methods ("male (17) versus female (3) islets") vs Extended Data Table 1 | 17 + 3 = 20 | 17 rows (side discrepancy) |
The serum-treatment rows in Extended Data Table 1 fall into two groups. Five rows are marked "Serum treatment, WB, FACS, qPCR", matching the five T2DM serum donors in the Results. Six rows are marked "Serum treatment, WB", with ages 22–57, which matches the age range in the Extended Data Fig. 5 legend for the non-T2D serum experiment ("Samples from individuals ranging from 22 to 57 years old, male, n=6/group"). On this reading the legend's n = 6 counts islet preparations; the legend itself does not say so. Every islet row in both groups is T2D = "NO". The split does not change the finding, but it is consistent with the Fig. 8 islets being five of the non-diabetic donors listed in Extended Data Table 1.
Figure
Figure 1. On the left (orange) is the wording reported in the abstract and repeated in the Results subsection heading and the Fig. 8 title. On the right (blue) are counts recomputed from the paper's Extended Data Tables 1 and 2. The abstract says the islets came from donors with T2DM. Extended Data Table 1 lists 17 islet donors, none with T2D, including the 11 used for serum treatment. T2D = YES appears only among the serum volunteers in Extended Data Table 2 (10 of 13), and the Results text attributes T2DM to the serum donors ("serum from five individuals with T2DM"; the paper does not say which five of the ten were used). All counts come from recompute.py.txt.
Figure 2. Each circle is one row of the paper's donor tables. Filled circles have T2D = YES and open circles have T2D = NO, as recomputed from the tables. The top row holds the 11 islet donors whose Applications column includes "Serum treatment", the middle row the 6 other islet donors, and the bottom row the 13 serum volunteers. Every islet donor is non-diabetic. The orange line gives the abstract's description of the islet donors for comparison.
Why it matters
The abstract sentence is the paper's only stated human result and its translational headline ("Remarkably, ..."). Taken at face value, it says exercise-conditioned serum reduced senescence markers in islets from people who already had T2DM, that is, in diabetic beta cells where senescence is the proposed therapeutic target. The experiment the paper actually documents is different: non-diabetic cadaveric islets cultured with serum from exercise-trained people with T2DM. These two designs answer different questions. The first tests whether diabetic islet tissue responds. The second tests whether circulating factors from trained diabetic individuals act on healthy islets.
No reported value, effect direction or P value changes, and the narrower conclusion (post-exercise serum from people with T2DM lowered senescence markers in cultured human islets) is unaffected. Three statements are framed in terms of people who already have T2DM, and the abstract's wording presents the human experiment as direct evidence for them: the abstract's closing claim of "promising therapeutic implications for T2DM", the Discussion's statement that the findings bear on "mechanisms that can restore β-cell function in individuals with T2DM", and the criterion stated in the Results subsection "Senolytic effects of exercise training" that exercise "must decrease the percentage of senescent cells in populations in which β-cell failure has already developed, such as in older individuals and those with insulin resistance or overt T2DM". These statements also draw on the mouse experiments (that criterion is introduced for the INK-ATTAC mouse model), so they do not depend on the human experiment alone. Because the abstract, the subsection heading and the Fig. 8 title all use the same wording, a reader sees the correct design only in the Results paragraph, the Methods and the Extended Data tables.
Explanations considered
Misplaced modifier or shorthand. The wording probably means "human islets treated with serum from male and female donors with T2DM". The Reporting Summary supports this: "Serum was obtained from male and female volunteers with established type 2 diabetes ... A total of 5 subjects were used in the manuscript, being 2 females and 3 males". The phrase "male and female ... with established type 2 diabetes" there describes the serum donors. This explains how the wording arose but does not make the published sentence accurate. Read normally, "human islets from male and female donors with T2DM" says the islet donors had T2DM, and the heading and Fig. 8 title ("T2DM islets") say the same.
Swapped tables. The footers of the two Extended Data tables appear swapped. Table 1 (islet donors) ends "Human serum information.", and Table 2 (serum volunteers) ends "IIDP cadaveric human islets information." If the tables themselves were swapped, the islets would be the 10 T2D = YES rows and the abstract would be right. The table contents rule this out: - Table 1 is captioned "Donor information for human pancreatic samples" and has an Applications column (serum treatment, WB, FACS, qPCR) that only makes sense for islet preparations. - Table 2 is captioned "Volunteer information for serum from moderate intensity training program", has subject IDs, and has 13 rows, matching the Methods' "seven female and six male participants". - The Methods say "Details of donors are given in Extended Data Table 1" under "Human islets". - The Results text attributes T2DM to the serum donors. - Fig. 3, Fig. 4 and the Results describe the human islets as non-diabetic.
The swapped footers are a separate, minor labelling error that adds to the confusion.
Conversion or archiving artifact. Ruled out. The conflict is categorical (YES/NO cells, the words "with T2DM"), not numeric. The version of record on nature.com has the same abstract sentence and the same Fig. 8 title. The version-of-record images of Extended Data Tables 1 and 2 have the same T2D columns (17 of 17 NO; 10 of 13 YES) and the same swapped footers.
Some islet donors had T2DM but were not listed. Nothing in the paper supports this. The Methods point to Extended Data Table 1 for donor details, the table has a T2D column, and no row is YES. The Methods do count 20 islet preparations ("male (17) versus female (3)") against 17 table rows. That gap is a separate inconsistency, but no passage anywhere in the paper says any islet donor had T2DM.
Open points about the serum donors (not the main finding). The Results say serum came from "five individuals with T2DM" and the Reporting Summary says "A total of 5 subjects were used in the manuscript", while Extended Data Table 2 lists 10 volunteers with T2D = YES. This need not be a contradiction: the Methods say serum samples were received from 13 participants, so the table may list every volunteer while only five T2DM sera were used for Fig. 8. The paper does not say which five. Similarly, the Extended Data Fig. 5 legend ("n=6/group") does not say whether n counts islet preparations or serum donors; six islet preparations treated with serum from the 3 non-T2D volunteers in Extended Data Table 2 would fit. Neither point affects the T2D status of the islet donors.
Status
As of 2026-10-23:
- Version of record. The nature.com article page, reached through doi.org, was fetched again on 2026-10-23. It carries the same abstract sentence ("human islets from male and female donors with T2DM treated with serum collected after a 10-week endurance exercise training programme...") and the same Fig. 8 title ("Postexercise serum reduces senescence markers in T2DM islets."). The page has no "Author Correction", "Change history" or "This article has been updated" notice. An earlier audit stage viewed the version-of-record images of Extended Data Tables 1 and 2 and found them identical to the PMC text (17 of 17 islet donors NO; 10 of 13 serum volunteers YES; same swapped footers). The full Results and Methods text on nature.com is paywalled, so those passages were checked in the PMC copy, which is the NIH author manuscript of the article.
- Supplements. The Reporting Summary (42255_2024_1130_MOESM2_ESM.pdf) was read on 2026-10-23. Page 2, "Reporting on sex and gender", reads: "Serum was obtained from male and female volunteers with established type 2 diabetes, no criteria was used to exclude sex. A total of 5 subjects were used in the manuscript, being 2 females and 3 males." It attributes diabetes to the serum volunteers and says nothing about diabetic islet donors. The Source Data workbook for Fig. 8 (42255_2024_1130_MOESM10_ESM.xlsx) has columns Pre, Post, dA1c, dP21CIP1, dP16INK4A and dGlucagon, most sheets holding five data rows, and no donor diabetes status. An earlier audit stage found that the Supplementary Information PDF contains only a gating-strategy figure.
- Corrections.
src/known.py(run 2026-10-23) lists no Crossref updates and no PMC or Europe PMC correction or retraction notices. A Crossreffilter=updates:query for the DOI returns 0 results. - PubPeer. PubPeer's site and API returned HTTP 403 on 2026-10-23, so PubPeer could not be checked directly. An earlier audit stage searched the web for the title and DOI together with PubPeer, correction and erratum and found no relevant thread or notice; no new web search could be run on 2026-10-23.
Sources: PMC12005094 (PubMed Central archive, read with src/show.py); https://www.nature.com/articles/s42255-024-01130-8; Reporting Summary https://static-content.springer.com/esm/art%3A10.1038%2Fs42255-024-01130-8/MediaObjects/42255_2024_1130_MOESM2_ESM.pdf; Source Data Fig. 8 (42255_2024_1130_MOESM10_ESM.xlsx, same location); https://api.crossref.org/works?filter=updates:10.1038/s42255-024-01130-8.
Suggested correction
Reword the abstract sentence so that T2DM describes the serum donors, for example: "Remarkably, human islets from non-diabetic donors treated with serum collected from male and female individuals with T2DM after a 10-week endurance exercise training programme showed a significant decrease in the levels of senescence markers." Make the same change to the Results subsection heading and the Fig. 8 title (e.g. "Postexercise serum from individuals with T2DM reduces senescence markers in human islets"). In the same correction, the authors could also:
- swap the footers of Extended Data Tables 1 and 2;
- identify which 5 of the 10 T2D = YES volunteers in Extended Data Table 2 supplied serum for Fig. 8, and state what n counts in the Extended Data Fig. 5 legend;
- reconcile the number of islet preparations (20 in the Methods, 17 in Extended Data Table 1).
Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC12005094 and regenerate fig1/fig2.
Run: python recompute.py.txt (writes fig1.svg/png and fig2.svg/png next to this file)
Inputs are transcribed from the article as archived in PubMed Central (PMC12005094):
Extended Data Table 1 ("Donor information for human pancreatic samples") and
Extended Data Table 2 ("Volunteer information for serum from moderate intensity
training program"). The same values appear in the version-of-record table images.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
# Extended Data Table 1: (age, T2D, BMI, applications), in table order
EDT1 = [
(54, "NO", 24.5, "Serum treatment, WB, FACS, qPCR"),
(45, "NO", 21.9, "Serum treatment, WB, FACS, qPCR"),
(62, "NO", 23, "Serum treatment, WB, FACS, qPCR"),
(58, "NO", 58, "Serum treatment, WB, FACS, qPCR"),
(53, "NO", 31.20, "Serum treatment, WB, FACS, qPCR"),
(53, "NO", 33.6, "Serum treatment, WB"),
(22, "NO", 31.79, "Serum treatment, WB"),
(57, "NO", 30.9, "Serum treatment, WB"),
(52, "NO", 31.6, "Serum treatment, WB"),
(52, "NO", 37.5, "Serum treatment, WB"),
(39, "NO", 32.4, "Serum treatment, WB"),
(49, "NO", 34.8, "FACS, qPCR"),
(37, "NO", 28.4, "FACS, qPCR"),
(56, "NO", 32.8, "FACS, qPCR"),
(42, "NO", 23.5, "FACS, qPCR"),
(39, "NO", 31.1, "FACS, qPCR"),
(58, "NO", 31.8, "FACS, qPCR"),
]
# Extended Data Table 2: (subject id, age, BMI, T2D), in table order
EDT2 = [
(1, 48, 26.6, "YES"), (3, 51, 34.1, "YES"), (4, 55, 30.1, "YES"), (8, 55, 25, "YES"),
(9, 54, 25.7, "YES"), (2, 41, 30.3, "YES"), (5, 41, 33.6, "YES"), (6, 50, 32.3, "YES"),
(7, 46, 33.1, "YES"), (10, 52, 28, "YES"), (11, 53, 33.6, "NO"), (12, 27, 30.2, "NO"),
(13, 54, 31.7, "NO"),
]
# ---------------------------------------------------------------- recomputation
n_isl = len(EDT1)
isl_t2d = sum(r[1] == "YES" for r in EDT1)
serum_isl = [r for r in EDT1 if "Serum treatment" in r[3]]
serum_isl_t2d = sum(r[1] == "YES" for r in serum_isl)
grpA = [r for r in serum_isl if "qPCR" in r[3]] # Serum treatment, WB, FACS, qPCR
grpB = [r for r in serum_isl if "qPCR" not in r[3]] # Serum treatment, WB
n_ser = len(EDT2)
ser_t2d = sum(r[3] == "YES" for r in EDT2)
ser_no_ages = sorted(r[1] for r in EDT2 if r[3] == "NO")
print("== Islet donors, Extended Data Table 1 ==")
print(f"rows listed: {n_isl}")
print(f"T2D = YES: {isl_t2d} of {n_isl}")
print(f"rows whose Applications include 'Serum treatment': {len(serum_isl)}")
print(f" of these T2D = YES: {serum_isl_t2d} of {len(serum_isl)}")
print(f" ages: {min(r[0] for r in serum_isl)}-{max(r[0] for r in serum_isl)}")
print(f" 'Serum treatment, WB, FACS, qPCR' rows: {len(grpA)} (ages {sorted(r[0] for r in grpA)})")
print(f" 'Serum treatment, WB' rows: {len(grpB)} (ages {min(r[0] for r in grpB)}-{max(r[0] for r in grpB)})")
print()
print("== Serum volunteers, Extended Data Table 2 ==")
print(f"rows listed: {n_ser}")
print(f"T2D = YES: {ser_t2d} of {n_ser}")
print(f"T2D = NO: {n_ser - ser_t2d} (ages {ser_no_ages})")
print()
print("== Cross-checks against the text ==")
print(f"Methods 'seven female and six male participants' = {7 + 6}; equals EDT2 rows: {7 + 6 == n_ser}")
print(f"Results 'serum from five individuals with T2DM' = 5; EDT2 T2D = YES rows listed: {ser_t2d} "
f"(the paper does not say which 5 of the {ser_t2d} supplied serum for Fig. 8)")
print(f"Methods 'male (17) versus female (3) islets' = {17 + 3}; EDT1 rows: {n_isl}")
print(f"Ext. Data Fig. 5 legend 'n=6/group', '22 to 57 years old'; EDT1 'Serum treatment, WB' rows: "
f"{len(grpB)}, ages {min(r[0] for r in grpB)}-{max(r[0] for r in grpB)}; EDT2 non-T2D volunteers: "
f"{n_ser - ser_t2d} (the legend does not say whether n counts islet preparations or serum donors)")
print()
print(f"Verdict: islet donors with T2D in EDT1 = {isl_t2d}; the abstract's 'human islets from male and "
f"female donors with T2DM' is not supported by the paper's own donor table.")
# ---------------------------------------------------------------- figure style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"text.color": INK})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# ---------------------------------------------------------------- fig1: text-vs-table diagram
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.6); ax.axis("off")
def panel(x, y, w, h, color, title, sub, body):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=color, lw=2))
ax.text(x + 0.14, y + h - 0.16, title, ha="left", va="top", fontsize=11, weight="bold", color=color)
ax.text(x + 0.14, y + h - 0.44, sub, ha="left", va="top", fontsize=10, color=INK2)
ax.text(x + 0.14, y + h - 0.80, body, ha="left", va="top", fontsize=10, color=INK, linespacing=1.4)
panel(0.15, 1.55, 3.15, 2.95, ORANGE, "As reported (abstract)",
"Repeated in a heading and Fig. 8 title",
"\u201chuman islets from male and\nfemale donors with T2DM\ntreated with serum collected\nafter a 10-week endurance\nexercise training programme\u201d\n\nResults heading: \u201c\u2026 in T2DM\nhuman islets\u201d\nFig. 8 title: \u201c\u2026 in T2DM islets\u201d")
panel(3.9, 1.55, 3.15, 2.95, BLUE, "The paper's own tables",
"Extended Data Tables 1 and 2",
f"Islet donors (Table 1): {n_isl} listed\n"
f" with T2D = YES: {isl_t2d} of {n_isl}\n"
f" used for serum treatment: {len(serum_isl)}\n"
f" of those with T2D: {serum_isl_t2d} of {len(serum_isl)}\n\n"
f"Serum volunteers (Table 2): {n_ser}\n"
f" with T2D = YES: {ser_t2d} of {n_ser}\n"
"Results: \u201cserum from five\nindividuals with T2DM\u201d\n"
"(which 5 is not stated)")
# conflict box joined to both panels
ax.add_patch(FancyBboxPatch((1.0, 0.22), 5.2, 0.86, boxstyle="round,pad=0.02,rounding_size=0.08",
fc=BG, ec=INK2, lw=1.2, ls="--"))
ax.text(3.6, 0.83, "Conflict: who had T2DM?", ha="center", va="center", fontsize=11, weight="bold", color=INK)
ax.text(3.6, 0.47, "Abstract: the islet donors. Tables and Results: only the serum donors;\n"
f"all {n_isl} islet donors are T2D = NO.", ha="center", va="center", fontsize=10, color=INK)
for x0 in (1.725, 5.475):
ax.add_patch(FancyArrowPatch((x0, 1.53), (3.6 + (x0 - 3.6) * 0.55, 1.10), arrowstyle="-|>",
mutation_scale=13, lw=2, color=INK2, connectionstyle="arc3,rad=0"))
save(fig, "fig1")
# ---------------------------------------------------------------- fig2: donor-level dot strip
fig, ax = plt.subplots(figsize=(7.2, 3.0))
fig.subplots_adjust(left=0.34, right=0.95, top=0.80, bottom=0.24)
rows = []
# row 0: islet donors used for serum treatment; row 1: other islet donors; row 2: serum volunteers
for i, r in enumerate(serum_isl):
rows.append((i, 2, r[1] == "YES"))
for i, r in enumerate([r for r in EDT1 if "Serum treatment" not in r[3]]):
rows.append((i, 1, r[1] == "YES"))
for i, r in enumerate(EDT2):
rows.append((i, 0, r[3] == "YES"))
for x, y, yes in rows:
ax.plot(x, y, "o", ms=11, mec=BLUE, mew=2, mfc=BLUE if yes else BG)
ax.set_yticks([0, 1, 2])
ax.set_yticklabels([f"Serum volunteers\n(Ext. Data Table 2, n={n_ser})",
f"Islet donors, other uses\n(Ext. Data Table 1, n={n_isl - len(serum_isl)})",
f"Islet donors, serum treatment\n(Ext. Data Table 1, n={len(serum_isl)})"])
ax.set_xlim(-0.7, max(len(serum_isl), n_ser) - 0.3); ax.set_ylim(-0.6, 2.6)
ax.set_xticks([]); ax.tick_params(axis="y", length=0, colors=INK)
for s in ax.spines.values():
s.set_visible(False)
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.text(len(serum_isl) - 0.3, 2, f" {serum_isl_t2d} of {len(serum_isl)} T2D", va="center", fontsize=10, color=BLUE)
ax.text(n_isl - len(serum_isl) - 0.3, 1, f" {isl_t2d} of {n_isl - len(serum_isl)} T2D", va="center", fontsize=10, color=BLUE)
ax.annotate(f"{ser_t2d} of {n_ser} T2D", xy=(ser_t2d - 1, 0), xytext=(ser_t2d - 1, -0.52),
ha="center", va="center", fontsize=10, color=BLUE)
handles = [Line2D([], [], ls="", marker="o", ms=9, mec=BLUE, mew=2, mfc=BLUE,
label="T2D = YES (recomputed from tables)"),
Line2D([], [], ls="", marker="o", ms=9, mec=BLUE, mew=2, mfc=BG, label="T2D = NO")]
ax.legend(handles=handles, loc="lower left", bbox_to_anchor=(-0.45, 1.02), ncol=2, frameon=False, fontsize=10)
fig.text(0.02, 0.10, "As reported in the abstract: the islets were \u201cfrom male and female donors with T2DM\u201d.",
fontsize=10, color=ORANGE, ha="left")
fig.text(0.02, 0.025, "Each circle is one table row; fill shows the T2D column of that row.", fontsize=10, color=INK2, ha="left")
save(fig, "fig2")
print("\nfigures written: fig1.svg, fig1.png, fig2.svg, fig2.png")
Abstract reports WGS spanned 49 (98%) of 48 P-VNTRs; the body gives 92% and 45
PLoS Comput Biol (2025) · doi:10.1371/journal.pcbi.1012885 · PMC11975116
Summary
The paper compares targeted and whole-genome (WGS) PacBio HiFi sequencing for genotyping a curated set of 48 phenotype-associated VNTRs (P-VNTRs). The Abstract says the WGS data spanned "49 (98%) of P-VNTRs with at least 15 spanning reads". That count cannot be right, because 49 of 48 is 102.1%. The body gives two other values for the same quantity: 92% (Results 2.5.2 and the Discussion) and "45 VNTRs (out of 48)" (Discussion), which is 93.75%, not 92%. The best-supported figure from the paper's own text is 45 of 48 (about 94%). The Abstract's 98% therefore overstates WGS coverage of the P-VNTR set by roughly 4 to 6 percentage points. The paper's comparison with targeted sequencing (4 of 48) is unaffected.
What the paper reports
In contrast, the 40X WGS HiFi dataset spanned 98% of all VNTRs and 49 (98%) of P-VNTRs with at least 15 spanning reads, albeit with lower coverage.
— Abstract
Similar to G-VNTRs, whole genome sequencing covered 92% of P-VNTRs with median spanning coverage ≥ 15 spanning reads in 28 samples (Fig 3b). CACNA1C, DUX4 and ABCA7 VNTRs were not well-covered likely due to VNTR length.
— Results, section 2.5.2
In contrast, whole genome sequencing was unbiased with at least 15 spanning reads for 98% of G-VNTRs and 92% of the P-VNTRs. For P-VNTRs, the gap in spanning reads between the two sequencing methods was even more stark, with only 4 VNTRs in targeted sequencing and 45 VNTRs (out of 48) in whole genome sequencing supported by at least 15 spanning reads.
— Discussion, paragraph 1
The size of the P-VNTR set is stated, and adds up to 48, in several places:
We curated a set of 10 , 787 gene-proximal (G-)VNTRs, and 48 phenotype-associated (P-)VNTRs of interest.
— Abstract (the spacing in "10 , 787" is as archived)
The P-VNTRs includes 39 pathogenic and 9 non-pathogenic phenotype-associated loci.
— Fig 3 legend, panel (a)
However, 19 (39%) of 48 P-VNTRs had high repeat-unit count (RC) variation, … Another 21 had low RC variation, while the remaining 8 VNTRs associated with phenotypes based not on length but on the presence of specific motifs …
— Results, section 2.5
Only 4 VNTRs of the 48 P-VNTRs were spanned by ≥ 15 targeted spanning reads.
— Results, section 2.5.1
What the paper's own numbers show
Method: take the denominator the paper states (48 P-VNTRs, confirmed by 39 + 9 and by 19 + 21 + 8). Convert each reported count to a percentage, and each reported percentage to the count it implies over 48. Then compare these with the count implied by the three loci that Results 2.5.2 names as not well covered. All arithmetic is in recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Size of the P-VNTR set | Abstract; Results 2; Fig 3a legend; Results 2.5; Discussion | 48 | 39 + 9 = 48; 19 + 21 + 8 = 48 |
| P-VNTRs spanned by ≥ 15 WGS reads (count) | Abstract | 49 | 49/48 = 102.08%, which exceeds the whole set |
| Same quantity (percentage) | Abstract | 98% | 0.98 × 48 = 47.04, i.e. 47/48 = 97.92%. This is neither 49 nor any count in the body |
| Same quantity (percentage) | Results 2.5.2; Discussion | 92% | 0.92 × 48 = 44.16, i.e. 44/48 = 91.67% |
| Loci named as not well covered by WGS | Results 2.5.2 | CACNA1C, DUX4, ABCA7 | 48 − 3 = 45, i.e. 45/48 = 93.75% (rounds to 94%) |
| Same quantity (count) | Discussion | 45 (out of 48) | 45/48 = 93.75%. The same paragraph says 92% |
| Abstract overstatement, in percentage points | Abstract vs body | — | 98 − 93.75 = 4.25 (vs 45/48); 98 − 91.67 = 6.33 (vs 44/48) |
| P-VNTRs spanned by ≥ 15 targeted reads | Abstract; Results 2.5.1; Discussion | 4 (8%) | 4/48 = 8.33%, consistent |
| High-RC P-VNTRs | Results 2.5 | 19 (39%) | 19/48 = 39.58% (would round to 40%), a minor rounding slip |
| Low-RC + motif P-VNTRs | Results 2.5 | 29 (60% of 48) | 29/48 = 60.42%, consistent |
The paper's statements give three different counts for one quantity: 49 (Abstract), about 44 (the 92% in Results and Discussion) and 45 (Discussion). Only 45 agrees with the three loci the Results names as poorly covered. No count over 48 can produce both "49" and "98%".
Figure
Figure 1. Three passages of the paper, quoted verbatim (orange boxes), report WGS coverage of the same 48 P-VNTRs at the same threshold (at least 15 spanning reads). The blue connectors give the arithmetic from the paper's own numbers. The Abstract's "49" is larger than the set (49/48 = 102.1%), and its 98% would correspond to 47 loci. Results gives 92% (about 44 loci) but names three loci as not well covered, which implies 45. The Discussion gives both 92% and 45 of 48 (93.75%). The set size of 48 is stated and adds up consistently throughout the paper.
Figure 2. Each reported value expressed as a count out of 48 (orange circles; the 92% statements are shown as 0.92 × 48 = 44.2), set against 45, the count left after removing the three loci that Results 2.5.2 names as not well covered (blue diamonds). The dashed line marks the whole set of 48, and the shaded region holds counts larger than the set. Only the Abstract's value falls there. The axis starts at zero.
Why it matters
The Abstract is the part of the paper most readers see and cite, and it states a P-VNTR count that is larger than the P-VNTR set. Its 98% looks plausible on its face, so a reader who does not check it against the stated set size of 48 would take WGS coverage of the clinically relevant loci to be about 4 to 6 points higher than the body supports. The 98% also matches the G-VNTR figure in the same sentence, which makes WGS look equally complete for both sets. The body itself says otherwise: three named loci (CACNA1C, DUX4, ABCA7) were not well covered, largely because of their length.
The error does not change the direction of the paper's main comparison. Every value the body supports (44 or 45 of 48, 92% to 94%) remains far above targeted sequencing (4 of 48, 8.3%). The paper's conclusion that WGS is more effective for broad VNTR genotyping stands. The G-VNTR figure (98% of 10,787) is reported consistently and is not affected. The problem is an impossible count, and an inflated percentage, in the Abstract for a secondary result. There is also a smaller internal inconsistency (92% vs 45/48) in the body.
Explanations considered
- A different denominator. The paper states 48 P-VNTRs everywhere, including in "45 VNTRs (out of 48)", and its subgroup counts (39 + 9; 19 + 21 + 8) both add up to 48. S2 Table has 49 rows but only 48 unique coordinates: one locus (MUC6) appears twice, and the table flags it as a duplicate entry because it is linked to both a pathogenic and a non-pathogenic phenotype. With 49 as the denominator, 92% would fit 45/49 (91.84%) and 98% would fit 48/49 (97.96%). But the abstract's "49" would then mean 49/49 = 100%, not 98%. So no single denominator reconciles "49 (98%)", and the paper never describes the set as 49 loci.
- A different coverage rule. The Abstract says "at least 15 spanning reads". Results says "median spanning coverage ≥ 15 spanning reads in 28 samples". Under any rule, at most 48 loci can meet the threshold, so 49 is impossible whichever definition was used. The Discussion uses the same threshold and the same WGS cohort, and its two adjacent figures (92% and 45 of 48) still disagree.
- Exclusions. CACNA1C is excluded only from the allele plot (Fig 3c), not from the coverage count. Results 2.5.2 counts it among the three loci that were not well covered.
- Rounding. 45/48 = 93.75% rounds to 94%, not 92%. 92% matches 44/48 (91.67%), but the text names exactly three poorly covered loci, not four. Rounding cannot turn 49 of 48 into a valid figure.
- Text conversion or archiving artifact. These are plain integers and percentages in running text, with no symbols, superscripts or table cells involved. The same wording appears in the publisher's HTML, the Europe PMC full-text XML and the PMC archive.
- Per-locus data. Fig 3b (a heatmap of spanning reads per locus and sample) holds the underlying data, but it cannot be read reliably as numbers. S2 Table lists coordinates and categories but not WGS read counts. The true count therefore cannot be settled beyond what the text implies, and the text's three named loci point to 45.
Status
- Version of record: checked on 2026-10-23. The publisher's HTML at journals.plos.org (reached through doi.org) has all three passages word for word, as do the Europe PMC full-text XML and the PMC archive. Every quotation above was checked verbatim against the archived PMC text.
- Supplements: S1 Table (PDF: genes, phenotypes, references), S2 Table (TSV: coordinates, category, pathogenicity), S3–S5 Tables (adapters, read-length summaries) and S1–S4 Figs are listed. S2 Table was downloaded and parsed: 49 rows, 48 unique coordinates and 1 flagged duplicate (MUC6). None of the supplements gives per-locus WGS spanning-read counts, so they do not resolve the discrepancy.
- Corrections: none found. Crossref shows no update-to or updated-by relations. PMC and Europe PMC have no correction, erratum, expression of concern or retraction notice (a Europe PMC correction query returned 0 hits). The PLOS article page shows no correction notice, and its comments tab shows "Reader Comments (0)".
- PubPeer: not confirmed. The PubPeer search API and search page returned a human-verification challenge (HTTP 403) on 2026-10-23. A web search found no PubPeer entry for the article.
- Sources: https://doi.org/10.1371/journal.pcbi.1012885; Europe PMC record PMID 40193344; PMC11975116; S2 Table, doi:10.1371/journal.pcbi.1012885.s002.
Status as of 2026-10-23.
Suggested correction
Replace "49 (98%) of P-VNTRs" in the Abstract with the count and percentage supported by the per-locus data behind Fig 3b, over the stated set of 48 P-VNTRs. If the three loci named in Results 2.5.2 are the only ones below the threshold, this would read "45 (94%) of P-VNTRs". Change the "92%" in Results 2.5.2 and in the Discussion to match (for example "94%", or "45 of 48"), so that the Discussion's "92%" and "45 VNTRs (out of 48)" agree. If the coverage rule in the Abstract ("at least 15 spanning reads") differs from the one in Results ("median spanning coverage ≥ 15 … in 28 samples"), state one rule and use it throughout. Optionally, also change "19 (39%)" in Results 2.5 to "19 (40%)".
Recomputation script (python recompute.py.txt)
# Recompute every number in the PMC11975116 write-up and regenerate its figures.
# Usage: python recompute.py.txt (needs numpy + matplotlib)
# Every input below is copied from the article text (doi:10.1371/journal.pcbi.1012885).
# Figures are written next to this file: fig1.*, fig2.* and five design candidates in candidates/.
import os
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, Rectangle
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs (paper)
N_PATH, N_NONPATH = 39, 9 # Fig 3a legend
N_HIGH, N_LOW, N_MOTIF = 19, 21, 8 # Results 2.5
N_STATED = 48 # Abstract, Results 2 intro, 2.5.1, Discussion ("out of 48")
ABS_COUNT, ABS_PCT = 49, 98 # Abstract: "49 (98%) of P-VNTRs"
RES_PCT = 92 # Results 2.5.2 and Discussion: "92% of (the) P-VNTRs"
DIS_COUNT = 45 # Discussion: "45 VNTRs (out of 48)"
NAMED_POOR = ["CACNA1C", "DUX4", "ABCA7"] # Results 2.5.2: "not well-covered"
TGT_COUNT, TGT_PCT = 4, 8 # Abstract / Results 2.5.1 / Discussion
S2_ROWS, S2_UNIQUE = 49, 48 # S2 Table: 49 rows, 48 unique coordinates (MUC6 listed twice)
# ---------------------------------------------------------------- arithmetic
def pct(a, b):
return 100.0 * a / b
print("== Size of the P-VNTR set")
print(f"pathogenic + non-pathogenic = {N_PATH} + {N_NONPATH} = {N_PATH + N_NONPATH}")
print(f"high-RC + low-RC + motif = {N_HIGH} + {N_LOW} + {N_MOTIF} = {N_HIGH + N_LOW + N_MOTIF}")
assert N_PATH + N_NONPATH == N_HIGH + N_LOW + N_MOTIF == N_STATED
N = N_STATED
print("\n== Abstract: '49 (98%)'")
print(f"49 / 48 = {pct(ABS_COUNT, N):.2f}% (exceeds 100%)")
print(f"98% of 48 = {ABS_PCT/100*N:.2f} loci; nearest count 47 -> 47/48 = {pct(47, N):.2f}%")
print("\n== Results 2.5.2 / Discussion: '92%'")
print(f"92% of 48 = {RES_PCT/100*N:.2f} loci; nearest count 44 -> 44/48 = {pct(44, N):.2f}%")
implied = N - len(NAMED_POOR)
print(f"48 - {len(NAMED_POOR)} named poorly covered loci ({', '.join(NAMED_POOR)}) = {implied}")
print(f"{implied}/48 = {pct(implied, N):.2f}% (rounds to {round(pct(implied, N))}%)")
print("\n== Discussion: '45 VNTRs (out of 48)'")
print(f"45/48 = {pct(DIS_COUNT, N):.2f}% (the same paragraph states 92%)")
print("\n== Counts that would round to each stated percentage (denominator 48)")
for k in range(43, 49):
print(f" {k}/48 = {pct(k, N):6.2f}%")
print("\n== Size of the abstract's overstatement (percentage points)")
print(f"98 - 93.75 = {ABS_PCT - pct(45, N):.2f} pp ; 98 - 91.67 = {ABS_PCT - pct(44, N):.2f} pp")
print("\n== Targeted sequencing (context)")
print(f"4/48 = {pct(TGT_COUNT, N):.2f}% (stated 8%)")
print("\n== Other P-VNTR percentages in Results 2.5 / 2.5.1")
print(f"19/48 = {pct(19, N):.2f}% (stated 39%); 29/48 = {pct(29, N):.2f}% (stated 60%); "
f"23/48 = {pct(23, N):.2f}% (stated 48%)")
print("\n== Alternative denominator (49 rows in S2 Table)")
for k in (45, 48, 49):
print(f" {k}/49 = {pct(k, S2_ROWS):.2f}%")
# ---------------------------------------------------------------- style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"axes.spines.top": False, "axes.spines.right": False, "svg.fonttype": "none",
"lines.linewidth": 2,
})
LW = 1.5 # points; matplotlib 1.5 pt = 2 px at 96 dpi
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg", bbox_inches="tight", pad_inches=0.12)
fig.savefig(p + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
def wrap(s, w):
return "\n".join(textwrap.wrap(s, w))
Q_ABS = ("“…the 40X WGS HiFi dataset spanned 98% of all VNTRs and "
"49 (98%) of P-VNTRs with at least 15 spanning reads…”")
Q_RES = ("“…whole genome sequencing covered 92% of P-VNTRs with median spanning "
"coverage ≥ 15 spanning reads in 28 samples (Fig 3b). CACNA1C, DUX4 and ABCA7 "
"VNTRs were not well-covered…”")
Q_DIS = ("“…at least 15 spanning reads for 98% of G-VNTRs and 92% of the P-VNTRs. "
"… 45 VNTRs (out of 48) in whole genome sequencing supported by at least "
"15 spanning reads.”")
# ---------------------------------------------------------------- candidate A: text panels
def cand_A():
fig = plt.figure(figsize=(W, 5.7))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 84); ax.axis("off")
def panel(x, y, w, h, title, quote, wid):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec=ORANGE, lw=LW))
ax.text(x + 1.5, y + h - 1.6, title, fontsize=11, fontweight="bold", va="top")
ax.text(x + 1.5, y + h - 6.2, wrap(quote, wid), fontsize=10, va="top", color=INK,
linespacing=1.35)
panel(2, 67, 96, 15, "Abstract", Q_ABS, 82)
panel(2, 18, 43, 29, "Results 2.5.2", Q_RES, 35)
panel(55, 18, 43, 29, "Discussion, paragraph 1", Q_DIS, 35)
arw = dict(arrowstyle="<->", color=BLUE, lw=LW)
ax.annotate("", xy=(14, 47.8), xytext=(14, 66.2), arrowprops=arw)
ax.text(16, 57, "49 of 48 = 102.1%, an impossible share\n98% of 48 \u2248 47 loci;"
" Results implies 44\u201345", va="center", fontsize=10, color=BLUE)
ax.annotate("", xy=(86, 47.8), xytext=(86, 66.2), arrowprops=arw)
ax.text(84, 57, "98% vs 92%\n49 vs 45 loci", va="center", ha="right", fontsize=10,
color=BLUE)
ax.annotate("", xy=(54.2, 32), xytext=(45.8, 32), arrowprops=arw)
ax.text(50, 12.5, "Results vs Discussion: 92% \u2260 45/48 = 93.75%; "
"48 \u2212 3 named loci = 45", ha="center", fontsize=10, color=BLUE)
ax.text(2, 6.3, "Set size stated throughout: 48 P-VNTRs = 39 pathogenic + 9 non-pathogenic "
"= 19 high-RC + 21 low-RC + 8 motif", fontsize=10, color=INK2)
ax.add_patch(Rectangle((2, 0.8), 2.6, 2.4, fc="white", ec=ORANGE, lw=LW))
ax.text(5.5, 2.0, "text as printed in the paper", va="center", fontsize=10, color=INK2)
ax.plot([42, 46], [2.0, 2.0], color=BLUE, lw=LW)
ax.text(47, 2.0, "recomputed from the paper\u2019s own numbers", va="center",
fontsize=10, color=INK2)
return fig
# ---------------------------------------------------------------- candidate B: count dot plot
ROWS = [ # label, reported count (or None), reported pct, recomputed count, recomputed note
("Abstract\n“49 (98%)”", ABS_COUNT, ABS_PCT),
("Results 2.5.2\n“92%”", RES_PCT / 100 * N, RES_PCT),
("Discussion\n“92%”", RES_PCT / 100 * N, RES_PCT),
("Discussion\n“45 (out of 48)”", DIS_COUNT, pct(DIS_COUNT, N)),
]
def cand_B():
fig, ax = plt.subplots(figsize=(W, 3.6))
ys = list(range(len(ROWS)))[::-1]
ax.axvline(N, color=INK2, lw=1, ls=(0, (4, 3)))
ax.text(N - 0.4, len(ROWS) - 0.45, "all 48 P-VNTRs", color=INK2, fontsize=10, va="bottom",
ha="right")
ax.axvspan(N, 52, color=GRID, alpha=0.6, lw=0)
ax.text(50, -0.75, "more loci\nthan exist", ha="center", va="center", color=INK2,
fontsize=10)
for y, (lab, cnt, _) in zip(ys, ROWS):
ax.plot([implied, cnt], [y, y], color=GRID, lw=3, zorder=1)
ax.scatter(cnt, y, s=170, color=ORANGE, zorder=2, edgecolor="white", linewidth=0.8)
ax.scatter(implied, y, s=70, color=BLUE, zorder=3, marker="D", edgecolor="white",
linewidth=0.8)
txt = f"{cnt:.0f}" if float(cnt).is_integer() else f"{cnt:.1f}"
ax.text(cnt + (0.5 if cnt >= implied else -0.5), y + 0.2, txt, color=ORANGE,
fontsize=10, ha="left" if cnt >= implied else "right")
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in ROWS], fontsize=10)
ax.set_xlim(0, 52); ax.set_ylim(-1.2, len(ROWS) - 0.2)
ax.set_xlabel("P-VNTRs spanned by ≥ 15 WGS reads (count out of 48)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.scatter([], [], s=120, color=ORANGE, label="reported (92% shown as 0.92 × 48 = 44.2)")
ax.scatter([], [], s=90, color=BLUE, marker="D",
label="48 − 3 named poorly covered loci = 45")
ax.legend(loc="lower left", frameon=False, fontsize=10, bbox_to_anchor=(0, -0.02))
return fig
# ---------------------------------------------------------------- candidate C: calculation flow
def cand_C():
fig = plt.figure(figsize=(W, 3.4))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 48); ax.axis("off")
def box(x, y, w, h, s, ec, fs=10):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.3,rounding_size=1",
fc="white", ec=ec, lw=LW))
ax.text(x + w / 2, y + h / 2, s, ha="center", va="center", fontsize=fs)
def arrow(x0, y0, x1, y1, c=INK2):
ax.annotate("", xy=(x1, y1), xytext=(x0, y0),
arrowprops=dict(arrowstyle="-|>", color=c, lw=LW))
box(1, 30, 22, 13, "P-VNTR set\n39 + 9 = 48\n19 + 21 + 8 = 48", BLUE)
box(28, 30, 28, 13, "minus 3 named as\nnot well-covered\n(CACNA1C, DUX4, ABCA7)", BLUE)
box(61, 30, 15, 13, "45 loci", BLUE, 11)
box(83, 30, 16, 13, "45/48\n= 93.75%", BLUE, 11)
arrow(23.6, 36.5, 27.4, 36.5); arrow(56.6, 36.5, 60.4, 36.5); arrow(76.6, 36.5, 82.4, 36.5)
ax.text(1, 24.5, "As printed:", fontsize=10.5, fontweight="bold", color=INK)
box(1, 9, 30, 12, "Abstract “49 (98%)”\n49/48 = 102.1%\n98% of 48 ≈ 47",
ORANGE)
box(35, 9, 30, 12, "Results / Discussion “92%”\n0.92 × 48 = 44.2\n"
"→ 44/48 = 91.67%", ORANGE)
box(69, 9, 30, 12, "Discussion “45 (out of 48)”\n45/48 = 93.75%\n"
"(matches the 3 named loci)", ORANGE)
ax.add_patch(Rectangle((1, 1.5), 2.6, 2.4, fc="white", ec=BLUE, lw=LW))
ax.text(4.5, 2.7, "derived from the paper’s own counts", va="center", fontsize=10,
color=INK2)
ax.add_patch(Rectangle((48, 1.5), 2.6, 2.4, fc="white", ec=ORANGE, lw=LW))
ax.text(51.5, 2.7, "value as reported, with its arithmetic", va="center", fontsize=10,
color=INK2)
return fig
# ---------------------------------------------------------------- candidate D: percentage dumbbell
def cand_D():
rows = [
("Abstract: “49 (98%)”", ABS_PCT, pct(ABS_COUNT, N), "49/48"),
("Results 2.5.2: “92%”", RES_PCT, pct(implied, N), "45/48 (3 loci named)"),
("Discussion: “92%” …\n“45 VNTRs (out of 48)”", RES_PCT,
pct(DIS_COUNT, N), "45/48"),
("Targeted (context): “4 (8%)”", TGT_PCT, pct(TGT_COUNT, N), "4/48"),
]
fig, ax = plt.subplots(figsize=(W, 3.4))
ys = list(range(len(rows)))[::-1]
ax.axvline(100, color=INK2, lw=1, ls=(0, (4, 3)))
ax.text(99.3, len(rows) - 0.55, "100%", color=INK2, ha="right", fontsize=10)
for y, (lab, rep, rec, note) in zip(ys, rows):
ax.plot([rep, rec], [y, y], color=GRID, lw=3, zorder=1)
ax.scatter(rep, y, s=90, color=ORANGE, zorder=3)
ax.scatter(rec, y, s=90, color=BLUE, marker="D", zorder=3)
lo, hi = min(rep, rec), max(rep, rec)
ax.text(lo - 2, y, f"{rep:g}%", color=ORANGE, ha="right", va="center", fontsize=10)
ax.text(hi + 2, y, f"{note} = {rec:.2f}%", color=BLUE, va="center", fontsize=10)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xlim(0, 106); ax.set_ylim(-0.5, len(rows) - 0.4); ax.set_xticks(range(0, 101, 20))
ax.set_xlabel("Share of the 48 P-VNTRs spanned by ≥ 15 reads (%)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.scatter([], [], s=90, color=ORANGE, label="percentage as printed")
ax.scatter([], [], s=90, color=BLUE, marker="D",
label="percentage implied by the paper’s own count")
ax.legend(loc="upper center", bbox_to_anchor=(0.45, -0.22), ncol=2, frameon=False,
fontsize=10)
return fig
# ---------------------------------------------------------------- candidate E: unit grid of 48 loci
def cand_E():
fig = plt.figure(figsize=(W, 3.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 40); ax.axis("off")
cols, size, gap, x0, y0 = 16, 4.2, 0.9, 2, 30
for i in range(N):
r, c = divmod(i, cols)
x, y = x0 + c * (size + gap), y0 - r * (size + gap)
poor = i >= N - len(NAMED_POOR)
ax.add_patch(Rectangle((x, y), size, size, fc="white" if poor else BLUE,
ec=BLUE, lw=LW))
if i == N - 2:
ax.text(x + size / 2, y - 0.8, ", ".join(NAMED_POOR), fontsize=10,
ha="center", va="top", color=INK2)
# the abstract's 49th locus
xg = x0 + cols * (size + gap) + 2.5
yg = y0 - 2 * (size + gap)
ax.add_patch(Rectangle((xg, yg), size, size, fc="none", ec=ORANGE, lw=LW, ls=(0, (3, 2))))
ax.text(xg + size + 1.2, yg + size / 2,
"Abstract: “49 (98%)”\nwould need a 49th locus\n(49/48 = 102.1%)",
fontsize=10, color=ORANGE, va="center")
ax.text(x0, 37.5, "The 48 P-VNTRs (39 + 9 = 19 + 21 + 8 = 48), one square per locus",
fontsize=10.5, fontweight="bold")
ax.add_patch(Rectangle((x0, 7.2), 2.8, 2.8, fc=BLUE, ec=BLUE, lw=LW))
ax.text(x0 + 4, 8.6, "45 not named as poorly covered: 45/48 = 93.75%", fontsize=10,
va="center", color=INK2)
ax.add_patch(Rectangle((x0, 2.2), 2.8, 2.8, fc="white", ec=BLUE, lw=LW))
ax.text(x0 + 4, 3.6, "3 named as “not well-covered” in Results 2.5.2", fontsize=10,
va="center", color=INK2)
ax.text(60, 8.6, "Results and Discussion: “92%” = 0.92 × 48 = 44.2",
fontsize=10, color=ORANGE, va="center")
ax.text(60, 3.6, "orange = as printed; blue = recomputed", fontsize=10, color=INK2,
va="center")
return fig
designs = {"A_text_panels": cand_A, "B_count_dots": cand_B, "C_calc_flow": cand_C,
"D_percent_dumbbell": cand_D, "E_unit_grid": cand_E}
for name, fn in designs.items():
save(fn(), os.path.join(CAND, name))
# Provisional picks for the write-up (swap after the design review):
save(cand_A(), os.path.join(HERE, "fig1"))
save(cand_B(), os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE)
Abstract gives Padj = 0.010 for a primary macrolide comparison; Results give 0.19 (Table 2: 0.187)
PLoS Med (2024) · doi:10.1371/journal.pmed.1004386 · PMC11073710
Summary
This MORDOR sub-study compares gut macrolide resistance determinants across four azithromycin treatment histories. For the comparison of 4.5 years of treatment (group 1) with 2.5 years (group 2), the abstract reports "fold change 0.80, 95% CI 0.50 to 1.00, Padj = 0.010". The Results report the same comparison as "0.80-fold, 95% CI: 0.52 to 1.00, Padj = 0.19", and Table 2(A) gives its min-P adjusted p-value as 0.187. That value rounds to 0.19, not to 0.010, and a 95% interval that reaches 1.00 is not compatible with an adjusted p of 0.010. The paper's own wording ("no significant increase") and its conclusion are consistent with the correct value, 0.19. But anyone who extracts the abstract's number gets a nominally significant result, apparently a significant decrease, for the comparison that underlies the conclusion that "resistance may plateau".
What the paper reports
Abstract, "Methods and findings" (Padj is printed with a subscript in the original):
However, there was no significant increase in macrolide resistance in communities treated 4.5 years (group 1) compared to just the most recent 2.5 years (group 2) (fold change 0.80, 95% CI 0.50 to 1.00, Padj = 0.010), or between communities that had been treated for 3 years in the past (group 3) versus just 1 year in the past (group 4) (fold change 1.00, 95% CI 0.78 to 2.35, Padj = 0.52).
Results, "Primary outcome", the same comparison:
Finally, we did not find a statistically significant difference in macrolide resistance determinants between 4.5 years of azithromycin treatment (group 1) compared to 2.5 years of treatment (group 2) (0.80-fold, 95% CI: 0.52 to 1.00, Padj = 0.19, Wilcoxon rank-sum, min-P adjustment) or 3 years of treatment (group 3) (1.60-fold, 95% CI: 1.00 to 3.08, Padj = 0.06, Wilcoxon rank-sum, min-P adjustment).
Results, "Primary outcome", group 1 vs group 4:
In particular, 4.5 years of azithromycin treatments (group 1) resulted in a 2.23-fold (95% CI: 1.20 to 5.17) increase in macrolide resistance determinants compared to 1 year of azithromycin treatment only during year 3 of the study (Padj = 0.01, Wilcoxon rank-sum (i.e., without covariate adjustment), min-P adjustment).
Table 2(A), rows 1 vs. 2 and 1 vs. 4. The columns are Groups | Treatment | Aminoglycosides | Cationic antimicrobial peptides | Macrolides | Metronidazole | Multi drug resistance | Tetracyclines | Trimethoprim | Beta-lactams:
1 vs. 2 | AA-AA-AA-AA-A vs.PP-PP-AA-AA-A | 0.218 | 0.918 | 0.187 | 0.255 | 0.945 | 0.140 | 0.112 | 0.408
1 vs. 4 | AA-AA-AA-AA-A vs.PP-PP-AA-PP-P | 0.218 | 0.791 | 0.014 | 0.884 | 0.848 | 0.927 | 0.730 | 0.957
Table 2 footnote (A):
The P-values derived from the min-P procedure that were prespecified in the Statistical Analysis Plan are presented in this table.
Methods, "Statistical methods > Primary outcome":
Pairwise Wilcoxon rank-sum tests for macrolide resistance determinants between each of the treatment history groups, accounting for multiple comparisons by means of the Westfall–Young min-P method [20], which are reported as adjusted P-values.
Methods, "Statistical methods > Secondary outcomes", on how fold changes and CIs were obtained:
Fold changes were estimated by transforming normalized read counts using the transformation log(x + epsilon) where epsilon = 0.001 (qualitative results are not sensitive to choice of small epsilon) and performing analysis using the unadjusted confidence interval from the Wilcoxon rank-sum test.
Specifically, the Hodges–Lehmann estimator was computed on the transformed scale, and the inverse transformation was used to yield an approximate ratio.
The statements that rest on this comparison. Abstract, "Conclusions":
resistance may plateau after 2 to 3 years of treatment
Author summary:
Longer periods of treatment with azithromycin were not associated with increased macrolide resistance
Discussion, first paragraph:
However, we were unable to find a difference between communities treated for 4.5 years versus just 2.5 years. Thus resistance in the gut may plateau after several MDAs, rather than continue to increase until it saturates the age-group [22].
What the paper's own numbers show
Method. I matched every macrolide p-value printed in the abstract and Results to its cell in Table 2(A) and checked that each text value is the table value rounded to the printed number of decimals. I then asked whether the abstract's 0.010 could come from any other reading of the same comparison, and listed the Table 2(A) cells closest to 0.010. Finally, I used the printed 95% CI to bound the unadjusted p-value for group 1 vs group 2. The interval is the unadjusted Wilcoxon/Hodges–Lehmann interval on the log scale, and its upper bound is 1.00, the null ratio. By test inversion that corresponds to an unadjusted two-sided p of about 0.05 or more. For a rough numerical check I also approximated the standard error on the log scale from the lower half-width, the upper half-width and the full width of the interval. The community-level data are not published, so the permutation test itself cannot be redone. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Macrolides, 1 vs 2, min-P Padj | Table 2(A) | 0.187 | source value |
| Macrolides, 1 vs 2, Padj | Results | 0.19 | 0.187 to 2 dp = 0.19, consistent |
| Macrolides, 1 vs 2, Padj | Abstract | 0.010 | does not match: the Table value is 18.7 times larger; 0.187 cannot round to 0.010 |
| Significance at 0.05 of the same comparison | Abstract vs Results / Table 2(A) | 0.010 vs 0.19 / 0.187 | 0.010 < 0.05; 0.187 > 0.05. The two classifications differ |
| Macrolides, 1 vs 3, Padj | Results / Table 2(A) | 0.06 / 0.063 | consistent |
| Macrolides, 1 vs 4, Padj | Results / Table 2(A) | 0.01 / 0.014 | consistent |
| Macrolides, 2 vs 3, Padj | Results / Table 2(A) | 0.003 / 0.003 | consistent |
| Macrolides, 2 vs 4, Padj | Results / Table 2(A) | < 0.001 / < 0.001 | consistent |
| Macrolides, 3 vs 4, Padj | Abstract / Table 2(A) | 0.52 / 0.517 | consistent; the abstract's only other Padj (3 vs 4) matches Table 2(A) |
| Table 2(A) cells closest to 0.010 | Table 2(A) | — | Aminoglycosides 2 vs 4 = 0.011; Macrolides 1 vs 4 = 0.014; Macrolides 2 vs 3 = 0.003. Macrolides 1 vs 2 is 0.187 |
| 1 vs 2 CI on the log scale | Results / Abstract | 0.52 to 1.00 / 0.50 to 1.00 | −0.654 to 0.000 / −0.693 to 0.000; both reach the null (log ratio 0) |
| Approximate unadjusted p implied by the Results CI (normal approximation, log scale) | derived | — | 0.18 from the full width; 0.05 to 0.31 depending on which half-width sets the SE |
| Same, from the Abstract CI | derived | — | 0.21 from the full width; 0.05 to 0.35 |
| One-sided reading of the Table value | derived | — | 0.187 / 2 = 0.094, still not 0.010 |
| 1 vs 2 CI lower bound | Abstract vs Results | 0.50 vs 0.52 | differ by 0.02; cannot be resolved without the data |
| Groups 1+2 vs 3+4 CI lower bound | Abstract vs Results | 1.5 vs 1.49 | 1.49 to 1 dp = 1.5, rounding only |
Two points make 0.010 impossible for this comparison, not merely inconsistent. First, a Westfall–Young min-P adjusted p-value can never be smaller than the unadjusted p-value for the same hypothesis. Second, the unadjusted p implied by an interval that reaches 1.00 is about 0.05 or more. Even if the printed 1.00 were rounded up from a bound just below 1 (for example 0.995), the unadjusted p would be only slightly below 0.05, far above 0.010. Every other macrolide Padj in the text matches Table 2(A). The abstract's 0.010 is the only exception, and it sits close to the 1 vs 4 value (0.014, reported in the Results as 0.01).
Figure
Figure 1. Short excerpts for the macrolide comparison of group 1 (4.5 years of azithromycin) with group 2 (2.5 years). The left panel is the abstract; the right panel is the Results text and the Table 2(A) macrolide cell for row "1 vs. 2". Orange values are printed in the paper. The blue lines below are checks derived from the paper's numbers by recompute.py.txt. The Results value is the Table 2(A) value rounded to two decimals. The abstract's 0.010 is 18.7 times smaller than the table value and cannot be reconciled with a 95% CI whose upper bound is 1.00. Read literally, with a fold change of 0.80, it would mean a significant decrease, which the abstract does not report and the Results deny ("did not find a statistically significant difference").
Figure 2. P-values for the group 1 vs group 2 macrolide comparison on one log-scale axis; the dashed line marks 0.05. Orange circles are the three printed values: the abstract (0.010), the Results (0.19) and Table 2(A) (0.187). Blue diamonds are approximate unadjusted two-sided p-values recomputed from each printed 95% CI (normal approximation on the log scale, SE from the full width); the blue bars span the values obtained when the SE is taken from either half-width instead. A min-P adjusted p cannot be smaller than the unadjusted p, and an upper CI bound of 1.00 puts the unadjusted p at about 0.05 or more (the left end of the blue bars). The Results and Table 2(A) values are above that floor; the abstract's 0.010 is well below it.
Why it matters
The comparison of group 1 with group 2 is part of the prespecified primary outcome. It is the evidence behind the abstract's conclusion that "resistance may plateau after 2 to 3 years of treatment", the Author summary's statement that "Longer periods of treatment with azithromycin were not associated with increased macrolide resistance", and the corresponding sentence of the Discussion. With the correct value (Padj = 0.19, Table 2(A) 0.187) these statements are supported as written. The analysis, the body text and the table are consistent with each other, and the paper's conclusions do not change.
The error matters because it sits in the abstract, which is the part most often read on its own, indexed and extracted into reviews and meta-analyses. Read literally, the abstract pairs a fold change of 0.80 and an upper bound of 1.00 with Padj = 0.010. That reads as a statistically significant decrease in macrolide resistance with longer treatment, which the data do not show. The abstract does not report a significant decrease, and the Results deny any significant difference ("did not find a statistically significant difference"). A reader who takes the number at face value would put this comparison in the wrong significance class. The 0.50 vs 0.52 mismatch in the CI lower bound is a separate, minor inconsistency and does not affect any interpretation.
Explanations considered
- A different test or adjustment. This does not account for the numbers. The abstract and the Results both label the value Padj. The Methods define Padj as the Westfall–Young min-P adjusted pairwise Wilcoxon p-value. Table 2(A) prints that value as 0.187.
- The abstract quotes the unadjusted p. This does not account for the numbers. The paper does not print an unadjusted p for this comparison. The printed CI is the unadjusted Wilcoxon/Hodges–Lehmann interval, and its upper bound of 1.00 corresponds to an unadjusted p of about 0.05 or more (roughly 0.18–0.21 by normal approximation with the SE from the full width; 0.05–0.35 across SE choices), not 0.010.
- A one-sided test. This does not account for the numbers. Nothing in the paper describes a one-sided test, and halving 0.187 gives 0.094.
- The FDR (Benjamini–Hochberg) row of Table 2(B). This does not account for the numbers. That row is "NA" for macrolides.
- A different outcome class, population or time point. This does not account for the numbers. The abstract and the Results describe the same groups, the same outcome and the same fold change (0.80). The value 0.010 does not appear in the macrolide 1 vs 2 cell. The nearest Table 2(A) values are aminoglycosides 2 vs 4 (0.011) and macrolides 1 vs 4 (0.014, reported as Padj = 0.01 in the Results).
- A conversion or typesetting artifact in PMC. This is excluded. The publisher's PDF, the publisher's HTML page, PMC and the Europe PMC full-text XML all print the same abstract (0.50 to 1.00, Padj = 0.010) and the same Results (0.52 to 1.00, Padj = 0.19). No minus sign, superscript or decimal separator is involved, and the Table 2(A) column order was checked against the header in the PDF.
- The abstract value is right and the table and Results are wrong. This is not plausible. 0.010 cannot be an adjusted p for an interval that reaches the null. The table and the Results agree with each other, and the Results state that the difference is not statistically significant.
- The abstract claims a significant difference. This is not the case. The abstract says "no significant increase" and reports no significant decrease. The error is the printed number: read literally, 0.010 with a fold change of 0.80 would mean a significant decrease, which the abstract does not report and which conflicts with the Results ("did not find a statistically significant difference") and Table 2(A).
- The CI lower bound (0.50 vs 0.52). This is unexplained. It is a 0.02 difference between two printed versions of the same interval; which one is correct cannot be determined without the data. It does not affect the conclusion.
Status
- Version of record. The publisher's PDF (journals.plos.org printable file, 16 pages, text extracted) and the PLOS HTML article page carry the same abstract text (CI 0.50 to 1.00, Padj = 0.010) and the same Results text (CI 0.52 to 1.00, Padj = 0.19) as the PMC archive. In the PDF's Table 2(A), the macrolide value for "1 vs. 2" is 0.187 and for "1 vs. 4" is 0.014. On the HTML page, Table 2 is an image. The Europe PMC full-text XML matches. Every quote above was confirmed verbatim against the PMC-archived text.
- Supplements. S1 Appendix (pmed.1004386.s001.pdf, 178 pages) holds community-level summary statistics by group and class (Table A), a trial schematic, the group list, the protocols and the statistical analysis plans. It has no pairwise fold changes, CIs or p-values, so it does not resolve the discrepancy. S1 CONSORT Checklist is not relevant. The data availability statement points only to raw sequences (SRA PRJNA1066963). No community-level analysis dataset is published, so the min-P p-value cannot be recomputed.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and its relation records list only the article's own components. PMC and Europe PMC (PMID 38709718) list no correction or comment links. The PLOS article page shows no correction notice. A web search for a correction notice citing the DOI returned nothing relevant. PubPeer could not be checked: both the PubPeer API and its search page returned a Cloudflare block (HTTP 403) to automated access.
- Sources. PMC article PMC11073710; publisher page and PDF at journals.plos.org (doi:10.1371/journal.pmed.1004386); S1 Appendix (pmed.1004386.s001.pdf); Europe PMC full-text XML; Crossref; PubPeer (not reachable).
Suggested correction
- In the abstract ("Methods and findings"), replace "Padj = 0.010" for group 1 vs group 2 with the Table 2(A) value, "Padj = 0.19" (0.187), after confirming it against the original analysis output.
- Reconcile the lower confidence limit for the same comparison: the abstract gives 0.50 and the Results give 0.52. Use whichever the analysis output supports in both places.
- Optionally, harmonize "1.5" (abstract) with "1.49" (Results) for the lower confidence limit of the groups 1+2 vs 3+4 comparison.
No change to the Results, Table 2, the Author summary, the Discussion or the conclusions appears to be needed.
Recomputation script (python recompute.py.txt)
"""Recompute the group 1 vs group 2 macrolide comparison of PMC11073710
(PLoS Med 2024, doi:10.1371/journal.pmed.1004386).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
All inputs are copied from the article as archived in PubMed Central (identical in the
publisher's PDF and HTML): the Abstract, the Results ("Primary outcome") and Table 2(A).
No raw data are used; the community-level data needed to redo the min-P permutation
test are not published.
"""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs (from the paper)
# Table 2(A): Westfall-Young min-P adjusted pairwise Wilcoxon p-values (100,000 permutations).
CLASSES = ["Aminoglycosides", "Cationic antimicrobial peptides", "Macrolides", "Metronidazole",
"Multi drug resistance", "Tetracyclines", "Trimethoprim", "Beta-lactams"]
TABLE2A = { # "<0.001" stored as 0.001 with a flag
"1 vs. 2": [0.218, 0.918, 0.187, 0.255, 0.945, 0.140, 0.112, 0.408],
"1 vs. 3": [0.379, 0.935, 0.063, 0.957, 0.291, 0.908, 0.690, 0.957],
"1 vs. 4": [0.218, 0.791, 0.014, 0.884, 0.848, 0.927, 0.730, 0.957],
"2 vs. 3": [0.055, 0.980, 0.003, 0.689, 0.530, 0.157, 0.690, 0.515],
"2 vs. 4": [0.011, 0.973, "<0.001", 0.840, 0.945, 0.193, 0.611, 0.508],
"3 vs. 4": [0.533, 0.980, 0.517, 0.957, 0.652, 0.908, 0.927, 0.959],
}
MAC = CLASSES.index("Macrolides")
# Macrolide comparisons as printed in the text: (fold change, CI low, CI high, Padj as printed)
RESULTS = { # Results, "Primary outcome"
"1 vs. 4": (2.23, 1.20, 5.17, "0.01"),
"2 vs. 3": (2.14, 1.38, 3.64, "0.003"),
"2 vs. 4": (3.12, 1.78, 7.31, "<0.001"),
"1 vs. 2": (0.80, 0.52, 1.00, "0.19"),
"1 vs. 3": (1.60, 1.00, 3.08, "0.06"),
}
ABSTRACT = { # Abstract, "Methods and findings"
"1 vs. 2": (0.80, 0.50, 1.00, "0.010"),
"3 vs. 4": (1.00, 0.78, 2.35, "0.52"),
}
ALPHA = 0.05
# ------------------------------------------------------------------ 1. text vs table
print("1. Macrolide comparisons: text vs Table 2(A) (min-P adjusted P)")
for pair in TABLE2A:
t = TABLE2A[pair][MAC]
r = RESULTS.get(pair, (None,) * 4)[3]
a = ABSTRACT.get(pair, (None,) * 4)[3]
if isinstance(t, str):
print(f" {pair}: Table {t}; Results {r}; Abstract {a}")
continue
parts = [f" {pair}: Table {t:.3f}"]
for lab, v in (("Results", r), ("Abstract", a)):
if v is None:
continue
dec = len(v.split(".")[1]) # decimals as printed in the text
rounded = round(t, dec)
ok = abs(rounded - float(v)) < 1e-9
parts.append(f"{lab} {v} (Table rounded to {dec} dp = {rounded:.{dec}f}: "
f"{'consistent' if ok else 'DOES NOT MATCH'})")
print("; ".join(parts))
t12 = TABLE2A["1 vs. 2"][MAC]
a12 = float(ABSTRACT["1 vs. 2"][3])
r12 = float(RESULTS["1 vs. 2"][3])
print(f" Abstract 1 vs. 2 Padj {a12:.3f} vs Table {t12:.3f}: ratio Table/Abstract = {t12 / a12:.1f}")
print(f" Significant at {ALPHA}? Abstract {a12} -> {a12 < ALPHA}; Results {r12} -> {r12 < ALPHA};"
f" Table {t12} -> {t12 < ALPHA}")
# ------------------------------------------------------------------ 2. nearest Table 2(A) entries to 0.010
print("\n2. Table 2(A) entries closest to the abstract's 0.010")
cells = []
for pair, row in TABLE2A.items():
for cls, v in zip(CLASSES, row):
if not isinstance(v, str):
cells.append((abs(v - a12), pair, cls, v))
for d, pair, cls, v in sorted(cells)[:3]:
print(f" {cls}, {pair}: {v:.3f} (|difference| = {d:.3f})")
print(f" Macrolides 1 vs. 2 itself: {t12:.3f} (|difference| = {abs(t12 - a12):.3f})")
# ------------------------------------------------------------------ 3. CI vs p-value
print("\n3. What the 95% CI implies about the unadjusted p-value (fold change 0.80)")
est = RESULTS["1 vs. 2"][0]
print(f" log(0.80) = {np.log(est):.3f}; log(upper bound 1.00) = {np.log(1.00):.3f}")
print(" The CI is the (unadjusted) Hodges-Lehmann/Wilcoxon interval. An interval from test inversion")
print(" that reaches the null (ratio 1, log 0) corresponds to an unadjusted two-sided p of about 0.05")
print(" or more; an unadjusted p of 0.010 would require an upper bound clearly below 1.")
APPROX = {}
for src, (e, lo, hi, _) in (("Results CI 0.52-1.00", RESULTS["1 vs. 2"]),
("Abstract CI 0.50-1.00", ABSTRACT["1 vs. 2"])):
res = {}
for lab, se in (("lower half-width", (np.log(e) - np.log(lo)) / 1.96),
("upper half-width", (np.log(hi) - np.log(e)) / 1.96),
("full width", (np.log(hi) - np.log(lo)) / (2 * 1.96))):
z = abs(np.log(e)) / se
p = 2 * norm.sf(z)
res[lab] = p
print(f" {src}, SE from {lab}: SE = {se:.3f}, z = {z:.2f}, approx. unadjusted p = {p:.3f}")
APPROX[src] = res
lo_p = min(min(r.values()) for r in APPROX.values())
hi_p = max(max(r.values()) for r in APPROX.values())
print(f" Range of normal-approximation p-values: {lo_p:.3f} to {hi_p:.3f}")
print(" A Westfall-Young min-P adjusted p is never smaller than the unadjusted p for the same")
print(f" comparison, so Padj = {a12:.3f} is below every value compatible with the printed CI.")
print(f" Halving the Table value (one-sided reading): {t12 / 2:.4f}, still not 0.010")
# ------------------------------------------------------------------ 4. minor CI mismatches
print("\n4. Other abstract-vs-Results numbers")
print(f" 1 vs. 2 CI lower bound: Abstract 0.50 vs Results 0.52, difference {0.52 - 0.50:.2f}")
print(f" Groups 1+2 vs 3+4 CI lower bound: Abstract 1.5 vs Results 1.49 -> 1.49 to 1 dp = {round(1.49, 1)}")
print(f" 1 vs. 2 CI on log scale (Results): {np.log(0.52):.3f} to {np.log(1.00):.3f}; "
f"(Abstract): {np.log(0.50):.3f} to 0.000")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: abstract vs Results / Table 2 text panels
fig = plt.figure(figsize=(7.2, 5.3), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(-1.0, 4.3)
ax.axis("off")
ax.set_facecolor(BG)
ax.text(0.2, 4.05, "Macrolide resistance determinants, group 1 (4.5 y azithromycin) vs group 2 (2.5 y)",
fontsize=10.5, weight="bold", va="center")
for x0, title in ((0.2, "Abstract (Methods and findings)"), (3.75, "Results and Table 2(A)")):
ax.add_patch(FancyBboxPatch((x0, 1.35), 3.25, 2.35, boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec="#c9c8c3", lw=1))
ax.text(x0 + 0.15, 3.5, title, fontsize=10, weight="bold", color=INK2, va="center")
L = 0.35
ax.text(L, 3.12, "\u201cno significant increase in", fontsize=10, va="center")
ax.text(L, 2.87, "macrolide resistance\u201d", fontsize=10, va="center")
ax.text(L, 2.5, "fold change 0.80,", fontsize=10, va="center")
ax.text(L, 2.25, "95% CI 0.50 to 1.00,", fontsize=10, va="center")
ax.text(L, 1.75, f"Padj = {a12:.3f}", fontsize=13, weight="bold", color=ORANGE, va="center")
R = 3.9
ax.text(R, 3.12, "\u201cdid not find a statistically", fontsize=10, va="center")
ax.text(R, 2.87, "significant difference\u201d", fontsize=10, va="center")
ax.text(R, 2.5, "0.80-fold, 95% CI: 0.52 to 1.00,", fontsize=10, va="center")
ax.text(R, 2.25, f"Padj = {r12:.2f}", fontsize=13, weight="bold", color=ORANGE, va="center")
ax.text(R, 1.75, f"Table 2(A), Macrolides, 1 vs. 2: {t12:.3f}", fontsize=10.5, weight="bold",
color=ORANGE, va="center")
ax.add_patch(FancyArrowPatch((1.75, 1.75), (3.85, 1.75), arrowstyle="<|-|>", mutation_scale=12,
color=INK2, lw=1.5))
ax.text(2.6, 1.93, "conflicting Padj", fontsize=10, color=INK2, ha="center",
va="center", style="italic")
ax.text(0.2, 0.98, "Same comparison, same Padj definition (Westfall\u2013Young min-P, Wilcoxon rank-sum):",
fontsize=10, color=INK, va="center")
lines = [
f"round({t12:.3f}, 2) = {round(t12, 2):.2f}: matches the Results; it cannot give {a12:.3f} "
f"(Table value is {t12 / a12:.1f}\u00d7 larger)",
f"Read literally (fold change 0.80), {a12:.3f} would mean a significant decrease; the Results",
"deny one: \u201cdid not find a statistically significant difference\u201d",
"An upper CI bound of 1.00 implies an unadjusted p of about 0.05 or more, and a",
"min-P adjusted p cannot be smaller than the unadjusted p for the same comparison",
f"Nearest macrolide entry in Table 2(A): 1 vs. 4 = {TABLE2A['1 vs. 4'][MAC]:.3f} "
f"(Results: Padj = 0.01)",
]
for i, s in enumerate(lines):
ax.text(0.35, 0.72 - 0.25 * i, s, fontsize=10, color=BLUE, va="center")
ax.text(7.0, -0.95, "orange = as printed in the paper; blue = checks derived from the paper's numbers",
fontsize=10, color=INK2, ha="right", va="bottom")
save(fig, "fig1")
# Figure 2: p-values for the same comparison on one log axis
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
ax.set_facecolor(BG)
rows = [
("Abstract: Padj", a12, None, ORANGE, "o", f"{a12:.3f}"),
("Results: Padj", r12, None, ORANGE, "o", f"{r12:.2f}"),
("Table 2(A): min-P Padj", t12, None, ORANGE, "o", f"{t12:.3f}"),
("Unadjusted p implied by\nResults CI 0.52–1.00", APPROX["Results CI 0.52-1.00"]["full width"],
(min(APPROX["Results CI 0.52-1.00"].values()), max(APPROX["Results CI 0.52-1.00"].values())),
BLUE, "D", None),
("Unadjusted p implied by\nAbstract CI 0.50–1.00", APPROX["Abstract CI 0.50-1.00"]["full width"],
(min(APPROX["Abstract CI 0.50-1.00"].values()), max(APPROX["Abstract CI 0.50-1.00"].values())),
BLUE, "D", None),
]
ys = np.arange(len(rows))[::-1]
for y, (lab, v, rng, c, m, txt) in zip(ys, rows):
if rng:
ax.plot(rng, [y, y], color=c, lw=2, solid_capstyle="round", zorder=2)
txt = f"{v:.2f} (range {rng[0]:.2f}–{rng[1]:.2f})"
ax.plot(v, y, marker=m, ms=9, color=c, mec=BG, mew=1.5, zorder=3, ls="none")
if rng:
ax.text(v, y + 0.32, txt, ha="center", va="center", fontsize=10, color=INK)
else:
ax.text(v * 1.18, y, txt, va="center", fontsize=10, color=INK)
ax.axvline(0.05, color=INK2, lw=0.8, ls="--", zorder=1)
ax.text(0.05 * 0.93, ys[0] + 0.55, "0.05", ha="right", va="center", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(0.004, 1.0)
ax.set_ylim(-0.6, len(rows) - 0.3)
ax.set_xticks([0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1.0])
ax.set_xticklabels(["0.005", "0.01", "0.02", "0.05", "0.1", "0.2", "0.5", "1"])
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_xlabel("p-value, macrolides, group 1 vs group 2 (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#bdbcb7")
ax.tick_params(axis="y", length=0)
ax.plot([], [], "o", color=ORANGE, ms=9, label="as printed in the paper")
ax.plot([], [], "D", color=BLUE, ms=9, label="recomputed from the printed CI (normal approx.)")
ax.legend(loc="lower left", bbox_to_anchor=(-0.45, 1.02), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")
print("\nFigures written: fig1.svg, fig1.png, fig2.svg, fig2.png")
The abstract says RNF213 loss lowers IFN-β-induced ISG expression, but Fig 2A shows no difference
PLoS Pathog (2025) · doi:10.1371/journal.ppat.1013333 · PMC12251097
Summary
The abstract states that RNF213 deficiency "decreases the expression of interferon-stimulated genes (ISGs) in response to IFN-β stimulation and Acinetobacter baumannii infection." For the IFN-β condition the Results say the opposite: the IFN-β-induced increases in Cxcl9 and Cxcl10 "were comparable between WT and Rnf213–/– MSFs," and all 8 IFN-β comparisons of Cxcl9/Cxcl10 in Fig 2A (two cell types, two genes, two time points) are labelled "ns." Bar heights read off the published Fig 2A give a knockout-to-wild-type ratio of about 0.91–1.31 (geometric mean 1.04) under IFN-β, with the knockout numerically higher in half of the comparisons, versus about 0.30–0.63 (geometric mean 0.44, all 16 significant) under A. baumannii or LPS. This matters because the paper uses the intact IFN-β response as its evidence that RNF213 acts upstream of the IFN-α/β receptor (IFNAR), so the abstract sentence points to the opposite location in the pathway from the one the paper argues for.
What the paper reports
In the abstract:
RNF213 deficiency impairs type I interferon (IFN-I) production and decreases the expression of interferon-stimulated genes (ISGs) in response to IFN-β stimulation and Acinetobacter baumannii infection.
— Abstract
In the body, the same experiment (Fig 2A) is described as follows:
The increases in the expression of Cxcl9 and Cxcl10 induced by A. baumannii infection and LPS stimulation were much lower in Rnf213–/– MSFs than in WT MSFs (Fig 2A).
However, the increases in the expression of Cxcl9 and Cxcl10 induced by IFN-β treatment were comparable between WT and Rnf213–/– MSFs (Fig 2A).
Taken together, our results indicate that RNF213 positively regulates IFN-I signaling possibly by mediating events upstream of IFNAR binding.
— Results, section "IFN-I signaling is impaired in Rnf213–/– MSFs and BMECs"
RNF213 deficiency inhibited TBK1 phosphorylation and IFN-I production in response to LPS stimulation and A. baumannii infection but not in response to IFN-β treatment, suggesting that RNF213 positively regulates IFN-I signaling by acting on targets upstream of IFNAR (Figs 2D, 2E, S3C and S3D).
— Results, section "RNF213 targets TRAF3", first sentence
The cited figure is described in its legend as:
qRT-PCR analysis of Rnf213, Cxcl9 and Cxcl10 in WT and Rnf213–/– MSFs and BMECs without treatment or infected with A. baumannii (50 MOI); or stimulated with LPS (200 ng/mL) and IFN-β (2 ng/mL) for indicated times (n = 4 technical replicates; 3 independent experiments).
— Fig 2 legend, panel A
Each quotation was checked character for character against the article as archived in PubMed Central (src/show.py PMC12251097). Each one occurs exactly once; recompute.py.txt reruns that check.
What the paper's own numbers show
Method. The claim is about direction (a decrease versus no change), so the check has two parts.
- The significance label printed above each wild-type/knockout pair in Fig 2A was transcribed exactly, for every stimulated Cxcl9 and Cxcl10 comparison (3 stimuli × 2 cell types × 2 genes × 2 time points = 24). The legend states these labels come from two-sided Student's t-tests without multiple-comparison correction.
- To show the size of each difference, the article's Data Availability statement reports no numeric source data, so the bar heights (gene/Hprt) were measured on the publisher's large Fig 2 image. For each panel a straight line was fit through the printed y-axis tick marks (largest residual 0.2% of the axis maximum), and the top edge of each outlined bar was read against it. Where the replicate dots cover a bar's top, the value is bracketed between the top of the bar outline and the middle of the dot cluster; that range is carried through to the ratio. These readings are approximate. The conclusion does not rest on them: the printed "ns" labels and the Results text already state there is no difference under IFN-β.
recompute.py.txt prints every value below and redraws both figures.
| Quantity | Location | Reported | Recomputed from the paper's own data |
|---|---|---|---|
| Effect of RNF213 deficiency on ISG expression after IFN-β | Abstract | "decreases the expression of interferon-stimulated genes (ISGs) in response to IFN-β stimulation" | Not supported: no IFN-β ISG comparison in the paper shows a decrease (rows below) |
| Same effect, as described in the body | Results, Fig 2A paragraph | "comparable between WT and Rnf213–/– MSFs" | Agrees with Fig 2A |
| Cxcl9/Cxcl10 after IFN-β, MSFs + BMECs, 3 h and 6 h (8 comparisons) | Fig 2A, IFN-β row | All 8 labelled "ns" | Knockout ÷ WT ≈ 0.91–1.31; geometric mean 1.04; knockout numerically higher in 4 of 8 |
| MSF Cxcl10, 6 h, IFN-β | Fig 2A | "ns" | Knockout ÷ WT ≈ 1.31 (≈ 0.61 vs ≈ 0.47; knockout higher) |
| Cxcl9/Cxcl10 after A. baumannii, MSFs + BMECs, 3 h and 6 h (8 comparisons) | Fig 2A, A. baumannii row | All *, ** or *** | Knockout ÷ WT ≈ 0.35–0.63; geometric mean 0.44 |
| Cxcl9/Cxcl10 after LPS, MSFs + BMECs, 6 h and 12 h (8 comparisons) | Fig 2A, LPS row | All ** or *** | Knockout ÷ WT ≈ 0.30–0.57; geometric mean 0.44 |
| A. baumannii and LPS together (16 comparisons) | Fig 2A | 16 of 16 significant | Knockout ÷ WT ≈ 0.30–0.63 (largest upper bound of a reading range 0.72); geometric mean 0.44 |
| IFN-I production / TBK1 phosphorylation after IFN-β | Results, "RNF213 targets TRAF3" | "but not in response to IFN-β treatment" | No panel cited there (Figs 2D, 2E, S3C–D) includes IFN-β-stimulated cells; their legends list only untreated, A. baumannii and LPS conditions |
So under bacterial infection or LPS, knockout cells induce roughly 40–45% as much Cxcl9/Cxcl10 as wild-type cells (geometric mean 0.44 for both stimuli), and every one of the 16 comparisons is marked significant. Under IFN-β the two genotypes are indistinguishable: the ratios scatter on both sides of 1 (geometric mean 1.04; knockout numerically higher in 4 of 8), and all 8 are marked "ns." The paper's body reports exactly this pattern. Only the abstract sentence departs from it.
Figure
Figure 1. The conflicting statements, side by side. Left (orange): the abstract sentence, with the IFN-β ISG clause highlighted. Right (blue): the Fig 2A Results sentence, the first sentence of "RNF213 targets TRAF3" (which concerns TBK1 phosphorylation and IFN-I production and cites Figs 2D, 2E, S3C–D), and a count of the significance labels in the IFN-β row of Fig 2A. The abstract's ISG clause and the Fig 2A sentence describe the same experiment and disagree only on the IFN-β condition; on A. baumannii and LPS the abstract and Results agree.
Figure 2. Size of the knockout effect for each stimulus in Fig 2A. Each filled blue dot is the ratio Rnf213–/– ÷ WT of mean Cxcl9 or Cxcl10 mRNA (gene/Hprt) for one cell type, gene and time point, from bar heights measured on the published Fig 2A; the short blue line is the range left by the measurement where replicate dots cover a bar top. Open blue diamonds are the per-stimulus geometric means. The orange label at right is the significance label printed in Fig 2A for that pair. The x-axis is logarithmic and the vertical line at 1 marks no difference. The 8 IFN-β ratios cluster around 1 (geometric mean 1.04) and all are "ns"; the 16 A. baumannii/LPS ratios sit between about 0.30 and 0.63 (geometric mean 0.44 for each stimulus) and all are *, ** or ***. The two clusters are far apart relative to the reading uncertainty.
Why it matters
- The abstract conflicts with the paper's own mechanistic inference. The body argues that RNF213 acts upstream of IFNAR, at TRAF3, precisely because knockout cells respond normally to exogenous IFN-β while producing less IFN-I after bacterial or LPS stimulation ("suggesting that RNF213 positively regulates IFN-I signaling by acting on targets upstream of IFNAR"). If RNF213 deficiency also reduced ISG induction by IFN-β, as the abstract states, RNF213 would be required downstream of the receptor, and this experiment would no longer support the upstream placement. A reader who sees only the abstract, as in indexing services, gets the mechanism wrong on this point.
- The main conclusions are unaffected and are in fact supported by the correct reading. RNF213 promotes K27-linked polyubiquitination of TRAF3 at K160, drives IFN-I production, and maintains blood–brain barrier integrity during A. baumannii infection. The title, author summary, figures, Results and Discussion all agree with the Results passages quoted above. The error is confined to one abstract sentence. The internal rescue experiment (Fig 4C), in which IFN-β restores Claudin-5/Occludin/Pecam1 expression in Rnf213–/– BMECs, likewise requires an intact IFN-β response and so agrees with the body, not the abstract.
- No numeric estimate is affected. The problem is a wrong statement of direction for one condition, not a miscalculated value.
Explanations considered
- "In response to IFN-β stimulation" modifies only "IFN-I production," not the ISG clause. This does not resolve the conflict. The paper never measures IFN-I production after IFN-β stimulation (the ELISA panels Fig 2B–C cover only A. baumannii and LPS), and IFN-β is itself a type I interferon, so "IFN-I production in response to IFN-β" is not a coherent separate readout. Under the parsing that attaches the phrase to ISG expression, it contradicts Fig 2A; under the other parsing, the Results already say IFN-I signaling is unaffected by IFN-β. Neither reading matches the body.
- The abstract refers to a different cell type or experiment. The only experiment in which Rnf213–/– cells are stimulated with IFN-β and ISGs are measured is Fig 2A, which covers both MSFs and BMECs; all 8 Cxcl9/Cxcl10 comparisons are "ns." Fig 1B–C uses IFN-β to induce Rnf213 in WT versus Ifnar–/– cells, a different genotype contrast. Fig 4C uses IFN-β pretreatment of BMECs, and the Results/Discussion describe the outcome as a rescue ("reversed by treatment with IFN-β" / "rescued by treatment with IFN-β"), which is the opposite of a reduced response. None of the eight supplementary figure legends reports IFN-β-stimulated ISG data in knockout versus WT cells.
- The Rnf213 transcript itself. In the IFN-β row of Fig 2A the Rnf213 panels are "***" because the gene is deleted in the knockout. Rnf213 is IFN-inducible (Fig 1B–C), but it is the targeted gene, not an ISG readout of the knockout's downstream signalling capacity, and neither the abstract nor the Results presents it as one.
- Low statistical power (absence of evidence). One could argue that "ns" with n = 4 technical replicates does not exclude a small decrease. But the abstract asserts a decrease, which "ns" does not support either way, and the point estimates show no tendency toward one: the geometric-mean ratio is 1.04 and the knockout is numerically higher in half the comparisons. With the same design the same figure detects 2- to 3-fold decreases under A. baumannii and LPS.
- Conversion or typesetting artifact. Ruled out. The same wording appears in the PMC-archived text, the Europe PMC full-text XML and the publisher's HTML version of record (labelled "Version 2").
- Already corrected or discussed. No correction, erratum, expression of concern or retraction was found (see Status).
None of these accounts for the conflict. The most plausible benign reading is that the abstract meant "LPS" where it wrote "IFN-β," since the Results support a decrease under LPS. That is a conjecture about wording and does not change the fact that the abstract as published contradicts the Results.
Status
As of 2026-10-23:
- Version of record: checked. The publisher's article page (journals.plos.org, reached through doi.org) offers "Version 2 / Version 1 / Uncorrected Proof." The current version carries all quoted passages verbatim: the abstract sentence, both Fig 2A Results sentences ("much lower" and "comparable"), the "but not in response to IFN-β treatment" sentence, and the Fig 4C "reversed by treatment with IFN-β" sentence. The same passages appear once each in the Europe PMC full-text XML.
- Figure data: Fig 2A has no numeric source data (Data Availability: all data are within the manuscript and its supporting files). The published Fig 2 image was measured for Fig 2 of this write-up, and the significance labels were transcribed from it. Every stimulated Cxcl9/Cxcl10 comparison was included.
- Supplements: the legends of all supporting items (S1–S8 Fig, S1–S2 Table) were read. None reports IFN-β-stimulated ISG data in Rnf213–/– versus WT cells. S3C–D, cited by the "RNF213 targets TRAF3" sentence, covers only untreated, A. baumannii and LPS conditions.
- Corrections and notices:
src/known.pyreturns only a publisher "New version" record dated 2025-07-11, which is PLOS's routine versioning (Uncorrected Proof → Version 1 → Version 2), not a correction. PubMed Central and Europe PMC list no notices. The publisher's page shows no correction, erratum, expression of concern or retraction. - Comments and PubPeer: the PLOS reader-comments page shows "Reader Comments (0)." PubPeer could not be queried directly (its search and API returned HTTP 403 / Cloudflare), and the one available web search for the title plus "pubpeer" could not run because the session's search budget was exhausted, so a PubPeer thread cannot be positively ruled out.
- Peer review: the open peer-review history published with the article was read (Europe PMC XML sub-articles r001–r004) along with the author response attachment. In Decision Letter 1, Reviewer #3 proposed the wording that became the "RNF213 targets TRAF3" sentence, including "but not in response to IFN-β treatment," and the authors adopted it. In the author response, the authors themselves wrote that "RNF213 deficiency did not affect the expression of ISGs during IFN-β stimulation (Fig 2)," which agrees with the Results and not the abstract. No review comment or author response addresses the abstract's IFN-β wording.
- Sources: PMC12251097 (
src/show.py);src/known.py 10.1371/journal.ppat.1013333; https://doi.org/10.1371/journal.ppat.1013333; the PLOS article, figure-image and reader-comments pages; https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12251097/fullTextXML (including peer-review sub-articles and the author response supplementary file).
Suggested correction
Restrict the IFN-β clause in the abstract to what the data show. For example:
RNF213 deficiency impairs type I interferon (IFN-I) production and decreases the expression of interferon-stimulated genes (ISGs) in response to Acinetobacter baumannii infection and LPS stimulation, whereas ISG induction by IFN-β is unaffected.
Alternatively, delete "IFN-β stimulation and" from the sentence. Separately, the first sentence of "RNF213 targets TRAF3" states that TBK1 phosphorylation and IFN-I production were unaffected after IFN-β treatment, yet none of the panels it cites (Figs 2D, 2E, S3C, S3D) includes IFN-β-treated samples; Fig 2A holds only Cxcl9/Cxcl10/Rnf213 mRNA after IFN-β, not TBK1 or IFN-I readouts. That sentence could restrict its IFN-β clause to ISG induction and cite Fig 2A (for example, "…but did not affect Cxcl9 and Cxcl10 induction by IFN-β treatment (Fig 2A)"), or drop the IFN-β clause for the TBK1/IFN-I readouts, for which no IFN-β-treated samples are shown.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12251097 (PLoS Pathog 2025, doi:10.1371/journal.ppat.1013333)
# Abstract vs Results / Fig 2A: does RNF213 deficiency lower ISG expression after IFN-beta?
#
# Run: python recompute.py.txt
# Prints every number used in writeup.md and regenerates fig1.{svg,png} and fig2.{svg,png}
# in the folder that holds this file. Needs numpy and matplotlib.
#
# Inputs
# * Quotes: copied verbatim from the article as archived in PubMed Central (PMC12251097).
# When run inside the /home/ubuntu/papers repo, the script re-checks each quote against
# the archived full text with src/show.py (skipped elsewhere).
# * Fig 2A: the article gives no numeric source data (Data Availability: all data are in
# the manuscript and its supporting files). The pixel coordinates below were measured on
# the publisher's large Fig 2 image
# https://journals.plos.org/plospathogens/article/figure/image?size=large&id=10.1371/journal.ppat.1013333.g002
# (2232 x 2576 px), by locating the y-axis tick marks and the top edge of each outlined
# bar (blue = WT, red = Rnf213-/-). Where the replicate dots hide a bar's top edge, the
# top is bracketed between the middle of the dot cluster and the highest visible pixel of
# the bar outline; otherwise +/-1 px. Tick values and significance labels are transcribed
# exactly as printed. All 24 stimulated Cxcl9/Cxcl10 comparisons are included (3 stimuli
# x 2 cell types x 2 genes x 2 time points); untreated (0 h) bars are not used.
# The finding does not rest on these measurements: the printed 'ns' labels and the
# Results text say the same thing. The measurements only show the size of each difference.
import math, os, statistics, subprocess
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ quotes
QUOTES = {
"abstract": ("Abstract",
"RNF213 deficiency impairs type I interferon (IFN-I) production and decreases the "
"expression of interferon-stimulated genes (ISGs) in response to IFN-β stimulation "
"and Acinetobacter baumannii infection."),
"results_down": ("Results, 'IFN-I signaling is impaired in Rnf213–/– MSFs and BMECs'",
"The increases in the expression of Cxcl9 and Cxcl10 induced by A. baumannii infection "
"and LPS stimulation were much lower in Rnf213–/– MSFs than in WT MSFs (Fig 2A)."),
"results_ifnb": ("Results, 'IFN-I signaling is impaired in Rnf213–/– MSFs and BMECs'",
"However, the increases in the expression of Cxcl9 and Cxcl10 induced by IFN-β treatment "
"were comparable between WT and Rnf213–/– MSFs (Fig 2A)."),
"results_bmec": ("Results, 'IFN-I signaling is impaired in Rnf213–/– MSFs and BMECs'",
"Similar patterns of A. baumannii infection- and LPS stimulation-induced Cxcl9 and "
"Cxcl10 expression were observed in BMECs (Fig 2A)."),
"results_upstream": ("Results, 'IFN-I signaling is impaired in Rnf213–/– MSFs and BMECs'",
"Taken together, our results indicate that RNF213 positively regulates IFN-I signaling "
"possibly by mediating events upstream of IFNAR binding."),
"results_traf3": ("Results, 'RNF213 targets TRAF3', first sentence",
"RNF213 deficiency inhibited TBK1 phosphorylation and IFN-I production in response to LPS "
"stimulation and A. baumannii infection but not in response to IFN-β treatment, suggesting "
"that RNF213 positively regulates IFN-I signaling by acting on targets upstream of IFNAR "
"(Figs 2D, 2E, S3C and S3D)."),
"fig2_legend": ("Fig 2 legend, panel A",
"qRT-PCR analysis of Rnf213, Cxcl9 and Cxcl10 in WT and Rnf213–/– MSFs and BMECs without "
"treatment or infected with A. baumannii (50 MOI); or stimulated with LPS (200 ng/mL) and "
"IFN-β (2 ng/mL) for indicated times (n = 4 technical replicates; 3 independent experiments)."),
"fig4c": ("Results, 'RNF213 modulates blood–brain barrier integrity via IFN-I signaling'",
"Notably, the reduced expression of Claudin-5, Occludin, and Pecam1 in Rnf213–/– BMECs "
"was reversed by treatment with IFN-β (Fig 4C)."),
}
def check_quotes():
repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
print("Quote check: src/show.py not available here; skipped.")
return
try:
full = subprocess.run([py, "src/show.py", "PMC12251097", "--full"], cwd=repo,
capture_output=True, text=True, timeout=300).stdout
except Exception as e:
print(f"Quote check: could not load archived text ({e}); skipped.")
return
norm = lambda s: " ".join(s.split())
full_n = norm(full)
print("Quote check against the PMC-archived full text (occurrences):")
for key, (loc, q) in QUOTES.items():
print(f" {key:17s} {full_n.count(norm(q))} ({loc})")
# ------------------------------------------------------------------ Fig 2A pixel data
# Panel: (stimulus, cell, gene) -> y-axis ticks [(pixel y, printed value)], and per time point:
# (time, WT top-edge pixel range, KO top-edge pixel range, printed significance label)
PANELS = [
("A. baumannii", "MSF", "Cxcl9", [(150.5, .18), (217.5, .12), (285.0, .06), (352.5, 0)],
[("3 h", (296, 296), (320, 320), "***"), ("6 h", (195, 195), (297, 297), "***")]),
("A. baumannii", "MSF", "Cxcl10", [(153.5, 4), (203.5, 3), (253.5, 2), (303.5, 1), (353.5, 0)],
[("3 h", (295, 303), (319, 319), "***"), ("6 h", (190, 190), (277, 277), "***")]),
("A. baumannii", "BMEC", "Cxcl9", [(152.5, .18), (220.5, .12), (287.5, .06), (355.0, 0)],
[("3 h", (262, 262), (321.5, 324), "**"), ("6 h", (205, 205), (302.5, 304), "***")]),
("A. baumannii", "BMEC", "Cxcl10", [(150.5, 1.5), (218.5, 1.0), (285.5, .5), (353.0, 0)],
[("3 h", (271, 271), (307, 307), "*"), ("6 h", (177, 177), (292, 292), "***")]),
("LPS", "MSF", "Cxcl9", [(435.5, .20), (486.0, .15), (536.5, .10), (587.0, .05), (637.5, 0)],
[("6 h", (553, 553), (586.5, 593), "**"), ("12 h", (492, 492), (594, 594), "**")]),
("LPS", "MSF", "Cxcl10", [(435.5, 4), (485.5, 3), (535.5, 2), (585.5, 1), (635.0, 0)],
[("6 h", (498, 498), (567, 567), "**"), ("12 h", (564, 564), (614, 614), "**")]),
("LPS", "BMEC", "Cxcl9", [(434.0, .24), (501.5, .16), (569.0, .08), (636.5, 0)],
[("6 h", (562, 562), (595, 596), "**"), ("12 h", (480, 480), (564, 564), "***")]),
("LPS", "BMEC", "Cxcl10", [(434.5, 1.8), (501.5, 1.2), (568.5, .6), (636.0, 0)],
[("6 h", (548.5, 558), (595.5, 596), "**"), ("12 h", (485, 485), (568, 568), "**")]),
("IFN-β", "MSF", "Cxcl9", [(738.5, .04), (789.0, .03), (839.5, .02), (890.0, .01), (940.5, 0)],
[("3 h", (843, 844), (826, 826), "ns"), ("6 h", (782, 785), (774, 774), "ns")]),
("IFN-β", "MSF", "Cxcl10", [(739.5, 2), (790.5, 1.5), (840.5, 1), (891.5, .5), (941.5, 0)],
[("3 h", (763.5, 767), (779, 779), "ns"), ("6 h", (894, 895), (878, 882), "ns")]),
("IFN-β", "BMEC", "Cxcl9", [(739.5, .06), (806.5, .04), (874.5, .02), (941.5, 0)],
[("3 h", (865, 865), (853, 861), "ns"), ("6 h", (764.5, 769), (777, 777), "ns")]),
("IFN-β", "BMEC", "Cxcl10", [(734.5, 2.4), (803.5, 1.6), (871.5, .8), (939.5, 0)],
[("3 h", (878, 878), (879, 879), "ns"), ("6 h", (767, 767), (782, 782), "ns")]),
]
PAD = 1.0 # px added on each side of every top-edge range (line width / anti-aliasing)
def calib(ticks):
"""Least-squares line value = a + b * pixel_y through the printed tick marks."""
ys = [t[0] for t in ticks]; vs = [t[1] for t in ticks]
my, mv = statistics.fmean(ys), statistics.fmean(vs)
b = sum((y - my) * (v - mv) for y, v in zip(ys, vs)) / sum((y - my) ** 2 for y in ys)
a = mv - b * my
resid = max(abs(a + b * y - v) for y, v in zip(ys, vs))
return a, b, resid
def measure():
out, worst = [], 0.0
for stim, cell, gene, ticks, bars in PANELS:
a, b, resid = calib(ticks)
worst = max(worst, resid / max(t[1] for t in ticks))
val = lambda y: a + b * y # b < 0: smaller pixel y = taller bar
for t, (w0, w1), (k0, k1), lab in bars:
wt_mid, ko_mid = val((w0 + w1) / 2), val((k0 + k1) / 2)
wt_hi, wt_lo = val(w0 - PAD), val(w1 + PAD)
ko_hi, ko_lo = val(k0 - PAD), val(k1 + PAD)
out.append(dict(stim=stim, cell=cell, gene=gene, time=t, label=lab,
wt=wt_mid, ko=ko_mid, r=ko_mid / wt_mid,
r_lo=ko_lo / wt_hi, r_hi=ko_hi / wt_lo))
print(f"\nAxis calibration: largest tick residual = {worst:.2%} of axis maximum")
return out
def summarise(rows, stim):
sel = [d for d in rows if d["stim"] == stim]
print(f"\n{stim}: {len(sel)} Cxcl9/Cxcl10 comparisons in Fig 2A (Rnf213-/- / WT)")
for d in sel:
print(f" {d['cell']:4s} {d['gene']:6s} {d['time']:>4s} WT {d['wt']:.3g} KO {d['ko']:.3g} "
f"ratio {d['r']:.2f} [{d['r_lo']:.2f}-{d['r_hi']:.2f}] printed: {d['label']}")
rs = [d["r"] for d in sel]
gm = statistics.geometric_mean(rs)
n_ns = sum(d["label"] == "ns" for d in sel)
n_above = sum(r > 1 for r in rs)
n_cover1 = sum(d["r_lo"] <= 1 <= d["r_hi"] for d in sel)
print(f" ratio range {min(rs):.2f}-{max(rs):.2f}; geometric mean {gm:.2f}; "
f"printed 'ns' {n_ns}/{len(sel)}; KO above WT in {n_above}/{len(sel)}; "
f"measurement interval includes 1 in {n_cover1}/{len(sel)}")
return sel, gm
# ------------------------------------------------------------------ style
BG, TXT, TXT2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": TXT2, "ytick.color": TXT2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
print(f"wrote {stem}.svg and {stem}.png")
# ------------------------------------------------------------------ figure 1
def fig1(n_ns, n_tot):
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.6); ax.axis("off")
def panel(x, w, title, sub, body, accent):
ax.add_patch(FancyBboxPatch((x, 0.92), w, 3.55, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=accent, lw=1.5))
ax.text(x + 0.14, 4.33, title, fontsize=11.5, fontweight="bold", va="top", color=TXT)
ax.text(x + 0.14, 4.03, sub, fontsize=10, va="top", color=accent)
yy = 3.68
for line, style in body:
ax.text(x + 0.14, yy, line, fontsize=10, va="top",
color={"n": TXT, "b": accent, "s": TXT2}[style],
fontweight="bold" if style == "b" else "normal",
fontstyle="italic" if style == "s" else "normal")
yy -= 0.235
left = [("“RNF213 deficiency impairs type I", "n"),
("interferon (IFN-I) production and", "n"),
("decreases the expression of", "b"),
("interferon-stimulated genes (ISGs)", "b"),
("in response to IFN-β stimulation", "b"),
("and Acinetobacter baumannii", "n"),
("infection.”", "n"),
("", "n"), ("", "n"), ("", "n"),
("States: ISGs lower in Rnf213–/–", "s"),
("cells after IFN-β.", "s")]
right = [("Fig 2A paragraph:", "s"),
("“…Cxcl9 and Cxcl10 induced by IFN-β", "n"),
("treatment were comparable", "b"),
("between WT and Rnf213–/– MSFs …”", "b"),
("“RNF213 targets TRAF3”:", "s"),
("“…inhibited TBK1 phosphorylation and", "n"),
("IFN-I production … but not in", "b"),
("response to IFN-β treatment …”", "b"),
("", "n"),
("Fig 2A, IFN-β row (MSFs and BMECs):", "s"),
(f"{n_ns} of {n_tot} Cxcl9/Cxcl10 comparisons", "n"),
("printed “ns”.", "n")]
panel(0.15, 3.05, "Abstract", "reported claim", left, ORANGE)
panel(4.0, 3.05, "Results and Fig 2A", "the paper's own text and labels", right, BLUE)
ax.add_patch(FancyArrowPatch((3.24, 3.0), (3.96, 3.0), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=TXT))
ax.text(3.6, 3.3, "conflict", ha="center", fontsize=10, fontweight="bold", color=TXT)
ax.text(3.6, 2.73, "IFN-β\nclause\nonly", ha="center", va="top", fontsize=10, color=TXT2)
ax.text(0.15, 0.47,
"After A. baumannii infection or LPS, abstract and Results agree: Fig 2A shows lower Cxcl9/\n"
"Cxcl10 in Rnf213–/– cells (16 of 16 comparisons marked *, ** or ***). The conflict is the\n"
"IFN-β condition, the only one in which Rnf213–/– cells were given IFN-β and ISGs measured.",
fontsize=10, color=TXT2, va="center")
save(fig, "fig1")
# ------------------------------------------------------------------ figure 2
def fig2(groups):
rows = []
for gname, sel, gm in groups:
rows.append(("header", gname, gm))
rows += [("pt", d, None) for d in sel]
n = len(rows)
fig, ax = plt.subplots(figsize=(7.2, 8.4))
fig.subplots_adjust(left=0.25, right=0.85, top=0.90, bottom=0.09)
ax.set_xscale("log")
ax.set_xlim(0.15, 2.0)
ax.set_ylim(n - 0.4, -0.8)
ax.axvline(1.0, color=TXT2, lw=0.8, zorder=1)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ticks = [0.2, 0.3, 0.5, 0.7, 1.0, 1.5, 2.0]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(TXT2)
ax.tick_params(axis="y", length=0)
ylabels, first = [], True
for i, (kind, d, gm) in enumerate(rows):
if kind == "header":
ylabels.append("")
ax.text(-0.02, i, d, transform=ax.get_yaxis_transform(), ha="right", va="center",
fontsize=10.5, fontweight="bold", color=TXT)
ax.plot([gm], [i], marker="D", ms=9, mfc="white", mec=BLUE, mew=2, ls="none", zorder=3,
label="geometric mean of the 8 ratios (recomputed)" if d == "IFN-β" else None)
ax.text(gm * 1.07, i, f"{gm:.2f}", va="center", fontsize=10, color=BLUE)
else:
ylabels.append(f"{d['cell']} {d['gene']} {d['time']}")
ax.plot([d["r_lo"], d["r_hi"]], [i, i], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([d["r"]], [i], marker="o", ms=8, color=BLUE, ls="none", zorder=3,
label="Rnf213–/– ÷ WT, bar means measured on Fig 2A\n(line: measurement range; recomputed)"
if first else None)
first = False
ax.text(1.03, i, d["label"], transform=ax.get_yaxis_transform(), va="center", ha="left",
fontsize=10.5, fontweight="bold", color=ORANGE)
ax.set_yticks(range(n)); ax.set_yticklabels(ylabels, fontsize=10, color=TXT)
ax.text(1.03, -0.75, "label printed\nin Fig 2A\n(reported)", transform=ax.get_yaxis_transform(),
va="bottom", ha="left", fontsize=10, color=ORANGE)
ax.set_xlabel("Rnf213–/– ÷ WT, mean Cxcl9 or Cxcl10 mRNA (gene/Hprt), log scale\n"
"← lower in Rnf213–/– 1 = no difference higher in Rnf213–/– →",
labelpad=6, color=TXT)
fig.legend(loc="upper left", bbox_to_anchor=(0.02, 0.997), ncol=1, frameon=False, fontsize=10,
handletextpad=0.4)
save(fig, "fig2")
if __name__ == "__main__":
check_quotes()
rows = measure()
groups = []
for stim in ("IFN-β", "A. baumannii", "LPS"):
sel, gm = summarise(rows, stim)
groups.append((stim, sel, gm))
infl = [d for d in rows if d["stim"] != "IFN-β"]
rs = [d["r"] for d in infl]
print(f"\nA. baumannii + LPS together: ratio range {min(rs):.2f}-{max(rs):.2f}; geometric mean "
f"{statistics.geometric_mean(rs):.2f}; largest upper bound {max(d['r_hi'] for d in infl):.2f}; "
f"printed *, ** or ***: {sum(d['label'] != 'ns' for d in infl)}/{len(infl)}")
ifnb = groups[0][1]
print(f"IFN-β: smallest lower bound {min(d['r_lo'] for d in ifnb):.2f}, "
f"largest upper bound {max(d['r_hi'] for d in ifnb):.2f}")
fig1(sum(d["label"] == "ns" for d in ifnb), len(ifnb))
fig2(groups)
The abstract reports increased heat sensitivity, but the paper's own data show increased heat resistance
PLoS Pathog (2018) · doi:10.1371/journal.ppat.1007004 · PMC5927469
Summary
The abstract states that loss of the outer spore protein CsA "increased spore sensitivity to heat, sodium hydroxide and hydrochloric acid," grouping heat with the two chemicals. For heat this reverses the direction of the paper's own result. The Results report that the csA-mutant spores had a 15% increase in colony-forming efficiency at 75 °C (p = 0.0286), and, by the paper's stated definition of spore viability, a higher post-treatment colony-forming efficiency means more spores survive — that is, greater resistance, i.e. less sensitivity. The Discussion agrees, stating the mutant spores "were more resistant to heat." The two chemical treatments are decreases in colony-forming efficiency (−51% and −29%), so "increased sensitivity" is correct for them; only the heat term is reversed. The error is confined to a secondary characterization phenotype and the body of the paper carries the correct direction, but the reversed claim sits in the abstract and is echoed in the Discussion's closing summary.
What the paper reports
The absence of CsA reduced sporulation levels and increased spore sensitivity to heat, sodium hydroxide and hydrochloric acid.
— Abstract
Spores of the csA mutant showed an 15% increase in colony forming efficiency when incubated at 75°C (Fig 4A), a 51% decrease in colony forming efficiency when incubated with hydrochloric acid (Fig 4B) and a 29% decrease in colony forming efficiency when incubated with sodium hydroxide (Fig 4C) when compared to wild-type spores (p = 0.0286).
— Results, "The csA and csB mutations do not affect the colony forming efficiency"
Although csA mutant spores had greater sensitivity to hydrochloric acid, they were more resistant to heat and did not show an increased sensitivity to ethanol in comparison to wild-type spores (Fig 4A and 4B, S5C Fig).
— Discussion
CsA is required for wild-type levels of spore resistance to heat, sodium hydroxide and hydrochloric acid, while CsB is required for normal levels of spore adherence to cervical cells.
— Discussion (closing summary; "resistance to heat" here implies the mutant was less resistant, again reversed)
Percentage spore viability (the ability of spores to form colonies) following treatment was calculated as follows: (log10 CFU ml-1 after plating spores on HIS agar post treatment/log10 CFU ml-1 after plating untreated spores on HIS agar) X 100.
— Materials and methods, "Spore resistance" (the paper's own outcome definition)
What the paper's own numbers show
Method. This is a sign/direction error, not an arithmetic slip, so there is no hidden quantity to recompute from raw data (no numerical CFU values for Fig 4 are deposited). The check is a mechanical mapping from the paper's stated outcome definition to the implied direction of each effect: by "percentage spore viability = (log₁₀ CFU after treatment ÷ log₁₀ CFU untreated) × 100," a higher post-treatment colony-forming efficiency (CFE) means more spores survive the treatment = more resistant = less sensitive, and a lower CFE means the opposite. Applying this to the three reported CFE changes gives the direction each should carry, which is then compared with the abstract. Separately, the single reported p-value is checked against the stated n = 4 per group design. Every value below is printed by recompute.py.txt.
| Quantity | Location | Reported | Direction implied by the paper's own definition |
|---|---|---|---|
| csA mutant heat (75 °C), CFE vs wild-type | Results; Fig 4A | +15% (increase) | more survival → more resistant = LESS sensitive — opposite to the abstract's "increased sensitivity to heat" |
| csA mutant hydrochloric acid, CFE vs wild-type | Results; Fig 4B | −51% (decrease) | less survival → more sensitive — abstract agrees (no contradiction) |
| csA mutant sodium hydroxide, CFE vs wild-type | Results; Fig 4C | −29% (decrease) | less survival → more sensitive — abstract agrees (no contradiction) |
| Heat direction, Discussion body | Discussion | "more resistant to heat" | = LESS sensitive; consistent with the Results, both disagree with the abstract |
| Reported p-value | Results; Fig 4 legend ("average of four independent experiments"); Methods (Mann-Whitney) | p = 0.0286 | 2 ÷ C(8,4) = 2/70 = 0.0286, the smallest exact two-sided rank-test p for n = 4 vs 4; internally consistent and not the error |
The body of the paper (Results, Fig 4 legend's neutral "altered ... resistance," and the Discussion sentence "they were more resistant to heat") is internally consistent on the direction of all three treatments. The abstract sentence is the single outlier, and the Discussion's closing one-line summary repeats it by listing heat among the resistances for which "CsA is required."
Figure
Figure 1. The direction conflict for heat. Left (orange): the abstract groups heat with sodium hydroxide and hydrochloric acid as treatments to which loss of CsA raised sensitivity. Right (blue): the Results report a +15% colony-forming efficiency at 75 °C and the Discussion calls the mutant spores "more resistant to heat." By the paper's own viability definition a higher post-treatment colony-forming efficiency is more survival, hence greater resistance and less sensitivity, so the two descriptions of heat point in opposite directions. The lower strip applies the same mapping to each treatment: the abstract's "increased sensitivity" matches the data for the two chemicals (both are efficiency decreases) but contradicts it for heat (an efficiency increase).
Figure 2. The three reported effects on one axis. The csA-mutant-versus-wild-type change in colony-forming efficiency is plotted from zero (no change). Heat alone falls on the positive side (more survival = less sensitive); the two chemicals fall on the negative side (less survival = more sensitive). The abstract's blanket "increased spore sensitivity" is correct for the two chemicals but has the wrong sign for heat.
Why it matters
Spore resistance to heat and chemicals is one of the characterization assays supporting the paper's claim that CsA contributes to spore integrity. The reversed heat term appears in the abstract — a high-exposure, headline location — and is repeated in the Discussion's closing summary, so a reader who takes the abstract at face value would conclude that CsA normally protects spores against heat, the opposite of what the data show (the mutant was more heat-resistant). The magnitude (15%) and the statistical significance are reported correctly in the Results; only the direction word is wrong, and only for heat. The paper's headline conclusions — that the outer spore proteins maintain spore structure, and that CsB loss affects gastrointestinal colonization and clearance — do not depend on the heat direction and are untouched. This is therefore a wrong secondary result rather than a wrong key result or conclusion.
Explanations considered
- A different definition of the measured quantity. The Materials and methods define percentage viability explicitly as post-treatment over untreated colony-forming ability. Under that definition a higher value is unambiguously more survival, so a +15% change is less sensitivity. No alternative reading of the stated definition makes +15% into increased sensitivity. Does not rescue the abstract.
- The log-ratio endpoint behaves non-monotonically. The endpoint is a monotone increasing function of post-treatment CFU for fixed untreated CFU, so a reported increase still means more survivors. Does not rescue the abstract.
- "Increased sensitivity" refers to the mutation process, not the phenotype. The Discussion states plainly "they were more resistant to heat," fixing the intended phenotype direction and matching the Results. The abstract and the Discussion's closing summary are the outliers. Does not rescue the abstract.
- A statistical artifact (the +15% is noise). The paper reports the heat comparison as significant at its own threshold (p = 0.0286, the minimum attainable for 4 vs 4), and the Discussion treats the heat result as real ("more resistant to heat"). The dispute is the sign of the reported, significant effect, not its reality. Does not rescue the abstract.
- A conversion or typesetting artifact in the archived text. The same wording is present in the version of record (see Status), so this is not an archiving artifact.
Status
- Version of record. The contradiction is present verbatim in the version of record: the abstract, Results and Discussion quotes above were confirmed in the PMC archive (
src/show.py PMC5927469). The independent re-derivation supplied with this case also confirmed the identical wording against the PLOS Pathogens HTML article and the Europe PMC full-text XML. - Supplements. Fig 4 (the heat/HCl/NaOH resistance figure) is supplied only as the rendered figure; no underlying numerical CFU values are deposited for it, and the related S5 figure is an image. There is no source-data file that could resolve the heat value independently; none is needed, since the dispute is a direction stated in the text.
- Corrections and comments. As of 2026-10-23, no correction, erratum, expression of concern or retraction is on record.
src/known.pyand the Crossref record show only a self-referential "new version" relation dated 2018-04-30 (the publication itself). Europe PMC returns the single article record (PMID 29668758) withcommentCorrectionList: None. PubPeer could not be queried in this session (the API and search page returned Cloudflare/HTTP 403 blocks); no PubPeer thread is known. - Sources. PMC5927469 (local archive via
src/show.py); Crossref and PMC/Europe PMC notice check viasrc/known.pyand the Europe PMC REST API.
Suggested correction
Correct the heat term in the abstract so that it is not grouped with the two chemicals — for example, "loss of CsA reduced sporulation levels, increased spore sensitivity to sodium hydroxide and hydrochloric acid, and altered (increased) resistance to heat" — and revise the Discussion's closing summary accordingly so that it no longer implies the mutant was less heat-resistant. The Results text, the Fig 4 legend and the Discussion sentence "they were more resistant to heat" already carry the correct direction and need no change.
Recomputation script (python recompute.py.txt)
"""Recompute / verify the heat-sensitivity direction in PMC5927469
(PLoS Pathog 2018, doi:10.1371/journal.ppat.1007004).
Run: python recompute.py.txt
Needs matplotlib (and the standard library). Prints every number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
The dispute is a SIGN/DIRECTION error, not an arithmetic slip, so the "recompute"
is (a) a mechanical mapping from the paper's own outcome definition to the implied
direction of each effect, and (b) an exact check that the single reported p-value is
consistent with the stated n = 4 per group design. All inputs are copied from the
paper as archived in PubMed Central (src/show.py PMC5927469).
Paper's own outcome definition (Materials and methods, "Spore resistance"):
percentage spore viability after a treatment =
(log10 CFU/ml after treatment / log10 CFU/ml untreated) x 100
-> a HIGHER colony-forming efficiency (CFE) after a treatment means MORE spores
survive it => MORE resistant => LESS sensitive.
-> a LOWER CFE after a treatment means fewer survive => LESS resistant
=> MORE sensitive.
"""
import os
from math import comb
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
# csA mutant vs wild-type, % change in colony-forming efficiency after each treatment
# (Results, "The csA and csB mutations do not affect the colony forming efficiency"; Fig 4)
REPORTED_CFE_CHANGE = { # treatment : % change in CFE for csA mutant vs WT
"heat (75 C)": +15,
"hydrochloric acid": -51,
"sodium hydroxide": -29,
}
# What the abstract says about each treatment. The abstract groups all three together:
# "increased spore sensitivity to heat, sodium hydroxide and hydrochloric acid"
ABSTRACT_SAYS = {t: "MORE sensitive" for t in REPORTED_CFE_CHANGE}
# What the Discussion body says about each treatment (Discussion #p4).
DISCUSSION_SAYS = {
"heat (75 C)": "MORE resistant / LESS sensitive", # "they were more resistant to heat"
"hydrochloric acid": "MORE sensitive", # "greater sensitivity to hydrochloric acid"
"sodium hydroxide": "MORE sensitive", # "increased sensitivity to sodium hydroxide"
}
REPORTED_P = 0.0286 # Results, single p-value for the csA vs WT comparisons
N_PER_GROUP = 4 # Fig 4 legend: "the average of four independent experiments";
# Methods: Mann-Whitney test
def cfe_change_to_sensitivity(pct_change):
"""Map a csA-vs-WT % change in post-treatment CFE to a sensitivity direction,
using the paper's own definition of percentage spore viability."""
if pct_change > 0:
return "LESS sensitive (= MORE resistant)"
if pct_change < 0:
return "MORE sensitive (= LESS resistant)"
return "no change"
# ---------------------------------------------------------------- 1. direction check
print("=== 1. Direction implied by the paper's own outcome definition, per treatment ===")
print(f"{'treatment':20s} {'reported CFE':>12s} {'implied direction':34s} "
f"{'abstract says':16s} {'discussion says':26s} abstract vs data")
print("-" * 118)
contradictions = []
for t, chg in REPORTED_CFE_CHANGE.items():
implied = cfe_change_to_sensitivity(chg)
implied_more = implied.startswith("MORE sensitive")
abstract_more = ABSTRACT_SAYS[t].startswith("MORE")
agree = (implied_more == abstract_more)
if not agree:
contradictions.append(t)
verdict = "agree" if agree else "CONTRADICTION"
print(f"{t:20s} {chg:+11d}% {implied:34s} {ABSTRACT_SAYS[t]:16s} "
f"{DISCUSSION_SAYS[t]:26s} {verdict}")
print()
print(f"Treatments where the abstract contradicts the paper's own CFE data: {contradictions}")
print(f"Results (+15% CFE at 75 C) implies: {cfe_change_to_sensitivity(+15)}")
print(f"Discussion body says for heat: {DISCUSSION_SAYS['heat (75 C)']}")
print("Results and Discussion agree with each other on heat: True")
print("Abstract agrees with them on heat: False "
"(abstract groups heat under 'increased sensitivity')")
print("Abstract agrees on the two chemicals (HCl, NaOH): True "
"(both are CFE decreases = increased sensitivity)")
# ---------------------------------------------------------------- 2. p-value sanity check
print("\n=== 2. Is the single reported p = 0.0286 consistent with n = 4 per group? ===")
total = comb(2 * N_PER_GROUP, N_PER_GROUP) # C(8,4)
smallest_two_sided = 2.0 / total
print(f"n1 = n2 = {N_PER_GROUP}: number of ways to split 8 ranks = C(8,4) = {total}")
print(f"smallest attainable exact two-sided Mann-Whitney p = 2/{total} = {smallest_two_sided:.4f}")
print(f"reported p = {REPORTED_P} matches 2/{total} = {smallest_two_sided:.4f}: "
f"{abs(smallest_two_sided - REPORTED_P) < 5e-4}")
print("=> the p-value is exactly the minimum a 4-vs-4 rank test can give; it is internally "
"consistent and is NOT the error.")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
GREEN_OK, PANEL = "#3a8a4f", "#f4f3ef"
import matplotlib # noqa: E402
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": TXT,
"axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# ---- Figure 1: two-panel abstract-vs-body contradiction diagram -----------------
fig, ax = plt.subplots(figsize=(7.2, 5.6))
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
def panel(x, y, w, h, title, title_col, body, foot, foot_col):
box = FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.6,rounding_size=2.5",
linewidth=1.6, edgecolor=title_col, facecolor=PANEL, zorder=1)
ax.add_patch(box)
ax.text(x + w / 2, y + h - 3.2, title, ha="center", va="top",
fontsize=12, fontweight="bold", color=title_col)
ax.text(x + w / 2, y + h - 11, body, ha="center", va="top",
fontsize=9.3, color=TXT, linespacing=1.32)
ax.text(x + w / 2, y + 3.5, foot, ha="center", va="bottom",
fontsize=9.1, color=foot_col, fontweight="bold", linespacing=1.25)
panel(3, 60, 43, 37, "Abstract", ORANGE,
'"The absence of CsA reduced\n'
'sporulation levels and increased\n'
'spore sensitivity to heat, sodium\n'
'hydroxide and hydrochloric acid."',
"→ heat grouped with the chemicals\nas MORE sensitive", ORANGE)
panel(54, 60, 43, 37, "Results + Discussion", BLUE,
'Results / Fig 4A: csA mutant had a\n'
'"15% increase in colony forming\n'
'efficiency when incubated at 75 C"\n'
'Discussion: "they were more\n'
'resistant to heat"',
"→ heat: MORE resistant\n= LESS sensitive", BLUE)
# connector naming the conflict
arr = FancyArrowPatch((46, 78), (54, 78), arrowstyle="<|-|>", mutation_scale=18,
linewidth=2, color="#b03020", zorder=3)
ax.add_patch(arr)
ax.text(50, 73, "heat\ndirection\nreversed", ha="center", va="top",
fontsize=9.5, fontweight="bold", color="#b03020", linespacing=1.15)
# bottom strip: the three treatments and whether abstract matches the data
ax.text(50, 52, "By the paper’s own definition "
"(viability = log₁₀ CFU after ÷ log₁₀ CFU untreated × 100), a higher\n"
"post-treatment CFE means more survival = more resistant = less sensitive:",
ha="center", va="top", fontsize=9.0, color=SEC, linespacing=1.3)
cells = [
("heat (75 C)", "+15% CFE", "less sensitive", "“more sensitive”", False),
("hydrochloric acid", "−51% CFE", "more sensitive", "“more sensitive”", True),
("sodium hydroxide", "−29% CFE", "more sensitive", "“more sensitive”", True),
]
cx = [17, 50, 83]
for i, (t, cfe, implied, says, ok) in enumerate(cells):
col = GREEN_OK if ok else "#b03020"
mark = "✓ abstract matches" if ok else "✗ abstract contradicts"
ax.text(cx[i], 38, t, ha="center", fontsize=9.6, fontweight="bold", color=TXT)
ax.text(cx[i], 31, f"data: {cfe} → {implied}", ha="center", fontsize=8.7, color=BLUE)
ax.text(cx[i], 25.5, f"abstract: {says}", ha="center", fontsize=8.7, color=ORANGE)
ax.text(cx[i], 18, mark, ha="center", fontsize=9.2, fontweight="bold", color=col)
legend = [Line2D([0], [0], color=ORANGE, lw=6, label="paper’s abstract claim"),
Line2D([0], [0], color=BLUE, lw=6, label="direction from the paper’s own numbers")]
ax.legend(handles=legend, loc="lower center", bbox_to_anchor=(0.5, 0.0),
ncol=2, frameon=False, fontsize=9, handlelength=1.4)
fig.subplots_adjust(left=0.01, right=0.99, top=0.99, bottom=0.04)
save(fig, "fig1")
# ---- Figure 2: per-treatment CFE change, heat as the sign outlier ---------------
fig, ax = plt.subplots(figsize=(7.2, 3.8))
treatments = list(REPORTED_CFE_CHANGE.keys())
vals = [REPORTED_CFE_CHANGE[t] for t in treatments]
ypos = list(range(len(treatments)))[::-1]
# shade the two half-planes and label what each side means
ax.axvspan(-80, 0, color="#f3e6df", zorder=0)
ax.axvspan(0, 60, color="#e3edf6", zorder=0)
ax.text(-40, 2.62, "CFE decrease → fewer survive\n= MORE sensitive",
ha="center", va="center", fontsize=8.6, color="#9a3412")
ax.text(30, 2.62, "CFE increase → more survive\n= LESS sensitive",
ha="center", va="center", fontsize=8.6, color="#1e4e79")
ax.axvline(0, color=SEC, lw=1.4, zorder=2)
for y, t, v in zip(ypos, treatments, vals):
ax.plot([0, v], [y, y], color="#b7b6b1", lw=2, zorder=1)
ax.plot(v, y, "o", ms=13, color=ORANGE, zorder=3)
ax.text(v + (4 if v > 0 else -4), y + 0.17, f"{v:+d}%", va="center",
ha="left" if v > 0 else "right", fontsize=11, fontweight="bold", color=TXT)
agree = (v < 0) # abstract says "increased sensitivity" for all three
if agree:
ax.text(v - 4, y - 0.26, "✓ abstract agrees", va="center", ha="right",
fontsize=8.6, fontweight="bold", color=GREEN_OK)
else:
ax.text(58, y - 0.26, "✗ abstract says “more sensitive” — contradicts",
va="center", ha="right", fontsize=8.6, fontweight="bold", color="#b03020")
ax.set_yticks(ypos)
ax.set_yticklabels(treatments, fontsize=10.5)
ax.set_ylim(-0.6, 3.1)
ax.set_xlim(-80, 60)
ax.set_xlabel("csA mutant vs wild-type: % change in colony-forming efficiency after treatment "
"(Results, Fig 4)", fontsize=9.3)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.set_title("Only heat is a CFE increase; the abstract groups all three as "
"“increased sensitivity”", fontsize=10, color=TXT, pad=8)
fig.tight_layout()
save(fig, "fig2")
print("\nWrote fig1.svg/png and fig2.svg/png to", HERE)
Abstract reports phantom-sensation scores as phantom range of motion in degrees
PNAS (2021) · doi:10.1073/pnas.2019555118 · PMC7936324
Summary
The abstract states that subjects with an agonist–antagonist myoneural interface below-knee amputation (AB, n = 15) reported "significantly greater phantom range of motion" than subjects with a traditional below-knee amputation (TB, n = 7), quantified as "AB: 12.47 ± 2.41, TB: 10.14 ± 1.45 degrees". In the body of the paper, 12.47 ± 2.41 and 10.14 ± 1.9 are the group means of the phantom sensation score, a unitless sum of 1–3 grades over foot regions. The per-subject scores in Fig. 1B reproduce these values exactly (recomputed 12.47 ± 2.42 and 10.14 ± 1.95). Phantom range of motion is a different measure: the paper reports it as a percentage of the contralateral limb, separately for each direction, and never as a single value in degrees. The abstract therefore attaches another outcome's numbers, a unit and a TB standard deviation (1.45) that none of the paper's data reproduce to one of its headline comparisons. Readers of the abstract would take the difference as about 2.3 degrees of phantom motion. It is in fact a 23% higher sensation score.
What the paper reports
The abstract:
"AB subjects reported significantly greater phantom range of motion postamputation (AB: 12.47 ± 2.41, TB: 10.14 ± 1.45 degrees) when compared to TB subjects (P < 0.05)." (Abstract)
The Results give the same means, with a different label and a different TB SD:
"AB subjects reported significantly higher phantom sensations (12.47 ± 2.41, P = 0.03) compared to TB subjects (10.14 ± 1.9) (P < 0.05, Student’s t test)." (Results, "Phantom Sensations and Pain")
The Methods define the phantom sensation score as a grade per foot region:
"Subjects were then specifically asked to relate the level of perception of their lateral and medial malleoli, heel pad, arch of the foot, balls of the footpad, and toes. The more thorough and tangible the phantom limb felt to subjects, the greater the phantom sensation scored. Responses were independently graded on a 1 to 3 scale" (Methods, "Phantom Limb Sensation")
Fig. 1B is a per-subject table. It has five region columns (Malleoli, Heelpad, Arch of foot, Balls of feet, Toes), a "Phantom Sensation Score" column and a "Phantom Pain Score" column, with summary rows:
"Average Std Dev 12.47 +/- 2.4" (AB) and "Average Std Dev 10.14 +/- 1.9" (TB) (Fig. 1B, Phantom Sensation Score column; transcribed from the version-of-record PDF)
"Phantom sensations in the AB cohort (n = 15) are significantly greater than those in the TB cohort (n = 7) (P < 0.05, Student’s t test)." (Fig. 1 legend, panel B)
Phantom range of motion (ROM) is measured and reported differently:
"Data were processed through custom scripts in MATLAB (Mathworks) to identify the maximum amplitude for each movement and normalized as a percentage of the amplitude of the contralateral limb." (Methods, "ROM Percepts")
"The demonstrated ROM of the phantom limb in each direction (PF, DF, IN, EV) were normalized as a percentage of the unaffected joints’ ROM (Fig. 2C)." (Results, "ROM Percepts")
"Further, AB subjects experienced significantly greater ROM than TB subjects following amputation for PF, DF, and EV (P < 0.05, Student’s two-tailed t test)." (Results, "ROM Percepts")
What the paper's own numbers show
Method. The 22 per-subject phantom sensation scores and pain scores were transcribed from Fig. 1B of the publisher's PDF into recompute.py.txt:
- AB scores: 14, 11, 8, 13, 15, 14, 14, 8, 13, 14, 10, 14, 14, 15, 10.
- TB scores: 8, 8, 11, 12, 13, 9, 10.
The script computes each group's mean, its sample SD (n − 1), its population SD (n) and its SEM, and runs Student’s and Welch's two-sample t tests on the raw scores and on the reported summary statistics. It also derives the possible range of the score from the number of graded regions. The Methods, the Results ("ROM Percepts") and the Fig. 2 legend were searched for any phantom ROM value expressed in degrees.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| AB value in the "phantom range of motion" sentence | Abstract | 12.47 ± 2.41 degrees | Equals the AB phantom sensation score: Fig. 1B mean 12.467 (187/15), SD 2.416 |
| TB value in the "phantom range of motion" sentence | Abstract | 10.14 ± 1.45 degrees | Mean equals the TB phantom sensation score: Fig. 1B mean 10.143 (71/7). SD is 1.952, not 1.45 |
| AB phantom sensation score | Results; Fig. 1B | 12.47 ± 2.41 (Fig. 1B: ± 2.4) | 12.467 ± 2.416 |
| TB phantom sensation score | Results; Fig. 1B | 10.14 ± 1.9 | 10.143 ± 1.952 |
| TB dispersion statistics that could produce 1.45 | Abstract | 1.45 | Sample SD 1.95, population SD 1.81, SEM 0.74: none equals 1.45 |
| Possible range of the phantom sensation score | Methods; Fig. 1B | 1–3 per region; 5 region columns | 5–15, with no unit (6–18 if the two malleoli were graded separately). Observed scores run from 8 to 15, and 2 AB subjects score 15 |
| Phantom ROM, the measure the abstract names | Methods and Results, "ROM Percepts"; Fig. 2C | % of the contralateral limb, per direction (PF, DF, IN, EV) | No pooled value and no value in degrees appears anywhere in the paper |
| AB − TB difference as stated in the abstract | Abstract | 12.47 − 10.14 "degrees" | 2.33 score points; ratio of the means 1.230 (23% higher score) |
| Phantom sensation test, raw Fig. 1B scores | Results | P = 0.03 (Student’s t test) | Student t = 2.220, P = 0.038; Welch t = 2.405, P = 0.030 |
| Phantom sensation test, summary statistics with TB SD 1.9 | Results | P = 0.03 | Student t = 2.243, P = 0.036; Welch P = 0.027 |
| Phantom sensation test, summary statistics with TB SD 1.45 | Abstract | P < 0.05 | Student t = 2.349, P = 0.029; Welch P = 0.011 |
Whichever SD is used, the phantom-sensation comparison remains significant at P < 0.05. (The reported P = 0.03 matches Welch's test on the raw scores, 0.030. Student's test, which the paper names, gives 0.038. Both are below 0.05.) The discrepancy lies in what the abstract's numbers are said to measure, in their unit and in the TB SD. It does not affect whether the comparison is significant.
Other observations (outside the main finding). The pain values in the text do not match Fig. 1B either. The pain score column of Fig. 1B gives AB 3.467 ± 4.96 and TB 9.286 ± 10.23 (printed summary rows: 3.46 ± 4.9 and 9.28 ± 10.2). The Results give 12.25 ± 4.9 and 17.29 ± 10.22: the SDs agree with Fig. 1B but the means do not. The abstract gives the AB pain SD as 5.37 rather than 4.9. Fig. 1B lists 7 AB subjects with a pain score of 0, and the Results say 6 of the 15 "indicated zero pain in all categories". From the reported summary statistics, the AB versus TB pain comparison gives Student P = 0.128 (SD 4.9) or 0.141 (SD 5.37), against the reported P = 0.08. The paper does not describe enough of how the pain score was built to resolve these differences, so they are listed here only for completeness.
Figure
Figure 1. The abstract sentence (top, orange outline) gives "12.47 ± 2.41" and "10.14 ± 1.45 degrees" as phantom range of motion. The same means appear in the Results (bottom left) as phantom sensation scores, which are sums of 1–3 grades over the five foot regions of Fig. 1B. The possible range is 5–15 and the score has no unit. The TB SD there is 1.9, and the Fig. 1B per-subject scores give 1.95 (blue, recomputed in recompute.py.txt). Phantom range of motion, the measure the abstract names (bottom right), is reported only as a percentage of the contralateral limb, separately for each direction, and never in degrees. Orange text marks values printed in the paper, blue text marks values recomputed from the paper's own per-subject data.
Figure 2. Dispersion of the seven TB phantom sensation scores listed in Fig. 1B (8, 8, 11, 12, 13, 9, 10). Orange circles: SDs printed in the abstract (1.45) and in the Results and Fig. 1B (1.9). Blue diamonds: the sample SD (1.95), population SD (1.81) and SEM (0.74) recomputed from the per-subject scores. The dashed line marks the abstract's 1.45. None of the recomputed statistics equals it, so the abstract's SD cannot be explained as a different summary of the same data.
Why it matters
The sentence is the abstract's only quantitative statement about phantom perception, and the paper's Significance statement and Discussion present phantom perception as a main benefit of the procedure. Read as written, the sentence tells the reader that AB subjects could perceive about 2.3 degrees more phantom joint motion than TB subjects. That would be a very small absolute range for whole-ankle motion. It would also conflict with the paper's own ROM method, which expresses ROM as a percentage of the unaffected limb. The numbers actually describe something else: AB subjects' summed phantom sensation grades were about 23% higher than TB subjects'. The abstract also gives a TB SD (1.45) smaller than the one in the Results (1.9) and the one the data support (1.95), which makes the TB group look less variable than it was.
The direction and statistical significance of both underlying findings are unaffected. In the Results, AB subjects have higher phantom sensation scores (P < 0.05 under every recomputation above), and significantly greater phantom ROM in three of the four directions (PF, DF, EV). The error is in how the abstract reports these findings. It merges two outcomes into one sentence, gives the magnitude of one under the name and a unit of the other, and leaves the actual ROM result (percent of contralateral ROM by direction) unquantified in the abstract. Anyone extracting effect sizes from the abstract, for example in a systematic review or meta-analysis of phantom limb outcomes after amputation, would record a degree-valued ROM difference that the study did not measure.
Explanations considered
- A separate pooled phantom-ROM measure in degrees exists. It does not. The Methods ("ROM Percepts") normalize goniometer amplitudes "as a percentage of the amplitude of the contralateral limb". The Results ("ROM Percepts") and Fig. 2C report ROM separately for PF, DF, IN and EV as a percentage of the unaffected joint. The paper gives no pooled mean ± SD for ROM and no value in degrees. The values 12.47 and 10.14 occur only in the phantom-sensation sentence of the Results and in Fig. 1B, apart from the abstract.
- The numbers coincide by chance. Both means match to two decimals, and so does the AB SD (2.41). Fig. 1B's per-subject phantom sensation scores reproduce them exactly (187/15 = 12.467, 71/7 = 10.143). Two measures agreeing by chance on three statistics is not a plausible explanation.
- "Degrees" is used loosely to mean "levels" or "grades". This would explain the unit but not the label. The sentence still names range of motion rather than sensation, and in this paper range of motion is a percentage. The TB SD of 1.45 would also remain unexplained.
- 1.45 is a different dispersion statistic (population SD, SEM) for the same data. The seven TB scores give a sample SD of 1.95, a population SD of 1.81 and an SEM of 0.74. None equals 1.45 (Fig. 2), and the AB SD in the same sentence (2.41) is the ordinary sample SD.
- A different time point or subgroup. Both the abstract and the Results describe the postamputation AB versus TB comparison with n = 15 and n = 7. The paper reports no other phantom-sensation or ROM summary with these means.
- A conversion or typesetting artifact in PMC. Ruled out: the publisher's typeset PDF (cover sheet: "Final published version") has the identical abstract and Results wording, and Europe PMC's full-text XML matches.
- The supplement holds the degree values. The SI Appendix (7 pages) covers methods figures, Fig. S4 (pain scores before and after surgery), Fig. S5 (correlations of fascicle strain with ROM and with phantom sensation) and Fig. S6 (F-test P values). It gives no group phantom-ROM values in degrees and no TB SD of 1.45. The five supplementary movies show ultrasound and fascicle dynamics and contain no summary numbers.
Status
- Version of record: checked. pnas.org returned HTTP 403 to automated access. The publisher's own typeset PDF was read from MIT DSpace (hdl.handle.net/1721.1/134133, marked "Version: Final published version"). Its abstract (p. 1) and Results (p. 3) wording is identical to the PMC archive, and Fig. 1B (p. 3 of 8) supplied the per-subject scores used here.
- Supplementary material: SI Appendix PDF checked (see above). Movies S1–S5 not opened, as they contain no tabulated values.
- Corrections: none found. Crossref lists no update relations, and PMC and Europe PMC list no correction, erratum, retraction or expression of concern linked to the article.
- PubPeer: not checked. pubpeer.com refused automated requests (HTTP 403 / human-verification challenge), and a web search for PubPeer threads could not be run in this session. Whether the issue has been raised there is therefore unknown.
- Sources: PMC7936324 (PubMed Central archive); doi:10.1073/pnas.2019555118 (publisher PDF via MIT DSpace); Crossref and Europe PMC metadata.
- As of: 2026-10-23.
Suggested correction
In the abstract, replace
"AB subjects reported significantly greater phantom range of motion postamputation (AB: 12.47 ± 2.41, TB: 10.14 ± 1.45 degrees) when compared to TB subjects (P < 0.05)."
with a sentence that keeps the two outcomes separate and reports the values given in the Results and Fig. 1B, for example:
"AB subjects reported significantly higher phantom sensation scores (AB: 12.47 ± 2.41, TB: 10.14 ± 1.9; P < 0.05) and significantly greater phantom range of motion, as a percentage of the unaffected limb, for plantarflexion, dorsiflexion and eversion (P < 0.05) when compared to TB subjects."
If the authors intended a different TB SD or a phantom-ROM summary in degrees, the source of 1.45 and of any degree values should be stated, since neither can be derived from the data presented. Separately, the authors could clarify how the text pain values (12.25 ± 4.9 and 17.29 ± 10.22, with 5.37 in the abstract) relate to the Fig. 1B pain scores (3.46 ± 4.9 and 9.28 ± 10.2).
Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC7936324 and regenerate fig1/fig2.
Run: python recompute.py.txt (writes fig1.svg/png and fig2.svg/png next to this file)
Inputs are transcribed from the article (PNAS 2021, doi:10.1073/pnas.2019555118):
- Abstract and Results text, as archived in PubMed Central (PMC7936324) and identical
in the publisher's typeset PDF ("Final published version").
- Fig. 1B, the per-subject table of Phantom Sensation Score and Phantom Pain Score
(15 AB rows, 7 TB rows, in printed order), read from the version-of-record PDF.
"""
import os
import textwrap
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Patch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs
# Fig. 1B, "Phantom Sensation Score" column
PS_AB = np.array([14, 11, 8, 13, 15, 14, 14, 8, 13, 14, 10, 14, 14, 15, 10])
PS_TB = np.array([8, 8, 11, 12, 13, 9, 10])
# Fig. 1B, "Phantom Pain Score" column
PAIN_AB = np.array([0, 2, 0, 1, 5, 0, 10, 15, 12, 3, 0, 0, 0, 0, 4])
PAIN_TB = np.array([0, 5, 0, 11, 11, 8, 30])
# Fig. 1B region columns: Malleoli, Heelpad, Arch of foot, Balls of feet, Toes
N_REGION_COLS = 5
GRADE_MIN, GRADE_MAX = 1, 3 # Methods: "graded on a 1 to 3 scale"
# Reported values
ABS = dict(ab_m=12.47, ab_s=2.41, tb_m=10.14, tb_s=1.45) # Abstract, "degrees"
RES = dict(ab_m=12.47, ab_s=2.41, tb_m=10.14, tb_s=1.9, p=0.03) # Results, phantom sensation
FIG1B = dict(ab_m=12.47, ab_s=2.4, tb_m=10.14, tb_s=1.9,
pain_ab_m=3.46, pain_ab_s=4.9, pain_tb_m=9.28, pain_tb_s=10.2)
PAIN_RES = dict(ab_m=12.25, ab_s=4.9, tb_m=17.29, tb_s=10.22, p=0.08)
PAIN_ABS_AB_S = 5.37
nA, nT = len(PS_AB), len(PS_TB)
def desc(x):
return x.mean(), x.std(ddof=1), x.std(ddof=0), x.std(ddof=1) / np.sqrt(len(x))
print("== 1. Scale of the phantom sensation score ==")
lo, hi = N_REGION_COLS * GRADE_MIN, N_REGION_COLS * GRADE_MAX
print(f"Fig. 1B region columns: {N_REGION_COLS}; graded {GRADE_MIN}-{GRADE_MAX} -> possible total {lo}-{hi}")
print(f"(if lateral and medial malleoli were graded separately, 6 regions -> {6*GRADE_MIN}-{6*GRADE_MAX})")
print(f"Observed Fig. 1B scores: AB {PS_AB.min()}-{PS_AB.max()}, TB {PS_TB.min()}-{PS_TB.max()}")
print(f"AB subjects at the {hi}-point maximum: {(PS_AB == hi).sum()}")
print()
print("== 2. Phantom sensation scores recomputed from Fig. 1B ==")
mA, sA, pA, seA = desc(PS_AB)
mT, sT, pT, seT = desc(PS_TB)
print(f"AB: n={nA}, sum={PS_AB.sum()}, mean={mA:.3f}, SD(n-1)={sA:.3f}, SD(n)={pA:.3f}, SEM={seA:.3f}")
print(f"TB: n={nT}, sum={PS_TB.sum()}, mean={mT:.3f}, SD(n-1)={sT:.3f}, SD(n)={pT:.3f}, SEM={seT:.3f}")
print(f"Reported Results/Fig. 1B: AB {RES['ab_m']} +/- {RES['ab_s']}, TB {RES['tb_m']} +/- {RES['tb_s']}")
print(f"Reported Abstract ('degrees'): AB {ABS['ab_m']} +/- {ABS['ab_s']}, TB {ABS['tb_m']} +/- {ABS['tb_s']}")
cands = {"SD (n-1)": sT, "SD (n)": pT, "SEM": seT}
print("TB dispersion candidates vs abstract 1.45:",
", ".join(f"{k} {v:.2f}" for k, v in cands.items()),
"-> any equal to 1.45 after rounding:", any(round(v, 2) == 1.45 for v in cands.values()))
print(f"Abstract AB-TB difference: {ABS['ab_m'] - ABS['tb_m']:.2f} (abstract unit: degrees); "
f"as a ratio of score means: {ABS['ab_m'] / ABS['tb_m']:.3f} "
f"({100 * (ABS['ab_m'] / ABS['tb_m'] - 1):.0f}% higher score)")
print()
print("== 3. Two-sample t tests ==")
def tt(m1, s1, n1, m2, s2, n2, eq):
r = stats.ttest_ind_from_stats(m1, s1, n1, m2, s2, n2, equal_var=eq)
return r.statistic, r.pvalue
for lab, s in [("Results TB SD 1.9", RES["tb_s"]), ("Abstract TB SD 1.45", ABS["tb_s"])]:
t, p = tt(12.47, 2.41, nA, 10.14, s, nT, True)
tw, pw = tt(12.47, 2.41, nA, 10.14, s, nT, False)
print(f"Summary stats, {lab}: Student t={t:.3f} p={p:.3f}; Welch t={tw:.3f} p={pw:.3f}")
r_s = stats.ttest_ind(PS_AB, PS_TB, equal_var=True)
r_w = stats.ttest_ind(PS_AB, PS_TB, equal_var=False)
print(f"Fig. 1B per-subject scores: Student t={r_s.statistic:.3f} p={r_s.pvalue:.3f}; "
f"Welch t={r_w.statistic:.3f} p={r_w.pvalue:.3f} (reported P = {RES['p']}, 'Student's t test')")
print()
print("== 4. Side observations: pain scores (not part of the main finding) ==")
pmA, psA, _, _ = desc(PAIN_AB)
pmT, psT, _, _ = desc(PAIN_TB)
print(f"Fig. 1B pain, recomputed: AB mean={pmA:.3f} SD={psA:.2f}; TB mean={pmT:.3f} SD={psT:.2f}")
print(f"Fig. 1B pain summary rows: AB {FIG1B['pain_ab_m']} +/- {FIG1B['pain_ab_s']}; "
f"TB {FIG1B['pain_tb_m']} +/- {FIG1B['pain_tb_s']}")
print(f"Results pain: AB {PAIN_RES['ab_m']} +/- {PAIN_RES['ab_s']}; TB {PAIN_RES['tb_m']} +/- {PAIN_RES['tb_s']}; "
f"Abstract AB SD {PAIN_ABS_AB_S}")
print(f"AB subjects with a Fig. 1B pain score of 0: {(PAIN_AB == 0).sum()} of {nA} "
f"(Results: '6 of the 15 subjects ... indicated zero pain in all categories')")
for lab, s in [("Results AB SD 4.9", 4.9), ("Abstract AB SD 5.37", 5.37)]:
t, p = tt(12.25, s, nA, 17.29, 10.22, nT, True)
print(f"Pain, summary stats, {lab}: Student t={t:.3f} p={p:.3f} (reported P = {PAIN_RES['p']})")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
"text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK2})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
def box(ax, x, y, w, h, title, body, width_chars, edge=INK2):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.004,rounding_size=0.012",
fc="white", ec=edge, lw=1.2, transform=ax.transAxes))
ax.text(x + 0.015, y + h - 0.022, title, fontsize=10, fontweight="bold", va="top",
transform=ax.transAxes)
ax.text(x + 0.015, y + h - 0.075, "\n".join(textwrap.wrap(body, width_chars)),
fontsize=10, va="top", color=INK, transform=ax.transAxes, linespacing=1.35)
# ---- Figure 1: abstract vs body text diagram
fig = plt.figure(figsize=(7.2, 5.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off(); ax.set_facecolor(BG)
box(ax, 0.03, 0.80, 0.94, 0.17, "Abstract",
"\u201cAB subjects reported significantly greater phantom range of motion postamputation "
"(AB: 12.47 \u00b1 2.41, TB: 10.14 \u00b1 1.45 degrees) when compared to TB subjects (P < 0.05).\u201d",
92, edge=ORANGE)
box(ax, 0.03, 0.15, 0.455, 0.285, "Results: Phantom Sensations and Pain",
"\u201cAB subjects reported significantly higher phantom sensations (12.47 \u00b1 2.41, P = 0.03) "
"compared to TB subjects (10.14 \u00b1 1.9)\u201d. Fig. 1B lists the per-subject scores: "
"five foot regions, each graded 1\u20133 (Methods).",
40)
box(ax, 0.515, 0.15, 0.455, 0.285, "Methods / Results: ROM Percepts",
"Phantom ROM was \u201cnormalized as a percentage of the amplitude of the contralateral limb\u201d "
"and reported per direction (PF, DF, IN, EV; Fig. 2C). No pooled phantom ROM in degrees "
"appears anywhere in the paper.",
40)
for x in (0.10, 0.585):
ax.add_patch(FancyArrowPatch((x, 0.79), (x, 0.445), transform=ax.transAxes,
arrowstyle="-|>", mutation_scale=14, lw=2, color=INK2))
def note(x, y, s, **kw):
ax.text(x, y, s, fontsize=10, va="top", transform=ax.transAxes, linespacing=1.35, **kw)
note(0.12, 0.76, "Same numbers, different measure", fontweight="bold")
note(0.12, 0.72, "Unitless score, possible range 5\u201315\n(Fig. 1B scores span 8\u201315)")
note(0.12, 0.635, "TB SD 1.45 abstract (reported)", color=ORANGE, fontweight="bold")
note(0.12, 0.595, "TB SD 1.9 Results (reported)", color=ORANGE, fontweight="bold")
note(0.12, 0.555, f"TB SD {sT:.2f} Fig. 1B (recomputed)", color=BLUE, fontweight="bold")
note(0.605, 0.76, "The measure the abstract names", fontweight="bold")
note(0.605, 0.72, "% of the contralateral limb,\nper direction; never pooled\nand never given in degrees")
ax.legend(handles=[Patch(fc="white", ec=ORANGE, lw=1.5, label="Text containing the conflicting statement"),
Line2D([], [], color=ORANGE, lw=0, marker="s", ms=9, label="Value reported in the paper"),
Line2D([], [], color=BLUE, lw=0, marker="s", ms=9,
label="Recomputed from the paper's Fig. 1B per-subject scores")],
loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=1, frameon=False, fontsize=10)
save(fig, "fig1")
# ---- Figure 2: TB dispersion, abstract value vs everything the Fig. 1B data give
rows = [
("Abstract TB SD", ABS["tb_s"], ORANGE, "o", "1.45"),
("Results / Fig. 1B TB SD", RES["tb_s"], ORANGE, "o", "1.9"),
("Sample SD (n−1), Fig. 1B scores", sT, BLUE, "D", f"{sT:.2f}"),
("Population SD (n), Fig. 1B scores", pT, BLUE, "D", f"{pT:.2f}"),
("SEM, Fig. 1B scores", seT, BLUE, "D", f"{seT:.2f}"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.3), facecolor=BG)
ax.set_facecolor(BG)
ys = np.arange(len(rows))[::-1]
for y, (lab, v, c, m, txt) in zip(ys, rows):
ax.plot([0, v], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(v, y, marker=m, ms=10, color=c, zorder=3, lw=0)
ax.text(v + 0.08, y, txt, va="center", fontsize=10, color=c, fontweight="bold")
ax.axvline(ABS["tb_s"], color=ORANGE, lw=1, ls=(0, (3, 3)), zorder=0)
ax.set_yticks(ys, [r[0] for r in rows], fontsize=10)
ax.set_xlim(0, 2.4)
ax.set_ylim(-0.6, len(rows) - 0.4)
ax.set_xlabel("Dispersion of the TB phantom-sensation score (score points), n = 7")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], color=ORANGE, marker="o", ms=9, lw=0, label="Reported in the paper"),
Line2D([], [], color=BLUE, marker="D", ms=9, lw=0,
label="Recomputed from Fig. 1B (TB scores 8, 8, 11, 12, 13, 9, 10)")],
loc="upper center", bbox_to_anchor=(0.35, -0.22), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png")
Abstract reverses the direction of the white-light calcium responses shown in Results and Fig. 4B
Proc Natl Acad Sci U S A (2018) · doi:10.1073/pnas.1802592115 · PMC6217433
Summary
The paper reports that a single bistable opsin, parapinopsin, gives zebrafish pineal photoreceptors opposite calcium responses to UV and visible light. In one experiment (Fig. 4B), white light was shifted toward UV or toward visible wavelengths. The abstract says these shifts "respectively increased vs. decreased" the calcium signal. The Results paragraph, the Fig. 4 legend, the transient responses in the plotted Fig. 4B traces, the 405-nm and 588-nm two-photon experiments, and the paper's proposed mechanism all give the opposite: UV-shifted light lowered calcium and visible-shifted light raised it. The existence of opponency and its dependence on parapinopsin are unaffected. But the abstract, the part most readers and indexers rely on, states the sign of the naturalistic-light result backwards, and nothing in the abstract lets a reader notice.
What the paper reports
Abstract:
Changes in the spectral composition of white light toward enhanced UV or visible wavelengths respectively increased vs. decreased calcium signal in parapinopsin-sufficient but not parapinopsin-deficient photoreceptors.
Results, section "Color Opponency Based on Photoequilibrium of Parapinopsin Two States", white-light paragraph:
We found decreased and increased calcium levels following exposure to UV and visible light-mixed white light, respectively, in WT and PT-/- but not in PP-/- and PP-/-/PT-/- (Fig. 4B).
Fig. 4 legend, panel B:
Note that the calcium levels decreased and increased transiently in response to exposure to UV and visible light-mixed white lights, respectively, in WT and PT-/-.
Results, section "Antagonistic Chromatic Responses in a Single Pineal Photoreceptor Cell" (two-photon imaging, Fig. 1C):
stimulation with 405-nm light caused a decrease in fluorescence intensity.
In contrast, stimulation with 588-nm light elevated the fluorescence intensity, demonstrating the antagonistic effect of UV and visible light on the calcium in individual PP cells to UV and visible light (Fig. 1C).
Results, mechanism paragraph (Fig. 3):
The 405-nm light stimulus increases the short-wavelength component of incident light and thus increases the fractional concentration of the signaling active photoproduct from that produced by the imaging light alone, leading to hyperpolarization (Fig. 3 B and D, calcium decrease).
Conversely, because the photoproduct is more sensitive to longer wavelengths, the 588-nm light stimulus decreases its fractional concentration, reducing the pool of signaling-active opsin, resulting in termination of hyperpolarization and increased calcium levels (Fig. 3 C and D, calcium increase).
Discussion, first paragraph:
with lights biased toward UV driving greater hyperpolarization than those biased toward visible light.
Fig. 4 legend, panels A and B (stimulus intensities and sample sizes):
The light intensities (360–780 nm) of white light, UV-mixed white light, visible light-mixed white light, and direct sunlight were ∼9.2 × 1014, ∼9.3 × 1014, ∼9.9 × 1014, and 1.1 × 1017 photons per cm2·s. The light intensities of UV and visible LED lights were ∼1.3 × 1013 and ∼7.6 × 1013 photons per cm2·s, respectively.
Changes in calcium level of PP cells in WT (red open circle; n = 24), PT-/- fish (green closed circle; n = 23), PP-/- fish (purple open triangle; n = 17), and PP-/-/PT-/- fish (black closed triangle; n = 16)
(The exponents read "1014", "1013" and "1017" in the archived text. They are superscripts lost in conversion: 10^14, 10^13, 10^17.)
What the paper's own numbers show
This is a check of direction, not of arithmetic. Each sentence that uses "respectively" pairs the i-th stimulus with the i-th direction word. recompute.py.txt pulls both word orders out of the quoted sentences with a regular expression, builds the stimulus-to-direction mapping for each sentence, and compares the mappings. For the Fig. 4B traces, the direction of the WT and PT-/- deflections at the UV-mixed and visible-mixed bars, and the later course of the traces, were read by eye from the published figure image. No values were measured from it. The script also checks every quotation above against the PMC archive and does the arithmetic on the Fig. 4A intensities.
| Quantity | Location | Reported | Recomputed from the paper's body |
|---|---|---|---|
| Calcium response to UV-enhanced white light | Abstract | increased | decreased |
| Calcium response to visible-enhanced white light | Abstract | decreased | increased |
| Calcium response to UV- / visible-mixed white light | Results, white-light paragraph (Fig. 4B) | decreased / increased | decreased / increased (agrees) |
| Calcium response to UV- / visible-mixed white light | Fig. 4 legend, panel B | decreased / increased | decreased / increased (agrees) |
| Transient deflection of WT and PT-/- traces at UV- / visible-mixed bars | Fig. 4B, plotted data | downward / upward | decreased / increased (agrees; for the later rebound see Explanations, item 4) |
| Fluorescence response to 405-nm / 588-nm light (two-photon) | Results, Fig. 1C | decrease / elevated | decreased / increased (agrees) |
| Predicted calcium change for 405-nm / 588-nm light | Results, mechanism (Fig. 3) | calcium decrease / calcium increase | decreased / increased (agrees) |
| UV-biased vs visible-biased light | Discussion | greater hyperpolarization for UV-biased | lower calcium under UV-biased light (agrees) |
| Body passages with UV → decrease and visible → increase | all of the above except the abstract | — | 5 of 5, plus the Discussion's comparison |
| White + UV LED; white + visible LED (photons per cm²·s) | Fig. 4 legend A | ~9.3 × 10^14; ~9.9 × 10^14 | 9.33 × 10^14 (LED adds 1.4%); 9.96 × 10^14 (LED adds 8.3%). Both agree with the legend within the rounding of its approximate (~) inputs; white light is itself ~9.2 × 10^14 |
| n per genotype, Fig. 4B | Fig. 4 legend B | WT 24, PT-/- 23, PP-/- 17, PP-/-/PT-/- 16 | unaffected by the direction issue |
The abstract gives UV → increase and visible → decrease. Every other passage that states a direction gives UV → decrease and visible → increase. Both added LEDs raise the total photon flux above the same white background (by 1.4% and 8.3%), so the two stimuli differ in spectral composition, not in the sign of the intensity change. The opposite responses are therefore spectral, as the paper argues, and the direction of each is fixed by the body of the paper.
Figure
Figure 1. The left panel quotes the abstract and the right panel quotes the Results sentence on Fig. 4B. Under each quotation is the stimulus-to-direction mapping its "respectively" construction implies. Orange marks the abstract as published. Blue marks what the paper's own body reports. Both directions are reversed between the two panels. The Fig. 4 legend, the transient responses in the plotted Fig. 4B traces, the 405/588-nm experiments and the mechanism all match the right panel. Every word is from the paper or from the output of recompute.py.txt.
Figure 2. For each passage of the paper that states a direction, this grid gives the stated calcium change for UV-shifted light and for visible-shifted light. The abstract (orange, top row) is the only passage with UV → increase and visible → decrease. Six body passages (blue) agree with each other, including the plotted Fig. 4B traces and the mechanism. The Discussion row is a comparison between the two lights rather than a change from baseline.
[Internal, to be removed with candidates/ before sending: five candidate figure designs are in candidates/ for selection. candidate_1 = Figure 1, candidate_2 = Figure 2, candidate_3 = a sign dot plot on one axis, candidate_4 = the "respectively" pairing, candidate_5 = the paper's mechanism chain against the abstract's outcome.]
Why it matters
- The abstract states the headline naturalistic-light result backwards. The Results present Fig. 4B as a test of whether color opponency is observed "under natural light-like conditions"; the abstract reports its outcome. A reader of the abstract alone learns that UV-enriched light raises calcium in parapinopsin-expressing cells, the opposite of what was measured.
- It contradicts the paper's own mechanism. In the paper's model, UV raises the fraction of signaling-active photoproduct, which drives hyperpolarization and a calcium decrease. The abstract's wording implies the reverse. The same holds for the earlier pineal ganglion-cell recordings the Introduction cites, in which firing is "suppressed by UV and enhanced by visible light", although that is a different quantity.
- The error goes further than the paper. PubMed, Europe PMC and Semantic Scholar carry the same abstract. Citing authors, reviewers and model builders who take the sign from the abstract would carry the reversal forward.
- What is not affected. The measured data, the Results, the figure, the sample sizes and the paper's main claim are not affected: opposite-sign responses that disappear without parapinopsin. Only the direction stated in the abstract is wrong.
Explanations considered
- The abstract describes a different quantity. If the abstract meant hyperpolarization, photoproduct amount or ganglion-cell firing, the sign could legitimately differ. It says "calcium signal", the same quantity as the Results and legend (GCaMP6s ΔF/F). This does not account for the conflict.
- The abstract describes a different experiment or population. "Toward enhanced UV or visible wavelengths" matches the UV-mixed and visible-mixed white light of Fig. 4A/B. "Parapinopsin-sufficient but not parapinopsin-deficient" matches WT and PT-/- versus PP-/- and PP-/-/PT-/-. The only experiments that shift the spectrum of white light are Fig. 4B and its repeats at other intensities and durations in SI Appendix Fig. S7. This does not account for the conflict.
- "Respectively" can be read the other way. The abstract lists "UV or visible" and then "increased vs. decreased". The only grammatical pairing is UV with increased. The Results and legend use the same construction with the verbs in the opposite order. This does not account for the conflict.
- The abstract refers to a different time point or phase, such as what follows the stimulus. The plotted traces do show a later change. In WT and PT-/-, calcium dips at the UV-mixed bar, then rebounds above the pre-stimulus baseline and stays elevated until the visible-mixed bar. Taken alone, that rebound has the sign of the UV half of the abstract (UV → increased). It does not account for the abstract, for three reasons. (a) The Results and the Fig. 4 legend name the transient decrease as the response to UV-mixed light: "the calcium levels decreased and increased transiently in response to exposure to UV and visible light-mixed white lights, respectively". The legend adds that the relaxation of these responses was "not observed as a result of time-lapse recoding with low time resolution". (b) The Discussion describes "slow elevation of calcium levels after these levels were greatly decreased" under continuous blue or xenon white light (SI Appendix, Figs. S2 and S6). It relates this to "the kinetics of signaling-active photoproduct", which "could constitute a mechanism of light adaptation", and not to the spectral content of a stimulus. Applying that to the rebound in Fig. 4B is this write-up's inference, but it is the only account of such a rise that the paper offers. (c) The visible half of the abstract (visible → decreased) has no counterpart in the figure. After the visible-mixed peak, the traces return to about the level they had just before the visible-mixed bar, and no decrease follows. So even read as a later phase, the abstract's full mapping is not shown. This does not account for the conflict.
- The difference reflects total intensity rather than spectrum. Both mixed lights add photons (1.4% and 8.3% above the white light), so intensity cannot give opposite signs, and it does not bear on which sign is which. This does not account for the conflict.
- One Discussion sentence uses "increase". The Discussion speaks of "maximum changes in increase of calcium level in PP cells (10–20 s; SI Appendix, Fig. S7)". It is ambiguous and does not say which stimulus produced the increase. The SI text speaks only of "calcium decrease profiles in Fig. S7 (~20 s for reaching the maximum)" and does not name a stimulus; Fig. S7 repeats the UV- and visible-mixed stimuli, so linking that phrase to the UV-mixed responses is an inference. Neither passage supports the abstract's mapping.
- Text-conversion artifact. The conflict is in words, not numbers, and the same abstract wording appears in the Europe PMC XML, PubMed and Semantic Scholar. A lost minus sign, superscript or table cell cannot produce it. This does not account for the conflict.
- Already corrected. No correction or erratum was found (see Status).
None of these accounts for the reversal. The abstract's direction words for the Fig. 4B experiment are in the opposite order to the rest of the paper.
Status
As of 2026-10-23:
- Version of record: checked. The PNAS article page (reached through doi.org) and the PMC HTML page refused automated access (HTTP 403 and a bot check), so the publisher's HTML could not be read directly. The publisher-supplied text was read in the Europe PMC full-text JATS XML for PMC6217433, in the PubMed record (PMID 30322939) and in Semantic Scholar. All three carry the abstract sentence word for word, and the XML carries the Results and legend sentences quoted above. The PMC archive used here (
src/show.py) gives the same passages. - Figure: the published Fig. 4 image (pnas.1802592115fig04.jpg, from the PMC / Europe PMC package) was inspected. The WT and PT-/- traces drop at the UV-mixed bar, then rebound above the pre-stimulus baseline and stay elevated until the visible-mixed bar. They peak at the visible-mixed bar and then return to about their level just before it. The PP-/- and PP-/-/PT-/- traces stay flat. The figure is not reproduced here.
- Supplements: the SI Appendix (pnas.1802592115.sapp.pdf, 18 pages) was read. The Fig. S6 legend refers the UV- and visible-mixed responses to Fig. 4B. The Fig. S7 legend describes repeats at other LED intensities and durations. The SI text speaks of "calcium decrease profiles in Fig. S7" and says that more signaling-active photoproduct "would facilitate depolarizing responses to visible-light stimuli". All of this agrees with the Results, not the abstract. No source data are deposited.
- Corrections and notices:
src/known.pyreturns no Crossref updates, PMC notices or Europe PMC notices. The Crossref record has no update-to or updated-by relation. No correction, erratum, expression of concern or retraction was found. - PubPeer: a web search for the title together with "pubpeer" returned no PubPeer page. The PubPeer site and API refused automated queries (HTTP 403, Cloudflare), so a thread there cannot be ruled out.
- Sources: PMC6217433 (
src/show.py);src/known.py 10.1073/pnas.1802592115; https://doi.org/10.1073/pnas.1802592115; https://www.ebi.ac.uk/europepmc/webservices/rest/PMC6217433/fullTextXML; https://www.ebi.ac.uk/europepmc/webservices/rest/PMC6217433/supplementaryFiles; PubMed PMID 30322939.
Suggested correction
In the Abstract, replace
Changes in the spectral composition of white light toward enhanced UV or visible wavelengths respectively increased vs. decreased calcium signal in parapinopsin-sufficient but not parapinopsin-deficient photoreceptors.
with
Changes in the spectral composition of white light toward enhanced UV or visible wavelengths respectively decreased vs. increased calcium signal in parapinopsin-sufficient but not parapinopsin-deficient photoreceptors.
This makes the abstract agree with the Results, the Fig. 4 legend, the plotted Fig. 4B data and the paper's proposed mechanism. No other part of the paper needs to change.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC6217433 (Proc Natl Acad Sci U S A 2018, doi:10.1073/pnas.1802592115)
# Abstract vs Results / Fig. 4B: direction of the calcium response to UV- and visible-enriched white light
#
# Run: python recompute.py.txt
# Prints every number and direction used in writeup.md and regenerates, in the folder that
# holds this file:
# fig1.svg / fig1.png two text panels: Abstract vs Results + Fig. 4 legend
# fig2.svg / fig2.png direction of the calcium change in every passage that states one
# candidates/candidate_{1..5}.svg/.png five candidate figure designs (1 = fig1, 2 = fig2), for selection
# Needs matplotlib (numpy only for the arithmetic).
#
# Inputs
# * Quotes: copied verbatim from the article as archived in PubMed Central (PMC6217433).
# When run inside the /home/ubuntu/papers repo, the script re-checks each quote against the
# archived full text with src/show.py --grep (skipped elsewhere).
# * Light intensities: Fig. 4 legend (A).
# * Fig. 4B plotted traces: the direction of the WT and PT-/- deflections was read by eye from
# the published figure image (pnas.1802592115fig04.jpg in the PMC / Europe PMC package).
# No values are measured from it and the image is not reproduced. The later course of the
# traces (rebound above baseline after the UV-mixed dip) was read the same way.
import os, re, subprocess, sys, textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
"axes.edgecolor": SEC, "axes.labelcolor": TXT, "xtick.color": TXT,
"ytick.color": TXT, "svg.fonttype": "none"})
# ------------------------------------------------------------------ quotes
Q = {
"abstract": ("Abstract",
"Changes in the spectral composition of white light toward enhanced UV or visible wavelengths "
"respectively increased vs. decreased calcium signal in parapinopsin-sufficient but not "
"parapinopsin-deficient photoreceptors."),
"results4b": ("Results, 'Color Opponency Based on Photoequilibrium of Parapinopsin Two States', white-light paragraph",
"We found decreased and increased calcium levels following exposure to UV and visible "
"light-mixed white light, respectively, in WT and PT-/- but not in PP-/- and PP-/-/PT-/- (Fig. 4B)."),
"legend4b": ("Fig. 4 legend, panel B",
"Note that the calcium levels decreased and increased transiently in response to exposure to "
"UV and visible light-mixed white lights, respectively, in WT and PT-/-."),
"fig1c_405": ("Results, 'Antagonistic Chromatic Responses in a Single Pineal Photoreceptor Cell'",
"stimulation with 405-nm light caused a decrease in fluorescence intensity."),
"fig1c_588": ("Results, 'Antagonistic Chromatic Responses in a Single Pineal Photoreceptor Cell'",
"In contrast, stimulation with 588-nm light elevated the fluorescence intensity, demonstrating "
"the antagonistic effect of UV and visible light on the calcium in individual PP cells to UV "
"and visible light (Fig. 1C)."),
"mech_405": ("Results, mechanism paragraph (Fig. 3)",
"The 405-nm light stimulus increases the short-wavelength component of incident light and thus "
"increases the fractional concentration of the signaling active photoproduct from that produced "
"by the imaging light alone, leading to hyperpolarization (Fig. 3 B and D, calcium decrease)."),
"mech_588": ("Results, mechanism paragraph (Fig. 3)",
"Conversely, because the photoproduct is more sensitive to longer wavelengths, the 588-nm light "
"stimulus decreases its fractional concentration, reducing the pool of signaling-active opsin, "
"resulting in termination of hyperpolarization and increased calcium levels (Fig. 3 C and D, "
"calcium increase)."),
"discussion": ("Discussion, first paragraph",
"with lights biased toward UV driving greater hyperpolarization than those biased toward visible light."),
"intro": ("Introduction, second paragraph",
"with firing of these neurons suppressed by UV and enhanced by visible light"),
"legend4a": ("Fig. 4 legend, panel A",
"The light intensities (360–780 nm) of white light, UV-mixed white light, visible light-mixed "
"white light, and direct sunlight were ∼9.2 × 1014, ∼9.3 × 1014, ∼9.9 × 1014, and 1.1 × 1017 "
"photons per cm2·s. The light intensities of UV and visible LED lights were ∼1.3 × 1013 and "
"∼7.6 × 1013 photons per cm2·s, respectively."),
"legend4b_n": ("Fig. 4 legend, panel B",
"Changes in calcium level of PP cells in WT (red open circle; n = 24), PT-/- fish (green closed "
"circle; n = 23), PP-/- fish (purple open triangle; n = 17), and PP-/-/PT-/- fish (black closed "
"triangle; n = 16)"),
}
def norm(s):
return re.sub(r"\s+", " ", s).strip()
def check_quotes():
repo = os.path.abspath(os.path.join(HERE, "..", "..", ".."))
show = os.path.join(repo, "src", "show.py")
if not os.path.exists(show):
print("[quote check skipped: src/show.py not found]")
return
py = "/home/ubuntu/papers-data/venv/bin/python"
py = py if os.path.exists(py) else sys.executable
print("=== Verbatim quote check against the PMC archive (src/show.py --grep) ===")
for k, (loc, q) in Q.items():
probe = q[:60]
out = subprocess.run([py, show, "PMC6217433", "--grep", probe], cwd=repo,
capture_output=True, text=True).stdout
ok = norm(q) in norm(out)
print(f" {'OK ' if ok else 'MISS'} {k:11s} ({loc})")
check_quotes()
# ------------------------------------------------------------------ direction mapping
# 'respectively' pairs the i-th stimulus with the i-th direction word; both orders are
# extracted from the quoted sentences themselves.
def dirword(w):
w = w.lower()
return "increase" if w.startswith("increas") else "decrease"
m = re.search(r"toward enhanced (\w+) or (\w+) wavelengths respectively (\w+) vs\. (\w+) calcium",
Q["abstract"][1])
ABS = {m.group(1): dirword(m.group(3)), m.group(2): dirword(m.group(4))}
m = re.search(r"found (\w+) and (\w+) calcium levels following exposure to (\w+) and (\w+) light-mixed",
Q["results4b"][1])
RES = {m.group(3): dirword(m.group(1)), m.group(4): dirword(m.group(2))}
m = re.search(r"calcium levels (\w+) and (\w+) transiently in response to exposure to (\w+) and (\w+) light-mixed",
Q["legend4b"][1])
LEG = {m.group(3): dirword(m.group(1)), m.group(4): dirword(m.group(2))}
M405 = "decrease" if "caused a decrease" in Q["fig1c_405"][1] else "?"
M588 = "increase" if "588-nm light elevated" in Q["fig1c_588"][1] else "?"
MECH405 = "decrease" if "calcium decrease" in Q["mech_405"][1] else "?"
MECH588 = "increase" if "calcium increase" in Q["mech_588"][1] else "?"
DISC = ("lower", "higher") if "biased toward UV driving greater hyperpolarization" in Q["discussion"][1] else ("?", "?")
FIG4B_IMAGE = ("decrease", "increase") # transient deflection at each bar, read by eye from the published Fig. 4B image (WT, PT-/-)
FIG4B_LATER = { # later course of the WT and PT-/- traces, read by eye from the same image
"after UV-mixed dip": "rebound above the pre-stimulus baseline, elevated until the visible-mixed bar",
"after visible-mixed peak": "back to about the level just before the visible-mixed bar; no decrease follows",
}
print("\n=== Direction of the calcium change stated for each stimulus ===")
print(f" Abstract UV-enhanced: {ABS['UV']:8s} visible-enhanced: {ABS['visible']}")
print(f" Results, Fig. 4B paragraph UV-mixed: {RES['UV']:8s} visible-mixed: {RES['visible']}")
print(f" Fig. 4 legend (B) UV-mixed: {LEG['UV']:8s} visible-mixed: {LEG['visible']}")
print(f" Fig. 4B traces (published image) UV-mixed: {FIG4B_IMAGE[0]:8s} visible-mixed: {FIG4B_IMAGE[1]}")
print(f" Results, Fig. 1C (two-photon) 405 nm: {M405:8s} 588 nm: {M588}")
print(f" Results, mechanism (Fig. 3) 405 nm: {MECH405:8s} 588 nm: {MECH588}")
print(f" Discussion (relative) UV-biased: {DISC[0]:8s} visible-biased: {DISC[1]} (calcium, via 'greater hyperpolarization')")
for k in ("UV", "visible"):
print(f" {k:8s} abstract == Results? {ABS[k] == RES[k]} Results == legend? {RES[k] == LEG[k]}")
body = [(RES["UV"], RES["visible"]), (LEG["UV"], LEG["visible"]), FIG4B_IMAGE, (M405, M588), (MECH405, MECH588)]
n_agree = sum(1 for b in body if b == ("decrease", "increase"))
print(f" Body statements with UV -> decrease and visible -> increase: {n_agree} of {len(body)}"
f" (plus the Discussion's relative statement)")
print(f" Abstract matches the body: {(ABS['UV'], ABS['visible']) == ('decrease', 'increase')}")
print(" Fig. 4B, later course of the WT and PT-/- traces (published image, by eye):")
for k, v in FIG4B_LATER.items():
print(f" {k:25s} {v}")
print(" -> the UV half of the abstract (UV -> increased) has a later counterpart (the rebound);")
print(" the visible half (visible -> decreased) has none, so the abstract's full mapping is not shown.")
# ------------------------------------------------------------------ light intensities (Fig. 4 legend A)
print("\n=== Fig. 4A light intensities (photons per cm2·s, 360-780 nm) ===")
white, uv_mix, vis_mix = 9.2e14, 9.3e14, 9.9e14
uv_led, vis_led = 1.3e13, 7.6e13
for lab, led, rep in (("UV-mixed", uv_led, uv_mix), ("visible-mixed", vis_led, vis_mix)):
tot = white + led
print(f" {lab:14s} white + LED = {tot:.3g} (reported ~{rep:.2g}); LED adds {led / white:.1%} to the white light")
# rounding check: each '~' value is taken as exact to its last printed digit (half a unit either way)
def rng(v, step):
return v - step / 2, v + step / 2
w_lo, w_hi = rng(white, 0.1e14)
for lab, led, rep in (("UV-mixed", uv_led, uv_mix), ("visible-mixed", vis_led, vis_mix)):
l_lo, l_hi = rng(led, 0.1e13)
r_lo, r_hi = rng(rep, 0.1e14)
lo, hi = w_lo + l_lo, w_hi + l_hi
print(f" {lab:14s} sum within rounding: {lo:.3e}-{hi:.3e}; reported ~{rep:.2g} covers {r_lo:.3e}-{r_hi:.3e};"
f" overlap: {lo <= r_hi and r_lo <= hi}")
print(" Both mixed lights add photons to the same white background (total intensity rises in both),")
print(" so the two stimuli differ in spectral composition, not in the sign of the intensity change.")
print("\n=== Fig. 4B sample sizes (legend; not affected by the direction issue) ===")
print(" WT 24, PT-/- 23, PP-/- 17, PP-/-/PT-/- 16; total", 24 + 23 + 17 + 16)
# ------------------------------------------------------------------ figure helpers
def newfig(h):
fig = plt.figure(figsize=(7.2, h), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2); ax.set_ylim(0, h); ax.axis("off"); ax.set_facecolor(BG)
return fig, ax
def save(fig, *names):
for n in names:
fig.savefig(n + ".svg", facecolor=BG)
fig.savefig(n + ".png", dpi=200, facecolor=BG)
plt.close(fig)
def panel(ax, x, y, w, h, color, title, sub, body, width_chars, fs=10.5):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=color, lw=2))
ax.add_patch(FancyBboxPatch((x, y + h - 0.08), w, 0.08, boxstyle="square,pad=0", fc=color, ec=color))
ax.text(x + 0.15, y + h - 0.27, title, fontsize=12, fontweight="bold", color=color, va="top")
ax.text(x + 0.15, y + h - 0.55, sub, fontsize=10, color=SEC, va="top")
ax.text(x + 0.15, y + h - 0.85, textwrap.fill(body, width_chars), fontsize=fs, va="top",
linespacing=1.35)
# ------------------------------------------------------------------ candidate 1 / fig1: two text panels
def cand1():
fig, ax = newfig(4.4)
ax.text(0.15, 4.25, "Abstract and Results give opposite directions for the Fig. 4B experiment",
fontsize=11.5, fontweight="bold", va="top")
panel(ax, 0.15, 0.95, 3.25, 3.0, ORANGE, "Abstract", "as published",
"“…white light toward enhanced UV or visible wavelengths respectively "
"increased vs. decreased calcium signal in parapinopsin-sufficient … photoreceptors.”", 36)
panel(ax, 3.8, 0.95, 3.25, 3.0, BLUE, "Results + Fig. 4 legend", "what the paper's body reports",
"“We found decreased and increased calcium levels following exposure to UV and visible "
"light-mixed white light, respectively, in WT and PT-/- … (Fig. 4B).”", 36)
# mapping rows inside panels
for x0, col, uvd, visd in ((0.3, ORANGE, ABS["UV"], ABS["visible"]), (3.95, BLUE, RES["UV"], RES["visible"])):
for yy, lab, dd in ((1.55, "UV-enriched", uvd), (1.2, "visible-enriched", visd)):
ax.text(x0, yy, lab, fontsize=10, color=col, fontweight="bold")
ax.text(x0 + 1.34, yy, f"→ calcium {dd}d", fontsize=10, color=col, fontweight="bold")
ax.add_patch(FancyArrowPatch((3.42, 1.48), (3.78, 1.48), arrowstyle="<->", mutation_scale=14,
color=SEC, lw=1.5))
ax.text(3.6, 0.72, "Conflict: both directions are reversed.", ha="center", fontsize=10.5,
fontweight="bold", color=TXT)
ax.text(3.6, 0.42, "The transient Fig. 4B responses, the 405/588-nm experiments (Fig. 1C) and the mechanism (Fig. 3)",
ha="center", fontsize=10, color=SEC)
ax.text(3.6, 0.17, "agree with the Results: UV-enriched light lowers calcium, visible-enriched light raises it.",
ha="center", fontsize=10, color=SEC)
return fig
save(cand1(), os.path.join(HERE, "fig1"), os.path.join(CAND, "candidate_1"))
# ------------------------------------------------------------------ candidate 2 / fig2: direction grid
ROWS = [
("Abstract", "enhanced UV / enhanced visible white light", ABS["UV"], ABS["visible"], ORANGE),
("Results, Fig. 4B paragraph", "UV- / visible-mixed white light", RES["UV"], RES["visible"], BLUE),
("Fig. 4 legend, panel B", "UV- / visible-mixed white light", LEG["UV"], LEG["visible"], BLUE),
("Fig. 4B traces (WT, PT-/-)", "transient, from the published image", FIG4B_IMAGE[0], FIG4B_IMAGE[1], BLUE),
("Results, Fig. 1C", "405-nm / 588-nm stimulus, two-photon", M405, M588, BLUE),
("Results, mechanism (Fig. 3)", "405-nm / 588-nm stimulus", MECH405, MECH588, BLUE),
("Discussion", "UV-biased / visible-biased light", "lower*", "higher*", BLUE),
]
SYM = {"decrease": "▼ decrease", "increase": "▲ increase", "lower*": "▼ lower*", "higher*": "▲ higher*"}
def cand2():
h = 0.95 + 0.48 * len(ROWS) + 0.75
fig, ax = newfig(h)
top = h - 0.25
ax.text(0.15, top, "Direction of the calcium change in every passage that states one", fontsize=11.5,
fontweight="bold", va="top")
ax.text(3.95, top - 0.5, "UV-shifted light", ha="center", fontsize=10.5, fontweight="bold")
ax.text(5.9, top - 0.5, "visible-shifted light", ha="center", fontsize=10.5, fontweight="bold")
y = top - 0.95
for i, (src, stim, d1, d2, col) in enumerate(ROWS):
if i == 0:
ax.add_patch(FancyBboxPatch((0.1, y - 0.2), 7.0, 0.44, boxstyle="round,pad=0.01,rounding_size=0.05",
fc="#fdf0ea", ec=ORANGE, lw=2))
else:
ax.plot([0.15, 7.05], [y + 0.24, y + 0.24], color=GRID, lw=1)
ax.text(0.2, y + 0.05, src, fontsize=10.5, fontweight="bold" if i == 0 else "normal", va="center", color=TXT)
ax.text(0.2, y - 0.13, stim, fontsize=10, color=SEC, va="center")
ax.text(3.95, y, SYM[d1], ha="center", va="center", fontsize=11, color=col, fontweight="bold")
ax.text(5.9, y, SYM[d2], ha="center", va="center", fontsize=11, color=col, fontweight="bold")
y -= 0.48
ax.text(0.15, y + 0.05, "Orange: the abstract as published. Blue: the paper's own body (text, legend, figure, mechanism).",
fontsize=10, color=SEC, va="top")
ax.text(0.15, y - 0.22, "* A comparison: UV-biased light drives “greater hyperpolarization”, i.e. lower calcium.",
fontsize=10, color=SEC, va="top")
return fig
save(cand2(), os.path.join(HERE, "fig2"), os.path.join(CAND, "candidate_2"))
# ------------------------------------------------------------------ candidate 3: sign dot plot on one axis
def cand3():
fig = plt.figure(figsize=(7.2, 3.9), facecolor=BG)
ax = fig.add_axes([0.14, 0.2, 0.52, 0.66]); ax.set_facecolor(BG)
sign = {"increase": 1, "decrease": -1}
body_src = [("Results text", RES), ("Fig. 4 legend", LEG),
("Fig. 4B traces", {"UV": FIG4B_IMAGE[0], "visible": FIG4B_IMAGE[1]}),
("Fig. 1C 405/588 nm", {"UV": M405, "visible": M588}),
("Fig. 3 mechanism", {"UV": MECH405, "visible": MECH588})]
for xi, k in enumerate(("UV", "visible")):
ax.plot(xi - 0.18, sign[ABS[k]], "o", ms=11, color=ORANGE, zorder=3)
for j, (_, d) in enumerate(body_src):
ax.plot(xi + 0.04 + 0.11 * j, sign[d[k]], "s", ms=8, color=BLUE, zorder=3)
ax.axhline(0, color=SEC, lw=0.8)
ax.set_xlim(-0.55, 1.85); ax.set_ylim(-1.6, 1.6)
ax.set_xticks([0.1, 1.1]); ax.set_xticklabels(["UV-shifted light", "visible-shifted light"])
ax.set_yticks([-1, 0, 1]); ax.set_yticklabels(["calcium\ndecrease", "no change", "calcium\nincrease"])
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
fig.text(0.02, 0.95, "Sign of the calcium response: abstract vs the paper's body", fontsize=11.5, fontweight="bold")
fig.text(0.69, 0.74, "● Abstract, as published", color=ORANGE, fontsize=10.5, fontweight="bold")
fig.text(0.69, 0.66, "■ Paper's body,\n 5 passages:", color=BLUE, fontsize=10.5,
fontweight="bold", va="top")
fig.text(0.71, 0.53, "\n".join(n for n, _ in body_src), color=SEC, fontsize=10, va="top", linespacing=1.4)
fig.text(0.02, 0.04, "Only the sign is plotted; the paper states directions, not a single summary value.",
fontsize=10, color=SEC)
return fig
save(cand3(), os.path.join(CAND, "candidate_3"))
# ------------------------------------------------------------------ candidate 4: 'respectively' pairing
def token(ax, x, y, s, col, w=1.55):
ax.add_patch(FancyBboxPatch((x - w / 2, y - 0.16), w, 0.32, boxstyle="round,pad=0.02,rounding_size=0.06",
fc="white", ec=col, lw=2))
ax.text(x, y, s, ha="center", va="center", fontsize=10.5, color=TXT)
def cand4():
fig, ax = newfig(3.9)
ax.text(0.15, 3.75, "How “respectively” pairs each light with a direction", fontsize=11.5,
fontweight="bold", va="top")
for x0, col, title, stims, verbs, mp in (
(0.25, ORANGE, "Abstract", ("enhanced UV", "enhanced visible"), ("increased", "decreased"), ABS),
(3.85, BLUE, "Results (Fig. 4B)", ("UV-mixed", "visible-mixed"), ("decreased", "increased"), RES)):
ax.text(x0, 3.25, title, fontsize=12, fontweight="bold", color=col)
ax.text(x0, 2.98, "word order in the sentence", fontsize=10, color=SEC)
for i in range(2):
xs = x0 + 0.85 + i * 1.65
token(ax, xs, 2.5, stims[i], col)
token(ax, xs, 1.55, verbs[i], col)
ax.add_patch(FancyArrowPatch((xs, 2.32), (xs, 1.73), arrowstyle="-|>", mutation_scale=12, color=col, lw=2))
ax.text(xs + 0.08, 2.02, f"{i + 1}st" if i == 0 else "2nd", fontsize=10, color=SEC)
ax.text(x0, 1.0, f"UV → calcium {mp['UV']}", fontsize=10.5, color=col, fontweight="bold")
ax.text(x0, 0.72, f"visible → calcium {mp['visible']}", fontsize=10.5, color=col, fontweight="bold")
ax.plot([3.6, 3.6], [0.5, 3.4], color=GRID, lw=1)
ax.text(3.6, 0.2, "Same pairing rule, opposite verb order: the two sentences assign opposite directions to each light.",
ha="center", fontsize=10, color=SEC)
return fig
save(cand4(), os.path.join(CAND, "candidate_4"))
# ------------------------------------------------------------------ candidate 5: mechanism chain
def box(ax, x, y, s, col, w=1.45, h=0.62, fc="white"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.06", fc=fc, ec=col, lw=2))
ax.text(x + w / 2, y + h / 2, s, ha="center", va="center", fontsize=10, color=TXT, linespacing=1.25)
def cand5():
fig, ax = newfig(4.3)
ax.text(0.15, 4.15, "The paper's mechanism predicts the Results' direction, not the abstract's",
fontsize=11.5, fontweight="bold", va="top")
ax.text(0.15, 3.82, "Chain from the Results (mechanism paragraph, Fig. 3, 405/588 nm), generalized in the Discussion", fontsize=10,
color=SEC, va="top")
rows = ((2.75, "UV-shifted\nwhite light", "more active\nphotoproduct", "hyperpolarization", "calcium\ndecreases",
f"calcium\n{ABS['UV']}d"),
(1.45, "visible-shifted\nwhite light", "less active\nphotoproduct", "hyperpolarization\nends",
"calcium\nincreases", f"calcium\n{ABS['visible']}d"))
for y, a, b, c, d, e in rows:
xs = [0.15, 1.6, 3.05, 4.5]
for x, s in zip(xs, (a, b, c, d)):
box(ax, x, y, s, BLUE if x == 4.5 else SEC, w=1.3, fc="#eef4fc" if x == 4.5 else "white")
for x in xs[:-1]:
ax.add_patch(FancyArrowPatch((x + 1.32, y + 0.31), (x + 1.44, y + 0.31), arrowstyle="-|>",
mutation_scale=12, color=SEC, lw=1.5))
box(ax, 6.1, y, e, ORANGE, w=0.98, fc="#fdf0ea")
ax.text(5.95, y + 0.31, "≠", ha="center", va="center", fontsize=16, color=TXT, fontweight="bold")
ax.text(5.15, 3.5, "Results / Fig. 4B", ha="center", fontsize=10, color=BLUE, fontweight="bold")
ax.text(6.59, 3.5, "Abstract", ha="center", fontsize=10, color=ORANGE, fontweight="bold")
ax.text(0.15, 1.0, "Blue: the outcome the paper's body reports, which follows from its mechanism.", fontsize=10, color=SEC)
ax.text(0.15, 0.72, "Orange: the outcome the abstract states, which reverses it for both lights.", fontsize=10, color=SEC)
ax.text(0.15, 0.44, "Same two lights, same calcium indicator (GCaMP6s), same parapinopsin-sufficient cells (WT, PT-/-).",
fontsize=10, color=SEC)
return fig
save(cand5(), os.path.join(CAND, "candidate_5"))
print("\nFigures written to", HERE, "and", CAND)
Reported P < 0.0001 for a light-versus-dark coiling difference is inconsistent with the paper's counts
Sci Adv (2020) · doi:10.1126/sciadv.abb8930 · PMC10662364
Summary
The Results report that offspring of a single N. pachyderma parent coiled sinistrally more often in the 12-hour light flask than in the 24-hour dark flask: 87.5% (n = 9) against 50% (n = 12), with P < 0.0001 by a proportion z test. The counts that fit the paper's own percentages and totals are 7 of 8 and 6 of 12. They add up to the stated 13 of 20 sinistral offspring, and no count out of 9 gives 87.5%. A two-proportion z test on these counts gives z = 1.72 and two-sided P = 0.085, and Fisher's exact test gives P = 0.158. None of the standard variants I tried comes close to 0.0001. The Discussion relies on this contrast when it suggests that light conditions, or the pH and O2 changes that come with them, could contribute to coiling direction. That inference loses its statistical support, but the paper's main observations of asexual reproduction and morphological variability among clones do not depend on it.
What the paper reports
Results, "Morphology of parent and offspring", second paragraph:
Coiling direction was ascertained in 20 offspring, 13 of which coiled sinistrally (65%). There were differences in coiling direction proportion between the 12-hour light and 24-hour dark treatments (see Materials and Methods): 87.5% were sinistral (n = 9) in the 12-hour light treatment (flask “O”), and 50% (n = 12) in the 24-hour dark treatment (flask “D”) (P < 0.0001, by a proportion z test).
Materials and Methods, third paragraph:
No shells we recovered from flask “A,” eight offspring and the parent were recovered from flask “O,” and 17 offspring from flask “D.”
Discussion, "Phenotypic variation in a genetically similar population", second paragraph:
The relative abundance of shells coiling in the nondominant direction was greater in the 24-hour dark treatment (50%) than in the 12-hour light treatment (12.5%), indicating that factors such as light conditions or associated changes in pH and O2 could contribute to coiling direction.
What the paper's own numbers show
Method. I first found the whole-number counts consistent with each printed percentage, its printed n, the Methods recovery counts and the stated total of 13 sinistral out of 20. I then ran the two-group test the paper names (a two-proportion z test with pooled standard error, two-sided) on those counts, along with the common variants: unpooled standard error, continuity correction, one-sided, Fisher's exact test, Boschloo's exact test, and the same tests with the sinistral parent counted as a ninth shell in flask O. Last, I searched for any calculation that yields P < 0.0001. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Flask O, sinistral | Results | 87.5% (n = 9) | No count out of 9 gives 87.5% (87.5% × 9 = 7.875; 7/9 = 77.8%, 8/9 = 88.9%). 7/8 = 87.5%, and Methods report 8 offspring (plus the parent) from flask O |
| Flask D, sinistral | Results | 50% (n = 12) | 6/12 = 50%; 12 of the 17 offspring recovered from flask D |
| All offspring scored | Results | 13 of 20 sinistral (65%) | 7 + 6 = 13, 8 + 12 = 20, 13/20 = 65%. With n = 9 the total would be 21 |
| Nondominant (dextral) coiling | Discussion | 12.5% vs 50% | 1/8 = 12.5%, 6/12 = 50%: the same contrast as above |
| Pooled two-proportion z test, 7/8 vs 6/12 | Results | P < 0.0001 | pooled p = 0.65, SE = 0.2177, z = 1.723, two-sided P = 0.085 (one-sided 0.043) |
| Same, continuity corrected | Results | P < 0.0001 | z = 1.244, P = 0.213 (equal to chi-square with Yates correction) |
| Unpooled (Wald) z test | Results | P < 0.0001 | z = 2.019, P = 0.044 |
| Fisher's exact test | Results | P < 0.0001 | P = 0.158 (odds ratio 7.0) |
| Boschloo's exact test | Results | P < 0.0001 | P = 0.119 |
| Parent counted, 8/9 vs 6/12 | Results | P < 0.0001 | pooled z = 1.871, P = 0.061; unpooled z = 2.181, P = 0.029; Fisher P = 0.159 |
| 95% intervals | (derived) | — | Wilson: flask O 52.9% to 97.8%, flask D 25.4% to 74.6%; Newcombe for the 37.5-point difference: −5.0 to 64.2 points, which includes 0 |
| Equal group size needed for P < 0.0001 at 87.5% vs 50% | (derived) | n = 9 and n = 12 | 47 per group (pooled z test) |
| One-sample z test, flask D against flask O's estimate | (candidate explanation) | P < 0.0001 | 6/12 against a fixed 0.875: z = −3.93, P = 8.6e-5 (exact binomial P = 0.0018), but this is not a two-group comparison |
| Percentages entered as counts out of 100 | (candidate explanation) | P < 0.0001 | 87.5/100 vs 50/100: z = 5.72, P = 1.1e-8, but this is not a valid test of these data |
Figure
Figure 1. Two-sided P values for the light-versus-dark comparison of sinistral coiling, on a logarithmic axis. The orange bar and arrow mark the paper's reported value, P < 0.0001, which is an upper bound. The filled blue dots are P values recomputed from the counts that fit the paper's percentages and totals (7 of 8 versus 6 of 12, or 8 of 9 if the sinistral parent is counted). They range from 0.044 to 0.213 and lie near or above the dashed 0.05 line, more than 400 times the reported bound. The two hollow blue dots are the only calculations found that give a value below 0.0001: a one-sample z test of flask D's 6 of 12 against flask O's 87.5% treated as a fixed value (P = 8.6e-5), and the two percentages treated as counts out of 100 (P = 1.1e-8). They are shown only for comparison; neither is a valid two-group comparison of these data. All values come from recompute.py.txt.
Why it matters
The P value attaches to a secondary comparison, and the direction of the observed difference stays as reported: 1 of 8 dextral shells in the light flask against 6 of 12 in the dark flask. What changes is the strength of the evidence. As printed, P < 0.0001 tells the reader that a difference this large is very unlikely to arise by chance between two groups that coil alike. With 8 and 12 shells, the correct pooled z test gives P = 0.085, and the 95% interval for the difference runs from about −5 to 64 percentage points. The data are compatible with no effect of the treatment on coiling.
The Discussion uses this contrast to suggest that "light conditions or associated changes in pH and O2 could contribute to coiling direction". That is presented as a possibility, but readers may take the printed P < 0.0001 as strong support for it. The paper notes that the two species are "generally distinguished by shell coiling direction", so a claimed environmental effect on coiling bears on how field specimens are assigned to species. The comparison involves one parent's clonal offspring in two flasks, so even a significant result would not separate light from any other difference between the flasks, and the recomputed P values (0.03 to 0.21, two-sided) leave it without strong statistical support.
The paper's headline findings do not rest on this test: that the parent reproduced asexually, that its clonal offspring spanned four morphotypes, and that offspring coiled in both directions (13 of 20 sinistral). The title and abstract do not report the light-versus-dark P value, and the paper has no tables.
Explanations considered
- Typing error in n. The printed n = 9 is impossible for 87.5% and inconsistent with the total of 20. Methods report "eight offspring and the parent" from flask O, so n = 9 may count the parent while 87.5% uses the 8 offspring. This explains the denominator but not the P value: with either 8 or 9 shells, the tests give P between 0.029 and 0.213.
- A different variant of the z test. Pooled or unpooled standard error, with or without continuity correction, one- or two-sided: the smallest P across all these, with either denominator, is 0.0146 (one-sided, unpooled, parent counted). None reaches 0.0001.
- A different two-group test. Fisher's exact test (P = 0.158), Boschloo's exact test (P = 0.119) and chi-square with Yates correction (P = 0.213) give larger P values than the z test. Chi-square without correction is the same test as the pooled z test and gives the same P = 0.085.
- More shells than the text states. P < 0.0001 at these percentages would need about 47 shells per group. The Results say coiling was ascertained in 20 offspring, and Methods report only 8 offspring from flask O, so the scored samples cannot be that large.
- Comparison against a fixed reference value. Testing flask D's 6 of 12 against a fixed proportion of 0.875 with a one-sample z test gives z = −3.93 and P = 8.6e-5, which would be reported as P < 0.0001 and fits the wording "by a proportion z test". That treats flask O's estimate, based on 8 shells, as a known constant; the exact binomial version gives P = 0.0018. Testing flask O's 7 of 8 against 0.5 gives one-sample z test P = 0.034 (exact binomial P = 0.070). Neither is the two-group comparison the text describes.
- Comparison with the population rate of nondominant coiling. The Results state that natural populations contain 1 to 3% nondominant coilers. Against 3%, flask D's 6 of 12 (one-sided exact P = 5.8e-7) and all offspring's 7 of 20 (P = 1.2e-6) are significant, but flask O's 1 of 8 is not (P = 0.22). This is a different question from the one the sentence reports, which is the difference between treatments.
- Percentages entered as counts. A two-proportion z test on 87.5 of 100 versus 50 of 100 gives z = 5.72 and P = 1.1e-8, which would be reported as P < 0.0001. It is not a valid test, because the sample sizes are 8 and 12, not 100.
- Which calculation, if any. Two calculations found give P < 0.0001: the one-sample z test of flask D against flask O's estimate (P = 8.6e-5; exact binomial P = 0.0018) and the percentages read as counts out of 100 (P = 1.1e-8). Neither is a valid comparison of the two flasks, and no two-group test on the paper's counts comes close. The source of the printed value cannot be established from the paper.
- Conversion or typesetting artifact. Ruled out (see Status): the publisher's typeset PDF, the Europe PMC XML and the PMC text all print the same digits, with a real less-than sign and no superscripts or minus signs.
Status
As of 2026-10-23:
- Version of record. The doi.org and science.org pages were not reachable by script (HTTP 403), so the text was checked in the publisher's typeset PDF as mirrored on NSF PAR (par.nsf.gov/servlets/purl/10197656; 8 pages, header "Sci. Adv. 2020; 6 : eabb8930 10 July 2020", published 10 July 2020), in the Europe PMC full-text XML, and in the PMC archived text. All three carry the same sentences and numbers quoted above.
- Supplement. The Europe PMC supplementary archive holds abb8930_SM.pdf (captions for Movies S1 to S3 only) and three movie files. It contains no counts per flask and no statistics. The data statement says additional data may be requested from the authors. The supplement does not resolve the discrepancy.
- Corrections and notices. No correction, erratum, expression of concern or retraction was found in Crossref (no update-to, updated-by or relation entries), Crossref Labs (which carries Retraction Watch data), PubMed Central or Europe PMC.
- PubPeer. The PubPeer publication lookup for the DOI returned no comments, and a web search found no PubPeer thread.
Suggested correction
In Results, "Morphology of parent and offspring", second paragraph, give the flask O denominator that matches 87.5% (n = 8 offspring), or, if n = 9 includes the parent, state that and give the matching percentage (88.9%). Replace "P < 0.0001, by a proportion z test" with the value for the counts actually compared, for example "7 of 8 versus 6 of 12; P = 0.085, two-sided two-proportion z test (Fisher's exact test, P = 0.16)", and state that the difference is not statistically significant. In the Discussion, "Phenotypic variation in a genetically similar population", second paragraph, the sentence on light conditions and coiling direction could be reworded to present the 12.5% versus 50% contrast as a small-sample observation from two flasks that was not statistically significant.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- re-derivation for PMC10662364 (Sci Adv 2020, doi:10.1126/sciadv.abb8930)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1.svg/.png
# plus five candidate figure designs (candidates/fig1_A..E) in this script's folder.
#
# Inputs, all taken from the article text:
# Results: "Coiling direction was ascertained in 20 offspring, 13 of which coiled sinistrally (65%)."
# Results: "87.5% were sinistral (n = 9) in the 12-hour light treatment (flask "O"), and 50% (n = 12)
# in the 24-hour dark treatment (flask "D") (P < 0.0001, by a proportion z test)."
# Methods: "eight offspring and the parent were recovered from flask "O," and 17 offspring from flask "D.""
# Discussion: nondominant coiling 50% (dark) vs 12.5% (light).
import os
import numpy as np
from scipy import stats
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- 1. counts
print("== 1. Which counts fit the printed percentages")
for k in range(0, 10):
print(f" {k}/9 = {k/9:.4f}", "<- 87.5%?" if abs(k/9 - 0.875) < 1e-9 else "")
print(f" 87.5% of 9 = {0.875*9:.3f} (not a whole number)")
print(f" 7/8 = {7/8:.4f}; 8/9 = {8/9:.4f}; 7/9 = {7/9:.4f}; 6/12 = {6/12:.4f}")
print(f" offspring total 8 + 12 = {8+12}; sinistral 7 + 6 = {7+6}; 13/20 = {13/20:.4f} (paper: 13 of 20, 65%)")
print(f" with n = 9 the total would be 9 + 12 = {9+12}, not 20")
print(f" nondominant (dextral): 1/8 = {1/8:.4f}, 6/12 = {6/12:.4f} (paper: 12.5% and 50%)")
print(f" flask D: 12 scored for coiling of 17 offspring recovered (Methods)")
xO, nO, xD, nD = 7, 8, 6, 12
# ---------------------------------------------------------------- 2. tests
def z2(x1, n1, x2, n2, pooled=True, cc=False):
p1, p2 = x1 / n1, x2 / n2
if pooled:
p = (x1 + x2) / (n1 + n2)
se = np.sqrt(p * (1 - p) * (1 / n1 + 1 / n2))
else:
se = np.sqrt(p1 * (1 - p1) / n1 + p2 * (1 - p2) / n2)
d = abs(p1 - p2)
if cc:
d = max(0.0, d - 0.5 * (1 / n1 + 1 / n2))
z = d / se
return z, 2 * stats.norm.sf(z), se
print("\n== 2. Two-group tests, flask O vs flask D")
p_pool = (xO + xD) / (nO + nD)
z, p, se = z2(xO, nO, xD, nD)
print(f" pooled p = {p_pool:.3f}; SE = sqrt({p_pool:.2f}*{1-p_pool:.2f}*(1/8+1/12)) = {se:.4f}")
print(f" difference = {7/8 - 6/12:.3f}")
R = {}
for lab, (a, b, c, d) in {"7/8 vs 6/12": (7, 8, 6, 12), "8/9 vs 6/12 (parent counted)": (8, 9, 6, 12)}.items():
for pooled in (True, False):
for cc in (False, True):
z, p, _ = z2(a, b, c, d, pooled, cc)
R[(lab, pooled, cc)] = (z, p)
print(f" {lab:30s} {'pooled ' if pooled else 'unpooled'} {'Yates' if cc else 'no cc':5s}"
f" z = {z:.3f} two-sided P = {p:.4f} one-sided P = {p/2:.4f}")
for a, b, c, d in [(7, 8, 6, 12), (8, 9, 6, 12)]:
tab = [[a, b - a], [c, d - c]]
fe = stats.fisher_exact(tab)
chy = stats.chi2_contingency(tab, correction=True)
chn = stats.chi2_contingency(tab, correction=False)
bo = stats.boschloo_exact(tab, alternative="two-sided")
R[("fisher", a, b)] = fe.pvalue
R[("yates", a, b)] = chy[1]
R[("boschloo", a, b)] = bo.pvalue
print(f" {a}/{b} vs {c}/{d}: Fisher exact P = {fe.pvalue:.4f} (OR = {fe.statistic:.1f}); "
f"chi-square Yates = {chy[0]:.3f}, P = {chy[1]:.4f}; no Yates = {chn[0]:.3f}, P = {chn[1]:.4f}; "
f"Boschloo exact P = {bo.pvalue:.4f}")
print(f" (uncorrected chi-square P equals the pooled z test P: "
f"{chn[1]:.4f} vs {z2(a, b, c, d)[1]:.4f})")
# Wilson intervals and Newcombe hybrid-score interval for the difference
def wilson(x, n, conf=0.95):
zc = stats.norm.ppf(1 - (1 - conf) / 2)
ph = x / n
den = 1 + zc ** 2 / n
cen = (ph + zc ** 2 / (2 * n)) / den
half = zc * np.sqrt(ph * (1 - ph) / n + zc ** 2 / (4 * n * n)) / den
return cen - half, cen + half
lO, uO = wilson(xO, nO)
lD, uD = wilson(xD, nD)
dd = xO / nO - xD / nD
dl = dd - np.sqrt((xO / nO - lO) ** 2 + (uD - xD / nD) ** 2)
du = dd + np.sqrt((uO - xO / nO) ** 2 + (xD / nD - lD) ** 2)
print(f" Wilson 95% CI flask O 7/8: {lO*100:.1f}% to {uO*100:.1f}%")
print(f" Wilson 95% CI flask D 6/12: {lD*100:.1f}% to {uD*100:.1f}%")
print(f" Newcombe 95% CI for the difference 37.5 points: {dl*100:.1f} to {du*100:.1f} points (includes 0)")
# ---------------------------------------------------------------- 3. routes to P < 0.0001
print("\n== 3. Calculations that would give P < 0.0001")
z_pc, p_pc, _ = z2(87.5, 100, 50, 100)
print(f" percentages treated as counts out of 100 (87.5/100 vs 50/100): z = {z_pc:.2f}, P = {p_pc:.1e}")
z1 = (0.5 - 0.875) / np.sqrt(0.875 * 0.125 / 12)
p1s = 2 * stats.norm.sf(abs(z1))
pb = stats.binomtest(6, 12, 0.875).pvalue
print(f" one-sample z, 6/12 against a fixed 0.875: z = {z1:.2f}, P = {p1s:.1e}; exact binomial P = {pb:.4f}")
z1b = (0.875 - 0.5) / np.sqrt(0.5 * 0.5 / 8)
p1b = 2 * stats.norm.sf(z1b)
pbb = stats.binomtest(7, 8, 0.5).pvalue
print(f" one-sample z, 7/8 against a fixed 0.5: z = {z1b:.2f}, P = {p1b:.4f}; exact binomial P = {pbb:.4f}")
print(" (neither one-sample route is a two-group comparison: each treats one flask's estimate as a known constant)")
need = None
curve_n = np.arange(4, 121)
curve_p = []
for n in curve_n:
zz, pp, _ = z2(0.875 * n, n, 0.5 * n, n)
curve_p.append(pp)
if need is None and pp < 1e-4:
need = n
curve_p = np.array(curve_p)
print(f" equal groups needed for pooled-z P < 1e-4 at 87.5% vs 50%: n = {need} per group "
f"(P = {z2(0.875*need, need, 0.5*need, need)[1]:.1e}); at n = {need-1}: "
f"P = {z2(0.875*(need-1), need-1, 0.5*(need-1), need-1)[1]:.1e}")
print(f" at n = 8 and 12 (actual): P = {R[('7/8 vs 6/12', True, False)][1]:.3f}")
print(" (different question) nondominant coiling against the 3% population upper bound, exact binomial, one-sided:")
for x, n, lab in [(1, 8, "flask O"), (6, 12, "flask D"), (7, 20, "all offspring")]:
print(f" {lab}: {x}/{n}, P = {stats.binomtest(x, n, 0.03, alternative='greater').pvalue:.2e}")
# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "axes.edgecolor": INK2, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK2, "text.color": INK, "axes.linewidth": 0.8,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
"lines.linewidth": 1.5, "lines.markersize": 8,
})
W = 7.2
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def clean(ax, xgrid=True, ygrid=False):
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.grid(False)
if xgrid:
ax.xaxis.grid(True, color=GRID, lw=0.8)
if ygrid:
ax.yaxis.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
# ---- Candidate A: P values on one log axis
def fmt_small(p):
m, e = f"{p:.1e}".split("e")
return f"P = {m}e{int(e)}"
def fig_A(path):
rows = [
("Reported in the paper\n(\"proportion z test\")", None, "rep"),
("Pooled two-proportion z test", R[("7/8 vs 6/12", True, False)][1], "ok"),
("Unpooled (Wald) z test", R[("7/8 vs 6/12", False, False)][1], "ok"),
("Pooled z test, parent counted (8/9)", R[("8/9 vs 6/12 (parent counted)", True, False)][1], "ok"),
("Fisher exact test", R[("fisher", 7, 8)], "ok"),
("Chi-square with Yates correction", R[("yates", 7, 8)], "ok"),
("One-sample z test, 6/12 against a\nfixed 0.875 (not a two-group test)", p1s, "bad"),
("Percentages entered as counts\nout of 100 (not a valid test)", p_pc, "bad"),
]
fig, ax = plt.subplots(figsize=(W, 5.3))
clean(ax)
ax.set_xscale("log")
ax.set_xlim(1e-9, 1)
ax.axvline(0.05, color=INK2, lw=0.8, ls="--", zorder=1)
ys = np.arange(len(rows))[::-1]
for y, (lab, p, kind) in zip(ys, rows):
if kind == "rep":
ax.annotate("", xy=(3e-9, y), xytext=(1e-4, y),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=14))
ax.plot([1e-4], [y], marker="|", ms=14, mew=2, color=ORANGE)
ax.text(1e-6, y + 0.18, "P < 0.0001", color=INK, va="bottom", ha="center", fontsize=10.5)
elif kind == "ok":
ax.plot([p], [y], "o", color=BLUE, ms=9, mec=BG, mew=1.5, zorder=3)
ax.text(p * 1.9, y, f"P = {p:.3f}", va="center", ha="left", fontsize=10.5, color=INK)
else:
ax.plot([p], [y], "o", mfc=BG, mec=BLUE, mew=2, ms=9, zorder=3)
ax.text(p * 2.6, y, fmt_small(p), va="center", ha="left", fontsize=10.5, color=INK2)
ax.text(0.05, -0.55, " 0.05", color=INK2, ha="left", va="bottom", fontsize=10)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10.5)
ax.set_ylim(-0.6, len(rows) - 0.3)
ax.set_xticks([1e-8, 1e-6, 1e-4, 1e-2, 1])
ax.set_xticklabels(["10⁻⁸", "10⁻⁶", "10⁻⁴", "0.01", "1"])
ax.set_xlabel("Two-sided P value, log scale; flask O vs flask D")
h = [Line2D([], [], color=ORANGE, marker="|", ms=12, mew=2, lw=2, label="Reported (upper bound)"),
Line2D([], [], color=BLUE, marker="o", ls="none", ms=8, label="Recomputed from paper's counts"),
Line2D([], [], mfc=BG, mec=BLUE, marker="o", ls="none", ms=8, mew=2, label="Recomputed, invalid")]
fig.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=3, frameon=False,
fontsize=10, handletextpad=0.4, columnspacing=1.2)
fig.tight_layout(rect=(0, 0.07, 1, 1))
save(fig, path)
# ---- Candidate B: proportions with 95% CIs and the difference, one percentage axis
def fig_B(path):
fig, ax = plt.subplots(figsize=(W, 4.0))
clean(ax)
rows = [("Flask O, 12-h light\n7 of 8 sinistral", 87.5, (lO * 100, uO * 100)),
("Flask D, 24-h dark\n6 of 12 sinistral", 50.0, (lD * 100, uD * 100)),
("Difference, O minus D\n(percentage points)", 37.5, (dl * 100, du * 100))]
ys = [2, 1, 0]
for y, (lab, v, (lo, hi)) in zip(ys, rows):
ax.plot([lo, hi], [y, y], color=BLUE, lw=2, solid_capstyle="round")
ax.plot([v], [y], "o", color=ORANGE if y > 0 else BLUE, ms=9, mec=BG, mew=1.5, zorder=3)
ax.text(hi + 2, y, f"{v:.1f} [{lo:.1f} to {hi:.1f}]", va="center", fontsize=10.5)
ax.axvline(0, color=INK2, lw=0.8)
ax.text(1, -0.55, "no difference", color=INK2, fontsize=10, va="center")
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows])
ax.set_xlim(-20, 130)
ax.set_xticks([-20, 0, 20, 40, 60, 80, 100])
ax.set_ylim(-0.8, 2.5)
ax.set_xlabel("Percent sinistral (rows 1-2); difference in percentage points (row 3)")
h = [Line2D([], [], color=ORANGE, marker="o", ls="none", ms=8, label="Reported percentage"),
Line2D([], [], color=BLUE, marker="o", lw=2, ms=8, label="Recomputed difference and 95% intervals")]
fig.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False, fontsize=10)
fig.suptitle("Paper: P < 0.0001. The 95%% interval for the difference includes 0\n"
"(pooled z test P = %.3f; Wilson intervals, Newcombe interval for the difference)"
% R[("7/8 vs 6/12", True, False)][1], fontsize=10.5, x=0.02, ha="left", color=INK)
fig.tight_layout(rect=(0, 0.08, 1, 1))
save(fig, path)
# ---- Candidate C: calculation-flow diagram
def fig_C(path):
fig, ax = plt.subplots(figsize=(W, 5.8))
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
def box(x, y, w, h, title, body, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.3,rounding_size=1.5",
fc=BG, ec=edge, lw=1.5))
ax.text(x + 1.5, y + h - 2.5, title, fontsize=10, color=INK2, va="top", fontweight="bold")
ax.text(x + 1.5, y + h - (15 if "\n" in title else 10), body, fontsize=10, color=INK, va="top", linespacing=1.4)
def arrow(x0, y0, x1, y1):
ax.add_patch(FancyArrowPatch((x0, y0), (x1, y1), arrowstyle="-|>", mutation_scale=14, color=INK2, lw=1.5))
pz = R[("7/8 vs 6/12", True, False)]
box(1, 54, 29, 42, "1 PRINTED",
"Flask O: 87.5%\n sinistral, n = 9\nFlask D: 50%\n sinistral, n = 12\nAll: 13 of 20 (65%)", ORANGE)
box(35.5, 54, 29, 42, "2 FITTING COUNTS",
f"87.5% x 9 = {0.875*9:.3f}\n (not whole)\nO: 7 of 8\n (Methods: 8\n offspring + parent)\nD: 6 of 12\n7+6 = 13, 8+12 = 20", BLUE)
box(70, 54, 29, 42, "3 RECOMPUTED",
f"Pooled z = {pz[0]:.2f}\n P = {pz[1]:.3f}\nFisher exact\n P = {R[('fisher', 7, 8)]:.3f}\n"
f"Parent counted\n (8/9), P = {R[('8/9 vs 6/12 (parent counted)', True, False)][1]:.3f}", BLUE)
box(1, 6, 29, 38, "4 PRINTED", "P < 0.0001,\n\"by a proportion\n z test\"", ORANGE)
box(35.5, 6, 63.5, 38, "5 ROUTES TO P < 0.0001 (NEITHER VALID)",
f"a) One-sample z test, 6/12 against a fixed 0.875:\n"
f" z = {z1:.2f}, {fmt_small(p1s)} (exact binomial P = {pb:.4f})\n"
f"b) Percentages entered as counts out of 100:\n"
f" 87.5/100 vs 50/100: z = {z_pc:.2f}, {fmt_small(p_pc)}\n"
f"A valid z test would need about {need} shells per group.", BLUE)
arrow(30.6, 75, 35.0, 75)
arrow(65.1, 75, 69.5, 75)
arrow(30.6, 25, 35.0, 25)
ax.plot([15.5, 15.5], [45, 53], color=INK2, lw=0.8, ls=":")
ax.text(16.5, 49, "same sentence", color=INK2, fontsize=10, va="center")
h = [Line2D([], [], color=ORANGE, lw=2, label="Box holds values printed in the paper"),
Line2D([], [], color=BLUE, lw=2, label="Box holds values recomputed from them")]
ax.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, -0.09), ncol=2, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.01, right=0.99, top=0.99, bottom=0.08)
save(fig, path)
# ---- Candidate D: P value against group size at 87.5% vs 50%
def fig_D(path):
fig, ax = plt.subplots(figsize=(W, 3.9))
clean(ax, xgrid=False, ygrid=True)
ax.set_yscale("log")
ax.plot(curve_n, curve_p, color=BLUE, lw=2, label="Pooled z test, 87.5% vs 50%, n per group")
pa = R[("7/8 vs 6/12", True, False)][1]
neq = 2 / (1 / 8 + 1 / 12)
ax.plot([neq], [pa], "o", color=BLUE, ms=9, mec=BG, mew=1.5)
ax.annotate(f"Actual groups, 8 and 12 (plotted at their\nequivalent equal size, {neq:.1f}): P = {pa:.3f}",
xy=(neq, pa), xytext=(18, 0.2), fontsize=10.5,
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
pn = z2(0.875 * need, need, 0.5 * need, need)[1]
ax.plot([need], [pn], "o", color=BLUE, ms=9, mec=BG, mew=1.5)
ax.annotate(f"P < 0.0001 first reached\nat n = {need} per group", xy=(need, pn), xytext=(need + 10, 3e-3),
fontsize=10.5, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
p100 = z2(87.5, 100, 50, 100)[1]
ax.plot([100], [p100], "o", mfc=BG, mec=BLUE, mew=2, ms=9)
ax.annotate("n = 100: percentages\nread as counts (invalid)", xy=(100, p100), xytext=(72, 4e-7), fontsize=10.5,
color=INK2, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.fill_between([0, 125], 1e-10, 1e-4, color=ORANGE, alpha=0.10, lw=0)
ax.axhline(1e-4, color=ORANGE, lw=1.5)
ax.text(122, 1.4e-4, "Reported: P < 0.0001 (shaded region)", color=INK, ha="right", va="bottom", fontsize=10.5)
ax.axhline(0.05, color=INK2, lw=0.8, ls="--")
ax.text(122, 0.058, "0.05", color=INK2, ha="right", va="bottom", fontsize=10)
ax.set_xlim(0, 125)
ax.set_ylim(1e-9, 1)
ax.set_yticks([1e-8, 1e-6, 1e-4, 1e-2, 1])
ax.set_yticklabels(["10⁻⁸", "10⁻⁶", "10⁻⁴", "0.01", "1"])
ax.set_xlabel("Shells scored per treatment group (equal groups)")
ax.set_ylabel("Two-sided P value (log scale)")
h = [Line2D([], [], color=BLUE, lw=2, marker="o", ms=8, label="Recomputed (paper's percentages, varying n)"),
Line2D([], [], color=ORANGE, lw=1.5, label="Reported threshold")]
ax.legend(handles=h, loc="lower left", frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
# ---- Candidate E: every scored shell as a marker
def fig_E(path):
fig, ax = plt.subplots(figsize=(W, 3.6))
ax.set_xlim(0, 30)
ax.set_ylim(-2.3, 3.4)
ax.axis("off")
x0, dx = 7.6, 0.78
rows = [(2.0, "Flask O, 12-h light", 7, 1, "7 of 8 = 87.5%\n(printed: 87.5%, n = 9)"),
(0.6, "Flask D, 24-h dark", 6, 6, "6 of 12 = 50%\n(printed: 50%, n = 12)")]
for y, lab, s_, d_, txt in rows:
ax.text(0.2, y, lab, ha="left", va="center", fontsize=10.5)
for i in range(s_ + d_):
x = x0 + i * dx
if i < s_:
ax.plot([x], [y], "o", ms=11, color=INK2)
else:
ax.plot([x], [y], "o", ms=11, mfc=BG, mec=INK2, mew=1.8)
ax.text(x0 + 12 * dx + 0.4, y, txt, va="center", fontsize=10.5, color=INK, linespacing=1.3)
ax.plot([x0 + 8 * dx], [2.0], "s", ms=11, mfc=BG, mec=INK2, mew=1.2, ls=(0, (2, 1)))
ax.text(x0 + 8 * dx, 2.5, "9th?", ha="center", fontsize=10, color=INK2)
ax.plot([0.5], [-0.75], "s", ms=10, color=ORANGE)
ax.text(1.2, -0.75, "Printed: P < 0.0001, \"by a proportion z test\"", fontsize=10.5, color=INK, va="center")
ax.plot([0.5], [-1.5], "o", ms=10, color=BLUE)
ax.text(1.2, -1.5, f"Recomputed from these counts: pooled z test P = {R[('7/8 vs 6/12', True, False)][1]:.3f};"
f" Fisher exact P = {R[('fisher', 7, 8)]:.3f}", fontsize=10.5, color=INK, va="center")
h = [Line2D([], [], marker="o", ls="none", ms=10, color=INK2, label="sinistral offspring"),
Line2D([], [], marker="o", ls="none", ms=10, mfc=BG, mec=INK2, mew=1.8, label="dextral offspring"),
Line2D([], [], marker="s", ls="none", ms=10, mfc=BG, mec=INK2, mew=1.2,
label="shell implied by n = 9 (the parent?)")]
ax.legend(handles=h, loc="upper left", bbox_to_anchor=(0.0, 1.04), ncol=3, frameon=False, fontsize=10,
handletextpad=0.3, columnspacing=1.0)
fig.tight_layout()
save(fig, path)
for name, fn in zip("ABCDE", (fig_A, fig_B, fig_C, fig_D, fig_E)):
fn(os.path.join(CAND, f"fig1_{name}"))
fig_A(os.path.join(HERE, "fig1")) # provisional choice; swap after design review
# contact sheet of the five candidates, for the design review
import matplotlib.image as mpimg
titles = {"A": "P values on one log axis (dot plot)", "B": "Proportions and difference with 95% intervals",
"C": "Calculation-flow diagram", "D": "P value vs group size at 87.5% vs 50%",
"E": "Every scored shell as a marker"}
imgs = {k: mpimg.imread(os.path.join(CAND, f"fig1_{k}.png")) for k in "ABCDE"}
hs = [imgs[k].shape[0] / imgs[k].shape[1] for k in "ABCDE"]
fig = plt.figure(figsize=(W, W * (sum(hs) + 5 * 0.06)))
gs = fig.add_gridspec(5, 1, height_ratios=hs, hspace=0.12)
for i, k in enumerate("ABCDE"):
ax = fig.add_subplot(gs[i])
ax.imshow(imgs[k])
ax.axis("off")
ax.set_title(f"{k} {titles[k]}", loc="left", fontsize=11, fontweight="bold", color=INK)
fig.savefig(os.path.join(CAND, "contact_sheet.png"), dpi=120, bbox_inches="tight")
plt.close(fig)
print("\nFigures written: fig1.svg/.png (= candidate A) and candidates/fig1_A..E .svg/.png")
Abstract says the mAb reduces neutrophil apoptosis; the body and Fig. 3 show an increase
Sci Adv (2021) · doi:10.1126/sciadv.abd1453 · PMC11057782
Summary
The abstract says the agonist anti-ChemR23 antibody "reduces neutrophil apoptosis at the site of inflammation". Every other statement in the paper about this effect says the opposite. The Results say the antibody "accelerated neutrophil apoptotic cell death", with more Caspase-3/7 activation and annexin V staining. The safety paragraph's premise is that it "accelerates neutrophil apoptosis". The Discussion reports "increased apoptosis of remaining neutrophils" at the inflammatory site. The Fig. 3 F–H data also show more apoptotic or dead neutrophils with the antibody than with the isotype control, in vitro and in mouse air-pouch exudates. The data and the headline conclusions are not affected. But the abstract, which is the part most readers and text-mining tools see, states one of the paper's two cellular mechanisms backwards.
What the paper reports
Abstract, fourth sentence:
We also identified an anti-ChemR23 agonist antibody that induces receptor signaling, promotes macrophage efferocytosis, and reduces neutrophil apoptosis at the site of inflammation.
Abstract, first sentence. This sentence sets out the effects that proresolving agonists are expected to have:
Resolution of inflammation is elicited by proresolving lipids, which activate GPCRs to induce neutrophil apoptosis, reduce neutrophil tissue recruitment, and promote macrophage efferocytosis.
Results, "Agonist anti-ChemR23 mAb as new proresolving therapeutic modality", paragraph 1:
In addition, agonist anti-ChemR23 mAb internalization accelerated neutrophil apoptotic cell death (Fig. 3G and fig. S10C), as revealed by an increase in Caspase-3/7 activation and annexin V staining (Fig. 3F and fig. S10E).
Mechanistically, this accelerated neutrophil apoptosis was associated with increased ROS production (fig. S10D).
Results, "ChemR23 activation with agonist mAb triggers resolution of acute and chronic intestinal inflammation", paragraph 4 (the safety assessment):
Given that anti-ChemR23 mAb accelerates neutrophil apoptosis (Fig. 3, F and G) and is a human immunoglobulin G1 (hIgG1) isotype with potential cell cytotoxicity (55), we assessed its in vivo effects on the peripheral immune cell population.
Discussion, paragraph 3:
We have identified an anti-human ChemR23 mAb with similar receptor signaling functions as RvE1, including in promoting macrophage resolutive polarization as well as inflammatory neutrophil migration and apoptosis.
However, as previously reported for RvE1 (33, 34), agonist anti-ChemR23 mAb significantly shortened the Ri, as revealed by an accelerated clearing of leukocytes from the inflammatory site, increased apoptosis of remaining neutrophils, and faster recovery from colitis.
Fig. 3 legend, panels F–H:
(F) Percentage of Caspase-3/7 human neutrophils determined by microcopy after 10 hours of culture with isotype hIgG1 or anti-ChemR23 mAb at 10 μg/ml (n = 3). (G) Percentage of dead human neutrophils determined using Live and Dead assay after 24 hours of culture with isotype hIgG1 or anti-ChemR23 mAb at 10 μg/ml (n = 8). (H) Neutrophils absolute number (left) and percentage of dead neutrophils (right) measured in mouse exudates of TNF-induced inflammation air pouch model at indicated time points.
The paper's framing of why apoptosis matters. Introduction, paragraph 2:
In IBD, delayed neutrophil apoptosis and neutrophil-mediated damage are associated with the development of UC and CD (22–26).
Discussion, paragraph 2:
Appropriate neutrophil apoptosis is fundamental for inducing resolution of inflammation
What the paper's own numbers show
Method. The discrepancy is one of direction, not arithmetic, so the check is a census. I extracted from the archived full text every sentence that mentions neutrophil apoptosis or death. I kept the sentences that state the direction of the antibody's effect, and recorded the word that sets the direction in each. I then compared that direction with the direction of the three data panels that measure it (Fig. 3F, G and the right side of H). The paper gives no numeric values for these panels. The values below were read by eye from the published figure image, are approximate, and are used only to show direction and rough size. recompute.py.txt holds all the quotes and confirms that each one appears verbatim in the PMC text (9 of 9). It prints the tally and the approximate readings, and draws the figure.
| Quantity | Location | Reported | Recomputed / observed from the paper's own content |
|---|---|---|---|
| Direction of the mAb's effect on neutrophil apoptosis | Abstract, 4th sentence | "reduces neutrophil apoptosis at the site of inflammation" (decrease) | Opposite to every statement and data panel below |
| Neutrophil apoptotic death in vitro | Results, para 1 of the mAb section (Fig. 3G, fig. S10C) | "accelerated neutrophil apoptotic cell death" | Increase |
| Caspase-3/7 activation and annexin V | Same sentence (Fig. 3F, fig. S10E) | "an increase in Caspase-3/7 activation and annexin V staining" | Increase |
| Mechanism | Results, next sentence (fig. S10D) | "this accelerated neutrophil apoptosis was associated with increased ROS production" | Increase |
| Premise of the safety assessment | Results, colitis section, para 4 | "anti-ChemR23 mAb accelerates neutrophil apoptosis (Fig. 3, F and G)" | Increase |
| Summary of in vitro activity | Discussion, para 3 | "promoting … inflammatory neutrophil migration and apoptosis" | Increase |
| Effect at the inflammatory site in vivo | Discussion, para 3 | "increased apoptosis of remaining neutrophils" | Increase |
| Caspase-3/7+ human neutrophils, 10 h (n = 3) | Fig. 3F | Marked * (P < 0.05) | About 45% with hIgG1 vs about 85% with the mAb (read by eye): higher with the mAb |
| Dead human neutrophils, Live/Dead, 24 h (n = 8) | Fig. 3G | Marked ** (P < 0.01) | About 20% vs about 55% (read by eye): higher with the mAb |
| Dead neutrophils in air-pouch exudate (n = 4 to 5 per group) | Fig. 3H, right | Marked * at 24 h and ** at 48 h | Higher with the mAb at both times |
| Tally of directional text statements | Whole article | — | 1 decrease (abstract only) vs 5 increase (Results 3, Discussion 2); 0 statements in the body say decrease |
| Tally of data panels | Fig. 3F, G, H right | — | 3 of 3 show more apoptosis or death with the mAb; 0 of 3 show less |
Figure
Figure 1. The left panel (orange) shows the abstract's sentence as published, which states a decrease in neutrophil apoptosis. Below it, in gray, is the abstract's own first sentence, which says proresolving GPCR agonists "induce neutrophil apoptosis". The right panel (blue) shows short verbatim excerpts from the Results and the Discussion, plus a description of Fig. 3F–H read from the published image. All of them state or show an increase. The connector marks the conflict: both panels describe the same experiments, but in opposite directions. Of the five body statements counted by recompute.py.txt, four are drawn here. The fifth, the Discussion's "promoting … inflammatory neutrophil migration and apoptosis", is listed in the table above. No values are plotted; the paper's own figure is not reproduced.
Why it matters
The abstract names two cellular mechanisms for the antibody: macrophage efferocytosis and an effect on neutrophil apoptosis. It gives the direction of the second one backwards. The paper argues that accelerated neutrophil apoptosis followed by efferocytosis is how resolution happens ("Appropriate neutrophil apoptosis is fundamental for inducing resolution of inflammation"), and that delayed apoptosis is part of IBD pathology. Read on its own, the abstract therefore describes a mechanism that runs against the paper's own model.
The direction also matters for safety. The Results say the antibody was screened for immune-cell depletion and neutropenia because it "accelerates neutrophil apoptosis" and is a cytotoxicity-capable hIgG1. An abstract that says it reduces apoptosis hides the reason for that check. Readers who see only the abstract (PubMed, indexing services, literature-mining tools that extract direction-of-effect relations) could record ChemR23 agonism as anti-apoptotic for neutrophils. That contradicts both this paper's data and the resolvin E1 literature it cites.
The data, the statistics and the headline conclusions are not affected. These are the title's "reduces tissue neutrophil accumulation", the faster resolution in the air-pouch model, and the reduced lesions, fibrosis and tumors in the colitis models. Only the wording of one abstract sentence is wrong.
Explanations considered
- Conversion or archiving error in PMC. Ruled out. The same wording appears in the publisher-deposited JATS XML held by Europe PMC (vol 7, issue 14, eabd1453, published 2 April 2021), in the PubMed abstract (PMID 33811066, checked on 2026-10-23) and in the institutional repository record (HAL, hal-03462811).
- The qualifier "at the site of inflammation" refers to a different measurement from the in vitro assays. This does not reconcile the two. The only in vivo measurement of neutrophil death at an inflammatory site is the percentage of dead neutrophils in air-pouch exudates (Fig. 3H, right). It is higher with the antibody at 24 h and 48 h, and the Discussion describes it as "increased apoptosis of remaining neutrophils" at the inflammatory site.
- "Reduces neutrophil apoptosis" means fewer apoptotic neutrophils remain because macrophages clear them faster. The paper reports no measurement showing fewer apoptotic neutrophils, and the exudate data show a higher fraction of dead neutrophils. Faster clearance would also be a statement about efferocytosis, which the same sentence already names separately. This reading is not supported.
- A different cell population or disease context. The only patient-derived neutrophil data (fig. S9, ANCA vasculitis) concern transmigration, not apoptosis. No sentence in the article reports reduced apoptosis in any population. The supplementary PDF (figs. S10C–E) could not be downloaded, but the main-text Results describe those panels as showing an increase.
- The intended phrase was "reduces neutrophil accumulation" (as in the title) or "induces neutrophil apoptosis" (as in the abstract's first sentence). Either would agree with the body. This explains how the sentence could be fixed, but it does not change what the published sentence says.
- The data, rather than the abstract, are wrong. Nothing points that way. Three separate assays in vitro (Caspase-3/7, annexin V, Live/Dead) and one in vivo all go the same way, with significance markers, and the text describes each one consistently.
A smaller wording problem sits in the same part of the paper. The Discussion's "promoting … inflammatory neutrophil migration and apoptosis" also reads as promoting migration, whereas the Results report that the antibody "significantly reduced human neutrophil transendothelial migration". This is noted for completeness and is not part of the main finding.
Status
- Version of record. The publisher page (science.org, reached via doi.org) returned HTTP 403 to automated access, so it was not read directly. The wording was confirmed in four independent copies: the publisher-deposited JATS XML in Europe PMC, the PMC text (via
src/show.py), the PubMed record (PMID 33811066) and the HAL repository record. All four have the abstract sentence as quoted. - Supplement. The supplementary PDF (sciadv.abd1453 SM, which holds figs. S10C–E) could not be retrieved: the PMC links returned 404 and the publisher link returned 403. The main Fig. 3 image (abd1453-f3.jpg, from the Europe PMC supplementary-files bundle) was viewed. Panels F, G and H go in the direction the Results describe. The discrepancy is in the abstract's wording, so the supplement would not resolve it either way.
- Corrections and comments. As of 2026-10-23, none found. Crossref lists no update relations for the DOI (
src/known.py). PMC and Europe PMC list no correction, erratum, retraction or expression of concern. The PubMed record has no comments or corrections links. - PubPeer. PubPeer's search page and API returned HTTP 403 (a Cloudflare challenge) to automated queries, so PubPeer could not be checked directly. A web search for the title together with "PubPeer" found no thread, only the article, repository records and a company announcement. Absence of a PubPeer thread is therefore not fully confirmed.
- Sources. PMC11057782; Europe PMC full-text XML and supplementary files for PMC11057782; PubMed 33811066; Crossref metadata for 10.1126/sciadv.abd1453; HAL hal-03462811.
Suggested correction
In the abstract, replace
…promotes macrophage efferocytosis, and reduces neutrophil apoptosis at the site of inflammation.
with wording that matches the Results and Discussion, for example
…promotes macrophage efferocytosis, and induces neutrophil apoptosis and reduces neutrophil accumulation at the site of inflammation.
The data, figures and statistics need no change. Optionally, the Discussion sentence "promoting macrophage resolutive polarization as well as inflammatory neutrophil migration and apoptosis" could be reworded to "promoting macrophage resolutive polarization and neutrophil apoptosis while reducing inflammatory neutrophil migration", which matches the transmigration result.
Recomputation script (python recompute.py.txt)
"""Direction-of-effect check for PMC11057782
(Sci Adv 2021;7(14):eabd1453, doi:10.1126/sciadv.abd1453).
Run: python recompute.py.txt
Needs matplotlib. Prints every number used in writeup.md and writes fig1.svg and
fig1.png next to this file.
The error is in the text, not the arithmetic. The abstract says the agonist
anti-ChemR23 mAb "reduces neutrophil apoptosis at the site of inflammation"; every
statement in the body, and the direction of the Fig. 3 F-H data, says it increases
(accelerates) it. This script lists every sentence in the article that states the
direction of the mAb's effect on neutrophil apoptosis or death, records the word
that sets the direction (classified by hand, and checked to occur in the quote),
and tallies them.
Quotes are copied verbatim from the article as archived in PubMed Central. If this
file sits in the papers repository, the script also re-checks each quote against
the archived text with src/show.py (skipped silently elsewhere).
The Fig. 3 values below were read by eye from the published figure image
(abd1453-f3.jpg); the paper prints no numeric values for these panels. They are
approximate and are used only for direction and rough size, never for a test.
"""
import os
import subprocess
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
PMCID = "PMC11057782"
# ------------------------------------------------------------------ statements
# (location, verbatim quote, word(s) that set the direction, direction)
# direction: +1 = increase / accelerate, -1 = decrease, 0 = context (not about the mAb)
STATEMENTS = [
("Abstract, 4th sentence",
"We also identified an anti-ChemR23 agonist antibody that induces receptor signaling, "
"promotes macrophage efferocytosis, and reduces neutrophil apoptosis at the site of inflammation.",
"reduces neutrophil apoptosis", -1),
("Results, 'Agonist anti-ChemR23 mAb as new proresolving therapeutic modality', para 1",
"In addition, agonist anti-ChemR23 mAb internalization accelerated neutrophil apoptotic cell death "
"(Fig. 3G and fig. S10C), as revealed by an increase in Caspase-3/7 activation and annexin V "
"staining (Fig. 3F and fig. S10E).",
"accelerated neutrophil apoptotic cell death", +1),
("Results, same paragraph, next sentence",
"Mechanistically, this accelerated neutrophil apoptosis was associated with increased ROS "
"production (fig. S10D).",
"accelerated neutrophil apoptosis", +1),
("Results, 'ChemR23 activation with agonist mAb triggers resolution...', para 4",
"Given that anti-ChemR23 mAb accelerates neutrophil apoptosis (Fig. 3, F and G) and is a human "
"immunoglobulin G1 (hIgG1) isotype with potential cell cytotoxicity (55), we assessed its in vivo "
"effects on the peripheral immune cell population.",
"accelerates neutrophil apoptosis", +1),
("Discussion, para 3 (in vitro summary)",
"We have identified an anti-human ChemR23 mAb with similar receptor signaling functions as RvE1, "
"including in promoting macrophage resolutive polarization as well as inflammatory neutrophil "
"migration and apoptosis.",
"promoting", +1),
("Discussion, para 3 (in vivo, inflammatory site)",
"However, as previously reported for RvE1 (33, 34), agonist anti-ChemR23 mAb significantly "
"shortened the Ri, as revealed by an accelerated clearing of leukocytes from the inflammatory "
"site, increased apoptosis of remaining neutrophils, and faster recovery from colitis.",
"increased apoptosis of remaining neutrophils", +1),
]
# Context sentences: what the paper says proresolving agonists are expected to do.
CONTEXT = [
("Abstract, 1st sentence",
"Resolution of inflammation is elicited by proresolving lipids, which activate GPCRs to induce "
"neutrophil apoptosis, reduce neutrophil tissue recruitment, and promote macrophage efferocytosis.",
"induce neutrophil apoptosis"),
("Introduction, para 2",
"In IBD, delayed neutrophil apoptosis and neutrophil-mediated damage are associated with the "
"development of UC and CD (22–26).",
"delayed neutrophil apoptosis"),
("Discussion, para 2",
"Appropriate neutrophil apoptosis is fundamental for inducing resolution of inflammation",
"Appropriate neutrophil apoptosis is fundamental"),
]
# Fig. 3 data panels (legend text is in the paper; values read by eye from the image).
# (panel, what is measured, n, control approx %, mAb approx %, significance marker)
FIG3 = [
("F", "Caspase-3/7+ human neutrophils, 10 h in vitro", "n = 3", 45, 85, "*"),
("G", "Dead human neutrophils (Live/Dead), 24 h in vitro", "n = 8", 20, 55, "**"),
("H right", "Dead neutrophils in mouse air-pouch exudate, 24 h and 48 h", "n = 4 to 5 per group",
None, None, "* at 24 h, ** at 48 h"),
]
def check_against_archive():
"""Re-check every quote against the PMC text when the repository tool is present."""
repo = os.path.abspath(os.path.join(HERE, "..", "..", ".."))
show = os.path.join(repo, "src", "show.py")
py = "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(show) and os.path.exists(py)):
return None
try:
full = subprocess.run([py, show, PMCID, "--full"], cwd=repo, capture_output=True,
text=True, timeout=120).stdout
except Exception:
return None
norm = " ".join(full.split())
out = []
for loc, q, *_ in STATEMENTS + CONTEXT:
out.append((loc, " ".join(q.split()) in norm))
return out
def main():
print(f"{PMCID}: statements on the direction of the mAb's effect on neutrophil apoptosis\n")
for loc, q, key, d in STATEMENTS:
assert key in q, (loc, key)
lab = {+1: "INCREASE", -1: "DECREASE"}[d]
print(f" {lab:8s} | {loc}\n governing words: '{key}'")
n_dec = sum(1 for s in STATEMENTS if s[3] == -1)
n_inc = sum(1 for s in STATEMENTS if s[3] == +1)
dec_locs = [s[0] for s in STATEMENTS if s[3] == -1]
print(f"\nText statements saying DECREASE: {n_dec} ({'; '.join(dec_locs)})")
print(f"Text statements saying INCREASE: {n_inc} (all in Results or Discussion)")
print(f"Body statements saying DECREASE: {sum(1 for s in STATEMENTS[1:] if s[3] == -1)}")
print("\nContext statements (direction proresolving agonists are expected to act):")
for loc, q, key in CONTEXT:
assert key in q
print(f" {loc}: '{key}'")
print("\nFig. 3 data panels (values read by eye from the image; approximate):")
n_up = 0
for p, what, n, c, m, sig in FIG3:
if c is not None:
print(f" Fig. 3{p}: {what} ({n}): hIgG1 ~{c}% vs anti-ChemR23 ~{m}% "
f"(difference ~{m - c} points, {m / c:.1f}x), marked {sig}")
else:
print(f" Fig. 3{p}: {what} ({n}): higher with anti-ChemR23 than hIgG1, marked {sig}")
n_up += 1
print(f"Fig. 3 panels in which apoptosis/death is higher with the mAb: {n_up} of {len(FIG3)}")
print(f"Fig. 3 panels in which it is lower: 0 of {len(FIG3)}")
chk = check_against_archive()
if chk is not None:
ok = sum(1 for _, v in chk if v)
print(f"\nQuotes found verbatim in the archived PMC text: {ok} of {len(chk)}")
for loc, v in chk:
if not v:
print(f" NOT FOUND: {loc}")
make_fig1()
# ------------------------------------------------------------------ figure
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#d9d8d4"
def wrap(s, width):
return "\n".join(textwrap.wrap(s, width))
def make_fig1():
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": TXT,
"svg.fonttype": "none", "figure.facecolor": BG, "savefig.facecolor": BG})
W, H = 7.2, 6.0
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, W)
ax.set_ylim(0, H)
ax.axis("off")
LH = 0.158 # line height in inches at 10 pt
def box(x0, y0, x1, y1, col):
ax.add_patch(FancyBboxPatch((x0, y0), x1 - x0, y1 - y0, boxstyle="round,pad=0,rounding_size=0.06",
fc="white", ec=col, lw=1.5))
def put(x, y, s, n_lines=None, **kw):
ax.text(x, y, s, va="top", ha="left", linespacing=1.25, **kw)
k = s.count("\n") + 1 if n_lines is None else n_lines
return y - k * LH * 1.0
ax.text(0.12, H - 0.14, "Direction of the mAb's effect on neutrophil apoptosis, as stated in the paper",
fontsize=11.5, fontweight="bold", va="top")
# ---- left: abstract
lx0, lx1, top, bot = 0.12, 2.55, H - 0.52, 0.86
box(lx0, bot, lx1, top, ORANGE)
y = top - 0.12
y = put(lx0 + 0.12, y, "ABSTRACT", fontsize=10.5, fontweight="bold", color=ORANGE)
y = put(lx0 + 0.12, y, "4th sentence (as published)", fontsize=10, color=SEC) - 0.06
s = STATEMENTS[0][1]
y = put(lx0 + 0.12, y, wrap("“" + s + "”", 28), fontsize=10)
y -= 0.14
y = put(lx0 + 0.12, y, "Direction stated:", fontsize=10, color=SEC)
y = put(lx0 + 0.12, y, "↓ DECREASE", fontsize=12, fontweight="bold", color=ORANGE) - 0.30
ax.plot([lx0 + 0.12, lx1 - 0.12], [y + 0.12, y + 0.12], color=GRID, lw=1)
y = put(lx0 + 0.12, y, "Same abstract, 1st sentence:", fontsize=10, color=SEC)
y = put(lx0 + 0.12, y, wrap("“…proresolving lipids, which activate GPCRs to induce "
"neutrophil apoptosis…”", 28), fontsize=10, color=SEC)
# ---- right: body
rx0, rx1 = 3.45, 7.08
box(rx0, bot, rx1, top, BLUE)
y = top - 0.12
y = put(rx0 + 0.12, y, "RESULTS, DISCUSSION, FIG. 3", fontsize=10.5, fontweight="bold", color=BLUE) - 0.06
items = [
("Results (Fig. 3F, G; in vitro)",
"“accelerated neutrophil apoptotic cell death … an increase in Caspase-3/7 activation "
"and annexin V staining”"),
("Results (mechanism)",
"“this accelerated neutrophil apoptosis was associated with increased ROS production”"),
("Results (safety, colitis section)",
"“anti-ChemR23 mAb accelerates neutrophil apoptosis (Fig. 3, F and G)”"),
("Discussion (in vivo, inflammatory site)",
"“an accelerated clearing of leukocytes from the inflammatory site, increased apoptosis "
"of remaining neutrophils”"),
("Fig. 3F, G, H (data, read from the image)",
"Caspase-3/7+ and dead neutrophils higher with the mAb than with hIgG1 in vitro, and dead "
"neutrophils higher in air-pouch exudate at 24 h and 48 h"),
]
for loc, txt in items:
y = put(rx0 + 0.12, y, loc, fontsize=10, color=SEC)
y = put(rx0 + 0.12, y, wrap(txt, 43), fontsize=10) - 0.08
y -= 0.04
y = put(rx0 + 0.12, y, "Direction stated:", fontsize=10, color=SEC)
n_inc = sum(1 for s in STATEMENTS if s[3] == +1)
y = put(rx0 + 0.12, y, "↑ INCREASE", fontsize=12, fontweight="bold", color=BLUE)
put(rx0 + 0.12, y - 0.04, f"all {n_inc} body statements; all 3 data panels",
fontsize=10, fontweight="bold", color=BLUE)
# ---- connector
ymid = top - 1.55
arr = FancyArrowPatch((lx1 + 0.04, ymid), (rx0 - 0.04, ymid), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=TXT)
ax.add_patch(arr)
ax.text((lx1 + rx0) / 2, ymid + 0.12, "opposite\ndirection", ha="center", va="bottom",
fontsize=10.5, fontweight="bold")
ax.text((lx1 + rx0) / 2, ymid - 0.12, "same\ndata\n(Fig. 3F–H)", ha="center", va="top",
fontsize=10, color=SEC, linespacing=1.2)
ax.text(0.12, 0.70, "Orange: the abstract's wording as published.", fontsize=10, color=SEC, va="top")
ax.text(0.12, 0.50, "Blue: what the Results, the Discussion and the paper's own Fig. 3 data show.",
fontsize=10, color=SEC, va="top")
ax.text(0.12, 0.30, "Quotes verbatim from the PMC archive copy (PMC11057782); "
"… marks omitted words.", fontsize=10, color=SEC, va="top")
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), **kw)
plt.close(fig)
print("\nwrote fig1.svg and fig1.png")
if __name__ == "__main__":
main()
Abstract and Introduction state the reverse of the paper's own Schwann cell depletion result
Sci Adv (2023) · doi:10.1126/sciadv.add6995 · PMC9891701
Summary
The Abstract states that "Depletion of pancreatic-specific TASc promoted the tumorigenesis of PDAC tumors", and the Introduction repeats it. (TASc are tumor-associated nonmyelinating Schwann cells.) The paper's only pancreas-specific depletion is intrapancreatic AAV8-GFAP-DTA in KPC mice. For that experiment the Results report improved survival (Fig. 2C; log-rank; n = 8, 12 and 13 mice), fewer PanIN 2/3 lesions and inhibited tumor growth (Fig. 2D–G). The Fig. 2 title reads "Depletion of TASc attenuates pancreatic cancer tumorigenesis". The main text contains 17 statements (14 sentences, one Results subsection heading and two figure titles) that state a direction for the effect of depleting, inhibiting or targeting TASc (or TASc-expressed PVT1) on tumors. Fifteen say depletion is antitumor. Only the Abstract and Introduction sentences say it is protumor. The data, statistics and the paper's central thesis are unaffected. Still, the Abstract, the paper's own summary of its findings, reverses the direction of its headline in vivo result, and no correction has been issued.
What the paper reports
The two sentences that state the depletion result as protumor:
Depletion of pancreatic-specific TASc promoted the tumorigenesis of PDAC tumors.
Abstract, sentence 4
Depletion of TASc promoted the tumorigenesis of PDAC.
Introduction, final (third) paragraph
The same experiment as described in the Results and in the Fig. 2 legend:
TASc promotes PDAC tumorigenesis
Results, subsection heading
KPC mice given AAV8-GFAP-DTA exhibited improved survival time compared to control group (Fig. 2C). Upon AAV8-GFAP-DTA–mediated Schwann cell depletion, KPC pancreases exhibited increased percentage of PanIN 1, reduced PanIN 2/3, and inhibited tumor growth (Fig. 2, D to G).
Results, "TASc promotes PDAC tumorigenesis", paragraph 1
Depletion of TASc attenuates pancreatic cancer tumorigenesis.
Fig. 2 title
Log-rank test; n = 8, 12, and 13 mice, respectively.
Fig. 2 legend, panel C (WT, AAV8-GFAP-Blank and AAV8-GFAP-DTA KPC mice)
Error bars, SDs; n = 12 and 13 animals, respectively; Student’s t test.
Fig. 2 legend, panel G (tumor volumes at 22 weeks)
The paper's own summary of the add-back experiment in the same subsection points the same way:
Together, these results suggested that the presence of TASc in PDAC facilitates tumorigenesis and resistance to immunotherapy.
Results, "TASc promotes PDAC tumorigenesis", paragraph 2, last sentence
So do the pharmacological depletion with fluorocitrate (FC), the Discussion, and a later sentence of the same Abstract:
Furthermore, depletion of TASc in PDAC may profoundly diminish tumor growth and improve the infiltration and cytotoxicity of CD8+ T cells.
Results, "Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy", last sentence
Depletion of TASc-expressed Pvt1 or TASc using FC restored the immunoresistant metabolic microenvironment and improved the antitumor effects of immune checkpoint inhibitors.
Discussion, paragraph 2
Furthermore, depletion of TASc using a small-molecule inhibitor effectively sensitized PDAC to immunotherapy, signifying the important roles of TASc in PDAC immune resistance.
Abstract, last sentence
Each quotation was checked character for character against the article as archived in PubMed Central (src/show.py PMC9891701 --grep), and the key sentences also against the publisher's typeset PDF (see Status).
What the paper's own numbers show
Method. No quantity can be recomputed, because the error is in the stated direction of an effect. The tumor volumes and survival curves appear only as plotted data in Fig. 2, and the text and figure title state their direction explicitly. recompute.py.txt therefore holds a hand-curated list of every statement (sentence, Results subsection heading or figure title) outside Materials and Methods that states a direction for the effect of depleting, inhibiting or targeting TASc, or TASc-expressed PVT1, on tumor growth, tumorigenesis, survival or response to immunotherapy. To build it, every sentence of the Abstract, Introduction, Results and Discussion, every Results subsection heading and every figure title was read. A statement was included when its subject is the removal or inhibition of TASc (by AAV8-GFAP-DTA, by fluorocitrate, by "a small-molecule" agent, or by targeting in general) or of TASc-expressed PVT1, and it states a direction for one of those outcomes. Statements left out were: those about the presence, rather than the removal, of TASc or PVT1 (such as the heading "TASc promotes PDAC tumorigenesis"); deletions of PVT1 nucleotides, RAF1 or TDO2; immune-cell or TASc-abundance readouts only; the cytotoxicity, safety or specificity of FC; and sentences calling TASc a therapeutic target without stating a tumor outcome. The script checks each listed statement verbatim against the archived text and confirms its section, labels each with the direction it states, and tallies them. Run with the publisher's PDF and supplement as text (instructions in the script header), it also checks those. It prints every number below.
| Quantity | Location | Reported | Recomputed from the paper's own text |
|---|---|---|---|
| Effect of depleting pancreatic TASc on tumorigenesis | Abstract, sentence 4 | "promoted the tumorigenesis" | Opposite: depletion attenuates tumorigenesis (Results; Fig. 2 title; fig. S4 title) |
| Same | Introduction, final paragraph | "promoted the tumorigenesis" | Opposite, as above |
| Survival after AAV8-GFAP-DTA depletion | Results; Fig. 2C (log-rank; n = 8 WT, 12 Blank, 13 DTA) | "improved survival time compared to control group" | Depletion is antitumor |
| PanIN grade and tumor growth after depletion | Results; Fig. 2D–G (Fig. 2F n = 8, 12, 13; Fig. 2G n = 12 vs 13) | "reduced PanIN 2/3, and inhibited tumor growth" | Depletion is antitumor |
| Fig. 2 title | Fig. 2 legend | "Depletion of TASc attenuates pancreatic cancer tumorigenesis." | Depletion is antitumor |
| Directional statements on depleting, inhibiting or targeting TASc or TASc-expressed PVT1 | Main text (Abstract, Introduction, Results incl. subsection headings, Discussion, figure titles) | — | 17 statements (14 sentences, 1 heading, 2 figure titles): 2 protumor (Abstract, Introduction), 15 antitumor |
| Statements about the AAV8-GFAP-DTA experiment only | Main text | — | 5 statements: 2 protumor (Abstract, Introduction), 3 antitumor (Results ×2, Fig. 2 title) |
| Same, including the supplementary figure titles | Main text + figs. S4, S13, S16 | — | 20 statements, 2 protumor, 18 antitumor (19 and 17 if the fig. S16 schematic is left out); DTA experiment 6 statements, 2 protumor, 4 antitumor |
| Fluorocitrate or small-molecule statements | Main text | — | 7, all antitumor |
| PVT1-loss statements | Main text | — | 3, all antitumor |
| Statements naming both PVT1 loss and FC | Main text (Discussion) | — | 2, both antitumor |
| Sections that contain a protumor statement | Main text | — | Abstract and Introduction only |
| Supplement sentences of the form "deplet… promot… tumorigen…" | Supplementary Materials PDF | — | 0 |
The Abstract itself contains three depletion statements in opposite directions. Sentence 4 says depletion is protumor. The PVT1 sentence and the last sentence say depletion is antitumor.
A minor side observation that does not affect the finding: the Fig. 2G legend names three groups (WT, AAV8-GFAP-Blank, AAV8-GFAP-DTA) but gives n for only two ("n = 12 and 13 animals").
Figure
Figure 1. The left panel (orange) shows the two sentences, in the Abstract and the Introduction, that state the depletion result as protumor. The right panel (blue) shows what the paper reports for the same experiment, intrapancreatic AAV8-GFAP-DTA in KPC mice: the Results phrases for Fig. 2C and Fig. 2D–G, with sample sizes from the Fig. 2 legend, and the Fig. 2 title. The connector marks the conflict: one experiment, two opposite directions. All text is quoted from the paper. None of the paper's figures or data are reproduced.
Figure 2. Each marker is one of the 17 main-text statements that state a direction for the effect of depleting, inhibiting or targeting TASc, or TASc-expressed PVT1, on tumors (inclusion rule under Method). Columns run in reading order from the Abstract to the Discussion, followed by the two figure titles. The Results FC column includes the subsection heading "Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy". The upper row holds statements that depletion promotes tumorigenesis. The lower row holds statements that it inhibits tumor growth or improves outcome. Marker shape gives the intervention: circles for AAV8-GFAP-DTA, squares for PVT1 loss, diamonds for fluorocitrate, triangles for sentences that name both PVT1 and FC. Markers in the same cell are stacked. Only the two orange markers, in the Abstract and the Introduction, sit in the upper row. Both concern the DTA experiment, whose three other statements all sit in the lower row. The supplementary titles of figs. S4, S13 and S16 (not drawn) would all sit in the lower row. Directions and locations come from recompute.py.txt.
Why it matters
- A headline result is stated in reverse in the Abstract. The Abstract sentence reports the paper's main in vivo test of function: whether removing TASc from the pancreas changes tumor development. Read literally, it says TASc restrain PDAC. That contradicts the Results heading ("TASc promotes PDAC tumorigenesis"), the Fig. 2 title, and the paper's therapeutic argument for depleting TASc with fluorocitrate plus anti–PD-1.
- Readers of the Abstract alone get the wrong direction. Many readers, indexing services, text-mining pipelines and automated summaries rely on the Abstract. Any of them could record "depletion of Schwann cells promotes pancreatic cancer". The Introduction repeats the sentence, so both places that summarize the findings carry the error. Crossref lists 68 citing works.
- The science itself is not affected. No reported value, test or P value changes, and the Results, figure legends and Discussion consistently give the antitumor direction. The same Abstract goes on to say that depleting TASc with a small molecule sensitized PDAC to immunotherapy, so a careful reader would notice the conflict. The error is in the wording of two summary sentences, not in the data or analysis.
Explanations considered
- The sentences refer to a different experiment or setting. This does not hold. "Pancreatic-specific" depletion matches only the intrapancreatic AAV8-GFAP-DTA delivery in KPC mice (Fig. 2, fig. S4). Every other statement about that experiment, in the Results, the Fig. 2 title and the fig. S4 title, reports an antitumor effect. The paper reports no subgroup, time point or model in which depletion accelerated tumors.
- "Tumorigenesis" is used in an unusual sense. This does not hold. The paper uses the word in its ordinary sense throughout, for example "TASc promotes PDAC tumorigenesis" and "Combining FC with the anti–PD-1 antibody significantly suppressed PDAC tumorigenesis".
- The increase in PanIN 1 lesions counts as "promoted". This does not hold. After depletion, the percentage of low-grade PanIN 1 rose while the share of high-grade PanIN 2/3 fell. The paper reports this shift in the same sentence as "inhibited tumor growth" and summarizes the panel under the title "Depletion of TASc attenuates pancreatic cancer tumorigenesis".
- The intended subject was TASc, not their depletion. "TASc promoted the tumorigenesis of PDAC", as in the Results heading, would be consistent with the data. So would "Depletion of TASc attenuated tumorigenesis", as in the Fig. 2 title. This explains how the wording could have arisen. It does not change the fact that the printed sentences state the opposite of the data.
- Conversion or archiving artifact. Ruled out. The publisher's typeset PDF in the PMC Open Access dataset (version 1, the only version; marked as the publisher's version, not an author manuscript) contains both sentences word for word. So does the Europe PMC full text. The conflict is in words ("promoted" against "attenuates"), not in symbols, signs or table cells that a conversion could alter.
- The supplement resolves it. It does not. The fig. S4 title ("Depletion of TASc inhibits PDAC tumorigenesis."), the fig. S13 title ("FC treatment sensitizes PDAC to immunotherapy.") and the title of the fig. S16 schematic ("Targeting tumor-associated non-myelinating Schwann cells improves anti-tumor immunity of immunotherapy-refractory pancreatic cancer.") all support the Results. The supplement contains no sentence saying depletion promotes tumorigenesis.
None of these explanations reconciles the Abstract and Introduction sentences with the paper's results.
Status
As of 2026-10-23:
- Version of record. The publisher's typeset PDF (16 pages, created 2023-01-26) held in the PMC Open Access dataset (
s3://pmc-oa-opendata/PMC9891701.1/;is_manuscript: false,is_retracted: false) was converted to text and searched. It contains the Abstract and Introduction sentences, the Results sentences and the Fig. 2 title as quoted. In the PDF's text layer the survival sentence is interrupted by a figure caption at a column break, and its text continues unchanged after the caption. The live PMC page and the Europe PMC full-text XML carry the same wording. The publisher's own page (doi.org redirects to science.org) returned HTTP 403 to automated access, so it could not be checked directly. - Supplements. The Supplementary Materials PDF (
sciadv.add6995_sm.pdf: methods and figs. S1 to S16) was converted to text and searched for depletion, DTA and FC statements. The fig. S4, fig. S13 and fig. S16 titles agree with the Results direction. The tables archive (Tables S1 to S7) was not opened, because the finding concerns the direction of a stated result, not a tabulated value. - Corrections. Crossref lists no update for the DOI: the record has no update-to field, and a filter for works that update it returned 0 results. PMC and Europe PMC list no correction, erratum, expression of concern or retraction (
src/known.py). The Europe PMC record (PMID 36724291) has no comment/correction list. It shows a revision date of 2024-05-08 without a stated reason. The PMC copy still carries the "promoted" wording. - PubPeer and the web. The PubPeer API returned HTTP 403, so PubPeer could not be queried directly. Earlier web searches by DOI and by title, combined with correction, erratum or PubPeer, found no notice or discussion of this point. A PubPeer comment therefore cannot be ruled out completely.
- Sources: PMC9891701 (via
src/show.py);src/known.py 10.1126/sciadv.add6995; PMC Open Access dataset files PMC9891701.1.pdf, PMC9891701.1.json and sciadv.add6995_sm.pdf; https://api.crossref.org/works/10.1126/sciadv.add6995 and https://api.crossref.org/works?filter=updates:10.1126/sciadv.add6995; Europe PMC REST record for DOI 10.1126/sciadv.add6995; https://doi.org/10.1126/sciadv.add6995 (HTTP 403); https://pubpeer.com/api/search/?q=10.1126/sciadv.add6995 (HTTP 403).
Suggested correction
Reverse the direction in the two sentences so that they match the Results, the Fig. 2 title and the fig. S4 title:
- Abstract: "Depletion of pancreatic-specific TASc attenuated the tumorigenesis of PDAC tumors." Alternatively: "Pancreatic-specific TASc promoted the tumorigenesis of PDAC tumors, and their depletion attenuated it."
- Introduction, final paragraph: "Depletion of TASc attenuated the tumorigenesis of PDAC."
No change to the data, figures, statistics or conclusions is needed. The Fig. 2G legend could also give the WT group size, since it currently gives n for two of its three groups.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC9891701 (Sci Adv 2023, doi:10.1126/sciadv.add6995)
# Abstract and Introduction vs Results / Fig. 2: what does depleting tumor-associated
# nonmyelinating Schwann cells (TASc) do to PDAC tumorigenesis?
#
# Run: python recompute.py.txt
# Prints every number used in writeup.md and regenerates fig1.{svg,png} and fig2.{svg,png}
# in the folder that holds this file. Needs matplotlib.
#
# The finding is about the direction of an effect stated in words; there is no numeric
# value to recompute. The script therefore does three things:
# 1. holds a hand-curated list of every statement (sentence, Results subsection heading or
# figure title; Materials and Methods excluded) that states a direction for the effect of
# depleting, inhibiting or targeting TASc, or TASc-expressed PVT1, on tumor growth,
# tumorigenesis, survival or response to immunotherapy, each copied verbatim from the
# article as archived in PubMed Central and labelled with the direction it states.
# The list was built by reading every sentence of the Abstract, Introduction, Results and
# Discussion, every Results subsection heading and every figure title (see INCLUSION below);
# 2. if run inside the /home/ubuntu/papers repo, re-checks each sentence against the
# archived full text (src/show.py) and confirms it sits in the stated section;
# skipped silently elsewhere;
# 3. tallies the directions and draws the two figures.
# Group sizes (n) are transcribed from the Fig. 2 legend.
#
# Optional checks of the publisher's typeset PDF and the Supplementary Materials PDF, both
# held in the PMC Open Access dataset (version 1 is the only version):
# aws s3 cp --no-sign-request s3://pmc-oa-opendata/PMC9891701.1/PMC9891701.1.pdf ./
# aws s3 cp --no-sign-request s3://pmc-oa-opendata/PMC9891701.1/sciadv.add6995_sm.pdf ./
# pdftotext PMC9891701.1.pdf vor.txt ; pdftotext sciadv.add6995_sm.pdf sm.txt
# VOR_TXT=./vor.txt SUPP_TXT=./sm.txt python recompute.py.txt
# Without those variables the two checks are reported as "not run", not as passed.
import os, re, subprocess
from collections import Counter, OrderedDict
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
# ------------------------------------------------------------------ statements
# INCLUSION: subject = depleting / inhibiting / targeting TASc (by AAV8-GFAP-DTA, fluorocitrate
# or "a small-molecule" agent) or TASc-expressed PVT1; predicate = a direction for tumor growth,
# tumorigenesis, survival or response to immunotherapy. Left out: statements about the
# presence (not removal) of TASc or PVT1 (see CONVERSE), deletion of PVT1 nucleotides, RAF1 or
# TDO2, immune-cell or TASc-abundance readouts only, cytotoxicity, safety or specificity of FC,
# and statements that TASc are a therapeutic target without a stated tumor outcome.
# key: (section as archived, direction, experiment, verbatim statement)
# direction "PRO" = depletion promotes tumorigenesis (protumor)
# direction "ANTI" = depletion inhibits tumor growth / improves outcome (antitumor)
# experiment: "DTA" = AAV8-GFAP-DTA depletion of Schwann cells in KPC mice (Fig. 2)
# "PVT1" = loss of PVT1 in TASc (Fig. 6), "FC" = fluorocitrate (Fig. 7)
STATEMENTS = OrderedDict([
("abs_pro", ("ABSTRACT", "PRO", "DTA",
"Depletion of pancreatic-specific TASc promoted the tumorigenesis of PDAC tumors.")),
("abs_pvt1", ("ABSTRACT", "ANTI", "PVT1",
"Depletion of TASc-expressed PVT1 suppressed PDAC tumor growth.")),
("abs_fc", ("ABSTRACT", "ANTI", "FC",
"Furthermore, depletion of TASc using a small-molecule inhibitor effectively sensitized "
"PDAC to immunotherapy, signifying the important roles of TASc in PDAC immune resistance.")),
("intro_pro", ("INTRODUCTION", "PRO", "DTA",
"Depletion of TASc promoted the tumorigenesis of PDAC.")),
("intro_pvt1", ("INTRODUCTION", "ANTI", "PVT1",
"Depletion of TASc-expressed PVT1 suppressed PDAC tumor growth.")),
("intro_fc", ("INTRODUCTION", "ANTI", "FC",
"Furthermore, using a small-molecule compound to inhibit TASc in vivo notably improved "
"the efficacy of immune checkpoint inhibitors.")),
("res_dta_surv", ("RESULTS > TASc promotes PDAC tumorigenesis", "ANTI", "DTA",
"KPC mice given AAV8-GFAP-DTA exhibited improved survival time compared to control group "
"(Fig. 2C).")),
("res_dta_growth", ("RESULTS > TASc promotes PDAC tumorigenesis", "ANTI", "DTA",
"Upon AAV8-GFAP-DTA–mediated Schwann cell depletion, KPC pancreases exhibited increased "
"percentage of PanIN 1, reduced PanIN 2/3, and inhibited tumor growth (Fig. 2, D to G).")),
("res_pvt1", ("RESULTS > TASc-expressed PVT1 facilitates PDAC tumorigenesis", "ANTI", "PVT1",
"on the contrary, depletion of Pvt1 in Schwann cells inhibited tumor growth (Fig. 6, A and B).")),
("res_fc_heading", ("RESULTS > Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy",
"ANTI", "FC",
"Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy")), # subsection heading
("res_fc_retraction", ("RESULTS > Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy",
"ANTI", "FC",
"Treatment with FC and control antibodies led to retraction of PDAC (Fig. 7, A to C).")),
("res_fc_combo", ("RESULTS > Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy",
"ANTI", "FC",
"Combining FC with the anti–PD-1 antibody significantly suppressed PDAC tumorigenesis in "
"comparison with anti–PD-1 treatment alone (Fig. 7, A to C, and fig. S13A).")),
("res_fc_summary", ("RESULTS > Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy",
"ANTI", "FC",
"Furthermore, depletion of TASc in PDAC may profoundly diminish tumor growth and improve the "
"infiltration and cytotoxicity of CD8+ T cells.")),
("disc_restored", ("DISCUSSION", "ANTI", "PVT1+FC",
"Depletion of TASc-expressed Pvt1 or TASc using FC restored the immunoresistant metabolic "
"microenvironment and improved the antitumor effects of immune checkpoint inhibitors.")),
("disc_therapy", ("DISCUSSION", "ANTI", "PVT1+FC",
"Considering the critical role of TASc in immunosuppressive microenvironments, depleting TASc "
"or TASc-expressed PVT1 in PDAC may be a promising therapeutic modality.")),
("fig2_title", ("Fig. 2 legend", "ANTI", "DTA",
"Depletion of TASc attenuates pancreatic cancer tumorigenesis.")),
("fig7_title", ("Fig. 7 legend", "ANTI", "FC",
"TASc inhibition sensitizes PDAC to immunotherapy.")),
])
KIND = {"res_fc_heading": "heading", "fig2_title": "figure title", "fig7_title": "figure title"}
# Statements about the presence (not depletion) of TASc; these imply that depletion should
# reduce tumorigenesis. Shown for context only, not part of the tally.
CONVERSE = OrderedDict([
("res_heading", ("RESULTS > TASc promotes PDAC tumorigenesis", "Results subsection heading",
"TASc promotes PDAC tumorigenesis")),
("res_s5", ("RESULTS > TASc promotes PDAC tumorigenesis", "Results, same subsection, paragraph 2",
"Together, these results suggested that the presence of TASc in PDAC facilitates "
"tumorigenesis and resistance to immunotherapy.")),
])
# Supplementary Materials (sciadv.add6995_sm.pdf), figure titles that state a direction for
# depletion/inhibition/targeting of TASc. Counted separately from the main-text tally.
# fig. S16 is a schematic (graphic illustration) of the whole model; its title is counted too.
SUPP_TITLES = OrderedDict([
("figS4_title", ("fig. S4 title", "ANTI", "DTA", "Depletion of TASc inhibits PDAC tumorigenesis.")),
("figS13_title", ("fig. S13 title", "ANTI", "FC", "FC treatment sensitizes PDAC to immunotherapy.")),
("figS16_title", ("fig. S16 title (schematic)", "ANTI", "TASc targeting",
"Targeting tumor-associated non-myelinating Schwann cells improves anti-tumor immunity of "
"immunotherapy-refractory pancreatic cancer.")),
])
# Fig. 2 legend, panels C, F and G (transcribed)
N_FIG2C = {"WT": 8, "AAV8-GFAP-Blank": 12, "AAV8-GFAP-DTA": 13} # "n = 8, 12, and 13 mice"
N_FIG2G = {"AAV8-GFAP-Blank": 12, "AAV8-GFAP-DTA": 13} # "n = 12 and 13 animals"
FIG2_LEGEND_SNIPPETS = [
"Log-rank test; n = 8, 12, and 13 mice, respectively.",
"Error bars, SDs; n = 12 and 13 animals, respectively; Student’s t test.",
]
# ------------------------------------------------------------------ verification
def load_archived_text():
repo = "/home/ubuntu/papers"
py = "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
return None
try:
out = subprocess.run([py, "src/show.py", "PMC9891701", "--full"], cwd=repo,
capture_output=True, text=True, timeout=120).stdout
except Exception:
return None
return out or None
def sections(txt):
"""Map section label -> text, using the --full layout of src/show.py."""
secs, cur = {}, None
for line in txt.splitlines():
if line.startswith("ABSTRACT:"):
cur = "ABSTRACT"; line = line[len("ABSTRACT:"):]
elif line.startswith("## "):
cur = line[3:].strip(); secs[cur] = secs.get(cur, "") + cur + "\n"; continue # heading kept
else:
m = re.match(r"\[Fig\. (\d+)\.\]", line)
if m:
cur = f"Fig. {m.group(1)} legend"
if cur:
secs[cur] = secs.get(cur, "") + line + "\n"
return secs
def verify(txt):
secs = sections(txt)
ok = True
print("Verbatim check against the archived full text (src/show.py PMC9891701 --full):")
for key, (sec, *_rest, s) in list(STATEMENTS.items()) + [
(k, (v[0], v[2])) for k, v in CONVERSE.items()]:
total = txt.count(s)
in_sec = secs.get(sec, "").count(s)
flag = "ok" if in_sec >= 1 else "MISSING"
ok &= in_sec >= 1
print(f" [{flag}] {key:18s} found {total}x in full text, {in_sec}x in '{sec[:45]}'")
leg2 = secs.get("Fig. 2 legend", "")
for snip in FIG2_LEGEND_SNIPPETS:
c = leg2.count(snip)
ok &= c == 1
print(f" [{'ok' if c == 1 else 'MISSING'}] Fig. 2 legend: {snip}")
print(" all checks passed" if ok else " SOME CHECKS FAILED")
return ok
def norm_pdf_text(t):
t = re.sub(r"-\n", "-", t) # keep hyphens split across lines
return re.sub(r"\s+", " ", t)
def verify_pdfs():
vor, supp = os.environ.get("VOR_TXT"), os.environ.get("SUPP_TXT")
print("\nPublisher PDF (version of record) check:", end=" ")
if vor and os.path.exists(vor):
t = norm_pdf_text(open(vor, encoding="utf-8", errors="replace").read())
print()
for key in ("abs_pro", "intro_pro", "res_dta_surv", "res_dta_growth", "fig2_title"):
q = STATEMENTS[key][3]
n_full = t.count(q)
if n_full:
print(f" {key:18s} found {n_full}x")
else: # the PDF text layer can interleave a figure caption inside a sentence
tail = q[q.index(" ", 12) + 1:]
print(f" {key:18s} found 0x as one string; tail '{tail[:40]}...' found "
f"{t.count(tail)}x (sentence interrupted by a figure caption in the PDF text layer)")
else:
print("not run (set VOR_TXT)")
print("Supplementary Materials check:", end=" ")
if supp and os.path.exists(supp):
t = norm_pdf_text(open(supp, encoding="utf-8", errors="replace").read())
print()
for key, (loc, d, exp, q) in SUPP_TITLES.items():
print(f" {key:18s} found {t.count(q)}x ({loc}: {q})")
n_pro = len(re.findall(r"(?i)deplet[^.]{0,80}promot[^.]{0,40}tumorigen", t))
print(f" sentences of the form 'deplet... promot... tumorigen...' in supplement: {n_pro}")
else:
print("not run (set SUPP_TXT)")
# ------------------------------------------------------------------ tally
def tally():
print("\nDirection stated for the effect of depleting/inhibiting/targeting TASc or TASc-expressed PVT1:")
for key, (sec, d, exp, s) in STATEMENTS.items():
lab = "depletion PROMOTES tumor" if d == "PRO" else "depletion INHIBITS tumor"
print(f" {key:18s} {sec[:40]:40s} {KIND.get(key, 'sentence'):12s} {exp:8s} {lab}")
c = Counter(d for _, d, _, _ in STATEMENTS.values())
n = len(STATEMENTS)
kinds = Counter(KIND.get(k, "sentence") for k in STATEMENTS)
print(f"\nTotal directional statements (main text): {n} ({dict(kinds)})")
print(f" depletion promotes tumorigenesis: {c['PRO']} "
f"({', '.join(k for k, v in STATEMENTS.items() if v[1] == 'PRO')})")
print(f" depletion inhibits tumor / improves outcome: {c['ANTI']}")
dta = [(k, v) for k, v in STATEMENTS.items() if v[2] == "DTA"]
cd = Counter(v[1] for _, v in dta)
print(f"Statements about the AAV8-GFAP-DTA depletion experiment itself: {len(dta)} "
f"({cd['PRO']} promotes [{', '.join(k for k, v in dta if v[1] == 'PRO')}], "
f"{cd['ANTI']} inhibits [{', '.join(k for k, v in dta if v[1] == 'ANTI')}])")
allst = list(STATEMENTS.values()) + list(SUPP_TITLES.values())
ca = Counter(v[1] for v in allst)
dta_all = [v for v in allst if v[2] == "DTA"]
cda = Counter(v[1] for v in dta_all)
print(f"Adding the supplementary figure titles ({', '.join(v[0] for v in SUPP_TITLES.values())}): "
f"{len(allst)} statements, {ca['PRO']} promote, {ca['ANTI']} inhibit; "
f"DTA experiment {len(dta_all)} statements, {cda['PRO']} promote, {cda['ANTI']} inhibit")
nos16 = [v for k, v in SUPP_TITLES.items() if k != "figS16_title"]
cn = Counter(v[1] for v in list(STATEMENTS.values()) + nos16)
print(f" (without the fig. S16 schematic title: {n + len(nos16)} statements, "
f"{cn['PRO']} promote, {cn['ANTI']} inhibit)")
fc = [v for v in STATEMENTS.values() if v[2] == "FC"]
print(f"Fluorocitrate/small-molecule statements (main text): {len(fc)}, all "
f"{'antitumor' if all(v[1] == 'ANTI' for v in fc) else 'MIXED'}")
pvt = [v for v in STATEMENTS.values() if v[2] == "PVT1"]
print(f"PVT1-loss statements (main text): {len(pvt)}, all "
f"{'antitumor' if all(v[1] == 'ANTI' for v in pvt) else 'MIXED'}")
both = [v for v in STATEMENTS.values() if v[2] == "PVT1+FC"]
print(f"Statements naming both PVT1 loss and FC (main text): {len(both)}, all "
f"{'antitumor' if all(v[1] == 'ANTI' for v in both) else 'MIXED'}")
pro_secs = sorted({v[0] for v in STATEMENTS.values() if v[1] == "PRO"})
print(f"Sections containing a 'promotes' statement: {pro_secs}")
print(f"\nFig. 2C group sizes (log-rank): {N_FIG2C}; total mice = {sum(N_FIG2C.values())}")
print(f"Fig. 2G group sizes (t test, tumor volume at 22 weeks): {N_FIG2G}; "
f"total = {sum(N_FIG2G.values())}")
return c, dta
# ------------------------------------------------------------------ figures
def style():
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
"axes.edgecolor": SEC, "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"svg.fonttype": "path", "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG,
})
def panel(ax, x, y, w, h, color, title, sub, body):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.008,rounding_size=0.015",
lw=2, ec=color, fc="white"))
ax.text(x + 0.02, y + h - 0.035, title, fontsize=12, fontweight="bold", color=color, va="top")
ax.text(x + 0.02, y + h - 0.095, sub, fontsize=10, color=SEC, va="top", style="italic")
yy = y + h - 0.165
for item in body:
txt, kw = item if isinstance(item, tuple) else (item, {})
ax.text(x + 0.02, yy, txt, fontsize=kw.get("size", 10.5), color=kw.get("color", TXT),
fontweight=kw.get("weight", "normal"), va="top", linespacing=1.3)
yy -= kw.get("step", 0.12)
def fig1(n2c, n2g):
fig = plt.figure(figsize=(7.2, 5.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
ax.text(0.03, 0.98, "Effect of TASc depletion on PDAC: two directions for one experiment",
fontsize=11.5, fontweight="bold", va="top")
panel(ax, 0.03, 0.255, 0.41, 0.66, ORANGE,
"Abstract and Introduction",
"summary of the depletion result",
[("Abstract, sentence 4:", {"color": SEC, "size": 10, "step": 0.045}),
("“Depletion of pancreatic-specific\nTASc promoted the tumorigenesis\nof PDAC tumors.”",
{"step": 0.17}),
("Introduction, final paragraph:", {"color": SEC, "size": 10, "step": 0.045}),
("“Depletion of TASc promoted the\ntumorigenesis of PDAC.”", {"step": 0.12}),
("Stated: depletion is protumor", {"color": ORANGE, "weight": "bold",
"size": 10.5})])
panel(ax, 0.56, 0.255, 0.41, 0.66, BLUE,
"Results and Fig. 2",
"AAV8-GFAP-DTA depletion in KPC mice",
[("Fig. 2C (log-rank; n = %d, %d, %d):" % tuple(n2c.values()),
{"color": SEC, "size": 10, "step": 0.045}),
("“improved survival time compared\nto control group”", {"step": 0.095}),
("Fig. 2D–G (volumes: n = %d vs %d):" % tuple(n2g.values()),
{"color": SEC, "size": 10, "step": 0.045}),
("“reduced PanIN 2/3, and inhibited\ntumor growth”", {"step": 0.095}),
("Fig. 2 title:", {"color": SEC, "size": 10, "step": 0.045}),
("“Depletion of TASc attenuates\npancreatic cancer tumorigenesis.”", {"step": 0.11}),
("Shown: depletion is antitumor", {"color": BLUE, "weight": "bold",
"size": 10.5})])
arr = FancyArrowPatch((0.448, 0.585), (0.552, 0.585), arrowstyle="<|-|>", mutation_scale=16,
lw=2, color=TXT)
ax.add_patch(arr)
ax.text(0.5, 0.615, "opposite\ndirection", ha="center", va="bottom", fontsize=10,
fontweight="bold", linespacing=1.15)
ax.text(0.03, 0.205,
"The paper's only pancreas-specific depletion of TASc is this experiment: AAV8-GFAP-DTA\n"
"delivered into the pancreas of KPC mice, compared with AAV8-GFAP-Blank controls.",
fontsize=10, color=TXT, va="top", linespacing=1.35)
ax.text(0.03, 0.10,
"Elsewhere the paper agrees with the right panel: Results heading “TASc promotes PDAC\n"
"tumorigenesis”; PVT1 loss in TASc and fluorocitrate depletion both reduce tumor growth.",
fontsize=10, color=SEC, va="top", linespacing=1.35)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), **kw)
plt.close(fig)
def fig2():
order = ["ABSTRACT", "INTRODUCTION",
"RESULTS > TASc promotes PDAC tumorigenesis",
"RESULTS > TASc-expressed PVT1 facilitates PDAC tumorigenesis",
"RESULTS > Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy",
"DISCUSSION", "Fig. 2 legend", "Fig. 7 legend"]
labels = ["Abstract", "Intro-\nduction", "Results\nDTA\n(Fig. 2)", "Results\nPVT1\n(Fig. 6)",
"Results\nFC\n(Fig. 7)", "Discus-\nsion", "Fig. 2\ntitle", "Fig. 7\ntitle"]
expmark = {"DTA": "o", "PVT1": "s", "FC": "D", "PVT1+FC": "^"}
fig, ax = plt.subplots(figsize=(7.2, 3.8))
fig.subplots_adjust(left=0.17, right=0.98, top=0.83, bottom=0.27)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
counts = Counter()
for key, (sec, d, exp, _) in STATEMENTS.items():
xi = order.index(sec)
yi = 1 if d == "PRO" else 0
off = counts[(xi, yi)]; counts[(xi, yi)] += 1
col = ORANGE if d == "PRO" else BLUE
ax.plot(xi, yi + 0.13 * off - 0.0, marker=expmark[exp], ms=10, color=col,
mec="white", mew=1, ls="none", zorder=3)
ax.set_xticks(range(len(order))); ax.set_xticklabels(labels, fontsize=10)
ax.set_yticks([0, 1])
ax.set_yticklabels(["depletion\ninhibits\ntumor", "depletion\npromotes\ntumor"], fontsize=10)
ax.tick_params(axis="y", length=0)
ax.set_ylim(-0.35, 1.45); ax.set_xlim(-0.6, len(order) - 0.4)
c = Counter(v[1] for v in STATEMENTS.values())
ax.text(-0.5, 1.3, f"{c['PRO']} statements (orange): the Abstract and Introduction sentences",
color=ORANGE, fontsize=10, fontweight="bold", va="center")
ax.text(1.6, 0.62, f"{c['ANTI']} statements (blue), in every section",
color=BLUE, fontsize=10, fontweight="bold", va="center")
# marker legend for experiment
from matplotlib.lines import Line2D
h = [Line2D([], [], marker=m, color=SEC, ls="none", ms=8, label=l) for l, m in
[("DTA depletion", "o"), ("PVT1 loss", "s"), ("fluorocitrate", "D"), ("PVT1 or FC", "^")]]
ax.legend(handles=h, ncol=4, loc="lower left", bbox_to_anchor=(-0.2, 1.02), frameon=False,
fontsize=10, handletextpad=0.3, columnspacing=1.0)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), **kw)
plt.close(fig)
if __name__ == "__main__":
txt = load_archived_text()
if txt:
verify(txt)
else:
print("(archived text not available here; verbatim check skipped)")
verify_pdfs()
tally()
style()
fig1(N_FIG2C, N_FIG2G)
fig2()
print("\nWrote fig1.svg, fig1.png, fig2.svg, fig2.png to", HERE)
Binomial p for 12 of 17 complexes is 0.072, not the reported 0.0245
eLife (2024) · doi:10.7554/eLife.93333 · PMC10942632
Summary
The paper reports that 12 of 17 protein complexes had an ROC-AUC above 0.5 and calls this "a significant excess (binomial test, p=0.0245)" against a null expectation of one half. The exact one-sided binomial p-value for 12 or more successes out of 17 at p = 0.5 is 9402/131072 = 0.0717, and the two-sided value is 0.143, so neither is significant at 0.05. The reported 0.0245 equals P(X ≥ 13), the upper tail starting one success above the observed count. The second binomial test in the same paragraph shows the same pattern: 0.0012 is reported, while P(X ≥ 4 | n = 17, p = 0.05) is 0.0088. That result remains significant. The correction removes one of the paper's three complex-level arguments for a detectable physical-interaction signal in ERC, and the corrected value is now consistent with the paper's own non-significant global permutation test (p = 0.08). The headline conclusion in the title and abstract does not depend on this test.
What the paper reports
The first binomial test, its stated null, and the second binomial test:
"Of the 17 complexes, 12 had an ROC-AUC > 0.5. Since the random expectation would be that half would exceed 0.5, this is a significant excess (binomial test, p=0.0245). Moreover, five complexes had AUCs > 0.7 (Figure 4). These results indicate that physically interacting domains tend to have a higher ERC than nonphysically interacting domains in those complexes. Likewise, four complexes had their physically interacting domains ranked significantly higher than noninteracting domains at an alpha of 5% (one-tailed Mann–Whitney U test), which is also a significant excess of complexes overall (binomial test, p=0.0012)." (Results, "ERC does not distinguish physical interactions from nonphysical interactions within a given protein complex", para 4, which opens "Of the 17 complexes, 12 had an ROC-AUC > 0.5")
The count is restated in the figure legend:
"Of the 17 complexes, 12 have an ROC-AUC > 0.5. The SEC23/24 complex (bright green) has the highest ROC-AUC at 1, and the NUP84 complex (marigold) has the lowest AUC of 0.247." (Figure 4 legend)
The global test on the same AUCs:
"We compared the true average ROC-AUC from the 17 complexes (0.596) to the permuted distribution of average ROC-AUCs, which resulted in a permutation p-value of 0.08." (Results, same subsection, para 5, which opens "We then applied a third test for a general pattern")
The Discussion relies on the 12-of-17 result:
"Looking across all complexes in our study, a proportion higher than random chance showed elevated ERC between interacting domains; this result was reflected in 12 of the 17 complexes with ROC-AUCs over 0.5 and the 4 complexes with individually significant rankings." (Discussion, para 2, which opens "We found that compensatory coevolution due to physical interaction contributes to ERC")
The abstract states the overall conclusion:
"We found no generalizable relationship between physical interaction and high ERC, although a few complexes ranked physical interactions higher than nonphysical interactions." (Abstract)
What the paper's own numbers show
Method. Under the null stated in the text, the number X of complexes with AUC > 0.5 is Binomial(n = 17, p = 0.5). The exact one-sided p-value for the observed 12 is P(X ≥ 12), which includes the observed outcome. It was computed by summing binomial coefficients exactly and checked with scipy.stats.binomtest. The same was done for the second test, X ~ Binomial(17, 0.05) with 4 observed. As an input check, the 17 per-complex AUCs printed in the version-of-record Figure 4 image were transcribed into recompute.py.txt. (The PMC text legend gives only the highest and lowest values.) They give 12 of 17 above 0.5, no values exactly at 0.5, and a mean of 0.596, matching the text. All numbers below are printed by recompute.py.txt when it is run. Paragraph numbers in this write-up are one-based, counted from the first paragraph of the named subsection, and each is given with its opening words.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Complexes with ROC-AUC > 0.5 | Results para 4 ("Of the 17 complexes, 12 had…"); Fig. 4 legend; Discussion para 2 | 12 of 17 | 12 of 17 (from the Fig. 4 AUCs) |
| Mean ROC-AUC | Results para 5 ("We then applied a third test…") | 0.596 | 0.596 (from the Fig. 4 AUCs) |
| Binomial p, 12 of 17 at p = 0.5, one-sided P(X ≥ 12) | Results para 4 | 0.0245 | 9402/131072 = 0.0717 |
| Same test, exact two-sided | — | — | 0.1435 |
| P(X ≥ 13) = P(X > 12) | — | — | 3214/131072 = 0.02452 (equals the reported value) |
| Smallest count out of 17 with one-sided p < 0.05 | — | — | 13 |
| Binomial p, 4 of 17 significant at α = 0.05, one-sided P(X ≥ 4) | Results para 4 | 0.0012 | 0.0088 (still < 0.05) |
| P(X ≥ 5) = P(X > 4) at p = 0.05 | — | — | 0.00116 (equals the reported value) |
| Global permutation test on mean AUC | Results para 5 | 0.08 | not recomputable (needs per-complex ranked lists); consistent with the corrected 0.072 |
| Complexes with AUC > 0.7 (side note) | Results para 4 | "five" | 6 in the Fig. 4 AUCs (1.00, 0.821, 0.736, 0.723, 0.714, 0.713) |
| Complexes with physical > nonphysical mean ERC (side note) | Results para 2 ("We split the proteins…") vs Table 1 | 12 of 17 | 14 of 17 in Table 1 |
Both reported binomial p-values equal the upper tail that starts one success above the observed count. That tail excludes the observed outcome, so it is not the p-value of the observation. For 12 of 17, the excluded term alone is P(X = 12) = 6188/131072 = 0.0472.
Figure
Figure 1. Reported p-values (orange) and exact binomial p-values recomputed from the paper's counts (blue), on one log-scaled axis; the dashed line marks α = 0.05. Top row: 12 of 17 complexes with ROC-AUC > 0.5 under the paper's stated null of one half. The reported 0.0245 falls below 0.05. The exact one-sided p (0.0717, filled blue) and two-sided p (0.143, open blue diamond) are both above it. Bottom row: 4 of 17 complexes individually significant at α = 0.05. The reported 0.0012 and the recomputed 0.0088 are both below 0.05, so only the size of that p-value changes. All values come from the paper's text or from recompute.py.txt.
Figure 2. The null distribution of the number of complexes with AUC > 0.5 out of 17, Binomial(17, 0.5). The blue bars (12 to 17) form the upper tail that includes the observed count, and they sum to the correct one-sided p = 0.0717 (blue bracket). The reported 0.0245 equals the sum over 13 to 17 only (orange bracket), leaving out the observed outcome, whose probability is 0.0472. This figure shows why a one-step shift in the tail boundary roughly triples the p-value here.
Why it matters
Results paragraphs 4 and 5 of the subsection use three complex-level tests to argue that physical coevolution contributes a detectable signal to ERC: the 12-of-17 sign test, the count of individually significant complexes, and the global permutation test. As reported, the first two are significant and the third is not, and the Discussion says the evidence "was tempered by a global permutation test". After correction, the 12-of-17 test (p = 0.072) is not significant either, so two of the three complex-wide tests fail to reach 0.05. The corrected value is in line with the permutation p of 0.08 rather than in tension with it.
Two statements depend on the 12-of-17 result as written. The Results paragraph calls it "a significant excess", and the Discussion's "a proportion higher than random chance ... this result was reflected in 12 of the 17 complexes with ROC-AUCs over 0.5" cites it as evidence. At α = 0.05 the 12-of-17 result is not a proportion higher than chance. The Discussion's broader claim of a "non-negligible signal from physical interaction" is weakened but keeps some support. The 4-of-17 count remains significant (corrected p = 0.0088), and five complexes are reported as significant in the separate proportional-rank analysis.
The headline conclusion is that ERC does not reliably predict physical interactions ("no generalizable relationship"). The error does not undermine it. If anything the corrected value makes the evidence for a general physical-interaction signal weaker, which fits that conclusion. The error is not in the title or the abstract.
Explanations considered
The count is 13, not 12. Ruled out. The text, the Figure 4 legend and the Discussion all say 12 of 17. The 17 AUCs printed in the version-of-record Figure 4 image have exactly 12 above 0.5, and their mean (0.596) matches the value in the text, so the transcription is consistent.
A tie at AUC = 0.5 was dropped (n = 16). Does not account for it. No Figure 4 AUC equals 0.5 (the closest is SMC5/6 at 0.504). Even with n = 16, P(X ≥ 12 | 16, 0.5) = 0.0384, not 0.0245.
A two-sided test. Does not account for it. The two-sided exact p is 0.143, larger than the one-sided value.
A different null proportion. Ruled out by the text, which states the expectation as "half would exceed 0.5".
A convention that defines the test as strictly greater than the observed count. No such definition appears in the Methods, which describe only the per-complex Mann–Whitney tests and the permutations. P(X > k) is not a valid p-value for an observation of k, because it excludes the observed outcome. The value 0.0245 is what an upper-tail call returns when the observed count itself is passed as the cutoff, for example R's pbinom(12, 17, 0.5, lower.tail = FALSE), which returns P(X > 12). The correct call passes 11. R was not run here; recompute.py.txt evaluates the equivalent upper-tail function, scipy.stats.binom.sf, with both cutoffs (0.02452 for 12, 0.07173 for 11). The second test reproduces the same shift exactly (0.00116 vs 0.0088), which points to one consistent computational step rather than two separate typos.
A conversion or archiving artifact. Ruled out. The eLife version-of-record web page, the version-of-record PDF, the PMC page, the Europe PMC full-text XML and the reviewed preprint v2 all give the same numbers. The only markup difference is in the second test: in the Europe PMC XML its equals sign is wrapped in italic markup, while the eLife web page displays it as "p = 0.0012". The value is the same.
The analysis used different data. This cannot be settled from the paper and does not change the finding as published. The data deposit (see Status) contains a per-complex spreadsheet. An earlier audit stage recomputed AUCs from it and found values that differ slightly from Figure 4 (for example NUP84 0.260 vs 0.247), with 14 of 17 above 0.5. Table 1 also shows physical > nonphysical mean ERC in 14 of 17 complexes, where the text says 12. A count of 14 of 17 would give P(X ≥ 14) = 0.0064. But every statement in the paper, and the published Figure 4, reports 12 of 17, and the p-value is evaluated here for that reported count. If a different count was intended, the text, the figure and the Discussion would all need to change.
Status
As of 2026-10-23:
- Version of record. The eLife version of record (https://elifesciences.org/articles/93333, published 2024-02-28) was fetched again on 2026-10-23. It still reads "binomial test, p=0.0245" and, for the second test, "binomial test, p = 0.0012" (typeset with an italic equals sign), and it shows no correction notice. An earlier audit stage found the same wording in the version-of-record PDF (elife-93333-v1.pdf), the Europe PMC full-text XML and the reviewed preprint v2 PDF ("Binomial test, P = 0.0245"). That stage also read the 17 AUCs from the Figure 4 image in the version-of-record PDF.
- Supplements and data. The version of record has no standalone supplementary or source-data files, only two figure supplements to Figure 2. The data availability statement points to the repository github.com/nclark-lab/erc and cites an archived copy on Software Heritage; Methods ("Calculating ERC") names the physical_interaction_paper subfolder of that repository as the location of the pipeline. An earlier audit stage opened all_complexes.xlsx (17 sheets of domain-pair ftERC scores with physical-interaction labels) and run_domain.R. The R script draws the ROC curves but contains no binomial test code, so the computation of the reported p-values could not be inspected. As noted above, AUCs recomputed from the spreadsheet do not exactly reproduce Figure 4.
- Corrections.
src/known.py(run 2026-10-23) lists no Crossref updates and no PMC or Europe PMC correction or retraction notices. - PubPeer. PubPeer's API returned HTTP 404 and its search page returned a bot-check page on 2026-10-23, so PubPeer could not be queried directly. A web search on 2026-10-23 for the title together with "pubpeer" returned only the article's own pages (PMC, bioRxiv, eLife) and no PubPeer thread.
Sources: PMC10942632 (PubMed Central archive, read with src/show.py); https://elifesciences.org/articles/93333; https://cdn.elifesciences.org/articles/93333/elife-93333-v1.pdf; reviewed preprint and reviews at https://elifesciences.org/reviewed-preprints/93333/reviews; preprint doi:10.1101/2023.10.18.562970; data deposit https://github.com/nclark-lab/erc (subfolder physical_interaction_paper, named in Methods, "Calculating ERC").
Suggested correction
In Results, "ERC does not distinguish physical interactions from nonphysical interactions within a given protein complex", para 4 (opening "Of the 17 complexes, 12 had an ROC-AUC > 0.5"), replace "this is a significant excess (binomial test, p=0.0245)" with "this excess is not statistically significant (one-sided exact binomial test, p=0.072)". Replace "(binomial test, p=0.0012)" with "(one-sided exact binomial test, p=0.0088)". In the Discussion, para 2 (opening "We found that compensatory coevolution due to physical interaction contributes to ERC"), revise "a proportion higher than random chance showed elevated ERC between interacting domains; this result was reflected in 12 of the 17 complexes with ROC-AUCs over 0.5 and the 4 complexes with individually significant rankings" so that only the 4 individually significant complexes are cited as an excess over chance, and note that the 12-of-17 count is not significant. The authors could also:
- state the test sidedness and the exact tail used in the Methods;
- reconcile "five complexes had AUCs > 0.7" (Results para 4) with the six values above 0.7 in Figure 4;
- reconcile "12 had higher average ERC for the physically interacting" (Results para 2, opening "We split the proteins") with Table 1, where 14 of 17 complexes show this;
- confirm which data underlie Figure 4, since the deposited spreadsheet gives slightly different AUCs.
Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC10942632 and regenerate fig1/fig2.
Run: python recompute.py.txt (writes fig1.svg/png and fig2.svg/png next to this file)
Inputs come from the article (eLife 2024, doi:10.7554/eLife.93333):
* Results, "ERC does not distinguish physical interactions ...", para 4
(opens "Of the 17 complexes, 12 had an ROC-AUC > 0.5"):
12 of 17 complexes with ROC-AUC > 0.5, "binomial test, p=0.0245";
4 of 17 complexes significant at alpha = 5%, "binomial test, p=0.0012".
* Figure 4 (version-of-record image): the 17 per-complex ROC-AUCs listed below.
* Same subsection, para 5 (opens "We then applied a third test"):
mean ROC-AUC 0.596, global permutation p = 0.08.
* Same subsection, para 2 (opens "We split the proteins"): "12 had higher
average ERC" for physically interacting domains.
* Table 1: mean ERC of physically vs nonphysically interacting domains.
Paragraph numbers are one-based, counted from the first paragraph of the subsection.
"""
import os
from math import comb
from fractions import Fraction
from scipy.stats import binom, binomtest
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
N = 17
# ------------------------------------------------------------ reported values
REP_P1 = 0.0245 # 12 of 17 AUC > 0.5, null proportion 0.5
REP_P2 = 0.0012 # 4 of 17 significant, null proportion 0.05
REP_PERM = 0.08 # global permutation test on mean AUC (not recomputable here)
# Figure 4 (VOR image legend), per-complex ROC-AUC
AUC = {"SEC23/24": 1.00, "ESCRT-I": 0.821, "Exportin": 0.736, "CUL8": 0.723,
"GET4_GET5": 0.714, "SWI_SNF": 0.713, "MCM": 0.666, "TREX": 0.600,
"ORC": 0.565, "PAN1": 0.543, "COMA": 0.540, "SMC5/6": 0.504,
"EXOCYST": 0.480, "ATG17/31/29": 0.478, "F1 ATPase": 0.467,
"EIF3": 0.341, "NUP84": 0.247}
# Table 1: (physically interacting domain ERC, nonphysically interacting domain ERC)
TABLE1 = {"EIF3": (8.852, 11.125), "MCM": (10.283, 8.000), "NUP84": (1.541, 4.411),
"ORC": (5.250, 4.035), "PAN1": (4.963, 4.716), "SMC5-6": (5.153, 4.110),
"TREX": (4.157, 4.129), "EXOCYST": (6.221, 5.974), "COMA": (3.051, 2.869),
"SWI/SNF": (4.121, 1.783), "CUL8": (2.611, 1.576), "GET4-GET5": (4.569, 3.302),
"ATG17-31-29": (3.444, 1.584), "ESCRT-I": (4.041, 2.402),
"F1 ATPase": (4.883, 5.654), "SEC23/24": (10.530, 7.754),
"Exportin": (8.871, 6.326)}
# ------------------------------------------------------------ Figure 4 checks
k_auc = sum(v > 0.5 for v in AUC.values())
print("== Inputs from Figure 4 ==")
print(f"complexes listed: {len(AUC)}")
print(f"AUC > 0.5: {k_auc} of {len(AUC)} (text: 12 of 17)")
print(f"AUC = 0.5 exactly (ties): {sum(v == 0.5 for v in AUC.values())}")
print(f"mean AUC: {sum(AUC.values())/len(AUC):.3f} (text: 0.596)")
print(f"AUC > 0.7: {sum(v > 0.7 for v in AUC.values())} (text: 'five complexes had AUCs > 0.7')")
print()
# ------------------------------------------------------------ test 1: 12 of 17, p0 = 0.5
k1 = 12
terms = [comb(N, k) for k in range(k1, N + 1)]
p_ge12 = Fraction(sum(terms), 2 ** N)
p_ge13 = Fraction(sum(comb(N, k) for k in range(k1 + 1, N + 1)), 2 ** N)
two_sided = binomtest(k1, N, 0.5).pvalue
print("== Test 1: 12 of 17 complexes with AUC > 0.5, null proportion 0.5 ==")
print(f"C(17,k), k=12..17: {terms}; sum = {sum(terms)}; 2^17 = {2**N}")
print(f"one-sided P(X>=12) = {sum(terms)}/{2**N} = {float(p_ge12):.4f}"
f" (scipy binomtest greater: {binomtest(k1, N, 0.5, alternative='greater').pvalue:.4f})")
print(f"two-sided exact p = {two_sided:.4f}")
print(f"P(X>=13) = P(X>12) = {sum(terms[1:])}/{2**N} = {float(p_ge13):.5f}"
f" (reported: {REP_P1})")
print(f"P(X=12), the observed outcome left out of P(X>12) = {comb(N, 12)}/{2**N} = "
f"{binom.pmf(12, N, 0.5):.4f}")
# R's pbinom(q, n, p, lower.tail = FALSE) returns P(X > q), i.e. scipy's binom.sf(q, n, p)
print(f"upper tail with the observed count as cutoff, P(X>12) "
f"[R: pbinom(12, 17, 0.5, lower.tail = FALSE)] = {binom.sf(12, N, 0.5):.5f}")
print(f"upper tail with cutoff 11, P(X>11) = P(X>=12) "
f"[R: pbinom(11, 17, 0.5, lower.tail = FALSE)] = {binom.sf(11, N, 0.5):.5f}")
print(f"tie excluded, P(X>=12 | n=16) = {binom.sf(11, 16, 0.5):.4f}")
print(f"smallest k out of 17 with P(X>=k) < 0.05: "
f"{min(k for k in range(N + 1) if binom.sf(k - 1, N, 0.5) < 0.05)}")
print()
# ------------------------------------------------------------ test 2: 4 of 17, p0 = 0.05
k2 = 4
p2_ge4 = binom.sf(k2 - 1, N, 0.05)
p2_ge5 = binom.sf(k2, N, 0.05)
print("== Test 2: 4 of 17 complexes significant at alpha 0.05, null proportion 0.05 ==")
print(f"one-sided P(X>=4) = {p2_ge4:.4f}")
print(f"P(X>=5) = P(X>4) = {p2_ge5:.5f} (reported: {REP_P2})")
print()
# ------------------------------------------------------------ side check: Table 1
k_t1 = sum(a > b for a, b in TABLE1.values())
print("== Side check: Table 1 ==")
print(f"physical ERC > nonphysical ERC: {k_t1} of {len(TABLE1)} (text: 12 of 17)")
print(f"for reference, a count of 14 of 17 would give P(X>=14 | 17, 0.5) = "
f"{sum(comb(N, k) for k in range(14, N + 1))}/{2**N} = {binom.sf(13, N, 0.5):.4f}")
print()
print(f"Global permutation test (reported, not recomputable): p = {REP_PERM}")
# ------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK2, "axes.edgecolor": INK2,
"svg.fonttype": "none", "font.family": "DejaVu Sans"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# Figure 1: reported vs recomputed p-values, one log axis, alpha = 0.05 reference
fig, ax = plt.subplots(figsize=(7.2, 3.5), facecolor=BG)
ax.set_facecolor(BG)
rows = [("12 of 17 complexes with ROC-AUC > 0.5\n(null proportion 0.5)",
REP_P1, float(p_ge12), two_sided),
("4 of 17 complexes significant at α = 0.05\n(null proportion 0.05)",
REP_P2, p2_ge4, None)]
ys = [1, 0]
for y, (lab, rep, rec, rec2) in zip(ys, rows):
hi = max(rep, rec, rec2 or 0)
ax.plot([min(rep, rec), hi], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(rep, y, "o", ms=10, color=ORANGE, zorder=3)
ax.plot(rec, y, "o", ms=10, color=BLUE, zorder=3)
ax.annotate(f"reported {rep}", (rep, y), xytext=(0, 11), textcoords="offset points",
ha="center", color=INK, fontsize=10)
ax.annotate(f"P(X≥{12 if y else 4}) = {rec:.4f}", (rec, y), xytext=(0, -18),
textcoords="offset points", ha="center", color=INK, fontsize=10)
if rec2 is not None:
ax.plot(rec2, y, "D", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=3)
ax.annotate(f"two-sided\n{rec2:.3f}", (rec2, y), xytext=(10, 0),
textcoords="offset points", ha="left", va="center", color=INK,
fontsize=10)
ax.plot([0.05, 0.05], [-0.6, 1.5], color=INK2, lw=1, ls="--", zorder=0)
ax.text(0.05, 1.55, "α = 0.05", ha="center", va="bottom", color=INK2, fontsize=10)
ax.set_xscale("log")
ax.set_xlim(5e-4, 0.8)
ax.set_ylim(-0.6, 1.75)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_xticks([0.001, 0.003, 0.01, 0.03, 0.1, 0.3])
ax.set_xticklabels(["0.001", "0.003", "0.01", "0.03", "0.1", "0.3"])
ax.minorticks_off()
ax.set_xlabel("binomial p-value (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
handles = [Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="reported in paper"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE,
label="recomputed, one-sided"),
Line2D([], [], marker="D", ls="", ms=8, mfc=BG, mec=BLUE, mew=2,
label="recomputed, two-sided")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.22, -0.25), ncol=3,
frameon=False, fontsize=10, handletextpad=0.3, columnspacing=1.2)
fig.tight_layout()
save(fig, "fig1")
# Figure 2: Binomial(17, 0.5) distribution; which tail each p-value sums
fig, ax = plt.subplots(figsize=(7.2, 3.6), facecolor=BG)
ax.set_facecolor(BG)
ks = list(range(N + 1))
pm = [binom.pmf(k, N, 0.5) for k in ks]
cols = ["#c9c8c3" if k < 12 else BLUE for k in ks]
ax.bar(ks, pm, width=0.8, color=cols, edgecolor=BG, lw=2, zorder=2)
ax.bar([12], [pm[12]], width=0.8, color=BLUE, edgecolor=BG, lw=2, zorder=2)
ytop = max(pm)
yb1, yb2 = pm[12] + 0.035, pm[12] + 0.07
ax.plot([12.6, 17.4], [yb1, yb1], color=ORANGE, lw=2)
for x in (12.6, 17.4):
ax.plot([x, x], [yb1 - 0.006, yb1], color=ORANGE, lw=2)
ax.text(15, yb1 + 0.004, f"reported p = P(X≥13) = {float(p_ge13):.4f}", ha="center",
va="bottom", fontsize=10, color=INK)
ax.plot([11.6, 17.4], [yb2, yb2], color=BLUE, lw=2)
for x in (11.6, 17.4):
ax.plot([x, x], [yb2 - 0.006, yb2], color=BLUE, lw=2)
ax.text(14.5, yb2 + 0.004, f"correct p = P(X≥12) = {float(p_ge12):.4f}", ha="center",
va="bottom", fontsize=10, color=INK)
ax.annotate(f"observed: 12 of 17\nP(X=12) = {binom.pmf(12, N, 0.5):.4f}",
(12, pm[12]), xytext=(9.0, pm[12] + 0.02), ha="right", fontsize=10,
color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.set_xticks(ks)
ax.set_xlabel("number of complexes with ROC-AUC > 0.5 (out of 17)")
ax.set_ylabel("probability under null (p = 0.5)")
ax.set_ylim(0, ytop * 1.08)
ax.set_xlim(-0.7, 17.7)
ax.grid(axis="y", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
fig.tight_layout()
save(fig, "fig2")
print("\nfigures written:", ", ".join(f"fig{i}.{e}" for i in (1, 2) for e in ("svg", "png")))
Abstract reports the deductive-reasoning test as F(2,387), an error df larger than the whole sample
Brain (2025) · doi:10.1093/brain/awaf062 · PMC12073978
Summary
The Abstract reports the omnibus group comparison on the Deductive Reasoning Test (DRT) as F(2,387) = 18.491, P < 0.001. The Results report the same test as F(2,142) = 7.73, P < 0.001. The Results value fits the paper's own numbers exactly: Table 2 lists 48 + 55 + 43 = 146 participants, and an age-adjusted ANCOVA across three groups leaves 146 − 4 = 142 error degrees of freedom. An error df of 387 would need 391 participants, more than the 328 people recruited in total. Significance and direction are the same under either value, and the effect size each tuple implies is similar (partial η² 0.087 and 0.098). But the Abstract makes the evidence for the deductive-reasoning difference look as strong as that for the analogical-reasoning difference (F = 18.93) and implies a sample more than twice the size of the one analysed.
What the paper reports
ART and DRT performance was significantly impaired in patients with frontal lesions [ART: F(2,238) = 18.93; P < 0.001; Frontal group worse than Posterior group and healthy controls, both P < 0.001; DRT: F(2,387) = 18.491; P < 0.001; Frontal group worse than healthy controls, P < 0.01].
— Abstract
There was a significant difference between the Frontal, Posterior and HC groups for overall DRT performance [F(2,142) = 7.73; P < 0.001; see Table 2].
— Results, "Deductive reasoning task", first paragraph
HC(n = 48) | Frontal(n = 55) | Posterior(n = 43) | Left frontal(n = 24) | Right frontal(n = 31) | Left posterior(n = 19) | Right posterior(n = 24)
— Table 2, "Performance on the Deductive Reasoning Task", column headers. The table footer reads: "Groups are compared using analysis of covariance (ANCOVA), controlling for age."
Frontal, Posterior and HC groups were compared on demographic variables, using ANOVA and Fisher’s exact test, and performance on neuropsychological tests, ART and DRT, using analysis of covariance (ANCOVA), controlling for age.
— Methods, "Behavioural analysis"
Behavioural analyses were conducted on the 176 patients with lesions falling within frontal or posterior areas and HCs using SPSS version 29 (https://www.ibm.com/spss).
— Methods, "Behavioural analysis"
Two-hundred and forty-seven patients with unilateral, focal lesions were prospectively recruited from the National Hospital for Neurology and Neurosurgery
Eighty-one HCs, with no neurological or psychiatric history, were recruited to match patients as closely as possible for age, gender, years of education and National Adult Reading Test (NART) scores.
— Methods, "Participants"
What the paper's own numbers show
Method. In a one-way ANCOVA with k = 3 groups and one covariate (age), the F test for the group effect has k − 1 = 2 numerator df and N − k − 1 = N − 4 error df. The group sizes printed in the table headers therefore fix the error df. The same rule was checked against every other F test in the Results whose groups are tabulated. The age-adjusted F itself cannot be rebuilt from summary tables, because that needs the covariance of age with the scores. As a reference point only, the unadjusted one-way ANOVA F was computed from each table's n, means and SDs. Every number below is printed by recompute.py.txt (Python, numpy/scipy).
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| DRT sample size | Table 2 header | HC 48, Frontal 55, Posterior 43 | N = 146; subgroups agree (24 + 31 = 55; 19 + 24 = 43) |
| DRT error df, ANCOVA with age | Results | 142 | 146 − 4 = 142 (matches) |
| DRT error df | Abstract | 387 | would need N = 391 (ANCOVA) or 390 (no covariate) |
| Largest possible sample | Methods, Participants | 247 patients + 81 HCs | 328 in total, so a three-group ANCOVA on all of them has error df 324 |
| Behavioural analysis set | Methods, Behavioural analysis | 176 patients + HCs | 176 + 81 = 257 |
| P for F(2,142) = 7.73 | Results | P < 0.001 | 6.5 × 10⁻⁴ |
| P for F(2,387) = 18.491 | Abstract | P < 0.001 | 2.1 × 10⁻⁸ |
| Partial η² implied by each tuple | Results / Abstract | — | 0.098 / 0.087 |
| Ratio of the two DRT F values | Abstract vs Results | 18.491 vs 7.73 | 2.39 |
| Unadjusted DRT F from Table 2 n, means, SDs | Table 2 | — | F(2,143) = 6.80, P = 0.0015 (Results F is 1.14 times this; Abstract F is 2.72 times) |
| Control: ART sample and error df | Table 1; Abstract; Results | n 72 + 100 + 70; df 238 | N = 242; 242 − 4 = 238 (matches) |
| Control: unadjusted ART F from Table 1 | Table 1 | F(2,238) = 18.93 | F(2,239) = 17.36 (reported is 1.09 times this) |
| Control: DRT indeterminate / determinate | Results | F(2,142) = 7.93 / 4.23 | unadjusted 7.18 / 3.81 (ratios 1.10 / 1.11); df 142 matches |
| Control: ART progression / odd-one-out | Results | F(2,238) = 15.20 / 11.74 | unadjusted 14.16 / 11.13 (ratios 1.07 / 1.06); df 238 matches |
| Control: pooled Frontal and Posterior DRT means | Table 2 | 71.67 / 79.37 | 71.68 / 79.37 from the hemisphere subgroups |
| Other error dfs in Results (RAPM, S fluency, Stroop) | Results | 184, 151, 150 | imply N = 188, 155, 154, all ≤ 257 |
Every ART and DRT F test in the Results has error df = N − 4 for its table, and each of the six tabulated ANCOVA Fs lies 6–14% above the unadjusted F from the same table. The Abstract's DRT tuple is the only one that fits neither pattern.
Figure
Figure 1. The two printed versions of the DRT omnibus test. Left: the Abstract's F(2,387) = 18.491 (orange). Under the ANCOVA described in Methods (three groups, age as covariate) an error df of 387 corresponds to N = 391 (gray, derived from the Abstract's df). Right: the Results' F(2,142) = 7.73 and the Table 2 group sizes (black, as printed), from which N = 146 and error df 146 − 4 = 142 are recomputed (blue). The study recruited 328 participants in total (247 patients and 81 healthy controls), so no analysis described in the paper could have 387 error df.
Figure 2. Reported error df (orange circles) and the error df recomputed as N − 4 from the group sizes in Table 1 or Table 2 (blue diamonds) for every ART and DRT omnibus F test in the paper. The seven further ART tests (progression, odd-one-out, intra- and cross-dimensional and others) all report df 238 and are drawn as one row. The dotted line marks 324, the error df of a three-group ANCOVA with age on all 328 recruited participants. The Abstract's DRT value is the only one that departs from N − 4, and it lies beyond that ceiling.
Why it matters
The Abstract gives the DRT omnibus test as one of two headline behavioural results, set beside the ART result as evidence that "ART and DRT performance was significantly impaired in patients with frontal lesions". Read from the Abstract, the evidence for the two group differences looks about equally strong (F = 18.93 and F = 18.491, both with P near 2 × 10⁻⁸). By the paper's own Results, the evidence for the DRT difference is clearly weaker (F = 7.73, P = 6.5 × 10⁻⁴), and it rests on 146 participants, not the roughly 391 that the printed df implies. The DRT effect size is smaller than the ART one under either tuple (partial η² 0.087 or 0.098, against 0.137 for the ART).
The stated conclusions do not change. Both tuples give P < 0.001, the Frontal group is worse than healthy controls under either, and the implied partial η² is similar (0.087 and 0.098). The paper's main findings — the right-versus-left frontal differences on the ART and on DRT indeterminate questions, and the lesion-deficit network mapping — rest on other analyses.
The readers most likely to be affected are those who take test statistics or sample sizes from abstracts, such as reviewers judging sample size and precision, or meta-analysts. A meta-analyst who converts the Abstract tuple gets a slightly smaller partial η² (0.087 instead of 0.098) and a much larger N (391 instead of 146), which would give the DRT result far more weight than its sample supports. The Abstract also presents the DRT as a promising clinical test. For a reader weighing that claim, the Abstract overstates the strength of evidence and the sample size behind the DRT result, while the effect size it implies is close to the Results value.
Explanations considered
- A different or larger analysis population. This does not account for the numbers. Methods state that behavioural analyses used the 176 frontal and posterior patients and the HCs, at most 257 people. Even a three-group ANCOVA on all 247 recruited patients and all 81 HCs together (328) would have an error df of 324, not 387.
- An analysis without the age covariate. This does not account for the numbers. Dropping the covariate raises the error df by one (390 needed instead of 391), which still exceeds the recruited sample.
- Repeated measures, or pooling determinate and indeterminate questions. This does not account for the numbers. The paper analyses the two question types in separate ANCOVAs, each with df 142. Pooling 146 participants' two scores would give 292 observations and an error df of 288, not 387, and between-subject effects in a repeated-measures model would keep df 142.
- A typing slip in the df alone. This does not account for the numbers. The F value also differs (18.491 against 7.73), and it is reported to three decimals where every other F in the paper has two.
- Age adjustment raising F. This does not account for the numbers on its own terms. The same paper reports the age-adjusted F for this test as 7.73. Across all six tabulated tests the reported ANCOVA F is 1.06–1.14 times the unadjusted F, while the Abstract's 18.491 would be 2.72 times.
- A conversion or typesetting artefact in the archived copy. This was ruled out. The publisher's version of record and the Europe PMC full-text XML give the same Abstract text ("F(2,387) = 18.491") and the same Results text ("F(2,142) = 7.73") as the PMC archive.
- An analysis reported only in the supplement. This was ruled out. The supplementary PDF contains task descriptions, lesion methods and two supplementary tables (demographics and background tests). It reports no DRT omnibus statistic and no group sizes that could give an error df of 387.
The Abstract's 18.491 is numerically close to the ART statistic, 18.93. That is an observation only; the paper's numbers do not show how the Abstract value arose.
Status
As of 2026-10-23:
- Version of record: checked. doi.org resolves to the publisher page (Brain 148(5):1757). Its Abstract reads "DRT: F(2,387) = 18.491; P < 0.001" and its Results read "F(2,142) = 7.73; P < 0.001". The Europe PMC JATS XML and the local PMC archive match, and the Table 2 header in the XML gives n = 48, 55 and 43. The PMC HTML page returned a CAPTCHA, so Europe PMC was used in its place.
- Supplement: checked. The supplementary bundle from Europe PMC holds the same supplementary PDF (14 pages) linked from the publisher page. Its text was searched for "387", "18.49", "F(2", "DRT" and "Deductive". It does not resolve the discrepancy (see above).
- Corrections: none found. Crossref lists no updates (no update-to or updated-by entries, and no Retraction Watch entry in the Crossref labs record), and PMC and Europe PMC list no notices, checked with
src/known.pyand the Crossref API. Europe PMC (PMID 40233941) links only to a scientific commentary in the same issue (doi:10.1093/brain/awaf117), which quotes no F statistics and mentions no error. The publisher page shows no correction notice. - PubPeer: the PubPeer API returned HTTP 403 to an automated query. A web search for the title with "PubPeer", "correction" or "erratum" found no thread, erratum or published comment. A manual search at pubpeer.com is still advisable.
Sources: doi.org/10.1093/brain/awaf062 · publisher article page · PMC12073978 · Europe PMC record · Crossref works record · scientific commentary
Suggested correction
The authors could check the DRT omnibus ANCOVA in the Abstract against the original output and align it with the Results. If the Results value is correct, the Abstract would read "DRT: F(2,142) = 7.73; P < 0.001; Frontal group worse than healthy controls, P < 0.01". If 18.491 comes from a different analysis, the Abstract should say which analysis it is and how many participants it included, since no analysis described in the paper has an error df of 387.
Recomputation script (python recompute.py.txt)
"""Recompute the Deductive Reasoning Test (DRT) omnibus statistics of PMC12073978
(Brain 2025, doi:10.1093/brain/awaf062) and regenerate the figures.
Run: python recompute.py.txt
Needs: numpy, scipy, matplotlib.
Prints every recomputed number used in writeup.md. Writes fig1.svg/.png and
fig2.svg/.png into the folder that holds this file, plus five candidate figure
designs (cand_A ... cand_E, .svg/.png) into ./candidates/.
All inputs are copied from the paper (Abstract; Methods, Participants and
Behavioural analysis; Results; Tables 1 and 2).
Model stated in Methods: one-way ANCOVA across three groups (Frontal, Posterior,
healthy controls) with age as the single covariate, so
group df = k - 1 = 2, error df = N - k - 1 = N - 4.
The age-adjusted F cannot be rebuilt from the tables (it needs the age-score
covariance), so the unadjusted one-way ANOVA F is computed from the printed
group n, means and SDs as a reference point only.
"""
import os
import numpy as np
from scipy import stats
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
# ---------------------------------------------------------------- inputs
N_PATIENTS_RECRUITED = 247 # Methods, Participants
N_PATIENTS_FP = 176 # frontal 102 + posterior 74
N_HC = 81 # Methods, Participants
DRT_N = {"HC": 48, "Frontal": 55, "Posterior": 43} # Table 2 header
DRT_SUB = {"LF": 24, "RF": 31, "LP": 19, "RP": 24}
ART_N = {"HC": 72, "Frontal": 100, "Posterior": 70} # Table 1 header
ABSTRACT_DRT = (2, 387, 18.491) # Abstract
RESULTS_DRT = (2, 142, 7.73) # Results, Deductive reasoning task, p0
ART_REPORTED = (2, 238, 18.93) # Abstract and Results
# (label, ns, means, sds, reported F, reported df2, location)
TESTS = [
("DRT overall", list(DRT_N.values()), [82.79, 71.67, 79.37], [15.22, 15.90, 16.03], 7.73, 142, "Results"),
("DRT indeterminate", list(DRT_N.values()), [76.06, 55.89, 70.56], [25.21, 30.53, 27.60], 7.93, 142, "Results"),
("DRT determinate", list(DRT_N.values()), [84.79, 76.69, 82.14], [15.42, 15.12, 15.17], 4.23, 142, "Results"),
("ART overall", list(ART_N.values()), [79.40, 67.79, 77.38], [11.93, 15.51, 13.51], 18.93, 238, "Abstract; Results"),
("ART progression", list(ART_N.values()), [83.22, 71.94, 81.19], [12.86, 16.60, 14.52], 15.20, 238, "Results"),
("ART odd-one-out", list(ART_N.values()), [75.58, 63.71, 73.57], [15.09, 20.06, 17.08], 11.74, 238, "Results"),
]
def anova_from_summary(ns, means, sds):
ns, m, s = (np.asarray(x, float) for x in (ns, means, sds))
N, k = ns.sum(), len(ns)
gm = (ns * m).sum() / N
ssb = (ns * (m - gm) ** 2).sum()
ssw = ((ns - 1) * s ** 2).sum()
F = (ssb / (k - 1)) / (ssw / (N - k))
return int(N), k, F, stats.f.sf(F, k - 1, N - k)
def p_of(F, d1, d2):
return stats.f.sf(F, d1, d2)
def eta_p2(F, d1, d2):
return F * d1 / (F * d1 + d2)
# ---------------------------------------------------------------- numbers
print("== Sample sizes ==")
n_drt = sum(DRT_N.values())
n_art = sum(ART_N.values())
print(f"DRT analysed (Table 2): 48 + 55 + 43 = {n_drt}")
print(f" subgroup check: LF + RF = {DRT_SUB['LF'] + DRT_SUB['RF']} (Frontal 55); "
f"LP + RP = {DRT_SUB['LP'] + DRT_SUB['RP']} (Posterior 43)")
print(f"ART analysed (Table 1): 72 + 100 + 70 = {n_art}")
max_behav = N_PATIENTS_FP + N_HC
all_recruited = N_PATIENTS_RECRUITED + N_HC
print(f"Behavioural analysis set (176 F/P patients + 81 HC) = {max_behav}")
print(f"All recruited (247 patients + 81 HC) = {all_recruited}")
print("\n== Error df under the stated ANCOVA (N - 3 groups - 1 covariate) ==")
print(f"DRT: {n_drt} - 4 = {n_drt - 4} (Results reports 142)")
print(f"ART: {n_art} - 4 = {n_art - 4} (Abstract and Results report 238)")
n_implied_abs = ABSTRACT_DRT[1] + 4
print(f"N implied by abstract F(2,387): ANCOVA {n_implied_abs}; ANOVA without covariate {ABSTRACT_DRT[1] + 3}")
print(f" excess over DRT sample: {n_implied_abs - n_drt}; over all recruited: {n_implied_abs - all_recruited}")
print(f"N implied by Results F(2,142): {RESULTS_DRT[1] + 4}")
print("Within-subject pooling check: 146 participants x 2 question types = "
f"{2 * n_drt} observations -> error df {2 * n_drt - 4}")
print("\n== P values and partial eta-squared of the reported tuples ==")
for lab, (d1, d2, F) in [("Abstract DRT", ABSTRACT_DRT), ("Results DRT", RESULTS_DRT), ("ART", ART_REPORTED)]:
print(f"{lab:13s} F({d1},{d2}) = {F}: P = {p_of(F, d1, d2):.2e}, partial eta^2 = {eta_p2(F, d1, d2):.3f}")
print(f"Abstract DRT F / Results DRT F = {ABSTRACT_DRT[2] / RESULTS_DRT[2]:.2f}")
print(f"Abstract DRT F vs ART F: {ABSTRACT_DRT[2]} vs {ART_REPORTED[2]} (difference {ART_REPORTED[2] - ABSTRACT_DRT[2]:.3f})")
print("\n== Unadjusted one-way ANOVA from the tables' n, mean, SD (reference only) ==")
UNADJ = {}
for lab, ns, m, s, Frep, d2rep, loc in TESTS:
N, k, F, p = anova_from_summary(ns, m, s)
UNADJ[lab] = F
print(f"{lab:18s} N={N} unadjusted F(2,{N - k}) = {F:5.2f}, P = {p:.2e} | "
f"reported ANCOVA F(2,{d2rep}) = {Frep} ({loc}); ratio reported/unadjusted = {Frep / F:.2f}; "
f"expected error df N-4 = {N - 4}")
print(f"Abstract DRT F / unadjusted DRT F = {ABSTRACT_DRT[2] / UNADJ['DRT overall']:.2f}")
print("\n== Pooled group means from the hemisphere subgroups (Table 2) ==")
print(f"DRT Frontal: (24*75.88 + 31*68.42)/55 = {(24 * 75.88 + 31 * 68.42) / 55:.2f} (reported 71.67)")
print(f"DRT Posterior: (19*77.05 + 24*81.21)/43 = {(19 * 77.05 + 24 * 81.21) / 43:.2f} (reported 79.37)")
# Other error dfs reported in Results (implied N under the same model)
OTHER_DF = {"RAPM": 184, "S fluency": 151, "Stroop": 150}
print("\n== Other reported error dfs -> implied N (N = df + 4) ==")
for k_, v in OTHER_DF.items():
print(f"{k_}: df {v} -> N {v + 4} (<= {max_behav})")
# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": TXT,
"axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2
def save(fig, path_noext):
os.makedirs(os.path.dirname(path_noext), exist_ok=True)
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def style_axis(ax):
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
def panel(ax, x, y, w, h, title, lines, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.012,rounding_size=0.02",
fc="white", ec=edge, lw=1.6))
ax.text(x + 0.03, y + h - 0.06, title, fontsize=11.5, weight="bold", va="top")
yy = y + h - 0.17
for txt, col, wt in lines:
ax.text(x + 0.03, yy, txt, fontsize=10.5, color=col, weight=wt, va="top", family="DejaVu Sans")
yy -= 0.085
# --- Candidate A: two text panels joined by an annotated connector
def cand_A(path):
fig = plt.figure(figsize=(W, 3.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
panel(ax, 0.03, 0.30, 0.40, 0.62, "Abstract", [
("“DRT: F(2,387) = 18.491;", ORANGE, "bold"),
("P < 0.001”", ORANGE, "bold"),
("", TXT, "normal"),
("error df 387 implies", SEC, "normal"),
(f"N = 387 + 4 = {n_implied_abs} participants", SEC, "normal"),
], ORANGE)
panel(ax, 0.57, 0.30, 0.40, 0.62, "Results / Table 2", [
("“F(2,142) = 7.73; P < 0.001”", TXT, "bold"),
("Table 2: n = 48 + 55 + 43", TXT, "normal"),
("", TXT, "normal"),
(f"recomputed: N = {n_drt}", BLUE, "bold"),
(f"error df = {n_drt} − 4 = {n_drt - 4} ✓", BLUE, "bold"),
], BLUE)
ax.annotate("", xy=(0.565, 0.61), xytext=(0.435, 0.61),
arrowprops=dict(arrowstyle="<->", color=TXT, lw=2))
ax.text(0.50, 0.66, "same test", ha="center", fontsize=10, color=SEC)
ax.text(0.50, 0.185, f"Conflict: the abstract's df needs {n_implied_abs} participants; the DRT sample is {n_drt},\n"
f"and only {all_recruited} people (247 patients + 81 controls) were recruited in total.",
ha="center", va="center", fontsize=10.5)
ax.text(0.03, 0.025, "Orange: printed in the Abstract. Black: printed in the Results and Table 2.\n"
"Blue: recomputed from the paper's group sizes. Gray: derived from the Abstract's df.",
fontsize=10, color=SEC, va="bottom")
save(fig, path)
# --- Candidate B: participants on one axis
def cand_B(path):
rows = [
("Implied by abstract F(2,387)", n_implied_abs, ORANGE, "o", "reported df + 4"),
("All recruited (247 + 81)", all_recruited, SEC, "s", "Methods"),
("Behavioural set (176 + 81)", max_behav, SEC, "s", "Methods"),
("Implied by Results F(2,142)", RESULTS_DRT[1] + 4, TXT, "o", "reported df + 4"),
("DRT groups, Table 2 (48+55+43)", n_drt, BLUE, "D", "recomputed sum"),
]
fig, ax = plt.subplots(figsize=(W, 3.6))
for i, (lab, v, c, mk, note) in enumerate(rows[::-1]):
ax.hlines(i, 0, v, color=GRID, lw=2)
ax.plot(v, i, mk, color=c, ms=10, mfc=c if mk != "o" or c == ORANGE else "white", mew=2)
ax.text(v + 8, i, f"{v}", va="center", fontsize=10.5, color=c, weight="bold")
ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows[::-1]])
ax.axvline(all_recruited, color=SEC, lw=1, ls=":")
ax.text(all_recruited - 4, len(rows) - 0.45, "everyone recruited", ha="right", fontsize=10, color=SEC)
ax.set_xlim(0, 440); ax.set_ylim(-0.6, len(rows) - 0.3)
ax.set_xlabel("Participants")
style_axis(ax)
fig.text(0.01, 0.01, "Orange: from the abstract's df. Blue: recomputed from Table 2.\nGray: counts in Methods. Black open circle: from the Results df.",
fontsize=10, color=SEC)
fig.tight_layout(rect=(0, 0.11, 1, 1))
save(fig, path)
# --- Candidate C: calculation flow
def cand_C(path):
fig = plt.figure(figsize=(W, 3.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
def box(x, y, w, h, t, ec, col=TXT, wt="normal"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.008,rounding_size=0.015",
fc="white", ec=ec, lw=1.6))
ax.text(x + w / 2, y + h / 2, t, ha="center", va="center", fontsize=10.5, color=col, weight=wt)
def arrow(x0, x1, y, t=None):
ax.annotate("", xy=(x1, y), xytext=(x0, y), arrowprops=dict(arrowstyle="->", color=TXT, lw=2))
if t:
ax.text((x0 + x1) / 2, y + 0.11, t, ha="center", va="bottom", fontsize=10, color=SEC)
# columns: x, width
cols = [(0.02, 0.21), (0.30, 0.15), (0.53, 0.15), (0.76, 0.22)]
ya, yb, h = 0.60, 0.17, 0.19
ax.text(0.02, 0.92, "From the paper's group sizes (Table 2) to the Results", fontsize=11, weight="bold")
box(*[cols[0][0], ya, cols[0][1], h], "Table 2 groups\n48 + 55 + 43", SEC)
box(cols[1][0], ya, cols[1][1], h, f"N = {n_drt}", BLUE, BLUE, "bold")
box(cols[2][0], ya, cols[2][1], h, f"error df\n= {n_drt - 4}", BLUE, BLUE, "bold")
box(cols[3][0], ya, cols[3][1], h, "Results:\nF(2,142) = 7.73 ✓", TXT, TXT, "bold")
arrow(0.235, 0.295, ya + h / 2, "sum")
arrow(0.455, 0.525, ya + h / 2, "− 4")
arrow(0.685, 0.755, ya + h / 2)
ax.text(0.02, 0.49, "From the abstract's df back to a sample size", fontsize=11, weight="bold")
box(cols[3][0], yb, cols[3][1], h, "Abstract:\nF(2,387) = 18.491", ORANGE, ORANGE, "bold")
box(cols[2][0], yb, cols[2][1], h, "error df\n= 387", ORANGE, ORANGE, "bold")
box(cols[1][0], yb, cols[1][1], h, f"N = {n_implied_abs}", ORANGE, ORANGE, "bold")
box(cols[0][0], yb, cols[0][1], h, f"more than all\n{all_recruited} recruited ✗", ORANGE, TXT)
arrow(0.755, 0.685, yb + h / 2)
arrow(0.525, 0.455, yb + h / 2, "+ 4")
arrow(0.295, 0.235, yb + h / 2)
ax.text(0.02, 0.04, "− 4 = three groups and one age covariate (ANCOVA, Methods). Blue: recomputed from\n"
"the paper's group sizes. Orange: printed in the abstract or derived from it.",
fontsize=10, color=SEC, va="bottom")
save(fig, path)
# --- Candidate D: F statistics, reported vs unadjusted reference
def cand_D(path):
rows = [("DRT overall (Abstract)", ABSTRACT_DRT[2], UNADJ["DRT overall"], True)]
for lab, ns, m, s, Frep, d2, loc in TESTS:
rows.append((f"{lab} ({loc.replace('; ', ', ')})", Frep, UNADJ[lab], False))
rows = rows[::-1]
fig, ax = plt.subplots(figsize=(W, 4.3))
for i, (lab, Frep, Fu, hl) in enumerate(rows):
ax.hlines(i, min(Frep, Fu), max(Frep, Fu), color=GRID, lw=2)
ax.plot(Fu, i, "D", color=BLUE, ms=8)
ax.plot(Frep, i, "s" if hl else "o", color=ORANGE, ms=9)
if hl:
ax.text((Frep + Fu) / 2, i + 0.17, f"{Frep} = {Frep / Fu:.1f} × the unadjusted F of {Fu:.2f}",
ha="center", va="bottom", fontsize=10, color=ORANGE, weight="bold")
ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows])
for t in ax.get_yticklabels():
if t.get_text().startswith("DRT overall (Abstract"):
t.set_weight("bold")
ax.set_xlim(0, 25); ax.set_ylim(-0.5, len(rows) - 0.15)
ax.set_xlabel("F statistic for the group effect")
style_axis(ax)
ax.plot([], [], "s", color=ORANGE, ms=9, label="reported F (Abstract; no stated analysis gives df 387)")
ax.plot([], [], "o", color=ORANGE, ms=9, label="reported F (Results, ANCOVA with age)")
ax.plot([], [], "D", color=BLUE, ms=8, label="unadjusted F recomputed from table n, mean, SD")
ax.legend(loc="upper center", bbox_to_anchor=(0.35, -0.2), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
# --- Candidate E: error df audit, reported vs N - 4
def cand_E(path):
rows = [
("DRT overall (Abstract)", 387, n_drt - 4),
("DRT overall (Results)", 142, n_drt - 4),
("DRT indeterminate (Results)", 142, n_drt - 4),
("DRT determinate (Results)", 142, n_drt - 4),
("ART overall (Abstract, Results)", 238, n_art - 4),
("ART, 7 further tests (Results)", 238, n_art - 4),
][::-1]
fig, ax = plt.subplots(figsize=(W, 3.5))
for i, (lab, rep, exp) in enumerate(rows):
ax.hlines(i, min(rep, exp), max(rep, exp), color=GRID, lw=2)
ax.plot(exp, i, "D", color=BLUE, ms=11)
ax.plot(rep, i, "o", color=ORANGE, ms=8)
if rep != exp:
ax.text(exp + 12, i - 0.3, f"387 vs {exp}", ha="left", va="center",
fontsize=10, color=TXT, weight="bold")
lim = all_recruited - 4
ax.axvline(lim, color=SEC, lw=1, ls=":")
ax.text(lim - 9, 2.5, f"largest error df for a\nthree-group ANCOVA on all\n{all_recruited} recruited: {lim}",
ha="right", va="center", fontsize=10, color=SEC)
ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows])
ax.set_xlim(0, 420); ax.set_ylim(-0.5, len(rows) - 0.5)
ax.set_xlabel("Error degrees of freedom")
style_axis(ax)
ax.plot([], [], "o", color=ORANGE, ms=8, label="reported error df")
ax.plot([], [], "D", color=BLUE, ms=11, label="N − 4 recomputed from the table's group sizes")
ax.legend(loc="upper center", bbox_to_anchor=(0.35, -0.22), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
cand_A(os.path.join(CAND, "cand_A_text_panels"))
cand_B(os.path.join(CAND, "cand_B_participants_axis"))
cand_C(os.path.join(CAND, "cand_C_calculation_flow"))
cand_D(os.path.join(CAND, "cand_D_F_statistics"))
cand_E(os.path.join(CAND, "cand_E_error_df_audit"))
# Provisional choice for the write-up until a design is picked: A as fig1, E as fig2.
cand_A(os.path.join(HERE, "fig1"))
cand_E(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE, "and", CAND)
Abstract calls a MELD–referral odds ratio of 1.13 "lower referral odds"
JAMA Netw Open (2026) · doi:10.1001/jamanetworkopen.2026.2567 · PMC13003376
Summary
The study follows 325 patients hospitalized with severe alcohol-associated hepatitis at 5 US transplant centers and models referral for liver transplant evaluation. The abstract states that "higher MELD scores were associated with lower referral odds" and gives, in the same sentence, an odds ratio of 1.13 (95% CI 1.07–1.18; P < .001). An odds ratio above 1, with a confidence interval wholly above 1, means that higher MELD scores go with higher odds of referral. The Results text, Table 2, Table 1 (mean MELD 32.3 in referred vs 27.4 in non-referred patients) and Supplement 1 eTable 3 all show that direction. All the numbers agree with each other. The error is a single direction word in the abstract, which reverses the stated direction of the association for one of the study's primary outcomes.
What the paper reports
Abstract, Results:
In multivariable analysis, higher MELD scores were associated with lower referral odds (odds ratio, 1.13 [95% CI, 1.07-1.18]; P < .001).
Results, "Factors Associated With Referral for Transplant Evaluation", paragraph 1:
In multivariable logistic regression, adjusting for age, sex, race, ethnicity, educational level, insurance, marital status, living with children, MELD score, ADI, and site of care, a higher MELD score was associated with increased odds of referral (odds ratio [OR], 1.13 [95% CI, 1.07-1.18]; P < .001) (Table 2).
Table 2 (Multivariable Logistic Regression Analyses at Each Step in the Liver Transplant Cascade), referral column (n = 299):
MELD score (≤40)c | 1.13 (1.07-1.18) | <.001
Table 2, footnote c (the same definition appears in Table 1 and in the Methods):
Scores range from 6 to 40, with higher scores indicating severe liver disease.
Results, "Patient Characteristics", paragraph 3, and Table 1:
Compared with those who were not referred, patients who were referred for LT evaluation had significantly higher mean (SD) MELD scores (32.3 [7.7] vs 27.4 [6.9]; P < .001)
Discussion, paragraph 2:
As expected, the MELD score was associated with the transplant cascade.
Supplement 1, eTable 3 (referral model with an insurance-by-site interaction), row "Meld_cap": estimate 0.1432, standard error 0.0284, Wald χ² 25.4024, P < .0001.
All quotes from the article were checked verbatim against the PMC-archived text with src/show.py PMC13003376 --grep. The eTable 3 values were read from the supplement PDF.
What the paper's own numbers show
Method. In a logistic model, OR = exp(β). An OR above 1 means a positive coefficient: the odds of the outcome (here, referral) rise as the covariate rises. I converted the printed OR and CI bounds to the log scale and took the standard error from the CI width to get a z statistic. I compared the abstract's numbers with the Results text and Table 2. I computed the OR that a "lower odds" statement would need, which is the reciprocal OR per one-point lower MELD. I converted the eTable 3 coefficient to an OR with a Wald CI. As a descriptive check, I compared the Table 1 MELD means by referral status with a Welch t test, and approximated the crude logistic slope with the normal-discriminant formula (difference in means divided by pooled variance). That approximation is unadjusted and assumes normal MELD distributions with equal variances. It is used only to check the sign. All calculations are in recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| MELD → referral, adjusted OR | Abstract, Results | 1.13 (1.07–1.18), P < .001, described as "lower referral odds" | ln OR = +0.122; ln CI bounds +0.068 and +0.166, both > 0; z = 4.90, p ≈ 1.0 × 10⁻⁶: higher odds |
| MELD → referral, adjusted OR | Results, paragraph 1 | 1.13 (1.07–1.18), P < .001, "increased odds" | Same numbers as the abstract; direction word consistent with them |
| MELD → referral, adjusted OR | Table 2, referral column | 1.13 (1.07–1.18), P < .001 | Geometric CI midpoint 1.124, consistent with 1.13 |
| Size of the effect | derived | — | +13% odds per MELD point (CI +7% to +18%); 1.13¹⁰ = 3.39 (1.97–5.23) per 10 points |
| OR that "lower odds" would require | derived | not reported anywhere | 1/1.13 = 0.885 (0.847–0.935), per 1-point lower MELD |
| Sensitivity referral model | Supplement 1, eTable 3, Meld_cap | β = 0.1432, SE 0.0284, χ² 25.40, P < .0001 | (β/SE)² = 25.42; OR = 1.154 (1.091–1.220): higher odds |
| Mean MELD, referred vs not referred | Table 1; Results, Patient Characteristics | 32.3 (7.7) vs 27.4 (6.9), P < .001; SMD 0.68 | Difference +4.9 points; Welch t = 5.75 (df 228), p = 2.9 × 10⁻⁸; SMD 0.67 |
| Crude OR per MELD point (approximation) | from Table 1 | — | Pooled SD 7.21; slope 0.0944; OR ≈ 1.099 (> 1) |
| Referral rate | Abstract; Results | 120 (36.9%) | 120/325 = 36.9% |
Every number in the abstract sentence matches the Results and Table 2. The adjusted estimates, the sensitivity model and the descriptive data all point the same way: patients with higher MELD scores were more likely to be referred. Only the word "lower" in the abstract disagrees.
Figure
Figure 1. On the left is the abstract's sentence and on the right the Results text and Table 2 row. Both are short excerpts, and both printed estimates are shown in orange: OR 1.13 (95% CI 1.07–1.18), P < .001. The double arrow marks the conflict: the numbers are identical, but the abstract says "lower" and the Results say "increased". The blue box gives values recomputed from the printed numbers. The log OR and both log CI bounds are positive (z = 4.9), which contradicts "lower" and supports "increased". The Table 1 means (32.3 in referred vs 27.4 in non-referred patients) point the same way.
Figure 2. Odds ratios for referral per 1-point higher MELD score, on a log axis with a thin reference line at OR = 1. Orange: the estimate as printed in the abstract and in the Results and Table 2. It is identical in both places and lies entirely to the right of 1. Blue: values recomputed from the paper's own numbers. These are the eTable 3 sensitivity model, exp(0.1432 ± 1.96 × 0.0284) = 1.154 (1.091–1.220), and the crude normal-discriminant approximation from the Table 1 means (1.099, point estimate only), both above 1. The hollow blue marker is the reciprocal, 1/1.13 = 0.885 (0.847–0.935). This is the estimate a "lower referral odds" statement would require. The paper does not report it.
Why it matters
The abstract is the part of a paper that most readers see, and the part that indexing services, evidence summaries and citing articles most often take statements from. Referral is one of the study's primary outcomes, and MELD is one of its two named exposures. As printed, the abstract sentence tells a reader that sicker patients were less likely to be referred for transplant evaluation. That would be a clinically surprising result, and it is the opposite of what the paper's analysis shows. A reader who knows how to read an odds ratio can see that the sentence contradicts itself. A reader who takes in only the words, or a tool that extracts the claim from the text, would record the reversed direction.
The consequences are limited. No number is wrong, and the significance and size of the estimate are as printed: each additional MELD point goes with about 13% higher adjusted odds of referral. The title, Key Points and abstract Conclusions do not mention the MELD main effect. The Results, Table 2, Table 1, the Discussion and the supplement are all consistent with each other. The paper's headline finding is unaffected. That finding is the MELD × ADI interaction at intermediate MELD scores (P for interaction = .01) and the referral probabilities by ADI band, and neither depends on this sentence. The ADI, wait-listing, transplant and mortality results are also unaffected.
Explanations considered
- A different model or population in the abstract. The abstract says "in multivariable analysis" and gives the same OR, CI and P value as the multivariable referral model in the Results and the referral column of Table 2 (n = 299). No other referral model with a MELD estimate of 1.13 appears in the paper or the supplement. This does not explain the wording.
- MELD coded so that the OR is per lower score. The Methods, Table 1 and Table 2 (footnote c) define MELD as a score from 6 to 40 "with higher scores indicating severe liver disease". The Results describe the same OR of 1.13 as "increased odds" with higher MELD. If the abstract meant an OR per point of lower MELD, it would report 0.885 (0.847–0.935), not 1.13. This does not explain the wording.
- A different outcome (non-referral, or mortality). The sentence is explicitly about "referral odds". Under a non-referral coding the OR would be 0.885. The 180-day mortality OR for MELD (1.07 [1.03–1.12]) is also above 1 and is a different number. This does not explain the wording.
- The adjusted association could differ in sign from the crude one. It could in principle, but it does not here. The adjusted OR printed in the sentence is itself above 1. The crude data (Table 1, referred patients' MELD 4.9 points higher) and the eTable 3 sensitivity model agree in sign. This does not explain the wording.
- A text-conversion artifact. The error is a word, not a sign, superscript or decimal mark. The publisher's full-text page, the Europe PMC full-text XML and the local PMC archive all carry the same wording. This does not explain the wording.
- Rounding, one-sided tests or another stated convention. None of these can change the direction of an OR whose whole CI lies above 1, and the paper states no convention that would. This does not explain the wording.
The evidence fits a single wrong direction word in the abstract's Results paragraph. Nothing in the paper shows how it arose.
Status
As of 2026-10-23:
- Version of record. doi.org resolves to the publisher's full-text page (jamanetwork.com/journals/jamanetworkopen/fullarticle/2846714, published online March 19, 2026), which was read in this workflow. Its abstract Results has the same sentence, "higher MELD scores were associated with lower referral odds (odds ratio, 1.13 [95% CI, 1.07-1.18]; P < .001)". Its body Results has "a higher MELD score was associated with increased odds of referral". Its Table 2 MELD row gives 1.13 (1.07-1.18), P < .001. The Europe PMC full-text XML and the PMC archive have identical text. The direct PMC web page returned only a bot-verification screen.
- Supplements. The Europe PMC supplementary-files archive was downloaded. It contains Supplement 1 (jamanetwopen-e262567-s001.pdf, eTables 1–4), Supplement 2 (Data Sharing Statement) and the figure images. Supplement 1 was read. eTable 3 gives the MELD (Meld_cap) coefficient as +0.1432 (SE 0.0284, P < .0001), which is a positive association with referral. eTable 1 gives mean MELD 29.2 overall and 32.3 among referred patients. Nothing in the supplement supports "lower" odds. The supplement agrees with the body text.
- Corrections. None found.
src/known.pyreturned no Crossref updates and no PMC or Europe PMC notices. The Crossref works record has no update-to or updated-by relation. Europe PMC (PMID 41854611) lists no comment or correction. The publisher page shows no correction notice and no posted comments. A web search for the title together with "correction", "erratum" or "PubPeer" (repeated for this write-up) found no notice, and a Retraction Watch site search found nothing. - PubPeer. The PubPeer search returned HTTP 403 and could not be queried directly. Web searches found no PubPeer thread for this article.
Sources: publisher full text; PMC13003376; Europe PMC REST full-text XML and supplementary files for PMC13003376; Crossref REST API for doi:10.1001/jamanetworkopen.2026.2567.
Suggested correction
Abstract, Results. Replace "In multivariable analysis, higher MELD scores were associated with lower referral odds (odds ratio, 1.13 [95% CI, 1.07-1.18]; P < .001)." with:
In multivariable analysis, higher MELD scores were associated with higher referral odds (odds ratio, 1.13 [95% CI, 1.07-1.18]; P < .001).
The numbers stay as printed, and they match the Results text, Table 2 and Supplement 1. No other part of the paper needs to change.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC13003376 write-up and regenerate fig1/fig2.
Run: python recompute.py.txt
All inputs are values printed in the article (Abstract, Results, Tables 1-2)
or in Supplement 1 (eTable 3). Nothing is fitted to patient-level data.
"""
import math
import os
import numpy as np
from scipy import stats
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
def se_log(lo, hi):
"""SE of log(OR) implied by a 95% CI."""
return (math.log(hi) - math.log(lo)) / (2 * Z)
# ------------------------------------------------------------------ inputs
OR, LO, HI = 1.13, 1.07, 1.18 # Abstract = Results p0 = Table 2 (referral, MELD <=40)
N_REFMODEL = 299 # Table 2 referral column header
N_ALL, N_REF = 325, 120 # Abstract / Results
M1, S1, N1 = 32.3, 7.7, 120 # Table 1: with referral
M0, S0, N0 = 27.4, 6.9, 205 # Table 1: without referral
B_E3, SE_E3, CHI_E3 = 0.1432, 0.0284, 25.4024 # Supplement 1 eTable 3, Meld_cap
print("=== 1. Direction implied by the reported OR (Abstract, Results, Table 2) ===")
b = math.log(OR); se = se_log(LO, HI); z = b / se
print(f" log OR = ln(1.13) = {b:+.3f}")
print(f" log CI bounds: ln(1.07) = {math.log(LO):+.3f}, ln(1.18) = {math.log(HI):+.3f} (both > 0: {math.log(LO) > 0})")
print(f" SE(log OR) from CI = {se:.4f}; z = {z:.2f}; two-sided p = {2*stats.norm.sf(abs(z)):.1e} (reported P < .001)")
print(f" geometric CI midpoint = {math.sqrt(LO*HI):.3f} (reported OR {OR})")
print(f" percent change in odds per +1 MELD point: {100*(OR-1):.0f}% (CI {100*(LO-1):.0f}% to {100*(HI-1):.0f}%)")
print(f" OR per +10 MELD points = 1.13^10 = {OR**10:.2f} (CI {LO**10:.2f} to {HI**10:.2f})")
print("\n=== 2. What 'lower referral odds' would require (reciprocal, not reported) ===")
print(f" OR per 1-point LOWER MELD = 1/1.13 = {1/OR:.3f} (CI {1/HI:.3f} to {1/LO:.3f})")
print("\n=== 3. Supplement 1 eTable 3 (sensitivity referral model, insurance x site interaction) ===")
print(f" Meld_cap coefficient {B_E3:+.4f}, SE {SE_E3:.4f}")
print(f" Wald chi2 = (b/SE)^2 = {(B_E3/SE_E3)**2:.2f} (printed {CHI_E3})")
lo3, hi3 = math.exp(B_E3 - Z*SE_E3), math.exp(B_E3 + Z*SE_E3)
OR3 = math.exp(B_E3)
print(f" OR = exp(b) = {OR3:.3f} (Wald 95% CI {lo3:.3f} to {hi3:.3f})")
print("\n=== 4. Descriptive cross-check, Table 1 MELD by referral status ===")
diff = M1 - M0
t = diff / math.sqrt(S1**2/N1 + S0**2/N0)
dfw = (S1**2/N1 + S0**2/N0)**2 / ((S1**2/N1)**2/(N1-1) + (S0**2/N0)**2/(N0-1))
print(f" mean difference = {M1} - {M0} = {diff:.1f} MELD points (referred higher)")
print(f" Welch t = {t:.2f}, df = {dfw:.0f}, p = {2*stats.t.sf(t, dfw):.1e} (reported P < .001)")
sp2 = ((N1-1)*S1**2 + (N0-1)*S0**2) / (N1+N0-2)
smd = diff / math.sqrt((S1**2 + S0**2)/2)
print(f" SMD (average-variance) = {smd:.2f} (Table 1 SMD 0.68)")
beta_crude = diff / sp2
OR_crude = math.exp(beta_crude)
print(f" pooled SD = {math.sqrt(sp2):.2f}; normal-discriminant crude logistic slope = {beta_crude:.4f}"
f" -> crude OR per MELD point ~ {OR_crude:.3f} (approximation; > 1)")
print("\n=== 5. Counts ===")
print(f" referral rate = {N_REF}/{N_ALL} = {100*N_REF/N_ALL:.1f}% (reported 36.9%)")
print(f" referral model n (Table 2 header) = {N_REFMODEL}")
# ------------------------------------------------------------------ figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
BG, TXT, SEC, ORA, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "font.family": "DejaVu Sans",
"svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, f"{name}.svg"), facecolor=BG, bbox_inches="tight")
fig.savefig(os.path.join(HERE, f"{name}.png"), facecolor=BG, bbox_inches="tight", dpi=200)
plt.close(fig)
# ---- Figure 1: abstract vs Results/Table 2 text diagram
fig = plt.figure(figsize=(7.2, 5.0), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 70); ax.axis("off")
def panel(x, title, lines, word, after, numbers):
ax.add_patch(FancyBboxPatch((x, 30), 38, 34, boxstyle="round,pad=0.4,rounding_size=1.5",
fc="white", ec=SEC, lw=1.2))
ax.text(x + 2, 61.5, title, fontsize=11.5, weight="bold", va="top")
ax.text(x + 2, 55.5, lines, fontsize=10.5, style="italic", va="top", linespacing=1.35)
ax.text(x + 2, 44.6, word, fontsize=12, weight="bold", color=TXT, va="center",
bbox=dict(boxstyle="round,pad=0.25", fc="#fde6dc", ec=ORA, lw=1.5))
ax.text(x + 2 + 1.45*len(word) + 2.2, 44.6, after, fontsize=10.5, style="italic", va="center")
ax.text(x + 2, 37.3, numbers, fontsize=11, color=ORA, weight="bold", va="center", linespacing=1.3)
ax.text(x + 2, 32.4, "as printed in the paper", fontsize=10, color=SEC, va="center")
panel(2, "Abstract, Results",
"“In multivariable analysis, higher\nMELD scores were associated with",
"lower", "referral odds”", "OR 1.13 (95% CI 1.07–1.18);\nP < .001")
panel(60, "Results text and Table 2",
"“a higher MELD score was\nassociated with",
"increased", "odds of referral”", "OR 1.13 (95% CI 1.07–1.18);\nP < .001")
ax.add_patch(FancyArrowPatch((41.0, 44.6), (59.5, 44.6), arrowstyle="<|-|>", mutation_scale=13,
lw=2, color=TXT, shrinkA=1, shrinkB=1))
ax.text(50.5, 53.5, "Conflict:\nsame OR,\nCI and P;\nopposite\ndirection", ha="center", va="center",
fontsize=10, linespacing=1.15)
# recomputed box
ax.add_patch(FancyBboxPatch((14, 4), 72, 18, boxstyle="round,pad=0.4,rounding_size=1.5",
fc="#e8f0fb", ec=BLU, lw=2))
ax.text(50, 19.6, "Recomputed from the printed OR and CI", ha="center", va="top", fontsize=11,
weight="bold", color=BLU)
ax.text(50, 14.6,
f"ln(1.13) = {b:+.3f}; ln(1.07) = {math.log(LO):+.3f}, ln(1.18) = {math.log(HI):+.3f} (all > 0)\n"
f"z = {z:.1f}: each higher MELD point goes with HIGHER referral odds\n"
f"Table 1 agrees: mean MELD {M1} referred vs {M0} not referred",
ha="center", va="top", fontsize=10.2, linespacing=1.4)
ax.add_patch(FancyArrowPatch((28, 23.6), (22, 29.6), arrowstyle="-|>", mutation_scale=12,
lw=1.5, color=BLU))
ax.add_patch(FancyArrowPatch((72, 23.6), (78, 29.6), arrowstyle="-|>", mutation_scale=12,
lw=1.5, color=BLU))
ax.text(23, 26.6, "contradicts “lower”", fontsize=10, color=SEC, va="center", ha="right")
ax.text(77, 26.6, "supports “increased”", fontsize=10, color=SEC, va="center", ha="left")
leg = [Line2D([], [], color=ORA, lw=0, marker="s", ms=9, label="Reported in the paper"),
Line2D([], [], color=BLU, lw=0, marker="s", ms=9, label="Recomputed from the paper's numbers")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 1.045), ncol=2, frameon=False,
fontsize=10)
save(fig, "fig1")
# ---- Figure 2: dot-and-interval plot on a log axis
rows = [
("Abstract (says “lower”)", OR, LO, HI, ORA, "o", True),
("Results and Table 2 (“increased”)", OR, LO, HI, ORA, "o", True),
("eTable 3 sensitivity model", OR3, lo3, hi3, BLU, "s", True),
("Crude, from Table 1 means", OR_crude, None, None, BLU, "D", True),
("1/1.13: what “lower” would need\n(not reported)", 1/OR, 1/HI, 1/LO, BLU, "o", False),
]
fig, ax = plt.subplots(figsize=(7.2, 4.3), facecolor=BG)
ax.set_facecolor(BG)
ys = np.arange(len(rows))[::-1].astype(float)
for (lab, o, lo, hi, col, mk, filled), y in zip(rows, ys):
if lo is not None:
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt")
txt = f"{o:.3f} ({lo:.3f}–{hi:.3f})" if col == BLU else f"{o:.2f} ({lo:.2f}–{hi:.2f})"
else:
txt = f"{o:.3f} (point only)"
ax.plot(o, y, marker=mk, ms=9, color=col, mfc=col if filled else BG, mew=2, lw=0)
ax.text(o, y + 0.33, txt, va="center", ha="center", fontsize=10, color=col)
ax.axvline(1.0, color=SEC, lw=0.9)
ax.set_xscale("log")
ax.set_xlim(0.80, 1.30)
ticks = [0.8, 0.9, 1.0, 1.1, 1.2, 1.3]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:.1f}" for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.tick_params(axis="y", length=0)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.set_xlabel("Odds ratio for referral per 1-point higher MELD score (log scale)")
ax.text(0.985, ys[0] + 0.8, "← lower odds", ha="right", va="center", fontsize=10, color=SEC)
ax.text(1.015, ys[0] + 0.8, "higher odds →", ha="left", va="center", fontsize=10, color=SEC)
ax.set_ylim(-0.5, ys[0] + 1.1)
leg = [Line2D([], [], color=ORA, lw=2, marker="o", ms=8, label="Reported in the paper"),
Line2D([], [], color=BLU, lw=2, marker="s", ms=8, label="Recomputed from the paper's numbers")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.35, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nFigures written: fig1.svg/png, fig2.svg/png")
Abstract reports P < 0.0001 for 3/170 vs 3/172 serious adverse events; the table gives P > 0.999
Nat Med (2025) · doi:10.1038/s41591-025-03674-y · PMC12092238
Summary
The abstract of this randomized trial (SUPPRESS-AF) says that procedure-related serious adverse events did not differ between PVI plus low-voltage-area ablation and PVI alone, and gives "1.7% versus 1.8%, P < 0.0001". The paper's own Extended Data Table 4 gives the counts behind those percentages, 3 of 170 and 3 of 172, and prints P > 0.999 for the same comparison with the test the Methods name (two-sided Fisher's exact test). Recomputed from those counts, Fisher's exact test gives P = 1.000. With six events in total, no split between the arms could give a two-sided Fisher P below 0.014. The words "no difference" are correct and the table is correct. The P value in the abstract is not, and a reader of the abstract alone sees a highly significant safety difference that the data do not show.
What the paper reports
Abstract:
There was no difference in the procedure-related serious adverse events between the two groups (1.7% versus 1.8%, P < 0.0001).
Extended Data Table 4, "Safety endpoints". This table is an image in the archived article; the text below was read from the publisher's image file (41591_2025_3674_Tab4_ESM.jpg). Columns: PVI+LVA-ABL n=170 | PVI-alone n=172 | p:
Procedure-related serious adverse events, n (%) | 3 (1.7) | 3 (1.8) | >0.999
Extended Data Table 4, footnote:
p for comparison between groups using the two-sided Fisher's exact test.
Methods, "Statistical analysis":
Safety endpoints were analyzed using Fisher's exact test. All tests were conducted at a two-sided α level of 0.05.
Results, "Safety endpoints":
There was no difference in procedure-related serious adverse events between the two groups. Case presentations of procedure-related serious adverse events are shown in Extended Data Table 6.
Discussion, fourth paragraph:
Serious, procedure-related, adverse event rates were not increased with the addition of LVA ablation.
Results, "Patients and follow-up" (the safety population):
A total of 342 patients were randomly assigned to either the LVA-ABL group (170 patients) or the PVI-alone group (172 patients) and safety outcomes were assessed.
Extended Data Table 6 (an image, read from 41591_2025_3674_Tab6_ESM.jpg) describes six cases: in PVI-alone, one cardiac tamponade, one stroke and one major bleeding; in PVI+LVA-ABL, one esophageal fistula and two major bleedings.
What the paper's own numbers show
Method. I took the counts and denominators from Extended Data Table 4 and checked them against the case list in Extended Data Table 6. I ran a two-sided Fisher's exact test on the 2×2 table [[3, 167], [3, 169]], the test that the Methods and the table footnote name. As robustness checks I also ran one-sided Fisher tests, chi-square tests with and without continuity correction, and Fisher's test with the efficacy-population denominator (171) in place of 172. Because Fisher's test conditions on the total number of events, I listed every possible split of the six events and its P value, to find the smallest P these margins allow. I also found how many events the LVA arm would need, against 3/172, for P < 0.0001. Finally I recomputed the other rows of Extended Data Table 4 with the same test to confirm that the method reproduces the paper's printed values. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Serious AEs, PVI+LVA-ABL | Abstract; Ext. Data Table 4 | 3 (1.7%) of 170 | 3/170 = 1.76% |
| Serious AEs, PVI-alone | Abstract; Ext. Data Table 4 | 3 (1.8%) of 172 | 3/172 = 1.74% (3/171 = 1.75%) |
| Case count | Ext. Data Table 6 | 6 cases | 3 PVI+LVA-ABL, 3 PVI-alone |
| P, serious AEs | Abstract | P < 0.0001 | Fisher two-sided P = 1.000 |
| P, serious AEs | Ext. Data Table 4 | >0.999 | Fisher two-sided P = 1.000 (odds ratio 1.012) |
| P with denominator 171 | robustness | – | 1.000 |
| One-sided Fisher P | robustness | – | 0.652 (greater) / 0.663 (less) |
| Chi-square P | robustness | – | 1.000 (Yates); 0.988 (uncorrected) |
| Risk difference | robustness | – | 0.02 percentage points (95% CI −2.76 to 2.80) |
| Lowest two-sided Fisher P over all splits of 6 events | robustness | – | 0.014 (all six in PVI-alone); 0.030 (all six in PVI+LVA-ABL) |
| LVA-arm events needed for P < 0.0001 vs 3/172 | robustness | – | 21/170 (12.4%), P = 8.7×10⁻⁵ |
| Bleeding events, 6 vs 2 | Ext. Data Table 4 | P = 0.173 | P = 0.1727 |
| Death from any cause, 2 vs 2 | Ext. Data Table 4 | >0.999 | P = 1.000 |
| Periprocedural AEs, 11 vs 4 | Ext. Data Table 4; Discussion | P = 0.069; OR 2.91 (0.91–9.31) | P = 0.0692; OR 2.91 (Woolf CI 0.91–9.31) |
| Symptomatic stroke, 4 vs 2 | Ext. Data Table 4 | 2 (1.7%), P = 0.723 | 2/172 = 1.16%, P = 0.447; with 3/172: 1.74%, P = 0.723 |
The same procedure reproduces the printed P values for bleeding, death and periprocedural events, and the table's own >0.999 for serious adverse events. Only the abstract's value disagrees.
Two smaller points, which do not bear on the main finding. First, 3/170 = 1.76% and 3/172 = 1.74%, so ordinary rounding would give 1.8% and 1.7%; the printed 1.7% and 1.8% appear in swapped order, in both the abstract and the table. Second, in the symptomatic stroke row the PVI-alone entry "2 (1.7)" is internally inconsistent: 2/172 is 1.2%, and Fisher's test on 4 vs 2 gives P = 0.447, whereas 3/172 gives both the printed 1.7% and the printed P = 0.723. The count may be 3 rather than 2; the paper does not say which entry is correct.
Figure
Figure 1. The two places the paper reports the comparison of procedure-related serious adverse events. The left panel quotes the abstract and the right panel reproduces the relevant row of Extended Data Table 4: the same percentages, the same counts and a footnote naming the two-sided Fisher exact test. The printed P values (orange) are P < 0.0001 and P > 0.999. The blue box gives the two-sided Fisher exact P recomputed from the table's counts, 3/170 vs 3/172, which is 1.000 and agrees with the table.
Figure 2. Given the arm sizes (170 and 172) and six events in total, these are the seven possible splits of the events. Bar heights are the probability of each split if the arms do not differ (hypergeometric distribution), and each bar is labelled with its two-sided Fisher exact P. The reported split, 3 vs 3, is the most likely split and has P = 1.000. The most extreme splits give P = 0.030 and 0.014, so no arrangement of six events in these arms could yield the P < 0.0001 printed in the abstract (orange text). All values are computed in recompute.py.txt.
Why it matters
The abstract is the part of the paper most readers see, and it is what indexing services and summaries reproduce. Read alone, "P < 0.0001" signals a highly significant difference in serious procedural harm between the two ablation strategies, the opposite of what the counts show and of what the same sentence says in words. A reader who notices the clash cannot tell from the abstract which part is wrong.
The effect on the paper's conclusions is small. The Results, Discussion and Extended Data Table 4 all report no difference, the table's P value is correct, and the primary efficacy result (61% vs 50% freedom from AF or AT, log-rank P = 0.127) does not depend on this value. The correct P (about 1.0) leaves the paper's safety reading unchanged: adding LVA ablation did not increase procedure-related serious adverse events in this trial. The trial is small for this endpoint, however (six events in total, risk-difference 95% CI about −2.8 to +2.8 percentage points), and the Discussion itself notes a tendency toward more periprocedural adverse events with LVA ablation (11 vs 4, P = 0.069). That point is separate and is not affected by this error.
Explanations considered
- Text conversion or typesetting artifact. Not supported. The same "P < 0.0001" appears in the publisher's PDF (PMC open-access copy, created by Springer on 2025-05-16) and in the Europe PMC full-text XML, where the "<" is an explicit
<entity between thin spaces. It is not a lost ">" sign read from an image, a mis-set superscript or a shifted table cell. - A different population, time point or event definition. Not supported. The abstract's percentages (1.7% and 1.8%) are exactly those printed in Extended Data Table 4 for procedure-related serious adverse events, and the case list in Extended Data Table 6 gives the same 3 vs 3. Using the efficacy denominator (171) instead of 172 also gives P = 1.000.
- A different test. Not supported. The Methods state that safety endpoints were analyzed with Fisher's exact test at a two-sided α of 0.05, and the table footnote says the same. One-sided Fisher tests (0.652, 0.663) and chi-square tests (0.988, 1.000) give values near 1. With these margins no split of the six events yields a two-sided Fisher P below 0.014.
- A non-inferiority or equivalence test. Not supported. A one-sided test against a margin could produce a small P for two near-identical rates (for example, a hypothetical margin of 5 percentage points would give P ≈ 2.3×10⁻⁴, and 3 points P ≈ 0.018), but the paper describes no non-inferiority or equivalence analysis and no safety margin, the trial was designed for superiority on the primary efficacy endpoint, and the sentence frames the result as "no difference". No margin is stated that would reproduce exactly "P < 0.0001".
- A correct value of a different comparison. No other comparison in the archived text is reported as P < 0.0001 (the smallest printed values, for procedural times and energy in Table 2, are given as <0.001), so there is no nearby value the sentence could have taken by mistake that can be identified. The cause of the discrepancy cannot be determined from the paper; what can be determined is that the printed value is incompatible with the printed counts and stated test.
Status
As of 2026-10-23:
- Version of record. The error is in the version of record. The nature.com article page could not be fetched automatically (it redirected to a login page), so I checked the publisher's PDF from the PMC open-access collection (PMC12092238.1.pdf, metadata creator "Springer", subject "Nature Medicine, doi:10.1038/s41591-025-03674-y") and the Europe PMC full-text XML. Both contain "(1.7% versus 1.8%, P < 0.0001)".
- Supplements. Extended Data Tables 4 (safety endpoints), 5 (periprocedural adverse events) and 6 (case descriptions of procedure-related serious adverse events) were read from the publisher's image files on media.springernature.com; the PMC text has captions only for these tables. Extended Data Tables 4 and 6 resolve the question: they give 3 vs 3 events and P > 0.999. The Supplementary Information PDF (protocol and statistical analysis plan) could not be downloaded in a readable form; it is not needed to settle the discrepancy.
- Corrections and comments. No correction, erratum, retraction or expression of concern was found. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction or comment notices. A web search for an author correction to this article returned none.
- PubPeer. The PubPeer API returned no publication record for the DOI, and the PubPeer search page refused automated access (HTTP 403), so PubPeer could not be checked directly. A web search for the title together with "PubPeer" found no thread. Absence of a PubPeer discussion is therefore not fully confirmed.
- Sources. PMC article PMC12092238; publisher PDF (PMC open-access copy); Europe PMC full-text XML; Extended Data Table images 41591_2025_3674_Tab4_ESM.jpg, Tab5_ESM.jpg and Tab6_ESM.jpg (media.springernature.com); Crossref; Europe PMC; PubPeer.
Suggested correction
- In the abstract, replace "(1.7% versus 1.8%, P < 0.0001)" with the value in Extended Data Table 4, for example "(3/170 (1.8%) versus 3/172 (1.7%), P > 0.999)".
- Check the order of the percentages for this row in the abstract and in Extended Data Table 4: 3/170 rounds to 1.8% and 3/172 to 1.7%.
- Check the symptomatic stroke row of Extended Data Table 4: the PVI-alone entry "2 (1.7)" with P = 0.723 is consistent with 3 events, not 2. Correct either the count or the percentage and P value.
- Optional: in the Discussion, the periprocedural adverse event percentages are labelled "95% CI" ("11 (95% CI = 6.5%) versus 4 (95% CI = 2.3%) patients"); they are proportions, not confidence intervals.
Recomputation script (python recompute.py.txt)
"""Recompute the procedure-related serious adverse event P value of PMC12092238
(Nat Med 2025, doi:10.1038/s41591-025-03674-y, SUPPRESS-AF).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes, next to this file:
fig1.svg/png (abstract vs Extended Data Table 4, text-panel diagram)
fig2.svg/png (all possible splits of the six events and their Fisher P values)
candidates/cand_A..cand_E.svg/png (five candidate designs for review; A = fig1, E = fig2)
Inputs are copied from the paper:
Abstract: "(1.7% versus 1.8%, P < 0.0001)"
Extended Data Table 4 (image 41591_2025_3674_Tab4_ESM.jpg), safety population
PVI+LVA-ABL n = 170, PVI-alone n = 172, P by two-sided Fisher's exact test
Extended Data Table 6: six case descriptions (3 PVI+LVA-ABL, 3 PVI-alone)
Discussion: OR = 2.91 (95% CI 0.91-9.31) for periprocedural adverse events
"""
import math
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs (from the paper)
N_LVA, N_PVI = 170, 172 # Extended Data Table 4 headers (safety population)
N_PVI_EFFICACY = 171 # abstract / efficacy population
SAE = (3, 3) # Procedure-related serious AEs, Extended Data Table 4
SAE_PRINTED_PCT = (1.7, 1.8) # Abstract and Extended Data Table 4
ABSTRACT_P = "P < 0.0001"
TABLE_P = ">0.999"
# Other Extended Data Table 4 rows: (a, b, printed % a, printed % b, printed P)
ROWS = {
"Bleeding events": (6, 2, 3.5, 1.2, "0.173"),
"Symptomatic stroke": (4, 2, 2.4, 1.7, "0.723"),
"Death from any cause": (2, 2, 1.2, 1.2, ">0.999"),
"Periprocedural adverse events": (11, 4, 6.5, 2.3, "0.069"),
"Procedure-related serious AEs": (3, 3, 1.7, 1.8, ">0.999"),
}
# Extended Data Table 6 case list
TABLE6 = [("PVI-alone", "Cardiac tamponade"), ("PVI-alone", "Stroke"),
("PVI+LVA-ABL", "Esophageal fistula"), ("PVI+LVA-ABL", "Major bleeding"),
("PVI-alone", "Major bleeding"), ("PVI+LVA-ABL", "Major bleeding")]
# ---------------------------------------------------------------- style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "lines.linewidth": 1.5,
})
LW = 1.5 # 1.5 pt = 2 px at 96 px/in
MS = 8 # marker size in points (>= 8 px)
def fisher2(a, n1, b, n2, alt="two-sided"):
return stats.fisher_exact([[a, n1 - a], [b, n2 - b]], alternative=alt)[1]
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def cp_ci(k, n, a=0.05):
lo = stats.beta.ppf(a / 2, k, n - k + 1) if k > 0 else 0.0
hi = stats.beta.ppf(1 - a / 2, k + 1, n - k) if k < n else 1.0
return lo, hi
# ---------------------------------------------------------------- 1. the serious-AE comparison
a, b = SAE
print("=== 1. Procedure-related serious AEs, Extended Data Table 4 ===")
print(f"PVI+LVA-ABL {a}/{N_LVA} = {100*a/N_LVA:.2f}% (printed {SAE_PRINTED_PCT[0]})")
print(f"PVI-alone {b}/{N_PVI} = {100*b/N_PVI:.2f}% (printed {SAE_PRINTED_PCT[1]})")
print(f"PVI-alone with efficacy denominator {b}/{N_PVI_EFFICACY} = {100*b/N_PVI_EFFICACY:.2f}%")
t = np.array([[a, N_LVA - a], [b, N_PVI - b]])
or_, p2 = stats.fisher_exact(t)
P_FISHER = p2
print(f"Fisher exact two-sided P = {p2:.4f} (odds ratio {or_:.3f}) reported: abstract {ABSTRACT_P}, table {TABLE_P}")
print(f"Fisher exact two-sided P with n = 171 = {fisher2(a, N_LVA, b, N_PVI_EFFICACY):.4f}")
P_GT = fisher2(a, N_LVA, b, N_PVI, "greater")
P_LT = fisher2(a, N_LVA, b, N_PVI, "less")
print(f"Fisher one-sided P: greater = {P_GT:.3f}, less = {P_LT:.3f}")
P_YATES = stats.chi2_contingency(t)[1]
P_CHI = stats.chi2_contingency(t, correction=False)[1]
print(f"chi-square P: Yates = {P_YATES:.3f}, uncorrected = {P_CHI:.4f}")
pA, pB = a / N_LVA, b / N_PVI
d = pA - pB
se = math.sqrt(pA * (1 - pA) / N_LVA + pB * (1 - pB) / N_PVI)
print(f"risk difference = {100*d:.3f} pp, Wald 95% CI {100*(d-1.96*se):.2f} to {100*(d+1.96*se):.2f} pp")
for m in (0.03, 0.05): # hypothetical non-inferiority margins (the paper describes none)
print(f"hypothetical one-sided non-inferiority test, margin {100*m:.0f} pp: P = {stats.norm.sf((m-d)/se):.1e}")
CI_LVA, CI_PVI = cp_ci(a, N_LVA), cp_ci(b, N_PVI)
print(f"Clopper-Pearson 95% CI: LVA {100*CI_LVA[0]:.2f}-{100*CI_LVA[1]:.2f}%, PVI {100*CI_PVI[0]:.2f}-{100*CI_PVI[1]:.2f}%")
cnt = {"PVI+LVA-ABL": 0, "PVI-alone": 0}
for g, _ in TABLE6:
cnt[g] += 1
print(f"Extended Data Table 6 case count: {cnt}")
# ---------------------------------------------------------------- 2. every split of the six events
print("\n=== 2. Every possible split of the six events (margins fixed: 170, 172, 6) ===")
TOT = a + b
hg = stats.hypergeom(N_LVA + N_PVI, TOT, N_LVA) # population, successes, draws
SPLIT = []
for k in range(TOT + 1):
pk = hg.pmf(k)
pf = fisher2(k, N_LVA, TOT - k, N_PVI)
SPLIT.append((k, pk, pf))
print(f" {k} vs {TOT-k}: probability under no difference = {pk:.4f}, two-sided Fisher P = {pf:.4f}")
PMIN = min(s[2] for s in SPLIT)
print(f"smallest attainable two-sided Fisher P with six events = {PMIN:.4f}")
# ---------------------------------------------------------------- 3. what would P < 0.0001 require
print("\n=== 3. Events needed in the LVA arm (vs 3/172) for two-sided Fisher P < 0.0001 ===")
CURVE = [(x, fisher2(x, N_LVA, b, N_PVI)) for x in range(0, 31)]
X_NEED = next(x for x, p in CURVE if x > b and p < 1e-4)
print(f" {X_NEED}/{N_LVA} = {100*X_NEED/N_LVA:.1f}% (P = {dict(CURVE)[X_NEED]:.2e}); "
f"{X_NEED-1}/{N_LVA}: P = {dict(CURVE)[X_NEED-1]:.2e}")
X05 = next(x for x, p in CURVE if x > b and p < 0.05)
print(f" first count with P < 0.05: {X05}/{N_LVA} (P = {dict(CURVE)[X05]:.4f})")
# ---------------------------------------------------------------- 4. other rows of Extended Data Table 4
print("\n=== 4. Other rows of Extended Data Table 4 ===")
for name, (x, y, ra, rb, rp) in ROWS.items():
print(f" {name:31s} {x}/{N_LVA} = {100*x/N_LVA:.2f}% (printed {ra}); "
f"{y}/{N_PVI} = {100*y/N_PVI:.2f}% (printed {rb}); Fisher P = {fisher2(x, N_LVA, y, N_PVI):.4f} (printed {rp})")
print(f" stroke row if PVI-alone count were 3: 3/172 = {300/172:.2f}%, Fisher P (4 vs 3) = {fisher2(4, N_LVA, 3, N_PVI):.3f}")
o = (11 * (N_PVI - 4)) / ((N_LVA - 11) * 4)
s = math.sqrt(1/11 + 1/(N_LVA-11) + 1/4 + 1/(N_PVI-4))
print(f" periprocedural AEs OR = {o:.2f}, Woolf 95% CI {math.exp(math.log(o)-1.96*s):.2f}-{math.exp(math.log(o)+1.96*s):.2f} (Discussion: 2.91, 0.91-9.31)")
# ================================================================= figures
def panel(ax, x, y, w, h, title, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc="white", ec=edge, lw=LW, transform=ax.transAxes))
ax.text(x + 0.02, y + h - 0.045, title, transform=ax.transAxes, fontsize=11,
fontweight="bold", va="top", color=INK)
def fig_A(path):
"""Text panels: abstract vs Extended Data Table 4, with recomputed value."""
fig = plt.figure(figsize=(7.2, 4.5))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
r = fig.canvas.get_renderer()
# left: abstract
panel(ax, 0.03, 0.42, 0.44, 0.54, "Abstract", INK2)
ax.text(0.05, 0.82, "\u201cThere was no difference in the\nprocedure-related serious adverse\n"
"events between the two groups\n(1.7% versus 1.8%,", transform=ax.transAxes,
fontsize=10.5, va="top", color=INK, linespacing=1.35)
tp = ax.text(0.05, 0.575, "P < 0.0001", transform=ax.transAxes, fontsize=13, fontweight="bold",
color=ORANGE, va="center")
bb = tp.get_window_extent(r).transformed(ax.transAxes.inverted())
ax.text(bb.x1 + 0.003, 0.575, ").\u201d", transform=ax.transAxes, fontsize=10.5, va="center")
# right: Extended Data Table 4
panel(ax, 0.53, 0.42, 0.44, 0.54, "Extended Data Table 4", INK2)
ax.text(0.55, 0.82, "Procedure-related serious\nadverse events, n (%)", transform=ax.transAxes,
fontsize=10.5, va="top", linespacing=1.35)
for xx, lab, val in [(0.55, "PVI+LVA-ABL\nn = 170", "3 (1.7)"), (0.705, "PVI-alone\nn = 172", "3 (1.8)")]:
ax.text(xx, 0.68, lab, transform=ax.transAxes, fontsize=10, color=INK2, va="top")
ax.text(xx, 0.565, val, transform=ax.transAxes, fontsize=10.5, va="center")
ax.text(0.855, 0.68, "P", transform=ax.transAxes, fontsize=10, color=INK2, va="top")
ax.text(0.855, 0.565, ">0.999", transform=ax.transAxes, fontsize=13, fontweight="bold",
color=ORANGE, va="center")
ax.text(0.55, 0.465, "footnote: two-sided Fisher\u2019s exact test", transform=ax.transAxes,
fontsize=10, color=INK2, va="center")
# connector between the two P values, below the panels
arr = FancyArrowPatch((0.13, 0.415), (0.90, 0.415), transform=ax.transAxes,
connectionstyle="arc3,rad=0.16", arrowstyle="<->", mutation_scale=12,
lw=LW, color=INK2)
ax.add_patch(arr)
ax.text(0.5, 0.275, "Conflict: same counts and same comparison, but P < 0.0001 vs P > 0.999",
transform=ax.transAxes, ha="center", fontsize=10.5, color=INK, fontweight="bold")
# recomputed
ax.add_patch(FancyBboxPatch((0.08, 0.075), 0.84, 0.15, boxstyle="round,pad=0,rounding_size=0.015",
fc="white", ec=BLUE, lw=LW, transform=ax.transAxes))
ax.text(0.5, 0.185, "Recomputed from the table\u2019s counts", transform=ax.transAxes,
ha="center", fontsize=10, color=INK2, va="center")
ax.text(0.5, 0.118, f"Fisher\u2019s exact test, 3/170 vs 3/172: two-sided P = {P_FISHER:.3f}",
transform=ax.transAxes, ha="center", fontsize=11, color=BLUE, fontweight="bold", va="center")
ax.text(0.5, 0.025, "Orange: values printed in the paper. Blue: recomputed from the paper\u2019s counts.",
transform=ax.transAxes, ha="center", fontsize=10, color=INK2)
save(fig, path)
def fig_B(path):
"""P values on one log axis."""
rows = [
("Abstract (printed)", 1e-4, ORANGE, "lt", "P < 0.0001"),
("Ext. Data Table 4 (printed)", 0.999, ORANGE, "gt", "P > 0.999"),
("Fisher, two-sided (stated test)", P_FISHER, BLUE, "", f"{P_FISHER:.3f}"),
("Chi-square, uncorrected", P_CHI, BLUE, "", f"{P_CHI:.3f}"),
("Fisher, one-sided (each direction)", min(P_GT, P_LT), BLUE, "", f"{P_GT:.3f} / {P_LT:.3f}"),
("Lowest P any split of 6 events allows", PMIN, BLUE, "", f"{PMIN:.3f}"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.6))
ys = np.arange(len(rows))[::-1]
for y, (lab, v, c, kind, txt) in zip(ys, rows):
mk = "o" if c == BLUE else "D"
if lab.startswith("Fisher, one-sided"):
ax.plot([P_LT, P_GT], [y, y], color=c, lw=LW)
ax.plot([P_GT, P_LT], [y, y], mk, color=c, ms=MS, mec="white", mew=1)
else:
ax.plot(v, y, mk, color=c, ms=MS, mec="white", mew=1)
if kind == "lt":
ax.annotate("", xy=(1.6e-6, y), xytext=(1e-4, y),
arrowprops=dict(arrowstyle="->", color=c, lw=LW))
ax.text(1.03, y, txt, color=INK, fontsize=10, ha="left", va="center",
transform=ax.get_yaxis_transform())
ax.axvline(0.05, color=INK2, lw=0.8, ls=(0, (3, 3)))
ax.text(0.05 * 1.12, len(rows) - 0.45, "α = 0.05", fontsize=10, color=INK2)
ax.set_xscale("log"); ax.set_xlim(1e-6, 2.2); ax.set_ylim(-0.6, len(rows) - 0.2)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xlabel("P value (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h1, = ax.plot([], [], "D", color=ORANGE, ms=MS, label="printed in the paper")
h2, = ax.plot([], [], "o", color=BLUE, ms=MS, label="recomputed from the paper\u2019s counts")
fig.legend(handles=[h1, h2], loc="upper center", ncol=2, frameon=False, fontsize=10)
fig.tight_layout(rect=(0, 0, 0.87, 0.92))
save(fig, path)
def fig_C(path):
"""Fisher P as a function of the LVA-arm count."""
xs = np.array([c[0] for c in CURVE]); ps = np.array([c[1] for c in CURVE])
fig, ax = plt.subplots(figsize=(7.2, 3.8))
ax.plot(xs, ps, color=BLUE, lw=LW, marker="o", ms=5, mec="white", mew=0.6,
label="two-sided Fisher P, k/170 vs 3/172 (recomputed)")
ax.axhline(0.05, color=INK2, lw=0.8, ls=(0, (3, 3)))
ax.text(30.3, 0.05, "0.05", va="center", fontsize=10, color=INK2)
ax.axhline(1e-4, color=ORANGE, lw=LW, ls=(0, (5, 2)), label="abstract: P < 0.0001 (printed)")
ax.text(30.3, 1e-4, "0.0001", va="center", fontsize=10, color=INK2)
ax.plot(3, P_FISHER, "o", color=BLUE, ms=11, mec=INK, mew=1.2, zorder=5)
ax.annotate(f"reported counts 3 vs 3:\nP = {P_FISHER:.3f}", (3, P_FISHER), xytext=(0.6, 2e-3),
fontsize=10, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
pn = dict(CURVE)[X_NEED]
ax.plot(X_NEED, pn, "o", color=BLUE, ms=11, mec=INK, mew=1.2, zorder=5)
ax.annotate(f"{X_NEED}/170 ({100*X_NEED/N_LVA:.1f}%) would be\nneeded for P < 0.0001",
(X_NEED, pn), xytext=(22.5, 3e-3), fontsize=10,
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.set_yscale("log"); ax.set_ylim(1e-8, 2.5); ax.set_xlim(-0.5, 30)
ax.set_xlabel("Events in the PVI+LVA-ABL arm (of 170), PVI-alone fixed at 3 of 172")
ax.set_ylabel("Two-sided P (log scale)")
ax.grid(color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.legend(loc="lower left", frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
def fig_D(path):
"""Per-arm event rates with exact 95% CIs."""
fig, ax = plt.subplots(figsize=(7.2, 2.9))
arms = [("PVI+LVA-ABL (3/170)", a / N_LVA, CI_LVA, SAE_PRINTED_PCT[0]),
("PVI-alone (3/172)", b / N_PVI, CI_PVI, SAE_PRINTED_PCT[1])]
for y, (lab, p, ci, pr) in zip([1, 0], arms):
ax.plot([100 * ci[0], 100 * ci[1]], [y, y], color=BLUE, lw=LW)
ax.plot(100 * p, y, "o", color=BLUE, ms=MS, mec="white", mew=1)
ax.plot(pr, y + 0.22, "D", color=ORANGE, ms=MS, mec="white", mew=1)
ax.text(100 * ci[1] + 0.15, y, f"{100*p:.2f}% (95% CI {100*ci[0]:.2f}–{100*ci[1]:.2f})",
va="center", fontsize=10)
ax.text(pr + 0.15, y + 0.22, f"printed {pr}%", va="center", fontsize=10, color=INK2)
ax.set_yticks([1, 0]); ax.set_yticklabels([r[0] for r in arms])
ax.set_xlim(0, 9.5); ax.set_ylim(-1.0, 1.6)
ax.set_xlabel("Procedure-related serious adverse events, % of patients")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h1, = ax.plot([], [], "D", color=ORANGE, ms=MS, label="printed in the paper")
h2, = ax.plot([], [], "o-", color=BLUE, ms=MS, lw=LW, label="recomputed, exact 95% CI")
ax.legend(handles=[h1, h2], loc="lower right", ncol=2, frameon=False, fontsize=10)
ax.set_title(f"Abstract prints P < 0.0001; these counts give Fisher P = {P_FISHER:.3f}",
fontsize=11, loc="left")
fig.tight_layout()
save(fig, path)
def fig_E(path):
"""All seven possible splits of six events, with their two-sided Fisher P."""
ks = [s[0] for s in SPLIT]; pk = [s[1] for s in SPLIT]; pf = [s[2] for s in SPLIT]
fig, ax = plt.subplots(figsize=(7.2, 3.8))
cols = [BLUE if k == a else "#bcd3ee" for k in ks]
ax.bar(ks, pk, width=0.72, color=cols, edgecolor=BG, lw=2)
for k, p, f in zip(ks, pk, pf):
ax.text(k, p + 0.008, f"P = {f:.3f}", ha="center", fontsize=10, color=INK)
ax.text(k, -0.045, f"{k} vs {TOT-k}", ha="center", fontsize=10, color=INK2)
ax.text(a, pk[a] + 0.035, "reported split", ha="center", fontsize=10, color=BLUE, fontweight="bold")
ax.text(-0.5, 0.35, "Abstract: P < 0.0001 (printed)", ha="left", va="top", fontsize=10.5,
color=ORANGE, fontweight="bold")
ax.text(-0.5, 0.315, f"No split of six events can give a\ntwo-sided Fisher P below {PMIN:.3f}",
ha="left", va="top", fontsize=10, color=INK)
ax.set_xticks([]); ax.set_ylim(-0.07, 0.37); ax.set_yticks(np.arange(0, 0.36, 0.05)); ax.set_xlim(-0.6, 6.6)
ax.axhline(0, color=INK2, lw=0.8)
ax.set_ylabel("Probability if the arms do not differ")
ax.set_xlabel("Split of the six events (PVI+LVA-ABL vs PVI-alone); bar labels: two-sided Fisher P", labelpad=4)
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.spines["left"].set_bounds(0, 0.35)
for s in ("top", "right", "bottom"):
ax.spines[s].set_visible(False)
fig.tight_layout()
save(fig, path)
for name, fn in [("cand_A", fig_A), ("cand_B", fig_B), ("cand_C", fig_C), ("cand_D", fig_D), ("cand_E", fig_E)]:
fn(os.path.join(CAND, name))
fig_A(os.path.join(HERE, "fig1"))
fig_E(os.path.join(HERE, "fig2"))
print("\nwrote fig1, fig2 and candidates/cand_A..E (.svg, .png) in", HERE)
Abstract reports 389 animals; the Methods list 116, and 389 sums per-context subject counts
eLife (2023) · doi:10.7554/eLife.87008 · PMC10547472
Summary
The abstract says facial behavior was coded "in 389 animals". The Methods list the study subjects by species and sex, and they add up to 116 adults (32 rhesus, 46 Barbary, 38 crested macaques). Table 2 agrees with the Methods: no per-context subject count is larger than the Methods headcount for its species. The figure 389 is exactly the sum of Table 2's "N subjects" column over all twelve species-by-context rows, so it counts each animal once for every context it was coded in. The study's analyses do not use this number and its conclusions are unaffected, but anyone who reads only the abstract will take the sample to be 3.35 times as large as it is.
What the paper reports
Abstract:
We coded facial behavior in >3000 social interactions across three social contexts (aggressive, submissive, affiliative) in 389 animals, using the Facial Action Coding System for macaques (MaqFACS).
Methods, "Study subjects and data collection", first paragraph:
Behavioral data and video recordings were collected on one adult male and 31 adult female rhesus macaques (M. mulatta), on 18 adult male and 28 adult female Barbary macaques (M. sylvanus), and 17 adult male and 21 adult female crested macaques (M. nigra).
Methods, "Facial behavior and social context coding":
We coded facial behavior of adult individuals but included their interactions with any other group member regardless of age or sex.
Table 2 shows the number of social interactions per species and context from which FACS codes were made.
Table 2, "Total number of social interactions per species and social context that were MaqFACS coded":
Species Context N interactions N subjects Duration (s) Rhesus Affiliative 193 29 1197 Rhesus Aggressive 413 32 2050 Rhesus Submissive 318 31 1262 Rhesus Unclear 121 30 802 Barbary Affiliative 683 43 4897 Barbary Aggressive 585 44 2128 Barbary Submissive 529 34 1890 Barbary Unclear 603 45 3500 Crested Affiliative 241 35 1918 Crested Aggressive 62 23 284 Crested Submissive 25 18 115 Crested Unclear 107 25 684
Discussion, on the comparison that limits inference:
Finally, effectively, our comparison is limited to three species which is a small sample.
What the paper's own numbers show
Method. I added the Methods counts by species and sex to get the number of distinct adults. For each species I summed the four Table 2 "N subjects" cells and found the largest of them. I then compared the sums with the abstract's 389, and the largest cell with the Methods headcount. As a separate check I summed the Table 2 interaction counts against the abstract's ">3000". Last, I counted animals in the authors' public data file (the file the paper's data availability statement points to): distinct animals per species, and the number of contexts each animal was coded in. Every number below is printed by recompute.py.txt. The data-file rows need the file itself: run python recompute.py.txt --data <path or URL>, where the URL is maqfacs.txt at the root of the GitHub repository the paper cites (Methods, "Specificity bias correction"), in raw form https://raw.githubusercontent.com/<owner>/macaque-facial-complexity/main/maqfacs.txt with <owner> taken from that citation. Its sha256 is printed and quoted under Status.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Animals in the study | Abstract | 389 animals | 116 distinct adults (Methods) |
| Rhesus adults | Methods | 1 male + 31 female | 32 |
| Barbary adults | Methods | 18 male + 28 female | 46 |
| Crested adults | Methods | 17 male + 21 female | 38 |
| Total distinct adults | Methods | not stated | 32 + 46 + 38 = 116 |
| Rhesus, sum of N subjects over four contexts | Table 2 | 29, 32, 31, 30 | 122 (3.81 × the 32 adults) |
| Barbary, sum of N subjects over four contexts | Table 2 | 43, 44, 34, 45 | 166 (3.61 × the 46 adults) |
| Crested, sum of N subjects over four contexts | Table 2 | 35, 23, 18, 25 | 101 (2.66 × the 38 adults) |
| Sum of all 12 N-subjects cells | Table 2 | — | 122 + 166 + 101 = 389, identical to the abstract |
| Same sum without the Unclear rows | Table 2 | — | 289, so 389 is not a three-context total |
| Largest per-context N subjects, by species | Table 2 | 32 / 45 / 35 | at most the Methods headcount (32 / 46 / 38), as it must be |
| Abstract count relative to headcount | Abstract vs Methods | 389 | 389 / 116 = 3.35; 273 more animals than were studied |
| Interactions, three named contexts | Abstract vs Table 2 | >3000 | 924 + 1797 + 328 = 3049 (3880 with Unclear); consistent |
| Distinct animals in the authors' data file | Data file maqfacs.txt |
— | 32 rhesus, 46 Barbary, 40 crested = 118. Crested: 17 male, 22 female, 1 with no sex recorded (Methods: 17 male, 21 female). Rhesus and Barbary match the Methods by sex |
| Per-context distinct subjects in the data file | Data file | — | Reproduces all 12 Table 2 N-subjects cells and durations (to 0.5 s) once the file's "agonistic" label is merged into "unclear"; they sum to 389 |
| Contexts per animal in the data file | Data file | — | Rhesus: 27 animals in 4 contexts, 4 in 3, 1 in 2. Barbary: 29, 16, 1. Crested: 10 in 4, 13 in 3, 5 in 2, 12 in 1. Sum over animals of contexts = 389 |
The arithmetic leaves one reading. The abstract's 389 is the total of the "N subjects" column, a column that counts each animal once per context. Taken as a headcount, the paper's own Methods and Table 2 give 116 adults (118 animals in the published data file).
Figure
Figure 1. On the left is the abstract's sentence giving the sample as 389 animals (orange: the value the paper reports). On the right is the Methods list of adults by species and sex, which sums to 116 (blue: recomputed from the paper's own counts). The lower panel gives Table 2's four per-context "N subjects" values for each species. Each value is no larger than the Methods headcount for its species, which is consistent with each one counting distinct animals, and together they add up to 122 + 166 + 101 = 389. The authors' data file settles the reading: it reproduces all 12 cells, and the number of contexts per animal sums to 389 over 118 animals. The abstract's figure is therefore the number of subject-context pairs, not the number of animals. All values are taken from the paper or printed by recompute.py.txt.
Figure 2. For each species, the open circles are the four Table 2 per-context subject counts, stacked vertically (affiliative, aggressive, submissive, unclear, top to bottom) so that close values such as 29 to 32 do not hide one another. The blue vertical line is the Methods headcount (32, 46, 38) and the orange circle is the sum of the four context counts (122, 166, 101; together 389). Every context count lies at or below the headcount, which is consistent with each one counting distinct animals (the data-file check confirms it). Their sum is 2.7 to 3.8 times the headcount (3.81 rhesus, 3.61 Barbary, 2.66 crested) because most animals appear in more than one context. The axis starts at zero.
Why it matters
The error is in the abstract, the part of a paper that is read, indexed and copied most widely. A reader, reviewer or meta-analyst who takes the sample size from the abstract or from a database built from it will record N = 389. The correct figure is about 116 (118 in the published data file), so the study would look 3.35 times as large as it is. That matters most when studies are weighed or compared by number of individuals, and in particular for the rhesus sample: 32 animals from one breeding group, 31 of them female, a limitation the Methods acknowledge.
The inferential results are not affected. Entropy, context specificity and the random-forest classifier are computed from 500 ms time blocks within interactions, and the species comparison rests on three species, as the Discussion states. The headcount enters none of these calculations, so every reported estimate and its direction stand. The abstract's other count, ">3000 social interactions", is consistent with Table 2 (3049 in the three named contexts).
Explanations considered
- Interaction partners counted as animals. The Methods say partners could be "any other group member regardless of age or sex", so more than 116 individuals may appear in the videos. But the abstract says facial behavior was coded "in 389 animals", and the Methods say facial behavior was coded only for adults. The paper gives no group sizes or partner counts that add up to 389, so this does not account for the number.
- Another population or time point. Each species has one sample in the Methods: one rhesus breeding group, one Barbary group and two wild crested groups. Table 2's per-context subject counts never exceed the Methods headcounts. Nothing in the paper describes a larger sample.
- A three-context total. The abstract mentions three contexts. Without the Unclear rows the N-subjects sum is 289, not 389, so the figure is not a three-context total either. Only the full twelve-cell sum matches.
- A text-conversion or version artifact. The values are plain integers, and the PMC archive, Europe PMC, the eLife version of record and all five bioRxiv preprint versions carry the same abstract sentence and the same Methods counts. The authors' data file reproduces every Table 2 subject count and duration. So this is not an artifact of conversion or of one version.
- A small difference in the crested count. The data file has 40 distinct crested animals (17 male, 22 female, 1 with no sex recorded) and the Methods list 38 (17 male + 21 female). It does not matter here: on any version the headcount is 116 to 118, not 389.
None of these explains the number. Adding up the per-context subject counts reproduces it exactly.
Status
As of 2026-10-23:
- Version of record. I checked the eLife version of record (published 2023-10-03,
status: vorin the eLife article API). Its abstract has the same sentence ("… in 389 animals …"), and its Methods counts and Table 2 match the PMC archive. Europe PMC's full text and all five bioRxiv preprint versions (2022-12-07 to 2023-08-16) also say "389 animals". - Supplements. Supplementary file 1 lists the Action Units and Action Descriptors, and the other file is the MDAR reporting checklist. Neither gives subject counts. The public data file
maqfacs.txt(sha2566e68e6e1…a5f, 41 454 rows of 500 ms blocks) gives 118 distinct animals and reproduces Table 2. - Corrections. Crossref update metadata, PMC and Europe PMC list no correction, erratum, expression of concern or retraction (
src/known.py). - PubPeer. A web search found no PubPeer thread for the DOI or the title. PubPeer's own search refused automated access (HTTP 403, human verification required), so I could not query it directly.
- Sources. PMC10547472 (text and Table 2); api.elifesciences.org/articles/87008; Europe PMC full-text XML; bioRxiv doi:10.1101/2022.12.07.519469; Crossref.
Suggested correction
Abstract, change "in 389 animals" to "in 116 adult animals" (or the authors' deduplicated count, 118 if the data file's 40 crested individuals are the correct figure). If a per-context total is wanted, it could be stated as such, for example: "116 adults (389 subject-context combinations)". Reconciling the crested count, 38 in the Methods and 40 in the data file, would make the Methods and the data file agree.
Recomputation script (python recompute.py.txt)
"""Recompute the sample size stated in the abstract of PMC10547472
(eLife 2023;12:RP87008, doi:10.7554/eLife.87008).
Run: python recompute.py.txt (paper's own numbers + figures)
python recompute.py.txt --data <path or URL> (also check the authors' data file)
The data file is maqfacs.txt at the root of the GitHub repository whose URL the paper
gives in Methods, "Specificity bias correction" (https://github.com/<owner>/macaque-facial-complexity);
its raw URL is https://raw.githubusercontent.com/<owner>/macaque-facial-complexity/main/maqfacs.txt.
Needs numpy and matplotlib; pandas only for the optional data-file check.
Prints every recomputed number used in writeup.md and writes, next to this file:
fig1.svg/png, fig2.svg/png (figures used in writeup.md)
candidates/cand_A..cand_E.svg/png (five alternative designs for fig1)
Inputs are copied from the paper as archived in PubMed Central (identical in the
eLife version of record):
Abstract: ">3000 social interactions ... in 389 animals"
Methods, "Study subjects and data collection": adults per species and sex
Table 2: N interactions, N subjects and duration per species x context
Optional: the authors' data file maqfacs.txt (file at the root of the public GitHub
repository given in the paper's data availability statement; sha256 printed below),
used only to count distinct animals and contexts per animal. Candidate E needs it.
"""
import os
import sys
import urllib.request
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs (from the paper)
ABSTRACT_ANIMALS = 389
ABSTRACT_INTERACTIONS_FLOOR = 3000 # ">3000"
SPECIES = ("Rhesus", "Barbary", "Crested")
METHODS = {"Rhesus": (1, 31), "Barbary": (18, 28), "Crested": (17, 21)} # (male, female) adults
CONTEXTS = ("Affiliative", "Aggressive", "Submissive", "Unclear")
T2 = { # Table 2: (N interactions, N subjects, duration s)
"Rhesus": {"Affiliative": (193, 29, 1197), "Aggressive": (413, 32, 2050),
"Submissive": (318, 31, 1262), "Unclear": (121, 30, 802)},
"Barbary": {"Affiliative": (683, 43, 4897), "Aggressive": (585, 44, 2128),
"Submissive": (529, 34, 1890), "Unclear": (603, 45, 3500)},
"Crested": {"Affiliative": (241, 35, 1918), "Aggressive": (62, 23, 284),
"Submissive": (25, 18, 115), "Unclear": (107, 25, 684)},
}
# ---------------------------------------------------------------- style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, NEUTRAL = "#eb6834", "#2a78d6", "#b9b7b1"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "path", "lines.linewidth": 2,
})
W = 7.2
def save(fig, path_noext):
fig.savefig(path_noext + ".svg", bbox_inches="tight", pad_inches=0.12)
fig.savefig(path_noext + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
# ---------------------------------------------------------------- recomputation
print("== Methods: adults per species ==")
adults = {}
for sp in SPECIES:
m, f = METHODS[sp]
adults[sp] = m + f
print(f"{sp:8s} {m} male + {f} female = {adults[sp]}")
total_adults = sum(adults.values())
print(f"Total distinct adults (Methods): {total_adults}")
print("\n== Table 2: N subjects per context ==")
subj_sum, subj_max = {}, {}
for sp in SPECIES:
vals = [T2[sp][c][1] for c in CONTEXTS]
subj_sum[sp] = sum(vals)
subj_max[sp] = max(vals)
print(f"{sp:8s} {' + '.join(map(str, vals))} = {subj_sum[sp]}; max {subj_max[sp]}"
f" <= Methods {adults[sp]}: {subj_max[sp] <= adults[sp]};"
f" sum / adults = {subj_sum[sp] / adults[sp]:.2f}")
total_subj = sum(subj_sum.values())
print(f"Sum over all 12 species x context cells: "
f"{' + '.join(str(subj_sum[s]) for s in SPECIES)} = {total_subj}")
print(f"Equals the abstract's {ABSTRACT_ANIMALS}: {total_subj == ABSTRACT_ANIMALS}")
no_unclear = sum(T2[s][c][1] for s in SPECIES for c in CONTEXTS if c != "Unclear")
print(f"Sum excluding the Unclear rows: {no_unclear}")
print(f"Ratio {ABSTRACT_ANIMALS} / {total_adults} = {ABSTRACT_ANIMALS / total_adults:.2f}")
print(f"Overstatement: {ABSTRACT_ANIMALS - total_adults} animals")
print("\n== Table 2: interactions ==")
ni_all = sum(T2[s][c][0] for s in SPECIES for c in CONTEXTS)
ni_3 = sum(T2[s][c][0] for s in SPECIES for c in CONTEXTS if c != "Unclear")
for sp in SPECIES:
print(f"{sp:8s} three named contexts {sum(T2[sp][c][0] for c in CONTEXTS[:3])};"
f" all four {sum(T2[sp][c][0] for c in CONTEXTS)}")
print(f"Three named contexts: {ni_3} (> {ABSTRACT_INTERACTIONS_FLOOR}: "
f"{ni_3 > ABSTRACT_INTERACTIONS_FLOOR}); all four contexts: {ni_all}")
# ---------------------------------------------------------------- optional: authors' data file
data = None
DATA_SRC = None
if "--data" in sys.argv:
i = sys.argv.index("--data")
if i + 1 < len(sys.argv) and not sys.argv[i + 1].startswith("--"):
DATA_SRC = sys.argv[i + 1]
else:
print("\n[--data needs an argument: a path or URL to maqfacs.txt]")
if DATA_SRC is None:
print("\n[data-file check not run: pass --data <path or URL to maqfacs.txt>]")
else:
try:
import hashlib
import io
import pandas as pd
if DATA_SRC.startswith("http"):
raw = urllib.request.urlopen(DATA_SRC, timeout=60).read()
else:
with open(DATA_SRC, "rb") as fh:
raw = fh.read()
print("\n== Authors' data file maqfacs.txt ==")
print(f"sha256 {hashlib.sha256(raw).hexdigest()} ({len(raw)} bytes)")
df = pd.read_csv(io.BytesIO(raw), sep="\t")
print(f"rows (500 ms blocks): {len(df)}")
print("context labels:", {k: int(v) for k, v in df["context"].value_counts().items()})
# Table 2's 'Unclear' row = the file's 'unclear' + 'agonistic' labels
df["ctx"] = df["context"].replace({"agonistic": "unclear"})
df["species"] = df["species"].str.capitalize()
ids = df.drop_duplicates(["species", "subject"])
n_animals = ids.groupby("species").size()
sex = ids.groupby(["species", "sex"], dropna=False).size()
print("distinct animals per species:", {k: int(v) for k, v in n_animals.items()}, "total", int(n_animals.sum()))
print("by sex (NaN = sex missing):")
print(sex.to_string())
g = df.groupby(["species", "ctx"]).agg(nsub=("subject", "nunique"), blocks=("block", "size"))
g["dur_s"] = g["blocks"] / 2
print("per species x context: distinct subjects, duration (s) = blocks / 2")
ok = True
for sp in SPECIES:
for c in CONTEXTS:
r = g.loc[(sp, c.lower())]
t = T2[sp][c]
match = int(r.nsub) == t[1] and abs(r.dur_s - t[2]) <= 0.5
ok &= match
print(f" {sp:8s} {c:12s} N subjects {int(r.nsub):3d} (Table 2 {t[1]:3d})"
f" duration {r.dur_s:7.1f} (Table 2 {t[2]:5d}) match: {match}")
print(f"All 12 N-subjects cells and durations reproduced (to 0.5 s): {ok}")
print(f"Sum of per-context distinct subjects in the data: {int(g.nsub.sum())}")
k = df.groupby(["species", "subject"])["ctx"].nunique()
dist = k.groupby(level=0).value_counts().unstack(fill_value=0)
print("animals by number of contexts they were coded in:")
print(dist.to_string())
print(f"sum over animals of contexts = {int(k.sum())}; distinct animals = {len(k)}")
data = {sp: sorted(k.loc[sp].tolist(), reverse=True) for sp in SPECIES}
except Exception as e: # network or pandas missing: figures from the paper still build
print(f"\n[data-file check skipped: {e!r}]")
# ---------------------------------------------------------------- figure helpers
def text_panel(ax, x, y, w, h, title, body, edge, title_color=INK):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc="white", ec=edge, lw=2, transform=ax.transAxes))
ax.text(x + 0.02, y + h - 0.045, title, transform=ax.transAxes, ha="left", va="top",
fontsize=10.5, color=title_color, fontweight="bold")
ax.text(x + 0.02, y + h - 0.15, body, transform=ax.transAxes, ha="left", va="top",
fontsize=10, color=INK, linespacing=1.45)
# ---- Candidate A: two text panels joined by an annotated connector (Abstract vs Methods/Table 2)
def cand_A(path):
fig = plt.figure(figsize=(W, 4.4))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
text_panel(ax, 0.01, 0.40, 0.38, 0.58, "Abstract (reported)",
"“We coded facial behavior in\n>3000 social interactions across\n"
"three social contexts (aggressive,\nsubmissive, affiliative) in\n"
"389 animals”", ORANGE)
rows = "\n".join(f"{sp:<8s} {METHODS[sp][0]:>2d} M + {METHODS[sp][1]:>2d} F = {adults[sp]:>3d}"
for sp in SPECIES)
text_panel(ax, 0.55, 0.40, 0.44, 0.58, "Methods, Study subjects",
rows + f"\n{'':23s}───\n{'Adults':<21s}= {total_adults:>3d}",
BLUE)
ax.texts[-1].set_family("DejaVu Sans Mono")
ax.annotate("", xy=(0.55, 0.67), xytext=(0.39, 0.67), xycoords="axes fraction",
arrowprops=dict(arrowstyle="<->", color=INK2, lw=2))
ax.text(0.47, 0.76, "conflict", ha="center", va="bottom", fontsize=10.5, color=INK,
fontweight="bold", transform=ax.transAxes)
ax.text(0.47, 0.58, f"389 vs {total_adults}\n= {ABSTRACT_ANIMALS / total_adults:.2f}×",
ha="center", va="top", fontsize=10, color=INK2, transform=ax.transAxes)
t2 = "\n".join(f"{sp:<8s} {' + '.join(f'{T2[sp][c][1]:2d}' for c in CONTEXTS)} = {subj_sum[sp]}"
for sp in SPECIES)
ax.add_patch(FancyBboxPatch((0.01, 0.01), 0.98, 0.33, boxstyle="round,pad=0,rounding_size=0.012",
fc="white", ec=NEUTRAL, lw=2, transform=ax.transAxes))
ax.text(0.03, 0.31, "Table 2, ‘N subjects’ per context\n(affil. + aggr. + subm. + unclear)",
fontsize=10.5, fontweight="bold", va="top", transform=ax.transAxes, linespacing=1.35)
ax.text(0.03, 0.17, t2, fontsize=10, va="top", transform=ax.transAxes, family="DejaVu Sans Mono",
linespacing=1.35)
ax.text(0.50, 0.29, f"122 + 166 + 101 = {total_subj}, the abstract's figure.\n"
"Each cell is ≤ the Methods headcount for its\nspecies; the authors' data file confirms "
"that the\nsum counts each animal once per context\nit was coded in (up to four times).",
fontsize=10, color=INK2, va="top", transform=ax.transAxes, linespacing=1.45)
save(fig, path)
# ---- Candidate B: two stacked bars on one count axis, segmented by species
def cand_B(path):
fig, ax = plt.subplots(figsize=(W, 2.9))
bars = [("Abstract: “389 animals”\n(= sum of Table 2 N subjects)", subj_sum, ORANGE),
("Methods: adults enrolled\n(distinct animals; same order)", adults, BLUE)]
for i, (lab, vals, col) in enumerate(bars):
y, left = 1 - i, 0
for sp in SPECIES:
v = vals[sp]
ax.barh(y, v - 2, left=left + 1, height=0.52, color=col, edgecolor="none")
ax.text(left + v / 2, y, f"{v}", ha="center", va="center", fontsize=10.5,
color="white", fontweight="bold")
if i == 0:
ax.text(left + v / 2, y - 0.34, sp, ha="center", va="top", fontsize=10, color=INK2)
left += v
ax.text(left + 6, y, f"{left}", ha="left", va="center", fontsize=11, fontweight="bold")
ax.set_yticks([1, 0], [b[0] for b in bars], fontsize=10)
ax.set_xlim(0, 420); ax.set_ylim(-0.75, 1.45)
ax.set_xlabel("Count (starts at zero)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_title(f"The abstract's count is {ABSTRACT_ANIMALS / total_adults:.2f}× the number of "
"animals listed in the Methods", fontsize=11, loc="left", color=INK)
save(fig, path)
# ---- Candidate C: per-species dot plot, contexts vs headcount vs sum
# Context circles are stacked vertically (affiliative, aggressive, submissive, unclear,
# top to bottom) so that close values such as 29-32 or 43-45 do not hide one another.
OFFS = (0.33, 0.11, -0.11, -0.33)
def cand_C(path):
fig, ax = plt.subplots(figsize=(W, 3.6))
for i, sp in enumerate(SPECIES):
y = 2 - i
vals = [T2[sp][c][1] for c in CONTEXTS]
ax.plot([0, subj_sum[sp]], [y, y], color=GRID, lw=2, zorder=1)
ax.scatter(vals, [y + o for o in OFFS], s=60, facecolor="white", edgecolor=INK2, lw=1.5,
zorder=3, label="Table 2 N subjects, one circle per context\n"
"(stacked: affiliative, aggressive, submissive, unclear)" if i == 0 else None)
ax.plot([adults[sp]] * 2, [y - 0.44, y + 0.44], color=BLUE, lw=2.5, zorder=4,
solid_capstyle="butt",
label="Methods: adults (distinct animals)" if i == 0 else None)
ax.scatter([subj_sum[sp]], [y], s=110, marker="o", color=ORANGE, zorder=4, ec=BG, lw=1.5,
label="Sum of the four context counts (adds to the abstract's 389)" if i == 0 else None)
ax.text(adults[sp] + 2, y + 0.30, f"{adults[sp]}", ha="left", va="center", fontsize=10,
color=BLUE, fontweight="bold")
ax.text(subj_sum[sp], y + 0.22, f"{subj_sum[sp]}", ha="center", va="bottom", fontsize=10,
color=INK)
ax.set_yticks([2, 1, 0], SPECIES)
ax.set_xlim(0, 180); ax.set_ylim(-0.55, 2.6)
ax.set_xlabel("Number of animals (starts at zero)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.22), ncol=1, frameon=False, fontsize=10)
ax.set_title("Every context count fits under the headcount; their sum does not",
fontsize=11, loc="left")
save(fig, path)
# ---- Candidate D: calculation-flow diagram, two parallel chains
def cand_D(path):
fig = plt.figure(figsize=(W, 3.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off"); ax.set_xlim(0, 100); ax.set_ylim(1, 60)
def box(x, y, w, h, txt, ec, bold=False, fs=10):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.8",
fc="white", ec=ec, lw=2))
ax.text(x + w / 2, y + h / 2, txt, ha="center", va="center", fontsize=fs,
fontweight="bold" if bold else "normal")
def arrow(x0, y0, x1, y1):
ax.annotate("", xy=(x1, y1), xytext=(x0, y0),
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.6, mutation_scale=12))
ax.text(1, 57, "Table 2: N subjects per species × context (12 cells)", fontsize=10.5,
fontweight="bold", va="top")
ax.text(1, 27, "Methods: adults per species and sex", fontsize=10.5, fontweight="bold", va="top")
for i, sp in enumerate(SPECIES):
y = 44 - i * 7
box(1, y, 40, 5.5, f"{sp}: " + " + ".join(str(T2[sp][c][1]) for c in CONTEXTS), NEUTRAL)
arrow(41.5, y + 2.75, 49.5, y + 2.75)
box(50, y, 12, 5.5, str(subj_sum[sp]), NEUTRAL)
arrow(62.5, y + 2.75, 72.5, 39.75)
y2 = 15 - i * 7
box(1, y2, 40, 5.5, f"{sp}: {METHODS[sp][0]} M + {METHODS[sp][1]} F", NEUTRAL)
arrow(41.5, y2 + 2.75, 49.5, y2 + 2.75)
box(50, y2, 12, 5.5, str(adults[sp]), NEUTRAL)
arrow(62.5, y2 + 2.75, 72.5, 10.75)
box(73, 36, 26, 7.5, f"{total_subj}\nsubject-context pairs", ORANGE, bold=True)
box(73, 7, 26, 7.5, f"{total_adults}\ndistinct adults", BLUE, bold=True)
ax.text(86, 47, "Abstract: “in 389 animals” (reported)", ha="center", fontsize=10,
color=INK)
ax.text(86, 3.2, "Methods headcount (recomputed)", ha="center", fontsize=10, color=INK2)
ax.annotate("", xy=(86, 15.2), xytext=(86, 35.3),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.6))
ax.text(87.5, 25.2, f"{ABSTRACT_ANIMALS / total_adults:.2f}×", fontsize=10.5,
fontweight="bold", va="center")
save(fig, path)
# ---- Candidate E: unit chart from the authors' data file, one column per animal
def cand_E(path):
if data is None:
print("[candidate E skipped: needs the authors' data file]")
return False
fig = plt.figure(figsize=(W, 3.3))
ax = fig.add_axes([0.0, 0.14, 1.0, 0.68])
cs, gap = 1.0, 0.18
for i, sp in enumerate(SPECIES):
y0 = (2 - i) * 6.2
for j, k in enumerate(data[sp]):
for r in range(k):
ax.add_patch(Rectangle((j * cs + gap / 2, y0 + r * cs + gap / 2), cs - gap, cs - gap,
fc=ORANGE, ec="none"))
n = len(data[sp])
ax.plot([0, n * cs], [y0 - 0.35, y0 - 0.35], color=BLUE, lw=2.5)
ax.text(-1, y0 + 2, sp, ha="right", va="center", fontsize=10.5)
ax.text(n * cs + 1, y0 + 2.6, f"{sum(data[sp])} squares", ha="left", va="center",
fontsize=10, color=INK)
ax.text(n * cs + 1, y0 + 1.0, f"{n} animals", ha="left", va="center", fontsize=10,
color=INK2)
ax.set_xlim(-10, 62); ax.set_ylim(-1.0, 16.6); ax.axis("off")
fig.text(0.02, 0.97, "Authors' data file: each column is one animal (blue rule spans the animals);\n"
"each orange square is one context that animal was coded in (1–4).", fontsize=10,
va="top", linespacing=1.4)
tot_sq = sum(sum(v) for v in data.values())
tot_an = sum(len(v) for v in data.values())
fig.text(0.02, 0.03, f"Total: {tot_an} animals; {tot_sq} squares, which is the abstract's "
"“389 animals”", fontsize=10.5, fontweight="bold", va="bottom")
save(fig, path)
return True
builders = {"A": cand_A, "B": cand_B, "C": cand_C, "D": cand_D, "E": cand_E}
for key, fn in builders.items():
fn(os.path.join(CAND, f"cand_{key}"))
# Figures used in writeup.md (provisional choice: A as fig1, C as fig2)
cand_A(os.path.join(HERE, "fig1"))
cand_C(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE, "and", CAND)
Registered primary outcome, door-to-balloon time, was replaced after completion; main text omits this
JAMA Netw Open (2024) · doi:10.1001/jamanetworkopen.2024.3729 · PMC10980970 · NCT05415085
Summary
This single-center randomized trial compared culprit lesion PCI before complete coronary angiography with complete angiography first in STEMI. The first ClinicalTrials.gov version of NCT05415085 (v0, 2022-06-08) registered one primary outcome, "Reduction in door-to-balloon time". The second and current version (v1, 2023-10-17) replaced it with "Needle-to-balloon time of 10 minutes or less". v1 was posted 474 days after the registered primary completion date (2022-06-30) and 164 days before publication. The paper reports needle-to-balloon time of 10 minutes or less as "the primary outcome of this study". Its main text does not say that the primary outcome was changed, and it describes the trial as "prospectively registered", although the record was first submitted about 13 months after enrollment began. The only disclosure is one undated sentence in the protocol supplement. The originally registered outcome appears in a single Results sentence as a non-significant difference (61.6 vs 73 minutes; P = .12), and the paper does not identify it as the registered primary.
What was registered
ClinicalTrials.gov NCT05415085 has two versions. The record history lists the modules each version changed, and its lastUpdateVersions field shows that the primary outcomes were last changed in v1 and the secondary outcomes in v0, so the secondary outcomes never changed.
v0, first version. First submitted 2022-04-27, QC 2022-06-08, first posted 2022-06-10. Status RECRUITING; study start 2021-05-01 (actual); primary completion 2022-06-30 (estimated); enrollment 250 (estimated). Primary outcome:
Reduction in door-to-balloon time | Change in door-to-balloon time in minutes | time frame: through study completion, an average of 1 year
The brief summary in v0 reads: "The aim of this study is to assess the impact of culprit-first versus culprit-last percutaneous coronary intervention on the door to balloon time and clinical outcomes in patients with ST-elevation myocardial infarction." The keywords include "door-to-balloon time".
v1, current version. Submitted and QC 2023-10-17, posted 2023-10-19. Changed modules: Study Status, Study Design, Outcome Measures, Contacts/Locations. Status COMPLETED; primary completion 2022-06-30 (actual); study completion 2023-04-13 (actual); enrollment 106 (actual). Primary outcome:
Needle-to-balloon time of 10 minutes or less | Change in needle-to-balloon time in minutes | time frame: through study completion, an average of 1 year
The brief summary and keywords were not changed: the current record still gives the aim as the impact "on the door to balloon time and clinical outcomes" and still lists "door-to-balloon time" as a keyword.
Secondary outcomes, identical in v0 and v1:
Need for hemodynamic (mechanical/medical) support during PCI | Number of patients that needed mechanical support during PCI
Need for respiratory support during PCI | Number of patients that needed respiratory support during PCI
Rate of failed PCI | As determined by the operator
All three have the time frame "through study completion, an average of 1 year".
What the paper reports
Abstract, "Main Outcomes and Measures":
A needle-to-balloon time of 10 minutes or less.
Methods, "Study Protocol and End Point":
The primary outcome of this study was a needle-to-balloon time of 10 minutes or less. Secondary outcomes included the need for hemodynamic support (ie, mechanical and/or medical) during PCI, the need for invasive or noninvasive ventilation during PCI, and the rate of failed PCI.
Methods, "Power Calculation":
Based on previously published data, the primary outcome of a needle-to-balloon time of 10 minutes or less was used for sample size calculation.
Methods, "Study Design and Population", the only statement about the registration:
This study was prospectively registered in Clinical Trials.gov (NCT05415085).
The same paragraph gives the enrollment period: "The study included patients with a diagnosis of STEMI who underwent primary PCI between April 1, 2021, and August 31, 2022."
Results, "Primary and Secondary Outcomes", which holds the paper's only result for door-to-balloon time:
The primary outcome of a needle-to-balloon time of 10 minutes or less was achieved in 46 of the 90 patients (51.1%) who underwent culprit lesion PCI before CAG and in 18 of the 94 patients (19.1%) who underwent CAG followed by culprit lesion PCI (odds ratio, 4.4 [95% CI, 2.2-9.1]; P < .001).
There was no significant difference in the mean (SD) door-to-balloon time (61.6 [45.8] vs 73 [59.8] minutes; P = .12).
The main text contains neither "amend" nor "changed". Door-to-balloon time appears in it only as background (Introduction and Discussion, citing earlier studies) and in the Results sentence above. The Abstract, Key Points and Conclusions do not mention it.
Supplement 1 ("Study Protocol and Statistical Analysis Plan", jamanetwopen-e243729-s001.pdf), section "3. Endpoints", is the only place the change is mentioned:
a. Primary endpoint: o Reduction in Door-to-Balloon (D2B) time Protocol amendment: The investigators ask the IRB for a study protocol amendment due to reconsideration of study protocol and primary outcome: o Revised primary outcome: Needle-to-balloon time within 10 minutes.
The protocol text carries no date for the amendment or for its approval. Its sample-size paragraph uses the revised outcome, in the same words as the paper's Power Calculation section. Its "Study Cohort" section gives an "Estimated number of 300 patients (~150 each group)". The case report form appended to it is labelled "February 2021" and "Ver 1.4". That label belongs to the form; it does not date the amendment.
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| Door-to-balloon time ("Reduction in door-to-balloon time"; change in minutes) | Sole primary in v0 (2022-06-08); removed in v1 (2023-10-17) | Not named as an outcome. One Results sentence: mean 61.6 vs 73 min, P = .12. Not in the Abstract, Key Points or Conclusions | demoted |
| Needle-to-balloon time of 10 minutes or less | Absent from v0; sole primary in v1 (2023-10-17, after primary completion) | Primary outcome: 46/90 (51.1%) vs 18/94 (19.1%), OR 4.4 (95% CI 2.2-9.1). Also the basis of the power calculation | matched to v1; added relative to v0 |
| Needle-to-balloon time in minutes (the v1 description, "Change in needle-to-balloon time in minutes") | Description text of the v1 primary | Reported as mean 11.4 vs 17.3 min, P < .001, not labelled primary | matched (wording difference: the registry title is a threshold, its description is continuous) |
| Hemodynamic support during PCI | Secondary, v0 and v1 | Secondary | matched |
| Respiratory support during PCI | Secondary, v0 and v1 | Secondary ("invasive or noninvasive ventilation") | matched |
| Rate of failed PCI | Secondary, v0 and v1 | Secondary | matched |
The main text gives no account of the change. A reader of the article alone would see a primary outcome that matches the current registry record and a statement that the trial was prospectively registered.
Figures
Figure 1. Registry events (blue, above the axis) and paper events (orange, below the axis) on one calendar axis. The paper's enrollment period (April 1, 2021 to August 31, 2022) is the orange bar. The light-grey band marks the period after the registered study start, and the darker band the period after the registered primary completion date (2022-06-30). The record was first submitted on 2022-04-27, 361 days after the registered start and 391 days after the paper's enrollment start, and its first version (v0, QC 2022-06-08) named door-to-balloon time as the primary outcome. The circled event, v1 on 2023-10-17, replaced the primary outcome with needle-to-balloon time of 10 minutes or less and changed enrollment from 250 (estimated) to 106 (actual). It falls after primary completion, after study completion (2023-04-13) and 164 days before publication on 2024-03-29.
Figure 2. Left column: the primary outcome in each registry version, quoted from the record (v0, 2022-06-08; v1, 2023-10-17). Right column: how the paper treats each one. The v1 outcome matches the paper's primary outcome, which carries the headline result (51.1% vs 19.1%; OR 4.4). The v0 outcome, door-to-balloon time, is not named as an outcome in the paper. Its only result is one Results sentence reporting no significant difference (61.6 vs 73 minutes; P = .12). The connector labels state the relation (REPLACED, DEMOTED, MATCHED). The bottom panel summarises the disclosure: none in the main text, one undated sentence in Supplement 1. The secondary outcomes are the same in both registry versions and in the paper.
Why it matters
A registered primary outcome lets readers judge whether the outcome that carries a trial's conclusions was chosen before the results were known. Here the first registered primary outcome, door-to-balloon time, is the one the trial's registry summary still names as its aim. It was replaced in the registry on 2023-10-17, after the registered primary completion date and after study completion. The paper's headline findings rest entirely on the replacement outcome: the Key Points finding (51.1% vs 19.1%), the abstract's odds ratio of 4.4, and the conclusion that culprit lesion PCI before complete CAG "resulted in shorter reperfusion times".
For the originally registered primary outcome the paper reports a mean (SD) door-to-balloon time of 61.6 (45.8) vs 73 (59.8) minutes, "no significant difference", P = .12. That result appears once, in the Results text. The Abstract, Key Points and Conclusions do not report it, and nothing in the paper tells the reader that it was the registered primary outcome.
The registry does not show when the decision to change the outcome was made. The protocol supplement records that an amendment was requested but gives no date. Without that date, a reader cannot tell whether the change came before enrollment, during it, or after outcome data were available. Each of these calls for a different reading of the result, and CONSORT 2010 item 6b asks trial reports to state any changes to trial outcomes after the trial commenced, with reasons.
The registration timing adds to this. The record was first submitted on 2022-04-27, 391 days after the paper's enrollment start (April 1, 2021) and 361 days after the registered start (2021-05-01). Under the ICMJE and WHO definition, which requires registration before the first participant is enrolled, the registration was retrospective. The paper's statement that the study was "prospectively registered" therefore does not match the record. The door-to-balloon primary was itself first recorded after about a year of enrollment.
Explanations considered
- Paper role (secondary analysis or a different paper for the registered outcome). Does not account for it. This is the main report of the trial. It cites only NCT05415085 and presents its outcome as "the primary outcome of this study". No other publication from the trial is cited.
- Disclosure. Partly addresses it, in a supplement only. Supplement 1 states that an amendment of the primary outcome was requested, "due to reconsideration of study protocol and primary outcome". The main text, which most readers and indexing services see, does not mention it. The supplement gives no date for the amendment or its approval, no reason beyond the quoted phrase, and no indication of whether outcome data had been seen. The main text's statement "This study was prospectively registered" does not reflect the change or the registration date.
- A protocol amendment approved before enrollment, with a late registry update. Possible, and it would be a legitimate reason for the change. The published material cannot confirm it. The protocol supplement does not date the amendment. The first registry version, submitted in April 2022 after about a year of enrollment, still named door-to-balloon time, and the registry was updated only after the trial was completed. If the amendment came before enrollment, the paper could say so with the IRB approval date.
- Wording differences (the two outcomes are the same measure described differently). Does not account for it. Door-to-balloon time runs from arrival at the hospital to first balloon inflation; needle-to-balloon time runs from the first arterial puncture to first balloon inflation (the paper defines both time points in Methods, "Data Collection"). They are different intervals, and the paper reports them separately, with different results (P = .12 vs P < .001). A smaller wording point concerns v1 itself: its title names a threshold ("10 minutes or less") while its description reads "Change in needle-to-balloon time in minutes". The paper's primary analysis uses the threshold, so it matches the v1 title.
- Correction or registry note. None found (see Status). The registry record has no note explaining the change, and the brief summary still states the door-to-balloon aim.
Separately, three other registry-paper differences are not part of the outcome change and are noted for completeness only. The registry gives enrollment as 106 (actual), while the paper reports 214 randomized in Results (106 and 108 per arm) and 216 in the Abstract. 106 is the number the paper gives for the culprit-lesion-first arm. The registry lists masking as "DOUBLE" (participant, care provider), while the paper describes an "open-label" trial. The registered primary completion date (2022-06-30) precedes the end of enrollment reported in the paper (August 31, 2022).
Status
As of 2026-10-23:
- Registry history. ClinicalTrials.gov API v2 current record, fetched 2026-10-23 (versionHolder 2026-10-22): last update submitted 2023-10-17, posted 2023-10-19. The version history (2 versions) and the archived v0 record were retrieved on 2026-10-22 from the internal history routes (
/api/int/studies/NCT05415085?history=trueand/api/int/studies/NCT05415085/history/0). The same history endpoints returned HTTP 403 when retried on 2026-10-23, and the history tab has no static content. The v1 content is identical to the current record. - Publisher version. The publisher's page (jamanetwork.com/journals/jamanetworkopen/fullarticle/2816961) carries the same registration statement and primary-outcome sentence as the PMC archive and shows no correction, erratum or editorial notice. Its comments section shows no comments. All paper quotes above were checked verbatim against the PMC-archived text. The supplement quotes were checked against the text extracted from Supplement 1 (jamanetwopen-e243729-s001.pdf).
- Corrections. Crossref lists no update relations for the DOI. PMC and Europe PMC list no correction, retraction or comment notices. A web search found no letter, comment or correction about the trial.
- PubPeer. Could not be checked: both the API and the search page refused automated access (HTTP 403).
- Sources. PMC10980970 and its Supplement 1; https://clinicaltrials.gov/study/NCT05415085 (current record and history); Crossref; Europe PMC; the publisher's article page.
Suggested correction
- In Methods, state that the primary outcome registered at ClinicalTrials.gov on 2022-06-08 was door-to-balloon time, that it was changed to needle-to-balloon time of 10 minutes or less, and give the date of the protocol amendment and its IRB approval, the reason, and whether the change was made before or after any outcome data were seen. The registry update date (2023-10-17) should also be stated.
- Report door-to-balloon time as the originally registered primary outcome, with its effect estimate and confidence interval, in the Results and in the Abstract.
- Replace "This study was prospectively registered" with the registration dates: first submitted 2022-04-27 and first posted 2022-06-10, after enrollment began on April 1, 2021.
- Minor: reconcile the number randomized (216 in the Abstract, 214 in Results). Ask the sponsor to update the registry's enrollment figure, masking field and brief summary so that they agree with the trial as reported.
Paper reports dose-limiting toxicity as primary endpoint; the only registered primary outcome is response rate
Cell Rep Med (2026) · doi:10.1016/j.xcrm.2026.102967 · PMC13522775 · NCT05459441
Summary
The ClinicalTrials.gov record for this trial has had one version (2022-07-12), and its sole primary outcome is "Clinical response rate", the percentage of patients with complete or partial response, at 24 months. The published report states that "The primary endpoint was the incidence of dose-limiting toxicities (DLTs)". DLT incidence appears nowhere in the registry, and the paper places radiographic response among "secondary or exploratory endpoints", reported as best overall response rather than at 24 months. The paper cites the registration number but does not mention that the registered primary outcome differs, and it gives no protocol amendment or reason for the difference. The paper reports that "There were no dose-limiting toxicities observed during dose escalation". The registered primary outcome is also reported, in Table 3 and outside the abstract: 2 of 17 efficacy-evaluable patients (11.8%) had a partial response and none had a complete response.
What was registered
The registry history (ClinicalTrials.gov record-history service, checked 2026-10-22 and 2026-10-23) lists exactly one version, version 0, dated 2022-07-12. The service reports outcomesUpdateCount: 0, and the current public record (API v2) has the same outcomes. The record was first submitted on 2022-06-24 and first posted on 2022-07-15. Its last update was posted on 2022-07-15. It gives the study start as 2022-01-01 (actual), so the trial was registered 174 days after it started, and no version exists from before the start date. The phase is "Early Phase 1". The brief title begins "Efficacy and Safety Evaluation of Percutaneous Ommaya Capsule Injection of Autologous Bi-dimensional Specific T Cells…".
| Version (date) | Type | Measure | Time frame | Description (verbatim) |
|---|---|---|---|---|
| v0 (2022-07-12) | Primary | "Clinical response rate" | "24 months" | "The percentage of patients with CR and PR in the total number of patients in the same period" |
| v0 (2022-07-12) | Secondary | "Overall survival (OS)" | "24 months" | "Frome the starting date of the enrollment until the date of the first documented disease progression or the date of the death from any cause,whichever comes first" |
| v0 (2022-07-12) | Secondary | "Safty(adverse events)" | "12 months" | "Number of participants with treatment-related adverse events as assessed by CTCAE v5.0" |
The other registered fields were never updated. The estimated primary completion date is 2023-06-30 and the estimated completion date is 2023-12-31. Enrolment is "30 (ESTIMATED)". The overall status is now "UNKNOWN", with last known status "RECRUITING" and status verified 2022-02. "Dose-limiting toxicity" does not appear anywhere in the record. The record's submission-tracking data show a results submission dated 2026-08-06, released and withdrawn the same day, and another released on 2026-08-08. As of the check, no results are posted and the submitted content is not public.
What the paper reports
The primary endpoint, from STAR Methods, "Objectives and endpoints":
The primary objective of this study was to evaluate the safety and tolerability of intracranial administration of non-chimeric autologous expanded T cells combined with pemetrexed. The primary endpoint was the incidence of dose-limiting toxicities (DLTs), assessed during a prespecified DLT evaluation window defined as inside the intracranial treatment cycle (once weekly for 5 weeks).
Secondary or exploratory endpoints included overall survival (OS), progression-free survival (PFS), and disease control rate (DCR).
Tumor response was determined based on the best overall response from treatment initiation until disease progression or last adequate tumor assessment.
The DLT definition, from STAR Methods, "Dose-escalation design":
DLT was defined as the occurrence of ≥1 treatment-related death or ≥3 unanticipated grade ≥3 treatment-related adverse events within the predefined DLT evaluation period.
From the Introduction:
The primary objective was to assess safety and tolerability, with comprehensive monitoring of both systemic and intracranial AEs.
Clinical outcomes, including radiographic response, progression-free survival (PFS), and overall survival (OS), were assessed as secondary or exploratory endpoints to characterize potential signals of therapeutic activity.
The result for this primary endpoint, from Results, "Safety of SCOPE-T treatment":
There were no dose-limiting toxicities observed during dose escalation; therefore, the highest dose tested (5 × 108 RAK cells) was selected for dose expansion and designated as the recommended phase 2 dose (RP2D).
(The "108" is 10^8 as rendered in the PMC text.) The response results appear in Results under the heading "Exploratory radiographic and clinical assessments", and in Table 3 ("Summary of clinical responses in the PPS and EES"):
Objective response rate | 2 (10.5%) | 2 (11.8%)
Benefit beyond imaging evaluationa | 19 (100.0%) | 17 (100.0%)
Clinical benefit rate | 19 (100.0%) | 17 (100.0%)
(The trailing "a" marks a footnote listing clinical and laboratory improvements.) The Results attribute clinical benefit to RANO assessment: "Using RANO-BM and RANO-LM criteria, all 17 patients in the EES demonstrated clinical benefit."
The sample size is discussed in STAR Methods, "Study design and patients" ("The selected sample size (n = 25) is consistent with typical phase 1/1b trials designed to evaluate safety and tolerability."), and in "Statistical analysis" ("No formal hypothesis testing or power calculation was prespecified.").
The paper refers to the registration only by its NCT number (Abstract, Introduction) and in the line in STAR Methods, "Clinical trial registration":
This trial was registered with https://clinicaltrials.gov/study/NCT05459441.
The paper never mentions that response rate was the registered primary outcome, or any protocol amendment or change of endpoints. It does not give a registration date. Apart from the per-protocol analysis set (defined in STAR Methods, "Analysis populations", as "patients who completed at least one treatment cycle without major protocol deviations"), the study protocol is referred to only in the Table 4 footer ("OS and PFS: According to protocol, data cutoff as of July 6, 2024."). The same footer adds "OS∗: Updated data follow-up as of May 01, 2026." for a separate survival column. Neither a protocol nor a statistical analysis plan is published with the article: the supplement is Document S1 (Figures S1–S9, Tables S1–S3), Data S1 (xlsx) and Document S2 (the article plus supplement).
Every quotation above was checked character for character against the article as archived in PubMed Central (src/show.py PMC13522775 --grep), and each occurs exactly once. The endpoint sentences, the registration line and the Table 3 rows also appear unchanged in the publisher's typeset PDF (Document S2). make_figures.py.txt reruns the PMC check, and the registry check against API v2.
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| Clinical response rate (CR + PR / total patients) | Primary, 24 months (v0, 2022-07-12) | Radiographic response among "secondary or exploratory endpoints". Not in the named list of secondary/exploratory endpoints (OS, PFS, DCR). Reported as best overall response in Table 3: ORR 2/19 (10.5%) in the PPS and 2/17 (11.8%) in the EES | Demoted; time point changed (best overall response, not 24 months) |
| Incidence of dose-limiting toxicities | Not registered at any level (v0) | Primary endpoint, with a 5-week DLT window | Added |
| Treatment-related adverse events (CTCAE v5.0) | Secondary, 12 months (v0) | Reported (Table 2; Table S3; abstract: "no treatment-related adverse events greater than grade 2"), but not listed as an endpoint. No time frame stated | Reported; not named an endpoint; time frame not stated |
| Overall survival | Secondary, 24 months (v0). The registered description reads like a progression-or-death definition | Secondary or exploratory: Kaplan-Meier median and 1-year rate | Matched by name; 24-month time frame not used |
| Progression-free survival | Not registered (v0) | Secondary or exploratory | Added |
| Disease control rate (CR + PR + SD) | Not registered (v0) | Secondary or exploratory. Table 3 has no row named DCR, but lists PR and SD counts and a "Clinical benefit rate" | Added |
| Enrolment (context) | 30, estimated (v0) | 25 enrolled; "The selected sample size (n = 25)…" | Differs; not explained |
Figures
Figure 1. Timeline of the trial and its registration. Blue circles are registry events: first submission (2022-06-24), version 0 (2022-07-12; the only version), first posting (2022-07-15) and the registered estimated primary completion date (2023-06-30, never updated), all on the axis. The results submissions (2026-08-06 and 2026-08-08; not posted as of 2026-10-23) share one blue circle, drawn on its stem above the version bar because on the axis it would sit under the publication marker one to three days earlier. Orange squares are trial and paper events taken from the article: study start and first enrolment (2022-01-01), last enrolment (2024-03-30), data cutoff (2024-07-06), journal submission (2025-11-28) and online publication (2026-08-05). The shaded band marks the 174 days between study start and registration. The blue bar shows that version 0 has been in force from 2022-07-12 to the check date (2026-10-23). No registry version changed the primary outcome, either before or after primary completion. The difference arises between the registry and the paper, not within the registry. The registered primary completion date is shown only as the registry's own, never-updated estimate; it is not a measure of reporting time (enrolment ended 2024-03-30 and the paper's data cutoff was 2024-07-06).
Figure 2. Registered outcomes compared with the outcomes in the paper. The left column (blue) shows the outcomes in version 0 (2022-07-12) with their registered type and time frame. The right column (orange) shows how the paper designates and reports each one. Every connector carries a text label. DLT incidence is labelled ADDED: it is the paper's primary endpoint and is absent from the registry. Its result box quotes the paper: no DLTs were observed during dose escalation. The registered primary, clinical response rate, is labelled DEMOTED and time point changed: the paper treats it as secondary or exploratory and reports it as best overall response, with the result from Table 3 (ORR 2/17, 11.8%, exact 95% CI 1.5–36.4%, in the EES; 2/19, 10.5%, in the PPS; no CR). The registered adverse-event outcome is labelled REPORTED. Overall survival is labelled MATCHED by name. PFS and DCR are labelled ADDED. The footnote gives the response rate over all 25 enrolled patients (2/25 = 8%, exact 95% CI 1.0–26.0%), computed here from Tables 3–4 because the registered description uses "the total number of patients" as its denominator; the paper does not report this figure. Exact intervals are Clopper-Pearson, the method the paper states for categorical variables.
Why it matters
- The registered primary outcome is reported, but it is framed as exploratory and left out of the abstract. The CR + PR rate is 2/17 (11.8%; exact 95% CI 1.5–36.4%) in the efficacy-evaluable set and 2/19 (10.5%) in the per-protocol set, with no complete responses. Over all 25 enrolled patients, the denominator the registered description implies, it would be 2/25 (8%). Of the two partial responses, one patient (Pt5) went from SD to PR and the other (Pt6) from PR to SD (Table 4). No response threshold was prespecified and no hypothesis was tested, so the result is not formally positive or negative. The abstract does not report the objective response rate.
- The abstract's efficacy statements rest on outcomes that were not registered. The abstract says "Exploratory radiographic and clinical assessments suggest disease stabilization in evaluable patients" and concludes that the findings "indicate preliminary signals of therapeutic activity". A Highlight reads "Preliminary disease control observed in a small early-phase cohort". These rely on disease control (CR + PR + SD) and on clinical benefit: Table 3 gives a "Clinical benefit rate" and a "Benefit beyond imaging evaluation" row, each 17/17 (100.0%) in the EES, and the Results attribute clinical benefit in all 17 to RANO-BM/RANO-LM assessment. Neither disease control nor clinical benefit appears in the registry.
- The headline safety conclusion rests on an unregistered primary endpoint. The abstract's main finding is safety and tolerability ("no treatment-related adverse events greater than grade 2"). The registry lists safety only as a secondary outcome, defined as a count of participants with treatment-related AEs at 12 months. The DLT endpoint the paper calls primary (result: "There were no dose-limiting toxicities observed during dose escalation"), and its "prespecified" 5-week window, cannot be checked against any public prespecified document. The safety data themselves are reported in full: Table 2 lists all AEs by grade for the 25 patients.
- The registered time frame is not used and is not addressed. The registry specifies response at 24 months. The protocol data cutoff for OS and PFS (2024-07-06, Table 4 footer) came about 3 months after the last enrolment (2024-03-30), so no patient enrolled after early July 2022 could have had 24 months of follow-up at that cutoff. The Table 4 footer also gives "OS∗: Updated data follow-up as of May 01, 2026.", so longer follow-up exists for some patients, but that update covers survival only: no response status at 24 months is reported. The paper reports best overall response instead and does not discuss the registered time frame.
- Readers comparing paper and registry get conflicting accounts. Anyone who follows the cited registration link finds response rate as the only primary outcome and no DLT endpoint. Systematic reviewers and outcome-reporting audits would code this as an undisclosed change of primary outcome.
On severity: safety as the primary endpoint is conventional for a first-in-human phase 1 study, the paper repeatedly calls its efficacy findings exploratory, and the registered primary result appears in Table 3. The safety conclusion therefore does not depend on the discrepancy. What is missing is any statement that the registered primary outcome was different.
Explanations considered
- Is this a secondary analysis or a partial report? No. The paper is the primary report of the trial. It covers all 25 patients enrolled between 1 January 2022 and 30 March 2024, at both sites, with safety, response and survival. The registry has a single primary outcome, so it cannot have been split across publications. The other NCT numbers in the paper (NCT02208362, NCT04185038, NCT04196413) belong to other trials and are cited to justify sample size and dosing.
- Does the paper disclose the change? No. Its only registration statement is "This trial was registered with https://clinicaltrials.gov/study/NCT05459441." No passage refers to the registered outcomes, an amendment, a change of endpoints or a reason for one. Searches of the text and the supplement for "primary outcome", "amend", "registered" and "response rate" found nothing relevant. The only "amend" hit is an acknowledgment of help with "amendments to enhance the manuscript".
- Was there a protocol amendment? The Table 4 footer refers to a protocol ("According to protocol, data cutoff as of July 6, 2024.") and the per-protocol set is defined by the absence of "major protocol deviations", but no protocol, amendment history or statistical analysis plan is published. The registry was never updated after 2022-07-12, so it records no amendment either. A change in the protocol may have happened, but there is no public record of one.
- Are these the same outcome under different wording? No. CR + PR response rate measures efficacy. DLT incidence ("≥1 treatment-related death or ≥3 unanticipated grade ≥3 treatment-related adverse events" within the DLT window) measures safety. The registered safety outcome, the number of participants with treatment-related AEs (CTCAE v5.0) at 12 months, is a secondary outcome. It is also a different measure from DLT incidence, with a different time window.
- Is a safety primary endpoint standard for phase 1? Yes. It is a legitimate reason a team might name DLT incidence as primary, and the paper's design (dose escalation guided by DLTs) is consistent with it. It does not account for the registry stating otherwise, though. The registered title begins "Efficacy and Safety Evaluation", and the registered primary outcome is efficacy. The paper does not explain the difference.
- Is the registry simply out of date? The record has not been updated since July 2022: the status is "UNKNOWN", and the completion dates and enrolment are still estimates. A stale record could explain why the registry was never aligned with a later protocol. It does not make the change disclosed. The registry is the only public prespecification, and the paper calls its DLT window "prespecified" without saying that the registry lists a different primary outcome.
- Did the registration timing play a part? The trial was registered 174 days after it started, so even the registered outcomes were entered after enrolment had begun. The paper neither claims prospective registration nor gives a registration date. This affects how the registration can be interpreted, but it does not explain the outcome difference: the registry and the paper still disagree.
- Is it a conversion or typesetting artifact? No. The same endpoint sentences and the same registration line appear in the PMC-archived text and in the publisher's typeset PDF (Document S2).
None of these accounts for the absence of disclosure. The issue documented here is that the difference is not acknowledged. It may well have had a legitimate reason.
Status
As of 2026-10-23:
- Registry history: checked. The ClinicalTrials.gov record-history service (
/api/int/studies/NCT05459441?history=trueand/history/0) lists one version (v0, 2022-07-12), withoutcomesUpdateCount: 0and the original primary and secondary outcomes unchanged. The current API v2 record matches v0. The website history tab (?tab=history) returns only a JavaScript shell through WebFetch. The internal routes return 403 to a cold client but 200 within a session that first calls API v2, which is how they were read. Results were submitted on 2026-08-06 and 2026-08-08 and are not posted. - Publisher version: checked through the typeset article in Document S2 (Europe PMC supplementary files: "Cell Reports Medicine 7, 102967, August 18, 2026"; Received 28 November 2025, Revised 25 March 2026, Accepted 7 July 2026, Published 5 August 2026). The primary-endpoint sentence, the secondary/exploratory sentences, the registration line and Table 3 match the PMC text. The publisher's HTML pages (cell.com, ScienceDirect) returned 403. The DOI resolves to Elsevier's linking hub.
- Corrections and notices: none found.
src/known.pyreturns no Crossref updates, no PMC notices and no Europe PMC notices, and Crossref has no update-to relation. A web search found no erratum or correction. - PubPeer: could not be checked. Its search page and API returned 403 (human-verification wall), so a PubPeer thread cannot be ruled out. COMPare does not apply: it covered 2015–2016 trials in five journals.
- Sources: https://clinicaltrials.gov/api/v2/studies/NCT05459441; https://clinicaltrials.gov/api/int/studies/NCT05459441?history=true; https://clinicaltrials.gov/api/int/studies/NCT05459441/history/0; PMC13522775 (
src/show.py, full text and Tables 2–4); https://www.ebi.ac.uk/europepmc/webservices/rest/PMC13522775/supplementaryFiles (mmc1.pdf, mmc2.xlsx, mmc3.pdf);src/known.py 10.1016/j.xcrm.2026.102967 --pmcid PMC13522775; https://doi.org/10.1016/j.xcrm.2026.102967.
Suggested correction
- A correction notice, or an added paragraph in "Objectives and endpoints". It should state that the ClinicalTrials.gov record (NCT05459441, registered 2022-06-24, after the 2022-01-01 start) lists "Clinical response rate" (CR + PR, 24 months) as the sole primary outcome and does not list DLT incidence. It should also give when and why the primary endpoint became DLT incidence (protocol version and date), or state that the registration was inaccurate from the outset.
- The protocol and statistical analysis plan, with version dates, added as supplementary material, so that the "prespecified" DLT window and the endpoint hierarchy can be checked.
- The registered primary outcome reported as registered, or a statement of why it cannot be. That means CR + PR over all enrolled patients (2/25 from the paper's Tables 3–4) alongside the EES and PPS figures, and the response status at 24 months (for instance from the May 2026 follow-up the paper already uses for survival), or a note that follow-up did not allow it. The abstract could give the objective response rate next to the disease-stabilization statement.
- The ClinicalTrials.gov record updated: outcome measures (with the change recorded in the registry's history), actual enrolment (25), actual primary completion and completion dates, and recruitment status. The results submitted in August 2026, not posted as of 2026-10-23, would let readers check the registered outcomes once they are public.
Registered 6-week re-intervention count reported instead as a 6-month total procedure count, without disclosure
Gut (2018) · doi:10.1136/gutjnl-2017-315335 · PMC6582745 · NCT02685865
Summary
This paper is the primary report of a single-centre randomised trial comparing lumen-apposing metal stents (LAMS) with plastic stents for EUS-guided drainage of walled-off necrosis (WON). In every ClinicalTrials.gov version (v0, 2016-02-18, to v7, 2019-02-08) the primary outcome has a 6-week time frame, and its description defines the counted events as re-interventions: repeat EUS-guided drainage, direct endoscopic necrosectomy (DEN) or a percutaneous catheter "due to inadequate treatment response". v0 to v3 title it "The number of re-interventions performed between metal and plastic stent groups to achieve treatment success". v4 (2017-09-13) changed "re-interventions" to "interventions" in the measure title and in the first sentence of the description, but left the definition, the hypothesis and the time frame unchanged; v4 to v7, including the current record, carry this wording. The paper's primary outcome is instead the "total number of procedures needed to achieve treatment success at 6-month follow-up". This count also includes the index procedure, ERCP, enteral feeding tubes and procedures for adverse events, so it differs from the registered outcome in both time point and construct. The paper does not mention the difference, and it reports no per-patient result or between-group test for the registered outcome.
What was registered
The record has eight versions, v0 to v7 (indices from the ClinicalTrials.gov history API). The record was first submitted on 2015-11-24. The earliest version, v0, is dated 2016-02-18 (first posted 2016-02-19), six days after the actual study start of 2016-02-12. v0 gave the start only as "2016-02", and from v3 (2017-04-21) the start is "2016-02-12" (actual). No version exists on or before the study start.
v0, 2016-02-18 (status Not yet recruiting; estimated enrolment 62). One primary outcome:
Measure: "The number of re-interventions performed between metal and plastic stent groups to achieve treatment success" (time frame: "6 weeks")
Description: "The primary objective of this study is to compare the number of re-interventions performed between metal and plastic stent groups to achieve treatment success at 6 week follow-up. Re-intervention is defined as the need for repeat EUS-guided drainage of WON using the same stent type and/or DEN and/or insertion of percutaneous catheter due to inadequate treatment response following initial WON drainage. As FCSEMS have larger diameter than plastic stents, the investigators hypothesize that the no. of re-interventions performed will be lower in the FCSEMS group compared to the plastic stent group."
The registered secondary outcome for treatment success was:
"Treatment success" (time frame: "6 week"): "Decrease in the size of WON to <33% of the original size and resolution of symptoms at 6-week outpatient follow-up (from day of hospital discharge) WITHOUT the need for cross-over to minimally invasive surgical debridement"
The other registered secondaries were technical success (day 0), treatment failure (6 months), recurrence (6 months), procedure duration (day 0), adverse events (6 months), duration of hospitalisation (6 months) and cost analysis (6 months).
v1 (2016-04-12), v2 (2017-02-13), v3 (2017-04-21). No change to the outcome measures. v1 changed the Study Identification module (brief title from "Trial Comparing FCSEMS and Plastic Stents" to "Comparison of FCSEMS and Plastic Stents") and the status, description, conditions and contacts modules; v2 and v3 changed the status module (status and dates) only.
v4, 2017-09-13 (status Active, not recruiting; actual enrolment 60; primary completion "2017-07-10", actual). The module labels include "Outcome Measures". The primary measure title became:
"The number of interventions performed between metal and plastic stent groups to achieve treatment success" (time frame: "6 weeks")
In the first sentence of the description, "re-interventions" became "interventions" ("...to compare the number of interventions performed between metal and plastic stent groups to achieve treatment success at 6 week follow-up."). The rest of the description was not changed: the definition still begins "Re-intervention is defined as the need for repeat EUS-guided drainage of WON using the same stent type and/or DEN and/or insertion of percutaneous catheter due to inadequate treatment response following initial WON drainage", and the hypothesis still refers to "the no. of re-interventions". The secondary treatment-success description became "Resolution of WON on CT scan and resolution of symptoms at 6-week outpatient follow-up (from day of hospital discharge) WITHOUT the need for cross-over to minimally invasive surgical debridement", still at "6 week". v4 also made wording edits to two other secondary-outcome descriptions, with no change to their time frames: treatment failure gained the words "or underlying diseaseor underlying disease." (as written), and procedure duration now starts when the "endoscope [is] inserted into the gastrointestinal lumen to perform transmural drainage" instead of "inserted into the oropharynx". This edit was made after the actual primary completion date then registered (2017-07-10; v6 later changed it to 2018-07-10) and nine days before the manuscript was received (2017-09-22).
v5, 2018-05-30. This is the last version before online publication (2018-06-01). The outcomes are the same as in v4.
v6, 2018-07-31, and v7, 2019-02-08. No outcome change. v6 changed the actual primary completion date from 2017-07-10 to 2018-07-10. v7 set the status to Completed. The current record (fetched from /api/v2/studies/NCT02685865 on 2026-10-23; last update posted 2019-02-11) matches v7, and it still gives the primary time frame as "6 weeks".
No version gives a 6-month time frame for the primary outcome. None defines the primary outcome as a count that includes the index procedure, ERCP, enteral feeding tubes or procedures for adverse events.
What the paper reports
The primary outcome, Abstract (Design):
"Primary outcome was comparing total number of procedures to achieve treatment success defined as symptom relief in conjunction with WON resolution on CT at 6 months."
Introduction, final paragraph:
"The primary objective of this randomised trial was to evaluate the effectiveness of LAMS by comparing the total number of procedures required to achieve treatment success between patients undergoing LAMS and plastic stent placement at 6-month follow-up."
Methods > Outcome measures:
"The main outcome measure was to compare the total number of procedures needed to achieve treatment success at 6-month follow-up."
Methods > Definitions:
"The total number of procedures required to achieve treatment success included the index endoscopic intervention, endoscopic reinterventions, percutaneous drain placement, ERCP, enteral feeding tube placement and any procedures performed for the management of adverse events or disease-specific events in each cohort during the 6-month follow-up period."
"Treatment success was defined as resolution of WON on CT scan in association with clinical resolution of symptoms at 6-month follow-up."
Methods > Sample size calculation:
"A two-sided sample size calculation was performed to detect a difference of one in the total number of procedures needed to achieve treatment success (assuming one for LAMS and two for plastic stents)"
Methods > Patient follow-up. This is the only role the 6-week visit has in the paper:
"On initial outpatient follow-up at 6 weeks postintervention, all patients underwent CT to assess treatment response prior to stent removal and an ERCP to assess for pancreatic duct integrity."
The primary result, Abstract (Results) and Table 3 ("Total no. of procedures for treatment success": mean 2.8 (SD 1.2) vs 3.2 (SD 1.5); median 2 vs 3; range 2–7 vs 2–7; p=0.192):
"There was no significant difference in total number of procedures performed (median 2 (range 2–7) LAMS vs 3 (range 2–7) plastic, p=0.192)"
The paper's statements about the registration and the protocol:
- The only registration statement is at the end of the Abstract: "Trial registration number NCT02685865." The words "registered" and "registration" (outside that line) do not appear in the text. The registry's hyphenated term "re-intervention" does not appear either, but "reinterventions" does, five times: as one component of the total procedure count (Methods > Definitions, "endoscopic reinterventions"), in the 3- and 6-month follow-up interviews ("disease-related reinterventions and readmissions") and three times in the Discussion. It is never reported as a separate outcome.
- The only protocol amendment described is in Results > LAMS cohort: "Therefore, the safety board recommended an amendment to the study protocol whereby a CT was obtained at 3 weeks in the LAMS cohort followed by stent removal if the WON had resolved." The Abstract also mentions it: "Interim audit resulted in protocol amendment where CT scan was obtained at 3 weeks postintervention followed by LAMS removal if WON had resolved."
- The only limitation about the primary outcome is in Discussion, limitations paragraph: "Four, we did not report outcomes from a patient perspective and the primary outcome was not adjudicated in a blinded fashion."
- Searches of the archived text for "registered", "primary outcome", "amend", "changed", "protocol", "6 week", "6-week" and "reintervention" found no statement about a change to the primary outcome's definition or time point.
The online supplement (gutjnl-2017-315335supp001.pdf, retrieved from the Europe PMC supplementary-files archive) has four tables: costs; "Procedures performed to achieve treatment success"; Poisson regression; and linear regression. Supplementary Table 2 lists procedure totals by type (LAMS vs plastic): "Repeat EUS-guided drainage 2 / 5", "Endoscopic necrosectomy 4 / 6", "Percutaneous drain placement 2 / 5", "Enteral feeding tube placement 7 / 13", "ERCP for pancreatic duct integrity 31 / 29" and "Management of adverse events 14 / 3". The supplement says nothing about the registration or any outcome change.
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| Number of re-interventions (repeat EUS drainage with the same stent type, DEN or percutaneous catheter, for inadequate response) to treatment success | Primary in v0–v3 (2016-02-18 to 2017-04-21). From v4 (2017-09-13) the title says "interventions", but the description keeps the re-intervention definition (through v7, 2019-02-08) | Not an outcome. The Results text and Supp. Table 2 give only procedure totals by type (LAMS 2+4+2 = 8; plastic 5+6+5 = 16), with no per-patient summary and no test | not reported |
| Time point of the primary outcome | "6 weeks" in every version, v0–v7; description "at 6 week follow-up" | "at 6-month follow-up" (Abstract, Introduction, Methods) | time point changed |
| Total number of procedures to treatment success (index procedure, endoscopic reinterventions, percutaneous drains, ERCP, enteral feeding tubes, procedures for adverse or disease-specific events) | In no version | Primary outcome and basis of the sample size calculation; median 2 vs 3, p=0.192 | added (as primary) |
| Treatment success | Secondary, "6 week"; v0: "<33% of the original size" plus symptoms; v4 (2017-09-13): "Resolution of WON on CT scan" plus symptoms | Secondary, "at 6-month follow-up"; 29 (93.5%) vs 28 (96.6%), p=0.999; also the end point of the primary count | time point changed |
| Technical success, procedure duration, adverse events, length of stay, costs, recurrence, treatment failure | Secondary, v0–v7 (day 0 or 6 months) | Secondary outcomes or definitions in Methods; reported in Table 3 and the text | matched |
Figures
Figure 1. Timeline of NCT02685865 and the paper. Blue marks are registry events (above the axis). Filled circles are the first submission (2015-11-24), v0 (2016-02-18, six days after the actual study start of 2016-02-12) and v4. The diamond is the registered actual primary completion (2017-07-10). Open squares are registry versions that did not change the outcomes (v1–v3, v5, v6, v7); v5 (2018-05-30) sits beside the publication marker. The ringed marker is v4 (2017-09-13), the only version that edited the outcome measures. It falls after the actual primary completion date then registered (2017-07-10; v6 later changed it to 2018-07-10) and nine days before the manuscript was received. Orange marks are the paper (below the axis): the randomisation period it reports (February 2016 to March 2017), receipt (2017-09-22; its axis marker sits under the v4 marker nine days before it, and its stem leads down to the label), online publication (2018-06-01) and print (2019-07). The paper's 6-month primary outcome appears at no point in the registry. v6 (2018-07-31), after publication, moved the actual primary completion date to 2018-07-10. Source: make_figures.py.txt.
Figure 2. On the left, in blue, are the registered outcomes. On the right, in orange, are what the paper reports. Each connector carries a label for its status. Top row: the registered primary (re-interventions to treatment success at 6 weeks) was replaced by a 6-month total procedure count. That count adds the index procedure, ERCP, feeding tubes and procedures for adverse or disease-specific events. Its result was median 2 vs 3, p=0.192, with a minimum of 2 in each arm. The v4 title edit is noted in grey. Middle row (dashed outline: an outcome with no reported result): for the registered primary as defined, the paper gives only procedure totals for its three components (8 vs 16), with no per-patient summary, no test and no 6-week split. Bottom row: the registered secondary, treatment success at 6 weeks, is reported at 6 months. The footer lists the other procedure types in the paper's count (Supplementary Table 2) and the paper's only registration statement and only protocol amendment.
Why it matters
The paper reports no result for the registered primary outcome. The registered measure is a per-patient count of re-interventions done for inadequate treatment response, reaching treatment success at 6 weeks, and the paper gives no between-group comparison of it at any time point. The closest data are procedure totals for the three registered components: 8 in the LAMS arm (2 extra transmural tracts, 4 necrosectomies, 2 percutaneous drains) against 16 in the plastic arm (5, 6 and 5). These come from Results > LAMS cohort and Plastic stent cohort ("Additional procedures undertaken due to suboptimal treatment response at 72 hours (for ongoing SIRS) included creation of additional transmural tracts (n=2), direct endoscopic necrosectomy (n=4) and percutaneous drainage catheter placement (n=2)" and "...(n=5), direct endoscopic necrosectomy (n=6) and percutaneous drainage catheter placement (n=5)") and Supplementary Table 2. These are event totals, not patient-level data. They are not split at 6 weeks and are not tested, so the registered comparison cannot be reconstructed from the paper.
The substituted outcome is built differently in ways that bear on its result:
- Shared fixed components. Every patient's count includes the index procedure. Supplementary Table 2 lists "ERCP for pancreatic duct integrity" as 31 and 29, one per randomised patient, in line with the protocolised 6-week ERCP. Both arms therefore have a floor of 2 procedures, and the observed minimum is 2 in each arm (Table 3). Components that every patient receives add the same amount to both arms and shrink the relative contrast between them.
- Procedures for adverse events. The count includes "any procedures performed for the management of adverse events". Supplementary Table 2 gives 14 such procedures in the LAMS arm and 3 in the plastic arm. These were coil embolisations, endoscopies for bleeding or buried stents, ERCP for biliary stricture, retrieval of migrated stents and removal of a PEG-J tube. The registered outcome did not count them, and in the published count they add to the LAMS total.
- Different window. The registered window ended at 6 weeks. The published window runs to 6 months and takes in readmissions and later procedures.
So the paper's null primary result (p=0.192) and its "non-superiority" headline concern a different end point from the one registered. The totals for the registered components are lower in the LAMS arm (8 vs 16), but no test is given and these are not per-patient data. Whether the registered comparison would have been null or not cannot be judged from what is published. The registered hypothesis itself ("the no. of re-interventions performed will be lower in the FCSEMS group") is never tested as stated.
Two further points make the published primary harder to interpret:
- Sample-size assumptions. The power calculation assumes means of "one for LAMS and two for plastic stents" in "the total number of procedures". Under the published definition, every patient has at least two procedures (index plus ERCP), so an assumed mean of one in the LAMS arm is not possible. The paper does not explain how the assumptions relate to the definition.
- Totals that do not add up. The procedure totals in Supplementary Table 2, plus one index procedure per patient, come to 91 (LAMS) and 90 (plastic). These correspond to means of 2.94 and 3.10, against the reported 2.8 and 3.2 in Table 3, so the per-patient composition of the published count cannot be checked from the supplement either.
This is not a case where a null registered result became a positive headline: the published primary is itself null, and both outcomes are procedure counts. But the paper presents the 6-month total count as the trial's primary outcome. Readers, reviewers and meta-analysts who take it at face value would not know that the registered primary, measured at 6 weeks and restricted to re-interventions, is not reported.
Explanations considered
- Paper role. This is the trial's primary report: 60 patients were randomised, matching the registered actual enrolment of 60, and the paper gives full efficacy, safety and cost results with a sample-size calculation for the primary outcome. It is not a secondary or follow-up analysis that could legitimately use a different end point. This does not explain the discrepancy.
- Disclosure. None. The registration is cited only as "Trial registration number NCT02685865." The text searches listed above found no statement that the primary outcome, its time point or its counting rules differ from the registration. The supplement contains only results tables. Not disclosed.
- Protocol amendments. The paper describes one amendment, recommended by the safety board: "a CT was obtained at 3 weeks in the LAMS cohort followed by stent removal if the WON had resolved." It concerns imaging and stent removal in one arm. It does not concern the primary outcome's definition, its time point or which procedures are counted. A legitimate amendment to the outcome could exist, but the paper cites no published protocol or statistical analysis plan, the registry has no posted documents, and the change is not described. A real but unreported amendment would explain why the outcome differs, but not why the paper does not say so.
- Registry edit in v4. The v4 edit (2017-09-13) changed the title from "re-interventions" to "interventions". It could be read as a move towards a broader count. It did not, however, change the time frame (still 6 weeks), and the description still defines the counted events as re-interventions for inadequate response. The registry never described the paper's 6-month count. The edit was made after the then-registered actual primary completion date (2017-07-10; changed to 2018-07-10 in v6) and is not mentioned in the paper either.
- Wording differences. The registry's "metal" and "FCSEMS" arm is the paper's LAMS. The registry lists a "Hot Axios Fully covered self-expandable metal stent", and the paper describes the Hot AXIOS LAMS as "fully covered self-expandable metal stents", so the device is the same. "Number of procedures" and "number of re-interventions" could look like loose synonyms. They are not, because the paper explicitly adds the index procedure, ERCP, feeding tubes and adverse-event procedures. Six weeks and six months cannot be a wording difference.
- The 6-week visit as the assessment point. One reading is that the registry's "6 weeks" refers to the first outpatient CT visit, which still exists in the paper. But the registry ties both the count and treatment success to "6 week follow-up". The paper defines treatment success, which closes the count, "at 6-month follow-up", and it counts procedures "during the 6-month follow-up period". The paper's 6-week visit is the CT and ERCP and stent-removal visit, not the end point of the primary outcome.
Status
As of 2026-10-23:
- Registry history. All eight versions of NCT02685865 were reviewed: v0 2016-02-18, v1 2016-04-12, v2 2017-02-13, v3 2017-04-21, v4 2017-09-13, v5 2018-05-30, v6 2018-07-31 and v7 2019-02-08. The source was the ClinicalTrials.gov history endpoints (
/api/int/studies/NCT02685865/historyand/history/<version>), from copies retrieved for this project on 2026-10-22 and 2026-10-23; the live history endpoint returned HTTP 403 on 2026-10-23. The current record (/api/v2/studies/NCT02685865, fetched 2026-10-23) matches v7. No results are posted to the registry. - Publisher version. The text checked is the PMC archive of the version of record: Gut 2019;68(7):1200–1209, received 2017-09-22, revised 2018-04-09, accepted 2018-05-02, epub 2018-06-01. The online supplement (Supplementary Tables 1–4) was read from the Europe PMC supplementary-files archive. The paper cites no published protocol or statistical analysis plan.
- Corrections. No correction, erratum, expression of concern or retraction was found in Crossref (no
update-torelation), in PMC notices or in Europe PMC notices. - Letters. Europe PMC lists five "Comment in" items (PMIDs 30030302, 30301769, 30368455, 30389713 and 30448773, the last an authors' reply). Only one was read, doi:10.1136/gutjnl-2018-317777 (PMID 30389713), from an institutional repository copy. It discusses the primary outcome "total number of procedures needed to achieve treatment success at 6-month follow-up" but does not raise the registration. The authors' reply (PMID 30448773, doi:10.1136/gutjnl-2018-317848, "Authors reply: treatment strategy for necrotising pancreatitis", Gut 2020;69:189–190) is the most likely place for any post-publication explanation of the outcome definition, but it could not be accessed: Europe PMC has no full text, and the free publisher PDF that Unpaywall lists, the article page and an alternate BMJ host all returned HTTP 403 on 2026-10-23. A search-engine excerpt of it covers only the drainage algorithm (multigate drainage, CT at 72 hours), so whether it mentions the outcome definition or the registration is unknown. The other three letters were not read; none is open access in Europe PMC.
- PubPeer. The PubPeer API and site returned a bot-verification page (HTTP 403), and a web search found no PubPeer thread for this article. Whether it has PubPeer comments is unconfirmed.
- Sources. ClinicalTrials.gov record and history tab; PMC6582745; Crossref metadata; Europe PMC.
Suggested correction
The authors and the journal could add a statement to Methods > Outcome measures, along these lines:
"The primary outcome registered at ClinicalTrials.gov (NCT02685865; first submitted 24 November 2015) was the number of re-interventions (repeat EUS-guided drainage using the same stent type, direct endoscopic necrosectomy, or percutaneous catheter insertion for inadequate treatment response) needed to achieve treatment success at 6-week follow-up. The primary outcome reported here, the total number of procedures (including the index procedure, ERCP, enteral feeding tubes and procedures for adverse events) needed to achieve treatment success at 6-month follow-up, differs from the registered outcome in its definition and time point. [Date of and reason for the change.]"
The correction could also report the registered primary outcome as prespecified: the per-patient number of re-interventions to treatment success at 6 weeks, by arm, with the planned between-group test, or say why it cannot be computed. Treatment success at 6 weeks, the registered secondary, could be reported too. If a protocol amendment or statistical analysis plan set up the 6-month total count, making that document and its date available would let readers check when the end point was fixed relative to unblinded data. The registry record could be updated to give the outcome that was analysed, with a note on when it changed.
Trial report names an unregistered primary outcome and omits registered primary HIV risk behavior
JAMA Intern Med (2014) · doi:10.1001/jamainternmed.2014.2556 · PMC4811188 · NCT00987961
Summary
ClinicalTrials.gov record NCT00987961 has listed the same two primary outcomes, "opioid use" and "HIV risk behavior", in every version from the first (2009-09-30) to the last (2015-11-10). The published primary report calls two outcomes its "prespecified primary outcome variables": entry into buprenorphine treatment and length of illicit opioid use. Illicit opioid use matches the registered "opioid use" outcome. Entry into treatment, which gives the abstract its headline result (72.2% vs 11.9%), is not a registered primary or secondary outcome, and the registered primary outcome HIV risk behavior is not reported or mentioned anywhere in the paper. The paper cites the registration only by its identifier and does not mention these differences.
What was registered
The registry record has 11 versions (v0 to v10). The ClinicalTrials.gov history summary reports primaryOutcomesSame: true, secondaryOutcomesSame: true, outcomesUpdateCount: 0 and lastUpdateVersions.primaryOutcomes: 0. The outcomes were therefore set in v0 and never edited. Versions v1 to v10 changed only the Study Status, Sponsor/Collaborators, Contacts/Locations or Study Design modules.
| Version | Submitted | Primary outcomes (registry wording) | Time frame |
|---|---|---|---|
| v0 (first version) | 2009-09-30 (first posted 2009-10-01) | "opioid use"; "HIV risk behavior" | "baseline, 1-month, 3-months, 6-months" (both) |
| v1-v6 | 2010-03-23 to 2013-07-08 | unchanged | unchanged |
| v7 (last before the print issue) | 2014-07-16 | unchanged | unchanged |
| v8-v10 (after publication) | 2015-05-21 to 2015-11-10 | unchanged | unchanged |
The registered secondary outcomes, also unchanged since v0, are "reduction in injection-related medical conditions" and "reduction in emergency department and hospital utilization", with the same time frame.
The descriptive fields say the same thing. The Detailed Description ends: "Primary outcomes are opioid use and HIV risk behavior." The Brief Summary says the study "tests whether starting Suboxone (buprenorphine) during a medical hospitalization, and then providing an appointment (a "link") for after discharge to maintenance buprenorphine in an outpatient setting will reduce HIV risk behavior in individuals who inject opioids." "HIV risk behavior" is also one of the record's three keywords, in v0 and in the current version.
Timing: the registry gives the study start as 2009-09. The paper says patients were identified "From August 1, 2009", so the first registration came about two months after recruitment began. No registry version exists from before enrollment started, but the outcomes in v0 were entered during recruitment, years before the data were complete. Primary completion is given as 2014-03 (actual); v0 estimated 2013-12.
What the paper reports
The paper's primary-outcome definition (Methods, "Primary Outcome Variables"):
The prespecified primary outcome variables were entry into buprenorphine treatment (in-person intake [anytime between study enrollment and 6 months after enrollment]) at the hospital-associated OAT program, confirmed by OAT electronic medical record review, and length of illicit opioid use (number of days of reported opioid use in the 30 days before the 1-, 3-, and 6-month interviews using a standard 30-day time-line follow-back method).
The Introduction (final paragraph) sets out the same two goals, treatment entry first:
This study had 2 primary goals. We sought to determine whether offering hospitalized, opioid-dependent patients initiation and linkage to office-based opioid addiction treatment with buprenorphine would facilitate entry into and increase persistence in buprenorphine treatment. We then examined whether this treatment initiation and linkage would decrease illicit opioid use at 6 months.
The Abstract, Main Outcomes and Measures:
Entry and sustained engagement with buprenorphine OAT at 1, 3, and 6 months (medical record verified) and prior 30-day use of illicit opioids (self-report).
Secondary outcomes (Methods, "Secondary Outcome Variables"): "Secondary outcome variables included time to entry into the buprenorphine program (days to in-person intake at the hospital-associated OAT program, confirmed by electronic medical record review) and OAT days".
The sample-size statement (Methods, Statistical Analysis) concerns opioid use only: "Specifically, the study was powered (1 – β > .8) to detect a 33% between-group difference in the rate of opioid use during follow-up (days of use per 30 follow-up days)." The Discussion's limitations paragraph refers to a single primary outcome: "our primary outcome measure (past 30-day illicit opioid use) also depended on self-report".
On the registration, the paper's only statement is the back-matter line "TRIAL REGISTRATION clinicaltrials.gov Identifier: NCT00987961". Searches of the archived text found no occurrence of "HIV" (the only related mention is "human immunodeficiency virus disease", once as background in the Introduction and once as a discharge diagnosis in Results), no occurrence of "risk behavior", "registered" or "amend", and no use of "protocol" except for the treatment protocol. The paper cites no trial protocol, statistical analysis plan or supplement (Europe PMC: no supplementary files).
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| Opioid use / illicit opioid use | Primary: "opioid use", "baseline, 1-month, 3-months, 6-months" (v0, 2009-09-30; unchanged to v10) | Primary: days of illicit opioid use in the 30 days before the 1-, 3- and 6-month interviews; IRR 0.60 (95% CI 0.46-0.73) | matched |
| HIV risk behavior | Primary: "HIV risk behavior", "baseline, 1-month, 3-months, 6-months" (v0, 2009-09-30; unchanged to v10) | Not named, defined or reported | not reported |
| Entry into buprenorphine treatment | Not registered as primary or secondary in any version | "Prespecified" primary: in-person intake at the hospital-associated OAT program by 6 months, record-verified; 52/72 (72.2%) vs 8/67 (11.9%) | added |
| Injection-related medical conditions | Secondary (v0, 2009-09-30; unchanged) | Not reported as an outcome | not reported |
| Emergency department and hospital utilization | Secondary (v0, 2009-09-30; unchanged) | Not reported as an outcome; the Discussion cites a prior study linking injection drug use to such use | not reported |
| Time to treatment entry; OAT days | Not registered | Secondary outcomes | added |
Figures
Figure 1. Timeline of the ClinicalTrials.gov record and the trial. The filled blue marker is v0 (2009-09-30), the only version that defined the outcomes. The hollow markers are v1 to v10, none of which edited the primary or secondary outcomes, before or after the registry's actual primary completion date (2014-03). The orange lane shows dates from the paper: recruitment from 2009-08-01 to 2012-10-31 (so v0 came about two months after recruitment began), presentation of the results at two meetings in April and June 2013, and the print issue of 2014-08-01. The Crossref DOI record was created on 2014-07-01; the online-first date could not be confirmed from the publisher. Study start and primary completion are month-level registry dates, drawn mid-month.
Figure 2. Registered primary outcomes (left; v0, 2009-09-30, unchanged in all later versions) against the outcomes the paper calls its prespecified primary outcomes (right; Methods, "Primary Outcome Variables"). Registered "opioid use" matches the paper's days of illicit opioid use, with the same 1-, 3- and 6-month time points; the paper reports IRR 0.60 (95% CI 0.46-0.73) for it. Registered "HIV risk behavior" has no counterpart in the paper. Entry into buprenorphine treatment appears as a primary outcome in the paper but in no version of the registration, and it carries the abstract's headline result. The note at the foot covers secondary outcomes: neither registered secondary is reported, and the paper's two secondaries were not registered.
Why it matters
The abstract's first and largest result is for an outcome the registration does not list: "linkage participants were more likely to enter buprenorphine OAT than those in the detoxification group (52 [72.2%] vs 8 [11.9%], P < .001)". The abstract conclusion that linkage "is an effective means for engaging medically hospitalized patients who are not seeking addiction treatment" rests on this outcome. Calling it "prespecified" leads readers to treat it as a confirmatory primary result, but the public record does not show that it was prespecified as an outcome at all. Entry into the linked program is also an outcome that follows closely from the intervention itself (the linkage arm was given an intake appointment and a bridging prescription), which makes the choice of primary outcome especially relevant to how the result is read.
For the registered primary outcomes, the paper reports one and not the other:
- Opioid use (registered primary): reported, favourable to linkage. "On the basis of complete case analysis, the estimated rate of illicit opioid use in those in the linkage group was approximately 40% lower (IRR, 0.60; 95% CI, 0.46-0.73; P < .01) than for participants randomized to the detoxification group." Worst-case substitution (IRR 0.73, 95% CI 0.65-0.83) and last observation carried forward (IRR 0.72, 95% CI 0.64-0.81) gave similar results. The second abstract conclusion ("reduces illicit opioid use 6 months after hospitalization") therefore rests on a registered primary outcome. Two caveats come from the paper itself. The abstract describes the IRR as applying to "the 30 days before the 6-month interview", while Results present it as a rate over the whole follow-up. And the outcome is self-reported, with 59.0% follow-up at 6 months that differed by arm ("The linkage group had a significantly higher rate of follow-up at 6 months (χ2 = 6.75, P = .009).").
- HIV risk behavior (registered primary): not reported. The paper gives no result, does not mention it as an outcome, and does not point to another report. Its direction and size cannot be judged from the paper. The registry's Brief Summary presents HIV risk behavior as the outcome the study tests, so its absence changes what the trial can be said to have shown.
The registered secondary outcomes (injection-related medical conditions; emergency department and hospital use) are also not reported. The Discussion suggests the reduction in opioid use "is likely to translate into improved health outcomes", and the Conclusions say future work "should evaluate whether decreased illicit opioid use and increased use of OAT in the 6 months after hospital discharge could have health benefits and prevent subsequent hospitalizations". Emergency department and hospital use was, however, a registered secondary outcome of this trial, with a time frame of baseline to 6 months.
The paper does acknowledge multiplicity in general terms ("Given multiple outcomes, readers concerned about overall type I error rates may consider P < .01 a conservative standard to evaluate statistical significance."), but a reader cannot tell from it how many outcomes were planned or which ones were dropped.
Explanations considered
- Paper role. This is the primary report of the randomized trial: it reports the full randomized sample (139 randomized; 67 detoxification, 72 linkage), and the registry links it as the trial's results publication. The only other publication the registry links (PMID 24045030, Addict Behav 2013) is, by its title, an analysis of days of heroin use and self-reported health among hospitalized heroin users, published before the registry's primary completion date; it is not a report of HIV risk behavior outcomes. The paper does not say HIV risk behavior is being reported elsewhere. This does not account for the discrepancy.
- Disclosure. The paper does not mention the registered outcomes, HIV risk behavior as a planned outcome, an amendment, or any difference from the registration. It calls its primary outcomes "prespecified" without saying where they were prespecified. No protocol or analysis plan is cited or supplied. Nothing in the paper accounts for the discrepancy.
- Protocol amendments. The registry history shows no outcome edit in any of the 11 versions (outcomesUpdateCount 0; primary outcomes last updated in v0), including the versions submitted after primary completion and after publication. An amendment that was made in a protocol but never entered in the registry or the paper cannot be excluded, but it is not documented anywhere available to readers. It may well have legitimate reasons; the issue documented here is that it is not disclosed.
- Wording differences. "Opioid use" and "length of illicit opioid use" are the same construct at the same time points, so that pair is treated as matched. Entry into buprenorphine treatment cannot be read as a rewording of "HIV risk behavior", or of either registered secondary outcome. The paper's "persistence in"/"sustained engagement" in treatment likewise has no registered counterpart.
- Time-point differences. None for opioid use: both use 1, 3 and 6 months (the registry adds baseline). Time points do not explain the other differences.
- Registration timing. v0 was submitted about two months after recruitment began. That makes v0 a registration made during recruitment rather than before it, but it is still the only public statement of the trial's outcomes and long predates the results. It does not explain why the paper's primary outcomes differ from it.
Status
As of 2026-10-23:
- Registry history. The ClinicalTrials.gov version list and history summary (11 versions, v0 to v10, 2009-09-30 to 2015-11-10; primaryOutcomesSame true, outcomesUpdateCount 0) were retrieved from the ClinicalTrials.gov history API on 2026-10-22, together with the full v0 record. Versions v6 (2013-07-08, the last before the likely online-first date) and v7 (2014-07-16, the last before the print issue) were fetched individually in an earlier check of this record and list the same two primary outcomes. On 2026-10-23 the history endpoints returned HTTP 403. The current record (v2 API, fetched 2026-10-23, last updated 2015-11-10) still lists "opioid use" and "HIV risk behavior" as the primary outcomes. An independent re-check, which could not reach the history endpoints, found an Internet Archive snapshot of the record dated 2011-10-19 with the same primary and secondary outcomes.
- Publisher version. The journal page (jamanetwork.com, by DOI and full-article URL) returned 403 or a CAPTCHA, so the quoted text could not be checked against the publisher's version of record, and no publisher correction notice could be looked for there. The quotes above are from the PubMed Central copy (released 2016-03-29), which appears to be the NIH-deposited author manuscript.
- Corrections. No corrections, retractions, expressions of concern or update notices were found in PubMed Central, Crossref (no update-to or relation entries) or Europe PMC. Europe PMC links one item, PMID 24978013 (JAMA Intern Med 2014;174(8):1377-8), as "Comment in"; that is a commentary, not a correction.
- PubPeer. The search returned a human-verification block, so no PubPeer result could be obtained.
- COMPare. Not applicable: it covered trials published in 2015-16 in five journals, not including JAMA Internal Medicine.
Sources: ClinicalTrials.gov NCT00987961 (current record, version history and v0, v6, v7 records); PMC4811188 as archived in PubMed Central; Crossref and Europe PMC metadata for doi:10.1001/jamainternmed.2014.2556.
Suggested correction
A correction or clarification notice could:
- State the primary and secondary outcomes as registered at ClinicalTrials.gov (NCT00987961, first submitted 2009-09-30 and unchanged since): opioid use and HIV risk behavior at baseline, 1, 3 and 6 months; secondary outcomes injection-related medical conditions and emergency department and hospital use.
- Say when and why entry into buprenorphine treatment became a primary outcome and HIV risk behavior was dropped, and point to the document (protocol, grant application or analysis plan) in which the outcomes described as "prespecified" were set out, ideally making that document available.
- Report the results for HIV risk behavior and for the registered secondary outcomes, or say where they are reported or why they cannot be.
- Label entry into buprenorphine treatment in the abstract and Methods as an outcome not listed in the trial registration, and make the abstract's description of the illicit opioid use IRR match the Results.
The registry record could also be updated with a note describing the outcomes as analysed, so that readers of either source can see the relationship between them.
Registered primary outcome is overall survival; the final report uses melanoma-specific survival without comment
N Engl J Med (2014) · doi:10.1056/NEJMoa1310460 · PMC4058881 · NCT00275496
Summary
Every version of the ClinicalTrials.gov record for MSLT-I (NCT00275496), from the first on 2006-01-10 to the current one dated 2015-09-01, gives one primary outcome: whether sentinel-node biopsy with selective lymphadenectomy "will effectively prolong overall survival", with a 10-year time frame added in 2009. The final trial report states instead that "The primary end point was melanoma-specific survival (survival until death from melanoma)." The paper does not mention the registered outcome or explain the difference, and it reports no overall-survival estimate, hazard ratio or curve. The only all-cause mortality information in the paper is a set of crude death counts in the Fig. 1B flow diagram. The trial was registered about 12 years after it started, so the registry entry may not reflect the original protocol. Even so, the registered primary outcome had been public for eight years before publication and the paper does not reconcile the two.
What was registered
ClinicalTrials.gov has 10 versions of the record. All of them postdate the study start (1993-11) and the end of enrollment (2002). The primary outcome measure text is the same, character for character, in all 10:
To determine whether wide excision of the primary with intraoperative lymphatic mapping (LM) followed by selective lymphadenectomy will effectively prolong overall survival compared to wide excision of the primary melanoma alone.
| Version | Date | Primary outcome | What changed in Outcome Measures |
|---|---|---|---|
| v0 | 2006-01-10 (first submitted; first posted 2006-01-12) | text above, no time frame | first registration |
| v1 | 2006-01-20 | unchanged | secondary only: "Sentinel node metastases in WEX arm; Incidence of nodal metastases" became "Incidence of Sentinel Node Metastases (biopys) vs clinical metastases (observation)" |
| v2, v3 | 2006-11-29, 2007-11-02 | unchanged | none (Study Status, References) |
| v4 | 2009-04-30 | unchanged; time frame "10 years" added | "10 years" added to primary and secondary; "biopys" corrected to "biopsy" |
| v5, v6 | 2011-11-07, 2012-07-30 | unchanged | none (v6 records primary completion 2012-06, actual) |
| v7 | 2012-11-19 (last version before publication) | unchanged, "10 years" | none (Study Status, References) |
| v8, v9 | 2014-05-13, 2015-09-01 | unchanged, "10 years" | none (Study Status only) |
The registered secondary outcomes in v4 to v9 are: "Disease-free survival; Incidence, timing, and anatomic distribution of distant metastases; Morbidity of procedures; Significance of TA90 levels; Incidence of Sentinel Node Metastases (biopsy) vs clinical metastases (observation); Accuracy of LM" (time frame "10 years"). Melanoma-specific survival does not appear as a primary or secondary outcome in any version.
Other registry dates: study start 1993-11; primary completion 2012-06 (actual); last update posted 2015-09-02. The current record lists this paper (PMID 24521106) as a "derived" citation and three "result" citations: the 2006 third interim report (N Engl J Med 2006 Sep 28, PMID 17005948); a 2010 Ann Surg Oncol paper on morbidity and length of stay after early versus delayed completion lymphadenectomy (PMID 20614193); and a 2012 Ann Surg Oncol paper on metastasectomy for distant metastases in MSLT-I patients (PMID 22648554). The two Ann Surg Oncol citations first appear in v7. The registry records masking as None (open label) in every version.
What the paper reports
The primary end point, in Methods, Statistical Analysis (first paragraph):
The primary end point was melanoma-specific survival (survival until death from melanoma). The secondary end points, described previously,11,12 included disease-free survival, survival with tumor-positive or tumor-negative sentinel nodes, and the incidence of sentinel-node metastases, as compared with the incidence of clinically detected nodal metastases.
References 11 and 12 are the 2006 third interim report in the same journal and a 1999 Ann Surg report on the accuracy of lymphatic mapping (PMC1420894). Neither is a protocol or a statistical analysis plan.
The registration appears once, at the end of the abstract:
(Funded by the National Cancer Institute, National Institutes of Health, and the Australia and New Zealand Melanoma Trials Group; ClinicalTrials.gov number, NCT00275496.)
The main results for the paper's own primary end point (abstract; Results, Survival Rates):
No significant treatment-related difference in the 10-year melanoma-specific survival rate was seen in the overall study population
hazard ratio for death from melanoma in the biopsy group, 0.84; 95% confidence interval [CI], 0.64 to 1.09; P = 0.18
This is for patients with intermediate-thickness melanomas (1.20 to 3.50 mm), the primary study group.
The analysis methods name only melanoma-specific and disease-free survival (Methods, Statistical Analysis, third and fourth paragraphs):
The Kaplan–Meier method was used to estimate mean 5-year and 10-year rates of melanoma-specific and disease-free survival.
Comparisons of melanoma-specific and disease-free survival were based on data from patients who underwent their assigned treatment.
The Discussion (fourth paragraph) refers to survival without saying which kind:
We did not see a significant survival advantage for all patients with intermediate-thickness melanomas.
The design changes the paper does disclose are a larger sample size ("After the second interim analysis, the sample size for the intermediate-thickness group was increased to 1200 patients") and the latent-subgroup method ("In post hoc analyses, a latent-subgroup analytic method was developed"). Both are in Methods, Statistical Analysis.
What the paper does not contain: searches of the full text as archived in PMC (abstract, body, table, figure legends) for "overall survival", "all-cause", "any cause" and "amend" find nothing. Every use of "protocol" refers to the per-protocol analysis. The one use of "registered" refers to enrollment ("Enrollment closed in 2002, after 2001 patients had been registered."). The Supplementary Appendix archived with the PMC manuscript (16 pages) has no overall-survival analysis and no statement about end points or protocol amendments. Its survival analyses are all melanoma-specific, disease-free or distant disease-free.
All quotations above were checked verbatim with src/show.py PMC4058881 --grep, and each full quotation occurs exactly once in the archived text. The Fig. 3 legend repeats the definition "survival until death from melanoma". Registry quotations were checked against the archived version records and the current API v2 record.
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| Overall survival (death from any cause), 10 years | Primary in every version, v0 2006-01-10 to v9 2015-09-01 (time frame "10 years" from v4 2009-04-30; v7 2012-11-19 is the last version before publication) | Not named as an end point. No estimate, hazard ratio or curve. Fig. 1B gives only crude counts of deaths by cause | not reported |
| Melanoma-specific survival, 10 years | In no version, as either primary or secondary | "The primary end point was melanoma-specific survival" | added (as primary) |
| Time frame of the primary outcome | 10 years (from v4 2009-04-30) | 10-year rates | matched |
| Disease-free survival, 10 years | Secondary, all versions (time frame "10 years" from v4 2009-04-30) | Secondary. The abstract's conclusions cite it first as a treatment benefit | matched |
| Incidence of sentinel-node metastases vs clinical metastases | Secondary, v1 onward | Secondary ("the incidence of sentinel-node metastases, as compared with the incidence of clinically detected nodal metastases") | matched |
| Distant metastases (incidence, timing, distribution) | Secondary, all versions | Reported as distant disease-free survival in the node-positive subgroup (Results; Fig. S2) | matched (as distant disease-free survival) |
| Survival with tumor-positive or tumor-negative sentinel nodes | In no version | Secondary | added (as secondary) |
The core finding is two rows: the registered primary outcome (overall survival) is not reported, and a different measure (melanoma-specific survival) is reported as the primary end point. Neither the substitution nor the reason for it is stated in the paper.
Figures
Figure 1. Timeline of the trial and its registry record. The trial started in 1993-11 (registry; the paper says it "commenced in 1994") and enrolled 2001 patients until 2002. It was first registered on 2006-01-10, about 12 years after the start. Every registry version therefore postdates the study start and the end of enrollment. Diamonds mark the versions that created or edited the Outcome Measures module: v0 (primary outcome overall survival), v1 (secondary wording only) and v4 (time frame "10 years" added to the unchanged primary). Open circles mark versions that changed other modules. The hatched band, bounded by the dashed line, is the period after primary completion (2012-06). The four versions in it did not touch outcomes: v6 (2012-07-30, Study Status; status set to Completed and primary completion 2012-06 recorded as actual), v7 (2012-11-19, Study Status and References; the last version before publication), and v8 (2014-05-13) and v9 (2015-09-01), Study Status only. The final report (2014-02-13) names melanoma-specific survival as its primary end point. Sources: ClinicalTrials.gov record history; the paper as archived in PMC; the registry's reference to the 2006 interim report.
Figure 2. Registered primary outcome versus the primary end point in the paper. Left, in blue: the registry's primary outcome as of v7 (2012-11-19), the last version before publication, worded the same since v0 (2006-01-10). Right, in orange: the paper's stated primary end point. The two use the same 10-year time frame but measure different things: overall survival counts deaths from any cause, while melanoma-specific survival counts only deaths from melanoma. The paper gives no overall-survival result. The nearest data are the crude counts of deaths by cause in the Fig. 1B flow diagram, which are not time-to-event estimates and do not account for losses to follow-up. Melanoma-specific survival appears in no registry version. Its result in the primary study group is null (hazard ratio 0.84, 95% CI 0.64 to 1.09, P = 0.18). Disease-free survival, a registered secondary outcome, is reported as a secondary end point.
Why it matters
Overall survival and melanoma-specific survival differ in more than wording. Melanoma-specific survival treats deaths from other causes as censored, so a treatment effect on non-melanoma mortality cannot appear in it. It also depends on cause-of-death attribution, which may not be blinded in an open surgical trial (the registry records masking: None in every version). For a comparison of an extra surgical procedure (sentinel-node biopsy, with immediate completion lymphadenectomy when positive) against observation, all-cause mortality is the outcome the registry named, and it captures any harm or benefit regardless of cause.
What the paper reports for the registered primary outcome: no estimate. No overall-survival rate, hazard ratio, P value or Kaplan–Meier curve appears in the abstract, main text, Table 1, figure legends or the archived Supplementary Appendix. The only related information is in the Fig. 1B flow diagram (image), which lists deaths by cause among patients included in follow-up. For intermediate-thickness melanomas: biopsy group 133 died from melanoma and 47 from other causes, 180 of 805 in all (22.4%); observation group 103 and 25, 128 of 522 (24.5%). For thick melanomas: biopsy 68 and 19, 87 of 185 (47.0%); observation 43 and 9, 52 of 126 (41.3%). These are crude proportions with unequal losses to follow-up (120 and 68 in the intermediate groups; 18 and 9 in the thick groups). They are not a substitute for the registered time-to-event analysis, and this write-up draws no inference about treatment effect from them. They show that the paper did record deaths from other causes, so an overall-survival analysis was possible from the trial data.
The abstract's positive conclusions rest on outcomes other than the registered primary: disease-free survival (a registered secondary), melanoma-specific and distant disease-free survival in the node-positive subgroup (a comparison the paper itself says raises "concern about ascertainment bias"), and a post hoc latent-subgroup analysis. The paper's own primary end point is stated openly as null in the whole primary study group. A reader comparing the paper with the registry cannot tell from the paper whether the registered overall-survival comparison was done or what it showed.
Explanations considered
- Paper role. This is the final report of the main trial: the title says "Final Trial Report", and the paper reports 10-year results for the primary study group and names its own primary end point. It is the report in which the registered 10-year primary outcome would be expected. It is not a secondary analysis or a subgroup paper. Does not account for it.
- Disclosure in the paper. The registry number appears only in the abstract's funding line. The paper does not mention overall survival, the registered outcome, or any difference from the registry. The secondary end points are said to be "described previously" (refs 11 and 12), but the paper gives no comparable reference or explanation for its primary end point. Not disclosed.
- Protocol amendment. The paper discloses two design changes, the sample-size increase after the second interim analysis and the post hoc latent-subgroup method, but no change to the primary end point. No protocol or statistical analysis plan is cited or provided. The archived Supplementary Appendix contains none. The registry's primary outcome never changed across 10 versions, so there is no registry-side amendment either. No amendment documented in either source.
- Wording differences. "Overall survival" and "melanoma-specific survival" are standard, distinct end points, and the paper defines its own narrowly as "survival until death from melanoma". The registry entry is phrased as an objective ("To determine whether…"), not a formal measure, but "overall survival" is explicit in it. Cannot be read as a rewording of the same outcome.
- Time point. Both are 10-year outcomes (registry time frame "10 years" from v4; paper reports 10-year rates). No time-point issue.
- Retrospective registration. The trial was registered in 2006, after enrollment had closed in 2002. The registry entry may therefore not match the original protocol, and there is no pre-start version to compare. The abstract of the 2006 third interim report already reported melanoma-specific and disease-free survival. I could not read that report's full text to see whether it names a primary end point. This could explain why the investigators used melanoma-specific survival. It does not explain why the final report leaves a public registration's primary outcome unmentioned and unreported. Mitigates the significance of the change, not the non-disclosure.
- Outcomes split across papers. The registry lists a single primary outcome. Results for thin melanomas are described as "exploratory" and "not reported on in this article", but that concerns a cohort, not the overall-survival end point. The registry links three MSLT-I "result" publications besides this paper. One is the 2006 interim report; neither of the other two reports 10-year overall survival by randomized group: the 2010 Ann Surg Oncol paper (PMID 20614193) compares morbidity and length of stay after early versus delayed lymphadenectomy, and the 2012 Ann Surg Oncol paper (PMID 22648554) compares survival after a stage IV diagnosis by treatment received (surgery versus systemic therapy alone). Both full texts were read in PMC (PMC2970739, PMC3405182). I did not find a separate MSLT-I publication reporting 10-year overall survival as the primary outcome. Does not account for it on present evidence.
Status
As of 2026-10-23:
- Registry history: all 10 versions read through the ClinicalTrials.gov internal history API (
/api/int/studies/NCT00275496?history=trueand/history/0to/history/9; the version list and v0 cached 2026-10-22, v1 to v9 cached 2026-10-23; these endpoints now return HTTP 403 from this machine, so the cached copies were used) and the current record through API v2 (re-fetched 2026-10-23). The primary outcome is unchanged in every version, and the current record still lists overall survival. - Publisher version: the text checked is the NIH author manuscript archived in PMC (PMC4058881), including the Fig. 1 flow-diagram image and the Supplementary Appendix PDF archived with it. The publisher's page at nejm.org returned HTTP 403, so the version of record there was not compared directly.
- Corrections:
src/known.pyfound no Crossref update-to records, no PMC notices and no Europe PMC correction or retraction notices for 10.1056/NEJMoa1310460. The reference list's erratum note belongs to the 2006 interim report (ref 11), not this paper. Europe PMC links published comments (an editorial in the same journal, N Engl J Med 2014;370:663-4, correspondence, and comments in other journals) but no erratum. - Correspondence: the May 2014 letters and authors' reply (N Engl J Med 2014;370(22):2148-50, doi:10.1056/NEJMc1402989). The authors' reply is archived in PMC (PMC4522311) and was read in full. It answers points about recurrence prevention, the nodal-metastasis curves, subgroup offsetting and earlier trials, and gives a whole-population distant disease-free survival result that is not in the paper (hazard ratio 0.89, P = 0.34). It does not mention overall survival, the registration or the choice of primary end point. The letters themselves (PMIDs 24869727 and 24869728) are not in PMC and nejm.org returned HTTP 403, so they were not read; what the reply answers suggests their topics, but whether any letter raised overall survival is unknown.
- PubPeer: the PubPeer API returned HTTP 403 (Cloudflare challenge), so PubPeer was not checked.
- Sources: ClinicalTrials.gov NCT00275496 (current record and version history); the article as archived in PubMed Central; Europe PMC records for this paper and for the 2006 and 2005 MSLT-I reports; the authors' reply to the May 2014 correspondence (PMC4522311); the two registry-linked Ann Surg Oncol papers (PMC2970739, PMC3405182); Crossref.
Suggested correction
The authors and journal could add a correction or a note to the article that:
- States that the ClinicalTrials.gov record (NCT00275496, first registered 2006-01-10) lists overall survival at 10 years as the primary outcome, and that the paper's primary end point is melanoma-specific survival.
- Says when and why the primary end point was set as melanoma-specific survival. If the original pre-2006 protocol named melanoma-specific survival, the correction could say so and make that protocol available.
- Reports the registered primary outcome: 10-year overall survival by randomized group in the intermediate-thickness (primary study) group, with the Kaplan–Meier estimates, hazard ratio, 95% CI and P value, analyzed the same way as the melanoma-specific survival results (and by intention to treat).
- If the registry entry does not reflect the protocol, the sponsor could update ClinicalTrials.gov with a dated note explaining the difference, so the record and the publication agree.
HTC uptake, the trial's first primary outcome, appears in no version of its registration
PLoS Med (2014) · doi:10.1371/journal.pmed.1001768 · PMC4267810 · NCT01459120
Summary
This cluster-randomized trial compared home-based HIV testing and counseling (HB-HTC) with mobile clinic HTC (MC-HTC) in rural Lesotho. The paper names uptake of HTC as its first primary outcome, states its main hypothesis in terms of uptake, bases its sample size on uptake, and leads the abstract results with uptake (92.5% vs 86.7%, aOR 2.06, p = 0.011). None of the four versions of ClinicalTrials.gov record NCT01459120 lists HTC uptake as a primary or a secondary outcome, and the registered outcomes never changed between versions. The paper has a section on deviations from the registration, but it discloses only that a registered cluster-level outcome was dropped, and it describes uptake as one of the endpoints that "were kept", which the record does not bear out.
What was registered
ClinicalTrials.gov holds four versions of NCT01459120. The record history lists them as v0 (submitted 2011-10-18, QC 2011-10-21, first posted 2011-10-25), v1 (2011-10-31; modules changed: Study Status, Contacts/Locations; the start date went from 2011-11 to 2011-10), v2 (2011-12-09; Study Status, Contacts/Locations) and v3 (2012-02-08, posted 2012-02-09; Study Status only; status COMPLETED; primary completion 2012-02, actual). The history summary reports primaryOutcomesSame: true, secondaryOutcomesSame: true and an outcomes update count of 0. I compared the outcomes module of v0, v1 and the current record (v3) field by field. They differ only in HTML character encoding (for example HIV/AIDS for HIV/AIDS) and line breaks. Enrollment is 1800 (ESTIMATED) in every version.
The five registered primary outcomes, identical in v0 (2011-10-18) through v3 (2012-02-08), are quoted here from the current record:
- "Proportion of newly tested HIV-positive clients over all clients tested for HIV during the campaigns". No time frame is given.
- "Proportion of clients newly tested HIV-positive who enrolled into HIV/AIDS care within one month over all clients newly tested HIV-positive during the campaigns". Time frame: "4 weeks after tested positive".
- "Overall number newly tested HIV-positive and enrolled into chronic HIV/AIDS-care at each facility". Time frame: "4 weeks after campaign". The description says this compares the facility-level total "Irrespective if these patients were tested during one of the campaigns or during routine activities at the facility" and the total "from the 5 villages where campaigns were held as compared to the five villages where no campaigns were held."
- "Absolute number of newly tested HIV-positive clients". No time frame is given.
- "Absolute number of clients newly tested HIV-positive who enrolled into HIV/AIDS care". Time frame: "4 weeks after tested HIV-positive".
The ten registered secondary outcomes cover CD4 count, WHO stage, four tuberculosis screening and sputum outcomes with their absolute-number counterparts, "Proportion of first-time HIV-testers among all clients accessing the testing services" and "Demographic characteristics of clients accessing the voluntary counseling and testing services". No primary or secondary outcome measures acceptance of testing among the people offered it. In outcome 1, "all clients tested" serves only as the denominator for HIV positivity.
What the paper reports
Methods, "Study Hypothesis, Design, and Setting":
The study's main hypothesis was that the HB-HTC approach would result in a higher uptake of HTC than the MC-HTC approach among individuals accessing the multi-disease campaigns.
The same section dates the trial "from 17 October to 25 November 2011".
Methods, "Outcomes at the Individual Level":
The trial had three primary and five secondary outcomes. The first primary outcome was the number of persons taking up HTC.
Uptake was defined as acceptance and completion of HTC with pre-test counseling, HIV test, and post-test counseling.
The second primary outcome was the proportion of participants newly tested HIV-positive among participants who took up HTC.
The third primary outcome was linkage to care within 28 d after a positive HIV test among participants who tested HIV-positive during the campaign.
Methods, "Sample Size and Statistical Analysis":
we estimated that HTC uptake would be about 65% in the MC-HTC arm
Abstract, "Methods and Findings":
The trial was powered to test the hypothesis of higher HTC uptake in HB-HTC campaigns than in MC-HTC campaigns.
Overall HTC uptake was higher in the HB-HTC group than in the MC-HTC group (92.5% versus 86.7%; adjusted odds ratio [aOR]: 2.06; 95% CI: 1.18–3.60; p = 0. 011)
(The space in "0. 011" is in the archived text.)
Methods, "Deviations from Initial Trial Registration", which is the paper's only statement about the registration:
Aggregate data from clusters on numbers of persons newly tested HIV-positive through routine facility-based HTC during the 28 d following the study campaigns were initially registered as one of the primary outcomes. While finalizing the internal study protocol, cluster data were dropped as a primary outcome because the trial was designed and powered to measure differences in outcomes at the individual level and not at the cluster level. Only individual endpoints were kept as primary outcomes—uptake of HTC, participants newly detected HIV-positive, and subsequent enrollment in HIV care—as these endpoints represent the three major steps of the HTC cascade. Nevertheless, description of cluster data was maintained and is reported in this paper.
Table 2 ("Primary outcomes") reports, for HB-HTC vs MC-HTC: uptake of HTC overall 92.5% (1,083/1,171) vs 86.7% (1,207/1,392), aOR 2.06 (1.18–3.60), p = 0.011; participants newly detected HIV-positive 3.6% (39/1,083) vs 6.2% (75/1,207), aOR 0.64 (0.48–0.86), p = 0.002; linkage to care 25.6% (10/39) vs 25.3% (19/75), aOR 0.99 (0.35–2.79), p = 0.978.
The supporting information includes Text S2, "Trial protocol", an undated internal protocol for a study "to be held from October to December 2011". Its outcome table lists "Proportion and absolute number of clients accepting HTC" first among the primary outcomes, and its sample-size section reads "Expected difference in HTC-uptake between study-arms (1st primary outcome): 80% (door-to-door) versus 65% (pitso)". The protocol does not mention the registry record.
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| Uptake of HTC (acceptance and completion of testing among eligible attendees) | Not registered, as primary or secondary, in v0 (2011-10-18), v1, v2 or v3 (2012-02-08) | First primary outcome; main hypothesis; basis of the sample size; first result in the abstract (aOR 2.06, p = 0.011) | added (not disclosed) |
| Proportion newly HIV-positive among those tested | Primary 1, v0–v3 | Second primary outcome: 3.6% vs 6.2%, aOR 0.64, p = 0.002 | matched |
| Proportion newly HIV-positive enrolled in care within one month / 4 weeks | Primary 2, v0–v3 | Third primary outcome, "within 28 d": 25.6% vs 25.3%, aOR 0.99, p = 0.978 | matched (4 weeks = 28 d) |
| Facility-level number newly HIV-positive and enrolled in care, 4 weeks after campaign | Primary 3, v0–v3 | Dropped as primary; reported descriptively (49 newly HIV-positive at facilities in 28 d; 7 from control villages); no arm comparison; facility-level enrolment not reported | demoted (disclosed) |
| Absolute number newly HIV-positive | Primary 4, v0–v3 | Counts 39 vs 75 shown within the second primary outcome; not named as a primary outcome | demoted (reported as count, not disclosed) |
| Absolute number newly HIV-positive enrolled in care, 4 weeks | Primary 5, v0–v3 | Counts 10 vs 19 shown within the third primary outcome; not named as a primary outcome | demoted (reported as count, not disclosed) |
Figures
Figure 1. Timeline of the trial. Squares are ClinicalTrials.gov versions; circles are dates from the paper and its Text S2 protocol. Clusters were randomised at a meeting "held on October the 3rd" (Text S2; its campaign timetable runs from 17 October 2011), and campaigns ran from 17 October to 25 November 2011. The record was first submitted on 18 October 2011, one day after the campaigns began. All four versions therefore came after the study started, and v3 (8 February 2012, status COMPLETED) falls in the month of primary completion. No version changed any outcome; the versions changed only status, dates and locations. The record was not updated again before publication on 16 December 2014 (the axis is broken between March 2012 and December 2014).
Figure 2. Registered primary outcomes (left, blue; v0 of 18 October 2011, unchanged through v3 of 8 February 2012) and the primary outcomes reported in the paper (right, orange), with each registered item's fate labelled. The paper's first primary outcome, HTC uptake, has no registered counterpart in any version (ADDED). The two registered proportions match the paper's second and third primary outcomes (MATCHED), whose results are shown in the boxes, including the null linkage result. The registered absolute numbers appear only as counts (COUNT ONLY). The cluster-level outcome was dropped as primary and reported descriptively (DEMOTED), which is the only change the paper discloses.
Why it matters
Uptake is the outcome on which the paper's design and headline rest. The main hypothesis, the power calculation and the first result in the abstract all concern uptake, and the abstract's conclusion that "HB-HTC was better in reaching children, individuals who had never tested before, and men" draws on it. A reader who checks the registration to see whether the main comparison was pre-specified finds no uptake outcome at all. The deviations section says uptake was "kept" as a primary outcome, which reads as if uptake had been registered.
The uptake result is also sensitive to how it is analysed. The paper reports that "This overall difference was due to a much higher uptake in the age group <12 y in the HB-HTC group (Table 2)", where uptake was 87.5% vs 58.7% (aOR 4.91). It also reports that "Among participants ≥12 y of age, uptake tended to be lower in the HB-HTC group compared to the MC-HTC group (Table 2)": 94.7% vs 97.1% at 12–24 y (aOR 0.59, p = 0.322) and 93.9% vs 96.1% at ≥25 y (aOR 0.61, p = 0.117). Without a registered definition, a reader cannot tell from the registry which population and analysis were intended to carry the primary comparison.
For the registered primary outcomes the paper does report results, in full. The registered first primary (proportion newly HIV-positive among those tested) favoured the comparator: 3.6% (39/1,083) with HB-HTC vs 6.2% (75/1,207) with MC-HTC, aOR 0.64 (0.48–0.86), p = 0.002. The registered second primary (linkage to care within one month) was null: 25.6% (10/39) vs 25.3% (19/75), aOR 0.99 (0.35–2.79), p = 0.978. The abstract states both. The registered facility-level outcome was not compared between arms, which the paper discloses. So no registered primary outcome is hidden. What the registry does not support is the paper's framing of uptake as the pre-specified, powered primary outcome. On the two registered individual-level primary outcomes, the intervention arm was not better.
Explanations considered
- Paper role. This is the main report of the trial, not a secondary or exploratory analysis. Its outcomes should be compared with the registered primary outcomes.
- Disclosure. The "Deviations from Initial Trial Registration" section discloses one change: the cluster-level outcome was dropped. It does not say that uptake was absent from the registration, or that it was added, and its wording ("were kept") implies the opposite. The paper does not disclose that the registered absolute-number outcomes are not treated as separate primary outcomes, though the counts are reported. The disclosure therefore does not account for the main discrepancy.
- Protocol amendments and registry edits. The registry history rules out a change in the record: the outcomes module is the same in all four versions, so uptake was never registered and then removed. The paper attributes its outcome set to "finalizing the internal study protocol". Text S2, the internal protocol, does list uptake as the first primary outcome and powers on it, so uptake was pre-specified there. But Text S2 is undated (it speaks of a study "to be held from October to December 2011") and does not say whether it was written before or after the registration, and the registration was never brought into line with it. A pre-specified internal protocol is a legitimate reason for the outcome set the paper uses; the issue is that the paper does not tell readers that the public registration differs from it.
- Wording differences. No registered outcome measures uptake under other words. Registered primary 1 uses "all clients tested" only as a denominator for HIV positivity. The registered secondary outcome "Proportion of first-time HIV-testers among all clients accessing the testing services" is a different quantity, and the paper reports it separately as its second secondary outcome. "Demographic characteristics of clients accessing the voluntary counseling and testing services" is descriptive and also does not measure acceptance of testing.
- Time point. The registered "within one month" / "4 weeks" for linkage and the paper's "within 28 d" are equivalent, so the matched outcomes are not affected.
- Registration timing. The record was first submitted on 18 October 2011, the day after campaigns began on 17 October and about two weeks after randomisation (Text S2). This is a separate, minor timing point; it does not explain the absence of uptake, since no later version added it.
- Enrollment. The registry's estimated enrollment of 1800 was never updated; the paper analysed 2,563 eligible participants from 3,197 attendees. This is a minor, separate point and does not bear on the outcome discrepancy.
Status
As of 2026-10-23:
- Registry history. Four versions, v0 to v3 (2011-10-21 QC through 2012-02-08). The version list, v0 and v1 were retrieved from the ClinicalTrials.gov history endpoints on 2026-10-22, and higher version numbers returned no version. On 2026-10-23 those endpoints returned HTTP 403, so v2 and v3 were not re-fetched; v3 is the current record, retrieved through the v2 API on 2026-10-23. The outcomes in v0, v1 and the current record are identical apart from character encoding, and the history summary reports no outcome update in any version.
- Publisher version. The PLoS Medicine article page (published 16 December 2014; received 13 September 2013; accepted 3 November 2014) links no correction, erratum, expression of concern or retraction. Its supporting information includes Text S1 (CONSORT table), Text S2 (trial protocol) and Table S1 (costs).
- Corrections. No Crossref update, PMC notice or Europe PMC notice was found (
src/known.py). - PubPeer. The PubPeer browser-extension lookup for the DOI returned no comments. An earlier web search found no thread.
- Sources. PMC4267810 as archived in PubMed Central; Text S2 from the publisher's supporting information; ClinicalTrials.gov NCT01459120 current record and record history; Crossref, PMC and Europe PMC notice searches; PubPeer.
Suggested correction
A correction to the article could state that HTC uptake, reported as the first primary outcome and the basis of the sample-size calculation, was not listed as a primary or secondary outcome in any version of ClinicalTrials.gov record NCT01459120. It could say that uptake was specified as a primary outcome in the internal protocol (Text S2), and give that protocol's date relative to the registration (18 October 2011) and the start of the campaigns (17 October 2011) if it is known. The sentence in "Deviations from Initial Trial Registration" that says uptake was "kept" as a primary outcome could be revised to say that uptake was added relative to the registration. The same section could note that registered primary outcomes 4 and 5 (absolute numbers) are reported as counts rather than as separate primary outcomes. The sponsor could also add a note to the registry record, or post results there, that points readers to the outcome set actually analysed.
The paper calls store sales primary; the registered primary, women's diet quality, is reported as secondary
PLoS Med (2021) · doi:10.1371/journal.pmed.1003729 · PMC8423266 · NCT03518151
Summary
ClinicalTrials.gov NCT03518151 (WRAPPED1) has a single version, submitted on 2018-04-25 (658 days after the registered study start and 360 days after primary completion), which names one primary outcome, women's dietary quality score from a 20-item food frequency questionnaire (FFQ) at 3 months, and lists weekly store sales of fresh fruit and vegetables and of confectionery over 12 weeks as secondary outcomes. The published primary report reverses that order: it names "store sales (primary outcome)", calls women's and children's diets "secondary outcomes", and reports every outcome at 3 and 6 months. The paper cites the registration three times and never mentions that its outcome order differs from the registered one. The registered primary outcome is reported in Table 2 (+0.29 SD, P = 0.04, adjusted for the Index of Multiple Deprivation, IMD; +0.26 SD, P = 0.06, fully adjusted), but the abstract gives no numbers for it.
What was registered
The registry record history lists a single version. It was retrieved through the ClinicalTrials.gov history endpoint (/api/int/studies/NCT03518151/history/0) and checked against the current public API v2 record, which matches it.
| Registry version | Date | Primary outcome | Secondary outcomes |
|---|---|---|---|
| v0 (the only version) | submitted 2018-04-25; first posted 2018-05-08; last update posted 2018-05-08 | Dietary quality score, 3-month follow-up | Child's dietary quality score (3 mo); women's daily fruit and vegetable portions (3 mo); women's weekly confectionery intake (3 mo); weekly sales of fresh fruit and vegetables (12 wk); weekly sales of confectionery (12 wk) |
The registered primary outcome reads, verbatim:
Dietary quality score — "The primary outcome is women's dietary quality measured using a 20-item food frequency questionnaire (FFQ) to create a standardised dietary quality score (validated against 100-item FFQ and red cell folate)." Time frame: "3 month follow-up post intervention commencement"
The two registered store-sales outcomes read:
Weekly sales of fresh fruit and vegetables (secondary) — "Total weekly sales of fresh fruit and vegetables collected via electronic sales data". Time frame: "12 week period following intervention commencement"
Weekly sales of confectionery (secondary) — "Total weekly sales of confectionery collected via electronic sales data". Time frame: "12 week period following intervention commencement"
Registry dates (all marked "actual"): study start 2016-07-06; primary completion 2017-04-30; study completion 2017-11-30. Enrollment: 150 (actual). Because v0 is the only version, the registry primary outcome was never changed. The record was created after data collection for the primary outcome had ended. The paper agrees: it says the study "took place between April 2016 and March 2017" and that "Recruitment occurred in 3 waves between July and September 2016."
What the paper reports
The paper names store sales as its primary outcome in two places:
It aims to assess whether creating a healthier layout in discount supermarkets in England improves the healthiness of store sales (primary outcome) and the purchasing and dietary behaviours of women customers aged 18 to 45 years (secondary outcomes) after 3 and 6 months.
— Introduction, final paragraph
The data collected included 9 months continuous store sale transaction data (primary outcome) and participant loyalty card data (secondary outcome) provided by the collaborating supermarket to cover 3 time periods
In order to obtain an understanding of intervention effects on household members’ diets (secondary outcomes), interview-administered telephone questionnaires obtained information about participants’ diets and diets of their children aged 2 to 6 years (where applicable) at 3 time points: baseline (time 1) and 3 (time 2) and 6 months (time 3) following intervention commencement.
— Section 2.4, Outcome measures
The store-sales analysis covers four product groups: "Weekly sales per store of fresh fruits and vegetables, frozen vegetables, confectionery, and intervention checkout items" (Section 2.6). The abstract's quantitative results are store-sales and loyalty-card purchasing results. Its first concluding sentence is "Healthier supermarket layouts can improve the nutrition profile of store sales and likely improve household purchasing and dietary quality."
The paper mentions the registration in three places, each time without comment on its content:
The trial registration number is NCT03518151.
— Abstract, Methods and findings
ClinicalTrials.gov NCT03518151 (pre-results)
— Abstract, "Trial registration" line
it was registered with ClinicalTrials.gov (NCT03518151, pre-results).
— Section 2.1, Study design and setting
The paper describes itself as "a pilot study with a prospective matched controlled cluster design" (Section 2.1). The article text (excluding the reference list) contains none of the strings "retrospectiv", "amend", "changed", "protocol", "analysis plan", "prespecified" or "pre-specified". The word "deviat" occurs only in "standard deviation". The one statement about decisions made after data collection concerns analysis methods, not which outcome is primary:
The distributions of the data were unknown in advance of study commencement because this was a pilot study. The research team therefore made decisions about the statistical analyses following (i) completion of initial analyses to determine the shape of the outcome data; and (ii) consultation with the literature to identify appropriate statistical methods for the various outcome variables.
— Section 2.6, Statistical analysis
The discrepancy
| Outcome | Registered (v0, 2018-04-25) | In the paper | Status |
|---|---|---|---|
| Women's dietary quality score (20-item FFQ, standardised) | Primary, 3-month follow-up | Secondary ("household members’ diets (secondary outcomes)"); reported at 3 and 6 months (Results 3.4, Table 2) | demoted; time point changed (6 months added) |
| Weekly sales of fresh fruit and vegetables | Secondary, 12-week period after intervention start | Primary ("store sales (primary outcome)"); interrupted time series at intervention, 3 and 6 months (Results 3.2) | added as primary; time point changed |
| Weekly sales of confectionery | Secondary, 12-week period after intervention start | Primary (same); at intervention, 3 and 6 months | added as primary; time point changed |
| Frozen-vegetable sales; intervention checkout-item sales | Not registered | Part of the store-sales (primary) analysis (Sections 2.6, 3.2) | added |
| Child's dietary quality score | Secondary, 3 months | Secondary, 3 and 6 months (Table 2) | matched; time point changed (6 months added) |
| Women's daily fruit and vegetable portions | Secondary, 3 months | Secondary, 3 and 6 months (Table 2) | matched; time point changed (6 months added) |
| Women's weekly confectionery intake | Secondary, 3 months | Secondary, 3 and 6 months (Table 2) | matched; time point changed (6 months added) |
| Loyalty-card purchasing (fruit and vegetables, confectionery, other categories) | Not registered | "participant loyalty card data (secondary outcome)" (Sections 2.4, 3.3) | added |
| Economic outcomes (grocery spend, total store sales) | Not registered | Reported (Section 3.5) | added |
Every registered outcome is reported, so nothing is missing. The difference is in the outcome hierarchy. The only registered primary outcome is presented as secondary. Two registered secondary outcomes, together with two unregistered product groups, become the primary outcome, and a 6-month time point is added throughout.
Figures
Figure 1. Trial timeline. Blue items are registry dates; orange items come from the paper or the publisher's article page. The only registry version (v0) was submitted on 2018-04-25. That is 658 days after the registered study start (2016-07-06) and 360 days after the registered primary completion (2017-04-30). The orange bars show the study period (April 2016 to March 2017) and the recruitment window (July to September 2016) as the paper reports them. No later registry version exists, so the primary outcome was never changed in the registry. The paper (received 2020-10-15, accepted 2021-07-12, published 2021-09-07) appeared while v0 was still the only version.
Figure 2. The registered outcome hierarchy (left, ClinicalTrials.gov v0, 2018-04-25) compared with the hierarchy in the paper (right). Each connector is labelled with what changed. The sole registered primary outcome, women's dietary quality score at 3 months, is reported as a secondary outcome (DEMOTED), and a 6-month time point is added. The registered secondary weekly sales outcomes become the paper's primary outcome (PROMOTED, in the table above "added as primary"), and the time frame changes from 12 weeks to 3 and 6 months. Loyalty-card purchasing, frozen-vegetable and checkout-item sales, and economic outcomes appear only in the paper (ADDED). The bottom panel gives the paper's results for the registered primary outcome (Table 2), with the 3-month headline store-sales results for comparison, and notes that the paper discloses nothing about the difference.
Why it matters
A primary outcome named in advance limits the choices that can be made after the data are seen: which outcome is the main test, which time point counts, and what the abstract leads with. Here the only registration was created after primary completion. The paper's hierarchy then differs from that registration and the paper does not say so. A reader who opens the cited NCT number finds a different primary outcome from the one the paper names, and the paper gives no explanation.
The two hierarchies lead to different headlines. The abstract's numerical results and its first conclusion are about store sales. Fruit and vegetable sales rose by +1.71 SD (95% CI 0.45, 2.96; P = 0.01) at 3 months and +2.42 SD (0.22, 4.62; P = 0.03) at 6 months. Confectionery sales fell by −1.05 SD (−1.98, −0.12; P = 0.03) at 3 months and −1.37 SD (−2.95, 0.22; P = 0.09) at 6 months (Abstract; Results 3.2). The registered primary outcome is reported in Results 3.4 and Table 2:
| Women's diet quality score, intervention vs control, change from baseline | Beta (SD) | 95% CI | P | n |
|---|---|---|---|---|
| 3 months, adjusted for IMD (registered time point) | 0.29 | 0.01, 0.57 | 0.04 | 138 |
| 3 months, adjusted for IMD and DAG-selected confounders | 0.26 | −0.01, 0.54 | 0.06 | 132 |
| 6 months, adjusted for IMD (not a registered time point) | 0.10 | −0.16, 0.36 | 0.45 | 125 |
| 6 months, fully adjusted | 0.10 | −0.17, 0.37 | 0.48 | 119 |
Table 2 labels the IMD-only model "Unadjusted"; its footnote says "* Unadjusted = adjusted for IMD (to control for similarities between pairs of stores)." The Results text reports the 3-month IMD-adjusted estimate first: "After adjustment for IMD, diet quality score at 3 months follow-up increased from baseline by 0.29 SDs ((95% CI 0.01, 0.57), P = 0.04) among intervention women compared to those shopping in control stores."
So the registered primary outcome points the same way as the headline sales results. Its 95% CI excludes zero only in the IMD-adjusted model, and the 6-month estimate is close to zero. The abstract summarises it only as "Changes in dietary variables were predominantly in the expected direction for health benefit." The Discussion calls the diet findings "more equivocal". The paper also says it is "a pilot study, and, therefore, not powered to detect an effect" (Abstract). If the registered order had been kept, the abstract would have led with a borderline 3-month dietary effect rather than with store-sales effects. Readers and systematic reviewers should know which of these the trial was registered to test.
Explanations considered
- Paper role (secondary analysis or companion paper). This does not explain the difference. This is the main outcomes report of WRAPPED1: it reports store sales, loyalty-card purchasing and all four registered dietary outcomes, and its sample (n = 150; 62 intervention, 88 control) matches the registered enrollment of 150. It cites no other report of the registered primary outcome. Reference 59 (BMJ Open 2020;10:e036758) is the protocol of the separate WRAPPED natural-experiment study and is cited only under "Directions for future research".
- Disclosure in the paper. None found. The NCT number appears three times with no comment on its content. No passage compares the reported outcomes with the registered ones, and none says that registration happened after the study. The limitations paragraph covers the lack of randomisation, being "underpowered", and implementation that was outside the team's control, but not the outcome hierarchy. The statement in Section 2.6 about decisions made after initial analyses covers the choice of statistical methods, not the choice of primary outcome. The only supplement (S1 File) contains a flow diagram, DAGs and store-pair time series; it includes no protocol, statistical analysis plan or note on outcome changes.
- Protocol amendment. Not applicable in the registry, which has a single version that was never amended. The paper cites no protocol or statistical analysis plan for this pilot, so no protocol-level designation could be checked.
- Different wording for the same outcome. Ruled out. An individual FFQ-based dietary quality score and aggregate store sales from electronic transaction data are different measures. The registry itself lists both, the first as primary and the second as secondary.
- Pilot status. The paper calls the study a pilot, which might explain a flexible approach to analysis. However, the registration was created after data collection and still names one primary outcome. Pilot status does not explain why the paper names a different primary outcome without saying so.
- Time points. The paper explains its 0–3 and 3–6 month windows (Section 2.4: habit formation, sustained change), but never compares them with the registered 3-month and 12-week time frames. This adds a second undisclosed difference and does not explain the swap of primary and secondary outcomes.
There may be legitimate reasons for the paper's hierarchy, for example the store-level sales data being the most complete data source. This write-up does not assess them. The issue documented is only that the difference from the registered primary outcome is not disclosed.
Status
As of 2026-10-23:
- Registry history: one version only (v0, submitted 2018-04-25, posted 2018-05-08; last update submitted 2018-04-25 and posted 2018-05-08). Retrieved through the ClinicalTrials.gov history endpoints (
/api/int/studies/NCT03518151?history=true,/history/0) and the public API v2 record, which agree. - Publisher version: PLoS Med 18(9): e1003729, received 2020-10-15, accepted 2021-07-12, published 2021-09-07. The PMC archive copy (PMC8423266) was used for all quotations, and each quotation was checked verbatim with
src/show.py PMC8423266 --grep. The publisher's article page shows no correction, expression of concern or retraction; the only matches for those terms are links to the journal's policy pages. The PLOS peer-review history page could not be retrieved (HTTP 404). - Corrections and notices: none in Crossref (no update records), PMC or Europe PMC (
src/known.py). - PubPeer: not checked. Both the search page and the API returned HTTP 403 (bot challenge).
- Sources: ClinicalTrials.gov NCT03518151 (v2 API and record-history endpoints); PMC8423266 full text; journals.plos.org article page for doi:10.1371/journal.pmed.1003729; Crossref, PMC and Europe PMC notice searches. The figures are drawn by
make_figures.py.txtfrom the dates and outcomes listed in that file.
Suggested correction
A correction or clarification would make the outcome hierarchy match the registry, or explain why it does not. It could do three things:
- State in the Methods (Sections 2.1 and 2.4) and in the abstract's Trial registration line that the trial was registered on 2018-04-25, after data collection, and that the registration names women's dietary quality score (20-item FFQ) at 3 months as the sole primary outcome, with weekly store sales of fresh fruit and vegetables and of confectionery over 12 weeks as secondary outcomes.
- Either present the registered primary outcome as primary, or say that store sales were designated primary in the paper, when this was decided and why. Also state that the 6-month time point and the frozen-vegetable, checkout-item, loyalty-card and economic outcomes were not in the registration.
- Give the registered primary result in the abstract with its estimate, CI and P value: 3 months, +0.29 SD (0.01, 0.57), P = 0.04 adjusted for IMD; +0.26 SD (−0.01, 0.54), P = 0.06 fully adjusted.
Paper says trial was registered prospectively; registry record was first submitted 33 months after study start
Circulation (2021) · doi:10.1161/CIRCULATIONAHA.121.054117 · PMC8849598 · NCT03476785
Summary
The Methods state that "The trial was registered prospectively on ClinicalTrials.gov (NCT03476785)." The ClinicalTrials.gov record gives the study start as 2015-06-01 (type ACTUAL). It also shows that the record was first submitted on 2018-03-01 and first posted on 2018-03-26, 33 months (1004 days) after that start, while its status was still "Recruiting". The record does describe this trial: its final enrollment (56, ACTUAL) equals the 56 participants the paper says were randomized. The outcome reported in the paper (the LV stiffness constant at 1 year) matches the registered primary outcome in every registry version, so the outcomes were not switched. However, no version of the record existed before enrollment began, so the registry cannot show that this outcome was fixed in advance. The paper does not mention the late registration, the registry edit of 2019-04-16 that made the primary outcome description more specific and deleted one secondary outcome, or the fact that one registered secondary outcome, pulse wave velocity, is not reported.
What was registered
The ClinicalTrials.gov record history has three versions. None of them is dated on or before the study start.
| Version | Date (submitted / posted) | Status | Primary outcome (measure; time frame; description) | Secondary outcomes |
|---|---|---|---|---|
| v0 | QC 2018-03-16; first submitted 2018-03-01; posted 2018-03-26 | Recruiting; enrollment 53 (estimated) | "Left ventricular stiffness"; "1 year"; "Change in end diastolic ventricular pressure and volume curve" | "Aerobic fitness": "Change in peak aerobic capacity assessed by VO2"; "Vascular stiffness": "Change in pulse wave velocity"; "Diastolic function": "Change in left ventricular diastolic function measured by echocardiography" (all at "1 year") |
| v1 | 2019-04-16 / posted 2019-04-18 (modules changed: Study Status, Outcome Measures, Contacts/Locations) | Active, not recruiting | "Left ventricular stiffness"; "1 year"; "Change in ventricular stiffness after 1 year of exercise training measured as the stiffness constant of the diastolic limb of the left ventricular pressure-volume curve" | "Aerobic fitness": "Change in peak aerobic capacity (VO2) after 1 year of exercise training"; "Vascular stiffness": "Change in pulse wave velocity after 1 year of exercise training". The "Diastolic function" outcome was deleted. |
| v2 (current) | 2019-10-10 / posted 2019-10-14 (modules changed: Study Status, Study Design) | Completed; enrollment 56 (actual) | Same as v1 | Same as v1 |
Key registry dates, all fields of the record: - Study start: 2015-06-01, ACTUAL, in every version. - Primary completion: 2019-07-02, ACTUAL (v2). In v0 and v1 it was given as 2019-06-01, ESTIMATED.
The only outcome edit, v1 on 2019-04-16, came after recruitment had closed and 77 days before primary completion. The primary outcome measure and time frame stayed the same. The new description names a specific quantity, the stiffness constant of the diastolic pressure-volume curve, where v0 described the end-diastolic pressure-volume curve in general.
What the paper reports
The PMC copy is the NIH author manuscript (NIHMS1731529).
Registration statement. Methods, "Participant Population and Study Design", second paragraph:
The trial was registered prospectively on ClinicalTrials.gov (NCT03476785).
The abstract ends with:
Clinical Trial Registration: https://clinicaltrials.gov/ct2/show/NCT03476785
Main hypothesis and endpoint. The paper never uses the words "primary outcome" or "primary endpoint". The abstract, "Background":
In a prospective, randomized-controlled trial, we hypothesized that 1-year prolonged exercise training would reduce LV myocardial stiffness in patients with LVH.
Methods, "Statistical Analysis", which has the only other use of "primary":
The primary analysis included all participants who completed the 1-year follow-up.
Result for the stiffness endpoint. Abstract, "Results":
LV myocardial stiffness was reduced (right and downward shift in the EDPVR; pre LV myocardial stiffness 0.062±0.020 to post 0.031±0.009), whereas there was no significant change in controls (pre 0.061±0.033 to post 0.066±0.031, interaction P=0.001).
Results, "LV pressure-volume relationship":
1-year of exercise training significantly reduced LV chamber and myocardial stiffness constants (LV chamber stiffness: from 0.060±0.031 to 0.042±0.025; LV myocardial stiffness: from 0.062±0.020 to 0.031±0.009), with no significant changes in the control group (LV chamber stiffness: from 0.041±0.016 to 0.049±0.020; LV myocardial stiffness: from 0.061±0.033 to 0.066±0.031; Figure 4A: interaction P value=0.015 and Figure 4B: interaction P value=0.023).
Fitness. Results, "Effect of Exercise Training on Oxygen Uptake and LVEDV", which matches Table 2 (V̇O2 max, mL/min/kg, interaction P 0.0004):
The exercise training program resulted in a significant increase in V̇O2 max in the exercise group (from 26.0±5.3 to 31.3±5.8 mL/min/kg, P<0.0001).
Enrollment. Methods, "Participant Population and Study Design":
Fifty-six participants with LVH and elevated biomarkers were enrolled (signed a consent form) and randomized, comprising 18 controls and 38 exercisers.
The paper gives no recruitment or enrollment dates. Searches of the full text for "2015", "2016", "2017" and "2018" outside the references return nothing. Searches for "pulse wave" and "PWV" also return nothing.
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| LV stiffness at 1 year (primary) | v0 2018-03-16: "Change in end diastolic ventricular pressure and volume curve"; v1 2019-04-16 and v2 2019-10-10: "...stiffness constant of the diastolic limb of the left ventricular pressure-volume curve" | LV myocardial stiffness constant "a" from the EDPVR (transmural pressure), pre vs 1 year; the main hypothesis and the headline result | matched |
| Same registered primary (alternative reading) | as above | LV chamber stiffness constant (PCWP-based EDPVR), pre vs 1 year | matched (the registry wording also fits this measure) |
| Aerobic fitness, peak VO2, 1 year (secondary) | v0, v1, v2 | V̇O2max, pre vs 1 year (Table 2, Figure 2) | matched |
| Vascular stiffness, pulse wave velocity, 1 year (secondary) | v0, v1, v2 | Not in the main text, tables or figure legends | not reported |
| Diastolic function by echocardiography, 1 year (secondary) | v0 only; deleted in v1 (2019-04-16) | Not reported. 3D echo is used only for LVEDV. | not reported (removed from the registry before primary completion) |
| LVEDV, Starling curves, preload-recruitable stroke work | Not registered | Reported (Figures 3, 5, 6), not labelled primary | added |
| Timing of registration (not an outcome) | First submitted 2018-03-01; study start 2015-06-01 (ACTUAL) | "registered prospectively" | contradicted by the registry; not disclosed |
Figures
Figure 1. Trial timeline from the ClinicalTrials.gov record history and the paper. The study start (2015-06-01, ACTUAL) comes 1004 days before the record was first submitted (2018-03-01; posted 2018-03-26, status "Recruiting"). No registry record exists for the shaded grey interval. The only version that changed the primary outcome text (v1, 2019-04-16, bold) is dated after the study start and after recruitment closed, and 77 days before primary completion (2019-07-02, ACTUAL). Version 2 (2019-10-10, open marker) changed only the status and enrollment fields. The paper's epub (2021-09-20) describes the registration as prospective.
Figure 2. Registered outcomes by registry version (left, blue) against outcomes reported in the paper (right, orange). The label on each connector names the relation. Both wordings of the registered primary outcome, LV stiffness at 1 year, correspond to the stiffness constants the paper reports. The myocardial constant is the paper's headline result and the chamber constant also fits the registry wording. The paper's result for the registered primary outcome is shown in the diagram (exercise 0.062 → 0.031; control 0.061 → 0.066). Its group × time interaction P value is 0.001 in the Abstract and 0.023 in the Results and Figure 4B. Of the registered secondary outcomes, peak VO2 is reported, pulse wave velocity is not, and echocardiographic diastolic function was deleted from the registry in v1 and is not reported. The bottom strip states the registration-timing discrepancy, which is the main finding.
Why it matters
The paper reports the registered primary outcome, and the result is positive. In completers (exercise n=20, control n=11), the LV myocardial stiffness constant fell from 0.062±0.020 to 0.031±0.009 with exercise and changed from 0.061±0.033 to 0.066±0.031 in controls. The LV chamber stiffness constant fell from 0.060±0.031 to 0.042±0.025 with exercise and changed from 0.041±0.016 to 0.049±0.020 in controls (interaction P=0.015). Nothing here suggests that a different primary outcome was substituted.
The problem is what the word "prospectively" tells a reader. Under the ICMJE recommendations, a trial counts as prospectively registered only if it was registered at or before the first participant's enrollment. Readers and systematic reviewers use that statement to decide whether the registry entry can be taken as evidence that the primary outcome, time point and analysis were fixed before any outcome data existed. Here the registry gives a study start of 2015-06-01, and its first record dates from March 2018. Primary completion was 2019-07-02 and each participant was followed for one year, so the last participant must have started the intervention by about July 2018. Of the roughly 37-month window in which participants could have been enrolled, at least 33 months came before the record existed. v0 still listed the trial as "Recruiting", so some participants may have been enrolled after registration. The paper gives no enrollment dates, so the exact split cannot be determined.
Three further points bear on how the registry can be used to check this paper: - The primary-outcome edit. The v1 change of 2019-04-16 was made after recruitment closed and before primary completion. By then most participants' 1-year data had probably been collected. The edit made the outcome more specific without changing the construct, and its new wording matches what the paper reports. It is still a post-recruitment change to the registered primary outcome, and the paper does not mention it. - Ambiguous wording. The registry says "ventricular stiffness ... stiffness constant", which could mean the chamber constant (PCWP-based) or the myocardial constant (transmural-pressure-based). The paper features the myocardial constant. Both constants show a significant interaction, so the conclusion does not depend on which one was intended. - Inconsistent P values. The abstract gives the interaction P for myocardial stiffness as 0.001. The Results and Figure 4B give 0.023 for what reads as the same comparison. Both are below 0.05, but a reader cannot tell which is correct.
The registered secondary outcome pulse wave velocity is not reported anywhere in the main text. The registered secondary outcome echocardiographic diastolic function was removed from the registry in 2019 and is also not reported. The paper explains neither omission.
Explanations considered
- Paper role. This is the primary report of the trial, not a secondary or ancillary analysis. Registered final enrollment (56, ACTUAL) equals the paper's "Fifty-six participants ... enrolled ... and randomized". The registry describes an exercise arm and a yoga arm, matching the paper's yoga-based attention control. The paper cites no other registration. Because the record describes this trial, the registration date applies to this trial's enrollment. The registry's broader title ("Prevention of Cardiovascular Stiffening With Aging and Hypertensive Heart Disease") does not change that.
- Could the study start date be wrong or refer to something other than enrollment? The start date is marked ACTUAL and is the same in all three versions. Even if it were too early, enrollment of 56 participants must have begun well before March 2018. The text reports 3,597 candidates screened, and the last participant must have started the 1-year intervention by about July 2018 to fit the ACTUAL primary completion of 2019-07-02. A March 2018 registration cannot have preceded the first enrollment unless nearly all 56 participants were enrolled in roughly four months, and nothing in the paper or the registry suggests that.
- Disclosure. The paper contains no statement that the registration was retrospective, no registration date and no reason for late registration. Its only statement about timing is the claim that registration was prospective. Searches for "registered", "protocol", "amend", "primary outcome" and "changed" found no further disclosure.
- Protocol amendments. The paper cites no published protocol or statistical analysis plan and mentions no amendments. The registry's v1 edit to the outcome measures is not mentioned. The registry also states that outcomes assessors were masked. The text says LVEDV was analysed "by an experienced cardiologist who was blinded to filling pressures". These statements are consistent with each other, but neither replaces prospective registration as public evidence of what was pre-specified.
- Wording differences. "Change in end diastolic ventricular pressure and volume curve" (v0) and "stiffness constant of the diastolic limb of the left ventricular pressure-volume curve" (v1/v2) both describe the EDPVR stiffness constant the paper reports. That accounts for the outcome match. It does not account for the registration-timing statement.
- A loose use of "prospectively". The authors might have meant that the trial was designed and conducted prospectively, which the paper also says elsewhere ("This study was a prospective randomized controlled trial"). The sentence in question, however, is specifically about registration on ClinicalTrials.gov, and in that context "prospectively" has a standard meaning that the registry dates contradict.
- Regulatory context. Whether a legal registration deadline applied to this behavioral trial was not assessed. It does not bear on whether the published statement is accurate.
Status
Checked as of 2026-10-23.
- Registry history. All three versions of NCT03476785 (v0 2018-03-16, v1 2019-04-16, v2 2019-10-10) were read from the ClinicalTrials.gov history API (/api/int/studies/NCT03476785/history/{0,1,2}), retrieved 2026-10-22/23. The current record (/api/v2/studies/NCT03476785) is identical to v2. On 2026-10-23 the history endpoint returned HTTP 403 when rechecked. The first-submitted date (2018-03-01) and study start (2015-06-01, ACTUAL) also appear in the current record, so the central finding does not depend on the history endpoint.
- Publisher version. The publisher page (ahajournals.org) returned HTTP 403, and a mirrored PDF on a university site refused the connection. The PMC text is the NIH author manuscript. The publisher-supplied PubMed abstract (PMID 34543068) has the same outcome values and the same abstract P value (interaction P=0.001), and ends "Registration: URL: https://www.clinicaltrials.gov; Unique identifier: NCT03476785." The Methods sentence "registered prospectively" could not be checked against the version of record. The online Data Supplement is not in PMC and was not checked. It could contain pulse wave velocity data, but the main text does not refer to any.
- Corrections. Crossref, PMC and Europe PMC list no correction, erratum, expression of concern or retraction (checked with src/known.py). A Letter to the Editor (doi:10.1161/CIRCULATIONAHA.121.058145, PMC8966638) raises left atrial size, atrial fibrillation, baseline BMI and follow-up length, not registration. The authors' published response was not read.
- PubPeer. The PubPeer API was blocked by Cloudflare (HTTP 403). A web search found no PubPeer thread for the DOI or title. The absence of comments is therefore not confirmed directly.
- Sources. The paper: PMC8849598, via src/show.py. The registry: ClinicalTrials.gov NCT03476785, current record and history versions 0 to 2. PubMed 34543068. Crossref, PMC and Europe PMC notices.
Suggested correction
- Replace "The trial was registered prospectively on ClinicalTrials.gov (NCT03476785)" with a statement such as: "The trial was registered retrospectively on ClinicalTrials.gov (NCT03476785; first submitted 1 March 2018), after enrollment began in June 2015." Give the reason for the late registration and, if known, how many participants were enrolled before it.
- State that the registered description of the primary outcome was revised on 16 April 2019, after recruitment closed and before primary completion, to "the stiffness constant of the diastolic limb of the left ventricular pressure-volume curve". Also state that the secondary outcome of echocardiographic diastolic function was removed at the same time.
- Specify which stiffness constant, myocardial (transmural pressure) or chamber (PCWP), is the pre-specified primary outcome. Reconcile the interaction P value for LV myocardial stiffness: 0.001 in the Abstract against 0.023 in the Results and Figure 4B.
- Report the registered secondary outcome, pulse wave velocity at 1 year, or explain why it is not reported.
Trial report omits one of two registered primary outcomes and does not disclose the omission
Lancet Reg Health Am (2025) · doi:10.1016/j.lana.2025.101339 · PMC12795688 · NCT04316221
Summary
ClinicalTrials.gov NCT04316221 has listed two primary outcomes since its first version (2020-03-18): the Caregiver Reported Child Development Instruments (CREDI) and the Guide for Monitoring Child Development (GMCD), both with a 6-month time frame. The Outcome Measures section was never edited in any of the record's six versions. The published protocol lists both instruments as primary in its abstract and data-collection schedule (Figure 7), although the defining sentence of its Primary Outcomes subsection names only the CREDI. The paper, which presents itself as the trial's primary report, names a single primary outcome (the CREDI global development score). It does not mention the GMCD anywhere, and it does not say that a registered primary outcome was dropped or why. The reported CREDI result is null. No GMCD result is reported.
What was registered
The record history (ClinicalTrials.gov history API, versions 0 to 5; the web history tab numbers the same versions 1 to 6) shows these versions and changed modules:
| Version (API) | Date | Modules changed | Primary outcomes |
|---|---|---|---|
| 0 | 2020-03-18 | first version (submitted 2020-03-16, posted 2020-03-20) | CREDI; GMCD |
| 1 | 2021-01-29 | Study Status (estimated start moved from 2020-06-01 to 2021-04-01) | unchanged |
| 2 | 2021-07-07 | Study Status (start 2021-04-01 marked actual; recruiting), Contacts/Locations | unchanged |
| 3 | 2023-01-30 | Study Status | unchanged |
| 4 | 2023-05-19 | Study Status, Study Design (enrollment changed to actual), Contacts/Locations | unchanged |
| 5 | 2026-03-12 | Study Status | unchanged |
The history summary reports "primaryOutcomesSame": true, "secondaryOutcomesSame": true and "outcomesUpdateCount": 0. The full records for versions 0, 1, 2, 4 and 5 were retrieved, and their primary-outcome entries are identical.
- Version 0 (2020-03-18), the first registration, gave an estimated start of 2020-06-01. Version 1 (2021-01-29) changed the estimated start to 2021-04-01 and was the version in force at the registered study start (2021-04-01) and when enrollment began (2021-05-28, per the paper); version 2, which marked the start as actual, was not posted until 2021-07-07. Version 1 has the same primary outcomes as version 0, word for word:
Caregiver Reported Child Development Instruments (CREDI) · time frame: 6 months · "The long form CREDI uses caregiver report and captures five domains: motor, language, cognition, social-emotional, and mental health. The CREDI is designed specifically to measure normal development within populations and has been tested in high, middle and low income countries The CREDI included a Guatemalan sample in the validation"
Guide for Monitoring Child Development (GMCD) · time frame: 6 months · "The GMCD has been used in low- and middle-income countries in children ages 0-24 months and focuses on seven areas: expressive language, receptive language, fine motor, gross motor, relating, play, and self-help." - Version 4 (2023-05-19), the last version before primary completion (2023-09-30) and before online publication (2025-12-20): the same two primary outcomes, with identical wording. - Version 5 (2026-03-12, after publication; current record): the same two primary outcomes. The record has no posted results (
hasResults: false).
The registered secondary outcomes, all at 6 months, are "changes in infant hemoglobin", "weight-for-age", "length-for-age", "Head circumference" and "Infant diet quality" (WHO Infant and Young Child Feeding indicators).
Published protocol (J Acad Nutr Diet 2022;122:432–444, doi:10.1016/j.jand.2021.03.015, online 2021-05-04; also the paper's supplement mmc2.pdf). The protocol is internally inconsistent about the GMCD. Its abstract lists both instruments as primary:
"Primary outcomes for this study are changes in global development scores, assessed using the Guide for Monitoring Global Development and the Caregiver Reported Child Development Instruments."
The body's "Primary Outcomes" subsection, however, opens with a defining sentence that names only the CREDI:
"Primary outcomes for this study are differences in global development scores between study arms, assessed using the Caregiver Reported Early Child Development Instruments (CREDI) long form at baseline (ie, infants aged 6 to 9 months) and endline."
The same subsection then describes the GMCD: "The Guide for Monitoring Child Development is extensively used alongside the CREDI in low- and middle-income countries in children aged 0 to 24 months", and "Together the Guide for Monitoring Child Development and the CREDI ... provide complementary information about child development." Figure 7 of the protocol ("Data collection description and schedule") lists the GMCD under "Primary outcome" and schedules it at endline in both groups. Under "Protocol Amendments" the protocol says: "Any modification or addition will be communicated by updating the information on the www.clinicaltrials.gov website."
Because the protocol can be read either way, the main benchmark here is the registry, which the protocol itself names as the place where changes are recorded and which lists the CREDI and the GMCD as primary outcomes in all six versions.
What the paper reports
The paper states a single primary outcome:
"The primary outcome was the overall global development score from the Caregiver Reported Early Childhood Development Instruments (CREDI) long form, which has been validated in Guatemala and piloted previously by our team."
— Methods, "Sociodemographic variables and outcomes", paragraph 2
"We hypothesized that infants receiving a single whole egg per day and local standard nutrition care during the complementary feeding period would have improved global development (primary outcome) and growth, anemia status, and diet quality (secondary outcomes) relative to infants who received only local standard nutrition care."
— Introduction, final paragraph, which opens "We report primary findings from the Saqmolo’ Project"
"Mixed linear or logistic regression models were used to estimate between-group differences in primary (global development score) and secondary (growth, anemia status, and diet quality) outcomes."
— Abstract, Methods
Its statements about the registration, protocol and analysis plan are:
"This trial was preregistered with ClinicalTrials.gov (NCT04316221). A detailed trial protocol was published previously."
— Methods, first paragraph, which also says "The study followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline."
"A detailed statistical analysis plan was made publicly available prior to data analysis."
— Methods, "Statistical analysis", first sentence (ref 51)
The archived PMC text, including the tables, figure legends and back matter, has no match for "GMCD" or "Guide for Monitoring". The supplementary tables (mmc1.docx) do not mention it either. The text contains no "amend" and no "deviat". The only departure from plan the paper describes concerns adherence monitoring: "More intensive weekly monitoring of self-reported egg consumption by telephone was planned but proved infeasible due to geographical limitations in cellular signal coverage."
The cited analysis plan (Harvard Dataverse doi:10.7910/DVN/1OSZOC, file "Saqmolo data analysis plan 1.19.2024.pdf", published 2024-01-23, after primary completion on 2023-09-30) says: "The primary endpoint for this study is differences between groups in child global development scores, as assessed using the Caregiver Reported Child Development Instruments (CREDI)." It does not mention the GMCD, and it does not say that the primary outcomes were changed from the registration or the protocol.
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| CREDI long form, global development score, 6 months | Primary; v0 2020-03-18 to v5 2026-03-12, unchanged | Sole primary outcome. Global raw scaled score at endline (6-month intervention), with domain scores as sensitivity analyses | matched |
| Guide for Monitoring Child Development (GMCD), 6 months | Primary; v0 2020-03-18 to v5 2026-03-12, unchanged. The published protocol (2021) lists it as primary in its abstract and data-collection schedule (Figure 7); its body's defining sentence names only the CREDI | Not mentioned anywhere: not as a primary, secondary or exploratory outcome, and not in the limitations | not reported |
| Infant hemoglobin, 6 months | Secondary; v0 to v5 | Secondary: altitude-adjusted hemoglobin and anemia | matched |
| Weight-for-age, length-for-age, head circumference, 6 months | Secondary; v0 to v5 | Secondary: z scores plus underweight, stunting and low head circumference. Weight-for-length z, which the protocol lists, is also reported | matched |
| Infant diet quality (WHO IYCF indicators), 6 months | Secondary; v0 to v5 | Secondary: minimum dietary diversity, minimum meal frequency, minimum adequate diet | matched |
The randomized sample (1200) matches the registered enrollment (1200, actual).
Figures
Figure 1. Timeline of the registration and publications. The upper lane shows the six ClinicalTrials.gov versions (blue; circles are versions before the registered study start, squares are versions after it). The darker band runs from the registered study start (2021-04-01) to primary completion (2023-09-30), and the lighter band covers the period after primary completion. No version changed the primary outcomes, so the figure has no outcome-change markers: every version lists the CREDI and the GMCD. The lower lane shows the registered start and primary-completion dates, the trial dates given in the paper (orange bar, 2021-05-28 to 2023-08-03), the published protocol (2021-05-04; CREDI and GMCD as primary in its abstract and Figure 7), the analysis plan deposited after primary completion (2024-01-23, CREDI only) and the paper (2025-12-20, CREDI only). The one version dated after primary completion, v5 (2026-03-12), also follows the paper's publication and still lists both primary outcomes.
Figure 2. Registered primary outcomes and the primary outcome reported. Left (blue): the two primary outcomes registered in v0 (2020-03-18), unchanged through v5 (2026-03-12). Right (orange): the primary outcome the paper reports, with its adjusted result (β −0.08 points, 95% CI −0.22 to 0.06, p = 0.26, Table 4). The CREDI connector is labelled "MATCHED" (same instrument, same 6-month time frame). The GMCD connector, labelled "NOT REPORTED", ends at a dashed box stating that the paper names the GMCD nowhere in the text, tables or supplementary tables and gives no reason. The note at the bottom summarises the protocol (both instruments primary in its abstract and Figure 7, only the CREDI in its body's defining sentence) and the analysis plan, which lists only the CREDI and states no change.
Why it matters
- Half of the registered primary set is missing without explanation. The registry has always listed two primary development instruments, and the paper reports one. A reader who checks the paper against the registration, as CONSORT item 6b invites ("any changes to trial outcomes after the trial commenced, with reasons"), finds the GMCD missing, with no statement of whether it was collected, analysed or dropped, or when.
- The registered primary that is reported is null. In the adjusted endline model the global CREDI score did not differ between groups ("β [SE] -0.08 [0.07] points [95% CI -0.22 to 0.06]; p = 0.26"; Results, "Modeling results for primary outcome"). The change-score sensitivity analysis and the four CREDI domain scores also showed no difference, and no prespecified effect modifier interacted significantly. Endline medians were 47.7 in both groups. The CREDI result is fully reported and the omission does not affect it.
- The development conclusion rests on one of the two registered instruments. The abstract concludes that egg provision "improved diet quality but did not benefit development, growth, or anemia status among Maya infants". For development, that claim rests on the CREDI alone. The paper itself names the CREDI's mode of administration as a limitation ("The CREDI is reported by caregivers and may be subject to social desirability bias") and suggests it may explain a contrast with another trial ("This might be because of our use of the CREDI, which relies on caregiver report, versus the BSID, which uses direct assessment by a trained observer"). The protocol says the GMCD and the CREDI "provide complementary information about child development". A GMCD result, in either direction, would bear directly on how far the null development finding generalises across instruments. Readers of the paper cannot tell that such a result was planned.
- Readers relying on the analysis plan would not see the change either. The plan the paper cites was deposited after primary completion and names only the CREDI. Neither the plan nor the paper notes that it differs from the registration and from the protocol's abstract and Figure 7.
Explanations considered
- The paper is a secondary or partial report. This does not apply. The paper says "We report primary findings from the Saqmolo’ Project". It reports the registered secondary outcomes alongside the primary outcome and does not point to a separate report. A Europe PMC search for "Saqmolo" (2026-10-23) returns only this paper, the protocol (J Acad Nutr Diet 2022), a protocol abstract (Curr Dev Nutr 2021) and an egg-composition study (Food Sci Nutr 2023). None of them reports GMCD results.
- The change is disclosed. It is not. The paper's only statements about the registration and plans are that the trial "was preregistered", that a protocol "was published previously" and that an analysis plan "was made publicly available prior to data analysis". None of them mentions the GMCD or a change in primary outcomes. The limitations section does not address it either.
- A documented protocol amendment explains it. None was found. The registry's Outcome Measures module was never edited in six versions (2020-03-18 to 2026-03-12), even though the protocol commits to communicating outcome changes "by updating the information on the www.clinicaltrials.gov website". The analysis plan (version 1.19.2024) lists only the CREDI, so a narrowing appears there. But the plan does not say that it departs from the registered or protocol primary outcomes, gives no reason, and is dated after primary completion. Citing the plan therefore does not, by itself, disclose the change to the reader. Whether the GMCD was administered at all is not stated in any of these documents.
- The two names refer to the same outcome. They do not. The CREDI and the GMCD are separate instruments. The registry lists them as two primary outcomes with different descriptions (caregiver-report CREDI with five domains; GMCD with seven areas), and the protocol cites them separately and schedules them separately (Figure 7). The protocol abstract's "Guide for Monitoring Global Development" is a naming variant of the same GMCD: the protocol's Outcomes section and Figure 7 call it "Guide for Monitoring Child Development". The paper's single primary outcome is explicitly the CREDI score.
- Time-point or metric differences. These are not at issue. The CREDI is analysed at the 6-month endline, consistent with the registered time frame. The difference concerns which outcomes are reported, not when they were measured.
- A later correction resolves it. None was found (see Status).
- The protocol already made the CREDI the sole primary outcome. Partly. The defining sentence of the protocol's Primary Outcomes subsection names only the CREDI, while its abstract and Figure 7 list the GMCD as primary as well, so the protocol is ambiguous. The registry, which the protocol designates for recording any modification, lists both as primary in every version, and the paper cites the registration. The protocol's ambiguity therefore does not disclose to the paper's readers that a registered primary outcome is not reported.
The documents do not show why the GMCD is absent, and this write-up makes no inference about the reason. A change of outcomes can have legitimate reasons, such as feasibility, instrument availability or field constraints. The issue documented here is that the omission is not disclosed.
Status
As of 2026-10-23:
- Registry history. The ClinicalTrials.gov history list endpoint (
/api/int/studies/NCT04316221/history) and the version records 0, 1, 2, 4 and 5 (/history/<version>, retrieved through the study's history page in a browser because direct requests returned HTTP 403) all list both primary outcomes. The current record (API v2, last update posted 2026-03-13) also lists both. No results are posted. - Publisher version. The text quoted here is the PMC archive of the article (Lancet Reg Health Am vol 54, February 2026, article 101339; online 2025-12-20). The publisher pages (thelancet.com, sciencedirect.com) returned HTTP 403. The Elsevier article API returned metadata only (cover date 2026-02-28, open access), so the wording was not compared against the publisher's own page.
- Corrections. No correction, erratum or retraction notice was found in PMC, Crossref (no
update-toorrelationentries) or Europe PMC. - PubPeer. Not checked. The site returned a Cloudflare verification challenge.
- Sources. ClinicalTrials.gov NCT04316221 (current record and history); PMC12795688 (full text and supplements mmc1.docx and mmc2.pdf, the latter the published protocol, via Europe PMC); the protocol, J Acad Nutr Diet 2022;122:432–444 (doi:10.1016/j.jand.2021.03.015, PMID 33962902); the statistical analysis plan, Harvard Dataverse doi:10.7910/DVN/1OSZOC.
Suggested correction
A correction to the article that:
- states that ClinicalTrials.gov NCT04316221 (and the abstract and Figure 7 of the published protocol) listed two primary outcomes at 6 months, the CREDI and the Guide for Monitoring Child Development (GMCD), and that the article reports only the CREDI;
- says whether GMCD data were collected and, if not, when and why the decision was made;
- if the GMCD was collected, reports the between-group result with the same models used for the CREDI (or a reason it cannot be reported), and says when the decision to make the CREDI the sole primary outcome was taken relative to the analysis plan (version 1.19.2024) and to the start of outcome analysis, as CONSORT item 6b asks for any change to trial outcomes;
- qualifies the abstract's conclusion that the intervention "did not benefit development" as resting on the caregiver-reported CREDI only.
Separately, the ClinicalTrials.gov record could be updated to reflect the outcomes actually analysed, with the change and its reason noted, as the protocol's amendment section anticipates.
HIV care trial report redefines registered retention outcome and omits hair-sample adherence without disclosure
Lancet Reg Health Am (2022) · doi:10.1016/j.lana.2022.100307 · PMC9536254 · NCT02846350
Summary
NCT02846350 (COPA2) registered five primary outcomes on 2016-07-22, before enrolment began in November 2016. No later registry version changed them. The paper is the trial's main outcomes report. It discloses and explains two departures: viral load was analysed as a 0.50-log reduction instead of suppression, and medication persistence was not presented. It does not disclose two others. Retention in care is reported as whether the participant completed the study visit at each time point, whereas the registry and the published protocol define it as the number of missed clinic appointments in the previous 6 months. The registered primary outcome "Medication adherence: Hair sample" is not reported and is never mentioned. All the randomised comparisons that are reported are null, and the paper says so in its abstract. Its positive interpretation rests on indirect (mediation) effects that it labels exploratory.
What was registered
ClinicalTrials.gov NCT02846350 was first submitted on 2016-07-01. Version 0 passed QC on 2016-07-22 with status "Not yet recruiting" and was first posted on 2016-07-27. The registry gives the study start as 2016-07 (actual) and primary completion as 2020-11-30 (actual). The paper says baseline visits ran from November 2016 to March 2018.
Version 0 lists five primary outcomes and no secondary outcomes. The wording is quoted from the record, with escape characters removed:
| # | Measure | Description | Time frame |
|---|---|---|---|
| 1 | "HIV viral Load (VL)" | "VL will be assessed by blood sampling. Viral suppression (HIV VL <200 copies/mL) will be the primary dichotomous outcome; a secondary dichotomization will be made among detectable patients with VL <1000 and those with VL >1000." | "Less than 6 months detectable VL" |
| 2 | "Retention in care" | "Defined as the number of missed clinic appointments (i.e., "no shows" not cancelled in advance by patients or clinic staff) in the 6 months preceding assessment." | "2 years" |
| 3 | "Medication persistence" | "Medication persistence is the time to treatment discontinuation, with a permissible gap of < 30 days, participants will be considered to have discontinued the regimen if a therapy gap = > 30 days occurs." | "2 years" |
| 4 | "Medication adherence: Hair sample" | "Adherence by hair sample to estimate an average concentration of ARV medication exposure" | "2 years" |
| 5 | "Medication adherence: Self-report" | "Proportion of self-reported adherent participants over time" | "2 years" |
The record history has 13 versions, v0 (2016-07-22) to v12 (2022-06-22). None of them edits the Outcome Measures module. The history summary gives outcomesUpdateCount = 0, lastUpdateVersions.primaryOutcomes = 0 and originalData.primaryOutcomesSame = true. Version 12 was submitted the day before the paper went online (2022-06-23). Its five primary outcomes match version 0 word for word, and so does the current record (last update posted 2022-06-29). The later versions change only Study Status, Contacts/Locations, Sponsor/Collaborators, Study Design, Study Identification, Oversight, Conditions and the Document Section.
The published protocol (Trials 2018, doi:10.1186/s13063-018-2758-5, PMC6056946; reference 17 of the paper) defines the outcomes the same way. On retention:
"In this study, retention in care will be defined as the number of missed clinic appointments (i.e., "no shows" not canceled in advance by patients or clinic staff) in the 6 months preceding assessment, including that timepoint's visit with the COPA2 physician. The primary retention analysis will utilize a dichotomized indicator 0/1 of any missed visits vs. no missed visits"
On hair samples and self-report:
"Participants will provide two hair samples, at the 12-month and 24-month assessments."
"Hair sample will estimate an average concentration of ARV medication exposure over 6 weeks with cut-off values used as categorical variables (12 months and 24 months)"
"Self-report will reflect percentage adherent over the last 30 days (Visual Analog Scale; VAS)"
The protocol's analysis plan (Hypothesis 1.2) treats self-reported adherence as continuous. It "will use a linear mixed model ... to examine percentage adherence in the past 30 days as a continuous outcome".
What the paper reports
The paper states its objective (Methods, "Objectives and design"):
"The primary objective of the study was to evaluate the impact of a physician-delivered, MI patient-oriented treatment program on ART adherence, HIV viral load, retention in care, and self-efficacy among PWH disengaged from HIV care."
It cites the registration in the same paragraph ("was registered at clinicaltrials.gov, NCT02846350") and in the abstract. Its only statement about departures from plan is in Methods, "Measures of main study outcomes":
"Per the original protocol, ART adherence, HIV viral suppression, retention in care, and medication persistence were the primary outcomes of the study. At the end of the data collection period, changes to how HIV viral load was evaluated as a primary outcome had to be made given that viral suppression at baseline was a constant (see description below). Self-efficacy and patient-provider communication were the MI physician targets in the original protocol, as well as the theorised mechanisms by which the intervention was intended to work. Contrary to the study protocol, in the current study, we do not present medication persistence given issues related to pharmacy pickup data medical abstraction."
On viral load (Methods, "HIV RNA Viral Load"):
"For analyses, condition differences were analysed as a clinically significant reduction in HIV viral load from one-time point to the next, 0·50log. Though we originally intended to analyse HIV RNA viral load as the proportion of participants who were virally suppressed in each condition, the complete absence of participants who were virally suppressed at baseline precluded such analyses; that is, viral suppression at baseline was a constant at baseline."
The Discussion repeats that this "precluded the analysis of viral suppression as an outcome, as originally proposed in the protocol."
On retention, the definition is given in Methods, "Retention in HIV Care", and repeated in Results, "Retention":
"Retention in HIV care was calculated as to whether a participant completed a study clinical visit at any given time point."
"Retention was evaluated at 6 months, 12 months, 18, and 24 months as to whether the participant completed a follow-up visit."
On self-reported adherence (Methods, "Self-reported antiretroviral adherence"):
"Self-reported adherence was assessed every six months at all time points using a 10-point Visual Analogue Scale (VAS), with 0 representing none and 10 representing 100% (perfect adherence)."
The analysis covers five outcomes (Methods, "Data analyses"):
"To test the effect of the intervention on ART adherence, HIV viral load, CD4+ count, retention, and self-efficacy, mixed modelling was used"
"As such, indirect effects were examined as exploratory analyses using bivariate parallel latent growth curve (LGC) models"
A full-text search of the PMC version finds no occurrence of "hair". Hair-sample adherence is not mentioned in the text, the tables or the figure legends. No passage explains the retention definition or relates it to the registered count of missed clinic appointments. The only supplementary item listed is "Supplementary materials Image, application 1".
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| HIV viral load | Primary. "Viral suppression (HIV VL <200 copies/mL) will be the primary dichotomous outcome" (v0, 2016-07-22; unchanged to v12, 2022-06-22) | Continuous viral load, with condition differences as a "clinically significant reduction ... 0·50log". Condition p = 0.757 | Not reported as registered. The change is disclosed, with a reason |
| Retention in care | Primary. "the number of missed clinic appointments ... in the 6 months preceding assessment" (v0, 2016-07-22; unchanged to v12) | "whether a participant completed a study clinical visit at any given time point". Condition p = 0.999 | Not reported as registered. The measure is redefined and the change is not disclosed |
| Medication adherence: hair sample | Primary. "Adherence by hair sample to estimate an average concentration of ARV medication exposure" (v0; unchanged to v12) | Absent: no result and no mention | Not reported, not disclosed |
| Medication adherence: self-report | Primary. "Proportion of self-reported adherent participants over time" (v0; unchanged to v12) | Mean VAS score (0–10) in a mixed model. Condition p = 0.075 | Matches the published protocol (continuous VAS). Differs from the registry wording only (minor) |
| Medication persistence | Primary. "time to treatment discontinuation ... gap = > 30 days" (v0; unchanged to v12) | Not presented ("Contrary to the study protocol ...") | Not reported. Disclosed, with a reason |
| CD4+ count | Not registered (the registry lists no secondary outcomes) | Analysed alongside the main outcomes. Condition p = 0.584 | Added |
| Self-efficacy (HIV-ASES) | Not registered. The protocol describes it as an MI patient target or mediator | Named in the "primary objective" and analysed as a main outcome. Condition p = 0.096 | Added. The paper notes its role in the original protocol |
The registry time frame for viral load, "Less than 6 months detectable VL", is unclear. The paper assesses viral load at 6, 12, 18 and 24 months, which matches the protocol, so no time-point change is claimed here.
Figures
Figure 1. Timeline of NCT02846350. The top lane shows the 13 registry versions. Only v0 (2016-07-22, filled marker) set the primary outcomes. Hollow markers are versions that left the Outcome Measures module untouched, including the three versions (v10 to v12) after primary completion on 2020-11-30 (dashed line; shaded area after it). The middle lane shows the registry study start (2016-07) and the baseline and follow-up visit periods given in the paper (November 2016 to March 2018, follow-up to May 2020). The bottom lane shows the published protocol (2018-07-24) and the paper (online 2022-06-23, issue September 2022). The registered outcomes were in place before enrolment and never changed, so the differences in the paper did not come from a registry update. Sources: the ClinicalTrials.gov history endpoint, the PMC article and Europe PMC.
Figure 2. The five registered primary outcomes (left, blue; v0 wording, identical through v12) and the outcomes reported in the paper (right, orange). Each connector is labelled with its status. Solid connectors join a registered outcome to the measure the paper reports in its place. Dashed connectors ending in "×" mark registered outcomes with no result in the paper. Two departures are disclosed in the paper: viral load and persistence. Two are not: retention, which is redefined, and hair-sample adherence, which is absent. The self-report adherence metric matches the published protocol and differs only from the registry wording. CD4+ count and self-efficacy are added. The p values in the orange boxes are the paper's longitudinal mixed-model tests for the main effect of condition and for time × condition. The footer gives the paper's indirect and direct LGC effects. All values are quoted from the Results section.
Why it matters
Results for the registered primary outcomes. For every registered primary outcome that the paper reports, the randomised comparison is null:
- Viral load, analysed as a continuous measure (a disclosed change): condition F(1,572) = 0·10, p = 0·757; time × condition p = 0·660; LGC direct effect B = 0·13, p = 0·322.
- Retention, analysed as study-visit completion (an undisclosed change): "condition (Wald X2 = 15.36 = 0·01, p = 0·999)" (as printed); interaction p = 0·750; direct effect B = 0·48, p = 0·131.
- Self-reported adherence, as a mean VAS score: condition F(1,858) = 3·18, p = 0·075; interaction p = 0·666; direct effect B = -0·20, p = 0·388.
The abstract states these null findings plainly: "Using mixed models, the intervention had no effect on ART adherence over time by condition on HIV RNA viral load, CD4+ count retention, or self-efficacy." So the departures did not turn a null primary result into a positive one.
Where the positive conclusion comes from. The abstract's Interpretation reads: "These findings suggest that physician-delivered MI may enhance the patient-provider relationship, self-efficacy, and ART adherence, and reduced HIV viral load in patients disengaged from HIV care." This rests on indirect effects through the patient–provider relationship: adherence B = 0·188, p = 0·009; viral load B = -0·095, p = 0·027; self-efficacy B = 0·063, p = 0·001. The paper calls these "exploratory analyses". The published protocol did plan exploratory mediation analyses, with "Patient-level skills (self-efficacy, motivation, patient-provider relationship) as potential mediators". Those were planned for retention, adherence and viral suppression, though, not for self-efficacy as an outcome. None of these analyses appears in the registry.
Retention. The registered measure counts no-shows at clinic appointments over the 6 months before each assessment; the protocol dichotomises it as any versus none. The reported measure records whether the participant attended the study visit itself. Study visits were "combined with regularly occurring HIV clinical care visits" and participants "were compensated with US $10 per study visit for travel". As a result, the reported measure largely tracks completion of study follow-up, which the paper also reports as attrition (for example, 25·6% did not complete the 12-month assessment). It cannot show missed clinical appointments between study visits, and those are the behaviour a re-engagement intervention targets. The Discussion itself suggests the null retention result "may have been related to the fact that participants were recruited and incentivized by study staff, given that study and clinical visits were often scheduled on the same day". A reader comparing the paper with the registration or the protocol is not told that the measure differs.
Hair-sample adherence. This was the only objective (drug-level) adherence measure among the registered primary outcomes. The protocol scheduled collection at 12 and 24 months. The paper's adherence findings, including the indirect effect on adherence, rest entirely on self-report. A reader cannot tell from the paper whether hair samples were collected, analysed or are still unreported. Leaving out a registered objective measure without comment is the kind of selective outcome reporting that registration exists to reveal.
Explanations considered
- Paper role. The paper describes the main trial results: all four registered constructs, n = 360, two-year follow-up, and the registration cited in both the abstract and the Methods. The paper does not point to any other report of the hair-sample or missed-appointment outcomes; the word "hair" does not occur in it. Being a secondary or partial report therefore does not explain the omissions.
- Disclosure. The paper discloses the viral-load change (reason: no participant was suppressed at baseline) and the omission of medication persistence (reason: problems abstracting pharmacy pickup data). These two departures are not at issue here. Its list of protocol primary outcomes ("ART adherence, HIV viral suppression, retention in care, and medication persistence") folds the two adherence measures into one. It does not mention hair samples. It does not say that retention is measured differently from the protocol. On the viral-load change, readers may note that the eligibility criteria required "at least one HIV viral load of >500 copies/mL in the prior 3 months", and that the protocol compared suppression "at 6, 12, 18, and 24 months post baseline". Because the paper discloses this change, it is noted only for completeness.
- Registry or protocol amendments. None of the 13 registry versions changes an outcome. The registry's Document Section holds only an informed consent form (2018-10-22); no protocol or SAP is uploaded. The paper cites no protocol amendment or SAP, and none is uploaded to the registry. No amendment accounts for the retention definition or the missing hair-sample outcome.
- Wording differences. For self-reported adherence, the registry's "proportion of self-reported adherent participants" differs from the paper's mean VAS score. However, the published protocol specifies VAS percentage adherence as a continuous outcome in a linear mixed model, and the paper follows that. This is treated as a registry wording issue, not an outcome switch. For retention, the difference is not one of wording. A count of missed clinic appointments over 6 months, taken from clinic records, and a binary indicator of attending the study visit measure different things. The paper does not present them as equivalent.
- Data unavailable. The paper gives a data-availability reason for persistence. It gives none for retention or hair samples. The consent form "included a release for medical record abstraction of treatment history and laboratory results", so clinic appointment records may have been available, but the paper does not say whether they were. The issue documented here is that neither departure is disclosed, whatever the reason for it.
Status
As of 2026-10-23:
- Registry history. The ClinicalTrials.gov history (
/api/int/studies/NCT02846350/history) and versions 0 and 12 (/history/0,/history/12) were retrieved and compared with the current record (/api/v2/studies/NCT02846350). After normalising whitespace and HTML escapes, the primary outcomes are identical in all three. The registry has no secondary outcomes and no posted results (hasResults = false). - Publisher version. The article pages on thelancet.com returned HTTP 403, so the version of record was not inspected directly. This write-up uses the PMC archived text (PMC9536254). Europe PMC gives the first publication date as 2022-06-23 (Lancet Reg Health Am, vol 13, September 2022, article 100307).
- Corrections. No Crossref update relations, PMC notices or Europe PMC comment/correction links were found for 10.1016/j.lana.2022.100307.
- PubPeer. The PubPeer search page returned 403 (bot challenge) and the API returned 404 for this DOI. A web search found no PubPeer thread. The PubPeer check is therefore incomplete.
- Sources. ClinicalTrials.gov NCT02846350 and its history; the paper (PMC9536254); the published protocol (PMC6056946); Europe PMC record; Crossref metadata.
Suggested correction
The journal and authors may wish to consider a correction or clarification notice that does the following:
- States that retention was registered, and specified in the protocol, as the number of missed clinic appointments in the 6 months before each assessment (analysed as any versus none). It should say that the paper instead reports whether the study visit was completed, and why. If clinic appointment records are available, it should report the registered retention measure.
- States whether hair samples were collected at 12 and 24 months as planned, and whether they were analysed. It should either report the registered hair-sample adherence outcome or explain why it is not reported. If the outcome is reported elsewhere, it should cite that report.
- Lists the registered primary outcomes in full: five, with the two adherence measures shown separately. Against each it should give the measure actually reported or the reason none is reported, and it should mark CD4+ count and self-efficacy as outcomes not registered as primary.
- Optionally notes that the registry's "proportion of self-reported adherent participants" was operationalised as the protocol's continuous VAS measure.
The trial team could also post results to ClinicalTrials.gov for the registered outcomes, and update the registry entry with any outcome changes and their dates.
Trial registered five months into enrollment and safety primary added later; paper says "prospectively registered"
Circulation (2017) · doi:10.1161/CIRCULATIONAHA.116.026493 · PMC5459585 · NCT01858363
Summary
The paper is the primary 12-month report of a randomized trial comparing a low-dose paclitaxel-coated balloon (DCB) with uncoated angioplasty (PTA). It states that "The study was prospectively registered". The ClinicalTrials.gov record was first submitted on 2013-05-08, but the paper reports that randomization began in December 2012 and the registry gives the study start as 2012-11. In the first registered version the only primary outcome was 12-month primary patency. The primary safety end point, one of the paper's two primary end points, was added as a registered primary outcome on 2014-02-12, about 14 months after randomization began and before primary completion. Both primary end points reported in the paper match the registration in force at publication, and neither the late registration nor the added primary outcome is mentioned in the paper.
What was registered
Five registry versions exist. They are numbered 1-5 here, in date order (indices 0-4 in the ClinicalTrials.gov history API). Every version gives the study start as "2012-11", and every version gives the first submission date as 2013-05-08.
Version 1, 2013-05-20 (first submitted 2013-05-08, first posted 2013-05-21; status Recruiting; estimated enrollment 360). This is the earliest version, and no version exists on or before the study start. It has one primary outcome:
Primary outcome: "Primary Patency at 12 months post-procedure" (time frame: "12 months")
Secondary outcome: "Major adverse event rate" (time frame: "1, 6, 12, 24 months")
No safety outcome is registered as primary.
Version 2, 2014-02-12 (status Recruiting). The module labels for this version include "Outcome Measures". It has two primary outcomes and no secondary outcomes:
"Primary Efficacy Endpoint" (time frame "12 months"); description: "Primary Patency at 12 months post-procedure"
"Primary Safety Endpoint" (time frame "30-Days and 12-Months"); description: "Freedom from device and procedure related target limb major amputation or death through 30-days post-procedure and clinically-driven target lesion revascularization at 12-month"
The secondary outcome "Major adverse event rate" was deleted in this version.
Version 3, 2015-06-25 and version 4, 2016-10-05. The outcomes are unchanged from version 2. Version 4, which changed only the Study Status and IPDSharing modules, records primary completion as "2016-03" (actual). It is the last version before the paper was received (2016-11-21) and published online (2017-04-19).
Version 5, 2021-10-11 (results first posted 2021-10-13; after publication). The primary outcomes were reworded for results reporting. The measures and time points did not change; the patency description adds the definition used in the paper (PSVR ≤2.5 and freedom from CD-TLR), which versions 1-4 did not spell out:
"Patency at 12-months" (12 months): "The primary efficacy endpoint was patency at 12 months post-procedure defined as the absence of target lesion restenosis determined by duplex ultrasound peak systolic velocity ratio (PSVR) ≤2.5 and freedom from clinically-driven target lesion revascularization (CD-TLR)."
"Percentage of Patients With Freedom From Device and Procedure Related Target Limb Major Amputation or Death Through 30-days Post-procedure and Clinically-driven Target Lesion Revascularization at 12-month" (30-Days and 12-Months)
The posted results (83.9% vs 60.6% for patency; 94.1% vs 83.3% for safety) agree with the paper. Actual enrollment is given as 294.
What the paper reports
The primary end points, from the Abstract (Methods):
"The primary safety end point was a composite of freedom from device- and procedure-related death through 30 days after the procedure and freedom from target limb major amputation and clinically driven target lesion revascularization through 12 months after the procedure. The primary effectiveness end point was primary patency at 12 months."
And from Methods > Outcomes:
"The primary effectiveness end point was primary patency (per lesion) through 12 months, defined as the absence of target lesion restenosis (duplex ultrasound–determined peak systolic velocity ratio ≤2.5) and freedom from CD-TLR."
The registration statement (Methods > Study Design, para 1):
"The study was prospectively registered (http://www.clinicaltrials.gov; unique identifier: NCT01858363)."
The enrollment period (Results > Patient Enrollment and Follow-Up, para 1):
"Between December 2012 and April 2015, 294 patients were randomized to DCB (222 patients, 254 lesions) or PTA (72 patients, 79 lesions) at 18 centers in Germany and Austria"
The paper's statements about prespecification are in Methods > Statistical Analysis. Para 1 sets out the two primary hypotheses (noninferiority with a 5% margin, then superiority, for safety; superiority for patency) and says of the safety hypothesis:
"A priori, if noninferiority was met and the lower 95% confidence limit was >0%, superiority would be claimed."
Para 2 says:
"All analyses were prespecified in a statistical analysis plan."
Neither statement gives a date, or says how the primary safety hypothesis relates to the registry entry of 2014-02-12 that first listed the safety composite as a primary outcome. No protocol amendment, no change to the primary end points and no registration date is mentioned anywhere in the text. The word "protocol" appears only in a sentence about the antiplatelet regimen (Methods > Study Device and Procedure). The Limitations paragraph does not mention registration.
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| 12-month primary patency | Primary in v1 (2013-05-20) as "Primary Patency at 12 months post-procedure"; primary "Primary Efficacy Endpoint" in v2-v4 (2014-02-12 to 2016-10-05) | Primary effectiveness end point: primary patency through 12 months (PSVR ≤2.5 and freedom from CD-TLR) | matched |
| Safety composite (device/procedure-related death through 30 days; target limb major amputation and CD-TLR through 12 months) | Absent from v1. Primary "Primary Safety Endpoint" from v2 (2014-02-12), during enrollment, before primary completion | Primary safety end point | added (in v2, about 14 months after randomization began; matches v2-v4; change not disclosed) |
| Major adverse event rate | Secondary in v1 only (1, 6, 12, 24 months); deleted in v2 (2014-02-12) | Reported as "Additional outcomes" (MAEs), not as a primary end point | not a primary outcome in either source; no primary-outcome status change |
| Registration timing | First submitted 2013-05-08 (posted 2013-05-21); study start 2012-11 in every version | "prospectively registered"; randomization "Between December 2012 and April 2015" | not prospective: registered roughly 5 months after the first randomization (about 6 months after the registry start date); not disclosed |
The safety composite has a minor wording difference. In the registry wording for v2-v4 ("Freedom from device and procedure related target limb major amputation or death through 30-days post-procedure and clinically-driven target lesion revascularization at 12-month"), the 30-day window could be read as applying to amputation as well as death. The paper puts amputation in the 12-month window. The version 5 description ("freedom from device and procedure-related death through 30 days post-procedure and freedom from target limb major amputation and clinically-driven target lesion revascularization (CD-TLR) through 12 months") matches the paper exactly. The wording also gives the same measure and time points, so this is classed as matched.
Figures
Figure 1. Timeline of NCT01858363 and the paper. Blue squares are registry versions: filled squares mark versions that changed the primary outcomes, and open squares mark versions that did not. Grey diamonds mark the registry study start (2012-11) and primary completion (2016-03). The orange bar is the randomization period reported in the paper (December 2012 to April 2015). Its hatched part is the stretch before the record was first submitted (2013-05-08). The orange circle marks online publication (2017-04-19). The safety composite was added as a primary outcome in version 2 (2014-02-12), after the study start and during enrollment, but before primary completion. The shaded area is the period after primary completion (axis broken between 2017 and 2021). Version 4 (2016-10-05) falls in it but changed only Study Status and IPDSharing. The only outcome change in that period is version 5 (2021-10-11), which reworded the primaries for results posting. All dates come from the registry history and the paper. Source: make_figures.py.txt.
Figure 2. On the left, in blue, are the registered outcomes across versions 1-4 (2013-05-20 to 2016-10-05), including the registered secondary outcome and the registration date. On the right, in orange, are the outcomes as reported in the paper, with the paper's results. Each connector carries a label for its status. The 12-month patency outcome was registered as primary from the first version and matches the paper's primary effectiveness end point. The safety composite was not in version 1. It was added as a primary in version 2 (2014-02-12) and matches the paper's primary safety end point. The registered secondary "Major adverse event rate" (dashed boxes) was deleted in version 2. The paper reports MAEs as an additional outcome, so no primary outcome was dropped or demoted. The bottom row sets the registration date beside the paper's "prospectively registered" statement.
Why it matters
The paper reports a result for each registered primary outcome, and both results are positive. For primary patency at 12 months, the paper reports "83.9% (188 of 224) for DCB and 60.6% (40 of 66) for PTA, for a difference of 23.3% (95% confidence interval, 10.6%–36.1%; P<0.001)" (Results > Effectiveness Outcomes). For the primary safety composite, it reports "94.1% (193 of 205) with DCB and 83.3% (50 of 60) with PTA, for a difference of 10.8% (95% confidence interval, 0.9%–23.0%)", meeting both noninferiority and superiority (Results > Safety Outcomes). The abstract concludes: "Superiority with a low-dose DCB for femoropopliteal interventions was demonstrated over PTA for both the safety and effectiveness end points."
The registry and the paper report the same outcomes, so this is not a case of a switched primary outcome. The issue is what the registry can and cannot vouch for:
- Registration was not prospective. Under the ICMJE definition, prospective registration means registration before the first patient is enrolled. Here the record was first submitted after randomization had started. The paper's own dates place that start in December 2012, and the registry places the study start in 2012-11. The paper's statement that the study "was prospectively registered" therefore does not agree with the registry or with the paper's own enrollment dates. Registration came early in a randomization period of about 29 months and well before primary completion. The registry cannot show that the first registered primary outcome was fixed before enrollment, but it does show that it was fixed before most patients were randomized.
- One of the two primary end points was registered during enrollment. The safety composite, which supports half of the abstract's conclusion of "superiority ... for both the safety and effectiveness end points", first appears as a registered primary outcome on 2014-02-12, about 14 months after randomization began and roughly 14 months before it ended. The paper does not date the primary safety hypothesis or its noninferiority margin. It states that the superiority rule was set "A priori" and that analyses were "prespecified in a statistical analysis plan", but gives no date for either. A reader of the paper cannot tell that, by the registry's account, this end point became primary while the trial was under way. The 12-month data were not complete at the time (primary completion 2016-03).
- Readers and evidence syntheses rely on the registration statement. Reviewers who use "prospectively registered" as a marker of low risk of selective reporting could score this trial differently if they had the registry dates.
Explanations considered
- Paper role. The paper is the primary 12-month report of the trial (Methods > Patient Follow-Up: "the primary effectiveness and safety results at 12 months of follow-up presented here"). It is not a secondary or post hoc analysis, so the registered primary outcomes apply to it in full. This does not explain the discrepancy.
- Disclosure. The paper contains no statement about the registration date, about any amendment of the primary end points, or about a registry update. The only registration statement calls the registration "prospective". A search of the archived text for the NCT number, "registered", "protocol", "amend", "primary outcome", "primary end", "changed" and "December 2012" found nothing that discloses either point. The online-only Data Supplement could not be retrieved (the publisher returned HTTP 403). The paper cites it only for eligibility criteria (Table I) and the list of centers, so a disclosure there is unlikely but not ruled out. Not disclosed.
- Protocol amendments. The addition of the safety composite in version 2 may reflect a protocol amendment, for example to set up a noninferiority safety hypothesis. A change made during enrollment and before outcome data were complete can be legitimate. But no protocol, amendment history or statistical analysis plan is published, cited, or posted to the registry (the registry record has no document section). Version 2 also revised other modules (Study Identification, Study Status, Oversight, Study Design, Contacts/Locations). A legitimate amendment would account for why the registry changed, but not for why the paper is silent about it and calls the registration prospective.
- Start date revised later. This is ruled out. Version 1 already gave the study start as 2012-11 with status Recruiting, and the paper independently dates the first randomization to December 2012. The late registration does not come from a start date edited afterwards.
- Earlier registration elsewhere. None was found. The paper cites only NCT01858363, and the registry record lists no other identifiers that predate it.
- Wording differences. Version 2 relabelled the patency outcome as "Primary Efficacy Endpoint" and moved the original wording into the description. Version 5 reworded both primaries for results posting and added the paper's operational definition of patency (PSVR ≤2.5 and freedom from CD-TLR). These changes did not alter the measure or the time point, and they do not account for the discrepancy. The ambiguous 30-day window in the v2-v4 safety wording is a drafting difference, and the version 5 text resolves it in line with the paper.
- "Prospective" referring to the design. The abstract uses "prospective" for the trial design ("This was a prospective, randomized, multicenter, single-blinded trial"), which is accurate. The Methods sentence applies the word to the registration itself ("was prospectively registered"), so this reading does not resolve the statement.
Status
As of 2026-10-23:
- Registry history. Five versions of NCT01858363 were reviewed: 2013-05-20, 2014-02-12, 2015-06-25, 2016-10-05 and 2021-10-11. The source was the ClinicalTrials.gov history endpoint (
/api/int/studies/NCT01858363/history/<version>), from copies retrieved earlier for this project; the live endpoint returned HTTP 403 on 2026-10-23. The current record (/api/v2/studies/NCT01858363, last update posted 2021-10-13) agrees with version 5. - Publisher version. The text checked is the PMC archive of the version of record, Circulation 2017;135(23):2227-2236 (received 2016-11-21, accepted 2017-03-29, online 2017-04-19, issue 2017-06-06). The online-only Data Supplement was not accessible (HTTP 403).
- Corrections. No correction, erratum, expression of concern or retraction was found in Crossref (no
update-toentries), in PMC notices or in Europe PMC notices. Europe PMC lists only a "Comment in" editorial (doi:10.1161/CIRCULATIONAHA.117.028308), which is not a correction. - PubPeer. The PubPeer search page was blocked by a bot check, and a web search found no PubPeer thread for this article. Whether it has PubPeer comments is unconfirmed.
- Sources. ClinicalTrials.gov record and its history tab; PMC5459585; Crossref metadata; Europe PMC.
Suggested correction
The authors and the journal could correct the registration statement in Methods > Study Design and add the registry history. Suggested wording:
"The study was registered at http://www.clinicaltrials.gov (unique identifier: NCT01858363) on May 8, 2013, after enrollment had begun (first randomization December 2012). The initial registration listed primary patency at 12 months as the primary outcome. The primary safety end point was added to the registered primary outcomes on February 12, 2014, during enrollment and before the 12-month follow-up was complete."
If a protocol amendment set up the safety end point and its noninferiority margin, the correction could also give the amendment's date and rationale and say when the statistical analysis plan was finalized. Making the protocol and the plan available, for example as supplementary material, would let readers check that both primary hypotheses were fixed before outcome data were unblinded. The reported results do not need to change.
Not written up after the deeper check (7)
These earlier-confirmed findings were dropped because the deeper check found a fixing correction, a disclosed or legitimate change, a version-of-record difference, or a failed blind re-check.
- PMC4108080 (Cell 2010): re-derivation confirmed; the error is only in the PMC copy (an author manuscript); the publisher's version of record is correct
- PMC12883204 (Lancet Oncol 2026): re-derivation confirmed; the error is only in the PMC copy (an author manuscript); the publisher's version of record is correct
- PMC11450708 (Ann Intern Med 2024): re-derivation confirmed; the error is only in the PMC copy (an author manuscript); the publisher's version of record is correct
- PMC4854752 (Lancet Haematol 2015): re-derivation confirmed; the error is only in the PMC copy (an author manuscript); the publisher's version of record is correct
- PMC4551397 (Ann Intern Med 2015): re-derivation confirmed; the error is only in the PMC copy (an author manuscript); the publisher's version of record is correct
- PMC7144257 (Eur Respir J 2020): re-derivation confirmed; version of record yes; correction: The error is real, and the authors have published a correction for it. Erratum: Eur Respir J 2020;56(3):2050524, doi:10.1183/13993003.50524-2020 (PMID 32973076, PMC8410236, Europe PMC pubType 'Publish; gate: The Author Correction (Eur Respir J 2020;56(3):2050524, doi:10.1183/13993003.50524-2020, PMC8410236) fixes this exact error. I read the correction text on PMC. It replaces the 95% CIs in both the Abstract and Table 4 and leaves the ORs and p-values unchanged. The corrected rows are: age >=65, 3.765
- PMC12980234 (JAMA Netw Open 2026): blind re-check refuted; paper role primary_report