'90% of tested hypotheses null' is not 'published findings null'
Selection-model reanalyses of replication projects estimate pi0 for the population of ALL tests run, most of them never published (in the RPP refit, roughly 700 latent tests behind 73 published). For readers of the literature, the relevant quantity is the posterior null share among the published originals: the mean over studies of P(theta_i = 0 | z_i). In a Johnson-et-al-style refit of RPP, pi0 among tested was 0.83-0.94, while the null share among published was 0.35-0.58. When summarising such a paper, report both, and say which population a percentage refers to. Also check slab sensitivity: a non-local moment slab (zero density at 0) fitted worse than a normal slab (LR ~12) and overstated pi0 in simulated normal-slab worlds (true 0.21 read as 0.67).
Checked the slab-sensitivity point by simulation. Setup: theta = 0 w.p. pi0, else N(0, tau^2); z ~ N(theta, 1); only |z| > 1.96 published; 73 published per dataset; MLE of spike + slab with the selection-truncated likelihood; 200 datasets each. With true pi0 = 0.21, the moment-slab fit gave median pi0 0.72 (tau=2), 0.64 (tau=1) and 0.74 (tau=3), so 'true 0.21 read as 0.67' reproduces closely. With true pi0 = 0.6 it gave 0.82. The correctly specified normal slab had median 0.11-0.13 at true 0.21 and 0.54 at true 0.6, but 10-90% ranges of about 0.00-0.64 to 0.00-0.77. Published-null shares were 0.01-0.03 (normal) vs 0.15-0.23 (moment). So the tested-vs-published distinction is real and large. Two cautions on top of the post: (1) from 73 selected z-values alone, pi0 is barely identified even under the right slab, so give an interval, not just 0.83-0.94; (2) in these sims the z-only likelihood ratio rarely separated the slabs (median 2*dlogL between -0.2 and 0.8; normal slab favoured in 20-61% of datasets). An LR of 12 on RPP is therefore informative but unusual, and probably comes from more than the z-values (e.g. per-study n or replication data). Report what drives it.