Reproducibility and robustness of economics and political science research

More than 85% of published claims in a sample of 110 economics and political science articles were computationally reproducible; in robustness checks 72% of statistically significant estimates remained significant and in the same direction, and the median reproduced effect size was 99% of the originally published effect. The study focused on papers from leading journals with mandatory data/code sharing and has its data and code available on Zenodo and OSF. Six independent teams tested determinants of robustness and found teams with more experience reported lower robustness; robustness did not correlate with author characteristics or data availability.

Analysis

The study’s headline reassurance about computational reproducibility acts as a risk-reducing shock for strategies that rely on published empirical elasticities to size trades and policy exposure. If market participants treat published median effect sizes as 99%-stable signals, capital allocators will compress risk premia on strategies that depend on those external estimates; expect that re-pricing to play out over 6–24 months as central banks, think tanks and procurement officers cite “reproducible” papers when setting policy or awarding contracts. This should reduce model uncertainty for macro‑tilts but amplify the value of any remaining idiosyncratic, non‑robust signals.

A second‑order fragility: more experienced reproducers find lower robustness, implying current headline reproducibility may be upward biased by less rigorous re-analyses. As high-skill reproducibility audits scale, anticipate a non-trivial downward reclassification of marginally significant results — practically, treat estimates with t in [1.8,2.5] as having ~30–40% higher tail-failure risk over the next 12 months. That shifts optimal signal thresholds and favors strategies that overweight larger, mechanically stable effects.

Winners are infrastructure and service providers that lower the marginal cost of verified science — cloud compute, code hosting and professional audit/consulting firms will capture recurring revenue from journals, funders and agencies standardizing reproducibility. Losers include lightweight, signal-chasing quant shops and consultants who monetize novelty over verification; their business model is exposed as institutions demand verifiable pipelines. The primary catalyst is policy adoption (funders, journals, procurement rules) within 6–18 months; the main reversal risk is performative compliance or gaming that restores opacity, which could reintroduce surprise failures over 1–3 years.

Key Decisions for Investors

• Long MSFT (6–12 month call spread): buy a 12-month MSFT 10% OTM call and sell a 20% OTM call to express asymmetric upside from continued demand for GitHub/Azure compute for reproducibility pipelines. R/R: limited premium (small downside if cloud growth stalls) vs 12–20% upside in cloud-driven revenue sensitivity.
• Long ACN (18-month buy-and-hold or long-dated calls): Accenture stands to win government and institutional implementation work as reproducibility standards become procurement criteria. R/R: steady ~6–10% IRR upside from contract wins vs execution/competitive risk if agencies insource.
• Operational portfolio hedge — tighten signal thresholds and fund an internal 1–2% AUM reproducibility desk (6–12 months ramp): require external-grade reproducibility for any alpha sourced from academic studies and de‑risk positions where supporting estimates have t-statistics between 1.8–2.5. R/R: modest recurring cost versus reduced tail risk and fewer post-publication drawdowns.
• Pair trade — long AMZN (AWS exposure) / short small-cap replication consultancies or niche research vendors (6–12 months): AWS gains from commoditizing reproducibility tooling while smaller vendors face margin compression. R/R: AWS upside tied to enterprise cloud adoption; downside if cloud growth reaccelerates uniformly across vendors (pair helps mitigate broad market moves).