Star Wars: The Empirics Strike Back

Abel Brodeur (1) , Mathias Lé (2, 3) , Marc Sangnier (4) , Yanos Zylberberg (5)

1 University of Ottawa [Ottawa]
2 PSE - Paris School of Economics
3 PSE - Paris-Jourdan Sciences Economiques
4 GREQAM - Groupement de Recherche en Économie Quantitative d'Aix-Marseille
5 School of Economics Finance and Management

Using 50,000 tests published in the AER, JPE, and QJE, we identify a residual in the distribution of tests that cannot be explained solely by journals favoring rejection of the null hypothesis. We observe a two-humped camel shape with missing p-values between 0.25 and 0.10 that can be retrieved just after the 0.05 threshold and represent 10-20 percent of marginally rejected tests. Our interpretation is that researchers inflate the value of just-rejected tests by choosing "significant" specifications. We propose a method to measure this residual and describe how it varies by article and author characteristics.

Domaines: Sciences de l'Homme et Société; Economies et finances

Abel Brodeur, Mathias Lé, Marc Sangnier, Yanos Zylberberg. Star Wars: The Empirics Strike Back. American Economic Journal: Applied Economics, 2016, 8 (1), pp.1-32. ⟨10.1257/app.20150044⟩. ⟨hal-01447851⟩