The standard error of regressions: a note on new evidence of significance misuse

Abstract

There is a body of literature dealing with the improper use of statistical significance within economic analysis. Among the problematic usages that have been identified are fundamental misunderstandings about the influence of sample design and size on statistical significance, an excessive focus on statistical significance to the exclusion of economic and policy significance, and a harmful conflation of these two very different types of significance. An analysis of 51 agricultural economics papers reviewed and presented at an African conference in 2010 finds improper usage of statistical significance that is comparable or worse in nature and extent to that found in a previous meta analysis focusing on published articles in the American Economic Review in the 1980s and 1990s: well over half of the papers employed what is termed 'sign' and 'Asterisks' econometrics. Overall, the findings underline the need for clearly stated and consistent analytical methods in producing papers as well as for careful review and selection of papers that employ regression analysis.