When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment

Когда проверка статистической значимости нулевой гипотезы непригодна для научных исследований: переоценка
John P. A. Ioannidis, Dénes Szűcs
2017-08-02

effect sizesnull hypothesis significance testingpre-registrationpre-study power calculationsreplication crisis
Null hypothesis significance testing (NHST) has several shortcomings that are likely contributing factors behind the widely debated replication crisis of (cognitive) neuroscience, psychology, and biomedical science in general. We review these shortcomings and suggest that, after sustained negative experience, NHST should no longer be the default, dominant statistical practice of all biomedical and psychological research. If theoretical predictions are weak we should not rely on all or nothing hypothesis tests. Different inferential methods may be most suitable for different types of research questions. Whenever researchers use NHST they should justify its use, and publish pre-study power calculations and effect sizes, including negative findings. Hypothesis-testing studies should be pre-registered and optimally raw data published. The current statistics lite educational approach for students that has sustained the widespread, spurious use of NHST should be phased out.
1
NHST should no longer be the default statistical practice across biomedical and psychological research after sustained negative experience with its use.
2
Researchers using NHST should justify its use and report pre-study power calculations, effect sizes, and negative findings.
3
The paper identifies shortcomings of null hypothesis significance testing as contributing factors to the replication crisis in cognitive neuroscience, psychology, and biomedical research.
4
The paper recommends preregistration of hypothesis-testing studies, optimal publication of raw data, and replacing inadequate statistics education that perpetuates spurious NHST use.
5
When theoretical predictions are weak, all-or-nothing hypothesis tests are inappropriate, and alternative inferential methods should be selected according to the research question.

null hypothesis significance testing (NHST) in biomedical, psychological, and cognitive neuroscience research

the shortcomings, inappropriate default use, and recommended alternatives and reporting practices for NHST

Publication Details
Publication Date
2017-08-02
Journal
Publisher
ISSN
Cited by
320
Access Type
Author Information
Authors
John P. A. Ioannidis
Dénes Szűcs
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%