incentive structures in sciencenovelty biaspositive-results biaspublication biasreplication crisis
Figures from the paper
Abstract (AI)
An academic scientist's professional success depends on publishing. Publishing norms emphasize novel, positive results. As such, disciplinary incentives encourage design, analysis, and reporting decisions that elicit positive results and ignore negative results. Prior reports demonstrate how these incentives inflate the rate of false effects in published science. When incentives favor novelty over replication, false results persist in the literature unchallenged, reducing efficiency in knowledge accumulation. Previous suggestions to address this problem are unlikely to be effective. For example, a journal of negative results publishes otherwise unpublishable reports. This enshrines the low status of the journal and its content. The persistence of false findings can be meliorated with strategies that make the fundamental but abstract accuracy motive-getting it right-competitive with the more tangible and concrete incentive-getting it published. This article develops strategies for improving scientific practices and knowledge accumulation that account for ordinary human motivations and biases.
Key Findings
1
Current publishing norms that prioritize novel, positive results incentivize practices that produce and report positive findings while ignoring negative results.
2
Journals dedicated to negative results are unlikely to be effective because they institutionalize low status for such findings and thus do not change incentive structures.
3
Mitigating persistent false findings requires making the accuracy motive (getting it right) competitive with the publication motive (getting it published) through strategies aligned with human motivations and biases.
4
The article develops practical strategies to improve scientific practices and knowledge accumulation by accounting for ordinary human motivations and biases.
5
These disciplinary incentives inflate the rate of false effects in the published literature and allow false results to persist unchallenged when replication is devalued.
Research Object
Scientific publishing system and disciplinary incentive structures in academic science
Research Subject
How incentives favoring novelty and positive results influence research design, analysis, reporting, replication, and the persistence of false findings, and strategies to align incentives with accuracy to improve knowledge accumulation
Publication Details
Publication Date
2012-11-01
Journal
Publisher
ISSN
Cited by
1473
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai6
The Structure of Scientific Revolutions1963
Initial sequencing and analysis of the human genome2001
Two new Later Stone Age sites from the Final Pleistocene in the Falémé Valley, eastern Senegal2024
Why Most Published Research Findings Are False2005
The file drawer problem and tolerance for null results.1979
How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data2009
Cited by8
Estimating the reproducibility of psychological science2015
The state of evidence for social and emotional learning: A contemporary meta-analysis of universal school-based SEL interventions2023
Replicability, Robustness, and Reproducibility in Psychological Science2021
An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science2012
Top 10 Replicated Findings From Behavioral Genetics2016
Assessing scientists for hiring, promotion, and tenure2018
When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment2017
The replication crisis has led to positive structural, procedural, and community changes2023