Bias in meta-analysis detected by a simple, graphical test
Смещение в метаанализе, выявляемое простым графическим тестом
1997-09-13
SCID: 54.1/9zybjdxu
Discuss with AI
funnel plot asymmetrymeta-analysis biaspublication biasregression of standard normal deviates
Figures from the paper
Abstract (AI)
Abstract Objective: Funnel plots (plots of effect estimates against sample size) may be useful to detect bias in meta-analyses that were later contradicted by large trials. We examined whether a simple test of asymmetry of funnel plots predicts discordance of results when meta-analyses are compared to large trials, and we assessed the prevalence of bias in published meta-analyses. Design: Medline search to identify pairs consisting of a meta-analysis and a single large trial (concordance of results was assumed if effects were in the same direction and the meta-analytic estimate was within 30% of the trial); analysis of funnel plots from 37 meta-analyses identified from a hand search of four leading general medicine journals 1993-6 and 38 meta-analyses from the second 1996 issue of the Cochrane Database of Systematic Reviews . Main outcome measure: Degree of funnel plot asymmetry as measured by the intercept from regression of standard normal deviates against precision. Results: In the eight pairs of meta-analysis and large trial that were identified (five from cardiovascular medicine, one from diabetic medicine, one from geriatric medicine, one from perinatal medicine) there were four concordant and four discordant pairs. In all cases discordance was due to meta-analyses showing larger effects. Funnel plot asymmetry was present in three out of four discordant pairs but in none of concordant pairs. In 14 (38%) journal meta-analyses and 5 (13%) Cochrane reviews, funnel plot asymmetry indicated that there was bias. Conclusions: A simple analysis of funnel plots provides a useful test for the likely presence of bias in meta-analyses, but as the capacity to detect bias will be limited when meta-analyses are based on a limited number of small trials the results from such analyses should be treated with considerable caution. Key messages Systematic reviews of randomised trials are the best strategy for appraising evidence; however, the findings of some meta-analyses were later contradicted by large trials Funnel plots, plots of the trials' effect estimates against sample size, are skewed and asymmetrical in the presence of publication bias and other biases Funnel plot asymmetry, measured by regression analysis, predicts discordance of results when meta-analyses are compared with single large trials Funnel plot asymmetry was found in 38% of meta-analyses published in leading general medicine journals and in 13% of reviews from the Cochrane Database of Systematic Reviews Critical examination of systematic reviews for publication and related biases should be considered a routine procedure
Key Findings
1
A simple graphical/regression analysis of funnel plots is a useful test for detecting likely bias in meta-analyses, but its detection power is limited when meta-analyses include few small trials.
2
Authors recommend routine critical examination of systematic reviews for publication and related biases using funnel plot analysis.
3
Discordance between meta-analyses and large trials observed in 4 of 8 pairs, with meta-analyses showing larger effects in all discordant cases.
4
Funnel plot asymmetry indicating bias was observed in 14 (38%) of journal meta-analyses and 5 (13%) of Cochrane reviews examined.
5
Regression-based funnel plot asymmetry predicts when meta-analyses disagree with subsequent large trials: asymmetry present in 3 of 4 discordant pairs and in none of 4 concordant pairs.
Research Object
Funnel plots used in meta-analyses
Research Subject
Asymmetry of funnel plots (measured by regression intercept of standard normal deviates against precision) as a graphical test to detect bias and predict discordance between meta-analyses and large trials; prevalence of such bias in published meta-analyses
Publication Details
Publication Date
1997-09-13
Journal
Publisher
ISSN
Cited by
57316
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai1
Cited by20
The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration2009
Two new Later Stone Age sites from the Final Pleistocene in the Falémé Valley, eastern Senegal2024
The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration2009
2017 ACC/AHA/AAPA/ABC/ACPM/AGS/APhA/ASH/ASPC/NMA/PCNA Guideline for the Prevention, Detection, Evaluation, and Management of High Blood Pressure in Adults: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines2017
Interventions for preventing obesity in children2019
Nicotine replacement therapy for smoking cessation2012
Exercise-based cardiac rehabilitation for coronary heart disease2011
Physical Activity and Incident Depression: A Meta-Analysis of Prospective Cohort Studies2018
Coping, emotion regulation, and psychopathology in childhood and adolescence: A meta-analysis and narrative review.2017
Controlled Interventions to Reduce Burnout in Physicians2016
Physical activity and risk of breast cancer, colon cancer, diabetes, ischemic heart disease, and ischemic stroke events: systematic review and dose-response meta-analysis for the Global Burden of Disease Study 20132016
Gestational diabetes mellitus and adverse pregnancy outcomes: systematic review and meta-analysis2022
The state of evidence for social and emotional learning: A contemporary meta-analysis of universal school-based SEL interventions2023
Nurses' burnout and associated risk factors during the COVID‐19 pandemic: A systematic review and meta‐analysis2021
Effectiveness of physical activity interventions for improving depression, anxiety and distress: an overview of systematic reviews2023
RETRACTED: Association Between Physician Burnout and Patient Safety, Professionalism, and Patient Satisfaction2018
The Global Prevalence of Anxiety Among Medical Students: A Meta-Analysis2019
A Meta-Analysis of School-Based Bullying Prevention Programs' Effects on Bystander Intervention Behavior2012
The benefits of statins in people without established cardiovascular disease but with cardiovascular risk factors: meta-analysis of randomised controlled trials2009
The Relationship Between Executive Functions and Academic Performance in Primary Education: Review and Meta-Analysis2019