Evaluating test accuracy study designs for rare and ultra-rare conditions: application to newborn blood spot screening

OBJECTIVES: To update and expand an existing framework for diagnostic accuracy study designs in low-prevalence settings, evaluate its applicability to newborn screening (NBS) for rare and ultra-rare conditions, and provide practical considerations tailored to test accuracy assessment. METHODS: Two literature searches were conducted: (1) an update of a prior review to identify methodological papers on test accuracy in low-prevalence settings (MEDLINE and Embase, 2017-2025) and (2) a de novo search for systematic reviews of test accuracy studies on five conditions relevant to NBS across multiple databases. A "best fit framework" synthesis was applied using an existing framework on diagnostic accuracy study designs as a base. Findings were reviewed and refined through expert consultation in a 1-day workshop. We summarize the characteristics, strengths, limitations, and unique challenges of each study design and provide practical considerations for their application in the NBS setting. RESULTS: The literature searches and expert input yielded 11 studies: seven methodological papers from search 1, three systematic reviews from search 2, and one methodological paper identified through expert in None of the studies introduced new study designs. The base framework for diagnostic accuracy study designs remains unchanged but applying it to rare conditions in NBS requires methodological adaptions. Challenges include extremely low prevalence, sample size limitations, broad disease spectra, variants of uncertain clinical significance, incidental findings, limited feasibility of timely, definitive reference standards, and temporal follow-up issues. While all study designs have potential utility in NBS, their suitability depends on the condition, available data, and context-specific challenges. CONCLUSION: No new study designs have emerged to address persistent challenges in low-prevalence settings. The foundational framework remains relevant, but its application in NBS-especially regarding large cohort studies-requires a flexible, context-specific approach. Given the need for sample enrichment and the rarity of conditions in NBS, two-gate studies are likely to be the most common, though other designs, such as adapted cohort studies and two-phase studies, are also viable, especially when enrichment strategies are applied. Future research should focus on developing tailored tools for assessing test accuracy in NBS.