64th ISI World Statistics Congress - Ottawa, Canada

64th ISI World Statistics Congress - Ottawa, Canada

Rapid evidence synthesis approach for limits on the search date: A tradeoff between accuracy and workload

Conference

64th ISI World Statistics Congress - Ottawa, Canada

Format: IPS Abstract

Session: IPS 94 - Advances in Research Synthesis in Healthcare Research

Tuesday 18 July 10 a.m. - noon (Canada/Eastern)

Abstract

Rapid reviews have become a popular tool for making quick decisions, and limiting the search date is a commonly used approach in published rapid reviews. To determine the most effective rapid strategy, we evaluated the accuracy and workload of search date limits on meta-analytical results. We collected data from the Cochrane Database of Systematic Reviews (CDSR), emulating rapid reviews by restricting the search date of the original CDSR to the past 40, 35, 30, 25, 20, 15, 10, 7, 5, and 3 years. We compared the results of these limited searches to the full meta-analyses, and also conducted a repeat literature search using a 10% random sample to measure relative workload reduction (RWR). By establishing the relationship between accuracy and RWR, we found that a search date limit of 20 years achieved good accuracy and saved the most workload, with a maximum bias tolerance of 5% or 10%. For a bias tolerance of 15% or 20%, a limit of 10 or 15 years might be considered. We identified over 21,000 meta-analyses from 2,693 CDSRs, encompassing both binary and continuous outcomes. Our results demonstrate that limiting the search date is a valid and efficient approach for producing credible evidence in a timely manner. When conducting rapid reviews, researchers should weigh both accuracy and workload to make an appropriate decision.