Estimating error rates in binary decisions with inconclusive outcomes
Conference
64th ISI World Statistics Congress - Ottawa, Canada
Format: CPS Abstract
Session: CPS 01 - Statistical methodology II
Monday 17 July 8:30 a.m. - 9:40 a.m. (Canada/Eastern)
Abstract
Binary decision-making occurs in many areas of science and policy; e.g., medicine (tumor present or absent),
forensics (ID or exclusion), finance (good or bad credit risk), agriculture (healthy or diseased plant). Lab or field studies may be conducted to assess the error rates in such binary decision-making processes (e.g., proficiency tests for radiologists or latent print examiners). In such tests, a true outcome is known (e.g., latent print and file print did or did not come from the same source), but study outcomes allow three responses (e.g., "same," "different," "inconclusive"). Many articles in forensic science report results of such studies by completely ignoring "inconclusive" responses, which can artificially increase the apparent accuracy rate. In this talk, I will discuss ways of estimating error rates in such studies that more fairly account for "inconclusive" decisions and enable fair comparisons of results across studies.