64th ISI World Statistics Congress - Ottawa, Canada

64th ISI World Statistics Congress - Ottawa, Canada

Estimating error rates in binary decisions with inconclusive outcomes


Karen Kafadar


  • S
    Sydney Campbell
  • J
    Jordan Rodu


64th ISI World Statistics Congress - Ottawa, Canada

Format: CPS Abstract

Session: CPS 01 - Statistical methodology II

Monday 17 July 8:30 a.m. - 9:40 a.m. (Canada/Eastern)


Binary decision-making occurs in many areas of science and policy; e.g., medicine (tumor present or absent),
forensics (ID or exclusion), finance (good or bad credit risk), agriculture (healthy or diseased plant). Lab or field studies may be conducted to assess the error rates in such binary decision-making processes (e.g., proficiency tests for radiologists or latent print examiners). In such tests, a true outcome is known (e.g., latent print and file print did or did not come from the same source), but study outcomes allow three responses (e.g., "same," "different," "inconclusive"). Many articles in forensic science report results of such studies by completely ignoring "inconclusive" responses, which can artificially increase the apparent accuracy rate. In this talk, I will discuss ways of estimating error rates in such studies that more fairly account for "inconclusive" decisions and enable fair comparisons of results across studies.