AFROC analysis of reporting radiographer's performance in CT head interpretation

Aim: A preliminary small scale study to assess the diagnostic performance of a limited group of reporting radiographers and consultant radiologists in clinical practice undertaking computed tomography (CT) head interpretation. Method: A multiple reader multiple case (MRMC) alternative free response receiver operating characteristic (AFROC) methodology was applied. Utilising an image bank of 30 CT head examinations, with a 1:1 ratio of normal to abnormal cases. A reference standard was established by double reporting the original reports using two additional independent consultant radiologists with arbitration of discordance by the researcher. Twelve observers from six southern National Health Service (NHS) trusts were invited to participate. The results were compared for accuracy, agreement, sensitivity, specificity. Data analysis used AFROC and area under the curve (AUC) with standard error. Results: The reporting radiographers results demonstrated a mean sensitivity rate of 88.7% (95% CI 82.3–95.1%), specificity 95.6% (96% CI 90.1–100%) and accuracy of 92.2% (95% CI 89.3–95%). The consultant radiologists mean sensitivity rate was 83.35% (95% CI 80–86.7%), specificity 90% (95% CI 86.7–93.3%) and accuracy of 86.65% (95% CI 83.3–90%). Observer performance between the two groups was compared with AFROC, AUC, and standard error analysis (p = 0.94, SE 0.202). Conclusion: The findings of this research indicate that within a limited study, a small group of reporting radiographers demonstrated high levels of diagnostic accuracy in the interpretation of CT head examinations that was equivalent to a small selection of consultant radiologists. - Highlights: • We assessed reporting radiographers and consultant radiologists in a clinical setting. • This was a small scale retrospective multi-reader multi-case multi-site study. • AFROC used lesion-based decisions rather than case-based decisions. • Within a limited study the observed performance was high in CT head interpretation. • Parallels were drawn with published results from other CT head interpretation studies