Welcome to Prof. Dev Chakraborty's FROC methodology web site
ROC is an acronym for receiver operating characteristic. An operating characteristic is a way of visually depicting an observer's performance in a diagnostic task.The ROC task is to correctly classify cases (images) as diseased or non-diseased.
FROC is an acronym for free-response receiver operating characteristic. The FROC task is to correctly classify suspicious regions as lesions or not-lesions (non-lesions). [A lesion is by definition a biopsy-proven true lesion. A suspicious region could be a true lesion or a non-diseased location that just happens to appear suspicious to the reader (radiologist).]
A suspicious region is reader and case specific. A suspicious region for one reader on a specific case need not be a suspicious region for another reader on the same case. Likewise, a suspicious region for one reader on a specific case need not be a suspicious region for the same reader on a different case.
A key mark of expertise is the ability to identify suspicious regions that are true lesions without identifying regions that are non-lesions.
The FROC curve is but one of several ways of visualizing search performance (in the medical imaging context search is the ability to find lesions while not finding non lesions - see book Chapter 17 for explanation and quantifying search performance). Since the FROC curve is not recommended as a summary of performance, it is more accurate to think of FROC as a search paradigm. Most imaging examinations involve search. The radiologist does not know a-priori how many lesions (0,1,2,...,) may be present in an image and consequently must search the image for suspicious regions and marking lesions while not marking non-lesions.
FROC analysis is used to objectively evaluate imaging systems or radiologists in search tasks. It is intended to answer questions like which is the better imaging system for detecting breast cancer? How good is the radiologist at detecting cancer? In the free-response paradigm the radiologist marks and rates suspicious locations on images. The rating is the confidence level that the marked region is malignant. The free-response paradigm closely parallels tasks such as finding lesions in mammograms or nodules in lung cancer screening. FROC analysis enables more precise evaluation of imaging systems (greater statistical power) than when location is ignored. Recent progress in this field has resulted from a connection made between this paradigm and the Kundel-Nodine model of human perception, which led to the Radiological Search Model (RSM) for observer performance in search tasks.
Images courtesy of Dr. Claudia Mello-Thoms and Prof. Harold Kundel
