Figure 1

(A): illustrates ROIs drawn from two different raters. Regions 1, 6, and 7 are examples where both raters agree in the detection of a lesion but disagree on how it should be outlined; we designate these regions as type CR12. Regions 2, 3, and 4 are examples where only rater 1 (blue) drew an ROI, thus we designate these regions as type CR1. Finally, only rater 2 (red) drew an ROI for region 5, thus we designate this region as type CR2. (B): (table). For each region depicted in Figure 1 (A), we calculate sizes of rater 1’s ROIs, rater 2’s ROIs, the intersection of their ROIs, and the union of their ROIs. For regions where only one of the raters drew an ROI, (types CR1 and CR2) we calculate the contribution to DE and not OE. In this case, DE will equal the size of the only drawn ROI. For regions where both raters drew an ROI (type CR12), we calculate the contribution to OE and not DE. In this case, OE will equal the area of the difference between the union and intersection of the ROIs. The bottom row shows the total area for the ROIs drawn by both raters, the total area of intersection and the union of the raters' ROIs, together with the DE and OE errors for the set of ROIs. These values can be used to calculate SI directly, or in terms of DE and OE. (C): Calculation of SI from traditional equation and in terms of DE and OE. (D): A plot of the total number of regions of types CR1 and CR2 smaller than a given area is shown for the ROIs displayed in Figure 1 (A). This plot can be useful in determining the average number of detection differences above a specified lesion size threshold. For instance, we have 1 detection error for ROIs larger than 3. We would expect two errors between raters if we threshold their ROIs to be larger than 2. (E): is the histogram of (Area of Rater 1’s ROI – Area of Rater 2’s ROI) / Area of the Union of Rater 1 and Rater 2 ROIs (i.e., (column 3 – column 2) / column 5, from the table (B), for regions of type C12. If more regions were used, this graph would begin to form a bell-shaped curve. Regions where experts closely agree in outlining a curve would contribute to the center of the graph, while a large disagreement would contribute to one of the edges.f