How to evaluate a segmentation?

How to evaluate a segmentation?#

This chapter has introduced the usual methods of segmentation, but there are many more! No method is the best because the result depends, among other things, on the image itself. Consequently, it is interesting to evaluate, for the type of image to process, the quality of the segmentation. To do that, different criteria can be used, defined below. In addition to the image to be segmented, we also need the expected result, which we call “ground truth” (French: vérité terrain).

Imagine that the ground truth and the segmentation are the images in Fig. 79 (binary segmentation).

../_images/segmentation-eval-gt-est.png

Fig. 79 Ground truth \(f^*\) (left) and segmentation \(f\) (right).#

Each image has two areas: the segmented object (shown in white) and the background (in black). So, we can define four types of zones (cf. Fig. 80):

  • the true positives (TP) represent the pixels considered as being in the object and being really in the object,

  • conversely, the true negatives (TN) are the pixels outside the object both in the segmentation and the ground truth,

  • the false positive (FP) are the pixels considered by the segmentation in the object, but which in reality are not part of it,

  • finally, the false negative (FN) are the pixels of the object that the segmentation has classified outside.

../_images/eval-zones.png

Fig. 80 Definition of true positive (TP), false positive (FP), true negative (VN) and false negative (FN).#

From these four quantities, one or the other of the criteria below can be used.

Sensibility (sensibilité)
\[ \frac{\text{TP}}{\text{TP}+\text{FN}} \]
Specificity (spécificité)
\[ \frac{\text{TN}}{\text{TN}+\text{FP}} \]
Dice coefficient (coefficient de Dice)
\[ \frac{2\,\text{TP}}{2\,\text{TP}+\text{FP}+\text{FN}} = \frac{2\,|f\, \cap f\,^*|}{|f\,| + |f\,^*|} \]
Jaccard coefficient (coefficient de Jaccard)
\[ \frac{\text{TP}}{\text{TP}+\text{FP}+\text{FN}} = \frac{|f\, \cap f\,^*|}{|f\, \cup f\,^*|} \]