Fig 1.
A typical example of image processing pipeline using different image processing steps.
The aim is to extract useful objects or regions, features for objects and / or object classification to assign object classes to each segment.
Fig 2.
Segmentation results using parameter set p = (w, t, s)T (image convolution filter size w, intensity threshold t and size s of a structuring element for image opening).
Fig 2(a) is the input grayscale image and Fig 2(b),(c) and (d) show segmentation outcomes using p = (1, 20, 1)T, (3, 120, 5)T and (1, 180, 1)T respectively.
Fig 3.
Segmentation results under high artifact levels (i.e. presence of both shading and background noise) using manual selection of p keeping parameters set the same for Fig 3(b),(c) and (d) as in Fig 2(b), 2(c) and 2(d) respectively.
Fig 4.
Exemplary feedforward pipeline vs. modified pipeline for the parameter adaptation of segmentation methods for benchmark images.
An input grayscale image is first pre-processed to remove noise and shading (parameter w is used to affect the pre-processing outcome). The pre-processed image is then used for image segmentation using either edge detection or intensity thresholding (thresholding parameter t is used in this step). The segmented image is post-processed using morphological operators to remove too big/too small objects (parameter s defines a structuring element for image opening). Features are then extracted from the remaining objects and fed into a classification routine. This pipeline could be modified using structural changes/ parameter adaptation, where evaluation measures are used for segmentation evaluation in order to calculate optimal parameter set popt. Using popt, an optimal image segmentation is obtained.
Fig 5.
(Case 1) Results for the selection of robust image segmentation parameters i.e. totsu in case of Otsu segmentation and tedge in case of Sobel edge detection for whole data set (r = 1).
R vs. totsu for the figure on top and R vs. tedge for the figure at the bottom. The green dot in both figures represents robust selection of the respective parameters i.e. trob,otsu = 0.24 for Otsu segmentation and trob,edge = 0.06 for Sobel edge detection.
Fig 6.
(Case 1) Results for Q(r, b, n) vs. A(r, b, n) and robustness with the selection of robust image segmentation parameters for Otsu segmentation (RobOtsu) and Sobel edge detection (RobEdge) for the whole data set (r = 1).
The robustness values are given in Table 1.
Table 1.
Robustness values of segmentation methods for different implementations of Case 1 (using explicit ground truth).
Fig 7.
(Case 1) Results of changing δ on image segmentation parameter adaptation for benchmark data set r = 1.
δ vs. R. R values against increasing δ for AutoOtsu (red) and for AutoEdge (blue).
Fig 8.
(Case 1) Results of image segmentation parameter adaptation (non-optimized) on benchmark data set r = 1.
Q(r, b, n) vs. A(r, b, n). The non-optimized results from r = 1, 2, 3 are shown in S8, S9 and S10 Figs respectively. The overall effect of using the best result is not a glaring one. The difference is fairly small between optimized and non-optimized result using δ = 0.02 for automatic tuning of intensity threshold and δ = 0.01 for the edge detection threshold.
Fig 9.
(Case 1) Results of image segmentation parameter adaptation (optimized) on benchmark data set r = 1.
Q(r, b, n) vs. A(r, b, n). In first row, original images from data set r = 1 are given. The second row shows corresponding segmentation and classification results using parametric feedback tuning of Otsu segmentation (AutoOtsu). The third row shows corresponding segmentation and classification results using similar tuning of edge detection method (AutoEdge). Red and green colors represent correct and wrong classification of the segmented BLOB respectively w.r.t ground truth BLOB. The robustness values for each method are given in Table 1.
Fig 10.
(Case 1) Segmentation outcome (shown in red) for Otsu segmentation and Sobel edge detection: feedforward (StdOtsu, StdEdge) and parameter adaptation (AutoOtsu, AutoEdge) at artifact level A(r, b, n), ≈ 0.75 in the first row and ≈ 0.94 in the second row.
First column: segmentation result for StdOtsu. Second column: segmentation result for AutoOtsu. Third column: segmentation result for StdEdge. Fourth column: segmentation result for AutoEdge.
Fig 11.
(Case 1) Segmentation outcome in terms of Q(r, b, n) against A(r, b, n): expert with one parameter (OneUser) vs. multiple parameter adaptation (MultiAuto).
Dotted line shows expert segmentation outcome where as solid line shows multiple parameter adaptation. The robustness results are given in Table 1.
Fig 12.
(Case 2) Object type to be found in the data set.
Set screws encircled in green are to be found in r = 1.
Fig 13.
(Case 2) Estimating the area feature of a certain object.
Set screws encircled in green on left and zoomed version on right to roughly calculate the number of pixels that constitute the area of a set screw.
Fig 14.
(Case 2) Segmentation outcome for Otsu segmentation and Sobel edge detection for abstract ground truth defined by end user using only one object type at A(r, b, n) = 0.5553.
First and second columns represent feedforward and feedback application of both methods respectively. First and second rows show Sobel edge detection and Otsu segmentation results respectively. The red outlines show the boundaries of segmented objects.
Fig 15.
(Case 2) Segmentation outcome for Otsu segmentation and Sobel edge detection for abstract ground truth defined by end user using only one object type with increasing A(r, b, n).
Dotted red and blue lines show outcome of Otsu segmentation and Sobel edge detection respectively with fixed parameter values respectively whereas solid red and blue lines show parameter adaptation using t. The robustness values for each method are given in Table 2.
Fig 16.
(Case 2) Segmentation/classification outcome for Otsu segmentation and Sobel edge detection of one object class using feedback method against increasing artifact levels along the columns from left to right.
First row: StdOtsu. Second row: AutoOtsu. Third row: StdEdge. Fourth row: AutoEdge. Green color shows right classification of segments and blue color shows the objects that are not classified as set screws.
Table 2.
Robustness values of standard segmentation methods (StdOtsu, StdEdge) vs. feedback adaptation with one parameter i.e. totsu for AutoOtsu and tedge for AutoEdge (for abstract ground truth).