Fig 1.
The distributions of public datasets are different from the distribution of our private data.
The left illustrates examples from public datasets and our private data to show the difference in image content and light conditions. The right illustrates the feature distribution of different data sets in low-dimensional space by using the UMAP algorithm [4]. The two axes represent the two main dimensions after dimensionality, which are new coordinate systems generated through the UMAP optimization process based on the geometric structure of the data distribution instead of corresponding to specific features in the original feature space.
Fig 2.
Our SATS integrated the newly proposed UDFusion and GANet into a teacher-student protocol to achieve the unlabeled colon polyp segmentation by leveraging the labeled public datasets.
Particularly, The GANet is employed to build multiple teacher modules and one student module. The teacher modules are well-trained with public datasets separately, thereby producing multiple pseudo-labels for training the student module on private data. The UDFuion evaluates the distribution between each private image and the public datasets to fuse the multiple pseudo-labels and thus outputs a reliable pseudo-label for the student modules. Therefore, the student module is able to segment private images after training without data annotation.
Fig 3.
The architecture of the proposed GANet.
The GANet firstly employs ResBlocks to build the encoder and thus extract features from the input image. Then, the GANet employs the Identify Transformer Block and the Refine Transformer Blocks to build the decoder, thereby segmenting the polyps progressively. Particularly, the Identify Transformer is constructed with a channel-attention module and a spatial-attention module to predict the rough location of the polyps from a global perspective. The Refine Transformer Blocks are constructed with foreground attention and background attention under the guidance of the Identify Transformer Block, thereby predicting more accurate segmentation masks in ambiguous areas.
Fig 4.
Five visualized examples to demonstrate the superiority of our SATS compared with existing colon polyp segmentation methods.
The color images are the polyp images, and the binary images are the labels or segmentation results. In each polyp image, the red boundary is the label, and the blue mask is the segmentation.
Table 1.
Our SATS attains a new state-of-the-art performance in the unsupervised colon polyp segmentation than all five comparative methods.
Fig 5.
The ablation study combines UDFusion with various colon polyp segmentation networks to demonstrate the superiority of our GANet.
Fig 6.
Three examples to demonstrate that our GANet is more capable of colon polyp segmentation compared with the existing four networks.
Table 2.
Ablation studies to validate the necessity of the Identify Transformer Block and the Refine Transformer Block.
Fig 7.
The ablation study combines our GANet with various pseudo-label generation methods to demonstrate the superiority of our UDFusion.
Fig 8.
Three visualized segmentation examples to demonstrate our UDFusion outperforming the other four pseudo-label generation methods.
Fig 9.
The experiment indicates 2d is the optimal for the coefficient in the multi-scale Tversky loss.
Fig 10.
The experiment indicates 2e − 4 is the optimal for the learning rate.
Table 3.
Segmentation results with fewer datasets and teacher networks.