NuSeT: A deep learning tool for reliably separating and analyzing crowded cells
Fig 3
NuSeT efficiently addresses common segmentation challenges.
(A) Implementing RPN-aided watershed algorithm improves touching cell separation. Bounding boxes and segmentation masks are computed by RPN and U-Net. Then the estimated centroid of each cell is computed from the coordinates of the bounding box. The watershed line is then estimated based on the binary mask and centroids. (B) Sample results showing that RPN successfully detects most of the cells, and watershed lines further separate touching cells. (C) Representative examples showing NuSeT detected more nuclei and better separated touching nuclei compared to Mask R-CNN and U-Net. (D,E) Examples nuclear masks generated using NuSeT for an image with high nuclei density (D). Comparison with the corresponding masks generated by Mask R-CNN and U-Net show subtle as well as prominent irregularities in boundary delineation that are circumvented by NuSeT (E).