Fig 1.
Visualization of the combined objective function.
An exemplary sample x is fed into the network, which outputs a training prediction . Each class is denoted with a unique color. The corresponding ground truth label
is missing the class on the lower right. The dotted line marks where the label would be in the sample. The dashed box on the right denotes
, with the output for both objective functions
and
. Prediction errors are marked as red areas. The number in the lower-left corners indicates the approximate output for the respective objective function. Since there is a prediction error in both given classes, the Dice loss will output a value
for them and 0 for the unlabeled class since it cannot make a statement without the ground truth mask. The class asymmetric loss will only output a value
if an unlabeled class’s predicted segmentation mask intersects with a given class’s ground truth mask. Since the network predicted a small portion of the unlabeled class as part of the class in the center-left, the output will be
for this class. For the other labeled class, no intersection occurred, and hence the output is 0. Since the unlabeled mask does not contain a ground truth, it is not evaluated by the class asymmetric loss, which will result in an output of 0.
Fig 2.
Labels of the extended heartSeg dataset.
Displaying image masks for the three semantic classes overlaid on two image samples of the medaka hatchlings’ cardiac system as part of the extended heartSeg dataset. The orange label masks depict the ventricle V (based on the heartSeg dataset [3]). The blue and green label masks depict the respective, newly added semantic classes bulbus B and atrium A.
Fig 3.
Visualization of the conducted experiments. Each row is an experiment, each column the applied statistic. The scatter points in each plot indicate the value for the respective method to measure performance, either the mean Performance Frugality Ratio PFR over the observed classes or the mean value (μ) over the respective statistic. The optimal performance is highlighted with a vertical dotted line. The plotted values are presented as a moving average with kernel size two. The left axis corresponds to the respective statistic, the right axis, if it is present, to the respective PFR. For details see S1 File.
Fig 4.
Predictions of different training schemes.
For reasons of comparability, the predictions are shown for the same sample (see Fig 2). The first column presents the development of the baseline model’s prediction during the training, followed by the ablation experiment with 60% dropped labels, the transfer learning approach with 70% dropped labels, and the merge experiment with a ratio of 63% to 37%. The rows depict the predictions’ state after e training epochs (10, 20, 30, 40, 50). For detailed results of the experiments, see Section 3. The last column shows the validation loss development during training, presented as a moving average with kernel size three.