Novel comparison of evaluation metrics for gene ontology classifiers reveals drastic performance differences
Fig 4
Six popular evaluation metrics compared with ADS.
Visual analysis of ADS results for six evaluation metrics: US AUC-ROC, Fmax, Smin2, Resnik score A and D, and Lin score D, obtained with CAFA data. Scores for AP sets at each ADS signal level are shown as boxplots and scores for FP sets as horizontal lines. RC value shows rank correlation of the AP sets with ADS signals and FPS shows the highest signal from FP sets. RC should be high and FPS should be low. Note the drastic differences between methods. We discuss these in the main text. We flipped the sign of Smin results for consistency. ADS signal is plotted on the x-axis and the evaluation metric, shown in headings, is plotted on the y-axis. Horizontal lines for FP sets are: Naive = red line, All Positive = blue line and Random = Green line.