Fig 1.
a) Workflow of a Citizen Science based annotation study. b) Workflow of our parametrized simulated Citizen Science study.
Fig 2.
Example image patches for the object classes found in the APEI-6 area.
Arthropods, Mollusca, Echinodermata, and also Cnidaria have a wide variety of shapes, but only one example image per label is shown for brevity.
Fig 3.
Images used for the CS primer-experiment including the expert annotations, whereas the color indicates the taxa/morphotype assigned by the expert.
Image a) features larger easier to spot objects, while image b) mostly features hard to spot Porifera.
Fig 4.
a) Deviation in x- and y-direction. Depicted are the deviation in the respective direction divided by the radius, i.e. . Each blue dot represents a CS annotation
of the Citizen Science Primer-experiment
. Please note that five outliers are not depicted in favor of a more detailed visualization. The red cross is just for orientation and depicts no deviation at all. b) Overlap displays the overlap of the expert annotation with the CS annotation. Relative radius
is the ratio of the radius r′ of the CS annotation and the radius r of the expert annotation. c) and d) show the same information on the same scale as a) and b) respectively but are protocol-corrected (see section Results for more details). e) and f) show the results for the Citizen Science Study
. Each dot represents a CS annotation
of the
.
Fig 5.
An overview of the modular architecture of AlexNet used for the classification of the (simulated) CS ROIs.
Fig 6.
Results of the annotation of a Porifera in the CS primer-experiment .
The expert annotation is in red while the CS annotations are in yellow and the protocol-corrected expert annotation is in blue. The radii selected by the CS are more than 10 times the expert annotation radius.
Table 1.
Classification accuracy for the expert, the simulated citizen scientist annotations, and the CS primer-experiment.
The arrows are just for illustration purposes. Differences of more than 3 basis points compared to the results of are depicted with ↓ or ↑. Differences of less than 3 basis points compared to
are depicted with ↘ or ↗.
Fig 7.
For illustration purposes we show the impact of different inaccuracies in position (IP) and radius (IR) on an annotation with the label Crustacea.
Fig 8.
Classification accuracy for the expert, the simulated citizen scientist annotations, and the CS primer-experiment.
a) Shows the accuracies aggregated by experiment and b) Each boxplot represents the aggregated results from each dataset () for each specific taxon.