Fig 1.
Schematic overview of the CrowdWater project, showing the connection between the CrowdWater app and game.
Fig 2.
Screenshot of the CrowdWater game.
The left picture shows the reference picture with the virtual staff gauge. The right picture shows an observation for the same spot at a different time. The player has to estimate the water level class for the picture on the right by comparing the water level and features in the stream or on the stream bank for both pictures. On the left hand side the scores of the top ten players for this round of the game are shown. The report button at the bottom can be used for pictures that cannot be classified. The squares at the top indicate the number of comparisons completed in this round of the game so far (black) and comparisons still to come (grey).
Fig 3.
Cumulative frequency distribution of all corrections.
Corrections of the original app value based on the mean game votes (between the 10th and 90th percentile). 100% = 841 classified observations.
Fig 4.
Agreement among players (in %) per classified observation.
Each column represents one observation, sorted according to the mean game vote (red triangle). Darker colours represent a higher agreement and lighter colours a lower agreement among the players. The original value of the water level class submitted via the app is indicated by the orange star.
Fig 5.
Cumulative number of classified observations per water level class based on the mean game value.
Fig 6.
Error distribution of the water level classes for the CrowdWater game and a field survey.
Comparison of the error distribution of the votes in the CrowdWater game (difference between the vote of a player and the mean game vote for that observation; n = 841) and for previously held field surveys (n = 517) [data from 49].
Fig 7.
Mean accuracy per player as a function of the number of observations that that player classified (each triangle represents one player). The lines indicate the median accuracy for all players (solid line) and the 25th and 75th percentile (dashed lines). The green shading indicates the novice players who played a maximum of two rounds (24 classifications). Note the log scale on the x-axis.
Fig 8.
Percentage of reports over all votes.
Reports for each of the 841 classified observations (i.e. at least 15 votes in total) that received at least one report. The red colour indicates that the observation received 15 reports, after which it is excluded from the game.
Table 1.
Reasons given for a report as a percentage of the overall number of reports.
Fig 9.
Percentage of observations with an error ≤ 0.05 (median in blue) or an error ≤ 0.2 (median in green) as a function of the number of votes per observation. The dashed blue and green lines indicate the 10th and 90th percentile, while the grey lines show the results for all 10 000 iterations. The solid red lines indicate that for 90% of observations the error is ≤ 0.2 after 16 votes. The dashed red lines indicate that for 95% of observations the error is ≤ 0.2 after 20 votes.
Fig 10.
Motivation of the participants to play the CrowdWater game.
The survey questions can be found in supplementary material 1 (S1 File).