Fig 1.
Crisis map produced by the Standby Task Force and Humanity Road in response to Typhoon Pablo, Dec. 6, 2012.
Reprinted from http://irevolutions.org/2012/12/08/digital-response-typhoon-pablo/ under a CC BY license, with permission from the Standby Task Force and Humanity Road, original copyright 2012.
Fig 2.
Map (top) and table (bottom) views of the open-source CrowdMapper platform.
(Bing Maps imagery reprinted under a CC BY license with permission from Microsoft Corporation).
Fig 3.
Randomized assignment of approximately 100 workers to different teams in one experiment run, as shown in TurkServer [39].
Table 1.
Number of teams and participating individuals assigned to each condition.
A Fisher exact test on the dropout rate is insignificant at the 10% level across all pairs of conditions, suggesting that the attrition rate did not significantly change based on the treatment.
Fig 4.
Bing Maps imagery reprinted under a CC BY license with permission from Microsoft Corporation.
Fig 5.
Performance (F1 score) vs time for teams of nominal size n = 1, 2, 4, 8, 16, 32.
Error bars show standard errors.
Fig 6.
A: F1 score; B: precision; and C: recall vs. actively worked person-hours. Each dot represents one team, and solid lines indicate least-squares best fit. Dashed lines indicate the performance of the SBTF, and the shaded area shows the 95% CI of the regression mean.
Fig 7.
A: Individual effort measured in effort hours (see text) as a function of mean person hours for teams of a given nominal size (nominal team sizes are given in parentheses). B: Allocation of effort to the four primary task-types as function of final average person hours, evaluated at 75% of final person-hours. We make the comparison at 75% of person hours rather than 100% because one of the groups of n = 32 succeeded in filtering all of the nearly 1600 tweets before the experiment completed, hence for the remaining time their computed effort allocation effectively under-counted filtering and displayed more verification. To check for robustness we also computed the same quantities at the end of the experiment. Naturally, the effort allocated to filtering is lower for the n = 32 treatment, but otherwise the pattern remains the same. In both panels nominal group sizes are given in parentheses, error bars denote standard error, and x-axes are on a log2 scale.
Fig 8.
Collaboration activity, defined as the mean effort-weighted number of contributors per event, as function of person hours.
Nominal group sizes are given in parentheses, error bars indicate one standard error, and the x-axis is on a log2 scale.
Fig 9.
A: F1 score; B: recall; and C: precision of synthetic teams relative to actual teams. Solid blue line indicates mean performance of synthetic teams and error bars show the interquartile range. Dots correspond to the performance of experimental teams, solid green line corresponds to maximum likelihood fit of quadratic model, and shaded area corresponds to 95% confidence interval (as in Fig 6). Dashed red line shows the SBTF performance.