Fig 1.
Overview of the IDEA protocol, as adopted in the repliCATS project.
Table 1.
Summary of different metrics commonly used to evaluate predictions.
Fig 2.
Summary of results from validation experiment.
Each dot represents a particular group’s aggregated assessment of replicability for each claim, with five groups assessing all 25 claims drawn from previous replication projects. Claims for which the replication study did not produce a statistically significant result are shown in the top pane, and claims for which the replication study did produce a statistically significant result are shown in the bottom pane.
Fig 3.
Anticlockwise from centre top (a) complete layout, plus expanded details of: (b) claim information; (c) and (d) responses to elicitation and aggregated feedback; and (e) participants’ reasoning comments.