Fig 1.
Author’s predictions on the probability of acceptance of their papers.
(a) Plot of authors’ predictions of chances of acceptance of the paper versus the actual acceptance rates for each response. The diagonal line represents perfect calibration and the (blue) dots represent authors’ responses. (b) Plots of authors’ predictions, for papers that were eventually accepted (thin yellow line) and rejected (thick blue line). The x-axis represents the fraction of responses with predicted percent chance greater or equal to the corresponding value on the y-axis. In other words, the x-axis is the fraction of responses with prediction greater than or equal to the corresponding y value.
Fig 2.
Comparing authors’ calibration error (Brier score) in prediction of acceptance across different subgroups based on gender and seniority level.
The error bars indicate 95% confidence intervals, obtained via bootstrapping. (a) Gender-based grouping. (b) Seniority-based grouping.
Fig 3.
Comparing authors’ (relative) predicted acceptance probability and perceived paper quality for any pair of papers authored by them.
This plot is based on 6,024 such responses. In particular, the first two bars enumerate the amount of agreement and disagreement respectively, among responses of any author that had a strict ranking between the two papers for both questions.
Fig 4.
Comparing authors’ ranking of their perceived scientific contribution (paper quality) and the decisions from the peer-review process.
This plot is based on 10,171 such responses. In particular, the first two bars enumerate the agreement and disagreement when the author-provided ranking is strict and where one of the papers is accepted and the other is rejected.
Fig 5.
Comparing co-authors’ rankings of their perceived scientific contribution (paper quality) of a pair of papers that both have authored.
This plot is based on 1,357 such responses. In particular, the first two bars enumerate the agreement and disagreement of co-authors when they both provide strict rankings of their papers.
Fig 6.
Change in authors’ perceptions of their own papers after seeing the reviews.
The five bars in each plot represent the five options: much more positive (“++”), slightly more positive (“+”), did not change (“0”), slightly more negative (“-”), much more negative (“- -”). The three subfigures depict responses pertaining to all, accepted, and rejected papers, and are based on 4435, 1767, and 2668 such responses respectively. (a) All papers. (b) Accepted papers. (c) Rejected papers.