Figure 1.
Open versus closed peer review systems for the peer review game.
Under the closed system of peer review (left column), reviewers know the identity of problem solvers, but problem solvers do not know the identity of the reviewers. Public information is limited to the number of accepted solutions for each player. Under the open system of peer review (right column) solvers and reviewers are known to each other, and both the number of accepted solutions and accepted reviews for each player are known publicly.
Figure 2.
Open peer reviewers spend a greater proportion of their time reviewing.
Each panel is a plot of the cumulative proportion of time each individual spends solving problems during the experiment over the course of the game. Under closed peer review, individuals spend a greater proportion of their time solving (top row). In two experiments (Closed Experiment 1, Closed Experiment 2), an individual spent almost 100% of their time solving problems. Under open peer review, individuals spent a smaller proportion of their time solving problems and a greater proportion of their time reviewing problems (bottom row).
Figure 3.
Open reviewers are more cooperative than closed reviewers.
Each panel shows the cooperation network for one of the peer review experiments. The thickness of the line indicates the amount of interaction and color indicates the type of interaction. Cooperation (blue) is defined as above average probability of both players accepting each others solutions. Obstruction (red) is defined as below average probability of both players accepting each others solutions. Under closed review (top row) there is less cooperation between players than under open review (bottom row).
Figure 4.
The task selection screens for the Peer Review Game.
Task selection under the (a) closed and (b) open modes. In each case a player may elect to solve or review a problem. In the open peer review mode, players know the identity of the players reviewing their solutions.
Figure 5.
The problem solving screen for the Peer Review Game.
The problem solving screen is the same for both versions of the game.
Figure 6.
The reviewing screen for the Peer Review Game.
The reviewing screen is the same for both versions of the game.
Figure 7.
The public information screens for the Peer Review Game.
Public information under the (a) closed and (b) open modes. In each case the number of solutions each player has had accepted are displayed. In the open review system, the number of solutions reviewed and accepted by each player is also displayed.