Fig 1.
A representative image of the encounters captured in the CongreG8 dataset.
In this case, a Pepper robot approaches a free-standing conversational group in order to join it. The complete dataset consists of trials of human approach behaviors and robot approach behaviors. Actual recordings took place in a motion capture facility in which all participants (apart from Pepper) wore motion capture suits.
Fig 2.
Each of the three players in the conversational group has a card with a word on it. Only one word card is different. The player who holds the odd card is the spy, but players do not know what cards the others have. They take turns to describe the word until the adjudicator (either a robot or a human) approaches the group to identify the spy. In this example, player B is identified as the spy and all the players show their cards to confirm that the identification is correct.
Fig 3.
The top-down view of the human-group interactions (room is not to scale). Three players were directed to stand in the center of the room and could do so freely (i.e., were not assigned specific positions). In an attempt to create a more natural situation with a variety of approach directions, the adjudicator was instructed to walk around the periphery of the group and, when directed to do so, approach them.
Fig 4.
Setup for the human-group interaction motion capture. A participant in a motion capture suit, T-pose (left). A group of three play Who’s the Spy, and the adjudicator approaches to join them (right).
Fig 5.
Setup for the robot-group interaction motion capture. The robot has 3 markers (red circles) attached on the base in order to track its position and orientation (left). The robot acts as the adjudicator to join and find the spy (right).
Fig 6.
Reconstructed skeleton (left). Marker positions (middle) and names of 37 markers (right).
Fig 7.
Real-time views that help the experimenter to control the robot.
The camera view from the robot’s forehead camera (left). The reconstructed scene from Motive including three group players and the robot (right).
Fig 8.
Diagram of the protocol for the data collection scenario.
The data or stream flows from orange dots to blue dots. In the robot-group condition, the wizard (2) controls the robot through a Python script (3), including body position and orientations. In addition, the robot (6) presents predefined gestures and dialogues when it identifies the spy after joining the group. Real-time constructed skeletons from Motive (4), and the forehead camera view from the robot are used to help the wizard send robot control commands.
Table 1.
Data acquisition protocol.
Table 2.
List of post-processed data in the CongreG8 dataset.
Fig 9.
Data post-processing including fixing labeling errors (left) and marker occlusions.
The three orange markers in the red circle represent unlabeled markers, and we manually assign correct labels to these markers in order to reconstruct the correct right-hand skeleton. The yellow marker in the green circle represents the occluded marker which causes a gap in the data trajectory (inside the green rectangle), and we make cubic interpolations to fill this gap.
Fig 10.
The visualization of group space and newcomer joining behaviors.
(a) The heatmap of all group members’ positions relative to the group center. (b) Two examples of joining group trajectories (the top-head marker is used to represent the position). (c) Full-body markers plots of joining behaviors corresponding to the examples in (b) respectively.
Fig 11.
Two group behaviors when the adjudicator (yellow character) approaches to join the group.
The red arrow indicates the movement of the adjudicator. (a) The group members stand still and ignore the adjudicator purposefully. (b) The group members accommodate the adjudicator, with one group member (red character) moving backward in order to make space for them. (c) The group members accommodate the adjudicator, one group member (red character) moves backward, and another (blue character) shifts weight from one foot to the other. These behaviors make space for the adjudicator.
Table 3.
Group behavior label definition.
Fig 12.
The violin plot of the big-five personality traits across all participants.
Fig 13.
The boxplot of the level of accommodation (left) and the ratio of accommodation behaviors (right).
Fig 14.
The averaged personality of each pool (left axis) and the percentage of Accommodate labels (right axis).
Fig 15.
The clustered group personalities on dimension-reduced data (left) and the boxplot of the ration of accommodate behaviors of two clusters (right).
Fig 16.
Three example use cases of the CongreG8 dataset.
Group behavior recognition in an online virtual chatroom (left and middle) and simulated group behaviors in modeling group animations (right).