Fig 1.
Visualization of the experimental setting of the Interaction session.
The Interaction Session followed one week after the Intake Session. Here, each participant was placed within a group (4-6 participants). Each participant interacted with all other participants in their group. Every participant repeated every stage (Photograph + Evaluation of Person Perceptions, Partner selection, Dyadic conversation + Post-conversation evaluation, Partner Selection, and the Joint Task) p-times (p = number of participants in their group - 1)). For each evaluation, partner selection, and decision in the Joint Task, participants were presented with a picture of the participant they would be rating, selecting, or interacting with (for more information see Procedure). This figure represents the experimental procedure experienced by two participants (black lines), while all participants interacted with each other (grey lines).
Fig 2.
Machine Learning Modeling Setup.
The figure represents: a) the data pairing used for machine learning analyses with time-series features of facial and acoustic cues automatically annotated in videos of each participant as input, while output were ratings of these participants on warmth or competence by others; b) presents the analysis process from multi-modal data preprocessing, where each audio-video recording was preprocessed by separating video and audio streams, fed to a pre-trained model used for vocal (OpenSmile) and facial (OpenFace) cues extraction. Lastly, Principal Component Analysis was only applied to reduce the dimensionality of the input that was used as input for the Support Vector Regression (SVR).
Table 1.
The proportion of explained variance (R2) in the Ridge Regression by acoustic and facial modalities on unseen participants.
Fig 3.
Interaction Between the Type of Task and Person Perceptions of Warmth and Competence Predicting Partner Selection.
The figure illustrates how the type of task moderates the relationship between perceptions of warmth and competence in predicting partner selection. Specifically, participants perceived as higher in competence were more likely to be selected as partners for the Joint Competence Task, but this relationship was not observed for the Joint Trust Task. Participants perceived as higher in warmth were more likely to be selected as partners for both tasks. However, partner selection was slightly stronger for the Joint Trust Task compared to the Joint Competence Task.