Figure 1.
Design with examples of the meaningful (iconic gesture: G+; german sentence: S+) and meaningless (control gesture: G−; russian sentence: S−) speech and gesture video stimuli.
The stimulus material consisted of video clips of an actor either speaking or performing gestures (exemplary screenshots). Speech bubbles (translations of the original German sentence “Der Fischer hat einen großen Fisch gefangen”) are inserted for illustrative purposes only. Note the dark- and light-colored spots on the actor's sweater that were used for the control task. The actor displayed in the photograph has given written informed consent to the publication of his photograph.
Figure 2.
Within-modality semantic processing for speech (A; S+>S−) and iconic gestures (B; G+>G−).
Table 1.
Regions activated for familiar versus unfamiliar language [S+>S−] and for meaningful iconic versus control gestures [G+>G−].
Figure 3.
Common areas of activation for the processing of semantics derived from speech and iconic gestures (S+>S− ∩ G+>G−).
Table 2.
Regions activated for both speech and gesture semantics ([S+>S−] ∩ [G+>G−]).
Figure 4.
Stronger activations for speech semantics than gesture semantics (A; [S+>S−] > [G+>G−]) and vice versa (B; [G+>G−] > [S+>S−]).
Table 3.
Regions activated specifically for speech and gesture semantics.