Introduction

Abundant research has suggested that gestures are not just used in the service of communication (e.g., [1]) but are also involved in the speakers’ cognitive processes and mental representations (e.g., [2]). Autobiographical memories are complex mental representations of the self that are constructed from different types of episodic and semantic knowledge, and thought to contain sensorimotor elements [3]. Provided that autobiographical memories are not fully amodal representations, here, we seek a link between gesture production and autobiographical memory representations. This idea complements well with the embodied cognition literature, where memory retrieval has been shown to benefit from sensorimotor activations during encoding (e.g., [4]). It has been shown that embodied forms of remembering involve different modalities, including eye-movements (e.g., [5]), body postures (e.g., [6]), and movements (e.g., [7]). Similarly, the beneficial role of gesture production has been shown during episodic memory (e.g., [8, 9]) and autobiographical memory retrieval [10]. However, to our knowledge, no previous studies attempted to explore the possible contribution of gesture production during the retrieval of complex autobiographical representations in adults. Given this backdrop (e.g., [6, 11], for a review), we aim to examine the link between gesture production and autobiographical memory representations.

Co-speech gestures are intertwined with speech, temporally and semantically [12]. Gestures are mainly considered in four categories: iconic gestures depict action, motion, and shape or indicate location and trajectory; and metaphoric gestures represent abstract ideas. Beat gestures, on the other hand, are simple motoric movements produced along with the rhythm of the speech. Last, deictic gestures include the type of hand movements that involve pointing or reaching to objects or abstract entities [12]. Recent frameworks mainly consider iconic and metaphoric gestures without deictic ones as representational and beat gestures as non-representational [1, 2]. In this paper, we follow the same categorization as we focus on specific representations of concrete and abstract concepts in iconic and metaphoric gestures (see [13, 14] for similar categorizations). Since representational gestures are visuospatial in nature [2], this categorization will allow us to investigate the possible contribution of visuospatial information to autobiographical remembering.

Recently, representational gestures have been highlighted as components that affect and even shape mental representations [11, 15]. According to the gesture-for-conceptualization framework [2], gestures may change the content of the thought by representing and then triggering and adding new sensory-spatial-motoric representations. Other roles they assume include exploring spatio-motoric information and organizing them into packages that allow an effective encoding [2, 16]. Taken together, these functions have great implications for how autobiographical representations would take their final form during retrieval, as autobiographical memory is thought to be “instantiating context for sensory-perceptual episodic memory” [3 (p. 1)].

What does an autobiographical event representation entail? A full representation includes episodic elements such as events, sensory-perceptual information (across different sensory modalities), actions, people, and spatial and temporal relations between them. An associated phenomenological sense of “autonoetic consciousness”, which is described as a sense of time as personal events are happening in relation to the self, is considered as the hallmark of thinking about the personal past [17 (p. 1)]. In addition to these episodic elements, semantic informational components, such as general knowledge about the self and the world, are also included. Therefore, multiple processes are in effect to retrieve, choose, and bind together relevant pieces of information [18, 19]. Our position in the present study is that, when there is an opportunity, co-speech hand gestures may contribute to those processes owing to the multimodal nature of memory representations in the gesture and speech interaction of narrating episodic events. To our knowledge, this is the first study to examine the link between gestures and various forms of episodic representations.

Representational gestures may support autobiographical memory retrieval in several ways. During recollection, relevant sensorimotor and perceptual components of past experiences are reactivated [11, 20]. When some of these episodic components are captured in gesture format, based on Kita et al.’s framework [2], gestures may activate and manipulate further spatio-temporal information, such as the location or relative positioning of the objects or people. Since the construction of an autobiographical event is a flexible and dynamic process that allows the details to be bound differently at each retrieval [21], the influence of gesture production during a retrieval attempt is critical for the type and amount of event details to be included. Supporting this notion, previous research has shown that gesture use and the number of correct details (e.g., shape, location) in (non-autobiographical) event accounts are related (e.g., [22]; see also [23]). Also, Cook et al. [8] showed that when participants were encouraged to use gestures during event description, both immediate and later recall performance increased, compared to the no-gesture group. This effect was present for events describing both complex everyday activities, which are highly visuospatial in nature, and static images that depict hand-movements. Therefore, we expect to see a positive association between the rate of representational gesture use and the number of episodic elements in an autobiographical event narrative.

Based on Kita et al. [2], representational gestures also assume a packaging function by chunking information into units that can be used in other operations. Applied to the mechanisms of autobiographical memory, packaging the sensory-perceptual details in co-speech gestures during recollection may facilitate other operations, such as recollective phenomenology; i.e., vividness and sense of reliving. Previous evidence has shown that what gives rise to recollective phenomenology, such as the sense of re-experiencing the event or vividness, may be the presence of sensory details [24]. Similarly, it has been suggested that the subjective sense of vividness may be determined by the amount of sensory information [25] and the number of retrieved memory details [26, 27]. Therefore, it is likely that as the representational gesture rate increases, the phenomenological experience of a memory would be stronger due to the availability of sensory-perceptual elements in the representation [28]. We maintain that owing to their nature involving spatio-motoric and sensory details, episodic representations would be in direct communication with the gesture system and would be open to their contribution to the episodic richness of the remembered events.

A frequent discussion in the autobiographical memory literature is the shared mechanisms of remembering the past and imagining the future events (e.g., [29, 30]). Owing to similar event construction processes, findings have shown that imagined personal events are based on the same informational and phenomenological components provided by episodic as well as semantic memory [19]. Therefore, the difference in the level of episodic content between past and future personal events is thought to be only quantitative (i.e., a matter of degree in the content details) rather than qualitative [18]. In fact, remembered events are typically shown to contain more perceptual content than imagined future events (e.g., [31, 32]). Thus, if gesture production supports detailed episodic representations, we would expect representational gesture rates to be also positively correlated with the episodic details in imagined personal events, but to a lesser degree than in remembered events.

Method

Results

Analytic plan

To examine gesture rate differences across conditions, we run a repeated measures ANOVA with the type of event (past, future, non-autobiographical) as the within-subjects factor. Then, we focus on the relationship between the individual differences (visuospatial and spatial imagery skills) and gesture rates with a Kendall’s tau rank correlation analysis. In line with the present study’s exploratory nature, we ran a series of Kendall’s tau rank correlation analyses in order to see the relations between gesture use and internal and external memory details. First, we test the association between internal details and representational gestures in the past, then in the future events. Next, we explore external details and their relationship with non-representational and representational gestures in past and future events, respectively. Fisher’s Z-transformation is used to compare correlation coefficients between conditions. As the final step, we explore the relationship between representational gesture use and memory phenomenology ratings (vividness, reliving, mental time travel).

In order to see whether there were any differences in terms of gesture rate across conditions, we calculated gesture per utterance (i.e., gesture use with respect to all verbal utterances) by dividing the total number of gestures (i.e., summation of representational, non-representational, deictic, and other gestures) for each event narrative by the total number of utterances (see Fig 1, for representational gestures). This analysis revealed that past, future, and non-autobiographical events differed from each other F(2, 80) = 3.44, p = .04, η2 = .02, yet the differences could not survive the Tukey’s test for post hoc analysis.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Scatterplots illustrating relations between event conditions and representational gesture rate.

Note. None-ABM corresponds to non-autobiographical memories.

https://doi.org/10.1371/journal.pone.0281748.g001

When we controlled for the influence of visuospatial skills and spatial imagery skills through participants’ CBT and MRT scores, we did not observe any relation of these skills on gesture rate in any condition (past: CBT, τ = .06, p = .62, MRT: τ = .10, p = .34; future: CBT, τ = .00, p = 1.0, MRT τ = .06, p = .60); non-ABM: CBT, τ = -.04, p = .76: MRT, τ = .03, p = .77; for CBT and MRT, respectively). Although we did not observe the effect of individual differences measured through these metrics, we have found that representational (τ = .34, p = .004) gestures were correlated in future and past conditions as well as non-ABM and past conditions (τ = .23, p = .05). Past and future conditions were also correlated (τ = .38, p = .001) in terms of the non-representational gesture rate. These correlations may reflect a gesturing tendency at the individual level regardless of the content, yet the measures in the current study were not able to detect it.

Table 1. presents the total number of gestures, means, and standard deviations broken down by event detail categories.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Means and standard deviations.

https://doi.org/10.1371/journal.pone.0281748.t001

The main question for the present study was: Are gesture types related to the types of event details? We started out with the most critical question by examining whether representational gestures were associated with internal event details. To be consistent with the literature using the Autobiographical Interview [37, 57], we calculated the total count scores both for internal details and for gesture use. Since the data were not normally distributed, we calculated Kendall’s tau-b for the correlations between memory details and gesture use. For summaries of the relevant correlations, please refer to Tables 2 and 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Correlation matrix for past events.

https://doi.org/10.1371/journal.pone.0281748.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Correlation matrix for future events.

https://doi.org/10.1371/journal.pone.0281748.t003

We confirmed our predictions in the past autobiographical event condition. For the past events, representational gestures correlated with internal details (τ = .27, p = .02). To delve into which particular episodic details were related to the representational gestures, we looked into internal detail categories. Representational gestures were correlated with internal place details (τ = .28, p = .02) as well as internal perceptual details (τ = .36, p = .002) as depicted in Table 2, confirming our prediction that gestures may help individuals to sustain spatial and sensorimotor information associated with the mental representations stored in the memory. Even though we did not have specific predictions for the relationship between external details and gestures, we sought a relationship with the non-representational (beat) gestures because they are discussed in the context of discourse processes. Confirming this, for the past events, non-representational gestures were correlated with external details (τ = .35, p = .002); specifically with personal semantic details, namely AF (τ = .27, p = .02) and SK (τ = .50, p < .001).

For the future autobiographical events, as presented in Table 3 we did not observe an association between representational gesture use and episodic (internal) details (τ = .10, p = .39). However, the analysis revealed that both representational (τ = .26, p = .02) and non-representational gestures were related to external details (τ = .27, p = .02). Non-representational gestures were specifically correlated with repetition (τ = .29, p = .02) and SK (τ = .35, p = .005). Overall, in both past and future events, external details correlated with non-representational gestures (i.e., beat). Lastly, a Fisher’s Z-transformation revealed that the association between representational gestures and internal details for past and future conditions are not significantly different (z = 1.20, p = .23).

An additional dimension in exploring the relationship between episodicity and gesture use in event narrations is the subjective sense of recollection. In other words, are representational gestures related to the phenomenology of autobiographical memories? First, we sought to investigate the association between overall gesture rate and subjective phenomenology. For the past events, we observed that gesture per utterance only correlated with reliving (τ = .25, p = .04), whereas the association between vividness (τ = .08, p = .54) and mental time travel (τ = .24, p = .053) did not reach significance. Regarding the phenomenological ratings, we observed that the feeling of reliving was correlated with both representational (τ = .29, p = .02) and non-representational gestures (τ = .24, p < .05) in past events. Also, the feeling of mental time travel was correlated with non-representational gestures (τ = .25, p = .04), suggesting that increased use of gestures is benefiting the reliving component of the autobiographical memories. In contrast to past events, neither the gesture categories nor gesture per utterance correlated with the phenomenology ratings in future event narrations (for the gesture per utterance: reliving, p = .52; vividness, p = .82; mental time travel, p = .09, as presented in Table 3).

Discussion

In the present study, we investigated the relationship between co-speech hand gestures -particularly representational gestures- and episodic autobiographical event representations. We asked participants to narrate past autobiographical, future autobiographical, and non-autobiographical events that differ in terms of their level of episodicity. Our study yielded a number of findings: First, we did not observe gesture rate differences across event conditions. Second, and most central to our current goal, we found that representational gesture production was associated with the specific episodic details as measured by the Autobiographical Interview [37]. Third, we have also observed a correlation between gesture use and subjective reports of recollective phenomenology in the past event condition. Surprisingly, this association holds both for representational and nonrepresentational gestures. Finally, non-representational gesture use and non-episodic information in the narratives of past and future episodic events were associated.

Regarding the first finding, our reasoning was that preparing to narrate an autobiographical event (as opposed to a non-autobiographical one) would trigger gesture use (due to the specific events’ multimodal representation) and gesture use would, then, represent and activate additional episodic information. Surprisingly, we did not observe differences in the rate of gesturing across these conditions. One possible explanation may lie in our choice of method in recruiting the events. Since we used spatial cue words (e.g., street) to sample the autobiographical events and spatial contexts (e.g., bank transaction) for the non-autobiographical conditions, the participants may have treated the events to be narrated as more scene-based than event-based [58]. when constructing them. For instance, this might have resulted in future events to resemble non-autobiographical scenes rather than autobiographical/episodic events (see Table 1 for gesture rates in conditions). Furthermore, given the evidence on the recruitment of gestures during spatial processing (e.g., [59–61]), the added scene-based treatment may have rendered the three conditions more similar in terms of the need to engage representational gestures. Given that in a previous study, participants gestured more in telling frightening experiences than when they were telling how to do grocery shopping [36], future designs need to vary the nature of autobiographical events to better outline the contribution of gestures.

Despite the lack of differences in the rate of gesture use across conditions, one finding lends strong support for an association between gesture use and episodic details: In the past event condition, we found a positive correlation between the episodic (internal) details and representational gestures. While the mechanism underlying these correlations between internal/episodic details and representational iconic gesture use is not completely clear based on the current design, our finding that representational gesture use was positively correlated with the existence of sensory details and place-related details suggests that gestures may indeed help with maintaining spatial and sensory information associated with the event representation (e.g., [9]). This is an important finding for the event construction literature according to which an image of the location serves as a scaffold for elaboration with additional details [30, 62]. If gestures support representing the spatial context in autobiographical episodic memory, it is possible that gesturing would have a corresponding benefit in recalling additional episodic details. This is the first study to our knowledge that sought a relationship between episodic informational units (e.g., sensory details) and gesture production. An alternative proposal to characterize the relationship between gestures and episodic representations reverses the directionality by arguing that gestures are supported by hippocampal episodic event representations [36]. Support for this argument comes from the finding that gesture use was compromised in patients with hippocampal damage, such that across different narratives (i.e., procedural and episodic) patients gestured less compared to the healthy group. The level of episodic details and gesture frequency was associated in a small group of neurotypical participants (N = 9); however, no such link was observed in the patient group. Although it is intuitive to think that gestures emerge from episodic representations, the evidence is not conclusive based on Hilverman et al.’s [36] selection of methods for two reasons: (1) Their choice of autobiographical events was emotional in nature; namely, the most frightening experience and personal context of JFK’s assassination. Given that hippocampus is also implicated in emotional processing [24, 63, 64], it is not possible to tease out whether it is the episodicity or the emotionality of the events driving the effects. (2) Their control -procedural event- condition included imagining scripted events that were non-episodic in nature. However, it is not possible to disentangle whether less frequent gesturing is due to the lack of episodic quality of the events or the fact that the participants needed to construct these events rather than remembering them. A good control condition would be to also include an imagined autobiographical event condition where construction processes would still be at work, but the content would be episodic. All in all, Hilverman’s finding that gestures are sensitive to episodic memory representations presents an interesting case for our purposes. Others have also alluded that the relationship between gesture use and cognitions is bidirectional [33]. We contend that “episodicity” of the narrative may trigger gesture production, and gestures in turn may guide the retrieval of further episodic details.

Interestingly, however, the association between gesture use and episodic details was observed only for the past autobiographical events. Two factors might have contributed to the lack of association between internal details and gesture use in future events. The narratives participants produced were rather short to allow for enough episodic elements, either in gesture or verbal form. Second, as Hostetter and Alibali [35] showed when verbally describing images, individuals gestured more than when they had previous physical experiences with the stimuli in the images. Therefore, since our participants did not have previous sensory and motor experience with the information they were describing in the future autobiographical event condition, gestures could not be effective in helping speakers sustain episodic information stored in memory to the extent that they did in the past event condition. Although gesture use can activate, manipulate, package, and explore spatio-motoric explanations and in turn help schematize information [2], it can be argued that particularly activation and manipulation of sensory and motor information is stronger for the past event representations than future event representations. Gesture use may activate the already established spatio-motoric event details better than formulating new event details. Then, when asked to imagine a personal event, not having a past sensory experience may lead to a less strong connection to the episodic details.

Furthermore, Addis [18] rightfully hinted that, in addition to the usual episodic elements, future event representations may also include memory for the imagination process itself. For example, engaging in reasoning or changing content to be more plausible would be manifested in external details, maybe in the form of repetitions or general semantics. This may have contributed to the unexpected relation between representational gestures and external events such that when information is captured and packaged in representational gestures, the freed-up cognitive resources may be dedicated to the discourse processes, i.e., the use of external details, in order to select and contextualize the internal event.

With regards to phenomenology, we found that gesture use was related to the sense of reliving the past events however this was not specific to the representational gestures as we predicted. Non-representational gesture use was also correlated with the subjective sense of recollection. Although prior work has not tested this relationship, the evidence on gestures alleviating cognitive load in narrative recall and problem-solving (e.g., [8, 65–67]) suggests that reduced cognitive demands during retrieval via the use of gestures may lead to richer reflective recollective experience.

Even though we did not have specific predictions regarding non-representational gestures, we did find rather strong correlations between nonrepresentational gestures and the external details in both past and future events. A tentative explanation relies on the possible communicative role played by both the non-representational gestures and external details. The beat (non-representational) gestures are thought to capitalize on communicative functions, such as fluency in discourse, rather than representative functions [68, 69]. Interestingly, evidence from the autobiographical literature also suggests that the reason external details, and therefore, semantic information, are included in the autobiographical narratives is to elaborate or “embellish” the discourse [70 (p. 1)]. It is not surprising, then, when non-representational gestures assume a fluency-resolving role, they would enhance the generation of semantic information in autobiographical narratives. Further studies are needed to establish whether there is a mapping between the semantic information in autobiographical memories and beat gestures or whether beat gestures are more frequently used during the search for specific episodic information.

Given this backdrop, the present findings provide good evidence that gestures support episodic event representations. Their role seems to be stronger in lived past experiences. To further qualify the relationship, future research is needed by carefully manipulating the gesture use (i.e., instructed gesture use) as well as the sampling of episodic events. The present study was a first step in establishing the contribution of co-speech gestures to autobiographical memories as complex mental representations. Future studies can be designed to further test the specific predictions based on the gesture-for-conceptualization framework [2]. In particular, the role of gestures’ specific functions (activation, manipulation, packaging, and exploring spatio-motoric information) for past and future event representations can be investigated. Future studies should also replicate the findings in other languages. Even though prior work has successfully used the Autobiographical Interview in Turkish (e.g., [31]), the way gestures represent concepts such as time and space, may be different across languages (e.g., [71, 72]) which may have implications on their influence on memory retrieval. Last, although our findings did not provide support for the role of visuo-spatial working memory and spatial skills on gesture use in narrating autobiographical memories, future research should also examine the role of individual differences in using cognitive resources, both for visuo-spatial and verbal skills, on episodicity in more detail [73].

The present research also represents a fertile research area with practical implications. A growing literature has shown that reduced episodic specificity (memories with a low level of episodic details) is associated with psychological health (e.g., [74]) as well as Alzheimer’s Disease (e.g., [75]). Since most of these studies are concerned with verbal and written accounts of episodic events, investigating gestures’ role in contributing to episodic specificity of memories would be critical. Given the evidence that information in gesture is sometimes repeated in speech, and sometimes only reserved for gesture (e.g., [76]), future work examining whether episodic details in gestures and in speech complement and influence each other would be a potential addition to the existing efforts in designing intervention programs to induce episodic specificity [77].

In conclusion, this study presents one of the first pieces of evidence about the role of representational gestures on episodic event details, such as sensory and place-related details and recollective quality in autobiographical memories. These associations were specific to recalling past events rather than imagining future events or non-autobiographical events. Our findings highlight the potential of gesture production in maintaining spatial and sensory information associated with the event representation that in turn aids the retrieval of details in autobiographical memories.

Acknowledgments

We would like to thank research assistants Nesli Aslan, Deniz Alp Domaniç, Berkay Toksöz, Lal Atatüre, Cansu Ağaoğlu, and Belgin Deryalar for their assistance in participant recruitment, data collection, and coding gestures. We would also like to thank the reviewers for their valuable feedback on this paper.