Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Effects of a brief online emotion word learning task on negative emotion differentiation, emotional self-efficacy, and prospective distress: Preliminary findings

  • Lindsey M. Matt ,

    Contributed equally to this work with: Lindsey M. Matt, T. H. Stanley Seah

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Writing – original draft, Writing – review & editing

    lmatt@lyrahealth.com

    Affiliation Lyra Health, Burlingame, California, United States of America

  • T. H. Stanley Seah ,

    Contributed equally to this work with: Lindsey M. Matt, T. H. Stanley Seah

    Roles Formal analysis, Writing – review & editing

    Affiliation Department of Psychiatry, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America

  • Karin G. Coifman

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Resources, Supervision, Writing – original draft, Writing – review & editing

    Affiliation Department of Psychological Sciences, Kent State University, Kent, Ohio, United States of America

Abstract

Background

Disruptions in emotion processing are common across psychological disorders. Research suggests that emotion differentiation (ED; specificity in language used to characterize one’s emotional experience) and emotional self-efficacy (ESE; perceived ability to understand and manage one’s emotions) are important transdiagnostic factors associated with various psychological benefits. Whether ED and ESE can be improved in adults remains largely unclear.

Methods

Using a longitudinal prospective design, we tested a brief online training targeting emotion word knowledge (vs. a control condition) to see if it improved negative ED (NED) and ESE in a college sample. Moreover, we tested if changes in NED or ESE mediated the effects of the training on levels of psychological distress one week and two-months post-intervention.

Results

Findings provided partial support for our hypotheses. Individuals whose ESE increased post-intervention reported lower levels of distress two months later. Moreover, exploratory analyses revealed those who demonstrated greater training engagement experienced increases in NED that in turn predicted lower distress one-week post-intervention. However, there were no direct effects of intervention group on NED or ESE and distress.

Conclusions

These findings highlight the potential of a remotely-administered emotion-language intervention to influence key dimensions of emotion processing and suggest avenues for further refinement. Both NED and ESE may be malleable for some, and that enhancements in ESE may produce long-term psychological benefits.

Introduction

To date, mounting evidence from decades of research suggests that disruptions in emotion-related processes are common across psychological disorders and linked to the development and maintenance of psychopathology [1]. These deficits can be understood as manifesting on various dimensions, including how individuals understand their emotional experiences and perceive their ability to manage or cope with emotions. Indeed, deficits in awareness and difficulties identifying one’s emotional experience have been robustly associated with elevated symptomatology [2,3]. Moreover, patients often appear to exhibit relatively simplistic or impoverished conceptualizations of emotional experiences (e.g., alexithymia) [4], report highly polarized affective states [5], and view their experience of negative emotions (e.g., anxiety) as aversive or unmanageable [6]. Indeed, avoidance of attention to negative feelings can result in the engagement in maladaptive behaviors (e.g., substance use; avoidance of feared situation) that interfere with mental health treatment and worsen prognosis [7]. As the mental health field continues to move toward transdiagnostic approaches to diagnosis and treatment, efforts to identify specific emotion processes that cut across mental health conditions are crucial.

Past research suggests that emotion differentiation (ED; also termed “granularity”), which is the specificity of the language individuals use to describe and label their emotional experience [8] may be a particularly important component in adaptive emotional processing and a worthy target for treatments [9]. While improving patients’ facility with emotion language (a process inherent in ED) is common amongst psychological treatments, this is only explicit in some treatments, where emphasis on increasing emotion language literacy and facility is emphasized (e.g., emotion-focused therapy) [10]. Further, relatively little is known as to under what conditions or processes ED ability can be enhanced. The present research aims to address this gap in the literature by examining the effects of an emotion word learning task (relative to a control condition) on ED ability assessed in daily life.

The field has witnessed a surge in ED research in recent years, with most studies reporting significant associations with clinically-relevant affective and behavioral outcomes [9,11]. Specifically, higher levels of negative ED (or NED) have been associated with use of adaptive emotion regulation strategies [8,12], higher levels of self-reported mindfulness and less emotional lability [13], as well as better social functioning [14,15]. Importantly, some studies have demonstrated that healthy individuals tend to exhibit greater levels of NED compared to clinical samples (e.g., depression [16]; borderline personality disorder [17]; autism spectrum disorder [18]), with NED associated with greater wellbeing [19], suggesting that it could be generally protective against psychopathology. More importantly, studies have consistently found that NED predicts less engagement in a broad range of maladaptive behaviors commonly enacted to reduce distress [20], even in times of heightened stress [21].

Indeed, recent meta-analyses revealed consistent protective effects suggesting reduced reliance on such risk behaviors as substance use, binge eating, aggression, and social avoidance for individuals with higher NED, regardless of diagnostic status [11]. Hence, for individuals with clinically heightened levels of distress, higher NED may be protective. For example, higher NED predicts fewer urges and acts of non-suicidal self-injury in borderline personality disorder [22] and less social avoidance in individuals diagnosed with social anxiety disorder and individuals with sub-threshold social anxiety [23]. Similarly, among non-clinical samples, NED appears to be protective against binge drinking [24], aggression [25], and impulsive behaviors [26]. Notably, even in highly challenging contexts, NED appears protective against risk behaviors (e.g., substance use and binge-eating) during the COVID-19 pandemic [27] and may also facilitate adaptive health behaviors during chronic illness [28,29]. Taken together, these findings highlight ED as a potential transdiagnostic factor, with higher levels conferring clear psychological benefits, and lower levels contributing to the development and maintenance of a variety of psychological conditions.

Several mechanisms may underlie the benefits of NED. Certainly, there are clear associations between NED and greater reports of adaptive emotion regulation strategy use, suggesting that the ability to specify emotional experience provides valuable insight and information to make better choices [9,30]. A complementary line of research has begun to explore how precise emotion labeling, a process underlying ED, may facilitate implicit emotion regulatory action. For instance, Kircanski and colleagues [31] tested if emotion labeling could enhance traditional exposure therapy in a group of spider-phobic individuals. In one of four conditions, a group was explicitly instructed to employ affect labeling to describe their negative feelings and worries about the spider during exposure. Affect labeling was found to be more effective than the other three conditions employing conventional emotion regulation strategies (i.e., reappraisal, distraction) and controls (i.e., exposure alone) in reducing emotional arousal (indexed via skin conductance), and more effective than distraction in reducing behavioral avoidance when viewing the spider one week later. These findings are largely consistent with a recent meta-analysis of neuroimaging research suggesting that just the act of conceptualizing emotional experience by activating emotion concepts can reduce reactivity in the amygdala [32].

Enhancing emotion differentiation

Given its broad potential impact on mental health, we must then wonder what processes could make ED more or less malleable. A recent investigation testing an emotion knowledge intervention demonstrated small benefits to participants by increasing NED during a lab task. In this study by Vedernikova and colleagues [33], participants received five days of training on emotion concepts and were encouraged to then apply them to their own lives during the training. After, they completed a hypothetical scenario task in which ED was derived. The experimental condition showed small but significantly higher NED than controls. Importantly, however, ED was measured from a lab task and not daily life experiences as is most commonly done. Other research by Van der Gucht et al. [34] explored enhancing NED through mindfulness-based intervention that included guided group meditation as well as informal practices focused on increasing in the moment awareness of experiences and sensations. They found that individuals exposed to an eight-week mindfulness-based stress reduction course showed improvements in NED both at post-task and four months later, though these results became non-significant when controlling for mean levels of negative affect. Taken together, these findings suggest that NED is malleable and that these changes could be influenced by targeted intervention.

More broadly, many common psychotherapies include explicit components that emphasize emotion concept and language learning such as Dialectical Behavior Therapy [35,36] and Coping Cat [37]. These treatments have been effective in reducing self-destructive behavior in adults [38] and distress/anxiety in children [39] compared to treatment-as-usual. Basic components include education about the function of emotions and practicing self-labeling of feelings. Randomized control trials of emotion-focused therapies (e.g. person-centered, Gestalt) which encourage explicit attendance to and use of emotion as a source of key information suggest that these treatments are also effective in managing depression equivalent to or better than other approaches [40]. A more recent study also found that an emotion regulation task, which included psychoeducation on emotions, including how to attend to and label emotions in oneself, and the link between emotions, behaviors, and triggers, was associated with less engagement in risk behaviors and greater use of emotion regulation strategies in a group of early adolescents [41].

Emotion language learning is also being both assessed and taught more commonly in schools. A theoretically based emotions course that focused on teaching toddlers to identify and label their emotions was shown to increase performance on tests of emotion knowledge and also contributed significantly to the development of emotion regulatory skills [42]. Other programs that teach specific emotion words by associating them with related antecedents and behaviors (i.e. I Can Problem Solve, ICPS [43], Promoting Alternative Thinking Strategies, PATHS [44]; Second Step [45]) are related to increases in prosocial behavior and reductions in behavioral problems [4648]. Finally, the RULER (Recognizing, Understanding, Labeling, Expressing, and Regulating Emotions) social-emotional learning program has been shown to consistently improve social cohesion in classrooms and also increase engagement and reduce problematic conduct in at-risk children and teens [49,50].

Emotion differentiation and emotional self-efficacy

The research described above has suggested that processes by which ED may exert benefits and reduce distress is twofold: 1) by informing individuals more precisely of their experience (perhaps facilitating more adaptive responses [30,51] and 2) by implicitly down-regulating arousal associated with the intensity of the experience [52]. However, one key alternative process may be that ED facilitates the individual’s feeling of emotional competence. This perception of one’s ability to perceive and understand, as well as to utilize and manage emotions is known as emotional self-efficacy (ESE) [53]. Broadly, self-efficacy has been defined as the belief individuals have that they are capable to produce change with their own actions [54]. While not considered a trait, self-efficacy develops alongside symbolic thought and language in childhood as we come to understand cause and effect relationships and self-awareness [54] and is then bolstered by responsive environments that can be influenced by personal action. When older, self-efficacy continues to grow when individuals engage in successful attempts to control the environment that they then attribute to their own behavior [55].

Low reported levels of domain-general self-efficacy are known to be associated with depression, anxiety, and avoidant behavior [55,56]. Alternately, higher levels of self -efficacy are associated with greater success in therapeutic interventions to manage substance abuse and eating disorders [57]. Moreover, self-efficacy is a key component of self-regulation with those high in self-efficacy more likely to set high goals and, in pursuit of them, modulate their level of effort as they engage in goal-directed behavior and appropriately persist when faced with challenges [58,59]. Importantly, self-efficacy may be best measured within specific domains [60]. As higher levels of self-efficacy are associated with better functioning in their given domain, it follows that higher ESE has been shown to be associated with more positive and less negative concurrent mood [53]. Interestingly, two recent investigations have demonstrated increases in ED just based on repeated demand to report emotions via experience sampling. No intervention was needed, only repeated practice of labeling current emotional experience increased the capacity to specify discrete experiences, improving ED [61,62]. Hence, because practice can drive an increase in self efficacy [63] a link between ESE and ED seems possible and could be a key pathway by which emotion conceptualization can come to reduce levels of distress.

Current investigation.

To date, a variety of promising interventions have attempted to enhance emotion differentiation alone or in conjunction with other components of mental health. Limitations to recent targeted approaches include a lack of a control group [34], use of a high time-demand intervention [34] and lab-based (vs. daily life) derivation of differentiation [33]. Additionally, the potential role of self-efficacy in the pathway between emotion differentiation and psychological benefits has not yet been explored.

In the current investigation we developed and tested a brief novel online emotion word learning task intended to specifically increase and reinforce participant knowledge of discrete emotion concepts. The development of this task was informed by current therapeutic approaches in work with both adults (DBT) [35] and children (Coping Cat) [37]. We tested the impact of this task (vs. neutral control task) on levels of ED and ESE and whether shifts in either of these constructs would influence changes in psychological distress across time.

We anticipated that exposure to the emotion word learning task would be associated with improvements in NED and potentially ESE, as well as reductions in psychological symptoms and distress. However, we also considered that NED could mediate the association between task condition and psychological symptoms/distress across time. Indeed, it could be that the intervention improves attention and capacity to label emotional experiences, thereby improving psychological health. Yet, it is possible that an alternative mechanism by which this process unfolds was through increases in ESE. Therefore, we considered ESE as an alternative mediator. Further, we explored whether individual differences in levels of engagement with the word learning task influenced the associations described above. Given the relative absence of prior evidence demonstrating that emotion language interventions can impact either process, as well as a marked absence even demonstrating associations between the two constructs, we generally treated all analyses as exploratory.

Materials and methods

Transparency and openness

Below we report how we determined our sample size, all data exclusions, all manipulations, and all measures in the study. Study was approved by Kent State University Institutional Review Board in 2014. Data were collected from 2015–2016 and analyzed using R and SPSS. This study’s design and its analysis were not pre-registered.

Participants

A priori power analyses were initially conducted based on medium effect sizes, which may have overestimated potential expected effect sizes [62]. This a priori estimation did suggest adequate power in a sample of 150 participants. However, the analytic plan changed during the peer review process and subsequent analyses revealed that effects were small and the sample potentially underpowered (see Results). Because post-hoc power estimation has been greatly criticized [64], we instead utilized tests of equivalence to evaluate null effects [65]. These are reported in detail in the results section.

N = 150 English-speaking adults from a large Midwestern university were recruited from 2015–2016 via open enrollment from the psychology research subject pool. Participants were required to have an Apple/Android OS smartphone to be eligible for the study. While the sample was unselected, participants displayed higher than average levels of distress compared to a previous undergraduate sample (see Measures section). From this initial sample, 32 individuals were excluded as they failed online accuracy checks (n = 2) or did not complete one or both parts of the online task (n = 30), leaving a sample of 118 participants. Note that actual sample size used for different analyses varied due to missing data. We noted this important information for each analysis in the results, as well as in Tables 1 and 2.

thumbnail
Table 1. Bivariate correlations of baseline and post-task variables (n = 118).

https://doi.org/10.1371/journal.pone.0299540.t001

thumbnail
Table 2. Comparison of demographics, self-report, and task measures by condition (n = 118).

https://doi.org/10.1371/journal.pone.0299540.t002

Procedure

Study participation took place over approximately two months. Participants completed a baseline lab session where they provided written consent and were assigned a unique study identification number. The consent form detailed risks of study participation including minor risk of mood change in response to the questions being asked or events being remembered. If participants did not wish to answer a question, they were permitted to skip it. They then provided demographic information and reported psychological symptoms, distress, and ESE and were then randomized to one of two word learning task conditions. Verbal intelligence was also estimated at that time. Finally, participants received training on how to respond to experience sampling diary. Following this session, participants engaged in one week of daily experience sampling where they indicated how they were feeling by rating specific emotion words (e.g. anger, sadness) five times a day for seven consecutive days using their smartphones. Afterwards, participants completed an online word learning task across two days, consisting of either emotion (experimental condition) or neutral (control condition) words. Following completion of the online word learning task, participants completed another week of daily experience sampling as above. Emotion word ratings during each phase of experience sampling were used to generate pre- and post-word learning task NED and emotional intensity. Finally, participants completed symptom, distress, and ESE measures again approximately one week and two months after the word learning task. See Fig 1 for flowchart of study procedures. All data collected were labeled by study identification number. Master list of study identification numbers and participant names was secured and able to be accessed only by corresponding author if necessary. Participants and research assistants were blinded to both experimental condition and study hypotheses, and participants were debriefed at the end of their participation.

thumbnail
Fig 1. Flowchart of study procedures.

SCL-90-R = Symptom Checklist 90-Revised, IDAS-II = Inventory of Depression and Anxiety Symptoms, ESES = Emotional Self-Efficacy Scale.

https://doi.org/10.1371/journal.pone.0299540.g001

Word learning task

Participants were randomly assigned to one of two possible task conditions (emotion vs. neutral) based on their study identification number. The word learning task was sent to participants via e-mail across two sequential days following completion of the first seven days of experience sampling. All parts of the intervention are available online (https://osf.io/sf7mw/).

Day one.

Participants received an email via Qualtrics asking them to complete a word learning task. They were informed that they would be presented with a series of words and corresponding definitions and asked to study and remember each word as they would be quizzed on them for the next two days. In the experimental condition, fifteen emotion words, including the seven used to derive NED, were presented in random order along with definitions that included detailed information such as the emotion’s function and associated features. For example, fear was described as a feeling that suggests a threat is present, though the danger may be real and present or based on your thoughts. Participants were told that fear may feel like shakiness, sweatiness, and/or jitteriness in the body and that they may respond by seeking shelter, running away, or freezing in place. This presentation was based on discrete emotions theory, which suggests each emotion has a unique antecedent, behavioral output or action [66]. and is modelled after the Emotion Regulation module of Dialectical Behavior Therapy [35] in which individuals are taught about the function of emotions and the role they play in understanding one’s environment.

Techniques from the learning literature were used to help enhance memory of novel information. Following presentation of each word, participants were asked to use a blank space below to write down a related memory that comes to mind. Known as the self-reference effect [67], research has shown that connecting novel information to the self through asking questions such as ‘Does the word describe you?’ results in enhanced memory even when compared to semantic or other-referent encoding strategies [68]. Following presentation of all words, participants completed a randomized-item quiz in which they matched parts of the definitions they learned to multiple choice word options. Use of quizzes in classroom settings is associated with better performance both on exams and in the overall course vs. studying of the material alone [20,69]. Further, multiple choice quizzes (vs. open ended or short answer) offer a concise approach that has been shown to result in enhanced recall of quiz information in the future [70]. Participants were also provided with explicit corrective feedback if they made an incorrect answer selection, as this type of feedback has been shown to significantly increase recall of knowledge, including in assessments of language learning [71].

Day two.

On the second day, participants received another email via Qualtrics asking them to complete a quiz about the information they read the previous day. This quiz was identical to those seen on day one but presented items in a novel randomized order. Use of repeated testing has been shown to both consolidate learning and enhance recall [72].

Control condition.

The neutral word condition used the same procedures. However, participants were presented with a variety of words describing household items (i.e. refrigerator, stove) and definitions that included detailed information about their function and history. The presence of a matched neutral word condition allowed for exploration of the unique effects of emotion vs. neutral word learning. These words were chosen as the control condition because we expected participants to demonstrate some variability in familiarity and frequency of contact with different household objects.

Measures

Tables 1 and 2 provide descriptive information (means, SDs, range) for the whole sample and by task condition respectively.

Current symptoms.

The Inventory of Depression and Anxiety Symptoms (IDAS-II) [73] is a measure of current symptoms of major depression and anxiety disorders. Given the interest in broad symptoms of psychopathology in the current study, an overall symptom score was created by summing scores from the Dysphoria, Lassitude, Insomnia, Appetite Loss, Appetite Gain, Ill Temper, Panic, Social Anxiety, Traumatic Intrusions, Traumatic Avoidance, and Mania subscales. The symptom score was found to be internally consistent (α = .94) and displayed adequate range (baseline M = 126.93, SD = 33.45, Range = 68.00–240.00).

Current distress.

The Symptoms Checklist-90-Revised (SCL-90-R) [74] is a widely-used measure of current symptoms of psychopathology. Here, a combination of the depression, hostility, and anxiety subscales were used to evaluate distress [75]. Scores were averaged to derive an estimate of general distress and showed good internal consistency (α = .93) and range of distress (baseline M = .82, SD = .54, Range = .03–3.07). Average scores were higher than those in other known undergraduate samples (0.67) [76] but not as high as those seen in clinical populations (1.30–2.00) [77].

Aggregate measure of symptoms and distress.

Bivariate correlations revealed a significant large positive (r = .63 - .78, p < .001) relationship between IDAS-II and SCL-90-R scores at each time point, and thus we chose to standardize and aggregate these variables in our analyses (see Tables 1 and 2).

Emotional self-efficacy.

The Emotional Self Efficacy Scale (ESES) is a measure of adaptive emotional functioning based on the four-factor model of emotional intelligence [78] and assesses the perceived ability to understand, regulate, and observe emotions in the self and others, and to use emotions to facilitate thought. Items were summed to create a single measure of ESE with good internal consistency (α = .95) and range (see Tables 1 and 2).

Verbal intelligence.

The Spot the Word task is designed to quickly and accurately estimate premorbid intelligence. It was included in the current study for use as a potential covariate as we sought to isolate the effects of the word-learning task from individual’s baseline vocabulary. It is comprised of sixty pairs of words, in which participants are asked to distinguish between a valid word in the English language (“true” target word) and a non-word foil. A sum score of correct responses was generated and serves as an estimated verbal intelligence quotient [79] (see Tables 1 and 2).

Experience sampling diary.

During the laboratory session, participants were trained in the use of a mobile phone application (The Personal Analytics Companion; PACO, [80] used to received daily prompts, and were guided through an example prompt to ensure understanding. Individuals were prompted five times within a 14-hour period, with each prompt remaining available for up to 60 minutes.

Each of the prompts assessed current emotional state by asking participants to rate a set of seven negative emotion words (Fear, Sadness, Distress, Guilt, Anger, Disgust, Shame) using a Likert-type scale (0 = ‘None’ to 7 = ‘Strongly’) and six positive emotion words, not relevant to this investigation. The words used represent a continuous range of arousal/activation within the negative valence of the affective circumplex [81] and are similar to those used in other studies of ED [11]. Ratings were made up to 5 times per day over 14 total days (excluding word-learning task days), for a total of 70 possible prompts for each participant. One prompt each day contained an accuracy check item in which participants were asked to select ‘0’ to ensure they were reading the presented content. Accuracy responses and number of prompts completed were reviewed. Of the whole sample (N = 150), five individuals did not provide any diary data. Overall, diary compliance rates were good (Overall: M = 55.42 out of 70 (79%), SD = 14.35; Pre-task: M = 30.15 out of 35 (86%), SD = 7.17; Post-task: M = 25.28 out of 35 (72%), SD = 10.05). In line with established experience sampling procedures [82], five participants (of the 118) who completed less than two SDs below the average number of total diaries (cutoff < 26.96 responses) were excluded in analyses where relevant.

Emotion differentiation. Two separate NED scores were generated for each participant from emotion ratings collected pre- and post-word learning task respectively. An index of NED was derived by calculating intraclass correlations coefficients (ICCs) with absolute agreement across the set of negative emotion words. Small correlations suggest that an individual can use more discrete terms to describe their emotional experiences, while large correlations suggest individuals describing their emotional experiences in a more uniformed manner. Scores were reversed by subtracting from 1 to aid interpretation, with larger values corresponding to higher ED [24]. Nine participants were excluded in our analyses because they did not have pre-/post-task NED scores due to limited variation across emotion word usage in a pattern that suggested inaccurate reporting and/or measurement error.

Negative emotion intensity. Negative emotion intensity (NEI) was derived by calculating the mean across participants’ ratings of negative emotion words across each 7-day phase (pre- and post-task) experience sampling (see Tables 1 and 2). NEI has been consistently explored in ED research as higher levels of intensity may demand more ED [11].

Word learning task engagement.

To assess the extent with which participants were engaged in the word learning task across both emotion and neutral conditions, we examined their performance on the memory recall task and multiple-choice quizzes (from both Day one and two).

Memory recall performance. Participants’ engagement with the memory task was assessed by coding the specificity of recalled memories associated with each emotion (vs. household item) word. A trained research assistant (blinded to study’s hypotheses) coded each response (range: 1–5) based on whether participants provided a specific memory (5; e.g., “I felt sad when I had to say goodbye to my family during freshman year of college.”), general memory (4; e.g., “I feel sad whenever I miss my friends.”), definition of the emotion (or household item) word (3), unsure (2), or incorrect and/or missing data (1). These scores were then averaged to provide an index of the extent with which participants were able to generalize knowledge acquired through the word learning task on self-relevant memories. The mean score for those in the emotion condition (n = 58) was 4.34 (SD = .41; range: 2.67–5.00) and those in the neutral condition (n = 60) was 4.57 (SD = .38; range: 3.07–5.00).

Multiple-choice quiz scores. To examine participants’ quiz performance, we calculated the total number of errors (out of all 15 questions) committed for each quiz taken on Day one and two respectively. Across both conditions, participants did not commit any errors for the quiz taken on Day one. Conversely, participants committed errors for the quiz taken on Day two across both conditions. Specifically, the mean error rate for those in the emotion condition (n = 58) was .06 (SD = .08; range: 0–.27) and those in the neutral condition (n = 60) was .07 (SD = .08; range: 0–.33).

Data analytic plan

First, we planned to confirm that groups were equivalent at baseline using chi-square and independent samples t-tests on all demographic and pre-task measures. If groups differed on any variables of interest, we included them as covariates in subsequent analyses. We also ran bivariate correlations between baseline and post-task variables to identify relationships between variables and potential covariates to include in subsequent analyses.

To examine the effects of the word learning task (emotion vs. neutral condition) on primary outcomes, we conducted a series of independent samples t-tests to assess group differences in NED, ESE, and aggregate measure of symptoms/distress. If non-significant findings emerged, we conducted equivalence testing using the two one-sided tests (TOST) approach as outlined in Lakens et al. [65] to ascertain whether non-significant results were true null effects or rather due to our sample being underpowered to detect between-group differences. This approach involves setting two equivalence bounds that enables examination of whether we can “reject effects that are as extreme or more extreme than our smallest effect size of interest” (p. 47) [65]. Due to limited prior work, we set equivalence bounds based on small effect size (Cohen’s d = ± 0.2) following convention to be conservative.

Finally, we were interested in understanding whether NED or ESE might underlie the potential relationship between task condition and distress (direct effects), where increases in NED or ESE post-task might result in reductions in distress over time (indirect effects). Therefore, we planned to conduct mediation analyses using Hayes’ PROCESS macro (Model 4) [83] to explore both direct and indirect effects. Further, to explore whether levels of task engagement (as assessed by memory specificity and quiz scores) influenced these associations, we conducted moderated mediation analyses using Hayes’ PROCESS macro (Model 7) [83]. Fig 2 provides an illustration of both mediation (2A) and moderated mediation (2B) models.

thumbnail
Fig 2.

Examples of mediation (2A) and moderated mediation (2B) models in which negative emotion differentiation or emotional self-efficacy mediates the relationship between task condition and post-task symptoms/distress, and whether these associations were moderated by level of task engagement. Fig 2C presents the effect of task condition on symptoms/distress at two-month follow-up as mediated by post-task emotional self-efficacy. The values on the c path are for the total effect, c’ path are for the direct effect, and ab path are for the indirect effect. Note. NED = Negative Emotion Differentiation; ESE = Emotional Self-Efficacy; emotion condition coded as 1, control condition coded as 0.

https://doi.org/10.1371/journal.pone.0299540.g002

Results

Preliminary analyses

We first confirmed that there were no differences in participants removed because of poor compliance or failure to meet accuracy checks and those that were maintained. Table 1 describes correlations and descriptive information between baseline and post-task variables for the sample (n = 118). We found that the aggregate measure of distress across time points was significantly negatively associated with baseline and post-task ESE, and significantly positively associated with baseline and post-task negative emotion intensity. Baseline NED was significantly negatively associated with post-task distress and at two-month follow-up. As in past research, NED was negatively associated with negative emotion intensity. Baseline verbal intelligence was not associated with any variables of interest. No other significant associations emerged between primary outcomes of interest.

Then, we confirmed that our experimental groups did not meaningfully differ on any key dimension at baseline. Indeed, as described in Table 2, chi-square and independent samples t-tests revealed no significant differences in age, gender, race/ethnicity, verbal intelligence, aggregate distress, ESE, NED, and negative emotion intensity between groups at baseline (ps > .05). These findings suggest both groups were generally equivalent at baseline. Further, there were no differences between groups in the average total number of diaries completed.

Primary analyses

Post-task group differences.

Separate independent samples t-tests were conducted to examine group differences in post-task NED, ESE, and aggregate measure of symptoms/distress (at one-week and two-month post-task). Results are described in Table 2.

NED. Results from independent samples t-tests indicated no significant difference between the emotion (n = 52, M = .49, SD = .27) and neutral (n = 52, M = .42, SD = .24) task condition on post-task NED, t(102) = -1.37, p = .174. Using the TOST procedure, results indicated that the observed effect size (d = -0.27) was not significantly within the equivalent bounds of d = -0.2 and d = 0.2, t(100.62) = -0.38, p = .647, mean difference: -.07, 90% CI [-.15; .01] and were not statistically equivalent to zero.

ESE. Results from independent samples t-tests indicated no significant difference between the emotion (n = 54, M = 109.35, SD = 22.79) and neutral (n = 57, M = 104.05, SD = 24.48) task condition on post-task ESE, t(109) = -1.18, p = .241. Results from the TOST procedure indicated that the observed effect size (d = -0.22) was not significantly within the equivalent bounds of d = -0.2 and d = 0.2, t(108.97) = -0.13, p = .550, mean difference: -5.30, 90% CI [-12.74; 2.14] and were not statistically equivalent to zero.

Distress at one-week post-task. Results from independent samples t-tests indicated no significant difference between the emotion (n = 54, M = -.11, SD = .86) and neutral (n = 55, M = .12, SD = 1.01) task condition on distress at one-week post-task, t(107) = 1.26, p = .210. Results from the TOST procedure indicated that the observed effect size (d = 0.24) was not significantly within the equivalent bounds of d = -0.2 and d = 0.2, t(104.88) = 0.22, p = .587, mean difference: .23, 90% CI [-0.07; 0.53] and were not statistically equivalent to zero.

Distress at two-month post-task. Results from independent samples t-tests indicated no significant difference between the emotion (n = 48, M = -.04, SD = .90) and neutral (n = 46, M = .04, SD = .96) task condition on distress at two-month post-task, t(92) = .41, p = .680. Results from the TOST procedure indicated that the observed effect size (d = 0.09) was not significantly within the equivalent bounds of d = -0.2 and d = 0.2, t(90.93) = -0.56, p = .289, mean difference: 0.08, 90% CI [-0.24; 0.40] and were not statistically equivalent to zero.

Overall, we did not observe significant between-group differences in post-task variables. In addition, results from equivalence tests suggest that we do not have sufficient information to affirm the null hypothesis either. Together, these findings indicate that group differences may be small and that our sample may be underpowered to detect them.

Mediation analyses.

To examine the direct effects of task condition on distress and NED/ESE, as well as the indirect effects of post-task NED/ESE on the relationship between task condition and distress at one-week and two-month post-task, separate bootstrap mediation analyses using Hayes’ PROCESS macro (Model 4) [83] were conducted using 10,000 bootstrap samples. Group (emotion [coded 1] vs. control [coded 0]) was entered as the independent variable, distress was entered as the dependent variable, and either post-task NED or ESE was entered as the mediator variable (Fig 2A).

NED. For the analyses examining post-task NED as a mediator, we also considered covariates such as baseline verbal intelligence and distress, pre-task NED, and post-task NEI but they did not significantly impact the results and are not reported here.

Distress at one-week post-task (Table 3). No significant direct effects were found between group (emotion: n = 50; neutral: n = 52) and post-task NED (b = .08, SE = 0.05, p = .120) or post-task distress (b = -.19, SE = 0.19, p = .333). There was also no direct effect between post-task NED and distress (b = -.31, SE = 0.37, p = .407). Additionally, post-task NED did not significantly mediate the relationship between group and one week post-task distress (b = -.03, SE = 0.04, 95% CI [-.11, .04]).

thumbnail
Table 3. Mediation analysis examining post-task NED as a mediator of the relationship between group and one-week post-task distress.

Estimates based on 10,000 bootstrap samples (n = 102).

https://doi.org/10.1371/journal.pone.0299540.t003

Distress at two-month post-task (Table 4). As above, there were no significant direct effects found between group (emotion: n = 45; neutral: n = 43) and post-task NED (b = .06, SE = 0.05, p = .261) or two-month post-task distress (b = -.11, SE = 0.20, p = .599). There was also no direct effect between post-task NED and two-month post-task distress (b = -.32, SE = 0.42, p = .447). Similarly, there was no evidence of mediation by post-task NED on two-month post-task distress (b = -.02, SE = 0.04, 95% CI [-.11, .04]).

thumbnail
Table 4. Mediation analysis examining post-task NED as a mediator of the relationship between group and two-month post-task distress.

Estimates based on 10,000 bootstrap samples (n = 88).

https://doi.org/10.1371/journal.pone.0299540.t004

ESE. For the analyses examining post-task ESE as a mediator, we also considered covariates such as baseline distress and ESE and reported if they significantly impacted the results.

Distress at one-week post-task (Table 5). No significant direct effects were observed between group (emotion: n = 53; neutral: n = 54) and post-task ESE (b = 6.71, SE = 4.56, p = .144) or post-task distress (b = -.12, SE = 0.17, p = .479). However, there was a direct effect of post-task ESE on one-week post-task distress (b = -0.01, SE = 0.004, p < .001) such that higher ESE was associated with lower levels of distress. As with NED, ESE did not significantly mediate the relationship between group and one-week post-task distress (b = -.09, SE = 0.07; 95% CI [-.24, .03]).

thumbnail
Table 5. Mediation analysis examining post-task ESE as a mediator of the relationship between group and one-week post-task distress.

Estimates based on 10,000 bootstrap samples (n = 107).

https://doi.org/10.1371/journal.pone.0299540.t005

Distress at two-month post-task (Table 6). No significant direct effect was observed between group (emotion: n = 45; neutral: n = 45) and two-month post-task distress (b = .06, SE = 0.19, p = .756). The direct effect of group on post-task ESE was trending towards significance (b = 9.44, SE = 4.83, p = .054). However, there was a direct effect of post-task ESE on two-month post-task distress (b = -0.01, SE = 0.004, p = .004) such that higher ESE was associated with lower levels of distress. Importantly, post-task ESE emerged as a significant mediator of the relationship between group and two-month post-task distress, such that higher ESE in those in the emotion condition were related to lower levels of distress at two months (b = -.12, SE = 0.07; 95% CI [-.28, -.002]). These results are illustrated in Fig 2C and detailed in Table 6. Note that the mediation effect disappeared after accounting for baseline distress and ESE.

thumbnail
Table 6. Mediation analysis examining post-task ESE as a mediator of the relationship between group and two-month post-task distress (controlling for baseline ESE and distress).

Estimates based on 10,000 bootstrap samples (n = 90).

https://doi.org/10.1371/journal.pone.0299540.t006

Moderated mediation analyses.

To explore participants’ level of engagement in the intervention itself (i.e., memory specificity and quiz error rate) as a moderator in the mediation model, separate bootstrap moderated mediation analyses using Hayes’ PROCESS macro (Model 7) [83] were conducted using 10,000 bootstrap samples. Group (emotion [coded 1] vs. control [coded 0]) was entered as the independent variable, distress was entered as the dependent variable, either post-task NED or ESE was entered as the mediator variable, and either memory specificity or quiz error rate was entered as the moderator variable (Fig 2B). As above, we considered baseline distress and pre-task NED/ESE as potential covariates and reported if they significantly impacted the results.

Memory specificity as moderator. We found significant moderated mediation effects for the model with post-task NED as the mediator of the relationship between group and one-week post-task distress, controlling for baseline distress and pre-task NED as covariates (n = 100). These effects are described in Table 7. Specifically, memory specificity during the intervention itself significantly moderated the association between group and post-task NED (b = .35, SE = .13, p = .011). Simple slopes analysis indicated that among those in the emotion condition, greater memory specificity (+1SD) was associated with higher levels of post-task NED (b = .20, SE = .07, p = .006) but this association was not observed among those at mean (b = .06, SE = .05, p = .241) and lower (-1SD) memory specificity (b = -.08, SE = .08, p = .320). Importantly, the results indicated that post-task NED significantly mediated the relationship between group and one-week post-task distress among individuals demonstrating higher memory specificity (or task engagement) in the emotion condition, b = -.10, SE = .06, 95% CI [-.22, -.01]. No significant mediation effects were observed among those at mean or lower memory specificity.

thumbnail
Table 7. Moderated mediation results for conditional indirect effects (n = 100).

https://doi.org/10.1371/journal.pone.0299540.t007

Quiz performance as moderator. No significant findings emerged across all mediation models testing quiz error rate as a moderator, even after controlling for baseline distress and pre-task NED/ESE. Specifically, the indirect effect of group on distress (at one-week or two-month post-task) through post-task NED/ESE did not vary as a function of quiz error rate.

Discussion

The current study examined the effects of a novel brief online emotion word learning task on NED, ESE, and distress in a college sample. NED and ESE were examined as potential mediators of the relationship between word learning task condition and distress. Finally, we also explored whether the strength of these associations was influenced by participants’ level of engagement with the word learning task. Our key findings indicate that a relatively simple and straightforward emotion language learning task can influence short and long term outcomes through increases in emotional self-efficacy and shifts in NED, when participants are well-engaged. These findings have clear implications for the development of novel interventions targeting underlying dimensions of disrupted transdiagnostic emotional processes like NED and ESE.

Overall, our findings provide partial support for the efficacy of the emotion word learning task on primary outcomes of interest. While we did not observe significant group differences in NED, ESE, and distress post-task (and at two months), however, post-hoc tests of equivalence suggest that these null findings were not statistically equivalent to zero either. Taken together, these results indicate that the efficacy of our task remains preliminary and would benefit from future replication, preferably in larger well-powered samples. Despite the lack of significant group differences in outcomes, we did observe significant indirect effects of post-task ESE on distress at two months follow-up, where increases in ESE among participants in the emotion word learning condition were related to lower levels of distress. This suggests that individuals in the experimental condition who experienced gains in ESE had some long-term psychological benefits despite the brief nature of the word learning task. Finally, we found indirect effects of post-task NED on distress at one-week post-task that depended on the extent with which participants engaged in the word learning task. Specifically, the emotion word learning task predicted increases in post-task NED among those who demonstrated higher task engagement (as indicated by greater specificity of emotional memory recalled), which in turn predicted lower levels of distress at one-week post-task. No other significant findings were observed.

Together these preliminary results suggest the possibility that a relatively simple emotion word learning task could influence how individuals attend to negative emotional experiences in daily life. Although there was no evidence of direct improvement in NED or ESE, there were some evidence of indirect short-term and long-term effects on psychological health that was mediated by increases in NED (depending on level of task engagement) and ESE respectively. These preliminary findings provide an exciting starting point for further research given the simplicity of the intervention and the potential for clinically meaningful impact on common treatment targets.

Emotion differentiation as a mediator

Critically, our results also demonstrate the importance of task engagement. Specifically, we found some evidence suggesting that the intervention was able to indirectly impact short-term distress/psychological health through post-task NED. However, this mediation was dependent on level of engagement in the intervention. Indeed, when individuals were more able to apply emotion knowledge to generate a specific personal memory, they were more likely to demonstrate improved differentiation abilities that in turn predicted lower distress one-week later. That NED emerged as a significant mediator between the emotion word learning task and post-task distress (among highly engaged participants) is in line with prior intervention research. For instance, Van der Gucht et al. [34] found that participants demonstrated improvements in NED following an 8-week mindfulness training while Vedernikova et al. [33] observed similar effects from a 5-day emotion knowledge intervention. Taken together, these findings suggest that exposure to interventions that cultivate emotional awareness and emotion word learning may be a potentially meaningful way to positively influence the relationship between ED and psychological health. Notably, our online word learning intervention was brief and, despite the demands, had excellent compliance and completion rates from participants. Therefore, it may be a promising basis for the development of a future low-cost and scalable transdiagnostic treatment to consider.

The finding that participants’ scores on the emotion knowledge multiple choice quiz did not impact differentiation abilities is largely in line with both learning science [68] and contemporary treatments such as CBT and DBT which focuses on not just skills/knowledge acquisition but also application and/or generalization of these skills/knowledge across contexts [35,84]. Moreover, this finding suggests that perhaps more attention should be devoted to scaffolding patients’ ability to apply emotion concepts to personally relevant experiences in treatment to maximize benefits. For example, the current intervention might have been enhanced if participants were asked to generate another personal memory the next day. Finally, that task engagement was important also suggests an important avenue to explore in future treatment research and development.

Emotional self-efficacy as mediator

Unlike NED, we found that higher ESE was significantly associated with lower levels of distress not only concurrently but also two months post-task. Importantly, ESE was found to significantly mediate the relationship between task condition and distress at two-month follow-up, with higher ESE among those in the emotion word learning condition demonstrating prospective reductions in distress. These findings bring forth the interesting possibility that feelings of competency around emotions, perhaps more so than specific emotion skills, may be important in predicting psychological health and related outcomes. Indeed, these findings are in line with decades of research on self-efficacy, which found that individuals’ perceived competency was predictive of a wide range of behavioral and clinical outcomes, above and beyond that of their actual abilities [54,85,86]. Given its benefits, future research should continue to explore additional ways to enhance ESE, as well as how ESE may impact other related constructs important for psychological health.

That ESE was associated with long-term psychological benefits in the current study also suggests that it may be an important and relatively simple target to consider in treatment development. This may be particularly relevant among patients experiencing high intensity negative emotions and difficulties with emotion regulation, both of which feature prominently across numerous psychological disorders [1]. Already, treatments such as DBT have included skills focused on increasing individuals’ ability to manage their emotions by building a sense of mastery/self-efficacy [35]. Moreover, interventions such as exposure therapy for anxiety-related disorders that focus on systematically increasing patients’ ability to tolerate feared stimuli have also been shown to influence levels of self-efficacy [87,88]. Indeed, rigorous experimental investigations have demonstrated that instructing individuals to label their affect during exposure can dampen physiological arousal, as well as increase willingness to approach feared stimuli at follow-up despite no apparent reductions in self-reported fear [31,89]. Together, these findings suggest that attention to changes in ESE could be a useful target in treatment research, one that can be explicitly and easily measured and monitored in relation to treatment progress in the clinic.

Strengths and limitations

In reviewing the results of the current study, it is important to do so in the context of its strengths and limitations. Notable strengths include the explicit focus on an underlying dimension of emotion processing (rather than a symptom specific to a disorder) assessed using robust methods including intensive sampling of emotions experienced in daily life via semi-random experience sampling. This is considered the gold standard of mood measurement [90] as it allows for assessment during the course of everyday life (vs. in the laboratory) and is minimally influenced by memory and other affective distortion biases [9193]. An additional strength includes the effort to isolate the effects of the task by including an active, well-matched, control task and focusing on explicit investigation of alternate processes, including both the development of perceived mastery via measurement of emotional self-efficacy and the assessment of engagement with the intervention itself.

Weaknesses include a relatively limited retrieval activity for the active intervention. The learning literature suggests that multiple opportunities for retrieval of new content [72] along with delayed feedback [94] can both enhance memory and recall. A slightly longer task may be able to make effective use of these techniques to generate more durable prospective changes in NED, ESE, and symptoms. However, our focus here was on the practicality of a brief, simple, and remotely-administered intervention. Undoubtedly, the duration and intensity of retrieval could be extended and tested in replication to maximize any benefits. In addition, unlike the multiple-choice quiz, we did not assess pre-post changes in memory recall performance. Therefore, it is unclear whether improvements in NED were driven by task performance or that participant-level factors (e.g., conscientiousness) contributed to these changes. This will be important to delineate in future research by including additional pre-post tests of task engagement. In addition, our findings show that while more NED and ESE appear to be associated with less distress, task condition accounted for a relatively small amount of variance in our models. This suggests that emotion word learning is one piece of the puzzle when it comes to enhancing emotion vocabulary and competency, and that exploration of other techniques to increase these important emotion skills is warranted. Finally, past research utilizing experience sampling has found that just the act of monitoring one’s emotions over time may enhance awareness and improve differentiation abilities [62,95]. Combined with the relatively brief nature of our intervention and limited sample size, this may explain the lack of direct effects on primary outcomes of interest. Indeed, emotion conceptualization processes are largely formed over years during childhood [96,97] and a brief intervention may not be sufficient for lasting change in some adults.

Clinical implications and future directions

Despite limitations, our preliminary findings have important implications for treatment development. Notably, we observed indirect effects of NED and ESE on psychological health. These findings are largely consistent with existing research on NED and ESE, and suggest that both NED and ESE may be important and relatively straightforward psychological mechanisms to consider in treatment. While facilitating emotion language use and mastery are common across psychotherapies, this is done with varied intention [10]. Our findings suggest that more explicit focus on these affective processes in treatment may be helpful in improving psychological outcomes, which is consistent with prior research demonstrating how affect labeling may enhance effectiveness of exposure therapy [98]. Indeed, the present study utilized a dismantling approach that focused on isolating the effects of emotion word learning on NED, ESE, and psychological health. Although preliminary, these results provide a starting point for informing the development of novel interventions further aimed at addressing key transdiagnostic emotional processes.

In the future, our task would benefit from replication in larger and more diverse samples beyond college students, including patient populations where we would expect greater variability in affective processes and psychological symptoms/distress. In doing so, focus should be placed on further elucidating the mechanism(s) by which emotion labeling comes to be beneficial and relevant individual factors that determine for who and when it is appropriate to offer such a task through rigorous experimental work.

Conclusions

The current study examined the effectiveness of a brief online emotion word learning task to determine whether it impacted key processes linked to emotion-related disorders, namely ED and ESE, and consequently tested if reduced psychological distress in the short and long-term. The results suggest that with good engagement in the training, emotion word learning may be one way to enhance important potential transdiagnostic processes such as ED and ESE and reduce distress in the short-term and long-term respectively. Moreover, our findings suggest that this task can be effectively delivered remotely without the aid of a trained mental health provider. Future work should seek to replicate and better understand the mechanisms underlying emotion labeling, its relationship with ED and ESE, and consider the further development of emotion word learning interventions as potential adjuvant tools to be implemented in clinical settings.

References

  1. 1. Kring AM. Emotion disturbances as transdiagnostic processes in psychopathology. Handbook of emotions, 3rd ed [Internet]. 2008;3:691–705. Available from: https://psycnet.apa.org/fulltext/2008-07784-043.pdf.
  2. 2. Kranzler A, Young JF, Hankin BL, Abela JRZ, Elias MJ, Selby EA. Emotional awareness: A transdiagnostic predictor of depression and anxiety for children and adolescents. J Clin Child Adolesc Psychol [Internet]. 2016;45(3):262–9. Available from: pmid:25658297
  3. 3. Vine V, Aldao A. Impaired emotional clarity and psychopathology: A transdiagnostic deficit with symptom-specific pathways through emotion regulation. J Soc Clin Psychol [Internet]. 2014;33(4):319–42. Available from: http://dx.doi.org/10.1521/jscp.2014.33.4.319.
  4. 4. Bagby M, Taylor G. Affect dysregulation and alexithymia. In: Disorders of Affect Regulation. Cambridge: Cambridge University Press; 1997. p. 26–45.
  5. 5. Barlow DH, Allen LB, Choate ML. Toward a unified treatment for emotional disorders. Behav Ther [Internet]. 2004;35(2):205–30. Available from: http://dx.doi.org/10.1016/s0005-7894(04)80036-4.
  6. 6. Mennin DS, Heimberg RG, Turk CL, Fresco DM. Preliminary evidence for an emotion dysregulation model of generalized anxiety disorder. Behav Res Ther [Internet]. 2005;43(10):1281–310. Available from: pmid:16086981
  7. 7. Hayes SC, Wilson KG, Gifford EV, Follette VM, Strosahl K. Experiential avoidance and behavioral disorders: A functional dimensional approach to diagnosis and treatment. J Consult Clin Psychol [Internet]. 1996;64(6):1152–68. Available from: http://dx.doi.org/10.1037/0022-006x.64.6.1152.
  8. 8. Barrett LF, Gross J, Christensen TC, Benvenuto M. Knowing what you’re feeling and knowing what to do about it: Mapping the relation between emotion differentiation and emotion regulation. Cogn Emot [Internet]. 2001;15(6):713–24. Available from: http://dx.doi.org/10.1080/02699930143000239.
  9. 9. Kashdan TB, Barrett LF, McKnight PE. Unpacking emotion differentiation: Transforming unpleasant experience by perceiving distinctions in negativity. Curr Dir Psychol Sci [Internet]. 2015;24(1):10–6. Available from: http://dx.doi.org/10.1177/0963721414550708.
  10. 10. Greenberg LS. Integrating an emotion-focused approach to treatment into psychotherapy integration. J Psychother Integr [Internet]. 2002;12(2):154–89. Available from: http://dx.doi.org/10.1037/1053-0479.12.2.154.
  11. 11. Seah THS, Coifman KG. Emotion differentiation and behavioral dysregulation in clinical and nonclinical samples: A meta-analysis. Emotion [Internet]. 2022;22(7):1686–97. Available from: pmid:34264705
  12. 12. Kalokerinos EK, Erbas Y, Ceulemans E, Kuppens P. Differentiate to regulate: Low negative emotion differentiation is associated with ineffective use but not selection of emotion-regulation strategies. Psychol Sci [Internet]. 2019;30(6):863–79. Available from: pmid:30990768
  13. 13. Hill CLM, Updegraff JA. Mindfulness and its relationship to emotional regulation. Emotion [Internet]. 2012;12(1):81–90. Available from: pmid:22148996
  14. 14. Erbas Y, Sels L, Ceulemans E, Kuppens P. Feeling me, feeling you: The relation between emotion differentiation and empathic accuracy. Soc Psychol Personal Sci [Internet]. 2016;7(3):240–7. Available from: http://dx.doi.org/10.1177/1948550616633504.
  15. 15. Kimhy D, Vakhrusheva J, Khan S, Chang RW, Hansen MC, Ballon JS, et al. Emotional granularity and social functioning in individuals with schizophrenia: an experience sampling study. J Psychiatr Res [Internet]. 2014;53:141–8. Available from: pmid:24561000
  16. 16. Demiralp E, Thompson RJ, Mata J, Jaeggi SM, Buschkuehl M, Barrett LF, et al. Feeling blue or turquoise? Emotional differentiation in major depressive disorder. Psychol Sci [Internet]. 2012;23(11):1410–6. Available from: pmid:23070307
  17. 17. Suvak MK, Litz BT, Sloan DM, Zanarini MC, Barrett LF, Hofmann SG. Emotional granularity and borderline personality disorder. J Abnorm Psychol [Internet]. 2011;120(2):414–26. Available from: pmid:21171723
  18. 18. Erbas Y, Ceulemans E, Boonen J, Noens I, Kuppens P. Emotion differentiation in autism spectrum disorder. Res Autism Spectr Disord [Internet]. 2013;7(10):1221–7. Available from: http://dx.doi.org/10.1016/j.rasd.2013.07.007.
  19. 19. Lennarz HK, Lichtwarck-Aschoff A, Timmerman ME, Granic I. Emotion differentiation and its relation with emotional well-being in adolescents. Cogn Emot [Internet]. 2018;32(3):651–7. Available from: pmid:28602148
  20. 20. Johnson BC, Kiviniemi MT. The effect of online chapter quizzes on exam performance in an undergraduate social psychology course. Teach Psychol [Internet]. 2009;36(1):33–7. Available from: pmid:20046908
  21. 21. Brown BA, Goodman FR, Disabato DJ, Kashdan TB, Armeli S, Tennen H. Does negative emotion differentiation influence how people choose to regulate their distress after stressful events? A four-year daily diary study. Emotion [Internet]. 2021;21(5):1000–12. Available from: pmid:33829837
  22. 22. Zaki LF, Coifman KG, Rafaeli E, Berenson KR, Downey G. Emotion differentiation as a protective factor against nonsuicidal self-injury in borderline personality disorder. Behav Ther [Internet]. 2013;44(3):529–40. Available from: pmid:23768678
  23. 23. Seah THS, Aurora P, Coifman KG. Emotion differentiation as a protective factor against the behavioral consequences of rumination: A conceptual replication and extension in the context of social anxiety. Behav Ther [Internet]. 2020;51(1):135–48. Available from: pmid:32005331
  24. 24. Kashdan TB, Ferssizidis P, Collins RL, Muraven M. Emotion differentiation as resilience against excessive alcohol use: an ecological momentary assessment in underage social drinkers: An ecological momentary assessment in underage social drinkers. Psychol Sci [Internet]. 2010;21(9):1341–7. Available from: http://dx.doi.org/10.1177/0956797610379863.
  25. 25. Edwards ER, Wupperman P. Emotion regulation mediates effects of alexithymia and emotion differentiation on impulsive aggressive behavior. Deviant Behav [Internet]. 2017;38(10):1160–71. Available from: http://dx.doi.org/10.1080/01639625.2016.1241066.
  26. 26. Dixon-Gordon KL, Chapman AL, Weiss NH, Rosenthal MZ. A preliminary examination of the role of emotion differentiation in the relationship between borderline personality and urges for maladaptive behaviors. J Psychopathol Behav Assess [Internet]. 2014;36(4):616–25. Available from: pmid:25750478
  27. 27. Seah THS, Sidney PG, Taber JM, Thompson CA, Coifman KG. Emotional complexity under high stress: Do protective associations for risk behaviors persist even during a pandemic? Emotion [Internet]. 2023;23(3):879–85. Available from: pmid:35939604
  28. 28. Coifman KG, Ross GS, Kleinert D, Giardina P. Negative affect differentiation and adherence during treatment for thalassemia. Int J Behav Med [Internet]. 2014;21(1):160–8. Available from: pmid:23086716
  29. 29. Seah THS, Almahmoud S, Coifman KG. Feel to heal: Negative emotion differentiation promotes medication adherence in multiple Sclerosis. Front Psychol [Internet]. 2021;12:687497. Available from: pmid:35082708
  30. 30. Barrett LF. The conceptual act theory: A précis. Emot Rev [Internet]. 2014;6(4):292–7. Available from: http://dx.doi.org/10.1177/1754073914534479.
  31. 31. Kircanski K, Lieberman MD, Craske MG. Feelings into words: Contributions of language to exposure therapy. Psychol Sci [Internet]. 2012;23(10):1086–91. Available from: pmid:22902568
  32. 32. Brooks JA, Shablack H, Gendron M, Satpute AB, Parrish MH, Lindquist KA. The role of language in the experience and perception of emotion: a neuroimaging meta-analysis. Soc Cogn Affect Neurosci [Internet]. 2016;nsw121. Available from: http://dx.doi.org/10.1093/scan/nsw121.
  33. 33. Vedernikova E, Kuppens P, Erbas Y. From knowledge to differentiation: Increasing emotion knowledge through an intervention increases negative emotion differentiation. Front Psychol [Internet]. 2021;12:703757. Available from: pmid:34899459
  34. 34. Van der Gucht K, Dejonckheere E, Erbas Y, Takano K, Vandemoortele M, Maex E, et al. An experience sampling study examining the potential impact of a mindfulness-based intervention on emotion differentiation. Emotion [Internet]. 2019;19(1):123–31. Available from: pmid:29578747
  35. 35. Linehan MM. Cognitive-behavioral treatment of borderline personality disorder. New York, NY: Guilford Publications; 1993.
  36. 36. Linehan MM. DBT (R) Skills Training Manual, Second Edition. 2nd ed. New York, NY: Guilford Publications; 2014.
  37. 37. Kendall PC, Hedtke KA. Coping cat workbook. 2nd ed. Workbook Publishing; 2006.
  38. 38. Panos PT, Jackson JW, Hasan O, Panos A. Meta-analysis and systematic review assessing the efficacy of dialectical behavior therapy (DBT). Res Soc Work Pract [Internet]. 2014;24(2):213–23. Available from: pmid:30853773
  39. 39. Keehn RH, Lincoln AJ, Brown MZ, Chavira DA. The Coping Cat program for children with anxiety and autism spectrum disorder: a pilot randomized controlled trial. J Autism Dev Disord [Internet]. 2013;43(1):57–67. Available from: pmid:22588377
  40. 40. Greenberg LS, Elliott RK, Lietaer G. Research on experiential psychotherapies. In: Bergin S AEL, editor. Handbook of psychotherapy and behavior change. John Wiley & Sons; 1994. p. 509–39.
  41. 41. Houck CD, Barker D, Rizzo C, Hancock E, Norton A, Brown LK. Sexting and sexual behavior in at-risk adolescents. Pediatrics [Internet]. 2014;133(2):e276–82. Available from: pmid:24394678
  42. 42. Izard CE, Trentacosta CJ, King KA, Mostow AJ. An emotion-based prevention program for head start children. Early Educ Dev [Internet]. 2004;15(4):407–22. Available from: http://dx.doi.org/10.1207/s15566935eed1504_4.
  43. 43. Spivack G, Shure M. Interpersonal cognitive problem solving (ICPS): A competence-building primary prevention program. J Prev Interv Community [Internet]. 1989;6(2):151–78. Available from: http://dx.doi.org/10.1080/10852358909511187.
  44. 44. Greenberg MT, Kuschz C. Promoting social and emotional development in deaf children: The PATHS project. Oxford, England: Premier Book Marketing; 1993.
  45. 45. Second Step: A violence-prevention curriculum. Grades 1–3, second edition. Teacher’s guide. 1989 [cited 2023 May 10]; Available from: http://files.eric.ed.gov/fulltext/ED365740.pdf.
  46. 46. Greenberg MT, Kusche CA, Cook ET, Quamma JP. Promoting emotional competence in school-aged children: The effects of the PATHS curriculum. Dev Psychopathol [Internet]. 1995;7(1):117–36. Available from: http://dx.doi.org/10.1017/s0954579400006374.
  47. 47. Greenberg P, Kusche C. Preventive intervention for school-age deaf children: the PATHS curriculum. J Deaf Stud Deaf Educ [Internet]. 1998 Winter;3(1):49–63. Available from: pmid:15579857
  48. 48. Domitrovich CE, Cortes RC, Greenberg MT. Improving young children’s social and emotional competence: a randomized trial of the preschool “PATHS” curriculum. J Prim Prev [Internet]. 2007;28(2):67–91. Available from: pmid:17265130
  49. 49. Cipriano C, Barnes TN, Rivers SE, Brackett M. Exploring changes in student engagement through the RULER approach: An examination of students at risk of academic failure. J Educ Stud Placed Risk [Internet]. 2019;24(1):1–19. Available from: http://dx.doi.org/10.1080/10824669.2018.1524767.
  50. 50. Hagelskamp C, Brackett MA, Rivers SE, Salovey P. Improving classroom quality with the RULER Approach to Social and Emotional Learning: proximal and distal outcomes. Am J Community Psychol [Internet]. 2013;51(3–4):530–43. Available from: pmid:23444004
  51. 51. Lindquist KA, Gendron M, Satpute AB. Language and Emotion: Putting Words into Feelings and Feelings into Words. In: Barrett LF, Lewis M, Haviland-Jones JM, editors. Handbook of Emotions. 4th ed. New York, NY: Guilford Publications; 2016. p. 579–94.
  52. 52. Lieberman MD. Why symbolic processing of affect can disrupt negative affect. In: Social Neuroscience. Oxford University Press; 2011. p. 188–209.
  53. 53. Kirk BA, Schutte NS, Hine DW. Development and preliminary validation of an emotional self-efficacy scale. Pers Individ Dif [Internet]. 2008;45(5):432–6. Available from: http://dx.doi.org/10.1016/j.paid.2008.06.010.
  54. 54. Bandura A. Self-efficacy: Toward a unifying theory of behavioral change. Psychol Rev [Internet].1977;84(2):191–215. Available from: pmid:847061
  55. 55. Williams SL. Self-efficacy, anxiety, and phobic disorders. In: Self-Efficacy, Adaptation, and Adjustment. Boston, MA: Springer US; 1995. p. 69–107.
  56. 56. Maddux JE, Meier LJ. Self-Efficacy and Depression. In: Self-Efficacy, Adaptation, and Adjustment. Boston, MA: Springer US; 1995. p. 143–69.
  57. 57. Maddux JE. Self-Efficacy Theory. In: Self-Efficacy, Adaptation, and Adjustment. Boston, MA: Springer US; 1995. p. 3–33.
  58. 58. Bandura A. The explanatory and predictive scope of self-efficacy theory. J Soc Clin Psychol [Internet]. 1986;4(3):359–73. Available from: http://dx.doi.org/10.1521/jscp.1986.4.3.359.
  59. 59. Locke A, Latham G. A theory of goal setting and task performance. London, England: Prentice-Hall; 1989.
  60. 60. Paunonen SV, Hong RY. Self-efficacy and the prediction of domain-specific cognitive abilities. J Pers [Internet]. 2010;78(1):339–60. Available from: pmid:20433622
  61. 61. Hoemann K, Barrett LF, Quigley KS. Emotional granularity increases with intensive ambulatory assessment: Methodological and individual factors influence how much. Front Psychol [Internet]. 2021;12:704125. Available from: pmid:34393942
  62. 62. Widdershoven RLA, Wichers M, Kuppens P, Hartmann JA, Menne-Lothmann C, Simons CJP, et al. Effect of self-monitoring through experience sampling on emotion differentiation in depression. J Affect Disord [Internet]. 2019;244:71–7. Available from: pmid:30321767
  63. 63. Holladay CL, Quiñones MA. Practice variability and transfer of training: the role of self-efficacy generality. J Appl Psychol [Internet]. 2003;88(6):1094–103. Available from: pmid:14640819
  64. 64. Zhang Y, Hedo R, Rivera A, Rull R, Richardson S, Tu XM. Post hoc power analysis: is it an informative and meaningful analysis? Gen Psychiatr [Internet]. 2019;32(4):e100069. Available from: pmid:31552383
  65. 65. Lakens D, McLatchie N, Isager PM, Scheel AM, Dienes Z. Improving inferences about null effects with Bayes factors and equivalence tests. J Gerontol B Psychol Sci Soc Sci [Internet]. 2020;75(1):45–57. Available from: pmid:29878211
  66. 66. Ekman P. An argument for basic emotions. Cogn Emot [Internet]. 1992;6(3–4):169–200. Available from: http://dx.doi.org/10.1080/02699939208411068.
  67. 67. Rogers TB, Kuiper NA, Kirker WS. Self-reference and the encoding of personal information. J Pers Soc Psychol [Internet]. 1977;35(9):677–88. Available from: pmid:909043
  68. 68. Symons CS, Johnson BT. The self-reference effect in memory: A meta-analysis. Psychol Bull [Internet]. 1997;121(3):371–94. Available from: pmid:9136641
  69. 69. Batsell WR Jr, Perry JL, Hanley E, Hostetter AB. Ecological validity of the testing effect: The use of daily quizzes in Introductory Psychology. Teach Psychol [Internet]. 2017;44(1):18–23. Available from: http://dx.doi.org/10.1177/0098628316677492.
  70. 70. Roediger HL 3rd, Marsh EJ. The positive and negative consequences of multiple-choice testing. J Exp Psychol Learn Mem Cogn [Internet]. 2005;31(5):1155–9. Available from: pmid:16248758
  71. 71. Ellis R, Loewen S, Erlam R. Implicit and explicit corrective feedback and the acquisition of l2 grammar. Stud Second Lang Acquis [Internet]. 2006;28(02). Available from: http://dx.doi.org/10.1017/s0272263106060141.
  72. 72. Karpicke JD, Roediger HL 3rd. The critical importance of retrieval for learning. Science [Internet]. 2008;319(5865):966–8. Available from: pmid:18276894
  73. 73. Nelson GH, O’Hara MW, Watson D. National norms for the expanded version of the inventory of depression and anxiety symptoms (IDAS-II). J Clin Psychol [Internet]. 2018;74(6):953–68. Available from: pmid:29327338
  74. 74. Derogatis LR, Melisaratos N. The Brief Symptom Inventory: an introductory report. Psychol Med [Internet]. 1983;13(3):595–605. Available from: http://dx.doi.org/10.1017/s0033291700048017. pmid:6622612
  75. 75. Almahmoud SY, Coifman KG, Ross GS, Kleinert D, Giardina P. Evidence for multidimensional resilience in adult patients with transfusion-dependent thalassemias: Is it more common than we think?: Multidimensional resilience in thalassemia. Transfus Med [Internet]. 2016;26(3):186–94. Available from: http://dx.doi.org/10.1111/tme.12296.
  76. 76. Coifman KG, Flynn JJ, Pinto LA. When context matters: Negative emotions predict psychological health and adjustment. Motiv Emot [Internet]. 2016;40(4):602–24. Available from: http://dx.doi.org/10.1007/s11031-016-9553-y.
  77. 77. Hennings JM, Owashi T, Binder EB, Horstmann S, Menke A, Kloiber S, et al. Clinical characteristics and treatment outcome in a representative sample of depressed inpatients—findings from the Munich Antidepressant Response Signature (MARS) project. J Psychiatr Res [Internet]. 2009;43(3):215–29. Available from: pmid:18586274
  78. 78. Mayer JD, Salovey P, Caruso DR. Emotional intelligence: Theory, findings, and implications. Psychol Inq [Internet]. 2004;15(3):197–215. Available from: http://dx.doi.org/10.1207/s15327965pli1503_02.
  79. 79. Baddeley A, Emslie H, Nimmo-Smith I. The Spot-the-Word test: A robust estimate of verbal intelligence based on lexical decision. Br J Clin Psychol [Internet]. 1993;32(1):55–65. Available from: pmid:8467274
  80. 80. Morris ME, Aguilera A. Mobile, social, and wearable computing and the evolution of psychological practice. Prof Psychol Res Pr [Internet]. 2012;43(6):622–6. Available from: pmid:25587207
  81. 81. Rafaeli E, Rogers GM, Revelle W. Affective synchrony: individual differences in mixed emotions. Pers Soc Psychol Bull [Internet]. 2007;33(7):915–32. Available from: pmid:17551163
  82. 82. Bolger N, Davis A, Rafaeli E. Diary methods: capturing life as it is lived. Annu Rev Psychol [Internet]. 2003;54(1):579–616. Available from: pmid:12499517
  83. 83. Hayes AF. Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York, NY: Guilford Publications; 2014.
  84. 84. Beck AT, Rush AJ, Shaw BF, Emery G. Cognitive therapy of depression. New York, NY: Guilford Publications; 1987.
  85. 85. Caprara GV, Fida R, Vecchione M, Del Bove G, Vecchio GM, Barbaranelli C, et al. Longitudinal analysis of the role of perceived self-efficacy for self-regulated learning in academic continuance and achievement. J Educ Psychol [Internet]. 2008;100(3):525–34. Available from: http://dx.doi.org/10.1037/0022-0663.100.3.525.
  86. 86. Gaudiano BA, Herbert JD. Self-Efficacy for Social Situations in Adolescents with Generalized Social Anxiety Disorder. Behavioural and Cognitive Psychotherapy. Cambridge University Press; 2007;35(2):209–23. Available from: https://doi.org/10.1017/S1352465806003377.
  87. 87. Breuninger C, Tuschen-Caffier B, Svaldi J. Dysfunctional cognition and self-efficacy as mediators of symptom change in exposure therapy for agoraphobia—Systematic review and meta-analysis. Behav Res Ther [Internet]. 2019;120(103443):103443. Available from: pmid:31374484
  88. 88. Voderholzer U, Hilbert S, Fischer A, Neumüller J, Schwartz C, Hessler-Kaufmann JB. Frequency and level of self-efficacy predict the effectiveness of therapist- and self-guided exposure in obsessive compulsive disorder. Behav Cogn Psychother [Internet]. 2020 [cited 2023 May 26];48(6):751–5. Available from: pmid:32814601
  89. 89. Niles AN, Craske MG, Lieberman MD, Hur C. Affect labeling enhances exposure effectiveness for public speaking anxiety. Behav Res Ther [Internet]. 2015;68:27–36. Available from: pmid:25795524
  90. 90. Fritzson A, Lieberman M, Tugade MM. Mood Assessment. In: Llewellyn CD, Ayers S, McManus C, Newman S, Petrie KJ, Revenson TA, Weinman J, editors. Cambridge Handbook of Psychology, Health and Medicine. 3rd ed. Cambridge: Cambridge University Press; 2019. p. 200–4. (Cambridge Handbooks in Psychology). Available from: https://doi.org/10.1017/9781316783269.
  91. 91. Robinson MD, Clore GL. Belief and feeling: Evidence for an accessibility model of emotional self-report. Psychol Bull [Internet]. 2002;128(6):934–60. Available from: pmid:12405138
  92. 92. Singer JA, Salovey P. Mood and memory: Evaluating the network theory of affect. Clin Psychol Rev [Internet]. 1988;8(2):211–51. Available from: http://dx.doi.org/10.1016/0272-7358(88)90060-8.
  93. 93. Derakhshan A, Wang Y, Wang Y, Ortega-Martín J. Towards Innovative Research Approaches to Investigating the Role of Emotional Variables in Promoting Language Teachers’ and Learners’ Mental Health. International Journal of Mental Health Promotion [Internet]. 2023 [cited 2023 Jul 4];25(7):823–32. Available from: https://www.techscience.com/IJMHP/v25n7/52898.
  94. 94. Butler AC, Karpicke JD, Roediger HL 3rd. The effect of type and timing of feedback on learning from multiple-choice tests. J Exp Psychol Appl [Internet]. 2007;13(4):273–81. Available from: pmid:18194050
  95. 95. Runyan JD, Steenbergh TA, Bainbridge C, Daugherty DA, Oke L, Fry BN. A smartphone ecological momentary assessment/intervention “app” for collecting real-time data and promoting self-awareness. PLoS One [Internet]. 2013;8(8):e71325. Available from: pmid:23977016
  96. 96. Denham SA. Social and emotional learning, early childhood. In: Encyclopedia of Primary Prevention and Health Promotion. Boston, MA: Springer US; 2003. p. 1009–18.
  97. 97. Nook EC, Somerville LH. Emotion concept development from childhood to adulthood. In: Nebraska Symposium on Motivation. Cham: Springer International Publishing; 2019. p. 11–41.
  98. 98. Craske MG, Treanor M, Conway CC, Zbozinek T, Vervliet B. Maximizing exposure therapy: an inhibitory learning approach. Behav Res Ther [Internet]. 2014;58:10–23. Available from: pmid:24864005