Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Couples extrinsic emotion regulation questionnaire: Psychometric validation in a Chilean population

  • Ana Kinkead,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Project administration, Resources, Writing – original draft, Writing – review & editing

    Affiliations Directorate of Research, Vice-rector for Research and Doctorates, Universidad Autónoma de Chile, Santiago, Metropolitan Region, Chile, Doctorate in Psychology, Faculty of Psychology, Universidad Diego Portales, Santiago, Metropolitan Region, Chile

  • Susana Sanduvete-Chaves ,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Methodology, Resources, Software, Validation, Visualization, Writing – original draft, Writing – review & editing

    schacon@us.es, salvador.chacon@uautonoma.cl (SCM); sussancha@us.es (SSC)

    Affiliation Departamento de Psicología Experimental, Facultad de Psicología, Universidad de Sevilla, Seville, Spain

  • Salvador Chacón-Moscoso ,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – review & editing

    schacon@us.es, salvador.chacon@uautonoma.cl (SCM); sussancha@us.es (SSC)

    Affiliations Departamento de Psicología Experimental, Facultad de Psicología, Universidad de Sevilla, Seville, Spain, Departamento de Psicología, Universidad Autónoma de Chile, Santiago, Chile

  • Christian E. Salas

    Roles Conceptualization, Project administration, Supervision, Writing – review & editing

    Affiliation Faculty of Psychology, Universidad Diego Portales, Santiago, Metropolitan Region, Chile

Abstract

The way couples regulate their emotions affects the quality of their relationship. Despite this, no evidence-based scales of validity and reliability can measure the intention to regulate emotions in the romantic dyad. In order to address this gap, we developed the Couples Extrinsic Emotion Regulation (CEER) questionnaire. First, we adapted the “Others” subscale from the Emotion Regulation of Others and Self questionnaire (EROS) for any close relationship to measure the intention to regulate emotions in couples; second, the psychometric properties of the CEER questionnaire were studied. For the content validity assessment, 23 experts (47.8% of whom worked in social and health psychology and the psychology of emotions, 17.4% in couples’ therapy, and 34.8% in social science methodologies) participated. A total of 528 Chileans completed the online CEER questionnaire, the relationship satisfaction scale (RAS) and dyadic adjustment scale (DAS): 27.8% were male, age M = 38.7, SD = 10.05, and time of the relationships M = 11.27, SD = 9.82. The content-based validity study made it possible to determine which items to include in the final version. Two unrelated first-order factors structure of the original test fit (RMSEA = .052, GFI = .97, AGFI = .95; CFI = .99; NFI = .98; and NNFI = .98). The CEER+ and CEER- factors presented adequate internal consistency (α = .79; ω = .80 and α = .85; ω = .85, respectively). The discrimination index of the factors were excellent (CEER+ = .55 and CEER- = .63). Validity evidence based on the relations to other variables showed a direct positive relation between CEER+, RAS and overall DAS, as well as their factors; and a negative relation between CEER-, RAS overall DAS, as well as their factors. The use of this instrument is recommended for the identification of Chilean couples where at least one of the partners has a less favorable opinion of their relationship, providing relevant data for couple’s therapy.

Introduction

Interpersonal emotion regulation (IER) refers to social interactions aimed at improving or worsening other people’s emotions [1]. It is based on the premise that people not only regulate their own emotions but also seek to influence, affect, or modify other people’s emotional experiences [2]. Niven et al. [3] refer to this as deliberate emotion regulation, the social regulation of affects through strategies such as deliberately attempting to influence the emotional states of others, thus improving, or worsening their affects. This is also known as extrinsic regulation [4]. Studies have shown a direct and significant association between a person’s intention to improve or worsen other people’s emotions and their own well-being or discomfort [5]. In addition, there are hedonic motives related to making others feel good, and the search for individual benefits to making them feel bad [6].

IER is essential to understanding the quality of close bonds such as partner relationships. According to the literature, the way a couple regulates their emotions may affect satisfaction and security with the relationship [7, 8]. In addition, issues associated with regulating emotions have always been associated with severe marital problems, more negative emotional expression, less positive expression, and risk of dissolution [9, 10]. Specifically, the evidence shows that when a person invalidates an expression of emotions by another person (by rejecting or judging them for it), this has negative emotional impacts such as emotional dysregulation [11]. Additional findings show a strong association between IER and positive/close relationships. It is also related to general well-being [12] and the dyadic adjustment, i.e., an adaptation process required for a couple to attain a functional and harmonious relationship as expressed in consensus, satisfaction, cohesion, and affectional expression [13].

Interaction paradigms are commonly used to study affective dynamics in couples. For instance, in video-recorded sessions, dyads discuss stressors, and their emotional responses are recorded [14]. Other approaches include the linguistic analysis of qualitative data such as that included in the Adult Attachment Interview, where trained researchers record the interviews and then code them, generating a conversational protocol that serves as the basis for further analysis [15]; or the use of life diaries [16]. Despite the methodological strengths of these modalities [17], other related phenomena such as extrinsic emotion regulation are not directly observable, and thus require other techniques such as questionnaires [18].

In response to this challenge, the Interpersonal Emotion Regulation Questionnaire (IERQ) was developed by Hofmann et al. [2] to assess the strategies people use to regulate their own emotions through others. Although the psychometric characteristics of this tool are adequate, it does not assess extrinsic regulation. Another instrument, the Brief Emotional Intelligence Scale (BEIS-10) [19], is a five-dimensional scale, also validated in an Italian population [20], which measures an individual’s willingness to explore his/her own emotions and the emotions of others. However, only two of the scale’s dimensions refer to the regulation and evaluation of the emotions of others. These dimensions focus on the individual’s ability to promote positive feelings in others, but not negative feelings or their intentionality, overlooking the fact that both types of emotions coexist in human relationships. Similarly, the Interpersonal Affect Improvement Strategies Questionnaire (IAISQ), developed by López-Pérez et al. [21], focuses on the intention to enhance other people’s emotions, but not on the desire to make the other person feel bad. It presented adequate construct validity evidence and acceptable reliability. Yet another questionnaire, the Intimate Partner Flooding Scale (IPFS), evaluates the effects that one person’s negative emotions have on their partner. Like the others, this instrument does not assess any deliberate intention to make the other person feel bad, although it presents reasonable unifactoriality, excellent internal consistency, and high test-retest reliability [22].

Conversely, the Managing the Emotions of Others Scale (MEOS) allows for a detailed description of the different prosocial and non-prosocial ways in which people manage the emotions of others; however, it does not focus on intimate partners or intentions [23, 24]. Another scale, the Difficulties in Interpersonal Regulation of Emotions (DIRE), designed by Dixon-Gordon et al. [25], relates regulatory difficulties with psychopathology; it does not measure the intention to regulate emotions, though it does present adequate internal coherence and provide evidence of construct and predictive validity. The one scale designed specifically for a Chilean population, the Perceived Emotional Synchrony Scale [26], but it targets people’s perception of synchrony after participating in group activities such as protests.

The Emotion Regulation of Others and Self (EROS) scale, created by Niven et al. [1], is an instrument for assessing individual differences in the strategies used to deliberately regulate emotions in a non-clinical population. It is formed by two subscales: the regulation of one’s own emotions (10 items) and the regulation of other people’s emotions (9 items). In turn, each subscale has two factors showing evidence of construct validity and internal consistency. The EROS was translated into Spanish and validated by Lozano et al. [27], who used it for a sample of elderly Spaniards. Just like the original scale, it had two factors for each subscale. The EROS comprises 24 items on a five-point Likert-type scale (from 1 = not at all, to 5 = a great deal), where 12 items assess the regulation of one’s own positive (ESR+) and negative (ESR-) emotions, and the other 12 assess the regulation of others’ positive (ERO+) and negative (ERO-) emotions. Table 1 presents the psychometric properties of instruments that assess interpersonal regulation of emotions, organized according to how closely each is related to extrinsic emotion regulation in couples.

thumbnail
Table 1. Instruments that evaluate constructs associated with interpersonal emotion regulation.

https://doi.org/10.1371/journal.pone.0252329.t001

In summary, to gauge extrinsic emotion regulation, scholars adapted and validated these instruments for close interpersonal relationships in general, but not specifically for couple relationships, and in countries other than Chile [27]. There are currently no instruments to assess extrinsic emotion regulation in non-pathological couples in any country. For these reasons, we considered fast, valid, and reliable self-reporting measures were necessary to assess the social interdependence used to configure and improve couple relationships [28, 29]. In this sense, the present study deepens the understanding of such interdependence and how it impacts a couple’s regulation of their emotional life [30, 31].

This article describes the adaptation of the “Others” subscale (positive and negative) from the Emotion Regulation of Others and Self (EROS) instrument for couples in Chile. Specifically, it examines validity evidence based on the adapted test content, its internal structure, and reliability. In addition, validity evidence is presented drawing on the association with other variables and criteria measures expected to predict the test’s outcome [32, 33]. Specifically, the extrinsic regulation of positive emotion is expected to be associated with increased satisfaction with one’s relationship and greater dyadic adjustment [8, 34]. Conversely, in the case of extrinsic regulation of negative emotion, the opposite outcome is expected.

Method

Participants

Based on the instrument’s content, forty Spanish speakers were chosen for the validity study. All who met the following inclusion criteria were contacted about assessing the CEER questionnaire: a bachelor’s degree or degree in psychology with experience in instrument design, and clinical experience in couples, or the field of emotion regulation. Twenty-three participants (57%) ultimately participated in the study. This percentage represented a moderate number of experts to assess a systematized procedure for a validity study based on test content (10 < N < 30) [35]. Twenty participants (87%) were Chilean nationals and three (23%) had been residents of Chile for at least five years. Their average age was M = 44 years old (SD = 7.09). Thirteen (56.5%) were men and 10 (43.5%) were women. In relation to professional experience, 11 (47.8%) worked in fields related to social and health psychology and the psychology of emotions, four (17.4%) in couples’ therapy, and eight (34.8%) in social science methodologies. In terms of academic degrees, 10 (43.5%) hold a Ph.D., 12 (52.2%) a master’s degree, and one (4.3%) a bachelor’s degree.

In accordance with the instrument’s internal structure, the reliability test, and validity evidence based on relations to other variables, a total of 528 people were chosen for the sample. All met the following inclusion criteria and participated in the study: (a) age ≥ 18 years, (b) were either Chilean citizens or residents (years ≥ 5), (c) were in a relationship of ≥ 6 months at the time of filling out the questionnaires, (d) had a high school degree. Participants ranged in age from 18 to 74, with an average M = 38.7 (SD = 10.05); the amount of time of their relationships ranged from one to 55 years, M = 11.27 (SD = 9.82); 147 (27.8%) were male and 381 (72.2%) were female; 503 (95.3%) declared to have a heterosexual, 14 (2.7%) homosexual, and 11 (2%) bisexual orientation; regarding educational attainment, 1 (0.2%) reported primary education, 21 (4%) high school, 426 (80.6%) technical training or university, and 80 (15.2%) graduate school; 392 (74.2%) lived with their partner, and 136 (25.8%) did not.

Instruments

The 12-item Spanish version of the Emotion Regulation of Others subscale (EROS) [27] was used to obtain validity evidence, based on the test content, and assessed by experts (available at https://forms.gle/bHneijX4mvB7rp7y5). These items, originally designed for any close relationship, were adapted to couple relationships, replacing the term “someone” with “my partner” in all cases. Furthermore, with the assistance of a language specialist, the survey was localized for its application in a Chilean sample, replacing some of the words used in Spain with their Chilean equivalents. Thus, a cultural and functional (rather than literal) equivalence was achieved [36, 37]. Specifically, “I have pointed out (he señalado)” was changed to “I said (hablé)”; the Spanish term for weak points, “puntos flacos” was changed to “puntos débiles,” which is more common in Chile (item 3); “I explained (expliqué)” was changed to “I expressed (expresé)” (item 4); “I discussed his/her positive characteristics (comenté sus características positivas)” was changed to “I spoke about his/her qualities (le hablé de sus cualidades)” (item 5); “unpleasant (antipático)” was changed to “rude (pesado)” (item 8), and “I reproached (recriminé)” was changed to “I complained (reclamé)” (item 10). Finally, item four was simplified by eliminating “me or others (a mi o a otros)” and “for having done so (por haberlo hecho).”

Two factors (made up of the same items from the original scale) were considered: (1) the intent of the Positive Couples Extrinsic Emotion Regulation, CEER+ (makes him/her feel good or better) with six items, and (2) the intent of Negative Couples Extrinsic Emotion Regulation, CEER- (makes him/her feel bad or worse) with six items. Each item was assessed as essential, adequate, or inadequate. A section was provided for recommendations on improving the scale.

The Couples Extrinsic Emotion Regulation (CEER) questionnaire is the instrument that resulted from the validity study based on the instrument’s content. It is comprised of the previously two mentioned subscales (CEER+ and CEER-), each with six items, on a five-point Likert scale (1 = never to 5 = always).

Hendrick’s Relationship Assessment Scale (RAS): This unifactor scale assesses a person’s overall satisfaction with the relationship, with a high predictive capacity for which relationships will last (91%) and which will end (86%) [38]. A self-administered survey, it is comprised of seven items that must be ranked on a five-point Likert scale (1 = low satisfaction, to 5 = high satisfaction). It showed excellent reliability indicators for the Chilean population (α = .92) [39].

Spanier’s Dyadic Adjustment Scale (DAS) [40]: This self-administered instrument is widely used to assess the quality and adjustment of couples [41]. With 32 items (one item on a seven-point Likert scale, 27 on a six-point Likert scale, two on a five-point scale, and two on a dichotomous scale), it yields an overall score along with a score for each of its four subscales (consensus, satisfaction, cohesion, and affectional expression). The Spanish version of the DAS had the same internal structure as the original scale [42], along with appropriate reliability (α = .80 and α = .92) [43, 44].

Procedure

The Research Ethics Committee at Diego Portales University (Chile) approved the study (approval number 012–2019). All participants provided written informed consent.

In order to perform the validity study based on the instrument’s content, a link to the questionnaire (Google Forms) was sent by email, after contacting most of the experts to explain the purpose of their involvement; the study was presented as voluntary and non-anonymous. Answers were obtained using a spreadsheet generated on Google Drive.

In order to perform the study of the validity evidence based on the instrument’s internal structure and relations to other variables, non-probabilistic sampling was used, and 4,380 individuals were recruited via email using two combined procedures: (a) incidental sampling: information taken from distribution lists accessed by the authors (from the university, professional training, the parents of pediatric patients, insurance companies, cultural and sports associations, and schools); and (b) snowball sampling: given that the Chilean population tends to be reluctant to talk about their private lives, some participants from the incidental sampling recommended other potential participants from their respective social networks. The final sample consisted of 528 participants, 12.1% of the total contacted, who anonymously responded to three online Google Form questionnaires (CEER, RAS, DAS). It should be noted that the digital questionnaire was configured so that answers for each item were mandatory in order to avoid missing data. Eight duplicates were eliminated. It is useful to note that some studies have supported the psychometric equivalence between the paper-and-pencil and online questionnaires used for intimate partner research [45].

Data analysis

Excel (Microsoft Corp., Chicago, IL, 2018), SPSS 23 (IBM Corp., Armonk, NY, 2015), PRELIS and LISREL 9.3 (Scientific Software International Inc., Lincolnwood, IL, 2017), JASP (JASP Team, Amsterdam, The Netherlands, 2020) and FACTOR (Ferrando & Lorenzo-Seva, Tarragona, Spain, 2017) were the software used for data recording and analysis.

The content validity calculation (CVC) was performed with Lawshe’s coefficient [46]: where n is the number of experts who deemed the item essential and N is the number of experts who evaluated the item. The CVC ranges from +1 to -1, with positive scores indicating adequate validity based on content. A minimum CVC of .42 for 20 experts (items with lower values would have been eliminated) is suggested [46].

In order to obtain validity evidence based on the internal structure of the instrument, Mardia’s multivariate skewness (S) and kurtosis (K) were tested [47] first, considering the items as continuous variables. When the significance of S or K was lower than .05, multivariate normality assumption was not met. In the case of an asymmetric distribution or excess kurtosis of ordinal items, the use of polychoric correlations was advised [48, 49]. The bivariate normality assumption needed to be fulfilled to use the polychoric correlation matrix [50]. This assumption was checked calculating Chi-square (X2) for each pair of correlations and the Bonferroni correction was used to calculate the number of times the null hypothesis was rejected. Thus, statistical significance was compared with the result obtained by applying the formula α/c, where α = .05, corresponded to a 95% confidence level and c was the number of contrasts (c = [number of items x number of items—1] / 2) [51]. Since X2 is very sensitive for large samples, it is often used to indicate a relative fit rather than an absolute fit. For this reason, the root mean square error of approximation (RMSEA) was also calculated [52]. When the RMSEA for each polychoric correlation was less than .1, the estimation of the parameter was not affected. Then, polychoric correlations matrix [53, 54] and the asymptotic variance-covariance matrix were estimated. The Diagonally Weighted Least Squares (DWLS) method was used for the estimates [54].

In order to assess data dimensionality, a parallel analysis based on minimum rank factor analysis was carried out [55]. Additionally, at the item-level, the overall assessment indexes were a) Unidimensional Congruence (UniCo), b) Explained Common Variance (ECV), and c) Mean of Item Residual Absolute Loadings (MIREAL) [56]. Data can be treated as essentially unidimensional when UniCo is above .95, ECV is above .85, and MIREAL is above .30.

According to the previous dimensionality analysis and the second objective of the study, a confirmatory factor analysis (CFA) was carried out to study the level of fitness of our data with the model in the original test [27]. It was a first-order structure with two unrelated factors, each with six items (see Fig 1).

thumbnail
Fig 1. Standardized factor loadings for the CEER questionnaire model of two unrelated first-order factors (CEER+ = positive Couples Extrinsic Emotion Regulation and CEER- = negative Couples Extrinsic Emotion Regulation).

https://doi.org/10.1371/journal.pone.0252329.g001

The model’s adjustment was assessed using different fit indexes [57]: (a) X2, accepting the null hypothesis as a good model fit (p ≥ .05); (b) the consistent Akaike information criterion (CAIC) stating that a good fit is when the index value was closer to the value of the saturated model instead of the independent one (low values) (c) the RMSEA, where values between .05 and .08 were considered a reasonable fit [58]; (d) the goodness-of-fit index (GFI) and adjusted GFI (AGFI); (e) the comparative fit index (CFI) [59], (f) the normalized fit index (NFI); and (g) the non-standardized NFI fit index (NNFI). Values above .9 were considered adequate for (d) to (g).

Once the factor structure of the model was tested, the internal consistency was ascertained using Cronbach’s α [60], George and Mallery’s criteria [61] and McDonald’s ω [62]. Values above .90 were considered excellent; between .80 - .90, good; and between .70 - .80, acceptable. The average discrimination index was assessed based on Linnette’s criteria [63], with values greater than .35 considered excellent.

The following item descriptive statistics and metric indexes by factor were presented in the CEER questionnaire: skewness (S), kurtosis (K), item discrimination (ID), Cronbach’s alpha if the item was deleted (α); McDonald’s omega if the item was deleted (ω); and reliability index (RI).

Before assessing validity based on the relation to other variables (satisfaction with the relationship and dyadic adjustment), normality assumptions were verified with the Kolmogorov-Smirnov test (normal distribution assumed p > .05); linearity was checked (met when p < .05); and error independence were verified with the Durbin-Watson test (values between 1.5 < d < 2.5 were considered adequate). Since not all assumptions were satisfactory, Spearman’s bivariate correlations (ρ) were calculated and interpreted following Cohen’s criteria [64] with values close to .1, .25, and .4 (absolute value) reflecting a low, moderate, and high effect magnitude, respectively. In order to estimate the true relationship between the test and the criterion variables, a structural equation model (SEM) was analyzed. X2 with p > .05 was interpreted as model fitness; and Student t was used to study the significance of each parameter (with a confidence level of 99%, values over 2.58 were considered significant).

Results

Validity study based on the instrument’s content

In relation to this study, raw data are available in S1 and S2 Datasets. In the first assessment (phase 1), of the 12 items, three (items 2, 4, and 5) met the inclusion criteria (see Table 2). Suggestions for improvement were collected for all items that failed to meet these criteria (four were assigned a negative value, and five, a value between 0 and .3).

All the suggested adjustments were made. Four people proposed changing the adjective “worse (peor)” to “bad (mal)” (items 4, 6, 8, 10, and 12), to reduce the callousness of the statement for Chileans, who are particularly sensitive to words with strong negative connotations. For the same reason, in item 3 “to make him/her feel worse (se sintiera peor)” was replaced by “to upset him/her (se molestara).” The remaining modifications were each suggested by one person. “To do (para hacer),” “to (para),” “to try (para intentar),” and “to achieve (para conseguir)” were replaced by “in an attempt to (con la intención de)” in items where this phrase was not originally included (items 3, 4, 5, 6, 7, 8, and 10). The purpose of these changes was to emphasize the intention behind the action and respond to existing semantic and cultural nuances between Chile and Spain. The remaining modifications involved the substitution of synonyms, sometimes in relation to nuanced meanings and in other cases, grammatical structures: “I have given him/her useful advice (le he dado un consejo útil)” was changed to “during difficult times, I supported my partner, giving advice (en momentos difíciles apoyé a mi pareja con un consejo)” (item 1); “I did something nice with … (hice algo agradable con …)” to “I did something nice together with … (hice algo agradable junto a …” (item 2); “weak points (puntos débiles)” to “flaws (defectos)” (item 3); “how he/she had made me suffer (cómo me había hecho sufrir)” to “how much he/she had made me suffer (cuánto me había hecho sufrir)” (item 4); “rude (pesado)” to “disagreeable (desagradable)” (item 8); I complained (reclamé)” to “I criticized (critiqué) (item 10); and “I did not (no hice)” was replaced by “I ignored (no le hice caso)” in item 12. For some items, words were added: “positive (positivas)” in item 5; “more (más)” was added before “time (tiempo)” in item 11; and “my partner (mi pareja)” was added.

Table 2 shows the items in their final version, after all amendments were incorporated. The CVC of the items after their modification (phase 2) ranged from .74 to 1, with all exceeding the inclusion criteria.

Validity evidence based on the instrument’s internal structure

In relation to the CEER questionnaire, the model test of two unrelated first-order factors, internal consistency, discrimination, descriptive and metric properties of items by factor, and validity study based on the relations to other variables, the raw data are available in S3 and S4 Datasets.

Multivariate skewness was met, S(364) = 3079.29 (p > .05), but multivariate kurtosis was not met, K = 46.49 (p < .001). As a result, we concluded that multivariate normality was not met.

After rejecting multivariate normality, a bivariate normality assumption was tested in order to be able to use a polychoric correlations matrix. A total of 66 contrasts were obtained (12 x 11/2). The premise of bivariate normality was accepted as the statistical significance associated with X2 was greater than .05 / 66 = .0007 on 59 occasions (89.4%). Additionally, all RMSEA values were below .1.

With regard to the analysis of dimensionality, the parallel analysis based on minimum rank factor advised for two dimensions. The following indexes suggested that the data should not be treated as essentially unidimensional: UniCo = .850, 95% CI [.744, .905]; ECV = .634, 95% CI [.567, .688]; and MIREAL = .466, 95% CI [.434, .503].

The CFA with the two unrelated first-order factors structure fit. There was no adjustment with X2(54) = 161.48, p < .001, possibly due to the large sample size. The other indexes showed an adequate fit of the proposed model as follows: CAIC = 304.02 (saturated CAIC = 566.99; independent CAIC = 5351.69); RMSEA = .052, 90% CI [.040, .063]; GFI = .97; AGFI = .95; CFI = .99; NFI = .98; and NNFI = .98. The model obtained appropriate λ, with values ranging from .65 to .85 (see Fig 1).

Instrument’s internal consistency, average discrimination index and items analysis

The CEER+ and CEER- factors presented good internal consistency (α = .79; ω = .80 and α = .85; ω = .85, respectively). The factors discrimination indexes were excellent (CEER+ = .55 and CEER- = .63). Table 3 introduces item descriptive statistics. All items presented negative S, ranging from -.81 to -2.33, and all but one obtained positive K, ranging from -.27 to 7.07, with excellent ID (.43, .71), good internal consistency if the items were deleted (α: .74 - .84; ω: .76 - .84) and adequate RI (.23, .74).

thumbnail
Table 3. Item descriptive statistics and metric indexes by factor in the CEER questionnaire.

https://doi.org/10.1371/journal.pone.0252329.t003

Validity evidence based on relations to other variables (test-criteria association)

None of the variables met the assumption of normality, with p ≤ .001. However, all associations met the assumptions of linearity (with p < .001) and error independence (1.825 < d < 2.057).

The validity evidence based on the relations of both CEER factors was calculated using the following criteria measures: RAS, DAS, and their respective factors (see Table 4). The reported measures also showed good consistency for RAS (α = .9; ω = .9), and for overall DAS (α = .89; ω = .9). In terms of the consensus and satisfaction subscales, results in general were adequate (α = .87; ω = .87 and α = .43; ω = .62 respectively); for the cohesion and affect expression subscales, they were acceptable (α = .67; ω = .68 and α = .68; ω = .75 respectively).

thumbnail
Table 4. Bivariate correlations as validity evidence based on the relation of the instrument to other variables (test—criteria correlation).

https://doi.org/10.1371/journal.pone.0252329.t004

Among all the variables mentioned, there were bivariate correlations with p < .01 in the expected sense: the higher the CEER+, the higher the RAS (ρ = .37) and overall DAS (ρ = .47), as well as their factors; and the higher the CEER-, the lower the RAS (ρ = -.37) and overall DAS (ρ = -.38), as well as their factors. In magnitude terms, CEER+ as well as CEER- correlations and the criterion variables were high to moderate. The correlation between both CEER factors was statistically significant, although with a low-medium magnitude (ρ = -.18).

SEM tested was formed by two predictors, i.e., CEER+ (X1) and CEER- (X2), on six criteria, i.e., RAS (Y1), Consensus–DAS–(Y2), Satisfaction–DAS–(Y3), Cohesion–DAS–(Y4), Affectional expression–DAS–(Y5) and Global DAS (Y6). The data fitted the model, X2(16) = .14, p > .05. Standardized effects and parameters significance of the test are presented in Table 5. Additionally, parameters significance is illustrated in Fig 2.

thumbnail
Fig 2. Parameters significance of the effects of X on Y based on SEM (CEER+ = positive Couples Extrinsic Emotion Regulation; CEER- = Negative Couples Extrinsic Emotion Regulation; RAS = Relationship Assessment Scale; DAS = Dyadic Adjustment Scale; Satisfac. = Satisfaction; A. expres. = Affectional expression.

Significant results -Student t values over 2.58 with a confidence level of 99%- are marked in bold).

https://doi.org/10.1371/journal.pone.0252329.g002

thumbnail
Table 5. Standardized effects and parameters significance (structural equation model).

https://doi.org/10.1371/journal.pone.0252329.t005

The model showed a direct positive relation between CEER+, RAS and overall DAS, as well as their factors; it revealed a negative relation between CEER-, RAS and overall DAS, as well as their factors. However, seven out of the twelve parameters did not yield statically significant values.

Discussion

The CEER questionnaire showed validity evidence based on the test content after its items were adapted from the original EROS [27] to Chilean couples. The existence of two independent factors was confirmed (CEER+ and CEER-). Both factors have adequate reliability and validity evidence based on the internal structure, in keeping with the results obtained when the instrument was initially used (for the general population) and validated with a Spanish sample [27]. Both scale factors correlated with other variables as hypothesized [8, 10, 13, 34]. This outcome provided some validity evidence based on the test-criteria association for the instrument.

By focusing on the IER’s deliberate intention and not only on the strategies they use, the CEER questionnaire allowed for the gathering of valuable data not always observable in studies about the emotional dynamics of romantic dyads [4, 5]. This could facilitate the identification of couples where at least one of the partners has a less favorable opinion of the relationship, providing relevant data for couple’s therapy. For example, as part of integrative behavioral couples’ therapy (IBCT), this tool could detect behavior considered problematic or harmful for the couple and usually contextualized [65] to determine the relationship differences and use acceptance or tolerance strategies to work through them.

It is important to emphasize that a validation process is a continuum in which evidence continually accumulates (content, construct, appearance, consequences, criteria, utility, etc.). The way in which the scores behave under other relevant conditions, though not examined in this work, will be important for future research. Studies could focus, for example, on couples at different stages of the relationship (newlyweds versus the most stable over time) and whether or not they cohabit. Another question is whether the intention underlying extrinsic regulation in a couple can be mediated by the other’s perception of this intention, since knowledge is limited to the behavioral level [66].

Some relevant limitations to the study include the use of non-probabilistic sampling (which could have generated biased data) and the low diversity of some characteristics (e.g., sexual orientation, as this was mainly a heterosexual sample; or language and location, i.e., a Spanish-speaking sample limited to Chile). Therefore, we suggest extrapolating the results with caution. In future research, a randomized procedure would be advisable, as would greater heterogeneity in terms of both the socio-demographic characteristics of the sample and the assessment of the relationship itself. The integration of these components would allow for an invariance study to determine whether the outcomes from this study are like those of different population groups. Couples who present clinical cases of abuse and domestic violence would be a suitable target population in future research, using the CEER questionnaire as another precise intervention tools along with other tests such as the Scale of Psychological Abuse in Intimate Partner Violence (EAPA) [67]. The CEER questionnaire could thus serve as an additional resource in a psychopathological diagnosis and the increase of the methodological quality of the interventions [68].

Supporting information

S1 Dataset. Raw data gathered in validity study based on the instrument’s content (English version).

https://doi.org/10.1371/journal.pone.0252329.s001

(XLSX)

S2 Dataset. Raw data gathered in validity study based on the instrument’s content (Spanish version).

https://doi.org/10.1371/journal.pone.0252329.s002

(XLSX)

S3 Dataset. Raw data gathered in validity study based on the instrument’s internal structure, internal consistency, average discrimination index, item analysis, and validity evidence based on the relations to other variables (English version).

https://doi.org/10.1371/journal.pone.0252329.s003

(XLSX)

S4 Dataset. Raw data gathered in validity study based on the instrument’s internal structure, internal consistency, average discrimination index, item analysis, and validity evidence based on the relations to other variable (Spanish version).

https://doi.org/10.1371/journal.pone.0252329.s004

(XLSX)

Acknowledgments

We would like to thank Danny Vargas Escobar for his support reviewing and reading the manuscript, the translator for their precise rendering of our work in English, and the reviewers for their thoughtful comments that contributed to improving this paper.

References

  1. 1. Niven K, Totterdell P, Stride C, Holman D. Emotion Regulation of Others and Self (EROS): The development and validation of a new individual difference measure. Curr Psychol. 2011; 30: 53–73. https://doi.org/10.1007/s12144-011-9099.
  2. 2. Hoffmann S, Carpenter J, Curtiss J. Interpersonal emotion regulation questionnaire (IERQ): Scale development and psychometric characteristics. Cognit Ther Res. 2016; 40(3): 341–356. https://doi.org/10.1007/s10608-016-9756-2 pmid:27182094
  3. 3. Niven K, Totterdell P, Holman D. A classification of controlled interpersonal affect regulation strategies. Emot. 2009; 9(4): 498–509. https://doi.org/10.37/a0015962 pmid:19653772
  4. 4. Zaki J, Williams C. Interpersonal emotion regulation. Emot. 2013; 13(2): 803–810. https://doi.org/10.1037/a0033839 pmid:24098929
  5. 5. Niven K, Totterdell P, Holman D, Headly T. Does regulating others’ feelings influence people’s own affective well-being? J Soc Psychol. 2012; 152(2): 246–260. https://doi.org/10.1080/00224545.2011.599823 pmid:22468424
  6. 6. Elliot AJ. Motivated emotion regulation: Principles, lessons, and implications of a motivational analysis of emotion regulation. Adv Motiv Sci. 2017; 4: 207–247. https://doi.org/ 10.1016/bs.adms.2016.12.001.
  7. 7. Altinok A, Kılıc N. Exploring the associations between narcissism, intentions towards infidelity, and relationship satisfaction: Attachment styles as a moderator. PLoS ONE. 2020; 15(11): e0242277. https://doi.org/10.1371/journal.pone.0242277 pmid:33186382
  8. 8. Vater A, Schröder-Abé M. Explaining the link between personality and relationship satisfaction: Emotion regulation and interpersonal behavior in conflict discussions. Eur J Pers. 2015; 29(2): 201–215. https://doi.org/10.1002/per.1993.
  9. 9. Becerra R, Preece DA, Gross JJ. Assessing beliefs about emotions: Development and validation of the Emotion Beliefs Questionnaire. PLoS ONE. 2020; 15(4): e0231395. https://doi.org/10.1371/journal.pone.0231395 pmid:32287328
  10. 10. Gottman J, Levenson R. Marital processes predictive of later dissolution: Behavior, physiology, and health. J Pers Soc Psychol. 1992; 63(2): 221–233. https://doi.org/10.1037//0022-3514.63.2.221 pmid:1403613
  11. 11. Weber D, Herr N. The messenger matters: Invalidating remarks from men provoke a more negative emotional reaction than do remarks from women. Psychol Rep. 2019; 122(1): 180–200. https://doi.org/10.1177/0033294117748618 pmid:29298582
  12. 12. Pedro M, Ribeiro T, Shelton K. Romantic attachment and family functioning: The mediating role of marital satisfaction. J Child Fam Stud. 2015; 24(11): 3482–3495. https://doi.org/10.1007/s10826-015-0150-6.
  13. 13. Scorsolini-Comin F, Santos M. [Dyadic adjustment and marital satisfaction: Correlations between the domains of two scales for assessing conjugality]. Psicol Reflex Crit. 2011; 24(3):439–447. Available from: https://www.redalyc.org/articulo.oa?id=18821107007.
  14. 14. Klein SR, Renshaw KD, Curby TW. Emotion regulation and perceptions of hostile and constructive criticism in romantic relationships. Behav Ther. 2016; 47(2): 143–154. https://doi.org/10.1016/j.beth.2015.10.007 pmid:26956648
  15. 15. Anolli L, Balconi M. Topic variability and linguistic convergence in marital couples’ speech as related to specific attachment style. Psychol Rep. 2005; 96(1): 83–106. https://doi.org/10.2466/pr0.96.1.83-106 pmid:15825910
  16. 16. Lüscher J, Stadler G, Scholz U. A daily diary study of joint quit attempts by dual-smoker couples: The role of received and provided social support. Nicotine Tob Res. 2018; 20(1): 100–107. https://doi.org/10.1093/ntr/ntx079.
  17. 17. McCormick MP, Hsueh J, Merrilees C, Chou P, Cummings EM. Moods, stressors, and severity of marital conflict: A daily diary study of low-income families. Fam Relat. 2017; 66: 425–440. https://doi.org/10.1111/fare.12258.
  18. 18. Rusu PP, Nussbeck FW, Leuchtmann L, Bodenmann G. Stress, dyadic coping, and relationship satisfaction: A longitudinal study disentangling timely stable from yearly fluctuations. PLoS ONE. 2020; 15(4): e0231133. https://doi.org/10.1371/journal.pone.0231133 pmid:32271854
  19. 19. Davies K, Lane A, Devonport T, Scott J. Validity and reliability of a brief emotional intelligence scale (BEIS-10). J Individ Differ. 2020; 31(4): 198–208. https://doi.org/10.1027/1614-0001/a000028.
  20. 20. Durosini I, Triberti S, Ongaro G, Pravettoni G. Validation of the Italian version of the Brief Emotional Intelligence Scale (BEIS-10). Psychol Rep. 2020; 1–21. https://doi.org/10.1177/0033294120959776.
  21. 21. López-Pérez B, Morillo D, Wilson E. Development and validation of the Interpersonal Affect Improvement Strategies Questionnaire. Eur J Psychol Assess. 2019; 35(2): 280–294. https://doi.org/10.1027/1015-5759/a000394.
  22. 22. Foran H, Lorber M, Malik J, Heyman R, Slep A. The Intimate Partner Flooding Scale. Assess. 2020; 27(6): 1151–1162. https://doi.org/10.1177/1073191118755911 pmid:29676164
  23. 23. Austin E, O’Donnell M. Development and preliminary validation of a scale to assess managing the emotions of others. Pers Individ Dif. 2013; 55(7): 834–839. https://doi.org/10.1016/j.paid.2013.07.005.
  24. 24. Jankowski K, Zajenkowski M, Stolarski M, Styła R, Zajenkowska A, Jędrasik-Styła M, et al. Polish version of the Managing the Emotions of Others Scale (MEOS): Psychometric properties and validation. Psychol Rep. 2016; 118(2): 532–543. https://doi.org/10.1177/0033294116636998 pmid:27154378
  25. 25. Dixon-Gordon KL, Haliczer LA, Conkey LC, Whalen DJ. Difficulties in interpersonal emotion regulation: Initial development and validation of a self-report measure. J Psychopathol Behav Assess. 2018; 40(3): 528–549. https://doi.org/10.1007/s10862-018-9647-9.
  26. 26. Páez D, Rimé B, Basabe N, Wlodarczyk A, Zumeta L. Psychosocial effects of perceived emotional synchrony in collective gatherings. J Pers Soc Psychol. 2015; 108(5): 711–729. https://doi.org/10.1037/pspi00000141 pmid:25822033
  27. 27. Lozano L, López L, Santas J, Cifre E. [Validation of the Emotion Regulation Others and Self (EROS) questionnaire. A pilot study in a sample of the elderly]. Àgora de Salut. 2015; 2: 89–97. https://doi.org/10.6035/AgoraSalut.2015.2.8.
  28. 28. Stokes J. Do “his” and “her” marriages influence one another? Contagion in personal assessments of marital quality among older spouses over a four-year span. Soc Psychol Q. 2017; 80(2): 204–215. https://doi.org/10.1177/019027251770170.
  29. 29. West CE. Clinical utility and validation of the Couple’s Communicative Evaluation Scale. Assess. 2005; 97(2): 599–622. https://doi.org/10.2466/pr0.97.2.599-622 pmid:16342591
  30. 30. Sels L, Ruan Y, Kuppens P, Ceulemans E, Reis H. Actual and perceived emotional similarity in couples’ daily lives. Assess. 2019; 11(2): 266–275. https://doi.org/10.1177/1948550619845927.
  31. 31. Williams W, Morelli S, Ong D, Zaki J. Interpersonal emotion regulation: Implications for affiliation, perceived support, relationships, and well-being. J Pers Soc Psychol. 2018; 115(2): 224–254. https://doi.org/10.1037/pspi0000132 pmid:29733662
  32. 32. American Educational Research Association, American Psychological Association, National Council on Measurement in Education, Joint Committee on Standards for Educational and Psychological Testing. Standards for Educational and Psychological Testing. Washington DC, USA: Authors, 2014.
  33. 33. Muñiz J, Fonseca-Pedrero E. [Ten steps for test development]. Psicothema, 2019; 31(1): 7–16. https://doi.org/10.7334/psicothema2018.291 pmid:30664405
  34. 34. Fernández J. [Emotional regulation and conflict in dyadic adjustment of couples]. Master’s Thesis. Faculty of Human and Social Sciences, Universidad Pontificia Comillas. Available from: http://hdl.handle.net/11531/32423.
  35. 35. Prieto G, Muñiz J. [A model to evaluate the quality of tests used in Spain]. Papeles del Psicol. 2000; 77: 65–75. Available from: https://www.redalyc.org/pdf/778/77807709.pdf.
  36. 36. International Test Commission. The ITC guidelines for translating and adapting tests (2nd ed.). Int J Test. 2018; 18(2): 101–134. https://doi.org/10.1080/15305058.2017.1398166.
  37. 37. Muñiz J, Elosua P, Hambleton RK. [International Test Commission guidelines for test translation and adaptation: Second edition]. Psicothema. 2013; 25(2): 151–157. https://doi.org/10.7334/psicothema2013.24 pmid:23628527
  38. 38. Hendrick S. A generic measure of relationship satisfaction. J Marriage Fam. 1988; 50(1): 93–98. https://doi.org/10.2307/352430.
  39. 39. León MJ, Olhaberry M, Hernández C, Sieverson C. Couple satisfaction and depression: Is the reflective functioning an intervening variable? Psykhe. 2018; 27(2): 1–14. https://doi.org/10.7764/psykhe.27.2.1139.
  40. 40. Spanier G. Assessing the strengths of the Dyadic Adjustment Scale. J Fam Psychol. 1988; 2: 92–94. https://doi.org/10.1037/h0080477.
  41. 41. Santos P, Vallejo P, Sierra JC. [Psychometric properties of a short version of the Dyadic Adjustment Scale in Spanish samples]. International Journal of Clinical and Health Psychology. 2009; 9(3): 501–517. Available from: https://www.redalyc.org/pdf/337/33712038009.pdf.
  42. 42. Cano-Prous A, Martín-Lanas A, Moyá-Querejeta J, Beunza-Nuin M, Lahortiga-Ramos F, García-Granero M. Psychometric properties of a Spanish version of the Dyadic Adjustment Scale. International Journal of Clinical and Health Psychology. 2014; 14(2): 137–144. https://doi.org/10.1016/s1697-2600(14)70047-x.
  43. 43. Cuenca M, Graña J, Peña M, Andreu J. Psychometric properties of the Dyadic Adjustment Scale (DAS) in a community sample of couples. Psicothema. 2013; 25(4): 536–541. https://doi.org/10.7334/psicothema2013.85 pmid:24124789
  44. 44. Vera C. [Marital satisfaction: Incident factors]. Dissertation. School of Psychology, Faculty of Education and Humanities, Universidad del Bío-Bío. Chillán, Chile. 2010. Available from: http://repobib.ubiobio.cl/jspui/handle/123456789/1557.
  45. 45. Brock R, Barry R, Lawrence E, Dey J, Rolffs J. Internet Administration of Paper-and-Pencil Questionnaires Used in Couple Research. Assess. 2010; 19(2): 226–242. https://doi.org/10.1177/1073191110382850 pmid:20881103
  46. 46. Lawshe CH. A quantitative approach to content validity. Pers Psychol. 1975; 28: 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x.
  47. 47. Mardia KV. Measures of multivariate skewness and kurtosis with applications. Biometrika. 1970; 57: 519–530. https://doi.org/10.2307/2334770.
  48. 48. Muthén B, Kaplan D. A comparison of some methodologies for the factor analysis of non-normal Likert variables. Br J Math Stat Psychol. 1985; 38: 171–189. https://doi.org/10.1111/j.2044-8317.1985.tb00832.x.
  49. 49. Muthén B, Kaplan D. A comparison of some methodologies for the factor analysis of non-normal Likert variables: A note on the size of the model. Br J Math Stat Psychol. 1992; 45: 19–30. https://doi.org/10.1111/j.2044-8317.1992.tb00975.x.
  50. 50. Coenders G, Satorra A, Saris WE. Alternative approaches to structural modeling of ordinal data: A Monte Carlo study. Struct Equ Modeling. 1997; 4: 261–282.
  51. 51. Holgado-Tello FP, Carrasco MA, del Barrio V, Chacón-Moscoso S. Factor analysis of the Big Five Questionnaire using polychoric correlations in children. Qual Quant. 2020; 43(1): 75–85. https://doi.org/10.1007/s11135-007-9085-3.
  52. 52. Lee S. Preliminary examination of psychometric properties of the Korean version of the specific interpersonal trust scale. Psychol Rep. 2007; 100(2): 355–364. https://doi.org/10.2466/pr0.100.2.355-364 pmid:17564209
  53. 53. Domínguez-Sánchez FJ, Lasa-Aristu A, Amor PJ, Holgado-Tello FP. Psychometric properties of the Spanish version of the Cognitive Emotion Regulation Questionnaire. Assess. 2011; 20(2): 253–261. https://doi.org/10.1177/1073191110397274 pmid:21467092
  54. 54. Holgado-Tello FP, Chacón-Moscoso S, Barbero-García M, Vila-Abad E. Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables. Quality and Quantity. 2010; 44: 153–166. https://doi.org/10.1007/s11135-008-9190-y.
  55. 55. Timmerman ME, Lorenzo-Seva U. Dimensionality assessment of ordered polytomous items with parallel analysis. Psychol Methods. 2011; 16: 209–220. https://doi.org/10.1037/a0023353 pmid:21500916
  56. 56. Ferrando PJ, Lorenzo-Seva U. Assessing the quality and appropriateness of factor solutions and factor score estimates in exploratory item factor analysis. Educ Psychol Meas. 2018; 78: 762–780. https://doi.org/10.1177/0013164417719308 pmid:32655169
  57. 57. Holgado-Tello FP, Suárez-Falcón JC, Morata-Ramírez MA. [Structural Equation Modeling from the path analysis to the multigroup analysis]. Madrid, Spain: Sanz y Torres, 2019. https://doi.org/10.1371/journal.pone.0215325 pmid:30973954
  58. 58. Petrowski K, Brähler E, Suslow T, Zenger M. Correction: Revised short screening version of the attachment questionnaire for couples from the German general population. PLoS ONE. 2020; 15(5): e0234228. https://doi.org/10.1371/journal.pone.0234228 pmid:32470097
  59. 59. Byrne B. Structural equation modeling with LISREL, PRELIS, and SIMPLIS: Basic concepts, applications, and programming: Mahwah, NJ: Lawrence Erlbaum Associates, 1998.
  60. 60. Millings A, Hirst SL, Sirois F, Houlston C. Emotional adaptation to relationship dissolution in parents and non-parents: A new conceptual model and measure. PLoS ONE. 2020; 15(10): e0239712. https://doi.org/10.1371/journal.pone.0239712 pmid:33112883
  61. 61. George D, Mallery P. SPSS for Windows step by step: A simple guide and reference. Boston, MA: Allyn & Bacon, 2003.
  62. 62. McDonald RP. Test theory: A unified treatment. Mahwah: Lawrence Erlbaum Associates, 1999.
  63. 63. Linnette-D’Sa1 J, Visbal-Dionaldo ML. Analysis of multiple-choice questions: Item difficulty, discrimination index and distractor efficiency. Int J Nurs Educ. 2017; 9(3): 109–114. https://doi.org/10.5958/0974-9357.2017.00079.4.
  64. 64. Cohen J. Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates, 1998.
  65. 65. Barraca J. Integrative Behavioral Couple Therapy (IBCT) as a third-wave therapy. Psicothema. 2015; 27(1): 13–18. https://doi.org/10.7334/psicothema2014.101 pmid:25633764
  66. 66. Vasquez C, Niven K, Madrid H. Leader interpersonal emotion regulation and follower performance. J Pers Psychol. 2020; 19(2): 97–101. https://doi.org/10.1027/1866-5888/a000249.
  67. 67. Porrúa-García C, Rodríguez-Carballeira A, Escartín J, Gómez-Benito J, Almendros C, Martín-Peña J. Development and validation of the scale of psychological abuse in intimate partner violence (EAPA-P). Psicothema. 2016; 28(2): 214–221. https://doi.org/10.7334/psicothema2015.197 pmid:27112821
  68. 68. Chacón-Moscoso S, Sanduvete-Chaves S, Sánchez-Martín M. The development of a checklist to enhance methodological quality in intervention programs. Front Psychol. 2016; 7: 1811. https://doi.org/10.3389/fpsyg.2016.01811 pmid:27917143