Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Seeking certainty about Intolerance of Uncertainty: Addressing old and new issues through the Intolerance of Uncertainty Scale-Revised

  • Gioia Bottesi ,

    Roles Conceptualization, Investigation, Project administration, Writing – original draft

    gioia.bottesi@unipd.it

    Affiliation Department of General Psychology, University of Padova, Padova, Italy

    ⨯
  • Stefano Noventa,

    Roles Conceptualization, Methodology, Writing – original draft

    Affiliation Methods Center, University of Tübingen, Tübingen, Germany

    ⨯
  • Mark H. Freeston,

    Roles Conceptualization, Project administration, Supervision

    Affiliation School of Psychology, Newcastle University, Newcastle upon Tyne, United Kingdom

    ⨯
  • Marta Ghisi

    Roles Conceptualization, Project administration, Supervision, Writing – original draft

    Affiliation Department of General Psychology, University of Padova, Padova, Italy

    ⨯

Abstract

Intolerance of Uncertainty is a trans-diagnostic process that spans a range of emotional disorders and it is usually measured through the Intolerance of Uncertainty Scale-12. The current study aims at investigating some issues in the assessment of Intolerance of Uncertainty (IU) through the Italian Intolerance of Uncertainty Scale-Revised, a measure adapted from the Intolerance of Uncertainty Scale-12 to assess IU across the lifespan. In particular we address the factor structure among a large community sample, measurement invariance across gender, age, and over time, together with reliability and validity of the overall scale and its subscales. The questionnaire was administered to community (N = 761; mean age = 35.86 ± 14.01 years) and undergraduate (N = 163; mean age = 21.16 ± 2.64 years) participants, together with other self-report measures assessing constructs theoretically related to IU. The application of a bifactor model shows that the Italian Intolerance of Uncertainty Scale-Revised possesses a robust general factor, thus supporting the use of the unit-weighted total score of the questionnaire as a measure of the construct. Furthermore, measurement invariance across gender, age, and over time is supported. Finally, the Italian Intolerance of Uncertainty Scale-Revised appears to possess adequate reliability and validity. These findings support the unidimensionality of the measure, a conceptually reasonable result in line with the trans-diagnostic nature of Intolerance of Uncertainty. In addition, this study and comparison with published factor structures of the Intolerance of Uncertainty Scale-12 and of the Intolerance of Uncertainty Scale-Revised identify some issues for the internal structure of the measure. In particular, concern is expressed for the Prospective IU subscale. In light of the promising psychometric properties, the use of the Italian Intolerance of Uncertainty Scale-Revised as a univocal measure is encouraged in both research and clinical practice.

Citation: Bottesi G, Noventa S, Freeston MH, Ghisi M (2019) Seeking certainty about Intolerance of Uncertainty: Addressing old and new issues through the Intolerance of Uncertainty Scale-Revised. PLoS ONE 14(2): e0211929. https://doi.org/10.1371/journal.pone.0211929

Editor: Michael B. Steinborn, University of Wuerzburg, GERMANY

Received: March 26, 2018; Accepted: January 24, 2019; Published: February 11, 2019

Copyright: © 2019 Bottesi et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the manuscript and its Supporting Information files.

Funding: The authors received no specific funding for this work.

Competing interests: We declare that we have read the journal's policy and the authors of this manuscript have the following competing interests: Prof. Mark H. Freeston declares financial conflict of interest (training honoraria and book royalties on closely related topics), whereas all the other authors declare that they have no conflict of interest. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Introduction

Intolerance of Uncertainty (IU) is the dispositional inability of an individual to tolerate the aversive reactions triggered by a perceived lack of sufficient/salient information and maintained by the related perception of uncertainty [1]. People with high levels of IU find uncertain future events as threatening, upsetting, and undesirable. In the attempt to control or avoid uncertainty, they usually endorse negative beliefs about their ability to cope with it, they experience high levels of distress, and they are likely to perform maladaptive behaviors like excessive information seeking, avoidance, or impulsive decision-making [2–6]. IU shares similarities with other psychological constructs (such as intolerance of ambiguity [7], distress tolerance [8], stress vulnerability [9], action vs. state orientation [10], indecisiveness [11], need for cognitive closure [12]), but its distinctiveness has been demonstrated [13–15] to the extent that fearing the unknown is posited to represent a fundamental fear [1]. Although IU was originally conceptualized as a cognitive vulnerability factor for worry, i.e. the core feature of Generalized Anxiety Disorder (GAD) [16, 17], it is currently considered a trans-diagnostic factor that putatively underlies neuroticism [1] and that spans a broad range of emotional disorders like Obsessive-Compulsive Disorder (OCD), social anxiety, panic disorder and agoraphobia, depression, post-traumatic stress disorder, and eating disorders [18– 21]. Additionally, recent evidence supports the notion of IU as a trans-diagnostic and trans-therapy change process. In particular, some studies demonstrated the effectiveness of unified treatment protocols focusing on IU when targeting multiple psychopathologies [22, 23]. In light of the relevance of IU as a clinical concept, the availability of a valid measure capable of reliably measuring it in both clinical and research settings appears crucial.

The Intolerance of Uncertainty Scale

Since 1994, the Intolerance of Uncertainty Scale (IUS) [17] has represented the most widely adopted standard measure of IU [24]. The original IUS consisted of 27 items developed from the clinical observation of people suffering from GAD. Given the broad definition of IU underlying the IUS, and the inevitable dependence of factor analytic solutions on sample composition, studies designed to examine its factor structure have led to inconsistent findings, revealing either one- [25], two- [26–28], three- [29], four- [2, 30], or even five-factor solutions [17]. Most of all, factors were difficult to interpret and showed cross-loaded items [24, 31]. In particular, results suggested that the IUS lacked cross-cultural validity, as different factor structures emerged with different languages (i.e., French, English, Dutch, Spanish, and German) and cultures (i.e., Canadian, American, Dutch, Spanish, and German). Furthermore, Norton [32] investigated the psychometric properties and the factor structure of the IUS in four different racial groups (African, Caucasian, Hispanic, and Southeast Asian) and reported differences in the factor structure across groups, despite substantially observing similar reliability and validity values. To note, most of these studies were based on under- or post-graduate students; factor analyses were carried out on adult community samples only in a few cases [27, 28]. A further issue concerning the validity of the IUS was the “GAD-specific” nature of some of its items (e.g. “My mind can’t be relaxed if I don’t know what will happen tomorrow”), which raised further questions about its capability to capture the core IU construct [3, 33] in a trans-diagnostic context. In the attempt to overcome these issues, the original IUS was reduced to a 12-item scale by Carleton et al. [26].

The Intolerance of Uncertainty Scale-12

Following the previous considerations, Carleton et al. [26] developed the IUS-12, a 12 item-questionnaire consisting of two factors they had identified in a subset of the items in the original measure. The first, Prospective IU, expresses the propensity of individuals toward active information seeking as a way to reduce uncertainty/increase certainty. The second, Inhibitory IU, refers to avoidance-oriented responses to uncertainty, i.e. an inhibition of actions or experience which is caused by uncertainty [24, 26, 34]. The authors recommended the use of the IUS-12 total score over the IUS to evaluate general IU as the latter was still too close to a measure of worry [26]. The IUS-12’s correlated two-factor structure was initially tested in two large undergraduate samples [26] and further replicated in several studies employing also community [3] and clinical [34–37] samples. Its cross-cultural validity (i.e., White and Black university students as well as Dutch undergraduate students) was also demonstrated [38, 39]. Good internal consistency of the IUS-12 total score and of the two subscales has been observed in non-clinical samples [3, 36, 39, 40], as well as in samples of people diagnosed with GAD [36], OCD [35], and comorbid anxiety and depression [37]. Furthermore, the IUS-12 has shown medium-high correlations with measures of worry, anxiety, depression, social anxiety, and general distress, and a stronger correlation with the Perfectionism/certainty scale of the Obsessive Beliefs Questionnaire-44 (OBQ-44) [26, 35–37, 39–41]. Taken together, the previous findings on the convergent and divergent validity of the IUS-12 support the trans-diagnostic nature of IU.

More recently, Hale et al. [31] pointed out that Carleton et al. [26] did not provide empirical justification for deriving a total score on the IUS-12. Therefore, they analyzed the factor structure of the IUS-12 in a sample of American psychology undergraduates and showed that a bifactor model fitted the data better than the unidimensional model and Carleton et al.’s correlated two-factor solution. They obtained evidence of a strong general factor and of weak and unreliable Prospective and Inhibitory group factors. Similarly, Lauriola, Mosca, and Carleton [42] tested the factor structure of an Italian translation of the IUS-12 in a sample of psychology university students and also observed that a bifactor model fitted the sample data better than the alternative models. Finally, Shihata, McEvoy and Mullan [43] recently reported that a bifactor model provided the best fit in Australian psychology undergraduate and clinical samples. In both these studies, similar findings to Hale et al. [31] were found, namely good reliability of the general factor and lower reliability for the group factors.

It should be stressed, however, that a better fit of the bifactor model might not necessarily imply that the factor structure of the IUS-12 follows a bifactor structure rather than a unidimensional or a correlated two-factor solution. Indeed, both the substantive interpretation of the bifactor solution as the true factor model underlying specific cognitive or psychological phenomena, and the fact that the bifactor might be advantaged in terms of fit indexes over other models, are still under debate [44–47]. Notwithstanding these issues, the bifactor model can be used to assess whether the group factors can be considered independent and auto-sufficient scales or alternatively whether the unit-weighted total score can be used as a univocal measure of a unidimensional construct without introducing excessive bias due to any multi-dimensionality [48–51]. In the present manuscript the bifactor model is thus considered and investigated more for the latter reasons rather than as a viable alternative model to the unidimensional or the correlated two-factor solution.

The current study

Different adaptations of the same questionnaire for children or adults often lead to different items and factor structures, thus making the interpretation and comparison of scores difficult [52]. In order to create a version of the IUS-12 which is suitable to people of different ages, Walker, Birrell, Rodgers, Leekam, and Freeston [53] rephrased some of the items of the IUS-12 and thus developed the IUS-Revised (IUS-R), a measure assessing IU across the lifespan. In particular, the language was simplified so that it can be easily read by an average 11-year-old student (Flesch Reading Ease = 81.5) [53]. To date, the IUS-R has proven to be a reliable and valid measure of IU in different samples: non-clinical samples of British and Spanish young people aged 6–8 years, 9–11 years, 12–14 years, and undergraduates [54]; healthy young people aged 11–17 [55]; children and adolescents with diagnoses of Autism or Asperger’s syndrome [56, 57]. Preliminary data supporting the adequacy of the IUS-R have been reported in a sample of Italian undergraduate students: Bottesi et al. [58] showed that Carleton et al.’s correlated two-factor model [26] fitted the data better than a single-factor model. Moreover, the Italian IUS-R showed acceptable internal consistency (total score: α = .80; Prospective IU: α = .68; Inhibitory IU: α = .79) and adequate construct validity. Furthermore, the IUS-R total score significantly predicted worry, even after controlling for somatic anxiety and depressive symptoms. However, Bottesi et al. [58] did not address some important issues, namely, the factor structure of the questionnaire in a community sample, an investigation of a bifactor model, gender and age invariance, and psychometric properties such as temporal stability.

The present study aims to fill some gaps in the extant literature on the assessment of IU as measured by means of the IUS, the IUS-12, or the IUS-R in a detailed psychometric examination of the Italian IUS-R. In our opinion, expanding knowledge about the measurement of IU through the IUS is highly desirable and warranted: providing clinical researchers and practitioners with a measure capable of adequately capturing the IU construct during screening and/or assessment phases has indeed clear implications in terms of both clinical evaluation and treatment planning.

First, we examine the factor structure of the IUS-R in a large community sample given that a significant proportion of studies on the various IUS measures have been conducted among psychology undergraduates [2, 17, 26, 31, 42, 43, 58] preventing generalization to community samples. Second, we address measurement invariance across gender and age. To our knowledge, to date only one study assessed gender invariance among university students [31], whereas none have examined age invariance. Third, we address one-month temporal stability and longitudinal measurement invariance in a sample of undergraduates: to our knowledge, only Khawaja and Yu [36] provided information about test-retest reliability (two-week interval) of the IUS-12 in a similar group (r = . 77), but they did not test measurement invariance over time. Finally, we address the reliability of the Prospective and Inhibitory subscales, which have been previously suggested to be unreliable [31, 43]. As the IUS-R is adapted from the IUS-12, most of the expected results were based on previous literature on the original version of the questionnaire. Specifically:

  1. The bifactor model was expected to show a superior fit when assessing the structure of the Italian IUS-R in a large community sample. Consistent with recent evidence [31, 42, 43], although the correlated two-factor model was expected to show better fit than the unidimensional model, the bifactor model was expected to support a unidimensional structure and high reliability of the Italian IUS-R unit-weighted total score (please note that, for readability reasons, from this point on we will simply define it as the “IUS-R total score”). In addition, although the Prospective and Inhibitory subscales were not expected to be reliable, item loadings on the group factors were expected to be stronger than in the IUS-12 as a result of the modifications of the items carried in the IUS-R version.
  2. The correlations between the Italian IUS-R scores and demographic variables such as age and education were expected to be small (e.g., r < |.10|).
  3. The Italian IUS-R was expected to possess good convergent validity. Based on previous literature on the IUS-12 and the preliminary data reported by Bottesi et al. [58], we anticipated to observe medium-to-large positive correlations (e.g., r ≥ .30 to < .50) with measures of negative problem orientation (NPO), worry, anxiety, depression, general distress, OC symptoms, and a strong association with a concurrent measure of IU [35].
  4. In line with Hale et al. [31], no significant bias in the items was expected across genders. As only a few studies have reported on gender differences [39, 58], and in line with preliminary findings by Bottesi et al. [58], no gender differences were expected. In particular, measurement invariance across gender was expected. Similarly, as the IUS-R was developed to assess IU across lifespan, measurement invariance was also expected across different age groups.
  5. Given that the original IUS and its derivatives are conceptualized as measuring a dispositional feature and in light of previous literature, temporal stability of the IUS-R was expected to be strong and invariant in an undergraduate sample [36].

Materials and methods

Participants and procedure

A community sample made up of 761 people (302 men and 459 women), living in different midsized communities in northern, central, and southern Italy, were recruited through advertisements requesting volunteers for psychological studies. Their mean age was 35.86 (SD = 14.01, range = [17, 78]) and their mean years of education were 14.36 (SD = 3.73, range = [5, 30]). Marital status was 54.7% single/living alone, 39.3% married/in a domestic relationship, 5.1% separated/divorced, and 0.9% widowed. The employment profile of the sample was: 43.8% full-time employment, 28.1% student, 7.9% part-time employment, the 3.8% full-time homemaker, 3.9% occasionally employed, 3.3% retired, 3.4% unemployed, 0.4% unable to work due to disability, and 5.4% other. Exclusion criteria were current psychological disorders, major neurocognitive disorder, and intellectual disability (intellectual developmental disorder). Data from this sample was used to assess the factor structure, reliability, measurement invariance, and convergent validity. Data were collected between 2012 and 2017 in four different but related studies, each employing slightly different batteries of self-report measures. The resulting datasets were merged to obtain the final community sample of this study. Therefore, not all individuals completed all the same self-report questionnaire measures, but all individuals completed the IUS-R.

A further non-clinical sample of 163 undergraduate students (47 males and 116 females) attending the second year of university at the School of Psychology was recruited to test one-month temporal stability and longitudinal measurement invariance. Exclusion criteria were the same as for the community sample. Their mean age was 21.16 (SD = 2.64, range = [19, 48]), and mean years of education were 13.22 (SD = .79, range = [12, 18]); 46.6% were single/living alone and 53.4% were in a relationship.

All participants were informed of the study’s aims and gave their written, informed consent before entering the study; no incentives were offered. Participants filled in a socio-demographic form and a battery of self-report questionnaires. The research was conducted in accordance with the Declaration of Helsinki and was approved by the Ethics Committee of Psychological Sciences of the University of Padova. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Measures

In the present subsection all the measures considered in the manuscript are listed together with their values of Cronbach’s alpha and the associated confidence intervals and sample sizes. It should be noted, however, that Cronbach’s alpha has been here reported mostly for comparative reasons with the extant literature. Readers should exercise caution in interpreting Cronbach’s alpha, as the reported measures largely differ both in number and homogeneity of the items, so that any interpretation should be driven by considerations on the context and the scope and length of the measure [59]. A more methodologically sound comparison between the measures should be based on cumulative reliability functions obtained by means of re-sampling techniques [60]. In the present study, examination of the reliability of the IUS-R scales and its Prospective and Inhibitory subscales will be addressed by means of omega composite reliability coefficients associated with the bifactor solution. As a final note, Table 1 lists published means and standard deviations of the Italian version of the measures, together with their estimated values in the current community sample.

thumbnail
Download:
Table 1. Means and standard deviations for all measures (Italian versions) from published studies (population parameters) and the current sample.

https://doi.org/10.1371/journal.pone.0211929.t001

The Intolerance of Uncertainty Scale-Revised (IUS-R; Italian version by Bottesi et al. [58]; the Italian version of the IUS-R can be found in the S1 File) consists of 12 items assessing IU. Respondents are asked to rate the extent to which each statement applies to themselves on a 5-point Likert scale; as previously outlined, promising preliminarily psychometric properties have been reported [58]. Almost all participants (758 community individuals and 163 undergraduate students) completed the IUS-R. In the community sample, Cronbach’s alpha was α = .87 (CI = [.85, .88], N = 758). The IUS-R contains two subscales, Prospective IU (IUS-R-P), and Inhibitory IU one (IUS-R-I). Cronbach’s alphas for the subscales were respectively α = .78 (CI = [.76, .81], N = 758) and α = .86 (CI = [.84, .87], N = 758).

The Obsessive Beliefs Questionnaire-87 (OBQ-87; Italian version by Sica et al. [61]) consists of 87 items assessing the dysfunctional belief domains believed to be involved in the onset and maintenance of OCD. The Italian version of the measure has demonstrated good internal consistency and test-retest reliability. Given the purpose of the present study, only the subscale measuring tolerance of uncertainty (OBQ-87-T) was used to assess convergent validity with the IUS-R. Cronbach’s alpha in the community sample was α = .89 (CI = [.88, .91], N = 213).

The Negative Problem Orientation Questionnaire (NPOQ; Italian version by Bottesi & Ghisi [62]) consists of 12 items assessing the individual’s approach to problems, including beliefs that problems are threatening, low self-confidence about abilities to solve problems, and pessimism about problem resolution (i.e., NPO). The Italian version of the NPOQ has shown adequate internal consistency, test-retest reliability, convergent, and discriminant validity. Cronbach’s alpha in the community sample was α = .93 (CI = [.92, .94], N = 429).

The Penn State Worry Questionnaire (PSWQ; Italian version by Morani, Pricci, & Sanavio [63]) consists of 16 items measuring the tendency to worry excessively and uncontrollably. Psychometric properties of the Italian version were adequate. In the community sample, Cronbach’s alpha was α = .91 (CI = [.89, .92], N = 429).

The Beck Anxiety Inventory (BAI; Italian version by Sica, Coradeschi, Ghisi, and Sanavio [64]) consists of 21 items assessing the severity of anxiety over the previous week. The Italian version of the BAI showed excellent internal consistency and good test-retest reliability; Cronbach’s alpha in the community sample was α = .90 (CI = [.89, .91], N = 256).

The Beck Depression Inventory-II (BDI-II; Italian version by Ghisi, Flebus, Montano, Sanavio, & Sica [65]) consists of 21 items evaluating the severity of depression over the previous two weeks. The BDI-II showed high internal consistency and good test-retest reliability in the Italian version. In the community sample, Cronbach’s alpha was α = .83 (CI = [.81, .85], N = 258).

The Depression Anxiety Stress Scales -21 (DASS-21; Italian version by Bottesi et al. [66]) consists of 21 items assessing depression, anxiety, and stress over the previous week. Findings on the Italian version suggested that the use of the total score, measuring a general distress factor, might be more appropriate than scoring the three subscales separately; the total score of the Italian version showed excellent internal consistency [66]. In the community sample, Cronbach’s alpha was α = .90 (CI = [.88, .91], N = 170).

The Obsessive Compulsive Inventory-Revised (OCI-R; Italian version by Sica et al. [67]) consists of 18 items assessing the distress caused by several OC symptoms in the past month; internal consistency in the Italian version was good. Given the purpose of the present study, the OCI-R total score was used. The Cronbach alpha value for the total score observed in the community sample was α = .84 (CI = [.82, .86], N = 127).

The Social Phobia Scale (SPS; Italian version by Sica et al. [68]) consists of 20 items assessing situations that involve being observed by other. The Italian version proved to be highly reliable and stable. Cronbach’s alpha in the community sample was α = .87 (CI = [.85, .89], N = 127).

Data analysis

Statistical analyses were performed with R [69] and the packages lavaan [70], semTools [71], and psych [72]. Confirmatory factor analyses (CFAs) were carried with the WLSMV robust estimator for ordinal data to test a unidimensional model, a correlated two-factor model, and a standard bifactor model accounting for both Prospective IU and Inhibitory IU group factors together with a general IU factor. As the bifactor model might be advantaged by traditional fit indexes in comparison to more parsimonious models [44–47], though it might not necessarily represent the true model behind the IU construct, significance of the fit indexes was not considered as the main criterion for model selection. Rather, considerations on model selection were driven by the results provided by the standard bifactor model as a tool to investigate the dimensionality of the test [48–51]. In addition, a modified version of the standard bifactor model was considered, in which the correlation between the group factors is freed (rather than constrained to be orthogonal under the standard model), in order to shed more light on the effects of a common factor in a correlated two-factor model. Considerations as to the factor structure of the IUS-R were then drawn based on a comparative discussion of these models. As to fit indexes, models were compared by means of chi-squared difference tests [73, 74] and a ΔCFI criterion (|ΔCFI| of < .01 suggests no significant difference) [75]. Fit indexes and tests were evaluated by the following criteria: χ2 should not be significant (although its dependence on sample size is largely acknowledged); Normed Chi square χ2/df < 2.0; Comparative Fit Index (CFI) and Tucker Lewis Index (TLI) > .95; Root Mean Square Error of Approximation (RMSEA) < .05; Standardized Root Mean square Residuals (SRMR) < .08; Weighted Root Mean square Residuals (WRMR) < 1.0 [76].

Similarly, considerations on the reliability of the IUS-R and its subscales IUS-R-P and IUS-R-I were given in terms of composite reliability rather than Cronbach’s α. In presence of broad (and possibly multidimensional) constructs with congeneric items, Cronbach’s alpha can underestimate reliability and is neither a measure of dimensionality nor of internal consistency [77]. For this reason, composite reliability coefficients have been developed to account for the different loadings of the items. In the present work, composite reliability was assessed by means of Raykov’s ρ [78], Bentler’s ω [79], and McDonald’s ωT and ωh [80]. Typically, values > .70 were considered evidence of good reliability. The first two coefficients can be seen as a ratio between the reliable variance and the total test variance. The main difference between Raykov’s ρ and Bentler’s ω is that the former accounts for uncorrelated measurement errors while the latter considers correlated errors. Both coefficients are based on the model implied covariance matrix to assess the total test variance. In contrast, Mc Donald’s ωh is based on the observed covariance matrix. The main difference between ωT and ωh, is that the former builds the reliable variance based on the loadings of all the factors while the latter only on the loadings of the general factor of the bifactor model. Of particular interest, ωh is unaffected by the number of items in the test.

In addition, Explained Common Variance (ECV > .60) was computed as the percent of common variance due to the general factor and as an indicator of unidimensionality [49]. In relation to ECV and ωh, the Percentage of Uncontaminated Correlations (PUC) was also controlled to verify whether any structural bias induced by forcing a multidimensional structure into a unidimensional scale could be disregarded so that practitioners are able to use the total score of the questionnaire as a univocal reliable measure of the common factor [51]. Generally, if PUC is higher than .80 then the other indexes are less relevant, otherwise they all need to be considered [51]. The Average Variance Extracted (AVE) was provided to compare variances due to construct and measurement error. Generally, values >.5 are considered acceptable and values >.7 are considered very good [81].

In the community sample, Pearson correlations were then calculated between the IUS-R total score with age and years of education; eta squared (η2) was used to quantify the magnitude of the effect. Following Cohen’s [82] criteria, .01 is considered small, .06 medium and .14 large. Correlations of the IUS-R total score and subscores with the other measures in the community sample were also computed to assess convergent validity. Partial correlations were calculated, controlling for the effect of the other constructs. Correlations were also calculated between all the measures and the predicted factor scores for the latent dimensions of the bifactor model in the community sample order to look for some evidence that the general common factor is not simply a method factor. Values of |.30| and |.50| were respectively considered moderate and large.

In addition, since the IUS-R was developed to be suitable across the lifespan and is also supposed to be gender invariant, Measurement Invariance (MI) with respect to gender and age was independently assessed using Multi-Group Confirmatory Factor Analysis (MGCFA) by inspecting the fit of 1) separate bifactor models for the groups (men and women, or different age groups); 2) configural invariance (loading and threshold parameters differ across groups); 3) metric invariance (only thresholds differ across groups); and 4) scalar invariance (loadings and thresholds cannot differ across groups). Since WLSMV was used, when testing scalar invariance a difference test for both factor loadings and thresholds simultaneously was applied, thus directly comparing configural and scalar models, as suggested by Muthén and Muthén [83].

Finally, temporal stability and reliability of the IUS-R in the undergraduate sample were assessed using Pearson’s correlations, Intra-Class Correlations (ICC), and longitudinal MI [84]. Bifactor models were fitted separately in the test and retest conditions, then configural, metric, and scalar invariance were assessed by fitting separate bifactor models to the two conditions, correlating all factors and all item residuals to account for their longitudinal dependence. MI between community and undergraduate samples was examined to test whether the results of the longitudinal MI on the undergraduate sample could be extended to the community sample.

Results

Descriptive statistics

All IUS-R items were approximately symmetric (|skewness|≤1), except for items 9, 11, and 12 that were skewed (|skewness|>1). Eight items showed negative excess kurtosis between 0 and -1, except item 3 (-1.05), and items 9, 11 and 12 which showed positive kurtosis (.65, 1.13, and 2.38 respectively). Mardia’s multivariate test indicated significant multivariate skewness and kurtosis.

Factor structure and reliability

As analyses were conducted on complete data, three participants were removed from the community sample (N = 758). Table 2 reports the fit indexes for a) the unidimensional model, b) the correlated two-factor model, c) the standard bifactor model, and d) a bifactor model in which the correlation between the group factors was freed, which is essentially model b) with the addition of a common factor. As can be seen in Table 2, the bifactor model c) showed the strongest fit indexes as expected when compared to the unidimensional and correlated two-factor models. Note that model d) could be considered the best fit, but this model was used only to investigate the relationships between the other models as will be discussed later. The unidimensional model showed the poorest robust fit indexes while all loadings were significantly different from zero and in the range [.371, .819], with robust standard errors in the range [.017, .031]. The correlated two-factor model showed moderate robust fit indexes; again, all loadings were also significantly different from zero and in the range [.404, .849], with robust standard errors in the range [.016, .032]. The estimated correlation between the Prospective IU and Inhibitory IU group factors in the bifactor model was .75 (SE = .022; z = 33.885, p < .001), which is slightly lower than the correlation between the predicted factor scores for the Prospective IU and Inhibitory IU group factors (r = .83, CI = [.808, .852], t(758) = 41.194, p < .001). Interestingly, both these correlations are higher than the observed correlation between the IUS-R-P and IUS-R-I subscale scores (r = .59, CI = [.545, .637], t(758) = 20.254, p < .001). This suggests a strong correlation between the latent factors in the correlated two-factor model, which possibly hints to the existence of a second order factor or of a general common factor as in the bifactor model. The standard bifactor model showed good robust fit indexes. As expected, results from the scaled χ2 difference test and the ΔCFI criterion indicated that the bifactor solution fitted the data significantly better than both the unidimensional and correlated two-factor models. Again, it is important to stress that the superior fit indexes of the bifactor model does not imply that IU is best described by such a model. Rather, the bifactor model can be used as a tool to examine the dimensionality and reliability of the scales.

thumbnail
Download:
Table 2. Fit statistics for the unidimensional, correlated two-factor, and bifactor models.

Scaled difference Δχ2 test and ΔCFI criterion are also reported.

https://doi.org/10.1371/journal.pone.0211929.t002

As can be seen in Table 3, the loadings on the general factor of the bifactor model were all well-estimated and in the .5 to .8 range, except for items 3 and 4 with values of .345 and .466 respectively which are still considered acceptable. Interestingly, although the loadings were slightly lower than those in previous reports [31, 43], their values were consistent (including the lower values for items 3 and 4) with those from the previous Italian sample [42]. Items 1 and 2 on the Prospective IU group factor were problematic as their loadings on the group factor had opposite signs to all other items on the same group factor. To note, when the correlation between the Prospective IU and Inhibitory IU group factors in the bifactor model was freed, most loadings of the general common factor decreased in magnitude, all loadings of the Inhibitory IU factor increased in magnitude, whereas almost all loadings of the Prospective IU group factor became negative (although some of them were not significantly different than zero). In addition, the correlation between the latent group factors was estimated as -.66 (SE = .066, z = -9.910, p < .001), which is of the same magnitude, but of the opposite sign than the correlation estimated in the correlated two-factor solution. This result suggests that, although both Prospective and Inhibitory group factors contribute in describing the IU construct, their specific group content might actually be of an opposite nature. For further details together with further discussion of the loadings in the present and previous studies [31, 42, 43], please see the discussion section.

thumbnail
Download:
Table 3. Standardized loadings for the standard bifactor model and the bifactor with correlated group factors.

For each loading, the associated robust standard error in round brackets, together with z- value and significance, are provided.

https://doi.org/10.1371/journal.pone.0211929.t003

Reliability indexes for the IUS-R general factor score as well as for the Prospective IU and Inhibitory IU group factors in the bifactor model, together with their respective correlations with the total score (unit-weighted scores), are reported in Table 4. As measures of unidimensionality we calculated the Percentage of Uncontaminated Correlations (PUC = .53, being lower than .80 is also necessary to control for the other indices), the Explained Common Variance (ECV = .70, good), and the Average Variance Extracted (AVE = .563, acceptable).

thumbnail
Download:
Table 4. Reliability of the Italian IUS-R (community sample).

Upper panel: Composite reliability coefficients for the general factor and group factors from the standard bifactor model and for the total score. Lower panel: Cronbach’s alpha for unrefined score and correlations with factor score.

https://doi.org/10.1371/journal.pone.0211929.t004

All estimates of composite reliability point towards a unidimensional scale and a reliable total score (mean IUS-R = 26.73, SD = 8.20). Together, ωT and ωh indicated that almost 87% of the reliable variance can be attributed to the general factor (obtained as the ratio between the omega for the general factor .781 and the omega for the total score .899 in Table 4) [50]. For comparison, the composite reliability calculated for the unidimensional model is Bentler’s ω = .88, and ωh = .91. In addition, both group factors may be unreliable, although the reliability for the Inhibitory IU group factor was higher than for the Prospective IU group factor. Cronbach’s α for the total scores based on the group factors might thus be inflated by the contribution of the general factor. While the general factor score shows a strong correlation with the associated IUS-R total score (see Table 4), the same does not hold for the group factors and the unrefined factor scores. Taken together these results indicate relative unidimensionality despite some multidimensionality and that the total score is a good enough indicator of the general factor; please see the next section and the discussion section for further comments on the subscales.

Associations with demographic variables and convergent validity of the IUS-R and of the predicted factor scores

Given that the previous analyses indicated acceptable unidimensionality (despite some underlying multidimensionality), correlations were examined for the IUS-R scale and its Prospective IU and Inhibitory IU subscales (see Table 5, upper panel). The same analyses were conducted for the predicted factor scores from the bifactor model (see Table 5, lower panel). The latter simultaneously aimed to assess whether the latent common factor correctly represented the IU construct rather than a method factor, and to show to what extent predicted Prospective IU and Inhibitory IU factor scores behaved like independent and reliable group factors. It should be noted that pairwise deletion of missing data was applied to the combined data set since, as previously mentioned, the community sample was obtained by merging four different studies with different batteries of measures. Readers should also be careful in interpreting the correlations for the same reason caution was advised in interpreting Cronbach’s alpha in the Measures section.

thumbnail
Download:
Table 5. Inter-correlations between the measures and the IUS-R scores (upper panel) or the IUS-R predicted factor scores (lower panel).

Sample size for each correlation is reported within round brackets. Non-available correlations are reported as NA.

https://doi.org/10.1371/journal.pone.0211929.t005

For the demographic variables, Pearson’s product-moment correlations of the IUS-R total score with age was significantly different than zero but shows a small effect (r = -.14, CI = [-.210, -.070], t(759) = -3.899, p < .001, η2 = .022). In the next section MI with respect to age is examined formally. Correlation with education was non-significant (r = -.002, CI = [-.070, .070], t(759) = -.060, p = .95). There was also no significant difference in the total score between genders (t(693.82) = -.28, p = .78, η2 = .0001). Correlations of the IUS-R and its subscales IUS-R-P and IUS-R-I with the other study measures in the community sample are reported in Table 5 (upper panel). As can be seen in the top row, the strongest correlation with external measures was found between the IUS-R and the NPOQ, immediately followed by medium correlations between the IUS-R and the OBQ-87 tolerance of uncertainty, PSWQ, SPS, and OCI-R, whereas correlations with BAI, BDI-II, and DASS-21 were weak.

Next, partial correlations were conducted on the largest possible subsample (N = 213) that simultaneously provided complete data for the OBQ-87 tolerance of uncertainty subscale, NPOQ, and PSWQ, i.e. the three measures most strongly correlated with the IUS-R. The partial correlations between the IUS-R and both the OBQ-87 tolerance of uncertainty (r = .26) and the NPOQ (r = .47) decreased in strength but remained significant (p < .001) after controlling for the other measures; the correlation between the IUS-R and PSWQ was no longer significant (r = .09).

Moving now to the factor scores, Table 5 (lower panel) reports the correlations of the predicted factor scores for the standard bifactor. As it can be seen, the predicted factor score of the general common factor (GEN) showed essentially the same strength of correlations with the other measures as the total score in Table 5 (upper panel) (which might be expected as from Table 4 their correlation is about .95). However, the predicted values of the factor scores for the Prospective IU (IUSP) and the Inhibitory IU (IUSI) group factors did not show convergent validity with the other external measures. This appears to support the essential unidimensionality of the construct as well as the unreliability of the subscales.

Measurement invariance of the bifactor model

Steps to verify the MI of the bifactor model in the community sample are summarized in Table 6 for the variable gender and in Table 7 for the (categorized) variable age.

thumbnail
Download:
Table 6. Fit statistics for the bifactor model tested for invariance across gender (community sample).

https://doi.org/10.1371/journal.pone.0211929.t006

thumbnail
Download:
Table 7. Fit statistics for the bifactor model tested for invariance across age groups (community sample).

https://doi.org/10.1371/journal.pone.0211929.t007

For the MI analysis for gender, item 11 in one group had an empty response category (5), so for this item categories 4 and 5 were merged. Separate CFAs showed good fit of the bifactor model for both genders. Configural invariance showed good robust fit indexes. Metric and scalar invariance showed slightly lower fit indexes but were still supported against configural invariance as the ΔCFI criterion was satisfied and normed chi-squares were close to 2. Interestingly, while there were no differences in latent means between genders for the general factor and the Inhibitory IU group factor, the Prospective IU group factor was significantly higher among men: 1.09 (SE = .144, z = 6.710, p < .001). A gender difference in the latent mean of the Prospective IU group factor provides further indication that it may be problematic. Importantly, the presence of full scalar invariance enables practitioners to make inferences on composite scores like test scores [85].

For the MI with respect to age, three age groups were defined: the first group contained people between 18–25 years of age, the second group between 26–45 years of age, and the last contained people 46 and older. Since in some groups the response category 5 was empty for items 10 and 11, categories 4 and 5 were merged for these items. As can be seen in Table 7, MI appears to be essentially satisfied up to the scalar level, hence there appears to be no substantial change in the factor structure of the IUS-R while describing IU across the adult lifespan. Please note that p = .088 for the last scaled chi square test of Table 7 is compatible with the presence of a small correlation between age and the IUS-R score.

Temporal stability and longitudinal measurement invariance

For the undergraduate sample, Pearson’s correlation coefficient between the total scores at one month was r = .74 (95% CI = [.665, .807], t(154) = 13.827, p < .001) and the ICC coefficient was .74 (95% CI = [.656, .801], F(155, 156) = 6.600, p < .001), suggesting good test-retest reliability. For MI (Table 8), since the response category 5 was empty for items 1, 2, 7 and 12 in at least one of the two temporal positions, categories 4 and 5 were merged for these items. The bifactor model was fitted to both test and retest conditions resulting in good robust fit indexes. Configural invariance also showed good fit indexes and the correlation between the general factor at the two administrations was .82 (SE = .028, z = 28.755, p < .001). Metric invariance showed good fit indexes as well, with a correlation of .81 (SE = .028, z = 29.428, p < .001). Scalar invariance showed robust fit indexes too, with again a correlation of .81 (SE = .028, z = 29.423, p < .001). There was a small decrease in the general latent factor mean of -.152 (SE = .068, z = -2.245, p = .025) between test and retest. Both metric and scalar invariances against configural invariance were supported.

thumbnail
Download:
Table 8. Fit statistics for the bifactor model tested for invariance across one-month test-retest (undergraduate sample).

https://doi.org/10.1371/journal.pone.0211929.t008

In order to extend the results of the previous MI to the community sample, an MI was conducted comparing the community and undergraduate samples. Since the response category 5 was empty for items 2 and 12 in the undergraduate sample, categories 4 and 5 were merged for these two items. Results of MI (Table 9) show that the bifactor structure is the same in both the samples.

thumbnail
Download:
Table 9. Fit statistics for the bifactor model tested for invariance across samples (community vs undergraduates).

https://doi.org/10.1371/journal.pone.0211929.t009

Discussion

IU is a trans-diagnostic factor that has been implicated in a wide range of psychological disorders and the utility of the IUS-12 in trans-diagnostic research and clinical practice is acknowledged. Walker et al. [53] adapted the IUS-12 to measure IU across the lifespan; importantly, their version of the test, i.e. the IUS-R, has proven to be a reliable and valid measure of IU in different samples [53–57], as well as in Italian undergraduates [58]. Beyond providing increasing evidence about the adequacy of the Italian IUS-R as a measure of IU, the current study addressed a number of yet unexplored or only partially explored issues which we will address in turn. Overall, the main results are in line with the expected results given in previous studies and further expand knowledge about the assessment of IU.

With respect to the factor structure of the questionnaire, results from the CFAs on the community sample suggested that the correlated two-factor model showed adequate fit indexes and might be a reasonable representation of the IU construct. However, the strength of the estimated correlation between the latent Prospective IU and Inhibitory IU domains (or between the predicted factor scores) suggested that there might be an unexpectedly high shared variance. Indeed, the application of a bifactor model as a tool to investigate the dimensionality of the scale and the strength of the group factors suggested that the Italian IUS-R is essentially a unidimensional scale. Such a result is supported by the presence of a strong common general factor in the bifactor model, and by the evaluation of composite reliability based on the bifactor solution. The general IU factor accounted for the 87% of variance in the IUS-R scores, consistent with, but slightly higher than the findings by Hale et al. (80%), Lauriola et al. (75%), and Shihata et al. (80% among undergraduates and 86% among clinical participants) [31, 42, 43]. Thus, there is increasing evidence across studies indicating that the total score is a good indicator of the general factor. Inspection of the standardized loadings across these three studies and the current study lead to some interesting similarities and differences.

First, a strong general factor was found in all samples. Loadings were generally comparable to those found in previous studies, with the only exception of items 3 and 4, which were lower in both Italian studies when compared to those in the studies by Hale et al. [31] and Shihata et al [43]. Interestingly, the content of these items explicitly refers to beliefs about the utility of planning under uncertain circumstances. Economics and business literature classifies Italy among the “strong uncertainty avoidance cultures” [86, 87]. Further, recent literature indicates that Italian people are likely to attribute a negative meaning to uncertainty and to be less confident about their own ability to solve problems and to manage uncertain events [88]. Consequently, Italian respondents may not fully endorse items about planning even when they are generally high in IU, perhaps explaining why lower loadings for these items have been observed in both the Italian studies. This supports the notion that cross-cultural differences may occur in the interpretation of (intolerance of) uncertainty.

Second, the Prospective IU domain has proved to be unreliable across all studies employing non clinical participants (in the clinical sample enrolled by Shihata et al. [43], the Prospective IU group factor even had to be removed in order to obtain an admissible bifactor model, consisting of a general IU factor and one group factor, inhibitory IU). The standardized loadings observed in the present study support the notion of a slightly stronger group factor than those reported by the other three studies [31, 42, 43] in undergraduate samples. The community sample and the different item phrasing of the IUS-R could have contributed to this result. However, none of the studies have retrieved robust group factors. The present study confirmed that some items on the group factor Prospective IU are particularly problematic. In particular, items 1 and 2 showed negative loadings, thus loading in the opposite directions to the other items on the factor. Although these items are coherent with the general factor representing IU, the part that does not load on the general factor is not coherent with the group factor. Negative loadings for item 1 were also observed also by Hale et al. [31] and Lauriola et al [42] but not Shihata et al. [43]. Both items refer to experiencing distress (i.e. feeling upset, feeling frustrated) in the face of uncertainty rather than engaging in activities that increase certainty/reduce uncertainty as indicated by the other items. As a consequence, it may be worth considering whether these items should be considered part of the Prospective IU domain in future research, since they clearly do not refer to the propensity to actively seek information to reduce uncertainty, although they may be a valid part of the general factor. Interestingly, item 2 was the only item showing a very high loading (.94) in the undergraduate sample reported by Shihata et al. [43], whereas other items showed extremely low loadings, ranging from -.03 to .18, indicating that in their sample the group factor may be more characterized by upset rather than information seeking.

Finally, when a correlation between Prospective IU and Inhibitory IU was permitted within the bifactor model, most of the items in the Prospective IU domain had negative loadings (which also resulted in a negative correlation between the latent group factors). Such a result suggests that, notwithstanding the fact that both Prospective IU and Inhibitory IU contribute to the IU construct, their specific content goes in opposite directions. This finding may be reasonable, given the original definition of the two dimensions: Prospective IU refers indeed to a strong desire for predictability, which drives individuals to collect as much information as possible in order to increase certainty/reduce uncertainty. In opposition, Inhibitory IU reflects “behavioural paralysis”, which might be intended as a sense of being stuck and unable to respond in the face of uncertainty [24, 30]. In other words, these two components represent conceptually opposite coping strategies employed to manage uncertainty, the former relying on active engagement and the latter on under-engagement [24].

Taken together, the results from this study indicate that the Prospective IU group factor is extremely unreliable and may collect unusual item behavior and distortions. Any further revision of the assessment of IU may require a more theory driven approach to adequately assess any group factors that may underlie the overall construct. In conclusion, our results support the presence of a strong and reliable general factor underlying the IUS-R items and provide good evidence of unidimensionality thus highlighting that the IUS-R total score is a reliable representation of the IU construct. Such a finding is conceptually sound and in line with the trans-diagnostic nature of IU, since unidimensional constructs are more likely than multidimensional ones to be invariant in terms of both form and function across individuals with different clinical phenotypes [31]. Some research has suggested that Prospective and Inhibitory IU may have differential links with specific disorders. For example, Prospective IU might drive several dysfunctional approach behaviors typical of GAD (i.e., worry) and OCD (i.e., compulsions), which are performed to reduce uncertainty/increase certainty; on the opposite, inhibitory IU would be more involved in disorders relying on avoidance behaviors to reduce exposure to uncertainty, as for example panic disorder, social anxiety, and depression [33, 34, 40]. However, findings from other studies [55, 89, 90] found that the two IU components were similarly associated with different symptom dimensions, consistent with the notion that people with high levels of IU might endorse both approach- and avoidance-oriented coping strategies to manage uncertainty and related distress but perhaps use them differentially in different situations. Consequently, the general trait IU is likely to have higher trans-diagnostic predictive utility than Prospective and Inhibitory IU separately [43].

As all the previous results are based on the application of a bifactor model, it should be noted that concern has been raised on its interpretation as structural and substantive models to describe scales [45]. Indeed, it has been argued that they might not be good representations of psychological phenomena, and that they might show superior fit performances only because they overfit data or are favored by fit statistics [46, 91]. In particular, some (but not all) of their better fit might derive from an ability to perform as “garbage collectors” for implausible response patterns, even though the data for most of the sample might actually be modeled with a unidimensional structure [44]. Although it appears that in general this issue should not arise between bifactor and correlated-factor models, but rather between bifactor and higher-order models [47], it is cautionary to not interpret them as substantive models. Notwithstanding these issues, they may still be good candidates to analyze the psychometric properties of test scores and of subscales based on group factors [48, 50]. In particular, they are helpful to investigate the reliability of the total score as a “univocal indicator of the latent variable, despite of the multidimensionality” ([49], p.232).

In the current study the bifactor model was used with this rationale, and it can be argued that the current version of the IUS-R has a reliable total score based on a unidimensional general factor, but that neither group factor achieves acceptable reliability. Practitioners who want to use the Inhibitory IU group factor as a separate scale (some group loadings were reasonably high thus indicating some degree of reliability) should consider that only about the 40% of the variance on the Inhibitory IU group factor is actually factor specific. The Prospective IU factor appears to be ill-conditioned (either by some items or by a theoretical concept that goes in the opposite direction of Inhibitory IU factor), hence it might not be advisable to use it as an independent subscale. Further refinements of the items or of the theoretical construct might enable the construction of a more reliable Prospective IU subscale. In such a case it would be reasonable that the correlated two-factor model (or a second order model) could be the best representation of the IU construct. Since, as previously outlined, Prospective and Inhibitory IU subscales might have differential relationships with clinically relevant constructs and specific measures of psychopathology [92], further analyses or indeed revision are needed to develop their reliabilities and relationships to the general construct. However, if the strong correlation between these two subscales is only a representation of their common relation with IU, it might be difficult to consistently find differential relationships with third variables. As an example, predicted factor scores on the two specific scales showed weak or non-significant correlations with other variables.

Analysis of correlations between the IUS-R total score with age and years of education provided results in line with our hypotheses. There were no associations between IUS-R total score and education, and only a weak correlation with age was observed. This also supports previous findings indicating that the IUS-R can be used with people of different ages and educational levels [53, 54]. The reasons for the decrease with age (as well as the non-significant trend in the associated MI analyses) are potentially an artefact, but may also be developmental as people gain experience and life unfolds with age; either uncertainty decreases, people’s intolerance of it does, or indeed both. Overall, demographic characteristics do not seem to affect the scores of the Italian IUS-R, which can be therefore administered in a wide range of samples.

As to convergent validity, current findings support both the established association between IU and GAD-related features (i.e., NPO and worry) [93] as well as the putative trans-diagnostic nature of IU where relationships with a variety of emotional disorders (such as GAD, social anxiety, panic disorder, depression, OCD) have been documented [18–20]. Medium-high correlations with NPOQ, OBQ-87 tolerance of uncertainty subscale, PSWQ, and SPS were observed, whereas correlations with OCI-R, BAI, BDI-II, and DASS-21 were weaker, though significant. Similar magnitudes are reported in the literature assessing associations between IU and these constructs in non-clinical samples, despite different studies using different questionnaires [26, 39, 40]. Furthermore, the strong association with the OBQ-87 tolerance of uncertainty subscale suggests that the IUS-R has adequate convergent validity [35]. Partial correlations between the IUS-R and the OBQ-87 tolerance of uncertainty subscale remained significant after controlling for the NPOQ and the PSWQ; the same occurred between the IUS-R and the NPOQ, whereas the association between the IUS-R and the PSWQ was no longer significant after controlling for the other measures. The particularly strong association between IU and NPO is in line with recent evidence suggesting that NPO might be better conceptualized as a facet of IU rather than as a distinct construct [93].

In addition to the previous considerations on the factor structure, we considered gender and age effects through MGCFA to investigate MI. In particular, scalar invariance of the bifactor model across gender was supported, as previously observed by Hale et al. [31]. In addition, MI was shown across three age groups. As a consequence, the IUS-R total score can be used to make inferences across genders, supplementing and extending findings from undergraduate samples [39, 58], and across the adult lifespan, thus supporting the rationale for which the IUS-R was developed [53]. Future research should consider further expanding these findings to children and adolescents. However, those wishing to use the subscales should be aware that Prospective IU might be heavily unreliable and that the associated total score might not be a reliable measure of the latent factor but only of the IU construct.

Finally, good one-month test-retest reliability in the undergraduate sample was observed in line with previous findings reported by Khawaja and Yu [36] on a similar group and consistent with the notion of IU as a dispositional, relatively stable feature [1, 16, 94]. To the authors’ knowledge, the current study is the first attempt to assess MI of the IU in a longitudinal study; the stability of the bifactor structure over time with a strong general factor was confirmed, further supporting the reliability of the IUS-R. In addition, this result can to a certain degree be extended to the community sample since MI of the bifactor structure was showed to hold when comparing community and undergraduate samples.

As to the main limitations of study, there are several points that require comment. First, the size of the undergraduate sample was relatively small and may not lead to stable solutions. Nevertheless, it should be noted that although the ratio of sample size to number of parameters [95] is not generally considered adequate to estimate the bifactor model and MI, it could be argued that with loadings of .5 or above and at least 8 indicators (requirements met here) the minimum required size is of about 160 observations [96]. Of course, results must be interpreted with caution and further studies are required to provide more reliable estimates. In addition, despite the evidence that the bifactor structure is the same in both undergraduates and community individuals, we did not directly test longitudinal MI in the community sample. Therefore, future studies investigating the temporal stability of the IUS-R in a non-student sample would be encouraged. Second, since the community sample was obtained by merging different samples collected in different waves, some of them did not complete all questionnaires which may have affected the results on convergent validity as they represent different subsamples or varying sizes. Third, despite the correlational data suggesting that the IUS-R scores are essentially insensitive to differences in educational level, MI across educational levels groups was not formally tested. Fourth, a measure of standard personality traits was not administered; similarly, we did not include measures assessing constructs theoretically unrelated to IU (e.g., sensation-seeking). Including such measures would have better situated our findings within a broader context as well as bolstered evidence for discriminant validity; consequently, the inclusion of such measures in future studies is desirable. Fifth, the MI of the Italian IUS-R factor model across non-clinical and clinical samples was not examined. This is an important issue, as results from community individuals may not generalize to patients with emotional disorders, although a very recent study reported that a bifactor model with a robust general factor best described IU in treatment-seeking individuals [43]. Finally, this was a correlational study with other self-report measures. Studies that use other indicators such as behavioral correlates of IU or real time responses to uncertainty would provide stronger evidence for the validity of the IUS-R. Moreover, the responsiveness of the IUS-R was not tested. We believe that data demonstrating sensitivity to change are required by the presence of evidences showing that IU is a trans-diagnostic and “trans-therapy” change process [22], and so changes in IU are correlated with or mediate treatment outcome (i.e. symptom reduction) in psychological interventions for emotional disorders [22, 97].

Despite these limitations, the coherence of the results suggests that the use of the Italian IUS-R in both research and clinical practice is warranted and suitable for people in a range of settings. Furthermore, the recommended use of the total score is fully consistent with recent literature and provides further support to the notion of IU as a trans-diagnostic factor [31]. Last but not least, the current findings suggest that future research on the measurement of IU through the IUS-12 and the IUS-R could focus on the refinement of item content from a conceptual standpoint, and on item phrasing, also taking into account the potential role played by cross-cultural differences in the interpretation and meaning of uncertainty. As a unidimensional scale, the Italian IUS-R seems fit for purpose but the Prospective IU domain (in particular) appears to require further work.

References

  1. 1. Carleton RN. Fear of the unknown: One fear to rule them all? J Anxiety Disord. 2016; 41: 5–21. pmid:27067453
  2. 2. Buhr K, & Dugas MJ. The Intolerance of Uncertainty Scale: Psychometric properties of the English version. Behav Res Ther. 2002; 40: 931–945. pmid:12186356
  3. 3. Carleton RN, Mulvogue MK, Thibodeau MA, McCabe RE., Antony MM, Asmundson GJG. Increasingly certain about uncertainty: Intolerance of uncertainty across anxiety and depression. J Anxiety Disord. 2012; 26: 468–479. pmid:22366534
  4. 4. Dugas MJ, Schwartz A,Francis K. Intolerance of uncertainty, worry, and depression. Cognit Ther Res. 2004; 28: 835–842.
  5. 5. Cougle JR, Timpano KR, Goetz AR. Exploring unique and interactive roles of distress tolerance and negative urgency in obsessions. Pers Individ Dif. 2012; 52: 515–520.
  6. 6. Pawluk EJ, Koerner N. A preliminary investigation of impulsivity in generalized anxiety disorder. Pers Individ Dif. 2013; 54: 732–737.
  7. 7. Frenkel-Brunswik E. Intolerance of ambiguity as an emotional and perceptual personality variable. J Pers. 1949; 18: 108–143.
  8. 8. Simons JS, Gaher RM. The distress tolerance scale: development andvalidation of a self-report measure. Motiv Emot. 2005; 29: 83–102.
  9. 9. Flehmig HC, Steinborn M, Langner R, Westhoff K. Neuroticism and the mental noise hypothesis: Relation to lapses of attention and slips of action in everyday life. Psychol Sci. 2007; 49: 343–360.
  10. 10. Kuhl J. A theory of self-regulation: Action versus state orientation, self-discrimination, and some applications. Appl Psychol. 1992; 41: 97–129.
  11. 11. Frost RO, Shows DL. The nature and measurement of compulsive indecisiveness. Behav Res Ther. 1993; 31: 683–692. pmid:8216169
  12. 12. Webster DM, Kruglanski AW. Individual differences in need for cognitive closure. J Pers Soc Psychol. 1994; 67: 1049–1062. pmid:7815301
  13. 13. Grenier S, Barrette AM, Ladouceur R. Intolerance of uncertainty and intolerance of ambiguity: Similarities and differences. Pers Individ Dif. 2005; 39: 593–600.
  14. 14. Rosen NO, Ivanova E, Knäuper B. Differentiating intolerance of uncertainty from three related but distinct constructs. Anxiety Stress Coping; 2014; 27: 55–73. pmid:23849047
  15. 15. Koerner N, Mejia T, Kusec A. What’s in a name? Intolerance of uncertainty, other uncertainty-relevant constructs, and their differential relations to worry and generalized anxiety disorder. Cogn Behav Ther. 2017; 46: 141–161. pmid:27684470
  16. 16. Dugas MJ, Gagnon F, Ladouceur R, Freeston MH. Generalized anxiety disorder: a preliminary test of a conceptual model. Behav Res Ther. 1998; 36: 215–226. pmid:9613027
  17. 17. Freeston MH, Rhéaume J, Letarte H, Dugas MJ, Ladouceur R. Why do people worry? Pers Individ Dif. 1994; 17: 791–802.
  18. 18. Carleton RN. Into the unknown: A review and synthesis of contemporary models involving uncertainty. J Anxiety Disord. 2016; 39: 30–43. pmid:26945765
  19. 19. Hong RY, Cheung MWL. The Structure of Cognitive Vulnerabilities to Depression and Anxiety: Evidence for a Common Core Etiologic Process Based on a Meta-Analytic Review. Clin Psychol Sci. 2015; 3: 892–912.
  20. 20. Shihata S, McEvoy PM, Mullan BA, Carleton RN. Intolerance of uncertainty in emotional disorders: What uncertainties remain? J Anxiety Disord. 2016; 41: 115–124. pmid:27212227
  21. 21. Einstein DA. Extension of the transdiagnostic model to focus on intolerance of uncertainty: a review of the literature and implications for treatment. Clin Psychol (New York). 2014; 21: 280–300. pmid:25400336
  22. 22. McEvoy PM, Erceg-Hurn DM. The search for universal transdiagnostic and trans-therapy change processes: Evidence for intolerance of uncertainty. J Anxiety Disord. 2016; 41: 96–107. pmid:26898177
  23. 23. Norton PJ, Paulus DJ. Toward a unified treatment for emotional disorders: update on the science and practice. Behav Ther. 2016; 47: 854–868. pmid:27993337
  24. 24. Birrell J, Meares K, Wilkinson A, Freeston M. Toward a definition of intolerance of uncertainty: A review of factor analytical studies of the Intolerance of Uncertainty Scale. Clin Psychol Rev. 2011; 31: 1198–1208. pmid:21871853
  25. 25. De Bruin GO, Rassin E, van der Heiden C, Muris P. Psychometric properties of a Dutch version of the intolerance of uncertainty scale. Neth J Psychol. 2006; 62: 91–97.
  26. 26. Carleton RN, Norton PJ, Asmundson GJG. Fearing the unknown: A short version of the Intolerance of Uncertainty Scale. J Anxiety Disord. 2007; 21: 105–117. pmid:16647833
  27. 27. González Rodriguez M, Leon RC, Rovella AT, Herrera MD. Adaptacion espanola de la escala de intolerancia hacia la incertidumbre: procesos cognitivos, ansiedad y depresion. Psicologia y Salud. 2006; 16: 219–233. Spanish.
  28. 28. Sexton KA, Dugas MJ. Defining distinct negative beliefs about uncertainty: Validating the factor structure of the intolerance of uncertainty scale. Psychol Assess. 2009; 21: 176–186. pmid:19485672
  29. 29. Gerlach AL, Andor T, Patzelt J. The significance of intolerance of uncertainty in generalized anxiety disorder: Possible models and development of a German version of the intolerance of uncertainty scale. Z Klin Psychol Psychother. 2008; 37: 190–199. German.
  30. 30. Berenbaum H, Bredemeier K, Thompson RJ. Intolerance of uncertainty: Exploring its dimensionality and associations with need for cognitive closure, psychopathology, and personality. J Anxiety Disord. 2008; 22: 117–125. pmid:17349774
  31. 31. Hale W, Richmond M, Bennett J, Berzins T, Fields A, Weber D, et al. Resolving Uncertainty About the Intolerance of Uncertainty Scale–12: Application of Modern Psychometric Strategies. J Pers Assess. 2016; 98: 200–208. pmid:26542301
  32. 32. Norton PJ. A psychometric analysis of the Intolerance of Uncertainty Scale among four racial groups. J Anxiety Disord. 2005; 6: 699–707. pmid:15927782
  33. 33. Gentes EL, Ruscio AM. A meta-analysis of the relation of intolerance of uncertainty to symptoms of generalized anxiety disorder, major depressive disorder, and obsessive-compulsive disorder. Clin Psychol Rev. 2011; 31: 923–933. pmid:21664339
  34. 34. McEvoy P, Mahoney AE. Achieving certainty about the structure of intolerance of uncertainty in a treatment-seeking sample with anxiety and depression. J Anxiety Disord. 2011; 25: 112–122. pmid:20828984
  35. 35. Jacoby R, Fabricant L, Leonard R, Riemann B, Abramowitz J. Just to be certain: Confirming the factor structure of the Intolerance of Uncertainty Scale in patients with obsessive–compulsive disorder. J Anxiety Disord. 2013; 27: 535–543. pmid:23973743
  36. 36. Khawaja N, Yu LNH. A comparison of the 27-item and 12-item intolerance of uncertainty scales. Clin Psychol (Aust Psychol Soc). 2010; 14: 97–106.
  37. 37. McEvoy PM, Mahoney EAJ. To be sure, to be sure: intolerance of uncertainty mediates symptoms of various anxiety disorders and depression. Behav Ther. 2012; 43: 533–545. pmid:22697442
  38. 38. Fergus T, Wu K. The Intolerance of Uncertainty Scale: Measurement invariance, population heterogeneity, and its relation with worry among self-identifying white and black respondents. Assessment. 2012; 20: 555–564. pmid:22984142
  39. 39. Helsen K, Van den Bussche E, Vlaeyen J, Goubert L. Confirmatory factor analysis of the Dutch Intolerance of Uncertainty Scale: Comparison of the full and short version. J Behav Ther Exp Psychiatry. 2013; 44: 21–29. pmid:22842528
  40. 40. Carleton RN, Collimore KC, Asmundson GJG. “It’s not just the judgements—It’s that I don’t know”: intolerance of uncertainty as a predictor of social anxiety. J Anxiety Disord. 2010; 24: 189–195. pmid:19931391
  41. 41. Obsessive Compulsive Cognitions Working Group. Psychometric validation of the Obsessive Belief Questionnaire and Interpretation of Intrusions Inventory–part 2: factor analyses and testing of a brief version. Behav Res Ther. 2005; 43: 1527–1542. pmid:16299894
  42. 42. Lauriola M, Mosca O, Carleton RN. Hierarchical factor structure of the Intolerance of Uncertainty Scale short form (IUS-12) in the Italian version. TPM Test Psychom Methodol Appl Psychol. 2016; 23: 377–394.
  43. 43. Shihata S, McEvoy PM, Mullan BA. A bifactor model of intolerance of uncertainty in undergraduate and clinical samples: Do we need to reconsider the two-factor model? Psychol Assess. 2018; 30: 893–903. http://dx.doi.org/10.1037/pas0000540. pmid:29553761
  44. 44. Reise SP, Kim DS, Mansolf M, Widaman KF. Is the Bifactor Model a Better Model or Is It Just Better at Modeling Implausible Responses? Application of Iteratively Reweighted Least Squares to the Rosenberg Self-Esteem Scale. Multivariate Behav Res. 2016; 51:818–838. http://dx.doi.org/10.1080/00273171.2016.1243461. pmid:27834509
  45. 45. Bonifay W, Lane SP, Reise SP. Three concerns with applying a bifactor model as a structure of psychopathology. Clin Psychol Sci. 2017; 5: 184–186.
  46. 46. Murray AL, Johnson W. The limitations of model fit in comparing the bi-factor versus higher-order models of human cognitive ability structure. Intelligence 2013; 41: 407–422. https://doi.org/10.1016/j.intell.2013.06.004.
  47. 47. Morgan GB, Hodge KJ, Wells KE, Watkins MW. Are fit indices biased in favor of bifactor models in cognitive ability research? A comparison of fit in correlated factors, higher-order, and bifactor models via Monte Carlo simulations. J Intell. 2015; 3: 2–20.
  48. 48. Reise SP, Moore TM, Haviland MG. Bifactor Models and Rotations: Exploring the Extent to which Multidimensional Data Yield Univocal Scale Scores. J Pers Assess. 2010; 92: 544–559. pmid:20954056
  49. 49. Rodriguez A, Reise SP, Haviland MG. Evaluating bifactor models: Calculating and interpreting statistical indices. Psychol Methods. 2016; 21: 137. pmid:26523435
  50. 50. Reise SP, Bonifay WE, Haviland MG. Scoring and modeling psychological measures in the presence of multidimensionality. J Pers Assess. 2013a; 95: 129–140. pmid:23030794
  51. 51. Reise SP, Scheines R, Widaman KF, Haviland MG. Multidimensionality and structural coefficient bias in structural equation modeling: A bifactor perspective. Educ Psychol Meas. 2013b; 73: 5–26.
  52. 52. McEvoy PM, Salmon K, Hyett MP, Jose PE, Gutenbrunner C, Bryson K, et al. Repetitive Negative Thinking as a Transdiagnostic Predictor of Depression and Anxiety Symptoms in Adolescents. Assessment. 2017; 1073191117693923.
  53. 53. Walker S, Birrell, J., L. Rogers, J. Leekam, S., and Freeston, M. H. Intolerance of Uncertainty Scale—Revised. 2010; Forthcoming.
    • 54. Bottesi G, Freeston MH. An examination of construct of Intolerance of Uncertainty with groups of typically developing young people of different ages. In: Freeston MH (Convenor). Intolerance of Uncertainty, Worry, and Generalized Anxiety Disorder among young people. Symposium conducted at the 40th British Association for Behavioural and Cognitive Psychotherapies (BABCP) Annual Conference; 2012, June 26–29. Leeds, United Kingdom. 2012; pp.54.
      • 55. Wright KD, Lebell MAA, Carleton RN. Intolerance of uncertainty, anxiety sensitivity, health anxiety, and anxiety disorder symptoms in youth. J Anxiety Disord. 2016; 41: 35–42. pmid:27302203
      • 56. Boulter C, Freeston M, South M, Rodgers J. Intolerance of uncertainty as a framework for understanding anxiety in children and adolescents with autism spectrum disorders. J Autism Dev Disord. 2014; 44: 1391–1402. pmid:24272526
      • 57. Joyce C, Honey E, Leekam SR., Barrett SL, Rodgers J. Anxiety, Intolerance of Uncertainty and Restricted and Repetitive Behaviour: Insights Directly from Young People with ASD. J Autism Dev Disord. 2017; 1–14.
      • 58. Bottesi G, Ghisi M, Novara C, Bertocchi J, Boido M, De Dominicis I, et al. Intolerance of Uncertainty Scale (IUS-27 e IUS-12): due studi preliminari. Psicot. Cogn. Comport. 2015; 21: 345–365. Italian.
      • 59. Cronbach LJ. Essentials of psychological testing. New York: Harper & Row, 1975.
        • 60. Steinborn MB, Langner R, Flehmig H.C, Huestegge L. Methodology of performance scoring in the d2 sustained-attention test: Cumulative-reliability functions and practical guidelines. Psychol Assess. 2018; 30: 339–357. pmid:28406669
        • 61. Sica C, Coradeschi D, Sanavio E, Dorz S, Manchisi D, Novara C. A study of the psychometric properties of the obsessive beliefs inventory and interpretations of intrusions inventory on clinical Italian individuals. J Anxiety Disord. 2004; 18: 291–307. pmid:15125978
        • 62. Bottesi G, Ghisi M. La valutazione dell’orientamento negativo al problema: validazione italiana del Negative Problem Orientation Questionnaire. Psicot. Cogn. Comport. 2017; 23: 275–288. Italian.
        • 63. Morani S, Pricci D, Sanavio E. Penn State Worry Questionnaire e Worry Domains Questionnaire. Presentazione delle versioni italiane ed analisi della fedeltà. Psicot. Cogn. Comport. 1999; 5: 195–209. Italian.
        • 64. Sica C, Coradeschi D, Ghisi M, Sanavio E. Beck Anxiety Inventory–BAI. Manuale. Firenze: Organizzazioni Speciali; 2006.
          • 65. Ghisi M, Flebus GB, Montano A, Sanavio E, Sica C. Beck depression inventory-II: manuale. Firenze: Organizzazioni Speciali; 2006.
            • 66. Bottesi G, Ghisi M, Altoè G, Conforti E, Melli G, Sica C. The Italian version of the Depression Anxiety Stress Scales-21: Factor structure and psychometric properties on community and clinical samples. Compr Psychiatry. 2015; 60: 170–181. pmid:25933937
            • 67. Sica C, Ghisi M, Altoè G, Chiri LR, Franceschini S, Coradeschi D, et al. The Italian version of the Obsessive Compulsive Inventory: Its psychometric properties on community and clinical samples. J Anxiety Disord. 2009; 23: 204–211. pmid:18701254
            • 68. Sica C, Musoni I, Chiri LR, Bisi B, Lolli V, Sighinolfi C. Social Phobia Scale (SPS) and Social Interaction Anxiety Scale (SIAS): Italian translation and adaptation. B Psicol Appl. 2007; 252: 59–71.
            • 69. R Core Team (2018). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/.
              • 70. Rosseel Y. (2012). Lavaan: An R Package for Structural Equation Modeling. J Stat Softw, 48, 1–36. Avalaible from: http://www.jstatsoft.org/v48/i02/.
              • 71. semTools Contributors. (2016). semTools: Useful tools for structural equation modeling. R package version 0.4–14. Retrieved from https://CRAN.R-project.org/package=semTools
                • 72. Revelle W. (2016). Psych: Procedures for Personality and Psychological Research. Northwestern University, Evanston, Illinois, USA. https://CRAN.R-project.org/package=psych Version = 1.6.12.
                  • 73. Satorra A, Bentler PM. A scaled difference chi-square test statistic for moment structure analysis. Psychometrika. 2001; 66: 507–514.
                  • 74. Satorra A, Bentler PM. Ensuring postiveness of the scaled difference chi-square test statistic. Psychometrika. 2010; 75: 243–248. pmid:20640194
                  • 75. Cheung GW, Rensvold RB. Evaluating goodness-of-fit indexes for testing measurement invariance. Struct Equ Modeling. 2002; 9: 233–255.
                  • 76. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling. 1999; 6: 1–55.
                  • 77. Sijtsma K. On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika. 2009; 74:107–120. pmid:20037639
                  • 78. Raykov T. Estimation of congeneric scale reliability using covariance structure analysis with nonlinear constraints. Br J Math Stat Psychol. 2001; 54: 315–323. pmid:11817096
                  • 79. Bentler PM. Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika. 2009; 74: 137–143. pmid:20161430
                  • 80. McDonald RP. Test theory: A unified treatment. Mahwah, NJ: Erlbaum; 1999.
                    • 81. Fornell C, Larker DF. Evaluating structural equation models with unobservable variables and measurement error. J Mark Res. 1981; 18: 39–50.
                    • 82. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale NJ: Erlbaum; 1988.
                      • 83. Muthén LK, Muthén BO. Mplus User’s Guide. 6th ed. Los Angeles (CA): Author; 1998–2010. https://www.statmodel.com
                        • 84. Widaman KF, Ferrer E, Conger RD. Factorial invariance within longitudinal structural equation models: Measuring the same construct across time. Child Dev Perspect. 2010; 4: 10–18. pmid:20369028
                        • 85. Steinmetz H. Analyzing observed composite differences across groups: Is partial measurement invariance enough? Methodology 2013; 9: 1–12.
                        • 86. Stremersch S, Tellis GJ. Understanding and managing international growth of new products. Int J Res Mark. 2004; 21: 421–438.
                        • 87. Wennekers S, Thurik R, van Stel A, Noorderhaven N. Uncertainty avoidance and the rate of business ownership across 21 OECD countries, 1976–2004. J Evol Econ. 2007; 17: 133–160.
                        • 88. Bottesi G, Ghisi M, Carraro E, Barclay N, Payne R, Freeston MH. Revising the Intolerance of Uncertainty Model of GAD: Evidence from British and Italian undergraduate samples. Front Psychol. 2016; 7: 1723.
                        • 89. Fourtounas A, Thomas SJ. Cognitive factors predicting checking, procrastination and other maladaptive behaviours: prospective versus inhibitory intolerance of uncertainty. J Obsessive Compuls Relat Disord 2016; 9:30–5.
                        • 90. O’Bryan EM, McLeish AC. An Examination of the Indirect Effect of Intolerance of Uncertainty on Health Anxiety Through Anxiety Sensitivity Physical Concerns. J Psychopathol Behav Assess. 2017; 39: 715–722.
                        • 91. Bonifay W, Cai L. On the Complexity of Item Response Theory Models. Multivariate Behav Res. 2017; 52: 465–484. pmid:28426237
                        • 92. Hong RY, Lee SS. Further Clarifying Prospective and Inhibitory Intolerance of Uncertainty: Factorial and Construct Validity of Test Scores from the Intolerance of Uncertainty Scale. Psychol Assess. 2015; 27: 605–620. pmid:25602690
                        • 93. Clarke JB., Ford M, Heary S, Rodgers J, Freeston MH. The Relationship Between Negative Problem Orientation and Worry: A Meta-Analytic Review. Psychopathol Rev. 2017; 4: 319–340.
                        • 94. Koerner N, Dugas MJ. An investigation of appraisals in individuals vulnerable to excessive worry: the role of intolerance of uncertainty. Cognit Ther Res. 2008; 32: 619–638.
                        • 95. Kline RB. Principles and practice of structural equation modeling. 3rd ed. New York, NY: Guilford; 2011.
                          • 96. Wolf EJ, Harrington KM, Clark SL, Miller MW. Sample size requirements for Structural Equations models: An evaluation of Power, Bias, and Solution Propriety. Educ Psychol Meas. 2013; 76: 913–934. pmid:25705052
                          • 97. Boswell JF, Thompson-Hollands J, Farchione TJ, Barlow DH. Intolerance of uncertainty: A common factor in the treatment of emotional disorders. J Clin Psychol. 2013; 69: 630–645. pmid:23381685