Internal and external validity of the brief version of the Multidimensional Personality Questionnaire: Exploratory structural equation modelling

The present study used exploratory structural equation modelling (ESEM) to examine the theorized dimension structure of the brief version of the Multidimensional Personality Questionnaire (MPQ-BR) at the scale-level (i.e., 11 lower-order primary factors loading on four higher-order factors) and item-level (sets of 12 items loading on 11 lower-order primary factors). A total of 214 adults from the community addressed the MPQ-BR and the Behavioral Inhibition System (BIS)/Behavioral Approach System (BAS) scales. The findings revealed poor fit and poorly defined factors at the item-level alongside adequate fit and well-defined factors at the scale-level. The higher-order factors in the latter model were supported for external validity in terms of demonstrating the expected theoretical and empirical correlations with the scales of the BIS/BAS scales. Result related implications for professional application, as well as potential revisions of the MPQ-BF are illustrated.

Positive Emotionality and Negative Emotionality. In a subsequent study, Eigenhuis, Kamphuis, and Noordhof [8] examined the factor structure of the primary trait scales of the MPQ-BF using EFA. As in Patrick et al.'s study, the primary trait scales loaded saliently on their target higher-order dimensions. However, there were considerable salient cross-loadings. Social Potency cross-loaded on the higher-order dimension of Constraint. Social Closeness loaded more strongly on the higher-order Negative Emotionality dimension, and Aggression had its primary loading on the higher-order Constraint dimension and not its designated higherorder Negative Emotionality dimension. Therefore, neither study showed clearly defined dimensions for the MPQ-BF at the primary trait scale-level. Also, as both these studies used PCA/EFA, the model fits were not available. Consequently, further investigation of the MPQ-BF at both the primary trait scale-level, and item-level are warranted.

Exploratory, confirmatory and combined evaluation processes
EFA and CFA have traditionally been standard approaches for assessing the dimensional structure of an instrument. For a new measure, EFA (an exploratory approach) is generally used to ascertain its factor structure. Once this is reasonably and clearly defined, CFA (a confirmatory approach) is applied to confirm the hypothesized structure [9]. Methodologically, the EFA approach involves no limitation on cross-loadings of questions, and thus scale-questions are enabled to freely load across different dimensions. The standard CFA perspective for first-order factor models (usually referred to as the independent cluster model of CFA [ICM-CFA]) is a model-informed procedure. This approach allows research to assess for a priori described construct-conceptualization (generally suggested in the EFA). As such, items associate exclusively on their associated dimensions, and all the loadings on non-target dimensions (cross-loadings) are specified to zero [10]. The cross-loadings limitation in the ICM-CFA approach is deemed as a significant source of an analysis' compromise, as items are often not purely associated to their allocated latent dimensions, and thus varying levels of constructclose links with non-target but conceptually-similar dimensions can be envisaged [11]. Given that cross-loadings for MPQ-BF questions have been frequently identified in EFA evidence [4,6,7], it can be argued that the CFA approach would not allow the reality of ratings for the MPQ-BF to be expressed correctly. Consequently, it can be expected to support poor applicability, even when this may not be the case. Related to this, some researchers (i.e., [11,12]) have advocated that it is almost unrealistic to achieve acceptable fitting structures for sufficient multi-dimensional scored instruments, when assessed exclusively with ICM-CFA procedures. A more appropriate model would be one that allows for cross-loadings alongside testing the fit for the proposed/assumed structure. The ESEM approach was developed for this purpose [13].
ESEM is a combination of the EFA and CFA processes, merging the positives of the EFA (enabling cross-loadings) and the CFA (conceptualization-inspired) procedures. Available evidence has revealed the comparative analytical advantages of the ESEM process over the both the EFA and ICM-CFA procedures [12,14]. Therefore, it is conceivable that application of the ESEM approach is more likely to provide a more realistic and comprehensive evaluation of the structure of the MPQ-BF than PCA/EFA and CFA approaches. To date, only one study has examined the factor structure of the MPQ-BF using ESEM (i.e., [6]). At the item-level, this study showed that while CFA did not support the theorized 11-factor model, ESEM did provide support in terms of good global fit. However, as this study did not report the factor loadings for the ESEM solution, it is not known how well the factors were defined. This is critical because a model that is claimed to be supported needs to have clearly defined factors [14]. Additionally, because the study did not examine the factor structure of the MPQ-BF at the primary trait scale-level, there are no ESEM findings on the structure of the MPQ-BF at this level.
In addition to the factor structure, another significant measurement feature for a valid instrument is support for the external validity of the factors in the structural model. Generally, for this purpose, the external variables need to be linked conceptually or/and empirically with the factors in the model that is under investigation (in this case, the MPQ-BF). In this context, for the MPQ-BF factors, the personality constructs in Gray's [15] reinforcement sensitivity theory (RST) of personality [15,16] could be considered useful. In Gray's original RST theory, personality primarily refers to differences among individuals in two primarily underpinning neurobiological systems, namely the Behavioral Inhibition System (BIS) and the Behavioral Approach System (BAS). These systems have been linked with various types of reinforcements, emotions, behaviors, and personality. As originally conceived, the BIS is sensitive to signals of punishment, frustrative non-reward, and novelty. Furthermore, it underlies anxiety-related personality traits. The BAS is sensitive to signals of reward and non-punishment, and its arousal prompts approach behaviors toward these stimuli. It underlies personality traits related to impulsivity. Additionally, the BAS can be conceptually linked to Tellegen's Positive Emotionality factor [17]. While it may appear that the BIS is linked conceptually to Tellegen's Negative Emotionality factor, Tellegen [18] has instead likened the BIS to his dimension of Constraint. This is because this factor encompasses traits related to tendency towards restraint versus impulsiveness and venturesomeness. The constructs in Gray's original RST model of personality are generally assessed using the BIS/BAS scales [19]. In the BIS/BAS scales, the BIS and BAS scales assess their namesake constructs. The BAS scale has three subscales: BAS-Reward Responsiveness, BAS-Drive (BAS-DR) and Fun Seeking (BAS-FS). BAS-Reward Responsiveness assesses approach motivation in anticipation of a future reward; BAS-Drive assesses goal-directed behavior; and BAS-Fun Seeking assesses motivation to approach immediately (a form of impulsivity). Related to these associations, the study by Eigenhuis et al. [8] examined the relationships of the MPQ-BF broad factors with the factors in the BIS/BAS scales. Findings showed that Positive Emotionality was correlated positively with all three BAS scale scores; Negative Emotionality was correlated negatively with BAS-Drive and BAS-Reward Responsiveness scale scores, and positively with the BIS scale score; and Constraint was correlated negatively with the BAS-Fun Seeking scale score, and positively with BAS-Drive and BIS scale scores. Given these considerations, it can be speculated that Tellegen's Positive Emotional dimensions and its designated trait scales would be positively associated with BAS-Reward Responsiveness, BAS-Drive, and Bas-Fun Seeking. Also, Tellegen's Constraint dimension would be positively associated with the BIS, and negatively with BAS-Fun Seeking. Although Tellegen's Negative Emotional dimension is viewed as being theoretically associated with the BIS, Tellegen has not supported this association.

The present study
Given the limitations in existing data concerning the MPQ-BF, the major aim of the present study was to use the ESEM procedure to examine the proposed factor structures at the itemlevel (i.e., items loading on the 11 lower-order primary trait factors) and the scale-level (i.e., the 11 primary trait scales loading on the four higher-order broad dimensions) among a large group of adults from the general community. In the ESEM model, as shown in Fig 1, the primary factor loads on their own targeted higher broad factors as well as non-targeted higher broad factors (at values close to zero). The present study examined the factors models separately at the item and scale levels because it is not possible to apply ESEM for second-order factor models using currently available SEM software programs, such as Mplus. Contingent on at least adequate model support, the external validity of the factors/dimensions in the model/s was also examined in terms of how the factors/dimensions were related to the scales in the

PLOS ONE
BIS/BAS scales. In terms of expectations, and based on the finding reported by Eigenhuis et al. [6], it was expected that ESEM would provide some level of support for the theorized MPQ-BF models at both the item-level and scale-level. Also, in terms of the external validity of the MPQ-BF factors, it was expected that the Positive Emotionality dimensions and its lowerorder primary trait factors would be positively associated with the three BAS scales, and that the Constraint dimension and its lower-order primary trait factors would be positively associated with the BIS.

Participants
Two-hundred and fourteen adults (115 females, M age = 34.64 years, SD = 15.57; 99; males, M age = 36.91 years, SD = 17.48; combined sample, M age = 31.71 years, SD = 16.48, range = 18 to 76 years) were recruited in Australia through several sources from the State of Victoria (see Procedure). The 95% confidence interval maximum sampling error for a population of 214 is -+ 6.7% (Z = .196). The mean age of females and males did not vary significantly, t (212) = 1.01, p < .05. As shown in Table 1, for the sample as a whole, the mean scores for the 11 lowerorder primary trait scales were all within the normal range.

Materials Multidimensional Personality Questionnaire-brief (MPQ-BF; [4]).
As aforementioned, the MPQ-BF has 155 items and11 primary personality trait dimensions. The personality trait scales and their allocation to the four higher dimensions (positive emotional temperament, absorption, negative emotional temperament, and constraint) were described above. It additionally embraces additional validity scale-items, assessing random responding (Variable Response Inconsistency), "yea-saying" or "nay-saying" (True Response Inconsistency), and social desirability (Unlikely Virtues). The T scores corresponding to the total scores for each of the personality trait scales were exclusively used in the present study (as observed indicators in the ESEM model). Each MPQ-BF item was scored as either true (rated 1) or false (rated 0) regarding the applicability of the statement to the participant. Therefore, for every sub-scale  [19] examine individual variations considering the BIS and BAS traits of the original RST [15]. The BIS scale (seven questions) measures the inclination of experiencing negative affect and behavioral inhibition when indications for punishment or risk apply. The BAS scale (13 items) assesses BAS sensitivity (i.e., the inclination to exhibit strong positive affect and behavioral arousal in the context of reward expectations). The BAS scale involves three subscales: BAS-Reward Responsiveness (five items), BAS-Drive (four items) and BAS-Fun Seeking (four items). BAS-Reward Responsiveness assesses approach motivation in anticipation of a future reward; BAS-Drive assesses goal-directed behavior; and BAS-Fun Seeking assesses the tendency to impulsively pursue pleasure. Questions are scored on a four-point Likert-scale (1 = "very false for me" to 4 = "very true for me"). Higher subscales rates indicate higher sensitivities. The total sum scores for the BIS, BAS-Reward Responsiveness, BAS-Drive, and BAS-Fun Seeking were used in the present study. The BIS/BAS Scales possess satisfactory convergent, discriminant, and concurrent validity [19]. The internal consistencies (Cronbach's α) of the BIS, BAS-Reward Responsiveness, BAS-Drive, and BAS-Fun Seeking Scales in the present study were .74, .80, .84, and .78 respectively.

Procedure
Following ethics approval from the Human Ethics Committee of the Federation (former Ballarat) University, participants were approached at various work and social situations. All participants were from various sources from the community. These included individuals enrolled at shopping centers, and sporting, recreational, hobby, and social cubs and associations. The procedure was explained and, if they were positive in participating, individuals were administered the questionnaires, the study plain language statement, and a prepaid reply envelope. The questionnaires included the MPQ-BF, and the BIS/BAS scales. Participants either mailed the questionnaires in prepaid envelopes or handed them to the researchers. A debriefing statement was distributed at the end of the study. Around 350 questionnaires were distributed to recruit the sample, resulting in a retention proportion approximating 61%.

Statistical analysis
The ESEM and CFA models employed here were calculated with the Mplus (Version 7) software [9], using target oblique rotation. Missing values were addressed based on the full-information maximum likelihood approach built into Mplus. This method is based on that data is missing completely at random. As the scores at the item-level were categorical, weighted least square mean and variance adjusted chi-square (WLSMV) estimator was employed for the ESEM calculations at this level. WLSMV can address issues of lack of normality and is indicated for items with four or less response categories (binary items in this case; [9,20,21]). Because the scores for the primary traits were continuous, the robust maximum likelihood (MLR) estimator was employed for the analysis at the primary traits level. This robust estimator also corrects for potential lack of normality concerns.
To ascertain if there were positive indications for the ESEM model, the study examined global fit, salience of loading of the primary trait scales, and presence/absence of primary trait scales with salient cross-loadings. To be considered an acceptable model at item-level, it had to have at least acceptable global fit alongside most of the items having salient loadings on their respective targeted primary trait dimensions, and absence of items having secondary cross-loadings on non-targeted primary trait dimensions. To be considered an acceptable model at the primary traits factor level, it had to have at least acceptable global fit, alongside most of the primary trait scale scores having salient loadings on their respective targeted broad factors, and absence of primary trait scale scores having secondary cross-loadings on non-target broad factors. Because all types of χ 2 values, including MLRχ 2 , are distorted by large sample cohorts, the global fit of ESEM model was evaluated ultilizing the root mean square error of approximation (RMSEA) and the comparative fit index (CFI). Hu and Bentler [22] have advocated that RMSEA values approximating .06 or lower support good fit, close to .07 and up to < .08 moderate fit, close to .08 and up to .10 marginal fit, and close to or over .10 poor fit. For the CFI, rates of .95 or higher indicate good fit, rates between .90 and up to < .95 are indicative as acceptable fit, and rates < .90 are considered as indicating poor fit. For the present study, the CFI add RMSEA had to show at least acceptable and marginal fit, respectively, for it to be considered accepted. Tabachnick and Fidell [23] suggested that the rotated factor loading has to be at least .32 (approximately 10% of the overlapping variance between item/indicator and factor) to be meaningful. Therefore, factor loadings of .32 or above were considered salient in the study.
To test for the relationships of the factors in the ESEM MPQ-BF with the factors in the BIS/ BAS scales, the correlations of these factors were computed. As is usual, positive correlations (p < .05) for the ESEM MPQ-BF factors/dimensions with the factors in the BIS/BAS scales were taken an indicative of significant relationships. For these correlations, their effect sizes were interpreted in terms of Cohen's [24] guidelines for r effect sizes: 0.1 = small, 0.3| = medium, and 0.5 = large.

ESEM evaluation of the theorized model at the item-level
The fit values for the ESEM model were χ 2 = 8715.51, df = 7249, p < 0.0001, CFI = 0.833, RMSEA = 0.022 (90% confidence interval = 0.020 to 0.024). For this model, the CFI value indicated poor fit, and the RMSEA value indicated good fit. Therefore, this model was interpreted as having inadequate fit. Table 2 shows the factor loadings for the MPQ-BF items in the ESEM model with 11 primary lower-order trait scales (factors). As demonstrated, not all the designated items loaded saliently on their respective primary trait factors. Additionally, all the primary trait factors also had several non-designated items loading saliently on them. Excluding non-salient items, negative salient items, and items that cross-loaded on other factors, there were (out of 12 in each case), 9, 6, 10, 7, 8, 6, 7, 6, 3, 6 and 7 salient items for the Wellbeing (

CFA and ESEM evaluation of the theorized model at the scale-level
Initially, for comparison, the fit of the theorized four-factor oblique model was computed using CFA. The fit values were χ 2 = 174.18, df = 39, p < 0.0001, CFI = 0.663, RMSEA = 0.127 (90% confidence interval = 0.108 to 0.147). Therefore, both the RMSEA and CFI values indicated poor fit. The fit values for the ESEM model were χ 2 = 46.83, df = 17, p < 0.0001, CFI = 0.926, RMSEA = 0.091 (90% confidence interval = 0.060 to 0.122). For this model, the CFI value indicated adequate fit, and the RMSEA value indicated marginal fit. Therefore, the fit findings for this model was interpreted as having some acceptable fit. Table 1 shows the factor loadings for the ESEM model. As demonstrated in the table, three of the four Positive Emotionality lower-order primary scales loaded positively and saliently (>.32) on the broad higher-order Positive Emotionality factor; the Absorption scale loaded positively and saliently on the Absorption factor; all three lower-order primary scales for Negative Emotionality loaded positively and saliently on the broad higher-order Negative Emotionality factor; and all three lower-order primary scales for Constraint scales loaded positively and saliently on the broad high-order Constraint factor. There was no salient secondary cross-loading (>.32 on a non-target factor). Although the lower-order Social Closeness factor only had a non-salient loading on it target Positive Emotionality higher-order broad factor, its loading on this factor was higher than its loadings on the other broad higher-order factors. Therefore, the four broad higher-order factors were considered to be well defined. Given this, and the adequate fir for this model at the global level, this model was interpreted as being adequate. Table 3 shows the correlations of the higher-order dimensions in the ESEM MPQ-BF model with the factors/scales in the BIS/BAS scales. As demonstrated, Positive Emotionality correlated positively with BAS-Drive. The magnitude of this correlation was of medium effect size, Absorption correlated positively with BAS-Drive, and the magnitude of this correlation was of small effect size. Negative Emotionality was not correlated significantly with any of the BIS/ BAS factor scales. Constraint was correlated positively with BIS, and negativity with BAS-Fun Seeking. The magnitude of the correlation with BAS-Fun Seeking was of medium effect size, and the magnitude of the correlation with BIS was of large effect size.

Discussion
The primary goal of the current empirical research was to use the ESEM procedure to assess the proposed dimensional structure of the MPQ-BF at the item-level (items loading on the 11 lower-order primary trait factors) and the scale-level (the 11 primary lower-order trait factors loading on the four higher-order broad dimensions) in a normative group of adults from the general community. At the item-level, although the RMSEA value indicated good fit, the CFI value indicated poor fit. Additionally, for this model, the factors were not clearly defined, with the primary lower-order trait factors not having all their designated items loading saliently on them, and having many non-designated items loading saliently on them. Therefore, overall, this model was interpreted as not demonstrating adequate support at the item level. In contrast to the findings here, the study by Eigenhuis et al. [6] that used the ESEM approach for examining the factor structure of the MPQ-BF at the item level found good global fit for the theorized model. However, because that study did not report the factor loadings for this solution, it is not known if the factors in the model were well-defined. As the present study examined the loadings and cross-loadings of the factor model at the item level, the conclusion reached is that this model is not an acceptable model for the MPQ-BF at the item level.

MPQ factor structure
At the scale-level, the findings showed some degree of acceptable fit for the ESEM model. For this model, all but one lower-order primary trait factor loaded saliently on their targeted higher-order broad factors. The exception was Social Closeness. Although this scale had a non-salient loading on it target Positive Emotionality broad factor, its loading on this factor was higher than its loadings on the other broad factors. The findings also showed no salient cross-loadings on the higherorder broad factors. Therefore, all four factors of the ESEM MPQ-BF model were reasonably well defined, thereby supporting the four-factor oblique ESEM model at the scale-level. Previous PCA/ EFA studies involving the MPQ-BF at the scales level have shown that although the primary factor scales loaded saliently on their target higher-order broad factors, there was considerable crossloadings on the higher-order broad factors [4,8], thereby indicating that the higher-order broad factors in the MPQ-BF were not clearly defined. A likely explanation for the differences in the present study and that of previous studies is that unlike previous studies that used PCA and EFA  [13]. The ESEM approach is generally considered a more realistic and superior approach than the PCA, EFA approach, and even the CFA approach [12,14]. Given this, it can be argued that the findings in the present study are more credible than that reported in past PCA/EFA studies. As far as it can be ascertained, this is the first study to examine the factor structure of the MPQ-BF at the scale level using ESEM. Although Eigenhuis et al. [6] used this approach, it only examined the factor structure of the MPQ-BF at the item-level. Additionally, because the ESEM approach was used, the global fit of our models were able to be examined. As already noted, the ESEM model showed adequate fit.
The support for the ESEM model at the scale-level was further enhanced in terms of support for the external validity of the factors in this model. Consistent with theory [6,17,18] and past findings [8], the findings here showed that Positive Emotionality correlated positively with BAS-Drive. Also, Constraint correlated positively with BIS, and negativity with BAS-Fun Seeking, with the correlation for BIS being stronger. Therefore, as suggested by Tellegen [18], the findings appear to suggest that the comparable constructs in the MPQ-BF for the BAS and BIS constructs in the initial version of RST are Positive Emotionality and Constraint, respectively. Another finding worthy of note is that Absorption correlated positively with BAS-Drive. Overall, taken together, these findings can be interpreted as providing reasonable support for the external validity of the factors in the ESEM model.

Implications, limitations & further research
The findings in the study have implications for the use and revisions of the MPQ-BF. The findings in the study have implications for the use of the MPQ-BF. As the findings showed inadequate support for the factor structure of the MPQ-BF at the item level, it follows that the primary traits in this measure are not well defined by the proposed targeted items. Thus the targeted items cannot be reliably used to measure the primary traits, and if used to do so, the findings need to be viewed cautiously. However as there was adequate support for the four-factor structure at the scale level, it follows that the targeted scales do adequately measure the proposed four higher order factors in the MPQ-BF, and that they can be used to measure these traits. In this respect, it is important to note that the four-factor higher order model referred to here comprise the same factors proposed for the MPQ-BF in terms of how the 11 primary scales map on to the higher order dimensions. Given these considerations and our findings, it can be argued that only the MPQ-BF components comprising the factor structure involving the scales as indicators of the higher order factors be used for research and clinical practice. Given our findings that items did not clearly define their proposed primary traits, we believe that our findings, as such, do not have implication for reconceptualization of Tallegen's personality model.
Notwithstanding all these, our finding do underscore the need for revisions in the items in the MPQ. Table 2 shows the items with clean loadings (salient and no cross-loadings on other non-target factors) and problem loadings (non-salient and/or cross-loadings on other non-target factors) on their respective primary trait factors: It can speculated that if scores for the primary lower-order trait factors are desired it would be preferable to obtain scores from the sets of items with clean loadings rather the complete set of the 12 items. What this also means is that the revisions of the remaining items in the 11 lower-order primary traits scales (those listed under "" in Table 2 may be needed in future revised visions of the MPQ-BF. We stress though that the proposed set of possible item in a future revised version of the MPQ-BF is highly speculative, and that the reliability and validity of the ensuring MPQ-BF structured needs to be comprehensively examined in future studies before it is used. In summary, the findings in the present study were interpreted as indicating that at the primary factor level, there was support for the ESEM model with four broad high-order factors (i.e., Positive emotionality, Absorption, Negative Emotionality, and Constraint). This model showed strong support in terms of external validity. In contrast, the proposed theoretical model for MPQ-BF at the item level was not supported. Although the present study provided valuable new psychometric findings for the MPQ-BF using an advanced methodology, the findings and interpretations in the study need to be considered in relation to several limitations. First, it is possible that factors such as age, gender, and ethnicity influenced ratings of MPQ-BF items. The failure to control for these effects in the study could have confounded the results. Second, because the study involved adults from the general community, it could be argued that the findings are unique to community samples, and cannot be generalized to clinical samples or special groups. Third, although the original intention was to obtain a random sample of adults, the eventual participants constituted a convenient sample. This may limit the generalizability of the findings and the conclusions made. Fourth, the external validity of the factors in the ESEM model were examined using a limited number of external variables, thereby limiting a more comprehensive evaluation of the external validity of the ESEM model. Fifth, is the relatively small sample size (N = ) in the study. Although a general rule of thumb is that a sample size of � 200 is adequate for testing the theoretical CFA model [25], the simulation study by Bandalos [26], involving WLSMV suggests that at least 500 cases may be needed for sufficient power to reject models. It would be useful for future studies to examine the factor structure, external validity and other psychometric properties (such as measurement invariance) of the MPQ-BF at both the scale and item levels, taking into consideration the methodology used in the present study and the limitations highlighted.