http://orcid.org/0000-0003-0821-1403
Figures
Abstract
The current study contributes to previous work on measuring the social phenotype in Autism Spectrum Disorder (ASD) by validating a multidimensional test of social competence developed for use with individuals with and without ASD. The “Multidimensional Social Competence Scale” (MSCS) was previously validated as a parent-rating scale with youth 11–18 years with ASD without intellectual disability and typically developing adolescents of comparable age. The current study presents a validation of a self-report version of the MSCS in a non-clinical young adult population (N = 1178, males = 360, females = 817, age range = 17–25 years). The MSCS consists of seven domains that represent social competence: social motivation, social inferencing, demonstrating empathic concern, social knowledge, verbal conversation skills, nonverbal sending skills, and emotion regulation. These domains are theorized to be indicative of the higher-order construct of social competence. A second higher-order theorization of the MSCS structure posits that 3 of these factors are indicative of social responsiveness, and the remaining 4 factors are indicative of social understanding and emotion regulation. Our findings indicated support for each of the theorized multidimensional factor structures. Reliability, optimal scoring, convergent and discriminant validity of the measure, as well as implications for future research are discussed.
Citation: Trevisan DA, Tafreshi D, Slaney KL, Yager J, Iarocci G (2018) A psychometric evaluation of the Multidimensional Social Competence Scale (MSCS) for young adults. PLoS ONE 13(11): e0206800. https://doi.org/10.1371/journal.pone.0206800
Editor: Alastair Smith, University of Plymouth, UNITED KINGDOM
Received: November 26, 2017; Accepted: October 19, 2018; Published: November 2, 2018
Copyright: © 2018 Trevisan et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data used for analyses are fully available with no restrictions in a Supporting Information file.
Funding: This research was funded by a grant from the Social Sciences and Humanities Research Council of Canada (No: #410-2010-0282; http://www.sshrc-crsh.gc.ca/home-accueil-eng.aspx) and a grant from the Michael Smith Foundation for Health Research (https://www.msfhr.org). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Autism Spectrum Disorder (ASD) is associated with a range of symptom severity and expression in children and adults [1]. Large individual differences in response to behavioural interventions imply etiological heterogeneity, and likely, heterogeneity at the level of underlying causal mechanisms [2–4]. “Person-oriented,” also referred to as dimensional, approaches have helped parse the heterogeneity observed in attention-deficit/hyperactivity disorder and antisocial behaviour [5–7]. The application of dimensional approaches represents a shift from the traditional emphasis on between-group differences to a focus on within-group differences [8–10]. Key to this approach is the identification of subgroups composed of individuals who are homogeneous regarding multiple variables considered simultaneously and throughout development [9]. Within a diverse population, there may be certain profiles that are particularly common and meaningfully categorized as subtypes. Such subgroups are not meant to represent “literally distinct groups” that map perfectly onto reality, but instead, are intended to “draw attention to differences in the causes and consequences” of their manifestations [11]. It is hypothesized that individuals within subgroups are more likely to share common etiological pathways and demonstrate similarities in treatment response and outcomes [12–13]. In contrast to categorical classifications systems (e.g., the Diagnostic and Statistical Manual of Mental Disorders[1]) that do not generally account for within-group heterogeneity, the person-centred or dimensional approach captures individual differences at the level of research data analysis as well as clinical assessment and treatment.
One approach to identifying homogeneous subgroups involves identifying common profiles with respect to a single domain of interest. By affording a greater level of specificity, this approach would allow researchers to ask more targeted questions of their data [12]. In the current research, we focus on the domain of social competence. We aim to contribute to previous work on assessing the social phenotype in ASD by validating a multidimensional test of social competence for use with adults with and without ASD.
The social phenotype in ASD
Given that the social domain may be the most defining area of impairment in ASD [13–14], the identification of homogeneous subgroups based on social functioning may shed light on meaningful sources of heterogeneity within ASD. Although all individuals with ASD meet the diagnostic criteria for social impairment, there is a large range of expression and severity of social deficits apparent within this population. Wing and Gould described three different categories of social impairment that might explain some of this variability: “aloof,” “passive,” and “active but odd” [15]. They described individuals in the first classification, “aloof,” as being socially withdrawn, while individuals in the second category, “passive,” as being open to social contact but unlikely to initiate it. The third classification, “active but odd,” describes individuals who initiate social interactions in an inappropriate manner. These categories appear to be differentiated primarily by varying levels of social motivation–ranging from a lack of interest (aloofness) to passive acceptance to heightened social interest but odd approach behaviour. Variability in social interest is a potentially discriminating factor as some individuals with ASD may be more likely to withdraw or show disinterest in others while others may desire social interaction but demonstrate their interest in inappropriate or awkward ways [16–18].
Yager and Iarocci developed and validated the “Multidimensional Social Competence Scale” (MSCS) as a parent-rating scale with youth aged 11–18 with Autism Spectrum Disorder (ASD) without intellectual disability and typically developing (TD) youth of comparable age [19]. This is because the social difficulties associated with ASD may in fact intersect with the variability seen in social competence within the general population [4]. As such, the MSCS was designed to assess a range of social behaviours frequently observed in high-functioning ASD individuals and that might also be seen in TD individuals exhibiting social impairment to a lesser degree.
As a next step toward the goal of developing a general measure of social competence, we extend the utility of the MSCS for use as a self-report scale in the general population and with older individuals, particularly young adults (ages 17–25). Given that this age range reflects the transition from adolescence into adulthood, variability observed in social competence should be particularly salient during this time. Thus, adapting procedures outlined by Slaney and Maraun [20], we conducted a test validation of a self-report version of the MSCS to assess whether its theorized multidimensional structure holds within a non-clinical young adult population.
Measures of social competence
Although several valid and widely used measures of social competence are available, these tests were designed for assessing specific aspects of social functioning in particular populations. For example, the “Social Skills Rating System” (SSRS) assesses social skills, problem behaviours, and academic competence [21]. The SSRS is useful for school assessments aimed at identifying individuals at risk for academic and behavioural problems. The social skills domain focuses primarily on broad-based social competencies specific to school settings (e.g., cooperation, responsibility and self-control) but is not likely to capture the finer-grained skills (e.g., social inferencing and verbal and nonverbal communication skills). Moreover, the scale is restricted to use with school-aged individuals.
Measures such as the “Autism Spectrum Quotient” (AQ), the “Broad Autism Phenotype Questionnaire” (BAPQ), and the “Social Responsiveness Scale” (SRS) were designed to assess the presence and extent of autistic traits [22–24]. Given that ASD is often conceptualized as a social disorder, measures of autistic traits such as the AQ are often used as measures of social ability. However, several clinical features and characteristics of ASD are non-social in nature—therefore, in addition to social items, the SRS measures repetitive behaviours/interests [25] and the AQ measures unusual and restricted interests, cognitive thinking styles, and executive functioning. Although these ASD screening tools are useful to screen for ASD symptoms and traits in the population at large, these measures do not extend well to measuring social competence in the general population because they include the assessment of features of autism that are not social in nature, such as, sensitivity to sensory stimuli and repetitive interests. This introduces construct-irrelevant variance when these measures are used to measure social competence with nonclinical populations. The MSCS differs in that it was designed based on a broad literature on social ability and development rather than an exclusive focus on social impairment related to symptoms of ASD.
Also widely used to assess social competence is the standardized “Vineland Adaptive Behaviour Scales” (VABS) [26–27]. The VABS measures social and non-socially adaptive and maladaptive behaviours. Although it is appropriate for use with all ages, the VABS was specifically designed to assess adaptive behaviour. According to the authors “the scales are organized using three domains—Communication, Daily Living Skills, and Socialization—that correspond to the three broad domains of adaptive functioning specified by the American Association on Intellectual and Developmental Disabilities and by DSM-5” [1,28]. The Socialization domain measures subdomains of Interpersonal relationships, Play and Leisure Skills and Coping Skills. The items on this subdomain and, more broadly on the VABS, focus on functional behaviours “Goes places with friends …”; “plans fun activities…” The MSCS has a somewhat different focus on social behaviours that reflect competencies in social situations. For example, “I recognize when people are trying to take advantage of me” or “I dominate conversations so that it can be hard to get a word in”. Thus the MSCS may better capture the heterogeneity in finer-grained social competencies in the general population, or in individuals with ASD with average and above average intelligence.
The “Emotional Quotient Inventory” (EQ-i) is a test of emotional-social intelligence that was originally designed for adults [29] but has since been adapted for use in child and adolescent populations [30]). This inventory measures some aspects of social and emotional competencies such as the abilities to identify and regulate emotions in others, relationship success, social awareness, stress management, adaptability, and general mood. The EQ-i is often administered in vocational settings used to make hiring decisions based on Bar-On’s[29] assertion that emotional intelligence contributes to real-world educational and workplace success. However, the EQ-i has been criticized for being based on a vague theoretical background that has not held up to empirical scrutiny [31]. For example, while emotional intelligence is conceptualized as a facet of intelligence, the EQ-i appears to be more strongly related to personality and psychopathology than related to intelligence, cognitive abilities or predictive of real-world success [30, 32–33] The MSCS includes emotion regulation as one dimension of social competence but also has 6 other dimensions that are theoretically and empirically related to the construct of social competence.
Development of the MSCS
According to the prism model, social competence can be measured at two overarching levels [19, 34]. The index level measures real-life indices of social competence, such as relationship quality, peer acceptance, and employment success. These indices do not provide detailed information about the social presentation of an individual but rather, are useful outcome measures. The motivation/skills level comprises the specific aptitudes demonstrated during social interactions including social-cognitive abilities and overt behaviours (e.g. eye contact). The MSCS was designed to parse the heterogeneity in social competence by identifying motivation/skills level behaviours and abilities that underlie successful social interactions at [19, 34]. Such indices have, for the most part, been overlooked in existing measures [35].
Originality of the MSCS
The MSCS is novel in that it recognizes that social competence exists in dynamic interactions between individuals and is multidimensional in nature. For example, the MSCS examines key transactional and relational aspects of social interactions (e.g., conversational turn taking, entering and leaving conversations, social referencing) in detail [36–37] and its items include individual and interpersonal behaviour [38], skillful coordination of multiple cognitive processes and the integration of contextual factors (e.g., social norms) to adequately reflect the social demands of different situations. We theorize that social competence 1) changes over the course of development, 2) emerges within a dynamic interaction between individuals and 3) is multidimensional in nature. However, we cannot capture all of these aspects of social competence in the MSCS. Thus, we developed the MSCS to primarily tackle the multidimensional aspect of SC; differentiating social motivation from social understanding and identifying 7 relatively distinct subdomains. The main goal is to identify more homogenous groups of participants for research studies or clinical interventions based on social competence profiles. We suspect that any item on the MSCS (e.g., demonstrating empathic concern) would have consistency but also differences across settings. However, the MSCS is mostly tapping consistency across contexts. That is, the individual would be rating how they feel they demonstrate their empathic concern across many contexts. For clinical purposes, the clinician on our team (GI) has found it helpful to further discuss an item where there is an uncertain response (e.g., “it depends”) and gather more specific information about the factors that contribute to differences across contexts (e.g., sensory overload) as this may be especially helpful in the design of therapeutic strategies. However, the MSCS is not yet fully developed for clinical use and thus, does not yet have this feature that allows one to qualify a response, however, it is something that is being considered in refining its use for clinical purposes.
Utility of the MSCS
The MSCS fills a gap in the current repertoire of social measures; it captures aspects of social functioning that are fine-grained, multi-dimensional, and not disorder-specific. Therefore, the MSCS will more precisely compare levels of social functioning among different populations and be especially useful for discerning shared and distinct patterns of social competence across different populations characterized by social impairment [39]. Direct comparisons across clinical populations can reveal disorder-specific aspects of social impairment missed when using traditional approaches comparing clinical groups to typically developing comparison groups [40]. The MSCS may also be useful alongside diagnostic tools such as the Autism Diagnostic Observation Scale (ADOS) and the Autism Diagnostic Inventory-Revised (ADI-R) for the purposes of developing quantitative profiles, parsing heterogeneity in social competencies and subtyping individuals with ASD [41–42].
Theoretical structure of the MSCS
Although not exhaustive, the motivations/skills selected for the MSCS were intended to be representative of the variables associated with peer acceptance and close friendship development. The MSCS consists of 7 content domains [19]. First, social motivation reflects an individual’s desire to interact with others [35]. This domain measures, to some degree, the reward value individuals obtain from social interactions. Accordingly, items on this domain measure the degree to which respondents initiate social contact, have apparent interest in friendships and other relationships, desire to be accepted by peers, and derive enjoyment from social interactions.
Social inferencing represents the capacity to detect social cues, especially regarding making accurate judgments of others’ mental states and reading social situations in a way that guides subsequent appropriate behaviour. These skills are important considering that people’s mental states may not always match their verbal expressions.
Empathic concern represents the ability to respond to others’ emotional states in a sensitive way. Empathic responses can occur through sharing the feelings of others or feeling compassion in response to someone’s distress [43]. Empathic concern is related to social inferencing. That is, to react to others’ emotional states with an appropriate empathic response, one must first be able to correctly interpret others’ emotional states [44].
Social knowledge refers to the ability to understand the “procedural rules” of social situations and to modify one’s social behaviour in response to the demands and features of such situations. Such knowledge also reflects understanding of how one might engage differently with peers as opposed to authority figures, as well as the ability to identify potentially risky or dangerous social situations.
Verbal conversation skills refer to the ability to engage in reciprocal conversation. Attributes in this domain reflect the ability to take turns, to understand how to initiate, join and leave conversations, and to know when to maintain or change conversational topics.
Nonverbal sending skills reflect the ability to use body language to facilitate communication (e.g., eye contact, facial expressions, gestures). This domain also reflects competencies such as knowing when to smile, looking concerned in response to another’s distress, as well as nonverbal aspects of speech related to prosody, style, and volume.
Finally, emotion regulation refers to the processes involved in monitoring, evaluating, and controlling the intensity of one’s internal emotional experiences and outward emotion-related behaviour to attain desired affective states. Table 1 lists all 77 of the MSCS items organized according to content domain. The scale is available in a usable format in the supplementary material for readers to download to use for their own research or clinical purposes.
In addition to the conceptualization of the theoretical structure of the MSCS as consisting of seven domains that feed into the higher-order construct of social competence, Yager and Iarocci [19] conducted exploratory factor analyses that suggested a possible second, higher level, structure of the MSCS. Based on factor correlations, they theorized that the domains of social motivation, empathic concern, and nonverbal skills assessed the “social responsiveness” aspect of social competence; whereas, the social inferencing, social knowledge, verbal skills, and emotion regulation domains related to the “social understanding/emotion regulation” aspects of social competence. These models are different but not mutually exclusive. The objective of the current analysis was not to find the “right” model (as many models may provide a similar fit to a given dataset; see Slaney and Maraun, 2008 [20]), but to determine whether both general and fine-grained conceptualizations of social competence are reflected in the seven domains of the MSCS. The goals were, first, to determine whether the two theoretical structures hypothesized by Yager and Iarocci [19] would be replicated for a self-report version of the MSCS and a population of young adults and, second, to demonstrate whether scoring is justified in the current sample at the domain, subscale, and total scale levels.
Self vs. parent report versions of the MSCS
The MSCS was originally developed as a parent-report measure and was adapted as a self-report measure for the current study. Self-report measures have generally been overlooked in research on individuals with ASD because of the concern that they may have difficulty engaging in introspection [45–46]. However, in the case of individuals with ASD with average intellectual functioning (IQ > 85), there is evidence to suggest agreement between youth and their parents in reporting social and psychiatric symptoms [47–48] as well as adequate validity indexes on broad base behavioural measures for youth with ASD [49]. The subjective viewpoint of experience provided by self-report measures is not available with other methods of assessment and offers unique insight on social competence.
The theoretical foundations of the MSCS remain the same for the adolescents in the previous parent-report study and young adults in the current study. Thus, the same theorized structures of the MSCS for the self- and parent-report versions of the test are hypothesized in the present study. On these grounds, we explored the psychometric properties of the test using confirmatory, rather than exploratory, factor analysis.
Current study
The objectives of the present study were twofold. First, we aimed to examine whether the theoretical structure of the MSCS identified by Yager and Iarocci [19] would hold for self-ratings on the MSCS in a non-clinical sample of young adults. To this end, we did preliminary testing of the unidimensional measurement models for each of the 7 domains of the MSCS before testing the multidimensional structure hypothesized by each of two structural models: a) a 7 first-order (i.e., 7 domains) and 1 second-order (i.e., social competence) hierarchical linear factor model; and b) a 7 first-order (i.e., 7 domains) and 2 second-order (i.e., social responsiveness and social understanding/emotion regulation) hierarchical linear factor model. Our second aim was to determine on the basis of our model fit tests which scoring procedures were justified for this sample to assess the external validity of the MSCS. Convergent validity of the MSCS was examined through correlations of MSCS model-derived weighted domain scores with theoretically related measures.
Method
Participants
Data was collected from 12 studies involving typically developing Canadian undergraduate university students (N = 1391). All participants provided written consent to participate in this study. In Canada, individuals aged 16 or older can provide written consent to participate in research without the consent of a guardian. This study received ethics approval from our Simon Fraser University’s Research Ethics Board (# 2014s0694). Participants who fell within the age range of 17.5–25.5 years were included in the analyses. Participants were excluded (N = 112) because they fell outside the specified age range or their age was missing. Three participants were missing information pertaining to one of the MSCS items. These missing items were imputed using mean substitution based on the average score for that person’s valid test items. Thirty-three participants had 5 or more missing items on the MSCS and were excluded from the analyses. Sixty-eight individuals participated in more than one of the 12 studies; in these instances, only the participant’s first testing session was included in the analyses. The final sample of 1178 young adults consisted of 360 males and 817 females. Ages ranged from 17 years, 6 months to 25 years, 6 months (M = 19.72, SD = 1.61).
Materials
Multidimensional Social Competence Scale (MSCS).
Of the sample of 1178, 1038 participants completed an online version of the self-report MSCS (administered using a university WebSurvey software program) and 140 participants completed the paper version. Both versions had identical questions and contained a total of 77 items, with each of the 7 content domains containing 11 items. Participants were asked to rate each of these items on a 5-point Likert scale, with higher scores indicating greater agreement with an item, and thus, higher levels of social competence.
Autism Spectrum Quotient (AQ).
The AQ [22] is a self-report questionnaire used to screen for social and non-social autism symptomology. As the AQ is commonly used to assess social competencies, the full scale AQ was correlated with the full scale MSCS to assess convergent validity. Additionally, the AQ has two subscales that are specifically related to social competencies: “social skills” and “communication skills.” These subscales were used to assess convergent validity of the MSCS social knowledge and MSCS verbal conversation skills domains, respectively. Two subscales of the AQ, “Attention to Detail” (tendency to notice small details and patterns, rather than focusing on the ‘big picture’) and “Attention Switching” (tendency to prefer ritualized routines and hyper-focused attention to special interests) which are less theoretically related to social competence, were used to demonstrate discriminant validity of these same MSCS domains. As higher scores on the MSCS represent higher social competency, whereas higher scores on the AQ represent social impairment, the observed correlations were expected to be negative.
Behaviour Rating Inventory of Executive Function (BRIEF).
The BRIEF assesses executive dysfunction related to behavioural regulation and metacognitive skills [50]. The “monitor” subscale of the BRIEF assesses impairment in the ability to monitor the effect of one’s own behaviour on others. We reasoned that accuracy in assessing one’s influence on others would require the ability to make accurate social inferences. As such, the monitor subscale of the BRIEF was used to measure convergent validity of MSCS social inferencing domain scores. The BRIEF “emotional control” subscale was used to assess convergent validity of MSCS emotion regulation domain scores given the similarity of content between these subscales. To demonstrate discriminant validity of these two MSCS domains, we used two subscales of the BRIEF, “Initiate” (ability to begin activities and independently generate ideas and solutions), and “Organization of Raw Materials” (ability to organize one’s work space and personal belongings), both of which are less theoretically relevant to social competence. We also used a subscale of the BRIEF, “Working memory” (ability to hold information when completing a task or encoding information), to demonstrate discriminant validity of “Empathic Concern.” Observed correlations were expected to be negative as higher social competence was expected to be associated with lower executive dysfunction.
Behaviour Assessment System for Children (BASC-2) self-report-college.
The BASC-2 assesses adaptive and maladaptive behaviours of college students [51]. We reasoned that higher scores on the BASC-2 “social stress” subscale would be negatively correlated with behaviours indicative of initiating, seeking, and enjoying social interactions as measured by the MSCS “social motivation” domain. We used a subscale of the BASC, “Inattention/Hyperactivity”(which measures traits associated with Attention Deficit and Hyperactivity Disorder) to demonstrate discriminant validity for the social motivation domain. We also used the “Anxiety” subscale of the BASC to demonstrate discriminant validity for “Nonverbal sending skills” and used the “Somatization” (complaints of physical symptoms that have no medical basis) subscale of the BASC to demonstrate discriminant validity for the full scale MSCS. Again, these subscales of the BASC were used for discriminant validity based on their minimal theoretical overlap with social competence.
Diagnostics
Prior to combining data from the different studies, we ran one-way analyses of variance (ANOVAs) to address any systematic differences between participants’ MSCS domain scores based on the study in which the participant was tested. For the domains of social motivation, social inferencing, empathic concern, verbal skills, and nonverbal skills, the omnibus F-test did not indicate any statistically significant differences in non-weighted composites between the 12 different studies at a pre-set alpha level of .05 (F’s (11, 1166) ranged from .666 to 1.324). However, the omnibus F statistics for the social knowledge and emotion regulation domains were statistically significant (F’s (11, 1166) for social knowledge = 2.343 and for emotional regulation = 2.737). Further, Tukey post-hoc comparisons showed that these differences were mainly attributed to 2 studies. Thus, we ran the subsequent CFA analyses both including and excluding these 2 studies. We observed only slight differences in fit indices and factor loadings. There is no theoretical reason for us to believe that these 2 samples might have characteristically differed from the rest of the samples in the study, therefore, we opted to report the results including all 12 samples.
We computed bivariate correlations between each of the item scores within the 7 domains to confirm that items within the same domain were positively correlated with one another as theorized. The results revealed that item 41 of the emotion regulation domain was negatively correlated with almost all other items in that domain. It may be the case that item 41, “I get over setbacks or disappointments quickly”, behaved differently due to the self-report nature of the MSCS used in the present study. Yager and Iarocci’s [19] results, which were based on a parent-report version of the test, indicated a positive factor loading for this item. One possibility is that this item is indicative of emotion regulation only when parents score their children, but not when a young adult self-reports. We recommend researchers using the MSCS examine bivariate correlations between item 41 and other domain items; if correlations are negative, the emotion regulation domain can be assessed with the omission of item 41. Accordingly, we chose to omit item 41 from the current analyses.
Data analysis plan
First, we tested the measurement model for each domain separately with 1-factor linear models using confirmatory factor analysis (CFA). For each model, we constrained the item responses for the 7 domains (each consisting of 11 items, apart from the emotion regulation domain which consisted of 10) to load onto a single factor; loadings and error variances were left to vary (i.e., were specified as 1-factor “congeneric” models [52]). Next, also using CFA models, we examined the two hypothesized multidimensional theoretical structures of the MSCS proposed by Yager and Iarocci [19]. The first model tested a 7 first-order factors (representing the 7 domains of the MSCS) and 1 second-order factor (social competence) structure (see Fig 1). Here, factor loadings could vary, whereas, the variance of the higher-order factor (“social competence”) and the error variances for each of 7 first-order factors were fixed at 1.
The final CFA tested a linear model with 7 first-order factors and 2 second-order factors identified in Yager and Iarocci [19] (see Fig 2). In that study, the first higher-order factor was labeled “social responsiveness” and the second higher-order factor was labeled “social understanding/emotion regulation”. Thus, we constrained domain scores for social motivation, empathic concern, and nonverbal skills to load on the first higher-order factor, and the domain scores from the remaining 4 domains to load on the second higher-order factor. As with the first multidimensional model, each of the factor loadings could vary, whereas, the variances of the 2 higher-order factors and the error variances of each of the 7 first-order factors were fixed at 1.
Next, on the basis of our model fit tests, we created weighted composites for each of the domain scores, as well as for the overall MSCS, social responsiveness, and social understanding/emotion regulation total scores. Weights were created by dividing the factor loadings by their corresponding error variances for each item across the different models. We used omega-squared to estimate reliability of the weighted composites across the 3 models and coefficient alpha to estimate the reliability of non-weighted composites.
Finally, given that data reported in this article were extracted from a larger database, convergent measures were only available for portions of the entire sample and for some of the MSCS domains. In total, we identified relevant measures for the full scale MSCS and 5 of the 7 MSCS domains. We ran correlational analyses to assess the convergent validity of each of these composites. All correlation estimates were corrected for attenuation.
Results
Confirmatory factor analyses
CFAs were conducted using the CALIS procedure in SAS Software 14.0 on the sample covariance matrix using diagonally weighted least squares estimation (based on the ordinal nature of the test data). We used two measures of absolute fit: the standardized root mean square residual (SRMR) and the root mean square error of approximation (RMSEA). Moschopoulos’ [53] approximation was used for computing the 90% confidence intervals around the RMSEA estimates. We employed Hu and Bentler’s [54] criteria for “relatively good fit” for a sample size N>250. SRMR values less than .08 are ideal, and RMSEA values less than .06 are ideal, although, RMSEA values between .06 and .08 are often considered acceptable. We also report the chi-square (χ2) estimate; however, given that chi-square is a biased measure of fit when N>200, we emphasize the SRMR and RMSEA estimates [55]. In addition, readers should note that Hu and Bentler’s [54] guidelines are specific to maximum likelihood estimation. However, Zhao [56] examined the use of fit indices and Hu and Bentler’s [48] guidelines with diagonally weighted least squares estimation. He recommended that if the χ2 statistic is small and the RMSEA value is small, then one can conclude adequate model fit. Alternatively, if the χ2 statistic is large and the RMSEA value is small, but the sample size is very large, then researchers might conclude that the model fits adequately because the inflated χ2 value is due to the large sample size.
Domain measurement models.
Table 2 presents the results of 1-factor CFAs. The SRMR and RMSEA indices were adequate for all dimensions. This indicates support for each of the domain measurement models and the forming of unweighted composites (i.e., simple sums of items.
Theoretical structural model 1: 7 first-order factors and 1 second-order factor.
The SRMR and RMSEA estimates for Model 1 indicate adequate fit (see Tables 3 and 4). The estimated loadings for each of the domains on the second-order factor were positive. Table 5 presents the estimated correlations for the model.
Model 2: 7 first-order factors and 2 second-order factors.
The fit indices for Model 2 were slightly better than that of the second model, again indicating reasonably good fit (see Table 3). The estimate of the covariance between the 2 second-order factors was .731. Tables 3 and 4 present the factor loadings for the domains on the two second-order factors.
Reliability and convergent validity
The results of the structural model tests indicate that there is justification for forming composite scores for the 7 MSCS domains, the overall MSCS, as well as the social responsiveness and social understanding/emotion regulation subscales. Given that the measurement models for all 7 domains indicated adequate fit, forming unweighted item composites for each of the 7 domains and interpreting these as indicators of unidimensional is supported for this data. Reliability estimates (i.e., Chronbach’s alpha) for the unweighted composites of each of the domains and higher order composites are presented in Table 6. Table 6 also presents the results of correlation analyses assessing convergent validity, and Table 7 presents the results of correlation analyses assessing discriminant validity. Domain scores and total scores correlated with each of the external measures in the expected directions.
Discussion
The objective of the current study was to assess two hypothesized multidimensional structures of a self-report version of the MSCS in a sample of young adults. First, the 7-first order and 1-second order model (i.e., Model 1) tested whether these 7 first-order domains all loaded onto the theorized construct of social competence. Second, the 7 first-order and 2 second-order model (i.e. Model 2) tested whether 3 of the first-order factors loaded onto the theorized construct of social responsiveness and the other 4 loaded onto the theorized construct of social understanding/emotion regulation. For each of these structural models, the SRMR and RMSEA indices suggested reasonably good fit. These results are consistent with previous findings [19].
Yager and Iarocci [19] theorized that the first-order domains of social motivation, empathic concern, and nonverbal sending skills might be indicative of individuals’ aptitudes for social responsiveness. In contrast, the first-order domains of social inferencing, social knowledge, verbal skills, and emotion regulation were theorized to represent social understanding and emotion regulation. These domains should be related as they each represent “the cognitive and behavioural skills needed to respond appropriately in social situations” [19]. Indeed, factor correlations between each of the first-order domains of the MSCS were comparable to that of Yager and Iarocci [19]. We found that the domains theorized to be related to the 2 higher order constructs were moderately to strongly correlated with one another. In contrast, we only found modest correlations between social motivation and verbal skills, and between emotion regulation and social motivation, empathic concern, social knowledge, and nonverbal sending skills. Emotion regulation scores were the least strongly correlated with the MSCS total score. The other domains all correlated quite strongly with the overall score.
A second aim was to determine whether forming scores at all three of the domain, subscale, and overall MSCS levels was justified for this sample. The results of our tests of the domain measurement models indicated adequate fit for all 7 domains of the MSCS. As such, forming sums of domain scores for each of these domains and interpreting these as indicators of single facets of social competence was justified for the current sample. In practice, researchers are encouraged to determine whether unidimensionality of each of these 7 domains holds for their own data and therefore whether non-weighted sums are justified.
Further validation of the MSCS in both clinical and non-clinical populations is warranted. In particular, item 41 in both parent-report and self-report versions of the MSCS needs further examination. Qualitative analyses of item descriptions might aid in identifying discrepancies in positive and negative factor loadings between different populations. In addition, item analyses to condense the items on the test will be useful. The current study presents a first step at validating the MSCS as a self-report and with diverse populations. However, to assess the utility of the MSCS to measure change over time requires longitudinal data.
Additionally, future research is needed to determine the convergent validity of the empathic concern and nonverbal sending skills domains, as the present study had no suitable scales to determine the convergent validity of these domains. In addition, some of the convergent validity indices presented in Table 5 were rather low small indicating questionable convergent validity of certain MSCS domains, although these small correlations may have been due to low reliability of the convergent measures (especially subscales of the AQ), or insufficient theoretical overlap with the MSCS domains. To provide a more complete validation study of the MSCS, future research will benefit from using convergent measures with stronger theoretical overlap, as well as the use of other methods besides self-report (e.g., other-report measures or “ecologically valid” measures of social competence based on real-world social behavior).
The MSCS is a first step to establishing some basic parameters of social competence that may be used as an index of a person’s social functioning. We do not mean for it to be a way to assess the “best” or only way to be socially competent as we acknowledge that social competence is a dynamic, context dependent construct that is socially constructed. There is a need for qualitative research to uncover the many ways that adults with ASD are resilient and competent in relationships that may be quite different from the TD way. However, the MSCS is based on the current literature on social development and, although limited, it can begin to point out the ways in which people (not just those with ASD) may be struggling to adapt to the mainstream social standards. This measure does not endorse mainstream social standards as the best or only way to be social. However, we acknowledge that people with ASD (and others who behave differently within a mainstream culture) are often in situations where they are interacting with people who abide by mainstream social standards and, as such, often struggle to adapt.
The MSCS has the potential to contribute to a variety of research applications in educational, clinical, and non-clinical settings. For example, MSCS profiles of social competence may prove useful in behavioural genetics research, in which a well specified phenotype or endophenotype is a more useful way of grouping participants than a clinical diagnosis [12]. Also, differentiating between “social responsiveness” and “social understanding/emotion regulation” components of social competence may be especially helpful in intervention studies.
Supporting information
S1 File. Multidimensional social competence scale.
MSCS available for download.
https://doi.org/10.1371/journal.pone.0206800.s001
(PDF)
S2 File. Data used for analyses.
Data file containing data used for all analyses in this study.
https://doi.org/10.1371/journal.pone.0206800.s002
(SAV)
Acknowledgments
We are grateful to the participants for contributing to this research and for providing helpful feedback on the scale.
References
- 1.
American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5th ed. Arlington: American Psychiatric Publishing, 2013.
- 2. McEachin JJ, Smith T, Lovaas OI. Long-term outcome for children with autism who received early intensive behavioural treatment. Am J Mental Retardation. 1993;97: 359–91.
- 3. Borden MC, Ollendick TH. An examination of the validity of social subtypes in autism. J Autism Devel Disord. 1994;24: 23–37.
- 4. Happe F, Ronald A, Plomin R. Time to give up on a single explanation for autism. Nat Neurosci. 2006;9: 1218–20. pmid:17001340
- 5. Mill J, Caspi A, Williams BS, Craig I, Taylor A, Polo-Tomas M, et al. Prediction of heterogeneity in intelligence and adult prognosis by genetic polymorphisms in the dopamine system among children with attention-deficit/hyperactivity disorder: Evidence from 2 birth cohorts. Arch Gen Psychiatry. 2006;63: 462–69. pmid:16585476
- 6. Nigg JT, Willcutt EG, Doyle AE, Sonuga-Barke EJ. Causal heterogeneity in attention-deficit/hyperactivity disorder: Do we need neuropsychologically impaired subtypes? Biol Psychiatry. 2005;57: 1224–30. pmid:15949992
- 7. Moffitt TE. Adolescence-limited and life-course-persistent antisocial behaviour: A developmental taxonomy. Psychology Rev. 1993;100: 674–701.
- 8. Beauchaine TP. Taxometrics and developmental psychopathology. Devel Psychopathol. 2003;15: 501–27.
- 9.
Bergman LR, von Eye A, Magnusson D. Person-oriented research strategies in developmental psychopathology. In: Cicchetti D, Cohen DJ, editors. Developmental psychopathology, Vol 1: Theory and method. 2nd ed. Hoboken: John Wiley & Sons; 2006. p. 850–888.
- 10. von Eye A, Bergman LR. Research strategies in developmental psychopathology: Dimensional identity and the person-oriented approach. Devel Psychopathol. 2003;15: 553–80.
- 11. Nagin DS. Analyzing developmental trajectories: A semiparametric, group-based approach. Psychological Methods. 1999;4: 139–57.
- 12. Iarocci G, Yager J, Elfers T. What gene-environment interactions can tell us about social competence in typical and atypical populations. Brain Cognition. 2007;65: 112–27. pmid:17628271
- 13.
Pennington BF, Ozonoff S. A neuroscientific perspective on continuity and discontinuity in developmental psychopathology. In: Cicchetti D, Toth SL, editors. Rochester Symposium on Developmental Psychopathology, Vol. 3: Models and integrations. University of Rochester Press; 1991, p. 117–59.
- 14.
Volkmar FR, Klin A. Issues in the classification of autism and related conditions. In: Volkmar FR, Paul R, Klin A, Cohen, D, editors. Handbook of autism and developmental disorders Vol. 1. 3rd ed. Hoboken: John Wiley & Sons; 2005. P. 5–41.
- 15. Wing L, Gould J. Severe impairments of social interaction and associated abnormalities in children: Epidemiology and classification. J Autism Devel Disord. 1979;9: 11–29.
- 16. Ghaziuddin M. Defining the behavioural phenotype of Asperger syndrome. J Autism Devel Disord. 2008;38: 138–42.
- 17. Tantam D. Asperger's syndrome. J Child Psychology Psychiatry. 1988;29: 245–55.
- 18.
Volkmar FR, Klin A. Diagnostic issues in Asperger Syndrome. In: Klin A, Volkmar F, Sparrow SS, editors. Asperger Syndrome. New York: Guilford; 2000. p. 25–71.
- 19. Yager J, Iarocci G. The development of the multidimensional social competence scale: A standardized measure of social competence in autism spectrum disorders. Autism Res. 2013;6: 631–41. pmid:24108618
- 20. Slaney KL, Maraun MD. A proposed framework for conducting data-based test analysis. Psychological Methods. 2008;13: 376–90. pmid:19072000
- 21. Gresham FM, Elliot SN. Social skills rating system (SSRS). Circle Pines: American Guidance Service; 1990.
- 22. Baron-Cohen S, Wheelwright S, Skinner R, Martin J, Clubley E. The autism-spectrum quotient (AQ): Evidence from asperger syndrome/high-functioning autism, males and females, scientists and mathematicians. J Autism Devel Disord. 2001;31: 5–17.
- 23. Hurley RS, Losh M, Parlier M, Reznick JS, Piven J. The broad autism phenotype questionnaire. J autism devel disord. 2007;37: 1679–90.
- 24.
Constantino JN, Gruber CP. Social responsiveness scale (SRS). Los Angeles: Western Psychological Services; 2005.
- 25. Constantino JN. Social responsiveness scale. Encyclopedia of Autism Spectrum Disorders. 2013; 2919–29.
- 26.
Sparrow SS, Balla DA, Cicchetti DV. Vineland adaptive behaviour scales. Circle Pine: American Guidance Services; 1984.
- 27.
Sparrow SS, Cicchetti DA, Balla DV. Vineland adaptive behaviour scales. 2nd ed. San Antonio: The Psychological Corporation; 2005.
- 28.
Schalock RL, Buntinx W, Borthwick-Duffy A, Luckasson R, Snell M, Tasse M, et al. User’s guide: Mental retardation: Definition, classification, and systems of supports (10th ed.). Washington, DC: American Association on Intellectual and Developmental Disabilities; 2007.
- 29.
Bar-On RM. The emotional intelligence inventory (EQ-I): Technical manual. Toronto: Multi-Health Systems, 1997.
- 30.
Bar-On R, Parker J. The emotional quotient inventory: Youth version: Technical manual. Toronto: Multi-Health Systems; 2000.
- 31.
Matthews G, Zeidner M, Roberts RD. Emotional intelligence: Science and myth. Cambridge, MA: MIT Press; 2004
- 32. Dawda D, Hart SD. Assessing emotional intelligence: Reliability and validity of the Bar-On Emotional Quotient Inventory (EQ-i) in university students. Personality Individ Diff. 1997;28(4): 797–812.
- 33. Newsome S, Day AL, Catano VM. Assessing the predictive validity of emotional intelligence. Personality Individ Diff. 2000;29: 1005–1016.
- 34. Rose-Krasnor L. The nature of social competence: A theoretical review. Soc Devel. 1997;6: 111–35.
- 35. Newcombe AF, Bukowski WM, Pattee L. Children's peer relations: a meta-analytic review of popular, rejected, neglected, controversial, and average sociometric status. Psychological Bull. 1993;113: 99–128.
- 36. Heerey EA. Decoding the dyad challenges in the study of individual differences in social behaviour. Current Directions in Psychological Sci. 2015;24: 285–91.
- 37. Shteynberg G. Shared attention. Perspectives Psychological Sci. 2015;10: 579–90.
- 38. Buhrmester D, Furman W, Wittenberg MT, Reis HT. Five domains of interpersonal competence in peer relationships. J Personality Soc Psychology, 1988;55: 991–1008.
- 39. Morrison KE, Pinkham AE, Penn DL, Kelsven S, Ludwig K, Sasson NJ. Distinct profiles of social skill in adults with autism spectrum disorder and schizophrenia. Autism Res. 2017;10: 878–87. pmid:28112487
- 40. Sasson NJ, Pinkham AE, Carpenter KL, Belger A. The benefit of directly comparing autism and schizophrenia for revealing mechanisms of social cognitive impairment. J Neurodevel Disord. 2011;3:87–100.
- 41. Lord C, Risi S, Lambrecht L, Cook EH, Leventhal BL, DiLavore PC, et al. The Autism Diagnostic Observation Schedule—Generic: A standard measure of social and communication deficits associated with the spectrum of autism. J Autism Devel Disord. 2000;30: 205–23.
- 42. Lord C, Rutter M, Couteur A. Autism Diagnostic Interview-Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. J Autism Devel Disord. 1994;24: 659–685.
- 43. Baron-Cohen S, Wheelwright S. The empathy quotient: an investigation of adults with Asperger syndrome or high functioning autism, and normal sex differences. J Autism Devel Disord. 2004;34: 163–75.
- 44. Smith A. The empathy imbalance hypothesis of autism: A theoretical approach to cognitive and emotional empathy in autistic development. Psychological Record. 2009;59: 489–510.
- 45. Baron-Cohen S. The extreme male brain theory of autism. Trends Cog Sci. 2002;6: 248–54.
- 46. Hill E, Berthoz S, Frith U. Brief report: Cognitive processing of own emotions in individuals with autistic spectrum disorder and in their relatives. J Autism Devel Disord. 2004;34: 229–35.
- 47. Hurtig T, Kuusikko S, Mattila ML, Haapsamo H, Ebling H, Jussila K, et al. Multi-informant reports of psychiatric symptoms among high-functioning adolescents with Asperger syndrome or autism. Autism. 2009;13: 583–98. pmid:19933765
- 48. Ozsivadjian A, Hibberd C, Hollocks M. Brief report: The use of self-report measures in young people with autism spectrum disorder to access symptoms of anxiety, depression and negative thoughts. J Autism Devel Disord. 2013;44: 969–74.
- 49. Bakhtiari R. Hutchison S, Iarocci G. How do Children and Adolescents with Autism Report about Themselves? A study of Validity Indexes in the BASC-2 Self-Report of Personality (SRP) (forthcoming).
- 50. Roth RM, Isquith PK, Gioia GA. Behaviour rating inventory of executive function–adult version. Odessa: Psychological Assessment Resources; 2005.
- 51.
Reynolds CR, Kamphaus RW. BASC-2 Behaviour assessment system for children. 2nd ed. Circle Pines: American Guidance Service; 2004.
- 52. Joreskog KG (1971) Statistical analysis of sets of congeneric tests. Psychometrika, 1971;36: 109–33.
- 53. Moschopoulos PG. On a new transformation to normality. Communications Statistics–Theory Methods, 1984: 12: 1873–1878
- 54. Hu L, Bentler P. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling. 1999;6: 1–55.
- 55.
Kenny DA. Measuring model fit. In: David A. Kenny’s Homepage [Internet]. 2015. Available from: http://davidakenny.net/cm/fit.htm
- 56.
Zhao Y. The performance of model fit measures by robust weighted least squares estimators in confirmatory factor analysis. Ph.D. Dissertation, Pennsylvania State University. 2015. Available from: https://etda.libraries.psu.edu/files/final_submissions/10679