http://orcid.org/0000-0002-6650-6018
Figures
Abstract
The Strengths and Difficulties Questionnaire (SDQ) is a screening tool used to measure psychological functioning among children and adolescents. It has been extensively used worldwide, but its psychometric properties, such as internal structure and reliability, seem to vary across countries. This is the first study exploring the construct validity and reliability of the Spanish version of SDQ among early adolescents (self-reported) and their parents in Latin America. A total of 1,284 early adolescents (9–15 years) and their parents answered the SDQ. We also collected demographic variables. A confirmatory factor analysis was conducted to assess the latent structure of the SDQ. We also used the multitrait-multimethod analysis to separate the true variance on the constructs from variance resulting from measurement methods (self-report vs. parent report), and evaluated the agreement between adolescents and their parents. We found that the original five-factor model was a good solution and the resulting sub-scales had good internal consistency. We also found that the self-reported and parental versions of SDQ provide different information, which are complementary and provide a better picture of the emotional, social, and conduct problems of adolescents. We have added evidence for the construct validity and reliability of the Spanish self-reported and parental SDQ versions in a Chilean sample.
Citation: Gaete J, Montero-Marin J, Valenzuela D, Rojas-Barahona CA, Olivares E, Araya R (2018) Mental health among children and adolescents: Construct validity, reliability, and parent-adolescent agreement on the ‘Strengths and Difficulties Questionnaire’ in Chile. PLoS ONE 13(2): e0191809. https://doi.org/10.1371/journal.pone.0191809
Editor: Andrea Martinuzzi, IRCCS E. Medea, ITALY
Received: March 16, 2017; Accepted: January 11, 2018; Published: February 5, 2018
Copyright: © 2018 Gaete et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data are from the AKA-EDU/15 and FONIS- EVS-13I0100 studies whose Principal Investigator may be contacted at jgaete@uandes.cl. The Committee of Ethics has provided restrictions to share the data publicly because it contains sensitive information regarding mental health of children and adolescents. For further information regarding restrictions to share the data publicly, please contact the President of the Committee of Ethics of Universidad de los Andes, Joaquin Garcia-Huidobro: jgh@uandes.cl.
Funding: This work was supported by: 1. Joint project—New Learning Environments and Technologies—AKA-EDU/15, Academy of Finland (http://www.aka.fi/en) and CONICYT, Chile (http://www.conicyt.cl) to JG; 2. Chilean National Grant for Research and Development in Health—FONIS- EVS-13I0100, CONICYT, Chile (http://www.conicyt.cl) to JG. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Mental health is an important problem worldwide, and Chile is no exception. In general, the main cause of years lost because of disability among the population aged 10–24 years old is neuropsychiatric disorder (45%) [1]. Mental health problems have an important effect, not only on the lives of the affected individuals, but also on the people around them (family, school, peers). Most of the psychiatric problems we see in adult life start during childhood or adolescence [2]. Recent reports show that the prevalence of psychiatric disorders among Chilean adolescents reaches 16.5%–18.2% [3, 4], but only a minority of affected adolescents make contact with the specialized services they require.
Detecting mental health problems early increases the chances of reduced burden and future complications [5]. In recent years, interest in developing instruments to detect these emotional and behavioural disorders has been increasing. Moreover, the World Health Organization has repetitively advocated using screening tools in primary care settings and schools [6–8]. These instruments should be valid and reliable for assessing mental health among adolescents, but should also be short, simple, and culturally adapted/tested.
The most widely used screening tests for detecting psychological and behavioural problems among young people are the Child Behavior Checklist (CBCL), the Pediatric Symptoms Checklist (PSC), and the Strengths and Difficulties Questionnaire (SDQ). The CBCL consists of 113 questions on specific problematic behaviours to be answered by the parents or surrogates of school-aged children between 6 and 18 years old. The items can be organized into three main scales (total problems, internalizing problems, and externalizing problems) and eight subscales (withdrawn, somatic complaints, anxious/depressed, social problems, thought problems, attention problems, delinquent behaviour, and aggressive behaviour) [9, 10]. The PSC consists of 35 questions about a range of emotional and behavioural problems reported by parents of children and adolescents between 4 and 18 years old [11, 12]. These items can be organized into three sub-scales: attention problems, internalizing problems, and externalizing problems. The SDQ consists of 25 items about a number of emotional and behavioural problems experienced by children aged between 4 and 16 years. It has five subscales [13]: emotional symptoms, conduct problems, hyperactivity/attention problems, peer problems, and pro-social behaviour. The first four sub-scales can be combined into a total difficulties sub-scale, while the last is considered the strengths sub-scale. It has three versions, to be answered, respectively, by parents/caregivers, by teachers, and by the subjects themselves if over 11 years old; however, the self-reported version has also been used in younger children (8–13 years old) with good psychometric results [14]. The SDQ offers several advantages over the other instruments: 1) it is a brief instrument, and thus less burdensome; 2) it provides information from three different sources using a similar item structure, allowing comparison of results and a more comprehensive overall assessment of children and adolescents’ mental status; and 3) it assess both strengths and difficulties at the same time.
Another important consideration when selecting an instrument is the availability of studies exploring its validity and reliability in a population of the target age. For instance, in Chile, there is a standardized version of the CBCL for children between 6 and 11 years old [15]. It is worth mentioning that this country-specific version is not based on the updated general version available today. A Chilean version of the new CBCL has been validated, but for a population of 1½ to 5-year-old children [16]. The PSC has also been validated in Chile, in a population of socioeconomically vulnerable students in the first grade of primary school (aged 6 years) [17]. The Chilean version of the PSC consists of 33 items and has been extensively used for school-aged pre-adolescent students (under 12 years old) [18–21]. However, validation studies for adolescents are lacking. Finally, the psychometric properties of the SDQ have been evaluated in Chile for the parent-answered version for children between 4 and 11 years old. Therefore, to the best of our knowledge, no studies have explored the construct validity, reliability, and degree of agreement between the SDQ self-report and the parental report for Chilean adolescents. Additionally, we know of no SDQ studies for adolescents from Spanish-speaking, Latin-American countries.
The psychometric properties of the SDQ self-report and parental formats have been well established for adolescents in several countries, such as the United Kingdom [22], Germany [23, 24], Netherlands [25], Nordic countries [26], Russia [27], Australia [28], Austria [29], India [27], Yemen [27, 30], Bangladesh [27, 31], China [32, 33], Brazil [27], and other European countries [34]. The only studies exploring the construct validity and reliability of the Spanish version of the young and parental formats of this instrument came from Spain [35, 36].
Studies have confirmed the five theoretical dimensions in the adolescent self-reported version and in the parental version of the questionnaire, using exploratory and confirmatory factor analyses [37, 38]. However, other studies have failed to replicate the originally postulated five-factor solution [39–42]. Additionally, they have proposed a three-factor solution [41], combining the conduct and hyperactivity/attention problems as an ‘externalizing’ dimension and the emotional and peer problems as an ‘internalizing’ dimension, while keeping the pro-social sub-scale as a separate factor [43]. Furthermore, despite evidence that the five-factor model fits well across gender and ethnic groups for young children [44], a study gathering information from five European countries found that the number of factors could be country-dependent in the case of adolescents [40], and a Norwegian study found that factor loadings were different between pre-adolescents and adolescents [45]. Regarding reliability, some studies have shown adequate internal consistency [46–49], while others have reported low values for some sub-scales, especially for conduct and peer problems [50].
For adolescents in Spanish-speaking countries in Latin America, we have found no studies exploring the construct validity of the SDQ by means of confirmatory factor analysis (CFA), using parental and self-reported data. As mentioned earlier, we found only two studies using the SDQ in Chile, one exploring the psychometric properties of the parental reports on children between the ages of 4 and 11 [51], and one presenting the results of comparing the scores between early-adolescent Aymara [an indigenous South American nation] and non-Aymara students, using the self-reported, parental, and teacher versions [52].
Therefore, we see a knowledge gap concerning the performance of the SDQ in Spanish-speaking, Latin-American countries, specifically, its construct validity and reliability for adolescent populations. The aims of the present study are: i) to evaluate competing models of the latent structure of the SDQ, using confirmatory factor analysis; ii) to explore the reliability of the resulting sub-scales having the best fit; iii) to compare the degree of agreement between adolescent self-reports and parental reports, and their respective explanatory power; iv) to provide normative data for the SDQ adolescent and parental versions.
Methods
Participants
For the purposes of this study, we used SDQ data from two separate studies in similar, school-aged populations. The first study (Study 1) is being conducted in a vulnerable urban population in San Felipe, a small city north of Santiago. This is an ongoing longitudinal study exploring the factors associated with the development of health-promoting behaviours in early adolescents. As part of the baseline assessment, we administered the SDQ to the students (10 to 15 years old) and the parents. Some preliminary results of the cross-sectional analysis have been published [53]. The second study (Study 2) gathered data from students 9 to 15 years old, aiming to test the validity of the SDQ and of the Chilean-adapted version of the Olweus Bully/Victim Questionnaire Revised. This preliminary validation study is part of a larger, ongoing research called ‘The KiVa antibullying program in primary schools in Chile, with and without the digital game component, a randomized controlled trial’ [54]. We decided to present here analytical results from both studies because Study 1 gathered information from low-income families, while Study 2 gathered information from low-, middle-, and high-income families, allowing the latter set of results to be more representative of the adolescent Chilean population as a whole.
Procedure and ethics
In Study 1, we invited all urban, municipal, state-funded primary schools in San Felipe (n = 10) to participate after obtaining authorization from school board authorities. All ten schools agreed to participate. We informed the parents or main caregivers about the study and asked them to sign and informed consent to allow their children to participate. A total of 1,035 parents were contacted, 682 consented and answered the parental questionnaire. On the day of the assessment, 560 students assented to participate and answered the questionnaire (10 did not assent, and 112 were absent that day). A total of 488 parent-child dyads provided complete data. In Study 2, we invited five schools to participate. A total of 1,945 parents or main caregivers were contacted, and 1,068 consented and answered the parent version of the SDQ questionnaire. On the day of the assessment, 913 students assented to participate and answered the questionnaire (50 did not assent, and 105 were absent that day). A total of 796 parent-child dyads provided complete data. See Fig 1: Flow chart. In both studies, the parental SDQ questionnaire was answered by the main caregiver, which most of the time was the mother. Other main caregivers were the father or other significant family member such as grandmother.
Both studies had been approved by the Ethics Committee of the Universidad de los Andes. Written informed consent was obtained from parents and written informed assent from adolescents. The study posed no risks.
Measures
Socio-demographic variables.
Sex (0 = male; 1 = female), age (years) and socio-economic status (0 = low-income families; 1 = middle-income; 2 = high-income families) were collected. The socioeconomic status was based on the criteria of the 2009 National System for the Measurement of Education Quality, which gathers information from parents or main caregivers about their household income, and it was collected from the Ministry of Education.
Strengths and Difficulties Questionnaire (SDQ) [55–57].
This questionnaire has 25 items, divided into five subscales: emotional symptoms, conduct problems, hyperactivity-inattention problems, peer problems, and pro-social behaviour. These five subscales can be organized into two major sub-scales: strengths (pro-social behaviour) and difficulties (the other four subscales). Each item uses a three-point ordinal format to be answered with one of the following: 0 = not true; 1 = somewhat true; and 2 = certainly true. Five of the items are negatively worded in the original (i.e. is obedient, thinks before acting, has good attention, has good friends, is generally liked). Therefore, for compatibility in combining subscales into major subscales, their scores were reversed. The mean score for each subscale was then calculated (range 0–10). All scores for the difficulties subscales were added up to a total difficulties score (range: 0–40). The scores on the pro-social subscale were analysed independently (range: 0–10). The SDQ has been translated into more than 50 languages [10]. We used the authorized Spanish version and the scoring algorithms proposed by its author (for more information see: sdqinfo.com).
Data analysis
For the purposes of this article, only responders with valid answers on all 25 items were included in the analyses.
Firstly, we summarized the socio-demographic variables and basic psychometric characteristics of the items using descriptive statistics, including means, standard deviations (SD), and, when necessary, frequencies and percentages. We used a structural equation modelling approach to CFA, to assess the structure of the proposed five-factor and three-factor models for the SDQ, in self-reports as well as in parental reports. Multivariate Mardia’s coefficients [58] and polychoric matrices were calculated to evaluate the distribution of the items. We ensured the adequacy of the matrices by assessment of the determinant, by the KMO index, and by Barlett’s test [59]. We also calculated the internal consistency of each factor by using McDonald’s Omega (ω), which can be interpreted as the square of the correlation between the scale score and the latent variable common to all the indicators [60]. The Omega index assumes a congeneric model, which means that factor loadings are allowed to vary, and it also takes into account the item-specific measurement error. Thus, it provides a more realistic estimate of true reliability than classical Cronbach’s Alpha values, being that both can be interpreted using the same threshold cut-off points.
We used the unweighted least squares (ULS) method for factor extraction, in view of its robustness [61]. Specifically, The ULS method does not provide inferential estimations based on the χ2 distribution (and therefore does not provide p-values), but it does not require any distributional assumption; it is robust and usually converges because of its computational efficiency; it tends to provide less biased estimates of the true parameter values than other procedures; and it shows good performance when working with polychoric matrices [62–65]. From a general perspective, we used we used the fit indices that the ULS reports such as the goodness-of-fit index (GFI), the adjusted goodness-of-fit index (AGFI), the normed-fit index (NFI), and the root-mean-square of the standardized residuals (RSMR). GFI and AGFI refer to the explained variance of the model, and values >0.90 are considered acceptable [66]. The NFI measures the proportional reduction in the adjustment function when going from null to the proposed model and is considered acceptable when >0.90 [67]. The RSMR is the standardized difference between the observed and the predicted covariance, indicating a good fit for values <0.08 [68]. From an analytical perspective, standardized saturations and the explained variance were considered. We also used a CFA approach to MTMM analysis [69]. This approach permits separation of the true variance on the constructs from variance resulting from measurement methods (self-report vs. parental report). The logic is that self-report and parental measures of the same construct should be highly correlated, but that measures of different constructs should have low correlations. We calculated squared factor loadings to estimate the explained variance in the sub-scales resulting from the underlying trait and the reporting method. The unexplained variance was termed uniqueness. We performed the same analyses according to age (≤11 vs >11) and socioeconomic status (Low vs Middle/High income) to assess potential differences.
We calculated the 25th, 50th, 75th, and 90th percentile scores for each generated sub-scale, for the total sample and for each sex, for both the adolescent and the parental SDQ versions. We also present the normative data for age groups ≤11 and >11 years old.
All analyses were performed using STATA-14, SPSS-19, and Amos-7.
Results
The materials used to produce the following results will be available upon request, including a detailed list of documents and all the data files needed in order for replication, as well as every step and the specific sequence the interested researchers should take into account to make data available [70]. Authors will post the referred materials in the group’s website, and/or will be send when asked for them [71].
Characteristics of participants
Missing data from item responses varied from 0.2% to 1.0%. The final number of respondents in children and early adolescents (age range: 9–15 years) and their parents was 1,284 (54.0%, female). The mean age was 11.3 years (SD = 1.4). Students attended 4th- (13.6%), 5th- (26.2%), 6th- (24.1%), 7th- (24.9%), and 8th- (11.3%) year primary school. Of the student sample, 40.6% came from low-income families, 21.0% from middle-income families, and 38.4% from high-income families. Table 1 shows the general features of the whole sample, and divided by study.
Psychometrics and construct validity of the SDQ
The polychoric matrix of the SDQ items using self-report data had a determinant of 0.008, the KMO test had a value of 0.87, Bartlett´s test gave 6,167.30 (df = 300; p < 0.001), and Mardia’s statistic was 48.24 (p < 0.001). In contrast, the matrix of the parental-report SDQ items had a determinant of 0.001, a KMO value of 0.87, a Bartlett´s test of 8,340.10 (df = 300; p < 0.001), and a Mardia’s statistic of 82.07 (p < 0.001).
Table 2 shows that the CFA fit indices for the SDQ were within acceptable values only in the case of the five-factor model ‒which in fact was originally proposed from a theoretical point of view‒ and they were better than in the case of the three-factor model, being that both the corresponding self-report and parental-report indices were adequate in the former.
Table 3 shows the descriptive statistics and McDonald’s Omega values for the SDQ items and factors, and Fig 2 the weights and correlations between factors for the CFA both for self-report and parental-report scores. As we can see, the values and variability of the ‘steal’ item are low in self-report scores (mean = 0.20, SD = 0.50), but especially so in parental-report scores (mean = 0.04, SD = 0.26). The factorial loadings are adequate, although the ‘good friend’ item is a low outlier in the case of self-report scores (w = 0.25). In terms of reliability, parental reports are more consistent than self-reports. McDonald’s Omega values for self-reports range from 0.65 (‘peer problems’) to 0.77 (‘hyperactivity’) and, in the case of parental reports, from 0.76 (‘peer problems’) to 0.85 (‘conduct problems’, and ‘hyperactivity’).
In self-report scores, the correlations among the five constructs are strongest between ‘emotional symptoms’ and ‘peer problems’ (r = 0.78), while in parental reports, the strongest correlations are between ‘conduct problems’ and ‘hyperactivity’ (r = 0.76). The lowest correlations in self-reports are between ‘pro-social behaviour’ and ‘emotional symptoms’ (r = -0.06), while the lowest correlations in parental reports are between ‘hyperactivity’ and ‘peer problems’ (r = 0.23). See Fig 2.
Trait and method components in the MTMM approach
The CFA approach to MTMM with two method factors and five trait factors has a very good fit to the model (Table 2). Fig 2 shows the CFA parameters for the MTMM approach, Fig 3 shows the structural parameters for the MTMM model, and Table 4 shows the trait and method variance components. As we can see, the trait variance components suggest that self-reports tend to be more discriminating on ‘pro-social behaviour’ (where they are practically unaffected by method) and ‘conduct problems’. On the other hand, parental reports seem to be particularly discriminating on ratings for ‘peer problems’ (where they are subject to low method effects) and ‘emotional symptoms’. The largest uniqueness value in self-reports is in ‘emotional symptoms’, while in parental reports it is in ‘pro-social behaviour’. Finally, we found moderately low correlations between the methods (r = 0.38).
We have produced additional results for CFA and MTMM analyses stratified by age (≤11 vs >11) and socioeconomic status (Low vs Middle/High income). These results are available in S1 File.
Normative data.
Regarding the percentiles of the total difficulties scale, the values are similar between girls and boys, being lower for girls with a difference of one point. See Tables 5 and 6 for the total sample, Tables 7 and 8 for participants aged ≤11 years old, and Tables 9 and 10 for participants aged >11 years old. The percentiles in the self-reported SDQ are slightly lower than those in the parental SDQ.
Discussion
This is the first study investigating the structure of the SDQ in a Spanish-speaking country in Latin America among adolescents and their parents. The results of our study support the originally proposed five-factor structure of the SDQ among early and middle adolescents and their parents/caregivers in Chile [13]. It appears to be a more plausible solution than the more recently proposed three-factor model [43]. However, we found high correlations between emotional symptoms and peer problems, and between conduct problems and hyperactivity, which may indicate latent, underlying internalizing and externalizing dimensions. Reliability values were in general adequate both for self-report and parent-report measures for all dimensions, although they were fair in the peer problems factor for self-report measures, being appropriate for parent-reports.
Exploring the structure of the SDQ stratified by age and socioeconomic status, the best fit was found in the parental report from middle/high households, and the worst fit was found in the self-report of students from low income households. The MTMM model was good in all strata. The correlation between self-report and parent report was similar among younger and older students, and among students coming from low income and middle/high households. Emotional problems were better explained by parental report among older students. While peer problems were better explained by parental report among students from middle/high households.
Some strengths of this study are the sample size, considering the challenge of collecting information from students and parents, and the representation of different socioeconomic backgrounds. Furthermore, normative data are provided, which may help future research to test cut-off points for determining the needs of adolescents with higher scores in difficulties sub-scales. This study, in line with most studies on SDQ, has been conducted in a population-based sample.
The factor loadings seem to be dissimilar between adolescents and parents. The factor loadings from the parents are higher than those from adolescents, which may indicate that adolescents experience their problems less distinctly than do their parents. We also notice that, in the case of adolescents, the lower, but still adequate, factor loadings were found in the negatively worded items. Furthermore, there was one item with a factor loading lower than de recommended cut-off point (<0.32) [72], namely ‘good friends’ (0.25). This methodological effect has been found in the SDQ previously [40, 73], and in other instruments [74, 75]. In our study, these results may be explained by the cognitive development of people at this age (9–15 years old), who may have had difficulty understanding the answer format or the direction of the intercalated questions. However, the results from parents all exceeded the recommended loading thresholds. Given the good factor structure, we recommend keeping these items for both adolescents and parents. However, in future research in Spanish-speaking countries, it will be important to report psychometric information bearing on whether to reconsider this recommendation, and to explore re-wording of the reverse items and assessing the effect on reliability.
Additionally, we found from the MTMM approach that the variance associated with the factor of methodological measure (self-report vs. parental report) was lower for hyperactivity problems (self-report), peer problems (parental report), and pro-social behaviour (both versions). However, for the remaining dimensions, the methodological component of the variance was high, suggesting the importance of having multi-informants when assessing psychopathology among adolescents. For example, when we see that the loading weight of the item ‘steals’ has half the weight in the parental report that it does in the self-report, we suspect that parents underestimate this symptom. This phenomenon is also found for other items such as ‘lies’ and ‘fights’, where adolescents may provide a better description of what they are doing than do parents.
This study has several limitations. Firstly, we did not have access to the entire range of ages addressed by the SDQ; therefore, our results are limited to the population studied here. Even though we have collected information from a large group of students and their main caregivers, there were many absent students the day of the survey, especially among low-income schools. Additionally, we could not access teacher reports to obtain a fuller picture of student behaviours. Several studies have shown the importance of having several informants to investigate students’ behaviours [76]. Even though our findings support a five-factor structure for the SDQ, it is possible that this instrument requires inversion of the wording of some of the items to improve understanding, especially among adolescents, which in turn may increase the reliability of some of the sub-scales. The usefulness of the normative data provided here is transitory; this data must be updated when we have better cut-off scores as a result of studies tapping both community and clinical populations. Therefore, next steps should be to explore diagnostic predictions made with the SDQ in Spanish-speaking countries in Latin America. Finally, the use of simple and short tools such as the SDQ may help to better investigate and understand the evolution of these symptoms during adolescence, and to explore mechanisms explaining the observed sex and cultural influences.
Supporting information
S1 File. Results for CFA and MTMM analyses stratified by age (≤11 vs >11) and socioeconomic status (Low vs Middle/High income).
https://doi.org/10.1371/journal.pone.0191809.s001
(PDF)
References
- 1. Gore FM, Bloem PJN, Patton GC, Ferguson J, Joseph V, Coffey C, et al. Global burden of disease in young people aged 10–24 years: a systematic analysis. Lancet. 2011;377(9783):2093–102. pmid:21652063
- 2. Costello DM, Swendsen J, Rose JS, Dierker LC. Risk and protective factors associated with trajectories of depressed mood from adolescence to early adulthood. J Consult Clin Psychol. 2008;76(2):173–83. pmid:18377115
- 3. Vicente B, Saldivia S, de la Barra F, Kohn R, Pihan R, Valdivia M, et al. Prevalence of child and adolescent mental disorders in Chile: a community epidemiological study. J Child Psychol Psychiatry. 2012;53(10):1026–35. pmid:22646991.
- 4. Vicente B, de la Barra F, Saldivia S, Kohn R, Rioseco P, Melipillan R. Prevalence of child and adolescent psychiatric disorders in Santiago, Chile: a community epidemiological study. Soc Psychiatry Psychiatr Epidemiol. 2012;47(7):1099–109. pmid:21796366.
- 5. Costello EJ. Early detection and prevention of mental health problems: developmental epidemiology and systems of support. J Clin Child Adolesc Psychol. 2016;45(6):710–7. pmid:27858462
- 6.
Hendren R, Birrell Weisen R, Orley JH, Organization WH. Mental health programmes in schools. 1994.
- 7. Fisher J, Cabral dMM, Izutsu T, Vijayakumar L, Belfer M, Omigbodun O. Adolescent Mental Health in Resource-Constrained Settings: A review of the evidence of the nature, prevalence and determinants of common mental health problems and their management in primary health care. Int J Social Psychiatry. 2011;57(1 Suppl):v–vi, 9–116.
- 8.
World Health Organization. Caring for children and adolescents with mental disorders; setting WHO directions. Geneva: World Health Organization; 2003.
- 9.
Achenbach TM, Rescorla L. Manual for the ASEBA School-Age Forms & Profiles: An Integrated System of Multi-informant Assessment. Burlington: University of Vermont, Research Center for Children, Youth & Families.; 2001.
- 10. Warnick EM, Bracken MB, Kasl S. Screening efficiency of the child behavior checklist and strengths and difficulties questionnaire: A systematic review. Child Adolesc Ment Health. 2008;13(3):140–7.
- 11. Jellinek MS, Murphy JM, Robinson J, Feins A, Lamb S, Fenton T. Pediatric Symptom Checklist: Screening school-age children for psychosocial dysfunction. J Pediatr. 1988;112(2):201–9. http://dx.doi.org/10.1016/S0022-3476(88)80056-8. pmid:3339501
- 12. Jellinek MS, Murphy JM, Burns BJ. Brief psychosocial screening in outpatient pediatric practice. J Pediatr. 1986;109(2):371–8. http://dx.doi.org/10.1016/S0022-3476(86)80408-5. pmid:3734977
- 13. Goodman R. The Strengths and Difficulties Questionnaire: A research note. J Child Psychol Psychiatry. 1997;38(5):581–6. pmid:9255702
- 14. Muris P, Meesters C, Eijkelenboom A, Vincken M. The self-report version of the Strengths and Difficulties Questionnaire: its psychometric properties in 8- to 13-year-old non-clinical children. Br J Clin Psychol. 2004;43(Pt 4):437–48. Epub 2004/11/09. pmid:15530213
- 15.
Montenegro H, Bralic S, Edwards M, Izquierdo T, Maltes S. [School mental health: Standardization of the inventory of behavioural problems and social skills of T. Achenbach in children aged 6 to 11 years]. Santiago: CIDE; 1983. Spanish.
- 16. Lecannelier F, Pérez Ewert JC, Groissman S, Gallardo D, Bardet AM, Bascuñan A, et al. Validation of child behavior inventory for children ages 1 1/2 to 5 years (CBCL 1 1/2-5) at the Santiago de Chile City. Universitas Psychologica. 2014;13(2):491–500.
- 17. George M, Siraquian X, Mores R. [Adaptation and validation of two screening instruments of mental health of first grade school children of Santiago]. Revista de Psicología. 1995;5:17–25. Spanish.
- 18. Guzman MP, Jellinek M, George M, Hartley M, Squicciarini AM, Canenguez KM, et al. Mental health matters in elementary school: first-grade screening predicts fourth grade achievement test scores. Eur Child Adolesc Psychiatry. 2011;20(8):401–11. pmid:21647553
- 19. Murphy JM, Guzmán J, McCarthy A, Squicciarini AM, George M, Canenguez K, et al. Mental health predicts better academic outcomes: A longitudinal study of elementary school students in Chile. Child psychiatry Hum Dev. 2015;46(2):245–56. pmid:24771270
- 20. Guzmán J, Kessler RC, Squicciarini AM, George M, Baer L, Canenguez KM, et al. Evidence for the effectiveness of a national school-based mental health program in Chile. J Am Acad Child Adolesc Psychiatry. 2015;54(10):799–807.e1. pmid:26407489
- 21.
Halpern ME. [Early detection of psychopathology in vulnerable schoolchildren in the metropolitan region]. Santiago: University of Chile; 2011. Spanish.
- 22. Goodman R, Ford T, Corbin T, Meltzer H. Using the Strengths and Difficulties Questionnaire (SDQ) multi-informant algorithm to screen looked-after children for psychiatric disorders. Eur Child Adolesc Psychiatry. 2004;13 Suppl 2:II25–31. pmid:15243783.
- 23. Woerner W, Becker A, Rothenberger A. Normative data and scale properties of the German parent SDQ. Eur Child Adolesc Psychiatry. 2004;13(2):ii3–ii10. pmid:15243780
- 24. Klasen H, Woerner W, Rothenberger A, Goodman R. [German version of the Strength and Difficulties Questionnaire (SDQ-German)—overview and evaluation of initial validation and normative results]. Prax Kinderpsychol Kinderpsychiatr. 2003;52(7):491–502. German. pmid:14526759.
- 25. van Widenfelt BM, Goedhart AW, Treffers PD, Goodman R. Dutch version of the Strengths and Difficulties Questionnaire (SDQ). Eur Child Adolesc Psychiatry. 2003;12(6):281–9. pmid:14689260.
- 26. Obel C, Heiervang E, Rodriguez A, Heyerdahl S, Smedje H, Sourander A, et al. The Strengths and Difficulties Questionnaire in the Nordic countries. Eur Child Adolesc Psychiatry. 2004;13 Suppl 2:Ii32–9. Epub 2004/07/10. pmid:15243784.
- 27. Goodman A, Heiervang E, Fleitlich-Bilyk B, Alyahri A, Patel V, Mullick MS, et al. Cross-national differences in questionnaires do not necessarily reflect comparable differences in disorder prevalence. Soc Psychiatry Psychiatr Epidemiol. 2012;47(8):1321–31. pmid:22033632.
- 28. van der Meer M, Dixon A, Rose D. Parent and child agreement on reports of problem behaviour obtained from a screening questionnaire, the SDQ. Eur Child Adolesc Psychiatry. 2008;17(8):491–7. pmid:18431536
- 29. Altendorfer-Kling U, Ardelt-Gattinger E, Thun-Hohenstein L. [The self-report version of the Strengths and Difficulties Questionnaire in an Austrian field sample]. Zeitschrift fur Kinder- und Jugendpsychiatrie und Psychotherapie. 2007;35(4):265–71. German. Epub 2007/11/01. pmid:17970370.
- 30. Almaqrami MH, Shuwail AY. Validity of the self-report version of the strengths and difficulties questionnaire in Yemen. Saudi Med J. 2004;25(5):592–601. Epub 2004/05/13. pmid:15138526.
- 31. Goodman R, Renfrew D, Mullick M. Predicting type of psychiatric disorder from Strengths and Difficulties Questionnaire (SDQ) scores in child mental health clinics in London and Dhaka. Eur Child Adolesc Psychiatry. 2000;9(2):129–34. pmid:10926063.
- 32. Liu SK, Chien YL, Shang CY, Lin CH, Liu YC, Gau SS. Psychometric properties of the Chinese version of Strength and Difficulties Questionnaire. Compr Psychiatry. 2013;54(6):720–30. Epub 2013/02/26. pmid:23433222.
- 33. Yao S, Zhang C, Zhu X, Jing X, McWhinnie CM, Abela JR. Measuring adolescent psychopathology: psychometric properties of the self-report strengths and difficulties questionnaire in a sample of Chinese adolescents. J Adolesc Health. 2009;45(1):55–62. Epub 2009/06/23. pmid:19541250.
- 34. Marzocchi GM, Capron C, Di Pietro M, Duran Tauleria E, Duyme M, Frigerio A, et al. The use of the Strengths and Difficulties Questionnaire (SDQ) in Southern European countries. Eur Child Adolesc Psychiatry. 2004;13 Suppl 2:Ii40–6. Epub 2004/07/10. pmid:15243785.
- 35. Gomez-Beneyto M, Nolasco A, Moncho J, Pereyra-Zamora P, Tamayo-Fonseca N, Munarriz M, et al. Psychometric behaviour of the strengths and difficulties questionnaire (SDQ) in the Spanish national health survey 2006. BMC Psychiatry. 2013;13:95. pmid:23522343.
- 36. Ortuno-Sierra J, Fonseca-Pedrero E, Paino M, Sastre i Riba S, Muniz J. Screening mental health problems during adolescence: psychometric properties of the Spanish version of the Strengths and Difficulties Questionnaire. J Adolesc. 2015;38:49–56. Epub 2014/12/03. pmid:25460680.
- 37. He JP, Burstein M, Schmitz A, Merikangas KR. The Strengths and Difficulties Questionnaire (SDQ): the factor structure and scale validation in U.S. adolescents. J Abnorm Child Psychol. 2013;41(4):583–95. Epub 2012/11/28. pmid:23183936.
- 38. Tsang KLV, Wong PYH, Lo SK. Assessing psychosocial well-being of adolescents: a systematic review of measuring instruments. Child Care Health Dev. 2012;38(5):629–46. pmid:22168129
- 39. Hoofs H, Jansen NWH, Mohren DCL, Jansen MWJ, Kant IJ. The Context Dependency of the Self-Report Version of the Strength and Difficulties Questionnaire (SDQ): A Cross-Sectional Study between Two Administration Settings. PLoS ONE. 2015;10(4):e0120930. pmid:25886464
- 40. Essau CA, Olaya B, Anastassiou-Hadjicharalambous X, Pauli G, Gilvarry C, Bray D, et al. Psychometric properties of the Strength and Difficulties Questionnaire from five European countries. Int J Methods Psychiatr Res. 2012;21(3):232–45. pmid:22890628
- 41. Dickey WC, Blumberg SJ. Revisiting the Factor Structure of the Strengths and Difficulties Questionnaire: United States, 2001. Journal of the American Academy of Child and Adolescent Psychiatry. 2004;43(9):1159–67. http://dx.doi.org/10.1097/01.chi.0000132808.36708.a9. pmid:15322420
- 42. Ruchkin V, Koposov R, Schwab-Stone M. The strength and difficulties questionnaire: Scale validation with Russian adolescents. J Clin Psychol. 2007;63(9):861–9. pmid:17674399
- 43. Goodman A, Lamping DL, Ploubidis GB. When to use broader internalising and externalising subscales instead of the hypothesised five subscales on the Strengths and Difficulties Questionnaire (SDQ): data from British parents, teachers and children. J Abnorm Child Psychol. 2010;38(8):1179–91. pmid:20623175.
- 44. Hill CR, Hughes JN. An examination of the convergent and discriminant validity of the strengths and difficulties questionnaire. Sch Psychol Q. 2007;22(3):380–406. pmid:18843384
- 45. Van Roy B, Veenstra M, Clench-Aas J. Construct validity of the five-factor Strengths and Difficulties Questionnaire (SDQ) in pre-, early, and late adolescence. J Child Psychol Psychiatry. 2008;49(12):1304–12. Epub 2009/01/06. pmid:19120709.
- 46. Gomez R. Correlated trait–correlated method minus one analysis of the convergent and discriminant validities of the Strengths and Difficulties Questionnaire. Assessment. 2014;21(3):372–82. pmid:22936782
- 47. Muris P, Meesters C, van den Berg F. The Strengths and Difficulties Questionnaire (SDQ)—further evidence for its reliability and validity in a community sample of Dutch children and adolescents. Eur Child Adolesc Psychiatry. 2003;12(1):1–8. Epub 2003/02/26. pmid:12601558.
- 48. Goodman R. Psychometric properties of the strengths and difficulties questionnaire. Journal of the American Academy of Child and Adolescent Psychiatry. 2001;40(11):1337–45. Epub 2001/11/09. pmid:11699809.
- 49. Ortuño-Sierra J, Chocarro E, Fonseca-Pedrero E, Riba SSi, Muñiz J. The assessment of emotional and behavioural problems: Internal structure of The Strengths and Difficulties Questionnaire. Int J Clin Health Psychol. 2015;15(3):265–73. http://dx.doi.org/10.1016/j.ijchp.2015.05.005.
- 50. Capron C, Thérond C, Duyme M. Psychometric properties of the French version of the self-report and teacher Strengths and Difficulties Questionnaire (SDQ). Eur J Psychol Assess. 2007;23(2):79–88.
- 51. Brown P, Capella C, Antivilo A. [Psychometric properties of the parent version of the Strengths and Difficulties Questionnaire]. Revista de Psicología. 2014;23(2):28–44. Spanish.
- 52. Caqueo A, Urzua A, Ferrer R, Zuniga F, Palma C, Escudero J. [Strengths and difficulties in emotional adjustment of Aymara children from the perspective of children, parents and teachers]. Rev Chil Pediatr. 2014;85(5):561–8. Epub 2015/02/24. pmid:25697432.
- 53. Gaete J, Rojas-Barahona CA, Olivares E, Araya R. Brief report: Association between psychological sense of school membership and mental health among early adolescents. J Adolesc. 2016;50:1–5. pmid:27149432.
- 54. Gaete J, Valenzuela D, Rojas-Barahona C, Valenzuela E, Araya R, Salmivalli C. The KiVa antibullying program in primary schools in Chile, with and without the digital game component: study protocol for a randomized controlled trial. Trials. 2017;18(1):75. pmid:28219403.
- 55. Goodman R, Ford T, Simmons H, Gatward R, Meltzer H. Using the Strengths and Difficulties Questionnaire (SDQ) to screen for child psychiatric disorders in a community sample. Br J Psychiatry. 2000;177:534–9. pmid:11102329.
- 56. Goodman R. The Strengths and Difficulties Questionnaire: a research note. J Child Psychol Psychiatry. 1997;38(5):581–6. pmid:9255702.
- 57. Goodman R, Meltzer H, Bailey V. The Strengths and Difficulties Questionnaire: a pilot study on the validity of the self-report version. Eur Child Adolesc Psychiatry. 1998;7(3):125–30. Epub 1998/11/24. pmid:9826298.
- 58. Mardia KV. Applications of Some Measures of Multivariate Skewness and Kurtosis in Testing Normality and Robustness Studies. Sankhya. 1974;36(2):115–28.
- 59.
García E, Gil J, Rodríguez G. [Factorial analysis]. Madrid: La Muralla; 2000, Spanish.
- 60.
McDonald RP. Test theory: A unified treatment. Mahwah: Lawrence Erlbaum; 1999.
- 61.
Jöreskog K. Factor analysis by least-squares and maximum-likelihood methods. In: Enslein K, Ralston A, Wilf HS, editors. Statistical methods for digital computers. New York: Wiley; 1977. p. 125–53.
- 62. Briggs NE, MacCallum RC. Recovery of weak common factors by maximum likelihood and ordinary least squares estimation. Multivariate Behav Res. 2003;38(1):25–56. pmid:26771123.
- 63. Knol DL, Berger MP. Empirical comparison between factor analysis and multidimensional item response models. Multivariate Behav Res. 1991;26(3):457–77. pmid:26776713.
- 64. Lee CT, Zhang G, Edwards MC. Ordinary least squares estimation of parameters in exploratory factor analysis with ordinal data. Multivariate Behav Res. 2012;47(2):314–39. pmid:26734852.
- 65. Parry CDH, McArdle JJ. An applied comparison of methods for least- squares factor analysis of dichotomous variables. Appl Psychol Meas. 1991;15(1):35–46.
- 66.
Byrne BM. Structural equation modeling with Amos: Basic concepts, applications and programming. Mahwah: Erlbaum; 2001.
- 67.
Lévy JP, Martín MT, Román MV. [Optimization according to covariance structures. In: Lévy JP, Varela J, editors. Modeling with covariance structures in the social sciences]. Coruña: Netbiblo; 2006. p. 21–2. Spanish.
- 68. Lt Hu, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6(1):1–55.
- 69. Marsh HW, Hocevar D. Confirmatory Factor Analysis of Multitrait-Multimethod Matrices. J Educ Meas. 1983;20(3):231–48.
- 70. King G. The future of the replication movement. Int Stud Perspect. 2003;4(1):100–5.
- 71. Russett B. The ‘Journal of Conflict Resolution′s’ policy on replication. Int Stud Perspect. 2003;4(1):88–9.
- 72. Costello AB, Osborne JW. Best Practices in Exploratory Factor Analysis: Four Recommendations for Getting the Most From Your Analysis. Pract Assess Res Eval. 2005;10(7):1–9.
- 73. van de Looij-Jansen PM, Goedhart AW, de Wilde EJ, Treffers PD. Confirmatory factor analysis and factorial invariance analysis of the adolescent self-report Strengths and Difficulties Questionnaire: how important are method effects and minor factors? Br J Clin Psychol. 2011;50(2):127–44. Epub 2011/05/07. pmid:21545447.
- 74. Gaete J, Montero-Marin J, Rojas-Barahona CA, Olivares E, Araya R. Validation of the Spanish Version of the Psychological Sense of School Membership (PSSM) Scale in Chilean Adolescents and Its Association with School-Related Outcomes and Substance Use. Front Psychol. 2016;7:1901. pmid:27999554.
- 75. Marsh HW. Positive and negative global self-esteem: a substantively meaningful distinction or artifactors? J Pers Soc Psychol. 1996;70(4):810–9. pmid:8636900.
- 76. Verhulst F, Dekker M, van der Ende J. Parent, teacher and self-reports as predictors of signs of disturbance in adolescents: whose information carries the most weight? Acta Psychiat Scand. 1997;96(1):75–81. pmid:9259229