https://orcid.org/0000-0002-8367-2229
Figures
Abstract
Background
The high prevalence of mental disorders in early adolescents, and their consequences, encourage the need for validated instruments to identify and assess psychosocial problems.
Objectives
i) To evaluate the psychometric properties of the Spanish version of the Pediatric Symptom Checklist (PSC) questionnaires (PSC-Y, 35 items, and PSC-17-Y) and its subscales (Attention, Internalizing and Externalizing subscales), including the assessment of the item structure, concurrent validity, and reliability; and ii) To assess possible associations between bullying experiences, school climate and school membership with psychological problems identified by the PSC questionnaire.
Methods
A cross-sectional study was carried out in 39 schools in Santiago, Chile. The sample consisted of 3,968 adolescents aged between 10 and 11 years. A descriptive analysis of the Pediatric Symptom Checklist was performed along with measures of dimensionality, reliability, and correlations with a validated questionnaire exploring similar constructs, the Strengths and Difficulties Questionnaire. Finally, associations of bullying, school climate, and school membership with the three subscales of the PSC were explored.
Results
Both versions of PSC had problems with item #7 (Act as if driven by motor), which did not load in any of the latent factors. It was removed from later analyses. The three-factor structure of PSC was confirmed. All remaining items had high factor loadings in their corresponded latent factors, and the reliability was high for the total scales (PSC-34-Y, ω = 0.78; PSC-16-Y, ω = 0.94) and the subscales of PSC-16-Y (Attention, ω = 0.77; Internalizing, ω = 0.79; Externalizing, ω = 0.78). The goodness of fit was adequate, and the correlation between PSC subscales and SDQ subscales was high. Victimization and perpetration were associated with all PSC subscales, and higher school climate and stronger school memberships were negatively associated with PSC symptoms.
Citation: Ramírez S, Gana S, Godoy MI, Valenzuela D, Araya R, Gaete J (2023) Validation of the Spanish version of the Pediatric Symptom Checklist (PSC) to identify and assess psychosocial problems among early adolescents in Chile. PLoS ONE 18(4): e0283921. https://doi.org/10.1371/journal.pone.0283921
Editor: Sergio A. Useche, University of Valencia: Universitat de Valencia, SPAIN
Received: April 20, 2021; Accepted: March 21, 2023; Published: April 6, 2023
Copyright: © 2023 Ramírez et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data files are available from UK Data Service: https://reshare.ukdataservice.ac.uk/cgi/users/home?screen=EPrint::View&eprintid=854843#t.
Funding: JG: Agencia Nacional de Investigación y Desarrollo (https://www.anid.cl) - AKA-EDU/15. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. JG: Agencia Nacional de Investigación y Desarrollo (https://www.anid.cl) - Millennium Science Initiative Program – NCS17_035. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Mental health problems among children and adolescents are prevalent and cause substantial disabilities over time [1,2]. Moreover, half of adult mental health disorders started their onset during adolescence [1,3]. Among adolescents, a meta-analysis published in 2015, which included data from 27 countries, reported a prevalence of 13.4% of psychiatric disorders [2], and a new meta-analysis published in 2020, collecting data from 19 countries, showed a prevalence of 31.0% [4]. The consequences of mental health problems in this population may have a lasting effect on their lives [5]. For instance, suicidal ideation and attempts are very frequent [6,7], and suicide is the second cause of mortality among young people [6,8]. Additionally, disability-adjusted life years (DALYs) among children aged 5–14 years old ranked second place among all causes of DALYs in the Americas and Europe [9], representing 10.7% and 11.4% of DALYs, respectively [9].
Another problem adolescents face today is the huge treatment gap found in different countries: most mental disorders become undetected, or there are many access barriers [10,11]. For instance, around 75% of children and adolescents with psychiatric disorders are not receiving treatment today [12]. Among these barriers, we can mention the lack of validated screening tools in several cultures, scarce implementation of their massive use, stigma and self-stigmatization [12–14], lacking of mental health infrastructure and services [11], few health professionals resources [11] and less than 1% of the health budget allocated to mental health in most low or middle-income countries [15].
Specifically, the availability and dissemination of instruments to identify psychosocial and behavioral problems [16] may help to reduce this gap. Indeed, several instruments are widely used to screen mental problems among adolescents [17–20], but many are not widely available in different cultures and contexts. Two recent systematic reviews found that the most commonly used instruments were the Child Behavior Checklist (CBCL), the Early Adolescent Temperament Questionnaire-Revised Version (EATQ-R), the Strengths and Difficulties Questionnaire (SDQ), Beck Youth Inventories (BYI), Behavior Assessment System for Children (BASC), Behavioral and Emotional Rating Scale (BERS), Child Health Questionnaire (CHQ), Child Symptom Inventories (CSI), and the Pediatric Symptoms Checklist (PSC) [17,18]. There are several advantages and disadvantages for each of them, but a detailed analysis of this issue is beyond the aims of this paper. However, we will provide information about some of these instruments to justify the necessity of validating the Pediatric Symptoms Checklist (PSC) in Chile.
The CBCL consists of 120 questions on problematic behaviors to be answered by the parents or carers of children aged 6–18 years old [21,22]. The current version of this questionnaire uses the algorithm of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) [21,22], and it is more specific about some disorders such as affective, anxiety, somatic, attention-deficit hyperactivity, oppositional defiant, and conduct problems. Among the disadvantages of this instrument are the extensiveness (which takes around 60 minutes to be answered) and that there is no self-reported version available for adolescents. The only validated version in Chile is for preschoolers (CBCL 1½-5) [23].
The EATQ-R measures the temperament and behavior of 15 years old adolescents [24,25]. It is available in self-report and parent-report formats. It has 103 items distributed in the following 13 subscales: activation control, activity level, affiliation, attention, fear, frustration, high-intensity pleasure, inhibitory control, perceptual sensitivity, pleasure sensitivity, shyness, aggression, and depressive mood [25]. In Chile, it has been validated among 12–18 years old adolescents [20]. However, it is a lengthy questionnaire, only two of the scales measure behavior problems, and we do not know the psychometric properties for a younger population (aged 9 to 11).
The SDQ consists of 25 items about emotional and behavioral problems in children and adolescents aged 4–16 [26]. It has three versions: self-reported, parents/carers-reported, and teachers-reported. It contains five subscales with five items each subscale: emotional symptoms, conduct problems, hyperactivity/attention problems, peer problems, and prosocial behavior [27]. The first four subscales are part of the total difficulties scale, while the last subscale is considered to be a part of the strengths [19]. In Chile, its psychometric properties have been studied with the parents/carers-reported version for children aged 4–11 years [28]. A recent validation study in Chile [19], conducted by the same research team, was conducted in a population of early adolescents (9–15 years old) with the self-report and parents/carers-reported versions. The main limitations found were related to the reliability of some of the subscales, especially for peer problems subscales (omega = 0.65, self-reported).
One important consideration when selecting an instrument is the availability of the psychometric properties of the scale for the age of the target population [29,30]. All of the studies carried out in Chile, except one [19], have this age limitation. Besides, most of the other studies showed less acceptable psychometric properties [20,23,28,31–34].
On the other hand, the PSC is an instrument that measures emotional and behavioral problems in children and adolescents between 4 and 18 years old [35]. The original questionnaire (PSC) has 35 items with a parent/carer-reported form, which showed good psychometric properties with high specificity (mean = 0.72) and sensitivity (mean = 0.88) values for identifying psychiatric disorders [36]. It has three subscales: attention, anxiety/depression (internalizing), and conduct (externalizing) [32]. Later on, a youth self-report questionnaire (PSC-Y, 35 items) was created. More recently, a shorter 17-item version for both parents (PSC-17) and youth (PSC-17-Y) [37] was produced based on the three original subscales, which were obtained from the three resulting factors [38]. This shorter version also showed good psychometric properties with an internal consistency test-retest of 0.85, for example [38]. This three-factor structure of the PSC provides evidence for the symptom clusterings of mental disorders initially proposed by Achenbach [39]: internalizing and externalizing, which currently are considered relevant to further incorporate the comorbidity concept in psychiatry [40].
In Chile, the first validation study was conducted over the parent version of the PSC 35-item [31] for first graders. This study found that two items ("absent from school and "worries a lot") were not reliable, so they were removed. The final version, the PSC-33-Spanish Chilean Version, has been widely used by the government of Chile and included as part of screening batteries of public programs, both in schools ("Life skills" program) [41] and in healthcare facilities [32]. Two other studies have used the parent-report format of the PSC-17 [32,33] but only among first graders. Therefore, no published studies have been conducted exploring the psychometric properties of the youth version of the Spanish version of the PSC-17-Y and PSC-Y, 35 items. Having self-reported information from adolescents may help to study the continuity of symptoms for those students who had participated in governmental programs when they were younger. Moreover, exploring the functioning of PSC-17-Y and PSC-Y, 35 items may give more options to professionals to select the suitable instruments according to population’ or patients’ necessities, measuring externalizing and internalizing problems, which appears to be a supported categorization to plan either prevention of treatment interventions [40].
Consequently, the aims of this study were: (1) to assess the validity of the internal factor structure of the subscales contained in the long (PSC-Y, 35 items) and brief (PSC-17-Y) young self-report questionnaires, (2) to explore the reliability of the subscales in both questionnaires, (3) to assess the concurrent validity of the resulting brief young self-report questionnaires with SDQ questionnaire; and (4) explore associations between several school risk and protective factors (bullying, school climate and sense of school membership) and the three subscales measured by the brief young self-report questionnaire.
Materials and methods
Study design
This is a cross-sectional study. This was part of a larger study aiming to assess the effectiveness of the antibullying program KiVa in Chile. Further details can be found in the clinical trial registration (ClinicalTrials.gov, NCT02898324)
Setting and sample
The study population consisted of students attending 5th or 6th grade (ages 10–11) from a total of 39 schools located in Santiago, Chile.
The inclusion criteria of the participating schools were the following: (1) Schools that have 5th -6th grades, (2) mixed schools, (3) located in Santiago city, (4) having two classes per grade, and (5) High vulnerability (>75%) measured by the Chilean index School Vulnerability Index-National System of Equality Allocation (IVE-SINAE). This index is built every year by the Ministry of Education of Chile and assigned to every school based on poverty conditions such as poor health, low family income, receiving state benefits, and the risk of school failure of the students. In other words, the IVE-SINAE index shows the proportion of students in a school who are in most need [42,43]. All participating students are required to be fluent in Spanish in spoken and written language. Exclusion criterium included: students who had a pre-existence cognitive disability.
A total of 4,495 students were eligible to participate and were invited. Of these, a total of 3,968 (88.3%) consented and responded to the questionnaire. From this total sample, and for analytical purposes, it was decided to divide it into two samples through a simple random sampling method. The first sample (n = 1,981) helped to explore the psychometric properties of the PSC-Y, 35 items, and the second sample (n = 1,987) helped to explore the psychometric properties of the PSC-17-Y.
Procedure and ethical considerations
The data was collected in November 2017 in accordance with the Declaration of Helsinki, with the approval of the ethics committee of the Universidad de los Andes (January 18th, 2016). First, school authorities were informed about the study aims, methods, and assessments, and afterward, they agreed to participate. Second, an informed and written consent form was sent to the parents or main caregivers in order to accept the participation of the students. Finally, informed and written assent was also provided by all students.
Measures
Sociodemographic variables.
We gathered the following information on sociodemographic factors: age (in years), gender (1 = Male; 2 = Female), grade class (1 = 5th grade; 2 = 6th grade), and School administrative dependency (1 = State school; 2 = Subsidized school). In Chile, around 93% of students attend state (40.4%) or subsidized (52.1%) schools [42].
PSC-Y, 35 item.
The Pediatric Symptom Checklist youth self-report version is a questionnaire that helps identify and assess changes in emotional and behavioral problems in children and early adolescents. It has 35 items. Each item is rated as "Never", "Sometimes" and "Often" (scored 0, 1, and 2, respectively). Item scores are summed. It has three subscales, with the following respective items (numbers are indicated in relation to the PSC 35 items): Attention (#4 "Fidgety, unable to sit still", #7 "Acts as if driven by a motor") #8 "Daydreams too much", #9 "Distracted easily "and #14 "Has trouble concentrating". Anxiety/Depression or Internalizing subscale (#11 "Feels sad or unhappy", 13 "Feels hopeless", #19 "Is down on him or herself", #22 "Worries a lot" and #27 "Seems to be having less fun") and Conduct or Externalizing subscale (#16 "Fights with others", #29 "Does not listen to rules", #31 "Does not understand other people’s feelings", #32 "Teases others", #33 "Blames others for his or her troubles", #34 "Takes things that do not belong to him or her" and #35 "Refuses to share"). This instrument has been validated in several languages internationally and demonstrated good psychometric properties [35,37,38,44,45]. No validation of the Spanish version of this instrument is available in Chile.
PSC-17-Y.
The PSC-17-Y is a brief youth self-reported questionnaire with a total of 17 items obtained from the three subscales of the youth self-reported 35-item version of PSC (PSC-Y). This version has also been validated [46] in several languages. No validation of the Spanish version of this instrument is available in Chile.
Strengths and Difficulties Questionnaire (SDQ).
The SDQ self-report questionnaire consists of 25 items aiming to assess emotional and behavioral problems among children and adolescents aged 4–17 years old [24]. It has five subscales with five items each: emotional symptoms, conduct problems, hyperactivity/attention problems, peer problems, and prosocial behavior [25]. The first four subscales are part of the difficulties scale, while the last subscale is considered the strengths scale [17]. It has good psychometric properties reported by its author [26] and in a Chilean validation study among adolescents [19]
Revised Olweus Bully/Victim Questionnaire.
The Revised Olweus Bully/Victim Questionnaire (OBVQ-R) is a self-reported 40 items questionnaire that measures bullying and victimization in different ways, such as physical, verbal, racial, sexual, etc. [47]. This questionnaire has been validated and has good psychometric properties [48–50]. Three items from this questionnaire were used to assess associations of three kinds of bullying participation: victimization (Have you been bullied at school in the last two months?), perpetrator (Have you bullied a student at school in the last two months?), and bystanders (Have you watched or witnessed that a student has been bullied at school in the last two months?). These items were answered using a 5-point scale (1 = Never to 5 = Several times a week). For analytical purposes, all these answers were dichotomized (0 = Never; 1 = Yes).
The Psychological Sense of School Membership (PSSM).
PSSM is a self-reported instrument developed to assess the sense of school belonging. The original PSSM scale comprises 18 items: 13 positively worded statements and five negatively worded statements [51]. All of these items are related to students’ perceptions of being accepted, respected, included, and supported by others in the school social environment. In Chile, this instrument has been validated [52], resulting in a 13-item questionnaire containing 13 positively worded statements. For each statement, students answer on a 5-point scale (1 = not at all true; 5 = completely true). Therefore, a higher score means a higher sense of school membership.
School climate.
This scale was used in the studies conducted in Finland to assess the effectiveness of the antibullying KiVa program [53], and it collects information about the perception of well-being by students at the school. Originally it had five items, but in the adaptation to Chile, four items had good psychometric parameters: "I feel safe at school", "I’m happy to be at my class", "I feel accepted when I am at the school" and "I like the school climate". For each statement, students answer on a 4-point scale (0 = strongly disagree; 4 = strongly agree). Therefore, a higher score means a better school climate. For this particular sample, the scale psychometric properties were good, showing a Cronbach’s alpha of 0.84, and also the variance explained by items of the scale was 82.4%.
Data analysis
Descriptive analysis.
First, a descriptive analysis of sociodemographic variables was performed with measures of variance by calculating 95% confidence intervals and standard deviation for the age by class grade variable. In addition, a measure of central tendency was determined by calculating the mean age. Second, to represent the relative frequencies, percentages were calculated for the gender, class grade, and administrative school dependency variables. Third, a description of correlations of the PSC-Y and PSC-17-Y items and their respective subscales with the total score was given by Spearman’s Rho test with their respective p-value, where a figure of +1 indicates a perfect positive association, a figure of -1 indicates a perfect negative association and 0 indicates no association. Ranges of 0.70–0.99 (+/-) will be considered a strong association, 0.40–0.69 (+/-) a moderate association, and 0.01–0.39 (+/-) a weak association [54]. A p-value ≤ 0.05 will be considered for statistical significance. Finally, the resulting scale, tested and adapted from the PSC-17-Y, was analyzed, calculating the means, medians, standard deviations, kurtoses, skewnesses, and factor loadings. Factor loading >0.70 was considered optimal, 0.40–0.70 was considered moderate, and 0.30–0.39 was considered minimal [55].
Dimensionality and reliability analyses.
The validity of the original versions of PSC-Y, PSC-17-Y, and the resulting versions PSC-34-Y and PSC-16-Y were tested using confirmatory factor analysis (CFA) with unweighted least squares (ULS) of the polychoric matrix due to the items ordinal distributions. The goodness of fit of the proposed models was evaluated using the following parameters: (1) the Root Mean Square Error of Approximation (RMSEA), where values below or equal to 0.05 were considered to be a good fit and values below or equal 0.08 were considered to be an acceptable fit [56,57]; (2) the Standardized Root Mean Square Residual (SRMR), where values below or equal to 0.10 were considered to be a good or acceptable fit [57]; (3) the Normalized Fit Index (NFI), where values above or equal to 0.95 were considered to be a good fit and values above 0.90 were considered to be an acceptable fit [58], (4) Non-Normed Fit Index (NNFI), where values above or equal to 0.97 indicates a good fit, and values above or equal 0.95 indicates an acceptable fit [59], (5) the Comparative Adjustment Index (CFI), where values above or equal to 0.97 were considered to be a good adjustment values above or equal 0.90 indicates an acceptable fit [57], (6) the Goodness of Fit Index (GFI), where values above or equal to 0.95 were considered to be a good fit and values above or equal to 0.90 were considered an acceptable fit [56]; and (7) the Adjusted Goodness of Fit Index (AGFI), where values above or equal to 0.90 were considered to be a good or an acceptable fit [56]. In addition, reliability was evaluated using the omega coefficient, where values of 0.65 or more are considered acceptable [60].
Concurrent validity was assessed between the three subscales of the resulting PSC-16-Y (attention, internalizing, and externalizing subscales) and the five subscales of the SDQ questionnaire (emotional problems, peer problems, conduct problems, hyperactivity problems, and prosocial behavior). This was evaluated with a Spearman correlation test [54]. A p-value ≤ 0.05 will be considered for statistical significance.
All dimensionality, reliability, and item descriptive analyses were performed in both instruments, PSC-Y and PSC-17-Y. We decided to present in the text the results of the PSC-17-Y, but all the analyses of the PSC-Y are included in the supplements.
Association analysis.
We explored the associations of bullying participation, such as victimization, perpetration, and bystander, from the OBVQ-R questionnaire with the resulting PSC-16-Y, using the total score and subscale scores as dependent variables. We also explored the association of sense of school membership and school climate with the resulting PSC-16-Y. All of these dependent variables were analyzed as continuous variables, performing multivariable regression analyses and adjusting each model by gender and age. The unstandardized regression coefficient with 95% confidence intervals is presented, and the cutoff for statistical significance was established with a p-value < 0.05.
Descriptive analyses of the scales, internal structure statistical analyses, and concurrent validity analysis were conducted using R 3.5.0 software, and CFA was performed using the lavaan package in R 3.5.0. The significance level was stated at 0.05. Finally, sociodemographic descriptive analyses and association analyses were conducted using Stata 15.
Results
Description of the sociodemographic variables
A total of 3,968 students participated. 53,8% [52.3–55.4] were male, 52.1 [50.6–53.7] attended 6th grade. The mean age was 9.8 (SD = 0.7) years old for 5th graders and 10.8 (SD = 0.8) years old for 6th graders. The students attended schools with different administrative dependencies: 50.1% [48.5–51.7] attended a state school, and 49.9% [48.3–51.5] attended a subsidized school. Both random samples had a similar distribution of sociodemographic variables (Table 1).
Description of PSC-17-Y items
Most correlation coefficients between each item of PSC-Y and the PSC-17-Y with the total score of the respective subscale were moderate but significant. The only exception was found for the item "#7, Acts as if driven by motor", which did not correlate with the "Attention subscale" nor with the total score. This result was used to eliminate this item from the questionnaire. Later, we tested all psychometric properties for the resulting new subscales: "PSC-34-Y" and "PSC-16-Y". It is worth mentioning that the authors had authorization from the original authors of the PSC questionnaire to modify the questionnaire if needed (Table 2).
Descriptive Statistics and factor loading of PSC-16-Y items
All items had moderate to optimal factor loading figures. The higher mean scores were found for items from the Attention (# 4, Fidgety, unable to sit still; # 9, Distracted easily) and Internalizing subscales (# 13, Feels hopeless; (# 27, Seems to be having less fun) (See Table 3 and Fig 1).
Confirmatory factor analysis and reliability of PSC
All goodness of fit indicators from the confirmatory factor analysis had a good or acceptable fit, for both the original versions of PSC (including the item "#7, Acts as if driven by motor") and the new PSC versions, with very small differences in performance in favor of the new PSC versions (Table 4).
Regarding reliability, all subscales had an acceptable omega coefficient in all of the questionnaire versions, except for the Attention subscale of the PSC-17-Y, which had an omega value of 0.67 (Table 5).
Concurrent validity with SDQ questionnaire
All PSC-16-Y subscales had a weak, although significant, correlation with all of the SDQ subscales. The Attention subscale had a higher correlation with the Hyperactivity subscale of the SDQ. The Internalizing subscale had a higher correlation with the Emotional Symptoms subscale of SDQ (0.38). The Externalizing subscale had a higher correlation with the Conduct Problems subscale of SDQ (0.39). All three PSC-16-Y subscales had a negative correlation with the Prosocial Behavior subscale of SDQ (Table 6).
School factors associated with PSC-16-Y
Regarding bullying experience, the strongest association with the PSC-16-Y total score was bullying perpetration (β = 3.19, p<0.001). Victimization was highly associated with Internalizing subscale score (β = 1.02, p<0.001), and the score in the perpetration subscale was highly associated with Externalizing subscale score (β = 1.50, p<0.001).
A better school climate and a stronger school membership were negatively associated with the PSC-16-Y total score, and with all subscales (Table 7A–7D).
b: Risk and protective factors associations with Attention subscale score of PSC-16-Y. c: Risk and protective factors associations with Internalizing subscale score of PSC-16-Y. d: Risk and protective factors associations with Externalizing subscale score of PSC-16-Y.
Discussion
To our knowledge, this is the first validation study of the Spanish version of the PSC-Y questionnaire among early adolescents in Latin America. Our findings support the decision to eliminate item #7, "Acts as if driven by motor" to improve the psychometric properties of the instruments. The resulting PSC-Y questionnaires, PSC-34-Y and PSC-16-Y, had a good structure of the items. The construct validation indicators using confirmatory factor analysis showed that all of the new subscales had good or adequate goodness of fit adjustment. We confirmed the original three-factor structure of the PSC questionnaire (Attention, Internalizing, and Externalizing subscales) [38], and there was a good correlation between the PSC-16-Y total score and its subscales and the subscales of the SDQ [19]. On the other hand, the reliability was good or acceptable for the total scale and for all subscales.
We decided to eliminate the item "Acts as if driven by motor" because the psychometric analyses indicated that this statement was, very likely, rather understood as a positive feature by adolescents instead of considering it as a negative behavior. In the translation and adaptation process, we explored several Spanish versions of the item, including the version proposed by authors ("Eres incansable"), and, in all the cases, the statement was considered as a positive behavior. Additionally, the literal translation of the item does not have a clear meaning in Spanish. Therefore, it is advisedly to eliminate this item, especially if we also consider that the Attention subscale, where this item belongs, already has a related item ("Fidgety, unable to sit still") which captures the same problematic symptom that "Acts as if driven by motor" intents to assess.
There are much fewer validation studies of the PSC-Y, 35 items, and PSC-17-Y (self-reported), compared with the number of studies exploring the validity of PSC, 35 items, and PSC-17 (parent/caregiver-reported). For instance, in one study [61] using the PSC-Y (35 items) to detect high-risk adolescents, it was found that this population had a higher likelihood of seeking help, a higher prevalence of previous mental health treatment, and a higher chance of coming from low-income families. Therefore, the PSC helped to identify relevant features of the population most at risk. Another study in the USA [62], using the PSC (35 items), reported good concurrent validity of the self-reported instrument with the teacher and parent-reported versions of the PSC. However, the reliability, using a test-retest procedure, was only acceptable (kappa of 0.50). Different contextual and cultural variables may affect the performance of this instrument, supporting the need for local studies of its validation.
On the other hand, regarding the PSC-17-Y, one study [63] found a good Cronbach’s alpha (alpha = 0.78) for the total scale, but the reliability of its subscales was not reported. Finally, in another study [64] on the child welfare population, it was found that internalizing subscale was strongly correlated, the attention subscale was moderately correlated, and externalizing subscale was weakly correlated with the Screen for Child Anxiety Related Disorders scales.
All these studies can be seen as complementary to each other; however, the scarce number of studies, limited information reported on the psychometric features, and the specificity of the population studied, justify the need for our study and the importance of the findings in a general population of adolescents.
Finally, there are only three other studies exploring the validity of the PSC questionnaire in Chile, but in all cases, they used the parent version of the PSC. The first study assessed the concurrent validity of the PSC (35 items), using the Teacher Observation of Classroom Adaptation–Revised (TOCA-R), and its four factors. The correlations between the total score of PSC and the four factors were acceptable (between 0.126 and 0.4399 [31]. Nevertheless, Cronbach’s alpha of the total score was good (0.853). Another study explored the structure of PSC (35 items); however, it did not find the proposed three-factor structure but a six-factor structure. They also explored the concordance of these six factors with the psychiatric diagnosis using the Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime version (Kiddie-SADS-PL), showing that only half of the cases had a good concordance, but the sample was small to reach clear conclusions [33]. Finally, a third study [32] explored the concurrent validity of the PSC-17 with the TOCA-R, finding moderate correlations between them [32,54]. In the same study, the reliability for the subscales was 0.72 for the externalizing subscale, 0.76 for the internalizing subscale, and 0.61 for the attention subscale. This study also performed a confirmatory factor analysis finding the three-factor structure of the PSC, similar to our study.
Among the strengths of our study, we can mention the following: we surveyed a large sample size (N = 3,968); the validation was explored in both versions of the PSC-Y (35 and 17 items); and we performed several statistical analyses in order to have a wide range to sources of validation, such as confirmatory factor analyses and Spearman’s correlation with another validated questionnaire. In addition, we explored the associations between different risk and protective factors and the PSC-Y-16 subscales as outcomes. These questionnaires can be administered easily in school settings which may help to identify students at risk and refer to proper early intervention or treatment in health services if needed.
This study has some limitations. First, this was a self-reported questionnaire, so information bias may exist, especially related to social desirability. Second, we only included students attending schools with high socioeconomic vulnerability, so the results may not represent adolescents from more affluent families. However, most students in Chile attend schools with high socioeconomic vulnerability; therefore, the results can be generalizable to most of the school population in Chile and perhaps in Latin America. Third, given the proposed changes in the content of the questionnaire in the Spanish version studied here compared with the original English version, the cross-cultural comparisons in the prevalence of mental health problems among adolescents should be made with caution. And finally, this was a cross-sectional study, so no causality can be concluded regarding the associations.
In all, the Spanish version of the PSC-Y questionnaire has good indicators of its validity in Chile, and our study reduces the knowledge gap of this information among early adolescents. This questionnaire may be used by the Government of Chile, other Latin American countries, and health and school professionals to identify at-risk populations. Future research also needs to determine the cutoff score in order to better detect adolescents who need help and to find other personal and contextual variables associated with mental health problems in order to plan preventive interventions.
Supporting information
S2 File. English version of PSC-Y Questionnaire.
https://doi.org/10.1371/journal.pone.0283921.s002
(DOCX)
S3 File. English version of PSC-17-Y Questionnaire.
https://doi.org/10.1371/journal.pone.0283921.s003
(DOCX)
Acknowledgments
Special thanks to all of the students and collaborators who participated in this study.
References
- 1. García-Carrión R, Villarejo-Carballido B, Villardón-Gallego L. Children and Adolescents Mental Health: A Systematic Review of Interaction-Based Interventions in Schools and Communities. Front Psychol. 2019;10:918. Epub 2019/05/10. pmid:31068881; PubMed Central PMCID: PMC6491840.
- 2. Polanczyk GV, Salum GA, Sugaya LS, Caye A, Rohde LA. Annual research review: A meta-analysis of the worldwide prevalence of mental disorders in children and adolescents. J Child Psychol Psychiatry. 2015;56(3):345–65. Epub 20150203. pmid:25649325.
- 3. Davidson LL, Grigorenko EL, Boivin MJ, Rapa E, Stein A. A focus on adolescence to reduce neurological, mental health and substance-use disability. Nature. 2015;527(7578):S161–6. Epub 2015/11/19. pmid:26580322.
- 4. Silva SA, Silva SU, Ronca DB, Gonçalves VSS, Dutra ES, Carvalho KMB. Common mental disorders prevalence in adolescents: A systematic review and meta-analyses. PLoS One. 2020;15(4):e0232007. Epub 2020/04/24. pmid:32324835; PubMed Central PMCID: PMC7179924.
- 5. Li T, Wang L, Camilleri JA, Chen X, Li S, Stewart JL, et al. Mapping common grey matter volume deviation across child and adolescent psychiatric disorders. Neurosci Biobehav Rev. 2020;115:273–84. Epub 2020/06/03. pmid:32485265.
- 6. Orri M, Scardera S, Perret LC, Bolanis D, Temcheff C, Séguin JR, et al. Mental Health Problems and Risk of Suicidal Ideation and Attempts in Adolescents. Pediatrics. 2020;146(1). Epub 2020/06/10. pmid:32513840.
- 7. Miché M, Hofer PD, Voss C, Meyer AH, Gloster AT, Beesdo-Baum K, et al. Mental disorders and the risk for the subsequent first suicide attempt: results of a community study on adolescents and young adults. Eur Child Adolesc Psychiatry. 2018;27(7):839–48. Epub 2017/10/14. pmid:29027588; PubMed Central PMCID: PMC6013520.
- 8. Gili M, Castellví P, Vives M, de la Torre-Luque A, Almenara J, Blasco MJ, et al. Mental disorders as risk factors for suicidal behavior in young people: A meta-analysis and systematic review of longitudinal studies. J Affect Disord. 2019;245:152–62. Epub 2018/11/06. pmid:30390504.
- 9. Baranne ML, Falissard B. Global burden of mental disorders among children aged 5–14 years. Child Adolesc Psychiatry Ment Health. 2018;12:19. Epub 2018/04/24. pmid:29682005; PubMed Central PMCID: PMC5896103.
- 10. Michaud PA, Visser A, Vervoort JPM, Kocken P, Reijneveld SA, Jansen D. Availability and accessibility of primary mental health services for adolescents: an overview of national recommendations and services in EU. Eur J Public Health. 2020. Epub 2020/08/21. pmid:32820338.
- 11. Kohn R, Ali AA, Puac-Polanco V, Figueroa C, López-Soto V, Morgan K, et al. Mental health in the Americas: an overview of the treatment gap. Rev Panam Salud Publica. 2018;42:e165. Epub 2019/05/17. pmid:31093193; PubMed Central PMCID: PMC6386160.
- 12. Aguirre Velasco A, Cruz ISS, Billings J, Jimenez M, Rowe S. What are the barriers, facilitators and interventions targeting help-seeking behaviours for common mental health problems in adolescents? A systematic review. BMC Psychiatry. 2020;20(1):293. Epub 2020/06/13. pmid:32527236; PubMed Central PMCID: PMC7291482.
- 13. Goodwin J, Savage E, Horgan A. Adolescents’ and Young Adults’ Beliefs about Mental Health Services and Care: A Systematic Review. Arch Psychiatr Nurs. 2016;30(5):636–44. Epub 2016/09/23. pmid:27654250.
- 14. Radez J, Reardon T, Creswell C, Lawrence PJ, Evdoka-Burton G, Waite P. Why do children and adolescents (not) seek and access professional help for their mental health problems? A systematic review of quantitative and qualitative studies. Eur Child Adolesc Psychiatry. 2020. Epub 2020/01/23. pmid:31965309.
- 15. Patel V, Chisholm D, Parikh R, Charlson FJ, Degenhardt L, Dua T, et al. Addressing the burden of mental, neurological, and substance use disorders: key messages from Disease Control Priorities, 3rd edition. Lancet. 2016;387(10028):1672–85. Epub 2015/10/12. pmid:26454360.
- 16. Salamanca-Buentello F, Seeman MV, Daar AS, Upshur REG. The ethical, social, and cultural dimensions of screening for mental health in children and adolescents of the developing world. PLoS One. 2020;15(8):e0237853. Epub 2020/08/25. pmid:32834012; PubMed Central PMCID: PMC7446846.
- 17. Tsang KL, Wong PY, Lo SK. Assessing psychosocial well-being of adolescents: a systematic review of measuring instruments. Child Care Health Dev. 2012;38(5):629–46. Epub 2011/12/16. pmid:22168129.
- 18. Deighton J, Croudace T, Fonagy P, Brown J, Patalay P, Wolpert M. Measuring mental health and wellbeing outcomes for children and adolescents to inform practice and policy: a review of child self-report measures. Child Adolesc Psychiatry Ment Health. 2014;8:14. Epub 2014/05/17. pmid:24834111; PubMed Central PMCID: PMC4022575.
- 19. Gaete J, Montero-Marin J, Valenzuela D, Rojas-Barahona CA, Olivares E, Araya R. Mental health among children and adolescents: Construct validity, reliability, and parent-adolescent agreement on the ’Strengths and Difficulties Questionnaire’ in Chile. PLoS One. 2018;13(2):e0191809. Epub 2018/02/06. pmid:29401472; PubMed Central PMCID: PMC5798763.
- 20. Hoffmann M, Pérez JC, García C, Rojas G, Martínez V. Chilean Adaptation and Validation of the Early Adolescent Temperament Questionnaire-Revised Version. Front Psychol. 2017;8:2131. Epub 2018/01/13. pmid:29326616; PubMed Central PMCID: PMC5742495.
- 21. Achenbach TM, Dumenci L, Rescorla LA. DSM-oriented and empirically based approaches to constructing scales from the same item pools. J Clin Child Adolesc Psychol. 2003;32(3):328–40. Epub 2003/07/26. pmid:12881022.
- 22. Cianchetti C, Faedda N, Pasculli M, Ledda MG, Diaz G, Peschechera A, et al. Predictive validity for the clinical diagnosis of a new parent questionnaire, the CABI, compared with CBCL. Clin Child Psychol Psychiatry. 2020;25(2):507–19. Epub 2020/01/03. pmid:31894698.
- 23. Lecannelier F, Pérez Ewert JC, Groissman S, Gallardo D, Bardet AM, Bascuñan A, et al. Validation of Child Behavior Inventory for Children ages 1½ to 5 years (CBCL 1½-5) at the Santiago de Chile City. Universitas Psychologica. 2014;13(2):491–500.
- 24. Capaldi DM, Rothbart MK. Development and validation of an early adolescent temperament measure. The Journal of Early Adolescence. 1992;12(2):153–73.
- 25.
Ellis LK, Rothbart MK, editors. Revision of the early adolescent temperament questionnaire. Poster presented at the 2001 biennial meeting of the society for research in child development, Minneapolis, Minnesota; 2001.
- 26. Goodman R. The Strengths and Difficulties Questionnaire: a research note. J Child Psychol Psychiatry. 1997;38(5):581–6. Epub 1997/07/01. pmid:9255702.
- 27. Harry ML, Acevedo J, Crea TM. Assessing the factor structure of the Spanish language parent Strengths and Difficulties Questionnaire (SDQ) in Honduras. PLoS One. 2019;14(3):e0214394. Epub 2019/03/29. pmid:30921369; PubMed Central PMCID: PMC6438563.
- 28. Brown P, Capella C, Antivilo A. Psychometric properties of the parent version of the Strengths and Difficulties Questionnaire. Revista de Psicología. 2014;23(2):28–44.
- 29. Carvajal A, Centeno C, Watson R, Martínez M, Rubiales AS. [How is an instrument for measuring health to be validated?]. An Sist Sanit Navar. 2011;34(1):63–72. Epub 2011/05/03. pmid:21532647.
- 30. Bolarinwa OA. Principles and methods of validity and reliability testing of questionnaires used in social and health science researches. Niger Postgrad Med J. 2015;22(4):195–201. Epub 2016/01/19. pmid:26776330.
- 31. George M, Siraqyan X, Morales R, De La FB, Rodríguez J, López C, et al. Adaptación y validación de dos instrumentos de pesquisa de problemas de salud mental en escolares de 1 básico. Revista de Psicología. 1994;5:17–26.
- 32. Leiva L, Rojas R, Peña F, Vargas B, Scquicciarini AM. Detectando las Dificultades Emocionales y Conductuales en la Escuela: Validación de PSC-17. Revista Iberoamericana de Diagnóstico y Evaluación Psicologica. 2019.
- 33. Halpern Gasman ME. Detección temprana de psicopatología en escolares vulnerables de la Región Metropolitana. 2011.
- 34. Cronbach LJ. Coefficient alpha and the internal structure of tests. psychometrika. 1951;16(3):297–334.
- 35. Jellinek MS, Murphy JM, Robinson J, Feins A, Lamb S, Fenton T. Pediatric Symptom Checklist: screening school-age children for psychosocial dysfunction. J Pediatr. 1988;112(2):201–9. Epub 1988/02/01. pmid:3339501.
- 36. Lavigne JV, Meyers KM, Feldman M. Systematic Review: Classification Accuracy of Behavioral Screening Measures for Use in Integrated Primary Care Settings. J Pediatr Psychol. 2016;41(10):1091–109. Epub 2016/06/12. pmid:27289069.
- 37. Jellinek MS, Murphy JM, Burns BJ. Brief psychosocial screening in outpatient pediatric practice. J Pediatr. 1986;109(2):371–8. Epub 1986/08/01. pmid:3734977.
- 38. Murphy JM, Bergmann P, Chiang C, Sturner R, Howard B, Abel MR, et al. The PSC-17: Subscale Scores, Reliability, and Factor Structure in a New National Sample. Pediatrics. 2016;138(3). Epub 2016/08/16. pmid:27519444; PubMed Central PMCID: PMC5005018 and owners of the Child Health and Development Interactive System and receive salary from its licensing and use. The other authors have no conflicts of interest relevant to this article to disclose.
- 39. Achenbach TM. The classification of children’s psychiatric symptoms: a factor-analytic study. Psychol Monogr. 1966;80(7):1–37. Epub 1966/01/01. pmid:5968338.
- 40. Achenbach TM, Ivanova MY, Rescorla LA, Turner LV, Althoff RR. Internalizing/Externalizing Problems: Review and Recommendations for Clinical and Research Applications. J Am Acad Child Adolesc Psychiatry. 2016;55(8):647–56. Epub 2016/07/28. pmid:27453078.
- 41. Guzmán J, Kessler RC, Squicciarini AM, George M, Baer L, Canenguez KM, et al. Evidence for the effectiveness of a national school-based mental health program in Chile. J Am Acad Child Adolesc Psychiatry. 2015;54(10):799–807.e1. Epub 2015/09/27. pmid:26407489.
- 42. Gaete J, Valenzuela D, Rojas-Barahona C, Valenzuela E, Araya R, Salmivalli C. The KiVa antibullying program in primary schools in Chile, with and without the digital game component: study protocol for a randomized controlled trial. Trials. 2017;18(1):75. Epub 2017/02/22. pmid:28219403; PubMed Central PMCID: PMC5319041.
- 43.
Agencia de Calidad de la Educación. Metodología de construcción de grupos socioeconómicos pruebas SIMCE 2013 2013 [cited 2020 18 Jun]. Available from: http://archivos.agenciaeducacion.cl/Metodologia_de_Construccion_de_Grupos_Socioeconomicos_Simce_2013.pdf.
- 44. Murphy JM, Jellinek M, Milinsky S. The Pediatric Symptom Checklist: validation in the real world of middle school. J Pediatr Psychol. 1989;14(4):629–39. Epub 1989/12/01. pmid:2607398.
- 45. Erdogan S, Ozturk M. Psychometric evaluation of the Turkish version of the Pediatric Symptom Checklist-17 for detecting psychosocial problems in low-income children. J Clin Nurs. 2011;20(17–18):2591–9. Epub 2011/03/10. pmid:21385256.
- 46. Borowsky IW, Mozayeny S, Ireland M. Brief psychosocial screening at health supervision and acute care visits. Pediatrics. 2003;112(1 Pt 1):129–33. Epub 2003/07/03. pmid:12837878.
- 47. Olweus D. Bully/victim problems in school. Prospects. 1996;26(2):331–59.
- 48. Kyriakides L, Kaloyirou C, Lindsay G. An analysis of the Revised Olweus Bully/Victim Questionnaire using the Rasch measurement model. Br J Educ Psychol. 2006;76(Pt 4):781–801. Epub 2006/11/11. pmid:17094886.
- 49. Gothwal VK, Sumalini R, Irfan SM, Giridhar A, Bharani S. Revised Olweus Bully/Victim Questionnaire: evaluation in visually impaired. Optom Vis Sci. 2013;90(8):828–35. Epub 2013/06/25. pmid:23792363.
- 50. Gaete J, Valenzuela D, Godoy MI, Rojas-Barahona CA, Salmivalli C, Araya R. Validation of the Revised Olweus Bully/Victim Questionnaire (OBVQ-R) Among Adolescents in Chile. Frontiers in Psychology. 2021;12. pmid:33912096
- 51. Goodenow C. The psychological sense of school membership among adolescents: Scale development and educational correlates. Psychology in the Schools. 1993;30(1):79–90.
- 52. Gaete J, Montero-Marin J, Rojas-Barahona CA, Olivares E, Araya R. Validation of the Spanish Version of the Psychological Sense of School Membership (PSSM) Scale in Chilean Adolescents and Its Association with School-Related Outcomes and Substance Use. Front Psychol. 2016;7:1901. Epub 2016/12/22. pmid:27999554; PubMed Central PMCID: PMC5138456.
- 53. Kärnä A, Voeten M, Little TD, Poskiparta E, Kaljonen A, Salmivalli C. A large-scale evaluation of the KiVa antibullying program: grades 4–6. Child Dev. 2011;82(1):311–30. Epub 2011/02/05. pmid:21291444.
- 54. Akoglu H. User’s guide to correlation coefficients. Turk J Emerg Med. 2018;18(3):91–3. Epub 2018/09/08. pmid:30191186; PubMed Central PMCID: PMC6107969.
- 55. Lloret-Segura S, Ferreres-Traver A, Hernandez-Baeza A, Tomas-Marco I. Exploratory item factor analysis: A practical guide revised and updated. Anales de Psicología. 2014;30(3):1151–69.
- 56. MacCallum RC, Hong S. Power Analysis in Covariance Structure Modeling Using GFI and AGFI. Multivariate Behav Res. 1997;32(2):193–210. Epub 1997/04/01. pmid:26788758.
- 57.
Harrington D. Confirmatory factor analysis: Oxford university press; 2009.
- 58. Lt Hu, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: a multidisciplinary journal. 1999;6(1):1–55.
- 59.
Jöreskog KG, Sörbom D. LISREL 8: Structural equation modeling with the SIMPLIS command language: Scientific Software International; 1993.
- 60.
McDonald R. Test Theory New York: Psychology Press; 2013 [cited 2020 18 Jun].
- 61. Murphy JM, Nguyen T, Lucke C, Chiang C, Plasencia N, Jellinek M. Adolescent Self-Screening for Mental Health Problems; Demonstration of an Internet-Based Approach. Acad Pediatr. 2018;18(1):59–65. Epub 2017/09/06. pmid:28870650.
- 62. Pagano ME, Cassidy LJ, Little M, Murphy JM, Jellinek MS. IDENTIFYING PSYCHOSOCIAL DYSFUNCTION IN SCHOOL-AGE CHILDREN: THE PEDIATRIC SYMPTOM CHECKLIST AS A SELF-REPORT MEASURE. Psychol Sch. 2000;37(2):91–106. Epub 2000/03/01. pmid:22328794; PubMed Central PMCID: PMC3274771.
- 63. Lowenthal E, Lawler K, Harari N, Moamogwe L, Masunge J, Masedi M, et al. Validation of the Pediatric Symptom Checklist in HIV-infected Batswana. J Child Adolesc Ment Health. 2011;23(1):17–28. Epub 2011/01/01. pmid:22685483; PubMed Central PMCID: PMC3370684.
- 64. Parker EM, Jacobson J, Pullmann MD, Kerns SEU. Identifying Psychosocial Problems Among Children and Youth in the Child Welfare System Using the PSC-17: Exploring Convergent and Discriminant Validity with Multiple Informants. Child Psychiatry Hum Dev. 2019;50(1):108–20. Epub 2018/07/02. pmid:29961167.