School mental health services are important contact points for children and adolescents with mental disorders, but their ability to provide comprehensive treatment is limited. The main objective was to estimate in mentally disordered adolescents of a nationally representative United States cohort the role of school mental health services as guide to mental health care in different out-of-school service sectors.
Analyses are based on weighted data (N = 6483) from the United States National Comorbidity Survey Replication Adolescent Supplement (participants' age: 13–18 years). Lifetime DSM-IV mental disorders were assessed using the fully structured WHO CIDI interview, complemented by parent report. Adolescents and parents provided information on mental health service use across multiple sectors, based on the Service Assessment for Children and Adolescents.
School mental health service use predicted subsequent out-of-school service utilization for mental disorders i) in the medical specialty sector, in adolescents with affective (hazard ratio (HR) = 3.01, confidence interval (CI) = 1.77–5.12), anxiety (HR = 3.87, CI = 1.97–7.64), behavior (HR = 2.49, CI = 1.62–3.82), substance use (HR = 4.12, CI = 1.87–9.04), and eating (HR = 10.72, CI = 2.31–49.70) disorders, and any mental disorder (HR = 2.97, CI = 1.94–4.54), and ii) in other service sectors, in adolescents with anxiety (HR = 3.15, CI = 2.17–4.56), behavior (HR = 1.99, CI = 1.29–3.06), and substance use (HR = 2.48, CI = 1.57–3.94) disorders, and any mental disorder (HR = 2.33, CI = 1.54–3.53), but iii) not in the mental health specialty sector.
Our findings indicate that in the United States, school mental health services may serve as guide to out-of-school service utilization for mental disorders especially in the medical specialty sector across various mental disorders, thereby highlighting the relevance of school mental health services in the trajectory of mental care. In light of the missing link between school mental health services and mental health specialty services, the promotion of a stronger collaboration between these sectors should be considered regarding the potential to improve and guarantee adequate mental care at early life stages.
Citation: Tegethoff M, Stalujanis E, Belardi A, Meinlschmidt G (2014) School Mental Health Services: Signpost for Out-of-School Service Utilization in Adolescents with Mental Disorders? A Nationally Representative United States Cohort. PLoS ONE 9(6): e99675. doi:10.1371/journal.pone.0099675
Editor: Kenji Hashimoto, Chiba University Center for Forensic Mental Health, Japan
Received: December 16, 2013; Accepted: May 19, 2014; Published: June 9, 2014
Copyright: © 2014 Tegethoff et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This project was financed by the Swiss National Science Foundation (SNSF, to MT, project no. PZ00P1_137023). GM received SNSF funding under project no. 100014_135328. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The National Comorbidity Survey Replication Adolescent Supplement (NCS-A) was funded by: United States Department of Health and Human Services, National Institutes of Health, National Institute of Mental Health (U01-MH60220); United States Department of Health and Human Services, National Institutes of Health, National Institute of Drug Abuse (R01-DA12058-05); United States Department of Health and Human Services, Substance Abuse and Mental Health Services Administration; Robert Wood Johnson Foundation (Grant 044780); John W. Alden Trust.
Competing interests: The authors have declared that no competing interests exist.
Mental disorders place a great challenge on the health care system. They are highly prevalent not only in adults  but also in adolescents and children , , , , , , with enormous implications for health-related quality of life  and health economy . The urgent need to integrate mental health into all aspects of health research and health-care delivery has been realized by major health institutions and scientific journals, as documented for example in The Lancet series on Global Mental Health and strategic research goals and health action plans by the Grand Challenges in Global Mental Health initiative and the American Academy of Pediatrics , , .
Many children and adolescents with mental disorders do not receive adequate care , , even though effective treatment approaches for mental disorders exist . The median interval between the first onset of a mental disorder and first contact with the treatment sector is nearly a decade , although early intervention offers the hope to prevent long-term mental health problems and increase wellbeing and productivity .
The school sector has been reported to be a major contact point for children and adolescents with emotional or behavioral problems , , , , and the further development of school-based mental health promotion programs for children and adolescents ranks among the top ten challenges identified by the Grand Challenges in Global Mental Health initiative . Results of the International School Psychology Survey reveal that the number of students per school psychologist is alarmingly high, counting from 470 students per school psychologist in Italy to more than 26000 students per school psychologist in parts of Germany . Besides this low ratio of school psychologists to students, the function of school psychologists is rather complex and involves a wide range of tasks among which only one is intervention supply , suggesting that sound and comprehensive treatment of children and adolescents with mental disorders remains hardly feasible in the school setting to date. Instead, the school sector has unrivaled access to children and adolescents and, with this unique feature, is exceptionally situated to link children and adolescents with mental disorders to out-of-school services addressing mental health problems and to high-quality care at early disorder stages. However, the role of the school sector as a guide to such out-of-school services remains poorly understood , . To improve our understanding of common trajectories of service use in children and adolescents, it has been claimed that it is important to clarify the sequence of health care sector utilization and to move beyond studying single sectors to the study of the potentially more relevant patterns of sector use , .
The main objective of this study was to estimate in a nationally representative United States cohort, focusing on adolescents with a lifetime mental disorder, the role of school mental health services as guide to care in out-of-school service sectors for the treatment of mental disorders.
Materials and Methods
This study was conducted using data from the National Comorbidity Survey Replication Adolescent Supplement (NCS-A). The NCS-A, a nationally representative face-to-face survey of 10148 United States adolescents (ages 13–18 years) that was carried out in a dual-frame design between February 2001 and January 2004 , , , has previously been described in detail , , . Further information on the NCS-A is available at http://www.icpsr.umich.edu/icpsrweb/ICPSR/studies/28581/version/2. Our analyses are based on a subsample of 6483 out of 10123 adolescents who were in school at the time of survey and for whom interviews as well as parent questionnaires (long versions) were available.
Adolescents and parents provided written informed consent, and the Human Subjects Committees of both Harvard Medical School and the University of Michigan approved the NCS-A.
Assessment of mental disorders
To examine lifetime mental disorders in adolescents, trained interviewers used the WHO Composite International Diagnostic Interview (CIDI) Version 3.0, a structured clinical interview assessing major classes of Diagnostic and Statistical Manual of Mental Disorders (DSM)-IV disorders, including affective disorders (i.e. major depressive disorder, dysthymia, bipolar disorder I, bipolar disorder II), anxiety disorders (i.e. agoraphobia, generalized anxiety disorder, social phobia, specific phobia, panic disorder, post-traumatic stress disorder, separation anxiety disorder), behavior disorders (i.e. attention deficit hyperactivity disorder, oppositional defiant disorder, conduct disorder), substance use disorders (i.e. alcohol abuse/dependence, drug abuse/dependence), and eating disorders (i.e. anorexia nervosa, bulimia nervosa, binge eating disorder). The applied version of the CIDI was computer-assisted and has been adjusted for adolescents, with ensured interview quality and concordance with a clinical reappraisal subsample , , , . In addition to the CIDI interview in the adolescents, parents or parent deputy were inquired to respond to a self-administered questionnaire (SAQ) assessing whether their offspring was suffering from attention-deficit/hyperactivity disorder, conduct disorder, oppositional defiant disorder, major depressive disorder, and dysthymic disorder, as for these disorders parental reports have been shown to be of additional diagnostic value , , . Information from adolescents and parents was integrated whenever it was available from both, and we defined mental disorder as present if DSM-IV criteria were fulfilled either based on adolescent or parent report. Age of onset was specified as the age at which a certain mental disorder developed for the first time. If more than one mental disorder was present within a mental disorder class, age of onset of the first mental disorder in this class was taken as age of onset of this class. Accordingly, if analyses were performed in the subsample of children and adolescents with “any mental disorder”, age of onset of the very first mental disorder was used as age of onset.
Assessment of service use
Adolescents and parents provided information on child/adolescent mental health service use across multiple sectors, based on the Service Assessment for Children and Adolescents . Levels of agreement between parent and adolescent reports were acceptable , . Respondents were asked whether the adolescents ever in their lifetime had gone to see any professionals or facilities for problems with their emotions, behavior, or use of alcohol and drugs, including in- and outpatient care. According to a previous study, service use was classified into the following categories : i) school services, including individual or group psychological counseling/therapy, and ii) out-of-school services, including a) the mental health specialty sector, including community mental health centers or outpatient mental health clinics, mental health professionals, partial hospitalization or day treatment programs, drug or alcohol treatment units/clinics, hospitals and residential treatment centers, b) the medical specialty sector, including emergency rooms, pediatricians or family doctors, and c) other services, including telephone hotlines, self help groups, counselors or family preservation workers, probation or juvenile justice corrections officers or court counselors, spiritual advisors, respite care providers, any other kind of healer, group homes, foster homes, detention centers, prisons/jails, or emergency shelters. Moreover, respondents were asked how old they had been when they first used the reported type of service.
All statistical analyses are based on post-stratification weighted data, which allowed for the correction of minor discrepancies in distributions of school and/or sociodemographic characteristics between the sample and the population . We performed the descriptive analyses of service utilization in different sectors and of the sample's sociodemograhic characteristics by calculating frequencies and percentages of these discrete variables. We computed separate discrete-time proportional hazard models with a non-parametric baseline hazard function, using complementary log-log regression to estimate temporal relationships between school and out-of-school service use for mental disorders, with school service use (versus no school service use) defined as time-varying predictor and mental health specialty service use, medical specialty service use, other service use, and any out-of-school service use defined as outcomes . Analyses were performed in the subsamples of children and adolescents with any i) affective disorder, ii) anxiety disorder, iii) behavior disorder, iv) substance use disorder, v) eating disorder, and vi) mental disorder. We present hazard ratios (HR) and their 95% confidence intervals (CI). To account for the complex structure of the survey data, we used the Taylor series linearization method.
To validate our results, we conducted the following secondary analyses: To control for potential confounding, we repeated the analyses after adjusting for several sociodemographic factors, as sociodemographic factors have previously been shown to influence mental health service utilization in children and adolescents , , , . We used those sociodemographic factors that have been included in previous epidemiological studies of child and adolescent mental health as predictors of mental disorder prevalence , and simultaneously included them in the analyses, with the categories indicated in Table 1.
Moreover, we repeated the analyses three times, controlling for either i) number of categories in which a mental disorder was present (1/2/> = 3), ii) age of onset of the type of mental disorder characterizing the subsample in which the analysis was performed (affective disorders: 1–10 years/11–12 years/13–14 years/15–18 years; anxiety disorders: 1–4 years/5–6 years/7–10 years/11–18 years; behavior disorders: 1–4 years/5–7 years/8–12 years/13–18 years; substance use disorders: 1–13 years/14 years/15 years/16–18 years; eating disorders: 1–12 years/13 years/14 years/15–18 years; any mental disorder: 1–4 years/5–6 years/7–11 years/12–18 years; limits between the categories were defined by cutoffs as close as possible to the quartiles of the respective age distributions), or iii) presence of further types of mental disorders not characterizing the subsample in which the analysis was performed (yes/no).
We interpreted with caution all results that were statistically significant in the main analyses but not stable after Holm correction or secondary analyses.
There were a low percentage of subjects with missing information on service use and we dealt with missing data by restricting each analysis to subjects with complete data (see Table 2). We evaluated the results using two-sided tests of significance set at a level of 0.05. Statistical analyses were performed using STATA/MP 13 (Stata Corporation, College Station, Texas).
Study cohort descriptives
Out of 6483 adolescents, 3656 adolescents (56.4%) were diagnosed with a lifetime mental disorder. Information on utilization of service sectors addressing mental health problems and on sociodemographic characteristics of the study cohort including these mentally disordered adolescents are presented in Table 1.
Prediction of out-of-school mental health service utilization by school mental health service use
Table 2 presents results of the discrete-time proportional hazard analyses of the associations between school and subsequent out-of-school service utilization for mental problems in children and adolescents with lifetime mental disorders. School service use predicted subsequent service utilization in the medical specialty sector in children and adolescents with affective disorders, anxiety disorders, behavior disorders, substance use disorders, and eating disorders, and any mental disorder, subsequent service utilization in other service sectors in children and adolescents with anxiety disorders, behavior disorders, and substance use disorders, and any mental disorder, subsequent service utilization in the mental health specialty sector in children and adolescents with any mental disorder, and subsequent service utilization in any out-of-school service sector in children and adolescents with anxiety disorders and any mental disorder.
After adjusting for multiple testing, all but the predictions for subsequent service utilization in the mental health specialty sector and in any out-of-school service sector in children and adolescents with any mental disorder remained statistically significant.
When we repeated the analyses after adjusting for either sociodemographic factors, number of categories in which a mental disorder was present, age of onset of the type of mental disorder characterizing the subsample in which the analysis was performed, or further types of mental disorders not characterizing the subsample in which the analysis was performed, levels of significance were mostly comparable to those presented in Table 2. However, the temporal prediction of mental health specialty service use by antecedent school service use was no longer significant in the subsample of children and adolescents with any mental disorder in the secondary analyses adjusting for sociodemographic factors (HR = 1.36, 95% CI = 0.85–2.19, p = 0.20), and number of disorder categories (HR = 1.50, 95% CI = 0.94–2.39, p = 0.08), and the temporal prediction of any out-of-school service use by antecedent school service use was no longer significant in the subsample of children and adolescents with any mental disorder in the secondary analyses adjusting for sociodemographic factors (HR = 1.48, 95% CI = 0.99–2.22, p = 0.06). Instead, the temporal prediction of other service use by antecedent school service use reached marginal significance in the subsample of children and adolescents with any affective disorder in the secondary analyses adjusting for number of disorder categories (HR = 1.81, 95% CI = 1.04–3.13, p = 0.04) and further types of mental disorder (HR = 1.79, 95% CI = 1.05–3.04, p = 0.03). Detailed results of the secondary analyses are available on request.
This study, conducted with 6483 adolescents of a nationally representative United States cohort, provides as yet missing association estimates of school and subsequent out-of-school service utilization for mental problems in children and adolescents with a lifetime mental disorder. Results indicate that school services may serve as guide to certain out-of-school service sectors addressing mental disorders, especially the medical specialty sector, but not the mental health specialty sector.
Our results complement previous descriptive evidence regarding service use patterns in adolescents with mental disorders , ,  by elucidating the role of the school service sector in the trajectory of child and adolescent service utilization for mental disorders, and suggest that the school sector plays a central role as guide to the medical specialty sector.
The roles of the medical and the mental health specialty sectors in mental health care have been addressed previously. Several studies have confirmed the prominent position of the medical specialty sector in providing services for mental disorders , even though the mental health specialty sector has been reported to be the major service sector for children and adolescents with mental disorders  and the most common subsequent service sector in adolescents entering the service system through the school mental health sector . While mental health care in the medical specialty sector has improved over the past decade through a better understanding of mental disorders, the availability of practicable screening tools, provision of psychotherapy in these settings, and further development and promotion of psychotropic medications that are mostly prescribed by primary care physicians , , , , most primary care pediatricians and child and adolescent psychiatrists believe that pediatricians should identify and refer, but not treat, their patients' mental health problems . Indeed, the appropriateness of treatment of mental disorders in the medical specialty sector as compared to the mental specialty sector remains a matter of debate , , , , , which should be kept in mind when evaluating the referral patterns observed in the present study.
There is considerable evidence that sociodemographic characteristics of children and adolescents with mental disorders influence whether or not services for mental disorders are utilized; such sociodemographic characteristics include, amongst others, ethnicity, family income, different patient, parental and familial factors, health insurance coverage, and service availability , , , , , , . Much less is known about what influences, in those making use of mental health care, the sequence of sector utilization on their pathway through the service system; for example what influences whether children/adolescents with mental disorders are seeking care in the medical sector or the mental health sector. Therefore, it has been claimed that it is important to clarify the role of potential factors, including referral patterns by professionals, service availability, child and family preferences, professional competencies, and financial considerations, in order to better understand what determines the service paths of children and adolescents seeking help for mental disorders .
Our study has several strengths, including the large nationally representative sample , a fully structured diagnostic interview covering a wide range of adolescent mental disorders, with good quality criteria , , and the integration of adolescent and parent information on adolescent mental disorders, as previously suggested . Given the good response rate and the small amount of missing data, it is unlikely that loss of subjects has introduced relevant selection bias.
Moreover, first, service utilization has been assessed by self-report, but the validity and reliability of such data has been challenged, especially in terms of non-response and recall bias, which is no issue in large health care utilization databases , . However, the NCS-A has not been linked with health care utilization databases, but instead provides self-reported information on service utilization for mental disorders based on a well-established instrument , . In adults, concordance between self-reported health care and mental health care utilization and registration data has been shown to be fair , , , and the rather low magnitude of bias in estimates of health care utilization due to non-response in health surveys has previously encouraged the continued use of interview health surveys . A study on the reliability of children's and adolescents' responses on the Child and Adolescent Services Assessment, the groundwork of the Service Assessment for Children and Adolescents , revealed that reports of lifetime service use were as reliable as were reports of service use in the preceding three months, even though it should be acknowledged that respondents reported inpatient, other overnight and juvenile justice services more reliably than outpatient and school services . It should also be noted that relying on health care utilization databases faces other limitations, including the exclusion of services provided outside the health care system , and the advantages of checklists over routine data sources have been previously documented .
Second, even though we controlled for several potential confounders, we cannot exclude confounding by unconsidered factors, including disorder severity. However, we did not reveal a relationship between school mental health service use and subsequent utilization of the mental health specialty sector, and controlling for number of disorder categories did not change the results remarkably, both of which suggests that disorder severity did not confound the presented results in a relevant manner.
Third, the observed associations do not inform about causal but about temporal relationships, and even though the study of causality is wanted, it remains challenging in such a large representative cohort. Until such studies are available, the elucidation of the position of the school sector in the trajectory of service use for mental disorders allows for a better understanding of common service paths of children and adolescents, as has previously been claimed .
Fourth, the aim of our analyses was to estimate whether school mental health service use serves as a guide to the use of any out-of-school service sectors; however, we did not compare school service use with use of a specific out-of-school service sector regarding their prediction of service use in further sectors. Therefore, it may be interesting to scrutinize in future studies whether the here observed referral patterns are specific for school mental health services.
The clinical and public health relevance of the role of the school sector in the trajectory of service use for mental disorders in children and adolescents becomes evident against the background of the urgent need to reduce unmet mental disorder treatment demands and improve access to mental health care , , together with the limited ability of school mental health services to provide comprehensive treatment on-site on the one hand and their unique position in accessing children and adolescents with mental disorders and linking them to out-of-school services on the other hand , . It has previously been criticized that the coordination of the different health care sectors is often insufficient, thus representing a barrier for the patients and impairing the supply with appropriate treatments . By raising the awareness of unidirectional referral patterns of school mental health services, our results may pave the way to improve the school mental health service system and multilateral collaborations between sectors where required, which is in line with current strategic research goals and task forces , . Notably, our findings raise the question as to whether the school mental health sector sufficiently takes advantage of its unique position in linking mentally disordered students with the out-of-school service system beyond the medical specialty sector.
The supply of mental health services in places where children and adolescents are easily accessible is only one among several action steps that have been proposed to increase access to mental health care for children and adolescents with mental disorders. Further suggestions include the improvement of access to information on available mental health care options, inclusion of children and adolescents in treatment planning, and offering assistance in finding one's way through the rather complicated service system . The bottom line is that, to be most effective in increasing access to child mental health care, strategies should target not only at structural barriers to care, such as inadequate insurance coverage, but also at barriers related to the understanding of mental health problems and services; for example, public education campaigns may help to increase the awareness and knowledge of mental disorders and mental health care services , .
Whether the here presented role of school mental health services as a guide to out-of-school services for mental disorders in the United States is generalizable to other countries remains to be elucidated, as not only the school mental health system but also insurance and health care systems, including access to mental health care systems, vary considerably worldwide , .
Future studies should focus on the role of school mental health services in the trajectory of mental health service use in other countries. Moreover, future research should include prospective data and the integration of self-report of service utilization and service utilization databases. It will also be important to evaluate the appropriateness of the mental health service use trajectory emanating from the school mental health sector and to learn about modifiable factors influencing referral patterns of school mental health services, in order to be able to take action and guide or optimize such referral patterns, if necessary.
To the best of our knowledge, this is the first comprehensive study of the role of the school mental health sector as a guide to mental health care in out-of-school sectors, using data of mentally disordered adolescents of a nationally representative United States cohort. Results indicate that school mental health services may serve as a guide to certain out-of-school service sectors for children and adolescents with various mental disorders, especially the medical specialty sector, but not the mental health specialty sector. While highlighting the relevance of school mental health services in the trajectory of mental care, our findings also suggest to further investigate into a stronger collaboration between school mental health services and mental health specialty services in order to improve and guarantee adequate mental care at early life stages.
Disclaimer: Hereby, we acknowledge that the original collector of the data, ICPSR, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
Conceived and designed the experiments: MT GM. Analyzed the data: MT ES AB GM. Wrote the paper: MT. Conceptualization and design of the study: MT GM. Data acquisition: MT ES GM. Statistical analyses: MT AB GM. Interpretation and analysis of data: MT ES AB GM. Draft of initial manuscript: MT. Critical review of the manuscript: MT ES AB GM. Approval of the final version of the manuscript: MT ES AB GM.
- 1. Wittchen HU, Jacobi F, Rehm J, Gustavsson A, Svensson M, et al. (2011) The size and burden of mental disorders and other disorders of the brain in Europe 2010. European neuropsychopharmacology: the journal of the European College of Neuropsychopharmacology 21: 655–679. doi: 10.1016/j.euroneuro.2011.07.018
- 2. Essau CA, Conradt J, Petermann F (2000) Frequency, comorbidity, and psychosocial impairment of depressive disorders in adolescents. Journal of adolescent research 15: 470–481. doi: 10.1177/0743558400154003
- 3. Essau CA, Conradt J, Petermann F (2000) Frequency, comorbidity, and psychosocial impairment of anxiety disorders in German adolescents. Journal of anxiety disorders 14: 263–279. doi: 10.1016/s0887-6185(99)00039-0
- 4. Brauner CB, Stephens CB (2006) Estimating the prevalence of early childhood serious emotional/behavioral disorders: challenges and recommendations. Public Health Rep 121: 303–310.
- 5. Costello EJ, Egger H, Angold A (2005) 10-year research update review: the epidemiology of child and adolescent psychiatric disorders: I. Methods and public health burden. J Am Acad Child Adolesc Psychiatry 44: 972–986. doi: 10.1097/01.chi.0000172552.41596.6f
- 6. Roberts RE, Attkisson CC, Rosenblatt A (1998) Prevalence of psychopathology among children and adolescents. Am J Psychiatry 155: 715–725.
- 7. Kessler RC, Avenevoli S, Costello EJ, Georgiades K, Green JG, et al. (2012) Prevalence, persistence, and sociodemographic correlates of DSM-IV disorders in the National Comorbidity Survey Replication Adolescent Supplement. Arch Gen Psychiatry 69: 372–380. doi: 10.1001/archgenpsychiatry.2011.160
- 8. Sawyer MG, Whaites L, Rey JM, Hazell PL, Graetz BW, et al. (2002) Health-related quality of life of children and adolescents with mental disorders. Journal of the American Academy of Child and Adolescent Psychiatry 41: 530–537. doi: 10.1097/00004583-200205000-00010
- 9. Guevara JP, Mandell DS, Rostain AL, Zhao H, Hadley TR (2003) National estimates of health services expenditures for children with behavioral disorders: an analysis of the medical expenditure panel survey. Pediatrics 112: e440. doi: 10.1542/peds.112.6.e440
- 10. Collins PY, Patel V, Joestl SS, March D, Insel TR, et al. (2011) Grand challenges in global mental health. Nature 475: 27–30. doi: 10.1038/475027a
- 11. Prince M, Patel V, Saxena S, Maj M, Maselko J, et al. (2007) No health without mental health. Lancet 370: 859–877. doi: 10.1016/s0140-6736(07)61238-0
- 12. Foy JM (2010) Enhancing pediatric mental health care: report from the American Academy of Pediatrics Task Force on Mental Health. Introduction. Pediatrics 125 Suppl 3S69–74. doi: 10.1542/peds.2010-0788c
- 13. Merikangas KR, He JP, Burstein M, Swendsen J, Avenevoli S, et al. (2011) Service utilization for lifetime mental disorders in U.S. adolescents: results of the National Comorbidity Survey-Adolescent Supplement (NCS-A). Journal of the American Academy of Child and Adolescent Psychiatry 50: 32–45. doi: 10.1016/j.jaac.2010.10.006
- 14. Offord DR, Boyle MH, Szatmari P, Rae-Grant NI, Links PS, et al. (1987) Ontario Child Health Study. II. Six-month prevalence of disorder and rates of service utilization. Archives of general psychiatry 44: 832–836. doi: 10.1001/archpsyc.1987.01800210084013
- 15. Weisz JR, Jensen-Doss A, Hawley KM (2006) Evidence-based youth psychotherapies versus usual clinical care: a meta-analysis of direct comparisons. The American psychologist 61: 671–689. doi: 10.1037/0003-066x.61.7.671
- 16. Wang PS, Berglund P, Olfson M, Pincus HA, Wells KB, et al. (2005) Failure and delay in initial treatment contact after first onset of mental disorders in the National Comorbidity Survey Replication. Archives of general psychiatry 62: 603–613. doi: 10.1001/archpsyc.62.6.603
- 17. Kieling C, Baker-Henningham H, Belfer M, Conti G, Ertem I, et al. (2011) Child and adolescent mental health worldwide: evidence for action. Lancet 378: 1515–1525. doi: 10.1016/s0140-6736(11)60827-1
- 18. Farmer EM, Burns BJ, Phillips SD, Angold A, Costello EJ (2003) Pathways into and through mental health services for children and adolescents. Psychiatric services 54: 60–66. doi: 10.1176/appi.ps.54.1.60
- 19. Burns BJ, Costello EJ, Angold A, Tweed D, Stangl D, et al. (1995) Children's mental health service use across service sectors. Health affairs 14: 147–159. doi: 10.1377/hlthaff.14.3.147
- 20. Farmer EM, Stangl DK, Burns BJ, Costello EJ, Angold A (1999) Use, persistence, and intensity: patterns of care for children's mental health across one year. Community mental health journal 35: 31–46. doi: 10.1023/a:1018743908617
- 21. Jimerson SR, Graydon K, Curtis MJ, Staskal R, Oakland TD, et al. (2007) The International School Psychology Survey: Insights From School Psychologists Around the World. The Handbook of International School Psychology. Thousand Oaks, CA, USA: Sage Publications, Inc.
- 22. Watkins MW, Crosby EG, Pearsson JL (2001) Role of the School Psychologist. School Psychology International 22: 64–73. doi: 10.1177/01430343010221005
- 23. Wood PA, Yeh M, Pan D, Lambros KM, McCabe KM, et al. (2005) Exploring the relationship between race/ethnicity, age of first school-based services utilization, and age of first specialty mental health care for at-risk youth. Mental health services research 7: 185–196. doi: 10.1007/s11020-005-5787-0
- 24. Wang PS, Demler O, Olfson M, Pincus HA, Wells KB, et al. (2006) Changing profiles of service sectors used for mental health care in the United States. The American journal of psychiatry 163: 1187–1198. doi: 10.1176/appi.ajp.163.7.1187
- 25. Kessler RC, Avenevoli S, Costello EJ, Green JG, Gruber MJ, et al. (2009) National comorbidity survey replication adolescent supplement (NCS-A): II. Overview and design. J Am Acad Child Adolesc Psychiatry 48: 380–385. doi: 10.1097/chi.0b013e3181999705
- 26. Kessler RC, Avenevoli S, Costello EJ, Green JG, Gruber MJ, et al. (2009) Design and field procedures in the US National Comorbidity Survey Replication Adolescent Supplement (NCS-A). Int J Methods Psychiatr Res 18: 69–83. doi: 10.1002/mpr.279
- 27. Kessler RC, Merikangas KR (2004) The National Comorbidity Survey Replication (NCS-R): background and aims. Int J Methods Psychiatr Res 13: 60–68. doi: 10.1002/mpr.166
- 28. Merikangas K, Avenevoli S, Costello J, Koretz D, Kessler RC (2009) National comorbidity survey replication adolescent supplement (NCS-A): I. Background and measures. J Am Acad Child Adolesc Psychiatry 48: 367–369. doi: 10.1097/chi.0b013e31819996f1
- 29. Kessler RC, Avenevoli S, Green J, Gruber MJ, Guyer M, et al. (2009) National comorbidity survey replication adolescent supplement (NCS-A): III. Concordance of DSM-IV/CIDI diagnoses with clinical reassessments. J Am Acad Child Adolesc Psychiatry 48: 386–399. doi: 10.1097/chi.0b013e31819a1cbc
- 30. Cantwell DP, Lewinsohn PM, Rohde P, Seeley JR (1997) Correspondence between adolescent report and parent report of psychiatric diagnostic data. Journal of the American Academy of Child and Adolescent Psychiatry 36: 610–619. doi: 10.1097/00004583-199705000-00011
- 31. De Los Reyes A, Kazdin AE (2005) Informant discrepancies in the assessment of childhood psychopathology: a critical review, theoretical framework, and recommendations for further study. Psychological bulletin 131: 483–509. doi: 10.1037/0033-2909.131.4.483
- 32. Grills AE, Ollendick TH (2002) Issues in parent-child agreement: the case of structured diagnostic interviews. Clin Child Fam Psychol Rev 5: 57–83.
- 33. Stiffman AR, Horwitz SM, Hoagwood K, Compton W 3rd, Cottler L, et al. (2000) The Service Assessment for Children and Adolescents (SACA): adult and child reports. Journal of the American Academy of Child and Adolescent Psychiatry 39: 1032–1039. doi: 10.1097/00004583-200008000-00019
- 34. Willett JB, Singer JD (2004) Discrete-time survival analysis. In: Kaplan D, editor. The SAGE Handbook of Quantitative Methodology for the Social Sciences. Thousand Oaks, London, New Delhi: Sage Publications, Inc.
- 35. Aickin M, Gensler H (1996) Adjusting for multiple testing when reporting research results: the Bonferroni vs Holm methods. American journal of public health 86: 726–728. doi: 10.2105/ajph.86.5.726
- 36. Holm S (1979) A Simple Sequentially Rejective Multiple Test Procedure. Scandinavian journal of statistics 6: 65–70.
- 37. Newson RB (2010) Frequentist q-values for multiple-test procedures. The Stata Journal 10: 568–584.
- 38. Zahner GE, Daskalakis C (1997) Factors associated with mental health, general health, and school-based service use for child psychopathology. American journal of public health 87: 1440–1448. doi: 10.2105/ajph.87.9.1440
- 39. Cunningham PJ, Freiman MP (1996) Determinants of ambulatory mental health services use for school-age children and adolescents. Health services research 31: 409–427.
- 40. Costello EJ, Janiszewski S (1990) Who gets treated? Factors associated with referral in children with psychiatric disorders. Acta psychiatrica Scandinavica 81: 523–529. doi: 10.1111/j.1600-0447.1990.tb05492.x
- 41. Marino S, Gallo JJ, Ford D, Anthony JC (1995) Filters on the pathway to mental health care, I. Incident mental disorders. Psychological medicine 25: 1135–1148. doi: 10.1017/s0033291700033110
- 42. Zito JM, Safer DJ, dosReis S, Gardner JF, Boles M, et al. (2000) Trends in the prescribing of psychotropic medications to preschoolers. JAMA: the journal of the American Medical Association 283: 1025–1030. doi: 10.1001/jama.283.8.1025
- 43. Olfson M, Marcus SC, Druss B, Pincus HA (2002) National trends in the use of outpatient psychotherapy. The American journal of psychiatry 159: 1914–1920. doi: 10.1176/appi.ajp.159.11.1914
- 44. Spitzer RL, Kroenke K, Williams JB (1999) Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. JAMA: the journal of the American Medical Association 282: 1737–1744. doi: 10.1001/jama.282.18.1737
- 45. Heneghan A, Garner AS, Storfer-Isser A, Kortepeter K, Stein RE, et al. (2008) Pediatricians' role in providing mental health care for children and adolescents: do pediatricians and child and adolescent psychiatrists agree? Journal of developmental and behavioral pediatrics: JDBP 29: 262–269. doi: 10.1097/dbp.0b013e31817dbd97
- 46. Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, et al. (2005) Twelve-month use of mental health services in the United States: results from the National Comorbidity Survey Replication. Archives of general psychiatry 62: 629–640. doi: 10.1001/archpsyc.62.6.629
- 47. Wells KB, Kataoka SH, Asarnow JR (2001) Affective disorders in children and adolescents: addressing unmet need in primary care settings. Biological psychiatry 49: 1111–1120. doi: 10.1016/s0006-3223(01)01113-1
- 48. Wang PS, Berglund P, Kessler RC (2000) Recent care of common mental disorders in the United States: prevalence and conformance with evidence-based recommendations. Journal of general internal medicine 15: 284–292. doi: 10.1046/j.1525-1497.2000.9908044.x
- 49. Wang PS, Demler O, Kessler RC (2002) Adequacy of treatment for serious mental illness in the United States. American journal of public health 92: 92–98. doi: 10.2105/ajph.92.1.92
- 50. Young AS, Klap R, Sherbourne CD, Wells KB (2001) The quality of care for depressive and anxiety disorders in the United States. Archives of general psychiatry 58: 55–61. doi: 10.1001/archpsyc.58.1.55
- 51. Burns BJ, Costello EJ, Erkanli A, tweed D, Farmer EMZ, et al. (1997) Insurance coverage and mental health service use by adolescents with serious emotional disturbances. Journal of Child and Family Studies 6: 89–111.
- 52. Verhulst FC, van der Ende J (1997) Factors associated with child mental health service use in the community. Journal of the American Academy of Child and Adolescent Psychiatry 36: 901–909. doi: 10.1097/00004583-199707000-00011
- 53. Saxena S, Thornicroft G, Knapp M, Whiteford H (2007) Resources for mental health: scarcity, inequity, and inefficiency. Lancet 370: 878–889. doi: 10.1016/s0140-6736(07)61239-2
- 54. Kessler RC, Avenevoli S, McLaughlin KA, Green JG, Lakoma MD, et al. (2012) Lifetime co-morbidity of DSM-IV disorders in the US National Comorbidity Survey Replication Adolescent Supplement (NCS-A). Psychological medicine 42: 1997–2010. doi: 10.1017/s0033291712000025
- 55. Bhandari A, Wagner T (2006) Self-reported utilization of health care services: improving measurement and accuracy. Medical care research and review: MCRR 63: 217–235. doi: 10.1177/1077558705285298
- 56. Schneeweiss S, Avorn J (2005) A review of uses of health care utilization databases for epidemiologic research on therapeutics. Journal of clinical epidemiology 58: 323–337. doi: 10.1016/j.jclinepi.2004.10.012
- 57. Reijneveld SA, Stronks K (2001) The validity of self-reported use of health care across socioeconomic strata: a comparison of survey and registration data. International journal of epidemiology 30: 1407–1414. doi: 10.1093/ije/30.6.1407
- 58. Reijneveld SA (2000) The cross-cultural validity of self-reported use of health care: a comparison of survey and registration data. Journal of clinical epidemiology 53: 267–272. doi: 10.1016/s0895-4356(99)00138-9
- 59. Killeen TK, Brady KT, Gold PB, Tyson C, Simpson KN (2004) Comparison of self-report versus agency records of service utilization in a community sample of individuals with alcohol use disorders. Drug and alcohol dependence 73: 141–147. doi: 10.1016/j.drugalcdep.2003.09.006
- 60. Gundgaard J, Ekholm O, Hansen EH, Rasmussen NK (2008) The effect of non-response on estimates of health care utilisation: linking health surveys and registers. European journal of public health 18: 189–194. doi: 10.1093/eurpub/ckm103
- 61. Farmer EMZ, Angold A, Burn BJ, Costello EJ (1994) Reliability of Self-Reported Service Use: Test-Retest Consistency of Children's Responses to the Child and Adolescent Services Assessment (CASA). Journal of Child and Family Studies 3: 307–325. doi: 10.1007/bf02234688
- 62. Knight M, Stewart-Brown S, Fletcher L (2001) Estimating health needs: the impact of a checklist of conditions and quality of life measurement on health information derived from community surveys. J Public Health Med 23: 179–186. doi: 10.1093/pubmed/23.3.179
- 63. Hogan MF (2003) New Freedom Commission Report: The President's New Freedom Commission: Recommendations to Transform Mental Health Care in America. Psychiatric Services 54.
- 64. Remschmidt H, Belfer M (2005) Mental health care for children and adolescents worldwide: a review. World psychiatry: official journal of the World Psychiatric Association 4: 147–153.
- 65. Report of the Surgeon Generals Conference on Children's Mental Health: A National Action Agenda. American Journal of Health Education 32: 179–182. doi: 10.1080/19325037.2001.10603461
- 66. Jorm AF (2000) Mental health literacy. Public knowledge and beliefs about mental disorders. The British journal of psychiatry: the journal of mental science 177: 396–401. doi: 10.1192/bjp.177.5.396
- 67. Owens PL, Hoagwood K, Horwitz SM, Leaf PJ, Poduska JM, et al. (2002) Barriers to children's mental health services. Journal of the American Academy of Child and Adolescent Psychiatry 41: 731–738. doi: 10.1097/00004583-200206000-00013
- 68. Jacob KS, Sharan P, Mirza I, Garrido-Cumbrera M, Seedat S, et al. (2007) Mental health systems in countries: where are we now? Lancet 370: 1061–1077. doi: 10.1016/s0140-6736(07)61241-0