https://orcid.org/0000-0002-2631-0205
Figures
Abstract
In the wake of the mental health crisis in children and adolescents, the coordination of education and mental health services has become a global priority. However, differing terminologies and classifications across sectors, hinder effective comparison. The classification in education focuses mainly on outputs like qualifications or throughputs like teaching programs. This proof-of-concept study tested the applicability of a standard classification of health services, the Description and Evaluation of Services and DirectoriEs (DESDE), to evaluate education services for mental health users in the context of Spain and The Netherlands. It was conducted alongside the PECUNIA project, that sought to develop methods for the assessment of mental health costs and outcomes in different sectors. The study followed an ontoterminology approach involving: 1) identification of services from a predefined list of 46 resource-use items, 2) disambiguation of identified services with the DESDE, and classifying them as accurate, ambiguous, vague or confuse; and 3) external validation by an expert panel. The analysis was conducted at the level of type of resource, target population and care provision. From the initial list, only ten of the resources could be categorized as services using DESDE, and not activities, interventions or professionals. Only four of them (8,65%) were accurate across all disambiguation categories. Experts were unaware of terminology problems in classification of service provision in the education sector. Classifications and glossaries can clarify service naming, description and costing allowing comparative effectiveness analysis and facilitating cross-sectoral planning. This should be grounded in common methodologies, tools, and units of analysis.
Citation: Pokhilenko I, Gutierrez-Colosia MR, Janssen LMM, Evers SMAA, Paulus ATG, Drost RMWA, et al. (2024) Clarifying terminology and definitions in education services for mental health users: A disambiguation study. PLoS ONE 19(7): e0306539. https://doi.org/10.1371/journal.pone.0306539
Editor: Faten Amer, An-najah National University Faculty of Medicine and Health Sciences, STATE OF PALESTINE
Received: November 8, 2023; Accepted: June 18, 2024; Published: July 3, 2024
Copyright: © 2024 Pokhilenko et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: The PECUNIA project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 779292. This study also received financial support by a grant from the Carlos III Health Institute (PI18/01521). The funding agreement ensured the authors’ independence in designing the study, interpreting the data, and writing and publishing the report.
Competing interests: The authors have declared that no competing interests exist.
Introduction
In the wake of the mental health crisis in children and adolescents, the coordination of education and mental health services has become a global priority. Integrated youth services which rely on evidence-based models, are designed to provide effective care through a holistic approach [1]. This integration is part of a broader discipline known as service science that has emerged in response to the growing complexity of the service sector, particularly in more developed economies [2]. It is a multidisciplinary approach studying services from a holistic or whole systems perspective that incorporates ontology, i.e. classification and explanation of entities [3]. Person-centered service provision tends to be multi-sectoral in nature and include a multitude of service sectors such as, for example, education, healthcare, and social care [2].
In line with the systems thinking theory, understanding, complex systems require analysing their interrelationships and behaviours. Interventions influence the entire system, and likewise, an entire system perspective is needed to assess a complex intervention [4,5]. Moreover, every system, while unique in its structure, comprising elements like financing and governance, is not isolated. In fact, all systems are interrelated components of a larger social system. For example, improvements in health services may enhance educational outcomes by increasing student mental health, wellbeing etc. and conversely educational advancements can lead to better health awareness [6,7].
Therefore, cross-sectoral classification of services that considers the context of provision and the interconnected nature of systems, is essential. It may enable policymakers, to mitigate negative effects, maximize synergies, and assess the wider effects of interventions on the whole system [4]. Furthermore, there is empirical evidence from the health sector that the application of the whole systems approach in practice, leads to efficiency and improved outcomes through better coordination between the involved stakeholders [8–10].
While the nature of service delivery in specific sectors can differ, there are common challenges that pertain to most if not all sectors. One example of such a challenge is terminological variability. It constitutes a major source of systematic bias in service research, with the magnitude of this problem remaining largely unnoticed until very recently [11]. This has led to the increased interest in, and the relevance of, intersectoral international classifications that allow for meaningful comparison of services within and across different sectors, thereby supporting the application of the whole systems approach to the evaluation of complex interventions.
While several international classifications are widely used in the education sector, their primary focus has been on the comparability of outputs (i.e. outcomes such as educational qualifications) [12] or throughputs like educational programs and procedures [13]. Surprisingly, little information is available regarding the classification of inputs such as types of education services, settings and education teams using a standard classification of human service provision. The OECD classifies education services into core and ancillary types [14]. Core services include all direct instructional expenses like teacher’s salaries, building maintenance, and administration. Ancillary services, though peripheral, focus on student welfare including meals, health services, transportation, and housing. Nevertheless, this classification lacks the tools needed for comparative analysis of services based on their features, within and across jurisdictions.
The International Standard Classification of Education (ISCED) represents another attempt to structure and organize education programs and educational qualifications by levels and fields [15]. However, it relies on country-level data implying important variabilities in classification and reporting. Also, there are programmes that span two or more ISCED levels making comparisons difficult.
There is a gap in the classification of provision of education services as current classifications do not establish common units of analysis that could be compared like-with-like. From an ecological perspective, several levels of units of analysis can be identified: macro-organizations (e.g. institutions, large foundations), meso-organizations (e.g. large schools), micro-organisations or services (e.g. small school, centres, courses), and nano-organisations that involve a dyad of users or groups of users and a provider delivering defined interventions and activities (e.g. a discipline within a school). Terminological variability and the lack of methods for classifying services according to common units of analysis lead to issues with service monitoring and planning. As mentioned above, this issue is relevant for many service sectors including the education sector and has been highlighted in previous studies focusing on school-based mental health interventions among other [16,17]. This issue was also raised in the impact analysis of the EdLINQ program for improving liaison between school services and child and adolescent mental health care in Queensland, Australia [18].
The Description and Evaluation of Services and DirectoriEs (DESDE) is an example of an international, intersectoral classification system valid for the evaluation of complex systems [19]. It originates from the field of mental health services research. In the past 25 years, it has been extended to incorporate other care sectors such as social [20], child and adolescent care, drug and alcohol, ageing, disabilities, long-term care [19,21]. DESDE has been validated and extensively used in the field of health services research and health economics for the comparison of resource utilization across different settings, efficiency analysis and the calculation of unit costs, among others [22]. Furthermore, DESDE is the only existing tool that provides local, bottom-up information that can be used across different sectors [5].
Building on the knowledge of classifications of health services [19] and its application to improve international comparability [23], this study aimed to implement a knowledge transfer from health to education services research. This study was designed as a proof of concept. That is the confirmation of the principle of the application of the emerging scientific knowledge, as the basis for subsequent activities, such as prototype development. Thus, we aimed to apply and test the applicability of a standardized classification system, i.e. DESDE, to the evaluation of education services relevant for mental health users in the context of two European countries, Spain and The Netherlands. The result of this study is expected to contribute to the understanding of the organization of education systems, identification of potential gaps in service provision, and facilitation of the application of the whole systems approach to the analysis of cross-sectoral service provision and complex interventions in the European context.
Methods
The study was designed as a proof-of-concept to test the applicability of the DESDE classification system to the education sector. It aimed to assess the clarity of service terms in this sector using an ontoterminology approach. Meaning the study of disambiguation of technical and scientific terms using standard classifications [24]. This method helps identify precise definitions and relationships between terms, enhancing the retrieval of domain-specific information. It also aids in creating controlled vocabularies, classifications, and glossaries [24].
The study was conducted alongside the ProgrammE in Costing, resource use measurement and outcome valuation for Use in multi-sectoral National and International health economic evaluAtions (PECUNIA) project (https://www.pecunia-project.eu/). PECUNIA was a European-funded consortium aimed at developing new standardised, harmonised and validated methods and tools for the assessment of costs and outcomes in European healthcare systems from a whole-system perspective including health and social care, employment, education and justice sectors. Although methods and tools developed within the PECUNIA project were meant to be generic, three mental health disorders (depression, post-traumatic stress disorder, and schizophrenia) were selected as illustrative examples. Building on the foundational goals of PECUNIA, this study’s focus on disambiguation and classification was related to the critical objectives of definition and harmonization within the project [11].The initial list of education resource-use items was compiled from literature reviews conducted within PECUNIA. Once classified with DESDE, these services were used in PECUNIA for cost calculation.
Education context in Spain and in the Netherlands
Structure.
In Spain, education is mandatory from 6 to 15 years [25]. Pre-school education while not compulsory, is available for children from birth to 6 years, and is divided into nursery school (0–3 years) and infant school (3 to 6 years) [25]. OECD reports indicate that in Spain 95% of 3 years old and 97% of 4 years old are enrolled in pre-school [26]. Education continues with primary school from ages 6 to 12, followed by secondary education until age 15. Post-secondary options such as upper secondary education or vocational training are available for 16 to 18 years, but are not compulsory [25]. Special education is organized in three sections: special early childhood education, adapted compulsory basic education, and training programs for transition to adult life from 16 to 20 years old, and it is not compulsory [27].
In the Netherlands, school attendance is compulsory from 5 to 16 years old, and for those between 16 and 18 who have not yet met basic qualifications [14]. Pre-school education is not compulsory [28]. Children typically start primary school at the age of 4 and transition to secondary education at 11–12. Secondary school education is structured into three distinct tracks [28]: pre-vocational education lasts four years and prepares students for entering vocational schools [29]; higher general continued education takes five years and prepares students to enter a university of applied sciences; preparatory scientific education lasts six years and prepares students to enter an academic university [30]. Special education is delivered in separate special schools and is organized into four clusters to address specific needs: visual impairments, hearing or speech impairments, physical or mental handicaps, and mental disorders and behavioural problems [31].
Governance.
In Spain, the Ministry of Education, Culture and Sport governs the education sector. Individual schools and budget are managed by the autonomous communities with guidelines set by the central government. Public schools receive government subsidies and must adhere closely to Ministry requirements. Private schools which may be subsidised or not, also depend on and are authorised by the regional government, but they have their own management. Although Spanish universities operate independently from the government, they are regulated and monitored by the national and regional quality and accreditation agencies [26].
In the Netherlands, the Ministry of Education, Culture and Science, governs the education sector, setting general rules while allowing institutes freedom in their teaching methods [32]. Primary and secondary schools are further governed by school boards, with several schools falling under one board. The quality of education is monitored nationally by the Inspectorate of Education, and funding for schools is allocated based on the number of students; with additional support for disadvantaged students or students with special needs [33]. Both public and private schools can receive government funding if they meet statutory requirements. Vocational and higher education institutes operate as separate legal entities and receive funding through a combination of tuition fees and government grants, which are based on the institution’s performance. Tertiary educational institutions are governed by boards that formulate and implement strategies within the government’s framework [33].
Key professionals.
In Spain, the main reference professional in public schools is the teacher/educator. They must complete four years of university studies and pass an official exam. Experience and training provide points that allow them to move between institutions. Special education adopts, the multitrack model described by Nash and Norwich [34], where all teachers receive generic training followed by specialized training therapeutic pedagogy or hearing and language [35]. In the private sector, a master’s degree is required in secondary though the official exam is not.
In the Netherlands, primary school teachers need a four-year degree from either a university of applied sciences or a regular university. Secondary school teachers, are differentiated by the level of education they can teach, depending on whether they have a first degree (lower education levels e.g pre-vocational education) or second degree (advanced educational levels e.g. preparatory scientific education). Teaching assistants, that support the delivery of education at primary and secondary schools, must complete a vocational education degree [36]. In special education, although the level might be lower compared to regular education, the pedagogical requirements are more stringent. Higher education professionals are often required to have a degree from a university of applied sciences or a regular university, along with a university teaching qualification, though there are no specific statutory requirements [36].
Tools: Description and Evaluation of Services and DirectoriEs (DESDE)
The DESDE system constitutes a unique approach for the standard description and classification of services across sectors [19,21]. It first identifies minimal organizational units with temporal stability arranged for delivering care, called Basic Stable Inputs of Care (BSIC). Second, the main activity of the BSIC is described through the ‘Main Type of Care’ (MTC) or code. There are 108 codes subdivided into six main branches of care (Residential, Day, Outpatient, Accessibility, Information, Self-help and Voluntary). DESDE provides a multiaxial system that contains information on the following aspects of a service or a BSIC: a) the sector cluster (health, social, education, employment, justice); b) the target population, Including the age (e.g. CX–children and adolescents), gender, and the diagnosis group for which the service is intended. Code is based on the International Classification of Diseases 10th edition (ICD-10) [37] (e.g. F7-F8—Mental retardation, disorders of psychological development), and the International Classification of Functioning, Disability and Health (ICF) [38]; c) a code of the main service function or type of care and its additional qualifiers. This code is the core component of DESDE (Fig 1). Two DESDE axes were used for the disambiguation study, target population and service type of care. In the example presented in Fig 1, the code thread refers to a special education needs class in a school. In this proof of concept study, the rationale for applying DESDE classification arose from the need to create a standardized framework for describing and assessing education services in a comparable manner. DESDE was originally developed for use in health and social care sectors. Its methodological rigor made it ideal to bring similar benefits of standardization to the education sector.
Procedure
This ontoterminology study followed a multistep process to assess the clarity of terms in education services, and involved three working groups:
Step 1—service identification.
Group A comprised members from Maastricht University (IP, LJ, SE, AP, RD). They undertook peer-reviewed and grey literature searches, as well as multi-national expert surveys that took place in six European countries prior to this study, as part of the PECUNIA project [39]. The aim was to build a list of resource-use items in the education sector along with their respective definitions relevant for mental health economic evaluations [39]. These identified services in the education sector, along with services in other sectors [40], formed the basis for developing new costing instruments such as the Resource Use Measurement (RUM) instrument, Reference Unit Cost (RUC) Templates, and RUC Compendium [41–43].
Step 2 –disambiguation.
Group B included two experts in disambiguation using DESDE (MR, LS) that analysed the list compiled by group A. In terminology, disambiguation is the process of making something clear and resolving ambiguity and vagueness of terms [44]. Ambiguity refers to the terms that can be interpreted in more than one way because of imprecise definitions. Vagueness refers to the terms that are not sufficiently specified and therefore cannot be translated into a code. Our team established a threshold for differentiating ambiguity and vagueness in the analysis of health services using DESDE [44,45]. A service that could be assigned two codes of the reference classification system was considered ambiguous, whilst a service that could be assigned more than two codes, was categorized as vague. Confusion in this study refers to the terms whose definition did not allow classification. Items were categorized into ‘accurate’, ‘ambiguous’, ‘vague’, or ‘confusing’ based on descriptions. The process involved two levels:
- 1. Defining the unit of analysis (type of resource): group B reviewed the general list and ensured that the items and their descriptions corresponded to the same unit of analysis (commensurability), in this case “services” and not to other units of analysis such as interventions, professionals, or outcomes. The standard description of services provided in DESDE was used to ensure that items could be compared like-with-like. Items classified as non-services were excluded from further disambiguation.
2. DESDE classification: group B assessed information about the target population—including age and diagnosis group-, as well as the type of care provided in the services. This enabled the assignment of prototype DESDE codes and additional qualifiers.
Step 3- external validation.
To finalize the disambiguation process and evaluate the suitability of DESDE for classifying education services, group C was consulted. This group consisted of a panel of external experts recruited from the current study’s co-authors’ professional network using purposive sampling (two from Spain, and three from the Netherlands). It included two academics, one public agent from the regional Ministry of Education and two third-sector professionals (AG, AR, TS, IO, MS). Their primarily role was to validate the services coded by group B, using their deep knowledge of the sector.
Consultation involved individual semi-structured interviews. Experts were guided through a series of generic inquiries about education within the contexts of Spain and The Netherlands. Interview questions were pre-defined by the co-authors of the current study (S1 Table). Experts critically reviewed the comprehensiveness and accuracy of the service list and verified the correctness of the assigned codes, taking into account the educational realities in both countries.
Results
Step 1 –identification
The item list for disambiguation contained 46 tentative services or resources, 38 items out of which were identified in the review of literature and expert surveys, and eight items were extracted from the PECUNIA RUM instrument [41]. For the full list of items, please refer to S2 Table. An additional listing of Spanish services and programmes was identified and provided in S3 Table. This list did not undergo disambiguation inasmuch as it was not included in the official listing of PECUNIA services, but it was reviewed by the Spanish experts.
Step 2—disambiguation
Level 1—unit of analysis (commensurability).
In the first step of disambiguation, ten items (22%) were classified as accurate and included in the subsequent disambiguation steps. Thirty-five items (76%) were classified as confusing, because they did not refer to services but rather to procedures (‘special needs diagnostics’), professionals (‘liaison teacher’) or consequences (‘low school adaptation/competence’) and were therefore excluded from the subsequent disambiguation steps. One item (2%), ‘student counselling’, was excluded because it was not clear what was understood by counselling in this term and admitted very different units of analysis. Please refer to S2 Table for more details.
Level 2—DESDE classification.
Ten items, initially classified as accurate in terms of unit of analysis and thus defined as “services” were further assessed for clarity concerning the target population. Five items were classified as accurate (50%). Two items were classified as ambiguous (20%) these included ‘special education services’ and ‘student transport to special education facility’ where both primary and secondary students could be admitted. Another two items were classified as vague (20%) because they lacked specific descriptions of age or diagnosis groups (‘special education day school’ and ‘day school’). One item was classified as confusing (10%) due to an incoherent description of the target population; ‘higher education school (university, college, vocational school)’ could include not only adults, but also adolescents. Please refer to Table 1 for more details.
Regarding service type, eight items were classified as accurate (80%). One item, ‘education support in a private setting’ was classified as ambiguous (10%) due to it having two different codes, with the second code varying by the frequency of care provided from high (O8) to low (O10). Another item, ‘special education services’ was classified as vague (10%), because it required a wide range of different codes. For more details, please refer to Tables 1 and 2.
Step 3—expert validation
Initially, the five experts from the panels (group C) were unaware of any terminology problem in the classification of educational services. Only one expert from Spain highlighted potential problems when comparing services between regions in the country. Experts considered ‘special education services’ and ‘higher education school’ umbrella terms that could refer to a variety of settings and discarded them as they were too broad to be useful for an operational classification. Regarding special education, they informed that in recent years there has been a shift towards a more inclusive school environment both in Spain and The Netherlands. Students with special education needs are educated in regular schools with additional support if required. For example, in the Netherlands regular schools receive additional funds to support students with special needs. Furthermore, partnerships between regular and special education schools facilitate the integration of well-performing students into regular schools. in Spain there were special centres for students with autism spectrum disorders and intellectual disabilities among other conditions, but the trend in the last decades has been to offer universal education in regular schools with specific programmes for the students with special needs. Only individuals with more severe disabilities are typically accommodated in ‘special education day schools’. Experts indicated that this distinction should be reflected in the DESDE code axis of the target population. Therefore, the ICF severity code (ICF3), was added in this case. ‘Boarding school’, ‘special education boarding school’, while no longer prevalent in public systems, were considered accurate in their definition and code. Similarly, ‘day school’ and ‘Student transport to special education facility’ were considered appropriate.
When asked about the comprehensiveness of the list of services, experts mentioned others not included in the more mental health-specific PECUNIA list. In Spain, teacher training centres offer voluntary and free training organized by teachers/educators who specialize in this area. This training represents an important part of the education budget. Temporary linguistic adaptation classrooms assist foreign students in learning Spanish, particularly in centres with high immigrant populations like temporary shelters. Sign language interpreters aid children with hearing difficulties. Schools can also request reinforcement, orientation, and support plans for students with specific learning difficulties, delivered after school. Furthermore, there are teachers providing domiciliary education for children with long term illnesses.
In the Netherlands, there are day care facilities aimed at providing learning basic life skills to people who dropped out of school. These services are not allowed to provide education; but they are often located within special education schools. Experts indicated that the type of funding (private or public) should be added to the DESDE code.
Finally, there are specific resources both in Spain and the Netherlands for juvenile penitentiary centres and for the children and adolescents undergoing treatment in the hospital. Following the DESDE hierarchical system, these services are classified respectively as justice and health services that provide education. The list of services, prototype codes, and comments from the expert interviews are listed in Table 3 below.
Discussion
This study on educational services relevant to mental health, presents the first analysis of terminological unclarity and highlights the need for a standard international terminology and classification of services in this sector. The rationale for applying DESDE classification arose from the need to create a standardized framework for describing and assessing education services in a comparable manner from a whole-system and cross-sectoral perspective [44]. The magnitude of the terminological problem in the education sector is at least as relevant as it is in the health and social sectors. From the original list of 46 resource-use items, only ten were typified as actual services (resource inputs) while other terms referred to professionals, interventions (throughputs), and consequences (outputs). Five services were accurate at target population level and eight were accurate at the level of service type (DESDE code). Only four services were accurate at all levels of disambiguation (8,65%) namely ‘boarding school’, ‘special education boarding school’, ‘education support at your place of study’ and ‘night school’. This ambiguity reflects the complexity and diversity of education services and the challenges inherent in defining and categorizing them. Accurate classification informs policy development by providing policymakers with a clear understanding of the types of services available and the population they serve. This allows for a more effective resource allocation and collaboration across sectors.
Most services were included in the ‘day’ typology of care, with the main code of ‘education related, structured and high intensity care, delivered for groups of people’ (D4.2g). However, DESDE has not been previously applied to assess the education sector, only to specific services providing education care for mental health users in the health and social sectors [46].
Collecting data using a top-down approach from sources such as administrative databases or professional boards, where data is interpreted through a national prism, may imply simplifications of the reality of the local system. Without clear terminology, applying national data to the local level could lead to misunderstandings and likely failures of any intended changes or interventions in the real world. Evidence about local conditions is important for assessing resource availability. A bottom-up approach that includes a comprehensive and systematic description of the context and knowledge at the service delivery level is essential to complete national data for evidence-informed research and policy [5].
Expert consultation was in this sense crucial for nuanced service descriptions, appropriate code assignments, and gaining deeper insights into the national education systems from a bottom-up perspective. Their input, based on their contextual background, contributed significantly to refining the outputs of the study and enhance the reliability of the findings: ‘boarding school’ and ‘special education boarding school’, were considered correctly classified despite being residual and absent from the public sector of both countries. Additionally, ‘day school’ and ‘student transport to special education facility’ were accurate at the level of DESDE code, although the latter lacked specificity regarding target population due to unspecified age groups. ‘Education support at your place of study’, and ‘education support in a private setting’ received high intensity codes within the provided range. Finally, ‘special education day school’ required a code for severity. ‘Higher education’ and ‘special education’, were considered umbrella terms and not useful for classification.
Suggestions made by experts for adapting the DESDE system to the education sector included the future development of new codes to differentiate day and evening school, reflecting differences in operational contexts such as hours, staffing, or resources. As well as differentiating regular education from special services provided within regular school premises. However, the information obtained in the study is established as the first step for the development of a classification of education services that will enable comprehensive mapping of the education sector in specific geographical areas. This standardized framework will facilitate systematic data collection and better comparison on the availability of services in a region at every level of the ecological system (micro, meso and macro).
Integrating international classifications has demonstrated a potential for disambiguation and therefore improving the comparability of resources across settings [20,44]. The combined use of complementary classification systems of services, interventions and professionals has proved its value in health (e.g. psychotherapy) [44] and social care (e.g. case management) [45]. In education, the description of service provision using DESDE can be combined with information on programs and qualification levels from ISCED and the types of professionals involved (ISCO-International Standard Classification of Occupations [47]. Thus, a more comprehensive overview of the education sector could be achieved in evaluating a primary school for special education, DESDE could provide insights into the facilities structured day care (i.e. D4.2 day care for a fixed number of hours/days); ISCED could clarify the level of education offered such as primary education focusing on basic skills like reading, writing and mathematics (ISCED level 1); and ISCO could detail the roles of professionals, like special education teachers (ISCO code 2340). This approach would enhance comparability and support evidence-informed decision-making [13].
Limitations
The findings of this study must be seen in light of some limitations. First, this study is a proof of concept for the development of an ontology-based classification of education services and its related terminology. It has been based on the knowledge transfer from health and social service research to the education sector. The DESDE classification system was developed for and has traditionally been used in health and social services research with corresponding wording and rationale. Nonetheless, the DESDE classification system has proven to be valid for evaluating care services in other target populations like old age and people with addiction and disabilities [22]. Second, this study relies on the list of specific education services related to mental health, as an inclusion criterion described in the PECUNIA project. Therefore, this study does not provide the full terminology and nomenclature of services provided in the education sector, although the systematic review and the expert survey were international and comprehensive [39]. Third, a limited number of experts was consulted in each country, which could have provided a limited perspective on the applicability of DESDE to classifying education services in Spain and in the Netherlands.
Implications for further research
Comparison of service provision, regardless of the sector should be grounded in standard methodologies. Using well-defined units of analysis ensures commensurability, i.e. like-with-like comparisons within or across areas [48]. Classifications and glossaries help overcome terminological bias preventing ambiguity and vagueness in the naming and description of existing services as well as avoid overshadowing duplication of services while recognizing the actual diversity of services [49]. This study presents a knowledge transfer from the health to the education sector by applying the DESDE system and lays out methods for applying this approach in countries other than Spain and the Netherlands.
As the first step, this proof of concept is important to set the ground for conducting a comprehensive mapping of an area and to study the feasibility and expanding the classification to encompass the entire education system. Standardized data enables policymakers and educators to make decisions based on robust evidence thereby enhancing effective service planning and delivery. Additionally, adopting a standardized classification for a whole system approach, would facilitate deeper integration with other sectors like healthcare, ensuring that policies and practices are interconnected across sectors, enhancing evidence-informed decision-making, and optimizing allocation of resources.
Supporting information
S2 Table. Commensurability assessment (unit of analysis).
https://doi.org/10.1371/journal.pone.0306539.s002
(DOCX)
Acknowledgments
Authors want to acknowledge the contribution of Ana Gimenez Ciruela, PhD, Counselor at the Regional Ministry of Education of Andalucía (Spain); Ana Rodriguez Meirinhos, Department of Education, Universidad Loyola Andalucía (Spain); Prof. Trudie Schills, Professor of Economics of Education at Maastricht University (The Netherlands); Ingrid Ottenheijm, Director of the Special Education Partnership in South Limburg (The Netherlands); Mrs Milou Samuels, Teach and Talent Coach for gifted children at Bernardinus College (The Netherlands), and the PECUNIA group for their contribution to the development of the list of services.
References
- 1. Richards MC, Benson NM, Kozloff N, Franklin MS. Remodeling Broken Systems: Addressing the National Emergency in Child and Adolescent Mental Health. Psychiatric services. 2024 Mar 1;75(3):291–3. pmid:37711021
- 2. Stoshikj M, Kryvinska N, Strauss C. Service Systems and Service Innovation: Two Pillars of Service Science. Procedia computer science. 2016 Jan 1; 83:212–20.
- 3.
Sterling SR. Whole systems thinking as a basis for paradigm change in education: Explorations in the context of sustainability. [Bath]: University of Bath; 2003.
- 4.
Savigny D de, Adam T, for Health Policy A, Research S, Organization WH. Systems thinking for health systems strengthening / edited by Don de Savigny and Taghreed Adam. World Health Organization; 2009. p. 107 p.
- 5. Furst MA, Gandré C, Romero López-Alberca C, Salvador-Carulla L. Healthcare ecosystems research in mental health: A scoping review of methods to describe the context of local care delivery. BMC Health services research. 2019 Mar; 19(173). pmid:30885186.
- 6. Wellander L, Wells MB, Feldman I. Does prevention pay? Costs and potential cost-savings of school interventions targeting children with mental health problems. The journal of mental health policy and economics. 2016 Jun; 19:91–101. pmid:27453456.
- 7. Murphy JM, Guzmán J, McCarthy AE, Squicciarini AM, George M, Canenguez KM, et al. Mental health predicts better academic outcomes: A longitudinal study of elementary school students in Chile. Child psychiatry & human development. 2015 April; 46(2):245–56. pmid:24771270.
- 8. Almeda N, Garcia-Alonso CR, Gutierrez-Colosia MR, Salinas-Perez JA, Iruin-Sanz A, Salvador-Carulla L. Modelling the balance of care: Impact of an evidence-informed policy on a mental health ecosystem. PLoS One. 2022 Jan; 17(1):e0261621. pmid:35015762.
- 9. Castleton B, Dunstan C, Mattocks R, Cpsychol CR. Whole system approach: from pilot to practice. Quality in Primary Care. 2003;11(2):91–5.
- 10. Rosen A, Salvador‐Carulla L. No service is an island: towards an ecosystem approach to mental health service evaluation. World Psychiatry. 2022 Jun; 21(2):237–8. pmid:35524610.
- 11. Gutierrez-Colosia MR, Hinck P, Simon J, Konnopka A, Fischer C, Mayer S, et al. Magnitude of terminological bias in international health services research: A disambiguation analysis in mental health. Epidemiology and psychiatric sciences. 2022 Aug; 31:e59. pmid:35993182.
- 12. Markowitsch J, Luomi-Messerer K. Development and Interpretation of Descriptors of the European Qualifications Framework. European journal of vocational training. 2008;42(1):33–58.
- 13.
OECD. Classifying Educational Programmes: Manual for ISCED-97 Implementation in OECD Countries. Paris: OECD Publishing; 1999. Available: chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.oecd.org/education/1841854.pdf.
- 14.
OECD. Education at a Glance 2019: OECD Indicators. Paris: OECD Publishing; 2019. Available: https://www.oecd-ilibrary.org/education/education-at-a-glance-2019_f8d7880d-en.
- 15.
UNESCO Institute for Statistics. International Standard Classification of Education: ISCED 2011. Montreal, Quebec: UIS; 2012.
- 16. Rowling L. Strengthening “school” in school mental health promotion. Health Education. 2009 Jun; 109(4):357–68.
- 17. O’reilly M, Svirydzenka N, Adams S, Dogra N. Review of mental health promotion interventions in schools. Social psychiatry and psychiatric epidemiology 2018 Jul; 53:647–62. pmid:29752493.
- 18. Salvador-Carulla L, Fernandez A, Sarma H, Mendoza J, Wands M, Gandre C, et al. Impact of Ed-LinQ: A Public Policy Strategy to Facilitate Engagement between Schools and the Mental Health Care System in Queensland, Australia. International journal of environmental research and public health. 2021 Jul; 18(15):7924. pmid:34360229.
- 19. Salvador-Carulla L, Alvarez-Galvez J, Romero C, Gutiérrez-Colosía MR, Weber G, McDaid D, et al. Evaluation of an integrated system for classification, assessment and comparison of services for long-term care in Europe: The eDESDE-LTC study. BMC Health services research. 2013 June; 13(1). pmid:23768163.
- 20. Romero-Lopez-Alberca C, Alonso-Trujillo F, Almenara-Abellan JL, Salinas-Perez JA, Gutierrez-Colosia MR, Gonzalez-Caballero JL, et al. A Semiautomated Classification System for Producing Service Directories in Social and Health Care (DESDE-AND): Maturity Assessment Study. Journal of medical internet research. 2021 Mar; 23(3):24930. pmid:33720035.
- 21. Salvador-Carulla L, Poole M, Gonzalez-Caballero JL, Romero C, Salinas JA, Lagares-Franco CM. Development and usefulness of an instrument for the standard description and comparison of services for disabilities (DESDE). Acta Psychiatrica Scandinavica. 2006 Nov; 114(SUPPL. 432):19–28. pmid:17087812.
- 22. Romero-López-Alberca C, Gutiérrez-Colosía MR, Salinas-Pérez JA, Almeda N, Furst M, Johnson S, et al. Standardised description of health and social care: A systematic review of use of the ESMS/DESDE (European Service Mapping Schedule/Description and Evaluation of Services and DirectoriEs). European Psychiatry. 2019 Sep 1;61:97–110. pmid:31426008.
- 23. Cetrano G, Salvador-Carulla L, Tedeschi F, Rabbi L, Gutiérrez-Colosía MR, Gonzalez-Caballero JL, et al. The balance of adult mental health care: Provision of core health versus other types of care in eight European countries. Epidemiology and psychiatric sciences. 2020 Oct; 29(e6). pmid:30328401.
- 24.
Roche C. Ontoterminology: How to unify terminology and ontology into a single paradigm. Proceedings of the 8th International Conference on Language Resources and Evaluation, LREC. 2012. Available: http://www.lrec-conf.org/proceedings/lrec2012/pdf/567_Paper.pdf.
- 25.
OECD. Education GPS—Spain—Overview of the education system (EAG 2022). 2022 [cited 19 Jul 2023]. Available: https://gpseducation.oecd.org/CountryProfile?primaryCountry=ESP&treshold=10&topic=EO.
- 26.
OECD. Education Policy Outlook—Spain. 2018 [cited 19 Jul 2023]. Available: https://web-archive.oecd.org/2021-01-22/492806-education-policy-outlook-country-profile-spain-2018.pdf.
- 27. Arizmendi A, Matarranz M. Política educativa y educación especial en la Unión Europea: estudio comparado en España e Italia. Revista de educación inclusiva. 2020;13: 125–144.
- 28.
OECD. Education GPS- THe Netherlands. 2023 [cited 20 Jul 2023]. Available: https://gpseducation.oecd.org/CountryProfile?primaryCountry=NLD.
- 29.
Government of the Netherlands. Pre-vocational secondary education (VMBO). 2020. Available: https://www.government.nl/topics/secondary-education/different-types-of-secondary-education/pre-vocational-secondary-education-vmbo.
- 30.
Government of the Netherlands. Senior general secondary education (HAVO) and pre university education. 2020. Available: https://www.government.nl/topics/secondary-education/different-types-of-secondary-education/senior-general-secondary-education-havo-and-pre-university-education-vwo.
- 31.
Government of the Netherlands. (Secondary) special education (In Dutch: (Voortgezet) speciaal onderwijs). In: Secondary) special education (In Dutch [Internet]. Available: https://www.rijksoverheid.nl/onderwerpen/passend-onderwijs/speciaal-onderwijs.
- 32.
Ministry of Education. Culture and Science Inspectorate of Education—The Dutch educational system. The Dutch educational system. [cited 20 Jul 2023]. Available: https://english.onderwijsinspectie.nl/inspection/the-dutch-educational-system.
- 33.
OECD. Education Policy Outlook Netherlands. 2014. Available: https://web-archive.oecd.org/2021-01-22/326698-education%20policy%20outlook_netherlands_en%20.pdf.
- 34. Nash T, Norwich B. The initial training of teachers to teach children with special educational needs: A national survey of English Post Graduate Certificate of Education programmes. Teaching and teacher education. 2010 Oct; 26(7):1471–80.
- 35. Vélez-Clavo X, Tárraga-Mínguez R, Fernández-Andrés MI, Sanz-Cervera P. Formación inicial de maestros en Educación Inclusiva: una comparación entre Ecuador y España. Revista de Educación Inclusiva. 2017; 9: 75–94.
- 36.
Government of the Netherlands. Working in Education (In Dutch: Werken in het onderwijs). 2020 [cited 20 Jul 2023]. Available: https://www.rijksoverheid.nl/onderwerpen/werken-in-het-onderwijs/vraag-en-antwoord/.
- 37.
World Health Organization. International Statistical Classification of Diseases and Related Health Problems 10th revision (ICD-10). Geneva; 2004 Jan. Available: https://icd.who.int/browse10/2019/en.
- 38.
World Health Organization. International Classification of Functioning, Disability and Health (ICF). 2001. Available: https://apps.who.int/iris/bitstream/handle/10665/42407/9241545429.pdf?sequence=1.
- 39. Pokhilenko I, Janssen LMM, Evers SMAA, Drost RMWA, Simon J, König HH, et al. Exploring the identification, validation, and categorization of costs and benefits of education in mental health: The PECUNIA project. International journal of technology assessment in health care. 2020 Jul; 36(4):325–31. pmid:32715991.
- 40. Fischer C, Mayer S, Peric N, Simon J, Group on behalf of the P. Establishing a comprehensive list of mental health-related services and resource use items in Austria: A national-level, cross-sectoral country report for the PECUNIA project. PLoS One. 2021 Jan; 17(1):e0262091. pmid:35061766.
- 41. Pokhilenko I, Janssen LMM, Paulus ATG, Drost RMWA, Hollingworth W, Thorn JC, et al. Development of an Instrument for the Assessment of Health-Related Multi-sectoral Resource Use in Europe: The PECUNIA RUM. Applied health economics Health Policy. 2023 Jan; 21:155–66. pmid:36622541.
- 42. Mayer S, Berger, Michael Konnopka Alexander Brodszky, Valentin Evers S, Hakkaart-van Roijen L, Guitérrez-Colosia M, et al. In Search for Comparability: The PECUNIA Reference Unit Costs for Health and Social Care Services in Europe. International journal of environmental research and public health. 2022 Mar; 19(6):3500. pmid:35329189.
- 43. Pokhilenko I, Kast T, Janssen LMM, Evers SMAA, Paulus ATG, Simon J, et al. International omparability of reference unit costs of education services: when harmonizing methodology is not enough (PECUNIA project). Expert review of pharmacoeconomics & outcomes research. 2023 Jan; 23(1):135–41. pmid:36472303.
- 44. Castelpietra G, Simon J, Gutierrez-Colosía M, Rosenberg S, Salvador-Carulla L. Disambiguation of psychotherapy—A search for meaning. British Journal of Psychiatry. 2021 Nov; 219(4):532–7. pmid:33143767.
- 45. Lukersmith S, Taylor J, Salvador-Carulla L. Vagueness and ambiguity in communication of case management: A content analysis in the Australian national disability insurance scheme. International journal of integrated care. 2021 Mar; 21(1). pmid:33776606.
- 46. Salvador-Carulla L, Furst MA, Tabatabaei-Jafari H, Mendoza J, Riordan D, Moore E, et al. Patterns of service provision in child and adolescent mental health care in Australia. Journal of child health care. 2022 Dec; 13674935221146381 pmid:36538047.
- 47.
International Standard Classification of Occupations. ISCO-08 Structure, Index Correspondence with ISCO-88. ISCO; 2008. Available: https://www.ilo.org/publications/international-standard-classification-occupations-2008-isco-08-structure.
- 48. Salvador-Carulla L, Amaddeo F, Gutiérrez-Colosía MR, Salazzari D, Gonzalez-Caballero JL, Montagni I, et al. Developing a tool for mapping adult mental health care provision in Europe: The REMAST research protocol and its contribution to better integrated care. International journal of integrated care. 2015 Dec; 15(e042). pmid:27118959.
- 49. Montagni I, Salvador-Carulla L, Mcdaid D, Straßmayr C, Endel F, Näätänen P, et al. The REFINEMENT Glossary of Terms: An International Terminology for Mental Health Systems Assessment. Administration and Policy in Mental Health and Mental Health Services Research. 2018 Mar; 45(2):342–51. pmid:28918502.