https://orcid.org/0000-0002-1371-1659
Figures
Abstract
Background
The first stage of diagnosing autism spectrum disorders usually involves population screening to detect children at risk. This study aims to assess the predictive convergent validity of the Polish version of the Communication and Symbolic Behavior Scales-Developmental Profile Infant-Toddler Checklist (CSBS-DP ITC) with the Autism Spectrum Rating Scales (ASRS), evaluate its sensitivity and specificity and assess the cut-off points for the possibility of using this questionnaire in population screening among children aged 6 to 24 months.
Method
The study was conducted among 602 children from the general population who had previously participated in the earlier phase of validation of the questionnaire for Polish conditions. The collected data were statistically processed to calculate the accuracy (i.e. sensitivity, specificity) of the questionnaire.
Results
In individual age groups, the sensitivity of the questionnaire varies from 0.667 to 0.750, specificity from 0.854 to 0.939, positive predictive value from 0.261 to 0.4 and negative predictive value—from 0.979 to 0.981. Screening accuracy ranges from 0.847 to 0.923 depending on the age group. The adopted cut-off points are 21 points for children aged 9–12 months, 36 for children aged 13–18 months, 39 for children aged 19–24 months. Cut-off points could not be established for children aged 6–8 months. The convergent validity values with the ASRS ranged from -0.28 to -0.431 and were highest in the group of the oldest children.
Citation: Sobieski M, Kopszak A, Wrona S, Bujnowska-Fedak MM (2024) Screening accuracy and cut-offs of the Polish version of Communication and Symbolic Behavior Scales-Developmental Profile Infant-Toddler Checklist. PLoS ONE 19(8): e0299618. https://doi.org/10.1371/journal.pone.0299618
Editor: Amin Nakhostin-Ansari, Tehran University of Medical Sciences, ISLAMIC REPUBLIC OF IRAN
Received: February 19, 2024; Accepted: June 6, 2024; Published: August 9, 2024
Copyright: © 2024 Sobieski et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data can be accessed at https://doi.org/10.6084/m9.figshare.26077021.v1.
Funding: The study was financed from the researchers' own funds and the internal funds of Wroclaw Medical University - the project is financed from the scientific grant: "Aspects of preventive care, diagnosis and therapy of patients of different ages under the care of a family doctor, including e-health, telemedicine solutions, coordination and analysis of care effectiveness indicators”; Task SIMPLE number: SUB.C290.19.054. The external funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Autism spectrum disorders (ASD) are neurodevelopmental conditions of undetermined and complex etiology, most often manifested in early childhood, which affect the everyday activities of individuals and are characterized primarily by difficulties in the sphere of communication and interpersonal interactions as well as restricted interests or repetitive behaviors [1]. The prevalence of ASD is not clearly defined—the World Health Organization (WHO) estimates that ASD affects 1 in 160 children worldwide [2]. More recent reports from a systematic review by Zeidan et al. estimate the worldwide prevalence of ASD as 1 in 100 children [3] at the same time indicating significant variability in prevalence depending on the country and research assumptions used in individual studies. According to the only official data from the Polish National Health Fund from 2012, the prevalence rate of ASD in individuals under the age of 18 in Poland is 3.4 cases per 10,000 children [4], but more accurate preliminary data from two Polish provinces estimate that ASD occurs in one in 286 children [5].
The first symptoms of ASD usually appear during early child development—prodromal symptoms may be visible as early as 6 months of age [6, 7]. Later in development (from 14 months to 3 years of age) communication and social behavioral symptoms become apparent [8]. It is also possible that development plateaus after a period when the age-appropriate milestones were achieved or that the skills acquired earlier are lost [9, 10]. It is assumed that a reliable diagnosis of ASD in a child can be made as early as the 2nd–3rd year of age [11]. However, due to delays associated with the diagnostic process (or its omission), this diagnosis is made much later—in a world-wide 2019 meta-analysis, the average age of ASD diagnosis is 60.48 months (range 30.90–234.57) [12].
Early diagnosis of ASD enables the initiation of an early, age-adjusted therapy of developmental delays and difficulties. The younger the child, the better the results of therapy can be achieved in the area of communication and social interaction, cognitive abilities, speech development, or behavior appropriate to the situation, which improves the quality of life of people with ASD, reduces the risk of mental disorders, and significantly reduces the burden of ASD [13–16].
Considering this, the American Academy of Pediatrics (AAP) recommends screening at 18 and 24 months of age as part of primary care during well-child care visits [17]. It appears that the increasing availability of screening significantly lowered the age of ASD diagnosis in the US, with diagnosis before the age of 4 made in 71% of children (2018) compared to 58% in 2014 [18, 19]. On the other hand, a 2016 report from the US Preventive Services Task Force shows insufficient evidence to recommend universal ASD screening [20]. However, there is evidence suggesting that including screening tools in routine medical appointments may result in earlier and more accurate identification of children who need further help compared to relying solely on clinical impressions, which is particularly important when care providers are less experienced in diagnosing ASD [21]. Moreover, the use of public ASD screening may reduce social inequities in terms of the age of diagnosis and access to further therapeutic activities [22, 23]. The conclusions of both reports indicate the need for further research on screening tools and their effectiveness, as well as on the effectiveness of further proceedings after screening [24].
Due to the lack of tools for early diagnosis of ASD available in Poland, the authors have prepared a linguistically and culturally adapted version of the original, American version of Communication and Symbolic Behavior Scales-Developmental Profile—Infant-Toddler Checklist (CSBS-DP ITC) [25]. CSBS-DP ITC is one of the available tools created for the early detection of symptoms of autism spectrum disorders. It is a 24-item questionnaire for parents or caregivers. The included questions are arranged into three composites (social, speech, and symbolic composite) and seven development predictors (emotion and eye gaze, communication, gestures, sounds, words, understanding, and object use). Original CSBS-DP ITC version can be used in universal ASD screening of children aged 6 to 24 months in a primary care setting [26]. The research showed that the result obtained in the original version of questionnaire could confidently predict the level of language development two years in advance and is an effective tool for the screening of children with special needs [25–27].
In order to conduct a validation study and disseminate knowledge about the research, the "Spojrzeć w oczy” (Pol. “Look into the eyes”) project was established. The project aimed to determine the psychometric properties (e.g. validity and reliability, sensitivity and specificity) of the Polish version of the CSBS-DP ITC. The questionnaire was adapted and translated using the back translate method by three independent translators, and the questions included were adjusted to the phonetics of the Polish language and the speech development of Polish children. A detailed description of the preparation of the tool for Polish cultural conditions is included in the earlier publication on CSBS-DP ITC validation [28]. The Polish version of CSBS-DP ITC is included as S1 File. Questionnaire and its English re-translation as S2 File. Data from the earlier stage of the project indicate a very good fit of the one-factor and three-factor models in confirmatory factor analysis. The total score of the Polish version of CSBS-DP ITC demonstrated satisfactory internal consistency, Cronbach’s α = .92, and McDonald’s ω = .92. The stability of the measurement was confirmed by performing interrater and test-retest reliability analysis, proving perfect and satisfactory level of stability, respectively.
This study aims to assess the predictive convergent validity of the Polish version of the CSBS-DP ITC with the Autism Spectrum Rating Scales (ASRS) questionnaire, used by psychologists for screening children with suspected ASD and those at risk of it, evaluate the sensitivity and specificity of the Polish version of the CSBS-DP ITC and finally assess the cut-off points for the use of the questionnaire in population screening.
Methods
Participants
The participants were children aged 30 months and their parents or caregivers who participated in the earlier phase of the project by correctly filling in the Polish version of CSBS-DP ITC questionnaire when children were between 6 and 24 months of age. The condition for inclusion was living in Poland, speaking Polish as one’s primary language, and giving informed, written consent to participate in the study. Project recruitment began on October 25, 2020 using advertisements placed in collaborating health care facilities and on social media, and ended on February 18, 2021. During this period, parents completed the Polish version of CSBS-DP ITC screening questionnaire. In addition, as part of the "Spojrzeć w oczy" project, materials on the symptoms, diagnosis and therapy of autism spectrum disorders were made available to parents and health care specialists—both in physical and online form [29]. The research phase lasted from April 28, 2021, to October 12, 2022 –until the last child included in the project turned thirty months of age. As some of the children (N = 34) were awaiting the final diagnosis, their parents were contacted by phone at a later date. By March 30, 2023, a final diagnosis was obtained in all of them, except one (due to diagnostic difficulties, it was not possible to unambiguously confirm the presence of autism spectrum disorders in that particular case). During and after the data collection, only MS and SW had the opportunity to identify individual study participants—this was to enable the provision of further psychological and pedagogical assistance and further diagnostics.
Invitations to the follow-up were sent to all 1461 parents of children who were included in the first phase of the project. A total of 678 submissions were received back, 76 reports were excluded from the study– 48 of them because parents had not previously participated in Polish CSBS-DP ITC screening, and in 11 cases siblings were assessed in the follow-up instead of the child who was screened, and another 17 were repeated submissions. When attempts were made to contact the other parents via e-mail or telephone (N = 859), as many as 722 of them admitted that they did not complete the further part of the study due to the lack of symptoms in their children and hence—lack of willingness to remain in the study. Finally, 602 children were enrolled in the study, whose parents had fully completed the ASRS questionnaire and follow-up interview, and who had participated in an earlier CSBS-DP ITC screening, which gives a return rate of 41.2%. The number of participants significantly exceeds the minimum sample size to have a confidence level of 95% and a margin of error of 5% in the Polish population (the calculated minimum sample size was 139) [30]. The mean age of the children was 30.21 months (SD = 1.03). The vast majority of questionnaire forms were filled in by the children’s mothers (N = 601, 99.83%)–only one of them was filled in by a father. Sample characteristics of the study participants are presented in Table 1. The individual phases of the project and the number of participants at each stage are collected in Fig 1.
Note. ASRS—Autism Spectrum Rating Scales, CSBS-DP ITC—Communication and Symbolic Behavior Scales-Developmental Profile—Infant-Toddler Checklist, Polish Version, DD—developmental delay, LD—language delay.
Approval from the Bioethics Committee of the Wroclaw Medical University was obtained to conduct the study (number KB– 641/2020; the full text of the consent is available as S3 File and its translation—as S4 File). All study participants gave written consent to participate in the study. All procedures were performed in accordance with the 1964 Helsinki Declaration and its later amendments.
Evaluation instruments
Autism Spectrum Rating Scales (ASRS).
ASRS is a set of questionnaires, used as an auxiliary tool in the diagnostic process or a screening tool, consisting of full and abbreviated versions of tests for younger children (2–5 years) and older children and adolescents (6–18 years), both for parents and teachers [31]. The ASRS was developed in 2009 and its structure corresponds to the diagnostic criteria of Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR) and International Statistical Classification of Diseases and Related Health Problems, 10th Revision (ICD-10). The Polish version was prepared in 2016 and is characterized by high reliability in the version for parents and has a confirmed discriminatory, convergent, and differential validity [32]. Additionally, the Polish version corresponds to of Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) diagnostic criteria. In the study, the short version for younger children was used, which contains 15 items that best differentiate between children diagnosed with ASD and the comparative group from the non-clinical population. The analysis of the discriminant function for raw scores revealed that the indicators of correct classification ranged from 88.2% to 91.4%. Cronbach’s alpha for the ASRS version used in the study is 0.85. Sensitivity and specificity are 87.8% and 83.5%, respectively.
Further evaluation instruments.
As ASRS is a Level 2 ASD diagnostic tool, parents were asked for additional information to determine the need for further diagnosis. In order to further evaluate children’s development, a short structured interview (follow-up interview) was created. They were also asked if their child had received an ASD diagnosis from a psychiatrist (understood as disorders included in the ICD-10 classification as F84.0, F84.1, or F84.5 or in the DSM-5 as autism spectrum disorder). They were also asked if their child had completed any of the available standardized diagnostic protocols available in Poland: Autism Diagnostic Interview-Revised (ADI-R), Psychoeducational Profile-3rd Edition-PL (PEP-3-PL), or Autism Diagnostic Observation Schedule, 2nd Version (ADOS-2) and what was the result of the conducted examination [33–35]. Information on the diagnostic provider’s credentials and experience was collected when possible to avoid the possible impact of extraneous variables on the results. A positive response in terms of receiving a nosological diagnosis of ASD from a psychiatrist or a positive ADOS-2 test result meant qualification to the group of children diagnosed with ASD. If the child received another diagnosis (e.g. language delay—LD), this was also recorded in the database used in the study.
Procedures
Due to the COVID-19 pandemic and difficulties in conducting such a study face-to-face in healthcare clinics, an electronic version of the questionnaire was prepared and made available on the project’s website examining the properties of the tool. Initially, the parents completed the Polish Version of CSBS-DP ITC questionnaire along with a short record of the mother’s and father’s age and the child’s comorbidities (genetic, hearing and vision disorders, movement disorders, family history), birth weight, and the week of pregnancy in which the delivery took place. Then, upon a child’s turning 30 months of age, the respective child’s parents were contacted again and asked to fill in the ASRS questionnaire and a short follow-up interview, which included questions about the occurrence of symptoms that were concerning to the parents and further diagnostics.
In the case of suspicion of ASD in a child (understood as a result above ASRS cut-off point or any persistent concerns parents have about their child’s development despite negative screening using the ASRS), the ADOS-2 was performed to make it possible to determine with high probability the presence of autism spectrum disorders in children from the risk group. At each stage during the study, parents were offered the possibility of carrying out the ADOS-2 test at the research center in Cieszyn, Silesian Voivodeship. However, due to the distribution of the population throughout the country, the majority of them preferred performing further diagnostics near their place of residence. If children underwent the ADOS test before our intervention or the ADOS test was performed in another center, we tried to verify the test by contacting the examiner to confirm whether the test result indicated the presence of autism spectrum disorders in the child.
Statistical analysis.
All the analyses presented in this manuscript were performed using Statistica 13.1 and R 4.3.0 software using packages ggplot2 3.5.1 and stats. Due to the wide variation of results in individual age groups (resulting from rapid development in the age group analyzed) and the insufficient number of participants at a particular age in months, it was necessary to group the participants into age cohorts gathering children at least partially at a similar level of psychomotor development. Participants were divided into four groups based on the substages of Piaget’s Sensorimotor Stage theory of cognitive development [36]. The first group included children aged from 6 to 8 months of age, the second group included children between 9 and 12 months, the third group included children between 13 and 18 months, and the fourth group included children above 19 months.
Because more complex tools are also used during further diagnostics of children with suspicion of ASD, first it was checked whether there was any correlation between the results achieved by children in the CSBS-DP ITC questionnaire and the shortened version of the ASRS (used as a Level 2 questionnaire) using the Spearman’s rank correlation coefficient method. In addition, since the children were grouped into four age cohorts, it was checked whether the shape of the relationship between CSBS-DP ITC Total Score and Total ASRS is influenced by the variable “Age”. The last aspect that was checked was the relationship between the variables of the child’s age (in months) and the Total Score obtained in the CSBS-DP ITC questionnaire. To verify the linearity, a normality analysis of the residuals of the linear model was performed using the Kolmogorov—Smirnov and Shapiro—Wilk tests. The tests showed no normality of the residuals in the model. For this reason, non-parametric methods were used for further analysis.
The discriminating ability of CSBS-DP ITC was examined to ensure that it was appropriate for each age cohort. This was achieved by conducting a receiver operating characteristics (ROC) analysis for all four age groups. The discriminating ability of the Polish Version of CSBS-DP ITC was determined by the Area Under the Curve (AUC) statistic. AUC values ranged from 0.5, representing a random chance efficacy, to 1.0, representing excellent performance in discriminating between the two conditions [37]. For each of the groups for which the ROC analysis was successful, potential cut-offs were determined to enable the tool to be used in practice as a clinical guide for further management. Two different methods were used to determine the cut-offs—the first was the cut-off point at which maximum sensitivity and specificity were determined using the Youden index, which represents the overall accuracy of the test [38, 39]. Additionally, the minimizing expected costs method was used by including a “decision threshold” in constructing tangents because potentially classifying a child with ASD as a healthy individual carries a higher cost (in terms of delay in treatment) than the opposite mistake (Zweig & Campbell, 1993). We assumed that the weight of these errors (misclassification costs) increases with the age of the child (and therefore with the delay in diagnosis). It was further assumed that the costs of misclassification (for children falsely classified as healthy—false negative) would be measured as 1, 2, 3, and 5, for the respective groups as age increases; values were adopted arbitrarily.
Due to the unclear prevalence of ASD in the Polish population, three different probabilities were used to determine the cut-off points using the tangent method—random (0.5), declared by WHO for the world population (1:160) and the probability estimated from the study population (depending on the number of cases in a given cohort). Thus, a total of twelve ROC curves were created and analyzed, and the results of these analyses were used to establish the sensitivity and specificity in specific age groups.
Results
Descriptive statistics
In the study population, among 602 children, 65 of them were further evaluated due to a positive ASRS test result or the presence of disturbing symptoms. Ultimately, 35 children from the entire group were diagnosed with ASD and 20 with language development delay. Data on the sex structure and incidence of developmental disorders in individual age-subgroups are detailed in Table 2.
The largest number of ASD cases (both percentage and total) were confirmed in the subgroup of children aged 13–18 months (n = 16; 6.61%), the smallest—in the subgroup of children aged 6 to 8 months (n = 2; 2.74%). In each age group, boys slightly dominated the study population (from 53.21% to 63.64%).
Correlation between total scores of ASRS and CSBS-DP ITC
To assess predictive accuracy between the earlier CSBS-DP ITC questionnaire result and the later ASRS, Spearman’s rank correlation coefficient analysis was used. Since the Total Score achieved in the CSBS-DP ITC questionnaire depends on the age of the examined child, it was additionally checked whether the child’s age affects the strength of the relationship between Total CSBS-DP ITC scores. The results are presented in the Table 3 and Figs 2 and 3.
Each dot indicates one child’s score on the CSBS-DP ITC questionnaire (Y-axis) in relation to the score on the ASRS questionnaire (X-axis), the color corresponds to the child’s age at the time of completing the CSBS-DP ITC questionnaire. Lower scores on the CSBS-DP ITC are also observed in the group achieving low ASRS scores due to lower age.
Each dot indicates the rank score one child’s score on the CSBS-DP ITC questionnaire (Y-axis) and ASRS (X-axis) with a marked trend (which is the highest for the oldest children and the lowest—for the youngest).
Statistically significant weak or moderate correlations are present between the total ASRS score and the total CSBS-DP ITC score in the three oldest age groups. It should be noted that in each age cohort, there are weak or moderate correlations between the child’s age and the total score obtained in CSBS-DP ITC. It should be borne in mind that these two questionnaires have an opposite scale—in the CSBS-DP ITC, the higher the score, the lower the risk of ASD in the child, while for the ASRS the opposite is true, which is clearly visible in Fig 2. The older the children examined, the more pronounced the inverse relationships between the results achieved in the CSBS-DP ITC questionnaire and the ASRS. In order to better present this relationship, an analysis was performed for age groups using ranking (due to the previously mentioned lack of normal distribution of variables).
Sensitivity, specificity of the Polish version of CSBS-DP ITC
In order to determine the sensitivity, specificity and other measures of the classification test for the Polish version of the CSBS-DP ITC, this study used the Youden method of ROC analysis and the tangential method to minimize the potential costs associated with misclassification. Results of the analyses and ROC charts for individual age groups using the Youden method are provided in Fig 4a–4c and Table 4. Results using the tangential method with different probabilities of ASD occurrence, due to its lower effectiveness compared to Youden’s method, are included in SF5.
A-C. ROC analysis charts in individual age groups using the Youden method.
Due to the small number of ASD cases in the first group (children aged 6 to 8 months), the ROC analysis in this subgroup could not be performed.
The highest sensitivity and specificity values were achieved using the Youden method and cut-offs did not change depending on the assumed probability of ASD occurrence—when using the prevalence "from the sample", random (0.5) and prevalence based on data from systematic reviews (1:160 according to WHO), the ROC curves plotted using the Youden method did not differ from each other. In the case of using the tangential method, the adopted cut-offs significantly differed depending on the assumed probability of ASD occurrence in the Polish population and they were usually characterized by lower accuracy or a significant disproportion between sensitivity and specificity in favor of one of these parameters. Moreover, adopting cut-offs determined using this method may be too volatile (e.g., 30 or 54 points in group 19–24 months of age), which is also probably due to the small number of cases of children diagnosed with ASD in the sample.
Among all performed ROC analyses, the area under the curve (AUC) index ranged from 0.782 to 0.856, which is a value allowing for the classification ability for the CSBS-DP ITC to be considered as at least above moderate. Among children in the age group of 9–12 months, a cut-off of 21 points was adopted, allowing for a sensitivity of 0.750 and a specificity of 0.862 with a Youden index of 0.612; in the age group of 13–18 months, a cut-off of 36 points was adopted, giving a sensitivity of 0.750 and a specificity of 0.854 with a Youden index of 0.604, and in the group of 19–24 months—a cut-off of 39 points, giving a sensitivity of 0.667 and specificity of 0.939 with a Youden index of 0.605.
Due to the size of the groups, analyzes of the CSBS-DP ITC subscales (Social, Speech, Symbolic components) using ROC curves did not allow for clear cut-off points for children. For this reason, if it is necessary to use subscales to determine a child’s risk, we suggest using the method used in the original version of the CSBS-DP ITC questionnaire (i.e. 1.25 SD below the mean score in the study population). Raw results and cut-off thresholds for this method were described in a previous publication on the Polish version of CSBS-DP ITC [28].
Discussion
The aim of this study was to determine the final psychometric values of the Polish version of the CSBS-DP ITC questionnaire as part of the nationwide project. The questionnaire enables testing children for developmental disorders from the age of 6 months, which is a distinctive feature of this study—previous studies on other diagnostic tools e.g. Modified-Checklist for Autism in Toddlers (Revised) (M-CHAT-R/F), Brief Infant Toddler Social Emotional Assessment (BITSEA), First Year Inventory (FYI) or Quantitative Checklist for Autism in Toddlers (Q-CHAT) were focused on older children [40–43]. In addition to the CSBS-DP ITC, the only screening questionnaire designed to be completed by parents (rather than trained observers) for children aged 6 months and older is the Canadian-validated Autism Parent Screen for Infants (APSI) [44]. Therefore, this study may provide evidence for the purposefulness of screening for developmental disorders among the youngest children in the primary care setting.
Due to the dynamic development of children’s skills in the studied period, it was necessary to take into account the age difference affecting the results achieved by children. In the original version of the CSBS-DP ITC questionnaire, there are separate cut-off points for each individual month of age, however, they have been set as a point below 1.25 SD from the average result in the general population. Due to the lack of sufficient participants in the study with a confirmed diagnosis of ASD, only solution to this problem was to group children into appropriate larger age cohorts.
Data from the analysis of the statistical model indicate that the correlation between the child’s age and the total score in the CSBS-DP ITC questionnaire is also present in age subgroups, which means that children close to the age limit of the cohort may be classified differently by the questionnaire. The influence of age on the total score would probably be even stronger if it was examined without grouping children into cohorts, however, due to its significance, this effect cannot be ignored. Moreover, the analyses do not confirm the linearity of this impact—the introduction of a simple correction based on a linear relationship would involve the risk of error. Therefore, this is another reason why it was necessary to use previously developed theoretical constructs (in this case, Piaget’s theory) to divide children into larger age groups. This, in turn, facilitated the conduct of analyses and the determination of threshold values according to the observed acquisition of particular abilities by children at the appropriate age. With the amount of data available, unfortunately, the only remaining solution is to take into account the correlation with age when interpreting the results and leave a margin of distrust for the scale. This is of particular relevance when evaluating children on the verge of age ranges. In case of doubt, as suggested by the authors of the original version, in the current study it is also recommended to check the child’s development in the next three months.
Based on a sample of 1,461 simple screening questionnaires, at least 35 children were diagnosed with ASD, which gives a prevalence of ASD in the Polish population of 2.39%–considering that the majority of parents who did not participate in the follow-up with the use of ASRS and follow-up interview dropped out due to the lack of suspicion of any symptoms of developmental disorders in their children. This percentage is even higher if the study considers only children assessed using the ASRS and subsequent tests (e.g. ADOS-2) and it may be estimated as 5.81%. Compared to the preliminary data from two Polish regions (which indicate 32–38 cases of ASD per 10,000 people), the prevalence of ASD in the group of respondents in the current study is markedly higher. The percentage of children diagnosed with ASD in whole project (2.39%) is therefore similar to the most recent data for American children (1:64) [19]. Similarly, the potentially unusual high percentage of children who underwent rehabilitation or with detected muscle tension disorders and constituted the study group does not seem to differ significantly from the general Polish population. This results from the high frequency of their detection in the Polish population in accordance with the guidelines of the Children and Youth Rehabilitation Section of the Polish Rehabilitation Society, as well as the high popularity of rehabilitation in Poland [29]. According to data collected on behalf of the National Chamber of Physiotherapists, as many as 49.9% of Poles use the services of physiotherapists; however, current accurate data on children are lacking [30].
However, it should be borne in mind that the selection of the method (due to the COVID-19 pandemic, the study was conducted online) may have resulted in some bias of the group—the questionnaire was filled in by willing parents, so parents observing the occurrence of any deficits in their children may account for a larger part of respondents than it actually could be the case in stationary primary care setting. Nevertheless, every effort was made to convince also parents who do not suspect any developmental disorders in their children to complete the questionnaire. It seems that these efforts were effective– 383 parents from 1461 included in the study reported concerns about their child’s development, which constitutes 25.7% of all participants. This is perfectly in line with the data from a study conducted at the C.S. Mott Children’s Hospital, Michigan, US, where it was shown that a quarter of parents also report suspected developmental disorders in their children [45]. Similar conclusions can be drawn from studies in the Netherlands, where even up to 50% of parents reported minor concerns about their children’s development [46]. Notwithstanding the foregoing, the data from the current study indicate that despite the potential bias of the group, there is still a large proportion of undiagnosed individuals in the general population who could benefit from appropriate diagnosis and treatment.
In order to reduce the rate of false positive results, it may be effective to include an observational study as a follow-up measure immediately after a positive result. The authors of the original version of the CSBS-DP created additional, further tools that aim to increase the sensitivity and specificity of the ITC questionnaire (i.e. Behavior Sample, where a pre-trained healthcare professional (HCP) assesses the child’s development, or Caregiver Questionnaire, where the child’s development is assessed by e.g. a caregiver in a nursery or nanny) [47]. Numerous evidence has also been described that the inclusion of follow-up methods increases the sensitivity and specificity also in the case of other ASD screening tools (e.g. M-CHAT R/F) [48]. In addition, the use of a Behavior Sample by HCPs immediately after an ITC assessment could reduce the potential dropout and be used as an immediate validation strategy. The use of a follow-up method right after the questionnaire-based screening method could be all the more important because some of the families, even after obtaining a result of the screening tests suggesting the possibility of ASD in the child and indicating the need for further diagnostics, still do not feel concerned about the development of their child as a study performed in Flanders suggests [49]. Major obstacles in trying to incorporate additional diagnostic methods in primary care settings are the excessive workload, insufficient time, and inadequate knowledge of HCPs about ASD [50–52].
Another issue worth mentioning is the possibility of using the CSBS-DP ITC questionnaire to detect developmental disorders other than ASD, e.g. language development delay (LD), which were observed in 20 respondents in our study (3.32%), or to assess strengths or weaknesses in a child’s developmental skills. Early detection of children with LD, especially those from high-risk groups, may benefit them in connection with the initiation of appropriate therapy, however, the effects of such therapy are less pronounced than in the case of ASD, as indicated by the results of the Danish SPELL longitudinal study conducted in the general population [53, 54]. The presence of three main CSBS-DP ITC subscales concerning social skills, symbolic skills, and speech development indicates the child’s potential resources and the most important deficits and may serve as a cue for early intervention therapists before further, more specialized diagnostics. However, the unequivocal use of this information in practice requires further research on the CSBS-DP ITC questionnaire specifically in risk groups.
Nevertheless, the evidence from this study may contribute to the discussion regarding the inclusion of ASD screening during well-child care visits. So far, no fully validated questionnaire for the diagnosis of ASD in children under 2 years of age has existed in Poland; only one study on the reliability of the Q-CHAT questionnaire was published, without an attempt to estimate the cut-off in the Polish population or assess the sensitivity and specificity of the tool [55]. Taking into account the positive data from the current study on the CSBS-DP ITC properties, the use of this tool in everyday practice of HCPs would seem justified.
Increasing early diagnostic capabilities through the use of CSBS-DP ITC in screening has the potential to provide large population benefits similar to those observed in the United States, associated with a significant reduction in the average age of diagnosis and an increase in the percentage of children diagnosed with ASD [19]. Taking into account the previously mentioned low official prevalence of ASD in Poland (which is most likely due to incomplete diagnosis, difficulties with access to specialists and lack of guidelines on how to deal with a child with suspected ASD), as well as the high average age at which the diagnosis of ASD in children is made implementing of ASD screening could improve the situation of children and their families [29]. It is estimated that the average age of receiving an ASD diagnosis in Poland is 7 years and 3 months, which is much higher than the world average, which worsens the potential outcomes of ASD therapy in terms of communication or language skills [12, 56].
The Polish version of CSBS-DP ITC is characterized by psychometric parameters slightly lower than the original version, and similar to the Taiwanese version (sensitivity 0.77) or the Italian version (maximum sensitivity 0.67, specificity 0.98, PPV 0.6, NPV 0.98); however, the Italian version achieved significantly lower diagnostic values in younger children [26, 57, 58]. The Polish version of the CSBS-DP ITC questionnaire, probably due to linguistic and cultural adjustment, does not have such a large impact of low age on psychometric values, which means that it can be used in screening children from 9 months of age [59]. This makes it possible to implement early intervention at a very early stage of the child’s development, which will probably enable achieving better final effects of tailored therapy [60].
There are several limitations to this study. The first is the relatively small number of study participants whose development was monitored throughout the study. Although this number fully meets the criteria of scientific research, a larger sample of respondents could provide better evidence of the usefulness of ASD screening in the Polish population and, at the same time, better estimate the prevalence of ASD in this age group in Poland. Another limitation is the method of electronic screening and the related self-selection of participants. It is believed to be one of the main factors for unrepresentativeness in studies conducted via online questionnaires. The attitudes of study participants may have influenced their decision to take part in the survey, which is why parents who suspected any developmental disorders in their children were more willing to participate in the study [61]. However, as mentioned earlier in the discussion, the present authors made every effort to avoid self-selection bias and, given the percentage of parents with any concerns about their child’s development compared to other inpatient studies, most likely the attempt to make it negligible have been successful.
Another possible limitation is the tracking of children’s development using online and telephone methods. Due to remote contact with parents, the authors relied on answers in questionnaires, surveys, and medical documentation provided. Nevertheless, in order to qualify the examined child to the group of children diagnosed with ASD, it was required to present appropriate documents prepared by a qualified psychologist or psychiatrist. The last flaw of the study is the time of observation of children—with the endpoint set at 30 months of age, and the possibility of extending this time if the child is undergoing further diagnostics, has received a positive ASRS result or parents have further concerns about the child’s development. At the same time, according to the DSM-V definition of ASD, the symptoms of autism spectrum disorders may become fully apparent also in later years of development. Despite providing psychological help to children who require it, some of them may develop symptoms later—when social demands exceed the limited capacities of the child [1, 62].
Conclusions
The Polish version of the CSBS-DP ITC questionnaire is characterized by reasonably high values of sensitivity, specificity, and accuracy in children aged from 9 to 24 months and children’s performance in the questionnaire may also be a predictor of outcomes in later examinations (i.e. ASRS). The conducted analyses indicate the potential usefulness of the Polish version of the CSBS-DP ITC questionnaire in the everyday practice of healthcare professionals for population screening for ASD. Further research is essential to more accurately define cut-offs among children who are on the verge of the accepted age ranges and among the youngest population (aged 6–8 months).
Supporting information
S1 File. Questionnaire—The Polish version of CSBS DP-ITC.
https://doi.org/10.1371/journal.pone.0299618.s001
(PDF)
S2 File. Questionnaire—English translation of the Polish version of CSBS DP-ITC.
Re-translated (from Polish to English) Polish version of CSBS-DP ITC.
https://doi.org/10.1371/journal.pone.0299618.s002
(PDF)
S3 File. Ethics committee approval.
Full text of Ethics Committee Approval for conducting this study obtained by the Wroclaw Medical University Ethics Committee.
https://doi.org/10.1371/journal.pone.0299618.s003
(PDF)
S4 File. Translated ethics committee approval.
Translated from Polish full text of Ethics Committee Approval for conducting this study obtained by the Wroclaw Medical University Ethics Committee.
https://doi.org/10.1371/journal.pone.0299618.s004
(DOCX)
S5 File. Results and discussion on sensitivity, specificity of the Polish version of CSBS-DP ITC calculated using tangential method.
https://doi.org/10.1371/journal.pone.0299618.s005
(DOCX)
Acknowledgments
Due to obtaining consent from the authors of the original version of the CSBS-DP ITC questionnaire and the general free availability of the questionnaire, the Polish version of the CSBS-DP ITC questionnaire is covered by the CC BY license, with particular emphasis on the original authors and Paul H. Brookes Publishing Co.
The researchers would like to thank Róża Hajkuś and Kamila and Tomasz Chadaj for their help in recruiting study participants.
References
- 1.
American Psychiatric Association. Cautionary Statement for Forensic Use of DSM-5. Diagnostic and Statistical Manual of Mental Disorders, 5th Edition. Washington DC: American Psychiatric Association Publishing; 2014.
- 2. Elsabbagh M, Divan G, Koh YJ, Kim YS, Kauchali S, Marcín C, et al. Global Prevalence of Autism and Other Pervasive Developmental Disorders. Autism Res. 2012;5: 160–179. pmid:22495912
- 3. Zeidan J, Fombonne E, Scorah J, Ibrahim A, Durkin MS, Saxena S, et al. Global prevalence of autism: A systematic review update. Autism Res. 2022;15: 778–790. pmid:35238171
- 4. Piskorz-Ogórek K, Ogórek S, Cieślińska A, Kostyra E. Autism in Poland in comparison to other countries. Polish Ann Med. 2015;22: 35–40.
- 5. Skonieczna-Żydecka K, Gorzkowska I, Pierzak-Sominka J, Adler G. The Prevalence of Autism Spectrum Disorders in West Pomeranian and Pomeranian Regions of Poland. J Appl Res Intellect Disabil. 2017;30: 283–289. pmid:26771078
- 6. Zwaigenbaum L, Bryson S, Garon N. Early identification of autism spectrum disorders. Behavioural Brain Research. Behav Brain Res; 2013. pp. 133–146. pmid:23588272
- 7. Canu D, Van der Paelt S, Canal-Bedia R, Posada M, Vanvuchelen M, Roeyers H. Early non-social behavioural indicators of autism spectrum disorder (ASD) in siblings at elevated likelihood for ASD: a systematic review. Eur Child Adolesc Psychiatry. 2021;30: 497–538. pmid:32088859
- 8. Jones EJH, Gliga T, Bedford R, Charman T, Johnson MH. Developmental pathways to autism: A review of prospective studies of infants at risk. Neurosci Biobehav Rev. 2014;39: 1–33. pmid:24361967
- 9. Landa RJ, Gross AL, Stuart EA, Faherty A. Developmental Trajectories in Children With and Without Autism Spectrum Disorders: The First 3 Years. Child Dev. 2013;84: 429–442. pmid:23110514
- 10. Pearson N, Charman T, Happé F, Bolton PF, McEwen FS. Regression in autism spectrum disorder: Reconciling findings from retrospective and prospective research. Autism Res. 2018;11: 1602–1620. pmid:30475449
- 11. Chawarska K, Klin A, Paul R, Volkmar F. Autism spectrum disorder in the second year: Stability and change in syndrome expression. J Child Psychol Psychiatry Allied Discip. 2007;48: 128–138. pmid:17300551
- 12. van ‘t Hof M, Tisseur C, van Berckelear-Onnes I, van Nieuwenhuyzen A, Daniels AM, Deen M, et al. Age at autism spectrum disorder diagnosis: A systematic review and meta-analysis from 2012 to 2019. Autism. 2021;25: 862–873. pmid:33213190
- 13. Orinstein AJ, Helt M, Troyb E, Tyson KE, Barton ML, Eigsti IM, et al. Intervention for optimal outcome in children and adolescents with a history of Autism. Journal of Developmental and Behavioral Pediatrics. 2014. pp. 247–256. pmid:24799263
- 14. MacDonald R, Parry-Cruwys D, Dupere S, Ahearn W. Assessing progress and outcome of early intensive behavioral intervention for toddlers with autism. Res Dev Disabil. 2014;35: 3632–3644. pmid:25241118
- 15. Smith T, Klorman R, Mruzek DW. Predicting Outcome of Community-Based Early Intensive Behavioral Intervention for Children with Autism. J Abnorm Child Psychol. 2015;43: 1271–1282. pmid:25778537
- 16. Lai MC, Kassee C, Besney R, Bonato S, Hull L, Mandy W, et al. Prevalence of co-occurring mental health diagnoses in the autism population: a systematic review and meta-analysis. The Lancet Psychiatry. 2019;6: 819–829. pmid:31447415
- 17. Hyman SL, Levy SE, Myers SM, Kuo DZ, Apkon CS, Davidson LF, et al. Identification, evaluation, and management of children with autism spectrum disorder. Pediatrics. 2020;145. pmid:31843864
- 18. Shaw KA, Maenner MJ, Baio J, Washington A, Christensen DL, Wiggins LD, et al. Early identification of autism spectrum disorder among children aged 4 years—Early autism and developmental disabilities monitoring network, six sites, United States, 2016. MMWR Surveill Summ. 2020;69: 1–11. pmid:32214075
- 19. Maenner MJ, Shaw KA, Baio J, Washington A, Patrick M, DiRienzo M, et al. Prevalence of autism spectrum disorder among children aged 8 Years-Autism and developmental disabilities monitoring network, 11 Sites, United States, 2016. MMWR Surveill Summ. 2020;69: 1–12. pmid:32214087
- 20. Siu AL. Screening for autism spectrum disorder in young children US preventive services task force recommendation statement. JAMA—J Am Med Assoc. 2016;315: 691–696. pmid:26881372
- 21. Sheldrick RC, Merchant S, Perrin EC. Identification of developmental-behavioral problems in primary care: A systematic review. Pediatrics. 2011. pp. 356–363. pmid:21727101
- 22. Fein D. Commentary on USPSTF final statement on universal screening for autism. J Dev Behav Pediatr. 2016;37: 573–578. pmid:27490701
- 23. Coury DL. Babies, bathwater, and screening for autism spectrum disorder: Comments on the USPSTF recommendations for autism spectrum disorder screening. J Dev Behav Pediatr. 2015;36: 661–663. pmid:26421531
- 24. Mandell D, Mandy W. Should all young children be screened for autism spectrum disorder? Autism. 2015;19: 895–896. pmid:26734703
- 25. Wetherby AM, Allen L, Cleary J, Kublin K, Goldstein H. Validity and reliability of the communication and symbolic behavior scales developmental profile with very young children. J Speech, Lang Hear Res. 2002;45: 1202–1218. pmid:12546488
- 26. Wetherby AM, Brosnan-Maddox S, Peace V, Newton L. Validation of the Infant-Toddler Checklist as a broadband screener for autism spectrum disorders from 9 to 24 months of age. Autism. 2008;12: 487–511. pmid:18805944
- 27. Wetherby AM, Woods J, Allen L, Cleary J, Dickinson H, Lord C. Early indicators of autism spectrum disorders in the second year of life. J Autism Dev Disord. 2004;34: 473–493. pmid:15628603
- 28.
Sobieski M, Wrona S, Flakus M, Pierchała K, Sobieska A, Podgórska K, et al. Reliability and Theoretical Validity of the Polish Version of Communication and Symbolic Behavior Scales-Developmental Profile—Infant-Toddler Checklist. [cited 11 Feb 2024].
- 29. Sobieski M, Grata-Borkowska U, Bujnowska-Fedak MM. Implementing an Early Detection Program for Autism Spectrum Disorders in the Polish Primary Healthcare Setting—Possible Obstacles and Experiences from Online ASD Screening. Brain Sci. 2024;14: 388. pmid:38672037
- 30. Schmidt SAJ, Lo S, Hollestein LM. Research Techniques Made Simple: Sample Size Estimation and Power Calculation. Journal of Investigative Dermatology. Elsevier B.V.; 2018. pp. 1678–1682. pmid:30032783
- 31.
Jones M. Autism Spectrum Rating Scales (ASRS). Encyclopedia of Special Education. New York: Multi-Health Systems; 2014.
- 32.
Wrocławska-Warchala E, Wójcik R. ASRS Zestaw Kwestionariuszy do Diagnozy Spektrum Autyzmu. Warszawa: Pracownia Testów Psychologicznych Polskiego Towarzystwa Psychologicznego; 2016.
- 33. Chojnicka I, Pisula E. Cross-Cultural Validation of the Polish Version of the ADI-R, Including New Algorithms for Toddlers and Young Preschoolers. Child Psychiatry Hum Dev. 2019;50: 591–604. pmid:30661145
- 34. Chojnicka I, Pisula E. Adaptation and validation of the ADOS-2, Polish version. Front Psychol. 2017;8. pmid:29163293
- 35.
Pisula E. Profil Psychoedukacyjny, Wydanie 3, Wersja polska. Podręcznik diagnosty. Warszawa: EduProf; 2019.
- 36. Lourenço OM. Developmental stages, Piagetian stages in particular: A critical review. New Ideas Psychol. 2016;40: 123–137.
- 37. Swets JA. Measuring the accuracy of diagnostic systems. Sci Sci. 1988;240: 1285–1293. pmid:3287615
- 38. Florkowski CM. Sensitivity, specificity, receiver-operating characteristic (ROC) curves and likelihood ratios: communicating the performance of diagnostic tests. Clin Biochem Rev. 2008;29 Suppl 1: S83–7. Available: http://www.ncbi.nlm.nih.gov/pubmed/18852864%0A, http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC2556590 pmid:18852864
- 39. Kumar R, Indrayan A. Receiver operating characteristic (ROC) curve for medical researchers. Indian Pediatr. 2011;48: 277–287. pmid:21532099
- 40. Briggs-Gowan MJ, Carter AS, Irwin JR, Wachtel K, Cicchetti D V. The Brief Infant-Toddler Social and Emotional Assessment: Screening for Social-Emotional Problems and Delays in Competence. J Pediatr Psychol. 2004;29: 143–155. pmid:15096535
- 41. Turner-Brown LM, Baranek GT, Reznick JS, Watson LR, Crais ER. The first year inventory: A longitudinal follow-up of 12-month-old to 3-year-old children. Autism. 2013;17: 527–540. pmid:22781058
- 42. Kleinman JM, Robins DL, Ventola PE, Pandey J, Boorstein HC, Esser EL, et al. The modified checklist for autism in toddlers: A follow-up study investigating the early detection of autism spectrum disorders. J Autism Dev Disord. 2008;38: 827–839. pmid:17882539
- 43. Allison C, Matthews FE, Ruta L, Pasco G, Soufer R, Brayne C, et al. Quantitative checklist for autism in toddlers (Q-CHAT). A population screening study with follow-up: The case for multiple time-point screening for autism. BMJ Paediatr Open. 2021;5: e000700. pmid:34131593
- 44. Sacrey LAR, Bryson S, Zwaigenbaum L, Brian J, Smith IM, Roberts W, et al. The Autism Parent Screen for Infants: Predicting risk of autism spectrum disorder based on parent-reported behavior observed at 6–24 months of age. Autism. 2018;22: 322–334. pmid:29671640
- 45. Freed GL, Singer DC, Gebremariam A, Schultz SL, Clark SJ. Milestones: How parents understand child development. CS Mott Child Hosp Natl Poll Child Heal. 2021;38: 1–2. Available: https://mottpoll.org/reports/milestones-how-parents-understand-child-development.
- 46. Reijneveld SA, De Meer G, Wiefferink CH, Crone MR. Parents’ concerns about children are highly prevalent but often not confirmed by child doctors and nurses. BMC Public Health. 2008;8: 124. pmid:18423036
- 47. Eadie PA, Ukoumunne O, Skeat J, Prior MR, Bavin E, Bretherton L, et al. Assessing early communication behaviours: Structure and validity of the Communication and Symbolic Behaviour Scales-Developmental Profile (CSBS-DP) in 12-month-old infants. Int J Lang Commun Disord. 2010;45: 572–585. pmid:19886849
- 48. Robins DL, Casagrande K, Barton M, Chen CMA, Dumont-Mathieu T, Fein D. Validation of the modified checklist for autism in toddlers, revised with follow-up (M-CHAT-R/F). Pediatrics. 2014;133: 37–45. pmid:24366990
- 49. Dereu M, Warreyn P, Raymaekers R, Meirsschaut M, Pattyn G, Schietecatte I, et al. Screening for autism spectrum disorders in flemish day-care centres with the checklist for early signs of developmental disorders. J Autism Dev Disord. 2010;40: 1247–1258. pmid:20198413
- 50. Al Maskari T, Melville C, Al Farsi Y, Wahid R, Willis D. Qualitative exploration of the barriers to, and facilitators of, screening children for autism spectrum disorder in Oman. Early Child Dev Care. 2020;190: 1762–1777.
- 51. Buckley C. Making your practice autism friendly. InnovAiT Educ Inspir Gen Pract. 2017;10: 327–331.
- 52. McCormack G, Dillon AC, Healy O, Walsh C, Lydon S. Primary Care Physicians’ Knowledge of Autism and Evidence-Based Interventions for Autism: A Systematic Review. Review Journal of Autism and Developmental Disorders. 2020. pp. 226–241.
- 53. Bleses D, Højen A, Justice LM, Dale PS, Dybdal L, Piasta SB, et al. The Effectiveness of a Large-Scale Language and Preliteracy Intervention: The SPELL Randomized Controlled Trial in Denmark. Child Dev. 2018;89: e342–e363. pmid:28598553
- 54. Bleses D, Dale PS, Justice L, Højen A, Vind BD, Jiang H. Sustained effects of an early childhood language and literacy intervention through second grade: Longitudinal findings of the SPELL trial in Denmark. PLoS One. 2021;16. pmid:34634097
- 55. Niedźwiecka A, Pisula E. Symptoms of Autism Spectrum Disorders Measured by the Qualitative Checklist for Autism in Toddlers in a Large Sample of Polish Toddlers. Int J Environ Res Public Health. 2022;19: 3072. pmid:35270764
- 56. Reder M, Brzezewska K. Developmental and family environment predictors of diagnostic decision-making in children with autism spectrum disorder and comorbid disorders. Heal Psychol Rep. 2022;10: 280–293. pmid:38084137
- 57. Lin CS, Chang SH, Cheng SF, Chao PC, Chiu CH. The preliminary analysis of the reliability and validity of the Chinese Edition of the CSBS DP. Res Dev Disabil. 2015;38: 309–318. pmid:25577181
- 58. Devescovi R, Monasta L, Bin M, Bresciani G, Mancini A, Carrozzi M, et al. A two-stage screening approach with I-TC and Q-CHAT to identify toddlers at risk for autism spectrum disorder within the italian public health system. Brain Sci. 2020;10: 184. pmid:32210022
- 59. Fischer VJ, Morris J, Martines J. Developmental screening tools: Feasibility of use at primary healthcare level in low- and middle-income settings. J Heal Popul Nutr. 2014;32: 314–326. pmid:25076668
- 60. Dawson G, Rogers S, Munson J, Smith M, Winter J, Greenson J, et al. Randomized, controlled trial of an intervention for toddlers with autism: the Early Start Denver Model. Pediatrics. 2010;125. pmid:19948568
- 61. Biele G, Gustavson K, Czajkowski NO, Nilsen RM, Reichborn-Kjennerud T, Magnus PM, et al. Bias from self selection and loss to follow-up in prospective cohort studies. Eur J Epidemiol. 2019;34: 927–938. pmid:31451995
- 62. Bacon EC, Courchesne E, Barnes CC, Cha D, Pence S, Schreibman L, et al. Rethinking the idea of late autism spectrum disorder onset. Dev Psychopathol. 2018;30: 553–569. pmid:28803559