Validation of a 4-item child perception questionnaire in Australian children

Objective To develop and validate a 4-item child oral health-related quality of life (OHRQoL) instrument that might be more amenable for uptake in large scale, multifaceted surveys of children’s health and wellbeing than current, longer-form child OHRQoL instruments. Methods Data were obtained from a study of the South Australian School Dental Service population designed to investigate OHRQoL among school children aged 8–13 years in 2002–2003. The Child Perception Questionnaire (CPQ8-10 and CPQ11-14) was utilised, which comprises 25 & 37 items representing four conceptual domains: oral symptoms, functional limitations, emotional wellbeing and social wellbeing. Initially, the psychometric properties of the short form 8-item CPQ were tested in both age groups using Confirmatory Factor Analysis. The rationale was that, if the 8-item CPQ8-10 and CPQ11-14 did not display good psychometric properties, there was no reason to proceed with further shortening into 4-item versions. Following a good fit of the 8-item CPQ, items with higher factor loadings in each domain were maintained and tested in the development of a 4-item CPQ. Exploratory Factor Analysis was conducted to determine dimensionality, followed by tests for reliability and validity. Model fits were assessed using Root Mean Square Error of Approximation (RMSEA), Comparative Fit Index (CFI) and Standardized Root Mean Square Residual (SRMR). Results There were 308 children aged 8–10 years who completed CPQ8-10 and 461 children aged 11–13 years who completed CPQ11-14. For the short-form 8-item instrument, satisfactory goodness of fit was demonstrated for the two age groups, with acceptable thresholds for RMSEA, CFI, and SRMR. The four items with the highest factor loading in each domain were the same for the 8-item CPQ8-10 and CPQ11-14. and these items were selected to comprise the 4-item CPQ8-10 and CPQ11-14. The 4-item short form displayed good criterion validity, with expected score patterns found in the majority of the known groups evaluated. Conclusions We developed short-form 4-item CPQ8-10 and CPQ11-14 instruments that were tested in a large convenience sample of South Australian school children. The instruments demonstrated acceptable reliability and validity. Implications for practice are discussed.

In summary, despite successful development of the 8-and 16-item CPQ [11][12][13][14] [16] and the 25-item CPQ [8][9][10] [12], there has been no uptake of these short forms in many governmentfunded surveys of school children in part because, for logistical, pragmatic and financial reasons, the instrument is still considered to be too long. Because a shorter form could have many potential applications, for example, inclusion in government surveys in Australia such as the Longitudinal Study of Australian Children and Longitudinal Surveys of Australian Youth, our aims were to develop and test the validity of 4-item CPQ 8-10 and CPQ [11][12][13][14] instruments. Our hypothesess were that: (1) 4-item versions of the CPQ 8-10 and CPQ [11][12][13][14] would have adequate psychometric properties and; (2) the 4-items measure a single overall dimension of children's OHRQoL.

Sample
Data were obtained from a study of the South Australian School Dental Service population designed to investigate OHRQoL among school children aged 8-13 years in 2002-2003 [13].
Children and their parents were approached with a package containing an information letter, a consent form and questionnaires. Parents provided signed informed consent. A total of 1,401 children were sampled, with 842 parent/child pairs responding with completed questionnaires. Data for this analysis comprises 308 children aged 8-10 years who completed CPQ [8][9][10] and 461 children aged 11-13 years who completed CPQ [11][12][13][14] . Ethical approval was granted by the Human Research Ethics Committee of the University of Adelaide.

The child perception questionnaire (CPQ)
The CPQ is designed to measure children's OHRQoL. The short form CPQ 8-10 from the 25-item CPQ 8-10 and 37-item CPQ [11][12][13][14] [13] comprises 8 items and represents 4 conceptual domains: oral symptoms (OS 1 : 'Pain' and OS 2 : 'Food stuck or caught'), functional limitation (FL 1 : 'Difficulty biting or chewing firm food' and FL 2 : 'Taken longer than others to eat a meal'), emotional wellbeing (EW 1 : 'Been upset because of problems with your teeth, lips, mouth or jaws' and EW 2 : 'Been irritable or frustrated because of problems with your teeth, lips, mouth or jaws') and social wellbeing (SW 1 : 'Missed school because of problems with your teeth, lips, mouth or jaws' and SW 2 : 'Not wanted to talk to other children because of problems with your teeth, lips, mouth or jaws') ( Table 1). Each item is ranked on 5-point Likert scale ranging from 1 to 5 (1 = Never, 2 = Once or twice, 3 = Sometimes, 4 = Often, and 5 = Very often). Scores were re-coded: 1 to 0, 2 to 1, 3 to 2, 4 to 3, and 5 to 4. Summary scores ranged from 0-16, after re-coding, with higher total scores indicating worse child OHQoL.

Criterion validity
Covariates used in the analysis of criterion validity included sociodemographic, dental healthrelated, self-rated oral health, and dental disease-related characteristics. Self-rated oral health was measured with the question "How would you rate your dental health?", with response options (Very good/Good/OK/Poor) for children aged 8 to 10 and response options (Excellent/Very good/Good/Fair/Poor) for children aged 11 to 13. The sociodemographic characteristics included children's sex (boy vs girl), parent answering the survey's highest education level (High school or less, Trade to Diploma or Tertiary) and; annual household income (<AUS$40K, AUS $40-60K or >AUS$60K: '1AUS$ = 0.7US$'). The dental health-related behaviours included: frequency of tooth brushing (�2/day vs <2/day), consumption of soft drinks (twice a day or more vs once a day or less), and consumption of sweet drinks (twice a day or more vs once a day or less). The response options for child's self-rated oral health included 'excellent/very good' vs

Statistical analysis
Data was stratified into two age groups: children aged 8-10 years and children aged 11-13 years. To evaluate the validity of 4-item versions of the CPQ 8-10 and CPQ [11][12][13][14] we conducted two steps. The first step was to investigate the validity of the 8-item CPQ 8-10 and CPQ [11][12][13][14] using Confirmatory Factor Analysis (CFA). In both instruments, the factorial structure evaluated was the 4-factor model (i.e. oral symptoms, functional limitation, emotional wellbeing and social wellbeing) with 2 items per factor ( Table 1). Although the 8-item version displayed good psychometric properties in other contexts, such as New Zealand [14], it is still necessary to ensure that it has good psychometric properties for children in the current study. The rationale is that, if the 8-item CPQ 8-10 and CPQ [11][12][13][14] do not display good psychometric properties for the children in question, there is no reason to proceed with further shortening into 4-item versions. The CFA models were estimated with maximum likelihood [19]. Model fit was assessed using the chi-square test statistic (χ 2 ), Comparative Fit Index (CFI), Root Mean Square Error of Approximation (RMSEA) and Standardized Root Mean Square Residual (SRMR). Model fit was considered acceptable when CFI>0.96, RMSEA<0.8 and SRMR <0.08 [20][21][22][23]. When acceptable fit of the 8-item version was established, the four items with the highest factor loading on each domain was selected to comprise the 4-item CPQ 8-10 and 4-item CPQ [11][12][13][14] . The factor loading indicates how much item responses were influenced by the underlying construct they intend to measure (rather than influenced by measurement error) [24]. Hence, the CPQ 8-10 and CPQ [11][12][13][14] items with highest factor loadings were the ones most influenced by (and, consequently, which better measured) their respective OHRQoL domains (i.e. oral symptoms, functional limitation, emotional wellbeing and social wellbeing).
After choosing the 4 items with the highest factor loading, the second step was to test the dimensionality of the 4-item CPQ 8-10 and CPQ 11-14 instruments using Exploratory Factor Analysis (EFA). Considering that 4-item CPQ 8-10 and CPQ [11][12][13][14] versions have never been evaluated, we did not have any prior information on possible factorial structures. In this case, when factorial structures are unknown, EFA should be preferred over CFA [25]. The quality of the factor analysis models was assessed using the Kaiser-Meyer-Olkin (KMO) test and Bartlett s test for sphericity. The KMO test measures the degree of multi-collinearity (based on partial correlations) between the included items, varies between 0 and 1 and should be greater than 0.5-0.6. Bartlett's test is a measure of the probability that the initial correlation matrix is an identity matrix and should be under 0.05. Factor retention was evaluated with Scree Plots, and remained satisfactory if their primary factor loading was more than 0.40 [26]. Finally, Cronbach's α and corrected item total correlations (CITCs) were used to assess the internal consistency of the instruments. A Cronbach's α coefficient of 0.70 or higher is considered satisfactory [27,28]. A CITC value lower than 0.30 indicates that an item displays poor internal consistency with the other items and should be excluded. Criterion validity of the 4-item CPQ 8-10 and CPQ [11][12][13][14] instruments was tested by examining known-groups comparisons [29]. That is, the extent to which groups expected to display worse OHRQoL (e.g. less educated, higher dmft, worse self-rated oral health) had higher scores (indicates worse OHRQoL) in the 4-item CPQ 8-10 and CPQ [11][12][13][14] . The 4-item CPQ 8-10 and CPQ [11][12][13][14] instruments were thus stratified by socio-demographic, dental behaviours, self-rated oral health and dental disease-related characteristics. All data were analysed using SAS statistical software (SAS 9.4, SAS Institute Inc., Cary, NC, USA).

Results
The CFA results of the 8-item CPQ 8-10 and CPQ [11][12][13][14] are presented in Table 2. Satisfactory goodness of fit was demonstrated with acceptable thresholds for RMSEA, CFI and SRMR. In addition, the four items with highest factor loading were the same for the 8-item CPQ 8-10 and the CPQ 11-14 (OS 1 , FL 1 , EW 1 and SW 2 ) (see Table 1: items in bold pertain). We selected these 4 items and proceeded to evaluate the validity and reliability of the 4-item CPQ 8-10 and CPQ [11][12][13][14]. Table 3 shows the EFA results of the 4-item CPQ 8-10 and CPQ [11][12][13][14] . The KMO values were over 0.6 and Bartlett's tests were under 0.05 in both the 4-item CPQ 8-10 and CPQ [11][12][13][14] instruments. This supports the factorability of correlation matrices and demonstrates the suitability of the data for the factor analysis. The EFA results indicate that the eigenvalues of the first factors were substantially higher than the eigenvalues of the three other factors. These results can be seen in the Scree Plots (Fig 1).
Furthermore, the eigenvalues of the second factor were smaller than 1 for both the CPQ 8-10 (Λ = 0.99) and CPQ [11][12][13][14] (Λ = 0.99). In the absence of measurement error, the second, third and fourth factor would explain less variance of the item responses than a single item, which is unacceptable. Hence, the evaluation of the Scree plots indicated that one factor should be retained. That is, the 4-item CPQ 8-10 and CPQ [11][12][13] are unidimensional instruments since they measure a single overall OHRQoL factor. Furthermore, factor loading values were nearly or higher than 0.7 for all items, with the exception of item SW 2 (λ = 0.41), indicating that the factor substantially explained the variance from item responses. Furthermore, factor loading values were higher than 0.40 for all items, indicating that the factor substantially explained the variance from item responses.
Cronbach's standardized alpha was nearly 0.7 in the 11-13 years group (0.68), which appeared to have good internal consistency; but less than 0.7 in the younger group (0.56). The

PLOS ONE
total item correlation values were more than 0.3, except for SW 2 (0.2) in the younger age group, which indicated that each item correlates well with the scale overall. In addition, the correlation between each item (representing each domain) of the 4-item CPQ, and each domain and total score of the 8-and 25-item CPQ 8-10 and 31-item CPQ [11][12][13][14] (see Table 4) indicated medium to strong correlations (ranging from 0.55 to 0.86) between each domain of the 4-item, and the 8-/25-/31-item CPQ. The exception was SW 2 of the 4-item with the total 25/31-item CPQ (0.37 and 0.44, respectively). After establishment of the factorial structure, the next step was evaluation of the 4-item CPQ 8-10 and CPQ 11-14 criterion validity. Tables 5 and 6 demonstrate the instrument scores stratified by related covariates. For those aged 8-10 years, higher scores were observed for either the total 4-item instrument and/or individual CPQ items among those with 'Ok/Poor' self-ratings of oral health, experience of dental caries, consumption of sweet drinks twice a day or more. For those aged 11-13 years, higher scores were observed for either the total 4-item CPQ instrument and/or individual CPQ items among those who rated their oral health as fair or poor, had experience of dental caries, consumed sweet drinks twice a day or more, had an annual household income of $40-60,000 Australian. In addition, 8-item CPQ 8-10 and CPQ [11][12][13][14] criterion validity was evaluated, and the same results were observed in most domains and total CPQ (see Tables 7 and 8).

PLOS ONE
unidimensional, indicating that both short forms measure a unique overall dimension of children's OHRQoL. The development of the 37-item CPQ [11][12][13][14] constituted the first questionnaire to evaluate OHRQoL among children. The importance of a tailored instrument was that children's cognitions about health, such as their perceptions regarding OHRQoL, are age-dependent, resulting from children's stage of cognitive and language development [11]. Despite its originality and excellent psychometric properties, the response burden associated with answering 37 items, especially for children aged 11 to 14, led researchers to advocate for shorter versions [30]. Over the last two decades, the 16-item and 8-item CPQ [11][12][13][14] [16] and 25-item CPQ 8-10 [12] were developed and their good psychometric properties were replicated by independent studies [31,32].

PLOS ONE
However, the 8-and 16-item CPQ forms have not been included in national and state-wide surveys applied in schools or longitudinal studies because child OHRQoL needs to be evaluated with a multitude of other outcomes, including general health, mental health and educational attainment. Managers of large surveys, as reported previously, have concerns regarding response burden. A meta-analysis conducted by Rolstad et al. [15] demonstrated the effect of questionnaire length on response burden, indicating a greater chance of response when patients were presented with a comparatively shorter questionnaire. Thus, reducing response burden is the rationale driving researchers to investigate the development of questionnaires with a minimum number of items [15].

PLOS ONE
Addition of child OHRQoL data would not only contribute to the overall richness and comprehensiveness of large, multidisciplinary surveys of child health, development and wellbeing, but provide tangible evidence for health policy translation in terms of the importance of oral health in school productivity, social engagement and quality of life across the life course. The data would yield important information for fair and equitable child dental service provision, and provide a valuable monitoring and surveillance tool in the policy, health and education sectors. There are three main ways, in a research sense, of how capturing oral health-related items in large, population-based surveys would give more power to answer key oral health research questions: (1) samples would be large and representative; (2) collection over time would enable trajectories to be calculated and downstream consequences (for example,

PLOS ONE
experience of dental disease in adulthood) more accurately predicted and; (3) it would be possible to stratify by geographic location or other subgroups of children (for example, socially disadvantaged), which is critical when oral health resources need to be targeted because of resource constraints. In our study, we evaluated the development of a CPQ version with a minimum number of items, comprising only one item for each of the 4 OHRQoL domains. Our findings showed that the 4-item CPQ had equivalent versions (that is, they should include the same 4 items) for children aged 8 to 10 and aged 11 to 14. These 4 items were chosen based on the highest factor loading on the overall OHRQoL factor, indicating that these items better discriminated the children according to their levels of OHRQoL. For example, item FL 1 ("Difficulty biting or chewing firm foods") was chosen over item FL 2 ("Taken longer than others to eat a meal") to measure "Functional limitation". It seems reasonable that item FL 1 measured "Functional limitation" more accurately since dental diseases, such as tooth decay, often lead to difficulties in chewing food [33]. On the other hand, when a child "Takes longer than others to eat a meal", this can happen due to other reasons than dental problems, such as time spent watching television or using smartphones [34,35]. The findings showed that the 4-item CPQ 8-10 and CPQ [11][12][13][14] are unidimensional measures, meaning the total score should be computed summing all 4 items. That is, the 4 items measure distinct domains of OHRQoL (Oral Symptoms, Functional Limitation, Emotional Wellbeing and Social Wellbeing) but these domains are part of an overall broader construct of OHRQoL. Despite the differences between domains, for example between a physiological domain such as Functional Limitation ("Difficulty biting or chewing firm foods") and a psychological domain such as Emotional wellbeing ("Been upset because of your teeth or mouth"), these 4 domains were all related to broader low quality of life due to oral health problems.
There is one point regarding the 4-item CPQ 8-10 and CPQ [11][12][13][14] which warrant further investigation. The reliability was below standards for group comparison (>.70) [36] in the children aged 8 to 10 group; while, for children aged 11 to 14, reliability had a bordering but still acceptable value. Reliability is a function of questionnaire length and trait variance. Hence, in the event that the 4-item CPQ 8-10 and CPQ [11][12][13][14] is included in government-funded surveys, these surveys will contain tens of thousands of children, with sample sizes more than ten times bigger than our study sample. With larger sample sizes, the higher sample heterogeneity regarding OHRQoL (i.e. higher trait variance) will possibly improve reliability despite the shortened questionnaire. This possibility, however, needs to be investigated and is a topic for future research. For now, due to the decreased reliability, the application and interpretation of the 4-item CPQ 8-10 results need to be conducted with caution. It is well-know, for example, that decreased reliability can lead to effect attenuation [37]. For example, diminished reliability can decrease the estimate of the OHRQoL effect (as measured by the 4-item CPQ 8-10 total score) on a chosen outcome (or vice-versa). While this study presented initial good results regarding the 4-item CPQ 8-10 psychometric properties, future independent studies need to further investigate the instrument's reliability.
Finally, the 4-item CPQ 8-10 and CPQ 11-14 displayed good criterion validity since expected score patterns were found in the majority of groups evaluated. For instance, higher mean scores (indicating worse OHRQoL) were found in children with worse self-rated oral health and higher soft drink consumption among others. Although the observed trend was expected -the higher score was in the lower income groups among children aged 8-10 years-there was no strong evidence demonstrated. One reason could be due to the many missing values (more than 12%).

Conclusion
This study was the first to propose and evaluate the psychometric properties of the 4-item CPQ 8-10 and CPQ [11][12][13][14] questionnaire. The upsurge of large governmental surveys that evaluate multiple health outcomes requires short questionnaires. We showed initial evidence that the 4-item CPQ 8-10 and CPQ [11][12][13][14] questionnaires are unidimensional instruments that can measure OHRQoL in children aged 8 to 10 and aged 11 to 14. The strengths of the study include: (1) it is not only the shortest form of CPQ, but also the only age-appropriate CPQ tool which can be used to assess school children's OHRQoL and; (2) the development and testing was conducted among a large population of school children, which is especially suitable for longitudinal cohort follow-up. We acknowledge that the 4 item CPQ [11][12][13][14] was better able to capture the oral health of children aged 11-14 years compared to the 4-item CPQ 8-11 that assesses children aged 8-11 years. We recommend further validation in child groups from other parts of the world.