https://orcid.org/0000-0001-5896-2753
Figures
Abstract
This study aimed to assess the validity and reliability of a questionnaire on patient acceptance of orthodontic retainers. The original questionnaire was forward- and backward-translated, followed by four validity tests (content validity, face validity, construct validity, criterion validity) and two reliability tests (test-retest reliability, internal consistency). Content validity was assessed by nine orthodontists who appraised the questionnaire’s representativeness, relevance, clarity, and necessity. Face validity was established through semi-structured in-depth interviews with 35 English-literate participants currently wearing orthodontic retainers. Construct validity was established through Exploratory Factor Analysis (EFA). For criterion validity, 107 participants concurrently answered the questionnaire and the Retainer-modified Malaysian Oral Health Impact Profile questionnaire. Test-retest reliability was verified by 34 subjects who responded to the questionnaire again after a two-week interval. Six revised items passed the threshold value of 0.78 for Item-Content Validity Index and Content Validity Ratio and were revised based on findings from the face validity test. Principal Component Analysis of EFA extracted information on only one component, and all items were positively correlated with the component matrix. Spearman’s rho value (rs = 0.490 and rs = 0.416) indicated a moderate correlation between the two questionnaires for criterion validity. Intraclass Correlation Coefficient ranged from 0.687 to 0.913, indicating moderate to excellent test-retest reliability. Cronbach’s alpha ranged from 0.687 to 0.913 indicating that none of the questionnaire items showed unacceptable or poor internal consistency. The questionnaire on patient acceptance of orthodontic retainers has been validated and can be used in both clinical and research settings.
Citation: Saw ZK, Yuen JJX, Ashari A, Ibrahim Bahemia F, Low YX, Nik Mustapha NM, et al. (2025) Forward-backward translation, content validity, face validity, construct validity, criterion validity, test-retest reliability, and internal consistency of a questionnaire on patient acceptance of orthodontic retainer. PLoS ONE 20(1): e0314853. https://doi.org/10.1371/journal.pone.0314853
Editor: Francisco Wilker Mustafa Gomes Muniz, Universidade Federal de Pelotas, BRAZIL
Received: June 14, 2024; Accepted: November 18, 2024; Published: January 3, 2025
Copyright: © 2025 Saw et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Most relevant data are included in the manuscript and its Supporting Information files. Data that cannot be shared publicly are withheld due to ethical considerations. However, data are available from the Department of Paediatric Dentistry & Orthodontics, Faculty of Dentistry, Universiti Malaya (contact Madam NOR SURIYANI BINTI SHAFEI at +603 7967 4802 or norsuriyanishafei@um.edu.my) for researchers upon request, after permission has been granted in accordance with the advice of the respective ethics committee.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Retention is the phase of orthodontic treatment in which teeth are maintained in their corrected positions after active tooth movement [1]. During this stage, retainers are prescribed to minimize relapse, which is the tendency of teeth to return to their pretreatment positions [2–4].
Retainers can be fixed, removable, or a combination of both and the retainer choice depends on factors include pretreatment occlusion, treatment results and oral hygiene [4]. It was found that orthodontists favor removable retainers over their fixed counterparts [5]. The two most common removable retainers prescribed are the Hawley retainer and the vacuum-formed retainer (VFR) [5].
In most cases, patients are instructed to wear removable retainers at night indefinitely [6]. However, achieving consistent long-term compliance remains challenging [7]. Discomfort caused by retainer wear has a negatively impact on patient’s acceptance and compliance [8]. A comfortable retainer that is well-accepted by patients for long-term use is key to improve retention outcomes and satisfaction [9].
To objectively assess patient acceptance on orthodontic retainers, a valid and reliable instrument is needed. Saleh et al. [10] modified a questionnaire originally developed by Ngan et al. [11] by reducing the questionnaire items from 14 to 10. In their questionnaire, Saleh et al. used visual analogue scale (VAS) to assess patient acceptance of orthodontic retainer in teeth biting, fitting of the appliance, speech, appearance, oral hygiene, durability, gingival irritation, swallowing, self-confidence, and comfort [10]. However, this questionnaire lacks validity and reliability testing. Similarly, Mohd Tahir et al. developed the Retainer-modified Oral Health Impact Profile for Malaysian adult (Retainer-modified S-OHIP(M)), assessing functional limitation, functional pain, physiological discomfort, physical disability, psychological disability, social disability, and handicap [12]. This questionnaire has only undergone face validity testing.
While both instruments aim to measure patient acceptance of orthodontic retainers, they have limitations in comprehensive validity and reliability testing, highlighting the need for a more robust tool. The present study aims to validate the questionnaire on patient acceptance of orthodontic retainers and assess its reliability.
Methods
Subjects and methods
This cross-sectional study was approved by the ethics committee of the Faculty of Dentistry Medical Ethics Committee (FDMEC), Universiti Malaya (DF CD2106/0016 (U)). Subject recruitment was conducted between June 1, 2021, and December 1, 2021. Four validity tests (content validity, face validity, construct validity, and criterion validity) and two reliability tests (test-retest reliability and internal consistency) were conducted. The STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines were followed to ensure rigorous and transparent reporting of this study [13].
Setting and participants
The inclusion criteria include those who were able to read and understand English, were currently wearing an orthodontic retainer, and were medically fit and healthy. Exclusion criteria include the inability to read or understand English, not currently wearing an orthodontic retainer, and the presence of medical conditions. Subjects were conveniently sampled from patients at the Faculty of Dentistry, Universiti Malaya. Written consent was obtained prior to the study via Google Form, including written consent from parents or guardians for minors.
The sample sizes of participants for each validity and reliability test are presented in Table 1. The sample size for each test was determined independently, as each test has a different recommended sample size.
Study instrument
A questionnaire assessing patient acceptance of orthodontic retainers, initially developed by Ngan et al. [11] and later modified by Saleh et al. [10], was utilized in this study. Subjects were asked to indicate their responses to 10 questionnaire items on a 100 mm visual analog scale (VAS), with the scale anchored by extremes: “very comfortable” at the left end, “very uncomfortable” at the right end and a neutral midpoint labeled “mildly comfortable”. The questionnaire items addressed ten aspects of retainer wear, including “teeth biting”, “appliance fit”, “speech”, “appearance”, “oral hygiene”, “durability”, “gingival irritation”, “swallowing”, “self-confidence”, and “comfort”. Their responses were then quantified using a ruler by measuring from left to right (in millimeters).
Forward-backward translation
Forward translation from English to Malay was performed independently by two translators. The two sets of translations were critically appraised by an expert panel consisting of three orthodontists and one expert in questionnaire translation and validation from the dental public health specialty. Amendments were made accordingly.
Backward translation into English was done independently by two other translators, who are native English speakers, were fluent in Malay, and were blinded to the original version. The two sets of translations were also critically appraised by an expert panel.
The final back-translated questionnaire was used for the subsequent content validity test.
Content validity
Content validity was assessed by nine orthodontists from Universiti Malaya (UM), Universiti Kebangsaan Malaysia (UKM), and Universiti Teknologi Mara (UiTM) with at least ten years of experience. This panel size adheres the recommendation to have an odd number of experts with at least three members [14]. The questionnaire was evaluated in terms of representativeness, relevance, clarity, and necessity [14].
Representativeness, relevance, and clarity were assessed using a 4-point ordinal scale (Table 1) and analyzed using Item-Content Validity Index (I-CVI). The I-CVI was calculated as the number of experts giving a rating of 3 or 4 divided by the number of experts. An I-CVI ≥ 0.78 was considered acceptable [15]. Necessity was assessed using a 3-point ordinal scale (Table 2) and analyzed using Content Validity Ratio (CVR). CVR was calculated using the formula “CVR = (Ne—N/2)/(N/2)”, where N is the total number of experts and Ne is the number of experts indicating that the item is essential (with a rating of 3). The minimum acceptable CVR for nine panelists was 0.78 [14]. The experts were also asked to comment on the grammar and order of the wording.
The data were critically reviewed by an expert panel of three orthodontists with at least three years of experience from UM, UKM, and UiTM. Based on the results, two questionnaire items were rephrased, and the rephrased items were appraised by the nine orthodontists. The revised questionnaire was used for subsequent face validity tests.
Face validity
Pilot testing and interviewer training were conducted prior to the actual face validity testing. Sample recruitment continued until data saturation was achieved or a minimum sample size of 30 subjects was reached, based on the guidelines provided by Yusoff [16]. Subjects were asked to answer the questionnaire via Google Form, and then took part in a one-on-one semi structured in-depth interview on their comprehension and interpretation of each questionnaire item. These interviews were conducted via Google Meet and were recorded. The recorded audio was transcribed and thematically analyzed [17]. The expert panel then critically reviewed the findings, emphasizing difficulties encountered and suggestions from subjects. Based on this feedback, a final version of the questionnaire (Fig 1) was developed and later subjected to validity and reliability testing.
The scale is 10 cm in length. The extent of patient discomfort is measured from the left end of the scale to the vertical line placed by the patient.
Construct validity
A total of 105 participants were recruited for construct validity, meeting the minimum sample size of 100 recommended by Anthoine et al [18]. The subjects answered the questionnaire via Google Forms.
The suitability of the data for factor analysis was tested using the Kaiser–Meyer–Olkin (KMO) test, with KMO value of more than 0.50 indicating suitability for factor analysis [19]. On the other hand, Bartlett’s Test of Sphericity assessed whether the original matrix was an identity matrix, a value of less than 0.05 indicates the significance of data and was therefore acceptable for factor analysis.
Principal Components Analysis (PCA) was used to extract the maximum variance from the dataset for each component. Total Variance Explained shows how the data are spread (variance) among the six questionnaire items. According to the Kaiser criterion, components with eigenvalues greater than 1 should be retained—as it accounts for more variance than a single item, justifying the combination of those items into a factor [20]. However, this holds true only if each item contributes one unit of variance. Pett et al. advised that the Kaiser Criterion should only be used in PCA when the total variance is accounted for during extraction [21].
The Component Matrix determines the correlation (also known as factor loadings) between the component and the variables. The values can range from -1 to 1, with values near -1 or 1 indicating strong correlations, and values near 0 indicating weak correlations. A cutoff of 0.5 was applied as such loadings were considered significant [22].
Criterion validity
A total of 107 subjects were recruited for criterion validity. Based on power calculations, a sample size of 92 was required to detect a medium-sized correlation (effect size = 0.3) between the two questionnaires, with 90% power [23, 24]. Additionally, it met the minimum requirement of 30 as recommended by Roscoe [25]. Subjects were asked to answer the current questionnaire and the Retainer-modified S-OHIP(M) concurrently [26].
The Retainer-modified S-OHIP (M) questionnaire uses the following scale and scoring system: ‘Very often’, ‘Quite often’, ‘Sometimes’, ‘Seldom’, and ‘Never’, which are assigned scores of 4, 3, 2, 1, and 0, respectively. Data with more than three “Don’t Know” responses were excluded, while for data with one or two “Don’t Know” responses, the mean score for that item was imputed [26]. The Retainer-modified S-OHIP(M) questionnaire uses two scoring methods, the additive score (ADD) and simple count (SC). ADD was counted as the sum of the scores of all 14 items, whereas SC was the count of the number of questionnaire items rated as ‘Very often’ and ‘Quite often’. The ranges of the scores for the ADD and SC were 0 to 56 and 0 to 14, respectively [25]. The data were analyzed for normality followed by Spearman Correlation Coefficient analysis.
Reliability
The questionnaire was self-administered by 34 subjects twice, two weeks apart, for test-retest reliability and internal consistency tests. According to Bujang and Baharum, a minimum sample size of 30 is required to detect a value of 0.50 for the Intraclass Correlation Coefficient (ICC), with an alpha of 0.05 and a power of 90% [27]. The sample size was inflated to 34 to include a 10% drop-out rate. A two-week interval was used to minimize “carry-over” or memory effects and potential changes in retainers’ or patients’ conditions [28]. Test-retest reliability was assessed by calculating the ICC from the repeated administration of the questionnaire with values less than 0.5 indicating poor reliability, 0.5 to less than 0.75 indicating moderate reliability, 0.75 to less than 0.9 indicating good reliability, and 0.90 to 1.00 indicating excellent reliability [29]. Internal consistency was measured using Cronbach’s Alpha and interpreted as excellent (> 0.9), good (> 0.8), acceptable (> 0.7), questionable (> 0.6), poor (> 0.5), and unacceptable (< 0.5) [30].
Statistical analysis
Statistical analyses were performed using the IBM Statistical Package for Social Sciences (SPSS) statistical analysis software Version 26.0 for Windows (IBM Corporation, NY). The collected data were treated as numerical data. The Shapiro‒Wilk test was used to assess the normality of the data distribution. Means and standard deviations were calculated for parametric data, while medians and 1st and 3rd quartiles were calculated for nonparametric data. The I-CVI and CVR were calculated for content validity. For criterion validity, a nonparametric Spearman Correlation Coefficient was calculated, as the ADD and SC scores of the Retainer-modified S-OHIP(M) were not normally distributed and had highly positively skewed data distributions. For reliability, a two-way mixed effects model was used to generate an ICC with 95% CI for continuous variables to assess test-retest reliability, and Cronbach’s Alpha was used to measure internal consistency.
Results
Content validity
Table 3 shows the content validity results. Based on expert feedback, the terms “tooth biting”, “gingival irritation”, “swallowing”, and “self-confidence” were discarded because they failed to meet the minimum acceptable CVR value of 0.78 for necessity [14].
The experts expressed concerns that they were uncertain of “oral hygiene” and “durability” constructs. These items were revised to “cleanliness of mouth” and “robustness of retainer”, respectively. As shown in Table 4, the revised items scored above the minimum acceptable CVR values for necessity when they were re-evaluated for content validity.
Face validity
A total of 35 respondents, with a mean age of 23.03 years and a standard deviation of 3.74 years, were interviewed for face validity. Table 5 shows the subjects’ demographic data, and S1 Table shows the thematic analysis data and verbatim consent for face validity.
The two items that caused confusion were “mouth cleanliness” and “robustness of retainer”. The expert panel decided to rephrase the items to “feeling of cleanliness of mouth” and “strength and durability of retainer”, respectively.
Construct validity
Exploratory Factor Analysis was applied to determine the factor structure among the 6 items related to patient acceptance of orthodontic retainers (Table 6). The KMO measure of sampling adequacy was 0.765, above the suggested value of 0.5, and Bartlett’s test of sphericity was significant, which is < 0.001 (with Χ2 = 177.545, Df = 15). The data were deemed suitable for Principal Component Analysis for construct validity. In the present study, all initial communalities were above the threshold of 0.3 [31], and all loading factors were above 0.5. The EFA results for all 6 items revealed only one component with eigenvalues greater than 1. The results showed that the component explained 49.76% of the variance.
Criterion validity
The Shapiro‒Wilk test showed that the p value of the mean VAS score for the current questionnaire was more than 0.05 (p = 0.087), indicating normally distributed data. However, the p values of the ADD and SC score for the Retainer-modified S-OHIP(M) were less than 0.05 (p = 0.005 and p < 0.001, respectively), indicating that the data were not normally distributed. The Spearman Correlation Coefficient test was selected for criterion validity testing.
The mean and standard deviation of the average VAS score for the current questionnaire were 3.56 and 1.51, respectively. The median and interquartile range for the ADD of Retainer-modified S-OHIP(M) were 12.00 (minimum 0; maximum 33) and 10.00, respectively. The median and interquartile range for SC of Retainer-modified S-OHIP(M) were 1.00 (minimum 0; maximum 7) and 2.00, respectively.
The Spearman Correlation Coefficient test showed a significant correlation between the average VAS score and the ADD (rs = 0.490, n = 105, p < 0.01). The correlation between the average VAS score and SC score was also significant (rs = 0.416, n = 105, p <0.01). Both Spearman’s rho values (rs = 0.490 and rs = 0.416) indicated a moderate correlation between the two questionnaires.
Reliability
A total of 34 subjects were recruited for reliability tests, with a mean age of 23.03 years and standard deviation of 3.80 years. All the questionnaire items exhibited moderate to excellent test-retest reliability, with ICC values ranging from 0.687 to 0.913. Cronbach’s alpha also ranged from 0.687 to 0.913, indicating that none of the questionnaire items showed unacceptable or poor internal consistency (Table 7).
Discussion
The present study validates the Patient Acceptance of Orthodontic Retainer Questionnaire, a self-report instrument that evaluates patients’ perception of their orthodontic retainer while wearing it. The questionnaire utilizes a Visual Analog Scale (VAS) for data presentation and demonstrates good validity and reliability.
Questionnaires are among the most common methodologic approaches used in research studies [32]. Valid and reliable instruments are paramount to collecting accurate data, which is the foundation for generating meaningful and impactful results [33, 34]. Currently, validated questionnaires exist to assess patient satisfaction with dentures [35], implant-supported prostheses [36], and active orthodontic treatment [37].
To date, there is no validated questionnaire that assesses patient acceptance and satisfaction with orthodontic retainers although prior studies have utilized patient questionnaires on orthodontic retainers. [10, 12, 38]. Hichens et al. piloted a patient satisfaction questionnaire with good reliability, but it remains unvalidated [38]. The Retainer-modified OHIP(M) questionnaire by Mohd Tahir et al. have only undergone face validity testing [12]. Similarly, questionnaire by Saleh et al. on patient acceptance of retainers lacked validation as well [10]. This research addresses the critical gap by presenting the first validated questionnaire on patient acceptance of orthodontic retainers.
The questionnaire in the present study was modified from that developed by Saleh et al., consisting of ten items related to acceptability of orthodontic retainers [10]. During content validation, the expert panel identified “tooth biting,” “gingival irritation,” “swallowing,” and “self-confidence” as less relevant. The exclusion of "tooth biting" and "swallowing" likely reflects their limited relevance as these issues are typically transient [39]. Similarly, "gingival irritation" can usually be resolved effectively through clinical adjustments. The removal of “self-confidence” may reflect that it is a complex and subjective outcome influenced by multiple factors beyond the retainer alone [40], making it less suitable for inclusion in a tool focused on the retainer’s direct effects on patient acceptance. Of the six retained items, “speech,” “appearance” and “comfort” remained unchanged. “Oral hygiene” and “durability” were modified to “cleanliness of mouth” and “robustness of retainer” respectively, to make it more understandable to laypeople. The content validated questionnaire was deemed relevant and representative of the construct of patient’s perception towards retainers [41].
Face validity ensures the intended audience finds the questionnaire items understandable, relevant, and easy to answer [42]. During structured interviews, participants highlighted that the phrases “mouth cleanliness” and “robustness of retainer” were confusing. Many were uncertain whether “mouth cleanliness” referred to the cleanliness of retainer or how easy it was to clean. Similarly, over half of the participants found “robustness of retainer” vague, unsure whether it referred to the thickness, quality, size, or condition of the retainer. In response to the feedback, the expert panel revised these items to “feeling of cleanliness of mouth” and “strength and durability of retainer” for clarity. The item “fitting of the appliance” was changed to “fitting of the retainer” to enhance specificity.
The moderate correlation between the present questionnaire and the Retainer-modified S-OHIP(M) suggests both instruments measure related constructs of patient acceptance of orthodontic retainers. However, the moderate strength of this correlation also suggests that the two instruments are not entirely equivalent, which is expected given the differences in scope. The Retainer-modified S-OHIP(M) [12] was adapted from short version of the Malaysian OHIP-14 [26] that primarily assesses a broader range of impacts of oral disorders on quality of life. In contrast, the present questionnaire focuses more specifically on patient acceptance and comfort with orthodontic retainers. Thus, the present questionnaire potentially captures a more targeted insights into specific aspects of retainer acceptance which might not be fully captured by the Retainer-modified S-OHIP(M).
The questionnaire has undergone a rigorous validation process and has demonstrated moderate to excellent test-retest reliability, with ICC values of 0.69 to 0.91. In comparison, a study evaluating a questionnaire about complete dentures in edentulous patients reported an ICC of 0.37–0.83 [43]. Another study on the validation of a Korean version of the oral health impact profile questionnaire had an ICC of 0.40–0.61 [44]. The lower ICC in these studies could be attributed to the longer test-retest interval of two to three months, compared to two weeks in the present study. While the test-retest interval may vary depending on the construct to be measured, a period of approximately 2 weeks is typically regarded as appropriate [45].
Tests for internal consistency revealed good to excellent internal consistency for five items, with Cronbach’s Alpha ranging from 0.81 to 0.91, while "appearance" showed a questionable value of 0.69. Unlike the other five items which are more closely related to functional aspects, “appearance” pertains to a highly subjective aesthetic component [46]. Subjects might consistently prioritize these functional qualities over aesthetics, leading to strong internal consistency among these items. Furthermore, the perception of appearance may not always align with functional priorities, as some subjects may place less emphasis on appearance [47] and focus more on how the retainer feels or functions. This misalignment could weaken the relationship between appearance with the other items, leading to a lower Cronbach’s Alpha value.
As patient compliance toward orthodontic retainers heavily depends on their acceptance of the retainer, this validated instrument has the potential to predict future compliance [8]. It may reveal retainer overlooked design flaws that potentially reduce patient acceptance such as discomfort, speech, and esthetics embarrassment [48]. Along this vein, this validated instrument empowers clinicians to deliver patient-specific recommendations and address potential compliance issues, ensuring long-term compliance to retention regimes.
Furthermore, it serves as an effective tool for comparing patient acceptance across different retainer types. The questionnaire has been utilized in a randomized-controlled trial to assess subjects’ perceptions of the traditional Hawley retainer versus a modified version of the VFR after orthodontic expansion treatment [49]. Optimizing patient comfort improves compliance, which in turn lead to reduced occurrence of relapse, saving time, cost, and manpower [50].
A limitation of the present study is that all the participants were drawn from patients attending a single university clinic and may not fully represent the Malaysian population. Therefore, further validation in a broader population may be necessary to ensure generalizability. However, none of the questionnaire items were socially or ethnically specific. Another potential limitation is that the VAS may pose challenges for patients unfamiliar with this format, potentially influencing their responses. Future research should compare responses obtained via VAS with those from other instruments, such as Likert scales. Furthermore, long-term follow-up studies are recommended to assess how patient perceptions evolve over time and how these perceptions relate to actual compliance with retainer use.
Conclusions
The ten-item questionnaire on patient acceptance of orthodontic retainers has been validated for content, face, construct, and criterion validity. Following the content validity test, four items were discarded, and three items were revised and rephrased. The final six-item questionnaire showed a moderate correlation with the relevant questionnaire, the Retainer-Modified S-OHIP(M). It also demonstrated moderate to excellent test-retest reliability, with none of the questionnaire items showing unacceptable or poor internal consistency. This validated questionnaire provides a reliable tool for assessing patient acceptance of orthodontic retainers and can be used effectively in clinical settings to gauge patient experiences and preferences.
Supporting information
S1 Table. Thematic analysis data and verbatim for face validity.
https://doi.org/10.1371/journal.pone.0314853.s001
(DOCX)
Acknowledgments
The authors extend their gratitude to Prof. Dr. Roslan Saub, as well as the orthodontists and all the participants who participated in this study.
References
- 1. Littlewood SJ, Millett DT, Doubleday B, Bearn DR, Worthington H V. Orthodontic retention: A systematic review. J Orthod. 2006;33:205–212. pmid:16926314
- 2. Yu Y, Sun J, Lai W, Wu T, Koshy S, Shi Z. Interventions for managing relapse of the lower front teeth after orthodontic treatment. Cochrane Database Syst Rev 2013. pmid:24014170
- 3. Blake M, Bibby K. Retention and stability: A review of the literature. Am J Orthod Dentofacial Orthop. 1998;114:299–306. pmid:9743135
- 4. Renkema AM, Hélène Sips ET, Bronkhorst E, Kuijpers-Jagtman AM. A survey on orthodontic retention procedures in the Netherlands. Eur J Orthod. 2009;31:432–437. pmid:19401355
- 5. Ab Rahman N, Low TF, Idris NS. A survey on retention practice among orthodontists in Malaysia. Korean J Orthod. 2016;46:36–41. pmid:26877981
- 6. Little RM, Riedel RA, Artun J. An evaluation of changes in mandibular anterior alignment from 10 to 20 years postretention. Am J Orthod Dentofacial Orthop. 1988;93:423–428. pmid:3163221
- 7. Almuqbil S, Banabilh S. Postretention phase: Patients’ compliance and reasons for noncompliance with removable retainers. Int J Orthod Rehabil. 2019;10:18.
- 8. Wong P, Freer TJ. Patients’ attitudes towards compliance with retainer wear. Aust Orthod J. 2005;21:45–53. pmid:16433081
- 9. Oates J, Weston WW, Jordan J. The impact of patient-centered care on outcomes. Fam Pract. 2000;49:796–804. pmid:11032203
- 10. Saleh M, Hajeer MY, Muessig D. Acceptability comparison between Hawley retainers and vacuum-formed retainers in orthodontic adult patients: a single-centre, randomized controlled trial. Eur J Orthod. 2017;39:453–461. pmid:28430890
- 11. Ngan P, Kess B, Wilson S. Perception of discomfort by patients undergoing orthodontic treatment. Am J Orthod Dentofacial Orthop. 1989;96:47–53. pmid:2750720
- 12. MohdTahir N, Wan Hassan WN, Saub R. Comparing retainers constructed on conventional stone models and on 3D printed models: a randomized crossover clinical study. Eur J Orthod. 2019;41:370–380. pmid:30321319
- 13. Cuschieri S. The STROBE guidelines. Saudi J Anaesth. 2019;13:S31. pmid:30930717
- 14. Lawshe CH. A quantitative approach to content validity. Pers Psychol. 1975;28:563–575.
- 15. Polit DF, Beck CT. The content validity index: Are you sure you know what’s being reported? critique and recommendations. Res Nurs Health. 2006;29:489–497. pmid:16977646
- 16. Yusoff MSB. ABC of Response Process Validation and Face Validity Index Calculation. Education in Medicine Journal. 2019;11:55–61.
- 17. Frantz A, Holmgren K. The Work Stress Questionnaire (WSQ)—Reliability and face validity among male workers. BMC Public Health. 2019;19:1–8.
- 18. Anthoine E, Moret L, Regnault A, Sbille V, Hardouin JB. Sample size used to validate a scale: a review of publications on newly-developed patient reported outcomes measures. Health Qual Life Outcomes. 2014;12:176. pmid:25492701
- 19. Kaiser HF. An index of factorial simplicity. Psychometrika. 1974;39:31–36.
- 20. Costello AB, Osborne J. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract Assess Res Eval. 2005;10:7.
- 21.
Pett M, Lackey N, Sullivan J. Making sense of factor analysis: The use of factor analysis for instrument development in health care research. 1st ed. SAGE Publications, Inc, Thousand Oaks, CA.
- 22.
Hair J F, Black WC, Babin BJ, Anderson RE. Multivariate data analysis, 7th ed. Pearson Education Limited.
- 23. Kang H. Sample size determination and power analysis using the G*Power software. J Educ Eval Health Prof. 2021;18:17. pmid:34325496
- 24. Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39:175–191. pmid:17695343
- 25. Roscoe JT. Fundamental research statistics for the behavioral sciences. Holt, Rinehart and Winston. 1969.
- 26. Saub R, Locker D, Allison P. Derivation and validation of the short version of the Malaysian Oral Health Impact Profile. Community Dent Oral Epidemiol. 2005;33:378–383. pmid:16128798
- 27. Bujang MA, Baharum N. A simplified guide to determination of sample size requirements for estimating the value of intraclass correlation coefficient: a review. Arch Orofac Sci. 2017;12:1–11.
- 28.
De Vet HCW, Terwee CB, Mokkink LB, Knol DL. Measurement in medicine: a practical guide, 1st ed. Cambridge University Press.
- 29. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016;15:155. pmid:27330520
- 30.
George D, Mallery P. IBM SPSS statistics 26 step by step: A simple guide and reference. Routledge.
- 31.
Child D. The essentials of factor analysis, 3rd ed. A&C Black.
- 32. Ranganathan P, Caduff C, Frampton CMA. Designing and validating a research questionnaire—Part 1. Perspect Clin Res. 2024;15:42–45.
- 33. Taherdoost H. and Reliability of the Research Instrument; How to Test the Validation of a Questionnaire/Survey in a Research. Int J Acad Res 5.
- 34. Boynton PM, Greenhalgh T. Hands-on guide to questionnaire research: Selecting, designing, and developing your questionnaire. BMJ: British Medical Journal. 2004;328:1312.
- 35. Vervoorn JM, Duinkerke ASH, Luteijn F, van de Poel ACM. Assessment of denture satisfaction. Community Dent Oral Epidemiol. 1988;16:364–367. pmid:3060310
- 36. Castillo-Oyagüe R, Suárez-García MJ, Perea C, Río J Del, Lynch CD, Gonzalo E, et al. Validation of a new, specific, complete, and short OHRQoL scale (QoLFAST-10) for wearers of implant overdentures and fixed-detachable hybrid prostheses. J Dent. 2016;49:22–32. pmid:27134045
- 37. Bennett ME, Tulloch JFC, Vig KWL, Phillips CL. Measuring Orthodontic Treatment Satisfaction: Questionnaire Development and Preliminary Validation. J Public Health Dent. 2001;61:155–160. pmid:11603319
- 38. Hichens L, Rowland H, Williams A, Hollinghurst S, Ewings P, Clark S, et al. Cost-effectiveness and patient satisfaction: Hawley and vacuum-formed retainers. Eur J Orthod. 2007;29:372–378. pmid:17702797
- 39. Moda LB, da Rosa Moreira Bastos RT, Flores-Mir C, Normando D. Patients’ perception of orthodontic retainers: a systematic review. Eur J Orthod. pmid:38071751
- 40. Perry P. Concept analysis: confidence/self-confidence. Nurs Forum (Auckl). 2011;46:218–230. pmid:22029765
- 41. Yusoff MSB. ABC of Content Validation and Content Validity Index Calculation. Education in Medicine Journal. 2019;11:49–54.
- 42. Allen MS, Robson DA, Iliescu D. Face Validity: A Critical but Ignored Component of Scale Construction in Psychological Assessment. Eur J Psychol Assess. 2023;39:153–156.
- 43. Komagamine Y, Kanazawa M, Kaiba Y, Sato Y, Minakuchi S. Reliability and validity of a questionnaire for self-assessment of complete dentures. BMC Oral Health. pmid:24885473
- 44. Bae KH, Kim HD, Jung SH, Park DY, Kim JB, Paik D Il, et al. Validation of the Korean version of the oral health impact profile among the Korean elderly. Community Dent Oral Epidemiol. 2007;35:73–79. pmid:17244140
- 45.
Streiner DL, Norman GR, Cairney J. Health measurement scales: a practical guide to their development and use, 5th ed. Oxford university press.
- 46. Ibáñez-Berganza M, Amico A, Loreto V. Subjectivity and complexity of facial attractiveness. Sci Rep. 2019;9:1–12.
- 47. Pratt MC, Kluemper GT, Lindstrom AF. Patient compliance with orthodontic retainers in the postretention phase. Am J Orthod Dentofacial Orthop. 2011;140:196–201. pmid:21803257
- 48. Al-Moghrabi D, Salazar FC, Pandis N, Fleming PS. Compliance with removable orthodontic appliances and adjuncts: A systematic review and meta-analysis. Am J Orthod Dentofacial Orthop. 2017;152:17–32. pmid:28651764
- 49. Ashari A, Nik Mustapha NM, Yuen JJX, Saw ZK, Lau MN, Xian L, et al. A two-year comparative assessment of retention of arch width increases between modified vacuum-formed and Hawley retainers: a multi-center randomized clinical trial. Prog Orthod. 2022;23:40. pmid:36018418
- 50. El-Huni A, Colonio Salazar FB, Sharma PK, Fleming PS. Understanding factors influencing compliance with removable functional appliances: A qualitative study. Am J Orthod Dentofacial Orthop. 2019;155:173–181. pmid:30712688