Standardized instruments are often used to monitor one’s progress in tinnitus relief although they were developed to screen and diagnose tinnitus. The need for the development for a tinnitus outcome assessment tool is high in the field of audiology and otolaryngology. The purpose of this study was to develop a tinnitus outcome questionnaire for sound management (listening to sound stimuli for tinnitus relief) and assess its content validity. A total of 32 questions with six domains (Tinnitus characteristics, the impact of tinnitus, tinnitus and hearing issues, handedness, tinnitus management, and sound management outcome) were generated after closely investigating major tinnitus questionnaires used worldwide (i.e. Tinnitus Handicap Inventory and Tinnitus Handicap Questionnaire) as well as literature. Ten healthcare professionals evaluated the appropriateness of the questionnaire items on a five-point Likert scale, where 1 is strongly inappropriate and 5 is strongly appropriate. Content relevance was assessed by computing the content validity index with the cut-off value of 0.75. Each response was first weighted as follows: 1 = 0; 2 = 0.25; 3 = 0.5; 4 = 0.75; and 5 = 1.0. The weighted average was then calculated. Items with a content validity index less than 0.75 were discarded and some items were revised according to the experts’ feedback. As a result, 31 out of the 32 items had the content validity index higher than 0.75, indicating that the items are appropriate to obtain information about the six domains. Reflecting the experts’ feedback, some questions were revised to be more specific. The study provides a baseline structure regarding potential questions to be included in a tinnitus outcome questionnaire for sound management. Development and standardization of such questionnaire would be a pathway to validating tinnitus relief via sound therapy.
Citation: Seol HY, Kim G-Y, Jo M, Kang S, Cho YS, Hong SH, et al. (2021) Content validity of the tinnitus outcome questionnaire for sound management. PLoS ONE 16(5): e0251244. https://doi.org/10.1371/journal.pone.0251244
Editor: Qian-Jie Fu, University of California, Los Angeles, UNITED STATES
Received: October 21, 2020; Accepted: April 22, 2021; Published: May 6, 2021
Copyright: © 2021 Seol et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Tinnitus, commonly known as “ringing” in the ears, refers to phantom sounds in the ear . Numerous literatures have reported various etiologies of tinnitus: otologic and neurologic disorders (i.e. hearing loss, otitis media, ear wax, head injury etc.), drugs (i.e. anti-inflammatory drugs, chemotherapy agents etc.) and anatomical issues (i.e. tensor tympani spasm, patulous eustachian tube etc.) [2, 3]. Along with the etiologies, tinnitus expresses a wide variety of characteristics in terms of onset, severity, laterality, frequency, and types of sound. It can occur abruptly or progressively and it can be constant or intermittent. For types of sound, tinnitus includes, but not limited to, buzzing, puretone, music, and hissing sounds. Tinnitus associated comorbidities have been well researched [4, 5]. For instance, one might experience non-bothersome tinnitus only at night in his or her own quiet bedroom while others might experience bothersome ringing to the point of feeling suicidal. Studies have also reported a high prevalence of tinnitus [6, 7]. Bhatt et al. performed a cross-sectional analysis on 75,764 respondents from the 2007 National Health Interview Survey and reported that approximately one in ten adults experience tinnitus in the United States . A high prevalence of tinnitus with increasing age in South Korea was reported by Lee et al. who performed a big data analysis from the National Health Information database. In 2015, 0.78 million per 51 million sought medical care regarding tinnitus .
There are several approaches for tinnitus management: sound amplification, surgery, tinnitus retraining or cognitive behavioral therapy, magnetic stimulation, and pharmaceuticals . Sound amplification refers to hearing technology, such as hearing aids, as well as background noise and noise maskers . A combination of counselling and ear-level noise generators can be utilized to separate tinnitus from one’s negative thoughts and responses . This approach is called tinnitus retraining therapy. Cognitive behavioral therapy, on the other hand, involves counselling and relaxation techniques to correct one’s negative thoughts and responses towards tinnitus . An appropriate approach is determined by healthcare professionals based on the patient’s complaints, possible underlying condition, and characteristics of tinnitus.
To monitor one’s progress in tinnitus relief, several standardized instruments are utilized, such as the Tinnitus Handicap Inventory (THI) and Tinnitus Handicap Questionnaire (THQ) . The Korean version of the THI shown to have good reliability and validity, is most widely used in Korea [9, 10]. However, it is important to note that these standardized questionnaires were not developed to measure treatment outcome. In the case of THI, it was originally developed for the purpose of screening and diagnosing tinnitus, but it is frequently utilized to examine the effectiveness of tinnitus treatments . Langguth et al. adds that there is need for the development of an outcome measurement tool to monitor treatment effects . A more recent study in 2017 reports that clinical trials regarding tinnitus have low methodological and reporting quality  and this lack of outcome assessment tool also partly contributes to the difficulty finding a “cure” for tinnitus . The purpose of this study is to design a tinnitus outcome questionnaire, specifically for sound management as a new outcome measure and assess its content validity.
Materials and methods
Phase I: Generating questionnaire items
Phase I involved generating questionnaire items by investigating widely accepted standardized tinnitus questionnaires, such as THI, THQ, Tinnitus Functional Index (TFI), and Tinnitus Sample Case History Questionnaire (TSCHQ) [12, 15–17], and literature on tinnitus and management using sound . Most standardized questionnaires contain a wide variety of domains including social, mental, and physical functioning, affective responses towards tinnitus, and outcome measurements [12, 17, 19]. Studies regarding sound management for tinnitus relief used the visual analogue scale (VAS) to measure annoyance, loudness, awareness caused by tinnitus [20, 21]. The questionnaire items were created based on the main purposes of the tinnitus outcome questionnaire for sound management: quantifying tinnitus characteristics and the improvement of tinnitus before and after listening to sound stimuli. Different domains were created upon examination of some standardized questionnaires to quantify tinnitus characteristics and the impact of tinnitus on individuals’ lives. For instance, Tinnitus Questionnaire had three categories: sleep disturbance, emotional distress, and auditory perceptual difficulties . The auditory perceptual difficulties domain included items related to hearing issues. Thus, the authors added a domain of tinnitus and hearing issues in the questionnaire. As a result, a total of 32 questions with six categories were generated: tinnitus characteristics, the impact of tinnitus, tinnitus and hearing issues, handedness, tinnitus management, and sound management outcome.
Phase II: Assessing the content validity of the items
Phase II entailed content validation, which is “determination of content representativeness or content relevance of the elements or items of an instrument” according to Lynn 1986 . Once all items were generated, an online anonymous survey was created via Google Forms and was distributed to ten healthcare professionals. The professionals were recruited from January 2020 to November 2020 from the following inclusion criteria: (a) adults 19 years old or older, (b) practitioners in the field of audiology or otolaryngology, and (c) native speakers of Korean. Exclusion criteria included individuals who were not willing to provide informed consent. All experimental procedures were approved by the regulations set by Samsung Medical Center’s Institutional Review Board and were carried out in accordance with approved guidelines. The experts were first informed about the purpose, risks and benefits, voluntary nature, and consequences of withdrawal of the survey. Verbal consent was obtained from all experts. The mean age of the professionals was 38.2 years (SD = 2.9). Average years of overall clinical experience and clinical experience in tinnitus were 11.1 (SD = 3.8) and 7.4 years (SD = 2.2), respectively. Demographics of the experts are described in Table 1. The experts judged how appropriate each question is to collect information regarding tinnitus characteristics and the effect of sound management on a five-point Likert scale, where 1 is strongly inappropriate and 5 is strongly appropriate. The experts were also requested to provide feedback, especially when they evaluated the item “1-strongly irrelevant” and “2-irrelevant” so that the authors could revise the items. Upon completion, the content validity index (CVI) with a cut-off value of 0.75, which was recommended in multiple studies, was calculated to validate the content of the questionnaire items [23, 24]. As the CVI is an index of inter-rater agreement, there is as an issue of chance agreement which Polit et al. tried to address utilizing the kappa statistic . Similar to findings in Cicchetti et al., the value of 0.75 was found to be “excellent” with 10 or more experts [25, 26]. Thus, 0.75 was selected to be the cut-off value for this study. To compute the CVI, each response was first weighted as follows: 1 = 0; 2 = 0.25; 3 = 0.5; 4 = 0.75; and 5 = 1.0 and the average was calculated. As the cut-off value for the CVI is 0.75, items with a CVI less than 0.75 were considered to be less relevant, and therefore, discarded. No statistical analysis was performed as it was not necessary for the study. Several items were revised according to the experts’ feedback.
Tinnitus outcome questionnaire for sound management
As described above, six domains and 32 questions were formed for the tinnitus management outcome questionnaire for sound management. The ‘Tinnitus characteristics’ domain consists of questions regarding the onset, cause, loudness, annoyance, and frequency of tinnitus. Items in ‘The impact of tinnitus’ include awareness of and stress caused by tinnitus. Items related to hyperacusis and hearing loss are in ‘Tinnitus and hearing issues’. The ‘Handedness’ domain asks whether one is right- or left-handed. In the ‘Tinnitus management outcome’, individuals are asked to report various methods they have tried and the effectiveness of each method in relieving tinnitus. Lastly, the ‘Sound management outcome’ involves the loudness and annoyance of tinnitus, changes in awareness or reaction to tinnitus after listening to sound stimuli, and other stimuli that individuals wish to listen to for tinnitus relief.
Assessing content validity of the questionnaire
CVIs were calculated for each item and are shown in Table 2. Out of the 32 items, only one question (How was the onset of tinnitus?) had a CVI of 0.73 which is less than the cut-off of 0.75. Thus, the question is considered to be less relevant and discarded from the questionnaire. 11 out of 32 items had the highest CVI which was 0.98 (Does stress cause tinnitus or worsen your tinnitus?; Have you received a hearing test and been diagnosed with hearing loss?; What have you tried to relieve your tinnitus?; How long have you tried the methods you selected above?; Was if effective? If so, how much did your tinnitus improve?; Are you currently wearing hearing aids or have you used hearing aids in the past?; How long have you been wearing your hearing aids or how long have you used your hearing aids?; How many hours in a day do you wear your hearing aids or how many hours in a day have you worn your device?; Which environments do you mostly wear your hearing aids or which environments have you mostly used your hearing aids?; Are there any changes in awareness or reaction to your tinnitus after listening to sound stimuli for N weeks?; and How annoying is your tinnitus now?). Four items (Which ear do you experience tinnitus?; When did you start experiencing tinnitus?; How annoying is your tinnitus now?; and Do you feel that certain sounds others hear as quiet or comfortable are loud to you?) had the CVI of 0.95. Six questions had the CVI of 0.93: Do you constantly experience tinnitus?; How much are you aware of your tinnitus?; Are you currently taking any medications due to your tinnitus?; If yes, what medications are you currently taking?; Which ear do you experience tinnitus?; and How loud is your tinnitus right now?. One question (What does your tinnitus sound like?) had a CVI of 0.90. Lastly, the rest of the items had a CVI below 0.90 ranging between 0.78 and 0.89. These items include: Which ear are you wearing your device or which ear did you wear the device?; If you experience tinnitus in both ears, which ear is experiencing more severe tinnitus?; How loud is your tinnitus right now?; How frequently do you experience tinnitus?; Are you right- or left-handed?; If not, do you experience difficulty hearing?; Are there any other sound stimuli you wish to listen to? If so, what are they?; What is the cause of your tinnitus?; and If yes, does your medication worsen your tinnitus?
Along with hearing loss, tinnitus has become a major health concern affecting various aspects of individuals’ lives [27, 28]. However, it is often difficult to manage tinnitus due to the lack of objective examination and consensus regarding the pathophysiology of tinnitus, and limited options for management [13, 28]. Clinical evaluations involving case history, audiometry, and tinnitogram could provide an insight as to one’s tinnitus characteristics and suggest possible intervention methods as an attempt to reduce tinnitus. However, when it comes to the efficacy of the intervention, healthcare professional typically rely heavily on self-report questionnaires . Even these questionnaires have a limited capability of measuring tinnitus treatment outcome as their main purpose is diagnosing tinnitus [12, 27]. Thus, developing an outcome questionnaire assessing changes in tinnitus before and after intervention is high in demand in clinical practice as well as research.
In this study, the authors designed a comprehensive tinnitus outcome questionnaire which has general items about tinnitus characteristics as well as items specific to sound management. The authors assessed the validity of its content which is the first step when developing questionnaires examining treatment effects . The questionnaire items were created upon reviewing standardized tinnitus questionnaires and research studies. Ten healthcare professionals with clinical experience in tinnitus assessed the content validity. Among 32 items, 31 items illustrated that their content representativeness is appropriate with the CVI over 0.75 which is its cut-off value. In ‘Tinnitus characteristics,’ the experts evaluated the following three questions as the most relevant questions in the section: Which ear do you experience tinnitus?; When did you start experiencing tinnitus?; and How annoying is your tinnitus right now?. These questions are available in many questionnaires that are already in use (THI, TFI, THQ, and TSCHQ). One question (How was the onset of tinnitus?) had the lowest CVI (0.73) overall with the experts’ feedback of vagueness of the question and this item was removed. In the “The impact of tinnitus” section, all questions (How much are you aware of your tinnitus? and Does stress cause tinnitus or worsen your tinnitus?) had higher CVI values of 0.93 and 0.98, respectively. In the ‘Tinnitus and hearing issues’ domain, two out of three questions had a CVI higher than 0.95 and one question asking if one thinks to have hearing loss had a CVI of 0.83. Experts suggested the following phrase to be included in the question: If not, do you experience difficulty hearing? and the authors reflected the feedback accordingly in the final version of the questionnaire. ‘Handedness’ is also discussed in many tinnitus literatures [12, 30–32] and was evaluated to be included for the tinnitus outcome questionnaire for sound management. In the domain of ‘Tinnitus management,’ all questions were evaluated to be appropriate except one question about medication (‘Are you currently taking any medications?’) had a relatively low CVI of 0.78. Applying the experts’ comment of adding “due to tinnitus” to the item, the authors revised the question to ‘Are you currently taking any medications due to your tinnitus?’ so that the question is more specific. Lastly, the experts evaluated all items in the ‘Sound management outcome’ domain to be suitable for the tinnitus outcome questionnaire for sound management.
To the best of our knowledge, this is the first study to develop a tinnitus outcome questionnaire which includes questions specific to sound management; currently, no instruments contain questions specific to sound therapy. Content validity assessment suggested a baseline for potential questionnaire items. Numerous studies have examined the effect of sound therapy, but most of them used questionnaires that did not include specific questions about sound therapy [33, 34]. Since the tinnitus outcome questionnaire contains both tinnitus characteristics and sound management items, it would be convenient for professionals to obtain tinnitus case history and outcome information in one questionnaire and monitor any changes in tinnitus characteristics after sound therapy.
However, this study has a limitation; only the content of the questionnaire was validated. Fackrell et al. mentions that reliability, validity, responsiveness, and interpretation as key factors for validating questionnaires . Further research is needed to examine internal consistency, reproducibility, and construct validity of the questionnaire as next steps. Other important concepts to consider are “responsiveness” and “minimally clinically important difference (MCID)”. Responsiveness refers to high sensitivity to any changes related to treatment and can be expressed as effect size [18, 27, 29]. Meikle et al. reviewed nine principal questionnaires (Tinnitus questionnaire, THQ, Tinnitus Severity Scale, Subjective Tinnitus Severity Scale Tinnitus Reaction Questionnaire, Tinnitus Severity Grading, Tinnitus Severity Index, THI, and Intake Interview for Tinnitus Retraining Therapy) for tinnitus and reported that they were not designed to have high “responsiveness” and lacked information about effect size . Thus, additional work is necessary to develop a tinnitus outcome assessment tool with high validity, reliability, and responsiveness. MCID is also related to data interpretation: how much change is clinically meaningful?. Adamchic et al. examined test-rest reliability, validity, and MCID of VAS for loudness and annoyance for individuals with chronic tinnitus. It was a 16-week clinical study and the results not only showed good test-retest reliability and validity, but also showed 10–15 points of MCID estimates . In spite of the study demonstrating the effectiveness of using VAS to measure tinnitus loudness and annoyance, many of the current tinnitus questionnaire do not describe MCID . The tinnitus outcomes questionnaire for sound management needs to consist of MCID information, such as a grading system. Additionally, possibility of using certain tinnitus characteristics information, such as annoyance, loudness, and frequency, to create various MCID scores based on the effectiveness of sound can be explored.
Once the outcome questionnaire is developed and standardized, it would be beneficial for tinnitus patients, healthcare professionals, and researchers. As evidence-based practice is critical, establishment of such questionnaire as a counselling tool for patients would demonstrate that offered intervention or management option is capable of providing benefit to the patients. The ability to directly see intervention effects over time could lead to more motivation and engagement in tinnitus management. For healthcare professionals and researchers, various subtypes of tinnitus could be characterized and an effective intervention method could be identified for each subtype, leading to the delivery of individualized care as well as the pathway of validating tinnitus relief via sound therapy .
- 1. Jastreboff PJ. Phantom auditory perception (tinnitus): mechanisms of generation and perception. Neurosci Res. 1990; 8:221–54. pmid:2175858
- 2. Folmer RL, Martin WH, Shi Y. Tinnitus: questions to reveal the cause, answers to provide relief. J Fam Pract. 2004; 53:532–40. pmid:15251092
- 3. Lockwood AH, Salvi RJ, Burkard RF. Tinnitus. N Engl J Med. 2002; 347:904–10. pmid:12239260
- 4. Pavaci S, Tortorella F, Fioretti AB, Angelone AM, Businco LDR, Lauriello M, et al. Analysis of the audiological characteristics and comorbidity in patients with chronic tinnitus. Audiol Res. 2019; 9:231. pmid:31929867
- 5. Martz E, Jelleberg C, Dougherty DD, Wolters C, Schneiderman A. Tinnitus, Depression, Anxiety, and Suicide in Recent Veterans: A Retrospective Analysis. Ear Hear. 2018; 39:1046–56. pmid:29624539
- 6. Lee HM, Han KD, Kong SK, Nam EC, Park SN, Shim HJ, et al. Epidemiology of Clinically Significant Tinnitus: A 10-Year Trend From Nationwide Health Claims Data in South Korea. Otol Neurotol. 2018; 39:680–7. pmid:29879090
- 7. Bhatt JM, Lin HW, Bhattacharyya N. Prevalence, Severity, Exposures, and Treatment Patterns of Tinnitus in the United States. JAMA Otolaryngol Head Neck Surg. 2016; 142:959–65. pmid:27441392
- 8. Wu V, Cooke B, Eitutis S, Simpson MTW, Beyea JA. Approach to tinnitus management. Can Fam Physician. 2018; 64:491–5. pmid:30002023
- 9. Oh S-J, Goh E-K, Kong S-K, Park SN, Choi J, Lee HM, et al. Development of a Korean-Type Qualitative Case History Tinnitus Questionnaire. Korean J Otorhinolaryngol-Head Neck Surg. 2017; 61:182–92.
- 10. Kim JH, Lee SY, Kim CH, Lim SL, Shin JN, Chung WH, et al. Reliability and validity of a Korean adaptation of the Tinnitus Handicap Inventory. Korean J Otorhinolaryngol-Head Neck Surg. 2002; 45:328–34.
- 11. Zeman F, Koller M, Schecklmann M, Langguth B, Landgrebe M, group TRIds. Tinnitus assessment by means of standardized self-report questionnaires: psychometric properties of the Tinnitus Questionnaire (TQ), the Tinnitus Handicap Inventory (THI), and their short versions in an international and multi-lingual sample. Health Qual Life Outcomes. 2012; 10:128. pmid:23078754
- 12. Langguth B, Goodey R, Azevedo A, Bjorne A, Cacace A, Crocetti A, et al. Consensus for tinnitus patient assessment and treatment outcome measurement: Tinnitus Research Initiative meeting, Regensburg, July 2006. Prog Brain Res. 2007; 166:525–36. pmid:17956816
- 13. Londero A, Hall DA. Call for an Evidence-Based Consensus on Outcome Reporting in Tinnitus Intervention Studies. Front Med (Lausanne). 2017; 4:42. pmid:28484698
- 14. McFerran DJ, Stockdale D, Holme R, Large CH, Baguley DM. Why Is There No Cure for Tinnitus? Front Neurosci. 2019; 13:802. pmid:31447630
- 15. Langguth B, Kreuzer PM, Kleinjung T, De Ridder D. Tinnitus: causes and clinical management. Lancet Neurol. 2013; 12:920–30. pmid:23948178
- 16. Kuk FK, Tyler RS, Russell D, Jordan H. The psychometric properties of a tinnitus handicap questionnaire. Ear Hear. 1990; 11:434–45. pmid:2073977
- 17. Newman CW, Jacobson GP, Spitzer JB. Development of the Tinnitus Handicap Inventory. Arch Otolaryngol Head Neck Surg. 1996; 122:143–8. pmid:8630207
- 18. Stein A, Wunderlich R, Lau P, Engell A, Wollbrink A, Shaykevich A, et al. Clinical trial on tonal tinnitus with tailor-made notched music training. BMC Neurol. 2016; 16:38. pmid:26987755
- 19. Hallam RS, Jakes SC, Hinchcliffe R. Cognitive variables in tinnitus annoyance. Br J Clin Psychol. 1988; 27:213–22. pmid:3191301
- 20. Bahaghighat S, Farahani S, Hoseinabadi R, Jalaie S. The effect of hearing amplification on tinnitus improvement. Auditory and Vestibular Research. 2018
- 21. Tass PA, Adamchic I, Freund H-J, von Stackelberg T, Hauptmann C. Counteracting tinnitus by acoustic coordinated reset neuromodulation. Restorative neurology and neuroscience. 2012; 30:137–59. pmid:22414611
- 22. Lynn MR. Determination and quantification of content validity. Nurs Res. 1986; 35:382–5. pmid:3640358
- 23. Yaghmaei F. Content validity and its estimation. J Med Educ. 2003; 3:25–7.
- 24. Fehring RJ. Methods to validate nursing diagnoses. Nursing Faculty Research and Publications. 1987:27. pmid:3679856
- 25. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in nursing & health. 2007; 30:459–67. pmid:17654487
- 26. Cicchetti DV, Sparrow SA. Developing criteria for establishing interrater reliability of specific items: applications to assessment of adaptive behavior. American journal of mental deficiency. 1981 pmid:7315877
- 27. Meikle MB, Stewart BJ, Griest SE, Henry JA. Tinnitus outcomes assessment. Trends Amplif. 2008; 12:223–35. pmid:18599500
- 28. Kleinjung T, Langguth B. Avenue for Future Tinnitus Treatments. Otolaryngol Clin North Am. 2020; 53:667–83. pmid:32381341
- 29. Fackrell K, Hall DA, Barry J, Hoare DJ. Tools for tinnitus measurement: development and validity of questionnaires to assess handicap and treatment effects. In: Signorelli F, Turjman F, editors. Tinnitus: causes, treatment and short and long-term health effects. New York: Nova Science; 2014. pp. 13–60.
- 30. Alhazmi F, Kay T, Mackenzie I, Kemp G, Sluming V. An investigation of the impact of tinnitus perception on the quality of life. Journal of Phonetics and Audiology. 2016
- 31. Sørensen M, Nielsen GE, Larsen L. A preliminary validation of a Norwegian version of the Tinnitus Sample Case History Questionnaire. Scandinavian journal of psychology. 2020; 61:549–59. pmid:31914212
- 32. Lanting CP, de Kleine E, Langers DR, van Dijk P. Unilateral tinnitus: changes in connectivity and response lateralization measured with fMRI. PloS one. 2014; 9:e110704. pmid:25329557
- 33. Barozzi S, Ambrosetti U, Behrens T, Passoni S, Del Bo L. Effects of tinnitus retraining therapy with different colours of sound. The international tinnitus journal. 2017; 21:139–43. pmid:29336133
- 34. Suzuki FAdB, Suzuki FA, Yonamine FK, Onishi ET, Penido NO. Effectiveness of sound therapy in patients with tinnitus resistant to previous treatments: importance of adjustments. Brazilian Journal of otorhinolaryngology. 2016; 82:297–303. pmid:26541232
- 35. Adamchic I, Langguth B, Hauptmann C, Tass PA. Psychometric evaluation of visual analog scale for the assessment of chronic tinnitus. Am J Audiol. 2012; 21:215–25. pmid:22846637