1.1. The RCADS-47

The RCADS is a measure of symptoms of anxiety and depression for young people aged between 8 and 18-years. It is a 47-item self-report questionnaire containing six subscales; Separation Anxiety Disorder (SAD), Social Phobia (SoP), Obsessive-Compulsive Disorder (OCD), Panic Disorder (PD), Generalised Anxiety Disorder (GAD) and Major Depressive Disorder (MDD). The RCADS is used extensively worldwide [e.g., 6,9], has been validated for use in many countries [e.g., 10: Denmark.; 11: Australia] and shows good psychometric properties in several countries [(e.g., 9; 10)]. It is widely-used in India across diverse samples in both clinical and research settings, indicating that it is considered culturally and conceptually valid by experts [12]. Despite this extensive use, it has not yet been subjected to a thorough test of cultural validation for use within children or young people in India, with previous research suggesting that modifications may be needed to ensure applicability within this population [(e.g., 13)]. Previously Long et al. [8] psychometrically tested the unmodified RCADS with a sample of Indian adolescents, to establish factor structure, this study will build on this examining Indian adolescents’ interpretations of items to identify linguistic and cultural discrepancies, thereby ensuring the measure more accurately captures culturally grounded experiences of anxiety and depression.

1.2. The process of culturally validating a measure

Cross-cultural validation involves taking a measure which was designed originally for use in one culture, and showing it to be meaningful, applicable and equivalent within another culture [14]. The cultural validation of mental health measures is important as culturally validated measures can identify constructs that they have been designed to more accurately, when compared to measures which have not been culturally validated [15; 16].

The International Test Commission [17] presents guidelines for adapting and translating measures. This consists of 18 guidelines across six stages: (1) pre-condition; obtaining the necessary permission for the holders of intellectual rights of the selected measure, (2) test development; ensuing that the adaptation process considers linguistic, psychological and cultural differences and includes experts with relevant experience, (3) confirmation; empirical analysis of the adaptation in an appropriate sample (e.g., establishing reliability and validity), (4) administration; the preparation of administration material (e.g., testing guidelines) to minimise any cultural-related difficulties, (5) score scales; interpretation of any group score differences based on all available information and (6) documentation; provide technical documentation of any changes, including evidence obtained to support equivalence. As the beginning of the cultural validation of the RCADS-47, this study focuses on stages 1 and 2 of the ITC guidelines, pre-condition and test development.

This study also drew upon the cultural validation guidelines presented by Caron [18]. This states that cultural validation is achieved through establishing the validity and reliability of the measure in the new population and consists of two stages. Broadly, cultural validations of mental health measures are reflective of this and are often conducted in two stages, with the first stage involving the verification of the cultural equivalence of the measure [17–19], often in the form of a Think Aloud study, to determine ‘inferential equivalence’ within the target culture. This process is also informed by wider cross-cultural adaptation literature, including Flaherty et al. [20], who identified several key dimensions of cultural equivalence in psychological tools, including semantic, content, technical, criterion, and conceptual equivalence.Following this, psychometric testing is utilised to establish the measures validity and reliability in the target culture [17–19]. This study demonstrates Stage 1; verification of the cultural equivalence of the measure. This confirms that the measure maintains its ‘inferential equivalence’ within another culture, enabling it to accurately identify the constructs it was designed to. This is of particular importance within this validation study as the cultural conceptions and perceptions of mental health can differ between India and western countries [e.g., 21]. Cultural equivalence can be assessed by presenting the measure to a small sample of the target population to identify any misunderstandings of measure items and any changes that may be needed. However, it is acknowledged that this study does not constitute a full cultural validation or establish inferential equivalence. Rather, this study examines semantic equivalence, by assessing whether participants understood item wording as intended, and item equivalence, by exploring whether items were interpreted as relevant within participants’ lived contexts. In line with Flaherty et al. [20], this represents an initial exploration of semantic and content equivalence within the target population. The Think Aloud methodology also provided preliminary insight into participants’ response processes. These processes represent an important initial stage in the broader cultural adaptation and validation of the measure prior to psychometric testing.

Think Aloud interviews were selected as they are widely-used in research involving young people [e.g., 22–24] and are effective in determining young people’s cognitive processes when completing a standardised measure [25]. Due to the diversity of languages spoken in India [(121 legitimised languages; 26)], we tested the English-version of the RCADS. Think Aloud interviews have been used extensive within health research [(e.g., 27; 28)] and have been identified as a valuable tool for insight into participant understanding of measure items, leading to measure improvement [29]. This study presents the first stage of the cultural validation of the RCADS-47 within a sample of Indian adolescents.

Due to the large number of languages spoken in India [(121 legitimised languages; 26)], underlying religious connotation associated with different languages (e.g., Hindi is associated with Hinduism), and that in Karnataka young people are often taught in English in schools (with English also being a compulsory subject in all schools for Grade 5), validating the English version of the RCADS-47 was considered neutral and inclusive, alongside providing a good anchor language for further research. Per the 2011 census, English is the second most commonly spoken language in India, with the majority of these individuals speaking it as their second or third language (Ministry of Home Affairs India (2011).

1.3. The present study

This study presents the first stage (of two) of the cultural validation of the RCADS-47 within a sample of Indian adolescents aged between 14 and 17 years. The aim of this study was to examine the semantic, item, and response process equivalence of the RCADS-47 using cognitive interviewing methods with Indian adolescents, as a first stage towards its cultural validation. To achieve this objective, 12 Indian adolescents completed Think aloud interviews as they completed each item of the measure, enabling the and amendment of any potential cultural misunderstandings within the measure, informing any changes for Stage 2 of this cultural validation [(psychometric testing; 30)].

2.1. Recruitment and Participants

Adolescents from a Youth Advisory Panel of an ongoing study were recruited [(Project SAMA; 31)], which is a collaboration between several UK and Indian universities and NIMHANS. The Youth Advisory Group was purposively recruited for the SAMA project based on their interest in adolescent mental health and their enthusiasm to shape research on this in India. Group members did not have to have experienced poor mental health themselves. They were indicative of the target population in terms of age, being school-going, from a mix of urban and rural locales and speaking English as well as local languages. Recruitment via this YPAG for the TA study was convenience sampling. The target sample was 10, as samples of five or more participants are adequate for Think Aloud studies [32] and is reflective of previous Think Aloud studies for measure analysis [(e.g., 23; 33)].

12 Indian adolescents aged between 14 and 17-years were recruited (5 males (41.7%), 7 females (58.3%); mean age = 15.67 years, SD = 0.98). The age group was selected as research indicates that adolescence is a critical time for the onset of mental health disorders [34]. Participants self-reported an overall English ability of 7.61 (cumulative of reading and speaking ability) on a scale of 1 (not well) to 10 (extremely well) and were taught in English in school, reducing the likelihood of problems encountered being due to English proficiency.

2.2. Ethics

All ethical safeguards were decided with NIMHANS and in-line with ethical guidelines in India [35]. The study was approved by the University of Leeds (PSYC-276) and NIMHANS ethics committee. Participants were encouraged to participate only if they were currently experiencing positive mental wellbeing at the point of consent. This criterion was adopted in part due to safeguarding considerations, as the study was not designed or resourced to appropriately support adolescents experiencing significant or clinical levels of distress, in line with ethical guidance for research with vulnerable populations [36; 37]. Although this may limit generalisability of findings, this study was intentionally focused on a non-clinical population. “Positive emotional wellbeing” was operationalised through adolescent self-identification at the point of recruitment. Participants and parents were provided, within the information sheet, with a brief and age-appropriate description of emotional wellbeing (e.g., generally feeling positive and not experiencing ongoing distress) and were advised to participate only if they felt this reflected their/ their child’s current experience. No formal screening tool was employed. Written parental and participant consent was obtained prior to the study, through online consent forms. Following initial written consent, participant consent was re-confirmed verbally at the start of the interview. Verbal consent was witnessed by the interviewer and audio recorded. Interviews were conducted between 21/09/2021 and 20/10/2021. Participants were reminded of their right to withdraw and given a handout containing sources of support for depression and anxiety, should they require them in the future.

2.3. Materials

The original 47-item English-version of the RCADS [9] was used. Thirty-seven items examine anxiety symptoms and ten examine depressive symptoms. Items are answered on a four-point scale depending on the frequency that the youth experiences the items from 0 (never) to 3 (always), producing a total internalising score (sum of all subscale), a total anxiety score (sum of the anxiety subscales) and a total depression score (sum of the depression subscale).

2.4. Procedure

Prior to commencement of the study, permission to use and culturally adapt the RCADS-47 was obtained from the copyright holders, consistent with Stage 1 of the ITC (2017) guidelines. Young people interested in participating were sent study information sheets for themselves and their parents. Parental and participant consent was obtained via an online written consent form. Participants also gave written consent for their data to be used in research via an online form. Following initial written consent, participant consent was re-confirmed verbally at the start of the interview, along with the collection of demographic information (Table 1). Verbal consent was witnessed by the interviewer and audio recorded. A unique ID code linked participant’s interview and demographic data. Both written and verbal instructions for the Think Aloud interviews were adapted from French et al. (2007). Participants completed an approximately 30-minute live, online audio-recorded Think Aloud interview with the first author (a UK national). An example of ‘thinking aloud’ was given to help participants familiarise themselves with the process During the interview, participants were asked to read and explain their comprehension of each item and their reason for their chosen response. Other than prompting the participant to continue talking if they fell silent for more than 10 seconds (‘keep thinking aloud’), the interviewer did not interrupt and remained off camera. Upon completion, participants were sent a debrief and a check on their wellbeing. This was deemed an appropriate length of interview for this measure based on cognitive interviewing research and practice, as this study aimed to identify how adolescents understand and respond to items rather than obtaining lengthy narrative focusing on comprehension, interpretation, and reasoning [38]. Research suggest that adequate responses can be obtained in this interview length [38]. Shorter interviews have also been shown to reduce fatigue and help maintain adolescents’ engagement and concentration [39]. Audio-recordings were transcribed verbatim.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Participant demographic information.

https://doi.org/10.1371/journal.pone.0352705.t001

2.5. Analysis

Analysis of transcripts was completed by the first author using a method adopted from previous Think Aloud studies [40; 41]. This involved coding each interview segment into at least one of six coding categories (Table 2), and if needed, more than one. Categories were; (1) no problem; (2) no sufficient thinking aloud; (3) reread/ stumbled; (4) problems understanding and (5) misinterpretation of question. As participants completed the interviews in a language which was not their first language, and as several responses did not fall into any of the pre-existing categories, a new coding category was added, namely (6) ‘responses did not make sense in English’. Consistent with recommendations for cognitive interviewing and Think Aloud methodology in questionnaire adaptation research, the frequency and type of identified comprehension and interpretation issues were reported to indicate the relative prominence of item-level difficulties within the sample, rather than to imply statistical generalisability [43; 44].

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Coding categories for the Think Aloud interviews.

https://doi.org/10.1371/journal.pone.0352705.t002

After full initial coding, 53% of item responses were randomly selected for blind second coding, yielding a good level of agreement [(83.33% agreement; K = 0.75; 45)]. Disagreements were resolved through discussion and coding was amended accordingly. Once coding was established, problematic items on the RCADS were identified and suggested rephrasing was developed. Changes were developed with guidance from Indian external advisors, for example, where difficulties in comprehension were identified, these terms were subsequently discussed with the Indian research team to identify more culturally and linguistically appropriate alternatives that would be more commonly understood within the target population.

As a general rule, we adopted a 25% cut-off, with items generating 3 or more problems (i.e., at least three participants having trouble with that item across any of the six coding categories) being candidates for rephrasing, similar to previous Think Aloud studies [(e.g., 24)]. Consistent with Step 2 of the ITC adaptation process, bilingual in-country mental health professionals from the SAMA project were involved throughout the adaptation process. Their expertise informed the interpretation of Think Aloud data, identification of potential semantic and cultural issues within items, and discussions regarding alternative phrasing to improve clarity and contextual relevance for adolescents in Karnataka. This expert input complemented the participant-based assessment of semantic and item equivalence presented in the current study.

3.1. Obsessive-Compulsive Disorder Subscale

The OCD subscale yielded the most problems (n = 27), with 4 of the 6 items being identified for rephrasing. A range of problems were encountered by participants in this subscale, with the most common being misinterpretation of the item (n = 17). Participants struggled with several aspects of the OCD items, with many failing to identify the obsessive repetitive element, instead responding as if the item was relating to aspects of their day-to-day life. For example, in response to item 16 (‘I have to keep checking that I have done things right (like the switch is off, or the door is locked’)), many participants did not identify that this item was addressing an obsessive need to continuously ensure that things have been done correctly, and instead referred to everyday tasks that have to be completed or checked (e.g., household chores). Additionally, participants struggled to understand the internal or self-focusing nature of items, and instead responses centred around societal or family expectations. For example, in response to item 44 (‘I have to do some things in just the way to stop bad things from happening’), instead of focusing on an internal drive to do things in a certain way, several participants discussed societal actions or changes, with one participants discussing studying social work to combat this.

3.2. Major Depressive Disorder Subscale

The MDD subscale yielded the second highest number of problems (n = 24), with the most commonly identified problem being participants re-reading/ stumbling, indicating that the participants expended notable ‘effort’ while understanding the item (n = 12). Overall, 5 of the 10 MDD items were identified for rephrasing. However, these suggested changes mostly consisted of minor modifications to wording, with limited significant problems surrounding understanding being identified. Analysis suggests these problems may be caused by items being worded in a complicated way, or including phrases which are uncommon in India. For example, three participants showed difficulty in reading item 15 (‘I have problems with my appetite’). It was confirmed by our external advisors that alternative phrasing, such as ‘eating patterns’, may aid understanding. This is further observed in item 6 (‘Nothing is much fun anymore’), where the word ‘much’ has been identified as not being commonly used in this context in India.

3.3. Panic Disorder, Generalised Anxiety Disorder and Social Phobia Subscales

Several problems were identified in the PD (N = 18), GAD (N = 12) and SoP (n = 9) subscales, with 3 PD items, 1 GAD items and 1 SoP item being identified for rephrasing. All subscales identified re-reading/ stumbling as a main problem, indicating some items required significant effort to understand. Items within these subscales were identified as including complicated language or language not commonly used in India, for example item 3 (‘when I have a problem I get a funny feeling in my stomach’), with the phrase ‘a funny feeling in my stomach’ proving difficult for our participants to understand.

4.1. Strengths, limitations and future research

The strengths of this study include the thorough approach towards measure validation. Some validation studies do not include this first stage and only conduct psychometric analysis, but this can limit the resulting measure as linguistic cultural equivalence has not been established.

However, findings should be considered in view of several limitations. Our sample is clearly not representative, nor aimed to be, of all young people in India. For example, the majority of participants in this study reported that they were Hindu, as discussed above, religion and religious practices may have some influence over daily norms and therefore experiences of anxiety symptoms [e.g., 51; 56], potentially influencing young people’s interpretation of the items. Therefore, although Hinduism is the prominent religion in India, findings between young people of different religions may differ (e.g., Muslims who make up 14.2% of the population (Ministry of Home Affairs India (2011)).

More broadly, India is highly culturally, linguistically, and socioeconomically diverse, and understandings, expressions, and communication of mental health symptoms may vary substantially across regions, communities, family structures, and social groups [e.g., 21]. Cultural norms relating to emotional expression, family relationships, academic expectations, social responsibility, and stigma surrounding mental health may all influence how adolescents interpret questionnaire items and decide upon responses. Consequently, certain RCADS items may hold different meanings, levels of relevance, or emotional salience in different Indian contexts. Although the current study provides preliminary evidence of semantic and item equivalence among adolescents in Karnataka, further work is needed to explore whether similar interpretations are observed across adolescents from different linguistic, religious, rural/urban, and socioeconomic backgrounds within India.

Additionally, all participants were from Karnataka. Although it has been verified by in-country advisors that young people in Karnataka do not speak a particularly unique version of Indian-English, suggesting that difficulties experienced by young people in Karnataka would be similar to young people from different states, some linguistic difference may still exist. Despite this, standardised English is taught in schools throughout India, suggesting that all school-going young people should have a similar understanding of standardised English, however findings should be applied to all areas of India with caution, due to potential linguistic differences that may exist.

Participants were recruited on the basis of self-identified positive emotional wellbeing, which may have introduced selection bias and restricts the generalisability of the findings. As a result, the perspectives captured are likely to reflect those of adolescents who are currently functioning well, and may not represent those experiencing emotional distress or clinical levels of difficulty.

Furthermore, participants were mostly from high Socio-Economic Status (SES) backgrounds and living in urban areas. This homogeneity may be partly due to the study being conducted online during COVID-19 travel restrictions, resulting in a sample consisting solely of adolescents with internet access. Consequently, adolescents from lower SES backgrounds were underrepresented, substantially limiting the generalizability of findings to the broader adolescent population in Karnataka and India more widely. This limitation is particularly important given the well-documented digital divide in India, where internet access differs significantly according to socioeconomic status and rural–urban location [57]. Adolescents from urban, high-SES backgrounds may also have had greater exposure to English and higher educational opportunities, potentially reducing the identification of language and comprehension difficulties within the RCADS items. Previous research has shown that digital exclusion and educational inequalities may disproportionately affect rural adolescents and those from disadvantaged backgrounds [57]. Therefore, future validation studies should prioritize recruiting adolescents from rural and lower-SES populations to improve representativeness and cultural applicability of the RCADS-47. A further limitation is that, although this study is a step towards validating the RCADS-47 in adolescents aged 13–17 years, no 13-year-olds were recruited at this stage of the study.

A further limitation of the study may be that participants were recruited through the SAMA Youth Advisory Panel, whose members had a pre-existing interest in adolescent mental health and research involvement. Although the group was broadly reflective of the target population in terms of age, school attendance, geographic background, and language use, participants may have been more familiar with mental health concepts and terminology than adolescents in the wider population. As such, the study may underestimate potential difficulties in understanding certain items or the broader concept of mental health measurement among adolescents with less exposure to these topics.

The absence of 13-year-old participants is a limitation as it reduces the representativeness of the sample across the full 13–17 age range and limits generalisability of findings to younger adolescents. No 13-year-olds were included in this study as none opted in to take part, despite the study being advertised to this age group. This is important because younger adolescents may differ in cognitive and linguistic development, which can affect how they interpret and articulate responses during Think Aloud tasks [58]. Their non-participation may reflect lower consent uptake or engagement among younger adolescents, a common issue in adolescent mental health research [59].

Finally, individual differences may influence a young person’s ability to think aloud [(e.g., 60; 61)], however, individual differences while thinking aloud are more prominent during tasks which cause a high cognitive load [(e.g., manually completing a task while thinking aloud; 61)], which was not present in our study. This was also addressed by including a relatively high number of participants.

Future research should aim to conduct think aloud interviews exploring the RCADS-47 with more diverse samples, this will enable any sub-cultural or demographic differences of item understanding to be identified. Identifications of further problems in different demographics will enable the RCADS-47 to be adapted further. Finally, the rephrased RCADS-47, as presented in this study, should be psychometrically tested in a large sample of Indian adolescents. If statistical testing reveals that the measure has met the acceptable criteria for the appropriate tests, cultural validity will have been established [18], increasing confidence in the use of the RCADS among Indian adolescents.