https://orcid.org/0000-0002-3903-4166
Figures
Abstract
Objectives
This study aimed to determine the efficacies of the Coping Camp app in reducing stress, depression, and anxiety and improving stress-coping behaviours and mental health wellbeing. Additionally, feasibility and acceptability of Coping Camp were evaluated.
Methods
In this unblinded cluster RCT, 540 participants from two high schools in China were randomly assigned to the Coping Camp intervention (n = 6 classes; 275 students) or treatment as usual (n = 5 classes; 265 students) at the class level. Coping Camp was an automated self-help app, consisting of 11 sessions delivered over 11 weeks, with primary outcomes including perceived stress, depression, anxiety, stress-coping behaviours, and mental health well-being. All outcomes were assessed at baseline, post-intervention (11 weeks), and follow-up (19 weeks), with efficacy analysed using linear mixed models and feasibility/acceptability measured by a 5-point Likert scale and qualitative feedback.
Results
At post-intervention and follow-up assessments, 75.4% and 81.7% of participants respectively attended. On average, participants logged in for 8.56 out of 11 sessions. Compared to the control group, the intervention group had significant reductions in levels of perceived stress (p = 0.01, d = 0.15 at T1; p < 0.001, d = 0.18 at T2), anxiety (p = 0.11; d = 0.08 at T1; p = 0.01; d = 0.13 at T2) and depression (p = 0.04, d = 0.11 at T1; p = 0.05, d = 0.10 at T2) but did not have a greater increase in stress-coping behaviours (p = 0.10 at T1; p = 0.97 at T2) or mental health wellbeing (p = 0.93 at T1; p = 0.08 at T2). The average ratings for each session were above 4, and qualitative feedback showed that most participants found the intervention to be “great,” “good,” and “useful.”
Citation: Zhou X, Edirippulige S, Jones A, Bai X, Smith AC, Bambling M (2023) The feasibility, acceptability and efficacy of an app-based intervention (the Coping Camp) in reducing stress among Chinese school adolescents: A cluster randomised controlled trial. PLoS ONE 18(11): e0294119. https://doi.org/10.1371/journal.pone.0294119
Editor: Chiedu Eseadi, University of Nigeria, Nsukka, NIGERIA
Received: March 23, 2023; Accepted: October 16, 2023; Published: November 27, 2023
Copyright: © 2023 Zhou et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: We are not able to share the data set in publicly available data repertoires, because it has been stated in the ethics application and participant information sheet that only aggregated data will be reported publicly. Depositing dataset publicly will violate the ethics approval requirements and the consent that participants gave us. As per ethical guidelines of the University of Queensland, all research data were kept in an online data management system—Research Data Manager (RDM). Only investigators have access to the data in this project. However, if it is available upon request to individual researcher who are interested. Here is the contact information for the University of Queensland Human Research Ethics Committee: Dr Tamara Periera Human Research Ethics Coordinator Phone: +61 7 344 31656 Email: humanethics@research.uq.edu.au Website for more contact information: https://www.uq.edu.au/research/contact.
Funding: XZ, UQ Research Training Scholarship, there is no grant number for this funding, https://scholarships.uq.edu.au/scholarship/graduate-school-scholarships-uqgss---includes-rtp, the sponsors did not play any role in the study design, data collection and analysis, decision to publish or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Adolescence is an important transitional period when adolescents experience diverse physical, psychological and social changes. The changes, including puberty, seeking independence, building peer relationships, and meeting academic and transition demands, all constitute acute or chronic stressors for adolescents. Stress, referring to the ‘condition or feeling that results when individuals perceive that the demands of a situation exceed their personal, psychological or social resources’ [1], has been proven by evidence to have a fundamental impact on adolescents. For example, stress is associated with maladaptive brain development in adolescence [2] and negative mental health outcomes such as anxiety [3], depression [4] and schizophrenia [5]. Stress is also reported to be associated with decreased academic performance [6] and school dropout [7]. Also, stress is reported to be related to conduct disorders such as substance abuse [8] and juvenile delinquency [9]. In order to reduce the incidence of these mental and conduct disorders, it is imperative to provide adolescents with stress management interventions.
China has an adolescent population of 146 million and 87% are enrolled in schools [10]. According to recent reports, nearly 90% of Chinese adolescents reported that they experienced high or very high levels of stress and nearly half (45.2%) said they spent two hours or more every day on their homework [6]. Apart from academic stress, Chinese adolescents are also reported to experience stress that arises from interpersonal relationships, and family issues [11]. Students also reported that these stressors have negative impacts on their studies and emotional wellbeing [12]. However, there are no formal mental health interventions to assist Chinese high school students manage their stress. The reasons are twofold: first, the mental health workforce in China is insufficient to meet the community demand [13]. Second, there is a significant disparity in the quantity of mental health services available in urban and rural areas and across regions in China [14].
Online mental health services may provide a solution to these issues of access and quality. Globally, mental health services have been increasingly delivered via internet such as videoconference, mobile apps and portable devices. Evidence suggest that these online mental health interventions enable patients who reside in remote areas to access services while saving time and money for traveling [15]. Online self-help mental health interventions can be delivered to school students and reach a large number of users thereby reducing the demand for human professionals. In the past decade, globally, online mental health interventions have been increasingly used to manage mental health issues among adolescents and have been proven by research to be effective [16]. For example, Ahmad et al [17] delivered an 8-week online intervention that was based on mindfulness to university students and found that the intervention could significantly reduce symptoms of stress, depression and anxiety as compared with the control group. Puolakanaho et al [18] delivered an online intervention that was based on acceptance commitment therapy to secondary school students and found that the intervention could significantly reduce stress levels. A systematic review conducted by Zhou et al [16] examined the evidence on online interventions for managing mental health issues among adolescents found that among 45 studies included, 43 were conducted in westernized countries, there are only two studies which tested the efficacies of online interventions among Chinese adolescents [19, 20]. One intervention was conducted in a university and the intervention was translated from MoodGYM that was developed in Australia [20]. The other study was conducted in Hongkong district and the intervention was translated from CATCH-IT that was developed in USA [19]. Both MoodGYM and CATCH-IT were online interventions delivering modularised cognitive behavioural therapy. Research has shown that Chinese adolescents experience and cope with stress in different ways from their western counterparts due to different cultural and societal factors [21]. For example, compared with their western counterparts, Chinese adolescents are more likely to suffer from high expectations to succeed in highly competitive academic environments. In addition, Chinese adolescents frequently used internalising stress as a coping strategy. Therefore, it is important to investigate whether the use of online stress management intervention could be effective in reducing stress among Chinese adolescents. However, to our best knowledge, there are no studies that investigate the efficacy of online interventions that are developed for Chinese adolescents in managing stress. Our study therefore aimed to test whether the use of a mobile app, named Coping Camp, which was developed to meet the specific needs and preferences of Chinese adolescents in high school settings, is effective. We also aimed to test whether the use of Coping Camp was acceptable in Chinese high school settings, and whether the use of Coping Camp was effective in reducing depression and anxiety, improving mental health wellbeing, decreasing the frequency of using maladaptive coping strategies and increasing the frequency of using adaptive coping strategies among Chinese adolescents. Additionally, we also assessed whether the intervention caused harm to participants.
Methods
We followed CONSORT-EHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile HEalth Applications and onLine TeleHealth) [22] when reporting the results of this randomized controlled trial.
Study design
This study was a two-arm cluster randomised controlled trial. Participants were randomised by classes (i.e., clusters) on a 1:1 ratio to either the intervention group or the control group (see Fig 1 for CONSORT diagram). As there were 11 classes in total, one more class was allocated to intervention group to benefit more students.
Notes: Data was collected by gathering participating students in an auditorium and providing them with time to complete the assessment. Missing participants fall into two categories: drop-out participants and absent participants. Drop-out participants occurred when they informed the intervention implementer of their withdrawal from the study. If they did not complete the post-intervention assessment (T1), they were also missing from the follow-up assessment (T2). Absent participants were those who were not present at school on the day of assessment. Therefore, participants who were missing from the post-intervention assessment (T1) might be present at T2 and completed the follow-up assessment (T2). Therefore, the number of lost participants at T2 were counted in comparison to the baseline participants.
Settings
The participants were recruited from two high schools in Mianyang City of Sichuan province in China. The recruitment was conducted in September 2021. One was a private high school and the other was a public high school. The schools were anonymised due to confidentiality. The private school had a student population of around 3400. The public school had a student population of around 6000. Mianyang City is a moderately sized city located in southwest China.
Participant recruitment
Participation in this study was voluntary. After consulting school administrators, only students who enrolled in grades 10 and 11 were available. The reason provided was that grade 12 students were preparing for the national college entrance examination, therefore, involving them was not practical. The recruitment strategies involved advertisement, posters, and announcements.
Participant eligibility criteria
Our inclusion criteria included: (1) own a smartphone and be permitted by parents and teachers to use the smartphone; (2) be enrolled in grades 10 and 11 of high school at the time of intervention. The exclusion criteria included: (1) students should not have suicidal ideation; (2) and should not be diagnosed with a mental disorder at the time of recruitment; (3) and not be taking psychiatric medications and (4) they should not be receiving regular mental health counselling services. The eligibility criteria were assessed via self-reported questionnaires.
Random allocation
The randomization was conducted by a research assistant who was independent of our research group. This research assistant was also blinded of which group was intervention group and which group was control group. An online randomizer (www.randomizer.org) was used to allocate classes into two groups.
Blinding
Given the nature of the intervention, it was not possible to blind either the participants or the intervention implementer. However, the outcome assessors and the researcher who conducted the randomization were blinded to the students’ condition.
Procedures
During the screening stage, 575 students interested in participating were invited by the first author to complete a short questionnaire to confirm their eligibility in the auditoriums at the schools. This took place on two separate weekdays during moral education class where students were engaged in non-academic activities (e.g., students watched news on TV or head teachers discussed daily affairs of the class). The participant information sheets were handed out to students to read information about the study, and they were given a chance to ask questions. After screening, 35 students were deemed ineligible (see Fig 1 for reasons) and 540 students from 11 classes were deemed eligible. These students were approached by the chief investigator, and consent forms were handed out for them and their parents to sign. All 540 students agreed to participate, and both they and their parents gave informed consent for the students to participate in the study. Intervention sessions were arranged once per week for 11 weeks during moral education classes. Each session lasted between 25–45 minutes to complete, there were no home assignments, however, the students were allowed to access the content of the app outside of intervention sessions.
Intervention condition
Participants assigned to the intervention group (IG) received a QR code to download the Coping Camp application, which had two versions: Android and IOS versions to suit different students. After downloading the Coping Camp, students were required to create a personal account, which they must use to log into the Coping Camp.
The intervention was based on Stress Inoculation Training (SIT) [23], a type of Cognitive Behavioural Therapy aimed at reducing stress. Although SIT has been frequently used to manage PTSD among veterans, it has been widely adopted in preventive and treatment interventions for stress management among nurses [24], teachers [25], and adolescents [26]. In recent years, SIT has been delivered via virtual reality technology and mobile phones and demonstrated potential efficacies [27, 28]. The standard SIT comprises three phases: the educational phase, skills training phase, and application phase. We modified the SIT to suit the Chinese culture and our adolescent population based on our previous studies with Chinese high school students and teachers [29]. Then, we invited Chinese academics and professionals to review the content of our SIT intervention, minor changes were made according to their comments. As a result, the modified SIT had 11 sessions. Modified SIT consisted same phases as standard SIT, the skill training phase included somatic skills (progressive relaxation and mindfulness), cognitive restructuring skills and behavioural skills (goal setting, time management, and problem-solving skills) and interpersonal skills. More details of the intervention can be found in our protocol: dx.doi.org/10.17504/protocols.io.n92ldmz69l5b/v1 [PROTOCOL DOI]. An app named Coping Camp was developed based on the modified contents of SIT. The app was in Chinese language and programmed by a Chinese company. To assess the initial feasibility and acceptability, we conducted a pilot test of the Coping Camp app among students prior to this trial, and bugs were identified and fixed.
The Coping Camp comprised 11 sessions (Fig 2), notification functions, three assessment and a discussion board. The 11 sessions were locked, and students could only access the next session after completing the previous one. The three assessments were also locked and only open at the time of each assessment. The discussion board was open during the intervention, and any participant could post or reply to other posts on the discussion board. The discussion board had two options, one named "Tell Us About Your Stress" and the other named "Share About Your Successful Coping Experiences." The aims of the discussion board were to provide a peer-support forum for students to discuss their stress, learn from and support each other, while still limiting the content to stress and coping. The discussion board was monitored by the chief investigator (XZ). The moderator only interacted with students on the discussion board under two circumstances: (1) when a question was directly asked to be answered by the moderator, and (2) when posts or replies were consulting other participants, the moderator would delete these posts or replies.
Control group
Participants who were allocated to a control group (CG) attended their regular moral education classes as arranged by the schools.
Measures
All the participants in the intervention or control group were assessed at three timepoints: baseline (T0; prior to intervention), post-intervention (T1; 11 weeks after the baseline), and follow-up (T2; 19 weeks after the baseline). The measures included demographic variables and study outcomes. Demographic variables were collected at baseline. We collected gender, year of birth, class, grade, name of the school, residence (rural or urban), and guardians for demographic information. Study outcomes were introduced in below sections. Data were collected via online questionnaires in the app and in the classrooms.
Perceived stress.
The primary outcome was perceived stress which was assessed by Perceived Stress Scale-10 Items (PSS-10). The PSS-10 is a self-reported scale comprising ten items used to evaluate respondents’ global level of perceived stress [30]. Respondents were asked to rate the frequency of certain feelings they experienced over the past month on a 5-point Likert Scale (1 = never, 2 = rarely, 3 = sometimes, 4 = often, 5 = always). The PSS-10 has been translated in Chinese and shown acceptable reliability and validity in both adults [31] and adolescents [32].
Depression and anxiety.
The Depression, Anxiety and Stress Scale (DASS-21) was used to measure symptoms of anxiety and depression as secondary outcomes. The DASS-21 is a 21-item scale that rates the degree to which each statement applies to participants over the past week on a four-point Likert scale from 0 to 3. Total scores range from 0 to 42, with higher scores indicating more severe symptoms. Subscale scores were also utilized in the study. The DASS-21 has been validated in Chinese adolescent populations and has satisfactory psychometric properties [33].
Mental health wellbeing.
Mental health wellbeing was measured using the four-item self-report instrument called the Outcome Rating Scale (ORS-4). The ORS-4 assesses the perceived change in three areas of client functioning: individual (or symptomatic) functioning, interpersonal relationships, and social role performance (work adjustment, quality of life). The ORS-4 has been found to have adequate reliability and validity [34], and the Chinese version of the ORS-4 has been recently validated and shown to be a reliable and valid measure for Chinese sub-clinical clients [35].
Coping behaviours.
Stress coping behaviours were measured using the short form of the Coping Inventory for Stressful Situations (CISS-SFC), which is a 21-item scale that measures task-oriented coping, emotion-oriented coping, and avoidance-oriented coping. Task-oriented coping was regarded as maladaptive coping, and emotion- and avoidance-oriented coping were regarded as maladaptive coping [36, 37]. The CISS-SFC has been validated and shown to be reliable among Chinese university students after being translated into Chinese [38].
Negative events.
To measure the negative effects of the intervention on a universal sample of school students, we defined negative events as any significant unfavorable changes perceived by participants in their life and study, rather than solely as significant changes in their mental condition during the intervention period. Serious negative events were defined as mortality, hospitalization, suicide, or suicide attempts. Short questionnaires with open-ended questions were used to measure negative events. Data related to negative events was collected at the end of the intervention period (T1).
Acceptability.
Acceptability of the intervention was measured through feedbacks at the end of each session which was assessed using a short satisfaction survey. The survey had two questions: the first was a quantitative question asking participants to rate the usefulness of Coping Camp on a 5-point scale, with 1 indicating "not useful at all" and 5 indicating "most useful." The second question was qualitative, asking participants to provide comments about each session.
Sample size calculation
According to a meta-analysis on internet interventions on stress reduction [39], the overall mean effect size for stress at post-test was Cohen’s d = 0.43 (95% CI 0.31–0.54). Small but significant effects were also found for depression (Cohen’s d = 0.34, 95% CI 0.21–0.48) and anxiety (Cohen’s d = 0.32, 95% CI 0.17–0.47). Therefore, effect sizes of d = 0.43 for perceived stress, d = 0.34 for depression, and d = 0.32 for anxiety were chosen. For a cluster randomized trial assuming the same values as for the original design, allowing for variation in cluster size up to a coefficient of variation of 5, an intra-cluster correlation up to 0.25, an individual autocorrelation as low as 0.5, and up to 10% drop out in each cluster, 5 clusters per group with an average size of 22 were needed to detect this effect (d = 0.43) with a power (1− β) of 0.80 at an alpha of 0.05, resulting in a total of 220 participants for all clusters across both groups.
Statistical analysis
Statistical analyses were conducted using software R version 4.2.2 (the R Foundation for Statistical Computing, Vienna, Austria). The accepted level of significance for this study was set at 95%. To determine baseline differences between the intervention group and control group, independent t-tests were performed on continuous baseline variables (e.g., age, PSS-10, DASS-21, ORS-4, CISS-SFC, MER), and Chi-squared tests were performed on categorical or nominal variables (e.g., gender, home location, guardian, etc.). To evaluate the efficacy of the intervention, intention-to-treat (ITT) analyses were conducted separately for each outcome [40]. The ITT analysis included data from the entire randomized sample. A repeated measures design with a linear mixed model (LMM) was used to investigate the effects of time and groups on outcome variables. The LMM was performed with time, group, time*group, gender, school, grade, guardian, and residence treated as fixed effects, while students and classes were treated as random effects. Covariates that are not significant will be removed from the model, therefore the main analyses will only include significant covariates. Cluster effects were examined by using regression model, where cluster (i.e., class) was used as a fixed effect, for each outcome at T1 and T2 separately, and the analysis results suggested that there were no significant cluster effects. In each linear mixed model, the school effects were assessed as a fixed effect. Upon analysis, these effects were found to be statistically insignificant in all models. Consequently, the final models did not incorporate the school effects. Additionally, given the large endpoints, p values (two-sided) will be adjusted using the Benjamini-Hochberg method to control for false discovery rate [41]. Last, we used raw scores rather than adjusting them for baseline difference in LMMs based on following reasons: for each baseline covariate presented in the table, we re-evaluated the p-values, taking into account adjustments for clustering, by employing a mixed effects model. This adjustment incorporated the treatment group as a fixed effect and the cluster as a random effect. Post-adjustment, only ORS-4 was statistically significant. All other variables were non-significant with the same randomization. This is likely to be attributed to the potential for false positives. Additionally, since the linear mixed model (LMM) introduces a random intercept for each class, baseline variations are expected to exert minimal influence on the results. Furthermore, additional analysis was performed for ORS-4, using the difference to baseline, and were similar to those of using the raw scores.
Missing participants and missing data handling
Data was collected by gathering participating students in an auditorium and providing them with time to complete the assessment. Therefore, missing participants fall into two categories: drop-out participants and absent participants. Drop-out participants occurred when they informed the intervention implementer of their withdrawal from the study. Except for one class (n = 48) students who withdrew from the study due to one subject teacher believed that the students should spend more time studying, after two weeks from the start of the intervention, there were no more students or classes who withdrew. In this case, they did not complete the post-intervention assessment (T1) or follow-up assessment (T2). Absent participants also occurred when they were not present at school on the day of assessment. Therefore, participants who were missing from the post-intervention assessment (T1) might be present at T2 and completed the follow-up assessment (T2). Therefore, the number of lost participants at T2 were counted in comparison to the baseline participants.
Missing data occurred in two different scenarios. The first scenario was when participants dropped out of the intervention or were not present on the day of assessment. At time point 1, there were 133 missing participants. Of those, 116 were from the intervention group, with 48 dropping out of the intervention and 68 not being present on the assessment day, while 17 were from the control group. At time point 2, there were 99 missing participants, with 17 from the control group and 82 from the intervention group. For the missing participants from the intervention group at time point 2, 48 had already dropped out at time point 1, and 34 were not present on the day of assessment at time point 2. Even though some participants did not complete all the time points, their data was still used in the analyses. The participants who dropped out did not differ significantly from those who remained in the trial regarding participant characteristics and study outcomes.
The second scenario was when a single participant skipped questions on a questionnaire such as PSS-10 or DASS-21. In this case, the missing data points were coded as NA and treated as missing in our analysis in R. Any participants with missing data for given outcomes were removed from the analysis of those outcomes. Missing data were not significantly associated with participants’ characteristics or study outcomes. To test the robustness of our results, we compared LMM results of including only those with complete data with results of including all the participants and found that the two results remained almost unchanged.
Ethical review
Ethics approvals were obtained from two institutions: the Human Ethics Office at The University of Queensland in Australia (approval number: 2021/HE000791) and the Research Ethics Office at Tianjin Normal University in China (approval number: 2021041901). Informed written consent forms were obtained offline from both the participants and their guardians as most of the participants were under the age of 18 at the time of the trial. Participation was voluntary, and all students were informed that their answers would remain confidential. Only the first author had access to identifiable information during and after data collection. All participants were anonymized prior to data analysis.
Trial registration
This randomised controlled trial was registered via Australian New Zealand Clinical Trials Registry (ANZCTR). Registration number: 380316. Link to the trial registration: https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=380316&isReview=true.
Results
Study participation
A flowchart of the study participants is shown in Fig 1. At the time of screening assessment in 11 school classes, a total of 613 students were present. Among them, 575 students (93.8%) agreed to participate. Of these, 540 students (88.1%) meet the inclusion criteria and participated in the study. Six classes consisting of 275 students were randomly allocated to the intervention group and 5 classes consisting of 265 students were allocated to the control group. Post-intervention assessment was completed in 159 participants (57.8%) in the intervention group and 248 participants (93.6%) in the control group. Follow up assessment was completed in 193 participants (70.2%) in the intervention group and 248 participants (93.6%) in the control group.
Sample characteristics
Table 1 presents the baseline characteristics for the study population. Baseline differences were found between intervention and control group participants for the following variables: the avoidance subscale of the CISS-SFC (t = -2.347, p = 0.019) and ORS (t = 3.123, p = 0.002), indicating that the intervention group had lower scores on avoidance coping skills (mean 19 vs. 20) and higher levels of mental health well-being (mean 26 vs. 24) than the control group at the start of the study. There were no other statistically significant differences found for demographic or outcome variables.
Program attrition and program use
During the 11-week program, the number of login times per student varied from 2 to 24 times (mean = 8.56, SD = 5.12). Each session lasted between 25 and 50 minutes.
Adherence
Adherence was defined as the frequency of sessions completed by students, ranging from 0 to 11. Out of 275 students, 19 students (6.9%) did not complete any sessions, 43 students (15.6%) completed 1–3 sessions, 92 students (33.5%) completed 4–7 sessions, and 121 students (44%) completed 8–11 sessions.
Intervention effects
Intent-to-treat analyses were conducted using linear mixed models (LMM) with random intercepts for students and classes to examine the effects of Coping Camp on all study outcomes. The final models for each study outcome are shown in Table 2, with results reported in the following sections, and Fig 3 summarizes the changes in all outcomes.
Stress (PSS-10).
The final mixed model for perceived stress, measured by PSS-10, included timepoint, group, and timepoint*group as fixed effects, along with the significant covariate gender. Results of the LMM analysis (Table 2) showed that participants in the intervention group had a highly significant reduction in stress levels compared with the control group (p = 0.01, d = 0.15) after the intervention, which was maintained at the 19-week follow-up (p < .001, d = 0.18). Fig 3a summarizes the changes in PSS-10 scores.
DASS-21, CISS-SFC, and ORS-4.
Fig 3b–3e summarizes the changes in scores of DASS-21 and its three subscales (depression, anxiety and stress). The LMM analysis (Table 2) showed that the participants in the intervention group had significant reductions as compared with those in the control group in scores of DASS-21 (p = 0.001, d = -0.14), depression subscale (p = 0.04, d = -0.11), stress subscale (p = 0.02, d = -0.14), but not in anxiety subscale (p = 0.11, d = -0.08) after the intervention. At the 19-week follow-up, the timepoint*group interaction effects were significant for DASS-21 total scale (p = 0.02, d = -0.12) and anxiety subscale (p = 0.01, d = -0.13). However, the timepoint*group interaction effects for DASS-21 depression subscale (p = 0.05, d = 0.11) and stress subscale were not significant at 19-week follow-up (p = 0.17, d = 0.10), indicating the treatment effect did not maintain at this timepoint.
Fig 3f–3i summarize the changes in CISS-SFC scores and its three subscales: emotion-oriented coping, task-oriented coping, and avoidance-oriented coping. The LMM analysis (Table 2) showed that the timepoint*group interaction effect was only significant for CISS-SFC avoidance subscale at post-intervention assessment (p = 0.02, d = −0.11), this indicated that the intervention was effective in increasing the frequency of using avoidance coping strategies.
Fig 3j summarizes the changes in scores of ORS-4. Results of LMM analysis (Table 2) showed that there was no significant timepoint*group interaction effect (p = 0.93, d = 0.00) after the intervention or at 19-week follow-up (p = 0.08, d = 0.09). This indicates that the participants in the intervention group did not achieve a significant change in their mental health well-being as compared with those in the control group.
Acceptability
Participants rated all 11 sessions on a 5-point scale, with average ratings ranging from 4.06 (SD = 0.89) to 4.49 (SD = 0.76), indicating high levels of perceived helpfulness. Qualitative feedback echoed these findings, with most participants reporting that Coping Camp was "great" (n = 69; 57%) and "helpful" (n = 21; 17%). However, a small number of participants reported difficulty mastering and relating to the app’s content (n = 11; 9%).
Negative events
Out of 221 intervention group participants and 278 control group participants, 56 (25.34%) and 129 (46.40%) respectively reported negative events in response to an open-ended question. Six students (2.71%) in the intervention group and 5 students (1.80%) in the control group did not respond. The difference was not significant (P = .91). The most common negative events reported were decreased monthly academic exam scores and conflicts with teachers, parents, and friends. No serious negative events were reported in either group.
Discussion
Principal findings
This study tested the efficacy of Coping Camp, a stress-managing app based on stress inoculation training (SIT), among Chinese school adolescents. The primary objective was to determine whether 11 weeks of use reduced stress levels. Secondary objectives included assessing acceptability in high school settings, reducing negative mental health outcomes, improving mental health wellbeing, and changing stress coping strategies. Results showed a significant reduction in levels of stress and depression. However, Coping Camp was not effective in improving mental health wellbeing or changing stress coping strategies.
Perceived stress
To our knowledge, this study was one of the first to demonstrate the stress-reducing effects of a mobile app based on stress inoculation training (SIT) in a universal sample of Chinese school adolescents. The results showed that the use of the Coping Camp significantly reduced the perceived stress level both at post-intervention and 19-week follow-up. The effect sizes were Cohen’s d = 0.15 at post-intervention and 0.18 at 19-week follow-up. The effect sizes were small as compared with previous online interventions among adolescents and young adults [42]. First, this intervention was arranged during moral classes which were followed by evening classes where students were engaged with homework and preparation for the next day’s classes, it is possible that students were not fully engaged with the intervention. Second, we recruited universal sample instead of a clinical sample, previous research has shown that interventions that recruited students with elevated symptoms are likely to produce larger effect sizes than those that recruited a universal sample [43]. Last, this is the exploratory study to investigate an app that was designed for Chinese high school students, further improvement needs to be made according to the user experience of participating students in order for the Coping Camp to generate larger effect size.
Depression and anxiety
Our study found that Coping Camp significantly reduced DASS-21 score at both post-intervention (P < .001) and 19-week follow-up (P < .01). Compared with the students in the controlled group, those who used Coping Camp decreased 2.69 at post intervention, which is similar to a previous study which tested the efficacy of online intervention among Chinese school adolescents [19]. Also, the effect sizes were Cohen’s d ranged from 0.08 to 0.13. The effect sizes were small compared with previous online interventions [19]. Similar reasons provided for perceived stress may provide an explanation.
Mental health wellbeing (ORS-4)
We found that Coping Camp was not effective in improving mental health well-being. This finding is similar to previous similar studies [44–46]. Outcome rating scale (ORS-4) was developed to measure the changes of global psychological distress and wellbeing during the psychotherapy sessions [34]. The possible reason why current intervention did not reduce the score of ORS-4 may be because current intervention was only designed for reducing stress and the scenarios might not suit to all the problems that these students confronted, according to the comments from students. Therefore, in order to achieve a more comprehensive change of students’ psychological wellbeing, it is recommended that stakeholders (e.g., teachers, parents etc.) could be involved and the scenarios should be more diverse.
Stress coping behaviours (CISS-SFC)
At the time of writing, there is limited number of studies examining the evidence on online and app-based mental health interventions’ effects on coping behaviour changes. Our study is one of the first to provide evidence on this subject and found that the use of an app-based mental health intervention, the Coping Camp, was not effective in changing stress coping strategies. This finding is in line with several studies that suggested face-to-face interventions were not effective in changing stress coping strategies among adolescents [47–49]. Several reasons were likely to contribute to the inefficacies. First, except for cognitive skills which took two sessions, other skills (e.g., mindfulness skills, progressive muscle relaxation skills, time management skills, problem-solving skills) are taught within one session which lasted for 30–40 minutes; and 30–40 minutes might not be enough for students to fully master each skill. In addition, except for the use during the intervention sessions, mobile phones were not available to participants during weekdays; the participants, therefore, did not have enough chance to practice and master these skills and apply these skills to their daily life. Second, the self-help nature of the intervention might be difficult for students too. Most of the students had no experience with coping skills training prior to the intervention. Future trials are warranted to investigate mobile app-based interventions’ effects on coping behaviour changes, which are critical for adolescents to maintain mental health wellbeing amid adverse life events [50]. For mental health apps providing coping strategy trainings, more guidance should be involved in the intervention to generate a larger effect. Third, according to behaviourism, reinforcements from environments are needed for new behaviours to sustain and consolidate, therefore, gamification features, such as incentives and rewards, may be involved in such apps to reinforce newly learnt coping behaviours. Also involving teachers/parents and creating a supportive classroom environment that encourages students to use positive coping behaviours would be beneficial.
Notably, we found that gender had significant effects on stress and coping behaviours. First, compared with male, females were under significantly more stress. Second, gender had significant effects on the coping styles. Compared with males, female participants used significantly more emotion- and avoidance-oriented coping strategies. Also, female participants used less task-oriented coping strategies. These findings are in line with previous research [51, 52]. We suggest future studies investigate gender-associated preferences in intervention development and investigate ways to improve female adolescents’ functional coping behaviours.
Acceptability
The results of this study showed that the Coping Camp was acceptable to Chinese high school students and the use of Coping Camp was feasible in Chinese high school settings. The rates of the Coping Camp were ranged from 4.05 to 4.49 across sessions, which was above the cut-off score of “useful”. This rating was similar to previous online interventions that was conducted in school settings [53]. Qualitative feedback reflected that the participants liked the app as they learnt skills and the app was helpful in reducing stress in their lives. This indicates that mental health apps need to be helpful in relieving stress to be acceptable.
Strength, limitations and implications for policymakers
To our best knowledge, this study is the first randomized controlled trial to investigate the efficacies of an app, the Coping Camp, which was developed for Chinese high school students and reflects the real-world effect of mobile intervention. This study showed that the use of the Coping Camp significantly reduced the stress levels, anxiety and depression among Chinese high school students and was feasible and acceptable among Chinese high school students. This has important implications for policymakers and educators. They can make use of internet resources (e.g., webpages and apps) as a mode to deliver psychoeducation and training about stress and coping where there is a lack of human resources in mental health professions. Several limitations should be noted. First, the sample was recruited from a moderate city in China, the results should be generated to big cities and remote/rural areas with caution. Second, as this is the first app that was developed for Chinese high school students, improvements should be made before it can be used for larger population. Third, participants were allocated randomly to two groups within the same school, which allowed for information sharing and cell phone exchanges. This might introduce contamination bias.
Conclusions
This study showed that the use of an app-based intervention, the Coping Camp, which was developed for managing stress among Chinese high school students was feasible, acceptable and effective, which provides an alternative for Chinese high schools where there are lack of mental health professionals and students experience excessive stress.
Supporting information
S2 File. Tianjin normal University scientific research ethics review form.
https://doi.org/10.1371/journal.pone.0294119.s002
(DOCX)
S1 Checklist. CONSORT 2010 checklist of information to include when reporting a randomised trial*.
https://doi.org/10.1371/journal.pone.0294119.s005
(DOC)
Acknowledgments
The authors acknowledge the statistical support received through the Research and Statistical Support Service, Faculty of Medicine, The University of Queensland, Australia.
References
- 1.
Lazarus RS. Psychological stress and the coping process. New York, NY, US: McGraw-Hill; 1966.
- 2. Romeo RD. The teenage brain: The stress response and the adolescent brain. Curr Dir Psychol Sci. 2013;22(2):140–5. pmid:25541572
- 3. McLaughlin KA, Hatzenbuehler ML. Stressful life events, anxiety sensitivity, and internalizing symptoms in adolescents. J Abnorm Psychol. 2009;118(3):659. pmid:19685962
- 4. Jayanthi P, Thirunavukarasu M, Rajkumar R. Academic stress and depression among adolescents: A cross-sectional study. Indian Pediatri. 2015;52(3):217–9. pmid:25848997
- 5. Gomes FV, Grace AA. Adolescent stress as a driving factor for schizophrenia development—a basic science perspective. Schizophr Bull. 2017;43(3):486–9. pmid:28419390
- 6. Ye L, Posada A, Liu Y. A review on the relationship between Chinese adolescents’ stress and academic achievement. New Dir Child Adolesc Dev. 2019(163):81–95. pmid:30614631
- 7. Dupéré V, Dion E, Leventhal T, Archambault I, Crosnoe R, Janosz M. High school dropout in proximal context: The triggering role of stressful life events. Child Dev. 2018;89(2):e107–e22. pmid:28369807
- 8. Charles NE, Mathias CW, Acheson A, Bray BC, Ryan SR, Lake SL, et al. Increased pre-and early-adolescent stress in youth with a family history of substance use disorder and early substance use initiation. J Youth Adolesc. 2015;44(10):1954–67. pmid:25788123
- 9. Jeon HS, Chun J. The influence of stress on juvenile delinquency: Focusing on the buffering effects of protective factors among Korean adolescents. Soc Work Public Health. 2017;32(4):223–37. pmid:28276895
- 10.
UNICEF China, UNFPA China. Population Status of Adolescents in China in 2015: Facts and Figures 2018 [https://www.unicef.cn/media/8856/file/2015%20Adolescent%20Population%20Status%20Facts&Figures.pdf.
- 11. Zhou X, Bambling M, Bai X, Edirippulige S. Chinese school adolescents’ stress experience and coping strategies: a qualitative study. BMC Psychol. 2023;11(1):1–15.
- 12. Liu X, Kurita H, Uchiyama M, Okawa M, Liu L, Ma D. Life events, locus of control, and behavioral problems among Chinese adolescents. J Clin Psychol. 2000;56(12):1565–77. pmid:11132571
- 13. Fang M, Hu SX, Hall BJ. A mental health workforce crisis in China: a pre-existing treatment gap coping with the COVID-19 pandemic challenges. Asian J Psychiatr. 2020;54:102265. pmid:32622031
- 14. Liang D, Mays VM, Hwang W-C. Integrated mental health services in China: challenges and planning for the future. Health Policy Plan. 2018;33(1):107–22. pmid:29040516
- 15. Hedman E, Ljótsson B, Lindefors N. Cognitive behavior therapy via the Internet: a systematic review of applications, clinical efficacy and cost–effectiveness. Expert Rev Pharmacoecon Outcomes Res. 2012;12(6):745–64. pmid:23252357
- 16. Zhou X, Edirippulige S, Bai X, Bambling M. Are online mental health interventions for youth effective? A systematic review. J Telemed Telecare. 2021;27(10):638–66. pmid:34726992
- 17. Ahmad F, El Morr C, Ritvo P, Othman N, Moineddin R, Team MVC. An eight-week, web-based mindfulness virtual community intervention for students’ mental health: randomized controlled trial. JMIR Ment Health. 2020;7(2):e15520. pmid:32074061
- 18. Puolakanaho A, Lappalainen R, Lappalainen P, Muotka JS, Hirvonen R, Eklund KM, et al. Reducing stress and enhancing academic buoyancy among adolescents using a brief web-based program based on acceptance and commitment therapy: a randomized controlled trial. J Youth Adolesc. 2019;48(2):287–305. pmid:30560515
- 19. Ip P, Chim D, Chan KL, Li TMH, Ho FKW, Van Voorhees BW, et al. Effectiveness of a culturally attuned Internet‐based depression prevention program for Chinese adolescents: A randomized controlled trial. Depress Anxiety. 2016;33(12):1123–31. pmid:27618799
- 20. Ren Z, Li X, Zhao L, Yu X, Li Z, Lai L, et al. Effectiveness and mechanism of internet-based self-help intervention for depression: The Chinese version of MoodGYM. Acta Psychol Sin. 2016;48(7):818–32.
- 21. Chen CF, Yang N, Sages J. A comparative investigation on the learning stress and extracurricular activities between senior high school students in China and Sweden. Journal of Huzhou University. 2014;36(12):97–102.
- 22. Eysenbach G, Consort EG. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res. 2011;13(4):e1923. pmid:22209829
- 23.
Meichenbaum D. Stress inoculation training. New York. 1985;304.
- 24. Mazlom SR, Darban F, Vaghei S, Modarres Gharavi M, Kashani Lotfabadi M, Shad M. The effect of Stress Inoculation Program (SIP) on nurses Perceived stress in psychiatric wards. Evid Based Care. 2012;2(1):35–44.
- 25. Cecil MA, Forman SG. Effects of stress inoculation training and coworker support groups on teachers’ stress. J Sch Psychol. 1990;28(2):105–18.
- 26. Kiselica MS, Baker SB, Thomas RN, Reedy S. Effects of stress inoculation training on anxiety, stress, and academic performance among adolescents. J Couns Psychol. 1994;41(3):335.
- 27. Serino S, Triberti S, Villani D, Cipresso P, Gaggioli A, Riva G. Toward a validation of cyber-interventions for stress disorders based on stress inoculation training: a systematic review. Virtual Real. 2014;18(1):73–87.
- 28.
Feindler EL, Fremouw WJ. Stress inoculation training for adolescent anger problems. Stress reduction and prevention. Boston, MA: Springer; 1983. p. 451–85.
- 29.
Zhou X, Edirippulige S, Bai X, Bambling M, editors. Chinese adolescents’ experience and perception of an app-based stress inoculation training intervention for stress management: a qualitative study. The 22nd Successes and Failures in Telehealth Conference (SFT-22) and the 14th Annual Meeting of the ATHS; 2022; Brisbane, QLD, Australia.
- 30.
Cohen S. Perceived stress in a probability sample of the United States. The social psychology of health. Thousand Oaks, CA, US: Sage Publications, Inc; 1988. p. 31–67.
- 31. Yang TZ, Huang HT. An epidemiological study on stress among urban residents in social transition period. Zhonghua liu xing bing xue za zhi = Zhonghua liuxingbingxue zazhi. 2003;24(9):760–4. pmid:14521764
- 32. Liu X, Zhao Y, Li J, Dai J, Wang X, Wang S. Factor Structure of the 10-Item Perceived Stress Scale and Measurement Invariance Across Genders Among Chinese Adolescents. Frontiers in psychology. 2020;11:537. pmid:32328009
- 33. Mellor D, Vinet EV, Xu X, Mamat NH, Richardson B, Román F. Factorial Invariance of the DASS-21 Among Adolescents in Four Countries. Eur J Psychol Assess. 2015;31(2):138–42.
- 34. Miller SD, Duncan BL, Brown J, Sparks JA, Claud DA. The outcome rating scale: A preliminary study of the reliability, validity, and feasibility of a brief visual analog measure. J Brief Ther. 2003;2(2):91–100.
- 35. She Z, Sun QW, Jiang GR. Reliability and validity of Chinese version of Outcome rating scale. Chin J Clin Psychol. 2017.
- 36. Chao RCL. Managing stress and maintaining well‐being: Social support, problem‐focused coping, and avoidant coping. J Couns Dev. 2011;89(3):338–48.
- 37. Endler NS, Parker JDA. Assessment of multidimensional coping: Task, emotion, and avoidance strategies. Psychol Assess. 1994;6(1):50.
- 38. Li C, Liu Q, Hu T, Jin X. Adapting the short form of the Coping Inventory for Stressful Situations into Chinese. Neuropsychiatr Dis Treat. 2017;13:1669–75. pmid:28721048
- 39. Heber E, Ebert DD, Lehr D, Cuijpers P, Berking M, Nobis S, et al. The Benefit of Web- and Computer-Based Interventions for Stress: A Systematic Review and Meta-Analysis. J Med Internet Res. 2017;19(2):e32. pmid:28213341
- 40. Ranganathan P, Pramesh C, Aggarwal R. Common pitfalls in statistical analysis: intention-to-treat versus per-protocol analysis. Perspect Clin Res. 2016; 7 (3): 144–6. pmid:27453832
- 41. Benjamini Y, Hochberg Y. Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. J R Stat Soc, B: Stat. 1995;57(1):289–300.
- 42. Davies EB, Morriss R, Glazebrook C. Computer-delivered and web-based interventions to improve depression, anxiety, and psychological well-being of university students: a systematic review and meta-analysis. J Med Internet Res. 2014;16(5):e3142. pmid:24836465
- 43. van Loon AWG, Creemers HE, Beumer WY, Okorn A, Vogelaar S, Saab N, et al. Can schools reduce adolescent psychological stress? A multilevel meta-analysis of the effectiveness of school-based intervention programs. J Youth Adolesc. 2020;49(6):1127–45. pmid:32034632
- 44. Farrer LM, Gulliver A, Katruss N, Fassnacht DB, Kyrios M, Batterham PJ. A novel multi-component online intervention to improve the mental health of university students: Randomised controlled trial of the Uni Virtual Clinic. Internet Interv. 2019;18:100276. pmid:31890625
- 45. Kenny R, Fitzgerald A, Segurado R, Dooley B. Is there an app for that? A cluster randomised controlled trial of a mobile app–based mental health intervention. Health Informatics J. 2020;26(3):1538–59. pmid:31702409
- 46. Dear BF, Fogliati VJ, Fogliati R, Johnson B, Boyle O, Karin E, et al. Treating anxiety and depression in young adults: A randomised controlled trial comparing clinician-guided versus self-guided Internet-delivered cognitive behavioural therapy. The Australian and New Zealand journal of psychiatry. 2018;52(7):668–79. pmid:29064283
- 47. Lang C, Feldmeth AK, Brand S, Holsboer-Trachsler E, Pühse U, Gerber M. Effects of a physical education-based coping training on adolescents’ coping skills, stress perceptions and quality of sleep. Phys Educ Sport Pedagog. 2017;22(3):213–30.
- 48. Dowling K, Simpkin AJ, Barry MM. A Cluster Randomized-Controlled Trial of the MindOut Social and Emotional Learning Program for Disadvantaged Post-Primary School Students. J Youth Adolesc. 2019;48(7):1245–63. pmid:31004264
- 49. Garcia C, Pintor J, Vazquez G, Alvarez-Zumarraga E. Project Wings, a coping intervention for Latina adolescents: a pilot study. West J Nurs Res. 2013;35(4):434–58. pmid:21593284
- 50. Shaheen F, Alam MS. Psychological distress and its relation to attributional styles and coping strategies among adolescents. J Indian Acad Appl Psychol. 2010;36(2):231–8.
- 51. Matud MP. Gender differences in stress and coping styles. Pers Individ Dif. 2004;37(7):1401–15.
- 52. Tamres LK, Janicki D, Helgeson VS. Sex differences in coping behavior: A meta-analytic review and an examination of relative coping. Pers Soc Psychol Rev. 2002;6(1):2–30.
- 53. Levin ME, Pistorello J, Seeley JR, Hayes SC. Feasibility of a prototype web-based acceptance and commitment therapy prevention program for college students. J Am Coll Health. 2014;62(1):20–30. pmid:24313693