LGBTQ+ community’s higher susceptibility to worse mental health outcomes and more help-seeking barriers compared to the cis-heterosexual population. Despite the LGBTQ+ population facing higher mental health risks, there has been a dearth of research focusing on developing tailored interventions targeting them. This study aimed to assess the effectiveness of a digital multicomponent intervention in promoting help-seeking for mental health issues in LGBTQ+ young adults.
Methods and findings
We recruited LGBTQ+ young adults aged between 18 and 29 who scored moderate or above on at least 1 dimension of the Depression Anxiety Stress Scale 21 and did not have help-seeking experiences in the past 12 months. Participants (n = 144) were stratified by gender assigned at birth (male/female) and randomly allocated (1:1 ratio) to the intervention or active control parallel condition by generating a random number table, so they were blinded to the intervention condition. All participants received online psychoeducational videos, online facilitator-led group discussions, and electronic brochures in December 2021 and January 2022, with the final follow-up in April 2022. The contents of the video, discussion, and brochure are help-seeking for the intervention group and general mental health knowledge for the control group. The primary outcomes were help-seeking intentions for emotional problems and suicidal ideation and attitudes toward seeking help from mental health professionals at the 1-month follow-up. The analysis was performed by including all participants based on their randomized group regardless of adherence to the protocol. A linear mixed model (LMM) was used for analysis. All models were adjusted for baseline scores. Chinese Clinical Trial Registry: ChiCTR2100053248.
A total of 137 (95.1%) participants completed a 3-month follow-up, and 4 participants from the intervention condition and 3 from the control condition did not complete the final survey. Compared with the control group (n = 72), a significant improvement was found in help-seeking intentions for suicidal ideation in the intervention group (n = 70) at post-discussion (mean difference = 0.22, 95% CI [0.09, 0.36], p = 0.005), 1-month (mean difference = 0.19, 95% CI [0.06, 0.33], p = 0.018), and 3-month follow-ups (mean difference = 0.25, 95% CI [0.11, 0.38], p = 0.001). There was also a significant improvement in the intervention condition on the help-seeking intention for emotional problems at 1-month (mean difference = 0.17, 95% CI [0.05, 0.28], p = 0.013) and 3-month follow-ups (mean difference = 0.16, 95% CI [0.04, 0.27], p = 0.022) compared with the control group. Participants’ depression and anxiety literacy and help-seeking encouragement related knowledge in intervention conditions showed significant improvements. There were no significant improvements in actual help-seeking behaviors, self-stigma toward seeking professional assistance, depression, and anxiety symptoms. No adverse events or side effects were observed. However, the follow-up time point was limited to 3 months which might not be long enough for drastic mindset and behavioral changes in help-seeking to occur.
The current intervention was an effective approach in promoting help-seeking intentions, mental health literacy, and help-seeking encouragement-related knowledge. Its brief yet integrated intervention format could also be utilized in treating other imminent concerns confronted by LGBTQ+ young adults.
Why was this study done?
- Members of the LGBTQ+ community, compared to their cis-heterosexual counterparts, are less likely to seek help for mental problems due to minority stress, stigma, and difficulty in acquiring sexual- and gender-affirmative care.
- Findings from past studies suggest that psychoeducational videos, facilitator-led group discussions, and help-seeking brochures are all key influential factors for individuals’ help-seeking.
- Nevertheless, few studies have investigated the effectiveness of multicomponent approaches and digital online-based interventions targeted at LGBTQ+ young adults’ help-seeking behaviors.
What did researchers do and find?
- We conducted a single-blind parallel-group randomized control trial to evaluate the efficacy of a digital online-based multicomponent intervention in promoting help-seeking among LGBTQ+ young adults with mental health problems.
- We included 144 participants recruited between December 3, 2021 and January 12, 2022 through online platforms who had no help-seeking experience within the past 12 months.
- We randomized 72 participants to the help-seeking promoting condition and compared them to 72 who received general health knowledge condition. The study was conducted over 5 months.
- Our findings indicated that an integrated intervention that combines psychoeducational videos, facilitator-led group discussions, and help-seeking brochures was effective in improving LGBTQ+ young adults’ general help-seeking intentions.
What do these findings mean?
- Our study offers preliminary evidence that digital interventions can improve help-seeking for LGBTQ+ young adults with mental health difficulties.
- Integration of digital interventions by policymakers to care provided to LGBTQ+ young adults may help to improve help-seeking and mental health outcomes on a wider societal level.
Citation: Han M, Wang Y, Zhang Y, Wang Y, Ou J, Ren D, et al. (2023) A multicomponent digital intervention to promote help-seeking for mental health problems and suicide in sexual and gender diverse young adults: A randomized controlled trial. PLoS Med 20(3): e1004197. https://doi.org/10.1371/journal.pmed.1004197
Academic Editor: Charlotte Hanlon, Addis Ababa University / King’s College London, ETHIOPIA
Received: June 7, 2022; Accepted: February 10, 2023; Published: March 6, 2023
Copyright: © 2023 Han et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: This work was supported by Research Fund of Vanke School of Public Health, Tsinghua University (No 100009001 to RC). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: AHSQ, Actual Help-Seeking Questionnaire; ATSPPH-SF, Attitudes Toward Seeking Professional Psychological Help Scale Short Form; DASS, Depression Anxiety and Stress Scale; GHSQ, General Help-Seeking Questionnaire; GLMM, generalized linear mixed model; HSERK, Help-Seeking Encouragement Related Knowledge Scale; IPT, Interpersonal Psychotherapy; LMM, linear mixed model; MHP, mental health professional; QR, quick response; SSOSH, Self-Stigma of Seeking Help; WHO, World Health Organization
The LGBTQ+ population is at greater risk than their cis-heterosexual counterparts for adverse mental health outcomes, including depression, anxiety disorders, and suicidality [1–5]. Such disparities are driven by stressors unique to the LGBTQ+ population [4,6,7]. For example, among various stressors, the family pressure faced by LGBTQ+ young people in China, exemplified in the parental generation’s implicit expectation of their LGBTQ+ offspring to conform to culturally ideal normative (e.g., heterosexual marriage and procreation), could be deemed as the most prevalent and pervasive, especially considering the key role of filial piety in traditional Chinese culture [8,9].
The minority stress model accentuates that the barriers resulting in the LGBTQ+ population’s reluctance to help-seeking can be categorized into distal and proximal stressors [6,10,11]. Findings from previous research indicate that distal stressors that heavily impact LGBTQ+ individuals’ externally based experiences contain the fear of being perceived negatively and the fear of being rejected by mental health professionals (MHPs) during treatments due to their implicit biases towards the LGBTQ+ community and their incompetency in offering LGBTQ+ affirmative care [6,10,12]; proximal stressors that profoundly influence LGBTQ+ individuals’ internally based experiences, in turn, are gradually shaped by abovementioned distal stressors and are constituted by over-reliance on self-coping, frightful perception of mental health conditions, and diffidence or insecurity when facing MHPs during treatments [6,10,12,13]. The combination of distal and proximal stressors contributes to the help-seeking stigma among LGBTQ+ individuals, manifested in their low help-seeking intentions and negative help-seeking attitudes (e.g., distrust in MHPs’ ethics and promise of confidentiality that is based on no solid evidence, one’s generalized prejudice, or hearsay) . Besides the interpretation formulated based on the minority stress model, a number of studies also reiterated specific barriers that LGBTQ+ individuals face, such as the lack of financial resources for transgender individual’s gender transition or for LGB individuals’ mental health services, that could lower their help-seeking intentions, worsen their help-seeking attitudes, and reduce the likelihood of actual help-seeking behaviors [15,16].
To our best knowledge, despite much research on promoting help-seeking intentions, improving help-seeking attitudes, and encouraging actual help-seeking behaviors in the cis-heterosexual population, there is little research targeting the LGBTQ+ population that utilizes an integrated approach with the aim to promote all 3 outcomes, leaving a critical gap in the literature. Specifically, 1 randomized clinical trial that centered on promoting LGBTQ+ individuals’ help-seeking was conducted in a German setting, with its emphasis on difficulties related to “coming out” . The study specifically used narratives from LGBTQ+ individuals who have been through the “coming-out” phase with the aim of alleviating their participants’ suicidal ideation, promoting help-seeking intention, reducing hopelessness, and promoting coping towards identity challenges. The study found that video narrative was effective in alleviating suicidal ideation and promoting help-seeking intentions (only with those who identify with the video), but the effects were short-lived. Another study that was centered on promoting help-seeking in the LGBTQ+ population was a randomized controlled trial that tested the effects of a game-based intervention, and the results showed, despite the intervention lowered the frequency of maladaptive behaviors (e.g., binge drinking and marijuana use), the study was not powered to find significant effects for help-seeking intentions, help-seeking self-efficacy, and help-seeking behaviors . Furthermore, the target sample of the study included all LGBTQ+ individuals who have experienced bullying/cyberbullying victimization and was not specified on LGBTQ+ individuals who have mental health concerns, as it was not shown in its eligibility criteria .
It is an important objective to improve LGBTQ+ young adults help-seeking for mental health conditions. It is equally important to improve their attitude and intention toward help-seeking from both professionals and the general population . Seeking help from professionals can effectively improve the mental health distress of the LGBTQ+ population and prevent suicide risk and self-injury. Nevertheless, considering (1) the LGBTQ+ population would often experience a strong stigma when seeking help for mental health issues; and (2) the lack of affirmative professionals specialized in LGBTQ+ care in China, it is difficult to seek professional help in many cases . When professionals are not available, seeking help from friends, family members, and other parties in the general population can also help to lower suicide risks and mental health concerns . Therefore, the present study shared a similar goal of nourishing LGBTQ+ young adults by improving the attitude or intention to seek help from both professionals and non-professional populations. Nevertheless, the target population and the intervention format between the previous and the present study were different. Compared to the previous study’s general LGBTQ+ population, the target population in the present study consisted of LGBTQ+ young adults with mental health conditions who had no help-seeking experience within the past 12 months, signaling increased clinical significance. Furthermore, regarding the intervention format, compared to the previous study’s sole concentration on utilizing self-disclosure videos, the present study focused on interaction and employed an integrated approach that was fully online and consisted of not only psychoeducational videos but also group discussions and help-seeking brochures.
The World Health Organization (WHO) postulates that mobile technology health interventions are featured by the effective utilization of mobile technologies to meet the unmet needs of traditional health service delivery . As previously noted, past research suggests that the LGBTQ+ population is disadvantaged in treatments that they need, often being restrained by limited financial resources and reduced availability of both medical and mental health services [23–25]. Therefore, we utilized the digital online-based format (i.e., the entire intervention was conducted via a video conferencing platform that incurs no costs for participants) to conduct our intervention. Past studies have demonstrated the efficacy of utilizing remote online conferencing as an intervention format to increase adherence, promote engagement, and reduce dropout rates [26,27]. In addition, the online-based format has greater accessibility and extensive outreach compared to traditional face-to-face intervention delivery [26–28]. Previous research has shown that online help-seeking interventions effectively promoted help-seeking outcomes in the cis-heterosexual population, resulting in a gap in the literature that the current study aims to fill .
Given the absence of research in promoting the LGBTQ+ population’s help-seeking intentions and attitudes using a digital online-based multicomponent approach, the present study sought to test the potential efficacy of this intervention. We employed a control condition in which participants would undergo the same intervention process with the same online format and time length while no mental help-seeking contents (focus on sleep hygiene, healthy diet, and exercise) in the intervention condition were involved. This study examined the efficacy of the digital online-based multicomponent help-seeking intervention targeting the LGBTQ+ population by comparing results of primary outcomes (i.e., help-seeking intention and attitude) and secondary outcomes (i.e., actual help-seeking behaviors, depression, anxiety literacy, help-seeking stigma, and help-seeking encouragement-related knowledge) from participants in the intervention condition against the control condition.
We conducted this single-blind parallel-group superiority randomized controlled trial to demonstrate the effectiveness of a brief digital online-based multicomponent intervention to promote help-seeking in LGBTQ+ young adults. The present study is reported in accordance with the Consolidated Standards of Reporting Trials (CONSORT) guideline (S1 CONSORT Checklist). This trial’s protocol was registered at the Chinese Clinical Trial Registry  (ChiCTR; identifier: ChiCTR2100053248) and has been published elsewhere .
Study design and participants
Recruitment was online, with recruitment posters being released through multiple social media platforms such as WeChat, QQ, Dou ban, and Sina Weibo, which are popular in China. By scanning the quick response (QR) code on the advertisement poster, participants who were willing to join the study answered the screening questionnaire and provided their contact numbers. Participants aged between 18 and 29 years old who self-identified as a member of the LGBTQ+ community were eligible to participate if they were living in China, had stable internet connections, scored moderate or above on at least 1 dimension (cut-off values were 14 points for depression, 10 points for anxiety, and 19 points for stress) of the Depression Anxiety Stress Scale 21 [32,33], and did not have help-seeking experiences in the past 12 months (asked by a single question that whether they had sought for professional help in the past 12 months).
Participants were excluded if they met one or more of the following criteria: had a history of being diagnosed with psychotic disorder, such as schizophrenia; have had suicidal attempt(s) in the last 6 months; currently have severe suicide ideations; could not complete all assessments and intervention procedures in a quiet, undisturbed space; or had help-seeking experience(s) from MHPs in the past 12 months. Participants have a one-to-one telephone conversation with a researcher to confirm whether they meet the inclusion and exclusion criteria.
Participants who met the study criteria went through the one-to-one consent-informing phase with one of the researchers via phone again. The electronic consent was sent to participants, and pictures of the handwritten signed consent or e-signature consent were returned to the researchers. Participants were recruited in 2 time periods: 80 participants were enrolled in December 2021, and 64 participants were enrolled in January 2022. All enrolled participants provided informed consent in written format. Participants have the right to withdraw their informed consent without reason at any time during the intervention and follow-up period, and their data will not be used in the analysis.
Randomization and masking
First, a random number table was generated by a researcher who was not involved in the intervention using IBM SPSS Statistics version 26.0. Each participant was assigned a unique number in the order of enrolling in the study. Then, the unique number of participants was separated from other information and sent to the researcher who generated the random number table, and this researcher, using the random number table, assigned participants to intervention or control condition. In this process, this researcher will not be exposed to other personal information of participants. Even though the recruitment of participants is divided into 2 periods, because the unified random number table generated in advance is used, the randomization process was guaranteed to be single-random.
A total of 144 participants who signed the informed consent form were stratified by gender assigned at birth (male/female) and then randomly allocated (1:1) to the experimental or control condition. Participants were masked to their group assignment, although they were informed of the existence of another group. The unified numbering procedure for all participants was used to minimize the possibility of participants knowing their groups according to the assigned number. Due to the nature of the present study, masking the study group facilitators was impossible. Nevertheless, we minimized the experimenter effect by having 3 group facilitators assigned to the experimental and control conditions, respectively.
Following randomization, participants were asked to complete the online baseline questionnaire and to choose their time slots to participate in the group discussion based on their schedules. Researchers then grouped participants according to their selected time slots. Each group, consisting of 4 to 6 participants, was assigned 1 researcher as the group facilitator to help participants complete all the intervention procedures. The entire intervention was conducted on an online video conferencing platform (Tencent Meeting, a conference software with functions very similar to ZOOM). Each participant received the encrypted online meeting link of one’s group and was notified to attend the meeting as scheduled. Due to the sudden changes in the schedules or dropouts, there were 2 groups that had only 3 participants.
This digital online-based intervention has incorporated 3 key therapeutic components, including (1) online animated psychoeducational videos (S1 Fig); (2) online facilitator-led group discussions (S2 Fig); and (3) electronic help-seeking brochures (S3 Fig/S4 Fig), with the efficacy of each proven by previous studies [34–38]. More specifically, animated psychoeducational videos have been shown to be more tone-neutral and acceptable than traditional psychoeducational videos [34,35,38]; according to the Interpersonal Psychotherapy (IPT) theory, facilitator-led group discussions have been recognized as an effective way to induce in-group empathy, promote attitude change, and advance positive thinking by facilitating group members’ self-disclosures ; the contents included in the help-seeking brochures were help-seeking encouragement, references to LGBTQ+ affirmative resources, and contact information of these resources, aligning with findings from previous research .
The current intervention was developed by the study team. Regarding the intervention videos, 5 modules in the intervention video were all presented from the perspective of LGBTQ+ populations, and the characters used were LGBTQ+ friendly. The intervention video also contains LGBTQ+-friendly doctors and practitioners in the psychology and psychiatry fields and 2 role models from the LGBTQ+ community (1 homosexual and 1 transgender). Regarding facilitator-led group discussions, structured interview guidelines were created by the supervisor of the project and were printed out during group discussions. During facilitator-led group discussions, 3 facilitators acquired participants’ verbal consent and introduced confidentiality to them. The help-seeking brochure was graphically designed by one of the research assistants. Its content included LGBTQ+-friendly knowledge and help-seeking resources. As mentioned in our study protocol , all materials prepared in the study were discussed and approved in the focus groups. The focus groups included 4 psychiatrists (average year working with LGBTQ+ was 3.6 years), 2 psychologists (average year working with LGBTQ+ was 7 years), 1 sociologist (average year working with LGBTQ+ was 3 years), and 5 LGBTQ+-friendly psychological consultants and psychotherapists (average year working with LGBTQ+ was 4 years). In addition, other researchers from UNSW Black Dog Institute were also involved in the development of the intervention. The development of the current intervention was based on previous research for the Chinese population [36,39]. For detailed descriptions of intervention contents for the experimental and the control conditions, please refer to our published study protocol .
Regarding the intervention process, group facilitators delivered this brief digital online-based multicomponent intervention by leading participants in completing the following 3 essential steps: (1) watching the entire 15-min intervention video (i.e., videos in both conditions were of the same length) with other group members in the online conference room; (2) actively participating in the structured discussions regarding intervention video and electronic psychoeducational brochure with other participants in the online conference room; and (3) freely accessing and reviewing the intervention video and electronic brochure post-discussion via web link given.
All 3 facilitators of the current study had a master’s degree in psychology and received 7 times internal training (2 to 3 h for each training) from 2 senior researchers (a doctoral degree in psychology and a doctoral degree in psychiatry research) before the start of the study (see investigators training section in appendix). The intervention process was recorded and checked to ensure intervention fidelity. Two senior researchers listened to the recordings of the intervention procedure independently and evaluated the fidelity of facilitators. The fidelity evaluation using a list developed by the research team, which included 6 items assessing video fluency, facilitators’ involvement, facilitators’ friendliness, participants’ activity, participants’ emotional involvement, and the overall evaluation of the process. Each item scored 0 to 5, using the average score of 6 items to evaluate the completion of a group discussion. Two senior researchers each listened to 6 recordings, including 3 intervention group recordings and 3 control group recordings from 3 facilitators, respectively. The average fidelity scores given by the 2 senior researchers were from 4.58 to 4.67 (full score of 5 points).
Intervention videos for both conditions were safely stored in a password-protected online server at Wenjuanxing, a Chinese online survey platform (i.e., similar to Qualtrics.com but can store videos and contents for guest viewing and downloading through the online address and password sent out by the researchers), during the course of the study. The electronic psychoeducational brochures for both conditions were sent out by researchers to participants directly for them to save and view at any moment during their free time. After group discussion and a brief introduction of the brochure, participants in the experimental and the control conditions were asked to complete an online post-discussion questionnaire. Participants were also asked to complete online questionnaires at 1-month and 3-month follow-ups. If participants did not reply within 5 days of receiving the online questionnaire link, they would receive a reminder text asking them to complete the questionnaire as soon as possible. If participants still did not reply within 2 days after the reminder, they would be marked as lost to follow up at this time point. Participants in the control condition also received the intervention video and psychoeducational brochure same as the intervention condition after the 3-month follow-up.
To ensure the safety of participants, we have developed a directory consisting of a list of psychiatric hospitals and their appointment registration methods in many cities in China, online and offline psychological intervention resources, and a list of the main Chinese crisis helplines. For participants who were excluded due to psychotic disorder or high suicide risk in the recruitment stage, we provided them with the directory and advised them to seek professional help. For participants involved in this study, whether in the intervention condition or control condition, the end of the brochures they received was also attached with the same directory mentioned above so they can get professional help at any time through this information if they need it. Ethical approval was granted by the Institution Review Board of Tsinghua University (Approval No. 20210146).
The primary outcomes were participants’ changes in general help-seeking intentions for emotional problems and suicidal ideation and attitudes toward seeking help from MHPs between baseline and 1-month follow-up. The general help-seeking intentions were measured by the General Help-Seeking Questionnaire (GHSQ), the validity of which in Chinese participants had been validated by previous research . GHSQ consisted of 22 items divided into 2 subscales (11 items for emotional problems and 11 items for suicidal ideation) and was scored on a 4-point Likert scale, with a higher score indicating a stronger intention to seek help. Participants’ attitudes toward seeking help from MHPs were measured by the Attitudes Toward Seeking Professional Psychological Help Scale Short Form (ATSPPH-SF), the validity of which had also been validated in Chinese participants by previous research . The validated Chinese version of the ATSPPH-SF consisted of 10 items on a 4-point Likert scale, with a higher score indicating a more positive attitude toward seeking professional help . Since GHSQ and ATSPPH-SF measured different aspects of help-seeking for mental health issues, and both are the focus of intervention in the current study, we defined that they are parallel primary outcomes.
Participants’ changes in general help-seeking intentions for emotional problems and suicidal ideation and attitudes toward seeking help from MHPs between baseline and post-discussion and 3-month post-intervention were defined as the secondary outcomes. Other secondary outcomes included participants’ actual help-seeking behaviors measured by the Chinese validated version of the Actual Help-Seeking Questionnaire (AHSQ) , participants’ depression and anxiety literacy measured by the Depression and Anxiety Literacy Questionnaire (D-A-Lit) [42,43], participants’ self-stigma toward seeking professional assistance measured by the Chinese validated version of Self-Stigma of Seeking Help (SSOSH) Scale , and participants’ help-seeking encouragement-related knowledge (i.e., knowledge in encouraging others for help-seeking) measured by the Chinese validated version of the Help-Seeking Encouragement-Related Knowledge Scale .
Since the GHSQ and AHSQ have the “not applicable” option, these 2 variables used the average value of valid items for analysis, while other variables used the total score. According to previous research, we did the additional analysis of the GHSQ by dichotomizing responses into “0” for “Highly unlikely” and “Unlikely,” and “1” for “Highly likely” and “Likely” .
Based on previous research, moderate effect sizes of help-seeking intentions (Cohen’s d = 0.53) and help-seeking attitudes (Cohen’s d = 0.58) were expected for the present study’s primary outcomes . The sample size was conservatively estimated (n = 144), assuming 20% attrition and a 1:1 allocation ratio during randomization , which would provide 80% power to detect an effect size of 0.53 at the α level of 0.05 for each primary outcomes.
Analyses for both primary and secondary outcomes were done from the following time points, respectively: (1) post-discussion; (2) 1-month follow-up; and (3) 3-month follow-up, with the exception of actual help-seeking behaviors of participants, which did not include the post-discussion measurement time point. The analysis was performed by including all participants based on their randomized group regardless of adherence to the protocol. The missing data was not processed since it is less than 5%. We used linear mixed model (LMM) to analyze the difference between the intervention effect on the primary outcomes and secondary outcomes, with the baseline score included as a covariate. The fixed effects included intervention, time of assessment, and interactions. The random intercepts of participants were considered in the random effects of the models, and all models were estimated using maximum-likelihood estimation methods.
For the additional analysis of AHSQ and GHSQ, each item was analyzed independently by using the generalized linear mixed model (GLMM), and the results were presented in the Appendix (S1 Table). In addition, we used the independent sample t test as a measure to investigate whether there were differences in participants’ evaluations of intervention quality between the intervention and control conditions. All statistical analyses were performed with IBM SPSS Statistics version 26.0 and R software version 4.2. The significance level of each analysis will be set to 0.05 while simultaneously reporting a 95% confidence interval. Considering the accuracy and robustness of the results, we have modified the analytical procedures mentioned in our protocol. Firstly, considering the multiple primary outcomes leading to a higher type I error rate, we used the Bonferroni correction method to adjust the P value of the 3 primary outcomes. Furthermore, we added a sensitivity analysis for all outcomes in the Appendix (S2 Table), which presented results using a mixed linear model that conducted a post hoc analysis calculating the difference from baseline. This trial was registered at www.chictr.org.cn, with the registration number ChiCTR2100053248.
Participants were recruited between December 3, 2021 and January 12, 2022. A total of 736 questionnaire respondents were assessed for eligibility, among which 592 respondents were either ineligible, met the exclusion criteria, lost contact, or stopped enrolling (Fig 1). After signing the consent, participants were notified that they were officially enrolled in the study and assigned a unique number. When the present study reached its planned number of participants (n = 144), recruitment was discontinued. During the post-discussion phase, 2 participants in the intervention condition withdrew their written consent and dropped out of the study, leaving 142 participants for primary analyses, and 139 out of 142 participants completed the intervention procedure, with 3 participants in the control condition dropping out before the start of the intervention. One participant in the intervention condition did not successfully submit baseline data, and 138 (95.8%) of 144 participants had successfully completed the 1-month follow-up questionnaire, and 137 (95.1%) of 144 participants had completed the 3-month follow-up questionnaire (Fig 1). The mean length of follow-up of all participants was 85.64 days (SD = 2.19), ranging from 83 to 93 days. The mean age of the participants was 21.84 (SD = 2.79) for the intervention condition and 22.49 (SD = 2.78) for the control. The gender identity, educational level, and other demographic characteristics were similar between conditions which were shown in Table 1.
For primary outcomes (Table 2), results from the LMM indicated that, compared with the control condition, participants in the intervention condition showed a significant improvement in the help-seeking intention of suicidal ideation at 1-month follow-up (mean difference = 0.19, 95% CI [0.06, 0.33], p = 0.018) (Fig 2a). For the help-seeking intention of emotional issues, the intervention effect showed a significant improvement at 1-month follow-ups (mean difference = 0.17, 95% CI [0.05, 0.28], p = 0.013) (Fig 2b). However, results showed that there was no significant improvement in attitudes toward seeking help from MHPs (Fig 2c).
(a) shows the results of help-seeking intention of suicidal ideation. (b) shows the results of help-seeking intention of emotional issues. (c) shows the results of attitudes toward seeking help from MHPs.
For secondary outcomes, results showed that compared with the control condition, participants in the intervention condition had a significant improvement in the help-seeking intention of suicidal ideation at post-discussion (mean difference = 0.22, 95% CI [0.09, 0.36], p = 0.005) and 3-month follow-up (mean difference = 0.25, 95% CI [0.11, 0.38], p = 0.001). For the help-seeking intention of emotional issues, the intervention effect showed a significant improvement at 3-month follow-ups (mean difference = 0.16, 95% CI [0.04, 0.27], p = 0.022) and no significant improvement at post-discussion (mean difference = 0.12, 95% CI [0.01, 0.23], p = 0.104).
For other secondary outcomes, there was no significant improvement in help-seeking behaviors and professional help-seeking stigma. Regarding the results of depression and anxiety literacy, participants in the intervention condition showed significantly improved depression and anxiety literacy at post-discussion (mean difference = 1.25, 95% CI [0.31, 2.20], p = 0.010; mean difference = 1.50, 95% CI [0.56, 2.44], p = 0.002) and 1-month follow-up (mean difference = 1.88, 95% CI [0.93, 2.83], p < 0.001; mean difference = 1.96, 95% CI [1.01, 2.90], p < 0.001). It was also worth noting that such intervention effect remained strongly significant at the 3-month follow-up (mean difference = 1.55, 95% CI [0.60, 2.50], p = 0.002; mean difference = 1.96, 95% CI [1.01, 2.90], p < 0.001). Furthermore, participants’ help-seeking encouragement-related knowledge had significant improvement at post-discussion, 1-month follow-up, and 3-month follow-up (mean difference = 1.30, 95% CI [0.08, 2.52], p = 0.037; mean difference = 1.44, 95% CI [0.22, 2.67], p = 0.021; mean difference = 1.62, 95% CI [0.39, 2.85], p = 0.010).
Besides, no differences in mental health (depression, anxiety, and stress) were observed between conditions in this study (Table 2). The results of intervention quality evaluated by participants showed that there were no differences between the intervention and control conditions (S3 Table). Results from follow-up evaluations for study participants also suggested high satisfaction rates for the study as a whole.
This randomized controlled trial aimed to examine the efficacy of a digital online-based multicomponent intervention promoting help-seeking in LGBTQ+ young adults with mental health problems. Compared to the control condition, participants who received the intervention showed improvements in general help-seeking intentions for both emotional problems and suicidal ideations. The significant differences at the primary time points (1-month follow-up) prove that the intervention program of the current study is effective. The mean differences between groups of help-seeking intention between baseline and 1-month follow-up were 0.17 for emotional problems and 0.19 for suicidal ideation. Compared with other similar studies [46,47], the mean differences of the current study at 1-month are significant and relatively higher, which indicates that the intervention program in this study can effectively improve the participants’ help-seeking intention at 1-month follow-up. The improvement in the help-seeking intention persisted during the 3-month follow-up, indicating that the effectiveness of the intervention can be better maintained. Additionally, participants who received the intervention showed significant improvements in depression and anxiety literacy at post-discussion, 1-month follow-up, and 3-month follow-up. Furthermore, participation in this intervention also significantly promoted participants’ help-seeking encouragement-related knowledge at post-discussion, 1-month follow-up, and 3-month follow-up. These findings underscore the promising potential of a brief, digital, online-based, yet multicomponent intervention as a practical option for ameliorating general help-seeking intentions and advancing depression and anxiety literacy in the LGBTQ+ population.
There is no cut-off value of the current scales used to assess the intentions of help-seeking [39,47]. We believe that future research should develop a validated scale, scores of which could reflect the impact of different levels of help-seeking intention on help-seeking behavior, to explore the correlation between the help-seeking intention and the actual help-seeking behavior. Nevertheless, these observed effects in the current study also showed that a lightweight psychological intervention could already play a role in improving the help-seeking intention in the LGBTQ+ population, underscoring the fact that such a highly cost-effective intervention format should be applied more extensively in LGBTQ+ health promotion campaigns. Realizing that one has mental health problems and need help is the first step to get effective mental health improvement. The results in our study showed that the intervention not only increased the participants’ mental health literacy, but also improved their intention to seek help, which means that the intervention program effectively help them take the first step to improve their mental health.
Aside from the improvements mentioned above, the intervention did not significantly promote participants’ professional help-seeking attitude and reduce participants’ help-seeking stigma. One potential explanation of these findings could be that the intricate connection between professional help-seeking attitude and help-seeking stigma has formulated a vicious cycle, blocking the intervention effect. Previous research indicated that help-seeking from LGBTQ+ individuals to professionals was deemed undesirable by the LGBTQ+ community in China due to the common idea embedded in the community culture that “help from each other (peer support)” could be more helpful compared to taking the risk of facing invalidation, discrimination, and prejudice to meet non-affirmative professionals, contributing to negative professional help-seeking attitudes and less motivation to seek help when in need [25,37,48–51]. Such a norm of seeking help from each other instead of professionals within the community could be the result of MHPs’ lack of sufficient training on LGBTQ+-affirmative care in China .
We also found the scores of primary outcomes of participants in the control condition increased over time. One possible explanation is that participants in the control condition have the placebo effect. That is, participants think they have accepted effective intervention, so they gave an increased score, which was also the effect of single-blinding. In addition, for the protection of participants at risk of suicide, we added a directory including 3 kinds of safety resources at the end of both brochures received by participants in the control condition and intervention condition. The increase in the help-seeking score of participants in the control condition may be due to this directory. With the purpose of avoiding the interference of these confounding factors, the current study was designed as a randomized controlled trial to verify the effectiveness of the intervention contents. Therefore, the impact of the directory on help-seeking was balanced between participants in 2 conditions.
Based on the structured discussion outline of the intervention condition (S2 Fig), participants in the intervention condition shared their feelings, new understanding of their emotional issues, and barriers to help-seeking after watching the intervention video. The sharing of experiences and feelings may lead other participants with similar experiences or feelings to feel that they are not alone and that their experiences are not unique . This may reduce the stigma of participants. Then, participants shared ideas on how to overcome the barriers to help-seeking and how to make good use of the resources mentioned in the video and brochure. These discussions can help them better understand the content of the video and brochure and use the resources in them to seek help independently after discussion. Although the facilitators turned on audio and video during the discussion, most participants merely used audio during the discussion. Without seeing each other, participants are limited to focusing on the audio discussion and understanding the contents of the intervention video and brochure rather than interacting with each other. In addition, the group discussion took place in an online conferencing group only once, and the duration of the group discussion was only about 30 min when excluding the duration of watching videos and answering questionnaires. Based on these, we believe that the group has not yet formed a stable group, and the interaction between participants was limited.
One of the strengths of the current study was the willingness of participants to trust the researchers and disclose personal information such as educational background and family income. The potential reason for this is that, at the informed consent stage before the research, the researchers introduced the confidential content in the informed consent to the participants in detail, which also promoted the participants’ trust in the researcher so that they were willing to authentically answer their personal information. Considering the effectiveness of the primary outcomes in the current study and the high completion rate of participants, we believe that the digital online-based multicomponent intervention (including psychoeducational videos, online facilitator-led group discussions, and informative electronic help-seeking brochures) could also be applied to other general populations, so as to effectively promote help-seeking for mental health.
There were several other strengths from an implementation standpoint that needed to be addressed. First, the present intervention was conducted with high fidelity and completed with low dropout rates. Three group facilitators have received fidelity training on delivering this evidence-based intervention pre-intervention and passed the fidelity assessment examining adherence to study protocol, quality of delivery, and participant responsiveness with high fidelity ratings. Second, under the benefits of high fidelity, the completion rate was extremely high (95%), which was much higher compared to the average completion rate (80%) for most randomized controlled trials, suggesting that not only did we precisely and accurately implement intervention procedures as planned but also the high level of urgency of such interventions for the LGBTQ+ population . The reasons for the high completion rate may include that: (1) all participants were contacted by a researcher in the way of one-to-one message, so when the participants failed to answer the questionnaires, the researcher would remind them one more time. (2) The participants’ educational level is relatively high, which also makes it easier for them to comply with the commitment to complete the research. Third, our intervention did not necessarily require the participation of MHPs. Once our intervention implementation process has been standardized and manualized in the near future, trainees who have undergone standard training based on our protocol could be expected to be able to carry out similar results. This will promote more LGBTQ+ young adults’ help-seeking intentions and mental health literacy, especially those who live in areas lacking LGBTQ+ mental health resources.
The primary limitation is that this study did not effectively promote participants’ actual help-seeking behaviors. The potential explanation could be the inference of the particular time period chosen to conduct both the present study and its post-intervention follow-ups. Chinese New Year (i.e., the Chinese Spring Festival) has always been the most celebrated national holiday in China, during which families, friends, and relatives reunite with each other. However, due to lower levels of both personal privacy and security arising from large family gatherings, it could also be the least possible occasion in which LGBTQ+ young adults would risk revealing their identities for help-seeking, subsequently blocking their actual help-seeking behaviors. Together with the impact of the pandemic (i.e., Coronavirus Disease 2019), even if they intended to seek help, significantly reduced availability and accessibility of resources was a difficulty they must overcome. The second limitation of this study was that the present study was limited to 3-month follow-ups, which might not be long enough for drastic mindset and behavioral changes in help-seeking to occur. Future studies should extend the follow-up time. The third limitation was that half of the participants of the current sample were transgender, nonbinary, genderqueer, or questioning population. This is because the promotion of online recruitment advertising of this study has been promoted by many LGBTQ+ community organizers. Online posters have been released through social media platforms, such as WeChat or QQ groups, which are popular in the Chinese LGBTQ+ community. Since our study received recognition from community organizers in the LGBTQ+ community, especially organizers from the transgender community, transgender young adults who usually could not be reached via posters and online research participation campaigns expressed their interest in participating in our study, explaining why there were more transgender participants in our study.
The present study strives to examine the efficacy of the digital online-based multicomponent intervention aiming to promote help-seeking in LGBTQ+ young adults. The intervention was found to be an effective integrated model for enhancing LGBTQ+ young adults’ general help-seeking intentions and promoting their depression and anxiety literacy. Although the clinical significance of the present study’s findings warrants future research, the results of the present study add to an emerging breadth of evidence indicating that a brief, digital, online-based, and multicomponent intervention holds great potential in downscaling currently existing mental health barriers faced by the LGBTQ+ population.
S1 CONSORT Checklist. CONSORT 2010 checklist of information to include when reporting a randomized trial.
S1 Fig. Samples of the intervention video.
S2 Fig. The structured discussion outline of the intervention condition.
S3 Fig. Samples of the help-seeking brochure.
S4 Fig. Samples of the help-seeking brochure (English-translation).
S1 Table. Results from the generalized mixed linear model.
S2 Table. Sensitivity Analyses of the linear mixed model.
S3 Table. Results of the intervention evaluation.
S1 Text. Items to include when reporting a randomized trial in a journal or conference abstract.
S1 Protocol. Protocol.
S1 Data. Outcome data in the main article.
S2 Data. Outcome data in supporting information.
- 1. Wang Y, Hu Z, Peng K, Rechdan J, Yang Y, Wu L, et al. Mapping out a spectrum of the Chinese public’s discrimination toward the LGBT community: results from a national survey. BMC Public Health. 2020;20(1):1–10.
- 2. Wang Y, Yu H, Yang Y, Drescher J, Li R, Yin W, et al. Mental health status of cisgender and gender-diverse secondary school students in China. JAMA Netw Open. 2020;3(10):e2022796-e. pmid:33107922
- 3. Gonzales G, Przedworski J, Henning-Smith C. Comparison of health and health risk factors between lesbian, gay, and bisexual adults and heterosexual adults in the United States: Results from the National Health Interview Survey. JAMA Intern Med. 2016;176(9):1344–1351. pmid:27367843
- 4. Eaton NR. Transdiagnostic psychopathology factors and sexual minority mental health: evidence of disparities and associations with minority stressors. Psychol Sex Orientat Gend Divers. 2014;1(3):244. pmid:25530981
- 5. Moagi MM, van Der Wath AE, Jiyane PM, Rikhotso RS. Mental health challenges of lesbian, gay, bisexual and transgender people: An integrated literature review. Health SA. 2021;26:1487. pmid:33604059
- 6. Douglass RP, Conlin SE. Minority stress among LGB people: Investigating relations among distal and proximal stressors. Curr Psychol. 2020:1–11.
- 7. Meyer IH. Prejudice, social stress, and mental health in lesbian, gay, and bisexual populations: conceptual issues and research evidence. Psychol Bull. 2003;129(5):674. pmid:12956539
- 8. Liu F, Chui H, Chung MC. The Moderating Effect of Filial Piety on the Relationship Between Perceived Public Stigma and Internalized Homophobia: a National Survey of the Chinese LGB Population. Sex Res Social Policy. 2021;18(1):160–169.
- 9. Hildebrandt T. The One-Child Policy, Elder Care, and LGB Chinese: A Social Policy Explanation for Family Pressure. J Homosex. 2019;66(5):590–608. pmid:29297777
- 10. Ramirez JL, Paz GM. Multiple minority stress: The role of proximal and distal stress on mental health outcomes among lesbian, gay, and bisexual people of color. J Gay Lesbian Ment Health. 2019;23(2):145–167.
- 11. Reisner SL, Greytak EA, Parsons JT, Ybarra ML. Gender minority social stress in adolescence: disparities in adolescent bullying and substance use by gender identity. J Sex Res. 2015;52(3):243–256. pmid:24742006
- 12. Mongelli F, Perrone D, Balducci J, Sacchetti A, Ferrari S, Mattei G, et al. Minority stress and mental health among LGBT populations: An update on the evidence. Minerva Psichiatr. 2019:60.
- 13. Spengler ES, Ægisdóttir S. Psychological help-seeking attitudes and intentions of lesbian, gay, and bisexual individuals: The role of sexual minority identity and perceived counselor sexual prejudice. Psychol Sex Orientat Gend Divers. 2015;2(4):482.
- 14. Scheer JR, Martin-Storey A, Baams L. Help-seeking barriers among sexual and gender minority individuals who experience intimate partner violence victimization. Intimate partner violence and the LGBT+ community. 2020:139–158.
- 15. Silveri G, Schimmenti S, Prina E, Gios L, Mirandola M, Converti M, et al. Barriers in care pathways and unmet mental health needs in LGBTIQ + communities. Int Rev Psychiatry. 2022;34(3–4):215–229. pmid:36151825
- 16. Higgins A, Downes C, Murphy R, Sharek D, Begley T, McCann E, et al. LGBT + young people’s perceptions of barriers to accessing mental health services in Ireland. J Nurs Manag. 2021;29(1):58–67. pmid:33068465
- 17. Kirchner S, Till B, Plöderl M, Niederkrotenthaler T. Effects of “It Gets Better” Suicide Prevention Videos on Youth Identifying as Lesbian, Gay, Bisexual, Transgender, Queer, or Other Sexual or Gender Minorities: A Randomized Controlled Trial. LGBT Health. 2022;9(6):436–446. pmid:35575732
- 18. Egan JE, Corey SL, Henderson ER, Abebe KZ, Louth-Marquez W, Espelage D, et al. Feasibility of a Web-Accessible Game-Based Intervention Aimed at Improving Help Seeking and Coping Among Sexual and Gender Minority Youth: Results From a Randomized Controlled Trial. J Adolesc Health. 2021;69(4):604–614. pmid:34140199
- 19. Nicholas A, Pirkis J, Reavley N. What responses do people at risk of suicide find most helpful and unhelpful from professionals and non-professionals? J Ment Health. 2022;31(4):496–505. pmid:32930018
- 20. Wang Y, Wilson A, Hu Z, Lu L, Li W, Peng K, et al. Counselling and psychotherapy service use in Chinese sexual minority populations: a nationwide survey. BMC Psychiatry. 2021;21(1):11. pmid:33413187
- 21. Chambi-Martínez CAA, Moraga-Escobar EI, Peralta-Jiménez GA, Vera-Calzaretta A, Barrientos J, Pihán R, et al. Social Support, Stress and Emotional Symptoms Among LGBTQ+ College Students in Chile. Int J Sex Health. 2022;34(2):277–290.
- 22. Kay M, Santos J, Takane M. mHealth: New horizons for health through mobile technologies. World Health Organization. 2011;64(7):66–71.
- 23. Ferlatte O, Salway T, Rice S, Oliffe JL, Rich AJ, Knight R, et al. Perceived Barriers to Mental Health Services Among Canadian Sexual and Gender Minorities with Depression and at Risk of Suicide. Community Ment Health J. 2019;55(8):1313–1321. pmid:31327106
- 24. Cronin TJ, Pepping CA, Halford WK, Lyons A. Mental health help-seeking and barriers to service access among lesbian, gay, and bisexual Australians. Aust Psychol. 2021;56(1):46–60.
- 25. Dunbar MS, Sontag-Padilla L, Ramchand R, Seelam R, Stein BD. Mental health service utilization among lesbian, gay, bisexual, and questioning or queer college students. J Adolesc Health. 2017;61(3):294–301. pmid:28549595
- 26. Wallin EEK, Mattsson S, Olsson EMG. The preference for internet-based psychological interventions by individuals without past or current use of mental health treatment delivered online: a survey study with mixed-methods analysis. JMIR Ment Health. 2016;3(2):e5324. pmid:27302200
- 27. Andersson G, Titov N. Advantages and limitations of Internet-based interventions for common mental disorders. World Psychiatry. 2014;13(1):4–11. pmid:24497236
- 28. Richardson LK, Frueh BC, Grubaugh AL, Egede L, Elhai JD. Current directions in videoconferencing tele-mental health research. Clin Psychol Sci Pract. 2009;16(3):323. pmid:20161010
- 29. Johnson JA, Sanghvi P, Mehrotra S. Technology-Based Interventions to Improve Help-Seeking for Mental Health Concerns: A Systematic Review. Indian J Psychol Med. 2021:02537176211034578. pmid:35949632
Chinese Clinical Trial Registry [cited 2022 Nov 2]. Available from: http://www.chictr.org.cn/index.aspx.
- 31. Ren D, Wang Y, Han M, Zhang Y, Cai C, Liu K, et al. Internet-based interventions to promote help-seeking for mental health in LGBTQ+ young adults: protocol for a randomized controlled trial. Internet Interv. 2022;28:100524. pmid:35342718
- 32. Lindley L, Bauerband L. Measurement invariance of the Depression, Anxiety, and Stress Scale (DASS-21) across cisgender sexual minority and transgender and nonbinary individuals. Psychology of Sexual Orientation and Gender Diversity. 2022. Advance online publication.
- 33. Gomez F. A guide to the depression, anxiety and stress scale (DASS 21). Central and Eastem Sydney primary health networks. 2016. [cited 2023 Feb 14]. Available from: https://www.academia.edu/25177167/A_Guide_to_the_Depression_Anxiety_and_Stress_Scale_DASS_21.
- 34. Conceição V, Rothes I, Gusmão R. The effects of a video-based randomized controlled trial intervention on depression stigma and help-seeking attitudes in university students. Psychiatry Res. 2022;308:114356. pmid:34972028
- 35. Matheson EL, Lewis-Smith H, Diedrichs PC. The effectiveness of brief animated films as a scalable micro-intervention to improve children’s body image: a randomised controlled trial. Body Image. 2020;35:142–153. pmid:33049455
- 36. Han J, Batterham PJ, Calear AL, Ma J. Seeking professional help for suicidal ideation: A comparison between Chinese and Australian university students. Psychiatry Res. 2018;270:807–814. pmid:30551329
- 37. Zay Hta MK, Tam CL, Au SY, Yeoh G, Tan MM, Lee ZY, et al. Barriers and facilitators to professional mental health help-seeking behavior: perspective of Malaysian LGBT individuals. J LGBTQ Issues Couns. 2021;15(1):38–58.
- 38. Pitaloka D, Thresia F. The effect of using animation video for mentally retarded students in vocabulary mastery. J English Educ Entrep. 2021;1(2):113–121.
- 39. Han J, Batterham PJ, Calear AL, Wu Y, Xue J, van Spijker BAJ. Development and pilot evaluation of an online psychoeducational program for suicide prevention among university students: A randomised controlled trial. Internet Interv. 2018;12:111–120. pmid:30135775
- 40. Fang S, Yang B, Yang F, Zhou Y. Study on reliability and validity of the Chinese version of the Attitudes toward Seeking Professional Psychological Help Scale-Short Form for community population in China. Chin Nurs Res. 2019;33(14):2410–2414.
Han J. Professional help seeking for suicidal ideation among Chinese and Australian university students. PhD. Thesis. The Australian National University. 2017. Available from: https://openresearch-repository.anu.edu.au/handle/1885/141152.
- 42. Griffiths KM, Christensen H, Jorm AF, Evans K, Groves C. Effect of web-based depression literacy and cognitive–behavioural therapy interventions on stigmatising attitudes to depression: Randomised controlled trial. Br J Psychiatry. 2004;185(4):342–349. pmid:15458995
- 43. Gulliver A, Griffiths KM, Christensen H, Mackinnon A, Calear AL, Parsons A, et al. Internet-based interventions to promote mental health help-seeking in elite athletes: an exploratory randomized controlled trial. J Med Internet Res. 2012;14(3):e1864. pmid:22743352
- 44. Zhou Y, Lemmer G, Xu J, Rief W. Cross-cultural measurement invariance of scales assessing stigma and attitude to seeking professional psychological help. Front Psychol. 2019;10:1249. pmid:31214074
- 45. Taylor-Rodgers E, Batterham PJ. Evaluation of an online psychoeducation intervention to promote mental health help seeking attitudes and intentions among young adults: randomised controlled trial. J Affect Disord. 2014;168:65–71. pmid:25038293
- 46. Nickerson A, Byrow Y, Pajak R, McMahon T, Bryant RA, Christensen H, et al. “Tell Your Story”: a randomized controlled trial of an online intervention to reduce mental health stigma and increase help-seeking in refugee men with posttraumatic stress. Psychol Med. 2020;50(5):781–792. pmid:30973115
- 47. Xu Z, Huang F, Kösters M, Staiger T, Becker T, Thornicroft G, et al. Effectiveness of interventions to promote help-seeking for mental health problems: systematic review and meta-analysis. Psychol Med. 2018;48(16):2658–2667. pmid:29852885
- 48. Ezhova I, Savidge L, Bonnett C, Cassidy J, Okwuokei A, Dickinson T. Barriers to older adults seeking sexual health advice and treatment: A scoping review. Int J Nurs Stud. 2020;107:103566. pmid:32380261
- 49. Shipherd JC, Green KE, Abramovitz S. Transgender clients: Identifying and minimizing barriers to mental health treatment. J Gay Lesbian Ment Health. 2010;14(2):94–108.
- 50. Mink MD, Lindley LL, Weinstein AA. Stress, stigma, and sexual minority status: The intersectional ecology model of LGBTQ health. J Gay Lesbian Soc Serv. 2014;26(4):502–521.
- 51. Herek GM, Gillis JR, Cogan JC. Internalized stigma among sexual minority adults: Insights from a social psychological perspective. Stigma Health. 2015;1(S):18–34.
- 52. Liu F, Chui H, Chung MC. Clients’ experience of LGBQ affirmative and nonaffirmative practice in China: An interpretative phenomenological analysis. Psychotherapy. 2022;59(1):1–12. pmid:35084885
- 53. Currie CL, Larouche R, Voss ML, Trottier M, Spiwak R, Higa E, et al. Effectiveness of Live Health Professional-Led Group eHealth Interventions for Adult Mental Health: Systematic Review of Randomized Controlled Trials. J Med Internet Res. 2022;24(1):e27939. pmid:34878409
- 54. Wood AM, White IR, Thompson SG. Are missing outcome data adequately handled? A review of published randomized controlled trials in major medical journals. Clin Trials. 2004;1(4):368–376. pmid:16279275