https://orcid.org/0000-0001-7628-882X
Figures
Abstract
Mental health conditions can have significant negative impacts on wellbeing and healthcare systems. Despite their high prevalence worldwide, there is still insufficient recognition and accessible treatments. Many mobile apps are available to the general population that aim to support mental health needs; however, there is limited evidence of their effectiveness. Mobile apps for mental health are beginning to incorporate artificial intelligence and there is a need for an overview of the state of the literature on these apps. The purpose of this scoping review is to provide an overview of the current research landscape and knowledge gaps regarding the use of artificial intelligence in mobile health apps for mental health. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) and Population, Intervention, Comparator, Outcome, and Study types (PICOS) frameworks were used to structure the review and the search. PubMed was systematically searched for randomised controlled trials and cohort studies published in English since 2014 that evaluate artificial intelligence- or machine learning-enabled mobile apps for mental health support. Two reviewers collaboratively screened references (MMI and EM), selected studies for inclusion based on the eligibility criteria and extracted the data (MMI and CL), which were synthesised in a descriptive analysis. 1,022 studies were identified in the initial search and 4 were included in the final review. The mobile apps investigated incorporated different artificial intelligence and machine learning techniques for a variety of purposes (risk prediction, classification, and personalisation) and aimed to address a wide range of mental health needs (depression, stress, and suicide risk). The studies’ characteristics also varied in terms of methods, sample size, and study duration. Overall, the studies demonstrated the feasibility of using artificial intelligence to support mental health apps, but the early stages of the research and weaknesses in the study designs highlight the need for more research into artificial intelligence- and machine learning-enabled mental health apps and stronger evidence of their effectiveness. This research is essential and urgent, considering the easy availability of these apps to a large population.
Author summary
Mental health concerns are a large burden for individuals, healthcare systems, and the economy. Although mental health concerns affect a lot of people, many find it difficult to access appropriate support. Mobile health apps are one potential way to address long wait times and a lack of mental health resources. With this review, we wanted to provide an overview of mobile health apps that are using artificial intelligence (AI) to provide some type of mental health support and to identify areas where more research is needed. We found 17 studies that evaluated an AI mental health app. There were many different uses of AI in the apps, including to provide conversational support to users, to predict moods, and to do risk assessments. However, the research was still in early stages; most of the studies had small numbers of participants and there is a need for more high-quality studies to evaluate whether the apps have significant benefits for users. This information that we have gathered will help guide the development of future studies and AI mental health apps.
Citation: Milne-Ives M, Selby E, Inkster B, Lam C, Meinert E (2022) Artificial intelligence and machine learning in mobile apps for mental health: A scoping review. PLOS Digit Health 1(8): e0000079. https://doi.org/10.1371/journal.pdig.0000079
Editor: Padmanesan Narasimhan, University of New South Wales, AUSTRALIA
Received: October 7, 2021; Accepted: June 22, 2022; Published: August 15, 2022
Copyright: © 2022 Milne-Ives et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data used in this scoping review was extracted from previously published papers and cited in the text and reference list.
Funding: This research was funded by the NIHR Artificial Intelligence in Health and Care Award (grant reference number: AI_AWARD02176). The views expressed in the paper belong to the authors and not necessarily those of NIHR, the University of Plymouth, or Wysa Ltd. The funding bodies were not involved in the study design, data collection or analysis, or the writing and decision to submit the article for publication.
Competing interests: ES is an employee of, and BI is an advisor for, Wysa Ltd. a company that has designed and developed an AI-enabled mental health app. ES and BI contributed to the initial conception of the study. No employees of Wysa were involved in the manuscript’s final drafting.
Introduction
Background
Mental health conditions, such as anxiety and depression, can have significant negative impacts on a range of mental and physical wellbeing, social, and employment outcomes [1,2]. People with severe, long-term mental illness have an average of 15 years shorter life expectancies than the general population [3]. Worldwide, there is a high prevalence of mental health issues and conditions [4]; in the UK, approximately a quarter of the population is seeking mental health treatment [5]. Despite this, it is estimated that 75% of people who need mental health support do not receive it, resulting in costs to the UK economy of approximately £100 billion annually [3,6]. Globally, this cost exceeds US$1 trillion each year [7]. There is a clear need for improved means of identifying and supporting mental health conditions among the general population.
Rationale
Many mobile health apps have been developed and made available to the public to try and address this need [8]. Several systematic reviews have recently been published focusing on various aspects and outcomes of mental health apps [8–13]. Most of these systematic reviews found methodological issues (such as a lack of control or comparison groups or representative samples, and a high risk of bias) [9,10] and insufficient evidence of effectiveness of mental health apps for changing behaviours or improving clinical outcomes [9,11,13]. However, one meta-analysis of randomised controlled trials found a significant difference between app interventions and control conditions (but not face-to-face or computer interventions) on certain outcomes [12]. A recent review of meta-analyses also found evidence for small to medium effects of apps on quality of life and stress [8]. This suggests that there is potential for mobile apps to support mental health, although there is a need for further, high-quality research to provide evidence of effectiveness. Following standardised guidelines for study design and reporting (such as the CONSORT statement [14] and CONSORT-EHEALTH extension [15]) would improve the quality of evidence available and help determine in what contexts mental health apps could provide benefits.
Despite this need for more rigorous evaluation, mobile apps for mental health are widely available to the general public and new ones are being designed to include innovative technologies. A number of mobile apps for mental health are available in app stores that have incorporated artificial intelligence (AI) and machine learning (ML) technologies into their service [16–18]. AI refers to the simulation of human intelligence in machines whereas ML allows machines to learn from data without being explicitly programmed [19]. AI/ML techniques have been widely applied in healthcare to generate insights from massive amounts of data [20–22] and are increasingly being incorporated in mobile health apps [23].
None of the systematic reviews that were identified examined evidence for the use of AI in mobile apps for mental health. A search of PROSPERO for registered reviews using the keywords “mental health apps” AND “AI OR artificial intelligence OR machine learning OR chatbot” also found no records. Given the increasing use of artificial intelligence in mobile health apps, a scoping review is needed to provide an overview of the state of the literature and to summarise the strengths and weaknesses of the existing research [24]. An overview of the state of research on AI/ML-enabled mental health apps will help to inform directions for future research and provide an initial assessment of the potential of these apps.
Objectives and research questions
The primary objective of the scoping review was to assess and summarize the current state of the literature on the use of AI/ML in mobile apps for mental health. Two research questions were defined to guide the review:
- How are artificial intelligence and machine learning techniques incorporated into mobile apps for mental health?
- What are the strengths and weaknesses of the research being conducted in this field?
Methods
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR; S1 Appendix) [25] was used to structure the review. The search strategy was developed using the Population, Intervention, Comparator, Outcome, and Study types (PICOS) framework (see Table 1). No separate protocol has been published.
A preliminary review of the literature was used to identify relevant MeSH terms and keywords for the search. These terms were grouped into three themes and were searched using the following search string structure: mobile applications (MeSH OR Keywords) AND mental health (MeSH OR Keywords) AND artificial intelligence (MeSH OR Keywords) (Table 2). PubMed was searched on 1 March 2021. S2 Appendix provides a list of the final search string and the number of results returned.
Inclusion criteria
The review included studies that evaluated AI/ML-enabled mobile apps that aim to provide any type of mental health support. ‘Support’ was not limited to treatment—apps aiming to identify risk, provide a means of monitoring risk factors or symptoms, or deliver education or therapeutic interventions were all eligible for inclusion. Participants of any age (children and adults) were eligible for inclusion. Any type of interventional, observational, or validation study design was eligible for inclusion as long as the study evaluated the mobile health app in question.
As the definition of mental health can be very broad, all studies of mobile apps that aimed to provide support for any mental health-related conditions or to improve well-being were included, except for substance abuse interventions. Although substance abuse disorders often co-exist with mental health conditions, mobile app interventions for substance abuse have specific aims of reducing use and were considered outside the general scope of this review [26]. Studies published since 2014 were included, because this marks the early stages of the introduction of artificial intelligence into smartphone apps. Between 2010–2014, Apple, Microsoft, Amazon, and Google all introduced AI-based voice assistants into phones and other Internet-of-Things devices [27]. Preliminary searches of PubMed also identified a significant increase in the number of retrieved publications starting in 2014, supporting the decision to use 2014 as the date cut-off.
Exclusion criteria
Studies that did not evaluate a specific mobile health app with a primary focus on providing mental health support (e.g. protocols, reviews, meta-analyses, perspectives, descriptions of design or development) were excluded. Studies of mobile apps without an artificial intelligence or machine learning component were excluded from the analysis, as were mobile health apps designed as substance abuse interventions.
Screening and article selection
The citation management software EndNote X9 was used to store the references and remove duplicates. As a large number of references were identified in the initial search, the EndNote X9 search function was used to screen them based on keywords from the search strategy (see S3 Appendix) before they were manually screened. The remaining titles and abstracts were screened by two reviewers (MMI and EM) and a full-text review was conducted by two authors to determine final eligibility (MMI and CL). The review was updated after a second screening of the retrieved studies to ensure that all of the relevant papers were included.
Data extraction
One reviewer examined the studies to extract key data (see Table 3). The data extraction table was developed based on the research questions and the PICOS framework.
Data analysis and synthesis
A descriptive analysis of the data extracted from the studies was conducted and summarised to provide a scoping overview of the state of the literature and the strengths and weaknesses of research about artificially-enabled mental health support apps. Data analysis was conducted collaboratively by two researchers (MMI and CL) to apply their different areas of expertise to the extraction and interpretation of the data. Implications of the findings for future research and limitations of the review are examined in the discussion; however, in accordance with accepted standards for scoping reviews, the methodological quality and risk of bias of the studies are not examined [25]. As the aim of the review was to provide an overview of how AI and ML are being incorporated into mobile apps for mental health support, a critical appraisal of the quality of the studies describing these technologies was not considered to be within the scope of this review.
Results
Included studies
The search of PubMed retrieved 1,228 articles (see S2 Appendix). After duplicates (n = 206) were removed using EndNote X9, 1,022 articles were screened using EndNote’s keyword search. 14 articles remained after 6 rounds of screening (see S3 Appendix) and 4 articles were determined to be eligible for final inclusion upon title/abstract and full-text review. A second screening was conducted with revised keyword searches and identified 17 articles for full text review (see S3 Appendix); 2 were already included, and 13 were determined to be eligible for inclusion, resulting in a total of 17 articles included in the final review (see Fig 1).
Study characteristics
There were a large variety of study types among the 17 studies, including pilot randomised controlled trials (RCTs) [28–30], observational studies [31–35], model evaluations [36,37], and a qualitative study [38].
The sample sizes included in the studies ranged from 6 [39] to 354 [40], but the number analysed was often lower than the recruited sample. 12 of the 17 included studies had sample sizes smaller than 100 [28–31,33–35,37–39,41,42]. All of the studies focused on adults.
App characteristics
The apps evaluated by the studies in this review had a range of uses of artificial intelligence. The most common uses of artificial intelligence in the mental health apps were prediction (of mood, stress, and risk) (8/17 studies) [31–37,41] and natural language conversations to deliver mental health support (6/17 studies) [29,30,38,40,42,43]. In other studies, AI/ML was used to provide diagnostic decision support [39] or to personalise aspects of the app such as notification delivery [28] and recommended interventions [44].
The apps also addressed diverse mental health challenges. 6 of the 17 studies focused on depression [29,31,36,40,42,44], 3 studies each focused on predicting mood in people with mood disorders [34,35,37], predicting or helping manage stress [28,32,33], or general well-being support [30,38,43], one aimed to identify suicide risk in inpatients [41], and one provided diagnostic support from a broad spectrum of mental disorders [39].
Sensors and data collection
The three main types of data collection via apps were passive phone-based sensors, active user input, and data from connected wearables (Table 4). The most common type of data collection was self-reported data about the users’ mood or stress levels (12/17 studies).
ML algorithms
Classification algorithms learn the patterns in input training data to predict the likelihood that subsequent data will fall into one of the predetermined categories [45]. The most common classification algorithm used in the included studies was a Random Forest classifier (5/17 studies, see Table 5). It is used for feature selection [41], risk-stratification of diabetic patients based on symptoms of depression [31], inferring context for depression level prediction [44], and predicting mood [34,35]. 5 of the 17 studies did not specify what machine learning algorithm was used and are thus not included in Table 5 [38–40,42,43].
Boosting algorithms are primarily used to reduce bias by iteratively learning weak classifiers and adding them to a final strong classifier to improve the model predictions of any given learning algorithm [46]. Only 2 out of the 17 studies reported used boosting algorithms [31,37].
Only one of the studies used a dimension reduction technique to reduce the number of features (e.g. sleep data, journal entries, mood) in order to aid further classification and improve classification reliability [41].
Study findings
The majority of the studies demonstrated feasibility of their AI/ML-enabled mental health solution. Although generally positive, the evidence gathered by the studies was preliminary. Many of the studies successfully modelled data collected by the app or associated inputs to identify behaviours or responses associated with mental health conditions [44], provide natural language support [29,30,38,40,42,43], develop risk profiles [31–37,41], and deploy context-sensitive notifications [28] or interventions [44].
The evidence of the impact of these apps on mental health was more limited. 7 of the 17 studies evaluated mental health as an outcome (rather than a model input) [29,30,35,38,42–44]. Two studies only reported user experiences; both found generally positive perceptions of the app in question [38,42]. Of the others, two found significant differences between app and control groups: one found a significant reduction in depression, but not anxiety, in the app group (Woebot) compared to the control group [29] and one found a significant reduction in amount and time of depressive, manic or hypomanic, and mood episodes in the app group (Circadian Rhythm for Mood) than the control group [35]. The three remaining studies observed differences associated with level of engagement: one observed a significant decrease in symptoms of depression, but only in a small subset of users who had high and prolonged adherence to the app [44], one found significant effects on well-being and stress in participants who adhered to the intervention [30], and one found higher mean improvement of mood in the high users compared to lower users [43]. This demonstrates the potential benefit of this approach, but identifies the importance of maintaining user engagement with the intervention for any potential mental health benefits.
The studies all recognized the need for further research, highlighting the early stage of the state of the literature. One study using AI/ML to send context-specific notifications found no difference between the ‘intelligent’ notifications and non-intelligent notifications [28]. This demonstrates the importance of rigorously evaluating the effectiveness of the use of AI/ML for specific purposes and intended outcomes.
Discussion
Summary of findings
Of the large number of mobile health apps for mental health [8], many apps—with high ratings and large numbers of downloads—have begun to incorporate artificial intelligence [16–18]. This review shows the diversity and feasibility of the use of artificial intelligence to support mental health care in a variety of different ways. However, it also demonstrates that, so far, there is limited research that can provide evidence of the effectiveness of these apps. Only three randomised controlled trials of an AI-enabled mental health app were identified in this review [28–30] and all of them were small-scale pilot RCTs, despite the availability of several highly-used AI-enabled mental health apps on the Apple App Store and Google Play (e.g. Woebot, Reflectly, Wysa, Youper). This review identifies the strengths and weaknesses in this field and highlights the need for high-quality, rigorous investigation of the AI-enabled mental health apps that are currently available and being used as well as those in development.
Strengths of AI/ML in apps
Several strengths of using AI in mental health apps were identified from these studies. Mobile devices can collect active data from participants and capture passive sensing data, providing a rich body of information for analysis and to tailor the provision of support. This application of AI/ML in real world situations provides significant data for modelling. All of the studies reviewed used data collected via mobile devices to inform AI models—for natural language analysis and output [29,30,38,40,42,43], risk stratification and prediction [31,41] or for personalising the user’s app experience [28,44]. They provided an initial demonstration of the feasibility of using AI for a wide variety of purposes to support mental health identification, risk assessment, and treatment.
Weaknesses of AI/ML app studies
Although the studies demonstrated feasibility and potential for the use of AI in mental health apps, they had a number of significant limitations. The sample sizes of the studies were relatively low, ranging from 6 [39] to 354 [40], with the latter being a retrospective descriptive study of real-world users, not recruited participants. The study duration was another weakness for several of the studies; of the 12 studies that included an intervention period, only 3 (25%) had intervention periods longer than 1 year [33–35]. 5 of the 12 studies (42%) had a duration of less than one month [29,30,32,41,42].
Limitations
The main limitation of this review is that only one database was searched to identify studies for inclusion, meaning that potentially relevant studies could have been overlooked. Another limitation is that the review was not conducted by two independent reviewers. Although two reviewers collaborated on the screening and data analysis, and the PRISMA-ScR framework was used to guide the structure of the review and ensure the necessary components were addressed, there was a lack of independent validation of the screening and analysis that would have strengthened the review.
Conclusion and future research
The purpose of this scoping review was to examine and provide an overview of the state of the research about AI-enabled mobile apps for mental health. A large body of research was initially identified, but the number of randomised control trials evaluating AI/ML technologies incorporated in mental health apps was limited. The studies included in the review demonstrated the potential feasibility of incorporating AI/ML into mental health apps as well as the variety of applications of AI/ML in mental health care to provide support to individuals and an overburdened healthcare system. However, the review demonstrates a clear need for more high-quality randomised controlled trials of these apps, to evaluate whether they are actually achieving their intended purposes and providing a benefit to users. This evidence will enable more effective apps to be recommended to the public and will further demonstrate the potential of AI/ML-enabled apps to support the identification of mental health conditions and risk assessments.
Supporting information
S4 Appendix. Data Extraction (first screening).
https://doi.org/10.1371/journal.pdig.0000079.s004
(DOCX)
S5 Appendix. Data Extraction (second screening).
https://doi.org/10.1371/journal.pdig.0000079.s005
(XLSX)
References
- 1.
The Improving Access to Psychological Therapies (IAPT) Pathway for People with Long-term Physical Health Conditions and Medically Unexplained Symptoms. NHS England; 2018.
- 2. Naylor C, Parsonage M, McDaid D, Knapp M, Fossey M, Galea A. Long-term conditions and mental health: The cost of co-morbidities. The King’s Fund; 2012.
- 3. The Five Year Forward View for Mental Health. The Mental Health Taskforce; 2016.
- 4.
Mental disorders. World Health Organisation n.d. https://www.who.int/news-room/fact-sheets/detail/mental-disorders (accessed March 24, 2021).
- 5.
Adult Psychiatric Morbidity Survey: Survey of Mental Health and Wellbeing, England, 2014. NHS Digital 2016. https://digital.nhs.uk/data-and-information/publications/statistical/adult-psychiatric-morbidity-survey/adult-psychiatric-morbidity-survey-survey-of-mental-health-and-wellbeing-england-2014 (accessed March 24, 2021).
- 6. Alonso J, Liu Z, Evans-Lacko S, Sadikova E, Sampson N, Chatterji S, et al. Treatment gap for anxiety disorders is global: Results of the World Mental Health Surveys in 21 countries. Depress Anxiety 2018;35:195–208. pmid:29356216
- 7.
Mental health in the workplace. World Health Organisation n.d. https://www.who.int/teams/mental-health-and-substance-use/mental-health-in-the-workplace (accessed March 24, 2021).
- 8. Lecomte T, Potvin S, Corbière M, Guay S, Samson C, Cloutier B, et al. Mobile Apps for Mental Health Issues: Meta-Review of Meta-Analyses. JMIR mHealth and uHealth 2020;8:e17458. pmid:32348289
- 9. Lui JHL, Marcus DK, Barry CT. Evidence-based apps? A review of mental health mobile applications in a psychotherapy context. Prof Psychol Res Pr 2017;48:199–210.
- 10. Donker T, Petrie K, Proudfoot J, Clarke J, Birch M-R, Christensen H. Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res 2013;15:e247. pmid:24240579
- 11. Weisel KK, Fuhrmann LM, Berking M, Baumeister H, Cuijpers P, Ebert DD. Standalone smartphone apps for mental health-a systematic review and meta-analysis. NPJ Digit Med 2019;2:118. pmid:31815193
- 12. Linardon J, Cuijpers P, Carlbring P, Messer M, Fuller-Tyszkiewicz M. The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials. World Psychiatry 2019;18:325–36. pmid:31496095
- 13. Milne-Ives M, Lam C, De Cock C, Van Velthoven MH, Meinert E. Mobile Apps for Health Behavior Change in Physical Activity, Diet, Drug and Alcohol Use, and Mental Health: Systematic Review. JMIR Mhealth Uhealth 2020;8:e17046. pmid:32186518
- 14. Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:c869–c869. pmid:20332511
- 15. Eysenbach G CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011;13:e126. pmid:22209829
- 16.
11 AI-based mental health apps. CMB Clinical Trials 2020. https://www.cmbclinicaltrials.com/post/11-ai-based-mental-health-apps (accessed March 24, 2021).
- 17. de Jesus A. Chatbots for Mental Health and Therapy–Comparing 5 Current Apps and Use Cases. Emerj 2018. https://emerj.com/ai-application-comparisons/chatbots-mental-health-therapy-comparing-5-current-apps-use-cases/ (accessed March 24, 2021).
- 18. D’Alfonso S. AI in mental health. Current Opinion in Psychology 2020;36:112–7. pmid:32604065
- 19. Russell S, Norvig P. Artificial intelligence: a modern approach 2002.
- 20. Balicer RD, Cohen-Stavi C. Advancing Healthcare Through Data-Driven Medicine and Artificial Intelligence. Healthcare and Artificial Intelligence 2020:9–15. https://doi.org/10.1007/978-3-030-32161-1_2.
- 21. Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol 2017;2:230–43.
- 22. Chen M, Hao Y, Hwang K, Wang L, Wang L. Disease Prediction by Machine Learning Over Big Data From Healthcare Communities. IEEE Access 2017;5:8869–79.
- 23. Sannino G, Bouguila N, De Pietro G, Celesti A. Artificial Intelligence for Mobile Health Data Analysis and Processing. Mobile Information Systems 2019;2019:1–2.
- 24. Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol 2018;18:1–7.
- 25. Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med 2018;169:467–73. pmid:30178033
- 26. Tofighi B, Chemi C, Ruiz-Valcarcel J, Hein P, Hu L. Smartphone Apps Targeting Alcohol and Illicit Substance Use: Systematic Search in in Commercial App Stores and Critical Content Analysis. JMIR mHealth and uHealth 2019;7:e11831. pmid:31008713
- 27. Hoy MB. Alexa, Siri, Cortana, and more: An introduction to voice assistants. Med Ref Serv Q 2018;37:81–8. pmid:29327988
- 28. Morrison LG, Hargood C, Pejovic V, Geraghty AWA, Lloyd S, Goodman N, et al. The Effect of Timing and Frequency of Push Notifications on Usage of a Smartphone-Based Stress Management Intervention: An Exploratory Trial. PLoS One 2017;12:e0169162. pmid:28046034
- 29. Fitzpatrick KK, Darcy A, Vierhile M. Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. JMIR Ment Health 2017;4:e19. pmid:28588005
- 30. Ly KH, Ly A-M, Andersson G. A fully automated conversational agent for promoting mental well-being: A pilot RCT using mixed methods. Internet Interv 2017;10:39–46. pmid:30135751
- 31. Sarda A, Munuswamy S, Sarda S, Subramanian V. Using Passive Smartphone Sensing for Improved Risk Stratification of Patients With Depression and Diabetes: Cross-Sectional Observational Study. JMIR mHealth and uHealth 2019;7:e11041. pmid:30694197
- 32. Sano A, Taylor S, McHill AW, Phillips AJ, Barger LK, Klerman E, et al. Identifying Objective Physiological Markers and Modifiable Behaviors for Self-Reported Stress and Mental Health Status Using Wearable Sensors and Mobile Phones: Observational Study. J Med Internet Res 2018;20:e210. pmid:29884610
- 33. Rozet A, Kronish IM, Schwartz JE, Davidson KW. Using Machine Learning to Derive Just-In-Time and Personalized Predictors of Stress: Observational Study Bridging the Gap Between Nomothetic and Ideographic Approaches. J Med Internet Res 2019;21:e12910. pmid:31025942
- 34. Cho C-H, Lee T, Kim M-G, In HP, Kim L, Lee H-J. Mood Prediction of Patients With Mood Disorders by Machine Learning Using Passive Digital Phenotypes Based on the Circadian Rhythm: Prospective Observational Cohort Study. J Med Internet Res 2019;21:e11029. pmid:30994461
- 35. Cho C-H, Lee T, Lee J-B, Seo JY, Jee H-J, Son S, et al. Effectiveness of a Smartphone App With a Wearable Activity Tracker in Preventing the Recurrence of Mood Disorders: Prospective Case-Control Study. JMIR Ment Health 2020;7:e21283. pmid:32755884
- 36. Mikus A, Hoogendoorn M, Rocha A, Gama J, Ruwaard J, Riper H. Predicting short term mood developments among depressed patients using adherence and ecological momentary assessment data. Internet Interv 2018;12:105–10. pmid:30135774
- 37. Busk J, Faurholt-Jepsen M, Frost M, Bardram JE, Kessing LV, Winther O. Forecasting Mood in Bipolar Disorder From Smartphone Self-assessments: Hierarchical Bayesian Approach. JMIR mHealth and uHealth 2020;8:e15028. pmid:32234702
- 38. Ta V, Griffith C, Boatfield C, Wang X, Civitello M, Bader H, et al. User Experiences of Social Support From Companion Chatbots in Everyday Contexts: Thematic Analysis. J Med Internet Res 2020;22:e16235. pmid:32141837
- 39. Jungmann SM, Klan T, Kuhn S, Jungmann F. Accuracy of a Chatbot (Ada) in the Diagnosis of Mental Disorders: Comparative Case Study With Lay and Expert Users. JMIR Formative Research 2019;3:e13863. pmid:31663858
- 40. Dosovitsky G, Pineda BS, Jacobson NC, Chang C, Escoredo M, Bunge EL. Artificial Intelligence Chatbot for Depression: Descriptive Study of Usage. JMIR Form Res 2020;4:e17065. pmid:33185563
- 41. Haines-Delmont A, Chahal G, Bruen AJ, Wall A, Khan CT, Sadashiv R, et al. Testing Suicide Risk Prediction Algorithms Using Phone Measurements With Patients in Acute Mental Health Settings: Feasibility Study. JMIR mHealth and uHealth 2020;8:e15901. pmid:32442152
- 42. Green EP, Lai Y, Pearson N, Rajasekharan S, Rauws M, Joerin A, et al. Expanding Access to Perinatal Depression Treatment in Kenya Through Automated Psychological Support: Development and Usability Study. JMIR Form Res 2020;4:e17895. pmid:33016883
- 43. Inkster B, Sarda S, Subramanian V. An Empathy-Driven, Conversational Artificial Intelligence Agent (Wysa) for Digital Mental Well-Being: Real-World Data Evaluation Mixed-Methods Study. JMIR Mhealth Uhealth 2018;6:e12106. pmid:30470676
- 44. Wahle F, Kowatsch T, Fleisch E, Rufer M, Weidt S. Mobile Sensing and Support for People With Depression: A Pilot Trial in the Wild. JMIR mHealth and uHealth 2016;4. pmid:27655245
- 45.
Aggarwal CC. Data Classification: Algorithms and Applications. CRC Press; 2014.
- 46. Schapire RE. A brief introduction to boosting. Ijcai, vol. 99, Citeseer; 1999, p. 1401–6.