The application of machine learning and artificial intelligence to clinical settings for prevention, diagnosis, treatment, and the improvement of clinical care have been demonstrably cost-effective. However, current clinical AI (cAI) support tools are predominantly created by non-domain experts and algorithms available in the market have been criticized for the lack of transparency behind their creation. To combat these challenges, the Massachusetts Institute of Technology Critical Data (MIT-CD) consortium, an affiliation of research labs, organizations, and individuals that contribute to research in and around data that has a critical impact on human health, has iteratively developed the “Ecosystem as a Service (EaaS)” approach, providing a transparent education and accountability platform for clinical and technical experts to collaborate and advance cAI. The EaaS approach provides a range of resources, from open-source databases and specialized human resources to networking and collaborative opportunities. While mass deployment of the ecosystem still faces several hurdles, here we discuss our initial implementation efforts. We hope this will promote further exploration and expansion of the EaaS approach, while also informing or realizing policies that will accelerate multinational, multidisciplinary, and multisectoral collaborations in cAI research and development, and provide localized clinical best practices for equitable healthcare access.
The development of effective and scalable algorithms that are executable at the frontlines of clinical care require robust data infrastructure, data sharing and usage capabilities that do not compromise patient confidentiality, and flexibility in creating partnerships across various departments, industries, and sectors. To achieve this in a sustainable and replicable manner, the EaaS approach has three components; 1. A global coalition of AI clinicians accessible via the MIT-CD network to support the translation, customization, innovation, and external validations of algorithms, 2. Training opportunities provided via MIT-CD affiliated institutions to educate clinically informed engineers and data literate clinicians, and introduce them to open source databases, and 3. Networking opportunities and events offered by the MIT-CD consortium to share best practices and offer chances for collaboration on innovative research in multidisciplinary teams.
Citation: Ishii-Rousseau JE, Seino S, Ebner DK, Vareth M, Po MJ, Celi LA (2022) The “Ecosystem as a Service (EaaS)” approach to advance clinical artificial intelligence (cAI). PLOS Digit Health 1(2): e0000011. https://doi.org/10.1371/journal.pdig.0000011
Editor: Mecit Can Emre Simsekler, Khalifa University of Science and Technology, UNITED ARAB EMIRATES
Published: February 3, 2022
Copyright: © 2022 Ishii-Rousseau et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The author(s) received no specific funding for this work.
Competing interests: The author(s) have no competing interests to declare for this work.
Machine learning algorithms have been lauded as indispensable tools in solving complex challenges across various fields ranging from photography to self-driving vehicles, to even AI-augmented legal defense . The wide-ranging applicability of these algorithms have brought about the development of services that aim to provide largely turnkey solutions for non-experts to build up their own algorithms given arbitrary datasets and labels. While the algorithms that are produced may initially seem of high quality, when put into practice they tend to suffer from significant issues such as the inability to generalize , systemic bias , and in some cases, inaccurate and non-reproducible predictions. The development of effective and scalable algorithms that are executable at the frontlines of clinical care require robust data infrastructure, data sharing and usage capabilities that do not compromise patient confidentiality, and flexibility in creating partnerships across various departments, industries, and sectors .
As interest in the application of machine learning in healthcare soars, similar turnkey approaches are being observed in healthcare. Not surprisingly, such efforts have translated poorly to clinical practice , as clinical AI (cAI) support tools are predominantly created by non-domain experts . Reliable cAI requires large-scale data availability for development with subsequent local population level re-development and tuning [2,5], while ensuring clinical workflow compatibility. Yet, healthcare facilities are reluctant to openly share data and few healthcare professionals are capable of integrating data science and biomedical informatics with clinical insight . Additionally, while cAI has been demonstrably cost-effective [6,7], only 4 of the 130 Food and Drug Administration (FDA) approved AI devices have been prospectively validated [8,9]. These challenges are further exacerbated when unvalidated and possibly non-generalizable algorithms are purchased by hospitals with the intention of improving clinical outcomes and resolving disparities in healthcare delivery. This not only marginalizes hospital systems that lack the resources for full-scale cAI development and procurement, but the product purchased may be unusable, with current algorithms demonstrating a lack of transparency behind their creation, and particularly with regard to any biases in the data used [10,11]. Furthermore, in the era of COVID-19, the demand for dynamic, lean, and multilateral efforts have become ever more apparent.
2. cAI and its current challenges
cAI is the application of machine learning and AI that assists with clinical decisions through process optimization, advancements in basic and clinical research and development, diagnostic and prognostic clinical decision support, in-hospital and extra-hospital patient support tools, and population-level applications to identify epidemics and understand non-communicable chronic diseases . cAI development requires large and well-managed datasets permitting initial algorithm construction, a clinical validation feedback loop consisting of technical and clinical experts, localized application that considers the target community’s demographic and socioeconomic contexts, infrastructure for visualization deployment, and financial resources, all of which are requirements that differ based on the clinical question posed. These requirements have also locked cAI within major academic medical centers, limiting current options for cAI integration to institutions independently devoting a large sum of resources so as to build an in-house team, or to institutions opting-out of using cAI tools entirely, potentially placing one’s hospital at distinct disadvantage (Table 1).
cAI benefits from multicentered, multi- or transnational, and culturally relevant approaches, which unfortunately may be incompatible with the structure of traditional academic medical systems. Although purchasing cAI algorithms may be efficient to implement [14,15], a vast number of factors determine and influence health systems, and the creation of cAI itself without extensive validation is insufficient to spur sustainable change. Collecting data in a manner that can train cAI to suit local populations and combat bias requires considerable amounts of scrutiny and resources; further, cAI specialists are also scarce  and the returns on investments are yet to be determined [16,17]. In addition, comprehensive understanding of each target clinical setting requires teamwork, domain expertise, and long-term commitment. Table 2 describes the overview of cAI stakeholders.
3. The “Ecosystem as a Service (EaaS)” approach
To overcome the challenges in cAI development and implementation, while leveraging the strengths of each stakeholder, the MIT Critical Data (MIT-CD) consortium developed the “Ecosystem as a Service (EaaS)” approach for sustainable and replicable cAI utilization. The MIT-CD consists of university departments, laboratories, independent academic institutions, research groups, national centers, industry representatives, and medical departments from around the globe in the field of computer science, electrical engineering, physics, or mathematics that are engaged in research in and around data that has a critical impact on human health. The EaaS approach began with the MIT Laboratory for Computational Physiology (LCP)’s efforts to use modeling, signal processing, pattern recognition, and machine learning for the development and refinement of methods to analyze data and generate predictive models for patient care. The LCP launched the Medical Information Mart for Intensive Care (MIMIC) database in the early 2000s, an open-source and free dataset of more than 60,000 de-identified intensive care unit admission records, which has since become the golden standard for public clinical databases [18,19]. The LCP network grew as interest in the MIMIC database and collaborators increased, and developed into its current MIT-CD coalition with the launch of the initial MIT Critical Data Conference in 2014 which invited academia, government, and industry leaders to discuss the pitfalls and challenges of big data in health care . This also brought about the emergence of “datathons”, (a portmanteau of “data” combined with “hackathon”, indicating the application of the hackathon model to data analytics) which convenes experts from various scientific disciplines and industry involved in creating and using patient data and algorithms for collaboration, group learning, error checking, and methodological review during the initial design and subsequent phases of cAI development and research . The combination of these efforts with iterative feedback and refinement with various stakeholders has shaped the current EaaS approach, which is described in Fig 1.
Overview of the EaaS approach for cAI with its three main components: 1. Global coalition of AI clinicians, 2. Training opportunities, and 3. Networking opportunities, integrated with an individual hospital system.
The EaaS approach has three components; 1. A global coalition of AI clinicians offered via the MIT-CD network to support the translation, customization, innovation, and external validations of algorithms, 2. Training opportunities provided via MIT-CD affiliated institutions to rear clinically informed engineers and data literate clinicians, and introduce them to open source databases, and 3. Networking opportunities and events offered by the MIT-CD consortium to share best practices and offer chances for collaboration on innovative research in multidisciplinary teams. Within the MIT-CD consortium, these areas are supported via grants or external sponsorships, and clinicians, engineers, and institutions are able to participate in the ecosystem and join the MIT-CD consortium at a very low cost or for free, depending on the type of partnership. The EaaS approach considers accessibility to cAI for all hospitals regardless of location and available resources and contributes to addressing and solving global health challenges. Further expansion of the ecosystem including availability of data and its sustainability is currently being discussed.
The MIT-CD’s EaaS approach provides a transparent educational and accountability platform for clinical and technical experts to collaborate, develop, and advance cAI. Hospital systems that monetize raw clinical data may not initially be incentivized by this approach; yet raw clinical data without curation is analogous to crude oil, which without distillation has minimal use or value. During the mid-2010s, MIT-CD and Philips recognized the importance of establishing a decentralized education platform for analyzing free and open source clinical data and have generated significant academic development and dataset analysis beyond what each group alone could achieve since. In other words, the EaaS approach permits the sharing of perspectives, experiences, and expertise, and leads to the development of new technologies that are clinically meaningful, academic publications that aid in providing evidence for unanswered clinical questions, and contribute to overall improved value-based patient care. The EaaS approach can be used for seamless and effective cAI adoption within large healthcare systems while benefiting smaller, independent, rural, and less resourced partners around the world (Fig 2).
High-level depiction of the stakeholders included in the EaaS approach, its operational components, as well as its iterative nature.
MIT-CD’s initial implementation efforts of the EaaS approach have shown promising results in improving healthcare access and clinical care through new and refined approaches to interpreting data along with partner universities and research laboratories around the world. Since 2018, MIT-CD has held the “Big Data Machine Learning in Healthcare” datathon events in Japan, which was co-hosted with Google in 2019, bringing together over 200 individuals and a variety of key academic institutions and organizations involved in patient care and healthtech service provision . These events have not only provided potential solutions to realize the Japanese government’s “Society 5.0” policy aimed to lead Japan to become the first country to achieve a human-centered society by incorporating advanced technologies in diverse industries and social activities and fostering innovation , but have also led to the development of national-level ICU mortality prediction models to further customized care and treatment [23,24].
In 2019, MIT-CD also held the first “Big Data Bhutan and Asia eHealth Information Network (AeHIN) Convergence Workshop on Enterprise Architecture” together with the Khesar Gyalpo University of Medical Sciences of Bhutan (KGUMSB) and the Bhutanese Ministry of Health (MoH) . Bhutan is in its inaugural stages of embracing and adopting innovative technological development of AI, quantum computing, blockchain, machine learning, big data, Internet of Things (IOT), virtual reality, and augmented reality , especially to build capacity in order to leverage the advantages of these modern technologies for the improvement of healthcare services and other socio-economic development. Throughout the course of preparing and executing the workshop as well as planning the subsequent steps for addressing the discovered challenges, the MIT-CD ecosystem aided the rearing of human resources necessary to develop Bhutan’s health data infrastructure for the collection, storage, and maintenance of all health-related data, while facilitating networking and collaborative efforts between the Bhutanese government, academic institutions, private sector, and oversea partners. The EaaS approach is also currently being deployed in Africa and in Asia through partnerships that integrate private sector resources to enhance public sector health care access and building capacity for collaborative research and patient-centric tools.
4. Conclusion: Future prospects
The MIT-CD EaaS approach offers a sustainable and cost-effective option for cAI implementation and development among researchers, service providers, and consumers. Realizing the EaaS approach for mass deployment still faces several hurdles; it requires 1. the pooling of best practices across wider regions in an open, collaborative, and holistic manner, 2. a pricing structure that offers a wide variety of solutions to meet the needs of diverse healthcare systems while considering insurance and reimbursement, and 3. the accumulation, de-identification, and sharing of health data at the hospital, regional, and country level, permitting equitable access to healthcare data. The EaaS approach’s core model of interdisciplinary teamwork has thrived with the enhanced promotion of digital communication and global coordination and has been resilient even in the era of COVID-19. Although questions remain in how to translate the movement to an independent industrialized consortium, with further exploration and expansion, the EaaS approach could permit acceleration of multinational, multidisciplinary, and multisectoral collaborations in cAI research and development, providing localized clinical best practices for equitable healthcare access.
- 1. Sawers P. DoNotPay’s legal bots help consumers fight the system during lockdown. In: VentureBeat [Internet]. 23 Jun 2020 [cited 12 Jun 2021]. Available: https://venturebeat.com/2020/06/23/donotpays-legal-bots-help-consumers-fight-the-system-during-lockdown/
- 2. Panch T, Mattie H, Celi LA. The “inconvenient truth” about AI in healthcare. NPJ Digit Med. 2019;2: 77. pmid:31453372
- 3. Z O, Obermeyer Z, Powers B, et al. Vogeli C. Dissecting racial bias in an algorithm used to manage the health of populations. Yearbook of Paediatric Endocrinology. 2020.
- 4. Cosgriff CV, Stone DJ, Weissman G, Pirracchio R, Celi LA. The clinical artificial intelligence department: a prerequisite for success. BMJ Health Care Inform. 2020;27. pmid:32675072
- 5. Winter is coming. HealthTech is here—Demos. 8 Jan 2019 [cited 13 Jun 2021]. Available: https://demos.co.uk/project/winter-is-coming-healthtech-is-here/
- 6. ARTIFICIAL INTELLIGENCE: Healthcare’s New Nervous System. [cited 13 Jun 2021]. Available: https://www.accenture.com/_acnmedia/PDF-49/Accenture-Health-Artificial-Intelligence.pdf#zoom=50
- 7. Hanna MG, Reuter VE, Samboy J, England C, Corsale L, Fine SW, et al. Implementation of Digital Pathology Offers Clinical and Operational Increase in Efficiency and Cost Savings. Arch Pathol Lab Med. 2019;143: 1545–1555. pmid:31173528
- 8. Wu E, Wu K, Daneshjou R, Ouyang D, Ho DE, Zou J. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nat Med. 2021;27: 582–584. pmid:33820998
- 9. Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan. [cited 13 Jun 2021]. Available: https://www.fda.gov/media/145022/download
- 10. Char DS, Shah NH, Magnus D. Implementing Machine Learning in Health Care—Addressing Ethical Challenges. N Engl J Med. 2018;378: 981–983. pmid:29539284
- 11. Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Med. 2019;17: 195. pmid:31665002
- 12. Smith H. Clinical AI: opacity, accountability, responsibility and liability. AI & SOCIETY. 2020.
- 13. Celi LA, Davidzon G, Johnson AE, Komorowski M, Marshall DC, Nair SS, et al. Bridging the Health Data Divide. J Med Internet Res. 2016;18: e325. pmid:27998877
- 14. Himmelstein DU, Wright A, Woolhandler S. Hospital Computing and the Costs and Quality of Care: A National Study. The American Journal of Medicine. 2010. pp. 40–46. pmid:19939343
- 15. Hospital Financial Analysis. 16 Jul 2017 [cited 13 Jun 2021]. Available: https://truecostofhealthcare.org/hospital_financial_analysis/
- 16. McCoy LG, Nagaraj S, Morgado F, Harish V, Das S, Celi LA. What do medical students actually need to know about artificial intelligence? NPJ Digit Med. 2020;3: 86. pmid:32577533
- 17. Esmaeilzadeh P. Use of AI-based tools for healthcare purposes: a survey study from consumers’ perspectives. BMC Med Inform Decis Mak. 2020;20: 170. pmid:32698869
- 18. Website. Available: Johnson A, Pollard T., Shen L et al. MIMIC-III, a freely accessible critical care database. Sci Data 3, 160035 (2016). pmid:27219127
- 19. Johnson AEW, Stone DJ, Celi LA, Pollard TJ. The MIMIC Code Repository: enabling reproducibility in critical care research. J Am Med Inform Assoc. 2017;25: 32–39.
- 20. Badawi O, Brennan T, Celi LA, Feng M, Ghassemi M, Ippolito A, et al. Making Big Data Useful for Health Care: A Summary of the Inaugural MIT Critical Data Conference. JMIR Medical Informatics. 2014;2: e3447. pmid:25600172
- 21. Celi LA, Lokhandwala S, Montgomery R, Moses C, Naumann T, Pollard T, et al. Datathons and Software to Promote Reproducible Research. J Med Internet Res. 2016;18: e230. pmid:27558834
- 22. Big Data Machine Learning in Healthcare in Japan. [cited 8 Nov 2021]. Available: http://datathon-japan.jp/
- 23. Society 5.0. [cited 8 Nov 2021]. Available: https://www8.cao.go.jp/cstp/english/society5_0/medical_e.html
- 24. Website. Available: Endo H., Uchino S., Hashimoto S. et al. Development and validation of the predictive risk of death model for adult patients admitted to intensive care units in Japan: an approach to improve the accuracy of healthcare quality measures. j intensive care 9, 18 (2021). pmid:33588956
- 25. Health Big Data and EA Convergence Workshop. [cited 8 Nov 2021]. Available: https://www.moh.gov.bt/health-big-data-and-ea-convergence-workshop/