US primary care in 2029: A Delphi survey on the impact of machine learning

Objective To solicit leading health informaticians’ predictions about the impact of AI/ML on primary care in the US in 2029. Design A three-round online modified Delphi poll. Participants Twenty-nine leading health informaticians. Methods In September 2019, health informatics experts were selected by the research team, and invited to participate the Delphi poll. Participation in each round was anonymous, and panelists were given between 4–8 weeks to respond to each round. In Round 1 open-ended questions solicited forecasts on the impact of AI/ML on: (1) patient care, (2) access to care, (3) the primary care workforce, (4) technological breakthroughs, and (5) the long-future for primary care physicians. Responses were coded to produce itemized statements. In Round 2, participants were invited to rate their agreement with each item along 7-point Likert scales. Responses were analyzed for consensus which was set at a predetermined interquartile range of ≤ 1. In Round 3 items that did not reach consensus were redistributed. Results A total of 16 experts participated in Round 1 (16/29, 55%). Of these experts 13/16 (response rate, 81%), and 13/13 (response rate, 100%), responded to Rounds 2 and 3, respectively. As a result of developments in AI/ML by 2029 experts anticipated workplace changes including incursions into the disintermediation of physician expertise, and increased AI/ML training requirements for medical students. Informaticians also forecast that by 2029 AI/ML will increase diagnostic accuracy especially among those with limited access to experts, minorities and those with rare diseases. Expert panelists also predicted that AI/ML-tools would improve access to expert doctor knowledge. Conclusions This study presents timely information on informaticians’ consensus views about the impact of AI/ML on US primary care in 2029. Preparation for the near-future of primary care will require improved levels of digital health literacy among patients and physicians.


Introduction Background
Attention in medicine and related fields has increasingly focused on the potential of big data, artificial intelligence (AI), and machine learning (ML) to change the delivery of healthcare (1)(2)(3)(4) Much of this debate has focused on the promise of AI/ML to augment or even disintermediate the clinical roles of physicians in gathering and monitoring patient health information, and to undertake core tasks such as diagnostics, prognostics, and the formulation of personalized patient healthcare plans. (1,(4)(5)(6)(7)(8)(9) Differentiating the hype from hope in the discourse about AI/ML in medicine is crucial to better understand the scope for the computerization of medicine. Although broad predictions of the impact of AI/ML on healthcare are ubiquitous, credible short-term predictions are necessary to address questions about resource allocation, and the adequacy of medical education and training.

Objectives
Recently a number of surveys have explored medical students', and physicians' views about the impact of AI/ML on the future of medical practice (10)(11)(12)(13)(14)(15)(16). Currently, there is scarce exploration of consensus views among informaticians on how AI/ML might meaningfully influence medical care in the short-term (17). To address this research gap, we designed a Delphi survey to explore leading health informaticians' predictions about the impact of machine learning on primary care in the US in 2029. To our knowledge, this is the first investigation of experts' opinions about the impact of AI/ML on the future of the near future of general medical practice.

The Delphi Method
The Delphi Method, developed by the Rand Corporation in the 1950s (18,19) is designed to pool the opinions of a purposive sample of identified experts in a given field to establish consensus predictions (20,21). Delphi polls rely on non-probability sampling techniques to identify a panel of experts: since participants are not randomly selected, representativeness is neither intended nor assured (22). The selected panel of experts is invited to answer a series of questions anonymously (22). Participants are next asked to reassess their initial judgments in light of group trends until consensus is obtained (23). This anonymous, iterative technique carries distinctive advantages over focus groups by avoiding the influences of individual dominant personalities, group-think, and helps to keep participants 'on topic' (18,22).
Delphi surveys are particularly well suited to exploring consensus views related to new lines of inquiry, and for establishing goal-setting, and needs assessments in policy-making (18,22). Since Delphi polls provide more accurate predictions than other forecasting methodologies, the approach is often used as a policy and practice heuristic for health care management, and resource allocation (24)(25)(26).

Approach
We used a modified Delphi technique which is structured into three discrete rounds (18,22,27,28). In Round One, questions are open-ended, requiring free-text answers. Responses are aggregated and coded into a series of statements. In Round Two, experts are provided with this list of statements, and requested to provide their level of agreement with each item.
Depending on the survey items, Round 2 and 3 questionnaires requested 'yes' or 'no' responses, or participants' level of agreement with statements on 7-point Likert scales: 1 = greatly decrease, 2 = moderately decrease, 3 = slightly decrease, 4 = remain the same, 5 = slightly increase, 6 = moderately increase, 7 = greatly increase; 1 = very unlikely, 2 = moderately unlikely, 3 = slightly 6 unlikely, 4 = uncertain, 5 = slightly likely, 6 = moderately likely, 7 = very likely; or 1 = strongly disagree, 2 = moderately disagree, 3 = slightly disagree, 4 = neutral, 5 = slightly agree, 6 = moderately agree, 7 = strongly increase. Those statements that reach a predefined level of agreement are omitted to reduce participant survey fatigue, and items that lack consensus are recirculated via a final anonymous poll. In the third and final round, panelists are reminded of their own response to the remaining statements as well as the median response of other experts, and are invited to preserve or revise their answer. A key aim of Delphi methodology is to maintain as high a response rate as possible rounds (22,29,30), and the accuracy of forecasts has been demonstrated to improve between each round (31).

The Expert Panel
Although there is no universally agreed sample size for Delphi polls (22), our aim was to balance the size of the panel with a high response rate between the three rounds. We therefore aimed to achieve a panel of around 12-15 individuals who would agree to share their expertise, and be committed to giving their time to respond to each round. Using purposive sampling methodology, the research team compiled a list of 27 highly trained and knowledgeable individuals with context-specific knowledge about health informatics and primary care in the US. Addressing the question about how to identify domain-specific 'experts', our goal was to prioritize panelists for their recognized competence in the field of health informatics. We defined expertise to mean a person who had published significant contributions within the field of health informatics, and/or individuals who were currently appointed as research leaders, or as health information officers. Acknowledging that heterogeneous panels have been shown to result in more accurate estimates (32), and that what counts as an expert can be influenced by goals, values, and the manner in which knowledge is generated, we aimed to recruit diverse participants 7 from across academia, healthcare, non-profit organizations, and industry; and to strive for panelists with a varied complementarity of interests within health informatics. Measures were also taken to ensure demographic diversity among invited participants along the lines of gender, age, nationality, and race/ethnicity. Prospective panelists were contacted via email in September 2019, with an invitation and internet link to the survey. Individuals were informed that we desired a commitment on the part of experts to respond to all three rounds, that adequate response time would be given to answer each round of the survey, participation was voluntary and unpaid, and that participants could withdraw at any time. Prospective participants were also informed that they would remain anonymous to other participants, their individual responses would not be shared with other panelists, and their contribution would be confidential. Respondents' names were also replaced with a study ID number by AK in order to preserve participant anonymity among other team members in data analysis.

The Questionnaire
We created an electronic questionnaire on JISC Online Surveys hosted at the University of Plymouth, UK (https://www.onlinesurveys.ac.uk/). The poll incorporated a three-step modified Delphi method which took place between September 2019 and January 2020. Ethical approval was provided by Beth Israel Deaconess Medical Center, Boston, USA, and the University of Plymouth, UK. Participants were sent 3 reminders after each round of the survey, and given 4-6 weeks to respond to Rounds 1 and 2, and 8 weeks to respond to Round 3 which fell over the New Year period.
In the first round, the Delphi survey requested demographic information; this was followed by 5 sections, with 7 open-ended questions, on the impact of machine learning on primary care by . CC-BY-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 11, 2020. . 8 2029 (see Appendix 1; Table 1  . CC-BY-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 11, 2020. . https://doi.org/10.1101/2020.05.08.20095224 doi: medRxiv preprint Responses to Round 1 were collated and coded into lists of statements. Coding was conducted by CB and independently reviewed by CL and AK, and subsequent revisions were made. Comments that were unrelated to the themes, or were deemed redundant were eliminated. Similar statements were grouped together and translated into concise items; whenever possible, replication of exact phrasing by participants was employed. These items were circulated in Round 2, and an online survey was sent to each individual member of the panel. Participants were requested to respond to categorical variables by selecting a 'yes' or 'no' response, and to questions with continuous variables by using predefined 7-point Likert scales (see Appendix 2).
Prior to consensus analysis of responses in Rounds 2 and 3, for categorical variables consensus was set at ≥ 75%, and for continuous variables consensus along 7-point semantic differential scales was set at an interquartile range of ≤ 1 (18,33). After analysis of Round 2 results, items that did not reach consensus were redistributed for Round 3. In Round 3, each participant received a personalized survey link. Panelists were reminded of their response to items in Round 2, and provided with the median collated response of the other participants.
. CC-BY-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted May 11, 2020. .

1
n -count, m -average value per sample, % -percentage of the sample, rounded to the nearest whole value, SD -standard deviation. Average value and SD were calculated only for age. 1 Round 1 calculations exclude the one non-eligible respondent. 2 Questions for which some participants selected more than one option.
Responses to Round 1 were translated into itemized lists of statements. As a result of this process, the was survey was expanded into 57 items arranged into 7 sections: (i) diagnostic accuracy (10 items), (ii) healthcare disparities (5 items), (iii) empathic care of patients (8 items), (iv) access to care (9 items), (v) primary care workforce (7 items), (vi) technological advancements in primary care (10 items), and (vii) the long-term future of the profession (8 items) (see Appendix 2). The panel was repeatedly prompted to forecast changes to primary care by 2029, and questions emphasized that predictions should be restricted to the US context.
Throughout the survey experts were reminded to "predict what you believe will happen and not what you personally would like to see happen". After completing each section, participants were also invited to provide free text comments, and following completion of the survey, offered to provide any additional feedback.

Rounds 2 and 3
In Round 2, 13/16 experts participated in the online survey (response rate of 81%). In Round 3, 13/13 experts responded (response rate of 100%). In Rounds 2 and 3, participants included 4 (31%) females, and 9 (69%) male participants (see Table 2 for demographic information). Table   3 presents the item means and standard deviations for item responses, and also indicates the items that reached consensus in Round 2, those that obtained consensus in Round 3, and items . CC-BY-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted May 11, 2020. . https://doi.org/10.1101/2020.05.08.20095224 doi: medRxiv preprint that failed to secure expert consensus. As described in the Methods, and as indicated in Table 3, items reflect three different 7-point Likert Scales. To undertake interpretation of panelists' predictions, these items were divided into three rational, a priori categories. For the scale identified as 'I', responses were bounded into items that experts expected to increase (item mean . CC-BY-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted May 11, 2020. .  .

CC-BY-ND 4.0 International license
It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted May 11, 2020. .

Primary Care Workforce
Informaticians disagreed with the prediction that in the US "there is a 90% chance primary care doctors will be obsolete 100 years from now". Panelists also agreed that primary care in the US would be one of the last specialties to be replaced by AI/ML. In the short-term, by 2029 in the US, experts forecast that advancements in AI/ML will incur a number of workforce changes in primary care (see Fig 2). Dependent on educational background some contrasting predictions emerged (see Table 4).
Beyond 2029, experts without a medical degree (MD) considered it likely that primary care doctors would always be needed to deliver empathic aspects of care -a prediction that did not engender consensus among panelists with a medical degree. Similarly, panelists without a medical education strongly agreed that the adoption of AI/ML tools in US healthcare will be slow, by 2029, due to the culture of medicine while those with a medical education did not reach consensus on this item. Conversely, experts with a medical degree forecast that by 2029 US doctors will transition from the role of dispensers of knowledge to managing teams and information systems; however, there was no consensus on this item among participants without an MD.

Diagnostic Accuracy
. CC-BY-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted May 11, 2020. .

8
Overall, our experts forecast that by 2029 AI/ML will increase rates of diagnostic accuracy especially for conditions where the markers of illness are relatively homogenous (see Fig 3).
Among their predictions, panelists envisaged that by AI/ML tools will improve diagnostic accuracy among persons with limited access to human experts, individuals identifying as from minority groups, or for those with rare conditions.

Access to Care
As a result of the disintermediation of physicians' expertise, our experts predicted that by 2029, AI/ML will increase access to primary care in the US (see Fig 4). Comparisons of ratings between participants with and without a medical education resulted in some divergence. Respondents without an MD predicted that, by 2029 as a result of AI/ML, patient access to medical care in the US will lag behind other developed countries; participants with an MD did not reach consensus on this item (see Table 4) . CC-BY-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted May 11, 2020. .  .

CC-BY-ND 4.0 International license
It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint 1 Scales were either increase (I) -1. greatly decrease to 7. greatly increase, likelihood (L) -1. very unlikely to 7. very likely, and agreement (A) -1. strongly disagree to 7. strongly agree. 2 One respondent had both medical and technological backgrounds so their data was excluded from both samples.

CC-BY-ND 4.0 International license
It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint

Empathic Care of Patients
Experts envisaged that by 2029 in the US, the availability of AI/ML tools will help to augment levels of empathic care (see Fig 5). Panelists were divided on whether, by 2029, AI will offer direct resources for delivering empathic care to patients (see Table 4). Participants with an MD considered this unlikely, while others failed to reach consensus on this item.

Summary of Major Findings
The collective forecasts of medical informaticians have been missing from discussions about how AL/ML will influence the short-term future of primary care (see Box 1). In this Delphi poll there was consensus that in the next decade in the US, AI/ML will engender training and primary care work forces changes, improve rates of diagnostic accuracy, and increase access to primary care.
Economists forecast that in the coming decades, AI/ML will revolutionize the workplace (34,35).
Taking a long view, informaticians in this Delphi poll predicted that 100 years from now it is unlikely that primary care doctors will be obsolete. Panelists further envisaged that primary care will be one of the last medical specialties to be displaced by technology. However, in the short . CC-BY-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted May 11, 2020. . https://doi.org/10.1101/2020.05.08.20095224 doi: medRxiv preprint term, by 2029, our experts did foresee workforce and training changes in US primary care as a result of AI/ML. Experts were collectively uncertain about whether AI/ML tools would enable lower-level clinicians to do higher level jobs, though it was not clear whether this prediction was driven by technological or regulatory considerations. Panelists anticipated a shift towards computing and engineering in the educational background of students entering medical school in 2029, and increased training demands on medical students to work with AI/ML in healthcare.
These predictions contrast with the currently limited debate about the need for curricular changes in medical education. (36)(37)(38) For many reasons, including financial, social, and geographical, timely access to primary care in the US remains a considerable problem. Compounding matters, with fewer medical students entering primary care, inefficiencies, and demographic changes -an ageing population, and more people suffering from chronic conditions for longer -it is widely envisaged that ambulatory medicine will become increasingly strained. (39,40) The results of this Delphi poll suggest that AI/ML tools may help to address some of these challenges. Experts envisaged that by 2029 there would be increased access to care via AI/ML-enabled tools for medical triage and routine patient self-diagnosis, and with the growth of telemedicine.
The panel also predicted increasing medical precision. By 2029, experts envisaged that the use of AI/ML-enabled tools among patients will help to reduce diagnostic errors both for diseases with homogenous symptoms, and for more difficult medical cases. Perhaps contributing to these reductions, experts anticipated that advancements in AI/ML will engender revisions in disease classifications. These positive predictions should be viewed against current evidence that diagnostic error is both common and harmful. In the US, recent estimates suggest a diagnostic error rate of 13-15% affecting the lives of around 12 million Americans annually, contributing to . CC-BY-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted May 11, 2020. . https://doi.org/10.1101/2020.05.08.20095224 doi: medRxiv preprint 10% of all deaths, and the highest proportion of medical malpractice claims. (41)(42)(43). Patients from racial and ethnic minorities, and those on low-incomes, are at higher risk of diagnostic error. (44) Our experts predicted that diagnostic accuracy will increase for individuals with limited access to care, minorities, or patients with rare conditions. While there are currently considerable concerns about the potential for algorithmic biases to be baked into AI/ML tools, driven in part by the underrepresentation of underprivileged demographic groups in training phases of machine learning (45), there was consensus among our Delphi panel that data collection in 2029 will be more representative of minority populations. This prediction may help to explain why the panel anticipated improved diagnostic accuracy for minorities.
Nonetheless, experts were less optimistic that AI/ML will narrow health disparities in the US by 2029. Current findings point to a "digital divide" in healthcare. Many factors drive current differential usage of digital health innovations including costs, lack of broadband access, and lower levels of digital and health literacy among underprivileged populations. (46,47) Research also suggests that in the US, health app usage is more common among people who are younger, better educated, on a higher income, or in better health (48). Our panel predicted that US healthcare will become increasingly productized. Although the poll provided no causal explanations for this prediction, in a growing health app economy, experts may have anticipated that disadvantaged patients will continue to be less likely to adopt eHealth tools. In addition, there was consensus that private hospitals will have greater access to AI/ML-enabled resources to improve diagnostic accuracy than public hospitals. Existing structural disparities in care may also have been perceived to be a factor that will perpetuate inequities in eHealth.
Our Delphi poll provided nuanced forecasts on the theme of physician empathy. There was collective consensus that, by 2029, AI/ML would not free up more time with patients in US . CC-BY-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted May 11, 2020. . primary care; however, experts did forecast that levels of empathy in primary care would increase in this time period. The survey did not fully illuminate the reasons for this but there was consensus that AI/ML-enabled tools will assist physicians in shared decision-making, and help provide information on patients' lifestyles and the social determinants of individuals' health.
Conceivably, the panel may have envisaged that such data might enhance physicians' personal knowledge about patients thereby fostering more empathic care. Again, these views appear to differ subtly from those of physicians. In qualitative research a common prediction among physicians is that, by liberating health professionals from administrative tasks, AI/ML will indirectly facilitate more time with patients thereby enhancing levels of empathy. (11,15) Survey research also indicates skepticism among physicians that AI/ML will be able to directly substitute for, or augment clinicians, in the provision of empathic care. (10)(11)(12)15) In terms of physicians' responsibilities, experts did not envisage that AI/ML will help to reduce documentation burdens by 2029 (10,12). This prediction contrasts with the more optimistic opinions of surveyed physicians. For example, in 2019, a global survey of psychiatrists found that the majority (83%, 657/791) judged it likely that future technology will fully replace physicians in the task of documentation with 84% (552/657) of these respondents predicting that this will happen in the next 10 years (12). Similarly, in 2018, survey research conducted among primary care physicians in the UK revealed comparable results: most UK general practitioners (80%, 578/720) anticipated that future technology will fully replace humans in the task of documentation with 79% (458/578) of these respondents believing that this will happen in the next decade (10).
. CC-BY-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted May 11, 2020. . https://doi.org/10.1101/2020.05.08.20095224 doi: medRxiv preprint Finally, experts in this Delphi poll did not weigh in on specific policy, legal, or ethical issues in relation to the impact of AI/ML on primary care. However, there was consensus that by 2029 regulatory issues will pose greater challenges than technical problems.

Strengths and Limitations
To our knowledge this is the first Delphi poll to explore experts' predictions about the short-term effects of AI/ML on a medical specialism. Major strengths of the survey were the high response rates between rounds, and the diversity of participants. Although only around one third of Round 3 panelists were female (31%), currently around 25% of health IT leaders in the US are women (49). The expert panel comprised leading health informaticians around half of whom also had a medical background. Panelists were drawn from diverse backgrounds, nationalities, and ethnicities including 3 participants in Round 1 who do not reside in the US but who are knowledgeable about the US healthcare system. We also note that the majority of experts primarily held allegiances to academia, and medicine, rather than industry; nonetheless, this may have been a strength rather than a limitation, resulting in more modest predictions. This survey has several limitations. As with all Delphi polls, there is no guarantee of accuracy in forecasts. No standardized guidelines exist for identifying, excluding, or selecting suitable experts from the field of interest (22,26). Reliability of predictions is dependent on the specialist knowledge of the participants which can be influenced by norms and values, motivational biases, and stakeholder interests (50,51). Although there was strong consensus among our panel of experts, we noted some divergence in opinions between participants with and without a medical degree. Conceivably, professional medical allegiances may have affected predictions; overall, however, we cannot speculate on how the composition of our panel strengthened or diminished the quality of predictions. Whilst participant retention rates between rounds were high, the . CC-BY-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted May 11, 2020. . https://doi.org/10.1101/2020.05.08.20095224 doi: medRxiv preprint number of panelists was limited, and more participants in the first round may have resulted in different consensus opinions (52,53).
Importantly, two events arising in the immediate period after data collection -one global and one in the US context -may affect the reliability of the Delphi poll. The coronavirus pandemic has (and currently is) exerting a significant impact on the delivery of primary care in the US.
Driven by this crisis, current evidence shows a substantial uptick in demand for telemedicine consultations, and in the use of AI/ML-driven triage tools. (54) Although it is too early to predict with certainty whether increase in these applications will persist after the pressure on frontline medicine has abated, it seems possible that our experts' forecasts on the influence of AI/ML on access to care may be especially well supported.
Second, the survey was administered prior to the finalized ruling, in March 2020, by the National Coordinator of Health Information Technology (ONC) on the 21 st Century Cures Act (55,56).
Designed to maximize innovation in healthcare by creating a competitive health app economy, this federal ruling sets out technical standards about how data must be shared, mandating patients' right to access their digital medical records. While the final ruling may have been anticipated by some of our experts in the months preceding the announcement, we cannot be certain about whether or how its publication might otherwise have influenced consensus predictions of our participants. However, we suggest that uncertainty prior to the ruling may have fostered more cautious predictions about the impact of AI/ML on primary care among our experts.

Conclusions
A good hockey player plays where the puck is. A great hockey player plays where the puck is going to be.
. CC-BY-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted May 11, 2020. This Delphi poll provides the consensus predictions of leading health informaticians on the impact of AI/ML on primary care in the US. The panel forecast that, in the long-term (100 years from now) primary care doctors will not be obsolete, and furthermore, that general medicine will be one of the last medical specialties to be displaced by technology. By 2029 in the US, however, experts did forecast that AI/ML will exert an impact on the delivery and quality of primary care.
Specifically, the panel predicted increased rates of diagnostic accuracy including for the most disadvantaged patient populations, greater access to primary care, and enhanced levels of empathic patient care. Against the panel's forecast that healthcare in the US would be increasingly productized, there was consensus that regulatory issues will pose greater challenges than technical ones in improving diagnostic accuracy. Experts were also less optimistic about the prospects of AI/ML to precipitate other desirable short-term changes in medicine. By 2029 in the US, the panel predicted that AI/ML would not narrow healthcare disparities, reduce documentation burdens on primary care physicians, or increase the total time spent with patients.
In the next decade, experts forecast increased AI/ML training requirements for medical students.
The central goal of Delphi polls is expert prediction. However, forecasts can also help us to exert control over the future by facilitating forward planning, and focusing attention on where, and how, relevant actors might intervene to create more preferable outcomes. Innovations in digital care pose myriad practical and regulatory issues including (but by no means limited to): the creations of standards for assessing the reliability and approval of medical algorithms and apps, questions about patient privacy, and the security of patients' online health information. In reviewing these findings we are struck by the contrastive predictions of our experts with those of . CC-BY-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 11, 2020. . https://doi.org/10.1101/2020.05.08.20095224 doi: medRxiv preprint surveyed physicians (10,11). As others have noted, medical schools have been slow to adapt curricula and offer courses aimed at promoting AI/ML literacy among students (36)(37)(38). We conclude that to empower both physicians and patients, and to rise to the challenges of the next decade, it is incumbent on the medical community, health and medical educators, and policymakers to take action to improve digital literacy both among patients and our future health professionals.

Declarations
Conflicting interests: The authors have declared no competing interests exist. . CC-BY-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 11, 2020. . https://doi.org/10.1101/2020.05.08.20095224 doi: medRxiv preprint