To describe the opinions of British general practitioners regarding the potential of future technology to replace key tasks carried out in primary care.
Main outcome measures
Investigators measured GPs’ opinions about the likelihood that future technology will be able to fully replace–not merely aid–the average GP in performing six primary care tasks; in addition, if GPs considered replacement for a particular task likely, the survey measured opinions about how many years from now this technological capacity might emerge.
A total of 720 (49%) responded to the survey. Most GPs believed it unlikely that technology will ever be able to fully replace physicians when it comes to diagnosing patients (489, 68%), referring patients to other specialists (444, 61%), formulating personalized treatment plans (441, 61%), and delivering empathic care (680, 94%). GPs were not in agreement about prognostics: one in two participants (380, 53%) considered it likely that technology will be fully capable of replacing physicians in performing this task, nearly half (187, 49%) of whom believed that the technological capacity will arise in the next ten years. Against these findings, the majority of GPs (578, 80%) believed it likely that future technology will be able to fully replace humans to undertake documentation; among them 261 (79%) estimated that the technological wherewithal would emerge during the next ten years. In general, age and gender were not correlated with opinions; nor was reported burnout and job satisfaction or whether GPs worked full time or part time.
The majority of UK GPs in this survey were skeptical about the potential for future technology to perform most primary care tasks as well as or better than humans. However, respondents were optimistic that in the near future technology would have the capacity to fully replace GPs’ in undertaking administrative duties related to patient documentation.
Citation: Blease C, Bernstein MH, Gaab J, Kaptchuk TJ, Kossowsky J, Mandl KD, et al. (2018) Computerization and the future of primary care: A survey of general practitioners in the UK. PLoS ONE 13(12): e0207418. https://doi.org/10.1371/journal.pone.0207418
Editor: Lars-Peter Kamolz, Medical University Graz, AUSTRIA
Received: September 20, 2018; Accepted: October 30, 2018; Published: December 12, 2018
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: This work was conducted with support from Harvard Catalyst | The Harvard Clinical and Translational Science Center (National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health Award UL1 TR002541 and financial contributions from Harvard University and its affiliated academic health care centers; it was also supported by an Irish Research Council-Marie Skłodowska Curie Global Fellowship (CLNE/2017/226) and a grant from the University of Basel, Switzerland. Additional funding: CB was supported by a Fulbright Award. MHB was supported by National Institute of Health grant number T32DA016184. The study funders played no role in the study design; writing of the report; or the decision to submit the manuscript for publication. Researchers were independent of influence from study funders.
Competing interests: The authors have declared that no competing interests exist.
In 2013 economists Carl Frey and Michael Osborne at Oxford University’s Martin School estimated that in “in the next decade or two” more than half of all jobs would be substituted by intelligent technology . In their forecast Frey and Osborne predicted that–at least in the short term–the work done by physicians and surgeons would be “low risk” (<1%) for automation, a prediction that is also shared by authors of a new working paper by the international Office of Economic Cooperation and Development . According to some artificial intelligence (AI) experts, however, these predictions are too modest and machine level intelligence will be capable of replacing the basic functions of medical professionals by 2050 .
Within the medical community there has also been an upsurge in debate about the potential of big data and machine learning to disrupt healthcare professions [4,5]. In the UK ongoing pilots of chatbot apps covering 7.5 million people and aimed at triaging NHS care have attracted concerns about safety, raising questions about the testing of these devices and the regulation of the digital health industry [6,7]. Beyond the potential–or otherwise–of chatbots, opinions about the threat of AI appear to be divided. Some physicians continue to express skepticism arguing that, “concerns about physician ‘unemployment’ and ‘de-skilling’ are overblown” . However, other physicians working in biomedical informatics and related fields predict that developments in AI are poised to revolutionize the delivery of health care [9,10,11,12], with some suggesting advancements may eventually obviate the need for physicians altogether .
Despite the abundant debate about the future of medicine, to our knowledge there is no exploration of the views of practicing physicians on the impact of AI on health professions. Given that much of the dialogue about machine learning technologies pivots on primary care, this survey seeks to determine the opinions of primary care physicians on the future of their job. We focus on currently registered general practitioners (GPs) working in the UK. Decomposing the work of primary care physicians into various foundational tasks (e.g. analyze patient information to reach diagnoses; provide empathic care, etc.) the survey asks UK GPs their views about whether, and when, future technology will be able to perform these tasks as well as or better than the average physician.
Participants in this cross-sectional national survey were randomly sampled from among UK-based general practitioners registered with the clinician marketing service Doctors.net.uk. This is the largest online medical network for medical doctors in the UK, providing information services to 228,317 physicians. In total, 87% of UK GPs are registered with the service . Ethical approval for the study was obtained from Beth Israel Deaconess Medical Center, Harvard Medical School and University College Dublin.
We required 678 responses to reflect the population of registered UK GPs with 99% confidence (±5%). Based on previous surveys using Doctors.net.uk we predicted a response rate of around 46% [15,16,17]. Invitations to participate were emailed and displayed on the Doctors.net.uk homepages of 1474 randomly selected physicians during a period of one week, from 12th June to 18th June 2018. Our sample was stratified by gender and age using demographic information about currently registered UK GPs provided by the General Medical Council (GMC) . All invited GPs were assured that their identities would not be disclosed to investigators, and participants gave informed consent before taking part in the survey. A small incentive of £10 ($13, €11) worth of ‘eSR’ points (exchangeable as shopping vouchers) was provided on completion, and participants were required to respond to every question to complete the survey. Further questions were embedded within the survey to determine whether respondents were currently practicing as GPs in the UK.
In light of the absence of research among physicians, the study team devised an original survey instrument to investigate the views of GPs about the potential impact of technology on the primary care profession (see S1 File). We developed the survey instrument in consultation with British and American general practitioners and piloted the survey with GP colleagues in the UK and USA (n = 12) to ensure face validity.
While the roles of primary care physicians vary from country to country, our aim was to formulate a generic list of common tasks performed in primary care with the ancillary goal that the survey might also be used for data collection beyond the UK. We used a validated primary care physician task list that encompasses 12 major professional functions and consulted with primary care physicians from the UK and USA to refine the list to six core tasks . This list included: “analyze patient information to reach diagnoses”; “analyze patient information to reach prognoses”; “evaluate when to refer patients to other health professionals”; “formulate personalized treatment plans”; “provide empathic care to patients”; and “provide documentation (e.g., update medical records) about patients”. We aimed to ensure that the language in the task list was neutral and not biased in favor of either human physicians or technology in carrying out these functions.
In designing the survey instrument, we recognized the widespread acceptance among AI experts and informaticians that machine learning is likely to surpass humans in performing a number of core medical tasks, yet also noted some widespread vagueness in predictions about how, if at all, humans and machines might work together . Many informaticians suggest that humans will be bypassed altogether for some tasks [9,10,13]; meanwhile, others argue that physicians will always be needed to manage patient care, and/or to undertake collaborative or ancillary responsibilities of key roles [8,21]. To avoid potential ambiguities in how respondents interpreted the questions and response options, we focused on whether tasks were likely to be fully rather than partially outsourced to technology. We also aimed to allow respondents to express discriminatory opinions about the tasks they considered most (or indeed least) vulnerable to replacement by machine learning. Finally, because the term “machine learning” may be unfamiliar among some physicians and considered too narrow a description among medical AI researchers, we employed generic language such as “machines” and “technology” to refer to AI innovations.
The first set of six items opened with a brief statement: “Some people believe that current and future innovations in artificial intelligence will lead to significant changes in medical practice and that machines will one day replace the work of physicians. Others deny that new technologies will ever have the capacity to replace this work”. We then asked respondents their opinion on the likelihood that “future technology will be able to fully replace and not merely aid human doctors in performing each task as well as or better than the average GP.” Employing 4-level Likert items we included the following response options: “extremely unlikely”, “unlikely”, “likely, and “extremely likely”. We avoided “don’t know”, “neutral” or “no opinion” options on the grounds that respondents often conflate these answers . Furthermore, inclusion of these choices may have precluded measurement of substantive opinions among GPs: research indicates that this is a risk in self-administered questionnaires where time-pressured individuals may invest less effort in their answers . Participants who responded that replacement was “likely” or “very likely” were asked a follow-up question about how soon in their estimation technology would have the capacity to perform the task and provided with a list of five response options.
A second set of items asked participants about their perceptions of the GP workforce. Using a five-point Likert scale we asked respondents for their opinions about whether the number of (i) GPs in the UK, and (ii) in their own practice, met patient demand. In the third section, respondents were surveyed on their current employment. We included two items about career satisfaction and burnout. To ensure that our survey was not over-long we employed Dolan et al.’s single-item measure of burnout that has been shown to be a reliable instrument for evaluating emotional exhaustion among primary care staff [24,25]. In this section we also asked participants about their work status (full time or part time); number of hours worked per week; and an estimate of the number of face-to-face patients that they saw each day. The purpose of these seven questions was to investigate whether physicians’ perceptions of the workforce, and/or current working conditions, were correlated with opinions about the future of medicine. A final section requested demographic information.
Data management and analysis
We used descriptive statistics to examine physicians’ characteristics and opinions about the likelihood of future technology replacing GPs on six primary care tasks. In our analysis responses to each of the six tasks were collapsed into positive (for “likely” or “extremely likely” responses) versus negative (for “unlikely” or “extremely unlikely”) opinions. All correlation analyses were conducted using Spearman’s rho; all group comparisons were conducted using the Mann-Whitney U test. These analyses were chosen because, unlike Pearson’s r and t-tests, the dependent variable does not need to be continuous, and because most dependent variables in our analyses were ordinal outcomes. Alpha was set at 0.05 (two-tailed) for all analyses. All analyses were conducted using SPSS version 24.
Of the 1474 GPs who were invited to participate 720 responded (an overall response rate of 49%). There were more male (55%) respondents, the average year of qualification was 1996 (range 1963 to 2012, mode = 1998), and 51% were aged 45 or older. Most participants studied medicine in the UK (87%), with a sizeable minority describing themselves as of Asian, Black or minority ethnicity (30%). Respondents were from all regions of the UK. More of our respondents worked part time (54%). 83% of full-time participants reported working 41 hours or more each week while 78% of part time GPs reported working 21 hours or more. Among GPs who worked full time, the average number of patients seen per day was 34.7 (SD = 9.13) compared with 31.35 (SD = 9.41) for part time GPs (Table 1).
Our participants varied from those registered with the General Medical Council (GMC) when it came to gender: 47% of UK general practitioners are male. However, respondents were of similar age to currently registered UK GPs where the mode year of qualification is 2000 and 55% are aged 45 and older. The GMC does not collect data on whether GPs work part time or full time.
Opinions about technological replacement
Almost seven in ten participants (68%, 489/720) believed it is unlikely or extremely unlikely that technology will ever be able to fully replace human physicians when it comes to diagnostics (Table 2). Among those who believed replacement was likely or extremely likely approximately one third (35%, 82/231) estimated that the technological capacity would emerge in the next ten years (Table 3). When asked about referral of patients to other health professionals, 61% (444/720) considered it unlikely or extremely unlikely that technology would ever be able to perform this task as well as or better than the average GP; however, of the 39% who believed replacement was likely or extremely likely, 58% (160/276) estimated that the technological capability would arise in 0 to 10 years, including 15% (41/276) who gave an estimate of 0 to 4 years from now. Similarly, 61% (441/720) considered it unlikely or extremely unlikely that technology could ever fully replace humans when it comes to formulating personalized patient treatment plans; of the 39% who disagreed, around one in ten (12%, 34/279) gauged that the technological capacity will be available in up to four years from now with a further 44% (124/279) giving an estimate of between 5 to 10 years. Most GPs (94% 680/720) believed that technology would never be able to provide empathic care as well as or better than the average GP.
Against these findings, however, just over half of the participants surveyed (53%, 380/720) considered it likely or extremely likely that humans would be fully replaced when it comes to making prognoses; of these respondents around one in two (49%, 187/380) estimated that the technological capacity to do so would emerge during the next ten years.
The majority of GPs (80%, 578/720) also judged it likely or extremely likely that future technology will be able to replace human physicians when it comes to the task of documentation. Among those who answered affirmatively, approximately one third (34%, 197/578) believed the technological wherewithal would emerge in the next four years, with an additional 45% (261/578) giving an estimate of 5 to 10 years from now.
Views about the GP workforce and employment
The overwhelming majority of participants (97%, 701/720) believed that the number of GPs in the UK is much less or somewhat less than patient demand. A similar number– 93% (667/720)–perceived the number of GPs in the community in which they practiced to be much less or somewhat less than patient demand. Nearly 4 in 10 GPs (38%, 272/720) reported that they were somewhat dissatisfied or very dissatisfied with their career, and 38% (269/720) reported burnout. These responses were similar to other reports on physician burnout in the UK . There was a strong correlation between respondents’ level of reported satisfaction and burn out (ρ = 0.511, p<0.001). The number of patients seen by GPs per day was weakly correlated with level of reported satisfaction (ρ = 0.172, p<0.001) and burnout (ρ = 0.261, p<0.001).
Correlates of opinions
In general, age and gender were not strongly correlated with opinions about technology replacing physicians. Only one task indicated a weak correlation, with more GPs in the younger age range (25–44 years old compared with 45 and older) believing that technology would replace physicians in formulating personalized treatment plans (ρ = 0.121, p<0.001, n = 720). A weak correlation between age and opinions about timescales was also found for diagnosis (ρ = -0.239, p<0.001, n = 231), prognosis (ρ = -0.148, p = 0.004, n = 380) and referral to other specialists (ρ = -0.167, p = 0.006, n = 276) with older GPs believing that the technological capacity would arise sooner. Gender of participants was weakly correlated with only one task–opinions about technology and the documentation of patient visits–with more male (82.6%) than female (77.4%) GPs believing physicians were likely to be replaced by technology (p = 0.013, n = 720). None of the following factors was associated with GPs’ opinions about technology and the future of primary care: reported burnout, level of job satisfaction, or whether GPs were full time or part time workers.
Participants who stated it is “extremely likely” that technology will replace the average GP in performing prognoses, making referrals, and in providing documentation estimated that technology would have this capacity sooner relative to those who said replacement was “likely” (p<0.001, n = 380; p<0.01, n = 276; p<0.001, n = 578, respectively).
Summary of major findings
The opinions of physicians have been missing from the debate about the future of the medical professions. Most respondents to this survey (61–68%) considered it unlikely or extremely unlikely that technology will be able to fully replace humans in performing key primary care tasks including diagnosis, referral to other specialists, and the formulation of personalized treatment plans; though, as these figures reflect, a significant minority of participants disagreed with these opinions. GPs were evenly split over prognostics with just over half (53%) considering it likely or extremely likely that technology will be able to replace them in performing this task. Nearly one in two of these respondents estimated that the technological capacity will arise in the next decade. Taken together these finding indicate skepticism but also appreciable disagreement among UK-based GPs about the capacity of future AI to disrupt their profession.
Our data show that the overwhelming majority of UK GPs studied (94%) believed it was unlikely or extremely unlikely that technology will be able to provide empathic care to patients as well as or better than the average GP. In contrast with opinions about other tasks, most GPs were optimistic about the potential for technology to fully replace them in performing tasks related to patient documentation. Eight in ten expected that future technology will be able to undertake this role, and 79% of these respondents estimated that the wherewithal will emerge up to ten years from now.
The majority of physicians surveyed appeared to disagree with predictions that humans will be replaceable in performing key medical tasks; yet, interestingly, most respondents expressed confidence in the prospects for technology to replace them in undertaking paperwork. These opinions should also be put in the context of GPs’ perceptions about the current workforce: 97% of respondents believed that patient demand in the UK exceeds the number of available GPs, and nearly 4 in 10 (38%) reported burn out with similar numbers expressed dissatisfaction with their job. Many AI experts maintain that, in the long-term, technological advancements will lead to efficiencies in the delivery of care through progress in preventative medicine and diagnostic accuracy; better and cheaper access to health care; and reductions in physician workloads [27,28,29]. Against these claims, and despite perceptions about high patient demand and reports of professional burdens, our results suggest that respondents viewed future technology as likely to play a somewhat limited role in improving workflow efficiencies–as restricted to documentation. However, it is also conceivable that GPs may have multiple, possibly conflicting, beliefs about the impact of technology in primary care and research is needed to explore these issues further.
Finally, we found a weak correlation between age of GPs and estimates about timescales. Perhaps surprisingly, more older GPs (45 years and over) than younger GPs estimated that the technological capacity to replace physicians on the tasks of diagnosis, prognosis, and referral to other specialists, would arise sooner. In light of the strong recommendation from biomedical informaticians that medical schools urgently start to train physicians in computer science and related fields, our data provides evidence to inform ongoing debates about curricula reform as well as changes to pre-medical requirements .
Most experts in AI fields argue that medicine will be “revolutionized” by innovations in machine learning [4,9–11]. To the extent that some of these experts view core tasks as replaceable by machines, our findings suggest a disconnect between the opinions of practicing GPs’ and informaticians. Evidence of disparities in opinions between respondents and AI experts also warrants further exploration. More information is needed to compare the beliefs and attitudes of physicians working across different specialties with those of AI experts. In particular, it would be valuable to compare views on the potential benefits, limitations, and possible timelines for major advancements in machine learning in medicine. Last, but not least, future research should examine patients’ views and preferences when it comes to the role of technology in medical care .
Strengths and limitations
This is the first survey to investigate physicians’ opinions about the future impact of technology on a medical specialty. While the majority of UK GPs are members of Doctors.net.org, a potential limitation of the study is that the participant response rate depended on individuals being logged into the network. The decision to complete the survey may have been influenced by the extent to which the responder was interested in the topic, and this, in turn, may have affected findings. The moderate response rate (49%) raises questions about representativeness, yet our respondents were reasonably similar to GPs registered with the GMC.
Some items on our list of tasks could be challenged on the grounds of vagueness. For example, it might be argued that the task “analyze patient information to reach diagnoses” does not encapsulate the subtasks or procedures involved in obtaining a diagnosis which might be said to include, among other things, efficient information gathering from the patient. However, to avoid building anthropocentric bias into our questions we deliberately avoided granular descriptions of GPs’ activities focusing on functions of primary care as opposed to how those functions might be realized.
While we obtained an optimal sample size for questions about the likelihood of technological replacement, this was reduced for items relating to time scales since only those participants who answered affirmatively were asked to respond to these questions. For some tasks this affected confidence intervals–e.g., empathy. Notwithstanding this limitation, since no other survey has been conducted on this issue, our investigation presents an important, foundational attempt to gauge physicians’ opinions on AI and future of a medical profession. In addition, the study investigated GPs’ opinions about whether technology will have the capacity to fully replace physicians on various tasks and not whether this will, in fact, happen. This latter, distinctive research question is also valuable and worth exploring. Finally, qualitative research might usefully probe physicians’ opinions about whether, and how, future innovations in technology might transform the designation of duties between humans and machines.
The majority of GPs in this UK-wide survey were unconvinced about the potential for technology to perform as well as or better than humans when it comes to key primary care tasks, including diagnostics and referral to other specialists. Respondents were most skeptical about the future capacity for technology to replace GPs in the delivery of empathic care. In contrast, the majority of respondents were optimistic that, in the near future, AI would be fully capable of undertaking tasks related to paperwork (See Box 1).
Box 1. Key questions and findings
What is already known about this topic
- Experts working in biomedical informatics and related fields variously predict that advancements in artificial intelligence and machine learning will transform the medical professions and improve patient care.
- The potential for technology to overhaul clinical practice raises ethical and policy questions, including the preparedness of the medical profession to lead and adapt to change.
What this study adds
- Most UK GPs believed that future technology would not be able to perform as well as or better than the average GP in key primary care tasks including: diagnostics, referral to other specialists, and the provision of empathic care.
- In contrast, half of respondents believed that future technology would be capable of fully replacing humans in the task of prognostics; eight in ten believed this would be the case when it comes to paper work.
- Disparities in opinions exist between GPs and experts working in artificial intelligence, with GPs studied more skeptical about the role of technology in the future of the primary care.
Reviewing our findings, the skepticism observed among the majority of GPs in this survey might appear justified. Last year, following extensive publicity and high anticipation, the five-year collaboration between IBM Watson and the MD Anderson Cancer Center collapsed, suggesting that the future role of AI in healthcare may be more hype than reality . Certainly, AI has yet to disrupt healthcare in ways that it has infiltrated other occupations and industries. However, as leading informaticians caution, cycles of hype camouflage broader historical trends: Amara’s Law is the observation that we tend to overestimate the effect of a technology in the short run and underestimate its effect in the long run [9,10]. We conclude by stressing that advancements in medical AI are unlikely to abate [28,29,32]. It is our hope that this survey provides prescient attention to the importance of the medical community leading, and fully engaging in, important but difficult debates about its future. Perhaps foremost among these questions is the adequacy of current medical education to inform future physicians about the scope as well as the current limitations of artificial intelligence, and data science . Retooling, we argue, is necessary if the profession is to manage and deeply influence foreseeable change, including in the delivery of primary care.
The authors thank Prof. Russell Philips for helpful advice on the design of the survey.
- 1. Frey CB & Osborne MA. The future of employment: how susceptible are jobs to computerization? Technological Forecasting and Social Change, 2017 Jan 1;114, 254–280.
- 2. Nedelkoska L & Quintini G. Automation, skills use and training. OECD Social, Employment and Migration Working Papers, 2018. No. 202, OECD Publishing, Paris. http://dx.doi.org/10.1787/2e2f4eea-en.
- 3. Walsh T. Android dreams: the past, present, and future of artificial intelligence. UK: Hurst. 2017.
- 4. Bauchner H, Berwick D, & Fontanarosa PB. Innovations in health care delivery and the future of medicine. Jama. 2016. Jan 5;315(1):30–31. pmid:26746453
- 5. Steinhubl SR, Muse ED, & Topol EJ. Can mobile health technologies transform health care? Jama. 2013. Dec 11;310(22):2395–6. pmid:24158428
- 6. Armstrong S. The apps attempting to transfer NHS 111 online. BMJ. 2018. 360. pmid:29335297
- 7. McCartney M. Innovation without sufficient evidence is a disservice to all. BMJ 2017. 358: j3980. https://doi.org/10.1136/bmj.j3980 pmid:28874353
- 8. Verghese A, Shah NH & Harrington RA. What This Computer Needs Is a Physician: Humanism and Artificial Intelligence. Jama. 2018. Jan 2;319(1):19–20. pmid:29261830
- 9. Hinton G. Deep learning–A technology with the potential to transform health care. Jama. 2018 Sept 18;320(11):1101–2. pmid:30178065
- 10. Naylor CD. On the prospects for a (deep) learning health care system. Jama. 2018 Sept 18;320(11):1099–100. pmid:30178068
- 11. Mandl KD & Bourgeois FT. The Evolution of patient diagnosis: From art to digital data-driven science. Jama, 2017 Nov 21;318(19):1859–60. pmid:29075757
- 12. Murdoch TB, Detsky AS. The inevitable application of big data to health care. Jama. 2013 Apr 3;309(13):1351–2. pmid:23549579
- 13. Darcy AM, Louie AK & Roberts LW. (2016). Machine learning and the profession of medicine. Jama. 2016 Feb 9;315(6):551–2. pmid:26864406
- 14. Doctors.net.uk. https://www.doctors.net.uk/about.html.
- 15. Howick J, Bishop FL, Heneghan C, Wolstenholme J, Stevens S, Hobbs FR et al. Placebo use in the United Kingdom: results from a national survey of primary care practitioners. PLoS One. 2013 Mar 20;8(3):e58247. pmid:23526969
- 16. Gonçalves-Bradley DC, Boylan AM, Koshiaris C, Vazquez Montes M, Ford GA, Lasserson DS. GPs’ adherence to guidelines for structured assessments of stroke survivors in the community and care homes. Family practice. 2015 Sep 30;32(6):659–63. pmid:26424724
- 17. Turner PJ, Van den Bruel A, Jones CH, Plüddemann A, Heneghan C, Thompson MJ et al. Point-of-care testing in UK primary care: a survey to establish clinical needs. Family practice. 2016 Apr 5;33(4):388–94. pmid:27048525
- 18. General Medical Council. GMC data explorer. https://data.gmc-uk.org/gmcdata/home/#/
- 19. Wetterneck TB, Lapin JA, Krueger DJ, Holman GT, Beasley JW & Karsh BT. Development of a primary care physician task list to evaluate clinic visit workflow. BMJ quality & safety 2011 Jan 1;21(1):47–53.
- 20. Cohen IG, Graver HS. Cops, Docs, and Code: A Dialogue Between Big Data in Health Care and Predictive Policing. UCDL Rev. 2017;51:437.
- 21. Obermeyer Z & Lee TH. Lost in Thought—The Limits of the Human Mind and the Future of Medicine. New England Journal of Medicine. 2017 Sep 28;377(13):1209–11. pmid:28953443
- 22. Armstrong RL. The midpoint on a five-point Likert-type scale. Perceptual and Motor Skills. 1987 Apr;64(2):359–62.
- 23. Krosnick JA, Holbrook AL, Berent MK, Carson RT, Michael Hanemann W, Kopp RJ et al. The impact of" no opinion" response options on data quality: non-attitude reduction or an invitation to satisfice? Public Opinion Quarterly. 2002 Sep 1;66(3):371–403.
- 24. West CP, Dyrbye LN, Sloan JA, & Shanafelt TD. Single item measures of emotional exhaustion and depersonalization are useful for assessing burnout in medical professionals. Journal of General Internal Medicine. 2009 Dec 1;24(12):1318. pmid:19802645
- 25. Dolan ED, Mohr D, Lempa M, Joos S, Fihn SD, Nelson KM et al. Using a single item to measure burnout in primary care staff: a psychometric evaluation. Journal of General Internal Medicine. 2015 May 1;30(5):582–7. pmid:25451989
- 26. Imo UO. Burnout and psychiatric morbidity among doctors in the UK: a systematic literature review of prevalence and associated factors. BJPsych bulletin. 2017 Aug;41(4):197–204. pmid:28811913
- 27. Topol EJ. The patient will see you now: The future of medicine is in your hands. New York: Basic Books; 2015.
- 28. Obermeyer Z, Emanuel EJ. Predicting the future—big data, machine learning, and clinical medicine. The New England Journal of Medicine. 2016 Sept 29;375(13):1216–19. pmid:27682033
- 29. Kohane IS, Drazen JM, Campion EW. A glimpse of the next 100 years in medicine. The New England Journal of Medicine. 2012 Dec 27;376(26):2538–2539.
- 30. Boeldt DL, Wineinger NE, Waalen J, Gollamudi S, Grossberg A, Steinhubl SR et al. How consumers and physicians view new medical technology: comparative survey. Journal of Medical Internet Research. 2015 Sep;17(9).
- 31. Freedman DH. A reality check for IBM’s AI ambitions. MIT Technology Review. 2017 June 27. https://www.technologyreview.com/s/607965/a-reality-check-for-ibms-ai-ambitions/
- 32. The Lancet. Artificial intelligence in health care: within touching distance. The Lancet (London: England). 2018 Dec 23;390(10114):2739.