http://orcid.org/0000-0003-3756-8831
Figures
Abstract
Introduction
A 2015 Institute Of Medicine statement “Transforming Health Care Scheduling and Access: Getting to Now”, has increased concerns regarding patient wait times. Although waiting times have been widely studied, little attention has been paid to the role of patient arrival times as a component of this phenomenon. To this end, we investigated patterns of patient arrival at scheduled ambulatory heart failure (HF) clinic appointments and studied its predictors. We hypothesized that patients are more likely to arrive later than scheduled, with progressively later arrivals later in the day.
Methods and results
Using a business intelligence database we identified 6,194 unique patients that visited the Cleveland Clinic Main Campus HF clinic between January, 2015 and January, 2017. This clinic served both as a tertiary referral center and a community HF clinic. Transplant and left ventricular assist device (LVAD) visits were excluded. Punctuality was defined as the difference between ‘actual’ and ‘scheduled’ check-in times, whereby negative values (i.e., early punctuality) were patients who checked-in early. Contrary to our hypothesis, we found that patients checked-in late only a minority of the time (38% of visits). Additionally, examining punctuality by appointment hour slot we found that patients scheduled after 8AM had progressively earlier check-in times as the day progressed (P < .001 for trend). In both a Random Forest-Regression framework and linear regression models the most important risk-adjusted predictors of early punctuality were: later in the day appointment hour slot, patient having previously been to the hospital, age in the early 70s, and white race.
Conclusions
Patients attending a mixed population ambulatory HF clinic check-in earlier than scheduled times, with progressive discrepant intervals throughout the day. This finding may have significant implications for provider utilization and resource planning in order to maximize clinic efficiency. The impact of elective early arrival on patient’s perceived wait times requires further study.
Citation: Gorodeski EZ, Joyce E, Gandesbery BT, Blackstone EH, Taylor DO, Tang WHW, et al. (2017) Discordance between 'actual' and 'scheduled' check-in times at a heart failure clinic. PLoS ONE 12(11): e0187849. https://doi.org/10.1371/journal.pone.0187849
Editor: Ashham Mansur, University Medical Center Goettingen, GERMANY
Received: May 2, 2017; Accepted: October 29, 2017; Published: November 14, 2017
Copyright: © 2017 Gorodeski et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
A 2015 Institute Of Medicine statement entitled “Transforming Health Care Scheduling and Access: Getting to Now”, has increased concerns regarding patient wait times. Although waiting times have been widely studied, little attention has been paid to the role of patient arrival times as a component of this phenomenon [1].
A lack of patient punctuality to a scheduled outpatient appointment, either in the patient being too early or too late, can lead a variety of undesirable consequences. A patient who shows up too early may have an extended wait time which can degrade perceived satisfaction with the visit [2]. At times providers may try to accommodate the early arriving patient at an earlier time slot [3], which can subsequently disrupt the timeliness of care for other patients who are scheduled in subsequent slots and arrive in a timely manner. On the other hand, a patient who shows up too late may have a prolonged wait time until the provider is able to see them on the same day, or may even have their appointment cancelled entirely. In both situations a lack of punctuality disrupts the flow of the outpatient clinic and creates stress for the patient and the provider. The ideal situation is minimization the absolute value of the deviation from the scheduled appointment time.
The aim of our study was to evaluate the discrepancy between ‘actual’ and ‘scheduled’ check-in times, or punctuality, at the HF clinic in Cleveland Clinic Main Campus, a large tertiary hospital in Cleveland, Ohio. We hypothesized that patients attending this clinic are more likely to arrive later than scheduled, with progressively later arrivals later in the day.
Methods
We performed a study of consecutive patients who presented to the Cleveland Clinic Main Campus HF clinic between January, 2015 and January, 2017, using an administrative database of patient visits. This outpatient clinic served both as a tertiary referral center and a community HF clinic, and was staffed by physicians and advanced practice practitioners (APPs), including nurse practitioners and physician assistants.
Patients had to have scheduled appointments prior to arriving. Visits were scheduled for 30 or 60 minutes depending on type: new to the HF clinic and the hospital (60 minutes), new to the HF clinic but not the hospital usually representing internal consults (60 minutes), and established patients previously seen in the HF clinic (30 minutes). Clinicians were assigned templates that alternated visit types (S1 File), and generally worked either whole or half days (mornings ending at 1:00PM, or afternoons ending at 5:00PM). Clinicians had the flexibility to add on patients between 7:30AM at the earliest and 4:30PM at the latest, as well as to double book. Upon arrival to the lobby outside the HF clinic, located on the 3rd floor of the hospital, patients were checked-in digitally by a medical secretary.
The cohort was limited to the first visit of each unique patient during the study’s time period (January, 2015 and January, 2017). The cohort was further limited to those patients scheduled to be seen by a single provider in the HF clinic on the day of visit. Heart transplant, left ventricular assist device (LVAD), social work, and research visits were excluded. The study was approved by Institutional Review Board of the Cleveland Clinic.
Study variables
Demographic and visit-specific variables were obtained from Cleveland Clinic’s hospital business intelligence database. There were no missing data. ‘Scheduled’ check-in times and ‘actual’ check-in times were recorded automatically via Cleveland Clinic’s appointment system. Weather-related variables were obtained from the National Oceanic and Atmospheric Administration’s National Centers for Environmental Information website (www.ncei.noaa.gov), and merged in by calendar date. These included temperature (°F) on day of visit, average wind speed (mph), rain fall (inch), snow fall (inch), snow depth (inch), presence of fog (yes/no), and presence of haze (yes/no).
End point
Punctuality, a continuous variable comprised of one minute intervals, was defined as the difference between ‘actual’ and ‘scheduled’ check-in times, whereby positive values were late check-in times and negative values were early check-in times.
Statistical analyses
Cohort characteristics were summarized with continuous variables expressed as mean and standard deviation, and categorical variables as frequencies. The Wilcoxon-Kruskal-Wallis test was used for continuous variables, the Pearson chi-square test was used for categorical variables, and the likelihood ratio chi-square test from the proportional odds model was used for ordinal variables. The distribution of punctuality was explored graphically, with point estimates expressed as mean and 95% bootstrapped confidence interval (CI).
We used random forest-regression (RF-R) and multiple linear regression methodologies to investigate the association between the independent variables and punctuality. The primary purpose of utilizing RF-R was for variable selection for linear regression modeling. A secondary purpose of RF-R methodology was to graphically explore the relation between the most predictive variables.
Random forests, a robust machine-learning statistical methodology, have been previously described in context of the medical literature by our group and others [4–7]. Briefly, RF-R allow for ranking of variables by importance, and investigation of each variable’s risk-adjusted behavior, using decision tree methods that are independent of parametric assumptions, and inherently account for interactions and outliers. We constructed an RF-R framework composed of 1,000 uncorrelated decision trees grown from the 20 variables shown in Table 1, for the outcome punctuality. Trees in this RF-R framework, essentially a set of uncorrelated massively complex prediction algorithms, were broad and densely branched with a mean 1,272 terminal nodes.
We identified the most important variables in the RF-R framework as those that most frequently split the branches near the tree trunks (i.e., minimal depth), averaged over the entire forest [5–7]. Minimal depth is a dimensionless statistic that measures the predictiveness of a variable in tree-based models. Smaller values for minimal depth indicate better predictiveness [8]; Just as “cream rises to the top,” the most predictive variables in random forest frameworks tend to aggregate closet to the root node (labeled level 0) of decision trees. The distribution of minimal depths (Dv) across the forest was calculated, and the mean Dv was derived. This was used as a filtering threshold for identifying those variables that were informative for prediction [7].
For presentation purposes we then explored the relation between the top four variables identified by minimal depth analysis and punctuality using RF-R framework-derived partial dependence plots. These can be interpreted as the effect on the response for a one unit change in the predictor, while averaging over the effects of all other variables [9]. Additionally, we created a marginal plot visualizing the simultaneous effect of the top four variables on the outcome.
Finally, to quantitatively describe the risk-adjusted behavior of ‘appointment hour slot’, our key variable of interest, we constructed fully-adjusted multiple linear regression models.
Computational methods
All analyses were performed with R version 3.2.3 (www.r-project.org), running off a Linux operating system, on a Dell PowerEdge R180 server. We used Wickham’s tidyr and dplyr libraries for data tidying and manipulation, lubridate [10] library for manipulation of dates and times, and ggplot2 [11] for data visualization and graphics. We used Ishwaran and Kogalur’s randomForestSRC [12] library for RF-R analyses, and Harrell’s rms and Hmisc libraries for regression analyses.
Results
We identified 6,194 patients that met the study’s inclusion and exclusion criteria (S1 Fig). Of 27,654 visits to the HF clinic in the study time period of January, 2015 and January, 2017, we excluded visits of patients with heart transplants (n = 1,834) or LVADs (n = 3,674), visits with multiple providers on same day (n = 2,886), and visits with non-clinicians such as those with social worker or research staff (n = 1,493). Finally, we excluded all subsequent visits beyond the first for those patients who had >1 visit in the study period (n = 11,573).
Among the 6,194 patients in the study cohort 38% checked-in late (n = 2,349 visits) (Fig 1). As compared to those who checked in late, patients who checked-in early/on time were (Table 1) proportionally older, more frequently white and male, and more frequently Medicare beneficiaries. Further, they were more likely to have previously been seen in the HF clinic. Patients tended to check-in early/on time for later appointment hour slots in the day (Table 1, Fig 2). Weather on day of appointment, including temperature, wind, rain, snow, fog, and haze, did not impact check-in time patterns in unadjusted analyses (Table 1).
Dotted line represents a visit for which a patient checked-in exactly on time (punctuality = 0).
Fig 3 shows all 20 variables and plots their average minimal depth as derived from the RF-R framework. All variables exceeded a threshold value for filtering variables, therefore confirming their contribution towards predictiveness. Of these, the most predictive variables are those seen in the top left, including appointment hour slot (minimal depth = 0.99), visit type (1.7), age (2.1), race (2.4), temperature on day of appointment (2.7), and total outpatient appointments on same day (3.2), among others. We constructed two sets of fully-adjusted multiple linear regression models (S1 Table). In the first model appointment hour slot was modeled as a continuous variable, showing a 3.4 minute progressively earlier predicted check-in time per hour, as the day progressed. In the second model appointment hour slot was modeled as an ordinal variable, showing that as compared to the earliest appointment hour slot of the day those visits scheduled for the latest slots in the day were estimated to check-in approximately 35 minutes earlier.
Appointment hour slot was the most predictive variable, followed by visit type, and so forth.
Fig 4 shows partial dependence plots of the top 4 most predictive variables (from Fig 3) in order. Appointment hour slot, the most predictive variable, retained its linear relation with the outcome, whereby patients scheduled later in the day were predicted to check-in progressively earlier. Visit type was the second most predictive variable, whereby as compared to patients who were new to the hospital, patients who had previously visited the hospital tended to show up earlier. Age, the third most predictive variable, had a U-shaped association with the outcome, whereby the youngest and the oldest patients were predicted to show up later, as compared to those in their early 70s who were predicted to show up earliest. Race was the fourth most predictive variable, whereby patients who were white were predicted to show up earlier than those of other races.
These plots were derived from the Random Forest-Regression (RF-R) machine learning framework, and can be interpreted as the effect on the response for a one unit change in the predictor, while averaging over the effects of all the other 20 variables (shown in Table 1) in the RF-R.
Fig 5 is a marginal plot simultaneously visualizing the association between appointment hour slot, visit type (categories collapsed to not new versus new to hospital), age (in quartiles), and race (categories collapsed to non-white versus white race), with the outcome of predicted punctuality.
These plots were derived from the Random Forest-Regression (RF-R) machine learning framework.
Discussion
The main finding of our study is that on average patients attending an ambulatory outpatient HF clinic show up early prior to their scheduled visit, and not late as we initially hypothesized. Further, we found that patients tend to show up progressively earlier as the day goes on. We found that appointment hour slot and visit type were the strongest risk-adjusted predictors of arrival patterns, with other variables contributing to prediction more modestly.
We are the first to describe patterns of patient arrival to an ambulatory outpatient HF clinic. Our findings of early arrival, with progressively early arrival as the day goes on, are consistent with observations from pediatric clinics, pain clinics, and general medical clinics both in the United Kingdom and the United States [3, 13–17]. Strengths of our study include digital documentation of check-in times, large cohort size, availability of relevant adjusting variables via administrative dataset without any missing data, and a contemporary approach to data analysis using robust novel machine learning techniques.
It is not entirely clear why patients tend to arrive to outpatient appointments progressively earlier as the day goes on. We expected that this association would be attenuated by one or more other appointments or tests on the same day, but did not find it to be the case after multi-variable risk adjustment. It may be that our findings highlight an aspect of human behavior not accounted for by other variables in our models: patients want to be at home in the evenings, and they show up progressively earlier to later in the day appointments in an effort to assure this. This should be studied further in future investigations.
Our findings are timely for another reason as well. The Institute of Medicine published a statement in 2015 entitled “Transforming Health Care Scheduling and Access: Getting to Now”, which has increased concerns regarding patient wait times [1]. While most work towards reducing global wait times has focused on improving access, little attention has been paid to the role of patient arrival times as a component of this phenomenon.
An area of active investigation is the impact of wait times on the patient experience. Previously published work showed that prolonged wait times in ambulatory clinics and the emergency department negatively impacted the patient experience [2, 18–23]. This included not only reduced overall satisfaction, but also worse perceptions of information, instructions, and the overall treatment provided by various clinicians [2]. Interestingly, a different study suggested that patients who arrive late have higher satisfaction due to shorter wait times [24]. Future work should focus on whether and how the patient experience is impacted by prolonged wait times related to early arrival to outpatient clinic.
Recognition that patients tend to arrive earlier as the day goes on may further allow for providers to optimize outpatient clinic operations. Other investigators have proposed decision algorithms about whether to see an early arriving patient immediately or wait for next scheduled patient to arrive first (the “Wait-Preempt Dilemma”) [17]. Clinicians may elect to adopt such algorithms in order to reduce total patient time in clinic, optimize clinic flow, and improve the patient experience. Even simpler may be the recognition that clinicians should be physically available to start their clinic 15 to 30 minutes early for afternoon sessions, as patients are anticipated to be already waiting to be seen in this frame of time.
Our study has several limitations. First, this was a single-center experience from a large tertiary referral hospital in an urban center, which may limit generalizability. Second, we used an administrative database that lacked more detailed clinical data specific to patients with HF. It may be possible that markers of HF disease severity could be associated with punctuality, this should be investigated in future studies. Third, we did not have any detailed data about daily traffic patterns (traffic jams, parking problems, etc.) that could have been associated with punctuality.
In summary, we found an unanticipated pattern of early arrival of patients to our HF clinic, with progressively earlier arrival over the course of the day. Our results, if confirmed, have significant implications regarding estimation of patient wait times before appointment. A more nuanced understanding of punctuality in this setting may impact how clinicians and administrators design and optimize outpatient HF clinics, and the subsequent impact on workflow and the patient experience.
Supporting information
S1 File. Ambulatory HF clinic schedule templates.
https://doi.org/10.1371/journal.pone.0187849.s003
(DOCX)
References
- 1.
Transforming health care scheduling and access: getting to now. Kaplan G, Lopez MH, McGinnis JM, editors. Washington, DC: Institute of Medicine; 2015. x, 143 pages p.
- 2. Bleustein C, Rothschild DB, Valen A, Valatis E, Schweitzer L, Jones R. Wait times, patient satisfaction scores, and the perception of care. Am J Manag Care. 2014;20(5):393–400. pmid:25181568.
- 3. Williams KA, Chambers CG, Dada M, McLeod JC, Ulatowski JA. Patient punctuality and clinic performance: observations from an academic-based private practice pain centre: a prospective quality improvement study. BMJ Open. 2014;4(5):e004679. pmid:24833686; PubMed Central PMCID: PMCPMC4024595.
- 4. Breiman L. Statistical modeling: The two cultures. Stat Sci. 2001;16(3):199–215.
- 5. Gorodeski EZ, Ishwaran H, Kogalur UB, Blackstone EH, Hsich E, Zhang ZM, et al. Use of hundreds of electrocardiographic biomarkers for prediction of mortality in postmenopausal women: the Women's Health Initiative. Circ Cardiovasc Qual Outcomes. 2011;4(5):521–32. pmid:21862719; PubMed Central PMCID: PMCPMC3893688.
- 6. Hsich E, Gorodeski EZ, Blackstone EH, Ishwaran H, Lauer MS. Identifying important risk factors for survival in patient with systolic heart failure using random survival forests. Circ Cardiovasc Qual Outcomes. 2011;4(1):39–45. pmid:21098782; PubMed Central PMCID: PMCPMC3991475.
- 7. Ishwaran H, Kogalur UB, Gorodeski EZ, Minn AJ, Lauer MS. High-Dimensional Variable Selection for Survival Data. J Am Stat Assoc. 2010;105(489):205–17.
- 8. Twyman-Saint Victor C, Rech AJ, Maity A, Rengan R, Pauken KE, Stelekati E, et al. Radiation and dual checkpoint blockade activate non-redundant immune mechanisms in cancer. Nature. 2015;520(7547):373–7. pmid:25754329; PubMed Central PMCID: PMCPMC4401634.
- 9. Cafri G, Bailey BA. Understanding Variable Effects from Black Box Prediction: Quantifying Effects in Tree Ensembles Using Partial Dependence. Journal of Data Science. 2016;14(1):67–95.
- 10. Grolemund G, Wickham H. Dates and times made easy with lubridate. Journal of Statistical Software. 2011;40(3):1–25.
- 11.
Wickham H. Ggplot2: elegant graphics for data analysis. New York: Springer; 2009. viii, 212 p. p.
- 12.
Ishwaran H and Kogalur UB. Random Forests for Survival, Regression and Classification (RF-SRC), R package version 2.3.0. 2016.
- 13. White MJB, Pike MC. Appointment Systems in Out-patients' Clinics and the Effect of Patients' Unpunctuality. Medical Care. 1964;2(3):133–45.
- 14. Swartzman G. The patient arrival process in hospitals: statistical analysis. Health Serv Res. 1970;5(4):320–9. pmid:5508522; PubMed Central PMCID: PMCPMC1067286.
- 15. Cayirli T, Veral E, Rosen H. Designing appointment scheduling systems for ambulatory care services. Health Care Manag Sci. 2006;9(1):47–58. pmid:16613016.
- 16. Hang SC, Hassmiller Lich K, Kelly KJ, Howell DM, Steiner MJ. Patient- and Visit-Level Variables Associated With Late Arrival to Pediatric Clinic Appointments. Clin Pediatr (Phila). Epub ahead of print October 5, 2016. pmid:27707900.
- 17. Samorani M, Ganguly S. Optimal Sequencing of Unpunctual Patients in High-Service-Level Clinics. Prod Oper Manag. 2016;25(2):330–46.
- 18. Shirley ED, Sanders JO. Patient satisfaction: Implications and predictors of success. J Bone Joint Surg Am. 2013;95(10):e69. pmid:23677370.
- 19. Taylor C, Benger JR. Patient satisfaction in emergency medicine. Emerg Med J. 2004;21(5):528–32. pmid:15333521; PubMed Central PMCID: PMCPMC1726409.
- 20. Newcomb P, Wilson M, Baine R, McCarthy T, Penny N, Nixon C, et al. Influences on Patient Satisfaction Among Patients Who Use Emergency Departments Frequently for Pain-Related Complaints. J Emerg Nurs. 2017. pmid:28527640.
- 21. Waters S, Edmondston SJ, Yates PJ, Gucciardi DF. Identification of factors influencing patient satisfaction with orthopaedic outpatient clinic consultation: A qualitative study. Man Ther. 2016;25:48–55. pmid:27422597.
- 22. Meyer C, Ringler A, Bartsch DK, Fendrich V. [Analysis of patient waiting times in a polyclinic for surgery]. Chirurg. 2016;87(11):964–70. pmid:27392761.
- 23. Inglehart MR, Lee AH, Koltuniak KG, Morton TA, Wheaton JM. Do Waiting Times in Dental Offices Affect Patient Satisfaction and Evaluations of Patient-Provider Relationships? A Quasi-experimental Study. J Dent Hyg. 2016;90(3):203–11. pmid:27340187.
- 24. Medway AM, de Riese WT, de Riese CS, Cordero J. Why patients should arrive late: The impact of arrival time on patient satisfaction in an academic clinic. Healthc (Amst). 2016;4(3):188–91. pmid:27637825.