Advertisement

  • Loading metrics

Open Access

Peer-reviewed

Research Article

From patient voices to policy: Data analytics reveals patterns in Ontario’s hospital feedback

  • Pourya Momtaz,

    Roles Data curation, Formal analysis, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada

    ⨯
  • Mohammad Noaeen,

    Roles Validation, Writing – original draft, Writing – review & editing

    Affiliation Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada

    ⨯
  • Konrad Samsel,

    Roles Writing – review & editing

    Affiliation Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada

    ⨯
  • Neil Seeman,

    Roles Resources, Writing – review & editing

    Affiliations Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada, Massey College, University of Toronto, Toronto, Ontario, Canada

    ⨯
  • Robert Cribb,

    Roles Resources, Writing – review & editing

    Affiliation Investigative Journalism Bureau, University of Toronto, Toronto, Ontario, Canada

    ⨯
  • Syed Ishtiaque Ahmed,

    Roles Writing – review & editing

    Affiliation Department of Computer Science, University of Toronto, Toronto, Ontario, Canada

    ⨯
  • Amol Verma,

    Roles Writing – review & editing

    Affiliations Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada, St. Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada

    ⨯
  • Dionne M. Aleman,

    Roles Writing – review & editing

    Affiliations Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada, Department of Mechanical & Industrial Engineering, University of Toronto, Toronto, Ontario, Canada

    ⨯
  • Zahra Shakeri

    Roles Conceptualization, Formal analysis, Funding acquisition, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    zahra.shakeri@utoronto.ca

    Affiliations Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada, Schwartz Reisman Institute, University of Toronto, Toronto, Ontario, Canada, Faculty of Information, University of Toronto, Toronto, Ontario, Canada

    ⨯

Abstract

Patient satisfaction is a central measure of high-performing healthcare systems, yet real-world evaluations at scale remain challenging. In this study, we analyzed 122,194 de-identified patient reviews from 45 Ontario general hospitals between January 2015 and July 2022. We applied a natural language processing (NLP) pipeline using a clinical named entity recognition (NER) model fine-tuned on biomedical literature to extract references to diseases, symptoms, and medical procedures from patient reviews. Geospatial analysis was conducted to examine sentiment patterns based on regional census data related to low-income status and visible-minority composition. Our primary objective was to investigate how the COVID-19 pandemic influenced patient satisfaction trends, with a specific focus on clinical units and hospitals serving marginalized populations. We assessed changes in the proportion of positive comments across time periods and socioeconomic groups using multivariate logistic regression.

Our findings show that over 80% of the hospitals studied had fewer than 50% positive reviews, highlighting possible systemic gaps in patient needs. Interestingly, the proportion of negative reviews decreased during the COVID-19 pandemic, suggesting possible changes in patient expectations or increased appreciation for healthcare workers. However, certain units, such as dentistry and radiology, experienced more negative ratings as a proportion of their total reviews. ‘Anxiety’ emerged as a recurrent concern in negative reviews, especially during the start of the pandemic, pointing to the growing awareness of mental health needs. Based on our geospatial analysis, hospitals located in regions with higher percentages of visible minority and low-income populations initially saw higher positive review proportions before COVID-19, but this trend reversed after 2020. Our statistical models confirmed that these shifts were significant, particularly for low-income-serving hospitals. Collectively, these results demonstrate how large-scale unstructured data can identify fundamental drivers of patient satisfaction, while underscoring the urgent need for adaptive strategies to address anxiety and combat systemic inequities.

Author summary

Understanding what patients think and feel about hospital care can lead to better health services and outcomes. We analyzed more than 120,000 patient reviews from 45 Ontario hospitals between 2015 and 2022. Our study combined text processing techniques to identify key concerns related to factors such as anxiety, billing difficulties, and interactions with staff. We also compared patient experiences before and during the COVID-19 pandemic, uncovering a drop in negative reviews and a rise in positive reviews. A particularly revealing finding was that hospitals located in regions with higher percentages of residents who are visible minorities and low-income groups received more positive feedback before the pandemic, but this trend reversed after the start of the pandemic. These patterns hint at deeper systemic issues, especially during times of crisis. By exploring the main drivers of satisfaction and dissatisfaction, our work highlights the need for healthcare services that prioritize kindness, clear communication, efficient operations, and equitable access for all. Lessons from this research could guide targeted improvements, ensuring that every patient, regardless of background or income, receives the compassionate and timely care they deserve. Our hope is that policymakers, hospital administrators, and community advocates will use these findings to shape policies that improve patient trust and well-being.

Citation: Momtaz P, Noaeen M, Samsel K, Seeman N, Cribb R, Ahmed SI, et al. (2026) From patient voices to policy: Data analytics reveals patterns in Ontario’s hospital feedback. PLOS Digit Health 5(2): e0000739. https://doi.org/10.1371/journal.pdig.0000739

Editor: Hadi Ghasemi, Shahid Beheshti University of Medical Sciences School of Dentistry, IRAN, ISLAMIC REPUBLIC OF

Received: January 7, 2025; Accepted: January 7, 2026; Published: February 5, 2026

Copyright: © 2026 Momtaz et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The data supporting this study’s findings are publicly accessible by request and were obtained by Investigative Journalism Bureau (IJB) reporters through Freedom of Information requests to Ontario hospitals via the provincial government’s online portal: https://www.ontario.ca/page/freedom-information-request. Note: Since the data were accessed through public records and did not involve direct interaction with live participants, formal consent was not required. Moreover, all data were anonymized to ensure the confidentiality and privacy of the individuals involved.

Funding: This research was supported by the Vector Scholarship in Artificial Intelligence (Vector Institute) to PM and the Institute for Pandemics (IFP) at the University of Toronto to ZSH. We also acknowledge funding from the Natural Sciences and Engineering Research Council of Canada (NSERC) through the Canada Research Chairs Program and Discovery Grant (RGPIN-2025-07037 to ZSH). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Patient-centeredness is essential for delivering high-quality healthcare, focusing on understanding patients’ perspectives, and ensuring safe and effective services [1–3]. Healthcare organizations are increasingly adopting patient-centric models, recognizing that patient satisfaction directly influences treatment adherence, reduces preventable errors, lowers staff turnover, decreases hospital readmissions, and enhances overall patient engagement and health outcomes [4–7]. The World Health Organization emphasizes the strong link between patient satisfaction and improved treatment adherence and health outcomes [6,7]. However, some studies indicate that an excessive focus on patient satisfaction might compromise clinical quality by prioritizing patient preferences over evidence-based practices [8]. This conflicting evidence demonstrates the need for further research, especially in outpatient primary care where data is limited and inconsistent.

A U.S.-based study of acute care hospitals found that those with high adherence to clinical guidelines also achieved better patient experience scores [9]. Interestingly, hospitals with the lowest risk-adjusted mortality rates for acute myocardial infarction had the highest patient experience scores [10]. These findings suggest that prioritizing patient experience does not necessarily undermine clinical quality. Further studies support this, showing that hospitals that excel in patient experience also report lower mortality and readmission rates and better adherence to surgical process measures [9,11,12]. This trend is consistent across various settings, including ambulatory care [11,13–15].

In Canada, focusing on patient experiences and patient-centered care is critical given the country’s health system performance challenges compared to other high-income nations. A 2019 Commonwealth Fund study ranked Canada last among 11 high-income countries for timeliness and efficiency of care [16]. A tragic illustration of these systemic delays comes from a recent case where a patient in Montreal, after waiting six hours for care and ultimately leaving the emergency department, died of an aneurysm [17]. With only 127 family physicians or nurse practitioners per 100,000 Canadians as of 2023 [18], the shortage of primary healthcare providers has led to increased hospital visits during health crises, potentially overwhelming the system [19]. Accessibility is further limited, with only 41% of Canadians able to see a healthcare provider the same or on the next day when needed [20], and even lower rates among economically disadvantaged populations [21], pointing to the need for equitable access to health services.

These accessibility issues significantly impact patient outcomes and experiences. In the 2021-2022 fiscal year, 14.6% of mental health patients in Ontario (Canada) had at least three hospital stays within a year, up from around 0.2% in 2017 [22]. Further compounding this burden, Canadians in the lowest income bracket are three to four times more likely than those in the highest income bracket to report poor or fair mental health [23]. This socioeconomic gradient highlights the disproportionate effects on vulnerable groups, including those with mental health conditions and lower socioeconomic status.

The COVID-19 pandemic exposed and intensified weaknesses in Canada’s healthcare system, such as provider shortages, accessibility issues, and disparities in patient outcomes [24]. Hospital overcrowding, staff shortages, and supply chain disruptions strained resources, with COVID-19 cases and deaths peaking in January 2022 [25]. The pandemic showed the need for resilient healthcare systems and patient-centered care, especially for those with chronic conditions or compromised immune systems [26,27]. Moreover, the mass departure of nursing professionals during the pandemic exacerbated healthcare access issues [28–30]. Unfortunately, even after the direct burden of the COVID-19 had passed, reports indicated continued challenges with long wait times and understaffed facilities [31], emphasizing the need for policy reform.

Prior research has effectively used natural language processing (NLP) and machine learning (ML) to analyze patient reviews, automating tasks such as topic classification and sentiment analysis [32–41]. However, most studies focus on online reviews collected from social media platforms or patient forums rather than from institutional sources. Moreover, there remains a gap in understanding the geographical disparities in patient sentiment, and how the socioeconomic status or marginalization of hospital catchments may relate to healthcare experiences.

In response to these gaps, our study applies NLP and other statistical approaches to identify factors that are associated with patient satisfaction. Our dataset includes 122,194 de-identified patient reviews from 45 out of Ontario’s 47 general hospitals. These reviews, collected between January 2015 and July 2022, offer the opportunity to examine how the COVID-19 pandemic shaped overall trends in patient satisfaction and explore possible differences in these findings when stratifying by clinical unit or catchment demographics. To our knowledge, this is the first study to apply these methods to a review dataset originating directly from multiple healthcare institutions, providing insights into pandemic-era patient experience and health equity within a publicly funded healthcare system.

Methods

Dataset

The dataset used in this analysis included 122,194 de-identified patient reviews collected between January 2015 and July 2022 from 45 Ontario hospitals. These reviews were compiled by the US-based National Research Corporation (NRC) and obtained through freedom of information requests by the Investigative Journalism Bureau (IJB) at the University of Toronto. Patient reviews were accompanied by unstructured metadata, including the experience date, hospital name, and hospital ward. Each comment’s sentiment classification was pre-assigned by NRC with one of the following labels: positive, neutral, negative, or mixed (positive/negative). These labels were derived from the patient’s quantitative survey ratings associated with each comment and were utilized as the ground truth for this study and we did not apply an additional automated sentiment model. We excluded comments without corresponding unit names (n = 38,929), resulting in a final analytic sample of 83,265 reviews for unit-level analyses.

Data labelling and pre-processing

A single-labeler approach was used to identify key themes related to each comment, such as transportation, access, and care coordination. Each review could be tagged with multiple themes, and a full list of thematic labels is included in S2 Table.

Next, patient review comments were pre-processed prior to NER classification, and included tokenization, lemmatization, and removal of non-informative tokens. Misspellings were corrected by mapping to the nearest word in a reference vocabulary. Experience dates were manually extracted and normalized to a consistent JavaScript Date format to support downstream visualization tools. Sentiment valence strings and satisfaction themes were harmonized using rule-based normalization (e.g., lowercasing and lookup tables) [42].

To resolve inconsistent hospital unit labels, we implemented a multi-step process by first defining a list of 27 general unit names (see S1 Table), and then mapping all unit labels in the dataset to these standardized categories using prompt engineering with Claude 3 LLM and manual verification. We used structured prompts to map raw unit labels to standardized unit categories, followed by manual verification. Obvious labels such as “Medical Assessment (MAU)” were directly mapped to “Medical Assessment Unit”, while abbreviations such as “H2C-Neph” were linked to “Nephrology Unit”. Ambiguous codes (e.g., “G4D”) were manually resolved using floor-plan references, and unclear entries (e.g., “YC2BC”) were tentatively categorized based on pattern similarity. Null or unrecognized units were labeled as “Unknown” and excluded from unit-level analyses; this affected 38,929 reviews.

Clinical named entity recognition

To extract clinically meaningful entities, we used the en_ner_bc5cdr_md model from the scispaCy library [43]. This spaCy-compatible model was trained on the BC5CDR corpus, which includes PubMed abstracts annotated for disease and chemical mentions. It identifies two entity types (DISEASE and CHEMICAL) and achieves an F1 score of approximately 0.86 on BC5CDR datasets [44].

The model returns each recognized entity with its text span and label (e.g., DISEASE or CHEMICAL). We kept entities returned by the model and performed post-processing to support frequency analysis. To capture context and granular meaning, we counted each entity mention separately, even if overlapped (e.g., both “fever” and “acute fever” were counted). We focused our analysis on clinically relevant disease entities (e.g., “pneumonia,” “cancer”) and chemical substances (e.g., specific medications). These extracted spans were aggregated to compute mention frequencies across all reviews.

Geospatial analysis by hospital catchment

We examined demographic factors using the HealthyPlan.City platform [45], developed by the Canadian urban environmental health research consortium (CANUE) [46,47], which integrates environmental indicators at the dissemination-block level with 2021 Canadian Census data across more than 125 Canadian cities [48]. For each hospital, we calculated a population-weighted average of visible-minority and low-income percentages across all dissemination blocks intersecting a geographic radius from the hospital’s location. While a 5 km radius was used by default, a smaller radius of 3 km was applied when two hospitals were located within 10 km of one another. This adjustment helped reduce overlap between catchment areas and ensured clearer attribution of surrounding neighborhood characteristics to each hospital. Hospitals were classified as serving higher visible-minority populations if ≥40% of nearby residents identified as visible minorities, and higher low-income populations if ≥10% of residents fell below the after-tax low-income cut-off (LIM-AT), as defined by Statistics Canada [49]. These population characteristics were computed as population-weighted averages across dissemination blocks within the designated radius. For the geospatial demographic analysis, data from 2015 were excluded due to low sample volume (n = 541) compared to subsequent years to ensure robust annual comparisons starting from 2016.

COVID-19 period temporal segmentation

‘COVID-19 period’ reviews were defined as those dated from January 2020 to July 2022. This range aligns with the World Health Organization’s emergency designation timeline [50,51], enabling consistent comparison of patient sentiment trends before and during the pandemic.

Statistical analysis

To evaluate differences in sentiment trends, we fitted multiple logistic regression models, each focusing on specific variable groupings (e.g., hospital units, thematic codes, demographic designations). The dependent variable across all models was binary sentiment polarity (1 = Positive, 0 = Negative). For binary sentiment models, we restricted analyses to reviews labeled positive or negative, excluding neutral and mixed (positive/negative) labels. Independent variables in these models included review year, hospital unit, thematic codes, and binary indicators for COVID-19 period, minority-serving hospitals, and low-income hospital regions. Interaction terms (e.g., CovidPeriod × Minority) were incorporated where applicable to assess whether temporal sentiment shifts differed across hospital and population groups.

Model significance was assessed using maximum likelihood estimation via the statsmodels library [52]. For each interaction model, we performed a likelihood ratio test comparing full and reduced models to determine whether interaction terms contributed significantly to model fit. Standardized z-scores, p-values, and 95% confidence intervals were reported for each coefficient.

We also conducted post-hoc statistical tests, including two-proportion z-tests, to compare sentiment proportions across pre-COVID and COVID periods. Chi-square test was used to assess temporal trends across years. All statistical analyses were conducted in Python using the following data analysis libraries: pandas(v2.2.2), scipy(v1.15.3), and statsmodels(v0.14.15). Significance was defined based on an alpha value of 0.05.

Results

Preliminary results

Based on our analysis of 122,194 de-identified patient feedback comments collected between 2015 and 2022 across 45 Ontario hospitals, we examined overall sentiment patterns and shifts over time. Our analysis showed that over 80% of these institutions received under 50% positive reviews, suggesting significant potential to enhance patient satisfaction. The total volume of reviews varied sharply from 2015 to 2022, rising from 541 reviews in 2015 to a peak of 26,243 in 2020 before dropping to 1,708 in 2022 (Fig 1). These fluctuations may reflect shifts in patient engagement through online feedback platforms, combined with an uptick in healthcare discussions during the early pandemic phase. The lower numbers for 2022 could stem from the dataset including reviews only until July of that year.

thumbnail
Download:
Fig 1. Temporal analysis of review sentiment distribution (2015-July 2022).

The upper panel presents a grouped bar chart depicting the absolute frequency of positive, negative, neutral, and mixed (positive/negative) reviews annually. The lower panel illustrates the relative proportion of each sentiment category as a stacked percentage bar chart. This dual representation reveals both absolute and relative changes in review sentiments over time, signaling trends in patient feedback and potential external influences.

https://doi.org/10.1371/journal.pdig.0000739.g001

Fig 1 illustrates trends in sentiments from 2015 to July 2022. Positive reviews were generally the largest category, ranging from 56.19% (2015) to 40.84% (2017), while negative reviews spanned 24.77% to 35.31%. Neutral reviews stayed below 5.73%. Although positive feedback climbed from 2017 to 2020, it slipped from 47.7% in 2020 to 45.8% in 2021. Concurrently, negative feedback rose from 29.11% to 31.4%. This result points to the pandemic’s probable impact on patient satisfaction. Year-by-year, positive sentiment rose through 2020 and then declined slightly in 2021. However, when we aggregated reviews into pre-COVID (2017–2019) versus COVID period (2020–2022), the odds of positive sentiment were higher during the COVID period (OR = 1.34, 95% CI 1.29–1.38). This suggests that sentiment shifts were not uniform across pandemic years. To assess this observation, we conducted a chi-square test comparing the distribution of positive, neutral, and negative reviews across pre-COVID and COVID periods. The test revealed a significant shift ( = 350.84, p < 0.0001), indicating that patient sentiment varied meaningfully across these periods. The shift in sentiment during the COVID-19 period was also confirmed through logistic regression and two-proportion z-tests. We fit a logistic regression model using only the COVID period as a binary predictor of review sentiment. The results showed a significant increase in the odds of positive sentiment during COVID-19 (p < 0.001), with an estimated 34% higher odds of positive reviews during the pandemic period. The model odds ratio for the COVID period was 1.34 (95% CI: 1.29–1.38), indicating a robust and consistent positive shift. This finding was supported by a two-proportion z-test comparing the fraction of positive reviews in the three years before COVID (2017–2019) versus the three years during the pandemic (2020–2022). The proportion of positive sentiment increased from 54.9% to 62.3%, (i.e., computed among reviews labeled positive or negative, excluding neutral and mixed labels) a statistically significant rise (Z = −18.37, p < 0.0001). This confirms that the pandemic period was associated with a meaningful upward shift in expressed patient satisfaction.

Word-frequency analysis suggested that staff behaviour, rather than clinical outcomes, drove both favorable and unfavorable reviews. Terms such as ‘nurse’ and ‘staff’ were widespread. Positive responses frequently included words like ‘professional’, ‘helpful’, and ‘caring’, indicating that qualities like competence and empathy form a foundation for trust. Negative reviews often cited ‘waiting’, ‘time’, ‘night’, and ‘hour’, suggesting long waits and logistical delays were a major source of frustration. An elevated mention of ‘doctor’ implied an opportunity to improve interactions with physicians, which could have shaped overall care perceptions. These insights signaled that interpersonal communication and efficient operations were top priorities for boosting satisfaction, mirroring global perspectives on patient-centered care [13].

Trends in negative and positive reviews

Analysis of patient reviews across different hospital units showed varied patterns in satisfaction. Some units reported more negative sentiments, while others held steady or improved positive ratings. The percentages listed below reflect the share of all positive (or all negative) reviews attributable to each unit in a given year (i.e., unit volume composition), not the within-unit positivity rate. To evaluate whether the pandemic affected sentiment differently across units, we first restricted the dataset to reviews labeled as either ‘Positive’ or ‘Negative’, excluding neutral or mixed responses. We then created a binary sentiment variable (1 = Positive, 0 = Negative) to serve as the outcome for a logistic regression model. The model included the review period (pre-pandemic vs during-pandemic), hospital unit type, and their interaction as predictors. Results showed that the effect of COVID-19 on sentiment varied significantly by unit (p = 0.012, likelihood-ratio test), indicating a heterogeneous response to the pandemic across hospital services.

Select results from this model are shown in Table 1. Day Surgery Units saw a significantly greater increase in positive sentiment during COVID (OR = 1.2760, p = 0.016), while Dentistry showed a significant decline (OR = 0.4140, p = 0.008). Oncology Units experienced a moderate but statistically significant increase in positivity (OR = 1.4569, p = 0.008). Emergency Departments remained significantly below average in baseline sentiment, but did not show a COVID-specific shift (p = 0.602, see Fig 5). Cardiology Units maintained a generally positive sentiment, with no significant change across the COVID boundary (p = 0.438, see Fig 5). The full results of this model, including all unit effects and interaction terms, are reported in S3 Table, along with model statistics and likelihood-ratio test.

thumbnail
Download:
Table 1. Effect of COVID-19 period on patient sentiment across hospital units: Selected results from logistic regression model with interaction terms.

https://doi.org/10.1371/journal.pdig.0000739.t001

The emergency unit reported a rise in negative reviews, moving from 36.1% in 2017 to 44.3% in 2021. This aligned with recent data showing Ontario’s ‘hallway health care’ challenges were worse than ever, as crowded hallways and bed shortages persisted [53]. Longer wait times, crowded facilities, and understaffing might explain this spike. Reports from Ontario’s Ministry of Health have noted that many emergency departments face staff shortages, which could have amplified patient dissatisfaction. Meanwhile, the maternity and pediatric units maintained low negative-review percentages. The maternity unit’s share of negative reviews shrank from 1.8% in 2017 to 0.3% in 2021, and the pediatric unit’s share went down from 4.1% to 1.1%. The medical/surgical unit also showed consistently lower negative reviews, varying from 4.1% in 2017 to 2.5% in 2021. This stability might reflect readiness for pandemic-related demands. Some local reports attribute this resilience to well-defined protocols that helped staff respond to workload surges without compromising care quality.

Although the emergency unit struggled with increased negative perceptions, it also showed a rise in positive reviews, climbing from 30.6% in 2017 to 44.7% in 2021. This shift may point to effective problem-solving measures in emergency care. In contrast, the maternity unit faced a drop in positive reviews, going from 3.6% in 2017 to 0.9% in 2021, while the pediatric unit’s positive reviews dropped from 5.8% to 2.5%. COVID-19 restrictions, such as limited visitors and face coverings, may have affected maternity and pediatric feedback [54–56]. These findings suggested a complex balance between providing safe care and addressing patient preferences, especially in units focused on women’s and children’s health.

Key clinical entities in patient reviews

Named entity recognition (NER) uncovered trends in frequently mentioned clinical topics in negative reviews. ‘Pain’ stood out as the most frequent term (Fig 2a), totaling 4,905 mentions (Fig 2b). Patients are often asked to describe their pain, which may drive these high counts.

thumbnail
Download:
Fig 2. Common entities in negative patient reviews (2015-July 2022).

(a) Bump chart showing annual ranking trends of frequently mentioned clinical entities, with 10 representing the most frequent. (b) Stacked bar chart (top) depicting raw counts and normalized area chart (bottom) showing the relative prevalence of the most frequent entities over time.

https://doi.org/10.1371/journal.pdig.0000739.g002

‘Infection’ was second most mentioned from 2017 to 2020 before dropping to seventh in 2022, indicating changing patient concerns over time. ‘Anxiety’ rose from being unranked in 2015 to among the top five from 2020 onward, reflecting a sharper focus on mental health in hospital reviews during the pandemic. ‘Bleeding’ and ‘stroke’ remained consistent, while ‘cancer’ and ‘fracture’ shifted in importance. These changes may match evolving hospital services or patient demographics. They also emphasize the need for flexible care models that respond to new demands, such as rising orthopedic cases or advanced cancer treatments.

In negative reviews, leading concerns included ‘pain’, ‘infection’, ‘bleeding’, ‘anxiety’, and ‘stroke’. In positive reviews, ‘pain’, ‘cancer’, ‘stroke’, ‘anxiety’, and ‘infection’ took the top spots. The changing positions of these terms between the pre-COVID and COVID-19 periods (Fig 2b) showed that patient priorities shifted under different conditions. Word frequency analysis also revealed that beyond clinical terms, patient satisfaction was linked to staff interactions and efficient service. Positive reviews featured words such as ‘professional’ and ‘caring’, whereas negative reviews included ‘waiting’ and ‘time’. This points to the importance of addressing operational hurdles and ensuring compassionate communication to strengthen patient trust.

Performance during COVID-19

Our analysis found that the percentage of negative reviews during the COVID-19 period was approximately 6 percentage points lower than during the non-COVID period, alongside a higher percentage of positive reviews. While this shift in sentiment may suggest increased public sympathy toward hospitals during the pandemic [57], or successful adaptations in service delivery, it should be interpreted as a correlation rather than a causal effect. Other possible explanations include a reduction in hospital visits, changes in triage protocols, or the expansion of telehealth services, which may have influenced patient sentiment during this time.

The analysis of key themes showed that during the COVID-19 period, ‘positive recognition’ had the highest percentage of positive reviews (73.18%), while ‘billing/accounting’ had the highest percentage of negative reviews (63.84%). In the non-COVID period, ‘positive recognition’ also had the highest percentage of positive reviews (73.77%), and ‘billing/accounting’ had the highest percentage of negative reviews (68.34%). This result implies that while many patients appreciated staff acknowledgment, billing issues remained a major source of dissatisfaction.

Comparisons of positive review percentages (Fig 3) suggested that ‘ICU/CCU’, ‘cardiology’, ‘radiology’, ‘families/friends’, ‘social services’, ‘discharge’, ‘information/education’, and ‘positive recognition’ received fewer positive reviews during COVID-19, suggesting the pandemic’s uneven impact. ICU/CCU and cardiology departments, for instance, were pushed to the limit by surging critical cases, possibly dampening patient satisfaction. Other themes showed higher positive review rates, pointing to the adaptability of some services.

thumbnail
Download:
Fig 3. Positive review percentages for key hospital themes before and during COVID-19.

Themes like ‘ICU/CCU’, ‘cardiology’, and ‘radiology’ experienced declines, indicating significant challenges during the pandemic. In contrast, other themes showed increased positive reviews, reflecting hospitals’ resilience in enhancing patient experiences amid unprecedented challenges.

https://doi.org/10.1371/journal.pdig.0000739.g003

To assess the impact of the COVID-19 pandemic on these patterns, we also used a logistic regression model that included both the themes and a binary indicator of whether the review occurred during the pandemic. We also included an interaction term between the theme and the COVID-19 period, allowing us to test whether the association between a given theme and sentiment changed during COVID.

The results showed that many themes were statistically significant predictors of patient sentiment (p < 0.001 for most), confirming their importance in shaping experiences. Moreover, several themes showed a clear shift during the pandemic. For example, while “emotional support” (OR = 0.73, − 0.83]) and “respect to patient” (OR = 0.78, − 0.87]) remained strong drivers of positivity overall, their sentiment impact weakened slightly during COVID, suggesting increased strain on interpersonal aspects of care. Conversely, themes like “infection prevention and control” (OR = 2.32, − 3.38]) and “housekeeping” (OR = 1.28, − 1.52]) saw a relative improvement in sentiment during the pandemic, potentially reflecting heightened public awareness or appreciation of these areas.

This statistical analysis supported the idea that not only did sentiment vary by theme, but the meaning and importance of those themes also shifted in response to the unique conditions of the pandemic. A complete summary of the regression model, including all themes and interaction terms, is provided in S4 Table.

Fig 4 details the trajectories of positive sentiment across specific hospital units and themes. Statistical analysis (Chi-square test) revealed significant declines in satisfaction in key areas. Notably, Pediatric units saw a sharp, statistically significant drop in positive sentiment regarding ‘Families/Friends’ (p < 0.001), likely reflecting the strain of visitation restrictions on family-centered care. Similarly, Inpatient units experienced significant decreases in positive feedback for ‘Social Services’ and ‘Parking/Transport’ (p < 0.001), indicated by the prominent red downward trajectories. In contrast, themes such as ‘Infection Control’ showed upward trends (blue trajectories) across several units (e.g., Emergency, p < 0.05), suggesting that despite operational challenges, patients recognized and appreciated the enhanced safety protocols implemented during the pandemic.

thumbnail
Download:
Fig 4. Shifts in positive patient sentiment by hospital unit and theme (Pre-COVID vs. COVID-19).

This comet chart visualizes the change in the percentage of positive reviews from the Pre-COVID (NC) period (comet tail) to the COVID-19 (C) period (comet head). On the left axis are hospital units, and along the top axis are key themes: information/education (I/E), medication/prescription (M/P), infection prevention & control (IC), families/friends (F/F), parking/transport (P/T), social services (SS), laboratory (Lab), and radiology (Rad). P-values displayed above each comet were calculated using a Chi-square test of independence comparing the proportion of positive vs. negative reviews between the two periods. An asterisk (*) indicates statistical significance (p < 0.05).

https://doi.org/10.1371/journal.pdig.0000739.g004

Positive review percentages rose for most hospital units during COVID-19 compared to the non-COVID period, with dentistry as a notable exception (31.25% vs. 20.57%). Cardiology and day surgery units saw relatively high positive ratings during COVID-19 (51.47% and 56.01%, respectively) (Fig 5a), which may reflect stronger infection control measures and patient safety protocols [58]. For dentistry and radiology, lower positive reviews (and higher negative ones) may stem from restricted service availability or heightened anxiety about close-contact procedures.

thumbnail
Download:
Fig 5. Positive and negative review percentages across different hospital units (COVID-19 vs. non-COVID periods).

(a) Percentages of positive reviews. (b) Percentages of negative reviews. This dual representation captures the shifts in patient satisfaction and dissatisfaction across different hospital units due to the pandemic, revealing critical insights into the impacts on healthcare services and areas needing targeted improvements.

https://doi.org/10.1371/journal.pdig.0000739.g005

Emergency and inpatient units had lower negative percentages during COVID-19 (32.81% and 26.89%) than before (38.91% and 31.92%), hinting that fewer non-urgent visits or increased public empathy may have contributed to slightly better perceptions (Fig 5b). These results illustrate how crisis responses can shape patient satisfaction [59]. Dentistry and radiology, on the other hand, indicate unmet needs that demand targeted solutions. A recent multicenter survey reported similar diverging trends, reinforcing the idea that some units handled acute pressure better than others [60].

Equity, Diversity, and Inclusion (EDI) analysis

Fig 6, generated with QGIS from shapefiles downloaded from the University of Toronto Maps Library, shows neighborhood boundaries used in our demographic sentiment analysis. Visible minority percentages ranged from <20% to >90%, yet high negative review rates did not always align with areas of higher minority density, indicating that other elements like socioeconomic conditions or local health resources may have been pivotal. Specifically non-COVID period reviews consistently received more negative reviews from the COVID period, suggesting persistent obstacles in everyday healthcare. Positive review rates varied by region, which can reflect possible differences in staffing or local health policies.

thumbnail
Download:
Fig 6. Visible minority demographics and hospital review sentiments in Toronto.

A heatmap of visible minority percentages across Toronto’s six hospital regions, alongside bar charts depicting patient review sentiments. Regions 1-4 exhibit >40% visible minority populations, while regions 5-6 show <40%. The base map data were obtained from the University of Toronto Maps and Data Library under a CC-BY license. The map was created in QGIS, with final styling completed for the manuscript

https://doi.org/10.1371/journal.pdig.0000739.g006

Visible minority analysis: Pandemic exacerbates existing inequalities– Analysis of annual sentiment averages reveals a widening gap during the pandemic. In the pre-COVID period (2016–2019), hospitals in high-minority areas maintained a positive review average of 43.6%. During the COVID-19 period (2020-2022), this average remained statistically unchanged (Mean = 44.9%, p=0.63). In contrast, hospitals in low-minority (‘Other’) areas saw a significant increase in positive sentiment, rising from a pre-COVID baseline of 41.6% to 47.6% during the pandemic (p = 0.015). This indicates that the general uplift in public gratitude observed in other regions was not equally felt in high-minority catchments (Fig 7a).

thumbnail
Download:
Fig 7. Comparison of mean sentiment before and during COVID-19 (2016-2022).

Grouped bar charts showing the percentage of positive (+) and negative (-) comments for (a) visible minority groups and (b) low-income groups compared to others. Dashed lines represent the mean sentiment for the Pre-COVID (2016–2019) and COVID period (2020–2022) periods. P-values (t-test) indicate the significance of the change in means between the two periods, where ns denotes not significant (p > 0.05), while (p < 0.05) indicates statistical significance.

https://doi.org/10.1371/journal.pdig.0000739.g007

To statistically examine this trend, we conducted a logistic regression using hospital minority status, COVID period, and their interaction as predictors of sentiment polarity. The model confirmed a strong overall increase in positive sentiment during the COVID period (p < 0.001). The interaction between COVID period and minority-serving hospitals was marginally significant (p = 0.053), suggesting a subtle but meaningful decline in sentiment among hospitals serving higher-minority populations during the pandemic. A detailed summary of the model is presented in S5 Table.

Low-income analysis: Economic vulnerability amplifies healthcare disparities– A similar pattern of stagnation was observed in low-income regions. While hospitals in higher-income (‘Other’) catchments saw their average positive reviews rise markedly from 41.9% (Pre-COVID) to 47.9% (COVID period) (p = 0.012), hospitals in low-income regions showed no statistically significant change, moving from a pre-COVID average of 44.9% to a COVID period average of 44.3% (p = 0.83). Furthermore, while higher-income areas benefited from a significant reduction in negative reviews (dropping from 34.4% to 29.8%, p = 0.046), low-income areas saw no such relief, with negative review proportions remaining stable (p = 0.86). As shown in Fig 7b, these trends highlight that economically vulnerable communities did not share in the broader improvements in hospital sentiment observed during the pandemic.

To statistically validate this reversal, we ran a logistic regression model that included low-income status, the COVID period, and their interaction. Prior to the pandemic, hospitals serving low-income populations were significantly more likely to receive positive sentiment (p < 0.001). However, the interaction term revealed a significant decline in positive sentiment during COVID (p < 0.001), confirming a regression of sentiment in these hospitals as the pandemic progressed. This shift aligns with our observed trends and supports the interpretation that economically vulnerable communities faced disproportionate strain during public health crises. A detailed summary of the model is included in S6 Table.

Discussion

Key observations

Below we present the main findings from our analysis, drawing attention to patterns that could reshape healthcare practices and policies.

Patient satisfaction as a launchpad for change

In today’s digital age, over 70% of patients research hospital reviews before deciding where to receive care, a significant increase from just 25% in 2013 [61], demonstrating the growing impact of online feedback on patient decisions. However, despite this trend, a recent report reveals that Ontario hospitals, while collecting substantial patient feedback, kept the findings confidential, leaving patient pleas for improvements unheard [60]. The finding that over 80% of hospitals had less than 50% positive reviews signals a pressing need for major overhaul in our healthcare system, aligning with Canada’s last-place ranking among 11 high-income nations in timeliness and efficiency of care [62], indicating a critical need for broad reform. The COVID-19 period was associated with measurable shifts in patient sentiment. While our aggregated models show higher overall odds of positive sentiment during COVID compared to pre-COVID, year-by-year patterns suggest that sentiment fluctuated across pandemic years, with some areas showing declines in positivity. From 2020 onwards, the gap between negative and positive reviews narrowed, suggesting a growing sense of dissatisfaction. This decline stems from intense pressure on healthcare resources, rapid adoption of new safety measures, and rising patient anxiety during an unpredictable global crisis [24]. Nevertheless, the high proportion of negative reviews may partly reflect self-selection bias, where patients who experience dissatisfaction are more inclined to leave feedback. Furthermore, the data collection methods used by hospitals or external review platforms could amplify this effect. However, the consistent pattern of concerns across multiple hospitals suggests that significant systemic issues remain unaddressed, and the urgency of such issues is not diminished by potential sampling biases.

Impact of staff behaviour on patient perceptions

Our analysis shows that human interaction is central in healthcare. Positive reviews often mentioned staff attributes such as ‘professionalism’, ‘helpfulness’, and ‘attentiveness’, aligning with research that calls attention to communication, empathy, and respectful treatment as key factors in patient experiences and outcomes. Negative reviews, in contrast, frequently pointed out long wait times, which is consistent with studies showing that excessive waits can lower satisfaction [63,64].

These patterns underline the importance of well-rounded training initiatives that target clinical skills, interpersonal engagement, emotional intelligence, and patient-focused dialogue. Creating a culture of compassion and respect can boost patient satisfaction and overall well-being [65–67]. The frequent mention of ‘doctor’ in negative reviews is especially concerning, suggesting issues such as high workloads, time constraints, or insufficient interpersonal training. Further investigation is warranted, possibly through targeted surveys, focus groups, and robust feedback analysis. We advocate for holistic training, more inclusive workplace cultures, effective strategies to shorten wait times [68,69], and real-time monitoring of patient feedback to address these critical issues.

Challenges and adaptations during the COVID-19 pandemic

The COVID-19 pandemic introduced extraordinary hurdles for healthcare systems. Our analysis showed evidence of changes in patient satisfaction trends during this period. Negative reviews often included ‘anxiety’ and ‘infection’, reflecting the pandemic’s mental and physical toll on patients [24]. Surprisingly, the share of negative reviews dropped while positive reviews rose during COVID-19 compared to pre-pandemic times. This might be attributed to increased patient empathy for overburdened hospitals, heightened focus on infection control, and adjusted patient expectations favoring basic care delivery over non-essential aspects [70,71].

Still, some hospital units showed disparities. The emergency unit consistently received high negative feedback, while the day surgery unit maintained higher positive ratings. This difference hints that certain services were prioritized, possibly resulting in uneven patient experiences across departments.

Drivers of patient satisfaction and dissatisfaction

Our analysis uncovered major factors that shape satisfaction or dissatisfaction. ‘Positive recognition’ consistently drove satisfaction, garnering the highest proportion of positive feedback before and during COVID-19. This finding reflects the value of recognition and gratitude in healthcare. Problems with ‘billing and accounting’, on the other hand, were a leading cause of dissatisfaction, with 68.34% negative reviews in the non-COVID period and 63.84% negative reviews in the COVID period, which indicates a need for clearer billing procedures and improved communication around financial matters.

We discovered that high costs for ancillary items and prescriptions contributed significantly to dissatisfaction, a phenomenon also noted in Canadian studies of out-of-pocket drug expenses [72]. Some patients also pay extra to skip long queues, intensifying their financial burden and unhappiness [72,73]. These data call for more transparent billing practices and action on healthcare affordability.

Notably, certain themes such as ‘ICU/CCU’, ‘cardiology’, ‘radiology’, ‘families/friends’, ‘social services’, ‘discharge’, ‘information/education’, and ‘positive recognition’ saw lower positive feedback during COVID-19 compared to previous years, implying that these services struggled to sustain patient satisfaction amid pandemic-related disruptions.

Demographic disparities and health equity

Our examination of hospitals in areas with different percentages of visible minorities and low-income populations revealed clear differences in patient satisfaction patterns. From 2016 to 2020, facilities in regions with higher minority and low-income concentrations generally had better reviews, but this reversed in 2021–2022, with these same hospitals receiving considerably worse feedback. This abrupt shift may reflect the following structural imbalances:

Resource gaps: Financial constraints have worsened, as several Ontario hospitals face significant challenges balancing their budgets [74], limiting opportunities for expansions or improved services. Infrastructure shortfalls further compound the crisis, with many hospital buildings in poor condition and in need of repair [75]. The pandemic hit all hospitals hard, but those serving minority and low-income areas experienced especially acute pressures. These safety-net hospitals, already underfunded, lacked the capacity to handle surging demands and were forced to halt income-generating elective services, compounding their financial strain [76].

Unequal health impacts: Studies show that visible minority groups had higher rates of COVID-19 infection, hospitalization, and mortality [77–80]. This likely fueled higher negative feedback from these communities.

Socioeconomic pressures: Crowded housing, job instability, and limited preventive care made low-income communities more vulnerable to COVID-19 [81]. Hospitals in these areas faced chronic resource shortages, further stretching staff and supply chains [82,83].

Language barriers: Non-English speakers faced substantial communication challenges. One U.S. study found a two-fold increase in COVID-19 hospitalizations and over two times higher risk of death among non-English speakers [84].

Trust issues: Historical mistreatment of minority communities contributed to vaccine hesitancy and reluctance to seek care. Statistics Canada found that only 56.4% of the Black population was open to vaccination, compared to 77.7% of White and 82.5% of South Asian populations [85].

These findings point to the need for resilient healthcare systems that prioritize equity, particularly in times of crisis. Future work should identify robust care models for marginalized populations and explore how social determinants of health shape patient experiences.

Implications for healthcare management and policy

AI-enhanced pain management ecosystems: An AI-based approach to pain management could transform patient care by using patient histories, treatment data, and real-time vitals to generate personalized pain relief strategies. This might help the 20% of Canadians who experience chronic pain and reduce the annual $38.2–$40.3 billion cost of pain management [86]. Yet, biases in AI algorithms, data privacy concerns, and the potential loss of human empathy remain real risks [87–89]. Emerging Canadian reports also convey the potential for rapid AI integration in healthcare settings could reshape patient-provider dynamics [90], while another recent study found that real-time AI alerts cut patient deaths by 43% [91].

Integrated mental health frameworks: Embedding mental health screening into regular medical workflows could address the uptick in ‘anxiety’ found in negative reviews from 2017 to 2022. With one in five Canadians facing mental illness each year, and around 4,000 annual suicides [92], building mental health support into hospital care could improve both physical and mental health outcomes.

Predictive emergency care systems: Advanced data analytics in emergency departments could improve patient flow and resource deployment. This strategy may counter the growing negative reviews in the Emergency unit, which rose from 36.1% in 2017 to 44.3% in 2021. Predictive models can forecast patient volumes with up to 90% accuracy [93], helping reduce wait times. Simulation-based methods and machine learning also show promise for identifying bottlenecks and providing real-time wait-time predictions [94,95], potentially increasing patient satisfaction in Canadian Emergency Departments (EDs), where 90% of non-admitted patients finish their visit within 9.1 hours [96]. Interestingly, after adopting SurgeCon, a platform that uses real-time analytics to monitor capacity and predict surges, Carbonear General Hospital in Newfoundland and Labrador cut physician wait times from 104 to 42 minutes and total ED time from 199 to 134 minutes [97].

Blockchain-enabled billing transparency: Blockchain, a decentralized digital ledger, provides real-time updates across a peer-to-peer network with tamper-resistant data integrity [98]. This technology is particularly suited to addressing healthcare billing challenges, where clarity and trust are paramount. Blockchain-based billing systems can provide transparent, immutable records, reducing disputes and enabling patients to better understand their medical expenses. This is especially critical given the significant patient dissatisfaction (68.34%) related to billing issues. Beyond transparency, the financial benefits are substantial. Research suggests that blockchain could save the healthcare industry $20 billion annually by streamlining data management and reducing inefficiencies [99]. Real-world implementations also showcase its potential. In 2019, Anthem, the second-largest health insurer in the U.S., announced plans to use blockchain to securely manage health data for 40 million patients [100]. Similarly, the United Arab Emirates (UAE) became the first nation to integrate blockchain and AI into its organ transplant system [101], further demonstrating the transformative potential of this technology.

Recognition-centric care environments: Environments that value staff recognition could boost patient satisfaction, reinforcing the 73.18% positive reviews tied to positive recognition during COVID-19. Studies suggest that better staff appreciation can lead to lower turnover rates and better patient safety [102,103]. However, the pandemic showed that recognition alone cannot solve deeper challenges like heavy workloads, stress, and inadequate pay [104,105]. For example, 90% of nurses reported increased workload since COVID-19 started, and 57% faced financial difficulties [105]. A comprehensive strategy that addresses these root issues is essential for lasting improvements in workforce morale and patient experiences.

Adaptive care matrices for specialized units: Flexible care frameworks in specialized areas such as dentistry or radiology could guard against service disruptions. Hospitals that shifted resources quickly during the pandemic saw fewer bottlenecks and better patient outcomes [106–110]. Implementing these strategies may prevent spikes in negative reviews when normal operations are disrupted.

Health equity zone frameworks: Targeted healthcare programs in high-minority regions might reverse the decline in positive feedback during the pandemic and honor Canada’s goal to reduce health inequalities. Research indicates that indigenous and low-income groups experience higher rates of mental illness and worse health outcomes [111], calling for policy interventions like equity zones to improve care and well-being [112,113].

Building on the interventions described above, putting them into practice will require strong privacy protections, regular performance reviews, and integration with existing hospital systems. For AI tools, whether in pain management, feedback dashboards, or predictive emergency care, this means secure data handling, safeguards against bias, and pilot testing in high-impact areas before wider rollout. Blockchain billing can be introduced in stages, starting with high-volume services to demonstrate transparency and train staff in secure processes. Interventions that focus on mental health, staff recognition, or flexible care models should be embedded into everyday workflows and supported by policies that address workload, resources, and equity. Health equity zone initiatives will benefit from targeted outreach, resource allocation, and partnerships with community organizations. Across all approaches, small-scale pilots and active involvement of staff and patients in design will help identify challenges early and improve adoption.

Limitations

This study offers valuable insights but has certain limitations. First, the absence of healthcare provider perspectives limits our view of the factors that shape satisfaction, although our analysis of patient reviews across sentiment, clinical entities, and themes provides a broad perspective. Second, our lack of detailed demographic data on patient ethnicity and gender might introduce bias, so we utilized regional statistics on visible minorities and low-income populations to approximate healthcare equity. Third, not every patient leaves a review, and dissatisfied individuals are often more likely to do so, which may skew findings and lead to a selection bias.

Fourth, several contextual factors that we could not fully model may also shape patient sentiment. During early pandemic waves, many hospitals postponed elective procedures, limited visitor access, and expanded telemedicine, changes that could have altered both the volume and tone of reviews. Public campaigns that celebrated front-line workers may likewise have encouraged more favourable posts. Furthermore, online feedback platforms capture a self-selected subset of patients, often those with strong opinions or reliable internet access. Future work that links review text with administrative records or population surveys could help to quantify these influences and refine the estimates reported here.

Fifth, we removed incomplete data, potentially excluding some experiences, and ambiguous hospital unit names required approximate mappings, possibly introducing inaccuracies. Sixth, the dataset from 2015 is relatively small (about 500 reviews), which may limit robustness for that year, but it still revealed consistent trends. Seventh, we do not know which sites are teaching hospitals, a factor that can affect patient perceptions. Finally, without contextual information about each patient’s circumstances, we cannot fully dissect the roots of their feedback. However, we employed advanced natural language processing methods to extract deeper insights, partly offsetting this gap.

Despite these constraints, our findings establish a strong platform for future investigations. They spotlight key areas like staff interactions, billing transparency, and health equity, offering clear pathways for policy and practice improvements.

Conclusion

This analysis of patient reviews indicates major gaps in hospital patient satisfaction before and during the COVID-19 pandemic, with many hospitals struggling to meet rising patient expectations. Staff behaviour stands out as a decisive factor, as kindness, professionalism, and helpfulness shape positive perceptions. Hospitals that invest in empathy, improved communication, and patient-focused care may see meaningful boosts in satisfaction. The pandemic affected patient satisfaction unevenly across hospital units. Some units maintained higher positive ratings, while others faced recurring problems linked to anxiety and infection. The dip in negative reviews, along with a rise in positive ones, may suggest greater public empathy or improvements in service quality. However, this also illustrates how quickly patient expectations can shift during crises.

Regional disparities further complicate the picture, as satisfaction patterns switched course during the pandemic, stressing the importance of coordinated health equity measures. These findings offer a clear framework for hospitals aiming to refine patient experiences, by tightening communication, resolving shortcomings in low-performing units, encouraging staff recognition, and tackling billing challenges. Interestingly, a recent Ontario-based study found that hospitals rarely shared patient feedback with the public, limiting opportunities for improvement [60]. Relying on ongoing analysis of patient reviews, supported by NLP techniques, can identify emerging concerns early and direct adjustments that keep pace with changing community needs.

Supporting information

S1 Table. Standardized hospital unit names.

Standardized list of hospital unit names used for NLP analysis.

https://doi.org/10.1371/journal.pdig.0000739.s001

(PDF)

S2 Table. Thematic satisfaction categories.

Thematic satisfaction categories used in patient review classification.

https://doi.org/10.1371/journal.pdig.0000739.s002

(PDF)

S3 Table. Full results of COVID × Unit logistic regression.

Model summary, likelihood-ratio test, and complete coefficient estimates for Sentiment ∼ CovidPeriod × Unit.

https://doi.org/10.1371/journal.pdig.0000739.s003

(PDF)

S4 Table. Logistic regression with CovidPeriod × ThemeCode interaction.

Model summary, likelihood-ratio test, and full coefficients for Sentiment ∼ CovidPeriod × ThemeCode.

https://doi.org/10.1371/journal.pdig.0000739.s004

(PDF)

S5 Table. Logistic regression with CovidPeriod × Minority interaction.

Model summary and coefficient estimates for Sentiment ∼ CovidPeriod × Minority.

https://doi.org/10.1371/journal.pdig.0000739.s005

(PDF)

S6 Table. Logistic regression with CovidPeriod × Low_income interaction.

Model summary and coefficient estimates for Sentiment ∼ CovidPeriod × Low_income.

https://doi.org/10.1371/journal.pdig.0000739.s006

(PDF)

References

  1. 1. Bastemeijer CM, Boosman H, Zandbelt L, Timman R, de Boer D, Hazelzet JA. Patient Experience Monitor (PEM): the development of new short-form picker experience questionnaires for hospital patients with a wide range of literacy levels. Patient Relat Outcome Meas. 2020;11:221–30. pmid:33312007
  2. 2. Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. National Academies Press; 2001.
    • 3. Coulter A, Locock L, Ziebland S, Calabrese J. Collecting data on patient experience is not enough: they must be used to improve care. BMJ. 2014;348:g2225. pmid:24671966
    • 4. Boulding W, Glickman SW, Manary MP, Schulman KA, Staelin R. Relationship between patient satisfaction with inpatient care and hospital readmission within 30 days. The American Journal of Managed Care. 2011;17(1):41–8.
    • 5. Bragge P, Delafosse V, Kellner P, Cong-Lem N, Tsering D, Giummarra MJ, et al. Relationship between staff experience and patient outcomes in hospital settings: an overview of reviews. BMJ Open. 2025;15(1):e091942. pmid:39773811
    • 6. Manary MP, Boulding W, Staelin R, Glickman SW. The patient experience and health outcomes. New England Journal of Medicine. 2013;368(3):201–3.
    • 7. Anhang Price R, Elliott MN, Zaslavsky AM, Hays RD, Lehrman WG, Rybowski L, et al. Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev. 2014;71(5):522–54. pmid:25027409
    • 8. Zgierska A, Rabago D, Miller MM. Impact of patient satisfaction ratings on physicians and clinical care. Patient Prefer Adherence. 2014;8:437–46. pmid:24729691
    • 9. Glickman SW, Boulding W, Manary M, Staelin R, Roe MT, Wolosin RJ, et al. Patient satisfaction and its relationship with clinical quality and inpatient mortality in acute myocardial infarction. Circ Cardiovasc Qual Outcomes. 2010;3(2):188–95. pmid:20179265
    • 10. Jaipaul CK, Rosenthal GE. Do hospitals with lower mortality have higher patient satisfaction? A regional analysis of patients with medical diagnoses. Am J Med Qual. 2003;18(2):59–65. pmid:12710554
    • 11. Tsai TC, Orav EJ, Jha AK. Patient satisfaction and quality of surgical care in US hospitals. Ann Surg. 2015;261(1):2–8. pmid:24887985
    • 12. Deloitte Center for Health Solutions. Value of patient experience: Hospitals with higher patient experience scores have higher clinical quality. Deloitte; 2017.
      • 13. Sacks GD, Lawson EH, Dawes AJ, Russell MM, Maggard-Gibbons M, Zingmond DS, et al. Relationship between hospital performance on a patient satisfaction survey and surgical quality. JAMA Surg. 2015;150(9):858–64. pmid:26108091
      • 14. Quigley DD, Reynolds K, Dellva S, Price RA. Examining the business case for patient experience: a systematic review. J Healthc Manag. 2021;66(3):200–24. pmid:33960966
      • 15. Congiusta S, Solomon P, Conigliaro J, O’Gara-Shubinsky R, Kohn N, Nash IS. Clinical quality and patient experience in the adult ambulatory setting. Am J Med Qual. 2019;34(1):87–91. pmid:29790371
      • 16. Canadian Institute for Health Information. How Canada compares: results from the Commonwealth Funds 2019 international health policy survey of primary care physicians in 11 countries—data tables. Ottawa, ON: CIHI; 2019.
        • 17. CTV News Montreal. This Montreal man died of an aneurysm after waiting in the ER for six hours; 2024.
          • 18. Statista. Number of family physicians by province Canada; 2024.
            • 19. Li K, Frumkin A, Bi WG, Magrill J, Newton C. Biopsy of Canada’s family physician shortage. Fam Med Community Health. 2023;11(2):e002236. pmid:37173094
            • 20. Canadian Institute for Health Information. Your health system. Canadian Institute for Health Information; 2021.
              • 21. Marchildon GP, Allin S. The public-private mix in the delivery of health-care services: its relevance for lower-income Canadians. Glob Soc Welf. 2016;3(3):161–70.
              • 22. Canadian Institute for Health Information. Repeat Hospital Stays for Mental Health and Substance Use. 2025.
                • 23. Centre for Addiction and Mental Health. Mental Illness and Addiction: Facts and Statistics. 2025. https://www.camh.ca/en/driving-change/the-crisis-is-real/mental-health-statistics
                  • 24. Filip R, Gheorghita Puscaselu R, Anchidin-Norocel L, Dimian M, Savage WK. Global challenges to public health care systems during the COVID-19 pandemic: a review of pandemic measures and problems. J Pers Med. 2022;12(8):1295. pmid:36013244
                  • 25. Moore D, Trottier H. Ongoing challenges three years after the start of the COVID-19 pandemic. Can J Public Health. 2023;114(3):325–7. pmid:37103776
                  • 26. Curnow E, Tyagi V, Salisbury L, Stuart K, Melville-Jóhannesson B, Nicol K, et al. Person-centered healthcare practice in a pandemic context: an exploration of people’s experience of seeking healthcare support. Front Rehabil Sci. 2021;2:726210. pmid:36188789
                  • 27. Aboumatar HJ. Three reasons to focus on patient and family engagement during the COVID-19 pandemic. Quality Management in Health Care. 2020.
                  • 28. Buerhaus PI, Auerbach DI, Donelan K. Nurse employment during the first 15 months of the COVID-19 pandemic. Health Affairs. 2021;40(10):1642–51.
                  • 29. Lasater KB, Aiken LH, Sloane DM, French R, Anusiewicz CV, Martin B. COVID-19 and the risk of burnout among nurses: a cross-sectional survey study. International Journal of Nursing Studies. 2021;121:104002.
                  • 30. Schroeder C, Schulte F. COVID-19: a crisis in care–the impact of COVID-19 on nursing and the workforce in Canada. The Lancet. 2021;397(10271):1102–3.
                  • 31. CUPE Ontario. No respite for Ontario patients: new report says five times more hospital beds needed than planned by Ford government. 2024. https://cupe.on.ca/no-respite-for-ontario-patients/
                    • 32. Alemi F, Torii M, Clementz L, Aron DC. Feasibility of real-time satisfaction surveys through automated analysis of patients’ unstructured comments and sentiments. Qual Manag Health Care. 2012;21(1):9–19. pmid:22207014
                    • 33. Bahja M, Lycett M. Identifying patient experience from online resources via sentiment analysis and topic modelling. In: Proceedings of the 3rd IEEE/ACM International Conference on Big Data Computing, Applications and Technologies. 2016. p. 94–9. https://doi.org/10.1145/3006299.3006335
                      • 34. Greaves F, Ramirez-Cano D, Millett C, Darzi A, Donaldson L. Machine learning and sentiment analysis of unstructured free-text information about patient experience online. The Lancet. 2012;380:S10.
                      • 35. Doing-Harris K, Mowery DL, Daniels C, Chapman WW, Conway M. Understanding patient satisfaction with received healthcare services: a natural language processing approach. AMIA Annu Symp Proc. 2017;2016:524–33. pmid:28269848
                      • 36. Khanbhai M, Anyadi P, Symons J, Flott K, Darzi A, Mayer E. Applying natural language processing and machine learning techniques to patient experience feedback: a systematic review. BMJ Health Care Inform. 2021;28(1):e100262. pmid:33653690
                      • 37. van Buchem MM, Neve OM, Kant IMJ, Steyerberg EW, Boosman H, Hensen EF. Analyzing patient experiences using natural language processing: development and validation of the artificial intelligence patient reported experience measure (AI-PREM). BMC Med Inform Decis Mak. 2022;22(1):183. pmid:35840972
                      • 38. Vashishtha E, Kapoor H. Enhancing patient experience by automating and transforming free text into actionable consumer insights: a Natural Language Processing (NLP) approach. Int J Health Sci Res. 2023;13(10):275–88.
                      • 39. Cammel SA, De Vos MS, van Soest D, Hettne KM, Boer F, Steyerberg EW, et al. How to automatically turn patient experience free-text responses into actionable insights: a natural language programming (NLP) approach. BMC Med Inform Decis Mak. 2020;20(1):97. pmid:32460734
                      • 40. Khanbhai M, Warren L, Symons J, Flott K, Harrison-White S, Manton D, et al. Using natural language processing to understand, facilitate and maintain continuity in patient experience across transitions of care. Int J Med Inform. 2022;157:104642. pmid:34781167
                      • 41. Nawab K, Ramsey G, Schreiber R. Natural language processing to extract meaningful information from patient experience feedback. Applied Clinical Informatics. 2020;11(2):242–52.
                      • 42. Loria S. Textblob Documentation. Release 015. 2018;2.
                        • 43. Neumann M, King D, Beltagy I, Ammar W. ScispaCy: fast and robust models for biomedical natural language processing. In: Proceedings of the 18th BioNLP Workshop and Shared Task. 2019. p. 319–27. https://doi.org/10.18653/v1/w19-5034
                          • 44. Allen Institute for AI. ScispaCy Documentation. 2025. https://allenai.github.io/scispacy/
                            • 45. City H. HealthyPlan.City: Our Research. 2024. https://healthyplan.city/en/our-research
                              • 46. Noaeen M, Doiron D, Syer J, Brook J. Advancing population health through open environmental data platforms. Principles and advances in population neuroscience. Springer Nature; 2024. p. 1–27.
                                • 47. Canadian Urban Environmental Health Research Consortium CANUE. CANUE: Canadian Urban Environmental Health Research Consortium. 2025. https://canue.ca
                                  • 48. HealthyPlan C. Methods. 2024. https://healthyplan.city/methods
                                    • 49. Doiron D, Setton EM, Syer J, Redivo A, McKee A, Noaeen M, et al. HealthyPlan.City: a web tool to support urban environmental equity and public health in canadian communities. J Urban Health. 2024;101(3):497–507. pmid:38587782
                                    • 50. World Health Organization. COVID-19 Public Health Emergency of International Concern (PHEIC) Global Research and Innovation Forum: Towards a Research Roadmap. 2020.
                                      • 51. World Health Organization. Statement on the fifteenth meeting of the International Health Regulations 2005 Emergency Committee regarding the coronavirus disease (COVID-19) pandemic. World Health Organization; 2023.
                                        • 52. Seabold S, Perktold J. Statsmodels: econometric and statistical modeling with python. In: Proceedings of the Python in Science Conference. 2010. p. 92–6. https://doi.org/10.25080/majora-92bf1922-011
                                          • 53. Chamandy A. Ontario’s hallway health care problem worse than ever, hospital data shows. BarrieToday. 2024.
                                          • 54. Verma A. The impact of COVID-19 on maternal and neonatal services: a systematic review. Journal of Maternal-Fetal & Neonatal Medicine. 2021;34(15):2472–80.
                                          • 55. Friedman KG, et al. The impact of COVID-19 on health care delivery for children. Journal of the American Academy of Pediatrics. 2020;146(3):e2020038286.
                                          • 56. Moroz L. The effect of COVID-19 on maternity services globally: a systematic review and meta-analysis. The Lancet Global Health. 2020;8(7):e1090–9.
                                          • 57. Barello S, Graffigna G. Caring for health professionals in the COVID-19 pandemic emergency: toward an “epidemic of empathy” in healthcare. Frontiers in Psychology. 2020;11.
                                          • 58. Canadian Institute for Health Information. Impact of COVID-19 on Canada’s Health Care Systems: Hospital Services. 2023.
                                            • 59. Canadian Institute for Health Information. Hospital staffing and hospital harm: trends throughout the pandemic. 2023.
                                              • 60. Institute for Journalism and Betterment. Ontario hospitals asked patients about the care they received. Results were kept secret and pleas for change went ignored. 2023.
                                                • 61. Advice S. How patients use online reviews when choosing healthcare providers. 2023.
                                                  • 62. Schneider EC, Shah A, Doty MM, Tikkanen R, Fields K, Williams RD. Mirror, mirror 2020 : reflecting poorly. Health care in the U.S. compared to other high-income countries. The Commonwealth Fund; 2020. https://www.commonwealthfund.org/publications/fund-reports/2020/aug/mirror-mirror-2020-reflecting-poorly
                                                    • 63. Xie Z, Or C. Associations between waiting times, service times, and patient satisfaction in an endocrinology outpatient department: a time study and questionnaire survey. Inquiry. 2017;54:46958017739527. pmid:29161947
                                                    • 64. Bleustein C, Rothschild DB, Valen A, Valatis E, Schweitzer L, Jones R. Wait times, patient satisfaction scores, and the perception of care. Am J Manag Care. 2014;20(5):393–400. pmid:25181568
                                                    • 65. Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3(1):e001570. pmid:23293244
                                                    • 66. Boissy A, Windover AK, Bokar D, Karafa M, Neuendorf K, Frankel RM, et al. Communication skills training for physicians improves patient satisfaction. J Gen Intern Med. 2016;31(7):755–61. pmid:26921153
                                                    • 67. Levinson W, Lesser CS, Epstein RM. Developing physician communication skills for patient-centered care. Health Aff (Millwood). 2010;29(7):1310–8. pmid:20606179
                                                    • 68. Roshanaei V, Luong C, Aleman DM, Urbach DR. Collaborative operating room planning and scheduling. INFORMS Journal on Computing. 2017;29(3):558–80.
                                                    • 69. Roshanaei V, Booth KEC, Aleman DM, Urbach DR, Beck JC. Branch-and-check methods for multi-level operating room planning and scheduling. International Journal of Production Economics. 2020;220:107433.
                                                    • 70. Aguirre S, Jogerst KM, Ginsberg Z, Voleti SS, Bhullar P, Spegman J. COVID-19 impact on the doctor-patient relationship: patient perspectives on emergency physician empathy and communication. Bulletin of Emergency & Trauma. 2021;9:125–32.
                                                    • 71. Czeisler ME, Tynan MA, Howard ME, Honeycutt S, Fulmer EB, Kidder DP, et al. Public attitudes, behaviors,, beliefs related to COVID-19, stay-at-home orders, nonessential business closures,, public health guidance - United States, New York City,, Los Angeles and May 5-12 2020. MMWR Morbidity and Mortality Weekly Report. 2020;69(24):751–8.
                                                    • 72. Law MR, Cheng L, Kolhatkar A, Goldsmith LJ, Morgan SG, Holbrook AM, et al. The consequences of patient charges for prescription drugs in Canada: a cross-sectional survey. CMAJ Open. 2018;6(1):E63–70. pmid:29440236
                                                    • 73. Canadian Institute for Health Information. Clinical factors contributing to high cost hospitalizations in a Canadian hospital. Healthcare Management Forum. 2017.
                                                    • 74. Bogdan S. Ontario hospitals failing to balance books amid financial challenges. Global News. 2024.
                                                    • 75. Callan I, D’Mello C. ‘Huge wake-up call’: Quarter of Ontario hospitals are in poor condition, docs reveal. Global News. 2023.
                                                    • 76. Cleveland Manchanda EC, Molina MF, Rodriguez RM. Racial equity in crisis standards of care-reassuring data or reason for concern?. JAMA Netw Open. 2021;4(3):e214527. pmid:33739428
                                                    • 77. Tai DBG, Shah AS, Doubeni CA, Sia IG, Wieland ML. The disproportionate impact of COVID-19 on racial and ethnic minorities in the United States. Clinical Infectious Diseases: An Official Publication of the Infectious Diseases Society of America. 2020.
                                                    • 78. Acosta AM, Garg S, Pham H, Whitaker M, Anglin O, O’Halloran AC, et al. Racial, ethnic disparities in rates of COVID-19–associated hospitalization, intensive care unit admission and in-hospital death in the United States from March 2020 to February 2021. JAMA Network Open. 2021;4.
                                                    • 79. Lopez LP, Hart LH, Katz MH. Racial and ethnic health disparities related to COVID-19. JAMA. 2021.
                                                    • 80. Boserup B, Mckenney MG, Elkbuli A. Disproportionate impact of COVID-19 pandemic on racial and ethnic minorities. The American Surgeon. 2020;86:1615–22.
                                                    • 81. Brakefield WS, Olusanya OA, White B, Shaban-Nejad A. Social determinants and indicators of COVID-19 among marginalized communities: a scientific review and call to action for pandemic response and recovery. Disaster Med Public Health Prep. 2022;17:e193. pmid:35492024
                                                    • 82. Douglas JA, Subica AM. COVID-19 treatment resource disparities and social disadvantage in New York City. Prev Med. 2020;141:106282. pmid:33035550
                                                    • 83. Weech-Maldonado R, Lord J, Davlyatov GK, Ghiasi A, Orewa GN. High-minority nursing homes disproportionately affected by COVID-19 deaths. Frontiers in Public Health. 2021;9.
                                                    • 84. Quadri NS, Knowlton G, Vazquez Benitez G, Ehresmann KR, LaFrance AB, DeFor TA, et al. Evaluation of preferred language and timing of COVID-19 vaccine uptake and disease outcomes. JAMA Netw Open. 2023;6(4):e237877. pmid:37043199
                                                    • 85. Olanlesi-Aliu A, Kemei J, Alaazi D, Tunde-Byass M, Renzaho A, Sekyi-Out A, et al. COVID-19 among Black people in Canada: a scoping review. Health Promot Chronic Dis Prev Can. 2024;44(3):112–25. pmid:38501682
                                                    • 86. Canadian Pain Task Force. Canadian Pain Task Force Report: March 2021. 2021.
                                                      • 87. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447–53. pmid:31649194
                                                      • 88. Price WN, Cohen IG. Privacy and the regulation of artificial intelligence. Columbia Law Review. 2019;119(7):1901–37.
                                                      • 89. Char DS, Shah NH, Magnus D. Implementing machine learning in health care - addressing ethical challenges. N Engl J Med. 2018;378(11):981–3. pmid:29539284
                                                      • 90. Verma AA, Stukel TA, Colacci M, Bell S, Ailon J, Friedrich JO, et al. Clinical evaluation of a machine learning-based early warning system for patient deterioration. CMAJ. 2024;196(30):E1027–37. pmid:39284602
                                                      • 91. Levin MA, Kia A, Timsina P, Cheng F-Y, Nguyen K-A-N, Kohli-Seth R, et al. Real-time machine learning alerts to prevent escalation of care: a nonrandomized clustered pragmatic clinical trial. Crit Care Med. 2024;52(7):1007–20. pmid:38380992
                                                      • 92. Centre for Addiction and Mental Health. Mental Illness and Addiction: Facts and Statistics; 2021. https://www.camh.ca/en/driving-change/the-crisis-is-real/mental-health-statistics
                                                        • 93. ALZA CARE. Empowering healthcare: Streamlining patient flow and resource allocation with predictive analytics. 2023.
                                                          • 94. Kim J-K. Enhancing patient flow in emergency departments: a machine learning and simulation-based resource scheduling approach. Applied Sciences. 2024;14(10):4264.
                                                          • 95. Benevento E, Aloini D, Squicciarini N. Towards a real-time prediction of waiting times in emergency departments: a comparative analysis of machine learning techniques. Business and International Management. 2023.
                                                          • 96. Canadian Institute for Health Information. NACRS Emergency Department Visits and Lengths of Stay. CIHI; 2023. https://www.cihi.ca/
                                                            • 97. Health Research (CIHR) CI. SurgeCon: Priming a Community Emergency Department for Patient Flow Management. 2023. https://cihr-irsc.gc.ca/e/53164.html
                                                              • 98. Nakamoto S. Bitcoin: a peer-to-peer electronic cash system. 2008.
                                                                • 99. Khezr S, Moniruzzaman M, Yassine A, Benlamri R. Blockchain technology in healthcare: a comprehensive review and directions for future research. Applied Sciences. 2019;9(9):1736.
                                                                • 100. Attaran M. Blockchain technology in healthcare: challenges and opportunities. International Journal of Healthcare Management. 2020;15(1):70–83.
                                                                • 101. Blockchain U. UAE first country in the world to use blockchain for organ donation. 2019.
                                                                  • 102. C A Short Company. The Impact of Recognition on Healthcare Management and HCAHPS Scores; 2021. https://www.cashort.com/
                                                                    • 103. Aiken LH, Sloane DM, Ball J, Bruyneel L, Rafferty AM, Griffiths P. Patient satisfaction with hospital care and nurses in England: an observational study. BMJ Open. 2018;8(1):e019189. pmid:29326193
                                                                    • 104. Registered Nurses’ Association of Ontario. Nursing through crisis: a comparative perspective. 2021. https://rnao.ca/sites/default/files/2021-05/Nursing
                                                                      • 105. Registered Practical Nurses Association of Ontario. Wellness, workload and wages – how RPNs are personally coping amid the pandemic. 2020. https://www.werpn.com/advisory/publications/rpn-covid-study/
                                                                        • 106. Kruskal JB, Rosen MP, Hara AK, Canon CL, Wald C. Enabling your radiology business to thrive strategic lessons learned during the initial and subsequent surges of the covid-19 pandemic. Academic Radiology. 2021.
                                                                        • 107. Brambilla A, Sun T-Z, Elshazly W, Ghazy A, Barach P, Lindahl G, et al. Flexibility during the COVID-19 pandemic response: healthcare facility assessment tools for resilient evaluation. Int J Environ Res Public Health. 2021;18(21):11478. pmid:34769993
                                                                        • 108. Vera San Juan N, Clark SE, Camilleri M, Jeans JP, Monkhouse A, Chisnall G, et al. Training and redeployment of healthcare workers to intensive care units (ICUs) during the COVID-19 pandemic: a systematic review. BMJ Open. 2022;12(1):e050038. pmid:34996785
                                                                        • 109. Chemali S, Mari-Sáez A, El Bcheraoui C, Weishaar H. Health care workers’ experiences during the COVID-19 pandemic: a scoping review. Hum Resour Health. 2022;20(1):27. pmid:35331261
                                                                        • 110. McCabe R, Schmit N, Christen P, D’Aeth JC, Løchen A, Rizmie D, et al. Adapting hospital capacity to meet changing demands during the COVID-19 pandemic. BMC Med. 2020;18(1):329. pmid:33066777
                                                                        • 111. Public Health Agency of Canada. Key health inequalities in Canada: a national portrait – executive summary. Government of Canada; 2020. https://www.canada.ca/en/public-health/services/publications/science-research-data/key-health-inequalities-canada-national-portrait-executive-summary.html
                                                                          • 112. Jackson B, Huston P. Advancing health equity to improve health: the time is now. Health Promot Chronic Dis Prev Can. 2016;36(2):17–20. pmid:26878490
                                                                          • 113. Public Health Agency of Canada. Social determinants of health and health inequalities. 2023.