https://orcid.org/0000-0003-2349-7219
Figures
Abstract
Background
Adverse event (AE) detection is a major patient safety priority. However, despite extensive research on AEs, reported incidence rates vary widely.
Objective
This study aimed: (1) to synthesize available evidence on AE incidence in acute care inpatient settings using Trigger Tool methodology; and (2) to explore whether study characteristics and study quality explain variations in reported AE incidence.
Methods
To identify relevant studies, we queried PubMed, EMBASE, CINAHL, Cochrane Library and three journals in the patient safety field (last update search 25.05.2022). Eligible publications fulfilled the following criteria: adult inpatient samples; acute care hospital settings; Trigger Tool methodology; focus on specialty of internal medicine, surgery or oncology; published in English, French, German, Italian or Spanish. Systematic reviews and studies addressing adverse drug events or exclusively deceased patients were excluded. Risk of bias was assessed using an adapted version of the Quality Assessment Tool for Diagnostic Accuracy Studies 2. Our main outcome of interest was AEs per 100 admissions. We assessed nine study characteristics plus study quality as potential sources of variation using random regression models. We received no funding and did not register this review.
Results
Screening 6,685 publications yielded 54 eligible studies covering 194,470 admissions. The cumulative AE incidence was 30.0 per 100 admissions (95% CI 23.9–37.5; I2 = 99.7%) and between study heterogeneity was high with a prediction interval of 5.4–164.7. Overall studies’ risk of bias and applicability-related concerns were rated as low. Eight out of nine methodological study characteristics did explain some variation of reported AE rates, such as patient age and type of hospital. Also, study quality did explain variation.
Conclusion
Estimates of AE studies using trigger tool methodology vary while explaining variation is seriously hampered by the low standards of reporting such as the timeframe of AE detection. Specific reporting guidelines for studies using retrospective medical record review methodology are necessary to strengthen the current evidence base and to help explain between study variation.
Citation: Eggenschwiler LC, Rutjes AWS, Musy SN, Ausserhofer D, Nielen NM, Schwendimann R, et al. (2022) Variation in detected adverse events using trigger tools: A systematic review and meta-analysis. PLoS ONE 17(9): e0273800. https://doi.org/10.1371/journal.pone.0273800
Editor: Mojtaba Vaismoradi, Nord University, NORWAY
Received: December 24, 2021; Accepted: August 15, 2022; Published: September 1, 2022
Copyright: © 2022 Eggenschwiler et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data files are available from https://doi.org/10.5281/zenodo.4892518.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
For the last two decades, patient safety has become and remained a key issue for health care systems globally [1]. One major driver of patient harm in acute care hospitals are adverse events (AEs)—“unintended physical injury resulting from or contributed to by medical care that requires additional monitoring, treatment or hospitalization, or that results in death” [2]. Reported AE rates vary between 7% and 40% [3], increasing health care costs by roughly 10,000 Euros per index admission [4]. Considering that approximately 40% of admissions can be associated with AEs, it is likely that the consequences, both on health care service costs and on patient suffering, are underestimated [4, 5]. While some AEs are hardly avoidable, others are: studies have indicated that 6%–83% of AEs are deemed to be preventable [6, 7].
Retrospective medical record reviews are commonly used when collecting data about patient safety such as AEs. Medical record review methodology using available data [8], was found to identify more AEs when compared with other methods [9, 10], can be repeated over time and can target specific AE types, or the overall AE rate [11].
There are several medical record review methods, and the most used ones are the Harvard Medical Practice Study (HMPS) methodology [12], with subsequently modifications [13], and the Global Trigger Tool (GTT) [2]. The GTT, popularised by the Institute for Healthcare Improvement (IHI) in the US, was primarily designed as a measurement tool in clinical practice to estimate and track AE rates over time, extending beyond traditional incident reports, and aiming to measure the effect of safety interventions [14, 15]. The GTT includes a two-step medical record review process. In the first step, knowledgeable hospital staff—mainly nurses, conduct primary reviews to identify potential AEs using predefined triggers as outlined in the GTT guidance. In the second step, physicians verify the reviews from the first step and authenticate their consensus. A "trigger" (or clue) is either a specific term or an event in a medical record that could indicate the occurrence of an AE, e.g., readmissions within 30 days or pressure ulcers [2]. Its main methodological advantage is that it is an open, inductive process, sensitive to detect various types of AEs [2]. GTT based studies typically report inter-rater reliability coefficients that represent satisfactory reliability (kappa 0.34 to 0.89; mean: 0.65) [16].
GTT’s triggers are grouped into six modules (e.g., Care Module, Medication Module). Some researchers use all six of these [17, 18] while most use only those relevant to their setting [19, 20]. Yet others either create additional modules (e.g., Oncology Module [21, 22]) or develop modified versions tailored specifically to their patient and care settings [3, 23]. While former versions diverge too importantly from the original GTT to label it as GTT, they are still considered as trigger tools (TTs).
When using the GTT outside of the USA, even in cases where translation is unnecessary, triggers need to be adapted to reflect local norms (e.g., blood level limits). Additionally, medication labels need to be adjusted as appropriate [24, 25]. Although the GTT was developed as a manual method, with the rise of electronic health records, the GTT process can be semi or fully automated [26].
Recent systematic reviews focussing on AEs detected via GTT or TT showed high detection rate variability [3, 6, 26]. Some of this variability may reflect differences in the studies’ methodological features. Adaptations in triggers, review processes or patient record selection protocols might influence detection rates, thereby impacting the comparability of detected AEs. Such differences in medical record review methodology have not yet been systematically addressed. Therefore, this study has two aims: (1) to synthesize the evidence identified by the TT methodology regarding AE incidence in acute care inpatient settings; and (2) to explore whether between study variation in the incidence of AEs can be explained by study characteristics and study quality.
Methods
Design
This systematic review and meta-analyses adhered to the preferred reporting items for PRISMA guideline [27, 28].
Search strategy and information sources
Our search strategy was developed and validated using methods suggested by Hausner et al. [29, 30]. This involves generating a test set, developing and validating a search strategy and documenting the strategy using a standardized approach [30]. The medical subject headings (MeSH) and keywords for titles and abstracts in our search string were: (trigger[tiab] OR triggers[tiab]) AND (chart[tiab] OR charts[tiab] OR identif*[tiab] OR record[tiab] OR records[tiab]) AND (adverse[tiab] OR medical error[mh]). We used this to query four electronic databases: PubMed, EMBASE, CINAHL and Cochrane Library. In addition, we also hand-searched the top three journals publishing about GTT/TT (BMJ Quality & Safety; Journal of Patient Safety; International Journal for Quality in Health) and screened all authors’ personal libraries. In all searches, publication dates were unrestricted. The detailed search strategy used for this review and further explanations on chosen journals is published in Musy et al. [26]. The index search was conducted in November 2015, additional five update searches in April 2016, July 2017, January 2020, September 2020, and the latest update on May 25 2022.
Eligibility criteria
We included publications fulfilling six criteria:1. publication in English, French, German, Italian or Spanish; 2. adult inpatient samples; 3. acute care hospital settings; 4. medical record review performed manually via GTT or other TT methods; 5. specialties in internal medicine, surgery (including orthopaedics), oncology, or any combination of these (mixed); and 6. outcome data relevant to our study, e.g., number of detected AEs. Systematic reviews and studies addressing only adverse drug events or exclusively deceased patients were excluded.
Study selection and data extraction
Titles and abstracts were screened independently by two researchers in a first round if they included any information on GTT or TT and in a second round on the eligibility criteria. After screening the titles and abstracts, two researchers individually assessed the full-text articles for eligibility. To ensure high-quality data entry, data were extracted by one researcher and verified by a second. Information on study characteristics (e.g., number of admissions, setting, patient demographics) and patient outcomes (incidence, preventability) were collected into an online data collection instrument (airtable.com). Where studies of authors of this report were considered, a pair without direct involvement in the primary study was chosen to abstract and appraise the study. Differences between researchers were then discussed in the research group to reach consensus.
Our main outcome of interest was AEs per 100 admissions ((number of AEs / number of admissions) * 100). In addition, we included three secondary outcomes: AEs per 1,000 inpatient days ((number of AEs / number of inpatient days) * 1,000), the percentage of admissions with one or more AEs (number of admissions with ≥1 AE / number of admissions) and percentage of preventable AEs (number of preventable AEs / number of AEs). We included nine TT methodology characteristics in our statistical analysis to assess their potentially influence on AE detection rates. We categorized these under four headings: setting (type of hospital, type of specialty), patient characteristics (age, length of stay), design (AE definition, timeframe of AE detection, commission/ omission) and reviewer (training, experience). Definitions of our variables, our categorisations of the selected characteristics and our rationale for the chosen variable and its categorisation are available in Table 1.
Quality assessment
To assess the risk of bias and applicability-related concerns for each included study, we developed and piloted a quality assessment tool (QAT) (see S1 File). This was inspired by the Quality Assessment Tool for Diagnostic Accuracy Studies 2 (QUADAS-2) tool and by the QAT developed by Musy et al. [41]. While assessing our included studies, we used both QUADAS-2 tool dimensions: the risk of bias and applicability-related concerns [41]. We assessed five domains: 1) patient selection; 2) rater or reviewer; 3) trigger tool method; 4) outcomes; and 5) flow and timing. Following the QUADAS-2 structure each domain included standardised signalling questions to help researchers’ rate each of the two dimensions, i.e., risk of bias and applicability-related concerns. Possible dimension classifications were low, high, or unclear. For each study, a QAT was completed by one researcher and reviewed by a second. To reach consensus, differences were discussed between the two and, if necessary, within the research group.
Statistical analysis
To analyse and plot our results we used R version 4.1.3 on Linux [42] with the meta [43] and metafor [44] packages. We determined the number of AEs per 100 admissions and the number of AEs per 1,000 patient days from the reported data. If the number of AEs was not explicitly described, we calculated it from the reported estimate of AEs per 100 admissions and number of patient admissions. The number of patient days could for example be calculated from the total number of AEs per 1,000 patient days. For studies published by this study’s co-authors or in some cases by their research colleagues, when samples overlapped, we asked them for additional information in order to avoid double counting of admissions and AEs [34, 45, 46]. Pooled estimates for AEs per 100 admissions and AEs per 1,000 patient days were derived using a random effects Poisson regression approach within the R metarate function [43, 44]. With the R metaprop function, a random effects logistic regression model was used to obtain summary estimates and confidence intervals (derived by the Wilson method) for the outcomes expressed as percentage of admissions with ≥1 AE and percentage of preventable AEs [43].
Subgroup analysis.
Heterogeneity was explored by stratified analyses, which were performed on the main outcome measure, i.e. number of AEs per 100 admissions to evaluate the influence of the nine study characteristics: type of hospital, type of specialty, patient age, length of stay, AE definition, timeframe of AE detection, commission and omission, reviewer training, and reviewer experience. In addition, we analysed five elements relating to risk of bias and the three for applicability-related concerns. P-values were derived from the likelihood ratio test for model fit (p < 0.05 was considered significant). Furthermore, between study heterogeneity was evaluated visually and by calculating the prediction intervals [47, 48]. To assess the risk of publication bias related to small study size, we created a funnel plot regressing the logit of the AEs per 100 admissions on the standard error, assessed the symmetry of the distribution and performed the Egger test [49].
Results
The index search and update searches produced 9,780 returns. Deleting duplicates left 6,685 separate entries. The more detailed screening process left 54 studies, which were published in 72 publications [5, 9, 10, 14, 15, 17–22, 24, 34, 37–40, 45, 46, 50–102]. Fig 1 depicts the complete review procedure.
From [27] (GTT, Global Trigger Tool, TT, Trigger Tool).
Study characteristics
The 54 included studies were all published between 2009 and 2022. Their study periods ranged from one month to six years (Table 2). They were conducted in 26 countries, most of them in Europe (34 studies, 63%), followed by the US (12 studies, 22%) and Others (8 studies, 15%).
Sorted by continent; within continent alphabetically by country code, and within the country by year.
Four studies (7%) did not report their clinical specialties [10, 17, 71, 77]. For those remaining, almost half (24 studies, 44%) involved mixed specialties. One study included no information on the number of included records [40]. The numbers of included records ranged from 50 to 56,447. Overall, we included 194,470 index admissions in our report.
Table 3 illustrates AE rates’ key characteristics. In seven studies, we could not retrieve the main outcome measure AEs per 100 admissions [14, 24, 40, 55, 70, 80, 94]; for the remaining 47, rates ranged from 2.5 to 140 per 100 admissions. Per 1,000 patient days, the 36 (67%) studies with sufficient data yielded counts ranging from 12.4 to 139.6. And in the 48 studies whose data allowed us to calculate percentages of admissions with one or more AEs, these ranged from 7% to 69%. AE preventability percentages, which 37 studies (69%) reported, ranged from 7% to 93%; however, four of these studies provided no relevant raw data [21, 45, 55, 56].
Quality assessment
Our quality assessment results (Fig 2) indicate that most of the domains of the risk of bias are rated as low (range: 48%–93%). However, the “patient selection” and “reviewer” domains received respectively 15% and 13% high ratings—considerably more than the other domains (range: 2%–6%). In two domains, risk of bias was largely unclear: “reviewer” and “trigger tool method” received this rating respectively in 39% and 30% of cases.
Assessments are presented in risk of bias and applicability-related concerns. (TT method, Trigger Tool method).
Overall applicability-related concerns were predominantly low (range of domains: 65%–87%). High ratings were most prevalent (17%) in the “patient selection” domain; unclear ratings were most common (28%) for “reviewer”. Quality assessment results on study-level are provided in S1 Table.
Summary estimates from meta-analyses
The forest plot in Fig 3 presents AEs per 100 admissions by sample size. Forty-five samples from single countries contributed, as well as two multi-country (n = 10) samples [61, 71]. The summary estimate was 30.0 AEs per 100 admissions (95% CI 23.9–37.5). Visual inspection of the forest plot indicated a high level of between study heterogeneity, which was confirmed by an I2 of 99.7% (95% CI 99.7–99.7). The prediction interval ranged from 5.4 to 164.7 AEs per 100 admissions. Four studies had exceptionally high detection rates [19, 20, 38, 87]. At the opposite side, seven study samples reported fewer than ten AEs per 100 admissions [17, 56, 71].
Ordered by sample size [5, 10, 15, 17–22, 34, 37–39, 45, 46, 50–54, 56–69, 71–79, 82–91, 93, 95–102]. In Wilson et al. 2012, countries were not further specified. (AEs, Adverse events; * pooled estimate; • mean estimate; ‡ calculated total number of AEs).
S1–S3 Figs present additional forest plots for the three secondary outcomes, respectively AEs per 1,000 patient days (n = 36 studies), percentages of admissions with AEs (n = 48 studies), and percentages of preventable AEs (n = 33 studies). Our meta-analysis showed a summary estimate of 48.3 AEs per 1,000 patient days (95% CI 40.4–57.8) with high level of between study heterogeneity (prediction interval 15.9–147.0). Twenty-six percent of admissions experienced one or more AEs (95% CI 22.0–29.5, prediction interval 7.8–58.3). Within the studies that rated preventability, 62.6% of AEs were classified as preventable (95% CI 54.0–70.5, prediction interval 16.8–93.3). Similarly, visual inspection indicated a high between study heterogeneity. Funnel plot exploration did not suggest evidence for publication bias or other biases related to small study size (P from Egger test = 0.3, S4 Fig).
Effect of study characteristics.
Eight of nine analysed study characteristics explained part of the heterogeneity between studies (Fig 4).
(AE, adverse event; CI, confidence interval; GTT, Global Trigger Tool; IHI, Institute for Healthcare Improvement; N Studies, number of studies).
As for the type of hospital study characteristic, academic medical centres (n = 25, 45%) detected more AEs per 100 admissions than non-academic hospitals (respectively 47.1, 95% CI 36.6–60.5 and n = 6, 11%; 35.8, 95% CI 30.8–41.7), but as the summary estimate for mixed types of hospitals (n = 21, 38%; 17.0, 95% CI 11.7–24.8) is lower than either academic and non-academic hospitals, this association is likely confounded by a third feature. For type of clinical specialty, the significant differences within categories were driven by the not reported category (n = 11, 20%), which had fewer AEs per 100 admissions compared to the others (10.6, 95% CI 6.8–16.7). The internal medicine specialty (n = 7, 13%) had the highest number of AEs per 100 admissions (56.4, 95% CI 40.5–78.5), followed by surgery/orthopaedics (n = 11, 20%; 41.7, 95% CI 29.5–59.0). Oncology (n = 4, 7%) had numbers similar to those of the mixed designation (respectively 40.0, 95% CI 26.2–61.3 vs. 33.5, 95% CI 25.0–44.8).
Older patients (mean > 70 years; n = 8, 15%) had a higher incidence of AEs than younger ones (mean ≤ 70 years; n = 38, 69%), although only eight studies specifically investigated older patients (respectively 63.7, 95% CI 43.6–93.0 and 25.9, 95% CI 19.6–34.2). As occurred with the type of clinical specialty, for the category length of stay, the not reported category (n = 20, 36%) has a driving effect, with a mean of 16.7 AEs per 100 admissions (95% CI 11.6–23.9). Greater lengths of stay (mean >5 days; n = 24, 44%) had slightly higher AE rates than shorter ones (<5 days; n = 11, 20%) (respectively 42.9, 95% CI 32.7–56.4 and 40.8, 95% CI 29.0–57.3).
Almost all studies reported an IHI-like definition of AEs (n = 45, 82%). Of the five (9%) that did not report such a definition, AE rates were lower (respectively 29.0, 95% CI 22.4–37.5 and 22.6, 95% CI 13.9–36.8). The remaining five (9%) studies applying a wider than IHI AE definition reported clearly higher AE rates (55.3, 95% CI 42.1–72.7).
For the two characteristics, timeframe of AE detection and commission and omission the studies failed to report in 69% and 82% of the cases, seriously hampering the analyses. Studies that employed a pilot phase as part of the reviewer training (n = 14, 25%) might have had slightly higher detection rates than training only (respectively 36.8, 95% CI 26.3–51.5 and n = 31, 56%; 24.9, 95% CI 18.0–34.4). Reviewers with no experience in medical record review (n = 11, 20%) detected fewer AEs than those with experience (respectively 12.4, 95% CI 7.3–21.2) and n = 16, 29%; 40.9, 95% CI 30.6–54.4). Half of all studies did not report (n = 28, 51%) whether their reviewers had experience in medical record review. In those cases, the reported AE rates were comparable to those of experienced reviewers (35.8, 95% CI 27.5–46.5).
Effect of risk of bias.
Our quality assessment explained some of the variation regarding AE detection rates (S5 Fig). In eight studies (15%), patient selection was rated as high risk of bias because they included a slightly different patient population than defined in the inclusion criteria. These studies had higher rates of AEs than studies with a low risk of bias (respectively 61.2 vs. 32.5 AEs per 100 admissions). In studies where the risk of bias for the trigger tool methodology, the outcome category and the flow and timing were rated as high or unclear, considerably lower AE rates were detected than in those with a low risk of bias.
Similarly, regarding the trigger tool methodology’s applicability-related concerns, ratings of unclear correlated with lower AE rates than those of low (respectively 10.7 vs. 38.7 AEs per 100 admissions).
Discussion
The aim of this systematic review and meta-analysis was to synthesize AE detection rates with TT methodology and to explore variations in AE rates and assess the study quality in acute care inpatient settings. Reporting of study characteristics varied widely, and non-reporting of characteristics ranged from 5% to 82%. The summary estimate for AEs per 100 admissions was 30 (95% CI 23.9–37.5). An AE rate of 48 per 1,000 patient days, which translates into, 48 AEs in 200 patients with a length of stay of 5 days. Twenty-six percent of patients experience at least one AE related to their hospital stay and 63% out of all AEs were deemed preventable. Eight out of nine study characteristics explained variation in reported AE results. Studies conducted in academic medical centres, or with older populations reported higher AE rates than non-academic centres or younger adult populations. For several risk of bias categories (e.g., outcome, flow and timing), a higher risk of bias in a study indicated lower AE rates, which points to an underestimation of AE detection rates in low quality studies.
Analysing 17 studies in general inpatients, Hibbert et al. [3] reported AE rates of 8–51 per 100 admissions—a far smaller range than we detected (2.5–140). Our studies’ larger range of AEs could result from our larger study sample (n = 54). Further, their rates of admissions with AEs ranged from 7% to 40%, with a cluster of nine falling between 20% and 29% [3]. We found a wider range—7%–69%, but the average (26%) is close to Hibbert et al. [3].
Schwendimann et al.’s scoping review [32] of multicentre studies reported a median of 10% of admissions with AEs, which is less than half what we found. But this is congruent with Zanetti et al.’s integrative review, which reported between 5% and 11% [7]. Both of those reviews, especially Schwendimann et al.’s, concentrated solely on studies applying the HMPS methodology, not TT methodology [7, 32]. One possible reason for the lower rates could be that TT methodology requires the research team to include all identified AEs (if present, several AEs for one patient, not only the most severe, like in HMPS) [2, 12].
Interestingly, Panagioti et al.’s meta-analysis [6] found that half of their sample’s AEs were preventable whereas our meta-analysis indicated an overall preventability of 61%. For an academic hospital with 32,000 annual admissions, a preventable percentage of 61 would mean roughly 5,000 AEs could be prevented annually–given effective prevention strategies could be implemented. The confidence intervals reported by Panagioti and our 95% CI largely overlaps despite the difference in selection criteria for inclusion. They included every study that explored AEs’ preventability and many of those used the HMPS methodology, i.e., targeting more severe AEs [6].
Our meta-analysis explained part of the broad variation in AE detection via the selected study characteristics. One unanticipated finding was that, for many of these characteristics, essential details (e.g., length of stay) were not provided. For those, the not reported group had a dominant influence on AE detection rates. Although four study characteristics—type of specialty, length of stay, timeframe of AE detection, and commission and omission—showed differences in the subgroups, as the differences were driven by the not reported category, these only slightly explain the variation between AE detection rates. For all four characteristics, eight countries from which Wilson et al. [71] drew their samples fell within the not reported category, which might explain some of this result.
Compared to other categories, academic hospitals [34], higher patient age [75], and experienced reviewers [39] all corresponded with more AEs per 100 admissions. Supporting Sharek et al. [39] we found that experienced reviewers were less likely to miss AEs than unexperienced reviewers. These results support many published medical record review studies [23, 31–33]. Nevertheless, the findings need to be interpreted with some caution. Regarding type of specialty, the data on internal medicine and surgery including orthopaedic both involve wide confidence intervals (respectively 95% CI 40.5–78.5, and 95% CI 29.5–59.0); therefore, their higher numbers of AEs per 100 admissions (respectively 56.4 and 41.7) are to be questioned: numerous publications have found that surgical patients typically experience more AEs during their hospital stay than medical patients [6, 37, 103].
Addressing the overall quality of the included studies, we rated both their risk of bias and applicability-related concerns as low. This finding is supported by those of two earlier systematic reviews. First, Klein et al.’s [104] assessment of 24 of our 66 included publications indicated reasonable overall quality; second, also using a study sample that overlapped somewhat with ours, Panagioti et al. [6] rated all of the overlapping studies’ risk of bias as low.
Nevertheless, regarding adherence to TT methodology, including data completeness and usability, our meta-analysis clearly showed that our overall study sample’s reporting quality was inadequate. Our QAT explained part of the AE detection rate’s high variability: where risk of bias is rated as high or unclear for “outcome”, “trigger tool method” and “flow and timing”, AE rates are lower than where risk of bias is rated as low. This suggests that insufficient reporting resulted in lower estimates, i.e., the actual AEs per 100 admissions are likely higher than reported here.
Although patterns of publication bias in the field of single arm studies measuring the incidence of AEs are not well understood, we decided to perform a funnel plot analysis to evaluate any association between small study size and the magnitude of the estimates of AEs per 100 admissions. Whenever an uncontrolled study evaluates effects and safety of a therapeutic intervention, publication bias may still be expected, where higher estimates of AE may be less likely to be published. If this type of publication bias is associated with small study size, funnel plot exploration may detect it. The studies included in our review were more about health services and delivery research and we did not anticipate to find obvious signs of publication bias [105], which was eventually confirmed. The vast majority of studies did not report the occurrence of AEs per patient days. Rather than considering this as potential selective reporting bias, we reason that the field is insufficiently aware of the advantage of using person-time incidence rates over incidence proportions, where former facilitates comparison across studies.
Strengths and limitations
Our systematic review was based on an exhaustive search strategy so that it is unlikely we missed studies that would have changed our findings. Throughout the search we have included two studies that were not identified with our search strategy. Those were lacking on of the core components like “adverse” [40] or “record” [86]. We did not do a systematic search of “grey literature” which may lead to remaining studies not identified.
In absence of a suitable risk of bias tool for the type of studies included, we adapted an existing QAT to simultaneously address risk of bias and applicability-related concerns of the included studies. We conducted stratified analyses not only to evaluate effects of studies’ characteristics but also to evaluate effects of QAT domains. Our systematic review included a considerable high number of included studies when compared to previous reviews and resulted in a proportionately higher number of index admissions.
However, we also acknowledge further limitations. One was the exclusion of psychiatric, rehabilitation, emergency departments and intensive care settings. We set this criterion to maximize comparability across study settings. Similarly, by excluding studies focussed only on adverse drug events, we avoided skewing AE rates based on single-event results. Despite their benefits, both decisions reduced the final sample size.
Also, although we consider the identification and labelling of adverse events vital, we chose not to address either the types of AEs or their severity. Furthermore, we did not conduct an analysis of the influence of reported conflict of interest or funding in the included studies, which could further explain some of the variation. For the future, we also acknowledge that the registration of the review protocol on an open access repository is necessary.
Still, the most important limitation is that high levels of not reported information that hampered a full appreciation of the findings. The data did not allow to run multivariable models in a meaningful manner, so that all findings from univariable analyses need to be interpreted with caution, as we cannot exclude that some of the observed association, such as the effect of type of hospital, are confounded. For future studies on AEs via retrospective medical record review, irrespective of the detection methods used, the certainty of the evidence base would benefit from the standard use of a dedicated reporting guideline. Such a guideline is currently lacking for the type of studies included.
Conclusion
Based on our analyses of 54 studies using TT methodology, we found an overall incidence of 30.0 AEs per 100 admissions—affecting 26% of patients. Of these we estimated that 63% were preventable, indicating a high potential to improve patient safety. However, lack of reporting and high levels of statistical heterogeneity limit these estimates’ reliability.
Of nine TT study characteristics evaluated, our analyses indicate that eight explained part of the wide variation in AE incidence estimates. In four of those, most of the variation was driven by the not reported category (type of specialty, length of stay, timeframe of AE detection, commission and omission). For two characteristics (time frame of AE detection, commission and omission), studies even failed to report the methodological information in 69% and 82%.
To enhance comparability—and the reporting of TT studies clearly needs improvement—we recommend the development and implementation of a reporting checklist accompanied with a guidance document specifically for studies on the use of retrospective medical record review methods for AE detection.
Supporting information
S1 Table. Assessments of risk of bias and applicability-related concerns.
https://doi.org/10.1371/journal.pone.0273800.s003
(PDF)
S1 Fig. Forest plot of AEs per 1000 patient days.
* = pooled estimate, • = mean estimate, ‡ = calculated total number of AEs, ~ = calculated total number of patient days [5, 15, 17–22, 34, 37, 39, 40, 45, 46, 50–52, 54, 57, 58, 60, 62–65, 67, 68, 72, 73, 76–79, 82, 84–87, 89–91, 93, 95, 96, 98–100, 102].
https://doi.org/10.1371/journal.pone.0273800.s004
(TIF)
S2 Fig. Forest plot percentage of admissions with at least one adverse event (AE).
CI, confidence interval; * = pooled estimate, • = mean estimate, + = calculated total number of admissions with ≥ 1 AE [5, 9, 14, 15, 17–22, 24, 34, 37, 39, 45, 46, 50–58, 60–68, 70, 72–87, 89–94, 96–101].
https://doi.org/10.1371/journal.pone.0273800.s005
(TIF)
S3 Fig. Forest plot percentage of preventable adverse events (AEs).
CI, confidence interval; * = pooled estimate, • = mean estimate, ¢ = calculated number of preventable AEs [15, 17–20, 34, 37–39, 46, 50, 51, 53, 59, 63–67, 71–75, 77, 78, 87, 89–91, 96–98, 100, 101].
https://doi.org/10.1371/journal.pone.0273800.s006
(TIF)
S4 Fig. Funnel plot for AEs per 100 admissions [5, 10, 15, 17–22, 34, 37–39, 45, 46, 50–54, 56–69, 71–79, 82–91, 93, 95–102].
https://doi.org/10.1371/journal.pone.0273800.s007
(TIF)
S5 Fig. Forest plot with stratified analysis of the risk of bias and applicability-related concerns.
AE, adverse events; N studies, number of studies; CI, confidence interval [5, 10, 15, 17–22, 34, 37–39, 45, 46, 50–54, 56–69, 71–79, 82–91, 93, 95–102].
https://doi.org/10.1371/journal.pone.0273800.s008
(TIF)
References
- 1.
Institute of Medicine. To Err is Human: Building a Safer Health System. Kohn LT, Corrigan JM, Donaldson MS, editors. Washington (DC): National Academies Press; 2000.
- 2.
Griffin F, Resar R. IHI Global Trigger Tool for Measuring Adverse Events (Second edition). IHI Innovation Series white paper. Cambridge, Massachusetts: Institute for Healthcare Improvement; 2009.
- 3. Hibbert PD, Molloy CJ, Hooper TD, Wiles LK, Runciman WB, Lachman P, et al. The application of the Global Trigger Tool: a systematic review. Int J Qual Health Care. 2016;28(6):640–9. Epub 2016/09/25. pmid:27664822
- 4. Kjellberg J, Wolf RT, Kruse M, Rasmussen SR, Vestergaard J, Nielsen KJ, et al. Costs associated with adverse events among acute patients. BMC Health Serv Res. 2017;17(1):651. Epub 2017/09/15. pmid:28903748
- 5. Adler L, Yi D, Li M, McBroom B, Hauck L, Sammer C, et al. Impact of Inpatient Harms on Hospital Finances and Patient Clinical Outcomes. J Patient Saf. 2018;14(2):67–73. Epub 2015/03/25. pmid:25803176
- 6. Panagioti M, Khan K, Keers RN, Abuzour A, Phipps D, Kontopantelis E, et al. Prevalence, severity, and nature of preventable patient harm across medical care settings: systematic review and meta-analysis. BMJ. 2019;366:l4185. Epub 2019/07/19. pmid:31315828
- 7. Zanetti ACB, Gabriel CS, Dias BM, Bernardes A, Moura AA, Gabriel AB, et al. Assessment of the incidence and preventability of adverse events in hospitals: an integrative review. Rev Gaucha Enferm. 2020;41:e20190364. Epub 2020/07/16. pmid:32667424
- 8. Thomas EJ, Petersen LA. Measuring errors and adverse events in health care. J Gen Intern Med. 2003;18(1):61–7. Epub 2003/01/22. pmid:12534766
- 9. Naessens JM, Campbell CR, Huddleston JM, Berg BP, Lefante JJ, Williams AR, et al. A comparison of hospital adverse events identified by three widely used detection methods. Int J Qual Health Care. 2009;21(4):301–7. Epub 2009/07/21. pmid:19617381
- 10. Classen DC, Resar R, Griffin F, Federico F, Frankel T, Kimmel N, et al. ’Global trigger tool’ shows that adverse events in hospitals may be ten times greater than previously measured. Health Aff (Millwood). 2011;30(4):581–9. Epub 2011/04/08. pmid:21471476
- 11.
Vincent C. Patient Safety: John Wiley & Sons, Ltd; 2010.
- 12. Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med. 1991;324(6):370–6. Epub 1991/02/07. pmid:1987460
- 13. Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD. The Quality in Australian Health Care Study. Med J Aust. 1995;163(9):458–71. Epub 1995/11/06. pmid:7476634
- 14. Naessens JM, O’Byrne TJ, Johnson MG, Vansuch MB, McGlone CM, Huddleston JM. Measuring hospital adverse events: assessing inter-rater reliability and trigger performance of the Global Trigger Tool. Int J Qual Health Care. 2010;22(4):266–74. Epub 2010/06/11. pmid:20534607
- 15. Good VS, Saldana M, Gilder R, Nicewander D, Kennerly DA. Large-scale deployment of the Global Trigger Tool across a large hospital system: refinements for the characterisation of adverse events to support patient safety learning opportunities. BMJ Qual Saf. 2011;20(1):25–30. Epub 2011/01/14. pmid:21228072
- 16. Hanskamp-Sebregts M, Zegers M, Vincent C, van Gurp PJ, de Vet HC, Wollersheim H. Measurement of patient safety: a systematic review of the reliability and validity of adverse event detection with record review. BMJ Open. 2016;6(8):e011078. Epub 2016/08/24. pmid:27550650
- 17. Hwang JI, Chin HJ, Chang YS. Characteristics associated with the occurrence of adverse events: a retrospective medical record review using the Global Trigger Tool in a fully digitalized tertiary teaching hospital in Korea. J Eval Clin Pract. 2014;20(1):27–35. Epub 2013/07/31. pmid:23890097
- 18. Kurutkan MN, Usta E, Orhan F, Simsekler MC. Application of the IHI Global Trigger Tool in measuring the adverse event rate in a Turkish healthcare setting. Int J Risk Saf Med. 2015;27(1):11–21. Epub 2015/03/15. pmid:25766063
- 19. Grossmann N, Gratwohl F, Musy SN, Nielen NM, Simon M, Donze J. Describing adverse events in medical inpatients using the Global Trigger Tool. Swiss Med Wkly. 2019;149:w20149. Epub 2019/11/11. pmid:31707720
- 20. Hommel A, Magneli M, Samuelsson B, Schildmeijer K, Sjostrand D, Goransson KE, et al. Exploring the incidence and nature of nursing-sensitive orthopaedic adverse events: A multicenter cohort study using Global Trigger Tool. Int J Nurs Stud. 2020;102:103473. Epub 2019/12/07. pmid:31810021
- 21. Gerber A, Da Silva Lopes A, Szüts N, Simon M, Ribordy-Baudat V, Ebneter A, et al. Describing adverse events in Swiss hospitalized oncology patients using the Global Trigger Tool. Health Sci Rep. 2020;3(2):e160. Epub 2020/05/15. pmid:32405540
- 22. Mattsson TO, Knudsen JL, Brixen K, Herrstedt J. Does adding an appended oncology module to the Global Trigger Tool increase its value? Int J Qual Health Care. 2014;26(5):553–60. Epub 2014/08/01. pmid:25080549
- 23. Unbeck M, Lindemalm S, Nydert P, Ygge BM, Nylen U, Berglund C, et al. Validation of triggers and development of a pediatric trigger tool to identify adverse events. BMC Health Serv Res. 2014;14:655. Epub 2014/12/22. pmid:25527905
- 24. Deilkas ET, Bukholm G, Lindstrom JC, Haugen M. Monitoring adverse events in Norwegian hospitals from 2010 to 2013. BMJ Open. 2015;5(12):e008576. Epub 2016/01/01. pmid:26719311
- 25.
Institute for Healthcare Improvement. Aktives Messinstrument der Patientensicherheit–das IHI Global Trigger Tool: Projekt-Version. Cambridge: Institute for Healthcare Improvement (IHI); 2009.
- 26. Musy SN, Ausserhofer D, Schwendimann R, Rothen HU, Jeitziner MM, Rutjes AW, et al. Trigger Tool-Based Automated Adverse Event Detection in Electronic Health Records: Systematic Review. J Med Internet Res. 2018;20(5):e198. Epub 2018/06/01. pmid:29848467
- 27. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. Epub 2021/03/31. pmid:33782057
- 28. Page MJ, Moher D, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ. 2021;372:n160. Epub 2021/03/31. pmid:33781993
- 29. Hausner E, Guddat C, Hermanns T, Lampert U, Waffenschmidt S. Development of search strategies for systematic reviews: validation showed the noninferiority of the objective approach. J Clin Epidemiol. 2015;68(2):191–9. Epub 2014/12/04. pmid:25464826
- 30. Hausner E, Waffenschmidt S, Kaiser T, Simon M. Routine development of objectively derived search strategies. Syst Rev. 2012;1:19. Epub 2012/05/17. pmid:22587829
- 31. Zegers M, de Bruijne MC, Wagner C, Hoonhout LH, Waaijman R, Smits M, et al. Adverse events and potentially preventable deaths in Dutch hospitals: results of a retrospective patient record review study. Qual Saf Health Care. 2009;18(4):297–302. Epub 2009/08/05. pmid:19651935
- 32. Schwendimann R, Blatter C, Dhaini S, Simon M, Ausserhofer D. The occurrence, types, consequences and preventability of in-hospital adverse events—a scoping review. BMC Health Serv Res. 2018;18(1):521. Epub 2018/07/06. pmid:29973258
- 33. Unbeck M. Evaluation of retrospective patient record review as a method to identify patient safety and quality information in orthopaedic care 2012. Available from: https://openarchive.ki.se/xmlui/handle/10616/40941.
- 34. Rutberg H, Borgstedt-Risberg M, Gustafson P, Unbeck M. Adverse events in orthopedic care identified via the Global Trigger Tool in Sweden—implications on preventable prolonged hospitalizations. Patient Saf Surg. 2016;10:23. Epub 2016/11/02. pmid:27800019
- 35.
OECD. Length of hospital stay (indicator) 2021 [cited 2021 03.01.]. Available from: https://data.oecd.org/healthcare/length-of-hospital-stay.htm.
- 36. Kable AK, Gibberd RW, Spigelman AD. Adverse events in surgical patients in Australia. Int J Qual Health Care. 2002;14(4):269–76. Epub 2002/08/31. pmid:12201185
- 37. Unbeck M, Schildmeijer K, Henriksson P, Jurgensen U, Muren O, Nilsson L, et al. Is detection of adverse events affected by record review methodology? an evaluation of the "Harvard Medical Practice Study" method and the "Global Trigger Tool". Patient Saf Surg. 2013;7(1):10. Epub 2013/04/17. pmid:23587448
- 38. Croft LD, Liquori ME, Ladd J, Day HR, Pineles L, Lamos EM, et al. Frequency of Adverse Events Before, During, and After Hospital Admission. South Med J. 2016;109(10):631–5. Epub 2016/10/06. pmid:27706501
- 39. Sharek PJ, Parry G, Goldmann D, Bones K, Hackbarth A, Resar R, et al. Performance characteristics of a methodology to quantify adverse events over time in hospitalized patients. Health Serv Res. 2011;46(2):654–78. Epub 2010/08/21. pmid:20722749
- 40. von Plessen C, Kodal AM, Anhoj J. Experiences with global trigger tool reviews in five Danish hospitals: an implementation study. BMJ Open. 2012;2(5). Epub 2012/10/16. pmid:23065451
- 41. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529–36. Epub 2011/10/19. pmid:22007046
- 42.
R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2022. Available from: https://www.R-project.org/.
- 43. Balduzzi S, Rucker G, Schwarzer G. How to perform a meta-analysis with R: a practical tutorial. Evid Based Ment Health. 2019;22(4):153–60. Epub 2019/09/30. pmid:31563865
- 44. Viechtbauer W. Conducting Meta-Analyses inRwith themetaforPackage. Journal of Statistical Software. 2010;36(3):1–48.
- 45. Nilsson L, Borgstedt-Risberg M, Soop M, Nylen U, Alenius C, Rutberg H. Incidence of adverse events in Sweden during 2013–2016: a cohort study describing the implementation of a national trigger tool. BMJ Open. 2018;8(3):e020833. Epub 2018/04/01. pmid:29602858
- 46. Nilsson L, Risberg MB, Montgomery A, Sjodahl R, Schildmeijer K, Rutberg H. Preventable Adverse Events in Surgical Care in Sweden: A Nationwide Review of Patient Notes. Medicine (Baltimore). 2016;95(11):e3047. Epub 2016/03/18. pmid:26986126
- 47. Higgins JP, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects meta-analysis. J R Stat Soc Ser A Stat Soc. 2009;172(1):137–59. Epub 2009/04/22. pmid:19381330
- 48. IntHout J, Ioannidis JP, Rovers MM, Goeman JJ. Plea for routinely presenting prediction intervals in meta-analysis. BMJ Open. 2016;6(7):e010247. Epub 2016/07/14. pmid:27406637
- 49. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629–34. Epub 1997/10/06. pmid:9310563
- 50. Kennerly DA, Kudyakov R, da Graca B, Saldana M, Compton J, Nicewander D, et al. Characterization of adverse events detected in a large health care delivery system using an enhanced global trigger tool over a five-year interval. Health Serv Res. 2014;49(5):1407–25. Epub 2014/03/19. pmid:24628436
- 51. Asavaroengchai S, Sriratanaban J, Hiransuthikul N, Supachutikul A. Identifying adverse events in hospitalized patients using global trigger tool in Thailand. Asian Biomedicine. 2009;3(5):545–50.
- 52. Bjorn B, Anhoj J, Ostergaard M, Kodal AM, von Plessen C. Test-Retest Reliability of an Experienced Global Trigger Tool Review Team. J Patient Saf. 2017. Epub 2017/10/13. pmid:29023303
- 53. Perez Zapata AI, Gutierrez Samaniego M, Rodriguez Cuellar E, Gomez de la Camara A, Ruiz Lopez P. [Comparison of the "Trigger" tool with the minimum basic data set for detecting adverse events in general surgery]. Rev Calid Asist. 2017;32(4):209–14. Epub 2017/03/21. pmid:28314619
- 54. Haukland EC, von Plessen C, Nieder C, Vonen B. Adverse events in hospitalised cancer patients: a comparison to a general hospital population. Acta Oncol. 2017;56(9):1218–23. Epub 2017/04/06. pmid:28379721
- 55. Lipitz-Snyderman A, Classen D, Pfister D, Killen A, Atoria CL, Fortier E, et al. Performance of a Trigger Tool for Identifying Adverse Events in Oncology. J Oncol Pract. 2017;13(3):e223–e30. Epub 2017/01/18. pmid:28095173
- 56. Mayor S, Baines E, Vincent C, Lankshear A, Edwards A, Aylward M, et al. Measuring harm and informing quality improvement in the Welsh NHS: the longitudinal Welsh national adverse events study. Health Serv Deliv Res. 2017;5(9). pmid:28252896
- 57. Mevik K, Griffin FA, Hansen TE, Deilkas ET, Vonen B. Is inter-rater reliability of Global Trigger Tool results altered when members of the review team are replaced? Int J Qual Health Care. 2016;28(4):492–6. Epub 2016/06/11. pmid:27283442
- 58. Mevik K, Griffin FA, Hansen TE, Deilkas ET, Vonen B. Does increasing the size of bi-weekly samples of records influence results when using the Global Trigger Tool? An observational study of retrospective record reviews of two different sample sizes. BMJ Open. 2016;6(4):e010700. Epub 2016/04/27. pmid:27113238
- 59. Croft LD, Liquori M, Ladd J, Day H, Pineles L, Lamos E, et al. The Effect of Contact Precautions on Frequency of Hospital Adverse Events. Infect Control Hosp Epidemiol. 2015;36(11):1268–74. Epub 2015/08/19. pmid:26278419
- 60. Mortaro A, Moretti F, Pascu D, Tessari L, Tardivo S, Pancheri S, et al. Adverse Events Detection Through Global Trigger Tool Methodology: Results From a 5-Year Study in an Italian Hospital and Opportunities to Improve Interrater Reliability. J Patient Saf. 2017. Epub 2017/06/10. pmid:28598897
- 61. Deilkas ET, Risberg MB, Haugen M, Lindstrom JC, Nylen U, Rutberg H, et al. Exploring similarities and differences in hospital adverse event rates between Norway and Sweden using Global Trigger Tool. BMJ Open. 2017;7(3):e012492. Epub 2017/03/23. pmid:28320786
- 62. Xu XD, Yuan YJ, Zhao LM, Li Y, Zhang HZ, Wu H. Adverse Events at Baseline in a Chinese General Hospital: A Pilot Study of the Global Trigger Tool. J Patient Saf. 2020;16(4):269–73. Epub 2016/09/10. pmid:27611772
- 63. Suarez C, Menendez MD, Alonso J, Castano N, Alonso M, Vazquez F. Detection of adverse events in an acute geriatric hospital over a 6-year period using the Global Trigger Tool. J Am Geriatr Soc. 2014;62(5):896–900. Epub 2014/04/05. pmid:24697662
- 64. Guzman Ruiz O, Perez Lazaro JJ, Ruiz Lopez P. [Performance and optimisation of a trigger tool for the detection of adverse events in hospitalised adult patients]. Gac Sanit. 2017;31(6):453–8. Epub 2017/05/27. pmid:28545741
- 65. Müller MM, Gous AG, Schellack N. Measuring adverse events using a trigger tool in a paper based patient information system at a teaching hospital in South Africa. Eur J Clin Pharm. 2016;18(2):103–12.
- 66. Pérez Zapata AI, Gutiérrez Samaniego M, Rodríguez Cuéllar E, Andrés Esteban EM, Gómez de la Cámara A, Ruiz López P. Detection of Adverse Events in General Surgery Using the “Trigger Tool” Methodology. Cirugía Española (English Edition). 2015;93(2):84–90. pmid:25443150
- 67. Guzman-Ruiz O, Ruiz-Lopez P, Gomez-Camara A, Ramirez-Martin M. [Detection of adverse events in hospitalized adult patients by using the Global Trigger Tool method]. Rev Calid Asist. 2015;30(4):166–74. Epub 2015/05/31. pmid:26025386
- 68. Mattsson TO, Knudsen JL, Lauritsen J, Brixen K, Herrstedt J. Assessment of the global trigger tool to measure, monitor and evaluate patient safety in cancer patients: reliability concerns are raised. BMJ Qual Saf. 2013;22(7):571–9. Epub 2013/03/01. pmid:23447657
- 69. Lipczak H, Knudsen JL, Nissen A. Safety hazards in cancer care: findings using three different methods. BMJ Qual Saf. 2011;20(12):1052–6. Epub 2011/06/30. pmid:21712371
- 70. Cihangir S, Borghans I, Hekkert K, Muller H, Westert G, Kool RB. A pilot study on record reviewing with a priori patient selection. BMJ Open. 2013;3(7). Epub 2013/07/23. pmid:23872292
- 71. Wilson RM, Michel P, Olsen S, Gibberd RW, Vincent C, El-Assady R, et al. Patient safety in developing countries: retrospective estimation of scale and nature of harm to patients in hospital. BMJ. 2012;344:e832. Epub 2012/03/15. pmid:22416061
- 72. Schildmeijer K, Nilsson L, Arestedt K, Perk J. Assessment of adverse events in medical care: lack of consistency between experienced teams using the global trigger tool. BMJ Qual Saf. 2012;21(4):307–14. Epub 2012/03/01. pmid:22362917
- 73. Rutberg H, Borgstedt Risberg M, Sjodahl R, Nordqvist P, Valter L, Nilsson L. Characterisations of adverse events detected in a university hospital: a 4-year study using the Global Trigger Tool method. BMJ Open. 2014;4(5):e004879. Epub 2014/05/30. pmid:24871538
- 74. O’Leary KJ, Devisetty VK, Patel AR, Malkenson D, Sama P, Thompson WK, et al. Comparison of traditional trigger tool to data warehouse based screening for identifying hospital adverse events. BMJ Qual Saf. 2013;22(2):130–8. Epub 2012/10/06. pmid:23038408
- 75. Najjar S, Hamdan M, Euwema MC, Vleugels A, Sermeus W, Massoud R, et al. The Global Trigger Tool shows that one out of seven patients suffers harm in Palestinian hospitals: challenges for launching a strategic safety plan. Int J Qual Health Care. 2013;25(6):640–7. Epub 2013/10/22. pmid:24141012
- 76. Mull HJ, Brennan CW, Folkes T, Hermos J, Chan J, Rosen AK, et al. Identifying Previously Undetected Harm: Piloting the Institute for Healthcare Improvement’s Global Trigger Tool in the Veterans Health Administration. Qual Manag Health Care. 2015;24(3):140–6. Epub 2015/06/27. pmid:26115062
- 77. Landrigan CP, Parry GJ, Bones CB, Hackbarth AD, Goldmann DA, Sharek PJ. Temporal trends in rates of patient harm resulting from medical care. N Engl J Med. 2010;363(22):2124–34. Epub 2010/11/26. pmid:21105794
- 78. Kennerly DA, Saldana M, Kudyakov R, da Graca B, Nicewander D, Compton J. Description and evaluation of adaptations to the global trigger tool to enhance value to adverse event reduction efforts. J Patient Saf. 2013;9(2):87–95. Epub 2013/01/22. pmid:23334632
- 79. Garrett PR Jr., Sammer C, Nelson A, Paisley KA, Jones C, Shapiro E, et al. Developing and implementing a standardized process for global trigger tool application across a large health system. Jt Comm J Qual Patient Saf. 2013;39(7):292–7. Epub 2013/07/31. pmid:23888638
- 80. Farup PG. Are measurements of patient safety culture and adverse events valid and reliable? Results from a cross sectional study. BMC Health Serv Res. 2015;15:186. Epub 2015/05/03. pmid:25934272
- 81. Bjertnaes O, Deilkas ET, Skudal KE, Iversen HH, Bjerkan AM. The association between patient-reported incidents in hospitals and estimated rates of patient harm. Int J Qual Health Care. 2015;27(1):26–30. Epub 2014/11/25. pmid:25417226
- 82. Brosterhaus M, Hammer A, Kalina S, Grau S, Roeth AA, Ashmawy H, et al. Applying the Global Trigger Tool in German Hospitals: A Pilot in Surgery and Neurosurgery. J Patient Saf. 2020;16(4):e340–e51. Epub 2020/11/21. pmid:33215895
- 83. Griffin FA, Classen DC. Detection of adverse events in surgical patients using the Trigger Tool approach. Qual Saf Health Care. 2008;17(4):253–8. Epub 2008/08/06. pmid:18678721
- 84. Gunningberg L, Sving E, Hommel A, Alenius C, Wiger P, Baath C. Tracking pressure injuries as adverse events: National use of the Global Trigger Tool over a 4-year period. J Eval Clin Pract. 2019;25(1):21–7. Epub 2018/07/22. pmid:30027549
- 85. Haukland EC, Mevik K, von Plessen C, Nieder C, Vonen B. Contribution of adverse events to death of hospitalised patients. BMJ Open Qual. 2019;8(1):e000377. Epub 2019/04/19. pmid:30997413
- 86. Hoffmann-Volkl G, Kastenbauer T, Muck U, Zottl M, Huf W, Ettl B. [Detection of adverse events using IHI Global Trigger Tool during the adoption of a risk management system: A retrospective study over three years at a department for cardiovascular surgery in Vienna]. Z Evid Fortbild Qual Gesundhwes. 2018;131–132:38–45. Epub 2017/11/07. pmid:29103832
- 87. Hu Q, Wu B, Zhan M, Jia W, Huang Y, Xu T. Adverse events identified by the global trigger tool at a university hospital: A retrospective medical record review. J Evid Based Med. 2019;12(2):91–7. Epub 2018/12/05. pmid:30511516
- 88. Lipczak H, Neckelmann K, Steding-Jessen M, Jakobsen E, Knudsen JL. Uncertain added value of Global Trigger Tool for monitoring of patient safety in cancer care. Dan Med Bull. 2011;58(11):A4337. Epub 2011/11/04. pmid:22047933
- 89. Magneli M, Unbeck M, Rogmark C, Rolfson O, Hommel A, Samuelsson B, et al. Validation of adverse events after hip arthroplasty: a Swedish multi-centre cohort study. BMJ Open. 2019;9(3):e023773. Epub 2019/03/10. pmid:30850403
- 90. Magneli M, Unbeck M, Samuelsson B, Rogmark C, Rolfson O, Gordon M, et al. Only 8% of major preventable adverse events after hip arthroplasty are filed as claims: a Swedish multi-center cohort study on 1,998 patients. Acta Orthop. 2020;91(1):20–5. Epub 2019/10/17. pmid:31615309
- 91. Menendez Fraga MD, Cueva Alvarez MA, Franco Castellanos MR, Fernandez Moral V, Castro Del Rio MP, Arias Perez JI, et al. [Compliance with the surgical safety checklist and surgical events detected by the Global Trigger Tool]. Rev Calid Asist. 2016;31 Suppl 1:20–3. Epub 2016/06/07. pmid:27265381
- 92. Mevik K, Hansen TE, Deilkas EC, Ringdal AM, Vonen B. Is a modified Global Trigger Tool method using automatic trigger identification valid when measuring adverse events? Int J Qual Health Care. 2019;31(7):535–40. Epub 2018/10/09. pmid:30295829
- 93. Sekijima A, Sunga C, Bann M. Adverse events experienced by patients hospitalized without definite medical acuity: A retrospective cohort study. J Gen Intern Med. 2020;34(2):S125. pmid:31251157
- 94. Toribio-Vicente MJ, Chalco-Orrego JP, Diaz-Redondo A, Llorente-Parrado C, Pla-Mestre R. [Detection of adverse events using trigger tools in 2hospital units in Spain]. J Healthc Qual Res. 2018;33(4):199–205. Epub 2018/01/01. pmid:31610975
- 95. Zadvinskis IM, Salsberry PJ, Chipps EM, Patterson ES, Szalacha LA, Crea KA. An Exploration of Contributing Factors to Patient Safety. J Nurs Care Qual. 2018;33(2):108–15. Epub 2018/02/22. pmid:29466259
- 96. Kelly-Pettersson P, Sköldenberg O, Samuelsson B, Stark A, Muren O, Unbeck M. The identification of adverse events in hip fracture patients using the Global Trigger Tool: A prospective observational cohort study. Int J Orthop Trauma Nurs. 2020;38:100779. Epub 2020/05/23. pmid:32439319
- 97. Kaibel Val R, Ruiz López P, Pérez Zapata AI, Gómez de la Cámara A, de la Cruz Vigo F. [Detection of adverse events in thyroid and parathyroid surgery using trigger tool and Minimum Basic Data Set (MBDS)]. J Healthc Qual Res. 2020;35(6):348–54. Epub 2020/10/30. pmid:33115613
- 98. Menéndez-Fraga MD, Alonso J, Cimadevilla B, Cueto B, Vazquez F. Does Skilled Nursing Facility Trigger Tool used with Global Trigger Tool increase its value for adverse events evaluation? J Healthc Qual Res. 2021;36(2):75–80. Epub 2021/01/30. pmid:33509727
- 99. Moraes SM, Ferrari TCA, Figueiredo NMP, Almeida TNC, Sampaio CCL, Andrade YCP, et al. Assessment of the reliability of the IHI Global Trigger Tool: new perspectives from a Brazilian study. Int J Qual Health Care. 2021;33(1). Epub 2021/03/07. pmid:33676370
- 100. Nowak B, Schwendimann R, Lyrer P, Bonati LH, De Marchis GM, Peters N, et al. Occurrence of No-Harm Incidents and Adverse Events in Hospitalized Patients with Ischemic Stroke or TIA: A Cohort Study Using Trigger Tool Methodology. Int J Environ Res Public Health. 2022;19(5). Epub 2022/03/11. pmid:35270487
- 101. Pérez Zapata AI, Rodríguez Cuéllar E, de la Fuente Bartolomé M, Martín-Arriscado Arroba C, García Morales MT, Loinaz Segurola C, et al. Predictive Power of the "Trigger Tool" for the detection of adverse events in general surgery: a multicenter observational validation study. Patient Saf Surg. 2022;16(1):7. Epub 2022/02/10. pmid:35135570
- 102. Pierdevara L, Porcel-Gálvez AM, Ferreira da Silva AM, Barrientos Trigo S, Eiras M. Translation, Cross-Cultural Adaptation, and Measurement Properties of the Portuguese Version of the Global Trigger Tool for Adverse Events. Ther Clin Risk Manag. 2020;16:1175–83. Epub 2020/12/11. pmid:33299318
- 103. de Vries EN, Ramrattan MA, Smorenburg SM, Gouma DJ, Boermeester MA. The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care. 2008;17(3):216–23. Epub 2008/06/04. pmid:18519629
- 104. Klein DO, Rennenberg R, Koopmans RP, Prins MH. The ability of triggers to retrospectively predict potentially preventable adverse events in a sample of deceased patients. Prev Med Rep. 2017;8:250–5. Epub 2017/11/29. pmid:29181297
- 105. Munn Z, Moola S, Lisy K, Riitano D, Tufanaru C. Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data. Int J Evid Based Healthc. 2015;13(3):147–53. Epub 2015/09/01. pmid:26317388