Introduction

In primary care, trigger tools have been used to evaluate and identify patient safety events [1, 2]. It is a method of conducting a structured review of a patient’s electronic medical record in order to identify specific indicators associated with adverse events and patient harm. The Institute for Healthcare Improvement (IHI) has developed the most well-known indicators or triggers [1]. This tool has been described as extremely useful for detecting any adverse events, whether preventable or not, in outpatient and primary care settings [3]. Detected preventable AEs using trigger tools may be problematic, but they were critical opportunities for many safety interventions and were easily accepted by providers due to their intuitive appeal as quality improvement targets [4]. The trigger tool has steadily assisted in the identification of adverse drug reactions, surgical complications, and other potentially avoidable harm [5]. Furthermore, it can be used for a variety of purposes, such as providing a practice alert in order to improve the quality and safety of care. Trigger tools can be used not only to detect errors or adverse events, but also to identify the progression of chronic disease from a patient’s EMR. It serves as a ‘backstop’ for clinicians viewing results and encourages reflection if clinical review or referral is indicated [6]. The trigger tools encourage clinicians to conduct a chart review and examination.

The use of trigger tools is advantageous to both clinicians and patients. It assists clinicians by reducing the number of work-hours required to review processes in detecting each adverse event from a patient’s electronic medical record in primary care [7]. Trigger tools enable physicians to take proactive precautions against patient safety events based on triggers detected in the patient’s electronic medical record [8]. Trigger tools benefit patients by preventing harmful events, eliminating complications in medication-related errors, and achieving a safe higher level of care [4].

The trigger tool development process begins with the development the triggers and the scanning of patient electronic health records (EHR) data for clinical and diagnostic clues in order to identify hazards or risks, which are then validated or evaluated [8, 9]. An expert panel evaluated the appropriateness of the set of triggers based on hospital settings after conducting a detailed literature review and study design [10]. An algorithm were also developed due to the automated identification process of information within an EHR that signal the potential error or an adverse event through the pilot hospital’s health information technology system [5].

The trigger tools must be validated or evaluated after the development process is completed. The validation process was carried out by developing the definition of each trigger and selecting random medical charts that had been identified as positive to be evaluated by experts for the presence of triggers [11]. The validation process includes a double review by physicians or other experts to assess the sensitivity and specificity of the triggers used [12]. There have only been a few previous studies on developed trigger tools for primary care settings [9].

The studies on the development and validation of trigger tools discussed above show that the majority of them are conducted in hospitals [5, 8, 10–12] rather than primary care [9]. It has been demonstrated that there is still a dearth of recent and focused studies on trigger development and validation in primary care settings. A specific trigger tool development and validation in the primary care setting has yet to be investigated. As a result, the purpose of this study is to investigate the trigger development and validation process in primary care settings.

Material and methods

Eligibility criteria and data selection

Studies that clearly describe the process development and validation of trigger tool in primary care or combination between primary and secondary care were considered eligible Furthermore, studies were included if the process development and validation of the trigger tool had an objectively defined outcome. However, due to the scarcity of research on this subject, a total of 196 articles were retrieved, with 6 duplications automatically removed using Mendeley Reference Manager. There were 190 records left after duplicates were removed. Ten full-text articles were determined to be eligible. Finally, five articles were chosen after passing the eligibility criteria. Fig 1 is a PRISMA flowchart depicting the entire study flow (S1 Fig PRISMA flowchart).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1.

https://doi.org/10.1371/journal.pone.0308906.g001

Data extraction and synthesis

To address the review question systematically, relevant data were carefully extracted from the included studies using the methodological framework provided by the Joanna Briggs Institute (JBI) [13]. This extraction process included a wide range of information, such as the authors’ names, publication year, country of origin, and various study characteristics such as setting, study design, aim, significant findings, and outcomes, which are listed in Table 1 (S2 Table. Summary Table of Included Studies).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Summary table of included studies.

https://doi.org/10.1371/journal.pone.0308906.t001

The extraction of study design was executed with precision to ascertain the rigor and relevance of the studies, adhering to the PO and PICO levels of evidence delineated by the Centre for Evidence-Based Medicine (CEBM) for a nuanced understanding of the research methodologies employed [15]. In order to map the included research, we emphasized on the complexities of process development and validation in the context of trigger tool implementation. Furthermore, key findings pertaining to the aforementioned processes were systematically extracted to summarize the essence of the studies. This entailed a careful examination of the complexities involved in the creation and validation of trigger tools, shedding light on the differences and challenges encountered. Moreover, the outcomes measured in each of the five selected studies were extracted, allowing for a more comprehensive synthesis of the findings and increasing the overall depth and clarity of the review.

Results

The final review included five articles. The studies were carried out in various regions, including the United States [16–19] and the Netherlands [20]. We examined the included studies to determine the main findings about the process development and validation of the trigger tool. S1 File contains a detailed extraction of the study (S1 File Detailed extraction of the study).

Discussion

Despite its importance, studies related to the development and validation of trigger tools in primary care are scarce. Therefore, we also examined combined settings to gather additional information. This scoping review discovered some similarities in the development and validation of trigger tools in both primary and combined primary and secondary care settings. In regards the triggers development, the primary process was started by clearly define the triggers or red flag criteria that were eligible for care measures and those that were not, based on the patient medical record [16, 19]. Triggers are flags or prompts discovered during a review of the medical record that prompt further investigation to determine the presence or absence of an adverse event [21]. Each trigger had its own adverse event analysis flowchart that described the criteria required to confirm or deny the occurrence of AEs when the trigger was detected [22]. To make the trigger more practical, the exclusion and appropriate follow-up criteria need to be clearly defined [16, 17]. The criteria were then programmed into a computerized algorithm, and drug-AE pairs that met the pre-defined criteria and trigger phrase rules were detected [16–18]. The AEs were more likely to be detected with high scores calculated automatically from the patients’ EMR, resulting in poor outcomes or even death [23, 24].

We also discovered some new ways to define trigger criteria from studies conducted in primary care settings by analyzing extracted cohort data from patients who met the initial denominator [17]. Clinicians must define and redefine the denominator and numerator definitions that represent all patients on a regular basis [17]. The denominator refers to patients who were eligible for the care measures, while the numerator refers to those who were not [17]. Another study conducted at the Children’s National Hospital used the same method to define the triggers criteria where clinicians devised a numerator and denominator measure to quantify trigger criteria [25]. The numerator was the percentage of cases that represented true AEs or near-miss events, and the denominator was the total number of triggers activated [25]. In contrast, a Chinese study used a literature review and Delphi method to develop ADE trigger tools for Chinese geriatric inpatients [26, 27]. Triggers were developed based on the literature review and clinical logic, then adapted to local healthcare settings. The next step was a Delphi panel, which was a structured and reliable method for gathering the opinions of an expert group in order to make a decision. Those triggers were first revised and presented to the two-round Delphi panel for revision and rating as part of the validation process [26, 27].

The validation approach was similar across most studies, involving manual chart reviews conducted by experts, including both physicians and non-physicians, with a specific emphasis on content and concurrent validity. In prior research conducted in combined hospitals and clinics in the mid-western US, red flags were defined, and algorithms were developed to identify them from patient data, thereby creating electronic "triggers" for diagnostic delays [28]. Additionally, the validation of trigger tools involved experts applying each criterion to the database and conducting a manual review of trigger-positive patient records to ensure accurate identification of the relevant information [29]. Following validation, all criteria were compiled into a stepwise algorithm that identified red flags, excluded clinical exclusion criteria, and determined appropriate follow-up criteria. These electronic trigger algorithms have demonstrated success in detecting delays in the follow up of red flags indicative of colorectal, hepatocellular, bladder, lung, and prostate cancer [30]. This underscores the effectiveness of the validation approach in ensuring the accuracy and reliability of trigger tools, particularly in the context of detecting diagnostic delays for various types of cancer.

The majority of previous attempts at validating trigger tools focused on positive predictive value [31]. Trigger tools with high positive predictive values are critical in reducing false positive rates, as long as their sensitivity is kept to a reasonable level [32]. This metric is useful for estimating the yield within the flagged records as well as the excess workload caused by false-positive flags [33]. The more sensitive the trigger tool, the more likely it will be developed using clinical decision rule methodology on the process of trigger tool development [34].

The process of developing and validating the trigger tool was also comparable to research done in hospital settings in Sweden and Singapore [12, 35, 36]. In Sweden, a pediatric trigger tool was developed with a broad literature review and expert opinion as validation using a modified Delphi process. Then, in Singapore, the process of developing mental health trigger tools was based on existing literature to develop the trigger list [36]. Furthermore, there were two stages of review during the validation process. For each trigger detected in the patient’s medical record, all records were reviewed by the respective registered nurse or pharmacist (non-physicians) [12, 35, 36]. The physicians then conducted a final independent review to validate the records that had been identified as having ADEs. Another study in Canada used a different validation process in hospital settings [32]. It indicates that the trigger tools were validated by comparing adverse events identified on the tools to events identified by clinical care providers at the point of care (retrospective methods). The significance of validating retrospective adverse event case finding methods against a robust prospective standard was to allow for refinement prior to widespread implementation [32].

The development and validation of trigger tools resulted in a promising and efficient safety review in emergency medicine in order to detect all harm from patient medical records [37]. Trigger tools have been developed in various clinical settings and countries, with excellent results as surveillance tools for detecting errors [38, 39]. Trigger tools are intended for AE surveillance in order to detect all causes of harm, both preventable and unpreventable [37, 38]. As a result, this greatly assists clinicians in preventing treatment errors that can result in adverse events [40]. The development of trigger tools could detect a broader range of events, allowing for the establishment of a baseline for assessing and allocating resources to improve quality care and patient safety [38]. Triggers that are detected will prompt a detailed review to look for evidence during the validation process, resulting in significant reductions in false errors [37, 40].

Recognizing the study’s limitations is critical, and one notable limitation is the scarcity of research conducted in primary care settings. This limitation stems from the use of a limited set of keywords as well as the limitation of databases to English-language papers. It is critical to understand that this focus may inadvertently exclude valuable insights from non-English sources and alternative terminologies. Despite these limitations, the study makes an important contribution by including updated information on the development and validation processes in both primary care and combined primary-secondary care settings. The inclusion of a time frame (within the last 5 years) ensures the data’s relevance and currency. Furthermore, the study adheres to the Centre for Evidence-Based Medicine’s (CEBM) rigorous standards, incorporating levels of evidence based on the PO (Patient/Population, Outcome) and PICO (Patient/Population, Intervention, Comparison, Outcome) frameworks. This methodological approach improves the analysis’s robustness and reliability.

Conclusion

This study outlined a thorough process for developing trigger tools, emphasizing the critical step of clearly defining triggers or red flag criteria through an extensive literature review. A critical aspect of ensuring accuracy is the validation process, which includes a dual evaluation by non-physician professionals followed by a final validation by physicians. The emphasis on identifying triggers with positive predictive value is consistent with the use of clinical decision rule methodology, which improves the precision and reliability of trigger tools in detecting adverse events. This study emphasized the similarities between the processes of developing and validating trigger tools in primary care and secondary care or hospital settings. The similarity of these processes suggests that primary care can benefit from the methodologies used in hospital settings. Primary care practitioners can improve their understanding and application of trigger tools by drawing parallels, contributing to improved patient safety and proactive adverse event detection.

Supporting information