Project design and local context

The Optimising Treatment of Community-Acquired Pneumonia (optiCAP) project was designed as a 16-month, prospective, open, interventional, controlled, multicentre quality improvement study, evaluating a change programme by applying statistical process control (SPC) [26].

The study was conducted at four regional hospitals in Denmark: Nordsjaellands Hospital (Site 1), Gentofte Hospital (Site 2), Silkeborg Regional Hospital (Site 3) and Hvidovre Hospital (Site 4). At each hospital, emergency departments (EDs), as well as departments of respiratory medicine and infectious diseases, served as study sites. S1 Fig in the S1 File presents an overview of the study sites, with further details available in S1 File.

Based on a baseline audit (November 2017 until February 2018), interventions were scheduled to take place from March until October 2018 at Sites 1–3. This included both technical and non-technical interventions based on the model for improvement, including Plan-Do-Study-Act (PDSA) cycles [27]. Site 4 served as control site without any interventions, and thus without deliberate attempts to improve the quality of CAP care.

During a follow-up period from November 2018 until February 2019, we evaluated the sustainability of the improvements.

Study population

The present study included adult patients admitted to our study sites (age ≥ 18 years) with CAP, treated with antibiotics. The diagnostic criteria used to define CAP are generally very heterogeneous [28]. In our study, we used one of the common definitions. Hence, CAP was defined by the presence of a new infiltrate on chest X-ray and at least one of the following signs and symptoms: cough, sputum production, dyspnoea, core body temperature >38.0°C and auscultatory findings of rales.

Exclusion criteria were hospital admission during the last 28 days, active tuberculosis and immunosuppression. Patients were classified as immunosuppressed if they had received treatment with corticosteroids (≥20 mg prednisolone-equivalent/day > 14 days), were HIV positive, had received chemotherapy during the last 28 days, had neutropenia (< 1000/μl), were an organ transplant recipient or received biological response modifier therapy.

Measures

Based on current evidence, patients admitted with CAP should be diagnosed and receive antibiotic treatment within 8 hours from admission, as this reduces mortality [29]. Furthermore, the collection of lower respiratory tract samples is recommended for all patients admitted with CAP in Denmark. As the Scandinavian countries continue to use narrow-spectrum antibiotics for most CAP patients, the microbiological results may eventually be used for altering the antibiotic therapy [4,30,31]. In Denmark, as in other countries, the CURB-65 score is used to assess CAP severity and ultimately to determine which type of empiric antibiotic to prescribe [4–6].

Based on these facts, we defined the following key indicators of adequate care for CAP:

• Process measures [27], i.e. actions completed within 8 hours after admission:
  1. Chest X-ray
  2. Collection of lower respiratory tract samples (LRTS)
  3. Documentation of mortality risk assessment by CURB-65 score (confusion, plasma urea, respiration frequency, blood pressure, age 65 or older) [32]
  4. Administration of an antibiotic
• Outcome measure [27]: Care-bundle treatment, i.e. proportion of patients who received all four elements as described in the previous point

Data collection

At each site, data were collected weekly by auditing electronic health records. An audit of 5 to 10 health records per site per week was considered appropriate, and these numbers could be reached by including all CAP patients at out study sites [27]. Local teams collected the data, which were transferred and stored in Research Electronic Data Capture (REDCap, version 9.3) software [33]. Raw data were subsequently cleansed using tidyverse (version 1.2.1) for R (version 3.6.0, R Core Team 2019). This implicated the detection and correction of missing values, negative values and outliers (values below the first and above the third quartile).

Baseline results and theory of change

Baseline data indicated variation in performance at the different sites, but also room for improvement regarding all of the defined process measures. Median time (interquartile range [IQR]) to chest X-ray was 2.3 (1.2–4.5) hours, corresponding to a completion rate of 88.7% within 8 hours. LRTS were collected in 60.6% of cases; the completion rate was 39.6% within 8 hours. CURB-65 score was documented in 31.2% of cases at a completion rate of 26.9% within 8 hours. The median time to administration of the first antibiotic (IQR) was 5.2 (3.4–8.0) hours, corresponding to a completion rate of 75.1% within 8 hours. The complete care bundle was delivered within 8 hours in 7.1% of cases.

Non-adherence to most of the given recommendations was an issue at all sites. To develop solutions and improve adherence to CAP guidelines, our team defined the following explanatory theories over the course of several seminars and meetings [34]:

• Lack of information regarding the disease as well as how and why to manage it as recommended by the guidelines
• Lack of skills regarding acquisition of LRTS
• Ineffective processes and unclear responsibilities
• Lack of effective electronic health record system (EHRS) tools to guide clinicians

Through the following approaches, we expected that we could reach our aim of achieving sustainable changes [34]:

• Provide more and easily accessible information on the disease, current guidelines and evidence-based practices, clearly explaining why they should be applied
• Provide feedback to clinicians when care delivery problems are discovered during the audit process
• Change ineffective processes using an integrative approach (feedback from clinicians)
• Streamline information and guideline content

The drivers and detailed change ideas are provided in the driver diagram in Fig 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Drivers and detailed change ideas derived from our theory of change.

CAP: community-acquired pneumonia, CURB-65: pneumonia mortality risk score (confusion, urea, respiration frequency, blood pressure, age 65 or older) [32], MD: medical doctors.

https://doi.org/10.1371/journal.pone.0234308.g001

Interventions

The interventions included both non-technical and technical activities (Table 1), which were derived after a baseline workflow assessment using optiCAP team members as observers. Furthermore, staff members were interviewed to identify limitations of the use of CAP guidelines and to gather suggestions for improvement.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. An overview of the implemented interventions.

https://doi.org/10.1371/journal.pone.0234308.t001

The technical interventions were designed to increase the amount of LRTS and consisted of hands-on training for physicians and nurses. Training in tracheal suction consisted of repeated sessions 10–20 minutes in duration with 5–15 participants. A senior physician explained and performed the procedure on a dummy. The training was offered to physicians (Sites 2 and 3) and nurses (Sites 1 and 3). After these sessions, participants were encouraged to continue with bed side training under the supervision of more experienced colleagues. At Site 2, an experienced senior nurse designed and implemented hands-on training in sputum induction, which consisted of a 5-10-minute theoretical introduction followed by bed-side training for 1–3 nurses per session.

Regarding the non-technical interventions, we divided the activities into (1) educational activities, (2) the design and distribution of educational material and (3) improvement of CAP patient flow.

The education of nurses and physicians involved repeated presentations lasting 15–45 minutes. Presentation topics included CAP pathogenesis, epidemiology, and guideline-based assessment and treatment. Later, during the intervention period, the results of the ongoing audit were presented and discussed. These educational sessions were held by an optiCAP team member, often as a part of the morning report or other regular teaching sessions involving 10–40 participants. Additionally, physicians received both confirming and corrective feedback from the study team during the health record audit. Feedback was always provided personally, either face-to-face, via secure work email or via the message system in the electronic health record system.

Educational material on the most important aspects of CAP included slide presentations for education sessions, newsletters to all physicians, and posters in all staff offices of the departments involved in the project, as well as pocket cards related to CAP. This material provided information on important steps in the assessment and treatment of CAP. The newsletters and presentations also included audit results and figures on the overall progress of the project.

Process changes involved authorising nurses to order X-rays and LRTS based on triage information at the ED, as well as an agreement with the Department of Clinical Microbiology to perform both standard microscopy, culture and sensitivity analysis (MCS) along with PCR for atypical bacteria from a single LRTS. Previously, this testing required two samples, and physicians often had to choose which test to perform. Furthermore, we developed several tools for the electronic health record systems (EHRS; Epic EMR and MidtEPJ) to assist physicians in treating CAP patients. These tools offered standard phrases for CURB-65 documentation as well as order sets consisting of guideline-based tests and antibiotic treatment packages.

Study of interventions and data analysis

For effect evaluation, we produced run charts with qicharts2 (version 0.6.0) for R [35]. Run charts are point-and-line graphs used to distinguish random from special cause (non-random) variation in time series data. The R code to reproduce the run charts and the final database are attached in the Supporting Information. For interpretation, we applied the Anhoej rules with the median as process centre [36]. The Anhoej rules are two tests for special cause variation: (1) unusually long runs of consecutive data points on the same side of the centre line, and (2) unusually few crossings of the centre line. Critical values for run length and number of crossings depend on the number of available data points and may either be calculated or found in statistical tables [36–38]. For example, in a run chart with 24–26 data points, a run of more than 8 data points or fewer than 8 crossings would indicate the presence of special cause variation. In the initial analysis, we used the median from the baseline period, including data from 12 ten-day periods each representing 12–23 patients at the control centre and 31–48 at the intervention sites as reference. With 49 data points in total, special cause variation was declared if any run exceeded 9 data points in length or if the curve crossed the centre line fewer than 19 times. Next, we recalculated the process centre (medians) for four periods: the baseline period, early intervention period (March 2018 to July 2018), late intervention period (August 2018 to October 2018) and the follow-up period.

Ethical considerations

The presented project was designed as a clinical audit and quality improvement project without direct patient contact. Therefore, the national authorities determined that ethical approval was not required to conduct this project. Local approval was granted by the respective hospital boards. Data for analysis were anonymized and handled according to the national regulations of the Danish Data Protection Agency (registration number HGH-2017-039).

Patient characteristics

In total, we audited health records of 2015 patients with CAP who were admitted at the study sites (Table 2). The cohort had a median age of 75 years and a balanced gender distribution. Chronic obstructive pulmonary disease (COPD) was the most common respiratory comorbidity among 30% of patients, followed by asthma in 9%, interstitial lung disease in 3%, lung cancer in 2% and bronchiectasis in 2%. Over half of all patients were former or current smokers; only 18% were never smokers. Smoking history was not documented in nearly one-third of the patient files.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Patient characteristics.

https://doi.org/10.1371/journal.pone.0234308.t002

More than 50% of all patients had a CURB-65 score of 0 or 1, corresponding to a very low risk of pneumonia-related mortality [32]. Overall in-hospital mortality was 7% (range 5–8%). There were no statistically significant demographical differences between the sites.

CAP care bundle utilisation over time

Since the beginning of the intervention period in March 2018, we noted a steady increase in care bundle utilisation, with special cause variation caused by a sustained shift at all the intervention sites (Fig 2, S2.1 and S2.2 Figs in S1 File) At the control site, we only observed random variation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Run chart showing the proportion of patients receiving the CAP care bundle (i.e. chest X-ray, LRTS, CURB-65 and antibiotics) within 8 hours from admission.

Each dot represents 12–48 cases of CAP. The vertical, grey, dashed line marks the beginning of the intervention period. The vertical, black, solid line denotes the beginning of the follow-up period. The process centre (horizontal line representing the median) was frozen after the baseline period. Special cause variation can be identified by a red, dashed process centre (sustained shift) [36]. See S2.1 Fig in S1 File for run charts for the individual intervention sites along with information on the timing of our interventions.

https://doi.org/10.1371/journal.pone.0234308.g002

In the follow-up period, the care bundle was completed in 41% of cases as compared to 11% at baseline at the intervention sites, indicating sustained improvements (S2.2 Fig in S1 File). Meanwhile, the completion rate remained below 3% at the control site.

The detailed, site-specific run charts including information on the implementation schedule are presented in S2.1 Fig in S1 File.

Utilisation of the individual elements of the care bundle over time

At the intervention sites, we observed special cause variation for all the elements of the care bundle: chest X-ray, LRTS, CURB-65 score and antibiotics within 8 hours from admission (Fig 3). At the control site, we saw random variation without substantial improvements regarding all bundle elements throughout the study period.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Run charts showing the proportion of patients receiving individual elements of the CAP care bundle within 8 hours from admission.

Each dot represents 12–48 cases of CAP. The vertical, grey, dashed line denotes the beginning of the intervention period. The vertical, black, solid line marks the beginning of the follow-up period. The process centre (horizontal line representing the median) is frozen after the baseline period. Special cause variation can be identified by a red, dashed process centre (sustained shift) [36]. See S3.1–3.8 Figs in S1 File for run charts for the individual intervention sites along with information on the timing of our interventions.

https://doi.org/10.1371/journal.pone.0234308.g003

Main results

Our project aimed to improve patient care by increasing adherence to CAP guidelines through the implementation of tailored interventions. Throughout the study, changes in key process measures following the interventions were subsequently assessed and summarised in a CAP care bundle. Overall, we achieved sustained improvement in the use of the bundle comprised of chest X-ray, CURB-65, LRTS and antibiotics administered within 8 hours from hospital admission.

At baseline, completion rates for all defined indicators varied, and the study sites faced different challenges regarding the management of CAP patients. This information highlights the necessity of focusing on the local challenges when designing improvement studies, and to tailor the interventions to these challenges [27].

Only approximately one-third of all patients were scored with CURB-65 or had an LRTS taken within 8 hours from admission at the hospital. Antibiotics were administered within the recommended 8 hours in 75% of all cases. Chest X-ray within 8 hours was the only component that reached a completion rate of 89% at baseline.

Throughout the intervention period, we observed significant improvements with special cause variation at the intervention sites, especially regarding the documentation of CURB-65, but also in the acquisition of LRTS and, to a lesser degree, chest X-rays and antibiotic administration within 8 hours. The last two measures were those with the highest completion rates at the start of the study. Thus, it seems that the improvement potential was higher for the other two elements of the care bundle.

Our outcome measure was an all-or-none indicator, combining the key process measures in a care bundle [16]. Thus, the system we studied reflected the effectiveness of the health care system, as opposed to patient outcomes [27]. We chose this approach based on evidence from larger clinical studies clearly documenting that guideline-based treatment, also with the application of care bundles, improves outcomes in CAP [24,25,29,39–41].

Care bundle completion rates within 8 hours increased considerably at our intervention sites from 11% at baseline to 41% in the follow-up period.

The finding that only 41% of all CAP patients received assessments and treatments conforming to established guidelines might seem concerning. However, given that the care bundle relies on four independent elements, we believe that this result still is remarkable when compared to other studies on CAP care bundles reporting compliance rates of 20–29% [16,23,42]. LRTS collection can be challenging to improve in CAP, but it is regarded as crucial in the Nordics for potentially altering the often narrow-spectred empiric antibiotic treatment [4,30,31,43]. Omitting this element of the care bundle from our analysis yielded a completion rate of over 70%.

We used a range of interventions during the intervention period, all of which likely contributed to the observed improvements. We believe that the repeated educational activities with reiterations every 1–2 months were crucial for increasing and sustaining care bundle completion [44]. This is also reflected in the run charts, where compliance with the care bundle increased almost every time an educational activity took place. Beyond that, process changes, such as authorising nurses to order tests (i.e. chest X-rays and LRTS) and providing standard phrases and order sets in the EHRS to guide clinicians also seemed to be effective. In this context, a thorough review of local processes was vital to understanding the local difficulties and needs for advancement, as is generally recommended in quality improvement [27].

Since all of our interventions were reasonably simple, dedicated clinicians should be able to integrate them in their daily routine at little expense if they are given time to do so. Apart from human resources, the only intervention incurring additional costs was the pocket cards on CAP treatment.

The most time-consuming aspects of our project were the screening for CAP patients via the X-ray systems and the health record audit. Approximately 15 hours per week per site were spent on this task throughout the study period. We chose this approach because of the well-documented inaccuracy of the International Classification of Diseases (ICD)-10 diagnosis of pneumonia. In a recent Danish study, more than one fourth of patients diagnosed with pneumonia did not fulfil the criterion of a new infiltrate on chest X-ray [31].

To ensure a consistent and streamlined audit and reliable data across all sites, clinicians were not permitted to audit the health records. A simpler and more feasible audit approach should be implemented in a daily routine and could involve continuous monitoring of CAP cases by a team of dedicated clinical staff members.

Strengths

The greatest strengths of our study are its size as well as the participation of one control and three intervention sites. The use of a control site is unusual in quality improvement, and the finding of improvements at the intervention sites together with a lack of change at the control site strengthens our conclusions [45]. In addition, we included a representative study population from study sites covering regions that serve approximately 15% of the Danish population. Hence, we think comparable programs could be designed throughout the country leading to comparable results.

Moreover, the study population was a real-life CAP cohort that is demographically comparable to another large Danish CAP cohort [31].

Limitations

The main limitation of our study is the reliance on the audit of health records that only contain the information that health professionals document. We could not identify a study investigating the quality of documentation in the EHRS used at our study sites; however, based on our experiences, it is generally high. Nevertheless, we cannot be sure whether a certain proportion of patients did not meet the bundle care criteria due to a lack of documentation.

Finally, the limited follow-up period of only four months does not allow an assessment of sustainability. This question will be the focus of a follow-up study starting in November 2019 and continuing until February 2020.