Study design

Understanding how complex processes such as sedation and ventilation are implemented in the ICU setting requires appropriate methodology [11]. We used a mixed methods rapid evaluation. Rapid evaluation methodology is used to identify priorities for and context to improvement of healthcare services in a timely manner [12]. The study design was informed by a rapid research decision tool [13]. The constituent methods we employed were: daily measurement of delivery of care processes, process mapping (to identify perceived gaps in quality), observations (to ascertain how care is actually delivered), focus group discussions (to consider the local context for improvement), semi-structured interviews (to understand individual perceptions of local systems). A structured assessment of local capacity for the Model for Understanding (MUSIQ) calculator [14] was also collected as the participating sites were planning to use the findings of this study to inform a future quality improvement project. A peer-reviewed protocol for the study has been published [15]. This manuscript has been written according to the Standards for Rapid Evaluation and Appraisal Methods (STREAM) [16].

Setting

The study was conducted in three ICUs in Vietnam. Vietnam has population of more than 100 million people [17]. The country has made significant progress toward achieving Universal Health Care through enacting the Law of Social Health Insurance in 2008 and its revision in 2014 [18]. As of 2021, 89.3% of Vietnamese people had personal health insurance (via a national government scheme), up from 62% in 2010 [19]. This already high proportion is expected to increase beyond 95% by 2025 [20]. However, according to the World Health Organization, in 2016, out-of-pocket payments accounted for 41% of total health expenditure in the country [21]. The Ministry of Health manages three levels of health service delivery: primary level in districts and communes, secondary level in provinces, and tertiary level in national institutions under central government control. Private hospitals also exist with payment by private healthcare insurance schemes and/or out-of-pocket payments by patients [17].

The Collaboration for Research, Improvement and Training in critical care in Asia and Africa (CCAA) works with clinical and administrative stakeholders to establish capacity to evaluate and improve the quality of critical care. The collaboration has implemented a cloud-based intensive care registry enabling replicable capture of measures of quality and providing feedback through analytics dashboards and reports displaying indicators of quality [15]. The registry platform has evaluated as high quality according to frameworks of data quality, structural and organisational performance [22]. In Vietnam, seven ICUs joined the network, of which three participated in this study. ICU1 is a tertiary hospital and the other two ICUs are provincial hospitals which manage patients with a variety of conditions. All three ICUs have an existing collaboration with CCAA and have established registry infrastructure to deliver audit and feedback.

Ethics

Ethical approvals for the interviews were given by the Oxford Tropical Research Ethics Committee (OxTREC) in October 2020 (Reference: 558−20) and hospital IRBs in November 2021 (refs 3355/QD-BVBND). Data were collected between 01/11/2021 and 31/12/2023. Ethical approval was granted for the rapid evaluation in June 2021 (OxTREC reference 525−21) and IRB approvals in December 2022 (ref 4365/QD-BVBND). These data were collected between 01/02/2022 and 31/12/2023.

Clinical processes being studied

RASS, SBT and SAT were chosen for study because local clinical leaders perceived sedation and ventilation to be an ongoing concern with staff and hoped to implement projects to improve them in the future. Unlike other quality processes, there were no Ministry of Health protocols describing RASS, SAT and SBT, and therefore different approaches were taken at different hospitals. The processes also had a strong evidence base, and had been recommended as feasible, reliable, valid and sensitive by a consensus-building study investigating critical care quality indicators in LMICs [23]. Definitions for the care processes can be found in Supplementary Information.

Eligibility, sampling and recruitment

Staff were considered eligible to participate if they were aged 18 years or older, were involved in the design and/or delivery of care for ventilated and/or sedated patients in the ICU and if they provided informed consent (in writing for interviews and FGDs, or verbally for observations) after receiving a participant information sheet (Supplementary Information). Patients were included in the registry data if they were admitted to ICU for at least 24 hours, received mechanical ventilation and/or sedation, and were aged 18 years or older. Participants were purposively sampled [24] to reflect a range of professional, clinical and organisational views. participate as they were not involved in the sedation-related processes of care being studied. Sampling continued until theoretical saturation was achieved [25].

Data collection

We conducted two cycles of data collection including process mapping, structured observations, focus group discussions and MUSIQ questionnaire completion. All data collection with participants was conducted in Vietnamese before subsequent translation into English. Procedures for collecting and reporting daily quantitative data via the registry to describe delivery of clinical processes are already published [22].

Process mapping was conducted during in-person sessions with between two and four multidisciplinary clinical members staff [12] (Supplementary Information). Participants created visual maps of patient pathways for the study processes (RASS, SAT and SBT) to represent the current typical process, preferences for how it should ideally be conducted. Contextual determinants influencing the study processes were integrated into the maps.

Structured observations were conducted in settings and scenarios where sedation and ventilation were being planned, delivered or discussed. Settings were identified prior to the start of data collection by consultation with ICU teams and adjusted based on findings during the study. Data were collected and contemporaneously documented by ALP using a structured observation template to ensure data consistency (Supplementary Information).

Focus group discussions were conducted with between two and eight participants. Discussions were semi-structured, using a guide (Supplementary Information) to maintain consistency between settings, but responsive to emerging themes and findings from the process mapping and observations. Audio recordings of discussions were collected and contemporaneous field notes made.

The MUSIQ questionnaire is a validated tool to identify organisational experience of and capacity for quality improvement [14]. The questionnaire was completed at each ICU at the beginning and end of this project. Data collection was coordinated by a member of the evaluation team with 3–6 participants agreeing consensus answers about local organisational contextual factors. The total possible score is 168, and there are seven thresholds which suggest the likelihood of success of QI projects given the context (Supplementary Information).

Semi-structured interviews were carried out to gain deeper insight into the processes of audit and feedback within the ICUs. These followed pre-designed interview guides and were conducted by research staff trained in social science methodology. Participants were interviewed twice: at the beginning of the study and again approximately 12 months later.

Data analysis

Data collection and interpretative analysis occurred in parallel. Analysis was coded according to the Consolidated Framework for Implementation Research (CFIR) framework [26]. The findings of the analysis were iteratively discussed by the study team, and then reconstructed using the Rapid Assessment Process (RAP) sheet (Supplementary Information), to maintain consistency and discover overarching themes. The validity of findings was discussed and checked with local clinical and registry participants at every site. Triangulation was performed by comparing and contrasting findings between qualitative methods, and with the registry data. Patients or the public were not actively involved in the design or conduct of this study.

Evaluation team

The team comprised four members to provide sufficient breadth of research expertise, language skills, and clinical experience in critical care. Three members were based in Vietnam, two of whom in a research capacity, and one in a combined clinical/research role at one of the study hospitals. One native Vietnamese speaker collected all qualitative data face to face in the study settings. One team member had undergone training in rapid evaluation methods, and three had extensive experience of qualitative methods. A reflexivity statement informed data analysis (Supplementary Information).

Description of current processes and perceived areas for improvement

RASS.

As summarized in Table 3, health professionals used qualitative labels for assessing sedation (such as ‘irritated/ awake/ asleep/ sedated’) much more commonly than quantitative or categorical scores such as RASS. Sedation, or more commonly ‘consciousness’, was discussed during ward round discussions in ICU1 and ICU2, but these rounds and discussions were less frequently observed in ICU3.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Quality of current care processes in ICU.

https://doi.org/10.1371/journal.pone.0339157.t003

There were parallel systems of assessment of sedation between doctors and nurses, using different metrics, and for different purposes. Nurses typically assessed and documented consciousness (using the Glasgow Coma Score (GCS) every 3 or 6 hours in all the ICUs. A project to improve documentation of RASS was being performed at ICU1, during which training was being delivered to nursing staff to increase familiarity with the tool. This led to RASS being documented on a dedicated audit sheet, separate to the nursing record, where it was not routinely viewed, discussed or used during normal daily care. These parallel systems explain the disagreement between observation findings (e.g., RASS wasn’t being used) and registry data indicating compliance of 83% patient days. In ICU2, doctors documented their assessment of RASS in their clinical notes, which were rarely discussed or accessed by nurses.

In all the ICUs, only doctors adjusted sedation dosing to achieve the depth of sedation they were seeking. Junior doctors were required to have their plans and orders approved by senior doctors. Nurses were not allowed to adjust sedation dosing (or any other medication) because of regulations to what they can practice in Vietnam; instead, they would report problems with sedation to doctors.

Many patients were deeply sedated due to the use of paralysing medications and/or their underlying disease (e.g., tetanus).

Documentation of sedation dose changes were made by doctors in the drug charts and/or medical records. Nurses hoped that these decisions could be communicated to them more consistently, and that using RASS might enable this. Doctors described strong personal ownership of sedation management for their patients, and this ownership was transferred during shift handovers to the next shift doctor.

‘I am in charge of that person... I am the one who set the [RASS] target. And if the shift changes on that day, a new doctor will examine that patient and set his/her own target for that day at the beginning of the shift’. [ICU3 FGD1]

Determinants of care quality

Contextual determinants of care quality are summarised according to the Consolidated Framework of Implementation Research (CFIR) [26] in Table 4, and are reported below.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Determinants of care quality according to the Consolidated Framework of Implementation Research (CFIR) [26].

https://doi.org/10.1371/journal.pone.0339157.t004

Intervention (key attributes of the clinical processes under study: RASS, SAT ad SBT)

Clinical staff reported not being persuaded of the relative advantages of using RASS, SAT and/or SBT. For example:

‘We are not proactive in [measuring RASS] it because we don’t see the benefits.’ [nurse, ICU1, FGD]

Mostly this was due to a lack of familiarity with these terms and/or processes, leading to persistence of existing care processes (such as qualitative descriptions of consciousness). The exception to this norm was in ICU2, where the clinical lead had instituted local protocols for measuring RASS and conducting SATs and SBTs. The other ICUs did not have protocols for sedation assessment.

There were only two examples of active opposition to using these processes. Firstly, there was reluctance to measure RASS as it was (erroneously) perceived to increase workload (‘costs’) by duplicating existing processes, such as assessment of consciousness. Secondly, at ICU1, doctors felt SAT was not helpful for specific groups of patients, for example those with tetanus. Hence participants felt that the processes would need adaptations to suit this different approach to weaning of sedation.

Implementation process (i.e., how RASS, SAT and SBT were implemented locally)

An in-person workshop had recently been held by CCAA to help engage staff at all three ICUs in improving sedation processes. The evidence base for the clinical processes being studied had been explained and participants had received training in quality improvement techniques, including process mapping.

Audit and feedback, via the registry platform, was planned to evaluate local delivery of RASS, SAT and SBT. Data collection processes varied between the ICUs. At ICU1, non-clinical staff collected data. These staff were not directly involved in routine clinical care of patients, and struggled to interpret clinical documentation; for example, documentation of ‘sedation weaning’ made it hard for them to ascertain whether SAT and SBT had been delivered. In contrast, clinical staff at ICUs 2 & 3 collected registry data, forcing them to engage with the data as they collected it. Furthermore, data completeness at all ICUs was regularly reviewed by a data quality officer, who highlighted gaps to all local sites. It is therefore possible that some data were entered retrospectively.

Challenging data collection inhibited the opportunity for reflection and evaluation which monthly feedback (via registry reports) provided. Data were discussed at monthly meetings, but attention was focused on technical issues of data completeness, or metrics which had to be reported externally (such as those describing operational issues such as admissions and discharges, or hospital acquired infections which the department of health needed reporting weekly) rather than conduct of the interventions themselves. Despite these challenges, interviewees at ICUs 2 & 3 stated that the reports helped them to look at sedation processes, and was helping them to make changes: “We prioritize to stop sedation early to prevent VAP since we have Registry.”

Outer setting (the context(s) in which the ICUs exist, e.g., the hospital, community and/or country)

Partnerships within the Vietnam network of ICUs, and with the registry team, facilitated implementation through training, registry infrastructure, resources for quality control, IT support and funding for data collectors. The three hospitals prioritised using the registry to obtain data aligned with external pressures for performance measurement by the Department of Health. There was a lack of overt financing incentives (or penalties) regarding sedation and ventilation from insurance companies (the most common healthcare payers in Vietnam) in contrast to countries like the USA. Since clinical records were designed to provide data that insurance companies required for payment, fields for RASS, SAT and SBT were omitted. Consequently, when ICU1 started recording RASS, a parallel set of medical records was created, creating duplication of work. National policies also influenced implementation because nurses were not authorised to manage doses of sedation and/or ventilator settings (or other medications).

Inner setting (the ICU settings)

Communication of sedation processes worked well within professional silos (i.e., between doctors during ward rounds, and between nurses at their handover meetings). However, nurses felt doctors didn’t always communicate sedation and/or ventilation weaning plans to them. For example, junior nurses were sometimes left confused:

‘Some doctors don’t tell the nurses that they are doing SBT; experienced nurses can notice [changes] from the ventilation mode, but junior nurses observe differences on patients and will ask the doctors.’ [ICU1]

At all three ICUs, there were 7–8 nurses looking after between 30–40 patients, of whom 45–65% were mechanically ventilated (Table 1). These high patient:nurse ratios increased fears of safety incidents if patients were not adequately sedated. These safety fears were exacerbated at night when fewer doctors were present. Furthermore, at ICU1, the night shift of doctors came from departments outside ICU, and so were less familiar with sedation processes. Participants felt that the variable level of experience of these doctors reduced the compatibility of using RASS, SAT and SBT consistently. Standardising sedation processes was more advanced at ICU2 then the other two settings, and as such it had greater relative priority there.

Individuals (the roles and characteristics of individuals involved)

Opinion leaders had a large influence on how sedation was performed, but the individuals who fulfilled these roles, and their impacts, varied between sites. At ICU2, the clinical lead of the ICU was proactive in implementing QI methods. ICU2 doctors recorded sedation and ventilation parameters in a dedicated spreadsheet to help them decide whether to initiate sedation breaks and/or wean ventilation; these data were also used for audit and research projects. He had also instigated local guidelines for sedation and ventilation weaning (which other ICUs were keen to borrow and implement in their own settings). He also ensured that sedation plans were discussed and documented during doctors’ handover meetings. Similarly, at ICU1, the clinical lead was perceived as influential, and was keen to improve documentation of RASS in parallel with this evaluation and delivered training sessions to nurses and doctors. We didn’t identify an opinion leader at ICU3.

There was variation between staff, but most participants agreed there was motivation to improve sedation and weaning processes. Staff perceptions of their own, and their peers’, capability was more mixed. Training was delivered at ICU2 for doctors and nurses to follow the local sedation guideline consistently. Only staff at ICU2 were familiar with using QI methods.

Organisational capacity for improvement

The results of the MUSIQ questionnaires are shown in Table 5. Scores at ICUs 1 & 2 indicate that a QI project focused on the three care processes under study would have a reasonable chance of success, and possible contextual barriers existed at ICU3. ICU2 had the highest score of 150 which tallies with the qualitative findings above regarding the influence of the QI Champion, presence of protocols and clinician familiarity with QI methods at that unit. The registry-enabled audit and feedback was important infrastructure that staff saw supporting implementation. Participants perceived that further engagement from departmental and hospital leaders would be necessary to support implementation. All three hospitals had dedicated general planning and quality management departments, mandated to work on priorities set by national policies. They possessed the skills (and some data) to measure and improve sedation and ventilation, but were disconnected from the clinical teams attempting to improve these care processes. This represented a missed opportunity to leverage relational connections within the hospital. Scores at all three ICUs remained within the same brackets at the end of the study in comparison to the start, suggesting there weren’t material changes in context during the period of data collection.

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Organisational readiness for quality improvement (MUSIQ score).

https://doi.org/10.1371/journal.pone.0339157.t005

Limitations

Firstly, this study was limited to ICUs in government hospitals in Vietnam and should not be considered representative of practices in other settings. However, by describing the contexts of the ICUs we studied, we hope readers may be able to judge how our findings apply to their own settings. Secondly, by triangulating interview and focus group findings with observations and process data we have minimised but not completely removed the effects of our institutional alignments. Thirdly, we considered three specific clinical processes, but of course these are not performed in isolation but are inextricably linked to multiple other clinical processes and patient outcomes which were beyond the scope of this study. Fourthly, the sample of clinical staff that contributed data to this study necessarily limits the generalizability of our findings. Finally, a hangover from the COVID-19 pandemic may have been present during this study (conducted in 2022−23). The pandemic influenced patient acuity, workload (such as prone positioning) and workforce constraints (such as use of personal protective equipment) and may have rendered fewer patients eligible for SAT/SBT [27].