Study design

We conducted an unblinded stepped-wedge cluster-randomised trial in 12 hospitals. We introduced pulse oximetry to all hospitals in November 2015, followed by stepped introduction of the full oxygen system intervention over four sequences (three hospitals every 4 months from March 2016 to March 2017) following a prespecified timetable (Fig 1, S1 Fig). We completed postintervention data collection at the end of October 2017. We collected 2 years of preintervention data retrospectively from all hospitals for extended analyses (November 2013 to October 2015).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Study diagram showing number of admissions in each hospital, by step.

Rows indicate individual hospitals (clusters). Columns indicate individual time periods (steps). Numbers indicate the number of neonatal (A) and child (B) participants included in analyses (excluding those admitted during the 2-week transition period at the beginning of each step). We provided all hospitals with pulse oximeters and training in November 2015 (step 7) and then introduced a multifaceted improved oxygen system to three hospitals every 4 months (randomised order). Blue bar graphs show the hospital-specific admission trends over time for quick reference, with the overall trend for neonates and children indicated with a spark-line. Notes: Data from November/December 2013 not available (first half of step 1).

https://doi.org/10.1371/journal.pmed.1002951.g001

We chose a stepped-wedge design as the most efficient and pragmatic trial design, enabling all hospitals to receive the full oxygen system intervention over a period that was commensurate with the logistics of implementation. Clusters were at the hospital level, the natural level at which the intervention would be implemented. Geographic separation between hospitals ensured minimal risk of contamination. Additional detail on study methods has been published elsewhere [27] (protocol DOI: https://doi.org/10.1186/s13063-017-2241-8).

Participants

We selected hospitals in collaboration with state health authorities, purposefully selecting hospitals that were representative of secondary-level health facilities and that provided inpatient care to children. Hospitals were eligible if they were registered as secondary-level health facilities and admitted at least 150 children per year. We included government and mission-run hospitals but not other private hospitals. We included hospitals in four states of southwest Nigeria (Oyo, Ondo, Ogun, and Osun states). We included nine general hospitals and three hospitals that focused on paediatric and/or maternity care (see S1 Text for additional detail on participating hospitals). AGF and AIA enrolled hospitals following an informational meeting with hospital representatives, with consent provided by hospital representatives prior to randomisation. We did not require individual patient/carer consent.

We collected data on all children (aged <15 years), including neonates (aged <28 days), admitted to participating hospitals during the study period. We had no additional exclusion criteria. We assumed participants admitted in different periods were independent. For effectiveness analysis, we excluded participants admitted during a 2-week ‘wash-in’ period at the beginning of each crossover period to implement the intervention and avoid contamination.

Randomisation

We conducted randomisation and allocation at a team meeting in Ibadan, Nigeria, using a computer-based random sequence generator to allocate hospitals to receive the full oxygen system at prespecified dates. We stratified hospitals by size (large > 1,500 paediatric admissions per year; small/medium < 1,500) prior to randomisation, ensuring one of the four largest hospitals was represented in each step. Concealment of the intervention or its start date was not possible, as the intervention required active participation from hospital staff in planning and implementation activities.

Intervention

The multifaceted intervention (full oxygen system) was delivered at the cluster (hospital) level and involved (1) a standardised oxygen equipment package, (2) clinical education and support, (3) technical training and support, and (4) infrastructure and systems support (Box 1, S2 Table). We designed the intervention based on experience from previous oxygen projects [7, 30, 31] and WHO guidelines [32–34] and were informed by educational, behaviour change, and implementation theories and strategies [35–39]. We adapted the intervention to local needs using a participatory planning process including a joint stakeholder meeting of representatives from all hospitals, site visits and needs assessment [13], and regular meetings between the project team and hospital staff. AGF, AIA, AAB, and HG conducted randomisation, invited hospital representatives to the major planning meeting, and then obtained consent to participate.

Procedures

We commenced prospective data collection and introduced pulse oximetry to all hospitals in November 2015. We introduced pulse oximetry prior to implementation of the full oxygen system to enable quantification of hypoxaemia prevalence and evaluation of the effect of the introduction of pulse oximetry alone (extended analysis). Hospitals received the full oxygen system in March 2016, July 2016, November 2016, or March 2017, in the randomised order. We conducted retrospective data collection in late 2015, covering the preintervention period (November 2013 to October 2015) for extended analysis.

During the preintervention period, hospitals provided care as usual, including oxygen therapy using their existing supplies. During the pulse oximetry period, hospitals continued usual care with the additional availability of project-supplied pulse oximeters. Following the full oxygen system intervention, hospitals provided clinical care using the new oxygen systems, clinical guidelines, and maintenance protocols.

Program theory

We hypothesised that to reduce mortality, effective oxygen systems must achieve (1) reliable oxygen access and (2) good clinical use of oxygen targeting hypoxaemic patients and that this would require technical, clinical, and administrative components [7]. We expected implementation of the full oxygen system to have greatest benefit for children with acute lower respiratory infection (ALRI) and preterm neonates, some effect on other conditions commonly associated with hypoxaemia (e.g., sepsis, cerebral malaria, meningitis, neonatal encephalopathy), and negligible effect on conditions rarely associated with hypoxaemia (e.g., diarrhoea, neonatal jaundice).

Outcomes

To evaluate effectiveness, our primary outcome was death (defined as death during hospitalisation or discharged expected to die), comparing the full oxygen system period to the pulse oximetry period. We report the primary outcome for four main subgroups: (1) children (excluding neonates), (2) children with ALRI, (3) neonates, and (4) preterm and/or low birth weight (LBW) neonates.

Secondary outcomes included (1) clinical outcome of death for additional subgroups (malaria, acute febrile encephalopathy [AFE], diarrhoea, neonatal encephalopathy, and neonatal sepsis) and (2) practice outcomes relating to oxygen-related quality of care (proportion of children/neonates with documented oxygen saturation, proportion of hypoxaemic children/neonates who received oxygen therapy, proportion of children/neonates who received oxygen therapy and who had a documented indication). We analysed outcomes (1) comparing the full oxygen system period to the pulse oximetry period and (2) comparing the full oxygen system period and the pulse oximetry period to the preintervention period (see ‘extended analysis’ description below).

We used standard WHO case definitions to classify diagnoses (ALRI = cough or difficult breathing and any of the following: fast breathing, lower chest wall indrawing; AFE = fever and any of the following: seizures, altered conscious state; diarrhoea = >3 loose stools per day, not >14 days’ duration; malaria = fever [or history of fever] and positive malaria test [rapid test or microscopy]; preterm/LBW = <37 weeks’ gestational age or <2,500-g birth weight) [34]. We based diagnostic classification on documentation in clinical records (i.e., if a particular sign was not recorded, we assumed that it was not present). Additional neonatal diagnostic classifications (neonatal sepsis, jaundice, neonatal encephalopathy) were classified according to the admission diagnosis.

All outcomes were determined using data extracted from case notes by trained data collection nurses who were based within each hospital but employed by the project. Standardised case report forms (CRFs) were completed by data collection nurses, returned to the project manager (AAB) for quality checking, and then submitted to the data manager (OBO) for double entry by trained data entry clerks using EpiData 3.1 (EpiData Association, Odense, Denmark). HG, AAB, and OBO reviewed datasets periodically in accordance with quality assurance and data management protocols [27].

Statistical analysis

We calculated power using the Hussey and Hughes method [40, 41], basing assumptions on admission, prevalence, and mortality data from our needs assessment [13], and informed by intracluster correlation coefficient (ICC) estimates from previous studies [42]. We estimated that we would enrol 16,770 children and 7,670 neonates (1,290 preterm/LBW) during the periods of pulse oximetry and full oxygen system and based our power calculations on this time period only (without the extended preintervention period). Our original estimation contained a mathematical error, and after correction, we calculated approximately 50% power to detect a 20% reduction in overall child mortality (4% to 3.2%, ICC 0.018), 35% power to detect a 35% reduction in child pneumonia mortality (6.1% to 4.0%, ICC 0.032), 50% power to detect a 20% reduction in neonatal mortality (8.2% to 6.6%, ICC 0.063), and 55% power to detect a 25% reduction in preterm/LBW mortality (18% to 13.5%, ICC 0.123). We calculated 99% power to detect a 10-percentage-point change in proportion of hypoxaemic children who received oxygen therapy (ICC 0.05).

We described patient characteristics by study period (preintervention, pulse oximetry, and full oxygen system periods). We summarised binary variables using frequencies and percentages. We summarised continuous variables using means and standard deviations or medians and interquartile ranges (IQRs) for non-normally distributed data.

We analysed outcomes using intention-to-treat principles. We excluded participants with missing outcome data from analysis. We report intervention effects as odds ratios (OR) with 95% confidence intervals (CIs) and report the crude proportions in tabular form and visually in heat-map tables (by hospital and step).

Fidelity and variation from protocol

The intervention was deployed as planned, with equipment installation and training occurring over 2–4 days at each hospital, usually in the first 2 weeks of the designated installation month (S2 Table). Solar-power installations were delayed because of technical problems with the initial solar installations (in hospitals H1, H2, and H3) requiring rectification, system redesign, and change of provider. This delayed subsequent solar installations and required temporary use of alternative power sources (mains power and/or generators). Despite this setback, all intervention hospitals had continuous access to oxygen therapy from the newly installed systems throughout the intervention period.

The project team visited each hospital at least once every 3 months and had regular communication by phone and email. Hospital staff generally reported equipment problems promptly, and our project team provided prompt assistance (including repair or replacement), ensuring patients had continuous access to oxygen therapy.

Hospitals took up some, but not all, aspects of the project’s support for systems. Most hospitals started multidisciplinary ‘oxygen team’ meetings. Few hospitals improved their medical record keeping. All hospitals continued to charge patients for oxygen, and most did not adopt measures to reduce costs to patients until late 2017. During the study period, Nigeria experienced an economic recession, resulting in reduced income for hospitals, unpaid healthcare worker wages, and associated industrial action [46]. Although hospital staff and administrators insisted that these financial pressures did not impede patient access to oxygen, we received anecdotal reports of more patients becoming indebted to the hospital and leaving against medical advice.

Intervention effects

Primary clinical outcomes.

Table 2 shows the crude rates and intervention effect sizes for the primary analysis of clinical outcomes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Effect of the full oxygen system intervention on mortality—Primary outcome analysis comparing the full oxygen system intervention to the prior standard of care plus pulse oximetry.

https://doi.org/10.1371/journal.pmed.1002951.t002

Primary analysis suggests that, compared to the pulse oximetry period, the full oxygen system was not associated with death for children overall (aOR 1.06; 95% CI 0.77–1.46; p = 0.721) or children with ALRI (aOR 1.09; 95% CI 0.50–2.41; p = 0.824) and was associated with increased risk of death for neonates overall (aOR 1.45; 95% CI 1.04–2.00; p = 0.026) but not preterm/LBW neonates (aOR 1.30; 95% CI 0.76–2.23; p = 0.366). Severity-adjusted analysis was qualitatively similar to the basic model, with stronger evidence of harm for neonates (Table 2).

At reviewer request, we conducted post hoc analysis on subgroups with hypoxaemia and children under 5 years of age, with qualitatively similar results (S3 Table).

Secondary clinical outcomes and extended analysis of preintervention period.

Table 3 shows the crude rates and intervention effect sizes for the primary and secondary clinical outcomes, including the extended analysis comparing pulse oximetry and full oxygen periods against the preintervention period.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Effect of the intervention(s) on mortality, showing the primary analysis (full oxygen period versus pulse oximetry period) and extended analysis (comparing pulse oximetry and full oxygen system periods to the preintervention period).

https://doi.org/10.1371/journal.pmed.1002951.t003

Compared to the preintervention period, the pulse oximetry and full oxygen system intervention were associated with decreased risk of death for children with ALRI (aOR 0.45, 95% CI 0.17–1.18, p = 0.104; aOR 0.57, 95% CI 0.31–1.04, p = 0.066), with qualitatively similar results from severity-adjusted analysis (aOR 0.33, 95% CI 0.12–0.92, p = 0.035; aOR 0.50, 95% CI 0.26–0.98, p = 0.044). Neither pulse oximetry nor the full oxygen system was associated with death for children overall, neonates overall, or any of the other subgroups, compared to the preexisting standard of care.

Practice outcomes.

Table 4 shows the crude rates and intervention effect sizes for the practice outcomes, including the extended analysis across all three study periods. Use of pulse oximetry in the clinical care of hospitalised children improved dramatically across the study periods (from <5% of admissions to >90%), with lesser change evident for oxygen practices.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Effect of the intervention(s) on practice outcomes, showing the primary analysis (full oxygen period versus pulse oximetry period) and extended analysis (comparing pulse oximetry and full oxygen system periods to the preintervention period).

https://doi.org/10.1371/journal.pmed.1002951.t004

For neonates, there was a statistically nonsignificant increase in overall oxygen use (aOR 1.34; 95% CI 0.96–1.87; p = 0.085) following introduction of the full oxygen system, with no change in oxygen administration for neonates with hypoxaemia (aOR 0.77; 95% CI 0.34–1.75; p = 0.531) compared to the pulse oximetry period. Pulse oximetry had minimal impact on overall oxygen use (aOR 1.29; 95% CI 0.82–2.03; p = 0.264) or the use of oxygen for neonates with signs of hypoxaemia (aOR 1.06; 95% CI 0.54–20.9; p = 0.857) compared to the preintervention period.

For children outside the neonatal period, results suggest that the full oxygen system increased overall oxygen use (aOR 1.89; 95% CI 1.34–2.66; p < 0.001) and improved oxygen administration for those with hypoxaemia (aOR 2.71; 95% CI 1.12–6.56; p = 0.027) compared to the pulse oximetry period. Pulse oximetry had minimal impact on overall oxygen use (aOR 0.72; 95% CI 0.43–1.19; p = 0.200) or the use of oxygen for those with signs of hypoxaemia (aOR 1.00; 95% CI 0.51–1.95; p = 0.995).

Heat-map graphs suggest that pulse oximetry and oxygen practice changes were achieved incrementally over a number of months, with most of this change occurring early in the pulse oximetry period (Fig 2). Of note, the proportion of hypoxaemic (SpO₂ < 90%) children receiving oxygen therapy increased to >75% coverage within 4–8 months of receiving pulse oximeters. Three hospitals are outliers (H9, H12, and H5’s children’s ward), being the only hospitals that genuinely had no oxygen access for children prior to the intervention and showed no improvement in oxygen use until after the full oxygen system was implemented.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Care practice outcome rates (proportions), by step and hospital.

Each cell contains the proportion of patients who received a particular element of care (pulse oximetry or oxygen therapy) on admission. For each of the 10 panels in this figure, the 12 hospitals are represented on the y-axis, whereas the steps of the trial are represented on the x-axis. The colour gradient extends from red (0%) through yellow to green (100%), providing a visual representation of practice change over time. Blue spark-lines show the hospital-specific trend for easy reference. When the denominator to compute the cell rate is 0, cells are coloured in white. Notes: Data from November/December 2013 not available in step 1.

https://doi.org/10.1371/journal.pmed.1002951.g002

Heat-map graphs suggest oxygen use was better targeted at neonates and children with hypoxaemia in the pulse oximetry period, but this is more weakly reflected in the adjusted effect estimates (aOR 2.00, 95% CI 0.80–4.95, p = 0.136 and aOR 1.21, 95% CI 0.55–2.68, p = 0.634 for neonates and children, respectively) (Fig 2, Table 4). The ‘indication for oxygen’ outcome uses a composite measure of hypoxaemia (SpO₂ < 90% on admission, or signs of hypoxaemia if no SpO₂ documented), which may be biased because of the dramatic change in SpO₂ documentation over the study periods (i.e., reflecting patients with signs of hypoxaemia in the preintervention period and patients with SpO₂ < 90% in the latter periods).

Interpreting results

We will now explore the clinical and practice outcomes in light of our program theory and the broader study context to provide a balanced and informed interpretation of results.

Limitations

Using a stepped-wedge design enabled us to gain the benefits of randomisation and learn valuable lessons about implementation along the way, but it did make adjusting for temporal effects harder than if it were a parallel cluster trial. Stepped-wedge designs work best when potential confounders remain relatively stable during the study period, and this was potentially compromised by the major economic and social changes in Nigeria that likely introduced additional confounding.

We used a 2-week ‘wash-in’ period; however, actual practice change took longer, and oxygen use changed less than we had anticipated. A heat map helped visualise this incremental change for practice outcomes, but low death rates make heat-map figures less useful for clinical outcomes.

We included over 24,000 participants in primary analysis; however, the small number of clusters and the unbalanced cluster size limited the power of our study. This was particularly challenging for neonatal data, with the four largest hospitals contributing 84% of the data. The power to detect significant change in mortality was lower than expected because of (1) lower-than-expected baseline CFRs, (2) lower overall admission numbers, and (3) higher-than-expected ICCs. We reported the primary clinical outcome for multiple subgroups and evaluated multiple secondary practice outcomes. We used prespecified analysis methods but recognise the risk of chance findings. Both positive and negative findings should be interpreted with caution.

We used data from clinical documents using an approach similar to others conducting implementation research through clinical research networks [52]. Our use of dedicated research nurses minimised the amount of missing data, and our audit of documentation practices prior to starting the study reassured us that documentation practices overall were excellent [13]. We used standardised case definitions but did not impose additional restrictions on healthcare workers regarding their documentation. This could result in diagnostic misclassification or missed signs of severity but is unlikely to have altered our findings substantively.

Practical implications

Our study was part of a pragmatic field trial to better inform efforts to improve oxygen access to children and newborns in Nigeria and other LMICs. We have reported results from our needs assessment and hypoxaemia prevalence estimates previously, showing that hypoxaemia is common in both respiratory and nonrespiratory childhood illnesses and that many hospitals have limited access to pulse oximetry and oxygen therapy [6, 29].

In recognition of this challenge, the Nigerian federal government recently released a national oxygen policy and strategic road map for the scale-up of oxygen services in Nigerian health facilities [53–55]. We hope that emerging findings from our project will help others seeking to improve oxygen access, and we offer the following humble suggestions.

1. Pulse oximetry is key to improving oxygen use and likely improves other aspects of inpatient care (e.g., assessment and monitoring, confidence and motivation among healthcare workers, communication and trust with caregivers). As a relatively affordable and potentially life-saving practice, pulse oximetry should be central to all efforts to improve oxygen systems and paediatric hospital care—including as a core vital sign in assessing all acutely unwell children and newborns.
2. Oxygen therapy is part of a system of care, and outcomes are influenced by many other factors. The challenges to oxygen access include many factors that also compromise other aspects of care, including weak equipment maintenance systems, poor power supplies, staff shortages, lack of clinical guidelines, and challenges of interdisciplinary cooperation. These challenges provide opportunities to use oxygen access as a means to reveal systemic weaknesses and incrementally improve the broader hospital system for improved patient outcomes.
3. Oxygen, administered as low-flow oxygen or with continuous positive airway pressure (CPAP), is an important part of neonatal care. We advise readers against making changes to neonatal oxygen policies or practices on the basis of this study and to adhere to WHO guidelines [32]. Oxygen, like any medication, should be given judiciously, accompanied by treatment of the specific medical condition and good supportive care. Future research and quality-improvement activities should seek to better understand how oxygen can be used most safely and effectively in neonatal care—particularly for the most at-risk preterm neonates.

Conclusion

For children, there is consistent evidence across primary and secondary analyses that the introduction of pulse oximetry improved oxygen practices in Nigerian hospitals and may have reduced ALRI mortality, with no additional clinical benefit from the full oxygen system. This finding is plausible and consistent with previous studies and may indicate that pulse oximetry improves the use of existing oxygen supplies and encourages hospitals to mobilise additional oxygen supplies for paediatric use (and may improve other aspects of care).

For neonates, the interpretation is more challenging. On one hand, there was suggestion of harm for neonates when comparing the full oxygen system period to the pulse oximetry period, without a clear explanation. On the other hand, we found no effect of the improved oxygen system on neonatal outcomes compared to baseline and negligible change to oxygen use for neonates.