Study design

This retrospective cohort study uses anonymized patient data collected between December 14, 2014 and November 15, 2015 at the GOAL ETC in Port Loko, Sierra Leone. Data comprised patient demographics, geographic location, clinical signs and symptoms, and laboratory results (for malaria infection and semi-quantitative Ebola viremia), as well as the final patient outcome of death or survival. We evaluate the potential of clinical characteristics in predicting EVD mortality and use these results to construct a symptom-based disease staging system, which corresponds to the prognostic power of the most prevalent symptoms.

Patient referral

The ETC was run by the humanitarian organization GOAL Global in cooperation with the Sierra Leonean Ministry of Health and Sanitation (MoHS). The ETC opened in December 2014 and accepted 600 patients from a catchment area spanning 200km. EVD surveillance in Sierra Leone was implemented through District Ebola Response Centres (DERCs). Individuals who were sick were encouraged to report their illness (or the suspected illness of others) via the national or district Ebola call-lines. Individuals that met the WHO case definition for EVD [25], as well as those with confirmed EVD infection, were referred to the ETC from surrounding communities, holding centres, health facilities, and quarantine houses. All EVD(+) patients were treated according to standard treatment protocols developed by WHO and Médecins Sans Frontières [13, 26]. This included empiric antimalarial treatment (Artesunate and Amodiaquine), broad-spectrum antibiotics, and nutritional supplementation for all patients, as well as oral or intravenous fluid rehydration.

Data collection

Signs and symptoms were recorded daily, on admission and throughout the patients’ length of stay at the ETC. Once triaged, blood was drawn and tested for EVD in on-site laboratories managed by Public Heath England. EVD diagnosis was determined by semi-quantitative reverse transcriptase-PCR (qRT-PCR) as previously described [27]. Briefly, the cycle threshold (Ct) value was used as an inverse proxy for viral load and a cut-off of 40 was used to discriminate between positive and negative values. Patients qualified as EVD(-) and were discharged from the ETC after returning two negative Ebola-specific qRT-PCR tests. Histidine-rich protein-II (HRP-II) antigen rapid diagnostic kits were used for testing of malaria infections, which were performed on admission at the ETC.

Signs and symptoms

Symptoms were reported by the patient during a comprehensive questionnaire by trained staff. Haemorrhaging, pyrexia, and disorientation were recorded by clinicians after examination. Haemorrhagic signs included visible blood loss such as hematochezia, hematemesis, haematuria, epistaxis, haemoptysis or persistent haemorrhage from an IV catheter site as well as subcutaneous haemorrhage such as purpura and petechiae. Pyrexia was defined as a body temperature over 38°C, measured using an infrared thermal sensor. Disorientation was measured by trained ETC clinicians as per the AVPU alertness scale (where pain and unconsciousness were considered “disorientated”). Additionally, any specific mention of “confusion” or “disorientation” in the medical notes was also considered a positive for this variable.

Cohorts and inclusion criteria

Of the 600 patients admitted to the ETC, 10 were declared dead on arrival and 24 were classified as late transfers from other ETCs or holding centres (treated elsewhere and thus convalescent on arrival) or had incomplete data. Thus, a total of 34 patients were excluded from this analysis. Of the 566 patients involved in the study, 100% had diagnostic test results for EVD, where 27.9% tested EVD(+) (n = 158). 543/566 patients had malaria test results, of which 34.6% were malaria(+) (n = 188). The cohort was evaluated for missing values in each variable. Referral time (the time in days from symptom onset to admission at the ETC) had 20 cases of missing data. Further analysis was undertaken to evaluate the aetiology of missingness, which included demographic variables (such as age and sex), clinical severity variables (such as EVD viral load) as well as the covariates used in the final scoring model. Here, we found that subjects with missing data did not differ systematically from those with observed referral time, which is in favour of the hypothesis that the data were missing completely at random. In addition, we performed a sensitivity analysis using the “Hotdeck” imputation technique, which showed that the model coefficients did not change when using complete data [28]. The patient catchment area and mortality rates can be visualised in S1 Fig.

Data entry

Ethical approval for this research was granted by the Sierra Leone Ethics and Scientific Review Committee (SLESRC). To maximize data fidelity, patient files were entered into a secure Microsoft Excel database and crosschecked by 3 independent and trained analysts. Entry of clinical data was overseen by members of the clinical ETC staff. Graphs were constructed using Graphpad Prism, version 6. Univariate and multivariate analysis was conducted using STATA software, version 14 (StataCorp). Score validation was performed using “RMS” R-Package (R Development Core Team. ISBN 3-900051-07-0, URL: http://www.R-project.org). Results were deemed statistically significant at a p-value of less than 0.05.

Primary data Analysis

Epidemiological data and symptoms were summarized by their frequencies and percentages. Univariate logistic regression was performed to assess the association between each predictor and the outcome of death (reported as Odds-Ratios (OR) and p-values). Potential interactions were tested (such as the effect of sex, age, referral time and Ebola contact). The functional relation between the outcome of death and continuous variables (age, days admitted, referral time and EVD viral load) were checked using a fractional polynomial model. The linearity assumption was confirmed for days admitted and referral time but not for age (S2 Fig) or Ct Value (S3 Fig). To simplify the prognostic score, age was coded into three categories: (1) <5 years + [25–45] years, (2) [5–25] years, and (3) >45 years. S2 Fig shows the rationale for the chosen categories on their polynomial curve. The 5–25 group is used as a reference, being the lowest risk group. Comparing the <5 years category to the reference, we obtained an OR of 5.35 (p = 0.006), while the 25–45 category returned an OR of 2.61 (p = 0.002). Comparisons between the 0–5 and 25–45 groups, however, showed that they were not significantly different (p = 0.2) and they could thus be grouped to simplify the user interface of the score. The area under the ROC curves for the scoring systems presented in this study were not statistically different when comparing these age categorisations with the polynomial function of age as a continuous variable. However, a continuous function would undoubtedly be more accurate on a larger sample size.

For PCR results, a Ct value lower than 20 cycles was categorized as “high viral load” and correlated to the natural threshold for the probability of fatal outcome (S3 Fig, dotted line). As there was an insufficient number of patients in the survival group (death = 96, survival = 62) compared to the number of 31 potential predictors, only the predictors associated to the outcome at level of p<0.20 were considered into a Stepwise Backward selection procedure to fit a multivariable logistic regression model [29]. The “daily” score for calculating risk after triage would ideally be handled with a time-dependent model in order to limit immortal time bias. However, these were not a good fit for our data, as the proportionality-hazards assumption was violated by non-parallel lines between categorical variables on log-log plots. Thus, a logistic multivariate model was privileged. Model diagnostics were performed to check for influential observations that impact coefficient estimates and a Hosmer-Lemeshow goodness-of-fit test was performed to assess calibration. Discriminative performance of the final model was assessed by calculating the Area Under the Receiver Operating Characteristics (ROC) Curve (AUC) and its 95% confidence interval.

Calculation of the prognostic scores and model validation

The β-coefficient = log(OR) of each covariate of the final model was converted into an integer-based point-scoring system. The score was then derived as the sum of the covariates’ weighted scores. Internal validation using the bootstrap method (repeated 1000 times) as described in Harrell et al [30] was used to provide a more accurate estimate of the performance of the original model (model based score: AUC_original). The algorithm calculates the optimism of the predictive discrimination in the original model. The difference (AUCoriginal−optimism) gives the bootstrap-corrected (i.e. internally validated) performance of the original model. As described in Steyerberg et al [31], bootstrapping has unavoidable limitations in small cohorts with a large number of predictors.

Role of the funding source

The funder had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to the data in the study and had final responsibility for the decision to submit for publication.

1. Epidemiological characteristics of EVD outcome

Of the 566 patients included in this study, 27.9% tested positive for EVD (n = 158). The crude mortality rate among EVD(+) patients was 60.8% (Fig 1A). Mortality rates were slighter higher in males (68.4% vs. 53.7%), with a statistically insignificant 1.9 fold increase in odds of death (p = 0.06) (Fig 1B). EVD survivors were on average 10 years younger than those who died (24.9 years vs. 34.3 years, p = 0.014) (Fig 1C). However, mean age of death did not differ among EVD(-) patients who died before being transferred out of the ETC (Fig 1C) or among genders (p>0.05). In general, case fatality rates for EVD were higher at the youngest and oldest extremes of age. The patient group aged between 5 and 24 years had the lowest mortality rate of 42.5%, which was significantly lower than other age groups. The over-45’s and under-5’s were particularly vulnerable, being 11.6 and 5.4 fold more likely to die, respectively (Fig 1D). Age groups were selected in order to ensure the mathematic simplicity of the final score. The polynomial curve of this continuous variable is shown in S2 Fig. Categorisation did not significantly alter the accuracy of the final scores (p>0.05).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Epidemiological characteristics of EVD outcome.

(A) Kaplan-Meier survival analysis of patients in the ETC according to their EVD status. (B) Mortality among EVD(-) and EVD(+) admissions according to gender. (C) Average age of death among EVD(-) and EVD(+) patients. (D) Mortality rate across age groups in EVD(-) and EVD(+) cohorts. Dotted lines represent the average mortality rate across all ages in the cohort. Statistics in (C) calculated by unpaired t test *: p<0.05, **: p<0.005, ***: p<0.001, ns: not significant.

https://doi.org/10.1371/journal.pntd.0005265.g001

2. Prognostic potential of clinical characteristics recorded at and after admission

In an effort to better predict the risk of EVD death, we analysed the prevalence and prognostic potential of the major clinical characteristics among EVD(+) patients. Triage symptoms reported by over 50% of fatal EVD(+) patients were asthenia, myalgia, diarrhoea, anorexia, vomiting, pyrexia, and headache (Fig 2A and Table 1). The prevalence of several of triage symptoms was notably different between fatal and non-fatal outcomes, as can be seen by comparing their ranking (Fig 2A) or their differential prevalence (Fig 2B). As expected, high viral load (Ct value <20) was approximately 50% more common among fatal outcomes and univariate analysis revealed it as a major correlate of mortality with 11.8 fold odds of death (p>0.0001) (Table 1). While disorientation on admission was not common in EVD(+) patients (11.4%), when present, it was associated with 94.4% of fatalities in EVD(+) patients, and was therefore the strongest indicator of fatal outcome (OR 13.1, p = 0.014) (Table 1). Other factors showing a statistically significant association with death were diarrhoea, hiccups, myalgia, dyspnoea and conjunctivitis (all p<0.05) (Table 1). While haemorrhagic signs were infrequent on admission (14.6%), developing haemorrhage at any point during admission at the ETC was associated with a 6-fold higher odds of mortality (p>0.0001) (Table 1). Finally, malaria infection was more prevalent in fatal outcomes (Fig 2B) and will be discussed further below.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Prognostic potential of clinical signs and symptoms recorded at and after admission.

(A) Prevalence of clinical characteristics at triage amongst EVD(+) patients who either survived or died, ranked according to the prevalence in fatal outcomes. Rankings from 1–19 are listed above each bar: black for the outcome of death and grey for survival. (B) Differences in symptom prevalence between EVD survivors and those who died. Positive values are more prevalent in fatal outcomes. Negative values are more prevalent in survivors.

https://doi.org/10.1371/journal.pntd.0005265.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Association of clinical and laboratory characteristics to EVD mortality.

https://doi.org/10.1371/journal.pntd.0005265.t001

3. Prognostic value of Ebola virus load (Ct value)

EVD diagnoses were routinely confirmed by qRT-PCR, where the cycle threshold (Ct) value is inversely proportional to the Ebola virus copy number. We used similar parameters presented in other studies to delineate high and low Ct values [14, 32], where a Ct value lower than a threshold of 20 cycles was categorized as “high viral load”. This also correlated to the natural threshold for the probability of fatal outcome in our cohort (S3 Fig). Ct values were available for 144/158 of the EVD(+) cohort (91.1%) and ranged from 13.5 to 37.9. Of these, 39% (n = 57) were classified as having with high viral loads (Fig 3A). The mean Ct value (22.0) did not vary by gender (p>0.05) but was differentially distributed across ages, where each 10 years corresponded to a decrease of 0.4 Ct points (i.e. an increase of viral load) (p = 0.035) (Fig 3B). Finally, the average Ct value for survivors was significantly higher than those who had a fatal outcome (24.9 vs. 20.6, p<0.01) (Fig 3C and 3D), where odds of death were 12.6 times higher for patients with Ct values of less than or equal to 20 (p<0.0001).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Prognostic value of Ebola virus load (Ct value).

(A) Distribution of Ct values¹ for EVD(+) patients considered to have a high viral load (Ct ≤ 20) and low viral load (Ct > 20). (B) Ct value distribution across age in the EVD(+) cohort. The red line plots the fractional polynomial prediction of the Ct value. (C) Ct values amongst survivors and fatalities in the EVD(+) cohort. (D) Kaplan-Meier survival analysis of EVD(+) patients according their Ebola virus loads, either considered as high viral (Ct ≤ 20) or low viral load (Ct > 20). ¹ Ct values represent Ebola-specific qRT-PCR results (inversely proportional to the viral load). Statistics in (C) calculated by unpaired t test *: p<0.05, **: p<0.005, ***: p<0.001, ns: not significant.

https://doi.org/10.1371/journal.pntd.0005265.g003

4. Impact of EVD referral sensitivity and admission time on outcome

An EVD staging model developed by the UK Defense Medical Services in reference to WHO guidelines on the pathogenesis of haemorrhagic fever, divides the temporal evolution of EVD into three symptomatic stages [13, 14]. Here, the “early” stage is comprised of non-specific symptoms lasting three days. Considering this timeline, we investigated the impact of early referral on mortality. Crude analysis showed that a fatal outcome was not associated to a later referral time. Oppositely, EVD survivors presented at the ETC one day later than those who died (4.6 vs. 3.6 days) albeit a statistically insignificant trend (p = 0.12). Indeed, those presenting within 3 days of symptom onset had a 15% higher mortality than those presenting later (p = 0.09) (Fig 4A). This counter-intuitive trend of earlier healthcare seeking behaviour resulting in higher death risk could be theoretically explained as the confounding effect of disease severity. Here, we propose that patients presenting earlier are doing so as they have a more severe acute disease and thus represent a population predisposed to mortality risk. To investigate this hypothesis, we compared viral loads as a proxy for disease severity and found that those presenting earlier had similar viral loads to those presenting later (Fig 4B). This result indicates that early presenters have more severe acute disease. Correcting for viral load as a confounding factor (where comparisons are only made between patients with equal viral loads), we found the more intuitive result that delayed treatment was significantly associated with mortality. Here, the probability of death increased by an average of 12% for each day of delayed treatment during the 1^st week of symptoms (p = 0.012) (Fig 4C) (Table 1). This quantifies the benefits of early health care seeking behaviour.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Impact of EVD referral sensitivity on diagnosis and patient outcome.

(A) Mortality rate for EVD(+) patients who were referred “early” (within 3 days of reported symptom onset) or “late” (after 3 days of symptom onset). (B) Cycle threshold (Ct) values¹ among early and late EVD(+) referrals. (C) Probability of death according the referral time corrected for Ct value. (D) Probability of death according the number of days spent in the ETC. The average number of days spent in the ETC for patients with fatal or survival outcomes are indicated in colour. (E) Mean referral time of EVD(+) patients according to quarantine (Qt) status. ¹ Ct values represent Ebola-specific qRT-PCR results (inversely proportional to the viral load). Statistics in (A, B and E) calculated by unpaired t test *: p<0.05, **: p<0.005, ***: p<0.001, ns: not significant

https://doi.org/10.1371/journal.pntd.0005265.g004

Oppositely, each day spent within the ETC increased the odds of survival by 1.4 fold (p<0.0001) irrespective of viral load (Fig 4D) (Table 1). The average admission duration for EVD(+) survivors was 10.4 days while deaths occurred, on average, within the first 4.2 days of admission (Fig 4D).

Finally, quarantine status upon admission was available for 96% of ETC admissions (n = 551), 25% of which were referred from quarantine houses. EVD(+) patients referred from quarantined homes had an earlier referral time than those not referred from quarantine facilities (3.8 days vs. 4.7 days, p = 0.03) (Fig 4E). This quantifies the potential patient benefit of quarantine in the region, however, we found no difference in mortality by quarantine status (OR = 1.8, p = 0.08).

5. Impact of malaria infection on patient outcome

Of the 543 EVD(+) and EVD(-) patients with a known malaria test, 34.8% tested malaria(+). Among EVD(+) patients, 24% were co-infected with malaria compared to 38% EVD(-) (OR = 2, p = 0.005). The prevalence for malaria infection varied drastically across age categories, where 5 year olds had an over 50% probability of being malaria(+) in both EVD(-) and EVD(+) cohorts (Fig 5A). Despite the WHO standard of care to treat all ETC admissions with Arteminisin Combination Therapy (ACT) upon admission (irrespective of malaria status)[33], EVD(+)/malaria(+) co-infected patients suffered a significantly higher mortality rate compared to EVD alone (74.3% vs. 53.6%, OR = 3.9, p = 0.03 after controlling for age and gender) (Fig 5B).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Impact of malaria co-infection on patient outcome.

(A) Probability of malaria infection across age for EVD(-) and EVD(+) cohorts. (B) Mortality rates among EVD(-) and EVD(+) cohorts according to the presence of malaria infection. (C) Ct value¹ among EVD(+) patients co-infected or not by malaria. (D) Probability of death among EVD(+) patients infected or not by malaria according to EVD Ct value¹. ¹ Ct values represent Ebola-specific qRT-PCR results (inversely proportional to the viral load). Statistics in (C) calculated by unpaired t test. *: p<0.05, **: p<0.005, ***: p<0.001, ns: not significant

https://doi.org/10.1371/journal.pntd.0005265.g005

Ebola viral load was differentially distributed across EVD(+) patients who were co-infected or not with malaria. Co-infected EVD(+)/malaria(+) patients had significantly higher viral loads compared to patients infected with EVD alone (mean Ct = 20.8 vs. 22.3, p<0.01) (Fig 5C). Controlling for viral load, the increased mortality in malaria co-infected EVD(+) patients was abrogated (p = 0.107) (Fig 5D). Taken together, these results reveal a potential pathogenic synergy between the malaria parasite and Ebola virus.

6. Derivation of a prognostic scoring system for EVD outcome

Performing multivariate analysis of the above data, we selected the clinical characteristics most predictive for EVD mortality using data collected at triage or at any time during the patients’ stay in the ETC (Table 1). By stepwise backwards elimination, and prioritizing the most prevalent symptoms, we identified several characteristics which yielded significant predictive values at triage and during admission. Characteristics that were statistically significant predictors of mortality at admission were vulnerable age groups (<5, 25–45 and >45 years), myalgia, disorientation and referral-time (normalised to viral load) (Table 1). Characteristics that were statistically significant predictors of mortality after admission were vulnerable age groups (<5 and >25 years), disorientation and haemorrhage. Oppositely, days spent in the ETC was a significant predictor of survival (OR 1.5-fold for each day, p<0.0001) (Table 1). Despite the strong association of malaria co-infection with fatality in our univariate analysis above, malaria infection was rendered insignificant in our multivariate analysis.

We then calculated weightings for both scores from the predictive coefficients with the aim to find a simplified scoring model using whole integers and calculations limited to subtraction or addition (Table 1). Testing the sensitivity and specificity of these weightings for the prediction of EVD infection, we found that the characteristics yielded an area under the ROC curve (AUC) of 91.4% (CI95%: 87–96%) for discriminating mortality at triage (Fig 6A) and 97.5% (CI95%: 95–99%) for calculations after admission (Fig 6B). The risk category cut-offs for each score are illustrated in Fig 6C and 6D. Each category contains at least 10% of the cohort. The 3 risk cut-offs (Low, Medium and High) were selected based on the linear risk curve (Fig 7), where “Low” and “High” categories represent risk plateaus on the extremes of the risk statistic (Low <7%, and High >98%).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Derivation of prognostic scoring system for EVD outcome.

The sensitivity and specificity of predicting mortality in EVD(+) patients using the scoring system developed with clinical parameters collected (A) at triage or (B) daily during the admission of the patient at the ETC. The area under the receiver-operator characteristic (ROC) curve represents the discriminative power of each score. (C-D) Sensitivity (green) and specificity (blue) according to the score points of (C) the triage score and (D) the daily score. Prevalence of survivors and those with fatal outcome are displayed as bar graphs and risk category cut-offs are shown as vertical lines. (E) Percentage of survivors and patients with fatal outcome classified in each risk category of the triage EVD mortality score and the (F) daily EVD mortality score.

https://doi.org/10.1371/journal.pntd.0005265.g006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Scorecards to extrapolate the Ebola severity risk.

Ebola Virus disease severity risk calculated at (A) triage and (B) after triage. Full page printable templates of these scorecards can be found in the supplementary information (S4 and S5 Figs).

https://doi.org/10.1371/journal.pntd.0005265.g007

Examining the accuracy of the triage mortality score, we found that the “high” risk classification was composed of 99% correctly classified fatal outcomes while the “low” risk category had a less than 10% mortality rate (Fig 6E). After triage, the “high” risk category of the daily score was composed of 98% fatalities compared to less than 1% in the “low” risk category (Fig 6F). An internal validation of the triage and daily scores yielded a final discriminative power of 89.12% and 97.04% respectively (Table 2). As described in Steyerberg et al [31], bootstrapping has unavoidable limitations in small cohorts with a large number of predictors and thus the optimism may be over-estimated. External validation is needed to best test these associations.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Internal validation of EVD prognostic scores.

https://doi.org/10.1371/journal.pntd.0005265.t002

Limitations

All prognostic tools carry the risk of becoming self-fulfilling prophecies if incorrectly used as an indicator for palliation: dooming severely ill patients to death when the score is not reflective of clinical advances. This score is specifically adapted to an Ebola response in resource-constrained settings, where clinical resources achieved a 40% survival rate. As an 80% survival rate was possible among patients in resource rich environments [5], it is clear that the interpretation of the score would need to evolve with anticipated clinical advancements. However, the major asset of this score is not limited to prediction of the binary outcome of death, but rather its use as a proxy for “disease severity” in resource limited environments. Thus, while the outcome of “death” may change with improved treatment options, patients scoring highly on this tool can still be shortlisted for intensive intervention. Additionally, with the exciting potential of machine-learning predictive tools [43], scoring systems such as these can become more durable and evolve with their developing environments, where a future of accurate EVD diagnosis and prognosis is a realistic possibility.

As we found for malaria, EVD is certainly not the only contributing factor to mortality within an ETC, and patients who have lived a lifetime within a poorly resourced health care system very probably have diverse and complex competing risks. This is an unavoidable bias, as accurate secondary diagnostics for co-morbidities were primitive at best for the bulk of the patients. We await retrospective analyses on patient samples that may reveal the presence of other co-infections or confounding genetic/immunologic anomalies.

Finally, external validation is an essential step before the endorsement of any clinical tool. Without external validation, the level of inaccuracy within this cohort cannot be estimated and thus these scoring systems must be used with this caution in mind.

Conclusion

This study identifies several epidemiological and clinical features, which are significantly predictive for the outcome of EVD infection and proposes several highly accurate statistical tools to predict the clinical severity of EVD and aid objective clinical prioritization.

Perspectives

External validation and systematic meta-analyses of the clinical features of EVD are needed to fine-tune the statistical weightings of this score to further improve its accuracy and geographical relevance.