Prognostic score to predict mortality during TB treatment in TB/HIV co-infected patients

Background Estimating mortality risk during TB treatment in HIV co-infected patients is challenging for health professionals, especially in a low TB prevalence population, due to the lack of a standardized prognostic system. The current study aimed to develop and validate a simple mortality prognostic scoring system for TB/HIV co-infected patients. Methods Using data from the CDC’s Tuberculosis Genotyping Information Management System of TB patients in Texas reported from 01/2010 through 12/2016, age ≥15 years, HIV(+), and outcome being “completed” or “died”, we developed and internally validated a mortality prognostic score using multiple logistic regression. Model discrimination was determined by the area under the receiver operating characteristic (ROC) curve (AUC). The model’s good calibration was determined by a non-significant Hosmer-Lemeshow’s goodness of fit test. Results Among the 450 patients included in the analysis, 57 (12.7%) died during TB treatment. The final prognostic score used six characteristics (age, residence in long-term care facility, meningeal TB, chest x-ray, culture positive, and culture not converted/unknown), which are routinely collected by TB programs. Prognostic scores were categorized into three groups that predicted mortality: low-risk (<20 points), medium-risk (20–25 points) and high-risk (>25 points). The model had good discrimination and calibration (AUC = 0.82; 0.80 in bootstrap validation), and a non-significant Hosmer-Lemeshow test p = 0.71. Conclusion Our simple validated mortality prognostic scoring system can be a practical tool for health professionals in identifying TB/HIV co-infected patients with high mortality risk.


Introduction
Tuberculosis (TB) is a leading cause of morbidity and mortality in HIV-infected individuals.In 2016, there were an estimated one million new TB cases amongst people who were HIVpositive worldwide with 374,000 deaths [1].As one of the four states (California, Texas, New York, and Florida) that accounted for 50.6% of the national total cases in the United States (U.S.) [2], Texas had 9,007 new TB cases and 30,979 HIV-positive individuals reported between 2010 and 2016 [3][4][5].In 2015, Texas had a TB incidence of 4.9 per 100,000 population, 5.1% of these new TB patients were HIV positive [4].In Texas, TB disease has been identified as a communicable disease having the highest standardized mortality ratio (SMR) relative to the national reference with 679 deaths between 2001 and 2010 [6].Studies in other settings have identified several prognostic factors associated with mortality in TB/HIV patients such as age!45, smear(+) pulmonary TB, antiretroviral therapy (ART), having initial TB regimen with rifamycin, isoniazid and pyrazinamide, drug susceptibility testing (DST), and CD cell count [7,8].Podlekareva et al. recently presented a health care index (HCI) score with components selected from commonly-used interventions and suggested its association with the outcome in HIV-positive patients [8].However, the HCI components were subjectively selected and not the result of objective multivariate regression modeling or statistical analyses.Although some TB mortality risk scores have been developed, they either do not include HIVinfected patients, have small sample size, or were developed based on single hospital to predict in-hospital mortality, or include variables that are not routinely available at community TB programs [9][10][11].In light of this, making an accurate prognosis for mortality risk during TB treatment in HIV co-infected patients is challenging for health professionals, especially in a low TB prevalence population, due to the lack of a standardized prognostic system.The current study aimed to develop and validate a simple prognostic scoring system using populationbased surveillance data, which are routinely collected by most TB programs and could predict patient mortality risk during TB treatment.The proposed mortality risk score could be a practical tool for TB clinicians and other health professionals in managing TB disease in patients with TB/HIV co-infection.

Study population
The study used retrospective de-identified surveillance data of all confirmed TB patients from the state of Texas (U.S.) reported to the National TB Surveillance System (NTSS).The dataset was downloaded from the Centers for Disease Control and Prevention (CDC) supported TB Genotyping Information Management System (TBGIMS) website.The inclusion criteria were defined as: (1) confirmed TB cases in the state of Texas from 01/2010 through 12/2016 (based on the date of "caseness" when the case was verified by the Texas Department of State Health Services included in the state's official case count) [12]; (2) age!15 years old; (3) positive HIV status; and (4) had documented TB treatment outcome in the dataset as either treatment completed ("completed") or dead ("died").As the dataset has only one pediatric TB/HIV coinfected patient, this patient was not included in this study.Given the main purpose of our study is to predict the mortality during TB treatment in HIV-infected patients against the treatment completion, patients who had an outcome coding other than "completed" or "died" (such as "adverse", "lost", "moved", "other", "refused", or "unknown"), i.e. vital status could not be verified, and had a negative or unknown HIV status were also excluded from the analyses.A confirmed TB case in the dataset is defined as either a laboratory confirmed case or a clinical confirmed case, which was identified and verified by the local and state TB program staff using the CDC's TB case definition [12].A patient with a Mycobacterium tuberculosis (Mtb) culture conversion was defined as a patient who had an initial positive sputum culture that converted to a documented negative culture without converting back to positive culture during the entire treatment course.Unknown conversion status was defined as a patient who had an initial Mtb positive sputum culture and completed TB treatment but the results of all follow-up cultures are not available.Abnormalities consistent with TB disease on chest radiograph (TB-CXR) were recorded in the dataset as a binary variable (normal versus abnormal) [13].

Ethics statement
As this was a retrospective study using de-identified data, ethical approval was not required.

Statistical analysis
Demographic and clinical data were reported as frequencies and proportions.Differences in demographic and clinical characteristics between the excluded and included patient population pools were determined using the Chi-square or Fisher's exact tests, as appropriate.Missing data were assessed for missing completely at random (MCAR) and covariate-dependent missingness (CDM) using the Little's chi-squared test [14].Univariate and multiple logistic regression models were used to determine the contribution of potential prognostic variables to the patient outcome.Variables for multiple logistic regression models were selected using the Bayesian model averaging (BMA) method [15,16].Briefly, Stata's BMA program was run to evaluate possible model sets from all variables having a p-value of <0.2 in the univariate analysis or variables deemed as clinically important.The BMA program suggested good models which included the variables with a high probability of being a risk factor.The Likelihood Ratio test was used to further reduce the model subsets.The best model was selected based on the small Bayesian information criterion (BIC).Significant risk factors were assigned weighted-points that were proportional to their β regression coefficient values.A prognostic score was calculated for each individual patient in the cohort.Patients were categorized in deciles of risk score and then collapsed into three groups which were significantly distinct in predictive risk for mortality (low, medium and high risk).Model discrimination was determined by the area under the receiver operating characteristic (ROC) curve (AUC).The model's good calibration was determined by a non-significant Hosmer-Lemeshow's goodness of fit test.Model validation was performed using the bootstrap resampling method with 2000 replications.All the analyses were performed using Stata version 14.2 (StataCorp LP, College Station, TX, USA).A p value of <0.05 was considered statistically significant.Findings of this study were reported according to the SRTOBE guidelines (Strengthening the Reporting of Observational Studies in Epidemiology [17].

Characteristics of the study sample
From January 2010 through December 2016, 569 HIV-infected adults in the state of Texas were confirmed with TB disease and reported in the National TB Surveillance System database.A total of 450 patients, including 57 individuals who died (12.7%) were used in the analysis after 119 patients with an outcome other than "completed" or "died" were excluded (Fig 1).There were no significant differences between characteristics of the excluded and included groups (Table 1).Data of the 434 included patients were used in the development and internal validation of the mortality prognostic scoring system.

Development of the mortality prognostic score system
The crude associations between potential risk factors and mortality were examined using univariate logistic regression analyses (Table 2).The variable selection process using the Bayesian model averaging method suggested seven variables with prognostic significance for further investigation in the final multiple logistic regression model: age group, homelessness, resident of long-term care facility, meningeal TB, TB-CXR, TB diagnosis confirmed by positive culture or Nucleic Acid Amplification (NAA), Mtb culture without conversion or unknown conversion status during the TB treatment.Six variables were used in the development of the TB mortality risk score, all except for the homelessness variable, which was not significant in the final model.Weighted points were assigned to each of the final six risk factors using the linear transformation of the corresponding regression coefficient [(divided by the smallest β coefficient (1.07, age), multiplied by a constant (5), and rounded to the nearest integer, (Table 3)].
A prognostic score was calculated for individual patients based on the following formula: All variables were binary with "No" = 0 and "Yes" = 1.Patients were divided into three groups that were significantly distinct in predictive risk for mortality: low-risk group (<20 points), medium-risk group (20-25 points), and high-risk group (>25 points).The mortality in low-, medium-, and high-risk groups were 2.6%, 11.9% and 44.4%, respectively (Table 4, Fig 2).The predicted probability of death during TB treatment can be calculated from the intercept (-6.994499) of the final model and corresponding regression coefficients of the variables included in the risk score based on the following formula:

Performance and validation of the prognostic score
The final model had good discrimination in the development (AUC = 0.82; 95% CI 0.76, 0.89) and bootstrap validation (AUC = 0.80; 95% CI 0.72, 0.88) (Table 4, Fig 2).The ROC analysis using the prognostic score itself also provided good discrimination in both the development and bootstrap validation (AUC = 0.82; 95% CI 0.76, 0.89 and AUC = 0.79; 95% CI 0.70, 0.87, respectively).The prognostic model had good calibration with a non-significant Hosmer- Comparisons of mortality between risk groups were conducted using Chi-square test.
Ã Overall p-value.A p<0.001 was also found for all pairwise comparisons among groups (i.e.low-risk vs. medium-risk, low-risk vs. high-risk and medium-risk vs. highrisk groups); a non-significant Hosmer-Lemeshow goodness of fit test indicates good calibration; Brier score: ranged 0-1, the smaller the score, the better performance.Lemeshow chi-square of 3.74 (p = 0.71) and excellent overall performance with a Brier score of 0.09 (Table 4).Shrinkage statistic calculated using the repeated 10-fold cross-validation (250 replications) indicated an in-sample shrinkage of 1.4% (standard error 1.8).This result together with a non-significant Hosmer-Lemeshow goodness-of-fit test suggested the model fit well with the data.Compared with low-risk patients, patients in the medium-and high-risk groups had significantly higher odds of mortality during TB treatment (Table 5).Given that the multivariate analysis requires non-missing data for all the included variables for each patient, a sample of 434/450 (96.4%) patients having complete data for all six included variables were used in the final model and in the development of the scoring system.The comparison between 434 patients who were included in the final model versus 16 excluded patients due to incomplete data found no significant difference neither in the mortality (12.4% versus 18.8%, p = 0.46) nor in all demographic and clinical characteristics (S1 Table ).Little's chisquared test for MCAR and CDM had non-significant p-values (0.13 and 0.44, respectively), which suggest that the missing values could be completely at random and do not influence on the outcome.

Online calculator application
We have created a free online application for our risk score calculator, which can be used on both android and iOS mobile devices.The calculator can be downloaded from the following link https://oaa.app.link/i0oYeyKsTK(registration for a free OpenAsApp account is required to access the calculator).The calculator provides a risk score (in points), risk group (low, medium or high), and probability of death (%) for an individual patient.

Discussion
In this study, we developed and internally validated a simple prognostic scoring system to predict the mortality risk during TB treatment for TB/HIV co-infected patients in an area having low TB incidence (4.9/100,000) [4].Our prognostic score was developed using populationbased surveillance data in an exclusively HIV-infected population in a low TB-burden setting.Using only six variables, which are routinely collected in TB programs, our mortality predictive model achieves excellent discrimination and good calibration and therefore, can provide clinicians and public health professionals with more information regarding the patient's risk of death during their treatment for TB disease.In order to enhance the practical implementation of the scoring system and to help allocate appropriate treatments and follow-up resources, we categorized patients into three distinctive risk groups.High-risk patients would need the most attention with urgent treatment and more aggressive medical support.Medium-risk patients could benefit from closer follow-up and prompt intervention if needed to prevent them from falling into aggravated health conditions.Low-risk group should be treated and managed as per routine protocols.Multiple approaches can be implemented to reduce a patient's mortality risk.For example, early combination antiretroviral therapy (cART) could be considered for high-risk patients even though their CD4+ level !50 cells/mm3 as the cART has been shown to reduce up to 68% TB-related deaths in TB/HIV co-infected patients [18].For individuals living in long-term care facilities, more aggressive nutritional support would be needed to improve the patient survival as these patients also often have other conditions that may increase the risk for TB mortality such as old age, poor living conditions, malnutrition and the presence of other comorbidities [19].Educational sections for patients and their families toward managing the patient's increased risk of mortality would need to be conducted to enhance the treatment adherence, improve the patient's nutrition condition, and provide the knowledge of how to seek medical assistant when needed.
In our model, being a resident in a long-term care facility appeared to be the strongest predictor of mortality.Older age, poor nutrition condition, presence of other comorbidities and lack of family support could contribute to the morality risk for individuals living in long-term care facilities [20,21].Delay in culture conversion has been suggested to be associated with a poor TB treatment outcome.Potential drug-resistant disease, failure to adhere to the treatment regimen and heavy initial bacillary load are among possible explanations for the adverse outcome [22,23].In our study, patients with TB meningitis had significantly higher odds of mortality, which is consistent with the observation of other authors [24].Patients with TB-CXR (abnormal chest radiograph consistent with TB disease) and Mtb positive culture or NAA results had significantly higher mortality rates, nearly four times and seven times the odds for death compared with patients who had normal chest radiograph or negative cultures.It is possible that patients with TB-CXR and positive cultures may have a higher Mtb bacillary load and more disseminated lesions, which may increase the risk for death.A similar result has been described by Christensen et al. in their study in which pulmonary TB patients had an almost two-fold increased long-term mortality than extrapulmonary TB patients [25].Although both TB-CXR and cavitation on CXR were evaluated in the initial multivariate model, only TB-CXR was significant.Additionally, TB/HIV co-infected patients may have a wide variety of radiographic findings rather than just cavitation [26].Therefore, TB-CXR was included in the final model.The association between older age and worse outcome has also been observed by other authors [27,28].Being significant in the univariate analysis, the diabetes and chronic kidney disease variables were evaluated in the initial multiple logistic regression model.However, these variables were not significant in multivariate analysis.Additionally, the model without diabetes nor chronic kidney disease had the same diagnostic performance as the model with these two variables included as confirmed by a non-significant Likelihood Ratio test result.Therefore, diabetes and chronic kidney disease were not included in our final model.
Our study has some limitations.First, the analysis excluded 119 (20.9%) out of 569 patients who had treatment outcome coded other than "completed" or "died".While this exclusion may be prone to misclassification bias, the similarity in demographic and clinical characteristics between the excluded and included patients suggested that potential misclassification, if any, was minimal.Second, although antiretroviral therapy (ART) has been suggested as being strongly associated with the mortality reduction in TB/HIV co-infected individuals [7,29], ART use and some other important prognostic factors such as HIV viral load, CD4 cell count, time of death and cause of death are not available in the NTSS data, and thus prevented us from developing an even more robust model for this population.Assuming the majority of our TB/HIV co-infected patients received ART given the extensive HIV management programs in the U.S. and as our model was developed and validated in a low TB prevalence country, external validation of our prognostic score in different populations would be needed.Although patients in the high-risk group had more than 30 time the odds of death (OR 30.24, 95% CI 10.93, 83.66) compared with patients in the low-risk group, this finding should be interpreted cautiously given the wide confidence interval, which may be due to the small number of high-risk patients.Lastly, given that our study sample only had one patient with multi-drug resistant TB (MDR-TB), this variable was not examined in the multivariate analysis.Given MDR-TB is known high-risk factor for a high risk for death, external validation or updating of the model with the inclusion of MDR-TB as a variable using data from populations with a high proportion of MDR-TB should be conducted.Lastly, given the nature of surveillance data where certain self-reported information was originally obtained from interviewing TB patients, the possibility of recall bias cannot be completely ruled out.
In spite of the limitations, our study has many strengths such as the data were obtained from a state-wide population-based surveillance program over a 7-year time period, the variables used are routinely collected by any TB program, and the model has good discrimination and calibration in both development and validation.While the mortality in all TB disease patients (both positive and negative HIV status) in Texas was 5.0% (66/1334) in 2015 [4], the significantly higher mortality rate in HIV-positive TB patients found in this study (12.7%, 57/ 540) also raises questions regarding the need for having better management strategies for this high-risk group of patients.In addition, with the availability of a free and convenient calculator app, which can be accessed from any android and iOS devices, clinicians and health professionals can easily use our scoring system in their daily practice in order to facilitate their decision-making process.

Conclusion
The present study developed and internally validated a simple and practical prognostic scoring system using the population-based surveillance data to predict mortality during TB treatment in TB/HIV co-infected patients.This TB mortality scoring system could help identify TB/HIV co-infected patients who have an increased risk of mortality.External validation of our risk score system using the provided formula in similar settings of low TB/HIV burden would be necessary.

Table 1 . Demographic and clinical characteristics of the study population compared with those not included in the study. Included (N = 450) Excluded (N = 119) P Value Ã
Note: Values are in number and % unless otherwise specified.Ã differences across groups were compared using the Chi-square or Fisher's exact tests, as appropriate.† TB-CXR: TB-specific abnormalities on chest radiograph.‡ NAA: Nucleic Acid Amplification.