https://orcid.org/0000-0002-0230-0766
Figures
Abstract
Background
Predicting waiting time for a deceased donor kidney transplant can help patients and clinicians to discuss management and contribute to a more efficient use of resources. This study aimed at developing a predictor model to estimate time on a kidney transplant waiting list using a machine learning approach.
Methods
A retrospective cohort study including data of patients registered, between January 1, 2000 and December 31, 2017, in the waiting list of São Paulo State Organ Allocation System (SP-OAS) /Brazil. Data were randomly divided into two groups: 75% for training and 25% for testing. A Cox regression model was fitted with deceased donor transplant as the outcome. Sensitivity analyses were performed using different Cox models. Cox hazard ratios were used to develop the risk-prediction equations.
Results
Of 54,055 records retrieved, 48,153 registries were included in the final analysis. During the study period, approximately 1/3 of the patients were transplanted with a deceased donor. The major characteristics associated with changes in the likelihood of transplantation were age, subregion, cPRA, and frequency of HLA-DR, -B and -A. The model developed was able to predict waiting time with good agreement in internal validation (c-index = 0.70).
Conclusion
The kidney transplant waiting time calculator developed shows good predictive performance and provides information that may be valuable in assisting candidates and their providers. Moreover, it can significantly improve the use of economic resources and the management of patient care before transplant.
Citation: Sapiertein Silva JF, Ferreira GF, Perosa M, Nga HS, de Andrade LGM (2021) A machine learning prediction model for waiting time to kidney transplant. PLoS ONE 16(5): e0252069. https://doi.org/10.1371/journal.pone.0252069
Editor: Stanislaw Stepkowski, University of Toledo, UNITED STATES
Received: March 17, 2021; Accepted: May 9, 2021; Published: May 20, 2021
Copyright: © 2021 Sapiertein Silva et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data are held in a public repository: https://www.kaggle.com/gustavomodelli/waitlist-kidney-brazil.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Brazil ranks first in number of kidney transplants in the world performed by Health Public System [1]. By July 2020, over 30,000 Brazilians were on waiting lists for a kidney transplant [2]. Although approximately 5,000 deceased donor kidney transplants take place yearly in the country, the supply of organs does not meet demand, and the gap is growing [2, 3]. As a result, recurrent tests and procedures are necessary every 2–3 years to maintain patients on active transplant list [4]. Since this poses a significant economic burden to the healthcare system, predicting a patient’s waiting time can help planning for pre-transplant evaluation, and thus promote a more efficient use of resources in countries such as Brazil [5]. For example, full pretransplant evaluation would be carried out at registration in candidates with high chances of being transplanted, while those less likely to be transplanted on the short term would undergo only the most necessary tests at registration and have their evaluation completed 6–8 hours before surgery. Moreover, estimating waiting time on the transplant list can help identifying the underprivileged, and thus impact allocation score, bringing more equity to transplantation programs [6].
The time deceased donor recipients spend on a waiting list, has been typically expressed as the median waiting time [7]. However, it does not convey the risks of death or removal from the waiting list.
Hart et al. [8] have used a competing risk model to develop a waitlist outcome calculator that demonstrates the probability of outcomes on the waiting list, including waiting time. However, their model reportedly requires updating and the online version of their calculator is still not available online.
The machine learning approach can contribute to the development of robust predictive models [5]. It includes both conventional statistical analyses, as well as linear and logistic regression and non-linear models, such as decision trees, neural network, nearest neighbors, and support vector machine, which can capture non-linear relationships. Machine Learning uses an approach based on steps: obtaining the data, excluding unusual data ("outliers"), selecting variables, model train, and validation. These approaches allow fitting algorithms capable of making predictions [9–11]. Thus, this study aimed at identifying the relevant predictors and combine them into a predictor model to estimate time on a kidney transplant waiting list using machine learning.
Methods
Study design and population
This retrospective cohort study included data of patients registered, between January 1, 2000 and December 31, 2017, in the waiting list of São Paulo State Organ Allocation System (SP-OAS) /Brazil. SP-OAS has a database that holds over 10 years of information and provides a good sample of the Brazilian transplant population. Indeed, of 5,923 kidney transplants performed in the country in 2018, one third of them (2,095) were carried out in the state.
SP-OAS adopts a policy of regional allocation, centralized and controlled organ distribution, and decentralized organ procurement and harvesting. SP-OAS serves as the state´s organ transplantation system and operates a single database of the entire transplant population in the state, but patients from the various transplant groups and dialysis centers are divided into 4 sub-regional waiting lists (FUNDERP; UNICAMP; UNIFESP; and HCFMUSP) according to location [12].
This study was approved by the Research Ethics Committee of the School of Medicine of Botucatu—UNESP (# 3.094.616; CAAE: 03660718.2.0000.5411). Informed consent was not required because the data were analyzed anonymously.
Eligibility criteria
All patients registered during the study period in the SP-OAS waiting list for a kidney-alone transplantation were included.
Living-donor kidney transplant recipients and prioritized patients were excluded. Patients with missing data were also excluded (n = 51 missing subregional information).
Allocation criteria
Allocation was performed as established by the National Transplantation System of the Brazilian Ministry of Health [12, 13]. For deceased donor transplants, allocation criteria are based on HLA matching (highest number of points for HLA DR, followed by HLA B and HLA A), recipient’s age (<18 years), date of registration on the waiting list, and panel reactive antibody (PRA). A point score system based on blood group and HLA match is used as follows:
- DR: 0 MM = 10 points; 1 MM = 5 points; 2 MM = 0 point;
- B: 0 MM = 4 points; 1 MM = 2 points; 2 MM = 0 point;
- A: 0 MM = 1 point; 1 MM = 0.5 point; 2 MM = 0 point.
Waiting time, allosensitization (cPRA >50), diabetes mellitus, and age < 18 years served as tiebreakers.
Predictors
The following variables were evaluated as predictors: age, sex, race, comorbidities, time on dialysis, blood group, calculated panel class I (cPRA), HLA-A, HLA-B, HLA-DR, number of blood transfusions, pregnancies, previous kidney transplants, and pre-transplant serology for Hepatitis B and C.
HLA frequency.
HLA frequency variables (A, B and DR) were calculated by dividing the number of times the allele of interest is observed in a population by the total number of copies of all the alleles at that genetic locus in the population. HLA frequency in the population of São Paulo state was obtained from the Allele Frequencies net database [14]. To standardize HLA antigen assignments an HLA Dictionary was used [15]. The serological equivalents were listed as expert assigned types.
Outcomes
Deceased donor transplantation was considered as the primary outcome.
Secondary outcomes include death on transplant list, and removal from the waiting list. Reasons for list removal include withdrawn from treatment, kidney function recovery, refusal to undergo transplantation, additional listings for the same person, deteriorating clinical conditions, and transplant performed outside the state.
Statistical analysis
Continuous variables were categorized according to a frequency histogram including- age and cPRA. Not normally distributed variables were transformed; the natural logarithm was applied to time on dialysis. HLA A, B, and DR were analyzed as continuous variables in order to reduce the number of categories. Homozygosis in HLA A, B, and DR were analyzed as independent variables. Variables with near-zero variance were removed from the analysis (Hepatitis C and Chagas’s disease) as they are uninformative predictors and cause algorithm convergence problems at the modeling stage.
Univariate analysis.
Univariate analyses were performed using the chi-square test for categorical variables and the Kurskall-Wallis test for continuous variables, considering transplant as the outcome. The analysis of the predictors associated with transplantation was performed using a Kaplan-Meier model (package survival, R software).
Prediction model.
The dataset was randomly split into two subsets–derivation (training 75%) and internal validation (testing, 25%). A Cox regression model (package survival, software R) was fit with transplantation as the outcome. All predictors were included in the univariate model, and collinear factors were removed. Collinearity was measured using the variance inflation factor (VIF). VIF values > 3 indicated collinearity. Multivariate stepwise Cox regression was performed using the MASS package, with AIC (“Akaike information criterion”) optimization. The lowest AIC indicated the final fitting model, which was used to develop the risk-prediction equations. Plots of scaled Schoenfeld residuals were used to check the proportionality assumption. Approximate proportional hazard effects were found in all cases. The final model was tested using 25% of the data. C statistics were used to assess the accuracy of prediction [16]. In addition, calibration plots were used to evaluate calibration for each model. Confidence intervals were reported according to the method of Louis and Zeger [17]. Sensitivity analyses were performed with Cox models fitted for each subregional list in order to address local variations. Regression analysis was also performed using competitive risk as described by Fine & Gray [18]. This analysis considers removal from the waiting list as a concurrent event (package cmprsk) [19]. Data analyses were performed using software R version 3.4.2.
Results
The total number of cases retrieved was 54,055. After removing all records on living-donor transplants and prioritized patients, 48,153 registries were included in the final analysis. During the study period, 28.4% of the patients were transplanted with a deceased donor (Fig 1). The median waiting time for transplantation was 26.3 months.
Patients who underwent transplantation showed lower median age, shorter time on dialysis, and lower cPRA, as well as higher frequencies of HLA-DR, -B and -A. Transplant frequency was higher in blood groups A and AB, in patients with homozygosity in HLA-A, -B or -DR, and in the FUNDERP and UNICAMP subregions (S1 Table).
Kaplan-Meier models
Kaplan-Meier univariate models show that the likelihood of transplantation was higher: in blood groups A and AB compared to groups B and O (Fig 2A); with zero cPRA compared to other cPRA values (Fig 2B); in patients with HLA-DR heterozygosity compared with those with HLA-DR homozygosity (Fig 2C); individuals under 18 years of age (Fig 2D); patients with positive anti-Hbc antibodies compared to those with negative anti-Hbc antibodies (Fig 2E).
Kaplan-Meier plot showing the Likelihood of transplantation in different groups: A: blood groups; B: cPRA (calculated panel class I) strata, C: HLA-DR homozygossity heterozygosity, D: age strata, E: presence of anti-Hbc (Hepatitis B surface antibody); F: between subregions.
Cox regression model
The Cox model showed that age, subregion, cPRA, and frequency of HLA-DR, -B and -A were associated with changes in likelihood of transplantation (Table 1).
Model validation.
The final multivariate Cox model was trained in 75% of the data (36,115) and tested in 25% of the data (12,038), with a c-index of 0.70 (c-Index) (Fig 3).
Sensitivity analysis
For the analysis of sensitivity, multivariate Cox models were fitted using stepwise selection for each allocation subregion. Considering three data subsets: FUNDERP + UNICAMP, UNIFESP, and USP. FUNDERP and UNICAMP data were combined into a single subset owing to the low number of cases of these subregions. Despite the small coefficient variations observed among subsets, no major differences in the final coefficients were found compared to the full model (S2 Table). C-indexes in the subsets FUNDERP+UNICAMP, USP, and UNIFESP (0.67, 0.69 and 0.68., respectively) were slightly lower than that in the full model.
Competitive risk analysis
A regression model was fitted using competitive risk as described by Fine & Gray15. considering the four different possible outcomes (staying on waiting list, transplant, removal from waiting list, or death). Thus, the patients removed from the waiting list could be re-enrolled and considered as new cases. Notably, the regression coefficients of the competitive risk model were very close to the Cox regression values (S3 Table).
Transplant waiting list calculator
A calculator of kidney transplant waiting time using Cox model hazard ratios can be accessed at: https://gustavomodelli.shinyapps.io/time_list_in_tx/
Discussion
Using the database of the largest state in number of transplants in Brazil, which covers a period of 17 years, we developed and validated a model for predicting waiting time to kidney transplantation with a good internal validation concordance (c-index = 0.70). Approximately 1/3 of the patients were transplanted during the study period and the likelihood of transplantation was greater in the first 50 months. The median transplant time was 26 months, about half of the 48 months reported in the American allocation system [20].
The methodology described here considered major factors that could affect waiting time for a deceased donor kidney transplant. As expected, age >18 years was associated with a greater likelihood of transplant, while age>60 years was associated with a lower chance of transplant, possibly due to the higher frequency of deteriorating clinical conditions seen in this age group [21]. Transplant likelihood was higher in the subregional centers where the number of candidates listed was lower. Expectedly, transplantation likelihood increased with HLA frequency, especially HLA-DR, as observed in the American allocation system, which also uses HLA scores [22]. Another significant factor that raised the probability of transplantation was the presence of positive serology for anti-HBc, which can be explained by the fact that our national allocation system offers organs from positive anti-Hbc donors to recipients who are either positive anti-Hbc or vaccinated (anti-Hbs positive) [13].
A lower likelihood of transplantation was seen in Blood group O as reported by others [23], where the export of blood group O donor kidneys to patients with other blood groups or zero mismatches leads to longer waiting times, to a higher death rate and to accumulation of blood group [23]. A high cPRA (> 80%) was associated with a much lower probability of transplantation (77%) consistently with information provided by the American [22] and French allocation systems [24]. The possibility of transplantation was also decreased among sensitized patients who, despite having higher allocation scores, often showed a positive crossmatch, which explains their reduced chance.
Prediction model
Prognostic scores currently implemented in transplant medicine mostly predict graft loss, renal function, and the likelihood of transplantation [25–27].
Hart et al. [8] also described a calculator for the likelihood of outcomes for kidney transplant candidates, which includes waiting time after listing for a deceased donor kidney transplant. C statistics for their models was 0.64. In this study, we fit a predictive model for the kidney transplant waiting list using clinical variables available at the time of registration on the waiting list. The model was validated in 25% of the data (internal validation), with a c-index of 0.70. Predictive models are considered useful when the C statistic is greater than 0.70, and strong when the C statistic exceeds 0.80. The iBox (integrative box risk prediction score—iBox), a risk prediction score combining demographic, functional, histological, and immunological factors, showed a c-index greater than 0.80 [28]. A similar study in the American population using the SRTR (“Scientific Registry of Transplant Recipients”) database found a c-index ranging from 0.64 to 0.73 [8]. In other fields, such as oncology, prediction results generally range between 0.60 and 0.70 [29]. These results suggest that our estimates are within useful ranges and that the statistical model used here is more accurate and performs better than the method of reporting median time to transplant [8, 21].
Limitations
This study is limited by the data source used. However, it is worth of note that the São Paulo State Organ Allocation System accounts for 50% of the kidney transplant activity in Brazil.
Cox regressions models could potentially overestimate waiting time because “deaths on the waiting-list” and “removal from the waiting list” are censored or not included in the calculation using this approach. Nonetheless, as the competitive risk model coefficients were very close to Cox regression values, the latter were used. Furthermore, competitive risk models are more difficult to be interpreted by non-specialists [24].
Practical applications
Our model provides estimates on waiting time for a deceased kidney transplant using hazard ratio. However, it can also calculate the probability of kidney transplant at a specific time point based on Cox survival time (See S4 Table for 7 simulated cases). Our online calculator for waiting time to kidney transplant can also be made available on mobile phone applications or incorporated into the transplant database itself. A better estimate of waiting time can improve counselling to kidney transplant candidates, help recruitment for trials, and even lead to changes in allocation strategies.
Conclusion
Our calculator of kidney transplant waiting time shows good predictive performance and provides information that may be valuable in assisting candidates and their providers in making informed decisions. As the number of patients on the waiting list grows, predicting the time frame for waiting to a kidney transplant can significantly improve the use of economic resources, as well as the management of patient care before transplant.
Supporting information
S1 Table. Characteristics of the patients on the kidney transplant waiting list between January 1, 2000, and December 31, 2017.
https://doi.org/10.1371/journal.pone.0252069.s001
(DOCX)
S2 Table. Sensitivity analysis of multivariate Cox model for each subregion (FUNDERP + UNICAMP), USP, and UNIFESP.
https://doi.org/10.1371/journal.pone.0252069.s002
(DOCX)
S3 Table. Competitive risk regression model for transplant considering the removal of the list or death as competitive events and the transplant as the outcome.
https://doi.org/10.1371/journal.pone.0252069.s003
(DOCX)
S4 Table. Estimation of the probability of transplantation in simulated cases.
https://doi.org/10.1371/journal.pone.0252069.s004
(DOCX)
References
- 1. Foresto RD, Pestana JOM, Silva HT Jr. Brasil: the leading public kidney transplant program worldwide. Rev Assoc Med Bras (1992). 2020;66(6):708–709. Epub 2020 Jul 20. pmid:32696867.
- 2. Dimensionamento dos transplantes no Brasil e em cada estado (2012–2019). Regist Bras Transplantes. 2019;25(4):1–104. Avaiable from: http://www.abto.org.br/abtov03/Upload/file/RBT/2019/RBT-2019-leitura.pdf
- 3. Medina-Pestana JO, Galante NZ, Tedesco-Silva H Jr, Harada KM, Garcia VD, Abbud-Filho M, et al. Kidney transplantation in Brasil and its geographic disparit. J Bras Nefrol. 2011;33(4):472–84. pmid:22189813
- 4. Kidney Disease: Improving Global Outcomes (KDIGO) Transplant Work Group. KDIGO clinical practice guideline for the care of kidney transplant recipients. Am J Transplant. 2009;9 Suppl 3:S1–155. pmid:19845597
- 5. Wahl B, Cossy-Gantner A, Germann S, Schwalbe NR. Artificial intelligence (AI) and global health: how can AI contribute to health in resource-poor settings? BMJ Glob Health. 2018;3(4):e000798. pmid:30233828
- 6. Gómez EJ, Jungmann S, Lima AS. Resource allocations and disparities in the Brazilian health care system: insights from organ transplantation services. BMC Health Serv Res. 2018;18(1):90. pmid:29415705
- 7.
Nation Kidney Fundation NK. [Cited 2021 March 10]. Avaiable from: https://www.kidney.org/atoz/content/transplant-waitlist.
- 8. Hart A, Salkowski N, Snyder JJ, Israni AK, Kasiske BL. Beyond “Median Waiting Time”: Development and Validation of a Competing Risk Model to Predict Outcomes on the Kidney Transplant Waiting List. Transplantation. 2016;100(7):1564–70. pmid:27286174
- 9. Hannun PGC, Andrade LGM. The future is coming: promising perspectives regarding the use of machine learning in renal transplantation. J Bras Nefrol. 2019;41(2):284–287. pmid:30353909
- 10. Obermeyer Z, Emanuel EJ. Predicting the Future—Big Data, Machine Learning, and Clinical Medicine. N Engl J Med. 2016;375(13):1216–9. pmid:27682033
- 11. Yoo KD, Noh J, Lee H, Kim DK, Lim CS, Kim YH, et al. A Machine Learning Approach Using Survival Statistics to Predict Graft Survival in Kidney Transplant Recipients: A Multicenter Cohort Study. Sci Rep. 2017;7(1):8904. pmid:28827646
- 12.
Resolução SS-114, de 29-09-2014. p. No 191 –DOE de 09/10/14 –Seção 1–22. [Cited 2021 March 10]. Avaiable from: http://ctxses.saude.sp.gov.br/
- 13.
Resolução SS 6, de 8-2-2019. Paulo, Diario Oficial do Estado de São. 2019. [Cited 2021 March 10]. Avaiable from: http://ctxses.saude.sp.gov.br/
- 14. Gonzalez-Galarza FF, McCabe A, Santos EJMD, Jones J, Takeshita L, Ortega-Rivera ND, et al. Allele frequency net database (AFND) 2020 update: gold-standard data classification, open access genotype data and new query tools. Nucleic Acids Res. 2020;48(D1):D783–D788. pmid:31722398
- 15. Holdsworth R, Hurley CK, Marsh SG, Lau M, Noreen HJ, Kempenich JH, et al. The HLA dictionary 2008: a summary of HLA-A, -B, -C, -DRB1/3/4/5, and -DQB1 alleles and their association with serologically defined HLA-A, -B, -C, -DR, and -DQ antigens. Tissue Antigens. 2009;73(2):95–170. pmid:19140825.
- 16. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361–87. pmid:8668867
- 17. Louis TA, Zeger SL. Effective communication of standard errors and confidence intervals. Biostatistics. 2009;10(1):1–2. Epub 2008 Jun 10. pmid:18550565
- 18. Fine JP, Gray RJ. A Proportional Hazards Model for the Subdistribution of a Competing Risk. J Am Stat Assoc. 1999;94:446–509.
- 19. Scrucca L, Santucci A, Aversa F. Competing risk analysis using R: an easy guide for clinicians. Bone Marrow Transplant. 2007;40(4):381–7. Epub 2007 Jun 11. pmid:17563735
- 20. Saran R, Robinson B, Abbott KC, Agodoa LY, Albertus P, Ayanian J, et al. US Renal Data System 2016 Annual Data Report: Epidemiology of Kidney Disease in the United States. Am J Kidney Dis. 2017;69(3 Suppl 1):A7–A8. Erratum in: Am J Kidney Dis. 2017 May;69(5):712. pmid:28236831; PMCID: PMC6605045.
- 21. Pruthi R, Steenkamp R, Feest T. UK Renal Registry 16th annual report: chapter 8 survival and cause of death of UK adult patients on renal replacement therapy in 2012: national and centre-specific analyses. Nephron Clin Pract. 2013;125(1–4):139–69. Epub 2014 Feb 14. pmid:24662172
- 22. Sanfilippo FP, Vaughn WK, Peters TG, Shield CF 3rd, Adams PL, Lorber MI, et al. Factors affecting the waiting time of cadaveric kidney transplant candidates in the United States. JAMA. 1992;267(2):247–52. pmid:1727521
- 23. Glander P, Budde K, Schmidt D, Fuller TF, Giessing M, Neumayer HH, et al. The ’blood group O problem’ in kidney transplantation—time to change? Nephrol Dial Transplant. 2010;25(6):1998–2004. Epub 2010 Jan 25. pmid:20100733
- 24. Trébern-Launay K, Kessler M, Bayat-Makoei S, Quérard AH, Briançon S, Giral M, et al. Horizontal mixture model for competing risks: a method used in waitlisted renal transplant candidates. Eur J Epidemiol. 2018;33(3):275–286. Epub 2017 Oct 30. pmid:29086099
- 25. Mark E, Goldsman D, Gurbaxani B, Keskinocak P, Sokol J. Using machine learning and an ensemble of methods to predict kidney transplant survival. PLoS One. 2019; 14(1):e0209068. pmid:30625130
- 26. Rao PS, Schaubel DE, Guidinger MK, Andreoni KA, Wolfe RA, Merion RM, et al. A comprehensive risk quantification score for deceased donor kidneys: the kidney donor risk index. Transplantation. 2009;88(2):231–6. pmid:19623019
- 27. Zhong Y, Schaubel DE, Kalbfleisch JD, Ashby VB, Rao PS, Sung RS. Reevaluation of the Kidney Donor Risk Index. Transplantation. 2019;103(8):1714–1721. pmid:30451742
- 28. Loupy A, Aubert O, Orandi BJ, Naesens M, Bouatou Y, Raynaud M, et al. Prediction system for risk of allograft loss in patients receiving kidney transplants: international derivation and validation study. BMJ. 2019;366:l4923. pmid:31530561
- 29. Prasad V, Kim C, Burotto M, Vandross A. The Strength of Association Between Surrogate End Points and Survival in Oncology: A Systematic Review of Trial-Level Meta-analyses. JAMA Intern Med. 2015;175(8):1389–98. pmid:26098871