Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Drug resistance from preferred antiretroviral regimens for HIV infection in South Africa: A modeling study

  • Ume L. Abbas ,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Writing – original draft, Writing – review & editing

    ume.abbase@bcm.edu

    Affiliations Department of Medicine, Section of Infectious Diseases and Department of Molecular Virology and Microbiology, Baylor College of Medicine, Houston, Texas, United States of America, Departments of Quantitative Health Sciences and Infectious Disease, Cleveland Clinic, Cleveland, Ohio, United States of America

  • Robert L. Glaubius,

    Roles Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Writing – review & editing

    Current address: Avenir Health, Glastonbury, Connecticut, United States of America

    Affiliation Departments of Quantitative Health Sciences and Infectious Disease, Cleveland Clinic, Cleveland, Ohio, United States of America

  • Yajun Ding,

    Roles Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Writing – review & editing

    Current address: PROS Holdings Inc, Houston, Texas, United States of America

    Affiliation Department of Medicine, Section of Infectious Diseases and Department of Molecular Virology and Microbiology, Baylor College of Medicine, Houston, Texas, United States of America

  • Gregory Hood

    Roles Data curation, Formal analysis, Investigation, Methodology, Resources, Software, Writing – review & editing

    Affiliation Pittsburgh Supercomputing Center, Carnegie Mellon University, Pittsburgh, Pennsylvania, United States of America

Drug resistance from preferred antiretroviral regimens for HIV infection in South Africa: A modeling study

  • Ume L. Abbas, 
  • Robert L. Glaubius, 
  • Yajun Ding, 
  • Gregory Hood
PLOS
x

Abstract

Background

Tenofovir-containing regimens comprise the preferred first-line antiretroviral therapy (ART) in many countries including South Africa, where utilization of second-line regimens is limited. Considerable HIV drug resistance has occurred among persons failing tenofovir-containing first-line ART. We evaluated drug resistance at the population level using mathematical modeling.

Setting

Heterosexual HIV epidemic in KwaZulu-Natal, South Africa.

Methods

We constructed a stochastic individual-based model and simulated scenarios of ART implementation, either CD4-based (threshold < 500 cells/mL) or Fast-track (81% coverage by 2020), with consideration of major drug-associated mutations (M184V, K65R and non-nucleoside reverse transcriptase inhibitor (NNRTI)). Using base case and uncertainty analyses, we assessed (majority) drug resistance levels.

Results

By 2030, the median total resistance (proportion of HIV-infected persons with drug resistance) is predicted to reach 31.4% (interquartile range (IQR): 16.5%-50.2%) with CD4-based ART, decreasing to 14.5% (IQR: 7.7%-25.8%) with Fast-track implementation. In both scenarios, we find comparably high prevalence (~80%) of acquired NNRTI-associated, M184V and K65R mutations. Over 48% of individuals with acquired resistance harbor dual, 44% triple and 7% just single drug mutations. Drug-resistant HIV is predicted to comprise 40% (IQR: 27%-50%) of incident infections, while 70% of prevalent transmitted resistance is NNRTI-associated. At 2018, the projected total resistance is 15% (IQR: 7.5%-25%), with 18% (IQR: 13%-24%) of incident infections from transmitted drug-resistant HIV.

Conclusions

WHO-recommended preferred first-line ART could lead to substantial drug resistance. Effective surveillance of HIV drug resistance and utilization of second-line as well as alternative first-line regimens is crucial.

Introduction

Global HIV incidence, though slowly receding, remains unacceptably high at 1.8 million new infections annually [1]. The remarkable efficacy of antiretroviral therapy (ART) for both HIV treatment and prevention led the Joint United Nations Programme on HIV/AIDS (UNAIDS) to recommend a Fast-track approach to ending the AIDS epidemic as a public health threat by 2030, and to establish ambitious targets including 90-90-90 by 2020 (90% of HIV-positive people know their serostatus, of whom 90% receive sustained ART, and 90% of these have viral suppression) and 95-95-95 by 2030 [2]. However, the expansion and maturity of ART programs globally are associated with a rise in HIV drug resistance, posing a threat to the success of ART scale-up and the overall HIV response [3]. Therefore, the World Health Organization (WHO) has launched a Global Action Plan [4], providing a framework, interventions and resources for counteraction, including the recommended consideration of change in the preferred ART regimens [5], especially in countries where the prevalence of pretreatment HIV drug resistance exceeds 10% (detected in antiretroviral naïve or antiretroviral exposed individuals initiating or reinitiating first-line ART) [6]. Concerns about HIV drug resistance are particularly relevant to the large epidemic in South Africa, where only 61% of the HIV-positive people were on treatment in 2017 [1], and in its hardest-hit province of KwaZulu-Natal, where 28% of adults are HIV-positive [7]. In this region, the scale, pace and nature of drug resistance at the population-level are unclear, amidst disparate data [817]. Therefore, we employed mathematical modelling to study HIV drug resistance from ART implementation in KwaZulu-Natal, South Africa.

Methods

We constructed and analyzed a stochastic individual-based mathematical model of the HIV epidemic in KwaZulu-Natal, with details of ART scale-up and HIV drug resistance, using discrete event systems modeling and simulation [1826]. This stochastic model is founded as an analogue [2730] to our prototype deterministic model of the HIV epidemic in KwaZulu-Natal [31, 32], and is detailed to extend and refine our modeling of antiretroviral drug resistance [3133]. Below, we describe the stochastic model structure, assumptions and analytic design, pertinent to this study. Complete specification of our deterministic HIV epidemic model of KwaZulu-Natal is available elsewhere [3133].

Model structure

The model is comprised of a set of modules that represent different categories of dynamic processes such as demographics, sexual behavior change, HIV transmission, disease progression, drug resistance and interventions for HIV prevention and treatment (Fig 1, Table 1 and S1 Table). The model tracks over time, the life histories of all individuals in a realistically sized population (2.5 million initially), characterized by various features (attributes) including gender, age (15–54 years), sexual behavior, infection status, disease stage, intervention status including first- and second-line ART, voluntary medical male circumcision (VMMC) and HIV drug susceptibility. HIV transmission is represented through heterosexual contact influenced by mixing patterns and behavioral factors including condom-use. The overall model state is updated dynamically based on randomly occurring events having specified rates [34] and about exponentially distributed time to next event [35, 36]. The stochastic model is specified using discrete event system specification (DEVS) [2022] and C++ [37]. It is formulated and parameterized as a stochastic analogue [2730] to our prototype deterministic model [31, 32] that was calibrated earlier using Bayesian methods [38]; thereby using the deterministic model’s median posterior input estimates [38] and fitting [39] to simulate the longitudinal HIV incidence and prevalence in KwaZulu-Natal and cross-sectional behavioral risk stratified HIV prevalence in South Africa (Fig 2).

thumbnail
Fig 1. Model structure.

A: Modular structure of the discrete event simulation model. B: Simplified model flow diagram.

https://doi.org/10.1371/journal.pone.0218649.g001

thumbnail
Fig 2. Calibration of the deterministic model and the stochastic individual-based model.

Model calibration to HIV prevalence among (A) women and (B) men by age. Error bars show 95% confidence intervals for data and 95% credible intervals for the IBM model (imperceptible because narrow) and ODE model (posterior) estimates. C: Model calibration to HIV incidence in the ACDSS and comparison to the UNAIDS’ Spectrum model [40]. D: Model validation against HIV prevalence in KwaZulu-Natal among adults aged 15–24 and 15–49 from the four South African national household surveys [7]. Abbreviations: ODE, ordinary differential equation; IBM, individual based model; ACDSS, Africa Centre Demographic Surveillance Site; CrI, credible interval.

https://doi.org/10.1371/journal.pone.0218649.g002

HIV drug resistance

The model distinguishes HIV-positive individuals by antiretroviral use (not on ART or on ART), HIV drug susceptibility (drug-sensitive or drug-resistant), type of drug resistance (transmitted or acquired), and virus population dynamics of drug-resistant HIV (majority or minority). Drug-resistant virus is either acquired through selection pressure from ART or transmitted from a donor with drug-resistant HIV. Drug-resistant virus can revert to drug-sensitive wild-type, off of ART or in a new host, but archived resistance can re-emerge with subsequent ART exposure. We assume that drug-resistant infection can reduce the efficacy of treatment (Table 1). The model represents the presence or absence of the major drug resistance mutations [70], either singly or in combination, associated with antiretrovirals in the WHO recommended preferred first and second-line ART regimens [71], excluding the alternative and/or interim recommended dolutegravir (DTG)-containing regimens [4, 5]. Thus, we consider tenofovir disoproxil fumarate (TDF) + lamivudine or emtricitabine (XTC) + efavirenz or nevirapine (EFV/NVP) as the first-line ART regimen, and zidovudine (AZT) + XTC + boosted lopinavir (LPV/r) as the second-line ART regimen. The following drug resistance mutations (associated with antiretrovirals) are modeled: i) Nucleoside reverse transcriptase (NRTI)-associated signature mutations [70]—M184V (XTC), K65R (TDF) and the thymidine analogue mutations / TAMs (AZT); ii) Non-nucleoside reverse transcriptase (NNRTI)-associated class mutations (EFV/NVP); and iii) Protease inhibitor (PI)-associated class mutations (LPV) [3]. Our assumptions regarding the emergence of acquired resistance to first-line ART are primarily informed by the TenoRes study [69] (Table 1), while those for resistance to second-line ART (not pertinent to this work) are informed by the SELECT study [72]. Though South Africa has the largest ART program in the world, less than 5% of HIV-positive individuals are on second-line regimens [73]. Thus, for clarity and focus on drug resistance from first-line ART, we do not implement the scale-up of second-line ART in this study.

Model-based analyses

CD4-based ART scenario.

Fifty-six percent of HIV-positive people were receiving ART in South Africa in 2016 [74], increasing to 61% in 2017 [1]. Thus, we assume a conservative ART scenario based on South Africa’s 2012–2016 National Strategic Plan [41], achieving 80% VMMC coverage among men and 80% ART coverage among HIV-positive individuals with CD4 cell counts ≤500 cells/μL [43] by 2020, with maintenance thereafter. We assume that VMMC reduces the risk of HIV acquisition in men by 60% [42] and that suppressive ART reduces the transmission risk by 96% [47] and prolongs the survival of HIV-positive individuals [4951, 75, 76].

Fast-track ART scenario.

In 2016, South Africa expanded treatment eligibility to include all HIV-positive individuals regardless of CD4 cell count, and adopted the UNAIDS Fast-track treatment targets [44]. Therefore, we simulate an accelerated ART scenario, assuming 80% VMMC implementation plus universal ART eligibility with expanded testing and treatment rollout, beginning in September 2016 and reaching Fast-track treatment coverage targets of 81% by 2020 and 90% by 2030. To achieve the corresponding Fast-track targets of 73% and 86% overall virologic suppression, we assume an aspirational concurrent adherence-support intervention that reduces virologic failure rates 50% by 2020 and by 80% ultimately (relative to conservative scenario (Table 1)) [77].

Study outcomes

We simulated the CD4-based ART and Fast-track ART, using 100 independent replications [18, 19] for each scenario, from 1978 to 2030 (base case analyses). Next, we repeated the above replications, with random variation of select resistance-related parameters over a specified range (Table 1), using Latin Hypercube Sampling [78], to determine the extent of uncertainty in model results (uncertainty analyses). We calculated the model output (outcome variables) means for base case, and the medians with interquartile range (IQR) for uncertainty analyses. Again, while our model incorporates both first- and second-line ART, the focus of this study is drug resistance associated with preferred first-line ART, reflecting limited utilization of second-line and alternative first-line regimens in South Africa [74, 79, 80]. Our primary outcome is the prevalence of HIV drug resistance in the majority virus by the end of 2030. Additionally, we assess the incidence of drug resistance and the resistance levels at 2018.

Outcome variable definitions.

Total resistance prevalence is defined as the number of HIV-positive individuals with virological non-suppression and acquired and/or transmitted drug-resistant majority virus, divided by the entire number of HIV-positive individuals, at a given time.

Prevalence of acquired resistance is specified as the number of HIV-positive individuals with virological non-suppression and acquired drug-resistant majority virus, divided by the entire number of HIV-positive individuals, at a given time.

Prevalence of acquired drug resistance mutations represents the number of HIV-positive individuals with virological non-suppression and majority virus harboring acquired NNRT-class, K65R and/or M184V mutations (occurring as single or multiple drug-resistant viral variants/mutants), divided by the number of HIV-positive individuals with virological non-suppression and acquired drug-resistant majority virus, at a given time.

Incidence of transmitted resistance is specified as the proportion of new infections due to drug-resistant HIV, in a given time. The prevalence of transmitted resistance is the number of HIV-positive individuals with virological non-suppression and transmitted drug-resistant majority virus, divided by the entire number of HIV-positive individuals, at a given time.

Prevalence of ART-adjusted (total, acquired or transmitted) drug resistance is defined as the number of HIV-positive individuals with virological non-suppression and (acquired and/or transmitted) drug-resistant majority virus, divided by the number of HIV-positive individuals with ART-experience and virological non-suppression, at a given time.

Results

Base case analyses

Prevalence of HIV drug resistance.

At 2018, the total resistance prevalence was similar in the two scenarios (~15%; ~230, 000 cases), with over a tenth contributed by transmitted resistance (~1.5%) (Fig 3A).

thumbnail
Fig 3. Prevalence of total, acquired and ART-adjusted HIV drug resistance.

A: Prevalence of total drug resistance at 2018. B: Prevalence of total drug resistance by 2030. C: Prevalence of acquired drug resistance mutations by 2030. D: Prevalence of acquired drug-resistant mutants by 2030. E: Prevalence of acquired drug-resistant mutants at 2018. F: ART-adjusted drug resistance prevalence at 2018.

https://doi.org/10.1371/journal.pone.0218649.g003

By 2030, the total resistance prevalence reached 30% from CD4-based ART (~375, 000 cases), while Fast-track ART reduced the total resistance to 13% (~150, 000 cases). In either scenario, over four-fifths of the total resistance was attributable to acquired resistance (Fig 3B).

Prevalence of acquired HIV drug resistance mutations.

Irrespective of the scenario, the prevalence of the NNRTI-associated (class), M184V and K65R (signature) mutations was comparable (~80%) in individuals with acquired resistance and virological non-suppression, by 2030 (Fig 3C). Among these individuals, over 48% harbored dual drug mutations, 44% had triple mutations and 7% just single mutations (Fig 3D). Similarly, at 2018, the proportions of these individuals with dual, triple and single mutations were 51%, 42% and 7% respectively (Fig 3E).

Prevalence and incidence of transmitted HIV drug resistance.

Transmitted drug resistance was relatively similar in the two scenarios. NNRTI-associated (class) mutations comprised about 70% of the prevalent transmitted drug resistance at 2018 and end of 2030 (Fig 4A). At 2018, an estimated 18% of the incident/new infections (incidence) were due to transmitted drug-resistant HIV (Fig 4B). By the end of 2030, the proportion of incident drug-resistant HIV infections had risen to 40% (Fig 4B).

thumbnail
Fig 4. Prevalence and incidence of transmitted HIV drug resistance.

A: Distribution of transmitted drug resistance by resistance type at 2018 and by 2030. B: Distribution of incident infections by resistance status at 2018 and by 2030.

https://doi.org/10.1371/journal.pone.0218649.g004

ART-adjusted HIV drug resistance.

In both scenarios at 2018, the ART-adjusted total resistance prevalence was approximately 80%; comprising about 10% transmitted and 70% acquired resistance (Fig 3F). NNRTI-only (transmitted and acquired) resistance prevalence was about 8% (data not shown) while acquired NNRTI+NRTI resistance prevalence was about 55% (Fig 3F).

Uncertainty analyses

The results of our base case analyses were confirmed by uncertainty analyses and are described below.

CD4-based ART scenario.

At 2018, the median total resistance prevalence in this scenario was at a level of 15.5% (IQR: 7.5%-25.3%), constituted by 13.9% (6.6%-22.7%) acquired and 1.6% (0.9%-2.5%) transmitted resistance, and comprising 247, 577 (117, 820–410, 492) resistance cases. By 2030, the median total resistance reached 31.4% (IQR: 16.5%-50.2%). This comprised 399, 504 (205, 366–654, 732) resistance cases and constituted 27% (14.3%-43.2%) acquired and 4.5% (2.3%-7%) transmitted resistance. The median proportion of incident infections attributable to transmitted drug-resistant HIV was 18.3% (IQR: 13%-23.7%) at 2018 which increased to 38.1% (IQR: 26.9%-49.2%) by 2030.

Fast-track ART scenario.

Similar to CD4-based ART, at 2018, the median total resistance prevalence in Fast-track scenario was 15.1% (IQR: 7.5%-25.3%), with an estimated 240,501 (114,615–399,151) cases and 13.5% (6.5%-22.1%) acquired and 1.5% (0.9%-2.5%) transmitted resistance. By 2030, the total resistance from Fast-track ART was less compared to 2018, and considerably less compared to the total resistance from CD4-based ART by 2030; its median value was 14.5% (IQR: 7.7%-25.8%), comprising 163,415 (84,784–299,174) cases of resistance and 13% (6.9%-23.6%) acquired and 1.5% (0.8%-2.2%) transmitted resistance. At 2018, the median proportion of drug-resistant incident infections was 17.9% (IQR: 12.9%-23.4%), increasing to 40.6% (IQR: 28.7%-49.5%) by 2030.

Survey-based versus model-based HIV drug resistance.

S1 Fig illustrates data from the 5th South African National HIV Prevalence, Incidence, Behaviour and Communication Survey, 2017; a cross-sectional, population-based, household survey [81]. Among all survey participants who were HIV-positive with virological non-suppression and successful drug resistance testing, the weighted total (any) drug resistance prevalence was 27.4% (95% confidence interval (CI): 22.8%-32.6%). However, the total resistance was about 10%, when calculated among all HIV-positive survey participants (virologically suppressed and non-suppressed), similar to our model-based estimates (median: 15%; IQR: 7.5%-25.3%). Survey-based drug resistance prevalence was 55.7% among participants with positive ARV detection, of which 14.3% was NNRTI-only and 40.4% was NNRTI+NRTI resistance. Among those who tested negative for ARVs, resistance prevalence was 22.8%, comprised by 20% NNRTI-only and 2.1% NNRTI+NRTI resistance; rising to 75.9% among ARV-experienced, while falling to 15.3% among ARV-naïve participants. Although differences in study design prevent a direct comparison, these survey data are similar to our model-based resistance estimates (10% vs 15% total, 42.6%-67.9% (ARV+) vs 80% (ART-adjusted) total and 29.6%-52.2% (ARV+) vs 55% (ART-adjusted) NNRTI+NRTI acquired resistance). Of note, i) our modeling context is KwaZulu-Natal, the South African province with the highest burden of HIV, ii) the denominator for our ART-adjusted estimates includes all HIV-positive individuals with ART-experience and virological non-suppression, and iii) our model allows for re-entry into ART; these factors may explain the somewhat lower survey-based compared to model-based estimates of drug resistance prevalence overall, as well as NRTI-resistant (versus NNRTI-resistant) majority viruses given their dynamics (Table 1).

Discussion

As the world gears up to end the HIV/AIDS epidemic as a public health threat by 2030, primarily through universal ART, this study addresses the spread of HIV drug resistance from failure of the preferred first-line antiretroviral regimens containing TDF + XTC + NNRTI. Using a novel, detailed and well-parametrized individual-based model of the HIV epidemic in KwaZulu-Natal, we project the prevalence and incidence of HIV drug resistance from ART, in the majority virus at the population level over time, through simulations of CD4-based or Fast-track approaches of ART implementation. The following are important insights from our study. 1) The total resistance prevalence (proportion of HIV-infected individuals with virological non-suppression and acquired and/or transmitted majority drug resistance) from treatment increases over time, especially with the CD4-based approach. 2) By 2030, total resistance from the implementation of Fast-track, is less compared to that from CD4-based ART. 3) Acquired resistance to first-line ART predominantly constitutes the total drug resistance prevalence. 4) Most individuals with acquired drug resistance harbor dual or triple major drug mutations. 5) The proportion of incident/new infections with transmitted drug-resistant HIV increases over time. 6) NNRTI-associated mutations in the majority virus are predominant among the prevalent, transmitted drug-resistant HIV infections. 7) Though differences in study design preclude an exact comparison, our model-based projections of moderate-to-high levels of drug resistance at 2018, are in general agreement with survey- and surveillance-based data [14, 17, 81].

Our base case and uncertainty analyses project increasing levels of total resistance prevalence in the scenario of CD4-based ART reaching a mean (median) value of 30% (31.4%) by 2030 compared to 15% at 2018. The total resistance appears to decrease to 13% (14.5%) in the Fast-track scenario by 2030. Elsewhere, we have found that the principal drivers for decrease in resistance with Fast-track ART are increasingly effective adherence support and rapid initiation of second-line after failure of first-line ART.

We predict that most individuals with acquired resistance have dual or triple drug mutations. In addition, the prevalence estimates of signature mutations associated with TDF and XTC and the NNRTI class mutations, are similarly high (~80%). These results underscore the importance of timely and equitable utilization of second-line ART. Though first-line ART associated mutations were not found to compromise the efficacy of second-line ART in clinical trials [72, 82, 83], our findings warrant caution, particularly over long term.

Our modeling shows that the proportion of incident infections with transmitted drug-resistant HIV increases over time with the implementation of ART. At 2018, our predicted value is 18%, which exceeds the WHO’s 10% threshold for pretreatment drug resistance (detected in antiretroviral naïve or antiretroviral exposed individuals initiating or reinitiating first-line ART) [6], and climbs to 40% by 2030. These data highlight the criticality of universal access to alternate first-line ART regimens and/or point of care drug resistance testing. We do not report the trade-off between averted infections and transmitted drug resistance due to ART, nor the epidemiological or economic impact of drug resistance; as these issues have been examined previously by us [31, 33, 84] and others [67, 85, 86]. Averted infections may offset the risk of resistance over short term [67]. By contrast, in contexts where current levels of pretreatment drug resistance exceed 10%, it is estimated that 16% (890, 000) of AIDS deaths, 9% (450, 000) of new infections, and 8% ($6.5 billion) of ART program costs in sub-Saharan Africa in 2016–2030 will be attributable to HIV drug resistance [85].

Similar to other modeling studies this work has limitations. Precise details of our model's projections will be affected by variations in the embedded structural and parameter assumptions, especially those regarding sexual behavior. Nevertheless, we used rigorous model construction, calibration, parameterization, and analyses. Our assumptions regarding drug resistance were derived from disparate literature sources; however, we carefully parameterized our model inputs, incorporating relevant current and local data, and explored plausible input ranges in uncertainty analyses. Our study focuses on drug resistance from the scale-up of WHO’s preferred first-line ART regimens containing TDF + XTC + EFV/NVP. We do not model the recommended consideration of non-NNRTI (dolutegravir-containing) first-line ART for all starters in countries with high (≥ 10%) pretreatment drug resistance to EFV/NVP [6, 87] or the interim guidelines that recommend changing the preferred first- and second-line regimens as dolutegravir-based [5]. Data from South Africa [817] are mixed regarding drug resistance at the population-level, though a study suggests that KwaZulu-Natal may have surpassed the 10% pretreatment resistance threshold [17]. Though dolutegravir rollout in South Africa appears imminent, the incorporation of this scenario in our model is challenging for the following reasons. The timing, pace and scale of dolutegravir roll out in South Africa are not precisely known [79, 80]. The policy of dolutegravir implementation may change from use in people initiating ART to use in all people on ART [88]. Several gaps in the evidence base need to be addressed by researchers as part of dolutegravir roll out [89]: More data are required to determine the risk of adverse birth outcomes when women initiate dolutegravir-based regimens before conception; While increasing access to viral load testing for monitoring the effectiveness of dolutegravir remains crucial, the best strategy to manage patients with viremia is unclear; The evidence to support the effectiveness of dolutegravir when given with tuberculosis treatment remains scarce. Finally, whether NRTI resistance will affect the long-term efficacy of dolutegravir-based regimens in first-line, and potentially second-line, ART is unknown [90] and dolutegravir-resistance patterns may differ across HIV type 1 non-B subtypes [91]. Clinical trials, cohorts, and surveillance of HIV drug resistance will be necessary to answer these questions, maximize the benefits of dolutegravir-based regimens, as well as inform future mathematical modeling. Nevertheless, our study provides critical insight into the potential trends and patterns of HIV drug resistance in South Africa, in the context of scale-up of TDF + XTC + EFV/NVP, as the preferred first-line ART, at present and in the future; lending support to the WHO recommendations for regimen change [5]. Future work will address model refinements informed by new available data. We do not include the prevention of mother to child transmission, which may be an important source of resistance [92]. Finally, our modeling context is the mature, generalized, high-prevalence HIV epidemic in KwaZulu-Natal, South Africa. While our quantitative findings may not be directly generalizable to other contexts, the qualitative insights from our modeling are robust.

Conclusion

Monitoring, prevention and treatment of drug-resistant HIV are vital components of the HIV response. Affordable and universal access to safe and effective first- and subsequent-line ART regimens, alongside reliable and convenient tests for HIV viral load and drug resistance, are crucial for the end of AIDS.

Supporting information

S1 Table. Behavioral, epidemiological and demographic model input parameters.

https://doi.org/10.1371/journal.pone.0218649.s001

(DOCX)

S1 Fig. Survey-based data.

Human Sciences Research Council survey-based HIV drug resistance in South Africa. The illustration was generated from publically available data.

https://doi.org/10.1371/journal.pone.0218649.s002

(TIF)

References

  1. 1. UNAIDS data 2018. http://www.unaids.org/en/resources/documents/2018/unaids-data-2018. (Accessed 30 December 2018).
  2. 2. UNAIDS. On the fast-track to end AIDS: UNAIDS 2016–2021 Strategy. http://www.unaids.org/sites/default/files/media_asset/20151027_UNAIDS_PCB37_15_18_EN_rev1.pdf (Accessed 30 December 2018). 2015.
  3. 3. World Health Organization. HIV drug resistance report 2017. http://www.who.int/hiv/pub/drugresistance/hivdr-report-2017/en/ (Accessed 30 December 2018). 2017.
  4. 4. World Health Organization. Global action plan on HIV drug resistance 2017–2021. https://www.who.int/hiv/pub/drugresistance/hivdr-action-plan-2017-2021/en/. (Accessed: 30 December 2018). Geneva 2017.
  5. 5. World Health Organization. Updated recommendations on first-line and second-line antiretroviral regimens and post-exposure prophylaxis and recommendations on early infant diagnosis of HIV: interim guidelines. Supplement to the 2016 consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection. https://www.who.int/hiv/pub/guidelines/ARV2018update/en/ (Accessed 30 Dec 2018). Geneva 2018.
  6. 6. World Health Organization. Guidelines on the public health response to pretreatment HIV drug resistance. https://www.who.int/hiv/pub/guidelines/hivdr-guidelines-2017/en/ (Accessed 30 December 2018). Geneva 2017.
  7. 7. Shisana O, Rehle T, Simbayi L, Zuma K, Jooste S, Zungu N, et al. South African national HIV prevalence, incidence and behaviour survey, 2012. Cape Town: HSRC Press; 2014.
  8. 8. Hunt GM, Ledwaba J, Basson AE, Moyes J, Cohen C, Singh B, et al. Surveillance of transmitted HIV-1 drug resistance in Gauteng and KwaZulu-Natal Provinces, South Africa, 2005–2009. Clin Infect Dis. 2012;54 Suppl 4:S334–8.
  9. 9. Manasa J, Katzenstein D, Cassol S, Newell ML, de Oliveira T. Primary drug resistance in South Africa: data from 10 years of surveys. AIDS Res Hum Retroviruses. 2012;28(6):558–65. pmid:22251009
  10. 10. Parboosing R, Naidoo A, Gordon M, Taylor M, Vella V. Resistance to antiretroviral drugs in newly diagnosed, young treatment-naive HIV-positive pregnant women in the province of KwaZulu-Natal, South Africa. J Med Virol. 2011;83(9):1508–13. pmid:21739439
  11. 11. Parikh UM, Kiepiela P, Ganesh S, Gomez K, Horn S, Eskay K, et al. Prevalence of HIV-1 drug resistance among women screening for HIV prevention trials in KwaZulu-Natal, South Africa (MTN-009). PLoS One. 2013;8(4):e59787. pmid:23585827
  12. 12. Steegen K, Bronze M, Papathanasopoulos MA, van Zyl G, Goedhals D, Van Vuuren C, et al. Prevalence of Antiretroviral Drug Resistance in Patients Who Are Not Responding to Protease Inhibitor-Based Treatment: Results From the First National Survey in South Africa. J Infect Dis. 2016;214(12):1826–30. pmid:27923946
  13. 13. Steegen K, Carmona S, Bronze M, Papathanasopoulos MA, van Zyl G, Goedhals D, et al. Moderate levels of pre-treatment HIV-1 antiretroviral drug resistance detected in the first south african national survey. PLoS One. 2016;11(12):e0166305. pmid:27907009
  14. 14. Steegen K, Bronze M, Papathanasopoulos MA, van Zyl G, Goedhals D, Variava E, et al. HIV-1 antiretroviral drug resistance patterns in patients failing NNRTI-based treatment: results from a national survey in South Africa. J Antimicrob Chemother. 2017;72(1):210–9. pmid:27659733
  15. 15. Boender TS, Kityo CM, Boerma RS, Hamers RL, Ondoa P, Wellington M, et al. Accumulation of HIV-1 drug resistance after continued virological failure on first-line ART in adults and children in sub-Saharan Africa. J Antimicrob Chemother. 2016;71(10):2918–27. pmid:27342546
  16. 16. Hunt GM, Dokubo EK, Takuva S, de Oliveira T, Ledwaba J, Dube N, et al. Rates of virological suppression and drug resistance in adult HIV-1-positive patients attending primary healthcare facilities in KwaZulu-Natal, South Africa. J Antimicrob Chemother. 2017;72(11):3141–8. pmid:28981637
  17. 17. Chimukangara B, Kharsany ABM, Lessells RJ, Naidoo K, Rhee SY, Manasa J, et al. Moderate-to-high levels of pretreatment HIV drug resistance in KwaZulu-Natal province, South Africa. AIDS Res Hum Retroviruses. 2019;35(2):129–38. pmid:30430843
  18. 18. Banks J, Carson JS II, Nelson BL, Nicol DM. Discrete-event system simulation, 5th Edition: Pearson; 2009. 648 p.
  19. 19. Law A, Kelton W. Simulation modeling and analysis. Third ed: McGraw-Hill; 2000.
  20. 20. Zeigler B. Theory of modeling and simulation. 1st ed. New York: Wiley Interscience; 1976.
  21. 21. Zeigler BP. Multifacetted modelling and discrete event simulation: Academic Press Professional, Inc.; 1984. 372 p.
  22. 22. Zeigler BP, Muzy A. From discrete event simulation to discrete event specified systems (DEVS). IFAC-PapersOnLine. 2017;50(1):3039–44. https://doi.org/10.1016/j.ifacol.2017.08.672.
  23. 23. Ingalls RG. Introduction to simulation. Proceedings of the 2013 Winter Simulation Conference: Simulation: Making decisions in a complex world; Washington, D.C. 2676019: IEEE Press; 2013. p. 291–305.
  24. 24. Karnon J, Stahl J, Brennan A, Caro JJ, Mar J, Moller J. Modeling using discrete event simulation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-4. Med Decis Making. 2012;32(5):701–11. pmid:22990085
  25. 25. Glynn PW. On the role of generalized semi-Markov processes in simulation output analysis. Proceedings of the 15th conference on Winter simulation—Volume 1; Arlington, Virginia, USA. 801471: IEEE Press; 1983. p. 39–44.
  26. 26. Yücesan E, Schruben L. Structural and behavioral equivalence of simulation models. ACM Trans Model Comput Simul. 1992;2(1):82–103.
  27. 27. Koopman JS, Jacquez G, Chick SE. New data and tools for integrating discrete and continuous population modeling strategies. Ann N Y Acad Sci. 2001;954:268–94. pmid:11797861
  28. 28. Chick SE, Adams AL, Koopman JS. Analysis and simulation of a stochastic, discrete-individual model of STD transmission with partnership concurrency. Math Biosci. 2000;166(1):45–68. pmid:10882799
  29. 29. Kadanoff LP. More is the same; phase transitions and mean field theories. Journal of Statistical Physics. 2009;137(5/6):777–97.
  30. 30. Simon PL, Kiss IZ. From exact stochastic to mean-field ODE models: a new approach to prove convergence results. IMA Journal of Applied Mathematics. 2013;78(5):945–64.
  31. 31. Glaubius RL, Hood G, Penrose KJ, Parikh UM, Mellors JW, Bendavid E, et al. Cost-effectiveness of injectable preexposure prophylaxis for HIV prevention in South Africa. Clin Infect Dis. 2016;63(4):539–47. pmid:27193745
  32. 32. Glaubius RL, Parikh UM, Hood G, Penrose KJ, Bendavid E, Mellors JW, et al. Deciphering the effects of injectable pre-exposure prophylaxis for combination HIV prevention. Open Forum Infectious Diseases. 2016. pmid:27703992
  33. 33. Glaubius R, Ding Y, Penrose KJ, Hood G, Engquist E, Mellors JW, et al. Dapivirine vaginal ring for HIV prevention: modelling health outcomes, drug resistance and cost-effectiveness. J Int AIDS Soc. 2019;22(5):e25282. pmid:31074936
  34. 34. Miller DK, Homan SM. Determining transition probabilities: confusion and suggestions. Med Decis Making. 1994;14(1):52–8. pmid:8152357
  35. 35. Gillespie DT. Exact stochastic simulation of coupled chemical reactions. J Phys Chem. 1977;81(25):2340–61.
  36. 36. Gillespie DT. Approximate accelerated stochastic simulation of chemically reacting systems. J Chem Phys. 2001;115(4):1716–33.
  37. 37. Stroustrup B. The C++ programming language: Addison-Wesley Professional; 2013. 1368 p.
  38. 38. Raftery AE, Bao L. Estimating and projecting trends in HIV/AIDS generalized epidemics using incremental mixture importance sampling. Biometrics. 2010;66(4):1162–73. pmid:20222935
  39. 39. Arlinghaus S. Practical handbook of curve fitting. 1st ed: CRC Press 1994. 272 p.
  40. 40. Mahy M, Lewden C, Brinkhof MWG, Dabis F, Tassie J-M, Souteyrand Y, et al. Derivation of parameters used in Spectrum for eligibility for antiretroviral therapy and survival on antiretroviral therapy. Sex Transm Infect. 2010;86(Suppl 2):ii28–ii34.
  41. 41. South Africa National AIDS Council. National strategic plan on HIV, STIs and TB, 2012–2016. Pretoria: SANAC; 2012.
  42. 42. Mehta SD, Moses S, Agot K, Odoyo-June E, Li H, Maclean I, et al. The long-term efficacy of medical male circumcision against HIV acquisition. AIDS. 2013;27(18):2709–899.
  43. 43. South Africa National Department of Health. National consolidated guidelines for the prevention of mother-to-child transmission of HIV (PMTCT) and the management of HIV in children, adolescents and adults. Pretoria: DoH; 2014.
  44. 44. South African National AIDS Council (SANAC). South Africa's national strategic plan for HIV, TB and STIs 2017–2022. 2017.
  45. 45. Joint United Nations Programme on HIV/AIDS. Ending AIDS: Progress towards the 90-90-90 targets. Geneva: Joint United Nations Programme on HIV/AIDS; 2017.
  46. 46. Joint United Nations Programme on HIV/AIDS. Fast-Track: ending the AIDS epidemic by 2030. Geneva: Joint United Nations Programme on HIV/AIDS; 2014.
  47. 47. Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, Kumarasamy N, et al. Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med. 2011;365(6):493–505. pmid:21767103
  48. 48. Grimsrud AT, Cornell M, Egger M, Boulle A, Myer L. Impact of definitions of loss to follow-up (LTFU) in antiretroviral therapy program evaluation: variation in the definition can have an appreciable impact on estimated proportions of LTFU. J Clin Epidemiol. 2013;66(9):1006–13. pmid:23774112
  49. 49. Johnson LF, Mossong J, Dorrington RE, Schomaker M, Hoffmann CJ, Keiser O, et al. Life expectancies of South African adults starting antiretroviral treatment: collaborative analysis of cohort studies. PLoS Med. 2013;10(4):e1001418. pmid:23585736
  50. 50. Clouse K, Pettifor AE, Maskew M, Bassett J, Van Rie A, Gay CL, et al. Initiating antiretroviral therapy when presenting with higher CD4 cell counts results in reduced loss to follow-up in a resource-limited setting. AIDS. 2013;27(4):645–50. pmid:23169326
  51. 51. When To Start Consortium, Sterne JAC, May M, Costagliola D, de Wolf F, Phillips AN, et al. Timing of initiation of antiretroviral therapy in AIDS-free HIV-1-infected patients: a collaborative analysis of 18 HIV cohort studies. Lancet. 2009;373(9672):1352–63. pmid:19361855
  52. 52. Hosseinipour MC, Gupta RK, Van Zyl G, Eron JJ, Nachega JB. Emergence of HIV drug resistance during first- and second-line antiretroviral therapy in resource-limited settings. J Infect Dis. 2013;207(Suppl 2):S49–S56.
  53. 53. Barth RE, van der Loeff MFS, Schuurman R, Hoepelman AIM, Wensing AMJ. Virological follow-up of adult patients in antiretroviral treatment programmes in sub-Saharan Africa: a systematic review. Lancet Infect Dis. 2010;10(3):155–66. pmid:20185094
  54. 54. Hamers RL, Schuurman R, Sigaloff KCE, Wallis CL, Kityo C, Siwale M, et al. Effect of pretreatment HIV-1 drug resistance on immunological, virological, and drug-resistance outcomes of first-line antiretroviral treatment in sub-Saharan Africa: a multicentre cohort study. Lancet Infect Dis. 2012;12(4):307–17. pmid:22036233
  55. 55. Li JZ, Paredes R, Ribaudo HJ, Svarovskaia ES, Metzner KJ, Kozal MJ, et al. Low-frequency HIV-1 drug resistance mutations and risk of NNRTI-based antiretroviral treatment failure: a systematic review and pooled analysis. JAMA. 2011;305(13):1327–35. pmid:21467286
  56. 56. Aghokeng AF, Monleau M, Eymard-Duvernay S, Dagnra A, Kania D, Ngo-Giang-Huong N, et al. Extraordinary heterogeneity of virological outcomes in patients receiving highly antiretroviral therapy and monitored with the World Health Organization public health approach in sub-Saharan Africa and southeast Asia. Clin Infect Dis. 2014;58(1):99–109. pmid:24076968
  57. 57. Hamers RL, Sigaloff KCE, Wensing AM, Wallis CL, Kityo C, Siwale M, et al. Patterns of HIV-1 drug resistance after first-line antiretroviral therapy (ART) failure in 6 sub-Saharan African countries: implications for second-line ART strategies. Clin Infect Dis. 2012;54(11):1660–9. pmid:22474222
  58. 58. World Health Organization. WHO HIV drug resistance report 2012. Geneva: World Health Organization; 2012.
  59. 59. Jain V, Sucupira MC, Bacchetti P, Hartogensis W, Diaz RS, Kallas EG, et al. Differential persistence of transmitted HIV-1 drug resistance mutation classes. J Infect Dis. 2011;203(8):1174–81. pmid:21451005
  60. 60. Paquet AC, Baxter J, Weidler J, Lie Y, Lawrence J, Kim R, et al. Differences in reversion of resistance mutations to wild-type under structured treatment interruption and related increase in replication capacity. PLoS One. 2011;6(1):e14638. pmid:21297946
  61. 61. Palmer S, Boltz V, Maldarelli F, Kearney M, Halvas EK, Rock D, et al. Selection and persistence of non-nucleoside reverse transcriptase inhibitor-resistant HIV-1 in patients starting and stopping non-nucleoside therapy. AIDS. 2006;20(5):701–10. pmid:16514300
  62. 62. Wirden M, Delaugerre C, Marcelin AG, Ktorza N, Ait Mohand H, Dominguez S, et al. Comparison of the dynamics of resistance-associated mutations to nucleoside reverse transcriptase inhibitors, nonnucleoside reverse transcriptase inhibitors, and protease inhibitors after cessation of antiretroviral combination therapy. Antimicrob Agents Chemother. 2004;48(2):644–7. pmid:14742228
  63. 63. Castro H, Pillay D, Cane P, Asboe D, Cambiano V, Phillips A, et al. Persistence of HIV-1 transmitted drug resistance mutations. J Infect Dis. 2013;208(9):1459–63. pmid:23904291
  64. 64. Deeks SG, Hoh R, Neilands TB, Liegler T, Aweeka F, Petropoulos CJ, et al. Interruption of treatment with individual therapeutic drug classes in adults with multidrug-resistant HIV-1 infection. J Infect Dis. 2005;192(9):1537–44. pmid:16206068
  65. 65. Trignetti M, Sing T, Svicher V, Santoro MM, Forbici F, D'Arrigo R, et al. Dynamics of NRTI resistance mutations during therapy interruption. AIDS Res Hum Retroviruses. 2009;25(1):57–64. pmid:19182921
  66. 66. Carlson JM, Schaefer M, Monaco DC, Batorsky R, Claiborne DT, Prince J, et al. HIV transmission. Selection bias at the heterosexual HIV-1 transmission bottleneck. Science. 2014;345(6193):1254031. pmid:25013080
  67. 67. Nichols BE, Sigaloff KC, Kityo C, Mandaliya K, Hamers RL, Bertagnolio S, et al. Averted HIV infections due to expanded antiretroviral treatment eligibility offsets risk of transmitted drug resistance: a modeling study. AIDS. 2014;28(1):73–83. pmid:23921620
  68. 68. Xu HT, Martinez-Cajas JL, Ntemgwa ML, Coutsinos D, Frankel FA, Brenner BG, et al. Effects of the K65R and K65R/M184V reverse transcriptase mutations in subtype C HIV on enzyme function and drug resistance. Retrovirology. 2009;6:14. pmid:19210791
  69. 69. Global epidemiology of drug resistance after failure of WHO recommended first-line regimens for adult HIV-1 infection: a multicentre retrospective cohort study. Lancet Infect Dis. 2016;16(5):565–75. pmid:26831472
  70. 70. Rhee SY, Jordan MR, Raizes E, Chua A, Parkin N, Kantor R, et al. HIV-1 drug resistance mutations: potential applications for point-of-care genotypic resistance testing. PLoS One. 2015;10(12):e0145772. pmid:26717411
  71. 71. World Health Organization. Consolidated guidelines on the use of antiretroviral drugs For treating and preventing HIV infection: recommendations for a public health approach. http://apps.who.int/iris/bitstream/10665/208825/1/9789241549684_eng.pdf?ua=1 (Accessed: July 22, 2018). 2016.
  72. 72. La Rosa AM, Harrison LJ, Taiwo B, Wallis CL, Zheng L, Kim P, et al. Raltegravir in second-line antiretroviral therapy in resource-limited settings (SELECT): a randomised, phase 3, non-inferiority study. Lancet HIV. 2016;3(6):e247–58. pmid:27240787
  73. 73. Moorhouse M, Maartens G, Venter WDF, Moosa MY, Steegen K, Jamaloodien K, et al. Third-line antiretroviral therapy program in the South African public sector: cohort description and virological outcomes. J Acquir Immune Defic Syndr. 2019;80(1):73–8. pmid:30334876
  74. 74. Joint United Nations Programme on HIV/AIDS (UNAIDS). UNAIDS Data 2017. 2017.
  75. 75. Fox MP, Ive P, Long L, Maskew M, Sanne I. High rates of survival, immune reconstitution, and virologic suppression on second-line antiretroviral therapy in South Africa. J Acquir Immune Defic Syndr. 2010;53(4):500–6. pmid:19838128
  76. 76. Johnston V, Cohen K, Wiesner L, Morris L, Ledwaba J, Fielding KL, et al. Viral suppression following switch to second-line antiretroviral therapy: associations with nucleoside reverse transcriptase inhibitor resistance and subtherapeutic drug concentrations prior to switch. J Infect Dis. 2014;209(5):711–20. pmid:23943851
  77. 77. Luque-Fernandez MA, Van Cutsem G, Geomaere E, Hilderbrand K, Schomaker M, Mantangana N, et al. Effectiveness of patient adherence groups as a model of care for stable patients on antiretroviral therapy in Khayelitsha, Cape Town, South Africa. PLoS One. 2013;8(2):e56088. pmid:23418518
  78. 78. McKay MD, Beckman RJ, Conover WJ. A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics. 1979;21(2):239–45.
  79. 79. (PEPFAR) Smith S. Dolutegravir / TLD roll out in South Africa. http://www.sahivsoc2018.co.za/wp-content/uploads/2018/11/17_Steven-Smith_SAHIVCS-Oct-2018-oct-26-draft.pdf. 2017.
  80. 80. Moorhouse MA, Carmona S, Davies N, Dlamini S, van Vuuren C, Manzini T, et al. Southern African HIV Clinicians Society Guidance on the use of dolutegravir in first-line antiretroviral therapy. South Afr J HIV Med. 2018;19(1):917. pmid:30473877
  81. 81. Moyo S, Hunt G, Khangelani Z, Zungu NP, Marinda E, Mabaso M, et al. HIV drug resistance in South Africa: results from a population-based household survey. The Conference on Retroviruses and Opportunistic Infections (CROI); March 4–7; Seattle, Washington2019.
  82. 82. Boyd MA, Moore CL, Molina JM, Wood R, Madero JS, Wolff M, et al. Baseline HIV-1 resistance, virological outcomes, and emergent resistance in the SECOND-LINE trial: an exploratory analysis. Lancet HIV. 2015;2(2):e42–51. pmid:26424460
  83. 83. Paton NI, Kityo C, Thompson J, Nankya I, Bagenda L, Hoppe A, et al. Nucleoside reverse-transcriptase inhibitor cross-resistance and outcomes from second-line antiretroviral therapy in the public health approach: an observational analysis within the randomised, open-label, EARNEST trial. Lancet HIV. 2017;4(8):e341–e8. pmid:28495562
  84. 84. Abbas UL, Glaubius R, Mubayi A, Hood G, Mellors JW. Antiretroviral therapy and pre-exposure prophylaxis: combined impact on HIV transmission and drug resistance in South Africa. J Infect Dis. 2013;208(2):224–34. pmid:23570850
  85. 85. Phillips AN, Stover J, Cambiano V, Nakagawa F, Jordan MR, Pillay D, et al. Impact of HIV Drug Resistance on HIV/AIDS-Associated Mortality, New Infections, and Antiretroviral Therapy Program Costs in Sub-Saharan Africa. J Infect Dis. 2017;215(9):1362–5. pmid:28329236
  86. 86. Wagner BG, Blower S. Universal access to HIV treatment versus universal 'test and treat': transmission, drug resistance & treatment costs. PLoS One. 2012;7(9):e41212. pmid:22957012
  87. 87. Phillips AN, Cambiano V, Nakagawa F, Revill P, Jordan MR, Hallett TB, et al. Cost-effectiveness of public-health policy options in the presence of pretreatment NNRTI drug resistance in sub-Saharan Africa: a modelling study. Lancet HIV. 2018;5(3):e146–e54. pmid:29174084
  88. 88. Phillips AN, Venter F, Havlir D, Pozniak A, Kuritzkes D, Wensing A, et al. Risks and benefits of dolutegravir-based antiretroviral drug regimens in sub-Saharan Africa: a modelling study. Lancet HIV. 2019; 6(2):e116–e127. pmid:30503325
  89. 89. Dorward J, Lessells R, Drain PK, Naidoo K, de Oliveira T, Pillay Y, et al. Dolutegravir for first-line antiretroviral therapy in low-income and middle-income countries: uncertainties and opportunities for implementation and research. Lancet HIV. 2018;5(7):e400–e4. pmid:29884404
  90. 90. Wijting I, Rokx C, Boucher C, van Kampen J, Pas S, de Vries-Sluijs T, et al. Dolutegravir as maintenance monotherapy for HIV (DOMONO): a phase 2, randomised non-inferiority trial. Lancet HIV. 2017;4(12):e547–e54. pmid:29107562
  91. 91. Han YS, Mesplede T, Wainberg MA. Differences among HIV-1 subtypes in drug resistance against integrase inhibitors. Infect Genet Evol. 2016;46:286–91. pmid:27353185
  92. 92. Samuel R, Julian MN, Paredes R, Parboosing R, Moodley P, Singh L, et al. HIV-1 drug resistance by ultra-deep sequencing following short course zidovudine, single-dose nevirapine, and single-dose tenofovir with emtricitabine for prevention of mother-to-child transmission. J Acquir Immune Defic Syndr. 2016;73(4):384–9. pmid:27327263