https://orcid.org/0000-0001-5855-5461
Figures
Abstract
Background
Knowledge of HIV drug resistance informs the choice of regimens and ensures that the most efficacious options are selected. In January 2014, a policy change to routine resistance testing was implemented in Ontario, Canada. The objective of this study was to investigate the policy change impact of routine resistance testing in people with HIV in Ontario, Canada since January 2014.
Methods
We used data on people with HIV living in Ontario from administrative databases of the Institute for Clinical Evaluative Sciences (ICES) and Public Health Ontario (PHO), and ran ordinary least squares (OLS) models of interrupted time series to measure the levels and trends of 2-year mortality, 2-year hospitalizations and 2-year emergency department visits before (2005–2013) and after the policy change (2014–2017). Outcomes were collected in biannual periods, generating 18 periods before the intervention and 8 periods after. We included a control series of people who did not receive a resistance test within 3 months of HIV diagnosis.
Results
Data included 12,996 people with HIV, of which 8881 (68.3%) were diagnosed between 2005 and 2013, and 4115 (31.7%) were diagnosed between 2014 and 2017. Policy change to routine resistance testing within 3 months of HIV diagnosis led to a decreasing trend in 2-year mortality of 0.8% every six months compared to the control group. No significant differences in hospitalizations or emergency department visits were noted.
Citation: Mbuagbaw L, Logie CH, Thabane L, Smaill F, Smieja M, Burchell AN, et al. (2021) The impact of routine HIV drug resistance testing in Ontario: A controlled interrupted time series study. PLoS ONE 16(4): e0246766. https://doi.org/10.1371/journal.pone.0246766
Editor: Yury E. Khudyakov, Centers for Disease Control and Prevention, UNITED STATES
Received: September 22, 2020; Accepted: January 25, 2021; Published: April 2, 2021
Copyright: © 2021 Mbuagbaw et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: These data represent third party data that are not owned nor collected by the study authors. The dataset from this study is held securely in coded form at ICES. While data sharing agreements with other institutions that contribute data prohibit ICES from making the dataset publicly available, access may be granted to those who meet pre-specified criteria for confidential access, available at https://www.ices.on.ca/das. These restrictions are imposed by the Information and Privacy Commissioner of Ontario. The full dataset creation plan and underlying analytic code are available from the authors upon request, understanding that the computer programs may rely upon coding templates or macros that are unique to ICES and are therefore either inaccessible or may require modification. If you are interested in requesting ICES Data & Analytic Services, please email or contact ICES DAS at 416-480-4092 (in Toronto) or 1-844-848-9855 (toll-free internationally). Email: das@ices.on.ca.
Funding: This work was supported by The Ontario HIV Treatment Network (OHTN) grant number EFP-1096-Junior Inv. This study was supported by an award from Constantine Douketis New Researcher Award of The Research Institute of St. Joe’s Hamilton. The study's analysis was supported by the Institute for Clinical Evaluative Sciences (ICES). Parts of this material are based on data and information compiled and provided by the MOHLTC CIHI and PHO. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: Ann N. Burchell is supported by an OHTN Leader Award, Canada Research Chair in Sexually Transmitted Infections and University of Toronto Department of Family and Community Medicine Non-Clinician Scientist Award. Carmen H. Logie is supported by a Canada Research Chair in Global Health Equity and Social Justice with Marginalized Populations and the Ontario Ministry of Research & Innovation. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
There are approximately 36.9 million people living with HIV worldwide [1]. Antiretroviral therapy (ART) is the mainstay of HIV management and is recommended for everyone with HIV and for people at high risk of HIV infection as a prevention strategy [2]. The increasing use of ART has contributed to an increase in resistant strains of HIV [3]. Drug resistance severely hampers the effectiveness of ART, reduces the number of effective therapeutic options, increases infectiousness, and reduces quality of life and survival [4, 5]. Some people with drug resistant HIV are infected with a resistant virus, i.e. transmitted drug resistance (TDR), and this is a recognised global health problem [6]. People with TDR are at a higher risk of treatment failure, treatment discontinuation and developing new drug resistant strains [7]. The rise in drug resistance is one of the greatest threats to global health, and can result in millions of deaths, an increase in new harder-to-treat strains of HIV and higher health-care costs [8]. The prevalence of TDR varies worldwide, but is highest in North America at ~13% [9], likely due to the long-standing and widespread availability of ART. The prevalence of TDR in Canada (based on administrative data) has increased over time, from 9.5% to 27.3% (2002 to 2009) in Ontario [10] and 12% to 18% (1996 to 2016) in British Columbia [11].
Effectively dealing with TDR requires monitoring and timely response to population and individual levels of resistance. In high-income countries that can afford it, resistance testing is performed before the patient starts ART [12]. This approach allows physicians to determine the most efficacious combination of drugs for each patient [12]. However, the merits of this approach are uncertain. Some American studies suggest that drug resistance testing prior to initiating therapy has limited benefits and may not be cost-effective [13, 14].
Close to 71,000 people are living with HIV in Canada, of which 42% live in Ontario [15, 16], Canada’s most populous province. Ontario also accounts for the largest number of new cases of HIV [17]. In Ontario, resistance testing prior to initiating ART became available for physicians around 2005 (discretionary resistance testing), but a formal recommendation for resistance testing to be used as standard of care was only made in January 2014 (routine resistance testing) by Public Health Ontario (PHO) [18]. This policy change led to a corresponding laboratory practice change, and since then, all newly diagnosed people with HIV in Ontario are expected to receive a genotypic resistance test (GRT) prior to initiating ART if their viral load is sufficiently high (>1000 copies/mL).
The aim of this study was to investigate the population-level impact of the resistance testing policy in Ontario by comparing outcomes in the cohort of people with HIV who started treatment between 2005 and 2013 (discretionary resistance testing) to outcomes in the cohort of people who started treatment in 2014 to 2017 (routine resistance testing).
Methods
Ethics
The use of data in this project was authorized under section 45 of Ontario’s Personal Health Information Protection Act, which does not require review by a Research Ethics Board. This project was conducted under section 45 of Ontario’s Personal Health Information Privacy Act and approved by the Privacy and Legal Office of ICES (formerly known as Institute for Clinical Evaluative Sciences. ICES is an independent, non-profit research institute whose legal status under Ontario’s health information privacy law allows it to collect and analyze health care and demographic data, without consent, for health system evaluation and improvement.
ICES is the health information custodian for this study and PHO houses the results of all HIV drug resistance tests in Ontario. All datasets were linked using unique encoded identifiers. All projects that make use of ICES data are assessed for re-identification risks prior to release, which ensures that research outputs do not contain information that could identify an individual. ICES policies and procedures are approved by the Information and Privacy Commissioner of Ontario, allowing ICES to receive personal health information of Ontarians without their consent for the purpose of evaluating, planning, or monitoring Ontario’s health system, or for the purpose of research. ICES’ last completed review was in 2017. ICES is currently undergoing a 2020 review by the Information and Privacy Commissioner (IPC) of Ontario.
Data were pulled on May 14, 2020.
Design
We conducted a controlled interrupted time-series (ITS) analysis of a cohort of people with HIV created from linked administrative databases (from the ICES and laboratory information from PHO).
Outcomes
We selected three patient important and policy relevant outcomes at two years: mortality, hospitalization rate, and emergency department visits among incident HIV cases per biannual period, measured as the fraction of incident HIV cases in each time interval who had the outcome of interest in the follow-up period (i.e., the proportion of incident HIV cases in each biannual period who had the outcome of interest).
Cohort
Cohort entry criteria was all incident cases of HIV in Ontario. HIV infection was determined based on a validated algorithm built upon physician claims for HIV-related care [19]. The index event date was defined as the first contact with the health system for HIV care determined by a positive HIV test or lab request for viral load or resistance testing. Participants were followed up until outcomes occurred or up to December 2019. The cohort was divided into two groups defined by whether a resistance test was used to inform care. The first group had a resistance test done within three months of HIV diagnosis, the second group had a resistance test done beyond 3 months of HIV diagnosis or had no record of a resistance test. A viral load test within 3 months of HIV diagnosis is an indicator for linkage to care [20], which has been used in other Canadian studies [21, 22]. People with a viral load test beyond 3 months of diagnosis are assumed to not be optimally engaged in care.
Statistical analysis
Sample size and power considerations are outlined in S1 Appendix. We used descriptive statistics to characterise the cohort of treatment-naïve people in the years prior to and following the implementation of the policy. Continuous data is described using means (standard deviation) or medians (quartile 1; quartile 3). Categorical data is described using counts (percentage). Age, sex, residence, and the Ontario Marginalization Index (ON-Marg) economic dependency quintiles are reported [23]. Given that these are population level data (and not a random sample), statistical testing may not be meaningful, as such p-values are not reported [24].
We compared outcomes (bi-annually) in people with HIV who initiated treatment before the policy (2005–2013) to outcomes in people who initiated treatment after the policy (2014–2017) adjusting for whether they received a resistance test within 3 months of HIV diagnosis or not, using a controlled ITS approach. The ITS is considered the “next” best strategy for interventions when randomization is not possible and clinical trial data do not exist. It is particularly useful for natural experiments and the evaluation of health policies [25]. However, other events occurring around the time of the intervention may influence outcomes. One way of addressing this concern is to include a control series. No effect in the control series provides stronger evidence in favor of the intervention [26]. In this context, it is expected that resistance tests conducted within 3 months of HIV diagnosis are used to inform first-line treatments and therefore the group of people who received a resistance test much later, or no resistance test at all, would provide an appropriate counterfactual, i.e. what would have happened if there was no policy change. In this analysis a control group is even more important given that before the policy, some people might have initiated treatment informed by a resistance test, and likewise after the policy, some people may not have received a resistance test prior to initiating treatment. Other Canadian studies have shown that access to resistance testing is not uniform [11, 27, 28]. However, ITS only informs inferences at the population level and we can not make inferences about individuals.
We used Ordinary Least Squares (OLS) models to assess the impact of the intervention on our outcomes of interest, both immediately and over time. The models included terms to evaluate the following variables: a constant to represent level of outcome (e.g. mortality) at baseline (before the policy), terms for the linear trend before the policy, the change in level of the outcome immediately after the policy, the change in trend after the policy, the difference in baseline level of outcome between treatment and control groups, the difference in trends between treatment and control before the policy, the difference in change in outcome between the treatment and control group immediately after the policy and the difference between treatment and control groups in the trend of the outcome variable after initiation of the intervention compared with before the policy.
For each modeled outcome of interest, we reported the maximum likelihood regression coefficient estimates from the final models, confidence intervals and p-values. The full ITS model and details on how we addressed potential autocorrelation is described in S2 Appendix and S3 Appendix, respectively.
These analyses were conducted with due consideration of sufficient equally spaced data points before and after the intervention, linearity of pre-intervention trends, stable measurement procedures and incorporate available data on a potential confounder (timely resistance testing) [29]. Trends are illustrated graphically. As a sensitivity analysis, we first ran similar models excluding outliers and secondly estimated our outcomes quarterly. Our findings are reported according to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines [30]. All analyses were performed in SAS statistical software (version 9.4) and figures produced in Stata (version 16.0).
Results
In this controlled ITS we included 18 points (bi-annual) before the intervention and 8 points after the intervention.
Cohort characteristics
We included data from a total of 12,996 people with HIV, of which 8,881 (68.3%) were diagnosed between 2005 and 2013, and 4,115 (31.7%) were diagnosed between 2014 and 2017. In the pre-policy period 25.4% (2262/8881) received a resistance test early (within 3 months of diagnosis), compared to 28.7% (1183/4115) in the post-policy period. The distribution of age, sex, residence, and economic dependency are reported in Table 1.
Two-year mortality
There was a 2% lower mortality in the intervention group prior to the policy change (estimated β = -0.022; 95% confidence interval [CI] -0.041 to -0.003; p = 0.030). Immediately after the policy change, there was a 5% higher mortality in the intervention group (estimated β = 0.048; 95% CI 0.013 to 0.083; p = 0.010). However, in the period following the policy change, mortality dropped by 0.8% every period, compared to the control group (estimated β = -0.008; 95% CI-0.014 to -0.002; p = 0.020) (Fig 1).
Two-year hospitalization
The difference in hospitalization rate between the control and intervention groups was not statistically significant prior to the policy change (β = -0.022; 95% CI -0.058 to -0.015; p = 0.249). Immediately after the policy change there was a 10% higher hospitalization rate in the intervention group compared to the control (β = 0.101; 95% CI 0.034 to 0.0167; p = 0.005), but no difference in the trends in hospitalization in the periods after (β = -0.010; 95% CI -0.022 to 0.002) (Fig 2).
Two-year emergency department visits
There was no significant difference between the intervention and control groups in the rate of emergency department visits before (β = 0.009; 95% CI-0.047 to 0.065; p = 0.751), immediately after (β = 0.061; 95% CI -0.042 to 0.164; p = 0.253) or in the period following the policy change (β = -0.004; 95% CI -0.022 to 0.015; p = 0.700) (Fig 3).
Full model details are presented in Table 2.
Sensitivity analysis
We identified 3 outlier data points and conducted a sensitivity analysis excluding these extreme data points, generating similar results. We also estimated our outcomes quarterly instead of bi-annually and got the same results. The results of the later sensitivity analyses are shown in S4 Appendix.
Discussion
In this cohort of 12,996 people with HIV in Ontario followed up for 13 years (2005 and 2019), we found that the implementation of a drug resistance testing policy of testing prior to treatment initiation in 2014 reduced two-year mortality, but not hospitalizations and emergency department visits. These findings are in line with American and European treatment guidelines that recommend resistance testing prior to initiating treatment in people with HIV [31, 32], and provide further additional evidence to support its use.
Our findings differ from one cohort study comparing the use of resistance testing to no resistance testing in European and Canadian patients with HIV which reported improvements in virological response, but no change in progression to AIDS or death at 5 years. The authors reported that the prevalence of TDR was low (5.8%) [33]. The population-level effect of resistance testing may be smaller if the prevalence of TDR is low.
We noted a 5% increase in mortality and a 10% increase in hospitalizations immediately after the policy. This was an unexpected finding. We hypothesize that with the expanded policy, more people diagnosed in late HIV infection (who have a higher underlying risk of death) underwent resistance testing than prior to the implementation of the policy. This might also be because the resistance testing policy led to more people who live in low income neighbourhoods or rural areas (who already have a higher underlying mortality rate) [29] to get resistance tests, this could increase the rates of mortality and hospitalization immediately after the policy. Confirmation of this hypothesis would require HIV clinical histories including CD4 cell counts, which were unavailable in health administrative data for this analysis.
Further research is required to understand why in the presence of a policy recommending routine resistance testing, some people do not get a resistance test. It is possible that there are provider or patient level factors that influence the use of resistance testing to inform treatment. Also, a resistance test would not be conducted if the viral load is ≤1000 copies/mL [18]. Despite universal access to health care in Canada, there are disparities in access to resistance testing [28]. For example, Indigenous people in British Columbia [28], men who have sex with men, and people who inject drugs in Ontario are less likely to have a drug resistance test [27, 34]. On the other hand, in British Columbia, males were more likely to be tested for drug resistance before initiating therapy than females [28], but Indigenous persons were less likely to be tested and when tested, were more likely to be drug resistant than non-Indigenous persons [11]. These differences likely, in part, have to do with complex social and structural factors, including stigma that interact to shape access to healthcare, including access to, and uptake of resistance testing [35]. Sex differences in resistance testing signal the need for researchers to conduct sex and gender-based analyses to examine how sex characteristics and/or gender norms, gender roles and institutionalized gender influence the barriers to uptake of practices such as drug resistance testing [36].
This study is not without limitations. First, the role of resistance testing in improving outcomes of HIV care depends in part on the prevalence of TDR, the efficacy of ART and adherence to ART, none of which was captured in this study. However, we do not believe that these factors would have been substantially different in the ‘no or late resistance testing’ group, and changes over time are factored into pre-post trend comparisons. We would also like to clarify that these factors, though relevant to individual patients, play a limited role in population-level inferences for which interrupted time series are intended. Further, given the relative novelty of this policy (2014), we were unable to estimate long term outcomes, i.e., 2-year mortality could not be estimated for patients enrolled beyond 2017. These results must also be viewed in the context of recent changes in ART such as the availability of integrase strand inhibitors [37], which exhibit a high barrier to resistance, as well as the widespread use of pre-exposure prophylaxis which may exacerbate the incidence of drug resistance [38].
Some of the strengths of this work include the use of a control group, the use of policy relevant and patient important outcomes, and its novelty. To the best of our knowledge this is the first study to investigate the population-level effect of a pre-treatment resistance testing policy and show the benefits of early routine resistance testing for everyone diagnosed with HIV. These findings would be of interest to countries that already have such policies in place and for countries that need to make evidence-informed decisions about implementing routine resistance testing.
Future work should incorporate relevant patient level characteristics that influence engagement in care, access to care and risk of TDR. Research should also examine physician-level factors and barriers associated with uptake of resistance testing, and if there is a need for specialized approaches to engage women and Indigenous persons.
Resistance testing prior to initiating therapy may lead to fewer deaths, but no difference was shown in emergency room visits or hospitalizations within 2 years. Exploring the individual-level impacts of drug resistance testing and the characteristics of patients who benefit the most from resistance testing may inform tailored approaches to care and provide further impetus for strengthening the HIV care cascade.
Supporting information
S2 Appendix. Full controlled interrupted time series model.
https://doi.org/10.1371/journal.pone.0246766.s002
(DOCX)
S3 Appendix. Approaches used for autocorrelation.
https://doi.org/10.1371/journal.pone.0246766.s003
(DOCX)
S1 File. STROBE statement—checklist of items that should be included in reports of observational studies.
https://doi.org/10.1371/journal.pone.0246766.s005
(DOC)
Acknowledgments
This analysis was supported by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). Parts of this material are based on data and information compiled and provided by the MOHLTC CIHI and PHO. The analyses, conclusions, opinions, and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended.
Ann N. Burchell is supported by an OHTN Leader Award, Canada Research Chair in Sexually Transmitted Infections and University of Toronto Department of Family and Community Medicine Non-Clinician Scientist Award. Carmen H. Logie is supported by a Canada Research Chair in Global Health Equity and Social Justice with Marginalized Populations and the Ontario Ministry of Research & Innovation.
References
- 1.
UNAIDS. Global HIV & AIDS statistics—2018 fact sheet 2019 [cited 2020 20 January]. Available from: http://www.unaids.org/en/resources/fact-sheet.
- 2.
WHO. HIV/AIDS: Treatment and care 2019 [cited 2019 09 September]. Available from: https://www.who.int/hiv/topics/treatment/en/.
- 3. Gupta RK, Gregson J, Parkin N, Haile-Selassie H, Tanuri A, Andrade Forero L, et al. HIV-1 drug resistance before initiation or re-initiation of first-line antiretroviral therapy in low-income and middle-income countries: a systematic review and meta-regression analysis. Lancet Infect Dis. 2017. pmid:29198909
- 4. Gupta RK, Jordan MR, Sultan BJ, Hill A, Davis DHJ, Gregson J, et al. Global trends in antiretroviral resistance in treatment-naive individuals with HIV after rollout of antiretroviral treatment in resource-limited settings: a global collaborative study and meta-regression analysis. Lancet. 2012;380(9849):1250–8. pmid:22828485
- 5. Cambiano V, Bertagnolio S, Jordan MR, Lundgren JD, Phillips A. Transmission of drug resistant HIV and its potential impact on mortality and treatment outcomes in resource-limited settings. J Infect Dis. 2013;207 Suppl 2:S57–62. pmid:23687290
- 6. Beyrer C, Pozniak A. HIV Drug Resistance—An Emerging Threat to Epidemic Control. New England Journal of Medicine. 2017;377(17):1605–7. pmid:29069566
- 7.
WHO. Guidelines on the public health response to pretreatment HIV drug resistance 2017 [cited 2020 25 November]. Available from: https://bit.ly/2C5GI7T.
- 8.
WHO. HIV drug resistance report 2019 2019 [cited 2019 19 September]. Available from: https://www.who.int/hiv/pub/drugresistance/hivdr-report-2019/en/.
- 9. Frentz D, Boucher C, Van De Vijver D. Temporal changes in the epidemiology of transmission of drug-resistant HIV-1 across the world. AIDs Rev. 2012;14(1):17–27. pmid:22297501
- 10. Burchell AN, Bayoumi AM, Rourke SB, Major C, Gardner SS, et al. Increase in transmitted HIV drug resistance among persons undergoing genotypic resistance testing in Ontario, Canada, 2002–09. Journal of Antimicrobial Chemotherapy. 2012;67(11):2755–65.
- 11. Rocheleau G, Franco-Villalobos C, Oliveira N, Brumme ZL, Rusch M, Shoveller J, et al. Sociodemographic correlates of HIV drug resistance and access to drug resistance testing in British Columbia, Canada. PLoS One. 2017;12(9):e0184848. pmid:28937991
- 12. Günthard HF, Saag MS, Benson CA, et al. Antiretroviral drugs for treatment and prevention of hiv infection in adults: 2016 recommendations of the international antiviral society–usa panel. JAMA. 2016;316(2):191–210. pmid:27404187
- 13. Palella FJ Jr., Armon C, Buchacz K, Cole SR, Chmiel JS, Novak RM, et al. The association of HIV susceptibility testing with survival among HIV-infected patients receiving antiretroviral therapy: a cohort study. Ann Intern Med. 2009;151(2):73–84. pmid:19620160
- 14. Hyle EP, Scott JA, Sax PE, Millham LRI, Dugdale CM, Weinstein MC, et al. Clinical Impact and Cost-effectiveness of Genotype Testing at Human Immunodeficiency Virus Diagnosis in the United States. Clinical Infectious Diseases. 2019.
- 15.
CATIE. The epidemiology of HIV in Canada 2017 [cited 2020 17 August]. Available from: http://www.catie.ca/en/fact-sheets/epidemiology/epidemiology-hiv-canada.
- 16.
CATIE. Progress on Ontario’s HIV care cascade 2015 [cited 2020 25 November]. Available from: http://www.catie.ca/en/catienews/2015-09-09/progress-ontario-s-hiv-care-cascade.
- 17. Haddad N, Li J, Totten S, McGuire M. Adult immunization: HIV in Canada—Surveillance Report, 2017. Canada Communicable Disease Report. 2018;44(12):348. pmid:30906230
- 18.
PHO. HIV Genotyping, Resistance, Tropism and HLA-B*57:01 Abacavir Hypersensitivity Testing 2020 [cited 2020 27 August]. Available from: https://www.publichealthontario.ca/en/laboratory-services/test-information-index/hiv-drug-resistance-testing-genotyping.
- 19. Antoniou T, Zagorski B, Loutfy MR, Strike C, Glazier RH. Validation of case-finding algorithms derived from administrative data for identifying adults living with human immunodeficiency virus infection. PloS one. 2011;6(6):e21748–e. pmid:21738786
- 20. Medland NA, McMahon JH, Chow EP, Elliott JH, Hoy JF, Fairley CK. The HIV care cascade: a systematic review of data sources, methodology and comparability. Journal of the International AIDS Society. 2015;18(1):20634. pmid:26626715
- 21. Wilton J, Liu J, Sullivan A, Rachlis B, Marchand-Austin A, Giles M, et al. Trends in HIV care cascade engagement among diagnosed people living with HIV in Ontario, Canada: A retrospective, population-based cohort study. PLOS ONE. 2019;14(1):e0210096. pmid:30608962
- 22. Kronfli N, Lacombe-Duncan A, Wang Y, de Pokomandy A, Kaida A, Logie C, et al. Access and engagement in HIV care among a national cohort of women living with HIV in Canada. AIDS Care. 2017;29(10):1235–42. pmid:28610435
- 23.
Matheson F, Dunn J, Smith K, Moineddin R, Glazier R. Ontario Marginalization Index user guide version 1.0. Toronto, ON, Centre for Research on Inner City Health. 2012.
- 24. Rubin A. Significance Testing with Population Data. Social Service Review. 1985;59(3):518–20.
- 25. Kontopantelis E, Doran T, Springate DA, Buchan I, Reeves D. Regression based quasi-experimental approach when randomisation is not an option: interrupted time series analysis. 2015;350:h2750. pmid:26058820
- 26. Lopez Bernal J, Cummins S, Gasparrini A. The use of controls in interrupted time series studies of public health interventions. International journal of epidemiology. 2018;47(6):2082–93. pmid:29982445
- 27. Burchell AN, Bayoumi AM, Rourke SB, Major C, Gardner S, Sandstrom P, et al. Increase in transmitted HIV drug resistance among persons undergoing genotypic resistance testing in Ontario, Canada, 2002–09. Journal of Antimicrobial Chemotherapy. 2012;67(11):2755–65.
- 28. Eyawo O, Fernandes KA, Brandson EK, Palmer A, Chan K, Lima VD, et al. Suboptimal use of HIV drug resistance testing in a universal health-care setting. AIDS Care—Psychological and Socio-Medical Aspects of AIDS/HIV. 2011;23(1):42–51.
- 29. Burchell AN, Raboud J, Donelle J, Loutfy MR, Rourke SB, Rogers T, et al. Cause-specific mortality among HIV-infected people in Ontario, 1995–2014: a population-based retrospective cohort study. CMAJ open. 2019;7(1):E1–e7. pmid:30622108
- 30. Elm Ev, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ. 2007;335(7624):806–8. pmid:17947786
- 31. Aberg JA, Kaplan JE, Libman H, Emmanuel P, Anderson JR, Stone VE, et al. Primary care guidelines for the management of persons infected with human immunodeficiency virus: 2009 update by the HIV medicine Association of the Infectious Diseases Society of America. Clin Infect Dis. 2009;49(5):651–81. pmid:19640227
- 32.
EACS. European Guidelines for the treatment of people living with HIV (PLWH) in Europe 2019 [cited 2020 27 May]. Available from: https://www.eacsociety.org/guidelines/eacs-guidelines/eacs-guidelines.html.
- 33. Lodi S, Günthard HF, Gill J, Phillips AN, Dunn D, Vu Q, et al. Effectiveness of Transmitted Drug Resistance Testing Before Initiation of Antiretroviral Therapy in HIV-Positive Individuals. JAIDS Journal of Acquired Immune Deficiency Syndromes. 2019;82(3):314–20. pmid:31609929
- 34. Zash R, Makhema J, Shapiro RL. Neural-Tube Defects with Dolutegravir Treatment from the Time of Conception. New England Journal of Medicine. 2018. pmid:30037297
- 35. Logie CH, James L, Tharao W, Loutfy MR. HIV, Gender, Race, Sexual Orientation, and Sex Work: A Qualitative Study of Intersectional Stigma Experienced by HIV-Positive Women in Ontario, Canada. PLOS Medicine. 2011;8(11):e1001124. pmid:22131907
- 36. Tannenbaum C, Greaves L, Graham ID. Why sex and gender matter in implementation research. BMC Medical Research Methodology. 2016;16(1):145. pmid:27788671
- 37. Raffi F, Wainberg MA. Multiple choices for HIV therapy with integrase strand transfer inhibitors. Retrovirology. 2012;9:110–. pmid:23253887
- 38. Gibas KM, van den Berg P, Powell VE, Krakower DS. Drug Resistance During HIV Pre-Exposure Prophylaxis. Drugs. 2019;79(6):609–19. pmid:30963509