Rectal and genital sampling in HIV prevention trials permits assessments at the site of HIV entry. Yet the safety and acceptability of circumcision and sigmoidoscopy (and associated abstinence recommendations) are unknown in uncircumcised men who have sex with men (MSM) at high risk of HIV infection.
Twenty-nine HIV-seronegative high-risk Peruvian MSM agreed to elective sigmoidoscopy biopsy collections (weeks 2 and 27) and circumcision (week 4) in a 28-week cohort study designed to mimic an HIV vaccine study mucosal collection protocol. We monitored adherence to abstinence recommendations, procedure-related complications, HIV infections, peripheral immune activation, and retention.
Twenty-three (79.3%) underwent a first sigmoidoscopy, 21 (72.4%) were circumcised, and 16 (55.2%) completed a second sigmoidoscopy during the study period. All who underwent procedures completed the associated follow-up safety visits. Those completing the procedures reported they were well tolerated, and complication rates were similar to those reported in the literature. Immune activation was detected during the healing period (1 week post-sigmoidoscopy, 6 weeks post-circumcision), including increases in CCR5+CD4+T cells and α4β7+CD4+T cells. Most participants adhered to post-circumcision abstinence recommendations whereas reduced adherence occurred post-sigmoidoscopy.
Rectosigmoid mucosal and genital tissue collections were safe in high-risk MSM. Although the clinical implications of the post-procedure increase in peripheral immune activation markers are unknown, they reinforce the need to provide ongoing risk reduction counseling and support for post-procedure abstinence recommendations. Future HIV vaccine studies should also consider the effects of mucosal and tissue collections on peripheral blood endpoints in trial design and analysis.
Citation: Lama JR, Karuna ST, Grant SP, Swann EM, Ganoza C, Segura P, et al. (2016) Transient Peripheral Immune Activation follows Elective Sigmoidoscopy or Circumcision in a Cohort Study of MSM at Risk of HIV Infection. PLoS ONE 11(8): e0160487. https://doi.org/10.1371/journal.pone.0160487
Editor: Alan Landay, Rush University, UNITED STATES
Received: March 22, 2016; Accepted: July 20, 2016; Published: August 18, 2016
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: Research reported in this publication was supported by the National Institute of Allergy and Infectious Diseases of the National Institutes of Health under award numbers UM1AI068614, UM1AI068618, UM1AI068635, and UM1AI069438. The content is the responsibility of the authors and does not necessarily represent the official views, policy or position of the U.S. National Institutes of Health, Department of the Navy, Department of Defense, or U.S. Government. Silvia Montano and Edith Swann are employees of the U.S. Government, who contributed to this work as part of their official duties. No copyright protection is available for their work as defined by Title 17 U.S.C. §105 and §101.
Competing interests: The authors have declared that no competing interests exist.
HIV exposure in rectal and genital compartments varies dramatically in HIV transmission risk, with receptive anal intercourse (RAI) being 12–35 times more likely to lead to infection that penile insertive behaviors [1, 2]. Such differences in compartments may require distinct immune potency and/or different mechanisms of protection to prevent HIV infection. In addition, rectal and genital sites can exhibit a degree of immune compartmentalization that cannot be studied from blood sampling [3–7]. Thus, to fill gaps in our understanding of immune responses to HIV vaccines, studies of mucosal immunity aim to complement assessments of systemic immunity. However, it is unclear and important to understand if intestinal and genital sampling can impact peripheral blood endpoints in HIV vaccine trials.
The collection of secretions is the least invasive method of sampling mucosal compartments, but it is limited to assessments of secreted soluble mediators, such as cytokines and antibodies. Secretions are not representative of the conditions of HIV-exposed mucosal tissue, where cellular and humoral components of a vaccine can act to prevent cell-associated and free virus transmission [8–10]. Therefore, genital and intestinal tissue sampling might be needed to fully understand vaccine responses in humans. For example, in the Step Study the lack of information about mucosal tissue responses to the vaccine in Ad5 seropositive individuals has made it difficult to interpret the increased risk observed with vaccination, or its association with circumcision status . T-cell mediated vaccines such as those using rhesus cytomegalovirus vectors may also benefit from tissue sampling at HIV exposure sites, where effector-memory T cells could play a significant role in the immune response .
In this manner, large efficacy trials in populations at risk of HIV have been complemented by smaller trials with mucosal sampling in populations at low HIV risk. However, multiple differences among mucosal compartments in high risk and low risk individuals, such as inflammation, the presence of other sexually transmitted infections (STIs), and potential microtrauma relating to sexual activity, can confound extrapolation of intestinal and genital response findings from low risk individuals to high risk ones [12–14]. Thus, some data from mucosal studies in low risk individuals may not represent mucosal and genital compartments of the populations at risk of HIV.
One gap in the HIV prevention clinical trials field is in our understanding of whether genital and intestinal sampling and the associated peri-procedure behavioral recommendations are tolerable, acceptable, and safe to sexually active men who have sex with men (MSM). For example, the acceptability and safety of circumcision was demonstrated in a largely heterosexual and HIV-negative population in East and South Africa [15–17], but data regarding circumcision in MSM is less clear . Although East and South African circumcision trials reported high adherence to post-procedure sexual abstinence recommendations and low complication rates [15–17, 19, 20], levels of baseline sexual activity in those populations varied, and complication rates were higher with earlier post-procedure resumption of sexual activity. A better understanding of adherence to post-procedure abstinence in sexually active MSM in the Americas is needed.
Information about the acceptability and safety of sigmoidoscopy with biopsy in high-risk MSM is even more limited than that for circumcision. Few groups have conducted flexible sigmoidoscopy studies in participants who practice RAI [21, 22]. Therefore, to better characterize foreskin and mucosal immune responses and understand the acceptability and safety of circumcision and rectosigmoid biopsies in high-risk South American MSM, the HIV Vaccine Trials Network (HVTN) conducted HVTN 914, a cohort study among MSM at high risk for HIV infection in Lima, Peru.
The study protocol and materials were approved by Institutional Bioethics Committee/Review Boards from Asociacion Civil Impacta Salud y Education (IMPACTA), US Naval Medical Research Unit No. 6, and the Fred Hutchinson Cancer Research Center. All study participants provided written informed consent prior to participation. Because the trial does not evaluate experimental interventions (circumcision and flexible sigmoidoscopy procedures are well established in clinical practice), the protocol team did not register the trial in ClinicalTrials.gov before participant enrollment. Upon request of the journal, the registration was completed (ClinicalTrials.gov NCT02630082). The study protocol and supporting TREND checklist are available as supplemental information; see S1 Protocol and S1 Checklist. The authors confirm that all ongoing and related trials for these interventions are registered.
In Lima, Peru, the MSM population has an HIV seroprevalence that is a 40-fold higher than the general male population , and high HIV risk MSM experience a 2.2–3.5% HIV incidence rate [11, 24]. HVTN 914 recruited healthy, HIV-seronegative, uncircumcised MSM 21 to 30 years old, who reported one or both criteria for HIV acquisition: 1) anal intercourse without condoms with one or more male or male to female transgender (MTF) partners in the preceding six months, and 2) reporting anal intercourse with two or more male or MTF partners in the preceding six months. Volunteers who had been in a monogamous relationship with an HIV-seronegative partner for > 6 months were excluded from participation due to low HIV transmission risk. Volunteers having sex with known HIV positive partners or a history of transactional sex in the past six months were ineligible due to concern for potential post-procedure exposures during healing, and/or economic difficulty adhering to abstinence recomendations.
Each participant’s healthy status was defined by medical history, physical examination, and having a normal hematology and coagulation profile. Men with a medical indication for circumcision (chronic balanitis or phimosis), foreskin covering less than half the glans, history of keloid scarring, or untreated clinical evidence of genitourinary or colonic infection were excluded.
Recruitment and Screening Procedures
A combination of social media and outreach was used to recruit participants from April 2011 through October 2012. No recruitment activity promoted male circumcision for HIV prevention, given the lack of evidence for MSM .
At the screening visit, a counselor obtained informed consent for study participation at the clinic. It included willingness to receive risk reduction counseling and HIV testing, described risks and benefits of circumcision and of sigmoidoscopy with biopsies, and described peri-procedure safety protocols. Participants were also informed about who had access to their personal information and study data. A different staff member conducted an assessment of understanding and clarified any misconceptions of key concepts. Screening included coagulation assessments, HIV-1/2 and Herpes Simplex Virus 2 (HSV-2) serology, and physical examination.
Enrollment and Follow-Up Study Visits
Fig 1 details screening, enrollment, and follow-up in Lima, Peru. The first 12 enrolled participants were scheduled to attend 10 visits during the 28 week study. They were tested for HIV-1/2 and HSV-2 at screening and at week 26. Six months into the protocol, another HIV test at week 16 was added for remaining participants to improve HIV surveillance. Syphilis was assessed at enrollment (week 0), Neisseria gonorrhea, and Chlamydia trachomatis were assessed at weeks 0 and 26. Participants diagnosed with any STI received treatment following standard guidelines .
Flow chart indicates the number and percentage (%) of participants at screening who underwent procedures at each study visit. Also included are those who left the study, and the reasons why. The dashed boxes indicate the visit added after 8 participants had already completed the protocol.
To ensure representation of MSM with different sexual behaviors  and differential HIV acquisition risk [26–28], we recruited a maximum of 5 men into each of 6 groups defined by self-reported anal sex preferences in preceding six months (insertive, receptive, or both) and by baseline HSV-2 serostatus (positive or negative). Questions used to assess sexual risk behavior, including those on anal sex preferences, were previously validated by study staff in an HIV sentinel surveillance study conducted with over 2,500 volunteers from Peru and Ecuador . Additionally, a 35-question computer-assisted self-interviewing (CASI) questionnaire was administered to assess tolerability and acceptability of interventional procedures (using a five-point Likert scale), occurrence of social impacts, hygiene patterns, and sexual behavior and satisfaction. The behavioral questions were previously evaluated for terminology and cultural competency through five in-depth interviews and two focus groups of MSM in Lima. Clinic staff was blinded to all CASI answers throughout the study.
At each visit except for week 16, a venous sample for serum (when indicated for STI serology testing), plasma and peripheral blood mononuclear cells (PBMC) was collected.
HIV risk reduction counseling followed local standard procedures, and included the provision of condoms and lubricants .Subjects also received counseling on abstinence from certain sexual activities, to help ensure safety after circumcision and rectosigmoid biopsy procedures.
In this observational study, clinic staff and participants were not blinded to the conduct of the interventional procedures, as all participants were to receive the same procedures and specific safety recommendations were discussed before and after each. Subjects were required to have no RAI nor put anything into their rectum from three days before through seven days after each sigmoidoscopy (with the exception of one Ringer´s lactate enema twelve hours before the procedure). Participants could undergo the procedures if no signs or symptoms of infection were detected on the day of procedure. Flexible sigmoidoscopies were conducted at weeks 2 and 27 by a certified gastroenterologist, after a second Ringer’s lactate enema was performed at an ambulatory surgery clinic. At each procedure, up to 25 sigmoid biopsy samples were collected with a 3.3mm diameter punch (six at 30cm from the anal margin, five at 25cm, six at 20cm, five at 15cm, and three at 10cm). Four additional biopsies were taken from the rectum. Clinical evaluations were performed at weeks 3 and 28 to confirm proper recovery, to assess stool changes, abdominal discomfort, signs and symptoms of bleeding or infection, and to provide medical clearance for resumption of RAI.
Circumcision was conducted by a certified urologist under sterile conditions at the ambulatory surgery clinic at week 4 post enrollment. The skin was checked for signs or symptoms of infection, and if negative, it was prepared with povidone-iodine before administration of local anesthesia via a dorsal penile nerve block with lidocaine and bupivacaine. The foreskin was removed using the sleeve or guided forceps procedures, per urologist discretion. Bleeding was controlled with electrocautery and skin edges were sutured with 4–0 absorbable sutures. Oral antibiotic prophylaxis post-circumcision was prescribed in all cases. Abstinence from masturbation and from insertive sexual activity (receiving oral sex, vaginal intercourse, or insertive anal intercourse [IAI]) was advised for six weeks after circumcision. Clinical evaluations were performed at weeks 5 and 10 to confirm proper healing, document any complications, and specifically at week 10 to provide medical clearance for resumption of insertive sexual activity.
In addition to reimbursement for a participant’s time, study staff facilitated on-time attendance to interventional procedures by providing dedicated transportation to ambulatory surgery facilities. All follow up from the study was concluded by May 2013.
Safety laboratory assays.
A medical evaluation and hemogram were conducted at each visit. The presence of serum antibodies to HIV-1/2, Treponema pallidum, and HSV-2 were assessed at specific study times as described above. HIV diagnosis followed the HVTN algorithm . The earliest date of detectable HIV-1 was determined by plasma HIV-1 RNA in stored samples at University of Washington Virology Laboratory (Seattle, WA). HSV-1/2 infection was diagnosed by a reactive EIA in samples with an index ratio >3.5 to improve the specificity of the assay . Positive and indeterminate serology HSV-1/2 test results were subject to HSV-2 western blot testing at University of Washington Virology Laboratory. Syphilis was diagnosed using RPR-TPHA at IMPACTA. A nucleic acid amplification test was used for the qualitative in vitro detection and differentiation of ribosomal RNA of C. trachomatis and N. gonorrhoeae at the King County Public Health Laboratories (Seattle, WA). All assays were validated for clinical diagnosis.
T-cell phenotypic analysis.
PBMC were isolated and cryopreserved following standard procedures , then shipped to the FHCRC laboratory in Seattle for later analysis. To evaluate T-cell activation, cryopreserved PBMC were thawed in R10 (RPMI 1640 supplemented with 2mML-glutamine, 25mM HEPES (Invitrogen) 10% FBS (Benchmark), 50μg/ml streptomycin, 50U/ml penicillin (Invitrogen), and 50U/ml benzonase (Novagen), and then washed and stained immediately. Sample viability was determined by LIVE/DEAD Aqua Cell Stain (Invitrogen). Surface staining used integrin β7 Phycoerythrin Cyanine (PECy) 5, Cluster of differentiation (CD) 49d (α4) Allophycocyanin (APC), C-C chemokine receptor type 5 (CCR5) PECy7, Cutaneous Leukocyte Antigen (CLA) biotin, CD14 V450 (all Becton Dickinson [BD] Biosciences), and CD103 (αE) Peridinin Chlorophyll Protein Cyanine (PerCPCy) 5.5 (Acris Antibodies). Intracellular staining used CD3 Ax700, Ki-67 Fluorescein Isothiocyanate FITC, B cell lymphoma 2 (Bcl-2) PE, Streptavidin APC-Cy7 (all BD), and CD4 Energy Coupling Dye ECD (Beckman Coulter). Flow cytometric analysis was performed using an LSR II flow cytometer (BD); post-acquisition analysis used FlowJo software (Tree Star, Inc). Personnel involved in thawing, staining and acquiring flow cytometric data were blinded from samples’ pre/post procedure status.
Lipopolysaccharide Binding Protein (LPB) quantitation.
To facilitate trial conduct, blood was collected on ACD tubes without fasting requirements. Plasma was isolated by two consecutive centrifugations: one at 300×g to separate plasma from cells, and a second at 800–1000×g to remove heme-containing contaminant. Plasma aliquots were stored at -80°C. Plasma LPB was quantified by ELISA (AntibodiesOnline.com) in the FHCRC laboratory with operators blinded as to pre/post procedure status. According to manufacturer instructions, normal LBP values range from 5–15 μg/ml, with an inter-assay variation coefficiency ranging from 9.8–17.8μg/ml depending on the concentration.
The privacy of participants was protected by assigning unique identifiers in place of the participant’s name on study data and specimens. In addition, each staff member signed a Confidentiality Agreement with the clinical site agreeing to protect confidential information.
Site staff recorded participant information on clinical research forms (CRFs). Independent data managers conducted quality control for data accurateness and consistency before faxing to HVTN Statistical and Data Managing Center. All study data has verifiable source documentation stored in secured locations.
Additionally, the study site was monitored under contract to the National Institute of Allergy and Infectious Diseases of the U.S. National Institutes of Health.
Regardless of enrollment group, participant results were analyzed as a single cohort. Descriptive statistics were used to summarize participant characteristics, tolerability and acceptability of interventional procedures, sexual behavior and occurrence of procedure-related events (by MedDRA preferred-term). Fisher’s exact tests were used to compare behavioral differences among dropout and retained participants.
Immune activation data (flow cytometry and LBP) were expressed as medians and interquartile ranges (IQR). Wilcoxon signed rank tests  were used to compare pre- and post-procedure immune activation, and participants with and without STIs. To prevent confounding associated with HIV disease process, immune activation data excludes all visits for two participants with confirmed HIV infection. Flow cytometry analysis used 165 PBMC samples, and excluded 3 with viability less than 70% after thawing and staining.
Statistical significance was defined at p<0.05. All analysis used SAS 9.2 (SAS Institute Inc, Cary, NC), and graphs were created using R 2.15.1 (R Foundation for Statistical Computing, Vienna, Austria).
Study Enrollment and Participation
Twenty-nine individuals (median age 24 years; all Latino/Hispanic) were enrolled in this cohort (Table 1). Twenty (69.0%) self-identified as homosexual, the rest as bisexual. In the six months preceding screening, participants had a median of eight male sex partners. Twenty-two (75.9%) stated they had practiced RAI with men (eight reported always using condoms) and 23 (79.3%) reported IAI with men (twelve reported always using condoms). Eight (27.6%) reported a female partner (median, 1) in the past six months (three reported using condoms during vaginal sex, two reported using condoms during anal sex). Fifteen participants reported having a steady job; eight reported being students (six with a part-time job); four non-students had part-time employment; and two were unemployed.
At week 0, HSV-2 infection was documented in 13 participants (44.8%). Syphilis seroreactivity was observed in five participants (17.3%), and in all cases it was diagnosed as a previously treated infection and required no further treatment. Of the 28 participants who underwent STI testing at week 0, three (10.3%) were diagnosed with chlamydia and gonorrhea, four (13.8%) with chlamydia, and one (3.4%) with gonorrhea. Two participants had urethral chlamydia (6.8%) in addition to rectal gonorrhea (n = 1) or rectal gonorrhea and chlamydia (n = 1). Four participants had rectal chlamydia only (13.6%), one had rectal gonorrhea only (3.4%), and one had rectal gonorrhea and chlamydia (3.4%) but no urethral infections. Of these, four participants diagnosed with chlamydia and/or gonorrhea dropped out before receiving diagnosis and treatment. Four received antibiotic treatment during the study period, after their first sigmoidoscopy (n = 4) and circumcision (n = 2) had been conducted. No participant tested positive for chlamydia and/or gonorrhea at week 26 before the second sigmoidoscopy.
At week 26, two participants had reactive HIV-1/2 serology tests. Retrospective testing revealed detectable plasma HIV-1 RNA in one participant at screening, indicating HIV exposure preceding the procedures. The second participant had no HIV-1 RNA detectable in plasma collected at week 10, suggesting he acquired HIV after the period of sexual abstinence post-circumcision. These participants were terminated from HVTN 914 per protocol and referred for HIV management.
Follow-Up and Tolerability of the Procedures
The main objective of this study was to assess the feasibility of performing safe and tolerable circumcision and rectosigmoid biopsy studies in Lima, Peru. To assess tolerability, we assessed pre- and post-procedure retention throughout the study visits.
Twenty-three participants (79.3%) underwent the first sigmoidoscopy at week 2 (Fig 1).Twenty-one participants (72.4%) completed the circumcision at week 4, including one who completed his circumcision 1 week later than the study plan, in accordance with a planned study deviation. Sixteen participants (55.2%) completed a second sigmoidoscopy at week 27.
Importantly, all participants who underwent any interventional procedures completed the associated post-procedure safety and tolerability evaluations (Fig 1). Retained participants reported very good/good tolerability and acceptance of sigmoidoscopies and circumcision (Fig 2A and 2B). Most participants leaving the study early also reported a good/very good opinion of the procedures at earlier visits (11/13), with only one participant reporting no opinion and another reporting a poor opinion of the sigmoidoscopies.
A) Opinion of the sigmoidoscopy procedures conducted at weeks 2 and 27. B) Opinion of the circumcision conducted at week 4. C) Perception of loss of work opportunities. D) Perception of reduction of income. E) Receptive anal behaviors (red lines) evaluated abstinence recommendations before and after the sigmoidoscopies at weeks 2 and 27, lasting through weeks 3 and 28 respectively. Receptive oral sex was (black lines) was also monitored but it was not part of abstinence recommendations. F) Insertive penile behaviors evaluated abstinence recommendations after circumcision at week 4, lasting through week 10. In A- F, S marks sigmoidoscopy visit, C indicates the circumcision visit. Confidence intervals were calculated using Wilson scores .
The most frequent dropout reason cited was difficulty adhering to the procedure schedule because participants had new job opportunities incompatible with convalescence (Fig 1). Some participants associated trial participation with loss of work opportunities (Fig 2C) and reduction in income (Fig 2D). Nevertheless, to examine if abstinence requirements encouraged drop-out, sexual behavior before the procedures (week 0) was compared post-hoc between drop-out and retained participants. There was no evidence that retained participants were less sexually active or more prone to using condoms than those who left the study early (data not shown). However, 91.7% (11/12) of participants who dropped-out had an STI at enrollment or screening visit, whereas only 43.8% (7/16) of those retained were STI-positive (p = 0.0159).
Procedure Related Events
To assess safety, procedure-related events and hemograms were monitored throughout the study. Clinical evaluations documented adverse drug reactions, edema, hemorrhage/hematoma, infection, pain, perforation or anatomic injury, and erection disabilities. No clinically significant changes in hematology values were observed over time (data not shown). Flexible sigmoidoscopies yielded few post-procedure events; all were mild and resolved on the procedure day (Table 2). Most circumcision-related events presented within the first 9 days post-circumcision (data not shown) and all resolved before study termination. The most common events were pain, edema, and hematoma/hemorrhage at the wound site, which were expected post-operatively and managed accordingly (Table 2).
No complications were reported by the six participants who underwent procedures in Lima before receiving their Chlamydia/gonorrhea PCR result (which was only validated in a US laboratory) and treatment. One participant with rectal chlamydia reported pain after the circumcision.
Adherence to Sexual Abstinence Recommendations
To assess safety of the procedures in a sexually active MSM population, we monitored HIV risk behaviors, and adherence to sexual abstinence recommendations using a CASI questionnaire at each study visit. There was a trend toward a decrease in the proportion of men reporting anal masturbation and RAI with male partners during the seven days pre- and post-sigmoidoscopies when receptive abstinence was advised, compared to the proportion reporting these behaviors outside of this period (Fig 2E). However, 17.4% (4/23 participants) and 12.5% (2/16 participants) continued to have RAI with men after the first and second sigmoidoscopies, respectively.
In contrast, participants did not report any penile masturbation, insertive oral sex, or IAI at week 5 and 10 study visits encompassing the six week post-circumcision abstinence period (Fig 2F). Only one participant of 23 reported insertive oral sex and vaginal intercourse without a condom during the week preceding his week 10 study visit, the last week of the advised post-circumcision abstinence period (Fig 2F).
To further assess tolerability, every participant reported their level of satisfaction with regard to sexual behaviors carried out in the week previous to the clinic visit. In the three weeks previous to the first flexible sigmoidoscopy, 19/24 RAI activities (79.1%) were reported as satisfying or very satisfying. In the four visits posterior to the recommended re-initiation of RAI, 18/20 (90%) RAI events were satisfying or very satisfying. Previous to flexible sigmoidoscopies, 3/4 rectal masturbation activities (75%) were satisfying or very satisfying, whereas 2/3 (66.6%) reported satisfaction after the flexible sigmoidoscopy procedure.
In the five weeks previous to circumcision, 100% of sex with women (21 vaginal events, 9 insertive oral events, and 4 IAI) was reported as satisfying or very satisfying; that percentage did not change after resuming insertive sex post-circumcision (10 vaginal events, 7 insertive oral, and 4 IAI). Of the 25 IAI with men reported before circumcision, 18 were reported as satisfying or very satisfying (72%), whereas 100% (6/6) reported satisfaction with these insertive behaviors in the 3 visits that followed the healing period from circumcision. 17 out of 22 insertive oral sex events with men (77%) reported being satisfied or very satisfied, whereas 6/6 (100%) reported satisfaction in the 3 visits that followed the healing period from circumcision. 81.3–87.5% of individuals reported being satisfied or very satisfied with penile masturbation in the visits previous to the circumcision (n = 16–21), whereas 66.7–75% reported satisfaction in the visits posterior to the abstinence from insertive activities (n = 8–12).
Systemic Immune Activation
Immune activation of CD4+ T cells was evaluated based on expression of CCR5 and Ki-67+Bcl2low, as these have been associated with high-risk sexual behavior, HIV-1 transmission, and in vitro infectivity [35, 36]. Within CD4+T cells we also assessed CLA, αEβ7, and α4β7, as surrogates of cell recruitment and migration to genital and intestinal surfaces [37, 38] (Fig 3A and S1 Table). No changes in CLA+, αEβ7+ or Ki-67+Bcl2low CD4+ T cells were observed following the sigmoidoscopies (S1 Table). However, the frequency of circulating CCR5+CD4+ T cells was slightly but significantly elevated one week post-sigmoidoscopy (week 3 median: 26.0%, IQR: 24.3–30.5%) compared to the day of the procedure (week 2 median: 24.2%, IQR: 19.5–27.7%; p = 0.0053; Fig 3B). Similar results were observed before (week 27 median: 23.6%, IQR: 21.2–26.5%) and after the second sigmoidoscopy (week 28 median: 27.3%, IQR: 21.9–29.9%; p = 0.0256; data not shown). A similarly small but significant increase in the frequency of α4β7+CD4+ T cells was also detected following the first sigmoidoscopy (median week 2: 7.7%, IQR: 6.4–8.7%; median week 3: 9.1%, IQR: 7.4–10.2%; p = 0.0070; Fig 3C). Together the results suggest sigmoidoscopies can lead to small, acute increases in CCR5+ and α4β7+CD4+ T cells in blood, without affecting other markers. No differences in the CCR5+CD4+ T (p = 1.000) or α4β7+CD4+ T cells (p = 1.000) were seen among men with STIs and men with negative STI tests.
A) Representative flow cytometry panels indicating live, CD3+, CD14-, CD4+ cells and the gating scheme for CCR5+, Ki-67+Bcl-2low, α4β7+, CD103+β7+, and CLA+ cells. 3 samples of 168 were excluded due to cell viability less than 70%.B) Percent CCR5+CD4+ T cells in blood over time. C) Percent α4β7+CD4+ T cells in blood over time. D) LBP concentration in serum over time. In panels B-D, dots indicate individual measurements, with box plots summarizing median and IQR; whiskers represent the range. S marks sigmoidoscopy visit, C indicates the circumcision visit. P-values are from Wilcoxon tests used to examine difference between sigmoidoscopy visits (week 2 and 27) and follow-up at weeks 3 and 28, respectively; only significant values are shown. Similar analysis was conducted comparing the circumcision visit (week 4) with the follow-up period including weeks 5 and 10.
CCR5+CD4+T-cell frequencies returned to baseline by two weeks post-sigmoidoscopy (week 4 median: 24.9%, IQR 20.3–28.1%; p = 0.1231; Fig 3B), when others have shown complete healing of the mucosal compartment post-sigmoidoscopies of men practicing RAI . Similar findings were observed among α4β7+CD4+ T cells (Fig 3C, p = 0.5459). Thus, the transient up-regulation of markers at week 3 returned to baseline within 2 weeks of biopsy.
Circumcision triggered more long-lasting changes in the frequencies of HIV target cells. Compared to week 4, a trend toward increased CCR5+CD4+ T cells was observed one week post-circumcision (week 5 median: 26.0%, IQR 20.5–30.5%; p = 0.1042), and a significant increase was detected at the end of the recommended abstinence period (week 10 median: 27.2%, IQR: 24.3–33.0%; p = 0.0001; Fig 3B).
Increases in α4β7+CD4+ T cells were also detected at weeks 5 (median: 9.9%, IQR: 8.4–10.8%; p = 0.0005) and 10 (median: 9.2%, IQR: 7.6–10.9%; p = 0.0062) compared to week 4 (median: 7.9%, IQR: 6.8.-9.0%; Fig 3C). At week 10, there was also a significant increase in Bcl2lowKi-67highCD4+ T cells compared to week 4 (p = 0.0204; S1 Table). No increases in αEβ7+or CLA+CD4+ T cells were observed (S1 Table). All circumcision-associated changes in CD4+ T-cell activation remained significant after excluding participants reporting multiple procedure-related events.
Significant increases in LBP, a proposed sensitive acute marker for bacterial infection [39, 40], were detected one week post-circumcision (week 5 median: 18.2μg/ml, IQR: 15.0–23.1μg/ml) when compared to the day of the procedure (week 4 median: 14.4 μg/ml, IQR: 10.6–21.2μg/ml; p = 0.0121; Fig 3D). LBP concentrations did not change after sigmoidoscopies.
This study was designed to be representative of a potential genital and rectal tissue sampling schedule after delivery of an HIV vaccine. Although not powered to assess the incidence of rare procedure-related events, our study suggests that circumcision and sigmoidoscopies can be performed safely in MSM with high-risk sexual behaviors. First, all participants who underwent interventional procedures completed the associated post-procedure evaluations. Second, hematological tests did not show clinically significant signs of blood loss, despite conduct of the procedures simultaneous with phlebotomy. And third, rates of complications such as infection, adverse drug reactions, problems with urination or erectile dysfunction were similar to the 0.5–10% adult male circumcision adverse event rates [20, 41–43] or the 0.004–0.00001% colonoscopy complications [44, 45] reported in the literature. From a safety standpoint, our data demonstrates the feasible implementation of these genital and mucosal collections in MSM at high HIV risk.
Interestingly, our data suggests that circumcision and sigmoidoscopy triggered activation in peripheral HIV target cells, which can be detected throughout the sexual abstinence periods. Systemically, the procedures were followed by an 8.2–15.3% increase in CCR5+CD4+T cells and a 16.8–32.9% increase in α4β7+CD4+ T cells compared to pre-procedure samples. As in other elective operations , a slight LBP increase was detected post-circumcision. Future studies combining these procedures and peripheral activation endpoints should time systemic assessments outside the procedure recovery window.
Because six participants had asymptomatic HSV-2, rectal chlamydia and/or gonorrhea at the time of their flexible sigmoidoscopies, we considered whether some of the systemic activation could be attributed to the rectal infections. The data argues against this possibility, as there was no difference among the CCR5+CD4+ and α4β7+CD4+ T-cell increases among participants with an STI versus those with negative STI tests. Additionally, an increase in CCR5+CD4+ T cells was observed again after the second sigmoidoscopy, a timepoint where all participants tested negative for the bacterial STIs. We conclude that the changes in peripheral activation can take place in men with or without asymptomatic rectal infections, although in this cohort we were not be powered to study each STI independently.
Peripheral increases in target cells may augment HIV susceptibility , Yet, as in studies of medical male circumcision, in this small study, no HIV infections were associated with healing periods with increased T-cell activation .
This may reflect how the combination of risk reduction counseling, frequent HIV testing, and abstinence recommendations can support the safe conduct of mucosal sampling in high risk MSM. After circumcision, when the highest increases in immune activation were observed, 20/21 participants reported abstaining from insertive activities for 6 weeks post-procedure, as recommended. Up to 17.4% of participants engaged in receptive anal behaviors within a week of the sigmoidoscopies, suggesting abstinence from receptive behaviors may be more challenging in this MSM population. In fact, receptive sexual behaviors also resumed by week 6 post-circumcision. While overall rates of adherence to insertive abstinence in this trial are consistent with the 78–95% rates seen in recent medical male circumcision trials in Africa [15–17, 19], further work is needed to identify incentives and barriers to compliance with receptive abstinence recommendations in high-risk MSM.
The reasons for differential compliance with insertive versus receptive abstinence are unknown. Visualization of the circumcision wound and penile sensitivity might have discouraged insertive activities during healing, whereas minimal discomfort and no visualization of the intestinal surface may have reduced compliance with post-sigmoidoscopy abstinence. Also, MSM in Lima might exert greater choice over insertive than receptive sexual behaviors .
Interestingly, screening data suggests many MSM are willing to undergo invasive procedures as part of an HIV prevention trial. There was a surplus of available volunteers for this trial: 27.8% of eligible volunteers were unable to be enrolled simply based on their HSV-2 sero-positivity or sex role criteria. Our data also reveals a positive evaluation of the circumcision and sigmoidoscopy procedures among the participants who were retained, as well as among the majority (85%) of those who dropped out. We also did not find evidence of changes in sexual satisfaction before the procedures and after resuming sexual activity. However, the low retention prevents us from fully interpreting the tolerability and satisfaction data, as there may be unmeasured barriers to participation in our study design, especially for the men with a previous STI history.
The major reported barriers to participation in the trial were the perceived loss of work opportunities and reduction in income. Repeated post-procedure clinic visits and recovery periods may have been disruptive to employment responsibilities. To provide more flexibility to participants, it will be important to define the kinetics of vaccine and microbicide effects in the mucosa to determine whether wider visit windows could be utilized without compromising mucosal endpoints.
In conclusion, this small study demonstrated that sexually active at-risk MSM in Lima, Peru were willing to participate in a trial of mucosal collections, including circumcision. The study design allowed intensive follow-up and post-procedure counseling and demonstrated the safety of the procedures in this population. Although the clinical implications of the increase in peripheral immune activation markers following the procedures are unknown, this finding reinforces the need for ongoing risk reduction counseling and support for post-procedure abstinence recommendations. This study also generated intriguing hypotheses regarding the acceptability of these procedures and their associated abstinence recommendations, which may inform improved retention efforts in future trials utilizing these procedures in a high-risk MSM population.
S1 Protocol. HVTN 914: A cohort study in Lima, Peru to evaluate the feasibility of measuring immune responses and activation levels in the foreskin and rectosigmoid mucosa in HIV-negative, uncircumcised men who have sex with men and who are at high risk for HIV acquisition.
Thanks to the study volunteers for their participation; to Dr. V.M. Barrientos, Dr. H. Agurto and the clinical and laboratory staff at IMPACTA for follow up of study participants; to Kevin Hawkins and Terri Stewart for LSRII support; to Nikki Klat for advice on the LBP assay; and to Stephen Voght for help with manuscript preparation. Special thanks to Nicole Frahm, Gina Escamilla, Tamra Madenwald, Maria Fernandez-Maldonado, and Cheryl de Boer for help with project and data management. We thank the James B. Pendleton Charitable Trust for their generous equipment donation.
Research reported in this publication was supported by the National Institute of Allergy and Infectious Diseases of the U.S. National Institutes of Health. The content is the responsibility of the authors and does not necessarily represent the official views, policy or position of the U.S. National Institutes of Health, the Department of the Navy, the Department of Defense, or the U.S. Government. Silvia Montano and Edith Swann are employees of the U.S. Government who contributed to this work as part of her official duties. No copyright protection is available for their work as defined by Titles 17 U.S.C. §105 and §101.
HVTN 914 Protocol Team: Javier R. Lama, Maria P. Lemos, Shelly T. Karuna, Yunda Huang, Dale Lawrence, M. Juliana McElrath, Jorge Sanchez, John Hural, Eric Ramos, Stephen C. De Rosa, Susan Buchbinder, Silvia M. Montano, Niles Eaton, Jin Bae, Heather Hildebrant, Rachel McClennen, Drienna Holman, Maria Fernandez-Maldonado, Carter Bentley, Patricia Espinoza, Carrie Schonwald, Karisse Torres, Patricia Segura, Pat Farrell, Shannon P. Grant, Raphael Gottardo, Gina Escamilla, Jacqueline Fitzpatrick, Cheryl de Boer, David Chambliss, Donna Smith, Jill Zeller, Tamra Madenwald, Carissa Karg, Eva Chung, On Ho, Robert de la Grecca, Maria Fernandez Maldonado, Jesus Peinado, Cecilia Chang, Sonia Minaya, Richard Negron, Jesus Jurupe, Maria Esther Gamero, Monica Nieto, Juanita Calderon, Mayu Rivera, and Nilda Gadea.
- Conceived and designed the experiments: MPL JL SK PS ML SB.
- Performed the experiments: ML JL ES SM.
- Analyzed the data: ML SG PS MPL SDR SB.
- Contributed reagents/materials/analysis tools: CG SM MJM SDR SG JS JL.
- Wrote the paper: ES SB SK JL ML JS MJM.
- 1. Vittinghoff E, Douglas J, Judson F, McKirnan D, MacQueen K, Buchbinder SP. Per-contact risk of human immunodeficiency virus transmission between male sexual partners. American journal of epidemiology. 1999;150(3):306–11. Epub 1999/08/03. pmid:10430236.
- 2. Patel P, Borkowf CB, Brooks JT, Lasry A, Lansky A, Mermin J. Estimating per-act HIV transmission risk: a systematic review. AIDS. 2014;28(10):1509–19. Epub 2014/05/09. pmid:24809629.
- 3. Belyakov IM, Isakov D, Zhu Q, Dzutsev A, Berzofsky JA. A novel functional CTL avidity/activity compartmentalization to the site of mucosal immunization contributes to protection of macaques against simian/human immunodeficiency viral depletion of mucosal CD4+ T cells. J Immunol. 2007;178(11):7211–21. Epub 2007/05/22. pmid:17513770.
- 4. Perreau M, Welles HC, Harari A, Hall O, Martin R, Maillard M, et al. DNA/NYVAC vaccine regimen induces HIV-specific CD4 and CD8 T-cell responses in intestinal mucosa. Journal of virology. 2011;85(19):9854–62. Epub 2011/07/22. pmid:21775454; PubMed Central PMCID: PMC3196391.
- 5. Ferre AL, Lemongello D, Hunt PW, Morris MM, Garcia JC, Pollard RB, et al. Immunodominant HIV-specific CD8+ T-cell responses are common to blood and gastrointestinal mucosa, and Gag-specific responses dominate in rectal mucosa of HIV controllers. Journal of virology. 2010;84(19):10354–65. Epub 2010/07/30. pmid:20668079; PubMed Central PMCID: PMC2937770.
- 6. Yang OO, Ibarrondo FJ, Price C, Hultin LE, Elliott J, Hultin PM, et al. Differential blood and mucosal immune responses against an HIV-1 vaccine administered via inguinal or deltoid injection. PloS one. 2014;9(2):e88621. Epub 2014/02/22. pmid:24558403; PubMed Central PMCID: PMC3928250.
- 7. Lemos MP, Karuna ST, Mize GJ, Fong Y, Montano SM, Ganoza C, et al. In men at risk of HIV infection, IgM, IgG1, IgG3, and IgA reach the human foreskin epidermis. Mucosal immunology. 2015. Epub 2015/10/29. pmid:26509877.
- 8. Li Q, Zeng M, Duan L, Voss JE, Smith AJ, Pambuccian S, et al. Live simian immunodeficiency virus vaccine correlate of protection: local antibody production and concentration on the path of virus entry. J Immunol. 2014;193(6):3113–25. Epub 2014/08/20. pmid:25135832; PubMed Central PMCID: PMC4157131.
- 9. Zeng M, Smith AJ, Shang L, Wietgrefe SW, Voss JE, Carlis JV, et al. Mucosal Humoral Immune Response to SIVmac239nef Vaccination and Vaginal Challenge. J Immunol. 2016. Epub 2016/02/13. pmid:26864031.
- 10. Hansen SG, Wu HL, Burwitz BJ, Hughes CM, Hammond KB, Ventura AB, et al. Broadly targeted CD8+ T cell responses restricted by major histocompatibility complex E. Science. 2016. Epub 2016/01/23. pmid:26797147.
- 11. Buchbinder SP, Mehrotra DV, Duerr A, Fitzgerald DW, Mogg R, Li D, et al. Efficacy assessment of a cell-mediated immunity HIV-1 vaccine (the Step Study): a double-blind, randomised, placebo-controlled, test-of-concept trial. Lancet. 2008;372(9653):1881–93. Epub 2008/11/18. pmid:19012954; PubMed Central PMCID: PMC2721012.
- 12. Francis SC, Hou Y, Baisley K, van de Wijgert J, Watson-Jones D, Ao TT, et al. Immune Activation in the Female Genital Tract: Expression Profiles of Soluble Proteins in Women at High Risk for HIV Infection. PloS one. 2016;11(1):e0143109. Epub 2016/01/28. pmid:26814891; PubMed Central PMCID: PMC4729472.
- 13. Passmore JA, Jaspan HB, Masson L. Genital inflammation, immune activation and risk of sexual HIV acquisition. Current opinion in HIV and AIDS. 2016;11(2):156–62. Epub 2015/12/03. pmid:26628324.
- 14. Masson L, Arnold KB, Little F, Mlisana K, Lewis DA, Mkhize N, et al. Inflammatory cytokine biomarkers to identify women with asymptomatic sexually transmitted infections and bacterial vaginosis who are at high risk of HIV infection. Sexually transmitted infections. 2015. Epub 2015/10/30. pmid:26511781.
- 15. Auvert B, Taljaard D, Lagarde E, Sobngwi-Tambekou J, Sitta R, Puren A. Randomized, controlled intervention trial of male circumcision for reduction of HIV infection risk: the ANRS 1265 Trial. PLoS medicine. 2005;2(11):e298. Epub 2005/10/20. pmid:16231970; PubMed Central PMCID: PMC1262556.
- 16. Gray RH, Kigozi G, Serwadda D, Makumbi F, Watya S, Nalugoda F, et al. Male circumcision for HIV prevention in men in Rakai, Uganda: a randomised trial. Lancet. 2007;369(9562):657–66. Epub 2007/02/27. pmid:17321311.
- 17. Bailey RC, Moses S, Parker CB, Agot K, Maclean I, Krieger JN, et al. Male circumcision for HIV prevention in young men in Kisumu, Kenya: a randomised controlled trial. Lancet. 2007;369(9562):643–56. Epub 2007/02/27. pmid:17321310.
- 18. Wiysonge CS, Kongnyuy EJ, Shey M, Muula AS, Navti OB, Akl EA, et al. Male circumcision for prevention of homosexual acquisition of HIV in men. Cochrane Database Syst Rev. 2011;(6):CD007496. Epub 2011/06/17. pmid:21678366.
- 19. Mehta SD, Gray RH, Auvert B, Moses S, Kigozi G, Taljaard D, et al. Does sex in the early period after circumcision increase HIV-seroconversion risk? Pooled analysis of adult male circumcision clinical trials. AIDS. 2009;23(12):1557–64. Epub 2009/07/03. pmid:19571722; PubMed Central PMCID: PMC2772053.
- 20. Kigozi G, Gray RH, Wawer MJ, Serwadda D, Makumbi F, Watya S, et al. The safety of adult male circumcision in HIV-infected and uninfected men in Rakai, Uganda. PLoS medicine. 2008;5(6):e116. Epub 2008/06/06. pmid:18532873; PubMed Central PMCID: PMC2408615.
- 21. Anton PA, Cranston RD, Kashuba A, Hendrix CW, Bumpus NN, Richardson-Harman N, et al. RMP-02/MTN-006: A phase 1 rectal safety, acceptability, pharmacokinetic, and pharmacodynamic study of tenofovir 1% gel compared with oral tenofovir disoproxil fumarate. AIDS research and human retroviruses. 2012;28(11):1412–21. Epub 2012/09/05. pmid:22943559; PubMed Central PMCID: PMC3484811.
- 22. McGowan I, Elliott J, Cortina G, Tanner K, Siboliban C, Adler A, et al. Characterization of baseline intestinal mucosal indices of injury and inflammation in men for use in rectal microbicide trials (HIV Prevention Trials Network-056). J Acquir Immune Defic Syndr. 2007;46(4):417–25. Epub 2007/09/25. pmid:17891044.
- 23. Sanchez J, Lama JR, Kusunoki L, Manrique H, Goicochea P, Lucchetti A, et al. HIV-1, sexually transmitted infections, and sexual behavior trends among men who have sex with men in Lima, Peru. J Acquir Immune Defic Syndr. 2007;44(5):578–85. Epub 2007/02/07. pmid:17279049.
- 24. Sanchez J, Lama JR, Peinado J, Paredes A, Lucchetti A, Russell K, et al. High HIV and ulcerative sexually transmitted infection incidence estimates among men who have sex with men in Peru: awaiting for an effective preventive intervention. J Acquir Immune Defic Syndr. 2009;51 Suppl 1:S47–51. Epub 2009/05/16. pmid:19384102; PubMed Central PMCID: PMC2725017.
- 25. Workowski KA, Berman S, Centers for Disease C, Prevention. Sexually transmitted diseases treatment guidelines, 2010. MMWR Recommendations and reports: Morbidity and mortality weekly report Recommendations and reports / Centers for Disease Control. 2010;59(RR-12):1–110. Epub 2010/12/17. pmid:21160459.
- 26. Freeman EE, Weiss HA, Glynn JR, Cross PL, Whitworth JA, Hayes RJ. Herpes simplex virus 2 infection increases HIV acquisition in men and women: systematic review and meta-analysis of longitudinal studies. AIDS. 2006;20(1):73–83. Epub 2005/12/06. pmid:16327322.
- 27. Wald A, Link K. Risk of human immunodeficiency virus infection in herpes simplex virus type 2-seropositive persons: a meta-analysis. The Journal of infectious diseases. 2002;185(1):45–52. Epub 2002/01/05. pmid:11756980.
- 28. Sanchez J, Sal YRVG, Hughes JP, Baeten JM, Fuchs J, Buchbinder SP, et al. Male circumcision and risk of HIV acquisition among MSM. AIDS. 2011;25(4):519–23. Epub 2010/11/26. pmid:21099672; PubMed Central PMCID: PMC3120051.
- 29. ClinicalTrials.gov. Estimating the Incidence of HIV Infection among Men who Have Sex with Men in Peru and Ecuador. http://www.clinicaltrials.gov/ct2/show/NCT002793312007 [cited 2016 January 12th].
- 30. Ministerio de Salud del Peru. Documento Técnico: Consejería en ITS / VIH y SIDA. April 23 ed. Lima2009.
- 31. Cooper CJ, Metch B, Dragavon J, Coombs RW, Baden LR, Force NHVTNV-IST. Vaccine-induced HIV seropositivity/reactivity in noninfected HIV vaccine recipients. JAMA: the journal of the American Medical Association. 2010;304(3):275–83. Epub 2010/07/20. pmid:20639561; PubMed Central PMCID: PMC3086635.
- 32. Ashley RL. Performance and use of HSV type-specific serology test kits. Herpes: the journal of the IHMF. 2002;9(2):38–45. Epub 2002/07/11. pmid:12106510.
- 33. Kierstead LS, Dubey S, Meyer B, Tobery TW, Mogg R, Fernandez VR, et al. Enhanced rates and magnitude of immune responses detected against an HIV vaccine: effect of using an optimized process for isolating PBMC. AIDS Res Hum Retroviruses. 2007;23(1):86–92. Epub 2007/02/01. pmid:17263637.
- 34. Wilcoxon F. Individual comparisons by ranking methods. Biometrics Bulletin. 1945;1(6):pp. 80–3.
- 35. Paxton WA, Liu R, Kang S, Wu L, Gingeras TR, Landau NR, et al. Reduced HIV-1 infectability of CD4+ lymphocytes from exposed-uninfected individuals: association with low expression of CCR5 and high production of beta-chemokines. Virology. 1998;244(1):66–73. Epub 1998/05/15. pmid:9581779.
- 36. Thomas SM, Tse DB, Ketner DS, Rochford G, Meyer DA, Zade DD, et al. CCR5 expression and duration of high risk sexual activity among HIV-seronegative men who have sex with men. AIDS. 2006;20(14):1879–83. Epub 2006/09/07. pmid:16954729; PubMed Central PMCID: PMC1630600.
- 37. Picker LJ. Immunopathogenesis of acute AIDS virus infection. Current opinion in immunology. 2006;18(4):399–405. Epub 2006/06/07. pmid:16753288.
- 38. Wagner N, Lohler J, Kunkel EJ, Ley K, Leung E, Krissansen G, et al. Critical role for beta7 integrins in formation of the gut-associated lymphoid tissue. Nature. 1996;382(6589):366–70. Epub 1996/07/25. pmid:8684468.
- 39. Gaini S, Koldkjaer OG, Pedersen C, Pedersen SS. Procalcitonin, lipopolysaccharide-binding protein, interleukin-6 and C-reactive protein in community-acquired infections and sepsis: a prospective study. Crit Care. 2006;10(2):R53. pmid:16569262; PubMed Central PMCID: PMC1550885.
- 40. Schumann RR, Zweigner J. A novel acute-phase marker: lipopolysaccharide binding protein (LBP). Clin Chem Lab Med. 1999;37(3):271–4. pmid:10353471.
- 41. Perera CL, Bridgewater FH, Thavaneswaran P, Maddern GJ. Safety and efficacy of nontherapeutic male circumcision: a systematic review. Annals of family medicine. 2010;8(1):64–72. Epub 2010/01/13. pmid:20065281; PubMed Central PMCID: PMC2807391.
- 42. Kigozi G, Musoke R, Kighoma N, Kiwanuka N, Makumbi F, Nalugoda F, et al. Safety of medical male circumcision in human immunodeficiency virus-infected men in Rakai, Uganda. Urology. 2014;83(2):294–7. Epub 2013/11/30. pmid:24286598.
- 43. Williams N, Kapila L. Complications of circumcision. The British journal of surgery. 1993;80(10):1231–6. Epub 1993/10/01. pmid:8242285.
- 44. Anderson ML, Pasha TM, Leighton JA. Endoscopic perforation of the colon: lessons from a 10-year study. The American journal of gastroenterology. 2000;95(12):3418–22. Epub 2001/01/11. pmid:11151871.
- 45. Levin TR, Conell C, Shapiro JA, Chazan SG, Nadel MR, Selby JV. Complications of screening flexible sigmoidoscopy. Gastroenterology. 2002;123(6):1786–92. Epub 2002/11/28. pmid:12454834.
- 46. Hiki N, Berger D, Dentener MA, Mimura Y, Buurman WA, Prigl C, et al. Changes in endotoxin-binding proteins during major elective surgery: important role for soluble CD14 in regulation of biological activity of systemic endotoxin. Clinical and diagnostic laboratory immunology. 1999;6(6):844–50. Epub 1999/11/05. pmid:10548574; PubMed Central PMCID: PMC95786.
- 47. Goodreau SM, Peinado J, Goicochea P, Vergara J, Ojeda N, Casapia M, et al. Role versatility among men who have sex with men in urban Peru. J Sex Res. 2007;44(3):233–9. pmid:17879166.
- 48. Agresti A, Coull BA. Approximate is better than "exact" for interval estimation of binomial proportions. Am Stat. 1998;52(2):119–26. pmid:ISI:000074022800005.