Introduction

Prospective studies are key study designs for assessing the risks of certain determinants in acquiring a specific disease in an attempt to unravel health mechanisms that are widely applicable [1,2]. Women’s participation in prospective studies is fundamental to understanding their health mechanisms [3]. Despite a continued emphasis by regulatory bodies, health institutions, research funding organizations, and scientific journals, the inclusion of women in research is not easy [4,5]. Significant benefits for prospective research in women can be reaped by sharing best practices on inclusion and retention.

First and foremost, a prospective study should have high internal validity to generate unconfounded insights [1]. Understanding a study’s internal validity is essential to judging its quality. Internal validity may be compromised when it is difficult to include sufficient participants. Low study power may result in associations that are spurious (“false”), imprecise (with wide confidence intervals), or missed altogether [6,7], and it becomes difficult to replicate the study. Similar threats to internal validity arise when insufficient participants are retained, i.e., when a large portion of included women is lost to follow-up or withdraws from the study (both called attrition). When attrition disproportionally affects a subset of the study population, it may lead to selection bias and an underestimation or an overestimation of the associations [1,8].

When a prospective study has high internal validity, the risks obtained may be applicable to a broad population of women and the study population does not need to have high external validity, i.e., be a representative sample [9–11]. Usually, representative frequency distributions are derived from other types of research, i.e., population-based surveys [9]. In practice, it has proven to be logistically difficult and cost ineffective to include and follow a representative sample in a study that, first and foremost, is designed to achieve high internal validity [12]. Still, insights in possible sampling bias and the external validity of a prospective study are useful to judge the applicability of a study’s findings. Internal and external validity reporting has improved with the STROBE and CONSORT guidelines [13] and is increasingly encouraged in the scientific journals [14].

Here, we report on the FemCure study, the first prospective research that systematically addresses rectal Chlamydia trachomatis (CT) in women [15]. FemCure included 560 women from three Dutch STI clinics and followed them for 3 months with systematic rectal and vaginal CT assessment. Chlamydia infection is the most commonly reported treatable bacterial sexually transmitted infection (STI) in high-income countries [16,17]. CT disproportionally affects women in terms of its occurrence and burden of sequelae, i.e., pelvic inflammatory disease, ectopic pregnancy, and tubal infertility [18]. The main treatments are doxycycline and azithromycin [19,20], although, the efficacy of azithromycin in the treatment of CT is debatable [21]. FemCure was set up to improve our understanding of posttreatment rectal and vaginal CT detection in women, by assessing risks due to sexual exposure (horizontal transmission) or exposure from adjacent body sites (self-infection), and by assessing the role of possibly suboptimal treatment effectiveness [15].

Here, we describe the multifaceted strategies that were implemented to include and retain a group of women attending STI clinics in a prospective cohort study. Insights in potential sampling bias and external validity of the FemCure study population as well as potential selection bias of the retained sample (internal validity) are presented by analyzing the factors associated with nonparticipation and attrition.

Materials and methods

Study design

STI clinic setting for the study.

The study population originated from the STI clinics of the Public Health Services in South Limburg, Rotterdam, and Amsterdam, which represent almost half of all STI clinic consultations in the Netherlands. Clinics apply the same care procedures [19] but vary in their annual number of clients (6,000–45,000) and urbanization degree (rural to highly urban) [22]. According to European guidelines [19], all women were tested for vaginal CT, and they were also rectally tested, when reporting anal sex or anal symptoms. Women who tested positive for rectal infection were treated with a seven-day course of oral doxycycline 100 mg twice daily [19]. Women who tested positive for vaginal infection, and who either tested negative for rectal infection or did not undergo the rectal test, received a 1 g single azithromycin oral dose. The first doxycycline and azithromycin doses were directly observed. In this context, we set up a cohort study with 3 months of follow-up (Fig 1) [15].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Design of the prospective cohort study, FemCure.

https://doi.org/10.1371/journal.pone.0230413.g001

Target population.

The target population of FemCure were women, who were 18 years or older, diagnosed with a vaginal or rectal CT infection during the inclusion period (April 2016 until September 2017), and negative for HIV, syphilis, and Neisseria gonorrhoeae (NG). FemCure aimed to have at least 400 women with complete follow-up, for sufficient power for the planned analyses [15]. To maximize participation, multifaceted strategies focused on helpful themes, identified previously [23–31], were simultaneously implemented (Table 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Themes and strategies (derived from previous studies 23–31) employed in the prospective multicenter FemCure study, aimed to maximize women’s participation.

https://doi.org/10.1371/journal.pone.0230413.t001

Who was preselected for inclusion?

Study information was provided to the women at least a day before the inclusion visit (the treatment consultation), to give them time to make an informed decision. Therefore, a standardized text and link to the study’s website were communicated via the routine channels that clinics use for contacting patients (i.e., online and by text message). Women from the target population could be preselected at the treatment consultation when the following applied: (a) a research nurse was available at that moment to handle inclusion (the predefined research capacity for handling study visits was four to six nurses per clinic; nurses were required to be available part-time for study duties, a priori restricting the number of patients who could be preselected); (b) study criteria applied (no recent antibiotic use, not pregnant, received study information at least 1 day before participation, understand the Dutch language, able to complete the study [e.g., not going abroad during the study, living close to the clinic]); (c) the patient was willing to, at that moment, be fully informed about the study and, when accepting inclusion, to comply with the study procedures; (d) the patient was willing to be transferred to a research nurse for possible inclusion in case if the patient was not already managed by a research nurse.

Who was included in the study?

For preselected women, the research nurse provided information and checked study criteria. A calendar was used to plan follow-up clinic visits. When a patient accepted the study procedures, providing written informed consent, the research nurse handled inclusion and provided treatment. Reasons for noninclusion were scored.

Who was followed after inclusion?

All included women were followed, but they could withdraw from the study or become lost-to-follow-up (both: attrition). Reasons for study withdrawal were recorded.

Data collection.

Clinical data (such as baseline characteristics of the target population) originated from the electronic clinic patient registries. Participants in the study self-collected rectal and vaginal samples for CT testing and completed structured online self-administered questionnaires (on antibiotic use, symptoms, and sexual behavior) at inclusion (week 0), and at weeks 4, 8, and 12 at the STI clinic, and at weeks 1, 2, 6, and 10 at home (Fig 1). A test package with clear instructions was provided for self-collection of samples at home. Week 12 data also included a study evaluation.

Results

Target population

In the study period, 4,916 women who were eligible for preselection came to the participating clinics (Table 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Baseline characteristics distributions in the target population, (not) preselected women, (not) included women, and women retained in the study, FemCure.

https://doi.org/10.1371/journal.pone.0230413.t002

Retention

Among the included women, 427 (76.2%) were retained (Fig 2, Table 2). Together, these 427 women collected data 3,416 scheduled times. By the seventeenth scheduled time (0.5%, 15 women), samples and questionnaire data were not provided in 3 of these moments. Further, at 14 moments, questionnaire data were not provided.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2.

Kaplan–Meier survival function (line) with 95% confidence intervals (grey area) on the probability of being retained, and the number of women retained (“Number at Risk”) in the FemCure study.

https://doi.org/10.1371/journal.pone.0230413.g002

Factors associated with nonparticipation

Associated factors for non-preselection from the target population.

In the multivariate analyses (Table 3), factors independently associated with non-preselection were study site Amsterdam (compared to South Limburg), non-Western migration background, and no STI history. A medium educational level (compared to a low educational level) was inversely associated.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Odds ratios for the association between characteristics and non-preselection among STI clinic women target population, and noninclusion among preselected women, and noninclusion among the target population, FemCure.

https://doi.org/10.1371/journal.pone.0230413.t003

Associated factors for noninclusion from preselected women.

In the multivariate analyses (Table 3), factors independently associated with noninclusion in the preselected women were study site Amsterdam (compared to South Limburg), non-Western migration background, no STI history, being 18–20 years of age (compared to 24 years or older) and a high educational level (rather than a low educational level). A medium educational level (compared to low education) was inversely associated.

Associated factors for noninclusion from the target population.

In the multivariate analyses (Table 3), factors independently associated with noninclusion were study site Amsterdam (compared to South Limburg), non-Western migration background, no STI history, and a high educational level (compared to low education). A medium educational level (compared to low educational level) was inversely associated.

Factors associated with attrition risk

The attrition risk was higher in women with low educational level (compared to high educational level) and in women aged 18–20 years (compared to 24 years or older) (Table 4).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Attrition (i.e., patients lost to follow-up or withdrawn) rates and Hazard Ratios, FemCure during the 3-month follow-up period (Apr. 2016–Dec. 2017).

https://doi.org/10.1371/journal.pone.0230413.t004

Study evaluation

The responses on the study evaluation questionnaires in women who were retained in the study showed a high study satisfaction (Table 5).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. End of follow-up study evaluation responses from the retained participants in the prospective multicenter FemCure cohort study on Chlamydia trachomatis in women.

https://doi.org/10.1371/journal.pone.0230413.t005

Discussion

We demonstrated that chlamydia-infected women can be included and retained in an intensive prospective cohort with 3 months of follow-up. The FemCure study was set up to determine risks for outcomes during follow-up in a group of CT infected women. Here, we assessed the external validity of the FemCure study, and demonstrated that the included population was not fully representative of all STI clinic women. Due to sampling bias, women who had a high educational level, no STI history, or with a non-Western migrant background, were underrepresented. We also assessed the internal validity, and concluded that the number of retained women was sufficient for the planned analyses, but young women and less educated women were more difficult to retain. This selective attrition may have introduced potential selection bias, which will be taken into account when interpreting the FemCure study’s findings.

To aid other researchers, we added a description of the strategies employed to engage women into the study: strategies shown by previous studies to be helpful to improve inclusion and retention (Table 1). A limitation of the current evaluation is that we could not assess the impact of the separate strategies as we did not have a control group and all strategies were simultaneously implemented.

To be included, women first had to be preselected, which was restricted (budgetary reasons) by the predefined research capacity to handle inclusions. Among the preselected women, 31% were included, which was lower than in other prospective studies in the area of sexual health that involved women, applying self-collection of samples at home [32–34]. In FemCure, not being able to attend the scheduled clinic visits or expecting too much effort to complete the study were important reasons for noninclusion. Among the included women, 76% completed the study, which was substantial and comparable to other prospective sexual health studies in women [32–35].

The design of our study was aimed at high internal validity [1], following a large group of women over 3 months to assess the risks for study outcomes (CT detection) after treatment. Thereby, we were interested in including a diverse sample of women from different subsets, to be able to adjust risks (confounding) and exploring heterogeneity of risks (effect modification).

In our sample, some subgroups participated less often due to exclusion criteria and self-selection, introducing sampling bias. The FemCure study sample underrepresented women with a high educational level, women without STI history, and non-Western migrant women. Women with full-time day jobs may not have had time to attend the clinic for required study visits, contributing to the underrepresentation of highly educated women. CT-infected women without an STI history possibly were likely less motivated to participate because of a lower health problem awareness than the previously infected women. As a substantial number of women with high educational level and women without STI history participated, statistical adjustment for these variables will be possible. To be included, women should be able and willing to free their time for the required clinic attendances and should understand the Dutch language. Likely, non-Western migrant women faced language barriers to participation. The small number of non-Western migrant women included in the study population hinders statistical adjustment for this characteristic. The diversity of our study population was, thus, confined by self-selection and by exclusion criteria, such as very young (<18 years) age (ethical reasons), recent antibiotics use, or STI co-infection (potential confounders that were expectedly too infrequent to statistically adjust for). The study nurses’ availability was fairly random and thereby the restricted capacity to handle inclusions presumably did not lead to the exclusion of specific subgroups. It should be noted that selective participation of any subset does not bias the risk estimates obtained in a prospective study, as the outcomes have not yet occurred at inclusion (they occur during follow-up) [1,14]. Further, statistical inferences to excluded subsets will not be possible, although the risks may be applicable to such subsets [14].

Importantly, the lower participation at the Amsterdam clinic resulted from the predefined target number of inclusions that were similar for each clinic, while the Amsterdam clinic served a much larger number of clients (large denominator). Further, young women were generally not underrepresented as young (18–20 years of age) women were included less often but more often preselected.

The sampling bias regarding education, no STI history, and non-Western migrant background resulted in a sample that was not completely representative of the target population. Even though data from the clinic-based cohorts are considered more representative of the “real world” than data from the randomized trials with numerous exclusion criteria [36], caution should be taken when making population inferences from the frequency distributions of FemCure’s baseline characteristics. Even with a representative study sample, inferring population impact can be a tricky undertaking as populations tend to change over time, such as STI clinic populations that vary in composition due to policy changes. Correspondingly, STI clinic populations may not represent the general population.

Our target number of women with retention was reached, ensuring sufficient power for the planned analyses [15]. Retained participants expressed a high study satisfaction and it is notable that nearly all (95%) of the 452 women who completed the first follow-up measurement were retained until the end of the study. However, young women and women with lower educational level were more difficult to retain; they tended to directly drop out after the initial inclusion contact, as was observed in previous studies [32]. Attrition does not necessarily invalidate the study as a sufficient number of women was still retained. However, selective attrition may affect the internal validity when the reasons for stopping data collection were related to both exposure and outcome [1,8]. This may introduce selection bias and lead to an underestimation or overestimation of the risk estimates. We will examine the education and age heterogeneity of risks by testing for effect modification, e.g., assess whether the association between the treatment type and post-treatment CT detection differs by educational level. Potentially, we may consider applying probability-of-exposure weights or inverse-probability-of-attrition weights to compensate for possible under-or-over estimation of risks [37].

By simultaneously implementing a mix of strategies, we aimed to optimize participation. We highlight a few previously found effective components [23–34] that in our practical experience were viewed as critical for the success of this study. Training and team building have resulted in a highly functioning and highly motivated interdisciplinary research team. Sufficient staffing was a challenge with the restricted budgetary funds and was extra invested in by the clinics. All nurses were women who at that time worked at the STI clinic in patient care and were experienced in addressing young women with various backgrounds on sexual health topics in a nonjudgmental way. They were committed to the protocol execution and patient follow-ups. Patient monitoring, reminders, and data handling were managed by a logistical team and a well-built computer program. The research was embedded in an existing clinical infrastructure providing a trusted and familiar research environment for patients. The health problem that this study dealt with affected the women themselves. Women received small monetary incentives and an additional STI checkup at week 12. Data collection was made easy with collection at home or at a nearby clinic and with clear instructions. A strategy that proved helpful in maintaining contact was text messaging. All women received a reminder a day before each data collection and when they did not fill in a questionnaire. Out of all women who were retained in the study 65% (n = 278/427) had received at least one such reminder. This proportion was 70% in retained women with a low educational level and 70% in women aged 18–20 years. A close-contact environment was further enabled by personalized contact with participants. Contact with study site staff was maintained by regular feedback from the coordinating site, expressing gratitude, and addressing any issues that arose regarding the study processes.

To conclude, it is feasible to include and retain CT-infected women in an intensive prospective cohort in busy clinical settings with sufficient investment in the design and study infrastructure and use multifaceted strategies to maximize participation. Highly educated women, women with non-Western background, and women without STI history were less likely to participate, while young women and women with low educational level were more difficult to retain.

Acknowledgments

We are grateful to the staff at the Public Health Service (GGD) South Limburg, Lisanne Eppings, Dr. Ronald van Hoorn, Maria Mergelsberg, Mandy Sanders, Emily Suijlen, Bianca Penders, Helen Sijstermans, and Ine de Bock, the staff at GGD Rotterdam, Beke Nuradini, Angie Martina, Roselyne Uwimana, Mieke Illidge, Klaas de Ridder, and Bram (A) Meima for data management, and the staff at GGD Amsterdam, Dieke Martini, Myra van Leeuwen, Claudia Owusu, Jacqueline Woutersen, Princella Felipa, Mayam Amezian, Arjdal Khadija, and Iris Deen, who are involved in the logistics, and inclusion, Martijn van Rooijen for data management, and Anders Boyd for statistical advice. We also thank the staff at the laboratories of Medical Microbiology of the Maastricht University Medical Center, especially Judith Veugen, Laura Saelmans, Mayk Luchessi, and Kevin Janssen. Also, we thank the staff of the microbiological laboratory of the GGD Amsterdam, Esther Heuser, and Michelle Himschoot. Finally, we thank the members of the advisory committee of this study for providing excellent input on the design of the study and feedback on this paper, from the National Institute for Public Health and the environment, Dr. Jan van Bergen, Birgit van Benthem, and from the University of Maastricht Gerjo Kok and Servaas Morré.