Introduction

The World Health Organization (WHO) has recommended syndromic management of sexually transmitted infections (STIs) in settings where diagnostic resources are not available as it is cheap, easy to implement and offers same-day treatment [1]. However, syndromic management for vaginal discharge is associated with unnecessary use of antibiotics which fuel the development of antimicrobial resistance (AMR), and importantly it leaves asymptomatic infections untreated [2].

Although WHO guidelines now include provision for aetiological testing, there are barriers to operationalising this in resource-limited settings [3]. Ideally, diagnostics should meet the ASSURED criteria (affordable, sensitive, specific, user-friendly, rapid, robust, easy-to-use, deliverable) [4]. Previously developed non-molecular tests for Neisseria gonorrhoeae have not met the minimum requirements for implementation [5–7] Nucleic acid amplifications tests (NAATs) are considered the current gold standard assay for detection of NG with several commercial assays available. However, cost and time-to-result limit their use in resource-limited settings.

Neisseria gonorrhoeae (NG) is associated with pelvic inflammatory disease, infertility, adverse birth outcomes, and increased risk of HIV transmission [8], and has been identified as a WHO priority pathogen for AMR research and development initiatives [9]. Improving antimicrobial stewardship for NG, by reducing unnecessary antibiotic use, requires diagnostic tests that can differentiate between individuals with and without infection.

A novel lateral flow assay for N. gonorrhoeae (NG-LFA) with an analytic time of 20 minutes, was reported to have sensitivities of 96% and 92%, and specificities of 97% and 96% among symptomatic men and women, respectively, in South Africa [10] and was found to be acceptable and usable for clients and health workers [11]. However, performance in the context of aetiological management cannot be extrapolated to screening amongst asymptomatic individuals. Clinical test performance may differ if bacterial loads are lower among asymptomatic than symptomatic individuals whilst the predictive values for both negative and positive results will be affected by the prevalence among different populations. Finally, perceptions of the utility of such a diagnostic as a screening tool rather than an aetiological test may differ, with implications for deployment within health services.

STI screening, particularly for NG, could be particularly effective among pregnant women. Both observational and interventional studies have demonstrated an association between NG in pregnancy and adverse birth outcomes such as prematurity and low birth weight, the incidence of which could be reduced by NG screening and treatment [12–14] An NG-LFA with high specificity and good sensitivity could potentially facilitate adoption of NG screening in pregnant women in resource constrained settings.

The aim of this study was to evaluate the performance and acceptability of the NG-LFA as a screening test in pregnant women, against molecular testing using the GeneXpert platform.

Methods

Results

Participant characteristics

Study recruitment is detailed in the flow diagram in Fig 1, with 912 individuals included in the final analyses. The median participant age was 25 (IQR 22 – 30) years. Most participants were attending ANC for a booking visit (71.5%; 652/912) and 27.9% (254/912) were attending for their first pregnancy. Of 815 participants with an estimated gestational age, 18 (2.2%), 350 (42.9%), and 447 (54.9%) were in the first, second, and third trimesters, respectively.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Flow diagram.

https://doi.org/10.1371/journal.pgph.0003839.g001

HIV prevalence was 9.9% (90/912), of whom 17 (18.9%) individuals were newly diagnosed. All 73 who reported HIV positive status prior to testing also reported antiretroviral therapy use. CT and TV prevalence was 18.8% (171/912) and 11.7% (107/912), respectively.

Most individuals (91.7%) reported no genital symptoms. The most commonly reported symptoms were abnormal vaginal discharge (4.8%; 44/912), vulval irritation or itching (3.3%; 30/912), and lower abdominal or pelvic pain (1.2%; 11/912). Only 2.0% (18/912) reported antibiotic use within the preceding two weeks, of whom four individuals reported receiving ceftriaxone (Table 1).

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of participants recruited into the study (N = 912 unless otherwise stated).

https://doi.org/10.1371/journal.pgph.0003839.t001

Performance of the NG-LFA

The prevalence of NG by Xpert was 4.2% (38/912). The number of true positives, true negatives, false positives, and false negatives using the NG-LFA compared to the reference Xpert assay was 25 (2.7%), 867 (95.1%), 7 (0.8%), and 13 (1.4%), respectively. The sensitivity, specificity, PPV, and NPV of the NG-LFA were 65.8% (25/38; 95% CI 48.6%–80.4%), 99.2% (867/874; 95% CI 98.4–99.7%), 78.1% (25/32; 95% CI 60.0-90.7%), and 98.5% (867/880; 95% CI 97.5-99.2%). Test accuracy was 97.8% (892/912; 95% CI 96.6-98.7%).

Sensitivity and specificity, disaggregated by symptom status, Ct value, antibiotic usage, HIV status, age, gestational age, clinic site, and reader, is shown in Table 2. For all eighteen individuals who reported antibiotic use within the previous two weeks, both Xpert and NG-LFA were negative.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Sensitivity and specificity of novel lateral flow assay for Neisseria gonorrhoeae infection, in pregnant women (N=912 unless otherwise stated).

https://doi.org/10.1371/journal.pgph.0003839.t002

There were six “invalid” NG-LFA results on initial testing, all of which on repeat testing were negative. All six were also negative on Xpert. Of the 912 enrolled participants, 17 had error codes reported on Xpert. Repeat testing returned negative results for all participants bar one who was positive. One further participant was excluded from analyses following an error code, as they did not wish to provide another sample for re-testing.

No adverse events were reported from performing either the NG-LFA or Xpert.

NG-LFA performance across cycle threshold values and within sub-groups

Figs 2 and 3 show the distribution of Xpert NG2 and NG4 target Ct values in Xpert positive samples, disaggregated by NG-LFA positivity. S1 Fig shows a scatter plot of NG2 and NG4 Ct values, disaggregated by NG-LFA positivity.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Histogram demonstrating distribution of Xpert NG2 target cycle threshold values in Xpert positive samples (N = 38).

https://doi.org/10.1371/journal.pgph.0003839.g002

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Histogram demonstrating distribution of Xpert NG4 target cycle threshold values in Xpert positive samples (N = 38).

https://doi.org/10.1371/journal.pgph.0003839.g003

The Ct values for both NG2 (median 29.5 vs 25.5; p = 0.0001) and NG4 (median 30.2 vs 25.6; p=0.0002) targets were higher in participants with false negative NG-LFA results, compared to true positive. Of the 13 discordant NG-LFA negative results, 8 had an NG2 and/or NG4 PCR Ct value of above 30; the remaining five had NG2 and NG4 Ct values between 25 and 30.

Among participants with positive Xpert results, 18.4% (7/38) self-reported presence of vaginal discharge. There was no difference in Ct values between those with and without symptoms, for both NG2 (median 26.3 vs 26.2; p = 0.90) and NG4 (median 26.3 vs 26.3; p=0.79) targets. S1 Table shows the characteristics of participants positive for NG on Xpert, disaggregated by NG-LFA positivity.

ROC curve

The ROC curve is shown in Fig 4. The AUC was 0.94 (95% CI 0.89 – 0.98). The optimal cut point according to the Youden Index was ≥0.0278785, which provided a sensitivity, specificity, PPV, and NPV, at this cut point of 89.5% (95% CI 75.2 – 97.1%), 90.4% (95% CI 88.2 – 92.2%), 28.8% (95% CI 20.8 – 37.9%), and 99.5% (95% ci 98.7 – 99.9%), respectively.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Receiver Operating Characteristic (ROC) curve of Neisseria gonorrhoeae lateral flow assay (NG-LFA) at different fluorescence thresholds.

https://doi.org/10.1371/journal.pgph.0003839.g004

Discussion

In this largely asymptomatic pregnant population, we demonstrated a very high specificity and NPV of the NG-LFA. However, the sensitivity was lower than reported in a symptomatic general population [10]. One in three cases of NG were not detected by the NG-LFA among pregnant women. Based on the findings of this study the NG-LFA potentially provides value in this context by allowing identification of asymptomatic women with NG who would otherwise be untreated, and may serve as a useful screening tool to rule in and treat NG infection in pregnant populations. However, more consideration is required when using the NG-LFA to rule out NG infection in ANC clinics, which should be supported by further demonstration studies. Modelling is also required to determine whether this level of clinical performance is still cost-effective compared to current standard of care, which is no screening.

Among Xpert-detected NG cases negative by NG-LFA, higher Ct values suggest that lower bacterial load in these samples is the likely reason for non-detection [22]. Other possibilities for false negative results include human error or issues with the readers. The four readers used in the study were prototype models and could have had technical issues. There were four false negative results within eight days at one site in August 2023. However, prior to re-training, practice tests using positive and negative control samples were correctly identified by both readers at the site. As no obvious errors could be identified retrospectively, and the relatively high Ct values among the false negative cases in this period (NG4 Ct > 30 for three out of four cases), this may have been a random increase in the frequency of false negative cases. There was also no difference in sensitivity between sites and readers, but confidence intervals were wide. Strengthening and ensuring quality assurance of the reader will be particularly important prior to commercialisation of the NG-LFA. Another consideration that may have contributed to lower bacterial loads was that the NG-LFA specimen was the third swab collected. In contrast, in the previous evaluation by Peters et al. the first swab collected was tested using the NG-LFA [10].

The previous evaluation of the NG-LFA focused on improving aetiological diagnosis and management [10]; in this study we evaluated its use for screening. The goals of testing vary markedly across these scenarios. In symptomatic patients, NG testing aims to prevent inappropriate treatment for NG in individuals who would otherwise receive treatment as part of syndromic management. In contrast, the goal of screening for NG in ANC would be to detect and treat NG cases that would otherwise not be treated, to avoid downstream sequelae, whilst also not providing inappropriate treatment to individuals without infection. The individual level benefits conferred by a test with a sensitivity of 65% and very high specificity may therefore be acceptable when compared to the alternative of no testing, until low-cost point-of-care tests with higher sensitivity are developed.

The evidence for screening for asymptomatic NG infection in pregnancy is not conclusive. The WANTAIM trial found that among pregnant women with NG at enrolment, screening and treatment led to a 53% improvement in prevalence of preterm birth and/or low birth weight, although this was in a small subset of trial participants [14]. Additionally, given that ophthalmia neonatorum is mainly contracted during delivery due to presence of NG in the birth canal, antenatal NG treatment should reduce this risk [23]. However, the clinical importance of the NG cases detected by Xpert but missed by the NG-LFA in this study is unclear. Lateral flow antigen tests have a lower analytical sensitivity than NAATs, which can detect prior or resolving infections, typically with high Ct, where DNA may be detectable but not antigen [24–26] In some cases, high Ct may also represent results that are falsely positive on Xpert, which is not a perfect diagnostic [27]. Previous studies have reported spontaneous clearance rates of up to 33% for NG in non-pregnant populations [28,29], Hypothetically, some low bacterial load infections might self-clear and the risks of treatment may outweigh the benefits. Understanding the downstream consequences of treating or not treating such infections is essential to determine the clinical and cost-effectiveness of screening. Of note, there were three false negative results that were culture positive, indicative of viability. This discrepancy may due to the different sampling sites for the NG-LFA and for culture. Whereas swabs for NG-LFA were vaginal, swabs for culture were taken from the cervix, where both bacterial load and organism viability is likely to be higher.

We compared sensitivity and specificity across a range of participant characteristics. However, due to the small sample sizes for sensitivity calculations, it is difficult to draw meaningful conclusions from these observations. Caution must also be exercised when comparing these findings to the study among symptomatic individuals in South Africa [10]. Individuals attending clinic because they have symptoms are likely very different to those attending for routine ANC and reporting symptoms when specifically asked. Firstly, physiological changes to vaginal discharge are common in pregnancy. Furthermore, the former’s symptoms are more likely to be of a higher severity or duration such that the individual has sought care. For these reasons we are also wary of interpreting the lack of difference in Ct values between participants with and without symptoms.

Such differences between populations are also important when considering the fluorescence threshold at which a sample is considered NG positive, and how optimal thresholds will vary between different populations. In both this study and the evaluation among symptomatic patients in South Africa, the threshold for a positive reading was ≥0.063. However, in our study a reduced threshold of ≥0.0278785 would have led to a sensitivity and specificity of 89.5% and 90.4%, respectively. By sacrificing a degree of specificity, only 4 NG infections out of 38 would have been missed. However, one in ten of those without NG would have been inappropriately treated. This demonstrates the importance of the aim of screening or testing in determining the most appropriate balance between sensitivity and specificity. For example, if the primary goal of ANC NG screening is to avert cases of ophthalmia neonatorum, then a relatively high rate of overtreatment may be tolerated. Finally we calculated the Youden index, which gives equal weight to sensitivity and specificity [20]. However, a different weighting may be used to find the most appropriate threshold. Importantly, as the NG-LFA has not yet gone through regulatory approval and is not commercialised, the threshold in both this study and the evaluation among symptomatic individuals was based on limited data. The threshold for a commercialised assay would be based on larger datasets covering different populations.

The NG-LFA was reported to be acceptable, easy-to-use, easy-to-interpret, and portable, concurrent with the SUS scores. These characteristics are particularly important when considering implementation in settings with limited infrastructure and where retention of a skilled workforce is challenging. However, interviewees reported issues with trusting the device following their experience of discordant results. Suggestions from team members of potential uses for the NG-LFA tended to include the need for a back-up, such as confirmatory testing, or the provision of presumptive treatment in individuals with persistent symptoms but a negative NG-LFA result. In contrast, in South Africa the NG-LFA was reported to engender trust among both healthcare workers and clients, as they could see the test being performed in front of them, in addition to providing increased confidence in clinical decision making [30]. However, there are some key differences between the studies in this regard. In South Africa, the aim was to rule out NG infection and reduce antimicrobial usage in symptomatic individuals who would have otherwise received treatment, and so having results to inform this management is positive. Furthermore, because of the higher sensitivity and higher NG prevalence, the impact of false positive and false negative results is reduced. In our study, the NG-LFA correctly reported 97.8% of results, but the small number of incorrect results had a notable effect on its perceived reliability. Another consideration is how the NG-LFA was communicated across both environments. In our study, Xpert was already integrated into ANC when the NG-LFA was introduced, and was used to guide treatment. The Xpert results were therefore considered the “gold standard”. In South Africa, testing procedures included additional tie breaker tests where discordant results were recorded, suggesting to staff that Xpert may not always be correct [10].

The strengths of this study include its pragmatic design, with participants attending for routine ANC within public sector primary healthcare clinics in Harare, reflecting real-world use of the NG-LFA. There were also important limitations. As the number of women with positive Xpert results was relatively small, the confidence intervals for the sensitivity estimates are wide, limiting comparisons of test performance across different participant characteristics. Importantly, the overall study sample size was not powered for the present evaluation, and was likely underpowered. Additionally, we used Xpert as the reference standard. Although a molecular test with high sensitivity and specificity [31], in the previous evaluation, laboratory-based confirmatory PCR was performed on discordant cases and identified that Xpert had three false negative and two false positive results [10]. Given the lower pre-test probability for screening in our study, false positive results on the Xpert are even more likely [25,32]. Furthermore, one of the “false positive” NG-LFA infections in this study was cultured and identified phenotypically as NG, indicating a false negative Xpert result. Unfortunately, overall yield from culture of NG isolates was relatively poor, with just over a third of individuals with Xpert-positive NG having a positive culture result. Although NG culture was never intended to be the gold standard, it was still less helpful than had been envisioned, particularly in terms of confirming false positive or false negative results. The lack of an on-site incubator to allow for immediate plating, and also variable times between sample collection and transport to the laboratory may have been contributing factors. Finally, due to the limited number of interviewees and limited time discussing the NG-LFA within broader interviews, our assessment of acceptability and usability is based on only four viewpoints. Furthermore, although the intervention team were blinded to Xpert results at the time of reading the NG-LFA, they did subsequently find out the Xpert results, due to the interventional nature of the study and having to deliver treatment to participants. This undoubtedly influenced their perceptions of the reliability of the NG-LFA. Of note, if such a test were implemented at scale, concerns regarding accuracy or trust are unlikely to be realised at a local level, if such confirmatory tests are not performed.

In conclusion, although the NG-LFA is promising for augmenting syndromic management in order to prevent inappropriate antimicrobial usage, as demonstrated among symptomatic individuals in South Africa [10], the potential usability of the NG-LFA as a screening tool in this context is less evident. NPV was high, and so the NG-LFA would correctly rule out infection in the majority of individuals. However, one-third of pregnant women with Xpert-detected NG were missed, which may have important implications for clinical and cost-effectiveness, and performance may not be considered acceptable for widespread implementation. However, this must be weighed against the current standard of care relying on syndromic management alone. Furthermore, most discordant NG-LFA negative cases had high Ct values suggestive of low bacterial load. More studies are needed to assess the NG-LFA in other settings and populations, across different sample types and sample collection methods, to further explore the relationship between NG-LFA positivity, bacterial load and infection viability, and to understand the clinical implications of these low bacterial load infections. Current NAATs are too expensive and/or logistically unfeasible for routine care, and further research is warranted to develop highly sensitive screening tests for asymptomatic pregnant women.

Supplementary information