Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

The Trade-Off between Accuracy and Accessibility of Syphilis Screening Assays

  • Pieter W. Smit,

    Affiliation Department of Infectious & Tropical diseases, London School of Hygiene & Tropical Medicine, London, United Kingdom

  • David Mabey ,

    david.mabey@lshtm.ac.uk

    Affiliation Department of Infectious & Tropical diseases, London School of Hygiene & Tropical Medicine, London, United Kingdom

  • John Changalucha,

    Affiliation National Institute for Medical Research, Mwanza, Tanzania

  • Julius Mngara,

    Affiliation National Institute for Medical Research, Mwanza, Tanzania

  • Benjamin Clark,

    Affiliation National Institute for Medical Research, Mwanza, Tanzania

  • Aura Andreasen,

    Affiliations Department of Infectious & Tropical diseases, London School of Hygiene & Tropical Medicine, London, United Kingdom, Mwanza intervention Trials Unit, Mwanza, Tanzania

  • Jim Todd,

    Affiliations Department of Infectious & Tropical diseases, London School of Hygiene & Tropical Medicine, London, United Kingdom, National Institute for Medical Research, Mwanza, Tanzania

  • Mark Urassa,

    Affiliation National Institute for Medical Research, Mwanza, Tanzania

  • Basia Zaba,

    Affiliation Department of Infectious & Tropical diseases, London School of Hygiene & Tropical Medicine, London, United Kingdom

  • Rosanna W. Peeling

    Affiliation Department of Infectious & Tropical diseases, London School of Hygiene & Tropical Medicine, London, United Kingdom

The Trade-Off between Accuracy and Accessibility of Syphilis Screening Assays

  • Pieter W. Smit, 
  • David Mabey, 
  • John Changalucha, 
  • Julius Mngara, 
  • Benjamin Clark, 
  • Aura Andreasen, 
  • Jim Todd, 
  • Mark Urassa, 
  • Basia Zaba, 
  • Rosanna W. Peeling
PLOS
x

Abstract

The availability of rapid and sensitive methods to diagnose syphilis facilitates screening of pregnant women, which is one of the most cost-effective health interventions available. We have evaluated two screening methods in Tanzania: an enzyme immunoassay (EIA), and a point-of-care test (POCT). We evaluated the performance of each test against the Treponema pallidum particle agglutination assay (TPPA) as the reference method, and the accessibility of testing in a rural district of Tanzania. The POCT was performed in the clinic on whole blood, while the other assays were performed on plasma in the laboratory. Samples were also tested by the rapid plasma Reagin (RPR) test. With TPPA as reference assay, the sensitivity and specificity of EIA were 95.3% and 97.8%, and of the POCT were 59.6% and 99.4% respectively. The sensitivity of the POCT and EIA for active syphilis cases (TPPA positive and RPR titer ≥1/8) were 82% and 100% respectively. Only 15% of antenatal clinic attenders in this district visited a health facility with a laboratory capable of performing the EIA. Although it is less sensitive than EIA, its greater accessibility, and the fact that treatment can be given on the same day, means that the use of POCT would result in a higher proportion of women with syphilis receiving treatment than with the EIA in this district of Tanzania.

Introduction

The prevalence of syphilis is high among pregnant women attending antenatal clinics in sub-Saharan Africa [1]. Syphilis in pregnancy can have devastating effects on the developing fetus and is a major cause of stillbirths and neonatal deaths in Africa [2]. Screening with an adequate diagnostic test and treatment of pregnant women with a single dose of benzathine penicillin before the third trimester could prevent more than 300, 000 stillbirths and neonatal deaths annually [3,4].

Latent syphilis can only be diagnosed serologically. Laboratory based assays such as the Treponema pallidum particle agglutination assay (TPPA) and rapid plasma reagin (RPR) test are widely used. As the agglutination is interpreted by a technician, the test result is subjective. Enzyme immunoassays (EIA) are now recommended for syphilis screening in Europe [5]. They are easy to use, provide objective results, and are well adapted to high throughput laboratories; but they are more expensive than the other assays, require equipment (a plate washer and a plate reader), and as TPPA and RPR, require cold storage of consumables, which is a limiting factor for some settings [6,7].

Most of the available syphilis point-of-care tests (POCT) are lateral flow based treponemal tests that provide a result in 10-30 minutes and do not require any equipment. In contrast to these laboratory based assays mentioned above, POCTs are easy to perform, require only a drop of blood collected by finger prick, and do not require refrigeration; they could enable same day testing and treatment for syphilis at any health facility, but are less sensitive than laboratory based assays [8].

Selecting a screening assay, particularly in an African country, should not be solely based on the performance of the assay. Besides the performance of a test, necessary equipment, cold chain requirements, and complexity of executing tests should be taken into consideration. Selecting a screening assay is therefore often a trade-off between the performance of the assays, cost, and accessibility for patients to be screened. Comparative studies have been performed [9-12] but, to our knowledge, evaluations that include performance and accessibility of syphilis screening tests have not been published previously. This study evaluates two screening assays, a POCT and an EIA, to review the trade-off between performance and accessibility of syphilis assays used to screen pregnant women in an African district.

Methods

Samples and Field Procedures

The study was approved by the ethical review committee of the National Institute for Medical Research (NIMR) in Tanzania and the ethical committee of the London School of Hygiene and Tropical Medicine. The Kisesa open cohort is a well-established on-going community-based study in Northern Tanzania [13,14]. The cohort study uses regular demographic surveillance with serological surveys, providing data on HIV incidence and prevalence [15]. Regular demographic surveillance ensures representation of all members of the population, with serological surveys providing data on HIV prevalence and incidence [15]. Subjects that accepted voluntary counseling and testing (VCT) were tested for HIV and syphilis using POCT performed by trained clinicians. All subjects with a positive syphilis result were given free medical treatment according to Tanzanian government guidelines, and all those positive for HIV were referred to the Tanzanian care and treatment center.

Whole blood was collected by venipuncture into heparinized tubes from consenting subjects, and transported to the NIMR laboratory in Mwanza. Within 24 hours the blood was centrifuged and stored at -20°C. Samples were collected from April 2010 until September 2010 and were tested until March 2011. Samples were bar-coded to ensure anonymous testing. Double data entry was used to enter the results. Results were entered automatically (EIA) or manually into the Laboratory Information Management System (LIMS).

Access to screening

We visited all 50 health facilities in one district of Mwanza Region (Geita District), to ask how many women attended each antenatal clinic per month. Geita District has one district hospital, 8 health centers, which tend to be in larger villages, and 41 rural dispensaries.

Point-of-care test

The SD bioline syphilis 3.0 POCT (Standard Diagnostics, Kyong gi-do, Korea) was included in this study as it was found to have acceptable performance (according to the WHO ASSURED criteria) [16]. Additionally, the SD bioline POCT was made available through the WHO Bulk Procurement Programme and has been previously used in Tanzania [17]. POCT testing was done by trained clinicians who were contracted and trained by the Kisesa open cohort study. The POCT was performed with whole blood samples collected by finger prickA timer was used to ensure that the test was read after exactly 15 minutes. The manufacturer’s instructions were followed.

TPPA

A total of 2099 plasma aliquots were raised to room temperature and tested by TPPA (Fujirebio, Tokyo, Japan). TPPA was performed according to the manufacturer’s instructions. TPPA results were read by two trained and experienced technicians. The reading of the TPPA occurred while masked to results of other tests. Discordant results between the two technicians were discussed and one outcome was agreed by consensus. Results were deemed indeterminate for biologically reactive samples or when a conclusive outcome could not be obtained due to difficulty in interpretation or lack of technician’s agreement.

Enzyme ImmunoAssay

The Syphilis Enzyme ImmunoAssay (EIA) (Lab21 healthcare, Kentford, UK) became available at the midpoint of the study and was performed on 1041 samples (49.6%). It was performed according to the manufacturer’s instructions and read by Optical Density (OD) 450/620nm using an automated reader (DTX 800, Beckman Coulter, USA) which calculated the cut-off according to the instruction manual.

RPR

Quantitative Rapid plasma Reagin (RPR) (BD Macro-vue RPR, Beckton Dickinson, Sparks MD, USA) was performed on all samples according to the manufacturer’s instructions. An active syphilis infection was defined as TPPA positive and RPR titer ≥1/8 [18].

Sensitivities and specificities, and exact binomial confidence intervals are given (diagt function in Stata). The agreement between various methods was tabulated. Microsoft Excel (Microsoft Corp., USA) and the statistical package Stata 11 (Stata corp LP, Texas, USA) were used to analyze the results.

Results

Point- of- care test

The POCT was evaluated with a set of 2099 samples. Of these, 17.1% of samples were positive by TPPA and 10.7% by POCT. With TPPA as reference, the POCT had 11 false positive and 145 false negative results, giving a sensitivity of 59.6% (95% confidence interval CI: 54.3-64.7%) and specificity of 99.4% (95% CI: 98.9-99.7%) (Table 1). There was a 92.6% agreement between POCT and TPPA.

TPPAPOCTRPRSamples
<1/8≥1/8Neg
++7141102214
+-219115145
-+30811
--44916761729
Total1395919012099

Table 1. The TPPA and POCT performances are given with RPR results divided into titres lower (<1/8) or higher than 1/8 (≥1/8).

+positive, - negative, Neg=negative
CSV
Download CSV

Of the 145 false negatives by POCT, 115 (79%) were RPR negative and 31 were RPR positive. Fifty out of 2099 samples tested by POCT had active syphilis (positive TPPA and RPR titre ≥ 1/8), of which 41 were detected by POCT, giving a sensitivity of 82% (69.2%-90.2%) and specificity of 100%.

EIA

The EIA was evaluated with a subset of 1041 samples that were available at the second half of the study. Of these, 18.1% of samples were positive by TPPA, 11.5% by POCT and 19.1% by EIA.

With TPPA as reference, the EIA had 20 false positive and 9 false negative results, giving a sensitivity of 95.2% (95% CI: 91.1-97.8%) and specificity of 97.7% (95% CI: 96.4-98.6%) (Table 2). There was a 97.3% agreement between TPPA and EIA. The EIA showed a sensitivity and specificity of 100% to detect active syphilis cases (Table 2).

TPPAEIARPRSamples
<1/8≥1/8Neg
++5326100179
+-0099
-+301720
--304799833
Total86309251041

Table 2. The TPPA and EIA performances are given with RPR results divided into titres lower (<1/8) or higher than 1/8 (≥1/8).

+positive, - negative, Neg=negative
CSV
Download CSV

All 29 samples for which a discordant result was obtained by the two laboratory methods were retested. Although retesting discordant results provides little certainty regarding the true syphilis status of the patients, it is informative to review the reproducibility of the tests. On retesting, 8 out of 9 TPPA positive samples became negative and one became TPPA indeterminate, and 6 out of 20 TPPA negative samples became positive. When retesting the samples with discordant results, only four EIA positive samples changed outcome (4 initially positive samples became negative out of 29 discordant results)

Comparison of EIA and POCT

The performance of the EIA and POCT were compared with the samples that were tested by EIA, TPPA and POCT. Figure 1 shows the distribution of 214 positive out of 1041 samples identified by one or more of the screening assays.

thumbnail
Figure 1. Distribution of positive samples among the three assays.

Numbers represents positive samples detected by three assays (number is given in 3 circles), or by two assays (overlapped by two circles) or by one assay (given as number below the assay name). 214 positives detected by any of the three methods out of 1041 samples.

https://doi.org/10.1371/journal.pone.0075327.g001

Accessibility

Syphilis screening is essential for antenatal care and we therefore decided to assess the accessibility of these syphilis tests in antenatal care clinics. Geita District has one district hospital, 8 health centers, which tend to be in larger villages, and 41 rural dispensaries. The EIA could only be performed at the district hospital, since health centers and dispensaries do not have centrifuges for plasma separation, cold storage capabilities, plate washers or readers. The POCT could successfully be implemented in the district hospital, health centers and dispensaries. On average, 517 pregnant women attend the district hospital ANC for the first time per month, whereas 1164 attend a health center and 1685 a dispensary. If the prevalence of active syphilis is 2.3%, as found in this study, 77 pregnant women with active syphilis would be expected to attend an ANC in Geita District each month. Twelve of these would be attending the district hospital, and 65 would be attending health centers or dispensaries. If POCT wereused at the ANC clinics (including the district hospital), with a sensitivity of 82%, it would detect 63 (82%) pregnant women and enable them to be offered immediate treatment, whereas without the POCT only 12 of 77 pregnant women with syphilis (16%) would be identified.

Discussion

To our knowledge, this is the first study comparing POCT and EIA performed in an African setting. Additionally, this is the first study that includes both the accessibility and diagnostic performances of syphilis screening assays into one evaluation.

It Is Important to Note That This Study Was Performed in Tanzania on Tanzanian Samples, Which Potentially Influences the Results. African Samples Can Contain High Immunoglobulin Levels Due to Other Infections Which Potentially Cause False Positive Test Results. This Is Particularly the Case for RPR Tests, Which Can Cause Biologically False Positive Results in HIV Positive Patients [19].

[20-22]. Although tests were performed and stored according to the manufacturer’s recommendations, potential environmental effects by transporting and using the kits under tropical conditions could not be completely ruled out. Performing tests in an African country relies more heavily on the robustness of the tests than in developed countries. This could potentially influence the test results. The need for quality control and appropriate training is important for any diagnostic test whether it is laboratory based or POCT. Most quality assurance methods are not suitable for monitoring POCT that are performed by healthcare workers at remote locations, making it more challenging to assure the quality of POC testing [17,23].

The reproducibility of the EIA evaluated in this study was good, with few samples giving discordant results on re-testing. The samples that were discordant had very low ODs, just above the cut-off. Therefore, EIA seems to be a more stable and reliable test compared to TPPA.

Adverse pregnancy outcomes due to syphilis are seen in women with RPR titers of ≥ 1:8 [2]. In this study, the EIA had a higher sensitivity for the detection of these cases than the POCT (100% versus 82%), but a major limiting factor is that the EIA can only be performed when sufficient laboratory infrastructure is available. Most importantly, in Mwanza region, the POCT would enable more pregnant women with syphilis to be identified due to its greater accessibility, since pregnant women can be screened not only at the hospital but also at health centers and dispensaries. The RPR test is easier to perform than the EIA and requires less equipment, but requires access to a laboratory with electricity. It has been used to screen pregnant women in many developing countries, but results obtained in rural health facilities have not been encouraging [24,25]. Therefore, we did not evaluate the option of on- or off-site RPR & TPPA screening in Mwanza region In some countries blood is taken from pregnant women at rural health facilities and sent to a central laboratory to be tested for syphilis (RPR and TPPA/TPHA), but under these circumstances women with syphilis will only be treated if they return for their results [26]. A study in Kenya found that less than 10% of pregnant women with syphilis received appropriate treatment when serological tests were performed at a central laboratory [27]. When same day testing and treatment were made available in these clinics, more than 90% of infected women were treated [28].

A test which gives a result in 15 minutes, allowing patients to be tested and treated on the same day, may result in more cases being treated than a more sensitive laboratory test which requires patients to return for treatment at a later date. The great advantage of the new POCTs for syphilis is that they do not require laboratory equipment or electricity, and can make same day testing and treatment for syphilis available at any health facility [17,29]. Based on the performance and accessibility of both tests, the POCT is the best option for this district in Tanzania and potentially for other settings in developing countries as well.

Acknowledgments

We are grateful to the study participants, the Kisesa cohort study staff, Leiden Cytology and Pathology Laboratory The Netherlands, the laboratory staff of the National Institute for Medical Research, Mwanza, and Mathilde E. Boon for their participation in this project.

Author Contributions

Conceived and designed the experiments: PWS DM JT RWP BZ. Performed the experiments: PWS JM AA MU. Analyzed the data: PWS JT BC DM RWP. Contributed reagents/materials/analysis tools: JT BZ RWP JC. Wrote the manuscript: PWS RWP DM JT AA.

References

  1. 1. Chico RM, Mayaud P, Ariti C, Mabey D, Ronsmans C et al. (2012) Prevalence of malaria and sexually transmitted and reproductive tract infections in pregnancy in sub-Saharan Africa: a systematic review. JAMA 307: 2079-2086. doi:10.1001/jama.2012.3428. PubMed: 22665107.
  2. 2. Watson-Jones D, Changalucha J, Gumodoka B, Weiss H, Rusizoka M et al. (2002) Syphilis in pregnancy in Tanzania. I. Impact of maternal syphilis on outcome of pregnancy. J Infect Dis 186: 940-947..
  3. 3. WHO (2012) Investment case for eliminating mother-to-child transmission of syphilis. Geneva: World Health Organization.
  4. 4. Newman L, Kamb M, Hawkes S, Gomez G, Say L et al. (2013) Global estimates of syphilis in pregnancy and associated adverse outcomes: analysis of multinational antenatal surveillance data. PLOS Med 10: e1001396. PubMed: 23468598.
  5. 5. Kingston M, French P, Goh B, Goold P, Higgins S et al. (2008) UK National Guidelines on the Management of Syphilis 2008. Int J STD AIDS 19: 729-740. doi:10.1258/ijsa.2008.008279. PubMed: 18931264.
  6. 6. Tsang RS, Martin IE, Lau A, Sawatzky P (2007) Serological diagnosis of syphilis: comparison of the Trep-Chek IgG enzyme immunoassay with other screening and confirmatory tests. FEMS Immunol Med Microbiol 51: 118-124. doi:10.1111/j.1574-695X.2007.00289.x. PubMed: 17854473.
  7. 7. Aktas G, Young H, Moyes A, Badur S (2005) Evaluation of the serodia Treponema pallidum particle agglutination, the Murex Syphilis ICE and the Enzywell TP tests for serodiagnosis of syphilis. Int J STD AIDS 16: 294-298. doi:10.1258/0956462053654195. PubMed: 15899082.
  8. 8. Mabey D, Peeling RW, Ballard R, Benzaken AS, Galbán E et al. (2006) Prospective, multi-centre clinic-based evaluation of four rapid diagnostic tests for syphilis. Sex Transm Infect 82 Suppl 5: v13–6: v13-. 16. PubMed: 17215274.
  9. 9. Binnicker MJ, Jespersen DJ, Rollins LO (2011) Treponema-specific tests for serodiagnosis of syphilis: comparative evaluation of seven assays. J Clin Microbiol 49: 1313-1317. doi:10.1128/JCM.02555-10. PubMed: 21346050.
  10. 10. Aktas G, Young H, Moyes A, Badur S (2007) Evaluation of the fluorescent treponemal antibody absorption test for detection of antibodies (immunoglobulins G and M) to Treponema pallidum in serologic diagnosis of syphilis. Int J STD AIDS 18: 255-260. doi:10.1258/095646207780658827. PubMed: 17509177.
  11. 11. Young H, Pryde J, Duncan L, Dave J (2009) The Architect Syphilis assay for antibodies to Treponema pallidum: an automated screening assay with high sensitivity in primary syphilis. Sex Transm Infect 85: 19-23. doi:10.1136/sti.2008.031872. PubMed: 18790859.
  12. 12. Maple PA, Ratcliffe D, Smit E (2010) Characterization of Treponema pallidum particle agglutination assay-negative sera following screening by treponemal total antibody enzyme immunoassays. Clin Vaccine Immunol 17: 1718-1722. doi:10.1128/CVI.00102-10. PubMed: 20844087.
  13. 13. TAZAMA (2011) Tazama project. pp.. Available: . http://www.tazamaproject. org/pub0006.shtml. Accessed 2011 June 10.
  14. 14. Cawley C, Wringe A, Isingo R, Mtenga B, Clark B et al. (2013) Low rates of repeat HIV testing despite increased availability of antiretroviral therapy in rural Tanzania: findings from 2003-2010. PLOS ONE 8: e62212. doi:10.1371/journal.pone.0062212. PubMed: 23626791.
  15. 15. Wambura M, Urassa M, Isingo R, Ndege M, Marston M et al. (2007) HIV prevalence and incidence in rural Tanzania: results from 10 years of follow-up in an open-cohort study. J Acquir Immune Defic Syndr 46: 616-623. doi:10.1097/QAI.0b013e31815a571a. PubMed: 18043316.
  16. 16. Jafari Y, Peeling RW, Shivkumar S, Claessens C, Joseph L et al. (2013) Are Treponema pallidum specific rapid and point-of-care tests for syphilis accurate enough for screening in resource limited settings? Evidence from a meta-analysis. PLOS ONE 8: e54695. doi:10.1371/journal.pone.0054695. PubMed: 23468842.
  17. 17. Mabey DC, Sollis KA, Kelly HA, Benzaken AS, Bitarakwate E et al. (2012) Point-of-Care Tests to Strengthen Health Systems and Save Newborn Lives: The Case of Syphilis. PLOS Med 9: e1001233. PubMed: 22719229.
  18. 18. Schmid G (2004) Economic and programmatic aspects of congenital syphilis prevention. Bull World Health Organ 82: 402-409. PubMed: 15356931.
  19. 19. Augenbraun MH, DeHovitz JA, Feldman J, Clarke L, Landesman S et al. (1994) Biological false-positive syphilis test results for women infected with human immunodeficiency virus. Clin Infect Dis 19: 1040-1044. doi:10.1093/clinids/19.6.1040. PubMed: 7888531.
  20. 20. Oboho IK, Gebo KA, Moore RD, Ghanem KG (2013) The Impact of Combined Antiretroviral Therapy on Biologic False Positive Rapid Plasma Reagin (RPR) Serologies in a Longitudinal Cohort of HIV-infected Persons. Clin Infect Dis.
  21. 21. Everett DB, Weiss HA, Changalucha J, Anemona A, Chirwa T et al. (2007) Low specificity of the Murex fourth-generation HIV enzyme immunoassay in Tanzanian adolescents. Trop Med Int Health 12: 1323-1326. doi:10.1111/j.1365-3156.2007.01933.x. PubMed: 17949396.
  22. 22. Gasasira AF, Dorsey G, Kamya MR, Havlir D, Kiggundu M et al. (2006) False-positive results of enzyme immunoassays for human immunodeficiency virus in patients with uncomplicated malaria. J Clin Microbiol 44: 3021-3024. doi:10.1128/JCM.02207-05. PubMed: 16891532.
  23. 23. Smit PW, van der Vlis T, Mabey D, Changalucha J, Mngara J et al. (2013) The development and validation of dried blood spots for external quality assurance of syphilis serology. BMC Infect Dis MS ID, 13: 1470439142778761. PubMed: 23442198.
  24. 24. Dorigo-Zetsma JW, Belewu D, Meless H, Sanders E, Coutinho RA et al. (2004) Performance of routine syphilis serology in the Ethiopian cohort on HIV/AIDS. Sex Transm Infect 80: 96-99. doi:10.1136/sti.2003.005827. PubMed: 15054167.
  25. 25. Mishra S, Naik B, Venugopal B, Kudur P, Washington R et al. (2010) Syphilis screening among female sex workers in Bangalore, India: comparison of point-of-care testing and traditional serological approaches. Sex Transm Infect 2009/12/08: 193–8 ed. pp. 193-198 PubMed: 19965800.
  26. 26. Gift TL, Pate MS, Hook EW 3rd, Kassler WJ (1999) The rapid test paradox: when fewer cases detected lead to more cases treated: a decision analysis of tests for Chlamydia trachomatis. Sex Transm Dis 26: 232-240. doi:10.1097/00007435-199904000-00010. PubMed: 10225593.
  27. 27. Temmerman M, Gilks CF, Sanghri HC (1993) Spontaneous and induced abortions at Kenyatta National Hospital, Nairobi, Kenya. Int J Gynecol Obstet 41: 182-183. doi:10.1016/0020-7292(93)90703-Y. PubMed: 8099035.
  28. 28. Fonck K, Claeys P, Bashir F, Bwayo J, Fransen L et al. (2001) Syphilis control during pregnancy: effectiveness and sustainability of a decentralized program. Am J Public Health 91: 705-707. doi:10.2105/AJPH.91.5.705. PubMed: 11344874.
  29. 29. Bronzan RN, Mwesigwa-Kayongo DC, Narkunas D, Schmid GP, Neilsen GA et al. (2007) On-site rapid antenatal syphilis screening with an immunochromatographic strip improves case detection and treatment in rural South African clinics. Sex Transm Dis 34: S55-S60. doi:10.1097/01.olq.0000245987.78067.0c. PubMed: 17139234.