http://orcid.org/0000-0002-1119-095X
Figures
Abstract
Introduction
The appropriate use of clinically accurate diagnostic tests is essential for the detection of pertussis, a poorly controlled vaccine-preventable disease. The purpose of this study was to estimate the sensitivity and specificity of different diagnostic criteria including culture, multi-target polymerase chain reaction (PCR), anti-pertussis toxin IgG (IgG-PT) serology, and the use of a clinical case definition. An additional objective was to describe the optimal timing of specimen collection for the various tests.
Methods
Clinical specimens were collected from patients with cough illness at seven locations across the United States between 2007 and 2011. Nasopharyngeal and blood specimens were collected from each patient during the enrollment visit. Patients who had been coughing for ≤ 2 weeks were asked to return in 2–4 weeks for collection of a second, convalescent blood specimen. Sensitivity and specificity of each diagnostic test were estimated using three methods—pertussis culture as the “gold standard,” composite reference standard analysis (CRS), and latent class analysis (LCA).
Results
Overall, 868 patients were enrolled and 13.6% were B. pertussis positive by at least one diagnostic test. In a sample of 545 participants with non-missing data on all four diagnostic criteria, culture was 64.0% sensitive, PCR was 90.6% sensitive, and both were 100% specific by LCA. CRS and LCA methods increased the sensitivity estimates for convalescent serology and the clinical case definition over the culture-based estimates. Culture and PCR were most sensitive when performed during the first two weeks of cough; serology was optimally sensitive after the second week of cough.
Conclusions
Timing of specimen collection in relation to onset of illness should be considered when ordering diagnostic tests for pertussis. Consideration should be given to including IgG-PT serology as a confirmatory test in the Council of State and Territorial Epidemiologists (CSTE) case definition for pertussis.
Citation: Lee AD, Cassiday PK, Pawloski LC, Tatti KM, Martin MD, Briere EC, et al. (2018) Clinical evaluation and validation of laboratory methods for the diagnosis of Bordetella pertussis infection: Culture, polymerase chain reaction (PCR) and anti-pertussis toxin IgG serology (IgG-PT). PLoS ONE 13(4): e0195979. https://doi.org/10.1371/journal.pone.0195979
Editor: Daniela Flavia Hozbor, Universidad Nacional de la Plata, ARGENTINA
Received: July 25, 2017; Accepted: April 3, 2018; Published: April 13, 2018
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The authors received no specific funding for this work. The funder IHRC Inc provided support in the form of salaries for athor ADL, but did not have any additional roles int he study design, data collection and analysis, decision to publish or preparation of the mansucript. The specific roles of this author are articulated in the author contributions section.
Competing interests: Adria D. Lee is employed by IHRC Inc. There are no patents, products in development or marketed products to declare. This does not alter our adherence to all the PLOS ONE policies on sharing data and materials.
Introduction
Pertussis is a highly contagious, prolonged respiratory illness caused by the Gram-negative bacterium Bordetella pertussis. Despite high childhood vaccination rates [1, 2], pertussis is one of the most poorly controlled bacterial vaccine-preventable diseases in the United States [3]. The annual incidence of pertussis has increased dramatically in recent years; 48,277 cases were reported in 2012, which is the highest number of cases reported since 1955 [4]. Incidence is highest among infants less than one year of age, who are too young to be fully vaccinated [4]. Additionally, over 90% of pertussis-related deaths occur in this age group [5].
To meet the Council of State and Territorial Epidemiologists (CSTE) clinical case definition, a person must have a cough illness lasting two or more weeks and at least one characteristic pertussis symptom: paroxysmal coughing, inspiratory whoop, or post-tussive vomiting [5]. Diagnosing pertussis is challenging because other respiratory pathogens cause pertussis-like illness, and the clinical presentation of pertussis varies with age and vaccination status [6, 7]. Thus, clinically accurate laboratory diagnostic tests are essential for the confirmation of infection by B. pertussis.
Due to its 100% specificity, culture is considered the “gold standard” pertussis diagnostic test. However, it has very low sensitivity, with estimates ranging from 12%-60% [8], and results may take 1–2 weeks to obtain [9]. To obtain viable bacteria for isolation, nasopharyngeal (NP) specimens should be collected within the first two weeks of illness, when symptoms are more likely to be non-specific and physicians might not consider pertussis as the diagnosis. Additionally, the organism can be difficult to isolate, especially if the patient has been recently vaccinated or received antibiotics against pertussis [10].
Polymerase chain reaction (PCR) is the most commonly used pertussis diagnostic test in the United States [11]. Real-time PCR (RT-PCR) can be completed in two to 24 hours, which allows for rapid diagnosis. Since PCR does not require the collection of viable bacteria to be positive, it is more sensitive than culture, with estimates ranging from 70%-99% when performed during the first three to four weeks of cough [12, 13]. Multi-target RT-PCR assays are used to differentiate between Bordetella species, and can be highly specific [14]. However, environmental contamination of clinical specimens in clinics and cross-contamination within laboratories has been associated with false positive PCR results and several pseudo-outbreaks of pertussis in recent years [15–17].
Although not included in the CSTE case definition, serologic tests are increasingly used to diagnose recent pertussis infections. Currently, the lack of commercially available, validated and standardized serodiagnostic assays limits their use in routine clinical practice in the United States and elsewhere [18, 19].
The Centers for Disease Control and Prevention (CDC) Pertussis and Diphtheria Laboratory has developed its own PCR and serological assays to diagnose pertussis. The multi-target RT-PCR assay combines a pertussis toxin subunit 1 (ptxS1) singleplex assay and a multiplex assay targeting the insertion sequence 481 (IS481), B. parapertussis IS1001 (pIS1001), and B. holmesii IS1001-like (hIS1001) to distinguish between Bordetella species and identify co-infections. Specimens with IS481 cycle threshold (Ct) values in the range 35–40 are classified as indeterminate for B. pertussis infection, as this range indicates specimens containing less than one bacterium per reaction, which may either be truly positive, or falsely positive due to contamination [14]. The anti-pertussis toxin IgG (IgG-PT) serological enzyme-linked immunosorbent assay (ELISA) is specific for B. pertussis and has been used to confirm pertussis as the causative agent of recent outbreaks [15, 20, 21]. It has been analytically validated using the World Health Organization (WHO) pertussis reference standard, and a study in adults suggests it can be used to diagnose pertussis as early as six months following receipt of Tdap, the adolescent and adult pertussis toxin-containing booster vaccine, as vaccine-induced antibody titers are expected to fall below the diagnostic cutoff by this time [18, 22].
Imperfect pertussis diagnostics may contribute to an underestimation or overestimation of disease burden across the age spectrum, and compromise prevention programs, surveillance activities, vaccine effectiveness studies, and outbreak management. For these reasons, we assessed the relative clinical validity by time since cough onset of CDC’s array of pertussis diagnostics by evaluating the sensitivities and specificities of culture, multi-target RT-PCR, and IgG-PT serology for the diagnosis of B. pertussis infection.
Methods
Study population
Individuals at least three years of age who met the inclusion criteria listed in Table 1 were invited to participate in the study. Participants were recruited from July 2007 to February 2011 by state and local public health investigators through routine surveillance and outbreak investigations, and by Emerging Infections Programs (EIP) investigators collaborating with health management organizations and private and public healthcare providers. Participants were identified through prospective reviews of hospital admissions, laboratory test orders, emergency department visits, and/or outbreak investigations in California, Colorado, Georgia, Minnesota, New Mexico, and New York. Participants were also recruited by CDC employees during outbreak investigations beginning in 2008.
Demographic and clinical information, including vaccination history, presence of pertussis symptoms, duration of cough illness, and recent use of antibiotics, were collected from each participant during the enrollment visit.
Ethics statement
The study was approved by the Centers for Disease Control and Prevention Institutional Review Board (IRB; #5029), Children’s Hospital and Research Center Oakland IRB (#2007–054), Committee for the Protection of Human Subjects, California Health and Human Services Agency (#07-04-03), Kaiser Permanente Northern California IRB (#CN-07TGree-02-H), Kaiser Permanente of Colorado IRB (#CO-07MRaeb-01), Georgia Department of Human Resources IRB (#070208), Kaiser Permanente Georgia IRB (#GA-07RDavi-01), Minnesota Department of Health IRB (#07–159), The University of New Mexico Health Sciences Human Research Review Committee (#07–273), and the Biomedical Research Alliance of New York IRB, Montefiore Medical Center (#07-10-245-01). Each enrolled participant provided written informed consent. Written parental permission was obtained for participants less than 18 years of age in addition to adolescent assent for participants aged 11–18 years.
Specimen collection and preparation
A NP aspirate or swab specimen and a blood specimen were collected from each participant during the enrollment visit. Swabs were immediately stored in tubes containing 2 mL of 1% casamino acid broth and vortexed for five minutes in a biosafety cabinet (BSC). Within 24 hours of specimen collection, the NP aspirate or swab suspension was divided into 300 μL aliquots in a Bordetella species DNA-free BSC in a room where no culture was performed. One aliquot was reserved for culture by the site laboratory, and the others were stored at −40°C to −80°C until shipped to CDC on dry ice for PCR testing. For NP aspirates collected by CDC employees during outbreak responses, all aliquots were shipped overnight to CDC for culture and PCR. Whole blood was allowed to clot for 30–45 minutes at room temperature and was centrifuged within two hours of collection. The serum was separated and divided into 300 μL aliquots within 24 hours of blood collection. Aliquots were stored at −40°C to −80°C until shipped to CDC on dry ice. Participants who had been coughing for ≤ 2 weeks at enrollment (i.e., 1–14 days) were asked to return in two to four weeks for collection of a second, convalescent blood specimen which was processed similarly.
Laboratory diagnostic tests
NP specimens were plated onto Regan-Lowe agar plates with and without cephalexin, which were incubated at 37°C with high humidity under ambient air and examined daily for seven to ten days. Colonies were stained to check for Gram-negative coccobacilli, and Bordetella species were confirmed by direct fluorescent antibody (DFA) testing, slide agglutination using specific antisera, and biochemical tests, as available, at each laboratory site. dx.doi.org/10.17504/protocols.io.kq9cvz6; dx.doi.org/10.17504/protocols.io.kvtcw6n.
Multi-target RT-PCR was performed at CDC as previously described [12, 14]. dx.doi.org/10.17504/protocols.io.kvxcw7n.
IgG-PT ELISAs were performed at CDC as previously described [18]. Blood specimens collected ≤ 2 weeks after cough onset were classified as “acute” specimens, and specimens collected > 2 weeks after onset were classified as “convalescent” specimens. For all specimens, antibody concentrations ≥ 94 EU/mL were considered positive for pertussis, concentrations 49–93 EU/mL were considered indeterminate, and concentrations < 49 EU/mL were considered negative [23]. Log-transformed antibody concentrations were calculated as log10 (antibody concentration +1). dx.doi.org/10.17504/protocols.io.kvwcw7e.
Sensitivity and specificity analyses
Statistical analyses were conducted using SAS 9.3 (SAS Institute, Inc., Cary, NC) and Latent Gold 4.0 (Statistical Innovations, Inc., Belmont, MA). Sensitivities and specificities of PCR, acute and convalescent serology, and the clinical case definition were estimated using culture as the “gold standard” test. However, due to its low sensitivity, culture is an imperfect gold standard and this method may produce biased estimates [24]. Therefore, sensitivities and specificities were also estimated using composite reference standard (CRS) analysis and latent class analysis (LCA).
CRS reduces bias by combining the results of the imperfect gold standard and a more sensitive diagnostic measure to create a new reference standard [24]. The CRS was created by combining culture and PCR results and was used to estimate the sensitivity and specificity of acute and convalescent serology and the clinical case definition. If either culture or PCR was positive, the CRS was defined as positive. If both tests were negative, the CRS was defined as negative. If one test result was negative and the other was missing, the CRS was defined as missing.
LCA was used to estimate the sensitivity and specificity of each diagnostic measure and the prevalence of pertussis in the study population. In LCA, gold standard bias is reduced by considering all diagnostic measures as imperfect. The statistical model combines the results of each measure to define an unmeasured latent variable that indicates true disease status. The model calculates the probability of each participant being classified as a case or a non-case, as well as the overall probability of participants being classified as cases (i.e., the prevalence of pertussis within the study population), based on participants’ results on at least three diagnostic measures and an assumption of conditional independence between the measures. A bivariate residual (BVR) substantially greater than 1 indicates a violation of this assumption [25]. To address this issue, a conditional dependence model is fit that includes a direct effect between two statistically associated measures. Good model fit is indicated by low, non-significant likelihood ratio statistics (L2) and a low proportion of classification errors [25].
Sensitivities and specificities were estimated among participants with non-missing data on culture, PCR, serology, and the clinical case definition. Several models were included to assess the effect of the timing of specimen collection on the sensitivity and specificity estimates. Model 1 included participants with all specimens collected 1–29 days after cough onset, Model 2 included participants with all specimens collected > 2 weeks after cough onset, and Model 3 included participants with all specimens collected ≤ 2 weeks after onset. Model 4 included participants with NP specimens collected ≤ 2 weeks after onset and a blood specimen collected 2–4 weeks later. Models 5A and 5B included participants with NP and blood specimens collected ≤ 2 weeks after onset, and a second blood specimen collected 2–4 weeks later.
Results
Demographics and clinical characteristics
A total of 868 persons with cough illness were enrolled in the study between July 2007 and February 2011 (Table 2). The mean age at enrollment was 32 years (range: 3–83 years), and participants enrolled in the study an average of 14 days after their cough onset (range: 1–29 days). Overall, 58.5% (508/868) of participants experienced at least one characteristic pertussis symptom. Of the 310 (35.7%) participants who responded as having been previously vaccinated against pertussis, 96.1% (298/310) were able to provide the date of their most recent vaccination. The median time between their most recent receipt of a pertussis-containing vaccine and study enrollment was four years (range: 0–40 years). Twenty participants had been vaccinated less than six months prior to their cough onset. Among the 190 participants who reported taking antibiotics prior to their enrollment in the study, the mean time between the start of antibiotic use and study enrollment was 10 days (range: 1–28 days). In addition, participants who reported antibiotic use were more likely to enroll later than those who did not report antibiotic use, on average (16 days vs. 13 days, p-value <0.0001).
Laboratory diagnostic test results
The laboratory diagnostic test results are described in Table 3. In total, 13.6% (118/868) of participants were positive for B. pertussis infection by at least one laboratory diagnostic test. B. pertussis was isolated from 2.5% of NP specimens, and there were no differences in the proportion of culture-positive specimens among those with and without prior antibiotic use (2.3% vs. 2.9%, p-value = 0.74). Seventeen of the 22 culture-positive participants were also PCR-positive for B. pertussis. Acute and convalescent blood specimens were collected an average of eight days (range: 1–14 days) and 24 days (range: 15–69 days) after cough onset, respectfully. Of those with positive convalescent serology, 77.4% (65/84) also met the clinical case definition, and 69.0% (58/84) were enrolled > 2 weeks after their cough onset and had negative results by both culture and PCR. Additionally, two convalescent serology-positive participants were vaccinated less than six months prior to cough onset (vaccinated 2.2 months and 5.7 months prior); both reported cough lasting three weeks and were culture- and PCR-negative, and one met the clinical case definition.
Fig 1 shows the proportion of participants in each age group who tested positive for B. pertussis by culture, PCR, acute serology, and/or convalescent serology. Overall, 61.9% (73/118) of those positive by at least one laboratory test were ≤ 19 years of age. No participants in the 65+ year age group tested positive for B. pertussis by culture, PCR, or acute serology; one participant was positive by convalescent serology. The effect of the timing of the clinical specimen collection on the laboratory diagnostic test result was also assessed (Fig 2). Overall, 68.2% (15/22) of the culture-positive NP specimens and 51.6% (16/31) of the PCR-positive NP specimens were collected ≤ 14 days after cough onset. In contrast, 78.5% (84/107) of the serology-positive blood specimens were collected 15–40 days after cough onset.
Acute sera are collected ≤ 2 weeks after cough onset, and convalescent sera are collected > 2 weeks after cough onset.
Three hundred and five participants provided both an acute and convalescent blood specimen. The median number of days between blood specimen collections was 20 days (range: 13–62 days). The log-antibody concentrations at both time points are shown in Fig 3. Eight of the 18 participants with discordant results between the acute and convalescent draws were only positive at the convalescent time point. Fig 4 shows the log-antibody concentrations of nine participants who were culture-positive and provided both an acute and convalescent blood specimen. On average, their blood specimens were collected 20 days apart (range: 14–26 days). Two participants were positive by serology at the acute time point. By the convalescent time point, five participants were positive and one was indeterminate. Despite being culture-positive, three participants were serology-negative at both time points, including two with undetectable antibody concentrations at both time points (Fig 4). One of these two participants was also PCR-negative for all Bordetella species, while the eight remaining participants were PCR-positive for B. pertussis.
Acute sera are collected ≤ 2 weeks after cough onset, and convalescent sera are collected > 2 weeks after cough onset. Open circles indicate participants with discordant results, and shaded circles indicate participants with concordant results at the acute and convalescent time periods. Dashed lines indicate the log-transformed diagnostic cutoff values. Log-transformed values ≥ 1.98 EU/mL were considered positive for pertussis, values 1.70–1.97 EU/mL were considered indeterminate, and values <1.96 EU/mL were considered negative.
Acute sera are collected ≤ 2 weeks after cough onset, and convalescent sera are collected > 2 weeks after cough onset. Two participants had undetectable antibody concentrations at both the acute and convalescent time points. Dashed lines indicate the log-transformed diagnostic cutoff values. Log-transformed values ≥ 1.98 EU/mL were considered positive for pertussis, values 1.70–1.97 EU/mL were considered indeterminate, and values <1.96 EU/mL were considered negative.
Sensitivity and specificity analyses
LCA requires participants to have a result for all diagnostic measures included in the model. Therefore, 275 participants with a missing result for at least one diagnostic measure were excluded from the sensitivity and specificity analyses. Additionally, 48 participants with an indeterminate PCR or serology result were excluded to reduce potential misclassification. Two participants who were PCR-positive for B. parapertussis and two who were PCR-positive for B. holmesii were categorized as “B. pertussis PCR-negative.” Sensitivity and specificity estimates are listed in Table 4.
Model 1.
Model 1 includes 545 participants with complete data on pertussis culture, PCR, convalescent serology, and the clinical case definition. Clinical specimens were collected 1–29 days after cough onset. When compared to culture, PCR was the most sensitive and specific diagnostic measure (Table 4). Convalescent serology was the least sensitive, and the clinical case definition was the least specific measure. Use of the CRS method increased the sensitivity estimates for convalescent serology and the clinical case definition (Table 4). Bivariate residuals of the initial LCA model indicated that there was an association between convalescent serology and the clinical case definition (BVR = 19.8). Therefore, a second LCA model was fit that included a direct effect between these two measures, which improved model fit (L2 = 8.4; p-value = 0.14). The rate of expected classification errors was 0.2%. PCR was the most sensitive and culture was the least sensitive diagnostic measure in the LCA model (Table 4). The sensitivity and specificity estimates for PCR were similar to those obtained when culture was considered the “gold standard,” and the sensitivity and specificity of both convalescent serology and the clinical case definition were similar to the estimates from the CRS method. The estimated prevalence of pertussis was 3.6% (95% CI: 1.8%- 5.4%). Nineteen participants had a probability of having pertussis ≥ 0.50, and were thus classified as cases (Table 5). The LCA model classified 80.6% (58/72) of participants with positive convalescent serology as non-cases.
Model 2.
Model 2 includes 281 participants with all clinical specimens collected > 2 weeks after cough onset. Only five participants had positive culture results, and all five were PCR-positive and met the clinical case definition (Table 4). The CRS-based sensitivity estimate for convalescent serology was higher than the culture-based estimate, while the specificity estimates for convalescent serology and the clinical case definition were similar to the corresponding culture-based estimates (Table 4). Bivariate residuals indicated an association between culture and PCR in the LCA model (BVR = 15.3), therefore a direct effect between these two measures was included. This model had good fit (L2 = 4.3; p-value = 0.51); however, the probability of classification errors was 6.3%, indicating poor separation between the latent classes. Convalescent serology was more sensitive than both culture and PCR in the LCA model (Table 4). The estimated prevalence of pertussis was 18.2% (95% CI: 9.7%- 26.8%). Fifty participants were classified as cases, and 15.8% (9/57) of participants with positive convalescent serology results were classified as non-cases (S1 Table).
Model 3.
To further assess the effect of the timing of blood specimen collection, 347 participants who had all clinical specimens collected ≤ 2 weeks after cough onset were included in Model 3. Acute serology was the least sensitive diagnostic measure by all three methods (Table 4). The CRS-based sensitivity and specificity estimates for acute serology and the clinical case definition were similar to the culture-based estimates (Table 4). The LCA model had good fit (L2 = 4.8; p-value = 0.57), and the probability of classification errors was 0.3%. The estimated prevalence of pertussis was 4.6% (95% CI: 2.02%- 7.12%). Fifteen participants were classified as cases, and 86.7% (13/15) of participants with positive acute serology results were classified as non-cases (S2 Table).
Model 4.
Of those who enrolled ≤ 2 weeks after cough onset, 264 participants returned for collection of a convalescent blood specimen, and were included in Model 4. The culture- and CRS-based sensitivity and specificity estimates for convalescent serology and the clinical case definition were similar (Table 4). The LCA model had good fit (L2 = 4.8; p-value = 0.57), and the probability of classification errors was 0%. Culture was the most sensitive diagnostic measure (Table 4). The sensitivities and specificities of PCR, convalescent serology, and the clinical case definition were similar to the culture- and CRS-based estimates. The estimated prevalence of pertussis was 3.2% (95% CI: 1.1%- 5.4%). Eight participants were classified as cases, and 66.7% (10/15) of participants with positive convalescent serology were classified as non-cases (S3 Table).
Models 5A and 5B.
To directly compare the sensitivity and specificity of acute and convalescent serology, Models 5A and 5B included 258 participants from Model 4 who had NP and blood specimens collected ≤ 2 weeks after cough onset, and returned for collection of a convalescent blood specimen. There were no differences in the culture- and CRS-based sensitivity and specificity estimates between the two models, and the culture-based estimates for acute and convalescent serology and the clinical case definition were similar to the corresponding CRS-based estimates (Table 4). Bivariate residuals indicated associations between several pairs of diagnostic measures in the LCA model. Thus, two LCA models were run, each controlling for a different pair of associations. LCA Model 5A (Table 4) accounted for the correlation between the serology results by including a direct effect between acute and convalescent serology (BVR = 3.3). This model had good fit (L2 = 7.0; p-value = 0.99), and the probability of classification errors was 0%. The LCA-based sensitivity and specificity estimates for PCR, acute serology, convalescent serology, and the clinical case definition were similar to the culture- and CRS-based estimates of Models 5A and 5B and the LCA-based estimates of Model 4 (Table 4). The estimated prevalence of pertussis was 3.3% (95% CI: 1.1%- 5.5%). Eight participants were classified as cases, and 80.0% (8/10) of participants with positive acute serology and 66.7% (10/15) with positive convalescent serology were classified as non-cases (S4 Table).
LCA Model 5B (S5 Table) accounted for the largest BVR in the model by including a direct effect between culture and PCR (BVR = 12.9). After controlling for this association, an additional association between culture and the clinical case definition was identified (BVR = 6.4). The LCA model controlling for both of these associations had good fit (L2 = 7.0; p-value = 0.99), and the probability of classification errors was 0.0%. However, 37.5% (3/8) of culture- and/or PCR-positive participants were classified as non-cases and 100% of acute and/or convalescent serology positive participants were classified as cases (S6 Table), which biased the sensitivity estimates for all diagnostic measures in the model (S5 Table).
Discussion
Our results demonstrate that the time since cough onset should be considered when determining which laboratory test to use. The sensitivity of culture and PCR decreased from greater than 70% and 80%, respectively, during the first two weeks following cough onset to 10% and 21%, respectively after the second week following cough onset. The IgG-PT ELISA was most sensitive > 2 weeks after cough onset. In contrast, serology performed on serum collected during the first two weeks following cough onset demonstrated unacceptably low sensitivity (13%-24%). Additionally, there were two study participants with positive cultures that failed to mount a detectable antibody response between their acute and convalescent specimens, emphasizing that serology is an imperfect diagnostic tool. Despite this limitation, serology provides an additional option for patients presenting to the physician late in their illness. Similarly, data from IgG-PT ELISAs used for routine diagnosis in Massachusetts and São Paulo, Brazil have demonstrated that the inclusion of serology provides a more accurate estimate of the true burden of disease by identifying adolescent and adult pertussis infections that were missed by culture and PCR [3, 26, 27].
Traditionally, the role of a gold standard test was assigned to pertussis culture, which requires the collection of viable bacteria from the nasopharynx. However, we found that culture was only 64% sensitive in Model 1, which is similar to previously reported estimates [3, 8]. Due to the insensitivity of culture, sensitivity and specificity estimates were also calculated using CRS and LCA, which attempt to reduce gold standard bias by incorporating information from additional diagnostic measures [24]. Despite using different methodologies, CRS and LCA tended to provide similar estimates for serology and the clinical case definition within each model.
Although CRS and LCA methods reduce gold standard bias, they do not eliminate this bias completely and are subject to other limitations. Since culture and PCR are most sensitive during the first few weeks of cough [13], CRS may not have corrected the potential misclassification of participants who enrolled several weeks after cough onset. With the exception of Model 2, LCA models tended to classify participants with negative culture and PCR results, but with positive convalescent serology, as non-cases; if these participants had pertussis, this classification would bias upwards the sensitivity of PCR and culture, while biasing downwards specificity estimates for serology. This bias likely explains the apparent exaggerated estimates of culture sensitivity in Models 4 and 5A. Additionally, the low prevalence of pertussis in the study population resulted in wide confidence intervals for all of our sensitivity and specificity estimates.
While LCA is based on the assumption that all diagnostic measures are independent of one another, this assumption is rarely met in practice [28]. Failure to account for associations between diagnostic measures produces a poor fitting model that gives too much weight to those diagnostic measures [28, 29]. Although including a direct effect will improve model fit, it may also change how cases are classified by the model. As demonstrated by Model 5B, which failed to account for the association between acute and convalescent serology, over 1/3 of culture-positive participants were incorrectly classified as non-cases, which resulted in biased sensitivity and specificity estimates for all diagnostic measures in the model. In contrast, Model 5A accounted for this association and correctly classified all culture-positive participants as cases. Thus, case classification should be examined in addition to model fit statistics to minimize bias.
Our analyses were limited by missing data on diagnostic test results. The convalescent serology result was frequently missing for those who enrolled ≤ 2 weeks after cough onset since it required participants to return at a later date to provide a second blood specimen. Indeterminate PCR or serology results were excluded from these analyses. The indeterminate PCR category indicates specimens containing less than one bacterial genome, which may either indicate truly infected persons or a false-positive result [14], and the indeterminate serology category may represent either an acute, recent infection, or possibly, prior vaccination [23]. Thus, these participants’ specimens were not able to be confidently classified as either positive or negative. Additionally, we were unable to assess the effect of age, previous antibiotic usage, and pertussis vaccination status on the performance of each diagnostic test due to the low number of positive clinical specimens in the study population and missing data (i.e., LCA models did not include these variables). Not controlling for these factors in the analysis may have affected our sensitivity and specificity estimates. Additional studies should be conducted to further understand the effect of age, previous antibiotic usage, and vaccination status on the performance of pertussis diagnostic tests.
Our analyses included two serology-positive participants who were vaccinated less than six months prior to specimen collection. As both were culture- and PCR-negative, potential false positive serology results for these two participants had little impact on the model classifications or the resulting sensitivity and specificity estimates. In an earlier study to estimate the effect of Tdap vaccination on serodiagnosis of pertussis infection, antibody titers rapidly declined below the diagnostic cutoff (94 EU/mL) by 75 days post-vaccination; a six-month waiting period between vaccination and serology testing was suggested to account for potential variations in antibody response following receipt of different vaccine formulations [22].
Surprisingly, there was no difference in the proportion of participants with positive culture results between those with and without previous antibiotic use. It is impossible to determine whether this finding is related to data quality issues (i.e., participants not reporting prior antibiotic use) or another explanation. Additionally, when the latent class modeling was restricted to those without prior antibiotic use, the overall results were similar (see S7 Table). For these reasons, participants with self-reported antibiotic use in the month prior to specimen collection were included in our main latent class models. Since previous antibiotic use was expected to decrease the sensitivity of culture [9] and PCR [30], the inclusion of participants with possible recent antibiotic use in the latent class models possibly resulted in biased estimates as described earlier.
Our sensitivity and specificity findings should not be viewed as generalizable to all commercially available pertussis diagnostic tests, as PCR and serology both suffer from a lack of standardization across US laboratories. PCR practices vary by DNA extraction method, PCR targets, and diagnostic cutoff values used [11, 12, 16, 31], which contribute to differences in analytic sensitivity between assays [11, 31]. PCR assays also vary in their ability to detect other Bordetella species, which has resulted in the misidentification of other Bordetella species as B. pertussis [16, 32]. Similarly, commercial serological assays utilize various combinations of antibodies, pertussis antigens, and diagnostic cutoff values, and are not all calibrated to international reference standards [33–35]. This variability has prevented the inclusion of serology as a confirmatory laboratory test in the United States [5]. A recent analysis of commercial assays in the United States found that assays with the best performance qualities were IgG-PT assays that were calibrated to a reference standard [19]. This message appears to be reaching the commercial manufacturers, as more and more calibrated, IgG-PT assays are appearing on the market to date, suggesting that potential harmonization of serologic assays may be possible. Ideally, in the near future, potential FDA approval of these high performing assays would then allow for increased clinical utility in the United States market and improve serodiagnostic practices throughout the country.
Overall, our findings demonstrate the importance of the timing of clinical specimen collection for the diagnosis of pertussis, and the need for an accurate, standardized diagnostic test that can be used several weeks after cough onset. Consideration should be given to including IgG-PT serology as a confirmatory test for non-recently vaccinated patients with specimens collected > 2 weeks after cough onset in the CSTE case definition for pertussis to identify additional cases, especially among adolescents and adults with clinical symptoms of pertussis who seek treatment after the optimal time period for culture and PCR testing. With the recent resurgence of pertussis, it is important that we further our understanding of the epidemiology and transmission of B. pertussis in the United States, and continue evaluating current pertussis control strategies through the appropriate use of highly accurate and validated diagnostic tests.
Supporting information
S1 Table. Diagnostic measure result patterns and case classification of participants in Model 2 (n = 281).
Participants in Model 2 enrolled in the study > 2 weeks after cough onset. Positive test results are indicated by (+), and negative test results are indicated by (−). Participants with missing data or indeterminate PCR or convalescent serology results were excluded from the analysis. The LCA model contains a direct effect between culture and PCR.
https://doi.org/10.1371/journal.pone.0195979.s001
(PDF)
S2 Table. Diagnostic measure result patterns and case classification of participants in Model 3 (n = 347).
Participants in Model 3 enrolled in the study ≤ 2 weeks after cough onset. Positive test results are indicated by (+), and negative test results are indicated by (−). Participants with missing data or indeterminate PCR or convalescent serology results were excluded from the analysis.
https://doi.org/10.1371/journal.pone.0195979.s002
(PDF)
S3 Table. Diagnostic measure result patterns and case classification of participants in Model 4 (n = 264).
Participants in Model 4 enrolled in the study ≤ 2 weeks after cough onset and returned for collection of a convalescent blood specimen. Positive test results are indicated by (+), and negative test results are indicated by (−). Participants with missing data or indeterminate PCR or convalescent serology results were excluded from the analysis.
https://doi.org/10.1371/journal.pone.0195979.s003
(PDF)
S4 Table. Diagnostic measure result patterns and case classification of participants in Model 5A (n = 258).
Participants in Model 5A enrolled in the study ≤ 2 weeks after cough onset and had both acute and convalescent blood specimens collected. Positive test results are indicated by (+), and negative test results are indicated by (−). Participants with missing data or indeterminate PCR or convalescent serology results were excluded from the analysis. The LCA model contains a direct effect between acute and convalescent serology.
https://doi.org/10.1371/journal.pone.0195979.s004
(PDF)
S5 Table. Sensitivity and specificity estimates of B. pertussis diagnostic measures in participants in Model 5B (n = 258).
Participants in Model 5B enrolled in the study ≤ 2 weeks after cough onset and had both acute and convalescent blood specimens collected. The LCA model contains direct effects between culture and PCR, and culture and the clinical case definition.
https://doi.org/10.1371/journal.pone.0195979.s005
(PDF)
S6 Table. Diagnostic measure result patterns and case classification of participants in Model 5B (n = 258).
Participants in Model 5B enrolled in the study ≤ 2 weeks after cough onset and had both acute and convalescent blood specimens collected. Positive test results are indicated by (+), and negative test results are indicated by (−). Participants with missing data or indeterminate PCR or convalescent serology results were excluded from the analysis. The LCA model contains direct effects between culture and PCR, and culture and the clinical case definition.
https://doi.org/10.1371/journal.pone.0195979.s006
(PDF)
S7 Table. Sensitivity and specificity estimates of B. pertussis diagnostic measures in participants in Model 6 (n = 434).
Participants in Model 6 had no reported antibiotic use one month prior to specimen collection, and enrolled in the study 1–29 days after cough onset. The latent class analysis (LCA) model contains a direct effect between convalescent serology and the clinical case definition.
https://doi.org/10.1371/journal.pone.0195979.s007
(PDF)
Acknowledgments
The Clinical Validation Study Group consists of Lauren Pasutti, Margret Wheeler, and Kathryn Wymore for the California Emerging Infections Program; Kathryn O. Boney for the Centers for Disease Control and Prevention; Heidi Flori for the Children’s Hospital of Oakland; Hugh Maguire and Justin Nucci for the Colorado Department of Public Health and Environment; Julie Gabel, Kim Kilgour, and Jessica Tuttle for the Georgia Department of Public Health; Marsha Raebel for the Kaiser Permanente Colorado Institute for Health Research; Cynthia Kenyon and Jennifer Palm for the Minnesota Department of Health; Pascale Léonard for the New Mexico Department of Health, Scientific Laboratory Division; Arthur Reingold for the University of California Berkeley School of Public Health; Tsoline Kojaoghlanian, Michael Levi, and Nathan Litman for The Children’s Hospital at Montefiore/Albert Einstein College of Medicine; and Manisha Patel for The Children’s Hospital at Montefiore and the Centers for Disease Control and Prevention.
We gratefully acknowledge the following institutions for their contributions to this study: California Emerging Infections Program, Alameda County Public Health Department, Alameda County Public Health Laboratory, Children’s Hospital Oakland, Colorado Department of Public Health and Environment, County of San Diego Health and Human Services Agency, Georgia Department of Public Health, Georgia Public Health Laboratory, Kaiser Permanente Colorado Institute for Health Research, Minnesota Department of Health, New Mexico Department of Health Scientific Laboratory Division, University of California Berkeley School of Public Health, and The Children’s Hospital at Montefiore/Albert Einstein College of Medicine. The findings and conclusions of this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
References
- 1. Seither R, Masalovich S, Knighton CL, Mellerson J, Singleton JA, Greby SM. Vaccination coverage among children in kindergarten—United States, 2013–2014 school year. MMWR. 2014;63(41):913–20. pmid:25321068
- 2. Elam-Evans LD, Yankey D, Singleton JA, Kolasa M. National, state, and selected local area vaccination coverage among children aged 19–35 months—United States, 2013. MMWR. 2014;63(34):741–8. pmid:25166924
- 3. CDC. Preventing tetanus, diptheria, and pertussis among adolescents: Use of tetanus toxoid, reduced diptheria toxoid, and acellular pertussis vaccines: Recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR. 2006;55(No. RR-3):1–46.
- 4. CDC. Summary of notifiable diseases—United States, 2012. MMWR. 2014;61(53):1–124. pmid:25233134
- 5.
Davis JP. Revision of the pertussis surveillance case definition to more accurately capture the burden of disease among infants <1 year of age. Council of State and Territorial Epidemiologists; 2013.
- 6. Frumkin K. Pertussis and persistent cough: practical, clinical and epidemiologic issues. The Journal of emergency medicine. 2013;44(4):889–95. pmid:23287746
- 7. Wirsing von Koenig CH. Pertussis diagnostics: Overview and impact of immunization. Expert Rev Vaccines. 2014;13(10):1167–74. pmid:25142439
- 8. Wendelboe AM, Van Rie A. Diagnosis of pertussis: a historical review and recent developments. Expert Rev Mol Diagn. 2006;6(6):857–64. pmid:17140372
- 9.
CDC. Pertussis. In: Atkinson W, Wolfe S, Hamborsky J, editors. Epidemiology and Prevention of Vaccine-Preventable Diseases. 12 ed. Washington, DC: Public Health Foundation; 2012. p. 215–32.
- 10.
Faulkner AE, Skoff T, Martin S, Cassiday P, Tondella ML, Liang J, et al. Pertussis. In: Roush SW, McIntyre L, Baldy LM, editors. VPD Surveillance Manual. 5 ed. Atlanta, GA: CDC; 2011. p. 1–12.
- 11. Tatti KM, Martin SW, Boney KO, Brown K, Clark TA, Tondella ML. Qualitative assessment of pertussis diagnostics in United States laboratories. The Pediatric infectious disease journal. 2013;32(9):942–5. pmid:23587980
- 12. Tatti KM, Wu KH, Tondella ML, Cassiday PK, Cortese MM, Wilkins PP, et al. Development and evaluation of dual-target real-time polymerase chain reaction assays to detect Bordetella spp. Diagnostic microbiology and infectious disease. 2008;61(3):264–72. pmid:18440175
- 13.
CDC. Diagnosis Confirmation 2014 [updated August 28, 2013. http://www.cdc.gov/pertussis/clinical/diagnostic-testing/diagnosis-confirmation.html.
- 14. Tatti KM, Sparks KN, Boney KO, Tondella ML. Novel multitarget real-time PCR assay for rapid detection of Bordetella species in clinical specimens. Journal of clinical microbiology. 2011;49(12):4059–66. pmid:21940464
- 15. Mandal S, Tatti KM, Woods-Stout D, Cassiday PK, Faulkner AE, Griffith MM, et al. Pertussis Pseudo-outbreak linked to specimens contaminated by Bordetella pertussis DNA From clinic surfaces. Pediatrics. 2012;129(2):e424–30. pmid:22250029
- 16. CDC. Outbreaks of respiratory illness mistakenly attributed to pertussis—New Hampshire, Massachusetts, and Tennessee, 2004–2006. MMWR. 2007;56(33):837–42. pmid:17717512
- 17. Salimnia H, Lephart PR, Asmar BI, Prebelich D, Paulson E, Fairfax MR. Aerosolized vaccine as an unexpected source of false-positive Bordetella pertussis PCR results. Journal of clinical microbiology. 2012;50(2):472–4. pmid:22162551
- 18. Menzies SL, Kadwad V, Pawloski LC, Lin TL, Baughman AL, Martin M, et al. Development and analytical validation of an immunoassay for quantifying serum anti-pertussis toxin antibodies resulting from Bordetella pertussis infection. Clinical and vaccine immunology: CVI. 2009;16(12):1781–8. pmid:19864485
- 19. Pawloski LC, Plikaytis BD, Martin MD, Martin SW, Prince HE, Lape-Nixon M, et al. Evaluation of commercial assays for single-point diagnosis of pertussis in the US. J Pediatric Infect Dis Soc. 2016.
- 20. Wei SC, Tatti K, Cushing K, Rosen J, Brown K, Cassiday P, et al. Effectiveness of adolescent and adult tetanus, reduced-dose diphtheria, and acellular pertussis vaccine against pertussis. Clinical infectious diseases: an official publication of the Infectious Diseases Society of America. 2010;51(3):315–21.
- 21. Rodgers L, Martin SW, Cohn A, Budd J, Marcon M, Terranella A, et al. Epidemiologic and laboratory features of a large outbreak of pertussis-like illnesses associated with cocirculating Bordetella holmesii and Bordetella pertussis—Ohio, 2010–2011. Clinical infectious diseases: an official publication of the Infectious Diseases Society of America. 2013;56(3):322–31.
- 22. Pawloski LC, Kirkland KB, Baughman AL, Martin MD, Talbot EA, Messonnier NE, et al. Does tetanus-diphtheria-acellular pertussis vaccination interfere with serodiagnosis of pertussis infection? Clinical and vaccine immunology: CVI. 2012;19(6):875–80. pmid:22539469
- 23. Baughman AL, Bisgard KM, Edwards KM, Guris D, Decker MD, Holland K, et al. Establishment of diagnostic cutoff points for levels of serum antibodies to pertussis toxin, filamentous hemagglutinin, and fimbriae in adolescents and adults in the United States. Clinical and diagnostic laboratory immunology. 2004;11(6):1045–53. pmid:15539504
- 24. Baughman AL, Bisgard KM, Cortese MM, Thompson WW, Sanden GN, Strebel PM. Utility of composite reference standards and latent class analysis in evaluating the clinical accuracy of diagnostic tests for pertussis. Clinical and vaccine immunology: CVI. 2008;15(1):106–14. pmid:17989336
- 25.
Vermunt J, Magidson J. Latent Gold 4.0 User's Guide. Belmont, Massachusetts: Statistical Innovations, Inc.; 2005.
- 26. Yih WK, Lett SM, des Vignes FN, Garrison KM, Sipe PL, Marchant CD. The increasing incidence of pertussis in Massachusetts adolescents and adults, 1989–1998. The Journal of infectious diseases. 2000;182:1409–16. pmid:11023464
- 27. Vaz-de-Lima LR, Martin MD, Pawloski LC, Leite D, Rocha KC, de Brito CA, et al. Serodiagnosis as adjunct assay for pertussis infection in Sao Paulo, Brazil. Clinical and vaccine immunology: CVI. 2014;21(5):636–40. pmid:24599531
- 28. Qu Y, Tan M, Kutner MH. Random effects models in latent class analysis for evaluating accuracy of diagnostic tests. Biometrics. 1996;52:797–801. pmid:8805757
- 29.
Vermunt J, Magidson J. Latent class cluster analysis. In: Hagenaars J, McCutcheon A, editors. Applied Latent Class Analysis. New York, NY: Cambridge University Press; 2002.
- 30.
CDC. Best practices for healthcare professionals on the use of polymerase chain reaction (PCR) for diagnosing pertussis 2015 [updated 9/8/15. http://www.cdc.gov/pertussis/clinical/diagnostic-testing/diagnosis-pcr-bestpractices.html.
- 31. Williams MM, Taylor TH Jr., Warshauer DM, Martin MD, Valley AM, Tondella ML. Harmonization of Bordetella pertussis real-time PCR diagnostics in the United States in 2012. Journal of clinical microbiology. 2015;53(1):118–23. pmid:25355770
- 32. Burgos-Rivera B, Lee AD, Bowden KE, Faulkner AE, Seaton BL, Lembke BD, et al. Evaluation of Level of Agreement in Bordetella Species Identification in Three U.S. Laboratories during a Period of Increased Pertussis. Journal of clinical microbiology. 2015;53(6):1842–7. pmid:25809969
- 33. Guiso N, Berbers G, Fry NK, He Q, Riffelmann M, Wirsing von Konig CH, et al. What to do and what not to do in serological diagnosis of pertussis: recommendations from EU reference laboratories. European journal of clinical microbiology & infectious diseases: official publication of the European Society of Clinical Microbiology. 2011;30(3):307–12.
- 34. Riffelmann M, Thiel K, Schmetz J, Wirsing von Koenig CH. Performance of commercial enzyme-linked immunosorbent assays for detection of antibodies to Bordetella pertussis. Journal of clinical microbiology. 2010;48(12):4459–63. pmid:20943873
- 35. Watanabe M, Connelly B, Weiss AA. Characterization of serological responses to pertussis. Clinical and vaccine immunology: CVI. 2006;13(3):341–8. pmid:16522775