Results of the National External Quality Assessment for Toxoplasmosis Serological Testing in China

Background Toxoplasmosis is typically diagnosed by serologic testing. External quality assessment (EQA) of clinical laboratories could ensure the accuracy and reliability of serological tests. We assessed the quality of toxoplasma serological assays in Chinese clinical laboratories by an EQA performed between 2004 and 2013 by the National Center for Clinical Laboratories. Methodology and Findings EQA panels were prepared and shipped at room temperature to participating laboratories that employed toxoplasma IgG and IgM serological detection. By 2013, 5,384 EQA test reports for toxoplasma-specific IgM and 2,666 reports for toxoplasma-specific IgG were collected. Enzyme-linked immunosorbent (ELISA) and chemical immunofluorescent assays were the most commonly used detection methods. The overall coincidence rates of negative samples were better than those of positive samples. The overall EQA score for toxoplasma-specific IgM detection ranged between 84.3% and 99.6%. The ratio of laboratories that achieved correct IgG detection ranged from 61.1% to 99.3%. However, the inter- and intra-assay variabilities were found to be considerable. The most common problem was failure to detect low titers of antibody. Conclusion The EQA scheme showed an improvement in toxoplasma serological testing in China. However, further optimization of assay sensitivity to detect challenging samples remains a future challenge.


Introduction
Toxoplasma gondii is a protozoan obligate intracellular parasite that causes the disease toxoplasmosis. It has been estimated that 10% to 70% of the world's population is infected by T. gondii, and the general infection rate of T. gondii in the northeast and south of China was found to be 12.3% [1]. Although the infection is generally asymptomatic or results in a clinical disease that is not recognized, it can cause severe health problems in individuals who are immunocompromised such as congenitally infected infants, transplant recipients, and AIDS patients. Infection of pregnant woman can lead to abortion, hydrocephalus, cerebral calcification, and/or chorioretinitis [2]. Therefore, programs aimed at detecting T. gondii infection in pregnant women by systematic screening have been performed in several countries such as France and Austria [3]. Serologic diagnosis, based on the detection of toxoplasma-specific immunoglobulin (Ig) G and IgM antibodies in the serum, is often used to determine the immune status of patients, and prenatal screening for antibodies is routine practice in many parts of the world [4]. As non-specific clinical manifestations can complicate diagnosis of the disease and because of the importance of early detection of in utero infection, accurate diagnostic tests are essential. Therefore, a number of more sensitive methods, such as a serum IgG avidity test, PCR, and western blotting using serum from mother-baby pairs, have been developed [5]. However, routine screening of toxoplasma-specific IgG and IgM in serum is still mostly used in clinical laboratories, especially in China [1,5].
Quality assurance of serologic testing is important to ensure accurate and reliable screening of susceptible individuals, especially for pregnant women. Therefore, laboratories should participate in external quality assessment (EQA) schemes conducted by independent organizations [6]. Participation in relevant EQA schemes allows for comparing test results between different clinical laboratories, it provides insight into national performance levels, and it allows for improving national performance levels [7]. An EQA program for the evaluation of clinical toxoplasma IgG and IgM serological detection assays in China, was established in 2004 by the National Center for Clinical Laboratories (NCCL). The program allowed clinical laboratories to review their testing process. Throughout the EQA, potential issues associated with the serological tests were identified. The aim of the present study was to assess the quality of toxoplasma serological tests in Chinese clinical laboratories. We based our study on data gathered between 2004 and 2013.

Panel preparation and distribution
The toxoplasma EQA scheme consisted of two distributions of human serum specimens for the examination of toxoplasma IgG and IgM antibodies with a request to report qualitative results twice a year (in May and October). Each distribution consisted of two panels of serum, one to test for toxoplasma IgG, another to test for toxoplasma IgM. Each panel consisted of five serum specimens. The panels were majorly prepared using sera pools of blood donors kindly donated by Shenzhen blood center and Liaoning blood center in China or minor sera pools donated by the company Viron/Serion (Würzburg, Germany) or minorly purchased from Guangzhou Kang Run biological products development co., LTD (Guangzhou, China) (Lot:L3720,W84,W64,W37). The NCCL tested the specimens for toxoplasma-specific IgG and IgM antibodies prior to dispatch, using several commercial enzyme-linked immunosorbent assays (ELISA) or chemiluminescence immunoassay (CIA) kit for toxoplasma IgG and IgM. The tests used to produce the EQA panel of sera for each year and precisely detail for positive sera were shown in Tables 1 and 2. The samples with lower titer (low S/CO ratio) detected by at least three of these kits were classified as weakly positive samples. The titers of negative sera provided were all smaller than cutoff of kits(data not shown). The specimens were divided in screw cap centrifuge tubes and stored at -40°C until distribution. The EQA panels, containing toxoplasma IgG-and IgM-positive specimens as well as negative specimens, were shipped at

Participants
Hospital laboratories from China that employ toxoplasma IgG and IgM serological detection were invited to participate in this EQA scheme. The participating clinical laboratories tested the EQA panels using their standard protocols and returned the test results to NCCL via a data submission form on the NCCL website. Participants using EIAs were asked to provide signal/ cut-off (S/CO) values if possible.

Evaluation of the results and statistical analysis
The EQA data were scored according to qualitative criteria; every correct positive or negative test result in accordance with the expected results of NCCL of the specimens in each panel was assigned a point of 20, while false negative or false positive results were not scored. The highest possible total EQA score per panel was thus 100 points, and an EQA points <80 was considered unqualified. Data were entered into Microsoft Excel spreadsheets and analyzed using Microsoft Excel and the Prism 6 software (GraphPad Software Inc., La Jolla, CA, USA) and SPSS 16.0(SataCorp).

Analysis of the toxoplasma-specific IgM scheme
All IgM test results, acquired by the different routine detection assays as used by the participating hospital laboratories, were analyzed and scored ( Table 3). The ratio of laboratories that correctly identified all five (EQA score = 100) samples within a panel ranged from 45.3% (panel 071) to 95.9% (panel 082). The overall coincidence rates for negative samples (98.4%) were higher than for positive samples (86.7%). A very low ratio of laboratories with an EQA score of 100 (45.3%) was observed for panel 071. Seventy-three laboratories (42.5%) failed to identify two positive samples within this panel, and 21 participants (12.2%) misidentified one positive sample. This was mainly due to the presence of two positive samples with low toxoplasma antibody titers in panel 071. The median S/CO values for the two positive samples were 1.0 (data not shown). Although such a low ratio was never again recorded because the quality of the tests improved, another weakly positive sample (0923) in panel 092, in 2009, remained undetected by 88 participants (37.6%), with a median S/CO value of 1.2. The results from these panels show that there were problems detecting low titers of toxoplasma-specific IgM antibodies. When the results for panel 071 are not taken into consideration, the overall ratio of laboratories with EQA score for toxoplasma-specific IgM detection 80, ranged from 84.3% to 99.6%, with a mean of 95.48%.

Analysis of the toxoplasma-specific IgG scheme
In general, the scores for IgG test results in the EQA scheme were better than those for IgM results ( Table 4). The overall coincidence rates for negative samples (99.1%) were higher than those for positive samples (94.5%). The ratio of laboratories that correctly identified all five samples within a panel ranged from 61.1% (panel 112) to 99.3% (panel 121). Over the period

Analysis of methods used in EQA
The diagnostic assays used by participants during the EQA, over ten years in the case of IgM and over six years in the case of IgG, are summarized in Tables 5 and 6, respectively. Overall, the coincidence rates of the different methods for negative samples were better than the coincidence rates for positive samples. The results showed that the detection rates for positive  Table 7. The results of Table 7 also illustrated that for IgG antibodies testing the total coincidence rates of domestic test and international test performed across the 5 participating labs were all larger than 94.9%, and for IgM antibodies testing the total coincidence rates of domestic test and international test performed across the 5 participating labs were all larger than 94.4% except for one To determine the inter-assay variability of EIA kits from different manufacturers that are currently used in the EQA by more than five laboratories, the variability of log S/CO values was estimated by determining the standard deviation (SD) of the results of all participating laboratories for all the positive samples distributed in EQA scheme. The mean and SD of log S/ CO values for each assay were determined based on the test results of all participants using EIAs, for the IgM-positive (Fig 1A) as well as the IgG-positive specimens (Fig 1B). The mean and SD of log S/CO values for all assays were also shown in Fig 1 as   The intra-assay variability for kits currently used by more than five participants was analyzed by comparing the assay results for individual positive samples distributed in EQA scheme. Results (mean (log S/CO) ± SD) for the IgM assay kits are given in Fig 2, and for the IgG assay kits in Fig 3. The results of individual positive samples for different kits were compared with the overall average result for all EIA kits (mean (log S/CO) ± 1.96 SD). Although the accuracy of IgG and IgM detection assays had improved during recent years [2], the present results demonstrated significant variability among test results from different EIA kits used for toxoplasma-specific IgG and IgM detection. For some samples, the mean (log S/CO) for various kits deviated far from the average mean of all kits. This was observed for the IgM assay kits Intra-assay Variability of EIAs Commonly Used in the EQA Scheme (IgM). Mean (log S/CO) ± SD for all assays are given per sample. Absence of error bars means that the assay was used by less than 5 laboratories. SD below the dotted line at 0.0 refers to false-negative assay results. The S/CO values for all positive samples used by each assay showed a skewed distribution, therefore, the results were transformed to the log normal distribution for analysis (P-P plot).
doi:10.1371/journal.pone.0130003.g002 (Fig 2) as well as for the IgG assay kits (Fig 3). Although the highest SDs for all kits were below +1.96 SD of the average for all individual samples, the lowest SD values were observed beyond -1.96 SD for some kits. It should be noticed that false-negative results were more frequently obtained for weakly positive samples (samples 0923, 1013,1114, and 1115 for IgM, and samples 1121 and 1124 for IgG). For these weakly positive samples, Tables 1 and 2 also illustrated the coincidence rates of the participating labs using the same tests used to produce the EQA panel were not good compared with other positive samples'. The lowest coincidence rates for IgG testing was 21.4%(6/28) by Medson EiA kit for IgG sample 1121 and the lowest coincidence rates for IgM testing was 60.0%(6/10 by Trinity EiA kit for IgM sample 0923.

Discussion
In this paper, we described the results of a long-term EQA scheme to evaluate the various immunoassays for the detection of toxoplasma-specific IgG and IgM used by the participating laboratories. Conducting such a program is the best method to detect problems with regard to assay sensitivity and accuracy, and to maintain and improve the quality of routine toxoplasmosis diagnosis in clinical laboratories.
The primary goal of EQA is to assess the participants' diagnostic assay performance. A UK National EQA scheme for toxoplasma-specific IgG was established in 1993, followed by a scheme for toxoplasma-specific IgM in 1998 [7]. These EQA schemes revealed that the performance of participating laboratories was consistently high, with a mean accuracy of 98% for IgG and of 95% for IgM. The Chinese National EQA scheme for toxoplasma-specific IgM was established in 2003, followed by a scheme for toxoplasma-specific IgG in 2008. If the strongly deviating results for panel 071 for IgM detection in the current study are not taken into account, the overall EQA score for toxoplasma-specific IgM and IgG detection in Chinese laboratories was consistently high, with a mean accuracy of 95.48% and 98%, respectively, similar to the results obtained in the UK study. Over the 10-year period during which this EQA was conducted, participation in the scheme increased, reflecting the increasing awareness of the importance of accurate diagnostic tests for an early detection of toxoplasma infection in order to appropriately treat infected individuals.
Although the overall high EQA scores in the current EQA program demonstrated an encouraging improvement in the accuracy of toxoplasma-specific IgG and IgM assays, we found a large variability in the assay results of the participating laboratories. For example, some of the participants reported false-negative results for some samples, while other participating laboratories correctly identified the same samples, using the same assay kit (Figs 2 and 3). One reason for the observed large variability could be measurement errors made by the participants during assay detection or in the selection of the cut-off value. The comparative test scores provide low-performance laboratories with the opportunity to examine their weaknesses and improve their methodologies. Participation in an interlaboratory comparison programme, such as EQA or PT(proficiency testing), is significant for clinical laboratories according to ISO 15189(Medical laboratories-Requirements for quality and competence) [8]. It allows for participating laboratories assess whether their testing results are comparable with other clinical laboratories testing results, and the participating laboratories should monitor the results of EQA and implement correctives action to improve its performance levels when their results are discrepant with expected results. Previous study domenstrated that assay interassay CVs for positive control were from 2.7% to 22.2% for six commercial kits for detection of IgM antibodies to toxoplasma gondii testing, while intra-assay precision varied from 4.9 to 13.4% for the manual assays [9]. Therefore, another reason might be the variability among kit lots from the same manufacturer. Our present study illustrated that the variability might exist in IgG and IgM testing. A clinical laboratory must institute its own verification procedures for each kit lot to ensure a continuing supply of comparable lots of reagents.
There is wide variety of assays available for the detection of toxoplasma-specific IgM and IgG. Some analyses based on commercial assay are necessary, because it is a key factor in clinical determination. Tough other techniques, such as Sabin-Feldman dye test(DT), latex agglutination test (LAT)for toxoplasma specific IgG detection, immunosorbent agglutination assay (ISAGA) for toxoplasma specific IgM detection, are mostly reserved for reference laboratories, while ELISA is mostly used in clinical laboratories for routine screening for IgG and IgM. ELISA and CIA are available as commercial kits and on automated platforms [10]. ELISA and CIA were the most commonly used methods in the laboratories that participated in this EQA scheme. Therefore, the panels with serum specimens containing toxoplasma IgG and IgM antibodies used in the EQA were tested by NCCL prior to dispatch, using various ELISA or CIA kits. Few participating laboratories used other techniques such as ECLA, ECA, ECL, DIGFA, and WB. However, the overall coincidence rates for IgM-and IgG-positive samples detected by these methods were lower than for ELISA and CIA. Especially DIGFA misidentified most positive samples. This demonstrates the different sensitivities of various methods and the importance of choosing an accurate and sensitive assay for clinical diagnosis.
Most commercial serological kits use native antigens prepared from tachyzoites grown in mice and/or tissue culture. The preparation methods for these antigens vary in different companies and the produced antigens may contain contaminants from culture media and eukaryotic host cells [10]. The major drawback is the poor standardization due to variation in antigen quality and subsequent variability in test results, which often leads to misinterpretation of the results [11]. Previous study [12] evaluated four commercial immunoassay systems and found that overall agreement rates among the four immunoassay systems were 91.7% for toxoplasma IgG and 89.8% for toxoplasma IgM. Especially the predictive value of IgM positivity strongly varies from one assay to another [13]. Therefore, we made a detailed comparison of S/CO values for different commercial ELISA kits. We found a large inter-assay variation for different ELISA kits for IgG and IgM detection (shown in Fig 1). This variation might be explained by the use of different antigens in various kits, with different antibody specificities. This complicates standardization of commercial kits. The lack of quality control and low specificity and/or sensitivity of many commercial kits form the main problems in serologic diagnosis of toxoplasmosis [3].
A detailed comparison of different commercial ELISA kits in the current study showed that some kits did not sufficiently allow for detecting weakly positive samples (Figs 2 and 3). The assay performance for panel 071 for IgM detection was rather low (42.5% failed to detect two or more positive samples and 12.2% missed one or two positive samples), and 88 participants (37.6%) failed to correctly identify sample 0923 in panel 092. Although these low scores never occurred again later in the EQA scheme, these results are noteworthy. The lower titers of toxoplasma antibodies in these samples could explain the poor assay results. The assay performance results for this sample in Fig 3 also illustrated this problem for IgG testing. In clinics, a falsenegative result can lead to misinformation of the patients, which can be especially fatal for fetuses when pregnant women are infected. Therefore, the manufacturers should thus pay attention to the sensitivity problem of the tests.
Most ELISAs can detect IgM for months or years after infection, so IgM detection is not an accurate marker of recent infection [14]. The differences in IgM in acute cases and chronic cases are lower levels of the antibody in the latter and different molecular weights of the antibodies. The low molecular weight, 7s IgM is more common in persistent infection while the 19s IgM is more common in acute infection; and may help in differentiating acute from persistent infection [15]. Because of the high titer of most IgM positive samples, the sera came from acute cases. The detection principles of EIAs or CIAs for IgM are capture method or indirect method aiming at the detection of IgM, therefore, these reagents could not distinguish 7s IgM or 19s IgM and these reagents could not recognize chronic IgM or acute IgM. When confirming an acute IgM result, additional tests such as the determination of IgG avidity are currently used in clinical laboratories [2,16]. However, specimens for IgG avidity testing were not included in the present EQA scheme and should be also included in future EQA schemes in order to discriminate between recent and past infection.
In conclusion, the long-term national EQA toxoplasma serology scheme in China has been beneficial to the participating clinical laboratories; the IgG and IgM assay performance improved and the sensitivity of routine tests as well as the accuracy of assay data analysis increased during the past years, thanks to the educational approach of the EQA scheme. This shows that a continuous monitoring of laboratories by an EQA can improve national performance, underlining the importance of such EQA.
Supporting Information S1 Table. Assays Currently Used by less than 5 Participants in the External Quality Assessment. (DOCX)