Meta-Analysis: Diagnostic Accuracy of Anti-Carbamylated Protein Antibody for Rheumatoid Arthritis

Objective The anti-carbamylated protein (CarP) antibody is a novel biomarker that might help in the diagnosis of rheumatoid arthritis (RA). We aim to assess the diagnostic value of anti-CarP antibody for RA. Methods We systematically searched PubMed, Embase, the Cochrane Library, Web of Science, and Scopus for studies published by December 15, 2015. Studies in any language that evaluated the utility of the anti-CarP antibody in the diagnosis of RA in which healthy donors or patients without arthritis or arthralgia served as controls were included. Two investigators independently evaluated studies for inclusion, assessed study quality and abstracted data. A bivariate mixed-effects model was used to summarize the diagnostic indexes from 7 eligible studies. Results The pooled sensitivity, specificity, and positive and negative likelihood ratios for anti-CarP antibody were 42% (95% CI, 38% to 45%), 96% (95% CI, 95% to 97%), 10.2 (95% CI, 7.5 to 13.9), and 0.61 (95% CI, 0.57 to 0.65), respectively. The summary diagnostic odds ratio was 17 (95% CI, 12 to 24), and the area under summary receiver operator characteristic curve was 80% (95% CI, 77% to 84%). Conclusion Anti-CarP antibody has a moderate value in the diagnosis of RA with high specificity but relatively low sensitivity.


Introduction
Rheumatoid arthritis (RA) is a common systemic autoimmune disease, characterized by persistent synovitis, systemic inflammation, and the presence of autoantibodies, particularly anticyclic citrullinated peptide (CCP) antibody and rheumatoid factor (RF). RA affects approximately 1% of the population globally [1], and 0.5-1.0% of the adult population in developed countries [2]. The disorder is more prevalent among women older than 65 years. For the development of RA, 50% of the risk is attributable to genetic factors, and the main environmental risk factor is smoking [2].
Irreversible damage to the joints is observed in RA; however early prevention is possible through early diagnosis and treatment. Currently, the anti-CCP antibody and RF are a part of the 2010 American College of Rheumatology (ACR)/The European League Against Rheumatism (EULAR) classification criteria for RA [3,4]. Despite the diagnostic contribution of the anti-CCP antibody and RF, approximately one-third of patients with RA remain seronegative [5]. Novel serological biomarkers are strongly needed to further improve the diagnosis of seronegative RA.
Anti-carbamylated protein (CarP) antibody, a novel autoantibody, has been detected in RA patients and predicts the development of the pathogenesis of RA, independent of the anti-CCP antibody [6,7]. This antibody recognizes proteins post-translationally modified by a process of carbamylation, rather than citrullination [8]. Carbamylation occurs when cyanate binds to primary amino or thiol groups presented in the body in equilibrium with urea [8,9]. The anti-CarP antibody has been described in RA, especially in anti-CCP-negative patients [10,11]. However, there are controversies regarding the diagnostic accuracy of the anti-CarP antibody in the literature. In this meta-analysis, published data on the sensitivity, specificity, and likelihood ratios of the anti-CarP antibody were summarized for the diagnosis of RA.

Data sources and searches
Without language restrictions, we searched PubMed, Embase, the Cochrane Library, Web of science, and Scopus for studies published by December 15, 2015 that detected the anti-CarP antibody. Our search combined the following index terms: autoantibody to carbamylated protein, autoantibody to CarP, anti-carbamylated protein antibody, anti-CarP antibody, rheumatoid arthritis, RA. The details of the search strategy are listed in the S1 File. We also searched the reference lists of retrieved studies and review articles for additional studies. Our meta-analysis was performed based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline (S2 File).

Data extraction and study quality assessment
Two investigators independently extracted data by using a standard form that included essential information on the eligible studies, including case number, type of article, trial design, places from which patient groups came, region where the studies were performed, plate and antibody brands of the ELISA, diagnostic criterion, testing method, the demographic characteristics of the participants, control participants, the cut-off of the testing method, and diagnostic indexes. When information from the identified studies was missing, we contacted the authors via email for detailed information. We assessed the study quality according to the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) [14]. Discrepancies were resolved through discussion or via consulting professionals.

Data analysis
We used a bivariate mixed-effects model to combine estimates of sensitivity, specificity, positive and negative likelihood ratios (LR), as well as diagnostic odds ratio (DOR). If heterogeneity existed (P0.05 or I 2 50%), the heterogeneity test was performed. We constructed a summary receiver operator characteristic (SROC) curve and calculated the area under the SROC curve (AUC) to evaluate the overall performance of the anti-CarP antibody in patients with RA. A sensitivity analysis was performed to evaluate stability by sequential omission of individual studies. Peter's test was also examined to explore publication bias. For analyses, we used STATA 12.0 software (Stata Corporation, 93 College Station, TX, USA).

Search results and characteristics of studies
We identified 150 published studies, of which 7 studies met the inclusion criteria [11,[15][16][17][18][19][20].  Table 1 and Table 2 summarize the characteristics of the included studies. Studies were published between 2011 and 2015, including 5 articles, 1 meeting abstract and 1 letter to the editor, as well as 6 documents from Europe and 1 research paper from the USA. We contacted the first author of the meeting abstract [15] and obtained useful information regarding their experiment. After gathering the information from the first author, we assessed the quality of the meeting abstract according to QUADAS-2. The result of the evaluation of the abstract's quality indicated that the meeting abstract was of high quality, and it was incorporated in the meta-analysis. The letter to the editor also contained enough information to be included. The anti-CarP antibody was identified by ELISA using fetal calf serum (FCS), while binding was determined using IgG. Regarding the characteristics of the control groups, studies used healthy persons as healthy controls and used patients with periodontitis, bronchiectasis, and cystic fibrosis as disease controls. The rationale for the studies included in qualitative synthesis but finally excluded is as follows: (1) three studies [10,20,21] did not provide sufficient data to allow the construction of a 2×2 table and (2) one study [22] lacked healthy controls, and only used patients with anti-citrullinated peptide antibody (ACPA)-positive arthralgia and inflammatory arthritis as disease controls. We think that if only patients with arthralgia or arthritis as disease controls were used to construct a 2×2 table for diagnostic accuracy of RA, the results may be biased. Shi et al. [6] reported that of the 340 patients with arthralgia, 133 patients (39%) presented with the anti-CarP antibody. There were 120 (35%) of 340 patients with arthralgia who met the 2010 ACR/EULAR criteria after a median of 12 months. In addition, Shi et al. [23] reported that 26% of the 2086 patients with arthritis were positive for the anti-CarP antibody. A total of 969 (47%) of all of the patients with arthritis met the 2010 ACR/EULAR criteria for RA. Similarly, Humphreys et al. [24] also reported that of 1995 patients with inflammatory polyarthritis, 1221 patients (61%) fulfilled the 2010 ACR/EULAR criteria. The Anti-CarP antibody was present in 460 (23%) of 1995 patients. Therefore, we know that patients with arthralgia or arthritis may subsequently develop RA, and the anti-CarP antibody is detected at a higher rate in these patients. If only patients with arthralgia or arthritis are used as controls, the specificity of the anti-CarP antibody for RA may be inaccurate. Therefore, the study [22] was excluded.

Study quality
The graphical display of the evaluation of the risk of bias and concerns regarding applicability of the selected studies, according to QUADAS-2, are reported in Fig 2. Concerning the domain of selection bias, three studies [15,16,18] did not explicitly report whether a consecutive or random sample of patients was enrolled. One study [11] did not explicitly report whether inappropriate exclusions were avoided. Four studies [11,[15][16][17] showed that the conduct or interpretation of the index test could introduce high bias, and three studies [18][19][20] introduced unclear bias. Regarding the domain of flow and timing, one study [15] did not explicitly report whether there was an appropriate interval between the index test and reference standard, while another study [18] did not explicitly indicate the reference standard used. One study [11] did not explicitly report whether all patients were included in the analyses. There were minimal concerns about patient selection and index test applicability, but unclear concern about the applicability of the reference standard in one study [18] as well as high concern in the remaining six studies. Overall, none of the 7 included studies indicated large methodological flaws, which would warrant their exclusion from the meta-analysis.

Exploration of heterogeneity and publication bias
We used a bivariate mixed-effects model in this meta-analysis (P = 0.471, I 2 = 0). No significant heterogeneity was found among the included studies. A sensitivity analysis indicated that the results of this meta-analysis were stable (Fig 6). Conducting Peter's test did not reveal any small-study effects or publication bias in this study (P = 0.437).

Discussion
RA can cause functional disability and reduce the quality of life. Early and accurate treatment interventions play an important role in preventing the development of massive erosions and deformities in the inflamed joints of RA patients. Specific laboratory tests are desirable to help in the early identification of RA. Robust biomarkers are also recommended for early diagnosis of RA to increase the benefits from aggressive interventions. The anti-CarP antibody was one of the newly discovered autoantibodies in the sera of patients with RA. Research by Shi J et al.
shows that anti-CarP and anti-CCP antibodies are different, but some cross-reactivity may exist [25]. Anti-CarP-FCS may have less cross-reactivity to citrullinated proteins than anti- carbamylated-fibrinogen (Fib) [26]. Thus far, several studies have reported that the anti-CarP antibody may provide additional benefit in the diagnosis of RA, especially for those with early and anti-CCP-negative diseases [5, 9-11, 27, 28]. The anti-CarP antibody are also detected juvenile idiopathic arthritis [10,22,29], psoriatic arthritis [30], and systemic sclerosis [31]. The association between the anti-CarP antibody and radiographic progression is strong in the total RA population as well as in the ACPA-negative subgroup [32]. The anti-CarP antibody may be used to predict radiographic progression within the RA population. Thus, the detection of the anti-CarP antibody may be a useful serological test for the identification and sub-classification of patients with RA. This meta-analysis is the first study to evaluate the diagnostic value of the anti-CarP antibody for RA. We included 7 studies in this meta-analysis. The potential diagnostic accuracy of the anti-CarP antibody was mainly due to its high specificity (96%) and positive LR (10.2), suggesting that patients with RA had a 10.2-fold higher chance of having a positive anti-CarP antibody test compared to controls. The sensitivity was lower (42%), while the negative LR was not low enough (negative LR<0.1) to exclude RA when the anti-CarP antibody tests were negative. The DOR (17) indicated that serum levels of the anti-CarP antibody could be helpful in diagnosing RA. Similarly, the AUC (80%) demonstrated that the anti-CarP antibody had a moderate diagnostic value for RA. Furthermore, 5 larger sample studies (cases100) [11,[15][16][17]20] were separately analyzed to evaluate the diagnostic value of anti-CarP antibody for RA. The pooled sensitivity, specificity, positive LR and negative LR were 41% (95% CI, 38% to 45%), 96% (95% CI, 95% to 97%), 11.4 (95% CI, 7.8 to 16.5) and 0.61 (95% CI, 0.57 to 0.65), respectively. The summary DOR was 19 (95% CI, 12 to 28), and the AUC was 84% (95% CI, 81% to 87%). The results of the larger sample studies are identical to the overall results.
In clinical applications, there are limitations of the 2010 ACR/EULAR classification criteria of RA with high sensitivity, such as failing to capture cases of symmetrical seronegative arthritis and limited joint involvement [11,20,33]. Anti-carbamylated-FCS IgG and IgA antibodies are present in both anti-CCP negative (IgG: 16%, IgA: 30%) and anti-CCP positive (IgG: 73%, IgA: 51%) RA patients [9]. The first presentation of the anti-CarP antibody was comparable with the anti-CCP antibody and presented earlier than IgM RF [27]. Therefore, the anti-CarP antibody might be a potential biomarker to identify anti-CCP-negative patients, who may benefit from early and aggressive interventions [34]. In addition, the simultaneous assessment of anti-CarP, anti-CCP, and RF might be beneficial in identifying RA.
Furthermore, other novel biomarkers found in patients with RA have been studied for their diagnostic value of RA. We summarized diagnostic performance of various antibody assays of RA in Table 3.
We excluded other RA-associated autoantibodies that had high specificity, such as anti-perinuclear factor and antikeratin antibodies [44,45], because rigorous technical requirements are needed for their detection. Compared with the antibodies in Table 3, the specificity of the anti-CarP antibody was high; however, the sensitivity was lower than that of the anti-CCP antibody and RF. It is possible that the assays used to detect these antibodies may differ in definition based on cut-off and diagnosis indexes, which may potentially limit their comparability. However, because the anti-CCP antibody has a high clinical utility, it is unlikely that the anti-CCP antibody will be replaced by new markers discovered in recent years.
Our position on the anti-CarP antibody is listed as follows: (1) The anti-CarP antibody is a novel biomarker that was recently discovered and, needs further studies to define its clinical utility; (2) The anti-CarP antibody is present before symptoms develop [5,46,47], suggesting that more research is necessary to determine its prognostic value; (3) The role of anti-CarP antibody in the immunological mechanisms of RA pathogenesis needs further study. So far, there has been research on this issue [48][49][50][51]; (4) According to the 2010 ACR/EULAR criteria, the weights for anti-CCP antibody and RF varied due to the titers. In addition, it needs to be determined whether a relationship exists between anti-CarP antibody titers and its diagnostic accuracy; and (5) Thus far, there has been no report in the literature of analysis of serum level of anti-CarP antibody between RA patients and relevant disease patients, who would be seen in rheumatology clinics. Our meta-analysis focused on healthy donors as controls. It is necessary to conduct research on the positive rate of the anti-CarP antibody in sera of patients with different diseases, which will allow a greater understanding of the diagnostic value of this antibody.
Our meta-analysis has some limitations. First, some articles published in other databases may have been missed. Second, certain concerns were raised during our assessment of the quality of the studies. For example, some included studies did not report whether the detection results of the anti-CarP antibody were performed blindly, which may cause measurement bias. However, these concerns did not influence our results as we found no heterogeneity or publication bias. The current studies were case-control studies. Well-designed prospective studies with larger sample sizes are needed to further confirm the value of the anti-CarP antibody for RA. In addition, in the present studies, the population was predominantly Caucasian, while only one study involved an Asian population. There were no Africans in the populations studied.
In conclusion, the anti-CarP antibody has a moderate diagnostic value, with high specificity but relatively low sensitivity in the diagnosis of RA.
Supporting Information S1 File. The search strategy using the five databases.