Diagnostic performance of congestion score index evaluated from chest radiography for acute heart failure in the emergency department: A retrospective analysis from the PARADISE cohort

Background Congestion score index (CSI), a semiquantitative evaluation of congestion on chest radiography (CXR), is associated with outcome in patients with heart failure (HF). However, its diagnostic value in patients admitted for acute dyspnea has yet to be evaluated. Methods and findings The diagnostic value of CSI for acute HF (AHF; adjudicated from patients’ discharge files) was studied in the Pathway of dyspneic patients in Emergency (PARADISE) cohort, including patients aged 18 years or older admitted for acute dyspnea in the emergency department (ED) of the Nancy University Hospital (France) between January 1, 2015 and December 31, 2015. CSI (ranging from 0 to 3) was evaluated using a semiquantitative method on CXR in consecutive patients admitted for acute dyspnea in the ED. Results were validated in independent cohorts (N = 224). Of 1,333 patients, mean (standard deviation [SD]) age was 72.0 (18.5) years, 686 (51.5%) were men, and mean (SD) CSI was 1.42 (0.79). Patients with higher CSI had more cardiovascular comorbidities, more severe congestion, higher b-type natriuretic peptide (BNP), poorer renal function, and more respiratory acidosis. AHF was diagnosed in 289 (21.7%) patients. CSI was significantly associated with AHF diagnosis (adjusted odds ratio [OR] for 0.1 unit CSI increase 1.19, 95% CI 1.16–1.22, p < 0.001) after adjustment for clinical-based diagnostic score including age, comorbidity burden, dyspnea, and clinical congestion. The diagnostic accuracy of CSI for AHF was >0.80, whether alone (area under the receiver operating characteristic curve [AUROC] 0.84, 95% CI 0.82–0.86) or in addition to the clinical model (AUROC 0.87, 95% CI 0.85–0.90). CSI improved diagnostic accuracy on top of clinical variables (net reclassification improvement [NRI] = 94.9%) and clinical variables plus BNP (NRI = 55.0%). Similar diagnostic accuracy was observed in the validation cohorts (AUROC 0.75, 95% CI 0.68–0.82). The key limitation of our derivation cohort was its single-center and retrospective nature, which was counterbalanced by the validation in the independent cohorts. Conclusions In this study, we observed that a systematic semiquantified assessment of radiographic pulmonary congestion showed high diagnostic value for AHF in dyspneic patients. Better use of CXR may provide an inexpensive, widely, and readily available method for AHF triage in the ED.

a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 CI 0.82-0.86) or in addition to the clinical model (AUROC 0.87, 95% CI 0.85-0.90). CSI improved diagnostic accuracy on top of clinical variables (net reclassification improvement [NRI] = 94.9%) and clinical variables plus BNP (NRI = 55.0%). Similar diagnostic accuracy was observed in the validation cohorts (AUROC 0.75, 95% CI 0.68-0.82). The key limitation of our derivation cohort was its single-center and retrospective nature, which was counterbalanced by the validation in the independent cohorts.

Conclusions
In this study, we observed that a systematic semiquantified assessment of radiographic pulmonary congestion showed high diagnostic value for AHF in dyspneic patients. Better use of CXR may provide an inexpensive, widely, and readily available method for AHF triage in the ED.

Author summary
Why was this study done?
• Chest radiography (CXR) is often performed in patients admitted for acute dyspnea in the emergency department (ED); however, its assessment is generally not standardized.
• Latest guidelines for heart failure (HF) emphasize the limitations of CXR in its current use.
• A standardized approach to quantify pulmonary congestion from CXR could potentially improve its diagnostic performance for acute HF (AHF).

What did the researchers do and find?
• We studied the diagnostic value of a semiquantitative approach (the congestion score index [CSI]) to pulmonary congestion on CXR for AHF in a retrospective cohort study of 1,333 patients with acute dyspnea admitted to the ED.
• This CSI was significantly associated with AHF diagnosis.
• The CSI also improved diagnostic accuracy over clinical parameters with or without inclusion of natriuretic peptide.
• This good diagnostic accuracy of the CSI was externally validated in independent cohorts (N = 224).
What do these findings mean?
• The quantification of radiographic pulmonary congestion using the CSI improved diagnostic accuracy for AHF on top of clinical parameters and natriuretic peptide. not authorized by the CNIL (Commission Nationale de l'Informatique et des Libertés) to perform the analysis. To comply with this national data regulation, we will provide access to the data on a secured server held by our institution upon reasonable request to the primary investigator of the study. Importantly, Nancy CIC-P works with a number of international groups using this secured server. Requests can be made at the following address: cic@chru-nancy.fr.

Introduction
Acute heart failure (AHF) is 1 of the leading causes of acute dyspnea in the emergency department (ED) [1] and is associated with a higher risk of morbidity and mortality [2,3]. In-hospital mortality is reported to be greater than 10% [4] and has remained stable in the last 30 years. As prognosis is associated with initiation time of specific therapies [5], current guidelines emphasize the importance of early diagnosis and treatment initiation to improve clinical outcomes [5,6]. However, a minority of patients with AHF receive treatment within 1 hour of admission [5], in contradiction with current recommendations [6]. In addition, a third of AHF diagnosis are missed in the ED [6], further delaying access to care. An increasing number of better diagnostic tools for AHF are available in the ED [7][8][9]. However, there is likely room for improving the diagnostic approach to AHF from widely available routine tools including chest radiography (CXR). CXR is a fast and inexpensive method performed systematically in the ED in patients with acute dyspnea [10][11][12]. It is the first-line diagnostic imaging modality advocated in current guidelines [13]. However, its diagnostic accuracy for HF has been reported to be relatively low [14][15][16]. In particular, diagnosing HF in patients with concomitant lung diseases such as chronic obstructive pulmonary disease (COPD) and pneumonia still remains a challenge [17].
A new semiquantitative approach to pulmonary congestion has recently emerged in the field of HF. Congestion score index (CSI) is a semiquantitative approach to pulmonary congestion based on a 6-zone evaluation of CXR, scoring each zone from 0 (no congestion) to 3 (intense alveolar pulmonary edema). CSI is a strong risk stratifier in patients with stable or worsening HF [18][19][20]. However, there are little available data regarding its diagnostic value for AHF.
The aims of the present study are to investigate the diagnostic value of pulmonary congestion assessed with CSI for AHF in patients admitted for acute dyspnea in the ED and to assess its discriminative value comparatively to and on top of currently used clinical diagnostic models and natriuretic peptide measurements.

Study population
This study is reported as per the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guideline (S1 Checklist). The Pathway of dyspneic patients in Emergency (PARADISE) cohort is a retrospective cohort study including consecutive patients aged 18 years or older who were admitted for acute dyspnea in the ED of the Nancy University Hospital (France) between January 1, 2015 and December 31, 2015 as detailed previously [21,22]. The hospital's electronic charts (resurgences) were systematically reviewed by investigators to search for the records of all patients admitted for acute dyspnea in the ED. All patients with signs or symptoms of dyspnea or requiring oxygen therapy for dyspnea during (or prior to) their ED stay were included. Patients with shock or cardiac arrest were excluded from this analysis as dyspnea was not the primary condition triggering ED admission. The PARADISE cohort is consequently a cohort of unselected consecutive patients with acute dyspnea in the ED. In the current study, a total of 1,333 dyspneic patients with available information on CXR at the ED were analyzed (S1 Fig). Demographic parameters, medical history, physical examination, laboratory findings, and treatment received in the ED were retrieved from the patients' electronic records.
External validation was performed on the merged dataset from the HF disease management program entitled "Insuffisance CArdiaque en LORraine (ICALOR)" [23,24] (included at a different time period that the data from the Nancy University Hospital used in the derivation set) and data from the Epinal Hospital (a secondary care hospital located 70 km from the Nancy University Hospital) within the Pathway and Urgent caRe of Dyspneic Patient at the Emergency Department in LorrainE District (PURPLE) multicenter cohort (NCT03194243). Briefly, we used the data from the previously described [19] 117 patients included in the ICA-LOR disease management cohort during a hospitalization for acutely decompensated HF in the Cardiology Department of the Nancy University Hospital (France). We also used the data of 107 consecutive patients admitted for acute dyspnea with available CXR data and discharge diagnosis in the ED of the Epinal Hospital. This external validation set allowed us to test CSI in a significantly different setting (mixing cardiology department from a tertiary care hospital and an ED of a secondary care hospital) than our derivation set.
Under French law, no formal Institutional Review Board approval is required for data extraction from patient medical records in single-center cohorts (the PARADISE and ICALOR cohorts). For the PURPLE cohort, patients were informed through a notice at admission and could refuse their inclusion in the study, although no formal consent was required in keeping with the framework of the Commission Nationale de l'Informatique et des Libertés (CNIL). The PARADISE cohort was recorded by the local hospital corresponding agent of the CNIL (Number R2016-08) and was registered on clinicaltrials.gov (NCT02800122). The PURPLE cohort was approved by an ethical board ("Comité de Protection des personnes"-Number 2016-63, ID RCB 2016-A01877-44) and CNIL (Number DR-2017-098) and was registered on clinicaltrials.gov (NCT03194243).

Diagnosis of heart failure
HF was diagnosed according to the European Society of Cardiology (ESC) guidelines [25]. Diagnosis of AHF was coded independently by 2 medical physicians (GG and TH) according to the ESC guidelines [25]. Each physician had access to all ED medical charts as well as additional hospital admission test results and records (e.g., echocardiography, natriuretic peptide level, and patient response to diuretic/bronchodilator therapy) but were blinded to CSI quantification on CXR. Homogenous coding was ensured by a trained senior physician (TC). Importantly, to determine the discharge diagnosis, we focused on the main cause of acute dyspnea rather than background medical history or coexisting conditions.

Radiographic congestion score index
Radiographic CSI was used to quantify the severity of pulmonary congestion in CXR as previously published [18,19]. After dividing the lung field into 6 topographical zones, each area was assessed as follows: Score 0, no congestion sign; Score 1, cephalization (superior area), perihilar haze or perivascular/peribronchial cuffing, or Kerley A lines (middle area), Kerley B, or C lines (inferior area); Score 2, interstitial or localized/mild alveolar pulmonary edema; Score 3, intense alveolar pulmonary edema (Fig 1). To enhance the reproducibility of the severity of confluent edema, a portion of the divided lung fields which was visually similar to the cardiac silhouette was regarded as an intense zone, whereas the field with weaker density was regarded as a mildly intense zone. Lung areas were not scored when more than one-third of the divided lung fields were occupied by pleural effusion (including vanishing tumor), atelectasis, or cardiac silhouette. CSI was calculated as the sum of the scores in each zone divided by the number of available zones. An examiner also assessed the presence of pneumonia, pleural effusion, cardiomegaly by cardiothoracic ratio (>50%), and the difficulty in assessing CSI.
CXR was analyzed by a single emergency physician (AD), blinded to clinical data and discharge diagnosis, with no previous training in congestion quantification on CXR prior to that in the present study. After a short training (approximately 3 hours) using a 20-patient sample with a radiographic CSI expert (MK), intraobserver and interobserver agreements (with MK) were tested on 30 randomly selected patients, while blinded to clinical status and diagnosis. Intraclass correlation coefficients showed good reproducibility (intraobserver agreement, 0.85, 95% CI 0.71 to 0.93 and interobserver agreement 0.81, 95% CI 0.64 to 0.90).

Brest score
The Brest score was calculated for every patient based on the patients' medical charts. This diagnostic score for AHF in dyspneic patients is based on age, comorbidities (i.e., prior history of HF, myocardial infarction, and COPD), pattern of dyspnea, ST segment abnormalities, and signs/symptoms of congestion (i.e., rales and leg edema) [26,27].

Statistical analysis
Categorical variables are expressed as frequencies (percentages) and continuous variables as means ± standard deviation (SD) or median (25th and 75th percentiles) according to the distribution of the variables. Comparisons of demographic, clinical, and biological parameters among quartiles of radiographic CSI were analyzed using χ 2 tests for categorical variables and ANOVA or Kruskal-Wallis test for continuous variables. Interobserver and intraobserver agreements of CSI were assessed with the intraclass correlation coefficient.
We analyzed 1,333 dyspneic patients with 289 AHF diagnosis-which provided a sizeable statistical power to assess diagnostic performance in a multivariable setting [28]. A logistic regression model was used to assess the association of CSI with diagnosis of AHF. Multivariable analyses included relevant confounders as previously shown [21]: model 1: age, sex, body mass index (BMI), presence of hypertension, diabetes mellitus, coronary artery disease, atrial fibrillation, prior HF admission, use of angiotensin converting enzyme inhibitor/angiotensin receptor blocker, beta-blocker, diuretics, leg edema, jugular venous distension, hemoglobin, white blood cell count, and estimated glomerular filtration rate (eGFR, as calculated by the Chronic Kidney Disease Epidemiology Collaboration formula [29]) at admission; model 2: Brest score. Receiver operating characteristics (ROC) curve was used to determine the diagnostic value of CSI in AHF. All correlation coefficients of variables included in the models were less than 0.50 with CSI, suggesting the absence of important in-model collinearity.
The increase in discriminative value of the addition of CSI for AHF diagnosis on top of the aforementioned potential covariates was assessed using continuous net reclassification improvement (NRI) [30]. In addition, in 498 (37.4%) patients who had available b-type natriuretic peptide (BNP) measurements, the added value of CSI on the top of the Brest score and BNP was assessed.
This analysis on the PARADISE cohort was planned in February 2019, although no formal analysis was written. The general analysis intention was to evaluate the diagnostic value of CSI for AHF. Of note, among all patients with available data, CXR was assessed, blinded for clinical data and diagnosis, and standard statistical approaches were used. Based on recommendations made during the peer-review process, we conducted restricted cubic spline regression analysis for the association between CSI and AHF diagnosis.
All analyses were performed using R version 3.4.0 (R Development Core Team, Vienna, Austria). A 2-sided p-value < 0.05 was considered statistically significant. No imputation was performed.

Baseline characteristics
Less than 10% of the population had no available data on CXR (S1 Table). These patients were markedly younger and had less comorbidities than patients who underwent CXR. The characteristics of the PARADISE cohort population across different discharge diagnoses such as AHF, COPD, and pneumonia are depicted in S2 Table. In a total of 1,333 patients included in this study, a half were men, mean age was 72.0 ± 18.5 years, mean BMI was 25.5 ± 5.5 kg/m 2 , and less than 10% had a prior admission for HF (7.1%) ( Table 1). CXR was considered as difficult to interpret during CXR reviewing in 502 patients (37.7%). Mean CSI was 1.42 ± 0.79. Patients with a higher CSI had more cardiovascular risk factors, comorbidities, more frequent prior HF admission, more severe congestion, inflammation status, higher BNP, poorer renal function, and more respiratory acidosis at admission (  49.35], p < 0.001-considering CSI score 0.5 as a reference) (Fig 2).

Diagnostic value of the congestion score index
In the whole population, CSI exhibited high discrimination for AHF as reflected by an area under the curve (AUC) of 0.84 (0.82 to 0.86) (Figs 3 and 4). Similarly, high AUC was observed across subgroups of age, sex, and comorbidities (obesity and COPD) or associated diagnosis (pneumonia and pleural effusion) (Fig 3)

Improvement in reclassification associated with acute heart failure diagnosis
The addition of CSI on top of the Brest score significantly improved reclassification of AHF diagnosis (NRI [95% CI] = 94.9 [83.5 to 106.2], p < 0.001) (Fig 4).

Validation in external cohorts
In the validation cohorts (N = 224), more than a half of the patients (56.7%) were men, mean age was 75.8 ± 13.9 years, mean CSI was 1.85 ± 0.87, and 72.7% had a diagnosis of AHF at discharge. The diagnostic performance of CSI was externally validated (AUC [95% CI] = 0.75 [0.68 to 0.82]).

Discussion
Our results show that pulmonary congestion quantified by a simple standardized CXR scoring can efficiently identify patients with AHF in the ED, consistently across the various subgroups (e.g., elderly, overweight, COPD, and pneumonia). Furthermore, CSI significantly improved the reclassification of AHF diagnosis on top of the recognized clinical diagnostic markers of  Values are mean ± SD, N (%), or median (25th and 75th percentiles). � p-value adjusted for age and BMI at admission.
ACEi, angiotensin converting enzyme inhibitor; ARB, angiotensin receptor blocker; BMI, body mass index; BP, blood pressure; BPM, beats per minute; BUN, blood urea nitrogen; BNP, b-type natriuretic peptide; CSI, congestion score index; eGFR, estimated glomerular filtration rate; HF, heart failure; PaCO2, partial pressure of carbon dioxide; PaO2, partial pressure of oxygen; SD, standard deviation. AHF and natriuretic peptides. Our findings suggest that a semiquantified assessment of congestion on CXR could represent a readily available and clinically useful diagnostic tool for AHF in acute dyspneic patients in the ED.

Radiographic congestion score index as a diagnostic tool for acute heart failure
CXR exhibited a higher diagnostic performance for AHF than usually reported in the literature [14][15][16]. Of note, previous reports assessed the diagnostic value of CXR using a single or combination of typical radiographic signs of congestion using a global evaluation of the lungs [15,16,[31][32][33] rather than a systematic approach, with lung segmentation as used in the CSI method. Our group recently showed that more severe pulmonary congestion, quantified by either CSI or lung ultrasound at admission, was associated with higher pulmonary artery systolic pressure [19,34]. This association of CSI with hemodynamic data emphasizes the mechanistic plausibility of our results. Recent registry data have shown that approximately 20% of patients hospitalized for AHF had concomitant lung diseases such as pneumonia and COPD [35][36][37], and these patients generally excluded in previous reports assessing the diagnostic value of CXR [16,17,38,39]. However, our subgroup analysis provided a remarkably homogenous diagnostic performance of CSI across various subgroups (i.e., elderly and BMI) and coexisting lung diseases (i.e., COPD and pneumonia). The only factor appearing to decrease the diagnostic value of CSI was cardiomegaly, possibly as a result of the overlapping of relevant information of the lung fields with cardiac silhouette in these patients.

Radiographic congestion score index and other diagnostic measurements: Clinical parameters and natriuretic peptide
In keeping with previous literature data [19,40], our results showed that patients with more severe pulmonary congestion were predominantly elderly, had higher BMI, more cardiovascular risk factors and comorbidities, severe congestion, poorer renal function, and higher inflammatory markers. In the latest guidelines, natriuretic peptide is recommended to rule out non-HF-related causes of acute dyspnea, although accumulated data suggested the overall diagnostic accuracy of natriuretic peptide [41][42][43][44][45]. However, multiple comorbidity burdens (i.e., older age and poor renal function) may lead to diagnostic uncertainty in a sizeable proportion of dyspneic patients [7]. BNP, in addition, requires time to measure and is not always available in routine clinical practice. In cases with unequivocal diagnosis of AHF based on clinical parameters, prompt treatment approach is recommended rather than wait for its On the receiving operating characteristic curve for the association between CSI and AHF diagnosis, the red shaded region represents the 90% or greater specificity zone (CSI � 1.3), whereas the green shaded region represents the 90% or greater sensitivity zone (CSI > 2.2). On the top portion of the figure, CXR panels illustrate typical examples of radiographies in the 3 zones, with CSI score of "1," "2," and "3," respectively, from left to right. AHF, acute heart failure; AUC, area under the curve; BNP, b-type natriuretic peptide; CSI, congestion score index; CXR, chest radiography; NRI, net reclassification improvement.
https://doi.org/10.1371/journal.pmed.1003419.g004 result [6]. In this regard, the Brest score, based on clinical parameters, may be a pragmatic tool as well as good diagnostic accuracy for diagnosing HF as previously shown [27]. However, in the current study, more than half of dyspneic patients have intermediate scores, suggesting that this clinical score may need to be complemented by more refined strategies in this relatively frequent "grey zone." The unmet need in clinical practice, even when using the Brest score and natriuretic peptides, thus remains high.
In the current study, CSI was found to improve reclassification of AHF diagnosis on top of the Brest score and BNP. In addition, the combination of CSI and the Brest score improved the diagnostic value (AUC 0.81) to a similar degree of the combination of BNP and the Brest score (AUC 0.82). CSI also significantly improved diagnostic accuracy on top of the Brest score and BNP, and the combined used of these 3 parameters resulted in an AUC of 0.85. Taken together, these findings further strengthen that CSI may play a complementary role to the clinical model and natriuretic peptides in diagnosing AHF.

Clinical implications
An early accurate diagnosis and consequently a prompt appropriate management improve outcomes in AHF patients admitted to the ED [5,46,47]. Our results show that a standardized evaluation of CXR, using CSI, improves diagnosis performance and potentially the ability to swiftly manage AHF in the ED. The assessment of radiographic pulmonary congestion requires training period. However, this approach may be easily scalable since training for this assessment is fairly simple; the operator (AD) who evaluated all CXRs of this cohort efficiently acquired the technique in the context of this study in a matter of a few hours.
Recent studies showed the clinical utility of lung ultrasound to diagnose AHF in dyspneic patients with high specificity and high sensitivity [9,26,48], and its diagnostic accuracy was better than that of CXR [8,50]. Of note, these previous studies did not include quantitative assessment of radiographic pulmonary congestion. In any event, based on the promising results herein, multicenter trials may be warranted to assess the impact of the implementation of CSI on AHF diagnosis in patients with acute dyspnea. Furthermore, further study to compare diagnostic value between CSI on CXR and lung ultrasound is a worthy undertaking.

Limitations and strengths
The main limitation of our derivation cohort was its single-center and retrospective nature, although the external validation of our results may lead to generalize our findings. The overall proportion of AHF diagnosis was relatively low, which may explain the fact that a low number of patients had congestion signs (i.e., leg edema, rales, and jugular venous distention) and cardiovascular diseases. In our center (as in most centers in France), some dyspneic patients known to have HF and all patients with obvious evidence of myocardial ischemia were admitted directly in the intensive cardiac care unit/cardiology ward, not through the ED [49]. The proportion of AHF diagnosis possibly due to the healthcare system may influence our results. However, it should be noted that hospitalization rates for worsening HF in the ED declined over the past decades [50], which may result from the development of a disease management program to prevent urgent HF hospitalization [51]. Indeed, the proportion of AHF diagnosis in the present study was similar to that of dyspneic patients in other contemporary cohorts [44,52]. Ejection fraction was not recorded; thus, we did not evaluate the diagnostic accuracy of CSI across levels of ejection fraction. This parameter, however, is usually not a major determinant of decision-making for acute dyspneic management in the ED [6,12].
In the derivation cohort, we had no data on CSI in 138 (9.4%) patients who did not undergo CXR or had no available lung field to assess CSI. These patients, however, had better clinical status (i.e., younger age and less severe congestion), and only 8% of these patients (N = 11) were diagnosed with AHF (S1 Table). In addition, all CSI readings were performed blinded for other parameters and diagnoses, suggesting that this limitation is unlikely to have major influence on our findings.
CSI is a semiquantitative tool with some subjectivity. Although accurate and reproducible scoring was achieved after about 3-hour training period in the current study and 1 of our previous study [19], more evidence may be needed to ascertain the appropriate learning period.
Lastly, the assessment of CSI was difficult in 502 (37.7%) patients, which may limit its applicability in routine clinical practice. However, the diagnostic value of CSI persisted in these patients (AUC = 0.80, 0.75 to 0.84) and the difficulty in assessing CSI did not influence its diagnostic accuracy (P interaction = 0.13).

Conclusions
Our study shows that a semiquantified assessment of radiographic pulmonary congestion provided diagnostic value for AHF in dyspneic patients of similar magnitude to that of BNP. These results suggest that implementing radiographic CSI in the diagnostic approach to AHF in addition to clinical parameters and BNP measurement could benefit the management of AHF patients. Better use of CXR may provide an inexpensive, widely, and readily available method for AHF triage in the ED. Multicenter prospective studies are nonetheless needed to confirm the diagnostic value of radiographic CSI.