Accuracy and precision of ScvO2 measured with the CeVOX-device: A prospective study in patients with a wide variation of ScvO2-values

Introduction Central-venous oxygen saturation (ScvO2) is a key parameter of hemodynamic monitoring and has been suggested as therapeutic goal for resuscitation. Several devices offer continuous monitoring features. The CeVOX-device (Pulsion Medical Systems) uses a fibre-optic probe inserted through a conventional central-venous catheter (CVC) to obtain continuous ScvO2. Objectives Since there is a lack of studies validating the CeVOX, we prospectively analyzed data from 24 patients with CeVOX-monitoring. To increase the yield of lower ScvO2-values, 12 patients were equipped with a femoral CVC. Methods During the 8h study period ScvO2_CeVOX was documented immediately before withdrawal of blood to measure ScvO2 by blood gas analysis (ScvO2_BGA) 6min, 1h, 4h, 5h and 8h after the initial calibration. No further calibrations were performed. Results In patients with jugular CVC (primary endpoint; 60 measurements), bias, lower and upper limits of agreement (LLOA; ULOA) and percentage error (PE) of the estimate of ScvO2 (ScvO2_CeVOX_jug) were acceptable with 0.45%, -13.0%, 13.9% and 16.6%, respectively. As supposed, ScvO2 was lower in the femoral compared to the jugular measurements (69.5±10.7 vs. 79.4±5.8%; p<0.001). While the bias (0.64%) was still acceptable, LLOA (-23.8%), ULOA (25.0%) and PE (34.5%) were substantially higher for femoral assessment of ScvO2 by the CeVOX (ScvO2_CeVOX_fem). Analysis of the entire data-pool with jugular as well as femoral CVCs allowed for a multivariate analysis which demonstrated that the position of the CVC per se was not independently associated with the bias ScvO2_CeVOX—ScvO2_BGA. The amount of the bias |ScvO2_CeVOX–ScvO2_BGA| was independently associated with the amount of the change of ScvO2_CeVOX compared to the initial calibration to ScvO2_BGA_baseline (|ScvO2_CeVOX—ScvO2_BGA_baseline|) as well as with low values of ScvO2_BGA_baseline. Furthermore, increasing time to the initial calibration was associated to the amount of the bias with borderline significance. A statistical model based on |ScvO2_CeVOX—ScvO2_BGA_baseline| and “time to last calibration” derived from an evaluation dataset (80 of 120 datasets, 16 of 24) provided a ROC-AUC of 0.903 to predict an amount of the bias |ScvO2_CeVOX–ScvO2_BGA| ≥5% in an independent validation group (40 datasets of 8 patients). Conclusion These findings suggest that the CeVOX device is capable to detect stability or instability of ScvO2_BGA. ScvO2_CeVOX accurately estimates ScvO2_BGA in case of stable values. However, intermittent measurement of ScvO2_BGA and re-calibration should be performed in case of substantial changes in ScvO2_CeVOX compared to baseline. Therefore, continuous measurement of ScvO2 with the CeVOX cannot replace ScvO2_BGA in instable patients. On the other hand, CeVOX might be useful for the monitoring of stable patients as a pre-test tool for more differentiated monitoring in case of changes in ScvO2_CeVOX.

Analysis of the entire data-pool with jugular as well as femoral CVCs allowed for a multivariate analysis which demonstrated that the position of the CVC per se was not PLOS

Introduction
In case of stable values for hemoglobin and the arterial oxygen saturation S a O 2 the oxygen saturation of blood returning to the right heart necessarily depends on cardiac output (CO) and oxygen consumption VO 2 . The oxygen saturation of blood drawn from a central venous catheter (CVC) or via a pulmonary arterial catheter (PAC) has been termed central-venous oxygen saturation S cv O 2 and mixed-venous oxygen saturation S mv O 2 , respectively. Both parameters reflect the relation of oxygen delivery and consumption. Normally, only about 25% of the delivered oxygen is withdrawn by the oxygen consuming tissues. Therefore, normal values for S cv O 2 and S mv O 2 are about 65-75%. In case of increased oxygen consumption and/or reduced delivery, S cv O 2 and S mv O 2 decrease. Consequently, decreasing values of S cv O 2 and S mv O 2 are used as warning signs indicating that mechanisms to compensate an impaired balance between oxygen consumption and delivery have been activated. While smaller decreases in S cv O 2 and S mv O 2 can be considered as physiological compensatory mechanism, more pronounced and prolonged decreases frequently precede anaerobic metabolism and hyperlactatemia. Based on this pathophysiological rationale, S cv O 2 and S mv O 2 have been suggested as therapeutic goals for resuscitation and as key targets of hemodynamic monitoring to avoid tissue hypoxia despite normal macro-circulatory parameters such as MAP and CVP [1,2,3,4,5]. Furthermore, S cv O 2 has been suggested as therapeutic goal for resuscitation and as a basic parameter of haemodynamic monitoring [3,6].
Several approaches have been established to facilitate these concepts: Since the use of a PAC has certain risks and it is costly and limited in time, S mv O 2 has largely been replaced by S cv O 2 which can easily been determined via a CVC [1,2,3].
Since repeated measurements are cumbersome and costly in long term critically ill patients, several devices continuously deriving ScvO 2 have been introduced. In addition to economic advantages, continuous monitoring offers the potential to increase the yield of pathological S cv O 2 measurements. Therefore, continuous monitoring is in particular attractive in stable patients at risk of sudden circulatory instability to provide a sensitive pre-test tool for more differentiated monitoring.
Several devices offer continuous monitoring features, including the CeVOX (Pulsion Medical Systems SE, Feldkirchen, Germany). Usually, continuous measurement is based on infrared oximetry which detects transmitted light of different wavelengths reflected by red blood cells varying with different concentrations of oxyhemoglobin and hemoglobin [1].
While some devices have integrated the infrared probe into special catheters, the use of the CeVOX is even further facilitated, since the probe can be introduced into a catheter already in place. Despite its use for more than a decade, there are only few studies available that prove validity and clinical usefulness [7,8,9,10,11]. Some validation studies suggest that accuracy and precision might depend on the absolute value of S cv O 2 with lower values resulting in imprecision compared to the gold-standard of blood gas analysis (BGA; Table 1).
Therefore, we performed a validation study in 24 patients equipped with the CeVOX device. To increase the yield of lower S cv O 2 -values, we included 12 patients equipped with a femoral CVC.

Materials and methods
The study was approved by the institutional review board (Ethikkommission Technische Universität München; Fakultät für Medizin; No. 5384/12). Written informed consent was obtained by all patients or their legal representatives. All patients were treated in a 14-bed university hospital general ICU with predominantly medical patients. Informed consent was obtained by all patients or their legal representatives. Between July and October 2016 we included 24 patients with hemodynamic monitoring comprising transpulmonary thermodilution (TPTD; PiCCO; Pulsion Medical Systems SE, Feldkirchen, Germany), central venous catheter and measurement of S cv O 2 irrespective of the study. For indicator injections for TPTD, blood withdrawal for blood gas analysis including S cv O 2 and insertion of the CeVOX-probe (PV2022-37; Pulsion Medical Systems SE, Feldkirchen, Germany) a 5-lumen CVC (Multicath 5, Vygon; Aachen, Germany) with a maximum intravascular length of 20 cm and a diameter of 3.15 mm (9.5 Fr) was used. The position of the tip was controlled (and corrected) according to X-ray in case of jugular, but not in case of femoral venous catheter access. The vascular part of the femoral venous catheter was completely inserted under ultrasound guidance. The CeVOX was inserted into the medial lumen of the CVC ending at the tip of the catheter according to the manufacturer´s recommendations with the aim to protrude the distal end of the catheter by 2 cm. For insertion a sterile Y-adapter was used which enables insertion of the probe through one lumen and withdrawal of blood for BGA through the other lumen of adapter. This provides withdrawal of blood at the most distal lumen of the CVC in close proximity to the fiber probe. For blood gas analysis a Siemens RapidPoint 500 (Siemens Healthcare, Erlangen, Germany) analyzer was used. Baseline BGA was performed to calibrate the CeVOX subunit of a PiCCO-2 or Pulsioflex monitor (Pulsion Medical Systems SE, Feldkirchen, Germany). No further calibrations of the CeVOX were performed during the study period. To investigate a potential impact of cardiac index (CI) on the accuracy of the estimate of S cv O 2 provided by the CeVOX (S cv O 2 _CeVOX) a triplicate TPTD was performed immediately before the baseline BGA. The registration of the arterial TPTD curve was performed as previously described [12,13]. During the 8h study period S cv O 2 _CeVOX was documented immediately before withdrawal of blood to measure S cv O 2 by BGA (S cv O 2 _BGA) 6min, 1h, 4h, 5h and 8h after the initial calibration.

Statistical analyses
Raw data were examined for input data error. Continuous variables were expressed as mean ±standard deviation. Categorical variables are expressed as percentages. Spearman´s coefficient of correlation was calculated to analyze the correlation of two parameters. To compare continuous variables we used Wilcoxon-test for paired samples.
Bland-Altman analysis was used to analyze the bias between S cv O 2 _CeVOX and S cv O 2 _BGA as well as to compute limits of agreement and percentage error. Bland-Altman analyses were corrected for repeated measurements allowing variability of true values within each subject [14].
With regard to clinical importance and comparability with previous studies Bland-Altmananalysis of the data derived from jugular BGA measurements was the primary endpoint of the study.
Prediction of the amount of the bias |S cv O 2 _CeVOX-S cv O 2 _BGA| was a major secondary endpoint. The amount of the bias is the absolute non-negative value of the bias without regard to its sign). To optimize the yield of our dataset, we used a three step approach: In a first step, we tried to assess the overall potential to derive a formula predicting inacceptable bias of S cv O 2 _CeVOX. Therefore, we performed multivariate analysis regarding the amount of the bias |S cv O 2 _CeVOX-S cv O 2 _BGA| in the total dataset (n = 120) including variables with a pvalue <0.2 in univariate analysis regarding this endpoint.
With regard to practical application only those variables were included in the regression analysis that would be available during continuous use of CeVOX after a single initial calibration.
In a second step, we randomly allocated the 24 patients in a 2:1 ratio to an evaluation group (n = 16 patients with 80 measurements) and to an independent validation group (n = 8 patients with 40 measurements). This was done to derive a prediction formula from two thirds of the datasets and to test its "robustness" in 40 "independent" measurements of the validation group that did not contribute to the derivation of the prediction formula (third step).
A similar approach has been described previously to derive and validate different models predicting inaccuracy of transpulmonary thermodilution with room temperature instead of ice-cold saline injectate [15].
Receiver operating characteristics (ROC)-analyses were performed to assess the discriminative ability of predictors regarding categorical endpoints.
Sample size was chosen according to the recommendation of Bland (https://www-user. york.ac.uk/~mb55/meas/sizemeth.htm). This publication suggests a number of n = 100 pairs in order to achieve an appropriate precision for the Bland-Altman analyses. Also accounting for potential drop-outs, incomplete datasets and pre-defined subgroup analyses we choose a number of n = 120.
All statistical analyses were performed using the IBM SPSS Statistics software version 23 (SPSS Inc., Chicago, IL, USA). Table 2 shows the patients baseline characteristics.

Patients characteristics
All patients were critically with a mean APACHE-II score of 21. Half of the patients suffered from ARDS, another 33% of sepsis. Consequently, 75% were under mechanical ventilation and vasopressors were necessary during 67% of the measurements. The baseline cardiac index derived from TPTD with the PiCCO-device was 4±1.6L/min/m 2 .

Comparison of S cv O 2 _CeVOX_jug and S cv O 2 _BGA_jug in patients with jugular CVC (primary endpoint)
In patients with jugular CVC (primary endpoint) S cv O 2 _BGA_jug and S cv O 2 _CeVOX_jug were significantly correlated (r = 0.567; p<0.001).

Analyses with jugular and femoral CVC
To further analyze the potential impact of catheter position (jugular or femoral), the amount of S cv O 2 _BGA, time to calibration of the CeVOX-device and other variables on accuracy and precision of S cv O 2 _CeVOX we analyzed the total dataset including jugular as well as femoral catheter positions.
S cv O 2 _BGA and S cv O 2 _CeVOX significantly correlated (r = 0.607; p<0.001). Mean values of S cv O 2 _CeVOX and S cv O 2 _BGA were comparable (75.0±11.9 vs. 74.5±9.9%; p = 0.262) with a mean bias of 0.55%, LLOA of -18.9%, ULOA of 20.0% and a PE of 25.9% (Fig 3).   Close association of one or more parameters to (the amount of) the bias S cv O 2 _CeVOX-S cv O 2 _BGA could be of interest for practical use, e.g. to provide an internal control for the device suggesting recalibration by measurement of S cv O 2 _BGA as a kind of recalibrationalarm.

Predictors of imprecision of S cv O 2 _CeVOX in all patients with jugular or femoral CVC
Therefore, we performed multivariate analysis regarding the amount of the bias |S cv O 2 _Ce-VOX-S cv O 2 _BGA|. With regard to practical application only those variables were included in the regression analysis that would be available during continuous use of CeVOX after a single initial calibration.
For this major secondary endpoint we used a three-step-approach: In a first step we performed a multiple regression analysis in the total dataset (n = 120) regarding the amount of the bias |S cv O 2 _CeVOX-S cv O 2 _BGA|. In a second step, we randomly divided the 120 datasets in 80 evaluation datasets and 40 independent validation datasets.

Variable Association to bias S cv O 2 _CeVOX-S cv O 2 _BGA Association to the amount |S cv O 2 _CeVOX-S cv O 2 _BGA| r-value p-value r-value p-value
The final model derived from the 80 evaluation datasets included |S cv O 2 _CeVOX-S cv O 2 _BGA_baseline| (p<0.001; t-value 11.610) as outstanding predictor of |S cv O 2 _CeVOX-S cv O 2 _BGA|. Furthermore, "time to last calibration" was independently associated with | S cv O 2 _CeVOX-S cv O 2 _BGA| with borderline significance (p = 0.065; t-value 1.875) in this model. The predictive capabilities of this model were high (R = 0.830; R 2 = 0.689).
A prediction model derived from these two parameters provided a ROC-AUC of 0.931 (p<0.001) in the evaluation group to predict an amount of the bias |S cv O 2 _CeVOX-S cv O 2 _BGA| !5%. The ROC-AUC for this model was slightly higher than that for |S cv O 2 _Ce-VOX-S cv O 2 _BGA_baseline| (AUC = 0.898; p<0.001) and substantially higher than for time to last calibration (AUC = 0.779; p<0.001; Fig 5).
In a third step, the prediction model derived from the evaluation group provided a ROC-AUC of 0.903 (p<0.001) in the validation group to predict an amount of the bias | S cv O 2 _CeVOX-S cv O 2 _BGA| !5%. This was in the same range as for |ScvO2_CeVOX-ScvO2_BGA_baseline| (AUC = 0.905; p<0.001), but substantially higher than for "time after last calibration" (AUC = 0.758; p = 0.020; Fig 6).

Discussion
S cv O 2 has been used to guide resuscitation with [3] and without success [16,17,18]. Most of the patients in the intervention group in these studies were equipped with a CVC with continuous S cv O 2 monitoring capability. Despite the use of continuous measurement of S cv O 2 and S mv O 2 for several decades, there are only few studies available that prove their accuracy, precision and clinical usefulness.
This also applies to the CeVOX-device. Our observational study compared S cv O 2 _CeVOX to S cv O 2 _BGA during an 8-hours period without recalibration, but repeated withdrawal of blood to determine S cv O 2 _BGA. We deliberately included patients with a femoral CVC to increase the yield in lower values of S cv O 2 , since in critically ill patients with (analgo)-sedation S cv O 2 in vena cava inferior usually is lower than in vena cava superior. Using this approach the following main results were found: 1. For higher values of S cv O 2 _BGA derived from jugular CVCs (primary endpoint) the S cv O 2 _CeVOX_jug provided acceptable bias, percentage error and limits of agreement.
2. By contrast, for lower values of S cv O 2 predominantly withdrawn from femoral CVCs, S cv O 2 _CeVOX_fem provided an acceptable bias, but inappropriately high values for the percentage error and the limits of agreement as well as a poor correlation to S cv O 2 _BGA (r = 0.488).
3. Analysis of the entire data-pool with jugular as well as femoral CVCs allowed for the multivariate analysis which demonstrated that the position of the CVC per se was not independently associated with the bias S cv O 2 _CeVOX-S cv O 2 _BGA. The amount of the bias was independently associated with the amount of the change of S cv O 2 _CeVOX compared to the initial calibration to S cv O 2 _BGA_baseline as well as to low values of S cv O 2 _BGA_baseline. Furthermore, increasing time to the initial calibration was associated to the amount of the bias with borderline significance. As for most of the few previously published studies the bias for S cv O 2 _CeVOX was acceptable and clearly within a range between -1% and +1%, irrespective of jugular (+0.45%) or femoral (+0.63%) position of the CVC [7,8,9,10,11].
In case of jugular CVC, lower and upper limits of agreement (-13,0%; +13.9%) for S cv O 2 _-CeVOX_jug were in the range of the two previous in vitro and of three in vivo studies with nearly identical values as for the study by Molnar and colleagues including patients from the same ICU as this study (-12.5; +13.2%; see Table 1).
However, for lower S cv O 2 _CeVOX_femderived from a femoral CVC LLOA (-23.8%) and ULOA (+25.0%) were not in the acceptable range. Consequently, percentage error values were acceptable for jugular measurements (16.6%), "borderline" for the totality of measurements (25.9%) and out of the acceptable range for femoral measurements (34.5%). At first glance, even the percentage error for femoral measurements seems in the same range as given by Molnar et al. (35.5%). However, a closer look at this study demonstrates that the percentage error was calculated by dividing the difference between ULOA and LLOA by the mean of S cv O 2 _Ce-VOX and S cv O 2 _BGA. This is an unusual method to calculate the percentage error which results in values twice as high as suggested by Critchley and colleagues [19]. Consequently, the percentage error according the Critchley-method would have been 17.8% in the Molnar-study which is the range of our findings for jugular measurements.
Our findings of a lower precision of S cv O 2 _CeVOX_fem in case of lower S cv O 2 _BGA derived from femoral measurements raise the key question, if imprecision is related to femoral measurement, to lower values of S cv O 2 _BGA or to any other variables. With a wide range of S cv O 2 _BGA between 33% and 90%, jugular as well as femoral measurements, different standardized intervals between measurements and concomitant TPTD measurement of CI our data allowed for univariate as well as multivariate analyses of different potential confounders of S cv O 2 _CeVOX.
Several previous studies (see Table 1) suggested a systematic imprecision resulting in an underestimation of high and an overestimation of high values of S cv O 2 _BGA [7,8,10]. By contrast, in our study there was a better agreement for higher values of S cv O 2 in general without a hint for a systematic underestimation.
One of five previous studies demonstrated an underestimation of S cv O 2 _BGA by the CeVOX-device in case of high cardiac index [8]. These findings were neither confirmed by our univariate nor by multivariate analyses. However, the study by Baulig et al. was performed in patients undergoing cardiac surgery with a markedly lower mean CI of 2.6 (range 1.0-4.5 L/min/m 2 ) compared to our study (mean CI 4.3±1.6; range 2.1-7.4 L/min/m 2 ).
On the contrary, our results with increasing imprecision of CeVOX with lower values of S cv O 2 _BGA are in line with the statement of Molnar et al. that "the scatter increases as S cv O 2 goes below 65%" [11].
Although our study suggests that imprecision of the CeVOX is associated rather to low values of S cv O 2 than to femoral measurement per se, a potential impact of femoral measurement has to be discussed: Kissoon and co-workers demonstrated in an animal experiment simulating various hemodynamic conditions that oximetry-based estimates of S cv O 2 derived with a different catheter were more imprecise for measurements in the inferior vena cava than in the superior vena cava. They concluded that "the oximetry catheter is capable of identifying changes in S cv O 2 under physiological conditions usually encountered in clinical medicine but was less accurate at the extremes of physiology and when placed in the inferior vena cava catheter especially during hypovolaemia and hypoxia" [20].
Finally, "drifts over time" have been reported [10], suggesting decreasing precision over time. These findings are supported by our study showing uni-and multivariate association of the amount of the bias to the time to last calibration. This clearly suggests the need for regular re-calibration of continuous oximetry catheters estimating S cv O 2 .
According to our analyses of the total data pool as well as of separate evaluation and validation groups changes in S cv O 2 _CeVOX compared to the baseline calibration were-by far-the strongest predictors of imprecision of the S cv O 2 _CeVOX.

Practical implications
Derivation of a formula-based on changes in S cv O 2 _CeVOX compared to calibration and on time to last calibration-predicting inappropriate precision of S cv O 2 _CeVOX might have practical implications. Since the usefulness of this formula was confirmed in an independent validation group of patients, this formula could be implemented as a kind of automated quality control suggesting re-calibration. A similar "calibration-index" has been suggested for re-calibration of continuous pulse contour analysis derived cardiac index by transpulmonary thermodilution [21]. A modification of the suggested formula has been included in the latest software of the PiCCO-2-device as a kind of decision support to optimize re-calibration and to improve the yield of relevant findings by thermodilution.

Strengths and limitations
Heterogeneity of the patients, also investigating measurements derived from femoral CVCs increasing the yield of low values of S cv O 2 , availability of TPTD to measure CI can be considered as strengths of the study. These characteristics also allowed for the multivariate analyses. Furthermore, the number of patients and measurements was sufficient to validate findings within an evaluation group in an independent validation group.
Finally, the findings of the study might have practical implications with regard to the above-mentioned "calibration-index" as a kind of automated quality control which might improve patient care and safe personal as well as material resources.
Nevertheless, this single-center study is based on a limited number of patients and measurements over a short period.

Conclusions
Continuous estimation of S cv O 2 by the CeVOX is accurate and precise, if the values are close to those at baseline. However, re-calibration should be performed in case of substantial changes compared to baseline. Furthermore, low values of S cv O 2 and longer time to calibration were associated with imprecision of the CeVOX-device. The integration of a calibration.index indicating the need for recalibration might help to improve the precision of the CeVOX device.
Finally, continuous measurement of S cv O 2 with the CeVOX cannot replace S cv O 2 _BGA in instable patients. On the other hand, CeVOX might be useful for monitoring of stable patients as a pre-test tool for more differentiated monitoring in case of changes in S cv O 2 _CeVOX. Improved continuous assessment of S cv O 2 including automated quality control might be useful also during anaesthesia in high risk surgery and during the stabilization phase in certain patients in an accident and emergency department.