Optimal threshold of three-dimensional echocardiographic fully automated software for quantification of left ventricular volumes and ejection fraction: Comparison with cardiac magnetic resonance disk-area summation method and feature tracking method

Aims Novel fully automated left chamber quantification software for three-dimensional echocardiography (3DE) has a potential for reliable measurement of left ventricular (LV) volumes and ejection fraction (LVEF). However, the optimal setting of global LV endocardial border threshold has not been settled. Methods and results We performed LV volumes and LVEF analysis using fully automated left chamber quantification software (Dynamic HeartModelA.I., Philips Medical Systems) in 65 patients who had undergone both 3DE and cardiac magnetic resonance (CMR) examinations on the same day. We recorded LV end-diastolic volume (LVEDV) and LV end-systolic volume (LVESV) according to the change in LV global border threshold settings from 0-point to 100-point with each increment of 10-point. These values were compared to the corresponding values of CMR with disk-area summation method and feature tracking (FT) method. Coverage probability (CP) was calculated as an index of accuracy and reliability. Fully automated software provided LV volumes and LVEF in 57 patients (Feasibility: 88%). LVEDV and LVESV increased steadily according to the increase in border threshold and reached minimal bias when border threshold setting was 80 against CMR disk-summation method and 90 against CMR FT method. Corresponding CP of LVEF was 0.74 and 0.84 against disk-area summation method and FT method. Conclusions With CMR values as a reference, LV endocardial border threshold value can be set around 80 to 90 with the same number of LV end-diastole and end-systole threshold to approximate LVEDV, LVESV and LVEF with clinically acceptable CP values of LVEF.


Methods and results
We performed LV volumes and LVEF analysis using fully automated left chamber quantification software (Dynamic HeartModel A.I. , Philips Medical Systems) in 65 patients who had undergone both 3DE and cardiac magnetic resonance (CMR) examinations on the same day. We recorded LV end-diastolic volume (LVEDV) and LV end-systolic volume (LVESV) according to the change in LV global border threshold settings from 0-point to 100-point with each increment of 10-point. These values were compared to the corresponding values of CMR with disk-area summation method and feature tracking (FT) method. Coverage probability (CP) was calculated as an index of accuracy and reliability. Fully automated software provided LV volumes and LVEF in 57 patients (Feasibility: 88%). LVEDV and LVESV increased steadily according to the increase in border threshold and reached minimal bias when border threshold setting was 80 against CMR disk-summation method and 90 against CMR FT method. Corresponding CP of LVEF was 0.74 and 0.84 against disk-area summation method and FT method. PLOS

Introduction
Left ventricular (LV) ejection fraction (LVEF) using transthoracic two-dimensional echocardiography (2DE) is still one of the most important cardiac metrics to quantify LV function. Several cut-off values of LVEF has been used to select candidates for implantable cardiac defibrillator [1] and to determine discontinuation of cancer therapy [2,3]. However, manual tracing on the endocardial border produces inter-observer measurement variabilities that may not be negligible. To overcome this problem, several ultrasound vendors have produced semiautomated LV border detection software that have still limitations due to several reasons. Recent advances in fully automated software (HeartModel A.I. ) for left heart chamber quantification with three-dimensional echocardiography (3DE) have been shown promising to quantify LV volumes and LVEF in a routine clinical setting [4][5][6]. If editing was not performed, the software provides identical numerical values of the measurements on the same 3DE datasets every time. Several studies had determined its accuracy against 3DE manual quantification analysis [4][5][6][7][8] or cardiac magnetic resonance (CMR) as a reference [7,[9][10][11][12]. However, it still underestimates LV volumes and overestimate LVEF against CMR [7,[9][10][11][12]. The software allows regional and global LV endocardial border editing. Although regional LV endocardial border editing produces a minor change of LV volumes, resulting in the subtle change in LVEF calculation, change of LV endocardial border threshold from 0-point to 100-point makes a remarkable change in LV volumes and LVEF. There are no reports to systematically determine optimal border setting for LV volumes as well as LVEF measurements. Recently, the updated version of the software (Dynamic HeartModel A.I. ) has been released, and it now provides LV volume curve using 3DE speckle tracking technology.
Accordingly, the aim of this study was to determine optimal LV border threshold at enddiastole (ED) and end-systole (ES) using Dynamic HeartModel A.I. and HeartModel A.I. for the measurements of LVEDV, LVESV, and LVEF against CMR as a reference.

Study population
We retrospectively searched patients who had clinically indicated CMR examination and also agreed to undergo 3DE examination on the same day in University of Occupational and Environmental Health Hospital from January 2017 to July 2018. Among 107 patients for potentially eligible patients, a total of 65 patients who had one-beat 3DE datasets acquisition with specific ultrasound machine that were required for fully automated left chamber quantification software analysis was finally selected for the analysis. All study patients were ethnic Japanese. The ethics committee of the University of Occupational and Environmental Health Hospital approved the study protocol, and informed consent was waived due to the retrospective nature of the analysis. MA) equipped with a fully sampled matrix-array transducer (X5-1). The depth and sector angle were adjusted to include the entire left and right chambers. Specific one-beat acquisition mode (3D HMQ) was used to acquire 3DE datasets. Several one-beat acquisitions were performed. These datasets were stored digitally for offline analysis.

3D echocardiographic analysis: Automated quantification
HeartModel A.I. . One-beat 3DE full-volume data sets were analyzed using fully automated quantification software (HeartModel A.I. , QLAB, version 10.5, Philips Medical Systems) that detects LV endocardial surfaces using an adaptive analytical algorithm that consists of knowledge-based identification of initial global shape and LV chamber orientation, followed by patient-specific adaptation [7]. After initiating the program, the software automatically determined the ED and ES frames using motion analysis, followed by construction of ED and ES LV casts. The software subsequently determined LVDEV, LVESV, and LVEF without geometric assumptions. To determine the effect of global editing on LV volumes and LVEF, we adjusted ED and ES endocardial border threshold with each increase of threshold value from 0-point to 100-point via 10-point increment, and obtained LVEDV and LVESV at each increment, although adjustments at increments of 1-point are possible. We did not perform any manual editing of endocardial border in every patient and every border setting. LVEF was calculated as LVEDV at specific border threshold-LVESV at specific border threshold divided by LVEDV at specific border threshold multiplied by 100.
This fully automated quantification software produces LV cast based on ultrasound knowledge database, and it occasionally creates erroneous LV cast. Erroneous LV cast constructions are defined if the border of the casts were constructed not on the actual myocardial-blood interface with manual editing correction impossible.
Dynamic HeartModel A.I. . In contrast to HeartModel that provides LVEDV and LVESV, the new software (Dynamic HeartModel A.I. , QLAB version 11.0, Philips Medical Systems) incorporates 3DE speckle tracking technology. It provides LV volume curve during one cardiac cycle. Like HeartModel analysis, we adjusted ED and ES endocardial border threshold with each increase of threshold value from 0-point to 100-point via 10-point increment and obtained LVEDV and LVESV at each increment. Again, we did not perform manual editing. LVEF was calculated as the same process.

CMR acquisition
CMR imaging was performed with a 3T scanner (Discovery 750W, GE Healthcare) with a phased-array cardiovascular coil. In each patient, retrospective electrocardiographically-gated localizing spin-echo sequences were used to identify the long axis of the heart. Steady state fully precession (SSFP) dynamic gradient-echo cine loops were acquired using retrospective electrocardiographic gating and parallel imaging techniques during 10-sec to 15-sec breathholds, with a following general parameters: slice thickness of the imaging planes 8 mm, field of view 40×40 cm, scan matrix 200×160, flip angle 50˚, repetition/echo times 3.8/1.7 ms, views per segment 20, number of reconstructed cardiac phases 20.

CMR analysis
CMR LV volumes were measured from multiple short-axis SSFP images using an analytical software (Segment v2.2, Medviso, Lund, Sweden), and the disk-area summation method was used for the calculation of the LVEDV and LVESV. LVEF was measured by standard formula.

Feature tracking (FT) CMR analysis
CMR images were analyzed using commercial FT software (2D CPA MR; TomTec Imaging Systems, Unterschleissheim, Germany), which is a vector-based analysis tool based on a hierarchical algorithm [13][14][15]. Using three apical long-axis cine SSFP images, the LV endocardial border at end-systole was semi-automatically drawn after 3-point clicking. Manual adjustment of endocardial border was performed when required. Subsequently, the software automatically propagates the contour and follows its features throughout the cardiac cycle to generate LV volume curve, from which the software provided LVEDV, LVESV and LVEF.

Statistical analysis
Continuous variables are expressed as mean ± SD or as median and 25 th to 75 th percentile according to data distribution. All statistical analyses were performed using commercially available software (JMP version 13.1.0, SAS Institute, Cary, NC; R version 3.4.3, The R foundation for Statistical Computing, Vienna, Australia; and Prism 8, GraphPad Software, Inc., La Jolla, CA). Freedman analysis with post-hoc comparison was performed to compare LVEDV, LVESV, and LVEF among different border settings and CMR. Linear correlations and Bland-Altman analysis were performed between the two methods. Since Dynamic HeartModel and CMR FT provide LV volume curves, we determined the reliability of the curves from Dynamic HeartModel against that obtained from CMR FT. Data were extracted using plots digitalization (WebPlotDigitizer), and each curve was digitized and interpolated 100 points during one cardiac cycle. A linear correlation was performed using 100 plots between the two methods, and r-values from linear correlation analysis were calculated in each patient. We calculated coverage probability (CP) as the percentage of the number of measurements whose difference fulfilled pre-defined criteria. Regarding LVEDV and LVESV, the cut-off value was 30 mL between 3DE and CMR measurements [16][17][18]. Regarding LVEF, the cut-off value was set as 10% between 3DE and CMR determined LVEFs [16][17][18]. Optimal threshold of fully automated software (Fig 1). The mean values of LVEF measured on standard CMR method and CMR FT method were 35 ±17% and 35 ±14%.   Table 2 presents statistical analysis of correlation, bias, 95% limit of agreement (LOA) and CP between Dynamic HeartModel with each border setting and CMR disk-area summation method. Smallest bias of LVEDV, LVESV, and LVEF between CMR and Dynamic HeartModel with high CP value (0.74) of LVEF was observed at border threshold value of 80-point at both ED and ES.  Table 3 represents statistical analysis of correlation, bias, 95% LOA and CP between Dynamic HeartModel with each border setting and CMR FT method. Smallest bias of LVEDV, LVESV, and LVEF between CMR and Dynamic HeartModel with high CP value (0.84) of LVEF was observed at border threshold value of 90-point at both ED and ES.  Comparison of HeartModel A.I. to CMR S1 and S2 Tables show analysis of correlation, bias, 95% LOA and CP between HeartModel with serial border setting and CMR disk-area summation method (S1 Table) and between HeartModel and CMR FT method (S2 Table). Smallest bias of LVEDV, LVESV, and LVEF between CMR disk-area summation method and HeartModel with high CP value (0.67) of LVEF was observed at border threshold value of 80 at both ED and ES. Smallest bias of LVEDV, LVESV, and LVEF between CMR FT method and HeartModel with high CP value (0.82) of LVEF was observed at border threshold value of 90 at both ED and ES. However, observed CP values of LVEDV, LVESV and LVEF were lower than that observed using Dynamic HeartModel.

Discussion
This is the first systematic study to investigate optimal endocardial border threshold values for determination of LV volumes and LVEF using 3DE fully automated left chamber quantification software (Dynamic HeartModel) and its earlier version (HeartModel) against CMR as a reference. Our results showed that the bias between both Dynamic HeartModel and HeartModel determined LVEDV and LVEDV were smallest when the border threshold value was set to 80-point against CMR disk-area summation method, or 90-point when compared to CMR FT method. Dynamic HeartModel could improve measurement accuracy over previous version of the software because the former was yielded with higher values of CP with equivalent border setting.

Previous studies
The importance of cardiac chamber size and function in cardiac imaging cannot be overemphasized. The limitations of 2DE have been inexorably mentioned in previous studies, including foreshortening and geometric assumptions [19][20][21]. The Recommendations for Cardiac Chamber Quantification by Echocardiography in Adults from American Society of Echocardiography and European Association of Cardiovascular Imaging suggested although LV size should be routinely assessed by 2DE, 3DE measurement and reporting of LV volumes is recommended when feasible [22]. Despite the good correlation with CMR, several previous studies using manual or semi-automated contour method showed underestimation of 3DE derived-LV volumes against CMR [19,20]. With the 3DE gaining popularity and momentum, efforts have been made to integrate its practice into daily routine of ultrasound laboratories. However, previous software were at best semi-automatic and often manual adjustment were required, true border delineation often necessitates additional extra times with observer variabilities that cannot be negligible. Therefore, even though 3DE offers significant improvements, there are impediments to its adoption into clinical practice due to tedious manual editing.
Fully automated left chamber quantification software with knowledge-based technology utilizes 3DE datasets with less technical demanding steps, and the time required to analyze 3DE LV volumes and LVEF is only less than 30 seconds without any manual editing [4,7]. Time required for the analysis is quite short compared with that using standard CMR disksummation method (5-10 minutes) or CMR FT method (2-3 minutes). The software allows global and regional editing to correct endocardial border. The global editing permits adjustment of the border threshold value from 0-point to 100-point on ED and ES borders to obtain an LV casts that fit the visual assessment. Previous validation studies consistently reported that 3DE with fully automated quantification software underestimate LVEDV (-13 mL to -30 mL) and LVESV (-7 mL to -29 mL) against CMR as a reference, albeit each author used different LV endocardial border settings [7,[9][10][11][12]. It is well known that the accurate endocardial border setting is essential for the accurate measurement of LV volumes and LVEF. However, there are still no recommendations regarding the optimal endocardial border threshold values that can be routinely employed for clinical practice.

Current study
In this study, feasibility of LV volumes and LVEF measurements was 88%, and the values were in agreement with the previous studies [4,6]. Fully automated quantification software is designed and trained to recognize two myocardial borders-the inner and outer extents of the myocardial tissue-those being at the blood-tissue interface and at the interface of the noncompacted and compacted myocardium. Border line can be changed according to global border threshold setting. We investigated how the ED and ES border threshold value setting in fully automated left chamber quantification software affected our quantification of LV volumes DHM "x" means DHM using threshold of "x". https://doi.org/10.1371/journal.pone.0211154.t003 Optimal threshold of fully automated software and LVEF, in comparison to the measurement by CMR as references. In addition to standard disk-area summation method with serial SSFP short axis images, we also applied CMR FT analysis in this study. CMR FT analysis is another method of choice to quantify LV function. Since FT analysis uses 3 apical views to generate LV volumes and LVEF like echocardiography, the comparison between the two methods are more appropriate. It also provides LV volume curves from 3 apical SSFP images. Thus, it had an opportunity to compare LV volume curves derived from Dynamic HeartModel and CMR FT. As seen in Figs 2 and 4, we found there was parallel increase in LVEDV and LVESV and corresponding parallel decrease in LVEF when ED and ES endocardial border threshold were adjusted simultaneously from value of 0-point to 100-point at increment of 10-point. We observed that larger border threshold values allowed LV volumes to closely approximate volumes obtained by CMR. These findings reaffirm the fact that previous routine echocardiographic endocardial border delineations usually result in smaller sizes of LV volumes compared to CMR quantifications, if echocardiographers do not specifically adjust outwardly their manual LV border tracing. Our results showed that the bias of LVEDV and LVESV between both Dynamic HeartModel/HeartModel and CMR disk-area summation method were smallest when the border threshold value was set to 80, or 90 when compared to CMR FT method. Using border threshold value 80 or 90 at Optimal threshold of fully automated software both ED and ES for determination of LVEF, the bias between fully automated quantification software and CMR was also smallest with acceptable CP values. Higher CP value of Dynamic HeartModel over HeartModel supports improved measurement accuracy, and this could be relating to updated algorithm of the software. The Dynamic HeartModel performs 3DE speckle tracking on the LV endocardial border in each frame over the entire cardiac cycle, resulting in the generation of LV volume curves. We determined its reliability against corresponding LV volume curves derived from CMR FT method. Median r -values by linear regression analysis from the whole subjects (r = 0.87) supported its accuracy and reliability.
In summary, this is the first study to perform systematically the determination of optimal border threshold by fully automated left chamber quantification software for the quantification of LV volumes and LVEF compared to the standard CMR disk-area summation method and Optimal threshold of fully automated software the novel CMR FT method. The fully automated quantification software with appropriate border threshold value setting has a potential for the evaluation of LV function with confidence.

Study limitations
There were several limitations that should be acknowledged in this study. First, the software was not associated with 100% feasibility. In this study, there were 2 cases of incorrect LV casts that could not be edited, and other 6 patients were excluded due to very poor image quality rejected 3DE datasets acquisition, resulting in overall feasibility of 88%. The number however, was similar to previously performed feasibility studies [6]. Second, we only investigated ED and ES border threshold settings at every 10-point basis. Third, to determine the reliability of LV volume curves by Dynamic HeartModel, we used corresponding LV volume curve using CMR FT that has not been validated as a reference standard. Fourth, temporal resolution of CMR (76ms) used in this study was above the Society for Cardiovascular Magnetic Resonance recommendations for LV function assessment [23]. It may overestimate LVESV, resulting in some underestimation of LVEF. Last, further study should be required to validate our results in the larger number of study patients with multicenter setting.

Conclusion
We concluded that LV endocardial border threshold value can be set around 80 to 90 with the same number of LV ED and ES threshold to approximate LVEDV, LVESV and LVEF with clinically acceptable CP values of LVEF as CMR values as a reference.

Fig 8. A linear correlation (upper panels) and Bland-Altman analysis (lower panels) of left ventricular end-diastolic volume (A), left ventricular end-systolic volume (B), and left ventricular ejection fraction (C) between Dynamic HeartModel and HeartModel with use of a border threshold value of 80 at both enddiastole and end-systole.
https://doi.org/10.1371/journal.pone.0211154.g008 Supporting information S1