Skip to main content
Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

CMRSegTools: An open-source software enabling reproducible research in segmentation of acute myocardial infarct in CMR images

  • William A. Romero R.,

    Roles Software, Validation, Writing – original draft, Writing – review & editing

    Affiliation Univ Lyon, INSA-Lyon, Université Claude Bernard Lyon 1, UJM-Saint Etienne, CNRS, Inserm, CREATIS UMR 5220, U1294, Saint Etienne, France

  • Magalie Viallon ,

    Roles Conceptualization, Data curation, Formal analysis, Methodology, Validation, Writing – original draft, Writing – review & editing

    Affiliation Univ Lyon, INSA-Lyon, Université Claude Bernard Lyon 1, UJM-Saint Etienne, CNRS, Inserm, CREATIS UMR 5220, U1294, Saint Etienne, France

  • Joël Spaltenstein,

    Roles Software, Validation, Writing – review & editing

    Affiliation Spaltenstein Natural Image, Geneva, Switzerland

  • Lorena Petrusca,

    Roles Validation, Writing – review & editing

    Affiliation Univ Lyon, INSA-Lyon, Université Claude Bernard Lyon 1, UJM-Saint Etienne, CNRS, Inserm, CREATIS UMR 5220, U1294, Saint Etienne, France

  • Olivier Bernard,

    Roles Software, Validation, Writing – review & editing

    Affiliation Univ Lyon, INSA-Lyon, Université Claude Bernard Lyon 1, UJM-Saint Etienne, CNRS, Inserm, CREATIS UMR 5220, U1294, Saint Etienne, France

  • Loïc Belle,

    Roles Data curation, Funding acquisition, Validation, Writing – review & editing

    Affiliation Department of Cardiology, Centre Hospitalier Annecy-Genevois, Annecy, France

  • Patrick Clarysse,

    Roles Methodology, Validation, Writing – review & editing

    Affiliation Univ Lyon, INSA-Lyon, Université Claude Bernard Lyon 1, UJM-Saint Etienne, CNRS, Inserm, CREATIS UMR 5220, U1294, Saint Etienne, France

  • Pierre Croisille

    Roles Conceptualization, Data curation, Funding acquisition, Methodology, Validation, Writing – review & editing

    Affiliation Univ Lyon, INSA-Lyon, Université Claude Bernard Lyon 1, UJM-Saint Etienne, CNRS, Inserm, CREATIS UMR 5220, U1294, Saint Etienne, France


In the last decade, a large number of clinical trials have been deployed using Cardiac Magnetic Resonance (CMR) to evaluate cardioprotective strategies aiming at reducing the irreversible myocardial damage at the time of reperfusion. In these studies, segmentation and quantification of myocardial infarct lesion are often performed with a commercial software or an in-house closed-source code development thus creating a barrier for reproducible research. This paper introduces CMRSegTools: an open-source application software designed for the segmentation and quantification of myocardial infarct lesion enabling full access to state-of-the-art segmentation methods and parameters, easy integration of new algorithms and standardised results sharing. This post-processing tool has been implemented as a plug-in for the OsiriX/Horos DICOM viewer leveraging its database management functionalities and user interaction features to provide a bespoke tool for the analysis of cardiac MR images on large clinical cohorts. CMRSegTools includes, among others, user-assisted segmentation of the left-ventricle, semi- and automatic lesion segmentation methods, advanced statistical analysis and visualisation based on the American Heart Association 17-segment model. New segmentation methods can be integrated into the plug-in by developing components based on image processing and visualisation libraries such as ITK and VTK in C++ programming language. CMRSegTools allows the creation of training and testing data sets (labeled features such as lesion, microvascular obstruction and remote ROI) for supervised Machine Learning methods, and enables the comparative assessment of lesion segmentation methods via a single and integrated platform. The plug-in has been successfully used by several CMR imaging studies.


Myocardial infarct size is a crucial parameter serving as a surrogate endpoint in clinical studies of new therapeutic interventions after Acute Myocardial Infarction (AMI). Therefore, considerable research engagement has been directed toward the automatic quantification of the infarct size as well as the size of the post-reperfusion no-reflow lesion based on CMR Late Gadolinium Enhanced (LGE) images [15]. These studies require reliable post-processing tools for the segmentation and quantification of the myocardial infarct lesion; in general, these tools are reported to be an in-house code development in a numeric computing environment (i.e. MATLAB) or a commercial application software. Not only do these tools restrict reproducibility of the studies but they can also prevent a full understanding of the use of computational methods. Because what is behind the mathematical model (constraints, parameters, etc.) is not clear, it is difficult to provide an objective assessment of lesion segmentation methods. For instance, a recent publication of Wu et al. [5] presents a general review of cardiac scar segmentation methods (including methods for non-LGE images). This review reports quantitative results across publications. The conclusion states the need of a public benchmarking of the methods as a fairer review of their performances. This is a challenge that requires tools for open collaboration in a growing culture focused on advancing methods by reproducible science [6, 7]. The neuroimaging research community is a clear example of this research and development methodology [8, 9].

In the field of Cardiac MRI, there is a gap between open-source software built within the scientific community and commercial applications. The issue is not the availability of computational methods for image processing in open-source packages per se, but the fact that these methods are not assembled in a suitable workflow for processing and carrying out statistical analysis on large clinical cohorts. A suitable workflow requires to select the structure of interest (myocardial segmentation), perform a computational method on the data from this structure, and analyse the output based on the American Heart Association (AHA) 17-segment model [10] (commonly named bullseye plot). Although it does not seem complex, it has been apparently embraced only by commercial applications which offer a custom-made workflow for myocardial segmentation and statistical analysis based on the AHA model. To provide an example, Table 1 presents an overview of well-known image processing tools in terms of two requirements for Cardiac MR image analysis: myocardial segmentation and statistical analysis following the standardised AHA 17-segment model. In general, a commercial software is a product designed to be marketed under a utilisation licence. The user must pay for a binary version which must be used as in accordance with the licence directives. There is no access to the source code of the commercial product (closed-source). Freeware software is free of charge but closed-source. In open-source software, the user has access to the source code and the different licences (i.e. BSD, GNU GPL, etc.) provide the rights to copy, modify, redistribute the code, as well as build and release binary versions for commercial purposes [11]. Open-source software promotes open collaboration and research communities have been attracted to this innovation methodology [12]. Overall, open-source image processing software is not closely linked to the CMR concepts and practice. Therefore, a customised computer-based tool such as the one presented in this paper, could provide on-the-job support to ease analysis task performance on large clinical cohorts.

Table 1. Overview of available post-processing tools. Requirement 1 (R1): myocardial segmentation, requirement 2 (R2): statistical analysis and visualisation following the AHA 17-segment model.

Moreover, a new demand has arisen with the advent of computational methods based on Machine Learning (ML). Using ML, the scientific community is now developing new solutions to solve the long-standing problem of a reliable quantification of infarct lesions. These approaches require annotated data sets which are generated and assessed by standard methods (i.e. signal threshold versus reference mean, full-width at half-maximum, etc.) [4, 5, 20, 21]. Consequently, there is a clear need for a unified reference of computational methods for lesion segmentation. This could be met with a common assessment platform that could easily be used by clinicians, radiologists and MRI researchers.

This paper presents CMRSegTools, a novel application software that allows users to benchmark new lesion segmentation methods as compared to state-of the-art methods in the field. CMRSegTools, includes customised tools for user-assisted myocardial segmentation; manual, semi- and automatic lesion segmentation which can be used in the generation of training data sets for Machine Learning (ML) strategies. This computer-based tool can be easily deployed on clinical infrastructures, closing the gap between analysis methods and their direct utilisation in clinical trials, as demonstrated in the studies [2225].

CMRSegTools plug-in

CMRSegTools is an application software specifically designed to ease myocardial segmentation, quantification and tissue characterisation on CMR images. CMRSegTools has been implemented as a plug-in for the widely available commercial OsiriX and open-source Horos DICOM viewers [26, 27] in order to take advantage of usability features and advanced visualisation tools (ROI outline tools, colour look-up tables, pixel interpolation algorithms for display, etc.) as well as the effortless deployment on a clinical or research infrastructure. Fig 1 shows the main components of the plug-in and its integration within the OsiriX/Horos environment.

Fig 1. CMRSegTools architecture diagram.

The plug-in has access to the DICOM viewer functionalities through the Application Program Interface (API) exposed by OsririX/Horos. The DICOM viewer runs on MacOS operating system. List of the frameworks and toolkits in each layer: ITK, Insight Toolkit; VTK, Visualisation Toolkit; DCMTK, DICOM Toolkit; Cocoa, Mac OSX Native AP; OpenGL, Computer Graphics API.

The user-friendly Graphical User Interface (GUI) empowers the user to: segment the Left-Ventricle (LV); calculate infarct size, Microvascular Obstruction (MVO) lesion size, Endocardial Surface Length (ESL), and Endocardial Surface Area (ESA) amongst others. The real-time feedback updates the GUI with statistical information (number of pixels, minimum and maximum pixel value, mean value, standard deviation, etc.) of the pixels selected by the segmentation along with an interactive histogram. The image of interest is also dynamically updated (Epicardium and Endocardium contours, LV/RV junction landmark, region segments statistics in the 17-segment model, AHA [10]) according to the user interaction (Fig 2). Furthermore, CMRSegTools saves the workspace state, which means that it restores the GUI and image viewer to the state they were in when the plug-in was last used. The source code of the plug-in is available at

Fig 2. CMRSegTools workspace.

The CMRSegTools GUI (left) shows the histogram highlighting the bins matching the lesion segmentation. The user can select the algorithms for the segmentation of the myocardium or some of its regions. The image viewer (right) displays the segmented area in magenta as well as the Epicardium (green) and Endocardium (red) contours, LV/RV junction landmark (yellow segment) and AHA region segments. The LGE image presented was obtained with a 3D IR GRE sequence.

Interoperability and extension

The CMRSegTools plug-in is independent of scanner vendor, sequence type and protocol. The OsiriX/Horos framework [26, 27] eliminates common requirements for data access (access to PACS), database management, transformation from the DICOM standard to final post-processing format, and data provenance; requirements which are critical in the development of clinical trials. In addition, the plug-in includes a functionality to import ROI from cvi42 (Circle Cardiovascular Imaging Inc., Calgary, Canada) file format.

This application software has been designed with the CMR lexicon in mind. All graphical elements generated by the plug-in (i.e. segmentation contours, set of pixels corresponding to the lesion, etc.) are tagged with keywords for identification within the OsiriX/Horos runtime environment. This enables the interaction with other plug-ins. For example, a specialised plug-in for myocardial segmentation (i.e. ML-based segmentation [21]) can export the segmentation output with the keywords “CMRSegTools: Epicardium” and “CMRSegTools: Endocardium” (ROI names can be set by the user in the OsiriX/Horos ROI manager). Therefore, this segmentation can be identified by CMRSegTools. Inversely, another plug-in can start working with the CMRSegTools ROIs (“CMRSegTools: Epicardium”, “CMRSegTools: Endocardium”, “CMRSegTools:Remote_stddev”, etc.) by searching for these keywords in the OsiriX/Horos ROI manager.

Statistical results include the number of pixels, minimum and maximum pixel values, mean value, and standard deviation, among others of each ROI (lesion, MVO and remote) per segment, slice and for the entire myocardium. This information can be exported to an Excel file or as a delimited text file (comma-separated values) for a more comprehensive examination over different studies or cohorts of interest using application software for statistical analysis (e.g. pandas [28], R scripts [29] or any statistical package).

The main computational methods are written in C++ using the Insight Segmentation and Registration ToolKit (ITK) [30] and the Visualisation ToolKit (VTK) [31], which grant extensibility and interoperability in terms of new functionalities required, that is new image processing methods.

Myocardial segmentation

The initialisation of the endocardial and epicardial contours is performed in 2 mouse clicks, and then automatically adjusted to the myocardium using the Boundary Enhanced Automated Surface (BEAS) algorithm [32, 33].

BEAS was originally developed for automatic segmentation of 3D cine-loop echocardiographic images to provide an efficient, fast and accurate solution for quantification of the main left ventricular volumetric indices used in clinical routine. In addition, the method was adapted for 3D+time CMR data sets (acquired by a cine steady state free precession sequence, SSFP) and benchmarked against the data sets available from the MICCAI 2009 Cardiac MR Left Ventricle Segmentation Challenge [34]. The benchmark demonstrated the technique to be robust, efficient and fast in terms of accuracy and computational load, which makes the BEAS algorithm suitable for a clinical practice [33].

The contours automatically determined by BEAS can be interactively corrected using OsiriX/Horos ROI edition functionalities such as the repulsor tool. Endocardial and epicardial contours can also be manually outlined. Moreover, the contours can be delineated on the basal, mid-cavity and apical slices and automatically propagated to the intermediate slices.

In order to remove pixels from the segmented myocardium close to the endocardial and epicardial contours, and avoid contamination from partial volume effect between myocardial tissue and blood (in the cavity) or air (at the myocardium to lung interface), an inner (epicardial) and outer (endocardial) offset in millimetres can be optionally set as a parameter of the segmentation method. These parameters can be configured by the user in the plug-in preferences.

Lesion segmentation

Methods for measuring infarct size can be divided into visual assessment, manual planimetry, and voxel-based approaches. Visual assessment scores hyperenhancement on a 5-point grading scale on the AHA 17-segment model [10]. Manual planimetry involves a manual definition of the hyperenhanced regions of interest (ROI) across contiguous short-axis slices in order to calculate the lesion size [10, 35]. Both methods are inherently operator dependent, which makes them time consuming as they rely on a manual process. Consequently, they are unsuitable for clinical routine or the analysis of large cohorts. Voxel-based thresholding techniques start with a comparative measurement of hyperenhanced regions with a remote healthy region set as a reference (Signal Threshold versus Reference Mean, STRM) [36]. Threshold calculation can be based on a statistical measure such as the Full-Width at Half-Maximum (FWHM) [37, 38] or on histogram information as in the case of the Gaussian mixture model (GMM) approaches [39]. Depending on whether the methods require an initialisation parameter to calculate the threshold or not they are classified as semi-automatic or fully automatic. These methods are more time efficient and suitable for large scale processing.

The methods for the segmentation of the hyperenhanced region in CMRSegTools include:

  • manual cut-off: the threshold is calculated as T = μ+ where μ is the mean of signal intensity in the remote and healthy myocardium and σ its the standard deviation, c is the parameter to set by the user (a positive integer usually between 2 and 10);
  • manual histogram-based segmentation: the pixel intensity range is defined by two interactive cursors on the histogram;
  • automatic cut-off: the threshold is calculated as where Imax is the maximum intensity within the myocardium in the infarct zone (method known in the literature as Full-Width at Half-Maximum, FWHM);
  • Gaussian mixture model: the threshold is defined as the intersection of a Rician-Gaussian mixture model fitted on the histogram [39];
  • Hsu modified: this method is based on the feature analysis and combined thresholding (FACT) algorithm where an iterative refining process eliminates false-positives based on regional analysis [37];
  • hidden Markov random field model with expectation-maximisation (HMRF-EM): this algorithm is founded on the method proposed in [40]. It labels pixels within the myocardium by calculating the set of parameters (mean and standard deviation) of the Probability Density Function (PDF) that defines each label (lesion and remote) [41].

These methods are described in detail in S1 Appendix.

Manual definition of the MVO region

Only Hsu and HMRF-EM methods provide a segmentation of the MVO (or no-reflow) regions. The MVO region can be selected (or edited) manually after segmentation of the infarct by using a pixel selection tool (brush) or region growing functionality. If there is a MVO region, this new class of tissue is automatically added to the infarct size, with corresponding pixels being automatically displayed in yellow (default color on the GUI for highlighting the MVO region). Similarly, if MVO pixels are categorised as false positive and highlighted in yellow by Hsu or HMRF-EM methods, they can be manually removed. MVO size and statistics (number of pixels, minimum and maximum pixel value, mean value, standard deviation, etc.) are calculated (Fig 3).

Fig 3. CMRSegTools segmentation output.

(a) Image viewer before executing CMRSegTools; (b) Epicardium and endocardium contours from the myocardial segmentation, lesion segmentation (pixels in magenta colour), histogram and quantification statistics without MVO region; (c) Segmented regions, histogram and quantification statistics after defining the MVO region (contour and pixels in yellow colour).

Transmural extent, endocardial surface length and endocardial surface area

An interactive 2D/3D wiper tool (white segments in Fig 2) allows the user to manually identify the lateral edges of myocardial infarct in order to determine the Endocardial Surface Length (ESL) of the infarct (in percent of the endocardial contour perimeter) [42]. When multiple slices are evaluated (wipers location is propagated across slices), the Endocardial Surface Area (ESA) is automatically calculated. This functionality also intrinsically allows the user to clean up the segmentation results a posteriori and provides a transmural extent estimation.

Testing data

A series of synthetic images emulating myocardial infarct lesions were generated with arbitrary size, transmural extent, percent Signal Enhancement (%SE) and CNR. In order to test the influence of the noise level on the results, several synthetic levels of noise were computed to reproduce what was previously measured in vivo using various CMR sequences in patients [43]. To achieve a realistic model of the cavity, myocardium, infarct size and shape, and signal intensity in each compartment, the numeric images were built using multi-slice 3D data sets acquired on patients. Synthetic images were generated with a resolution similar to the one used in clinical acquisitions [41].

In vivo images from the MIMI study ( Identifier: NCT01360242, [44]) were used in this study to validate the plug-in functionalities on real clinical data. The MIMI (Minimalist Immediate Mechanical Intervention) study was a multicentric randomised trial aiming at comparing immediate stenting and 24–48h delayed stenting in patients treated with primary percutaneous coronary intervention (PCI); the MR protocol included T1, T2, T2*, Cine and 3D/2D Late Gadolinium Enhancement (LGE) scans. The data set is available through the Human Heart Project [45] under research collaboration agreement.


The validation of the implemented methods within CMRSegTools includes a functional validation of the myocardial segmentation method; verification of the lesion segmentation output on synthetic data, and an assessment of the automated segmentation functionalities based on a reference made by a radiology expert.

Myocardial contour segmentation with BEAS

An example of the CMRSegTools LV segmentation output is presented in Fig 4. These results show the variation of the segmented contours across different image types (LGE, EGE and cine images) as a consequence of the image contrast difference and myocardial mass.

Fig 4. Performance of BEAS segmentation on images with different contrast type.

(a) User initialisation on the LGE image; (b) first and (c) second iteration of the automatic segmentation based on (a). (d) User initialisation on the EGE image and (e) segmentation output. (f) Segmentation output on cine bSSFP. All images were taken at the same slice location and from the same patient at different breath-hold scans.

Although the BEAS algorithm was originally developed for segmentation of echocardiographic images, the method has shown very good performance on images with homogeneous myocardium wall (well-defined ring structure) such as cardiac cine MRI based on balanced SSFP contrast. As this method is based on active contours principle [46, 47], the segmentation is affected by the heterogeneity of the myocardium wall with multiple tissue classes (lesion, MVO and remote healthy pixels). These tissue classes have an impact on the energy function that penalises the deviation to the initial model (LV template presented in Fig 4a and 4d), which means that the user may have to manually correct the LV segmentation contours.

Validation on synthetic data

The numerical phantom generation from real clinical data (2D and 3D) is illustrated in Fig 5 and a performance benchmark across segmentation methods is presented in Fig 6. Fig 6 shows the relative and absolute error in the calculation of the lesion size as a function of the CNR. Absolute relative error is plotted on logarithmic scale to highlight the results of 2-SD, 3-SD and 5-SD for CNR > 5. Overall, the accuracy improves as the CNR increases. The segmentation by STRM (n-SD) methods is strongly determined by the remote region and CNR. Lower CNR (CNR < 6) leads to an under-estimation (less pixels categorised as MI) by 2-SD, 3-SD,5-SD and FWHM Region methods, and to an over-estimation (more pixels categorised as MI) by FWHM Max, GMM, Hsu and HMRF-EM methods. This is due to the incorrect categorisation of the infarct edges or some noisy pixels. On a real image coil sensitivity variation, cardiac motion and partial-volume effects may generate this kind of intermediate SI pixels that may be included within the n-SD constraint or statistical model. However, at lower CNR scenarios, a lower cut-off provides a more accurate infarct segmentation with an error less than 3.8%. Lower CNR impacts the performance of the GMM method reaching about 60% of mean relative error. The best performance corresponds to STRM with 5-SD, FWHM, Hsu and HMRF-EM providing a very low margin of error starting at CNR > 3.

Fig 5.

Example of numerical phantom generation (c), (d), (e) and (f) from a real clinical data set (a) and (b). Images were acquired with a 3D-IR-GRE sequence on a patient with a myocardial infarct. (a) Traced endocardial and epicardial contours on one slice (2D), (b) manual lesion segmentation, (c) output synthetic image (2D). The figures (d), (e) and (f) illustrate transmural extent and ESL calculation on a numerically generated infarct with known size and variable transmural extent around a user-defined mean value. (d) Red, green and yellow contours are respectively the endocardial and epicardial borders, and the mean simulated transmural extent; (e) final synthetic image. (f) CMRSegTools segmentation output (HMRF-EM); magenta pixels on the synthetic image correspond to those classified as myocardial infarct on the numerical phantom. The orange wipers delineate the ESL.

Fig 6. Performance of the lesion segmentation methods across different Contrast-to-Noise Ratio (CNR) scenarios.

The curves show the relative error (RE) and bars the absolute relative error (ARE, logarithmic scale) for each segmentation method. The CNR was calculated as the ratio between the SI difference of the infarct and healthy myocardium over the standard deviation of the SI in the myocardium.

Radiology expert segmentation vs automated segmentation methods

An example of the manual segmentation made by an expert radiologist (P.C. with 20 years of experience in cardiovascular imaging) against each one of the methods within CMRSegTools on a patient with MVO is presented in Figs 7 and 8. The relative error between methods output and expert segmentation of a ROI was calculated as

Fig 7. Comparison between the manual segmentation made by an expert (without the definition of the MVO region), and the segmentation made by each of the methods within CMRSegTools.

For each segmentation, the figure shows: epicardium (green) and endocardium (red) contours from the myocardial segmentation, lesion segmentation (magenta), remote healthy myocardium (turquoise), no-reflow region by HMRF (yellow), histogram, quantification statistics and relative error.

Fig 8. Comparison between the manual segmentation made by an expert (including a manual definition of the MVO region), and the segmentation made by each of the methods within CMRSegTools.

For each segmentation, the figure shows: epicardium (green) and endocardium (red) contours from the myocardial segmentation, lesion segmentation (magenta), remote healthy myocardium (turquoise), no-reflow region (yellow), histogram, quantification statistics and relative error.


Although a considerable research has been conducted to develop and assess computational methods for the quantification of lesion size in CMR, to date and to the best of our knowledge, there has been no initiative to provide a unified reference for lesion segmentation methods within a common assessment platform. The baseline of existing publications reporting infarct lesion sizing is usually an offline analysis based on in-house (closed-source) implementations of conventional or, less often, innovative segmentation algorithms. Following this approach it is difficult to make an objective evaluation of existing methods. The assessment would only be performed when a medical device vendor releases the methods in a commercial software product. In this scenario, the benchmarking of computational methods implemented by research groups and commercial solutions is biased by the vendor constraints for the algorithms. Since manufacturers are under regulatory restrictions, the integration of the latest hardware and software on their platforms typically requires long production time, often resulting in regrettable delays in early testing of new, advanced and potentially valuable algorithms. CMRSegTools is therefore a contribution to enable reproducible research by assembling the most used methods for infarct segmentation in a suitable workflow executed in a single and widely available environment: Osirix/Horos DICOM viewers. This platform was chosen because it provides an Application Programming Interface (API) that grants access to the data management, user interaction and visualisation functionalities. It was also chosen because it has an open-source licence. However, OsiriX/Horos is a macOS-only software, which limits its deployment on platforms based on a different operating system.

One of the contributions to the infarct segmentation workflow is the implementation of a semi-automatic method for the LV segmentation based on active contours [32, 33]. The BEAS algorithm utilises a LV template provided by the user in order to automatically fit the template to the epicardium and endocardium boundaries. Although the outcome of the BEAS segmentation may be affected by the different tissue classes in the myocardial wall, this functionality enhances automation capabilities for clinical routine or the analysis of large cohorts as demonstrated in [33]. A reliable segmentation of the myocardium is important as the edge of the endocardium may constitute up to 50% of the infarct perimeter; this means that a wrong determination of the endocardial infarct border is a large source of variability in final infarct size measurements. At present, there are no automated algorithms that can reliably distinguish the bright LV cavity from the bright endocardial border of the infarct. This challenge is being addressed by new segmentation methods (including non-LGE images) based on ML [5, 21, 48, 49].

To validate the correspondence between the mathematical model (S1 Appendix) and the code implementation, the correctness of the computational methods has been verified with synthetic data. This also allowed the benchmarking of infarct segmentation methods (Fig 6) on synthetic data. The main lesson emerging from the analysis is that STRM (n-SD) methods with n < 5 and GMM are easily influenced by CNR. At lower CNR conditions the outcomes from these methods are unreliable, making them unsuitable for real clinical conditions. When CNR > 5, 2-SD and 3-SD are insensible to CNR. The HMRF-EM has shown a good performance in most of the different scenarios (CNR > 3). A comparison between expert and automated segmentation methods is presented in Figs 7 and 8 as an example of the output of the current version. CMRSegTools has been tested and used successfully in several studies [2225].

An extended assessment or benchmarking of infarct segmentation methods is beyond the scope of this paper. The goal is to provide a computer-based tool which includes well-known lesion segmentation methods (S1 Appendix) in order to empower research scientists to reproduce results and assess comparatively new methods. In this direction, CMRSegTools source code has been released under the CeCILL licence (a French free software license compatible with GPL) that grants users the right to copy, modify, and distribute new versions of the plug-in.

Machine learning approaches for SI classification would require reference data that can be applied to a learning process [50]. With CMRSegTools, it is possible to generate labeled features (infarct, MVO and remote ROIs) extracted from the data and the expected quantification based on reference segmentation methods. These specific features are the main input of supervised learning approaches. As shown in Figs 7 and 8, outcomes from ML methods can be easily compared to manual, semi- or full-automated state-of-the-art methods. Additionally, errors performed by ML methods may be labeled as well in order to improve the accuracy, specially in strategies involving active learning.

The analysis scenario presented in this paper (methods benchmarking and expert vs computational methods assessment) can be replicated by research scientists working in CMR on their own data or by downloading the testing data used for this work from the Human Heart Project [45]. This is particularly interesting as it allows reproducible research in this field.


CMRSegTools is an application software for the comparative assessment of infarct segmentation methods applied to CMR images via a single and integrated platform (widely available Osirix and Horos DICOM viewers). This helps to improve the reproducibility of post-processing methods in clinical studies. This application software works on native DICOM images; the OsiriX/Horos functionalities enable direct connection to PACS making the plug-in independent of scanner vendor, imaging sequence and protocol. Interoperability functionalities allow importing data from external software such as ROI created in other application software (e.g. cvi42). New segmentation methods can be easily integrated into the plug-in by developing components based on well-known image processing and visualisation libraries such as ITK and VTK in C++ programming language. By using CMRSegTools, it is possible to create training and testing data sets that can be used in Machine Learning approaches. In addition, statistical measurements can be exported for further examination in specialised data analysis tools.

Supporting information

S1 Appendix. Lesion segmentation methods in CMRSegTools.



The authors would like to thank Alessandro Volz, M.Sc. (Medical imaging software developer. Bellinzona, Switzerland.) for his valuable contributions to this project.


  1. 1. Kim HW, Farzaneh-Far A, Kim RJ. Cardiovascular magnetic resonance in patients with myocardial infarction: current and emerging applications. Journal of the American College of Cardiology. 2009;55(1):1–16. pmid:20117357
  2. 2. Kim RJ, Wu E, Rafael A, Chen EL, Parker MA, Simonetti O, et al. The use of contrast-enhanced magnetic resonance imaging to identify reversible myocardial dysfunction. New England Journal of Medicine. 2000;343(20):1445–1453. pmid:11078769
  3. 3. Kim RJ, Shah DJ, Judd RM. How we perform delayed enhancement imaging: HOW I DO…. Journal of cardiovascular magnetic resonance. 2003;5(3):505–514. pmid:12882082
  4. 4. Karim R, Bhagirath P, Claus P, Housden RJ, Chen Z, Karimaghaloo Z, et al. Evaluation of state-of-the-art segmentation algorithms for left ventricle infarct from late Gadolinium enhancement MR images. Medical image analysis. 2016;30:95–107. pmid:26891066
  5. 5. Wu Y, Tang Z, Li B, Firmin D, Yang G. Recent Advances in Fibrosis and Scar Segmentation From Cardiac MRI: A State-of-the-Art Review and Future Perspectives. Frontiers in Physiology. 2021;12. pmid:34413789
  6. 6. Nosek BA, Alter G, Banks GC, Borsboom D, Bowman SD, Breckler SJ, et al. Promoting an open research culture. Science. 2015;348(6242):1422–1425. pmid:26113702
  7. 7. Donoho DL. An invitation to reproducible computational research. Biostatistics. 2010;11(3):385–388. pmid:20538873
  8. 8. Nichols TE, Das S, Eickhoff SB, Evans AC, Glatard T, Hanke M, et al. Best practices in data analysis and sharing in neuroimaging using MRI. Nature neuroscience. 2017;20(3):299–303. pmid:28230846
  9. 9. Eglen SJ, Marwick B, Halchenko YO, Hanke M, Sufi S, Gleeson P, et al. Toward standard practices for sharing computer code and programs in neuroscience. Nature neuroscience. 2017;20(6):770–773. pmid:28542156
  10. 10. on Myocardial Segmentation AHAWG, for Cardiac Imaging: R, Cerqueira MD, Weissman NJ, Dilsizian V, Jacobs AK, et al. Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart: a statement for healthcare professionals from the Cardiac Imaging Committee of the Council on Clinical Cardiology of the American Heart Association. Circulation. 2002;105(4):539–542.
  11. 11. Guadamuz A. Free and open source software. LAW AND THE INTERNET, 3rd Ed, Edwards L, Waelde C, eds, Oxford: Hart. 2009;.
  12. 12. Levine SS, Prietula MJ. Open collaboration for innovation: Principles and performance. Organization Science. 2014;25(5):1414–1433.
  13. 13. Fedorov A, Beichel R, Kalpathy-Cramer J, Finet J, Fillion-Robin JC, Pujol S, et al. 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magnetic resonance imaging. 2012;30(9):1323–1341. pmid:22770690
  14. 14. Schneider CA, Rasband WS, Eliceiri KW. NIH Image to ImageJ: 25 years of image analysis. Nature methods. 2012;9(7):671–675. pmid:22930834
  15. 15. Koenig M, Spindler W, Rexilius J, Jomier J, Link F, Peitgen HO. Embedding VTK and ITK into a visual programming and rapid prototyping platform. In: Medical Imaging 2006: Visualization, Image-Guided Procedures, and Display. vol. 6141. International Society for Optics and Photonics; 2006. p. 61412O.
  16. 16. Medis Suite MR, Medis Medical Imaging Systems BV, Leiden, The Netherlands;.
  17. 17. cvi42 for Cardiovascular MRI, Circle Cardiovascular Imaging Inc., Calgary, Canada;.
  18. 18. Heiberg E, Sjögren J, Ugander M, Carlsson M, Engblom H, Arheden H. Design and validation of Segment-freely available software for cardiovascular image analysis. BMC medical imaging. 2010;10(1):1–13. pmid:20064248
  19. 19. CAAS MR Solutions, Pie Medical Imaging, Maastricht, The Netherlands;.
  20. 20. Ranschaert ER, Morozov S, Algra PR. Artificial intelligence in medical imaging: opportunities, applications and risks. Springer; 2019.
  21. 21. Chen C, Qin C, Qiu H, Tarroni G, Duan J, Bai W, et al. Deep learning for cardiac image segmentation: a review. Frontiers in Cardiovascular Medicine. 2020;7:25. pmid:32195270
  22. 22. Bernelin H, Mewton N, Si-Mohamed S, Croisille P, Rioufol G, Bonnefoy-Cudraz E, et al. Neprilysin levels at the acute phase of ST-elevation myocardial infarction. Clinical Cardiology. 2019;42(1):32–38. pmid:30284298
  23. 23. Paccalet A, Crola Da Silva C, Mechtouff L, Amaz C, Varillon Y, De Bourguignon C, et al. Serum Soluble Tumor Necrosis Factor Receptors 1 and 2 Are Early Prognosis Markers After ST-Segment Elevation Myocardial Infarction. Frontiers in pharmacology. 2021; p. 2083. pmid:34539391
  24. 24. Hayek A, Paccalet A, Mechtouff L, Da Silva CC, Ivanes F, Falque H, et al. Kinetics and prognostic value of soluble VCAM-1 in ST-segment elevation myocardial infarction patients. Immunity, inflammation and disease. 2021;9(2):493–501. pmid:33559404
  25. 25. Zeilinger MG, Kunze KP, Munoz C, Neji R, Schmidt M, Croisille P, et al. Non-rigid motion-corrected free-breathing 3D myocardial Dixon LGE imaging in a clinical setting. European Radiology. 2022;32. pmid:35184220
  26. 26. Rosset A, Spadola L, Ratib O. OsiriX: an open-source software for navigating in multidimensional DICOM images. Journal of digital imaging. 2004;17(3):205–216. pmid:15534753
  27. 27. Horos, free and open source code software (FOSS) program;.
  28. 28. Wes McKinney. Data Structures for Statistical Computing in Python. In: Stéfan van der Walt, Jarrod Millman, editors. Proceedings of the 9th Python in Science Conference; 2010. p. 56 – 61.
  29. 29. R Core Team. R: A Language and Environment for Statistical Computing; 2018. Available from:
  30. 30. McCormick MM, Liu X, Ibanez L, Jomier J, Marion C. ITK: enabling reproducible research and open science. Frontiers in neuroinformatics. 2014;8:13. pmid:24600387
  31. 31. Schroeder W, Martin KM, Lorensen WE. The visualization toolkit an object-oriented approach to 3D graphics. Prentice-Hall, Inc.; 1998.
  32. 32. Barbosa D, Pedrosa J, Heyde B, Dietenbeck T, Friboulet D, Bernard O, et al. heartBEATS: A hybrid energy approach for real-time B-spline explicit active tracking of surfaces. Computerized Medical Imaging and Graphics. 2017;62:26–33. pmid:28784271
  33. 33. Queirós S, Barbosa D, Heyde B, Morais P, Vilaça JL, Friboulet D, et al. Fast automatic myocardial segmentation in 4D cine CMR datasets. Medical image analysis. 2014;18(7):1115–1131. pmid:25042098
  34. 34. Radau P, Lu Y, Connelly K, Paul G, Dick A, Wright G. Evaluation framework for algorithms segmenting short axis cardiac MRI. The MIDAS Journal-Cardiac MR Left Ventricle Segmentation Challenge. 2009;49.
  35. 35. Mewton N, Revel D, Bonnefoy E, Ovize M, Croisille P. Comparison of visual scoring and quantitative planimetry methods for estimation of global infarct size on delayed enhanced cardiac MRI and validation with myocardial enzymes. European journal of radiology. 2011;78(1):87–92. pmid:19962260
  36. 36. Bondarenko O, Beek AM, Hofman MB, Kühl HP, Twisk JW, Van Dockum WG, et al. Standardizing the definition of hyperenhancement in the quantitative assessment of infarct size and myocardial viability using delayed contrast-enhanced CMR. Journal of Cardiovascular Magnetic Resonance. 2005;7(2):481–485. pmid:15881532
  37. 37. Hsu LY, Natanzon A, Kellman P, Hirsch GA, Aletras AH, Arai AE. Quantitative myocardial infarction on delayed enhancement MRI. Part I: Animal validation of an automated feature analysis and combined thresholding infarct sizing algorithm. Journal of Magnetic Resonance Imaging: An Official Journal of the International Society for Magnetic Resonance in Medicine. 2006;23(3):298–308. pmid:16450367
  38. 38. Hsu LY, Ingkanisorn WP, Kellman P, Aletras AH, Arai AE. Quantitative myocardial infarction on delayed enhancement MRI. Part II: Clinical application of an automated feature analysis and combined thresholding infarct sizing algorithm. Journal of Magnetic Resonance Imaging: An Official Journal of the International Society for Magnetic Resonance in Medicine. 2006;23(3):309–314. pmid:16450368
  39. 39. Friman O, Hennemuth A, Peitgen H. A Rician-Gaussian mixture model for segmenting delayed enhancement MRI images. In: Proc. 16th Sci. Meeting Int. Soc. Magn. Reson. Med.(ISMRM’08); 2008. p. 1040.
  40. 40. Zhang Y, Brady M, Smith S. Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm. IEEE Transactions on Medical Imaging. 2001;20(1):45–57. pmid:11293691
  41. 41. Viallon M, Spaltenstein J, de Bourguignon C, Vandroux C, Ammor A, Romero W, et al. Automated quantification of myocardial infarction using a hidden Markov random field model and the EM algorithm. In: International Conference on Functional Imaging and Modeling of the Heart. Springer; 2015. p. 256–264.
  42. 42. Ibanez B, Aletras AH, Arai AE, Arheden H, Bax J, Berry C, et al. Cardiac MRI endpoints in myocardial infarction experimental and clinical trials: JACC scientific expert panel. Journal of the American College of Cardiology. 2019;74(2):238–256. pmid:31296297
  43. 43. Viallon M, Jacquier A, Rotaru C, Delattre BM, Mewton N, Vincent F, et al. Head-to-head comparison of eight late gadolinium-enhanced cardiac MR (LGE CMR) sequences at 1.5 tesla: From bench to bedside. Journal of magnetic resonance imaging. 2011;34(6):1374–1387. pmid:21972032
  44. 44. Belle L, Motreff P, Mangin L, Rangé G, Marcaggi X, Marie A, et al. Comparison of Immediate With Delayed Stenting Using the Minimalist Immediate Mechanical Intervention Approach in Acute ST-Segment–Elevation Myocardial Infarction: The MIMI Study. Circulation: Cardiovascular Interventions. 2016;9(3):e003388. pmid:26957418
  45. 45. The Human Heart Project;.
  46. 46. Kass M, Witkin A, Terzopoulos D. Snakes: Active contour models. International journal of computer vision. 1988;1(4):321–331.
  47. 47. Caselles V, Catté F, Coll T, Dibos F. A geometric model for active contours in image processing. Numerische mathematik. 1993;66(1):1–31.
  48. 48. Xu C, Xu L, Gao Z, Zhao S, Zhang H, Zhang Y, et al. Direct delineation of myocardial infarction without contrast agents using a joint motion feature learning architecture. Medical image analysis. 2018;50:82–94. pmid:30227385
  49. 49. Guo S, Xu L, Feng C, Xiong H, Gao Z, Zhang H. Multi-level semantic adaptation for few-shot segmentation on cardiac image sequences. Medical Image Analysis. 2021;73:102170. pmid:34380105
  50. 50. Maes F, Robben D, Vandermeulen D, Suetens P. The role of medical image computing and machine learning in healthcare. In: Artificial Intelligence in Medical Imaging. Springer; 2019. p. 9–23.