Skip to main content
Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Biomechanical rupture risk assessment of abdominal aortic aneurysms using clinical data: A patient-specific, probabilistic framework and comparative case-control study

  • Lukas Bruder,

    Roles Data curation, Formal analysis, Methodology, Software, Visualization, Writing – original draft

    Affiliation Mechanics & High Performance Computing Group, Technical University of Munich, Garching, Germany

  • Jaroslav Pelisek,

    Roles Data curation, Investigation, Project administration, Resources, Supervision, Validation, Writing – review & editing

    Affiliations Department of Vascular Surgery, University Hospital Zurich, Zurich, Switzerland, Clinic for Vascular and Endovascular Surgery, Technical University of Munich, Munich, Germany

  • Hans-Henning Eckstein,

    Roles Conceptualization, Data curation, Funding acquisition, Investigation, Project administration, Resources, Supervision, Writing – review & editing

    Affiliation Clinic for Vascular and Endovascular Surgery, Technical University of Munich, Munich, Germany

  • Michael W. Gee

    Roles Conceptualization, Funding acquisition, Methodology, Software, Writing – review & editing

    Affiliation Mechanics & High Performance Computing Group, Technical University of Munich, Garching, Germany


We present a data-informed, highly personalized, probabilistic approach for the quantification of abdominal aortic aneurysm (AAA) rupture risk. Our novel framework builds upon a comprehensive database of tensile test results that were carried out on 305 AAA tissue samples from 139 patients, as well as corresponding non-invasively and clinically accessible patient-specific data. Based on this, a multivariate regression model is created to obtain a probabilistic description of personalized vessel wall properties associated with a prospective AAA patient. We formulate a probabilistic rupture risk index that consistently incorporates the available statistical information and generalizes existing approaches. For the efficient evaluation of this index, a flexible Kriging-based surrogate model with an active training process is proposed. In a case-control study, the methodology is applied on a total of 36 retrospective, diameter matched asymptomatic (group 1, n = 18) and known symptomatic/ruptured (group 2, n = 18) cohort of AAA patients. Finally, we show its efficacy to discriminate between the two groups and demonstrate competitive performance in comparison to existing deterministic and probabilistic biomechanical indices.

1 Introduction

An abdominal aortic aneurysm (AAA) is a slowly progressing vascular disease, causing an enlargement of the infrarenal aorta and is considered pathological if the aortic diameter exceeds 30 mm [1]. AAA prevalence has been reported within a range of 1.2% to 3.3% in men older than 60 years based on several studies in western societies [2]. In most cases, AAAs develop asymptomatically over several years, but they can rapidly turn into a serious clinical emergency in case of rupture. More than 50% of patients with a ruptured AAA die before reaching the hospital [1] and perioperative mortality rates range from 40% to 60% [3].

To prevent such a disastrous scenario, the clinical guidelines from the US-based Society for Vascular Surgery recommend elective repair for AAA patients with an aortic diameter greater or equal to 55 mm, regular screening intervals for patients with smaller-sized AAAs and one-time screenings for AAAs in men and women above a certain age and based on established risk factors [1]. This maximum diameter recommendation is based on a risk assessment, where the risk of rupture is weighed against the mortality risk of an elective repair. While the latter risks are relatively well-known, aneurysm rupture is a complex biomechanical failure event. With the increasing use of endovascular repair (EVAR) over open surgical repair (OSR) [4], however, which can be attributed to the significant short term mortality benefit of EVAR (1.4% compared to 4.2%) [1], interventional risks have become a less important factor in the risk assessment process.

Nonetheless, a biomechanical rupture risk assessment can provide an additional important piece of information. It enables the possibility to provide patient-specific screening guidelines, avoid unnecessary interventions [5] and support the clinical decision process for cases that are not covered by the clinical guidelines. The Society for Vascular Surgery’s 55 mm recommendation, e.g., only holds for patients “at low or acceptable surgical risk with a fusiform AAA” [1]. Furthermore, there are no clear or only weak recommendations for women with AAAs of size 50-54 mm, aneurysms with non-fusiform geometries, smaller AAAs [6], or patients at higher surgical risk. In addition to that, not all AAAs are suitable for EVAR, with higher complication rates for AAA cases that are not covered by the instructions for use [7]. Lastly, recent meta-studies (e.g. [8, 9]) on the long term outcomes of EVAR versus OSR could not detect any differences with regards to the all-cause mortality or even concluded in favor of OSR.

In this paper, we present a highly personalized, probabilistic framework for the biomechanical quantification of AAA rupture risk. The framework builds upon a comprehensive database, consisting of tensile experiments that were carried out on 305 AAA tissue samples from 139 patients and corresponding non-invasively and clinically accessible patient data. The approach consistently incorporates the available statistical information in terms of probability distributions in order to account for patient-specific uncertainties about relevant vessel wall properties. We emphasize the importance of accounting for these uncertainties and demonstrate that this leads to a more accurate individualized rupture risk assessment as compared to deterministic approaches.

Our work builds upon previous efforts by our group and collaborators regarding the biomechanical modeling and characterization of AAA in-vivo behavior [1016], as well as several previous studies indicating that biomechanical indices are more accurate predictors for AAA rupture risk than the clinically established maximum diameter criterion [1724]. In contrast to the approaches in [1720, 22, 24], however, we advocate a probabilistic treatment to account for uncertain vessel wall properties. Our work thus goes along the lines of [21], but with the key difference that it includes the stiffness parameters of the AAA vessel wall as statistical quantities, uses patient-specific vessel wall properties and accounts for statistical correlations among these properties.

The paper is organized as follows. Motivated by a failure-based criterion, our rupture risk index is formulated in Section 2.1 incorporating patient-specific statistical information. Section 2.2 defines the biomechanical AAA model and specifies the probabilistic regression model to obtain personalized vessel wall properties. In Section 2.3, a method for the efficient evaluation of the rupture risk index is proposed and in Section 3, the framework is applied on a total of 36 retrospective, diameter matched asymptomatic (group 1, n = 18) and known symptomatic/ruptured (group 2, n = 18) cohort of AAA patients.

2 Materials and methods

2.1 Failure-based probabilistic quantification of rupture risk

2.1.1 Rupture as an event of material failure.

From a mechanical point of view, rupture represents an event of local material failure at a point x in the aneurysm wall, which motivates its definition via a failure function φ(x) and the failure criterion (1)

We limit ourselves to stress-based failure and define rupture as an event where the local wall stress measure σ(x) exceeds the local wall strength σγ(x). This results in the failure function φ(x) = σ(x) − σγ(x), or the criterion (2)

Using the equivalent von Mises stress σvm(x) as the local stress measure σ(x) and an assumed spatially constant wall strength σγ, this criterion can be evaluated as (3) where is the maximum von Mises stress .

It is important to note that the above definition does not incorporate any aspect about failure over time. In order to be able to include time in the analysis, i.e. to make a statement about the risk of rupture in the next year, one would require knowledge about the future progression of the AAA for this patient, such as a model for the aneurysm growth and change in vessel wall properties. Since there is hardly any knowledge about these aspects, we limit the further discussion to a rupture risk assessment at the point of time of the acquired data. While there are sudden events like calcification-induced formation of saccular aneurysms, we assume that in most cases an AAA is a slowly progressing disease and thus our approach has, at least for the near future, sufficient predictive capability.

2.1.2 Existing criteria and rupture risk indices.

Rupture risk estimation for AAAs has been an ongoing research topic over several decades, with many attempts to establish decision criteria for clinical practice. The maximum diameter criterion [1] still represents the most widely used criterion for decision making today. It is often justified by Laplace’s law, which states that the vessel wall stress is proportional to its diameter in spherical geometries. Based on this and with data obtained from several clinical studies, a very simple criterion, (4) has been formulated, relating the patient’s AAA diameter d to a critical maximum diameter dmax. While established in clinical practice and easy to apply using CT or ultrasound imaging, this criterion has often been criticized [25] and is an ongoing subject for discussion [6].

With growing computational resources and advances in the modeling of biomechanical material behavior, the simulation of patient-specific AAA models has been advanced by several research groups. Experiments on harvested AAA samples were able to reveal material parameters and failure properties. In addition with regression models [13, 26, 27] for the prediction of the individual wall strength, this enabled the definition of biomechanics-based indices [19, 20, 22, 28], such as the rupture potential index (RPI) (5) relating the von Mises stress to the wall strength. Furthermore, it could be shown [19, 20, 24] that these indices can be better rupture risk indicators than the maximum diameter criterion.

Experimental testing [13, 26, 27] also revealed significant inter- and intra-patient variabilities in the mechanical properties of AAA tissue, motivating a probabilistic approach to rupture risk estimation [14, 16, 21] and resulting in the probabilistic rupture risk index (PRRI) [21] (6) where the authors used distributions for the wall thickness and wall strength that were fitted on cohort data published by our group [13].

2.1.3 A novel probabilistic approach.

In this work, we propose a novel failure-based, probabilistic rupture risk indicator that consistently incorporates all available statistical information and accounts for correlations among vessel wall properties. Fig 1 (left) illustrates the rationale for our approach, showing how part of the available data is directly involved in the estimation of the risk of rupture, while another part affects the evaluation of the computational model. In general, this data will be correlated, resulting in correlated quantities for the evaluation of rupture risk and necessitating a reformulation.

Fig 1. Rationale for our novel formulation (left) and exemplary visualization of its estimation (right).

The probability of rupture, , is calculated as the volume of the probability distribution within the triangular-shaped area marked in red.

To that end and recalling the rupture criterion from Eq (3), we can calculate the probability of rupture over the joint probability distribution as (7) where is the indicator function defined as (8)

This formulation can be easily extended to, e.g., spatially varying vessel properties using Eqs (1) or (2) as failure events. Furthermore, it includes the PRRI in Eq (6) as a special case, when choosing .

Lastly, it allows for a straightforward visual interpretation as illustrated in Fig 1 (right). The plot shows the joint probability distribution and visualizes the rupture event area in red. The blue area implies a high probability for the joint occurrence of the corresponding stress and strength values. The probability of rupture is simply the volume of this density within the triangular rupture event area. Thus, the larger the overlap between and the red area, the higher .

2.2 Data-informed patient-specific AAA models

2.1.1 Geometry creation from CT imaging and meshing.

Patient-specific 3D AAA geometries are reconstructed via a semi-automatic segmentation process from CT imaging data using the software ScanIP (Synopsys, Mountain View, California) and based on a protocol as described in [12]. The minimal requirement for the spatial resolution of CT scans was 1 mm and for the slice thickness 3 mm. The upper boundary for the segmentation was the branching of the renal arteries and the lower boundary below the bifurcation at the iliac arteries. Due to the small thickness of the AAA wall, its low contrast and the limited resolution of the CT images, it is only possible to extract the blood lumen and intraluminal thrombus (ILT) geometries. After segmentation, the ILT geometry is exported as a surface model for meshing.

In a next step, we use the software Trelis (csimsoft, American Fork, Utah) and bi-linear quadrilateral elements to mesh the abluminal ILT surface. From this surface mesh, the arterial wall layer is extruded with a specified, spatially constant thickness t, resulting in a tri-linear, single layer, hexahedral mesh for the AAA wall. Finally, linear tetrahedral elements are employed for the meshing of the complex ILT geometry and a layer of linear pyramid elements as a transition for mesh compatibility between AAA wall and thrombus. Element sizes were set to 1.6 mm, corresponding to the median of measured thicknesses of AAA wall specimens in our database and leading to hexahedral elements of shape 1.6 mm × 1.6 mm × t for the AAA wall. A mesh convergence study has been performed to assess that the chosen spatial mesh resolution is sufficient in the context of our application. The meshing procedure is also described in [29] in more detail.

2.2.2 Biomechanical modeling.

Previous studies have shown that in order to accurately describe the biomechanical behavior of AAAs, a sufficient model complexity is required [20, 30], while results by [31, 32] indicate that also simpler models might be appropriate. For our purposes, we employ the finite deformation boundary value problem of nonlinear elasticity: (9) (10) (11) where Ω0 is the reference configuration of the AAA, u denotes the displacement field, F = I+ ∇u the deformation gradient, S the second Piola-Kirchhoff stress tensor and σ the Cauchy stress tensor.

On the Neumann boundary γσ, i.e. the luminal ILT surface, an orthonormal load is applied, with the pressure value p and the unit outward surface normal n in the current configuration. Furthermore, at the proximal and distal end surfaces of the AAA model, Γu, we employ a Robin-type boundary condition with spring supports following [29, 33]. The stiffness parameter ks is per unit reference area and set to 100 kPa/mm in this study, while N is the unit outward surface normal in the reference configuration.

To model the constitutive behavior of the ILT, we use the strain energy function proposed in [34] (12) and a linearly decreasing stiffness c from the luminal to the abluminal ILT surface [12]. and are the first and second invariants of the modified right Cauchy-Green deformation tensor , with C = FT F and J = det(F) [29]. The strain energy function employed for the AAA wall material is [35, 36] (13) with stiffness parameters α and β. Both strain energy functions are equipped with an additive volumetric component (14) including the bulk modulus (15) with parameters and for the employed ILT and wall material models and a Poisson’s ratio of ν = 0.48 [12].

To obtain a pressurized in vivo configuration of the AAA, the MULF prestressing method [10, 11] is used, where the applied load corresponds to the mean arterial pressure (MAP = 1/3 systolic pressure + 2/3 diastolic pressure). From this prestressed configuration, the pressure is raised by 50% to simulate elevated blood pressure conditions [21]. Following [19] and for comparability reasons, the values for the systolic and diastolic pressures were set to 121 mmHg and 87 mmHg for all cases, respectively, resulting in a MAP of 98.33 mmHg.

With the finite element discretization from Section 2.2.1, a nonlinear system of equations is obtained, which is solved using an in-house finite element code. We note that in this study we neglect the effect of calcifications in the AAA for simplicity and assume constant vessel wall thickness t and stiffness parameters α and β throughout the aneurysm. Furthermore, we evaluate the maximum von Mises stress as the 99th percentile of the von Mises stress field in the aneurysm.

For the remainder of this work, we will use the parameter to quantity of interest (QoI) map (16) with parameter vector and QoI to denote the forward problem. Thus, calculating for one realization of t, α and β will involve one evaluation of the nonlinear finite element model.

2.2.3 Patient database.

The modeling of patient-specific vessel wall properties here is based on data that has been collected during several research projects between 2008 and 2017 on the mechanobiological behavior of AAAs [13, 15]. The study was approved by the ethics committee of the University Hospital rechts der Isar, Technical University of Munich. AAA patients undergoing elective OSR (including emergency repair due to rupture) at the University Hospital rechts der Isar in Munich, Germany, were added to the database, whenever it was possible to extract tissue samples for mechanical testing. Apart from anamnesis and CT imaging data, hemograms were evaluated and one or more AAA tissue samples harvested during OSR. These samples were mechanically and histologically investigated, resulting in an exhaustive retrospective AAA database. Further information on data collection and experimental testing can be found in [13, 15]. To date, the database contains a total number of 305 entries from an equal number of tissue samples that were collected from 139 patients.

The data can be split into two groups. Invasive properties (cf. Table 1), denoted as , are properties, which have been determined retrospectively from AAA tissue samples and cannot be obtained for a prospective patient by using clinically established methods. They are, however, essential for the biomechanical modeling and simulation of AAAs and the calculation of the probability of rupture using Eq (7). Non-invasive properties (cf. Table 2), denoted by ξ, on the other hand, can be determined with standard methods in the clinic. The subrenal diameter in Table 2 is measured directly below the renal arteries. If the aneurysm reached the renal arteries, the aortic diameter between the celiac artery and the superior mesenteric artery minus 2.5 mm was used instead [12].

Table 1. Invasive properties represent key vessel wall characteristics for a biomechanical rupture risk assessment.

Based on correlations between the invasive and non-invasive properties [13], the goal is to construct a statistical model for the patient-individualized prediction of vessel wall properties Θ(ξ) for a prospective new patient with non-invasive properties ξ. While this process is described in Section 2.2.4, a preprocessing step for the dataset is essential, since values are missing both in the invasive and non-invasive properties for several cases in our database. Moreover, the relatively small number of available data, but high number of non-invasive properties, requires a feature selection process to identify the most important properties in ξ. Similar to [15], we conduct the following preprocessing steps.

Non-invasive features in ξ, where more than 30% of the data points had missing values and patients with more than 30% of missing features were excluded and all other missing non-invasive properties imputed with the corresponding median value across the population. As a consequence, the four parameters calcium, high-sensitivity C-reactive protein (hsCRP), creatine kinase and fibrinogen were disregarded. Afterwards, all non-invasive features were normalized.

Based on a correlation analysis using Spearman’s rank correlation coefficient (cf. S1 Table), the total number of features was reduced to a final selection of 8 variables: maximum AAA diameter, maximum thrombus thickness, AAA length, subrenal diameter, thrombocytes, hemoglobin, mean corpuscular hemoglobin (MCH), mean corpuscular volume (MCV). The restriction was done using a sequential forward selection algorithm similar to [15]. In an attempt to keep the number of non-invasive parameters small, we iteratively added the highest correlating non-invasive parameters to the GP model (see Section 2.2.4) until no further improvement in the leave-one-out cross-validation (LOOCV) scores could be observed. We note, however, that this does not imply that other non-invasive features such as sex, medication or anamnesis parameters do not have an influence on the biomechanical properties of the AAA wall. The resulting dataset , that was used for the analysis in Section 3, consisted of ndata = 251 data points from 113 individual patients and is available as supplementary information to this study (cf. S2 and S3 Tables).

2.2.4 Prediction of invasive vessel wall properties.

Previous approaches to create models for the AAA wall thickness, stiffness parameters or strength were either deterministic [12, 26], based on cohort statistics [21], or did not account for correlations among the vessel wall quantities [15]. In the following, we make use of a multivariate Gaussian process regression model [3739] to address these shortcomings and achieve the following desiderata:

  1. Patient-specific modeling: obtain personalized estimates for the vessel wall quantities Θ based on correlations with the non-invasive properties ξ of a specific, prospective patient.
  2. Probabilistic treatment: take into account the uncertainties in the predictions for Θ (do not ignore statistical information).
  3. Dependencies: model the correlations among the invasive properties Θ in order to obtain a more accurate probabilistic description and avoid physically implausible parameter configurations.

As a result, given the non-invasive properties ξ of a prospective AAA patient, the logarithm (acting as a positivity constraint) of the corresponding prediction Θ(ξ) will follow a multivariate Gaussian distribution with predicted mean μlogΘ and covariance matrix Σlog Θ, i.e. (17)

As we will see in Section 3.2, our approach leads to more accurate estimates for Θ and also a lower variance in the predictions. All relevant details regarding this model are provided in Appendix A.1.

2.3 A Kriging surrogate model for the maximum stress

2.3.1 Estimating the probability of rupture.

Since the calculation of the probability of rupture from Eq (7) using the high-fidelity, nonlinear finite element model from Section 2.2.2 is infeasible for a clinical application, we propose a Kriging surrogate model to speed up computations [4042]. The surrogate model will effectively serve as a proxy for the maximum von Mises stress in the AAA vessel wall (cf. Eq (16)) and allows to make computationally cheap predictions at an arbitrary combination of θ = [t, α, β]T, i.e. (18) with the predicted mean and standard deviation , respectively. For all relevant details, we refer to Appendix A.2. The high-fidelity model can then be simply approximated as , allowing for a direct Monte Carlo estimation of the probability of rupture (19) where (20) and Θip(log Θ), i = 1…neval.

2.3.2 An active learning approach to training.

The Kriging surrogate training process is carried out under the following two demands:

  1. As few as possible high-fidelity model evaluations.
  2. Ensure that the Kriging model is accurate where necessary.

To that end, we adopt and extend the Active Learning-MacKay (ALM) strategy from [43] and choose points for high-fidelity model evaluations such as to minimize a density- and stress-weighted predictive standard deviation objective function (21) where p(log Θ) is the patient-specific probability distribution for the invasive model parameters Θ = [t, α, β, σγ]T from the regression model in Section 2.2.4. The reasoning behind this choice follows from the ALM approach, where only the predictive standard deviations are considered in the objective function. In our case, we are equipped with a probability distribution, p(log Θ), so we can attribute a higher weight to the more probable regions in Θ. Additionally, we pay special attention to points in the input space, where the predicted maximum von Mises stresses are high to ensure the surrogate model accurately replicates the full model in these regions. The problem of choosing an appropriate point θnext for evaluation results in the optimization problem (22) which is approximated by creating a grid over the input space, calculating using the Kriging surrogate and determining (23)

The next evaluation point θnext = [tnext, αnext, βnext]T can then simply be extracted from Θnext. During the active learning, we monitor the average (24) and stop the training process, when there are no more significant changes in with an increasing number of high-fidelity model evaluations.

3 Results

3.1 Framework summary

Based on our retrospective AAA database of non-invasive and invasive data pairs and a multi-output Gaussian process model fitted to this dataset (cf. Section 2.2.4), the necessary steps to estimate the probability of rupture for a prospective patient are:

  • Step 1: Data generation in the clinic: CT imaging, determination of the non-invasive parameters ξ from Table 2
  • Step 2: Geometry creation: segmentation and meshing of the AAA geometry
  • Step 3: Model specification: modeling of the invasive properties Θ(ξ) using the multi-output Gaussian process model from Sections 2.2.4 and A.1.
  • Step 4: Surrogate training: fitting of the Kriging model using active learning
  • Step 5: Post-processing: estimating the probability of rupture

While CT imaging is essential for geometry creation, the rupture risk analysis can also be carried out if no non-invasive properties ξ are available for a prospective patient by using cohort statistics (cf. Model 1, Section 3.2) without personalization. The computational procedure is summarized in Algorithm 1. In practice, it has proven feasible to choose ninit = 8 (where it makes sense to include the predicted mean μlog Θ in the set of initial samples), ngrid = neval = 10, 000 and tol = 1.0 × 10−4.

Algorithm 1 Calculating the probability of rupture

1: Input: Input uncertainties p(logΘ(ξ)), simulation model , tol, ninit, ngrid, neval

2: Set

3: Generate ninit samples and calculate

4: Train a Kriging surrogate using the training data

5: Create a grid over the input space and calculate (cf. Eq (24))

6: while do

7:  Determine θnext using Eq (23) and calculate

8:   Update the Kriging model with the new data point and calculate

9:  Set iter = iter + 1

10: end while

11: Generate neval samples and calculate according to Eq (19) using the Kriging surrogate

12: Output:

3.2 Regression model benchmark

Before demonstrating the framework in full detail, a brief comparison between the multi-output Gaussian process regression model (cf. Section 2.2.4) with existing probabilistic modeling approaches used in the context of AAA rupture risk is provided. To that end, we employ leave-one-out-cross-validation (LOOCV) on our dataset (cf. Section 2.2.3) to test the predictive capabilities of three different models for p(logΘ):

  • Model 1: assuming all variables are log-normally distributed and independent, the joint distribution (25) is obtained, where the means and variances are calculated across the whole population using the dataset , that is (26) with κ ∈ {t, α, β, σγ}. This corresponds to the approach chosen in [21].
  • Model 2: by training single-output Gaussian processes for each output variable separately following [15], the same decomposition of Gaussian distributions as in Eq (25) is obtained, however, with means and variances predicted individually for each patient.
  • Model 3: our proposed multi-output Gaussian process (cf. Eq (17)).

In addition to the mean of the patient standardized mean square error (PSMSE) [15], we also report the mean of the patient predictive entropy (PPE), (27) where is the entropy of the distribution p(logΘ) and a measure of uncertainty or variance for multivariate distributions. With regards to the different measures, it is desirable for both PSMSE and PPE to be small, corresponding to a model which is accurate and produces low-variance estimates. For conciseness, values for the mean of the PSMSE are averaged over the four predictive variables Θ. We refer to [15] for an exhaustive discussion of the LOOCV and calculation of the PSMSE. The obtained results for the three models are shown in Table 3. We note that our proposed model (Model 3) was able to consistently achieve the lowest scores, although the differences are rather small.

Table 3. Leave-one-out-cross-validation (LOOCV) results for the three probabilistic models.

3.3 Framework demonstration for AAA Pat17

To illustrate the application of our proposed framework we demonstrate all steps in detail below, following the outline as presented in Section 3.1. We assume we are provided with CT imaging data and non-invasive properties ξ for one specific prospective AAA (Step 1), referred to as Pat17 in the following.

Fig 2 shows the AAA as seen via CT imaging (I), a 3D rendering of the segmentation result (II) as well as the generated finite element mesh (III) (Step 2). The mesh consists of 117, 218 finite elements and 93, 840 nodal degrees of freedom, with an approximate element size of 1.6 mm.

Fig 2. AAA Pat17 as seen via CT imaging (I), a 3D rendering of the segmentation result (II), the generated finite element mesh (III) and a visualization of the von Mises stress field corresponding to the mean μlogΘ of the predictive distribution p(logΘ) for that AAA (IV).

Table 4 shows the relevant 8 non-invasive properties ξ that are used by the regression model (cf. Section 2.2.4) to obtain the predictive distribution p(logΘ(ξ)), which is specific to Pat17. Along with that, means and standard deviations based on all 113 patients in are provided. Based on this data, we can predict the mean μlogΘ and covariance ΣlogΘ for this patient (Step 3). The obtained distribution is visualized in Fig 3 and the predictive means and standard deviations are provided in Table 5 along with reference values from the cohort. The entropy of p(logΘ) is 3.3050 and thus slightly lower than the LOOCV mean (cf. Table 3). Highest correlations among the invasive properties for Pat17 can be found between t and σγ (), β and σγ (), t and β (ρt,β = −0.1966) as well as α and β (ρα,β = 0.1413).

Table 4. Non-invasive properties ξ for AAA Pat17 as well as cohort means and standard deviations (based on all 113 patients in ) for comparison.

Fig 3. Visualization of the predictive distribution p(logΘ) transformed to the physical parameter range for AAA Pat17.

Plots (I)-(VI) show 2D marginal distributions over all possible parameter combinations between t, α, β and σγ. Highest correlations are observed between t and σγ (), β and σγ (), t and β (ρt,β = −0.1966) as well as α and β (ρα,β = 0.1413).

Table 5. Predicted means and standard deviations for the invasive properties of AAA Pat17 along with cohort values over all ndata = 251 samples for comparison.

Given p(logΘ), the forward model (cf. Eq (16)) for Pat17 is defined. The probability of rupture for this AAA is approximated using a Kriging surrogate model (Step 4). Fig 2 (IV) provides a visualization of the von Mises stresses corresponding to μlogΘ, the mean parameter combination of p(logΘ). Fig 4 shows the decrease of the objective function over the number of iterations on the left as well as a comparison of the Kriging-based approximate distribution together with a Monte Carlo reference calculated using 10, 000 samples on the right.

Fig 4.

Left: Decrease of the objective function over the number of training iterations, where the first training iteration corresponds to the Kriging surrogate after ninit = 8 model evaluations. 11 model evaluations were used for the surrogate creation. Right: Estimated Kriging-based distribution along with a Monte Carlo reference. All densities were calculated using kernel density estimation with Gaussian kernels based on 10, 000 samples of the maximum von Mises stress .

Lastly, the probability of rupture can be estimated using the Kriging surrogate (Step 5), which amounts to 0.47% for Pat17 (cf. Fig 5 for a visualization). We stress that this value must not be compared to the operative risks associated with OSR or EVAR in order to make decisions. Rather, it needs to be put into context with results for other AAA patients that have been computed using the same methodology, which is discussed below in Section 3.4.

3.4 Comparative case-control study using diameter matched groups

To test the efficacy of the framework as a rupture risk indicator and to compare it with existing biomechanical indices, we consider diameter matched groups of asymptomatic (group 1, n = 18) and known symptomatic/ruptured (group 2, n = 18) AAA patients from our database. The groups were chosen such that their maximum diameter mean and standard deviation approximately match (group 1: 62.17±7.18 mm, group 2: 63.06±7.56 mm), rendering a differentiation between the groups based on the maximum diameter criterion ineffective.

For a detailed overview regarding the selection of the two groups, we refer to Table 6. After preprocessing of our original dataset (cf. Section 2.2.3), we restricted the cohort to AAAs with a maximum diameter between 50 and 80 mm in order to obtain an intermediate-sized group of patients. As a result, 64 patients remained, of which 47 had asymptomatic and 17 had symptomatic or ruptured AAAs. The latter were put into one group, since symptomatic AAAs are known to be at an elevated risk of rupture [44]. The reason for the much lower number of symptomatic/ruptured AAAs is that these AAAs often have very large diameters (>80mm). We included AAA patients from a previous case-control study by our group [19], which examined 13 asymptomatic and 12 symptomatic AAA patients. Finally, we manually selected 18 asymptomatic and 18 symptomatic/ruptured patients based on the following criteria:

  • Find two groups with the best match in diameter.
  • Preferably include cases where non-invasive data is available and thus patient-specific invasive properties can be predicted.
  • Disregard cases, where CT images are not available or lack a sufficient image quality to create simulation models.
Table 6. Overview: Selection process for the diameter matched groups.

Detailed information for all AAAs of both groups is provided in Tables 7 and 8 and a visualization of their rupture risk indices, , in Figs 5 and 6 (cf. Appendix A.3). No patient had known connective tissue disorders. For 10 out of 18 AAAs in group 1 and for 9 out of 18 AAAs in group 2 we had non-invasive data and were thus able to use the multi-output regression model to determine a personalized input density p(logΘ). For the remaining 8 (group 1) and 9 (group 2) AAAs, we used cohort statistics, i.e. Model 1 from Section 3.2.

Table 7. Group 1 (asymptomatic, 18 ♂, 0 ♀) overview and obtained results for , RPI, PRRI and .

Table 8. Group 2 (symptomatic/ruptured, 13♂, 5 ♀) overview and obtained results for , RPI, PRRI and .

We apply our framework to all 36 AAAs using an individual prospective scenario, i.e. before starting the analysis for one AAA, this patient is removed from the database, while the other 35 AAAs are included. In order to provide a comparison of with other biomechanical indices, we calculate the following additional quantities:

  • Maximum von Mises stress at the input parameter mean (neglects any statistical information): (28)
  • Rupture potential index [28] at the input parameter mean (neglects any statistical information, but takes into account the wall strength): (29)
  • Probabilistic rupture risk index [21] (takes into account cohort-based uncertainties in the wall thickness and wall strength according to Model 1, Section 3.2): (30)

Comprehensive results for all patients are listed in Tables 7 and 8 (cf. Appendix A.3). The average number of high-fidelity model evaluations to train the Kriging surrogate was 11. Based on these results and to evaluate the performance of the individual quantities, we provide:

  1. Relative mean and median differences between group 1 and group 2 (cf. Table 9).
  2. Boxplots for both groups (cf. Fig 7).
  3. Receiver operating characteristic (ROC) curves and the area under the ROC curve (AUC) (cf. Fig 8) [45]. Computed true positive rates (TPR), false positive rates (FPR) and corresponding threshold values are provided for as supplementary information (cf. S4 Table).
Fig 7. Boxplots comparing dmax, , RPI, PRRI and for the asymptomatic and symptomatic/ruptured group.

The plots illustrate the interquartile range (green and red color) including the sample median as well as the first and third quartiles. Whiskers indicate minimum and maximum values and black dots represent all values from the respective group.

Table 9. Relative mean and median differences (in %) of dmax, , RPI, PRRI and between the asymptomatic and the symptomatic/ruptured group.

4 Discussion

The obtained values for the relative mean and median differences in Table 9 confirm that group 1 and group 2 are indistinguishable based on the maximum diameter criterion. While the relative differences are higher for and RPI, PRRI and in particular our proposed index feature a significantly larger mean and median difference. Recalling that the maximum diameter, dmax, is one important non-invasive parameter in our framework (cf. Section 2.2.3), we emphasize that its influence has been rendered ineffective through the study design. A similar trend as in Table 9 can be observed in Fig 7, with RPI and PRRI providing a slightly better separation between the two groups than , while for the interquartile ranges of the two groups are non-overlapping. Finally, in Fig 8 we can observe that outperforms the remaining classifiers and achieves the best performance among all quantities in terms of the AUC score, followed by PRRI, RPI and . We further note that from the 18 patients in the symptomatic/ruptured group, 11 had ruptured AAAs (Pat19, Pat23, Pat24, Pat26, Pat27, Pat28, Pat29, Pat30, Pat32, Pat34, Pat35). The mean scores for the 11 ruptured AAAs is 6.57 and thus slightly higher than the mean 5.89 for the 7 symptomatic AAAs. To summarize our key observations:

  • The maximum diameter criterion, by design, clearly fails to separate the two groups in all our comparisons.
  • The proposed index consistently achieves the best separation.
  • The results indicate that the more statistical information taken into account, the better the capability to distinguish between group 1 and group 2.
Fig 8. Receiver operating characteristic (ROC) curves showing true positive rates (TPR) over false positive rates (FPR) and area under the ROC curve (AUC) scores for dmax, , RPI, PRRI and .

Before translating these findings into any clinical application, however, there are several limitations that have to be kept in mind. First, this is a non-randomized, retrospective case-control study with a relatively small cohort size (group 1: n = 18, group 2: n = 18) and the database described in Section 2.2.3. Second, there was no matching based on other risk factors such as sex, age or family history, which could be a confounder. Third, since we only have access to electively repaired or symptomatic/ruptured AAAs for mechanical testing, the mean diameters of the two groups (group 1: 62.17 mm, group 2: 63.03 mm) are larger than the Society for Vascular Surgery’s decision criterion for elective repair (55 mm) [1]. In the future, due to the increasing use of EVAR, it will be even harder to obtain representative tissue samples from AAAs of relevant size for a database. As a result, caution is advised when interpreting the results presented here for smaller AAAs, e.g. of size 45 − 55 mm. Furthermore, all discussed approaches are unable to make any prediction about the future development of the AAA, such that the rupture risk assessment only holds for the point in time of data generation. In addition to that, the biomechanical model does not take into account factors like growth, calcifications and surrounding organs, which might be important for the analysis.

5 Conclusion

We presented a novel data-informed, highly personalized, probabilistic framework for the quantification of abdominal aortic aneurysm (AAA) rupture risk and demonstrated competitive performance in comparison to existing approaches. Our framework results in the calculation of a rupture risk index, , which can be introduced as a relevant additional piece of information in the clinical decision process for AAA cases that are not or not unambiguously covered by existing guidelines and recommendations. In view of our results it is suggested to incorporate personalized, or at least cohort-based, statistical information and choose a probabilistic approach for the biomechanical rupture risk assessment. Deterministic indices were shown to be less accurate and do not account for possible sensitivities due to uncertain vessel wall quantities.

In order to advance this framework to a clinical application, several further aspects need to be examined. Challenges lie especially in the fully automatic segmentation of the CT imaging data, which at the moment requires manual steps by a trained expert and can be time consuming. In view of the limitations discussed in Section 4, a larger, randomized study with risk factor matched groups is desirable to confirm this study’s findings regarding its clinical use. Future work will also address how further model parameters such as the blood pressure influence the rupture risk index and whether this quantity should be treated probabilistically as well. Lastly, to be able to make predictions over time, it is required to incorporate AAA growth [46, 47] into the framework and analyze its effect on the biomechanical rupture risk assessment.

A Appendix

A.1 Multi-output Gaussian process regression

The data generation process for logΘ is assumed to underly a function that is contaminated by additive Gaussian noise, such that (31)

It is further postulated that the vector follows a multivariate Gaussian distribution with the positive semi-definite covariance matrix and it is assumed that , with the diagonal matrix S and noise levels . Demanding that every entry of the vector corresponds to the same zero mean Gaussian process with covariance function k(ξ, ξ′), i.e. (32) can be expressed as a multivariate Gaussian process [39] (33)

As a result, the collection follows a matrix-variate Gaussian distribution (34) with the covariance matrix K and entries Kij = k(ξi, ξj), modeling the covariance between two inputs ξi and ξj. Expressing the matrix Gaussian distribution as a multivariate Gaussian distribution and incorporating the additive noise ϵ, one obtains (35) where ⊗ denotes the Kronecker product and the ndata × ndata identity matrix. For our purposes, we choose the covariance function (36) with hyperparameters ζ1, ζ2, ζ3 and ζ4. Following [3739], the matrix Ω is parameterized via the entries Lij of a Cholesky decomposition Ω = LLT. Together with the noise parameters from the matrix S, this results in the hyperparameter vector (37) where and . The predicted mean for an arbitrary point ξ becomes (38) and the predicted covariance (39) where k denotes the vector of covariance function evaluations between ξ and the data , i.e. (40)

Finally, the log marginal likelihood is (41) and can be optimized with respect to its hyperparameters ζ.

A.2 Kriging surrogate incorporating explicit basis functions

Kriging can be regarded as a special case of a Gaussian process, where data points are assumed noise-free to interpolate the high-fidelity model at the provided high-fidelity evaluations. To find an adequate function for this purpose, we use a Kriging interpolation model that incorporates explicit basis functions as described in [42]. Using such a model, it is possible to exactly represent functions that can be described by the provided basis. Ensuring positive predictions via a log transformation, we approximate the high-fidelity model as (42) where is a zero mean Gaussian process with covariance function k(θ, θ′) and h(θ) denotes the chosen basis functions with coefficients η. A simple squared exponential kernel (43) is chosen, where the matrix is diagonal, leading to the vector of hyperparameters ζ = [ζ1, ζ2, ζ3, ζ4]T, with . Furthermore, trilinear basis functions, i.e. (44) are employed. Assuming a Gaussian prior for the coefficients, , this results in the Gaussian process (45)

The dependence on the prior parameters b and B can be resolved, if a vague prior for η is chosen, i.e. if the limiting case is considered, where B−1 approaches the zero matrix 0. In that case, the predicted mean for a new point θ becomes (46) and the predicted variance (47) where , r = hHK−1 k and is the vector of neval model evaluations. K is the data covariance matrix between all training data points such that (48)

Moreover, k is a vector with the covariances between training and test points, i.e. (49) H a matrix containing vectors h(θ) at all training data points and h = h(θ). It is interesting to note, how the terms in Eqs (46) and (47) consist of a contribution from the zero mean Gaussian process predicted mean and variance, and , and additional terms involving the provided basis functions, respectively. Finally, the marginal log likelihood is (50) where A = HK−1 HT C = K−1 HT A−1 HK−1 and m is the rank of HT. Maximizing the marginal log likelihood, optimal values for the hyperparameters ζ of the Kriging covariance model (cf. Eq (43)) and for the provided and can be determined.

Supporting information

S1 Table. Correlation analysis.

Computed correlations between all non-invasive and invasive AAA vessel wall properties using Spearman’s rank correlation coefficient.


S2 Table. Non-invasive data.

Dataset containing the eight non-invasive AAA vessel wall properties: maximum AAA diameter, maximum thrombus thickness, AAA length, subrenal diameter, thrombocytes, hemoglobin, mean corpuscular hemoglobin (MCH), mean corpuscular volume (MCV). All values are provided in their respective unit as specified in Table 2. The first column is a consecutive number for the individual patients and allows for the identification of data that was collected from the same patient.


S3 Table. Invasive data.

Dataset containing the four invasive AAA vessel wall properties: wall thickness, alpha stiffness, beta stiffness, wall strength. All values are provided in their respective unit as specified in Table 1. The first column is a consecutive number for the individual patients and allows for the identification of data that was collected from the same patient.


S4 Table. ROC values.

Computed true positive rates (TPR), false positive rates (FPR) and corresponding threshold values for .



  1. 1. Chaikof EL, Dalman RL, Eskandari MK, Jackson BM, Lee WA, Mansour MA, et al. The Society for Vascular Surgery Practice Guidelines on the Care of Patients with an Abdominal Aortic Aneurysm. Journal of Vascular Surgery. 2018;67(1):2–77.e2. pmid:29268916
  2. 2. US Preventive Services Task Force, Owens DK, Davidson KW, Krist AH, Barry MJ, Cabana M, et al. Screening for Abdominal Aortic Aneurysm: US Preventive Services Task Force Recommendation Statement. JAMA. 2019;322(22):2211. pmid:31821437
  3. 3. Reite A, Søreide K, Kvaløy JT, Vetrhus M. Long-Term Outcomes After Open Repair for Ruptured Abdominal Aortic Aneurysm. World Journal of Surgery. 2020;44(6):2020–2027.
  4. 4. Trenner M, Haller B, Storck M, Reutersberg B, Kallmayer MA, Eckstein HH. Trends in Patient Safety of Intact Abdominal Aortic Aneurysm Repair: German Registry Data on 36,594 Procedures. European Journal of Vascular and Endovascular Surgery. 2017;53(5):641–647.
  5. 5. Bergqvist D, Björck M, Wanhainen A. Abdominal Aortic Aneurysm—To Screen or Not to Screen. European Journal of Vascular and Endovascular Surgery. 2008;35(1):13–18.
  6. 6. Spanos K, Eckstein HH, Giannoukas AD. Small Abdominal Aortic Aneurysms Are Not All the Same. Angiology. 2020;71(3):205–207.
  7. 7. Oliveira-Pinto J, Oliveira N, Bastos-Gonçalves F, Hoeks S, VAN Rijn MJ, Ten Raa S, et al. Long-Term Results of Outside “Instructions for Use” EVAR. The Journal of Cardiovascular Surgery. 2017;58(2):252–260. pmid:27998049
  8. 8. Antoniou GA, Antoniou SA, Torella F. Editor’s Choice—Endovascular vs. Open Repair for Abdominal Aortic Aneurysm: Systematic Review and Meta-Analysis of Updated Peri-Operative and Long Term Data of Randomised Controlled Trials. European Journal of Vascular and Endovascular Surgery. 2020;59(3):385–397.
  9. 9. Giannopoulos S, Kokkinidis DG, Armstrong EJ. Long Term Outcomes of Endovascular vs Open Surgical Repair for Abdominal Aortic Aneurysms: A Meta-Analysis of Randomized Trials. Cardiovascular Revascularization Medicine. 2020.
  10. 10. Gee MW, Reeps C, Eckstein HH, Wall WA. Prestressing in Finite Deformation Abdominal Aortic Aneurysm Simulation. Journal of Biomechanics. 2009;42(11):1732–1739.
  11. 11. Gee MW, Förster C, Wall WA. A Computational Strategy for Prestressing Patient-Specific Biomechanical Problems under Finite Deformation. International Journal for Numerical Methods in Biomedical Engineering. 2010;26(1):52–72.
  12. 12. Maier A. Computational Modeling of Rupture Risk in Abdominal Aortic Aneurysms. München: Dr. Hut; 2013.
  13. 13. Reeps C, Maier A, Pelisek J, Härtl F, Grabher-Meier V, Wall WA, et al. Measuring and Modeling Patient-Specific Distributions of Material Properties in Abdominal Aortic Aneurysm Wall. Biomechanics and Modeling in Mechanobiology. 2013;12(4):717–733. pmid:22955570
  14. 14. Biehler J, Gee MW, Wall WA. Towards Efficient Uncertainty Quantification in Complex and Large-Scale Biomechanical Problems Based on a Bayesian Multi-Fidelity Scheme. Biomechanics and Modeling in Mechanobiology. 2015;14(3):489–513.
  15. 15. Biehler J, Kehl S, Gee MW, Schmies F, Pelisek J, Maier A, et al. Probabilistic Noninvasive Prediction of Wall Properties of Abdominal Aortic Aneurysms Using Bayesian Regression. Biomechanics and Modeling in Mechanobiology. 2017;16(1):45–61. pmid:27260299
  16. 16. Biehler J, Wall WA. The Impact of Personalized Probabilistic Wall Thickness Models on Peak Wall Stress in Abdominal Aortic Aneurysms. International Journal for Numerical Methods in Biomedical Engineering. 2018;34(2):e2922.
  17. 17. Fillinger MF, Raghavan ML, Marra SP, Croonenwett JL, Kennedy FE. In Vivo Analysis of Mechanical Wall Stress and Abdominal Aortic Aneurysm Rupture Risk. Journal of Vascular Surgery. 2002;36:589–597.
  18. 18. Fillinger MF, Marra SP, Raghavan ML, Kennedy FE. Prediction of Rupture Risk in Abdominal Aortic Aneurysm during Observation: Wall Stress versus Diameter. Journal of Vascular Surgery. 2003;37:724–732.
  19. 19. Maier A, Gee MW, Reeps C, Pongratz J, Eckstein HH, Wall WA. A Comparison of Diameter, Wall Stress, and Rupture Potential Index for Abdominal Aortic Aneurysm Rupture Risk Prediction. Annals of Biomedical Engineering. 2010;38(10):3124–3134.
  20. 20. Gasser TC, Auer M, Labruto F, Swedenborg J, Roy J. Biomechanical Rupture Risk Assessment of Abdominal Aortic Aneurysms: Model Complexity versus Predictability of Finite Element Simulations. European Journal of Vascular and Endovascular Surgery. 2010;40(2):176–185.
  21. 21. Polzer S, Gasser TC. Biomechanical Rupture Risk Assessment of Abdominal Aortic Aneurysms Based on a Novel Probabilistic Rupture Risk Index. Journal of The Royal Society Interface. 2015;12(113):20150852.
  22. 22. Gasser TC. Biomechanical Rupture Risk Assessment: A Consistent and Objective Decision-Making Tool for Abdominal Aortic Aneurysm Patients. Aorta. 2016;4(2). pmid:27757402
  23. 23. Polzer S, Gasser TC, Vlachovský R, Kubíček L, Lambert L, Man V, et al. Biomechanical Indices Are More Sensitive than Diameter in Predicting Rupture of Asymptomatic Abdominal Aortic Aneurysms. Journal of Vascular Surgery. 2019;71(2):617–626.e6. pmid:31176634
  24. 24. Doyle BJ, Bappoo N, Syed MBJ, Forsythe RO, Powell JT, Conlisk N, et al. Biomechanical Assessment Predicts Aneurysm Related Events in Patients with Abdominal Aortic Aneurysm. European Journal of Vascular and Endovascular Surgery. 2020.
  25. 25. Vorp DA. Biomechanics of Abdominal Aortic Aneurysm. Journal of Biomechanics. 2007;40(9):1887–1902.
  26. 26. Vande Geest JP, Wang DHJ, Wisniewski SR, Makaroun MS, Vorp DA. Towards A Noninvasive Method for Determination of Patient-Specific Wall Strength Distribution in Abdominal Aortic Aneurysms. Annals of Biomedical Engineering. 2006;34(7):1098–1106.
  27. 27. Raghavan ML, Kratzberg J, Castro de Tolosa EM, Hanaoka MM, Walker P, da Silva ES. Regional Distribution of Wall Thickness and Failure Properties of Human Abdominal Aortic Aneurysm. Journal of Biomechanics. 2006;39(16):3010–3016.
  28. 28. Vande Geest JP, Di Martino ES, Bohra A, Makaroun MS, Vorp DA. A Biomechanics-Based Rupture Potential Index for Abdominal Aortic Aneurysm Risk Assessment: Demonstrative Application. Annals of the New York Academy of Sciences. 2006;1085(1):11–21.
  29. 29. Hemmler A, Lutz B, Reeps C, Kalender G, Gee MW. A Methodology for in Silico Endovascular Repair of Abdominal Aortic Aneurysms. Biomechanics and Modeling in Mechanobiology. 2018;17(4):1139–1164.
  30. 30. Reeps C, Gee M, Maier A, Gurdan M, Eckstein HH, Wall WA. The Impact of Model Assumptions on Results of Computational Mechanics in Abdominal Aortic Aneurysm. Journal of Vascular Surgery. 2010;51(3):679–688.
  31. 31. Joldes GR, Miller K, Wittek A, Doyle B. A Simple, Effective and Clinically Applicable Method to Compute Abdominal Aortic Aneurysm Wall Stress. Journal of the Mechanical Behavior of Biomedical Materials. 2016;58:139–148.
  32. 32. Zelaya JE, Goenezen S, Dargon PT, Azarbal AF, Rugonyi S. Improving the Efficiency of Abdominal Aortic Aneurysm Wall Stress Computations. PLoS ONE. 2014;9(7):e101353.
  33. 33. Moireau P, Xiao N, Astorino M, Figueroa CA, Chapelle D, Taylor CA, et al. External Tissue Support and Fluid–Structure Simulation in Blood Flows. Biomechanics and Modeling in Mechanobiology. 2012;11(1-2):1–18. pmid:21308393
  34. 34. Gasser TC, Görgülü G, Folkesson M, Swedenborg J. Failure Properties of Intraluminal Thrombus in Abdominal Aortic Aneurysm under Static and Pulsating Mechanical Loads. Journal of Vascular Surgery. 2008;48(1):179–188.
  35. 35. Raghavan ML, Webster MW, Vorp DA. Ex Vivo Biomechanical Behavior of Abdominal Aortic Aneurysm: Assessment Using a New Mathematical Model. Annals of Biomedical Engineering. 1996;24(5):573–582.
  36. 36. Raghavan ML, Vorp DA. Toward a Biomechanical Tool to Evaluate Rupture Potential of Abdominal Aortic Aneurysm: Identification of a Finite Strain Constitutive Model and Evaluation of Its Applicability. Journal of Biomechanics. 2000;33(4):475–482.
  37. 37. Bonilla EV, Chai KM, Williams C. Multi-Task Gaussian Process Prediction. In: Platt JC, Koller D, Singer Y, Roweis ST, editors. Advances in Neural Information Processing Systems 20. Curran Associates, Inc.; 2008. p. 153–160.
  38. 38. Wang B, Chen T. Gaussian Process Regression with Multiple Response Variables. Chemometrics and Intelligent Laboratory Systems. 2015;142:159–165.
  39. 39. Chen Z, Wang B, Gorban AN. Multivariate Gaussian and Student-t Process Regression for Multi-Output Prediction. Neural Computing and Applications. 2020;32(8):3005–3028.
  40. 40. Sacks J, Welch WJ, Mitchell TJ, Wynn HP. Design and Analysis of Computer Experiments. Statistical Science. 1989;4(4):409–423.
  41. 41. Kennedy MC, O’Hagan A. Predicting the Output from a Complex Computer Code When Fast Approximations Are Available. Biometrika. 2000;87(1):1–13.
  42. 42. Rasmussen CE, Williams CKI. Gaussian Processes for Machine Learning. Adaptive Computation and Machine Learning. Cambridge, Mass: MIT Press; 2006.
  43. 43. Gramacy RB, Lee HKH. Adaptive Design and Analysis of Supercomputer Experiments. Technometrics. 2009;51(2):130–145.
  44. 44. Matsushita M, Ikezawa T, Sugimoto M, Idetsu A. Management of Symptomatic Abdominal Aortic Aneurysms Following Emergency Computed Tomography. Surgery Today. 2014;44(4):620–625.
  45. 45. Fawcett T. An Introduction to ROC Analysis. Pattern Recognition Letters. 2006;27(8):861–874.
  46. 46. Stevens RRF, Grytsan A, Biasetti J, Roy J, Liljeqvist ML, Gasser TC. Biomechanical Changes during Abdominal Aortic Aneurysm Growth. PLOS ONE. 2017;12(11):e0187421.
  47. 47. Kehl S, Gee MW. Calibration of Parameters for Cardiovascular Models with Application to Arterial Growth. International Journal for Numerical Methods in Biomedical Engineering. 2016;33(5):e2822.