http://orcid.org/0000-0001-7564-4394
Figures
Abstract
In this paper we suggest a new Bayesian approach to network meta-analysis for the case of discrete multiple outcomes. The joint distribution of the discrete outcomes is modeled through a Gaussian copula with binomial marginals. The remaining elements of the hierarchial random effects model are specified in a standard way, with the logit of the success probabilities given by the sum of a baseline log-odds and random effects comparing the log-odds of each treatment against the reference and having a Gaussian distribution centered at the vector of pooled effects. An adaptive Markov Chain Monte Carlo algorithm is devised for running posterior inference. The model is applied to two datasets from Cochrane reviews, already analysed in two papers so to assess and compare its performance. We implemented the model in a freely available R package called netcopula.
Citation: Graziani R, Venturini S (2020) A Bayesian approach to discrete multiple outcome network meta-analysis. PLoS ONE 15(4): e0231876. https://doi.org/10.1371/journal.pone.0231876
Editor: Xiaofeng Wang, Cleveland Clinic Lerner Research Institute, UNITED STATES
Received: August 15, 2019; Accepted: April 2, 2020; Published: April 28, 2020
Copyright: © 2020 Graziani, Venturini. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data are available in the directory \data of the netcopula package that is freely downloadable from the following repository: https://github.com/sergioventurini/netcopula/tree/master/data.
Funding: Rebecca Graziani has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement no. 694262), project DisCont - Discontinuities in Household and Family Formation, PI F.C. Billari. Sergio Venturini has received funding from the Management Department of Università degli Studi di Torino, under the project “CULF_AUTOF_18_01 - Progetto di ricerca dipartimentale_2019"
Competing interests: The authors have declared that no competing interests exist.
Introduction
In the last decades, as the need of evidence based techniques in medical research and clinical practice has been more and more recognized, the use of meta-analysis, introduced with a high level of debate, has become widespread. Nowadays, areas of application of meta-analysis extend beyond medicine and health, being widely used in both natural and social sciences. See [1] for a critical review of the main methodological developments in meta-analysis. Traditionally, meta-analytic techniques make it possible to summarize evidence provided by several studies comparing the same treatments and considering in general one outcome at time. The basic methods combine study-specific treatment effect estimates under a fixed effect or a random effect model (see [2] and [3]). Study specific covariates and individual patients data can be incorporated as well (see for IPD meta-analysis among others [4–6] and [7]). Bayesian methods are widely used, making it possible to allow for all parameter uncertainty in the model, to include all relevant information and to extend the models to accommodate more complex scenarios. Advantages of the Bayesian approach are discussed and reviewed in several papers and books, see among others [8] and [9].
Meta-analytic techniques have been developed in the recent years along several directions. In the present work, we focus on one of the most recent development, multiple outcomes network meta-analysis. Network meta-analysis in the past few years has become increasingly popular and its advantages and disadvantages are discussed in several published articles, see among others [10–14] and [15]. In particular [16] provide a complete and detailed review of network meta-analysis techniques. Network meta-analysis makes it possible to combine both direct and indirect evidence provided by different studies with respect to different treatments. The main challenge is to use the totality of trial evidence to determine an internally consistent set of estimated treatment effects between all treatments, while respecting randomization. As long as the included trials and treatments form a connected network, network meta-analysis allows to borrow strength across treatments in the estimation of relative effect sizes. Full-fledged ranking of all considered treatments can be also obtained. Several methods for running univariate network meta-analysis have been proposed and coded in statistical packages. However, there are relatively few attempts to extend these methods to the multivariate setting.
The need of multiple outcomes network meta-analytic techniques stems from the fact that often studies report several outcomes, that in general are correlated. Such correlation arises when several outcomes are measured on the same participants, when one event is nested in another (as in the case of disease survival nested in total survival) or when outcomes are measured repeatedly on the same participants. The outcomes correlation entails a correlation of the treatments effect, which is clearly neglected if separate univariate network meta-analyses are run for each outcome. Multiple outcomes network meta-analysis makes it possible to account for such within-study correlation of the treatment effects so to simultaneously borrow strength across treatments and outcomes. In this way, more studies contribute towards each outcome and treatment comparison. Indeed, summary results for each outcome depend on correlated results from other outcomes, and summary results for each treatment comparison incorporate indirect evidence from related treatment comparisons, in addition to any direct evidence.
Multiple outcomes network meta-analysis faces the same issues that multiple outcomes meta-analysis addresses, but in a more complicated setting, see [17] and [18] for an overview of advantages and disadvantages of multiple outcomes techniques. We can identify two main challenges. The first stems from the fact that while studies report estimates and standard errors of the treatments relative effect for each outcome, rarely the corresponding covariance matrix is provided. In multiple outcomes meta-analysis, a common choice is to assume that the within-study covariance is known and to focus on the estimation of the between-study covariance matrix. In a frequentist approach the entries of such matrix are estimated resorting to maximum likelihood techniques as in [19] or restricted maximum likelihood techniques as in [20]. Method of moments techniques are used as well, [21] and [22] suggest a multivariate generalization of the DerSimonian and Laird’s methodology. [23] suggest a structural equation modeling approach (see [24] for a review on the use of such approach in meta-analysis). The within-study correlation coefficients are then imputed on the basis of individual patients data, when available for similar studies. When individual patients data are not available, plausible values can be assumed on the basis of clinical considerations as in [25]. As well empirical correlations can be used as in [20]. [26] use delta methods to approximate the within-study correlation on the basis of information on the outcomes correlation. [27] suggest a Bayesian approach to multiple outcome meta-analysis, where the within study variances are assumed to be known and simplifying assumptions are made on the correlations so to reduce the number of parameters. Noninformative priors are assigned to such correlations. [28] use external sources of information to construct informative priors for both within-study correlations and the between-study covariance matrix.
The second issue to be addressed in a multivariate generalization of network meta-analytic techniques comes from the fact that not all studies report results on all the considered outcomes. In multiple outcomes meta-analysis, the missing outcomes issue is addressed relying on traditional imputation and data augmentation techniques both in a frequentist and a Bayesian approach ([29, 30]). In a Bayesian approach [26] express the likelihood as a product of marginal distributions over reported outcomes following the approach suggested by Glester and Olkin in [31]. Such techniques make it possible to borrow strength across outcomes and this, as pointed out by [32], reduces the impact of a selective non reporting of the outcomes on the pooled treatment effect estimates.
In this work we adopt a contrast-based perspective to estimate the treatment effects in a network meta-analysis, see [33, 34]. Contrast-based models currently represent the most popular methodology in the network meta-analysis literature. However, another approach, called arm-based, has also been recently advanced (see [35–38]). In contrast-based models, a baseline treatment is defined for each study and the focus of the analysis is on the estimation of the relative treatment effects (for example using log odds ratios, or another suitable metric). In this context, the baseline effects are treated as nuisance parameters and they are usually modeled with noninformative prior distributions. This implies that absolute treatment effects cannot be directly obtained unless a reference treatment absolute effect is first estimated using information that are external to the model see among others [39]. On the contrary, arm-based models aim to model the absolute effect of each treatment in a study (for example using the log odds) and the relative treatment effects are then constructed from the arm estimates. Contrast-based models are usually advocated as more theoretically grounded compared to the arm-based approach because the latter discards the randomization structure of the evidence. Moreover, arm-based models are more likely to provide biased estimates of relative treatment effects with increased posterior variances, and they often show a slower convergence especially when some treatments are only included in few studies. On the contrary, arm-based models are more advantageous because they can also incorporate the information provided by single-arm studies. For more details on the pros and cons of the two approaches see [40] and the discussion rejoinder by [41], while for a more technical comparison between arm-based and contrast-based models for network meta-analysis we suggest the recent work by [42].
In this paper we present a new approach to network meta-analysis in the case of discrete multiple outcomes. The model we suggest is a Bayesian hierarchical random effects model that is based on a Gaussian copula likelihood, which allows to incorporate the estimation of within-study variances and correlations. Our approach draws on and generalizes the method suggested by [43] based on a Clayton copula model. However the switch from a Clayton copula model to a Gaussian copula model is not straightforward due to the implications in terms of computational problems to be addressed. Indeed in the case of Clayton copula model the correlation between outcomes is modeled through a single univariate parameter, while in the case of a Gaussian copula model the association is modeled through a correlation matrix. Posterior inferences are based on a latent variables adaptive Markov Chain Monte Carlo algorithm, that draws on the suggestions by [44] and [45] for copula regression models and by [46] for the simulation of correlation matrices. The uncertainty due to the missingness problem is addressed and accounted for through a posterior based imputation of missing outcomes at each stage of the algorithm. All codes, data and examples are available in a R package called netcopula, that can be freely downloaded from the following public repository https://github.com/sergioventurini/netcopula.
The set-up of the paper is as follows. Section 2 provides a description of the suggested model and of the MCMC algorithm devised for running posterior inferences. In Section 3 two applications of the model are provided with a comparison with other two different approaches. Section 4 provides a discussion with final conclusions.
Method
Introduction
Copula models have become widely used in all applied fields since they make it possible to split the specification of a multivariate model into two parts: the marginal distributions on one side and the dependence structure on the other side. In this way, any univariate distribution can be used for modelling the marginal behavior of the considered variables, which can be discrete and continuous. Moreover, marginal distributions belonging to different families can be selected, ensuring a higher flexibility with respect to a traditional modeling with multivariate distributions. The dependency across the variables is then modelled through a copula function that “glues together” the marginal distributions.
In the following, we briefly review the copula based approach for the case of two variables, Y1 and Y2, with marginal cumulative distributions F1 and F2 respectively. We want to obtain a bivariate distribution for the vector (Y1, Y2) having these two margins. Sklar ([47]) proved that we can always find a function C such that
(1)
where C(y1, y2) is the joint distribution function for a pair of bivariate uniform random variables. Sklar called C copula function and showed three relevant properties (see [48] for a detailed introduction to copula models). The distribution in Eq (1) is constructed from the marginal distributions F1 and F2, while the role of the copula function is to determine the dependence between Y1 and Y2. If the marginal distributions are continuous, differentiating Eq (1) gives the joint density
(2)
where c(F1(y1), F2(y2)) is the copula density. Eq (2) shows that the copula density controls the level of dependence between Y1 and Y2. The copula function does not determine the distribution of the margins. It merely determines the dependence between the two random variables. There are many copula functions, one of the most popular is the Gaussian copula. In the continuous case a useful way to think at the copula method is that, based on the probability integral transformation on each margin, the original variables are each transformed into uniform random variables Uj = Fj(Yj). Indeed no matter is the marginal distribution Fj, Uj has a uniform distribution and the dependency between the original variables carries through to the transformed uniform distributions. In this way, assuming a copula model as in Eq (1) for the pair (Y1, Y2) reduces to considering the following model
In the case of discrete marginal distributions, the marginal are steps function, we define Uj as associated with the variable Yj through the following inequality
Nevertheless this still ensures Yj is uniformly distributed in the interval (0, 1). The method can be easily generalized to the case of more than two variables.
Model specification
We consider a sample of n multi-arm randomized trials and let yik denote the vector of the number of times each of M outcomes is observed in study i for treatment k, that is yik = (yik1, …, yikM)⊤, , where
is the set of treatments compared in study i, with the treatment labelled as 1 representing the control (i.e. the baseline) treatment in study i whose efficacy is compared with that of the remaining (ai − 1) treatments. Note that the term “treatment 1” may refer to distinct treatments in the different studies. We suggest to model yik as the realization of a multivariate discrete random variable Yik, i = 1, …, n and
, with distribution built from a Gaussian copula with binomial margins. We assume that for each study i, each arm k and each outcome m
where
is the inverse cumulative distribution function of a binomial random variable with parameters nik and pikm, nik is the number of patients randomized to arm k in study i, pikm the treatment-specific probability of an outcome of type m in study i and xikm is the m-th component of vector xik, that is the realization of a random vector having a multivariate Gaussian distribution with a arm-specific correlation matrix, Γk. As previously emphasized, Fikm as the cumulative distribution function of a discrete random variable is a step function, therefore its inverse is a many-to-one function. This indeed complicates the calculations as compared to the continuous case (see [49]). The variables xikm are latent, not observed variables to be associated with each yikm. The logistic transformations of the treatment-specific probabilities
are modelled for
, as follows
where, μi = (μi1, …, μiM)⊤ denotes the vector of study-specific baseline effects and δik = (δik1, …, δikM)⊤ indicates the trial-specific log-odds ratios of treatment
relative to the baseline treatment (i.e. treatment 1) in study i.
Finally, we assume that the random effects have a multivariate normal distribution
(3)
where ti1 denotes the baseline treatment in study i, tij is the j-th treatment compared in study i and dj,k = (dj,k,1, …, dj,k,M)⊤ represents the vector of pooled effects (across trials) of treatment k relative to treatment j. The dj,k are usually the main quantities of interest in a meta-analysis. The consistency equations
(4)
where r identifies a treatment chosen as reference, ensure that the correct treatment comparison is used in the network meta-analysis (see [33] and [50]). Note that to guarantee consistency, it is also required that dr,r,1 = ⋯ = dr,r, M = 0.
The matrix Σ in Eq (3) contains the variances of the random effect δi,k, j for each treatment k ∈ 2, …, ai and each outcome m = 1…, M, and all possible covariances between any two random effects. As the pooled treatment effects, it is common to all studies and for this reason commonly referred to as a matrix that defines the between-study covariance structure, in opposition to the Γi that models the within-study correlation structure across outcomes. To keep the number of parameters manageable and to allow identifiability of Σ, we follow [51]and make the following simplifying assumption
where
describes the common between-study covariance structure. In this way, we assume that the variances and covariances of the random effects for each treatment comparison are the same and differ only with respect to the considered outcome. Moreover, such homogeneity assumption along with the structural relationships between the dj,k within trial i imply that the between-arm correlations are assumed to be all equal to 0.5 (see [52]). These correlations between the treatment differences come from the fact that all differences are taken relative to the same control arm, that is, they depend on the same trial baseline effect μi.
Priors choice
As for the prior assignment, proper priors are selected and we specify them in the application so to be vague. The study-specific baseline effects μi = (μi1, …, μiM) are assumed to be independent and distributed according to a normal distribution with mean zero and variance . As well the pooled (across trial effects) dr,q = (dr,q,1, …, dr,q, M) are assumed to be independent and identically distributed according to a normal distribution with mean zero and variance
.
As for the matrix ΣM, it has been shown in the literature that a standard conjugate Wishart prior is overly influential on the corresponding posterior distribution (see among others [26, 53] and [54]). Moreover, explicitly representing an informative prior distribution for a covariance matrix is difficult. We therefore follow an alternative strategy by adopting a log-Cholesky parameterization (see [55]) for the precision matrix . More specifically, we define
, with R = {rm,p} being an upper triangular Cholesky factor, with p = 1, …, M, m ≤ p. To guarantee that the Cholesky factorization is unique, one has to require the diagonal elements of R to be positive. To avoid constrained estimation, we use the logarithms of the diagonal elements of R. Hence, the covariance matrix is parameterized in terms of the parameter vector
(5)
Finally, we assume that the components of β, β ℓ with ℓ = 1, …, M(M+ 1)/2, are a priori independent and all distributed according to a normal distribution with zero mean and variance .
The correlation matrices Γq, with q ∈ T are assumed to be independent and uniformly distributed on the space of all correlation matrices.
Posterior computations
Fig 1 summarizes the formulation of our hierarchical model. Since the joint posterior of the model parameters cannot be obtained in closed form, we devise a latent variables and adaptive Markov Chain Monte Carlo (MCMC) algorithm for the posterior inference. At each iteration, for each unit i (i.e. each study), two sets of latent variables are introduced: the random effects and the latent vectors related to the specification of the copula model, xik, i = 1, …, N. Drawing on [45] and [44], we suggest to jointly update all latent variables and the baseline vectors μi. This is the most delicate step of the algorithm. An adaptive metropolis step is as well foreseen for the updates of the random effects and the baseline log-odds.
The Figure depicts the elements of the suggested hierachical model.
The updates of the copula parameters Γq are obtained by applying the two-stage parameter expanded reparameterization and Metropolis-Hastings (PX-RPMH) algorithm for simulating a correlation matrix proposed by [46]. Finally, the full conditionals for d and ΣM are obtained from standard results for Bayesian analysis of multiple regression models, an adaptive metropolis step is used for their simulation.
It is worth emphasizing here that the algorithm also allows for the possibility that the outcomes are reported differently in the studies. In this situation, a simple strategy one can implement consists in analyzing only the subset of events reported by all studies. Even if this suggestion allows to bypass the problem, the risk is that a considerable amount of data may be discarded. In our approach missing data are imputed at each iteration of the algorithm.
A detailed description of devised MCMC algorithm is provided in the Appendix.
Results and discussion
We apply the suggested model to two datasets from two Cochrane reviews. The two datasets have been analysed based on two different models, so that the performance of our model can be assessed and compared against two different approaches. In both cases we assume that the studies share the same Gaussian copula correlation matrix Γ and diffuse priors are chosen for all parameters by setting in particular and
equal to 103 and
equal to 101/8. In both cases not all outcomes are investigated in all studies. Missing outcomes value yikm are imputed at each iteration of the algorithm, by imputing the corresponding latent variable value xikm, from the model predictive distribution. The results come from a long run of the devised algorithm with 350000 iterations of which 300000 are discarded as burn-in. In both cases the convergence of the algorithm is assessed through the R coda package. Both examples can be reproduced since the corresponding scripts are provided as demos.
Home safety
In the first example we consider the data from a subset of a Cochrane review of safety education and provision of safety equipment for injury prevention, see [56] for a description of the methods. The focus is on the evidence relating to the prevention of poisoning injuries. Data come from twenty-two studies on three outcomes are recorded: Medicines Safe storage, Household Products Safe Storage and Poison Center Number Possession. Nine treatments are considered: Usual Care (UC), Education (EDU), Education + Provision of free/low cost equipment (EDU+FE), Education + Provision of free/low cost equipment + Fitting of Equipment (EDU+FE+F), Education + Home Safety Inspection (EDU+HSI), Education + Provision of free/low cost equipment + Home Safety Inspection Fitting of Equipment (EDU+FE+HSI+F), Education + Home Visit (EDU + HV), Provision of Free/Low Cost Equipment (FE). Overall there are three studies considering all three outcomes, nineteen studies considering two outcomes. All studies but one are two arms. Fig 2 depicts the network graph for the three outcomes.
The figure shows the network structure: A: Medicines Safe storage, B: Household Products Safe Storage and C: Poison Center Number Possession. The thicker the lines the higher the number of studies reporting results on the considered outcome.
[51] analyse the same dataset, based on a different approach. Indeed, [51] model the outcomes log odds ratio so that a continuous, Gaussian, multivariate distribution can be used as likelihood. Moreover, in [51] the within-study covariances are taken as known, as they are estimated from the data. In our approach, such matrices are assumed to be unknown so that the model provides as well an estimate of them. The analysis can be replicated by running the script example_homesafety.R to be found in the folder demo of the netcopula package. Fig 3 displays the trace plots of the pooled treatment effects against the baseline Usual Care.
The figure shows the trace plots of the pooled treatments effect against the baseline treatment Usual Care, after discarding the burnin.
Tables 1 to 3 display median estimates along with the highest posterior density (HPD) credibility intervals for the pooled effects estimated according to our model and according to [51], referred to as Model 3 in the original paper. Usual care is taken as baseline effect. In all cases our estimates show a smaller variability, in particular in the estimation of the pooled effect of Education + Home Visit versus Usual Care, of Education + Home Safety Inspection versus Usual Care and of Provision of Free/Low Cost Equipment versus Usual Care. The EDU+HV and EDU+HSI are indeed considered only for one outcome and the effect directly compared against Usual Care. The FE treatment is considered for two outcomes and only indirectly compared against Usual Care. The smaller uncertainty of the median estimates for such treatments shows that our model succeeds in borrowing strength across treatments and outcomes as foreseen. There are differences in the estimates of the pooled-effects for some treatment comparisons. In almost all cases our estimates belong to the corresponding credibility intervals in [51]. A simulation study (not reported here but available as a further demo in the netcopula R package) shows that, for different settings of the true parameters value, our model is able to recover the true values of the pooled effects.
Table 4 reports median estimates and highest posterior density (HPD) credibility intervals for the between-study standard deviations and correlations. Again, the estimates produced by the fit of our model show a smaller variability especially in the estimation of the within-study correlations.
Alcohol dependence
In the second example, the data come from a Cochrane systematic review of pharmacology treatments for alcohol dependency. See [57, 58] for a detailed description of the methods and [59] for an update. The same data are also analysed in [43]. In particular, the authors model the outcome correlations resorting to a Clayton copula model, with one single parameter then fine-tuning such correlations. Moreover they allow for heterogeneity of the random effects. Again, our analysis can be replicated by running the script example_alcoholdependence.R to be found in the folder demo of the netcopula package.
The data come from forty-one studies and three outcomes are considered: Return to Heavy Drinking, Return to Drinking and Discontinuation. Eleven studies consider all three outcomes, twenty-two studies consider two outcomes and height report results only on one outcome. Three treatments are considered: naltrexone (NAL), acamprosate (ACA) and naltrexone + acamprosate (NAL+ACA). Fig 4 depicts the network graph for the three outcomes.
The figure shows the network structure: A: Return to Heavy Drinking, B: Return to Drinking and C: Discontinuation. The thicker the lines the higher the number of studies reporting results on the considered outcome.
Tables 5 to 7 reports median estimates along HPD credibility intervals of the treatments pool effects obtained according to our model and [43] model. We can see that in all cases, our estimates show a smaller variability. Table 8 reports median estimates and credibility intervals for the within outcome correlations. Our model estimates a positive weak association between Return to Drinking and Discontinuation, even if there is high uncertainty on the estimation of the correlation between Return to Heavy Drinking and Return to Drinking and Return to Heavy Drinking and Discontinuation.
Conclusion
In this paper we suggest a new model for a multiple outcomes network meta-analysis in the case of discrete outcomes. Our model accounts for both correlation between outcomes and between treatments. Moreover, we deal with the case of missing at random outcomes. In a comparison with two approaches previously proposed in the literature our results show a lower uncertainty. The model we suggest can be extended to the case of outcomes of different kinds, both discrete and continuous. We use here the Gaussian copula, but the model can be easily modified to include a different kind of copula.
However, there are some limitations in the suggested approach. The first problem arises in the estimation of the copula correlation matrix. The algorithm should be improved in order to reduce the running time and the uncertainty of the derived estimates. In the proposed approach the correlation matrix is assumed to be unstructured. This in the case of high-dimensional outcome might slow down the convergence of the algorithm. In this case a parametrization of the correlation matrix might be a reasonable choice as investigated in [60] and [61]. Moreover the performance of the model is particularly affected by the number of studies that in general is not high. More specifically, our model includes three orders of latent variables, the variables for the copula, the random effects and the latent variables to be introduced in the imputation step. The smaller number of studies considered in the first example (22) compared those in the second example (41) is in our opinions at the basis of the higher uncertainty in the estimation of the correlation parameters.
Supporting information
S1 Appendix. In the following we provide a description of the algorithm used for fitting the multiple outcome network meta-analysis model suggested.
For the notation, we refer to the model depicted in Fig 1.
https://doi.org/10.1371/journal.pone.0231876.s001
(PDF)
Acknowledgments
The authors would like to acknowledge two anonymous referees for the extremely useful comments and suggestions.
References
- 1. Sutton AJ, Higgins JP. Recent developments in meta-analysis. Statistics in Medicine. 2008;27(5):625–650. pmid:17590884
- 2. DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials. 1986;7(3):177–188. pmid:3802833
- 3. DerSimonian R, Kacker R. Random-effects model for meta-analysis of clinical trials: an update. Contemporary Clinical Trials. 2007;28(2):105–114. pmid:16807131
- 4. Stewart LA, CWGOMAUIP. Practical methodology of meta-analyses (overviews) using updated individual patient data. Statistics in Medicine. 1995;14(19):2057–2079. pmid:8552887
- 5. Higgins JP, Whitehead A, Turner RM, Omar RZ, Thompson SG. Meta-analysis of continuous outcome data from individual patients. Statistics in Medicine. 2001;20(15):2219–2241. pmid:11468761
- 6. Whitehead A, Omar RZ, Higgins JP, Savaluny E, Turner RM, Thompson SG. Meta-analysis of ordinal outcomes using individual patient data. Statistics in Medicine. 2001;20(15):2243–2260. pmid:11468762
- 7. Simmonds MC, Higgins JP, Stewart LA, Tierney JF, Clarke MJ, Thompson SG. Meta-analysis of individual patient data from randomized trials: a review of methods used in practice. Clinical Trials. 2005;2(3):209–217. pmid:16279144
- 8. Sutton AJ, Abrams KR. Bayesian methods in meta-analysis and evidence synthesis. Statistical Methods in Medical Research. 2001;10(4):277–303. pmid:11491414
- 9.
Spiegelhalter DJ, Abrams KR, Myles JP. Bayesian Approaches to Clinical Trials and Health-Care Evaluation. Wiley; 2004.
- 10. Cipriani A, Higgins JP, Geddes JR, Salanti G. Conceptual and technical challenges in network meta-analysis. Annals of Internal Medicine. 2013;159(2):130–137. pmid:23856683
- 11. Cooper NJ, Peters J, Lai MC, Juni P, Wandel S, Palmer S, et al. How valuable are multiple treatment comparison methods in evidence-based health-care evaluation? Value in Health. 2011;14(2):371–380. pmid:21296599
- 12. Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Research Synthesis Methods. 2012;3(2):80–97. pmid:26062083
- 13. Sutton A, Ades A, Cooper N, Abrams K. Use of indirect and mixed treatment comparisons for technology assessment. Pharmacoeconomics. 2008;26(9):753–767. pmid:18767896
- 14. Dias S, Sutton AJ, Ades A, Welton NJ. Evidence synthesis for decision making 2: a generalized linear modeling framework for pairwise and network meta-analysis of randomized controlled trials. Medical Decision Making. 2013;33(5):607–617. pmid:23104435
- 15. Ioannidis JP. Indirect comparisons: the mesh and mess of clinical trials. The Lancet. 2006;368(9546):1470–1472.
- 16. Efthimiou O, Debray TP, van Valkenhoef G, Trelle S, Panayidou K, Moons KG, et al. GetReal in network meta-analysis: a review of the methodology. Research Synthesis Methods. 2016;7(3):236–263. pmid:26754852
- 17. Jackson D, Riley R, White IR. Multivariate meta-analysis: potential and promise. Statistics in medicine. 2011;30(20):2481–2498. pmid:21268052
- 18. Mavridis D, Salanti G. A practical introduction to multivariate meta-analysis. Statistical Methods in Medical Research. 2013;22(2):133–158. pmid:22275379
- 19. Hardy RJ, Thomson SG. A likelihood approach to meta-analysis with random effects. Statistics in Medicine. 1996;15(6):619–629. pmid:8731004
- 20. Van Houwelingen HC, Arends LR, Stijnen T. Advanced methods in meta-analysis: multivariate approach and meta-regression. Statistics in Medicine. 2002;21(4):589–624. pmid:11836738
- 21. Jackson D, White IR, Thompson SG. Extending DerSimonian and Laird’s methodology to perform multivariate random effects meta-analyses. Statistics in Medicine. 2010;29(12):1282–1297. pmid:19408255
- 22. Chen H, Manning AK, Dupuis J. A method of moments estimator for random effect multivariate meta-analysis. Biometrics. 2012;68(4):1278–1284. pmid:22551393
- 23. Cheung MWL. Multivariate meta-analysis as structural equation models. Structural Equation Modeling: A Multidisciplinary Journal. 2013;20(3):429–454.
- 24. Cheung MWL, Cheung SF. Random-effects models for meta-analytic structural equation modeling: review, issues, and illustrations. Research Synthesis Methods. 2016;7(2):140–155. pmid:27286900
- 25. Riley RD, Abrams KR, Sutton AJ, Lambert PC, Thompson JR. Bivariate random-effects meta-analysis and the estimation of between-study correlation. BMC Medical Research Methodology. 2007;7(1):3. pmid:17222330
- 26. Wei Y, Higgins JPT. Bayesian multivariate meta-analysis with multiple outcomes. Statistics in Medicine. 2013;32(17):2911–2934. pmid:23386217
- 27. Nam IS, Mengersen K, Garthwaite P. Multivariate meta-analysis. Statistics in Medicine. 2003;22(14):2309–2333. pmid:12854095
- 28. Bujkiewicz S, Thompson JR, Sutton AJ, Cooper NJ, Harrison MJ, Symmons DP, et al. Multivariate meta-analysis of mixed outcomes: a Bayesian approach. Statistics in Medicine. 2013;32(22):3926–3943. pmid:23630081
- 29. Dempster AP, Laird NM, Rubin DB. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society Series B (methodological). 1977; p. 1–38.
- 30. Tanner MA, Wong WH. The calculation of posterior distributions by data augmentation. Journal of the American statistical Association. 1987;82(398):528–540.
- 31.
Cooper H, Hedges LV, Valentine JC. The handbook of research synthesis and meta-analysis. Russell Sage Foundation; 2009.
- 32. Kirkham JJ, Riley RD, Williamson PR. A multivariate meta-analysis approach for reducing the impact of outcome reporting bias in systematic reviews. Statistics in Medicine. 2012;31(20):2179–2195. pmid:22532016
- 33. Lu GB, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Statistics in Medicine. 2004;23(20):3105–3124. pmid:15449338
- 34. Lu GB, Ades AE. Modeling between-trial variance structure in mixed treatment comparisons. Biostatistics. 2009;10(4):792–805. pmid:19687150
- 35. Zhang J, Carlin BP, Neaton JD, Soon GG, Nie L, Kane R, et al. Network meta-analysis of randomized clinical trials: reporting the proper summaries. Clinical Trials. 2014;11(2):246–262. pmid:24096635
- 36. Zhang J, Yuan Y, Chu H. The impact of excluding trials from network meta-analyses—an empirical study. PLoS ONE. 2016;11(12):e0165889. pmid:27926924
- 37. Hong H, Chu H, Zhang J, Carlin BP. A Bayesian missing data framework for generalized multiple outcome mixed treatment comparisons. Research Synthesis Methods. 2016;7(1):6–22. pmid:26536149
- 38. Lin L, Zhang J, Hodges J, Chu H. Performing arm-based network meta-analysis in R with the pcnetmeta package. Journal of Statistical Software. 2017;80(5):1–25.
- 39.
Welton NJ, Sutton AJ, Cooper NJ, Abrams KR, Ades AE. Evidence Synthesis for Decision Making in Healthcare. Wiley; 2012.
- 40. Dias S, Ades AE. Absolute or relative effects? Arm-based synthesis of trial data. Research Synthesis Methods. 2016;7(1):23–28. pmid:26461457
- 41. Hong H, Chu H, Zhang J, Carlin BP. Rejoinder to the discussion of “a Bayesian missing data framework for generalized multiple outcome mixed Treatment comparisons”, by S. Dias and A. E. Ades. Research Synthesis Methods. 2016;7(1):29–33. pmid:26461816
- 42. White IR, Turner RM, Karahalios A, Salanti G. A comparison of arm-based and contrast-based models for network meta-analysis. Statistics in Medicine. 2019;38(17):5197–5213. pmid:31583750
- 43. Liu Y, DeSantis SM, Chen Y. Bayesian mixed treatment comparisons meta-analysis for correlated outcomes subject to reporting bias. Journal of the Royal Statistical Society: Series C (Applied Statistics). 2018;67(1):127–144.
- 44. Pitt M, David C, Kohn R. Efficient Bayesian inference for Gaussian copula regression models. Biometrika. 2006;93(3):537–554.
- 45. Smith MS, Khaled MA. Estimation of Copula Models With Discrete Margins via Bayesian Data Augmentation. Journal of the American Statistical Association. 2012;107(497):290–303.
- 46. Liu X, Daniels MJ. A New Algorithm for Simulating a Correlation Matrix Based on Parameter Expansion and Reparameterization. Journal of Computational and Graphical Statistics. 2006;15(4):897–914.
- 47. Sklar A. Fonctions de répartition à n dimensions et leurs marges. Publication the l’Institut de Statistiques de l’Université de Paris 8. 1959;229–231.
- 48.
Nelsen R.B. An Introduction to Copulas. Second Edition. Springer, New York. 2006.
- 49.
Smith MS. Bayesian Approaches to Copula Modelling. In: Damien P, Dellaportas P, Polson NG, Stephens DA, editors. Bayesian Theory and Applications. Oxford University Press; 2013.
- 50.
Dias S, Welton NJ, Sutton AJ, Ades AE. A Generalised Linear Modelling Framework for Pairwise and Network Meta-analysis of Randomised Controlled Trials. NICE DSU; 2011.
- 51. Achana FA, Cooper NJ, Bujkiewicz S, Hubbard SJ, Kendrick D, Jones DR, et al. Network meta-analysis of multiple outcome measures accounting for borrowing of information across outcomes. BMC Medical Research Methodology. 2014;14(92):1–16.
- 52. Higgins JP, Whitehead A. Borrowing strength from external trials in a meta-analysis. Statistics in Medicine. 1996;15(24):2733–2749. pmid:8981683
- 53. Leonard T, Hsu JS. Bayesian inference for a covariance matrix. Annals of Statistics. 1992;20(4):1669–1696.
- 54. Danaher PJ, Smith MS. Modeling Multivariate Distributions Using Copulas: Applications in Marketing. Marketing Science. 2011;30(1):4–21.
- 55. Pinheiro JC, Bates DM. Unconstrained parameterisations for variance–covariance matrices. Statistics in Computing. 1996;6(3):289–296.
- 56. Kendrick D, Coupland C, Mulvaney C, Simpson J, Smith S, Sutton A, et al. Home safety education and provision of safety equipment for injury prevention. Cochrane Database Syst Rev. 2007;1(1):197–204.
- 57. Srisurapanont M, Jarusuraisin N. Naltrexone for the treatment of alcoholism: a meta-analysis of randomized controlled trials. International Journal of Neuropsychopharmacology. 2005;8(2):267–280. pmid:15850502
- 58. Rosner S, Hackl-Herrwerth A, Leucht S, Lehert P, Vecchi S. Soyka. M. (2010). Acamprosate for alcohol dependence. Cochrane Database of Systematic Reviews;9.
- 59. DeSantis SM, Zhu H. A Bayesian mixed-treatment comparison meta-analysis of treatments for alcohol dependence and implications for planning future trials. Medical Decision Making. 2014;34(7):899–910. pmid:24935915
- 60. Webb EL, Forster JJ. Bayesian model determination for multivariate ordinal and binary data. Computational Statistics & Data Analysis. 2008;52(5):2632–2649.
- 61. Talhouk A, Doucet A, Murphy K. Efficient Bayesian inference for multivariate probit models with sparse inverse correlation matrices. Journal of Computational and Graphical Statistics. 2012;21(3):739–757.