http://orcid.org/0000-0002-4649-5016
Figures
Abstract
Background
Structural approach disparities were minimally addressed in past systematic reviews of model-based cost-effectiveness analyses addressing Tuberculosis management strategies. This review aimed to identify the structural approach disparities in model-based cost-effectiveness analysis studies addressing Tuberculosis diagnosis and describe potential hazards caused by those disparities.
Methods
A systematic search to identify studies published before October 2015 was performed in five electronic databases. After removal of duplication, studies’ titles and abstracts were screened based on predetermined criteria. The full texts of potentially relevant studies were subsequently screened and excluded when they did not address active pulmonary Tuberculosis diagnosis. Quality of the studies was assessed using the “Philips’ checklist.” Various data regarding general information, cost-effectiveness results, and disease modeling were extracted using standardized data extraction forms. Data pertaining to models’ structural approaches were compared and analyzed qualitatively for their applicability in various study settings, as well as their potential influence on main outcomes and cost-effectiveness conclusion.
Results
A total of 27 studies were included in the review. Most studies utilized a static model, which could underestimate the cost-effectiveness of the diagnostic tools strategies, due to the omission of indirect diagnosis effects, i.e. transmission reduction. A few structural assumption disparities were found in the dynamic models. Extensive disparities were found in the static models, consisting of varying structural assumptions regarding treatment outcomes, clinical diagnosis and empirical treatment, inpatient discharge decision, and re-diagnosis of false negative patients.
Conclusion
In cost-effectiveness analysis studies addressing active pulmonary Tuberculosis diagnosis, models showed numerous disparities in their structural approaches. Several structural approaches could be inapplicable in certain settings. Furthermore, they could contribute to under- or overestimation of the cost-effectiveness of the diagnosis tools or strategies. They could thus lead to ambiguities and difficulties when interpreting a study result. A set of recommendations is proposed to manage issues related to these structural disparities.
Citation: Padmasawitri TIA, Frederix GW, Alisjahbana B, Klungel O, Hövels AM (2018) Disparities in model-based cost-effectiveness analyses of tuberculosis diagnosis: A systematic review. PLoS ONE 13(5): e0193293. https://doi.org/10.1371/journal.pone.0193293
Editor: Jan-Willem Alffenaar, University Medical Center Groningen, NETHERLANDS
Received: December 16, 2016; Accepted: January 30, 2018; Published: May 9, 2018
Copyright: © 2018 Padmasawitri et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This work received support from the Indonesian Endowment Fund for Education, PRJ-336, to Ms. T.I. Armina Padmasawitri. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: TIAP reported a grant from Indonesian Endowment Fund for Education (LPDP). This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
The current Tuberculosis (TB) epidemic is mainly driven by untreated or improperly treated TB cases. Largely, those cases are caused by challenges in diagnosis, [1] and to reduce them, a rapid and accurate diagnosis process is required. This need is answered by the development of various novel diagnostic tools, such as the new generation of smear microscopy, e.g. Light Emitting Diode (LED) smear microscopy, and liquid culture. Both have higher sensitivity compared to their older generation tools, i.e. Ziehl-Niessen (ZN) smear microscopy and solid culture. [2,3] Another example is the new generation Nucleic Acid Amplification Test (NAAT), i.e. Xpert.MTB/Rif (Xpert) which has a significantly higher sensitivity compare to ZN microscopy, a rapid turnaround time for result, and ability to simultaneously detect drug resistant TB (i.e. rifampicin resistant strain). [3] Another novel diagnostic tool is the urine lipoarabinomannan (LAM) assay, which can be delivered at a point-of-care, although its high sensitivity is limited to TB cases with HIV comorbidity. [4]
These novel diagnostic tools often have higher cost than the existing tools. They may also require extensive resources, such as, in the case of Xpert, stabile electricity and highly trained laboratory personnel. Hence, their implementation is often hindered by budget constraints. [5] To reconcile policy and budget constraints, that is, to locate the diagnosis strategy which produces maximum health benefit at an affordable cost, decision making can be guided by the valuable information generated from model-based cost-effectiveness analyses.
In the last five years, the number of model-based cost-effectiveness analyses addressing TB diagnosis has increased tremendously. [6] This progress is, however, often outweighed by inconsistencies in modeling practices. These inconsistencies contribute to disparities among studies results, which consequently reduce studies’ comparability and transferability. [7] This situation creates complications in utilizing study results to guide decision making. Hence, identifying these inconsistencies is critical. [8]
Past systematic reviews identified modeling practice inconsistencies in cost-effectiveness analyses of TB management and intervention. They documented several methodological inconsistencies, [6,9–11] including variations in cost measurement method, choice of health outcome measures (e.g. QALY, DALY, or natural units), and study perspective (e.g. health system or societal). These reviews did not identify inconsistencies pertaining to the structure of the model used to depict disease progression. A more recent systematic review described various structure of models used to predict the cost-effectiveness of TB screening strategies and addressed their quality. [12] Although it highlighted the influence of several parameters, such as tool’s accuracy, it did not analyze the potential influence of different structural approaches on the cost-effectiveness estimates. [12] As shown by studies in other disease areas, models’ structural approaches disparities, such as different choices of health states (e.g. inclusion or exclusion of latent disease state), may substantially influence cost-effectiveness results. [13,14] Strong evidence of this influence was shown by a recent study which measured the cost-effectiveness of various TB interventions using multi models. [15] All models in the study utilized the same methodology and input parameters; however, disparities appeared in their cost-effectiveness results. These disparities could mostly be explained by the differences in the models’ structural approaches.
To the best of our knowledge, structural approach disparities in TB diagnosis model-based cost-effectiveness analyses are minimally addressed. Hence, in this study, a systematic review of model-based cost-effectiveness analyses addressing active pulmonary TB diagnosis was conducted to identify structural approach disparities and describe potential hazards caused by those disparities.
Methods
A systematic literature review was performed to identify model-based cost-effectiveness analyses addressing active pulmonary TB diagnosis, which were published before October 2015. The review was performed and reported following the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guideline. [16]
Search strategy and selection process
Electronic search was performed in several databases, namely PubMed, EMBASE, Centre for Reviews and Dissemination (CRD), Cost-effectiveness Analysis (CEA) Registry, and EconLit, using a broad search term, i.e.: ("Tuberculosis") AND ("cost" OR "economic") AND ("model" OR "mathematical model”). An example of a detailed search strategy and the search date for each database is detailed in S1 File.
After removal of duplication, the title and the abstract of the studies were screened by one reviewer (TIAP) based on predetermined criteria. A study was included in the review if: 1.) it evaluated the cost-effectiveness of active pulmonary TB diagnosis process, 2.) it was published in English, 3.) it was a full economic evaluation (both economic and health consequences were evaluated), 4.) it utilized a model to generate outcome (including decision-analytic, Markov and dynamic transmission model), and 5.) its cost effectiveness result was extractable and adjustable to the current value. A study was excluded following exclusion criteria detailed in Table 1. The study’s full text report was referenced when the abstract contained ambiguous information.
Following the initial screening, the full texts of the included studies were further screened. At this stage, studies were excluded when their main objective did not focus on active pulmonary TB diagnosis. This was done to remain consistent with the objective of the review. The excluded study objectives are detailed in Table 1.
A sample of 50 studies was taken from all studies found during the search process. The sample was re-screened by a different author (AMH) to validate the screening process. Articles excluded due to language restriction and inaccessible detailed results were analyzed further to avoid exclusion of relevant studies.
Quality assessment
The framework for quality assessment of decision analytic models (Philips’ checklist) [17] was utilized to assess the quality of the included studies. Philips’ checklist is the recommended tool for quality assessment of model-based studies. Several items in the Philips’ checklist were modified to accommodate specific quality characteristics pertaining to diagnosis. [18]
The quality assessment for each study was performed by two reviewers (TIAP and AMH; or TIAP and GWF). Discrepancies between the two reviewers were resolved through a discussion. When the discrepancies could not be resolved, a third reviewer was involved. Details on our approach to utilize Philips’ checklist for quality assessment can be found in S1 File.
The quality assessment results were analyzed qualitatively. The assessment items that were associated with the structural approaches of the model were highlighted.
Data extraction and analysis
Data extraction was performed on items related to the following topics: 1.) general information, 2.) main outcomes, and 3.) TB progression modeling. It was performed using standardized forms, developed in Microsoft Excel™ guided by previous studies. [18–20] The forms and complete extracted data can be found in S2–S4 Tables of the supporting information. The data extraction was performed by one reviewer (TIAP). However, the results were discussed with other authors, mainly authors who performed quality assessment on the studies (AMH or GWF).
The general information covered the study’s characteristics, including study’s methodological approaches, choice of modeling framework, and cost-effectiveness conclusion. It also included the settings in which the analysis was performed. Study settings were categorized as high burden TB settings if they were listed by World Health Organization (WHO) as among the 22 high TB burden and 41 high TB/HIV burden countries. [21] Settings were categorized as moderate or low burden settings based on the claim of the studies.
Extracted main outcomes were Incremental Cost Effectiveness ratio (ICER) and/or Average Cost Effectiveness Ratio (ACER). ICERs or ACERs published in United States dollars (USD) were adjusted to 2015 value using the Consumer Price Index (CPI). [22] ICERs or ACERs published in other currencies were adjusted to 2015 value using the CPI followed by conversion to USD using Purchase Power Parity (PPP). [23–27] The adjustment method followed the recommendation from WHO. [28]
TB progression modelling topic covered the inclusion/exclusion of disease and setting characteristics in the model, as well as structural assumptions. Structural assumptions encompassed all study’s assumptions which influenced the structure of its model.
A pilot study performed following the development of the data extraction forms, tested the applicability of the forms. Its result was discussed among authors and necessary changes, as well as additional fields were incorporated into the forms.
The extracted data were analyzed qualitatively. The cost-effectiveness conclusions and main cost-effectiveness outcomes of similar studies (i.e. those performed in the same setting and addressed similar diagnostic tools) were assessed to find variability. The methodological approaches of the studies were assessed to confirm inconsistencies found in previous reviews. In the main analysis, models’ structural approaches were assessed to find variability. Their applicability in various settings and potential influence on the cost-effectiveness result were also analyzed. The structural approaches assessed were the choice of modelling framework, the inclusion/exclusion of disease and setting characteristics in the model, and the structural assumptions.
Results
After removing duplications, the search in five electronic databases identified 1126 articles. Of these, only 27 fulfilled the inclusion criteria and were included in the review. [29–55] This article selection process is depicted in Fig 1.
In total, 1099 studies were excluded. The number of studies excluded for each exclusion criterion is given in Table 1.
Most studies (78%) were performed in high burden TB settings, i.e. Brazil, [34,35,38,47,48] India, [36,40,42,50,52,53] several countries in Southern Africa (Botswana, Lesotho, Namibia, South Africa and Swaziland), [31,35,43,51–53] and East Africa (Kenya, Malawi, Uganda, Tanzania and Zambia). [30,35,41,44,46,49,51–53,55] One study performed analysis in a high burden region, i.e. Sub-Saharan Africa. [29] The remaining studies were performed in low to moderate burden settings, i.e. Finland, [45] UK, [39] US, [32,33,37] and Hong Kong P.R. China. [54] More than half of the studies (63%) were performed within 2011–2015. [29,32,36,38–43,48–55] The earliest study dated from 1998. [46]
The included studies evaluated the cost-effectiveness of various active pulmonary TB diagnosis tools and strategies. Two studies evaluated the negative impact of serology based testing for active disease diagnosis in high burden settings. [36,42]
Almost all studies concluded that novel diagnosis tools or strategies were cost effective compared to the best available diagnosis, as depicted in Fig 2. Exceptions were found in three studies assessing NAAT (e.g. Xpert and Mycobacterium Tuberculosis Direct Test or MTD) in low-burden settings, [33,39,45] two studies evaluating serology based testing in high burden setting, [36,42] and one study investigating TB diagnosis in private and public health sector. [50]
NAAT = Nucleic Acid Amplification Technique, MTD = Mycobacterium Tuberculosis Direct Test, LM = Light-Emitting Diode Microscopy, SSM = Sputum Smear Microscopy, LAM = Lipoarabinomanan assay, MGIT = Mycobacteria Growth Indicator Tube, LJ = Löwenstein–Jensen, CXR = Chest X-Ray. * Secondary analysis by Rajahlati et al. showed different cost-effectiveness conclusion from the primary analysis (NAAT for all TB presumptive cases). NAAT was cost effective when applied selectively on smear positive cases due to its high sensitivity and the high rate of true positive in this group. [45].
Despite the high degree of agreement in cost-effectiveness conclusion, ICER and/or ACER value for similar tools or strategies varied between studies and settings. This can be seen in Figs 3 and 4, which depicts the ICER of various diagnosis strategies from studies utilizing generic health outcomes measurement, i.e. Disability Adjusted Life Years (DALY) and Quality Adjusted Life Years (QALY). The adjusted cost-effectiveness outcome for all included studies can be found in S3 Table.
DALY = Disability Adjusted Life Years, SSM = Sputum Smear Microscopy, Xpert = Xpert MTB/RIF, IGRA = Interferon-Gamma Release Assay (e.g. anda-TB), MGIT = Mycobacteria Growth Indicator Tube, LM = Light-Emitting Diode Microscopy, LF-LAM = Lateral Flow Urine Lipoarabinomannan Assay, LJ = Löwenstein–Jensen.
QALY = Quality Adjusted Life Years, MTD = Mycobacterium tuberculosis Direct Test, Xpert = Xpert MTB/RIF, SSM = Sputum Smear Microscopy, NAAT = Nucleic Acid Amplification Technique, PPM = Public Private Mix Program.
Two studies which evaluated the cots-effectiveness of implementing Xpert as an initial test for all TB cases in South Africa exemplified the variations of cost-effectiveness outcome. [43,53] Both studies, published one year apart, utilized similar comparator and Xpert test price. However, after adjustment to 2015 value, one study predicted an ICER value of USD 749.82, [43] while the other predicted an ICER of USD 149.55. [53]
As detailed in Table 2, the included studies utilized various methodological approaches. Several studies were found to neglect certain good quality attributes, especially those associated with the models’ structural approach. Around half of the studies failed to report structural assumptions and their justification. [29–34,36–38,40,42,44,46–49,51] Moreover, none of the studies performed adequate structural uncertainty analysis. All studies also failed to report adequately the synthesis process of the diagnostic tools’ accuracy. A complete quality assessment of the studies can be found in S1 Table.
Disparities in the models’ structural approaches were also found among the studies, including different choices of modeling framework. The static model framework, i.e. single cohort Decision Analytic Model (DAM) consisted of a decision tree, was most commonly utilized by the studies (89%). [29–40,42,44–49,51–55] One study utilized DAM consisted of a decision tree in combination with a Markov model to simulate re-diagnosis of false-negative patients. [34] The dynamic model framework was used by three studies which conducted their analysis in high burden settings. [41,43,50] Studies’ choices of modeling framework are summarized in Fig 5.
As detailed in Fig 5, studies generally included important setting characteristics, i.e. Human Immunodeficiency Virus (HIV) comorbidity and Multi Drug Resistance (MDR) TB. The inclusion of these characteristics reflected the disease burden in the study settings. However, these characteristics were omitted by several studies conducted in important settings with three overlapping burdens (high prevalence of TB, TB-HIV comorbidity, and MDR TB), i.e. India and South Africa. [31,35,36,40,50,51] Studies that modeled HIV comorbidity differed in modeling the influence of HIV on TB progression, e.g. influence of HIV towards smear status.
Two studies included setting characteristic in addition to HIV comorbidity and MDR TB, i.e. different quality of TB care in public and private sector. [43,50] One study, in which the setting was dominated by sub-standard private sector care, showed that Xpert alone without a program to transfer patients from private to public sector was not cost effective. [50] Another study in the same setting, which did not consider this characteristic, concluded that Xpert alone was cost effective. [53]
Studies using a dynamic model framework showed few structural assumptions disparities. One of the studies omitted the active transmission process of MDR TB strain. [41].
In contrast, the 24 static models showed extensive structural assumption disparities. These included different assumptions regarding treatment outcomes, clinical diagnosis and empirical treatment, inpatient discharge decision, and re-diagnosis of false negative patients. The disparities are detailed in Table 3.
Discussion
The wide range in the cost-effectiveness ratio (ICER and/or ACER) observed in the included studies could be the result of different study settings and populations. However, it could also have been influenced by modeling practice inconsistencies. As reported in one study, modeling practice inconsistencies, such as utilization of different methodological and structural approaches, were strongly related to the wide range of results. [7] Modeling practice inconsistencies identified by previous reviews, i.e. variations in methodological approaches, [6,9–11] were still prevalent among the included studies. The included studies were also afflicted by quality issues pertaining to structural approaches identified by a previous review, such as failure to address structural uncertainty. [12] In addition, the studies also showed extensive disparities in the structural approaches.
As in previous reviews, [6,10] a static model framework, which omitted TB transmission process, was the preferred options among the included studies. This is understandable, since static models development is relatively straightforward and can be easily understood by non-modelers, including decision makers. In contrast, dynamic models require complicated mathematical computation. Furthermore, their development is challenged by the difficulties in modeling the transmission process itself [73] especially when data and knowledge regarding the transmission process is limited.
However, the use of static models to address infectious disease intervention strategies may potentially underestimate the indirect effect of the strategies in preventing secondary cases. [73,74] This issue was recognized by almost all studies which utilized a static model framework; however, they argued that the underestimation would result in a more conservative estimate. The argument should be considered carefully, since the underestimation may not only affect health benefit but also the demand of intervention and its related cost. [74]
The use of a dynamic model framework is recommended for investigating an infectious disease management strategy that influences the disease transmission process. [75] Diagnostic strategies may influence the TB transmission process through mitigating the main infection source, e.g. untreated active disease consisting of undiagnosed (e.g. diagnostic loss to follow up) and false negative cases. They may also influence transmission process due to their operational aspect, such as a rapid process to obtain diagnosis result. The rapid process causes shorter delay in starting treatment, which consequently causes shorter period of infectiousness. [76] The influences of diagnosis strategies on the TB transmission process were consistently observed in all included studies which utilized dynamic models. [41,43,50]
The ongoing TB transmission process in high burden TB settings is an important factor that might hinder countries progressing toward TB control goals. [77] The impact of a diagnostic strategy on the ongoing transmission process is better explained by a dynamic model framework; hence, the use of such framework may be more beneficial for high burden settings.
A dynamic model framework may also be useful in facilitating long term analysis. Studies utilizing a dynamic model framework in a setting with a high prevalence of TB and HIV comorbidity could identify budget increase in the long term (10 to 20 years), not only for TB management, but also for Anti-Retroviral Treatment (ART), due to the higher survival rate of patients. [41,43] This finding was not observed in the static models. [43]
HIV comorbidity and MDR TB were included in the model governed by underlying assumptions driven by currently limited knowledge and data. [77] Thus, we found various assumptions regarding HIV influence on TB as well as the MDR TB transmission process. Furthermore, these characteristics were excluded in several relevant settings. This exclusion potentially influences study result, e.g. by overestimating the effectiveness of diagnosis tools with lower accuracy in HIV positive individuals. An evidence of this influence was shown by a recent multi-model study. [78] In the study, the inclusion and exclusion of HIV interaction with TB in the models led to differences in TB outcome projections.
Other important setting characteristics include various key drivers of TB epidemics and health system characteristics, such as domination of sub-standard private sector TB care. [79] As shown by one study, including a sub-standard private sector in the model could influence cost-effectiveness results. [50] Unfortunately, currently, data and knowledge for this characteristic are limited. One example of this limitation is the uncertainty regarding the impact of shifting patients from private sector to the high quality public sector care. [15] Another example is the uncertainty surrounding the number of TB cases managed in private sector due to underreporting. [80] More research into this characteristic will be valuable to inform model construction.
Other disparities found concerned the structural assumptions, which were more prevalent among the static models. Assuming a successful treatment outcome for all correct diagnosis was applied by several studies. This assumption would not apply in settings with significant numbers of unsuccessful treatment outcome such as treatment loss-to-follow-up. Assuming only successful treatment outcome in such settings may lead to an overestimation of the cost-effectiveness. This was proven by an empirical study in South Africa which showed that extensive loss to follow up mitigated the benefit of an accurate diagnosis by a novel tool. [81]
Another structural assumption disparity concerned the practice of clinical diagnosis, consisting of additional diagnosis, such as Chest X-ray, followed by an empirical TB treatment. It is usually performed in highly suspected TB cases which obtain negative results for the main diagnosis tests such as smear microscopy.
As argued by one study, the effectiveness of novel diagnosis tools could have been overestimated by modeling studies when clinical diagnosis practice was underestimated. [82] Novel diagnosis tools’ effectiveness is influenced by the number of additional diagnosed TB cases, which would have been undiagnosed by the existing diagnosis practice. When a rigorous clinical diagnosis practice is applied, as found in several high burden settings, [82,83] most TB cases will be diagnosed and treated under the existing diagnosis practice. Consequently, this will minimize the number of additional cases diagnosed by the novel diagnosis tools and lower their effectiveness. [82,83]
Another study argued that assuming clinical diagnosis practice to be unchanged by the availability of novel diagnostic tools with better accuracy could underestimate the effectiveness of the novel tools. [41] This was due to the expectation that novel tools would increase physicians’ certainty of a negative result; thus, reducing the need to perform additional clinical diagnosis which could introduce false positive cases due to its low specificity. Unfortunately, the influences of novel diagnosis tools on clinical diagnosis practice have not been fully understood. [83] However, clinical diagnosis practice had a significant influence on novel tool’s cost-effectiveness, as shown by a recent empirical study in South Africa. [81] Thus, it is important to consider this practice in the model.
In settings where TB is managed in an inpatients-setting, the assumption regarding decision to discharge from hospital or isolation can also be influential. The assumption, for instance, that a patient could be discharged solely based on Nucleic Acid Amplification Technique (NAAT) negative result, could lead to a positive cost-effectiveness for NAAT use in low burden TB settings. [32, 45] The speed with which NAAT result was obtained leads logically to a faster decision to discharge, thus minimizing hospital stay and its related cost.
Disparities were also found in assumptions around the false negative re-diagnosis process. Since most settings do not actively screen for TB, the re-diagnosis process may rely heavily on individuals’ discretion to seek another diagnosis. In high burden settings, this may be challenged by several circumstances such as long distances between home and the health center. Hence, the re-diagnosis process could have been overestimated, and could eventually cause diagnosis tools’ or strategies’ effectiveness overestimation.
Despite its important findings, this review could have overlooked some relevant studies, such as those published in languages other than English and those with inaccessible detailed result. However, additional analysis performed to studies excluded based on these two criteria did not result in additional inclusion of studies. The complete description of the additional analysis can be found in S1 File.
A meta-analysis could be performed on the cost and health benefit estimates of the reviewed studies to produce a single estimate of ICER. However, the single estimate might be futile since it is considered non-transferable to any setting. [84–86] An ICER estimate addresses an isolated problem in certain health-system, thus its component cannot be freely transferred to different settings. This transferability issue has resulted in the absence of a recommended meta-analysis method for cost-effectiveness analyses results. [86] Furthermore, a meta-analysis was not instrumental in achieving the aim of the study.
Bias could be introduced during the screening process, which was done by only one author. However, this risk of bias was reduced by performing validation. It showed that the level of agreement among two authors for abstract and title screening reached 88%, and a consensus was reached following a short deliberation. Bias could also be introduced during data extraction, since it was also conducted by one author. It was, however, reduced through extensive discussions among the co-authors; especially with those who also analyzed the studies for quality assessment.
This review included studies conducted in multiple settings and covered a variety of diagnostic tools and strategies. It also managed to confirm past systematic review findings. Thus, inclusion of more studies in this review would most likely substantiate current findings.
PRISMA guideline mandates systematic review to assess the individual study’s risk of bias. Indication of bias was shown by the extensive disparities of cost-effectiveness result among studies with similar diagnostic strategies and settings. This bias could be caused by methodological inconsistencies, including structural approach disparities. Another source of bias, which is not discussed in this review, is input parameters. Input parameters may have reliability issue. For example, several settings in Southeast Asia showed a high burden of TB, but population screening found that actual TB prevalence was significantly higher than what had been estimated through TB program case notification. [80] Another example of this reliability issue is the lack of transparency in reporting the method to synthesis diagnostic tools’ accuracy.
The structural approaches of the models were often reported inadequately. This practice could cause difficulties for decision makers when assessing the suitability of an approach to their setting, which emphasizes the need to report structural approaches and their justification transparently. In addition, model validation should be reported to ensure decision makers’ confidence in the model. This can be done following a recent guidance on model validation reporting. [87] Another important practice, often omitted, is to address uncertainties introduced by different choices of structural approaches. An example of this was shown in a recent study that used multi models and summarized the models’ cost-effectiveness results. [15] A summary of practice and reporting recommendation to manage issues related to model’s structural disparities can be found in Table 4.
Modeling practice inconsistencies, found in the past and in the current review, highlight the need for a clear standard for study conduct and reporting, especially in high burden TB settings, which mostly consists of Low-Middle Income Countries. Moreover, the standard should be enforced by important stakeholders, such as shown by The Bill and Melinda Gates Foundation, which developed a case-reference containing general methodologic specifications and reporting standards. [89] A disease-specific standard may also give additional benefit to model’s structure development.
Conclusion
This systematic review identified extensive structural approach disparities in model-based cost-effectiveness analyses addressing TB diagnosis strategies. It shows that several structural approaches could be inapplicable in certain settings. Furthermore, certain approaches could potentially contribute to under- or overestimation of the cost-effectiveness of a diagnosis tool or strategy. Eventually, they will lead to ambiguities and difficulties when interpreting the study result. This issue can be managed by reporting the model’s structural approaches and their justification transparently. Furthermore, this lack of clarity may also be reduced by addressing and reporting model’s validity and structural uncertainty.
Supporting information
S1 File. Details of the search and screening strategy, and quality assessment approach.
https://doi.org/10.1371/journal.pone.0193293.s001
(PDF)
S1 Table. Quality assessment of the included studies.
https://doi.org/10.1371/journal.pone.0193293.s003
(PDF)
S2 Table. Data extraction for general information.
https://doi.org/10.1371/journal.pone.0193293.s004
(PDF)
S4 Table. Data extraction of tuberculosis progression modeling approaches.
https://doi.org/10.1371/journal.pone.0193293.s006
(PDF)
Acknowledgments
We would like to thank Prof. Mirjam Kretzschmar for the comments and input on a draft of this article, as well as Linda McPhee for the input on the redaction of the article.
References
- 1. Zumla A, George A, Sharma V, Herbert RHN, Oxley A, Oliver M. The WHO 2014 Global tuberculosis report—further to go. Lancet Glob Health. 2015;3: e10–e12. pmid:25539957
- 2. Pai M, Minion J, Sohn H, Zwerling A, Perkins MD. Novel and improved technologies for tuberculosis diagnosis: progress and challenges. Clin Chest Med. 2009;30: 701–716, viii. pmid:19925962
- 3. Pai M, Dowdy D. Tuberculosis: progress and challenges in product development and delivery. Lancet Respir Med. 2014;2: 25–27. pmid:24461893
- 4. Lawn SD. Point-of-care detection of lipoarabinomannan (LAM) in urine for diagnosis of HIV-associated tuberculosis: a state of the art review. BMC Infect Dis. 2012;12: 103. pmid:22536883
- 5. Chiang C-Y, Van Weezenbeek C, Mori T, Enarson DA. Challenges to the global control of tuberculosis. Respirology. 2013;18: 596–604. pmid:23551328
- 6. Zwerling A, White RG, Vassall A, Cohen T, Dowdy DW, Houben RMGJ. Modeling of novel diagnostic strategies for active tuberculosis—a systematic review: current practices and recommendations. PloS One. 2014;9: e110558. pmid:25340701
- 7. Boehler CEH, Lord J. Mind the Gap! A Multilevel Analysis of Factors Related to Variation in Published Cost-Effectiveness Estimates within and between Countries. Med Decis Mak Int J Soc Med Decis Mak. 2016;36: 31–47. pmid:25878194
- 8.
Walker DG, Wilson RF, Sharma R, Bridges J, Niessen L, Bass EB, et al. Best Practices for Conducting Economic Evaluations in Health Care: A Systematic Review of Quality Assessment Tools [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2012. Available: http://www.ncbi.nlm.nih.gov/books/NBK114545/
- 9. Santatiwongchai B, Chantarastapornchit V, Wilkinson T, Thiboonboon K, Rattanavipapong W, Walker DG, et al. Methodological Variation in Economic Evaluations Conducted in Low- and Middle-Income Countries: Information for Reference Case Development. PLoS ONE. 2015;10. pmid:25950443
- 10. Oxlade O, Pinto M, Trajman A, Menzies D. How methodologic differences affect results of economic analyses: a systematic review of interferon gamma release assays for the diagnosis of LTBI. PloS One. 2013;8: e56044. pmid:23505412
- 11. Koufopoulou M, Sutton AJ, Breheny K, Diwakar L. Methods Used in Economic Evaluations of Tuberculin Skin Tests and Interferon Gamma Release Assays for the Screening of Latent Tuberculosis Infection: A Systematic Review. Value Health J Int Soc Pharmacoeconomics Outcomes Res. 2016;19: 267–276. pmid:27021762
- 12. Auguste P, Tsertsvadze A, Court R, Pink J. A systematic review of economic models used to assess the cost-effectiveness of strategies for identifying latent tuberculosis in high-risk groups. Tuberc Edinb Scotl. 2016;99: 81–91. pmid:27450009
- 13. Frederix GWJ, Severens JL, Hövels AM, Raaijmakers JAM, Schellens JHM. Reviewing the cost-effectiveness of endocrine early breast cancer therapies: influence of differences in modeling methods on outcomes. Value Health J Int Soc Pharmacoeconomics Outcomes Res. 2012;15: 94–105. pmid:22264977
- 14. Mauskopf J. Modelling technique, structural assumptions, input parameter values: which has the most impact on the results of a cost-effectiveness analysis? PharmacoEconomics. 2014;32: 521–523. pmid:24743914
- 15. Menzies NA, Gomez GB, Bozzani F, Chatterjee S, Foster N, Baena IG, et al. Cost-effectiveness and resource implications of aggressive action on tuberculosis in China, India, and South Africa: a combined analysis of nine models. Lancet Glob Health. 2016; pmid:27720689
- 16. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62: e1–34. pmid:19631507
- 17. Philips Z, Ginnelly L, Sculpher M, Claxton K, Golder S, Riemsma R, et al. Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health Technol Assess Winch Engl. 2004;8: iii–iv, ix–xi, 1–158.
- 18. Handels RLH, Wolfs CAG, Aalten P, Joore MA, Verhey FRJ, Severens JL. Diagnosing Alzheimer’s disease: a systematic review of economic evaluations. Alzheimers Dement J Alzheimers Assoc. 2014;10: 225–237. pmid:23727080
- 19. Dowdy DW, Dye C, Cohen T. Data Needs for Evidence-Based Decisions: A TB Modeler’s “Wish List.” Int J Tuberc Lung Dis Off J Int Union Tuberc Lung Dis. 2013;17: 866–877. pmid:23743307
- 20. White PJ, Garnett GP. Mathematical modelling of the epidemiology of tuberculosis. Adv Exp Med Biol. 2010;673: 127–140. pmid:20632534
- 21.
World Health Organization. Global Tuberculosis Report 2016 [Internet]. World Health Organization; 2016. Available: http://apps.who.int/iris/bitstream/10665/250441/1/9789241565394-eng.pdf?ua=1
- 22.
United States Department of Labor. CPI Inflation Calculator [Internet]. 2015. Available: http://data.bls.gov/cgi-bin/cpicalc.pl
- 23.
Office for National Statistics United Kingdom. Consumer Price Inflation: December 2015, CPI index values (2005 = 100), 1-month and 12-month rates: December 2014 to December 2015 [Internet]. 2016. Available: http://www.ons.gov.uk/economy/inflationandpriceindices/bulletins/consumerpriceinflation/december2015
- 24.
Office for National Statistics United Kingdom. Consumer Price Indices, December 2011,CPI Indices (2005 = 100), 1-month change and 12-month change: December 2010 to December 2011 [Internet]. 2012. Available: http://webarchive.nationalarchives.gov.uk/20160105160709/http://www.ons.gov.uk/ons/dcp171778_250279.pdf
- 25.
National Institute of Statistics and Economic Studies. Consumer Price Index—Base 2015—All households—France—All items [Internet]. 2016. Available: http://www.insee.fr/en/bases-de-donnees/bsweb/serie.asp?idbank=001759970
- 26.
Official Statistics of Finland (OSF). Consumer Price Index 2000 = 100, December 2015, Appendix table 5 [Internet]. 2016. Available: http://www.stat.fi/til/khi/2015/12/khi_2015_12_2016-01-14_tau_005_en.html
- 27.
Organisation for Economic Co-operation and Development (OECD). National Accounts: PPPs and Exchange Rates [Internet]. OECD.Stat. Available: http://stats.oecd.org/Index.aspx?datasetcode=SNA_TABLE4
- 28.
Tan-Torres Edejer T, Baltussen R, Adam T, Hutubessy R, Acharya A, Evans DB, et al., editors. Making choices in health: WHO guide to cost-effectiveness analysis. Geneva: World Health Organization; 2003.
- 29. Abimbola TO, Marston BJ, Date AA, Blandford JM, Sangrujee N, Wiktor SZ. Cost-effectiveness of tuberculosis diagnostic strategies to reduce early mortality among persons with advanced HIV infection initiating antiretroviral therapy. J Acquir Immune Defic Syndr 1999. 2012;60: e1–7. pmid:22240465
- 30. Bonnet M, Tajahmady A, Hepple P, Ramsay A, Githui W, Gagdnidze L, et al. Added value of bleach sedimentation microscopy for diagnosis of tuberculosis: a cost-effectiveness study. Int J Tuberc Lung Dis Off J Int Union Tuberc Lung Dis. 2010;14: 571–577.
- 31. Chihota VN, Grant AD, Fielding K, Ndibongo B, van Zyl A, Muirhead D, et al. Liquid vs. solid culture for tuberculosis: performance and cost in a resource-constrained setting. Int J Tuberc Lung Dis Off J Int Union Tuberc Lung Dis. 2010;14: 1024–1031.
- 32. Choi HW, Miele K, Dowdy D, Shah M. Cost-effectiveness of Xpert® MTB/RIF for diagnosing pulmonary tuberculosis in the United States. Int J Tuberc Lung Dis Off J Int Union Tuberc Lung Dis. 2013;17: 1328–1335. pmid:24025386
- 33. Dowdy DW, Maters A, Parrish N, Beyrer C, Dorman SE. Cost-Effectiveness Analysis of the Gen-Probe Amplified Mycobacterium Tuberculosis Direct Test as Used Routinely on Smear-Positive Respiratory Specimens. J Clin Microbiol. 2003;41: 948–953. pmid:12624014
- 34. Dowdy DW, Lourenço MC, Cavalcante SC, Saraceni V, King B, Golub JE, et al. Impact and cost-effectiveness of culture for diagnosis of tuberculosis in HIV-infected Brazilian adults. PloS One. 2008;3: e4057. pmid:19129940
- 35. Dowdy DW O’Brien MA, Bishai D. Cost-effectiveness of novel diagnostic tools for the diagnosis of tuberculosis. Int J Tuberc Lung Dis Off J Int Union Tuberc Lung Dis. 2008;12: 1021–1029.
- 36. Dowdy DW, Steingart KR, Pai M. Serological testing versus other strategies for diagnosis of active tuberculosis in India: a cost-effectiveness analysis. PLoS Med. 2011;8: e1001074. pmid:21857810
- 37. Guerra RL, Hooper NM, Baker JF, Alborz R, Armstrong DT, Kiehlbauch JA, et al. Cost-effectiveness of different strategies for amplified Mycobacterium tuberculosis direct testing for cases of pulmonary tuberculosis. J Clin Microbiol. 2008;46: 3811–3812. pmid:18799694
- 38. Guerra RL, Dorman SE, Luiz RR, Conde MB. Cost-effectiveness of routine diagnostic evaluation of pulmonary tuberculosis in a primary care unit in Brazil. Int J Tuberc Lung Dis Off J Int Union Tuberc Lung Dis. 2013;17: 1336–1340. pmid:24025387
- 39. Hughes R, Wonderling D, Li B, Higgins B. The cost effectiveness of Nucleic Acid Amplification Techniques for the diagnosis of tuberculosis. Respir Med. 2012;106: 300–307. pmid:22137190
- 40. Kelly V, Sagili KD, Satyanarayana S, Reza LW, Chadha SS, Wilson NC. Cost-utility analysis of LED fluorescence microscopy in the diagnosis of pulmonary tuberculosis in Indian settings. Int J Tuberc Lung Dis Off J Int Union Tuberc Lung Dis. 2015;19: 696–701. pmid:25946362
- 41. Langley I, Lin H-H, Egwaga S, Doulla B, Ku C-C, Murray M, et al. Assessment of the patient, health system, and population effects of Xpert MTB/RIF and alternative diagnostics for tuberculosis in Tanzania: an integrated modelling approach. Lancet Glob Health. 2014;2: e581–591. pmid:25304634
- 42. Little KM, Pai M, Dowdy DW. Costs and Consequences of Using Interferon-γ Release Assays for the Diagnosis of Active Tuberculosis in India. PloS One. 2014;10: e0124525. pmid:25918999
- 43. Menzies NA, Cohen T, Lin H-H, Murray M, Salomon JA. Population health impact and cost-effectiveness of tuberculosis diagnosis with Xpert MTB/RIF: a dynamic simulation and economic evaluation. PLoS Med. 2012;9: e1001347. pmid:23185139
- 44. Mueller DH, Mwenge L, Muyoyeta M, Muvwimi MW, Tembwe R, McNerney R, et al. Costs and cost-effectiveness of tuberculosis cultures using solid and liquid media in a developing country. Int J Tuberc Lung Dis Off J Int Union Tuberc Lung Dis. 2008;12: 1196–1202.
- 45. Rajalahti I, Ruokonen EL, Kotomäki T, Sintonen H, Nieminen MM. Economic evaluation of the use of PCR assay in diagnosing pulmonary TB in a low-incidence area. Eur Respir J. 2004;23: 446–451. pmid:15065837
- 46. Roos BR, van Cleeff MR, Githui WA, Kivihya-Ndugga L, Odhiambo JA, Kibuga DK, et al. Cost-effectiveness of the polymerase chain reaction versus smear examination for the diagnosis of tuberculosis in Kenya: a theoretical model. Int J Tuberc Lung Dis Off J Int Union Tuberc Lung Dis. 1998;2: 235–241.
- 47. Scherer LC, Sperhacke RD, Ruffino-Netto A, Rossetti ML, Vater C, Klatser P, et al. Cost-effectiveness analysis of PCR for the rapid diagnosis of pulmonary tuberculosis. BMC Infect Dis. 2009;9: 216. pmid:20043842
- 48. Schmid KB, Scherer L, Barcellos RB, Kuhleis D, Prestes IV, Steffen RE, et al. Smear plus Detect-TB for a sensitive diagnosis of pulmonary tuberculosis: a cost-effectiveness analysis in an incarcerated population. BMC Infect Dis. 2014;14: 678. pmid:25510328
- 49. Shah M, Dowdy D, Joloba M, Ssengooba W, Manabe YC, Ellner J, et al. Cost-effectiveness of novel algorithms for rapid diagnosis of tuberculosis in HIV-infected individuals in Uganda. AIDS Lond Engl. 2013;27: 2883–2892. pmid:25119690
- 50. Suen S-C, Bendavid E, Goldhaber-Fiebert JD. Cost-effectiveness of improvements in diagnosis and treatment accessibility for tuberculosis control in India. Int J Tuberc Lung Dis Off J Int Union Tuberc Lung Dis. 2015;19: 1115–1124, i–xv. pmid:26260835
- 51. Sun D, Dorman S, Shah M, Manabe YC, Moodley VM, Nicol MP, et al. Cost utility of lateral-flow urine lipoarabinomannan for tuberculosis diagnosis in HIV-infected African adults. Int J Tuberc Lung Dis Off J Int Union Tuberc Lung Dis. 2013;17: 552–558. pmid:23485389
- 52. van’t Hoog AH, Cobelens F, Vassall A, van Kampen S, Dorman SE, Alland D, et al. Optimal triage test characteristics to improve the cost-effectiveness of the Xpert MTB/RIF assay for TB diagnosis: a decision analysis. PloS One. 2013;8: e82786. pmid:24367555
- 53. Vassall A, van Kampen S, Sohn H, Michael JS, John KR, den Boon S, et al. Rapid diagnosis of tuberculosis with the Xpert MTB/RIF assay in high burden countries: a cost-effectiveness analysis. PLoS Med. 2011;8: e1001120. pmid:22087078
- 54. You JHS, Lui G, Kam KM, Lee NLS. Cost-effectiveness analysis of the Xpert MTB/RIF assay for rapid diagnosis of suspected tuberculosis in an intermediate burden area. J Infect. 2015;70: 409–414. pmid:25573001
- 55. Zwerling AA, Sahu M, Ngwira LG, Khundi M, Harawa T, Corbett EL, et al. Screening for Tuberculosis Among Adults Newly Diagnosed With HIV in Sub-Saharan Africa: A Cost-Effectiveness Analysis. J Acquir Immune Defic Syndr 1999. 2015;70: 83–90. pmid:26049281
- 56. Ang M, Nguyen HV, Kiew SY, Chen S, Chee S-P, Finkelstein E. Cost-effectiveness of alternative strategies for interferon-γ release assays and tuberculin skin test in tuberculous uveitis. Br J Ophthalmol. 2015;99: 984–989. pmid:25573149
- 57. Sugarman J, Alvarez GG, Schwartzman K, Oxlade O. Sputum induction for tuberculosis diagnosis in an Arctic setting: a cost comparison. Int J Tuberc Lung Dis Off J Int Union Tuberc Lung Dis. 2014;18: 1223–1230. pmid:25216837
- 58. Drobniewski F, Cooke M, Jordan J, Casali N, Mugwagwa T, Broda A, et al. Systematic review, meta-analysis and economic modelling of molecular diagnostic tests for antibiotic resistance in tuberculosis. Health Technol Assess Winch Engl. 2015;19: 1–188. pmid:25952553
- 59. Dowdy DW, van’t Hoog A, Shah M, Cobelens F. Cost-effectiveness of rapid susceptibility testing against second-line drugs for tuberculosis. Int J Tuberc Lung Dis Off J Int Union Tuberc Lung Dis. 2014;18: 647–654. pmid:24903933
- 60. Mears J, Vynnycky E, Lord J, Borgdorff MW, Cohen T, Crisp D, et al. The prospective evaluation of the TB strain typing service in England: a mixed methods study. Thorax. 2015; pmid:25882538
- 61. Andrews JR, Lawn SD, Rusu C, Wood R, Noubary F, Bender MA, et al. The cost-effectiveness of routine tuberculosis screening with Xpert MTB/RIF prior to initiation of antiretroviral therapy: a model-based analysis. AIDS Lond Engl. 2012;26: 987–995. pmid:22333751
- 62. Samandari T, Bishai D, Luteijn M, Mosimaneotsile B, Motsamai O, Postma M, et al. Costs and consequences of additional chest x-ray in a tuberculosis prevention program in Botswana. Am J Respir Crit Care Med. 2011;183: 1103–1111. pmid:21148723
- 63. Porco TC, Lewis B, Marseille E, Grinsdale J, Flood JM, Royce SE. Cost-effectiveness of tuberculosis evaluation and treatment of newly-arrived immigrants. BMC Public Health. 2006;6: 157. pmid:16784541
- 64. Winetsky DE, Negoescu DM, DeMarchis EH, Almukhamedova O, Dooronbekova A, Pulatov D, et al. Screening and rapid molecular diagnosis of tuberculosis in prisons in Russia and Eastern Europe: a cost-effectiveness analysis. PLoS Med. 2012;9: e1001348. pmid:23209384
- 65. Marchand R, Tousignant P, Chang H. Cost-effectiveness of screening compared to case-finding approaches to tuberculosis in long-term care facilities for the elderly. Int J Epidemiol. 1999;28: 563–570. pmid:10405865
- 66. Dowdy DW, Andrews JR, Dodd PJ, Gilman RH. A user-friendly, open-source tool to project impact and cost of diagnostic tests for tuberculosis. eLife. 2014;3. pmid:24898755
- 67. Langley I, Doulla B, Lin H-H, Millington K, Squire B. Modelling the impacts of new diagnostic tools for tuberculosis in developing countries to enhance policy decisions. Health Care Manag Sci. 2012;15: 239–253. pmid:22674467
- 68. Langley I, Adams E, Doulla B, Squire SB. Operational modelling to guide implementation and scale-up of diagnostic tests within the health system: exploring opportunities for parasitic disease diagnostics based on example application for tuberculosis. Parasitology. 2014;141: 1795–1802. pmid:25035934
- 69. Walker D. Economic analysis of tuberculosis diagnostic tests in disease control: how can it be modelled and what additional information is needed? Int J Tuberc Lung Dis Off J Int Union Tuberc Lung Dis. 2001;5: 1099–1108.
- 70. de Perio MA, Tsevat J, Roselle GA, Kralovic SM, Eckman MH. Cost-effectiveness of interferon gamma release assays vs tuberculin skin tests in health care workers. Arch Intern Med. 2009;169: 179–187. pmid:19171815
- 71. Tsevat J, Taylor WC, Wong JB, Pauker SG. Isoniazid for the tuberculin reactor: take it or leave it. Am Rev Respir Dis. 1988;137: 215–220. pmid:3276255
- 72. Gold MR, Franks P, McCoy KI, Fryback DG. Toward consistency in cost-utility analyses: using national measures to create condition-specific values. Med Care. 1998;36: 778–792. pmid:9630120
- 73. Drake TL, Devine A, Yeung S, Day NPJ, White LJ, Lubell Y. Dynamic Transmission Economic Evaluation of Infectious Disease Interventions in Low- and Middle-Income Countries: A Systematic Literature Review. Health Econ. 2016;25 Suppl 1: 124–139. pmid:26778620
- 74. Lugnér AK, Mylius SD, Wallinga J. Dynamic versus static models in cost-effectiveness analyses of anti-viral drug therapy to mitigate an influenza pandemic. Health Econ. 2010;19: 518–531. pmid:19382106
- 75. Pitman R, Fisman D, Zaric GS, Postma M, Kretzschmar M, Edmunds J, et al. Dynamic transmission modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-5. Med Decis Mak Int J Soc Med Decis Mak. 2012;32: 712–721. pmid:22990086
- 76. Lin H-H, Dowdy D, Dye C, Murray M, Cohen T. The impact of new tuberculosis diagnostics on transmission: why context matters. Bull World Health Organ. 2012;90: 739–747A. pmid:23109741
- 77. Yates TA, Khan PY, Knight GM, Taylor JG, McHugh TD, Lipman M, et al. The transmission of Mycobacterium tuberculosis in high burden settings. Lancet Infect Dis. 2016;16: 227–238. pmid:26867464
- 78. Houben RMGJ, Menzies NA, Sumner T, Huynh GH, Arinaminpathy N, Goldhaber-Fiebert JD, et al. Feasibility of achieving the 2025 WHO global tuberculosis targets in South Africa, China, and India: a combined analysis of 11 mathematical models. Lancet Glob Health. 2016;
- 79.
Stop TB Partnership. The Global Plan to End TB 2016–2020, The Paradigm Shift. In: The Global Plan to End TB 2016–2020 [Internet]. 2015 [cited 2 Feb 2016]. Available: http://www.stoptb.org/global/plan/plan2/
- 80.
National Institute of Health Research and Development, Directorate Generale of Disease Control and Environmental Health. Indonesia Tuberculosis Prevalence Survey 2013–2014. Jakarta: Ministry of Health Republic of Indonesia; 2015.
- 81. Vassall A, Siapka M, Foster N, Cunnama L, Ramma L, Fielding K, et al. Cost-effectiveness of Xpert MTB/RIF for tuberculosis diagnosis in South Africa: a real-world cost analysis and economic evaluation. Lancet Glob Health. 2017;5: e710–e719. pmid:28619229
- 82. Menzies NA, Cohen T, Murray M, Salomon JA. Effect of empirical treatment on outcomes of clinical trials of diagnostic assays for tuberculosis. Lancet Infect Dis. 2015;15: 16–17. pmid:25541164
- 83. Theron G, Peter J, Dowdy D, Langley I, Squire SB, Dheda K. Do high rates of empirical treatment undermine the potential effect of new diagnostic tests for tuberculosis in high-burden settings? Lancet Infect Dis. 2014;14: 527–532. pmid:24438820
- 84. Gomersall J.S., Jadotte Y.T., Xue Y., Lockwood S., Riddle D., Preda A. Conducting systematic reviews of economic evaluations. Int. J. Evid. Based Healthc. 2015;13, 170–178. pmid:26288063
- 85. Mathes T., Jacobs E., Morfeld J.-C., Pieper D. Methods of international health technology assessment agencies for economic evaluations—a comparative analysis. BMC Health Serv. Res. 2013;13, 371. pmid:24079858
- 86. Shemilt I., McDaid D., Marsh K., Henderson C., Bertranou E., Mallander J., Drummond M., Mugford M., Vale L., Campbell and Cochrane Economics Methods Group. Issues in the incorporation of economic perspectives and evidence into Cochrane reviews. Syst. Rev. 2013;2, 83. pmid:24050504
- 87. Vemer P, Corro Ramos I, van Voorn GAK, Al MJ, Feenstra TL. AdViSHE: A Validation-Assessment Tool of Health-Economic Models for Decision Makers and Model Users. Pharmacoeconomics. 2016;34: 349–361. pmid:26660529
- 88. Frederix GWJ, van Hasselt JGC, Schellens JHM, Hövels AM, Raaijmakers JAM, Huitema ADR, et al. The impact of structural uncertainty on cost-effectiveness models for adjuvant endocrine breast cancer treatments: the need for disease-specific model standardization and improved guidance. PharmacoEconomics. 2014;32: 47–61. pmid:24263964
- 89. Wilkinson T, Chalkidou K, Walker D, Lopert R, Teerawattananon Y, Chantarastapornchit V, et al. Bill and Melinda Gates Foundation: Methods for Economic Evaluation Project [Internet]. 2014. Available: https://www.york.ac.uk/media/che/documents/MEEP%20report%20final%20PDF.pdf