Reader Comments
Post a new comment on this article
Post Your Discussion Comment
Please follow our guidelines for comments and review our competing interests policy. Comments that do not conform to our guidelines will be promptly removed and the user account disabled. The following must be avoided:
- Remarks that could be interpreted as allegations of misconduct
- Unsupported assertions or statements
- Inflammatory or insulting language
Thank You!
Thank you for taking the time to flag this posting; we review flagged postings on a regular basis.
closeLack of Clarity over Review of the Private Sector
Posted by domdiep on 29 Aug 2012 at 23:19 GMT
In their recent article (Comparative Performance of Private and Public Healthcare Systems in Low- and Middle-Income Countries: A Systematic Review. PLoS Medicine, 2012 9(6) e1001244) Basu et al. acknowledge the highly contentious debate that exists regarding the appropriate scale of public and private health sectors in low- and middle-income countries, and set out to review the evidence on the relative performance of these sectors on each essential health system element identified by the World Health Organization.
The evidence presented in their review draws attention to the complexity of such a comparison across countries, health areas, and types of private healthcare provider. For example, how to disaggregate information on formal vs. informal private providers in a useful way? They find wide variety in the relative quality and performance of public and private providers. Not surprisingly, the authors highlight the need for more rigorous comparative research prior to making determinations about the proper role of public or private sector health services. Given the lack of strong data, the authors conclude that there is insufficient evidence to support assertions of greater efficiency and effectiveness in the private sector.
This conclusion is broadly in line with prior reviews of healthcare delivery systems that show distinct strengths and weaknesses within both public and private sectors (Berendes et al. 2011; Montagu et al. 2011). However despite the good intentions of their work, the review conclusions are of limited utility due to a number of methodological weaknesses. The authors’ lack of transparency regarding their review process – particularly the methods used for including papers and assessing data – call into question the validity of the findings and strength of the conclusions.
Of greatest concern is the authors’ approach to selecting studies for inclusion in the review. The accurate assessment of the relative performance by private and public healthcare delivery systems requires analysis of only those studies that directly compare these two sectors. This would seem obvious, and is a stated intention of the paper, however the study inclusion criteria do not limit the review to comparative studies, and the authors include many studies that evaluate the performance of a single sector on dimensions of quality of care, patient outcomes, fairness and equity, and efficiency.
The resulting narrative summary is therefore misleading. To demonstrate that the private sector has poorer patient outcomes, the authors point to a study in Botswana on the impact of out-of-pocket costs for HAART (Bisson et al. 2006). However, this study was conducted exclusively within a private clinic and its conclusions cannot support the relative cost or outcomes of private vs. public facilities. Similarly, the authors use a study on infertility treatment services in Brazil to demonstrate lower service availability in the public sector; despite the fact that the study does not evaluate private service availability (Makuch et al. 2010). Over-reaching conclusions such as these, and the inclusion of both comparative and, selectively we presume, non-comparative studies, suggests biases in the review, a perspective which is supported in other areas of the paper. What can we make of a statement such as: “nine retrospective chart reviews and survey-based studies found that diagnostic accuracy and adherence to medical management standards were worse among private than public sector care providers”, when the papers referenced include studies which show no such comparison (Auer et al. 2006, Udwadia et al. 2010)?
Given the low quality of health care in low- and middle-income countries, it is clear that interventions to improve the equity and efficacy of healthcare delivery systems are needed. The contentious political debate about the role of private and public healthcare sectors in strengthening overall healthcare delivery requires impartial research on the relative performance of these sectors. The paper by Basu et al. is a commendable effort to add to what is a thin body of knowledge. We do not disagree with the conclusions, but we are dismayed that they are reached on the basis of insufficient and non-transparent data, weakening them significantly. Because the inclusion criteria used by Basu et al. are not clear, and their mixing of both comparative and single-sector data appears to be agenda-driven, much of the value of this paper is lost when read by an informed audience. This is a great shame, but effectively puts the challenge to researchers around the world to conduct a more rigorous version of this analysis to usefully inform policy makers, funders, and other academics.
Sincerely,
Dominic Montagu and Naomi Beyeler
Private Sector Healthcare Initiative, Global Health Group, University of California, San Francisco
References:
Auer C et al. (2006). Diagnosis and Management of Tuberculosis by Private Practitioners in Manila, Philippines. Health Policy, 77:172 – 181.
Basu J et al. (2012). Comparative Performance of Private and Public Healthcare Systems in Low- and Middle-Income Countries: A Systematic Review. PLoS Medicine, 9(6) e1001244
Berndes S. et al. (2011) Quality of Private and Public Ambulatory Health Care in Low and Middle Income Countries: Systematic Review of Comparative Studies. PLoS Med 8(4): e1000433.
Bisson, GP et al. (2006). Out-of-Pocket Costs of HAART Limit HIV Treatment Responses in Botswana’s Private Sector. AIDS, 20(9):1333 – 1336.
Makuch, MY et al. (2010). Low Priority Level for Infertility Services Within the Public Health Sector: A Brazilian Case Study. Human Reproduction, 25(2):430 – 435.
Montagu D et al. (2011) Private versus public strategies for health service provision for improving health outcomes in resource-limited settings. San Francisco, CA: Global Health Sciences, University of California, San Francisco, ISBN: 978-1-907345-18-0
Udwadia ZF et al. (2010). Tuberculosis Management by Private Practitioners in Mumbai, India: Has Anything Changed in Two Decades? PLoSONE 5(8): e12023.
RE: Lack of Clarity over Review of the Private Sector
sanjaybasu replied to domdiep on 07 Sep 2012 at 22:36 GMT
We appreciate the commentary from our colleagues and actually agree with their conclusions concerning data quality, which was a major conclusion from our article. However, we'd like to clarify misunderstandings of the study's methods and respond to their comments, which are at times inaccurate.
First, we were struck by the undertone of their commentary, and specifically wondered why these commentators appeared so vexed by our results as to curiously alter the nature of our review when making their claims. When we investigated further, it appears that these commentators have not disclosed notable conflicts of interest—for example, they have been funded by Exxon and related groups to focus on, and apparently promote, the private sector delivery of healthcare in developing countries [1,2,3,4].
Second, the authors’ claim that a review should be isolated to studies which directly compare public and private sectors; they took several citations out of context to claim that we inaccurately cited them, without noting that we compared data between studies, not just data within studies. We specifically noted in the introduction to our manuscript that a parallel review published in this journal was incorporating only internally comparative studies [5], and our review would focus on a broader field of studies. Indeed, private-sector proponents had previously criticized academic scientists for excluding documents from private sector and development institutions, which often only contain information on the private sector. A peer reviewer of our study noted that the study “included a wider range of articles, which is perhaps a better representation of what is currently known about this area. “ Simply isolating a review to only documents with internal comparison of two systems would of course obviate some of the utility of systematic reviews, which is to provide comparisons among multiple data sources. For instance, if two hospitals serve a similar catchment population or are supposed to follow a standard international protocol, we would expect similar outcomes from them for a given healthcare metric, or to be able to compare them against an international goal. In their response to our article, the commentators take references out of context when we performed such comparisons, and instead presented citations inappropriately to claim inaccuracy in the analysis.
Next, the commentators suggest the study lacks clarity and transparency, but in fact our manuscript fulfills all PRISMA systematic review criteria (the highest international standard for systematic reviews [6]) and includes a replicated protocol and full inclusion and exclusion criteria as well as complete review process details. It was peer-reviewed by four reviewers who noted “the authors appear to have made great efforts to perform an objective review through both the use of rigorous and standardized methods, a data extraction auditing process, transparency of reporting, and carefully developed and worded conclusions.”
We do, however, agree with the commentators when they suggest that contentious debate seems to derail data-driven discussions about the private and public sectors. We found that these commentators appear to make similar commentaries in other journals when study results disagree with their agenda; for instance, these commentators previously attacked a report in the BMJ, making claims that were later found to be grossly distorted [7]. The commentators’ recent reports about private versus public sector outcomes also seems to include marked misuse of data and failure to report conflicts of interest [8].
For example, the authors cite two studies and claim that they demonstrate “the higher cost of health services provision in the public sector.” The first was a study by Eggleston and colleagues [9], which contained no data, much less comparative data, on the costs of care in public versus private sector. Montagu and colleagues base their inference of higher costs on the comment in the paper that government hospitals, which are substantially larger and had more beds occupied, have more expensive medical equipment and physicians. A per capita cost is impossible to deduce, particularly as no cost data was presented in the referenced article. For the authors to claim that costs are shown to be higher based on such reports is incredible and betrays profound bias.
The second paper cited for this conclusion presented virtually no information at all about costs and resource use, unless the 65% default rate from treatment in the private sector was inferred to somehow represent cost-savings [10]. The authors went on to summarize this study has having “equal outcomes” despite, as the title suggests, statistically worse outcomes in private compared with public clinics (35% treatment success compared with 82% treatment success for smear positive TB patients, 55% versus 95% for smear negative TB patients).
Comparative studies indeed are valuable but not when their results are imagined or reversed. These are but two examples of what we find to be a troubling broader agenda of these commentators. The commentators should shed light on their own biases, and truthfully divulge their financial and political conflicts of interest when making their commentaries.
References
[1] http://globalhealthscienc...
[2] http://globalhealthscienc...
[3] http://www.sf4health.org/
[4] http://www.ps4h.org
[5] Berendes S, Heywood P, Oliver S, Garner P (2011) Quality of private and public ambulatory health care in low and middle income countries: systematic review of comparative studies. PLoS Med. 8. doi:10.1371/journal.pmed.1000433.
[6] http://www.plosmedicine.o...
[7] http://www.bmj.com/rapid-...
[8] Montagu D et al. (2011) Private versus public strategies for health service provision for improving health outcomes in resource-limited settings. San Francisco, CA: Global Health Sciences, University of California, San Francisco, ISBN: 978-1-907345-18-0.
[9] Eggleston K et al. (2010) Comparing public and private hospitals in China: Evidence from Guandong. BMC Health Serv Res; 10: 76.
[10] Lonnroth K et al. (2003) Private tuberculosis care provision associated with poor treatment outcome: comparative study of a semi-private lung clinic and the NTP in two urban districts in Ho Chi Minh City, Vietnam. Int J Tuberc Lung Dis 7(2): 165-71.