Conflict of Interest Policies for Organizations Producing a Large Number of Clinical Practice Guidelines

Background Conflict of interest (COI) of clinical practice guideline (CPG) sponsors and authors is an important potential source of bias in CPG development. The objectives of this study were to describe the COI policies for organizations currently producing a significant number of CPGs, and to determine if these policies meet 2011 Institute of Medicine (IOM) standards. Methodology/Principal Findings We identified organizations with five or more guidelines listed in the National Guideline Clearinghouse between January 1, 2009 and November 5, 2010. We obtained the COI policy for each organization from publicly accessible sources, most often the organization's website, and compared those polices to IOM standards related to COI. 37 organizations fulfilled our inclusion criteria, of which 17 (46%) had a COI policy directly related to CPGs. These COI policies varied widely with respect to types of COI addressed, from whom disclosures were collected, monetary thresholds for disclosure, approaches to management, and updating requirements. Not one organization's policy adhered to all seven of the IOM standards that were examined, and nine organizations did not meet a single one of the standards. Conclusions/Significance COI policies among organizations producing a large number of CPGs currently do not measure up to IOM standards related to COI disclosure and management. CPG developers need to make significant improvements in these policies and their implementation in order to optimize the quality and credibility of their guidelines.


Introduction
Clinical practice guidelines (CPGs) are ''statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options,'' according to the Institute of Medicine (IOM) [1]. CPGs can influence the care delivered by a large number of healthcare providers and thus the outcomes of patients [2], so their quality is critically important. High-quality guidelines have the potential to promote the use of effective clinical services, decrease undesirable practice variation, reduce the use of services that are of minimal or questionable value, increase the use of effective but underused services, and target services to populations most likely to benefit [3]. Highquality guidelines are valid (unbiased), reproducible, clinically applicable to the populations of interest, flexible, clearly presented, developed through a multidisciplinary process, reviewed on a regular schedule, and well-documented [3]. CPGs may improve processes of care [2,4,5], however data on the effectiveness of CPGs on health outcomes are sparse and conflicting [1,2,[4][5][6].
Conflict of interest (COI) is an important potential source of bias in the development of CPGs. A COI is a set of conditions in which professional judgment concerning a primary interest (such as the health and well being of a patient or the validity of research), is unduly influenced by a secondary interest [7]. There are data that suggest an association between author or funder COI and study outcomes [8][9][10][11] and between COI and systematic review conclusions [12]. Data on the prevalence of industry relationships of guideline sponsors and authors are limited and dated, however [13][14][15][16], and evidence of the effect of these relationships on guideline recommendations is confined to case studies [17]. Nonfinancial interests may also be powerful motivators for guideline authors [18], including the advancement of medical science, career advancement, fulfillment of a desire to do good, opportunity to publish, notoriety, future success in obtaining funding for research, and increased sense of self worth [19].
Recent IOM reports highlight the importance of COI in healthcare [20], systematic reviews [21], and CPGs [1], and provide standards for disclosure and management of COI. The extent to which current CPG organizational policies align with IOM standards has not yet been reported. The objective of this study was to describe the COI policies of organizations currently producing a large number of CPGs, to compare these policies to the recent IOM standards related to COI in CPG development [1], and to identify characteristics of organizations that correlate with policies that meet these standards.

Methods
We identified organizations that had five or more guidelines listed in the National Guideline Clearinghouse (NGC) (http:// www.guideline.gov) that were published between January 1, 2009 and the date of our search (November 5, 2010). We selected this cohort in order to examine the policies of organizations that have the potential for a major effect on physician behavior and patient outcomes. The NGC, funded by the Agency for Healthcare Research and Quality (AHRQ), was established in January of 1999, ''to provide physicians and other health professionals, health care providers, health plans, integrated delivery systems, purchasers, and others an accessible mechanism for obtaining objective, detailed information on clinical practice guidelines and to further their dissemination, implementation, and use'' [22]. The inclusion criteria for guidelines within the NGC are: 1) The CPG contains systematically developed statements that include recommendations, strategies, or information that assists physicians and/or other health care practitioners and patients to make decisions about appropriate health care.

2)
The CPG was produced under the auspices of medical specialty associations; relevant professional societies, public or private organizations, government agencies at the Federal, State, or local level; or health care organizations or plans.

3)
Corroborating documentation can be produced and verified that a systematic search and review of existing scientific evidence was performed during the guideline development.

4)
The full text guideline is available upon request in the English language.

5)
The guideline is current, in that it was developed, reviewed, or revised within the last 5 years.
Two authors (HKH, BUB) independently searched for each organization's COI policy on the organization's website and through a general internet search. Only policies that were specific to CPG development were examined further: the policy was required to either include a statement specifically regarding CPGs or be included in a CPG development handbook, and at a minimum address COI for CPG panel members. Policies that covered a variety of organizational activities were not examined further because such general policies did not address adequately address the issues of COI relevant to CPG development. If the guideline developer was US-based, the authors also determined if the CPG organization was signed onto the Council of Medical Specialty Societies' (CMSS) Code for Interactions with Companies, by examining the CMSS website [23]. CMSS, a collaboration of almost 40 US-based medical specialty organizations, issued a revised code in March, 2011 to provide guidance for members on how to minimize and manage conflicts of industry that arise from their relationships with companies that produce drugs, devices, and other healthcare interventions. Underpinned by the principles of independence and transparency, the CMSS code provides specific guidance for the conduct of educational programs, business interactions, receipt of funding support, sponsorship, and advocacy, among others.
We did not include information contained within the CPG or in the summary provided by the NGC as the former was invariably insufficient to assess the organization's policy and the latter was information provided by the organization to the NGC after the guideline was completed. We did not contact organizations for additional information, nor did we try to obtain information from organizations with protected websites for members only, because we wanted to include only information that was publicly available to users of CPGs, and our past experiences trying to obtain unpublished COI policies suggested that such an approach was not likely to be productive.
One author abstracted information on each organization's COI policy into a pre-specified template in ExcelH, and those data were checked by a second author. The data abstracted from each policy included when disclosures were obtained, from whom, over what time period, the types of COI (financial or intellectual), and how disclosures were managed. Data were synthesized across guidelines using descriptive statistics and a qualitative synthesis. We used a logistic regression model to explore the association between whether an organization had a CPG-specific COI policy and pre-specified predictors included type of organization, the number of CPGs produced between 2009 and our search date, and whether or not the organization was based in the US.
We compared the COI policy of each guideline organization to the COI-related standards in the 2011 IOM report (Table 1) [1] for the purpose of describing the degree to which organizations are already meeting these standards and the gaps that need to be addressed. We selected the IOM standards as a comparator for our cohort of guidelines because this report is the most recent, large-scale compendium of evidence and recommendations for guideline development in healthcare that we are aware of. We are aware of extensive efforts across North American guideline groups to implement these standards. We assessed the seven IOM standards that were deemed evaluable at the level of the organizational policy (standards 2.1, 2.2a, 2.2b, 2.3, 2.4a, 2.4c, 2.4d). We did not assess standards that were best assessed at the level of the CPG (standard 1.1) or standards that were unlikely to be included in an organization's COI policy (standard 2.4b, 2.4e).
Three authors (SLN, HKH, BUB) independently made a determination as to whether each organization's policy met each IOM standard and then consensus was achieved through discussion. Inter-rater reliability was assessed by using Kappa statistics for each standard and overall for the seven standards combined.
We then explored pre-specified predictors for organizations fulfilling these IOM standards, using the same variables that we used in our examination of whether each organization had a COI policy or not. The association between meeting each standard or at least one standard and the predictors was assessed using a logistic regression model. The association between meeting any IOM standard and the predictors was assessed using a logistic generalized estimating equation (GEE) model, accounting for correlation among multiple standards within each organization. The association between the proportion of standards met in each organization and the predictors was explored using a linear regression model.

Characteristics of Organizations Producing CPGs
596 CPGs were published in NGC between 2009 and the date of our search in 2010, by 118 organizations listing at least one guideline. Of these organizations, 37 fulfilled our inclusion criteria, publishing 392 guidelines (66% of the total of 596 CPGs) ( Table 2).
The majority of the 37 organizations (86.5%) published fewer than 20 guidelines, while one organization published more than 40 [24]. Professional (56.7%) and government (29.7%) organizations made up the majority of the 37 organizations.

Conflict of Interest Policies
Seventeen of the 37 organizations (45.9%) had a COI policy that was publicly accessible and applicable to the development of CPGs ( Table 3). The remaining organizations either did not have a policy or it was not publicly available (43.2%), or the only identified policy was not explicitly related to CPG development (10.8%). Organizational characteristics were not significantly associated with the presence of a COI policy, including the production of fewer CPGs (p = 0.24), governmental compared to a professional organization (p = 0.65), and based in the US compared to other countries (p = 0.17).
The types of non-financial COI described in the policies also varied ( Table 3). Eleven organizations (65% of 17) specifically required that intellectual or other non-financial COI be disclosed [25,26,28,29,[32][33][34][35][36][37][38]. Four of these organizations did not provide a definition for non-financial COI, but rather used terms such as ''intellectual'' or ''other related'' conflicts [25,29,35,38]. The remaining seven organizations provided more detailed, and very heterogeneous, descriptions [26,28,32-34,36,37]. One organiza-tion required the disclosure of competing beliefs, academic institutions, societies and publications [28], while another organization required the reporting of substantial career efforts (effort devoted to procedures currently performed in clinical practice) [29]. One organization [26] used the definition of intellectual COI as proposed by Guyatt and colleagues [18] where an attachment to a specific viewpoint as developed through academic and other activities could unduly affect an individual's judgment about a specific guideline recommendation.
Disclosures are collected at various time points across the organizations, (Table 4) most commonly at the time of appointment to a guideline panel or prior to beginning deliberations. Two organizations addressed updating of COI, either throughout the CPG process [25] or just prior to CPG publication [29]. The reporting period for COI ranged between the prior 12 to 60 months (median 12 months).
Seven organizations (41.2%) limited the disclosure to only those that were relevant to the content of the CPG (Table 2) [25,28,29,[32][33][34]36]. In most organizations relevance was not defined, however, although three organizations that shared a common COI policy require disclosure for personal payments, ''which may either relate to the manufacturer or owner of a product or service being evaluated, in which case it is regarded as 'specific' or to the industry or sector from which the product or service comes, in which case it is regarded as 'non-specific' '' [32][33][34]. Table 1. Institute of Medicine standards for developing trustworthy clinical practice guidelines (2011) [1].

IOM Standard
1.1: * The process by which a CPG is developed and funded should be detailed explicitly and be publicly accessible.
2.1:{ Prior to selection of the CPG group, individuals being considered for membership should declare all interests and activities potentially resulting in COI with development group activity, by written disclosure to those convening the guideline development group. Disclosure should reflect all current and planned commercial, noncommercial, intellectual, institutional, and patient/public activities pertinent to the potential scope of the CPG.
2.2a: All COI of each GDG member should be reported and discussed by the prospective development group prior to the onset of his or her work.
2.2b: Each panel member should explain how his or her COI could influence the CPG development process or specific recommendations.
2.3: ¥ Members of the GDG should divest themselves of financial investments they or their family members have in, and not participate in marketing activities or advisory boards of, entities whose interests could be affected by CPG recommendations. 2.4a: Whenever possible, GDG members should not have COI.
2.4b:* In some circumstance, a GDG may not be able to perform its work without members who have COI, such as relevant clinical specialists who receive a substantial portion of their income from services pertinent to the CPG.
2.4c: Members with COI should represent not more than a minority of the guideline development group.
2.4d: The chair or co-chairs should not be a person(s) with COI.

Management of Disclosures
Eleven organizations (64.7% of 17) delineated strategies to manage COI after disclosure, with considerable variation in these approaches (Table 4). Advisory panels (5.8%), the CPG committee (23.5%), committee chairs (5.8%), or another party (e.g., funder of the CPG) (17.6%) assessed the disclosures. In some cases the same body of individuals assessed and managed the disclosures (17.6%). More commonly, however, a party separate from those involved in the assessment of disclosures devised a management plan: advisory panel (17.6%), the CPG chair (5.8%), or an ethics committee (11.8%). For 47% of organizations, a significant conflict resulted in recusal from the CPG development group or abstention from the final recommendation process. One policy detailed penalties for non-compliance with the organization's policy [28], while two other organizations stated that non-compliance would lead to disciplinary action [27,29].

IOM Standards for Reporting Conflicts of Interest in Clinical Practice Guidelines
None of the 17 organizations with a CPG-specific COI policy met all seven IOM standards specific to COI ( Table 5). The policies of eight organizations [25,26,28,29,[32][33][34]36,38], however, met at least one of the seven standards, with five organizations meeting at least four [25,29,[32][33][34]. The standard most frequently met was 2.1 (n = 7) and 2.2a (n = 8). None of the organization characteristics was statistically significantly associated with meeting any standard, including number of CPGs listed in NGC by that organization in our 2009 to 2010 cohort (p = 0.21); professional organization compared to government sponsor (p = 0.81); and USbased organization versus other countries (p = 0.85). Regression models examining these same predictors for meeting each individual standard, at least one standard, as well as the proportion of standards met for each organization were also not significant (p,0.05).
The agreement among the three assessors for all seven IOM standards across all 17 policies was 94.1% (95% CI, 88.3% to 97.6%). Agreement for specific standards ranged from 76.5% (standard 4a) to 100.0% (standards 2b, 3, 4d). The overall Kappa statistic for inter-rater reliability among the three reviewers was 0.88.

Discussion
Fewer than half of organizations that have recently listed a large number of guidelines in the NGC have a COI policy directly related to CPGs. Of organizations with such policies, these vary widely with respect to types of COI addressed, from whom disclosures are collected, monetary thresholds for disclosure, approaches to management, and updating requirements. Not one organization's policy adhered to all seven of the IOM 2011 standards applicable to COI policies for CPGs, and approximately half of these organizations do not adhere to a single one of these standards.
There is a need for extensive improvements in COI policies for organizations producing CPGs. Every developer must have a COI policy specific to CPGs: a general COI policy for the organization or a policy that relates to research subjects, for example, is not adequate to address the issues of secondary interests relevant to guideline development. Guidelines involve multiple groups of individuals and processes, and the COI of all of these must be disclosed: authors of the systematic review, the guideline panel members, and other individuals involved in the formulation and approval processes. COI policies for CPGs must also address  AASLD [26] If a person or their immediate family might gain financially or non-financially from the actions of the person while acting on behalf of AASLD. If a person who serves AASLD also serves another organization, whose mission overlaps that of AASLD, in a capacity that can potentially put that organization in direct competition with AASLD thereby impeding the ability of AASLD to fulfill its mission. Conflicts of interest are further defined as any circumstances that create a risk that professional judgment or actions regarding a primary interest will be unduly influenced by a secondary interest. Primary interests are those associated with the stated mission of the AASLD. Secondary interests may be financial or non-financial in nature There are important implications of our findings for users of CPGs. Given the evidence that financial relationships of researchers are associated with study findings [8][9][10][11]39] and that guideline panel composition correlates with panel recommendations [40], the reader needs to have ready access to accurate and up-to-date disclosures, and be assured that secondary interests have been appropriately managed.
Nonfinancial COI may be an important source of bias, and twothirds of organizations make some mention of this type of COI in their policies. Definitions are variable, however, and often vague and difficult to interpret, potentially leading to inconsistent and incomplete reporting. The current knowledge base on the impact of nonfinancial COI on decision making is sparse, so that the development of evidence-based guidance on the disclosure and management of nonfinancial COI is difficult at the present time.
The IOM standards related to COI do not address all of the important issues for CPG developers and users. In particular, these standards do not address accessibility of COI policies, the relevance of secondary interests to the primary interest, and the optimal presentation of disclosures. We often encountered a great deal of difficulty locating COI policies: these policies should be readily accessible from the guideline organization's website. Policies on disclosures should address both relevance and an appropriate level of detail. Inadequate guidance within COI policies on these issues may reflect lack of evidence on both the relationship between specific types of conflicts and risk of bias, and what constitutes optimal disclosures for the reader. IOM standards also do not address accountability for accurate disclosures and adherence to the policy within the organization. In other words, adherence to these seven IOM standards does not ensure an optimal COI policy.
There are limitations to our approach. Most importantly, we did not assess if and how these policies were applied during CPG development and publication. Nor did we examine the impact of each organization's policy on their processes, disclosures, recommendations, and the use and interpretation of the guidelines by healthcare providers. Our descriptive work provides the basis for these important next steps in evaluating the nature and effect of COI policies in CPG development.
It is possible that our summary of these organizations' policies is not up to date. Given the current attention to COI policies by biomedical journals [41,42] and organizations producing CPGs [1,23], the COI policies for the organizations that we examined may have changed between development of the CPGs listed on the NGC website, our examination of the policy, and the present. Our information on specific policies may be incomplete in spite of having two independent persons identifying COI policies due to difficulty locating policies in various locations in linked websites.
We did not contact organizations for additional, unpublished information, nor did we try to obtain information from organizations with protected websites. We felt that if the guideline was in the public domain the COI policy should be there also, *Financial COI declared in ranges (less than $10,000, $10,000 to $25,000, and greater than $25,000).

{
Having a financial interest of greater than $5,000 in an individual pharmaceutical company or an individual manufacturer of medical instruments, devices or equipment; acceptance of anything with a value greater than $25 for their personal use from an individual pharmaceutical company or an individual manufacturer of medical instruments, devices or equipment.
{ Required to disclosure amounts greater than $10,000 per year or as determined through reference to public prices or other reasonable measure of fair market value, or represents more than 5% ownership interest in any single entity.  Table 1 Table 5. Organizational policies and the Institute of Medicine standards [1]. 3 (43%) 2 (29%) 2 (29%) 0 (0%) 0 (0%) 0 (0%) 28 (23%) Each cell depicts whether or not the organization's policy met all of the criteria for each standard. Abbreviations: COI, conflict of interest; INF, information not found; NR, not reported; see Table 2 for abbreviations for the guideline organizations. doi:10.1371/journal.pone.0037413.t005 given the presumed importance of transparency in COI processes and disclosures. We also encountered challenges in determining if an organization's COI policy was specific to CPGs. Two independent reviewers made this assessment and then came to consensus, but it is possible that some of the organizations that we assessed as not having a CPG-specific policy would consider themselves to have such a policy. We would counter that these policies need to be more transparent and to address the processes and individuals involved in CPGs specifically. The applicability of our findings to other organizations producing CPGs may be limited. The COI policies of organizations producing a small number of CPGs (less than 5) may differ from the policies that we examined. In addition, we focused only on CPGs published in English and those listed in NGC: the policies of other organizations may differ from our findings. The IOM standards were developed by a US-based group of experts, and thus encompass the American perspective on guideline topics, methodology, available resources, and the standards target U.S. patient populations and health care providers. The evidence base examined in the IOM report encompasses studies from a variety of international settings, and thus the IOM standards are likely applicable to guideline development in the Western world.
The IOM standards [1] were released after the publication of the CPGs examined herein. Thus our goal in comparing the IOM standards to existing organizational polices was to describe the current status of these policies and to identify areas where improvements are needed and not to criticize organizations for not having met these standards. Our reporting of the gaps in current approaches may aid organizations in planning updates of their COI policies and in comparing policies across organizations.
The IOM standards were not intended to be a quality assessment tool, thus implementing them as such was problematic. Standard 2.1 addresses both financial and nonfinancial COI, thus the two types of COI are not distinguished. Many of the standards are too vague to implement as quality assessment criteria: for example, standard 2.4a (Table 1).
COI policies among organizations producing a large number of CPGs currently do not measure up to IOM standards. Policymakers, guideline funders, sponsors, and developers, as well as users need to address and demand improvements. Patients and populations need trustworthy CPGs, and the accurate disclosure and subsequent management of COI is essential to achieve that goal.