Policies on Conflicts of Interest in Health Care Guideline Development: A Cross-Sectional Analysis

Objective To assess whether organisations that develop health care guidelines have conflict of interest (COI) policies and to review the content of the available COI policies. Methods Survey and content analysis of COI policies available in English, French, Spanish, and Italian conducted between September 2014 and June 2015. A 24-item data abstraction instrument was created on the basis of guideline development standards. Results The survey identified 29 organisations from 19 countries that met the inclusion criteria. From these organisations, 19 policies were eligible for inclusion in the content analysis. Over one-third of the policies (7/19, 37%) did not report or did not clearly report whether disclosure was a prerequisite for membership of the guideline panel. Strategies for the prevention of COI such as divestment were mentioned by only two organisations. Only 21% of policies (4/19) used criteria to determine whether an interest constitutes a COI and to assess the severity of the risk imposed. Conclusions The finding that some organisations, in contradiction of widely available standards, still do not have COI policies publicly available is concerning. Also troubling were the findings that some policies did not clearly report critical steps in obtaining, managing and communicating disclosure of relationships of interest. This in addition to the variability encountered in content and accessibility of COI policies may cause confusion and distrust among guideline users. It is in the interest of guideline users and developers to design an agreed-upon, comprehensive, clear, and accessible COI policy.


Introduction
The problem of conflict of interest (COI) in health care guidelines has received growing attention from health care guideline developers and users in recent years. Inadequate management of COI can result in bias and have important implications for public confidence in both the guidelines and the organisations that produce them [1][2][3][4][5]. In response to this problem, numerous recommendations for improving the identification and management of COI in guideline development have been issued [6][7][8][9][10][11][12]. The U.S. Institute of Medicine (IOM) and Guidelines International Network (G-I-N), a global network of individuals and organisations interested in guidelines [www.g-i-n.net/about-g-i-n], have both produced standards for creating trustworthy guidelines [13][14][15][16] which include guiding principles to identify and manage COI. The importance of recording and addressing COI is also reflected in the appraisal criteria of AGREE II [17], an international tool for assessing the quality and reporting of health care guidelines.
For guideline developers, a first step in addressing COI is to have relevant policies and procedures in place. To our knowledge only one previous study [30] has surveyed and examined the content of the COI policies of guideline developers. This cross-sectional study of 37 organisations producing a 'large' number of guidelines (defined as five or more guidelines listed in the National Guidelines Clearinghouse in [2009][2010] compared COI policies to IOM standards. Only 17 (17/37, 46%) of the guideline developers studied had a COI policy for guidelines in place and not one policy adhered to all 7 relevant IOM standards [14]. The study was limited to English-language guideline developers and nearly two-thirds of these organisations (24/37, 65%) were in the United States, with the remainder representing four other countries: Canada (5), the United Kingdom (5), New Zealand (2), and the Netherlands (1).
This study expands upon previous research to examine the COI policies of an international sample that includes non-English-language guideline developers.

Objectives
The aim of our study was to assess whether and how organisations address COI issues related to guideline development, and to identify and quantify problematic areas of underreporting.
We set out to capture a cross-sectional sample of worldwide guideline development organisations and to examine the content of their policies on COI available in English, French, Spanish and Italian through a predefined data abstraction instrument.

Selection of guideline development organisations
Between September 2014 and June 2015 we conducted an Internet search and content analysis of the COI policies of organisations who have a programme of health care guideline development covering a wide range of health topics i.e. a programme not limited to a particular disease or condition or to group or subgroup of diseases or conditions.
We obtained a preliminary list of 53 organisations by searching in the G-I-N database (http://www.g-i-n.net/membership/members-around-the-world) filtered by "activities" and "guideline development" (accessed September 25, 2014).
This list was integrated with a list of organisations obtained through an evidence inventory [31] of surveys of guideline developer organisations. We performed the evidence inventory to list what surveys were available on guideline developer organisations. Studies were sought from MEDLINE (1 January 2009 to 1 August 2014) and limited to English language (S1 Table). The search identified 217 articles. One author (CM) screened the titles and abstracts of all retrieved articles. A survey of European guideline developers was found and included [32] with one companion report [33]. We also included an article [34] which listed guideline handbooks/manuals to identify further guideline developers.
From these studies 60 additional organisations were identified, of which 21 were duplicates, leaving a sample of 92 organisations (S2 Table). In order to identify a manageable subsample of organisations that had the potential to produce guidelines with the most wide-ranging impacts on health, we pragmatically restricted our sample further to exclude organisations whose guideline programmes are limited to single disease areas or single health professions.

Identification of COI policies
We searched the websites of included organisations for documents that described the guideline development process (manuals, handbooks, methodology articles, webpages), disclosure of relationships of interest (DOI) forms, and any other available documents that mentioned COI issues related to guideline development and available in English, French, Italian or Spanish, with no date restrictions. We did not contact organisations directly to obtain additional information, but used only information publicly available.
We aimed to include documents that addressed COI in guideline development and that provided data for at least two of the items included in each of the following overarching items of the data abstraction instrument: "Information required for disclosure of financial and nonfinancial relationships" and "Procedures for collecting, reviewing disclosure of relationships of interest and managing conflict of interest" (Table 1). We called a document meeting these criteria a "COI policy". In the content analysis we distinguished the term "relationship (financial and nonfinancial) of interest" and "conflict of interest". The term "relationship of interest" refers to any interest or activity requested to be declared that may be judged to constitute a COI. Consequently we used the term "disclosure of relationships of interest" to indicate the step that precedes the review of DOI.

Data extraction and analysis
We developed a 24-item data abstraction instrument (Table 1) based on recommendations and standards of COI proposed by the IOM [13,14] and G-I-N [15]. The types of financial relationships and nonfinancial relationships considered in this study were based on those listed by IOM [13,14] and G-I-N [15].
One author (CM) abstracted information on each organisation and each COI policy into the predefined template and those data were checked by a second author (CF, ST). When there was disagreement regarding the content of a particular COI policy this was resolved by discussion. If the policy was updated during the period of data collection, we updated the abstracted information. Where disclosure forms were publicly available, data from these were integrated with the data extracted from the policies. Where inconsistencies were found, the information in the policy prevailed.

Results
Twenty-six organisations were excluded from the 92 identified as their remit was restricted to single disease specialties or single professional groups (S2 Table). Of the 66 remaining organisations, 11 were subsequently excluded because they did not provide documents on guideline development in English, French, Italian, or Spanish; 15 because they did not provide publicly accessible documents on guideline development or COI issue guideline related; 5 because their website was not found or not accessible and 6 because they were not guideline developers.
The documents of the remaining 29 organisations (Table 2) were assessed for the presence of a COI policy according to our definition. Of these organisations 21 were governmental, 4 were not for profit and 4 were professional associations. Ten organisations (10/29, 34%) were excluded from content analysis for the following reasons: 8 because their documents did not mention or provided insufficient information on COI, 1 because its handbook referred to an included parent organisation's COI policy and 1 because its handbook was superseded by the handbook of its collaborating organisation. In the end 19 COI policies were included in the      [49,58,62] with reference to the COI policy posted in the website. The remainder of the organisations included information on COI policy on a webpage as a flow chart [43] or in a methodological article posted on the organisation's website [35,40] ( Table 2). The date of the COI policy was determined by considering the date of the document itself if published as stand alone document or of the corresponding date of the handbook/manual, methodological article, or webpage where the COI policy was included. The date of the 18

Disclosure of relationships of interest
All included policies outlined categories of financial relationships but characterization of specific types of nonfinancial relationships was lacking in some cases [39,40,43,47,63] (4/19, 21%) ( Table 4). Policies varied in the categories of information that need to be disclosed and in whether or not disclosures were limited to information relevant to the content/topic of the guideline.  Table 3. Definition of conflict of interest by organisation.

Organisation Definition of conflict of interest
ACP [35] "Potential financial or nonfinancial conflicts of interest that refer to relationships that a reasonable reader of a guideline would wish to know about and that if not disclosed could compromise the interpretation of the ACP guideline" CTFPHC [37] Not reported CDC-ACIP [39,40] Not reported CENETEC [41] Not reported FMSD [43] Not reported IETS [44] The activities that may create potential conflicts of interest occur in those circumstances where professional judgment on a primary interest, such as patient's welfare or the validity of research, may be influenced by a secondary interest, such as financial gain, prestige, personal or professional advancement AWMF [45] Not reported GS [47] The activities that may create potential conflicts of interest occur in those circumstances where professional judgment on a primary interest, such as patient's welfare or the validity of research, may be influenced by a secondary interest, such as financial gain, prestige, personal or professional advancement [69] HAS [48,49] The relationships of interests may give rise to the conflict of interest. These are therefore two distinct concepts. A conflict of interest arises from a situation in which the relationships of interest of a person may affect, by their nature or intensity, his impartiality or independence in the exercise of his mission regarding the matter under discussion NCEC [55,56] "In the context of the work of the NCEC, a COI is any interest that could result in bias in the work or decision making processes of the NCEC" NHMRC [57] "Financial interests: an interest must be declared as a potential conflict when benefits or losses either in money or in kind have occurred or may occur at a level that might reasonably be perceived to affect a person's judgment in relation to a fair decision about evidence and their participation in group decision making". "Other relationship: an interest must be declared as a potential conflict when a strong position or prejudice or familial connection or other relationship held by a person could reasonably, or be perceived to, affect a person's judgment in relation to fair decision about evidence and their participation in group decision-making including making an effort to arrive at a consensus" NICE [58,59] "A conflict of interest arises when the judgment of someone involved in the work of NICE may be compromised, by the financial or other considerations set out in this policy" SIGN [61,62] "Competing interests are defined as any interest of the person, their partners or close relatives (personal) or their department/employer/business (non-personal) which may potentially influence the content, including recommendations, of SIGN guidelines" SNLG [63] Not reported TGL [64] "Interest statements must comprise a declaration of any interests that may be capable of influencing advice or decisions relating to the operation or activities of TGL, or that may affect the integrity and reputation of TGL" GETS [65] The conflict of interest arises when an individual or organisation considers alternatives where interests or benefits coexist and there is a duality of commitment [70]. The conflict is evident when the option that provides personal benefits is selected at the detriment of the option more adherent to ethical principles and general interest HE-TU [66] "Any interest which may affect or may reasonably be perceived to affect, the expert objectivity and independence" USPSTF [67] Not reported WHO [68] "A conflict of interest is a set of circumstances that creates a risk that professional judgment or actions regarding a primary interest will be unduly influenced by a secondary interest" [14]. "Any interest declared by an expert that may affect or reasonably be perceived to affect the expert's objectivity and independence in providing advice to WHO" doi:10.1371/journal.pone.0166485.t003     Concerning the item "who is required to report on relationships of interest", all policies stated that a declaration is collected from members of the guideline development group/panel/ committee but a minority [37,44,58,59,61,62,68] (5/19, 26%) detailed additional disclosures from staff involved in a standard guideline development process, for example, guideline methodologists, systematic review team, reviewers and advisors. In contrast, policies were much more likely [35,37,39,40,44,45,48,49,57,58,59,61,62,[64][65][66][67][68] (14/19, 74%) to require the disclosure of close personal relationships.
Of those policies that described activities to record DOI, the original completed DOI forms were reported to be publicly accessible on the website and/or in the published guideline by seven organisations [37,44,47,48,49,57,64,66] (7/15, 47%) with some exceptions. For example, one organisation stated DOI will be made public on the organisation website but"exception to this requirement will be by Chief Executive Officer in appropriate circumstances" [57].
Two organisations published in the guideline and/or website a summary of the DOI information [61,62,68] while other two [55,56,67] kept this information confidential although one of these mentioned the possibility of access upon request under state law [67].

Discussion
We found that of the 29 organisations corresponding to our eligibility criteria we were not able to identify COI policies for approximately one in four organisations (8/29, 28%). Failure of guideline developer organisations to adopt a publicly accessible COI policies related to guideline development has also been reported elsewhere in literature. For example, in the above mentioned cross-sectional study of Norris et al. [30] only 46% of the 37 surveyed organisations had a COI policy directly related to health care guidelines. These findings along with the data from our study suggest that the absence of a COI policy might be one of the reason  [23,26,27,29] as well as the observed uneven adherence to current standards on COI in guidelines [18][19][20][21]. We also found that many organisations provided no or unclear information pertaining to some items of our data abstraction instrument. In our view, a particularly troubling area in which COI policies need to be improved is that of guideline panel selection and composition. More attention to this issue has been advocated to reduce of the potential for COI to create bias [10,12,28]. Nevertheless, in our investigation about one-third of the policies did not report or did not clearly report whether disclosure was a prerequisite for panel selection. Additionally, preventive strategy such as divestment [13,15] and prohibition [13] were rarely mentioned.
A second key area in which COI policies need to be improved is the management of the disclosed information, particularly regarding the practices of "the assessment of risk" which is defined as "practices and criteria used to determine if a relation of interest constitutes a COI and to assess the potential for harm of the COI identified" [13]. It is encouraging that the policies surveyed were directed at the most common relationships of interest in support of the identification of COI. However a policy for systematic and transparent management of the disclosed information was uncommon. Few organisations clearly stated what practices and criteria were used to determine whether a relation of interest constitutes a COI and to assess the potential for harm of the COI identified. The translation of the disclosed information into a decision or measures taken to limit the likelihood of undue influence of COI is a critical step. This step should be explicit and transparent to reduce the flexibility of the organisations in addressing financial and nonfinancial ties as well as to assure the ability of the informed reader to appraise the validity of decisions on COI issue.
In terms of the management and monitoring of the disclosed information IOM recommends that organisations should create a COI committee (Recommendation 3.1) [13]. In this regard we found that few organisations have felt the need or identified the necessary resources to have a dedicated infrastructure to act as a third party providing independent review of DOI and managing COI across all the phases of guideline development process.
Most organisations relied on the same group of individuals who develop the guidelines to handle the disclosed cases. This is of particular concern if considered in the context of the evidence of guideline chairs and panel members with a high proportion of conflicts [20,25]. Apart from an improved consistency in decisions taken across guidelines, in our view a "conflict of interest committee" could avoid administrative burden for the chair, co-chair and members of the guideline group and more importantly could consistently implement procedures for the monitoring and enforcement of the policies.
Furthermore our study underlines the issue of variation across policies in keeping with the findings of Norris et al. [30]. We found that the policies varied in their content, for example in the definition of COI (when provided), in the categories and details of the information required to be disclosed, in the time frame for disclosure, in establishing or not establishing a threshold, and in detailing who is required to declare a relationship of interest. Our sample included organisations from 13 countries and one intergovernmental organisation, so this variation in part probably reflects cultural, legal and administrative differences across countries as well as differences in organisational values and interests.
We noted that the policies were also variable in format and accessibility across the organisations, limiting the ability of the guideline user to obtain a clear and complete picture of the strategy to address COI throughout the guideline development process. An important finding is that less than half of the included organisations reported that the information from completed DOI forms was publicly accessible without restriction. Again, notwithstanding cultural, legal and administrative issues, the guideline user has the right to assess the completeness of the disclosure of COI as well as the consistency with which the policy is implemented.
This study contributes to the literature on the quality of guidelines by demonstrating, using an innovative sampling method and highly detailed data extraction, that COI remains an area that requires attention from guideline users and improvement from many guideline development organisations. The study confirms previous research that has drawn the same conclusions about COI from different samples of guidelines. The study also has some limitations that deserve comment.
First, organisations may have practices which are not fully reflected in the written COI policies or do have policy but not publicly available. However, we decided to rely only on information on COI policies publicly accessible on website since we consider accessibility an essential element of a transparent COI policy. As pointed out by IOM, COI policies should be comprehensible and accessible to the individuals and institutions that may be affected [13]. We recommend that the organisations that disseminate their own health care guidelines should provide public information on their strategies to identify and manage COI through an understandable and accessible policy.
Second, information about the policies and information about COI within the policies might be incomplete. We experienced several difficulties in locating COI policies from websites and in obtaining a complete and clear picture of the actual information required to be disclosed, the process of reviewing and the management of COI for the entire guideline development process. Some organisations published their COI policies as a complete self-contained document on their website, whereas others included information on identification and management of COI in their guideline methodology document. In some instances information was interspersed among diverse parts of the guideline methodological document, the disclosure of interest form and other supporting policies. Given the unstructured nature of the documents retrieved we checked as far as possible for presence of information about COI. We also encountered problems in the interpretation and coding of the information (e.g., on what should be disclosed) given that there is not an agreed taxonomy of terms relating to COI.
Third, we restricted our study to organisations that produce guidelines on a wide range of health topics and we excluded some professional organisations, which may limit the generalizability of our results. However, we would argue that our inclusion criteria would bias the sample towards organisations with greater resources for guideline development and therefore, if anything, our results would underestimate the extent of the problem of inadequate COI management. However our study has the strength of including 29 organisations from 19 different countries and one intergovernmental organisation in the sample and a content analysis of documents in four different languages. In terms of countries and languages represented, this may be a more representative sample of guideline development organisations than those obtained in other studies of COI in guidelines.

Conclusions
The finding that some organisations despite the recommendations and standards issued respectively in 2009 [13], 2011 [14] and 2012 [15] still do not have COI policies publicly available is concerning. Also troubling was the failure of many policies to clearly report on critical steps of obtaining, managing and communicating disclosure of relationships of interest. These problems in addition to the existing variation among policies may increase the scope for inconsistency in addressing COI issues; furthermore, unclear and incomplete COI policies may confuse readers, erode public confidence, and decrease trust in guidelines and the organisations that produce them. Thus, it is in the interest of guideline users and developers to promote strong adherence to the standards available on COI in conjunction with better scientific journal policies so that every guideline is completed by a harmonized, complete, understandable, and accessible COI policy.