Objectives

Our primary objectives were to describe the number (%) of guidelines that

1. Published the panel members’ COI disclosures
2. Did so in a distinct, clearly identified, paragraph
3. Reported, and described the potential impact COI of panel members on the guideline recommendations
4. Reported, and described the potential impact of the COI of the chairperson on the guideline recommendations
5. Reported and described the potential impact of a sponsor of the guideline on the recommendations
6. Described how the COI, when present, have been managed.

Our secondary objective was to examine reporting trends over time.

Study design

This study was designed as a cross-sectional analysis of systematically searched guidelines published from January 2007 to June 2018 in ten leading anaesthesia journals.

Inclusion criteria

We used InCites Journal Citation Reports (Thompson Reuters) to identify the ten anaesthesia journals, excluding pain journals, with the highest impact factor (IF) in 2017. These were: Anesthesiology (IF 6.523), British Journal of Anaesthesia (6.499), Anaesthesia (IF 5.431), Regional Anesthesia and Pain Medicine (IF 4.382), European Journal of Anaesthesiology (IF 3.979), Anesthesia & Analgesia (IF 3.463), International Journal of Obstetric Anesthesia (IF 3.404), Canadian Journal of Anesthesia (IF 3.377), Journal of Neurosurgical Anesthesiology (IF 3.238) and Minerva Anestesiologica (IF 2.693). We then searched in Medline (via Pubmed) for all guidelines that had been published in these journals between 1^st of January 2007 and 30^th of June 2018 using two search strategies. In the first search strategy, we used the criterion [name of the journal] and the filters “Guideline” or “Practice guideline”. In the second, we used the criterion [name of the journal] AND (guideline[Title] OR guidelines[Title] OR consensus statement[Title]). We included reports with the term “guideline” or “consensus statement” in the title, or that were described as a “guideline” in the subtitle. We excluded guidelines on methodological (for instance, statistical) topics. When a guideline on a given topic was updated every year, published in the same format and in the same journal, we included the latest update and excluded previous versions. When a guideline on a given topic was published as multiple articles in the same issue of a journal, stratified according to the studied population for example, we examined all articles but counted them as one unique guideline.

Setting

Searches were performed in July 2018 by one author (NE) and inclusion criteria were checked by a second author (DW). Data were extracted from July to September 2018 by one author (DW) and were checked by another (NE). Searches and access to guidelines were made through the library of the Medical Faculty of the University of Geneva, Geneva, Switzerland.

Data extraction

We searched for specific keywords in the electronic document using the “Search for” tool in .pdf documents. For example, to search for statements regarding sponsorship of a guideline, we used the keywords “funding”, “financial”, “sponsor” or “support”. When none of the keywords was identified in the text, the complete guideline was read to make sure the information had not been missed.

Variables

From each included guideline, we extracted information regarding the name of the journal, year of publication, number of panel members, name of the chairperson, and sources of sponsorship of the guideline. Sponsorship was categorised into four categories: academic or institutional, medical society, industry, or mixed (more than one source of funding disclosed). We specifically checked the six items described in Table 1. Additionally, we considered that a detailed description was needed on how COI and potential bias due to sponsorship of the guideline were managed if at least one panel member, or the chairperson of the guideline, declared having a COI, or if the guideline had received sponsorship. If the information was lacking for one or more of these items (for example, panel members declared lack of COI, but there was no statement regarding sponsorship of the guideline) we considered that the need for a detailed description of how COI were managed was “unclear”. If it was explicitly reported that none of the panellists had any COI and that no funding had been received, we considered that a description of how COI were managed was not needed.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Degrees of quality of reporting of conflicts of interest.

https://doi.org/10.1371/journal.pone.0212327.t001

Bias

To limit bias in the selection of guidelines and data extraction, two authors separately performed the search strategies and the data extraction (DW, NE). Any discrepancy in data extraction was discussed with the third author (MRT) until a consensual decision could be reached.

Study size

This investigation was designed as a descriptive study. There was no intention to search for associations. There was a pre-hoc assumption that the incidence of anaesthesia guidelines fulfilling all requirements of adequate reporting of COI would be low (<10%). In order to have results that may be generalisable to the entire field of anaesthesia, and to allow for examination of time trends, we decided to search in ten major anaesthesiology journals over a 12-year period.

Statistical methods

Categorical variables are described as numbers (percentages) and a summary proportion of each variable over the whole period of time is provided. For the time trend analyses, variables are reported stratified according to three periods of publication (2007–2010; 2011–2014; 2015–2018). For this analysis, we dichotomised all six items described in Table 1 into two groups: Group 1 “best case” and “adequate” reporting; Group 2 “ambiguous” and “worst case” reporting. We computed the odds of “best-case or adequate” versus “ambiguous or worst case” reporting for each item, and we computed odds ratios (OR) with 95% confidence intervals (CI) comparing each time period with the baseline (2007–2010). We performed a similar analysis by using year of publication as a continuous variable. All analyses presented are two-sided, alpha level was fixed at 5% and analyses were performed using STATA 15 (StataCorp, 4905 Lakeway Dr, College Station, Tx 77845, USA).

Selection of guidelines and descriptive data

Our two search strategies identified 463 references, of which we excluded 353 (Fig 1).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Flowchart or retrieved and eventually analysed guidelines.

https://doi.org/10.1371/journal.pone.0212327.g001

We eventually analysed 110 guidelines published from January 2007 to June 2018; 92% (101/110) were freely accessible. The number of guidelines published per year varied from 6 to 15 (median 9). The median number of panel members per guideline, including chairpersons, was 9 (interquartile range 7 to 13; min 1, max 34) (See S1 Table for a detailed description of all included guideline).

Primary outcomes

1. Accessibility of COI disclosures.

Seventy of 110 guidelines (64%) reported COI disclosures of panel members in the published guideline and 40 (36%) did not. Of the 40 guidelines that did not, one^E15 published them on the journal’s website (freely accessible), one^E10 stated that disclosures were available “on request”, and another ^E18 published them on the journal’s website, but the internet link was not available (Fig 2).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Disclosures of conflict of interest of panel members and chairs and sponsorship of guidelines.

COI: conflict of interest; *Guidelines describing how COI were managed.

https://doi.org/10.1371/journal.pone.0212327.g002

Trend over time of reporting

The proportion of guidelines reporting disclosed COI and sponsors of the guideline in the published report increased over time (Tables 2 and S2). The proportion of guidelines reporting disclosures of COI of panel members in the published report has significantly increased from 28% in the period 2007–2010, to 87% in the period 2015–2018, p-trend <0.001. The proportion of guidelines reporting presence or absence of sponsorship of the guideline also increased significantly over time from 13% (2007–2010) to 54% (2015–2018), p-trend<0.001. There was no evidence that, among the guidelines that published COI disclosures, the format or content of the declaration had changed over time (S2 Table). When considering all the 110 guidelines however, the odds of having panel members (or chairpersons) disclosing no COI, or disclosing COI with a clear explanation of how the COI may have influenced the results (compared to those not reporting COI or reporting COI without explanation), have increased over time. Also, the odds of reporting on how the COI were managed improved over time, but remained marginal (Table 2).

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Trends of reporting of conflicts of interest and funding over time.

https://doi.org/10.1371/journal.pone.0212327.t002

Main results

We set out to investigate how COI of panel members and sponsorship of guidelines have been reported in guidelines published in leading anaesthesia journals over the last 12 years. Although the proportion of guidelines that disclosed, in the published report, the COI of panel members has increased over time, less than half of them only did so in a distinct, clearly identifiable, paragraph. When COI were reported, it often remained unclear what the nature of the conflict was, and how it could have influenced the recommendations of the guidelines. Although half of the guidelines disclosed COI of panel members or the presence of a sponsor of the guideline, only seven described in the published report how the conflicts were managed. Finally, a minority of the panel members and chairpersons reported having no COI.

Results in context

Although the impact of sponsors on the conclusions of clinical trials has been demonstrated [14][15], the impact of sponsors and of COI of panel members on the recommendations of guidelines has not yet been formally shown [6][16]. Recent scandals that were related to COI of panel members [17][18], some of which have led to the retraction of guidelines [19], suggest that this topic is sensitive, and highlights that transparent and explicit disclosures of COI of panellists are required. The introduction in the US of the “Sunshine Act” in 2010, which aims to openly disclose the relationships between pharmaceutical companies and physicians, also illustrates the importance given to the transparent disclosures of COI [20]. Recommendations on the management of COI during the guideline process have been proposed by the Guideline International network (GIN) and the Committee on Conflict of Interest in Medical Research, Education, and Practice of the US Institute of Medicine. The aim of such recommendations is to limit biases and to preserve scientific integrity. Both recommendations suggest that the chairpersons should be free of any COI. A chairperson’s COI may impact on the recommendations by the biased selection of panel members who are known to support a desired outcome [21]. Both also recommend that COI of all panel members should be declared, with a clear description of how each of the COI has been managed during the guideline process. Finally, presence or absence of a sponsor of the guideline should be clearly reported, and the potential impact of the sponsor on the guidelines must be explained [8][22]. Despite these recommendations, several studies have highlighted weaknesses in the management of COI in guidelines, and have called for improvement [1][7][11][23].

Reporting of COI in guidelines has been the object of previous studies. Some have focused on the proportion of guidelines reporting (or not reporting) COI of panel members. In 2001, a study of guidelines in internal medicine that were published between 1994 et 1999, reported that only 3.7% had reported COI of panel members [24]. Ten years later this proportion had grown to 15% [25], and even fewer guidelines provided information on how the COI had been managed. The proportion of guidelines that disclose COI of panel members has been shown to have further increased from 2006 to 2016 [26][27]. These data are in line with the finding of the present study. Reporting of COI of panel members of guidelines in anaesthesiology has increased from 28% to about 87% during the last 12 years. Up to now, however, very little research has been performed to examine how “informative” disclosures were. Hilda Bastian, in 2016, highlighted the lack of clarity of COI disclosures in reports of clinical trials and underlined that the visibility of COI disclosures needed to be improved [26]. Alhazzani et al stressed that intellectual and institutional COI were still largely overlooked during guideline development [2]. The problem regarding the lack of attention given to the impact of sponsors and authors’ COI has been previously highlighted in the context of systematic reviews [28].

What does this study add?

The present study is the first to examine, from a reader’s point of view, the accessibility, transparency and clarity of information that can be derived from reporting of COI in guidelines in anaesthesia. COI are not systematically reported and when reported, statements very rarely provide a clear description of what the nature of the conflict was, and how it could have influenced the recommendations. Also, guidelines still rarely report on how COI were managed. Whether they were managed, but their management not reported, or not managed at all, remains unclear. We also show that although the reporting rate of COI has increased over the years, there is still room for improvement. For example, we were quite surprised by the diversity of the labels of the COI disclosures, even in one given journal. We suggest that COI disclosures should be presented in a separate paragraph, clearly identified as “conflicts of interest disclosures”. This work may be a basis for future recommendations on how COI ought to be reported in guidelines.

Strengths and limitations

One of the strengths of our study is that we examined systematically searched guidelines across ten major anaesthesiology journals over a 12-year period, regardless of the topic. Two authors extracted the data independently, to minimise the risk of data extraction errors and inconsistencies.

One limitation of our study is that we did not check for the veracity of the disclosures. About 25% of the guidelines reported that none of the panel members had any COI, which may not necessarily be true. In fact recent studies have suggested that authors often fail to report real COI [3][29][30]. Also, we have not attempted to differentiate between financial COI (sometimes called direct COI) and non-financial COI (sometimes called indirect COI or academic COI). It has been shown that non-financial COI are frequent and may be as important as financial ones [11]. We have not performed subgroup analyses according to the type of sponsorship (industry or medical societies). As medical societies may receive sponsorship from industry, the distinction becomes debatable [31]. We cannot exclude the possibility that we have missed a guideline. Interestingly, we realised that when searching for guidelines using the filter “guideline” in PubMed, some guidelines we knew of could not be identified. It remains unclear on which basis recommendations are indexed as “guidelines” in electronic databases. Some reports included in our analyses may not be regarded by everyone as “real” guidelines since some looked more like short reports of expert opinions. However, we included these reports on a consistency basis; all were published with a sub-heading “guideline” or the term “guideline” or “consensus statement” was in the title.

Where do we go from here—research agenda

It may be hypothesised that in the specific context of clinical practice guidelines, COI influence the direction or strength of recommendations [11]. However, this has never been formally demonstrated in the field of anaesthesia. Future research should examine the impact of COI of panel members, or of chairpersons, on the guideline recommendations. Also, recent concerns have focussed on non-financial COI [32]. It has been suggested that those are more likely to arise when panel members are specialists in the area under review, since they may have a strong opinion on the topic and may influence their peers [33]. The relative importance of both financial and non-financial COI requires further information [2]. Responsibility is an important issue that was highlighted in other settings and that needs further clarification [34][35]. Who should check for veracity, transparency and clarity of COI disclosures in guidelines? It has been suggested that journals should provide more detailed information on which COI need to be declared, and that authors must be warned about potential sanctions for those who fail to comply [36]. Others have suggested that the education of authors and reviewers should be improved [37], or that the chairpersons should be better trained on how to manage COI [23]. A more extreme view would be to refuse to publish guidelines where panellists have COI or when a description of how COI have been managed is lacking [26].