‘Trial Exegesis’: Methods for Synthesizing Clinical and Patient Reported Outcome (PRO) Data in Trials to Inform Clinical Practice. A Systematic Review

Purpose The CONSORT extension for patient reported outcomes (PROs) aims to improve reporting, but guidance on the optimal integration with clinical data is lacking. This study examines in detail the reporting of PROs and clinical data from randomized controlled trials (RCTs) in gastro-intestinal cancer to inform design and reporting of combined PRO and clinical data from trials to improve the ‘take home’ message for clinicians to use in practice. Materials and Methods The case study was undertaken in gastro-intestinal cancer trials. Well-conducted RCTs reporting PROs with validated instruments were identified and categorized into those combining PRO and clinical data in a single paper, or those separating data into linked primary and supplemental papers. Qualitative methods were developed to examine reporting of the critical interpretation of the trial results (trial exegesis) in the papers in relation of the PRO and clinical outcomes and applied to each publication category. Results were used to inform recommendations for practice. Results From 1917 screened abstracts, 49 high quality RCTs were identified reported in 36 combined and 15 linked primary and supplemental papers. In-depth analysis of manuscript text identified three categories for understanding trial exegesis: where authors reported a “detailed”, “general”, or absent PRO rationale and integrated interpretation of clinical and PRO results. A total of 11 (30%) and 6 (16%) combined papers reported “detailed” PRO rationale and integrated interpretation of results although only 2 (14%) and 1 (7%) primary papers achieved the same standard respectively. Supplemental papers provide better information with 11 (73%) and 3 (20%) achieving “detailed” rationale and integrated interpretation of results. Supplemental papers, however, were published a median of 20 months after the primary RCT data in lower impact factor journals (median 16.8 versus 5.2). Conclusion It is recommended that single papers, with detailed PRO rationale and integrated PRO and clinical data are published to optimize trial exegesis. Further work to examine whether this improves the use of PRO data to inform practice is needed.


Introduction
The updated Consolidated Standards of Reporting Trials (CONSORT) extension for patient reported outcomes (PROs) aims to facilitate the use of PRO data in health policy and practice through the transparent reporting of PROs in randomized controlled trials (RCTs) [1]. It makes recommendations for reporting of PRO instrument validity, presentation and handling of missing data and reporting of PRO sample size calculations, data analyses and results within the main text and abstract. The statement endorses reporting the rationale/hypotheses for PRO assessment and it highlights the need for integrated reporting of the PROs with the clinical findings of the paper (extensions and elaborations 2a, 2b and P20/21 and 22). These latter recommendations are particularly essential for critical interpretation of the trial, so called exegesis or a "take-home message", for clinicians to understand and use results in clinical practice.
The CONSORT extension provides illustrations of how to report these issues. For example, elaboration 2a and extension P2b state that authors should "briefly establish the rationale for including PROs and why specific outcomes were selected", and "report the rationale for the selection of specific patient-reported outcomes". Furthermore, the guidelines state (in items P20/21 and elaboration 22) that "the clinical significance of PRO results is often not discussed in RCT reports but should be interpreted in relation to other important clinical outcomes such as survival". Whilst this is helpful, the level of detail required for reporting the PRO data is unclear and this may have a detrimental impact on the overall clinically relevant trial conclusions.
This problem is further compounded by the way that PRO data are published. Some trials publish results in a single paper combined with clinical findings, which may limit full explanation of PRO results in the context of finite manuscript word limits. Publication of clinical and PROs separately is therefore attractive, however, practicing oncologists may be less likely to read the supplemental paper and thus not use PRO data in decision-making. The PRO CON-SORT statement does not provide guidance for PRO reporting within these different scenarios. Whether PRO data are published together with clinical outcomes or separately in two articles, there is a need for optimal reporting of clinical and PROs so that they can be used in clinical practice. The aim of this paper, therefore, is to explore current standards and make recommendations for reporting a combined PRO and clinical 'take home' message to use in reporting RCTs in oncology.

Materials and Methods
This study was conducted in two parts. Part 1: systematic identification of well-designed and conducted RCTs reporting PROs with validated instruments, categorisation of papers into combined PRO and clinical reports or linked primary and supplemental reports, and assessment of PRO reporting within each RCT (PRO CONSORT extension). Part 2: development of novel methods to examine the 'take home/trial exegesis' message of the RCT and application to the papers identified in Part 1.

Part 1(a) Identification of well-designed and conducted RCTs reporting PROs with validated instruments
Systematic review methodology was used to identify RCTs at a low risk of bias reporting PROs with validated instruments in radical treatments of gastro intestinal oncology. Full-text articles were obtained. Gastro-intestinal oncology trials of radical treatment were chosen because the research team were familiar with the clinical and PRO data in this area, and trials at a low risk of bias are examples of best practice.
Electronic searches were performed in MEDLINE, Embase and Cochrane databases using the OVID SP gateway and Cochrane library. Search terms for esophageal, gastric and colorectal cancer were combined, as were terms for chemotherapy, radiotherapy, surgery or combined treatment. Results were restricted with the application of terms for "randomized clinical trials" and "patient reported outcomes", and limited to articles published between January 2000 and October 2012 (see full search in S1 Appendix). The search output was imported into Reference Manager software and duplicate records removed. References for relevant studies before the year 2000 were obtained from a previous systematic review [2]. Titles and abstracts were screened by two researchers (AGKM and RM). Serial publications for the same trial (e.g. articles reporting short and long term PROs) were included. Excluded were phase II studies, RCTs of endoscopic and non-biomedical interventions, or trials limited to palliative treatment, screening or premalignant conditions. Only English-language publications were considered. Articles were assessed for risk of bias in the trials by three researchers (AGKM, RM, NB) using the Cochrane tool [3]. Studies classified with potential high or unascertainable risk of bias were excluded. Independent data extraction was conducted by at least two reviewers (AGKM, NB, RM, JMB) using a pre-designed and piloted form. Details of the trial were recorded including disease site, treatment intervention, primary and secondary outcomes, number of participants, and main trial results. The systematic review PRISMA checklist is presented in S2 Appendix.

(b) Categorisation of papers into combined PRO and clinical reports or linked primary and supplemental reports
Where included papers indicated the presence of previously published results from the same trial, these additional papers were sought and included in the analysis. These linked papers are hereafter considered in the order by which they were published, with those published first and second defined as "primary" and "supplemental" respectively. Thus, trials were categorized into those reporting PROs and clinical outcomes in a single, combined paper and those reporting results in linked primary and supplemental papers. The journal impact factor (Thomson Reuters, 2012) and the date of publication for each paper were recorded. Descriptive statistics compared journal impact factor for combined and linked primary and supplementary papers and median times between publications were summarized.
(c) Assessment of PRO reporting using the new CONSORT extension The PRO CONSORT extension was applied to all trials to establish standards of PRO reporting, with the exception of item P6a which was an inclusion criterion (use of a validated PRO measure). Reporting of item 7a (PRO sample size) was recorded as present if a sample size calculation was completed in trials with patient reported primary outcomes, or if it were not applicable (for example, if there were no PROs as primary outcomes). Descriptive statistics are presented to consider PRO CONSORT standards within trials reporting in a single, combined publication or in linked primary and supplementary papers. Part 2 (a) Development of methods to present the 'take home/trial exegesis' message of PRO with clinical data in trials Methods to report the 'take home' messages of clinical and PROs in trials were developed through an in-depth analysis of items 2a and P2b (rationale/hypotheses for PRO measurement), and P20/21 and 22 (limitations and implications for clinical practice, and interpretation of PROs in relation to clinical outcomes) to identify good practice and produce methods to inform a PRO take home message from trials. All papers were read and re-read independently by at least two (AM, RM and JMB) researchers to become immersed in the data and relevant text was independently coded, copied verbatim into an electronic database and analysed for consistency between researchers. Discrepancies in coding were discussed within the study team (AM, RM and JMB). Methods for reporting trial 'take home' message were developed and applied iteratively to relevant text. Deviant examples were sought to challenge theories. Primary quotations are provided in accordance to methods of qualitative rigor. It was also noted whether primary papers indicated the future publication of a supplemental PRO paper (defined as "signposting").

(b) Application of the novel methods to included trials
The methods described above were applied to the included trials and reported data examined by whether trials were published in combined or linked primary and supplemental reports.

Part 1(a) Identification of well-designed and conducted RCTs reporting PROs with validated instruments
OVID (MEDLINE and Embase) and Cochrane database search yields were 1815 and 939 records. After de-duplication, 1917 abstracts were screened, 1716 excluded, and 201 full text articles further assessed for eligibility, and these were supplemented with 13 studies from a previous systematic review [2]. Sixty-seven articles met the inclusion criteria describing trials at low risk of bias (Fig 1). The 66 included articles reported PROs from 49 RCTs, the majority of which were chemotherapy interventions (22/49, 44.9%) in colorectal disease (36/49, 73.5%, Table 1).
(b) Categorisation of papers into combined PRO and clinical reports or linked primary and supplemental reports Some 36 (71%) single papers reported combined PRO and clinical results and the remaining 30 were linked primary and supplemental papers. The median journal impact factor for both combined and primary papers was the same (16.8), but supplemental PRO papers were reported in journals with a lower median impact factor (5.2). The mean time between publication of linked primary and supplementary papers was 20 months (range 5 to 51).

(c) Assessment of PRO reporting using the new CONSORT extension
Overall, the reporting of most papers did not meet the new PRO CONSORT standards (S1 Table). There were six (2 combined; 0 primary and 4 supplemental) papers reporting all PRO CONSORT items (Table 2). Primary papers reported the fewest items (median 3, range 2 to 7), typically lower than combined papers (median 6, range 2 to 12) and supplemental papers (median 10, range 4 to 12). The least frequently reported items related to "results" (13a, 15, 16, 17a and 18), and were reported in less than one third of papers. The least frequently reported Part 2 (a) Development of novel methods to examine the take home message of PRO with clinical data in trials The new method for understanding and improving combined reporting practice was developed based upon the in-depth analyses, emergent data and iterative discussions with AM and JMB. Sections of verbatim text were coded to items 2a and P2b were classified as providing a 1) detailed rationale/hypothesis-when authors included a specific PRO domain and/or hypothesized effect, 2) general rationale/hypothesis-when authors included non-specific rationale (i.e. "to examine quality of life") or 3) no rationale provided. Verbatim quotes of PRO rationales are • 100-199 4 (10) • 200-499 13 (35) • 500-999 8 (21) • 1000-1999 3 (8) • presented in Tables 3 and 4. An example of a detailed rationale included: ". . .we hypothesized a priori that [intervention] would result in a decrease in the magnitude and rate of decline in HRQL, particularly in physical function and overall well-being" [4], and an illustration of a general rationale included ". . .to compare. . .quality of life. . .between [intervention] and [control]" [5]. Likewise, sections of text coded to the items P20/21 and 22 were developed as 1) detailed interpretations-where authors discussed the hypothesized effect of a specific PRO in relation to the hypothesized effect of a clinical outcome, 2) general interpretations-when authors include non-specific interpretation of PROs in relation to clinical outcomes, or 3) no integrated PRO and clinical interpretation of results in the paper. An example of detailed interpretation of findings includes ". . .
[Intervention] resulted in significantly improved TTP [time to progression] (primary end point), OS [overall survival], and overall response rate (secondary end points), with global health status (quality of life). . . preserved for a longer time." [6] Part 2, b) Application of the methods to included trials Of the 36 trials reporting combined papers, there were 11 (30%) papers that provided a PRO rationale/hypothesis, 10 (30%) providing general information and 15 (40%) not providing a The PRO should be identified in the abstract as a primary or secondary outcome. * For more detailed analysis of items 2a/P2b see Table 3 ** Study inclusion criteria † Only applicable to trials with PROs as primary outcome (n = 6, all combined papers). ‡ For more detailed analysis of items P20/21/22 see Table 3 doi:10.1371/journal.pone.0160998.t002  PRO rationale ( Table 5). The interpretation of PRO data in the context of clinical outcomes were detailed in six (16%), general in 24 (65%) and absent in 7 (17%) papers (Items P20/21 and 22, Table 3). There were seven papers that described both detailed PRO rationale/hypotheses and detailed interpretations of PROs in relation to clinical outcomes [5,6,13,26,30,68,70]. Where PRO and clinical results were published separately (n = 15), most primary papers did not provide any PRO rationale (n = 10, 66%) or text interpreting PROs in relation to clinical findings (n = 10, 66%). One primary paper provided detailed descriptions of both these issues. In comparison, all supplemental papers described PRO rationales and most (14, 93%) contained detailed interpretation of PROs in relation to clinical findings, although only two had detailed descriptions of both of these. Of the 15 primary papers, 66% signposted the presence of the supplemental PRO report.

Conclusions
Reporting PRO rationale linked to clinical hypotheses, and clear reporting of PRO results interpreted appropriately in the context of the clinical outcomes are critical to ensure that oncologists gather a "take-home" message to communicate to patients which encompasses clinical and PROs. This review explored this issue in detail. Patient reported outcome reporting standards were at lowest levels in primary clinical papers (where clinical trial data was reported separately to the supplementary PROs). Whilst supplemental papers provided more detail there was a 5 to 51 month delay in publication in less well cited journals, thus diminishing their impact. Most (71%) trials did report combined results, demonstrating that it is possible to do so. New methods to examine reporting of trial "take home" messages recommend that detailed information about domain specific PRO rational and interpretation with clinical data are supplied within a main trial paper to arm clinicians with relevant outcomes to use in decisionmaking. Authors need to be allowed space to report these details alongside clinical outcomes to inform the take home message from papers to help clinicians in practice. Where this is not appropriate for scientific reasons, for example, if primary outcome data are available before secondary PROs, then this could be explicitly stated.  Other systematic reviews have shown that PRO reporting standards are poor [71][72][73][74][75][76][77] which contributed to the need for the development of the PRO CONSORT extension. Similarly, other papers have recommended clear PRO hypotheses and integration with clinical findings, however, there is no empirical data presented on how to best achieve this [78]. Reviews also have examined how PROs in RCTs influence decision-making and confirm that PRO information is not used in practice [77,79,80]. Previous work, however, has not provided solutions to these problems or considered the conceptual reasons why PRO data are not used in practice. Theoretically, failure of PRO data to have an impact on clinical practice may stem from problems that start with the trial design and conduct, compounded by poor reporting and separate PRO and clinical publications. Further research is now needed to investigate whether improved reporting will have the desired effect of informing patient-centred care, clinical decision making and health policy decisions. For example, work could include a study directly exploring oncologists' views of trial reports following the introduction of new reporting guidelines would be informative. Additionally in-depth research of clinical decision-making in multi-disciplinary teams or of oncology consultations may be undertaken to examine how PRO data are used. Research needs to be targeted into each of these areas in order to understand how improvements, such as the recently published SPIRIT statement [81] for improving RCT protocols or the CONSORT PRO extension, impact clinical practice. What is clear is that cancer patients want information about PROs and indeed rate such data of similar importance to survival information [82][83][84]. It is therefore critical that oncologists communicate PRO data in the context of a shared doctor-patient consultation and methods to do so are being established [85][86][87]. This may have occurred because authors split trial results and deliberately left the PRO methodology and findings to the supplemental report.
This review included a systematic search for studies using PRISMA guidelines [88] and transparent methodology for an in-depth analyses if the textual data, but there were some limitations. Trials of radical treatments of gastrointestinal cancers were selected for analyses because the authors were familiar with the PRO and clinical data in this field. It is conceivable that including trials in other diseases or in the palliative setting may have identified different reporting standards. For example, in the palliative setting, authors may make greater reference to PROs and their integration with clinical outcomes because the main focus of treatment is not to cure disease. Further work is needed to examine this area in detail. In addition, studies with high or unascertainable risk of bias were excluded because lower standards of reporting are associated with bias and likely poor PRO reporting [89], and it is possible that important data were missed. This is considered to be unlikely however, because even the included "highquality" trials demonstrated significant reporting weaknesses and inclusion of poor quality trials would probably not yield exemplar practice. In summary, this review presents and evidence based way of implementing the new CON-SORT PRO extension items 2a, P2b, P20/21 and 22 based on current literature. It is recommended that RCTs report domain-specific PRO rationale with anticipated treatment effects, and integrate these findings with specific clinical outcomes in a single combined report. It is acknowledged that trials are structured around their primary (often clinical) endpoints, and it is appropriate to prioritise these data at the expense of other outcomes. It seems unnecessary, however, to relegate evaluation of patient experience to reports that may be less likely to influence practice. The adoption of better standards for PRO reporting could facilitate the use of PRO data by oncologists and patients for informed decision making.