Figures
Abstract
Background
Evidence suggests there are inconsistencies in patient-reported outcome (PRO) assessment and reporting in clinical trials, which may limit the use of these data to inform patient care. For trials with a PRO endpoint, routine inclusion of key PRO information in the protocol may help improve trial conduct and the reporting and appraisal of PRO results; however, it is currently unclear exactly what PRO-specific information should be included. The aim of this review was to summarize the current PRO-specific guidance for clinical trial protocol developers.
Methods and Findings
We searched the MEDLINE, EMBASE, CINHAL and Cochrane Library databases (inception to February 2013) for PRO-specific guidance regarding trial protocol development. Further guidance documents were identified via Google, Google scholar, requests to members of the UK Clinical Research Collaboration registered clinical trials units and international experts. Two independent investigators undertook title/abstract screening, full text review and data extraction, with a third involved in the event of disagreement. 21,175 citations were screened and 54 met the inclusion criteria. Guidance documents were difficult to access: electronic database searches identified just 8 documents, with the remaining 46 sourced elsewhere (5 from citation tracking, 27 from hand searching, 7 from the grey literature review and 7 from experts). 162 unique PRO-specific protocol recommendations were extracted from included documents. A further 10 PRO recommendations were identified relating to supporting trial documentation. Only 5/162 (3%) recommendations appeared in ≥50% of guidance documents reviewed, indicating a lack of consistency.
Conclusions
PRO-specific protocol guidelines were difficult to access, lacked consistency and may be challenging to implement in practice. There is a need to develop easily accessible consensus-driven PRO protocol guidance. Guidance should be aimed at ensuring key PRO information is routinely included in appropriate trial protocols, in order to facilitate rigorous collection/reporting of PRO data, to effectively inform patient care.
Citation: Calvert M, Kyte D, Duffy H, Gheorghe A, Mercieca-Bebber R, Ives J, et al. (2014) Patient-Reported Outcome (PRO) Assessment in Clinical Trials: A Systematic Review of Guidance for Trial Protocol Writers. PLoS ONE 9(10): e110216. https://doi.org/10.1371/journal.pone.0110216
Editor: Matthias Briel, University Hospital Basel, Switzerland
Received: June 9, 2014; Accepted: September 10, 2014; Published: October 15, 2014
Copyright: © 2014 Calvert et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The authors confirm that all data underlying the findings are fully available without restriction. All relevant data are within the paper and its Supporting Information files.
Funding: This research was funded by the National Institute for Health Research School for Primary Care Research. Professors Calvert and Draper are members of the Medical Research Council Midland Hub for Trials Methodology Research (Medical Research Council grant G0800808). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Patient-reported outcomes (PROs), including health-related quality of life (HRQL), symptoms such as pain or fatigue, and health utility, are increasingly assessed in clinical trials as a measure of effectiveness.[1]–[3] PRO trial data may be used to inform clinical care and decision-making, predict long-term outcomes and influence health-policy; but to do so, as with any trial outcome, they must be collected with rigor. Unfortunately, evidence shows that the quality of PRO data can be undermined in some trials by inconsistencies in data collection [4] and, in particular, by high rates of missing data [5]; this adversely affects the integrity and usefulness of such data in clinical practice.
To help ensure optimal PRO data collection, PRO-specific components should be considered during clinical trial design and clearly documented in the trial protocol. [6], [7] The trial protocol is the cornerstone of a well-conducted trial, and should provide specific instruction on how to conduct all aspects of the study. [8] The protocol also allows external funding bodies, regulators, research ethics committees, journal editors, health care providers, systematic reviewers and policy makers to evaluate the design and methods. [6] Despite the importance of PROs, recent data suggests that some trial staff feel protocols provide little guidance regarding PRO-specific aspects of the trial, leading to ambiguity and the potential for significant inconsistency in the way PRO data are gathered, analysed, acted upon, and reported. [4], [9], [10].
The recent publication of the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) guidance aims to promote the inclusion of important general methodological components in trial protocols [6]; however, it does not provide specific guidance related to PROs. It is currently unclear exactly what PRO-specific information should be included in trial protocols. The aim of this systematic review was to summarize current PRO-specific guidance for clinical trial protocol developers.
Methods
Ethics
This study received ethical approval from the University of Birmingham ethical review board (ERN_13-0047).
This review was conducted according to PRISMA guidelines (Checklist S1) [11] and a protocol is available. [12] Given the methodological focus of the review this was not registered with PROSPERO at http://www.crd.york.ac.uk/PROSPERO/.
Eligibility criteria
Papers were deemed eligible if they contained guidance (in the form of advice or formal recommendations) and/or a checklist on PRO related trial protocol content. As PROs is an ‘umbrella term’ papers including protocol guidance relating to specific types of PRO (such as HRQL) were also deemed eligible.
Information Sources and Search Strategy
The MEDLINE (Ovid), EMBASE, CINHAL and Cochrane Library databases were searched from inception to February 2013 (electronic search strategies are presented in full in Appendix S1). Other relevant articles were identified via two Internet search engines (Google and Google Scholar) using the key words ‘Patient-Reported Outcomes’ or ‘Health-Related Quality of Life’ in combination with ‘Guidance’, ‘Guidelines’ or ‘Checklist’. Only the first 30 results (3 pages) of each search were reviewed as article relevance diminishes substantially with each page of results. [13] In addition, an international advisory group (MB, AG, JB, RMB and MK) were consulted via email to identify additional ‘grey literature’ directly relevant to the research question. Finally, PRO guidance/checklists and Standard Operating Procedures (SOPs) were requested from all members of the UK Clinical Research Collaboration registered clinical trials units (UK CRC-CTU) via email, with one follow up reminder. All citations were downloaded into Endnote software version X7 and duplicates deleted. Records were then screened by title/abstract before full-text articles/documents were retrieved for eligibility evaluation. Remaining articles were subject to a citation search before a final hand-search of all reference lists.
Guidance selection
Papers and other guidance documents were deemed eligible if they provided guidance (advice or formal recommendations) and/or a checklist describing key PRO-specific information that should be specified in clinical trial protocols. Non-English papers were screened by language specialists in the School of Health and Population Sciences, University of Birmingham. When more than one edition of a book was available, the latest edition was screened.
Data Extraction
Two reviewers (HDu and AG) independently screened the titles and abstracts of all citations. Full text versions of potentially eligible documents were independently reviewed (HDu and AG) with uncertainties resolved through discussion with a third investigator (MC/DK). Two investigators (MC and DK), independently and in duplicate, extracted both the publication details and all PRO-specific protocol recommendations from the final included documents. Data were extracted into Excel on pre-specified forms to capture: publication source, year of publication, clinical and regulatory focus. Each recommendation was identified through independent review by MC and DK and iteratively added to the spreadsheet following checks for consistency.
Both explicit (‘stated clearly and in detail, leaving no room for confusion or doubt’ [14]) and implicit (‘suggested though not directly expressed’ [14]) recommendations for PRO related protocol content were extracted. Explicit recommendations specifically stated that an item of PRO information should be included in the trial protocol. Implicit recommendations described important PRO trial design issues, and were written in such a way as to suggest items should be included in the protocol – without specifically stating so.
Summary of Guidance
For ease of interpretation, PRO protocol recommendations were extracted and grouped as per SPIRIT guidance headings. [6] Duplicate recommendations within each of these sections were identified by MC and DK, and were merged where necessary following discussion with the international advisory group. The proportion of guidance documents associated with each recommendation was identified. To assess general trends in guidance over time, the proportion of guidance documents per recommendation was analysed retrospectively over 5 year time periods.
Results
Selection of Guidance Documents
The literature search yielded 21,175 unique references. Following application of the inclusion criteria, 54 guidance documents[1], [15]–[67] were included in the review (Figure 1). DK and MC independently included 53 of the 54 articles. One paper was initially excluded by MC but later included following discussion by the research team. [52] Of the 54 included documents, 8 were identified from searches of electronic databases, 5 from citation tracking, 27 from hand searching the reference lists of included articles, 7 from the grey literature review and 7 from expert recommendations.
Guidance Characteristics
Document characteristics are summarised in Table 1. The included materials dated from 1989 to 2013 and included 42 journal articles, 5 books and 7 organizational guideline documents, with the majority focused on HRQL/PRO assessment in cancer trials (n = 35, 64.8%) and written from a non-regulatory perspective (n = 44, 81.5%).
PRO Protocol Guidance
The included guidance documents contained 162 unique recommendations regarding PRO study design/conduct information that should be included in trial protocols. There was disagreement between data extractors on the inclusion of 10 items (6.2%), 6 of these items were included following discussion by the team, whilst 4 were felt to be duplicates of existing items. Of the 162 included items, 134 recommendations were explicit (e.g. ‘All analyses should be clearly defined a priori in the research protocol’ [41]), and 28 were implicit (e.g. ‘…investigators need to provide a rationale for the selection of a particular HRQL instrument’ [46]). Protocol recommendations are summarised in Table 2 and presented in full in Appendix S2; in both places, they are grouped under SPIRIT guidance headings, with additional subheadings used to organise content. In addition, 10 PRO recommendations were related to other supporting trial documentation (Appendix S3).
Administrative information
There were n = 4 recommendations regarding trial administration, which centred around identifying the roles and responsibilities of PRO personnel and ranged from advocating involvement of the research nurse in PRO protocol development, to providing the contact details of the Quality of Life (QOL) sub-study coordinator where appropriate.
Introduction: Background, rationale, and objectives/hypotheses
Eleven unique recommendations related to the inclusion of PROs in the introductory sections of the protocol. These focused on aspects surrounding: PRO specific background information (n = 2), for instance, the need to describe the PRO population of interest; specification of the PRO rationale (n = 5), for example, justifying the relevance of PRO assessment in the disease and population under investigation; or outlining the PRO hypothesis and objectives (n = 4).
Methods: Participants, interventions and outcomes
There were n = 25 unique recommendations within this section, focused on a number of areas, including: the PRO study setting (n = 1), the PRO-specific eligibility criteria (n = 3), the need to specify the PRO as an endpoint (n = 5), the PRO-specific sample size (n = 2) and blinding considerations (n = 2). Twelve different recommendations related to timing of the PRO assessment, ranging from: including PRO assessment timings in the main protocol assessment schedule and specifying time windows, to justifying timings according to the study research questions, length of recall of the questionnaire, the natural history of the disease under study, and any planned analysis.
Methods: assignment of interventions
There were no PRO-specific recommendations identified under this heading.
Methods: Data Collection, management and analysis
Ninety-four recommendations related to PRO-specific protocol guidance for data collection, management and analysis. These focused on data collection aspects including: identification/description of the PRO instrument (n = 4), for instance, the need to outline the questionnaire domains and number of items; justifying the choice of instrument (n = 13), for example, the importance of referencing the validity, reliability and responsiveness of the tool; detailing the data collection plan (n = 10), for example, stating who should administer the questionnaire; and describing the data collection/training guidelines (n = 16) (e.g. outlining the certification process for staff involved in PRO assessment) and plans to minimise missing data (n = 19) for example specifying who would check questionnaires for missing data. There were n = 7 recommendations concerning PRO specific quality assurance, ranging from the inclusion of guidance for data entry coding decisions regarding missing or ambiguous responses; to specifying procedures for a central PRO data monitoring system aimed at identifying and rectifying potential data collection problems. Finally, n = 13 recommendations focused on PRO analysis, specifically: the PRO-specific components of the statistical analysis plan (n = 13), for instance, the need to include an a priori estimation of expected change in PRO score; plans to address multiple hypothesis testing (n = 2), such as pre-specification of sequence of testing; defining clinical significance (n = 6), for example, describing and justifying the minimal clinically important difference/change); and specifying methods to deal with missing PRO data (n = 4), for instance, defining proposed sensitivity analyses for imputation methods.
Methods: Monitoring
There were four recommendations regarding PRO specific trial monitoring, ranging from the need to define the role of the Data Monitoring Committee in relation to PROs, to the inclusion of a plan to manage PRO Alerts.
Ethics and Dissemination
There were n = 3 recommendations focused on PRO-specific consent information, for example, the need to include information for patients regarding who should be contacted for help with completing the PRO questionnaire. Two recommendations addressed PRO specific confidentiality issues, such as the need to specify whether QOL data will be used to influence patient management. Two recommendations focused on the need to include PRO-specific dissemination plans, through both peer-reviewed scientific publication and direct participant contact.
Appendices
Fourteen recommendations focused on the inclusion of relevant PRO documents as protocol appendices, including: a copy of the PRO questionnaire(s), sample patient information and consent materials containing PRO information and a PRO-specific administration flow chart/checklist.
Other Trial Documentation
Ten recommendations focused on PRO information that should be included in protocol-related trial documents such as Standard Operating Procedures (SOPs), Case Report Forms (CRFs) or training manuals (Appendix S3).
Time trends and Common Recommendations
The availability of PRO-specific guidance over time is shown in Figure 2. The data suggest that there has been consistent publication of PRO protocol guidance, across all areas, over the last 25 years (Table 3). In addition, over 75% of recommendations extracted for this study have been available for at least 10 years.
Only 3% of recommendations appeared in more than half of the documents included in the study, highlighting a lack of consistency in the PRO guidance literature reviewed (Table 2). These included (in order of frequency): the need to specify the timing of QOL assessment, the provision of PRO data collection guidelines and/or a training plan, specification (and justification) for the chosen PRO questionnaire, routine inclusion of a priori defined PRO analyses plans and specifying a named person within the trial with responsibility for overseeing QOL assessment.
Discussion
Summary of Findings
Our review is the first to summarise the current PRO-specific guidance for clinical trial protocol developers. In total we identified 54 guidance documents[1], [15]–[67], which provided 162 recommendations regarding PRO-specific information that should be included in protocols containing a PRO endpoint.
Unfortunately, although PRO protocol guidance has been in existence for over 25 years, our findings suggest it remains difficult to implement in practice. First, with the exception of 8 publications[16], [22], [39]–[41], [59], [62], [63] sourced via electronic database searches, the guidance literature was particularly difficult to access. The remaining 46 documents, which provided more than half (56.7%) of all PRO protocol recommendations, were obtained via citation tracking, hand-searching reference lists of included articles, grey literature review and expert contact. It is unlikely that protocol developers would have the time or resources to carry out such a comprehensive search. As such, developers may be reliant on a small proportion of guidance documents available via easily accessible scientific databases. This is problematic, as these publications provide relatively little coverage of the current PRO protocol recommendations in circulation. As our findings show, recommendations are spread over a wide variety of sources, thus, over reliance on a small number of guidance documents may mean important PRO design considerations are overlooked. For example, even the two publications that provided most recommendations, Chassany [41] and Fairclough [58] (42 recommendations each, 24 shared), provided just 37.04% of the total in circulation.
Second, developers wishing to use the guidance summarised in this review face the challenge of trying to incorporate a large number of recommendations into what is usually a rather limited amount of space within the protocol. For instance, we identified 94 unique recommendations concerning data collection, management and analysis, of which, 19 alone addressed minimising missing data. Tackling missing PRO data is clearly an important design consideration since it helps reduce bias and preserves statistical power [68], however, it may be unrealistic to expect protocol developers to incorporate all 19 recommendations within a study protocol.
It is important, therefore, for the scientific community to reach consensus on the essential PRO protocol content required to preserve trial integrity; and to provide guidance that is useful in practice. We advocate the development of consolidated, easily accessible and internationally endorsed consensus guidelines addressing this objective. Our review provides a useful starting point as it presents a comprehensive list of the PRO protocol guidance currently available, however, it remains unclear at this stage exactly which of the recommendations identified in this study should be incorporated into consolidated guidelines. A number of recommendations are supported by multiple sources and appear to be underpinned by a clear theoretical justification (for example, the need to provide a rationale for PRO measurement (recommended in 48.15% of guidance documents)), and may be promising candidates for inclusion. There were, however, a number of other recommendations that were less frequently cited, but still may have important implications for trial conduct, reporting and the quality of PRO results. For example, referencing the PRO instrument validity and reliability in the protocol (recommended in 25.93% of guidance documents) will help ensure that the psychometric properties of the PRO have been duly considered during the trial design and will help facilitate later reporting in accordance with the CONSORT-PRO extension. [69] In addition, only four publications provided guidance describing plans for the identification and management of PRO Alerts, that is: ‘concerning levels of psychological distress or physical symptoms that may require an immediate response’ [67]. However, evidence suggests that without clear, pre-specified, plans for the management of PRO Alerts, either in the trial protocol or supporting documentation, variation may occur in their management within/across trial sites risking co-intervention bias and suboptimal patient care. [67].
Consolidated PRO guidance for protocol developers is clearly necessary, however, our findings also show that guidelines should be developed using robust consensus methodology to ensure that the merits of all individual recommendations are carefully considered prior to selection/rejection. The definitive guidelines should aim to improve the quality of PRO trial design and reporting, resulting in more robust PRO trial data that will exert a greater influence on clinical practice and will provide an improved information base for future patients. Researchers should be supported in implementing the guidance through training and online resources. Furthermore, endorsement by funding bodies and Institutional Review Boards/Ethical Committees, who review the content of protocols, and journal editors, who are responsible for their publication, is important to ensure widespread adoption.
Strengths and Limitations
Our review has for the first time collated and summarised the existing PRO guidance available for protocol developers using systematic methods and multiple reviewers. A limitation of our approach is that the PRO item categorisation and indexing employed during our analysis is influenced by reviewer interpretation. Also, publications included in the study had to provide guidance on PRO-protocol content; however, such guidance was not always the main focus or aim of some of the included articles. Again, the interpretation of the reviewer may subtly alter the original meaning of the text drawn from such material. The use of independent dual data extraction by 2 investigators (with a third to mediate) sought to reduce these effects, however, they remain a legitimate concern. Furthermore no formal quality appraisal was undertaken given the diverse nature of the guidance documents. Relevant PRO guidance literature was difficult to source and appeared to be particularly poorly indexed. Whilst we employed a number of resources to comprehensively search the literature (including electronic databases, citation tracking and hand searching, internet search engines and expert contact) further PRO guidance probably exists that was not included in our study, such as disease specific documents citing EMA and FDA guidance included in our review. Consideration of alternative search strategies, such as ‘citation pearl methodology’ [70], may have further increased the comprehensiveness of our review. The International Society for Quality of Life Research Best Practice for PROs in Trials Taskforce (Co-chaired by authors, MC, MK and MB) is leading work in this area and would value the identification of further grey literature across a range of clinical settings on this topic by interested readers to inform future guideline development. A further limitation is that most of the international advisory group work in an oncology setting therefore are likely to have greater knowledge of grey literature in this field. However, oncologists have routinely used PROs for many years and as such we may anticipate more literature in this clinical area.
Conclusion
PRO-specific protocol guidance is difficult to access, lacks consistency and is unwieldy; with over 160 recommendations spread across 54 different publications. It is therefore extremely challenging to implement in practice. There is a need to develop easily accessible consolidated, and consensus-driven, PRO protocol guidelines. Guidance should aim to ensure key PRO information is routinely included in trial protocols with a PRO endpoint, in order to facilitate the rigorous collection and reporting of PRO data, thus maximising its capacity to effectively inform patient care.
Supporting Information
Appendix S2.
Protocol recommendations by source publication.
https://doi.org/10.1371/journal.pone.0110216.s003
(XLSX)
Appendix S3.
PRO recommendations relating to other supporting trial documentation.
https://doi.org/10.1371/journal.pone.0110216.s004
(XLSX)
Acknowledgments
Thanks to Saeeda Bashir for coordinating requests for information from the UK CRC-CTU Network, to Karen Biddle for administrative support and to Janine Dretzke and Simon Stevens, University of Birmingham UK, for screening non-English language papers.
Author Contributions
Conceived and designed the experiments: MC DK H. Duffy AG RM-B JI H. Draper MB JB MK. Performed the experiments: AG H. Duffy MC DK RM-B. Analyzed the data: MC DK RM-B MB JB MK. Contributed to the writing of the manuscript: MC DK H. Duffy AG RM-B JI H. Draper MB JB MK. Obtained funding: MC DK H. Draper JI AG RM-B MB MK JB. Searched the literature and screened titles, abstracts and full text articles with input from a third reviewer (MC/DK) where necessary: AG H. Duffy. Searched the grey literature: H. Duffy RM-B. Extracted data from included publications: MC DK. Formed an expert advisory panel that identified additional grey literature relevant to the research question: MB JB MK RM-B. Wrote the first draft of the manuscript: MC. Critically revised the manuscript and approved the final version for publication: MC DK H. Duffy AG RM-B. JL H. Draper MB JB MK.
References
- 1.
FDA (2009) Patient Reported Outcome Measures: Use in Medical Product Development to Support Labelling Claims. Guidance for Industry.
- 2. Basch E (2010) The missing voice of patients in drug-safety reporting. N Engl J Med 362: 865–869.
- 3. Calvert M, Brundage M, Jacobsen PB, Schunemann HJ, Efficace F (2013) The CONSORT Patient-Reported Outcome (PRO) extension: implications for clinical trials and practice. Health Qual Life Outcomes 11: 184.
- 4.
Kyte D, Ives J, Draper H, Keeley T, Calvert M (2013) Inconsistencies in quality of life data collection in clinical trials: a potential source of bias? Interviews with research nurses and trialists. PloS one 8.
- 5. Fielding S, Maclennan G, Cook JA, Ramsay CR (2008) A review of RCTs in four medical journals to assess the use of imputation to overcome missing data in quality of life outcomes. Trials 9: 51.
- 6. Chan (2013) Tetzlaff JM, Altman DG, Laupacis A, Gøtzsche PC, et al. (2013) SPIRIT 2013 Statement: Defining Standard Protocol Items for Clinical Trials. Annals of Internal Medicine 158: 200–207.
- 7. The Lancet (2010) Strengthening the credibility of clinical research. The Lancet 375: 1225.
- 8. Chan A, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, et al. (2013) SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ 346: 1–42.
- 9. Kyte DG, Draper H, Ives J, Liles C, Gheorghe A, et al. (2013) Patient Reported Outcomes (PROs) in Clinical Trials: Is ‘In-Trial’Guidance Lacking? A Systematic Review. PloS one 8: e60684.
- 10. Brundage M, Bass B, Davidson J, Queenan J, Bezjak A, et al. (2011) Patterns of reporting health-related quality of life outcomes in randomized clinical trials: implications for clinicians and quality of life researchers. Quality of Life Research 20: 653–664.
- 11.
Moher D, Liberati A, Tetzlaff J, Altman DG (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 339.
- 12.
University of Birmingham Patient Reported Outcomes Research Group Study Protocol Available: http://www.birmingham.ac.uk/Documents/college-mds/groups/Patient-Reported-Outcomes/Study-Protocol-Evaluation-of-Patient-Reported-Outcomes-%28PROs%29-in-Clinical-Trials-Systematic-Review-of-Trial-Protocols.pdf Accessed 2014 September 5.
- 13.
iPROSPECT (2006) iPROSPECT Search Engine User Behaviour Study. Available: http://district4.extension.ifas.ufl.edu/Tech/TechPubs/WhitePaper_2006_SearchEngineUserBehavior.pdf Accessed 2014 September 22.
- 14.
Oxford University Press (2014) Oxford Dictionaries.Available: http://www.oxforddictionaries.com/Accessed 2014 September 22.
- 15. Moinpour CM, Feigl P, Metch B, Hayden KA, Meyskens FL, et al. (1989) Quality of life end points in cancer clinical trials: review and recommendations. Journal of the National Cancer Institute 81: 485–496.
- 16. Schron E, Shumaker S (1991) The integration of health quality of life in clinical research: experiences from cardiovascular clinical trials. Progress in cardiovascular nursing 7: 21–28.
- 17. Gotay CC, Korn EL, McCabe MS, Moore TD, Cheson BD (1992) Quality-of-life assessment in cancer treatment protocols: research issues in protocol development. Journal of the National Cancer Institute 84: 575–579.
- 18. Osoba D (1992) The quality of life committee of the clinical trials group of the National Cancer Institute of Canada: organization and functions. Quality of Life Research 1: 211–218.
- 19.
Gotay C, Korn E, McCabe M, Moore T, Cheson B (1992) Building quality of life assessment into cancer treatment studies. Oncology (Williston Park, NY) 6: 25–28; discussion 30–22, 37.
- 20. Nayfield S, Ganz P, Moinpour C, Cella D, Hailey B (1992) Report from a National Cancer Institute (USA) workshop on quality of life assessment in cancer clinical trials. Quality of Life Research 1: 203–210.
- 21. Sadura A, Pater J, Osoba D, Levine M, Palmer M, et al. (1992) Quality-of-Life Assessment: Patient Compliance With Questionaire Completion. Journal of the National Cancer institute 84: 1023–1026.
- 22.
Hayden K, Moinpour C, Metch B, Feigl P, O'Bryan R, et al.. (1993) Pitfalls in quality-of-life assessment: lessons from a Southwest Oncology Group breast cancer clinical trial; pp. 1415–1419.
- 23. Cella D, Wiklund I, Shumaker S, Aaronson N (1993) Integrating health-related quality of life into cross-national clinical trials. Quality of life research 2: 433–440.
- 24.
Spilker (1996) Quality of Life & Pharmacoeconomics in Clinical Trials: Applying quality of life principles in International cancer clinical trials. Chapter 72 Bernhard J et al al.
- 25. Molin C, Arrigo C (1995) Clinical trials and quality of life assessment: The nurses' viewpoint. European Journal of Cancer 31: S8–S10.
- 26. Fayers P, Hopwood P, Harvey A, Girling D, et al. (1997) Quality of life assessment in clinical trials-guidelines and a checklist for protocol writers: the UK medical research council experience. European Journal of Cancer 33: 20–28.
- 27. Kiebert G (1997) Quality of life as an outcome in EORTC clinical trials. European Journal of Cancer 33: S31–S34.
- 28. Bernhard J, Cella DF, Coates AS, Fallowfield L, Ganz PA, et al. (1998) Missing quality of life data in cancer clinical trials: serious problems and challenges. Statistics in medicine 17: 517–532.
- 29. Bernhard J, Peterson H, Coates A, Gusset H, Isley M, et al. (1998) Quality of Life Assessment in International Breast Cancer Study Group (IBCSG) Trials: Practical Issues and Factors Associated with Missing Data. Statistics in Medicine 17: 587–601.
- 30. Osoba D, Zee B (1998) Completion rates in health-related quality-of-life assessment: approach of the National Cancer Institute of Canada Clinical Trials Group. Statistics in medicine 17: 603–612.
- 31. Simes RJ, Greatorex V, Gebski V (1998) Practical approaches to minimize problems with missing quality of life data. Statistics in medicine 17: 725–737.
- 32. Moinpour CM, Lovato LC (1998) Ensuring the quality of quality of life data: the Southwest Oncology Group experience. Statistics in medicine 17: 641–651.
- 33. Brooks MM, Jenkins LS, Schron EB, Steinberg JS, Cross JA, et al. (1998) Quality of Life at Baseline: Is Assessment After Randomization Valid? Medical care 36: 1515–1519.
- 34. Leidy NK, Revicki DA, Genesté B (1999) Recommendations for evaluating the validity of quality of life claims for labeling and promotion. Value in Health 2: 113–127.
- 35. Osoba D (1999) What has been learned from measuring health-related quality of life in clinical oncology. European journal of cancer 35: 1565–1570.
- 36. De Haes J, Curran D, Young T, Bottomley A, Flechtner H, et al. (2000) Quality of life evaluation in oncological clinical trials–the EORTC model. European Journal of Cancer 36: 821–825.
- 37. Revicki DA, Osoba D, Fairclough D, Barofsky I, Berzon R, et al. (2000) Recommendations on health-related quality of life research to support labeling and promotional claims in the United States. Quality of Life Research 9: 887–900.
- 38. Bottomley A (2001) Developing Clinical Trial Protocols for Quality of Life. Applied Clinical Trials 10: 40–44.
- 39. Hakamies-Blomqvist L, Luoma ML, Sjöström J, Pluzanska A, Sjödin M, et al. (2001) Timing of quality of life (QoL) assessments as a source of error in oncological trials. Journal of advanced nursing 35: 709–716.
- 40. Santanello NC, Baker D, Cappelleri JC, Copley-Merriman K, DeMarinis R, et al. (2002) Regulatory Issues for Health-Related Quality of Life-PhRMA Health Outcomes Committee Workshop, 1999. Value in Health 5: 14–25.
- 41. Chassany O, Sagnier P, Marquis P, Fullerton S, Aaronson N (2002) Patient-reported outcomes: the example of health-related quality of life-a European guidance document for the improved integration of health-related quality of life assessment in the drug regulatory process. Drug Information Journal 36: 209–238.
- 42.
EORTC (2002) Guidelines for Assessing Quality of Life in EORTC Clinical Trials.Available: http://groups.eortc.be/qol/sites/default/files/archives/clinical_trials__guidelines_qol.pdf Accessed 2014 22nd Sept.
- 43.
Movsas B (2003) Quality of life in oncology trials: a clinical guide. Elsevier. 235–247.
- 44. Calvert M, Freemantle N (2004) Use of health-related quality of life in prescribing research. Part 2: methodological considerations for the assessment of health-related quality of life in clinical trials. Journal of clinical pharmacy and therapeutics 29: 85–94.
- 45. Wiklund I (2004) Assessment of patient-reported outcomes in clinical trials: the example of health-related quality of life. Fundamental & clinical pharmacology 18: 351–363.
- 46. Buchanan DR, O'Mara AM, Kelaghan JW, Minasian LM (2005) Quality-of-life assessment in the symptom management trials of the National Cancer Institute-supported Community Clinical Oncology Program. Journal of clinical oncology 23: 591–598.
- 47.
Fayers PM, Hays RD (2005) Assessing quality of life in clinical trials: methods and practice: Oxford University Press.
- 48.
Lipscomb J (2004) Outcomes Assessment in Cancer: Use of health related quality of life measures by industry and regulatory agencies in evaluating oncology therapies. Chapter 27 Revicki DA p.
- 49. Avery K, Blazeby JM (2006) Quality of life assessment in surgical oncology trials. World journal of surgery 30: 1163–1172.
- 50.
TROG (2007) Assessing Health Related Quality of Life in Cancer Patients: A research clinicians guide. TROG Policy and Procedures.
- 51. Ganz PA, Gotay CC (2007) Use of patient-reported outcomes in phase III cancer treatment trials: lessons learned and future directions. Journal of Clinical Oncology 25: 5063–5069.
- 52. Lipscomb J, Reeve BB, Clauser SB, Abrams JS, Bruner DW, et al. (2007) Patient-reported outcomes assessment in cancer trials: taking stock, moving forward. Journal of Clinical Oncology 25: 5133–5140.
- 53. Land SR, Ritter MW, Costantino JP, Julian TB, Cronin WM, et al. (2007) Compliance with patient-reported outcomes in multicenter clinical trials: methodologic and practical approaches. Journal of Clinical Oncology 25: 5113–5120.
- 54. Patrick DL, Burke LB, Powers JH, Scott JA, Rock EP, et al. (2007) Patient-Reported Outcomes to Support Medical Product Labeling Claims: FDA Perspective. Value in Health 10: S125–S137.
- 55. Sloan JA, Dueck AC, Erickson PA, Guess H, Revicki DA, et al. (2007) Analysis and Interpretation of Results Based on Patient-Reported Outcomes. Value in Health 10: S106–S115.
- 56. Revicki DA, Erickson PA, Sloan JA, Dueck A, Guess H, et al. (2007) Interpreting and reporting results based on patient-reported outcomes. Value in Health 10: S116–S124.
- 57.
Fayers P, Machin D (2007) Quality of life: the assessment, analysis and interpretation of patient-reported outcomes: John Wiley & Sons.
- 58.
Fairclough DL (2012) Design and analysis of quality of life studies in clinical trials: CRC press.
- 59.
Hao Y (2010) Patient-reported outcomes in support of oncology product labeling claims: regulatory context and challenges.
- 60. NCIC CTG Quality of Life Committee Orientation Handbook (2014)
- 61.
Basch (2011) Recommendations for Incororating Patient-Reported Outcomes into the Design of Clinical Trials in Adult Oncology. Effective Guidance Document 1.0.
- 62. Efficace F, Taphoorn M (2012) Methodological issues in designing and reporting health-related quality of life in cancer clinical trials: the challenge of brain cancer studies. Journal of neuro-oncology 108: 221–226.
- 63. Jensen RE, Moinpour CM, Fairclough DL (2012) Assessing health-related quality of life in cancer trials. Clinical Investigation 2: 563–577.
- 64.
Novik et al. European Hematology Association:Scientific Working Group Guidelines:Patient-Reported Outcomes in Hematology: Chapter 2.
- 65. Macefield RC, Avery KN, Blazeby JM (2013) Integration of clinical and patient-reported outcomes in surgical oncology. Br J Surg 100: 28–37.
- 66.
King M University of Sydney, Psycho-oncology Cooperative Research Group (PoCoG) (2011) Quality of Life Protocol Checklist.
- 67. Kyte D, Draper H, Calvert M (2013) Patient-reported outcome alerts: ethical and logistical considerations in clinical trials. JAMA 310: 1229–1230.
- 68. Curran D, Molenberghs G, Fayers PM, Machin D (1998) Incomplete quality of life data in randomized trials: missing forms. Stat Med 17: 697–709.
- 69. Calvert M, Blazeby J, Altman DG, Revicki DA, Moher D, et al. (2013) Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension. Jama 309: 814–822.
- 70. Dolan P, Shaw R, Tsuchiya A, Williams A (2005) QALY maximisation and people's preferences: a methodological review of the literature. Health Econ 14: 197–208.