Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Children and young people’s contributions to public involvement and engagement activities in health-related research: A scoping review

  • Alison Rouncefield-Swales ,

    Contributed equally to this work with: Alison Rouncefield-Swales, Jane Harris, Bernie Carter, Lucy Bray, Toni Bewley, Rachael Martin

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Validation, Writing – original draft

    rouncefia@edgehill.ac.uk

    Affiliation Faculty of Health, Social Care and Medicine, Edge Hill University, Ormskirk, United Kingdom

  • Jane Harris ,

    Contributed equally to this work with: Alison Rouncefield-Swales, Jane Harris, Bernie Carter, Lucy Bray, Toni Bewley, Rachael Martin

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Validation, Writing – original draft

    Affiliation Faculty of Health, Public Health Institute, Liverpool John Moores University, Liverpool, United Kingdom

  • Bernie Carter ,

    Contributed equally to this work with: Alison Rouncefield-Swales, Jane Harris, Bernie Carter, Lucy Bray, Toni Bewley, Rachael Martin

    Roles Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Supervision, Validation, Writing – review & editing

    Affiliation Faculty of Health, Social Care and Medicine, Edge Hill University, Ormskirk, United Kingdom

  • Lucy Bray ,

    Contributed equally to this work with: Alison Rouncefield-Swales, Jane Harris, Bernie Carter, Lucy Bray, Toni Bewley, Rachael Martin

    Roles Conceptualization, Formal analysis, Investigation, Validation, Writing – review & editing

    Affiliation Faculty of Health, Social Care and Medicine, Edge Hill University, Ormskirk, United Kingdom

  • Toni Bewley ,

    Contributed equally to this work with: Alison Rouncefield-Swales, Jane Harris, Bernie Carter, Lucy Bray, Toni Bewley, Rachael Martin

    Roles Conceptualization, Formal analysis, Investigation, Validation, Writing – review & editing

    Affiliation Faculty of Health, Social Care and Medicine, Edge Hill University, Ormskirk, United Kingdom

  • Rachael Martin

    Contributed equally to this work with: Alison Rouncefield-Swales, Jane Harris, Bernie Carter, Lucy Bray, Toni Bewley, Rachael Martin

    Roles Formal analysis, Investigation, Validation

    Affiliation Faculty of Health, Social Care and Medicine, Edge Hill University, Ormskirk, United Kingdom

Abstract

Background

There has been an increasing interest in how children and young people can be involved in patient and public involvement and engagement (PPIE) in health research. However, relatively little robust evidence exists about which children and young people are reported as being involved or excluded from PPIE; the methods reported as being used to involve them in PPIE; and the reasons presented for their involvement in PPIE and what happens as a result. We performed a scoping review to identify, synthesise and present what is known from the literature about patient and public involvement and engagement activities with children and young people in health related research.

Methods

Relevant studies were identified by searches in Scopus, Medline, CINAHL, Cochrane and PsychInfo databases, and hand checking of reference lists and grey literature. An adapted version of the Guidance for Reporting Involvement of Patients and the Public (GRIPP2) was used as a framework to collate the data. Two reviewers independently screened articles and decisions were consensually made.

Main findings

A total of 9805 references were identified (after duplicates were removed) through the literature search, of which 233 full-text articles were assessed for eligibility. Forty studies published between 2000 and 2019 were included in the review. The review reveals ambiguities in the quality of reporting of PPIE with children with clear reporting on demographics and health conditions. The review found that children and young people were commonly involved in multiple stages of research but there was also significant variation in the level at which children and young people were involved in PPIE. Evaluation of the impact of children and young people’s involvement in PPIE was limited.

Conclusions

Consultation, engagement and participation can all offer children and young people worthwhile ways of contributing to research with the level, purpose and impact of involvement determined by the children and young people themselves. However, careful decisions need to be made to ensure that it is suited to the context, setting and focus so that the desired PPIE impacts are achieved. Improvements should be made to the evaluation and reporting of PPIE in research. This will help researchers and funders to better understand the benefits, challenges and impact of PPIE with children and young people on health research.

Introduction

The ten year plan for patient and public involvement and engagement (PPIE) for the National Institute for Health Research in the UK was published in 2015 [1, 2] and talks about public involvement as being central to “going the extra mile” to developing high-quality research [1]. This ten year plan commits to having “a population actively involved in research to improve health and wellbeing for themselves, their family and their communities” and the “public as partners in everything we do” [1]. The plan draws upon INVOLVE’s definitions of involvement being “research being carried out ‘with’ or ‘by’ members of the public rather than ‘to’, ‘about’ or ‘for’ them” [3], and engagement being “where information and knowledge about research is provided and disseminated” [3]. More recently, the UK standards for Public Involvement in Research [4] provides a framework for what good public involvement in research should look like and against which improvement can be assessed. The terminology in the field is subject to on-going debate with PPIE often used interchangeably, and in some cases incorrectly, with patient and public involvement (PPI). This review has taken a broad perspective and uses PPIE as an umbrella term to encompass both patient and public involvement (PPI) and patient and public engagement (PPE).

Although there is some good evidence being produced [2, 5] and many positive impacts arising from PPIE are reported, there is limited empirical ‘proof’ of the impact of PPIE on the quality of research [68]. Genuine PPIE is increasingly accepted as a necessary and fundamental component of well-designed and executed research [9, 10] that has meaningful benefits for patients [11]; yet, despite this, meaningful PPIE in research is neither consistently nor effectively utilised within all studies.

Despite pleas that the engagement of children and young people within a PPIE capacity in research studies should no longer be perceived as an optional extra [12], there is still considerable room for improvement [12]. Children and young people who participate in PPIE may benefit directly in many ways such as skills acquisition [8, 13] and gaining confidence [14]. Less direct, but nonetheless important benefits, are those accruing to the children and young people who gain meaningful benefit via guidelines and changes to clinical practice arising from research outcomes grounded in the high quality PPIE [8]. A key challenge is to ensure that the involvement of children and young people is authentic and not merely a tick box exercise [15] with tokenistic involvement operating at the bottom of Hart’s ladder of participation [16].

Typically, children and young people are involved in a variety of ways in PPIE in relation to health research ranging from consultation, through to involvement, collaboration and to user-led research [15, 17]. Child and young person advisory groups [8] are one way of involving children and young people. Their input to research studies can be focused on engagement at one time point such as the development of materials [18, 19] or involvement in data analysis [20], through to more active and continuous engagement across the lifespan of a study as seen in studies using co-design [21] and co-production [22] methodologies.

Despite there being noted benefits and challenges to PPIE, there remains inconsistency and a lack of transparency in how PPIE activities are reported which creates challenges for people wishing to review and appraise PPIE. Transparent reporting of PPIE supports future research, improves standards and best practice [23], reduces research ‘waste’ [24], helps increase public support and improves understanding about methods of effective involvement in research [25]. However, under-reporting and inconsistent quality of reporting of PPIE are noted in the literature [26]. The Guidance for Reporting Involvement of Patients and the Public reporting checklists (GRIPP2) [27] were developed as a means of improving the reporting of PPI. GRIPP2 aims to improve the quality, transparency, and consistency of reporting and to ensure that PPI practice is based on the best evidence. Reporting on PPIE in published research is an increasing requirement [28], but historically this was not the case and inconsistencies in the requirements of publishers mean that high-quality and consistent reporting remains low [26].

A scoping review [29] was undertaken with the aim to identify, synthesise and present what is known from the literature about patient and public involvement and engagement activities with children and young people in health related research.

Our objectives focused on:

  1. which children and young people are reported as being involved or excluded in PPIE;
  2. the methods reported as being used to involve them in PPIE; and
  3. the reasons presented for their involvement in PPIE and what happens as a result.

Methods

We structured the review using Arksey and O’Malley’s [30] five-stage framework (identifying the research questions; identifying relevant studies; study selection; charting data; and collating, summarising, and reporting the results) for conducting a scoping review. A sixth stage was added as we also undertook a parallel ‘consultation exercise’ as advised by Arksey and O’Malley [30]. Additionally, we were guided by Levac et al.’s [31] recommendations that clarify and enhance the study identification, study selection, and charting the data stages of a review.

Defining PPIE for this review

Our working definition for PPIE with children and young people for this review builds on the previously mentioned INVOLVE definitions [3] which are the most widely referred to definitions within the UK. Our intentionally broad definition of PPIE for this review–‘health related research being carried out with or by children and young people rather than to, about or for them’; or, ‘where information and knowledge about research is provided and disseminated’ [3].

Stage 1: Identifying the research question

An overarching research question guided our systematic search strategy and reporting of results: What is known about public involvement and engagement activities with children and young people in health related research? This question enabled us to adequately capture a broad range of existing literature while providing the opportunity for further research objectives to be added and modified throughout the review. This iterative process was useful as we became increasingly familiar with the literature.

Stage 2: Identifying relevant studies

Search terms were developed based on consideration of the population (children and young people aged under 25 years); concept (public involvement defined as doing research ‘with’ or ‘by’ the public, rather than ‘to’, ‘about’ or ‘for’ the public [32] and public engagement defined as “where information and knowledge about research is provided and disseminated” [1]; and context (health research).

A comprehensive list of search terms was identified and refined through searches on Scopus and Medline by the review team. AR and JH led the development of the search strategy with the support of a health research librarian. Truncation and proximity operators were employed to increase the sensitivity of the search (see S1 Appendix for full Medline search). Searches were undertaken in Scopus, Medline, CINAHL, Cochrane and PsychInfo databases in December 2019.

The reference lists of included documents were reviewed for additional papers, and Scopus and Google Scholar were consulted to identify the citing literature. A search of the grey literature (pdf files and webpages), including a hand search, was also completed in December 2019. Grey literature was identified via Open Grey, Google and from the websites of the Royal College of Paediatrics and Child Health, Royal College of Nursing, Barnardo’s, Department of Health, National Institute of Health Research Portfolio, Childlink, WellChild, Shine, Generation R, UNICEF, Save the Children, INVOLVE and iCan for research and conference publications.

Stage 3: Study selection

Studies were included in the review if they met the criteria outlined in Table 1. The search parameters were papers published 1st January 2000 – 16th December 2019. The age range of 0-24 years was used to encompass children and young people and was determined based on the World Health Organization (2014) [33] definition of a ‘child’ as a person under the age of 18 years and a ‘young person as under 24 years of age’ (10–24 years).

Citations were imported into Covidence (an online review tool that simplifies the screening and extraction process) and duplicates were removed. Following this, study selection occurred using a two-stage screening process. In Stage 1, titles and abstracts of all papers were each blind reviewed by two members of the review team (AR, JH) and conflicts resolved by a third reviewer from the team (TB, LB, BC). Seven papers, unavailable in full-text format, were excluded. In Stage 2, all members of the review team (AR, JH, TB, LB, BC) were allocated papers with two reviewers each screening all full-text papers. Conflicts were resolved through discussion between the two reviewers or where further advice needed by a third reviewer. Where papers relating to the same study reported on different aspects of the PPIE, all relevant papers were included.

Stage 4: Charting the data

We iteratively developed and refined a data extraction table reporting on nine key sections (Table 2) based on the Guidance for Reporting Involvement of Patients and the Public 2 (GRIPP2 short and long-form) [27]. The Guidance for Reporting Involvement of Patients and the Public–short-form and long-form (GRIPP2-SF and GRIPP2-LF) [27] reporting checklists provide key items for the reporting of PPI. The short version (GRIPP2-SF) is for reporting PPI in any study and a long version (GRIPP2-LF) is for when the study is mainly about PPI in research.

thumbnail
Table 2. Overview of data extraction items based on adapted Guidance for Reporting Involvement of Patients and the Public 2 (GRIPP2) form [27].

https://doi.org/10.1371/journal.pone.0252774.t002

Core data were extracted on the author(s), year of publication, country of origin, research study aim, methodology, method and sample. PPIE-specific data extraction (Table 2) focused on PPIE aim, PPIE terminology, definition and underpinning concepts/theory; PPIE population; PPIE design and stages of involvement; methods by which PPIE was evaluated; evidence of the impact of PPIE; conclusions and lessons learned from PPIE; limitations related to PPIE; and recommendations arising from PPIE. Five reviewers (AR, JH, TB, LB, BC) independently extracted data from their allocated articles with extractions checked for consistency and condensed by one reviewer (BC).

Quality assessment.

Although assessing the methodological and other qualities of studies within a scoping review can help contextualise findings and enable interpretation, Arksey and O’Malleyhttps://onlinelibrary.wiley.com/doi/full/10.1111/hex.13069 - hex13069-bib-0019 [30] suggest that quality assessment is not required and Levac [31] recognises the difficulties in assessing quality across a wide range of published and grey literature. Despite awareness of these issues, we initially sought to report on the quality of the studies using the Mixed Methods Appraisal Tool (MMAT) [34]. However, the MMAT directed our attention towards methodological issues when we were most interested in the quality of reporting of the PPIE.

No tool existed that enabled us to assess the quality of reporting from a broad range of papers, and which was not overly onerous. Having used elements of the GRIPP2 (LF- and SF) [27] to chart and extract the relevant information from the papers, we developed a short appraisal tool based on key areas of the GRIPP2-LF [27] using a similar format to the MMAT, to support our review of the quality of PPIE reporting. This tool focused on items necessary to assess the quality of the reporting for the focus of our review. One reviewer (AR) led the development of this appraisal tool. Through an iterative and condensing process involving the whole team, items from each section of the GRIPP2-LF were selected and turned into questions to develop a tool we named the QRIPPAT (Quality of Reporting Involvement of Patients and the Public Appraisal Tool). The QRIPPAT focuses on four broad themes each with several questions: PPIE aim and definition, PPIE methods, PPIE findings and discussion, and PPIE learning and reflections, all of which require only a Yes/No response (Table 3).

thumbnail
Table 3. QRIPPAT (Quality of Reporting Involvement and Engagement of Patients and the Public Appraisal Tool).

https://doi.org/10.1371/journal.pone.0252774.t003

Assigning scores to quality appraisal can be seen as contentious, especially if the scoring system is unweighted or unvalidated. However, scoring papers can reflect a measure of their worth and the score can be used, along with other considerations, as part of the critique of a paper. With this in mind, we have equally weighted each element of the QRIPPAT and assigned a score of 1 for each Yes and 0 for each No, giving a range of 0–10, where 0 is the lowest score for quality and 10 the highest quality score. The results of the QRIPPAT assessment (Table 4) reveals the overall quality in the reporting of the PPIE in the studies under review.

thumbnail
Table 4. Quality of Reporting Involvement and Engagement of Patients and the Public Appraisal Tool (QRIPPAT).

https://doi.org/10.1371/journal.pone.0252774.t004

Stage 5: Collating, summarising, and reporting results

Collated and summarised information from the included papers was added to the data extraction table in S1 Table and it proved invaluable in helping to develop initial themes that were finalised through discussions involving all members of the research team. Due to the wide variation in PPIE approaches in the included studies and significant variability in the quality of reporting of PPIE, a narrative approach offered the best fit to summarise the results of the review. The NIHR’s research process model [68] which outlines eight stages in the research process in which research-oriented PPIE can take place was condensed to create a better fit for reporting our findings. The findings outlined were explicitly reported by the study authors and not inferred by the authors of this review.

Stage 6: Consultation

Although not commonly used within scoping reviews, we were determined to ensure our review modelled good PPIE practice. Before commencing the review, we engaged with a young man who is an experienced service user via a service user group within our Faculty. This discussion helped to refine the scope of the review and, through his involvement, we incorporated exploring the wider impacts of PPIE and a critical consideration of not only which children and young people are included in PPIE but also those who may be excluded. Furthermore, the review benefited from the expertise of a team member who is a skilled Family Engagement Officer (RM) who challenged and influenced the team’s thinking through her insights into the priorities and areas which might be most important for children and young people involved in PPIE activities. RM provided feedback on the QRIPPAT extraction and the overall data extraction, checked the credibility of the evidence synthesis, identified gaps in the data and made suggestions as to present the findings.

Building on our earlier consultation work, the lay summary of the key points of the evidence generated from the review and recommendations for researchers is being co-produced by a group of young service users.

Findings

In the findings, we initially report on the study characteristics, including the settings and core matters relating to research studies linked to the reported PPIE and the findings from the QRIPPAT. We then present findings relating to our core objectives:

  1. which children and young people are reported as being involved or excluded in PPIE;
  2. the methods reported as being used to involve them in PPIE; and
  3. the reasons presented for their involvement in PPIE and what happens as a result.

Study characteristics

The 40 articles included 38 primary research studies (the PPIE from one study was reported in three articles) [39, 60, 67]. The PRISMA diagram below (Fig 1) illustrates the article search, screening, and review process. The PPIE was undertaken as part of research conducted in the United Kingdom (n = 32) [1820, 22, 3639, 42, 44, 45, 4767], Australia (n = 3) [14, 43, 46], United States (n = 3) [21, 35, 41], Canada (n = 1) [13] and New Zealand (n = 1) [40]. The majority of the articles (n = 37) were published after 2010 [13, 1822, 3545, 4752, 5467] with three published between 2000–2010 [14, 46, 53].

Overview of study designs.

Most articles described a single health research study although four articles [39, 46, 55, 56] described multiple health research studies within which PPIE was embedded. The most commonly reported research designs were qualitative (n = 20) [13, 18, 2022, 36, 4143, 45, 48, 50, 51, 53, 54, 5759, 63, 65] and mixed-methods (n = 11) [14, 35, 39, 40, 46, 47, 55, 60, 64, 66, 67]; of these studies, five (qualitative n = 1, and mixed methods n = 4) drew on a participatory methodology [35, 36, 4143]. The remaining studies included randomised control trials (n = 4) [19, 37, 38, 44], case study (n = 2) [49, 52] or systematic review (n = 1) [61]. The study design was unclear in two articles [56, 62].

Quality assessment of reporting of PPIE.

The majority of articles (n = 25) focussed almost exclusively on reporting the PPIE. Fifteen articles devoted only a small segment of the paper to reporting PPIE [14, 22, 4044, 46, 49, 51, 5759, 64, 67]. The diversity in the nature of the articles led to difficulties in comparing the quality of the reporting. There was a wide variation in the quality of reporting of PPIE, as is evident in our assessment using the QRIPPAT (Table 4).

Overall, there was the greater clarity in reporting on the aim of the PPIE (n = 39) [13, 14, 1822, 3543, 4567], the demographics of the PPIE populations involved (n = 35) [13, 14, 1822, 3539, 41, 42, 4552, 5456, 5867] and the methods used to engage young people in PPIE (n = 40) [13, 14, 1822, 3567]. The clarity of reporting was mixed on the PPIE definition used (n = 22) [13, 18, 20, 22, 3739, 41, 43, 45, 47, 48, 50, 51, 53, 58, 6063, 66, 67], the impact of PPIE (n = 28) [13, 14, 1822, 35, 36, 3841, 4346, 49, 52, 54, 55, 6066], contextual and process factors affecting PPIE (n = 26) [13, 18, 19, 21, 22, 3539, 41, 43, 45, 46, 5055, 6063, 66, 67] and critical learning from the PPIE (n = 27) [13, 14, 1820, 3539, 41, 42, 4547, 5052, 55, 5963, 6567]. There was a lack of clarity in the quality of reporting on the conceptual influences underpinning the PPIE (n = 16) [18, 2022, 3537, 4143, 45, 57, 62, 63, 66, 67] and the methods of evaluating the impact of PPIE (n = 7) [18, 35, 45, 55, 58, 62, 66].

PPIE Terminology, definitions and conceptual and theoretical influences.

S1 Table summarises the data extraction of Public and Patient Involvement and Engagement (PPIE) for studies included in the scoping review. A variety of terminology was reported in papers to describe the PPIE with children and young people. Commonly, the term patient and public involvement (PPI) [1820, 36, 37, 44, 45, 4749, 5458, 60, 6467] was used, alongside terms such as involvement [38, 53, 61], user involvement [50, 52], advisory [40], and engagement [39]. Where PPIE was integral to the research methodology, participatory methodology terms were frequently used included participatory design [13, 14, 43, 46, 51, 59], community-based participatory research (CBPR) [41], peer research [35], co-research [42, 63], co-design [21], and co-production [62]. One study described adopting a child rights based approach (CRBA) [22].

Explicit definition of the terminology used with reference to relevant literature was provided in 16 papers [18, 22, 37, 41, 43, 4648, 50, 51, 53, 58, 6062, 67]. In 14 papers [13, 19, 20, 35, 36, 38, 39, 42, 45, 54, 57, 63, 65, 66], looser definitions often without reference to literature, were offered using terms such as research ‘with’ and ‘by’ young people. In ten papers, no definition of the terminology was provided [14, 21, 40, 44, 49, 52, 55, 57, 59, 64]. No difference in the inclusion and clarity of definition for PPIE activity was found between older or more recent publications, and the provision of a clear definition was not dependent on the terminology used.

Twenty-seven papers [13, 14, 19, 3640, 4455, 57, 58, 60, 61, 6466] gave no explicit reference to any conceptual models or theoretical influences which had informed their approach to PPIE in the study. Thirteen papers referred to a conceptual or theoretical model as informing their approach to PPIE: three referred to participatory design/methodology [42, 43, 59] and the remainder to a child rights-based approach (CRBA) [22], co-operative enquiry [63], co-production model [62], empowerment theory [41], experience based co-design (EBCD) [20], patient experience framework [57], responsive and managerial public involvement [18], systems design [21], user-centred design [67] and youth participatory action research [35].

Children and young people reported as being involved or excluded in PPIE

Most studies gave demographics of the children and young people involved in the PPIE in terms of age, with fewer (n = 22) specifying more detailed demographics, for example, gender or ethnicity, health conditions or health experiences of the children and young people [14, 18, 35, 36, 38, 4044, 46, 48, 49, 51, 53, 5659, 61, 62, 64]. Five studies did not report the age range of the children and young people [40, 44, 46, 53, 59]. One study specified they had engaged children younger than four years [37] and one had engaged with children and young people from 4–24 years old [22]. Seven studies had solely engaged those in young adulthood (18–25 years) [13, 18, 20, 35, 39, 60, 67] and twelve had solely engaged with adolescents (10–18 years) [21, 36, 42, 43, 45, 50, 51, 55, 58, 6163]. No studies had engaged solely with children <11 years. Five studies had spanned childhood and adolescence (aged between 5 to 18 years) [19, 48, 54, 64, 65] and nine had spanned adolescence and young adulthood (10–25 years) [14, 38, 41, 47, 49, 52, 56, 57, 66].

Eleven studies referred to children and young people having specific conditions: arthritis [45], asthma [37], cancer [66], diabetes [21, 39, 60, 67], physical disability [22, 50], PICU experiences [19, 54], and depression or stroke [20]. Three studies addressed more general health experiences, such as the experience of accessing healthcare services [47] or a range of health experiences or conditions [55, 65]. Three studies had engaged with children and young people with other experiences that impacted health, for example, drug use [13], care leavers [52], asylum seekers [52] and experiences of therapy [63].

Typically, the numbers of children and young people involved in PPIE were small, although the range was wide from three [18] to 300 [43]. Of the 33 studies which reported the number of children and young people involved, seventeen had involved ten or fewer children and young people [13, 18, 20, 21, 3537, 39, 41, 42, 48, 50, 59, 60, 63, 65, 67], eight had involved 11–20 children and young people [14, 19, 38, 45, 47, 51, 52, 55], six had involved 21–100 children and young people [22, 49, 54, 58, 61, 62] and two between 200–300 children and young people [43, 66].

In thirteen studies, the PPIE work involved a broader population than just children and young people and included parents [1821, 45, 55, 64], adult stakeholders [14, 54], professionals [58] and combinations of families and professionals [22, 50, 65].

How children and young people are recruited/identified for PPIE.

Twenty-six studies had recruited children and young people from specific populations with direct, relevant experience [13, 14, 18, 2022, 3537, 3943, 4547, 50, 52, 54, 55, 58, 60, 63, 66, 67]. Studies reported that they had recruited children and young people through schools [36], specialist conferences [66], multimedia campaigns [39, 60] and specialist clinics or services [21, 50, 54, 60]. One study had recruited through distributing flyers and posters to groups and organisations associated with people with lived experience [20] while another outlined recruiting through existing research study stakeholders and advertising through existing networks, advisory groups and via Twitter [58]. Recruitment and retention were commonly reported as being challenging [18, 19, 21, 37, 55, 58], often resulting in the adoption of pragmatic approaches such as working with existing PPIE groups. Involvement was reported as being more meaningful and embedded rather than tokenistic when the children and young people involved in PPIE and the researchers were able to form effective working relationships [38]. Recruiting smaller subgroups of children and young people with direct experience of the topic under study was identified as essential to create a good ‘fit’ [55].

Methods reported as being used to involve children and young people in PPIE

A condensed version of the NIHR research process model [68] (Fig 2) was used to guide the findings of children and young people’s involvement in PPIE. PPIE activities were reported as occurring throughout various stages of a research study, and most commonly, PPIE with children and young people occurred across multiple stages. However, in twelve studies children and young people’s involvement was limited to one stage of the research [19, 36, 37, 40, 44, 46, 50, 54, 61, 63, 64, 66], this was frequently within the initial stages of prioritising, design and development the research [19, 37, 44, 50, 54, 63, 64, 66]. In eight studies PPIE with children and young people occurred across two stages of a research study: prioritising, design and development, and undertaking stages [14, 48, 58, 65]; undertaking and analysis stages [42, 47]; analysis and dissemination stages [20]; and prioritising, design and development, and dissemination stages [18]. In twelve studies PPIE with children and young people occurred across three stages: prioritising, design and development, undertaking research, and analysis stages [35, 41, 43]; prioritising, design and development, undertaking research, and dissemination stages [21, 56, 57]; prioritising, design and development, analysis, and dissemination stages [67]; and, undertaking research, analysis, and dissemination stages [13, 39, 49, 51, 52]. In eight studies [22, 38, 45, 53, 55, 59, 60, 62] children and young people were involved across all four stages.

thumbnail
Fig 2. PPIE involvement in the stages of the research process.

https://doi.org/10.1371/journal.pone.0252774.g002

PPIE in design and development.

Twenty-nine papers reported on children and young people’s involvement in the development of research studies. Such activity was directed on establishing a focus relevant to children and young people [19, 37, 43, 45, 54, 62], defining research questions [14, 41, 50, 62, 65], reviewing protocols [48, 57], helping to establish interventions [21, 35, 38, 58], reviewing data collection methods and tools [13, 14, 18, 22, 35, 38, 40, 41, 45, 53, 5557, 59, 60, 6265, 67], advising on the participant recruitment strategy [44, 57, 59], advising on research population [40, 45], defining outcome measures [19, 21, 43, 44], advising on lay summaries and materials [18, 37], contributing to research ethics [19, 37, 59], and designing the study branding and logo [37, 50, 55, 56, 60, 66]. Some of this activity occurred pre-bid [37].

PPIE in undertaking and management.

In twenty-six papers, there was evidence of children and young people contributing to the undertaking and management of the research stage. Children and young people were involved in helping to monitor and manage the research [21, 39, 46, 47, 49, 60, 65], developing participant information and materials [14, 21, 22, 38, 46, 48, 49, 52, 53, 56, 57, 59, 60, 62, 65], contributing to researcher training and support [45], designing research and publicity updates [43, 55, 62], and co-facilitating/conducting data collection [13, 14, 35, 39, 4143, 49, 51, 55, 58].

PPIE in analysis.

Twenty-two papers reported on children and young people’s involvement in the analysis and reporting of studies. Children and young people were involved in data entry [35], developing themes and contributing to data analysis [20, 22, 35, 36, 38, 41, 42, 47, 49, 51, 52, 55, 59, 61, 62] and in supporting the interpretation of research findings [13, 39, 43, 45, 53, 60, 67].

PPIE in dissemination.

Nineteen papers reported children and young people’s involvement in PPIE in dissemination. Of these, twelve papers reported co-presenting at conferences, workshops, exhibitions and webinars [13, 38, 43, 52, 53, 5557, 60, 62, 67]. Seven papers reported supporting the development of interventions including online campaigns [43], videos [20, 22] and health management tools [21, 39, 60, 67]. Other types of reported PPIE with children and young people included supporting further grant applications [39, 60, 67] and contributing to research papers and reports [18, 22, 38, 45, 49, 51, 56, 59, 60].

Why PPIE is taking place and what happens as a result

Of the studies which presented evaluation methods most reported relatively simple descriptions of the impact of PPIE based on informal evaluations and often insufficient detail was presented. However, one paper acknowledged that an embedded evaluation methodology was able to offer robustness and determination of real-world meaningfulness [48].

Methods used to evaluate the PPI.

Twenty-eight studies did not report the methods used to evaluate the impact of the PPIE interventions or strategies [13, 14, 1921, 36, 37, 3944, 4648, 5054, 57, 59, 61, 6365, 67]. Eleven studies reported internal evaluations undertaken by the research team [18, 22, 35, 38, 45, 49, 56, 58, 60, 62, 66].

Only one study [55] formally evaluated the impact of children and young people’s involvement in PPIE through an independent mixed-methods evaluation. This evaluation highlighted the success of the study and the model of participation which had substantial involvement of children and young people as co-researchers and co-producers. The young people, project personnel, parents and carers and civil servants gave positive feedback on recruitment, training and support provided to the young people finding it to be effective, well planned and positively received. Furthermore, the evaluation suggested that children and young people’s life experiences shaped the study and ensured that there was a resonance with the issues under investigation.

Seven out of the eleven studies which reported internal evaluations did not report the methods used [18, 22, 35, 38, 49, 60, 66]. However, some included reflections and feedback from children and young people [18, 22, 35, 60], researchers [22, 49, 60] and post-event evaluation [66]. Reported methods of evaluation of PPIE activity [45, 57, 58, 62] included anonymous questionnaires [62], writing workshops with children and young people and study staff [57], confidential feedback [58] and measures of effectiveness from postings on social media [45].

Impact of involvement in PPIE on children and young people.

The reporting of the impact on children and young people was frequently anecdotal. Fourteen papers directly reported impacts on children and young people [13, 14, 20, 22, 35, 38, 4143, 47, 55, 57, 60, 62]. Reported impacts on children and young people included gaining skills such as research and technical skills [13, 14, 35, 41, 62] and skills and confidence in public speaking [13, 62] which were viewed as improving the children and young people’s chances of future employment [13, 35]. Through involvement with, and the support of professional networks [13, 60], some children and young people had found employment opportunities [13], roles within the community [41] and interest in pursuing a research career [20]. Young people were also reported as gaining softer skills such as confidence [14, 62] and advocacy skills [13, 14, 41]. Personal impacts on children and young people included empowerment and creating positive change [38, 43, 57], enjoyment [22, 42], feeling part of a team [47], feeling listened to [22], and developing a greater understanding of their rights [22]. One participatory study [41] reported a negative impact and a risk to children and young people’s safety when racial discrimination was experienced during data collection. The incident resulted in improvements to the study safety protocols.

Impact of involvement of children and young people in PPIE on the research and researchers.

Thirty-two papers reported on the impact of the involvement of children and young people in PPIE on the research and researchers [13, 14, 1922, 3541, 43, 44, 46, 4952, 54, 55, 57, 5967]. Two studies [35, 55] reflected on the considerable impact of the involvement of children and young people in PPIE throughout the whole research process which had strengthened the design, conduct and rigour of the study [35] and led to the research study having greater resonance with the issues under investigation [55]. Involvement also positively impacted on the researchers’ individual understanding of PPIE processes and their skills in involving children and young people throughout the research process [36, 38, 55, 60]. For one study, this led to securing further funding and having their work recognised as having a “robust and meaningful inclusion of patient perspectives” [60].

One study reflected on the impact being “primarily about change in researchers’ focus and approach to their field” [52] while another reported on the importance of involving children and young people in designing the structure and processes of the PPIE itself to ensure that timelines and expectations had a better fit with the children and young people’s lives [60].

There were also clear impacts on the different stages of the research process. During the development stages, positive impacts were noted with children and young people helping to identify priorities [54], ensuring studies were informed by a lived understanding [14, 52], reducing bias [36] and improving the ethical basis of the study [37]. Studies reported the impact of children and young people in the development of patient focused outcome measures [19, 44, 63] and in enhancing the researchers’ knowledge and perspective of the topic under study [52]. Four studies [49, 52, 56, 65] documented a change in the use of language due to the involvement of children and young people and in one study [56] this led to further research exploring language use in clinical settings.

During the stages of active research, children and young people provided insight into their perception of the relevance and appropriateness of the research which shaped the study focus, aims and design [19, 37, 41, 49, 50, 52, 55, 62, 65]. Involvement of children and young people had assisted in assessing the appropriateness of research tools and led to the use of more user-relevant tools [14, 3941, 44, 46, 48, 59, 6265]; this led to a complete change in the data collection methods in three studies [41, 62, 65]. Children and young people had assisted the design of tools by identifying lines of inquiry, helping with the wording and ensuring the acceptability of the approach to other children and young people [14, 44, 46, 52, 59, 63, 64] as well as suggesting changes to the study population [37, 40, 59]. Children and young people helped ensure study documentation were accessible and relevant [18, 19, 37, 65] (e.g., improving the wording of patient information and invitation letters) and improved the process and wording of the consent process [19, 48]. Evidence also suggests that the involvement of children and young people assisted in improving the attractiveness of the research study to the target population by suggesting enhancements to branding [66], recruitment strategies [39, 62] and through increased credibility [14].

During the stages of analysis and reporting, children and young people helped to ensure that the identification of themes, the interpretation of data incorporated children and young people’s perspectives and experiences [13, 14, 20, 22, 36, 49, 51, 52, 59, 61]. Children and young people’s input ensured that children and young people perspectives were privileged in writing up [51], that key issues were raised [22] and that recommendations and implications were based on the priorities of children and young people [59, 61]. Furthermore, children and young people provided clear advice on accessible versions of documentation [59]. One study [20] reflected on improvements to engaging children and young people in data analysis and moved towards conversational engagement in the analytical process.

During the stages of dissemination and implementation, the involvement of children and young people was reported as improving dissemination due to the influence and awareness-raising in the community [14]. Furthermore, the impact of children and young people was recognised in campaigns that were reported as more inclusive and mindful of children and young people’s views and experiences [21, 43].

Impact of involvement of children and young people on policy/community.

Several studies recognised that the impact of PPIE needs to be understood more broadly in how researchers may approach research in the future [20] and should question “how studies are positioned within a highly politicised funding-agenda” [52]. Few studies discussed the impact of the involvement of children and young people on policy with only two studies reporting a direct impact on policy [22, 41]. One study [41] led to the children and young people involved becoming vocal advocates influencing a local policy change, while the recommendations from the other study [22] were accepted by Government and local authorities. While evidence of broader impact was limited, there was an acknowledgement that papers such as those included in this review are contributing to the growing literature on the involvement of children and young people in health research [37].

Limitations of involvement of children and young people in PPIE

Typically, the limitations of the involvement of children and young people in PPIE were reported as population issues such as recruitment and retention issues [19, 21, 37, 41, 45, 48, 50, 51] and non-representative samples/selection bias [21, 41, 45, 50, 51, 53, 54, 60, 61]. Constraints on time reduced the extent and depth that children and young people were able to be involved [13, 22, 39, 43, 51] as did resource and financial restrictions [22, 37, 43, 51, 60]. There was a need for more comprehensive training for researchers to support the involvement of children and young people [36, 52] and for the improvements to the design of activities to ensure accessibility and engagement with children and young people [55, 61]. Delays to studies can impact on young people’s on-going involvement in a study and diminish the positive impact of training and capacity building for children and young people [41]. Several studies acknowledged that the robust involvement and participation of children and young people must commence at the earliest stages of a study to ensure children and young people views are integrated into planning and design [39, 46, 54, 55].

Other limitations arose relating to managing group dynamics [48], ensuring meaningful engagement and accountability [41, 43, 46, 52] and either under or over-estimation by the researchers of the children and young people’s knowledge and skills [47, 52] and ability to commit to time-intensive activities [13, 52, 63]. In some studies, children and young people actively resisted the limited roles assigned to them and negotiated more active and involved roles [13, 42]. Other limitations reflect the need for researchers to be more flexible in response to young people’s personal circumstances and to ensure the voices of young people “less frequently heard” are able to play an active role in the process [38]. This closely aligns with ensuring that children and young people’s perspectives are not lost in translation during the research process [43, 61].

Discussion

This scoping review has systematically examined the empirical evidence on PPIE with children and young people in health research. Our deliberately broad search terms aimed to capture the full range of PPIE activity meeting INVOLVE’s definitions of involvement as research “with or by” children and young people (rather than “to, about or for them”) and engagement as “where information and knowledge about research is provided and disseminated” [1, 3]. Our review found a broad spectrum of terminology to describe activities with children and young people spanning engagement and involvement through to participatory and co-production approaches. Traditionally, youth participation has been influenced by the hierarchical approach proposed in Hart’s Ladder of Participation [16] with engagement and involvement of children and young people in adult initiated decisions ranked lower than approaches that provide children and young people the opportunity for shared decision-making and initiating action. However, this hierarchical approach has been criticised for failing to acknowledge that different levels of participation will be valuable in different cultural contexts [69]. In particular, these hierarchical approaches place too much emphasis on adults controlling children and young people’s levels of participation, rather than acknowledging that children and young people should be able to choose how much participation they wish to have [69]. Our review found significant variation in how children and young people were being involved in PPIE with researchers emphasising the need for pragmatism and flexibility to achieve the desired PPIE impacts. Children and young people were reported as choosing to consult only on some elements of PPIE and lead others. They also selected to be less involved in some activities and more involved in others. This suggests that hierarchical approaches fail to recognise that the success of PPIE activities depend upon the preferences, abilities and availability of children and young people. Being able to judge and negotiate children and young people’s preferences and abilities was facilitated by ongoing and established authentic relationships.

Our review found good quality reporting on the methods and populations involved in PPIE. However, there was mixed reporting on the definitions and impacts of PPIE and poor reporting on the underpinning concepts and evaluation of PPIE impact. This aligns with systematic reviews of PPIE reporting in health and social care research, which also found inconsistent and poor quality reporting and limited evidence on PPIE impact [26, 70]. A before and after comparison study found that these inconsistencies have persisted despite the introduction of the GRIPP reporting standards [27], suggesting uncertainty about reporting will continue until PPIE is more clearly embedded into research practice [26].

Our review found that the reporting of PPIE with children and young people was largely dependent on 1) which children and young people were involved, 2) the methods used to involve them, and 3) the impact on children and young people, the research process and on policy. We address each of these contextual factors in turn before considering how the findings of our review can influence our reporting of effective, good quality PPIE for children and young people.

Which children and young people are reported as being involved in PPIE

Across the studies, children and young people aged 4–24 years were involved in health related PPIE. Only seven studies [19, 22, 37, 48, 54, 64, 65] had engaged with children under ten years old. A lack of clear reporting on additional demographic factors and limited reporting on health conditions make it difficult to comment on which children and young people were excluded from PPIE involvement. However, some studies did note the underrepresentation of ethnic minority groups [53, 54] and children and young people with severe or complex needs [45, 53, 54]. Wider literature on healthcare access and transition note the additional barriers faced by children and young people from ethnic minority groups [7173] and with chronic conditions [74, 75], suggesting PPIE is currently falling short of the ambition of involving all children and young people in research to “improve health and wellbeing for themselves, their families and their communities” [1, 2].

While the majority of studies recruited children and young people from populations with direct experience of the research topic, a substantial minority [19, 38, 44, 48, 49, 51, 53, 56, 57, 59, 61, 62, 64, 65] recruited from existing PPIE youth groups. Members of these established PPIE groups were reported as having good engagement with [38], favourable views towards health services and high health and research literacy [61]. This aligns with previous research findings which report that children and young people who are involved in PPIE tend to be more outgoing, self-confident, critical and comfortable in formal education than their non-involved peers [76, 77]. The barriers to recruiting and retaining children and young people identified in our review included constrained research timeframes [22, 37, 43], lack of funding [22, 37, 43, 60], and limits to young people’s availability to attend [13, 21, 41, 51], may compound this lack of diverse representation. This suggests the limits set by research and budgetary timelines can lead to PPIE which does not fit in with the lives of children and young people, particularly if they are not involved in negotiating preferable and feasible activities and timelines in the early planning stages [77, 78].

The methods reported as being used to involve children and young people in PPIE

The reviewed studies involved children and young people across a variety of research designs including qualitative, mixed methods, RCTs and systematic reviews. PPIE involvement took place at all phases of the research project, including development (n = 29), undertaking and management (n = 66), analysis (n = 22) and dissemination (n = 19). This breadth of involvement resonates with findings from a previous scoping review [79] and confirms that is it is feasible to effectively involve children and young people in PPIE at every stage of the research process. However, only half of the studies (n = 20) had PPIE involvement across more than two of these phases suggesting that PPIE involvement is often either not sufficiently planned and/or sustained across the life cycle of research projects.

As noted in a previous scoping review of the involvement of children and young people in research design [80], much of the PPIE involvement reported exists on the lower levels of Hart’s ladder of participation [16], where young people are consulted and involved in adult initiated decisions but do not have opportunities for shared-decision making or initiating action [16]. Many of the reviewed studies noted that shorter-term, consultation based activities limited the opportunities for children and young people and researchers to build relationships [18, 39, 51] but that more meaningful and authentic engagement was achieved when young people were given or chose to adopt more active roles [13, 40, 41, 46, 48, 62, 63]. This concurs with previous reviews which found that more active PPIE involvement can lead to more positive impacts [79, 81]. However, we would argue as do others [69], that these hierarchical views of what constitutes ‘good’ PPIE can create fewer opportunities for flexible PPIE based on what young people would choose, want, and have the time and assets to participate in. This is not to say that expectations of what PPIE activities can be achieved should be lowered. One study in our review described children and young people negotiating a more active PPIE role by leading workshops, data collection and analysis [42], emphasising the importance of being guided by the desire of children and young people, if evident, for greater research involvement. However, as reported by several studies in our review, moving PPIE involvement towards more active roles for children and young people is complex and requires flexibility, planning [18, 41, 47, 54, 62] and training for both children and young people and researchers [45].

Why PPIE is taking place and what happens as a result

The impact of PPIE is a key aspect of the UK standards [4] and encourages researchers to identify and share the benefits, changes and learning gained from public insights. Despite increased commitment among researchers, funders and policymakers to demonstrate the positive impacts of PPIE involvement [1, 2, 82], the evidence base on the impacts of PPIE remains patchy and largely observational [80, 83]. It has been suggested that difficulties in assessing the impact of PPIE arise due to the complexity of PPIE [84] and that impact is highly contingent on context and the precise nature of involvement [85].

In agreement with a previous systematic review on the impacts of adult PPIE [81], our review suggests a wide range of positive PPIE impacts for children and young people, research, policy and the wider community across all phases of the research process. Evidence from the reviewed studies suggests that the involvement of children and young people help to refine research priorities, increase the accessibility and attractiveness of research methods and ensure children and young people’s perspectives are represented in analysis, outputs and dissemination. A range of positive individual (for example, new research skills or increased confidence) and interpersonal (for example, professional and community networks) impacts were reported for children and young people. Two studies suggested longer-term impacts including employment [35] and community advocacy roles [13]. However, in line with previous research [80, 83], the majority of this evidence on impacts was anecdotal, with only seven studies using formal methods to evaluate and evidence the impact of PPIE. While the GRIPP-2 reporting standards have the potential to improve the reporting of PPIE impact, this evidence base can only be developed through the inclusion of formal, planned evaluation activities during the early research development stages [83]. However, we would again argue against the hierarchical view of evaluation that prioritises statistical evidence of impact [86]. The diverse range of impacts noted in the reviewed studies suggests a context specific approach to impact evaluation should be taken [86] which consults with children and young people, to ensure that PPIE has positive impacts not only for researchers and research funders but also for the children and young people who have volunteered their time to participate [82].

Consistent with the search strategy which was focused on health research rather than health policy, only two studies had reported positive impacts of PPIE with children and young people on policy which is unsurprising as policy impacts tend to take longer to be realised and often fall outside of the timescales of a research project. The numerous life transitions experienced by children and young people such as moving schools, colleges and universities; transitioning from child to adult services; and age restrictions on organised groups can make it more difficult for researchers to sustain relationships and measure long term outcomes once their research has been concluded. Research funders should therefore consider how resources allocated for PPIE can be most effectively used, including allowing continued activity and follow-up beyond the time-restrictions of the research project.

Implications for practice

Our scoping review found that PPIE with children and young people is feasible across all phases of research for a range of ages, populations and contexts. Efforts should be made to prevent the exclusion of children and young people from minority ethnic groups, younger children, those with severe or complex need, lower socio-economic groups and other marginalised populations from PPIE. Smaller groups, varied opportunities for participation according to abilities and preferences, and community organisation partnership can allow more diverse groups of children and young people to be recruited and retained. Engagement with children and young people appears more meaningful and authentic when it moves beyond consultation to allow them opportunities for active involvement and shared decision making across the life course of the project.

PPIE is widely regarded as having positive impacts on children and young people and research, but evaluation of PPIE activity must be incorporated at the planning stage to ensure these impacts are evidenced.

Overall, our review found that reporting of PPIE activity with children and young people was variable and there was poor quality reporting of definitions, underpinning theory, and the evaluation of PPIE impact. Despite the existence of the GRIPP-2 quality reporting standards [80], uncertainty is evident around reporting of PPIE for children and young people, particularly as it is often not a requirement of research publications.

We would encourage researchers and practitioners to embrace a broader understanding of what effective, good quality PPIE can be in the context of their work with children and young people. Based on the findings of this scoping review we recommend a new, non-hierarchical perspective on PPIE for children and young people that encompasses, supports and values all elements of participation by acknowledging that a flexible, context specific approach to involvement and engagement, led by children and young people’s choice, is necessary.

PPIE with children and young people should be pragmatic, flexible, and suited to the context, setting and the focus of the study. It should be led by children and young people’s preferences, choices, abilities and interests and should respect their time, skills, and commitment. Consultation, engagement and participation can all offer children and young people worthwhile ways of contributing to research with the level, purpose and impact of involvement determined by the children and young people themselves.

Strengths and limitations of the scoping review

Although we used a wide range of search terms to capture the variation in PPIE terminology, inconsistencies in the keywords and terms used for PPIE means some studies may not have been included and this may have limited the findings. The lack of formal evaluation and the lack of standardised and clear reporting of PPIE led to difficulties in extracting relevant information from the papers and limits what can be reported in our review. Few journals require information about PPIE, which may lead to the under-reporting of PPIE activities. While the British Medical Journal (BMJ) has introduced the requirement for specific information about PPI, a recent study found that only 11% of studies published in the BMJ reported PPI activity [26].

Despite these limitations, our study has strengths such as our commitment to PPIE. Our search was strengthened by the involvement of a PPIE expert (RM) and a service user who helped us define our research questions, focus our data extraction and this input strengthened our findings. A further strength of our study was our commitment to overcome the lack of a suitable tool for appraising the quality of PPIE reporting. To address this, we have developed a simple tool for quality appraisal of PPIE research. This tool, the QRIPPAT (Quality of Reporting Involvement of Patients and the Public Appraisal Tool), was developed from a validated reporting checklist, the GRIPP2 [27], and has potential to be developed further and used to evaluate ca the quality of PPIE reporting across multiple studies.

Conclusion

Our review notes that PPIE extends beyond the boundaries of INVOLVE’s accepted definition of “research being carried out ‘with’ or ‘by’ members of the public rather than ‘to’, ‘about’ or ‘for’ them” and engagement being “where information and knowledge about research is provided and disseminated”[3]. PPIE is more complex and involved than this simple definition suggests and allows. Children and young people need researchers to adopt pragmatic and flexible ways of working and their preferences, choices, abilities and interests need to be central to the way that researchers work with them. Consultation, engagement and participation can all offer children and young people worthwhile ways of contributing to research with the level, purpose and impact of involvement determined by the children and young people themselves. However, careful decisions need to be made to ensure that it is suited to the context, setting and focus so that the desired PPIE impacts are achieved.

Improvements should be made to the evaluation and reporting of PPIE in research. This will help researchers and funders to better understand the benefits, challenges and impact of PPIE with children and young people on health research. Our review further concludes that PPIE should be consistently and clearly reported in all studies and with further development the QRIPPAT could offer a clear way forward for quality appraisal of PPIE research.

Implications

PPIE with children and young people should be pragmatic, flexible and led by children and young people’s preferences, choices, abilities and interests and should respect their time, skills, and commitment.

Supporting information

References

  1. 1. Denegri S, Coldham T, Eglin S, Frost R, Kerridge L, Matthews R, et al. Going the extra mile: Improving the nation’s health and wellbeing through publicinvolvement in research. London: National Institute for Health Research; 2015 2015.
  2. 2. Research NIfH. Taking stock—NIHR public involvement and engagement. London: National Institute for Health Research; 2019 26/06/2019.
  3. 3. INVOLVE. What is public involvement in research?: NIHR; 2015 [Available from: https://www.invo.org.uk/find-out-more/what-is-public-involvement-in-research-2/.
  4. 4. NIHR. UK standards for Public Involvement in Research London; 2019.
  5. 5. Young K, Kaminstein D, Olivos A, Burroughs C, Castillo-Lee C, Kullman J, et al. Patient involvement in medical research: what patients and physicians learn from each other. Orphanet journal of rare diseases. 2019;14(1):21–7. pmid:30678705
  6. 6. Brett J, Staniszewska S, Mockford C. The PIRICOM Study. A Systematic Review of the Conceptualisation, Measurement, Impact and Outcomes of Patient and Public Involvement in Health and Social Care Research. London; 2010.
  7. 7. Chambers E, Gardiner C, Thompson J, Seymour J. Patient and carer involvement in palliative care research: An integrative qualitative evidence synthesis review. Palliative medicine. 2019;33(8):026921631985824–984. pmid:31250702
  8. 8. Gaillard S, Malik S, Preston J, Escalera BN, Dicks P, Touil N, et al. Involving children and young people in clinical research through the forum of a European Young Persons’ Advisory Group: needs and challenges. Fundamental & Clinical Pharmacology. 2018;32(4):357–62. pmid:29457267
  9. 9. Davies HT, Phillips B, Preston J, Stones SR. Making research central to good paediatric practice. Archives of Disease in Childhood. 2019;104(4):385–8. pmid:30902887
  10. 10. Beier K, Schweda M, Schicktanz S. Taking patient involvement seriously: a critical ethical analysis of participatory approaches in data-intensive medical research. BMC medical informatics and decision making. 2019;19(1):90–10. pmid:31023321
  11. 11. Slattery P, Saeri AK, Bragge P. Research co-design in health: a rapid overview of reviews. Health research policy and systems. 2020;18(1):17–3. pmid:32046728
  12. 12. Molloy EJ, Mader S, Modi N, Gale C. Parent, child and public involvement in child health research: core value not just an optional extra. Pediatric research. 2019;85(1):2–3. pmid:30531982
  13. 13. Funk A, van Borek N, Taylor D, Grewal P, Tzemis D, Buxton JA. Climbing the "Ladder of Participation": Engaging experiential youth in a participatory research project. Canadian Journal of Public Health. 2012;103(4):288–92.
  14. 14. Holmes W, Stewart P, Garrow A, Anderson I, Thorpe L. Researching Aboriginal health: experience from a study of urban young people’s health and well-being. Social science & medicine. 2002;54(8):1267–79. pmid:11989962
  15. 15. Preston J, Stones SR, Davies H, Preston J, Phillips B. How to involve children and young people in what is, after all, their research. Archives of Disease in Childhood. 2019;104(5):494–500. pmid:31000534
  16. 16. Hart R. Children’s Participation: From Tokenism to Citizenship. Florence; 1992.
  17. 17. Bailey S, Boddy K, Briscoe S, Morris C. Involving disabled children and young people as partners in research: a systematic review: Involving disabled children and young people as partners in research. Child: care, health & development. 2015;41(4):505–14. pmid:25323964
  18. 18. Sheridan R, Preston J, Stones S, Ainsworth S, Horton-Taylor D, Challinor R, et al. Public Involvement—why bother? Reflections from a study informing the design of multimedia information resources to improve engagement of children, young people and their families with trials. 2019(Journal Article).
  19. 19. Tume LN, Preston J, Blackwood B. Parents’ and young people’s involvement in designing a trial of ventilator weaning. Nursing in Critical Care. 2016;21(3):e10–e8. pmid:26486094
  20. 20. Locock L, Kirkpatrick S, Brading L, Sturmey G, Cornwell J, Churchill N, et al. Involving service users in the qualitative analysis of patient narratives to support healthcare quality improvement. Research Involvement and Engagement. 2019;5(1).
  21. 21. Hannon TS, Moore CM, Cheng ER, Lynch DO, Yazel-Smith LG, Claxton GEM, et al. Codesigned shared decision-making diabetes management plan tool for adolescents with type 1 diabetes mellitus and their parents: Prototype development and pilot test. Journal of Medical Internet Research. 2018;20(5). pmid:33052121
  22. 22. Larkins C, Thomas N, Judd D, Lloyd J, Carter B, Farrelly N. “We want to help people see things our way”: a rights-based analysis of disabled children’s experience living with low income. Office of the Children’s Commissioner. 2013.
  23. 23. Jones EL, Williams-Yesson BA, Hackett RC, Staniszewska SH, Evans D, Francis NK. Quality of reporting on patient and public involvement within surgical research: a systematic review. Annals of surgery. 2015;261(2):243–50. pmid:24950279
  24. 24. Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, et al. How to increase value and reduce waste when research priorities are set. The Lancet. 2014;383(9912):156–65. pmid:24411644
  25. 25. Price A, Albarqouni L, Kirkpatrick J, Clarke M, Liew SM, Roberts N, et al. Patient and public involvement in the design of clinical trials: An overview of systematic reviews. Journal of evaluation in clinical practice. 2018;24(1):240–53. pmid:29076631
  26. 26. Price A, Schroter S, Snow R, Hicks M, Harmston R, Staniszewska S, et al. Frequency of reporting on patient and public involvement (PPI) in research studies published in a general medical journal: a descriptive study. BMJ Open. 2018;8(3):e020452. pmid:29572398
  27. 27. Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ. 2017;358:j3453. pmid:28768629
  28. 28. Aldcroft A. Patient partnership at The BMJ: Walking the talk http://blogs.bmj.com/bmjopen/2018/03/23/new-requirements-for-patient-and-public-involvement-statements-in-bmj-open/2018.
  29. 29. Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology. 2018;18(1):143. pmid:30453902
  30. 30. Arksey H O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005;8(1):19–32.
  31. 31. Levac D, Colquhoun H, O’Brien KK. Scoping studies: Advancing the methodology. Implementation Science. 2010;5(1):69–. pmid:20854677
  32. 32. Boote J, Baird W, Beecroft C. Public involvement at the design stage of primary health research: A narrative review of case examples. Health Policy. 2010;95(1):10–23. pmid:19963299
  33. 33. World Health O. Health for the world’s adolescents: a second chance in the second decade: summary. Geneva: World Health Organization; 2014 2014. Contract No.: WHO/FWC/MCA/14.05.
  34. 34. Hong QN, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, et al. MIXED METHODS APPRAISAL TOOL (MMAT) VERSION 2018. 2018.
  35. 35. Beatriz ED, Lincoln AK, Alder J, Daley N, Simmons F, Ibeh K, et al. Evaluation of a Teen Dating Violence Prevention Intervention among Urban Middle-School Youth Using Youth Participatory Action Research: Lessons Learned from Start Strong Boston. Journal of Family Violence. 2018;33(8):563–78. pmid:31341349
  36. 36. Best P, Badham J, Corepal R, O’Neill RF, Tully MA, Kee F, et al. Network methods to support user involvement in qualitative data analyses: An introduction to Participatory Theme Elicitation. Trials. 2017;18(1). pmid:29169378
  37. 37. Boote J, Julious S, Horspool M, Elphick H, Smithson WH, Norman P. PPI in the PLEASANT trial: Involving children with asthma and their parents in designing an intervention for a randomised controlled trial based within primary care. Primary Health Care Research and Development. 2016;17(6):536–48. pmid:26856306
  38. 38. Brady LM, Templeton L, Toner P, Watson J, Evans D, Percy-Smith B, et al. Involving young people in drug and alcohol research. Drugs and Alcohol Today. 2018;18(1):28–38.
  39. 39. Byrne M. Increasing the impact of behavior change intervention research: Is there a role for stakeholder engagement? Health Psychology. 2019;38(4):290–6. pmid:30896215
  40. 40. Carroll P, Witten K, Calder-Dawe O, Smith M, Kearns R, Asiasiga L, et al. Enabling participation for disabled young people: Study protocol. BMC Public Health. 2018;18(1). pmid:29884146
  41. 41. Chopel A, Lee RE, Ortiz-Matute E, Peoples N, Vagadori KH, Curtis A, et al. The META-oak project: Using photovoice to investigate youth perspectives on tobacco companies’ marketing of e-cigarettes toward adolescents in Oakland. SAGE Open. 2019;9(3).
  42. 42. Coad J. Involving young people as co-researchers in a photography project. Nurse Researcher. 2012;19(2):11–6. pmid:22338803
  43. 43. Collin P, Swist T. From products to publics? The potential of participatory design for research on youth, safety and well-being. Journal of Youth Studies. 2016;19(3):305–18.
  44. 44. Cooper C, Beard D, Jain A, Greig A, Sierakowski A, Gardiner M, et al. Patient and public involvement into the design of a paediatric surgical trial: the ninja study. 2017;18(Supplement 1) (no pagination).
  45. 45. Costello W, Dorris E. Laying the groundwork: Building relationships for public and patient involvement in pre-clinical paediatric research. Health Expectations. 2019. pmid:31625656
  46. 46. Curtin M, Murtagh J. Participation of Children and Young People in Research: Competence, Power and Representation. The British Journal of Occupational Therapy. 2007;70(2):67–72.
  47. 47. Dovey-Pearce G, Walker S, Fairgrieve S, Parker M, Rapley T. The burden of proof: The process of involving young people in research. Health Expectations. 2019;22(3):465–74. pmid:30770609
  48. 48. Forsyth F, Saunders C, Elmer A, Badger S. ‘A group of totally awesome people who do stuff’—a qualitative descriptive study of a children and young people’s patient and public involvement endeavour. Research Involvement and Engagement. 2019;5(1). pmid:30915233
  49. 49. Griffiths FE, Armoiry X, Atherton H, Bryce C, Buckle A, Cave JAK, et al. The role of digital communication in patient–clinician communication for NHS providers of specialist clinical services for young people [the Long-term conditions Young people Networked Communication (LYNC) study]: a mixed-methods study. 2018.
  50. 50. Hunt A, Brown E, Coad J, Staniszewska S, Hacking S, Chesworth B, et al. ’Why does it happen like this?’ Consulting with users and providers prior to an evaluation of services for children with life limiting conditions and their families. Journal of child health care: for professionals working with children in the hospital and community. 2015;19(3):320–33.
  51. 51. Kendal SE, Milnes L, Welsby H, Pryjmachuk S, Co-Researchers G, Shafeeah C, et al. Prioritizing young people’s emotional health support needs via participatory research. Journal of Psychiatric and Mental Health Nursing. 2017;24(5):263–71. pmid:28170132
  52. 52. Liabo K. Care leavers’ involvement in research: an ethnographic case study on impact. Sage. 2018.
  53. 53. Lightfoot J, Sloper P. Having a say in health: involving young people with a chronic illness or physical disability in local health services development. Children & Society. 2003;17(4):277–90.
  54. 54. Manning JC, Hemingway P, Redsell SA. Survived so what? Identifying priorities for research with children and families post-paediatric intensive care unit. Nursing in Critical Care. 2018;23(2):68–74. pmid:28516470
  55. 55. McLaughlin H. Involving Children and Young People in Policy, Practice and Research. London: NCB; 2015.
  56. 56. Mitchell SJ, Slowther A-M, Coad J, Akhtar S, Hyde E, Khan D, et al. Ethics and patient and public involvement with children and young people. Archives Of Disease In Childhood Education And Practice Edition. 2019;104(4):195–200. pmid:29440127
  57. 57. Mitchell S, Slowther AM, Coad J, Dale J. The journey through care: Study protocol for a longitudinal qualitative interview study to investigate the healthcare experiences and preferences of children and young people with life-limiting and life-threatening conditions and their families in the West Midlands, UK. BMJ Open. 2018;8(1). pmid:29358429
  58. 58. Morton KL, Atkin AJ, Corder K, Suhrcke M, Turner D, Van Sluijs EMF. Engaging stakeholders and target groups in prioritising a public health intervention: The Creating Active School Environments (CASE) online Delphi study. BMJ Open. 2017;7(1). pmid:28087549
  59. 59. Commissioner OotCs. “They still need to listen more”: a report about disabled children and young people’s rights in England. Office of the Children’s Commissioner. 2014.
  60. 60. O’Hara MC, Cunningham Á, Keighron C, Allen G, Caulfield A, Duffy C, et al. Formation of a type 1 diabetes young adult patient and public involvement panel to develop a health behaviour change intervention: The D1 now study. Research Involvement and Engagement. 2017;3(1).
  61. 61. Oliver K, Rees R, Brady LM, Kavanagh J, Oliver S, Thomas J. Broadening public participation in systematic reviews: A case example involving young people in two configurative reviews. Research Synthesis Methods. 2015;6(2):206–17. pmid:26099487
  62. 62. Pavarini G, Lorimer J, Manzini A, Goundrey‐Smith E, Singh I. Co‐producing research with youth: the NeurOx young people’s advisory group model. Health Expectations. 2019. pmid:31095837
  63. 63. Perry S, Carpenter S. Preliminary development and piloting of a user‐generated routine outcome measure in a children and young people’s counselling service. Counselling and Psychotherapy Research. 2016;16(3):171–82.
  64. 64. Royal College of P, Child H. The Development of a Patient Reported Experience Measure for Paediatrics Patients (0–16 years) in Urgent and Emergency Care: Research Report. London: RCPCH; 2012.
  65. 65. Snodin J, Bray L, Carter B, Jack B. Consulting with children, parents and a teacher to shape a qualitative study. Nurse Researcher. 2017;25(2):39–43. pmid:29115754
  66. 66. Taylor RM, Mohain J, Gibson F, Solanki A, Whelan J, Fern LA. Novel participatory methods of involving patients in research: Naming and branding a longitudinal cohort study, BRIGHTLIGHT. BMC Medical Research Methodology. 2015;15(1). pmid:25888163
  67. 67. Walsh D, Hynes L, O’Hara M, Mc Sharry J, Dinneen S, Byrne M. Embedding a user-centred approach in the development of complex behaviour change intervention to improve outcomes for young adults living with type 1 diabetes: The D1 Now Study [version 2; peer review: 2 approved]. HRB Open Research. 2018;1(8). pmid:32002502
  68. 68. Service NRD. Patient and public involvement in health and social care research: a handbook for researchers. 2018.
  69. 69. Treseder P. Empowering Children and Young People. London; 1997. pmid:9066249
  70. 70. Blackburn S, McLachlan S, Jowett S, Kinghorn P, Gill P, Higginbottom A, et al. The extent, quality and impact of patient and public involvement in primary care research: a mixed methods study. Research Involvement and Engagement. 2018;4(1):16. pmid:29850029
  71. 71. Robards F, Kang M, Usherwood T, Sanci L. How Marginalized Young People Access, Engage With, and Navigate Health-Care Systems in the Digital Age: Systematic Review. Journal of Adolescent Health. 2018;62(4):365–81.
  72. 72. Elster A, Jarosik J, VanGeest J, Fleming M. Racial and Ethnic Disparities in Health Care for Adolescents: A Systematic Review of the Literature. Archives of Pediatrics & Adolescent Medicine. 2003;157(9):867–74. pmid:12963591
  73. 73. Markkula N, Cabieses B, Lehti V, Uphoff EP, Astorga S, Stutzin F. Use of health services among international migrant children–a systematic review. Globalization and Health. 2018;14. pmid:29382348
  74. 74. Betz CL, Lobo ML, Nehring WM, Bui K. Voices not heard: A systematic review of adolescents’ and emerging adults’ perspectives of health care transition. Nursing Outlook. 2013;61(5):311–36. pmid:23876260
  75. 75. Chu PY, Maslow GR, von Isenburg M, Chung RJ. Systematic Review of the Impact of Transition Interventions for Adolescents With Chronic Illness on Transfer From Pediatric to Adult Healthcare. J Pediatr Nurs. 2015;30(5):e19–e27. pmid:26209872
  76. 76. van Schelven F, Boeije H, Mariën V, Rademakers J. Patient and Public Involvement of young people with a chronic condition in projects in health and social care: A scoping review. Health Expectations. 2020;n/a(n/a). pmid:32372423
  77. 77. Fox R. Resisting participation: critiquing participatory research methodologies with young people. Journal of Youth Studies. 2013;16(8):986–99.
  78. 78. Preston J, Stones SR, Davies H, Phillips B. How to involve children and young people in what is, after all, their research. Archives of Disease in Childhood. 2019;104(5):494–500. pmid:31000534
  79. 79. Flynn R, Walton S, Scott SD. Engaging children and families in pediatric Health Research: A scoping review. Research Involvement and Engagement. 2019;5(1). pmid:31700676
  80. 80. Larsson I, Staland-Nyman C, Svedberg P, Nygren JM, Carlsson I-M. Children and young people’s participation in developing interventions in health and well-being: a scoping review. BMC Health Services Research. 2018;18(1):507. pmid:29954392
  81. 81. Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expectations. 2014;17(5):637–50. pmid:22809132
  82. 82. Staniszewska S, Adebajo A, Barber R, Beresford P, Brady L-M, Brett J, et al. Developing the evidence base of patient and public involvement in health and social care research: the case for measuring impact. International Journal of Consumer Studies. 2011;35(6):628–32.
  83. 83. Johns T, Whilbley C, Crossfield S. Closed Study Evaluation: Measuring Impacts of Patient and Public Involvement and Research Quality Performance. London; 2015.
  84. 84. Crocker JC, Boylan A-M, Bostock J, Locock L. Is it worth it? Patient and public views on the impact of their involvement in health research and its assessment: a UK-based qualitative interview study. Health Expectations. 2017;20(3):519–28. pmid:27338242
  85. 85. Staley K, Buckland SA, Hayes H, Tarpey M. ’The missing links’: understanding how context and mechanism influence the impact of public involvement in research. Health Expect. 2014;17(6):755–64. pmid:23107054
  86. 86. Staley K. ‘Is it worth doing?’ Measuring the impact of patient and public involvement in research. Research Involvement and Engagement. 2015;1(1):6. pmid:29062495