Skip to main content
Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Digital technology and patient and public involvement (PPI) in routine care and clinical research—A pilot study

  • Yang Chen ,

    Contributed equally to this work with: Yang Chen, Ali A. Hosin

    Roles Conceptualization, Data curation, Methodology, Resources, Writing – original draft, Writing – review & editing

    Affiliations Institute of Health Informatics, Faculty of Population Health Sciences, University College London, London, United Kingdom, Clinical Research Informatics Unit, University College London Hospitals, London, United Kingdom

  • Ali A. Hosin ,

    Contributed equally to this work with: Yang Chen, Ali A. Hosin

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Project administration, Resources, Writing – original draft, Writing – review & editing

    Affiliation Clinical Pharmacology Department, University College London Hospitals, London, United Kingdom

  • Marc J. George,

    Roles Funding acquisition, Methodology, Project administration, Writing – review & editing

    Affiliation Clinical Pharmacology Department, University College London Hospitals, London, United Kingdom

  • Folkert W. Asselbergs,

    Roles Resources, Supervision, Writing – review & editing

    Affiliations Institute of Health Informatics, Faculty of Population Health Sciences, University College London, London, United Kingdom, Clinical Research Informatics Unit, University College London Hospitals, London, United Kingdom, Division Heart & Lungs, Department of Cardiology, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands, Institute of Cardiovascular Science, Faculty of Population Health Sciences, University College London, London, United Kingdom

  • Anoop D. Shah

    Roles Methodology, Project administration, Supervision, Writing – review & editing

    Affiliations Institute of Health Informatics, Faculty of Population Health Sciences, University College London, London, United Kingdom, Clinical Research Informatics Unit, University College London Hospitals, London, United Kingdom, Clinical Pharmacology Department, University College London Hospitals, London, United Kingdom



Patient and public involvement (PPI) has growing impact on the design of clinical care and research studies. There remains underreporting of formal PPI events including views related to using digital tools. This study aimed to assess the feasibility of hosting a hybrid PPI event to gather views on the use of digital tools in clinical care and research.


A PPI focus day was held following local procedures and published recommendations related to advertisement, communication and delivery. Two exemplar projects were used as the basis for discussions and qualitative and quantitative data was collected.


32 individuals expressed interest in the PPI day and 9 were selected to attend. 3 participated in person and 6 via an online video-calling platform. Selected written and verbal feedback was collected on two digitally themed projects and on the event itself. The overall quality and interactivity for the event was rated as 4/5 for those who attended in person and 4.5/5 and 4.8/5 respectively, for those who attended remotely.


A hybrid PPI event is feasible and offers a flexible format to capture the views of patients. The overall enthusiasm for digital tools amongst patients in routine care and clinical research is high, though further work and standardised, systematic reporting of PPI events is required.


Patient and public involvement (PPI) work has increasingly gained prominence in healthcare settings. Organisations such as the National Institute for Health Research (NIHR) [1] and the Patient Centered Outcomes Research Institute (PCORI) [2] advocate for PPI and engagement (PPIE) to play a central role in research design. Greater PPI participation in routine clinical care was also highlighted in a recent scoping review [3]. Developing alongside these innovations has been the increasing use of digital technologies in healthcare [4]. Most recently, a systematic review of patient and public views on artificial intelligence (AI)–at the frontier of digital technologies–highlighted that whilst there were positive attitudes toward AI, reservations remained and human supervision of AI in healthcare decision-making was a preferred [5]. As PPI and digital tools continue to increase their mark in healthcare, there is growing need for research that lies at the intersection of these two trends. This study helps to address this gap in the literature and had the following aims:

  1. To determine the feasibility of hosting a PPI event using digital tools and to gather feedback on the usefulness of a hybrid participation format
  2. To gather patient views on the role of digital technologies in routine care delivery and clinical research, with specific focus on remote monitoring of blood pressure, consent processes and patient reported outcome measures (PROMs) as exemplar themes.


To minimise digital exclusion [6], patients were recruited through online and offline means (UCLH NIHR Biomedical Research Centre, British Heart Foundation Networks and UCLH clinical pharmacology clinic). Box 1 outlines the specific projects discussed during the event.

Box 1. Projects discussed during the PPI event

Our discussions were framed around two projects that were both in their early design/ pilot phase:

1. A quality improvement project (QIP) which related to the remote monitoring of blood pressure through a bespoke smartphone application or website and safe data linkage to the patient electronic care record. The specific platform discussed which would be used to host remotely collected data was the ‘Mycare UCLH patient portal’ [7].

2. A research study proposal examining the use of digital tools during hospital stays to both as interventions such as clinician-facing alerts, as well as platforms to document patient views.

After registration of interest, individuals were invited to complete a short survey to confirm their availability and to optionally volunteer anonymised demographic data to aid the selection of a diverse cohort of PPI participants. Other background information including social determinants of health were not collected in order to prioritise accessibility and ease of completion. The venue was chosen to ensure adherence to social distancing with appropriate audio-visual facilities to allow two-way communication between live and remote participants. An upper limit of 10 participants for the event was used based on previous guidance for focus group size [8]. In line with NHS governance policy, Microsoft Teams was the chosen online platform [9]. A shortlist of 10 participants was purposively selected to generate a balanced group. This was based on protected characteristics such as age and gender, as well as pre-event confidence in using digital tools.

A scribe was appointed amongst the study group who contemporaneously recorded the key themes of the verbal discussion. Direct quotations were used, and quantitative feedback was collected in selected data capture forms that were distributed. The pre-attendance short survey allowed prospective participants to provide written consent before the event, and verbal consent was taken on the day for use of non-attributable quotations that arose from discussions. A short feedback form was distributed to participants after the event. The delivery and contents of the day including slide sets used to trigger co-design discussions are available in the S1 File. This study is reported according to the GRIPP2 long form where relevant and a checklist is supplied in the S1 File [10]. In accordance with the UK Health Research Authority decision tool on ethical requirements for research [11] and specific guidance from the National Institute for Health and Care Research on the ethics of PPI events, no approval from an ethics committee was sought [12].


A total of 32 participants expressed interest in the event. 19 patients were available to attend on the day and a summary of their characteristics is found in Table 1. A total of 9 participants attended on the day (3 face-to-face, 6 via Microsoft Teams).

Outcomes generated from the day and in feedback were grouped into qualitative and quantitative domains.

Qualitative outcomes

The following selected quotations were informative in driving changes to the proposed care pathway for remote monitoring of BP:

You can spot trends, I wore a heart monitor, especially post-surgery

But the downsides arehow do you motivate people, is it anxiety provoking”—Participant 3

I’m really interested in what stress and anxiety can do to your body but what about doing it everyday obsessively and a bit more than I need to. At what point should I be making contact with my GP.”—Participant 8

These quotes were indicative of the enthusiasm for remote monitoring that the group had and their inquisitiveness about how such their BP data would be shared with their usual care team. However, there was an emphasis additionally placed on safeguarding required to mitigate against over-measurement and the possible anxiety that elevated readings could create, though this was balanced with points made about elevated readings being positively viewed, in terms of being a motivating factor to engage in lifestyle behaviours such as exercise to reduce BP.

As a result of the group discussions, the following changes to the remote BP project were made:

  • Adjusting the pathway to share BP data and feedback with more members of the care team
  • Refinement of the data entry system to include app-based reminders and advice on when to recommend initiating further discussions
  • A proposal to incorporate elements of counselling and support at the time of diagnosis

In terms of the research proposal, discussions focused primarily on the ethics and logistics of delivering a clinician-facing digital alert as an intervention within a research study. A selected quotation with regards to waiver of consent was:

seems the most practical way of avoiding any behavioural effects that come from knowing you are in a research trial” Participant 8

This quote summarised the consensus view of the group in terms of how to balance the feasibility and safety of doing research studies of electronic alerts aimed at clinicians which seek to change their behaviour and downstream management strategies that can affect patients, In this regard, the group understood that thresholds for tolerating waived consent may vary case by case. There was a majority consensus for waived consent in the context of comparing existing non-pharmaceutical interventions such as fluid restriction versus no restriction, given the patient is the final arbiter of oral fluid intake. Support for waiver was not unanimous though and so additional review and communication strategies were sought by the trial management group because of the diverse and helpful patient feedback.

Quantitative outcomes

For the research proposal, different models of consent were rated on a 5-point Likert scale for suitability and responses were individually completed following discussion (Table 2). The average suitability for each consent model was: waiver of consent: 4.5/5; research registry: 3/5; verbal consent: 2.7/5. A significant amount of missing data was noted for participants who attended remotely.

Table 2. Patient views on different models of consent for the research proposal.

Participants (i) to (iii) were those who attended in person. First, participants were asked to rank the models of consent in order of preference (1 = most preferred, 3 = least preferred). Next, participants were asked to rate each model out of 5 for suitability (1 = not at all suitable; 5 = entirely suitable).

Table 3 additionally summarises views collected on patient reported outcome measures (PROMs). Data was only available for the three participants who attended in person.

Table 3. Participant views on PROMs.

Participants were asked to what extent they agreed with specific statements on how patient reported outcome measures (PROMs) may be used (1 = not at all, 5 = very much agree).

Feedback on PPI event and impact of context

The event was held on December 3rd, 2021, and strict Covid-19 protocols were observed for those who attended in person. The hybrid format proved challenging in relation to timings observed for the day. Whilst the majority of participants (6 out of 9) were logged in remotely, ensuring an equitable distribution of time was difficult though partly helped by all participants having webcam access. Table 4 highlights feedback collected from 8 participants after the event with similar mean ratings between in person (overall rating 4/5, interactivity 4/5) and remote attendance (overall quality 4.5/5, interactivity 4.8/5).

Table 4. Summary of rated quality and interactivity of the day from participants.


We have shown in this study that a hybrid PPI event was feasible to deliver and received positive feedback that was similar between in person and remote attendance. The engagement from participants and their views was used to shape two distinct projects in their design phase.

Comparison with literature

We are unaware of existing published reports of hybrid PPI events which not only examine digital tools as a theme, but which used such tools to allow the safe conduct of the day. Whilst reducing commute times and accommodating those who are risk averse in the pandemic era are advantages, there is the caveat of ensuring factors such as digital exclusion [7] do not reinforce biases and underrepresentation of key groups.

Thus systematic design and delivery of PPI will require dedicated focus to equality, diversity and inclusion, particularly as upstream PPI begins to shape both research (including improving recruitment to clinical trials [13]) and routine service design (such as increased participation in co-design).

When compared to the distribution of current PPI events, [14] it is unclear if participation levels are representative of patients affected by each condition or treatment studied, or if the distribution of known and published PPI events match up to patient priorities more generally.


Our findings should be considered in light of several limitations. First, our sample size was small and there is the risk of ascertainment bias, given that recruitment of patients occurred through a limited number of sources. We mitigated this in part through a pre-event survey but nevertheless the views of the focus day may not be representative of the wider population, something that all PPI is at risk of [15].

Second, our event was time limited–at 4.5hours in duration including breaks, we sought to balance factors such as breadth and depth of coverage as well as risk of fatigue. Written feedback collected for the day noted that timing was tight and the afternoon session in particularly was abbreviated to adhere to time, ensuring that for in-person attendees would not miss their return travel home.

Third, the data collection form used to quantify views was a 1–5 Likert scale. We elected to forgo using a formal validated measuring instrument in favour of ease and speed of use. As a pilot event, we regarded the data collection exercise as more exploratory in nature.

Fourth, we did not use a formal framework to support our PPI day design. This was motivated by pragmatism, with existing evidence to suggest that most recognised frameworks have poor useability [16]. However, there remain questions regarding how best to harness PPI for implementation-focused clinical and research questions and previous work in the field has offered suggestions which require further exploration [17].

Future directions

The optimal way to harness PPI to help implement digital technologies and to inform how we test them as part of pragmatic research, [18] remain areas for further work. PPI efforts will be central to the maintenance of trust in their use. [19, 20].


A hybrid PPI event is feasible and offers a flexible format to capture patient and public views when planning clinical service design, quality improvement or research projects. It may be of particular benefit in capturing the views of those unable to attend otherwise, for reasons including mobility and clinical vulnerability. Striking a balance of ‘informal PPI’ with pragmatic recruitment and delivery of more formal events will be a consideration for all research and clinical teams. The challenges related to adequate time to facilitate discussions and to capture views during future hybrid formats will require additional staff training and sharing of feedback and experiences, allied to better reporting of PPI events.

Supporting information

S1 File. Additional material including information on study delivery, slides used and GRIPP2 reporting checklist.



We are grateful to the British Heart Foundation Heart Voices network were instrumental in helping to recruit patients and the assistance from Chloe Goldman in this regard. We are additionally grateful to Tom Lumbers for help with reviewing slides for the PPI day and contributing to the content and themes for the research element of the day. Lastly, we are grateful to all patients who expressed interest in the day and who kindly volunteered their time and views in helping to support this study and to inform the design of the two projects that have been discussed during this paper.


  1. 1. National Institute for Health Research. Be part of Research. (online) Accessed on 21 December 2021. Available at:
  2. 2. Patient-centered outcomes research institute. Pcori (Online). Accessed on 22 February 2022. Available at:
  3. 3. Goodridge D, McDonald M, New L, Scharf M, Harrison E, Rotter T et al. Building patient capacity to participate in care during hospitalisation: a scoping review. BMJ Open 2019;9:e026551. pmid:31272973
  4. 4. Ting DSW, Carin L, Dzau V, Wong TY. Digital technology and COVID-19. Nat Med 26, 459–461 (2020). pmid:32284618
  5. 5. Young AT, Amara D, Bhattacharya A, Wei ML. Patient and general public attitudes towards clinical artificial intelligence: a mixed methods systematic review. Lancet Digit Health. 2021 Sep;3(9):e599–e611. pmid:34446266
  6. 6. Clark M, van Vliet E, Collins M. Reflections from the COVID-19 pandemic on inequalities and patient and public involvement and engagement (PPIE) in social care, health and public health research. Health Expect. 2021 Oct;24(5):1547–1550. pmid:34369628
  7. 7. University College London Hospitals. MyCare UCLH (Online). Accessed on 7 March 2022. Available at:
  8. 8. Doria N, Condran B, Boulos L, Curtis Maillet DG, Dowling L, Levy A. Sharpening the focus: differentiating between focus groups for patient engagement vs. qualitative research. Res Involv Engagem. 2018 Jun 25;4:19 pmid:29983994
  9. 9. NHS Digital. NHSmail and Microsoft Teams. (Online), Accessed on 10 May 2022. Available at:
  10. 10. Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ. 2017 Aug 2;358:j3453. pmid:28768629
  11. 11. NHS Health Research Authority. HRA-Decision tools (Online). Accessed on 18 May 2022. Available at:
  12. 12. NIHR School for Primary Care Research Do I need ethical approval to run an involvement activity? (Online). Accessed on 18 May 2022. Available at:
  13. 13. Crocker JC, Ricci-Cabello I, Parker A, Hirst JA, Chant A, Petit-Zeman S et al. Impact of patient and public involvement on enrolment and retention in clinical trials: systematic review and meta-analysis. BMJ. 2018 Nov 28;363:k4738. pmid:30487232
  14. 14. National Institute for Health Research. Participant in Research Experience Survey (PRES) 2020/21 (online). Available at: Accessed on 21 December 2021
  15. 15. Lander J, Langhof H, Dierks ML. Involving patients and the public in medical and health care research studies: An exploratory survey on participant recruiting and representativeness from the perspective of study authors. PLoS One. 2019 Jan 7;14(1):e0204187. pmid:30615619
  16. 16. Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, Chant A. Frameworks for supporting patient and public involvement in research: Systematic review and co-design pilot. Health Expect. 2019 Aug;22(4):785–801 pmid:31012259
  17. 17. Gray-Burrows KA, Willis TA, Foy R, Rathfelder M, Bland P, Chin A et al. Role of patient and public involvement in implementation research: a consensus study. BMJ Qual Saf. 2018 Oct;27(10):858–864. pmid:29666310
  18. 18. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015 May 8;350:h2147. pmid:25956159
  19. 19. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A’Court Cet al. Beyond Adoption: A New Framework for Theorizing and Evaluating Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies. Journal of medical Internet research vol. 19,11 e367. 1 Nov. 2017 pmid:29092808
  20. 20. Kalkman S, van Delden J, Banerjee A, Tyl B, Mostert M, van Thiel G. Patients’ and public views and attitudes towards the sharing of health data for research: a narrative review of the empirical evidence. J Med Ethics. 2022 Jan;48(1):3–13. pmid:31719155