Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

From intent to implementation: Factors affecting public involvement in life science research

  • John A. Burns,

    Roles Conceptualization, Methodology, Project administration, Validation, Writing – original draft, Writing – review & editing

    Affiliations Lamont-Doherty Earth Observatory of Columbia University, Palisades, New York, United States of America, American Museum of Natural History, New York, New York, United States of America, eLife Ambassador for Good Practice in Science, Cambridge, United Kingdom

  • Sinead Holden,

    Roles Formal analysis, Writing – review & editing

    Affiliation Clinical Research Centre, School of Medicine, University College Dublin, Dublin, Ireland

  • Kora Korzec,

    Roles Formal analysis, Methodology, Project administration, Resources, Validation, Writing – original draft, Writing – review & editing

    Affiliation eLife Sciences Publishing, Cambridge, United Kingdom

  • Emma R. Dorris

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Validation, Visualization, Writing – original draft, Writing – review & editing

    emma.dorris@ucd.ie

    Affiliations eLife Ambassador for Good Practice in Science, Cambridge, United Kingdom, School of Medicine, University College Dublin, Dublin, Ireland

Abstract

Public involvement is key to closing the gap between research production and research use, and the only way to achieving ultimate transparency in science. The majority of life science research is not public-facing, but is funded by the public and impacts communities. We undertook an exploratory survey of researchers within the life sciences to better understand their views and perceived challenges to involving the public in their research. As survey response rate could not be determined, interpretation of the results must be cautious. We had a valid response cohort of n = 110 researchers, of whom 90% were primarily laboratory based. Using a mixed methods approach, we demonstrate that a top-down approach is key to motivate progression of life scientists from feeling positive towards public involvement to actually engaging in it. Researchers who viewed public involvement as beneficial to their research were more likely to have direct experience of doing it. We demonstrate that the systemic flaws in the way life sciences research enterprise is organised, including the promotion system, hyper-competition, and time pressures are major barriers to involving the public in the scientific process. Scientists are also apprehensive of being involuntarily involved in the current politicized climate; misinformation and publicity hype surrounding science nowadays makes them hesitant to share their early and in-progress research. The time required to deliberate study design and relevance, plan and build relationships for sustained involvement, provide and undertake training, and improve communication in the current research environment is often considered nonpragmatic, particularly for early career researchers. In conclusion, a top-down approach involving institutional incentives and infrastructure appears most effective at transitioning researchers from feeling positive towards public involvement to actually implementing it.

Introduction

There is increasing focus on the translation and implementation of new scientific evidence in real-world settings, in line with the increase in citizen science and public and patient involvement (PPI) initiatives [1]. Across the life sciences, narrowing the gap between research production and research use is a key challenge. There is increasing evidence that public involvement and stakeholder engagement is key to achieving impact and true provision and use of scientific knowledge for the benefit of society [28].

The field of public engagement has adopted a varied vocabulary where the meanings of specific terms can vary by discipline and by geography. In this study we focus on public involvement in research (Box 1). Public involvement sits in the sphere of public engagement activities (Fig 1). However, public involvement is a collaborative effort between researchers and the public in which the public can have a high level of influence on research. In life science fields not directly related to human health, such as ecological and environmental research, research projects that include members of the public as active participants are often called "citizen science" efforts. Citizen science efforts where members of the public work with professional scientists can be classified into three major categories: contributory, collaborative, and co-created projects [9]. The categories reflect the level of input and participation from members of the public (Fig 1). We use the term public in our context to mean people who are not career researchers in the given field. Although some initiatives may be targeted to reach an interested ‘general public’, targeting the general public as if they were an undifferentiated group of people is rarely successful [10]. The term “public” refers to a targeted audience, be they communities of people based on their interests, passions, or other shared circumstances.

Box 1. Definitions provided to survey participants prior to answering questions

Participation: public take part in an experiment, trial, or study as a subject. The person who participates is the target of observation by researchers. They are data providers. Examples: participant in a clinical trial.

Engagement: a one-way dialogue from researcher to public where the researcher explains/educates/informs the public about their research. Examples: public lectures, lab tours, research demonstrations

Consultation: a one-way dialogue from public to researcher where the public’s input on matters affecting them is sought. The public are usually not decision makers. Examples: Surveys or focus groups to inform policy or governance

Involvement: two-way dialogue between public and researchers. The active involvement between people who use services, the public and researchers. Public are active collaborators, akin to researchers, clinicians/professionals or managers who are asked for their views and experiences when contributing to research. The public usually have some degree of decision making. Examples: Research advisory groups members, citizen science, public educators or mentors.

thumbnail
Fig 1. Public involvement within the spectrum of public engagement.

Public Engagement is a broad term that includes the multi-faceted and varied ways researchers engage with others outside their field of expertise. Public engagement in research consists of different levels of activities, characterised based on their ability to influence research. These frequently overlap, as successful higher-level engagement is usually built on strong foundational (lower level) engagement.

https://doi.org/10.1371/journal.pone.0250023.g001

Collaborative projects are often designed by scientists but receive input and feedback from participants and legislators to achieve a conservation goal, such as maintaining water quality or protecting habitat for a threatened species [11,12]. Co-created projects occur when members of the public have a say in study design, data collection, data interpretation, and dissemination of results [9]. Co-created projects can help frame concerns of community members as scientific questions and can levy specialized local knowledge and engagement from community members. One such co-created project, "Gardenroots" involved measuring arsenic exposure routes in a community based near a superfund site in Arizona [13]. Involving members of the community in all aspects of the study aided the overall goal of risk communication to the community since members of the community were involved in the actual risk analysis [13]. Successful completion of projects that include a citizen science component is aided by careful and deliberate study design including project goals, scientific questions, plans for sustained participation, training, communication, and desired project outcomes [14].

Public and patient involvement (PPI) in health research is akin to citizen science applied specifically to the health and social care field. The most commonly used PPI definition is “research carried out ‘with’ or ‘by’ members of the public rather than ‘to’, ‘about’ or ‘for’ them” [15]. Both citizen science and PPI are most commonly practiced by public-facing disciplines. Expanding public involvement to laboratory-based and non-public facing disciplines is more challenging for all stakeholders [1618].

There has been increasing dialogue about public engagement in science. However, there is a risk of public engagement practices being siloed; never quite embedding broadly in fundamental practices, assumptions and research cultures [19]. If publicly engaged science is to have purchase, it must happen even in non-traditional disciplines as well as those scientific communities where the benefits are well-defined and direct [20]. Public engagement with science encompasses the spectrum of intentional, meaningful interactions that provide opportunities for mutual learning between scientists and members of the public. Whereas scientific outreach to the public is common in non-public facing disciplines (Fig 1); public involvement in science (Box 1), which entails collaborations between professional scientists and members of the public who are directly involved in a scientific research project, is far less common [21,22]. The provision of information (outreach) is a necessary aspect of publicly engaged research but is often emphasized at the expense of listening to, acknowledging and incorporating the interests of the public [23].

To embed public involvement practices into non-public facing disciplines will require support and cultivation by researchers themselves. In this study we use a survey of international researchers within the life sciences as a method of understanding the attitudes and challenges of public involvement and to identify the factors that promote public involvement in life science research.

Materials and methods

An ethics exemption was obtained from the Institutional Review Board of UCD Dublin. Participation in the survey was voluntary, no incentives were offered. This is an exploratory study that was not a priori registered. Informed consent was sought from all participants. Participants were informed of survey anonymity, contact details for the investigators and the purpose of the survey in advance of consenting. No IP addresses were collected. Participants who refused consent, were under the age of 18, or were not life scientists were removed from the dataset.

Survey development

The survey was developed as per Kelley et al (2003 [24]). Google forms was used as the survey platform. Only nominated members for the project team had access to response data. The usability of the questionnaire was tested by native (n = 3) and non-native (n = 3) English speaking life scientists. An open format survey was used, with participants self-declaring if they were life scientists. The survey consisted of seven sections:1. About the survey; 2. Consent (n = 3 questions); 3. Research Details (n = 4 questions); 4. Researcher Demographics (n = 6 questions); 5. Public Involvement Awareness and Experience (n = 10 questions); 6. Views on Public Involvement (n = 11 questions); and 7. Thanks and Follow-up. The survey was presented as one section per page. Sections 2–5 contained nominal questions whereas section five contained both nominal and unrestricted textbox questions. Only the questions on consent were mandatory. A responder progress bar was visible whilst completing the survey. Questions were not randomised. A single question on the age that parents finished education was used as a proxy for researcher socio-economic status (SES) during upbringing [25]. The survey was only available in the English language. The survey can be found in S1 Appendix.

Sampling and survey administration

Convenience sampling was used via electronic distribution of survey to members of the international working group of life scientists, the eLife Ambassadors for Good Practice in science (n = 206). Advertising occurred via a private online cloud-based team collaboration hub and via email, with a request for them to also share it within their scientific communities, societies within the life sciences, and through internal institutional communications. The survey was also advertised via a blog on the National Co-ordinating Centre For Public Engagement (UK) [18]. Data was collected between August-December 2018.

Quantitative analysis

Log files were checked for duplicate entries. Data was analysed in IBM SPSS v24. Data was analysed via binary regression analysis with public involvement experience (PIE) as the binary (yes/no) dependent variable. ‘No’PIE was used as the reference category.

Variable identification.

To determine the predictor variables in the model, a binary regression analysis was run with PIE as the binary dependent variable and demographic variables as covariates. Any variables with p<0.1 was carried forward in block 1 to the final model. The demographic variables were as follows: sex (categories: female/male/prefer not to say); age (in years); disability (coded as dichotomous yes/no); ethnic background (coded into 5 categories: 1. Asian (including Asian American & Asian (all other backgrounds), Asian European had a null response; 2. Black (including Black American, the categories Black African, Black (Arab) & Black (Any other background) were excluded as they had a null response); 3. Latino (including Latino American and Latino (Any Other Background); 4. White (including White European, White American, White Arab, White (Any other background); and 5. Other. Nationality, a free test answer, was coded by continent as dichotomous yes/no variables for European, North American, South American, Asian or Oceania. Those who answered countries spanning multiple continents (such as Turkey) were included in both. Africa was excluded as there were no answers from this continent. Socio-economic background (categories: up to age 15, 15–18, 18–23, >23). All covariates except for age were categorical.

A binary regression was run with PIE as the binary dependent variable and research-associated variables as covariates. Any variables with p<0.1 was carried forward in block 2 to the final model. Where do you conduct your primary research was recoded into dichotomous yes/no variables for lab (all lab types), field (all field types), community (any community including national/international) and government (including government facilities and public institutions (museums)). The region researchers are based in was coded into dichotomous variables by continent (Europe, North America, South America, Asian and Oceania). Career stage (categories: 1. Graduate & postgraduate, 2. Postdoctoral and research fellow, 3. Principal Investigator). Respondents could select up to four primary research disciplines. To capture this data for regression, categories were combined into larger groups (1. Cell & Molecular Biology including biochemistry, molecular biology, structural biology, biotechnology, cell biology, nanobiology; 2. Genetics including genetics and genomics, chromosomes and gene expression; 3. Neuroscience; 4. Biomedical and Health-associated including cancer biology, epidemiology and global health, human biology and medicine, immunology, infectious diseases, regenerative medicine, anatomy and physiology; 5. Ecology and conservation biology; 6. Botany and plant science; 7. Developmental and evolutionary science and 8. Computational and Theoretical Biology). These groups were coded dichotomously as yes/no. The final variable included was a dichotomous yes/no response to the question “Have you applied for funding or ethics where there was a specific question on public involvement in research?”.

Final model.

Data was analysed via logistic regression with public involvement experience (PIE) as the binary (yes/no) dependent variable. ‘No’PIE was used as the reference category. Demographic variables age and disability status (categorical) were included as covariates in block 1. Research associated variables community- based research, government-based research and “Have you applied for funding or ethics where there was a specific question on public involvement in research?” were included as categorical variables in block 2.

Qualitative analysis

Free text answers underwent qualitative textual analysis. Data were analysed using the pragmatist approach of inductive thematic analysis [26] combined with frequency analysis of themes, and managed using excel [27]. Two researchers (KK; ED, both of whom have experience in open science and qualitative analyses) analysed data independently, with a third independent researcher nominated to mediate potential discrepancies. We used the standard 6-stage analysis: 1. Becoming familiar with the data. 2. Generating initial codes: All relevant phrases, sentences, or text units that related to the research question were extracted into a data display table and given a code 3. Themes: Broad overarching themes were established based on code similarities. 4. Reviewing themes: independently derived themes were reviewed for internal homogeneity and external heterogeneity. 5. Defining and naming themes: the refined themes were discussed and the most apt name given to each identified theme. A quote from the data that captured a distinct aspect of each theme was identified. 6. Producing the report: A detailed account of each theme was composed. Stages 1–3 were performed independently by each researcher. Stages 4–6 were performed as a research team.

Results

There were 122 respondents, of which 117 individuals self-reported as life scientists. Further five respondents did not consent to the use of their data, and two were under the age of 18, resulting in a final dataset of n = 110 respondents. There was a high completeness rate of the survey. Sections on consent, research details, and public involvement awareness and experience had 100% completeness rates. Research demographics had completeness rates ranging from 93.6%-99.1%. The closed questions (n = 4) in the Views on Public Involvement section had 100% completion whereas the free text question had a completeness rate between 66.4%-75.6%. The dataset can be found at 10.6084/m9.figshare.12249671. Of the 110 respondents, n = 35 (31.8%) had public involvement experience and n = 75 did not (68.2%).

Research demographics

In the dataset of 110 researchers, respondents were overwhelmingly laboratory-based at a university or research institute (n = 100 (90.9%); Table 1). Respondents were based in research locations across five continents with no respondents from research institutes in Africa. Most respondents were based in North America or Europe (n = 47 (42.8% of all responses) and n = 34 (30.9%) respectively) followed by Central and South America (n = 16, 14.4%) and Oceania (n = 4, 3.6%).

Characteristics associated with public involvement experience

Five variables were brought forward for inclusion in the final model: age, disability, community- based research setting, government-based research setting and “Have you applied for funding or ethics where there was a specific question on public involvement in research?”. When these variables were included in the final model (Model Chi Squared statistic 30.21, p<0.001), only answering “Yes” to applying for funding or ethics where there was a specific question on public involvement in research remained associated with PIE (p<0.001, OR 6.36 (95% CI 2.31–17.51), Table 2).

thumbnail
Table 2. Binary logistic regression model for characteristics associated with PIE.

https://doi.org/10.1371/journal.pone.0250023.t002

Researcher-perceived barriers to public involvement

As per Maccarthy et al (2018), challenges surrounding public involvement were asked in the context of institutional barriers, personal worries and research concerns [17]. There were 81 responses to the question on institutional barriers [In your view, what are the institutional barriers to including public involvement in your research?]. The well documented challenges of regulations, resources and funding were major themes emerging from the data [7,17,28,29]. The other major theme was that of career benefit.

Not valued as academic currency for TT*/jobs.”

  • Postdoctoral researcher in cell biology, based in North America (*TT: Tenure Track)

We had 79 responses to the question on personal barriers to involvement, of which n = 9 (11.4%) indicated they did not have any personal barriers. Within the remaining responses to the question [In your view, what are the personal barriers (worries) that would prevent you including public involvement in your research?] two main, and highly related, themes emerged: fear of misrepresentation and personal communication skills.

When asked expressly about the concerns of public involvement to their research [In your view, what are the research concerns that would prevent you including public involvement in your research?]; of the 70 respondents, 15 (21.4%) said they had no concerns about its impact on their research. Of the remaining respondents: misrepresentation, workload, effect on research focus and relevance of public involvement for their research emerged as concerns with approximately equal frequency.

The main barrier is time—being involved in public outreach takes precious time away from research activities, getting grants and publications.”

  • Principal Investigator, cancer and cell biology, Europe.

Time is a precious resource in the life sciences and burnout is common [3033]. Even when researchers acknowledged the potential benefit of public involvement to their research, there was concern that there were not sufficient resources in place such that the benefit to their research would not outweigh the cost in terms of time.

Discussion

Science can be a public good and a social enterprise. In addition to inordinate technological development, the rapid growth in life sciences research and knowledge has occurred due to a long-standing public investment in life science research [3437]. The fact that much research is not public-facing does not mean that it is not publicly relevant. In order to continue sustained investment into life science research, we must build more comprehensive, interactive and mutually beneficial opportunities for dialogue and exchange with our largest funders- the public [38]. Openness in science does not solely relate to article processing fees [39]. Responsibility and openness in science also relates to increasing the accessibility and use of scientific knowledge. To this end, responsible science should include appropriate dialogue with the community upon which it has an impact; or which it may be perceived to impact [20,40].

Here we demonstrate that a top-down approach encourages public involvement in life sciences research. Our analyses found that the strongest correlate to actual experience of public involvement in research was applying to institutional bodies that specifically asked researchers about their public involvement activities. Funders and institutions providing the mandate and resources to support public involvement is paramount to foster public involvement. Simply feeling positively towards involving the public in research was not sufficient for researchers to implement it. Researchers who considered public involvement to be beneficial to the researcher themselves or to their research, rather than just to the general public, were more likely to engage in it. This suggests that provision of greater context or immersive experience of public involvement in the life sciences would allow researchers to understand and conceptualise the potential benefit of public involvement to their own research.

Previous findings from public engagement and public communications have demonstrated the likelihood of having involved the pubic in research increased with age. Although age was a significant factor in the initial model where only demographic variables were included, age was not significant in our final model that also included research-associated variables (p = 0.054) [41,42]. Our findings therefore diverge with the literature demonstrating that both scientific engagement and active dissemination increase with age and experience [41,42]. It should be noted, however, that those studies focused predominantly on outreach activities, and were centred on researchers in a single university [42] or a single, albeit large, national research organisation [41]. This, in combination with the fact that those studies were conducted pre the social media age, may explain this divergence with our findings.

Our qualitative analysis identified that the career pathway, including funding pressures and the promotion system of research institutions has a key role to play in the pragmatic ability of early-career researchers to truly consider public involvement. This highlights that simply providing training and resources for public involvement, without implementing changes in the science career pathways, may have limited benefit at an early career stage [43].

“Fear of being misrepresented or being "less serious"”

  • Postgraduate conservation biologist, South America

Involving the public leaves the researcher and their research vulnerable to misrepresentation and being negatively viewed by your peers and colleagues. This finding is consistent with Poliakoff and Webb [42] who found that colleagues participating had a positive influence on a researcher’s intent to engage the public. However, other studies of scientists’ willingness to engage found descriptive norms to be relatively limited predictors of willingness to engage[44,45].

Public involvement is demanding both in terms of time and resources. The academic career structure and the increasingly high bar to earn job stability or tenure as an academic researcher is a major barrier to public involvement [46,47].

“The extremely competitive science system preclude me to really consider it”

  • Research Fellow in biochemistry, based in Europe.

Institutions need to consider not only the funding and resources available for public involvement, but the environment necessary to promote it. The continued focus on academic productivity, at the expense of academic activities that grow or improve research, will continue to restrict researchers in their innovative potential and growth as pragmatism and limited time and energy necessitates focus on tenure track and promotion variables. The attitude of “waiting for tenure to do what you actually love” has become pervasive in the life sciences. However, many excellent and innovative scientists are lost to science during the long road to tenure [36,48,49]. Engaging stakeholders is beneficial to science and to the translation and use of scientific knowledge [2,6,50] and institutions need to consider the life sciences system as a whole in their efforts to encourage stakeholder engagement and knowledge transfer [36].

Researchers who had involved the public in their research were more knowledgeable about the local and electronic resources available to them and more likely to have engaged in training for public involvement. These resources seem helpful downstream, once public involvement has been initiated. Science communication is a critical skill for public involvement. Without some grasp of the relevant science, the public cannot be expected to make informed decisions about research [51]. In the absence of a pre-existing relationship, how can a researcher understand the public’s information needs, and make it accessible in a format useful for public involvement? Can we expect laboratory-based life scientists to also be skilled in bridging science and decision making? There are frameworks available to assist in science communication but the evidence-base for their effectiveness in the life sciences is understudied [5254].

“Once misinformation has been disseminated, it seems very difficult to rectify in the current information sharing climate”

  • Postdoc in Neuroscience, based in Europe

Science has become increasingly politicised in the public sphere [55]. Political orientations and ideologies can shape public trust in science [55,56]. Fear of misrepresentation of their research, either intentional or otherwise, was a major theme identified in our data.

“There is always a fear that your words might be taken out of context to push an agenda that you don’t agree with. Also, putting yourself out there may make you vulnerable to verbal or even physical attacks on you”

  • Principal Investigator in Biochemistry, based in Europe

Complicating the issue is the reach of misinformation. Once misinformation has entered the public sphere, rebuttal with corrections to the misinformation tend to be muted compared with the propagation of the misinformation, leading to false and misrepresented information remaining within the pool of common knowledge [57]. Our data shows this has made researchers wary of opening their in-progress research to the public, even when they recognise there are potential benefits to their research of doing so.

Politicized environments often induce suspicions about science communicators’ true motives or expertise and questions may arise as to whether a scientist can really be trusted [58]. There is a fear among scientists about the potential personal backlash stemming from public involvement. This links to the theme of career benefit.

Researcher vulnerability is an important issue often overlooked in public involvement. For institutions wishing to encourage increased public involvement they may need to consider what protections are in place for researchers exposing their early stage and in-development research to the public sphere. The time required to deliberate study design and relevance, plan and relationship build for sustained involvement, provide and undertake training, and improve communication in the current research environment is not considered feasible within their research and/or institutional environment for many researchers and is an area in need of further research into the best ways to overcome these challenges. Misrepresentation (intentional or otherwise) coupled with the ecosystem of rapid spread of false information on public platforms, and the fear thereof, is a complex issue [59]. It is made more complex in the context of non-public interfacing research, which does not have a high level of public understanding in the first place [6062]. To truly integrate public involvement in a mutually beneficial way in non-public-facing disciplines requires deeper understanding and improved holistic support for this already over-burdened researcher demographic [36].

This study highlights the challenges faced in compelling researchers, particularly early-career laboratory science researchers, to engage directly with the public by involving them in their research projects. Despite the challenges highlighted here, citizen science efforts have been growing in terms of number of projects [63], outcomes in terms of publications [63], and the number or resources available to researchers and members of the public alike including open source, peer reviewed journals on citizen science and public and patient involvement [6467]. Our results emphasize the need for integration and incentivization of the use of these resources by institutions and funding agencies to maximize researcher’s interests in public involvement [43]. Effective change towards a more open and inclusive model of life sciences will require more than just training and funding. Policy makers and institutions can greatly influence the implementation of public involvement in research through the working conditions and environment they promote.

Limitations

The sample size of 110 researchers is small, limiting the capacity to conduct more in-depth statistical associations. The high rate of consensus on the beneficial nature of public involvement indicates selection bias in the respondents toward those who view public involvement positively. The survey was only available in English, limiting the potential respondents to those proficient in written English. We are unable to estimate a response rate due to the method of survey advertising. As such, caution must be used in interpretation of the results.

Conclusions

Fundamentally, a researcher’s decision to involve the public will be decided by whether they believe it has sufficient potential value to both themselves and their research as to warrant the investment in terms of time, energy and potential or perceived career consequences. Policy makers and institutions can greatly influence this decision by creating an environment supportive of responsible research practices, including public involvement. Creating this environment in many cases will require a paradigm shift if public involvement is to become more than a marginal curiosity or tokenistic effort in laboratory-based research [23].

Supporting information

S4 Table. Ethnic/Cultural background frequency data.

https://doi.org/10.1371/journal.pone.0250023.s004

(DOCX)

S7 Table. Respondents who have been asked a specific question on public involvement on an ethics or funding application frequency data.

https://doi.org/10.1371/journal.pone.0250023.s007

(DOCX)

S1 Appendix. Public involvement in research questionnaire.

https://doi.org/10.1371/journal.pone.0250023.s008

(XLSX)

Acknowledgments

This study was conducted as an initiative of the eLife Ambassadors for Good Practice in Science; we would like to acknowledge the help and support of the eLife Ambassadors and community members. In particular Dr. Erin Wissink, Dr. Hannah Wang and Dr. Vinodh Ilangovan.

References

  1. 1. Goodman MS, Sanders Thompson VL. The science of stakeholder engagement in research: classification, implementation, and evaluation. Translational behavioral medicine. 2017;7(3):486–91. Epub 2017/04/10. pmid:28397159.
  2. 2. Crocker JC, Ricci-Cabello I, Parker A, Hirst JA, Chant A, Petit-Zeman S, et al. Impact of patient and public involvement on enrolment and retention in clinical trials: systematic review and meta-analysis. BMJ. 2018;363:k4738. pmid:30487232
  3. 3. Wilkins CH, Spofford M, Williams N, McKeever C, Allen S, Brown J, et al. Community representatives’ involvement in Clinical and Translational Science Awardee activities. Clinical and translational science. 2013;6(4):292–6. Epub 2013/06/10. pmid:23919364.
  4. 4. Yarborough M, Edwards K, Espinoza P, Geller G, Sarwal A, Sharp R, et al. Relationships hold the key to trustworthy and productive translational science: recommendations for expanding community engagement in biomedical research. Clinical and translational science. 2013;6(4):310–3. Epub 2013/01/14. pmid:23919367.
  5. 5. Kost RG, Reider C, Stephens J, Schuff KG, Clinical, Translational Science Award Research Subject Advocacy Survey T. Research subject advocacy: program implementation and evaluation at clinical and translational science award centers. Academic medicine: journal of the Association of American Medical Colleges. 2012;87(9):1228–36. pmid:22836849.
  6. 6. Boaz A, Hanney S, Borst R, O’Shea A, Kok M. How to engage stakeholders in research: design principles to support improvement. Health Research Policy and Systems. 2018;16(1):60. pmid:29996848
  7. 7. Ryan SF, Adamson NL, Aktipis A, Andersen LK, Austin R, Barnes L, et al. The role of citizen science in addressing grand challenges in food and agriculture research. Proceedings of the Royal Society B: Biological Sciences. 2018;285(1891):20181977. pmid:30464064
  8. 8. Peters MA, Besley T. Citizen science and post-normal science in a post-truth era: Democratising knowledge; socialising responsibility. Educational Philosophy and Theory. 2019;51(13):1293–303.
  9. 9. Shirk J, Ballard H, Wilderman C, Phillips T, Wiggins A, Jordan R, et al. Public participation in scientific research: a framework for deliberate design. Ecology and society. 2012;17(2).
  10. 10. Engagement NCCfP. Who are the public? 2019 [cited 2019]. https://www.publicengagement.ac.uk/about-engagement/who-are-public.
  11. 11. Deutsch W, Busby A, Orprecio J, Labis J, Cequina E. Community-based water quality monitoring: from data collection to sustainable management of water resources. Los Baños, Philippines: Philippine Council for Agriculture, Forestry and …; 2001.
  12. 12. Low B, Sundaresan SR, Fischhoff IR, Rubenstein DI. Partnering with local communities to identify conservation priorities for endangered Grevy’s zebra. Biological Conservation. 2009;142(7):1548–55.
  13. 13. Ramirez-Andreotta MD, Brusseau ML, Artiola J, Maier RM, Gandolfi AJ. Building a co-created citizen science program with gardeners neighboring a Superfund site: The Gardenroots case study. International public health journal. 2015;7(1). pmid:25954473
  14. 14. Chandler M, Rullman S, Cousins J, Esmail N, Begin E, Venicx G, et al. Contributions to publications and management plans from 7 years of citizen science: use of a novel evaluation tool on Earthwatch-supported projects. Biological Conservation. 2017;208:163–73.
  15. 15. INVOLVE. What is public involvement in research UK: INVOLVE (NHS); [cited 2018 08 August 2018]. http://www.invo.org.uk/find-out-more/what-is-public-involvement-in-research-2/.
  16. 16. Dobbs TW, Iain. The BMJ Opinion [Internet]. UK: BMJ. 2016. https://blogs.bmj.com/bmj/2016/05/11/ppi-in-basic-science-research-are-we-doing-enough/.
  17. 17. Maccarthy J, Guerin S, Wilson AG, Dorris E. Facilitating Patient and Public Involvement in basic and preclinical health research. bioRxiv. 2018:425371.
  18. 18. Dorris ER. Blog of the NCCPE [Internet]. UK: National Co-ordinating centre for Public Engagement. 2018 23/11/2018. https://www.publicengagement.ac.uk/whats-new/blog/involve-public-your-research-your-research-will-thank-you-it.
  19. 19. Wynne B, Stilgoe J. The Public Value of Science: Or How to Ensure that Science Really Matters: Demos; 2005.
  20. 20. Stilgoe J, Lock SJ, Wilsdon J. Why should we promote public engagement with science? Public understanding of science (Bristol, England). 2014;23(1):4–15. Epub 2014/01/16. pmid:24434705.
  21. 21. Haywood BK, Besley JC. Education, outreach, and inclusive engagement: Towards integrated indicators of successful program outcomes in participatory science. Public understanding of science. 2014;23(1):92–106. pmid:23887249
  22. 22. Golumbic YN, Baram-Tsabari A, Koichu B. Engagement and Communication Features of Scientifically Successful Citizen Science Projects. Environmental Communication. 2020;14(4):465–80.
  23. 23. Lavery JV. Building an evidence base for stakeholder engagement. Science. 2018;361(6402):554. pmid:30093590
  24. 24. Kelley K, Clark B, Brown V, Sitzia J. Good practice in the conduct and reporting of survey research. International Journal for Quality in Health Care. 2003;15(3):261–6. pmid:12803354
  25. 25. Thomson S. Achievement at school and socioeconomic background—an educational perspective. npj Science of Learning. 2018;3(1):5. pmid:30631466
  26. 26. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006;3(2):77–101.
  27. 27. Bree RT G Gerry. Using Microsoft Excel to code and thematically analyse qualitative data: a simple, cost-effective approach. The All Ireland Journal of Teaching and Learning in Higher Education (AISHE-J). 2016;8(2):2811.
  28. 28. Staniszewska S, Denegri S. Patient and public involvement in research: future challenges. Evidence Based Nursing. 2013;16(3):69. pmid:23749800
  29. 29. Boivin A, Lehoux P, Lacombe R, Burgers J, Grol R. Involving patients in setting priorities for healthcare improvement: a cluster randomized trial. Implementation science: IS. 2014;9:24-. pmid:24555508.
  30. 30. Holleman W, Gritz ER. Biomedical burnout. Nature. 2013;500(7464):613–4.
  31. 31. Holleman WL, Cofta-Woerpel LM, Gritz ER. Stress and Morale of Academic Biomedical Scientists. Academic Medicine. 2015;90(5):562–4. pmid:25340366
  32. 32. Messias E, Gathright MM, Freeman ES, Flynn V, Atkinson T, Thrush CR, et al. Differences in burnout prevalence between clinical professionals and biomedical scientists in an academic medical centre: a cross-sectional survey. BMJ Open. 2019;9(2):e023506. pmid:30782882
  33. 33. Powell K. Work–life balance: Break or burn out. Nature. 2017;545(7654):375–7.
  34. 34. Technology NRCUCoTiSa. The Pace of Developments in the Life Sciences. National Research Council (US) Committee on Trends in Science and Technology Relevant to the Biological Weapons Convention: An International Workshop. US: National Academies Press; 2011.
  35. 35. Committee on Dimensions C, Implications of Recent Trends in the Careers of Life S. Trends in the early careers of life scientists. Preface and Executive Summary. Molecular biology of the cell. 1998;9(11):3007–15. pmid:9802892.
  36. 36. Alberts B, Kirschner MW, Tilghman S, Varmus H. Rescuing US biomedical research from its systemic flaws. Proceedings of the National Academy of Sciences. 2014;111(16):5773. pmid:24733905
  37. 37. Kelly T, Marians K. Rescuing US biomedical research: Some comments on Alberts, Kirschner, Tilghman, and Varmus. Proceedings of the National Academy of Sciences. 2014;111(26):E2632. pmid:24920595
  38. 38. Science AAftAo. Why public engagement matters: AAAS. https://www.aaas.org/resources/communication-toolkit/what-public-engagement.
  39. 39. Levin N, Leonelli S, Weckowska D, Castle D, Dupré J. How Do Scientists Define Openness? Exploring the Relationship Between Open Science Policies and Research Practice. Bulletin of science, technology & society. 2016;36(2):128–41. Epub 2016/06/01. pmid:27807390.
  40. 40. Barben D, Fisher E, Selin C, Guston DH, Hackett EJ, Amsterdamska O, et al. The handbook of science and technology studies. MIT Press Cambridge, MA; 2008.
  41. 41. Jensen P, Rouquier J-B, Kreimer P, Croissant Y. Scientists who engage with society perform better academically. Science and Public Policy. 2008;35(7):527–41.
  42. 42. Poliakoff E, Webb TL. What factors predict scientists’ intentions to participate in public engagement of science activities? Science Communication. 2007;29(2):242–63.
  43. 43. Alperin JP, Muñoz Nieves C, Schimanski LA, Fischman GE, Niles MT, McKiernan EC. How significant are the public dimensions of faculty work in review, promotion and tenure documents? eLife. 2019;8:e42254. pmid:30747708
  44. 44. Besley JC, Dudo A, Yuan S, Lawrence F. Understanding Scientists’ Willingness to Engage. Science Communication. 2018;40(5):559–90.
  45. 45. Besley JC. What do scientists think about the public and does it matter to their online engagement? Science and Public Policy. 2014;42(2):201–14.
  46. 46. Moher D, Naudet F, Cristea IA, Miedema F, Ioannidis JPA, Goodman SN. Assessing scientists for hiring, promotion, and tenure. PLoS biology. 2018;16(3):e2004089–e. pmid:29596415.
  47. 47. Gewin V. Tactical tenure manoeuvres. Nature. 2014:421–3. pmid:24745066
  48. 48. Lee O. I have one of the best jobs in academia. Here’s why I’m walking away. Vox. 2015 08 Sept 2015.
  49. 49. Ruben A. Another tenure-track scientist bites the dust. Science. 2017 19 July 2017.
  50. 50. Bowen DJ, Hyams T, Goodman M, West KM, Harris-Wai J, Yu JH. Systematic Review of Quantitative Measures of Stakeholder Engagement. Clinical and translational science. 2017;10(5):314–36. Epub 2017/05/29. pmid:28556620.
  51. 51. Fischhoff B. The sciences of science communication. Proc Natl Acad Sci U S A. 2013;110 Suppl 3(Suppl 3):14033–9. Epub 2013/08/13. pmid:23942125.
  52. 52. Edwards W, Miles RF Jr, Von Winterfeldt D. Advances in decision analysis: from foundations to applications: Cambridge University Press; 2007.
  53. 53. von Winterfeldt D. Bridging the gap between science and decision making. Proc Natl Acad Sci U S A. 2013;110 Suppl 3(Suppl 3):14055–61. Epub 2013/08/12. pmid:23940310.
  54. 54. Boivin A, L’Espérance A, Gauvin F-P, Dumez V, Macaulay AC, Lehoux P, et al. Patient and public engagement in research and health system decision making: A systematic review of evaluation tools. Health expectations: an international journal of public participation in health care and health policy. 2018;21(6):1075–84. Epub 2018/07/30. pmid:30062858.
  55. 55. Gauchat G. Politicization of Science in the Public Sphere: A Study of Public Trust in the United States, 1974 to 2010. American Sociological Review. 2012;77(2):167–87.
  56. 56. Schmid-Petri H. Politicization of science: how climate change skeptics use experts and scientific evidence in their online communication. Climatic Change. 2017;145(3):523–37.
  57. 57. Starbird K, Maddock J, Orand M, Achterman P, Mason RM. Rumors, false flags, and digital vigilantes: Misinformation on twitter after the 2013 boston marathon bombing. IConference 2014 Proceedings. 2014.
  58. 58. Lupia A. Communicating science in politicized environments. Proc Natl Acad Sci U S A. 2013;110 Suppl 3(Suppl 3):14048–54. Epub 2013/08/12. pmid:23940336.
  59. 59. Pennycook G, Rand DG. Fighting misinformation on social media using crowdsourced judgments of news source quality. Proceedings of the National Academy of Sciences. 2019;116(7):2521. pmid:30692252
  60. 60. National Academies of Sciences E, Medicine. Communicating science effectively: A research agenda: National Academies Press; 2017.
  61. 61. Scheufele DA, Krause NM. Science audiences, misinformation, and fake news. Proceedings of the National Academy of Sciences. 2019:201805871. pmid:30642953
  62. 62. National Academies of Sciences E, Medicine. Science literacy: Concepts, contexts, and consequences: National academies press; 2016.
  63. 63. Follett R, Strezov V. An analysis of citizen science based research: usage and publication patterns. PloS one. 2015;10(11):e0143687. pmid:26600041
  64. 64. Hoffman C, Cooper CB, Kennedy EB, Farooque M, Cavalier D. Scistarter 2.0: A digital platform to foster and study sustained engagement in citizen science. Analyzing the Role of Citizen Science in Modern Research: IGI Global; 2017. p. 50–61.
  65. 65. Wang Y, Kaplan N, Newman G, Scarpino R. CitSci. org: A new model for managing, documenting, and sharing citizen science data. PLoS biology. 2015;13(10):e1002280. pmid:26492521
  66. 66. London IC. PPI Resource Hub. 2018.
  67. 67. Ambassadors e. ECR Central: Public Involvement in Research Resources.