Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Implementing evidence ecosystems in the public health service: Development of a framework for designing tailored training programs

  • Laura Arnold ,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    arnold@akademie-oegw.de

    Affiliations Academy of Public Health Services, Duesseldorf, Germany, Department of International Health, Care and Public Health Research Institute—CAPHRI, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands

    ⨯
  • Simon Bimczok,

    Roles Formal analysis, Investigation, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Academy of Public Health Services, Duesseldorf, Germany

    ⨯
  • Timo Clemens,

    Roles Supervision, Writing – review & editing

    Affiliation Department of International Health, Care and Public Health Research Institute—CAPHRI, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands

    ⨯
  • Helmut Brand,

    Roles Supervision, Writing – review & editing

    Affiliation Department of International Health, Care and Public Health Research Institute—CAPHRI, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands

    ⨯
  • Dagmar Starke,

    Roles Funding acquisition, Supervision, Writing – review & editing

    Affiliation Academy of Public Health Services, Duesseldorf, Germany

    ⨯
  • on behalf of the EvidenzÖGD study consortium

    Membership of the EvidenzÖGD study consortium is provided in the acknowledgments.

    ⨯

Abstract

The COVID-19 pandemic has highlighted the importance of local evidence ecosystems in which academia and practice in the Public Health Service (PHS) are interconnected. However, appropriate organizational structures and well-trained staff are lacking and evidence use in local public health decision-making has to be integrated into training programs in Germany. To address this issue, we developed a framework incorporating a toolbox to conceptualize training programs designed to qualify public health professionals for working at the interface between academia and practice. We conducted a scoping review of training programs, key-informant interviews with public health experts, and a multi-professional stakeholder workshop and triangulated their output. The resulting toolbox consists of four core elements, encompassing 15 parameters: (1) content-related aspects, (2) context-related aspects, (3) aspects relevant for determining the training format, and (4) aspects relevant for consolidation and further development. Guiding questions with examples supports the application of the toolbox. Additionally, we introduced a how-to-use guidance to streamline the creation of new training programs, fostering knowledge transfer at the academia-practice interface, equipping public health researchers and practitioners with relevant skills for needs-based PHS research. By promoting collaborative training development across institutions, our approach encourages cross-institutional cooperation, enhances evidence utilization, and enables efficient resource allocation. This collaborative effort in developing training programs within local evidence ecosystems not only strengthens the scientific and practical impact but also lays a foundation for implementing complex public health measures effectively at the local level.

Citation: Arnold L, Bimczok S, Clemens T, Brand H, Starke D, on behalf of the EvidenzÖGD study consortium (2024) Implementing evidence ecosystems in the public health service: Development of a framework for designing tailored training programs. PLoS ONE 19(4): e0292192. https://doi.org/10.1371/journal.pone.0292192

Editor: Kuo-Cherh Huang, Taipei Medical University, TAIWAN

Received: October 21, 2023; Accepted: March 18, 2024; Published: April 18, 2024

Copyright: © 2024 Arnold et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are included in the paper and in the supporting information files. To ensure data protection, excerpts from the qualitative interviews are provided anonymously

Funding: The project is part of the funding priority "Strengthening Collaboration between Public Health Services and Public Health Research" of the Federal Ministry of Health and was funded by the federal government Germany.

Competing interests: The authors have declared that no competing interests exist.

1. Introduction

The systematic incorporation of research evidence into health policy is crucial for the provision of high-quality public health services. Public health professionals are increasingly required to engage in evidence-informed decision-making (EIDM) [1], which aims to identify the most effective and cost-efficient interventions, while also minimizing harm, assessing adverse consequences, and enhancing health outcomes for individuals and communities [2, 3]. Accomplishing this requires that public health policy and practice are informed by the best available evidence, as well as a comprehensive consideration of normative and contextual factors, including political and societal preferences, setting-specific considerations, feasibility, affordability, and sustainability [4–6]. Proper implementation and execution of EIDM has the potential not only to enhance the effectiveness of health policies and public health interventions but also to promote transparency, acceptability, and accountability [7, 8].

A system that encompasses the formal and informal connections and interactions among various stakeholders involved in the production, implementation, and utilization of evidence can be referred to as evidence ecosystem [9]. The formulation and adoption of effective policies and programs relies on the presence of a functional evidence ecosystem encompassing diverse actors from various domains and with diverse agendas [10]. Within such a system, two distinct but interconnected conceptual domains must be considered: evidence generation and evidence utilization, usually linked either to policy or practice [11, 12]. The effectiveness of such an ecosystem relies on robust relationships and active collaboration between public health academia (e.g., universities, research institutes, or academies of public health services) and public health practice (e.g., public health service on federal, state, or local level) [13]. To date, the responsibility of generating evidence has predominantly been ascribed to academic institutions, whereas the utilization of evidence, encompassing the identification, processing, and contextual adaptation of scientific knowledge, has commonly been linked to the domain of public health practice. However, meeting the diverse needs of individuals and communities necessitates a close intertwining of evidence generation and evidence utilization, rather than maintaining a strict separation of responsibilities and competencies. This calls for a symbiotic relationship between academia and practice, enabling the harmonization of scientific inquiry with prevailing practical needs and imperatives [10, 14]. Institutionally anchored, this in turn facilitates informed decision-making in the implementation of evidence-based public health interventions [15, 16].

However, the COVID-19 pandemic highlighted considerable disparities between the envisioned integration of EIDM within a comprehensive evidence ecosystem and the practical implementation of evidence transfer between academia and practice in the German Public Health Service (PHS) [17–19]. In Germany, the health system governance follows a decentralized approach, with responsibilities divided between the federal and state levels, along with corporatist bodies of self-governance [20]. The implementation of federal legislation pertaining to PHS, lies within the purview of the 16 state governments, who wield legislative powers and provide functional and disciplinary oversight of the PHS on local level [21]. Consequently, the local health authorities (LHA) operating at the level of municipalities and independent cities assume a pivotal role in addressing wide range of public health tasks [22]. These LHAs are key actors in promoting and protecting population health and well-being, encompassing health surveillance, evidence gathering, and providing guidance on the prevention of communicable and non-communicable diseases.

Ensuring effective resource utilization and successful intervention implementation requires not only access to scientific knowledge but also its judicious application [6], including the thoughtful integration of evidence into practice, consideration of the specific context, ongoing evaluation and adaptation of measures and strategies, and continuous stakeholder engagement to ensure ethical, effective, and sustainable outcomes. A judicious approach has the potential to provide decision-makers with better access to information on effective strategies, thereby, optimizing the allocation of limited resources and increasing confidence in the successful implementation of different intervention options [23]. Especially during periods of financial constraints, considering carefully the intricacies of practical implementation within public health decision-making processes safeguards the well-being of the population. This is particularly relevant for the municipal level, as LHAs possess an encompassing understanding of local conditions and contexts predestining them to ensure EIDM processes at the local level [24–26].

Nevertheless, the COVID-19 pandemic has exposed significant challenges in LHAs’ implementation and embedding of EIDM processes. These challenges include among others, limited access to scientific information [27, 28], a lack of institutionally anchored collaboration between academia and practice in many places [29, 30], and substantial personnel, material and time constraints that make accessing, reviewing, and applying research findings into local contexts much more difficult [28, 31].

Overcoming these barriers necessitates the establishment of institutionalized evidence ecosystems that foster large-scale collaboration between public health practice and academia and enable inter-agency collaboration in line with the Health in All Policies (HiAP) approach [32, 33]. At the community level, this entails the presence of a well-qualified public health workforce equipped with scientific expertise, administrative skills, and methodological knowledge to ensure a two-way theory-practice transfer.

In Germany, several established training programs provide high-quality education and training opportunities for public health professionals. These programs include, in addition to various public health chairs that primarily prepare students for scientific activities, in particular the Academies of Public Health Services that offer advanced trainings and practice-oriented qualifications. However, to implement and enhance EIDM processes at the academia-practice interface, both comprehensive scientific competencies coupled with extensive expertise in local administrative tasks, activities and responsibilities are needed. This interface encompasses activities aimed at enhancing knowledge transfer between the municipal PHS (e.g., LHA) and research institutions (e.g., universities or academies), as well as fostering collaboration. For brevity, we will refer to this domain as "work at the interface" in the following. Effective dissemination and exchange of evidence at the community level requires a profound understanding of public health services. Consequently, public health professionals working at the interface require competencies in evidence-based policy advice and design, coordination and management skills, and a comprehensive understanding of organizational leadership and decision-making structures [34, 35]. Acquiring these competencies necessitates a consolidation of theory and practice, which is often facilitated through postgraduate training approaches. Noteworthy examples of postgraduate training programs in Germany include the medical residency program for public health specialists [36], the trainee program for public health specialists at LHA Fulda [37], or the postgraduate training in applied epidemiology at the Robert Koch Institute [38]. However, these existing approaches predominantly focus on one occupational group or prepare for single areas of activity. To address this gap, developing training programs tailored to the complex regional characteristics of PHS at the local level is imperative.

The overarching aim of this study was to develop a framework incorporating a toolbox for conceptualizing tailored training programs to equip public health professionals with the necessary skills to enhance evidence-informed approaches at the interface. The study followed a stepwise approach with the following research objectives:

  1. Explore the relevant fields of activity, tasks, required skills, and competencies for a position at the interface through several semi-structured key informant interviews (Fig 1: RQ-1/2).
  2. Conduct a scoping review to identify, characterize, and analyze available training programs that prepare public health professionals for working at the interface (Fig 1: RQ-3).
  3. Develop a toolbox that encompasses key parameters for creating tailored training programs qualifying individuals to facilitate evidence transfer at the local level, drawing from the findings of steps 1 and 2.
thumbnail
Download:
Fig 1. Schematic of the methodological approach to develop a framework for conceptualizing tailored training programs for working at the interface (RQ = research question).

https://doi.org/10.1371/journal.pone.0292192.g001

3. Materials and methods

A stepwise iterative mixed-methods approach was employed, marked by a systematic and cyclical approach in data collection, analysis, and synthesis. This methodology was instrumental in ensuring each phase built upon and refined insights from previous phases for a comprehensive understanding of the topic. Firstly, qualitative interviews were conducted to explore the tasks, fields of activities, skills and competencies relevant to working at the interface. Secondly, a scoping review was carried out to identify suitable training programs. In the third step, a stakeholder workshop was executed, consisting of the critical appraisal and prioritization of the main findings from the previous steps. The final phase, triangulation and synthesis, involved a core team of the project consortium reflecting on and synthesizing the results from all previous steps. This iterative process of reflection, appraisal, and synthesis culminated in the development of a comprehensive toolbox that encompasses both essential content and contextual factors required for the development of tailored training programs.

In our study, the terms "framework" and "toolbox" are closely interlinked: The “framework” refers to the overarching theoretical construct guiding the development of tailored training programs. In contrast, the “toolbox” consists of a comprehensive collection of essential elements and parameters, identified during our research. It provides researchers and practitioners with customized guiding questions to identify and discuss all components relevant to their field. Together with its accompanying short application guide, the toolbox is a pivotal aspect of the framework, integral to its function and effectiveness.

The methodological approach is displayed in Fig 1. Details on the activities in each step are described below.

3.1 Qualitative interview study

In the first step, 23 semi-structured key informant interviews (KII) were conducted to explore aspects related to public health research, knowledge transfer, collaboration between academia and practice in the PHS, and training programs. Participants were purposefully selected using a sampling plan that utilized professional networks and snowballing. Key informants were carefully chosen based on their expertise in public health academia and practical experience in the PHS. We aimed to achieve a diverse representation, considering a variety of professional backgrounds and levels of experience. Recruitment of study participants took place from December 17, 2021 to March 31, 2022. The first interview was conducted on January 12, 2022, the last on May 03, 2022. During recruitment, all participants were informed about the study procedure and objectives. Written informed consent was obtained from all participants. The consent forms were filed in accordance with German data protection regulations. Prior to recording the interview, all participants were informed about the procedure and verbally asked if they agreed to be recorded. To incorporate the heterogeneity of the PHS accordingly and to reach saturation, the sampling plan considered two dimensions: representation of experts from federal, state, and local governments, and inclusion of experts from public health practice, public health academia, experts with teaching background, and young professionals. A semi-structured interview guide was developed based on existing literature and through brainstorming and iteratively refined by a group of five researchers (LA, DD, SB, SG, and SW) who are all part of the project consortium. After incorporating minor adjustments from the pilot test, the final guide was used for virtual video call interviews. In most cases, participants were invited by e-mail or telephone.

All interviews were conducted by a pair of researchers and the resulting audio files were content-semantic transcribed [39], pseudonymized, and afterwards deleted (LA, DD, SB, SG, and SW). If requested, the transcripts were returned to the participants for correction or comments. No repeat interviews were conducted. The transcripts underwent a deductive-inductive qualitative content analysis following Mayring’s approach [40, 41]. After calibration of the coding frame within a group of five researchers from the project consortium, all interviews were coded by two researchers independently using MAXQDA Analytics Pro 2022 (VERBI Software GmbH, Berlin) (LA, DD, SB, SG). Inductive additions to the coding frame were made as required. Any discrepancies were resolved through discussion. The final coding tree can be found together with anonymized example passages in the appendix (S1 Table).

Relevant tasks, activities, skills, and competencies associated with working at the interface and facilitating evidence transfer at the local level were identified and extracted (RQ-1 and RQ-2, Fig 1). The insights obtained from the interviews were utilized to determine priority areas of activity at the interface. Furthermore, these results informed the subsequent development of the toolbox.

3.2 Scoping review of relevant training programs

The second step involved a scoping review aiming to identify, characterize, and analyze available training programs that prepare public health professionals for working at the interface. This methodological approach was chosen due to its comprehensive format, allowing for efficient mapping the existing literature within a limited timeframe while capturing the scope and characteristics of current research activity. We did not attempt to identify all records on training programs at the interface, but enough to assume that saturation had been reached regarding the parameters to be identified in the analysis. To the best of our knowledge, there is currently no such overview.

The scoping review was conducted following the framework proposed by Arksey and O’Malley [42]. To answer RQ-3 (Fig 1), we developed a search strategy focusing on two themes:

  • PHS workforce, referring to professionals providing essential public health services within local, state, or national level public health authorities.
  • Practice-oriented training programs, such as continuing education, training opportunities, and professional development programs, prepare individuals for academic work in PHS practice or practice-oriented roles in public health academia, including hybrid training programs that bridge both areas.

After piloting and refining the search strategy by two researchers (LA, SB), we searched the scientific databases PubMed and LIVIVO on October 26, 2021 (Appendix: S2 and S3 Tables). Retrieved records were de-duplicated in the bibliographic management software CITAVI (Swiss Academic Software GmbH) and imported in Rayyan, a web-based tool for conducting systematic reviews [43]. Initially, the screening process was calibrated and the predefined inclusion and exclusion criteria were tested for practicability and applicability. Records were assessed for eligibility based on the following criteria:

  • Population: Included records referred to the PHS workforce as defined above or individuals engaged in academic public health. Excluded records focused primarily on professionals involved in patient treatment.
  • Context: Included records referred to practice-oriented training and qualification approaches and programs as defined above. Excluded records encompassed training exclusively focused on one area (e.g., new master programs unrelated to the PHS) or lacked the objective of qualifying for the interface.
  • Setting: Included records focused on training approaches implemented in Germany as well as results from neighboring countries with comparable healthcare systems, including Austria—AT, Switzerland—CH, and the Netherlands—NL. Examples from the United Kingdom—UK, known for its Public Health Specialist Program within the National Health Service (NHS), were also considered exemplary. Excluded records concentrated on training approaches from other countries.
  • Publication date and language: The search was restricted to articles published between 2011 and 2021 in English, German, or Dutch.
  • Publication type: No restrictions were imposed based on publication type.
  • Full-text accessibility: Excluded records were those for which full-text access was unavailable.

Following the calibration of the screening process, a title-abstract screening and subsequent full-text screening was conducted independently by two researchers (LA, SB). Relevant publications were transferred to MAXQDA Analytics Pro 2022 (VERBI Software GmbH, Berlin) for coding and thematic synthesis analysis based on the approach outlined by Thomas and Harden [44]. A critical appraisal was not conducted in accordance with Arksey and O’Malley’s scoping review framework methodology [42]. Initially, three sample documents were coded line-by-line inductively by both researchers, demonstrating a high level of consensus. Subsequently, the remaining documents were coded individually by one researcher each, with new codes developed and added inductively as needed. All coded passages were then grouped, defined, and labeled according to identified similarities and differences, resulting in a hierarchical tree structure (Appendix: S4 Table). From the emerging descriptive themes, a first version of the toolbox was developed (LA, SB).

3.3 Stakeholder workshop

The third step involved a multi-professional and interdisciplinary stakeholder workshop, which aimed to critically appraise the first version of the toolbox by prioritizing skill-sets, competencies, and key elements for the development of tailored training programs. The in total 44 participants were purposely selected using a sampling plan similar to the interview approach. The participant pool was intentionally diverse, extending beyond the key informants from the qualitative interviews. It included representatives from various public health organizations, public health practitioners from PHS on local, state and federal level, academic researchers from different universities and research centers, and young professionals. The selection was guided by the intent to include individuals who shared professional and communication skills but offered a range of opinions, to encourage dynamic and critical discussions. This approach aimed to prevent the occurrence of "group think", where decisions are influenced by conformity or dominance of certain individuals within the group [45].

The recruitment phase lasted from April 01, 2022 to June 14, 2022, and the workshop itself took place from July 04–05, 2022.

The workshop utilized the Strategic Orientation Mapping (SOR) approach for the decision-making process proposed by Schlicht and Zinsmeister [45]. Prior to the workshop, participants completed an online survey to prioritize fields of activity at the interface. The results of this survey were presented at the beginning of the workshop. The workshop consisted of two parts: firstly, a critical appraisal of the tasks and activities deemed as priorities in the survey, and secondly, an assessment of purposefully selected training programs using the SWOT analysis approach [46, 47]. The selection of these training programs was a collaborative decision within the consortium, informed by the outcomes of both the qualitative interview study and the scoping review. This selection process was guided by a dual focus: the programs chosen were those deemed most compatible with the operational environment of the PHS workforce, and simultaneously, those that were practice-oriented, facilitating effective work at the interface between scientific research and practical application. The workshop results were documented, processed by the project team, and shared with participants for feedback and final adaptations.

3.4 Triangulation and synthesis

The results from step one to three were combined and consolidated through triangulation in workshops with the research consortium members. Triangulation in this context refers to a systematic integration of results obtained from the various methodological approaches employed in the previous steps [48]. Through an iterative process, data from steps one to three were subjected to reflexive analysis, where the results were compared and synthesized. These consolidated outcomes, summarized by two researchers (LA, SB), were subsequently presented to the project consortium members for critical assessment. The feedback received informed further revisions. This iterative cycle continued until a consensus was achieved. As a result of this cyclical process, a comprehensive toolbox was developed, aimed at facilitating the creation of customized training programs for public health professionals who work at the interface.

Subsequently, this toolbox was utilized in a series of workshops within the project consortium to develop a training program, specifically tailored to the unique structural conditions of all consortium member institutions. A detailed description of the application of the toolbox alongside with the resulting training program can be found in forthcoming publications (Arnold et al., in preparation).

3.5 Ethics and reflexive statement

Since the qualitative interviews and the stakeholder workshops contained exclusively technical questions, study-related stresses and risks were expected to be minimal. As some information provided by participants might involve criticism of their own agency or partner organizations, the pseudonymity of data was ensured, and no detailed information was disclosed. The study was conducted in accordance with the Declaration of Helsinki and data collection was approved by the Institutional Ethics Committee of Faculty of Medicine at the Heinrich-Heine-University Düsseldorf, protocol codes 2021–1646 (2021-10-28) and 2021–1646_1 (2021-12-16). We adhered to the Consolidated Criteria for Reporting Qualitative Research (COREQ) checklist in reporting the qualitative interviews, stakeholder workshops, and final triangulation and synthesis [49]. Results of the scoping review were reported according to the Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist [50].

Throughout the qualitative interviews and the stakeholder workshop, our research team maintained active engagement with both the research process and the participants. In line with the COREQ guidelines, we acknowledge that personal biases were not entirely avoidable. This recognition led us to adopt a reflexive stance throughout the research, where we remained aware of our potential biases and their potential impact on data collection and analysis. To mitigate the influence of these biases, we implemented several measures: We critically examined our assumptions, fostering a culture of self-reflection and open discussion. Regular monthly meetings with the project consortium were instrumental in this regard, providing an interdisciplinary platform for evaluating research progress and perspectives.

Our research team brought together a diverse range of expertise, including epidemiology (LA, DD), medicine (DD, HB), public health (LA, SB, HB, TC, SG, DS), sociology (SW, DS), and social sciences (LA, DS). All authors possessed experience in conducting qualitative research projects. This diversity in professional and academic backgrounds played a pivotal role in shaping the research process. During the triangulation and synthesis phase, the multifaceted perspectives within our team proved to be particularly valuable. Each member’s insights and interpretations, shaped by their unique disciplinary lenses, were critical in the development of the final toolbox.

4. Results

As a result of the methodology employed, two main outcomes have emerged. The first outcome involves the mapping of relevant areas of activity at the interface. The second outcome pertains to the development of a toolbox designed to create tailored training programs aimed at equipping public health professionals for work at these interface. Both outcomes laid the groundwork for the currently ongoing development and pilot testing of a customized training program tailored to the specific requirements and local needs of the EvidenzÖGD research consortium (Link). A comprehensive report on this aspect will be provided once the piloting phase is completed (Arnold et al., in preparation).

4.1 Fields of activity at the interface

A total of 24 individuals were interviewed in 23 semi-structured interviews. Most of the participants had multiple specialties and areas of expertise due to their professional background, which is shown accordingly in Table 1. Of the interviewees, sixteen primarily worked in the PHS and ten at a university. Five participants identified themselves as early career professionals, and eight brought an international perspective.

thumbnail
Download:
Table 1. Characteristics of the interview participants.

https://doi.org/10.1371/journal.pone.0292192.t001

The duration of the interviews ranged from 31 to 56 minutes. The interviews included an equal representation of men and women. Based on the interviews, areas of knowledge transfer activity at the municipal level were identified and clustered, along with identification of requisite skills and competencies.

The obtained results were further discussed and prioritized in the multi-professional and interdisciplinary stakeholder workshop with 48 experts. Of these, 40 participants responded to the initial question regarding the positioning of their professional expertise in a triangle between practice, academia, and teaching. Among them, 18 experts primarily identified with public health practice, 14 experts primarily with public health academia, and four experts indicated a prior focus on teaching and conceptualization of training programs. Four experts situated themselves at the intersection of all three domains. Additionally, 30 experts reported having five or more years of experience in the field of public health, while 21 experts reported having five or more years of experience specifically in the PHS. Furthermore, 21 experts mentioned their participation in the design of a training program at least once.

The results of the interview analysis and the stakeholder workshop were further elaborated by the interdisciplinary research consortium through a series of conceptual workshops. Finally, the consortium consolidated six fields of activity for establishing evidence-informed processes and structures at the interface between academia and practice in LHAs in Germany:

  • Networking and committee activities: This field focuses on creating and fostering networking opportunities among stakeholders at the community level. It also includes the mapping of needs, goals, and expectations to promote ongoing engagement.
  • Knowledge management (evidence use): This field entails building sector-specific and cross-sectoral expertise on municipal PHS ("practical knowledge"). It also includes the development of practice-oriented research questions, the conduction of evidence syntheses including quality assessments, and the identification of suitable implementation approaches.
  • Knowledge communication: This field involves tailoring expertise and research findings to local conditions by developing appropriate communication materials, such as policy briefs, statements, or evidence synthesis. It also includes the development, implementation, and promotion of a joint communication strategy to facilitate cross-institutional knowledge exchange between stakeholders, team members, leaders, and decision-makers.
  • Project management: This field encompasses evidence-informed identification, adaptation and subsequent implementation, administration, and evaluation of (complex) public health interventions and measures. It also includes the securement of project funding.
  • Capacity building and change management: This field focuses on opportunities to strengthen the structural conditions of knowledge-transfer processes, aiming to enhance the capacity of individuals and organizations to apply research evidence effectively.
  • Consolidation of knowledge-transfer processes: This field involves the sustainable implementation of evidence-informed processes and structures. It also includes establishing feedback mechanisms and promoting individual as well as organizational self-reflection in evidence use.

A detailed description of the knowledge, skills, attitudes, and attributes required for each field can be found in the competency framework developed as part of the EvidenzÖGD project (Link).

4.2 Toolbox for the development of relevant training programs

The identification and extraction of relevant aspects for the development of tailored training programs for the PHS workforce on local level was informed by the Six-Step Model for Developing Competency Frameworks proposed by Batt et al. [51]. Emphasis was placed on defining desired outcomes and considering relevant process aspects, including inputs and activities. Continuous evaluation of this process took place throughout the iterative development phase, enabling the utilization of findings to enhance the ongoing processes of model development and revision.

A total of 1,706 records (PubMed: n = 1,468; LIVIVO: n = 238) were identified in the scoping review. Following the title-abstract screening, 1,590 records were excluded, and an additional 24 records were excluded during the full-text screening. Exclusion reasons are displayed in Fig 2. Ultimately, 25 records were included in the review. The characteristics of the included records can be found in Appendix (S5 Table).

thumbnail
Download:
Fig 2. PRISMA flowchart.

https://doi.org/10.1371/journal.pone.0292192.g002

Based on these 25 records, four core elements were identified as relevant for the development of a training program intended to qualify professionals for work at the interface: (1) context-related aspects, (2) content-related aspects, (3) aspects relevant for determining the training format and (4) aspects relevant for consolidation and further development of the program. A total of 15 parameters were assigned to these core elements, and specific guiding questions with examples were developed to facilitate the application of the toolbox. Subsequently, each of the four aspects, along with corresponding guiding questions, will be presented.

4.2.1 Context-related aspects.

Context-related aspects involve essential program parameters and factors that need to be established prior to program implementation. These aspects encompass (a) agreement on program objectives, (b) involvement of relevant stakeholders, (c) identification of required resources, and (d) definition of the program setting. To facilitate the conceptualization of a training program considering contextual factors, Table 2 presents these parameters and corresponding guiding questions.

thumbnail
Download:
Table 2. Context-related aspects for the development of a training program.

https://doi.org/10.1371/journal.pone.0292192.t002

4.2.2 Content-related aspects.

Content-related aspects are essential for structuring the program content effectively. These aspects include determining the (a) relevant program content, (b) selecting suitable training and education formats, (c) didactical concepts, and specifying (d) appropriate measurability and assessment methods. Table 3 illustrates these parameters and provides guiding questions to aid determining the program’s content.

thumbnail
Download:
Table 3. Content-related aspects for the development of a training program.

https://doi.org/10.1371/journal.pone.0292192.t003

4.2.3 Aspects relevant for determining the training format.

Once the context- and content-related aspects have been established, the next step is to specify the preferred program format. Based on the findings of the scoping review, four parameters were identified as relevant for the transition into a training program (Table 4). These parameters include determining the program format (a), program duration and density (b), professional credentialing requirements (c), and the recruitment process and selection strategy (d).

thumbnail
Download:
Table 4. Aspects relevant for determining the training format.

https://doi.org/10.1371/journal.pone.0292192.t004

4.2.4 Program parameters of selected training programs.

Within the scoping review, we identified various qualification models, with five being particularly suitable for facilitating evidence use and knowledge transfer within the local PHS in Germany. These models include trainee programs, PhD programs, rotational concepts, continuing professional development (CPD) courses that integrate both academia and practice, and PHS-related modules in academic degree programs. The selection of these models was based on a collaborative endeavor within the consortium, deeply informed by insights from the qualitative interviews, the scoping review, and the stakeholder workshop. Our criteria for choosing these training programs were twofold: First, we concentrated on programs compatible with the operational environment of the PHS workforce in Germany, ensuring alignment with the specific requirements and challenges faced by professionals. Second, we prioritized programs that were practice-oriented, designed to effectively bridge the gap between scientific research and practical application in public health service. This dual-focused selection process was aimed at identifying programs that meet the contextual needs of the PHS workforce while also enhancing knowledge transfer and application at the interface of academia and practice.

Table 5 presents the selected training programs based on their program parameters. All five qualification models were subject of a SWOT analysis during the stakeholder workshop. Special attention was paid to the feasibility of ensuring knowledge transfer at the local level.

thumbnail
Download:
Table 5. Program parameters of selected training programs.

https://doi.org/10.1371/journal.pone.0292192.t005

4.2.5 Aspects relevant for consolidation and further development of the training program.

Furthermore, parameters necessary for the long-term existence of the program were classified as aspects relevant for consolidation and further development of the program. These aspects encompass a range of factors that can be grouped into three overarching parameters, including (a) piloting and implementation, (b) evaluation and quality assessment, and (c) advancement and transferability. Table 6 provides an overview of these aspects, along with guiding questions that aim to support a comprehensive understanding and careful consideration of the fundamental elements required for ensuring the ongoing success of the program.

thumbnail
Download:
Table 6. Aspects relevant for consolidation and further development of the training program.

https://doi.org/10.1371/journal.pone.0292192.t006

5. Discussion

5.1 Summary of findings

Current research-oriented training opportunities provided by well-established public health programs effectively prepare students for academic roles but often neglect the complexities of working in and with local governments [93]. Conversely, practice-based training programs often do not adequately manage to equip practitioners with the skills necessary for academic tasks and roles.

To address the lack of available training offerings that meet the unique needs and requirements of the local context, we have developed a comprehensive toolbox for conceptualizing integrated training programs. These programs aim to enhance evidence transfer between academia and practice in local PHS. Through an iterative process involving key informant interviews, a scoping review, and a multidisciplinary stakeholder workshop, we identified essential aspects and parameters for such a toolbox. The toolbox presented comprises four core elements, encompassing a total of 15 parameters:

  1. content-related aspects (incl. four parameters),
  2. context-related aspects (incl. four parameters),
  3. aspects relevant for determining the training format (incl. four parameters), and
  4. aspects relevant for consolidation and further development (incl. three parameters).

Specific guiding questions with illustrative examples have been developed for each parameter to assist in the development of tailored training models aligned with local needs and requirements. These guiding questions enable program developers to effectively assess the complexities associated with developing, implementing, and sustaining the program’s effectiveness and impact.

5.2 Short guidance on how to apply the toolbox

The toolbox presented is deliberately generic so that it can be applied to different contexts. To this end, the accompanying guiding questions are intended to be supportive to determine the relevant focus and content. While the toolbox can be used standalone by systematically answering all questions in sequence, we believe the toolbox to be most useful in an iterative deliberative process, as recommended by experts in the field [51]. To facilitate this, we propose a seven-step process, visually depicted in Fig 3, with a brief explanation of each step outlined below.

thumbnail
Download:
Fig 3. Schematic how-to-use guidance of the toolbox.

https://doi.org/10.1371/journal.pone.0292192.g003

To initiate the process (step 1), it is crucial to form a core team comprising representatives from relevant public health institutions on the local level. This core team should include experts with broad professional backgrounds from universities, research institutions, academies of public health services, local public health authorities, local administration, and ideally, government agencies. Reflecting on activities and structures relevant for evidence transfer between academia and practice provides insights into existing practices and highlights areas for improvement. This reflection enhances understanding regarding the dissemination of research findings and utilization of evidence in current practice. Ensuring alignment and collaboration throughout the development process is further supported by each entity represented in the core team initially defining their goals, needs, resources, and requirements.

In the preparation phase (step 2), the defined goals, interests, needs, resources and requirements should be aligned with underlying competencies of each member institution. This mapping exercise provides a deeper understanding of the strengths and expertise within the core team while identifying any overlaps or conflicts. Promptly resolving potential conflicts enables effective collaboration and smooth progress throughout the development process. Performing a stakeholder analysis based on the results of the mapping will provide insights into the broader ecosystem and helps identifying relevant experts not yet involved. These experts can be engaged by expanding the core team or involving them in the program advisory group, which should be formed to ensure comprehensive guidance throughout the development process.

During the development phase (step 3), the core team uses the toolbox to reach a consensus on the fundamental elements of the training program. An iterative process is recommended refining content- as well as context-related aspects by systematically assessing the provided guiding questions (see Tables 2 and 3) in successive workshops conducted by the core team. To ensure continuous exchange within the core team, a predetermined schedule should be agreed upon in the initial meeting. Once program objectives and content are specified, the program format, including duration and density, professional credentialing requirements, and desired recruitment and selection criteria, can be defined (see Tables 4 and 5).

In the refinement phase (step 4), the core team consolidates the first version of the qualification model based on the agreed-upon parameters. Subsequently, the model is presented to the program advisory group, including future participants, policy makers, and experts not involved in prior steps. Gathering input and feedback allows for refining and improving the model. The core team adapts the model according to the feedback received, ensuring its relevance, effectiveness, and alignment with stakeholders’ expectations.

To test the feasibility and practicability of the model, it is recommended to pilot the model in a real-world setting at the outset the implementation phase (step 5) (see Table 6(A)). Careful assessment of results and feedback from the pilot phase enables necessary adaptations to the model. Once refined, the model can be implemented in the respective setting.

During the evaluation phase (step 6), efforts should be dedicated to developing a comprehensive evaluation and quality assessment strategy for the program. The core team defines relevant quality assurance tools to assess the program’s effectiveness, impact, and adherence to standards (see Table 6(B)). Continuous monitoring and evaluation, accompanied by adjustments, ensure the ongoing quality improvement of the tailored qualification model.

In the final advancement phase (step 7), a strategy for continuous development to update and maintain the program over time should be established (see Table 6(C)). Assessing the transferability of the adapted model to other contexts and settings, while considering scalability and applicability, is crucial. Defining a dissemination strategy facilitates sharing the knowledge and experiences gained during the development and implementation process, ensuring broader adoption and impact of the model.

5.3 Need for institutional anchored evidence eco-systems on local level

Effective implementation of needs-based public health interventions requires the assessment, synthesis, and appropriate utilization of research evidence, in alignment with the broader policy system. The practical challenges encountered by LHAs in this regard became evident during the COVID-19 pandemic, where time-sensitive decisions had to be made amidst epistemic uncertainties [5, 17]. This was aggravated by absence of structures facilitating rapid knowledge transfer and exchange in many municipalities, particularly at the pandemics’ onset. To bridge the gap between evidence generation and utilization in the local PHS in Germany, the establishment of evidence ecosystems is deemed highly relevant.

Comprehensive methodological skills, including formulating appropriate research questions and conducting evidence syntheses, are essential to obtain timely and robust evidence on public health-related challenges. Successful implementation of complex interventions relies on the effective translation of new research findings from academia into practice and vice versa. Therefore, sector-specific and cross-sectoral expertise in public policy and administration is vital for aligning research questions with local needs, as emphasized by experts in the field [23, 94–96], particularly on the local level. To recommend and implement locally tailored strategies, scientific evidence must be communicated in a meaningful and usable manner for policymakers, decision-makers, and practitioners [2, 97]. This necessitates the production of succinct and user-friendly evidence syntheses, specifically tailored to meet informational demands of the intended users. Additionally, addressing local needs entails active engagement and involvement of key stakeholders, interdisciplinary teams of experts, and collaborative and continuous efforts between evidence generators and evidence users [10].

The toolbox presented aims to serve as a guide to conceptualize training programs that teach skills related to evidence generation and to train participants in evidence utilization by translating and applying parts of a generic body of evidence to the community context in which LHAs operate. Within the EvidenzÖGD-project we utilized the toolbox by following the proposed step-wise approach of the how-to-use-guide to develop a customized training program tailored to the specific needs of the research consortium. The resulting qualification model is currently undergoing piloting and evaluation (Arnold et al., in preparation).

Tailored training programs are intended to equip public health researchers and practitioners with the relevant skills to design and implement needs-based PHS research. The joint development of training approaches seeks to strengthen cross-institutional collaboration and enhance understanding of evidence generation and utilization. In the long term, this process is meant to enable public health researchers and practitioners to conduct high-quality PHS research, aligned with local needs, thereby saving resources and enhancing the evidence base for successfully conducting complex public health measures on site. Consequently, co-developing tailored solutions within a local evidence ecosystem can contribute not only to scientific impact but, potentially, to practical impact as well.

However, the successful implementation and effectiveness of such tailored programs will also be influenced by contextual conditions. In addition to adequate material and financial resources, laws and regulations that mandate evidence-informed decision-making processes are essential. Therefore, a clear political endorsement and support for the integration of evidence-informed practices into the policy and practice on local level are needed. Moving forward, attention to these contextual factors is vital to ensure the optimal impact and sustainability of tailored training programs.

6. Strengths and limitations

The design of this study incorporates some noteworthy strengths. The novelty of our overarching approach, in which we looked at aspects relevant to strengthen knowledge-translation and exchange at the interface of academia and practice in local PHS in Germany by taking into account evidence required from a scoping review, several key-informants and a multi-professional group of experts. The comprehensive mixed-methods approach allowed us to incorporate a broad variety of methods and types of evidence. The iterative approach enabled critical evaluation of our own research results, contributing to continuous quality assurance and consolidation. Based on this, we assume that our study, which is jointly organized by academia and practice, can contribute significantly to the improvement of knowledge-transfer processes at the municipal level by means of the proposed toolbox and its application.

However, the study possesses some methodological limitations that warrant consideration. Firstly, the search of the scoping review was restricted to five European countries. This decision was primarily made to identify training programs that have been implemented or tested within a context comparable to the German public health system (AT, CH, and NL) or in a well-established context (UK). We are aware of other good examples from the United States and Canada, among others [3, 98–101]. Due to the fact that we supplemented the results with 23 key-informant interviews and finally reflected and prioritized the entire toolbox with 48 experts, we assume relevant coverage. It is important to note, that our aim was not to capture all available records within the scoping review. However, the achieved saturation gives us confidence that the approach captured the majority of relevant publications and reflects on the majority of relevant parameters. Some parameters (e.g., “piloting and implementation” or “advancement and transferability”) were underrepresented in the identified training programs, which might be due to the fact that these aspects are so far not regularly considered in training program development.

Secondly, the qualitative interview study had a strong focus on the German context, with limited inclusion of international experiences. Nevertheless, a small sample of well-informed international interviewees provided valuable insights into current debates within their country contexts. While this emphasis on individuals’ experiences within the system strengthens the development of tailored training programs, it may limit the emergence of "out of the box" thinking.

Lastly, the multidisciplinary stakeholder workshop allowed for open discussions, benefiting from participants diverse professional backgrounds, ranging from students to retired public health experts. This facilitated the integration of different disciplines and varying levels of expertise and interests. Although all topics were successfully discussed and prioritized as planned, it is worth considering alternative approaches, such as a DELPHI process, to generate additional solutions. However, it should be noted that the time and availability of the experts involved was limited and other approaches would have been much more time-consuming. In this regard, the commenting phase following the stakeholder workshop proved valuable in mitigating potential biases associated with group thinking processes, while also partially addressing the time constraint limitation.

While our study explicitly focused on enhancing knowledge translation and exchange within the decentralized governance structure of the public health system in Germany, we recognize the potential limitations of the toolbox’s applicability to systems with different structures. Our interviews and scoping review are tailored to the specific needs of the German context and therefore focus on five selected European countries, which we consider a strength. However, we intentionally designed the toolbox with a generic framework to enable adaptation to various regions and contexts. We acknowledge the potential utility and necessity of this adaptability in designing customized training programs for different systems. The toolbox’s transferability to other geographical and administrative contexts is currently being explored in the ongoing EvidenzÖGD study. The extent to which the toolbox can effectively serve in diverse administrative systems and settings without a decentralized governance structure must be examined in the future.

7. Conclusions

This study aimed to develop a toolbox that serves as a guide to develop training programs to equip public health professionals and researchers with the knowledge, skills, and capacities relevant to implementing evidence-informed approaches at the interface of academia and practice. Training programs that are explicitly tailored to local needs have the potential to foster a shared research culture focusing on topics relevant for the PHS and establish a sustainable cross-institutional infrastructure known as a local evidence ecosystem.

Applying the toolbox for training program development by following the proposed how-to-use-guide can contribute to the strengthening and enhancement of the local evidence ecosystem in which they are embedded. The efficacy of this approach should and will be evaluated in future studies. If successful, a well-established evidence ecosystem can provide the much-needed bridge between the evidence-generating and the evidence-utilizing system. This mutually beneficial relationship benefits both public health academia and practice, as research questions and projects tailored to local needs can lead to the development of customized solutions. Consequently, an institutionally anchored knowledge-transfer and exchange ecosystem helps to ensure a transparent and evidence-informed fulfillment of local PHS tasks and activities.

Supporting information

S1 Table. Coding tree structure and anonymized exemplary passages from the qualitative interviews.

https://doi.org/10.1371/journal.pone.0292192.s001

(XLSX)

S2 Table. Search strategy for the database PUBMED.

https://doi.org/10.1371/journal.pone.0292192.s002

(DOCX)

S3 Table. Search strategy for the database LIVIVO.

https://doi.org/10.1371/journal.pone.0292192.s003

(DOCX)

S4 Table. Overarching coding tree structure from the scoping review.

https://doi.org/10.1371/journal.pone.0292192.s004

(DOCX)

S5 Table. Characteristics of the studies included in the scoping review.

https://doi.org/10.1371/journal.pone.0292192.s005

(DOCX)

Acknowledgments

The authors express their gratitude to the study participants for their time and dedication. Additionally, we would like to extend our appreciation to Katharina Kreffter, Lena Raith, Meret Reuther, Joy Pirig, and Luisa Urban for their valuable support throughout the project. We also thank Jan M. Stratil for his valuable time and constructive feedback on earlier manuscript versions.

In this study, the contributions of the EvidenzÖGD study project consortium were integral to the research progress. The consortium—a collaborative group of researchers and practitioners from the Academy of Public Health Services (AÖGW), the Heinrich-Heine-Universität Düsseldorf (HHU) and the Local Health Authority Düsseldorf (LHA)—includes the following experts: Laura Arnold (AÖGW, project lead and corresponding author), Simon Bimczok (AÖGW), Hannah Schütt (AÖGW), Dagmar Starke (AÖGW), Nico Dragano (HHU), Delbar Dilmaghani (HHU), Annika Höhmann (HHU), Simon Götz (HHU), Simone Weyers (HHU), Ravina Ambalavanar (LHA), Anke Kietzmann (LHA), Andrea Melville-Drewes (LHA), Guido Schenuit (LHA), Trudpert Schoner (LHA). and Michael Schäfer (Freelancer).

Additional information

Informed consent statement

Written Informed consent was obtained from all subjects involved in the qualitative study.

References

  1. 1. Yost J, Dobbins M, Traynor R, DeCorby K, Workentine S, Greco L. Tools to support evidence-informed public health decision making. BMC Public Health 2014; 14(728). pmid:25034534
  2. 2. Armstrong R, Waters E, Dobbins M, Anderson L, Moore L, Petticrew M, et al. Knowledge translation strategies to improve the use of evidence in public health decision making in local government: intervention design and implementation plan. Implement Sci 2013; 8(121). pmid:24107358
  3. 3. Dobbins M, Greco L, Yost J, Traynor R, Decorby-Watson K, Yousefi-Nooraie R. A description of a tailored knowledge translation intervention delivered by knowledge brokers within public health departments in Canada. Health Res Policy Syst 2019; 17(1): 63. pmid:31221187
  4. 4. Baltussen R, Jansen M, Oortwijn W. Evidence-Informed Deliberative Processes for Legitimate Health Benefit Package Design—Part I: Conceptual Framework. Int J Health Policy Manag 2022; 11(10): 2319–26. pmid:34923808
  5. 5. Stratil JM, Voss M, Arnold L. WICID framework version 1.0: criteria and considerations to guide evidence-informed decision-making on non-pharmacological interventions targeting COVID-19. BMJ Glob Health 2020; 5(11). pmid:33234529
  6. 6. Brownson RC, Fielding JE, Maylahn CM. Evidence-based public health: a fundamental concept for public health practice. Annu Rev Public Health 2009; 30: 175–201. pmid:19296775
  7. 7. Langlois EV, Becerril Montekio V, Young T, Song K, Alcalde-Rabanal J, Tran N. Enhancing evidence informed policymaking in complex health systems: lessons from multi-site collaborative approaches. Health Res Policy Syst 2016; 14(20). pmid:26983405
  8. 8. Oliver K, Pearce W. Three lessons from evidence-based medicine and policy: increase transparency, balance inputs and understand power. Palgrave Commun 2017; 3(43).
  9. 9. Stewart R, Dayal H, Langer L, van Rooyen C. The evidence ecosystem in South Africa: growing resilience and institutionalisation of evidence use. Palgrave Commun 2019; 5(90).
  10. 10. WHO. Evidence, policy, impact: WHO guide for evidence-informed decision-making: WHO guide for evidence-informed decision-making. Geneva 2021.
    • 11. Straus SE, Tetroe J, Graham ID (Ed.). Knowledge translation in health care: Moving from evidence to practice. Second edition. Chichester, West Sussex, Hoboken, NJ: Wiley/BMJ Books; 2013.
      • 12. Knowledge Translation in Health Care: Moving from Evidence to Practice—CIHR; 2015 [cited 2024 January 22] Available from: https://cihr-irsc.gc.ca/e/40618.html.
        • 13. Caron RM, Hiller MD, Wyman WJ. Engaging local public health system partnerships to educate the future public health workforce. Journal of community health 2013; 38(2): 268–76. pmid:22940868
        • 14. van den Driessen Mareeuw F, Vaandrager L, Klerkx L, Naaldenberg J, Koelen M. Beyond bridging the know-do gap: a qualitative study of systemic interaction to foster knowledge exchange in the public health sector in The Netherlands. BMC Public Health 2015; 15(922). https://doi.org/10.1186/s12889-015-2271-7
          • 15. OECD (Ed.). Building Capacity for Evidence-Informed Policy-Making. Paris: OECD; 2020.
            • 16. Parkhurst JO. The politics of evidence: From evidence-based policy to the good governance of evidence. Abingdon, Oxon, New York, NY: Routledge 2017.
              • 17. Greenhalgh T, Engebretsen E. The science-policy relationship in times of crisis: An urgent call for a pragmatist turn. Soc Sci Med 2022; 306(115140). pmid:35717826
              • 18. SVR. Resilienz im Gesundheitswesen: Wege zur Bewältigung künftiger Krisen Gutachten 2023Sachverständigenrat zur Begutachtung der Entwicklung im Gesundheitswesen (Ed.). 1. Auflage. Berlin: MWV Medizinisch Wissenschaftliche Verlagsgesellschaft; 2023.
                • 19. Arnold L, Teichert U. Politischer Reformprozess im Zuge der COVID-19-Pandemie: Der Pakt für den Öffentlichen Gesundheitsdienst. Public Health Forum 2021; 29(1): 47–50.
                • 20. Blümel M, Spranger A, Achstetter K, Maresso A, Busse R. Germany: Health System Review. Health Systems in Transition 2020; 22(6): 1–272. pmid:34232120
                • 21. Federal Ministry of Health (Ed.). The German healthcare system: Strong. Reliable. Proven. Berlin; 2020. Available from: https://www.bundesgesundheitsministerium.de/fileadmin/Dateien/5_Publikationen/Gesundheit/Broschueren/200629_BMG_Das_deutsche_Gesundheitssystem_EN.pdf [cited 2024 January 23]
                  • 22. Busse R, Riesberg A. Health Care Systems in Transition: GermanyWHO on behalf of the European Observatory on Health Systems and Policies (Ed.). Copenhagen; 2004.
                    • 23. Kneale D, Rojas-García A, Thomas J. Obstacles and opportunities to using research evidence in local public health decision-making in England. Health Res Policy Syst 2019; 17(61). pmid:31248422
                    • 24. Starke D, Arnold L. Der ÖGD im 21. Jahrhundert. Chancen und Herausforderungen. In: BARMER Gesundheitswesen aktuell 2021 2021; 68–95.
                    • 25. Kuhn J, Wildner M. Corona-Krise und öffentlicher Gesundheitsdienst. GGW 2020; 20(4): 15–22.
                    • 26. Szagun B, Kuhn J, Starke D. Kommunale Gesundheitsförderungspolitik und das Präventionsgesetz. Präv Gesundheitsf 2016; 11(4): 265–70.
                    • 27. Holst J. COVID-19: Besondere Herausforderungen an Public Health in Theorie und Praxis. Gesundheitswesen 2020; 82(11): 829–35.
                    • 28. Beirat Pakt ÖGD. Wissenschaft und Forschung im und für einen zukunftsfähigen ÖGDBeirat zur Beratung zukunftsfähiger Strukturen im Öffentlichen Gesundheitsdienst in Umsetzung des Paktes für den Öffentlichen Gesundheitsdienst (Beirat Pakt ÖGD) (Ed.). Berlin. Strukturelle und zukunftsorientierte Weiterentwicklung des Öffentlichen Gesundheitsdienstes; 2023. Available from: https://www.bundesgesundheitsministerium.de/themen/gesundheitswesen/pakt-fuer-den-oegd/beirat-pakt-oegd [cited 2024 January 22]
                      • 29. BMG. Stärkung der Zusammenarbeit zwischen Öffentlichem Gesundheitsdienst und Public-Health-ForschungBundesministerium für Gesundheit (BMG) (Ed.). Öffentliche Bekanntmachung des Bundesministeriums für Gesundheit (BMG); veröffentlicht am 27.02.2020; 2020. Available from: https://projekttraeger.dlr.de/de/foerderung/foerderangebote-und-programme/staerkung-der-zusammenarbeit-zwischen-oeffentlichem-gesundheitsdienst-und-public-health [cited 2024 January 22]
                        • 30. Leopoldina. Public Health in Deutschland: Strukturen, Entwicklungen und globale HerausforderungenDeutsche Akademie der Technikwissenschaften, Deutsche Akademie der Naturforscher Leopoldina, Union der Deutschen Akademien der Wissenschaften (Ed.). Halle (Saale), München, Mainz; 2015. Available from: http://nbn-resolving.org/urn:nbn:de:gbv:3:2-106000 [cited 2024 January 23]
                          • 31. Zukunftsforum Public Health. Der Pakt für den Öffentlichen Gesundheitsdienst. Empfehlungen für Umsetzung und Ausgestaltung: Empfehlungen für Umsetzung und AusgestaltungZukunftsforum Public Health (Ed.). ; 2020. Available from: https://zukunftsforum-public-health.de/stellungnahme-pakt-oegd/ [cited 2024 January 23]
                            • 32. Geene R, Gerhardus A, Grossmann B, Kuhn J, Kurth BM, Moebus S, et al. Health in All Policies–Entwicklungen, Schwerpunkte und Umsetzungsstrategien für DeutschlandZukunftsforum Public Health (Ed.). ; 2020. Available from: https://zukunftsforum-public-health.de/health-in-all-policies/ [cited 2024 January 22]
                              • 33. Geene R, Kurth BM, Matusall S. Health in All Policies–Entwicklungen, Schwerpunkte und Umsetzungsstrategien für Deutschland. Gesundheitswesen 2020; 82(7): e72–e76. pmid:32392589
                              • 34. Czabanowska K, Smith T, Könings KD, Sumskas L, Otok R, Bjegovic-Mikanovic V, et al. In search for a public health leadership competency framework to support leadership curriculum-a consensus study. Eur J Public Health 2014; 24(5): 850–6. pmid:24121098
                              • 35. Arnold L, Starke D. Evidenzinformiertes Planen für Gesundheit—Koordination und Steuerung. In: Klapper B, Cichon I, editors. Neustart! Für die Zukunft unseres Gesundheitswesens. Berlin: MWV 2021; 581–8.
                                • 36. AÖGW, SAMA, Regierungspräsidium Stuttgart, AGL und LGL, Bayerisches Staatsministerium für Gesundheit und Pflege, Sächsisches Staatsministerium für Soziales und Verbraucherschutz (Ed.). Curriculum Kursweiterbildung Öffentliches Gesundheitswesen. Düsseldorf; 2019.
                                  • 37. Hoffmann D. Der perfekte Start in eine ÖGD-Karriere: arzt&karriere; 2022 [cited 2024 January 23] Available from: https://arztundkarriere.com/oegd/statt-weiterbildung-trainee-oegd-karriere-fulda/.
                                    • 38. Robert Koch-Institut. Postgraduierten-ausbildung für angewandte Epidemiologie (PAE) am RKI: (Deutsches Feldepidemiologie-Trainingsprogramm, FETP): Abteilung für Infektionsepidemiologie am RKI; 2021 [cited 2024 January 23] Available from: https://www.rki.de/DE/Content/Institut/OrgEinheiten/Abt3/Infektepidem_Training.html.
                                      • 39. Dresing T, Pehl T. Praxisbuch Interview, Transkription & Analyse: Anleitungen und Regelsysteme für qualitativ Forschende. 8. Auflage. Marburg: Eigenverlag 2018.
                                        • 40. Mayring P. Qualitative Inhaltsanalyse: Grundlagen und Techniken. 12., überarb. Aufl. Weinheim: Beltz 2015.
                                          • 41. Mayring P. Einführung in die qualitative Sozialforschung: Eine Anleitung zu qualitativem Denken. 5., überarb. und neu ausgestattete Aufl. Weinheim: Beltz 2002.
                                            • 42. Arksey H, O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology 2005; 8(1): 19–32.
                                            • 43. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev 2016; 5(210). pmid:27919275
                                            • 44. Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol 2008; 8(45). pmid:18616818
                                            • 45. Schlicht W, Zinsmeister M. Gesundheitsförderung systematisch planen und effektiv intervenieren. Berlin, Heidelberg: Springer Berlin Heidelberg 2015.
                                              • 46. Renault V. SWOT Analysis: Strengths, Weaknesses, Opportunities, and Threats: Center for Community Health and Development; 2023 [cited 2024 January 23] Available from: https://ctb.ku.edu/en/table-of-contents/assessment/assessing-community-needs-and-resources/swot-analysis/main.
                                                • 47. David FR. Strategic management: Concepts and cases. 13th ed. Upper Saddle River N.J.: Prentice Hall 2011.
                                                  • 48. Moran-Ellis J, Alexander VD, Cronin A, Dickinson M, Fielding J, Sleney J, et al. Triangulation and integration: processes, claims and implications. Qualitative Research 2006; 6(1): 45–59.
                                                  • 49. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007; 19(6): 349–57. pmid:17872937
                                                  • 50. Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med 2018; 169(7): 467–73. pmid:30178033
                                                  • 51. Batt A, Williams B, Rich J, Tavares W. A Six-Step Model for Developing Competency Frameworks in the Healthcare Professions. Front Med (Lausanne) 2021; 8(789828). pmid:34970566
                                                  • 52. Cheetham M, Wiseman A, Khazaeli B, Gibson E, Gray P, van der Graaf P, et al. Embedded research: a promising way to create evidence-informed impact in public health? J Public Health (Oxf) 2018; 40: 64–70. pmid:29538721
                                                  • 53. Currie CA, Nottage C, Spurrier N, Madden DL. Comparison of public health specialty training in Australia and England. Public Health 2020; 188: 35–41. pmid:33069009
                                                  • 54. Dorner TE, Diem G. Public Health Ausbildung in Österreich. Ein Überblick. Wiener Medizinische Wochenschrift 2014; 164: 131–40.
                                                  • 55. Erwin PC, Brownson CA, Eyler AA, Hefelfinger J, Deshpande AD, Macchi M, et al. Long-Term Evaluation of a Course on Evidence-Based Public Health in the U.S. and Europe. American Journal of Preventive Medicine 2021; 61(2): 299–307.
                                                  • 56. Gray SF, Evans D. Developing the public health workforce: training and recognizing specialists in public health from backgrounds other than medicine: experience in the UK. Public Health Reviews 2018; 39(14). pmid:29942671
                                                  • 57. Harrison RA, Gemmell I, Reed K. The effect of using different competence frameworks to audit the content of a masters program in public health. Front Public Health 2015; 3(143). pmid:26042213
                                                  • 58. Heller RF, Zurynski R, Barrett A, Oaiya O, Madhok R. Open Online Courses in Public Health: experience from Peoples-uni. F1000Research 2017; 6(170). pmid:28491283
                                                  • 59. Idler N, Huber J, Mutius S, Welbergen L, Fischer MR. Prevention and health promotion from theory to practice: The interprofessional MeMPE Summer University for students of Medicine, Master of Public Health and Epidemiology. GMS journal for medical education 2016; 33(5). pmid:27990468
                                                  • 60. Jansen MWJ, Hoeijmakers M. A masterclass to teach public health professionals to conduct practice-based research to promote evidence-based practice: a case study from The Netherlands. J Public Health Manag Pract 2013; 19(1): 83–92. pmid:23169408
                                                  • 61. Könings KD, Jong N, Lohrmann C, Sumskas L, Smith T, O’Connor SJ, et al. Is blended learning and problem-based learning course design suited to develop future public health leaders? An explorative European study. Public Health Reviews 2018; 39(13).
                                                  • 62. Ramsay A, Harries AD, Zachariah R, Bissell K, Hinderaker SG, Edginton M, et al. The Structured Operational Research and Training Initiative for public health programmes. Public health action 2014; 4(2): 79–84. pmid:26399203
                                                  • 63. Baxter M, Islam G, Parsons H, Enoch DA, Fry C, Kiernan M, et al. A new infection trainee education programme from the Healthcare Infection Society. J Hosp Infect 2016; 93(2): 157–8. pmid:27105750
                                                  • 64. Dey P, Brown J, Sandars J, Young Y, Ruggles R, Bracebridge S. The United Kingdom Field Epidemiology Training Programme: meeting programme objectives. Euro Surveill 2019; 24(36). pmid:31507267
                                                  • 65. Diem G, Brownson RC, Grabauskas V, Shatchkute A, Stachenko S. Prevention and control of noncommunicable diseases through evidence-based public health: implementing the NCD 2020 action plan. Global health promotion 2016; 23(3): 5–13. pmid:25758171
                                                  • 66. Peik SM, Mohan KM, Baba T, Donadel M, Labruto A, Loh LC. Comparison of public health and preventive medicine physician specialty training in six countries: Identifying challenges and opportunities. Medical Teacher 2016; 38(11): 1146–51. pmid:27093229
                                                  • 67. McCulloch J, Bracebridge S, Thompson M, van de Venter E, Oliver I. Developing capacity in field epidemiology in England. Perspectives in public health 2014; 134(5): 297–9. pmid:25169617
                                                  • 68. Smith T, Stankunas M, Czabanowska K, Jong N, O’Connor S, Fowler Davis S. Principles of all-inclusive public health: developing a public health leadership curriculum. Public Health 2015; 129(2): 182–4. pmid:25707932
                                                  • 69. Gillam S, Rodrigues V, Myles P. Public health education in UK medical schools-towards consensus. J Public Health (Oxf) 2016; 38(3): 522–5. pmid:26045473
                                                  • 70. Turner-Wilson AL, Mills AM, Rees K. Can nurses rise to the public health challenge? How a novel solution in nurse education can address this contemporary question. Nurse education today 2017; 57: 65–7. pmid:28735247
                                                  • 71. Buunaaisie C, Manyara AM, Hugh A, Bird EL, Bray I, Ige J, et al. Employability and career experiences of international graduates of MSc Public Health: a mixed methods study. Public Health 2018; 160: 62–9.
                                                  • 72. Chastonay P, Vu NV, Humair J-P, Mpinga EK, Bernheim L. Design, implementation and evaluation of a community health training program in an integrated problem-based medical curriculum: a fifteen-year experience at the University of Geneva Faculty of Medicine. Medical education online 2012; 17(16741). pmid:22778541
                                                  • 73. Bennetts C, Elliston K, Maconachie M. Continuing professional development for public health: an andragogical approach. Public Health 2012; 126(6): 541–5. pmid:22633081
                                                  • 74. Gerhardus A, Schilling I, Voss M. Public Health als anwendungsorientiertes Fach und Multidisziplin– „Forschendes Lernen”als Antwort auf die Herausforderungen für Lehren und Lernen? Gesundheitswesen 2017; 79(3): 141–3.
                                                  • 75. Tran A. In-house peer supported literature search training: a public health perspective. Health information and libraries journal 2017; 34(3): 258–62. pmid:30917210
                                                  • 76. Salway S, Piercy H, Chowbey P, Brewins L, Dhoot P. Improving capacity in ethnicity and health research: report of a tailored programme for NHS Public Health practitioners. Primary health care research & development 2013; 14(4): 330–40. pmid:22883709
                                                  • 77. WHO. European Action Plan for Strengthening Public Health Capacities and ServicesWorld Health Organization (WHO) (Ed.). Malta; 2012. Available from: https://iris.who.int/bitstream/handle/10665/340447/WHO-EURO-2012-2219-41974-57687-eng.pdf?sequence=1 [cited 2024 January 23]
                                                    • 78. WHO. The Helsinki Statement on Health in All PoliciesWorld Health Organization (Ed.). Helsinki; 2013. Available from: https://www.who.int/publications/i/item/9789241506908 [cited 2024 January 23]
                                                      • 79. Verschuuren M, van Oers H. Population Health Monitoring. Cham: Springer International Publishing 2019.
                                                        • 80. WHO. WHO-ASPHER Competency Framework for the Public Health Workforce in the European RegionWorld Health Organization (WHO) (Ed.). Kopenhagen; 2020. Available from: https://iris.who.int/bitstream/handle/10665/347866/WHO-EURO-2020-3997-43756-61569-eng.pdf?sequence=1 [cited 2024 January 23]
                                                          • 81. Bornioli A, Evans D, Cotter C. Evaluation of the UK Public Health Skills and Knowledge Framework (PHSKF): implications for international competency frameworks. BMC Public Health 2020; 20(956). pmid:32552689
                                                          • 82. European Centre for Disease Prevention and Control. Core competencies in applied infectious disease epidemiology in Europe; 2022 [cited 2024 January 22] Available from: https://www.ecdc.europa.eu/en/publications-data/core-competencies-applied-infectious-disease-epidemiology-europe.
                                                            • 83. Working Group Teaching in Epidemiology, GMDS, DGEpi (Ed.). Catalogue of Learning Objectives Epidemiology: Basics of Epidemiology. Available from: http://www.epiteaching.org/Catalogue-Learning-Objectives-Epi.pdf [cited 2024 January 23]
                                                              • 84. European Education Area. European Credit Transfer and Accumulation System (ECTS); 2023 [cited 2024 January 22] Available from: https://education.ec.europa.eu/education-levels/higher-education/inclusive-and-connected-higher-education/european-credit-transfer-and-accumulation-system.
                                                                • 85. Sell K, Hommes F, Fischer F, Arnold L. Multi-, Inter-, and Transdisciplinarity within the Public Health Workforce: A Scoping Review to Assess Definitions and Applications of Concepts. Int J Environ Res Public Health 2022; 19(10902). pmid:36078616
                                                                • 86. Gershuni O, Orr JM, Vogel A, Park K, Leider JP, Resnick BA, et al. A Systematic Review on Professional Regulation and Credentialing of Public Health Workforce. Int J Environ Res Public Health 2023; 20(5). pmid:36901111
                                                                • 87. University of Bielefeld. Promotionsstudiengang Doctor of Public Health—Universität Bielefeld; 2022 [cited 2024 January 23] Available from: https://www.uni-bielefeld.de/fakultaeten/gesundheitswissenschaften/studiengaenge/drph/.
                                                                  • 88. Pettenkofer School of Public Health, Munich Medical Research School. Ph.D. Program—Medical Research in Epidemiology & Public Health—IBE—LMU München; 2022 [cited 2024 January 23] Available from: https://www.ibe.med.uni-muenchen.de/phd-studiengang/index.html.
                                                                    • 89. Hecker K, Moritz KB. Ärztemangel–Hamburg geht neue Wege. Gesundheitswesen 2018; 80(04): 390.
                                                                    • 90. Akademie für Öffentliches Gesundheitswesen (AÖGW) (Ed.). Master of Science in Public Health/Öffentliches Gesundheitswesen. Available from: https://www.akademie-oegw.de/aus-und-weiterbildung/master-of-science-in-public-health [cited 2024 January 23]
                                                                      • 91. RWU Hochschule Ravensburg-Weingarten University of Applied Sciences. Master Angewandte Gesundheits-wissenschaft | Studiengänge | RWU Hochschule Ravensburg-Weingarten University of Applied Sciences; 2022 [cited 2024 January 23] Available from: https://www.rwu.de/studieren/studiengaenge/angewandte-gesundheits-wissenschaft.
                                                                        • 92. Schloemer T, Schröder-Bäck P. Criteria for evaluating transferability of health interventions: a systematic review and thematic synthesis. Implement Sci 2018; 13(88). pmid:29941011
                                                                        • 93. APHA. Supporting Research and Evidence-Based Public Health Practice in State and Local Health AgenciesAmerican Public Health Association (Ed.). APHA Policy Statement; 20171; 2017. Available from: https://apha.org/policies-and-advocacy/public-health-policy-statements/policy-database/2018/01/18/supporting-research-and-evidence-based-public-health-practice [cited 2024 January 22]
                                                                          • 94. Kneale D, Rojas-García A, Raine R, Thomas J. The use of evidence in English local public health decision-making: a systematic scoping review. Implement Sci 2017; 12(53). pmid:28427465
                                                                          • 95. Orton L, Lloyd-Williams F, Taylor-Robinson D, O’Flaherty M, Capewell S. The use of research evidence in public health decision making processes: systematic review. PLoS ONE 2011; 6(7). pmid:21818262
                                                                          • 96. Hanney SR, Gonzalez-Block MA, Buxton MJ, Kogan m. The utilisation of health research in policy-making: concepts, examples and methods of assessment. Health Res Policy Syst 2003; 1(2): 1–28. pmid:12646071
                                                                          • 97. Petticrew M, Whitehead M, Macintyre SJ, Graham H, Egan M. Evidence for public health policy on inequalities: 1: the reality according to policymakers. JECH 2004; 58(10): 811–6. pmid:15365104
                                                                          • 98. MacVarish K, Kenefick H, Fidler A, Cohen B, Orellana Y, Todd K. Building Professionalism Through Management Training: New England Public Health Training Center’s Low-Cost, High-Impact Model. J Public Health Manag Pract 2018; 24(5): 479–86. pmid:28991053
                                                                          • 99. Dopson SA, Griffey S, Ghiya N, Laird S, Cyphert A, Iskander J. Structured Mentoring for Workforce Engagement and Professional Development in Public Health Settings. Health Promot Pract 2017; 18(3): 327–31. pmid:28125904
                                                                          • 100. Bogaert K, Castrucci BC, Gould E, Rider N, Whang C, Corcoran E. Top Training Needs of the Governmental Public Health Workforce. J Public Health Manag Pract 2019; 25(2): S. 134-S144. pmid:30720626
                                                                          • 101. Ablah E, Weist EM, McElligott JE, Biesiadecki LA, Gotsch AR, Keck CW, et al. Public health preparedness and response competency model methodology. Am J Disaster Med 2013; 8(1): 49–56. pmid:23716373