Systems thinking and its application to health research

The systems approach, also known as a holistic view or systems thinking, recognises the interconnectedness of various sectors and actors and helps stakeholders understand how influences in one part of the system can impact the entire patient journey. The systems approach is derived from Operations Research (OR), which seeks to understand the totality of the situation (see [18] for a review of the evolution of the field and its focus on optimisation of resource use). Systems thinking as a dimension of OR approaches the issue of optimisation by examining the system in question (e.g., a hospital) through a lens of complexity as well as taking into account uncertainty, prediction, and interdependencies within the system so that critical relationships are understood. Among the underpinning concepts used is interconnectedness: it emphasises the importance not only of drawing together observations to create a synthesis but also of looking at the way in which interconnections are linked through feedback loops to try to identify relationships where causality exists. Different research methods from systems thinking have been deployed to approach healthcare systems, including dynamic modelling, agent-based modelling, causal loop diagrams, and other methods from social science such as social network analysis [19]. Most applied research applications of systems thinking in the health field typically commence with a discussion of the systems thinking theory; then the modelling is applied to a case study to demonstrate innovation or change that would lead to system improvement (see [20]). Among some of the obvious benefits of systems thinking in healthcare is its application to the analysis of workforce dynamics and of the value and effect of fostering collaborative coordination among partners to create joint solutions that mitigate impacts upon patient outcomes and care quality. Moreover, OR supports the optimisation of resource allocation, aiding policymakers in crafting inclusive policies that address shortages or service interruptions throughout the system by helping to recognise where breakpoints or bottlenecks exist. Through engaging stakeholders who are informed by differing paradigms and agendas, a systems approach helps identify how a comprehensive, collaborative, versatile, and flexible approach that cuts across boundaries and insular thinking may help address the myriad of challenges encountered within healthcare systems. However, the systems approach can be complex, and the available software is difficult for non-experts to use. Developing a decision support tool (DST) based on a systems approach not only makes the methodology more appealing but also enhances its practical usability. This enables end-users to interact with the system more effectively, thereby expediting the decision-making process. A DST in healthcare is a software program that aids key opinion leaders (KOLs) in making well-informed decisions by delivering pertinent and timely information. It consolidates data from multiple sources, including patient records and medical research, to provide recommendations, predict outcomes, and propose best practices. These tools help in planning by identifying trends, optimising the use of resources, and enhancing patient care, which leads to more efficient and effective healthcare services.

This article, therefore, contributes to the development of research policies for organisations aiming to advance ICM by utilising a DST that facilitates better decision-making. It does so by building an innovative conceptual framework that uses systems approach principles to help with ICM implementation, using a simulation environment to grow and support integration-care-friendly policies. The purpose of this initiative is to demonstrate how the theory and practice of ICM can be invoked to bring about change and continuous improvements in patient care and outcomes through the use of innovative methodology in the form of a DST. Some of the key questions that need to be asked at the outset about the efficacy of using ICM in a hospital setting are framed in Table 1, which aims to help scope out what systems thinking may hope to achieve with its multidimensional and multidisciplinary approach to helping to improve the management of patient outcomes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Questions on how operational research could achieve enhanced ICM in a hospital setting.

https://doi.org/10.1371/journal.pone.0321994.t001

Table 1 presents a set of questions that serve as a guideline. These questions help the framework address a wide range of inquiries related to the implementation of ICM. Their global relevance lies in their ability to focus attention on the resources utilised—both financial and human—in the pursuit of patient-centred care and improved patient outcomes. To understand the broader system of healthcare delivery and the way ICM can be implemented, it is necessary to use systems theory to create a conceptual framework; this framework will then help to inform decision-making by depicting and quantifying the effects of any proposed changes in healthcare systems. The framework is a way to visualise and depict a complex system in a simplified manner, as many models in social science seek to do, but it differs from them in its adoption of a positivist approach. What makes a systems thinking approach fundamentally different is that this is not an abstract theoretical activity, as the framework is constructed only after detailed stakeholder engagement activities and dialogue have taken place to refine the system and adapt it to the local setting. Systems thinking is a powerful tool which enables stakeholders to have ownership over the visualisation process; it does so by helping to build in the local nuances and human dimension that positivist research methods have often been criticised as lacking.

In this paper, the conceptual framework was supported with a simulation environment that acted as a decision support tool (DST) to enhance retinal services at a prominent UK hospital. A DST is a software-based system that contains patient information from a multitude of sources, including details of individual conditions, previous interventions, and other scientific data that may help with clinical diagnosis and treatment. Alongside the DST is the care pathway, which is the specific continuum the patient follows after [21]. Vissers and Beech [22] identified five specific forms of healthcare planning whereby the care plan was a patient-focused element within the wider planning functions of healthcare management. These were:

• a care plan for each individual patient (patient planning and protocol);
• the planning of care in care pathways (patient group planning and control);
• the capacity planning of professionals, equipment, and space (resource planning and control);
• the planning of the number of patients to be treated and care activities to be carried out (patient volume planning and control), and
• the long-term policy of the institution (strategic planning).

Applying a simulation methodology makes it possible to combine many of these planning functions into a single system. This makes it possible to identify efficiencies and cost savings that could be made within the care pathway by integrating key services like community and primary care. It is also possible to target innovations at each level of planning, although we focus on the strategic level in this paper. Due to its holistic nature, systems thinking was identified in this study as a means by which the most effective and cost-efficient interventions could be identified in the hospital system to address the increasing demand for retinal services. The DST presented in this study constitutes a significant contribution to knowledge and was an innovative health research policy approach that was co-created with the hospital staff. It is significant because it provided key decision makers with evidence to help them prioritise those interventions that could lead to improved patient outcomes. The co-creation process is critical; NHS England [13] noted that system drivers (e.g., quality improvements) were insufficient to create change if they were not accompanied with staff engagement in co-leading the implementation of such innovations. The key communication process in systems thinking concerns illustrating the potential impact of interventions before practical implementation takes place, thereby informing strategic decision-making.

The paper commences with a discussion of the concept of ICM and its recent use to illustrate the theory and practice which it is embedded within. This is followed by a section on the application of systems thinking’s contributions to achieving ICM objectives and its value in policymaking. We then introduce the conceptual framework we adopt for enhancing healthcare policies within the realm of integrated care. Next, we present a real case study, in which we developed a discrete event simulation model in collaboration with a retinal services unit at an NHS trust in England. The paper goes on to discuss the simulation’s results and to draw conclusions from this example as regards the implications for ICM.

ICM and its application in health-care management

Integrated care is a relatively new conceptual framework for healthcare management that has become a feature in several healthcare systems worldwide (see [17]), operating with varying degrees of partnership [23–25], but a formal inception at the national level is a relatively new ideology for the UK (see Appendix 1). The integrated care system (ICS) was formally established in July 2022 in England/UK [26]. This system consists of 42 geographically based partnerships, fostering collaboration among NHS service providers, commissioners, local authorities, and other regional partners. It replaced the existing 106 clinical commissioning groups, a structure which had been criticised for its fragmentary nature. As a new framework for healthcare management, the potential effectiveness of ICSs at the national level remains to be seen [27], but many local success stories highlight their benefits, emphasising how they can be adapted to meet diverse needs. For example, in Derbyshire, the implementation of an integrated neighbourhood team approach resulted in a reduction of urgent ambulance callouts by 2,300 per year and further led to a decrease in hospital stays by 1,400 [28] (also see the NHS website for examples of successful changes https://www.england.nhs.uk/integratedcare/resources/case-studies/). As ICM is in its early stages in England, these ICSs are providing an important laboratory for research into how best to transition towards full integration. This process will require active participation from all levels of stakeholders and is described by NHS England [13] as pivoting upon motivating colleagues to embrace innovations, and implementing change management measures by means of shared and change leadership. As Thune and Mira [11] indicated, transitioning towards ICM requires involving all stakeholders in the process of designing, managing, implementing, and evaluating initiatives. Deriving empirical evidence of successful implementation and quantitative evidence of national outcomes whilst encouraged by WHO and OECD, it remains limited. As the evidence base to benchmark enhancements in patient-centred care are notably absent, van Harten [59] identified one example of breast cancer care and the qualitative changes in patient-centred care as well as the challenge of making comparative assessments. Zonneveld et al.’s [60] systematic review of the field reiterated these points, highlighting the values that were associated with behaviour change to achieve ICM. Clearly, this endeavour poses its own set of challenges. As already highlighted (e.g., [17]), organisational innovation is critical to integrate the wide range of stakeholders around a single model of delivery where diverse services and partnerships can be better coordinated by professionals and local partners who have a single goal - integration-friendly care that creates better health outcomes for the local population. Against that background, systems thinking has considerable potential to communicate the innovation proposed to a wide range of stakeholders and publics. It can do so by visualising and explaining the weaknesses in the current system in order to shift practice from a structure based on a complex amalgam of units and divisions towards a more harmonised model of delivery.

It is important to distinguish between the macro health policy objectives of government to improve citizen healthcare and the pragmatic requirements of local delivery, which are often focused on anchor institutions like hospitals. Whilst there is perennial criticism of the frequent policy changes at national level that have been made to try and address demand and supply issues for public healthcare across England (and the devolved nations), the most persistent criticisms of the NHS often stem from the fact that it consumes 20% of public spending to meet those needs [29]. Frequent policy changes have left local care in a constant state of flux over the last 20 years, as different governments have sought to reinvent the NHS and its interface with its publics. The current ideology of ICM has placed a considerable onus on hospitals to reorganise their delivery of services so as to address local concerns. These concerns typically revolve around historical criticisms and often unfounded assumptions about the bureaucracy and lack of joined-up thinking in the NHS [29]. Consequently, the national policy of pursuing ICM is the latest in a series of reforms that have impacted the NHS over its 77-year history, leaving frontline staff and managers to cope with constant changes over that period. Ideologically, ICM is premised on the notion that hospital services, as one facet of NHS delivery, are disconnected complex systems, and that this creates inefficiencies associated with poor resource use. As Wickens’s [29] study illustrates, criticisms of NHS delivery are not necessarily borne out in practice, and delivery issues are not unique to the UK [30]. However, the new realities of ICS now mean that health managers need to embrace change once again and adopt a new perspective on healthcare provision. That means those managers need to extend their purview beyond the confines of individual hospitals. ICSs must extend their modelling beyond hospital boundaries to encompass workforce needs, making sure the community dynamics, demographics, service demands, health and social care providers, mental health, and other services are combined in it.

Systems thinking and the application to ICM

With the shift towards ICM and growing interest in how health systems operate, systems thinking has been widely used in healthcare as a method of real-world problem-solving (see Peters [31] for a review of the development of systems thinking, the theories underpinning its development, and the methods of analysis it deploys). As a tool designed to understand complexity in large systems like a hospital, it has great potential to help people understand how different components in a system can impact one another and how they are all related. This holistic approach contrasts with traditional linear problem-solving methods, offering deeper insights into system dynamics and interdependencies.

As a scientific research method, it has been adopted outside of academia to assist with policymaking. As Slater [32] explains in relation to the UK Civil Service:

“Most policy problems occur within a complex system that is constantly changing with levels of uncertainty and a variety of cause-and-effect loops. Systems have multiple elements, interconnections, stakeholders and drivers, and stakeholders’ views often do not align. Sometimes even defining the policy problem can be complex and hard to understand, let alone exploring possible solutions to that problem. Rather than understanding a system as the sum of its individual parts, systems thinking seeks to view every part as an element of the whole and it’s the connections between these elements that are critical.”

Yet studies, such as Kwamie et al. [33] (p. 1715), have commented that policymakers have not fully recognised the importance of systems thinking. This oversight may stem from the inherent complexity and the required shift in mindset from viewing problems in isolation to considering them as part of a larger system. Peters [31], however, highlighted its wider applications in healthcare research. It would be suited to this field because of the very large complex organisations involved and the nature of the systems that have been created to manage public health from the primary care stage, e.g., Tako et al. [34] through to hospital systems. Simulation is a widely utilised approach within systems thinking, as it enables the system to be modelled and then experimented with to create different outcomes by using mathematical algorithms. Among the most notable methods used in healthcare research are discrete event simulation (DES), agent-based simulation (ABS), and system dynamics (SD), as well as hybrid variations of these techniques. The selection of methodology is heavily influenced by the objectives of the study [35]. For instance, when the study aims to analyse operational metrics at discrete time intervals (such as daily resource utilisation), DES is typically preferred [36]. Conversely, when evaluating actions (such as changes in the behaviour of medical staff and patients), ABS is more suitable [37], whereas SD is suited to strategic-level analysis regarding the causality and effects of relationships [38]. A comparison of the simulation methods is provided in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2.. Comparison of simulation methods.

https://doi.org/10.1371/journal.pone.0321994.t002

DES is one of the most widely used system tools in a healthcare setting, as Zhang’s [39] review documented. What is apparent from the wide application of DES in healthcare settings is the tendency for its use to focus on specific forms of treatment (e.g., Jahn et al. [40]), whether these be individual or multiple units (e.g., Huynh et al. [41]; Pendharkar et al. [42]), and its use as a means of examining how increased capacity and investment might improve waiting times. Other studies have included an analysis of a system of clinics offered in a particular hospital (e.g., Jun [43]) and examples of how patient flow can be improved through such systems (e.g., Bard et al. [44]; Santos et al. [45]). However, a notable feature of the majority of the early studies of DES in healthcare settings is the reluctance to model whole hospital systems, an idea in which the concept of ICM is deeply embedded. However, a notable feature of early DES studies in healthcare is the reluctance to model entire hospital systems, where the concept of ICM is deeply embedded.

While these studies provide valuable insights, there is a need for a more critical examination of the assumptions and limitations inherent in these models. For instance, many DES applications focus narrowly on operational efficiency without fully addressing patient outcomes or systemic issues such as healthcare equity and accessibility. Additionally, the literature often highlights the technical aspects of simulation without sufficient consideration of the integration of care in which these healthcare systems operate.

System modelling

The process of system analysis and modelling commences with the diagnosis of issues and the establishment of objectives to be pursued. Moreover, prior to conducting simulations, interventions or scenarios intended for testing must be clearly defined to allow seamless integration into the model. Additionally, the critical performance indicators expected from the simulation outcomes are identified during this phase. Depending on the project’s complexity and overarching goals, the selection of an appropriate modelling or simulation methodology becomes paramount: possible options include DES, SD, ABS, and Monte Carlo simulation.

Qualitative mapping, also called conceptualisation, is the most important task in systems thinking and modelling. The task includes capturing and verifying patient flow across the providers (at the specialty level) and establishing key differences and features in a health system. Therefore, several meetings were organised with key stakeholders; this group consisted of key decision makers, the management team (hospital/specialty level), and key healthcare professionals. A focus group methodology was used to capture the free-flowing discussion around the visualisation of the system and its components and interdependencies.

To facilitate the assessment of healthcare policies pertaining to integrated care, we formulated a comprehensive conceptual framework. This framework encompasses primary care, secondary care, specialist services, mental health provisions, community services, involvement of local authorities, and engagement with other potential stakeholders. It is imperative to emphasise that our conceptual framework remains adaptable to the specific objectives and challenges encountered in each study. For instance, one study might require adding mental health services alongside hospitals, whilst another might prioritise community services working in conjunction with hospitals. Therefore, the framework can be customised to suit the specific needs and goals of each study, accommodating various simulation or modelling methodologies, and ensuring its applicability to diverse integrated care models.

Firstly, we present an example structure of an ICS in Fig 1. An ICS covers various catchment areas (called NHS trusts in the UK). Each catchment area hosts a variety of health and social care organisations, such as primary care, hospitals (including specialties), mental health services, and community services, as originally envisaged in the formation of the NHS. Secondly, Fig 2 illustrates the potential flow of patients through healthcare services within a catchment area, along with the structure of the key services available. The patient flow comprises healthcare services, both inpatient and outpatient, as well as primary care, community services, mental health facilities, and others.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. An example structure of an integrated care system.

PC: Primary Care, H: Hospital, MH: Mental Health, CS: Community Services, O: Other.

https://doi.org/10.1371/journal.pone.0321994.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Patient flow in healthcare services in a catchment area and the service structure.

GP: General Practitioner, DNAs: Did Not Attends, WL: Waiting List, C: Consultation, D: Diagnostics, Tr: Treatment, B: Beds, Th: Theatre, ED: Emergency Department.

https://doi.org/10.1371/journal.pone.0321994.g002

Patients who are referred to inpatient and outpatient care wait for appointments within the community. Consultation, diagnostics, and treatment are processes in outpatient services; patients use them and are discharged the same day (i.e., no overnight stay is required) with the possibility of a follow-up appointment (if necessary). In inpatient services, which include diagnostics and surgeries, patients are often required to remain hospitalised (i.e., overnight stay); this can last up to several days, except for day-case admissions. Unplanned admissions to inpatient care may occur through direct referrals from emergency departments (EDs) or other healthcare providers. Depending on the specific research objective and case study, the scope of EDs can be expanded as needed.

Patients receive examination and treatment in primary care settings, such as GPs’ or optometry clinics. Community services comprise a wide area, including adult and children’s services, various therapies (e.g., speech and language), and rehabilitation services (e.g., pulmonary or cardiac rehabilitation). Mental health services and outpatient care (e.g., physiotherapy) are also integral components. The seamless integration of these services is imperative, as they play a central role in delivering care, support, and preventive measures across health, social, and mental health domains. Without their inclusion, any attempt at an integrated care model would be incomplete. It is important to note that the patient flow in the real world is highly complex and transitions may take place between any combination of these services/departments. Thus, the conceptualised flow diagram is generic and adaptable to any setting, country, or integrated care service.

Data collection and analysis

Conducting meticulous data collection and analysis is imperative for projects of this magnitude, particularly given the complexity of ICS. Such endeavours demand a substantial investment of time and expertise due to the multifaceted nature of services which operate at an ICS level. Data for analysis may be sourced from national and/or hospital databases, published literature, and expert insights. The granularity of data can vary, encompassing patient-level or activity-level information, contingent upon the modelling methodology and scope of the study. Key input parameters typically include demand, capacity, referrals, wait times, treatment durations, length of stay (LoS), patient demographics, rates, cost-revenue considerations, and routing mechanisms. Statistical analysis is undertaken to examine the distribution of data (e.g., length of stay is a critical driver in the utilisation of resources) to inform modelling and simulation processes, with demand forecasting being necessary in certain instances.

Model development

The subsequent step involves translating the flow or relationship diagram into a computerised environment, integrating established inputs derived from comprehensive data collection and analysis. The simulation may be developed utilising off-the-shelf commercial software (e.g., Simul8, Vensim, AnyLogic) or custom-coded through advanced programming languages, such as JAVA, Python, or R. As previously highlighted, the framework remains neutral towards specific simulation methodologies, refraining from endorsing any single approach. It is imperative to reiterate that the selection of the modelling method should be made at the outset, depending on the problem and the objectives of the study during the system modelling phase, as the subsequent mapping and data collection processes are intricately shaped by this initial decision.

This article presents a decision support tool (DST) specifically designed for retinal services. The tool’s main objective is to evaluate the impact of interventions and policies at the operational level, for example, in terms of their impact on resource utilisation within the ICM framework. To achieve this, DES emerges as the preferred methodology. DES enables modelling of hospital operations and healthcare services, which in turn provides insights into patient pathways within the system and generating operational metrics for scenario analysis [47]. Therefore, DES was selected as the modelling approach on the grounds that it was the most suitable method for the case study.

It is important to reiterate that the conceptual framework is also usable with any other simulation method; the choice of method should depend on the project objective and focus. For example, if we wanted to study how the entire retinal services system changes over time (i.e., at the strategic level), including factors like improved vision and patient satisfaction, we would choose system dynamics simulation. However, in this case, we are focusing on specific events like treatments and resource usage in retinal services at discrete time points (at the operational level), rather than the overall system dynamics and feedback loops.

Validation and verification

The model or simulation that is developed must undergo rigorous scrutiny to ensure its accuracy and reliability, a process commonly referred to as validation and verification. This entails scrutinising the logic of the model, as well as input generation to ensure proper functionality. The generated outputs are then meticulously compared against real-world data. Additionally, each component of the model is examined to confirm it accurately reflects the flow diagram. Crucially, key stakeholders within the system, who initially took part in the mapping process, are then asked to participate in validating whether the model accurately portrays the system’s dynamics, in order to help co-create a robust real-world situation. Subsequent statistical analysis is conducted to determine the appropriate warm-up period and number of replications required for simulation runs [48]. Once a validated and verified model is obtained, it can be utilised to conduct testing experiments and evaluate various scenarios. The model’s outputs are then analysed to assess the potential impact of changes, guiding decision-making based on the analysis.

Case study: Application of the model to an NHS hospital

The incessant strain on NHS services in the UK is a recognised feature that has severely impacted patient care and access, especially since the COVID-19 pandemic, as services grapple with a backlog on top of the existing demand. Evidence from September 2023 statistics reveals a staggering waiting list of 7.77 million patients: once patients awaiting more than one type of procedure have been accounted for, this amounts to roughly 6.5 million individual patients, with nearly 3.29 million of them enduring waits exceeding 18 weeks [49]. The second most affected service is ophthalmology: one in every 11 patients on the NHS waiting list, and almost 628,502 individuals, are awaiting these services [50]. Within ophthalmology, retinal services face the highest demand. This department treats conditions like age-related macular degeneration and diabetic retinopathy (DR), predominantly among the elderly population, through treatments like laser therapy or intraocular injections. Based on historical data from the NHS trust in this study, there has been a significant rise in DR follow-up attendances. Forecasts indicate this trend will persist, which will require retinal services to operate beyond 100% capacity. This situation leaves no space for new patients, and consequently, the waiting list and treatment times will be extended. The resulting delays in treatment could potentially lead to vision loss.

System modelling in the case study.

To showcase the use of the conceptual framework to improve the system, we focused on retinal services as a case study. Semi-structured interviews and focus group meetings were carried out with specialist nurses from retinal services and managers from an ICS. The interviewees’ insights and inputs allowed us to map the patient flow in the service in detail as well as the structure of the ICS. Also, inputs such as resource, capacity, and patient routing were collected via the interviews. The patient flow was verified by the nurses from the services.

In collaboration with the trust and using the conceptual framework as a guide, we developed a DST for the entire patient pathway of retinal services. This tool integrates the primary care, hospital, and community services responsible for comprehensive care for individuals with eye-related conditions. The goal was to identify the most effective and efficient interventions to alleviate the pressures on retinal services.

Based on the conceptual framework, a diagram of retinal services, showing the structure of services and the hierarchy, is represented in Fig 3. It consists of two layers. A high-level depiction of retinal services and its hierarchy under an ICS is shown in part (a) of Fig 3. The patient flow for retinal services was conceptualised by the trust’s stakeholders, including the service manager and nurses. Part (b) of the diagram shows diagnosis, treatment, and consultation, with the possible options listed under each stage.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Conceptual diagram of retinal services.

a) High-level depiction of retinal services and its hierarchy under an integrated care system; b) Patient flow in retinal services at trust level.

https://doi.org/10.1371/journal.pone.0321994.g003

Fig 3. and other parts of the DST are established through a co-creation process, which involved extensive collaboration with key stakeholders from the Trust’s retinal services, including consultant ophthalmologists, specialist nurses, and the service manager. This engagement was structured through three comprehensive rounds of semi-structured interviews and focus groups, each serving a distinct purpose in the model development process. University ethical approval was obtained for this study. For all interviews and workshops, formal participant consent was obtained from healthcare professionals, with clear information provided about data usage, storage protocols, and confidentiality measures in accordance with institutional requirements.

The first round focused on understanding and mapping the current patient pathways. Through semi-structured interviews, we explored:

• What is the complete patient journey from referral to discharge?
• What are the key decision points throughout the pathway?
• What resources (human and equipment) are required at each stage?
• What are the typical treatment times for different procedures?
• How do patient flows vary between different conditions and treatments?

The second round concentrated on verification and detailed data collection. Stakeholders were presented with initial pathway mappings and asked to verify and enhance them. Key questions included:

• Are these pathway representations accurate and complete?
• What specific resources are consumed at each touch-point?
• What variations exist in treatment times between different patient types?
• What factors influence clinical decision-making at each stage?
• What are the current bottlenecks and resource constraints?

The final round focused on model validation and scenario development. Stakeholders were presented with the complete pathway mapping and preliminary Simul8 model. Questions explored:

• Does this model accurately represent your service operations?
• Are there any missing elements or inaccuracies in the model?
• What scenarios would be most valuable to test?
• What metrics would be most useful for evaluating different scenarios?

Each round of stakeholder engagement led to specific refinements in the model. The iterative nature of this process ensured that the final simulation model accurately represented the real-world operation of the retinal service. All stakeholder interactions were thoroughly documented through meeting minutes, pathway diagrams, and data collection sheets, ensuring transparency and traceability in the model development process. This structured approach to stakeholder engagement was crucial in developing a model that not only accurately represented the current service but also enabled meaningful scenario testing for service improvement.

Patients are mostly referred to these services via optometrists and GPs. After the initial diagnosis and treatment, patients either attend for the remaining treatments (e.g., injections) or a follow-up review (monitoring/assessment). Treatment and follow-up patterns vary depending on patient type and the type of condition: namely, age-related macular degeneration (AMD), diabetic retinopathy (DR), or retinal vein occlusion (RVO) conditions. Re-bookings can be made for patients who cancelled or ‘did not attend’ (DNA) appointments.

Data collection and analysis in the case study.

To run the simulation model for the case study, we collected various data which was mainly patient level. The main data sources were the hospital, expert views and published literature/reports. The data obtained from the hospital were from a period of 36 months. The process involved comprehensive data collection and analysis due to the large size and complexity of the dataset. Working closely with the hospital’s health informatics team, we extracted relevant data from various services, including inpatient, outpatient, and emergency services. This data was used to estimate various input parameters, establish statistical distributions, and forecast future demand. Also, several statistical distributions were set to capture the variation in real life for factors such as length of stay and waiting time.

The full list of the input parameters that are used in the model is provided in Table 3 along with the source. The inputs are related to demand, human and non-human resources (such as ophthalmology consultants, nurses, and diagnostic equipment), routing rates, and treatment rates, as well as costs and revenues.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Input parameters.

https://doi.org/10.1371/journal.pone.0321994.t003