Background

With an increasing burden of multimorbidity and higher demands for care under constrained health system resources, remote patient monitoring (RPM) has garnered interest in its potential to enhance healthcare access and coordination of care facilitated at a distance [1,2]. They are valued for enabling monitoring of a patient’s clinical conditions outside of in-person or acute care settings, facilitating timely detection and responsive clinical action at the deterioration of illness [3]. Health system shocks in the wake of the COVID-19 pandemic resulted in growing demands on a reduced workforce, and patients with complex needs required more hands-on support and wraparound care [4]. Preference has also increased for people to age at home or in their communities [5–9]. Yet if not properly managed, there is a risk that frequent hospitalizations and emergency department (ED) visits will continue to be the norm [10–12]. Digital health solutions may be well positioned to support management of chronic conditions with improved clinical outcomes [13,14]. However, not everyone who desires to be monitored remotely has access to these technologies and attention must be paid to bridge the digital divide and inequitable access to care [15–17]. For example, data shows that 41% of 12,052 surveyed Canadians (20% of whom are 65 + years) have an unmet demand for RPM programs and services to manage chronic conditions [18].

Traditionally, RPM has been designed and implemented with a condition-centric approach (i.e., to support specific clinical needs such as surgical transitions, diabetes, heart failure) separately, rather than as a model of care that centers the patient throughout their care needs. As such, limited attention has been given to variations in technological dependence, clinical workflows, patient populations, and resources - making it difficult to (i) understand which programs are effective and why, (ii) assess their stage of technological implementation or maturity, and (iii) identify key features that characterise good RPM solutions. A realist review that aimed to identify the factors that may influence effectiveness of RPM interventions presented six theories that can inform strategies to ensure the right kind of program model for different targeted populations [19]. These include targeting population focus, responsiveness of escalation parameters, enhancing self-management, and strengthening coordinated care. Evidence from systematic reviews show widely varying outcomes from RPM programs which has been explained as being due to variations in patient selection, clinical models, implementation strategies, and features of the technology [20,21]. When compared to usual care, a recently published pooled analysis of 58 chronic disease RPM programs showed reduced mortality and improved physiologic measures, but higher risk of hospitalization [22]. However, the study did not establish if these were avoidable hospitalizations, since RPM is intended to result in just-in-time interventions that prevent mortality. Furthermore, although digital health innovations are touted as being able to equitably improve access to care, the COVID-19 pandemic revealed that technological innovations in healthcare may perpetuate or create inequities [23,24]. In particular, communities who are structurally marginalized by age, race, gender, geography, disability, amongst others, are likely to experience an increased digital divide due to intervention-generated inequalities [25]. Similarly, RPM programs can also inherently restrict access due to equity related factors [17,26].

Between 2021 and 2022, the provincial health bodies in Ontario, Canada - the Ontario Ministry of Health (MOH) and Ontario Health (OH), funded the implementation of more than 40 RPM programs across the province with diverse clinical use cases including for COVID-19 management, chronic conditions, alternate levels of care, and surgical transitions. These were implemented by multiple healthcare organizations using different technology vendors. To understand which programs were better positioned to scale and were on track to advance the quintuple aim of healthcare (i.e., cost effectiveness, positive health outcomes, good patient and provider experiences, and equity [27]), the MOH and OH sought to evaluate this heterogenous group of RPM programs. While such an evaluation could guide future planning and investments in digitally enabled healthcare and inform knowledge on the impact of RPM, the heterogeneity of the programs – a situation not unique to the Ontario cohort, posed an initial challenge. To ensure a like-to-like comparison of programs despite varying clinical pathways, implementation timeline, enrolled users, etc. the body of evidence and decision-making on RPM needed a common point of reference. Recognising this challenge and in the absence of an existing solution, a research team at the Centre for Digital Health Evaluation (CDHE), Women’s College Hospital Institute for Health System Solutions and Virtual Care (WIHV) decided to develop a novel typology of RPM programs.

The rationale for developing a typology is that a disease-agnostic categorisation of RPM that goes beyond clinical use cases and spans the full spectrum of care - from self-management to active management - can advance thinking around the design, implementation, and delivery of RPM programs, and address the high variation in effectiveness evaluations. In this paper, we describe the development of such a first-of-its-kind tool which provides a systematic way to assess, compare, and potentially guide the implementation and evaluation of remote care interventions.

Study design

Two rapid reviews of literature on RPM programs supplemented by expert consultation from clinicians, health system implementers, and patient evaluators constituted our research methodology. This approach aligns with the knowledge creation phase of the Knowledge to Action (KTA) Framework [28], which focuses on producing new or synthesizing existing knowledge in a way that is practical, relevant, and accessible for real-world implementation. A similar methodological approach has been used to develop other program typologies; for example, on service models for inpatient mental health [29], allowing users to categorise healthcare programs in a standardized way while capturing variation. The KTA knowledge creation phase comprises three main stages which are mapped below to the steps we followed in developing the RPM typology: knowledge inquiry (Stage 1), knowledge synthesis (Stage 2), and the creation of knowledge tools and products (Stage 3). For distinction between the approach and the outcome, we report stages 1 and 2 under the methods section and stage 3 in the results section of this paper.

Stage 1. Knowledge inquiry: Rapid review of literature

Driven by the need identified by OH and MOH, the aim of Stage 1 was to establish the knowledge gap and scope of inquiry by reviewing documented RPM programs from the literature. This was preceded by a non-systematic scan of documents on the funded RPM programs in the Ontario context which described to varying levels of detail their programs and features. This activity allowed us to familiarise ourselves with the variety of features that could be presented in real-world programs before systematically reviewing the literature and identifying characteristics relevant for classifying programs. Next, we carried out two rapid literature reviews in 2022 – one focusing on RPM programs for chronic diseases, and the other on COVID-19 (see S1 Table).

Chronic disease literature review.

The rapid review on RPM for chronic diseases used search criteria with variations of remote monitoring, and included related and overlapping terms like telemonitoring, telemedicine, mHealth, apps, eHealth, and virtual care. The articles were limited to four chronic conditions, specifically chronic obstructive pulmonary disease (COPD), heart failure (HF), Diabetes (type 1 and 2, gestational diabetes) and hypertension - conditions where RPM has been most widely used. Articles were limited to Canadian and International peer-reviewed articles published between 2017–2022 and in the English language. The search strategy was applied to Embase and Medline. From an initial 27,248 identified articles, 200 were included (Fig 1)

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. PRISMA diagram for combined chronic disease and COVID-19 literature reviews.

A final 87 publications were included for sense-making to define characteristics common across RPM programs, which were subsequently reviewed by an expert panel.

https://doi.org/10.1371/journal.pdig.0001595.g001

Stage 2. Knowledge synthesis: Information mapping and sensemaking

From the 87 articles identified in the two rapid reviews (stage 1) the research team held meetings to discuss a random selection of 10 articles, going over the program features reported in each paper, specifically those unrelated to the clinical use case of the RPM program. We created a running list of these characteristics through an information mapping exercise, identifying up to 20 information categories by which RPM programs were reported (see Table 1). To extract data from each article into an Excel spreadsheet against these categories, there was a need for the research team to establish two things: (i) the meaning of each category such that it could be easily communicated and uniformly understood, and (ii) the different ways in which information on each category could be reported across RPM programs, such that there was a limited range of options to pick from. For example, to avoid an unmanageable number of choices for the category ‘Program size’ we defined 5 sub-categories- small, medium, large, scaled-up, and unknown, each with its own definition. Establishing these early on ensured coherent interpretation and systematic extraction during the information mapping process. In addition to biweekly meetings between the research team and MOH and OH program teams, we also convened a 6-person expert panel comprising three patient partner evaluators (who have experience with chronic diseases, health service delivery or RPM programs) and three physicians who are experts in virtual care. Collectively these advisors contributed to the sensemaking exercise by providing feedback on definitions, relevance and clarity of each characteristic, and its applicability for classifying diverse RPM programs. The patient partner evaluators were particularly invaluable in ensuring every definition was clear and easy to understand (commonly dubbed ‘lay language’) to someone unfamiliar with the technical details of how RPM programs are designed or function. These series of meetings with the expert panel, resulted in the development of a dictionary or glossary that captured all definitions and sub-options for each RPM category. Somewhat like a “code book”, this was iteratively refined throughout the project as we extracted more data, expanding our RPM vocabulary and encountering new pieces of information from articles. Where information about a specific characteristic was unavailable, we labelled it as ‘unknown’.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Initial list of information categories on RPM programs and those that were subsequently excluded.

https://doi.org/10.1371/journal.pdig.0001595.t001

Following the first round of definitions and sub-options for each category, up to 10 new articles were independently piloted for data extraction into a Macros-enabled Excel sheet by two research team members. The team then met to discuss uniformity and agreed on the approach for systematically mapping and extracting the rest of the articles. Using a macros-enabled workbook provided the functionality of using Visual Basic for Applications code- a programming language, to track our extracted data, build custom functions into the spreadsheet and configure the envisioned typology with automation features. Quality assurance processes were implemented such that extracted data from up to 40% of the articles were vetted by at least one other individual. During the extraction stage, we progressively excluded six information categories that were identified by group consensus as not relevant for standardizing RPM classification, as being ambiguous, or for being either too narrow or too wide in the range of options they encompassed, narrowing the number of categories from 20 to a final list of 12 (see Tables 1 and 2). The final list of categories intentionally makes no assumptions about scalability (i.e., ability to spread or support a particular number of patients served). Additionally, concepts pertaining to value and effectiveness were deemed out of scope due to a lack of empirical evidence to justify classification against these parameters. The draft version of the glossary outlining the 12 characteristics and completed extraction sheet were presented to the expert panel and other health system partners/RPM stakeholders in a meeting where the research team facilitated discussions on what a user-friendly tool for grouping RPM programs at a higher level, starting from the 12 identified characteristics and their associated subcategories could look like.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Final list of 12 disease-agnostic characteristics of RPM programs.

https://doi.org/10.1371/journal.pdig.0001595.t002

Defining and differentiating four domains of an RPM Typology

Building up from knowledge of implemented provincial RPM programs, expert feedback, and data from the extracted articles, and informed by project objectives, we clustered the 12 disease-agnostic characteristics of RPM programs into four domains, each defined below.

Technology - the level of automation and technological complexity of a given RPM program. It includes characteristics of manual data entry (e.g., patient reported outcome measures) and its frequency, the data entry modality (e.g., fully automated, semi-automated, manual for physiologic measurements), whether an alert protocol exists (e.g., triaging or flagging patients requiring follow-up from the clinical team), and data access (e.g., degree to which patients have access to their data).

Touch - the level of monitoring and interaction between patients and the clinical team. It includes characteristics related to the cadre of staff and level of clinical specialization of staff on the RPM team (e.g., less specialized nurse practitioners versus more specialized staff such as a cardiologist), how follow-up communication is facilitated, availability of the RPM team (e.g., 24/7 versus during usual business hours), and patients’ risk profile.

Integration - the extent to which the RPM program is linked to or leverages existing systems such as established clinical workflows, data interoperability, human resources, and vendor interconnectedness. It includes characteristics of device linkages (e.g., platform vendors, cloud services), and integration considerations (e.g., existing services and resources, workflows, systems, and infrastructure for patient records).

Equity - the extent to which the RPM program proactively enables inclusion, access, and patient-centricity. It includes the nature of device ownership and equity considerations (e.g., language, digital literacy, enabling offline functionality, cultural adaptations, provision of hardware/software and devices, data ownership and access). The domains are illustrated in Fig 2.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. The four domains of the RPM Typology and their related characteristics.

The 12 characteristics were defined through sense-making informed by two rapid literature reviews and feedback from an expert panel.

https://doi.org/10.1371/journal.pdig.0001595.g002

To allow for differentiation within the four domains, we considered that each could be characterized as high or low. The domains Technology (called Tech for short) and Touch were called Groups and discussed as representative of expected variability of RPM program features with characteristics that exist on a sliding scale from low to high. The rationale for this is that the appropriate mix of technological complexity and monitoring intensity will likely be guided by pragmatic decisions related to budget, workforce availability, market competition with vendor options, program maturity, the population served, clinical pathway, available resources, etc. High-low combinations of these two domains were thus assumed to be equivalent (i.e., high tech is not considered to be superior to low tech RPM programs). We assigned alphabets to denote the four possible group combinations for Tech and Touch respectively - group A (high:high), group B (low:high), group C (high:low), and group D (low:low). See Fig 3.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Overview of 16 distinct RPM typologies combining types and groups of four domains.

A given RPM program can be mapped onto one of the 16 distinct typologies based on the program’s combination of characteristics. The greyed-out cells are to be ignored when reading the matrix.

https://doi.org/10.1371/journal.pdig.0001595.g003

In contrast, the other two domains - Integration and Equity (also known as patient-centricity), which together we called Types, were agreed to represent inherent aspirations of all RPM programs. As such they were seen as program features that should be less prone to variation and more representative of a standard of aspiration. That is, all RPM programs should ideally be high integration and high equity. By following this approach in our distinction, we automatically created hierarchies for integration and equity – from low (Type 4) to high (Type 1). However, we assumed equivalence for mixed high and low combinations between Integration and Equity. That is, a high integration, low equity classification (Type 2) is equivalent to a low integration, high equity (Type 3) one. Finally, by combining the four groups with the four types we produced an alphanumeric categorization of RPM programs allowing users to categorize them as any of 16 distinct typologies (e.g., 1A, 4D, 2C, 3B, etc.). For example, a RPM program with typology 2B means it is characterized as low tech:high touch:high integration:low equity (see Fig 3). This can alternatively be written as ↓Tech ↑Touch ↑Intg ↓Eqty. If a RPM program is categorized as low tech:low touch:high integration:high equity (i.e., ↓ Tech ↓Touch ↑Intg ↑Eqty), it means it has a typology of 1D, and so on.

Stage 3. Creation of knowledge products: Developing a survey and automated tool

Developing an RPM typology survey and automated typology categorization tool.

Using the defined typology characteristics from Stage 2, we created a typology survey (aka Typology Checklist) built into Microsoft Forms, designed to be completed electronically by program administrators (with the option to print out and complete a physical copy) and with questions that aligned with the information needed for Table 2. The Checklist can be completed within 10 minutes by those familiar with the RPM program in question to collect detailed semi-structured data about the features and characteristics needed to establish the categorization of the RPM program. The Typology Checklist contains 19 questions with multiple options to select from and includes an open-ended text box for additional comments and notes.

To map information systematically and reliably following completion of the Checklist by a RPM program administrator, we developed a detailed and user-friendly RPM typification tool, in Microsoft Excel, leveraging its developer feature which activates in-built advanced automation and interactive functionality. The final product automatically categorizes, in real time, any RPM program using a simple algorithm that responds to information about the program’s unique characteristics (Table 3).

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Snapshot of RPM Typology Tool.

https://doi.org/10.1371/journal.pdig.0001595.t003

The algorithm consists of two interim calculations: one at the characteristic level (n = 12), and one at the domain level (n = 4). For the characteristic-level calculation, each characteristic is rated as being high (score: 3), moderate (score: 2), or low (score: 0 or 1). The domain-level calculation then computes the average across characteristic-level interim scores. When the domain-level score output is ≥ 2, that domain is automatically categorized as “high”, while domain-level scores < 2 are categorized as “low” for the respective domain. It is important to note that these interim calculations are used solely for the purposes of categorization and higher scores have no value implications- i.e., they do not imply “better” or “worse” RPM programs.

The full version of the typification tool is a multiple-tab Excel spreadsheet that includes a detailed user guide, typology glossary, user instructions, examples of six different RPM programs that have been typified and blank templates that could be duplicated and used to classify multiple RPM programs, with outputs visible in real time as selections of program features are made from pre-programmed drop-down functions in different cells. To download the RPM Typology Tool and Checklist please visit https://forms.gle/ofxSNKQMewo62uoj7

Benefit for health system payors- informing selection of technology vendors

Returning to the rationale for developing a disease-agnostic classification model for RPM, this typology can support evaluation of outcome data of RPM programs through contextualizing and characterizing different programs similarly. The problem our study aimed to solve is the current inability to compare between or across disease-focused programs due to high variability resulting from a solely clinical focus for describing RPM programs. Using this novel typology addresses this problem- four programs that are similarly classified as 4C can be reasonably compared to another group of programs that are similarly classified as 1D, even when each program serves different clinical groups or socio demographic populations.

This characterization allows for clustering of different types of RPM programs to compare “apples to apples” given the high variability and variety of different types of RPM programs within and across disease domains. A shortlist of vendors of record can assist organizations and health teams with selecting an approved and verified vendor to suit their program needs. For example, type 2A and 2B RPM programs vary only by the level of technology. If effectiveness evidence is compared and the low-tech options have comparable effectiveness, this could considerably reduce the cost of adopting a similar program (either within the same clinical domain or applied to a different one). Similarly, if a hypothetical new RPM program is in the planning and preparation stage, comparing effectiveness across 1A and 1C typologies, which differ only by high vs low touch, can inform optimised approaches to balance cost and time with anticipated effectiveness.

Future directions and limitations

The RPM typology as a tool to classify diverse RPM programs is primarily based on evidence from an environmental scan of international programs in the chronic disease and COVID-19 domains at various stages of implementation (e.g., pre-pilot, pilot, fully integrated, etc.). Sensemaking that informed which characteristics and domains were prioritized was thus influenced by the limits of information provided within journal articles. It is possible to include additional domains of interest that were not captured in the literature but might be relevant to decision-makers and innovators. For example, although we recognized its importance, scalability, such as the spread (number of sites) and caseload (number of patients served), was not sufficiently captured in the literature and thus is not included in the current version of the typology. Similarly, the current tool did not consider program maturity or implementation context, which could inform additional layers of granularity to how each typology category is understood. The aspiration of high integration and high equity models may therefore not necessarily be optimal depending on the context and environment. Notably the literature knowledge base which informed the typology focused on OECD countries, resulting in underrepresentation of low- and middle-income countries. The implications of these limitations are linked to critiques of knowledge to action models like the KTA, wherein community and non-formalised knowledge are deprioritised and there is limited utility in engaging with social justice and equity [41]. Furthermore, the current typology has not yet been validated and the scoring system used assumes equal weighting for all characteristics. The immediate next step is to conduct validity testing of this tool. A future cluster analysis would be beneficial for comparing efficacy across programs with different clinical pathways to further inform baseline features for different typologies and to validate the assumption regarding integration and equity features existing on a hierarchy from low (Type 4) to high (Type 1).