Introduction

Foot ulceration is a common complication of diabetes, a chronic condition that is a highly prevalent among US military Veterans [1]. Foot ulceration negatively impacts mobility and quality of life. With comprehensive care, only about 77% of ulcers heal within a year and recurrence is common [2]. Furthermore, individuals who are Black, Hispanic, and Native American [3–5] or live in rural areas [5, 6] have higher rates of ulceration and amputation than those who are white and live in urban areas, respectively. The reasons for the disparities are not well understood, but proposed reasons include more advanced presentation of foot problems and socioeconomic factors that may impact access to care [5–7]; systemic racism is likely a major contributing factor [8].

Elevated skin temperatures are an early sign of damage and in randomized controlled trials, risks of ulceration were 24% to 90% lower in the foot temperature monitoring groups compared to the usual care control groups [9–13]. Monitoring of foot skin temperatures is now recommended in several clinical practice guidelines [14–16]. Despite guideline endorsement, foot temperature monitoring is rarely practiced because the approach tested in the trials, which involved a handheld thermometer and individual tracking and comparison of temperatures, is time consuming and onerous. Fortunately, new technologies, including temperature sensing mats, “smart” insoles/socks, and smartphone applications have made foot temperature monitoring easier [17, 18].

This study focuses on the daily-use telemedicine foot temperature monitoring SmartMat made by Podimetrics; Somerville, Massachusetts, USA. The in-home foot temperature monitoring requires no configuration or set up by the patient. A temperature scan takes 20 seconds, and the temperature data are transmitted to the cloud using an embedded cellular component; patients are not required to have Wi-Fi or home cellular service. The software detects “hot spots”, defined as asymmetries of ≥2.2˚C between the same region on the left and right foot or different regions on the same foot. Temperature asymmetries that persist for at least two days are predictive of ulceration [19], including in those with foot deformities and partial foot amputations [20]. When a hot spot is detected, the company notifies and works with the patient to change behaviors (e.g., reduce standing/walking, wear protective footwear, and check feet for injury/infection). If the temperature asymmetries persist, the company notifies the patient’s healthcare provider who determines the next steps, often including an in-person examination.

The Veterans Health Administration (VHA), the largest integrated healthcare system in the United States, began national implementation of remote foot temperature monitoring using Smartmats in 2019. Over 6 million Veterans received care from VHA in 2019 at one of over 170 medical centers and 1200 outpatient facilities [21]. The mean and median number of patients at each medical center (including satellite outpatient clinics) was 52,162 and 44,637 (interquartile range: 31,40 to 64,881), respectively [22]. The VHA Innovation Ecosystem [23] launched the Initiative to End Diabetic Limb Loss [24] in partnership with the VHA Podiatry Service, Office of Health Equity and Office of Connected Care to design new care models that incorporated emerging technologies like the SmartMat in early detection of diabetic foot ulcers. This initiative encouraged providers (primarily podiatrists) to enroll patients in remote foot temperature monitoring, a program that was offered free of charge to the Veteran. The standard operating procedures in place in 2019 detailed circumstances for appropriate provision of temperature monitoring devices (Kyle Nordrum, personal communication). These guidelines declared suitability for RTM was based on patients having either 1) peripheral neuropathy with peripheral artery disease and/or a foot deformity or 2) peripheral neuropathy and history of foot ulcer or lower extremity amputation.

In a separate report, we assessed effectiveness of RTM in VA; compared to usual care, RTM was not associated with a reduction in lower extremity amputation or hospitalization, though it was associated with a reduced risk of death [25]. Although that study [25] was unable to determine the reasons for the absence of a reduction in amputation risk, hypothesized reasons include patients not using the SmartMat as directed, patients not having been alerted when there was a hot spot or not complying with instructions when alerted, and/or enrolling patients who may not benefit from RTM (e.g., patients with peripheral artery disease who would not mount an inflammatory response to injury or infection). A recent study of patients who monitored foot skin temperatures via a handheld thermometer found that skin temperatures were frequently not elevated prior to ulcer development, calling into question the foot temperature increase-ulcer association [26]. Given these confusing results, more research is needed to confirm or refute the recent findings and to help illuminate the reasons for the discrepancies; a multi-site randomized controlled trial is underway to rigorously evaluate outcomes (and mediators of outcomes) [27].

Meanwhile, because RTM has potential to reduce the risk of ulceration and amputation, assessing the extent to which it has been implemented in proportion to need is valuable. Thus, the goal of this study was to evaluate reach and adoption of remote temperature monitoring (RTM) in VHA, using the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework [28]. The RE-AIM framework was designed to make research findings more generalizable by encouraging scientists and evaluators to balance internal and external validity when developing and testing interventions. Reach is typically defined as the number, proportion of the intended audience, and the representativeness of participants compared with the intended audience. Adoption is defined as the number and proportion of settings and staff members that agree to initiate program or policy change and how representative they are of the intended audience. We evaluated adoption by first describing the geographic distribution of RTM throughout VHA over time and then assessing difference in characteristics of patients who received care at facilities with higher vs. lower (or no) use of RTM. The goal of these adoption analyses was to assess whether there were differences in facility-level characteristics based on RTM use. Next, to understand whether there was equitable use of RTM in the facilities that employed RTM, we compared patient characteristics of those enrolled in RTM relative to a comparison group of patients eligible for, but not enrolled in RTM (reach analyses). Information from this study can be used to identify geographic areas and subpopulations that may be underutilizing this technology and may benefit from interventions to increase uptake.

Materials and methods

Study design, population, and data source

We conducted an observational study using national data from VHA electronic medical records. Data were accessed between July 2020 and December 2022. To be eligible for inclusion, patients had to have diabetes and at least two diagnosis or procedure codes for an ulceration between July 1, 2018, and June 30, 2021 (Fig 1). We chose July 1, 2018 to allow for at least six months before the first possible enrollment date for these analyses. These criteria applied both to patients enrolled and not enrolled in RTM. Current procedural terminology and International Classification of Diseases (ICD)-9 and ICD-10 codes were used to identify ulcers. See S1 Table for specific codes. The date of the earliest encounter for the ulcer was used as the baseline date for assessing covariates unless noted otherwise. We excluded patients from both groups if they had codes in the two years prior to baseline indicating that they were not good candidates for RTM including: dementia; inability to walk, as determined by diagnosis codes for quadriplegia, paraplegia, hemiplegia, and/or spinal cord injury; or bilateral major lower extremity amputation (because they did not have at least one foot that could be monitored via temperature scans). We also excluded patients if they had no primary care encounters in the two years prior to baseline, or they were enrolled in RTM prior to January 1, 2019.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Diagram describing inclusion and exclusion criteria for study analyses.

The figure shows the number of people included in the Table 1 and Table 2 analyses. Table 2 includes 46,294 patients, including 45,228 who never enrolled in RTM and 1066 who were enrolled in RTM.

https://doi.org/10.1371/journal.pone.0301260.g001

Covariates

We evaluated demographic, geographic, clinical, and facility factors, as well as utilization. Details about the data sources, definitions, and categories are included in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Covariates.

https://doi.org/10.1371/journal.pone.0301260.t001

Results

We identified 1066 patients who met study inclusion criteria and were enrolled in RTM and 45,228 patients who met study inclusion criteria but were not enrolled in RTM. Of the 45,228, 27,166 patients were at a facility where at least one patient was enrolled in RTM (Fig 1). Fig 2 shows how RTM use varied over time and was distributed across VHA facilities. Each map covers a 7.5-month period. Average monthly enrollment per facility among patients who met this evaluation’s eligibility criteria was 11.2 during the first 7.5-month period (January 1, 2019-August 15, 2019); 26.9 patients per month in the second time period; 61.7 patients per month in the third time period, which began shortly after the COVID-19 pandemic was declared; and 42.3 patients per month in the fourth time period. The number of facilities that enrolled at least 10 patients (1.3 per month) over a 7.5-month time period was 4, 7, 11, and 7, during the first, second, third, and fourth time periods, respectively, including 8 facilities that enrolled at least 10 patients in more than one time period. Notably, in the third time period (April 1, 2020-November 15, 2020), the top two facilities (Cincinnati and Palo Alto VHA Medical Centers) each enrolled more than 12 patients per month (13.5 patients per month in Palo Alto and 12.4 in Cincinnati), while the next three highest facilities enrolled 7.3, 4.3, and 3.6 patients per month, respectively. In addition to an increase in the number of facilities that enrolled 10 patients or more during a time period, the number of facilities that enrolled 1–9 patients increased over time, from 7 facilities in the first time period to 35 in the fourth time period. Fig 2 also makes clear that there are several states with no patients enrolled in RTM (e.g., Montana, North and South Dakota, Maine, New Hampshire, and Vermont). Additionally, facility growth in RTM was not linear or predictable, as some facilities, like the Manhattan VHA, enrolled 2.7 patients per month in the first time period, 1.7 patients per month in the second, and none in the third and fourth time periods.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Remote foot temperature monitoring enrollment over time by facility.

The map on the top left represents enrollments between January 1, 2019, and August 15, 2019. The map on the top right represents enrollment between August 16, 2019, and March 31, 2020. The map on the bottom left represents enrollment between April 1, 2020, and November 15, 2020. The map on the bottom right represents enrollment between November 16, 2020, and June 30, 2021. “Parent” facilities may include more than one medical center. For example, the Palo Alto VHA Medical Center parent facility includes the Palo Alto VHA, as well as facilities in Menlo Park and Livermore and the Portland VHA Medical Center parent facility includes facilities in Portland, OR as well as Vancouver, WA.

https://doi.org/10.1371/journal.pone.0301260.g002

Among the 46,294 patients eligible for RTM, 39%, 36%, 15% and 10% received care from facilities with no, low, moderate, and high RTM use, respectively (Table 2). Differences across facility categories were statistically significant for all variables. Here, we highlight larger absolute differences across categories (≥5 percentage points between high and no patient categories). Although there was variation across use groups in terms of race and ethnicity, the differences were relatively small and did not show a clear trend. Compared to patients at non-high-use facilities, patients at high-use facilities had better geographic access to care based on drive-time. Patients at high-use facilities were disproportionately urban and in the Pacific region, and small proportion of patients at moderate- and high-use facilities were in the Southeast. Based on the Area Deprivation Index, a composite measure of deprivation where lower scores represent less deprivation, a greater proportion of patients at high-use facilities lived in areas with low deprivation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Patient characteristics by site remote temperature monitoring patient volume (between January 1, 2019 and June 30, 2021) for all eligible patients.

https://doi.org/10.1371/journal.pone.0301260.t002

Lastly, a greater proportion of the high-use facilities were high complexity and had lower patient to podiatrist ratios (40.6% in the lowest quartile in high-use facilities vs. 23.7% in the lowest quartile for patients at facilities with no patients enrolled). Findings were mixed in terms of quality measures. Specifically, high-use facilities had better above the knee (AK) to below the knee (BK) amputation ratios (when considering the lower two quartiles), possibly indicating better care quality, though patients at high-use facilities were less likely to have been at the best performing facilities based on the proportion of patients who were assessed for loss of protective sensation (a second measure of quality).

Next, we compared patient characteristics among those who were enrolled and not enrolled in RTM when only considering patients receiving care at facilities with at least one patient enrolled in RTM (Table 3). There was no evidence of substantial collinearity. Among eligible patients at facilities where at least one patient was enrolled in RTM, 3.7% (1006/27,166) were enrolled in RTM. In the multivariable model, factors that were independently and inversely associated (based on a 95% CI excluding 1.00) with RTM enrollment were age 80+ (vs. 70–79), Black race (vs. white), low income (vs. copayments required), living 60+ minutes’ drive time from specialty care (vs. <60), and having a moderate to high number of telehealth encounters in the prior two years (2^nd, 3^rd, and 4^th quartiles vs. the 1^st). Factors that were positively associated with RTM enrollment were having osteomyelitis, Charcot foot, and a partial foot amputation (vs. no amputation), intermediate Gagne comorbidity index (vs. the lowest category), body mass index ≥30 kg/m² (vs. 18.5–24.9), having more outpatient encounters (2^nd, 3^rd, and 4^th quartiles vs. first) and 2 or more ER/urgent care visits (vs. none). Based on the AIC (Table 4), foot conditions (especially osteomyelitis and Charcot foot) were most strongly associated with RTM enrollment, with model fit deteriorating most when these covariates were removed. After foot conditions, utilization and demographics were the next conceptual categories most strongly associated with RTM enrollment, followed by other health conditions/comorbidities, and finally geographic access.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Distribution and associations of demographic, geographic, and utilization characteristics among eligible VHA patients who were and were not enrolled in remote temperature monitoring between January 1, 2019, and June 30, 2021.

https://doi.org/10.1371/journal.pone.0301260.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Summary of Akaike information Criterion (AIC) for conceptual groups and variables.

https://doi.org/10.1371/journal.pone.0301260.t004

Discussion

RTM use increased substantially over the 30-month observation period, including 15 months during the COVID-19 pandemic. Beginning in March 2020, VHA shifted from in-person encounters to virtual care (video and phone-based care) [39]. In fact, a study of Veterans 65 and older found that there were 824% more virtual visits in April-November 2020 compared to the pre-pandemic period [40]. RTM allowed providers to monitor patients’ feet despite restrictions on in-person care; organizations like the International Working Group on the Diabetic Foot recommended temperature monitoring to more effectively care for patients with, or at risk for DFU when in-person care was limited [41]. Notably, though the number of patients enrolled and the number of facilities employing RTM increased substantially during the observation period, growth was concentrated in a small number of high-use facilities. RTM use was greater in facilities with a higher proportion of urban patients, lower area deprivation, higher complexity and those with a lower patient-to-podiatrist ratio. These findings indicate a need to identify and address barriers to RTM use in facilities with a high proportion of Veterans who live in rural areas or areas with high deprivation to prevent exacerbating existing disparities. Among facilities offering RTM, Black patients, those with a low income, and those with longer drive times from specialty care were less likely to be enrolled in RTM. A greater proportion of patients enrolled in RTM had a second foot condition that put them at high risk of a future ulcer or amputation (osteomyelitis, Charcot foot, or a partial foot amputation). Those enrolled were no more likely to have other comorbidities such as CKD/ESRD, depression, or poor glucose control, but were more likely to have an intermediate level of comorbidities and obesity. Those enrolled were also more likely to be heavier utilizers of face-to-face healthcare services, suggesting that patients who are seen more frequently by clinicians are more likely to be invited and enrolled (opportunistic enrollment). If RTM lowers healthcare costs (a question we and others hope to answer in future research), targeting patients with high utilization may be a cost-effective strategy. However, to ensure equitable implementation of RTM, it will also be necessary to take a more systematic approach to enrolling patients.

To facilitate implementation of RTM during the period under study, the Innovation Ecosystem aimed to understand challenges providers and sites faced in incorporating RTM into preventative care (Suzanne Shirley, personal communication). The Innovation Ecosystem includes Innovator Network sites, which have designated “Innovations Specialists” who are clinicians, researchers, and administrators who work with frontline employees to identify, test, and spread innovative products and practices throughout VHA. The Innovation Ecosystem met with providers and learned that they found the ordering process to be complicated, time consuming and confusing. They also learned that there was insufficient knowledge about the availability and impact of RTM. The Innovation Ecosystem convened key stakeholders to identify potential solutions, which included approaches to make ordering easier; efforts to provide a stronger evidence base through evaluations of clinical, economic and equity related impacts (of which this study is a part); and clearer communication to providers on the impact and availability of this technology.

A few limitations should be considered when interpreting our results. First, identifying a comparison group was challenging. When evaluating appropriateness for RTM, clinicians might consider not only whether a patient had a prior ulcer, but whether prior ulcers were on the plantar surface of the foot in addition to other factors. However, because ICD-10 codes do not specify ulcer location, we were not able to consider ulcer location as an inclusion criterion. Thus, we may have included patients who would not have been considered good candidates for RTM, including those with severe peripheral artery disease, as this condition is difficult to determine based on diagnosis and procedure codes and we lacked the resources to do chart review. Additionally, because we relied on diagnosis and procedure codes, we did not have data that might have been useful for risk adjustment, such as diabetes-related treatments, duration of diabetes, or time since last ulceration. Utilization measures were based solely on VHA care; we may have underestimated total healthcare utilization among patients who received some of their care outside VHA. Furthermore, codes may sometimes be applied erroneously. However, we have no reason to suspect that errors would be differential by RTM use status. We included only two measures of quality of care. A more thorough evaluation of quality-of-care measures (which was outside the scope of this study) may have resulted in a more clear and consistent association with RTM facility volume. We assumed that those not enrolled were not offered the opportunity to enroll, but some patients refuse RTM, though this information is not tracked, and refusals are relatively uncommon. Lastly, we only evaluated the reach and adoption components of the RE-AIM framework. Effectiveness was evaluated separately [25]. It would be useful for future studies to assess implementation and maintenance. Important strengths of this study include its national scope and ascertainment of information beyond medical conditions, including characteristics of the patient’s social environment (e.g., Area Deprivation Index) and healthcare access and utilization, to provide a more comprehensive picture. Furthermore, our multivariable model allowed us to evaluate the independent associations between factors and RTM enrollment.

In conclusion, this study provides important information on the adoption and reach of RTM in the initial years of national implementation in VHA, including characteristics of facilities with higher and lower use, and characteristics of enrolled patients relative to those who are targeted. Our findings indicate that adoption of RTM was not equitable as this innovation was primarily employed by higher-complexity facilities with patients who had better geographic access and who lived in areas with less deprivation. Furthermore, within facilities employing RTM, there were indications of inequitable distribution of RTM. It will be valuable to continue to identify barriers to RTM and potential solutions to ensure that this technology is equitably disseminated. Equity will require matching of RTM need and use across facilities and continued surveillance for disparities.

Supporting information

Acknowledgments

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government. A portion of this paper was presented as a poster at the International Symposium of the Diabetic Foot (the Hague Netherlands) in May 2023.