https://orcid.org/0000-0002-0224-3482
Figures
Abstract
Diabetic Retinopathy stands as a leading cause of irreversible blindness, necessitating frequent examinations, especially in the early stages where effective treatments are available. However, current examination rates vary widely, ranging from 25–60%. This study scrutinizes the Point-of-Care Diabetic Retinopathy Examination Program at the University of Pittsburgh/UPMC, delving into its composition, evolution, challenges, solutions, and improvement opportunities. Employing a narrative approach, insights are gathered from key stakeholders, including ophthalmologists and staff from primary care clinics. A quantitative analysis from 2008 to 2020 provides a comprehensive overview of program outcomes, covering 94 primary care offices with 51 retinal cameras. Program components feature automated non-mydriatic 45° retinal cameras, a dedicated coordinator, rigorous training, and standardized workflows. Over this period, the program conducted 21,960 exams in 16,458 unique individuals, revealing a diverse population with an average age of 58.5 and a balanced gender distribution. Average body mass index (33.96±8.02 kg/m2) and hemoglobin A1c (7.58%±1.88%) surpassed normal ranges, indicating prevalent risk factors for diabetes-related complications. Notably, 24.2% of patients underwent more than one exam, emphasizing program engagement. Findings indicated that 86.3% of exams were gradable, with 59.0% within normal limits, 12.1% showing some evidence of diabetic retinopathy, and 6.4% exhibiting vision-threatening diabetic retinopathy. Follow-up appointments with ophthalmologists were recommended in 31.5% of exams due to indeterminate results, positive diabetic retinopathy (≥moderate or macular exudate), or other findings like age-related macular degeneration or suspected glaucoma. The program demonstrated high reproducibility across diverse healthcare settings, featuring a sustainable model with minimal camera downtime, standardized workflows, and financial support from grants, health systems, and clinical revenues. Despite COVID-19 pandemic challenges, this research emphasizes the program’s reproducibility, user-friendly evolution, and promising outcomes. Beyond technical contributions, it highlights human factors influencing program success. Future research could explore adherence to follow-up ophthalmological recommendations and its associated factors.
Citation: Bonilla-Escobar FJ, Ghobrial AI, Gallagher DS, Eller A, Waxman EL (2024) Comprehensive insights into a decade-long journey: The evolution, impact, and human factors of an asynchronous telemedicine program for diabetic retinopathy screening in Pennsylvania, United States. PLoS ONE 19(7): e0305586. https://doi.org/10.1371/journal.pone.0305586
Editor: Andrzej Grzybowski, University of Warmia, POLAND
Received: February 6, 2024; Accepted: June 1, 2024; Published: July 12, 2024
Copyright: © 2024 Bonilla-Escobar et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting information files. Further Data cannot be shared publicly because it consist of medical records and clinical data of patientes helded by the University of Pittsburgh Medical Center (UPMC/UP). Data are available from request to Ethics Committee (contact via email research@pitt.edu) for researchers who meet the criteria for access to confidential data.
Funding: EW The Eye and Ear Foundation of Pittsburgh. https://eyeandear.org/ The funders did not play any role in the study design, data collection and analysis, nor decision to publish or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Diabetic Retinopathy (DR) is a leading cause of irreversible vision loss [1]. While there are effective treatments for DR, these are most successful in the early stages when patients are asymptomatic [2–6]. Consequently, patients with diabetes should be examined annually for DR [7]; however, this currently occurs at rates between 25–60% [7–10].
Known barriers to examination include finances, transportation, lack of education, time constraints, and access [8, 11–13], all of which are more prevalent in people living in underserved communities in which diabetes, diabetic complications, and DR are more prevalent [14].
The standard for DR examination is an in-person retinal examination by an ophthalmologist or optometrist through dilated pupils [15, 16]. Retinal photography with remote interpretation has been used as a method of overcoming barriers to an in-person examination, demonstrating to be sensitive, specific, and cost-effective [9, 12].
There are several programs with remote interpretation of retinal photographs of diabetic patients in the US and abroad [8, 9, 17–31]. The American Teleophthalmology Association (ATA) has created technical guidelines for teleophthalmology programs, including those for DR [32]. Programs using remote interpretation for DR examination (DRE) are often described from a technological standpoint in terms of imaging technology, image transmission, and interpretation platform, and in term of outcomes emphasizing the quality of images and DR diagnosis and classification [18, 19, 21].
The implementation of an effective, scalable, and sustainable program for photographic retinal examination must consider the needs of patients, physicians, staff, ophthalmologists, and payors in the context of existing systems of documentation, communication, and referral patterns [32, 33]. We document the current composition, workflow, and outcomes (participants characteristics, quality of images and diagnosis) of the Point-of-Care Diabetic Retinopathy Examination Program (POCDREP) at the University of Pittsburgh/UPMC, and describe the challenges faced as the program grew, solutions to those challenges, and opportunities for continued improvement.
Methods
In this study, we provide a narrative approach to describe the implementation of the POCDREP, its evolution, and its status. A quantitative approach is used to report program outcomes based on a retrospective chart review. The period reported on includes the exams carried out from September 1st, 2008, to December 31st, 2020.
Description of the program and its evolution
As part of a quality improvement project (Code 630, July 5th, 2016), we interviewed key stakeholders—specifically, two ophthalmologists involved in program development, primary care clinic managers, and 31 primary care physicians—to identify barriers and facilitators to program implementation. Participants were selected based on their history of involvement in the program’s implementation within their respective clinics. Surveys were conducted from July 5th, 2016, and June 30th, 2017. Interview participation was voluntary, with verbal informed consent obtained.
We also reviewed communications, reports, guides, and instruction materials to present a description of the program’s evolution, coupled with the construction and annotation of a timeline ranging from program predecessors and early efforts to the current state of the program (Fig 1).
A narrative was formulated to depict pivotal moments and the rationale for changes in the program. This narrative is tailored for readers interested in implementing or enhancing a comparable program within their organization.
Program outcomes
Following the approval by the Institutional Review Board of the University of Pittsburgh on May 22nd, 2020, under the code STUDY20010159 (S1 File), we conducted secondary data collection. This process involved gathering data from medical records and the program’s automatic monthly reports from June 1st, 2020, to May 1st, 2021. Our focus was on exam results and basic sociodemographic and clinical information. Data from exams carried out between September 1st, 2008, and June 14th, 2015, from both UPMC and participating non-UPMC sites, were manually collected from their respective EHR systems. Data from June 15th, 2015, to December 31st, 2020, were gathered from automatic monthly reports generated by the UPMC Electronic Health Record (EHR) system (Epic). All data were consolidated into a unified database covering the study period.
Variables
Analyzed data included retrospectively collected information about the (1) participating sites (location, number of offices using a camera, clinic affiliation to a healthcare system—i.e., UPMC, Federally Qualified Health Center (FQHC)—, clinic main specialty, and ordering provider), (2) patients´ characteristics (age, gender, ZIP code, most recent HgbA1C level, smoking status, race & ethnicity, and primary language spoken), and exam characteristics (date of the image, date of reading, image quality, findings and diagnosis, and recommendations). One participant could have more than one DRE during the analyzed period; therefore, exams and participants were analyzed separately.
We created a poverty level variable using information from the “Percentage of Individuals Living Below the Federal Poverty Level” based on the ZIP code from the Census 2000 Summary File 3 [34, 35]. Poverty status was categorized in quintiles in ascending order from the lowest to highest levels of poverty in the neighborhood as follows: <10% (Q1), 10–19.9% (Q2), 20–29.9% (Q3), 30–39.9% (Q4), >40% (Q5). Cut points were based on the US Census Bureau’s definitions of poverty areas [36].
Image graders assessed and categorized fundoscopic image quality for each eye as “good,” “fair,” “poor,” or “non-gradable.” If an image was not uploaded, it was graded as “no image.” Photographs without smears, artifacts, or movement were defined as “good” quality images. A “fair” picture entailed a degree of smear or artifact, but was of sufficient quality to be graded confidently. Images were graded as having "poor" quality if artifacts would have prevented an assessment of the absence of retinopathy but nevertheless demonstrated pathology. In images that were “non-gradable,” graders could not evaluate whether the region of the eye that should be photographed was within normal limits.
DRE findings were described based on DR severity levels as “within normal limits”, “mild non-proliferative DR (NPDR),” “moderate NPDR,” “severe NPDR,” and “proliferative DR” [37]. Another relevant finding was the presence of “macular exudate,” defined as a surrogate marker of clinically significant diabetic macular edema (DME) [38, 39]. Exams identified with DR were further classified as with vision-threatening DR (VTDR: macular exudate or ≥ moderate non-proliferative DR) or without VTDR. Graders also reported incidental findings and diagnoses, including suspected glaucoma, age-related macular degeneration (AMD), hypertensive retinopathy, among others [40].
Exams were classified as positive, indeterminate, with other findings requiring follow-up, or within normal limits. Positive exams were those with levels of DR severity ≥ moderate or with macular exudate. Exams with “non-gradable” image(s) or with “no image,” were classified as indeterminate for DR. Other findings such as suspected glaucoma, AMD, vein occlusions, nevus, and epiretinal membranes, among others, were classified as other findings requiring follow-up.
As the purpose of this study is to describe the program and its longitudinal outcomes, we used descriptive statistics to present outcomes based on data characteristics. Hence, we are describing the characteristics of (1) primary care clinics, (2) participants, and (3) exams.
We used frequencies and percentages for categorical variables and central tendency and dispersion measurements (mean and standard deviation, SD, or median and interquartile range, IQR) for quantitative variables. All analyzes were carried out in Stata16® (StataCorp, TX).
Results
1. Program structure
1.1. Origins and evolution of the program.
The first attempt to use remote photography to address diabetic eye examination care gaps at UPMC was in 2008 (Fig 1). Four retinal cameras, Topcon TRC NW400, were installed in various settings, including three in primary care offices and one in a mobile unit with funding from the US Department of Defense (DoD Project). However, the project faced limited success, despite efforts involving multiple focus group sessions with staff to facilitate the implementation of the cameras. However, participating sites engaged solely when compliance with the Healthcare Effectiveness Data and Information Set (HEDIS) measures was required.
The selected early non-mydriatic cameras, though operationally complex, were operated by non-ophthalmic technicians trained for image capture. These technicians also served as study coordinators in the feasibility research protocol, managing also consent forms. Challenges in camera maintenance resulted in suboptimal image quality, and the infrastructure for transmitting and reporting images was cumbersome, causing delays in interpretation. By 2012, only one primary care practice had a functioning camera, and the mobile unit was mainly used for DR exams at health fairs. The DoD Project concluded with approximately 750 patients screened for diabetic retinopathy.
In 2006, our department initiated a community outreach project to address unmet eye care needs in Pittsburgh and the surrounding areas. The Guerrilla Eye Service (GES) provides comprehensive eye examinations—refraction, anterior segment examination, dilated retina examination, and perimetry—to patients at free care clinics. GES is staffed by medical students, ophthalmology residents, and attendings. The service remains active in clinical sites around Pittsburgh. A large proportion of the patients served by GES have diabetes and in addition a there is a substantial no-show rate at the missions.
High no-show rates led to the idea that permanently stationing a non-mydriatic fundus camera at a GES site might improve exam rates and DR detection, as photographic exams would be captured when patients were seen for a medical exam. Automated, non-mydriatic cameras were becoming available, and with the help of a local foundation, a Centervue DRS® nonmydriatic automatic camera was purchased for the McKeesport 9th Street clinic in 2012. McKeesport’s population is a heterogeneous mix of ethnicities and nationalities, with 30.3% living below the poverty line [41].
In contrast with the earlier non-mydriatic retinal cameras and the high-end ophthalmic cameras, the Centervue DRS required minimal training for clinic staff. The transmission of images for reading and interpretation was conducted utilizing Centervue’s cloud-based reading platform under a subscription arrangement.
Over the following year, a neighboring primary care clinic periodically borrowed and used the camera; eventually, funding was secured to purchase two additional cameras in McKeesport, one of which was placed at the Latterman Family Health Center, with the other at the McKeesport Hospital Internal Medicine clinic. Both sites are UPMC facilities and share the UPMC EHR infrastructure.
The program’s success at indigent care sites in McKeesport was shared with the leadership of the Internal Medicine clinics at UPMC flagship sites in Oakland and Shadyside (UPMC University of Pittsburgh Physicians, UPP), where DR exam rates were approximately 20%. After a trial period with a loaned camera in 2014, UPMC purchased a camera for each flagship site.
The annual subscription to the cloud-based reading platform was deemed financially unsustainable. Because of this, the platform was eventually discontinued. The program moved to secure-email based interpretation or UPMC Epic EHR transmission of images and interpretations.
After the growth in McKeesport, the program received much greater interest, and group of three UPMC Family Medicine sites (St. Margaret Family Medicine) applied for a grant following a loaned trial, subsequently receiving funds for a camera at each site. Federal Qualified Health Centers (FQHC) that were part of the GES missions also acquired cameras.
In 2015, the remaining cameras from the DoD funded program were rolled into the point-of-care diabetic retinopathy examination program (POCDREP). Over this period, cameras were placed in additional non-UPMC indigent care sites around Pittsburgh [42], and a coordinator was hired to work on quality improvement for the program.
UPMC Community Medicine Inc. (UPMC CMI), the community-based medicine arm of UPMC, acquired 30 cameras for their sites. Over the next several years, the program reached sites outside the Pittsburgh region. Once again, having trialed the concept with a loaned camera, four clinics in Harrisburg, PA—about 200 miles from Pittsburgh—purchased cameras. In addition, the program now serves Erie, PA (~130 miles from Pittsburgh) and Susquehanna, PA (200 miles from Pittsburgh).
The program continues to grow, primarily by word of mouth between medical directors, who are then put in contact with our program, and loaned a camera for trial purposes. Following this, funding is obtained from UPMC or a local health-focused neighborhood foundation when success is demonstrated.
1.2. Retinal cameras.
Automated, non-mydriatic retinal cameras have been essential to the program’s success, primarily because they are easier to use, resulting in greater staff acceptance, albeit at the expense of the adjustability and versatility of the cameras intended for an ophthalmologist’s office. Clinic staff utilized either the Centervue DRS or Topcon TRC NW400 digital retinal camera to capture macula-centered, 45-degree images of each eye. Images from UPMC offices were stored and forwarded using Epic®, whereas those from non-UPMC facilities were forwarded via secure email. There are currently many cameras available, with each one requiring an investment of approximately $17,000. Currently, all but one of the cameras used in the UPMC program were manufactured by Centervue (now iCare). We have found that using a single camera model at nearly all sites simplifies deployment, configuration, training, and fosters peer support for troubleshooting. Cameras requiring repair can be replaced by a drop-in loaner camera from the program, with minimal downtime.
1.3. Personnel.
Initially, a department faculty member (EW) managed the program’s administration with support from department administrative assistants. However, as the program grew, it became evident that additional administrative support was required. As a result, a graduate student (FJBE) was recruited to serve as the program coordinator. Salary support for the coordinator position was secured from a grant from the Beckwith Institute, an endowed fund of UPMC, and subsequently from department funds as the program became large enough to generate sufficient revenue.
The coordinator is responsible for:
- Providing training to working personnel at new sites;
- Serving as a clinical site liaison to troubleshoot and train new personnel; and
- Working with local site personnel to arrange follow-up for patients with photographic exams that suggest the need for an in-person evaluation with an eye care provider.
At each clinical site, it has been essential to identify and engage an attending physician and office manager responsible for local oversight of the program. At each site, providers must understand the program, indications, and procedures for ordering photographic examinations; likewise, it is crucial that they familiarize nurses and medical assistants with the EHR workflow, and the camera’s operation.
1.4. Training.
Early in the program, we depended exclusively on representatives from the camera vendor to provide on-site training for the camera operation. While this was helpful, we found that it was insufficient as despite its relative ease of use, it became clear that it was necessary to provide multiple initial training sessions and further refresher training to help clinic staff routinely capture images of acceptable quality. In addition, training was required to implement the workflow specific to other EHRs. To ensure competence, the minimum requirements were one hour of online training, and a further hour in person; however, the trainer remained on site for a minimum of two days following this initial period to conduct further hands-on training, role play, and to provide in-person advice and recommendations.
Local clinic personnel were essential in the creation and iterative improvement of training materials that facilitated the acquisition of acceptable quality photographs while avoiding common imaging pitfalls. In addition, these staff were equally helpful in designing and improving EHR workflows and training materials for onboarding new personnel.
1.5. Sustainability.
Program expenses included those related to equipment, deployment, and personnel. Equipment expenses included the camera and table purchase cost for each site, along with associated infrastructure adjustments like electricity or internet port installations, stools, replacement of chair or stool casters with bell glides, lens cleaners, and occasional repair costs. Late in the program, service contracts were routinely purchased with each new camera. Deployment expenses could have included the costs of networking each new camera, but these were routinely absorbed by UPMC and, as such, were not visible to the program. Personnel expenses were limited to the salary of the project coordinator. The program director and clinic personnel did not receive additional compensation. Image graders were compensated as a result of additional generated clinical productivity.
Equipment costs were covered by grants or by the health system, and program revenues resulted from the clinical revenue attributed to clinical sites and image graders, clinical practice revenues, and healthcare payer revenues. The most significant, measurable revenue is from improved quality scores; additionally, the program may result in decreased costs to healthcare payers by detecting retinopathy early and preventing blindness. However, these cost savings are not easily calculated.
1.6. Workflow.
Epic is the EHR for outpatient care in the UPMC system. The workflow for DRE goes as follows:
- Patients are identified for inclusion in the program. On some sites, patients who meet the criteria for the program are identified on review of the daily patient schedule, and a photographic examination takes place as part of that visit. At other sites, the patient panel is reviewed regularly to determine which patients are overdue for DRE, following which they are called to schedule an exam specifically for this purpose.[15]
- An order for the photographic exam is placed in Epic;
- Clinic staff perform the photographic exam;
- Images resulting from the exam are uploaded to the Epic Media Manager and attached to the order;
- The order is forwarded to an Epic pool for interpretation of the photos;
- Two ophthalmologists in the reading pool, working from their Epic in-basket, examine each image and create a report;
- The report is entered into the interpretation section of the order and copied back to the ordering provider;
- The ordering provider is then responsible for following up with the patient and initiating any further actions as required.
Report creation is simplified and standardized with the help of software created in house. The physician using the software checks off findings in each eye, and the program then ‘calculates’ diagnoses and ICD-10 codes, determines a disposition, and creates a formatted report copied into Epic (Fig 2).
Photographic exams from sites that do not use Epic require additional steps at the beginning and end of the Epic workflow. Examination photos and patient data are communicated to the program coordinator via secure email and an Epic testing encounter is created to initiate the standard workflow. In addition to recording the interpretation in Epic, a secure email containing the report is sent to the provider.
1.7. Coding and billing.
A program-specific billing system was implemented in 2016. Decision-makers identified an opportunity for reimbursement after screening insured patients, with a reimbursement for both the clinic taking the photos and the graders. In addition, there are reimbursements and stimulus packages for sites improving their HEDIS measures, a widely used measure of care performance, and quality scores based on DRE rates. Once the program provided evidence of change in quality scores, the support from stakeholders increased. Creating a standard coding and billing infrastructure greatly facilitated the sustainability of program implementation.
1.8. Patient follow-up.
The need for documentation of patient follow-up after DRE was informed by the prevalence of patients who underwent multiple DREs without documentation follow-up. Within the first year of documentation (2016), we observed that the follow-up rate of patients who were recommended to seek further care with an eye doctor was 42%. This low rate suggested that DRE and diagnosis alone may not be sufficient to prevent DR progression, and that a more significant effort would have to be directed toward facilitating follow-up care after the initial DR diagnosis.
We helped enhance this process through a quality improvement project, instructing sites on correctly documenting the reports from community eye doctors in the EHR, asking primary care staff to address the reports, and noting instances of contact or attempts to contact the patient. Attempts to create an automated post-DRE tracking system in Epic were unsuccessful; consequently, post-DRE follow-up and tracking remain a manual process.
1.9. COVID-19 pandemic.
The program stopped imaging patients from March to May 2020 due to the COVID-19 pandemic and following this period, it was essential to identify how to safely restart the strategy. When using tabletop cameras, standard practice dictates cleaning the areas of patient contact before and after each patient, including the headrest, chinrest, and table. This practice did not change, with disposable alcohol swabs and wipes used to clean the surfaces. Upon site reopening, DRE numbers increased but have not yet returned to peak pre-pandemic levels.
Issues brought to our attention after the pandemic included the lack of in-person training, and staff being unable to recall specific configurations or procedures. Sites distant to Pittsburgh (+2 hours’ drive) were our main concern as the program was halted there for a longer period of time.
2. Program outcomes
2.1. Participating sites.
The program covers 35 Western and Central PA municipalities across a total area of approximately 51,000 Km2 (approximately 2 million inhabitants). Since 2008, cameras have been placed in 51 offices; by 2020, 48 sites had a camera, with three sites having discontinued participation. Additional sites refer patients for DREs to sites with a camera (n = 46). This referral system has increased the total number of sites in the program to 94.
A total of 87 sites are owned by UPMC, most of which are primary care clinics in Pittsburgh and its surrounding areas (Fig 3). One camera is stationed on a mobile unit and used to close care gaps at nursing homes and community centers; seven have been placed at sites greater than 100 miles from Pittsburgh (UPMC Annville, UPMC Susquehanna UPMC Hammot-Erie); and there are also 4 FQHC and 3 free clinics. In terms of specialties, most of the sites are focused on primary care, and 3 cameras have been placed in endocrinology clinics.
Legend: Locations marked with a blue pin are UPMC Community medicine Inc. (CMI) practices, with a black pin are University of Pittsburgh Physicians (UPP), with a red pin are remote locations (>100 miles from Pittsburgh), and with a yellow pin are non-UPMC clinics.
Reprinted from NASA Earth Observatory website under a CC BY 4.0 license, Public domain, original copyright 2024, retrieved from (https://earthobservatory.nasa.gov/map#6/41.179/-78.223).
2.2. Participants.
From August 2008 to December 2020, there were 21,960 DREs. The monthly distribution of photographs between 2008 and 2020 is shown in Fig 1. A total of 16,458 unique individuals were examined during this period. The age average of the examined patients was 58.5 ± 13.7 years and 46.5% (7,657) were female. The majority of patients identified themselves as White (65.0%, 10,699). Only 1.5% (246) were Hispanic or Latino, almost half (44.6%, 7,342) were unmarried (divorced, legally separated, single, or widowed), and most had type 2 diabetes (93.4%). See Table 1.
Out of the total individuals, 75.8% (12,469) had a single DRE, 17.7% (2,909) had two, 4.6% (752) had 3, and 2% (328) had 4 or more. There was a trend of growth in the number of images until the COVID-19 pandemic, when the program was paused (Fig 1).
2.3. Exams.
At the time of DRE (n = 21,960, Table 2), most participants used Medicare (32.2%, 7,073) as insurance, followed by commercial insurance (29.2%, 6,406). In terms of socioeconomic status at examination based on zip codes, 6.9% (1,524) of examined patients were below the poverty line (>Q2, 20%-39.9%). None of the patients imaged lived in the zip code designated as extreme poverty areas.
When examined, most people reported that they never had smoked (37.6%, 8,265) or were former smokers (30.4%, 6,687). Participants’ average BMI at the moment of imaging was 33.96±8.02 (n = 19,700, range 12.6–88), and their mean hemoglobin A1c (HbA1c) was 7.58±1.88 (n = 18,617, range 3.5–18.5) with 9.75% with levels ≥10% (Table 2).
2.2.1 Findings. From the images collected (n = 43,665), 63.3% were of good quality, 18.62 fair, 4.41% just gradable, and 2.64% of poor quality; and 10.45% were non-gradable. Only 0.6% (261) of participants’ eyes were not imaged due either to monocular status, or the staff not uploading the image for reading. Non-gradable images rates per eye were 2.7% (592) and 3.7% (828) for the right and left eye, respectively.
Exams were gradable in 86.3% (18,957) of the tests. The percentages of indeterminate exams (non-gradable and no image) declined in the first four years of the program and stabilized until the inclusion of many new clinics in 2017; however, the rate appears to have stabilized afterwards. Additionally, the prevalence of positive DRE findings consistently remained within the range of 20% to 30% throughout the entire program duration (Fig 4).
Hypertensive retinopathy was diagnosed less in the first years of the program, while AMD and VTDR were the most common findings. After 2016, there appears to have been a stabilization of the percentages of these diseases with a higher incidence of VTDR and suspected glaucoma. Fig 5 shows the trends of diagnosis of VTDR, AMD, suspected glaucoma, and hypertensive retinopathy.
Almost one-third (6,928, 31.5%) of the exams resulted in a recommendation for a follow-up appointment with an ophthalmologist. This determination relied on the exam being labeled as positive, other findings necessitating follow-up, indeterminate results, or a combination of these factors. Among exams that were gradable (18,957), 59.0% were within normal limits, 12.1% showed at least some evidence of DR, 7.8% suspected glaucoma, 2.9% hypertensive retinopathy, and/or 2.7% AMD. Some images were not gradable for DR but demonstrated visible pathology other than DR, and therefore required further evaluation (2.0%, 441).
Of those exams showing a level of DR, 10.9% (2,404) had nonproliferative disease, whether mild (6.3%, n = 1,385), moderate (3.6%, n = 791), or severe (1.0%, n = 221). Proliferative disease was found in 0.8% (177) of exams, macular exudate in 3.0% (661), and panretinal photocoagulation in 0.7% (144) of exams. VTDR was identified in 6.4% (1,409) of the exams.
Discussion
We documented the origins and evolution, current state, and outcomes of a point-of-care diabetic retinopathy examination program (POCDREP) at UPMC, an academic health center in Pittsburgh. Since 2008, the program has overcome several challenges and proven to be a user-friendly, reproducible tool for DRE with promising results across different sites and healthcare systems (i.e., FQHC and free clinics). Other programs have been previously described particularly with focus on the technology and the imaging transmission [19, 22–24, 32]; however, to the best of our knowledge, this is the first description of a program that includes not only technical aspects but emphasizes in the human factors of a DRE program, and the most extensive data review period (spanning 13 years) among programs of a similar nature.
Program structure
One of the program’s critical features is the use of cameras that do not require a mydriatic pupil for acquiring retinal images [43, 44], facilitating image-taking in primary healthcare centers after a short course of training and supervision, followed by remote image interpretation by ophthalmologists who analyze and detect abnormalities that indicate DR.
The program has demonstrated high reproducibility as we developed a standardized workflow that ensures consistent results across different users and settings. It allows the program to be used in multiple locations, serving the entire population without regard for insurance or economic status, even in areas where access to specialist medical professionals is limited (from a low-resource point of care, free clinics to multispecialty sites).
Another highlight of this program is its participatory approach, rarely mentioned in other programs [45, 46]. Primary care sites personnel have a voice in the program to help with its continue growth and removal of barriers for examinations. They, as a team, helped with implementation and development of protocols for image transmission, provided feedback about training, imaging strategies, and established their own collaborations with other sites for them to become part of the program (i.e., loaning a camera between clinics).
Patients’ characteristics
Based on patients’ characteristics, the program had an equal gender distribution, with a 50/50 male-to-female ratio. Their mean age is 60 years, which includes both early (<40 years) and late-onset diabetes (>40 years). Our program included a widely distributed population not limited to a specific age group, which is relevant for DR screening as the disease can affect individuals at any age and stage of their disease [47]. This real-life study also shows rates of continuity of care, where around 25% of the patients had at least another yearly exam within the program, which are similar to those previously reported [25]. Continuity of DR care is relevant as shown in a cohort study analyzing changes in DR levels using DRE over a 5-year period, where 6% of patients without retinopathy and 27% with mild non-proliferative DR progressed to VTDR [26]. Further research on the incidence of new retinopathy or its progression within the POCDREP is required.
Participants in our study reflect the Pittsburgh’s demographic composition. For instance, Medicaid recipients comprised 14.5%, closely aligning with the 16.5% reported in the 2021 census. Similarly, the representation of Veterans Affairs beneficiaries at 0.4% mirrors the census figure of 0.91%. Despite challenges with sustainability and implementation, we reached a part of the uninsured segment of the population (1%); however, not enough when the 2021 Census described a 5.46% rate of uninsured people [48]. Barriers to obtaining specialty care for uninsured patients encompassed out-of-pocket costs, stigma, scarcity of local specialists accepting uninsured patients, and logistical challenges like transportation. Consequently, teleophthalmology in primary care emerges as a tool to address these barriers and bridge healthcare gaps not only for uninsured patients but also for those with limited coverage [49].
In the US, Hispanic and Black patients have higher prevalence of DR and VTDR.[50] As the patients within this study mostly identified as White, further research and quality improvement efforts are needed to reach high-risk populations.
Obesity, is a well-recognized, prevalent risk factor for multiple diseases, including type 2 diabetes, and can be a predictor and monitoring parameter for diabetic patients [51]. Patients participating in the POCDREP were found to have an average BMI of 34±8 and only 8.6% of the population had a normal BMI. A 2021 US study, demonstrates a direct correlation between obesity and increased prevalence of severe DR, including VTDR [52]. Furthermore, obesity has a significant correlation with multiple microvascular (retinopathy, neuropathy, nephropathy) complications in diabetic patients, thereby emphasizing the need for sustainable strategies to prevent obesity and its complications [53].
The HbA1c goal in diabetic patients varies depending on individual factors such as age, comorbidities, and diabetes duration [54]. However, in general, the American Diabetes Association recommends an HbA1c target of less than 7% for most non-pregnant adults with diabetes [55]. Diabetic patients with an HbA1c ≥ 7% are 6.9 times more likely to develop DR than those with an HbA1c < 7%, including VTDR [56]. During the program, we observed individuals had an average hemoglobin A1c of 7.6 ±1.9, with almost 10% of patients with levels above 9.9%, which suggests inadequate disease control and an increased risk of microvascular complications such as VTDR [54].
Program outcomes
Non-gradable exams in our study (10.5% either or both eyes) were similar to those reported in similar DR programs in the US and internationally [8, 17, 20, 23, 25, 27–31]. The use of dilating drops has been suggested as a way to overcome this barrier, a strategy that has been implemented with training recently in our program. However, certain diabetic patients encounter multiple constraining factors for acquiring good quality retinal images even after pupil dilation, such as cataracts, smoking, and small pupils due to an autonomic neuropathy [17, 57, 58].
Approximately one-third of individuals examined for DR demonstrated positive results, indicating a substantial prevalence of the condition in asymptomatic individuals. VTDR, AMD, glaucoma suspicion [59], were identified as the most common findings which is similar to global literature on the topic [8, 17, 20, 23, 25, 27–31]. Implementing secondary prevention programs may improve prognosis and disease control while also helping identify other prevalent causes of blindness [60]. This may also reduce the incidence of vision loss in diabetic patients, thereby reducing the cost of treatment by preventing complications.
A follow-up appointment with an ophthalmologist was requested in 31.5% of the exams. In the U.S., research indicates that less than 40% of patients with diabetes receive annual eye examinations, and this rate is even lower among underserved populations [61, 62]. Even in the population already receiving treatment for proliferative DR, the non-follow-up rate is 16.3% [63, 64]. Further research is needed to identify follow-up rates and the factors preventing patients from adhering to the recommendations after a positive exam.
As for the revenue generated by this program, conducting a cost analysis to evaluate the impact of early disease detection on overall costs could provide valuable insights for healthcare providers and policymakers. Researchers can better understand the cost-effectiveness of such interventions by assessing the costs associated with early disease detection and comparing them to the potential savings from preventing or reducing the severity of DR [65]. Identifying the impact of DRE using non-mydriatic retinal cameras in Pennsylvania also has the potential to support the expansion of such programs and serve as an impetus for further research in this domain. These initiatives will necessitate collaboration between healthcare providers, policymakers, and researchers to develop comprehensive approaches to prevent DR.
Limitations
Retrospective studies have limitations that must be considered when interpreting their results. One of the primary limitations of our study was the lack of an early standardization of the program data collection tools, which limited the number of variables to analyze in the full cohort of patients, including information on their follow-up status. The implications of information loss are twofold: it can substantially influence the statistical robustness of the study and restrict the applicability of its outcomes to a broader context. Nonetheless, we rigorously sought to mitigate this limitation. Thorough scrutiny of available data sources was conducted to maximize the information gleaned from the program. This meticulous approach aimed to minimize potential biases that could result from the loss of information, thereby enhancing the reliability and validity of our study’s findings. The absence of data on follow-up assessments during the first decade of the program prevented us to compare rates between different implementation periods. Further research is being carry out to identify factors associated with lost to follow-up after the exams. The issue of missing data can be fixed with a prospective data collection method or a registry that includes more variables and continuously captures accurate data about the patient and the program.
Conclusions
The Point of Care Diabetic Retinopathy Examination Program (POCDREP) has successfully reached comparable outcomes with other national and international DRE programs. The POCDREP has proven to be a reproducible tool for detecting DR with promising results within a range of point-of-care sites. Sustainability of the program can be attributed to the development of explicit guidelines, building a participatory and collaborative network of stakeholders from participating primary care sites, ongoing training for healthcare providers in performing DRE, and the establishment of monitoring systems to track DREs over time. Considering that the program primarily focuses on DRE, the detection of other diseases enhances overall healthcare.
Acknowledgments
To those individuals who have contributed to this endeavor, many for more than a decade, special recognition is extended to the dedicated personnel at primary care clinics and participating sites. Their pivotal role in ensuring access to the program for their patient population and fostering a sense of community effort and collaboration are a driving force behind the program’s success.
References
- 1. Bourne RR, Stevens GA, White RA, Smith JL, Flaxman SR, Price H, et al. Causes of vision loss worldwide, 1990–2010: a systematic analysis. Lancet Glob Health. 2013;1(6):e339–49. Epub 2014/08/12. pmid:25104599.
- 2. Moutray T, Evans JR, Lois N, Armstrong DJ, Peto T, Azuara-Blanco A. Different lasers and techniques for proliferative diabetic retinopathy. Cochrane Database Syst Rev. 2018;3:CD012314. Epub 2018/03/16. pmid:29543992.
- 3. Virgili G, Parravano M, Evans JR, Gordon I, Lucenteforte E. Anti-vascular endothelial growth factor for diabetic macular oedema: a network meta-analysis. Cochrane Database Syst Rev. 2017;6:CD007419. Epub 2017/06/24. pmid:28639415.
- 4. Vashist P, Singh S, Gupta N, Saxena R. Role of early screening for diabetic retinopathy in patients with diabetes mellitus: an overview. Indian J Community Med. 2011;36(4):247–52. pmid:22279252.
- 5. Simó R, Hernández C. New Insights into Treating Early and Advanced Stage Diabetic Retinopathy. Int J Mol Sci. 2022;23(15). Epub 2022/08/13. pmid:35955655.
- 6. Kollias AN, Ulbig MW. Diabetic retinopathy: Early diagnosis and effective treatment. Dtsch Arztebl Int. 2010;107(5):75–83; quiz 4. Epub 2010/02/27. pmid:20186318.
- 7. Flaxel CJ, Adelman RA, Bailey ST, Fawzi A, Lim JI, Vemulakonda GA, et al. Diabetic Retinopathy Preferred Practice Pattern(R). Ophthalmology. 2020;127(1):P66–P145. Epub 2019/11/24. pmid:31757498.
- 8. Daskivich LP, Vasquez C, Martinez C Jr., Tseng CH, Mangione CM. Implementation and Evaluation of a Large-Scale Teleretinal Diabetic Retinopathy Screening Program in the Los Angeles County Department of Health Services. JAMA Intern Med. 2017;177(5):642–9. Epub 2017/03/28. pmid:28346590.
- 9. Jani PD, Forbes L, Choudhury A, Preisser JS, Viera AJ, Garg S. Evaluation of Diabetic Retinal Screening and Factors for Ophthalmology Referral in a Telemedicine Network. JAMA Ophthalmol. 2017;135(7):706–14. Epub 2017/05/19. pmid:28520833.
- 10. Paz SH, Varma R, Klein R, Wu J, Azen SP, Los Angeles Latino Eye Study G. Noncompliance with vision care guidelines in Latinos with type 2 diabetes mellitus: the Los Angeles Latino Eye Study. Ophthalmology. 2006;113(8):1372–7. Epub 2006/06/14. pmid:16769120.
- 11. Shaw J. TeleOphthalmology-ready for prime time. Eyenet Magazine. 2016:41–5.
- 12. Pasquel FJ, Hendrick AM, Ryan M, Cason E, Ali MK, Narayan KM. Cost-effectiveness of Different Diabetic Retinopathy Screening Modalities. J Diabetes Sci Technol. 2015;10(2):301–7. Epub 2016/01/01. pmid:26719134.
- 13. Ryan ME, Rajalakshmi R, Prathiba V, Anjana RM, Ranjani H, Narayan KV, et al. Comparison Among Methods of Retinopathy Assessment (CAMRA) study: smartphone, nonmydriatic, and mydriatic photography. Ophthalmology. 2015;122(10):2038–43. pmid:26189190
- 14. Jackson BM, Gutierrez ML, Relyea GE, Carlton EL, Ahn S, Binkley BL, et al. Unmet Primary Care Needs in Diabetic Patients with Multimorbidity in a Medically Underserved Area. Health Serv Res Manag Epidemiol. 2017;4:2333392817702760. Epub 2017/05/26. pmid:28540336.
- 15. Hartstra WW, Holleman F, Hoekstra JB, Schlingemann RO. [Screening for diabetic retinopathy]. Ned Tijdschr Geneeskd. 2007;151(4):228–33. Epub 2007/02/28. pmid:17323877.
- 16. Das T, Takkar B, Sivaprasad S, Thanksphon T, Taylor H, Wiedemann P, et al. Recently updated global diabetic retinopathy screening guidelines: commonalities, differences, and future possibilities. Eye. 2021;35(10):2685–98. pmid:33976399
- 17. Owsley C, McGwin G Jr., Lee DJ, Lam BL, Friedman DS, Gower EW, et al. Diabetes eye screening in urban settings serving minority populations: detection of diabetic retinopathy and other ocular findings using telemedicine. JAMA Ophthalmol. 2015;133(2):174–81. Epub 2014/11/14. pmid:25393129.
- 18. Shi L, Wu H, Dong J, Jiang K, Lu X, Shi J. Telemedicine for detecting diabetic retinopathy: a systematic review and meta-analysis. Br J Ophthalmol. 2015;99(6):823–31. Epub 2015/01/08. pmid:25563767.
- 19. Tozer K, Woodward MA, Newman-Casey PA. Telemedicine and Diabetic Retinopathy: Review of Published Screening Programs. J Endocrinol Diabetes. 2015;2(4). Epub 2015/01/01. pmid:27430019.
- 20. Toy BC, Aguinaldo T, Eliason J, Egbert J. Non-Mydriatic Fundus Camera Screening for Referral-Warranted Diabetic Retinopathy in a Northern California Safety-Net Setting. Ophthalmic Surg Lasers Imaging Retina. 2016;47(7):636–42. Epub 2016/07/21. pmid:27434895.
- 21. Kirkizlar E, Serban N, Sisson JA, Swann JL, Barnes CS, Williams MD. Evaluation of telemedicine for screening of diabetic retinopathy in the Veterans Health Administration. Ophthalmology. 2013;120(12):2604–10. Epub 2013/10/03. pmid:24084501.
- 22. Cavallerano AA, Conlin PR. Teleretinal imaging to screen for diabetic retinopathy in the Veterans Health Administration. J Diabetes Sci Technol. 2008;2(1):33–9. Epub 2008/01/01. pmid:19885175.
- 23. Massin P, Chabouis A, Erginay A, Viens-Bitker C, Lecleire-Collet A, Meas T, et al. OPHDIAT©: A telemedical network screening system for diabetic retinopathy in the Île-de-France. Diabetes & metabolism. 2008;34(3):227–34.
- 24. Mamillapalli CK, Prentice JR, Garg AK, Hampsey SL, Bhandari R. Implementation and challenges unique to teleretinal diabetic retinal screening (TDRS) in a private practice setting in the United States. J Clin Transl Endocrinol. 2020;19:100214. Epub 2020/01/21. pmid:31956513.
- 25. Schulze-Dobold C, Erginay A, Robert N, Chabouis A, Massin P. Ophdiat((R)): five-year experience of a telemedical screening programme for diabetic retinopathy in Paris and the surrounding area. Diabetes Metab. 2012;38(5):450–7. Epub 2012/07/07. pmid:22765981.
- 26. Fonda SJ, Bursell SE, Lewis DG, Clary D, Shahon D, Cavallerano J. Incidence and Progression of Diabetic Retinopathy in American Indian and Alaska Native Individuals Served by the Indian Health Service, 2015–2019. JAMA Ophthalmol. 2023;141(4):366–75. Epub 2023/03/10. pmid:36892822 from Social Eyes Corp outside the submitted work. No other disclosures were reported.
- 27. Ogunyemi O, Terrien E, Eccles A, Patty L, George S, Fish A, et al. Teleretinal screening for diabetic retinopathy in six Los Angeles urban safety-net clinics: initial findings. AMIA Annu Symp Proc. 2011;2011:1027–35. Epub 2011/12/24. pmid:22195163.
- 28. Avendano-Veloso A, Parada-Hernandez F, Gonzalez-Ramos R, Dougnac-Osses C, Carrasco-Saez JL, Scanlon PH. Teleophthalmology: a strategy for timely diagnosis of sight-threatening diabetic retinopathy in primary care, Concepcion, Chile. Int J Ophthalmol. 2019;12(9):1474–8. Epub 2019/09/24. pmid:31544045.
- 29. Vujosevic S, Pucci P, Casciano M, Daniele A, Bini S, Berton M, et al. A decade-long telemedicine screening program for diabetic retinopathy in the north-east of Italy. J Diabetes Complications. 2017;31(8):1348–53. Epub 2017/05/30. pmid:28551296.
- 30. Hou X, Wang L, Zhu D, Guo L, Weng J, Zhang M, et al. Prevalence of diabetic retinopathy and vision-threatening diabetic retinopathy in adults with diabetes in China. Nat Commun. 2023;14(1):4296. Epub 2023/07/19. pmid:37463878.
- 31. Cavallerano AA, Cavallerano JD, Katalinic P, Blake B, Rynne M, Conlin PR, et al. A telemedicine program for diabetic retinopathy in a Veterans Affairs Medical Center—the Joslin Vision Network Eye Health Care Model. Am J Ophthalmol. 2005;139(4):597–604. Epub 2005/04/06. pmid:15808153.
- 32. Horton MB, Brady CJ, Cavallerano J, Abramoff M, Barker G, Chiang MF, et al. Practice Guidelines for Ocular Telehealth-Diabetic Retinopathy, Third Edition. Telemed J E Health. 2020;26(4):495–543. Epub 2020/03/27. pmid:32209018.
- 33. Li H, Horton M, Bursell S-E, Cavallerano J, Zimmer-Galler I, Tennant M, et al. Telehealth Practice Recommendations for Diabetic Retinopathy, Second Edition. Telemedicine journal and e-health: the official journal of the American Telemedicine Association. 2011;17:814–37. pmid:21970573
- 34.
United States Census Bureau. QuickFacts. Pittsburgh city, Pennsylvania: U.S. Department of Commerce; 2019 [cited 2023 Jul 13]. https://www.census.gov/quickfacts/pittsburghcitypennsylvania.
- 35. Krieger N, Chen JT, Waterman PD, Soobader MJ, Subramanian SV, Carson R. Geocoding and monitoring of US socioeconomic inequalities in mortality and cancer incidence: does the choice of area-based measure and geographic level matter?: the Public Health Disparities Geocoding Project. Am J Epidemiol. 2002;156(5):471–82. Epub 2002/08/28. pmid:12196317.
- 36.
United States Census Bureau. Poverty Datasets Washington D.C.: U.S. Census Bureau; 2021 [updated Nov 22, 2021; cited 2021 Nov 25, 2021https://www.census.gov/topics/income-poverty/poverty/data/datasets.html.
- 37. Wilkinson CP, Ferris FL 3rd, Klein RE, Lee PP, Agardh CD, Davis M, et al. Proposed international clinical diabetic retinopathy and diabetic macular edema disease severity scales. Ophthalmology. 2003;110(9):1677–82. Epub 2003/09/18. pmid:13129861.
- 38. Fong DS, Ferris FL 3rd, Davis MD, Chew EY. Causes of severe visual loss in the early treatment diabetic retinopathy study: ETDRS report no. 24. Early Treatment Diabetic Retinopathy Study Research Group. Am J Ophthalmol. 1999;127(2):137–41. Epub 1999/02/25. pmid:10030553.
- 39. Litvin TV, Ozawa GY, Bresnick GH, Cuadros JA, Muller MS, Elsner AE, et al. Utility of hard exudates for the screening of macular edema. Optom Vis Sci. 2014;91(4):370–5. Epub 2014/02/25. pmid:24561961.
- 40. Vloka C, Waxman E. Look beyond the retinopathy: The incidental finding of a brain tumour. J Telemed Telecare. 2022;28(3):230–2. Epub 2020/09/03. pmid:32873139.
- 41.
USA D. MCKEESPORT, PA, CENSUS PLACE 2020 [cited 2023 05-18-2023]. https://datausa.io/profile/geo/mckeesport-pa.
- 42. Bonilla-Escobar FJ. Fighting blindness with a Guerrilla: The Guerrilla Eye Service of Pittsburgh. Int J Med Students. 2017;5(3):117–20.
- 43. Gajiwala UR, Pachchigar S, Patel D, Mistry I, Oza Y, Kundaria D, et al. Non-mydriatic fundus photography as an alternative to indirect ophthalmoscopy for screening of diabetic retinopathy in community settings: a comparative pilot study in rural and tribal India. BMJ Open. 2022;12(4):e058485. Epub 2022/04/10. pmid:35396308.
- 44. Bedard C, Sherry Liu S, Patterson C, Gerstein H, Griffith L. Systematic review: Can non-mydriatic cameras accurately detect diabetic retinopathy? Diabetes Res Clin Pract. 2017;129:154–9. Epub 2017/05/22. pmid:28528076.
- 45. Vujosevic S, Limoli C, Luzi L, Nucci P. Digital innovations for retinal care in diabetic retinopathy. Acta Diabetol. 2022;59(12):1521–30. Epub 2022/08/13. pmid:35962258.
- 46. Riordan F, Racine E, Phillip ET, Bradley C, Lorencatto F, Murphy M, et al. Development of an intervention to facilitate implementation and uptake of diabetic retinopathy screening. Implement Sci. 2020;15(1):34. Epub 2020/05/21. pmid:32429983.
- 47. Cho Y, Park HS, Huh BW, Seo SH, Seo DH, Ahn SH, et al. Prevalence and risk of diabetic complications in young-onset versus late-onset type 2 diabetes mellitus. Diabetes Metab. 2022;48(6):101389. Epub 2022/10/19. pmid:36255061.
- 48.
Data USA, Pittsburgh, PA [Internet]. 2021 [cited 03-29-2024]. https://datausa.io/profile/geo/pittsburgh-pa/#:~:text=Between%202020%20and%202021%2C%20the,from%205.35%25%20to%205.46%25.
- 49. Bifulco L, Grzejszczak L, Velez I, Angelocci T, Anderson D. A qualitative investigation of uninsured patient and primary care provider perspectives on specialty care eConsults. BMC Health Serv Res. 2023;23(1):1133. Epub 2023/10/21. pmid:37864170.
- 50. Lundeen EA, Burke-Conte Z, Rein DB, Wittenborn JS, Saaddine J, Lee AY, et al. Prevalence of Diabetic Retinopathy in the US in 2021. JAMA Ophthalmol. 2023;141(8):747–54. Epub 2023/06/15. pmid:37318810
- 51. Piché ME, Tchernof A, Després JP. Obesity Phenotypes, Diabetes, and Cardiovascular Diseases. Circ Res. 2020;126(11):1477–500. Epub 2020/05/22. pmid:32437302.
- 52. Yang GR, Li D, Xie Z. Association of obesity with diabetic retinopathy in US adults with diabetes in a national survey. Endocr Connect. 2021;10(7):725–30. Epub 2021/06/12. pmid:34114968.
- 53. Gao S, Zhang H, Long C, Xing Z. Association Between Obesity and Microvascular Diseases in Patients With Type 2 Diabetes Mellitus. Front Endocrinol (Lausanne). 2021;12:719515. Epub 2021/11/13. pmid:34764938.
- 54. Lautsch D, Boggs R, Wang T, Gonzalez C, Milligan G, Rajpathak S, et al. Individualized HbA(1c) Goals, and Patient Awareness and Attainment of Goals in Type 2 Diabetes Mellitus: A Real-World Multinational Survey. Adv Ther. 2022;39(2):1016–32. Epub 2021/12/25. pmid:34951678.
- 55. Standards of Care in Diabetes-2023 Abridged for Primary Care Providers. Clin Diabetes. 2022;41(1):4–31. Epub 2023/01/31. pmid:36714254.
- 56. Almutairi NM, Alahmadi S, Alharbi M, Gotah S, Alharbi M. The Association Between HbA1c and Other Biomarkers With the Prevalence and Severity of Diabetic Retinopathy. Cureus. 2021;13(1):e12520. Epub 2021/02/11. pmid:33564524.
- 57. Negi A, Vernon SA. An overview of the eye in diabetes. J R Soc Med. 2003;96(6):266–72. Epub 2003/06/05. pmid:12782689.
- 58. Erdem U, Gundogan FC, Dinc UA, Yolcu U, Ilhan A, Altun S. Acute effect of cigarette smoking on pupil size and ocular aberrations: a pre- and postsmoking study. J Ophthalmol. 2015;2015:625470. Epub 2015/02/24. pmid:25699189.
- 59. Jo K, Chang DJ, Min JW, Yoo YS, Lyu B, Kwon JW, et al. Long-term prediction models for vision-threatening diabetic retinopathy using medical features from data warehouse. Sci Rep. 2022;12(1):8476. Epub 2022/05/20. pmid:35589921.
- 60. Nentwich MM, Ulbig MW. Diabetic retinopathy—ocular complications of diabetes mellitus. World J Diabetes. 2015;6(3):489–99. Epub 2015/04/22. pmid:25897358.
- 61. Liu Y, Torres Diaz A, Benkert R. Scaling Up Teleophthalmology for Diabetic Eye Screening: Opportunities for Widespread Implementation in the USA. Curr Diab Rep. 2019;19(9):74. Epub 2019/08/04. pmid:31375932.
- 62. Jeong IS, Lee EJ. Current Status and Associated Factors of Annual Eye Examination Among People with Type 2 Diabetes Mellitus: Using the 7th National Health and Nutrition Examination Survey. Asian Nurs Res (Korean Soc Nurs Sci). 2021;15(4):239–46. Epub 2021/07/28. pmid:34314881.
- 63. Abdelmotaal H, Ibrahim W, Sharaf M, Abdelazeem K. Causes and Clinical Impact of Loss to Follow-Up in Patients with Proliferative Diabetic Retinopathy. J Ophthalmol. 2020;2020:7691724. Epub 2020/02/25. pmid:32089871.
- 64. Suresh R, Yu HJ, Thoveson A, Swisher J, Apolinario M, Zhou B, et al. Loss to Follow-Up Among Patients With Proliferative Diabetic Retinopathy in Clinical Practice. Am J Ophthalmol. 2020;215:66–71. Epub 2020/03/27. pmid:32209344.
- 65. Avidor D, Loewenstein A, Waisbourd M, Nutman A. Cost-effectiveness of diabetic retinopathy screening programs using telemedicine: a systematic review. Cost Eff Resour Alloc. 2020;18:16. Epub 2020/04/14. pmid:32280309.