Outcome.

To determine if DR was present in the first phase of the ShECS, participants were asked about their DR diagnosis status. Morovere, participants recieved two occular examinations in the first and second phase of the ShECS. During these stages of the ShECS, ophthalmologists used direct or indirect ophthalmoscopy to examine the retina. To diagnose DR, fundus imaging was also used. A Nidek AFC-230 Fundus Camera (Nidek, Chiyoda-ku, Japan) was used for fundus photography.

Predictors.

As a review of the literature is a common approach for narrowing down the list of candidate predictors [15], the most relevant candidate predictors, that could be readily available in clinical practice and did not require an ophthalmological examination, were identified based on a review of the literature. All measurements were based on the ShECS protocol.

Participants were asked about their birth date, smoking status (cigarette, pip, or hookah), and diabetes duration (time since diabetes diagnosis) via a face-to-face interview [11]. The participants’ sex was determined based on the investigator’s observations. Trained nurses measured SBP and diastolic blood pressure (DBP) using a digital blood pressure sphygmomanometer [16]. Mean blood pressure (MBP) was defined as (2 × DBP + 1 × SBP)/3. Fasting blood glucose (FBG), HbA1c, and blood lipid levels were measured after 12–14 hours of fasting [17]. Non-fasting blood glucose (BG) was assessed [18]. Body weight was measured using digital scales, and body height was measured in a standing position [19] and BMI was subsequently calculated as the ratio of weight (in kilograms) to the square of height (in meters). Waist circumference (WC) was measured at the highest part of the iliac crest using a non-stretchable tape [17]. A non-elastic tape was used to measure hip circumference (HC) in the biggest pelvic circumference region while the subject was standing (personal communication). Then, WC divided by HC was used to compute the waist-to-hip ratio (WHR). Pulse rate was recorded as number of beats per minute (bpm) using a digital sphygmomanometer (personal communication).

Descriptive statistics.

For categorical variables, frequencies and percentages were used as the descriptive measures. Quantitative variables were described using mean, and standard deviation. In cases where quantitative variables were not near a normal distribution, the median and interquartile range were reported along with the mean and standard deviation.

Development and validation of the prediction models.

Because not all the predictors were measured simultaneously during the phase one of the ShECS, three separate models were developed, as illustrated in the Fig 1. Models M1 and M3 predict 5-year risk of DR, and M2 predicts the 10-year risk. Log-binomial, logistic, and Poisson regression models with robust standard error estimates were used to achieve the optimal risk prediction model. However, only the results of the logistic regression were presented since the log-binomial regression was unable to converge and there was no simple way to evaluate the calibration of the Poisson models. The predictors included in each model are described in the Fig 1. Since SBP in M1, DBP in M2, and both blood pressures in M3 were statistically significant predictors, each of the three models was fitted separately regarding SBP-DBP and MBP to utilize all available information on the blood pressure variable. We used a complete case analysis so that participants with missing values were excluded from the model-building and model-validation processes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. The model development process.

Predictors considered for developing M1 included: sex, smoking, age, diabetes duration, blood pressure, body mass index, and non-fasting blood glucose. Predictors considered for developing M2 included: sex, smoking, age, diabetes duration, blood pressure, body mass index, and non-fasting blood glucose. Predictors considered for developing M3 included: sex, smoking, age, diabetes duration, blood pressure, body mass index, HbA1c, total cholesterol, high-density lipoprotein cholesterol, triglycerides, fasting blood glucose, pulse rate, abdomen circumference, hip circumference, abdominal to hip ratio. a: systolic and diastolic blood pressure were included as separate predictors. b: mean blood pressure ((2*DBP + 1*SBP)/3) was included.

https://doi.org/10.1371/journal.pone.0325814.g001

Model development. 

A univariable regression analysis was conducted, following the criterion of a P-value0.2, to examine unadjusted associations. However, key predictors were not excluded solely based on statistical non-significance in the univariable analysis [20]. Subsequently, a multivariable analysis was performed using a backward elimination approach, focusing on the remaining predictors. In multivariable regression, predictors and interaction terms with P-value0.05 were retained in the final models.

We considered a generalized additive model (GAM) and corresponding plots in univariable and multivariable analyses to check the nonlinearity of associations between outcome and continuous predictors. An appropriate polynomial term was supposed to be included in the model if there was evidence of a nonlinear association; if not, the predictors were thought to be linear. To find potentially influential observations, a modified Cook’s distance plot was used. If any influential observations were found, a sensitivity analysis was carried out, in which the final model was repeated without the influencing points. To detect multicollinearity, which is a high correlation among predictors, we used the variance inflation factor (VIF). A VIF above 10 was considered as high collinearity.

Model performance measures.

• Discrimination, that is the ability of a prediction model to discriminate among the patients with and without future outcomes was quantified using Harrell’s concordance (c) statistic, which was equal to the area under the Receiver Operating Characteristic curve (AUC).
  • Calibration, which specifies the agreement between the observed, and predicted outcomes of all participants, was graphically illustrated using calibration plots and statistically tested using Hosmer–Lemeshow goodness-of-fit test. A p-value of greater than 0.05 indicated a good fit. Moreover, the expected (predicted) to observed (E/O) ratio was calculated by dividing the expected to observed outcome risks.

Model validation.

• Three distinct approaches were applied to assess internal validation of final models: 1) for apparent validation, performance measures were assessed directly on the data used to drive the models, resulting in optimistic performance estimates, 2) following the approach outlined by Kuhn and Johnson [21] cross-validated performance measures were derived using a stratified 10-fold cross-validation method, 3) following the approach outlined by Steyerberg, performance measures were additionally obtained using a bootstrap validation with 200 bootstrap samples, that reflect future performance in new patients [22]. However, there was a significant amount of random error in the performance measures in the 10-fold cross-validation since there weren’t many outcomes in certain folds. Rather, more stable measures were established using bootstrap validation. Therefore, only performance measures using bootstrap validation approach were reported.
• We compared the apparent c-statistic and calibration plots with those after bootstrap validation. In the case of a considerable drop in performance measures, it was intended to recheck the models in terms of the number of included predictors in relation to the sample size.

Decision-curve analysis.

A decision curve was formed using net benefit (NB) on y-axis, and thresholds of predicted probabilities on x-axis. The NB was defined as (TP –wFP)/N, where “w” indicated a weight equal to the odds of selected threshold (Pt/1-Pt), which is considered as the harm to benefit ratio of treatment. For example, at the threshold of 10%, the false-positives is valued at one-ninth of true-positives. “Pt” is the threshold probability, “TP” was the number of true-positives, “FP” is the number of false-positives, and N was the total number of subjects [23]. To determine the model line, we plotted NBs against a broad range (0–1) of predicted probability thresholds. Additionally, two additional lines were drawn to illustrate the default strategies of “treat none” and “treat all” without the use of the prediction model. We regarded NB as zero for the “treat none” line, and we drew the “treat all” line depending on the thresholds and event rate [24,25].

Using the decision curve of each model, the interval between the intersection points of model’s line with both lines of default strategies was considered the reasonable interval of thresholds for prediction. As is usual, the statistical optimum decision threshold was determined based on Youden’s J statistic (J = model sensitivity model specificity−1). Decision curves were plotted using both apparent and bootstrap validation approaches.

Regarding the use of cluster sampling, robust standard errors and p-values were considered to correct the design effect. STATA₁₇ (StataCorp, USA), and R_4.2.3 statistical software were used to perform the statistical analyses.

Disease course.

Previous studies established that a longer diabetes duration is a major non-modifiable risk factor for the onset of DR; a recent systematic review revealed that diabetes duration was associated with DR development with a pooled odds ratio (OR) of 1.03 (95% CI: 1.02, 1.03) [26]. Other previous epidemiological studies confirmed a relationship between long duration of diabetes and DR [27–29]. The present study discovered a statistically significant association between the length of diabetes and DR, which is consistent with previous research. The impact of chronic exposure to hyperglycemia, which causes microvascular damage, is probably the cause of this connection. However, because of the impossibility of intervention in this factor, it can only be considered a significant predictor to identify high-risk patients and instead focus on modifiable risk factors.

Blood glucose.

After the diabetes duration, hyperglycemia is reported to be the most relevant risk factor for the development of DR [30]. In a systematic review, and meta-analysis, 2-hour postprandial blood glucose, FBG, and HbA1c were reported to be associated risk factors of DR with the pooled OR of 1.94 (95% CI: 0.81, 4.65), 1.33 (95% CI: 1.12, 1.59), and 1.15 (95% CI: 1.09, 1.20), respectively [31]. The results of our study revealed that BG was a statistically significant predictor across all multivariable models where this variable was included. A similar pattern was observed in the case of FBG. These results are in agreement with those of prior research, suggesting that elevated blood glucose levels may contribute to the development of DR. Furthermore, HbA1c is a widely recognized glycemic control indicator, and its influence on the development of DR has been thoroughly examined in prior research [26,31,32]. Nevertheless, our investigation demonstrated a detrimental association. One possible explanation is that HDL-C has a protective effect, and the regression models included an interaction effect between HbA1c and HDL-C. Therefore, HbA1c may have different effects on DR depending on HDL-C levels.

Blood pressure.

Several studies suggested that high blood pressure can contribute to the development of DR. A large population-based cross-sectional study discovered an association between hypertension and DR with an adjusted OR of 1.08 (95% CI: 1.04, 1.11) [33]. Moreover, in another population-based study, poorly controlled and untreated hypertension were associated with DR with adjusted OR of 1.97 (95% CI: 1.39, 2.83), and 2.01 (95% CI: 1.34, 3.05), respectively. Systolic blood pressure was associated with DR with an adjusted OR of 1.45 (95% CI: 1.28, 1.65) [34]. The results of present study showed that SBP in M1, DBP in M2, and both variables in M3 were significantly associated with DR. These findings indicate the predictive capability of both variables. Therefore, prediction models were developed by combining these two variables as the MBP. Mean blood pressure was a statistically significant predictor of both M1 and M2.

Lipid profile.

Dyslipidemia is a common comorbidity of diabetes [35,36], and can be involved in the development of DR [37]. Research findings on the association between HDL-C and DR are conflicting. While some studies suggest that HDL-C plays a protective role against DR development [38], others indicate a positive association between HDL-C levels and DR occurrence [39]. Therefore, a study revealed an inverted U-shaped association between HDL-C and DR [40]. Our findings indicate a statistically significant protective effect of HDL-C on DR in univariable and multivariable regression models. There is a discrepancy among the studies regarding the association of TG and DR [37,41,42]. Triglycerides was identified as a detrimental risk factor in our investigation. A positive association between TC and DR has been substantiated by numerous investigations [37,43]. In contrast, the current study did not demonstrate a statistically significant relationship between TC levels in either univariable or multivariable analyses. This discrepancy may be due to the small sample size and consequent lack of statistical power to detect a significant association.

Model development.

After incorporating the significant predictors from the univariable regression analyses of this study, three predictor variables, including diabetes duration, MBP, and BG constructed final suggested M1 and M2. In addition to these variables, HbA1c, FBG, HDL-C, and TG levels were involved in the development of suggested M3.

Several DR risk prediction models were developed and validated in various populations. A recent population-based cohort study in China developed a model to predict the risk of vision-threatening DR including predictors of age, BMI, SBP, diabetes duration, and HbA1c, with a c-statistic of 0.72 and 0.68, respectively, for the training and testing set, and a good calibration [44]. In addition, a model was fitted using diabetes duration, HbA1c, FBG, SBP, proteinuria, BMI, and education level, as determined by cross-sectional study data from Iran. The model exhibited a c-statistic of 0.76 and a high degree of agreement between the predicted and observed risk [7]. Another risk prediction algorithm by Aspelund et al. was created to calculate the risk of sight-threatening DR in a time period ranging from 6 to 60 months based on clinical data, including type and duration of diabetes, HbA1c, SBP, presence and DR grade, from an Icelandic type 1 and 2 population with diabetes. The c-statistic of the algorithm was found to be 0.76 [45]. A validation study by Rao et al., applying the data of T2DM patients from Australia, New Zealand, and Finland, showed that the Aspelundˈs algorithm predicted two- and five-year risk of sight-threatening DR well with a c-statistic of 0.86 and 0.86, respectively [46].

Several predictors in our risk prediction models are common to existing models, and their significance in DR development has been previously documented in the literature, as mentioned earlier.

Model performance.

Discrimination analysis of all models in this study, using the c-statistic, demonstrated acceptable power to discriminate between low- and high-risk individuals, even after correcting for optimism. The c-statistic shows the probability that, of two people, one will have DR within the next 5 or 10 years and the other will not, with the patient who will acquire DR having a larger predicted risk than the patient who will not. The discriminatory ability of prediction models in our study was consistent with that in previous evidence [7,44,45]. Including key predictors may be a reason for achieving moderate-to-good discrimination power in this study, and consistency with other studies. However, we observed lower discrimination than that in the model validated by Rao et al. [46], which may be due to differences among populations and outcome spectrum. Furthermore, in all the suggested models, the mean predicted values were close to the mean observed outcomes (E/O ratio close to 1), indicating good calibration, except in higher risks (graphically tested). The latter implies a relatively poor calibration at higher risks, which may be due to the lower number of observed DR cases.

Clinical usefulness.

In addition to discrimination and calibration, we assessed the ability of models to make better clinical decisions than settings without using the model. For M1, no difference was observed between the “treat all” and “treat none” strategies and the model line at predicted risk levels below 0.12 and above 0.6, with a proposed cut-off point of 0.24. Using this model in clinical settings, individuals with a predicted risk over 0.24 should be categorized as positive for developing DR in the next five years and need care, while other patients will be classed as negative. This cut-off point was thought of as a decision threshold. Moreover, due to M2, the appropriate threshold interval was found to be 0.22 to 0.43 with a proper cut-off point of 0.32 to classify patients based on 10-year predicted risk. Decision curve of M3 also showed that in the predicted risk of less than 0.12, there is no difference between “treat all” approach and the prediction model. Hence, in the predicted risk higher than 0.37, there is no difference between “treat none” and the model line. The most appropriate decision threshold for M3 was 0.20. Previously, a clinically useful predicted risk threshold interval of 21–51% by Mo et al. [47], 2–85% by Wang et al. [48], and 11–95% by Liu et al. [49] were proposed for Chinese population. However, to our knowledge, such an interval has not been reported in the Iranian population. We suggested the most suitable cut-off point based on statistical considerations to categorize people as future cases or non-cases of DR, but healthcare providers are free to select any other reasonable classification cutoff within the clinically useful interval based on the patient’s condition or the importance of false positives or false negatives.