https://orcid.org/0000-0002-4504-7721
Figures
Abstract
Background and objective
Diabetes Mellitus (DM) is a chronic disease with a high worldwide prevalence. Diabetic foot is one of the DM complications and compromises health and quality of life, due to the risk of lower limb amputation. This work aimed to build a risk classification system for the evolution of diabetic foot, using Artificial Neural Networks (ANN).
Methods
This methodological study used two databases, one for system design (training and validation) containing 250 participants with DM and another for testing, containing 141 participants. Each subject answered a questionnaire with 54 questions about foot care and sociodemographic information. Participants from both databases were classified by specialists as high or low risk for diabetic foot. Supervised ANN (multi-layer Perceptron—MLP) models were exploited and a smartphone app was built. The app returns a personalized report indicating self-care for each user. The System Usability Scale (SUS) was used for the usability evaluation.
Results
MLP models were built and, based on the principle of parsimony, the simplest model was chosen to be implemented in the application. The model achieved accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of 85%, 76%, 91%, 89%, and 79%, respectively, for the test data. The app presented good usability (93.33 points on a scale from 0 to 100).
Citation: Ferreira ACBH, Ferreira DD, Barbosa BHG, Aline de Oliveira U, Aparecida Padua E, Oliveira Chiarini F, et al. (2023) Neural network-based method to stratify people at risk for developing diabetic foot: A support system for health professionals. PLoS ONE 18(7): e0288466. https://doi.org/10.1371/journal.pone.0288466
Editor: Melissa Orlandin Premaor, Federal University of Minas Gerais, BRAZIL
Received: September 27, 2022; Accepted: June 27, 2023; Published: July 13, 2023
Copyright: © 2023 Ferreira et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data file is available from the Mendeley database (https://data.mendeley.com/datasets/hd7wtgs7dz/1). Ferreira, Danton; Resende, Igor; Oliveira, Henrique; Ferreira, Ana Claudia; Lopes, Maria Helena; Anjos, Andre (2022), “Data for: Competitive Neural Layer-based Method to Identify People with High Risk for Diabetic Foot”, Mendeley Data, V1, doi: 10.17632/hd7wtgs7dz.1.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
1. Introduction
Diabetes Mellitus (DM) and its complications are becoming one of the biggest causes of morbidity and mortality in the world. In 2017, 4 million people died from diabetes and its complications. In 2019 there were 463 million people, between 20 and 79 years of age, diagnosed with DM worldwide, with a projected increase of 51% by 2045 [1, 2].
Neuropathy is a complication often found in people with diabetes, which compromises the functioning of nerves throughout the body and alters sensory, motor, and autonomic functions. The presence of neuropathy makes the person even more prone to the development of diabetic foot, especially due to the loss of sensation in the feet [3, 4]. Diabetic foot is a complication that affects up to 25% of people with diabetes throughout their lives and is a serious health problem among chronic patients, possibly leading to lower limb amputations. It represents a relevant socioeconomic impact due to the loss of ability to work, affects social life and the quality of life, with high morbidity, disability, and mortality, and can be classified as one of the most devastating complications of diabetes [1, 4–6].
The International Diabetes Federation argues that a greater focus on the prevention of foot ulcers of people with diabetes is needed, rather than focusing on treating them, as there is enormous difficulty in treating and recuperating the limb after the injury has already been established [1]. For the prevention of diabetic foot, it is usually necessary to screen people with diabetes, through clinical examination, which generates a risk score for the patient to develop the diabetic foot [7, 8]. Screening requires time, knowledge, training, and skill from the professional (nurse or doctor) for an effective result. Furthermore, there are a lot of services that need to be performed daily by the health unit professionals [9], which makes it difficult to examine all people with diabetes [10]. To prioritize the clinical examination of those at most risk for developing this complication, alternative methods are needed to quickly identify, people more susceptible to develop diabetic foot.
Artificial intelligence (AI) has been widely used in the health area to assist health professionals in decision making with a focus on screening, classification, and diagnostics, among other possibilities. The work reported in [11] developed a computer vision approach for the classification of skin cancer using hybrid texture features. They achieved an overall accuracy of 97.13% using a multi-layer perceptron (MLP). An MLP is an ANN capable of handling both linearly separable and non-linearly separable data. It belongs to a class of neural networks known as feed-forward neural networks, which connect neurons in one layer to the next layer in a forward manner. The interconnected neurons process data through three or more layers. The basic structure of an MLP consists of an input layer, one or more hidden layers and an output layer. Due to their simplicity, MLPs usually require short training times to learn the representations in data and to produce an output. Alternatively, deep neural networks (DNNs), which are artificial neural networks with deep layers, have also been employed for developing health decision making systems. Due to the automatic feature extraction ability, deep learning methods have been successfully applied in different areas, especially in the field of medical imaging [12]. To investigate spine pathologies, the authors of the work reported in [13] proposed deep learning primitives and stacked Sparse autoencoder-based patch classification modeling for Vertebrae segmentation from Computed Tomography (CT) images. A deep learning approach was also developed in [14] for automatic CT vertebra segmentation, which significantly reduced computational cost and achieved excellent accuracy in vertebra segmentation. The Convolutional Neural Network, which is a type of DNN, was employed for Liver Segmentation in Medical Diagnosis [15]. Liver segmentation in CT scan images is a significant step toward the development of a quantitative biomarker for computer-aided diagnosis [16]. In the work reported in [17], the Convolutional Neural Network (CNN) and Variational Autoencoder deep learning models were used for feature extraction to build a binary and multiclass classification system for supporting COVID-19 triage based on computed tomography data. Despite their ability to deal with complex problems, DNNs usually require longer training periods of time. Additionally, they rely on powerful computers with specialized processing units such as Tensor Processing Units (TPU) and Neural Processing Units (NPU).
Regarding diabetic foot, some AI-based methods have been developed to identify people at risk for diabetic foot [18–22] while others focus on changes already present in feet of people with DM [23–28]. In the research presented in [19] the authors used a questionnaire and investigated the presence of risk factors for diabetic foot. Data analysis was performed using Self-Organizing Maps (SOM), which led to the identification of two different groups of participants. A high-risk group was identified based on having the highest number of risk factors present among the participants. In the research reported in [20], the authors used the same database introduced in [19]; however, they employed a competitive neural layer (CNL) with only two neurons for the separation of groups. The CNL approach outperformed the SOM approach, with this finding being confirmed by specialists in diabetic foot and statistical tests.
All the aforementioned AI-based works used unsupervised algorithms. Unlike these, the present study aimed to build a supervised ANN for the identification of people at risk for diabetic foot. For the construction of this new model, a database with the information of participants with diabetes was labeled according to the risk for diabetic foot (high or low risk) by diabetic foot specialists, which guided the supervised learning of the ANN. In addition, a further study was carried out on the available variables to reduce the high risk for diabetic foot contributory factors of high risk for diabetic foot and make the data collection more efficient for the end-user.
The primary goal of the proposed risk classifier is to identify people with diabetes who require immediate intervention. By doing so, health professionals can anticipate and prioritize their care. This early evaluation followed by a clinical examination of the feet may direct personalized actions that could help prevent diabetic foot.
2. Database
To design (training and validation) the proposed method, a database consisting of 54 variables identified in 250 people with diabetes was used. This database is presented in [20]. These data were collected via questionnaire in a health institution of a city in the state of Minas Gerais, Brazil, from people aged 18 years or over, who agreed to participate in the research study and signed the consent form. The anonymity of the participants’ data was ensured, and the data collection started only after authorization from the institution (CAAE 66815617.3.00005404).
The participants were chosen by convenience sampling. The instrument (questionnaire) used for data collection covered the risk factors for the development of diabetic foot proposed by the International Consensus on Diabetic Foot [29], and by the Primary Care Booklet to Assist People with DM from the Brazilian Ministry of Health [30], which resulted in a total of 54 variables. Variables related to people’s self-care habits concerning their health, foot care, and perceived changes in the feet, as well as sociodemographic and socioeconomic variables were included. As detailed in [20], each entry in the dataset contained the subject’s income, profession, gender, age, years of education, marital status, body mass index (BMI), smoking habit, alcohol use, type of diabetes, presence of hypertension, circulatory problems, burning sensation in the feet, presence of bunions, examination of the feet, wrinkles between the toes, habit of washing the feet, calluses on the feet, type of footwear, time between buying shoes, walking habits, blood glucose value, whether blood glucose is monitored, food control, current or previous ringworm in the feet, pain in the legs and feet, current or previous cracks in the feet, current or previous blister on the feet, shock in feet, warm and reddish feet, removal of cuticles from toes, type of toenail cutting, sock type, action taken when diabetic foot occurred, checking inside the shoe, position when watching television/reading, length of time standing or walking, type of treatment, length of time with diabetes, practice of physical activity, tingling in the feet, loss of foot sensitivity, numbness in the feet, sitting on the feet, edema in the feet, change in foot structure, moisturizing of the feet, vision problems, current or previous ingrown toenails, use of warm bag on feet, current or previous foot injuries, and the presence of leg/foot amputation. In total, each sample (patient input) in the database contained 54 variables. The data were encoded in which the categorical variables were transformed into floating-point numbers -1.0 (meaning that the value presents lower risk for the diabetic foot condition) or +1.0 (meaning that the value presents higher risk for the diabetic foot condition) [20].
The data of each participant was analyzed, independently, by two diabetic foot experts that classified each person as low or high risk for diabetic foot. A third expert analyzed the discordant cases to obtain a final consensus. Based on their professional experiences and the literature, the experts indicated the risk (high risk or low risk) that each of the participants had for the development of diabetic foot. In total, 169 people were identified by the experts as being at high risk for diabetic foot development and 81 at low risk.
To test the designed method, a new database was constructed. This test database was composed of 141 participants, according to the sample calculation for finite populations [31]. This sample included people, 18 years of age or more, with diabetes registered in a Primary Health Unit (PHU) with the Family Health Strategy Team (Brazilian mode of health care) of a city in Minas Gerais, Brazil, who agreed to participate in the research by signing the consent form. Data collection started after authorization from the institution (CAAE 66815617.3.00005404).
For data collection, a computer program was built with all the questions in the questionnaire. The literal information was automatically coded by the program. After data collection, the information of each participant was analyzed individually by the three experts in diabetic foot. The experts indicated the risk that the 141 participants had for the development of diabetic foot, classifying them as high or low risk. The data were presented to the supervised ANN for its final performance evaluation.
3. Data normalization
Data normalization is important to avoid polarization in the training of the neural network [32]. The non-categorical variables of both databases were standardized within the range [–1 1] according to:
(1)
where x is the database variable to be normalized and xmin and xmax are the maximum and minimum values of the corresponding variable.
4. Method design
The design steps of the proposed method are presented by the block diagram shown in Fig 1.
The data were randomly divided into 10 folds of 25 patients each (totaling 250), following the Cross-validation methodology [33].
The use of a multilayer perceptron (MLP) neural network was proposed, in which three approaches were developed (I, II, and III):
I—Approach that does not require a variable selection step. In this case, all 54 variables are used by the MLP model;
II—Approach that uses as inputs only the variables identified as most relevant through multivariate analysis, as presented in [20];
III—Approach that uses as inputs only variables selected by the Fisher’s Discrimination Ratio (FDR) [34];
In summary, the input of the MLP for approach I is a vector with 54 values corresponding to the 54 variables described in Section 2, after data codification and normalization. Approaches II and III used fewer variables (only the selected ones) than approach I as input for the MLP, due to the specific feature selection step employed by each approach.
The MLP models were configured with an intermediate layer, in which the number of neurons was chosen experimentally, and the logarithmic sigmoid activation function was used. The output layer was composed of two neurons (each one representing a class: high risk or low risk), the target was defined as [1 0] for high risk and [0 1] for low risk. The softmax activation function was used in the output layer since it normalizes the neurons’ output to a probability distribution.
The training algorithm was the Bayesian regularization backpropagation [35]. This minimizes the linear combination of the square of errors and weights, determining the best combination that produces a network that generalizes well. This algorithm updates the weights according to the Levenberg-Marquardt optimization algorithm.
As performance measures of the models, accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated. To compare the generated models, the Friedman tests and the Dunn-Bonferroni post-test [36] were applied. The analysis was performed using the statistical software SAS version 9.4 and SPSS version 23, considering a significance level of 5%.
Finally, an app, called CARPeDia, implementing the best MLP model achieved, was built, providing a simple and easy-to-use interface. The app was designed by using the JavaScript, Hypertext Markup Language (HTML) and Cascading Style Sheets (CSS) languages. The System Usability Scale (SUS) [37] was employed for the usability evaluation of the developed app.
5. Results
This section presents the results for both the validation and testing datasets.
5.1 Results for validation data
Mean accuracy values over the validation data of the 10 folds used during the models’ training are presented.
For approach I, the number of neurons in the intermediate layer was varied from 5 to 30 and the best results were obtained with 10 neurons. Therefore, the final model for this approach (MI) had 54 entries, referring to all 54 variables in the database, 10 neurons in the middle layer, and 2 output neurons, that is, 54x10x2.
For approach II, a model based on 29 database variables (MII) was proposed, these being the variables indicated as the most relevant in the study reported in [20]. The final configuration of the best model architecture was 29x10x2.
For approach III (MIII), the FDR was applied to select the most relevant variables, that is, those that most contribute to the classifier’s performance. The FDR result is shown in Fig 2, where variables with the highest J value are considered the most relevant.
From this result, three models were developed in approach III, namely:
- MIII1: considers the 10 most relevant variables, being represented by the following questions (in order of relevance): 23-Do you feel tingling in your legs and feet?; 19- Have you ever felt a loss of sensation in your feet?; 24-Do you feel numbness in your legs and feet?; 28-Do you feel stinging in your legs and feet?; 26-Do you feel shocks in your legs and feet?; 53-Do you have/have you had any sores and/ or bruises on your feet?; 25-Do you feel a burning sensation in your legs and feet?; 38- Do you have/have you had ingrown toenails?; 43- Do you use or have you used a hot water bottle on your feet, and/or soaked your feet in hot water?; 29-Have you noticed any difference in your foot bones? Fig 3 demonstrates the distributions of the answers to these questions over the groups.
- MIII2: considers the 20 most relevant variables—including the previous ten, and
- MIII3: considers the 30 most relevant variables—including the previous twenty.
The number of neurons in the middle layer was varied from 5 to 30 and the final configurations for each model were 10x10x2 (MIII1), 20x20x2 (MIII2), and 30x20x2 (MIII3).
The mean values of the 10 folds of these models for sensitivity, specificity, accuracy, PPV, and NPV were compared and are shown in Table 1. Note that there were no statistically significant differences between the models. Considering the principle of parsimony [38], it was conclude that for approach III, the MIII1 model (10x10x2—with ten variables at the entrance) was the best model to be used.
Models MI, MII and MIII1 were compared with the results shown in Table 2. A significant difference in relation to the PPV in the Friedman test was found, however, the Dunn-Bonferroni post-test showed that there was no significant difference between the models. Therefore, as the model MIII1 was the simplest architecture, 10x10x2, and following the principle of parsimony, this was the most suitable.
5.2 Results for testing data
This section presents the results of the best model (approach III—MIII1) for the testing database (141 patients). The results are shown in Table 3. The result of the competitive network, developed in [20] was also presented, for the same testing database, for comparison purposes. It was observed that the MIII1 stood out for obtaining the best accuracy, best sensitivity, and best NPV. Also, using a questionnaire with only 10 questions instead of 54 is preferable for a risk stratification system since it takes less time to apply.
6. Discussion
The high sensitivity of the system is important to identify those patients who are at high risk for diabetic foot, helping healthcare professionals to identify those who need immediate care.
The high specificity of the system also demonstrates that people with DM, when using the system and finding a result that confirms that they are not at high risk for diabetic foot, will be able to continue to monitor the health of their feet at home. They can do this without seeking healthcare service, either by using the system on their own cell phone or through a health agent who can apply the system to the patient during home visits. The high PPV and NPV values found confirm the strong capacity of the proposed system.
When analyzing the variables that were most discriminating for the classification of participants according to approach III, it was observed that they are constantly presented in the literature as risk variables for diabetic foot [1, 4, 30]. There is a strong relationship between the variables and the triad of risk factors for ulceration: neuropathy, deformity, and trauma. All ten variables indicated by the approach III model are related to the triad, which reinforces the system’s ability to search for the characteristics of greatest risk for ulceration [4, 39].
The complication is characterized as a neuropathic foot when the presence of changes in the sensitivity of the feet is perceived, which often comes with sensations such as tingling, claudication, numbness, burning, and shocks [4]. Once present, the loss of protective sensitivity is characterized as a strong point for the appearance of a foot injury, since, when injured, the person will not notice the pain, and will not treat the injury. Fig 3 shows that these sensations are less perceived by the participants who were classified as low risk for diabetic foot. Conversely, the presence of these sensations was frequently mentioned by the participants in the group classified as high risk.
With the loss of sensation, factors such as the presence of ingrown toenails, and the use of hot water on the feet can be considered precursor factors for trauma, which can cause great tissue damage in the feet [2, 13].
In addition to these findings, it is also possible to perceive the atrophy of the interosseous musculature, enlargement of the plantar arch and "claw" toes, which can characterize the observation of the participants regarding the "difference in the bones of the feet", that is, the deformity. Areas of the foot that experience high pressure are more prone to the appearance of injuries, and tissue damage. Therefore, it is important to identify these areas early in order to provide adequate protection, with adequate shoes and equipment that provide comfort and support for the feet [4, 39].
The presence of foot injury has been considered by the literature as a risk factor for diabetic foot, because once the patient had an injury, they are vulnerable to new occurrences. The patient should investigate the causes of the current and/or previous injury and take step to prevent its recurrence [13, 39].
The presence of characteristics considered to be risk factors for diabetic foot was lower in the group classified as low risk, mainly when the ten most relevant variables (Fig 3) were considered. This corroborates the results achieved by approach III (MIII1), which presented high specificity. Once all these risk characteristics, or some of them, are present in the individual and are identified by the system, a risk score for diabetic foot may be created [4, 30].
Regarding the use of machine learning techniques to assess the risk of developing diabetic foot, recent research has focused on capturing the images of the feet for processing [40, 41] and/or clinical exams [20, 41–44]. The work reported in [40] compared traditional classifiers, such as MLP and Rando Forest using different feature selection and optimization techniques with several state-of-the-art Convolutional Neural Networks (CNNs) on foot thermogram images to identify diabetic foot. They found that the AdaBoost Classifier achieved an F1 score of 97% by processing ten features extracted from foot thermogram images achieving better results than the CNN-based classifiers. The authors of the work reported in [41] used a real-time Single Snapshot Multiple-frequency Demodulation—Spatial Frequency Domain Imaging (SSMD-SFDI) system to map the optical properties of the forefoot skin. They used Logistic Regression and Support Vector Machines (SVM) to process the 2D maps of optical and physiological parameters, including cutaneous hemoglobin concentration, oxygen saturation, scattering properties, melanin content, and epidermal thickness, from every single snapshot. An area under the ROC curve (AUC) of 97.2% was achieved (distinguishing the diabetic foot patients from all other subjects). Despite the high performance achieved by the image-based classifiers, image processing by Deep Neural Networks usually requires lengthy periods of time for training. Additionally, it requires powerful computers with specialized processing units such as Tensor Processing Units (TPU) and Neural Processing Units (NPU). In some cases, a strict experimental setup with specific illumination to capture the images of the feet, which may decrease the usability of the system, must be used as in the SSMD-SFDI imaging system reported in [41].
In relation to machine learning-based systems that avoid the use of images for diabetic foot development risk classification, Table 4 summarizes the main features of four existing methods and the proposed one. Most papers exploited feature selection methods to reduce the number of variables. Logistic Regression (LR) was the classifier most used. The size of the sample studied varied from one work to another. It should be highlighted that the proposed method is the only one that does not require clinical exams to perform the risk classification, which makes the method easier and cheaper for the user. The number of variables varied from 18 to 54 and the final number of variables (after feature selection) varied from 4 to 12. The studies normally used accuracy as the performance metric. It is emphasized that the proposed method achieved a competitive accuracy (85.00%) in comparison to the other studies shown in Table 4, mainly considering that it does not require any further clinical exams to perform the final risk classification.
6.1 The CARPeDia app
The use of apps for smartphones has been widely experienced in the health field due to the following reasons: (i) they enable the dissemination and updating of knowledge; (ii) they support the clinical decision-making of professionals in the area, contributing to the elaboration of reliable diagnoses and qualified therapeutic guidelines/conducts for patients/users; and (iii) they can optimize the work of health professionals, particularly in situations with a high number of patients to be treated.
In addition to appealing to those who are able to access it, seeking to monitor and verify the risk for diabetic foot, the CARPeDia also generates a personalized report. This report highlights any incorrect self-care actions performed by the user, and provides guidelines for improving their health and foot care.
The app developed implements the proposed approach III (MIII1), which uses only 10 input variables to stratify the risk for diabetic foot, making the app simpler to be used. The result obtained after the usability evaluation is an index of user satisfaction (ranging from 0 to 100). Following the calculations proposed in [37], a score of 93.33 points was achieved, which demonstrates that the app fulfils the usability requirements.
Fig 4 shows the CARPeDia app interfaces. After completing the information requested in the app, a personalized report is generated, which provides the user guidelines and suggests behavior/attitudes changes. It also guides the changes already perceived by the patient, and addresses aspects that could increase the risk for diabetic foot. Furthermore, the collected information is stored for future analysis and improvements in the method.
7. Conclusion
This work presents a new supervised neural network model for diabetic foot risk assessment. The results showed that with only 10 variables out of 54 in the database, the proposed supervised model achieved an accuracy of 85% and a sensitivity of 84%, outperforming the unsupervised model presented in [20]. The dimension reduction from 54 to 10 implies lower computational cost, which is desirable for implementation considering hardware requirements, and also requires less time for patient information collection.
Given the high number of people with DM and the high demand for health services, the system helps by stratifying cases, so that health professionals can organize the flow of care according to priorities, immediately attending to those in need (greater risk).
For future works, the authors intend to exploit Deep Neural Network models [12, 45] to segment and process images of the feet for patients classified as high risk for diabetic foot.
Acknowledgments
The authors thank John Brooke, the author of the SUS scale, which has been made freely available for use in usability assessments. We also acknowledge the National Council for Scientific and Technological Development (CNPq/Brazil, Proc 306262/2017-7) for supporting this work.
References
- 1.
International Diabetes Federation. IDF Clinical Practice Recommendations on the Diabetic Foot– 2017, A guide for healthcare professionals. 2017. ISBN: 978-2-930229-86-7
- 2. International Diabetes Federation. IDF Diabetes Atlas, 9 ed; 2019.
- 3. American Diabetes Association (ADA). Diagnosis and Classification of Diabetes Mellitus. Diabetes Care 2015;38:8–16. http://care.diabetesjournals.org/content/38/Supplement_1/S8.
- 4. International Working Group on the Diabetic Foot. IWGDF Guideline on the prevention of foot ulcers in persons with diabetes. Part of the 2019 IWGDF Guidelines on the Prevention and Management of Diabetic Foot Disease. 2019. www.iwgdfguidelines.org
- 5. Brooks E, Burns M, Ma R, Scholten HJ, Becker S. Remote Diabetic Foot Temperature Monitoring for Early Detection of Diabetic Foot Ulcers: A Cost-Effectiveness Analysis. Clinicoecon Outcomes Res. 2021 Oct 7;13:873–881. pmid:34675567
- 6. Dias A, Ferreira G, Vilaça M, Pereira MG. Quality of Life in Patients with Diabetic Foot Ulcers: A Cross-sectional Study. Adv Skin Wound Care 2022 Dec 1;35(12):661–668. pmid:36409188
- 7. Wagner FW Jr. The dysvascular foot: a system for diagnosis and treatment. Foot Ankle 1981 Sep;2(2):64–122. pmid:7319435
- 8. Armstrong DG, Lavery LA, Harkless LB. Validation of a diabetic wound classification system: The contribution of depth, infection and vascular disease to the risk of amputation. Diabetes Care 1998; 21:855–859.
- 9. Braghetto GT, Sousa LA, Beretta D, Vendramini SHF. Difficulties and facilities of the Family Health nurse in the work process. Cad. Saúde Colet 2019; 27: 420–426.
- 10. Borges DB, Lacerda JT. Actions aimed at the Diabetes Mellitus control in Primary Health Care: a proposal of evaluative model. Saúde Debate 2018; 42: Jan-Mar. https://doi.org/10.1590/0103-1104201811613
- 11. Syeda Shamaila Zareen Sun Guangmin, Li Yu, Kundi Mahwish, Qadri Salman, Syed Furqan Qadri, et al. A Machine Vision Approach for Classification of Skin Cancer Using Hybrid Texture Features, Computational Intelligence and Neuroscience. Volume 2022, Article ID 4942637, https://doi.org/10.1155/2022/4942637
- 12. Qadri SF, Shen L, Ahmad M, Qadri S, Zareen SS, Akbar MA. SVseg: Stacked Sparse Autoencoder-Based Patch Classification Modeling for Vertebrae Segmentation. Mathematics. 2022; 10(5):796. https://doi.org/10.3390/math10050796
- 13. Lazzarini PA, Hurn SE, Fernando ME, Jen SD, Kuys SS, Kamp MC, et al. Prevalence of foot disease and risk factors in general inpatient populations: a systematic review and meta-analysis. BMJ Open. 2015 Nov 23;5(11):e008544. pmid:26597864
- 14. Furqan Qadri S, Ai D, Hu G, Ahmad M, Huang Y, Wang Y, et al. Automatic Deep Feature Learning via Patch-Based Deep Belief Network for Vertebrae Segmentation in CT Images. Applied Sciences. 2019; 9(1):69. https://doi.org/10.3390/app9010069
- 15. Ahmad M, Qadri SF, Qadri S, Saeed IA, Zareen SS, Iqbal Z, et al. A Lightweight Convolutional Neural Network Model for Liver Segmentation in Medical Diagnosis. Comput Intell Neurosci. 2022 Mar 30;2022:7954333. pmid:35755754
- 16. Ahmad M, Ai D, Xie G, Qadri S.F , Song H, Huang Y, et al. Deep Belief Network Modeling for Automatic Liver Segmentation. IEEE Access. 2019; 7:20585–20595,
- 17. Alvarenga TA, Santos O, Rodriguez DZ, Ferreira DD, Barbosa BHG, Seixas JM. Unsupervised Class-Expert Learning for Supporting Covid-19 Triage Based on Computed Tomography Data. Learning and nonlinear models. 2022, 20:74–88.
- 18. Silva RN, Ferreira ACBH, Ferreira DD, Barbosa BHG. Non-invasive method to analyse the risk of developing diabetic foot. Healthcare Technology Letters 2014;1:109–13. pmid:26609394
- 19. Ferreira ACBH Fernandes BM, Ferreira DD. Noninvasive Approach based on Self Organizing Maps to Classify the Risk of Diabetic Foot. IEEE Latin America Transactions 2018; 16:75–79.
- 20. Ferreira ACBH Ferreira DD, Oliveira HC Resende IC, Anjos A Lopes MHBM. Competitive neural layer-based method to identify people with high risk for diabetic foot. Computers in Biology and Medicine 2020; 120: 1037–44. pmid:32421649
- 21. Schäfer Z, Mathisen A, Svendsen K, Engberg S, Rolighed Thomsen T, Kirketerp-Moller K. Toward Machine-Learning-Based Decision Support in Diabetes Care: A Risk Stratification Study on Diabetic Foot Ulcer and Amputation. Front Med (Lausanne). 2021 Feb 18;7:601602. pmid:33681236
- 22. Shields C, Cunningham SG, Wake DJ, Fioratou E, Brodie D, Philip S, et al. User-Centered Design of A Novel Risk Prediction Behavior Change Tool Augmented With an Artificial Intelligence Engine (MyDiabetesIQ): A Sociotechnical Systems Approach JMIR Hum Factors 2022; 9(1):e29973 pmid:35133280
- 23. Hernandez-Contreras D, Barreto HP, Rangel-Magdaleno JJ, Gonzalez JA. Narrative review: Diabetic foot and infrared thermography. Infrared Physics & Technology 2016; 78:105–17.
- 24. Goyal M, Reeves ND, Davison AK, Rajbhandari S, Spragg J, Yap MH. DFUNet: Convolutional Neural Networks for Diabetic Foot Ulcer Classification. IEEE Transactions on Emerging Topics in Computational. 2018; 1–12. Electronic.
- 25. Ohura N, Mitsuno R, Sakisaka M, Terabe Y, Morishige Y, Uchiyama A, et al. Convolutional neural networks for wound detection: the role of artificial intelligence in wound care. Journal of Wound Care 2019; 28(10). pmid:31600101
- 26. Kai SC, Yam MC, Audrey HMT, Shanying L, Yuan TC, Qiantai H, et al. Clinical validation of an artificial intelligence-enabled wound imaging mobile application in diabetic foot ulcers. International Wound Journal 2021, 19(1). https://doi.org/10.1111/iwj.13603
- 27. Swerdlow M, Shin L, D’Huyvetter K, Mack WJ, Armstrong DG. Initial Clinical Experience with a Simple, Home System for Early Detection and Monitoring of Diabetic Foot Ulcers: The Foot Selfie. J Diabetes Sci Technol 2023 Jan;17(1):79–88. pmid:34719973
- 28. Ahmed A, Aziz S, Abd-Alrazaq A, Farooq F, Sheikh J. Overview of Artificial Intelligence-Driven Wearable Devices for Diabetes: Scoping Review. J Med Internet Res 2022 Aug 9;24(8):e36010. pmid:35943772
- 29. International Working Group on the Diabetic Foot. International Consensus on the Diabetic Foot. M Wijnandts, Department of Internal Medicine, Academic Hospital Maastricht, Netherlands, fl3, 2001.
- 30. Brasil. Ministério da Saúde. Diabetes Mellitus. Cadernos de Atenção Básica n.° 16, Série A. Normas e Manuais Técnicos. Brasília, DF; 2006. (Portuguese only). Available in: http://bvsms.saude.gov.br/bvs/publicacoes/diabetes_mellitus_cab16.pdf.
- 31. Fontelles MJ, Simões MG, Almeida JCA, Fontelles RGS. Scientific research methodology: Guidelines for size sample calculation. Revista Paraense de Medicina, 2010; 24:abril–junho.
- 32. Haykin S. Neural Networks and Learning Machines. 3rd ed. Prentice Hall; 2008.
- 33. Kohavi R. A study of cross-validation and bootstrap for accuracy estimation and model selection. In: International joint Conference on artificial intelligence 1995; 14:1137–1145.
- 34.
Theodoridis S, Koutroumbas K. Pattern Recognition. 4th edition. Academic Press; 2009.
- 35. MacKay D.J.C . A Practical Bayesian Framework for Backpropagation Networks. Neural Comput 1992; 4 (3): 448–472. https://doi.org/10.1162/neco.1992.4.3.448
- 36. Pagano M, Gauvreau K. Princípios de Bioestatística, Ed. Thomson São Paulo, 2004. (Portuguese only)
- 37. Brooke J. SUS—A quick and dirty usability scale. 1986. Available in http://www.usabilitynet.org/trump/documents/Suschapt.doc.
- 38. Cabral Filho JE. Methodology and Ethical Principles Interrelation in Scientific Research and Publication. Rev. Bras. Saúde Matern. Infant 2004; 4: 225–28.
- 39. Boulton AJ, Armstrong DG, Albert SF, Frykberg RG, Hellman R, Kirkman MS, et al. Comprehensive foot examination and risk assessment: a report of the task force of the foot care interest group of the American Diabetes Association, with endorsement by the American Association of Clinical Endocrinologists. Diabetes Care 2008 Aug;31(8):1679–85. pmid:18663232
- 40. Khandakar A, Chowdhury MEH, Ibne Reaz MB, Md Ali SH, Hasan MA, Kiranyaz S, et al. A machine learning model for early detection of diabetic foot using thermogram images. Comput Biol Med. 2021 Oct;137:104838. pmid:34534794
- 41. Li Y, Guo M, Qian X, Lin W, Zheng Y, Yu K, et al. Single snapshot spatial frequency domain imaging for risk stratification of diabetes and diabetic foot. Biomed Opt Express. 2020, Jul 20;11(8):4471–4483. pmid:32923057
- 42. Guiotto A, Bortolami G, Ciniglio A, Spolaor F, Guarneri G, Avogaro A, et al. Machine learning approach to diabetic foot risk classification with biomechanics data. Gait & Posture. 2022, 97(2):30–31. https://doi.org/10.1016/j.gaitpost.2022.09.052.
- 43. Jiang M, Gan F, Gan M, Deng H, Chen X, Yuan X, et al. Predicting the Risk of Diabetic Foot Ulcers From Diabetics With Dysmetabolism: A Retrospective Clinical Trial. Front Endocrinol (Lausanne). 2022 Jul 12;13:929864. pmid:35903284; PMCID: PMC9317529.
- 44. Naemi R, Chockalingam N, Lutale JK, et al. Predicting the risk of future diabetic foot ulcer occurrence: a prospective cohort study of patients with diabetes in Tanzania, BMJ Open Diabetes Research and Care. 2020, 8. pmid:32371531
- 45. Hirra I, Ahmad M, Hussain A, Ashraf MU, Saeed IA, Qadri SF., et al. Breast Cancer Classification From Histopathological Images Using Patch-Based Deep Learning Modeling. IEEE Access. 2021; 9:24273–24287.