Fig 1.
Examples of the progression of ophthalmic disease.
(a)–(f) The slit-lamp images of six consecutive re-examination stages: the 3rd, 6th, 9th, 12th, 18th and 24th month. The first two rows are negative samples defined as manageable patients during the whole recovery period, while the third and fourth rows represent positive samples who require Nd-YAG laser surgery at the 6th re-examination stage. Notes: Nd-YAG: neodymium-doped yttrium aluminum garnet.
Fig 2.
The architecture of the TempSeq-Net model.
(a) Temporal sequence data inputs. The sequence images are sorted according to the re-examination stages and then entered into the convolutional neural network sequentially. (b) Convolutional neural network. The CNN is used for extracting the high-level features of temporal sequence images. (c) Long short term memory. The LSTM is used for mining and summarizing the internal rules of temporal sequence images. (d) The prediction output. The model predicts the probability of the progression of ophthalmic disease at an upcoming stage, where F and S represent follow-up and laser surgery, respectively. (e) The internal structure of the LSTM. Notes: 3M: the 3rd month of re-examination; FC: full-connected layer; TempSeq-Net: temporal sequence network.
Fig 3.
The overall prediction framework for the progression of ophthalmic disease.
(a) The 6,090 slit-lamp sequence images consist of six consecutive re-examination stages (the 3rd, 6th, 9th, 12th, 18th and 24th month) of the 1,015 patients. Each image was examined and labeled independently by three experienced ophthalmologists. (b) Seeking the optimal classifier. The 5-fold cross-validation was employed to evaluate the performance of six combinations of three CNNs and two sequence methods (LSTM and RNN) to obtain the optimal TempSeq-Net model. (c) Training classifiers with different sequence lengths. Sequence datasets with different lengths (five, four and three) and their labels are employed to train three classifiers TempSeq-Net, TempSeq-Net-1 and TempSeq-Net-2, respectively. (d) Evaluating classifiers with different sequence lengths. The classifiers trained in the (c) are compared using sequence images with lengths of 2–5. Notes: CNN: convolutional neural network; LSTM: long short term memory; RNN: recurrent neural network.
Table 1.
The quantitative evaluation of six different temporal sequence networks.
Fig 4.
The ROC curves and performance comparison of six temporal sequence networks.
(a) The ROC curves and AUC values of six temporal sequence networks: AlexNet-LSTM, GoogLeNet-LSTM, ResNet-LSTM, AlexNet-RNN, GoogLeNet-RNN and ResNet-RNN. (b) The performance comparison of LSTM models (AlexNet-LSTM, GoogLeNet-LSTM and ResNet-LSTM) in terms of accuracy, the model size and time consumption per sequence data. Notes: ROC: receiver operating characteristic curve.
Table 2.
The efficiency and resource utilization comparison of three LSTM models.
Fig 5.
The performance comparison of TempSeq-Net model over different sequence lengths.
(a) The ROC curves and AUC values of TempSeq-Net model over sequence lengths of 2–5. (b) The ACC, SPE and SEN indicators comparison of TempSeq-Net model over sequence lengths of 2–5. Notes: SL: sequence length.
Table 3.
The performance comparison of TempSeq-Net model for prediction with different sequence lengths.
Table 4.
The performance comparison of the models trained with different sequence lengths.
Fig 6.
The convergence analysis of the TempSeq-Net model.
The blue and red curves represent the changing trends of the loss function value and accuracy with iterations, respectively.