Comparison of repeatability of swept-source and spectral-domain optical coherence tomography for measuring inner retinal thickness in retinal disease

Purpose To compare repeatability between SS-OCT and SD-OCT for measurement of macular, macular retinal nerve fiber (mRNFL), and ganglion cell-inner plexiform layer (GC-IPL) thickness in various retinal diseases. Methods One hundred and fourteen eyes of 114 subjects were investigated. Seventy-eight eyes with retinal disease and 36 normal eyes underwent two consecutive measurements of macular, mRNFL, and GC-IPL thickness using SS-OCT and SD-OCT. The data were obtained using the Early Treatment Diabetic Retinopathy Study (ETDRS) protocol. The eyes with retinal diseases were divided into three subgroups according to central macular thickness (CMT) for analysis. The intraclass correlation coefficient (ICC) was calculated to determine the repeatability of OCT device. Results In normal eyes, both OCT devices showed excellent repeatability of macula, mRNFL, and GC-IPL thickness measurements with high ICCs in all ETDRS subfields. In eyes with retinal disease, although SS-OCT showed better repeatability for inner retinal thickness measurements than SD-OCT, the overall ICCs were lower than those in normal eyes. In subgroup analysis, the ICCs in the low CMT group were lower than those in the normal and high CMT groups, particularly when using SD-OCT. Conclusions Both OCT devices had comparable repeatability for retinal thickness measurement in normal eyes and eyes with retinal disease. However, the possibility of measurement error should be considered in eyes with a thin and atrophic retina.

Introduction the tenets of the Declaration of Helsinki (IRB no. 2017-10-013). Written informed consent was obtained from all participants, which was approved by the IRB. One hundred and twentyseven subjects from the Department of Ophthalmology of Hanyang University Guri Hospital were enrolled in the study between October 2017 and January 2018. The subjects underwent a complete ophthalmic examination, including best-corrected visual acuity (BCVA), intraocular pressure (Goldman applanation tonometry), refractive errors, slit lamp biomicroscopy, fundoscopy, and OCT, along with a complete medical history. Only one eye per subject was included. In the normal group, the right eye was selected. In the disease group, if both eyes were affected, the right eye was selected, otherwise the affected eye was selected.
The normal group (36 eyes, 36 healthy subjects) was age-matched and included subjects at least 19 years with a BCVA of 20/25 or higher, a normal fundus, and an intraocular pressure <21 mmHg. Participants in both groups were excluded if they had a spherical equivalent greater than ± 6 D, cataract graded as more severe than grade 3 (Lens Opacities Classification System III), [22] a history of glaucoma or optic nerve disease, or a history of vitrectomy.

Optical coherence tomography imaging
All subjects underwent imaging with an SS-OCT (deep range imaging OCT, Triton, Topcon, Tokyo, Japan) and SD-OCT (3D-OCT-2000, Topcon) in the same clinical setting. The two OCT examinations were performed twice by one experienced technician in the order of SD-OCT and SS-OCT, respectively. There was a 10-minute interval between examinations using the two OCT devices. The pupil was dilated before examination and an internal fixation target in the OCT device was used. Unacceptable images, such as those with poor scan quality (an image quality score provided by the onboard OCT software lower than 45, according to manufacturer's recommendation [23,24]) obtained by either OCT device, were excluded. The images were interpreted using the same image viewer (IMAGEnet 6 Version 1.20.11109, Topcon) for both OCT devices. A three-dimensional vertical scan protocol composed of 512 × 128 scans (512 A-scans for each of 128 B-scans, for a total of 65,536 axial scans/volume) was used for both OCT devices. The scan length was 7.0 × 7.0 mm for both SS-OCT and SD-OCT. SD-OCT uses a super luminescence diode at 840 nm wavelength as light source, and provides 5-6 μm of axial resolution, 20 μm of transverse resolution and maximum scan velocity of 27,000 A-scans per second. SS-OCT uses a tunable laser as a light source to provide a 1050 nm centered wavelength and provides 8 μm of axial resolution, 20 μm of transverse resolution and maximum scan velocity of 100,000 A-scans per second. [25] The macular, macular RNFL (mRNFL), and GC-IPL thicknesses were measured automatically. [26] All measurements were presented in the Early Treatment Diabetic Retinopathy Study (ETDRS) protocol with the central circle, inner ring, and outer ring using 1-, 3-, and 6-mm diameters. Manual correction of segmentation was not applied.

Statistical analysis
All statistical analyses were performed using SPSS version 22.0 (IBM Corp., Armonk, NY USA). Data are presented as mean values ± standard deviation. The repeatability of the two consecutive measurements was assessed using the intraclass correlation coefficient (ICC) and the coefficient of variation (CV). The ICC was used to determine the repeatability for each device using a one-way random-effects model. The degree of reliability was classified according to the ICC as follows: slight (0-0.2), fair (0.21-0.4), moderate (0.41-0.6), substantial (0.61-0.8), and almost perfect (0.81-1). [27] To compare the 2 ICC values for intradevice repeatability between the 2 types of OCT systems, we used the statistical method of Donner and Zou. [28] A CV of <10% indicates high repeatability. Bland-Altman plots with regression analysis were used to analyze the agreement between the thickness values obtained by each OCT device. Statistical significance was defined as P < 0.05. We divided the retinal disease group into 3 subgroups for analysis according to central macular thickness (CMT), i.e., low CMT (<200 mm), normal CMT (200-300 mm), and high CMT (>300 mm) based on a previous study. [20]

Demographics
One hundred and fourteen eyes (114 patients) met the study inclusion criteria. Five eyes were excluded because of poor image quality (n = 2) or decentration (n = 3). There were no significant differences in age, sex, laterality, refractive errors, or intraocular pressure between the normal and retinal disease groups. However, the retinal diseases group showed significantly lower BCVA than the normal group (p = 0.009). There was no significant difference in clinical features in the retinal disease group, except for a significantly lower BCVA in the low CMT group (p = 0.012; Table 1). The low CMT group included 20 eyes with diagnoses such as geographic atrophy in AMD, macular atrophy associated with RVO, macular atrophy associated with DR, and ocular Behcet's disease. The normal CMT group included 35 eyes with AMD, RVO, DR, CSC, and ocular Behcet's disease. The high CMT group included 23 eyes with AMD, macular edema secondary to RVO, diabetic macular edema, CSC, and ERM with macular edema. The demographics and clinical characteristics of the study subjects are shown in Table 1.

Repeatability in normal eyes
In the normal eyes, the ICCs for the macula, mRNFL, and GC-IPL thickness measurements exceeded 0.9 in most subfields and were classified as almost perfect (ICC = 0.81-1) for SS-OCT and SD-OCT. The CV values were generally high. Only the central thickness (0.817) and outer temporal subfield (0.840) measurements of the mRNFL using SD-OCT were under 0.9. The ICC was 0.993 for SS-OCT and 0.990 for SD-OCT for the average macula thickness (p = 0.136), 0.989 for SS-OCT and 0.969 for SD-OCT, respectively, for the average mRNFL thickness (p < 0.001), and 0.991 for SS-OCT and 0.989 for SD-OCT, respectively, for the average GC-IPL thickness (p = 0.269) (Fig 1, Table 2).

Repeatability in eyes with retinal disease
In the eyes with retinal disease, the ICCs for the average thickness measurements were lower for both OCT measurements than those in the normal group (normal vs retinal disease group: were higher than those for SD-OCT, and the CV results were similar. In terms of the repeatability of measurement for the mRNFL, the inner superior subfield and the outer temporal subfield of the quadrant map showed ICCs lower than 0.8 (inner superior, 0.771 on SS-OCT and 0.739 on SD-OCT; outer temporal, 0.858 on SS-OCT and 0.784 in SD-OCT). No subfield had an ICC lower than 0.8 for the macular or GC-IPL thickness measurement (Table 2). We compared the repeatability of thickness measurements for the macula, mRNFL, and GC-IPL in three subgroups according to CMT (low, normal and high). The ICCs for macular thickness showed high repeatability in all groups using both OCT devices, but were relatively higher for SS-OCT than for SD-OCT (low CMT, 0.  Table 3). A similar tendency was found Table 2. Intra-device repeatability of the two consecutive measurements using the intraclass correlation coefficient and coefficient of variation in normal eyes and eyes with retinal disease.

Normal group
Retinal disease group   Table 5).

Discussion
In this study, we found that both SS-OCT and SD-OCT had high repeatability for measurement of the thickness of the macula, mRNFL, and GC-IPL in the group with normal eyes. However, the ICCs for SS-OCT were higher than those for SD-OCT overall. In the group with retinal disease, the overall repeatability of thickness measurements was higher for SS-OCT than for SD-OCT; however, the repeatability was lower than that in the group with normal eyes. In the CMT subgroup analysis, SS-OCT showed higher repeatability than SD-OCT regardless of the CMT value, and both OCT devices had relatively low repeatability of mRNFL thickness measurements in eyes with low CMT. With the recent advent of SS-OCT, researchers have investigated the repeatability of this OCT device in normal eyes. Mansouri et al. investigated the repeatability of all SS-OCT scan protocols in normal eyes and found that repeatability was high for choroidal and retinal thickness measurements when using automated segmentation (ICC = 0.8-0.9). [18] However, they suggested that SS-OCT also had scan artifacts in up to 9% of cases, which should be taken into account when interpreting measurement data. Another study by Lee et al. compared the repeatability of peripapillary RNFL and GC-IPL thickness measurements between SD-OCT and SS-OCT in normal eyes. [19] They reported that the ICCs for peripapillary RNFL thickness measurements were higher on SS-OCT than on SD-OCT, but both OCT devices had ICCs > 0.9 for most clinically relevant subfields. The same trend was found for the GC-IPL thickness measurements, i.e., no clinically relevant difference was detected between two devices. The study by Lee et al obtained macular GC-IPL thickness measurements in 6 subfields, [19] whereas we used the ETDRS map, which is used mainly in retinal research, and there was a similar trend of high repeatability for both OCT devices in normal subjects. Although the study by Lee et al was the first to compare SD-OCT and SS-OCT with regard to  measurement of thickness values, the OCT devices used in their study was not from same manufacturer, in that their SS-OCT device was from Topcon and their SD-OCT device was from Carl Zeiss Meditec Inc. (Dublin, CA, USA). This limits the relevance of the findings of their study because use of different segmentation algorithm software may affect repeatability. In contrast, our study compared the repeatability of SD-OCT with that of SS-OCT using devices manufactured by same company (Topcon). We analyzed the OCT images using the same image viewer software, so we were able to evaluate the influence of the laser source more accurately when measuring retinal thickness by excluding segmentation algorithm software factors.
We assessed the repeatability of macular, mRNFL and GC-IPL thickness measurements obtained by SS-OCT in eyes with retinal disease for the first time and divided the retinal disease group into three subgroups according to CMT. The overall ICCs for the retinal disease group were lower than those for the normal group using both OCT devices, with the exception of macular thickness using SD-OCT. We found that the low and high CMT groups had ICCs that were lower than those in the normal CMT group using both OCT devices. This trend was consistent with that seen in previous studies comparing normal eyes with eyes affected by retinal disease using SD-OCT. [20,21,29] Lee et al. reported that the ICCs for thickness measurements in groups with normal eyes, eyes with macular edema, and eyes with atrophy were 0.998, 0.985, and 0.903, respectively. Our data also showed high repeatability for macular thickness measurements (ICC > 0.8-0.9 on both OCT devices). In our study, both OCT devices showed similar repeatability (ICC > 0.8-0.9) for GC-IPL thickness measurements, which is similar to previous findings using SD-OCT. [20] Therefore, both OCT devices had similar intra-device repeatability for measurement of macular and GC-IPL thickness in eyes with retinal disease, regardless of CMT. However, another study performed by Lee et al. suggested that SD-OCT had low repeatability in eyes with ERM, especially in those with high CMT (>450 μm). Shin et al. also reported that repeatability of SD-OCT for GC-IPL measurements was low in eyes with AMD that were out of the normal CMT range. [29] In the present study, retinal diseases were classified by CMT rather than by specific disease because of the small sample sizes. Further studies are warranted to confirm the repeatability of SS-OCT for each specific retinal disease.
Macular RNFL is the thinnest auto-segmentation layer measurable by current OCT devices. We found that both OCT devices showed high repeatability for mRNFL thickness measurements in eyes with retinal disease and normal or high CMT. An SS-OCT study by Min et al. reported that the ICCs for mRNFL thickness measurements were high in eyes with DR with or without macular edema. [30] However, in our study, the repeatability of the mRNFL measurements was significantly lower for both OCT devices in the low CMT group (average ICC 0.669 on SS-OCT vs. 0.639 on SD-OCT). In the low CMT group, SS-OCT had 6 subfields with high ICCs (>0.8), while SD-OCT had no such subfields. This finding may reflect the fact that mRNFL in the low CMT group was far thinner than that in the normal and high CMT groups, which may lead to segmentation error in the detection of the mRNFL. In contrast, high repeatability was observed for the macular and GC-IPL thickness measurements, even in the low CMT group. These differences might reflect differences in the research methods used in the studies. The study by Min et al. used a wide scan protocol (12 × 9 mm), whereas we used a standard-size scan protocol (6 × 6 mm or 7 × 7 mm). In addition, our study included a mixture of retinal diseases, whereas their study included one specific retinal disease. Our study did not investigate the repeatability of peripapillary RNFL thickness measurements, which is important for diagnosis of glaucoma. According to previous reports on the repeatability of peripapillary RNFL thickness measurements obtained by SD-OCT in eyes with retinal disease, the repeatability was relatively higher (with ICCs in the range of 0.8-0.9) than that of our mRNFL measurements. Lee et al. and Shin et al. demonstrated high repeatability when measuring peripapillary RNFL thickness in eyes with ERM and AMD. [21,29] We attribute this difference to the fact that the peripapillary RNFL is thicker than the mRNFL.
There are several potential explanations for why the ICCs were lower in the retinal disease group than in the normal group in our study. One reason may be the low average visual acuity and another may be an unstable gaze. Auto-segmentation errors because of the change in macular contour are other possible reasons. [31,32] Overall, SS-OCT showed slightly higher repeatability than SD-OCT for measurements of inner retinal thickness in eyes with retinal diseases in our study. As mentioned above, the SS-OCT system provides faster scan speeds and better image resolution, and uses a longer wavelength light source (1050 nm) than SD-OCT, so can improve the repeatability of inner retinal thickness measurements by reducing segmentation artifacts. [16,17] The strength of our study is in the fact that it is the first to compare the repeatability of SS-OCT with that of SD-OCT for measurement of inner retinal thickness in both normal eyes and eyes with retinal disease. Although the measurement of inner retinal layer thickness provided by the OCT device is useful and widely used to evaluate the macular ganglion cell layer and the nerve fiber layer, the reliability of the measurement should be considered when interpreting data from patients with retinal diseases, especially with low CMT. In addition, the scan protocols, image viewer, and OCT devices used in our study were made by same manufacturer, which made it possible to compare only the effects of the laser source, while excluding other confounding factors.
However, there are several limitations to this study. The first is the relatively small sample size. Further prospective studies in larger numbers of patients are needed to confirm our present findings. Second, the retinal diseases included in this study were heterogeneous. In practice, segmentation errors are more likely to be more influenced by specific retinal layer abnormalities rather than CMT. For example, ERM can cause inner retinal segmentation errors, while AMD can cause outer retinal segmentation errors. Finally, the purpose of this study was to compare the repeatability of inner retinal thickness according to the difference in light source between SD-OCT and SS-OCT. However, other than the light source, SD-OCT and SS-OCT also differ in axial resolution (5-6 μm of axial resolution in SD-OCT vs. 8 μm of axial resolution in SS-OCT). This point should be considered in interpreting the results of this study.
Although SS-OCT showed relatively better repeatability for inner retinal thickness measurements in both normal eyes and eyes with retinal disease when compared with SD-OCT, the two OCT devices showed comparable performance. However, the repeatability of mRNFL in the condition of retinal atrophy is low even for SS-OCT. The changes in the retinal layer configuration of the atrophic retina may affect thickness measurements, which should be considered when assessing patients with retinal diseases, especially when combined with glaucoma or neuro-ophthalmologic disorders. A future study comparing the repeatability of the two OCT devices in eyes with glaucoma, in which mRNFL and GC-IPL thickness is clinically significant, would be worthwhile.