Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Suitability of UK Biobank Retinal Images for Automatic Analysis of Morphometric Properties of the Vasculature

  • Thomas J MacGillivray ,

    T.J.MacGillivray@ed.ac.uk

    Affiliations VAMPIRE project, Clinical Research Imaging Centre, University of Edinburgh, Edinburgh, United Kingdom, Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom, Clinical Research Facility, University of Edinburgh, Edinburgh, United Kingdom

  • James R. Cameron,

    Affiliation The Anne Rowling Regenerative Neurology Clinic, University of Edinburgh, Edinburgh, United Kingdom

  • Qiuli Zhang,

    Affiliation Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom

  • Ahmed El-Medany,

    Affiliation Medical School, University of Edinburgh, Edinburgh, United Kingdom

  • Carl Mulholland,

    Affiliation Medical School, University of Edinburgh, Edinburgh, United Kingdom

  • Ziyan Sheng,

    Affiliation Medical School, University of Edinburgh, Edinburgh, United Kingdom

  • Bal Dhillon,

    Affiliation VAMPIRE project, School of Clinical Sciences, University of Edinburgh, Edinburgh, United Kingdom

  • Fergus N. Doubal,

    Affiliation Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom

  • Paul J. Foster,

    Affiliation National Institute for Health Research, Biomedical Research Centre at Moorfields Eye Hospital & University College London Institute of Ophthalmology, London, United Kingdom

  • Emmanuel Trucco ,

    Contributed equally to this work with: Emmanuel Trucco, Cathie Sudlow

    Affiliation VAMPIRE project, School of Computing, University of Dundee, Dundee, United Kingdom

  • Cathie Sudlow ,

    Contributed equally to this work with: Emmanuel Trucco, Cathie Sudlow

    Affiliation Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom

  • UK Biobank Eye and Vision Consortium

    Membership details for the UK Biobank Eye and Vision Consortium are provided in the Acknowledgements.

Suitability of UK Biobank Retinal Images for Automatic Analysis of Morphometric Properties of the Vasculature

  • Thomas J MacGillivray, 
  • James R. Cameron, 
  • Qiuli Zhang, 
  • Ahmed El-Medany, 
  • Carl Mulholland, 
  • Ziyan Sheng, 
  • Bal Dhillon, 
  • Fergus N. Doubal, 
  • Paul J. Foster, 
  • Emmanuel Trucco
PLOS
x

Abstract

Purpose

To assess the suitability of retinal images held in the UK Biobank - the largest retinal data repository in a prospective population-based cohort - for computer assisted vascular morphometry, generating measures that are commonly investigated as candidate biomarkers of systemic disease.

Methods

Non-mydriatic fundus images from both eyes of 2,690 participants - people with a self-reported history of myocardial infarction (n=1,345) and a matched control group (n=1,345) - were analysed using VAMPIRE software. These images were drawn from those of 68,554 UK Biobank participants who underwent retinal imaging at recruitment. Four operators were trained in the use of the software to measure retinal vascular tortuosity and bifurcation geometry.

Results

Total operator time was approximately 360 hours (4 minutes per image). 2,252 (84%) of participants had at least one image of sufficient quality for the software to process, i.e. there was sufficient detection of retinal vessels in the image by the software to attempt the measurement of the target parameters. 1,604 (60%) of participants had an image of at least one eye that was adequately analysed by the software, i.e. the measurement protocol was successfully completed. Increasing age was associated with a reduced proportion of images that could be processed (p=0.0004) and analysed (p<0.0001). Cases exhibited more acute arteriolar branching angles (p=0.02) as well as lower arteriolar and venular tortuosity (p<0.0001).

Conclusions

A proportion of the retinal images in UK Biobank are of insufficient quality for automated analysis. However, the large size of the UK Biobank means that tens of thousands of images are available and suitable for computational analysis. Parametric information measured from the retinas of participants with suspected cardiovascular disease was significantly different to that measured from a matched control group.

Introduction

Around 20 million people in the UK alone suffer from at least one long term health condition, and over the next decade this is set to increase three-fold with the ageing population placing an enormous economic and human-resource strain on the health care system [13]. In the US, 7 out of 10 deaths each year are from chronic diseases [4]. The retina, through the analysis of blood vessels in fundus images, is a well-established non-invasive proxy of systemic non-retinal vascular disease [57], and detailed clinical observations of characteristic fundus features have led to the identification of early indicators of a diverse range of conditions, including diabetes mellitus, cardiovascular disease and neurological disorders [5, 810].

The UK Biobank includes the largest retinal data repository in a prospective population-based cohort and represents an important and valuable resource, accessible to international researchers, for improving the prevention, diagnosis and treatment of a wide range of long-term conditions and chronic illness [11]. Considerable potential exists in using this dataset to discover biomarkers that will in the future enable systematic patient surveillance and the identification of high-risk patients as well as surrogate end points that will accelerate the discovery of new interventions.

We accessed fundus images held in the UK Biobank and subjected them to computational analysis using VAMPIRE (Vascular Assessment and Measurement Platform for Images of the REtina) software [12, 13], which yields parametric information relating to the retinal vasculature that may be indicative of systemic disease [5, 14]. Our objective was to assess the suitability of UK Biobank images for computational analysis and this paper presents our findings. Validation of relevant VAMPIRE modules has been reported elsewhere [12, 13, 15, 16].

Methods

The UK Biobank recruited around half a million people aged between 40–69 years in the UK from 2006–2010. With informed consent, participants completed a detailed touchscreen questionnaire about their lifestyle, environment, medical and family history, underwent a range of physical measures and provided blood, urine and saliva samples for future analysis. They also agreed to long term follow-up of their health. A subset of 68,544 participants underwent retinal fundus camera imaging—a single non-mydriatic digital image was obtained from right and left eyes using a Topcon 3D OCT 1000 Mark 2 (45° field-of-view, centred to include both optic disc and macula). Colour images were stored in PNG format with dimensions 2048 × 1536 pixels. The UK Biobank obtained approval for all its data collection procedures from its governing Research Ethics Committee.

We accessed fundus images from 2,690 participants—people with a self-reported history of myocardial infarction (n = 1,345), and a control group without such a history (n = 1,345), matched on assessment centre location, age and sex. This yielded 5,380 images for analysis with VAMPIRE version 2.0 (Universities of Edinburgh and Dundee, UK). The selection criterion for cases was chosen in order to harvest a participant group with a long-term condition (i.e. cardiovascular disease) that is widely considered to have retinal manifestations [5]. Our access to the resource was checked and approved by the UK Biobank as being consistent with their Access Procedures and Ethics and Governance Framework [17, 18]. Our research adhered to the tenets of the Declaration of Helsinki.

A team of four operators, consisting of a consultant level ophthalmologist (author Cameron) and three medical students (authors El-Medany, Mulholland and Sheng), was trained to use the VAMPIRE software with clear protocols to quantify retinal blood vessel parameters—tortuosity, bifurcation branching angle (BA) and bifurcation coefficient (BC)—that might reveal abnormalities in the vasculature that are considered indicative of disease processes and microvascular damage [5, 14]. Training for each operator consisted of two sessions lasting approximately four hours in total: an introductory phase where the protocols and software were presented and familiarity with them gained through practice on a demonstration image set (n = 20); and an assessment phase where competency in operation was assessed on a testing image set (n = 20). Fundus images were divided amongst the four operators based on their availability for conducting analysis—Cameron (40%), El-Medany (4%), Mulholland (40%) and Sheng (16%). Software ran on UK Biobank hardware, accessed via a remote Windows desktop session.

The correct procedure for operating the software was as follows. First, the boundaries of the optic disc (OD) in a retinal image were detected [15] and a conventional OD-centred circular measurement zone established, stretching from the OD boundary to twice the OD diameter. Next, the software attempted to detect the retinal blood vessels present in the image and locate bifurcations. If VAMPIRE was able to detect the vasculature within the measurement zone the image was classed as processable; otherwise it was deemed non-processable.

For processable images, the operator was asked to identify, visually, the 2 thickest arterioles and 2 thickest venules in the measurement zone. The operator then used the software to select the path for a vessel as it extends from the edge of the OD to the outer edge of the zone for the calculation of tortuosity [16]. Taking the mean of these 2 measurements gave the summarised arteriolar and venular tortuosity values, respectively for an image. The operator was further asked to identify 3–5 arteriolar and 3–5 venular bifurcations per image from the automatically detected candidates located within the measurement zone, with the software returning BA and BC for each one. Due to the macula centred nature of the images, measuring 3–5 bifurcations was a realistic sampling target to realise. The software calculated the median for each set of measurements in order to summarise into arteriolar and venular parameters for an image. Successful completion of the measurement protocol categorised the image as analysable, i.e. there was sufficient detection of retinal vessels in the image to enable the measurement of the target parameters; otherwise it was recorded as non-analysable.

A chi-square test was used to compare the proportions of processable and analysable images by sex, age (in groups 40–50, 51–60 and 61–70 years), and case-control status. A logistic regression model was used to calculate odds ratio (OR) between female and male, age groups, and cases and controls. Inter-operator variability was assessed for the measured retinal parameters using images from 40 participants selected at random and by calculating the intraclass correlation coefficient (ICC) using a two-way mixed, consistency agreement, average measure model. We further evaluated our measurement protocols by calculating the correlation of retinal parameters between right and left eyes for participants with analysable images for both eyes using Spearman’s correlation coefficient. This assumes that in a (non-ocular disease) population an effective measurement strategy for characterising a particular retinal feature will produce a high inter-ocular correlation [19]. Finally, a logistic regression model was used to estimate associations of retinal measures with cases versus controls. ORs were calculated for an increase of 1 SD of VAMPIRE measures and adjusted for age and sex. Values of p < 0.05 (2-tailed) were considered statistically significant. Statistical analyses were conducted using SAS software version 9.3 (SAS Institute, Cary NC) and SPSS version 19 (IBM Corp, Chicago IL, USA).

Results

Table 1 shows the mean age and sex distribution of our study population, of all the UK Biobank participants who underwent fundus imaging, and of the entire UK Biobank population. The UK Biobank participants who had fundus imaging had almost identical mean age (57 years) and sex distribution (54% female) to the entire UK Biobank population, while participants in the case-control dataset used in this study were slightly older (mean age 61 years) and a lower proportion were female (20%).

Total operator time for using VAMPIRE on the study population images came to approximately 360 hours or around 4 minutes per image, with an additional 16 hours of training time (i.e. 4 hours per operator). Computational experiences are summarised in Table 2 which indicates that from the 2,690 UK Biobank participants that we assessed for our study, 2,252 people (84%) had a processable fundus image for at least one eye while 1,817 people (68%) had processable images for both eyes. 1,604 (60%) participants had an analysable fundus image of at least one eye, while 957 (36%) had analysable images for both eyes.

thumbnail
Table 2. Results of using the VAMPIRE software on the study population images.

https://doi.org/10.1371/journal.pone.0127914.t002

Further examination by participant sex, age group and case-control status is presented in Table 3. A higher proportion of female than male participants had at least one processable image (88% versus 83%), but the proportion of participants with analysable images did not differ by sex. Increasing age was associated with a reduced proportion of participants with processable and with analysable images (e.g., 93% processable and 78% analysable in those aged 40–50 years; 82% processable and 56% analysable in those aged 61–70 years). However, case-control status was not associated with either processablity or analysability, suggesting no major influence on image quality of cardiovascular disease or vascular risk factors such as smoking, hypertension and diabetes (which would be more common among cases than controls).

thumbnail
Table 3. Proportions of participants with at least one image that was processable or analysable by gender, age group and case-control status.

https://doi.org/10.1371/journal.pone.0127914.t003

Analysis of inter-operator variability is presented in Table 4. From the images of 40 participants selected at random, 12 could not be analysed due to poor image quality. For the remaining 28 images there was very good agreement between operators for measurements of BC and BA (ICC > 0.8). For measurements of tortuosity the level of agreement was lower (<0.3 ICC < 0.5).

thumbnail
Table 4. Analysis of VAMPIRE measures regarding the inter-operator reliability.

https://doi.org/10.1371/journal.pone.0127914.t004

Table 5 shows the correlations of retinal parameters between right and left eyes for participants with analysable images for both eyes, regardless of case-control status. Positive but low correlations (< 0.25) were present and significant (≤ 0.05) for all parameters. Associations of retinal vascular measures for cases and controls are reported in Table 6 (and S1 Table). We found narrower arteriolar BA (p = 0.02) and also lower arteriolar and venular tortuosity (p<0.0001) in cases compared to controls. No significant differences in either arteriolar or venular BC or in venular BA were observed.

thumbnail
Table 6. Association of VAMPIRE measure between cases and controls in any eye.

https://doi.org/10.1371/journal.pone.0127914.t006

Discussion

The proportion of UK Biobank participants in this study who had at least one image of sufficient quality for automated processing (84%) is similar to the proportion of participants reported to have retinal images that were gradable for retinal vascular parameters in other prospective, population-based studies conducting retinal photography (82–91%) [2023]. However, UK Biobank has the advantages that the number of participants with fundus imaging is substantially larger than in any previous prospective study, and that the retinal measures derived can be studied in association with a very wide range of other data of unparalleled depth and breadth from the baseline assessment, subsequent enhancements and follow-up.

Operator experience during this study suggested that non-processable images were distributed amongst four categories: around 80% were completely or largely blurred or obscured, hence unusable for computational analysis with any software; around 15% were hazy or blurred, which again precludes them from any computational analysis, though the images may still be suitable for visual grading of certain retinal characteristics such as lesions of diabetic retinopathy; only around 5% showed the retinal vasculature and hence might conceivably be processable and analysable with either VAMPIRE (after future refinements to the software) or other software.

Our study sample had a higher proportion of males and a higher mean age than the whole UK Biobank population who underwent retinal examination. Since male sex was associated with reduced image processability, and increasing age was associated with both reduced processability and reduced analysability, a higher proportion of all 68,544 participants with retinal imaging in UK Biobank are likely to have images which are processable and analysable with VAMPIRE software than we report here for our study sample.

We found that a substantially smaller proportion of images could be analysed than were processable. From our experiences with automated analysis, signs of retinopathy (e.g. microaneurysms, exudates, and haemorrhages) do not interfere with measuring retinal vessel parameters using VAMPIRE. However, the macula-centred nature of the UK Biobank fundus images affects software analysis of vascular parameters: as the macula is an avascular region of the retina, most of the visible vasculature appears concentrated towards the side of the image containing the OD. So opportunities to measure retinal vascular parameters such as bifurcation geometry and tortuosity can be limited, contributing to the lower proportion of analysable compared with processable images. This issue will affect the performance of other vascular analysis software tools as well as the quantification of other vascular parameters, particularly vessel diameters, which are commonly measured in the vicinity around the OD [2427].

A key aim for future work will be to reduce the proportion of processable but non-analysable images without compromising measurement validity or reliability. This will require further careful study of the main determinants of analysability. For example, our experience suggests that some images are processable but not analysable with VAMPIRE as a result of insufficient or incorrect bifurcation detection, i.e. either the software does not detect a bifurcation or it incorrectly labels the parent vessel and the measurement has to be discarded. Ongoing software refinements will address these issues through improved identification and evaluation of bifurcations and with a manual override option for the operator to correct software detection errors, thus enabling a higher proportion of analysable images in the future.

Similar protocols have been reported by other authors in computer-assisted studies on retinal biomarkers measuring in similar zones. For instance in [28], 7 arteriolar and 7 venular vessels were selected for tortuosity analysis along with 5 arteriolar and 5 venular bifurcations. In line with our results, arteriolar tortuosity was significantly lower among ischaemic heart disease cases (n = 124) compared to controls, although no difference was observed in arteriolar BA. To assess retinopathy of prematurity [29], tortuosity was quantified using 4 arteriolar and 4 venular vessels, i.e. 2 major vessels in each of 4 quadrants wherever possible. In [30], a protocol choosing only the most significant arterioles originating from the OD and that crossed a significant part of the fundus was proposed, giving 2–3 vessels per image. In an investigation into the relationship of retinal vascular tortuosity with age, blood pressure, and other cardiovascular risk factors, 6 large arteriolar segments and 6 large venular segments were considered [31]. Less tortuous arterioles were independently associated with older age, higher blood pressure and higher body mass index. To study cardiovascular risk factors and retinal arteriolar tortuosity in a multi-ethnic child population, the protocol for tortuosity involved 5–8 arteriolar and 5–8 venular segments [32]. Finally, approximately 15 bifurcations (an assortment of arteriolar and venular) were measured per image in patients with diabetes [33].

Inter-operator agreement ranged from excellent for bifurcations to fair and poor for tortuosity (arteriolar and venular, respectively) in our tests (n = 28). One possible explanation for this inconsistency is the difference in the number of samples required to calculate each parameter, i.e. 2 for tortuosity and 3–5 for bifurcations. With tortuosity there exists more opportunity for the operators to select different vessels while for bifurcations the requirement for a larger sample means they are more likely to choose the same ones as each other. Also, we have not tested intra-operator agreement in this study and so our results should be interpreted with a degree of caution.

While it is common to consider measurement reliability through repeat analysis on subsets of images, reporting the correlation between right and left eyes to show efficacy of a measurement protocol in characterising the retinal vasculature is rare. Although our correlations were significant, they were also low which suggests that further refinement of our protocols is required in order to improve their effectiveness. This may yet increase the utility of the retinal vascular parameters and warrant their further investigation as candidate biomarkers of systemic disease.

The average operator time per image suggests that fundus images for all 68,544 participants could be processed and analysed in around 18 months by a 4-strong team or 9 months by 8 operators, under reasonable assumptions (42 working weeks per year per operator, with 5 working days per week and 7 working hours per day). However, operator time required to analyse large image sets is anticipated to decrease as the software becomes increasingly automated in future versions (for example, computerised classification of vessels as arterioles or venules, which will lessen manual decision making and accelerate the analysis of retinal parameters).

As an alternative to automated image analysis for handling large numbers of retinal images, the process of outsourcing (or ‘crowdsourcing’) specific retinal image analysis tasks to an online community of non-experts has been explored recently in the context of lesion grading [34], but further studies are necessary to assess the reliability of crowdsourcing and to identify for which tasks it is most suited. The approach seems unlikely to be suitable for annotating morphometric properties of the vasculature, which is a very different task from lesion grading.

As a final point, despite the fact that a sizeable proportion of images from among those initially accessed were unsuitable for computational analysis, the number of images potentially available for this type of analysis from the total collection of UK Biobank retinal images is still of substantial size, and sufficient to support statistically well-powered studies featuring retinal parameters.

Supporting Information

S1 Table. Association of VAMPIRE measurement between cases and control in left eye, right eye and both eyes.

https://doi.org/10.1371/journal.pone.0127914.s001

(DOCX)

Acknowledgments

This research has been conducted using the UK Biobank Resource. The collection of eye & vision data in UK Biobank was supported in part by a grant from the NIHR Biomedical Research Centre at Moorfields Eye Hospital and UCL Institute of Ophthalmology. The UK Biobank Eye and Vision Consortium is supported by a grant from The Special Trustees of Moorfields Eye Hospital. The main contact for this consortium is Prof Paul Foster (p.foster@ucl.ac.uk) and the list of members is available from the consortium website (http://www.ukbiobankeyeconsortium.org.uk/people). We would like to acknowledge the contributions of our colleagues and collaborators to the work reported: R Annunziata, L Ballerini, S Borooah, L Hughes, E Pellegrini, D Relan, G Robertson, and K Zutis.

Author Contributions

Conceived and designed the experiments: TM BD ET. Performed the experiments: JR AE CM ZS. Analyzed the data: TM JR QZ AE CS. Contributed reagents/materials/analysis tools: TM ET. Wrote the paper: TM JR FD PF ET CS. Designed the software used in the analysis: TM FD ET. Statistical analysis of the data: QZ CS.

References

  1. 1. Department of Health, UK Government. Long Term Conditions Compendium of Information (Third Edition); 2012 [updated on 30 May 2012; cited on 7 November 2013]. Available from: https://www.gov.uk/
  2. 2. The Scottish Government. Long term Conditions; 2013 [updated on 22 November 2013; cited on 27 November 2013]. Available from: http://www.scotland.gov.uk/Topics/Health/Services/Long-Term-Conditions
  3. 3. Department of Health, Social Services and Public Safety, Northern Ireland Government. Living with Long Term Conditions; April 2012 [cited on 22 November 2013]. Available from: http://www.dhsspsni.gov.uk/long-term-condition.htm
  4. 4. Centers for Disease Control and Prevention. Chronic Diseases and Health Promotion; 2013 [updated on 13 August 2012; cited on 9 December 2013]. Available from: http://www.cdc.gov/chronicdisease/overview/index.htm
  5. 5. Liew G, Wang JJ. Retinal Vascular Signs: A Window to the Heart? Revista Española de Cardiología (English Edition). 2011;64(6):515–521.
  6. 6. Patton N, Aslam T, MacGillivray TJ, Pattie A, Deary IJ, Dhillon B. Retinal vascular image analysis as a potential screening tool for cerebrovascular disease: a rationale based on homology between cerebral and retinal microvasculatures. Journal of anatomy. 2005;206(4):319–348. pmid:15817102
  7. 7. Doubal FN, Hokke PE, Wardlaw JM. Retinal microvascular abnormalities and stroke: a systematic review. J Neurol Neurosurg Psychiatry. 2009 Feb;80(2):158–65. pmid:18931002
  8. 8. Cheung N, Mitchell P, Wong TY, Diabetic retinopathy. The Lancet. 2010 July;376(9735):124–136. pmid:20580421
  9. 9. Baker ML, Hand PJ, Wang JJ, Wong TY. Retinal signs and stroke: revisiting the link between the eye and brain. Stroke. 2008;39(4), 1371–1379. pmid:18309171
  10. 10. London A, Benhar I, Schwartz M. The retina as a window to the brain—from eye research to CNS disorders. Nature Reviews Neurology. 2013 Jan;9:44–53 pmid:23165340
  11. 11. The UKBiobank; 2013 [cited on 9 December 2013]. Available from: http://www.ukbiobank.ac.uk/
  12. 12. MacGillivray TJ, Perez-Rovira A, Trucco E, Chin KS, Giachetti A, Lupascu C, et al. VAMPIRE: Vessel Assessment and Measurement Platform for Images of the Retina. Human Eye Imaging and Modeling.CRC Press; 2012.
  13. 13. Trucco E, Ballerini L, Relan D, et al. Novel VAMPIRE algorithms for quantitative analysis of the retinal vasculature. In: Proceedings fromthe 4th IEEE Biosignals and Biorobotics Conference (ISSNIP/BRC). 2013:1–4.
  14. 14. Patton N, Aslam TM, MacGillivray TJ, MacGillivray T, Zutis K, Lupascu C, et al. Retinal image analysis: Concepts, applications and potential. Progress in Retinal and Eye Research. 2006 January;25(1):99–127. pmid:16154379
  15. 15. Giachetti A, Ballerini L, Trucco E, Wilson PJ. The use of radial symmetry to localize retinal landmarks. Computerized Medical Imaging and Graphics. 2013 Sep; 37(5):369–376.
  16. 16. Trucco E, Azegrouz H, Dhillon B. Modelling the tortuosity of retinal vessels: does calibre play a role? IEEE Trans on Biomedical Engineering. Sep 2010;57(9):2239–2247. pmid:20515707
  17. 17. The UK Biobank Ethics and Governance Council; 2013 [cited on 9 December 2013]. Available from: http://www.egcukbiobank.org.uk/
  18. 18. The UK Biobank Access Procedures; 2014 [cited on 11 June 2014]. Available from: http://www.ukbiobank.ac.uk/wp-content/uploads/2012/09/Access-Procedures-2011.pdf
  19. 19. Murdoch IE, Morris SS, Cousens SN. People and eyes: statistical approaches in ophthalmology. Br J Ophthalmol. 1998 Aug;82(8):971–3. Review. pmid:9828786; PubMed Central PMCID: PMC1722711.
  20. 20. Wong TY, Klein R, Sharrett AR, Duncan BB, Coupar DJ, Tielsch JM, et al. Retinal arteriolar narrowing and risk of coronary heart disease in men and women. The Atherosclerosis Risk in Communities Study. JAMA. 2002 Mar 6;287(9):1153–9. pmid:11879113
  21. 21. Ikram MK, Witteman JC, Vingerling JR, Breteler MM, Hofman A, de Jong PT. Retinal vessel diameters and risk of hypertension: the Rotterdam Study. Hypertension. 2006 Feb;47(2):189–94. pmid:16380526
  22. 22. Kawasaki R, Xie J, Cheung N, Lamoureux E, Klein R, Klein BE, et al. Retinal microvascular signs and risk of stroke: the Multi-Ethnic Study of Atherosclerosis (MESA). Stroke. 2012 Dec;43(12):3245–51. pmid:23111439
  23. 23. Bertelsen G, Erke MG, von Hanno T, Mathiesen EB, Peto T, Sjølie AK, et al. The Tromsø Eye Study: study design, methodology and results on visual acuity and refractive errors. Acta Ophthalmol. 2013 Nov;91(7):635–42. pmid:22963377
  24. 24. Hubbard LD, Brothers RJ, King WN, Clegg LX, Klein R, Cooper LS, et al. Methods for evaluation of retinal microvascular abnormalities associated with hypertension/sclerosis in the Atherosclerosis Risk in Communities Study. Ophthalmology. 1999 Dec;106(12):2269–80. pmid:10599656
  25. 25. Knudtson MD, Lee KE, Hubbard LD, Wong TY, Klein R, Klein BE. Revised formulas for summarizing retinal vessel diameters. Curr Eye Res. 2003 Sep;27(3):143–9. pmid:14562179
  26. 26. Patton N, Aslam T, MacGillivray T, Dhillon B, Constable I. Asymmetry of retinal arteriolar branch widths at junctions affects ability of formulae to predict trunk arteriolar widths. Invest Ophthalmol Vis Sci. 2006 Apr;47(4):1329–33. pmid:16565364
  27. 27. Niemeijer M, Xu X, Dumitrescu AV, Gupta P, van Ginneken B, Folk JC, et al. Automated measurement of the arteriolar-to-venular width ratio in digital color fundus photographs. IEEE Trans Med Imaging. 2011 Nov;30(11):1941–50. pmid:21690008
  28. 28. Witt N, Wong TY, Hughes AD, Chaturvedi N, Klein BE, Evans R, et al. Abnormalities of retinal miscovascular structure and risk of mortality from ischemic heart disease and stroke. Hypertension. 2006 May;47(5): 975–981. pmid:16585415
  29. 29. Kiely AE, Wallace DK, Freedman SF, Zhao. Computer-assisted measurement of retinal vascular width and tortuosity in retinopathy of prematurity. Archives of Ophthalmology. 2010 Jul; 128(7):847–852. pmid:20625044
  30. 30. Dougherty G, Johnson MJ, Wiers MD. Measurement of retinal vascular tortuosity and its application to retinal pathologies. Med Biol Eng Comput. 2010 Jan;48(1):87–95. pmid:20012560
  31. 31. Cheung CY, Zheng Y, Hsu W, Lee ML, Lau QP, Mitchell P, et al. Retinal vascular tortuosity, blood pressure, and cardiovascular risk factors. Ophthalmology. 2011 May;118(5):812–8 pmid:21146228
  32. 32. Owen CG, Rudnicka AR, Nightingale CM, Mullen R, Barman SA, Sattar N, et al. Retinal Arteriolar Tortuosity and cardiovascular risk factors in a multi-ethnic population study of 10-year-Old children; the Child Heart and Health Study in England (CHASE). ArteriosclerThrombVasc Biol. 2011 Aug;31(8):1933–8.
  33. 33. Habib MS, Al-Diri B, Hunter A, Steel DHW. The association between retinal vascular geometry changes and diabetic retinopathy and their role in prediction of progression—an exploratory study. BMC Ophthalmol. 2014 Jul 7;14:89. pmid:25001248
  34. 34. Mitry D, Peto T, Hayat S, Morgan JE, Khaw KT, Foster PJ. Crowdsourcing as a novel technique for retinal fundus photography classification: analysis of images in the EPIC Norfolk cohort on behalf of the UK Biobank Eye and Vision Consortium. PLoS One. 2013 Aug 21;8(8):e71154. pmid:23990935