Cross-sectional blindness prevalence surveys are essential to plan and monitor eye care services. Incomplete or inaccurate reporting can prevent effective translation of research findings. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement is a 32 item checklist developed to improve reporting of observational studies. The aim of this study was to assess the completeness of reporting in blindness prevalence surveys in low and middle income countries (LMICs) using STROBE.
MEDLINE, EMBASE and Web of Science databases were searched on April 8 2016 to identify cross-sectional blindness prevalence surveys undertaken in LMICs and published after STROBE was published in December 2007. The STROBE tool was applied to all included studies, and each STROBE item was categorized as ‘yes’ (met criteria), ‘no’ (did not meet criteria) or ‘not applicable’. The ‘Completeness of reporting (COR) score’ for each manuscript was calculated: COR score = yes / [yes + no]. In journals with included studies the instructions to authors and reviewers were checked for reference to STROBE.
The 89 included studies were undertaken in 32 countries and published in 37 journals. The mean COR score was 60.9% (95% confidence interval [CI] 58.1–63.7%; range 30.8–88.9%). The mean COR score did not differ between surveys published in journals with author instructions referring to STROBE (10/37 journals; 61.1%, 95%CI 56.4–65.8%) or in journals where STROBE was not mentioned (60.9%, 95%CI 57.4–64.3%; p = 0.93).
Citation: Ramke J, Palagyi A, Jordan V, Petkovic J, Gilbert CE (2017) Using the STROBE statement to assess reporting in blindness prevalence surveys in low and middle income countries. PLoS ONE 12(5): e0176178. https://doi.org/10.1371/journal.pone.0176178
Editor: Rohit C. Khanna, LV Prasad Eye Institute, INDIA
Received: August 17, 2016; Accepted: April 6, 2017; Published: May 8, 2017
Copyright: © 2017 Ramke et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Cross-sectional prevalence surveys are essential to plan and monitor local and national eye care service delivery, as well as to monitor the prevalence and causes of blindness and visual impairment globally. The latest estimate of global blindness drew on 227 prevalence surveys from 84 countries undertaken between 1990 and 2012. In the current global eye health action plan the World Health Organization (WHO) called for more prevalence surveys to be undertaken in low and middle income countries (LMICs) to enable evidence-informed planning.
Resources for health research are scarce—especially in LMICs—so it is essential that research findings are translated into health care policy and practice. Incomplete or inaccurate reporting of study methods and results hinders understanding of a study’s rigour, and limits interpretation and effective translation of the findings. The manuscripts reporting blindness and visual impairment prevalence surveys (hereafter referred to as blindness prevalence surveys) do not always allow the reader to understand “what was planned, what was done, what was found, and what conclusions were drawn”. Incomplete reporting can also make synthesis of evidence difficult or impossible.
To improve reporting and facilitate critical appraisal of observational studies, including cross-sectional surveys, the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement was developed from a collaboration of epidemiologists, methodologists, statisticians, researchers and journal editors. The STROBE statement was disseminated in 2007 and provides a checklist of 32 items recommended for inclusion in the reporting of observational studies. The items include aspects of study design, sample selection, data collection, analysis, and potential bias. To date it has been endorsed by more than 120 journals, including The Lancet and PLoS (see list at http://www.strobe-statement.org/).
This study aimed to assess the extent to which items in the STROBE cross-sectional checklist (available for download at http://www.strobe-statement.org/) were reported in blindness prevalence surveys published after STROBE was disseminated.
The search and study selection process is outlined in Box 1.
Box 1. Study search and selection process
Date—April 8 2016
Databases—MEDLINE, EMBASE and Web of Science
Algorithm—‘blindness or vis* impairment or low vision’ and ‘prevalence or rapid assessment or population-based’.
Limits—Published between 1 January 2008 and 31 March 2016
Additional searching—Examined reference lists of published reviews of blindness prevalence[6–9]
Process—Two researchers screened the titles of all citations identified during the initial search and the full-text manuscript was retrieved for review if the citation was potentially relevant
- conducted in countries classified by The World Bank as LMIC in 2014 (Gross National Income <US$8,498) 
- presented cross-sectional population-based data
- provided information on blindness prevalence based on subjective visual acuity measurement
- published in English
- undertaken in specific populations (e.g. in hospitals, ‘institutionalised’, ‘diabetics’)
- only included children
- only reported disease-specific blindness prevalence
- only reported blindness as an explanatory variable rather than as an outcome
Following discussion, we considered three of the 32 STROBE items not applicable to blindness prevalence reporting a priori. Sensitivity analysis (item 12e) was considered not applicable because we were focused on the reporting of the main result; choice of category boundaries for continuous variables (item 16b) because blindness (the outcome) is categorical and dichotomous (i.e. blind or not blind); and translating relative risk into absolute risk (item 16c) because the outcome was a prevalence rate, not a measure of association. The remaining 29 items were converted to questions to assist with consistent assessment. Three authors (JR, JP, VJ) independently reviewed one manuscript (not included in the sample) against the 29 STROBE items to develop the questions and corresponding reviewer guidelines through discussion and consensus. Five of the included manuscripts were then reviewed by the same three authors to clarify any discrepancies in application of the questions and finalise the guidelines. Each remaining manuscript in the sample was then reviewed by two authors (JR, JP, VJ, AP), with one author (JR) reviewing all. Discrepancies were resolved by discussion and consensus. In addition to the STROBE items, each included study was checked for a reference to STROBE, and whether the Rapid Assessment of Avoidable Blindness (RAAB) methodology was used. The 2015 SCImago Journal Rank (SJR) quartile was extracted for each ophthalmology journal with an included study (http://www.scimagojr.com) and categorised as ophthalmology journal top-quartile (Q1), ophthalmology journal outside top-quartile (Q2–4), or not ophthalmology journal.
Data were collated using an Excel spreadsheet (Microsoft Corp, Redmond, Washington). Each question was categorized as ‘yes’ (met criteria), ‘no’ (did not meet criteria) or ‘not applicable’. We did not include a ‘partly’ option, to avoid it being used to negate having to choose between yes and no. Where a STROBE item contained several components, data were collected for each component. For example, item 5 relates to reporting “….setting, location and relevant dates…..” so data were collected separately on i) setting/location and ii) relevant dates. For item 5 to be assessed as adequate, both components needed to be reported.
To assess the extent to which STROBE was referred to by the journals in which included studies were published, two authors (JR and AP) checked the publicly available instructions for authors and reviewers. Each set of accessible instructions was categorised as STROBE being: required, recommended, listed (without recommendation), or not mentioned.
Analysis was completed using Excel and Stata 12.0 (StataCorp LP, TX). Descriptive analysis included the number and proportion of manuscripts reporting i) each of the 29 STROBE items, ii) use of RAAB methodology and iii) use of STROBE to guide reporting; the number and proportion of manuscripts published in the top-quartile of ophthalmology journals; and the number of journals including STROBE in instructions to i) authors and ii) reviewers.
The ‘Completeness of reporting (COR) score’ for each manuscript was calculated as the ‘yes’ answers as a proportion of the ‘yes + no’ answers: COR score (%) = [yes / (yes + no)] * 100. Consequently, the ‘not applicable’ items did not impact results.
Normality of COR scores was confirmed using distribution and probability plots. The mean COR score and 95% confidence interval (CI) were calculated for each category of author and reviewer instructions (i.e. required, recommended, listed, not mentioned, not accessible); use of RAAB methodology (used RAAB or other methodology); and publication in the top-quartile of ophthalmology journals (in the top-quartile, not in the top-quartile, not in an ophthalmology journal). One-way ANOVA or two-sample t-test was used to test for a difference between the mean scores of the categories. The median and inter-quartile range of the COR score within each manuscript section was calculated.
Some of the studies published in 2008 may have been prepared and reviewed prior to the dissemination of STROBE in 2007, so we performed sensitivity analysis to assess whether the inclusion of studies published in 2008 impacted our results. We calculated the COR score for studies published in 2008 and published after 2008, and used the two-sampled t-test to assess the statistical significance of the difference between mean COR scores in the two time periods.
Summary of included studies
The search identified 2942 studies (Fig 1), of which 89 met the inclusion criteria (listed in S1 Table). The studies were undertaken in 32 countries and published in 37 different journals. Eight of the 37 journals were in the top-quartile of ophthalmology journals (SJR quartile), and published almost half of the included studies (n = 42). More than one-third of studies (37%, n = 33) reported using the RAAB protocol. No manuscript reported using STROBE to guide reporting.
Journals and STROBE
Of the 37 journals that published included studies, ten (27%) mentioned STROBE in instructions to authors—STROBE was listed without further instruction in four journals, recommended in five journals, and required by one journal (PLOS ONE). Reviewer instructions were available for 15 (41%) journals; 12 of these did not mention STROBE, two listed STROBE without further instruction and one (PLOS ONE) required adherence to reporting guidelines (including STROBE) for publication (Table 1). Only one of the eight top-ranked ophthalmology journals (that published three included studies) recommended the use of STROBE by authors, and none with reviewer comments available referred to STROBE. In contrast, of the 15 ophthalmology journals outside the top-quartile, four journals (publishing seven studies) listed STROBE and two journals (publishing four studies) recommended STROBE in instructions to authors; none mentioned STROBE in available reviewer instructions.
Completeness of reporting
COR scores ranged from 30.8% to 88.9%, with a mean of 60.9% (95%CI 58.1–63.7%; Table 1). The mean COR score of studies that used RAAB methodology (58.8%, 95%CI 54.9–62.8%) was not different to studies using other methodologies (62.2%, 95%CI 58.4–65.9%; t(87) = 1.15, p = 0.25). The mean COR score of studies published in the top-quartile of ophthalmology journals was higher than studies published in lower-ranked ophthalmology journals or non-ophthalmology journals (Table 1; F(2,86) = 7.06, p = 0.001).
Inclusion of STROBE in instructions to authors did not tend to improve reporting—the mean score of studies published in journals with instruction to authors that required, recommended or listed STROBE (61.1%, 95%CI 56.4–65.8%) was not different to the score for studies published in journals not mentioning STROBE (60.9%, 95%CI 57.4–64.3%; t(87) = 0.09, p = 0.93); the mean score of studies published in the journal requiring STROBE (68.2%, 95%CI 64.0–72.4%) was higher than the score for the other studies combined (60.0%, 95%CI 57.0–63.1%) but this was not statistically significant (t(87) = 1.87, p = 0.065). Inclusion of STROBE in reviewer instructions also failed to improve reporting (64.3%, 95%CI 57.1–71.6% for required/recommended/listed versus 64.6%, 95%CI 59.8–69.5% for not mentioned; t(36) = 0.07, p = 0.95).
Sensitivity analysis showed there was no difference in the mean COR score of those studies published during 2008 and those published after 2008 (t(87) = 0.99, p = 0.32).
Reporting of STROBE items
Nine items were reported in 90% or more of the included manuscripts (Table 2): providing an informative abstract (item 1b, 97%), scientific background (item 2, 96%) and study objectives (item 3, 99%); presenting key elements of study design (item 4, 91%); describing eligibility criteria (item 6, 92%); providing characteristics of study participants (item 14a, 90%) and number of outcome events (item 15, 91%); and summarising (item 18, 90%) and interpreting (item 20, 94%) results.
Seven items were reported in less than one-third of manuscripts: defining outcomes and diagnostic criteria (item 7, 31%); describing methods of measurement (item 8, 27%); explaining how missing data were managed (item 12c, 3%); reporting missing data (item 14b, 33%); discussing study limitations (item 19, 17%) and generalisability (item 21, 30%); and providing the name and role of the funder (item 22, 22%). Partial reporting within a number of these items reduced the overall score. For example, for item 7 blindness was commonly defined (in 92% of manuscripts) while diagnostic criteria were not (33%); and for item 22 the funder was often named (78%) but their role was not defined (17%; Table 2).
Overall, the extent of reporting varied notably across and within manuscript sections, with applicable items being most commonly reported in the Introduction (median COR Score = 97%; 2 items: 99% and 96%), followed by the Abstract (76%; 2 items: 97% and 55%), Results (65%, IQR 41–87), Methods (47%, IQR 33–71) and Discussion (51%, IQR 22–90).
The WHO’s global eye health action plan has called for more blindness prevalence surveys to be undertaken in LMICs. Our results indicate that reporting of such surveys must be improved to maximise effective translation of the findings, and to aid synthesis. No single blindness prevalence manuscript published since the dissemination of STROBE adhered to more than 90% of STROBE items, and no single STROBE item was adequately addressed across all manuscripts (Table 2). Reporting was weakest in the Methods and Discussion sections. If authors improved adherence to STROBE, readers would better understand what was done and therefore whether the survey findings (i.e. blindness prevalence estimates) are valid. Further, better adherence to STROBE would enable evidence-informed planning and monitoring of eye care services, and more accurate global estimates when results are synthesised.
Journal endorsement and use of reporting guidelines are expected to improve completeness of reporting. However, with the exception of CONSORT (CONsolidated Standards of Reporting Trials), few evaluations have assessed whether journal endorsement of guidelines improves reporting. The CONSORT guidelines were launched in 1997 to improve reporting of randomised control trials (RCTs). A systematic review found that journals endorsing CONSORT improved reporting of RCTs, but the review also identified a gap between journals endorsing CONSORT and consistently implementing it. Our results reflect a similar gap for STROBE—endorsement of STROBE in a journal’s instructions for authors and reviewers was not reflected by better reporting of the blindness prevalence surveys they published (Table 1). This lack of translation suggests that endorsement of reporting guidelines without enforcement by journal editors and reviewers is ineffectual. Some journals are trying to strengthen author application of STROBE. For example, The BMJ and PLOS Medicine require authors to complete the STROBE checklist to identify accomplishment of each item, with the checklist published alongside the study.[15, 16] This strategy could be implemented by ophthalmology journals, and further strengthened by requiring reviewers to assess adherence to STROBE when undertaking reviews. We identified weaker reporting in lower-ranked ophthalmology journals (2015 SJR quartiles 2−4), suggesting these journals may benefit most from the use of STROBE to assist authors and reviewers.
STROBE development intentionally avoided being prescriptive, which resulted in generalized guidelines, some of which are not applicable to blindness prevalence surveys. We omitted three items from our assessment a priori, and others (such as the use of a flow diagram [13c]) seem less applicable after undertaking the review. The COR score calculation gave equal weight to each item in the STROBE checklist. As some items are arguably less important for reporting blindness prevalence surveys, the COR score is an imperfect summary measure. With the anticipated increase in surveys in the coming years, development of specific reporting guidelines for blindness prevalence surveys may be beneficial, and could include the identification of ‘essential’ items, as well as items that are not applicable. Fortunately, the process followed to develop such guidelines in other fields are available from the Equator Network (http://www.equator-network.org/). Until specific guidelines become available, the existing STROBE checklist can be used by authors of all blindness prevalence surveys.
Excluding some STROBE items from our assessment limits the comparability of our findings with other studies. Also, few studies report analysis for cross-sectional surveys separately to cohort and case-control studies. Despite these differences, our mean COR score (60.9%, 95%CI 58.1–63.7%) was similar to equivalent rates of ‘yes’/ ‘applicable’ items reported from STROBE assessment in diverse fields including orthopaedics (58%), hand surgery (58% post-STROBE dissemination), and dermatology (58% post-STROBE).
RAAB surveys are the most commonly published blindness prevalence survey methodology. Reasons for this include that RAABs are shorter and less expensive than traditional surveys; the protocol and analysis software are readily available; and RAABs have been designed to be implemented by local eye health workers following standardised training. We found that the completeness of reporting of RAAB and non-RAAB surveys was similar. With the number of RAABs set to increase in the coming years, expanding the focus on reporting and the use of guidelines during RAAB training is a promising strategy to improve reporting of blindness prevalence surveys. In addition, ongoing monitoring of reporting quality may be beneficial, using the COR score (or similar), as well as graphical displays.
This analysis must be interpreted in the context of its limitations. Assessment of included studies was unblinded, which may have introduced bias through presumed reputation of some authors. This potential for bias was lessened by two of the four reviewers being outside the field of eye care. Reviewer fatigue may also have been an issue, as data collection took place over a five-month period―we attempted to reduce this by developing assessment guidelines, using two reviewers for each study, and requiring any discrepancy to be discussed and resolved. In addition to English language studies, the search found 12 non-duplicate studies in Chinese and four in Spanish that we were unable to assess. The STROBE statement is available in these languages, and could be applied by researchers fluent in reading them. Finally, we were unable to assess whether STROBE was included in instructions to authors at the time of submission of each individual manuscript. We believe this had little influence on our results―less than one-third of journals referred to STROBE in 2016, and it is unlikely that journals previously included STROBE in their instructions and subsequently removed it.
While not a limitation, to assist interpretation of our results it must be noted that we assessed completeness of reporting and not methodological quality or risk of bias of the included studies. That is, we recorded whether an item was reported in sufficient detail to enable appraisal (e.g. steps taken to reduce bias in cluster selection), not whether the methods employed were sufficient to achieve the specific aim (e.g. avoid selection bias). Further, it is possible some of the checklist items were fulfilled during study design, implementation or analysis, but without complete reporting this remains unknown.
A scan of the literature suggests this is the first analysis of completeness of reporting of blindness prevalence surveys. It has revealed strong reporting in some areas, with other areas in need of improvement. The anticipated increase in surveys in LMICs in the coming years provides the impetus for stakeholders to identify strategies to improve the quality of reporting and therefore increase the utility of survey findings. Supporters of research (international and national non-government organisations as well as academic institutions) could extend support beyond study implementation to also strengthen local research capabilities, including reporting. Endorsement of STROBE by journals should help, but only if its use is consistently enforced by editorial staff and reviewers. We recommend that ophthalmology journals adopt the STROBE checklist and ensure it is used by authors and reviewers.
S2 File. STROBE and blindness prevalence surveys data.
- Conceptualization: JR AP VJ JP CG.
- Data curation: JR.
- Formal analysis: JR.
- Investigation: JR AP VJ JP.
- Methodology: JR AP VJ JP CG.
- Writing – original draft: JR.
- Writing – review & editing: AP VJ JP CG.
- 1. Stevens GA, White RA, Flaxman SR, Price H, Jonas JB, Keeffe J, et al. Global prevalence of vision impairment and blindness: Magnitude and temporal trends, 1990–2010. Ophthalmology. 2013;120(12):2377–84. pmid:23850093
- 2. World Health Organization. Universal Eye Health: A global action plan 2014–2019. Geneva: World Health Organization, 2013.
- 3. World Health Organization. The WHO strategy on research for health. Geneva: World Health Organization, 2012 9241503254.
- 4. Vandenbroucke JP, Von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. Plos Med. 2007;4(10):e297. pmid:17941715
- 5. da Costa BR, Cevallos M, Altman DG, Rutjes AWS, Egger M. Uses and misuses of the STROBE statement: bibliographic study. BMJ Open. 2011;1(1):e000048. pmid:22021739
- 6. Pascolini D, Mariotti SP. Global estimates of visual impairment: 2010. Br J Ophthalmol. 2012;96(5):614–8. pmid:22133988
- 7. Bourne R, Price H, Taylor H, Leasher J, Keeffe J, Glanville J, et al. New Systematic Review Methodology for Visual Impairment and Blindness for the 2010 Global Burden of Disease Study. Ophthalmic Epidemiol. 2013;20(1):33–9. pmid:23350553
- 8. Furtado JM, Lansingh VC, Carter MJ, Milanese MF, Peña BN, Ghersi HA, et al. Causes of Blindness and Visual Impairment in Latin America. Surv Ophthalmol. 2011;57(2):149–77. pmid:22137039
- 9. Murthy GV, Johnson GJ. Prevalence, incidence and distribution of visual impairment. In: Johnson GJ, Minassian DC, Weale RA, West SK, editors. The epidemiology of eye disease. Third ed. London: Imperial College Press; 2012.
- 10. World Bank. Countries and economies 2014 [1 July 2015]. http://data.worldbank.org/country.
- 11. Hoy D, Brooks P, Woolf A, Blyth F, March L, Bain C, et al. Assessing risk of bias in prevalence studies: modification of an existing tool and evidence of interrater agreement. J Clin Epidemiol. 2012;65(9):934–9. pmid:22742910
- 12. Ebrahim S, Clarke M. STROBE: new standards for reporting observational epidemiology, a chance to improve. Int J Epidemiol. 2007;36(5):946–8. pmid:17890280
- 13. Stevens A, Shamseer L, Weinstein E, Yazdi F, Turner L, Thielman J, et al. Relation of completeness of reporting of health research to journals’ endorsement of reporting guidelines: systematic review. BMJ. 2014;348:g3804. pmid:24965222
- 14. Turner L, Shamseer L, Altman DG, Schulz KF, Moher D. Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review. Syst Rev. 2012;1(1):60.
- 15. PLOS Medicine Editors. Observational studies: getting clear about transparency. PloS Med. 2014;11(8):e1001711. pmid:25158064
- 16. The British Medical Journal. Resource for authors. http://www.bmj.com/about-bmj/resources-authors/article-types/research.
- 17. Cobo E, Cortés J, Ribera J, Cardellach F, Selva-O’Callaghan A, Kostov B, et al. Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial. BMJ. 2011;343:d6783. pmid:22108262
- 18. Parsons NR, Hiskens R, Price CL, Achten J, Costa ML. A systematic survey of the quality of research reporting in general orthopaedic journals. J Bone Joint Surg Br. 2011;93(9):1154–9. pmid:21911523
- 19. Sorensen AA, Wojahn RD, Manske MC, Calfee RP. Using the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement to assess reporting of observational trials in hand surgery. J Hand Surg Am. 2013;38(8):1584–9.e2. pmid:23845586
- 20. Bastuji-Garin S, Sbidian E, Gaudy-Marqueste C, Ferrat E, Roujeau J-C, Richard M-A, et al. Impact of STROBE statement publication on quality of observational study reporting: interrupted time series versus before-after analysis. PloS One. 2013;8(8):e64733. pmid:23990867
- 21. International Centre for Eye Health. Rapid Assessment of Avoidable Blindness 2014. http://iceh.lshtm.ac.uk/rapid-assessment-of-avoidable-blindness/.
- 22. Kuper H, Polack S, Limburg H. Rapid assessment of avoidable blindness. Community Eye Health. 2006;19(60):68–9. pmid:17515970
- 23. Morissette K, Tricco AC, Horsley T, Chen MH, Moher D. Blinded versus unblinded assessments of risk of bias in studies included in a systematic review. Cochrane Database Syst Rev. 2011;(Issue 9):Art. No.: MR000025.
- 24. Institute of Social and Preventive Medicine. University of Bern. Translations of STROBE Statement 2009. http://www.strobe-statement.org/index.php?id=strobe-translations.