Using the STROBE statement to assess reporting in blindness prevalence surveys in low and middle income countries

Objective Cross-sectional blindness prevalence surveys are essential to plan and monitor eye care services. Incomplete or inaccurate reporting can prevent effective translation of research findings. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement is a 32 item checklist developed to improve reporting of observational studies. The aim of this study was to assess the completeness of reporting in blindness prevalence surveys in low and middle income countries (LMICs) using STROBE. Methods MEDLINE, EMBASE and Web of Science databases were searched on April 8 2016 to identify cross-sectional blindness prevalence surveys undertaken in LMICs and published after STROBE was published in December 2007. The STROBE tool was applied to all included studies, and each STROBE item was categorized as ‘yes’ (met criteria), ‘no’ (did not meet criteria) or ‘not applicable’. The ‘Completeness of reporting (COR) score’ for each manuscript was calculated: COR score = yes / [yes + no]. In journals with included studies the instructions to authors and reviewers were checked for reference to STROBE. Results The 89 included studies were undertaken in 32 countries and published in 37 journals. The mean COR score was 60.9% (95% confidence interval [CI] 58.1–63.7%; range 30.8–88.9%). The mean COR score did not differ between surveys published in journals with author instructions referring to STROBE (10/37 journals; 61.1%, 95%CI 56.4–65.8%) or in journals where STROBE was not mentioned (60.9%, 95%CI 57.4–64.3%; p = 0.93). Conclusion While reporting in blindness prevalence surveys is strong in some areas, others need improvement. We recommend that more journals adopt the STROBE checklist and ensure it is used by authors and reviewers.


Introduction
Cross-sectional prevalence surveys are essential to plan and monitor local and national eye care service delivery, as well as to monitor the prevalence and causes of blindness and visual impairment globally. The latest estimate of global blindness drew on 227 prevalence surveys from 84 countries undertaken between 1990 and 2012. [1] In the current global eye health action plan the World Health Organization (WHO) called for more prevalence surveys to be undertaken in low and middle income countries (LMICs) to enable evidence-informed planning. [2] Resources for health research are scarce-especially in LMICs-so it is essential that research findings are translated into health care policy and practice. [3] Incomplete or inaccurate reporting of study methods and results hinders understanding of a study's rigour, and limits interpretation and effective translation of the findings. The manuscripts reporting blindness and visual impairment prevalence surveys (hereafter referred to as blindness prevalence surveys) do not always allow the reader to understand "what was planned, what was done, what was found, and what conclusions were drawn". [4] Incomplete reporting can also make synthesis of evidence difficult or impossible. [5] To improve reporting and facilitate critical appraisal of observational studies, including cross-sectional surveys, the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement was developed from a collaboration of epidemiologists, methodologists, statisticians, researchers and journal editors. [4] The STROBE statement was disseminated in 2007 and provides a checklist of 32 items recommended for inclusion in the reporting of observational studies. The items include aspects of study design, sample selection, data collection, analysis, and potential bias. [4] To date it has been endorsed by more than 120 journals, including The Lancet and PLoS (see list at http://www.strobe-statement.org/). This study aimed to assess the extent to which items in the STROBE cross-sectional checklist (available for download at http://www.strobe-statement.org/) were reported in blindness prevalence surveys published after STROBE was disseminated.

Methods
The search and study selection process is outlined in Box 1.

Data collection
Following discussion, we considered three of the 32 STROBE items not applicable to blindness prevalence reporting a priori. Sensitivity analysis (item 12e) was considered not applicable because we were focused on the reporting of the main result; choice of category boundaries for continuous variables (item 16b) because blindness (the outcome) is categorical and dichotomous (i.e. blind or not blind); and translating relative risk into absolute risk (item 16c) because the outcome was a prevalence rate, not a measure of association. The remaining 29 items were converted to questions to assist with consistent assessment. Three authors (JR, JP, VJ) independently reviewed one manuscript (not included in the sample) against the 29 STROBE items to develop the questions and corresponding reviewer guidelines through discussion and consensus. Five of the included manuscripts were then reviewed by the same three authors to clarify any discrepancies in application of the questions and finalise the guidelines. Each remaining manuscript in the sample was then reviewed by two authors (JR, JP, VJ, AP), with one author (JR) reviewing all. Discrepancies were resolved by discussion and consensus. In addition to the STROBE items, each included study was checked for a reference to STROBE, and whether the Rapid Assessment of Avoidable Blindness (RAAB) methodology was used. The 2015 SCImago Journal Rank (SJR) quartile was extracted for each ophthalmology journal with an included study (http://www.scimagojr.com) and categorised as ophthalmology journal top-quartile (Q1), ophthalmology journal outside top-quartile (Q2-4), or not ophthalmology journal.
Data were collated using an Excel spreadsheet (Microsoft Corp, Redmond, Washington). Each question was categorized as 'yes' (met criteria), 'no' (did not meet criteria) or 'not Box 1. Study search and selection process Search Date-April 8 2016 Databases-MEDLINE, EMBASE and Web of Science Algorithm-'blindness or vis Ã impairment or low vision' and 'prevalence or rapid assessment or population-based'.
Limits-Published between 1 January 2008 and 31 March 2016 Additional searching-Examined reference lists of published reviews of blindness prevalence [6][7][8][9] Process-Two researchers screened the titles of all citations identified during the initial search and the full-text manuscript was retrieved for review if the citation was potentially relevant • presented cross-sectional population-based data • provided information on blindness prevalence based on subjective visual acuity measurement

Exclusion
• undertaken in specific populations (e.g. in hospitals, 'institutionalised', 'diabetics') • only included children • only reported disease-specific blindness prevalence • only reported blindness as an explanatory variable rather than as an outcome applicable'. We did not include a 'partly' option, to avoid it being used to negate having to choose between yes and no. [11] Where a STROBE item contained several components, data were collected for each component. For example, item 5 relates to reporting ". . ..setting, location and relevant dates. . ..." so data were collected separately on i) setting/location and ii) relevant dates. For item 5 to be assessed as adequate, both components needed to be reported.
To assess the extent to which STROBE was referred to by the journals in which included studies were published, two authors (JR and AP) checked the publicly available instructions for authors and reviewers. Each set of accessible instructions was categorised as STROBE being: required, recommended, listed (without recommendation), or not mentioned.

Analysis
Analysis was completed using Excel and Stata 12.0 (StataCorp LP, TX). Descriptive analysis included the number and proportion of manuscripts reporting i) each of the 29 STROBE items, ii) use of RAAB methodology and iii) use of STROBE to guide reporting; the number and proportion of manuscripts published in the top-quartile of ophthalmology journals; and the number of journals including STROBE in instructions to i) authors and ii) reviewers.
The 'Completeness of reporting (COR) score' for each manuscript was calculated as the 'yes' answers as a proportion of the 'yes + no' answers: COR score (%) = [yes / (yes + no)] Ã 100. Consequently, the 'not applicable' items did not impact results.
Normality of COR scores was confirmed using distribution and probability plots. The mean COR score and 95% confidence interval (CI) were calculated for each category of author and reviewer instructions (i.e. required, recommended, listed, not mentioned, not accessible); use of RAAB methodology (used RAAB or other methodology); and publication in the topquartile of ophthalmology journals (in the top-quartile, not in the top-quartile, not in an ophthalmology journal). One-way ANOVA or two-sample t-test was used to test for a difference between the mean scores of the categories. The median and inter-quartile range of the COR score within each manuscript section was calculated.
Sensitivity analysis. Some of the studies published in 2008 may have been prepared and reviewed prior to the dissemination of STROBE in 2007, so we performed sensitivity analysis to assess whether the inclusion of studies published in 2008 impacted our results. We calculated the COR score for studies published in 2008 and published after 2008, and used the twosampled t-test to assess the statistical significance of the difference between mean COR scores in the two time periods.

Summary of included studies
The search identified 2942 studies (Fig 1), of which 89 met the inclusion criteria (listed in S1 Table). The studies were undertaken in 32 countries and published in 37 different journals. Eight of the 37 journals were in the top-quartile of ophthalmology journals (SJR quartile), and published almost half of the included studies (n = 42). More than one-third of studies (37%, n = 33) reported using the RAAB protocol. No manuscript reported using STROBE to guide reporting.

Journals and STROBE
Of the 37 journals that published included studies, ten (27%) mentioned STROBE in instructions to authors-STROBE was listed without further instruction in four journals, recommended in five journals, and required by one journal (PLOS ONE). Reviewer instructions were available for 15 (41%) journals; 12 of these did not mention STROBE, two listed STROBE without further instruction and one (PLOS ONE) required adherence to reporting guidelines (including STROBE) for publication (Table 1). Only one of the eight top-ranked ophthalmology journals (that published three included studies) recommended the use of STROBE by authors, and none with reviewer comments available referred to STROBE. In contrast, of the 15 ophthalmology journals outside the top-quartile, four journals (publishing seven studies) listed STROBE and two journals (publishing four studies) recommended STROBE in instructions to authors; none mentioned STROBE in available reviewer instructions.
Sensitivity analysis showed there was no difference in the mean COR score of those studies published during 2008 and those published after 2008 (t(87) = 0.99, p = 0.32).

Discussion
The WHO's global eye health action plan has called for more blindness prevalence surveys to be undertaken in LMICs. [2] Our results indicate that reporting of such surveys must be improved to maximise effective translation of the findings, and to aid synthesis. No single blindness prevalence manuscript published since the dissemination of STROBE adhered to more than 90% of STROBE items, and no single STROBE item was adequately addressed

18
Summarise key results with reference to study objectives Are key results summarised with reference to the study objectives? Items 12e, 16b, 16c removed a priori as they were considered not applicable to reporting blindness prevalence surveys. *to be assessed as a yes, individual components must be assessed as yes; NA not included; proportion (%) = yes/(yes+no). † as enumeration and participation were the only stages involved, we categorized this as yes or NA. ‡ one study found no blind participants (only vision impaired) so could not report this. § The role of funders was NA for 6 studies that received no specific funding; these studies were included in the numerator for the overall assessment of this item.
https://doi.org/10.1371/journal.pone.0176178.t002 across all manuscripts (Table 2). Reporting was weakest in the Methods and Discussion sections. If authors improved adherence to STROBE, readers would better understand what was done and therefore whether the survey findings (i.e. blindness prevalence estimates) are valid. [12] Further, better adherence to STROBE would enable evidence-informed planning and monitoring of eye care services, and more accurate global estimates when results are synthesised.
Journal endorsement and use of reporting guidelines are expected to improve completeness of reporting. However, with the exception of CONSORT (CONsolidated Standards of Reporting Trials), few evaluations have assessed whether journal endorsement of guidelines improves reporting. [13] The CONSORT guidelines were launched in 1997 to improve reporting of randomised control trials (RCTs). A systematic review found that journals endorsing CONSORT improved reporting of RCTs, but the review also identified a gap between journals endorsing CONSORT and consistently implementing it. [14] Our results reflect a similar gap for STROBE-endorsement of STROBE in a journal's instructions for authors and reviewers was not reflected by better reporting of the blindness prevalence surveys they published (Table 1). This lack of translation suggests that endorsement of reporting guidelines without enforcement by journal editors and reviewers is ineffectual. Some journals are trying to strengthen author application of STROBE. For example, The BMJ and PLOS Medicine require authors to complete the STROBE checklist to identify accomplishment of each item, with the checklist published alongside the study. [15,16] This strategy could be implemented by ophthalmology journals, and further strengthened by requiring reviewers to assess adherence to STROBE when undertaking reviews. [17] We identified weaker reporting in lower-ranked ophthalmology journals (2015 SJR quartiles 2−4), suggesting these journals may benefit most from the use of STROBE to assist authors and reviewers. STROBE development intentionally avoided being prescriptive, [4] which resulted in generalized guidelines, [12] some of which are not applicable to blindness prevalence surveys. We omitted three items from our assessment a priori, and others (such as the use of a flow diagram [13c]) seem less applicable after undertaking the review. The COR score calculation gave equal weight to each item in the STROBE checklist. As some items are arguably less important for reporting blindness prevalence surveys, the COR score is an imperfect summary measure. With the anticipated increase in surveys in the coming years, development of specific reporting guidelines for blindness prevalence surveys may be beneficial, and could include the identification of 'essential' items, as well as items that are not applicable. Fortunately, the process followed to develop such guidelines in other fields are available from the Equator Network (http://www.equator-network.org/). Until specific guidelines become available, the existing STROBE checklist can be used by authors of all blindness prevalence surveys.
Excluding some STROBE items from our assessment limits the comparability of our findings with other studies. Also, few studies report analysis for cross-sectional surveys separately to cohort and case-control studies. Despite these differences, our mean COR score (60.9%, 95%CI 58.1-63.7%) was similar to equivalent rates of 'yes'/ 'applicable' items reported from STROBE assessment in diverse fields including orthopaedics (58%), [18] hand surgery (58% post-STROBE dissemination), [19] and dermatology (58% post-STROBE). [20] RAAB surveys are the most commonly published blindness prevalence survey methodology. Reasons for this include that RAABs are shorter and less expensive than traditional surveys; the protocol and analysis software are readily available; [21] and RAABs have been designed to be implemented by local eye health workers following standardised training. [22] We found that the completeness of reporting of RAAB and non-RAAB surveys was similar. With the number of RAABs set to increase in the coming years, expanding the focus on reporting and the use of guidelines during RAAB training is a promising strategy to improve reporting of blindness prevalence surveys. In addition, ongoing monitoring of reporting quality may be beneficial, using the COR score (or similar), as well as graphical displays.
This analysis must be interpreted in the context of its limitations. Assessment of included studies was unblinded, which may have introduced bias through presumed reputation of some authors. [23] This potential for bias was lessened by two of the four reviewers being outside the field of eye care. Reviewer fatigue may also have been an issue, as data collection took place over a five-month period-we attempted to reduce this by developing assessment guidelines, using two reviewers for each study, and requiring any discrepancy to be discussed and resolved. In addition to English language studies, the search found 12 non-duplicate studies in Chinese and four in Spanish that we were unable to assess. The STROBE statement is available in these languages,[24] and could be applied by researchers fluent in reading them. Finally, we were unable to assess whether STROBE was included in instructions to authors at the time of submission of each individual manuscript. We believe this had little influence on our results-less than one-third of journals referred to STROBE in 2016, and it is unlikely that journals previously included STROBE in their instructions and subsequently removed it.
While not a limitation, to assist interpretation of our results it must be noted that we assessed completeness of reporting and not methodological quality or risk of bias of the included studies. [5] That is, we recorded whether an item was reported in sufficient detail to enable appraisal (e.g. steps taken to reduce bias in cluster selection), not whether the methods employed were sufficient to achieve the specific aim (e.g. avoid selection bias). Further, it is possible some of the checklist items were fulfilled during study design, implementation or analysis, but without complete reporting this remains unknown.
A scan of the literature suggests this is the first analysis of completeness of reporting of blindness prevalence surveys. It has revealed strong reporting in some areas, with other areas in need of improvement. The anticipated increase in surveys in LMICs in the coming years [2] provides the impetus for stakeholders to identify strategies to improve the quality of reporting and therefore increase the utility of survey findings. Supporters of research (international and national non-government organisations as well as academic institutions) could extend support beyond study implementation to also strengthen local research capabilities, including reporting. Endorsement of STROBE by journals should help, but only if its use is consistently enforced by editorial staff and reviewers. We recommend that ophthalmology journals adopt the STROBE checklist and ensure it is used by authors and reviewers.