In the following paragraphs, we (Drs. E. Laurin, and I. Gardner from Atlantic Veterinary College, Prince Edward Island; and Dr. J. Christensen from Epidemiologic Surveillance and Analysis Consulting, Prince Edward Island) describe our concerns about the conclusions and interpretations of data in Morton et al. (2017) on how piscine orthoreovirus (PRV) infection of wild Pacific salmon may be associated with exposure to farmed Atlantic salmon and with fitness of Pacific salmon to complete particular migration routes in British Columbia (BC), Canada. In their manuscript, the authors use the disease name “heart and skeletal muscle inflammation (HSMI)” synonymously with PRV, even though HSMI is a histopathologically-identified condition and an infrequent cause of disease in farmed Atlantic salmon (Laurin et al., 2019).

We believe that the Morton et al. (2017) manuscript fails to adhere to reporting standards for observational studies (e.g. cross-sectional studies, as in the health sciences as recommended by PloS ONE (http://journals.plos.org/...)). The relevant reporting standard is named STROBE (STRengthening the reporting of OBservational studies in Epidemiology statement; Elm et al., 2007). There is a veterinary sciences adaption of STROBE, named STROBE-Vet (strobevet-statement.org), which is endorsed by various veterinary peer-reviewed journals.

The purpose of reporting standards for observational studies is to allow unbiased presentation of results, less biased inferences to target populations, candid discussion of a study’s strengths and weaknesses, as well as to improve comparability with other related peer-reviewed literature to improve the quality of future systematic reviews and meta-analyses. Transparent, accurate, and complete reporting is underpinned by an appropriate and valid study design to support the research questions being addressed (Elm et al., 2007; Sauerbrei et al., 2014). Further initiatives are underway to address the quality of design and statistical analysis of observational studies (Sauerbrei et al. 2014), but we did not apply them to this paper, as they are still under development.

We applied the STROBE-Vet checklist of 22 items (47 points) to the Morton et al. (2017) manuscript, and identified areas that may have led to biased results and misleading conclusions. The STROBE-Vet checklist was completed blindly by three of us, and results (yes, no, partial, or not applicable for each point) were sent independently to a fourth person for compilation. Discrepant assessments were re-reviewed by two of us, and a consensus report was used to summarize findings (Table 1).

In brief, Morton et al. (2017) reported an observational study “to assess if PRV infection is epidemiologically linked between wild and farmed salmon in the eastern Pacific” and evaluated the effect (better termed “statistical associations”) with (1) high or low exposure to Atlantic salmon farms tested to be PRV infected and (2) the possible effects of challenge via more arduous return migration routes (high elevation regions). Their sampling scheme involved obtaining fresh (using best-before date) farmed salmon (262 Atlantic salmon and 35 Steelhead salmon) bought in 10 southwestern BC markets over 93 different dates. They reported that fishmongers confirmed that the salmon were from BC net-pens, but exact net-pen identification was not available. Regions for salmon bought at these markets corresponded to areas where Pacific salmon followed high or low exposure to Atlantic salmon farms and farmed Atlantic salmon that were tested for PRV. Then 601 wild Pacific salmon were sampled from marine and freshwater sites in southern BC (9 distinct regions classified based on either low or high exposure to Atlantic salmon farms and areas where they overcome significant migration challenges) and an additional 402 from another site, two years following the first study period. The latter were analysed separately as were data from the 2 study years.

Four tissue samples (gill, heart, head kidney, and spleen) were collected from wild Pacific salmon. In the methods, due to sampling limitations, they report that their study was exploratory in nature (“Our study, aimed at exploring potential geographic patterns and generating epidemiological evidence providing provisional support for key hypotheses…”). Statistical methods involved cluster analysis, logistic regression (stepwise model selection), and comparative analysis using likelihood-based inference. They stated that their results “suggest that PRV transfer is occurring from farmed Atlantic salmon to wild Pacific salmon, that infection in farmed salmon may be influencing infection rates in wild salmon, and that this may pose a risk of reduced fitness in wild salmon impacting their survival and reproduction.” In brief, they concluded that exposure to farmed Atlantic salmon was associated with PRV infection and that PRV-infected wild Pacific salmon may be less likely to finish a more arduous spawning migration, even though a longitudinal design and appropriate comparison groups (e.g. PRV-negative fish) were not used for inferences.

In their methods, the authors state: “Because so little is known about the potential epidemiological interactions between farmed and wild salmon in the North Pacific, an exploratory approach to our analyses was used. Thus, in keeping with the spirit of exploratory data analysis, we adopted a flexible approach to the selection of statistical methods and models, and put forward our conclusions as hypotheses worthy of further attention.” However, in their discussion, many conclusions stated as evidence are presented, without transparent discussion of limitations and biases within their study design and methods and subsequent interpretations and extrapolations to the greater fish populations. Such limitations and biases become evident after benchmarking the publication against reporting standards for observational studies. Based on our consensus results of the STROBE-Vet checklist (Table 1, posted in a separate comment), the study was deficient completely on 10 full and 2 half points (out of 47 individual points). An additional 13 full and 1 half point were followed only partially. Only 9 full and 2 half points were fulfilled. The majority of the shortcomings involved reporting of design and methods (e.g. sampling, statistical methods, data quality, limitations of the study, and compliance with standards).

Examples of where the study design and methods used in the study were weak and thereby likely to introduce bias in the results and interpretation include: (1) sample source was unknown for Atlantic salmon in the study and should have been verified with records from stores, as cross-contamination is likely in markets; (2) use of mixed ages and mixed species, which may be important confounders or effect modifiers, and lack of data on age and size of Atlantic salmon; (3) no explanation was provided for possible roles of non-salmonid species in transfer of PRV; and (4) there was no accounting of time-in-water for Atlantic salmon, as this can affect exposure level and disease status to infectious agents, particularly those with chronic and recovery stages.

Potential confounders were inherent in their design and sampling, but were not examined, and statistical methods chosen in that regard were not always well justified. In addition, assumptions were made about migration success that were unlikely to be biologically valid. For example, different species and ages of salmon were assumed to have the same susceptibility to infection with PRV and were combined in the data analysis. Microbes other than PRV (e.g. infectious hematopoietic necrosis virus) may be contributing to migration failure, but the assumption was made that all losses were accounted for by PRV exposure, and this is likely incorrect.

In addition, the conclusion was weak (“cannot be definitive”) and the interpretation of the results of this study to the target wild salmon population was biased. Morton et al. (2017) state that the primary plan or purpose of the study was to find an epidemiological link between farmed and wild salmon and to assess the impact of PRV on the wild salmon; but then, in their methods, they strongly point out that their study is very much exploratory in nature. In contrast, in the discussion, they interpret the results as evidence, including provisional and precautionary evidence, rather than as hypotheses that require further investigation

We recommend that stronger statements be made by PloS ONE about adherence to reporting standards and that manuscript submissions should be supported by the completed checklist of STROBE or STROBE-Vet items with page numbers to facilitate the review process. Strengthening the quality of primary studies of health outcomes in wild fish will help inform risk assessments and policy decisions.

References:
1. Elm E von, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. STROBE initiative. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Epidemiology (Cambridge, Mass.) 2007; 18(6):800–804.
2. Laurin E, Jaramillo D, Vanderstichel R, Ferguson H , Kaukinen KH, Schulze AD, Keith I, Gardner IA, Miller KM. 2019. Histopathological and novel high-throughput molecular monitoring data from farmed salmon (Salmo salar and Oncorhynchus spp.) in British Columbia, Canada, from 2011-2013, Aquaculture 499:220-234.
3. Morton A, Routledge R, Hrushowy S, Kibenge M, Kibenge F. 2017. The effect of exposure to farmed salmon on piscine orthoreovirus infection and fitness in wild Pacific salmon in British Columbia, Canada. PLoS ONE 12(12):e0188793 https://doi.org/10.1371/j...