Post a new comment on this article
Post Your Discussion Comment
Please follow our guidelines for comments and review our competing interests policy. Comments that do not conform to our guidelines will be promptly removed and the user account disabled. The following must be avoided:
- Remarks that could be interpreted as allegations of misconduct
- Unsupported assertions or statements
- Inflammatory or insulting language
Why should this posting be reviewed?
See also Guidelines for Comments and Corrections.
Thank you for taking the time to flag this posting; we review flagged postings on a regular basis.close
1st review & comments on auotocorelation & impossible result of no effect of # wolves on wolf depredations ignored by authors
Posted by wielgus on 17 Feb 2016 at 17:38 GMT
Review of PONE-D-15-15642 “Wolf lethal control and livestock depredations: counter-evidence from correctly specified models” by Niraj Poudyal, Nabin Baral, and Stanley T. Asah.
This paper shows what can go wrong when non-population biologists (1 economist & 2 sociologists) weigh in on wildlife population ecology and demography.
The authors attempt to refute the findings of my paper Wielgus and Peebles (2014), whereby numbers of wolves, numbers of breeding pairs, and numbers of wolves killed at time t were all found to be associated with increased livestock depredations at time t+1. The authors claim that the original models used by Wielgus and Peebles (2014) were mis-specified (using only demographic variables) and should have used a time index variable (trend), as they did, because livestock depredations were increasing over time as the wolf populations increased.
In their re-analysis, they showed that time accounted for almost all of the variability (coefficient for time = 0.16, P<0.001) vs. number of wolf breeding pairs (-0.02, P = 0.21), and number of wolves killed (-0.01, P< 0.01), see Table 1. The number of breeding pairs was not even a significant variable for livestock depredations! The number of wolves killed was counter-indicated.
The problem: Using time as their major explanatory variable in this context is exactly similar to using time to explain 10 year snowshoe hare cycles. Time explains everything…but explains nothing. The question is not “did livestock depredations increase over time” (everyone already knows that!). The question is how did the number of wolves, number of breeding pairs, and most importantly, the number of wolves killed contribute (or not) to the increased livestock depredations over time?
In this case, time is a completely synthetic or “garbage can” variable – which already includes the combined demographic effects of number of wolves each year, numbers of wolves killed each year, number of breeding pairs each year, and resulting wolf population growth rate each year. The authors themselves state “The statistically significant positive coefficient of the trend variable (time index) indicated that there were unobserved factors that led to the increase in the number of cattle killed” (italics mine).
Unobserved? The whole point of our paper was to disentangle the important population demography parameters from the effects of time – to arrive at plausible, biological causative mechanisms of increased depredations. We did not report on the statistical effects of unobserved factors subsumed in time (as did these authors).
As we stated on page 12 of our paper “The secondary effects of time…..were already subsumed in the primary effect terms of breeding pairs and wolves, so those secondary effects cannot account for the positive effects of wolf kills on depredations”. Time explains everything…but nothing. That is why we, and other population biologists, do not use time as an explanatory variable in investigations of population dynamics (Sinclair, Fryxell and Caugley2006). Of course livestock depredations increased over time as wolves increased! This is circular reasoning at its worst. We wanted to know what the effects of wolf kills were on subsequent livestock depredations, not simply point out that depredations increased over time, as the wolf populations increased.
We showed that increased wolf kills were associated with increased breeding pairs (increased growth rate), and that increased breeding pairs were associated with increased livestock depredations - until the wolf kill rate exceeded the intrinsic growth rate of wolves, at which time livestock depredations declined. By contrast, these authors had no putative causal chain, other than unobserved factors subsumed in time. The factors were observed in our paper. The authors of this paper chose to ignore them, use an all-encompassing time variable instead, and use the real demographic variables of interest simply as additional smoothing variables in their synthetic model.
Because of this massive, fatal error (using time as the primary predictor), I will not comment any further on all the other problems evident in this paper. I will leave that to my data analyst and consulting statistician (if they so desire).
Sinclair, A.R.E., Fryxell, J.M., and Caughley, G. 2006. Wildlife ecology , conservation, and management. Second Edition. Blackwell Publishing.
Dr. Robert Wielgus, Director
Large Carnivore Conservation Lab
Washington State University
RE: 1st review & comments on auotocorelation & impossible result of no effect of # wolves on wolf depredations ignored by authors
Authors’ Response: We felt that the above statement is an attempt to discredit the source of information without first addressing whether the information is reliable or not. Information should be evaluated on its own merit. The criticism of the authors has nothing to do with the rigor of the analysis conducted and reported in the manuscript. Irrespective of users, statistics has certain rules that are applicable to all. Although biostatistics might appear different from econometrics in the application, but they are similar in the adoption of statistical principles. Claiming that only biologists are the experts on statistical modeling of the phenomenon rather than refuting the results on statistical grounds sounds like an ‘ad hominem’ argument to us. While modeling observational data, statistical adequacy often precedes the substantive explanations. The rebuttal is all about what can go wrong when the assumptions of statistics are not satisfied, but not about the ecology or behavior of wolves in particular. This argument is consistent with Reviewer 1’s claims of ecological understanding that is needed to correctly specify time series models. It does not suffice to say that ecological reasoning informs model specification. What is that reasoning? Statistical rigor is not something that “population biologists” should be immune to.
Authors’ Response: We greatly appreciate your effort in parsing out the effects of the demographic variables of wolves on livestock depredation. It is important to remember that had there been only three variables that explained the phenomenon of livestock depredation, theoretically the time index would have been insignificant statistically. Because this was not the case, there could be potentially other factors that influence livestock depredation, but there is no way to rule out them in an observational study like this. When the results do not hold while respecifying the models, we suspect that there are other variables influencing the phenomenon. For example, food availability to wolves is one critical variable that influences livestock depredation, but this is not controlled for in the models. Using the time index as a control variable helps to parse out the effect of demographic variables while controlling for other unobserved but relevant variables that might influence the phenomenon. This is the gist of our argument in modeling the data in this case.
Authors’ Response: The above statement appears to conflate the effect size with the variance explained by the model. Because of the different units of measurement of the predictor variables, it is difficult to make a direct comparison of the regression coefficients. For example, there was a 16% increase in cattle depredation on average from one year to next. But one wolf killed reduces the number of cattle depredated by 1%. Thus the effect of one wolf and one year are not directly comparable. This should be obvious whether one is a population biologist or not.
Nevertheless, the coefficients of the explanatory variables in the original and respecified models do not differ much in the practical sense. This observation also refutes that the time index is capturing all the variation in the data. The small magnitude of the coefficients of the explanatory variables suggests that there might be other relevant factors that are missing from the models.
Authors’ Response: Nowhere in the rebuttal have we mentioned that time explains anything. We used the time index as a control variable. We agree with your research question and we feel that the research question is very intriguing. The only problem is that this is an observational study and claiming that only the above mentioned variables explain the phenomenon without exploring the data and testing the assumptions is untenable. We assume that the author is alluding to a paper by Krebs et al. 2001 published in BioScience for the treatment of time variable and using it to justify their modeling approach. The two cases are quite different and not comparable at all. It is worth noting that this is the second time we have to make obvious that we should not be comparing that which are non-comparable. The authors (Wielgus and Peebles) did not design any field experiment to answer their research question. In the Krebs et al. paper, they come up with some hypotheses and tested them using the data collected through a field study design regarding what could explain the observed cycles in snowshoe hare populations. Even in their case, the authors are very humble in reporting the limitations for establishing causality. The wolf data are not collected from any sort of experiment; they are observational data. The essence of the argument is that causality is better inferred from experimental designs rather than statistical procedures.
Krebs, C. J., R. Boonstra, S.Boutin and A. R. E. Sinclair 2001. What drives the 10-year cycle of snowshoe hares? BioScience 51(1): 25-34.
Authors’ Response: We greatly appreciate your efforts in disentangling the population demography parameters from the effects of time to propose plausible ecological explanation of the phenomenon. As you mentioned, had the time included the combined demographic effects of number of wolves each year, number of wolves killed each year, number of breeding pairs each year and resulting wolf population growth each year, the time index variable would have been statistically insignificant in the models. That assumption could have been easily tested and we simply took the pain to test the assumption and found that it did not hold. On a substantial ground, livestock depredation is not only associated with the demographics of wolves, but also with ranchers’ behaviors and operational contexts. For example, the effort of ranchers to keep their livestock safe is one critical social variable which is absent in the models. We are calling such relevant variables as “unobserved” in the models.
Authors’ Response: As copied in the above comment, in the original paper, it was assumed that the “secondary effects of time … were already subsumed in the primary main effect …” but this assumption was not tested. Our replication shows that this assumption does not hold for the data. The effect captured by the time index was not subsumed in the primary main effect terms of breeding pairs and wolves. We have not ignored any of the demographic variables that were included in the original paper. How do we know that the demographic variables are the only variables that influence livestock depredation in an observational study? Where in the Sinclair et al.’s 2006 book is a reference for not using the time as an index in statistical models? Had the author provided more specific reference (e.g., page number), it would have been easier to address that comment more specifically. In our view, Morris and Doak’s 2002 book provides an excellent reference regarding how to deal with time, temporal environmental trends, and environmental autocorrelation while modeling population dynamics (Chapter 2, pages 18-20, 93-94 and 133-135 are particularly helpful).
We wonder how the authors could completely negate circular reasoning in a correlational design (or observational study). We would like to reiterate here too that we have not made any causal claims in the rebuttal. We would like to share with the author one plausible explanation for wolf population growth. When more livestock is depredated there is more food for the wolves which in turn increases reproductive potential of wolves leading to their population increase. When the population of wolves increases, there might be more instances of livestock depredation. In this case, there is a positive feedback, one event causes another. We don’t think that we can establish an asymmetrical relationship in this study. We feel that logic should be the cornerstone of every scientific discipline.
Morris, W. F. and Doak, D. F. 2002. Quantitative conservation biology: theory and practice of population viability analysis. Sinauer Associates, Inc. Publishers, Massachusetts.
Based on the information provided in the original paper, it is difficult to replicate Figure 3 exactly. We assumed that the authors added the numbers from three states for the figures (1-6). In our speculation, there could be four possible versions for Figure 3, but none of them quite resembles to Figure 3. When the outlier is ignored there is a linear relationship between the wolf kill rate and the number of livestock depredated, when accounting for the trend. We reproduced the figures for your reference.
Fitting a quadratic line in the presence of an obvious outlier does not resonate well with statistical reasoning. There is a trend in the data and when it is accounted for, the figures look quite different, but there is still positive association between the two variables. We did not reproduce other figures because this one is enough to refute the author’s claim about a curvilinear relationship.
Authors’ Response: We respectfully disagree with your conclusive statement regarding the “fatal error”. Using the time index in time-series data like this is not a fatal error, but omitting the time index is. The contrast between the original and replicated results speaks for itself. The inclusion of statistically insignificant variables in the best models built by the forward selection method reported in the original paper gives enough information to suspect that those models were not reliable. We would like to reiterate that statistical evidence is generally circumstantial evidence when it is related to causality in observational studies, so it must be used carefully. Its persuasiveness depends on the likelihood of other causes being responsible for the phenomenon at issue.
We understand your feelings and wish that the original findings were robust enough to be acceptable. We agree with you that time itself does not explain anything, but the index of time is required for modeling some time-series data, which is the point in this case.