Hi Phidot,
We got an interesting comment in a review on a recent manuscript. Could someone could check my response? The Reviewer is clearly knowledgeable, but the approach has thrown me somewhat and I’d appreciate advice.
Background: We’ve been using spatial variants of the CJS live-recaptures model to estimate survival and detection efficiency for migratory salmon. In this particular study, we released 151 fish to swim over 4 different detection sites. The fish were ~evenly divided into two groups by the type of acoustic tag they were implanted with (V7=larger tag; V4=smaller tag), and each of the tag type groups were evenly divided into two groups depending on if they had a small gill biopsy collect at tagging (biopsy=no/yes). We want to know if tag type (i.e. tag size/tag burden) or gill biopsy affected fish survival.
For the manuscript, we came up with a series of 20 candidate models to address this question where survival was modelled as a function of time (i.e. migration segment, and/or the type of tag and the presence/absence of gill biopsy. Detection probability was modelled the same way in all models as time*tag_type because it was not likely that gill biopsy would affect detection probability. I used AIC to rank the models and then model-averaged.
Reviewer’s comment: (The Reviewer refers to “Model 16” which is the fully-varying model Phi(time*tag_type*biopsy)p(time*tag_type).)
“The value of the study is in the comparison of treatment effects at the three points in the migration. As currently presented, with 20 hypotheses with combinations of treatments in different segments, it detracts from the most relevant information which is the comparison of treatments at different points in the migration. Model selection for parsimony and model averaging would be appropriate if the purpose was to estimate survival through the migratory stage controlling for experimental factors (tag type, biopsies) that can affect survival of individual fish. But that is not the purpose of this experiment, to obtain the most parsimonious model. Rather, the purpose is to determine if the treatments result in statistically significant differences in inferred survivals at three points in time. The analysis is already done, you simply present the parameter estimates from model 16.
Model 16 from table 1 is much clearer to me if you write out the equation for the two-factor design of the experiment. I assume MARK or the R package uses the logit transformation for both detection and survival.
In each of segment j, tag type V4 no biopsy (mu), tag type V4 with biopsy (gamma), tag type V7 without biopsy (alpha) and tag type v7 with biopsy (delta)
logit(∅_i,j)= mu_j + alpha_j *T1_i + gamma_j * T2_i + (delta_j * T3_i ) (model 16)
With i fish id subscript
j subscript for the segment of river for which survival is inferred from detections
mu the survival rate of fish with V4 tags and no biopsy (in this structure, it is the reference condition)
alpha is the incremental change in survival of fish with V7 tag and no gill biopsy relative to the reference state
gamma is the incremental change in survival of fish with V4 tag and a gill biopsy relative to the reference state
delta is the incremental change in survival of fish with V7 tag and a gill biopsy relative to the reference state.
T1_i to T3_i are fish specific binary variables identifying treatments T1 = V7 no biopsy, T2 = V4 with biopsy, and T3 = V7 with biopsy
So treatment effects, relative to V4 as the reference, are directly inferred from the CJS model estimates of the parameters.”
Suggested Response: The Reviewer is correct that our main goal for this paper is the comparison of treatments at three points in the migration rather than estimating survival, and has suggested an interesting approach. However, the multi-model comparison approach we used is appropriate for our study for a number of reasons. First, we are dealing with a limited sample size. The Reviewer suggests that we base our inferences on the most highly-parameterized model which means that the errors on the beta parameter estimates are larger than they would be for less-parameterized candidate models and reduces our ability to detect effects (e.g. for just tag type, just biopsy, or for additive effects only). Later in the review, the Reviewer acknowledges that we may have sample size issues. Secondly, the reviewer is advocating that we base statistical significance on the size of the 95% beta parameter estimates which is a traditional significance test. In contrast, we would prefer to continue with the evidence-based approach not based on a specific significance level. Finally, although the survival estimates are of reduced importance in our study, we do wish to present the best possible estimates given the data.
The Reviewer’s suggested parameterization for our fully-varying model (Model 16) is interesting and we were able to reproduce it in program Mark; however, we’d like to clarify that we did not have the model ‘already done’ using this method. The RMark interace to Program Mark (which we used for this analysis) has a different approach. While Program Mark has a flexible design matix that allows specification of the beta parameters in different ways, the method proposed by the Reviewer is advanced.
We would also like to clarify that the Reviewer is talking about comparing the beta parameters to identify statistical significance rather than the real survival estimates. There can be multiple beta parameters combined to estimate each real parameter and each beta has an associated error. In our case, the real parameters contain error for the estimation of survival in each segment (time) in addition to the variables that we are interested in testing (tag_type and biopsy).
Further questions: Is the reviewer’s approach preferable in situations where there is a large sample size? What does the reviewer mean that our goal is not to obtain the most parsimonious model? I know what a parsimonious model is, but not why one would prefer to use a more parameterized model.
Thank you!!
Aswea