Model averaging parameter estimates and effect sizes

[kcduerr]

I have a general question about model averaging. I have run simple CJS models and want to report differences in phi between two groups (in my case, site 1 vs. site 2). I model averaged phi over all models (n=11) and can report the effect sizes and SE's. Not a problem. I was always taught that averaging over all models was the "appropriate" way. However, in a recent conversation with a colleague, it was suggested to me that there is debate about whether you should always average over all models, or if it may also be ok to average over a reduced model set that includes only models that contain the variable(s) of interest. In my case, only two models drop out of the set. These two models also happen to be the best supported. When I model average phi using the reduced set, my effect size increases a little, but could have a larger impact biologically. Perhaps I've done a bit of dredging here. So, my question is, should I only report model averaged estimates calculated from the full set, or is it ok to average over the reduced set?

[egc]

I have a general question about model averaging. I have run simple CJS models and want to report differences in phi between two groups (in my case, site 1 vs. site 2). I model averaged phi over all models (n=11) and can report the effect sizes and SE's. Not a problem. I was always taught that averaging over all models was the "appropriate" way. However, in a recent conversation with a colleague, it was suggested to me that there is debate about whether you should always average over all models, or if it may also be ok to average over a reduced model set that includes only models that contain the variable(s) of interest. In my case, only two models drop out of the set. These two models also happen to be the best supported. When I model average phi using the reduced set, my effect size increases a little, but could have a larger impact biologically. Perhaps I've done a bit of dredging here. So, my question is, should I only report model averaged estimates calculated from the full set, or is it ok to average over the reduced set?

Moved this from the MARK subforum, since the question is 'general', and not really software-specific (model averaging is an issue regardless of whether or not you use MARK).

[tjk]

Because model averaging allows you to account for the variation of each model in your model set, it seems like averaging across all of your models would be your best approach, especially since you mention that the two models not included in your reduced model set are the two with the most support. In this example, you would be ignoring the contribution of those two estimates in your model averaging, when those two models seem to provide the best fit to your data. This is generally a good question however, because it seems like there are varied approaches on which set of models should be used when averaging (i.e., average across all models, or only for a reduced model set). It seems to me that this has to be addressed by the researcher on a case by case approach because some analyses may show the opposite of your example, where model averaging across a reduced model set may be the best approach. Probably the most important aspect to all of this is being very transparent when explaining your approach.