V-C matrix when model averaging

[achatinella]

Hi All,

I’m using a Pradel model to estimate survival and recruitment for two groups. Four of my candidate models had equal support in the data, so I am using model averaging across those four models. This is straightforward using the equations in Burnham and Anderson (2002 p. 152). But I also need to use the Delta Method to estimate standard errors, because each real parameter is a function of > 1 beta parameter. My problem is analogous to worked example (4) in Appendix B “variance and SE of back-transformed estimates – harder.” I am not sure what to use for the variance-covariance matrix, since the beta estimates I use in the back transformation are model-averaged. Can I simply model average the elements from the 4 V-C matrices to generate a model-averaged V-C matrix? Seems too easy.

Thanks,

Jim

[jlaake]

Why not model average the real parameters and I believe MARK will construct the v-c matrix of the model averaged real parameters for you. Sorry if I'm misunderstanding your post. B&A explain how to compute a model-averaged v-c matrix which actually averages the correlation matrix and then scales to a v-c with the std errors.

--jeff

[achatinella]

Jeff,

You're right about the V-C matrix from MARK when model averaging real parameters. Unfortunatley, I cannot very easily use the model averaged real parameter estimates from MARK. This is because my model includes a covariate, like example 4 in Appendix B of the MARK book. So I have a separate estimate of survival for each individual. Thus, I am using model averaged beta values to back transform from the logit scale to the 0 - 1 probability scale. I can calculate standard errors for these model avergaed betas using the delta method, but to do so requires a V-C matrix, and I'm not sure how to get a V-C matrix for model averaged betas.

Do you have a reference or page number for the B&A explanation of a model-avergaed V-C matrix?

I guess one solution would be to use the "user specified covariate value" option. I could do this for a range of covariate values. Maybe that would be the easiest approach.

Thanks,

Jim

[jlaake]

Got it. I didn't understand that you were using covariates from your first post. I'd go with the simple solution of user-defined covariates. Unfortunately the way it is implemented means MARK needs to be re-run each time which is unforunate. That is one of the reasons I created the RMark code. Once the model output is in R, any number of predictions can be computed without re-running the optimization.

--jeff

[cooch]

Got it. I didn't understand that you were using covariates from your first post. I'd go with the simple solution of user-defined covariates. Unfortunately the way it is implemented means MARK needs to be re-run each time which is unforunate. That is one of the reasons I created the RMark code. Once the model output is in R, any number of predictions can be computed without re-running the optimization.

--jeff

As noted in the appendix, if (i) you can take the partials of the model function (since the terms are typically all scalar products or something equally trivial, this should be easy - if it isn't, you probably shouldn't be using MARK , and (ii) have the V-C matrix (which MARK gives you), then creating a program in almost any scripting environment ([R], SAS, Excel, pretty well everything) to generate values and CI's over some range of the covariate is trivial - for a given model. Realy, with even rudimentary programming skills, this is about a 2 minute task.

But, what is less trivial is how to model average over models with different invididual covariates (or, even among models with different functions of the same covariates). There are, in fact, differences between B&A first and second editions on this point. I'm trying to come up with some canonical text which will clarify.

[bmitchel]

Jim -

I put together an Excel spreadsheet that can be used to produce model-averaged results for different covariate values; it is at http://www.uvm.edu/~bmitchel/software.html. Took me a bit longer than 2 minutes to write (so I suppose I'm not yet a rudimentary programmer ;-), and it may save you some time. The spreadsheet is designed to be general (i.e., not specific to MARK), so you will need to export model-specific VC matrices, parameter estimates, etc. If you are only working with a few scenarios (e.g. a couple of different covariate values) it will probably be faster to stick with user-defined covariates in MARK. As Evan mentions, there are some differences in the formulas and methods in the different B&A publications; the spreadsheet includes both variance formulas and assumes that variance = 0 when a parameter is not in the model (per personal communication with David Anderson). The spreadsheet will generate a model-averaged VC matrix, per B&A 2004, but I have seen this formula produce negative variances, so be aware that it might not be a good approach.

Brian

[achatinella]

Thanks to all for your thoughts and comments. Estimating the model averaged standard errors is easy enough. I wrote a simple SAS program that does this (but I confess it also took me me than 2 minutes : ). What I needed to know what how to compute a model averaged VC matrix to use as input for calculating standard errors. My models do have different covariates, and different functions of covariates, so I can see now that model averaging will not be as straightforward as I initially thought.

Probably what I will do instead is to interpret the top few models separately without model averaging. This way I can base my inference on > 1 model, recognizing the uncertainty in model selection, while still considering the generalities / distinctions that emerge from the set of well-supported models.

Jim