Model averaging real parameters

[jCeradini]

Hi folks,

I apologize if these questions don't make sense due to a fundamental (statistical) misunderstanding on my part....

1) I understand that in RMark model averaging is over the real parameters and not the betas (due to additional complications involved with averaging the betas - good discussion here: viewtopic.php?f=1&t=996&p=2620&hilit=model+average+betas#p2620). The help for model.average.list states that RMark uses eqtn 4.1 from B&A. What I cannot tell from the eqtn is how averaging is done when a predictor is not in every model (same as the beta discussion in the above link): are real parameters averaged only over the models that contain a given predictor (the "natural" average) or they are averaged over the entire model set, where a given real parameter is considered zero for a model that does not contain that predictor (the "zero" method)? Or does this question only make sense in the context of model averaging betas?

2) Assume a model set where some models contain interactions and other models contain the same predictors but without an interaction. If this set is model averaged, can one interpret the 'main effects' of the predictors? Grueber et al. 2011 (J. of Evo Bio) discuss the importance of 'centralizing' predictors when model sets have interactions. I understand that MARK standardizes covariates internally but I'm not sure whether that's relevant for this example.

Thanks!
Joe

[cooch]

Hi folks,

I apologize if these questions don't make sense due to a fundamental (statistical) misunderstanding on my part....

1) I understand that in RMark model averaging is over the real parameters and not the betas (due to additional complications involved with averaging the betas - good discussion here: viewtopic.php?f=1&t=996&p=2620&hilit=model+average+betas#p2620). The help for model.average.list states that RMark uses eqtn 4.1 from B&A. What I cannot tell from the eqtn is how averaging is done when a predictor is not in every model (same as the beta discussion in the above link): are real parameters averaged only over the models that contain a given predictor (the "natural" average) or they are averaged over the entire model set, where a given real parameter is considered zero for a model that does not contain that predictor (the "zero" method)? Or does this question only make sense in the context of model averaging betas?

MARK averages reals for a given interval, over all models that are structurally consistent (i.e., same underlying data type and likelihood). Where or not the model contains a particular covariate is irrelevant -- all models (with or without a covariate) generate interval-specific estimates for a given parameter. Model averaging model averages those interval-specific estimates.

You questions about what to do with models that 'don't contain a particular covariate' applies to model-averaging over betas, which is one of several reasons why model averaging over betas is discouraged (since it isn't always clear how to appropriately do so).

2) Assume a model set where some models contain interactions and other models contain the same predictors but without an interaction. If this set is model averaged, can one interpret the 'main effects' of the predictors? Grueber et al. 2011 (J. of Evo Bio) discuss the importance of 'centralizing' predictors when model sets have interactions. I understand that MARK standardizes covariates internally but I'm not sure whether that's relevant for this example.

Thanks!
Joe

if your question is 'what is the relative importance of a predictor', then your best bet is cumulative AIC weights. For simple main effects, and a symmetrical candidate model set, this works quite well. But, jury is out on what to do with interaction terms -- even more complicated if the interactions are among linear covariates.

[jCeradini]

Thanks so much for the quick response.

MARK averages reals for a given interval, over all models that are structurally consistent (i.e., same underlying data type and likelihood). Where or not the model contains a particular covariate is irrelevant -- all models (with or without a covariate) generate interval-specific estimates for a given parameter. Model averaging model averages those interval-specific estimates

So, just to make sure I'm following: I'm predicting values for apparent survival across the range of a covariate (while holding other covariates at their mean), model averaged over the whole set (as when you give covariate.predictions a marklist). Models that do not contain the covariate of interest will still contribute a weighted survival estimate to the model average, which will represent the survival estimate when the covariate of interest is zero?

Thanks again.
Joe

[cooch]

Thanks so much for the quick response.

MARK averages reals for a given interval, over all models that are structurally consistent (i.e., same underlying data type and likelihood). Where or not the model contains a particular covariate is irrelevant -- all models (with or without a covariate) generate interval-specific estimates for a given parameter. Model averaging model averages those interval-specific estimates

So, just to make sure I'm following: I'm predicting values for apparent survival across the range of a covariate (while holding other covariates at their mean), model averaged over the whole set (as when you give covariate.predictions a marklist). Models that do not contain the covariate of interest will still contribute a weighted survival estimate to the model average, which will represent the survival estimate when the covariate of interest is zero?

Thanks again.
Joe

I'll let Jeff answer the question was to what covariate.predictions does, or doesn't do.

More broadly, I'm generally not a big fan of trying to interpret model averages from models with a slew of (what appears to be) individual covariates, since interpretation depends an awful lot on how you code things (standardized, or otherwise), and any number of other things. It isn't even clear to me how you interpret generated averages -- 'average survival over interval t -> t+1 for an individual of a given covariate value, given all the other covariates are 'controlled' for in some fashion'. What most folks really want is the average functional response between a parameter and some covariate. Which, of course, brings us back to model averaging betas, which isn't something easily done in a general, defensible way.

[jCeradini]

Thanks for elaborating.

For the sake of clarification, as one example, I'm predicting apparent survival across a range of shrub cover estimates while holding other veg metrics (like grass cover, etc) at their mean. For individual covariates, I make predictions for adults and non-adults separately and males and females separately. I agree, I would love to have a model averaged beta estimate for shrub cover, but I'm not gunna go there.

Hopefully Jeff can help me better understand whats happening when covariate.predictions is given a marklist, although I imagine it's something you've already covered in this thread.

Thanks,
Joe

[jlaake]

I was in the field without email.

Let me give an example. Let's say you have a single Phi and two models. Model 1 has a covariate and Model 2 is Phi(.). For every value of the covariate, Model 1 will return the real value survival prediction of Phi at the covariate value and Model 2 will return the constant phi survival value. Those are model averaged and that is done for each of the covariate values that you specify. It's that simple.

--jeff

[jCeradini]

Thanks Jeff. That's perfectly clear!

One follow-up question, at the risk of beating a dead horse:

What about model averaging with interactions? Evan already mentioned that the jury is still out on this, but I may have asked the question poorly before. If you model average a set where model 1 has covariates x1 * x2 and model 2 has x1 + x3, is the model averaged estimate for x1 interpretable? Isn't the x1 estimate from model 1 (where x1 is in an interaction) contributing to the model averaged estimate?

Thanks again,
Joe

[jlaake]

I don't see any reason why that would be a problem. It is no different than model averaging x1*x2 and x1+x2. If the models make sense I can't think of any reason why model averaging the real parameters wouldn't make sense. The betas are a different story especially if the covariates are factor variables.

--jeff

[jCeradini]

Got it - thanks Jeff.

As with most of this thread, the key for me was realizing that real parameters can be dealt with much differently than betas....which is obviously why they're used instead of betas for model averaging.

Joe

[cooch]

A useful addition to this thread -- see viewtopic.php?f=2&t=3070