Trouble interpreting interaction model/"problem model"

Forum for discussion of general questions related to study design and/or analysis of existing data - software neutral.

Trouble interpreting interaction model/"problem model"

Postby heg90 » Sun May 26, 2019 7:15 pm

Hi,

I'm running a single-season occupancy model, and my top two models are 1) psi(elevation) and 2) psi(elevation*SC5), where SC5 = % of size class 5 trees within the sample unit. Although the interaction model appears to be significant, I'm questioning whether something strange is occurring with the interaction model that might be pulling it to the top of the set. Both of the models share nearly identical delta AIC (0.00 vs 0.01), AIC weights, and deviance, which seems strange as a novice to occupancy modeling (please correct me if that's not odd). I've double checked my design matrix and nothing appears to be wrong in the matrix that would make them almost identical (i.e. one beta for elevation, one for SC5, and one for product(elev,SC5)).

The beta estimates for elevation seem reasonable for both models, but the odds ratio derived from the beta estimate for SC5 in this interaction model seems unreasonable (i.e. SC5 beta = 0.164, OR = 1.178, and with a 1 unit change reflecting a 1% change in SC5 per sample unit this would translate to: for every 20% increase in SC5 per sample unit, probability of occupancy increases by 356%). In the additive model psi(elev + SC5) and the univariate model psi(SC5) the odds ratios are reasonable, so I'm wondering if there is something going on with the interaction model that is pulling it to the top. I've tried looking into 'pretending variables' or reading through the MARK book to find any information on why this might be occurring, so any insight or direction in where to look for help/interpretation would be greatly appreciated.

I noticed in MARK Ch. 4 - 51 it talks about 'problem models' when model averaging parameter estimates. It states that "if the 'problem models' have appreciable support in the data, you'll need to be more careful [when considering dropping models for estimating parameters]. You might choose simply to average only those 'well-estimated' parameters, but you need to first confirm that those models aren't well-supported because of poorly estimated parameters." This is what I'm confused by. I'm not sure 1) how to determine if a model is a 'problem model'/what is considered a 'problem model', and 2) how to determine if my 2nd best model is well-supported simply because of poorly estimate parameters.

Thank you!

~Holly
Last edited by heg90 on Sun May 26, 2019 7:36 pm, edited 1 time in total.
heg90
 
Posts: 20
Joined: Sun Nov 04, 2018 4:52 pm

Re: Trouble interpreting interaction model

Postby darryl » Sun May 26, 2019 7:33 pm

Hi Holly
You have to interpret beta estimates carefully when you have interactions between variables in your model. In you case with the psi(ele*SC5) model, the beta parameter for the SC5 main effect indicates how occupancy changes with respect to SC5 in the specific case when ele=0 (as entered in your data file; as the interaction term will be 0 in that case, so have no effect). The interaction beta then indicates how the effect of SC5 as elevation changes (and vice versa).

I posted a video on interpreting interactions with continuous covariates a few months ago that you might find useful (https://www.proteus.co.nz/news-tips-and ... ovariates/). Apologies to everyone for the shameless self promotion ;-)

Cheers
Darryl
darryl
 
Posts: 495
Joined: Thu Jun 12, 2003 3:04 pm
Location: Dunedin, New Zealand

Re: Trouble interpreting interaction model/"problem model"

Postby heg90 » Sun May 26, 2019 7:38 pm

Beautiful! A video for how to interpret this was exactly what I was looking for. Thank you so much! I may be back with more questions to follow.
heg90
 
Posts: 20
Joined: Sun Nov 04, 2018 4:52 pm

Re: Trouble interpreting interaction model/"problem model"

Postby heg90 » Sun May 26, 2019 8:55 pm

Hi,

I spent some time reviewing some of your videos, in particular the interpretation of OR, and I had a few questions relating to interpreting the beta estimates for the interaction model. You had mentioned that "the beta parameter for the SC5 main effect indicates how occupancy changes with respect to SC5 in the specific case when ele=0."

Before running models in MARK I scaled elevation, so elev = 0 reflects the mean elevation (I'm not sure if that would influence the case of elev = 0, as 0 reflects a mean of ~700m). SC5% ranges from ~0-40% between my data points. If my beta estimate for the SC5 parameter is 0.1641, and this reflects the influence of SC5 when elev = 0 and the interaction effect is 0, wouldn't that still translate to a 1% increase in SC5% has a 17.8% increase on probability of occupancy (e^0.1641 = 1.178), or similarly a 20% increase has a 356% increase on psi? That seems off, which led me to believe that either I'm interpreting this wrong or perhaps there is something wrong with the estimates this model is producing. Also, it is strange that the top two models would have nearly identical delta AIC, AIC weights, and deviance?
heg90
 
Posts: 20
Joined: Sun Nov 04, 2018 4:52 pm

Re: Trouble interpreting interaction model/"problem model"

Postby darryl » Sun May 26, 2019 11:33 pm

Before running models in MARK I scaled elevation, so elev = 0 reflects the mean elevation (I'm not sure if that would influence the case of elev = 0, as 0 reflects a mean of ~700m).


So, interpretation of the SC5 main effect, is the effect on occupancy at 700m.

wouldn't that still translate to a 1% increase in SC5% has a 17.8% increase on probability of occupancy (e^0.1641 = 1.178), or similarly a 20% increase has a 356% increase on psi?


Increase not on the probability of occupancy, but the odds of occupancy (ie for every 1% increase the odds of occupancy are 1.178 times greater; assuming SC5 values entered in data so 1% = 1). Not sure how you get 356%; calculation should be e^(0.1641*20) = 26.63. Again this is effect on odds, not probability, so you can have numbers greater than 100% (in case that was what was concerning you).

Also, it is strange that the top two models would have nearly identical delta AIC, AIC weights, and deviance?


Not necessarily, but you need to check that all the results make sense. In particular, check that the deviances get smaller as you include the additive effect, then the interaction. If they don't, the optimisation routine may not have converged properly (ie to the global maximum for the likelihood).
darryl
 
Posts: 495
Joined: Thu Jun 12, 2003 3:04 pm
Location: Dunedin, New Zealand

Re: Trouble interpreting interaction model/"problem model"

Postby heg90 » Sun May 26, 2019 11:51 pm

Hi,

Thank you so much for the clarification. I was under the impression that the odds ratio could be translated into the probability of occupancy, as I've read in a recent paper similar to my study that they reported effect size as, for example: "A 10% increase in relative percent conifer was associated with a 27% increase in marten occurrence (odds ratio = 1.27)." When I tried applying that to my OR I thought that I could translate that as "a 1% increase in SC5 was associated with a 17.8% increase in marten occurrence," or if you multiply that to 20% "a 20% increase is associated with a 17.8 *20 = 356% increase in marten occurrence." I suspect I'm calculating it wrong, as it sounds like you're suggesting that to get insight on the effect that 20% SC5 would have on occupancy I would calculate that as e^(0.1641*20) rather than what I had initially done by taking (e^0.1641) = 1.178, and then multiplying 17.8 * 20.
heg90
 
Posts: 20
Joined: Sun Nov 04, 2018 4:52 pm

Re: Trouble interpreting interaction model/"problem model"

Postby darryl » Mon May 27, 2019 12:05 am

If you're interpreting effect sizes in terms of odds ratios, then you have to word the results in terms of odds of occurrence (or similar; unless the paper previously defined 'occurrence' as odds of occurrence).

Yes, you're calculating the 20% change incorrectly.
darryl
 
Posts: 495
Joined: Thu Jun 12, 2003 3:04 pm
Location: Dunedin, New Zealand

Re: Trouble interpreting interaction model/"problem model"

Postby heg90 » Mon May 27, 2019 12:29 am

Okay, thank you for the clarification. Probability of occupancy is close to zero around the mean for elevation. If I am interested in knowing what the effects of 20% size class 5 are at an elev = -1 (translates to ~400m), is it correct to calculate this as follows:

logit(psi) = Bo + B1(elev) + B2(SC5) + B3(elev*SC5)
= -4.77 - 10.1791(-1) + 0.1641(20) + 0.337(-1*20)
= 1.941
OR = e^1.941 = 7.022

Is there a way to change the value of elev in MARK to elev = -1 instead of defaulting to elev = 0?
heg90
 
Posts: 20
Joined: Sun Nov 04, 2018 4:52 pm

Re: Trouble interpreting interaction model/"problem model"

Postby darryl » Mon May 27, 2019 12:49 am

e^1.941 = odds of occurrence at elev=-1 and SC5=20, not the odds ratio. I suggest you take another look at how probability, odds and odds ratios related to each other, and with logistic regression. It does take a bit of practice.

If you want to isolate the effect of SC5 at elev=-1, then that would be (ie using only the terms that relate to SC5):
B2+B3*elev = 0.1641 + 0.337(-1) = -0.1729

so OR = e^(-0.1729) for a 1% change in SC5, or OR=e^(-0.1729*20) for a 20% change in SC5
darryl
 
Posts: 495
Joined: Thu Jun 12, 2003 3:04 pm
Location: Dunedin, New Zealand

Re: Trouble interpreting interaction model/"problem model"

Postby cooch » Mon May 27, 2019 9:37 am

heg90 wrote:Is there a way to change the value of elev in MARK to elev = -1 instead of defaulting to elev = 0?


See first sub-section on 'covariates' in Chapter 21 in the MARK book.
cooch
 
Posts: 1628
Joined: Thu May 15, 2003 4:11 pm
Location: Cornell University


Return to analysis & design questions

Who is online

Users browsing this forum: No registered users and 11 guests