Adjusted AIC values for stacked models

[kdavis79]

Hello,

I have run single-season multi-state occupancy models (with the conditional binomial parameterization) for my system, using a stacked approach where some sites appear twice in the detection history (once for the first year of survyes, again for the second year of surveys). Not all sites were visited twice (121/234). Someone told me I would need to adjust my AIC values to take into account the non-indepence of the detections histories (i.e. reduce my sample size to the number of unique wetlands: 234, not the total number of detection histories (345).

I've calculated AICc manually using the formula: AICc = AIC+ 2K(K +1) n-K +1. However, I've run into downstream issues where I would like to use the summedWgt function, but that obviously defaults to using the normal model weight. I've tried modifying the function but I'm running into some odd issues, so I figured I woudl revisit whether or not there are any built-in options earlier in the process to adjust my sample size. I hadn't seen any, but I'm hoping I just missed it.

Thanks for the input!

[jhines]

I'll try to think about a modification to RPresence to allow QAICc model weights, but in the meantime, the following code will replace the AIC, and model weights in the aictable with QAICc values:

# replace aic, delta-aic, and model weight with values based on qaicc.

# values with no small-samp-size adjustment and no lack-of-fit adjustment
ness=9e9; chat=1;
# change ness to adjust for small sample size
# change chat to adjust for lack-of-fit

t1=aictable$table; # get aic table
K=t1$npar # set K=number of parameters of each model
# compute qaicc...
t1$AIC=t1$neg2ll/chat+2*K+2*K*(K+1)/(ness-K-1)
t1$DAIC = t1$AIC - min(t1$AIC)
# compute qaicc weight, qwgt
qwgt=exp(-t1$DAIC/2)
t1$wgt=round(qwgt/sum(qwgt),4)
print(t1[,-1])
# now, we can compute weights based on qaicc
weights <- summedWgt(covs, parameter, aictable)

[kdavis79]

Hi Jim!

Thanks for the quick response. I did figure out my previous issue when calculatingthe AICc following the formula I initially described, but using QAICc as you described would probably be better. However, I believe to get the c-hat value I need to run the bootstrap goodness of fit test ("modfitboot number of bootstraps for assessing model fit. Only works for type = "so".")? and from what I see in the documentation, that's only possible for the basic single-season model (not any other varities, including multi-state, which is what I'm running).

If c-hat values are not available, is standard procedure to keep c-hat = 1?

Thanks again for the help!

[jhines]

You're right that multi-season gof tests are not available in RPresence at this time. One option would be to analyze each year individually, computing a gof chi-square value with degrees of freedom for each, then add the chi-square values and df's to get a probability. If there aren't many surveys, small expected values may not be a problem and the chi-square test should be OK.

I think what some folks do is acknowledge that no gof test is available and variances might be underestimated. To try to account for this, they make the model as general as possible, while still being useful to address the hypotheses at hand.

[kdavis79]

Okay, I think that makes sense, but I'd like to clarify I'm not doing multi-season models, I'm doing a single-season model with multiple states (absence, species presence, species presence w/ breeding). I have data from two years for some sites, but have opted to stack that data (leading to non-independent sites, which is the sample size issue I need to adjust for).

I believe the only implementation of multi-state models is the "do.ms.1" or "do.ms.2" options, which are dynamic models, but I'm just not providing a second season when running them (and only provide formulas for psi, R, and p).

How would I implement a GOF chi-square test in this context?

[jhines]

OK, there isn't a gof test for the single-season, multi-state model at the moment. With your data containing only 3 surveys, I thought I could create an ad-hoc gof test fairly quickly, but my ad-hoc test resulted in no statistic being able to be computed due to insufficient data. The only other option I can think of is to simulate data like the single-state case, but that isn't something I can do quickly at this time. For now, I suggest leaving c-hat at 1 and saying that data are insufficient to do a gof test. With such a low number of surveys and detections, I doubt a gof test would have enough power to detect whether the model fits anyway.