Models with extremely high SE and errors

[Happyme]

So if I understand this correctly most of the psi estimates are not 0 or 1 and this is not a boundary issue? I am also having trouble understanding when psi is held constant and p (water temp) my SE is extremely high. But with the model psi(cover objects), p(water temp) my SE is within reason?

[Happyme]

Site estimates for water temp in the model psi(cover objects), p(water temp)
Individual Site estimates of <P[1]>
Site estimate Std.err 95% conf. interval
P[1] 1 site1 : 0.0000 0.0000 0.0000 - 0.0106
P[1] 2 site2 : 0.0000 0.0000 0.0000 - 0.0105
P[1] 3 site3 : 0.0000 0.0000 0.0000 - 0.0105
P[1] 4 site4 : 0.0000 0.0000 0.0000 - 0.0104
P[1] 5 site5 : 0.0000 0.0000 0.0000 - 0.0105
P[1] 6 site6 : 0.0000 0.0000 0.0000 - 0.0104
P[1] 7 site7 : 0.0000 0.0000 0.0000 - 0.0107
P[1] 8 site8 : 0.0000 0.0000 0.0000 - 0.0108
P[1] 9 site9 : 0.0000 0.0000 0.0000 - 0.0102
P[1] 10 site10 : 0.0000 0.0000 0.0000 - 0.0099
P[1] 11 site11 : 0.0000 0.0000 0.0000 - 0.0131
P[1] 12 site12 : 0.0000 0.0000 0.0000 - 0.0130
P[1] 13 site13 : 0.0000 0.0000 0.0000 - 0.0129
P[1] 14 site14 : 0.0000 0.0000 0.0000 - 0.0132
P[1] 15 site15 : 0.0000 0.0000 0.0000 - 0.0142
P[1] 16 site16 : 0.0000 0.0000 0.0000 - 0.0143
P[1] 17 site17 : 0.0000 0.0000 0.0000 - 0.0129
P[1] 18 site18 : 0.0000 0.0000 0.0000 - 0.0142
P[1] 19 site19 : 0.0000 0.0000 0.0000 - 0.0132
P[1] 20 site20 : 0.0000 0.0000 0.0000 - 0.0133
P[2] 1 site1 : 0.1540 0.0761 0.0548 - 0.3637
P[2] 2 site2 : 0.1397 0.0733 0.0468 - 0.3493
P[2] 3 site3 : 0.1330 0.0719 0.0432 - 0.3425
P[2] 4 site4 : 0.1354 0.0724 0.0445 - 0.3449
P[2] 5 site5 : 0.1106 0.0665 0.0320 - 0.3188
P[2] 6 site6 : 0.1337 0.0721 0.0436 - 0.3431
P[2] 7 site7 : 0.1374 0.0729 0.0456 - 0.3470
P[2] 8 site8 : 0.1358 0.0725 0.0447 - 0.3453
P[2] 9 site9 : 0.0127 0.0171 0.0009 - 0.1573
P[2] 10 site10 : 0.0143 0.0187 0.0011 - 0.1631
P[2] 11 site11 : 0.0150 0.0193 0.0012 - 0.1654
P[2] 12 site12 : 0.0148 0.0191 0.0011 - 0.1648
P[2] 13 site13 : 0.0051 0.0085 0.0002 - 0.1187
P[2] 14 site14 : 0.0110 0.0154 0.0007 - 0.1505
P[2] 15 site15 : 0.0069 0.0108 0.0003 - 0.1304
P[2] 16 site16 : 0.0112 0.0156 0.0007 - 0.1513
P[2] 17 site17 : 0.0045 0.0076 0.0002 - 0.1136
P[2] 18 site18 : 0.0050 0.0084 0.0002 - 0.1181
P[2] 19 site19 : 0.0052 0.0086 0.0002 - 0.1192
P[2] 20 site20 : 0.0050 0.0083 0.0002 - 0.1179
P[3] 1 site1 : 0.5711 0.1321 0.3163 - 0.7931
P[3] 2 site2 : 0.5610 0.1309 0.3108 - 0.7837
P[3] 3 site3 : 0.5314 0.1270 0.2944 - 0.7551
P[3] 4 site4 : 0.5621 0.1310 0.3113 - 0.7846
P[3] 5 site5 : 0.5870 0.1339 0.3249 - 0.8076
P[3] 6 site6 : 0.6141 0.1366 0.3396 - 0.8312
P[3] 7 site7 : 0.6506 0.1392 0.3593 - 0.8608
P[3] 8 site8 : 0.6033 0.1356 0.3337 - 0.8220
P[3] 9 site9 : 0.6026 0.1355 0.3334 - 0.8214
P[3] 10 site10 : 0.6720 0.1400 0.3710 - 0.8768
P[3] 11 site11 : 0.6588 0.1396 0.3638 - 0.8670
P[3] 12 site12 : 0.6850 0.1403 0.3781 - 0.8860
P[3] 13 site13 : 0.7191 0.1398 0.3973 - 0.9086
P[3] 14 site14 : 0.2574 0.0901 0.1209 - 0.4661
P[3] 15 site15 : 0.4374 0.1133 0.2397 - 0.6572
P[3] 16 site16 : 0.6553 0.1394 0.3619 - 0.8644
P[3] 17 site17 : 0.6305 0.1379 0.3484 - 0.8448
P[3] 18 site18 : 0.6433 0.1388 0.3553 - 0.8550
P[3] 19 site19 : 0.6437 0.1388 0.3556 - 0.8554
P[3] 20 site20 : 0.6473 0.1390 0.3575 - 0.8582
P[4] 1 site1 : 0.0932 0.0614 0.0241 - 0.2993
P[4] 2 site2 : 0.1079 0.0658 0.0307 - 0.3159
P[4] 3 site3 : 0.0979 0.0629 0.0262 - 0.3047
P[4] 4 site4 : 0.1320 0.0717 0.0427 - 0.3414
P[4] 5 site5 : 0.0880 0.0597 0.0219 - 0.2932
P[4] 6 site6 : 0.0910 0.0607 0.0232 - 0.2968
P[4] 7 site7 : 0.0911 0.0608 0.0232 - 0.2969
P[4] 8 site8 : 0.0926 0.0612 0.0239 - 0.2987
P[4] 9 site9 : 0.3505 0.1012 0.1842 - 0.5632
P[4] 10 site10 : 0.3352 0.0992 0.1740 - 0.5469
P[4] 11 site11 : 0.3266 0.0982 0.1682 - 0.5377
P[4] 12 site12 : 0.3525 0.1014 0.1856 - 0.5654
P[4] 13 site13 : 0.3284 0.0984 0.1694 - 0.5396
P[4] 14 site14 : 0.3267 0.0982 0.1683 - 0.5378
P[4] 15 site15 : 0.2799 0.0927 0.1363 - 0.4890
P[4] 16 site16 : 0.3194 0.0973 0.1633 - 0.5301
P[4] 17 site17 : 0.3065 0.0957 0.1545 - 0.5165
P[4] 18 site18 : 0.2972 0.0947 0.1482 - 0.5069
P[4] 19 site19 : 0.2955 0.0945 0.1470 - 0.5051
P[4] 20 site20 : 0.2764 0.0923 0.1339 - 0.4854

Sorry to ask so many questions. I have taken an occupancy modeling class in graduate school but most labs had predictable outcomes. My data from my thesis research is not being predictable and I just want to understand the output. There is limited resources to troubleshoot and come to valid conclusions. This forum has been the best source of information for me. Again, thank you for any input and time spent looking over my output.

[jhines]

The results of the psi(cover objects),p(water temp) look reasonable, given the sparse data. I didn't see an estimate of psi for the psi(.)p(water temp) model. Was it 1.0?

My guess is that you probably want to say something about the effect of the covariates, water temp and cover objects. In that case, you can build models with and without those covariates and let AIC determine which best describes the data or has the most support. So your model-set would include these models:

psi(.)p(.)
psi(.)p(water temp)
psi(cover)p(.)
psi(cover)p(water temp)

If the psi(.)p(.) has the lowest AIC, then the effect of the covariates is not much different from zero, either due to biology or too small sample size.

[Happyme]

The psi for the psi(.), p(Water temp) model was 1.0. I ran the models you suggested and psi(.), p(water_temp) is still the most supported model (deltaAIC 0.00, weight 0.6399) with psi(cover_objects), p(water_temp) second with a deltaAIC of 1.15 and weight of 0.3601. However, psi(.), p(.) does not have the lowest AIC, psi(Cover_objects), p(.) does. Does this increase the effect of the covariates or would it still not be much different than zero?

[darryl]

Your estimated effect size for cover_objects is the associated beta parameter/regression coefficient. That value may be close to zero if the psi(.)p(.) had most support.

If you're struggling with the interpretation of some of the results, then if you can't find any resources that you find suitable in the occupancy modelling context, you can also look into the logistic regression literature as the modelling is essentially 2 simultaneous logistic regression analyses, one of occupancy and one on detection.

Two other points. AIC-type approaches should be used as a guide, you still need to check that the results from each model are giving biologically sensible results. If they're not, you may want to consider excluding that model (although making sure your reason is justifiable, and not just because you don't like the result). Especially with small sample sizes, you could have a spurious result (i.e. a model that fits the data well, but isn't useful otherwise).

The other point is that these models are statistical, not magical. No matter how much you wish otherwise, sometimes there simply isn't enough information in the data you've collected to reliable tell you what's going on. This is where study design is really important so you can get realistic expectations of what can be achieved with the amount of effort you're planning to put into the field.

Cheers
Darryl

[Happyme]

Ok. Thank you for the guidance. I will look into the logistic regression analysis. The psi(cover objects), p(water temp) model does look pretty good. Can you explain to me what is happening in my first sampling event and all of the parameter estimates are 0.00 but the other three sampling events have a broad range of parameter estimates?

[jhines]

Your model for detection has p(water temp), so you're making detection a linear (on logit scale) function of water temperature. Since you have no detections in the first two surveys, the function starts with p=0 for whatever water temperature you have in the first survey. Then, p increases linearly over the next 3 surveys. If you look at the "beta" estimates for p, you should have an "intercept" and "effect" beta estimates. The real estimates of p are computed as:

Code: Select all

                          exp(beta-intercept + beta-effect)
p(survey t) = -----------------------------------------------------
                       1 + exp(beta-intercept + beta-effect)

Since the first p=0, the beta-intercept estimate is probably a large negative value (eg., -20 or less). Since the p's increase over the surveys, the beta-effect is a value > 0.

[Happyme]

Ah ok. This makes sense. So what do you think is the biggest design mistake here? At least from a protocol development point of view for future research? Better to have more study sites and less replication? I did rerun the models how Jim suggested with only the two seasons in March and April. I still have crazy SE for my untransformed beta estimates for psi when psi is held constant. But by c hat is 0.98. My constant model, psi(.), p(.) has an AICc of 3.54 with 6 digit SE. The real parameter estimate is 1 and SE is 0. It was third on the list of 5 possible models. Two only had AICc < 2.0. My question is if the constant model is not supported and not biologically relevant are the two models with AICc with AICc < 2.0? Holding one or the other constant is when I run in to trouble. When they both are not held constant is when I get the best output. I read on the USGS site that high SE are not necessarily a problem for untransformed beta parameters which is what I have but they are for real parameter estimates. But mine are ok there.

And you all are the best. I know I keep asking questions but I really want to understand what’s going on here. I am reading through the literature and the USGS site is helpful as well. My particular data just seems to have a set of unusual issues I just want to make sure I work through and interpret correctly. Thank you all for being so helpful.

[jhines]

To get an idea of where to invest your effort, program GENPRES (included with Presence) allows you to simulate studies with different numbers of sites and surveys to see how standard errors are affected. In general, with studies with high occupancy and low detection, you're better off adding surveys than sites. With low occupancy and high detection, you're better off with more sites. There is a table in the Occupancy Estimation and Modeling book which gives the optimum number of sites and surveys for given values of occupancy and detection. The table was produced using GENPRES.

Unfortunately, GENPRES does not allow simulating data with covariates. For that, I suggest using the R package, RPRESENCE. I think there is an example of generating data with covariates in that package.

My feeling is that the psi(.)p(water temp) model makes the most sense. Occupancy is obviously fairly high and may not be 1.0 as estimated in the model, but it is so high that you probably can't model it as a function of any covariates (ie., if sites are almost all occupied, you can't say occupancy varies by anything since occupancy doesn't seem to vary). Modeling detection as a function of water temperature seems to make sense, especially since it corresponds to your pre-conceived notion.

[Happyme]

This helps so much. Now the output is making sense. I was struggling with why the psi estimate SE was so high even when I would just use the two sampling periods in March and April. Thank you!

So when discussing my results is it acceptable to list all of the a priori models and then explain why the models with psi(cov) were removed from analysis because of the estimate of 1 and high SE but water temperature is a valid model because it explains the variation in detection for each sampling period?