Inconsistencies in Presence output: DM and model dependent

[jdiffen]

Hi All,

I decided to give Presence a whirl and experimented with different design matrices. To make sure I was specifying models correctly I started comparing some results to MARK, where I'm more up to speed.

I stumbled upon some inconsistencies in Presence and am wondering if other folks have observed this.

My data consists of 31 sites, 5 primary sessions, each with 3 surveys in each session. In all cases I ran equivalent models in MARK and Presence (MARK: RD occupancy Estimation with psi, gamm. PRESENCE: Seasonal occupancy and colonization).

I ended up comparing output from the programs for 4 design matrices, all modelling temporal variation in Psi.

Identity:
10000
01000
00100
00010
00001

Intercept 1
10000
11000
10100
10010
10001

Intercept 2
11000
10100
10010
10001
10000

Means
11000
10100
10010
10000
1-1-1-1-1

Psi(T), gam(.), p(.) produced identical results across all design matrices and both programs (8 runs). Woo Hoo!!

Psi(T), gam(,), p(T) (p varies across primary sessions, but not within), produces problems.


In MARK, all models were identical (same AIC, loglikelihood, etc).

In Presence, the Means and Intercept 1 model match each other and MARK. For these models, -2LL = 493.339853

For Intercept 2: -2LL = 498.574913 (=deltaAIC ~5!!)
For, Identity: -2LL = 494.082105

All models have the same # of parameters and I've checked and double checked the 2 design matrices causing the problems.

Any ideas? I'm using Presence 2.0 <070626.1646>

Thanks, Jay

[darryl]

Hi Jay,
First, I don't think your 'means' DM is quite right; I think the final column in the second last row should be a 1, but that shouldn't be causing your problem.

Second, make sure you have the most recent update of PRESENCE, there was a little bug that sneaked it's way into the multi-season model around May-June and was given whacky output at times (recall I said there was a problem with the spotted owl results at the workshop?).

Third (and this is my best guess for what may be causing the problem), for these reparameterized multi-season models (ie not using initial psi, colonization and extinction), then in PRESENCE there is a constraint to check that the probability that is not directly in the model, which is now a derived parameter (local extinction in your case), is between the values of 0 and 1. If any of the derived values is outside of 0-1, then the numerically procedure tries a different value within the likelihood maximization process. I've found this constraint can make the numerical algorithm a bit flaky, but have never tried what you have here which is a bit worrisome. What I think is happening here is that with the different DM's you're moving through slightly different areas of the parameter space and in 1 case there's no problems while in another case you start butting up against this constraint so PRESENCE sends you off in another direction where you end up at a different local maximum.

Now, we included this constraint when modelling annual occupancy to ensure that the derived estimates are biologically reasonable given the formulation of the multi-season occupancy model. MARK does not perform a check and most times there's not going to be a problem, but we have come across situations where MARK and PRESENCE don't agree and have found that the derived estimates in MARK is outside 0-1.

If Jim Hines is reading this thread he may have another suggestion.

Cheers
Darryl

[jdiffen]

Hi Darryl,
Yup, my means DM is wrong..doh! (5b0-b4)/5 doesn't make much sense!

Jim has the .pao and .pa2 files so I'll soon be publicly flogged for doing something very wrong.

I had wondered if some of the algorithms might vary between MARK and Presence.

Though I was using 070626.1646, I've replicated the results in version 07013.1626, though this was not exhaustive and I did not start from scratch..just opened the pa2 file and reran a few of the models.

The time specific Episilon estimates from MARK don't look troublesome....unless the lower CI's affect the decision to alter the ML search.

Epi CI lower upper
0.0240234 0.1357162 -0.2419804 0.2900273
0.1460153 0.1209907 -0.0911265 0.3831572
0.3011489 0.1097842 0.0859719 0.5163260
0.1259405 0.1927269 -0.2518042 0.5036852


Thanks again, Jay

[darryl]

Jay,
Ok, we'll wait for Jim's decision. It does sound like the constraint may be the cause of the problem if you're getting the same results with different builds. The first derived estimate of epsilon is pretty close to zero so the algorithm may be having problems getting to that maximum depending on what direction it's approaching it from. If it is, we'll working on a fix for PRESENCE; i's not very desirable behaviour!

Cheers
Darryl