lcl and ucl values

posts related to the RMark library, which may not be of general interest to users of 'classic' MARK

lcl and ucl values

Postby Bermad » Mon Apr 10, 2017 1:01 pm

Hi, someone could explain me why the lcl and ucl values include the values [0-1] when the estimated value are close to a boundary (e.g. the value 1; see the image).


Image

I mean that the upper and lower credibility intervals are close to the estimated value all time, except when the estimated value reach the value 1. In that moment the ucl value is 1 and the lcl value is 0.


The results are from a simulated capture-recapture history for a multistate model. I'm doing it for assessing the accuracy of capture-recapture models in some situations. Specifically, I simulated a capture history with 3 status (A, B and C), and I ran the next model (phi{t}, p{.}, psi{.}).
I think that the wide range of the lcl and ucl are not related with the code I used in RMark. Nevertheless, I put the code below just in case.

Thanks in advance,

Code:
(1) Specify the model
Code: Select all
model_process <- process.data (input, model="Multistrata")


(2) Specify the model parameters
Code: Select all
Phi.time=list(formula=~time+stratum)
p.dot=list(formula=~1+stratum)
Psi.dot=list(formula=~1)


(3) Calculate some transition probabilities by substraction
Code: Select all
model_make=make.design.data(model_process,                               parameters=list(Psi=list(subtract.stratum=c("A", "C", "B"))))


(4) Fix as 0 the transitions from B and C to A because such transitions are impossible:
Code: Select all
model_make$Psi=model_make$Psi[!(model_make$Psi$stratum=="B" & model_make$Psi$tostratum=="A"),]
model_make$Psi=model_make$Psi[!(model_make$Psi$stratum=="C" & model_make$Psi$tostratum=="A"),]


(5) Run the model
Code: Select all
model_mark <- mark (model_process, model_make, model.parameters = list(S=Phi.time, p=p.dot, Psi=Psi.dot))
Bermad
 
Posts: 7
Joined: Wed Jun 01, 2016 4:19 am

Re: lcl and ucl values

Postby jlaake » Mon Apr 10, 2017 1:11 pm

Don't use design data deletion for Psi or any other parameter with a multinomial logit link. In general I have asked folks to no longer use the design data deletion as a way to fix parameters. I wish I never used that approach. There is a much better way now. See the first sticky post on RMark subforum at

http://www.phidot.org/forum/viewtopic.php?f=21&t=2887

This could be causing your problems. But if not parameters at a boundary often end up with large std errors and conf intervals. Also if you are fixing mlogit parameters to 0, see the mlogit0 argument to speed up execution time.
jlaake
 
Posts: 1417
Joined: Fri May 12, 2006 12:50 pm
Location: Escondido, CA

Re: lcl and ucl values

Postby egc » Mon Apr 10, 2017 1:14 pm

Generally, estimates at/near boundary are often problematic, as if often reflected in SE and/or CI being quite non-sensical. Short of knowing how you simulated your data, its hard to say if part of the prlblem may be there, but it is a possiblity. Also, worth having a look at Appendix F for a general discussion of parameter identifiability.
egc
Site Admin
 
Posts: 201
Joined: Thu May 15, 2003 3:25 pm

Re: lcl and ucl values

Postby Bermad » Tue Apr 11, 2017 5:16 am

My results show that one parameters was not counted. Specifically, the R message was:
Note: only 12 parameters counted of 13 specified parameters
AICc and parameter count have been adjusted upward


According to the information provided in the appendix C: Coping with identifiability, this parameter may be not counted because end up at boundaries. I have two candidates for explaining it:

Code: Select all
   Survival_A time_6 (S=~stratum+time)
      lcl == 0
      estimate == 1
      ulc == 1

   Recapture_A (p=~1+stratum)
      lcl == 0
      estimate == 0.9999
      ulc == 1


I suppose that the not counted parameter was the Survival for the status A at time 6 because the other parameter is constant in time.

The appendix C of the Mark book suggests the sparse data as one potential reason for the wide credibility intervals. However I added 50 individuals per time step (24 time steps) in my simulations. Specifically the number of individuals in the time steps 5-7 was:

time_5 <- State_0==1165 State_A==51 State_B==159 State_C==272
time_6 <- State_0==1055 State_A==52 State_B==178 State_C==365
time_7 <- State_0==932 State_A==51 State_B==224 State_C==443


(To clarify that the lower number of individuals in the status A is because I did not allow the transition to A as arrival state).
Looking the number of marked individuals, I would discard that the sparse data is the reason of the wide credibility intervals this possibility. Nevertheless, I kept following the appendix C, and I fixed the parameter "Survival_A time_6" to 1 typing:

Code: Select all
model_make_2$S$fix=NA
model_make_2$S$fix[model_make_2$S$time==6]<-1


I fixed the warning message from R related to the number of the counted parameters. However, I do not understand why I should fix the survival parameter if I have no sparse data.
Could you explain me why I have wide credibility intervals without sparse data?
Is it possible that Mark has problems, different to the input, for calculating estimates close to the boundaries?
Bermad
 
Posts: 7
Joined: Wed Jun 01, 2016 4:19 am

Re: lcl and ucl values

Postby jlaake » Tue Apr 11, 2017 1:06 pm

Did you do what I suggested in regards to fixing real Psi values? Until you do it is not clear to me that the results you are getting are correct. Appendix C needs to be updated and the Workshop notes from 2015 in the Documentation Archive is a more recent source of information. See http://www.phidot.org/software/mark/rma ... tation.zip
.
jlaake
 
Posts: 1417
Joined: Fri May 12, 2006 12:50 pm
Location: Escondido, CA

Re: lcl and ucl values

Postby Bermad » Thu May 04, 2017 10:07 am

Yes, I followed your recomendation, and I solved the problem with the number of counted parameters.

However I would like to know whether I always should repeat the analysis when I got an estimate close to the boundaries, but fixing the parameter that was close at the boundaries to 0 or 1.
Bermad
 
Posts: 7
Joined: Wed Jun 01, 2016 4:19 am

Re: lcl and ucl values

Postby jlaake » Tue May 09, 2017 9:02 am

You should only be fixing real parameter values for structural reasons (ie p=0 when you did not sample) or when parameters are truly confounded (p(t)Phi(t) for last occasion). Once you fix a real parameter it is no longer counted for AIC so unless that parameter(s) is(are) fixed across all models it will tend to overfit.
jlaake
 
Posts: 1417
Joined: Fri May 12, 2006 12:50 pm
Location: Escondido, CA


Return to RMark

Who is online

Users browsing this forum: No registered users and 11 guests