www.phidot.org

I am having some issue with models not working in secr version 4.3.1 (and also 4.2.x) when using hcov="sex" (where sex is a factor with F, M and NA). This is a large multi-survey dataset that was working fine in secr 3.1.5.

Code: Select all

> secr.0<- secr.fit (capthist ,model = list(g0~1,sigma~1,D~1), hcov='sex',  buffer=buffer)
Checking data
Preparing detection design matrices
Preparing density design matrix
Finding initial parameter values...
Initial values  D = 2e-05, g0 = 0.00527, sigma = 6574.44594
Maximizing likelihood...
Eval     Loglik        D       g0    sigma pmix.h2M
   1         NA -10.6871  -5.2411   8.7909   1.5000
   2         NA -10.6871  -5.2411   8.7909   1.5000
   3         NA -10.6871  -5.2411   8.7909   1.5000
  .....
  19         NA -10.6871  -5.2411   8.7918   1.5002
  20         NA -10.6871  -5.2411   8.7909   1.5003
Completed in 69.84 seconds at 22:36:17 01 Sep 2020

If I remove hcov everything works fine:

Code: Select all

> secr.0<- secr.fit (capthist ,model = list(g0~1,sigma~1,D~1),  buffer=buffer)
Checking data
Preparing detection design matrices
Preparing density design matrix
Finding initial parameter values...
Initial values  D = 2e-05, g0 = 0.00503, sigma = 6574.44594
Maximizing likelihood...
Eval     Loglik        D       g0    sigma
   1  -2219.929 -10.7166  -5.2868   8.7909
   2  -2219.929 -10.7166  -5.2868   8.7909
   3  -2219.928 -10.7166  -5.2868   8.7909
   .....
  65  -2086.951  -9.7860  -4.4418   8.4692
  66  -2086.953  -9.7860  -4.4419   8.4700
Completed in 99.19 seconds at 22:39:02 01 Sep 2020


Digging a bit deeper I found that the issue only occurs if at least one of the surveys in capthist only has individuals of one sex. Any suggestions on how to fix this? The sex covariate is already coded as a factor with two levels (F and M).

Here code to reproduce the issue:

Code: Select all

temptrap <- make.grid(nx = 6, ny = 6, spacing = 20, detector ="proximity")
capthist1<-sim.capthist (temptrap, detectpar = list(g0 = 0.2,sigma = 20))
capthist2<-sim.capthist (temptrap, detectpar = list(g0 = 0.2,sigma = 20))
covariates(capthist1)<-data.frame(sex=c("F","M")[rbinom(dim(capthist1)[1],1,0.5)+1])
covariates(capthist2)<-data.frame(sex=rep("M",dim(capthist2)[1]))

capthist<-MS.capthist(capthist1,capthist2)

secr.fit(capthist,hcov="sex")
secr.fit(capthist)

Thanks,

Mathias

Hi Mathias
I've found the bug and will provide a fix soon. Thanks for your report.
Murray

OK. The bug was fixed by replacing 'factor' with 'as.factor' in internal function getknownclass().

The new windows binary is available at secr_4.3.2.zip, and the source package at secr_4.3.2.tar.gz. It will be a month or more before 4.3.2 appears on CRAN, and this 'pre-release' version will change. There's another issue to sort out (vignettes not compiling to pdf). See secr 4.3.2 NEWS.txt for recent changes.

Murray

Hi Murray,

thanks for the quick fix. It works for my case. The only thing to keep in mind with this fix is that users will need to manually define the correct factor levels for each capture history in the multi-session capture history list. If they just provide a string or the default factor levels (see code below) it won't work.

Best,

Mathias

Code: Select all

temptrap <- make.grid(nx = 6, ny = 6, spacing = 20, detector ="proximity")
capthist1<-sim.capthist (temptrap, detectpar = list(g0 = 0.2,sigma = 20))
capthist2<-sim.capthist (temptrap, detectpar = list(g0 = 0.2,sigma = 20))

#still does not work
covariates(capthist1)<-data.frame(sex=c("F","M")[rbinom(dim(capthist1)[1],1,0.5)+1])
covariates(capthist2)<-data.frame(sex=rep("M",dim(capthist2)[1]))

#this works now
covariates(capthist1)<-data.frame(sex=factor(c("F","M")[rbinom(dim(capthist1)[1],1,0.5)+1],levels=c("F","M")))
covariates(capthist2)<-data.frame(sex=factor(rep("M",dim(capthist2)[1]),levels=c("F","M")))

capthist<-MS.capthist(capthist1,capthist2)

secr.fit(capthist,hcov="sex")

That's a good point. The function shareFactorLevels() was meant to fix inconsistencies between sessions, and it does so with a couple of minor adjustments in a new version accessible by the links given previously. Following your first example, this now works:

Code: Select all

CH <- MS.capthist(capthist1, capthist2)
CH <- shareFactorLevels(CH)
secr.fit(CH, hcov = 'sex')


I have also amplified the warning message to suggest using shareFactorLevels.
Murray

Hello,

I am having the same issue where all the log likelihood values are NA when I include hcov = "Sex", but using version 4.3.2 and shareFactorlevels() haven't helped.

For context, I am using a dataset with 224 sessions over 12 years. There are some sessions with very few individuals captured (total capture occasions ranges from 1 to 46), which I would think might cause some issues. However, other models I've tried have converged and seemed to work well, including a finite mixture model.

Is there something I'm missing or am I using it incorrectly?

Code: Select all

reducedcapthist = shareFactorLevels(reducedcapthist)
m_sex_hcov =  secr.fit(reducedcapthist,
                   ncores = 4,
                   model = list(g0 ~ 1,
                   sigma ~ 1, D ~ 1),
                   masks_reduced,
                   verify = FALSE,
                   CL = FALSE,
                   details =list(autoini = 220),
                   hcov = "Sex"
                   )


Code: Select all

Preparing detection design matrices
Preparing density design matrix
Finding initial parameter values...
Initial values  D = 7.66332, g0 = 0.10035, sigma = 27.49728
Maximizing likelihood...
Eval     Loglik        D       g0    sigma pmix.h2M
   1         NA   2.0364  -2.1933   3.3141   1.5000
   2         NA   2.0364  -2.1933   3.3141   1.5000
   3         NA   2.0364  -2.1933   3.3141   1.5000
   4         NA   2.0364  -2.1933   3.3141   1.5000
   5         NA   2.0364  -2.1933   3.3141   1.5000
   6         NA   2.0364  -2.1933   3.3141   1.5000
   7         NA   2.0366  -2.1933   3.3141   1.5000
   8         NA   2.0364  -2.1932   3.3141   1.5000

This may have nothing much to do with hcov or factor levels - is there direct evidence of that? Sometimes it's hard to find a set of parameter values that 'work' across all sessions of a multisession dataset (and yours has way more sessions than any I've handled). If other models fitted OK you can provide one in the 'start' argument to avoid the autoini algorithm altogether.

Hi Murray,

I'm not quite sure what you mean by finding parameter values that work across all sessions. Do you think allowing for flexibility in values for pmix would help? For instance, including year or site as a covariate in pmix or something similar?

Thanks,
Ryan

Your very first likelihood evaluation fails. My guess is that the starting values are somehow very highly improbable with respect to the data for at least one session. Hence my suggestion you fish around for better starting values. I doubt messing with the model structure (including covariates or other variation) itself will fix the initial-NA problem.

If all else fails I would attempt some debugging by separately evaluating the likelihood for each of your numerous sessions with the initial parameter values:

for (i in 1:224) {
cat('i = ', i, ' LL = ', secr.fit(reducedcapthist[[i]], start = c(2.0364, -2.1933, 3.3141, 1.5000),
hcov = 'Sex', details = list(LLonly = TRUE)), '\n')
}

Which ones fail?

(I can't remember how hcov handles sessions with no detections - that may be the problem.)

Hi Murray,

I took at a look at the capthist dataframes for the eight sessions it failed for, and they had only one capture occasion with 1 or 2 females. The sessions where there were any males worked, although the coefficient estimates for pmix.h2M were ridiculously large (36 in one session). I've played around with changing the starting value for pmix.h2M but also had no luck.

I currently have all the empty sessions removed because it was the only way I could include individual covariates without getting errors.

Any thoughts? I'm thinking of just including sex as a group, because the majority of the values are known.