change group structure in process.data (error)

[abreton]

In MARK under the huggins parameterization the derived abundance estimates are partitioned based on the group structure specified when the data were imported. If the user specified four groups on import - males2001, males2002, females2001, females2002 - then mark will derive an estimate of abundance +/- SE for these groups no matter what model the user specifies (dot, time varying, etc.). But what if the user wants a combined estimate WITH a SE of males and females in 2001 and the same for 2002? Sure, delta method, but RMark offers a much more attractive option, for individuals with average brain cells, since the group structure can be quickly manipulated (re-process.data).

I've been trying to do this and have run into an error. In the next set of code, I ran the dot model with the same group structure (combined age and year, 14 groups) as my MARK file,

Code: Select all

# convert
LYC=convert.inp("C:/Users/abreton.CNRDOM/Documents/CSU_Project/MARK_Analyses/LYC_2004_2008/EH_File_LYC_2004_10_Bins.inp",
group.df=data.frame(age.yr=c("SA04","A04","SA05","A05","SA06","A06","SA07","A07","SA08","A08","SA09","A09","SA10","A10"),
aclass=rep(c("SA","A"),7),
syear=c(rep("2004",2),rep("2005",2),rep("2006",2),rep("2007",2),rep("2008",2),rep("2009",2),rep("2010",2))),
covariates=c("L1","L2","L3","L4","L5","L6","L7","L8","L9","L10","L11","may1Bin","RMiBin"),use.comments=FALSE)

# process
LYC.huggins=process.data(LYC,model="Huggins",begin.time=1,mixtures=1,
 time.intervals=c(1,1,1,1,1,1,1,1,1,1), groups=("age.yr"))

# make design data
LYC.ddl=make.design.data(LYC.huggins)

# specify parameters to fix
p.fix.indices=c(83:88,6:11,93:99,16:22,107:110,30:33,119:121,42:44,129:132,52:55,154,77)
c.fix.indices=c(229:234,159:164,238:244,168:174,251:254,181:184,262:264,192:194,271:274,201:204,275:277,205:207,285:286,
 294,215:216,224)
p.fix.count=rep(0,length(p.fix.indices))
c.fix.count=rep(0,length(c.fix.indices))

# specify pim
p.dot.fix.shared=list(formula=~1,share=TRUE,fixed=list(index=c(p.fix.indices,c.fix.indices),value=c(p.fix.count,c.fix.count)))

# fit dot model
initial=-3.0337564
LYC.huggins.M0=mark(LYC.huggins,LYC.ddl, model="dot",
 model.parameters=list(p=p.dot.fix.shared),output=FALSE, simplify=FALSE, initial=initial)

..and the results match MARK perfectly. Here are elements from the dm,

Code: Select all

> LYC.huggins.M0$design.matrix[1:14,]
 1  2  3  4  5  6  7  8  9 10 11 12 13 14
 1  1  1  1  1  1  1  1  1  1  1  1  1  1
Levels: 1
> LYC.huggins.M0$design.matrix[155:168,]
155 156 157 158 159 160 161 162 163 164 165 166 167 168
  1   1   1   1   1   1   1   1   1   1   1   1   1   1
Levels: 1

Next, I changed the group specified in process.data from age.yr to syear (sample year). This resulted in an error. Here is the process.data with the alternative group structure,

Code: Select all

LYC.huggins=process.data(LYC,model="Huggins",begin.time=1,mixtures=1,
 time.intervals=c(1,1,1,1,1,1,1,1,1,1), groups=("syear"))

The error,

Error in extract.mark.output(out, model, adjust, realvcv) :
MARK did not run properly. If error message was not shown, re-run MARK with invisible=FALSE
********Following model failed to run : p(~1)c() **********

...and elements of the design matrix,

Code: Select all

> LYC.huggins.M0$results$beta
NULL
> LYC.huggins.M0$design.matrix[1:14,]
 1  2  3  4  5  6  7  8  9 10 11 12 13 14
 1  1  1  1  1  1  1  1  1  1  1  1  1  1
Levels: 1
> LYC.huggins.M0$design.matrix[155:168,]
<NA> <NA> <NA> <NA> <NA> <NA> <NA> <NA> <NA> <NA> <NA> <NA> <NA> <NA>
<NA> <NA> <NA> <NA> <NA> <NA> <NA> <NA> <NA> <NA> <NA> <NA> <NA> <NA>

Thanks for having a look.

andre

[jlaake]

Andre-

Did you use simplify=FALSE on that run? I didn't see the run command that failed. If so, please try with default of simplify=TRUE. There is no reason to not simplify and I've been meaning to remove that argument because I've not been checking to see if it works since I implemented the simplification process.
If that doesn't do it, please look in the mark.out file that the run create and send the error that it reports.

This is not likely a problem with process.data but in building/running model with simplify=F.

--jeff

[abreton]

Previously, I experienced an error when I specified fix parameters in combination with (the default) simplify=TRUE. When I instead set the mark function argument to simplify=FALSE I had no problems as long as I didn't change the group structure. Nonetheless, I tried your suggestion with the dot model. Here is the code when I DON'T FIX any parameters,

Code: Select all

# convert
LYC=convert.inp("C:/Users/abreton.CNRDOM/Documents/CSU_Project/MARK_Analyses/LYC_2004_2008/EH_File_LYC_2004_10_Bins.inp",
group.df=data.frame(age.yr=c("SA04","A04","SA05","A05","SA06","A06","SA07","A07","SA08","A08","SA09","A09","SA10","A10"),
aclass=rep(c("SA","A"),7),
syear=c(rep("2004",2),rep("2005",2),rep("2006",2),rep("2007",2),rep("2008",2),rep("2009",2),rep("2010",2))),
covariates=c("L1","L2","L3","L4","L5","L6","L7","L8","L9","L10","L11","may1Bin","RMiBin"),use.comments=FALSE)

# process
LYC.huggins=process.data(LYC,model="Huggins",begin.time=1,mixtures=1,
 time.intervals=c(1,1,1,1,1,1,1,1,1,1), groups=("age.yr"))

# make design data
LYC.ddl=make.design.data(LYC.huggins)

# specify pim
p.dot.shared=list(formula=~1,share=TRUE)

# fit dot model
initial=-3.0337564
LYC.huggins.M0=mark(LYC.huggins,LYC.ddl, model="dot",
 model.parameters=list(p=p.dot.shared),output=FALSE,initial=initial)

This code runs in RMark without any errors and the results match what I get when I specify the same model/run directly in MARK.

Here's what happens when I specify parameters to be FIXED in the PIM specifications. First the code,

Code: Select all

# specify parameters to fix
p.fix.indices=c(83:88,6:11,93:99,16:22,107:110,30:33,119:121,42:44,129:132,52:55,154,77)
c.fix.indices=c(229:234,159:164,238:244,168:174,251:254,181:184,262:264,192:194,271:274,201:204,275:277,205:207,285:286,
 294,215:216,224)
p.fix.count=rep(0,length(p.fix.indices))
c.fix.count=rep(0,length(c.fix.indices))

# specify pim
p.dot.fix.shared=list(formula=~1,share=TRUE,fixed=list(index=c(p.fix.indices,c.fix.indices),value=c(p.fix.count,c.fix.count)))

# fit dot model
initial=-3.0337564
LYC.huggins.M0=mark(LYC.huggins,LYC.ddl, model="dot",
 model.parameters=list(p= p.dot.fix.shared),output=FALSE,initial=initial)

Here's the error reported in the R console,

[code]Error in extract.mark.output(out, model, adjust, realvcv, vcvfile) :
MARK did not run properly. If error message was not shown, re-run MARK with invisible=FALSE
********Following model failed to run : p(~1)c() **********

And here's the error from the bottom of the MARK...out file,
ERROR -- Error with integer read of COVARIATES

A fix for this error may/may not solve the problem I was having when I started this post. Regardless, I though it made sense to start here...let me know if you need any more details.

*** Update: Jeff resolved this problem on 10-10-11. ***