restore models

[Karin]

Dear all,

I (or better R) calculated lots of CJS models which took quite a while. Unfortunately, after that R crashed. Now I don’t know how I can access those models. As expected collect.models() did not work after I started R again. Therefore I tried what was written on p. C-10 in the appendix:

> myexample=mark(name,filename="mark040")
Create mark model with existing file (Y/N)?Y

But I only got an error message (and model mark040 was not the model with constant Phi and p):

Note: only 47 parameters counted of 2 specified parameters
AICc and parameter count have been adjusted upward
Error in `row.names<-.data.frame`(`*tmp*`, value = c("Phi g1 c1 a0 t1", :
invalid 'row.names' length


********Following model failed to run : Phi(~1)p(~1) **********


Does anyone have a suggestion on how I can access the stored models? And is it possible to restore all at once?

Kind regards,
Karin

[dhewitt]

Technically R did not do any model calculations. MARK took care of that part. (Unless you are using the CJS routines that Jeff has built directly into RMark, of course.)

So what versions of R, RMark, and MARK are you using? And how many models are you talking about? I ask because R has literally never crashed on me when using RMark (and thus MARK) and fitting up to 50 big models with huge data sets.

Jeff may be able to help in tracking down which models are related to which output files, but I would suggest just re-running them to be safe.

- Dave Hewitt

[Karin]

I’m using R 2.10.1, RMark 1.9.6, and the one doing all the work: MARK 5.1.

I think the main problem is that I have so many models. I have 4 possible Factors influencing survival and recapture rates and as I want to find out which combination of factors describes my data best, I have several hundred models. But computing the models is not the problem, it just takes a long time. The problem is that after the last model has been successfully calculated, R crashes. So the models are all there, I just need to find a way to access them.

I had this problem once before and re-running all models ended the same way.

Karin

[cooch]

I’m using R 2.10.1, RMark 1.9.6, and the one doing all the work: MARK 5.1.

I think the main problem is that I have so many models. I have 4 possible Factors influencing survival and recapture rates and as I want to find out which combination of factors describes my data best, I have several hundred models. But computing the models is not the problem, it just takes a long time. The problem is that after the last model has been successfully calculated, R crashes. So the models are all there, I just need to find a way to access them.

I had this problem once before and re-running all models ended the same way.

Karin

Switch to MARK 6.0 (the current release), and try again.

Moreover, model sets consisting of 'hundreds of models' implies you might want to spend more time thinking hard about what models to include. In my experience, model sets that large are diagnostic of more 'dredging' than 'a priori inference.

[dhewitt]

Switch to MARK 6.0 (the current release), and try again.

Note that Jeff has not checked the RMark import features in the MARK 6.0 GUI yet (presumably still named mark_int.exe -- I haven't upgraded!). The interaction with mark.exe should be OK, but I am sure Jeff would be interested in reports of any issues.

Moreover, model sets consisting of 'hundreds of models' implies you might want to spend more time thinking hard about what models to include. In my experience, model sets that large are diagnostic of more 'dredging' than 'a priori inference.

Agreed. An interesting (and in some respects troublesome) thread on this topic recently kicked off on ESA's Ecolog listserv:

https://listserv.umd.edu/cgi-bin/wa?A2= ... D=1&P=1282
[I hope this link works. Note a link back to the Phidot forums at one point.]

[jlaake]

Sorry I didn't reply sooner. I've not been getting emails from the forum. I was wondering why the forum was so quiet.

The approach you took to restore the models should work but you need to specify the entire mark command that would create the model rather than just a name. Contact me offlist and we'll work out the issue. Not sure why R is crashing unless it is due to lack of memory. You may want to try using external=TRUE which stores the results external to the .Rdata workspace which reduces the memory requirements. Or you may want to try it with fewer models at once to see if that works first.

--jeff

[dhewitt]

[quote="dhewitt"][quote="cooch"]Switch to MARK 6.0 (the current release), and try again.[/quote]

Note that Jeff has not checked the RMark import features in the MARK 6.0 GUI yet (presumably still named mark_int.exe -- I haven't upgraded!). The interaction with mark.exe should be OK, but I am sure Jeff would be interested in reports of any issues.[/quote]

Just an FYI for RMark users wondering about an upgrade to MARK 6.0...

In a quick but decently thorough test run just now, I found that R 2.10.1 with RMark 1.9.6 worked great with MARK 6.0 (mark.exe = Feb 27 2010, mark_int.exe [GUI] = Feb 24 2010). I checked the RMark Import in the GUI and it worked as expected for a reasonably complex set of models.

[Karin]

Dear Jeff,

Thanks for your reply! Unfortunately, I didn’t get any mails from the forum either, and I noticed your message not before today. (It seems that I’m not permitted to send you an e-mail.)

I tried to specify the entire mark command (although I’m not sure if I didn’t miss something important), but I only got more error messages.

myexample=mark(name, ddl = name.ddl, begin.time = 1, model.name = "Phi(~age * Sex + Year)p(~1)", model = "CJS", title = "", model.parameters = list(Phi=list(formula=~age * Sex + Year),p=list(formula=~1)), covariates = NULL, initial = NULL, design.parameters = list(), right=TRUE, groups = c("Age","Sex"), age.var = 1, initial.ages = c(0.6,1.2,2), age.unit = 1/12, time.intervals = c(1,1,1,1,1,7,1,1,1,1,1,7,1,1,1,1,1,7,1,1,1,1,1), nocc=NULL, output = TRUE, invisible = TRUE, adjust = TRUE, mixtures = 1, se=TRUE, simplify=TRUE, filename="mark029", default.fixed=TRUE, silent=FALSE, retry=0, options=NULL, brief=FALSE, realvcv=FALSE, delete=FALSE, external=FALSE, profile.int=FALSE, chat=NULL)
Warning: specification of ddl ignored, as data have not been processed
Error in make.mark.model(data.proc, title = title, covariates = covariates, :

Error: Variable Year used in formula is not defined in data

The Factor Year was specified via the merge.design.covariates function and for “age” I used age bins. I don’t know whether (and how) they should be specified within the mark function as design.parameters or covariates.

“external=TRUE” should be used when I’m creating the original model? (in mark.wrapper ?)

Anyway, I’m not sure if addressing all the models exactly would solve my problem. As I mentioned before, I have several hundred models (I know that some may disagree with this approach). Therefore, I would prefer a method (if possible) to restore all models at once.

Thank you very much for your time!

Best wishes,
Karin

[cooch]

Dear Jeff,

Thanks for your reply! Unfortunately, I didn’t get any mails from the forum either, and I noticed your message not before today. (It seems that I’m not permitted to send you an e-mail.)
...

You didn't receive an email because in all likelihood you weren't subscribed to the RMark subforum. Have a look at

new.forum.instructions.pdf

(especially item 3).

And no, 'private messaging' through the board is not permitted - part of the various 'anti-spammer security layers' (i.e. if a spammer actually manages to register, the first thing they do is private message everyone on the board).