MARK v. 9.0 | now in full release...

announcements (new versions, changes, bugs, installation problems...) related to program MARK

MARK v. 9.0 | now in full release...

Postby cooch » Mon Apr 02, 2018 9:25 pm

After a fair bit of 'behind the curtain' testing, Gary has announced the full, general release of MARK v. 9.0 (upgrading from v. 8.2, which was the last release before 9.2). Transition from 'Hawaian goose' to 'black bellied whistling duck' on the splash screen. Here are the major enhancement as per Gary's website - the key one (for most users) will be 282' -- major changes/improvemenets to how parameters are counted. Beyond the text (below), these changes are documented (and demonstrated) in the addendum to Chapter 4 in the book.

Personal observation: upgrading to v. 9 will potentially (occasionally) change the number of estimated parameters for analysis of messy data sets with lots of parameters, so you're advised to watch for these possible changes if you re-run models you previously ran with older versions of MARK (especially for models based on a DM, or where you didn't use a sin link). Having said that, v. 9.0 is designed to do a 'better job' at handling parameter counting, so if you find a difference, take that under advisement.

April, 2018. Version 9.0 — Black-bellied Whistling Ducks

282. The numerical computation of the first and second derivatives has been tweaked to improve accuracy.

282. A major change in estimating the number of parameters that were estimated in a model has been implemented. Two methods are now used. First, a numerical threshold is estimated from the gradient (G) vector as 2 times the maximum absolute value in the gradient. This numerical threshold is then used to determine the number of values in the singular-value decomposition (S) vector that exceed the numerical threshold, with this value taken as the number of parameters estimated. Second, the S vector is searched for the largest ratio S(i)/S(i + 1) between 2 consecutive values, as well as the next largest ratio between 2 consecutive values. If the ratio is >50, the index of the numerator for the maximum ratio is taken as the number of parameters estimated. When both of these estimates agree, all is well. If the 2 estimates disagree, the maximum of the 2 is reported as the number of parameters estimated, and a warning is printed in the full output that the 2 estimates disagree. An option has been provided in the File | Preferences menu choice to make this warning very explicit. The model name has the phrase “Check Par. Cnt.” added to the front of the name, and the model name is shown in blue in the Results Browser. The user should then check the full output to see if the estimate reported is reasonable, or if the number of parameters estimated should be changed. Once an appropriate value is set, the blue coloring can be eliminated by clicking on the model name and deleting the phrase “Check Par. Cnt.”. Unfortunately, neither of the 2 methods can detect that a parameter estimated at its boundary should be counted, e.g., p-hat = 1 with a logit link, or pent-hat = 0 with a MLogit link. Improved numerical precision of the derivatives just made this problem worse. Users should use the sin link when possible to detect parameters estimated at the boundary.
cooch
 
Posts: 1628
Joined: Thu May 15, 2003 4:11 pm
Location: Cornell University

Re: MARK v. 9.0 | now in full release...

Postby egc » Mon Apr 02, 2018 10:03 pm

Linux and OS/X versions to follow in the next few days.
egc
Site Admin
 
Posts: 201
Joined: Thu May 15, 2003 3:25 pm

Re: MARK v. 9.0 | now in full release...

Postby egc » Wed Apr 04, 2018 9:36 am

Linux version now compiled and online -- details and links here:

http://www.phidot.org/software/mark/rmark/linux/

OS/X version sometime in the near future.
egc
Site Admin
 
Posts: 201
Joined: Thu May 15, 2003 3:25 pm

Re: MARK v. 9.0 | now in full release...

Postby nesmonde » Wed Apr 13, 2022 7:36 am

Hi,

I'm a little unsure on how to choose a reasonable reported estimate when Check Par. Cnt. shows up. Can you clarify this or is there a rule of thumb when the parameter estimates in gap method and threshold method differ?

Thank you.
nesmonde
 
Posts: 7
Joined: Tue Nov 02, 2021 4:29 pm

Re: MARK v. 9.0 | now in full release...

Postby gwhite » Wed Apr 13, 2022 8:59 am

Your best strategy is to examine the beta estimates. You will have to look for parameters on the boundary that were estimated, but causing problems with actually being counted as estimated.
gwhite
 
Posts: 329
Joined: Fri May 16, 2003 9:05 am

Re: MARK v. 9.0 | now in full release...

Postby cooch » Wed Apr 13, 2022 5:41 pm

Its important to remember that what you're asking about relates to extrinsically nonidentifiable parameters (i.e., poorly estimated parameters, either because of 'problems' with the data, or simply because the parameter is so close to a 0 or 1 boundary that you can't reasonably estimate it). The 'warnings' MARK gives you tells you to be suspicious of the overall parameter count, which if there is a problem, reflects an 'undercount' (i.e., MARK will never report more parameters than are structurally possible -- if there is an 'error' in the reported count, it will be because MARK estimates too few parameters...).

This leads to 2 considerations:

1\ the need to manually adjust the number of parameters 'up' to what should be identifiable given the structure of the model. This is important for model selection, since the default AIC criterion uses a penalty that is a simple function (2K) of the number of conceptually estimable parameters (K).

2\ what to report? If there is extrinsic nonidentifiability, you can first try confirming that the problem is 'data' by looking at data cloning (Appendix F). If its 'data', or if the estimate really is at either boundary, there isn't much you can do -- I usually tell people to simply not report that estimate (if you're reporting from a single model), or report it with caveats. The model-averaging that MARK does will generally 'skip' problem estimates, but you can always handle this manually.
cooch
 
Posts: 1628
Joined: Thu May 15, 2003 4:11 pm
Location: Cornell University

Re: MARK v. 9.0 | now in full release...

Postby shannonbarbermeyer » Thu May 05, 2022 7:28 pm

Would any of these changes affect POPAN results (and some other model formulation results) given in Chapter 12 - even after manually correcting for number of parameters estimated? So far, when working through chapters 1-7 I am generally able to match results almost exactly (if not exactly depending on estimation methods) - but in Chapter 12, I am finding a number of results that don't match my outputs. I have gone through the exercises several times and in different ways (PIM chart, PIM matrix, various links - sin, etc.) to make sure I'm doing exactly what they did - but sometimes the AIC (even after fixing parameters per authors' instructions) is off. For examples, in Ch 12 (version downloadable by individual chapters - not dated though but just downloaded it recently) on page 12-23 I am getting AICc ~941.938 for the last model listed whereas they got 940.088 (even after any No Par adjustments are made). Similarly, slightly different results on page 12-26 in the PARM-SPECIFIC link function image. And similar AICc differences on page 12-28 in the results browser even after adjusting for number of parameters to match the authors' numbers. On page 12-40 the AICc I got for their 4th model was 4912 - they got 5030. This was the biggest difference. None of these appreciably changed the main take home results (model with the most support) - but I was curious if the POPAN, etc. versions in MARK that the authors used when they wrote this Chapter maybe differ enough to cause these discrepancies (I want to minimize the probability that I'm not doing something wrong - but I can't find it if I am). Thank you!
shannonbarbermeyer
 
Posts: 24
Joined: Tue Aug 05, 2008 1:32 pm
Location: Wailuku, HI

Re: MARK v. 9.0 | now in full release...

Postby cooch » Thu May 05, 2022 8:23 pm

Not a clue -- I had nothing to do with the POPAN chapter. That was a Carl Schwarz production number. Carl has retired, so I rather doubt he'd be inclined to go through everything with newer versions of MARK.

My generic advice is - the newer version of MARK is more rigorous wrt handling some parts of the numerical optimization. If the the results are largely the same as those in Chapter 12, don't fuss about it. AIC differences? Don't start there -- look at -2ln(L) first. If its the same, then the difference in AIC is because new MARK does a better job counting parameters. And if number of parameters changes, then so will AIC>
cooch
 
Posts: 1628
Joined: Thu May 15, 2003 4:11 pm
Location: Cornell University

Re: MARK v. 9.0 | now in full release...

Postby shannonbarbermeyer » Thu May 05, 2022 9:50 pm

Thank you - yes, in all cases I had adjusted all the number of parameters to what the authors had reported so the number of parameters should not affect the comparison of their vs. my AICc results. Unfortunately, for most of the cases they did not include their -2ln(L) results, so I generally only had AICc comparisons to go on. In one case where they did report the -2ln(L) (page 12-29), I had different AICc (even after correcting parameter count to what they indicated it should be) although I had the same -2ln(L)...so that made me think something else beyond parameter count must be going on to create these differences, but I didn't know what (or perhaps they reported a different parameter count than they actually used to generate the results they reported in some of these cases). Anyhow - sounds like in general this is probably nothing for me to worry about as long as I'm understanding and following their overall procedures. Thank you again for your help!
shannonbarbermeyer
 
Posts: 24
Joined: Tue Aug 05, 2008 1:32 pm
Location: Wailuku, HI


Return to software problems/news

Who is online

Users browsing this forum: No registered users and 6 guests