negative degrees of freedom and Goodness of fit testing

questions concerning analysis/theory using program MARK

Re: negative degrees of freedom and Goodness of fit testing

Postby gwhite » Thu Nov 22, 2018 1:37 pm

After seeing the input data and models, I've figured out what is going on. All the animals were released on the first occasion, so that means there are 2^5 = 32 possible encounter histories. Only 10 of these were observed because p=1 for the first 2 occasions. And for most of the 4 groups, fewer were observed.

To keep this simple, consider just a single group with p=1 for each occasion. MARK computes the DF of the data by assuming a multinomial distribution for each cohort. Further assume p = 1, so only 6 histories are possible, giving 5 DF. The global model of {phi(t) p(t)} has 9 parameters if you let p be estimated. Thus, 5 - 9 = -4, and negative DF for the deviance. But this situation is more subtle than that given that we are interested in GOF. What you actually have are 5 binomial distributions, i.e., animals that are alive at each occasion either survive or not for 5 occasions. There is no GOF possible because you have fit a saturated model. All 5 DF in the data are estimated by the 5 parameters.

So for the actual data with all releases at occasion 1, there were 10 observed values over 4 groups. Given that the DF for the multinomial is 1 less than the number of cells in the multinomial distribution, there there are 9 DF for each group, except that not all groups had each of the 10 observed encounter histories. But to make this simple again, assume that each group contributes 9 DF, giving 36 DF from the data. Then when you fit the {phi(g*t) p(g*t)} global model, you have 36 parameters. So you would have 0 DF for GOF, again because of p=1 removes DF from the data.

The result is that you are not fitting a saturated model like described above for all occasions with p=1, but you are coming close. The negative DF is telling you that you have no DF for GOF. Were you to run the data one group at a time, you might have 1 or more of the groups with a GOF test, but the rest would be saturated models with no GOF possible.

In summary, if you really have data with p=1, then thinking about the data as a known fate model makes it clear that you are fitting a saturated model. For the actual data here, you have some hybrid between the CJS and known fate model.

Gary
gwhite
 
Posts: 329
Joined: Fri May 16, 2003 9:05 am

Re: negative degrees of freedom and Goodness of fit testing

Postby aswea » Thu Nov 22, 2018 5:57 pm

Thank you for the response and for writing so clearly. And thanks Evan for reformatting those m-arrays.

What I mostly need now is advice on a defensible course of action. Can I just say that a GoF test was not possible because there were no DoF left after fitting the most general model? Can I then proceed as usual to use AIC to compare the competing hypotheses with c-hat=1? Or do I run GoF tests on some of the less parameterized models in the set and use one of those values? The Gentle Intro recommends testing how the model ranking changes with c-hat so I’ve done that.

To summarize the answers so far:

• Are the degrees of freedom (DoF) for the saturated model the number of unique capture histories (in the dataset, not the number possible) minus 1? Answer= “Yes, Mark computes the DF by assuming a multinomial distribution for each cohort. The DF for the multinomial is 1 less than the number of cells in the multinomial distribution.”

• Does this mean that even very large datasets can have models where the DoF is negative if the recapture or survival probabilities are 100% for some intervals? Answer= “Yes. If p=1 then should use a known fate model. If p=1 sometimes, then have some hybrid between CJS and known fate”.
aswea
 
Posts: 27
Joined: Sat Oct 17, 2009 3:32 pm
Location: Gander NL

Re: negative degrees of freedom and Goodness of fit testing

Postby gwhite » Thu Nov 22, 2018 10:04 pm

You are going to get the same result whether you just assume c = 1, or try to detect lack of fit with the meager amount of data in the last few occasions of a few of the groups. Detecting lack of fit requires lots of data.

Gary
gwhite
 
Posts: 329
Joined: Fri May 16, 2003 9:05 am

Re: negative degrees of freedom and Goodness of fit testing

Postby aswea » Mon Nov 26, 2018 9:10 am

In case it is useful in future, there's a section "Goodness of fit and known fate models" in the Gentle Intro that is relevant to this thread.
aswea
 
Posts: 27
Joined: Sat Oct 17, 2009 3:32 pm
Location: Gander NL

Re: negative degrees of freedom and Goodness of fit testing

Postby dpeters » Tue Aug 16, 2022 10:37 am

Hi Gary,

I've run into something similar with a CJS analysis with salmon smolts. There are a large number of releases (see below), there are decent numbers of recaptures (detections), and the real pararmeter estimates are reasonable. However, GOF testing is problematic because of 0 Deviance df. Perhaps this is because not all the possible capture histories are observed? Or that the data have some removals at recapture (barging in this case)?

There are six "groups" and three occasions (including release) and I'm trying to test GOF for the global model phi(g*t) p(g*t).

Code: Select all
     Live Encounters

   Group 1 CIRC1
Occ.  R(i)    j= 2     3 Total
--- ------   ----- ----- -----
  1   3572    1096   106  1202
  2   1030           161   161
  3    191                   0

   Group 2 CIRC2
Occ.  R(i)    j= 2     3 Total
--- ------   ----- ----- -----
  1   3561    1355   118  1473
  2   1295           179   179
  3    229                   0

   Group 3 CIRC3
Occ.  R(i)    j= 2     3 Total
--- ------   ----- ----- -----
  1   3529    1207   118  1325
  2   1163           167   167
  3    220                   0

   Group 4 2B
Occ.  R(i)    j= 2     3 Total
--- ------   ----- ----- -----
  1   1197     527    83   610
  2    462           113   113
  3    121                   0

   Group 5 4B
Occ.  R(i)    j= 2     3 Total
--- ------   ----- ----- -----
  1   1195     464    89   553
  2    417            97    97
  3    127                   0

   Group 6 6B
Occ.  R(i)    j= 2     3 Total
--- ------   ----- ----- -----
  1   1189     483    82   565
  2    437           100   100
  3    114                   0



From reading the thread, I gather one approach is to fit a model one group at a time and do a GOF on that?

Thanks!

Doug
dpeters
 
Posts: 7
Joined: Thu Dec 12, 2019 1:06 pm

Re: negative degrees of freedom and Goodness of fit testing

Postby gwhite » Tue Aug 16, 2022 1:37 pm

With only 3 occasions, you don't have enough degrees of freedom to test GOF. It is the same situation as a straight survival estimate based on the binomial distribution. No matter how many animals, there are no df left for testing after you estimate S.
gwhite
 
Posts: 329
Joined: Fri May 16, 2003 9:05 am

Re: negative degrees of freedom and Goodness of fit testing

Postby dpeters » Wed Aug 17, 2022 12:15 pm

Thanks Gary!

The Mij array I posted represents a "collapsed" version of this dataset where the focus was estimating apparent survival between release and the first recapture. The final occasion is actually a composite of whether it was detected at any one of a subsequent set of PIT-tag antenna arrays.

I could expand the encounter history to show those individually occasions and perhaps provide an avenue for GOF testing. However, I'm not sure that would be the same thing or even valid for the model fit to the collapsed data.

cheers, Doug
dpeters
 
Posts: 7
Joined: Thu Dec 12, 2019 1:06 pm

Previous

Return to analysis help

Who is online

Users browsing this forum: No registered users and 13 guests

cron