Individual Covariate testing - zero SE estimate

questions concerning analysis/theory using programs M-SURGE, E-SURGE and U-CARE

Individual Covariate testing - zero SE estimate

Postby simone77 » Fri Mar 07, 2014 7:25 am

Hi,

I am modelling a data set made of nine occasions with individuals observed as seropositive, seronegative or unknown serological status. Since I have found significant trap-dependence effect I am using the approach from Pradel & Sanz-Aguilar (2012) which considers for each underlying biological state (e.g. seropositive and seronegative) an extra one regarding the trap-awareness (aware or unaware).
Then, I have five states in the order they appear in GEPAT:
SN-U, SeroNegative trap Unaware
SN-A, SeroNegative trap Aware
SP-U, SeroPositive trap Unaware
SP-A Seropositive trap Aware)
and four events:
0, not captured
1, captured as SN
2, captured as SP
3, captured unknown serological status

My paramater types are:
IS (Initial States), Survival (Transition step 1), Seroconversion (Transition step 2), Capture (Transition step 3) (following Pradel & Sanz-Aguilar 2012), and Serological Determination (Event step 1). There are two groups (1= females, 2=males) and defined only to age classes (first interval after first capture vs all others).

I want to investigate the effect of an individual covariate (body mass) on seroconversion (i.e. transition from SN to SP and vice versa). Given that I have the body mass only for the first capture of each individual, I am going to use two age-classes to focus only in the first interval after first capture.

In a first step, I simplified the structure of Capture, Serological Determination and Initial State. Then I wanted to test the effect of body mass (standardized in the data set, mean=0 StDev=1) on Seroconversion. With this aim I compare two models differing only for the Seroconversion structure:

Phrase for Initial State t
* - p -

Phrase Survival g(1,2).f(1 2,3 4)
y - - - *
- y - - *
- - y - *
- - - y *
- - - - *

Model no body mass effect
Phrase for Seroconversion a.g(1,2).f(1 2,3 4)
* - p - -
- * - p -
p - * - -
- p - * -
- - - - *

Model Body mass effect on transition from SN to SP of males
Phrase for Seroconversion a(1).g(2).f(1 2).[i+xind]+a(1).g(2).f(3 4)+a(1).g(1).f(1 2,3 4)+a(2).g(1,2).f(1 2,3 4)

Phrase for Capture sex+f(1 3,2 4).t
p * - - -
p * - - -
- - p * -
- - p * -
- - - - *

Phrase for Serological Determination f+t
- p - *
* - - -
- - p *
* - - -
* - - -
Since in the fifth occasion no captured individual was bled, for that occasion I fixed this parameter to be equal to zero.

When the model with Body Mass effect terminates, I am asked to give a number of parameters and I give one more with respect to model without body mass, is this correct?
Deviance estimates make sense, however when I have a look at the beta estimates (model with Body Mass effect is run with C-I intervals activated), I found that the best for the Body Mass has zero SE. This happens also for Initial State 5th and 6th occasion. From previous model I have no serious concerns on parameter estimability. Any idea on what is going on?
simone77
 
Posts: 197
Joined: Mon Aug 10, 2009 2:52 pm

Re: Individual Covariate testing - zero SE estimate

Postby Guillaume Souchay » Mon Mar 17, 2014 9:53 am

Hi Simone,
When the model with Body Mass effect terminates, I am asked to give a number of parameters and I give one more with respect to model without body mass, is this correct?

Yes, it is correct (you add only 1 more parameter - the slope).

Deviance estimates make sense, however when I have a look at the beta estimates (model with Body Mass effect is run with C-I intervals activated), I found that the best for the Body Mass has zero SE. This happens also for Initial State 5th and 6th occasion. From previous model I have no serious concerns on parameter estimability. Any idea on what is going on?

I just have a naive question. what are the values for the CI? is it also zero?
Is the slope estimated at the boundary?

Guillaume
Guillaume Souchay
 
Posts: 64
Joined: Wed Feb 27, 2008 5:20 am
Location: Nantes, FRANCE

Re: Individual Covariate testing - zero SE estimate

Postby simone77 » Mon Mar 17, 2014 11:47 am

Guillaume Souchay wrote:Hi Simone,
When the model with Body Mass effect terminates, I am asked to give a number of parameters and I give one more with respect to model without body mass, is this correct?

Yes, it is correct (you add only 1 more parameter - the slope).

Deviance estimates make sense, however when I have a look at the beta estimates (model with Body Mass effect is run with C-I intervals activated), I found that the best for the Body Mass has zero SE. This happens also for Initial State 5th and 6th occasion. From previous model I have no serious concerns on parameter estimability. Any idea on what is going on?

I just have a naive question. what are the values for the CI? is it also zero?
Is the slope estimated at the boundary?

Guillaume



Hi Guillaume,

Thanks for your answer.
By checking the output file for the same model without individual covariate testing (the other one has only info on beta estimates), I have noticed that Initial State values at the 5th and 6th occasion are not estimable. It must be due to the fact that at the 5th occasion no individuals were state assessed (correspondent p in the IVFV has been set to zero).

Values for CI are identical to the point estimates in all these cases.

How could I know if the slope associated to body mass is at the boundary? there is no information on which parameter is at the boundary from the output of a model with individual covariate. I have not yet gone deeper into applying the delta method to back transform this beta value in order to get a representation of the effect of body mass on the specific transition parameter. By the way, any reference on how to do it in R is very welcome from me and, probably, other E-Surge users.

I have nine case studies, each one with a different data set and I have seen that this happens in most of but not all the cases. To add details or confusion: I have also noticed that body mass slope values yielded by using different Non-linear solvers (Quasi-Newton, EM, and PEM) can be quite different (e.g. respectively: -77.2, -38.5, -56.8). This is not the case for the other betas.

Simone
simone77
 
Posts: 197
Joined: Mon Aug 10, 2009 2:52 pm

Re: Individual Covariate testing - zero SE estimate

Postby Guillaume Souchay » Mon Mar 17, 2014 12:15 pm

Hi Simone,

By checking the output file for the same model without individual covariate testing (the other one has only info on beta estimates),

The Output file from a model with individual covariate only included the estimation of the beta. The beta are the estimated parameters and the biological parameters are only backtransformation of the beta. Thus, even if there is only beta, you have all the information you need :)

How could I know if the slope associated to body mass is at the boundary? there is no information on which parameter is at the boundary from the output of a model with individual covariate.

A parameter estimated at the boundary will have either the value of 0 or 1 on the biological scale, thus either -36 or +36 on R scale. When a parameter is estimated at these values, it's possible that the SE is not calculated. Remi could say more about that.

Usually, you have some information about the number of parameters estimated at the boundary in the output text file, with the information of the number of mathematical/estimated parameters. However, you don't have all the parameters involved.

By the way, any reference on how to do it in R is very welcome from me and, probably, other E-Surge users.

Concerning the Delta Method, you can find lots of information in the MARK BOOK, which is the link between a program manual and an handbook of CMR method. See Appendix B about the Delta Method, section B.4. Cooch and White provided several examples of back-transformation => http://www.phidot.org/software/mark/docs/book/pdf/app_2.pdf

I have nine case studies, each one with a different data set and I have seen that this happens in most of but not all the cases. To add details or confusion: I have also noticed that body mass slope values yielded by using different Non-linear solvers (Quasi-Newton, EM, and PEM) can be quite different (e.g. respectively: -77.2, -38.5, -56.8). This is not the case for the other betas.

My first guess would be related to local minima. Using different solvers should lead to different solutions but to different method to achieve the unique convergence point.
Another care is to standardize the individual covariate before using it within E-SURGE.

Did you run your model without ind covariate several time with the Multiple-Random option?
Did your model well converge when you found these different values for the slope? Was the optimization achieved rather than the number of iterations/function value was exceeded? (information from the DOS-Windows).

A good way is to run several time your model without the body mass covariate, and to use the result from this best model as the Initial Value for your model with the covariate: After running your model without the cov, use the "from Last Model" option for IVFV, include the covariate in GEMACO, fixed the parameters, if needed (be sure to be on the biological scale!) and run your model.

I just noticed this point:
From previous model I have no serious concerns on parameter estimability.

Is your model full-ranked? Do you have some identifiability issues related to the parameters with ind covariate?

Good luck

Guillaume
Guillaume Souchay
 
Posts: 64
Joined: Wed Feb 27, 2008 5:20 am
Location: Nantes, FRANCE

Re: Individual Covariate testing - zero SE estimate

Postby simone77 » Wed Mar 19, 2014 6:34 am

Hi Guillaume,

Thank you again for your willingness to help. Below there are answer to your comments and some other detail on this case study. In italic are my previous commentaries.

By checking the output file for the same model without individual covariate testing (the other one has only info on beta estimates),
Guillaume Souchay wrote:The Output file from a model with individual covariate only included the estimation of the beta. The beta are the estimated parameters and the biological parameters are only backtransformation of the beta. Thus, even if there is only beta, you have all the information you need :)

Yes, I wasn't clear enough, I was just referring to the fact that for model with individual covariate there is no info on which parameters are not identifiable and other details.

How could I know if the slope associated to body mass is at the boundary? there is no information on which parameter is at the boundary from the output of a model with individual covariate.
Guillaume Souchay wrote:A parameter estimated at the boundary will have either the value of 0 or 1 on the biological scale, thus either -36 or +36 on R scale. When a parameter is estimated at these values, it's possible that the SE is not calculated. Remi could say more about that.

R scale refers to the mathematical parameter scale, isn't it? You are right I should have checked it as the first thing. As said in the previous post, my estimates (from different non-linear solver) on the mathematical scale are far beyond -36 and it does mean they are at the boundary. The strange thing, and this is why I haven't paid much attention to the boundary issue, is that the QAICc and Deviance of the model with the individual covariate is much better than the model without. I would believe that if deviance is much better, it is because body mass is absorbing deviance and this is not very coherent with the slope of body mass being very very close to zero (as it would be my case).

Guillaume Souchay wrote:Usually, you have some information about the number of parameters estimated at the boundary in the output text file, with the information of the number of mathematical/estimated parameters. However, you don't have all the parameters involved.



By the way, any reference on how to do it in R is very welcome from me and, probably, other E-Surge users.
Guillaume Souchay wrote:Concerning the Delta Method, you can find lots of information in the MARK BOOK, which is the link between a program manual and an handbook of CMR method. See Appendix B about the Delta Method, section B.4. Cooch and White provided several examples of back-transformation

Thanks for the tip, I forgot about that chapter in MARK manual (read quite a long time ago): it is clear enough to be easily implemented in R (or even in Excel itself).

I have nine case studies, each one with a different data set and I have seen that this happens in most of but not all the cases. To add details or confusion: I have also noticed that body mass slope values yielded by using different Non-linear solvers (Quasi-Newton, EM, and PEM) can be quite different (e.g. respectively: -77.2, -38.5, -56.8). This is not the case for the other betas.
Guillaume Souchay wrote:My first guess would be related to local minima. Using different solvers should lead to different solutions but to different method to achieve the unique convergence point. Another care is to standardize the individual covariate before using it within E-SURGE.
Did you run your model without ind covariate several time with the Multiple-Random option?
Did your model well converge when you found these different values for the slope? Was the optimization achieved rather than the number of iterations/function value was exceeded? (information from the DOS-Windows).

Answering this:
1. Yes, Individual covariates were standardized before running the model (mean=0, SE=1).
2. Yes, I run the model without individual covariate (also the one with individual covariate) by checking "Multiple Random" initial values = 6 (made some trial before and saw this was a value big enough to get the same value of deviance).
3. Yes, my model converged without any at least apparent problem either by using Quasi-Newton, EM or PEM.
4. Regarding last question, I didn't pay to much attention to this because in my experience when some problem occurs with that, a pop up window advises you. However, I have seen that in many cases (from the MS-DOS window) the optimization was reached at the 201th iteration (the cycle is set on default value=200) and then started the next cycle (remember I had Multiple Random activated). I have been doing some trial by excluding a few individuals for which the body mass was a clear outlier (likely due to field work errors) and saw that the slope is a bit better from a boundary issue perspective, i.e. -39.4 vs prior -77.2 (Quasi-Newton). However, it is still beyond the -32 limit you mentioned for the Logit link estimates, and still have SE=0. On the other hand, I have seen that last iteration used to be around the 90th, far before the 200th.
Do you believe the number of iteration at which model stops running is an important point to be considered, also in case no warning is given?

Guillaume Souchay wrote:A good way is to run several time your model without the body mass covariate, and to use the result from this best model as the Initial Value for your model with the covariate: After running your model without the cov, use the "from Last Model" option for IVFV, include the covariate in GEMACO, fixed the parameters, if needed (be sure to be on the biological scale!) and run your model.

In the past I have never been lucky with the "from last model" option to get rid of local minima. However, I have tried to make what you suggest but the deviance I get results to be higher than the model without the individual covariate (should be equal or less than that). It doesn't seem to work in my case.
Guillaume Souchay wrote:I just noticed this point: From previous model I have no serious concerns on parameter estimability.
Is your model full-ranked? Do you have some identifiability issues related to the parameters with ind covariate?

Considering the model without the individual covariate: it has three parameters that are not identifiable which are two Initial State (for occasion 5th and 6th as mentioned in the previous post) and one that is quite logic since it refers to the probability a previously captured individual is seen at occasion 1th. I have tried to fix it to zero and nothing changes except this parameter is no more in the parameter estimability list.
From the *.out file of this model results that:
No. of mathematical parameters: 46
Estimated model rank applicable to the data: 44
Estimated No. of boundary parameters: 2

About the model with individual covariate: this is the problem I was referring to in the first point, i.e. as far as I know there is no info about which specific parameters are not estimable. However, from the *out file results that:
No. of mathematical parameters: 46
Estimated model rank applicable to the data: 47
Estimated No. of boundary parameters: 4

Really I would have thought it has only one more parameter than the model without the individual covariate and therefore I manually modified no. of parameters of individual covariate model to be 45.

Then, in conclusion I have not yet a clear idea where the problem is coming from. The strangest thing for me is that the individual covariate model would seem to perform consistently better but still the apparent slope for body mass is very close to zero boundary (no effect at all).

Simone
simone77
 
Posts: 197
Joined: Mon Aug 10, 2009 2:52 pm

Re: Individual Covariate testing - zero SE estimate

Postby CHOQUET » Wed Mar 19, 2014 8:24 am

Do you have standardised our individual covariates ?
CHOQUET
 
Posts: 211
Joined: Thu Nov 24, 2005 4:58 am
Location: CEFE, Montpellier, FRANCE.

Re: Individual Covariate testing - zero SE estimate

Postby simone77 » Wed Mar 19, 2014 8:50 am

Hi Remi,

Yes, I have.
simone77
 
Posts: 197
Joined: Mon Aug 10, 2009 2:52 pm

Re: Individual Covariate testing - zero SE estimate

Postby Guillaume Souchay » Wed Mar 19, 2014 10:22 am

Hi Simone,

R scale refers to the mathematical parameter scale, isn't it? You are right I should have checked it as the first thing. As said in the previous post, my estimates (from different non-linear solver) on the mathematical scale are far beyond -36 and it does mean they are at the boundary.

Yes, the R scale is ]-inf;+inf[ and refers to mathematical parameter scale.
The value of |36| is just an indication, because when you backtransformed the value, it gives you 2.3e-19 and 1 on the biological scale for -36 and 36 respectively.
Thus, values far beyond -36 are extremely close to 0. maybe this is why the SE can't be estimated, it's too close of 0. When an estimate is at the boundary, the SE can't be estimated, however, Remi needs to confirm this.

Regarding last question, I didn't pay to much attention to this because in my experience when some problem occurs with that, a pop up window advises you.

No. when a saddle point is discovered, yes there is a warning message, but if the solver stopped and due to the number of iterations, you can be on a point that is not a saddle point and thus, no warning.

I have seen that in many cases (from the MS-DOS window) the optimization was reached at the 201th iteration (the cycle is set on default value=200) and then started the next cycle (remember I had Multiple Random activated).

When the solver reached the maximal number of iterations, the solver stops, keeps the deviance of the last iteration but it doesn't mean that convergence was achieved ... When you used the Multiple Random option, then a new cycle with new initial values starts, and the solver runs again.
Sometimes, you need more than 200 iterations for your model to converge. So, you can either increase the number of iterations in the Advanced Numerical box, or you can select the "continue after n cycle" rather than "Stop after 1 cycle" in the Convergence option in the same box.

I have seen that last iteration used to be around the 90th, far before the 200th.
Is this for your model with or without the cov?

Really I would have thought it has only one more parameter than the model without the individual covariate and therefore I manually modified no. of parameters of individual covariate model to be 45.

The model with the covariate has only 1 more parameter than the model without. Thus, the number of mathematical parameter is 46+1=47. For the rank, it should be the same: 44 (rank of the model wihtout cov)+1=45.

I have one more question: perhaps this is about it, perhaps not. What is the distribution of your covariate (non-standardized)? I am wondering if all values are closed to each other, maybe, there is not enough variation, and so the slope can't be different from zero, and then, the SE can't be estimated.
However, this is just an open question.

Cheers,

Guillaume
Guillaume Souchay
 
Posts: 64
Joined: Wed Feb 27, 2008 5:20 am
Location: Nantes, FRANCE

Re: Individual Covariate testing - zero SE estimate

Postby simone77 » Wed Mar 19, 2014 11:36 am

Again, in italic my previous comments.

R scale refers to the mathematical parameter scale, isn't it? You are right I should have checked it as the first thing. As said in the previous post, my estimates (from different non-linear solver) on the mathematical scale are far beyond -36 and it does mean they are at the boundary.
Guillaume wrote:Yes, the R scale is ]-inf;+inf[ and refers to mathematical parameter scale.
The value of |36| is just an indication, because when you backtransformed the value, it gives you 2.3e-19 and 1 on the biological scale for -36 and 36 respectively.
Thus, values far beyond -36 are extremely close to 0. maybe this is why the SE can't be estimated, it's too close of 0. When an estimate is at the boundary, the SE can't be estimated, however, Remi needs to confirm this.


Regarding last question, I didn't pay to much attention to this because in my experience when some problem occurs with that, a pop up window advises you.
Guillaume wrote:No. when a saddle point is discovered, yes there is a warning message, but if the solver stopped and due to the number of iterations, you can be on a point that is not a saddle point and thus, no warning.

Yes, I believe to remember that sometimes I have seen a pop up window asking you if you want to continue with iteration since the maximum number of iteration has been reached. I didn't refer to a warning again saddle point.

I have seen that in many cases (from the MS-DOS window) the optimization was reached at the 201th iteration (the cycle is set on default value=200) and then started the next cycle (remember I had Multiple Random activated).
Guillaume wrote:When the solver reached the maximal number of iterations, the solver stops, keeps the deviance of the last iteration but it doesn't mean that convergence was achieved ... When you used the Multiple Random option, then a new cycle with new initial values starts, and the solver runs again.
Sometimes, you need more than 200 iterations for your model to converge. So, you can either increase the number of iterations in the Advanced Numerical box, or you can select the "continue after n cycle" rather than "Stop after 1 cycle" in the Convergence option in the same box.

When stopping at the 201th iteration I have noticed it was giving the same deviance from quite a few lines before (around 30 or more) so I wouldn't expect it would jump to another minimum from there. However I will give it a try tonight to see if something changes.

(...talking about model without outliers)I have seen that last iteration used to be around the 90th, far before the 200th.
Guillaume wrote:Is this for your model with or without the cov?

With the individual covariate.

Really I would have thought it has only one more parameter than the model without the individual covariate and therefore I manually modified no. of parameters of individual covariate model to be 45.
Guillaume wrote:The model with the covariate has only 1 more parameter than the model without. Thus, the number of mathematical parameter is 46+1=47. For the rank, it should be the same: 44 (rank of the model wihtout cov)+1=45.

Think the same.

Guillaume wrote:I have one more question: perhaps this is about it, perhaps not. What is the distribution of your covariate (non-standardized)? I am wondering if all values are closed to each other, maybe, there is not enough variation, and so the slope can't be different from zero, and then, the SE can't be estimated.
However, this is just an open question.

Yes, this is the first came to my mind, something strange in the individual covariate distribution but there is nothing excessively strange in it, just a bit left-skewed but I don't believe it can be causing the problem.
Image
So, the strange thing to me is that even though the model seems to fit the data much better than the other (the deviance is much better and the DeltaQAICc is around 10), the slope for the body mass is so close to zero.
simone77
 
Posts: 197
Joined: Mon Aug 10, 2009 2:52 pm

Re: Individual Covariate testing - zero SE estimate

Postby Guillaume Souchay » Wed Mar 19, 2014 12:02 pm

Hi Simone,

thanks for the reply and the clarification.
Based on what you said, it seemed that the model converged but with a slope of 0 for the body mass and a decreased in Deviance.

About the deviance being the same over 30 iterations, sometimes, the deviance can really decrease after several other iterations. But, it's not a problem for your model with the covariate anyway.

It is really strange ...
I'm quite lacking other ideas ...

Did you try to fix the slope to a value of 0.05 to see if the model converges to the same deviance or not? To fix a parameter estimate at the boundary can help to check if this parameter is really estimated or not, and if indeed estimated, by looking at different fixed values of this parameter, you can estimate the 95%CI.

I also have just a suggestion about the modelling with the covariate. You could also try to include an individual random effect in the model to relax a little the strength of the relationship between the survival and to get a estimated of the residual variance.
You could also see if the deviance is still decreasing or being the same than previous models.
Guillaume Souchay
 
Posts: 64
Joined: Wed Feb 27, 2008 5:20 am
Location: Nantes, FRANCE

Next

Return to analysis help

Who is online

Users browsing this forum: No registered users and 13 guests