Overlapping confidence intervals POPAN

questions concerning analysis/theory using program MARK

Overlapping confidence intervals POPAN

Postby j.harv3y » Sun Mar 22, 2020 7:33 am

Hi again,

I've run a POPAN model, and the most parsimonious model has pent varying by time and group covariate (injury).

All looks great (reasonable SE's, not overlapping between the two groups) until I look at the confidence intervals. There is large overlap between injured vs non-injured at each year.

What can I conclude from this? It has me a little stumped. I've seen a couple of published papers where they state that overlapping confidence intervals means there's no difference between the groups, is this a reasonable conclusion to make? I'd expect the SE to be much larger? Should I not have allowed pent to vary across group covariate in the first place, and just kept it as varying across time?

Pent output is as follows;

year injuries est se uci lci
2014-15 major 0.06 0.03 0.02 0.16
2015-16 major 0.09 0.03 0.04 0.17
2016-17 major 0.06 0.02 0.03 0.13
2017-18 major 0.04 0.02 0.01 0.10
2018-19 major 0.04 0.02 0.02 0.10
2014-15 none 0.14 0.06 0.06 0.31
2015-16 none 0.22 0.06 0.12 0.37
2016-17 none 0.15 0.05 0.07 0.29
2017-18 none 0.10 0.04 0.04 0.22
2018-19 none 0.11 0.04 0.05 0.23

Thanks a million!
j.harv3y
 
Posts: 45
Joined: Mon Oct 08, 2018 4:45 am

Re: Overlapping confidence intervals POPAN

Postby cooch » Sun Mar 22, 2020 8:58 am

In simple terms (meaning, the purists - and Bayesians ;-) - might object about large conceptual details), if two statistics have non-overlapping confidence intervals, they are necessarily significantly different but if they have overlapping confidence intervals, it is not necessarily true that they are not significantly different.

This can be demonstrated by a simple example. Suppose we are interested in comparing means from two independent samples. The mean of the first sample is 9 and the mean of the second sample is 17. Let’s assume that the two group means have the same standard errors, equal to 2.5. The 95 percent confidence interval for the first group mean can be calculated as: 9 +/- 1.96 x 2.5, where 1.96 is the critical t-value. The confidence interval for the first group mean is thus (4.1, 13.9). Similarly for the second group, the confidence interval for the mean is (12.1, 21.9). Notice that the two intervals overlap. However, the t-statistic for comparing two means is:

t=\displaystyle\frac{17-9}{\sqrt{2.5^2+2.5^2}}=2.26

which reflects that the null hypothesis, that the means of the two groups are the same, should be rejected at the alpha = 0.05 level. To verify the above conclusion, consider the 95 percent confidence interval for the difference between the two group means: (17-9) +/- 1.96 x sqrt(2.5^2+2.5^2), which yields (1.09, 14.91). The interval does not contain zero, hence we reject the null hypothesis that the group means are the same.

So, overlapping CI's is not a perfect diagnostic about lack of a significant difference.
cooch
 
Posts: 1628
Joined: Thu May 15, 2003 4:11 pm
Location: Cornell University

Re: Overlapping confidence intervals POPAN

Postby j.harv3y » Sun Mar 22, 2020 10:53 am

Hi Cooch,
Amazing thanks so much for explaining! Was just concerned I was missing something obvious and shouldn't be using group covariates in pent. All makes sense :)
Thanks so much again!
j.harv3y
 
Posts: 45
Joined: Mon Oct 08, 2018 4:45 am

Re: Overlapping confidence intervals POPAN

Postby cooch » Sun Mar 22, 2020 7:13 pm

j.harv3y wrote:Hi Cooch,
Amazing thanks so much for explaining!


No worries -- a certain virus has given me a bit more time than usual...

Was just concerned I was missing something obvious and shouldn't be using group covariates in pent. All makes sense :)
Thanks so much again!


Your question was largely about whether overlapping CI's was (always) diagnostic about non-siginficance. As per my reply, that isn't the case. [This is a relatively common question, so I gave the answer a bit more substance...]

Using group covariates for pent is a different issue. The following (from the MARK helpfile) might be informative (keeping in mind that you can code 'group' using an individual covariate):

One limitation of the POPAN data type is with the use of individual covariates. Because the super-populations size (N) estimates the number of animals never captured, this parameter includes animals for which the individual covariate is not known. Thus, modeling N as a function of individual covariates is inappropriate. Further, the B(i) and N(i) parameters are functions of N, as well at the pent(i) and phi(i). Thus, if the pent(i) or phi(i) are modeled as functions of individual covariates, the derived parameters will also be functions of these individual covariates, creating a illogical estimate. The best strategy for use of individual covariates with the POPAN data type is to use the mean values of the individual covariates for providing the estimates of the real and derived parameters.
cooch
 
Posts: 1628
Joined: Thu May 15, 2003 4:11 pm
Location: Cornell University

Re: Overlapping confidence intervals POPAN

Postby j.harv3y » Wed Mar 25, 2020 7:45 am

All sounds good, thanks so much for the explanation.

Re. the individual covariates, I'm not sure I understand 100%. I had come across the individual covariates bit before, but nothing was mentioned about group covariates, so assumed it was fine. Should the same approach be taken? I've seen a couple of papers that do just add in a group covaraite but perhaps I've misunderstood?

Thanks so much again!
j.harv3y
 
Posts: 45
Joined: Mon Oct 08, 2018 4:45 am


Return to analysis help

Who is online

Users browsing this forum: No registered users and 11 guests