Interpretting 95% Confidence Intervals

questions concerning analysis/theory using program PRESENCE

Interpretting 95% Confidence Intervals

Postby saldanh » Mon Jun 20, 2022 6:28 pm

Hello,

I am wondering whether there is a way to calculate the 95% confidence interval around the difference between two occupancy estimates (produced in PRESENCE) to determine if the two estimates are significantly different (like in the video here: https://www.youtube.com/watch?v=bz2cFDoBDB4). Is it possible without knowing the standard deviation?


Thanks,
Dan
saldanh
 
Posts: 5
Joined: Tue Mar 30, 2021 9:51 pm

Re: Interpretting 95% Confidence Intervals

Postby jhines » Mon Jun 20, 2022 9:21 pm

As Darryl said in his video, comparing two 95% confidence intervals for overlap doesn't give you a hypothesis test at the 5% level. One way to test if the difference is significant is to create the design matrix such that one estimate is the "intercept" and the other estimate is the difference between the two occupancy estimates. For example, if you have a site covariate for occupancy, with two possible categories for the covariate, then the beta associated with that site covariate would represent the difference between the two occupancy estimates (on the logit scale). Since Presence prints the 95% confidence limits for the betas, you just see if the confidence interval for that beta covers zero. If not, then the difference between the occupancy estimates is "significantly" different from zero.

Another method is to use a tool on the Patuxent website named "CONTRAST". That program can be used to compare estimates with standard errors, testing the hypothesis that all estimates come from the same distribution. You can enter the two occupancy estimates, with their standard errors (or variance-covariance matrix) and CONTRAST will produce a chi-square value and probability level testing that hypothesis. If the probability < 0.05, then you would reject the null hypothesis (both occupancy estimates come from the same distribution). If you have more than two occupancy estimates to compare, CONTRAST can compare them, but the comparison relates to the hypothesis that ALL occupancy estimates come from the same distribution. If any of the estimates are significantly different, the result from CONTRAST will be significant.

No, it is not possible to test for a significant difference between estimates if you don't have an idea of the variability in the estimates.
jhines
 
Posts: 598
Joined: Fri May 16, 2003 9:24 am
Location: Laurel, MD, USA

Re: Interpretting 95% Confidence Intervals

Postby cooch » Tue Jun 21, 2022 9:14 am

To follow from Jim's detailed response -- the issue of interpreting 'overlap' in CI's in terms of 'significance' (in the statistical sense), is illustrated by means of a simple numerical example in the following: viewtopic.php?f=1&t=3987&p=13190&hilit=overlapping#p13189

As Jim suggests, its important to realize that the approach you take would depend largely on whether you're trying to compare difference between groups in a single analysis (Jim's first suggestion using a offset-intercept design matrix), or if you have two separate estimates from two separate analyses, where the CONTRAST approach might work for you.

More 'conceptually', it might also be worth considering the important distinction between 'statistical' significance and 'biological' significance. One of the important ideas the Burnham & Anderson tomes reminded everyone of was the idea that statistical significance at some nominal alpha level can be (and often is) fairly arbitrary (why is alpha=0.05 any better than, say, alpha=0.025, or 0.1?). Rather, an alternative (and IMO, often better) approach is to specify a priori how big a difference at minimum you think is 'biologically' important, and why, and then use the statistical tools to determine if the differences between groups (as in your example, say) is bigger than your designated minimum critical (important) difference, and if based on the CI for that difference, does it bound this minimum threshold? This concept is discussed in some detail in the MARK book - chapter, 6, section 6.13: http://www.phidot.org/software/mark/doc ... ap6.pdf#68

Of course, if the issue at hand is 'difference between psi estimates', the challenge might be to determine what a 'biological significant difference' might be.
cooch
 
Posts: 1628
Joined: Thu May 15, 2003 4:11 pm
Location: Cornell University


Return to analysis help

Who is online

Users browsing this forum: No registered users and 1 guest

cron