model-averaged estimate output

questions concerning analysis/theory using program PRESENCE

model-averaged estimate output

Postby J.Vanek » Sun Dec 31, 2017 4:03 pm

Hi everyone,

1) Are there any guiding documents on interpreting the outputs of the model-averaged estimates?

2) What does the dialogue box "Would you like to have estimates from each model printed? (If you select 'No', only the model averaged estimates for each site are printed.)" actually mean?

When I click Yes, I receive an output like this (first two sites only for example):

nsites=159 nsrvys=8 nmods=14
site:1 BER-A-101
model wgt cumwgt psi1 se(psi1) model name
1 0.321 0.321 0.0233 0.0173 psi(GROUP+PC1+PC2),gamma(GROUP+PC1+PC2),eps(.),p(SEASON)
2 0.139 0.461 . . psi(GROUP+PC1+PC2),gamma(GLOBAL),eps(.),p(SEASON)
3 0.097 0.558 . . psi(GROUP+PC1+PC2),gamma(GROUP+PC1+PC2+PC3),eps(.),p(SEASON)
4 0.090 0.648 . . psi(GROUP+PC1+PC2),gamma(GROUP+PC1),eps(.),p(SEASON)
5 0.074 0.722 . . psi(GROUP+PC1+PC2),gamma(GROUP+URBAN),eps(.),p(SEASON)
6 0.058 0.779 . . psi(GROUP+PC1+PC2),gamma(GROUP+PC2),eps(.),p(SEASON)
7 0.052 0.831 . . psi(GROUP+PC1+PC2),gamma(SEASON),eps(.),p(SEASON)
8 0.034 0.865 . . psi(GROUP+PC1+PC2),gamma(GROUP),eps(.),p(SEASON)
9 0.029 0.894 . . psi(GROUP+PC1+PC2),gamma(GROUP+PC1+PC3),eps(.),p(SEASON)
10 0.028 0.921 . . psi(GROUP+PC1+PC2),gamma(GROUP+ROAD),eps(.),p(SEASON)
11 0.027 0.949 . . psi(GROUP+PC1+PC2),gamma(.),eps(.),p(SEASON)
12 0.023 0.972 . . psi(GROUP+PC1+PC2),gamma(GROUP+URBAN+ROAD),eps(.),p(SEASON)
13 0.018 0.989 . . psi(GROUP+PC1+PC2),gamma(GROUP+PC2+PC3),eps(.),p(SEASON)
14 0.011 1.000 . . psi(GROUP+PC1+PC2),gamma(GROUP+PC3),eps(.),p(SEASON)

psi1(site 1) model-avg-est: . se= . BER-A-101


site:2 BER-A-102
model wgt cumwgt psi1 se(psi1) model name
1 0.321 0.321 0.1117 0.0632 psi(GROUP+PC1+PC2),gamma(GROUP+PC1+PC2),eps(.),p(SEASON)
2 0.139 0.461 . . psi(GROUP+PC1+PC2),gamma(GLOBAL),eps(.),p(SEASON)
3 0.097 0.558 . . psi(GROUP+PC1+PC2),gamma(GROUP+PC1+PC2+PC3),eps(.),p(SEASON)
4 0.090 0.648 . . psi(GROUP+PC1+PC2),gamma(GROUP+PC1),eps(.),p(SEASON)
5 0.074 0.722 . . psi(GROUP+PC1+PC2),gamma(GROUP+URBAN),eps(.),p(SEASON)
6 0.058 0.779 . . psi(GROUP+PC1+PC2),gamma(GROUP+PC2),eps(.),p(SEASON)
7 0.052 0.831 . . psi(GROUP+PC1+PC2),gamma(SEASON),eps(.),p(SEASON)
8 0.034 0.865 . . psi(GROUP+PC1+PC2),gamma(GROUP),eps(.),p(SEASON)
9 0.029 0.894 . . psi(GROUP+PC1+PC2),gamma(GROUP+PC1+PC3),eps(.),p(SEASON)
10 0.028 0.921 . . psi(GROUP+PC1+PC2),gamma(GROUP+ROAD),eps(.),p(SEASON)
11 0.027 0.949 . . psi(GROUP+PC1+PC2),gamma(.),eps(.),p(SEASON)
12 0.023 0.972 . . psi(GROUP+PC1+PC2),gamma(GROUP+URBAN+ROAD),eps(.),p(SEASON)
13 0.018 0.989 . . psi(GROUP+PC1+PC2),gamma(GROUP+PC2+PC3),eps(.),p(SEASON)
14 0.011 1.000 . . psi(GROUP+PC1+PC2),gamma(GROUP+PC3),eps(.),p(SEASON)

psi1(site 2) model-avg-est: . se= . BER-A-102


Why are the last lines for each site (underlined) for each site blank? Are the model averaged estimates for each site actually the numbers I have in bold?

When I click "No", I get an output that is all empty:

psi1(site 1) model-avg-est: . se= . BER-A-101
psi1(site 2) model-avg-est: . se= . BER-A-102
psi1(site 3) model-avg-est: . se= . CAH-A-101
psi1(site 4) model-avg-est: . se= . CAH-A-102
psi1(site 5) model-avg-est: . se= . CUB-B-101
psi1(site 6) model-avg-est: . se= . CUB-D-102
psi1(site 7) model-avg-est: . se= . CUB-E-104
psi1(site 8) model-avg-est: . se= . CUB-F-103
psi1(site 9) model-avg-est: . se= . CUB-F-105
psi1(site 10) model-avg-est: . se= . CUB-F-106
psi1(site 11) model-avg-est: . se= . CUB-H-107
psi1(site 12) model-avg-est: . se= . CUB-H-108
psi1(site 13) model-avg-est: . se= . ETH-A-101
psi1(site 14) model-avg-est: . se= . ETH-B-102
psi1(site 15) model-avg-est: . se= . ETH-D-103
psi1(site 16) model-avg-est: . se= . ETH-E-104
psi1(site 17) model-avg-est: . se= . ETH-G-105
psi1(site 18) model-avg-est: . se= . FOR-B-102

etc. etc.


Thank you!
J.Vanek
 
Posts: 7
Joined: Sat Jun 04, 2016 9:23 pm

Re: model-averaged estimate output

Postby jhines » Fri Jan 05, 2018 1:19 pm

Sorry, I don't think I've written any documentation for the model-averaged tool yet.

The model-averaging tool computes the (weighted) average value for each parameter (psi, p) for each site and survey. To do this, it reads the output of each model to get the value for each parameter. The value of psi (or p), for each site is printed, along with the model weight when you select "Yes". Those values are used to compute the weighted average, which is printed after all (14) models. So, if you are only interested in the model-averaged estimates, select "No" when asked about printing estimates from each model. If you want to see how the model-averaged estimates are computed, select "Yes".

The numbers you hi-lighted in your output are the estimates from the top-ranked model, not the model-averaged estimates. The model-averaged estimates are the ones you underlined which have the label, "model-avg-est". Unfortunately, it appears that there was a problem reading the psi parameter from the output of models 2-14, which is why you see the ". ." in each of those lines and in the model-avg-est line.

If you would like to send me (jhines@usgs.gov) the most recent backup zipfile in your project folder, I will diagnose why the model-averaging tool was unable to read those estimates.
jhines
 
Posts: 425
Joined: Fri May 16, 2003 9:24 am
Location: Laurel, MD, USA

Re: model-averaged estimate output

Postby J.Vanek » Tue Jan 09, 2018 3:57 am

Thank you very much!
J.Vanek
 
Posts: 7
Joined: Sat Jun 04, 2016 9:23 pm

Re: model-averaged estimate output

Postby jthomas » Wed Feb 07, 2018 11:23 pm

Hi there. I’m using Presence v 12.7 and am having the same problem with model-averaged estimates. The output contains blank estimates, or estimates are incorrect (i.e. averaged psi estimates are much lower than individual model estimates, standard errors too large). I never had this problem with previous versions, using the same data. Is it possible this is a software problem, or is it coming from my end?

Side note: I thought of model averaging in RPresence, but (as far as I can tell) there is currently no option to use QAIC.

Thanks!
jthomas
 
Posts: 4
Joined: Sun Feb 04, 2018 8:40 pm

Re: model-averaged estimate output

Postby jhines » Thu Feb 08, 2018 10:56 am

Hi,

Presence tries to read the output file for each model when computing model-averaged estimates. If the output files have been deleted, or the models didn't produce reasonable estimates, you should see a "." for the model-averaged estimate.

If you send me your most recent backup zipfile, I can diagnose the problem for you.

Jim
jhines
 
Posts: 425
Joined: Fri May 16, 2003 9:24 am
Location: Laurel, MD, USA

Re: model-averaged estimate output

Postby jhines » Mon Feb 12, 2018 2:18 pm

I've fixed a bug in the model-averaging tool in PRESENCE. Version 2.12.9 should work now.
jhines
 
Posts: 425
Joined: Fri May 16, 2003 9:24 am
Location: Laurel, MD, USA

Re: model-averaged estimate output

Postby jthomas » Thu Feb 15, 2018 7:17 pm

Great, thank you Jim!
jthomas
 
Posts: 4
Joined: Sun Feb 04, 2018 8:40 pm

Re: model-averaged estimate output

Postby Albert » Wed Feb 28, 2018 12:56 pm

I’m carrying out a single-species multiple-season occupancy analysis (118 sites, 2 seasons and 4 surveys/season) and I’m making a model averaging to estimate the parameters. However, I think that the results of model averaging aren’t correct.

After searching in the forum I updated with the last version Presence 2.12.10, but the estimations aren’t still correct (as a previous post, low estimations, high SE and some missing estimations).

Here I show the first part of results with a subset of models:

site:1 site1
model wgt cumwgt psi1 se(psi1) model name
1 0.339 0.339 0.8364 0.0448 psi(.),gam(.),eps=1-gam,p()
2 0.244 0.583 0.8361 0.0448 psi,gamma(),eps(),p()
3 0.208 0.791 . . psi(cte),gamma(),p()
4 0.203 0.994 . . psi,gamma(),eps(),p(s)
5 0.006 1.000 . . psi,gamma(),eps(),p(p+s)

psi1(site 1) model-avg-est: 0.0706 se=4170.0302 site1
psi1(avg.allsites) model-avg-est: 0.0706 sd= 0.0000


site:1 site1
model wgt cumwgt gam1 se(gam1) model name
1 0.339 0.339 0.9142 0.0331 psi(.),gam(.),eps=1-gam,p()
2 0.244 0.583 . . psi,gamma(),eps(),p()
3 0.208 0.791 . . psi(cte),gamma(),p()
4 0.203 0.994 . . psi,gamma(),eps(),p(s)
5 0.006 1.000 . . psi,gamma(),eps(),p(p+s)

gam1(site 1) model-avg-est: . se= . site1
gam1(avg.allsites) model-avg-est: 0.0000 sd= 0.0000

Maybe the problem persists?
Thank you for your help!

Albert
Albert
 
Posts: 2
Joined: Tue Feb 27, 2018 7:23 pm

Re: model-averaged estimate output

Postby jhines » Wed Feb 28, 2018 3:15 pm

The model-averaging function works by reading the design matrices to get the parameters to model-average. Since you have some models with the psi(.)gam(t)eps(t) parameterization, and some models with the psi(t)gam(t) parameterization, the parameters are not in the same places in the design matrices. If you send me your most recent backup zipfile, I will try to get the model-averaging tool to accommodate this.

Jim
jhines
 
Posts: 425
Joined: Fri May 16, 2003 9:24 am
Location: Laurel, MD, USA

Re: model-averaged estimate output

Postby Albert » Wed Feb 28, 2018 8:12 pm

Dear Jim,
Thank you vey much for your help!

Albert
Albert
 
Posts: 2
Joined: Tue Feb 27, 2018 7:23 pm


Return to analysis help

Who is online

Users browsing this forum: No registered users and 1 guest