Delta method vs. computationally intensive variance estimate

Forum for discussion of general questions related to study design and/or analysis of existing data - software neutral.

Delta method vs. computationally intensive variance estimate

Postby eng » Wed Sep 16, 2020 7:01 pm

Hi everyone, I am hoping you can help check my intuition or find errors in my approach to estimating the variance of a function of parameters estimated from mark-recapture models. I used the Delta method and the computationally-intensive simulation approach described in the Addendum of Appendix B. However, my estimates from the two methods differ by several orders of magnitude.

My multi-variable transformation is simple, it is the ratio of two proportions: Y(ψ,α)=α/ψ, where ψ,α ∈
(0,1].

One catch is that they are estimated from separate models, so I don’t have a joint variance-covariance matrix, but I work around that (see below). The other catch (maybe more significant?) is that across data sets (I’m doing this for several different data sets) many of the parameters are close to the bound at 1. I think this may be violating assumptions of the delta method, causing it to perform poorly.


The Delta Method Approach
For the delta method, I found D the 1×2 vector of partial derivatives of Y with respect to each of the parameters, and then I used the maximum likelihood estimates from MARK to construct D = [(-ψ^-2*α) (-ψ^-1)]

As I mentioned above, I don’t have a variance-covariance matrix for ψ and α, but I took the conservative approach of assuming they are independent (which should give me the largest estimated variance for Y), and constructed the variance-covariance matrix Σ by setting the diagonal elements equal to the squared standard error estimated from MARK, and the off diagonals to 0.

I estimated the variance of Y as var(Y) = DΣD'.


Computationally Intensive Approach
For this approach, I used the simulation method introduced in the Addendum of Appendix B, where I simulated each of the parameters, did the transformation for a large sample of draws, and then computed analytical standard errors.

I specified ψ and α as independent Beta distributions, parameterized such that the means were equal to the MLEs and the standard deviations were equal to the standard errors from MARK.

I simulated 10^6 draws from each distribution, computed α/ψ for each pair of values, and then estimated the mean and variance of the ratio as the sample mean and variance of the draws.


Questions
When I compared estimates of the standard error of Y from both methods, they differ by several orders of magnitude (e.g., 0.0007 versus 0.02). The estimates from the computationally intensive approach (with larger SEs) seem more plausible, but I want to understand why the results are so different. Can anyone provide insight into why that is?

Aside from a coding or math error, my thought was that it could be because ψ and α are frequently close to the bound of 1, so the transformation is not very linear, which means the Delta method won’t provide a very good approximation.

I was also wondering if the simulation approach somehow better accounts for the missing covariance between α and ψ, as implied in the last paragraph of the discussion in Powell (2007).
eng
 
Posts: 4
Joined: Sun Feb 24, 2013 2:17 am

Re: Delta method vs. computationally intensive variance esti

Postby cooch » Thu Sep 17, 2020 3:47 pm

Congratulations on (clearly) doing a bit of homework.

eng wrote:
Computationally Intensive Approach
For this approach, I used the simulation method introduced in the Addendum of Appendix B, where I simulated each of the parameters, did the transformation for a large sample of draws, and then computed analytical standard errors.

I specified ψ and α as independent Beta distributions, parameterized such that the means were equal to the MLEs and the standard deviations were equal to the standard errors from MARK.

I simulated 10^6 draws from each distribution, computed α/ψ for each pair of values, and then estimated the mean and variance of the ratio as the sample mean and variance of the draws.


So, did you simulate the data on the logit scale, and back-transform? If you didn't, then simulating data when the mean is near either 0/1 boundary can be problematic.
cooch
 
Posts: 1628
Joined: Thu May 15, 2003 4:11 pm
Location: Cornell University

Re: Delta method vs. computationally intensive variance esti

Postby eng » Mon Sep 21, 2020 4:02 pm

While digging further, I discovered that my error was a pretty silly one, sqrt(0.0007) is about 0.02, so my main issue was comparing variances and SEs. There is a small difference (probability-scale simulation-based SEs were generally about 4% larger than when I used the Delta method on the probability-scale) but the difference is much more reasonable now.

I had simulated the data on the probability scale, using independent beta distributions (using the betaval function from the popbio package in R). The summary statistics of the simulated data seemed to match the input values.

Thank you for your help!
eng
 
Posts: 4
Joined: Sun Feb 24, 2013 2:17 am


Return to analysis & design questions

Who is online

Users browsing this forum: No registered users and 18 guests