Don't do inverse metric decomposition every draw #2894
Conversation
… once each time the inverse metric is set (instead of every sample, Issue #2881). This involved switching to setter/getters for interfacing with dense_e_point so I made the change for diag_e_point as well. Also changed set_metric verbage to set_inv_metric.
|
The current design is intentional. The Cholesky is needed only once per transition which is a relatively small cost compared to the many gradient evaluations needed within each transition. Saving the Cholesky decomposition introduces an additional At the same time the members of |
There was a problem hiding this comment.
lgtm! Though I'd like to wait to merge till @betanalpha approves the test example is satisfying
Jenkins Console Log Machine informationProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010CPU: G++: Clang: |
Jenkins Console Log Machine informationProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010CPU: G++: Clang: |
src/stan/mcmc/hmc/static_uniform/adapt_diag_e_static_uniform.hpp
Outdated
Show resolved
Hide resolved
| @@ -31,10 +31,15 @@ class adapt_diag_e_static_uniform | |||
| this->stepsize_adaptation_.learn_stepsize(this->nom_epsilon_, | |||
| s.accept_stat()); | |||
|
|
|||
| bool update = this->var_adaptation_.learn_variance(this->z_.inv_e_metric_, | |||
| this->z_.q); | |||
| Eigen::VectorXd inv_metric; | |||
There was a problem hiding this comment.
Can you know the size of this when you make it?
There was a problem hiding this comment.
I could, but it gets resized in learn_variance. Should I still set the size when declaring it?
There was a problem hiding this comment.
If you declare the sizes there then the resize is a no-op since it's already the correct size. But idt it's that big of a deal either way
There was a problem hiding this comment.
I just left it off. It's kindof a weird thing happening here anyway (like learn_variance is effectively returning two things).
…-dense-metric-decomposition
Jenkins Console Log Machine informationProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010CPU: G++: Clang: |
Jenkins Console Log Machine informationProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010CPU: G++: Clang: |
Thinking about this again, don't you pay the O(N^2) memory burden still computing the cholesky but it's just temporary data? If a program failed because of N is too large it would fail either way wouldn't it? |
Oh yeah, the temporary would still be O(N^2) since you compute the Cholesky at one point or another. Good point. I'll add the forwarding real quick |
…4.1 (tags/RELEASE_600/final)
…-dense-metric-decomposition
…b.com:stan-dev/stan into feature/issue-2881-dense-metric-decomposition
Jenkins Console Log Machine informationProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010CPU: G++: Clang: |
…-dense-metric-decomposition
…-dense-metric-decomposition
Jenkins Console Log Machine informationProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010CPU: G++: Clang: |
…4.1 (tags/RELEASE_600/final)
Fix for Issue #2881 .
This involved switching to setter/getters for interfacing with dense_e_point so I made the change for diag_e_point as well.
I also changed all the set_metric verbage to set_inv_metric.
I didn't know what tests needed added for this. Lemme know.
Submission Checklist
./runTests.py src/test/unitmake cpplintCopyright and Licensing
Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company): Columbia University
By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses: