-
-
Notifications
You must be signed in to change notification settings - Fork 34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Moment matching with iwmm package #212
Comments
From my perspective all options are good with one expection: I would prefer to not add another hard dependency of loo. That is, if iwmm was to continue its own package used by loo, it should perhaps better be under |
I agree with @paul-buerkner. If this can work by adding a "soft" dependency then that's fine and would avoid copying code, but we should avoid adding it as a "hard" dependency. Assuming we can Suggest iwmm instead of Import it, then maybe that's the best option since it avoids copying any code between the packages, right? |
Thanks for the replies.
Completely agree, I think it would fit under
Yes, this should avoid code duplication. I think |
We have also discussed adding Pareto-k diagnostic to posterior package (and had one person last summer to do that, but there is no PR yet), so that in cases when we just need that it would be better to depend on posterior than on loo |
Currently
loo_moment_match
only works onstanfit
objects (due to the reliance on thelog_prob
methods fromrstan
), but notCmdStanFit
or just a matrix of draws (see e.g. #209). I was thinking about the best way to expand support ofloo_moment_match
to other objects, and am opening this issue mostly for discussion at this stage.@topipa has created a generic implementation of moment matching (iwmm) which works on a matrix or
stanfit
object. Recently we have also addedCmdStanFit
support to this package (using the new model methods in cmdstanr which are currently not yet in a release).iwmm is not (yet) on CRAN, so now might be a good time to discuss the best way to interoperate with loo.
After some discussion with @avehtari, I currently see three options (there may be others I haven't thought of):
Code from iwmm is copied into loo and adapted for use in
loo::loo_moment_match
(made specific for leave-one-out importance posteriors). iwmm would remain an independent package for generic importance sampling.iwmm is submitted to CRAN and
loo::loo_moment_match
is rewritten to useiwmm::moment_match
. loo would then import and depend on iwmm.All iwmm functions are moved into loo, and kept generic (i.e. not specific for leave-one-out posteriors). Keeping the functions generic has some precedence as loo is the home of
loo::psis
which is used in cases other than leave-one-out CV (e.g. in priorsense and adjustr).@jgabry @avehtari @topipa @paul-buerkner , do you have any thoughts on this?
The text was updated successfully, but these errors were encountered: