rstan fails to read cmdstanr output .csv files

[santikka]

Summary:

rstan::read_stan_csv() no longer works for for .csv files produced by cmdstanr

Description:

It seems that rstan expects the save_warmup field to be an integer value, but cmdstanr instead writes this as a logical value (e.g., false) to the conversion to integer fails in rstan:::parse_stancsv_comments() resulting in an NA value for save_warmup. This used to work with earlier versions of cmdstanr/CmdStan.

Reproducible Steps:

library("cmdstanr")
#> This is cmdstanr version 0.8.1
#> - CmdStanR documentation and vignettes: mc-stan.org/cmdstanr
#> - CmdStan path: C:/Users/Santtu/.cmdstan/cmdstan-2.35.0
#> - CmdStan version: 2.35.0
file <- file.path(cmdstan_path(), "examples", "bernoulli", "bernoulli.stan")
mod <- cmdstan_model(file)
data_list <- list(N = 10, y = c(0, 1, 0, 0, 0, 0, 0, 0, 0, 1))
samples <- mod$sample(
  data = data_list,
  refresh = 0,
  chains = 1,
  iter_sampling = 10,
  iter_warmup = 10
)
#> Running MCMC with 1 chain...
#> 
#> Chain 1 WARNING: No variance estimation is 
#> Chain 1          performed for num_warmup < 20 
#> Chain 1 finished in 0.0 seconds.
rstan::read_stan_csv(samples$output_files())
#> Warning in parse_stancsv_comments(comments): NAs introduced by coercion
#> Error in if (max(save_warmup) == 0L) {: missing value where TRUE/FALSE needed

^{Created on 2024-07-10 with reprex v2.1.1}

RStan Version:

2.32.6

R Version:

R version 4.4.0 (2024-04-24 ucrt)

Operating System:

Windows 11 Pro 23H2

[martinmodrak]

I can reproduce the issue - outputs from CmdStan 2.34.1 are read fine, outputs from 2.35.0 cause problems.

Although I get a different error message (maybe because of using R 4.3.2)

Error in rstan::read_stan_csv(samples$output_files()) : 
  object 'n_kept' not found

I'll also note that the problem is made worse by the code at:

rstan/rstan/rstan/R/stan_csv.R

Line 220 in 9b8d8fe

if (max(save_warmup) == 0L) { # all equal to 0L

where if save_warmup is not all 0 or all 1, n_kept never gets assigned.

[jgabry]

Thanks. I think this might be fixed by #1131. Or at least the save_warmup issue, not sure about the n_kept issue @martinmodrak ran into.

[martinmodrak]

@jgabry Yes, this doesn't resolve the problamatic logic around n_kept. Also #1131 seems likely to break the old .CSVs, though. I didn't do a full checkout, but notice:

as.integer(as.logical("0"))
# [1] NA
as.integer(as.logical("false"))
# [1] 0

[jgabry]

Also #1131 seems likely to break the old .CSVs, though.

Ah good point. Perhaps we should revert #1131 and figure out a more general approach? @bgoodri @andrjohns we probably need to decide whether we want to keep supporting this functionality in RStan. If there continue to be changes in the CSV files created by CmdStan I don't think RStan can keep up with releases in order to make sure read_stan_csv keeps working. Do we even need it anymore given that CmdStanR exists?

[andrjohns]

The approach I took for cmdstanr for compatibility with both was to convert any true/false to 0/1, so that the parsing logic for the rest stayed the same

[martinmodrak]

Note that using cmdstanr with brms relies on read_stan_csv. We probably should add a test with fixed CSVs from s couple CmdStan versions and it should be OK.... Dne st 31. 7. 2024 19:37 uživatel Andrew Johnson ***@***.***> napsal:

…

The approach I took for cmdstanr for compatibility with both was to convert any true/false to 0/1 <https://github.com/stan-dev/cmdstanr/blob/0e3a99a52bc5ad15d9e386e9993c10ad6905818b/R/csv.R#L843>, so that the parsing logic for the rest stayed the same — Reply to this email directly, view it on GitHub <#1133 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACILKU3PJQLPI4IAEB7ONHDZPEOFFAVCNFSM6AAAAABKVJVGVGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRRGAZDMNZXGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[andrjohns]

Note that using cmdstanr with brms relies on read_stan_csv.

It doesn't look like that's the case anymore (which makes sense, otherwise there would have been a ton of bug reports in brms once 2.35 was released)

[jgabry]

otherwise there would have been a ton of bug reports in brms once 2.35 was released

Yeah that would have been a mess!