RegressionAndOtherStories.jl v0.13

Project Status	Build Status

Note

After many years I have decided to step away from my work with Stan and Julia. My plan is to be around until the end of 2024 for support if someone decides to step in and take over further development and maintenance work.

At the end of 2024 I'll archive the different packages and projects included in the Github organisations StanJulia, StatisticalRethingJulia and RegressionAndOtherStoriesJulia if no one is interested (and time-wise able!) to take on this work.

I have thoroughly enjoyed working on both Julia and Stan and see both projects mature during the last 15 or so years. And I will always be grateful for the many folks who have helped me on numerous occasions. Both the Julia and the Stan community are awesome to work with! Thanks a lot!

Purpose (once completed)

RegressionAndOtherStories.jl contains supporting (Julia) functions and the data files used in "Regression and Other Stories" by Andrew Gelham, Jennifer Hill and Aki Vehtari.

The package is also used in project SR2StanPluto.jl v9+, a revised inplementation of the Statistical Rethinking support functions using Makie.jl, CausalInference.jl and GraphViz.jl.

Contents

The supporting functions are intended to be used in (currently) 3 Julia projects (also under development), ROSStanPluto.jl, ROSTuringPluto.jl and SR2StanPluto.

All currently available data files are in .csv format and located in the data directory.

If RegressionAndOtherStories.jl is loaded, the files can be read in as a DataFrame using e.g.:

hibbs = CSV.read(ros_datadir("ElectionsEconomy", "hibbs.csv"), DataFrame)

For that purpose ros_datadir() is exported.

If needed, Stata files (.dat) have been converted to .csv files as is illustrated by a few scripts in the scripts directory, e.g. see scripts\hdi.jl. To access the Stata files in the R package ROS-Examples RegressionAndOtherStories.jl expects the environment variable JULIA_ROS_HOME to be defined, e.g.:

ENV["JULIA_ROS_HOME"] = expanduser("~/Projects/R/ROS-Examples")

R itself does not necessarily need to be installed for this to work.

If so desired, direct use of the Stata files is also possible as the Stata to .csv file conversion scripts mentioned above show.

Approach

RegressionAndOtherStories.jl v9+ is using Julia's package extension option. In particular Turing, Stan, Makie, GraphViz and CausalInference, if needed, are included as extensions.

Over time I might minimize the use of AlgebraOfGraphics.jl. It is a nice package but also a bit more difficult to tailor (compared to Makie/CairoMakie).

I expect I can use ParetoSmoothedImportanceSampling.jl as is but will take another look at PSIS.jl and ParetoSmooth.jl when revising the relevant chapters.

Project maintenance for Pluto notebooks

In the subdirectory src/Maintenance/reset_notebooks.jl is a function I use in the Pluto notebook projects (SR2StanPluto, ROSStanPluto, etc.). The function potentially makes three changes to selected notebooks:

1. If it finds a line starting with Pkg.activate it disables that line if reset_activate = true.

2. If it finds a line starting with #Pkg.activate it enables that line if set_activate = true.

3. It removes the Project and Manifest sections of all notebooks selected for reset. See the maintenance notebooks in projects such as SR2StanPluto and ROSStanPluto.

Using Pkg.activate(...) is useful if your workflow uses many different notebooks. E.g., in my workflow I ] up the project SR2StanPluto in the REPL before I start opening individual notebooks in Pluto.

Issues, comments and questions

Please file issues, comments and questions here.

Pull requests are also welcome.

Versions

Versions 0.12 and 0.13

1. Switching to PCDAG and FCIDAG when using CausalInference.jl.

Version 0.11

1. Redone working with CausalInference.jl.
2. Added introductory CausalInference notebooks.

Version 0.10

1. Redone DAG struct.
2. Use GraphViz with CairoMakie.
3. switch to use CairoMakie instead of GLMakie.

Version 0.9

1. Switch to extensions.
2. Added simulate function.
3. Added scale_df_cols! (scale! conflicted with Makie and other packages).
4. Switching to CausalInference.jl as a replacement for StructuralCausalModels.jl.
5. Possibly switching to either PSIS.jl or ParetoSmooth.jl as a replacement for ParetoSmoothedImportanceSampling.jl.
6. Switched to Makie.jl and GLMakie.jl as back-end.
7. Use of GraphViz.jl to display DAGs.

Versions 0.7 and 0.8

1. Primarily following package updates.

Version 0.6.1

1. Changed back to use DataFrames directly as basis for summaries.
2. Use getindex to access single elements in summary DataFrames (first argument taken vrom parameters column in df)
3. For Stan use array() to group nested columns into a matrix. For Turing continue to use nested_column_to_array.

Release 0.5.0

1. Added DataFrame operatior function (not exported).
2. Added errorbars_mean and errorbars_draws.
3. Added nested_column_to_array.
4. Made model_summary String/Symbol agnostic.

Release 0.4.5

1. Doc fixes by Pietro Monticone
2. Added model_summary(::SampleModel).

Release 0.4.x

1. Model_summary and plot_chains (accept both Symbol and Strings)
2. Focus on Appendices A and B.
3. Focus on chapters 4, 5, 6, 7

Versions 0.3.6 - 0.3.10

1. Fine tuning working with ros_functions and ros_notebooks.

Release 0.3.5

1. Added maintenance functions for a (large) set of notebooks.

Release 0.3.4

1. Is tagging using JuliaHub with setting branch name working?

Version 0.3.3

1. Add initial version of notebook maintenance routines.
2. Tag this version (if not done by TagBot)

Version 0.3.2

1. Fix Makie and AoG glue scripts.

Version 0.3.1

1. StatsFuns compat entry to 1.0.

Version 0.3.0 (under development)

1. Switch back to using Requires.jl
2. Switch to using eachindex() where appropriate.
3. Experimental versions for chapter 3.

Version 0.2.4

1. Chapter 2 mostly done
2. Added trankplot function

Version 0.2.0

1. Support for the 5 examples from chapter 1 done.
2. Added plot_chains() and model_summary() functions.
3. Added Makie and AlgebraOfGraphics as dependencies.

Note: Source files for Makie/AoG are all in src/Makie/ to simplify moving those to a separate repo (not my intention right now, but still).

4. In sync with both ROS[Turing|Stan]Pluto projects tagged 2.3 and up.

Version 0.1.0

1. Initial commit (to registrate the package for usage in projects).

References

Of course this package is focused on:

0. Gelman, Hill, Vehtari: Regression and Other Stories

which in a sense is a major update to item 3. below.

There is no shortage of other good books on Bayesian statistics. A few of my favorites are:

1. Bolstad: Introduction to Bayesian statistics

2. Bolstad: Understanding Computational Bayesian Statistics

3. Gelman, Hill: Data Analysis Using Regression and Multilevel/Hierarchical Models

4. McElreath: Statistical Rethinking

5. Kruschke: Doing Bayesian Data Analysis

6. Lee, Wagenmakers: Bayesian Cognitive Modeling

7. Betancourt: A Conceptual Introduction to Hamiltonian Monte Carlo

8. Gelman, Carlin, and others: Bayesian Data Analysis

9. Pearl, Glymour, Jewell: Causal Inference in Statistics: A Primer

10. Pearl, Judea and MacKenzie, Dana: The Book of Why

11. Scott Cunningham: Causal Inference - the mixtapes

A good book to understand most of the Julia constructs used in this book is:

12. Bogumił Kamiński: Julia for Data Analysis.