Initial Interface and perfect-model experiment

[nefrathenrici]

This PR implements the initial interface with ClimaAtmos.jl, EnsembleKalmanProcesses.jl, and the Caltech HPC.
This is tested by the only experiment, a moist Held-Suarez perfect-model calibration found in experiments/sphere_held_suarez_rhoe_equilmoist

Content

Folder-by-folder:
src:

• atmos_interface.jl: contains functions to configure and run the atmos model
• ekp_interface.jl contains functions to initialize and update EKP

docs:

• Updated docbuilding
• Small doc page for the moist held suarez experiment.
• Comprehensive guide on using the pipeline

experiments:

• job_array.sh job scheduler for the pipeline
• initialize.sbatch Initializes the ensemble
• model_run.sbatch Runs the atmos model for a single ensemble member
• update.sbatch Computes the observation map and updates the EKI in the pipeline each iteration.
• Project.toml Environment for running experiments, includes preferences for distributed simulations
• Manifest.toml this is the bulk of the lines in the PR, needed for full reproducibility
• folder sphere_held_suarez_rhoe_equilmoist, contains the perfect-model calibration setup on parameter equator_pole_temperature_gradient_wet. Detailed below

experiments/sphere_held_suarez_rhoe_equilmoist:

• prior.toml: prior distribution for equator_pole_temperature_gradient_wet
• observation_map.jl: contains function for the observation map for the perfect model calibration. Used in EKI update step
• generate_observations.sbatch: Script for generating "truth" observations from a model run and process them using the same observation map
• obs_mean.jld2 and obs_noise_cov.jld2 Truth and noise data generated by generate_observations.sbatch
• ekp_config.yml: contains info for EKP such as # iterations, ensemble size, locations of truth data and prior
• atmos_config.yml Contains ClimaAtmos model configuration, diagnostics, location of the restart file
• day200.0.hdf5: Steady-state restart file for the experiment

test:

• Tests for atmos model configuration
• Tests for EKI initialization

.github:

• Add a testing github action to run tests on push

[codecov]

Welcome to Codecov 🎉

Once merged to your default branch, Codecov will compare your coverage reports and display the results in this comment.

Thanks for integrating Codecov - We've got you covered ☂️

[odunbar]

Hey! Here are a bunch of comments on the Docs which look really good so far.

My primary comment is that you need to talk about the overall pipeline at the top (e.g. the generate_observations and job_array scripts) as this is what users actually call

My secondary comments are largely just to use snippets where possible rather than explaining so much through sentences about the snippets.

Thanks

[odunbar]

Hi Nat - this looks much better,

• Could you attach a final section called something like "Plotting results"
• Add the function you call to plot the convergence, and the image that we have from our current convergence plot.

I think otherwise this is excellent for what we need. Thanks for the work in making this clear.

[odunbar]

The docs and experiment output look good, also we have had many outside discussions regarding the source code.

Please add any outstanding work you have on this to a checklist in a new issue (e.g. failure handling, updating the prior.toml with constrained_gaussian constructors, etc.)

Then LGTM! Thanks for all the work, squash and merge!