Add dynamic_zarr_store module

[emfdavid]

Grib Index Aggregations

The functions in this module allow building kerchunk aggregations of NODD grib2 weather forecasts fast.

The module supports a 3 step process

1. Extract and persist metadata directly from a few arbitrary grib files for a given product such as HRRR SUBH
2. Use the metadata mapping to build an index table of every grib message from the idx text files
3. Combine the index data with the metadata to build any FMRC slice (Horizon, RunTime, ValidTime, BestAvailable)

Once the metadata is created for the grib files from one complete forecast run (for instance, 48 hourly files from the 00Z HRRR SFC product), it takes less than a minutes to index a whole year of forecasts in a single python process - no parallelism required. This speeds up building the aggregations. It does not speed up reading the data (that is next).

A juptyer notebook provides a brief demonstration of the capability.

Camus Energy is using this operationally with GEFS, GFS and HRRR grib2 files, available on NODD hosted cloud storage buckets. There is no requirements file or docker file included in this PR. There are extensive tests that can be shared later. To run the code you must install kerchunk from github as the grib_tree code is not in the version 2.2 release.

This excerpt of our production code is a prototype for the community discussion that we hope can move into Kerchunk.

[mpiannucci]

Can't wait to go through this!!!

[emfdavid]

PR got a little bigger with the test fixtures 😂

I need to cleanup any lingering references to private Camus Energy buckets in the test fixtures.