Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempt Use Of Dates With idata Objects #30

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

AFg6K7h4fhy2
Copy link
Contributor

@AFg6K7h4fhy2 AFg6K7h4fhy2 commented Oct 21, 2024

Tests for this will be found here: #29

This PR ought to be merged before further work on #9 occurs.

@AFg6K7h4fhy2 AFg6K7h4fhy2 added the enhancement New feature or request label Oct 21, 2024
@AFg6K7h4fhy2 AFg6K7h4fhy2 added this to the [October 14, October 25] milestone Oct 21, 2024
@AFg6K7h4fhy2 AFg6K7h4fhy2 self-assigned this Oct 21, 2024
@AFg6K7h4fhy2 AFg6K7h4fhy2 linked an issue Oct 21, 2024 that may be closed by this pull request
@AFg6K7h4fhy2
Copy link
Contributor Author

Forecasts in this PR are implicitly assumed to be sequential days — not weeks, 2-weeks, a sequence of Saturdays, etc... Other resolution may be supported. Some critical tasks of this PR include (some of these already exist, albeit inadequately):

  • Designing a function that adds "dates" as coordinates OR as a group to an az.InferenceData object that was generated from AT LEAST posterior and posterior_predictive groups (should have observed_data by default and might not have a predictions group).
  • Designing a function that receives an az.InferenceData object with date coordinates OR a dates group and converts it to a polars dataframe.
  • (similar to previous) Designing a function that receives an az.InferenceData object with date coordinates OR a dates group and converts it to tidy dataframe.

Key considerations:

  • Dates as coordinates?
  • Dates as groups in idata object?
  • Names of idata groups to expect (e.g. obs or y in the group observed_data)?
  • Function in forecasttools (not on user-side) for idata wo/ dates → idata w/ dates?
  • Function in forecasttools (not on user-side) for idata w/ dates → [dataframe formats]?

@AFg6K7h4fhy2
Copy link
Contributor Author

From DB:

  1. TPM is working on an experiment to create some idata with dates to demo functionality in forecasttools. This is fine and I don't care how it is done in the immediate future.
  2. There should be some utility to add dates to an existing idata object. It should be agnostic as possible. It should live in forecasttools (although I initially made an issue for it in PyRenew) (1) should be implemented using (2), once (2) exists.

@AFg6K7h4fhy2
Copy link
Contributor Author

AFg6K7h4fhy2 commented Oct 22, 2024

Some notes extracted from the experimentation file for adding dates to idata objects:

  • Function for adding dates to idata accept either IS08601 str or datetime date.
  • Any functions that receives a polars dataframe with dates ought to have those dates typed date (datetime).
  • .draw was artifact from R → use draw.
  • There should be name agnostic idata reference to dimensions (e.g. reference what is stored in idata.observed_data["obs_dim_0"] without using obs_dim_0...

NOTE:

# idata.observed_data.sizes["obs_dim_0"] is suggested by Arviz over
# idata.observed_data["obs_dim_0"].size

idata.observed_data.sizes
# output: Frozen({'obs_dim_0': 488})

# idata.observed_data.dims gets one:
# output: FrozenMappingWarningOnValuesAccess({'obs_dim_0': 488})


# ^ "obs_col" name still needed ^
  • An alternate name, which could be a forecasttools default, for a key like obs_dim_0 could be day_index.
  • Dates being added to idata objects ought to be "as agnostic as possible".
  • A function that add to an idata object an interval of one-day sequential dates is one possible function for forecasttools; other functions could provide other resolutions.
  • TPM PR task:
    • Experimentation using assign_coords with dates to idata.
    • Refactor function that takes idata wo/ dates and converts to polars dataframe.
    • Experimentation using add_group with dates to idata (shows plot_ts).
    • Using assign_coords to create new .nc example forecast file.
    • Refactor function that aggregates values by epiweek to use idata w/ dates.
    • Ensure vignette still works.

@AFg6K7h4fhy2 AFg6K7h4fhy2 marked this pull request as ready for review October 23, 2024 20:58
.pre-commit-config.yaml Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Attempt Use Of Dates With idata Objects
2 participants