Skip to content

Merge hierarchical pipeline #1083

Merged
merged 22 commits into from
Jan 27, 2023
Merged

Merge hierarchical pipeline #1083

merged 22 commits into from
Jan 27, 2023

Conversation

alex-hse-repository
Copy link
Collaborator

@alex-hse-repository alex-hse-repository commented Jan 26, 2023

Before submitting (must do checklist)

  • Did you read the contribution guide?
  • Did you update the docs? We use Numpy format for all the methods and classes.
  • Did you write any new necessary tests?
  • Did you update the CHANGELOG?

Proposed Changes

  • HierarchicaStructure -- container to hold the hierarchical structure of the time series
  • Hierarchical series support in TSDataset
  • Random hierarchical dataset generation
  • HierarchicalPipeline -- pipeline to forecast time series with hierarchical structure
  • TopDownReconciler and BottomUpReconciler -- hierarchical reconciliation methods

Closing issues

@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@github-actions
Copy link

github-actions bot commented Jan 26, 2023

@github-actions github-actions bot temporarily deployed to pull request January 26, 2023 14:18 Inactive
@Mr-Geekman
Copy link
Contributor

Mr-Geekman commented Jan 26, 2023

Is there is CHANGELOG.md?

@codecov-commenter
Copy link

codecov-commenter commented Jan 26, 2023

Codecov Report

Merging #1083 (0844bd6) into master (c04bf64) will decrease coverage by 18.08%.
The diff coverage is 31.75%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@             Coverage Diff             @@
##           master    #1083       +/-   ##
===========================================
- Coverage   86.51%   68.44%   -18.08%     
===========================================
  Files         164      170        +6     
  Lines        8901     9304      +403     
===========================================
- Hits         7701     6368     -1333     
- Misses       1200     2936     +1736     
Impacted Files Coverage Δ
etna/datasets/datasets_generation.py 55.22% <3.33%> (-44.78%) ⬇️
etna/datasets/hierarchical_structure.py 22.01% <22.01%> (ø)
etna/datasets/utils.py 45.97% <23.68%> (-49.95%) ⬇️
etna/datasets/tsdataset.py 74.83% <30.55%> (-15.91%) ⬇️
etna/reconciliation/bottom_up.py 35.29% <35.29%> (ø)
etna/pipeline/base.py 79.15% <37.50%> (-15.50%) ⬇️
etna/pipeline/hierarchical_pipeline.py 40.38% <40.38%> (ø)
etna/reconciliation/top_down.py 42.37% <42.37%> (ø)
etna/reconciliation/base.py 53.33% <53.33%> (ø)
etna/datasets/__init__.py 100.00% <100.00%> (ø)
... and 67 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@github-actions github-actions bot temporarily deployed to pull request January 27, 2023 06:51 Inactive
sigma: float = 1,
random_seed: int = 1,
) -> pd.DataFrame:
"""Create DataFrame with hierarchical structure and AR process data.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From method name and docstring I can't understand how many levels are generated during this method.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add n_levels parameter to make it more explicit

from etna.transforms.base import Transform


class HierarchicalPipeline(Pipeline):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After all, it is somewhat confusing that HierarchicalPipeline mimics Pipeline, but we can't e.g. mimic the logic of AutoRegressivePipeline or predict lower levels with ensembles.

May be it should be decorator over pipeline class that only adds logic with reconciliation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have we thought about this possibility?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We thought about solving it via composition, however it turns out that we should implement too much methods. But it is interesting point that levels might be predicted via different pipelines type, we haven't thought in this direction. May be we will improve this in next iteration of this track

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should definitely think about it in the future.

Copy link
Contributor

@Mr-Geekman Mr-Geekman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look at comments above.

@@ -0,0 +1,1539 @@
{
Copy link
Contributor

@martins0n martins0n Jan 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some issues with rendering of \begin{equation}  y_{A,t} = y_{AA,t} + y_{AB,t}  \  y_{t} = y_{A,t} + y_{B,t}  \end{equation}


Reply via ReviewNB

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

examples/hierarchical_pipeline.ipynb Show resolved Hide resolved
@github-actions github-actions bot temporarily deployed to pull request January 27, 2023 09:10 Inactive
@github-actions github-actions bot temporarily deployed to pull request January 27, 2023 09:19 Inactive
@github-actions github-actions bot temporarily deployed to pull request January 27, 2023 09:44 Inactive
@github-actions github-actions bot temporarily deployed to pull request January 27, 2023 10:17 Inactive
@alex-hse-repository alex-hse-repository self-assigned this Jan 27, 2023
@github-actions github-actions bot temporarily deployed to pull request January 27, 2023 10:25 Inactive
@github-actions github-actions bot temporarily deployed to pull request January 27, 2023 10:40 Inactive
@alex-hse-repository alex-hse-repository enabled auto-merge (squash) January 27, 2023 10:45
@alex-hse-repository alex-hse-repository merged commit e5ce280 into master Jan 27, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants