Skip to content

[BUG] Fix inference test on subset of segments for DeepARModel #1089

Closed
1 task done
Mr-Geekman opened this issue Jan 30, 2023 · 2 comments
Closed
1 task done

[BUG] Fix inference test on subset of segments for DeepARModel #1089

Mr-Geekman opened this issue Jan 30, 2023 · 2 comments
Assignees
Labels
bug Something isn't working
Milestone

Comments

@Mr-Geekman
Copy link
Contributor

Mr-Geekman commented Jan 30, 2023

🐛 Bug Report

We have failing test in tests/test_modes/test_inference : test_forecast_subset_segments_failed_assertion_error.

We want to figure out why is it failing and try to fix this.
If we understand that it cannot be easily fixed, add link to this issue near the tests and explain the problem inside the issue.

Expected behavior

Tests are passed.

In this test we check that model can generate predictions on subset of segments used during training.

How To Reproduce

Run the tests.

Environment

No response

Additional context

No response

Checklist

  • Bug appears at the latest library version
@Mr-Geekman Mr-Geekman added the bug Something isn't working label Jan 30, 2023
@Mr-Geekman Mr-Geekman added this to the Inference 2.1 milestone Jan 30, 2023
@Mr-Geekman Mr-Geekman self-assigned this Feb 13, 2023
@Mr-Geekman
Copy link
Contributor Author

If I try to add n_samples=None like suggested in #802 , test for DeepARModel is working without AssertionError. However, it doesn't seem to work correctly because in this mode many tests in test_models/nn/test_deepar.py fail.

Probably, it isn't a correct way to make predictions deterministic. As I understand, with n_samples=None we make model predict values in non-autoregressive fashion and take true values as input. We usually expect this during training. But during prediction it isn't really possible because true values aren't available and predictions become poor.

Setting n_samples=1 doens't fix the test.

@Mr-Geekman
Copy link
Contributor Author

Closed by #1115.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
Status: Done
Development

No branches or pull requests

1 participant