Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Calling sort_and_finalize_staged_data throws non informative exception #1737

Closed
Tracked by #1679
vasil-pashov opened this issue Aug 1, 2024 · 0 comments · Fixed by #1763
Closed
Tracked by #1679

Calling sort_and_finalize_staged_data throws non informative exception #1737

vasil-pashov opened this issue Aug 1, 2024 · 0 comments · Fixed by #1763
Assignees
@vasil-pashov
Labels
bug Something isn't working

Comments

@vasil-pashov
Copy link
Collaborator

vasil-pashov commented Aug 1, 2024 •
edited
Loading

Describe the bug
If nothing is added to the staged segment and sort_and_finalize_staged_data an exception is thrown.

Steps to reproduce

import pandas as pd
import numpy as np
import arcticdb as adb

ac = adb.Arctic("lmdb://test")
lib = ac.get_library("test", create_if_missing=True)
lib.sort_and_finalize_staged_data("sym")

Output:

[2024-08-01 12:48:34.144] [arcticdb] [error] E_ASSERTION_FAILURE Stream descriptor not found in pipeline context
Traceback (most recent call last):
  File "...\test.py", line 7, in <module>
    lib.sort_and_finalize_staged_data("sym")
  File "...arcticdb\version_store\library.py", line 959, in sort_and_finalize_staged_data
    self._nvs.version_store.sort_merge(symbol, None, mode == StagedDataFinalizeMethod.APPEND, prune_previous_versions=prune_previous_versions)
arcticdb_ext.exceptions.InternalException: E_ASSERTION_FAILURE Stream descriptor not found in pipeline context

Expected result
Improve the exception message.

Related issues
Possibly related to: #1735, #1736

OS, Python, Arctic versions

Python: 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
OS: Windows-10-10.0.22631-SP0
ArcticDB: 4.5.0rc1
@vasil-pashov vasil-pashov mentioned this issue Aug 1, 2024
Closed
10 tasks
@vasil-pashov vasil-pashov changed the title sort_and_finalize throws E_ASSERTION_FAILURE Stream descriptor not found in pipeline context there are no staged segments. What's the correct behavior? Calling sort_and_finalize_staged_data throws there are no staged segments Aug 1, 2024
@vasil-pashov vasil-pashov self-assigned this Aug 1, 2024
@vasil-pashov vasil-pashov added the bug Something isn't working label Aug 1, 2024
This was referenced Aug 1, 2024
Closed
Closed
@vasil-pashov vasil-pashov changed the title Calling sort_and_finalize_staged_data throws there are no staged segments Calling sort_and_finalize_staged_data throws non informative exception Aug 14, 2024
@vasil-pashov vasil-pashov mentioned this issue Aug 14, 2024
Merged
5 tasks
vasil-pashov added a commit that referenced this issue Aug 21, 2024
@vasil-pashov
Fixes for sort and finalize (#1763)
e62a048 
#### Reference Issues/PRs
Fixes: #1737 #1736 #1735

#### What does this implement or fix?

#### Any other comments?

#### Checklist

<details>
  <summary>
   Checklist for code changes...
  </summary>
 
- [ ] Have you updated the relevant docstrings, documentation and
copyright notice?
- [ ] Is this contribution tested against [all ArcticDB's
features](../docs/mkdocs/docs/technical/contributing.md)?
- [ ] Do all exceptions introduced raise appropriate [error
messages](https://docs.arcticdb.io/error_messages/)?
 - [ ] Are API changes highlighted in the PR description?
- [ ] Is the PR labelled as enhancement or bug so it appears in
autogenerated release notes?
</details>

<!--
Thanks for contributing a Pull Request to ArcticDB! Please ensure you
have taken a look at:
- ArcticDB's Code of Conduct:
https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md
- ArcticDB's Contribution Licensing:
https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing
-->

---------

Co-authored-by: Vasil Pashov <[email protected]>
vasil-pashov added a commit that referenced this issue Sep 30, 2024
@vasil-pashov
Fixes for sort and finalize (#1763)
a51ff8b 
Fixes: #1737 #1736 #1735

<details>
  <summary>
   Checklist for code changes...
  </summary>

- [ ] Have you updated the relevant docstrings, documentation and
copyright notice?
- [ ] Is this contribution tested against [all ArcticDB's
features](../docs/mkdocs/docs/technical/contributing.md)?
- [ ] Do all exceptions introduced raise appropriate [error
messages](https://docs.arcticdb.io/error_messages/)?
 - [ ] Are API changes highlighted in the PR description?
- [ ] Is the PR labelled as enhancement or bug so it appears in
autogenerated release notes?
</details>

<!--
Thanks for contributing a Pull Request to ArcticDB! Please ensure you
have taken a look at:
- ArcticDB's Code of Conduct:
https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md
- ArcticDB's Contribution Licensing:
https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing
-->

---------

Co-authored-by: Vasil Pashov <[email protected]>
vasil-pashov added a commit that referenced this issue Sep 30, 2024
@vasil-pashov
Fixes for sort and finalize (#1763)
946cbf9 
Fixes: #1737 #1736 #1735

<details>
  <summary>
   Checklist for code changes...
  </summary>

- [ ] Have you updated the relevant docstrings, documentation and
copyright notice?
- [ ] Is this contribution tested against [all ArcticDB's
features](../docs/mkdocs/docs/technical/contributing.md)?
- [ ] Do all exceptions introduced raise appropriate [error
messages](https://docs.arcticdb.io/error_messages/)?
 - [ ] Are API changes highlighted in the PR description?
- [ ] Is the PR labelled as enhancement or bug so it appears in
autogenerated release notes?
</details>

<!--
Thanks for contributing a Pull Request to ArcticDB! Please ensure you
have taken a look at:
- ArcticDB's Code of Conduct:
https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md
- ArcticDB's Contribution Licensing:
https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing
-->

---------

Co-authored-by: Vasil Pashov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vasil-pashov vasil-pashov

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fixes for sort and finalize man-group/ArcticDB
1 participant
@vasil-pashov