Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Raise retryable Python errors during daft.read_parquet #2136

Closed
2 tasks done
jaychia opened this issue Apr 15, 2024 · 0 comments
Closed
2 tasks done

Raise retryable Python errors during daft.read_parquet #2136

jaychia opened this issue Apr 15, 2024 · 0 comments
Labels
p0 Priority 0 - to be addressed immediately

Comments

@jaychia
Copy link
Contributor

jaychia commented Apr 15, 2024 •
edited
Loading

Is your feature request related to a problem? Please describe.

Currently when errors happen during daft.read_parquet execution, they get propagated as generic ValueError or FileNotFoundError. This makes it difficult for applications to decide intelligently whether these errors may be transient (and hence should be retried on the application level).

Daft should expose a RetryableError type, and raise that appropriately for any network-based errors that the user should retry.

  • Raise a "retryable error" for any network-based errors such as connection timeouts or DNS erorrs
  • Figure out how to correctly propagate the error when S3 streams get terminated by S3
@jaychia jaychia added the p1 Important to tackle soon, but preemptable by p0 label Apr 15, 2024
@jaychia jaychia added p0 Priority 0 - to be addressed immediately and removed p1 Important to tackle soon, but preemptable by p0 labels Apr 23, 2024
@samster25 samster25 mentioned this issue Apr 30, 2024
Merged
samster25 added a commit that referenced this issue Apr 30, 2024
@samster25
[FEAT] Rust side exceptions for Transient Errors (#2197)
c439cbc 
#2136
* Enables Python checkable exceptions when performing IO in Daft
* Currently raises the following:
* DaftTypeError
* ConnectTimeoutError
* ReadTimeoutError
* ByteStreamError
* SocketError
@jaychia jaychia closed this as completed May 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
p0 Priority 0 - to be addressed immediately
Projects
No open projects
Daft-OSS
Archived in project
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jaychia