-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle logging non-JSON-serializable classes in stream slices #22118
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems like a good idea. Out of curiosity, are there slices today where the value is not JSON serializable?
Note that you will probably need to run |
Yes! The run I linked is from a connector which defines streamslices as a StreamSlice dataclass (not a dictionary) with the following definition:
obviously, this goes against typing to be using a data class instead of a dict or other mapping format - it should still probably be updated to be a TypedDict. However, if you fix that, then you still run into the error because I didn't notice they came from pendulum, not the |
And thanks, will format and add changelog info etc. so that I can release it :) Edit: just remembered changelog and versioning are handled by the action, so convenient 💯 |
What
Describe what the change is solving
It helps to add screenshots if it affects the frontend.
Avoid issues like these when trying to log stream slices which contain values that are not serializable. Note that in the case of the linked action, the
streamslice
class is not aMapping
, so it's not the most valid case here. However, when I implemented that as a TypedDict instead, I ran into the same issue fordatetime
formats, and I think they're a valid case we should support, hence that being the test case.How
Describe the solution
When dumping, if a class is not json serializable, dump it as a string.