Skip to content
This repository has been archived by the owner on Sep 26, 2023. It is now read-only.

Truncate columns after escaping #98

Open
chuwy opened this issue Jan 31, 2020 · 2 comments
Open

Truncate columns after escaping #98

chuwy opened this issue Jan 31, 2020 · 2 comments

Comments

@chuwy
Copy link
Contributor

chuwy commented Jan 31, 2020

#42 introduced a column truncation in Transformer. But it truncates human-readable strings (with quotes and newlines) before they're escaped. As a result if string was truncated to 4096 and it has on " symbol that has to be truncated - this field will end up as 4097-characters length. Most common offender is se_label.
@chuwy chuwy added this to the Version 0.6.0 milestone Jan 31, 2020
@chuwy chuwy changed the title Truncate Truncate columns after escaping Jan 31, 2020
@chuwy chuwy mentioned this issue Jan 31, 2020
Closed
@chuwy chuwy removed this from the Version 0.6.0 milestone Jan 31, 2020
@chuwy
Copy link
Contributor Author

chuwy commented Jan 31, 2020

Not possible to implement right now, as this escaping happens in circe's noSpaces

@chuwy
Copy link
Contributor Author

chuwy commented Jan 31, 2020

Technically, we can count special symbols (\s) and extract number we get from final length.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@chuwy