Dont sanitize strings filters #625
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mstemm
force-pushed
the
dont-sanitize-strings-filters
branch
from
143bc58
to
9563e90
Compare
It turns out that with -O3 (release flags), using erase/remove_if with an operator struct is slightly faster than the inline version that was here, as the compiler can inline the operators. It isn't faster with -O2 and is much slower without optimization. Optimize for the release build by adding the operators back.
Making whitespace changes in separate commit.
Modify sinsp_filter_check::extract to take an optional argument sanitize_strings, defaulting to true. It controls whether the filtercheck should sanitize any string arguments when extracting the value from the event. The only place where it is set to false is in sinsp_filter_check::compare(). This means that when comparing events against a filter expression, any string values are not sanitized. They will still be sanitized when the events are displayed. This significantly speeds up users like falco that have complicated filter expressions that run against lots of events.
mstemm
force-pushed
the
dont-sanitize-strings-filters
branch
from
9563e90
to
729bbef
Compare
gianlucaborello
added a commit
that referenced
this pull request
Aug 16, 2016
* Switch back to erase/remove_if for sanitization. It turns out that with -O3 (release flags), using erase/remove_if with an operator struct is slightly faster than the inline version that was here, as the compiler can inline the operators. It isn't faster with -O2 and is much slower without optimization. Optimize for the release build by adding the operators back. * Whitespace diffs. Making whitespace changes in separate commit. * Make sanitize_string cfgable and off for filters. Modify sinsp_filter_check::extract to take an optional argument sanitize_strings, defaulting to true. It controls whether the filtercheck should sanitize any string arguments when extracting the value from the event. The only place where it is set to false is in sinsp_filter_check::compare(). This means that when comparing events against a filter expression, any string values are not sanitized. They will still be sanitized when the events are displayed. This significantly speeds up users like falco that have complicated filter expressions that run against lots of events.
dmyerscough
pushed a commit
to dmyerscough/sysdig
that referenced
this pull request
Mar 3, 2017
* Switch back to erase/remove_if for sanitization. It turns out that with -O3 (release flags), using erase/remove_if with an operator struct is slightly faster than the inline version that was here, as the compiler can inline the operators. It isn't faster with -O2 and is much slower without optimization. Optimize for the release build by adding the operators back. * Whitespace diffs. Making whitespace changes in separate commit. * Make sanitize_string cfgable and off for filters. Modify sinsp_filter_check::extract to take an optional argument sanitize_strings, defaulting to true. It controls whether the filtercheck should sanitize any string arguments when extracting the value from the event. The only place where it is set to false is in sinsp_filter_check::compare(). This means that when comparing events against a filter expression, any string values are not sanitized. They will still be sanitized when the events are displayed. This significantly speeds up users like falco that have complicated filter expressions that run against lots of events.
dmyerscough
pushed a commit
to dmyerscough/sysdig
that referenced
this pull request
Mar 3, 2017
While merging together draios#624 (multi string search) and draios#625 (don't sanitize strings), the length was only being set when string sanitization was enabled. Additionally, the length was not being initialized to 0, meaning that it was not defined. This led to some FPs for some of the falco tests.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stop sanitizing string values when comparing events to filtering expressions. String values are still sanitized when displaying events.
To give an idea of the cpu savings, running falco with a fairly optimized set of rules (but not the complete set of changes like optimized in (...) and per-event filters) had a cpu usage of 18% with the phoronix apache test. Disabling string optimization reduced the cpu usage to 14%.