-
Notifications
You must be signed in to change notification settings - Fork 46
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Avoid to raise exception when the version header field doesn't exist #99
Comments
As-described, this plugin handles failure in decoding by catching and logging the exception, and emitting a I agree that we should catch invalid CEF payloads as early as possible so that we can emit a more helpful log message. In this particular case, we encountered a payload that did not have any of the 7 required leading headers, and should fail with a clear indication of such instead of letting a partial- or empty-headers event through to fail in later processing. |
When encountering malformed-CEF or non-CEF payloads, this plugin now emits helpful descriptive log messages, and prevents data-loss and corruption by emitting an event tagged with `_cefparsefailure` containing the bytes it received. This set of changes catches 3 distinct cases of malformed payloads. - missing one or more of the 7 required CEF header fields; a payload that does not have all 7 unescaped-pipe-terminated header fields cannot be reliably interpreted as CEF (prevents corruption). - containing something OTHER than a sequence of key=value pairs in the extensions space (prevent data-loss; previously when extensions were invalid they were silently omitted) - containing unescaped newlines (prevents corruption; previously data after the first newline was injected into the currently-parsed extension field) In catching these classes of malformed inputs, this changeset also resolves logstash-plugins#99 in which our failure to detect a malformed input proactively caused an unhelpful `NoMethodError` message to be logged before a `_cefparsefailure`-tagged event was emitted.
* decode: multiline support in extension values Per CEF spec v25 (2017-09-28): > *Multi-line* fields can be sent by `CEF` by encoding the newline character > as `\n` or `\r`. Note that multiple lines are only allowed in the value part > of the extensions While this plugin has long _encoded_ multiline extension values and other escape sequences, our _decode_ has only supported escaped backslashes or escaped equals signs, and with this change becomes compliant with this portion of the spec. Note that due to the preexisting newline-centric normalization, round-trip encode/decode cycle is only _semantically_ guaranteed. * version bump & changelog for 6.2.6 * noop: remove $stderr debug logging * fix: proactively detect malformed CEF When encountering malformed-CEF or non-CEF payloads, this plugin now emits helpful descriptive log messages, and prevents data-loss and corruption by emitting an event tagged with `_cefparsefailure` containing the bytes it received. This set of changes catches 3 distinct cases of malformed payloads. - missing one or more of the 7 required CEF header fields; a payload that does not have all 7 unescaped-pipe-terminated header fields cannot be reliably interpreted as CEF (prevents corruption). - containing something OTHER than a sequence of key=value pairs in the extensions space (prevent data-loss; previously when extensions were invalid they were silently omitted) - containing unescaped newlines (prevents corruption; previously data after the first newline was injected into the currently-parsed extension field) In catching these classes of malformed inputs, this changeset also resolves #99 in which our failure to detect a malformed input proactively caused an unhelpful `NoMethodError` message to be logged before a `_cefparsefailure`-tagged event was emitted. * fix: when using `delimiter`, ensure codec flush consumes buffer Resolves #100
Logstash information:
Please include the following information:
bin/logstash --version
) anyJVM (e.g.
java -version
): bundled oneIf the affected version of Logstash is 7.9 (or earlier), or if it is NOT using the bundled JDK or using the 'no-jdk' version in 7.10 (or higher), please provide the following information:
java -version
)JAVA_HOME
environment variable if set.OS version (
uname -a
if on a Unix-like system):Description of the problem including expected versus actual behavior:
When an invalid CEF payload is presented to the codec, it should tag the event with
_cefparsefailure
instead of let throw anNoMethodError
Steps to reproduce:
Please include a minimal but complete recreation of the problem,
including (e.g.) pipeline definition(s), settings, locale, etc. The easier
you make for us to reproduce it, the more likely that somebody will take the
time to look at it.
Provide logs (if relevant):
The text was updated successfully, but these errors were encountered: