LOG-5998: fix max_line_bytes for audit logs #2844
Conversation
openshift-ci-robot
added
the
jira/valid-reference
|
@jcantrill: This pull request references LOG-5998 which is a valid jira issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/cherrypick master |
|
@jcantrill: once the present PR merges, I will cherry-pick it on top of master in a new PR and assign it to you. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: jcantrill The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
bot
added
the
approved
|
/cherrypick release-5.9 |
|
@jcantrill: once the present PR merges, I will cherry-pick it on top of release-5.9 in a new PR and assign it to you. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
| max_line_bytes = 3145728 | ||
| max_read_bytes = 262144 |
There was a problem hiding this comment.
Shouldn't this be the other way around?
| max_line_bytes = 3145728 | |
| max_read_bytes = 262144 | |
| max_read_bytes = 3145728 | |
| max_line_bytes = 262144 |
max_line_bytes
optional
uint
The maximum size of a line before it is discarded.
This protects against malformed lines or tailing incorrect files.
default: 102400 (bytes)
max_read_bytes
optional
uint
Max amount of bytes to read from a single file before switching over to the next file. Note: This does not apply when oldest_first is true.
This allows distributing the reads more or less evenly across the files.
default
There was a problem hiding this comment.
agree. in the docs the defaults identify read < max which should apply in our case too
There was a problem hiding this comment.
Actually, strike my last comment. The "max_line_bytes" is essentially the largest length a line can be and the assumption is that when completely read, the result is a well formed line message (e.g. json). max_read_bytes is the amount of bytes read in a "read" operation but doesn't say it has completely read a log entry
There was a problem hiding this comment.
Agreed but under normal circumstances you could read a batch of lines that in total are longer than the longest acceptable single line.
There was a problem hiding this comment.
If max "read from a single file before switching over to the next file" is set 12 times lower than before, are we sure of the impact?
There was a problem hiding this comment.
Scanning the discord, it does seem that if not set to the same value, the "read" is approx 10x smaller in general
ex.
max_line_bytes: 16777216 max_read_bytes: 16777216
or
max_line_bytes = 1024000 max_read_bytes = 102400
|
/test e2e-target |
|
/lgtm |
|
@jcantrill: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Description
This PR:
Links
https://issues.redhat.com/browse/LOG-5998