Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[filebeat][threatintel] MISP splitting fix for empty responses #38917

Conversation

chrisberkhout
Copy link
Contributor

@chrisberkhout chrisberkhout commented Apr 13, 2024

Proposed commit message

[filebeat][threatintel] MISP splitting fix for empty responses (#)

Two fixes related to empty server responses:

- Set `response.split.ignore_empty_value` to `true` to avoid indexing
  `{response:[]}`, which is the MISP server's empty response body.
  According to the `response.split` [documentation][1], "If the split
  target is empty the parent document will be kept. If documents with
  empty splits should be dropped, the `ignore_empty_value` option should
  be set to `true`."

- Use the null-safe operator for a chained method invocation following a
  null-safe field access. All other null-safe operator usages were also
  reviewed and corrected where necessary.

[1]: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#response-split

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Related issues

@chrisberkhout chrisberkhout added Team:Service-Integrations Label for the Service Integrations team bugfix labels Apr 13, 2024
@chrisberkhout chrisberkhout self-assigned this Apr 13, 2024
@chrisberkhout chrisberkhout requested a review from a team as a code owner April 13, 2024 08:13
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Apr 13, 2024
Copy link
Contributor

mergify bot commented Apr 13, 2024

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @chrisberkhout? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

@chrisberkhout chrisberkhout added the Team:Security-Service Integrations Security Service Integrations Team label Apr 13, 2024
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@elasticmachine
Copy link
Collaborator

elasticmachine commented Apr 13, 2024

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Duration: 137 min 59 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@chrisberkhout chrisberkhout added backport-v8.12.0 Automated backport with mergify backport-v8.13.0 Automated backport with mergify labels Apr 13, 2024
@chrisberkhout chrisberkhout force-pushed the threatintel-misp-ignore-empty-responses branch from d973caf to 1ee99a4 Compare April 14, 2024 00:37
Copy link
Contributor

@efd6 efd6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit then LGTM

CHANGELOG.next.asciidoc Outdated Show resolved Hide resolved
Co-authored-by: Dan Kortschak <[email protected]>
@chrisberkhout chrisberkhout requested a review from efd6 April 15, 2024 06:55
@chrisberkhout chrisberkhout enabled auto-merge (squash) April 15, 2024 08:27
@chrisberkhout chrisberkhout merged commit 692658c into elastic:main Apr 15, 2024
41 of 42 checks passed
mergify bot pushed a commit that referenced this pull request Apr 15, 2024
Two fixes related to empty server responses:

- Set `response.split.ignore_empty_value` to `true` to avoid indexing
  `{response:[]}`, which is the MISP server's empty response body.
  According to the `response.split` [documentation][1], "If the split
  target is empty the parent document will be kept. If documents with
  empty splits should be dropped, the `ignore_empty_value` option should
  be set to `true`."

- Use the null-safe operator for a chained method invocation following a
  null-safe field access. All other null-safe operator usages were also
  reviewed and corrected where necessary.

[1]: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#response-split

---------

Co-authored-by: Dan Kortschak <[email protected]>
(cherry picked from commit 692658c)
mergify bot pushed a commit that referenced this pull request Apr 15, 2024
Two fixes related to empty server responses:

- Set `response.split.ignore_empty_value` to `true` to avoid indexing
  `{response:[]}`, which is the MISP server's empty response body.
  According to the `response.split` [documentation][1], "If the split
  target is empty the parent document will be kept. If documents with
  empty splits should be dropped, the `ignore_empty_value` option should
  be set to `true`."

- Use the null-safe operator for a chained method invocation following a
  null-safe field access. All other null-safe operator usages were also
  reviewed and corrected where necessary.

[1]: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#response-split

---------

Co-authored-by: Dan Kortschak <[email protected]>
(cherry picked from commit 692658c)
chrisberkhout added a commit that referenced this pull request Apr 15, 2024
Two fixes related to empty server responses:

- Set `response.split.ignore_empty_value` to `true` to avoid indexing
  `{response:[]}`, which is the MISP server's empty response body.
  According to the `response.split` [documentation][1], "If the split
  target is empty the parent document will be kept. If documents with
  empty splits should be dropped, the `ignore_empty_value` option should
  be set to `true`."

- Use the null-safe operator for a chained method invocation following a
  null-safe field access. All other null-safe operator usages were also
  reviewed and corrected where necessary.

[1]: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#response-split

---------

Co-authored-by: Dan Kortschak <[email protected]>
(cherry picked from commit 692658c)
chrisberkhout pushed a commit that referenced this pull request Apr 18, 2024
…r empty responses (#38927)

Two fixes related to empty server responses:

- Set `response.split.ignore_empty_value` to `true` to avoid indexing
  `{response:[]}`, which is the MISP server's empty response body.
  According to the `response.split` [documentation][1], "If the split
  target is empty the parent document will be kept. If documents with
  empty splits should be dropped, the `ignore_empty_value` option should
  be set to `true`."

- Use the null-safe operator for a chained method invocation following a
  null-safe field access. All other null-safe operator usages were also
  reviewed and corrected where necessary.

[1]: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#response-split

---------

Co-authored-by: Dan Kortschak <[email protected]>
chrisberkhout pushed a commit that referenced this pull request Apr 18, 2024
…r empty responses (#38928)

Two fixes related to empty server responses:

- Set `response.split.ignore_empty_value` to `true` to avoid indexing
  `{response:[]}`, which is the MISP server's empty response body.
  According to the `response.split` [documentation][1], "If the split
  target is empty the parent document will be kept. If documents with
  empty splits should be dropped, the `ignore_empty_value` option should
  be set to `true`."

- Use the null-safe operator for a chained method invocation following a
  null-safe field access. All other null-safe operator usages were also
  reviewed and corrected where necessary.

[1]: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#response-split

---------

Co-authored-by: Dan Kortschak <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-v8.12.0 Automated backport with mergify backport-v8.13.0 Automated backport with mergify bugfix Team:Security-Service Integrations Security Service Integrations Team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[filebeat][threatintel][MISP] "cannot access method/field [size] from a null def reference"
3 participants