refactor!: remove K8S audit logs from Falco #1952

jasondellaluce · 2022-03-22T16:30:01Z

What type of PR is this?

/kind cleanup

/kind design

Any specific area of the project related to this PR?

/area build

/area engine

What this PR does / why we need it:

This is part of the ongoing effort of porting the current K8S Audit Logs support implementation out of Falco and turn it into a Falcosecurity PR. The goals of this PR are to:

Remove the current audit log implementation from Falco
Adapt all integration tests to use new K8S Audit plugin
Adapt the K8S default ruleset to use the new K8S Audit plugin
Drop the civetweb dependency
Re-implement the healthz endpoint with a lightweight HTTP server

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

This new plugin-based K8S Audit implementation introduces the following breaking changes:

The falco.yaml piece of configuration responsible of K8S Audit is now located under the plugins section
List field types now have explicit list typing: a field can either extract a single value or a list of values. For list values, extracting one value means extracting a list containing only one value, which is not the same thing as extracting regular single values
List field types now only support the in and intersects operators. For example, checks such as ka.req.role.rules.verbs contains create would be rejected and would need to be changed in the equivalent ka.req.role.rules.verbs intersects (create)
Failed field value extraction should be checked with the exists operator, and not by comparing with the <NA> string.
The <NA> string literal is not returned anymore, neither in single-valued fields nor in list fields. For example, in the past field existence was occasionally checked with expressions like ka.target.subresource != <NA>, which would now inherently never be true, because if the field is absent the string comparison fails by default. Instead, prefer using the analoguous ka.target.subresource exists, which explicitly checks for missing values.
The /healtz endpoint cannot bind to the same port of the K8S Audit Log endpoint (e.g. /k8s-audit), due to the fact that they are now managed by two different webservers (one in Falco, one in the plugin)
Syscalls and K8S Audit event sources cannot be active at the same time anymore. This is caused by the current restriction of having 1 event source active per time in the libs, so that this may change in the future

Blocking PRs:

Does this PR introduce a user-facing change?:

refactor!: remove K8S audit logs from Falco

poiana · 2022-03-22T16:32:42Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jasondellaluce

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jasondellaluce]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

jasondellaluce · 2022-03-24T09:13:40Z

/milestone 0.32.0

jasondellaluce · 2022-04-21T12:33:33Z

This is currently blocked by the changes of #1976.

jasondellaluce · 2022-04-22T07:58:06Z

userspace/engine/falco_engine_version.h

@@ -21,4 +21,4 @@ limitations under the License.
 // This is the result of running "falco --list -N | sha256sum" and
 // represents the fields supported by this version of Falco. It's used
 // at build time to detect a changed set of fields.
-#define FALCO_FIELDS_CHECKSUM "77c4c549181b8aac1b9698c0101ac61acb5b2faede84a2c4fecb34834c6de2b9"
+#define FALCO_FIELDS_CHECKSUM "a557747a209f2d16e90a3324d84d56c02cf54d000b6e3ee44598413f19885fcc"


This changes because the ka.* and jevt.* are not listed anymore with the default falco.yaml configuration.

Signed-off-by: Jason Dellaluce <[email protected]>

…plib Signed-off-by: Jason Dellaluce <[email protected]>

Signed-off-by: Jason Dellaluce <[email protected]>

…ests Signed-off-by: Jason Dellaluce <[email protected]>

Signed-off-by: Jason Dellaluce <[email protected]> Co-authored-by: Leonardo Grasso <[email protected]>

Signed-off-by: Jason Dellaluce <[email protected]>

leogr

Just left a comment, otherwise SGTM! 🤩

userspace/falco/webserver.cpp

Signed-off-by: Jason Dellaluce <[email protected]> Co-authored-by: Leonardo Grasso <[email protected]> Co-authored-by: Leonardo Grasso <[email protected]>

leogr

💥

mstemm

Just a couple of questions and one suggested change. Thanks for making this great change!

userspace/engine/json_evt.h

mstemm · 2022-04-29T18:31:24Z

userspace/falco/app_actions/init_falco_engine.cpp

 	}

 	for(const auto &src : m_options.disable_sources)
 	{
 		m_state->enabled_sources.erase(src);
 	}

-	// XXX/mstemm technically this isn't right, you could disable syscall *and* k8s_audit and configure a plugin.
+	// todo(jasondellaluce,leogr): change this once we attain multiple active source


This will increase the need to support multiple event sources within one falco process. k8s and syscalls together used to be supported together obviously, and that isn't possible atm.

Definitely. I left this comment as a remainder for when we'll work on supporting multiple sources.

mstemm · 2022-04-29T18:33:34Z

CMakeLists.txt

@@ -148,8 +148,8 @@ if(NOT MINIMAL_BUILD)
  # libcurl
  include(curl)

-  # civetweb
-  include(civetweb)
+  # cpp-httlib


Just curious, why the switch to cpp-httplib?

Civetweb caused lots of compilation problems in the past, and was in general a troublesome dependency for us in the past. This new http server is header-only and pretty lightweight, which better serves out purposes for now since we only implement /healthz.

mstemm

I'll approve it to just save a round-trip on monday. @jasondellaluce promised to update the comment and I'll check back for answers on the other questions.

Thanks again!

poiana · 2022-04-29T18:46:45Z

LGTM label has been added.

Git tree hash: 093cd38cf8219e09e240c466c1ea51cd9a7afd6d

mstemm · 2022-04-29T18:48:54Z

(whoops, didn't see that @leogr had also approved). Well I'll check back for the answers and it would be nice to clean up the comment in a later PR. Not the end of the world as that code is pretty retired anyway.

poiana added release-note do-not-merge/work-in-progress kind/cleanup dco-signoff: yes kind/design area/build area/engine labels Mar 22, 2022

poiana requested review from fntlnz and mstemm March 22, 2022 16:32

poiana added approved size/XXL labels Mar 22, 2022

jasondellaluce force-pushed the refactor/k8saudit-porting branch 2 times, most recently from 579f680 to 92548f7 Compare March 23, 2022 16:41

poiana added this to the 0.32.0 milestone Mar 24, 2022

jasondellaluce force-pushed the refactor/k8saudit-porting branch 2 times, most recently from 8255a55 to edebac8 Compare March 24, 2022 10:09

jasondellaluce force-pushed the refactor/k8saudit-porting branch from edebac8 to e429ecf Compare April 20, 2022 08:29

jasondellaluce mentioned this pull request Apr 21, 2022

refactor!: deprecate PSP regression tests and warn for unsafe usage of <NA> in k8s audit filters #1976

Merged

jasondellaluce changed the title ~~wip: refactor: remove K8S audit logs from Falco~~ wip: refactor!: remove K8S audit logs from Falco Apr 21, 2022

jasondellaluce force-pushed the refactor/k8saudit-porting branch from e429ecf to a969dd1 Compare April 22, 2022 07:49

jasondellaluce commented Apr 22, 2022

View reviewed changes

jasondellaluce force-pushed the refactor/k8saudit-porting branch from a969dd1 to d638ff2 Compare April 22, 2022 09:13

jasondellaluce changed the title ~~wip: refactor!: remove K8S audit logs from Falco~~ refactor!: remove K8S audit logs from Falco Apr 22, 2022

poiana removed the do-not-merge/work-in-progress label Apr 22, 2022

jasondellaluce mentioned this pull request Apr 22, 2022

Only processing K8s Audit Logs #1757

Closed

jasondellaluce force-pushed the refactor/k8saudit-porting branch from 952bd5b to 83b9c1c Compare April 22, 2022 14:22

refactor(userspace/falco): remove k8s audit references from falco

26c12cb

Signed-off-by: Jason Dellaluce <[email protected]>

jasondellaluce and others added 5 commits April 29, 2022 08:23

refactor: drop civetweb dependency and implement healtz using cpp-htt…

3b6a263

…plib Signed-off-by: Jason Dellaluce <[email protected]>

refactor: add k8saudit plugin and adapt config, tests, and rulesets

0cbd528

Signed-off-by: Jason Dellaluce <[email protected]>

refactor(test): use SKIP_PLUGINS_TESTS to skip k8s audit regression t…

a17861f

…ests Signed-off-by: Jason Dellaluce <[email protected]>

chore: solve compilation issues and polish code

c4b9140

Signed-off-by: Jason Dellaluce <[email protected]> Co-authored-by: Leonardo Grasso <[email protected]>

chore(falco.yaml): comment-out k8s plugin default config values

1e04c6e

Signed-off-by: Jason Dellaluce <[email protected]>

jasondellaluce force-pushed the refactor/k8saudit-porting branch from 6047bb8 to 1e04c6e Compare April 29, 2022 08:24

leogr reviewed Apr 29, 2022

View reviewed changes

userspace/falco/webserver.cpp Outdated Show resolved Hide resolved

fix(userspace/falco): listen to proper host in webserver

1658cf7

Signed-off-by: Jason Dellaluce <[email protected]> Co-authored-by: Leonardo Grasso <[email protected]> Co-authored-by: Leonardo Grasso <[email protected]>

leogr approved these changes Apr 29, 2022

View reviewed changes

poiana assigned leogr Apr 29, 2022

poiana added the lgtm label Apr 29, 2022

mstemm requested changes Apr 29, 2022

View reviewed changes

poiana assigned mstemm Apr 29, 2022

poiana removed the lgtm label Apr 29, 2022

poiana requested a review from leogr April 29, 2022 18:34

mstemm approved these changes Apr 29, 2022

View reviewed changes

poiana added the lgtm label Apr 29, 2022

poiana merged commit dbbc93f into master Apr 29, 2022

poiana deleted the refactor/k8saudit-porting branch April 29, 2022 18:47

jasondellaluce mentioned this pull request May 11, 2022

refactor(docs): update documentation of k8s audit events support falcosecurity/falco-website#584

Merged

jasondellaluce mentioned this pull request Jun 20, 2022

[Tracking] Opening multiple event sources in the same Falco instance #2074

Closed

15 tasks

leogr mentioned this pull request Jul 26, 2023

Update info in "Build Falco from source" page falcosecurity/falco-website#1091

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor!: remove K8S audit logs from Falco #1952

refactor!: remove K8S audit logs from Falco #1952

jasondellaluce commented Mar 22, 2022 •

edited

Loading

poiana commented Mar 22, 2022

jasondellaluce commented Mar 24, 2022

jasondellaluce commented Apr 21, 2022

jasondellaluce Apr 22, 2022

leogr left a comment

leogr left a comment

mstemm left a comment

mstemm Apr 29, 2022

jasondellaluce Apr 29, 2022

mstemm Apr 29, 2022

jasondellaluce Apr 29, 2022

mstemm left a comment

poiana commented Apr 29, 2022

mstemm commented Apr 29, 2022

refactor!: remove K8S audit logs from Falco #1952

refactor!: remove K8S audit logs from Falco #1952

Conversation

jasondellaluce commented Mar 22, 2022 • edited Loading

poiana commented Mar 22, 2022

jasondellaluce commented Mar 24, 2022

jasondellaluce commented Apr 21, 2022

jasondellaluce Apr 22, 2022

Choose a reason for hiding this comment

leogr left a comment

Choose a reason for hiding this comment

leogr left a comment

Choose a reason for hiding this comment

mstemm left a comment

Choose a reason for hiding this comment

mstemm Apr 29, 2022

Choose a reason for hiding this comment

jasondellaluce Apr 29, 2022

Choose a reason for hiding this comment

mstemm Apr 29, 2022

Choose a reason for hiding this comment

jasondellaluce Apr 29, 2022

Choose a reason for hiding this comment

mstemm left a comment

Choose a reason for hiding this comment

poiana commented Apr 29, 2022

mstemm commented Apr 29, 2022

jasondellaluce commented Mar 22, 2022 •

edited

Loading