diff --git a/x-pack/auditbeat/module/system/socket/_meta/docs.asciidoc b/x-pack/auditbeat/module/system/socket/_meta/docs.asciidoc index fc63d5ab6db..fbaf6e94157 100644 --- a/x-pack/auditbeat/module/system/socket/_meta/docs.asciidoc +++ b/x-pack/auditbeat/module/system/socket/_meta/docs.asciidoc @@ -2,5 +2,147 @@ beta[] -This is the `socket` dataset of the system module. +This is the `socket` dataset of the system module. It allows to monitor network +traffic to and from running processes. It's main features are: +- Supports TCP and UDP sockets over IPv4 and IPv6. +- Outputs per-flow bytes and packets counters. +- Enriches the flows with https://www.elastic.co/guide/en/ecs/current/ecs-process.html[process] +and https://www.elastic.co/guide/en/ecs/current/ecs-user.html[user] information. +- Provides information similar to Packetbeat's flow monitoring with reduced CPU +and memory usage. +- Works on stock kernels without the need of custom modules, external libraries +or development headers. + +This dataset does not analyze application-layer protocols nor provide any other +advanced features present in Packetbeat: +- Monitor network traffic whose destination is not a local process, as is the +case with traffic forwarding. +- Monitor layer 2 traffic, ICMP or raw sockets. + +[float] +=== Implementation + +It is implemented for Linux only and currently supports x86 (32 and 64 bit) +architectures. + +The dataset uses +https://www.kernel.org/doc/Documentation/trace/kprobetrace.txt[KProbe-based event tracing] +to monitor TCP and UDP sockets over IPv4 and IPv6, providing flow monitoring +that includes byte and packet counters, as well as the local process and user +involved in the flow. It does so by plugin into the TCP/IP stack to generate +custom tracing events avoiding the need to copy network traffic to user space. + +By not relying on periodic polling, this approach enables the dataset to perform +near real-time monitoring of the system without the risk of missing short lived +connections or processes. + +[float] +=== Requirements + +Features used by the `socket` dataset require a minimum Linux kernel version +of 3.12 (vanilla). However, some distributions have backported those features +to older kernels. The following (non-exhaustive) lists the different +distributions under which the dataset is known to work: + +[options="header"] +|============================================== +| Distribution | kernel version | Works? +| CentOS 6.5 | 2.6.32-431.el6 | NO^<>^ +| CentOS 6.9 | 2.6.32-696.30.1.el6 | ✓ +| CentOS 7.6 | 3.10.0-957.1.3.el7 | ✓ +| Debian 8 | 3.16.0-6 | ✓ +| Debian 9 | 4.9.0-8 | ✓ +| Debian 10 | 4.19.0-5 | ✓ +| SLES 12 | 4.4.73-5 | ✓ +| Ubuntu 12.04 | 3.2.0-126 | NO^<>^ +| Ubuntu 14.04.6 | 3.13.0-170 | ✓ +| Ubuntu 16.04.3 | 4.4.0-97 | ✓ +| AWS Linux 2 | 4.14.138-114.102 | ✓ +|============================================== + +[[anchor-1]] +<>: These systems lack +https://lore.kernel.org/patchwork/patch/399251/[PERF_EVENT_IOC_ID ioctl.] +Support might be added in a future release. + +The dataset needs CAP_SYS_ADMIN and CAP_NET_ADMIN in order to work. + +[float] +==== Kernel configuration + +A kernel built with the following configuration options enabled is required: + +- `CONFIG_KPROBE_EVENTS`: Enables the KProbes subsystem. +- `CONFIG_DEBUG_FS`: For kernels laking `tracefs` support (<4.1). +- `CONFIG_IPV6`. + +These settings are enabled by default in most distributions. + +The following configuration settings can prevent the dataset from starting: + +- `/sys/kernel/debug/kprobes/enabled` must be 1. +- `/proc/sys/net/ipv6/conf/lo/disable_ipv6` must be 0 +(IPv6 enabled in the loopback device). + +[float] +==== Running on docker + +The dataset can monitor the Docker host when running inside a container. However +it needs to run on a `privileged` container with `CAP_NET_ADMIN` and IPv6 +support. The docker container running {beatname_uc} needs access to the host's +tracefs or debugfs directory. This is achieved by bind-mounting `/sys`. + +[float] +=== Configuration + +The following options are available for the `socket` dataset: + +- `socket.tracefs_path` (default: none) + +Must point to the mount-point of `tracefs` or the `tracing` directory inside +`debugfs`. If this option is not specified, {beatname_uc} will look for +the default locations: `/sys/kernel/tracing` and `/sys/kernel/debug/tracing`. +If not found, it will attempt to mount `tracefs` and `debugfs` at their +default locations. + +- `socket.flow_inactive_timeout` (default: 30s) + +Determines how long a flow has to be inactive to be considered closed. + +- `socket.flow_termination_timeout` (default: 5s) + +Determines how long to wait after a socket has been closed for out of order +packets. With TCP, some packets can be received shortly after a socket is +closed. If set too low, additional flows will be generated for those packets. + +- `socket.perf_queue_size` (default: 4096) + +The number of tracing samples that can be queued for processing. A larger value +uses more memory but reduces the chances of samples being lost when the system +is under heavy load. + +- `socket.lost_queue_size` (default: 128) + +The number of lost samples notifications that can be queued. + +- `socket.ring_size_exponent` (default: 7) + +Controls the number of memory pages allocated for the per-CPU ring-buffer +used to receive samples from the kernel. The actual amount of memory used is +Number_of_CPUs x Page_Size(4KB) x 2^ring_size_exponent^. That is 0.5 MiB of RAM +per CPU with the default value. + +- `socket.clock_max_drift` (default: 100ms) + +Defines the maximum difference between the kernel internal clock and +the reference time used to timestamp events. + +- `socket.clock_sync_period` (default: 10s) + +Controls how often clock synchronization events are generated to measure drift +between the kernel clock and the dataset's reference clock. + +- `socket.guess_timeout` (default: 15s) + +The maximum time an individual guess is allowed to run. diff --git a/x-pack/auditbeat/module/system/socket/config.go b/x-pack/auditbeat/module/system/socket/config.go index a82c8e0f19f..e3a4258b23d 100644 --- a/x-pack/auditbeat/module/system/socket/config.go +++ b/x-pack/auditbeat/module/system/socket/config.go @@ -21,7 +21,7 @@ type Config struct { LostQueueSize int `config:"socket.lost_queue_size,min=1"` // ErrQueueSize defines the size of the error queue. A single error is fatal. - ErrQueueSize int `config:"socket.err_buffer_size,min=1"` + ErrQueueSize int `config:"socket.err_queue_size,min=1"` // RingSizeExp configures the exponent size for the per-cpu ring buffer used // by the kernel to pass tracing events.