Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Agent] Add agent standalone manifests for system module & Pod's log collection #23938

Merged
merged 12 commits into from
Feb 22, 2021
Merged

[Agent] Add agent standalone manifests for system module & Pod's log collection #23938

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
fc9e7fe
Add agent standalone manifests for system module
ChrsMark Feb 9, 2021
3aec0b3
Rename file
ChrsMark Feb 16, 2021
33df263
Merge remote-tracking branch 'upstream/master' into add_standalone_ag…
ChrsMark Feb 16, 2021
5fdd099
rebase on top of k8s module
ChrsMark Feb 16, 2021
34da3b2
Split manifests as per pattern
ChrsMark Feb 16, 2021
e414c59
Add Pod's logs collection
ChrsMark Feb 17, 2021
7573987
Update configmap subfile
ChrsMark Feb 17, 2021
97018db
Add more verobse logging
ChrsMark Feb 17, 2021
c5340e3
Fix version placeholders
ChrsMark Feb 18, 2021
75c6f16
Remove ids & set log selector
ChrsMark Feb 19, 2021
e2b008f
Remove verbose logging
ChrsMark Feb 19, 2021
292c360
Remove secrets from clusterrole
ChrsMark Feb 22, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
163 changes: 159 additions & 4 deletions deploy/kubernetes/elastic-agent-standalone-kubernetes.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,10 @@ spec:
dnsPolicy: ClusterFirstWithHostNet
containers:
- name: elastic-agent
image: docker.elastic.co/beats/elastic-agent:7.12.0-SNAPSHOT
image: docker.elastic.co/beats/elastic-agent:8.0.0-SNAPSHOT
args: [
"-c", "/etc/agent.yml",
"-e", "-d", "composable.providers.kubernetes",
"-e", "-d", "*",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we have the debug selector for all on by default? That seems like it would produce probably more than it should.

Maybe remove it with a comment on how to add it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏼

]
env:
- name: ES_USERNAME
Expand All @@ -52,11 +52,35 @@ spec:
mountPath: /etc/agent.yml
readOnly: true
subPath: agent.yml
- name: proc
mountPath: /hostfs/proc
readOnly: true
- name: cgroup
mountPath: /hostfs/sys/fs/cgroup
readOnly: true
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
- name: varlog
mountPath: /var/log
readOnly: true
volumes:
- name: datastreams
configMap:
defaultMode: 0640
name: agent-node-datastreams
- name: proc
hostPath:
path: /proc
- name: cgroup
hostPath:
path: /sys/fs/cgroup
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: varlog
hostPath:
path: /var/log
---
apiVersion: v1
kind: ConfigMap
Expand Down Expand Up @@ -87,6 +111,137 @@ data:
node: ${NODE_NAME}
scope: node
inputs:
- id: 4ae27079-6cd4-4ab7-a459-abbae74ffc44
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should be able to remove the id from everywhere. You really do not need those, being that those are normally generated by Fleet.

Elastic Agent will work without id on the inputs and streams.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏼 thanks for clarifying

name: log-1
revision: 1
type: logfile
use_output: default
meta:
package:
name: log
version: 0.4.6
data_stream:
namespace: default
streams:
- id: logfile-log.log-4ae27079-6cd4-4ab7-a459-abbae74ffc44
data_stream:
dataset: generic
symlinks: true
paths:
- /var/log/containers/*${kubernetes.container.id}.log
- id: 1f6112ef-9847-4294-aba4-0dc9e7240551
name: system-3
revision: 2
type: system/metrics
use_output: default
meta:
package:
name: system
version: 0.10.9
data_stream:
namespace: default
streams:
- id: system/metrics-system.core-1f6112ef-9847-4294-aba4-0dc9e7240551
data_stream:
dataset: system.core
type: metrics
metricsets:
- core
core.metrics:
- percentages
- id: system/metrics-system.cpu-1f6112ef-9847-4294-aba4-0dc9e7240551
data_stream:
dataset: system.cpu
type: metrics
period: 10s
cpu.metrics:
- percentages
- normalized_percentages
metricsets:
- cpu
- id: system/metrics-system.diskio-1f6112ef-9847-4294-aba4-0dc9e7240551
data_stream:
dataset: system.diskio
type: metrics
period: 10s
diskio.include_devices: null
metricsets:
- diskio
- id: system/metrics-system.filesystem-1f6112ef-9847-4294-aba4-0dc9e7240551
data_stream:
dataset: system.filesystem
type: metrics
period: 1m
metricsets:
- filesystem
processors:
- drop_event.when.regexp:
system.filesystem.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
- id: system/metrics-system.fsstat-1f6112ef-9847-4294-aba4-0dc9e7240551
data_stream:
dataset: system.fsstat
type: metrics
period: 1m
metricsets:
- fsstat
processors:
- drop_event.when.regexp:
system.fsstat.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
- id: system/metrics-system.load-1f6112ef-9847-4294-aba4-0dc9e7240551
data_stream:
dataset: system.load
type: metrics
period: 10s
metricsets:
- load
- id: system/metrics-system.memory-1f6112ef-9847-4294-aba4-0dc9e7240551
data_stream:
dataset: system.memory
type: metrics
period: 10s
metricsets:
- memory
- id: system/metrics-system.network-1f6112ef-9847-4294-aba4-0dc9e7240551
data_stream:
dataset: system.network
type: metrics
period: 10s
network.interfaces: null
metricsets:
- network
- id: system/metrics-system.process-1f6112ef-9847-4294-aba4-0dc9e7240551
data_stream:
dataset: system.process
type: metrics
process.include_top_n.by_memory: 5
period: 10s
processes:
- .*
process.include_top_n.by_cpu: 5
process.cgroups.enabled: false
process.cmdline.cache.enabled: true
metricsets:
- process
process.include_cpu_ticks: false
system.hostfs: /hostfs
- id: >-
system/metrics-system.process_summary-1f6112ef-9847-4294-aba4-0dc9e7240551
data_stream:
dataset: system.process_summary
type: metrics
period: 10s
metricsets:
- process_summary
system.hostfs: /hostfs
- id: >-
system/metrics-system.socket_summary-1f6112ef-9847-4294-aba4-0dc9e7240551
data_stream:
dataset: system.socket_summary
type: metrics
period: 10s
metricsets:
- socket_summary
system.hostfs: /hostfs
david-kow marked this conversation as resolved.
Show resolved Hide resolved
- id: 934ef8aa-ed19-405b-8160-ebf62e3d32f8
name: kubernetes-node-metrics
revision: 1
Expand Down Expand Up @@ -215,10 +370,10 @@ spec:
serviceAccountName: elastic-agent
containers:
- name: elastic-agent
image: docker.elastic.co/beats/elastic-agent:7.12.0-SNAPSHOT
image: docker.elastic.co/beats/elastic-agent:8.0.0-SNAPSHOT
ChrsMark marked this conversation as resolved.
Show resolved Hide resolved
args: [
"-c", "/etc/agent.yml",
"-e", "-d", "composable.providers.kubernetes",
"-e", "-d", "*",
]
env:
- name: ES_USERNAME
Expand Down
Loading