Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use NamespaceAwareResourceMetaGenerator for all generic kubernetes resources #33763

Merged
merged 13 commits into from
Dec 7, 2022
Merged

Use NamespaceAwareResourceMetaGenerator for all generic kubernetes resources #33763

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
f8638db
update elastic-agent-autodiscover; use NamespaceAwareResourceMetaGene…
tetianakravchenko Nov 22, 2022
f4ace20
update configuration comments
tetianakravchenko Nov 22, 2022
25b3e4c
add PR number; fix conflicts
tetianakravchenko Nov 22, 2022
841fb4b
run make notice
tetianakravchenko Nov 22, 2022
622ec2d
fix go.sum file after rebase
tetianakravchenko Nov 23, 2022
d115383
make update
tetianakravchenko Nov 23, 2022
1353bab
clean up changelog
tetianakravchenko Nov 23, 2022
a0234fe
test llc
tetianakravchenko Nov 28, 2022
5c7a985
Merge branch 'main' into k8s-metricsets-add-resource-meta
tetianakravchenko Nov 28, 2022
6e1b95f
Merge branch 'main' into k8s-metricsets-add-resource-meta
tetianakravchenko Nov 28, 2022
2db057a
revert local test
tetianakravchenko Nov 28, 2022
c8f7c08
Merge branch 'main' into k8s-metricsets-add-resource-meta
tetianakravchenko Dec 2, 2022
94fde5a
Merge branch 'main' into k8s-metricsets-add-resource-meta
tetianakravchenko Dec 7, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,7 @@ https://github.com/elastic/beats/compare/v8.2.0\...main[Check the HEAD diff]
- Update README file on how to run Metricbeat on Kubernetes. {pull}33308[33308]
- Add per-thread metrics to system_summary {pull}33614[33614]
- Add GCP CloudSQL metadata {pull}33066[33066]
- Add namespace metadata to all namespaced kubernetes resources. {pull}33763[33763]

*Packetbeat*

Expand Down
4 changes: 2 additions & 2 deletions NOTICE.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9656,11 +9656,11 @@ SOFTWARE.

--------------------------------------------------------------------------------
Dependency : github.com/elastic/elastic-agent-autodiscover
Version: v0.4.0
Version: v0.5.0
Licence type (autodetected): Apache-2.0
--------------------------------------------------------------------------------

Contents of probable licence file $GOMODCACHE/github.com/elastic/elastic-agent-autodiscover@v0.4.0/LICENSE:
Contents of probable licence file $GOMODCACHE/github.com/elastic/elastic-agent-autodiscover@v0.5.0/LICENSE:

Apache License
Version 2.0, January 2004
Expand Down
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ require (
github.com/aws/smithy-go v1.12.0
github.com/awslabs/kinesis-aggregation/go/v2 v2.0.0-20220623125934-28468a6701b5
github.com/elastic/bayeux v1.0.5
github.com/elastic/elastic-agent-autodiscover v0.4.0
github.com/elastic/elastic-agent-autodiscover v0.5.0
github.com/elastic/elastic-agent-libs v0.2.13
github.com/elastic/elastic-agent-shipper-client v0.4.0
github.com/elastic/elastic-agent-system-metrics v0.4.5-0.20220927192933-25a985b07d51
Expand Down
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -615,8 +615,8 @@ github.com/elastic/bayeux v1.0.5 h1:UceFq01ipmT3S8DzFK+uVAkbCdiPR0Bqei8qIGmUeY0=
github.com/elastic/bayeux v1.0.5/go.mod h1:CSI4iP7qeo5MMlkznGvYKftp8M7qqP/3nzmVZoXHY68=
github.com/elastic/dhcp v0.0.0-20200227161230-57ec251c7eb3 h1:lnDkqiRFKm0rxdljqrj3lotWinO9+jFmeDXIC4gvIQs=
github.com/elastic/dhcp v0.0.0-20200227161230-57ec251c7eb3/go.mod h1:aPqzac6AYkipvp4hufTyMj5PDIphF3+At8zr7r51xjY=
github.com/elastic/elastic-agent-autodiscover v0.4.0 h1:R1JMLHQpH2KP3GXY8zmgV4dj39uoe1asyPPWGQbGgSk=
github.com/elastic/elastic-agent-autodiscover v0.4.0/go.mod h1:p3MSf9813JEnolCTD0GyVAr3+Eptg2zQ9aZVFjl4tJ4=
github.com/elastic/elastic-agent-autodiscover v0.5.0 h1:fiDJQKB148RsRISqqbGINtJ/ZPupeVjwn0fbPz9ZDOc=
github.com/elastic/elastic-agent-autodiscover v0.5.0/go.mod h1:p3MSf9813JEnolCTD0GyVAr3+Eptg2zQ9aZVFjl4tJ4=
github.com/elastic/elastic-agent-client/v7 v7.0.0-20221121201703-4b23a52d0ebe h1:fpM0Lw4JTA9XYbQMQhVrMzJ0GrwUaXHVSsrm5n/b23E=
github.com/elastic/elastic-agent-client/v7 v7.0.0-20221121201703-4b23a52d0ebe/go.mod h1:FEXUbFMfaV62S0CtJgD+FFHGY7+4o4fXkDicyONPSH8=
github.com/elastic/elastic-agent-libs v0.2.11/go.mod h1:chO3rtcLyGlKi9S0iGVZhYCzDfdDsAQYBc+ui588AFE=
Expand Down
6 changes: 2 additions & 4 deletions metricbeat/docs/modules/kubernetes.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,8 +237,7 @@ metricbeat.modules:
# If kube_config is not set, KUBECONFIG environment variable will be checked
# and if not present it will fall back to InCluster
#kube_config: ~/.kube/config
# To configure additionally node and namespace metadata, added to pod, service and container resource types,
# `add_resource_metadata` can be defined.
# To configure additionally node and namespace metadata `add_resource_metadata` can be defined.
# By default all labels will be included while annotations are not added by default.
# add_resource_metadata:
# namespace:
Expand Down Expand Up @@ -284,8 +283,7 @@ metricbeat.modules:
#kube_config: ~/.kube/config
# Set the namespace to watch for resources
#namespace: staging
# To configure additionally node and namespace metadata, added to pod, service and container resource types,
# `add_resource_metadata` can be defined.
# To configure additionally node and namespace metadata `add_resource_metadata` can be defined.
# By default all labels will be included while annotations are not added by default.
# add_resource_metadata:
# namespace:
Expand Down
6 changes: 2 additions & 4 deletions metricbeat/metricbeat.reference.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -506,8 +506,7 @@ metricbeat.modules:
# If kube_config is not set, KUBECONFIG environment variable will be checked
# and if not present it will fall back to InCluster
#kube_config: ~/.kube/config
# To configure additionally node and namespace metadata, added to pod, service and container resource types,
# `add_resource_metadata` can be defined.
# To configure additionally node and namespace metadata `add_resource_metadata` can be defined.
# By default all labels will be included while annotations are not added by default.
# add_resource_metadata:
# namespace:
Expand Down Expand Up @@ -553,8 +552,7 @@ metricbeat.modules:
#kube_config: ~/.kube/config
# Set the namespace to watch for resources
#namespace: staging
# To configure additionally node and namespace metadata, added to pod, service and container resource types,
# `add_resource_metadata` can be defined.
# To configure additionally node and namespace metadata `add_resource_metadata` can be defined.
# By default all labels will be included while annotations are not added by default.
# add_resource_metadata:
# namespace:
Expand Down
6 changes: 2 additions & 4 deletions metricbeat/module/kubernetes/_meta/config.reference.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@
# If kube_config is not set, KUBECONFIG environment variable will be checked
# and if not present it will fall back to InCluster
#kube_config: ~/.kube/config
# To configure additionally node and namespace metadata, added to pod, service and container resource types,
# `add_resource_metadata` can be defined.
# To configure additionally node and namespace metadata `add_resource_metadata` can be defined.
# By default all labels will be included while annotations are not added by default.
# add_resource_metadata:
# namespace:
Expand Down Expand Up @@ -70,8 +69,7 @@
#kube_config: ~/.kube/config
# Set the namespace to watch for resources
#namespace: staging
# To configure additionally node and namespace metadata, added to pod, service and container resource types,
# `add_resource_metadata` can be defined.
# To configure additionally node and namespace metadata `add_resource_metadata` can be defined.
# By default all labels will be included while annotations are not added by default.
# add_resource_metadata:
# namespace:
Expand Down
3 changes: 1 addition & 2 deletions metricbeat/module/kubernetes/_meta/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,7 @@
#kube_config: ~/.kube/config
# Set the namespace to watch for resources
#namespace: staging
# To configure additionally node and namespace metadata, added to pod, service and container resource types,
# `add_resource_metadata` can be defined.
# To configure additionally node and namespace metadata `add_resource_metadata` can be defined.
# By default all labels will be included while annotations are not added by default.
# add_resource_metadata:
# namespace:
Expand Down
4 changes: 3 additions & 1 deletion metricbeat/module/kubernetes/util/kubernetes.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,11 +105,13 @@ func NewResourceMetadataEnricher(
}
cfg, _ := conf.NewConfigFrom(&commonMetaConfig)

metaGen := metadata.NewResourceMetadataGenerator(cfg, watcher.Client())
podMetaGen := metadata.GetPodMetaGen(cfg, watcher, nodeWatcher, namespaceWatcher, config.AddResourceMetadata)

namespaceMeta := metadata.NewNamespaceMetadataGenerator(config.AddResourceMetadata.Namespace, namespaceWatcher.Store(), watcher.Client())
serviceMetaGen := metadata.NewServiceMetadataGenerator(cfg, watcher.Store(), namespaceMeta, watcher.Client())

metaGen := metadata.NewNamespaceAwareResourceMetadataGenerator(cfg, watcher.Client(), namespaceMeta)

enricher := buildMetadataEnricher(watcher, nodeWatcher, namespaceWatcher,
// update
func(m map[string]mapstr.M, r kubernetes.Resource) {
Expand Down
3 changes: 1 addition & 2 deletions metricbeat/modules.d/kubernetes.yml.disabled
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,7 @@
#kube_config: ~/.kube/config
# Set the namespace to watch for resources
#namespace: staging
# To configure additionally node and namespace metadata, added to pod, service and container resource types,
# `add_resource_metadata` can be defined.
# To configure additionally node and namespace metadata `add_resource_metadata` can be defined.
# By default all labels will be included while annotations are not added by default.
# add_resource_metadata:
# namespace:
Expand Down
6 changes: 2 additions & 4 deletions x-pack/metricbeat/metricbeat.reference.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -888,8 +888,7 @@ metricbeat.modules:
# If kube_config is not set, KUBECONFIG environment variable will be checked
# and if not present it will fall back to InCluster
#kube_config: ~/.kube/config
# To configure additionally node and namespace metadata, added to pod, service and container resource types,
# `add_resource_metadata` can be defined.
# To configure additionally node and namespace metadata `add_resource_metadata` can be defined.
# By default all labels will be included while annotations are not added by default.
# add_resource_metadata:
# namespace:
Expand Down Expand Up @@ -935,8 +934,7 @@ metricbeat.modules:
#kube_config: ~/.kube/config
# Set the namespace to watch for resources
#namespace: staging
# To configure additionally node and namespace metadata, added to pod, service and container resource types,
# `add_resource_metadata` can be defined.
# To configure additionally node and namespace metadata `add_resource_metadata` can be defined.
# By default all labels will be included while annotations are not added by default.
# add_resource_metadata:
# namespace:
Expand Down