win custom metrics #1221
Merged
win custom metrics #1221
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jatakiajanvi12
changed the base branch from
user/amagraw/custom-metrics-migration
to
ci_prod
jatakiajanvi12
changed the base branch from
ci_prod
to
user/amagraw/custom-metrics-migration
jatakiajanvi12
force-pushed
the
jajataki/win_custom_metrics
branch
from
1a5b5d0
to
c0c3e6c
Compare
pfrcks
reviewed
Mar 29, 2024
| @@ -22,4 +22,11 @@ | |||
| flush_thread_count 5 | |||
| </buffer> | |||
| keepalive true | |||
There was a problem hiding this comment.
am I correct to assume that this file is inserted into rs fluentd config if AZMON_WINDOWS_FLUENT_BIT_DISABLED is disabled?
pfrcks
approved these changes
Mar 29, 2024
jatakiajanvi12
merged commit 7f5991e
into
user/amagraw/custom-metrics-migration
2 checks passed
pfrcks
pushed a commit
that referenced
this pull request
Apr 6, 2024
move LOGS_AND_EVENTS_ONLY telemetry to DS (#1212) * move LOGS_AND_EVENTS_ONLY telemetry to DS * Add CVE-2024-24557 to trivyignore --------- Co-authored-by: Amol Agrawal <[email protected]> fix input plugin bugs (#1207) * fix input plugin bugs --------- Co-authored-by: Amol Agrawal <[email protected]> collect logs from pods from excluded system ns (#1138) (#1146) * collect logs from pods from excluded system ns Enable disable addon test pipeline (#1208) * Update daily_addons_enablement_test.yaml for Azure Pipelines * Bump github.com/docker/docker in /source/plugins/go/input (#1194) Bumps [github.com/docker/docker](https://github.com/docker/docker) from 24.0.6+incompatible to 24.0.7+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](moby/moby@v24.0.6...v24.0.7) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * start telegraf only when fluent tcp listener up and running (#1205) * start telegraf only when fluent tcp listener up and running * remove duplicate plugin * update trivy ignore * consistent with linux * Containerlogv2 Metadata and annotations based filtering (#1148) * add fluent-bit kubernetes filter (#1115) * add fluent-bit kubernetes filter --------- Co-authored-by: Amol Agrawal <[email protected]> * add metadata feilds including labels, annotations, uid and image (#1120) * add metadata feilds including labels, annotations, uid and image * set the include_fields if customer set it otherwise go with default if enabled * fix minor bugs * change placeholder * add exclude specific kubernetes config * update tls verify for linux * make log filtering opt in explicit (#1140) Co-authored-by: Amol Agrawal <[email protected]> * Longw/metadata containerlogv2 kubernetes (#1139) * add metadata feilds including labels, annotations, uid and image * rename to podUid and add nil check * set the include_fields if customer set it otherwise go with default if enabled * add workload to testing clusters * containerlogv2 metadata backend change with feature flag on * adjust the marshal and log the output * address comments add check and scenario for empty list * remove ADX support * go fmt for better format * fix space in tomlparser.rb * update trivy ignore * fix Kube_Tag_Prefix for windows * update trivy * add ttl for metadata cache * update containerlogv2 test workloads * change typo and update tests * Longw/metadata containerlogv2 address feedback (#1166) * add argument for sendMetric * update SendMetric * update SendMetric * Longw/metadata containerlogv2 address feedback2 (#1186) * extend to support more configs on metadata * add warning message if feilds not match * add plugin for geneva path * add kubernetesMetadataCollection for geneva path * Longw/metadata containerlogv2 address feedback2 (#1188) * remove send metric * update trivy * Longw/metadata containerlogv2 address feedback3 (#1195) * update doc for containerlogv2 linux * add kube_meta_cache_ttl to agent settings * update logic to adjust colonLocation as the end of the list if it is not found * address comments and feedback * update conf file for geneva * adjust geneva and add new metrics * address feedback for geneva path * add telemetry in go * address comments for geneva conf filter config --------- Co-authored-by: Amol Agrawal <[email protected]> Co-authored-by: Amol Agrawal <[email protected]> * move LOGS_AND_EVENTS_ONLY telemetry to DS (#1212) * move LOGS_AND_EVENTS_ONLY telemetry to DS * Add CVE-2024-24557 to trivyignore --------- Co-authored-by: Amol Agrawal <[email protected]> * fix input plugin bugs (#1207) * fix input plugin bugs --------- Co-authored-by: Amol Agrawal <[email protected]> * collect logs from pods from excluded system ns (#1138) (#1146) * collect logs from pods from excluded system ns --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Janvi Jatakia (from Dev Box) <[email protected]> Co-authored-by: Ganga Mahesh Siddem <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Long Wan <[email protected]> Co-authored-by: Amol Agrawal <[email protected]> Co-authored-by: Amol Agrawal <[email protected]> Update the pipeline to not trigger for builds (#1214) Co-authored-by: Janvi Jatakia (from Dev Box) <[email protected]> Gangams/add flag to enable service account timebound token (#1213) * add flag for service account timebound token * fix pr feedback * fix pr feedback Solving pipeline bugs for windows ama url variable (#1215) Co-authored-by: Janvi Jatakia (from Dev Box) <[email protected]> Bump github.com/docker/docker in /source/plugins/go/input (#1209) Bumps [github.com/docker/docker](https://github.com/docker/docker) from 24.0.7+incompatible to 24.0.9+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](moby/moby@v24.0.7...v24.0.9) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Add telemetry for KubernetesMetadata size (#1216) * add telemetry for KubernetesMetadata size * go fmt for style * add condition check with KubernetesMetadataEnabled use single output change tags for combined output fix bugs update win conf fix formatting pv metrics fix formatting pv metrics (#1219) Co-authored-by: Amol Agrawal <[email protected]> update logPaths for tests and fix TODOs Dont start fluentd for MSI mode and fluentbit enabled (#1221) Co-authored-by: Janvi Jatakia (from Dev Box) <[email protected]> address PR comments change test log extension Add release notes for 3.1.19 (#1220) Containerlogv2 Kubernetes Metadata Grafana Dashboard Private Preview (#1218) * Containerlogv2 Kubernetes Metadata Grafana Dashboard Private Preview * update dashboard * dashboard cleanup * update raw template * template cleanup * update Update the start fluentd condition (#1223) Co-authored-by: Janvi Jatakia (from Dev Box) <[email protected]> add more checks add debug statement better logging Add release notes for 3.1.19 (#1220) Containerlogv2 Kubernetes Metadata Grafana Dashboard Private Preview (#1218) * Containerlogv2 Kubernetes Metadata Grafana Dashboard Private Preview * update dashboard * dashboard cleanup * update raw template * template cleanup * update Update the start fluentd condition (#1223) Co-authored-by: Janvi Jatakia (from Dev Box) <[email protected]> add more checks add debug statement better logging
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request involves changes to the configuration and behavior of Fluentd and Fluent Bit in a Kubernetes environment. The most critical changes include the relocation of the
custom_metrics_mdmfilter plugin for performance data from Windows nodes, and the conditional start of Fluentd as a Windows service based on certain environment variables.Here are the key changes:
Configuration Changes:
build/linux/installer/conf/kube.conf: Removed thecustom_metrics_mdmfilter plugin for performance data from Windows nodes. This plugin was responsible for collecting certain metrics likecpuUsageNanoCores,memoryWorkingSetBytes, andpvUsedBytes.build/linux/installer/conf/windows_rs_containerinventory.conf: Added thecustom_metrics_mdmfilter plugin for performance data from Windows nodes. This is the same plugin that was removed from thekube.conffile.Behavior Changes:
kubernetes/windows/main.ps1: Modified theStart-Fluent-Telegraffunction to conditionally start Fluentd as a Windows service. The service will start if theAZMON_WINDOWS_FLUENT_BIT_DISABLEDenvironment variable is not set to 'false', or if theUSING_AAD_MSI_AUTHenvironment variable is not set to 'true', or if thegenevaLogsIntegrationvariable is set to 'true'.These changes suggest a shift in the strategy for collecting performance data from Windows nodes in the Kubernetes environment.