add metadata feilds including labels, annotations, uid and image #1120
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pfrcks
reviewed
Nov 17, 2023
pfrcks
reviewed
Nov 17, 2023
pfrcks
reviewed
Nov 17, 2023
|
This PR is stale because it has been open 7 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
wanlonghenry
merged commit c83aed3
into
develop/fluent-bit-kubernetes-plugin
2 checks passed
pfrcks
reviewed
Nov 28, 2023
| @@ -211,6 +212,8 @@ def populateSettingValuesFromConfigMap(parsedConfig) | |||
| begin | |||
| if !parsedConfig[:log_collection_settings][:metadata_collection].nil? && !parsedConfig[:log_collection_settings][:metadata_collection][:enabled].nil? | |||
| @logEnableKubernetesMetadata = parsedConfig[:log_collection_settings][:metadata_collection][:enabled] | |||
| if !parsedConfig[:log_collection_settings][:metadata_collection][:include_fields].nil? | |||
| @logKubernetesMetadataiIncludeFields = parsedConfig[:log_collection_settings][:metadata_collection][:include_fields] | |||
There was a problem hiding this comment.
missing end for the if block. also add a puts statement inside the block "config::Using config map setting for kubernetes metadata include fields"
pfrcks
reviewed
Nov 28, 2023
| @@ -211,6 +212,8 @@ def populateSettingValuesFromConfigMap(parsedConfig) | |||
| begin | |||
| if !parsedConfig[:log_collection_settings][:metadata_collection].nil? && !parsedConfig[:log_collection_settings][:metadata_collection][:enabled].nil? | |||
| @logEnableKubernetesMetadata = parsedConfig[:log_collection_settings][:metadata_collection][:enabled] | |||
| if !parsedConfig[:log_collection_settings][:metadata_collection][:include_fields].nil? | |||
| @logKubernetesMetadataiIncludeFields = parsedConfig[:log_collection_settings][:metadata_collection][:include_fields] | |||
| puts "config::Using config map setting for kubernetes metadata" | |||
There was a problem hiding this comment.
shift this line to above line 214
wanlonghenry
added a commit
that referenced
this pull request
Mar 19, 2024
* add fluent-bit kubernetes filter (#1115) * add fluent-bit kubernetes filter --------- Co-authored-by: Amol Agrawal <[email protected]> * add metadata feilds including labels, annotations, uid and image (#1120) * add metadata feilds including labels, annotations, uid and image * set the include_fields if customer set it otherwise go with default if enabled * fix minor bugs * change placeholder * add exclude specific kubernetes config * update tls verify for linux * make log filtering opt in explicit (#1140) Co-authored-by: Amol Agrawal <[email protected]> * Longw/metadata containerlogv2 kubernetes (#1139) * add metadata feilds including labels, annotations, uid and image * rename to podUid and add nil check * set the include_fields if customer set it otherwise go with default if enabled * add workload to testing clusters * containerlogv2 metadata backend change with feature flag on * adjust the marshal and log the output * address comments add check and scenario for empty list * remove ADX support * go fmt for better format * fix space in tomlparser.rb * update trivy ignore * fix Kube_Tag_Prefix for windows * update trivy * add ttl for metadata cache * update containerlogv2 test workloads * change typo and update tests * Longw/metadata containerlogv2 address feedback (#1166) * add argument for sendMetric * update SendMetric * update SendMetric * Longw/metadata containerlogv2 address feedback2 (#1186) * extend to support more configs on metadata * add warning message if feilds not match * add plugin for geneva path * add kubernetesMetadataCollection for geneva path * Longw/metadata containerlogv2 address feedback2 (#1188) * remove send metric * update trivy * Longw/metadata containerlogv2 address feedback3 (#1195) * update doc for containerlogv2 linux * add kube_meta_cache_ttl to agent settings * update logic to adjust colonLocation as the end of the list if it is not found * address comments and feedback * update conf file for geneva * adjust geneva and add new metrics * address feedback for geneva path * add telemetry in go * address comments for geneva conf filter config --------- Co-authored-by: Amol Agrawal <[email protected]> Co-authored-by: Amol Agrawal <[email protected]>
jatakiajanvi12
added a commit
that referenced
this pull request
Mar 22, 2024
* Update daily_addons_enablement_test.yaml for Azure Pipelines * Bump github.com/docker/docker in /source/plugins/go/input (#1194) Bumps [github.com/docker/docker](https://github.com/docker/docker) from 24.0.6+incompatible to 24.0.7+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](moby/moby@v24.0.6...v24.0.7) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * start telegraf only when fluent tcp listener up and running (#1205) * start telegraf only when fluent tcp listener up and running * remove duplicate plugin * update trivy ignore * consistent with linux * Containerlogv2 Metadata and annotations based filtering (#1148) * add fluent-bit kubernetes filter (#1115) * add fluent-bit kubernetes filter --------- Co-authored-by: Amol Agrawal <[email protected]> * add metadata feilds including labels, annotations, uid and image (#1120) * add metadata feilds including labels, annotations, uid and image * set the include_fields if customer set it otherwise go with default if enabled * fix minor bugs * change placeholder * add exclude specific kubernetes config * update tls verify for linux * make log filtering opt in explicit (#1140) Co-authored-by: Amol Agrawal <[email protected]> * Longw/metadata containerlogv2 kubernetes (#1139) * add metadata feilds including labels, annotations, uid and image * rename to podUid and add nil check * set the include_fields if customer set it otherwise go with default if enabled * add workload to testing clusters * containerlogv2 metadata backend change with feature flag on * adjust the marshal and log the output * address comments add check and scenario for empty list * remove ADX support * go fmt for better format * fix space in tomlparser.rb * update trivy ignore * fix Kube_Tag_Prefix for windows * update trivy * add ttl for metadata cache * update containerlogv2 test workloads * change typo and update tests * Longw/metadata containerlogv2 address feedback (#1166) * add argument for sendMetric * update SendMetric * update SendMetric * Longw/metadata containerlogv2 address feedback2 (#1186) * extend to support more configs on metadata * add warning message if feilds not match * add plugin for geneva path * add kubernetesMetadataCollection for geneva path * Longw/metadata containerlogv2 address feedback2 (#1188) * remove send metric * update trivy * Longw/metadata containerlogv2 address feedback3 (#1195) * update doc for containerlogv2 linux * add kube_meta_cache_ttl to agent settings * update logic to adjust colonLocation as the end of the list if it is not found * address comments and feedback * update conf file for geneva * adjust geneva and add new metrics * address feedback for geneva path * add telemetry in go * address comments for geneva conf filter config --------- Co-authored-by: Amol Agrawal <[email protected]> Co-authored-by: Amol Agrawal <[email protected]> * move LOGS_AND_EVENTS_ONLY telemetry to DS (#1212) * move LOGS_AND_EVENTS_ONLY telemetry to DS * Add CVE-2024-24557 to trivyignore --------- Co-authored-by: Amol Agrawal <[email protected]> * fix input plugin bugs (#1207) * fix input plugin bugs --------- Co-authored-by: Amol Agrawal <[email protected]> * collect logs from pods from excluded system ns (#1138) (#1146) * collect logs from pods from excluded system ns --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Janvi Jatakia (from Dev Box) <[email protected]> Co-authored-by: Ganga Mahesh Siddem <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Long Wan <[email protected]> Co-authored-by: Amol Agrawal <[email protected]> Co-authored-by: Amol Agrawal <[email protected]>
pfrcks
pushed a commit
that referenced
this pull request
Apr 6, 2024
move LOGS_AND_EVENTS_ONLY telemetry to DS (#1212) * move LOGS_AND_EVENTS_ONLY telemetry to DS * Add CVE-2024-24557 to trivyignore --------- Co-authored-by: Amol Agrawal <[email protected]> fix input plugin bugs (#1207) * fix input plugin bugs --------- Co-authored-by: Amol Agrawal <[email protected]> collect logs from pods from excluded system ns (#1138) (#1146) * collect logs from pods from excluded system ns Enable disable addon test pipeline (#1208) * Update daily_addons_enablement_test.yaml for Azure Pipelines * Bump github.com/docker/docker in /source/plugins/go/input (#1194) Bumps [github.com/docker/docker](https://github.com/docker/docker) from 24.0.6+incompatible to 24.0.7+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](moby/moby@v24.0.6...v24.0.7) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * start telegraf only when fluent tcp listener up and running (#1205) * start telegraf only when fluent tcp listener up and running * remove duplicate plugin * update trivy ignore * consistent with linux * Containerlogv2 Metadata and annotations based filtering (#1148) * add fluent-bit kubernetes filter (#1115) * add fluent-bit kubernetes filter --------- Co-authored-by: Amol Agrawal <[email protected]> * add metadata feilds including labels, annotations, uid and image (#1120) * add metadata feilds including labels, annotations, uid and image * set the include_fields if customer set it otherwise go with default if enabled * fix minor bugs * change placeholder * add exclude specific kubernetes config * update tls verify for linux * make log filtering opt in explicit (#1140) Co-authored-by: Amol Agrawal <[email protected]> * Longw/metadata containerlogv2 kubernetes (#1139) * add metadata feilds including labels, annotations, uid and image * rename to podUid and add nil check * set the include_fields if customer set it otherwise go with default if enabled * add workload to testing clusters * containerlogv2 metadata backend change with feature flag on * adjust the marshal and log the output * address comments add check and scenario for empty list * remove ADX support * go fmt for better format * fix space in tomlparser.rb * update trivy ignore * fix Kube_Tag_Prefix for windows * update trivy * add ttl for metadata cache * update containerlogv2 test workloads * change typo and update tests * Longw/metadata containerlogv2 address feedback (#1166) * add argument for sendMetric * update SendMetric * update SendMetric * Longw/metadata containerlogv2 address feedback2 (#1186) * extend to support more configs on metadata * add warning message if feilds not match * add plugin for geneva path * add kubernetesMetadataCollection for geneva path * Longw/metadata containerlogv2 address feedback2 (#1188) * remove send metric * update trivy * Longw/metadata containerlogv2 address feedback3 (#1195) * update doc for containerlogv2 linux * add kube_meta_cache_ttl to agent settings * update logic to adjust colonLocation as the end of the list if it is not found * address comments and feedback * update conf file for geneva * adjust geneva and add new metrics * address feedback for geneva path * add telemetry in go * address comments for geneva conf filter config --------- Co-authored-by: Amol Agrawal <[email protected]> Co-authored-by: Amol Agrawal <[email protected]> * move LOGS_AND_EVENTS_ONLY telemetry to DS (#1212) * move LOGS_AND_EVENTS_ONLY telemetry to DS * Add CVE-2024-24557 to trivyignore --------- Co-authored-by: Amol Agrawal <[email protected]> * fix input plugin bugs (#1207) * fix input plugin bugs --------- Co-authored-by: Amol Agrawal <[email protected]> * collect logs from pods from excluded system ns (#1138) (#1146) * collect logs from pods from excluded system ns --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Janvi Jatakia (from Dev Box) <[email protected]> Co-authored-by: Ganga Mahesh Siddem <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Long Wan <[email protected]> Co-authored-by: Amol Agrawal <[email protected]> Co-authored-by: Amol Agrawal <[email protected]> Update the pipeline to not trigger for builds (#1214) Co-authored-by: Janvi Jatakia (from Dev Box) <[email protected]> Gangams/add flag to enable service account timebound token (#1213) * add flag for service account timebound token * fix pr feedback * fix pr feedback Solving pipeline bugs for windows ama url variable (#1215) Co-authored-by: Janvi Jatakia (from Dev Box) <[email protected]> Bump github.com/docker/docker in /source/plugins/go/input (#1209) Bumps [github.com/docker/docker](https://github.com/docker/docker) from 24.0.7+incompatible to 24.0.9+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](moby/moby@v24.0.7...v24.0.9) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Add telemetry for KubernetesMetadata size (#1216) * add telemetry for KubernetesMetadata size * go fmt for style * add condition check with KubernetesMetadataEnabled use single output change tags for combined output fix bugs update win conf fix formatting pv metrics fix formatting pv metrics (#1219) Co-authored-by: Amol Agrawal <[email protected]> update logPaths for tests and fix TODOs Dont start fluentd for MSI mode and fluentbit enabled (#1221) Co-authored-by: Janvi Jatakia (from Dev Box) <[email protected]> address PR comments change test log extension Add release notes for 3.1.19 (#1220) Containerlogv2 Kubernetes Metadata Grafana Dashboard Private Preview (#1218) * Containerlogv2 Kubernetes Metadata Grafana Dashboard Private Preview * update dashboard * dashboard cleanup * update raw template * template cleanup * update Update the start fluentd condition (#1223) Co-authored-by: Janvi Jatakia (from Dev Box) <[email protected]> add more checks add debug statement better logging Add release notes for 3.1.19 (#1220) Containerlogv2 Kubernetes Metadata Grafana Dashboard Private Preview (#1218) * Containerlogv2 Kubernetes Metadata Grafana Dashboard Private Preview * update dashboard * dashboard cleanup * update raw template * template cleanup * update Update the start fluentd condition (#1223) Co-authored-by: Janvi Jatakia (from Dev Box) <[email protected]> add more checks add debug statement better logging
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
add metadata feilds including labels, annotations, uid and image
default include all the feilds