Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(kafka plugins): Add metadata full to kafka config #9833

Merged
merged 1 commit into from
Oct 21, 2021
Merged

feat(kafka plugins): Add metadata full to kafka config #9833

merged 1 commit into from
Oct 21, 2021

Conversation

gquintana
Copy link
Contributor

@gquintana gquintana commented Sep 28, 2021 •
edited
Loading

Required for all PRs:

  • Updated associated README.md.
  • Wrote appropriate unit tests.
  • Pull request title or commits are in conventional commit format (e.g. feat: or fix:)

resolves #9825

Adds a metadata_full setting to kafka input/output in order to be able to disable full Kafka cluster metadata fetching.

@telegraf-tiger
Copy link
Contributor

Thanks so much for the pull request!
🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

@telegraf-tiger telegraf-tiger bot added the feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin label Sep 28, 2021
@gquintana gquintana force-pushed the feat/kafka-metadata-full-setting branch 2 times, most recently from 07fd437 to d77b6ef Compare September 28, 2021 14:37
@gquintana gquintana force-pushed the feat/kafka-metadata-full-setting branch from d77b6ef to e09f918 Compare September 28, 2021 15:15
srebhan
srebhan approved these changes Oct 20, 2021
View reviewed changes
Copy link
Member

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks for the contribution @gquintana!

@srebhan srebhan self-assigned this Oct 20, 2021
@srebhan srebhan added area/kafka ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. labels Oct 20, 2021
@MyaLongmire MyaLongmire merged commit 8265969 into influxdata:master Oct 21, 2021
phemmer added a commit to phemmer/telegraf that referenced this pull request Oct 26, 2021
@phemmer
Merge remote-tracking branch 'origin/master' into postgres
674c971 
* origin/master: (176 commits)
  fix: Linter fixes for plugins/inputs/[h-j]* (influxdata#9986)
  fix(inputs/kube_inventory): don't skip resources with zero s/ns timestamps (influxdata#9978)
  fix: update gjson to v1.10.2 (influxdata#9998)
  fix: procstat tags were not getting generated correctly (influxdata#9973)
  chore: create bug report form (influxdata#9976)
  fix: Allow for non x86 macs in Go install script (influxdata#9982)
  test: add sqlserver plugin integration tests (influxdata#9943)
  feat: plugins/common/tls/config.go: Filter client certificates by DNS names (influxdata#9910)
  feat: add option to skip table creation in azure data explorer output (influxdata#9942)
  docs: update nightlies links (influxdata#9989)
  fix: add s390x to nightlies (influxdata#9990)
  feat: Add more details to processors.ifname logmessages (influxdata#9984)
  docs: Create SECURITY.md (influxdata#9951)
  fix: set NIGHTLY=1 for correctly named nightly artifacts (influxdata#9987)
  feat: Kafka Add metadata full to config (influxdata#9833)
  chore: Update to AWS SDK v2 (influxdata#9647)
  chore: lint ignore fmt.Printf unhandled error (influxdata#9967)
  fix: starlark pop operation for non-existing keys (influxdata#9954)
  feat: update etc/telegraf.conf and etc/telegraf_windows.conf (influxdata#9876)
  fix: Check return code of zfs command for FreeBSD. (influxdata#9956)
  chore: update go to 1.17.2 (influxdata#9873)
  fix: Graylog plugin TLS support and message format (influxdata#9862)
  docs: update README with info on package repos (influxdata#9964)
  feat: Modbus connection settings (serial) (influxdata#9256)
  fix: segfault in ingress, persistentvolumeclaim, statefulset in kube_inventory (influxdata#9585)
  fix: add normalization of tags for ethtool input plugin (influxdata#9901)
  chore: remove empty build.py (influxdata#9958)
  fix: internet_speed input plugin not collecting/reporting latency (influxdata#9957)
  chore: reference db2 external plugin (influxdata#9952)
  chore: update readme go version from 1.14 to 1.17 (influxdata#9944)
  fix: decode Prometheus scrape path from Kuberentes labels (influxdata#9662)
  docs: fix broken link (influxdata#9812)
  fix: Correct conversion of int with specific bit size (influxdata#9933)
  fix: update golanci-lint to v1.42.1 (influxdata#9932)
  feat: Azure Event Hubs output plugin (influxdata#9346)
  feat: more fields for papertrail event webhook (influxdata#9940)
  fix: solve compatibility issue for mongodb inputs when using 5.x relicaset (influxdata#9892)
  docs: Add symlink to command documentation (influxdata#9926)
  docs: update contributing.md (influxdata#9914)
  chore: reference oracle external plugin (influxdata#9934)
  ...
phemmer added a commit to phemmer/telegraf that referenced this pull request Oct 26, 2021
@phemmer
Merge remote-tracking branch 'origin/master' into postgres
4552a53 
* origin/master: (176 commits)
  fix: Linter fixes for plugins/inputs/[h-j]* (influxdata#9986)
  fix(inputs/kube_inventory): don't skip resources with zero s/ns timestamps (influxdata#9978)
  fix: update gjson to v1.10.2 (influxdata#9998)
  fix: procstat tags were not getting generated correctly (influxdata#9973)
  chore: create bug report form (influxdata#9976)
  fix: Allow for non x86 macs in Go install script (influxdata#9982)
  test: add sqlserver plugin integration tests (influxdata#9943)
  feat: plugins/common/tls/config.go: Filter client certificates by DNS names (influxdata#9910)
  feat: add option to skip table creation in azure data explorer output (influxdata#9942)
  docs: update nightlies links (influxdata#9989)
  fix: add s390x to nightlies (influxdata#9990)
  feat: Add more details to processors.ifname logmessages (influxdata#9984)
  docs: Create SECURITY.md (influxdata#9951)
  fix: set NIGHTLY=1 for correctly named nightly artifacts (influxdata#9987)
  feat: Kafka Add metadata full to config (influxdata#9833)
  chore: Update to AWS SDK v2 (influxdata#9647)
  chore: lint ignore fmt.Printf unhandled error (influxdata#9967)
  fix: starlark pop operation for non-existing keys (influxdata#9954)
  feat: update etc/telegraf.conf and etc/telegraf_windows.conf (influxdata#9876)
  fix: Check return code of zfs command for FreeBSD. (influxdata#9956)
  chore: update go to 1.17.2 (influxdata#9873)
  fix: Graylog plugin TLS support and message format (influxdata#9862)
  docs: update README with info on package repos (influxdata#9964)
  feat: Modbus connection settings (serial) (influxdata#9256)
  fix: segfault in ingress, persistentvolumeclaim, statefulset in kube_inventory (influxdata#9585)
  fix: add normalization of tags for ethtool input plugin (influxdata#9901)
  chore: remove empty build.py (influxdata#9958)
  fix: internet_speed input plugin not collecting/reporting latency (influxdata#9957)
  chore: reference db2 external plugin (influxdata#9952)
  chore: update readme go version from 1.14 to 1.17 (influxdata#9944)
  fix: decode Prometheus scrape path from Kuberentes labels (influxdata#9662)
  docs: fix broken link (influxdata#9812)
  fix: Correct conversion of int with specific bit size (influxdata#9933)
  fix: update golanci-lint to v1.42.1 (influxdata#9932)
  feat: Azure Event Hubs output plugin (influxdata#9346)
  feat: more fields for papertrail event webhook (influxdata#9940)
  fix: solve compatibility issue for mongodb inputs when using 5.x relicaset (influxdata#9892)
  docs: Add symlink to command documentation (influxdata#9926)
  docs: update contributing.md (influxdata#9914)
  chore: reference oracle external plugin (influxdata#9934)
  ...
VladislavSenkevich pushed a commit to gwos/telegraf that referenced this pull request Nov 23, 2021
@gquintana
feat: Kafka Add metadata full to config (influxdata#9833)
f830919
phemmer added a commit to phemmer/telegraf that referenced this pull request Nov 30, 2021
@phemmer
Merge remote-tracking branch 'origin/master' into postgres
7f8a962 
* origin/master: (176 commits)
  fix: Linter fixes for plugins/inputs/[h-j]* (influxdata#9986)
  fix(inputs/kube_inventory): don't skip resources with zero s/ns timestamps (influxdata#9978)
  fix: update gjson to v1.10.2 (influxdata#9998)
  fix: procstat tags were not getting generated correctly (influxdata#9973)
  chore: create bug report form (influxdata#9976)
  fix: Allow for non x86 macs in Go install script (influxdata#9982)
  test: add sqlserver plugin integration tests (influxdata#9943)
  feat: plugins/common/tls/config.go: Filter client certificates by DNS names (influxdata#9910)
  feat: add option to skip table creation in azure data explorer output (influxdata#9942)
  docs: update nightlies links (influxdata#9989)
  fix: add s390x to nightlies (influxdata#9990)
  feat: Add more details to processors.ifname logmessages (influxdata#9984)
  docs: Create SECURITY.md (influxdata#9951)
  fix: set NIGHTLY=1 for correctly named nightly artifacts (influxdata#9987)
  feat: Kafka Add metadata full to config (influxdata#9833)
  chore: Update to AWS SDK v2 (influxdata#9647)
  chore: lint ignore fmt.Printf unhandled error (influxdata#9967)
  fix: starlark pop operation for non-existing keys (influxdata#9954)
  feat: update etc/telegraf.conf and etc/telegraf_windows.conf (influxdata#9876)
  fix: Check return code of zfs command for FreeBSD. (influxdata#9956)
  chore: update go to 1.17.2 (influxdata#9873)
  fix: Graylog plugin TLS support and message format (influxdata#9862)
  docs: update README with info on package repos (influxdata#9964)
  feat: Modbus connection settings (serial) (influxdata#9256)
  fix: segfault in ingress, persistentvolumeclaim, statefulset in kube_inventory (influxdata#9585)
  fix: add normalization of tags for ethtool input plugin (influxdata#9901)
  chore: remove empty build.py (influxdata#9958)
  fix: internet_speed input plugin not collecting/reporting latency (influxdata#9957)
  chore: reference db2 external plugin (influxdata#9952)
  chore: update readme go version from 1.14 to 1.17 (influxdata#9944)
  fix: decode Prometheus scrape path from Kuberentes labels (influxdata#9662)
  docs: fix broken link (influxdata#9812)
  fix: Correct conversion of int with specific bit size (influxdata#9933)
  fix: update golanci-lint to v1.42.1 (influxdata#9932)
  feat: Azure Event Hubs output plugin (influxdata#9346)
  feat: more fields for papertrail event webhook (influxdata#9940)
  fix: solve compatibility issue for mongodb inputs when using 5.x relicaset (influxdata#9892)
  docs: Add symlink to command documentation (influxdata#9926)
  docs: update contributing.md (influxdata#9914)
  chore: reference oracle external plugin (influxdata#9934)
  ...
phemmer added a commit to phemmer/telegraf that referenced this pull request Jan 3, 2022
@phemmer
Merge remote-tracking branch 'origin/master' into postgres
02ee794 
* origin/master: (176 commits)
  fix: Linter fixes for plugins/inputs/[h-j]* (influxdata#9986)
  fix(inputs/kube_inventory): don't skip resources with zero s/ns timestamps (influxdata#9978)
  fix: update gjson to v1.10.2 (influxdata#9998)
  fix: procstat tags were not getting generated correctly (influxdata#9973)
  chore: create bug report form (influxdata#9976)
  fix: Allow for non x86 macs in Go install script (influxdata#9982)
  test: add sqlserver plugin integration tests (influxdata#9943)
  feat: plugins/common/tls/config.go: Filter client certificates by DNS names (influxdata#9910)
  feat: add option to skip table creation in azure data explorer output (influxdata#9942)
  docs: update nightlies links (influxdata#9989)
  fix: add s390x to nightlies (influxdata#9990)
  feat: Add more details to processors.ifname logmessages (influxdata#9984)
  docs: Create SECURITY.md (influxdata#9951)
  fix: set NIGHTLY=1 for correctly named nightly artifacts (influxdata#9987)
  feat: Kafka Add metadata full to config (influxdata#9833)
  chore: Update to AWS SDK v2 (influxdata#9647)
  chore: lint ignore fmt.Printf unhandled error (influxdata#9967)
  fix: starlark pop operation for non-existing keys (influxdata#9954)
  feat: update etc/telegraf.conf and etc/telegraf_windows.conf (influxdata#9876)
  fix: Check return code of zfs command for FreeBSD. (influxdata#9956)
  chore: update go to 1.17.2 (influxdata#9873)
  fix: Graylog plugin TLS support and message format (influxdata#9862)
  docs: update README with info on package repos (influxdata#9964)
  feat: Modbus connection settings (serial) (influxdata#9256)
  fix: segfault in ingress, persistentvolumeclaim, statefulset in kube_inventory (influxdata#9585)
  fix: add normalization of tags for ethtool input plugin (influxdata#9901)
  chore: remove empty build.py (influxdata#9958)
  fix: internet_speed input plugin not collecting/reporting latency (influxdata#9957)
  chore: reference db2 external plugin (influxdata#9952)
  chore: update readme go version from 1.14 to 1.17 (influxdata#9944)
  fix: decode Prometheus scrape path from Kuberentes labels (influxdata#9662)
  docs: fix broken link (influxdata#9812)
  fix: Correct conversion of int with specific bit size (influxdata#9933)
  fix: update golanci-lint to v1.42.1 (influxdata#9932)
  feat: Azure Event Hubs output plugin (influxdata#9346)
  feat: more fields for papertrail event webhook (influxdata#9940)
  fix: solve compatibility issue for mongodb inputs when using 5.x relicaset (influxdata#9892)
  docs: Add symlink to command documentation (influxdata#9926)
  docs: update contributing.md (influxdata#9914)
  chore: reference oracle external plugin (influxdata#9934)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sspaink sspaink sspaink approved these changes

@srebhan srebhan srebhan approved these changes

Assignees

@srebhan srebhan

Labels
area/kafka feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Kafka output metadata collection
4 participants
@gquintana @sspaink @srebhan @MyaLongmire