Salt support for splunk-otel-collector #1050

Mukeshr-Ram · 2021-12-17T16:23:20Z

Salt support for splunk-otel-connector
Ticket: OTL-1270 OTL-1358
Changes:

Install splunk-otel-collector and fluentd on all supported Linux environments
Updated documentation, README.md
Adding salt-test Github actions for salt deployment test which uses pytest.
Note:
Fluent YUM dependency package module pkgconfig has been changed to pkgconf-pkg-config on RHEL8 (https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/considerations_in_adopting_rhel_8/index). So, made changes based on package name.
With Salt's older version(e.g - 2019.2) we are not able to pass the custom version for all the packages (e.g. fluentd and splunk-otel-collector) as there may be a bug in the older version of salt(pkg.installed succeeded but says it failed saltstack/salt#27400 (comment)) we are using the latest Salt version. All the testings for this formula are done using the latest Salt version.

jeffreyc-splunk · 2021-12-21T17:08:19Z

internal/buildscripts/packaging/tests/deployments/salt/images/rpm/Dockerfile.opensuse-12

+
+ENV container docker
+
+RUN zypper -n install -l curl dbus-1 systemd-sysvinit tar wget python3-pip ca-certificates


Please include the 2 lines from https://github.com/signalfx/splunk-otel-collector/pull/1056/files that I added for the other tests.

Added these 2 lines in opensuse-12 Dockerfile which is used in the salt test.

jeffreyc-splunk · 2021-12-21T17:11:24Z

deployments/salt/Readme.md

@@ -0,0 +1,119 @@
+# Splunk OpenTelemetry Collector Salt Formula


Please rename this file to README.md.

Thanks, Updated

deployments/salt/Readme.md

deployments/salt/splunk-otel-collector/fluentd.sls

jeffreyc-splunk · 2021-12-21T17:22:44Z

deployments/salt/splunk-otel-collector/init.sls

+        exit 1
+{% else %}
+
+{% set install_fluentd = salt['pillar.get']('splunk-otel-collector:install_fluentd', 'True') %}


Should install_fluentd be a boolean instead of string? If so, we should also use https://docs.saltproject.io/en/latest/topics/jinja/index.html#to-bool to convert the value to boolean.

Yes, install_fluentd is a boolean variable.
As per your suggestion, I have updated the script to convert the value in boolean type with to_bool.
Thanks for the suggestion!

jeffreyc-splunk · 2021-12-21T17:38:53Z

deployments/salt/splunk-otel-collector/fluentd.sls

+    - require:
+      - pkg: Install FluentD Linux capability module dependencies
+
+Start FluentD service:


Please add a systemctl daemon-reload step before starting the td-agent service.

Thanks for the suggestions.
Added systemctl daemon-reload step.
From this, my understanding is when we are making any change in td-agent service config so we need to restart/reload the service of td-agent, hear we are doing that by reloading daemon. Correct me if I am wrong.

deployments/salt/Readme.md

Co-authored-by: Jeff Cheng <[email protected]>

theletterf

Tagging @tmorrisonsfx for docs updates.

deployments/salt/README.md

jeffreyc-splunk · 2022-01-10T16:01:00Z

.github/workflows/salt-test.yml

+      - name: Check out the codebase.
+        uses: actions/checkout@v2
+
+      - name: Lint


Let's move the lint step to a separate job since it doesn't need to be run twice.

Thanks for the suggestions.
Added salt-lint-test job

Sorry, I meant that the lint job should be separate from the test job, but they can both be in the same workflow. Also, please add the lint job as a dependency to the test job.

Thanks for the guidance!
I have added dependency on salt-test and salt-lint-test. can you please verify it

jeffreyc-splunk · 2022-01-10T16:18:57Z

deployments/salt/splunk-otel-collector/config.sls

+{% if splunk_otel_collector_config_source  != '' %}
+
+Push custom config for Splunk Otel Collector, if provided:
+  file.managed:


Please make sure the collector service is restarted if the config file has changed.

@jcheng-splunk
We tried using the Watch module for restarting the collector service however it's only working with files changed using Salt, So added the file.copy module to copy the default config. This ensures that the service will restart if the config file and environment file get changed.

jeffreyc-splunk · 2022-01-10T16:19:45Z

deployments/salt/splunk-otel-collector/collector_config.sls

+{% set splunk_ballast_size_mib = salt['pillar.get']('splunk-otel-collector:splunk_ballast_size_mib', '') %}
+
+/etc/otel/collector/splunk-otel-collector.conf:
+  file.managed:


Please make sure the collector service is restarted if this environment file has changed.

Thanks for the suggestions.
Updated with the suggested changes.

jeffreyc-splunk · 2022-01-10T16:23:06Z

deployments/salt/splunk-otel-collector/fluentd.sls

@@ -0,0 +1,115 @@
+{% set splunk_fluentd_config = salt['pillar.get']('splunk-otel-collector:splunk_fluentd_config', '/etc/otel/collector/fluentd/fluent.conf') %}


Please make sure that the td-agent service is restarted if the fluentd config file has changed.

We have added fluentd_service.sls file for restarting td-agent service if the fluentd config file has changed.

jeffreyc-splunk · 2022-01-10T16:39:34Z

deployments/salt/splunk-otel-collector/fluentd_config.sls

+{% if splunk_fluentd_config_source != '' %}
+
+Push custom FluentD config, if provided:
+  file.managed:


Please make sure that the td-agent service is restarted if the fluentd config has changed.

We have added fluentd_service.sls file for restarting td-agent service if the fluentd config file has changed.

Salt support for splunk-otel-collector

b23ed1e

Mukeshr-Ram requested review from a team as code owners December 17, 2021 16:23

Adding pytest scripts and GitHub actions for salt deployment test

8ccf398

jeffreyc-splunk reviewed Dec 21, 2021

View reviewed changes

nitaliya and others added 4 commits December 22, 2021 09:53

Update deployments/salt/Readme.md

a257af5

Co-authored-by: Jeff Cheng <[email protected]>

Update deployments/salt/splunk-otel-collector/fluentd.sls

2118b80

Co-authored-by: Jeff Cheng <[email protected]>

Update deployments/salt/Readme.md

dded238

Co-authored-by: Jeff Cheng <[email protected]>

Merge branch 'main' into salt

9b9ea55

nitaliya force-pushed the salt branch from 5f7071c to a7b68cb Compare December 22, 2021 07:08

nitaliya force-pushed the salt branch from a7b68cb to 1c4afcb Compare December 31, 2021 05:52

theletterf reviewed Dec 31, 2021

View reviewed changes

deployments/salt/README.md Outdated Show resolved Hide resolved

Making suggested changes.

953b3d1

nitaliya force-pushed the salt branch from bc19fc7 to 953b3d1 Compare December 31, 2021 09:25

jeffreyc-splunk reviewed Jan 10, 2022

View reviewed changes

nitaliya added 3 commits January 11, 2022 10:30

Making salt-lint-test job.

89208db

Making suggested changes.

022def3

Adding dependency on salt-test and salt-lint-test

fce2c36

jeffreyc-splunk approved these changes Jan 19, 2022

View reviewed changes

jeffreyc-splunk merged commit 2fffa8f into signalfx:main Jan 19, 2022

nitaliya deleted the salt branch January 20, 2022 07:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Salt support for splunk-otel-collector #1050

Salt support for splunk-otel-collector #1050

Mukeshr-Ram commented Dec 17, 2021 •

edited

Loading

jeffreyc-splunk Dec 21, 2021

nitaliya Dec 22, 2021

jeffreyc-splunk Dec 21, 2021

nitaliya Dec 22, 2021

jeffreyc-splunk Dec 21, 2021

nitaliya Dec 22, 2021 •

edited

Loading

jeffreyc-splunk Dec 21, 2021

nitaliya Dec 22, 2021

theletterf left a comment

jeffreyc-splunk Jan 10, 2022

nitaliya Jan 11, 2022

jeffreyc-splunk Jan 12, 2022

nitaliya Jan 13, 2022

jeffreyc-splunk Jan 10, 2022

Mukeshr-Ram Jan 12, 2022

jeffreyc-splunk Jan 10, 2022

Mukeshr-Ram Jan 12, 2022

jeffreyc-splunk Jan 10, 2022

Mukeshr-Ram Jan 12, 2022

jeffreyc-splunk Jan 10, 2022

Mukeshr-Ram Jan 12, 2022


		ENV container docker

		RUN zypper -n install -l curl dbus-1 systemd-sysvinit tar wget python3-pip ca-certificates

		@@ -0,0 +1,119 @@
		# Splunk OpenTelemetry Collector Salt Formula

		@@ -0,0 +1,115 @@
		{% set splunk_fluentd_config = salt['pillar.get']('splunk-otel-collector:splunk_fluentd_config', '/etc/otel/collector/fluentd/fluent.conf') %}

Salt support for splunk-otel-collector #1050

Salt support for splunk-otel-collector #1050

Conversation

Mukeshr-Ram commented Dec 17, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nitaliya Dec 22, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

theletterf left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mukeshr-Ram commented Dec 17, 2021 •

edited

Loading

nitaliya Dec 22, 2021 •

edited

Loading