Unpin td-agent and cap elasticsearch gem

With [0] the version of td-agent being installed was pinned, which isn't sustainable in the long run, so we drop the pin. Latest version of the elasticsearch gem no longer works with older (OSS) versions of Elasticsearch. This is fixed by downgrading the version of the elasticsearch gems. Backport includes 711ec08 [1]. [0] Iefcdd3100b7e3c5320bc5f1286a18251bdeab885 [1] I353ef9b6b8da0e726c17ad2e06995d3690731e1d Related-Bug: 1930867 Closes-Bug: 1954759 Closes-Bug: 1960444 Signed-off-by: Dr. Jens Harbott <[email protected]> Co-Authored-By: Radosław Piliszek <[email protected]> Change-Id: I3045786e863b098d7339d1066aef6c857aa5f97f (cherry picked from commit a8e6924)
stackhpc · Feb 9, 2022 · 5c6eb17 · 5c6eb17
1 parent c6a972e
commit 5c6eb17
Show file tree

Hide file tree

Showing 4 changed files with 16 additions and 13 deletions.
diff --git a/docker/base/apt_preferences.debian b/docker/base/apt_preferences.debian
@@ -7,11 +7,6 @@ Package: erlang*
 Pin: version 1:23.*
 Pin-Priority: 1000
 
-# FIXME(mgoddard): Pinning to 4.0.* to avoid bug 1930867.
-Package: td-agent*
-Pin: version 4.0.*
-Pin-Priority: 1000
-
 # NOTE(mgoddard): logstash 7.9.x is the last version that supports
 # Elasticsearch OSS.
 Package: logstash-oss

diff --git a/docker/base/apt_preferences.ubuntu b/docker/base/apt_preferences.ubuntu
@@ -6,11 +6,6 @@ Package: erlang*
 Pin: version 1:23.*
 Pin-Priority: 1000
 
-# FIXME(mgoddard): Pinning to 4.0.* to avoid bug 1930867.
-Package: td-agent*
-Pin: version 4.0.*
-Pin-Priority: 1000
-
 # NOTE(mgoddard): logstash 7.9.x is the last version that supports
 # Elasticsearch OSS.
 Package: logstash-oss

diff --git a/docker/fluentd/Dockerfile.j2 b/docker/fluentd/Dockerfile.j2
@@ -27,9 +27,8 @@ LABEL fluentd_version="0.14" fluentd_binary="td-agent"
     ] %}
 
     {% if base_arch in ['aarch64', 'x86_64'] %}
-        # FIXME(mgoddard): Pinning to 4.0.* to avoid bug 1930867.
         {% set fluentd_packages = fluentd_packages + [
-            'td-agent-4.0.*'
+            'td-agent'
         ] %}
     {% else %}
         {% set fluentd_packages = fluentd_packages + [
@@ -62,7 +61,14 @@ LABEL fluentd_version="0.14" fluentd_binary="td-agent"
 {% endif %}
 
 {{ macros.configure_user(name=fluentd_user, groups='mysql') }}
-{{ macros.install_packages(fluentd_packages | customizable("packages")) }}
+
+# NOTE(frickler): Downgrading elasticsearch gems for compatibility with OSS versions of ES
+{{ macros.install_packages(fluentd_packages | customizable("packages")) }}{% if base_arch in ['aarch64', 'x86_64'] %} \
+    && td-agent-gem install elasticsearch:7.13.0 \
+    && td-agent-gem uninstall "elasticsearch:>7.13.0" \
+    && td-agent-gem uninstall "elasticsearch-api:>7.13.0" \
+    && td-agent-gem uninstall "elasticsearch-transport:>7.13.0"
+{% endif %}
 
 # Distro specific files and operations
 # Fluentd on rpm-based non-x86_64 is installed from rpm

diff --git a/releasenotes/notes/cap-fluentd-elasticsearch-18c0ca8e90c1234c.yaml b/releasenotes/notes/cap-fluentd-elasticsearch-18c0ca8e90c1234c.yaml
@@ -0,0 +1,7 @@
+---
+fixes:
+  - |
+    Latest version of the elasticsearch gem no longer works with older
+    (OSS) versions of Elasticsearch. This is fixed by capping the version
+    of the elasticsearch gem installed into the fluentd container.
+    `LP#1954759 <https://launchpad.net/bugs/1954759>`__