Add fluentd images #5303

repeatedly · 2019-01-16T10:35:15Z

New proposal to add fluentd image. This replaces old PR: #3724

Checklist for Review

NOTE: This checklist is intended for the use of the Official Images maintainers both to track the status of your PR and to help inform you and others of where we're at. As such, please leave the "checking" of items to the repository maintainers. If there is a point below for which you would like to provide additional information or note completion, please do so by commenting on the PR. Thanks! (and thanks for staying patient with us ❤️)

associated with or contacted upstream?
does it fit into one of the common categories? ("service", "language stack", "base distribution")
log collector
is it reasonably popular, or does it solve a particular use case well?
logging part of CNCF
does a documentation PR exist? (should be reviewed and merged at roughly the same time so that we don't have an empty image page on the Hub for very long) Add fluentd document docs#1403
dockerization review for best practices and cache gotchas/improvements (ala the official review guidelines)?
2+ dockerization review?
existing official images have been considered as a base? (ie, if foobar needs Node.js, has FROM node:... instead of grabbing node via other means been considered?)
~~if FROM scratch, tarballs only exist in a single commit within the associated history?~~
passes current tests? any simple new tests that might be appropriate to add? (https://github.com/docker-library/official-images/tree/master/test)

repeatedly · 2019-01-16T12:10:09Z

Send PR to docs: docker-library/docs#1403

yosifkit · 2019-01-16T21:36:10Z

Diff:

diff --git a/_bashbrew-arches b/_bashbrew-arches
index e69de29..d6d56d3 100644
--- a/_bashbrew-arches
+++ b/_bashbrew-arches
@@ -0,0 +1,26 @@
+fluentd:v1.3-1 @ amd64
+fluentd:v1.3-1 @ arm32v6
+fluentd:v1.3-1 @ arm64v8
+fluentd:v1.3-1 @ i386
+fluentd:v1.3-1 @ ppc64le
+fluentd:v1.3-1 @ s390x
+fluentd:v1.3-debian-1 @ amd64
+fluentd:v1.3-debian-1 @ arm32v5
+fluentd:v1.3-debian-1 @ arm32v7
+fluentd:v1.3-debian-1 @ arm64v8
+fluentd:v1.3-debian-1 @ i386
+fluentd:v1.3-debian-1 @ ppc64le
+fluentd:v1.3-debian-1 @ s390x
+fluentd:v1.3-debian-onbuild-1 @ amd64
+fluentd:v1.3-debian-onbuild-1 @ arm32v5
+fluentd:v1.3-debian-onbuild-1 @ arm32v7
+fluentd:v1.3-debian-onbuild-1 @ arm64v8
+fluentd:v1.3-debian-onbuild-1 @ i386
+fluentd:v1.3-debian-onbuild-1 @ ppc64le
+fluentd:v1.3-debian-onbuild-1 @ s390x
+fluentd:v1.3-onbuild-1 @ amd64
+fluentd:v1.3-onbuild-1 @ arm32v6
+fluentd:v1.3-onbuild-1 @ arm64v8
+fluentd:v1.3-onbuild-1 @ i386
+fluentd:v1.3-onbuild-1 @ ppc64le
+fluentd:v1.3-onbuild-1 @ s390x
diff --git a/_bashbrew-list b/_bashbrew-list
index e69de29..acf2661 100644
--- a/_bashbrew-list
+++ b/_bashbrew-list
@@ -0,0 +1,8 @@
+fluentd:v1.3-1
+fluentd:v1.3-debian-1
+fluentd:v1.3-debian-onbuild-1
+fluentd:v1.3-onbuild-1
+fluentd:v1.3.3-1.0
+fluentd:v1.3.3-debian-1.0
+fluentd:v1.3.3-debian-onbuild-1.0
+fluentd:v1.3.3-onbuild-1.0
diff --git a/fluentd_v1.3-1/Dockerfile b/fluentd_v1.3-1/Dockerfile
new file mode 100644
index 0000000..41841b8
--- /dev/null
+++ b/fluentd_v1.3-1/Dockerfile
@@ -0,0 +1,45 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM alpine:3.8
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.3.3"
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apk delete' has no effect
+RUN apk update \
+ && apk add --no-cache \
+        ca-certificates \
+        ruby ruby-irb ruby-etc ruby-webrick \
+        tini \
+ && apk add --no-cache --virtual .build-deps \
+        build-base \
+        ruby-dev gnupg \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.1.0 \
+ && gem install fluentd -v 1.3.3 \
+ && gem install bigdecimal -v 1.3.5 \
+ && apk del .build-deps \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+# for log storage (maybe shared with host)
+RUN mkdir -p /fluentd/log
+# configuration/plugins path (default: copied from .)
+RUN mkdir -p /fluentd/etc /fluentd/plugins
+
+RUN addgroup -S fluent && adduser -S -g fluent fluent
+RUN chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+
+ENV LD_PRELOAD=""
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.3-1/entrypoint.sh b/fluentd_v1.3-1/entrypoint.sh
new file mode 100755
index 0000000..231fc92
--- /dev/null
+++ b/fluentd_v1.3-1/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    source $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/fluent.conf
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.3-1/fluent.conf b/fluentd_v1.3-1/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.3-1/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>
diff --git a/fluentd_v1.3-debian-1/Dockerfile b/fluentd_v1.3-debian-1/Dockerfile
new file mode 100644
index 0000000..38369f9
--- /dev/null
+++ b/fluentd_v1.3-debian-1/Dockerfile
@@ -0,0 +1,63 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM debian:stretch-slim
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.3.3"
+ENV TINI_VERSION=0.18.0
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apt-get purge' has no effect
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends \
+            ca-certificates \
+            ruby \
+ && buildDeps=" \
+      make gcc g++ libc-dev \
+      ruby-dev \
+      wget bzip2 gnupg dirmngr \
+    " \
+ && apt-get install -y --no-install-recommends $buildDeps \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.1.0 \
+ && gem install fluentd -v 1.3.3 \
+ && dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')" \
+ && wget -O /usr/local/bin/tini "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch" \
+ && wget -O /usr/local/bin/tini.asc "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch.asc" \
+ && export GNUPGHOME="$(mktemp -d)" \
+ && gpg --batch --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 6380DC428747F6C393FEACA59A84159D7001A4E5 \
+ && gpg --batch --verify /usr/local/bin/tini.asc /usr/local/bin/tini \
+ && rm -r /usr/local/bin/tini.asc \
+ && chmod +x /usr/local/bin/tini \
+ && tini -h \
+ && wget -O /tmp/jemalloc-4.5.0.tar.bz2 https://github.com/jemalloc/jemalloc/releases/download/4.5.0/jemalloc-4.5.0.tar.bz2 \
+ && cd /tmp && tar -xjf jemalloc-4.5.0.tar.bz2 && cd jemalloc-4.5.0/ \
+ && ./configure && make \
+ && mv lib/libjemalloc.so.2 /usr/lib \
+ && apt-get purge -y --auto-remove \
+                  -o APT::AutoRemove::RecommendsImportant=false \
+                  $buildDeps \
+ && rm -rf /var/lib/apt/lists/* \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+# for log storage (maybe shared with host)
+RUN mkdir -p /fluentd/log
+# configuration/plugins path (default: copied from .)
+RUN mkdir -p /fluentd/etc /fluentd/plugins
+
+RUN groupadd -r fluent && useradd -r -g fluent fluent
+RUN chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+
+ENV LD_PRELOAD="/usr/lib/libjemalloc.so.2"
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.3-debian-1/entrypoint.sh b/fluentd_v1.3-debian-1/entrypoint.sh
new file mode 100755
index 0000000..231fc92
--- /dev/null
+++ b/fluentd_v1.3-debian-1/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    source $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/fluent.conf
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.3-debian-1/fluent.conf b/fluentd_v1.3-debian-1/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.3-debian-1/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>
diff --git a/fluentd_v1.3-debian-onbuild-1/Dockerfile b/fluentd_v1.3-debian-onbuild-1/Dockerfile
new file mode 100644
index 0000000..51c7fdf
--- /dev/null
+++ b/fluentd_v1.3-debian-onbuild-1/Dockerfile
@@ -0,0 +1,65 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM debian:stretch-slim
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.3.3"
+ENV TINI_VERSION=0.18.0
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apt-get purge' has no effect
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends \
+            ca-certificates \
+            ruby \
+ && buildDeps=" \
+      make gcc g++ libc-dev \
+      ruby-dev \
+      wget bzip2 gnupg dirmngr \
+    " \
+ && apt-get install -y --no-install-recommends $buildDeps \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.1.0 \
+ && gem install fluentd -v 1.3.3 \
+ && dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')" \
+ && wget -O /usr/local/bin/tini "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch" \
+ && wget -O /usr/local/bin/tini.asc "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch.asc" \
+ && export GNUPGHOME="$(mktemp -d)" \
+ && gpg --batch --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 6380DC428747F6C393FEACA59A84159D7001A4E5 \
+ && gpg --batch --verify /usr/local/bin/tini.asc /usr/local/bin/tini \
+ && rm -r /usr/local/bin/tini.asc \
+ && chmod +x /usr/local/bin/tini \
+ && tini -h \
+ && wget -O /tmp/jemalloc-4.5.0.tar.bz2 https://github.com/jemalloc/jemalloc/releases/download/4.5.0/jemalloc-4.5.0.tar.bz2 \
+ && cd /tmp && tar -xjf jemalloc-4.5.0.tar.bz2 && cd jemalloc-4.5.0/ \
+ && ./configure && make \
+ && mv lib/libjemalloc.so.2 /usr/lib \
+ && apt-get purge -y --auto-remove \
+                  -o APT::AutoRemove::RecommendsImportant=false \
+                  $buildDeps \
+ && rm -rf /var/lib/apt/lists/* \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+# for log storage (maybe shared with host)
+RUN mkdir -p /fluentd/log
+# configuration/plugins path (default: copied from .)
+RUN mkdir -p /fluentd/etc /fluentd/plugins
+
+RUN groupadd -r fluent && useradd -r -g fluent fluent
+RUN chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+ONBUILD COPY fluent.conf /fluentd/etc/
+ONBUILD COPY plugins /fluentd/plugins/
+
+ENV LD_PRELOAD="/usr/lib/libjemalloc.so.2"
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.3-debian-onbuild-1/entrypoint.sh b/fluentd_v1.3-debian-onbuild-1/entrypoint.sh
new file mode 100755
index 0000000..231fc92
--- /dev/null
+++ b/fluentd_v1.3-debian-onbuild-1/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    source $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/fluent.conf
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.3-debian-onbuild-1/fluent.conf b/fluentd_v1.3-debian-onbuild-1/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.3-debian-onbuild-1/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>
diff --git a/fluentd_v1.3-onbuild-1/Dockerfile b/fluentd_v1.3-onbuild-1/Dockerfile
new file mode 100644
index 0000000..0046291
--- /dev/null
+++ b/fluentd_v1.3-onbuild-1/Dockerfile
@@ -0,0 +1,47 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM alpine:3.8
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.3.3"
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apk delete' has no effect
+RUN apk update \
+ && apk add --no-cache \
+        ca-certificates \
+        ruby ruby-irb ruby-etc ruby-webrick \
+        tini \
+ && apk add --no-cache --virtual .build-deps \
+        build-base \
+        ruby-dev gnupg \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.1.0 \
+ && gem install fluentd -v 1.3.3 \
+ && gem install bigdecimal -v 1.3.5 \
+ && apk del .build-deps \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+# for log storage (maybe shared with host)
+RUN mkdir -p /fluentd/log
+# configuration/plugins path (default: copied from .)
+RUN mkdir -p /fluentd/etc /fluentd/plugins
+
+RUN addgroup -S fluent && adduser -S -g fluent fluent
+RUN chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+ONBUILD COPY fluent.conf /fluentd/etc/
+ONBUILD COPY plugins /fluentd/plugins/
+
+ENV LD_PRELOAD=""
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.3-onbuild-1/entrypoint.sh b/fluentd_v1.3-onbuild-1/entrypoint.sh
new file mode 100755
index 0000000..231fc92
--- /dev/null
+++ b/fluentd_v1.3-onbuild-1/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    source $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/fluent.conf
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.3-onbuild-1/fluent.conf b/fluentd_v1.3-onbuild-1/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.3-onbuild-1/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>

repeatedly · 2019-02-04T17:02:31Z

@yosifkit Do we need more work for this patch?

yosifkit · 2019-02-06T23:21:47Z

Sorry for the delay! 🙇‍♂️ Thanks for your patience. The only comments I have are hopefully minor:

chown or chgrp in its own layer has had problems with some docker storage drivers in the past, so we try to ensure that file creation and owner/permissions are done in a single layer. In this case, it seems to just be some directories, so it should work most of the time, but can be improved later.
onbuild images: we don't really encourage their use ([Proposal] Deprecate onbuild tags #2076), but if it is something your users actively use and expect, that is up to you.
- If keeping them, rather than repeating all the content from the base fluentd image, they can be FROM it.
```
# as specific as you like here (depending on how often you want to regenerate the file)
FROM fluentd:v1.3-1
# maybe a "--chown fluent:fluent"  on the COPYs?
ONBUILD COPY fluent.conf /fluentd/etc/
ONBUILD COPY plugins /fluentd/plugins/
```
side note, did you want to update to Alpine 3.9?
I usually shy away from packages like build-base and instead just install exactly what I need, but it doesn't include so many extras like build-essential in debian, so that is your decision
did you want to have a latest tag (ie default)? If you do not have one, docker pull fluentd will fail.

Heading to double check the docs. 🚀

cc @tianon for second review.

Diff:

diff --git a/_bashbrew-arches b/_bashbrew-arches
index e69de29..d6d56d3 100644
--- a/_bashbrew-arches
+++ b/_bashbrew-arches
@@ -0,0 +1,26 @@
+fluentd:v1.3-1 @ amd64
+fluentd:v1.3-1 @ arm32v6
+fluentd:v1.3-1 @ arm64v8
+fluentd:v1.3-1 @ i386
+fluentd:v1.3-1 @ ppc64le
+fluentd:v1.3-1 @ s390x
+fluentd:v1.3-debian-1 @ amd64
+fluentd:v1.3-debian-1 @ arm32v5
+fluentd:v1.3-debian-1 @ arm32v7
+fluentd:v1.3-debian-1 @ arm64v8
+fluentd:v1.3-debian-1 @ i386
+fluentd:v1.3-debian-1 @ ppc64le
+fluentd:v1.3-debian-1 @ s390x
+fluentd:v1.3-debian-onbuild-1 @ amd64
+fluentd:v1.3-debian-onbuild-1 @ arm32v5
+fluentd:v1.3-debian-onbuild-1 @ arm32v7
+fluentd:v1.3-debian-onbuild-1 @ arm64v8
+fluentd:v1.3-debian-onbuild-1 @ i386
+fluentd:v1.3-debian-onbuild-1 @ ppc64le
+fluentd:v1.3-debian-onbuild-1 @ s390x
+fluentd:v1.3-onbuild-1 @ amd64
+fluentd:v1.3-onbuild-1 @ arm32v6
+fluentd:v1.3-onbuild-1 @ arm64v8
+fluentd:v1.3-onbuild-1 @ i386
+fluentd:v1.3-onbuild-1 @ ppc64le
+fluentd:v1.3-onbuild-1 @ s390x
diff --git a/_bashbrew-list b/_bashbrew-list
index e69de29..acf2661 100644
--- a/_bashbrew-list
+++ b/_bashbrew-list
@@ -0,0 +1,8 @@
+fluentd:v1.3-1
+fluentd:v1.3-debian-1
+fluentd:v1.3-debian-onbuild-1
+fluentd:v1.3-onbuild-1
+fluentd:v1.3.3-1.0
+fluentd:v1.3.3-debian-1.0
+fluentd:v1.3.3-debian-onbuild-1.0
+fluentd:v1.3.3-onbuild-1.0
diff --git a/fluentd_v1.3-1/Dockerfile b/fluentd_v1.3-1/Dockerfile
new file mode 100644
index 0000000..41841b8
--- /dev/null
+++ b/fluentd_v1.3-1/Dockerfile
@@ -0,0 +1,45 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM alpine:3.8
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.3.3"
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apk delete' has no effect
+RUN apk update \
+ && apk add --no-cache \
+        ca-certificates \
+        ruby ruby-irb ruby-etc ruby-webrick \
+        tini \
+ && apk add --no-cache --virtual .build-deps \
+        build-base \
+        ruby-dev gnupg \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.1.0 \
+ && gem install fluentd -v 1.3.3 \
+ && gem install bigdecimal -v 1.3.5 \
+ && apk del .build-deps \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+# for log storage (maybe shared with host)
+RUN mkdir -p /fluentd/log
+# configuration/plugins path (default: copied from .)
+RUN mkdir -p /fluentd/etc /fluentd/plugins
+
+RUN addgroup -S fluent && adduser -S -g fluent fluent
+RUN chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+
+ENV LD_PRELOAD=""
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.3-1/entrypoint.sh b/fluentd_v1.3-1/entrypoint.sh
new file mode 100755
index 0000000..231fc92
--- /dev/null
+++ b/fluentd_v1.3-1/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    source $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/fluent.conf
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.3-1/fluent.conf b/fluentd_v1.3-1/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.3-1/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>
diff --git a/fluentd_v1.3-debian-1/Dockerfile b/fluentd_v1.3-debian-1/Dockerfile
new file mode 100644
index 0000000..38369f9
--- /dev/null
+++ b/fluentd_v1.3-debian-1/Dockerfile
@@ -0,0 +1,63 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM debian:stretch-slim
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.3.3"
+ENV TINI_VERSION=0.18.0
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apt-get purge' has no effect
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends \
+            ca-certificates \
+            ruby \
+ && buildDeps=" \
+      make gcc g++ libc-dev \
+      ruby-dev \
+      wget bzip2 gnupg dirmngr \
+    " \
+ && apt-get install -y --no-install-recommends $buildDeps \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.1.0 \
+ && gem install fluentd -v 1.3.3 \
+ && dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')" \
+ && wget -O /usr/local/bin/tini "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch" \
+ && wget -O /usr/local/bin/tini.asc "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch.asc" \
+ && export GNUPGHOME="$(mktemp -d)" \
+ && gpg --batch --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 6380DC428747F6C393FEACA59A84159D7001A4E5 \
+ && gpg --batch --verify /usr/local/bin/tini.asc /usr/local/bin/tini \
+ && rm -r /usr/local/bin/tini.asc \
+ && chmod +x /usr/local/bin/tini \
+ && tini -h \
+ && wget -O /tmp/jemalloc-4.5.0.tar.bz2 https://github.com/jemalloc/jemalloc/releases/download/4.5.0/jemalloc-4.5.0.tar.bz2 \
+ && cd /tmp && tar -xjf jemalloc-4.5.0.tar.bz2 && cd jemalloc-4.5.0/ \
+ && ./configure && make \
+ && mv lib/libjemalloc.so.2 /usr/lib \
+ && apt-get purge -y --auto-remove \
+                  -o APT::AutoRemove::RecommendsImportant=false \
+                  $buildDeps \
+ && rm -rf /var/lib/apt/lists/* \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+# for log storage (maybe shared with host)
+RUN mkdir -p /fluentd/log
+# configuration/plugins path (default: copied from .)
+RUN mkdir -p /fluentd/etc /fluentd/plugins
+
+RUN groupadd -r fluent && useradd -r -g fluent fluent
+RUN chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+
+ENV LD_PRELOAD="/usr/lib/libjemalloc.so.2"
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.3-debian-1/entrypoint.sh b/fluentd_v1.3-debian-1/entrypoint.sh
new file mode 100755
index 0000000..231fc92
--- /dev/null
+++ b/fluentd_v1.3-debian-1/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    source $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/fluent.conf
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.3-debian-1/fluent.conf b/fluentd_v1.3-debian-1/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.3-debian-1/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>
diff --git a/fluentd_v1.3-debian-onbuild-1/Dockerfile b/fluentd_v1.3-debian-onbuild-1/Dockerfile
new file mode 100644
index 0000000..51c7fdf
--- /dev/null
+++ b/fluentd_v1.3-debian-onbuild-1/Dockerfile
@@ -0,0 +1,65 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM debian:stretch-slim
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.3.3"
+ENV TINI_VERSION=0.18.0
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apt-get purge' has no effect
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends \
+            ca-certificates \
+            ruby \
+ && buildDeps=" \
+      make gcc g++ libc-dev \
+      ruby-dev \
+      wget bzip2 gnupg dirmngr \
+    " \
+ && apt-get install -y --no-install-recommends $buildDeps \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.1.0 \
+ && gem install fluentd -v 1.3.3 \
+ && dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')" \
+ && wget -O /usr/local/bin/tini "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch" \
+ && wget -O /usr/local/bin/tini.asc "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch.asc" \
+ && export GNUPGHOME="$(mktemp -d)" \
+ && gpg --batch --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 6380DC428747F6C393FEACA59A84159D7001A4E5 \
+ && gpg --batch --verify /usr/local/bin/tini.asc /usr/local/bin/tini \
+ && rm -r /usr/local/bin/tini.asc \
+ && chmod +x /usr/local/bin/tini \
+ && tini -h \
+ && wget -O /tmp/jemalloc-4.5.0.tar.bz2 https://github.com/jemalloc/jemalloc/releases/download/4.5.0/jemalloc-4.5.0.tar.bz2 \
+ && cd /tmp && tar -xjf jemalloc-4.5.0.tar.bz2 && cd jemalloc-4.5.0/ \
+ && ./configure && make \
+ && mv lib/libjemalloc.so.2 /usr/lib \
+ && apt-get purge -y --auto-remove \
+                  -o APT::AutoRemove::RecommendsImportant=false \
+                  $buildDeps \
+ && rm -rf /var/lib/apt/lists/* \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+# for log storage (maybe shared with host)
+RUN mkdir -p /fluentd/log
+# configuration/plugins path (default: copied from .)
+RUN mkdir -p /fluentd/etc /fluentd/plugins
+
+RUN groupadd -r fluent && useradd -r -g fluent fluent
+RUN chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+ONBUILD COPY fluent.conf /fluentd/etc/
+ONBUILD COPY plugins /fluentd/plugins/
+
+ENV LD_PRELOAD="/usr/lib/libjemalloc.so.2"
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.3-debian-onbuild-1/entrypoint.sh b/fluentd_v1.3-debian-onbuild-1/entrypoint.sh
new file mode 100755
index 0000000..231fc92
--- /dev/null
+++ b/fluentd_v1.3-debian-onbuild-1/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    source $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/fluent.conf
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.3-debian-onbuild-1/fluent.conf b/fluentd_v1.3-debian-onbuild-1/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.3-debian-onbuild-1/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>
diff --git a/fluentd_v1.3-onbuild-1/Dockerfile b/fluentd_v1.3-onbuild-1/Dockerfile
new file mode 100644
index 0000000..0046291
--- /dev/null
+++ b/fluentd_v1.3-onbuild-1/Dockerfile
@@ -0,0 +1,47 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM alpine:3.8
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.3.3"
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apk delete' has no effect
+RUN apk update \
+ && apk add --no-cache \
+        ca-certificates \
+        ruby ruby-irb ruby-etc ruby-webrick \
+        tini \
+ && apk add --no-cache --virtual .build-deps \
+        build-base \
+        ruby-dev gnupg \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.1.0 \
+ && gem install fluentd -v 1.3.3 \
+ && gem install bigdecimal -v 1.3.5 \
+ && apk del .build-deps \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+# for log storage (maybe shared with host)
+RUN mkdir -p /fluentd/log
+# configuration/plugins path (default: copied from .)
+RUN mkdir -p /fluentd/etc /fluentd/plugins
+
+RUN addgroup -S fluent && adduser -S -g fluent fluent
+RUN chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+ONBUILD COPY fluent.conf /fluentd/etc/
+ONBUILD COPY plugins /fluentd/plugins/
+
+ENV LD_PRELOAD=""
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.3-onbuild-1/entrypoint.sh b/fluentd_v1.3-onbuild-1/entrypoint.sh
new file mode 100755
index 0000000..231fc92
--- /dev/null
+++ b/fluentd_v1.3-onbuild-1/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    source $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/fluent.conf
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.3-onbuild-1/fluent.conf b/fluentd_v1.3-onbuild-1/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.3-onbuild-1/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>

tianon · 2019-02-07T00:22:55Z

I would also recommend removing the use of apk update -- using --no-cache on apk add will download the index files, and will remove them (effectively making apk update do nothing).

tianon · 2019-02-07T00:23:40Z

(Is it also intentional that bigdecimal is only installed in the Alpine variant?)

lag-linaro · 2019-02-18T12:31:50Z

Any luck with this @repeatedly?

repeatedly · 2019-04-03T22:53:15Z

@yosifkit Sorry for the delay response.

chown or chgrp ~ In this case, it seems to just be some directories, so it should work most of the time, but can be improved later.

If you have an dockerfile example, I will apply the changes for it.

onbuild images: we don't really encourage their use (#2076), but if it is something your users actively use and expect, that is up to you.

Instead of onbuild, users need to write own Dockerfile(e.g. like https://github.com/fluent/fluentd-docker-image#31-for-current-images), right?
We can't delete onbuild images immediately but we can put the note in RAEADME for the deprecation.

side note, did you want to update to Alpine 3.9?

Yes. I forgot to update this. I will update soon.

did you want to have a latest tag (ie default)? If you do not have one, docker pull fluentd will fail.

I will add latest.

@tianon

Is it also intentional that bigdecimal is only installed in the Alpine variant?

Yes. Alpine Linux removes bigdecimal library from default library to reduce the ruby package size and it causes the error for fluentd.

ghatwala · 2019-04-08T12:02:19Z

@yosifkit , @tianon - Any updates on above , to help adding "fluentd' as official docker image ?

yosifkit · 2019-04-10T23:17:27Z

If you have an dockerfile example, I will apply the changes for it.

This will combine the folder creation with their permissions and the user/group creation, which could still be in its own RUN line if desired.

RUN addgroup -S fluent \
	&& adduser -S -g fluent fluent \
# for log storage (maybe shared with host)
	&& mkdir -p /fluentd/log \
# configuration/plugins path (default: copied from .)
	&& mkdir -p /fluentd/etc /fluentd/plugins \
# fix ownership
	&& chown -R fluent:fluent /fluentd

Instead of onbuild, users need to write own Dockerfile(e.g. like https://github.com/fluent/fluentd-docker-image#31-for-current-images), right?

Correct, but since the only ONBUILD parts are two COPY instructions, they would have a 3 line Dockerfile vs an opaque 1 line FROM

# For users the change would be minimal
# current
FROM fluentd:v1.3.3-onbuild-1.0

# vs without onbuild:
FROM fluentd:v1.3.3
COPY fluent.conf /fluentd/etc/
COPY plugins /fluentd/plugins/

We can't delete onbuild images immediately but we can put the note in README for the deprecation.

That is fine, but they do need to be updated to just be the 3 lines rather than being a full copy of the non-onbuild image Dockerfile.

# as specific as you like here (depending on how often you want to regenerate the file)
FROM fluentd:v1.3-1
# maybe a "--chown fluent:fluent"  on the COPYs?
ONBUILD COPY fluent.conf /fluentd/etc/
ONBUILD COPY plugins /fluentd/plugins/

Edit: also, the apk update is unnecessary since you already have --no-cache on the add

ghatwala · 2019-04-11T04:49:26Z

@yosifkit - thanks for the review comments , @repeatedly -Could you please help check and update accordingly .

repeatedly · 2019-04-24T16:18:55Z

Okay. I will apply above review to fluentd images.

- Use latest commit for fluentd v1.4 - Add latest tag - Remove onbuild images

repeatedly · 2019-04-25T21:30:20Z

Just updated the content. I decied to stop providing onbuild for official images.

yosifkit · 2019-04-25T23:25:41Z

Diff:

failed fetching repo "fluentd"
unable to find a manifest named "fluentd" (in "/tmp/tmp.sn5azwuMOt/oi/library" or as a remote URL)
failed fetching repo "fluentd"
unable to find a manifest named "fluentd" (in "/tmp/tmp.sn5azwuMOt/oi/library" or as a remote URL)
failed fetching repo "fluentd"
unable to find a manifest named "fluentd" (in "/tmp/tmp.sn5azwuMOt/oi/library" or as a remote URL)
failed fetching repo "fluentd"
unable to find a manifest named "fluentd" (in "/tmp/tmp.sn5azwuMOt/oi/library" or as a remote URL)
diff --git a/_bashbrew-arches b/_bashbrew-arches
index e69de29..09f700b 100644
--- a/_bashbrew-arches
+++ b/_bashbrew-arches
@@ -0,0 +1,13 @@
+fluentd:latest @ amd64
+fluentd:latest @ arm32v6
+fluentd:latest @ arm64v8
+fluentd:latest @ i386
+fluentd:latest @ ppc64le
+fluentd:latest @ s390x
+fluentd:v1.4-debian-2 @ amd64
+fluentd:v1.4-debian-2 @ arm32v5
+fluentd:v1.4-debian-2 @ arm32v7
+fluentd:v1.4-debian-2 @ arm64v8
+fluentd:v1.4-debian-2 @ i386
+fluentd:v1.4-debian-2 @ ppc64le
+fluentd:v1.4-debian-2 @ s390x
diff --git a/_bashbrew-list b/_bashbrew-list
index e69de29..3ba2bd7 100644
--- a/_bashbrew-list
+++ b/_bashbrew-list
@@ -0,0 +1,5 @@
+fluentd:latest
+fluentd:v1.4-2
+fluentd:v1.4-debian-2
+fluentd:v1.4.2-2.0
+fluentd:v1.4.2-debian-2.0
diff --git a/fluentd_latest/Dockerfile b/fluentd_latest/Dockerfile
new file mode 100644
index 0000000..5259f04
--- /dev/null
+++ b/fluentd_latest/Dockerfile
@@ -0,0 +1,47 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM alpine:3.9
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.4.2"
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apk delete' has no effect
+RUN apk update \
+ && apk add --no-cache \
+        ca-certificates \
+        ruby ruby-irb ruby-etc ruby-webrick \
+        tini \
+ && apk add --no-cache --virtual .build-deps \
+        build-base \
+        ruby-dev gnupg \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.2.0 \
+ && gem install fluentd -v 1.4.2 \
+ && gem install bigdecimal -v 1.3.5 \
+ && apk del .build-deps \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+RUN addgroup -S fluent && adduser -S -g fluent fluent \
+    # for log storage (maybe shared with host)
+    && mkdir -p /fluentd/log \
+    # configuration/plugins path (default: copied from .)
+    && mkdir -p /fluentd/etc /fluentd/plugins \
+    && chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+
+ENV FLUENTD_CONF="fluent.conf"
+
+ENV LD_PRELOAD=""
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_latest/entrypoint.sh b/fluentd_latest/entrypoint.sh
new file mode 100755
index 0000000..876b424
--- /dev/null
+++ b/fluentd_latest/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    . $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/${FLUENTD_CONF}
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_latest/fluent.conf b/fluentd_latest/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_latest/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>
diff --git a/fluentd_v1.4-debian-2/Dockerfile b/fluentd_v1.4-debian-2/Dockerfile
new file mode 100644
index 0000000..5a92a0a
--- /dev/null
+++ b/fluentd_v1.4-debian-2/Dockerfile
@@ -0,0 +1,63 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM ruby:2.6-slim-stretch
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.4.2"
+ENV TINI_VERSION=0.18.0
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apt-get purge' has no effect
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends \
+            ca-certificates \
+ && buildDeps=" \
+      make gcc g++ libc-dev \
+      wget bzip2 gnupg dirmngr \
+    " \
+ && apt-get install -y --no-install-recommends $buildDeps \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.2.0 \
+ && gem install fluentd -v 1.4.2 \
+ && dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')" \
+ && wget -O /usr/local/bin/tini "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch" \
+ && wget -O /usr/local/bin/tini.asc "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch.asc" \
+ && export GNUPGHOME="$(mktemp -d)" \
+ && gpg --batch --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 6380DC428747F6C393FEACA59A84159D7001A4E5 \
+ && gpg --batch --verify /usr/local/bin/tini.asc /usr/local/bin/tini \
+ && rm -r /usr/local/bin/tini.asc \
+ && chmod +x /usr/local/bin/tini \
+ && tini -h \
+ && wget -O /tmp/jemalloc-4.5.0.tar.bz2 https://github.com/jemalloc/jemalloc/releases/download/4.5.0/jemalloc-4.5.0.tar.bz2 \
+ && cd /tmp && tar -xjf jemalloc-4.5.0.tar.bz2 && cd jemalloc-4.5.0/ \
+ && ./configure && make \
+ && mv lib/libjemalloc.so.2 /usr/lib \
+ && apt-get purge -y --auto-remove \
+                  -o APT::AutoRemove::RecommendsImportant=false \
+                  $buildDeps \
+ && rm -rf /var/lib/apt/lists/* \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+RUN groupadd -r fluent && useradd -r -g fluent fluent \
+    # for log storage (maybe shared with host)
+    && mkdir -p /fluentd/log \
+    # configuration/plugins path (default: copied from .)
+    && mkdir -p /fluentd/etc /fluentd/plugins \
+    && chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+
+ENV FLUENTD_CONF="fluent.conf"
+
+ENV LD_PRELOAD="/usr/lib/libjemalloc.so.2"
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.4-debian-2/entrypoint.sh b/fluentd_v1.4-debian-2/entrypoint.sh
new file mode 100755
index 0000000..876b424
--- /dev/null
+++ b/fluentd_v1.4-debian-2/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    . $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/${FLUENTD_CONF}
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.4-debian-2/fluent.conf b/fluentd_v1.4-debian-2/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.4-debian-2/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>

yosifkit · 2019-04-25T23:47:39Z

I don't love the non-verified download of jemalloc, but it is at least https and they don't provide signatures or a SHASUMS file. I'm assuming that the jemalloc in Debian Stretch was too old (3.6.0) and so my only suggestion is to look to using the Debian package once the image moves to Buster (5.1.0, https://packages.debian.org/buster/libjemalloc2).

In summary, LGTM. (~~pending docs check~~) ping @tianon.

Build test of #5303; 241306e; amd64 (fluentd):

$ bashbrew build fluentd:v1.4.2-2.0
Building bashbrew/cache:46f777a7d44567ae51a262c7a2c8567dc9ebfd2e1ad89b3d177d67a75634f5de (fluentd:v1.4.2-2.0)
Tagging fluentd:v1.4.2-2.0
Tagging fluentd:v1.4-2
Tagging fluentd:latest

$ test/run.sh fluentd:v1.4.2-2.0
testing fluentd:v1.4.2-2.0
	'utc' [1/4]...passed
	'cve-2014--shellshock' [2/4]...passed
	'no-hard-coded-passwords' [3/4]...passed
	'override-cmd' [4/4]...passed


$ bashbrew build fluentd:v1.4.2-debian-2.0
Building bashbrew/cache:1232cddc848588df2a0cd75560b88855094b97e92b0bde5eba14f19c0d7ec1a9 (fluentd:v1.4.2-debian-2.0)
Tagging fluentd:v1.4.2-debian-2.0
Tagging fluentd:v1.4-debian-2

$ test/run.sh fluentd:v1.4.2-debian-2.0
testing fluentd:v1.4.2-debian-2.0
	'utc' [1/4]...passed
	'cve-2014--shellshock' [2/4]...passed
	'no-hard-coded-passwords' [3/4]...passed
	'override-cmd' [4/4]...passed

tianon · 2019-04-26T21:31:00Z

LGTM (removing onbuild for official images seems like a totally sane way to communicate the transition IMO 👍)

A few other thoughts that can be considered / dealt with post-merge:

apk update is unnecessary with apk add --no-cache ...
LABEL keys should be adjusted according to https://github.com/opencontainers/image-spec/blob/v1.0.1/annotations.md
I would recommend adding --no-network to apk del
echo $@ | grep ' \-c' is liable to have false positives and edge cases for detecting whether the user supplied -c -- is -c in fluentd similar to other programs in that the latest value wins so the default could simply be prepended to the argument list? (see discussion in Convert to Docker Official Images fluent/fluentd-docker-image#152 (comment)) if not, I'd recommend switching to a loop that checks for an argument of exactly -c
adding a checksum or something to jemalloc would be a great addition

docker-library-bot added library/fluentd new-image labels Jan 16, 2019

Add fluentd images

1161718

repeatedly force-pushed the add-fluentd branch from 83d4464 to 1161718 Compare January 16, 2019 10:46

repeatedly mentioned this pull request Jan 16, 2019

Add fluentd document docker-library/docs#1403

Merged

repeatedly mentioned this pull request Jan 16, 2019

Create a new official image for fluentd #3724

Closed

9 tasks

repeatedly mentioned this pull request Feb 4, 2019

Adding support for power(ppc64le) for travis builds fluent/fluentd-docker-image#120

Closed

ghatwala mentioned this pull request Apr 18, 2019

Adding power support fluent/fluentd#2386

Merged

Update fluentd setting

241306e

- Use latest commit for fluentd v1.4 - Add latest tag - Remove onbuild images

repeatedly force-pushed the add-fluentd branch from 24fa7fd to 241306e Compare April 25, 2019 21:25

tianon merged commit 092096a into docker-library:master Apr 26, 2019

ghatwala mentioned this pull request Jul 23, 2019

Confusion Over Docker Hub Fluentd vs Official fluent/fluentd-docker-image#189

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add fluentd images #5303

Add fluentd images #5303

repeatedly commented Jan 16, 2019 •

edited by yosifkit

Loading

repeatedly commented Jan 16, 2019

yosifkit commented Jan 16, 2019

repeatedly commented Feb 4, 2019

yosifkit commented Feb 6, 2019

tianon commented Feb 7, 2019

tianon commented Feb 7, 2019

lag-linaro commented Feb 18, 2019

repeatedly commented Apr 3, 2019

ghatwala commented Apr 8, 2019

yosifkit commented Apr 10, 2019 •

edited

Loading

ghatwala commented Apr 11, 2019

repeatedly commented Apr 24, 2019

repeatedly commented Apr 25, 2019

yosifkit commented Apr 25, 2019

yosifkit commented Apr 25, 2019

tianon commented Apr 26, 2019

Add fluentd images #5303

Add fluentd images #5303

Conversation

repeatedly commented Jan 16, 2019 • edited by yosifkit Loading

Checklist for Review

repeatedly commented Jan 16, 2019

yosifkit commented Jan 16, 2019

repeatedly commented Feb 4, 2019

yosifkit commented Feb 6, 2019

tianon commented Feb 7, 2019

tianon commented Feb 7, 2019

lag-linaro commented Feb 18, 2019

repeatedly commented Apr 3, 2019

ghatwala commented Apr 8, 2019

yosifkit commented Apr 10, 2019 • edited Loading

ghatwala commented Apr 11, 2019

repeatedly commented Apr 24, 2019

repeatedly commented Apr 25, 2019

yosifkit commented Apr 25, 2019

yosifkit commented Apr 25, 2019

tianon commented Apr 26, 2019

repeatedly commented Jan 16, 2019 •

edited by yosifkit

Loading

yosifkit commented Apr 10, 2019 •

edited

Loading