From 5e90beeff5f37ed7295de0792650fa35ab77b31c Mon Sep 17 00:00:00 2001 From: Sebastian Serth Date: Fri, 19 Apr 2024 18:51:13 +0000 Subject: [PATCH 1/4] Upgrade to Ubuntu 24.04 Noble --- Makefile | 2 +- README.md | 2 +- README_ZH_cn_.md | 2 +- README_zh_tw.md | 2 +- Vagrantfile | 21 ++++++++++----------- image/Dockerfile | 2 +- 6 files changed, 15 insertions(+), 16 deletions(-) diff --git a/Makefile b/Makefile index fe24996c7..6d0538648 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -VERSION ?= focal-1.0.0-pre +VERSION ?= noble-1.0.0 ifdef BASE_IMAGE BUILD_ARG = --build-arg BASE_IMAGE=$(BASE_IMAGE) ifndef NAME diff --git a/README.md b/README.md index 28f456afa..168b9e186 100644 --- a/README.md +++ b/README.md @@ -86,7 +86,7 @@ You can configure the stock `ubuntu` image yourself from your Dockerfile, so why | Component | Why is it included? / Remarks | | ---------------- | ------------------- | -| Ubuntu 22.04 LTS | The base system. | +| Ubuntu 24.04 LTS | The base system. | | A **correct** init process | _Main article: [Docker and the PID 1 zombie reaping problem](http://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/)._

According to the Unix process model, [the init process](https://en.wikipedia.org/wiki/Init) -- PID 1 -- inherits all [orphaned child processes](https://en.wikipedia.org/wiki/Orphan_process) and must [reap them](https://en.wikipedia.org/wiki/Wait_(system_call)). Most Docker containers do not have an init process that does this correctly. As a result, their containers become filled with [zombie processes](https://en.wikipedia.org/wiki/Zombie_process) over time.

Furthermore, `docker stop` sends SIGTERM to the init process, which stops all services. Unfortunately most init systems don't do this correctly within Docker since they're built for hardware shutdowns instead. This causes processes to be hard killed with SIGKILL, which doesn't give them a chance to correctly deinitialize things. This can cause file corruption.

Baseimage-docker comes with an init process `/sbin/my_init` that performs both of these tasks correctly. | | Fixes APT incompatibilities with Docker | See https://github.com/dotcloud/docker/issues/1024. | | syslog-ng | A syslog daemon is necessary so that many services - including the kernel itself - can correctly log to /var/log/syslog. If no syslog daemon is running, a lot of important messages are silently swallowed.

Only listens locally. All syslog messages are forwarded to "docker logs".

Why syslog-ng?
I've had bad experience with rsyslog. I regularly run into bugs with rsyslog, and once in a while it takes my log host down by entering a 100% CPU loop in which it can't do anything. Syslog-ng seems to be much more stable. | diff --git a/README_ZH_cn_.md b/README_ZH_cn_.md index 7e6ad5787..3c7954ab0 100644 --- a/README_ZH_cn_.md +++ b/README_ZH_cn_.md @@ -82,7 +82,7 @@ Baseimage-docker让这一切完美。在"内容"部分描述了所有这些修 | 模块 | 为什么包含这些？以及备注 | | ---------------- | ------------------- | -| Ubuntu 22.04 LTS | 基础系统。 | +| Ubuntu 24.04 LTS | 基础系统。 | | 一个**正确**的初始化进程 | *主要文章：[Docker和PID 1 僵尸进程回收问题](http://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/)*

根据Unix进程模型，[初始化进程](https://en.wikipedia.org/wiki/Init) -- PID 1 -- 继承了所有[孤立的子进程](https://en.wikipedia.org/wiki/Orphan_process)，并且必须[进行回收](https://en.wikipedia.org/wiki/Wait_(system_call))。大多数Docker容器没有一个初始化进程可以正确的完成此操作，随着时间的推移会导致他们的容器出现了大量的[僵尸进程](https://en.wikipedia.org/wiki/Zombie_process)。

而且，`docker stop`发送SIGTERM信号给初始化进程，照理说此信号应该可以停止所有服务。不幸的是由于它们对硬件进行了关闭操作，导致Docker内的大多数初始化系统没有正确执行。这会导致进程强行被SIGKILL信号关闭，从而丧失了一个正确取消初始化设置的机会。这会导致文件损坏。

Baseimage-docker配有一个名为`/sbin/my_init`的初始化进程来同时正确的完成这些任务。 | | 修复了APT与Docker不兼容的问题 | 详情参见：https://github.com/dotcloud/docker/issues/1024 。 | | syslog-ng | 对于很多服务－包括kernel自身，都需要一个syslog后台进程，以便可以正确的将log输出到/var/log/syslog中。如果没有运行syslog后台进程，很多重要的信息就会默默的丢失了。

只对本地进行监听。所有syslog信息会被转发给“docker logs”。 | diff --git a/README_zh_tw.md b/README_zh_tw.md index 63c4be9a9..d0e7feb32 100644 --- a/README_zh_tw.md +++ b/README_zh_tw.md @@ -82,7 +82,7 @@ Baseimage-docker讓這一切完美。在"內容"部分描述了所有這些修 | 模塊 | 爲什麼包含這些？以及備註 | | ---------------- | ------------------- | -| Ubuntu 22.04 LTS | 基礎系統。 | +| Ubuntu 24.04 LTS | 基礎系統。 | | 一個**正確**的初始化行程 | *主要文章：[Docker和PID 1 殭屍行程回收問題](http://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/)*

根據Unix行程模型，[初始化行程](https://en.wikipedia.org/wiki/Init) -- PID 1 -- 繼承了所有[孤立的子行程](https://en.wikipedia.org/wiki/Orphan_process)，並且必須[進行回收](https://en.wikipedia.org/wiki/Wait_(system_call))。大多數Docker容器沒有一個初始化行程可以正確的完成此操作，隨着時間的推移會導致他們的容器出現了大量的[殭屍行程](https://en.wikipedia.org/wiki/Zombie_process)。

而且，`docker stop`發送SIGTERM信號給初始化行程，照理說此信號應該可以停止所有服務。不幸的是由於它們對硬體進行了關閉操作，導致Docker內的大多數初始化系統沒有正確執行。這會導致行程強行被SIGKILL信號關閉，從而喪失了一個正確取消初始化設置的機會。這會導致文件損壞。

Baseimage-docker配有一個名爲`/sbin/my_init`的初始化行程來同時正確的完成這些任務。 | | 修復了APT與Docker不兼容的問題 | 詳情參見：https://github.com/dotcloud/docker/issues/1024 。 | | syslog-ng | 對於很多服務－包括kernel自身，都需要一個syslog後臺行程，以便可以正確的將log輸出到/var/log/syslog中。如果沒有運行syslog後臺行程，很多重要的信息就會默默的丟失了。

只對本地進行監聽。所有syslog信息會被轉發給“docker logs”。 | diff --git a/Vagrantfile b/Vagrantfile index 3a376a009..3f2f9411b 100644 --- a/Vagrantfile +++ b/Vagrantfile @@ -9,37 +9,37 @@ Vagrant.configure("2") do |config| # The most common configuration options are documented and commented below. # For a complete reference, please see the online documentation at # https://docs.vagrantup.com. - + # Every Vagrant development environment requires a box. You can search for # boxes at https://atlas.hashicorp.com/search. - config.vm.box = "ubuntu/focal64" + config.vm.box = "ubuntu/noble64" config.disksize.size = '50GB' - + # Disable automatic box update checking. If you disable this, then # boxes will only be checked for updates when the user runs # `vagrant box outdated`. This is not recommended. # config.vm.box_check_update = false - + # Create a forwarded port mapping which allows access to a specific port # within the machine from a port on the host machine. In the example below, # accessing "localhost:8080" will access port 80 on the guest machine. # config.vm.network "forwarded_port", guest: 80, host: 8080 - + # Create a private network, which allows host-only access to the machine # using a specific IP. # config.vm.network "private_network", ip: "192.168.33.10" - + # Create a public network, which generally matched to bridged network. # Bridged networks make the machine appear as another physical device on # your network. # config.vm.network "public_network" - + # Share an additional folder to the guest VM. The first argument is # the path on the host to the actual folder. The second argument is # the path on the guest to mount the folder. And the optional third # argument is a set of non-required options. # config.vm.synced_folder "../data", "/vagrant_data" - + # Provider-specific configuration so you can fine-tune various # backing providers for Vagrant. These expose provider-specific options. # Example for VirtualBox: @@ -54,14 +54,14 @@ Vagrant.configure("2") do |config| # # View the documentation for the provider you are using for more # information on available options. - + # Define a Vagrant Push strategy for pushing to Atlas. Other push strategies # such as FTP and Heroku are also available. See the documentation at # https://docs.vagrantup.com/v2/push/atlas.html for more information. # config.push.define "atlas" do |push| # push.app = "YOUR_ATLAS_USERNAME/YOUR_APPLICATION_NAME" # end - + # Enable provisioning with a shell script. Additional provisioners such as # Puppet, Chef, Ansible, Salt, and Docker are also available. Please see the # documentation for more information about their specific syntax and use. @@ -72,4 +72,3 @@ Vagrant.configure("2") do |config| config.vm.provision :shell, path: "vagrant-libs/bootstrap.sh" end - diff --git a/image/Dockerfile b/image/Dockerfile index dc86eb957..0c0345880 100644 --- a/image/Dockerfile +++ b/image/Dockerfile @@ -1,4 +1,4 @@ -ARG BASE_IMAGE=ubuntu:22.04 +ARG BASE_IMAGE=ubuntu:24.04 FROM $BASE_IMAGE ARG QEMU_ARCH From c2142db49ccfdbaf819d53e9740009d345aac4de Mon Sep 17 00:00:00 2001 From: Sebastian Serth Date: Fri, 19 Apr 2024 22:03:33 +0200 Subject: [PATCH 2/4] Update syslog-ng.conf to version 4.3 --- image/services/syslog-ng/syslog-ng.conf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/image/services/syslog-ng/syslog-ng.conf b/image/services/syslog-ng/syslog-ng.conf index b6cf2e44c..b6b306167 100644 --- a/image/services/syslog-ng/syslog-ng.conf +++ b/image/services/syslog-ng/syslog-ng.conf @@ -1,4 +1,4 @@ -@version: 3.35 +@version: 4.3 @include "scl.conf" # Syslog-ng configuration file, compatible with default Debian syslogd @@ -7,7 +7,7 @@ # First, set some global options. options { chain_hostnames(off); flush_lines(0); use_dns(no); use_fqdn(no); dns_cache(no); owner("root"); group("adm"); perm(0640); - stats_freq(0); bad_hostname("^gconfd$"); + stats(freq(0)); bad_hostname("^gconfd$"); }; ######################## From 74cb5902884b5c114d3dc17584cdf59f56972810 Mon Sep 17 00:00:00 2001 From: Sebastian Serth Date: Sat, 20 Apr 2024 12:02:18 +0200 Subject: [PATCH 3/4] Add default smart-multi-line.fsm config for syslog-ng The config file is taken from the syslog-ng upstream repository. It got added for the syslog-ng 4.2.0 release and remained unchanged since then (as of April 2024). https://github.com/syslog-ng/syslog-ng/blob/syslog-ng-4.2.0/lib/multi-line/smart-multi-line.fsm --- image/services/syslog-ng/smart-multi-line.fsm | 83 +++++++++++++++++++ image/services/syslog-ng/syslog-ng.sh | 1 + 2 files changed, 84 insertions(+) create mode 100644 image/services/syslog-ng/smart-multi-line.fsm diff --git a/image/services/syslog-ng/smart-multi-line.fsm b/image/services/syslog-ng/smart-multi-line.fsm new file mode 100644 index 000000000..126a30e1c --- /dev/null +++ b/image/services/syslog-ng/smart-multi-line.fsm @@ -0,0 +1,83 @@ +# +# Copyright 2023 Balazs Scheidler +# Copyright 2016 Google Inc. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# The regular expressions were extracted from +# https://github.com/GoogleCloudPlatform/fluent-plugin-detect-exceptions +# and converted into a TSV format by Balazs Scheidler. +# +# List of tab separated fields +# +# comma-separated-states /regexp/ new_state +# + +# java +start_state,java_start_exception /(?:Exception|Error|Throwable|V8 errors stack trace)[:\r\n]/ java_after_exception +java_after_exception /^[\t ]*nested exception is:[\t ]*/ java_start_exception +java_after_exception /^[\r\n]*$/ java_after_exception +java_after_exception,java /^[\t ]+(?:eval )?at / java +java_after_exception,java /^[\t ]+--- End of inner exception stack trace ---$/ java +java_after_exception,java /^--- End of stack trace from previous location where exception was thrown ---$/ java +java_after_exception,java /^[\t ]*(?:Caused by|Suppressed):/ java_after_exception +java_after_exception,java /^[\t ]*... \d+ (?:more|common frames omitted)/ java + +# python +start_state /^Traceback \(most recent call last\):$/ python +python /^[\t ]*File / python_code +python_code /[^\t ]/ python +python /^(?:[^\s.():]+\.)*[^\s.():]+:/ start_state + +# PHP +start_state /(?:PHP\ (?:Notice|Parse\ error|Fatal\ error|Warning):)|(?:exception\ '[^']+'\ with\ message\ ')/ php_stack_begin +php_stack_begin /^Stack trace:/ php_stack_frames +php_stack_frames /^#\d/ php_stack_frames +php_stack_frames /^\s+thrown in / start_state + +# Go +start_state /\bpanic: / go_after_panic +start_state /http: panic serving/ go_goroutine +go_after_panic,go_after_signal,go_frame_1 /^$/ go_goroutine +go_after_panic /^\[signal / go_after_signal +go_goroutine /^goroutine \d+ \[[^\]]+\]:$/ go_frame_1 +go_frame_1 /^(?:[^\s.:]+\.)*[^\s.():]+\(|^created by / go_frame_2 +go_frame_2 /^\s/ go_frame_1 + +# Ruby +start_state /Error \(.*\):$/ ruby_before_rails_trace +ruby_before_rails_trace /^ $/ ruby +ruby_before_rails_trace /^[\t ]+.*?\.rb:\d+:in `/ ruby +ruby /^[\t ]+.*?\.rb:\d+:in `/ ruby + +# Dart +start_state /^Unhandled exception:$/ dart_exc +dart_exc /^(Instance of)|(Exception)|(Bad state)|(IntegerDivisionByZeroException)|(Invalid argument)|(RangeError)|(Assertion failed)|(Cannot instantiate)|(Reading static variable)|(UnimplementedError)|(Unsupported operation)|(Concurrent modification)|(Out of Memory)|(Stack Overflow)/ dart_stack +dart_exc /^'.+?':.+?$/ dart_type_err_1 +dart_type_err_1 /^#\d+\s+.+?\(.+?\)$/ dart_stack +dart_type_err_1 /^.+?$/ dart_type_err_2 +dart_type_err_2 /^.*?\^.*?$/ dart_type_err_3 +dart_type_err_3 /^$/ dart_type_err_4 +dart_type_err_4 /^$/ dart_stack +dart_exc /^FormatException/ dart_format_err_1 +dart_format_err_1 /^#\d+\s+.+?\(.+?\)$/ dart_stack +dart_format_err_1 /^./ dart_format_err_2 +dart_format_err_2 /^.*?\^/ dart_format_err_3 +dart_format_err_3 /^$/ dart_stack +dart_exc /^NoSuchMethodError:/ dart_method_err_1 +dart_method_err_1 /^Receiver:/ dart_method_err_2 +dart_method_err_2 /^Tried calling:/ dart_method_err_3 +dart_method_err_3 /^Found:/ dart_stack +dart_method_err_3 /^#\d+\s+.+?\(.+?\)$/ dart_stack +dart_stack /^#\d+\s+.+?\(.+?\)$/ dart_stack +dart_stack /^$/ dart_stack diff --git a/image/services/syslog-ng/syslog-ng.sh b/image/services/syslog-ng/syslog-ng.sh index 610d80c3b..3b545d68c 100755 --- a/image/services/syslog-ng/syslog-ng.sh +++ b/image/services/syslog-ng/syslog-ng.sh @@ -9,6 +9,7 @@ SYSLOG_NG_BUILD_PATH=/bd_build/services/syslog-ng $minimal_apt_get_install syslog-ng-core cp $SYSLOG_NG_BUILD_PATH/syslog-ng.init /etc/my_init.d/10_syslog-ng.init cp $SYSLOG_NG_BUILD_PATH/syslog-ng.shutdown /etc/my_init.post_shutdown.d/10_syslog-ng.shutdown +cp $SYSLOG_NG_BUILD_PATH/smart-multi-line.fsm /usr/share/syslog-ng/smart-multi-line.fsm mkdir -p /var/lib/syslog-ng cp $SYSLOG_NG_BUILD_PATH/syslog_ng_default /etc/default/syslog-ng touch /var/log/syslog From a2409df832a5b25610610da8a8eb1820b9975047 Mon Sep 17 00:00:00 2001 From: Sebastian Serth Date: Sat, 20 Apr 2024 19:40:26 +0200 Subject: [PATCH 4/4] Use latest workflow run for release badge --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 168b9e186..842304061 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # A minimal Ubuntu base image modified for Docker-friendliness -[![Release](https://github.com/phusion/baseimage-docker/actions/workflows/main.yml/badge.svg?branch=focal-1.1.0)](https://github.com/phusion/baseimage-docker/actions/workflows/main.yml) +[![Release](https://github.com/phusion/baseimage-docker/actions/workflows/main.yml/badge.svg)](https://github.com/phusion/baseimage-docker/actions/workflows/main.yml) _Baseimage-docker only consumes 8.3 MB RAM and is much more powerful than Busybox or Alpine. See why below._