Skip to content
This repository has been archived by the owner on May 22, 2020. It is now read-only.

create a chroot aci for docker and kubelet #34

Closed
wants to merge 1 commit into from
Closed

create a chroot aci for docker and kubelet #34

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions node-aci/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
.acbuild
library-debian-jessie.aci
node.aci
39 changes: 39 additions & 0 deletions node-aci/build
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
#! /bin/bash

set -o errexit
set -o pipefail
set -o nounset
set -o xtrace

rm -f node.aci

docker2aci docker://debian:jessie
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need debian base image? Why not alpine?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vishh One reason I avoid Alpine: it lacks security metadata http://lists.alpinelinux.org/alpine-devel/5228.html and quay/clair#12


acbuild begin ./library-debian-jessie.aci

acbuild run -- apt-get update
acbuild run -- apt-get install -y -q apparmor curl iptables
acbuild run -- apt-get autoremove
acbuild run -- apt-get clean

acbuild run -- \
curl -sSL --fail \
"https://get.docker.com/builds/Linux/x86_64/docker-1.11.1.tgz" \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it too much to assume that docker will already be installed?

the version of docker to pull is very OS specific, and I struggle to think users will not have it already if they plan to use it.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My increasing concern here is the "kubernetes supports docker vX". This has already become a little fragile with the CoreOS release cycle, where we have shipped docker too soon from k8s perspective (k8s v1.1 & docker v1.10 iirc), then with the upcoming k8s- v1.3 release it was unclear if v1.11 was going to be the blessed version (while even CoreOS alpha still had v1.10).

Decoupling this from the underlying host would (hopefully) make this a little easier to reason about / ship updates as atomic units.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with @aaronlevy that I am more motivated by coupling kubelet/docker versions then making it easy to install docker.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The community itself doesn't agree on the version of docker to use in kube
1.3, I think this blesses a particular version moving forward. I prefer
it's removed. Maybe I just don't understand the expectation for this PR
generally.

On Thursday, June 23, 2016, Mike Danese [email protected] wrote:

In node-aci/build
#34 (comment):

+set -o xtrace
+
+rm -f node.aci
+
+docker2aci docker://debian:jessie
+
+acbuild begin ./library-debian-jessie.aci
+
+acbuild run -- apt-get update
+acbuild run -- apt-get install -y -q apparmor curl iptables
+acbuild run -- apt-get autoremove
+acbuild run -- apt-get clean
+
+acbuild run -- \

Agree with @aaronlevy https://github.com/aaronlevy that I am more
motivated by coupling kubelet/docker versions then making it easy to
install docker.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/kube-deploy/pull/34/files/3173ffd06edce04392130f4d4a9f27458cd33022#r68332519,
or mute the thread
https://github.com/notifications/unsubscribe/AF8dbMZ0la5he-1RJ21R2EvGIFZuUG7zks5qOxWVgaJpZM4IisM6
.

-o /opt/docker.tgz
acbuild run -- tar xzfv /opt/docker.tgz --strip=1 -C "/usr/local/bin"
acbuild run -- rm /opt/docker.tgz

acbuild run -- \
curl -sSL --fail \
"https://storage.googleapis.com/kubernetes-release/release/v1.3.0-alpha.4/bin/linux/amd64/kubectl" \
-o "/usr/local/bin/kubectl"
acbuild run -- chmod +x "/usr/local/bin/kubectl"

acbuild run -- \
curl -sSL --fail \
"https://storage.googleapis.com/kubernetes-release/release/v1.3.0-alpha.4/bin/linux/amd64/kubelet" \
-o "/usr/local/bin/kubelet"
acbuild run -- chmod +x "/usr/local/bin/kubelet"

acbuild write node.aci
acbuild end
21 changes: 21 additions & 0 deletions node-aci/docker.service
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target

[Service]
Type=notify
RootDirectory=/opt/kubelet/rootfs
ExecStart=/usr/local/bin/docker daemon
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

[Install]
WantedBy=multi-user.target
18 changes: 18 additions & 0 deletions node-aci/kubelet.service
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
[Unit]
Description=Kubernetes Kubelet Server
Documentation=https://github.com/kubernetes/kubernetes
After=network.target docker.socket

[Service]
RootDirectory=/opt/kubelet/rootfs
ExecStart=/usr/local/bin/kubelet \
--address=0.0.0.0 \
--allow-privileged=true \
--enable-server \
--config=/etc/kubernetes/manifests \
--cluster-dns=10.0.0.10 \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can one customize kubelet configuration? Config Map?
Is cluster-dns required by the kubelet before it can read its config from a ConfigMap?

--cluster-domain=cluster.local \
--v=2

[Install]
WantedBy=multi-user.target
31 changes: 31 additions & 0 deletions node-aci/unpack
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#! /bin/bash

set -o nounset
set -o errexit
set -o pipefail

ROOTFS=/opt/kubelet/rootfs

mount_in() {
local path="${1}"
local shared="${2:-false}"
mkdir -p "${path}"
mkdir -p "${ROOTFS}${path}"
mount --rbind "${path}" "${ROOTFS}${path}"
if [[ "${shared}" == "true" ]]; then
mount --bind "${ROOTFS}${path}" "${ROOTFS}${path}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK this extra bind is not necessary..

mount --make-shared "${ROOTFS}${path}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be make-rshared?

fi
}

mkdir -p /opt/kubelet
tar xzvf node.aci -C /opt/kubelet

mount_in /proc
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead why not mount in root, followed by a chroot?

mount_in / true
chroot $ROOTFS

mount_in /sys
mount_in /dev
mount_in /run
mount_in /var/run
mount_in /etc
mount_in /var/lib/docker
mount_in /var/lib/kubelet true