crun.1

.nh
.TH crun 1 "User Commands"

.SH NAME
.PP
crun - a fast and lightweight OCI runtime


.SH SYNOPSIS
.PP
crun [global options] command [command options] [arguments...]


.SH DESCRIPTION
.PP
crun is a command line program for running Linux containers that
follow the Open Container Initiative (OCI) format.


.SH COMMANDS
.PP
\fBcreate\fP
Create a container.  The runtime detaches from the container process
once the container environment is created.  It is necessary to
successively use \fB\fCstart\fR for starting the container.

.PP
\fBdelete\fP
Remove definition for a container.

.PP
\fBexec\fP
Exec a command in a running container.

.PP
\fBlist\fP
List known containers.

.PP
\fBkill\fP
Send the specified signal to the container init process.  If no signal
is specified, SIGTERM is used.

.PP
\fBps\fP
Show the processes running in a container.

.PP
\fBrun\fP
Create and immediately start a container.

.PP
\fBspec\fP
Generate a configuration file.

.PP
\fBstart\fP
Start a container that was previously created.  A container cannot be
started multiple times.

.PP
\fBstate\fP
Output the state of a container.

.PP
\fBpause\fP
Pause all the processes in the container.

.PP
\fBresume\fP
Resume the processes in the container.

.PP
\fBupdate\fP
Update container resource constraints.

.PP
\fBcheckpoint\fP
Checkpoint a running container using CRIU

.PP
\fBrestore\fP
Restore a container from a checkpoint


.SH STATE
.PP
By default, when running as root user, crun saves its state under the
\fB/run/crun\fP directory.  As unprivileged user, instead the
\fIXDG_RUNTIME_DIR\fP environment variable is honored, and the directory
\fB$XDG_RUNTIME_DIR/crun\fP is used.  The global option \fB--root\fP
overrides this setting.


.SH GLOBAL OPTIONS
.PP
\fB--debug\fP
Produce verbose output.

.PP
\fB--log\fP=\fILOG-DESTINATION\fP
Define the destination for the error and warning messages generated by
crun.
If the error happens late in the container init process, when crun
already stopped watching it, then it will be printed to the container
stderr.

.PP
It is specified in the form \fIBACKEND:SPECIFIER\fP\&.

.PP
These following backends are supported:

.RS
.IP \(bu 2
file:PATH
.IP \(bu 2
journald:IDENTIFIER
.IP \(bu 2
syslog:IDENTIFIER

.RE

.PP
If no backend is specified, then \fIfile:\fP is used by default.

.PP
\fB--log-format\fP=\fIFORMAT\fP
Define the format of the log messages.  It can either be \fBtext\fP, or
\fBjson\fP\&.  The default is \fBtext\fP\&.

.PP
\fB--no-pivot\fP
Use \fB\fCchroot(2)\fR instead of \fB\fCpivot_root(2)\fR when creating the container.
This option is not safe, and should be avoided.

.PP
\fB--root\fP=\fIDIR\fP
Defines where to store the state for crun containers.

.PP
\fB--systemd-cgroup\fP
Use systemd for configuring cgroups.  If not specified, the cgroup is
created directly using the cgroupfs backend.

.PP
\fB--cgroup-manager\fP=\fIMANAGER\fP
Specify what cgroup manager must be used.  Permitted values are \fBcgroupfs\fP,
\fBsystemd\fP and \fBdisabled\fP\&.

.PP
\fB-?\fP, \fB--help\fP
Print a help list.

.PP
\fB--usage\fP
Print a short usage message.

.PP
\fB-V\fP, \fB--version\fP
Print program version

.SH CREATE OPTIONS
.PP
crun [global options] create [options] CONTAINER

.PP
\fB--bundle\fP=\fIPATH\fP
Path to the OCI bundle, by default it is the current directory.

.PP
\fB--config\fP=\fIFILE\fP
Override the configuration file to use.  The default value is \fBconfig.json\fP\&.

.PP
\fB--console-socket\fP=\fISOCKET\fP
Path to a UNIX socket that will receive the ptmx end of the tty for
the container.

.PP
\fB--no-new-keyring\fP
Keep the same session key

.PP
\fB--preserve-fds\fP=\fIN\fP
Additional number of FDs to pass into the container.

.PP
\fB--pid-file\fP=\fIPATH\fP
Path to the file that will contain the container process PID.

.SH RUN OPTIONS
.PP
crun [global options] run [options] CONTAINER

.PP
\fB--bundle\fP=\fIBUNDLE\fP
Path to the OCI bundle, by default it is the current directory.

.PP
\fB--config\fP=\fIFILE\fP
Override the configuration file to use.  The default value is \fBconfig.json\fP\&.

.PP
\fB--console-socket\fP=\fISOCKET\fP
Path to a UNIX socket that will receive the ptmx end of the tty for
the container.

.PP
\fB--no-new-keyring\fP
Keep the same session key.

.PP
\fB--preserve-fds\fP=\fIN\fP
Additional number of FDs to pass into the container.

.PP
\fB--pid-file\fP=\fIPATH\fP
Path to the file that will contain the container process PID.

.PP
\fB--detach\fP
Detach the container process from the current session.

.SH DELETE OPTIONS
.PP
crun [global options] delete [options] CONTAINER

.PP
\fB--force\fP
Delete the container even if it is still running.

.PP
\fB--regex\fP=\fIREGEX\fP
Delete all the containers that satisfy the specified regex.

.SH EXEC OPTIONS
.PP
crun [global options] exec [options] CONTAINER CMD

.PP
\fB--apparmor\fP=\fIPROFILE\fP
Set the apparmor profile for the process.

.PP
\fB--console-socket\fP=\fISOCKET\fP
Path to a UNIX socket that will receive the ptmx end of the tty for
the container.

.PP
\fB--cwd\fP=\fIPATH\fP
Set the working directory for the process to PATH.

.PP
\fB--cap\fP=\fICAP\fP
Specify an additional capability to add to the process.

.PP
\fB--detach\fP
Detach the container process from the current session.

.PP
\fB--cgroup\fP=\fIPATH\fP
Specify a sub-cgroup path inside the container cgroup.  The path must
already exist in the container cgroup.

.PP
\fB--env\fP=\fIENV\fP
Specify an environment variable.

.PP
\fB--no-new-privs\fP
Set the no new privileges value for the process.

.PP
\fB--preserve-fds\fP=\fIN\fP
Additional number of FDs to pass into the container.

.PP
\fB--process\fP=\fIFILE\fP
Path to a file containing the process JSON configuration.

.PP
\fB--process-label\fP=\fIVALUE\fP
Set the asm process label for the process commonly used with selinux.

.PP
\fB--pid-file\fP=\fIPATH\fP
Path to the file that will contain the new process PID.

.PP
\fB-t\fP \fB--tty\fP
Allocate a pseudo TTY.

.PP
**-u \fIUSERSPEC\fP \fB--user\fP=\fIUSERSPEC\fP
Specify the user in the form UID[:GID].

.SH LIST OPTIONS
.PP
crun [global options] list [options]

.PP
\fB-q\fP \fB--quiet\fP
Show only the container ID.

.SH KILL OPTIONS
.PP
crun [global options] kill [options] CONTAINER SIGNAL

.PP
\fB--all\fP
Kill all the processes in the container.

.PP
\fB--regex\fP=\fIREGEX\fP
Kill all the containers that satisfy the specified regex.

.SH PS OPTIONS
.PP
crun [global options] ps [options]

.PP
\fB--format\fP=\fIFORMAT\fP
Specify the output format.  It must be either \fB\fCtable\fR or \fB\fCjson\fR\&.
By default \fB\fCtable\fR is used.

.SH SPEC OPTIONS
.PP
crun [global options] spec [options]

.PP
\fB-b\fP \fIDIR\fP \fB--bundle\fP=\fIDIR\fP
Path to the root of the bundle dir (default ".").

.PP
\fB--rootless\fP
Generate a config.json file that is usable by an unprivileged user.

.SH UPDATE OPTIONS
.PP
crun [global options] update [options] CONTAINER

.PP
\fB--blkio-weight\fP=\fIVALUE\fP
Specifies per cgroup weight.

.PP
\fB--cpu-period\fP=\fIVALUE\fP
CPU CFS period to be used for hardcapping.

.PP
\fB--cpu-quota\fP=\fIVALUE\fP
CPU CFS hardcap limit.

.PP
\fB--cpu-rt-period\fP=\fIVALUE\fP
CPU realtime period to be used for hardcapping.

.PP
\fB--cpu-rt-runtime\fP=\fIVALUE\fP
CPU realtime hardcap limit.

.PP
\fB--cpu-share\fP=\fIVALUE\fP
CPU shares.

.PP
\fB--cpuset-cpus\fP=\fIVALUE\fP
CPU(s) to use.

.PP
\fB--cpuset-mems\fP=\fIVALUE\fP
Memory node(s) to use.

.PP
\fB--kernel-memory\fP=\fIVALUE\fP
Kernel memory limit.

.PP
\fB--kernel-memory-tcp\fP=\fIVALUE\fP
Kernel memory limit for TCP buffer.

.PP
\fB--memory\fP=\fIVALUE\fP
Memory limit.

.PP
\fB--memory-reservation\fP=\fIVALUE\fP
Memory reservation or soft_limit.

.PP
\fB--memory-swap\fP=\fIVALUE\fP
Total memory usage.

.PP
\fB--pids-limit\fP=\fIVALUE\fP
Maximum number of pids allowed in the container.

.PP
\fB-r\fP, \fB--resources\fP=\fIFILE\fP
Path to the file containing the resources to update.

.SH CHECKPOINT OPTIONS
.PP
crun [global options] checkpoint [options] CONTAINER

.PP
\fB--image-path\fP=\fIDIR\fP
Path for saving CRIU image files

.PP
\fB--work-path\fP=\fIDIR\fP
Path for saving work files and logs

.PP
\fB--leave-running\fP
Leave the process running after checkpointing

.PP
\fB--tcp-established\fP
Allow open TCP connections

.PP
\fB--ext-unix-sk\fP
Allow external UNIX sockets

.PP
\fB--shell-job\fP
Allow shell jobs

.SH RESTORE OPTIONS
.PP
crun [global options] restore [options] CONTAINER

.PP
\fB-b DIR\fP \fB--bundle\fP=\fIDIR\fP
Container bundle directory (default ".")

.PP
\fB--image-path\fP=\fIDIR\fP
Path for saving CRIU image files

.PP
\fB--work-path\fP=\fIDIR\fP
Path for saving work files and logs

.PP
\fB--tcp-established\fP
Allow open TCP connections

.PP
\fB--ext-unix\fP
Allow external UNIX sockets

.PP
\fB--shell-job\fP
Allow shell jobs

.PP
\fB--detach\fP
Detach from the container's process

.PP
\fB--pid-file\fP=\fIFILE\fP
Where to write the PID of the container


.SH Extensions to OCI
.SH \fB\fCrun.oci.seccomp.receiver=PATH\fR
.PP
If the annotation \fB\fCrun.oci.seccomp.receiver=PATH\fR is specified, the
seccomp listener is sent to the UNIX socket listening on the specified
path.  It can also set with the \fB\fCRUN_OCI_SECCOMP_RECEIVER\fR environment variable.
It is an experimental feature, and the annotation will be removed once
it is supported in the OCI runtime specs.  It must be an absolute path.

.SH \fB\fCrun.oci.seccomp.plugins=PATH\fR
.PP
If the annotation \fB\fCrun.oci.seccomp.plugins=PLUGIN1[:PLUGIN2]...\fR is specified, the
seccomp listener fd is handled through the specified plugins.  The
plugin must either be an absolute path or a file name that is looked
up by \fB\fCldopen(3)\fR\&.  More information on how the lookup is performed
are available on the \fB\fCld.so(8)\fR man page.

.SH \fB\fCrun.oci.seccomp_fail_unknown_syscall=1\fR
.PP
If the annotation \fB\fCrun.oci.seccomp_fail_unknown_syscall\fR is present, then crun
will fail when an unknown syscall is encountered in the seccomp configuration.

.SH \fB\fCrun.oci.seccomp_bpf_data=PATH\fR
.PP
If the annotation \fB\fCrun.oci.seccomp_bpf_data\fR is present, then crun
ignores the seccomp section in the OCI configuration file and use the specified data
as the raw data to the \fB\fCseccomp(SECCOMP_SET_MODE_FILTER)\fR syscall.
The data must be encoded in base64.

.PP
It is an experimental feature, and the annotation will be removed once
it is supported in the OCI runtime specs.

.SH \fB\fCrun.oci.keep_original_groups=1\fR
.PP
If the annotation \fB\fCrun.oci.keep_original_groups\fR is present, then crun
will skip the \fB\fCsetgroups\fR syscall that is used to either set the
additional groups specified in the OCI configuration, or to reset the
list of additional groups if none is specified.

.SH \fB\fCrun.oci.systemd.force_cgroup_v1=/PATH\fR
.PP
If the annotation \fB\fCrun.oci.systemd.force_cgroup_v1=/PATH\fR is present, then crun
will override the specified mount point \fB\fC/PATH\fR with a cgroup v1 mount
made of a single hierarchy \fB\fCnone,name=systemd\fR\&.
It is useful to run on a cgroup v2 system containers using older
versions of systemd that lack support for cgroup v2.

.PP
\fBNote\fP: Your container host has to have the cgroup v1 mount already present, otherwise
this will not work. If you want to run the container rootless, the user it runs under
has to have permissions to this mountpoint.

.PP
For example, as root:

.PP
.RS

.nf
mkdir /sys/fs/cgroup/systemd
mount cgroup -t cgroup /sys/fs/cgroup/systemd -o none,name=systemd,xattr
chown -R the_user.the_user /sys/fs/cgroup/systemd

.fi
.RE

.SH \fB\fCrun.oci.timens_offset=ID SEC NSEC\fR
.PP
Specify the offset to be written to /proc/self/timens_offsets when creating
a time namespace.

.SH \fB\fCrun.oci.systemd.subgroup=SUBGROUP\fR
.PP
Override the name for the systemd sub cgroup created under the systemd
scope, so the final cgroup will be like:

.PP
.RS

.nf
/sys/fs/cgroup/$PATH/$SUBGROUP

.fi
.RE

.PP
When it is set to the empty string, a sub cgroup is not created.

.PP
If not specified, it defaults to \fB\fCcontainer\fR on cgroup v2, and to \fB\fC""\fR
on cgroup v1.

.PP
e.g.

.PP
.RS

.nf
/sys/fs/cgroup//system.slice/foo-352700.scope/container

.fi
.RE

.SH \fB\fCrun.oci.delegate-cgroup=DELEGATED-CGROUP\fR
.PP
If the \fB\fCrun.oci.systemd.subgroup\fR annotation is specified, yet another
sub-cgroup is created and the container process is moved here.

.PP
.RS

.nf
/sys/fs/cgroup/$PATH/$SUBGROUP/$DELEGATED-CGROUP

.fi
.RE

.PP
The runtime doesn't apply any limit to the \fB\fC$DELEGATED-CGROUP\fR
sub-cgroup, the runtime uses only \fB\fC$PATH/$SUBGROUP\fR\&.

.PP
The container payload fully manages \fB\fC$DELEGATE-CGROUP\fR, the limits
applied to \fB\fC$PATH/$SUBGROUP\fR still applies to \fB\fC$DELEGATE-CGROUP\fR\&.

.PP
Since cgroup delegation is not safe on cgroup v1, this option is
supported only on cgroup v2.

.SH \fB\fCrun.oci.hooks.stdout=FILE\fR
.PP
If the annotation \fB\fCrun.oci.hooks.stdout\fR is present, then crun
will open the specified file and use it as the stdout for the hook
processes.  The file is opened in append mode and it is created if it
doesn't already exist.

.SH \fB\fCrun.oci.hooks.stderr=FILE\fR
.PP
If the annotation \fB\fCrun.oci.hooks.stderr\fR is present, then crun
will open the specified file and use it as the stderr for the hook
processes.  The file is opened in append mode and it is created if it
doesn't already exist.

.SH \fB\fCrun.oci.handler=HANDLER\fR
.PP
It is an experimental feature.

.PP
If specified, run the specified handler for execing the container.
The only supported value is \fB\fCkrun\fR\&.  When \fB\fCkrun\fR is specified, the
\fB\fClibkrun.so\fR shared object is loaded and it is used to launch the
container using libkrun.

.SH \fB\fCrun.oci.handler=wasm\fR
.PP
If specified, run the wasm handler for container.
Allows running wasm workload natively. Accepts a \fB\fC\&.wasm\fR binary as input
and if \fB\fC\&.wat\fR is provided it will automatically compiled into a wasm module.
Stdout of wasm module is relayed back via crun.

.SH tmpcopyup mount options
.PP
If the \fB\fCtmpcopyup\fR option is specified for a tmpfs, then the path that
is shadowed by the tmpfs mount is recursively copied up to the tmpfs
itself.

.SH rro mount options
.PP
If the \fB\fCrro\fR option is specified then the mount is made recursively read-only.

.SH Automatically create user namespace
.PP
When running as user different than root, an user namespace is
automatically created even if it is not specified in the config file.
The current user is mapped to the ID 0 in the container, and any
additional id specified in the files \fB\fC/etc/subuid\fR and \fB\fC/etc/subgid\fR
is automatically added starting with ID 1.

.SH Intermediate user namespace
.PP
If the configuration specifies a new user namespace made of a single
mapping to the root user, but either the UID or the GID are set as
nonzero then crun automatically creates another user namespace to map
the root user to the specified UID and GID.

.PP
It enables running unprivileged containers with UID and GID different
than zero, even when a single UID and GID are available, e.g. rootless
users on a system without newuidmap/newgidmap.


.SH CGROUP v2
.PP
\fBNote\fP: cgroup v2 does not yet support control of realtime processes and
the cpu controller can only be enabled when all RT processes are in the root
cgroup. This will make crun fail while running alongside RT processes.

.PP
If the cgroup configuration found is for cgroup v1, crun attempts a
conversion when running on a cgroup v2 system.

.PP
These are the OCI resources currently supported with cgroup v2 and how
they are converted when needed from the cgroup v1 configuration.

.SH Memory controller
.TS
allbox;
l l l l 
l l l l .
\fB\fCOCI (x)\fR	\fB\fCcgroup 2 value (y)\fR	\fB\fCconversion\fR	\fB\fCcomment\fR
limit	memory.max	y = x	
swap	memory.swap.max	y = x - memory_limit	T{
the swap limit on cgroup v1 includes the memory usage too
T}
reservation	memory.low	y = x	
.TE

.SH PIDs controller
.TS
allbox;
l l l l 
l l l l .
\fB\fCOCI (x)\fR	\fB\fCcgroup 2 value (y)\fR	\fB\fCconversion\fR	\fB\fCcomment\fR
limit	pids.max	y = x	
.TE

.SH CPU controller
.TS
allbox;
l l l l 
l l l l .
\fB\fCOCI (x)\fR	\fB\fCcgroup 2 value (y)\fR	\fB\fCconversion\fR	\fB\fCcomment\fR
shares	cpu.weight	T{
y = (1 + ((x - 2) * 9999) / 262142)
T}
	T{
convert from [2-262144] to [1-10000]
T}
period	cpu.max	y = x	T{
period and quota are written together
T}
quota	cpu.max	y = x	T{
period and quota are written together
T}
.TE

.SH blkio controller
.TS
allbox;
l l l l 
l l l l .
\fB\fCOCI (x)\fR	\fB\fCcgroup 2 value (y)\fR	\fB\fCconversion\fR	\fB\fCcomment\fR
weight	io.bfq.weight	y = x	
weight_device	io.bfq.weight	y = x	
rbps	io.max	y=x	
wbps	io.max	y=x	
riops	io.max	y=x	
wiops	io.max	y=x	
.TE

.SH cpuset controller
.TS
allbox;
l l l l 
l l l l .
\fB\fCOCI (x)\fR	\fB\fCcgroup 2 value (y)\fR	\fB\fCconversion\fR	\fB\fCcomment\fR
cpus	cpuset.cpus	y = x	
mems	cpuset.mems	y = x	
.TE

.SH hugetlb controller
.TS
allbox;
l l l l 
l l l l .
\fB\fCOCI (x)\fR	\fB\fCcgroup 2 value (y)\fR	\fB\fCconversion\fR	\fB\fCcomment\fR
\&.limit_in_bytes	hugetlb.\&.max	y = x	
.TE