-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: Avi Deitcher <[email protected]>
- Loading branch information
Showing
1 changed file
with
141 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,141 @@ | ||
| # Terminals and Standard IO | ||
| All processes on Unix (and Unix-like) operating systems have 3 standard file descriptors (fd) passed to them at start, collectively standard-IO (`stdio`): | ||
|
|
||
| * `0`: standard-in (`stdin`), the input stream into the process | ||
| * `1`: standard-out (`stdout`), the output stream from the process | ||
| * `2`: standard-error (`stderr`), the error stream from the process | ||
|
|
||
| When creating and running a container via `runc`, it is important to take care to structure the stdio the new container's process receives. | ||
|
|
||
| ## Terminal Modes | ||
|
|
||
| `runc` supports two distinct methods for passing stdio to the container's primary process: | ||
|
|
||
| * pass-through | ||
| * new terminal | ||
|
|
||
| ### Pass-Through | ||
| In pass-through mode, the `stdio` passed to the `runc` call will be passed as is to the primary container. This means that if you do the following: | ||
|
|
||
| ``` | ||
| $ echo input | runc run some_container > /tmp/log.out 2>& /tmp/log.err | ||
| ``` | ||
|
|
||
| Then the stdout of `some_container` will appear in the file `/tmp/log.out`, the stderr will appear in `/tmp/log.err`, and the process will receive input of `input`. `runc`'s stdout was redirected to `/tmp/log.out`, and `runc`, in turn, passed that handle to the container itself. | ||
|
|
||
| To invoke pass-through mode, configure your container's `config.json` with `terminal: false` (which is the default). | ||
|
|
||
| ### New Terminal | ||
| If you do _not_ want to pass your current stdio through to the container, tell it to create a new terminal. | ||
|
|
||
| When you do so, `runc` will do the following: | ||
|
|
||
| 1. Create a new pseudo-terminal (`pty`). | ||
| 2. Pass this pty's slave to the container's primary process as its stdio. | ||
| 3. Send the pty's master to a process to read/write stdio for the container's primary process (details below). | ||
|
|
||
| To invoke new terminal mode, configure your container's `config.json` with `terminal: true`. | ||
|
|
||
| ## Which Mode To Use | ||
| `runc` itself runs in two modes: | ||
|
|
||
| * foreground | ||
| * detached | ||
|
|
||
| You can use either terminal mode with either runc mode. However, there are considerations that may indicate preference for one mode over another. | ||
|
|
||
| ### Foreground | ||
| When running in foreground, `runc` and the container you invoke are most like running a regular foreground process. This is the scenario most likely to be useful for running in pass-through terminal mode. As the example above showed: | ||
|
|
||
| ``` | ||
| $ echo input | runc run some_container > /tmp/log.out 2>& /tmp/log.err | ||
| ``` | ||
|
|
||
| Nonetheless, you still have the option, when running in foreground runc mode, of using New Terminal mode. However, if the process is short-lived, you will not have much time to retrieve the console's fds from the socket and doing anything with it. | ||
|
|
||
| ### Detached | ||
| When running in detached runc mode, you can run your terminal either in Pass-Through or New-Terminal mode. | ||
|
|
||
| You run in detached mode by doing one of the following: | ||
|
|
||
| * `runc -d` | ||
| * `runc create` followed by `runc start`, i.e. container lifecycle management | ||
|
|
||
| #### Pass-Through Detached | ||
| When running runc detached with terminal Pass-Through, the stdin/stdout/stderr of the calling process will be passed to `runc` and from there to the container's process. However, since `runc` detaches, there are several side effects to take into account: | ||
|
|
||
| * The stdout/stderr of the process can become intermixed with the stdout/stderr of the calling process. | ||
| * Any input into the calling process can cause indeterminate issues with which process actually receives the input. | ||
|
|
||
| For example: | ||
|
|
||
| ``` | ||
| $ runc create foo # container created, inheriting stdio from this terminal | ||
| $ runc start foo # container started, inheriting stdio from this terminal | ||
| $ curl https://containerd.io # get contents of this site to the terminal | ||
| <!DOCTYPE html> | ||
| I am container output | ||
| <html lang="en"> | ||
| <head> | ||
| ASashcned more container output | ||
| <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/> | ||
| <meta name="viewport" content="width=device-width, initial-scale=1"> | ||
| <title>containerd</title> | ||
| ``` | ||
|
|
||
| Two problems occurred above: | ||
|
|
||
| 1. The output from the `curl` command and the output of the container `foo` became intermingled. | ||
| 2. Our typing `curl https://containerd.io` actually was passed to both our shell and the container `foo`. The results can be unpredictable. Our example here was simulated and might not even work. | ||
|
|
||
| Further, a third, possibly temporary problem exists. Pass-Through Detached only works correctly when stdout/stderr are either an actual terminal or a file. If you use _anything_ else, e.g. a pipe, it will hang, for reasons unknown. There is an [open issue](https://github.com/opencontainers/runc/issues/1721) on this. | ||
|
|
||
| For this reason, we **strongly recommend** using New Terminal mode when running `runc` detached. | ||
|
|
||
| #### New Terminal Detached | ||
| When running runc detached with New Terminal mode, runc behaves exactly as described above, creating a new pty, passing it to the container, sending the fd to a socket, and exiting. | ||
|
|
||
| ## How To Get the FD for New Terminal | ||
| When running `runc` in New Terminal mode, i.e. `terminal:true` in `config.json` - whether foreground or detached `runc` - you **must** tell `runc` what process will control the master end of the pty. To do that: | ||
|
|
||
| 1. Open a Unix-domain socket | ||
| 2. Pass the path to the socket to `runc create <container> --console-socket <path_to_socket>` | ||
| 3. Listen on the socket to get the master file descriptor | ||
| 4. Use that file descriptor to control stdio | ||
|
|
||
| Note the one shortcoming: the single descriptor means that you have mixed stdout/stderr. There is no way to separate them. Essentially, this is a _console_, which has no separate stdout and stderr. | ||
|
|
||
| The following sample code, partially taken from the reference implementation [recvtty](https://github.com/opencontainers/runc/blob/master/contrib/cmd/recvtty/recvtty.go), shows how to accept a file descriptor for the socket. It does not contain the usual `err` handling and `defer`red closing. | ||
|
|
||
| ```go | ||
| // only partial imports listed | ||
| import ( | ||
| "github.com/containerd/console" | ||
| "github.com/opencontainers/runc/libcontainer/utils" | ||
| ) | ||
|
|
||
| // path is path to a Unix domain socket | ||
| ln, err := net.Listen("unix", path) | ||
| conn, err := ln.Accept() | ||
| unixconn, ok := conn.(*net.UnixConn) | ||
| socket, err := unixconn.File() | ||
|
|
||
| master, err := utils.RecvFd(socket) | ||
| c, err := console.ConsoleFromFile(master) | ||
|
|
||
| // Copy from our stdio to the master fd. | ||
| quitChan := make(chan struct{}) | ||
| go func() { | ||
| io.Copy(os.Stdout, c) | ||
| quitChan <- struct{}{} | ||
| }() | ||
| go func() { | ||
| io.Copy(c, os.Stdin) | ||
| quitChan <- struct{}{} | ||
| }() | ||
|
|
||
| // Only close the master fd once we've stopped copying. | ||
| <-quitChan | ||
| c.Close() | ||
| ``` |