diff --git a/README.md b/README.md index 5dbd5d27f..8af280f73 100644 --- a/README.md +++ b/README.md @@ -76,6 +76,24 @@ and execute the following command in the root of the repository: The build files are found in the `book` directory. +### Regenerating `stages.png` + +This requires a LaTeX toolchain installed. +On Debian-based distros you can install it with +``` +$ sudo apt install texlive-latex-base +``` + +Then run +``` +$ pdflatex -output-directory=target src/building/stages.tex +``` + +and take a screenshot of the resulting PDF in your browser: +``` +$ x-www-browser target/stages.pdf +``` + ### Link Validations We use `mdbook-linkcheck` to validate URLs included in our documentation. To perform link checks, uncomment the `[output.linkcheck]` field in the `book.toml` configuration file and install `mdbook-linkcheck` with: diff --git a/src/building/bootstrapping.md b/src/building/bootstrapping.md index dd24699fe..318304dd7 100644 --- a/src/building/bootstrapping.md +++ b/src/building/bootstrapping.md @@ -57,6 +57,75 @@ first build the new compiler with an older compiler and then use that to build the new compiler with itself. For development, you usually only want the `stage1` compiler: `x.py build library/std`. +## Where do stages start and end? + +A common question is what exactly happens when you run `x.py build` or `x.py test`. +Does `--stage 1` mean to _build_ the stage 1 artifacts or to _run_ them? +In fact, it means both! + + + +![stages](./stages.png) + +So, for example, when you run `x.py test [--stage 1]`, +that means to build the compiler in row 1 and column 0, then run it on the testsuite. +This corresponds to the `run-stage` diagram. +However, when you run `x.py build [--stage 1]`, that means to build the compiler in +row 2 and column 1. This corresponds to the `build-stage` diagram. +Building any of the items in the diagram also requires first building all +items with arrows pointing to it. + +The diagram just says `rustc` for simplicity, but this also includes all +programs linked to rustc: + +- `rustdoc` +- `rustfmt` +- `clippy` +- `miri` +- compiler plugins + +Similarly, `std` refers to the whole standard library: + +- `core` +- `alloc` +- `std` +- `test` +- `proc_macro` + +### What are `run-stage` and `build-stage`? + +`run-stage` means that this deals with _running_ the compiler, +so `--stage N` refers to the artifacts in `build/stageN`. + +`build-stage` means that this deals with _building_ the compiler, +and it refers to `build/stageN-component`. + +`build/stageN` is suitable for use with `rustup toolchain link`, +but `stageN-component` never has enough components to be usable (since it only has one). +Copying these artifacts from `stage(N-1)-component` to `stageN` +is called _uplifting_ the artifacts to `stageN`. + +### Why have `build-stage` at all? + +`stage0/bin/rustc` can't open an rlib from stage1-* or vice-versa. +They are completely separate worlds, and `build-stage` reflects those worlds quite directly. +Say you want to build a custom driver and you've run +`rustup toolchain link build/*/stage1`: you have to run +`x.py build --stage 1 src/librustc_driver` to have it available. +The stage number corresponds to the "world" you have to be in to use it. +If this used `run-stage` instead, you'd need `x.py build --stage 1` to build +a regular program, but `x.py build --stage 2 src/librustc_driver` to build a +custom driver. + +### Are there other ways to think about stages? + +Yes! The above explanation focuses on what `rustc` is being _referred_ to - +it describes `build --stage 1 src/rustc` as `build-stage` 1 and `run-stage 2`. +However, another way to think about it is that `--stage 1` refers to +the compiler _being run_, so `build --stage 1 src/rustc` means to _run_ +`stage1/rustc` on the `src/rustc` crate. This can be slightly more confusing +at first, but leads to a more consistent view of 'stage'. + ## Complications of bootstrapping Since the build system uses the current beta compiler to build the stage-1 @@ -152,7 +221,8 @@ Keep in mind this diagram is a simplification, i.e. `rustdoc` can be built at different stages, the process is a bit different when passing flags such as `--keep-stage`, or if there are non-host targets. -The following tables indicate the outputs of various stage actions: +The following tables indicate the outputs of various stage actions +(in this context, 'stage' refers to `build-stage`): | Stage 0 Action | Output | |-----------------------------------------------------------|----------------------------------------------| @@ -192,7 +262,7 @@ The following tables indicate the outputs of various stage actions: `--stage=2` stops here. -Note that the convention `x.py` uses is that: +Note that the convention `x.py` uses for `build-stage` is that: - A "stage N artifact" is an artifact that is _produced_ by the stage N compiler. - The "stage (N+1) compiler" is assembled from "stage N artifacts". - A `--stage N` flag means build _with_ stage N. @@ -276,6 +346,8 @@ artifacts respectively. Additionally, the `RUSTFLAGS_STAGE_NOT_0` variable, as its name suggests, pass the given arguments if the stage is not 0. +In this context, `STAGE` refers to `build-stage`. + ## Environment Variables During bootstrapping, there are a bunch of compiler-internal environment diff --git a/src/building/stages.png b/src/building/stages.png new file mode 100644 index 000000000..75042f4a7 Binary files /dev/null and b/src/building/stages.png differ diff --git a/src/building/stages.tex b/src/building/stages.tex new file mode 100644 index 000000000..81cc742e4 --- /dev/null +++ b/src/building/stages.tex @@ -0,0 +1,79 @@ +\documentclass{standalone} + +\usepackage{tikz} +\usetikzlibrary{arrows.meta} + +\begin{document} + +\noindent +\begin{tikzpicture} + +\node[text width=5in] at (2.5, 2) { +\noindent Arrows represent build dependencies. +Columns are things that are linked together. +Rows are things that can be used together to build a program. +\\ + +\noindent \verb|./x.py build --stage N| builds programs in \emph{column} \verb|N| (except rustdoc - it builds rustdoc from column \verb|N - 1|).\\ +\noindent \verb|./x.py test --stage N| builds and tests everything in \emph{row} \verb|N|.\\ +\noindent \verb|./x.py doc --stage N| generates docs with rustdoc from \emph{row} \verb|N|.\\ +}; + +\draw[draw=black,fill=white,fill opacity=0.5] (-3, -0.5) rectangle ++(12, 1); +\draw[fill=teal,fill opacity=0.5] (-3, -1.5) rectangle ++(12, 1); +\draw[fill=olive,fill opacity=0.5] (-3, -2.5) rectangle ++(12, 1); +\draw[fill=pink,fill opacity=0.5] (-3, -3.5) rectangle ++(12, 1); + +\draw[draw=black,fill=white,fill opacity=0.5] (1, 0.5) rectangle ++(2, -5); +\draw[fill=teal,fill opacity=0.5] (3, 0.5) rectangle ++(2, -5); +\draw[fill=olive,fill opacity=0.5] (5, 0.5) rectangle ++(2, -5); +\draw[fill=pink,fill opacity=0.5] (7, 0.5) rectangle ++(2, -5); + +\node[rotate=90] at (-3.5, -1.5) {run-stage}; + +\node[] at (-2, 0) {stage 0}; +\node[] at (-2, -1) {stage 1}; +\node[] at (-2, -2) {stage 2}; +\node[] at (-2, -3) {stage 3}; + +\node[] at (5, -5) {build-stage}; + +\node[] at (2, -4) {stage 0}; +\node[] at (4, -4) {stage 1}; +\node[] at (6, -4) {stage 2}; +\node[] at (8, -4) {stage 3}; + +\begin{scope}[every node/.style={thick,draw,fill=white}] + \node (s0r) at (0,0) {beta rustc}; + \node (s0s) at (2,0) {std}; + \node (s1r) at (2,-1) {rustc}; + \node (s1s) at (4,-1) {std}; + \node (s2r) at (4,-2) {rustc}; + \node (s2s) at (6,-2) {std}; + \node (s3r) at (6,-3) {rustc}; + \node (s3s) at (8,-3) {std}; +\end{scope} + +\begin{scope}[>={Stealth[black]}, every edge/.style={draw=black,very thick}] + \path [->] (s0r) edge node {} (s0s); + \path [->] (s0r) edge node {} (s1r); + \path [->] (s0s) edge node {} (s1r); + \path [->] (s1r) edge node {} (s1s); + \path [->] (s1r) edge node {} (s2r); + \path [->] (s1s) edge node {} (s2r); + \path [->] (s2r) edge node {} (s2s); + \path [->] (s2r) edge node {} (s3r); + \path [->] (s2s) edge node {} (s3r); + \path [->] (s3r) edge node {} (s3s); +\end{scope} + +\node[text width=5in] at (2.5, -6) { +\noindent \verb|build| excludes \verb|rustc| by default. + +Use \verb|build --stage N src/rustc| to build the \verb|rustc| in \emph{column} +\verb|N|. +}; + +\end{tikzpicture} + +\end{document}