Auto merge of rust-lang#64882 - ehuss:stabilize-bare-extern, r=eddyb

Stabilize --extern flag without a path.

This stabilizes the `--extern` flag without a path, implemented in rust-lang#54116.

This flag is used to add a crate that may be found in the search path to the extern prelude. The intent of stabilizing this now is to change Cargo to emit this flag for `proc_macro` when building a proc-macro crate. This will allow the ability to elide `extern crate proc_macro;` for proc-macros, one of the few places where it is still necessary.

It is intended that Cargo may also use this flag for other cases in the future as part of the [std-aware work](https://github.com/rust-lang/wg-cargo-std-aware/). There will likely be some kind of syntax where users may declare dependencies on other crates (such as `alloc`), and Cargo will use this flag so that they may be used like any other crate. At this time there are no short-term plans to use it for anything other than proc-macro.

This will not help for non-proc-macro crates that use `proc_macro`, which I believe is not too common?

An alternate approach for proc-macro is to use the `meta` crate, but from my inquiries there doesn't appear to be anyone interested in pushing that forward. The `meta` crate also doesn't help with things like `alloc` or `test`.

cc rust-lang#57288

src/doc/rustc/src/codegen-options/index.md

@@ -197,7 +197,11 @@ in software.
 ## prefer-dynamic
 
 By default, `rustc` prefers to statically link dependencies. This option will
-make it use dynamic linking instead.
+indicate that dynamic linking should be used if possible if both a static and
+dynamic versions of a library are available. There is an internal algorithm
+for determining whether or not it is possible to statically or dynamically
+link with a dependency. For example, `cdylib` crate types may only use static
+linkage.
 
 ## no-integrated-as
 

src/doc/rustc/src/command-line-arguments.md

@@ -21,8 +21,7 @@ to `#[cfg(verbose)]` and `#[cfg(feature = "serde")]` respectively.
 <a id="option-l-search-path"></a>
 ## `-L`: add a directory to the library search path
 
-When looking for external crates or libraries, a directory passed to this flag
-will be searched.
+The `-L` flag adds a path to search for external crates and libraries.
 
 The kind of search path can optionally be specified with the form `-L
 KIND=PATH` where `KIND` may be one of:
@@ -262,9 +261,30 @@ This flag, when combined with other flags, makes them produce extra output.
 <a id="option-extern"></a>
 ## `--extern`: specify where an external library is located
 
-This flag allows you to pass the name and location of an external crate that
-will be linked into the crate you are building. This flag may be specified
-multiple times. The format of the value should be `CRATENAME=PATH`.
+This flag allows you to pass the name and location for an external crate of a
+direct dependency. Indirect dependencies (dependencies of dependencies) are
+located using the [`-L` flag](#option-l-search-path). The given crate name is
+added to the [extern prelude], which is the same as specifying `extern crate`
+within the root module. The given crate name does not need to match the name
+the library was built with.
+
+This flag may be specified multiple times. This flag takes an argument with
+either of the following formats:
+
+* `CRATENAME=PATH` — Indicates the given crate is found at the given path.
+* `CRATENAME` — Indicates the given crate may be found in the search path,
+  such as within the sysroot or via the `-L` flag.
+
+The same crate name may be specified multiple times for different crate types.
+If both an `rlib` and `dylib` are found, an internal algorithm is used to
+decide which to use for linking. The [`-C prefer-dynamic`
+flag][prefer-dynamic] may be used to influence which is used.
+
+If the same crate name is specified with and without a path, the one with the
+path is used and the pathless flag has no effect.
+
+[extern prelude]: ../reference/items/extern-crates.html#extern-prelude
+[prefer-dynamic]: codegen-options/index.md#prefer-dynamic
 
 <a id="option-sysroot"></a>
 ## `--sysroot`: Override the system root

src/librustc/session/config.rs

@@ -1800,7 +1800,7 @@ pub fn rustc_optgroups() -> Vec<RustcOptGroup> {
             "",
             "extern",
             "Specify where an external rust library is located",
-            "NAME=PATH",
+            "NAME[=PATH]",
         ),
         opt::multi_s(
             "",
@@ -2164,7 +2164,6 @@ fn collect_print_requests(
     cg: &mut CodegenOptions,
     dopts: &mut DebuggingOptions,
     matches: &getopts::Matches,
-    is_unstable_enabled: bool,
     error_format: ErrorOutputType,
 ) -> Vec<PrintRequest> {
     let mut prints = Vec::<PrintRequest>::new();
@@ -2206,7 +2205,7 @@ fn collect_print_requests(
         "tls-models" => PrintRequest::TlsModels,
         "native-static-libs" => PrintRequest::NativeStaticLibs,
         "target-spec-json" => {
-            if is_unstable_enabled {
+            if dopts.unstable_options {
                 PrintRequest::TargetSpec
             } else {
                 early_error(
@@ -2370,7 +2369,6 @@ fn parse_externs(
     matches: &getopts::Matches,
     debugging_opts: &DebuggingOptions,
     error_format: ErrorOutputType,
-    is_unstable_enabled: bool,
 ) -> Externs {
     if matches.opt_present("extern-private") && !debugging_opts.unstable_options {
         early_error(
@@ -2392,13 +2390,6 @@ fn parse_externs(
         let name = parts.next().unwrap_or_else(||
             early_error(error_format, "--extern value must not be empty"));
         let location = parts.next().map(|s| s.to_string());
-        if location.is_none() && !is_unstable_enabled {
-            early_error(
-                error_format,
-                "the `-Z unstable-options` flag must also be passed to \
-                 enable `--extern crate_name` without `=path`",
-            );
-        };
 
         let entry = externs
             .entry(name.to_owned())
@@ -2483,12 +2474,10 @@ pub fn build_session_options(matches: &getopts::Matches) -> Options {
         );
     }
 
-    let is_unstable_enabled = nightly_options::is_unstable_enabled(matches);
     let prints = collect_print_requests(
         &mut cg,
         &mut debugging_opts,
         matches,
-        is_unstable_enabled,
         error_format,
     );
 
@@ -2521,7 +2510,7 @@ pub fn build_session_options(matches: &getopts::Matches) -> Options {
         );
     }
 
-    let externs = parse_externs(matches, &debugging_opts, error_format, is_unstable_enabled);
+    let externs = parse_externs(matches, &debugging_opts, error_format);
 
     let crate_name = matches.opt_str("crate-name");
 

src/librustdoc/config.rs

@@ -615,10 +615,6 @@ fn parse_externs(matches: &getopts::Matches) -> Result<Externs, String> {
         let mut parts = arg.splitn(2, '=');
         let name = parts.next().ok_or("--extern value must not be empty".to_string())?;
         let location = parts.next().map(|s| s.to_string());
-        if location.is_none() && !nightly_options::is_unstable_enabled(matches) {
-            return Err("the `-Z unstable-options` flag must also be passed to \
-                        enable `--extern crate_name` without `=path`".to_string());
-        }
         let name = name.to_string();
         // For Rustdoc purposes, we can treat all externs as public
         externs.entry(name)

src/librustdoc/lib.rs

@@ -140,7 +140,7 @@ fn opts() -> Vec<RustcOptGroup> {
         }),
         stable("cfg", |o| o.optmulti("", "cfg", "pass a --cfg to rustc", "")),
         stable("extern", |o| {
-            o.optmulti("", "extern", "pass an --extern to rustc", "NAME=PATH")
+            o.optmulti("", "extern", "pass an --extern to rustc", "NAME[=PATH]")
         }),
         unstable("extern-html-root-url", |o| {
             o.optmulti("", "extern-html-root-url",

src/test/run-make-fulldeps/extern-flag-fun/Makefile

@@ -4,7 +4,6 @@ all:
 	$(RUSTC) bar.rs --crate-type=rlib
 	$(RUSTC) bar.rs --crate-type=rlib -C extra-filename=-a
 	$(RUSTC) bar-alt.rs --crate-type=rlib
-	$(RUSTC) foo.rs --extern hello && exit 1 || exit 0
 	$(RUSTC) foo.rs --extern bar=no-exist && exit 1 || exit 0
 	$(RUSTC) foo.rs --extern bar=foo.rs && exit 1 || exit 0
 	$(RUSTC) foo.rs \
@@ -15,3 +14,6 @@ all:
 		--extern bar=$(TMPDIR)/libbar.rlib \
 		--extern bar=$(TMPDIR)/libbar-a.rlib
 	$(RUSTC) foo.rs --extern bar=$(TMPDIR)/libbar.rlib
+	# Try to be sneaky and load a private crate from with a non-private name.
+	$(RUSTC) rustc.rs -Zforce-unstable-if-unmarked --crate-type=rlib
+	$(RUSTC) gated_unstable.rs --extern alloc=$(TMPDIR)/librustc.rlib 2>&1 | $(CGREP) 'rustc_private'

src/test/run-make-fulldeps/extern-flag-fun/gated_unstable.rs

@@ -0,0 +1,3 @@
+extern crate alloc;
+
+fn main() {}

src/test/run-make-fulldeps/extern-flag-fun/rustc.rs

@@ -0,0 +1 @@
+pub fn foo() {}

src/test/run-make-fulldeps/extern-flag-pathless/Makefile

@@ -0,0 +1,18 @@
+-include ../tools.mk
+
+# Test mixing pathless --extern with paths.
+
+all:
+	$(RUSTC) bar-static.rs --crate-name=bar --crate-type=rlib
+	$(RUSTC) bar-dynamic.rs --crate-name=bar --crate-type=dylib -C prefer-dynamic
+	# rlib preferred over dylib
+	$(RUSTC) foo.rs --extern bar
+	$(call RUN,foo) | $(CGREP) 'static'
+	$(RUSTC) foo.rs --extern bar=$(TMPDIR)/libbar.rlib --extern bar
+	$(call RUN,foo) | $(CGREP) 'static'
+	# explicit --extern overrides pathless
+	$(RUSTC) foo.rs --extern bar=$(call DYLIB,bar) --extern bar
+	$(call RUN,foo) | $(CGREP) 'dynamic'
+	# prefer-dynamic does what it says
+	$(RUSTC) foo.rs --extern bar -C prefer-dynamic
+	$(call RUN,foo) | $(CGREP) 'dynamic'

src/test/run-make-fulldeps/extern-flag-pathless/bar-dynamic.rs

@@ -0,0 +1,3 @@
+pub fn f() {
+    println!("dynamic");
+}

src/test/run-make-fulldeps/extern-flag-pathless/bar-static.rs

@@ -0,0 +1,3 @@
+pub fn f() {
+    println!("static");
+}

src/test/run-make-fulldeps/extern-flag-pathless/foo.rs

@@ -0,0 +1,3 @@
+fn main() {
+    bar::f();
+}

src/test/run-make-fulldeps/save-analysis-rfc2126/Makefile

@@ -1,8 +1,7 @@
 -include ../tools.mk
 
 all: extern_absolute_paths.rs krate2
-	$(RUSTC) extern_absolute_paths.rs -Zsave-analysis --edition=2018 \
-		-Z unstable-options --extern krate2
+	$(RUSTC) extern_absolute_paths.rs -Zsave-analysis --edition=2018 --extern krate2
 	cat $(TMPDIR)/save-analysis/extern_absolute_paths.json | "$(PYTHON)" validate_json.py
 
 krate2: krate2.rs

src/test/rustdoc/inline_cross/use_crate.rs

@@ -2,7 +2,7 @@
 // aux-build:use_crate_2.rs
 // build-aux-docs
 // edition:2018
-// compile-flags:--extern use_crate --extern use_crate_2 -Z unstable-options
+// compile-flags:--extern use_crate --extern use_crate_2
 
 // During the buildup to Rust 2018, rustdoc would eagerly inline `pub use some_crate;` as if it
 // were a module, so we changed it to make `pub use`ing crate roots remain as a `pub use` statement

src/test/ui-fulldeps/pathless-extern-unstable.rs

@@ -0,0 +1,10 @@
+// ignore-stage1
+// edition:2018
+// compile-flags:--extern rustc
+
+// Test that `--extern rustc` fails with `rustc_private`.
+
+pub use rustc;
+//~^ ERROR use of unstable library feature 'rustc_private'
+
+fn main() {}

src/test/ui-fulldeps/pathless-extern-unstable.stderr

@@ -0,0 +1,12 @@
+error[E0658]: use of unstable library feature 'rustc_private': this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via `Cargo.toml` instead?
+  --> $DIR/pathless-extern-unstable.rs:7:9
+   |
+LL | pub use rustc;
+   |         ^^^^^
+   |
+   = note: for more information, see https://github.com/rust-lang/rust/issues/27812
+   = help: add `#![feature(rustc_private)]` to the crate attributes to enable
+
+error: aborting due to previous error
+
+For more information about this error, try `rustc --explain E0658`.

src/test/ui/pathless-extern-ok.rs

@@ -0,0 +1,9 @@
+// edition:2018
+// compile-flags:--extern alloc
+// build-pass
+
+// Test that `--extern alloc` will load from the sysroot without error.
+
+fn main() {
+    let _: Vec<i32> = alloc::vec::Vec::new();
+}