Ensure LLVM is in the link path for rustc tools #70123

cuviper · 2020-03-18T23:32:31Z

The build script for rustc_llvm outputs LLVM information in cargo:rustc-link-lib and cargo:rustc-link-search so the compiler can be linked correctly. However, while the lib is carried along in metadata, the search paths are not. So when cargo is invoked again later for rustc tools, they'll also try to link with LLVM, but the necessary paths may be left out.

Rustbuild can use the environment to set the LLVM link path for tools -- LIB for MSVC toolchains and LIBRARY_PATH for everyone else.

Fixes #68714.

This reverts commit 8b9c539.

rust-highfive · 2020-03-18T23:32:35Z

r? @Mark-Simulacrum

(rust_highfive has picked a reviewer for you, use r? to override)

Mark-Simulacrum · 2020-03-18T23:42:06Z

Seems good to me, and presuming it works for you then great! It looks like this is just a straight propagation of current logic/improvement on the status quo though.

@bors r+

bors · 2020-03-18T23:42:08Z

📌 Commit e1a6a30 has been approved by Mark-Simulacrum

bors · 2020-03-18T23:42:08Z

🌲 The tree is currently closed for pull requests below priority 5, this pull request will be tested once the tree is reopened

rust-highfive · 2020-03-19T01:15:22Z

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2020-03-18T23:33:02.2656388Z ========================== Starting Command Output ===========================
2020-03-18T23:33:02.2658827Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/f1b5852e-4d91-4c65-9041-f880bca363e0.sh
2020-03-18T23:33:02.2659115Z 
2020-03-18T23:33:02.2662805Z ##[section]Finishing: Disable git automatic line ending conversion
2020-03-18T23:33:02.2682468Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/70123/merge to s
2020-03-18T23:33:02.2686159Z Task         : Get sources
2020-03-18T23:33:02.2686484Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-03-18T23:33:02.2686792Z Version      : 1.0.0
2020-03-18T23:33:02.2687011Z Author       : Microsoft
---
2020-03-18T23:33:03.3354189Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-03-18T23:33:03.3410211Z ##[command]git config gc.auto 0
2020-03-18T23:33:03.3448103Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-03-18T23:33:03.3470464Z ##[command]git config --get-all http.proxy
2020-03-18T23:33:03.3554459Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/70123/merge:refs/remotes/pull/70123/merge
---
2020-03-19T00:26:42.2496494Z .................................................................................................... 1700/9796
2020-03-19T00:26:46.1377914Z .................................................................................................... 1800/9796
2020-03-19T00:26:56.3510329Z ..........................................................................i......................... 1900/9796
2020-03-19T00:27:02.4418453Z .................................................................................................... 2000/9796
2020-03-19T00:27:09.5157381Z ................................................................iiiii............................... 2100/9796
2020-03-19T00:27:25.4620988Z .................................................................................................... 2300/9796
2020-03-19T00:27:27.4332707Z .................................................................................................... 2400/9796
2020-03-19T00:27:30.3685753Z .................................................................................................... 2500/9796
2020-03-19T00:27:47.8910166Z .................................................................................................... 2600/9796
---
2020-03-19T00:30:10.6755250Z ....................................i...............i............................................... 5000/9796
2020-03-19T00:30:18.9001140Z .................................................................................................... 5100/9796
2020-03-19T00:30:25.2362588Z ...............................................................................i.................... 5200/9796
2020-03-19T00:30:30.1598014Z .................................................................................................... 5300/9796
2020-03-19T00:30:39.2547436Z ............................................................ii.ii........i...i...................... 5400/9796
2020-03-19T00:30:42.9802851Z ...................................................................................................i 5500/9796
2020-03-19T00:30:54.8089358Z .................................................................................................... 5700/9796
2020-03-19T00:31:00.4134331Z .....................................................i.............................................. 5800/9796
2020-03-19T00:31:06.1361689Z .................................................................................................... 5900/9796
2020-03-19T00:31:14.6945075Z .................................................................................................... 6000/9796
2020-03-19T00:31:14.6945075Z .................................................................................................... 6000/9796
2020-03-19T00:31:20.3980963Z ...............................................ii...i..ii...........i............................... 6100/9796
2020-03-19T00:31:38.4400033Z .................................................................................................... 6300/9796
2020-03-19T00:31:41.6646810Z .................................................................................................... 6400/9796
2020-03-19T00:31:41.6646810Z .................................................................................................... 6400/9796
2020-03-19T00:31:45.7005829Z .............................................................................i..ii.................. 6500/9796
2020-03-19T00:32:05.1380665Z .................................................................................................... 6700/9796
2020-03-19T00:32:13.5450765Z ...........................................................................i........................ 6800/9796
2020-03-19T00:32:15.3928615Z .................................................................................................... 6900/9796
2020-03-19T00:32:17.3572004Z .................................................................................................... 7000/9796
---
2020-03-19T00:33:50.1366966Z .................................................................................................... 7800/9796
2020-03-19T00:33:55.0201055Z .................................................................................................... 7900/9796
2020-03-19T00:34:00.5772330Z .............................................................i...................................... 8000/9796
2020-03-19T00:34:09.5061123Z .................................................................................................... 8100/9796
2020-03-19T00:34:14.3946860Z ..........iiiiiiiiii.i.............................................................................. 8200/9796
2020-03-19T00:34:26.6251013Z .................................................................................................... 8400/9796
2020-03-19T00:34:33.8832972Z .................................................................................................... 8500/9796
2020-03-19T00:34:46.1632801Z .................................................................................................... 8600/9796
2020-03-19T00:34:52.3112576Z .................................................................................................... 8700/9796
---
2020-03-19T00:36:59.3056458Z Suite("src/test/codegen") not skipped for "bootstrap::test::Codegen" -- not in ["src/tools/tidy"]
2020-03-19T00:36:59.3238062Z Check compiletest suite=codegen mode=codegen (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-03-19T00:36:59.5326649Z 
2020-03-19T00:36:59.5327316Z running 183 tests
2020-03-19T00:37:02.2731088Z iiii......i............ii.i..iiii....i....i...........i............i..i..................i....i..... 100/183
2020-03-19T00:37:04.5020683Z .......i.i.i...iii..iiiiiiiiiiiiiiii.......................iii.............ii......
2020-03-19T00:37:04.5026210Z 
2020-03-19T00:37:04.5026671Z  finished in 5.178
2020-03-19T00:37:04.5027657Z Suite("src/test/codegen-units") not skipped for "bootstrap::test::CodegenUnits" -- not in ["src/tools/tidy"]
2020-03-19T00:37:04.5203453Z Check compiletest suite=codegen-units mode=codegen-units (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
---
2020-03-19T00:37:06.5222062Z Suite("src/test/assembly") not skipped for "bootstrap::test::Assembly" -- not in ["src/tools/tidy"]
2020-03-19T00:37:06.5392273Z Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-03-19T00:37:06.6758540Z 
2020-03-19T00:37:06.6758832Z running 9 tests
2020-03-19T00:37:06.6759837Z iiiiiiiii
2020-03-19T00:37:06.6760854Z 
2020-03-19T00:37:06.6771686Z  finished in 0.136
2020-03-19T00:37:06.6772583Z Suite("src/test/incremental") not skipped for "bootstrap::test::Incremental" -- not in ["src/tools/tidy"]
2020-03-19T00:37:06.6938557Z Check compiletest suite=incremental mode=incremental (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
---
2020-03-19T00:37:25.6668631Z Suite("src/test/debuginfo") not skipped for "bootstrap::test::Debuginfo" -- not in ["src/tools/tidy"]
2020-03-19T00:37:25.6904172Z Check compiletest suite=debuginfo mode=debuginfo (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-03-19T00:37:25.8726314Z 
2020-03-19T00:37:25.8730151Z running 115 tests
2020-03-19T00:37:38.3334858Z iiiii..i.....i..i...i..i.i.i..i..i..ii....i.i....ii..........iiii.........i.....i..i.......ii.i.ii.. 100/115
2020-03-19T00:37:39.7482008Z ...iiii.....ii.
2020-03-19T00:37:39.7486231Z 
2020-03-19T00:37:39.7486382Z  finished in 14.058
2020-03-19T00:37:39.7491719Z Suite("src/test/ui-fulldeps") not skipped for "bootstrap::test::UiFullDeps" -- not in ["src/tools/tidy"]
2020-03-19T00:37:39.7497600Z Uplifting stage1 rustc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
---
2020-03-19T00:49:08.6474340Z 
2020-03-19T00:49:08.6484537Z    Doc-tests core
2020-03-19T00:49:13.0211673Z 
2020-03-19T00:49:13.0239830Z running 2481 tests
2020-03-19T00:49:21.5863287Z ......iiiii......................................................................................... 100/2481
2020-03-19T00:49:29.6969674Z .....................................................................................ii............. 200/2481
2020-03-19T00:49:48.4053764Z ....................i............................................................................... 400/2481
2020-03-19T00:49:48.4053764Z ....................i............................................................................... 400/2481
2020-03-19T00:49:57.3715522Z .........................................................................i..i..................iiii. 500/2481
2020-03-19T00:50:12.5912510Z .................................................................................................... 700/2481
2020-03-19T00:50:20.4182450Z .................................................................................................... 800/2481
2020-03-19T00:50:28.2651046Z .................................................................................................... 900/2481
2020-03-19T00:50:36.2701408Z .................................................................................................... 1000/2481
---
2020-03-19T00:53:54.4379781Z 
2020-03-19T00:53:54.4380125Z running 1010 tests
2020-03-19T00:54:10.8645565Z i................................................................................................... 100/1010
2020-03-19T00:54:20.5562360Z .................................................................................................... 200/1010
2020-03-19T00:54:27.1225274Z ..................iii......i......i...i......i...................................................... 300/1010
2020-03-19T00:54:31.9155299Z .................................................................................................... 400/1010
2020-03-19T00:54:38.3380924Z ............................................i..i......................................ii............ 500/1010
2020-03-19T00:54:49.9359106Z .................................................................................................... 700/1010
2020-03-19T00:54:49.9359106Z .................................................................................................... 700/1010
2020-03-19T00:54:56.2836306Z ....................................iiii............................................................ 800/1010
2020-03-19T00:55:09.3737958Z .................................................................................................... 900/1010
2020-03-19T00:55:15.5829357Z ..........................................................iiii...................................... 1000/1010
2020-03-19T00:55:15.9801328Z test result: ok. 990 passed; 0 failed; 20 ignored; 0 measured; 0 filtered out
2020-03-19T00:55:15.9801721Z 
2020-03-19T00:55:15.9899915Z  finished in 154.551
2020-03-19T00:55:15.9916419Z Set({"src/libterm"}) not skipped for "bootstrap::test::Crate" -- not in ["src/tools/tidy"]
---
2020-03-19T01:13:05.7956150Z Suite("src/test/run-make-fulldeps") not skipped for "bootstrap::test::RunMakeFullDeps" -- not in ["src/tools/tidy"]
2020-03-19T01:13:05.8243608Z Check compiletest suite=run-make-fulldeps mode=run-make (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-03-19T01:13:06.0251152Z 
2020-03-19T01:13:06.0251485Z running 210 tests
2020-03-19T01:13:35.5770358Z ......................i...ii.......................................................................i 100/210
2020-03-19T01:14:09.9781951Z ........................................iiiiii......i..............iii.............................. 200/210
2020-03-19T01:14:14.4623828Z test result: ok. 195 passed; 0 failed; 15 ignored; 0 measured; 0 filtered out
2020-03-19T01:14:14.4627742Z 
2020-03-19T01:14:14.4628834Z  finished in 68.638
2020-03-19T01:14:14.4631321Z Set({"src/doc/rustdoc"}) not skipped for "bootstrap::test::RustdocBook" -- not in ["src/tools/tidy"]
---
2020-03-19T01:15:13.8889172Z   local time: Thu Mar 19 01:15:13 UTC 2020
2020-03-19T01:15:14.5039793Z   network time: Thu, 19 Mar 2020 01:15:14 GMT
2020-03-19T01:15:14.5041430Z == end clock drift check ==
2020-03-19T01:15:17.3507109Z 
2020-03-19T01:15:17.3582330Z ##[error]Bash exited with code '1'.
2020-03-19T01:15:17.3596434Z ##[section]Finishing: Run build
2020-03-19T01:15:17.3651864Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/70123/merge to s
2020-03-19T01:15:17.3657002Z Task         : Get sources
2020-03-19T01:15:17.3657375Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-03-19T01:15:17.3657700Z Version      : 1.0.0
2020-03-19T01:15:17.3657931Z Author       : Microsoft
2020-03-19T01:15:17.3657931Z Author       : Microsoft
2020-03-19T01:15:17.3658304Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2020-03-19T01:15:17.3658721Z ==============================================================================
2020-03-19T01:15:17.6897466Z Cleaning any cached credential from repository: rust-lang/rust (GitHub)
2020-03-19T01:15:17.6943308Z ##[section]Finishing: Checkout rust-lang/rust@refs/pull/70123/merge to s
2020-03-19T01:15:17.7023600Z Cleaning up task key
2020-03-19T01:15:17.7024875Z Start cleaning up orphan processes.
2020-03-19T01:15:17.7198536Z Terminate orphan process: pid (3901) (python)
2020-03-19T01:15:17.7758006Z ##[section]Finishing: Finalize Job

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

cuviper · 2020-03-19T01:24:02Z

Hmm, failed in bootstrap's own tests. I can reproduce locally, but it's not helpful:

     Running /home/jistone/rust/rust/build/bootstrap/debug/deps/bootstrap-f9994770a6875357

running 11 tests
test builder::tests::build_default ... error: test failed, to rerun pass '--lib'


command did not execute successfully: "/home/jistone/rust/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "test" "--" "--test-threads=1"
expected success, got: exit code: 1

I'll investigate more tomorrow...

@bors r-

cuviper · 2020-03-19T17:30:28Z

OK, in a debugger I found the silence was because the test exits via build_helper::fail, which bypasses the normal test harness. Printing s from there, or running with --nocapture, I see:

failed to execute command: "/home/jistone/rust/rust/build/tmp-rustbuild-tests/main/A/llvm/build/bin/llvm-config" "--libdir"
error: No such file or directory (os error 2)

I pushed a new commit which should be more robust about missing llvm-config.

Mark-Simulacrum · 2020-03-19T17:38:17Z

r=me with CI passing, just in case

cuviper · 2020-03-19T19:37:17Z

@bors r=Mark-Simulacrum

bors · 2020-03-19T19:37:19Z

📌 Commit 3a2a442 has been approved by Mark-Simulacrum

@ghost

Rollup of 8 pull requests Successful merges: - rust-lang#69080 (rustc_codegen_llvm: don't generate any type debuginfo for -Cdebuginfo=1.) - rust-lang#69940 (librustc_codegen_llvm: Replace deprecated API usage) - rust-lang#69942 (Increase verbosity when suggesting subtle code changes) - rust-lang#69968 (rustc: keep upvars tupled in {Closure,Generator}Substs.) - rust-lang#70123 (Ensure LLVM is in the link path for rustc tools) - rust-lang#70159 (Update the bundled wasi-libc with libstd) - rust-lang#70233 (resolve: Do not resolve visibilities on proc macro definitions twice) - rust-lang#70286 (Miri error type: remove UbExperimental variant) Failed merges: r? @ghost

This is a follow-up to rust-lang#70123, which added `llvm-config --libdir` to the `LIBRARY_PATH` for rustc tools. We need the same for "run-make-fulldeps" and "ui-fulldeps" tests which depend on compiler libraries, implicitly needing to link to `-lLLVM` as well.

…rk-Simulacrum Ensure LLVM is in the link path for "fulldeps" tests This is a follow-up to rust-lang#70123, which added `llvm-config --libdir` to the `LIBRARY_PATH` for rustc tools. We need the same for "run-make-fulldeps" and "ui-fulldeps" tests which depend on compiler libraries, implicitly needing to link to `-lLLVM` as well.

cuviper added 3 commits March 18, 2020 13:49

Rename add_lib_path to add_dylib_path

71f5aed

Ensure LLVM is in the link path for rustc tools

0536b8d

Revert "Fix missing libLLVM.so in stage0 sysroot."

e1a6a30

This reverts commit 8b9c539.

rust-highfive assigned Mark-Simulacrum Mar 18, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 18, 2020

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 18, 2020

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Mar 19, 2020

Avoid llvm-config in more situations, like bootstrap test runs

3a2a442

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Mar 19, 2020

Centril mentioned this pull request Mar 23, 2020

Rollup of 8 pull requests #70305

Merged

bors merged commit 9423c4f into rust-lang:master Mar 23, 2020

cuviper mentioned this pull request Mar 30, 2020

Ensure LLVM is in the link path for "fulldeps" tests #70591

Merged

cuviper deleted the library-path branch April 3, 2020 18:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ensure LLVM is in the link path for rustc tools #70123

Ensure LLVM is in the link path for rustc tools #70123

cuviper commented Mar 18, 2020

rust-highfive commented Mar 18, 2020

Mark-Simulacrum commented Mar 18, 2020

bors commented Mar 18, 2020

bors commented Mar 18, 2020

rust-highfive commented Mar 19, 2020

cuviper commented Mar 19, 2020

cuviper commented Mar 19, 2020

Mark-Simulacrum commented Mar 19, 2020

cuviper commented Mar 19, 2020

bors commented Mar 19, 2020

Ensure LLVM is in the link path for rustc tools #70123

Ensure LLVM is in the link path for rustc tools #70123

Conversation

cuviper commented Mar 18, 2020

rust-highfive commented Mar 18, 2020

Mark-Simulacrum commented Mar 18, 2020

bors commented Mar 18, 2020

bors commented Mar 18, 2020

rust-highfive commented Mar 19, 2020

cuviper commented Mar 19, 2020

cuviper commented Mar 19, 2020

Mark-Simulacrum commented Mar 19, 2020

cuviper commented Mar 19, 2020

bors commented Mar 19, 2020