Migrate `libs-through-symlinks` and `translation` run-make tests to rmake #129011

Oneirical · 2024-08-12T16:07:37Z

Part of #121876 and the associated Google Summer of Code project.

Everything here works locally... except line 57 (the test works if it is commented out):

    rustc()
        .arg("-Ztranslate-lang=zh-CN")
        .input("test.rs")
        .sysroot("fakeroot")
        .run_fail()
        .assert_stderr_contains("this is a test message");

This will therefore initially fail CI.

try-job: aarch64-gnu
try-job: aarch64-apple
try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu

rustbot · 2024-08-12T16:07:45Z

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

rustbot · 2024-08-12T16:07:48Z

This PR modifies tests/run-make/. If this PR is trying to port a Makefile
run-make test to use rmake.rs, please update the
run-make port tracking issue
so we can track our progress. You can either modify the tracking issue
directly, or you can comment on the tracking issue and link this PR.

cc @jieyouxu

bors · 2024-08-14T17:43:40Z

☔ The latest upstream changes (presumably #129092) made this pull request unmergeable. Please resolve the merge conflicts.

tests/run-make/libs-through-symlinks/rmake.rs

jieyouxu · 2024-08-16T08:31:51Z

tests/run-make/translation/rmake.rs

Suggestion: can we please group the test cases alongside their comments, like the original Makefile? E.g.

fn test_builtin_fallback_bundle() { // TODO }

Dumping them together is a bit hard to follow.

jieyouxu · 2024-08-16T08:38:02Z

tests/run-make/translation/rmake.rs

+
+fn main() {
+    // Check that the test works normally, using the built-in fallback bundle.
+    rustc().input("test.rs").run_fail().assert_stderr_contains("struct literal body without path");


Suggestion: the RUSTC_TRANSLATION_NO_DEBUG_ASSERT=1 env var might be load-bearing, see

rust/compiler/rustc_errors/src/translation.rs

Lines 109 to 118 in 69e36d6

// Always yeet out for errors on debug (unless

// `RUSTC_TRANSLATION_NO_DEBUG_ASSERT` is set in the environment - this allows

// local runs of the test suites, of builds with debug assertions, to test the

// behaviour in a normal build).

Some(Err(primary))

if cfg!(debug_assertions)

&& env::var("RUSTC_TRANSLATION_NO_DEBUG_ASSERT").is_err() =>

{

do yeet primary

}

We probably want to have a Rustc factory fn that presets that env var:

fn rustc_no_translation_assert() -> Rustc { rustc().env("RUSTC_TRANSLATION_NO_DEBUG_ASSERT", "1") }

@rustbot author

rustc() returns a temporary &mut Rustc which cannot be returned at first glance (unless there is some lifetime magic to pull it off).

I added the environment variable to the rustc calls in the test manually, though in my local tests it seemed to have no effect. Then again, this might be due to me using stage 0 compilation.

Oh right, my bad. Hm.

jieyouxu · 2024-08-16T08:43:16Z

tests/run-make/translation/rmake.rs

+    // translation of rustc's error messages.
+    rustc()
+        .arg("-Ztranslate-lang=tlh")
+        // .input("test.rs")


Remark: commented out

Original test:

sysroot-missing: $(RUSTC) $< -Ztranslate-lang=tlh 2>&1 | $(CGREP) "missing locale directory"

Note how the ususal test.rs next to the section header sysroot-missing: is absent. I have no idea what the $< is slotting in.

rust-log-analyzer · 2024-08-16T15:30:52Z

The job x86_64-gnu-llvm-17 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

------
 > importing cache manifest from ghcr.io/rust-lang/rust-ci-cache:20d3b4d4a2629cbf7865cdbf92fe47512a7c96658c24253a045ff38e8075cd7fb37ca6fcadfa6e6d093333943ad24f6fc4f163ec5b74fd940de9d5bb03eb4d3b:
------
##[endgroup]
Setting extra environment values for docker:  --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-17]
---
sccache: Starting the server...
##[group]Configure the build
configure: processing command line
configure: 
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-17', '--enable-llvm-link-shared', '--set', 'rust.thin-lto-import-instr-limit=10', '--set', 'change-id=99999999', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--set', 'rust.lld=false', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling']
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-17/bin/llvm-config
configure: llvm.link-shared     := True
configure: rust.thin-lto-import-instr-limit := 10
configure: change-id            := 99999999
---
  Downloaded boml v0.3.1
   Compiling boml v0.3.1
   Compiling y v0.1.0 (/checkout/compiler/rustc_codegen_gcc/build_system)
    Finished `release` profile [optimized] target(s) in 3.79s
     Running `/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-codegen/x86_64-unknown-linux-gnu/release/y test --use-system-gcc --use-backend gcc --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/cg_gcc --release --mini-tests --std-tests`
Using system GCC
[BUILD] example
[AOT] mini_core_hello_world
/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/cg_gcc/mini_core_hello_world
abc
---
---- [run-make] tests/run-make/translation stdout ----

error: rmake recipe failed to complete
status: exit status: 101
command: cd "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/translation/rmake_out" && env -u RUSTFLAGS AR="ar" CC="cc" CC_DEFAULT_FLAGS="-ffunction-sections -fdata-sections -fPIC -m64" CXX="c++" CXX_DEFAULT_FLAGS="-ffunction-sections -fdata-sections -fPIC -m64" HOST_RPATH_DIR="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools-bin:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" LD_LIB_PATH_ENVVAR="LD_LIBRARY_PATH" LLVM_BIN_DIR="/usr/lib/llvm-17/bin" LLVM_COMPONENTS="aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets amdgpu amdgpuasmparser amdgpucodegen amdgpudesc amdgpudisassembler amdgpuinfo amdgputargetmca amdgpuutils analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter avr avrasmparser avrcodegen avrdesc avrdisassembler avrinfo binaryformat bitreader bitstreamreader bitwriter bpf bpfasmparser bpfcodegen bpfdesc bpfdisassembler bpfinfo cfguard codegen codegentypes core coroutines coverage debuginfobtf debuginfocodeview debuginfodwarf debuginfogsym debuginfologicalview debuginfomsf debuginfopdb demangle dlltooldriver dwarflinker dwarflinkerparallel dwp engine executionengine extensions filecheck frontendhlsl frontendopenacc frontendopenmp fuzzercli fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interfacestub interpreter ipo irprinter irreader jitlink lanai lanaiasmparser lanaicodegen lanaidesc lanaidisassembler lanaiinfo libdriver lineeditor linker loongarch loongarchasmparser loongarchcodegen loongarchdesc loongarchdisassembler loongarchinfo lto m68k m68kasmparser m68kcodegen m68kdesc m68kdisassembler m68kinfo mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts objcopy object objectyaml option orcjit orcshared orctargetprocess passes perfjitevents powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo riscvtargetmca runtimedyld scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target targetparser textapi transformutils ve veasmparser vecodegen vectorize vedesc vedisassembler veinfo webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo webassemblyutils windowsdriver windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info x86targetmca xcore xcorecodegen xcoredesc xcoredisassembler xcoreinfo xray xtensa xtensaasmparser xtensacodegen xtensadesc xtensadisassembler xtensainfo" LLVM_FILECHECK="/usr/lib/llvm-17/bin/FileCheck" NODE="/usr/bin/node" PYTHON="/usr/bin/python3" RUSTC="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" RUSTDOC="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" SOURCE_ROOT="/checkout" TARGET="x86_64-unknown-linux-gnu" TARGET_RPATH_DIR="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" TARGET_RPATH_ENV="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/translation/rmake_out:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/translation/rmake"
--- stderr -------------------------------
--- stderr -------------------------------
=== HAYSTACK ===
error: failed to load fluent bundle: could not read ftl file: No such file or directory (os error 2)

=== NEEDLE ===
this is a test message
thread 'main' panicked at /checkout/tests/run-make/translation/rmake.rs:88:10:

jieyouxu · 2024-08-22T02:15:55Z

tests/run-make/translation/Makefile

-	rm -f $(FAKEROOT)/share
-	mkdir -p $(FAKEROOT)/share/locale/zh-CN/
-	ln -s $(CURDIR)/working.ftl $(FAKEROOT)/share/locale/zh-CN/basic-translation.ftl
-	$(RUSTC) $< --sysroot $(FAKEROOT) -Ztranslate-lang=zh-CN 2>&1 | $(CGREP) "this is a test message"


Remark (to myself): $< ?

bors · 2024-08-23T11:44:11Z

☔ The latest upstream changes (presumably #129443) made this pull request unmergeable. Please resolve the merge conflicts.

jieyouxu · 2024-08-23T14:08:09Z

@rustbot blocked (on fixing symlink cleanup in compiletest and bootstrap)

jieyouxu · 2024-09-27T13:47:56Z

The symlink trouble I observed previously from compiletest and bootstrap likely came from somehow under non-admin mode artifacts can be produced under admin ownership, which doesn't block this PR now that run_make_support symlink handling is rectified.

rustbot assigned jieyouxu Aug 12, 2024

jieyouxu mentioned this pull request Aug 12, 2024

Tracking Issue for porting run-make tests to use Rust #121876

Open

This comment has been minimized.

Sign in to view

jieyouxu reviewed Aug 16, 2024

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 16, 2024

Oneirical added 2 commits August 16, 2024 10:13

rewrite libs-through-symlinks to rmake

91661a5

rewrite translation to rmake

af4bdbf

Oneirical force-pushed the dont-blink-they-sys-you branch from ea3ebb5 to af4bdbf Compare August 16, 2024 14:37

jieyouxu added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 16, 2024

jieyouxu reviewed Aug 22, 2024

View reviewed changes

rustbot added S-blocked Status: Marked as blocked ❌ on something else such as an RFC or other implementation work. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 23, 2024

jieyouxu added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-blocked Status: Marked as blocked ❌ on something else such as an RFC or other implementation work. labels Sep 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate `libs-through-symlinks` and `translation` run-make tests to rmake #129011

Migrate `libs-through-symlinks` and `translation` run-make tests to rmake #129011

Oneirical commented Aug 12, 2024 •

edited by jieyouxu

Loading

rustbot commented Aug 12, 2024

rustbot commented Aug 12, 2024

This comment has been minimized.

bors commented Aug 14, 2024

jieyouxu Aug 16, 2024

jieyouxu Aug 16, 2024

jieyouxu Aug 16, 2024

Oneirical Aug 16, 2024

jieyouxu Aug 16, 2024

jieyouxu Aug 16, 2024

Oneirical Aug 16, 2024

rust-log-analyzer commented Aug 16, 2024

jieyouxu Aug 22, 2024

bors commented Aug 23, 2024

jieyouxu commented Aug 23, 2024

jieyouxu commented Sep 27, 2024

	// Always yeet out for errors on debug (unless
	// `RUSTC_TRANSLATION_NO_DEBUG_ASSERT` is set in the environment - this allows
	// local runs of the test suites, of builds with debug assertions, to test the
	// behaviour in a normal build).
	Some(Err(primary))
	if cfg!(debug_assertions)
	&& env::var("RUSTC_TRANSLATION_NO_DEBUG_ASSERT").is_err() =>
	{
	do yeet primary
	}

Migrate libs-through-symlinks and translation run-make tests to rmake #129011

Are you sure you want to change the base?

Migrate libs-through-symlinks and translation run-make tests to rmake #129011

Conversation

Oneirical commented Aug 12, 2024 • edited by jieyouxu Loading

rustbot commented Aug 12, 2024

rustbot commented Aug 12, 2024

This comment has been minimized.

bors commented Aug 14, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rust-log-analyzer commented Aug 16, 2024

Choose a reason for hiding this comment

bors commented Aug 23, 2024

jieyouxu commented Aug 23, 2024

jieyouxu commented Sep 27, 2024

Migrate `libs-through-symlinks` and `translation` run-make tests to rmake #129011

Migrate `libs-through-symlinks` and `translation` run-make tests to rmake #129011

Oneirical commented Aug 12, 2024 •

edited by jieyouxu

Loading