[Bug]: `cargo uitest` fails with `[E0463]: can't find crate for std` #324

tigerros · 2023-11-26T02:42:57Z

Summary

I'm not positive that this is a bug, but running Marker on a different crate instead of testing does work, so I think something is wrong with the testing.

Reproducer

Windows 10

git clone https://github.com/rust-marker/lint-crate-template.git
cd lint-crate-template
cargo uitest

Using the latest nightly toolchain and latest Marker dependencies (0.4.2) doesn't help.

Version

cargo-marker 0.4.2

Logs

Updating crates.io index
   Compiling proc-macro2 v1.0.70
   Compiling unicode-ident v1.0.12
   Compiling serde v1.0.193
   Compiling cfg-if v1.0.0
   Compiling once_cell v1.18.0
   Compiling lazy_static v1.4.0
   Compiling windows_x86_64_msvc v0.48.5
   Compiling windows_x86_64_msvc v0.42.2
   Compiling winapi v0.3.9
   Compiling unicode-width v0.1.11
   Compiling sharded-slab v0.1.7
   Compiling tracing-core v0.1.32
   Compiling thread_local v1.1.7
   Compiling pin-project-lite v0.2.13
   Compiling serde_json v1.0.108
   Compiling windows-targets v0.48.5
   Compiling semver v1.0.20
   Compiling cc v1.0.83
   Compiling libc v0.2.150
   Compiling windows-sys v0.48.0
   Compiling quote v1.0.33
   Compiling windows-targets v0.42.2
   Compiling syn v2.0.39
   Compiling tracing v0.1.40
   Compiling tracing-subscriber v0.3.18
   Compiling thiserror v1.0.50
   Compiling portable-atomic v1.5.1
   Compiling regex-syntax v0.8.2
   Compiling ryu v1.0.15
   Compiling eyre v0.6.9
   Compiling crossbeam-utils v0.8.16
   Compiling anyhow v1.0.75
   Compiling itoa v1.0.9
   Compiling camino v1.1.6
   Compiling tracing-error v0.2.0
   Compiling regex-automata v0.4.3
   Compiling backtrace v0.3.69
   Compiling windows-sys v0.45.0
   Compiling rustc-demangle v0.1.23
   Compiling encode_unicode v0.3.6
   Compiling indenter v0.3.3
   Compiling owo-colors v3.5.0
   Compiling console v0.15.7
   Compiling color-spantrace v0.2.1
   Compiling serde_derive v1.0.193
   Compiling thiserror-impl v1.0.50
   Compiling ansi_term v0.12.1
   Compiling yansi-term v0.1.2
   Compiling is-terminal v0.4.9
   Compiling pad v0.1.6
   Compiling number_prefix v0.4.0
   Compiling log v0.4.20
   Compiling memchr v2.6.4
   Compiling fastrand v2.0.1
   Compiling indicatif v0.17.7
   Compiling prettydiff v0.6.4
   Compiling tempfile v3.8.1
   Compiling colored v2.0.4
   Compiling annotate-snippets v0.9.2
   Compiling bstr v1.8.0
   Compiling crossbeam-channel v0.5.8
   Compiling regex v1.10.2
   Compiling color-eyre v0.6.2
   Compiling levenshtein v1.0.5
   Compiling comma v1.0.0
   Compiling marker_api v0.4.2
   Compiling marker_utils v0.4.2
   Compiling cargo-platform v0.1.5
   Compiling rustc_version v0.4.0
   Compiling cargo_metadata v0.15.4
   Compiling rustfix v0.6.1
   Compiling ui_test v0.21.2
   Compiling marker_uitest v0.4.2
   Compiling lint-crate-template v0.1.0 (C:\Users\aurel\source\repos\lint-crate-template)
    Finished test [unoptimized + debuginfo] target(s) in 22.29s
     Running tests\uitest.rs (target\debug\deps\uitest-ab45871b164387e1.exe)
marker_rustc_driver 0.4.2 (c753799 2023-11-25)


FAILED TEST: tests\ui\hello_marker.rs
command: "C:\\Users\\aurel\\.rustup\\toolchains\\nightly-2023-11-16-x86_64-pc-windows-msvc\\bin\\marker_rustc_driver.exe" "--error-format=json" "-Aunused" "--out-dir" "\\\\?\\C:\\Users\\aurel\\source\\repos\\lint-crate-template\\target\\ui_test\\tests\\ui" "tests\\ui\\hello_marker.rs" "--edition" "2021"

error: actual output differed from expected
Execute `cargo test -- -- --bless` to update `tests\ui\hello_marker.stderr` to the actual output
--- tests\ui\hello_marker.stderr
+++ <stderr output>
-warning: hello, main (From Marker)
+error[E0463]: can't find crate for `std`
~ --> $DIR/hello_marker.rs:5:1
   |
-5 | / fn main() {
+6 |     println!("Hello, World!");
~6 | |     println!("Hello, World!");^^^^^^^

-warning: 1 warning emitted
+error: requires `sized` lang_item

+error: aborting due to 3 previous errors
+
+For more information about this error, try `rustc --explain E0463`.


full stderr:
error[E0463]: can't find crate for `std`

error: cannot find macro `println` in this scope
 --> tests\ui\hello_marker.rs:6:5
  |
6 |     println!("Hello, World!");
  |     ^^^^^^^

error: requires `sized` lang_item

error: aborting due to 3 previous errors

For more information about this error, try `rustc --explain E0463`.

full stdout:


FAILURES:
    tests\ui\hello_marker.rs

test result: FAIL. 1 failed;

Error: tests failed

Location:
    C:\Users\aurel\.cargo\registry\src\index.crates.io-6f17d22bba15001f\ui_test-0.21.2\src\lib.rs:388:13
error: test failed, to rerun pass `--test uitest`

Caused by:
  process didn't exit successfully: `C:\Users\aurel\source\repos\lint-crate-template\target\debug\deps\uitest-ab45871b164387e1.exe` (exit code: 1)

The text was updated successfully, but these errors were encountered:

xFrednet · 2023-11-29T14:11:45Z

Thank you for the report! Have you been able to run Marker on a normal crate, or does that also happen there?

tigerros · 2023-11-29T17:25:35Z

Thank you for the report! Have you been able to run Marker on a normal crate, or does that also happen there?

Running cargo marker on a crate that uses the same lint crate works, if that's what you mean.

xFrednet · 2023-11-29T21:46:16Z

Yes, that's what I meant. Then it seems to be a problem with the configuration of Marker's testing framework 🤔

I'm not quite sure what could cause it, but that's what debugging is for. Since it works on my Linux machine and the CI of that crate, I'm guessing that it has something to do with windows.

@tigerros, would you mind running two tests for me?

Add the "rust-std" component in the component list of the rust-toolchain.toml file of the lint crate, and run the tests again?
And could you enable more debugging information, via the following environment variables MARKER_ERROR_TRACE=1 MARKER_LOG=info?

That would be super helpful!

tigerros · 2023-11-29T22:12:43Z

I'm pretty sure I enabled logging in the first log... Anyway, same issue.

Log

    Updating crates.io index
   Compiling proc-macro2 v1.0.70
   Compiling unicode-ident v1.0.12
   Compiling cfg-if v1.0.0
   Compiling serde v1.0.193
   Compiling lazy_static v1.4.0
   Compiling once_cell v1.18.0
   Compiling windows_x86_64_msvc v0.48.5
   Compiling winapi v0.3.9
   Compiling windows_x86_64_msvc v0.42.2
   Compiling unicode-width v0.1.11
   Compiling sharded-slab v0.1.7
   Compiling serde_json v1.0.108
   Compiling tracing-core v0.1.32
   Compiling thread_local v1.1.7
   Compiling windows-targets v0.48.5
   Compiling libc v0.2.150
   Compiling semver v1.0.20
   Compiling pin-project-lite v0.2.13
   Compiling cc v1.0.83
   Compiling tracing v0.1.40
   Compiling windows-sys v0.48.0
   Compiling windows-targets v0.42.2
   Compiling quote v1.0.33
   Compiling tracing-subscriber v0.3.18
   Compiling syn v2.0.39
   Compiling portable-atomic v1.5.1
   Compiling ryu v1.0.15
   Compiling itoa v1.0.9
   Compiling thiserror v1.0.50
   Compiling crossbeam-utils v0.8.16
   Compiling anyhow v1.0.75
   Compiling regex-syntax v0.8.2
   Compiling camino v1.1.6
   Compiling eyre v0.6.9
   Compiling windows-sys v0.45.0
   Compiling rustc-demangle v0.1.23
   Compiling tracing-error v0.2.0
   Compiling encode_unicode v0.3.6
   Compiling indenter v0.3.3
   Compiling backtrace v0.3.69
   Compiling owo-colors v3.5.0
   Compiling yansi-term v0.1.2
   Compiling regex-automata v0.4.3
   Compiling console v0.15.7
   Compiling color-spantrace v0.2.1
   Compiling ansi_term v0.12.1
   Compiling is-terminal v0.4.9
   Compiling pad v0.1.6
   Compiling serde_derive v1.0.193
   Compiling thiserror-impl v1.0.50
   Compiling memchr v2.6.4
   Compiling number_prefix v0.4.0
   Compiling log v0.4.20
   Compiling fastrand v2.0.1
   Compiling indicatif v0.17.7
   Compiling prettydiff v0.6.4
   Compiling tempfile v3.8.1
   Compiling crossbeam-channel v0.5.8
   Compiling colored v2.0.4
   Compiling color-eyre v0.6.2
   Compiling regex v1.10.2
   Compiling bstr v1.8.0
   Compiling annotate-snippets v0.9.2
   Compiling levenshtein v1.0.5
   Compiling comma v1.0.0
   Compiling marker_api v0.4.2
   Compiling marker_utils v0.4.2
   Compiling cargo-platform v0.1.5
   Compiling rustc_version v0.4.0
   Compiling rustfix v0.6.1
   Compiling cargo_metadata v0.15.4
   Compiling ui_test v0.21.2
   Compiling marker_uitest v0.4.2
   Compiling lint-crate-template v0.1.0 (C:\Users\aurel\source\repos\lint-crate-template)
    Finished test [unoptimized + debuginfo] target(s) in 18.56s
     Running tests\uitest.rs (target\debug\deps\uitest-ab45871b164387e1.exe)
marker_rustc_driver 0.4.2 (c753799 2023-11-25)


FAILED TEST: tests\ui\hello_marker.rs
command: "C:\\Users\\aurel\\.rustup\\toolchains\\nightly-2023-11-16-x86_64-pc-windows-msvc\\bin\\marker_rustc_driver.exe" "--error-format=json" "-Aunused" "--out-dir" "\\\\?\\C:\\Users\\aurel\\source\\repos\\lint-crate-template\\target\\ui_test\\tests\\ui" "tests\\ui\\hello_marker.rs" "--edition" "2021"

error: actual output differed from expected
Execute `cargo test -- -- --bless` to update `tests\ui\hello_marker.stderr` to the actual output
--- tests\ui\hello_marker.stderr
+++ <stderr output>
-warning: hello, main (From Marker)
+error[E0463]: can't find crate for `std`
~ --> $DIR/hello_marker.rs:5:1
   |
-5 | / fn main() {
+6 |     println!("Hello, World!");
~6 | |     println!("Hello, World!");^^^^^^^

-warning: 1 warning emitted
+error: requires `sized` lang_item

+error: aborting due to 3 previous errors
+
+For more information about this error, try `rustc --explain E0463`.


full stderr:
error[E0463]: can't find crate for `std`

error: cannot find macro `println` in this scope
 --> tests\ui\hello_marker.rs:6:5
  |
6 |     println!("Hello, World!");
  |     ^^^^^^^

error: requires `sized` lang_item

error: aborting due to 3 previous errors

For more information about this error, try `rustc --explain E0463`.

full stdout:


FAILURES:
    tests\ui\hello_marker.rs

test result: FAIL. 1 failed;

Error: tests failed

Location:
    C:\Users\aurel\.cargo\registry\src\index.crates.io-6f17d22bba15001f\ui_test-0.21.2\src\lib.rs:388:13
error: test failed, to rerun pass `--test uitest`

Caused by:
  process didn't exit successfully: `C:\Users\aurel\source\repos\lint-crate-template\target\debug\deps\uitest-ab45871b164387e1.exe` (exit code: 1)

Veetaha · 2023-11-29T22:46:26Z

@xFrednet I believe I already saw this issue when debugging something unrelated long ago, but didn't think it would be reproducable outside of my debugging setup. The core problem is with custom handling of --sysroot parameter in the driver here:

marker/marker_rustc_driver/src/lib.rs

Lines 309 to 353 in e1dfde8

    
           /// Get the sysroot, looking from most specific to this invocation to the least: 
        
           /// - command line 
        
           /// - runtime environment 
        
           ///    - `SYSROOT` 
        
           ///    - `RUSTUP_HOME`, `MULTIRUST_HOME`, `RUSTUP_TOOLCHAIN`, `MULTIRUST_TOOLCHAIN` 
        
           /// - sysroot from rustc in the path 
        
           /// - compile-time environment 
        
           ///    - `SYSROOT` 
        
           ///    - `RUSTUP_HOME`, `MULTIRUST_HOME`, `RUSTUP_TOOLCHAIN`, `MULTIRUST_TOOLCHAIN` 
        
           fn find_sys_root(sys_root_arg: Option<&str>) -> String { 
        
               sys_root_arg 
        
                   .map(Utf8PathBuf::from) 
        
                   .or_else(|| std::env::var("SYSROOT").ok().map(Utf8PathBuf::from)) 
        
                   .or_else(|| { 
        
                       let home = std::env::var("RUSTUP_HOME") 
        
                           .or_else(|_| std::env::var("MULTIRUST_HOME")) 
        
                           .ok(); 
        
                       let toolchain = std::env::var("RUSTUP_TOOLCHAIN") 
        
                           .or_else(|_| std::env::var("MULTIRUST_TOOLCHAIN")) 
        
                           .ok(); 
        
                       toolchain_path(home, toolchain) 
        
                   }) 
        
                   .or_else(|| { 
        
                       Command::new("rustc") 
        
                           .arg("--print") 
        
                           .arg("sysroot") 
        
                           .output() 
        
                           .ok() 
        
                           .and_then(|out| String::from_utf8(out.stdout).ok()) 
        
                           .map(|s| Utf8PathBuf::from(s.trim())) 
        
                   }) 
        
                   .or_else(|| option_env!("SYSROOT").map(Utf8PathBuf::from)) 
        
                   .or_else(|| { 
        
                       let home = option_env!("RUSTUP_HOME") 
        
                           .or(option_env!("MULTIRUST_HOME")) 
        
                           .map(ToString::to_string); 
        
                       let toolchain = option_env!("RUSTUP_TOOLCHAIN") 
        
                           .or(option_env!("MULTIRUST_TOOLCHAIN")) 
        
                           .map(ToString::to_string); 
        
                       toolchain_path(home, toolchain) 
        
                   }) 
        
                   .map(Utf8PathBuf::into_string) 
        
                   .expect("need to specify SYSROOT env var during marker compilation, or use rustup or multirust") 
        
           }

I don't remember the conditions under which the problem reproduces, and I'm not sure that's windows specific. Some combination of RUSTUP_TOOLCHAIN and rust-toolchain or +{toolchain} plus the CARGO env vars in the test process causes this. The behavior that is implemented in that referenced function differs from real rustc's behavior. At that moment I wondered why that function event exists in the first place, and why not to forward processing of that parameter to RunCompiler, but didn't go further.

xFrednet · 2023-11-30T09:59:00Z

I had the feeling this code will come back to bite me. This way of getting the sysroot was taken from Clippy, with some refactorings for readability. I never use the --sysroot flag and don't know why it requires special handling in tools.

Some months back, there was a PR in rustc to move this special handling code into rustc
rust-lang/rust#103660. I've created #48 to migrate Marker as well, but haven't gotten too it yet, since I assumed it worked fine as is.

Checking Clippy's driver code, it looks like there is still some code to forward the sysroot flag, but it's definitely cleaner than the old setup:
https://github.com/rust-lang/rust-clippy/blob/8b0bf6423dfaf5545014db85fcba7bc745beed4c/src/driver.rs#L191-L200

Edit: I just looked it up, apparently the flag was added for cross compilation: rust-lang/cargo#2312 It might be used for other things as well, but that seems to be the origin.

xFrednet · 2023-12-02T16:48:13Z

@tigerros We've created a release candidate for v0.4.3, which might fix the bug. Would you mind testing it on your machine?

This is the install script for the release candidate. You might have to specify v0.4.3-rc in the lint crate template, instead of the v0.4 version which is the default. If this fixes the bug, we'll do a proper v0.4.3 release.

curl.exe -fsSL https://raw.githubusercontent.com/rust-marker/marker/v0.4.3-rc/scripts/release/install.ps1 | powershell -command -

xFrednet · 2023-12-02T16:52:41Z

I think I figured out another problem... When I released v0.4.1 I accidentally did that from the master branch, which updated the used nightly toolchain. This update never made it to the master of the lint crate template. I'll fix that! Let's hope that fixes everything

Edit: The upside of this mess, is that I learned to never release hotfix versions from the master branch again

xFrednet · 2023-12-02T16:58:34Z

Okay, now everything should be back in order. Sorry 😅

tigerros · 2023-12-02T17:16:02Z

@xFrednet It works. And yes, it's necessary to specify 0.4.3-rc for the marker deps in the template. Thanks!

xFrednet · 2023-12-02T19:28:16Z

Perfect, thank you!

tigerros added C-bug Category: Something isn't working S-needs-triage Status: This issue needs triage labels Nov 26, 2023

tigerros changed the title ~~[Bug]: cargo uitest fails with [E0463]: can't find crate for std``~~ [Bug]: cargo uitest fails with [E0463]: can't find crate for std` Nov 26, 2023

tigerros changed the title ~~[Bug]: cargo uitest fails with [E0463]: can't find crate for std`~~ [Bug]: cargo uitest fails with [E0463]: can't find crate for std Nov 26, 2023

tigerros changed the title ~~[Bug]: cargo uitest fails with [E0463]: can't find crate for std~~ [Bug]: cargo uitest fails with [E0463]: can't find crate for \std\`` Nov 26, 2023

tigerros changed the title ~~[Bug]: cargo uitest fails with [E0463]: can't find crate for \std\``~~ [Bug]: cargo uitest fails with [E0463]: can't find crate for std Nov 26, 2023

xFrednet added this to the v0.5.0 milestone Nov 29, 2023

xFrednet added D-rustc-driver Driver: Rustc Driver A-marker-uitest Area: ui-test infrastructure (marker_uitests) and removed S-needs-triage Status: This issue needs triage labels Nov 29, 2023

xFrednet mentioned this issue Dec 1, 2023

Rustc: Simplify --sysroot code after rust#103660 #325

Merged

xFrednet mentioned this issue Dec 2, 2023

Update toolchain after Marker v0.4.2 🔧 rust-marker/lint-crate-template#8

Merged

tigerros closed this as completed Dec 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: `cargo uitest` fails with `[E0463]: can't find crate for std` #324

[Bug]: `cargo uitest` fails with `[E0463]: can't find crate for std` #324

tigerros commented Nov 26, 2023 •

edited

Loading

xFrednet commented Nov 29, 2023

tigerros commented Nov 29, 2023

xFrednet commented Nov 29, 2023 •

edited

Loading

tigerros commented Nov 29, 2023

Veetaha commented Nov 29, 2023 •

edited

Loading

xFrednet commented Nov 30, 2023 •

edited

Loading

xFrednet commented Dec 2, 2023

xFrednet commented Dec 2, 2023 •

edited

Loading

xFrednet commented Dec 2, 2023

tigerros commented Dec 2, 2023 •

edited

Loading

xFrednet commented Dec 2, 2023

[Bug]: cargo uitest fails with [E0463]: can't find crate for std #324

[Bug]: cargo uitest fails with [E0463]: can't find crate for std #324

Comments

tigerros commented Nov 26, 2023 • edited Loading

Summary

Reproducer

Version

Logs

xFrednet commented Nov 29, 2023

tigerros commented Nov 29, 2023

xFrednet commented Nov 29, 2023 • edited Loading

tigerros commented Nov 29, 2023

Veetaha commented Nov 29, 2023 • edited Loading

xFrednet commented Nov 30, 2023 • edited Loading

xFrednet commented Dec 2, 2023

xFrednet commented Dec 2, 2023 • edited Loading

xFrednet commented Dec 2, 2023

tigerros commented Dec 2, 2023 • edited Loading

xFrednet commented Dec 2, 2023

[Bug]: `cargo uitest` fails with `[E0463]: can't find crate for std` #324

[Bug]: `cargo uitest` fails with `[E0463]: can't find crate for std` #324

tigerros commented Nov 26, 2023 •

edited

Loading

xFrednet commented Nov 29, 2023 •

edited

Loading

Veetaha commented Nov 29, 2023 •

edited

Loading

xFrednet commented Nov 30, 2023 •

edited

Loading

xFrednet commented Dec 2, 2023 •

edited

Loading

tigerros commented Dec 2, 2023 •

edited

Loading