rustc private crates have unresolved references due to rustc_macros not being expandable

[Veykril]

When using the rustc_private crates in a project (like rust-clippy for example), some things won't be resolved by r-a, because it can't expand the macros used in the rustc_private crates. The cause for that is that cargo doesn't rebuild the crates when we build the project for build-scripts and proc-macros, so we never built the necessary proc-macros, causing r-a to fall back to using the dummy expander for the relevant proc-macros.

[thomcc]

It's worth noting that this seems to fail for some declarative macros and not just proc-macros as far as I can tell.

It'd be nice if there were some work-around, as it makes working on out-of-tree rustc_private code a fair bit more annoying.

[Veykril]

Which declarative macros? That's a different issue.

[bjorn3]

For proc macros rust-analyzer could look in the rustc sysroot for a .so with the correct name, right? This only works if rustc-dev is installed, but if you are working on code that uses rustc private crates, you probably have it installed anyway. (By the way you can use rustc --print target-libdir to get the right dir to look at even if rustc changes the sysroot layout for sysroot deps)

[Veykril]

Oh I didn't know those were shipped, yes we can most likely use them then. Though I guess for that we'd need to know which lib belongs to what crate, since we aren't invoking the buildscripts that mapping is foreign to us.

[bjorn3]

For the proc macro crate rustc_macros you did look at librustc_macros-*.so on Linux with prefix and suffix changed as necessary on other targets. Some for other proc macros.

rustc-dev contains all proc macros as rustc requires that all dependencies are available when depending on a crate, even if a dependency is a proc macro that isn't used by the current crate.

[Veykril]

Right that should work fine, I assumed there might be duplicate versions of some of them but that doesn't seem to be the case.

[bjorn3]

Right, duplicate versions would be a problem. If it every happens, rustc -Zls may help with figuring out which proc macro corresponds to which source version by listing the dependencies of crates that depend on the proc macro and then solving the constraints this produces to find which proc macro version corresponds to which source version.

[Veykril]

Ye let's ignore that for now, it should hopefully not happen :) #14282 fixes this, @thomcc if you could open a new issue with an example of a declarative macro you are having issues with that would be great.

[thomcc]

Wow, this owns, thank you!

It seems that my case was a declarative macro that expanded into a proc macro (I thought I had checked that it wasn't, but I got lost in my mental expansion of it).

[nilehmann]

Is there anything special I need to enable to make #14282 work?

I'm trying to get completions or use go to definition for stuff in rustc_span::sym in a fresh clone of Clippy without luck. I'm in v0.3.1435 which I think should include the changes. Or maybe I misunderstood what #14282 was supposed to fix?

These are all my settings in vscode

{
  "rust-analyzer.inlayHints.maxLength": 40,
  "rust-analyzer.inlayHints.renderColons": false,
  "rust-analyzer.procMacro.attributes.enable": true,
  "rust-analyzer.inlayHints.parameterHints.enable": false,
  "rust-analyzer.rustc.source": "discover",
  "rust-analyzer.completion.privateEditable.enable": true
}

[Veykril]

"rust-analyzer.rustc.source": "discover", is the only setting you'll need that deviates from the default for this, so I'm not sure why its not working for you.

[nilehmann]

Ok I think I found the issue, in my laptop the names of the dynamic libraries are prefixed with lib so it fails to find matches here

rust-analyzer/crates/project-model/src/build_scripts.rs

Line 433 in 9549753

proc_macro_dylibs.iter().find(|(name, _)| *name == package.name)

I assume some linux/mac mismatch.

The following patch fixes it for me

diff --git a/crates/project-model/src/build_scripts.rs b/crates/project-model/src/build_scripts.rs
index 6df1273ed..ca768ebcc 100644
--- a/crates/project-model/src/build_scripts.rs
+++ b/crates/project-model/src/build_scripts.rs
@@ -429,8 +429,9 @@ impl WorkspaceBuildScripts {
             for p in rustc.packages() {
                 let package = &rustc[p];
                 if package.targets.iter().any(|&it| rustc[it].is_proc_macro) {
-                    if let Some((_, path)) =
-                        proc_macro_dylibs.iter().find(|(name, _)| *name == package.name)
+                    if let Some((_, path)) = proc_macro_dylibs
+                        .iter()
+                        .find(|(name, _)| name == &format!("lib{}", package.name))
                     {
                         bs.outputs[p].proc_macro_dylib_path = Some(path.clone());
                     }