Changes to name resolution

[nrc]

Some internal and language-level changes to name resolution.

Internally, name resolution will be split into two parts - import resolution and
name lookup. Import resolution is moved forward in time to happen in the same
phase as parsing and macro expansion. Name lookup remains where name resolution
currently takes place (that may change in the future, but is outside the scope
of this RFC). However, name lookup can be done earlier if required (importantly
it can be done during macro expansion to allow using the module system for
macros, also outside the scope of this RFC). Import resolution will use a new
algorithm.

The observable effects of this RFC (i.e., language changes) are some increased
flexibility in the name resolution rules, especially around globs and shadowing.

rendered

[ticki]

I am especially excited about the implications for macro resolution, since these are, as of now, really annoying. Having them fitting better into the module system would be an awesome change.

[nrc]

cc @petrochenkov @jseyfried

[sgrif]

This RFC, specifically the ability to import the same item multiple times without conflict, solves one of the bigger unresolved pains in Diesel. The ability to have a local name shadow an import feels like a loss to me, though. When you take macros into account, it's possible for defining a struct to silently break something, with a fairly non-local error. I'd much prefer something akin to hiding from Haskell.

mod foo {
    pub struct Bar;
}

use foo::* hiding Bar;

struct Bar;

[retep998]

Would this be legal as a result of this RFC?

mod foo {
pub mod apple {
    use super::*;
    pub type Carrot = u32;
}
pub mod banana {
    use super::*;
    pub type Carrot = u32;
}
pub use self::apple::*;
pub use self::banana::*;
}
fn main() {
    let x: foo::Carrot = 5;
}

[petrochenkov]

@retep998
No, types are still processed after name resolution, so apple::Carrot and banana::Carrot are considered different items.
Instead you can use primitive types like u32 in imports through one more level of indirection:

mod primitive {
    pub type u32_ = u32;
}
pub mod apple {
    pub use primitive::u32_ as Carrot;
}
pub mod banana {
    pub use primitive::u32_ as Carrot;
}

(Or even nicer, on >=1.9 with two levels of indirection: gist)

[petrochenkov]

To clarify, this RFC doesn't propose the ability to leak private items through macros, right? I.e.

struct S;
pub macro s {
    // Still an error, privacy is checked in the context
    // of macro expansion and not macro definition,
    // private `S` cannot be leaked.
    () => (S)
}

[nrc]

No, there are no user-visible changes to privacy or macros in this RFC. I do plan to propose this later though.

[retep998]

@petrochenkov The reason I ask is due to winapi, where sometimes two (or more!) headers will both define the same typedef, and I'd rather not have to pick a header to be the canonical one (nevermind that if I want to be able to cfg out some modules, I'd have really complex cfg conditions to re-enable the non-canonical typedefs). If I define a typedef via type or use in a crate, is there a difference between the two for downstream consumers (assuming it is a primitive type or a non-tuple struct)?

[nrc]

@retep998 There is a difference between type and use - the former are processed later in the compiler and are not part of the module system. At name resolution time two different aliases to the same type are considered different items. Two different use imports of the same item are considered the same for the purposes of 'overlapping' imports.

One could imagine erasing aliases earlier in the compilation process and thus making them transparent to name resolution (and the shadowing rules). I think that should be a separate RFC and I'm not sure if it is a good idea or not.

[glaebhoerl]

I suspect that would be a bad idea if extended to generic type aliases.

[eddyb]

When in doubt, combine your options: lazy lowering (similar to HIR+Ty -> HAIR for MIR building), triggered by type collection (top-level items) or type inference (expressions inside functions) that uses binding tables to determine what nodes to produce.

[Ericson2314]

What exactly does this mean? Don't parse macro definitions? invocations? the result of macro expansion?

[eddyb]

We don't attempt to parse the invocations of a macro in any way (other than storing them as TTs).

[Ericson2314]

Ok, that of the 3 makes the most sense but just wanted to check :)

[Ericson2314]

In order to keep macro expansion comprehensible to programmers, we must enforce that all macro uses resolve to the same binding at the end of resolution as they do when they were resolved.

I think this is required for basic soundness, not just peace of mind. Say two macro use sites, a and b, expand to a macro definition used at a third use site, c. Expanding a then c vs b then c leads to a different result, yet at each expansion of c the binding for c's macro was hitherto unambiguous. The only red flag at the end is the overlapping macro bindings once a and b are expanded.

[Ericson2314]

Worse yet, if a and b expand to glob imports providing binding the macro expanded at c, there isn't actually an error at the end of the [a, c, b] or [b, a, c] expansions (in the final states themselves), because overlapping glob imports are not a problem.

[xeno-by]

How are you planning to mitigate this risk?

[Ericson2314]

Would this be a good time to fix rust-lang/rust#26775 ?

[jseyfried]

@Ericson2314 #26775 was fixed by #31362.

[Ericson2314]

@jseyfried pub extern crate seem to still not work for me, though?! It doesn't look that that was addressed by your fix yet rust-lang/rust#21757 was closed as dupe of rust-lang/rust#26775. edit nevermind, sorry for confusion.

Also does https://github.com/nrc/rfcs/blob/aac02932e3c3b5749d1c15b83813ca1c3ef00281/text/0000-name-resolution.md#non-public-imports apply to (non-pub) extern crate?

[jseyfried]

@Ericson2314 Perhaps you are using stable? The fix is on beta but hasn't landed on stable yet. If it doesn't work for you on beta or nightly, could you provide an example?

extern crates are currently treated the way that section proposes we treat use imports, so it doesn't really apply.

[Ericson2314]

@jseyfried edited my comment but my mistake it does indeed work; for the record I am on nightly.

[Ericson2314]

Ah and yes I somehow also missed the current semantics of extern crate are indeed that, so this + your fix unifies everything as is desired. Yay!

[nikomatsakis]

I'm finding it hard to reason about the precise model proposed here, I admit. I wonder if there is a way to make the write up a bit more declarative.

In any case, I think the only change here that I found problematic when working on my name resolution prototype was this one:

Note that shadowing is namespace specific. I believe this is consistent with our general approach to name spaces.

I think the problem had to do with the possibility that, in some round, we would bring in a name from a glob that would later have to be shadowed, but we didn't know at the time whether it ought to be shadowed or not. I'll try to see if I can come up with a precise example.

[jseyfried]

I think the problem had to do with the possibility that, in some round, we would bring in a name from a glob that would later have to be shadowed, but we didn't know at the time whether it ought to be shadowed or not.

This is primarily why we currently distinguish between Indeterminate and Failed imports, which adds quite a bit of complexity (cf. this code and this code). For example,

mod bar {
    pub fn foo() {}
    pub use baz::*;
}
mod baz {
    pub use bar::*;
}   

pub mod foo {}
mod quux {
    pub use super::*; // This defines `quux::foo` in the type namespace ...
    pub use bar::foo;
    //^ ... but we can't know that until we know that `bar::foo` fails in the type namespace,
    //| which requires us to detect that `bar -> baz -> bar -> ...` is a glob cycle.
}

If we changed the semantics so that a single import (and perhaps also an ordinary item for consistency) always shadowed glob imports in both namespaces (even if it only defined one of them), then we wouldn't need to distinguish between failing and indeterminate resolutions.

I think this change would cause too much breakage to be practical, but if you'd like I could implement it and we could do a crater run to see if doing a warning cycle would be feasible.

[jseyfried]

Also, while the complexity is unfortunate, it does generalize well in the presence of unexpanded macros -- we would just have to report that an otherwise failing name in a module with unexpanded macros is indeterminate.

[Ericson2314]

Oh, it just occurred to me that this should mean that a single glob use at the top-level module should suffice for desguaring an automatic prelude import, right?

[petrochenkov]

I forgot to mention a drawback - names (and paths) don't have a definitive "point of introduction" anymore.
I.e. you can't say "the name was defined/imported here", there is always can be >1 of such definition/import points.
Unless all of these points are tracked, this complicates, for example, stability checking for reexports or error reporting if errors refer to these points.