NLL should identify and respect the lifetime annotations that the user wrote #48482

davidtwco · 2018-02-23T21:04:01Z

Part of #47184.

davidtwco · 2018-02-23T21:05:13Z

As of writing, this PR is not complete. It doesn't actually solve the issue yet, it just contains what I've got so far.

It adds the majority of what was included in the mentoring instructions (I have not yet added the special case in the renumbering code as I wasn't sure about that).

nikomatsakis · 2018-02-23T22:27:13Z

@davidtwco nice! will look soon

nikomatsakis · 2018-02-27T19:03:33Z

src/librustc_mir/build/block.rs

                                })
                            }));
                    } else {
                        this.visit_bindings(&pattern, &mut |this, _, _, node, span, _| {
                            this.storage_live_binding(block, node, span);
+                            if let Some(ty) = ty {


So, this won't work for something like let (a, b): (u32, u32) = (22, 44); -- it will assert that both a and b have type (u32, u32), I believe.

I'm not sure how most elegantly to handle this case right now -- one way would be to track the ty as we visit the bindings, so that we can destructure it appropriately. Another would be to "build back up" the value from the types of the bindings. I don't like either one, unfortunately :(

(I wonder if we can push the problem to typeck, actually, since it already has to do "destructure" the type hint, I think?)

nikomatsakis · 2018-02-27T19:05:44Z

src/librustc_mir/build/matches/mod.rs

@@ -145,8 +145,18 @@ impl<'a, 'gcx, 'tcx> Builder<'a, 'gcx, 'tcx> {
        end_block.unit()
    }

+    pub fn user_assert_ty(&mut self, block: BasicBlock, ty: Ty<'tcx>, var: NodeId, span: Span) {


I've been debating whether my idea to make this a statement was good or not -- but I do see some advantages. For example, it neatly addresses the question of where this subtyping needs to be enforced.

nikomatsakis · 2018-02-27T19:10:16Z

src/librustc_mir/transform/cleanup_post_borrowck.rs

+                       statement: &mut Statement<'tcx>,
+                       location: Location) {
+        if let StatementKind::UserAssertTy(..) = statement.kind {
+            statement.kind = StatementKind::Nop;


Nit: statement.make_nop();

Fixed this.

nikomatsakis · 2018-02-27T19:10:34Z

src/librustc/session/config.rs

@@ -1117,6 +1117,8 @@ options! {DebuggingOptions, DebuggingSetter, basic_debugging_options,
        "make unnamed regions display as '# (where # is some non-ident unique id)"),
    emit_end_regions: bool = (false, parse_bool, [UNTRACKED],
        "emit EndRegion as part of MIR; enable transforms that solely process EndRegion"),
+    include_user_assert_ty: bool = (false, parse_bool, [UNTRACKED],
+        "include UserAssertTy as part of MIR dumps"),


I think we should just always include it...?

Will remove this flag in a future commit.

Removed this.

nikomatsakis · 2018-02-27T19:11:36Z

src/librustc_mir/transform/cleanup_post_borrowck.rs

@@ -54,8 +54,10 @@ impl MirPass for CleanupPostBorrowck {
            delete.visit_mir(mir);
        }

-        let mut delete = DeleteUserAssertTy;
-        delete.visit_mir(mir);
+        if !tcx.sess.opts.debugging_opts.include_user_assert_ty {


Also, it seems like this flag isn't quite what it suggests -- that is, it doesn't "include it" in the dump, it "stops deleting it". Can't you just use -Ζdump-mir and inspect the results from earlier passes? For example, I generally do -Zdump-mir=nll

I completely forgot I could view the MIR from earlier passes, so yeah, I'll remove this flag.

Removed this.

nikomatsakis · 2018-02-27T19:13:08Z

src/librustc_mir/borrow_check/nll/type_check/mod.rs

@@ -776,6 +776,7 @@ impl<'a, 'gcx, 'tcx> TypeChecker<'a, 'gcx, 'tcx> {
            | StatementKind::InlineAsm { .. }
            | StatementKind::EndRegion(_)
            | StatementKind::Validate(..)
+            | StatementKind::UserAssertTy(..)


I think we want to do a check here -- something pretty similar to what Assign does. i.e., get the type of the place whose type is being asserted and invoke self.sub_types() (or actually eq_type perhaps).

I intended to add a check here in a future commit, I initially included the variant here just so I could see if being included in the MIR without it doing anything.

Added something that uses eq_type, not sure it's doing 100% of what it needs to though.

nikomatsakis · 2018-03-01T19:21:27Z

src/librustc_mir/borrow_check/nll/type_check/mod.rs

+                let local_ty = mir.local_decls()[*local].ty;
+                debug!("check_stmt: user_assert_ty ty={:?} local_ty={:?}", ty, local_ty);
+                if let Err(terr) =
+                    self.eq_types(ty, local_ty, location.at_successor_within_block())


I think this should be at_self_within_block, though it probably doesn't matter. But this basically means that the type must be equal "on entry" to the statement, which feels "intuitively" right to me.

davidtwco · 2018-03-02T14:42:36Z

So, with everything that is implemented at current, I'm not seeing the error from UserAssertTy being produced.

Currently, the eq_type call is comparing the following:

DEBUG:<unknown>: check_stmt: user_assert_ty ty=std::vec::Vec<&'static std::string::String> local_ty=std::vec::Vec<&'static std::string::String>

I've experimented with ways to support the let (a, b): (u32, u32) = (22, 44); case mentioned in the previous review regarding this, but also couldn't find a particularly elegant way to handle this in the parts of the code that are currently being changed.

I'm uncertain what the next steps for this PR are? @nikomatsakis

bors · 2018-03-05T21:47:27Z

☔ The latest upstream changes (presumably #48208) made this pull request unmergeable. Please resolve the merge conflicts.

bors · 2018-03-08T11:37:16Z

☔ The latest upstream changes (presumably #46882) made this pull request unmergeable. Please resolve the merge conflicts.

nikomatsakis

You are also missing changes to the renumberer. In particular, I think we want to override visit_user_assert_ty in that impl to do nothing, so that we preserve the user's types.

I think this should fix your test case!

Sorry for the super delayed feedback.

nikomatsakis · 2018-03-09T23:18:40Z

src/librustc_mir/transform/cleanup_post_borrowck.rs

@@ -42,15 +42,15 @@ impl MirPass for CleanEndRegions {
                          tcx: TyCtxt<'a, 'tcx, 'tcx>,
                          _source: MirSource,
                          mir: &mut Mir<'tcx>) {
-        if !tcx.emit_end_regions() { return; }
+        if tcx.emit_end_regions() {


any particular reason for this hunk, or just cleanup? (I sort of prefer the "return early" variant)

So do I actually, this was just left-over from when I had CleanEndRegions and CleanUserAssertTy combined as a single pass and I didn't want an early return to stop the deletion of the UserAssertTy. I can change it back.

nikomatsakis · 2018-03-09T23:30:44Z

src/librustc/mir/visit.rs

@@ -619,6 +629,11 @@ macro_rules! make_mir_visitor {
                }
            }

+            fn super_user_assert_ty(&mut self,
+                                    _ty: & $($mutability)* Ty<'tcx>,
+                                    _local: & $($mutability)* Local) {


You are missing some things here:

self.visit_ty(ty); self.visit_local(local);

davidtwco · 2018-03-10T23:23:39Z

It's now causing an error as we would expect (though, almost certainly not the error we want in the end). If I'm not mistaken, we still need to handle the let (a, b): (u32, u32) = (22, 44); case and where let x: &u32 = ...; needs to be distinguished from let x: &'static u32 = ...; (as per the original instructions).

Normally I'd copy the error from the AST borrowck, but I notice that on nightly with the AST borrowck the error is an ICE while on stable with the AST borrowck it is a normal error. Just noticed this while writing, so until now I wasn't sure what error it should have been - was always seeing the nightly ICE while working locally. Might have been going a bit mad when I wrote this, will be addressed in future commits.

nikomatsakis · 2018-03-12T17:16:31Z

src/test/ui/issue-47184.rs

+
+fn main() {
+    let vec: Vec<&'static String> = vec![&String::new()];
+    //~^ ERROR


What error do we get now?

As far as I can remember (not able to run this at the moment), we got an ICE produced by this section of the code.

Though, strangely, Travis isn't showing this. Perhaps I got mixed up and saw something else.

nikomatsakis · 2018-03-12T17:17:02Z

src/librustc_mir/transform/cleanup_post_borrowck.rs

+    }
+}
+
+pub struct CleanUserAssertTy;


why are these two distinct things?

If you are referring to the separation of the passes, in my experimentation locally, I found that the passes had to happen at different times otherwise the UserAssertTy statement wouldn't be in the MIR when the NLL type check code gets it (since the type check pass doesn't run when #![feature(nll)] was used - as far as I could tell - ordering it relative to that wasn't viable). Therefore they had to be separate. If this doesn't sound right, I can take another look.

Hmm. I didn't realize that these passes were running at distinct times. At minimum, could use a comment clarifying what is happening here =)

I'll add one with my next push.

This discussion should be resolved - I updated the comment on the module to mention this. It doesn't affect these lines specifically so this isn't marked as "outdated".

nikomatsakis · 2018-03-12T17:18:33Z

src/librustc_mir/borrow_check/nll/renumber.rs

@@ -118,6 +118,11 @@ impl<'a, 'gcx, 'tcx> MutVisitor<'tcx> for NLLVisitor<'a, 'gcx, 'tcx> {
        debug!("visit_closure_substs: substs={:?}", substs);
    }

+    fn visit_user_assert_ty(&mut self, _ty: &mut Ty<'tcx>, _local: &mut Local,
+                            _location: Location) {
+        debug!("visit_user_assert_ty: skipping renumber");


It'd be nice to have a comment here. Something like:

User-assert-ty statements represent types that the user added explicitly.
We don't want to erase the regions from these types: rather, we want to
add them as constraints at type-check time.

Will add this as soon as I get a chance.

davidtwco · 2018-03-13T14:56:48Z

I've updated the test in this PR with the expected error that we see in the AST borrowck. What was happening is the following:

We were seeing an ICE occuring in the to_region_vid call from this section of the code. This was what I mistakenly thought was the expected error from above - oops!

This is just because it is visiting the type that didn't get renumbered, which we don't want, so I've overridden the visit_user_assert_ty fn here in the newest commits. It might be the case that those statements should have been removed by our pass by the time it got to this point.

I'm not entirely sure where our DeleteUserAssertTy pass should happen, if I put it at the same place the CleanEndRegions pass happens then the statement is removed before the user type assertion check. That's why those two passes are separate and for now, I've just put the DeleteUserAssertTy pass somewhere that I found it worked.

Upon doing this, I expected to see no error be reported - as I previously wasn't seeing the ICE that I thought would be thrown where our user type assertion is performed. Digging in a little, it seems I made a bad assumption regarding all that eq_types actually does (from a cursory glance, I assumed that when it returned Err(..) then the types didn't match and that's where we'd handle error throwing, that evidently is not the case). Given that, it seems like we're actually seeing this do what it is supposed to now (locally the issue-47184 test passes with the expected UI output) - yay!

Looking over initial instructions and previous comments, this still doesn't handle the let (a, b): (u32, u32) = (22, 44); case and the case where let x: &u32 = ...; needs to be distinguished from let x: &'static u32 = ...;.

davidtwco · 2018-03-13T15:24:28Z

~~After chatting with @nikomatsakis on Gitter, here's a list of everywhere within function bodies where we annotate types.~~ Moved this to the issue.

nikomatsakis · 2018-03-13T15:53:21Z

@davidtwco turbofish covers more cases than that. Example:

let x = SomeStruct::<'static> { field: u32 };

So really it's most paths. I guess it'd be useful though to try and go over the kinds of places paths can appear that matter:

struct names
value references (e.g., let x = foo::<...>; where foo is a fn or something)
method calls (foo.method::<arg>)
fully qualified paths (<T as ..>::method)

(Also worth noting is that these paths can all include inference variables like _ or '_, and we don't want to record those variables, but let's ignore that for now, I have thoughts there but #48411 has to land first)

nikomatsakis · 2018-03-13T15:53:34Z

that said, can we move this list to the issue, not this PR?

nikomatsakis

Awesome. Left a few nits but this is basically good to go.

nikomatsakis · 2018-03-23T17:38:13Z

src/librustc/mir/mod.rs

+    ///     let (a, b): (T, U) = y;
+    ///
+    /// Here we would insert a `UserAssertTy<(T, U)>(y)` instruction to check that the type of `y`
+    /// is the right thing.


It is worth adding to the docs that the Canonical is used to capture "inference variables" from user's types.

For example:

let x: Vec<_> = ...; let y: &u32 = ...;

would result in Vec<?0> and &'?0 u32 respectively (where ?0 is a canonicalized variable).

This has been added but it is slightly below the reviewed line so GitHub hasn't noticed.

nikomatsakis · 2018-03-23T17:39:11Z

src/librustc/ty/context.rs

@@ -344,6 +345,9 @@ pub struct TypeckTables<'tcx> {
    /// method calls, including those of overloaded operators.
    type_dependent_defs: ItemLocalMap<Def>,

+    /// Stores the canonicalized types provided by the user.
+    user_provided_tys: ItemLocalMap<CanonicalTy<'tcx>>,


Maybe add a "see also: the UserAssertTy statement in MIR" or something like that?

nikomatsakis · 2018-03-23T17:39:42Z

src/librustc_mir/borrow_check/nll/type_check/mod.rs

+                let (ty, _) = self.infcx.instantiate_canonical_with_fresh_inference_vars(
+                    stmt.source_info.span, c_ty);
+                debug!("check_stmt: user_assert_ty ty={:?} local_ty={:?}", ty, local_ty);
+                if let Err(terr) = self.eq_types(ty, local_ty, location.at_self()) {


nikomatsakis · 2018-03-23T17:40:16Z

src/librustc_mir/build/block.rs

                                })
                            }));
                    } else {
+                        // FIXME: We currently only insert `UserAssertTy` statements for patterns


Can we say FIXME(#47184)? We no longer mandate it, but I still prefer to have issue labels where we can.

nikomatsakis · 2018-03-23T17:58:43Z

@bors r+

bors · 2018-03-23T17:58:44Z

📌 Commit 4161ae7 has been approved by nikomatsakis

@nikomatsakis

NLL should identify and respect the lifetime annotations that the user wrote Part of rust-lang#47184. r? @nikomatsakis

bors · 2018-03-24T02:08:29Z

⌛ Testing commit 4161ae7 with merge ab0ef14...

@nikomatsakis

NLL should identify and respect the lifetime annotations that the user wrote Part of #47184. r? @nikomatsakis

bors · 2018-03-24T04:36:06Z

☀️ Test successful - status-appveyor, status-travis
Approved by: nikomatsakis
Pushing ab0ef14 to master...

rust-highfive assigned nikomatsakis Feb 23, 2018

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 23, 2018

davidtwco mentioned this pull request Feb 23, 2018

NLL should identify and respect the lifetime annotations that the user wrote #47184

Closed

27 tasks

davidtwco closed this Feb 25, 2018

davidtwco deleted the issue-47184 branch February 25, 2018 12:17

davidtwco restored the issue-47184 branch February 27, 2018 17:28

davidtwco reopened this Feb 27, 2018

nikomatsakis requested changes Feb 27, 2018

View reviewed changes

davidtwco force-pushed the issue-47184 branch 2 times, most recently from 386ddec to 12d6706 Compare March 1, 2018 14:33

nikomatsakis reviewed Mar 1, 2018

View reviewed changes

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 5, 2018

davidtwco force-pushed the issue-47184 branch from b09c238 to 1ea5920 Compare March 6, 2018 15:50

davidtwco force-pushed the issue-47184 branch from 1ea5920 to e65c6b3 Compare March 8, 2018 14:22

nikomatsakis requested changes Mar 9, 2018

View reviewed changes

davidtwco force-pushed the issue-47184 branch 2 times, most recently from d062708 to e66fa1b Compare March 10, 2018 23:17

nikomatsakis reviewed Mar 12, 2018

View reviewed changes

davidtwco and others added 3 commits March 22, 2018 21:11

Temporarily only adding UserAssertTy on binding patterns.

fc5c4da

Debug logs for replace_bound_regions_with_nll_infer_vars

04aeef8

Added flag to disable user type assertion.

447ae76

davidtwco force-pushed the issue-47184 branch from aad73f2 to 447ae76 Compare March 22, 2018 22:48

Updated MIR with UserAssertTy in mir-opt tests.

03481f1

davidtwco force-pushed the issue-47184 branch from 8ef1375 to 03481f1 Compare March 23, 2018 11:34

Fixed issues with incremental tests.

3a0162b

davidtwco changed the title ~~WIP: NLL should identify and respect the lifetime annotations that the user wrote~~ NLL should identify and respect the lifetime annotations that the user wrote Mar 23, 2018

nikomatsakis approved these changes Mar 23, 2018

View reviewed changes

nikomatsakis requested changes Mar 23, 2018

View reviewed changes

Improved comments for UserAssertTy statement.

4161ae7

davidtwco force-pushed the issue-47184 branch from 896eedf to 4161ae7 Compare March 23, 2018 17:50

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 23, 2018

nikomatsakis approved these changes Mar 23, 2018

View reviewed changes

alexcrichton mentioned this pull request Mar 23, 2018

Rollup of 20 pull requests #49317

Closed

bors added a commit that referenced this pull request Mar 24, 2018

Auto merge of #48482 - davidtwco:issue-47184, r=nikomatsakis

ab0ef14

NLL should identify and respect the lifetime annotations that the user wrote Part of #47184. r? @nikomatsakis

bors merged commit 4161ae7 into rust-lang:master Mar 24, 2018

davidtwco deleted the issue-47184 branch March 24, 2018 12:57

NLL should identify and respect the lifetime annotations that the user wrote #48482

NLL should identify and respect the lifetime annotations that the user wrote #48482

Conversation

davidtwco commented Feb 23, 2018 • edited Loading

davidtwco commented Feb 23, 2018

nikomatsakis commented Feb 23, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidtwco commented Mar 2, 2018

bors commented Mar 5, 2018

bors commented Mar 8, 2018

nikomatsakis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidtwco commented Mar 10, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidtwco commented Mar 13, 2018 • edited Loading

davidtwco commented Mar 13, 2018 • edited Loading

nikomatsakis commented Mar 13, 2018

nikomatsakis commented Mar 13, 2018

nikomatsakis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nikomatsakis commented Mar 23, 2018

bors commented Mar 23, 2018

bors commented Mar 24, 2018

bors commented Mar 24, 2018

davidtwco commented Feb 23, 2018 •

edited

Loading

davidtwco commented Mar 10, 2018 •

edited

Loading

davidtwco commented Mar 13, 2018 •

edited

Loading

davidtwco commented Mar 13, 2018 •

edited

Loading