cargo update behaves different with and without workspaces

[staktrace]

It appears that cargo update on a workspace will pull in all the optional dependencies of the members, whereas doing cargo update on a single member crate will not. This behaviour difference seems like a bug. It is also blocking use of a workspace to reduce build times in the Firefox build, because the extra optional dependencies need to get vendored into the Firefox tree even though they are never used.

To illustrate the problem, I created a little test repo with a workspace - https://github.com/staktrace/cargo-bug. The STR to reproduce are in the README file.

/cc @froydnj

[staktrace]

Sorry my STR aren't quite correct... it worked before but for some reason doesn't work now. Gimme a bit to figure out what changed.

[staktrace]

Ok, I updated the cargo-bug repo from my initial comment so that it reproduces the problem properly. It turns out there needs to be one extra "optional" layer in between. But still, the cargo update on the workspace behaves differently from cargo update on the only crate it contains, which seems like a bug.

[alexcrichton]

Thanks for the report and the reproduction! I think you're running into expected behavior, however.

Currently a workspace contains one lock file for all its members, and all that lock file contains all optional dependencies for all members. So following the instructions in your README the first situation doesn't have subproj1 as a member of the workspace, so ipc-channel is omitted. In the second scenario, however, subproj1 is a member so its optional dependency is included.

Does that make sense? Currently there's no way to explicitly remove a member from a workspace, but that's tracked by #3192

[staktrace]

Hm, ok. I guess my mental model of how a workspace is supposed to work is wrong then. We can close this. However, I'd also like to document why my mental model was wrong, so that you can take it into account in future changes to cargo workspaces (although I suspect for backwards compat we're kind of stuck with this now).

Upon reading the cargo documentation, it seems that there's two ways to include members into a workspace: one is to put it inside the subtree rooted at the workspace, and the other is to list it in the "members" list. Because these are orthogonal there are four possible combinations:

If W is the workspace and A is a crate, then:
(1) W lists A as a member, and A is in the subtree of W => A gets built as part of W
(2) W doesn't list A as a member, and A is in the subtree of W => building A produces error
(3) W lists A as a member, and A is outside the subtree of W => A can be built standalone OR as part of W
(4) W doesn't list A as a member, and A is outside the subtree of W => A is built standalone

In my mind both outcomes (2) and (3) are unexpected. I would expect a member to either be in a workspace or standalone, whereas in (2) it is neither and in (3) it is both. Since I was not expecting this behaviour, I subconsciously figured that the more "explicit" behaviour (listing A as a member of W) would take priority so that (2) would get resolved as "A is standalone" and (3) would get resolved as "A gets built as part of W". In this world, subproj1 would be standalone and not get included into the top-level workspace in my reproduction project.

Perhaps the documentation on cargo workspaces should be made more explicit to cover these four options?

[luser]

I don't think your mental model with "subtree" is quite right, per http://doc.crates.io/manifest.html#the-workspace-section

All path dependencies residing in the workspace directory become members. You can add additional packages to the workspace by listing them in the members key. Note that members of the workspaces listed explicitly will also have their path dependencies included in the workspace.

There's sort of a weird non-symmetric property here which is that you can have a workspace defined at the root of a directory tree, and it can specify members, so you could leave out some crates that are in subdirectories, but [package.workspace] defaults to "the first crate whose manifest contains [workspace] upwards in the filesystem". It's unclear to me what happpens to crates in that situation, although if you're not building them it's probably not really an issue.

[staktrace]

Ah, you're right. While doing more experimentation I found what appears to be a real bug: https://github.com/staktrace/cargo-bug-2

In this scenario we have this structure:

test-repo/ <-- workspace
    subproj1  <-- belongs to test-repo workspace
    subproj2 <-- belongs to test-repo2 workspace
test-repo2/ <-- another workspace
    subproj3 <-- belongs to test-repo workspace

Where all the Cargo.toml files are consistently pointing to their workspaces and members. But running cargo update in subproj3/ or in subproj2/ results in an error. I don't see why.

[staktrace]

@alexcrichton for my previous comment - Is my understanding wrong, or should I file an issue for that?

[alexcrichton]

@staktrace yeah that looks like it's a bug, but I wonder if that's just #3586?