Forbid wildcard dependencies on crates.io #1241
Conversation
|
👍 I warn people not to do so with hyper, but it still happens, and I break On Thu, Aug 6, 2015, 10:12 PM Steven Fackler [email protected]
|
|
An alternative is to continue allowing wildcard bounds, but let maintainers tighten the bounds post-hoc if any breakage comes up. Another thing to consider is an Both solutions are used by the Haskell community, which has grappled with dependency issues for quite some time. (EDIT: I'm not for or against this proposal; just putting these ideas out there.) |
|
Definitely +10000 for post-hoc changing of the constraints. I'd also love it for users to suggest changes to the constraints, and a flag for cargo to go by the user consensus instead of the official constraints (consider abandoned libraries) This is still useful if we decide its better to start conservative and then relax constraints. |
|
Changing dependency constraints for published versions of crates seems like it would wreak havoc on build reproducibility. We could perhaps add a fourth "level" that could only be incremented on the crates.io side, so version 3.2.4's dependencies could be corrected in version 3.2.4.1. |
|
Well definitely log changes. I suppose one depend on a specific subsets of that "fourth level" but I'd wait to see if reproducibility is a problem in practice. I guess my initial impression is that the change is supposed to be unobservable, but I haven't thought that through. |
|
Hmm, I think I might have been thinking about that wrong and reproducibility wouldn't be an issue. |
nrc
added
T-libs-api
|
Crazy idea: disallow wildcard dependencies, but allow lower bounds. In that case, when a dependency is updated to a new major version, automatically build and test with the updated dependency. If the build, or any of the tests fail, automatically insert the correct upper bound. Realistically, that's all the crate maintainer is going to do anyway, so may as well automate it! Combined with the ability for maintainers to modify dependency bounds after publishing in response to bug reports, this would provide very precise and fine-grained control of dependency bounds, while minimizing the burden on maintainers of responding quickly and publishing new versions when one of their dependencies is updated. |
|
I'm in favor of this RFC, the motivation is quite right in that not having an upper bound on a version constraint is basically guaranteed to break at some point in the future.
I agree with @sfackler that this is not a great idea for reproducibility. A lock file indicates a solution to the dependency graph provided by the set of It could in theory be possible to track revisions and then you lock to a particular index file at a particular point in time, but that seems much more complicated than just rejecting
Unfortunately I don't think this can be done in a bulletproof fashion. Your dependencies can become part of your own public API if you just export or return one of the types, so an upgrade across the major version of a dependency may not break you but could break your own downstream clients. It generally seems like the best practice is no matter what still "have an upper bound". If you really want to keep the current behavior then dependencies can look like: foo = ">= 0.0, <= 10000"(or something like that) |
|
I wish to solve this too, the way I imagine a few crates actually want the convenience of |
I was specifically interested in avoiding the time pressure: when an upstream crate upgrades to a new major version, it basically has a domino effect, where all downstream crates have to update as quickly as possible, as soon as all their dependencies are updated to use the new version. Invariably, there's one crate whose maintainer is on holiday or busy or whatever, which results in the entire ecosystem being held back. On the other hand, allowing upper bounds to get ahead of the actual versions of crates seems to defeat the point of removing wildcard support: if you can continue to use "<= 1000" then for all intents and purposes, it's a wildcard bound. While automated upgrades as I described would only be as good as the tests, there's nothing to say that their maintainers do more than just rerun their test suite with the new version anyway. Another option would be to allow upper bounds to only go say one or two major versions ahead of the actual published crate. With a little help from cargo, that could be quite powerful (eg. an easy way for the maintainer to switch from using the latest published version of a crate, to one tagged 'beta' in the associated git repo, would allow them to also test one version ahead, and allow everyone to update their crates in parallel ready for when one of their upstream crates has a new major version) |
|
While I'm mostly for this RFC, I'm concerned about the point @bluss brings up that there may be some legitimate uses for wildcard dependencies. Maybe there could be some opt-in way to allow wildcard dependencies, which would discourage the majority of people from enabling it unless it was really needed? Or maybe if this is something only unstable projects need, we could allow this for projects with a major revision number of zero (I don't know if this actually means unstable in semver, but it's how I use it)? |
|
Super unstable projects are not really served well by crates.io's current setup - you end up with tons of versions that live in S3 forever wasting space. Maven has |
|
Could "super unstable projects" just be depended on via a git dependency? [dependencies.super_unstable]
git = "https://githuib.com/super/unstable" |
|
Yeah, that is definitely the other alternative, and now that I think about it, seems like a better method than SNAPSHOT publishes given that Cargo has built in git support. |
|
I'm not really a fan of this. Most of the time, my code is fine with newer versions of other crates. Forbidding wildcards just means that I'll try to work around it (by trying to set very high max versions) or just have to spend more time needlessly bumping versions in my Cargo.toml files, pushing new versions to git, and publishing a new version on crates.io. We'll be trading an occasional need to fix things with an ongoing desire to bump version numbers of dependencies. Without some type of automation to handle updating Cargo.toml files like we have for Cargo.lock files, I think this is definitely a bad idea. Even with that automation, I'm concerned this will result in more things than necessary being linked into projects (more than 1 version of a crate) even in cases where 1 version of a crate would work. |
|
"Most of the time" isn't really good enough if your constraint choices end up breaking random downstream users. |
|
@jmesmon very likely scenario: your crate Worse still, what if you no longer care about crate Compare to you depending on |
|
I just realized one reason I use wildcards in a lot of my projects is I don't want to bother looking up the version number of the project. I often know the name of the crate, but don't know the version off the top of my head. I'd be much more willing to ditch wildcard dependencies if I could ask Cargo to add a dependency, and Cargo would add a reasonable default version (maybe only the major and minor version numbers, i.e. 1.0) or maybe the default could be configured. There are some package managers that have this functionality (i.e. |
|
As convenient as it would be, I think it is hypocritical to allow "git dependencies" but not wildcards. Both offer the same impossible promise that some library will never make a breaking change to it's interface. If we need one, I suggest we allow both, but banish package (releases) that use them to some "unsafe sublisting" of packages. @alexcrichton Revising the bounds is done because it is impossible to get the bounds both sound and complete---because upstream revisions that haven't been made yet. I am OK with shrinking the bound to make it sound being a patch version bump---since one can rely on bugs in the code, one can rely on bugs produced by an unsound bound that nevertheless results in a successful build. But extending the bound (in attempt to make it complete) is always a harmless change---even when the bound is extended unsoundly---since lock files ensure reproducible builds. This reminds me, does cargo attempt to avoid earlier patch versions? Since the only reason to publish a new patch version is a performance increase or bug fix, it seems between foolish and dangerous to accept earlier patch versions---don't want to get downstream "accustomed" to an upstream bug, especially one that has been fixed. By contrast, a bump in the minor version may just indicate an extended interface, in which case there is there no harm whatsoever in using the older version. |
|
@gsingh93 Cargo will still support "*", so you can use that during initial development. Then just write in the versions you used before publishing? |
|
@Ericson2314 git dependencies are already forbidden on crates.io. Nothing in here proposes to change that. I tentatively think that loosening constraints after publish would be a reasonable thing to do, but that seems out of scope for this RFC. |
|
|
||
| * `*` | ||
| * `> 0.3` | ||
| * `>= 0.3` |
There was a problem hiding this comment.
I assume 0.3.* will still be ok (it is in common use). What about 0.*?
There was a problem hiding this comment.
Yeah, both of those would be fine (I didn't even realize they existed!)
There was a problem hiding this comment.
0.* is a poor choice, but I'm not sure if it's worth explicitly prohibiting. I think we may want to wait and see.
|
@seanmonstar @sfackler Isn't using/shipping a Cargo.lock file supposed to prevent some of the issues you're concerned about? Right now downstream crates can avoid bumping the versions by controlling their Cargo.lock file a bit more directly. Perhaps an alternate that doesn't increase the burden of maintaining crates would be to provide tools that allow more fine grained control of Cargo.lock? |
|
In the current version the build can break if someone uses In the proposed version, the upgrade of a library can be blocked by any crate in your dependency tree. This is also bad and cannot be worked around currently, so I don't think this RFC should be accepted until it addresses this, currently it just trades one issue for another, where the former has a work-around and the latter does not. |
| pick dependencies that have reasonable version restrictions, there could be a | ||
| wildcard constraint hiding five transitive levels down. Manually searching the | ||
| entire dependency graph is an exercise in frustration that shouldn't be | ||
| necessary. |
There was a problem hiding this comment.
Bad version restrictions are also viral the other way around: There can always be a unneccessarily strict version constraint hiding "five transitive levels down". Manually searching the entire dependency graph is not a solution to this, as you'd have to publish your own versions of all the crates that recursively depend on this overly strict crate if the author does not update the version requirements by themselves – which means that the alternative this RFC is suggesting also carries this risk.
There was a problem hiding this comment.
NB. this exact point is made above #1241 (comment) .
|
@tbu- I'm not sure how this RFC trades one issue for another. Even today, if a crate has an overly strict version bound, then you'll have the problem you described. This RFC does nothing to change that. |
|
@BurntSushi Well, it pushes people away from the |
|
@tbu- I don't really think I mean, |
|
@daboross It's a work around in the way that that root node of the dependency tree can decide about all library's version, including library's dependencies. But as you say, all libraries would have to rely on the binary crate actually using them maintains a proper |
|
The current situation is not great, but I see @tbu-'s point in that sloppy versions are a way to work around the upgrade churn issue. |
|
cc @wycats |
Also update to a warning-first rollout
|
Updated to only block wildcard deps, and roll out with a warning only for one release cycle. |
|
This libs team discussed this RFC today, and decided that we were in favor but in light of the recent change by @sfackler we're going to re-up this for another week-long FCP. |
|
I appreciate the change, thank you! |
|
I have no problems with this proposal now that it has been changed to target only wildcards. |
| downloads every day. 50% of the [libraries that depend on it](https://crates.io/crates/openssl/reverse_dependencies) | ||
| have a wildcard constraint on the version. None of them can build against every | ||
| version that has ever been released. Indeed, no libraries can since many of | ||
| those releases can before Rust 1.0 released. In addition, almost all of them |
There was a problem hiding this comment.
I think came is meant here instead of can twice
|
I've been thinking about this a lot, and I have a few thoughts. I'm in favor of forbidding wildcard dependencies, as the current draft of the RFC proposes. Version ranges make me uneasy, because they make it much more likely that a resolved dependency graph will still fail to compile, which is something I would like to eventually rule out altogether. I am not suggesting any changes to the current draft of this RFC related to version ranges, but I'll explore this topic further. Shared TypesThe core tension here is about shared types. To give a concrete example, let's say that // format-time
extern crate time;
pub fn format_time<T: time::Time>(time: T) -> String {
// format the time
}// enriched-time
extern crate time;
pub struct EnrichedTime { /* ... */ }
impl time::Time for EnrichedTime { /* ... */ }If a parent crate depends on both Cargo's Coarse-Grained SolutionToday's Cargo attempts to address this issue coarsely: First, Cargo will automatically coalesce dependencies within the same major version into a single shared dependency across the entire graph. This means that if Second, Cargo assumes compliance with semantic versioning to make this work. This assumption is fundamental: if Cargo was not allowed to assume that Third, depending on a range of versions ( If you are sharing those types in your public interface, or passing instances of those types to other dependencies, this creates a "matrix" problem: the only way to be confident that your crate will compile for all downstream crates is to build your crate will all combinations of upstream versions. Shared vs. Internal DependenciesOne of the weaknesses of the current model is that it doesn't distinguish between dependencies used purely internally, as part of your implementation and dependencies exposed as part of your public interface. Internal-only dependencies have a much higher-tolerance for using version ranges to select a subset of a crate's types that are compatible across major versions, because the code with the internal-only dependency knows exactly how the type is being used. For example, even if a particular method on In contrast, dependencies exposed as part of your public API by definition do not know all of the ways those types are being used by downstream crates. As a result, it can only use version ranges if all of the types exposed in their public interface are compatible across those versions. The Role of Small Crates and Semantic VersioningInstead of trying to teach all crate authors the above rules, which are devilishly hard to get right, Cargo has so far relied on relatively small crates and semantic versioning to allow crate authors to describe compatibility for a small cluster of types. Instead of asking The fact that people want sometimes to express their dependencies as ranges implies two things to me:
A ProposalI think we can address this problem with a small improvement: Add a new Version Ranges Are ProblematicThat said, I think that depending on version ranges for crates that you use as part of your public interface is a smell. It assumes that there is a subset of a package's types and functions that are stable across major versions. For example, if the If the In this case, it behooves the author of the I see dependency ranges as an interim workaround, rather than a long-term solution. Matrix BuildsWe can reduce the risk of dependency hell caused by version ranges through However, that can only work if dependency ranges are used as a last resort, and crate authors are encouraged to break out more stable interfaces from less stable implementation. The simple advice amounts to:
This doesn't mean that everyone needs to preemptively create interface crates just in case. It just means that once a crate discovers that downstream crates are often depending on version ranges, the "right thing to do" is to break out the more stable interfaces from the less stable implementation. (if you're building a crate you intend to be widely shared, maybe you should do it preemptively.) |
|
The library subteam discussed this RFC yesterday and the conclusion was to accept it, thanks again @sfackler! |
A `*` version constraint makes it hard/impossible to release a breaking change (e.g. clap 2.0) without causing people's code to break. `cargo update` with a `*` dependency will happily move from (say) 1.4 to 2.0, even though semver says these may be a breaking change. Using a more precise dependency constraint gives the crate author more flexibility in making breaking changes, and makes users of the crate happier if/when they do happen. rust-lang/rfcs#1241 contains a bit more discussion about this.
A `*` version constraint makes it hard/impossible to release a breaking change (e.g. clap 2.0) without causing people's code to break. `cargo update` with a `*` dependency will happily move from (say) 1.4 to 2.0, even though semver says these may be a breaking change. Using a more precise dependency constraint gives the crate author more flexibility in making breaking changes, and makes users of the crate happier if/when they do happen. rust-lang/rfcs#1241 contains a bit more discussion about this.
Avoid suggesting star dependencies. A `*` version constraint makes it hard/impossible to release a breaking change (e.g. clap 2.0) without causing people's code to break. `cargo update` with a `*` dependency will happily move from (say) 1.4 to 2.0, even though semver says these may be a breaking change. Using a more precise dependency constraint gives the crate author more flexibility in making breaking changes, and makes users of the crate happier if/when they do happen. rust-lang/rfcs#1241 contains a bit more discussion about this.
Centril
added
A-versioning
Rendered