Skip to content

Commit

Permalink
distribution-types: flip the ordering of index preference
Browse files Browse the repository at this point in the history
Previously, we would prioritize `--index-url` over all
`--extra-index-url` values. But now, we prioritize all
`--extra-index-url` values over `--index-url`. That is, `--index-url`
has gone from the "primary" index to the "fallback" index. In most
setups, `--index-url` is left as its default value, which is PyPI.

The ordering of `--extra-index-url` with respect to one another remains
the same. That is, in `--extra-index-url foo --extra-index-url bar`,
`foo` will be tried before `bar`.

Finally, note that this specifically does not match `pip`'s behavior.
`pip` will attempt to look at versions of a package from all indexes in
which in occurs. `uv` will stop looking for versions of a package once
it finds it in an index. That is, for any given package, `uv` will only
utilize versions of it from a single index.

Ref #171, Fixes #1377, Fixes #1451, Fixes #1600
  • Loading branch information
BurntSushi committed Feb 29, 2024
1 parent 1c57893 commit dd82e71
Show file tree
Hide file tree
Showing 3 changed files with 124 additions and 4 deletions.
4 changes: 2 additions & 2 deletions crates/distribution-types/src/index_url.rs
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,7 @@ impl Default for IndexUrls {
}

impl<'a> IndexUrls {
/// Return the primary [`IndexUrl`] entry.
/// Return the fallback [`IndexUrl`] entry.
///
/// If `--no-index` is set, return `None`.
///
Expand Down Expand Up @@ -318,7 +318,7 @@ impl<'a> IndexUrls {
/// If `no_index` was enabled, then this always returns an empty
/// iterator.
pub fn indexes(&'a self) -> impl Iterator<Item = &'a IndexUrl> + 'a {
self.index().into_iter().chain(self.extra_index())
self.extra_index().chain(self.index())
}
}

Expand Down
60 changes: 60 additions & 0 deletions crates/uv/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -284,10 +284,25 @@ struct PipCompileArgs {
refresh_package: Vec<PackageName>,

/// The URL of the Python package index (by default: <https://pypi.org/simple>).
///
/// The index given by this flag is given lower priority than all other
/// indexes specified via the `--extra-index-url` flag.
///
/// Unlike `pip`, `uv` will stop looking for versions of a package as soon
/// as it finds it in an index. That is, it isn't possible for `uv` to
/// consider versions of the same package across multiple indexes.
#[clap(long, short, env = "UV_INDEX_URL")]
index_url: Option<IndexUrl>,

/// Extra URLs of package indexes to use, in addition to `--index-url`.
///
/// All indexes given via this flag take priority over the index
/// in `--index-url` (which defaults to PyPI). And when multiple
/// `--extra-index-url` flags are given, earlier values take priority.
///
/// Unlike `pip`, `uv` will stop looking for versions of a package as soon
/// as it finds it in an index. That is, it isn't possible for `uv` to
/// consider versions of the same package across multiple indexes.
#[clap(long, env = "UV_EXTRA_INDEX_URL")]
extra_index_url: Vec<IndexUrl>,

Expand Down Expand Up @@ -421,10 +436,25 @@ struct PipSyncArgs {
link_mode: install_wheel_rs::linker::LinkMode,

/// The URL of the Python package index (by default: <https://pypi.org/simple>).
///
/// The index given by this flag is given lower priority than all other
/// indexes specified via the `--extra-index-url` flag.
///
/// Unlike `pip`, `uv` will stop looking for versions of a package as soon
/// as it finds it in an index. That is, it isn't possible for `uv` to
/// consider versions of the same package across multiple indexes.
#[clap(long, short, env = "UV_INDEX_URL")]
index_url: Option<IndexUrl>,

/// Extra URLs of package indexes to use, in addition to `--index-url`.
///
/// All indexes given via this flag take priority over the index
/// in `--index-url` (which defaults to PyPI). And when multiple
/// `--extra-index-url` flags are given, earlier values take priority.
///
/// Unlike `pip`, `uv` will stop looking for versions of a package as soon
/// as it finds it in an index. That is, it isn't possible for `uv` to
/// consider versions of the same package across multiple indexes.
#[clap(long, env = "UV_EXTRA_INDEX_URL")]
extra_index_url: Vec<IndexUrl>,

Expand Down Expand Up @@ -619,10 +649,25 @@ struct PipInstallArgs {
output_file: Option<PathBuf>,

/// The URL of the Python package index (by default: <https://pypi.org/simple>).
///
/// The index given by this flag is given lower priority than all other
/// indexes specified via the `--extra-index-url` flag.
///
/// Unlike `pip`, `uv` will stop looking for versions of a package as soon
/// as it finds it in an index. That is, it isn't possible for `uv` to
/// consider versions of the same package across multiple indexes.
#[clap(long, short, env = "UV_INDEX_URL")]
index_url: Option<IndexUrl>,

/// Extra URLs of package indexes to use, in addition to `--index-url`.
///
/// All indexes given via this flag take priority over the index
/// in `--index-url` (which defaults to PyPI). And when multiple
/// `--extra-index-url` flags are given, earlier values take priority.
///
/// Unlike `pip`, `uv` will stop looking for versions of a package as soon
/// as it finds it in an index. That is, it isn't possible for `uv` to
/// consider versions of the same package across multiple indexes.
#[clap(long, env = "UV_EXTRA_INDEX_URL")]
extra_index_url: Vec<IndexUrl>,

Expand Down Expand Up @@ -892,10 +937,25 @@ struct VenvArgs {
prompt: Option<String>,

/// The URL of the Python package index (by default: <https://pypi.org/simple>).
///
/// The index given by this flag is given lower priority than all other
/// indexes specified via the `--extra-index-url` flag.
///
/// Unlike `pip`, `uv` will stop looking for versions of a package as soon
/// as it finds it in an index. That is, it isn't possible for `uv` to
/// consider versions of the same package across multiple indexes.
#[clap(long, short, env = "UV_INDEX_URL")]
index_url: Option<IndexUrl>,

/// Extra URLs of package indexes to use, in addition to `--index-url`.
///
/// All indexes given via this flag take priority over the index
/// in `--index-url` (which defaults to PyPI). And when multiple
/// `--extra-index-url` flags are given, earlier values take priority.
///
/// Unlike `pip`, `uv` will stop looking for versions of a package as soon
/// as it finds it in an index. That is, it isn't possible for `uv` to
/// consider versions of the same package across multiple indexes.
#[clap(long, env = "UV_EXTRA_INDEX_URL")]
extra_index_url: Vec<IndexUrl>,

Expand Down
64 changes: 62 additions & 2 deletions crates/uv/tests/pip_install.rs
Original file line number Diff line number Diff line change
Expand Up @@ -36,14 +36,24 @@ fn decode_token(content: &[&str]) -> String {

/// Create a `pip install` command with options shared across scenarios.
fn command(context: &TestContext) -> Command {
let mut command = command_without_exclude_newer(context);
command.arg("--exclude-newer").arg(EXCLUDE_NEWER);
command
}

/// Create a `pip install` command with no `--exclude-newer` option.
///
/// One should avoid using this in tests to the extent possible because
/// it can result in tests failing when the index state changes. Therefore,
/// if you use this, there should be some other kind of mitigation in place.
/// For example, pinning package versions.
fn command_without_exclude_newer(context: &TestContext) -> Command {
let mut command = Command::new(get_bin());
command
.arg("pip")
.arg("install")
.arg("--cache-dir")
.arg(context.cache_dir.path())
.arg("--exclude-newer")
.arg(EXCLUDE_NEWER)
.env("VIRTUAL_ENV", context.venv.as_os_str())
.current_dir(&context.temp_dir);
command
Expand Down Expand Up @@ -802,6 +812,56 @@ fn install_no_index_version() {
context.assert_command("import flask").failure();
}

/// Install a package via --extra-index-url.
///
/// This is a regression test where previously `uv` would consult test.pypi.org
/// first, and if the package was found there, `uv` would not look at any other
/// indexes. We fixed this by flipping the priority order of indexes so that
/// test.pypi.org becomes the fallback (in this example) and the extra indexes
/// (regular PyPI) are checked first.
///
/// (Neither approach matches `pip`'s behavior, which considers versions of
/// each package from all indexes. `uv` stops at the first index it finds a
/// package in.)
///
/// Ref: <https://github.com/astral-sh/uv/issues/1600>
#[test]
fn install_extra_index_url_has_priority() {
let context = TestContext::new("3.12");

uv_snapshot!(command_without_exclude_newer(&context)
.arg("--index-url")
.arg("https://test.pypi.org/simple")
.arg("--extra-index-url")
.arg("https://pypi.org/simple")
// This tests what we want because BOTH of the following
// are true: `black` is on pypi.org and test.pypi.org, AND
// `black==24.2.0` is on pypi.org and NOT test.pypi.org. So
// this would previously check for `black` on test.pypi.org,
// find it, but then not find a compatible version. After
// the fix, `uv` will check pypi.org first since it is given
// priority via --extra-index-url.
.arg("black==24.2.0"), @r###"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
Resolved 6 packages in [TIME]
Downloaded 6 packages in [TIME]
Installed 6 packages in [TIME]
+ black==24.2.0
+ click==8.1.7
+ mypy-extensions==1.0.0
+ packaging==23.2
+ pathspec==0.12.1
+ platformdirs==4.2.0
"###
);

context.assert_command("import flask").failure();
}

/// Install a package from a public GitHub repository
#[test]
#[cfg(feature = "git")]
Expand Down

0 comments on commit dd82e71

Please sign in to comment.