Add ability to specify package sources that are not scanned/used by default

[jgentil]

• I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.

Feature Request

When specifying additional sources in pyproject.toml as "secondary", the current behavior of Poetry is to include this source in all of it's scans for dependency resolution for adding or installing packages. This is almost entirely unnecessary in many cases, as a secondary source is normally only needed to install very specific packages and would not normally host any other packages from pypi (and if they do, it's likely not "secondary" then but "default".)

Since this behavior is already defined, I'd like to propose a third option, targeted which specifies that a source should be used only if a dependency definition explicitly specifies it in it's source parameter. That package's dependencies would be scanned from that repository, but it would be otherwise ignored.

One specific reason that this is desperately needed is that in several of my corporate projects, we use Azure DevOps artifacts system private PyPI. This system is notoriously slow and a small project's dependency resolution scan take upwards of 10 minutes with Poetry scanning it for every single project dependency. To mitigate this, we currently force source="pypi" in all dependency specifications to target it specifically.

[zevisert]

+1 for this!

Adding fuel to the fire, I can view the access log for my private package repository -- and poetry always scans [{tool.poetry.source = { default=false, secondary=true }] sources for all transitive dependencies, even if we use your source="pypi" mitigation.

I tested with a project that has no private dependencies, and by adding a new source, but without changing dependencies at all the poetry update time went from 0.32 to over 200 seconds!

[zevisert]

With transitive dependencies in mind, the only way I can think to address this is with your suggestion for a targeted or alike key in the tool.poetry.source table.

[zevisert]

I'd really love to move this feature forward! Every time I need to update packages at work I have to sit there watching poetry 1.1.11 take 300+ seconds in most of our projects to resolve dependencies is annoying when I think I have a PR I have can do it in less than 15..

I just heard about @python-poetry/triage Hi! 👋

[IWillPull]

I'm also experiencing it and having builds take 30+ minutes longer when using one secondary dependency sounds more like a bug, not a feature request.

[dimbleby]

this should be absorbed into #6713

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.