Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

warn if cuda packages are installed implicitly #428

Merged
merged 6 commits into from
Jun 15, 2023

Conversation

minrk
Copy link
Contributor

@minrk minrk commented Jun 8, 2023

closes #426

doesn't change default behavior, but adds a warning if cuda is not requested but 'cudatoolkit' is pulled in, as suggested by @maresb. The presence of 'cudatoolkit' itself in the dependency list is considered sufficient for an 'explicit cuda' signal.

I could leave out that condition because the available version is still implicit, which may trip folks up.

I initially wanted to allow specifying a version with --with-cuda [version], but click doesn't seem to want folks to have options with 0 or 1 args. Plus, virtual package specs already solve this, so it's not worth the fight with click.

@minrk minrk requested a review from a team as a code owner June 8, 2023 14:50
@netlify
Copy link

netlify bot commented Jun 8, 2023

Deploy Preview for conda-lock ready!

Name Link
🔨 Latest commit cd68557
🔍 Latest deploy log https://app.netlify.com/sites/conda-lock/deploys/6486fa6dcf817300080d7301
😎 Deploy Preview https://deploy-preview-428--conda-lock.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@maresb
Copy link
Contributor

maresb commented Jun 8, 2023

Wow, this is very exciting! Too bad about the Click annoyance. What if we made the CUDA version mandatory?

Now that I have your attention, can I ask a few quick questions about CUDA support, most of which are out-of-scope for this PR?

  • Are there other potential CUDA indicators we might want to consider besides cudatoolkit?
  • We seem to have this hard-coded default 11.4 version. Should we regularly update, and if so, at what frequency and with which criteria / delay?
  • There seems to be a lot of ugly code associated with the default virtual package spec. Do you think it be feasible to instead replace most of this code with the parsing of a default-virtual-packages.yml file?

now you can't specify that you want cuda without specifying a version

Default behavior is still 11.4
@minrk
Copy link
Contributor Author

minrk commented Jun 9, 2023

mandatory version should work fine.

I'm afraid I can't really answer the cuda questions, since I'm not a cuda user. I'm mostly here for making it easier to not use cuda. In my experience, cudatoolkit is ~always present when cuda's depended upon, but I can't say for sure. It's probably a 'good enough' indicator to start with.

Re: the version bump, I think making the version string required makes it easier to leave the default implicit version alone. Then you need to specify the version if you want to avoid the warning or if you want a more recent version. The implicit cuda message can reflect that.

There seems to be a lot of ugly code associated with the default virtual package spec. Do you think it be feasible to instead replace most of this code with the parsing of a default-virtual-packages.yml file?

I'll have a look if I get a chance. Would need #429

@minrk
Copy link
Contributor Author

minrk commented Jun 9, 2023

flake8 complains about complexity. I don't really know how to respond to that other than suggesting that the complexity check should be removed. I don't think I have the authority to refactor to reduce complexity.

@maresb
Copy link
Contributor

maresb commented Jun 9, 2023

Thanks a lot for this great work! Sorry about the complexity issue. Would it work to add # noqa: C901 to the definition line of make_lock_files?

@minrk
Copy link
Contributor Author

minrk commented Jun 9, 2023

noqa comment works, thanks

@minrk minrk changed the title warn if cuda packages are installed explicitly warn if cuda packages are installed implicitly Jun 9, 2023
Copy link
Contributor

@maresb maresb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a few minor stylistic nits to help improve readability.

But really I think this needs a look from a CUDA expert, unfortunately not me. @mariusvniekerk, could you please suggest someone?

_implicit_cuda_message = """
'cudatoolkit' package added implicitly without specifying that cuda packages
should be accepted.
Add a minimum cuda version via `--with-cuda VERSION` or via virtual packages
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm not mistaken, the version is not "minimum" but represents the exact version of CUDA which is available on the (hypothetical) target system.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's the version used for the solve, but in practice cuda versions are treated as lower bounds. So setting a version in the hypothetical solve is effectively setting the minimum version for which the locked environment can be used. But I switched it to 'specify' to avoid trying to say too much.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this is good to know... I would have assumed that it's semver.

I'm still slightly confused, and I want to make sure I understand what you're writing. Would a typical pin be cuda >=11.4 or cuda <12? Thanks for your patience!

conda_lock/conda_lock.py Outdated Show resolved Hide resolved
conda_lock/conda_lock.py Outdated Show resolved Hide resolved
conda_lock/virtual_package.py Outdated Show resolved Hide resolved
conda_lock/virtual_package.py Outdated Show resolved Hide resolved
minrk and others added 3 commits June 12, 2023 10:34
in practice it is a minumum, but what matters is it's solved assuming an exact version is available
@maresb
Copy link
Contributor

maresb commented Jun 14, 2023

I think this is really good and unlikely to break anything. I'm just going to merge in a few days if there's no further feedback.

@mariusvniekerk mariusvniekerk merged commit ce4ab70 into conda:main Jun 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove cuda from default virtual packages?
3 participants