Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding pixi export command #800

Closed
guru-desh opened this issue Feb 11, 2024 · 10 comments
Closed

Adding pixi export command #800

guru-desh opened this issue Feb 11, 2024 · 10 comments
Labels
🆘 help wanted Maintainers would like community input ✨ enhancement Feature request 👩‍💻 cli Issue related to CLI

Comments

@guru-desh
Copy link

Problem description

It would be super awesome if pixi had an option to export the dependencies from the pixi.toml file to a conda environment .yaml file. It would work similar to how poetry export is able to export the dependencies from the pyproject.toml file into a requirements.txt.

There are valid use cases for this, and I have one. I'm a Head Teaching Assistant for the Machine Learning course at Georgia Tech. I found pixi pretty recently since pixi is one of the best package managers for managing regular, host, and build dependencies especially for anaconda environments (which we use extensively in our course assignments). Currently, we're using conda + poetry, which requires maintaining both a conda environment and a poetry environment, which is clunky for us.

An export feature in pixi would allow us to use pixi for managing all our dependencies and allowing us to easily export the conda environment files for all the platforms we support with full confidence that students won't run into issues.

Thank you guys so much for solving a VERY known problem in the python world. I really love this work, and I'm planning on using pixi extensively in the future.

@guru-desh guru-desh added the ✨ enhancement Feature request label Feb 11, 2024
@liquidcarbon
Copy link
Contributor

Hi @guru-desh agreed that this feature would be useful.
There's a hacky option to produce requirements.txt in #518 if you want to give it a try.

@ruben-arts
Copy link
Contributor

We're currently working on the opposite function where you can import an env.yml to a pixi project. I;m not sure if we would also support the export but we are open to contributions!

The translation to requirements.txt conda-lock pyproject.toml should all be relatively easy. They can all be constructed from the information in the pixi.lock. We're even working on a python library to read them: conda/rattler#510.

@ruben-arts ruben-arts added needs-decision Undecided if this should be done 👩‍💻 cli Issue related to CLI labels Feb 29, 2024
@baszalmstra
Copy link
Contributor

We would be open to adding this to pixi but won't have the capacity to pick this up anytime soon. We welcome PRs!

@baszalmstra baszalmstra added 🆘 help wanted Maintainers would like community input and removed needs-decision Undecided if this should be done labels Mar 4, 2024
abkfenris added a commit to abkfenris/pixi that referenced this issue May 22, 2024
Adds an export command group, and a subcommand for exporting in Conda environment.yml files.

Currently it just exports the direct dependency names, but I could see adding flags to include all locked dependencies. It also currently does not export the versions, but that could also be a flag to select if it should export the specs as in the manifest, or as locked.

works on prefix-dev#800
@akhmerov
Copy link

I find this feature especially important for backwards compatibility reasons: our cluster is running RHEL 8, and pixi requires newer kernel versions. My intended workaround would be to export data from pixi and use something like a conda lock.

@synapticarbors
Copy link
Contributor

@akhmerov -- You might find the following which I wrote as a prototype useful: https://github.com/synapticarbors/pixi2ces

I'm hoping to contribute some version of it part to pixi as part of a pixi export command.

@liquidcarbon
Copy link
Contributor

I find this feature especially important for backwards compatibility reasons: our cluster is running RHEL 8, and pixi requires newer kernel versions. My intended workaround would be to export data from pixi and use something like a conda lock.

@akhmerov I'm on Oracle Linux Server 8 and this satisfies Pixi

[tool.pixi.system-requirements]
linux = "5.4.17"

is it different in RHEL?

@akhmerov
Copy link

@synapticarbors thank you for the pointer!

@liquidcarbon 🤯 indeed, overriding system requirements seems to work. I wonder, though, whether something would explode later down the line. Where can I find more information?

@liquidcarbon
Copy link
Contributor

My main prod stuff is still under poetry, so can't say with full confidence it'll be fine, but I don't see why not

@akhmerov
Copy link

Ah, it was my misunderstanding of the error message. It's not that pixi itself requires recent kernel and libc, but rather that it has default virtual package requirements for those. In that case my remark should be disregarded because exporting to conda environment, wouldn't change the virtual package requirements. Apologies for the noise.

@ruben-arts
Copy link
Contributor

The pixi project export command is coming in the next release. And more formats are being added as we speak. Feel free to add more formats like @abkfenris is doing in: #2003 for environment.yml

Closing this issue as it can now be implemented.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🆘 help wanted Maintainers would like community input ✨ enhancement Feature request 👩‍💻 cli Issue related to CLI
Projects
None yet
Development

No branches or pull requests

6 participants