Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Binding tasks to environments directly #1519

Open
hugokerstens opened this issue Jun 17, 2024 · 3 comments
Open

Binding tasks to environments directly #1519

hugokerstens opened this issue Jun 17, 2024 · 3 comments
Labels
✨ enhancement Feature request

Comments

@hugokerstens
Copy link

hugokerstens commented Jun 17, 2024

Problem description

From the docs I understand that you can configure a task to be bound to a specific feature. As far as I understand, binding a task to an environment is not possible. However, when running an (unbounded) task from the CLI it's the other way around: you of course have to specify/select the environment to run the task in, not the feature. This behavior is unintuitive to me.

Is it by design that tasks are not able to be bound to environments? Or is this something that is not implemented yet?

Example use case

Consider the following use case, where a task requires dependencies from multiple features.

I have a python package that includes a documentation with several scripts to generate some example plots using matplotlib (which is not a dependency of the python package). Locally, I use jupyter to develop the scripts. In CI, I use mkdocs to execute the scripts and build the documentation. My pixi.toml contains the following:

[feature.analysis.dependencies]
matplotlib-base = ">=3.8"

[feature.local.dependencies]
jupyter = ">=1.0"

[feature.docs.dependencies]
mkdocs = ">=1.6"

[environments]
local = { features = ["local", "analysis"] }
docs = { features = ["docs", "analysis"] }

[tasks]
# I want to bind this to the environment docs, not the feature docs,
# as the command requires both the analysis and docs feature.
build_docs = "mkdocs build"

The task build_docs belongs to the docs environment, but there's no way to define it that way. Binding it to the feature docs is not correct, as the command requires dependencies of multiple features.

I am interested to hear your thoughts on this!

@hugokerstens hugokerstens added the ✨ enhancement Feature request label Jun 17, 2024
@vigneshmanick
Copy link
Contributor

if tasks that depend on each other are defined, then the plots from analysis will be executed before the mkdocs command is run.

In the example below based on your config, i have tried to create plots from the analysis feature which is then executed before the build_docs command is run.

Output

> pixi run -e docs build_docs
✨ Pixi task (plots in docs): python data/plot.py

✨ Pixi task (build_docs in docs): mkdocs build

Example

[dependencies]
python = ">=3.12.4,<3.13"

[feature.analysis.dependencies]
matplotlib-base = ">=3.8"

[feature.analysis.tasks]
plots = "python data/plot.py"

[feature.local.dependencies]
jupyter = ">=1.0"

[feature.docs.dependencies]
mkdocs = ">=1.6"


[feature.docs.tasks]
build_docs = {cmd="mkdocs build", depends-on=["plots"]}


[environments]
local = { features = ["local", "analysis"] , solve-group="default"}
docs = { features = ["docs", "analysis"] , solve-group="default"}

@hugokerstens
Copy link
Author

Thanks for your reply. This approach is only feasible when the commands can be separated, but this is not always the case. In my example, mkdocs build executes the scripts internally.

My question can be summarized as: how do you handle when a single task requires dependencies from different features?

I think of it this way. When a feature-bound task is run, pixi will find an environment that includes that feature and execute the task in there. Why not allow to bind to a specific environment directly?

@hugokerstens hugokerstens changed the title Binding tasks to environments or features Binding tasks to environments directly Jul 1, 2024
@akhmerov
Copy link

akhmerov commented Jul 1, 2024

I have another case that doesn't seem to quite map on the set of concepts implemented in pixi right now.

I have a package with basic and optional dependencies. These seem to belong to different features, and packaged used for testing into a yet another feature. I would then like to make test_comprehensive task which requires 3 features: basic_dependencies, optional_dependencies, and testing.

The question I fail to answer is: within which feature should I define test_comprehensive? It seems to be equally wrong to put it in any of the 3 features.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✨ enhancement Feature request
Projects
None yet
Development

No branches or pull requests

3 participants