Run task under multiple enviroments

[nichmor]

Problem description

Using polarify example:

[feature.py39.dependencies]
python = "3.9.*"
[feature.py310.dependencies]
python = "3.10.*"
[feature.pl017.dependencies]
polars = "0.17.*"
[feature.pl018.dependencies]
polars = "0.18.*"

and

[environments]
pl017 = ["pl017", "py39", "test"]
pl018 = ["pl018", "py310", "test"]

it would be very useful to define task to run under list of environments automically

[tasks]
run_tests =  { "pytest", environments = ["pl017", "pl018" ] }

so in this case in will run pytest in every enviroment.
This potentially could be useful to use for cross-testing.

[baszalmstra]

Mm interesting!

One problem I see is that currently tasks are tied to features, tasks dont have any relation with environments as in your example. I think that could be quite confusing. Im wondering what an alternative syntax would be.

E.g. this could be confusing:

[feature.pl019.task]
run-test = { cmd = "pytest", environments = ["pl017", "pl018"]

[nichmor]

Mm interesting!

One problem I see is that currently tasks are tied to features, tasks dont have any relation with environments as in your example. I think that could be quite confusing. Im wondering what an alternative syntax would be.

E.g. this could be confusing:

[feature.pl019.task]
run-test = { cmd = "pytest", environments = ["pl017", "pl018"]

good point! Could we extend and apply tasks for environments tables?

[environments.tasks]
run_test =  { "pytest", environments = ["pl017", "pl018" ] }

I think that usually you will want to run tasks under different environments rather than features ( because you can combine features in one environment )

[renan-r-santos]

Chiming in to add another example that could possibly benefit from this. I couldn't find a more concise way to express my test matrix in pixi.toml.
Note that I need pixi run test to run tests against an editable install in a local/default environment but in CI I need to build a wheel, install it and then test it, somewhat similar to what nox and nox-poetry do.

[nichmor]

Chiming in to add another example that could possibly benefit from this. I couldn't find a more concise way to express my test matrix in pixi.toml. Note that I need pixi run test to run tests against an editable install in a local/default environment but in CI I need to build a wheel, install it and then test it, somewhat similar to what nox and nox-poetry do.

thanks for sharing this example!
It's true, maybe we can somehow help with more easy-way of handling this kind of matrix:
https://github.com/renan-r-santos/pixi-kernel/blob/75c4ad15c4d2dea58f08c49d255e6ea6dd66278d/.github/workflows/ci.yml#L21

so you don't need to maintain them manually

[bollwyvl]

I think the task graph depends_on could do a lot of the heavy lifting, basically overloading the jarring interactive prompt if --environment is omitted. Consider a web-like thing:

[tasks.test-all]
depends_on = [
  { task = "test-unit", environment = ["test-oldest", "test-newest"] },
  { task = "test-acceptance", environment = ["test-oldest", "test-newest"] },
  { task = "test-acceptance", environment = "test-newest", env = { JS_COVERAGE="1" } },
]

[tasks.test-unit]
cmd = "pytest tests"
depends_on = [
  { task = "preflight", environment = "$PIXI_ENVIRONMENT_NAME" },
]

[tasks.test-acceptance]
cmd = "robot atest"
depends_on = [
  { task = "preflight", environment = "$PIXI_ENVIRONMENT_NAME" },
  { task = "build-js", env = { JS_COVERAGE = "$JS_COVERAGE" } },
]

(with some inputs and outputs sprinkled in to make that not... awful)

[pmlandwehr]

Might be more trivial (or even already possible, but not obviously documented), it would be nice to specify that a task belongs even to just a single environment, e.g. for running pytest within a "test" environment, etc.

[nichmor]

Might be more trivial (or even already possible, but not obviously documented), it would be nice to specify that a task belongs even to just a single environment, e.g. for running pytest within a "test" environment, etc.

hey!
you can run a specify a environment for task using --environment flag.

Or you have other usecase ?

[ruben-arts]

What about extending the idea from @bollwyvl with a simple syntax extension for the taskname with an environment prefix?

[task]
test = "pytest"
all-tests = {depends_on = ["py39:test", "py310:test"]}
local-test = {cmd = "echo TESTING", depends_on = ["py39:test"]}

[dependencies]
pytest = "*"

[feature.py310.dependencies]
python = "3.10"

[feature.py39.dependencies]
python = "3.9"

[environments]
py39 = ["py39"]
py310 = ["py310"]

The rule being, split on : where it is {environment}:{task_name}

When the --environment is used it would overwrite the test it self but the defined environments say. So in the local-test example it would echo TESTING in the --environment specified environment but the test in py39

[bollwyvl]

Ha, gotta be careful with the grammar: there are only so many good separators, as suggested on #1285 for a per-file "namespace".

[bollwyvl]

Given no movement here, I have started down the dark path of pixi-in-pixi:

[tasks.test-all]
cmd = """
     pixi r -e test         test  -m unit
&&   pixi r -e test-oldest  test  -m unit
&&   pixi r -e test         test  -m integration
&&   pixi r -e test-oldest  test  -m integration
"""

This feels far worse than an easy-to-reason-about (and therefore visualize, document) depends-on graph, but i basically never want the interactive prompt, and typing --environment (or more) also feels bad. Further, this seems like it would never work with a future pixi run -j8, which is what I actually want in this case.

More thinking on typography: reusing the pixi run CLI grammar wouldn't feel quite as bad, if pixi can pull that meaning back out in pixi task list --json (#1344).

[tasks]
test-all = {depends-on=[
  "-e test         test -m unit", 
  "-e test-oldest  test -m unit", 
  "-e test         test -m integration", 
  "-e test-oldest  test -m integration",
]}

TOML is real picky, though: one couldn't mix a string and an inline table in the same list... so I'm still mostly in favor of the "fat" depends-on which can build matrices with (decent, no task name lookup) completion:

[tasks]
test-all = {depends-on=[
  {task = "test", environment=["test", "test-oldest"], args=[["-m unit"], ["-m integration"]]}
]}

[hugokerstens]

Issue #1519 is related: binding a task to a specific environment.

[hugokerstens]

Allowing to specify an environment in depends-on would be a really useful feature. As a minimal starting point even specifying a single environment is a great improvement already.

Consider my current config, using pixi in pixi:

[tool.pixi.feature.test.tasks]
tests = "pytest"

[tool.pixi.tasks]
tests-minimal = "pixi run -e minimal tests"
tests-mid = "pixi run -e mid tests"
tests-latest = "pixi run -e latest tests"
tests-all = { depends-on = ["tests-minimal", "tests-mid", "tests-latest"] }

This would become:

[tool.pixi.feature.test.tasks]
tests = "pytest"

[tool.pixi.tasks]
tests-all = { depends-on = [
    { task = "tests", environment = "minimal" },
    { task = "tests", environment = "mid" },
    { task = "tests", environment = "latest" },
]}