tox.ini: Add wsl as a technology, to complement local and docker

[mkoeppe]

This is for portability testing of the Sage distribution when developing on Windows, as an alternative to using Docker.

wsl would be a "technology", like local or docker -
https://doc.sagemath.org/html/en/developer/portability_testing.html#automatic-docker-based-build-testing-using-tox

The GH workflow for wsl testing from #30216 could then be simplified and generalized by going through tox.

(follow-up from #30216)

---

References:

Manual install: https://docs.microsoft.com/en-us/windows/wsl/install-manual

Windows Subsystem for Linux Command Reference | Microsoft Docs
https://docs.microsoft.com/en-us/windows/wsl/reference

windows subsystem for linux - How do you install multiple, separate instances of Ubuntu in WSL? - Stack Overflow
https://stackoverflow.com/questions/51584765/how-do-you-install-multiple-separate-instances-of-ubuntu-in-wsl

DDoSolitary/LxRunOffline: A full-featured utility for managing Windows Subsystem for Linux (WSL)
https://github.com/DDoSolitary/LxRunOffline

Manually download Windows Subsystem for Linux (WSL) Distros | Microsoft Docs
https://docs.microsoft.com/en-us/windows/wsl/install-manual

Install Windows Subsystem for Linux (WSL) on Windows 10 | Microsoft Docs
https://docs.microsoft.com/en-us/windows/wsl/install-win10

WSL - Ubuntu Wiki
https://wiki.ubuntu.com/WSL

Depends on #30923

CC: @tobiasdiez @sagetrac-tmonteil

Component: porting

Issue created by migration from https://trac.sagemath.org/ticket/30505

[mkoeppe]

comment:1

gh-tobiasdiez at #97:
#97:

mkoeppe wrote:

In the long run (on some follow up ticket), I think it would actually be good to push wsl provisioning down one level, into tox.ini, by defining environments such as wsl-ubuntu-focal-standard (in analogy to docker-ubuntu-focal-standard). So that on a Windows box, you would be able to just invoke tox to create an isolated wsl container that is only used for testing.

I'm not sure how helpful it is to push it to the tox.ini. For me as a Windows user, WSL is the most convenient way to run sage. But for this I have a central WSL which I setup once and then reuse. Moreover, VS Code can connect to this WSL instance and run all commands trough it as needed. Thus, the workflow is different than say with docker containers.

This ticket is for a different purpose, namely portability testing - where you run Sage against various Linux distributions (and configurations, i.e., what system packages are installed). We do this with Docker, but on a Windows development box it may be more convenient to do it with isolated WSL instances instead.

[mkoeppe]

comment:5

Tobias, can we try to get this ticket going? Basically I want to add to the top-level tox.ini a way to create a new (isolated) WSL instance (with several options for the Linux distribution to use).

[tobiasdiez]

comment:6

What exactly do you want to accomplish? Do you want to add a tox command that downloads a given ubuntu distro, installs it as a wsl, installs sage including all dependencies and then run all tests in this wsl?

If that's the case, why do you want to manage the wsl installation from within tox? For integration tests, it would be easier to install wsl with a given linux distro in the github action and then run tox relative to this wsl. For development, it is also easier to let the user setup the initial wsl and reuse it.

(By the way, the same also applies to docker images where you can easily run the github action jobs in a docker container and don't need to manage the container from tox, https://www.petefreitag.com/item/903.cfm. This should also somewhat speedup the workflow as github caches the container.)

[mkoeppe]

comment:7

Replying to @tobiasdiez:

What exactly do you want to accomplish? Do you want to add a tox command that downloads a given ubuntu distro, installs it as a wsl, installs sage including all dependencies and then run all tests in this wsl?

If that's the case, why do you want to manage the wsl installation from within tox?

For local testing and debugging of portability issues. GH Actions only goes so far, at some point one needs to get an actual local installation.

For development, it is also easier to let the user setup the initial wsl and reuse it.

No, the point is that the tox command will automate this step so that one has a reproducible way to set up a (disposable) wsl copy. In other words, there is not "the" initial wsl but perhaps a dozen wsls with different configurations. By default, a wsl would be reused but the developer would be able to set it up from scratch using tox -r.

It's the same automation win that the existing docker-... tox environments give us: Of course I can set up a Docker container, install packages with apt-get, and then run my tests of the Sage build in it. But with the tox this just becomes one command, tox -e docker-ubuntu-bionic-standard.

[mkoeppe]

comment:8

One question that I am unsure about: Where would tox come from in a typical Windows developer's system, and what shell would be used to execute the tox commands?

[tobiasdiez]

comment:9

Thanks for your explanation. But I still don't really get why you need/want to manage the wsl installation via tox. If tox is able to configure a normal linux install, then it's only one additional step for the user to create a wsl installation with the said linux.

Moreover, typical WSL installs are of the order of 2 to 5gb (mine is 22gb, which is used exclusively for sage). That's not something you would use in a use-once-then-throw-away spirit, like docker images.

But, of course, you could move the current code in the github wsl workflow to a new tox environment. As a first step towards this, and what I would find actually more important, is to have local-ubuntu environments similar to local-homebrew.
Then you can simply install ubuntu in wsl via the Windows store, and run

wsl
> tox -e local-ubuntu-standard

That's the same amount of characters as tox -e local-wsl-ubuntu-standard ;-)

Replying to @mkoeppe:

One question that I am unsure about: Where would tox come from in a typical Windows developer's system, and what shell would be used to execute the tox commands?

You would need to have a python installed in windows, then install tox via pip. Tox would usually be invoked from the "Windows Command line" or powershell console.

[mkoeppe]

comment:11

Replying to @tobiasdiez:

of course, you could move the current code in the github wsl workflow to a new tox environment. As a first step towards this, and what I would find actually more important, is to have local-ubuntu environments similar to local-homebrew.
Then you can simply install ubuntu in wsl via the Windows store, and run

wsl
> tox -e local-ubuntu-standard

That's the same amount of characters as tox -e local-wsl-ubuntu-standard ;-)

OK, sounds good, let's do this as the first step. That's now #30923

[mkoeppe]

Dependencies: #30923

[mkoeppe]

comment:13

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.