Move Mill process working directory to sandbox

[lihaoyi]

This makes os.pwd an empty directory: out/mill-worker-*/sandbox/ for Mill's server mode, out/mill-no-server/sandbox/ otherwise

In general, code outside of tasks should use millSourcePath, while code inside of tasks should use T.dest or the PathRefs given to it by upstream tasks. Accessing project files via os.pwd can result in tasks not invalidating when files changed.

Like #3347, this does not prevent people intentionally walking the filesystem to work mischieve, but it does help mitigate "accidental" usage of os.pwd to access project files. The original os.pwd is still made available as mill.api.WorkspaceRoot.workspaceRoot outside of tasks, or T.workspace inside of tasks, for the occasional scenarios that need it. But at least this adds some partial guardrails against accidentally using os.pwd in a way that subverts Mill's expectations, and the APIs that expose the workspace root can be appropriately documented to warn people against doing the wrong thing

[lefou]

I understand the motivation, yet I find it a bit uncommon for a CLI tool to not have the current working directory the same as the directory, from where I started the process. I have to make myself comfortable with this change, I guess.

So, is this only changing the pwd of the Mill server process? What's the pwd of the client? And what's the pwd, when Mill runs in --no-server mode?

[lihaoyi]

The client pwd doesn't change, but the user can't write any code that runs in the Mill client, so the pwd of the client doesn't really matter.

For -i/--no-server, we still always spawn a sub-process for the Mill build logic (as of #3363), and thus are able to control it's PWD. The sandbox for that lives in out/mill-no-server/, since it doesn't have a out/mill-worker-*/ folder to use.

It's definitely a bit unusual, but even today people shouldn't really be reading or writing stuff out of os.pwd, since it bypasses any logic around cache invalidation, watch-and-re-run, and breaks the isolation guarantees that come with T.dest. There is a workaround of copying stuff into os.pwd when the build starts, and it can probably be made kinda-sorta robust via interp.watch or something, if people really need to grandfather in old logic e.g. in the example/thirdparty/acyclic. But using os.pwd should not be the first thing people reach for, and should be made suitably inconvenient to use

[lefou]

For -i/--no-server, we still always spawn a sub-process for the Mill build logic (as of #3363), and thus are able to control it's PWD. The sandbox for that lives in out/mill-no-server/, since it doesn't have a out/mill-worker-*/ folder to use.

We might need to add something unique to that path, e.g. the process ID, since it's possible to start many mill --no-server processes in parallel.