[Windows] Add get-user-info utility

[gabriel-samfira]

This utility allows us to mount the buildkitd executable inside a Windows container and fetch the SID of any existing user. This is to work around the fact that the SAM hive data structures are undocumented and there is no API to inspect an offline SAM hive to fetch the security info of an existing user.

We will use this utility when determining the SID of the user when we conduct FileOps and is part of a replacement for: #3248

Depends on: moby/moby#44847

Signed-off-by: Gabriel Adrian Samfira [email protected]

[crazy-max]

Is this supposed to be called inside a Windows container? It looks similar to: https://github.com/containerd/containerd/blob/main/integration/images/volume-ownership/tools/get_owner_windows.go. Might be worth to have a dedicated repo for this utility?

[crazy-max]

Should we use "golang.org/x/sys/windows" instead?

[gabriel-samfira]

x/sys is where new stuff gets added. The syscall package only changes when those changes need to support the standard library.

The LookupSID() function from the syscall package is identical to the one in x/sys/windows, and calls into a stable Windows API which, judging by Microsofts track record for backwards compatibility, will probably remain unchanged for many many years.

Unless we want to use an old version of go that does not have this function in the standard library, I think we can use syscall.

If you prefer, I can change it.

[crazy-max]

We are already using this pkg

buildkit/session/sshforward/sshprovider/agentprovider_windows.go

Line 13 in c3eddbf

"golang.org/x/sys/windows"

that's why I suggested to use it instead but I don't have a strict pref.

[TBBle]

My recollection of a discussion in the go-winio repo was that we were leaning towards preferring x/sys/windows over syscall where possible. I have a PR there for example which replaces all uses of syscall with x/sys/windows versions.

Which I would like to come back to at some point; it was delayed because there was a syscall.Handle in a public API being changed to windows.Handle and they wanted to batch that up some other API breaks.

[gabriel-samfira]

Then it's settled. Will replace with x/sys/windows ASAP.

[TBBle]

For the record, the PR and discussion I mentioned is microsoft/go-winio#197.

[crazy-max]

Suggested change

	sid, _, _, err := syscall.LookupSID("", username)
	sid, _, _, err := windows.LookupSID("", username)

[gabriel-samfira]

Is this supposed to be called inside a Windows container? It looks similar to: https://github.com/containerd/containerd/blob/main/integration/images/volume-ownership/tools/get_owner_windows.go. Might be worth to have a dedicated repo for this utility?

The short answer is: yes. This is meant to be called inside a Windows container, whenever buildkitd itself needs to resolve a username to a SID, and that username is not a well known account.

Slightly longer answer bellow:

This utility is meant to be used here: https://github.com/moby/buildkit/pull/3517/files#diff-8265d89f801913b3b2f4c1585304405c661d143ede3ed61ec6885d0581cc1650R96-R105

Which is made available to the container, here: https://github.com/moby/buildkit/pull/3517/files#diff-9d45df0583b268e8e8fee8d2a71f04728c1ba11519660b6702f00d3f3e0d55c5R18-R30

A more detailed explanation for the need for this, can be found here: https://github.com/moby/buildkit/pull/3517/files#diff-8265d89f801913b3b2f4c1585304405c661d143ede3ed61ec6885d0581cc1650R52-R66

This spares us from having to ship a separate binary along side buildkitd. We simply mount os.Executable() inside the container, somewhere in PATH.

The idea is to be as self contained as possible. This felt like the best course of action. A separate binary is also possible if preferred.

[crazy-max]

LGTM

PTAL @TBBle

[TBBle]

Seems short and reasonable to me, in isolation. I'm still on sabbatical so haven't gone through the rest of the (greatly appreciated) work done around this.

I'm not sure off-hand where in the process re-exec is supposed to drop privs; I'm actually not sure if that's necessary here either, it's just something I recall being noted around the re-exec system in dockerd (in the Windows graphdriver code) and also scattered across the Windows filesystem code in containerd.

I also don't have a particular risk-vector in mind.

[thaJeztah]

Is this supposed to be called inside a Windows container? It looks similar to: https://github.com/containerd/containerd/blob/main/integration/images/volume-ownership/tools/get_owner_windows.go. Might be worth to have a dedicated repo for this utility?

This similar to https://github.com/docker-archive/windows-container-utility ? (something we should consider swapping in moby ?)

[gabriel-samfira]

This similar to https://github.com/docker-archive/windows-container-utility ? (something we should consider swapping in moby ?)

This returns a marshaled idtools.Identity{}. The output is something like:

#10 [6/6] RUN get-user-info administrator
#10 1.662 {"UID":0,"GID":0,"SID":"S-1-5-21-2702878673-795188819-444038987-500"}
#10 DONE 2.1s

It could be extended to do more if needed, and spares us from shipping a separate binary.

[thaJeztah]

So there's two things that should be verified (I haven't looked closely how/where this exactly is used);

• currently this re-execs buildkitd. When we use BuildKit in moby, there won't be a buildkitd (as BuildKit is vendored, and embedded in dockerd). So if we need this functionality in moby, we'd have to look at how to do the same.
• second bit is related to the code that was moved in your PR in moby (Disable chrootarchive.init() on Windows moby#44847). (trying to describe in my simplistic view of that issue); If we re-exec in the container, and the re-exec would load a DLL, that would be from within the container. Those are loaded in memory, and can mean that running it within an untrusted container could potentially load a malicious DLL into memory, which means that any other uses of the DLL would use that.

not sure if any of that applies here, but (just in case it's relevant) 😅

[gabriel-samfira]

currently this re-execs buildkitd. When we use BuildKit in moby, there won't be a buildkitd (as BuildKit is vendored, and embedded in dockerd). So if we need this functionality in moby, we'd have to look at how to do the same.

Ahh. Interesting. But the proposed PR that uses this, mounts os.Executable() as get-user-info. So even if this gets embedded into moby, it will bind mount whatever executable embedded this code. Then get-user-info is executed, a new process is created in the context of the container, not on the host, so we get a new, separate process.

Edit: Here is where this gets mounted: https://github.com/moby/buildkit/pull/3517/files#diff-9d45df0583b268e8e8fee8d2a71f04728c1ba11519660b6702f00d3f3e0d55c5R18-R30
Later edit: Here is where it gets used: https://github.com/moby/buildkit/pull/3517/files#diff-8265d89f801913b3b2f4c1585304405c661d143ede3ed61ec6885d0581cc1650R96

• Those are loaded in memory, and can mean that running it within an untrusted container could potentially load a malicious DLL into memory, which means that any other uses of the DLL would use that.

I am not extremely familiar with the internals of how Windows containers manage memory, but I expect that the host and the container running on top don't share DLLs loaded into memory. As soon as the container exists, I expect anything loaded by those containers into memory would get released. Any user space apps on the host should be unaffected by what DLLs get loaded in guests. Will investigate in either case.

[thaJeztah]

Yes, situation on Windows could be "different". The issue in Moby was related to docker cp. To copy files from the container to the host, a process (re-exec) was launched in the container namespace (or chrooted environent; I'd have to dig up details). Golang's archive/tar package would do a user-lookup to resolve usernames, which could result in a dlopen within the container's filesystem. As the daemon and "re-exec" share the same Go runtime, both would share memory, and the loaded dll therefore be used by the daemon itself.

Perhaps others can fill be in on the more technical details 😅

[gabriel-samfira]

Ahh. In this case we don't really re-exec the already running binary. We just create a new container with the binary mounted inside with a different name (get-user-info.exe) and execute it as a new task. There would be two separate processes, one on the host and one inside the container.

It should be the equivalent of doing:

PS C:\> cmd.exe /c dir C:\test
 Volume in drive C has no label.
 Volume Serial Number is 1A83-F08F

 Directory of C:\test

01/19/2023  04:03 PM    <DIR>          .
01/19/2023  01:24 PM        48,101,888 buildkitd.exe
01/19/2023  04:03 PM    <SYMLINK>      get-user-info.exe [C:\test\buildkitd.exe]
               2 File(s)     48,101,888 bytes
               1 Dir(s)  23,039,586,304 bytes free
PS C:\> 
PS C:\> 
PS C:\> docker run -it -e 'PATH=%PATH%;C:\test' --rm -v C:\test:C:\test a1cc73b9ec99 cmd
Microsoft Windows [Version 10.0.20348.1249]
(c) Microsoft Corporation. All rights reserved.

C:\>get-user-info administrator
{"UID":0,"GID":0,"SID":"S-1-5-21-2702878673-795188819-444038987-500"}

[corhere]

The issue in moby with docker cp had to do with the re-exec chroot(2)ing into the container filesystem without actually entering the container's namespaces, so malicious code loaded into the re-exec'ed process from the container's filesystem would be running with root privileges on the host. There was no sharing of Go runtimes or anything like that. Nowadays docker cp uses a thread rather than a whole process, and so does share a Go runtime and memory space with the daemon, but that happened long after the docker cp container escape was discovered and patched.

Projecting a binary into the container and exec'ing it as a container process should provide all the same isolation guarantees as any other container process...so long as there is no way to overwrite the host binary from inside the container. Since Windows has mandatory file locking for binaries of running processes I would imagine it wouldn't be an issue, though who knows; Windows is full of surprises.