Remove binary cloaks from the VMR's main branch

[MichaelSimons]

There is a shared desire to build both the Microsoft flavor of .NET as well as all of the source-build distro versions from the same source (one repo/commit/tag). To support this binaries will be allowed within the VMR going forward but require an explicit action. The current set of OSS-license binaries will be allowed wholesale as to not hinder the Unified Build project.

When doing this work, it is important to preserve cloaks that remove non-oss licensed components e.g. https://github.com/dotnet/installer/blob/main/src/VirtualMonoRepo/source-mappings.json#L193.

As part of this work, it will have to be decided what to do with the binary scan detector. That tool may still be needed by #4089

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ellahathaway]

I'm thinking that we should keep the binary scanner. It will be necessary in preventing any new and unwanted binaries in the VMR. However, I'd like to modify the scanner to use the source-mapping file rather than the allowed-binaries.txt. The reasoning for this is that a developer would only have to touch a single file if they decide to include or cloak a file. Using source-mapping file, the binary scanner will search for any binaries not explicitly listed as "includes".

This work will involve a few steps:

1. Move all inclusions in allowed-binaries.txt to source-mapping.json as "includes"
2. Move all allowed cloaked binaries from "excludes" to "includes". This way, the binary scanner knows to ignore them.
3. Update the binary scanner to use "includes" from source-mappings as binaries to ignore.
4. Change how SB uses the binary scanner. In SB, the binaries will be removed while running ./prep.sh. The binary scanner runs as part of tag-and-scan which occurs before "prep". I propose having two separate jobs for "tag" and "scan". That way, tag can run first (probably not at all for lite builds), and scan can run after prepping.

@premun It is my understanding that you originally implemented the binary scan functionality. Do you think that these changes would be possible? I've taken a look through the binary scanner source code, but I'd like your thoughts on these proposed changes since you're more well-versed with the tool than I am.

[premun]

Yeah, this sounds like a good plan to me. It should be fairly straightforward to extend the file model and transition onto it.

The scan itself was implement by @dkurepa but we're both able to assist with the changes. There are even E2E tests in the arcade-services repo which execute everything on a local VMR so this should help validate everything upfront.

[MichaelSimons]

The scan itself was implement by @dkurepa but we're both able to assist with the changes. There are even E2E tests in the arcade-services repo which execute everything on a local VMR so this should help validate everything upfront.

I wonder if there is value in pulling this outside of arcade-services so that is can be run by distro-maintainers in their source-build environment? e.g. treat is more as a smoke-test.