Use PayloadReader in System.Resources.Extensions

[adamsitnik]

High level overview.

1. All PayloadReader related code was moved to System.Runtime.Serialization.BinaryFormat library, the library is marked with IsPackable set to false so we don't publish any NuGet package by accident. Other projects reference it's code via links (to ensure nothing relies on the project before the API gets approved).
2. The deserializer authored by @JeremyKuhne for WinForms was modified only to allow support of older monikers (see the commits).
3. System.Resources.Extensions is now using simplified deserializer by default.
4. It's possible to request using BinaryFormatter by setting System.Resources.Extensions.UseBinaryFormatter app context switch.
5. All tests have been migrated and are passing.
6. I've not reduced the code duplication between System.Resources.Extensions.BinaryFormat.Tests and actual BinaryFormatter test project as I assumed the latter is soon going to be removed from this repo.

[adamsitnik]

@JeremyKuhne I am not sure if you want to review all the files (you already did that in dotnet/winforms#11341). In case you don't, you can simply review the following commits:

• f0c20fc where I applied some non trivial changes to get the code compile for older monikers
• 3a90452 where I've fixed bugs the commit above has introduced
• 00608e4 where I've made the switch
• 6a36687 where I added the compat switch

What is still on my TODO list is to move the tests from https://github.com/dotnet/winforms/tree/main/src/System.Private.Windows.Core/tests/BinaryFormatTests. It will be another commit with no product code changes (only copy paste of the tests and maybe some minor MSBuild changes)

[JeremyKuhne]

Note that this is effectively a straight copy of the existing BinaryFormatter logic. All catch blocks in this class match the existing behavior.

[JeremyKuhne]

I haven't found a need to requeue multiple times. One could, however, make this a bit more complicated and keep requeuing as long as the count continues to go down.

[JeremyKuhne]

Do the record type checks prevent any observably bad behavior? I worry a little bit about things ending up in orders that don't match the constraints of the spec, but it otherwise being ok.

[bartonjs]

This will grow from 0 to 4 to 8 to ...

The 99% case for large arrays will be bytes, and that is covered for most inputs by the "if I have that much data remaining, just read it" above. But large arrays of TimeSpan (or int when not on new.NET) are going to have to go through many reallocs.

The easiest, of course, is

Suggested change

	List<T> values = [];
	List<T> values = new List<T>(count);

If the threat model indicates that this puts too much power in the hands of the payload, then it needs to change to a page based allocation and a custom type.

For expediency, I recommend just pre-sizing the list and taking this as an issue to follow up on later (as part of, or prior to, the threat model). As current usage is limited to resources, it's a fine thing to do.

[adamsitnik]

The list is not pre-sized on purpose: adamsitnik/SafePayloadReader#4

Since the code for decoding is already optimized for .NET Core, I am going to add the optimization for other TFMs

[steveharter]

FYI test failure:

Microsoft.DotNet.XUnitExtensions.SkipTestException : Not enough memory available to test max array size support
Stack Trace
   at System.Runtime.Serialization.BinaryFormat.Tests.EdgeCaseTests.CanReadArrayOfAnySize(Int32 length) in /_/src/libraries/System.Runtime.Serialization.BinaryFormat/tests/EdgeCaseTests.cs:line 85
   at InvokeStub_EdgeCaseTests.CanReadArrayOfAnySize(Object, Span`1)
   at System.Reflection.MethodBaseInvoker.InvokeWithOneArg(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)