No charset in url-encoded form in RestSharp 109.0.1

[RudiSzalai]

Describe the bug
no cookie in the restresponse cookies containter with .NET 7.0 and RestSharp 109.0.1

To Reproduce

var options109 = new RestClientOptions($"{realm.ConfigSource}")
{
         RemoteCertificateValidationCallback = (restclientsender, certificate, chain, sslPolicyErrors) => true,
         MaxTimeout = -1,
};

var restclient = new RestClient(options109);

var loginrequest = new RestRequest();
                
loginrequest.Resource = "login";
loginrequest.Method = Method.Post;
loginrequest.AddParameter("username", realm.Username);
loginrequest.AddParameter("password", realm.Password);
               
var loginresult = await restclient.ExecuteAsync(loginrequest).ConfigureAwait(false);
               

The exact same code returns a cookie on .NET 6 and RestSharp 108.0.3

Stack trace
no exception or any errors

Desktop (please complete the following information):

• OS: Azure Function
• .NET version .NET 7.0
• Version 109.0.1

Additional context
The differences in the restresponse between the two versions:

108
ContentHeaders: (there is only 1, and the ContentType is empty)
{HeaderParameter { Name = Content-Type, Value = text/html; charset=utf-8, Type = HttpHeader, Encode = False, ContentType = }}

ContentLength = null
RawBytes = {byte[695223]}

109
ContentHeaders: (there are 1, and the ContentType is text/plain)
{HeaderParameter { Name = Content-Type, Value = text/html; charset=utf-8, Type = HttpHeader, Encode = False, ContentType = text/plain }}
{HeaderParameter { Name = Content-Length, Value = 2352, Type = HttpHeader, Encode = False, ContentType = text/plain }}

ContentLength = 2352
RawBytes = {byte[2352]}

I need to use .NET 7.0 and on that platform:
108.0.3 has a bug and throws an exception explained here: https://github.com/restsharp/RestSharp/issues/1969
109.0.1 doesn't work with the above issue.

If someone gives me a solution/workaround I'll happily contribute $50 for this project, because currently I can't do my work as nothing works since the .NET 7.0 upgrade.

[alexeyzimarev]

Do you have an example of the raw response? How did the cookies work in previous versions, were you able to get the cookies in the client container?

[RudiSzalai]

With 108/.NET 6.0 the cookie is in the restresponse Cookies collection and in the restclient CookieContainer as expected.
With 109.0.1/.NET 7.0 there is no cookie anywhere. - I've just realized that the authentication fails as if I feed the wrong password and this is why the Content of the response is much smaller and no cookie.
Most definitely I feed the same username/password in both instances and the server/API is the same in both cases. So it looks like something is not right with the request. Any idea what can I look for?

[alexeyzimarev]

You can send the request to requestbin using both versions and compare them. It's usually obvious what the difference is.

[RudiSzalai]

The only difference I can see is that the restrequest Parameters with 108 has no content type and on .NET 6.0 this works for authentication:
[0] = {GetOrPostParameter { Name = username, Value = svc_0365_ipam, Type = GetOrPost, Encode = True, ContentType = }}
(with 108 and .NET 7.0 this empty content type causes and exception, this is fixed in 109.0.1)

with 109 the same parameter in Parameters look like this:
[0] = {GetOrPostParameter { Name = username, Value = svc_0365_ipam, Type = GetOrPost, Encode = True, ContentType = text/plain }}

It seems if the ContentType is set in the request param to 'text/plain' then the authentication fails, so no cookie. (for the testing I rolled back the function to .NET 6.0 so I'm using the same configuration/username/password for both cases for the Azure function, so the username cannot be not the same for the two test cases)

[alexeyzimarev]

We took the default content type from botnet runtime code https://github.com/dotnet/runtime/blob/e21b7ea213386701a4904c70c09005c9bc3cb95c/src/libraries/System.Net.Http/src/System/Net/Http/StringContent.cs#L16

As I suggested, if you make your requests to requestbin, you will see the actual difference. Inspecting parameters doesn't tell you how the raw request looks like.

[RudiSzalai]

here are two requests sent from the same function to requestbin/pipedream:
109.0.1.0: Content-Type: application/x-www-form-urlencoded (with this authentication fails)
108.0.3.0: Content-Type: application/x-www-form-urlencoded; charset=utf-8 (with this authentication is OK)

==================================================================
Host: enkv8qp9bpmwd.x.pipedream.net
X-Amzn-Trace-Id: Root=1-6422ee84-4649646d6512c3c708a08c01
Content-Length: 27
Accept: application/json, text/json, text/x-json, text/javascript, application/xml, text/xml
User-Agent: RestSharp/109.0.1.0
Accept-Encoding: gzip, deflate, br
Content-Type: application/x-www-form-urlencoded

username=asg&password=sdfgd

======================================================================
Host: enkv8qp9bpmwd.x.pipedream.net
X-Amzn-Trace-Id: Root=1-6422ed67-1a1a5fa0072d4b312a6a52d9
Content-Length: 27
Accept: application/json, text/json, text/x-json, text/javascript, application/xml, text/xml
User-Agent: RestSharp/108.0.3.0
Accept-Encoding: gzip, deflate, br
Content-Type: application/x-www-form-urlencoded; charset=utf-8

username=asg&password=sdfgd

===================================================================

[alexeyzimarev]

RestSharp doesn't touch the content type charset unless the DisableCharset option is set to true. You can actually see what happens if you use v109 with .NET 6. If it will add the charset, it's yet another breaking change in .NET 7.

[RudiSzalai]

both of these are from .NET 6.0. It seems the new restsharp 109 doesn't put the charset in the query, and the authentication fails without that, while 108 and earlier did. This is the code I used to generate the requests:

var testop = new RestClientOptions("https://enkv8qp9bpmwd.x.pipedream.net")
{
     RemoteCertificateValidationCallback = (restclientsender, certificate, chain, sslPolicyErrors) => true,
     MaxTimeout = -1,
};

var testrestclient = new RestClient(testop);
var testloginrequest = new RestRequest();
testloginrequest.Method = Method.Post;
testloginrequest.AddParameter("username", "asg");
testloginrequest.AddParameter("password", "sdfgd");
var testloginresult = await testrestclient.ExecuteAsync(testloginrequest).ConfigureAwait(false);

So 109 doesn't work for me regardless of the .NET version, because it omits the charset and without that the authentication fails. (so no cookie)
108 works on .NET 6, throws an exception on .NET 7 due to a known breaking change in .NET 7. The AddParameter would need the contentType as a workaround, but it seems I can't add it any ways I tried.

So how can I add the charset to to 109 or how can I add the contentType to 108 in order to make any of them work on .NET 7.0?

[alexeyzimarev]

I am not sure what exactly happens because there's a test that makes a call with multi-part form and it ensures that the charset is there as part of the content type:

RestSharp/test/RestSharp.Tests.Integrated/MultipartFormDataTests.cs

Lines 25 to 32 in 93ca86a

	const string CharsetString = "charset=utf-8";
	const string ContentTypeString = $"{KnownHeaders.ContentType}: text/plain; {CharsetString}";
	const string ContentDispositionString = $"{KnownHeaders.ContentDisposition}: form-data;";
	const string Expected =
	$"--{{0}}{LineBreak}{ContentTypeString}{LineBreak}{ContentDispositionString} name=foo{LineBreak}{LineBreak}bar{LineBreak}" +
	$"--{{0}}{LineBreak}{ContentTypeString}{LineBreak}{ContentDispositionString} name=\"a name with spaces\"{LineBreak}{LineBreak}somedata{LineBreak}" +
	$"--{{0}}--{LineBreak}";

[alexeyzimarev]

Ok, wrong test. It's the multi-part and your request uses url-encoded

[alexeyzimarev]

I don't see any changes in this code for almost a year. Have no idea why the behaviour could change

            var formContent = new FormUrlEncodedContent(
                postParameters
                    .Select(x => new KeyValuePair<string, string>(x.Name!, x.Value?.ToString() ?? string.Empty))!
            );
            Content = formContent;

[alexeyzimarev]

The only way the charset is getting set now is when the compilation branch for .NET Framework or .NET Standard is used. But not .NET 6 or 7. The FormUrlEncodedContent class doesn't add anything to the charset. The "legacy" branch constructs the content by itself, and uses StringContent, which sets the charset.

[htve]

I encountered the same problem.

console.exe x86 at win11 22h2 test:

1. TargetFramework = net7.0 or net6.0, use v108.0.3, result: working
2. TargetFramework = net7.0 or net6.0, use v109.0.1, result: not working