HttpWebRequest throws an uncatchable task exception when processing requests simultaneously with timeout specified

[david4neblio]

Steps to Reproduce

Just open up a new project and copy and paste the following code to replicate the error.

using System;
using System.Net;
using System.IO;
using System.Threading;
using System.Threading.Tasks;

namespace TestProgram
{
    class MainClass
    {
		static bool close_program = false;

        public static void Main(string[] args)
		{
			//Create a catch all Task exception handler
			//This will catch the eventual task exception
			TaskScheduler.UnobservedTaskException += (s, e) =>
                LastExceptionHandler(e.Exception, "TaskScheduler.UnobservedTaskException");

			Console.WriteLine("This program contains 2 types of exceptions");
			Console.WriteLine("One that is catchable, the other that is not. The latter will be displayed here.");
			Console.WriteLine("Now wait for the uncatchable exception (less than 60 seconds in most cases), do not press any key.");
			int counter = 0;
			while(true){
				if(Console.KeyAvailable){
					break;
				}
				Thread.Sleep(1000);
				counter++;

                //We will run 2 queries to the same URL in an alternate task
				Task.Run(() => QueryHttp());
				Task.Run(() => QueryHttp());

				Console.WriteLine(counter + " Second(s)");
				if(close_program == true){
					break;
				}
			}
        }

        public static string QueryHttp()
		{
			try
			{
				ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;
                
				HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://httpstat.us/200?sleep=3000");
				request.Credentials = System.Net.CredentialCache.DefaultCredentials;
				request.Timeout = 1000;
                            
				using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
				{
					using (StreamReader respreader = new StreamReader(response.GetResponseStream()))
					{
						return respreader.ReadToEnd();
					}
				}
			}
			catch (Exception)
			{
				//This exception will normally be a timeout exception which is catchable
			}
			return "";
		}

		public static void LastExceptionHandler(Exception e, string source)
        {
			//This will catch the eventual task exception
			Console.WriteLine("Uncatchable thread exception: " + e);
			close_program = true;
        }
    }
}

Current Behavior

This code throws a task exception by method MyGetResponseAsync found within HttpWebRequest. The exception here is uncatchable due to the fact that the MyGetReponseAsync is a task created by BeginGetResponse and there is no current way (I'm aware of) to add an exception handler to the MyGetResponseAsync task. So when the exception is thrown, it bubbles to the global task exception handler if one is set.

Expected Behavior

That the exception created by MyGetResponseAsync should be catchable outside of global exception handlers. The exception itself is probably caused by multithreading issues with HttpWebRequest (as it only happens when multiple requests are sent around the same time) or the threadpool used by the created task.

On which platforms did you notice this

[ ] macOS
[X] Linux
[ ] Windows

Version Used:
This error happens also in the most recent version of Mono as well.

Mono JIT compiler version 5.12.0.226 (tarball Thu May  3 09:42:09 UTC 2018)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           __thread
	SIGSEGV:       altstack
	Notifications: epoll
	Architecture:  amd64
	Disabled:      none
	Misc:          softdebug 
	Interpreter:   yes
	LLVM:          supported, not enabled.
	GC:            sgen (concurrent by default)

Stacktrace

N/A - Run the code to see the stacktrace

[marek-safar]

@baulig another async rewrite side effect :-(. Please look into it

[baulig]

I thought we fixed this in 8facc4a, but it looks like we didn't.

[bocciab]

I just tried installing Mono 5.18 along with Xamarin.Mac 5.2.1.13 and I still see the uncachable exception in the sample repro project. Is there something else needed to get the fix working?

[baulig]

It is certainly possible that your version of Xamarin.Mac has not picked up that fix yet.

[bocciab]

This is a critical issue. I'm currently at a decision point to keep this crashing my app or downgrade users back to that 10.14 invisible-UI issue from prior mono releases.

Is the fix going to come to mono 5.12 or at least 5.16 (current VS-Stable)? The other posts on this make it sound like we're waiting for VS 2019 to go to full public release before we'll get this.

[baulig]

This should've been fixed in #10732.

[bocciab]

I don't know what release version that is or should be in (we really could use public release versioning when talking about things - I can't follow your branching/release milestone naming). What I know is that if I look in the code in the latest stable 5.18 framework install I have (5.18.0.245), the code in the commit is not there. It wasn't in 5.16 either, but I've been installing all different versions trying to test things, so I lost my 5.16 install and VSMac is erroring trying to check updates to re-install VS Stable. I've also tried installing several versions of Mono and trying the sample repro posted in this issue and it's still throwing the same uncatchable task exception.

[NickAVolpe]

If it helps anyone, when I replace HttpWebRequest with HttpClient the error is not reproducible. Which is frustrating on my end because my code is written with HttpClient and I am still seeing this exception in our production environment. Suggestions welcome :)

`

var client = new HttpClient();
client.Timeout = new System.TimeSpan(0, 0, 1000);

var request = new HttpRequestMessage(HttpMethod.Get, uri);

resp = await client.SendAsync(request);

var foo = await resp.Content.ReadAsStringAsync();
return foo;

`

Visual Studio Community 2017 for Mac
Version 7.7.3 (build 43)

Mono Framework MDK
Runtime:
Mono 5.16.0.221 (2018-06/b63e5378e38) (64-bit)
Package version: 516000221

Xamarin.Mac
Version: 5.2.1.13 (Visual Studio Community)

Xamarin.Android
Version: 9.1.5.1 (Visual Studio Community)

[baulig]

Are you seeing this in Xamarin.Mac or Desktop Mono?

The fix is definitely in 5.18 (5.18.0.245 is this commit). I just downloaded the package and the test case from the top of this issue runs fine with it.

The latest release of Xamarin.Mac seems to be 5.2.1.11 (which is this commit). The Mono in it is commit 000780, which is from the 2018-04 branch. Your 5.2.1.13 is also using the same Mono commit.

We did not back port the fix into 2018-04.

[NickAVolpe]

Forgot to mention, I am running this in a Xamarin.Android project which I do not believe has mono 5.18 yet.

[bocciab]

I was running the latest stable Xamarin.Mac. Updating to the latest stable version of it, and even trying to update to 5.18, I still see the crash occurring.

I downgraded to XM 4.4 (a known prior build that wasn't seeing this crash) and the sample app had no uncatchable unobserved task exceptions.

As noted before, I'm not able to get updates for VS Mac, so I can't easily re-update to the latest stable. So I'll just ask if you can clarify if there's a Visual Studio (Mac / Xamarin.Mac) Stable version that has the fix in it? If not, will it be pack ported to whatever is out in the Stable channel or do we have to wait for VS 2019 or something?

[rocuh]

Is there anyway to escalate a backport? This regression has been in Xamarin.Android since October.

[xleon]

For those using Xamarin.Android, this pre-release is using mono 5.18

Not sure about iOS though

[gautambjain]

Badly need this fix. How to get it in VS 2017 without upgrading to VS 2019 Preview 2?

[gautambjain]

Badly need this fix. How to get it in VS 2017 without upgrading to VS 2019 Preview 2?

It works now after I downgraded Visual Studio 2017 to version 15.7.6 from this link: https://docs.microsoft.com/en-us/visualstudio/productinfo/installing-an-earlier-release-of-vs2017

[MagicAndre1981]

look if this latest VS 2017 15.9 compatible version has the fix:

https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-builds-d15-9/

The commit shows it has landed end January 2019 for 15.9 branch, so it should be included.

[MagicAndre1981]

@gautambjain @Roceh

15.9.7 has the fix:

https://github.com/xamarin/xamarin-android/releases/tag/xamarin-android-9.1.7.0
https://docs.microsoft.com/en-us/visualstudio/releasenotes/vs2017-relnotes#15.9.7