Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Getting timeout error for operations above 30 seconds. #2368

Closed
icardosos opened this issue Nov 10, 2017 · 31 comments
Closed

Getting timeout error for operations above 30 seconds. #2368

icardosos opened this issue Nov 10, 2017 · 31 comments
Assignees
Labels
bug This is a product bug.
Milestone

Comments

@icardosos
Copy link

icardosos commented Nov 10, 2017

Hi guys,

I'm getting timeout exception for operations that took more than 30 seconds. Here is my scenario:

Server Side: Basic Wcf Application (.NET full 4.6.1)
Basically, I have one endpoint that call Thread.Sleep

image

Client Side: Console Application (.NET Core 2.0)
A console application that call the service above, I used Wcf Connect Service to generate the proxy class (https://marketplace.visualstudio.com/items?itemName=erikcai-MSFT.VisualStudioWCFConnectedService)

image

When I set Thread.Sleep < 30, everything works fine but as soon as I put more 30 seconds and I'm always getting the timeout error in 30 seconds, I've already set all the possibles timeout but it does not work (considering the default timeout for binding does not make sense to set it)

@zhenlan
Copy link
Member

zhenlan commented Nov 10, 2017

@icardosos Thank you for reporting the issue. We are able to repro it and will investigate further.

@zhenlan zhenlan added this to the S127 milestone Nov 10, 2017
@zhenlan zhenlan added the bug This is a product bug. label Nov 10, 2017
@mconnew
Copy link
Member

mconnew commented Nov 10, 2017

The cause of this is a little complicated. We used to instantiate our own instance of WinHttpHandler as at the time, HttpClientHandler didn't have sufficient API's to be able to do things like set a certificate validation callback handler. WinHttpHandler has some extra properties that can be set for various timeouts. One of these is ReceiveHeadersTimeout which is how long to wait until the response headers are returned. The default value for this is 30 seconds. We weren't setting this to a specific value and so this is the timeout which is applying.
Now that HttpClientHandler has more properties, I made a change to WCF to directly use that. HttpClientHandler disables these timeouts where WCF doesn't. I made this change in the master branch before the 2.0 release but I don't know why it didn't make it into the 2.0 release branch.
As the next release isn't very soon, you have 3 options. 1) You could pick up a development build from myget which has this issue fixed. 2) You could build your own WCF assembly from source and use that. 2) You can use reflection to replace the WinHttpHandler with a modified one. We cache them in a static Dictionary so after the first request you would be able to replace it. If you want to do the reflection approach, let me know and I'll work out the code you would need.

@icardosos
Copy link
Author

Thanks @mconnew would be great if you help me out with reflection approach

@trichling
Copy link

@mconnew I am running into the same issue and would be very glad to get assistance!

@mconnew
Copy link
Member

mconnew commented Nov 14, 2017

I have some code I've tested to fix this. I was about to post the solution when power went out in the office. We're having some unusual weather in Seattle right now. I'll post the code tomorrow.

@mconnew
Copy link
Member

mconnew commented Nov 14, 2017

You can find the example code in a gist here, The type SimpleServiceClient is a client created by the WCF connected service VS extension against a basic service. The operation DelayedResponseAsync just sleeps for the amount of time that was passed. This code is fragile and will break on the next release. Would switching to WebSockets work for you as that won't have this problem? You can use WebSockets by using NetHttpBinding and setting NetHttpBinding.WebSocketSettings.TransportUsage to WebSocketTransportUsage.Always.

@icardosos
Copy link
Author

icardosos commented Nov 16, 2017

@mconnew , although your code works I'm not able to use it in my project because I don't want to call an endpoint twice and I can't change that service either.

I've already try to change to a development build from myget but it didn't work, I created the project and the proxy by the WCF connected service it compiles but I got a runtime error seems like the proxy is looking for another version of System.ServiceModel.Primitives.

Anyway, I will have to change my project to the full framework.

Thank you

@zhenlan
Copy link
Member

zhenlan commented Nov 16, 2017

After you ran WCF connected service, please update your .csproj to the version of System.ServiceModel.* packages to the version you want to use from myget (suppose you already added the myget feed to the list of your package feeds).

@jpjohnson
Copy link

Thanks @mconnew, the reflection solution worked!

Will that solution break in the next release?

@mconnew
Copy link
Member

mconnew commented Nov 28, 2017

@jpjohnson, yes it will break in the next release, but the next release won't have this problem. In the next release we don't use a DelegatingHandler as the handler in HttpClient so you could add code similar to this at line 29 of the gist:

            DelegatingHandler delegatingHandler = handlerField.GetValue(httpClient) as DelegatingHandler;
            if(delegatingHandler == null) return; // HttpClientHandler is not a DelegatingHandler

@jpjohnson
Copy link

Thanks @mconnew, looking forward to next release.

@rahul-raut
Copy link

@mconnew I am still facing this issue. I am using third party vendors WCF service and that service takes more 30 seconds every time. I also tried to use reflection to fix this issue but not bale to fix it. Could you please assist me what I am doing wrong?

@zhenlan zhenlan modified the milestones: S129, S130 Jan 12, 2018
@MelnikovIG
Copy link

MelnikovIG commented Jan 19, 2018

@mconnew I think in you timeout fix gist executing real request to fill _httpClientCache is bad idea. Maybe it is better to fill HttpClient cache with fake request like

try
{
    (client.ChannelFactory.CreateChannel(new EndpointAddress(uri)) as IRequestChannel).
    Request(Message.CreateMessage(MessageVersion.None, null))
}
catch (Exception e)
{
}

Or maybe anyone found a more elegant solution?

@Dmitry44
Copy link

Hi @mconnew,
I have similar problem with web service (non WCF). Is it possible that the root case of the problem the same?

@ttugates
Copy link

ttugates commented Feb 5, 2018

@mconnew , Re: Option 1). What is the myget package feed URL to be able to reference the development build?

~Thx

@carlottokjellkvist
Copy link

Is there any estimation on when this issue will be released in 2.0.x?

Br
Carl-Otto

@zhenlan
Copy link
Member

zhenlan commented Mar 6, 2018

The issue is already fixed in recent 2.1 preview1 release.
https://github.com/dotnet/wcf/releases/tag/v2.1.0-preview1

@carlottokjellkvist
Copy link

@zhenlan Tnx for your answer, but that is a preview and we are soon going for production. So my question then is: will there not be any more 2.0.x releases and/or when can we expect the preview to be released non preview. I suppose it will take some time for the preview so make it there?

Br
Carl-Otto

@aifrim
Copy link

aifrim commented Mar 12, 2018

@zhenlan is there a ETA on a stable 2.1 (non-preview) release? Or at least a 2.0.6 release?

@danielsmf
Copy link

danielsmf commented Mar 13, 2018

Hi guys, I just installed the 2.1.300-preview1-008174 and it doesn't solve this problem yet.

and if you use docker with linux container, neither reflection solution works. CAST ERROR, WinHttpHandler /

@liri2006
Copy link

@danielsmf, Preview did fix issue for me. I do not run the inside docker though.

@danielsmf
Copy link

@liri2006 It's strange, in my local environment i had timeout, how did you set your timeout?

Like this?
startup.cs>

client.Address = new System.ServiceModel.EndpointAddress(Configuration["Endpoint"]);
                client.Endpoint.Binding.ReceiveTimeout = new TimeSpan(0, 20, 0);
                client.Endpoint.Binding.OpenTimeout = new TimeSpan(0, 20, 0);
                client.Endpoint.Binding.CloseTimeout = new TimeSpan(0, 20, 0);
                client.Endpoint.Binding.SendTimeout = new TimeSpan(0, 20, 0);
                client.InnerChannel.OperationTimeout = new TimeSpan(0, 20, 0);

and

web.config>
<system.web>
    <httpRuntime maxRequestLength="8096"
        useFullyQualifiedRedirectUrl="true"
        executionTimeout="1200"/>
  </system.web>
  <system.webServer>
    <handlers>
      <remove name="aspNetCore"/>
      <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModule" resourceType="Unspecified"/>
    </handlers>
    <aspNetCore requestTimeout="00:20:00"  processPath="%LAUNCHER_PATH%" arguments="%LAUNCHER_ARGS%" stdoutLogEnabled="false" stdoutLogFile=".\logs\stdout"
                />
  </system.webServer>

  <system.applicationHost>
    <webLimits connectionTimeout="00:21:00"
       headerWaitTimeout="00:21:00"
       minBytesPerSecond="500"
      />
  </system.applicationHost>

@zhenlan
Copy link
Member

zhenlan commented Mar 14, 2018

@danielsmf please make sure you are using 4.5.0-preview1-26228-01 version of the System.ServiceModel.* packages and your app targets netcoreapp2.1.

@danielsmf
Copy link

That is it... I change to Package System.ServiceModel.Http -Version 4.5.0-preview1-26228-01 and it works!!!

Thanks!!

@zhenlan
Copy link
Member

zhenlan commented Mar 23, 2018

@carlottokjellkvist, @aifrim I understand it may take a little bit for 2.1 to reach stable, so we are working on a servicing release with the fix of this issue for 2.0. Currently, you can find these packages from myget below. Please give them a try if you can.

https://dotnet.myget.org/feed/dotnet-core/package/nuget/System.ServiceModel.Duplex/4.4.2
https://dotnet.myget.org/feed/dotnet-core/package/nuget/System.ServiceModel.Http/4.4.2
https://dotnet.myget.org/feed/dotnet-core/package/nuget/System.ServiceModel.NetTcp/4.4.2
https://dotnet.myget.org/feed/dotnet-core/package/nuget/System.ServiceModel.Primitives/4.4.2
https://dotnet.myget.org/feed/dotnet-core/package/nuget/System.ServiceModel.Security/4.4.2
https://dotnet.myget.org/feed/dotnet-core/package/nuget/System.Private.ServiceModel/4.4.2

Once we finish validation internally, we will release them to nuget.org.

@elvirdolic
Copy link

Hi, is it already on nuget when we expect it on nuget?

@carlottokjellkvist
Copy link

@zhenlan is there a estimation for when it will be released?

@jiimaho
Copy link

jiimaho commented Apr 13, 2018

Hello, i'm also waiting for when this will be released stable on nuget.

Am I understanding all this correctly if running .NET Core 2.0/2.1 and with System.ServiceModel.* 4.4.2 will solve the timeout problems?

@zhenlan
Copy link
Member

zhenlan commented Apr 14, 2018

We got a few higher priority tasks lately. We are still testing packages. I am aiming to release them next week. Has anyone tried the packages and do they work for you?

Am I understanding all this correctly if running .NET Core 2.0/2.1 and with System.ServiceModel.* 4.4.2 will solve the timeout problems?

@jiimaho, for .NET Core 2.0, yes, the v4.4.2 S.SM.* packages will solve the timeout problem. For .NET Core 2.1, the problem is already solved with our recently released v4.5.0-preview* packages.

@carlottokjellkvist
Copy link

@zhenlan We are actually trying it right now and it seems to work as expected :-)

@zhenlan
Copy link
Member

zhenlan commented Apr 17, 2018

Version 4.4.2 System.ServiceModel.* packages are released to nuget.org this morning. Please let us know if this solves your issue.

https://github.com/dotnet/wcf/releases/tag/v2.0.7

@zhenlan zhenlan closed this as completed Apr 24, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This is a product bug.
Projects
None yet
Development

No branches or pull requests