Query: Except/Intersect methods in Ix-Async does not work well with buffering #8393

smitpatel · 2017-05-05T20:44:15Z

Line in the source code:
await Task.WhenAll((Task) moveNextTask, this.fillSetTask).ConfigureAwait(false);

Disabled tests
Except_nested
Intersect_nested

Stack trace

Stacktraces: 
(new) Test collection for Microsoft.EntityFrameworkCore.SqlServer.FunctionalTests.AsyncQuerySqlServerTest (1): Microsoft.EntityFrameworkCore.SqlServer.FunctionalTests.AsyncQuerySqlServerTest.Intersect_nested 
System.InvalidOperationException : There is already an open DataReader associated with this Command which must be closed first.
   at System.Data.SqlClient.SqlCommand.<>c.<ExecuteDbDataReaderAsync>b__108_0(Task`1 result)
   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.<ExecuteAsync>d__26.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Query.Internal.AsyncQueryingEnumerable.AsyncEnumerator.<BufferlessMoveNext>d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Storage.Internal.SqlServerExecutionStrategy.<ExecuteAsync>d__6`2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Query.Internal.AsyncQueryingEnumerable.AsyncEnumerator.<MoveNext>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.SelectEnumerableAsyncIterator`2.<MoveNextCore>d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.AsyncIterator`1.<MoveNext>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.IntersectAsyncIterator`1.<MoveNextCore>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.AsyncIterator`1.<MoveNext>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.IntersectAsyncIterator`1.<MoveNextCore>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.AsyncIterator`1.<MoveNext>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.SelectEnumerableAsyncIterator`2.<MoveNextCore>d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.AsyncIterator`1.<MoveNext>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Query.Internal.AsyncLinqOperatorProvider.ExceptionInterceptor`1.EnumeratorExceptionInterceptor.<MoveNext>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Collections.Generic.AsyncEnumerableHelpers.<ToArrayWithLength>d__1`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Collections.Generic.AsyncEnumerableHelpers.<ToArray>d__0`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Specification.Tests.AsyncQueryTestBase`1.<AssertQuery>d__394`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Specification.Tests.AsyncQueryTestBase`1.<AssertQuery>d__388`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Specification.Tests.AsyncQueryTestBase`1.<Intersect_nested>d__348.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

Happens with MARS is false (buffering needed)

The text was updated successfully, but these errors were encountered:

- Fixes a race in async ShapedQuery execution. - Optimizes ShapedQuery by pushing down shaper to inner enumerable. - Removes IValueBufferCursor.Current - no longer used.

divega · 2018-10-20T12:20:54Z

Given we want to stop doing tricks to serialize async query enumeration (#10914), if in 3.0 we end still using System.Interactive.Async, we should ask it to stop using WhenAll in the regular query operators (it can still have a “special”, parallel version that implements it that way).

There is a good explanation at
https://github.com/dotnet/corefx/issues/32640#issuecomment-431572661.

Also related, issues with Union have been reported a few times although they have been hard to reproduce:
#12549.

Clearing up milestone to discuss tracking this for 3.0.

smitpatel · 2019-01-25T00:46:19Z

In 3.0, we are not going to do client eval so the only place where we need to call into linq operators over async enumerable would be InMemory (or any other provider which wants to evaluate something on client.
This issue is also related with #12048 if we need to get rid of IX-Async.
Further, we should be able to implement our own enumerator to deal with this.

smitpatel · 2019-06-05T23:07:42Z

We have removed IX-Async dependency. As noted above only InMemory would need to do client side operation but that would be done sync way so this issue won't arise. We don't need to do anything here.

smitpatel changed the title ~~Except/Intersect methods in Ix-Async does not work well with buffering.~~ Except/Intersect methods in Ix-Async does not work well with buffering May 5, 2017

This was referenced May 5, 2017

Skipped tests report #7160

Closed

Disable flaky tests #8395

Merged

ajcvickers assigned anpete May 12, 2017

ajcvickers added this to the 2.0.0 milestone May 12, 2017

ajcvickers added area-test type-bug and removed area-test labels May 12, 2017

anpete added a commit that referenced this issue May 16, 2017

Query: Testing out a fix for #8393:

973f98e

- Fixes a race in async ShapedQuery execution. - Optimizes ShapedQuery by pushing down shaper to inner enumerable. - Removes IValueBufferCursor.Current - no longer used.

anpete added a commit that referenced this issue May 16, 2017

Query: Testing out a fix for #8393:

7750fb4

- Fixes a race in async ShapedQuery execution. - Optimizes ShapedQuery by pushing down shaper to inner enumerable. - Removes IValueBufferCursor.Current - no longer used.

anpete added a commit that referenced this issue May 16, 2017

Query: Testing out a fix for #8393:

929bbf6

- Fixes a race in async ShapedQuery execution. - Optimizes ShapedQuery by pushing down shaper to inner enumerable. - Removes IValueBufferCursor.Current - no longer used.

anpete added a commit that referenced this issue May 16, 2017

Query: Testing out a fix for #8393:

3335d24

- Fixes a race in async ShapedQuery execution. - Optimizes ShapedQuery by pushing down shaper to inner enumerable. - Removes IValueBufferCursor.Current - no longer used.

anpete added a commit that referenced this issue May 16, 2017

Query: Testing out a fix for #8393:

4dee02c

- Fixes a race in async ShapedQuery execution. - Optimizes ShapedQuery by pushing down shaper to inner enumerable. - Removes IValueBufferCursor.Current - no longer used.

anpete added a commit that referenced this issue May 16, 2017

Query: Testing out a fix for #8393:

cc2cf8f

- Fixes a race in async ShapedQuery execution. - Optimizes ShapedQuery by pushing down shaper to inner enumerable. - Removes IValueBufferCursor.Current - no longer used.

anpete added a commit that referenced this issue May 16, 2017

Query: Testing out a fix for #8393:

3d1f294

- Fixes a race in async ShapedQuery execution. - Optimizes ShapedQuery by pushing down shaper to inner enumerable. - Removes IValueBufferCursor.Current - no longer used.

anpete added a commit that referenced this issue May 16, 2017

Query: Testing out a fix for #8393:

612642e

- Fixes a race in async ShapedQuery execution. - Optimizes ShapedQuery by pushing down shaper to inner enumerable. - Removes IValueBufferCursor.Current - no longer used.

anpete added a commit that referenced this issue May 16, 2017

Query: Testing out a fix for #8393:

99d6df6

- Fixes a race in async ShapedQuery execution. - Optimizes ShapedQuery by pushing down shaper to inner enumerable. - Removes IValueBufferCursor.Current - no longer used.

anpete added the propose-punt label Jun 26, 2017

ajcvickers modified the milestones: 2.0.0, Backlog Jun 29, 2017

ajcvickers added punted-for-2.0 and removed propose-punt labels Jun 30, 2017

ajcvickers changed the title ~~Except/Intersect methods in Ix-Async does not work well with buffering~~ Query: Except/Intersect methods in Ix-Async does not work well with buffering May 16, 2018

ajcvickers unassigned anpete Jun 6, 2018

divega removed this from the Backlog milestone Oct 20, 2018

ajcvickers added this to the 3.0.0 milestone Oct 22, 2018

ajcvickers assigned smitpatel Oct 22, 2018

smitpatel added the closed-not-needed label Jun 5, 2019

smitpatel removed this from the 3.0.0 milestone Jun 5, 2019

smitpatel removed their assignment Jun 5, 2019

smitpatel closed this as completed Jun 5, 2019

ajcvickers added the closed-out-of-scope This is not something that will be fixed/implemented and the issue is closed. label Mar 10, 2022

ajcvickers reopened this Oct 16, 2022

ajcvickers closed this as not planned Won't fix, can't repro, duplicate, stale Oct 16, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Query: Except/Intersect methods in Ix-Async does not work well with buffering #8393

Query: Except/Intersect methods in Ix-Async does not work well with buffering #8393

smitpatel commented May 5, 2017 •

edited

Loading

divega commented Oct 20, 2018

smitpatel commented Jan 25, 2019

smitpatel commented Jun 5, 2019

Query: Except/Intersect methods in Ix-Async does not work well with buffering #8393

Query: Except/Intersect methods in Ix-Async does not work well with buffering #8393

Comments

smitpatel commented May 5, 2017 • edited Loading

divega commented Oct 20, 2018

smitpatel commented Jan 25, 2019

smitpatel commented Jun 5, 2019

smitpatel commented May 5, 2017 •

edited

Loading