Using GroupBy with auto-generated expression passed to Select throws an exception in RC1 and RC2

[aradalvand]

(Here's the runnable repo for the issue described below)

I have an expression visitor that replaces every occurrence of a method that's marked with a certain attribute with an expression that that method points to. It's not that complicated, and it used to work perfectly until I updated to RC1, and then RC2, it used to work in preview 7 and before.

This problem arises when I'm working with GroupBy.

The code:

Imagine you have a Purchase entity that has a DatePurchased property on it of type DateTime.
I'm executing a query to figure out how many purchases took place on each day. The query will look like this. Note that the expression passed to the Select method gets generated on the fly by my expression visitor:

var result = dbContext.Purchases
    .GroupBy(p => p.DatePurchased.Date)
    .Select(g => new {
        Date = g.Key,
        Count = g.Count(),
    })
    .ToList();

Looks simple enough, huh? But I actually get an exception in RC1 and RC2. Prior to that there was no problem.

I even tried to compare two expressions, one written directly by hand and the other one generated by my visitor, using ExpressionEqualityComparer, and it returns true, which confirms they're the same. Yet I get an exception for one and not for the other! Incredibly strange!

I strongly recommend you clone the repo I linked to above, and see everything for yourself.

The errors:

I actually get two different exceptions on RC1 and RC2.
Here's the exception I get on RC2:

Unhandled exception. System.InvalidOperationException: The LINQ expression 'DbSet<Purchase>()
    .GroupBy(p => p.DatePurchased.Date)
    .Select(g => new CountOnDateDto{
        Date = g.Key,
        Count = g
            .AsQueryable()
            .Count()
    }
    )' could not be translated. Additional information: Translation of 'Select' which contains grouping parameter without composition is not supported. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.
   at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.Expand(Expression query)
   at Microsoft.EntityFrameworkCore.Query.QueryTranslationPreprocessor.Process(Expression query)
   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass9_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.GetEnumerator()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Program.<Main>$(String[] args) in C:\Users\Arad\source\repos\EfCoreGroupByProblem\EfCoreGroupByProblem\Program.cs:line 42

What does this even mean?! I googled the error message but no relevant search results came up.

And here's the one I get on RC1:

Unhandled exception. System.ArgumentException: Property 'System.DateTime Key' is not defined for type 'System.Linq.IQueryable`1[EfCoreGroupByProblem.Purchase]' (Parameter 'property')
   at System.Linq.Expressions.Expression.Property(Expression expression, PropertyInfo property)
   at System.Linq.Expressions.Expression.MakeMemberAccess(Expression expression, MemberInfo member)
   at System.Linq.Expressions.MemberExpression.Update(Expression expression)
   at System.Linq.Expressions.ExpressionVisitor.VisitMember(MemberExpression node)
   at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.GroupingElementReplacingExpressionVisitor.VisitMember(MemberExpression memberExpression)
   at System.Linq.Expressions.MemberExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.GroupingElementReplacingExpressionVisitor.Visit(Expression expression)
   at System.Linq.Expressions.ExpressionVisitor.VisitMemberAssignment(MemberAssignment node)
   at System.Linq.Expressions.ExpressionVisitor.VisitMemberBinding(MemberBinding node)
   at System.Linq.Expressions.ExpressionVisitor.Visit[T](ReadOnlyCollection`1 nodes, Func`2 elementVisitor)
   at System.Linq.Expressions.ExpressionVisitor.VisitMemberInit(MemberInitExpression node)
   at System.Linq.Expressions.MemberInitExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.GroupingElementReplacingExpressionVisitor.Visit(Expression expression)
   at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.ProcessSelect(GroupByNavigationExpansionExpression groupBySource, LambdaExpression selector)
   at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.Expand(Expression query)
   at Microsoft.EntityFrameworkCore.Query.QueryTranslationPreprocessor.Process(Expression query)
   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass9_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.GetEnumerator()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Program.<Main>$(String[] args) in C:\Users\Arad\source\repos\EfCoreGroupByProblem\EfCoreGroupByProblem\Program.cs:line 42

Provider and version information

EF Core version: from 6.0 RC1 (previously worked in 6.0 Preview 7)
Database provider: (e.g. Microsoft.EntityFrameworkCore.SqlServer)
Target framework: (e.g. .NET 6.0 RC)
Operating system: Windows 10 version 20H2 (build 19042.1288)
IDE: (e.g. Visual Studio 2022 RC - version 17.0.0 RC3)

I've been beating my head against the wall for the last few days and I have run out of ideas for what could be causing this. I'm convinced this is probably a subtle bug somewhere in the new RC versions of EF Core, as there was absolutely no problem prior to those versions.

[smitpatel]

Probably user is not replacing the "g" correctly in the lambda. Since this works writing query directly, this is likely bug in the user defined expression visitor.

[aradalvand]

@smitpatel But I doubt that's the case for three reasons:

1. As I mentioned in my original comment when you compare the two expressions with ExpressionEqualityComparer it returns true. Correct me if I'm wrong but if the parameters were being replaced incorrectly in the generated expression then this would return false.
2. This used to work in every version prior to RC1. If there's a problem with the parameters or something, I guess it must've failed in the past as well.
3. This works properly in every case except for when I'm dealing with GroupBy. Again, if there was something wrong with the parameters, it should throw an error in every case. Right?

Although I could definitely be wrong here... 😬

[smitpatel]

• ExpressionEqualityComparer is not perfect truth in the sense it doesn't verify each parameter being same reference (just name and type match). That is by design so EEC saying 2 expressions are same is not a guarantee that they are the same.
• Not necessarily failed in past. Especially if the check in past was somewhat more relaxed then it could have allowed faulty tree to pass through.
• I assume it works in every other case than group by then it is with different lambda whose parameter is not of type grouping.

Looking at error message in rc1/rc2 & changes happened between both of those g.Key doesn't seem to be correct form. We have a code to intercept and replace g.Key when processing in the tree. This code was added in rc1. Both exception message indicates that it is not being intercepted which leads to exact failures as above.