ArgumentNullException on query that worked in EF 6

[Gaz83]

Note sure if this issue is classed as the same as the currently open one here Issue 12603

So I have converted an application from wpf and EF6 to .net core 3 and EF core and
I have a query that seemed to work fine in EF 6 but gives me a ArgumentNullException
in EF core.
The annoying thing is that I tried creating a new app and a new database, stripping out tables and columns that are not required by the query, just so I could try and recreate the issue. Query works fine in this new recreated app, although the database only contains 2 records in each table.

Here is the query code.

var items2 = DbContext.UutResult.Where(x =>
                x.StartDateTime != null && (x.StartDateTime.Value.Date >= startDate &&
                                            x.StartDateTime.Value.Date <= endDate &&
                                            x.StationId == testerId)).Select(x => new TestResultItem()
            {
                TesterId = (int) x.TestSocketIndex,
                TesterType = x.StationId,
                TestDateTime = (DateTime) x.StartDateTime,
                TestResult = x.UutStatus,
                PanelBarcode = x.BatchSerialNumber,
                UutBarcode = x.UutStatus,
                TestTimeSec = (double) x.ExecutionTime,
                TestSteps = x.StepResult.Where(y => y.StepGroup == "Main").Select(y => new TestResultStepItem()
                {
                    StepName = y.StepName,
                    Report = y.ReportText,
                    TestResult = y.Status,
                    UpperLimit = y.StepNumericlimit1.FirstOrDefault().StepNumericlimit2.FirstOrDefault().HighLimit.ToString(),
                    OrderId = (int)y.OrderNumber
                }).ToList()
            }).ToList();

The exception is on this line, because if I comment it out then the query runs fine.

UpperLimit = y.StepNumericlimit1.FirstOrDefault().StepNumericlimit2.FirstOrDefault().HighLimit.ToString()

I assumed it was because StepNumericlimit1 could be null or StepNumericlimit2 could be null, but in the new dummy database I made sure these null conditions were there i.e. I made sure there was 1 StepNumericlimit1 with 0 child StepNumericlimit2, and it still worked.

Exception message: System.ArgumentNullException: 'Value cannot be null. Parameter name: source'
Stack trace: at System.Linq.ThrowHelper.ThrowArgumentNullException(ExceptionArgument argument) at System.Linq.Enumerable.TryGetFirst[TSource](IEnumerable1 source, Boolean& found)
     at lambda_method(Closure , QueryContext , TransparentIdentifier2 ) at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.Internal.ProjectionShaper.TypedProjectionShaper3.Shape(QueryContext queryContext, ValueBuffer& valueBuffer)
     at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.Internal.ProjectionShaper.TypedProjectionShaper3.Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.Internal.IShaper.Shape(QueryContext queryContext, ValueBuffer& valueBuffer) at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable1.Enumerator.BufferlessMoveNext(DbContext _, Boolean buffer)
     at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func3 operation, Func3 verifySucceeded)
     at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable1.Enumerator.MoveNext() at Microsoft.EntityFrameworkCore.Query.Internal.QueryBuffer.CorrelateSubquery[TInner,TOut,TCollection](Int32 correlatedCollectionId, INavigation navigation, Func2 resultCollectionFactory, MaterializedAnonymousObject& outerKey, Boolean tracking, Func1 correlatedCollectionFactory, Func3 correlationPredicate)
     at lambda_method(Closure , ValueBuffer )
     at System.Linq.Enumerable.SelectEnumerableIterator2.MoveNext() at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.ExceptionInterceptor1.EnumeratorExceptionInterceptor.MoveNext()
     at System.Collections.Generic.List1..ctor(IEnumerable1 collection)
     at System.Linq.Enumerable.ToList[TSource](IEnumerable1 source) at EPQT.ServiceLayer.TesterService.GetTestData(DateTime startDate, DateTime endDate, String testerId, String productFilter) in C:\Visual Studio 2017\Projects\Extractor\Core\EPQT.ServiceLayer\TesterService.cs:line 120 at EPQT.Modules.TesterQuery.Views.TestQueryViewModel.<>c__DisplayClass54_0.b__0() in C:\Visual Studio 2017\Projects\Extractor\Modules\EPQT.Modules.TesterQuery\Views\TestQueryViewModel.cs:line 218 at System.Threading.Tasks.Task.InnerInvoke() at System.Threading.Tasks.Task.<>c.<.cctor>b__277_0(Object obj) at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)

Further technical details

EF Core version: 2.2.4
Database Provider: Microsoft.EntityFrameworkCore.SqlServer
Operating system: Windows 10
IDE: Visual studio 2019

EF Code files - scaffold generated files.

EFCode_NonWorking.zip
EFCode_Working.zip

[Gaz83]

ok so I got the query working by changing

UpperLimit = y.StepNumericlimit1.FirstOrDefault().StepNumericlimit2.FirstOrDefault().HighLimit.ToString()

to this

UpperLimit = y.StepNumericlimit1.Select(z => z.StepNumericlimit2.FirstOrDefault().HighLimit).FirstOrDefault().ToString()

[smitpatel]

Query falls into client eval for projection. And
y.StepNumericlimit1.FirstOrDefault().StepNumericlimit2.FirstOrDefault().HighLimit.ToString()
will throw above exception on client side eval when first FirstOrDefault returns null. (even after applying null safety of navigation)