IndexOutOfRange thrown from In-Memory when using union on owned types

[jmgarnica]

While using in-memory db with an entity that has an owned entity the combination of filter + concat/union and select produces an IndexOutOfRange when requesting the results.

Project with minimun to reproduce the error here

Entities:

record Entity(int Id, OwnedEntity Owned)
{
    // for efcore
    private Entity() : this(default, new OwnedEntity(default))
    {
    }
}

record OwnedEntity(bool Property2);

Contex configuration

modelBuilder.Entity<Entity>(builder =>
{
    builder.HasKey(e => e.Id);
    builder.Property(e => e.Id)
        .ValueGeneratedOnAdd();

    builder.OwnsOne(e => e.Owned);
});

Query

Please note that this is a very simplified version of our production code, so query.Union(query) might not make sense at first but they have different filters, but with only this i can reproduce the error.

For the error to occur all of these have to be in the query:

• .Where(e => e.Owned.Property2) or any filter by a property of the owned entity
• query.Union(query) or Concat
• .Select(e => e.Owned.Property2) or any select group that includes a property of the owned entity

and the table must contain at least one element.

var query = context.Set<Entity>().Where(e => e.Owned.Property2);
var total = query.Union(query).Select(e => new {e.Id, e.Owned.Property2 });
var result = total.ToList(); // <- IndexOutOfRange here

Stack Traces

Unhandled exception. System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at lambda_method55(Closure, ValueBuffer, ValueBuffer)
   at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryQueryExpression.<>c__DisplayClass50_0`4.<LeftJoin>b__3(<>f__AnonymousType0`2 t, TInner i)
   at System.Linq.Enumerable.SelectManyIterator[TSource,TCollection,TResult](IEnumerable`1 source, Func`2 collectionSelector, Func`3 resultSelector)+MoveNext()
   at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
   at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.Enumerator.MoveNextHelper()
   at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.Enumerator.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)

efcore debug output

dbug: 25/10/2023 10:09:51.429 CoreEventId.QueryCompilationStarting[10111] (Microsoft.EntityFrameworkCore.Query)
      Compiling query expression:
      'DbSet<Entity>()
          .Where(e => e.Owned.Property2)
          .Union(DbSet<Entity>()
              .Where(e => e.Owned.Property2))
          .Select(e => e.Owned.Property2)'
dbug: 25/10/2023 10:09:51.542 CoreEventId.QueryExecutionPlanned[10107] (Microsoft.EntityFrameworkCore.Query)
      Generated query execution expression:
      'queryContext => new QueryingEnumerable<bool>(
          queryContext,
          InMemoryQueryExpression.LeftJoin<ValueBuffer, ValueBuffer, int, ValueBuffer>(
              outer: InMemoryQueryExpression.LeftJoin<ValueBuffer, ValueBuffer, int, ValueBuffer>(
                  outer: InMemoryShapedQueryCompilingExpressionVisitor.Table(
                      queryContext: queryContext,
                      entityType: EntityType: Entity),
                  inner: InMemoryShapedQueryCompilingExpressionVisitor.Table(
                      queryContext: queryContext,
                      entityType: EntityType: OwnedEntity Owned),
                  outerKeySelector: valueBuffer => ExpressionExtensions.ValueBufferTryReadValue<int>(
                      valueBuffer: valueBuffer,
                      index: 0,
                      property: Property: Entity.Id (int) Required PK AfterSave:Throw ValueGenerated.OnAdd),
                  innerKeySelector: valueBuffer => ExpressionExtensions.ValueBufferTryReadValue<int>(
                      valueBuffer: valueBuffer,
                      index: 0,
                      property: Property: OwnedEntity.EntityId (no field, int) Shadow Required PK FK AfterSave:Throw),
                  resultSelector: (outer, inner) => new ValueBuffer(new object[]
                  {
                      (object)ExpressionExtensions.ValueBufferTryReadValue<int>(
                          valueBuffer: outer,
                          index: 0,
                          property: Property: Entity.Id (int) Required PK AfterSave:Throw ValueGenerated.OnAdd),
                      (object)ExpressionExtensions.ValueBufferTryReadValue<int?>(
                          valueBuffer: inner,
                          index: 0,
                          property: Property: OwnedEntity.EntityId (no field, int) Shadow Required PK FK AfterSave:Throw),
                      (object)ExpressionExtensions.ValueBufferTryReadValue<bool?>(
                          valueBuffer: inner,
                          index: 1,
                          property: Property: OwnedEntity.Property2 (bool) Required)
                  }),
                  defaultValue: ValueBuffer,
                  comparer: null)
                  .Where(valueBuffer => ExpressionExtensions.ValueBufferTryReadValue<bool?>(
                      valueBuffer: valueBuffer,
                      index: 2,
                      property: Property: OwnedEntity.Property2 (bool) Required) == True)
                  .Select(valueBuffer => new ValueBuffer(new object[]{ (object)ExpressionExtensions.ValueBufferTryReadValue<int>(
                      valueBuffer: valueBuffer,
                      index: 0,
                      property: Property: Entity.Id (int) Required PK AfterSave:Throw ValueGenerated.OnAdd) }))
                  .Union(InMemoryQueryExpression.LeftJoin<ValueBuffer, ValueBuffer, int, ValueBuffer>(
                      outer: InMemoryShapedQueryCompilingExpressionVisitor.Table(
                          queryContext: queryContext,
                          entityType: EntityType: Entity),
                      inner: InMemoryShapedQueryCompilingExpressionVisitor.Table(
                          queryContext: queryContext,
                          entityType: EntityType: OwnedEntity Owned),
                      outerKeySelector: valueBuffer => ExpressionExtensions.ValueBufferTryReadValue<int>(
                          valueBuffer: valueBuffer,
                          index: 0,
                          property: Property: Entity.Id (int) Required PK AfterSave:Throw ValueGenerated.OnAdd),
                      innerKeySelector: valueBuffer => ExpressionExtensions.ValueBufferTryReadValue<int>(
                          valueBuffer: valueBuffer,
                          index: 0,
                          property: Property: OwnedEntity.EntityId (no field, int) Shadow Required PK FK AfterSave:Throw),
                      resultSelector: (outer, inner) => new ValueBuffer(new object[]
                      {
                          (object)ExpressionExtensions.ValueBufferTryReadValue<int>(
                              valueBuffer: outer,
                              index: 0,
                              property: Property: Entity.Id (int) Required PK AfterSave:Throw ValueGenerated.OnAdd),
                          (object)ExpressionExtensions.ValueBufferTryReadValue<int?>(
                              valueBuffer: inner,
                              index: 0,
                              property: Property: OwnedEntity.EntityId (no field, int) Shadow Required PK FK AfterSave:Throw),
                          (object)ExpressionExtensions.ValueBufferTryReadValue<bool?>(
                              valueBuffer: inner,
                              index: 1,
                              property: Property: OwnedEntity.Property2 (bool) Required)
                      }),
                      defaultValue: ValueBuffer,
                      comparer: null)
                      .Where(valueBuffer => ExpressionExtensions.ValueBufferTryReadValue<bool?>(
                          valueBuffer: valueBuffer,
                          index: 2,
                          property: Property: OwnedEntity.Property2 (bool) Required) == True)
                      .Select(valueBuffer => new ValueBuffer(new object[]{ (object)ExpressionExtensions.ValueBufferTryReadValue<int>(
                          valueBuffer: valueBuffer,
                          index: 0,
                          property: Property: Entity.Id (int) Required PK AfterSave:Throw ValueGenerated.OnAdd) }))),
              inner: InMemoryShapedQueryCompilingExpressionVisitor.Table(
                  queryContext: queryContext,
                  entityType: EntityType: OwnedEntity Owned),
              outerKeySelector: valueBuffer => ExpressionExtensions.ValueBufferTryReadValue<int>(
                  valueBuffer: valueBuffer,
                  index: 0,
                  property: Property: Entity.Id (int) Required PK AfterSave:Throw ValueGenerated.OnAdd),
              innerKeySelector: valueBuffer => ExpressionExtensions.ValueBufferTryReadValue<int>(
                  valueBuffer: valueBuffer,
                  index: 0,
                  property: Property: OwnedEntity.EntityId (no field, int) Shadow Required PK FK AfterSave:Throw),
              resultSelector: (outer, inner) => new ValueBuffer(new object[]
              {
                  (object)ExpressionExtensions.ValueBufferTryReadValue<int>(
                      valueBuffer: outer,
                      index: 0,
                      property: Property: Entity.Id (int) Required PK AfterSave:Throw ValueGenerated.OnAdd),
                  (object)ExpressionExtensions.ValueBufferTryReadValue<int?>(
                      valueBuffer: outer,
                      index: 1,
                      property: Property: OwnedEntity.EntityId (no field, int) Shadow Required PK FK AfterSave:Throw),
                  (object)ExpressionExtensions.ValueBufferTryReadValue<bool?>(
                      valueBuffer: outer,
                      index: 2,
                      property: Property: OwnedEntity.Property2 (bool) Required),
                  (object)ExpressionExtensions.ValueBufferTryReadValue<int?>(
                      valueBuffer: inner,
                      index: 0,
                      property: Property: OwnedEntity.EntityId (no field, int) Shadow Required PK FK AfterSave:Throw),
                  (object)ExpressionExtensions.ValueBufferTryReadValue<bool?>(
                      valueBuffer: inner,
                      index: 1,
                      property: Property: OwnedEntity.Property2 (bool) Required)
              }),
              defaultValue: ValueBuffer,
              comparer: null)
              .Select(valueBuffer => new ValueBuffer(new object[]{ (object)ExpressionExtensions.ValueBufferTryReadValue<bool?>(
                  valueBuffer: valueBuffer,
                  index: 4,
                  property: Property: OwnedEntity.Property2 (bool) Required) }))
              .Select(valueBuffer => new ValueBuffer(new object[]{ (object)ExpressionExtensions.ValueBufferTryReadValue<bool?>(
                  valueBuffer: valueBuffer,
                  index: 0,
                  property: Property: OwnedEntity.Property2 (bool) Required) })),
          Func<QueryContext, ValueBuffer, bool>,
          Context,
          False,
          True
      )'

Version information

• net8.0:

  • Microsoft.EntityFrameworkCore Version=8.0.0-rc.2.23480.1
  • Microsoft.EntityFrameworkCore.InMemory Version=8.0.0-rc.2.23480.1

• net7.0:

  • Microsoft.EntityFrameworkCore Version=7.0.13
  • Microsoft.EntityFrameworkCore.InMemory Version=7.0.13

[ajcvickers]

We recommend against using the in-memory provider for testing--see Testing EF Core Applications. While we have no plans to remove the in-memory provider, we will not be adding any new features to this provider because we believe valuable development time is better spent in other areas. When feasible, we plan to still fix regressions in existing behavior.