Support Jagged and Multidimensional Arrays (Cosmos DB)

[calebeno]

I've just started working with the entity framework and am very pleased to find a solid ORM in C# land. I have an Azure Function app set up with the Cosmos DB provider. While many types are supported out of the box, I encountered an issue when using Jagged and Multidimensional Arrays (either [][] or [,] respectively).

I am currently using the 6.0.0 version of Microsoft.EntityFrameworkCore.Cosmos in a net6.0 framework app.

System.Private.CoreLib: Exception while executing function: MyFunc. Microsoft.EntityFrameworkCore: The property 'MyProperty' is of type 'MyType[][]' which is not supported by the current database provider. Either change the property CLR type, or ignore the property using the '[NotMapped]' attribute or by using 'EntityTypeBuilder.Ignore' in 'OnModelCreating'.

My current solution is to serialize the array into a string on store and deserialize on retrieval. This is done like so in the model builder:

class MyType { }

class MyParentType
{
    MyType[][] MyProperty { get; set; }
}

// Within the OnModelCreating function
modelBuilder.Entity<MyParentType>()
    .Property(p => p.MyProperty).HasConversion(
        g => JsonConvert.SerializeObject(g),
        g => JsonConvert.DeserializeObject<MyType[][]>(g)
    );

I did try making an OwnsMany relationship just to see if that worked but it throws a different error:

// Within the OnModelCreating function
modelBuilder.Entity<MyParentType>()
    .OwnsMany(p => p.MyProperty);

System.Private.CoreLib: Exception while executing function: MyFunc. Microsoft.EntityFrameworkCore: The specified type 'MyType[]' must be a non-interface reference type to be used as an entity type.

In the context of Cosmos, Jagged and Multidimensional arrays should ideally be supported by default as the data structures in JSON already support this (both would render out to the same JSON and could be converted back to their original respective type). Quick example of json serialization for both types (for thoroughness):

var test = new string[2][];
test[0] = new string[2] { "my string test1-1", "my string test1-2" };
test[1] = new string[2] { "my string test2-1", "my string test2-2" };
log.LogInformation(JsonConvert.SerializeObject(test));
// [["my string test1-1","my string test1-2"],["my string test2-1","my string test2-2"]]

var test2 = new string[2,2]
{
    { "my second test1-1", "my second test1-2" },
    { "my second test2-1", "my second test2-2" }
};
log.LogInformation(JsonConvert.SerializeObject(test2));
// [["my string test1-1","my string test1-2"],["my string test2-1","my string test2-2"]]

Presumably, this could be extended to 3D and 4D multidimensional arrays as well but my case only needs 2d.

I would love to see these data types handled automatically instead of requiring conversion to a string in a future release of the Entity Framework Cosmos DB provider. Thank you for your time and consideration!

[roji]

While jagged arrays should be OK, does JSON (and therefore CosmosDB) support multidimensional arrays? I'm not too sure how that would work. We could certainly serialize a .NET multidimensional array to a JSON jagged array, but then deserialization would yield a .NET jagged array.

[calebeno]

JSON doesn't support multidimensional arrays as such. Both Multi and Jagged would serialize to the same JSON format:

[ [ ], [ ] ]

Would it be possible to deserialize based on the intended receiving type? As an example, deserializing the same 2D JSON string array works for both types using JsonConvert:

var arrayString = "[ [\"mystring1-1\", \"mystring1-2\"], [\"mystring2-1\", \"mystring2-2\"] ]";

var test = JsonConvert.DeserializeObject<string[][]>(arrayString);
log.LogInformation(test.ToString());         // System.String[][]
log.LogInformation(test[0][0]);              // mystring1-1

var test2 = JsonConvert.DeserializeObject<string[,]>(arrayString);
log.LogInformation(test2.ToString());        // System.String[,]
log.LogInformation(test2[1,1]);              // mystring2-2

[roji]

Would it be possible to deserialize based on the intended receiving type?

That may indeed be possible.

[ajcvickers]

/cc @AndriySvyryd

[LeaFrock]

I have a similar case, with a small difference about exception.

My table has a json column, which is like:

[{"option": ["as", "around"], "answer": ["4", "0"], "position": [[16, 6], [23, 2]]}]

And the code model is like:

public class MyTableEntity
{
    public List<QuestionContent> Value { get; set; }
}

public sealed class QuestionContent
{
    public string[] Option { get; set; }

    public string[] Answer { get; set; }

    public int[][] Position { get; set; }
}

When I want to use JSON column support in EF Core 8 like the following,

            entity.OwnsMany(e => e.Value, ob =>
            {
                ob.ToJson();
                ob.Property(c => c.Option)
                    .HasJsonPropertyName("option");
                ob.Property(c => c.Answer)
                    .HasJsonPropertyName("answer");
                ob.Property(c => c.Position)
                    .HasJsonPropertyName("position")
                    .HasConversion( 
                        v => JsonSerializer.Serialize(v, JsonSerializerOptions.Default),
                        v => JsonSerializer.Deserialize<int[][]>(v, JsonSerializerOptions.Default));
            });

an exception is thrown as,

System.InvalidOperationException: Cannot get the value of a token type 'StartArray' as a string.
   at System.Text.Json.ThrowHelper.ThrowInvalidOperationException_ExpectedString(JsonTokenType tokenType)
   at System.Text.Json.Utf8JsonReader.GetString()
   at Microsoft.EntityFrameworkCore.Storage.Json.JsonStringReaderWriter.FromJsonTyped(Utf8JsonReaderManager& manager, Object existingObject)
   at Microsoft.EntityFrameworkCore.Storage.Json.JsonConvertedValueReaderWriter`2.FromJsonTyped(Utf8JsonReaderManager& manager, Object existingObject)
   at lambda_method201(Closure, QueryContext, Object[], JsonReaderData)
   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.MaterializeJsonEntityCollection[TEntity,TResult](QueryContext queryContext, Object[] keyPropertyValues, JsonReaderData jsonReaderData, INavigationBase navigation, Func`4 innerShaper)
   at lambda_method200(Closure, QueryContext, DbDataReader, ResultContext, SingleQueryResultCoordinator)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)

  (Skipped......)

If I comment out the last HasConversion line of Position, the exception changes, which is the same as reported above.

And if I ignore the Position with ob.Ignore(c => c.Position), no exceptions throw. So I'm sure there’re some details here that I don't understand.

[roji]

@LeaFrock EF doesn't currently supported nested (jagged) arrays. That's what this issue (and #30713) track.