Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query support for nested collections of primitive types #30713

Open
Tracked by #30731
roji opened this issue Apr 18, 2023 · 2 comments
Open
Tracked by #30731

Query support for nested collections of primitive types #30713

roji opened this issue Apr 18, 2023 · 2 comments
Assignees
Labels
area-primitive-collections area-query consider-for-next-release punted-for-9.0 Originally planned for the EF Core 9.0 (EF9) release, but moved out due to resource constraints. type-enhancement
Milestone

Comments

@roji
Copy link
Member

roji commented Apr 18, 2023

#29427 allows mapping arbitrary collections of primitive types to JSON text columns, and #30426 allows querying them with arbitrary LINQ operators. This tracks mapping nested collections as well (e.g. int[][]).

Some ideas for query translations over nested collections:

-- Index to get a nested array, returns 3, 4
SELECT o.value FROM OPENJSON((
    SELECT i.value FROM OPENJSON('[[1, 2], [3, 4]]') AS i
    ORDER BY i.[key]
    OFFSET 1 ROWS FETCH NEXT 1 ROWS ONLY)) AS o;

-- Index to get a nested element, returns 4
SELECT o.value FROM OPENJSON((
    SELECT i.value FROM OPENJSON('[[1, 2], [3, 4]]') AS i
    ORDER BY i.[key]
    OFFSET 1 ROWS FETCH NEXT 1 ROWS ONLY)) AS o
ORDER BY o.[key]
OFFSET 1 ROWS FETCH NEXT 1 ROWS ONLY;

-- c.Nested.Where(n => n[0] == 1)
SELECT o.value FROM OPENJSON('[[1, 2], [3, 4]]') AS o
WHERE (
    SELECT i.value FROM OPENJSON(o.value) AS i
    ORDER BY i.[key]
    OFFSET 0 ROWS FETCH NEXT 1 ROWS ONLY) = 1;

-- Flatten (SelectMany), returns 1, 2, 3, 4
SELECT i.value FROM OPENJSON('[[1, 2], [3, 4]]') AS o
CROSS APPLY OPENJSON(o.value) AS i;

Note that IIRC PG doesn't support nested arrays (but does support multidimensional ones).

@roji roji mentioned this issue Apr 19, 2023
34 tasks
@ajcvickers ajcvickers added this to the Backlog milestone Apr 20, 2023
@ajcvickers ajcvickers self-assigned this Apr 1, 2024
@ajcvickers ajcvickers modified the milestones: Backlog, 9.0.0 Apr 1, 2024
ajcvickers added a commit that referenced this issue Apr 2, 2024
This change starts using the EF8 primitive collection infrastructure for primitive collections in Cosmos. This involves adding support for nested collections and read-only collections.

The main challenge here is supporting all the different collection types when, for example, `List<List<string>>` cannot be cast to `IEnumerable<IEnumerable<string>>`. To support this, we use exact collection types in snapshots, and we use non-generic methods in the nested comparers and serializers.

General dictionary mapping still not supported.
No extensive query testing done yet.

Model building and change tracking for #30713
Fixes #25364
Fixes #25343
Part of #4179
Fixes #31722
ajcvickers added a commit that referenced this issue Apr 3, 2024
This change starts using the EF8 primitive collection infrastructure for primitive collections in Cosmos. This involves adding support for nested collections and read-only collections.

The main challenge here is supporting all the different collection types when, for example, `List<List<string>>` cannot be cast to `IEnumerable<IEnumerable<string>>`. To support this, we use exact collection types in snapshots, and we use non-generic methods in the nested comparers and serializers.

General dictionary mapping still not supported.
No extensive query testing done yet.

Model building and change tracking for #30713
Fixes #25364
Fixes #25343
Part of #4179
Fixes #31722
ajcvickers added a commit that referenced this issue Apr 3, 2024
* Consolidate primitive collections across relational and Cosmos

This change starts using the EF8 primitive collection infrastructure for primitive collections in Cosmos. This involves adding support for nested collections and read-only collections.

The main challenge here is supporting all the different collection types when, for example, `List<List<string>>` cannot be cast to `IEnumerable<IEnumerable<string>>`. To support this, we use exact collection types in snapshots, and we use non-generic methods in the nested comparers and serializers.

General dictionary mapping still not supported.
No extensive query testing done yet.

Model building and change tracking for #30713
Fixes #25364
Fixes #25343
Part of #4179
Fixes #31722

* Remove dead files

* Updates based on review feedback
@ajcvickers
Copy link
Member

@roji I'm trying to figure out what work this is tracking for 9. I implemented model building and change tracking support for nested collections in #33456. It's possible I should have then assigned this to you to do query stuff. Are we still missing query support for nested collections? If so, everywhere, just in relational, or just in Cosmos? (If it's not working in Cosmos, then that's a regression, and we'll have to decide how to handle it. If it's not working in relational, then we should block nested collections in relational for 9.)

@ajcvickers ajcvickers changed the title Nested collections of primitive types Query support for nested collections of primitive types Jul 31, 2024
@ajcvickers ajcvickers assigned roji and unassigned ajcvickers Jul 31, 2024
@ajcvickers ajcvickers added punted-for-9.0 Originally planned for the EF Core 9.0 (EF9) release, but moved out due to resource constraints. consider-for-next-release and removed consider-for-current-release labels Jul 31, 2024
@ajcvickers ajcvickers modified the milestones: 9.0.0, Backlog Jul 31, 2024
@ajcvickers
Copy link
Member

Design meeting decision: leave as-is for Cosmos. Block nested collections on 9. See #34324.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-primitive-collections area-query consider-for-next-release punted-for-9.0 Originally planned for the EF Core 9.0 (EF9) release, but moved out due to resource constraints. type-enhancement
Projects
None yet
Development

No branches or pull requests

3 participants