[pick][GraphQL] Object DataLoader (#17332) #18384
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implement data loaders for fetching historical object versions, objects
bounded by their parent versions, and the latest versions of an object
at a given checkpoint.
By implementing an Object DataLoader, we also implicitly get support for
data-loading all derived types (`MoveObject`, `MovePackage`,
`MoveModule`, `DynamicField`, `Coin`, etc).
These implementations (particularly historical queries and queries where
the version can be bounded by a parent version) can be made even more
efficient with the existence of an index/side table that maps an
object's ID and version to the checkpoint it is part of. This change has
not been included in this PR, but we will follow up on this as part of
Object query benchmarking.
As part of this change, I enabled queries for historical objects outside
the available range. Later (with the use of an `obj_version` index) it
will also be possible to enable dynamic field look-ups on historical
objects as well.
```
sui$ cargo nextest run -p sui-graphql-rpc
sui$ cargo nextest run -p sui-graphql-e2e-tests --features pg_integration
```
Run the following query -- after this change, it takes about 8s to
complete on the server, fetching about 80 objects, while previously it
would either timeout or squeak in *just* under the 40s timeout. I expect
this number to improve further once we have an efficient way to map
object ids and versions to a checkpoint sequence number.
```graphql
query {
transactionBlocks(last: 5) {
nodes {
effects {
objectChanges(first: 50) {
pageInfo { hasNextPage }
nodes {
idCreated
idDeleted
inputState { asMoveObject { contents { json } } }
outputState { asMoveObject { contents { json } } }
}
}
}
}
}
}
```
---
Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.
For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.
- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [x] GraphQL: Queries for historical versions of objects will now
return data even if that version of the object is outside the available
range.
- [ ] CLI:
- [ ] Rust SDK:
wlmyng
requested review from
amnn,
stefan-mysten,
emmazzz and
suiwombat
as code owners
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
3 Ignored Deployments
|
amnn
approved these changes
Jun 24, 2024
wlmyng
merged commit 9615c0b
into
releases/sui-graphql-rpc-v2024.4.0-release
47 of 48 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implement data loaders for fetching historical object versions, objects bounded by their parent versions, and the latest versions of an object at a given checkpoint.
By implementing an Object DataLoader, we also implicitly get support for data-loading all derived types (
MoveObject,MovePackage,MoveModule,DynamicField,Coin, etc).These implementations (particularly historical queries and queries where the version can be bounded by a parent version) can be made even more efficient with the existence of an index/side table that maps an object's ID and version to the checkpoint it is part of. This change has not been included in this PR, but we will follow up on this as part of Object query benchmarking.
As part of this change, I enabled queries for historical objects outside the available range. Later (with the use of an
obj_versionindex) it will also be possible to enable dynamic field look-ups on historical objects as well.Run the following query -- after this change, it takes about 8s to complete on the server, fetching about 80 objects, while previously it would either timeout or squeak in just under the 40s timeout. I expect this number to improve further once we have an efficient way to map object ids and versions to a checkpoint sequence number.
Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.
For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.
Description
Describe the changes or additions included in this PR.
Test plan
How did you test the new or updated feature?
Release notes
Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.
For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.