[Indexer/TiDB][EASY] Add pre-requisites to README. #17686

amnn · 2024-05-13T15:09:18Z

Description

Explain what needs to be installed to get the MySQL version of the Indexer working.

Test plan

Try the steps and make sure they work.

vercel · 2024-05-13T15:09:22Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
sui-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 14, 2024 11:14am

3 Ignored Deployments

Name	Status	Preview	Updated (UTC)
multisig-toolkit	⬜️ Ignored (Inspect)	Visit Preview	May 14, 2024 11:14am
sui-kiosk	⬜️ Ignored (Inspect)	Visit Preview	May 14, 2024 11:14am
sui-typescript-docs	⬜️ Ignored (Inspect)	Visit Preview	May 14, 2024 11:14am

ronny-mysten

## Description Explain what needs to be installed to get the MySQL version of the Indexer working. ## Test plan Try the steps and make sure they work.

This table maps an object's ID and version to a checkpoint sequence number, in a table partitioned by the first byte of the object ID. This speeds up look ups into `objects_history` by offering a path for a first look-up to the correct partition in that table for a given object's ID and version. This PR introduces the table, and the logic to populate it in the indexer. ``` sui$ cargo nextest run -p sui-indexer sui$ cargo nextest run -p sui-graphql-rpc sui$ cargo nextest run -p sui-graphql-e2e-tests --features pg_integration ``` A future PR will make use of this table from GraphQL, which will test it further. - #17686 - #17687 - #17688 - #17689 - #17691 - #17694 - #17695 --- Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK:

) ## Description Recreating #17690 which was accidentally closed. Adding data to the `packages` table, to support the following GraphQL queries: ```graphql type Query { # Fetch all packages created strictly `afterCheckpoint` and strictly # before `beforeCheckpoint`. packages( afterCheckpoint: Int, beforeCheckpoint: Int, first: Int, after: String, last: Int, before: String, ): MovePackageConnection # Fetch all packages in the same family as the package at `address` # with versions strictly after `afterVersion` and strictly before # `beforeVersion`. packageVersions( address: SuiAddress!, afterVersion: Int, beforeVersion: Int, first: Int, after: String, last: Int, before: String, ): MovePackageConnection # Fetch a package by its address, and optionally supplying a # version. If the version is supplied, returns the package whose # Original ID matches the Original ID of the package at `address`, # but whose version is `version`, otherwise just fetches the package # directly. package(address: SuiAddress!, version: Int): MovePackage } type MovePackage { # Return the package whose Original ID matches this package's but # whose version matches `version`. If `version` is not supplied, # defaults to the latest version. atVersion(version: Int): MovePackage # Fetch all packages in the same family as this package, with # versions strictly after `afterVersion` and strictly before # `beforeVersion`. versions( afterVersion: Int, beforeVersion: Int, first: Int, after: String, last: Int, before: String, ): MovePackageConnection } ``` These queries are important for writing tools that perform whole-chain package analyses, and also for the .move Registry. ## Test plan Make sure nothing is broken: ``` sui$ cargo nextest run -p sui-indexer ``` Tests of new features will be included in a stacked change that uses these tables and indices in GraphQL. ## Stack - #17686 - #17687 - #17688 - #17689 - #17691 - #17694 - #17695 - #17542 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [x] Indexer: Adds the following fields: `packages.original_id`, `packages.package_version`, `packages.checkpoint_sequence_number` to support queries about package upgrades. - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK:

This table maps an object's ID and version to a checkpoint sequence number, in a table partitioned by the first byte of the object ID. This speeds up look ups into `objects_history` by offering a path for a first look-up to the correct partition in that table for a given object's ID and version. This PR introduces the table, and the logic to populate it in the indexer. ``` sui$ cargo nextest run -p sui-indexer sui$ cargo nextest run -p sui-graphql-rpc sui$ cargo nextest run -p sui-graphql-e2e-tests --features pg_integration ``` A future PR will make use of this table from GraphQL, which will test it further. - #17686 - #17687 - #17688 - #17689 - #17691 - #17694 - #17695 --- Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK:

## Description Use the `objects_version` table to speed up point look-ups (via data loaders) for historical objects (ID + version), and dynamic fields (object look-up bounding version by parent ID). With this change, the restriction of accessing dynamic fields only within the available range is dropped. ## Test plan ``` sui$ cargo nextest run -p sui-graphql-rpc sui$ cargo nextest run -p sui-graphql-e2e-tests --features pg_integration. ``` Perform a query that involves fetching a large number of dynamic fields, which should now be fast. The following example, fetching dynamic fields on a deepbook pool loads 50 dynamic fields in about 5s from cold (which also requires loading packages for resolution), and then 2s from there: ``` query { owner( address: "0x029170bfa0a1677054263424fe4f9960c7cf05d359f6241333994c8830772bdb" ) { dynamicFields { pageInfo { hasNextPage endCursor } nodes { name { type { repr } json } value { ... on MoveValue { type { repr } json } ... on MoveObject { contents { json type { repr } } } } } } } } ``` ## Stack - #17686 - #17687 - #17688 - #17689 - #17691 - #17694 - #17695 - #17542 - #17726 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [x] GraphQL: Dynamic fields can now be looked up on any historical object (not just objects in the available range). - [ ] CLI: - [ ] Rust SDK:

## Description Implement `Query.package` and `MovePackage.atVersion` to query a package at a specific version, using the new fields added to the `packages` table, exposed via some new data loaders. ## Test plan New transactional tests: ``` sui$ cargo nextest run -p sui-graphql-e2e-tests \ --features pg_integration \ -- packages/versioning ``` ## Stack - #17686 - #17687 - #17688 - #17689 - #17691 - #17694 - #17695 - #17542 - #17726 - #17543 - #17692 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [x] GraphQL: Introduce `Query.package` and `MovePackage.atVersion` to query packages at specific versions. - [ ] CLI: - [ ] Rust SDK:

## Description Add a new kind of package point look-up to get the latest version of the package at a given ID (or from another `MovePackage`). For system packages, this is analogous to getting the latest version of the object at that ID, but the versions of other packages all exist at different IDs. ## Test plan New transactional tests: ``` sui$ cargo nextest run -p sui-graphql-e2e-tests \ --features pg_integration \ -- packages/versioning ``` ## Stack - #17686 - #17687 - #17688 - #17689 - #17691 - #17694 - #17695 - #17542 - #17726 - #17543 - #17692 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [x] GraphQL: Add `Query.latestPackage` and `MovePackage.latest` for fetching the latest version of a package. - [ ] CLI: - [ ] Rust SDK:

## Description Adds a query, `Query.packages` for fetching all packages that were introduced within a given checkpoint range. Useful for fetching package contents in bulk, to do local analyses. ## Test plan New E2E tests: ``` sui$ cargo nextest run -p sui-graphql-e2e-tests \ --features pg_integration \ -- packages/versioning ``` Also tested for performance against a large read replica (the query planner quotes a high estimate for the query but the actual results do not take very long to run because queries on many sub-partitions are eliminated). ## Stack - #17686 - #17687 - #17688 - #17689 - #17691 - #17694 - #17695 - #17542 - #17726 - #17543 - #17692 - #17693 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [x] GraphQL: Introduces `Query.packages` for paginating through all packages (optionally bounding by the checkpoint the package was introduced in). - [ ] CLI: - [ ] Rust SDK:

## Description Introduce two new queries: `Query.packageVersions` and `MovePackage.versions` for iterating over all the different versions of a given package. This kind of query is useful for understanding package history. These were introduced as a separate query, instead of having a single query for iterating over packages that could optionally take a checkpoint bounds or version bounds because of how system packages interact with the `packages` table: Because system packages are updated in-place, they only have one row in the `packages` table. This makes sense for paginating packages in bulk (e.g. by checkpoint) where the primary aim is to get a snapshot of the packages available at a certain point in time, but doesn't work for answering package version queries for system packages, and it prevents us from creating a combined query. A combined query would also allow someone to create a filter that bounds checkpoints and versions, but doesn't bound the package itself (or would require us to prevent that combination), which is complicated to implement efficiently and not particularly useful. ## Test plan New E2E tests: ``` sui$ cargo nextest run -p sui-graphql-e2e-tests \ --features pg_integration \ -- packages/versioning ``` & Testing against a read replica to make sure system package tests work well, and performance is reasonable. ## Stack - #17686 - #17687 - #17688 - #17689 - #17691 - #17694 - #17695 - #17542 - #17690 - #17543 - #17692 - #17693 - #17696 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [x] GraphQL: Introduces `Query.packageVersions` and `MovePackage.versions` for paginating over the versions of a particular package. - [ ] CLI: - [ ] Rust SDK: