Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_: added tests for updating fees when after router provides the best route #5795

Open
wants to merge 8 commits into
base: develop
Choose a base branch
from

Conversation

saledjenic
Copy link
Contributor

This PR introduces anvil network on the status-go side and tests:

  • subscribing/unsubscribing for fees updates
  • receiving fees updates

Copy link

github-actions bot commented Sep 3, 2024

Thank you for opening this pull request!

Looks like you have BREAKING CHANGES in your PR.
Please make sure to update status-desktop and status-mobile clients accordingly.

@status-im-auto
Copy link
Member

status-im-auto commented Sep 3, 2024

Jenkins Builds

Click to see older builds (99)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 6d2b162 #1 2024-09-03 14:43:29 ~2 min tests-rpc 📄log
✔️ 6d2b162 #1 2024-09-03 14:44:48 ~4 min linux 📦zip
✔️ 6d2b162 #1 2024-09-03 14:45:20 ~4 min ios 📦zip
✔️ 6d2b162 #1 2024-09-03 14:46:12 ~5 min android 📦aar
✔️ 6d2b162 #1 2024-09-03 15:12:52 ~32 min tests 📄log
✔️ 552b757 #2 2024-09-04 06:28:42 ~1 min android 📦aar
✔️ 552b757 #2 2024-09-04 06:29:13 ~2 min linux 📦zip
✔️ 552b757 #2 2024-09-04 06:29:31 ~2 min tests-rpc 📄log
✔️ 552b757 #2 2024-09-04 06:30:08 ~3 min ios 📦zip
✔️ 552b757 #2 2024-09-04 06:59:05 ~31 min tests 📄log
✔️ d52622f #3 2024-09-04 06:36:13 ~1 min android 📦aar
✔️ d52622f #3 2024-09-04 06:36:43 ~1 min linux 📦zip
✔️ d52622f #3 2024-09-04 06:37:18 ~2 min tests-rpc 📄log
✔️ d52622f #3 2024-09-04 06:37:26 ~2 min ios 📦zip
✔️ d52622f #3 2024-09-04 07:31:31 ~32 min tests 📄log
✖️ db02a14 #4 2024-09-04 10:40:48 ~1 min tests 📄log
✔️ db02a14 #4 2024-09-04 10:41:14 ~1 min android 📦aar
✔️ db02a14 #4 2024-09-04 10:41:43 ~2 min linux 📦zip
✔️ db02a14 #4 2024-09-04 10:42:32 ~2 min tests-rpc 📄log
✔️ db02a14 #4 2024-09-04 10:42:56 ~3 min ios 📦zip
✖️ 257a72b #5 2024-09-04 11:14:02 ~51 sec tests 📄log
✔️ 257a72b #5 2024-09-04 11:14:41 ~1 min android 📦aar
✔️ 257a72b #5 2024-09-04 11:15:16 ~2 min linux 📦zip
✔️ 257a72b #5 2024-09-04 11:15:32 ~2 min tests-rpc 📄log
✔️ 257a72b #5 2024-09-04 11:16:08 ~3 min ios 📦zip
✔️ 0a4156b #6 2024-09-04 12:01:03 ~1 min android 📦aar
✔️ 0a4156b #6 2024-09-04 12:01:33 ~2 min linux 📦zip
✖️ 0a4156b #6 2024-09-04 12:01:49 ~2 min tests 📄log
✔️ 0a4156b #6 2024-09-04 12:01:55 ~2 min tests-rpc 📄log
✔️ 0a4156b #6 2024-09-04 12:03:11 ~3 min ios 📦zip
✔️ f785e06 #7 2024-09-04 14:33:30 ~1 min android 📦aar
✖️ f785e06 #7 2024-09-04 14:33:44 ~1 min tests-rpc 📄log
✔️ f785e06 #7 2024-09-04 14:35:08 ~3 min ios 📦zip
✔️ f785e06 #7 2024-09-04 14:36:12 ~4 min linux 📦zip
✔️ f785e06 #7 2024-09-04 15:02:50 ~30 min tests 📄log
✔️ 6129a75 #8 2024-09-04 17:25:10 ~1 min android 📦aar
✔️ 6129a75 #8 2024-09-04 17:25:45 ~2 min linux 📦zip
✔️ 6129a75 #8 2024-09-04 17:25:54 ~2 min tests-rpc 📄log
✔️ 6129a75 #8 2024-09-04 17:26:39 ~3 min ios 📦zip
✖️ 6129a75 #8 2024-09-04 17:54:31 ~30 min tests 📄log
✔️ a0869cc #9 2024-09-04 17:35:02 ~1 min android 📦aar
✔️ a0869cc #9 2024-09-04 17:35:37 ~2 min linux 📦zip
✔️ a0869cc #9 2024-09-04 17:35:56 ~2 min tests-rpc 📄log
✔️ a0869cc #9 2024-09-04 17:36:54 ~3 min ios 📦zip
✔️ a0869cc #9 2024-09-04 18:26:05 ~31 min tests 📄log
✔️ 7b1795c #10 2024-09-05 07:08:24 ~2 min android 📦aar
✔️ 7b1795c #10 2024-09-05 07:09:44 ~3 min tests-rpc 📄log
✔️ 7b1795c #10 2024-09-05 07:10:20 ~4 min ios 📦zip
✔️ 7b1795c #10 2024-09-05 07:10:49 ~4 min linux 📦zip
✔️ 7b1795c #10 2024-09-05 07:37:49 ~31 min tests 📄log
✔️ 9b09420 #11 2024-09-05 07:27:19 ~1 min android 📦aar
✔️ 9b09420 #11 2024-09-05 07:28:06 ~2 min linux 📦zip
✔️ 9b09420 #11 2024-09-05 07:28:37 ~2 min tests-rpc 📄log
✔️ 9b09420 #11 2024-09-05 07:28:45 ~2 min ios 📦zip
✔️ 9b09420 #11 2024-09-05 08:09:06 ~31 min tests 📄log
✔️ a359cd6 #12 2024-09-05 08:12:56 ~1 min tests-rpc 📄log
✔️ a359cd6 #12 2024-09-05 08:13:04 ~1 min android 📦aar
✔️ a359cd6 #12 2024-09-05 08:13:36 ~1 min linux 📦zip
✔️ a359cd6 #12 2024-09-05 08:14:28 ~2 min ios 📦zip
✖️ a359cd6 #2 2024-09-05 08:22:42 ~1 min tests-anvil 📄log
✔️ a359cd6 #12 2024-09-05 08:42:32 ~30 min tests 📄log
✔️ 0206dff #13 2024-09-05 08:43:27 ~1 min android 📦aar
✔️ 0206dff #13 2024-09-05 08:43:59 ~2 min linux 📦zip
✔️ 0206dff #13 2024-09-05 08:44:21 ~2 min tests-rpc 📄log
✔️ 0206dff #13 2024-09-05 08:44:55 ~3 min ios 📦zip
✔️ 0206dff #13 2024-09-05 09:13:09 ~30 min tests 📄log
✔️ d0e6e66 #14 2024-09-05 08:51:41 ~1 min android 📦aar
✔️ d0e6e66 #14 2024-09-05 08:52:12 ~2 min linux 📦zip
✔️ d0e6e66 #14 2024-09-05 08:52:34 ~2 min tests-rpc 📄log
✔️ d0e6e66 #14 2024-09-05 08:52:55 ~2 min ios 📦zip
✔️ d0e6e66 #14 2024-09-05 09:43:53 ~30 min tests 📄log
✖️ 5f7184b #15 2024-09-05 09:24:16 ~1 min tests-rpc 📄log
✖️ 5f7184b #5 2024-09-05 09:24:23 ~17 sec tests-anvil 📄log
✔️ 5f7184b #15 2024-09-05 09:24:23 ~1 min android 📦aar
✖️ 5f7184b #6 2024-09-05 09:24:59 ~16 sec tests-anvil 📄log
✔️ 5f7184b #15 2024-09-05 09:25:59 ~3 min ios 📦zip
✔️ 5f7184b #15 2024-09-05 09:26:26 ~3 min linux 📦zip
✖️ 29ed647 #7 2024-09-05 09:28:31 ~16 sec tests-anvil 📄log
✔️ 29ed647 #16 2024-09-05 09:29:43 ~1 min android 📦aar
✔️ 29ed647 #16 2024-09-05 09:30:10 ~1 min linux 📦zip
✔️ 29ed647 #16 2024-09-05 09:30:55 ~2 min tests-rpc 📄log
✔️ 29ed647 #16 2024-09-05 09:31:14 ~3 min ios 📦zip
✔️ 29ed647 #15 2024-09-05 10:14:36 ~30 min tests 📄log
✖️ 2ae40f8 #8 2024-09-05 09:46:43 ~9.8 sec tests-anvil 📄log
✔️ 2ae40f8 #17 2024-09-05 09:48:07 ~1 min android 📦aar
✔️ 2ae40f8 #17 2024-09-05 09:48:40 ~2 min linux 📦zip
✔️ 2ae40f8 #17 2024-09-05 09:48:41 ~2 min tests-rpc 📄log
✔️ 2ae40f8 #17 2024-09-05 09:49:41 ~3 min ios 📦zip
✖️ 2ae40f8 #9 2024-09-05 09:50:56 ~8.8 sec tests-anvil 📄log
✔️ b0ecc04 #18 2024-09-05 09:58:57 ~1 min tests-rpc 📄log
✔️ b0ecc04 #18 2024-09-05 09:59:27 ~1 min android 📦aar
✔️ b0ecc04 #18 2024-09-05 09:59:41 ~1 min linux 📦zip
✔️ b0ecc04 #18 2024-09-05 10:00:46 ~2 min ios 📦zip
✖️ a34efcd #11 2024-09-05 09:59:49 ~47 sec tests-anvil 📄log
✔️ a34efcd #19 2024-09-05 10:00:12 ~1 min tests-rpc 📄log
✔️ a34efcd #19 2024-09-05 10:01:18 ~1 min android 📦aar
✔️ a34efcd #19 2024-09-05 10:01:53 ~1 min linux 📦zip
✔️ a34efcd #19 2024-09-05 10:04:10 ~3 min ios 📦zip
✖️ a34efcd #16 2024-09-05 10:15:54 ~1 min tests 📄log
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 0867efd #20 2024-09-05 12:32:04 ~1 min android 📦aar
✔️ 0867efd #20 2024-09-05 12:32:31 ~2 min linux 📦zip
✔️ 0867efd #20 2024-09-05 12:32:34 ~2 min tests-rpc 📄log
✔️ 0867efd #20 2024-09-05 12:33:34 ~3 min ios 📦zip
✔️ 0867efd #17 2024-09-05 13:01:36 ~31 min tests 📄log
✔️ 4d1a48f #21 2024-09-05 13:05:02 ~2 min tests-rpc 📄log
✔️ 4d1a48f #21 2024-09-05 13:05:07 ~2 min linux 📦zip
✔️ 4d1a48f #21 2024-09-05 13:05:56 ~3 min ios 📦zip
✔️ 4d1a48f #21 2024-09-05 13:08:20 ~5 min android 📦aar
✔️ 4d1a48f #18 2024-09-05 13:34:52 ~31 min tests 📄log
✔️ 4d1a48f #22 2024-09-11 11:54:39 ~2 min tests-rpc 📄log
✔️ 4d1a48f #22 2024-09-11 11:56:13 ~4 min ios 📦zip
✔️ 4d1a48f #22 2024-09-11 11:56:20 ~4 min linux 📦zip
✔️ 4d1a48f #22 2024-09-11 11:56:31 ~4 min android 📦aar
✔️ 4d1a48f #19 2024-09-11 12:25:52 ~33 min tests 📄log

api/backend_test.go Show resolved Hide resolved

closeFn := func() {
close(suggestedRoutesCh)
signal.SetMobileSignalHandler(nil)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for keeping an eye on the reset of MobileSignalHandler ❤️

I recently added a ResetMobileSignalHandler, please checkout this

TLDR: Just add t.Cleanup(signal.ResetMobileSignalHandler) instead, it's a bit simpler.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

Comment on lines 71 to 72
var allGeneratedPaths []string
allGeneratedPaths = append(allGeneratedPaths, pathEIP1581Root, pathEIP1581Chat, pathWalletRoot, pathDefaultWalletAccount)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔

Suggested change
var allGeneratedPaths []string
allGeneratedPaths = append(allGeneratedPaths, pathEIP1581Root, pathEIP1581Chat, pathWalletRoot, pathDefaultWalletAccount)
allGeneratedPaths := []string{pathEIP1581Root, pathEIP1581Chat, pathWalletRoot, pathDefaultWalletAccount}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

err = backend.StartNodeWithAccountAndInitialConfig(account, password, *defaultSettings, nodeConfig, profileKeypair.Accounts, nil)
require.NoError(t, err)

return backend, stop, nil
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here for stop, using t.Cleanup help

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

@saledjenic saledjenic force-pushed the router-updates-tests-1 branch 2 times, most recently from 552b757 to d52622f Compare September 4, 2024 06:34
@saledjenic
Copy link
Contributor Author

@igor-sirotin thanks for the review, and comments addressed.

@igor-sirotin
Copy link
Collaborator

Tests seem to be broken, investigating

Copy link
Collaborator

@igor-sirotin igor-sirotin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't take this personally, just to ensure it's not merged while the tests are failing 😄
Details: #5797

@saledjenic
Copy link
Contributor Author

Don't take this personally, just to ensure it's not merged while the tests are failing 😄 Details: #5797

Yes, I know that. Discussed with Anton, already. :)

@igor-sirotin
Copy link
Collaborator

Accidents happen, you know 😄

@saledjenic
Copy link
Contributor Author

@igor-sirotin @antdanchenko please take a look at the tests now.
I moved updates TestRouterFeesUpdate to another package anviltests. We can use that package for all integration tests that need anvil running, and also I excluded that package from running unit tests. Also added run-integration-anvil to makefile, and what we need now is just to add that call to CI.

…as router v2

Breaking change!

Since the old router logic is removed, there is no need to have files and structs marked as v2.

Renamed endpoints:
- `GetSuggestedRoutesV2` to `GetSuggestedRoutes`
- `GetSuggestedRoutesV2Async` to `GetSuggestedRoutesAsync`
- `StopSuggestedRoutesV2AsyncCalcualtion` to `StopSuggestedRoutesAsyncCalcualtion`
It's a breaking change due to errors' changes, the list of mapped old error codes:
- `"WR-001"` is now `"WRR-001"`
- `"WR-002"` is now `"WRR-002"`
- `"WR-003"` is now `"WRR-003"`
- `"WR-004"` is now `"WRR-004"`
- `"WR-005"` is now `"WRR-005"`
- `"WR-006"` is now `"WRR-006"`
- `"WR-007"` is now `"WRR-007"`
- `"WR-008"` is now `"WRR-008"`
- `"WR-009"` is now `"WRR-009"`
- `"WR-010"` is now `"WRR-010"`
- `"WR-011"` is now `"WRR-011"`
- `"WR-012"` is now `"WRR-012"`
- `"WR-013"` is now `"WRR-013"`
- `"WR-014"` is now `"WRR-014"`
- `"WR-015"` is now `"WRR-015"`
- `"WR-019"` is now `"WRR-016"`
- `"WR-020"` is now `"WRR-017"`
- `"WR-021"` is now `"WRR-018"`
- `"WR-025"` is now `"WRR-019"`

- `"WR-016"` is now `"WR-001"`
- `"WR-017"` is now `"WR-002"`
- `"WR-018"` is now `"WR-003"`
- `"WR-022"` is now `"WR-004"`
- `"WR-023"` is now `"WR-005"`
- `"WR-024"` is now `"WR-006"`
- `"WR-026"` is now `"WR-007"`
- `"WR-027"` is now `"WR-008"`

Other changes:
- `RouteInputParams` type moved to `requests` package and code updated accordingly
- `SuggestedFees` type moved to a new `fees` package and code updated accordingly
- `SendType` type moved to a new `sendtype` package and code updated accordingly
- the following functions moved to `common` package
  - `ArrayContainsElement`
  - `ArraysWithSameElements`
  - `SameSingleChainTransfer`
  - `CopyMapGeneric`
  - `GweiToEth`
  - `WeiToGwei`
- the following consts moved to `common` package
  - `HexAddressLength`
  - `SupportedNetworks`
  - `SupportedTestNetworks`
Following the approach we did for keeping requests at the same location, these changes introduce
new location for responses. `SuggestedRoutesResponse` is moved there and renamed to
`RouterSuggestedRoutes` and code is updated accordingly.

New type `Route` defined (since a single route is composed of zero or more paths).

Types `Route`, `Path`, `Graph` and `Node` belong to a new `routs` package now.
Copy link
Member

@jakubgs jakubgs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_assets/ci/Jenkinsfile.tests-anvil is awfully similar to _assets/ci/Jenkinsfile.tests-rpc. But I'm assuming it's going to change in the future.

@@ -3,7 +3,7 @@ services:
image: ghcr.io/foundry-rs/foundry:latest
platform: linux/amd64
command:
- anvil --host 0.0.0.0
- anvil --host 0.0.0.0 --block-time 2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't need this change for integration-tests

- ../:/go/src/github.com/status-im/status-go/
working_dir: /go/src/github.com/status-im/status-go/api
command: |
go test -tags gowaku_skip_migrations -run ^TestRouterFeesUpdate github.com/status-im/status-go/api/... -v -- -anvil-host=http://anvil:8545
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right now we are launching a single test here, would be great to replace it with launching all go tests that require anvil if possible

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

@@ -499,3 +499,6 @@ run-integration-tests:
run-anvil: SHELL := /bin/sh
run-anvil:
docker-compose -f integration-tests/docker-compose.anvil.yml up --remove-orphans

run-integration-anvil:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI run won't finish because anvil will continue to run forever, we need to do something similar to https://github.com/status-im/status-go/blob/develop/Makefile#L492 with running silently and then reading exit code from the container with tests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

@igor-sirotin
Copy link
Collaborator

igor-sirotin commented Sep 5, 2024

I probably should have looked at this closer in the very beginning, but better to speak late than never 🙂
I think we're a bit messing up things here.

  1. I don't think that making defaultSettings and ensureDBOpened public is a good idea.
    ensureDBOpened is for sure a low-level API, it should be kept private. Should not be used in integrations tests.

    The fact that we have to make them public for this is a sign of that we're doing something wrong.

  2. The test itself if ~50 lines of code at the end of TestRouterFeesUpdate.
    Yet we have 300 more lines just to set it up.

    I believe this is also a sign of that we're doing something wrong.

  3. Can't we implement TestRouterFeesUpdate with the integration-tests that we already have?
    It even imports the same seed phrase 😄
    Looks like it would be much simpler to implement it that way.

cc @antdanchenko @saledjenic @status-im/status-go-guild


These are some major questions I have.
I still have minor concerns for the current implementation itself, I will write them if we decided to keep everything as is.

@saledjenic
Copy link
Contributor Author

I don't think that making defaultSettings and ensureDBOpened public is a good idea.
ensureDBOpened is for sure a low-level API, it should be kept private. Should not be used in integrations tests.

Exactly, that's what I said to you in private chat, we have to sort that out a bit better cause defaultSettings, defaultNodeConfig and other things are used in the same/similar form in many places in statusgo and I could copy them, but I didn't want to have one more place with those functions, and made them public, with the hope to have better organization and reusable function there, soon.

The test itself if ~50 lines of code at the end of TestRouterFeesUpdate.
Yet we have 300 more lines just to set it up.
I believe this is also a sign of that we're doing something wrong.

Yes, looks like that, but we need rpc.Client set with Anvil for those tests, and in this particular case subscribeForUdates and unsubscribeFeesUpdateAccrossAllChains are private to router package, not exposed and can be tests only via SuggestedRoutesAsync and SuggestedRoutes calls if the route is generated. In order to generate the route we need all other things to be set, like settings, node config... that's where other 300 lines came from.

Can't we implement TestRouterFeesUpdate with the integration-tests that we already have?
It even imports the same seed phrase 😄
Looks like it would be much simpler to implement it that way.

I am not sure that we can, cause those integration-tests are written in python and they can call only exposed endpoints (we cannot test just updates, but the router, and that's maybe ok), but I am not sure if it's possible to connect to signal that is sent when fees are recacluated? Guess not.

The point of having them this way is that we will add more tests later here that will test the router, sending transaction process and that way we can get rid of if params.TestsMode checks that we temporarily introduced. Maybe for this one we can use integration-tests (not sure @antdanchenko ?) but we need to be sure that rpc.Client will be set correctly, also a full control over base fees, gas estimation, balances per different chains and so on in order to compare a real with the expected result.


I agree that we should think about this, so no rush in making a decision. This PR can stay like this, at least as a PoC, till we figure out what's the best approach to proceed with.

@igor-sirotin thanks for the review, there is a long way in front of us till we set everything properly.

@igor-sirotin
Copy link
Collaborator

We agreed to postpone this PR.

The main reason why we can't implement this with existing integration tests, is the disability of getting events from status-go. I opened an issue about it here:

It shouldn't be difficult to implement, I will take a look at this ASAP.

cc @saledjenic @antdanchenko

@igor-sirotin
Copy link
Collaborator

igor-sirotin commented Sep 7, 2024

Here's the PR with required functionality implemented:

@saledjenic saledjenic force-pushed the router-clean-up branch 2 times, most recently from cbb74b7 to 900ad9b Compare September 10, 2024 07:32
Base automatically changed from router-clean-up to develop September 11, 2024 11:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants