GETH - filter not found

[maaktweluit]

Phillip, Ubuntu 20.04 , Latest version

    ~/yajsapi/examples(b0.3*) » yagna payment fund --driver erc20                                                                                                                                130 ↵ phillip@phillip

    Error: Library error: Mailbox has closed

    [2021-02-09T11:31:23Z INFO  actix_server::builder] Starting "actix-web-service-127.0.0.1:7465" service on 127.0.0.1:7465
    thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: EthereumClientError(Rpc(Error { code: ServerError(-32000), message: "filter not found", data: None }))', core/payment-driver/gnt/src/gnt/sender.rs:598:18
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    [2021-02-09T11:35:04Z INFO  ya_gnt_driver::gnt::faucet] Requesting Eth from faucet
    [2021-02-09T11:35:07Z INFO  ya_gnt_driver::gnt::faucet] Successfully requested Eth.
    [2021-02-09T11:35:07Z INFO  ya_gnt_driver] Waiting for ETH from faucet...
    [2021-02-09T11:35:24Z INFO  ya_gnt_driver] Received ETH from faucet.
    [2021-02-09T11:35:24Z INFO  ya_gnt_driver] Requesting NGNT from Faucet...

[ederenn]

nieznanysprawiciel - linux-py - Panic after payment fund erc20

Note: `yagna payment fund --driver erc20` returned:
`Funds obtained from faucet.`
But I don't have any funds after rerunning daemon.

[2021-02-09T11:12:17Z INFO  ya_zksync_driver::zksync::faucet] Requesting NGNT from zkSync faucet... address = 0x67c1e464d07c9c5f74c7073e5f602f7ec98b80ed
[2021-02-09T11:12:19Z WARN  ya_zksync_driver::zksync::faucet] Retrying (1/6) to request NGNT from Faucet after failure: GenericError { inner: "Failed to connect to host: Internal error: connector has been disconnected" }
[2021-02-09T11:12:32Z WARN  ya_zksync_driver::zksync::faucet] Retrying (2/6) to request NGNT from Faucet after failure: GenericError { inner: "Failed to connect to host: Internal error: connector has been disconnected" }
[2021-02-09T11:12:44Z INFO  ya_zksync_driver::zksync::faucet] Waiting for NGNT from faucet...
[2021-02-09T11:13:12Z INFO  ya_zksync_driver::zksync::faucet] Received NGNT from faucet.
[2021-02-09T11:13:24Z INFO  ya_payment::service::local] get status: GetStatus { address: "0x67c1e464d07c9c5f74c7073e5f602f7ec98b80ed", driver: "zksync", network: Some("rinkeby"), token: None }
[2021-02-09T11:14:00Z INFO  ya_gnt_driver::gnt::faucet] Requesting Eth from faucet
[2021-02-09T11:14:02Z INFO  ya_gnt_driver::gnt::faucet] Waiting for Eth faucet next donation in 30s
[2021-02-09T11:14:32Z WARN  ya_gnt_driver::gnt::faucet] Retrying (1/6) to request Eth from Faucet after failure: LibraryError("faucet request is queued, try again")
[2021-02-09T11:14:35Z INFO  ya_gnt_driver::gnt::faucet] Waiting for Eth faucet next donation in 30s
[2021-02-09T11:15:05Z WARN  ya_gnt_driver::gnt::faucet] Retrying (2/6) to request Eth from Faucet after failure: LibraryError("faucet request is queued, try again")
[2021-02-09T11:15:08Z INFO  ya_gnt_driver::gnt::faucet] Successfully requested Eth.
[2021-02-09T11:15:08Z INFO  ya_gnt_driver] Waiting for ETH from faucet...
[2021-02-09T11:15:08Z INFO  ya_gnt_driver] Received ETH from faucet.
[2021-02-09T11:15:08Z INFO  ya_gnt_driver] Requesting NGNT from Faucet...
[2021-02-09T11:16:09Z ERROR ya_gnt_driver::gnt::sender] Error sending transaction: EthereumClientError(Transport("Unexpected response status code: 504 Gateway Timeout"))
[2021-02-09T11:16:09Z INFO  ya_gnt_driver] send new tx: []
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: EthereumClientError(Rpc(Error { code: ServerError(-32000), message: "filter not found", data: None }))', core/payment-driver/gnt/src/gnt/sender.rs:598:18
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

yagna_rCURRENT (2).log
payment (1).zip
gnt-driver (1).zip

[etam]

This is related to #58: geth nodes were failing and were temporarily removed from proxy. Then a client connected to such failing geth could get redirected to another healthy geth node. But filters are stored on geth node, they make the protocol stateful.

I suggest using local_filter_middleware: https://web3py.readthedocs.io/en/stable/middleware.html#locally-managed-log-and-block-filters
See also: ethereum/web3.py#551, ethereum/web3.py#732

Edit: Oh, I forgot it's Rust, not Python. IDK if rust web3 has something similar.

[etam]

Additional idea: When using server statelessly it would make sense to repeat failed request few times. If a request fails, the next one might get redirected by proxy to another geth instance.