EVM errors with automine / Transaction Queuing

[teknoxic]

i am building a centralised token exchange for ERC20 tokens with ganache-cli.
It's a simple order book system for trading, and each user has their own wallet on Ganache.
It works like this:
User sends order from frontend to backend NodeJS server > NodeJS sends order to ganache-cli > Ganache-cli sends result to frontend

It works quite well when one user makes an order for a token, however when multiple users make an order at the same time i get some errors on ganache-cli:

• EVM errors like this:

- Error: Transaction has been reverted by the EVM:
{
  "transactionHash": "0x6166209fdff8db5d611de59b6b2da4065c30a4191217c43b3bed15fa17eebb81",
  "transactionIndex": 0,
  "blockNumber": 7989,
  "blockHash": "0x5ac49c1e4cdfab30b1f009a3c3559a2042c1a66924728bc3a24ef1f16aa4fb95",
  "from": "0x7f300f7b7374aa64e7a627e6135419bf858625b8",
  "to": "0x74a259bb89e22abfb9dd1b21b48237830045a53b",
  "cumulativeGasUsed": 25940,
  "gasUsed": 25940,
  "contractAddress": null,
  "logs": [],
  "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
  "status": false,
  "effectiveGasPrice": 0,
  "type": "0x0"
}

• Errors due to incorrect nonces (nonce is always 1 ahead)
  Error: Returned error: the tx doesn't have the correct nonce. account has nonce of: 4996 tx has nonce of: 4995
  I'm generating a nonce and sending transactions like this:

                nonce = await web3.eth.getTransactionCount(addr);
                transaction = {
                    "from": user,
                    "nonce": web3.utils.toHex(nonce),
                    "gasPrice": web3.utils.toHex(gasPrice * 1e9),
                    "gasLimit": web3.utils.toHex(gasLimit),
                    "to": useraddress,
                    "value": amountToSend,
                };

My current settings are:
ganache-cli --database.dbPath /opt/db -a 20 --wallet.defaultBalance 100 --wallet.mnemonic "*" -g 0 -p 7545 --chain.networkId 987 -h 127.0.0.1 --miner.blockTime 0 --chain.hardfork berlin

Ganache-cli vesion:
ganache v7.0.1 (@ganache/cli: 0.1.2, @ganache/core: 0.1.2)
So my questions are

1. is auto-mining suitable for this use-case?
2. Does ganache-cli have any built in transaction queuing for my use case? i would like to be able to process large numbers of transactions on a contract in the order of the time Ganache receives them. eg. execute the first transaction received first and the last transaction last.
3. Is there any settings or improvements to my current ganache-cli settings that you can suggest that would work better for the backend of a centralised token exchange?
4. Is there a better way to generate nonces before a transaction?

I would really like to solve these issues with Ganache directly, rather than building an order queuing system in between the frontend and Ganache!

Thanks

[davidmurdoch]

You've almost got it set up correctly.

If you omit the nonce field from the transaction object Ganache will auto increment the nonce automatically and process the transactions in FIFO (first in, first out) order (with exceptions[1]). note this won't work with signed transactions (eth_sendRawTransaction), as those always require a nonce.

You can also send transactions with a nonce that is "too high" to Ganache v7, so you could fetch the transaction count just once on start up (depending on your use case you may need to check txpool_content for transactions that are in the pool but not yet mined in order to calculate the correct nonce -- but with a blocktime of 0 that shouldn't be an issue) and then increment the nonce by 1 each time. Here is a somewhat naive way that should work in non-exceptional circumstances:

const nonces = new Map();
async function exchange(user, to, value) {
  let currentNonce = nonces.get(user) || 0;
  try {
    const txHash =  await sendTransaction(user, to, value, currentNonce);
    nonces.set(currentNonce + 1); // transaction was accepted by Ganache
    return await waitForReceipt(txHash);
  } catch(e) {
    // handle the case where the transaction could not be sent (like if the user doesn't have the funds to send the transaction)
  }
}

A said the above function "naive" because transaction monitoring can get pretty complex, especially when many transactions from a single account are in the transaction pool. You'll need to worry about sending "replacement" transactions (sending another transaction with the same nonce but a higher "effective" gas price) if sendTransaction throws an exception but another transaction has already been sent in its stead. I'll leave the intricacies for handling that up to you.

• If you send a transaction with a "too high" nonce so it gets put the non-executable pending pool, and then send a second transaction without a nonce, that second transaction will be processed before the first transaction you send, since it automatically gets the next available nonce.

[cds-amal]

Closing for issue maintenance.