Unable to create a new connection in the presence of concurrent relayers

[arirubinstein]

Summary of Bug

On a network with other relayers running, it's not possible to create a new channel as hermes fails to create the connection since another relayer opens it connection state is not INIT or TRYOPEN (got STATE_OPEN).

full logs: https://gist.github.com/arirubinstein/a37aa0441617e539546d5fc77dbcef0b

Version

0.13.0

Steps to Reproduce

1. Run a hermes relayer on a network
2. on another hermes instance, attempt to create a new channel with the same config

Acceptance Criteria

Hermes should be able to reliably create new channels and be tolerant of other relayers changing state

• create connection CLI should be able to tolerate cases when the connection handshake is finished by other relayer processes; in this case, the CLIs should return success instead of retrying to resubmit useless transactions and fail
• integration test
• Unable to create a new chanel in the presence of concurrent relayers #2254

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate milestone (priority) applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned