Custom epmd to avoid race conditions in DNS name resolutions

[arcusfelis]

Requires esl/MongooseIM#4179
How to test (clone, checkout this branch first):

helm install test-mim MongooseIM --set replicaCount=10 --set image.tag=PR-4179 --set persistentDatabase=rdbms --set rdbms.username=mongooseim --set rdbms.database=mongooseim --set volatileDatabase=cets

(only PR-4163 works with this build because of new config options for wait_for_dns).
Nodes are joined:

kubectl exec -it mongooseim-0 -- mongooseimctl cets systemInfo

And logs are cleaner:

kubectl logs mongooseim-0

Upgrades should work too:

helm upgrade mim-test MongooseIM --set replicaCount=10 --set image.tag=PR-4179 --set persistentDatabase=rdbms --set rdbms.username=mongooseim --set rdbms.database=mongooseim --set volatileDatabase=cets --set image.pullPolicy=Always

You should see during install :

when=2023-11-21T18:48:03.352629+00:00 level=warning what=nodeup pid=<0.428.0> at=cets_discovery:handle_info/2:236 time_since_startup_in_milliseconds=20895 alive_nodes=2 remote_node=mongooseim@mongooseim-1.mongooseim.default.svc.cluster.local
when=2023-11-21T18:48:23.573738+00:00 level=warning what=nodeup pid=<0.428.0> at=cets_discovery:handle_info/2:236 time_since_startup_in_milliseconds=41116 alive_nodes=3 remote_node=mongooseim@mongooseim-2.mongooseim.default.svc.cluster.local
when=2023-11-21T18:48:43.764690+00:00 level=warning what=nodeup pid=<0.428.0> at=cets_discovery:handle_info/2:236 time_since_startup_in_milliseconds=61286 alive_nodes=4 remote_node=mongooseim@mongooseim-3.mongooseim.default.svc.cluster.local
when=2023-11-21T18:49:04.694150+00:00 level=warning what=nodeup pid=<0.428.0> at=cets_discovery:handle_info/2:236 time_since_startup_in_milliseconds=82236 alive_nodes=5 remote_node=mongooseim@mongooseim-4.mongooseim.default.svc.cluster.local
when=2023-11-21T18:49:25.349933+00:00 level=warning what=nodeup pid=<0.428.0> at=cets_discovery:handle_info/2:236 time_since_startup_in_milliseconds=102892 alive_nodes=6 remote_node=mongooseim@mongooseim-5.mongooseim.default.svc.cluster.local
when=2023-11-21T18:49:45.800172+00:00 level=warning what=nodeup pid=<0.428.0> at=cets_discovery:handle_info/2:236 time_since_startup_in_milliseconds=123322 alive_nodes=7 remote_node=mongooseim@mongooseim-6.mongooseim.default.svc.cluster.local
when=2023-11-21T18:50:05.832652+00:00 level=warning what=nodeup pid=<0.428.0> at=cets_discovery:handle_info/2:236 time_since_startup_in_milliseconds=143375 alive_nodes=8 remote_node=mongooseim@mongooseim-7.mongooseim.default.svc.cluster.local
when=2023-11-21T18:50:25.832449+00:00 level=warning what=nodeup pid=<0.428.0> at=cets_discovery:handle_info/2:236 time_since_startup_in_milliseconds=163375 alive_nodes=9 remote_node=mongooseim@mongooseim-8.mongooseim.default.svc.cluster.local
when=2023-11-21T18:50:46.931023+00:00 level=warning what=nodeup pid=<0.428.0> at=cets_discovery:handle_info/2:236 time_since_startup_in_milliseconds=184453 alive_nodes=10 remote_node=mongooseim@mongooseim-9.mongooseim.default.svc.cluster.local

During update - something similar in logs.

[chrzaszcz]

👌