Disappearing values from Horde.Registry after spawning more than two nodes.

[LKlemens]

Problem: disappearing values from Horde.Registry after spawning more than two nodes.

Expected behavior: After adding more nodes, values in the Horde.Registry from all nodes will be synced without losing any value.

Elixir: 1.13.3
horde: 0.8.7

Reproduce:

• clone and setup project

   git clone https://github.com/LKlemens/horde_behavior
   cd horde_behavior
   mix deps.get

• start first node

  iex --sname node1@localhost --cookie foo -S mix

• start second node

  iex --sname node2@localhost --cookie foo -S mix

• check state of registry on node1 - should be two processes - one per node

  nodes = [node() | Node.list()]
  Enum.map(nodes, fn node -> Horde.Registry.lookup(Registry.State, node); end)
  [[{#PID<0.276.0>, nil}], [{#PID<19886.268.0>, nil}]]

• start third node

  iex --sname node3@localhost --cookie foo -S mix

• check state of registry on node1 - should be three processes but there are only two processes - first and last - the middle one disappearing

  nodes = [node() | Node.list()]
  Enum.map(nodes, fn node -> Horde.Registry.lookup(Registry.State, node); end)
  [[{#PID<0.276.0>, nil}], [], [{#PID<21155.273.0>, nil}]]

Notes

• I checked google and I found two issues related to it, but reading that threads didn't help me :P
  But maybe I missed sth.

  Registry drops all entries when second node joins #116
  https://elixirforum.com/t/horde-dynamicsupervisor-bypasses-registry/49562

• I thought that processes are adding to Horde.Registry too fast, but adding Agents manually by command below didn't help, issue still occures.

Agent.start_link(fn -> 553 end, name: {:via, Horde.Registry, {Registry.State, node()}})

• That might be related to issue 65 from delta_crdt_ex

  delta_crdt appears to lose data when running on more than one node. delta_crdt_ex#65

[LKlemens]

Hi,
@derekkraan I know you are busy and have much more important stuff to do, but could you (if you have time) just confirm that problem is reproducible on your side?
BTW Thank you for your awesome work! :)

[derekkraan]

Hi @LKlemens yes, I believe I have reproduced this error in the past.

It will likely be a while before I find time to work on it again.

Hope this answers your question.