spread not working

[alcir]

spread is not working, while the package from the EC repository works, 32 and 64 bit version.

[mamash]

Can you be a little more specific? I can confirm a typo in the spread SMF manifest (missing 'type' attribute on the stop method) in 2013Q2, and I'm fixing that now. However, with that fixed, spread works for me across two nodes with a simple configuration; e.g. services come up, and I can use spuser to join a group.

[alcir]

really? I use wackamole. And in a two nodes config, the vip never rise.

[mamash]

Hm, isn't that more of a wackamole problem then? Spread is just the message bus.

[alcir]

I don't think.
Wackamole and Spread, from EC repositroy works fine. With the same configuration, the packages from pkgsrc don't work.
Again, if I compile spread from sources (but using the -m32 flag), then wackamole from pkgsrc works fine.
If I compile spread from sources without using -m32, wackamole doesn't work.

[mamash]

I had a look at how EC builds spread. There are some patches that fix the build process - none of which is an issue under pkgsrc - and also some that modify the source files. Alas, with no history.

[alcir]

I've tried to apply these patches to the pkgsrc package. But wackamole doesn't work.
Have you tried wackamole?

[mamash]

No, but Richard from Joyent support has worked with a big JPC customer, and based on his notes on an internal ticket he made it work just fine. I'll get his notes.

[mamash]

I just made a new cluster of two SmartOS VMs and spread/wackamole work fine. Here's how to reproduce:

• Provision two VMs with "allow_ip_spoofing":1 set, using the base-13.2.1 (base64-13.2.1) image.
• Install spread and wackamole (pkgin -y in spread wackamole).
• Put down basic spread.conf and wackamole.conf files (see below)
• Make sure the node names (e.g. 'node1') match the VM hostnames (/etc/nodename) and the name/IP resolve (via /etc/hosts)
• Start spread (svcadm enable spread)
• Start wackamole (svcadm enable wackamole)

I can see the VIP show up as net0:1 on both nodes, and is constantly available as I start/stop the first/second nodes.

Files used:

/opt/local/etc/spread.conf

Spread_Segment  10.88.88.127:4803 {
        node1 10.88.88.221
        node2 10.88.88.222
}
DebugFlags = { PRINT EXIT }
EventPriority = pDEBUG
RuntimeDir = /var/spool/spread
EventTimeStamp

/opt/local/etc/wackamole.conf

Spread = 4803
SpreadRetryInterval = 5s
Group = wack1
Control = /var/run/wack.it
Prefer None
VirtualInterfaces {
        net0:10.88.88.220/32
}
Arp-Cache = 60s
balance {
        AcquisitionsPerRound = all
        interval = 4s
}
mature = 5s

/etc/nodename

node1

/etc/hosts

::1        localhost
127.0.0.1  localhost loghost f8e0e411-5e26-4c51-8153-01ac9eb4d1ea
10.88.88.221  node1
10.88.88.222  node2

Notes:

• Alternatively, you can modify the SMF start method for spread and force an arbitrary node name, and not mangle your /etc/nodename:

svccfg -s spread setprop start/exec = astring: '"/opt/local/sbin/spread -n node1"'

• To have spread log to a file, make sure spread:spread has access, as that's the user/group the service is run as.

[alcir]

Yes, I can confirm that it works.
I don't really know where the problem was, where my mistake was.
Thanks.