support backup of v2 and v3 stores

[heyitsanthony]

Right now etcd backups must choose between v2's backup and v3's snapshot. For systems that store v2 and v3 data at once, there is no way to fully back up the cluster. This behavior is somewhat unexpected since presumably a backup of the cluster should restore all the keys that were on the system.

[raoofm]

I think this should be a really high priority rather than just an enhancement. It puts all the existing v2 users who moved to v3 without migrating data (or with migrating data but not all clients migrated yet) at a huge risk.

[r7vme]

i've just experimented, after reading blog post from @mattymo , and found out that v2 backup actually contains both v2 and v3 data.

I have etcd 3.2.0-rc.0+git

To check

Start etcd

mkdir /tmp/etcd-data
etcd -debug -data-dir /tmp/etcd-data

Create v2 and v3 data

for i in `seq 1 10`; do curl http://127.0.0.1:2379/v2/keys/message$i -XPUT -d value="Hello world"; done
# create v3 foo bar (base64 encoded)
curl -L http://127.0.0.1:2379/v3alpha/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'

Create backup

mkdir /tmp/etcd-backup
etcdctl backup --data-dir /tmp/etcd-data --backup-dir /tmp/etcd-backup

Stop existing etcd and start new one from backup

etcd -debug -data-dir /tmp/etcd-backup --force-new-cluster

Check values

~|⇒ etcdctl ls                                                 
/message4
/message5
/message8
/message9
/message2
/message3
/message7
/message10
/message1
/message6
~|⇒ ETCDCTL_API=3 etcdctl get "" --prefix=true 
foo
bar

Is this smth unexpected?

[heyitsanthony]

@r7vme there's probably v3 entries in the wal that are being copied over and then replayed back into the v3 store. It's not an actual backup

[raoofm]

@xiang90 @heyitsanthony Should this issue be closed in favor of #6925. Then the users can migrate v2 data to v3 and still support v2 clients. So the current problem of taking both v2 and v3 backup as a whole can be dropped as we would then need only v3 backup.

[heyitsanthony]

Fixed by #8479