fsrepo: improve migrations copy

[ghost]

No:

$ cmd/ipfs/ipfs daemon
Initializing daemon...
Found outdated fs-repo, migrations need to be run.
Run migrations now? [y/N] n
Not running migrations of fs-repo now.
Please get fs-repo-migrations from https://dist.ipfs.io
Error: fs-repo requires migration

Programmatic No:

$ cmd/ipfs/ipfs daemon --migrate=false
Initializing daemon...
Found outdated fs-repo, migrations need to be run.
Not running migrations of fs-repo now.
Please get fs-repo-migrations from https://dist.ipfs.io
Error: fs-repo requires migration

Yes with 404: (e.g. when we haven't pushed a migration to dist.ipfs.io yet)

$ cmd/ipfs/ipfs daemon --migrate=true
Initializing daemon...
Found outdated fs-repo, migrations need to be run.
  => Looking for suitable fs-repo-migrations binary.
  => None found, downloading.
  => Failed to download fs-repo-migrations.
The migrations of fs-repo failed:
  failed to find latest fs-repo-migrations: GET https://ipfs.io/ipns/ipfs.io/fs-repo-migrations/versions error: 404 Not Found: Path Resolve error: no link named "fs-repo-migrations" under QmTzQ1JRkWErjk39mryYw2WVaphAZNAREyMchXzYQ7c15n
If you think this is a bug, please file an issue and include this whole log output.
  https://github.com/ipfs/fs-repo-migrations
Error: failed to find latest fs-repo-migrations: GET https://ipfs.io/ipns/ipfs.io/fs-repo-migrations/versions error: 404 Not Found: Path Resolve error: no link named "fs-repo-migrations" under QmTzQ1JRkWErjk39mryYw2WVaphAZNAREyMchXzYQ7c15n

Yes with fs-repo-migrations error: (this is an fs-repo v4 where I did echo 3 > version)

$ cmd/ipfs/ipfs daemon --migrate=true
Initializing daemon...
Found outdated fs-repo, migrations need to be run.
  => Looking for suitable fs-repo-migrations binary.
  => None found, downloading.
  => Running: /tmp/go-ipfs-migrate893559663/fs-repo-migrations -to 4 -y
Found fs-repo version 3 at /home/user/.ipfs
===> Running migration 3 to 4...
applying 3-to-4 repo migration
locking repo at "/home/user/.ipfs"
  - verifying version is '3'
  - opening datastore at "/home/user/.ipfs"
transfering blocks to new key format
[1 / 293]failed to decode: /home/user/.ipfs/blocks/CIQA4/CIQA42T54MUH5XHODGGRH6HT3XZRADGA2GY3BXMNOGWBUSKFH2QTQTI.data
ipfs migration:  migration 3 to 4 failed: encoding/hex: odd length hex string
  => Failed: /tmp/go-ipfs-migrate893559663/fs-repo-migrations -to 4 -y
The migrations of fs-repo failed:
  migration failed: exit status 1
If you think this is a bug, please file an issue and include this whole log output.
  https://github.com/ipfs/fs-repo-migrations
Error: migration failed: exit status 1

Yes with success:

$ IPFS_PATH=ipfspath cmd/ipfs/ipfs daemon
Initializing daemon...
Found outdated fs-repo, migrations need to be run.
Run migrations now? [y/N] y
  => Looking for suitable fs-repo-migrations binary.
  => None found, downloading.
  => Running: /tmp/go-ipfs-migrate455985879/fs-repo-migrations -to 4 -y
Found fs-repo version 3 at ipfspath
===> Running migration 3 to 4...
applying 3-to-4 repo migration
locking repo at "ipfspath"
  - verifying version is '3'
  - opening datastore at "ipfspath"
transfering blocks to new key format
[15 / 15] Approx time remaining: 203s  
transferring stored public key records
gathering keys...
got 1 keys, beginning transfer. This will take some time.
[1 / 1]
transferring stored ipns records
gathering keys...
got 1 keys, beginning transfer. This will take some time.
[1 / 1]
updated version file
===> Migration 3 to 4 succeeded!
  => Success: fs-repo has been migrated to 4.
Swarm listening on /ip4/10.230.22.40/tcp/4001
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/172.17.0.1/tcp/4001
Swarm listening on /ip4/192.168.4.219/tcp/4001
Swarm listening on /ip4/78.52.201.197/tcp/4001
Swarm listening on /ip6/::1/tcp/4001
Swarm listening on /ip6/fc3d:7777:a6a4:fcdb:f218:5856:5de:eb1a/tcp/4001
Swarm listening on /ip6/fd0a:562b:34e4:0:a4df:52b7:7446:2229/tcp/4001
Swarm listening on /ip6/fd0a:562b:34e4::b85/tcp/4001
API server listening on /ip4/127.0.0.1/tcp/5001
Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Daemon is ready

[jbenet]

nice, thanks!

[jbenet]

@lgierth possible for a script to be made that compiles all this output so we can check it every once in a while?

[jbenet]

This LGTM

[ghost]

I'll try to add sharness tests

[ghost]

I'll try to add sharness tests

Ah @whyrusleeping already updated the tests. thanks!