11-to-12 migration fails with s3ds plugin

[ShadowJonathan]

Currently i'm in the middle of an angry and frustrated debug session for this, but so far I have observed:

• It does not properly query beyond 1000 keys (the default max for one list call)
• It does not swap anything because it "cannot find the key":

root@infinistore:~/fs-repo-migrations# ~/unpack/fs-repo-migrations/fs-repo-migrations -to 12                                                                                                                                                                                                                                
Found fs-repo version 11 at /data/ipfs                                                                                                                                                                                                                                                                                               
Do you want to upgrade this to version 12? [y/n] y                                                                                                                                                                                                                                                                                   
2023/01/22 17:27:03 Looking for suitable migration binaries.                                                                                                                                                                                                                                                                         
2023/01/22 17:27:03 Running migration fs-repo-11-to-12 ...                                                                                                                                                                                                                                                                           
2023/01/22 17:27:03   => Running: /root/bin/fs-repo-11-to-12 -path=/data/ipfs -verbose=true                                                                                                                                                                                                                                          
applying 11-to-12 repo migration                                                                                                                                                                                                                                                                                                     
locking repo at "/data/ipfs"                                                                                                                                                                                                                                                                                                         
  - verifying version is '11'                                                                                                                                                                                                                                                                                                        
  - loading repo configurations                                                                                                                                                                                                                                                                                                      
  - starting CIDv1 to raw multihash block migration                                                                                                                                                                                                                                                                                  
  - backup file will be written to /data/ipfs/11-to-12-cids.txt                                                                                                                                                                                                                                                                      
  - Adding keys in prefix /blocks to backup file                                                                                                                                                                                                                                                                                     
1000 CIDv1 keys added to backup file for /blocks                                                                                                                                                                                                                                                                                     
  - Adding keys in prefix /filestore to backup file                                                                                                                                                                                                                                                                                  
0 CIDv1 keys added to backup file for /filestore                                                                                                                                                                                                                                                                                     
  - backup file will be read from /data/ipfs/11-to-12-cids.txt                                                                                                                                                                                                                                                                       
ERROR: could not swap /blocks/ipfs/AFKREIA234FMYJUK6JEMCMKMNOOVP4DH2JEIKLMXSFHXICRT23GTWOENXA->/blocks/ipfs/CIQBVXYKZQTIV4SIYEYUY245K7YGPUSIQUWZPEKPOQFDHVWNHM4I3OA. Could not find /blocks/ipfs/AFKREIA234FMYJUK6JEMCMKMNOOVP4DH2JEIKLMXSFHXICRT23GTWOENXA even though it was in the backup file. Skipping.                         
ERROR: could not swap /blocks/ipfs/AFKREIA2464TM23YDSLXW55WZQFZFCJ4BYQ3LFUX3XX3O3UHJWGWOA27AM->/blocks/ipfs/CIQBVZ5ZGZVXQHEXPN33NTALSKETYDRBWWLJPXPPW5XIOTMNM4BV6AY. Could not find /blocks/ipfs/AFKREIA2464TM23YDSLXW55WZQFZFCJ4BYQ3LFUX3XX3O3UHJWGWOA27AM even though it was in the backup file. Skipping.                         
ERROR: could not swap /blocks/ipfs/AFKREIA246V6J6SNG7BXZKS4R7WDYY6YESKD3JB7PNG3LVGN7SF6OW26OY->/blocks/ipfs/CIQBVZ5L4T5E2N6DPSVFZD7MHRR5QJEUHWSD662NWXKM37EL45NV45Q. Could not find /blocks/ipfs/AFKREIA246V6J6SNG7BXZKS4R7WDYY6YESKD3JB7PNG3LVGN7SF6OW26OY even though it was in the backup file. Skipping.                         
...

I built this with: ./build-plugin.sh 11-to-12 github.com/ipfs/[email protected]

[ShadowJonathan]

I have finally gotten this working, using my own fork:

./build-plugin.sh 11-to-12 github.com/shadowjonathan/[email protected]