Release 2.0.0b11 - beta quality, for testing only

[ThomasWaldmann]

borgbackup 2.0.0b11 beta release - do not use this on production backup repositories.

Long changelog:
https://borgbackup.readthedocs.io/en/2.0.0b11/changes.html#version-2-0-0b11-2024-09-26

If you use pip to install this, use: pip install "borgbackup==2.0.0b11"

Docs: https://borgbackup.readthedocs.io/en/2.0.0b11/

Discussion:

Note: Only limited platform testing and binaries this time, see "vagrant" tag in issue tracker.

---

This discussion was created from the release Release 2.0.0b11 - beta quality, for testing only.

[bjo81]

Thanks, will give it a try with Hetzners Object Storage Beta. Are b10 repos compatible with b11?

[bjo81]

Seems like there is nothing in the target repo: rclone -v delete fsn:mail-b2 returns nothing deleted, maybe fsn/mail-b2 was wrong before, but it also throws borgstore.backends.errors.BackendAlreadyExists: rclone storage base path already exists: fsn:mail-b2

[ThomasWaldmann]

Maybe "rclone rmdir ..."?

[bjo81]

That deleted the whole bucket:

2024/09/27 11:52:05 INFO  : S3 bucket mail-b2: Removing directory
2024/09/27 11:52:05 INFO  : S3 bucket mail-b2: Bucket "mail-b2" deleted

Edit: created a new bucket, now it throws

borgstore.backends.errors.BackendAlreadyExists: rclone storage base path already exists: fsn:mailb2/

but it's definitely empty.

[ThomasWaldmann]

There will be a fix for these "base directory" issues in next borgstore release (soon).

[ThomasWaldmann]

borgstore 0.0.5 with a fix was just released!

[rmiceli]

Likely user error, but seems like borg2 is not able to find rclone (v1.64.2) on my system (Fedora 40). Let me know if more details would help or if helpful to open an issue.

borg2b11 -r rclone://onedrive:/borg-beta/framework repo-create -e repokey-blake2-chacha20-poly1305
2024/09/27 17:45:44 Failed to rc: loopback call failed: strconv.ParseInt: parsing "": invalid syntax
Error: rclone binary not found on the path or not working properly

which borg2b11
/usr/bin/borg2b11

which rclone
/usr/bin/rclone

rclone lsd onedrive:/
          -1 2024-09-27 17:13:05         0 borg-beta

[ThomasWaldmann]

I downloaded some of the macOS arm64 rclone binaries and this is the output:

{
	"arch": "arm64",
	"decomposed": [
		1,
		57,
		0
	],
	"goTags": "cmount",
	"goVersion": "go1.17.2",
	"isBeta": false,
	"isGit": false,
	"linking": "dynamic",
	"os": "darwin",
	"version": "v1.57.0"
}

{
	"arch": "arm64",
	"decomposed": [
		1,
		64,
		2
	],
	"goTags": "cmount",
	"goVersion": "go1.21.3",
	"isBeta": false,
	"isGit": false,
	"linking": "dynamic",
	"os": "darwin",
	"version": "v1.64.2"
}

{
	"arch": "arm64",
	"decomposed": [
		1,
		67,
		0
	],
	"goTags": "cmount",
	"goVersion": "go1.22.4",
	"isBeta": false,
	"isGit": false,
	"linking": "dynamic",
	"os": "darwin",
	"version": "v1.67.0"
}

{
	"arch": "arm64",
	"decomposed": [
		1,
		68,
		1
	],
	"goTags": "cmount",
	"goVersion": "go1.23.1",
	"isBeta": false,
	"isGit": false,
	"linking": "dynamic",
	"os": "darwin",
	"version": "v1.68.1"
}

[ncw]

$ rclone-v1.64.2 rc --loopback core/version
{
	"arch": "amd64",
	"decomposed": [
		1,
		64,
		2
	],
	"goTags": "none",
	"goVersion": "go1.21.3",
	"isBeta": false,
	"isGit": false,
	"linking": "static",
	"os": "linux",
	"version": "v1.64.2"
}

That works fine for me with rclone v1.64.2. I wonder if Fedora have broken rclone somehow? This command works all the way back to rclone v1.48 for me.

[rmiceli]

Yes, seems to be an issue with Fedora somehow. Output below is from a fresh Fedora 41 image. I can try to troubleshoot some more this week and file a bug in their bug tracker. Let me know if helpful to provide more details.

$ rclone rc --loopback core/version
{
	"error": "loopback: call failed: strconv.ParseInt: parsing \"\": invalid syntax",
	"path": "core/version",
	"status": 500
}
2024/09/30 07:51:06 Failed to rc: loopback: call failed: strconv.ParseInt: parsing "": invalid syntax

$ rclone --version
rclone 1.67.0
- os/version: fedora 41 (64 bit)
- os/kernel: 6.10.11-200.fc40.x86_64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.23rc2
- go/linking: dynamic
- go/tags: none

[ncw]

I looked at the Fedora packaging for rclone and it looks sensible. First thought is that Fedora have modified the version number to be non-numeric which is causing the ParseInt problem. It doesn't look like that in the packaging though (nor in your rclone --version output so it is probably something else along those lines.

[rmiceli]

The issue with the Fedora package is that the rclone fs.Version is not set with a leading "v" (e.g. 1.64.2 instead of v1.64.2). The package maintainer just pushed an update to fix this: https://src.fedoraproject.org/rpms/rclone/c/120873a89d0e80fc936b4bc8fe196e1d55c30e1f

[rmiceli]

Did some basic tests on Fedora 40 of a remote repo via rclone to OneDrive and everything worked. Let me know if more info or tests would be helpful. Excited to see these features move out of beta!

• Ran repo-create to initialize repo on onedrive. This worked once I updated rclone from v1.64.2 to v1.68.1
• Created an archive of a 5.5 GB directory. Stats below

  Repository: ssh://None/./onedrive:/borg-beta/framework
  Archive name: test
  Archive fingerprint: ***
  Time (start): Sat, 2024-09-28 09:38:54 -0400
  Time (end):   Sat, 2024-09-28 15:03:54 -0400
  Duration: 5 hours 24 minutes 59.847 seconds
  Number of files: 5050
  Original size: 5.52 GB
  Deduplicated size: 5.38 GB
  Time spent in hashing: 14.445 seconds
  Time spent in chunking: 37.894 seconds
  Added files: 5050
  Unchanged files: 0
  Modified files: 0
  Error files: 0
  Files changed while reading: 0
  Bytes read from remote: 0
  Bytes sent to remote: 0
  Saving files cache
  Saving chunks cache
  Saving cache config
  

• Deleted ~850 files and added ~20 MB, then created a second archive

  Repository: ssh://None/./onedrive:/borg-beta/framework                                                                                        
  Archive name: test
  Archive fingerprint: ***
  Time (start): Sat, 2024-09-28 15:11:11 -0400
  Time (end):   Sat, 2024-09-28 15:12:54 -0400
  Duration: 1 minutes 43.301 seconds
  Number of files: 4203
  Original size: 5.56 GB
  Deduplicated size: 22.09 MB
  Time spent in hashing: 0.078 seconds
  Time spent in chunking: 0.107 seconds
  Added files: 4
  Unchanged files: 4199
  Modified files: 0
  Error files: 0
  Files changed while reading: 0
  Bytes read from remote: 0
  Bytes sent to remote: 0
  Saving files cache
  Saving chunks cache
  Saving cache config
  

• Deleted first archive and compacted

  Starting compaction / garbage collection...
  Getting object IDs present in the repository...
  Computing object IDs used by archives...
  Analyzing archive test 2024-09-28 19:11:11.080889+00:00 *** (1/1)
  Determining unused objects...
  Deleting 865 unused objects...
  Overall statistics, considering all 1 archives in this repository:
  Source data size was 6 GB in 5502 files.
  Repository size is 5 GB in 5951 objects.
  Compaction saved 1 MB.
  Finished compaction / garbage collection...
  

• Restored ~50 MB from the second archive and files matched the originals

[ThomasWaldmann]

For all beta testers: please upgrade to borgstore 0.0.5, it fixes some issues:

https://github.com/borgbackup/borgstore/blob/master/CHANGES.rst#version-005-2024-10-01

[bjo81]

After upgrading borgstore, the repo-create went fine.

But the question for the passphrase of the rclone repo looks wrong (ssh://None./.. ):

 borg2 transfer --repo rclone://fsn:borg2store/mail  --from-borg1 --other-repo /tmp/sb/mail
Enter passphrase for key /tmp/sb/mail: 
Enter passphrase for key ssh://None/./fsn:borg2store/mail: 

[ThomasWaldmann]

#8446