Release 2.0.0b7, beta quality, for testing only #7827

ThomasWaldmann · 2023-09-14T21:51:28Z

ThomasWaldmann
Sep 14, 2023
Maintainer

borgbackup 2.0.0b7 beta release - do not use this on production backup repositories.

Long changelog:
https://borgbackup.readthedocs.io/en/2.0.0b7/changes.html#version-2-0-0b7-2023-09-14

If you use pip to install this, use: pip install "borgbackup==2.0.0b7"

Docs: https://borgbackup.readthedocs.io/en/2.0.0b7/

Feedback:

This discussion was created from the release Release 2.0.0b7, beta quality, for testing only.

bket · 2023-09-15T07:27:51Z

bket
Sep 15, 2023

Tested on OpenBSD amd64 current. Builds OK, all tests pass, and so far no issues found with my use case.

Interestingly, several source files (*.c, *.h, *.pyx) are being packaged/installed (2.0.0b7 versus 2.0.0b6). I believe this is caused by 61e96eb. Is this intended?

--- 2.0.0b6
+++ 2.0.0b7
@@ -45,6 +45,10 @@ site-packages
 site-packages/borg/${MODPY_PYCACHE}version.${MODPY_PYC_MAGIC_TAG}pyc
 site-packages/borg/${MODPY_PYCACHE}xattr.${MODPY_PYC_MAGIC_TAG}${MODPY_PYOEXTENSION}
 site-packages/borg/${MODPY_PYCACHE}xattr.${MODPY_PYC_MAGIC_TAG}pyc
+site-packages/borg/_chunker.c
+site-packages/borg/_endian.h
+site-packages/borg/_hashindex.c
+site-packages/borg/_item.c
 site-packages/borg/_version.py
 site-packages/borg/archive.py
 site-packages/borg/archiver/
@@ -134,12 +138,24 @@ site-packages
 site-packages/borg/archiver/tar_cmds.py
 site-packages/borg/archiver/transfer_cmd.py
 site-packages/borg/cache.py
+site-packages/borg/cache_sync/
+site-packages/borg/cache_sync/cache_sync.c
+site-packages/borg/cache_sync/sysdep.h
+site-packages/borg/cache_sync/unpack.h
+site-packages/borg/cache_sync/unpack_define.h
+site-packages/borg/cache_sync/unpack_template.h
+site-packages/borg/checksums.c
 @so site-packages/borg/checksums.${MODPY_PYC_MAGIC_TAG}so
 site-packages/borg/checksums.pyi
+site-packages/borg/checksums.pyx
+site-packages/borg/chunker.c
 @so site-packages/borg/chunker.${MODPY_PYC_MAGIC_TAG}so
 site-packages/borg/chunker.pyi
+site-packages/borg/chunker.pyx
+site-packages/borg/compress.c
 @so site-packages/borg/compress.${MODPY_PYC_MAGIC_TAG}so
 site-packages/borg/compress.pyi
+site-packages/borg/compress.pyx
 site-packages/borg/constants.py
 site-packages/borg/crypto/
 site-packages/borg/crypto/__init__.py
@@ -155,11 +171,15 @@ site-packages
 site-packages/borg/crypto/file_integrity.py
 site-packages/borg/crypto/key.py
 site-packages/borg/crypto/keymanager.py
+site-packages/borg/crypto/low_level.c
 @so site-packages/borg/crypto/low_level.${MODPY_PYC_MAGIC_TAG}so
+site-packages/borg/crypto/low_level.pyx
 site-packages/borg/fuse.py
 site-packages/borg/fuse_impl.py
+site-packages/borg/hashindex.c
 @so site-packages/borg/hashindex.${MODPY_PYC_MAGIC_TAG}so
 site-packages/borg/hashindex.pyi
+site-packages/borg/hashindex.pyx
 site-packages/borg/helpers/
 site-packages/borg/helpers/__init__.py
 ${MODPY_COMMENT}site-packages/borg/helpers/${MODPY_PYCACHE}/
@@ -210,8 +230,10 @@ site-packages
 site-packages/borg/helpers/shellpattern.py
 site-packages/borg/helpers/time.py
 site-packages/borg/helpers/yes_no.py
+site-packages/borg/item.c
 @so site-packages/borg/item.${MODPY_PYC_MAGIC_TAG}so
 site-packages/borg/item.pyi
+site-packages/borg/item.pyx
 site-packages/borg/locking.py
 site-packages/borg/logger.py
 site-packages/borg/manifest.py
@@ -227,7 +249,19 @@ site-packages
 site-packages/borg/platform/${MODPY_PYCACHE}xattr.${MODPY_PYC_MAGIC_TAG}${MODPY_PYOEXTENSION}
 site-packages/borg/platform/${MODPY_PYCACHE}xattr.${MODPY_PYC_MAGIC_TAG}pyc
 site-packages/borg/platform/base.py
+site-packages/borg/platform/darwin.c
+site-packages/borg/platform/darwin.pyx
+site-packages/borg/platform/freebsd.c
+site-packages/borg/platform/freebsd.pyx
+site-packages/borg/platform/linux.c
+site-packages/borg/platform/linux.pyx
+site-packages/borg/platform/posix.c
 @so site-packages/borg/platform/posix.${MODPY_PYC_MAGIC_TAG}so
+site-packages/borg/platform/posix.pyx
+site-packages/borg/platform/syncfilerange.c
+site-packages/borg/platform/syncfilerange.pyx
+site-packages/borg/platform/windows.c
+site-packages/borg/platform/windows.pyx
 site-packages/borg/platform/xattr.py
 site-packages/borg/platformflags.py
 site-packages/borg/remote.py
@@ -283,6 +317,14 @@ site-packages
 site-packages/borg/testsuite/${MODPY_PYCACHE}patterns.${MODPY_PYC_MAGIC_TAG}pyc
 site-packages/borg/testsuite/${MODPY_PYCACHE}platform.${MODPY_PYC_MAGIC_TAG}${MODPY_PYOEXTENSION}
 site-packages/borg/testsuite/${MODPY_PYCACHE}platform.${MODPY_PYC_MAGIC_TAG}pyc
+site-packages/borg/testsuite/${MODPY_PYCACHE}platform_darwin.${MODPY_PYC_MAGIC_TAG}${MODPY_PYOEXTENSION}
+site-packages/borg/testsuite/${MODPY_PYCACHE}platform_darwin.${MODPY_PYC_MAGIC_TAG}pyc
+site-packages/borg/testsuite/${MODPY_PYCACHE}platform_freebsd.${MODPY_PYC_MAGIC_TAG}${MODPY_PYOEXTENSION}
+site-packages/borg/testsuite/${MODPY_PYCACHE}platform_freebsd.${MODPY_PYC_MAGIC_TAG}pyc
+site-packages/borg/testsuite/${MODPY_PYCACHE}platform_linux.${MODPY_PYC_MAGIC_TAG}${MODPY_PYOEXTENSION}
+site-packages/borg/testsuite/${MODPY_PYCACHE}platform_linux.${MODPY_PYC_MAGIC_TAG}pyc
+site-packages/borg/testsuite/${MODPY_PYCACHE}platform_posix.${MODPY_PYC_MAGIC_TAG}${MODPY_PYOEXTENSION}
+site-packages/borg/testsuite/${MODPY_PYCACHE}platform_posix.${MODPY_PYC_MAGIC_TAG}pyc
 site-packages/borg/testsuite/${MODPY_PYCACHE}remote.${MODPY_PYC_MAGIC_TAG}${MODPY_PYOEXTENSION}
 site-packages/borg/testsuite/${MODPY_PYCACHE}remote.${MODPY_PYC_MAGIC_TAG}pyc
 site-packages/borg/testsuite/${MODPY_PYCACHE}repoobj.${MODPY_PYC_MAGIC_TAG}${MODPY_PYOEXTENSION}
@@ -421,6 +463,10 @@ site-packages
 site-packages/borg/testsuite/nanorst.py
 site-packages/borg/testsuite/patterns.py
 site-packages/borg/testsuite/platform.py
+site-packages/borg/testsuite/platform_darwin.py
+site-packages/borg/testsuite/platform_freebsd.py
+site-packages/borg/testsuite/platform_linux.py
+site-packages/borg/testsuite/platform_posix.py
 site-packages/borg/testsuite/remote.py
 site-packages/borg/testsuite/repoobj.py
 site-packages/borg/testsuite/repository.py
@@ -431,6 +477,7 @@ site-packages
 site-packages/borg/version.py
 site-packages/borg/xattr.py
 site-packages/borgbackup-${MODPY_EGG_VERSION}.dist-info/
+site-packages/borgbackup-${MODPY_EGG_VERSION}.dist-info/AUTHORS
 site-packages/borgbackup-${MODPY_EGG_VERSION}.dist-info/LICENSE
 site-packages/borgbackup-${MODPY_EGG_VERSION}.dist-info/METADATA
 site-packages/borgbackup-${MODPY_EGG_VERSION}.dist-info/RECORD

2 replies

ThomasWaldmann Sep 15, 2023
Maintainer Author

Thanks for testing and review.

I guess the added source files are due to these lines:
61e96eb#diff-fa602a8a75dc9dcc92261bac5f533c2a85e34fcceaff63b3a3a81d9acde2fc52L52

I didn't find a corresponding setting for pyproject.toml.

ThomasWaldmann Sep 15, 2023
Maintainer Author

Found something: https://setuptools.pypa.io/en/stable/userguide/datafiles.html

Rather new setuptools seems to support this in pyproject.toml:

[tool.setuptools.packages.find]
where = ["src"]

[tool.setuptools.exclude-package-data]
mypkg = [".gitattributes"]

awgcooper · 2023-10-11T11:16:27Z

awgcooper
Oct 11, 2023

I've been using this for a month now on Debian 12 amd64, replicating two of my Borg 1.2x repos. Nothing to report other than there have been no problems and, subjectively speaking, it seems to be faster than 1.2x.

Dare I ask how close you think the final release may be?

7 replies

ThomasWaldmann Oct 14, 2023
Maintainer Author

Precise refcounting of chunks -> only "exists or not" and garbage collection, going away from transactions and log-like storage. That would simplify it a lot and could prepare for more parallelism in future. See the "breaking" issues for more about that.

awgcooper Oct 14, 2023

Certainly does seem like a lot of work. Regarding this: #7377 (comment), you mention the need for a scalable fs. At a guess, which filesystems would and wouldn't work?

ThomasWaldmann Oct 14, 2023
Maintainer Author

Depends a lot on the repo size and maybe also the size/count of input files (the latter could be worked around by not chunking individual files (as now), but the concatenated stream of all files' content). But guess older filesystems like *FAT and ext2/3 might struggle. Some issues (like too many files in a single directory) can be easily worked around.

awgcooper Oct 14, 2023

What's your guesstimate for the maximum 'safe' repo size with Borg2, assuming you make these 'bigger changes'?

ThomasWaldmann Oct 14, 2023
Maintainer Author

Can't say. The limit (if any) comes from the fs used.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release 2.0.0b7, beta quality, for testing only #7827

{{title}}

Replies: 2 comments 9 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Release 2.0.0b7, beta quality, for testing only #7827

ThomasWaldmann Sep 14, 2023 Maintainer

Replies: 2 comments · 9 replies

bket Sep 15, 2023

ThomasWaldmann Sep 15, 2023 Maintainer Author

ThomasWaldmann Sep 15, 2023 Maintainer Author

awgcooper Oct 11, 2023

ThomasWaldmann Oct 14, 2023 Maintainer Author

awgcooper Oct 14, 2023

ThomasWaldmann Oct 14, 2023 Maintainer Author

awgcooper Oct 14, 2023

ThomasWaldmann Oct 14, 2023 Maintainer Author

ThomasWaldmann
Sep 14, 2023
Maintainer

Replies: 2 comments 9 replies

bket
Sep 15, 2023

ThomasWaldmann Sep 15, 2023
Maintainer Author

ThomasWaldmann Sep 15, 2023
Maintainer Author

awgcooper
Oct 11, 2023

ThomasWaldmann Oct 14, 2023
Maintainer Author

ThomasWaldmann Oct 14, 2023
Maintainer Author

ThomasWaldmann Oct 14, 2023
Maintainer Author