borg init: add a --additional-free-space=X option? default?

[fschulze]

On a remote backup the disk quota was reached and borg failed to connect because borg serve just dies and locally I get:
Connection closed by remote host. Is borg working on the server?

I couldn't find a way to get it working again, so I ended up having to delete the full backup and loose all history.

I found the storage_quota and additional_free_space options, but no way to set them on an existing backup except by manually using sftp. It seems only --storage-quota is exposed via the init command, but not the other option (which from what I read seems more useful).

Locally I have borg 1.1.17, not sure what version Hetzner has on its storage boxes/backup spaces.

Shouldn't additional_free_space be set by default to prevent unrecoverable errors like this?

[ThomasWaldmann]

Did you try borg config?

[ThomasWaldmann]

About a default additional_free_space: how would you compute the default value to fit most future scenarios?

[fschulze]

I tried borg config, but it is local only for security reasons according to the docs.

As I understand it the size of additional_free_space mostly depends on the segment size. But it doesn't really matter as long as there is enough space to let borg serve still run and let borg prune --save-space work to be able to recover.

[ThomasWaldmann]

Right, only local borg config.

But I think there's also some temporary free space needed for repo index, hints, etc - and the size of these depend on the size of the repo.

Also, there could be outside influence on the free space if there is other stuff running on the borg repo server (so even if borg lets some space free / checks for free space according to additional_free_space, that could go away any time).

[ThomasWaldmann]

i had a quick look and yes, i think an option --additional-free-space=X similar to --storage-quota=Y would make sense.

[ThomasWaldmann]

and even without that, we could also change the repo config default value for additional_free_space to a non-zero value.
just wondering what value would fit most users / scenarios?

[fschulze]

Maybe ask if not given and suggest 2G with some hints or a pointer to the docs?

And borg config should allow setting these variables. Since the server actually changes the values, I see no security problem, as ssh and the server process stand in between.

[ThomasWaldmann]

@m3nu I think we should have 2G .. 5G as default value there.

Usually people will have hundreds of GBs local space for their backups. OTOH, there are borg storage providers like borgbase.com with relatively small (10GB) "for free" accounts. Do you have an idea how we can raise the default without impacting such an offer too much?

[ThomasWaldmann]

Hmm, guess whether that would make a difference for services like borgbase.com depends on whether they are using the borg-based quota (--storage-quota) or some external mechanism.

[m3nu]

We use both, file system quotas and Borg quotas. The latter don't work on append-only repos, I think.

We also set additional_free_space to 5M to have enough space for the lock file.

[m3nu]

Keeping 2GB available by default may be too much, since users can set the quota for a repo as low as 1 GB.

[ThomasWaldmann]

Hmm, difficult to come up with a good value for everybody.

[ThomasWaldmann]

See also #5313.