Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(backup): add parallel backup creation using pigz #4509

Merged

Conversation

MicLieg
Copy link
Contributor

@MicLieg MicLieg commented Feb 25, 2024

Description

This pull request changes the creation of a backup to use pigz to reduce compression times by parallelizing compression. To do that pigz was added to the dependencies.

Backup times on i7-13700K bare metal server

Default Backup

Small Minecraft Server (117 MB) took ~ 5 seconds

user@hostmachine:~/minecraft$ time ./mcserver backup
[ INFO ] Backing up mcserver: Starting backup: No previous backups found
[  OK  ] Backing up mcserver: Completed: mcserver-2024-02-25-163835.tar.gz, total size 107M
core_exit.sh exiting with code: 4

real    0m5,122s

Big Minecraft Server (18 GB) took ~ 285 seconds (4 minutes 45 seconds)

user@hostmachine:~/minecraft$ time ./mcserver backup
[ INFO ] Backing up mcserver: Starting backup: No previous backups found
[  OK  ] Backing up mcserver: Completed: mcserver-2024-02-25-165601.tar.gz, total size 15G
core_exit.sh exiting with code: 4

real    4m44,821s

Parallel Backup

Small Minecraft Server (117 MB) took ~ 3 seconds

user@hostmachine:~/minecraft$ time ./mcserver backup
[ INFO ] Backing up mcserver: Starting backup: No previous backups found
[  OK  ] Backing up mcserver: Completed: mcserver-2024-02-25-164608.tar.gz, total size 107M
core_exit.sh exiting with code: 4

real    0m3,197s

Big Minecraft Server (18 GB) took ~ 23 seconds

user@hostmachine:~/minecraft$ time ./mcserver backup
[ INFO ] Backing up mcserver: Starting backup: No previous backups found
[  OK  ] Backing up mcserver: Completed: mcserver-2024-02-25-170138.tar.gz, total size 15G
core_exit.sh exiting with code: 4

real    0m23,154s

Fixes #3422

Type of change

  • Bug fix (a change which fixes an issue).
  • New feature (a change which adds functionality).
  • New Server (new server added).
  • Refactor (restructures existing code).
  • Comment update (typo, spelling, explanation, examples, etc).

Checklist

PR will not be merged until all steps are complete.

  • This pull request links to an issue.
  • This pull request uses the develop branch as its base.
  • This pull request subject follows the Conventional Commits standard.
  • This code follows the style guidelines of this project.
  • I have performed a self-review of my code.
  • I have checked that this code is commented where required.
  • I have provided a detailed enough description of this PR.
  • I have checked if documentation needs updating.

Documentation

If documentation does need updating either update it by creating a PR (preferred) or request a documentation update.

Thank you for your Pull Request!

@MicLieg
Copy link
Contributor Author

MicLieg commented Feb 25, 2024

To improve this, a new setting for the backupmode="fast" / "slow" could be added in the _default.cfg file to configure whether the slower, resource-saving or the faster backup mode should be used; which can easily cause 100% CPU utilisation on all cores.

@dgibbs64 dgibbs64 self-requested a review February 26, 2024 13:56
@dgibbs64
Copy link
Member

Looks good to me.

@dgibbs64 dgibbs64 added this to the v24.2.0 milestone Feb 26, 2024
@dgibbs64 dgibbs64 merged commit 476ed02 into GameServerManagers:develop Feb 26, 2024
3 checks passed
MicLieg added a commit to MicLieg/LinuxGSM that referenced this pull request Mar 1, 2024
@MicLieg MicLieg deleted the feature/parallel-backup-creation branch March 9, 2024 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants