Regression when creating backup: backup file equal to disk size is created

[pat-s]

• Gitea version: 1.13-rc2
• Git version: 2.26.2
• Operating system: ubuntu 20.04 (Gitea via docker)
• Database (use [x]):
  • PostgreSQL
  • MySQL
  • MSSQL
  • SQLite
• Can you reproduce the bug at https://try.gitea.io:
  • Yes (provide example URL)
  • No
• Log gist:

Description

When dumping a backup of a gitea docker install via

docker exec -u git gitea bash -c 'gitea dump -w /tmp -c /data/gitea/conf/app.ini -f /data/gitea/conf/backup_gitea_$(date +\%d-\%m-\%Y).zip'

the output file that gets created fills up the whole disk space within ~10 seconds.

However, when executing the commands from above in parts, the backups is created just fine and finishes in < 1 sec:

docker exec -it gitea bash
su - git
gitea dump -w /tmp -c /data/gitea/conf/app.ini -f /data/gitea/conf/backup_gitea_assets_$(date +\%d-\%m-\%Y).zip

I also tried this on 1.14.0+dev-233-g0615b668d.

docker version
Client: Docker Engine - Community
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:02:52 2020
 OS/Arch:           linux/amd64
 Experimental:      false

[pat-s]

I digged a bit deeper and turning on verbosity I found the following

bad case (docker exec)

docker exec --user git gitea bash -c 'gitea dump -V -w /tmp -c /data/gitea/conf/app.ini -f /data/gitea/conf/backup_gitea_assets_$(date +\%d-\%m-\%Y).zip'

2020/11/19 15:16:41 cmd/dump.go:278:runDump() [I] Dumping database...
2020/11/19 15:16:41 cmd/dump.go:32:addFile() [I] Adding file gitea-db.sql
2020/11/19 15:16:41 cmd/dump.go:290:runDump() [I] Adding custom configuration file from /data/gitea/conf/app.ini
2020/11/19 15:16:41 cmd/dump.go:32:addFile() [I] Adding file app.ini
2020/11/19 15:16:41 cmd/dump.go:55:addRecursive() [I] Adding dir  custom
2020/11/19 15:16:41 cmd/dump.go:55:addRecursive() [I] Adding dir  custom/conf
2020/11/19 15:16:41 cmd/dump.go:32:addFile() [I] Adding file custom/conf/backup_gitea_assets_19-11-2020.zip
Failed to include custom: custom/conf/backup_gitea_assets_19-11-2020.zip: copying contents: write /data/gitea/conf/backup_gitea_assets_19-11-2020.zip: no space left on device

good case (interactive in the container)

gitea dump -V -w /tmp -c /data/gitea/conf/app.ini -f /data/gitea/conf/backup_gitea_assets_$(date +\%d-\%m-\%Y).zip

2020/11/19 15:18:40 cmd/dump.go:278:runDump() [I] Dumping database...
2020/11/19 15:18:41 cmd/dump.go:32:addFile() [I] Adding file gitea-db.sql
2020/11/19 15:18:41 cmd/dump.go:290:runDump() [I] Adding custom configuration file from /data/gitea/conf/app.ini
2020/11/19 15:18:41 cmd/dump.go:32:addFile() [I] Adding file app.ini
2020/11/19 15:18:41 cmd/dump.go:306:runDump() [I] Custom dir /usr/local/bin/custom doesn't exist, skipped
2020/11/19 15:18:41 cmd/dump.go:55:addRecursive() [I] Adding dir  log
2020/11/19 15:18:41 cmd/dump.go:370:runDump() [I] Finish dumping in file /data/gitea/conf/backup_gitea_assets_19-11-2020.zip

Note the difference after Adding file app.ini.

Unfortunately setting -C somevalue did not work: docker exec --user git gitea bash -c 'gitea dump -V -w /tmp -c /data/gitea/conf/app.ini -C /foo -f /data/gitea/conf/backup_gitea_assets_$(date +\%d-\%m-\%Y).zip'.

2020/11/19 15:24:09 cmd/dump.go:278:runDump() [I] Dumping database...
2020/11/19 15:24:09 cmd/dump.go:32:addFile() [I] Adding file gitea-db.sql
2020/11/19 15:24:09 cmd/dump.go:290:runDump() [I] Adding custom configuration file from /data/gitea/conf/app.ini
2020/11/19 15:24:09 cmd/dump.go:32:addFile() [I] Adding file app.ini
2020/11/19 15:24:09 cmd/dump.go:306:runDump() [I] Custom dir /foo doesn't exist, skipped
2020/11/19 15:24:09 cmd/dump.go:310:runDump() [I] Packing data directory...data
2020/11/19 15:24:09 cmd/dump.go:32:addFile() [I] Adding file data/gitea/conf/backup_gitea_assets_19-11-2020.zip
Failed to include data directory: data/gitea/conf/backup_gitea_assets_19-11-2020.zip: copying contents: write /data/gitea/conf/backup_gitea_assets_19-11-2020.zip: no space left on device
2020/11/19 15:24:13 cmd/dump.go:171:fatal() [F] Failed to include data directory: data/gitea/conf/backup_gitea_assets_19-11-2020.zip: copying contents: write /data/gitea/conf/backup_gitea_assets_19-11-2020.zip: no space left on device

[pat-s]

Is there a better way of doing backups via docker?

This blocks us since we are unable to do automatic backups.

I also added docker version info to the OP.

[6543]

Is there a better way of doing backups

@pat-s you just have to save: the database, config-file & data folder. so you could simply create your own backup solution based on your enviroment ... e.g. docker

[pat-s]

right! I was loosing focus.

We are already backing up the DB and I could simply zip the other assets via the mounted volume 🤦

DBNAME=gitea; zip -q -r backup_$(echo $DBNAME)_$(date +\%d-\%m-\%Y)_assets.zip /home/ubuntu/gitea/gitea
s3cmd put /home/ubuntu/backup_$(echo $DBNAME)_$(date +\%d-\%m-\%Y)_assets.zip s3://s3-name/backup_$(echo $DBNAME)_$(date +\%d-\%m-\%Y)_assets.zip
rm /home/ubuntu/backup_gitea_$(date +\%d-\%m-\%Y)_assets.zip

[Alsiri0n]

Same problem on 1.13.1 version.

[Alsiri0n]

Same problem on 1.13.2 version.

[zeripath]

OK this is a not a regression - just a change in behaviour. It was never expected that you would store your backups in the custom directory and within the path of things added to the dump.

I've provided a PR to prevent adding the current dump file to the dump and to provide an option to prevent adding the custom dir.