corrupted container package

[Fogapod]

Description

i have a problem with a specific docker package in my instance. seems like some file or database corruption took place and this package cannot be pulled properly anymore. all other containers work.
i tried deleting all instance container packages using script multiple times, pushing new images to that container, it does not help.

first got this issue on version 1.17.1, updating to 1.17.3 did not help.

pulling image

$ docker pull gitea.example.org/org/repo:test
test: Pulling from org/repo
e9995326b091: Pull complete 
71689475aec2: Pull complete 
f88a23025338: Pull complete 
0df440342e26: Retrying in 1 second 
eef26ceb3309: Retrying in 3 seconds 
8e3ed6a9e43a: Retrying in 4 seconds 
f22600e0d2b9: Waiting 
2aac9dd0923a: Waiting

meanwhile in gitea

Nov 09 08:31:11 ip-172-31-34-149 gitea[548]: 2022/11/09 08:31:11 ...ntainer/container.go:84:apiError() [E] [636b654f-6] open /var/lib/gitea/data/packages/0d/f4/0df440342e265c89de536643c3376dadf44c810fe2fb2b2ee44711f8661ce531: no such file or directory
Nov 09 08:31:11 ip-172-31-34-149 gitea[548]: 2022/11/09 08:31:11 [636b654f-6] router: completed GET /v2/org/repo/blobs/sha256:0df440342e265c89de536643c3376dadf44c810fe2fb2b2ee44711f8661ce531 for 1.1.1.1:0, 500 Internal Server Error in 4.8ms @ container/container.go:391(container.GetBlob)
Nov 09 08:31:12 ip-172-31-34-149 gitea[548]: 2022/11/09 08:31:12 [636a958d-23] router: polling   GET /user/events for 180.180.72.41:0, elapsed 3750.4ms @ events/events.go:19(events.Events)
Nov 09 08:31:12 ip-172-31-34-149 gitea[548]: 2022/11/09 08:31:12 ...ntainer/container.go:84:apiError() [E] [636b6550] open /var/lib/gitea/data/packages/ee/f2/eef26ceb3309d6e72a4402c3f16b047416adecaf91bc5360ebd4205d1ef5e310: no such file or directory
Nov 09 08:31:12 ip-172-31-34-149 gitea[548]: 2022/11/09 08:31:12 [636b6550] router: completed GET /v2/org/repo/blobs/sha256:eef26ceb3309d6e72a4402c3f16b047416adecaf91bc5360ebd4205d1ef5e310 for 1.1.1.1:0, 500 Internal Server Error in 4.8ms @ container/container.go:391(container.GetBlob)
Nov 09 08:31:12 ip-172-31-34-149 gitea[548]: 2022/11/09 08:31:12 ...ntainer/container.go:84:apiError() [E] [636b6550-2] open /var/lib/gitea/data/packages/ee/f2/eef26ceb3309d6e72a4402c3f16b047416adecaf91bc5360ebd4205d1ef5e310: no such file or directory
Nov 09 08:31:12 ip-172-31-34-149 gitea[548]: 2022/11/09 08:31:12 [636b6550-2] router: completed GET /v2/org/repo/blobs/sha256:eef26ceb3309d6e72a4402c3f16b047416adecaf91bc5360ebd4205d1ef5e310 for 1.1.1.1:0, 500 Internal Server Error in 5.4ms @ container/container.go:391(container.GetBlob)
Nov 09 08:31:13 ip-172-31-34-149 gitea[548]: 2022/11/09 08:31:13 ...ntainer/container.go:84:apiError() [E] [636b6551] open /var/lib/gitea/data/packages/8e/3e/8e3ed6a9e43aaf17bc2175fa2c7ae2ee24cff4d32bf82adf1ea689db4dcf15e1: no such file or directory
Nov 09 08:31:13 ip-172-31-34-149 gitea[548]: 2022/11/09 08:31:13 [636b6551] router: completed GET /v2/org/repo/blobs/sha256:8e3ed6a9e43aaf17bc2175fa2c7ae2ee24cff4d32bf82adf1ea689db4dcf15e1 for 1.1.1.1:0, 500 Internal Server Error in 4.4ms @ container/container.go:391(container.GetBlob)
Nov 09 08:31:13 ip-172-31-34-149 gitea[548]: 2022/11/09 08:31:13 ...ntainer/container.go:84:apiError() [E] [636b6551-2] open /var/lib/gitea/data/packages/8e/3e/8e3ed6a9e43aaf17bc2175fa2c7ae2ee24cff4d32bf82adf1ea689db4dcf15e1: no such file or directory
Nov 09 08:31:13 ip-172-31-34-149 gitea[548]: 2022/11/09 08:31:13 [636b6551-2] router: completed GET /v2/org/repo/blobs/sha256:8e3ed6a9e43aaf17bc2175fa2c7ae2ee24cff4d32bf82adf1ea689db4dcf15e1 for 1.1.1.1:0, 500 Internal Server Error in 4.4ms @ container/container.go:391(container.GetBlob)

can i somehow completely reset container packages in instance?

Gitea Version

1.17.3

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

No response

Operating System

Debian 5.10.149-2 (2022-10-21) x86_64 GNU/Linux

How are you running Gitea?

locally compiled

Database

PostgreSQL

[lunny]

You can delete all packages from admin panel -> packages

[Fogapod]

You can delete all packages from admin panel -> packages

there is no such option for me (gitea 1.17.3)

[KN4CK3R]

You do not need to delete all packages. Just delete the failing package (and all other images which share the missing blobs). Then add/modify your app.ini:

[cron.cleanup_packages]
;; Unreferenced blobs created more than OLDER_THAN ago are subject to deletion
OLDER_THAN = 1s

Then you can run the package cleanup job from the admin panel to remove the non-existing package blobs from the database. After that you should be able to upload the package again.

Or you delete the missing blobs from the package_blob table directly and upload the image again.

[Fogapod]

Running cron solved this, thanks. My issue is solved but i dont know what caused it initially, never touched any files on server. Could be deleting packages from admin tab