Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update operations are non atmoic across processes #2023

Open
Alphasite opened this issue Aug 1, 2024 · 1 comment
Open

Update operations are non atmoic across processes #2023

Alphasite opened this issue Aug 1, 2024 · 1 comment
Labels
bug Something isn't working needs-investigation

Comments

@Alphasite
Copy link

Alphasite commented Aug 1, 2024

What happened:
Ran multiple instances of grype CLI and the database ended up invalid, failing the integrity checks.

What you expected to happen:
It should be parallel safe.

How to reproduce it (as minimally and precisely as possible):
Spin up multiple scan tasks in an environment without the database downloaded.

Anything else we need to know?:

Its likely this specific section of code:

	// activate the new db cache
	return file.CopyDir(c.fs, dbDirPath, c.dbDir)

I don't know if there is a complete solution but as i understand it using a symlink or a rename would probably be an atomic operation which doesn't have as much risk of concurrent issues which this current approach of delete and then copy has.

This covers a similar issue: https://stackoverflow.com/questions/307437/moving-a-directory-atomically for ideas.

Environment:

  • Output of grype version:
  • OS (e.g: cat /etc/os-release or similar): Ubuntu Trusty
@Alphasite Alphasite added the bug Something isn't working label Aug 1, 2024
@Alphasite Alphasite changed the title Update operations are non atmoic Update operations are non atmoic across processes Aug 1, 2024
@willmurphyscode
Copy link
Contributor

Hi @Alphasite, thanks for the detailed report!

You are right that a symlink update or rename would probably be more atomic, but we've had issues in the past, for example renaming fails if the old path and new path point to different volumes, at least on some operating systems.

That said, I think this is a reasonable expectation for users of Grype to have. We'll need to investigate how to do it in a cross platform way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs-investigation
Projects
Status: Backlog
Development

No branches or pull requests

2 participants