Allow per project cache #15
Comments
|
That's a good feature. But it would be helpful to make it optional with a flag, as there is use case for different projects sharing cache objects |
|
I started a prove-of-concept patch for on-disk storage here: https://github.com/gjasny/bazel-remote/tree/project_name I noticed that for |
|
the CAS only needs to enforce this for the last segment in the URL, so it's pretty easy to do this. there's an implementation here that inspired this issue: https://github.com/kubernetes/test-infra/tree/master/greenhouse |
In some client setups, untracked local files can be used by an action without being included in the Action message, which causes action cache collisions: bazelbuild/bazel#4558 Ideally this should be fixed on the client side (either in the client, or in the build configuration), but it is not always easy to do in practice. As a workaround, this patch adds a setting to mangle ActionCache keys with the instance name provided by the client (if it is not empty), to produce a new ActionCache key. Clients are then able to specify a different instance name whenever a change is made that could affect these untracked inputs. The instance name value could be something like the hash of the compiler version. This allows multiple ActionCache items to exist in the cache, without requiring a change to the on-disk storage format. This feature is disabled by default, since it would cause cache invalidations for existing users. Fixes buchgr#15.
In some client setups, untracked local files can be used by an action without being included in the Action message, which causes action cache collisions: bazelbuild/bazel#4558 Ideally this should be fixed on the client side (either in the client, or in the build configuration), but it is not always easy to do in practice. As a workaround, this patch adds a setting to mangle ActionCache keys with the instance name provided by the client (if it is not empty), to produce a new ActionCache key. Clients are then able to specify a different instance name whenever a change is made that could affect these untracked inputs. The instance name value could be something like the hash of the compiler version. This allows multiple ActionCache items to exist in the cache, without requiring a change to the on-disk storage format. This feature is disabled by default, since it would cause cache invalidations for existing users. Fixes #15.
|
@mostynb Looking through some logs with this in place I see the following:
|
|
@sitsofe: This flag only mangles "AC" hash keys, which are prone to key collisions if the client can't hash all of the inputs. |
@BenTheElder mentioned to me that k8s is using a different remote cache for different projects and they distinguish between projects via a path component in the URL:
https://cache.com/<project name>/(ac|cas)/I think that's a good idea and I believe @nicolov already laid the foundations for that.
The text was updated successfully, but these errors were encountered: