Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(repo-server): excess git requests, improve cache performance by adding two level cache #3

Open
wants to merge 4 commits into
base: lsremote-part3
Choose a base branch
from

Conversation

nromriell
Copy link
Owner

Fixes argoproj#14725

Fourth part of a split of the argoproj#16309 and follow-up to argoproj#16410

Another piece of the fixes for argoproj#14725. This PR improves the cache performance and removes the added overhead from introducing the cache level lock to solve the excess ls-remote calls

  • Extends the CacheActionOpts introduced in part 3 to add CacheType. This field can be used to skip the local in-memory or external cache allowing us to replace current redis only caches with two level caches in place without having to swap everything over to using both.
  • Swaps the cache type for reposerver/cache to two level cache
  • Adds logic to two level cache to support the CacheType action option
  • Modifies the Cache interface to move the Get call to match the Set call interface with just a single Item input which accepts the action options.

Baseline v2.9.0 with 200 multi-source applications with a ref only source:
Screenshot from 2023-11-18 18-24-30

With all of the changes from the original PR:
Screenshot from 2023-11-18 19-33-39

Checklist:

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
  • The title of the PR states what changed and the related issues number (used for the release note).
  • The title of the PR conforms to the Toolchain Guide
  • I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
  • I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
  • Does this PR require documentation updates?
  • I've updated documentation as required by this PR.
  • Optional. My organization is added to USERS.md.
  • I have signed off all my commits as required by DCO
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My build is green (troubleshooting builds).
  • My new feature complies with the feature status guidelines.
  • I have added a brief description of why this PR is necessary and/or what this PR solves.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant