Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding support for native Kubernetes executor #755

Merged
merged 6 commits into from
Sep 12, 2024

Conversation

KrKOo
Copy link

@KrKOo KrKOo commented Dec 12, 2023

As stated in #748, running Docker-in-Docker for each job is not nearly as effective or secure as utilizing the Kubernetes API for the same purpose. This PR introduces optional native Kubernetes job execution.

How does it work:
When the worker is configured to use the Kubernetes executor, it initializes the environment in the same way as when using Docker. Then instead of running a Docker container, it creates another Kubernetes Job.

TODO:

  • Update the documention
  • Implement continuous log collection from the job (if needed)

@KrKOo KrKOo marked this pull request as ready for review December 12, 2023 00:23
@lbeckman314
Copy link
Member

Thank you for the PR @KrKOo! We'll start reviewing it and hopefully get in approved for the release pipeline. 🚀

@lbeckman314 lbeckman314 self-assigned this Dec 12, 2023
@lbeckman314 lbeckman314 linked an issue Dec 12, 2023 that may be closed by this pull request
@lbeckman314 lbeckman314 changed the base branch from master to develop July 30, 2024 18:06
@lbeckman314
Copy link
Member

Apologies for the glacial pace with merging this issue — we just tested it locally and everything looks great, thank you for the contribution!

Rebasing onto latest development branch now...

Copy link

netlify bot commented Sep 12, 2024

Deploy Preview for ohsu-comp-bio-funnel canceled.

Name Link
🔨 Latest commit da9d083
🔍 Latest deploy log https://app.netlify.com/sites/ohsu-comp-bio-funnel/deploys/66e37a9a6f67b60008fcdd4e

@lbeckman314 lbeckman314 merged commit e6877b2 into ohsu-comp-bio:develop Sep 12, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature Request: Add support for native K8s Executor
2 participants