Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kubernetes PodManager's xcom sidecar kill command is invalid for containers where /bin/sh is dash #37903

Closed
2 tasks done
pdebelak opened this issue Mar 5, 2024 · 0 comments · Fixed by #37905
Closed
2 tasks done
Assignees
Labels
area:core kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet

Comments

@pdebelak
Copy link
Contributor

pdebelak commented Mar 5, 2024

Apache Airflow version

2.8.2

If "Other Airflow 2 version" selected, which one?

No response

What happened?

When using a kubernetes connection that has an XCom sidecar image value set to an image where /bin/sh is dash, the pod manager attempts to run kill -s SIGINT 1 using /bin/sh which prints kill: invalid signal number or name: SIGINT and fails to kill the process. This works fine with the default alpine image, but not with debian-based ones (like the default python image). See

def extract_xcom_kill(self, pod: V1Pod):
for the code for this.

What you think should happen instead?

The pod manager should either let you customize the shell used (so I can specify /bin/bash instead of /bin/sh) or should use the more portable signal numbers instead (kill -2 1, which is the equivalent command) which should work with any container with a POSIX /bin/sh.

How to reproduce

Set the XCom sidecar image for a kubernetes connection to python:latest and then use that connection to start a task with the KubernetesPodOperator with do_xcom_push=True. Notice that the sidecar prints an error and doesn't terminate.

Operating System

Debian GNU/Linux

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:core kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant