Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty log folders are not removed when clean up #33162

Closed
1 of 2 tasks
ckljohn opened this issue Aug 7, 2023 · 1 comment · Fixed by #33252
Closed
1 of 2 tasks

Empty log folders are not removed when clean up #33162

ckljohn opened this issue Aug 7, 2023 · 1 comment · Fixed by #33252
Labels
affected_version:main_branch Issues Reported for main branch area:core good first issue kind:bug This is a clearly a bug

Comments

@ckljohn
Copy link
Contributor

ckljohn commented Aug 7, 2023

Apache Airflow version

main (development)

What happened

Empty log folders use up all Inodes and they are not removed by clean-logs.sh

This is the diff after cleaning empty folders. (50GB disk used)

airflow@airflow-worker-2:/opt/airflow/logs$ df -i
Filesystem       Inodes   IUsed    IFree IUse% Mounted on
/dev/nvme2n1    3276800 1311542  1965258   41% /opt/airflow/logs
airflow@airflow-worker-2:/opt/airflow/logs$ find . -type d -empty -delete
airflow@airflow-worker-2:/opt/airflow/logs$ df -i
Filesystem       Inodes  IUsed    IFree IUse% Mounted on
/dev/nvme2n1    3276800 158708  3118092    5% /opt/airflow/logs

What you think should happen instead

No response

How to reproduce

Have lots of frequent DAGs.

Operating System

Debian 11

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@ckljohn ckljohn added area:core kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet labels Aug 7, 2023
@potiuk
Copy link
Member

potiuk commented Aug 7, 2023

Marked as good first issue, but mayeb you would like to fix it yourself? it does not seem complex and I bet you already have a solution for that in your environment? Contributing it back seems like a good idea if you do

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affected_version:main_branch Issues Reported for main branch area:core good first issue kind:bug This is a clearly a bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants