-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bugfix clean-logs, avoid crushed container when folder not found #36050
Conversation
scripts/docker/clean-logs.sh
Outdated
@@ -35,7 +35,7 @@ while true; do | |||
-type f -mtime +"${RETENTION}" -name '*.log' -print0 | \ | |||
xargs -0 rm -f | |||
|
|||
find "${DIRECTORY}"/logs -type d -empty -delete | |||
trap find "${DIRECTORY}"/logs -type d -empty -delete ERR |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This fix is simple and works but the disadvantage of it that every ERR is except here.
There is other solution that will except the specific file not found exception:
error_file=$(mktemp)
find "${DIRECTORY}"/logs -type d -empty -delete
if grep -q "No such file or directory" "$error_file"; then
cat "$error_file"
echo "Info: if you run with hight available scheduler probably the other scheduler already delete it"
fi
rm "$error_file"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not just find "${DIRECTORY}"/logs -type d -empty -delete || true
? That will ignore any errors. I think it should be pretty enough.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep. you are right. :)
d7e5ac5
to
6a4b06e
Compare
(cherry picked from commit 61fd166)
Follow up: #33162
In my case I have high available scheduler (2 schedulers).
When I upload the chart on kubernetes I got exception inside the container scheduler-log-groomer:
and the container got restart.
I think it's a race condition when more then 1 scheduler runs log-grommer container and try to delete the same directory.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.