Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rework curator to make less slice allocations and close opend dirs #120

Merged
merged 1 commit into from
Apr 14, 2022
Merged

Rework curator to make less slice allocations and close opend dirs #120

merged 1 commit into from
Apr 14, 2022

Conversation

vlvasilev
Copy link
Collaborator

@vlvasilev vlvasilev commented Apr 14, 2022 •
edited
Loading

How to categorize this PR?

/kind enhancement
/area logging

What this PR does / why we need it:
With this PR Loki's curator does fewer slice allocations and closes the opened directories after each deletion.
Also, the pprof API is open on 2718 port.

Which issue(s) this PR fixes:
Could fixes #119

Special notes for your reviewer:

Release note:

Loki's curator does fewer slice allocations when deleting files.

Loki's curator closes the opened directories after each deletion. 

Loki's curator profiling is available via  HTTP `pprof` API open on `2718` port.

@gardener-robot gardener-robot added area/logging Logging related kind/enhancement Enhancement, improvement, extension needs/review Needs review size/xl Size of pull request is huge (see gardener-robot robot/bots/size.py) needs/second-opinion Needs second review by someone else labels Apr 14, 2022
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Apr 14, 2022
@gardener-robot-ci-3 gardener-robot-ci-3 added needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Apr 14, 2022
@Kristian-ZH
Copy link
Contributor

Do you need the test/loki-curator-benchmark/*.png files?

@vlvasilev vlvasilev force-pushed the rework-curaor-to-use-less-memory-when-deleting-diles branch from f21d90c to e1ce329 Compare April 14, 2022 12:45
@gardener-robot gardener-robot added size/m Size of pull request is medium (see gardener-robot robot/bots/size.py) and removed size/xl Size of pull request is huge (see gardener-robot robot/bots/size.py) labels Apr 14, 2022
@gardener-robot-ci-3 gardener-robot-ci-3 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Apr 14, 2022
@gardener-robot-ci-1 gardener-robot-ci-1 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Apr 14, 2022
Kristian-ZH
Kristian-ZH approved these changes Apr 14, 2022
View reviewed changes
Copy link
Contributor

@Kristian-ZH Kristian-ZH left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@gardener-robot gardener-robot added reviewed/lgtm Has approval for merging and removed needs/review Needs review needs/second-opinion Needs second review by someone else labels Apr 14, 2022
@vlvasilev vlvasilev force-pushed the rework-curaor-to-use-less-memory-when-deleting-diles branch from e1ce329 to fdd9b3d Compare April 14, 2022 13:44
@gardener-robot gardener-robot added size/l Size of pull request is large (see gardener-robot robot/bots/size.py) needs/second-opinion Needs second review by someone else and removed size/m Size of pull request is medium (see gardener-robot robot/bots/size.py) reviewed/lgtm Has approval for merging labels Apr 14, 2022
@gardener-robot-ci-3 gardener-robot-ci-3 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Apr 14, 2022
@gardener-robot-ci-1 gardener-robot-ci-1 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Apr 14, 2022
@vlvasilev vlvasilev merged commit c2d0137 into gardener:master Apr 14, 2022
@vpnachev vpnachev changed the title Rework curaor to make less slice allocations and close opend dirs Rework curator to make less slice allocations and close opend dirs Apr 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Kristian-ZH Kristian-ZH Kristian-ZH approved these changes

Assignees
No one assigned
Labels
area/logging Logging related kind/enhancement Enhancement, improvement, extension needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) needs/second-opinion Needs second review by someone else size/l Size of pull request is large (see gardener-robot robot/bots/size.py)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Loki curator container memory usage starts to gradual increase until is OOMKiled.
6 participants
@vlvasilev @Kristian-ZH @gardener-robot-ci-1 @gardener-robot-ci-2 @gardener-robot-ci-3 @gardener-robot