Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Throttle requests from the beginning #1199

Merged
merged 1 commit into from
Mar 10, 2021
Merged

Throttle requests from the beginning #1199

merged 1 commit into from
Mar 10, 2021

Conversation

danielmitterdorfer
Copy link
Member

With this commit we consult the scheduler prior to issuing a request
instead of afterwards. If we don't do that, clients can coordinate
creating a large initial load spike in Elasticsearch. With this
countermeasure, it is possible that clients avoid this initial spike if
a non-determistic scheduler, such as the poisson scheduler is chosen.

Relates #1195

@danielmitterdorfer
Throttle requests from the beginning
cabdc63 
With this commit we consult the scheduler prior to issuing a request
instead of afterwards. If we don't do that, clients can coordinate
creating a large initial load spike in Elasticsearch. With this
countermeasure, it is possible that clients avoid this initial spike if
a non-determistic scheduler, such as the poisson scheduler is chosen.

Relates elastic#1195
@danielmitterdorfer danielmitterdorfer added enhancement Improves the status quo :Load Driver Changes that affect the core of the load driver such as scheduling, the measurement approach etc. labels Feb 24, 2021
@danielmitterdorfer danielmitterdorfer added this to the 2.1.0 milestone Feb 24, 2021
@danielmitterdorfer danielmitterdorfer self-assigned this Feb 24, 2021
gingerwizard
gingerwizard approved these changes Feb 25, 2021
View reviewed changes
Copy link

@gingerwizard gingerwizard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Maybe worth noting in the migration guide that this may cause different results where large number of clients are used?

@danielmitterdorfer danielmitterdorfer merged commit c0b33fb into elastic:master Mar 10, 2021
@danielmitterdorfer danielmitterdorfer deleted the throttle-at-start branch March 10, 2021 10:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gingerwizard gingerwizard gingerwizard approved these changes

Assignees

@danielmitterdorfer danielmitterdorfer

Labels
enhancement Improves the status quo :Load Driver Changes that affect the core of the load driver such as scheduling, the measurement approach etc.
Projects
None yet
Milestone
2.1.0
Development

Successfully merging this pull request may close these issues.
2 participants
@danielmitterdorfer @gingerwizard