-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ui,admission: observability improvements for admission control #68595
Conversation
@dhartunian I don't know what to make of the lint failure -- I did not add a Tooltip. Could you help?
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we'll want to do a review of all new admission-control-related graphs before release, together with PM
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @ajwerner and @sumeerbhola)
pkg/ui/src/views/cluster/containers/nodeGraphs/dashboards/overload.tsx, line 149 at r1 (raw file):
<LineGraph title="Admission Latency Rate" sources={nodeSources}> <Axis label="latency rate (micros/sec)">
This is pretty obscure to be looking at directly.. It's hard to reason about what this rate means. I guess it is not possible to show the wait_sum / admitted
graph without calculating a new metric?
pkg/ui/src/views/cluster/containers/nodeGraphs/dashboards/overload.tsx, line 179 at r1 (raw file):
<LineGraph title="Admission Latency: 75th percentile" sources={nodeSources}> <Axis label="latency for requests that waited (nanos)">
"latency" can be confusing (could refer to the overall execution latency). Maybe admission delay or wait time?
@sumeerbhola Filling in for @dhartunian, the several warnings about using
We use a code formatter called Prettier as a plugin for our linter Eslint. Often folks working with React will have tools builtin to handle the formatting automatically. We do have a yarn command to auto-format files, but I'm sorry to say it is not something that is a part of the regular build in
Please let me know if you run into any issues or if I can provide more clarification. |
- Trace statements for latency incurred in admission queues. - Certain admission control metrics are now included in the overload dashboard. Specifically, - Resource bottlenecks can be identified using the "KV Admission Slots" and "KV Admission IO Tokens Exhausted Duration Per Second" graphs. - The rate at which admission control is admitting requests is in the "Admission Work Rate" graphs and the corresponding delay rate (for all requests) is in "Admission Delay Rate". Dividing the latter by the former gives the mean admission delay. - The 75th percentile delay for those requests that actually waited for admission is in the "Admission Delay: 75th percentile" graph. When admission control is off most of these graphs will be empty or zero, and the total KV admission slots will be 1. Informs cockroachdb#65955 Release note (ui change): admission control metrics are added to Overload dashboard.
a32b990
to
5e44a7f
Compare
TFTRs @RaduBerinde @nathanstilwell ! |
bors r+ |
Build succeeded: |
overload dashboard. Specifically,
"KV Admission Slots" and "KV Admission IO Tokens Exhausted
Duration Per Second" graphs.
is in the "Admission Work Rate" graphs and the corresponding
latency rate (for all requests) is in
"Admission Latency Rate". Dividing the latter by the former
gives the mean admission latency.
waited for admission is in the
"Admission Latency: 75th percentile" graph.
When admission control is off most of these graphs will be
empty or zero, and the total KV admission slots will be 1.
Informs #65955
Release note (ui change): admission control metrics are added to
Overload dashboard.