-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support throttling vstreamer copy table work on source tablets #9923
Conversation
5d4a46a
to
b0e7f4c
Compare
f924c86
to
ca8468c
Compare
5c27419
to
4bbabfb
Compare
94b8a91
to
f33ee77
Compare
Signed-off-by: Matt Lord <[email protected]>
f33ee77
to
ac8564a
Compare
Signed-off-by: Matt Lord <[email protected]>
7af1c00
to
e89f98b
Compare
Signed-off-by: Matt Lord <[email protected]>
e89f98b
to
9de8182
Compare
Signed-off-by: Matt Lord <[email protected]>
fed244e
to
4fca26b
Compare
Signed-off-by: Matt Lord <[email protected]>
42e2108
to
07bd5c9
Compare
And make it a gauge so that it's always showing the current value as we allow this to be changed in the running process via /debug/env. Signed-off-by: Matt Lord <[email protected]>
2fce166
to
55cee55
Compare
55cee55
to
2c9b869
Compare
Signed-off-by: Matt Lord <[email protected]>
2c9b869
to
e60a3ce
Compare
Signed-off-by: Matt Lord <[email protected]>
Signed-off-by: Matt Lord <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
* Document work in vitessio/vitess#9923 Signed-off-by: Matt Lord <[email protected]> * Move to two dashes for long flag names Signed-off-by: Matt Lord <[email protected]> * Add detailed flag info and related info Signed-off-by: Matt Lord <[email protected]>
Starting this discussion here to follow up on @deepthi's DM observation that these flag names are too long. @mattlord and I did note that, but we thought more descriptive the better since they would anyway reside in scripts or config files. However as @deepthi points out this may contribute to hitting OS command line length limits and also shorter the better, for recall purposes. @mattlord, thoughts? To start off suggestions!: |
I'm not opposed to keeping the full name of the innodb variable in the flag for clarity. At a minimum, we can drop |
IMO related flags should all have the same prefix. I would have used
The I don't really agree that shorter is easier to remember, but I don't feel too strongly here. I'm not sure that OS limits are a major concern, but it's possible (1 MiB on macOS and 2MiB on Linux):
Please note that the entire --help output — which includes the help text — is currently less than 60KiB:
IMO what we really need is a project/ticket to define some Vitess policies for command-line options/flags and then apply them to all flags across all of vitess. Want me to create an issue for that? I don't mind being the one to work on it either — the lack of a policy is the real issue IMO (don't feel too strongly on the specifics, but the lack of uniformity and predictability is bad for users). |
Let's create an issue/task saying "we need to define a policy", and once there is a concrete proposal it can become an RFC. The current state is not good. We also have some flags with dashes and some with underscores. |
Description
This provides mechanisms to limit the impact of large vreplication copy table (row streamer) workflow phases on production source tablets.
It supports throttling based on the InnoDB history list length and the MySQL replication lag seen (if the source is a replica). This gives vreplication source tablets an opportunity to catch up in between copy table phase cycles (copy, catchup, fastforward). You can control this using two new vttablet flags:
Related Issue(s)
Checklist