Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mimir-operations: Ensure integer auto-scaling thresholds #3512

Merged
merged 4 commits into from
Nov 24, 2022

Conversation

aknuds1
Copy link
Contributor

@aknuds1 aknuds1 commented Nov 24, 2022

What this PR does

Ensure that auto-scaling thresholds in mimir-operations are integers.

Which issue(s) this PR fixes or relates to

We've sometimes made the mistake of setting non-integer KEDA thresholds due to this logic not enforcing the expectation of them being integers.

Checklist

  • Tests updated
  • [na] Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@grafanabot

This comment was marked as off-topic.

@aknuds1 aknuds1 force-pushed the chore/autoscaling-int-threshold branch 3 times, most recently from c278206 to 37b2170 Compare November 24, 2022 09:55
@aknuds1 aknuds1 force-pushed the chore/autoscaling-int-threshold branch from 37b2170 to 94cdacd Compare November 24, 2022 09:57
@aknuds1 aknuds1 marked this pull request as ready for review November 24, 2022 09:58
@aknuds1 aknuds1 requested a review from a team as a code owner November 24, 2022 09:58
@aknuds1 aknuds1 changed the title WIP: mimir-operations: Ensure integer auto-scaling thresholds mimir-operations: Ensure integer auto-scaling thresholds Nov 24, 2022
@aknuds1 aknuds1 force-pushed the chore/autoscaling-int-threshold branch from 94cdacd to 6a3deb7 Compare November 24, 2022 10:24
)
);

std.split(std.toString(siToBytesDecimal(str)), '.')[0]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pracucci @jhesketh I followed your advice of fixing siToBytes to return an integer represented as a string), WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are using std.toString(std.parseInt(trigger.threshold)) above, do we need this here?

siToBytes could still return a number (preferably an integer, but decimals shouldn't matter if it's easier to ignore it for now). It's KEDA itself that expects a number in a string so we could do the string conversion just at that point.

Copy link
Contributor Author

@aknuds1 aknuds1 Nov 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are using std.toString(std.parseInt(trigger.threshold)) above, do we need this here?

@jhesketh std.parseInt will fail if e.g. it's a string representing a decimal nmber, so it is necessary.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we guarantee that siToBytesDecimal() always return a number, then here we just use std.ceil(siToBytesDecimal(str))?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pracucci aside from the technique for converting to a number, do you prefer rounding up (ceil) to rounding down, since I'm currently doing the latter?

Copy link
Contributor Author

@aknuds1 aknuds1 Nov 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I converted siToBytesDecimal to return a number, awaiting @pracucci's reply regarding floor vs ceil. I figured rounding down would be OK personally, but not sure what others think.

@aknuds1 aknuds1 force-pushed the chore/autoscaling-int-threshold branch from 6a3deb7 to 128f7cb Compare November 24, 2022 10:28
Copy link
Contributor

@jhesketh jhesketh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to do this as it should work, but also wondering if we can neaten the type handling up a bit. The type expectation from KEDA is silly :-s

operations/mimir/autoscaling.libsonnet Show resolved Hide resolved
)
);

std.split(std.toString(siToBytesDecimal(str)), '.')[0]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are using std.toString(std.parseInt(trigger.threshold)) above, do we need this here?

siToBytes could still return a number (preferably an integer, but decimals shouldn't matter if it's easier to ignore it for now). It's KEDA itself that expects a number in a string so we could do the string conversion just at that point.

Copy link
Contributor

@bboreham bboreham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you state in a comment what the conversion is - looks like truncate-towards-zero.

Also I would clarify that siToBytes is now returning a string. Perhaps name it siToBytesString.

@aknuds1
Copy link
Contributor Author

aknuds1 commented Nov 24, 2022

Can you state in a comment what the conversion is - looks like truncate-towards-zero.

Also I would clarify that siToBytes is now returning a string. Perhaps name it siToBytesString.

Thanks @bboreham, will do.

@aknuds1 aknuds1 force-pushed the chore/autoscaling-int-threshold branch from 128f7cb to c366640 Compare November 24, 2022 13:09
In Jsonnet logic to generate KEDA auto-scaling thresholds, ensure that
thresholds are integers since KEDA requires this.

Signed-off-by: Arve Knudsen <[email protected]>
@aknuds1 aknuds1 force-pushed the chore/autoscaling-int-threshold branch from c366640 to 08a8209 Compare November 24, 2022 13:12
@@ -137,21 +139,27 @@
},
},

local siToBytes(str) = (
local siToBytesString(str) = (
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From a design perspective, I think this should be a generic utility returning a number and then if the call needs a string it will cast it to string (which is what were were doing before).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pracucci actually before it could return either a number or a string.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pracucci I converted it now into returning an integer.

Signed-off-by: Arve Knudsen <[email protected]>
Signed-off-by: Arve Knudsen <[email protected]>
@aknuds1 aknuds1 enabled auto-merge (squash) November 24, 2022 13:38
@aknuds1 aknuds1 merged commit 3ddf418 into main Nov 24, 2022
@aknuds1 aknuds1 deleted the chore/autoscaling-int-threshold branch November 24, 2022 13:39
masonmei pushed a commit to udmire/mimir that referenced this pull request Dec 16, 2022
* mimir-operations: Ensure integer auto-scaling thresholds

In Jsonnet logic to generate KEDA auto-scaling thresholds, ensure that
thresholds are integers since KEDA requires this.

Signed-off-by: Arve Knudsen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants