Stale action doesn't seem to be working #35144
Comments
|
Thanks @mmarchini! I'll see if there's any obvious optimisations we can try to avoid hitting that limit. |
|
We can probably optimize it by using the |
The auto closing of issues & PRs labelled with `stalled` doesn't seem to be working as expected. The GitHub Action UI gives the impression the stale action tries to execute more operations than it is allowed to do. Previously there was no filtering on issues & PRs. So when it tries to fetch all the currently open issues, checking whether or not they should be get closed, it would have to perform quite a few requests pagination requests to get the information needed. Knowing that we only care about issues & PRs already labelled `stalled`, we can provide the [`only-labels`](https://github.com/actions/stale/blob/13b324e4b28a2708236aadb11361fa65af60d201/action.yml#L38) option to make sure we only fetch relevant issues. Refs nodejs#35144
The auto closing of issues & PRs labelled with `stalled` doesn't seem to be working as expected. The GitHub Action UI gives the impression the stale action tries to execute more operations than it is allowed to do. Previously there was no filtering on issues & PRs. So when it tries to fetch all the currently open issues, checking whether or not they should be get closed, it would have to perform quite a few requests pagination requests to get the information needed. Knowing that we only care about issues & PRs already labelled `stalled`, we can provide the [`only-labels`](https://github.com/actions/stale/blob/13b324e4b28a2708236aadb11361fa65af60d201/action.yml#L38) option to make sure we only fetch relevant issues. Refs #35144 PR-URL: #35159 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Mary Marchini <[email protected]> Reviewed-By: Michaël Zasso <[email protected]>
|
The action closed a few issues but still reach a maximum number of operations: https://github.com/nodejs/node/actions/runs/252939090 I don't understand, because the number of operations per run is a required action input but we don't set it and the action still runs. |
|
Very strange indeed. Expanding the logs of the run you mentioned, shows quite a lot of verbose logging; 405 lines logged before being killed 🤔
I'll dive in and see if there's any hints in there, or GitHub docs describing what kind of limits there are. |
|
Just found the culprit; the maximum number of operations error is self imposed by the stale GitHub Action. It counts the number of operations (read: github API requests) performed, and exits with that error when having reached a certain threshold. Luckily it's configurable via the As described in the option docs, it's a countermeasure to avoid hitting the request per hour rate limits. Meaning it might not be the best idea to bump There's improvements to be made in the stale action project to reduce the API requests made, which we in practise never use the result of (un-labelling due to comments), but that will naturally take some time to fix. In the near future, we've got two alternatives as far as I see things:
Any thoughts? |
|
30 is a very low number of operations, which APIs are used by the Action? Most endpoints tolerate up to 5000 requests per hour [1], 30 requests per minute for Search API [2], and 5000 point per hour (whatever that means) for GraphQL queries [3]. Unless the search API is being used, we can increase it without problem. If the Search API is being used, we could have the stale action running every half hour or so, that way we'll drain the queue way faster (probably in a day or two). |
|
Here's the relevant requests it performs when fetching issues/PRs labelled
Cool. I haven't found any use of the Search API. I'm more than happy to open a new PR to bump |
|
Yeah let's go with 500 :) |
|
Also I'm almost sure the request limits is scoped to the action run, meaning it wouldn't affect other runs (not entirely sure though) |
|
Huh, that would be nice indeed. In worst case it doesn't and we end up blowing our rate limits sometime in the future, we can adjust appropriately. Appreciate the lightning quick responses! |
The second attempt at getting the auto closing of issues & PRs to work as expected without hitting a maximum operations allowed error we've been seeing. Recently discovered that the mentioned error is actually self imposed by the stale action itself. It keeps track of how many outgoing GitHub API requests it performs, and if that count exceeds the configured `operations-per-run` option, it exits to avoid hitting API rate limits. Default `operations-per-run` value is set to `30`. That's a very low limit and we're not at all concerned hitting that rate limit as of now, so we're bumping `operations-per-run` to `500` with these changes. Refs nodejs#35144
The auto closing of issues & PRs labelled with `stalled` doesn't seem to be working as expected. The GitHub Action UI gives the impression the stale action tries to execute more operations than it is allowed to do. Previously there was no filtering on issues & PRs. So when it tries to fetch all the currently open issues, checking whether or not they should be get closed, it would have to perform quite a few requests pagination requests to get the information needed. Knowing that we only care about issues & PRs already labelled `stalled`, we can provide the [`only-labels`](https://github.com/actions/stale/blob/13b324e4b28a2708236aadb11361fa65af60d201/action.yml#L38) option to make sure we only fetch relevant issues. Refs #35144 PR-URL: #35159 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Mary Marchini <[email protected]> Reviewed-By: Michaël Zasso <[email protected]>
The second attempt at getting the auto closing of issues & PRs to work as expected without hitting a maximum operations allowed error we've been seeing. Recently discovered that the mentioned error is actually self imposed by the stale action itself. It keeps track of how many outgoing GitHub API requests it performs, and if that count exceeds the configured `operations-per-run` option, it exits to avoid hitting API rate limits. Default `operations-per-run` value is set to `30`. That's a very low limit and we're not at all concerned hitting that rate limit as of now, so we're bumping `operations-per-run` to `500` with these changes. Refs #35144 PR-URL: #35235 Reviewed-By: Mary Marchini <[email protected]> Reviewed-By: Daijiro Wachi <[email protected]> Reviewed-By: Luigi Pinca <[email protected]>
|
Bumping The first close stalled action run after those changes excited cleanly with Still not as effective closing issues as I'd thought beforehand to be honest. That's because most of the issues it checks ends up evaluating to true for one/both of the below: a. has been commented on by someone else than the issue author or a bot since it was labelled The results of those checks are logged into something like this per issue: Are we okey with this as is? Or do anyone fee there's improvements to be made the stale action to be more effective? |
|
Does it close 30 days after someone comments? If so I think we're fine, otherwise we might need to revisit. |
Yes, as long as no updates to the issue/PR has been made after the last comment (reflected by that issue's Would that cause headache for us? Do we often update |
|
That sounds totally reasonable for us. If it becomes a problem we can revisit. For now I believe we can close this issue. |
The second attempt at getting the auto closing of issues & PRs to work as expected without hitting a maximum operations allowed error we've been seeing. Recently discovered that the mentioned error is actually self imposed by the stale action itself. It keeps track of how many outgoing GitHub API requests it performs, and if that count exceeds the configured `operations-per-run` option, it exits to avoid hitting API rate limits. Default `operations-per-run` value is set to `30`. That's a very low limit and we're not at all concerned hitting that rate limit as of now, so we're bumping `operations-per-run` to `500` with these changes. Refs #35144 PR-URL: #35235 Reviewed-By: Mary Marchini <[email protected]> Reviewed-By: Daijiro Wachi <[email protected]> Reviewed-By: Luigi Pinca <[email protected]>
The auto closing of issues & PRs labelled with `stalled` doesn't seem to be working as expected. The GitHub Action UI gives the impression the stale action tries to execute more operations than it is allowed to do. Previously there was no filtering on issues & PRs. So when it tries to fetch all the currently open issues, checking whether or not they should be get closed, it would have to perform quite a few requests pagination requests to get the information needed. Knowing that we only care about issues & PRs already labelled `stalled`, we can provide the [`only-labels`](https://github.com/actions/stale/blob/13b324e4b28a2708236aadb11361fa65af60d201/action.yml#L38) option to make sure we only fetch relevant issues. Refs #35144 PR-URL: #35159 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Mary Marchini <[email protected]> Reviewed-By: Michaël Zasso <[email protected]>
The second attempt at getting the auto closing of issues & PRs to work as expected without hitting a maximum operations allowed error we've been seeing. Recently discovered that the mentioned error is actually self imposed by the stale action itself. It keeps track of how many outgoing GitHub API requests it performs, and if that count exceeds the configured `operations-per-run` option, it exits to avoid hitting API rate limits. Default `operations-per-run` value is set to `30`. That's a very low limit and we're not at all concerned hitting that rate limit as of now, so we're bumping `operations-per-run` to `500` with these changes. Refs #35144 PR-URL: #35235 Reviewed-By: Mary Marchini <[email protected]> Reviewed-By: Daijiro Wachi <[email protected]> Reviewed-By: Luigi Pinca <[email protected]>
The auto closing of issues & PRs labelled with `stalled` doesn't seem to be working as expected. The GitHub Action UI gives the impression the stale action tries to execute more operations than it is allowed to do. Previously there was no filtering on issues & PRs. So when it tries to fetch all the currently open issues, checking whether or not they should be get closed, it would have to perform quite a few requests pagination requests to get the information needed. Knowing that we only care about issues & PRs already labelled `stalled`, we can provide the [`only-labels`](https://github.com/actions/stale/blob/13b324e4b28a2708236aadb11361fa65af60d201/action.yml#L38) option to make sure we only fetch relevant issues. Refs nodejs#35144 PR-URL: nodejs#35159 Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Mary Marchini <[email protected]> Reviewed-By: Michaël Zasso <[email protected]>
The second attempt at getting the auto closing of issues & PRs to work as expected without hitting a maximum operations allowed error we've been seeing. Recently discovered that the mentioned error is actually self imposed by the stale action itself. It keeps track of how many outgoing GitHub API requests it performs, and if that count exceeds the configured `operations-per-run` option, it exits to avoid hitting API rate limits. Default `operations-per-run` value is set to `30`. That's a very low limit and we're not at all concerned hitting that rate limit as of now, so we're bumping `operations-per-run` to `500` with these changes. Refs nodejs#35144 PR-URL: nodejs#35235 Reviewed-By: Mary Marchini <[email protected]> Reviewed-By: Daijiro Wachi <[email protected]> Reviewed-By: Luigi Pinca <[email protected]>
https://github.com/nodejs/node/actions/runs/247170963
It seems like the Action is not faring well with the size of our repository. I'm not sure if it is closing some issues, or if it tries to detect all issues before closing, but fails to do so because it reaches some limit. Based on the number of lines in the output, we're probably iterating over all issues (even though we don't have to) and therefore we're hitting GitHub API rate limit (but that's just a guess). We probably need some changes on the upstream action to better handle
days-before-stale: -1scenarios.@phillipj fyi
The text was updated successfully, but these errors were encountered: