[RFC 0051] Mark stale nixpkgs issues #51
Conversation
rfcs/0051-close-stale-issues.md
Outdated
| markComment: > | ||
| This issue has been automatically marked as stale because it has not had | ||
| recent activity. It will be closed if no further activity occurs. Thank you | ||
| for your contributions. |
There was a problem hiding this comment.
I think the framing here is very important to (1) don't make anyone feel like they're just a drag on the project, make clear that it's totally fine to just keep bumping an issue; (2) give constructive feedback on how to get attention for an issue, like actively searching for maintainers / previous people that touched the code and pinging them.
There was a problem hiding this comment.
I think this is a great point. Phrasing it as helping the user seems much more likely to turn out well. "It looks like this issue hasn't attracted much attention. Here are n suggestions for how to progress this issue further."
rfcs/0051-close-stale-issues.md
Outdated
| [alternatives]: #alternatives | ||
|
|
||
| 1. Do nothing | ||
| 2. Make custom tooling to do something more sophisticated |
There was a problem hiding this comment.
There is also a whole set of other bots in the same project that might better fit our needs: https://probot.github.io/apps/
rfcs/0051-close-stale-issues.md
Outdated
|
|
||
| Marking issues stale might dissuade contributors who already feel | ||
| their contribution was being ignored. | ||
|
|
There was a problem hiding this comment.
There are some security considerations to have to giving access to the org to a third-party application. I don't know what access rights the stale bot needs, hopefully just access to the issues. Alternatively it would also be possible to deploy https://probot.github.io/apps/stale/ ourselves.
rfcs/0051-close-stale-issues.md
Outdated
| [motivation]: #motivation | ||
|
|
||
| The Nixpkgs GitHub page has a large number of open issues causing | ||
| community angst and misrepresenting the responsiveness of the project. |
There was a problem hiding this comment.
It would be good to find metrics that better represent our project's health and communicate that instead. It seems quite natural to me that due to the size of nixpkgs we have many known problems that needs to be addressed.
There was a problem hiding this comment.
Yes, when I first came here, I remember being shocked by the numbers. But now they just seem realistic. We'd all like them to go down, but I feel like closing issues prematurely would cause more angst, not less. Imagine how it would feel when, in addition to having your issue "ignored", you then suffer the further indignity of having it closed by a bot!
It's already easy to get the impression that nobody cares about your issue or PR. Wouldn't closing outstanding issues would basically confirm that?
There was a problem hiding this comment.
I think it really depends on the bot's abilities. For example I am never annoyed by the existence of ofborg as it helps me move my PR along. Even when it was posting build results as comments, I was happy to get the notifications because it was an asynchronous prompt for me to react on the feedback.
Stale bots are less clear cut. It depends how precise their heuristics are. If implemented correctly I can see how it could prompt people to react and move the issue or PR along. But if they generate too much noise it can also lead to notification fatigue. It really depends on the implementation.
There was a problem hiding this comment.
Sure, I wouldn't be opposed to a kind of ping-bot. But I understood this RFC as being primarily about closing issues, with the bot being an implementation detail.
I actually wasn't thinking about the case where the submitter has not responded; in that case, I'm not opposed to automated closing if it's a "support ticket" kind of issue where others can't reproduce. That would help make the list of open issues more relevant, I think.
There was a problem hiding this comment.
Maybe it would be an option to only ask for a follow up if the last message is not from the submitter? This way the submitter would only have to say "no, this is still an issue" once and the bot won't bother them again until someone else asks for some followup.
There was a problem hiding this comment.
Either this, or maybe it would be possible to have the bot only trigger on 9.needs: reporter feedback?
That said, my experience using codetriage is that the issues I click on usually would not fall under this label -- it's only maybe one in four or five. But it does happen that when I ping from triage to re-request the requested information, the reporter just states they've stopped using NixOS altogether, and then it's possible to close the ticket.
All that to say: I think a stale bot would make sense, but more for the “regularly poke people with a message to help them figure out how to push it forward” part of it and less for the “automatically close issues” part of it. If it's done this way, we should be careful to not have it poke too many issues on the first start, or maintainers would be overwhelmed by people directly reaching to them at once, though.
There was a problem hiding this comment.
Another scenario is when a PR gets stuck because the author and the reviewers don't agree on the outcome. In that case, if the bot is pinging, it's not bringing in new information that could help move the PR forward. If it's closing it favours the status quo over innovation.
There was a problem hiding this comment.
Not necessarily. If for example the bot links to a page containing as many suggestions as possible on how to move on in such situations (which in my opinion should be a part of this RFC), I would consider it very helpful.
rfcs/0051-close-stale-issues.md
Outdated
|
|
||
| ``` | ||
| # Number of days of inactivity before an issue becomes stale | ||
| daysUntilStale: 60 |
There was a problem hiding this comment.
I think this should be much higher, perhaps ~1 stable release of NixOS? There are a lot of useful issues older than 60 days!
There was a problem hiding this comment.
stale does not mean "not-useful" — neither does it mean "not valid"...
It just reflects the fact that nobody has bothered interacting.
|
I am completely against this idea. In my experience, this this the archetypal way of trying to solve social issues with technical solutions. Apart from “making that number go down and feel better”, introducing arbitrary time-outs for mostly still valid issues is a sure way to lose potential contributors. Github provides good tooling to filter issues down to the ones that are interesting to review, no need to confuse “issue open” with “failure of project”. |
rfcs/0051-close-stale-issues.md
Outdated
| # Number of days of inactivity before an issue becomes stale | ||
| daysUntilStale: 60 | ||
| # Number of days of inactivity before a stale issue is closed | ||
| daysUntilClose: 7 |
There was a problem hiding this comment.
I would prefer never. Let humans to decide.
|
To add to what @Profpatsch is saying my previous issues that has gone "stale" in other repos has been a source of incredible frustration when they auto-close just to artificially decrease an issue number. Another thing I have noticed about Nixpkgs issues is that some of them are very long-running for technical reasons. One example of this is NixOS/nixpkgs#33798. Just because there isn't much happening on this issue doesn't mean that it's an issue that we want to close, it's still an ambition to drop Qt 5.6 at some point, but as of right now a few packages requires it to run so we don't drop it. I'd be angry and frustrated if such issues were auto-closed. |
rfcs/0051-close-stale-issues.md
Outdated
|
|
||
| ``` | ||
| # Number of days of inactivity before an issue becomes stale | ||
| daysUntilStale: 60 |
There was a problem hiding this comment.
I think it must be not less than 6 months. One year is even better.
In this way, it'll help get rid of issues that can't be reproduced anymore, but also it'll not cause frustration of issue creator.
|
The idea is to ping old issues and close one that do not respond. I'm not sure if automatic closing is a good idea, but notification and labeling is, if it's done rarely (once in a year). |
I honestly don't want to sound dismissive of your point of view. There is just one thing that I don't understand. If you have been motivated before to work on that issue, then why had there been no progress and no activity whatsoever for 15+ months at the time when I closed it? I mean, when you say you feel demotivated working on it now, I'm not sure how to parse that. What was your motivation level before I closed it exactly? IMHO, if an issue has been completely inactive for, say 12 months, then it's dead and I see no point in keeping it open. I would tag it with OBSOLETE or AUTO-CLOSED or whatever and close it. If you actually care about an issue, then surely you should be able to make some kind of headway within a freaking year, no? |
|
The purpose of issues is to keep track of existing problems. Closing issues for existing problems defeats this purpose as well as keeping track of issues that nobody can reproduce anymore. |
|
I agree. I would find it really upsetting if an issue I reported still existed but was closed just because it was old. |
|
I nominate @grahamc and myself for shepherding |
Not everyone agrees that this is their purpose. |
|
my 2¢: I really hate it when projects do this and I don't think it improves software quality at all. |
The thing is, issues are not only meant for developers. They are meant for users to report issues so that developers can fix them. And for a regular user, it is very normal that in 15+ months they make no progress -- they would maybe need to learn everything from scratch just to make headway on this. (and, in this specific case, I guess the priority was just below other more-important stuff) Saying that we should close issues when, after a bot ping, no one answers for some time kind of makes sense (no one is interested in it any longer, and it quite possibly doesn't exist any longer). Saying that we should close issues when there is no progress made means that we decide to have no way for non-developers to report issues.
I think it is the first time I hear of someone opposing this statement. What is their purpose, for you? |
I think this argument arises in part from a missing (conceptual) distinction between issues and tickets. Consider that a single underlying issue may manifest for multiple users in ways that cause them to file multiple reports. Currently, the Nixpkgs GitHub is a main venue for user support, so we have both; but that's not a reason to give them the same blanket treatment. (That said, it is also not appropriate to mark unresolved tickets as resolved, and a sophisticated support system distinguishes between resolved and expired.) GitHub has only a binary open/closed status, so how you think that status should be used is a direct result of what you think that the items in the GitHub tracker are. |
domenkozar
added
the
status: open for nominations
I am sorry, but that is not my experience. You say it's normal for people to actively work on an open source software issue without any observable progress whatsoever for 15+ months, but I don't believe that. My experience is that once a ticket has gone over a year without any activity, chances that it will be fixed in the future are very, very slim. I don't think it's normal at all for tickets to get picked up again after lying there, dead, for over a year. I dare say, it hardly ever happened in my free software career. That's why people write things like the Github stale bot in the first place, so that those dead tickets can be closed.
OK, we both agree. I believe that this is the proposal made in this RFC? The bot closes issues who remain inactive of X days even after the ping, right? That sounds perfectly reasonable.
Oh, come on, that's is not true. Non-developers are reporting issues in Nixpkgs all the time!!!! Nobody is going to prevent them from doing that.
I perceive Github issues as a way to track important, actionable tasks that someone is currently working on or that someone should be working on in the near future. The issue tracker is a way to coordinate the work done by the developers, so that everyone knows what the others are working on and we avoid doing the same thing multiple times by accident. Tickets that lie around for years without anyone even attempting to solve them don't meet those criteria. Clearly, those issue are not actionable or not important or neither of the two. |
|
It's rare that someone will work continuously on an issue for 15 months without visible process, but I think it's hardly uncommon in open source for an issue to lie around for 15 months until someone with the interest, expertise and time decides to investigate and fix it. I think incrementalism is the only way a project the size of nixpkgs can advance. |
Can you point out any concrete examples in the Nixpkgs issue tracker to back up that opinion? |
|
@peti This is an easy challenge. This kind of issues is easy to find, just in the issues that were closed in the last month we can find some examples:
|
It's a good thing I did not say that, then. To quote myself, “for a regular user [as opposed to developer], it is very normal that in 15+ months they make no progress -- they would maybe need to learn everything from scratch just to make headway on this” The regular user is not going to actively work on an open-source software issue for 15+ months. They are going to shoot the issue, and let people with relevant knowledge fix it, when such people will come onto the issue. Being better at pointing people with relevant knowledge to relevant issues is what we need to be better at, and I hope a stale-bot-like thing can do it properly without having this hugely-annoying-for-the-bug-reporter behavior of just closing things no one decides to work on. |
Interesting. Clearly you were right. I hardly ever see cases like those myself, but apparently that does not mean that they don't exist. |
|
Would it make sense to get more data points about issues and PRs? One danger I see of this RFC is that it's natural to judge the RFC based on our personal history. Maybe if we make it more about data then the discussions can become more specific and objective. I think we all agree that we have many issues open and that automation is a good thing to leverage (that's why we are developers isn't it? :D). All we seem to disagree on is on the implementation and how it should look like. |
It would. What would be the questions that you would like to use data to answer? |
|
Three minor, disconnected observations from my very minor contributor's perspective:
|
|
Nit: could someone remove the |
domenkozar
added
status: accepted
and removed
status: FCP
Why is that? I might be missing something here. |
Your avatar and its both have rectangular heads, chest-mounted monitors, claw-hands, and ball-tipped antennæ atop the head. :) |
|
This bot is great; I found and closed several long-since resolved issues from its pings! The above wording does seem like an improvement, though. |
|
A little less enthusiastic feedback: The bot does create quite a bit of work for me. Just in the last 4 days I've received 60 emails:
On pretty much all of them I have to write I find it somewhat stressful to go through that amount of (mainly unnecessary) work, because I do want to point out when issues are not stale, and remove the label to make that clear. I've also tried to see if others unstale these issues if I wait a couple days, but that doesn't seem to happen, so it does seem to be on me to do it (at least for those issues that arrive in my mailbox). As an aside, for some reason
It feels like the task of marking issues that are still important to me as non-stale is something I soon have to automate, because it takes a significant amount of my daily time. 🤔 |
|
In my case, the bot helped me recover an issue (NixOS/nixpkgs#24613) that I had commented in 2017 to say: it can be closed. The only other comment it had received since then was someone else saying: it can be closed. It's a bit sad that nobody with the right permission had reacted to close it but now I have triage permissions, so I was finally able to close it! [But if I hadn't, this kind of message wouldn't have been nice, unless you can react to ask the bot to close the issue.] EDIT: this was also the case of another issue (NixOS/nixpkgs#30476). 👍 for modifying the message to be more friendly and explicit on what people could do |
|
I agree it's annoying (I get multiple dozen emails per day), but once the bot goes through the backlog the noise will turn down. |
|
I guess this has to affect most of contributors that have been very active over long term. @nh2: perhaps you mistakenly thought that the bot would close the issues? If some are very important to you it surely does make sense to express that on the issue, but if not... keeping the "stale" label doesn't seem a big deal to me. (arguably, just the fact that noone is working on the issue for many months)
It appears to me, but "1000+" isn't very informative. You can try https://github.com/NixOS/nixpkgs/issues?q=involves%3Anh2 |
|
What it says aside, I've found the bot to have its use as an automated
version of @Profpatsch <https://github.com/Profpatsch> going around asking
"(triage) Is this still a problem": it's reminded me to check whether I
still have issues, and in at least one case I didn't (NixOS/nixpkgs#47385
<NixOS/nixpkgs#47385>).
Yes, that’s why I’m glad this exists now. Regular triage was purely
mechanical (and I didn’t even do it regularly, more like once a year).
Improving the phrasing is always possible, thus I’m glad people are
discussing it.
I agree that it should be formulated as positively as possible. in general,
you don’t want to give people negative emotions if not necessary, and
positive and encouraging phrasing contributes a lot to that.
…On Thu, Jun 4, 2020 at 3:59 AM 8573 ***@***.***> wrote:
Three minor, disconnected observations from my very minor contributor's
perspective:
1.
I join in supporting @abathur <https://github.com/abathur>'s phrasing.
2.
What it says aside, I've found the bot to have its use as an automated
version of @Profpatsch <https://github.com/Profpatsch> going around
asking "(triage) Is this still a problem": it's reminded me to check
whether I still have issues, and in at least one case I didn't (
NixOS/nixpkgs#47385 <NixOS/nixpkgs#47385>).
3.
When not looking carefully, I've confused it with @timokau
<https://github.com/timokau>... :p
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#51 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAYB5ZVQI6QWWEYIFD6UINTRU35YTANCNFSM4IPHMSPQ>
.
|
I hope so :) Can anybody clarify the working mechanism of the bot: Via which method does it decide how to comment on issues that have no activity for longer than 6 months? E.g. this one had the last human comment in 2017. Is it slowly working through a backlog? If yes, is there a way to see how far it is, or know when it's done? The fact that old issues were not all commented on instantly but over a time of multiple days may falsely have created the impression that what I see in my inbox now is the "stable throughput" of just the "every 180 days" logic. If it's really just an initial burst and afterwards the daily volume is much lower, that's probably fine!
I did not think that it would close the issues; if I remember correctly, I was part of a discussion before where most people agreed that a stale-bot should only set labels, not auto-close, and I was also an advocate of labeling-only. I do feel obliged to comment on these issues as the bot asks, not because they are "very important" to me but because it would be a waste of time (mine and others') if somebody filed the same bug twice or did the same investigation twice, and to make clear to others that yes, this bug still exists (in other words, keeping issue tracker information accurate).
Unfortunately that method does not work for issues where you've clicked the Subscribe button on the right, only for issues where you're I checked yesterday; Github's API does not seem to provide a way to get at the total number of subscribed issues. I tried That said, the |
Updated: Oops. Overconfident in it making sense. Guess I shoulda just gone to find the code. :D After taking a closer look, I think:
|
|
Now that I've seen this in action, as someone with a couple of fairly old open PRs: I don't know how to proceed with them. They need reviewing and eventually committing, but there's no obvious, clear-cut way of figuring out who I should alert to do that. I'd like to see the stale stale bot providing a link to some general guidance on getting nixpkgs PRs moving. More generally, I think PR handling really shouldn't be almost dependent on the PR author identifying and pinging the 'appropriate' people; surely that could be automated? |
|
At least the PR part can (and hopefully will) be improved by marvin. Instead of marking PRs as stale, we'd assign labels to it which indicate for whom it is actionable (author, reviewer, merger). Then stale labels would either be unnecessary or could be restricted to I hope this will already help in getting you a reviewer for your PRs, since reviewers can now easily find actionable PRs and everybody can help out, regardless of whether they have the commit bit or not. Once that is working, we could explore timokau/marvin-mk2#2 to get some steady "baseline-throughput" of PRs. We should make sure that people don't sign up for too much there though to avoid burnout. |
|
I once had an idea, which may be worth mentioning here, also considering timokau/marvin-mk2#2 : What if there was a website or a database that I'd upload to it the list of packages in my |
|
@doronbehar I'm all for improving the stale bot wording! I think we can discuss the details in a regular nixpkgs PR. |
|
OK, here's my suggestion: NixOS/nixpkgs#92254 |
|
#124 Follow-up RFC for Nix itself is here. |
* Stalled RFCs RFC * Rename with PR number * Spelling fix. Co-authored-by: Luna Nova <[email protected]> * Spelling and Grammar Fixes Co-authored-by: Ryan Mulligan <[email protected]> * Update shepherds. * Fix diagram rendering. Co-authored-by: Linus Heckemann <[email protected]> * Add reference to #51 * Fix brain-typo. Co-authored-by: Ryan Mulligan <[email protected]> Co-authored-by: Luna Nova <[email protected]> Co-authored-by: Ryan Mulligan <[email protected]> Co-authored-by: Linus Heckemann <[email protected]>
* rfc51: close stale issues initial draft * rfc0051: use staleLabel consistent with nixpkgs labeling conventions * rfc0051: do not close issues * rfc0051: reword motivation * record the members of the shepherd team * Update rfcs/0051-mark-stale-issues.md Co-Authored-By: asymmetric <[email protected]> * [RFC 0051] remove references to "issue" in stale message
* Stalled RFCs RFC * Rename with PR number * Spelling fix. Co-authored-by: Luna Nova <[email protected]> * Spelling and Grammar Fixes Co-authored-by: Ryan Mulligan <[email protected]> * Update shepherds. * Fix diagram rendering. Co-authored-by: Linus Heckemann <[email protected]> * Add reference to NixOS#51 * Fix brain-typo. Co-authored-by: Ryan Mulligan <[email protected]> Co-authored-by: Luna Nova <[email protected]> Co-authored-by: Ryan Mulligan <[email protected]> Co-authored-by: Linus Heckemann <[email protected]>
Summary:
Rendered