Draft starter SLA for issues and PRs

[thomashchan1]

As discussed in the 6/18 meeting regarding "Due process for spec issues/PRs", would like to propose some starter/draft SLA for issues/PRs:

Issue SLA:
-For issue originator, after opening an urgent issue, allow at least (proposed) 1 business day before following up with additional info/questions or relevant individual(s). Use URGENT label to distinguish from other issues and provide reason behind the urgency.
-For issue originator, after opening a non-urgent issue, allow at least (proposed) 3 business days before follow up with additional info/questions or relevant individual(s).
-For issue resolver, after providing answers/resolution, allow at least (proposed) 2 business days for originator to respond before closing an issue.

PR SLA
-Minimum review period for PRs before they could be merged: proposing 3 business days.
-Maximum response time from reviewers/contributors: proposing 3 business days.
-If there is an exception to the general SLA, please highlight reasons in comments. Valid exceptions are typo PRs or documenting/implementing what have already been discussed/agreed, etc..
-If SLA is violated and no reason/justification comments are found, please be prepared to accept another PR which could modify or rollback prior changes.
-Proposing to have 2 reviews from different companies for interface/design impacting changes.

Let me know if these are reasonable and if there are others we want to add to the list. We can add these to the appropriate sections in the CONTRIBITING.md file for all relevant repos. Let me know if you want me to open a PR for these changes. Thanks.

[bogdandrutu]

Do you want to apply this for all the repos or just for specs? Also I think we need to make sure fixes like P0 bugs are address way faster.

[thomashchan1]

Ideally that would be for all repos, but we can start with specs given that's where the key discussions are happening. Agreed that P0 bugs would move way faster, and that's where the classifications/labels and exception comments/justifications come in so everyone would be synced on expectations.

[iredelmeier]

Issue SLA:
-For issue originator, after opening an urgent issue, allow at least (proposed) 1 business day before following up with additional info/questions or relevant individual(s). Use URGENT label to distinguish from other issues and provide reason behind the urgency.

Use of the URGENT label makes sense to me.

I don't think I understand the suggestion re: waiting before asking followup questions. It seems like questions should be asked ASAP? Guessing I'm missing or misunderstanding the intent.

-For issue originator, after opening a non-urgent issue, allow at least (proposed) 3 business days before follow up with additional info/questions or relevant individual(s).

Same as above. I believe that the more info and questions we have upfront, the better.

-For issue resolver, after providing answers/resolution, allow at least (proposed) 2 business days for originator to respond before closing an issue.

This seems like, at the very least, it should vary by context.

For issues that are "discussions" of some sort, I think 2 days may sometimes be insufficient. Mostly, the cost of prematurely stopping discussions seems greater than the potential value of closing the issue.

For issues that require specific feedback from the originator (or anyone else in the thread), such as suggested workarounds, I believe that we could even increase from 2 to something like 5.

For most other issues, however, I think that keeping them open after something like a merged PR will be more confusing than closing them immediately with a suggestion that, if the issue remains, you can reopen the original issue or open a new one.

Importantly, GitHub provides certain keywords that will auto-close issues through a PR merge. This feels like the desired behaviour. I'd also be surprised if we can disable, so there would be ambiguity and/or inconsistency across PRs that did and didn't use canonical keywords.

PR SLA
-Minimum review period for PRs before they could be merged: proposing 3 business days.
-Maximum response time from reviewers/contributors: proposing 3 business days.

How should these account for active discussion? (Also, how should we define "active"?)

-If there is an exception to the general SLA, please highlight reasons in comments. Valid exceptions are typo PRs or documenting/implementing what have already been discussed/agreed, etc..

I strongly believe that, moving forward, all agreements and discussion should be confirmed on GitHub - ideally through an RFC - rather than only in meetings or other venues since a) not everyone can attend all meetings and b) large group meetings tend to drown out certain voices.

Agreed re: typos. I'd also add bug fixes, documentation improvements, and certain "chores" (e.g., backfilling tests, adding CI).

-If SLA is violated and no reason/justification comments are found, please be prepared to accept another PR which could modify or rollback prior changes.

👍

-Proposing to have 2 reviews from different companies for interface/design impacting changes.

I believe we should also be prepared to handle scenarios in which reviewers do not have unanimous responses. (I proposed something for the RFCs here... which I now see needs some Markdown fixes 😅)

[thomashchan1]

Issue SLA:
-For issue originator, after opening an urgent issue, allow at least (proposed) 1 business day before following up with additional info/questions or relevant individual(s). Use URGENT label to distinguish from other issues and provide reason behind the urgency.

Use of the URGENT label makes sense to me.

I don't think I understand the suggestion re: waiting before asking followup questions. It seems like questions should be asked ASAP? Guessing I'm missing or misunderstanding the intent.

-For issue originator, after opening a non-urgent issue, allow at least (proposed) 3 business days before follow up with additional info/questions or relevant individual(s).

Same as above. I believe that the more info and questions we have upfront, the better.

[tc] Agreed that additional questions/infos should be posted ASAP. The intention here is to give the assignees time to work the issues and respond. Will make this clear about following up on issue status with the assignees.

-For issue resolver, after providing answers/resolution, allow at least (proposed) 2 business days for originator to respond before closing an issue.

This seems like, at the very least, it should vary by context.

For issues that are "discussions" of some sort, I think 2 days may sometimes be insufficient. Mostly, the cost of prematurely stopping discussions seems greater than the potential value of closing the issue.

For issues that require specific feedback from the originator (or anyone else in the thread), such as suggested workarounds, I believe that we could even increase from 2 to something like 5.

[tc] 5 days seem a bit excessive if we want to move a bit faster on our projects. Would recommend we start with 2 days, and if the issue originator needs more time, can always request that in the comment section. We want to balance the need for closing the issue/discussion too early versus dead air time waiting for originator to provide initial response. Thoughts?

For most other issues, however, I think that keeping them open after something like a merged PR will be more confusing than closing them immediately with a suggestion that, if the issue remains, you can reopen the original issue or open a new one.

[tc] Yes, for issues that have merged PRs, closing it right-a-way seems to be a better option. The above recommended guidelines would be for things that we don't take action or recommending workarounds, etc..

Importantly, GitHub provides certain keywords that will auto-close issues through a PR merge. This feels like the desired behaviour. I'd also be surprised if we can disable, so there would be ambiguity and/or inconsistency across PRs that did and didn't use canonical keywords.

PR SLA
-Minimum review period for PRs before they could be merged: proposing 3 business days.
-Maximum response time from reviewers/contributors: proposing 3 business days.

How should these account for active discussion? (Also, how should we define "active"?)

[tc] PR with outstanding questions not yet answered? Clock starts when all pending questions/issues have been addressed?

-If there is an exception to the general SLA, please highlight reasons in comments. Valid exceptions are typo PRs or documenting/implementing what have already been discussed/agreed, etc..

I strongly believe that, moving forward, all agreements and discussion should be confirmed on GitHub - ideally through an RFC - rather than only in meetings or other venues since a) not everyone can attend all meetings and b) large group meetings tend to drown out certain voices.

[tc] That would be ideal. Let's help to push through and trial the RFC process so that we can iterate and improve with the goals to keep the community more open and more efficient.

Agreed re: typos. I'd also add bug fixes, documentation improvements, and certain "chores" (e.g., backfilling tests, adding CI).

[tc] Sounds good.

-If SLA is violated and no reason/justification comments are found, please be prepared to accept another PR which could modify or rollback prior changes.

👍

-Proposing to have 2 reviews from different companies for interface/design impacting changes.

I believe we should also be prepared to handle scenarios in which reviewers do not have unanimous responses. (I proposed something for the RFCs here... which I now see needs some Markdown fixes 😅)

[tc] given the RFC process proposes 2 business days for FCP, may be we should start with 2 instead of 3 days for PRs?

[SergeyKanzhelev]

no progress here. The best place for this is specification repo anyway. Closing the issue