Release cadence

[fjetter]

This ticket is to not derail any changelog related discussions going on in #199

Xref #101

[jrbourbeau]

Copying over @jakirkham's related comment from #199 here...

don't understand how the release cadence affects our ability to fix regressions. In my mind, we agreed to just release where we are at every 2 weeks

I believe two weeks are not sufficient to find and fix the regressions. Due to stability problems in dask/distributed we recently needed to delay multiple releases because we didn't manage to fix them in the given time.

On the flip side releasing less frequently could mean more regressions accumulate in main (as others are only testing the latest release) and we only find out about more of them later (once we finally release).

[jsignell]

I misunderstood what you meant about the cycle being too short to fix regressions. It thought you meant regressions that were introduced in a previous release. In which case releasing again, even if the regression isn't fixed, doesn't necessarily make things worse. It seems like what you were intending to consider are the cases where the regression is only on main (unreleased) and we are trying to avoid releasing any versions that contain the regression.

To fix that I think we need to think about introducing a code freeze before releases. Maybe that would end up resulting in a longer release cycle, but the main mechanism for preventing undiscovered regressions from entering release has to be a freeze right?

[quasiben]

I think it's also worth linking an older discussion from last year where it was proposed to increase releases: #84. The community and focused uses cases may have shifted since that was introduced but the discussion is worth reviewing.

[jakirkham]

It's good that you raise that Ben.

Several discussions evolved out of that may be worth revisiting as we figure out how best to improve the release process. In particular we observed there were a few common needs:

• Quick releases (to get changes in users faster)
• Slower more stable releases (or need to get more complex changes in that take time)
• Need for blessed releases and/or maintenance releases and/or RCs

This motivated a few discussions most notably how CalVer was implemented here ( #100 ). These also came up:

• Publishing nightly packages #76
• Blessed Dask releases #101
• Add release candidates to release procedure #94

Also more recent, but conceptually related:

• Integration test suite before release #163

While there are still needs for each of these cases, there has been some effort to address them with CalVer. It may be worth revisiting how to address those as they get at the sticky issues that make releasing challenging.