Update release flow

[clrcrl]

Description & motivation

As per our discussions — fixing a process problem with documentation rather than fancy tech.

🚨 We need to update our default branches to get this right.

[jtcohen6]

[EDIT: overtaken by events]

While the version of the world where all dbt and dbt-utils minor releases match their numbering, and all dependent packages defensively place upper boundaries on both, it introduces more complexity than we have capacity to handle proactively today. Instead, we're going to continue to let dbt and dbt-utils minor versions diverge, and react swiftly to breaking changes as they relate to each other and dependent packages.

Below is my original message for posterity, or for when we code up dbt-dependabot:

My feelings have gone back and forth on this several times. At the end of the day, I think it is the right move. There are two big preconditions for this to work well:

• For every minor version release of dbt, we need to have a dbt-utils release ready to go on the same day ("gameday").
• What to do about all our other packages? I don't think we need to move them into semantic version lockstep just yet. I think those packages should have the freedom to make minor releases (breaking changes) separate from the dbt release cycle. That said, we should pin versions defensively by adding the identical upper bounds for dbt + dbt-utils.

Logistics

The Core dev team is committing to putting out a release candidate two weeks before minor versions are released. In that crucial two-week window, we need to:

• Test and ready the matching minor version of dbt-utils, for release on gameday.
• If there are no breaking changes, cut a patch release of every other (non-utils) package to bump up the upper bounds of dbt + dbt-utils by one minor version. If there are breaking changes, prepare new minor versions of those packages to release on gameday.

Next steps

This is kind of messy, because we don't have a dev/0.17.x branch that people should make bug-fix PRs against.

What do you think about taking these steps now:

• Create a dev/0.17.x branch off master, and make it the default branch
• Cut a 0.17.0 release that's effectively identical to 0.5.0
• Cut new patch release of every other Fishtown-supported package to pin all three of the following at ">=0.17.0,<0.18.0":
  • require-dbt-version
  • dbt-utils revision
  • pip install dbt into

0.18.0

We hope to have a release candidate of dbt v0.18.0 by the end of next week.

• Should we cut a dbt-utils v0.18.0-rc1, pre-gameday?
• Or is it enough to install dbt-utils from the dev/0.18.x branch when testing other packages that depend on it?

If it turns out that dbt v0.18.0 and dbt-utils v0.18.0 don't have any breaking changes for (e.g.) the Snowplow package, we can cut a new patch release of the Snowplow package right then and there, pre-gameday, just to update its dependency boundaries to ">=0.17.0,<0.19.0".

[clrcrl]

@jtcohen6 — I rewrote almost all of this and just did a force push. I think there's probably still some details to hash out once you've read through it, especially w.r.t. branching strategy.

[jtcohen6]

Thank you for making something (a saner, proportional-to-our-efforts approach) out of nothing (my contributions to a meandering discussion this morning).

I think the biggest trick is going to be adjudicating what counts as "new functionality." New macros makes sense to me. What about (e.g.) adding a new argument to the pivot macro, with no breaking changes for existing users?

IMO we want to be in the habit of releasing dbt-utils minor versions less often than dbt minor versions, since there's a lot of dependency wrangling any time we do it. That may mean making net-new contributors wait longer than they've had to in the past.

[jtcohen6]

I was thinking, "Shouldn't this be dev/0.x-1.0 (currently dev/0.5.0)? But I think having a safe, clear, main branch to PR against for small fixes or README updates is going to be much more user-friendly.