Book & playground don't match the latest published version

[mklopets]

Because the website (book + playground) get published from main, they are frequently out-of-sync with the reality that's been published & taken into use by others. This can cause a lot of confusion when "known good" queries suddenly start breaking in the playground, or when docs don't match the playground.

We've discussed this briefly in the past, but I wanted to open an issue since I've seen my team run into this multiple times in the past few weeks.

As the project gains popularity, I think we need to solve this in one way or another. Otherwise, this will make for a lot of frustrated would-be users. IMO, the published docs (+playground) should match the latest published version of the library, and they should also explicitly mention which version they're for.

[max-sixty]

I agree this isn't ideal. I can see a couple of ways of improving the situation:

1. Release faster. That would involve

• Mandating Changelogs & docs
  • Basically anything that makes @aljazerzen slower or less keen to write code is a negative for the project, so this depends on him mostly!
• Release every couple of days, either an automated workflow, a button anyone can press, or someone just does it.
• "Holding" breaking changes until we bump non-patch version
  • Another approach would be to implement versioning and have a feature flag for nightly. That would be a fun project, but a difficult one. (And some things would be very difficult / not worth doing, like changing the pest grammar.) Most of the payoff of that work is beyond this request

2. An option in the playground for which version is running. That would involve

• Having multiple wasm files when building the playground, either built from switching to a tag, or publish artifacts to GitHub and use those
• A UI for selecting them
• I'm not sure how difficult the JS side would be

3. Publish dev releases to npm. This would let folks like @mklopets work with a more up-to-date version. This is probably the easiest but least conclusive option?

@mklopets I totally get that having the playground & book match the currently released version would make development easier for you, and I really want development to be easy for you! But having it match the current main also makes development easier for the project, so I'd be a -0.5 on just making that switch, assuming one of the above is viable. That said, others should weigh in. Is that reasonable?

[richb-hanover]

Today, PRQL has conflicting priorities:

1. We need people to try out PRQL (because it's now "worth criticizing") to find the next most important improvement, notice the sharp edges, and discover actual bugs
2. Those people "crash up on the rocks" when the language changes out from under known-good queries.

My perspective: We're still very much in an experimental phase. There are lots of known "language issues" that need to be ironed out that will break queries. We should target intrepid users who can tolerate this breakage, all the while minimizing their pain. How can we do that?

• A clear sense amongst project members about how the language will evolve/stabilize over time, and about the amount of breakage to be expected in the next month, three months, six months, etc. (If we can... Maybe we're still so early in the process that we can't see an endpoint yet.)

• More frequent releases of patch versions. (Breakage be damned... for now.) This does require effort on the documentation: How can we get "descriptions scribbled on napkins" from developers that others can perfect?

• NOT versioning: I remain hopeful that PRQL can evolve so that every new language addition will be a strict superset of the 1.0 version. Even if that can't happen, it's way too early to make this investment in time and effort.

Thanks for listening.

[aljazerzen]

We're still very much in an experimental phase

This is the important part. It implies that we don't give any guarantees about stability. So there is point of having nightly builds or even different versions in playground.

I can vow to include CHANGELOG.md entries in my feat: commits and we can have a rule to also release a new minor version every two weeks.

[max-sixty]

We're still very much in an experimental phase

This is the important part. It implies that we don't give any guarantees about stability. So there is point of having nightly builds or even different versions in playground.

I can vow to include CHANGELOG.md entries in my feat: commits and we can have a rule to also release a new minor version every two weeks.

OK, this sounds very reasonable if you're up for that.

If the thing is just one line, I'm also happy to take it from the GitHub suggestion. (So maybe "feat" is a good filter)

[eitsupi]

Simple question: how hard is it to deploy the root docs only updated at release time and the edge docs in a sub directory like /dev?

[aljazerzen]

Not easy but doable. The problematic part is that we build the whole website+book+playground in a single build for a commit and book requires prql-compiler for compiling PRQL snippets to SQL.

[eitsupi]

One update per day may be sufficient, as in the Apache Arrow website?
I don't fully understand, but I believe Arrow does one nightly release every day and updates the dev docs once a day by nightly release (I often see failed deployments).
https://github.com/apache/arrow-site

[snth]

I agree with @mklopets and was going to raise the same thing.

@max-sixty What is the cost of operating the website? Or rather, would it be possible to get prql-dev.org and have main branch deploy there and only deploy to prql-lang.org on releases?

There presumably wouldn't be much traffic to prql-dev.org so egress charges and the like would be much lower.

I don't know how hard coded the prql-lang.org URL is everywhere. In my experience with internal sites, putting things behind a /dev/ prefix tends to be quite a pain but redirection at the DNS name level is relatively straightforward.

[richb-hanover]

I hear a consensus for putting out a release frequently. Before we design and build a lot of new machinery, let's see if this release schedule minimizes the problems that have been identified.

[max-sixty]

@max-sixty What is the cost of operating the website? Or rather, would it be possible to get prql-dev.org and have main branch deploy there and only deploy to prql-lang.org on releases?

It's approximately zero, and even if it were not, that would be fine.

FWIW something like readthedocs supports lots of versions (e.g. here's xarray: https://docs.xarray.dev/en/v0.21.0/). We don't use them because we're using mdbook, but there are probably options around that too. The only constraint is someone owning it...

[max-sixty]

Let's try the new release cadence, and see how that goes. I trust @mklopets will give an honest review!

I'll leave this open for the moment.

[mklopets]

I'm all for the other improvements on the release cadence but I do think the most "correct" solution would be to either:
a) either let the website's visitors specify the version via a select (more difficult), or
b) just always deploy the latest released version (easy).

having it match the current main also makes development easier for the project

This is obviously an important point, but I also think it's probably a simple one to address. Other than running the playground with the code of your active local compiler branch, are there any other difficulties here that I'm missing? I'm sure we could figure those out (& I'm more than happy to help).

While releases e.g. at most every 2 weeks would be great for continuing the momentum of the project (irregardless of this issue), I think it's incredibly user-unfriendly to have the docs & playground (not only the playground!) be incorrect for even a few days.

As we start to have more and more real, live use cases, these things will only get more important, and I'd hate for us to lose potential users because it externally looks like it's a "broken thing".

[snth]

While releases e.g. at most every 2 weeks would be great for continuing the momentum of the project (irregardless of this issue), I think it's incredibly user-unfriendly to have the docs & playground (not only the playground!) be incorrect for even a few days.

I agree with that.

I also got the impression that we only picked up some issues on release and deployment so I would vote in favour of doing some DEV/UAT/QA deployment somewhere which can be triggered on every merge to main.

[richb-hanover]

In a project this new and experimental, I don't think a small delay between implementing a new feature and seeing its documentation is bad. What makes newcomers suspect a project is broken is the observation that there isn't new development and that the documentation is out of date.

In an ideal world, new features and their documentation would roll out simultaneously.

1. One way to do this would be to have the Playground, Book, Website, VScode extension, etc. track the "main" release (0.4.0 as of this date). Anyone who wants to try out newer features can build their own system natively or with the Dockerfile.

2. Another way to approximate the ideal is with the stepped-up release cadence. But it means that we are (all) accepting new responsibilities:

   • Developers who change the language need to give an overview of how they work. This includes an entry in the CHANGELOG; a overview of the syntax/semantics; a link to a Issue or Discussion that gives background. That give a hint to our intrepid testers.
   • Others in the project need to step up to bring all the supporting materials (Playground, Book, Website, VScode extension - what else?) to a point that they reasonably describe the current state.

3. I suppose there are other paths, but any one requires both the new feature implementation and the documentation. The first gives us rigorously up-to-date documentation that lags the "state of the art" until there's a new release. When we hit 1.0, we'll probably choose this strategy. The second lets everyone play with the newest stuff, with the proviso that it may not be well documented.

Where do we think we are with PRQL's evolution? Thanks.

[aljazerzen]

I think we should go for option 1 here. It's easiest to implement while making sure that latest release, book & playground always match.

[TaxFairness]

I'm confused: Option 1 seems to lock us at a particular version until we get the documentation complete. This means that fewer people (only those who have set up a development environment) can try out any new features.

I prefer our current practice, which is more like option 2. It seems to me that model allows developers to make progress (and automatically push their updates into the Playground) as long as they leave hints (in the CHANGELOG) about how to use the new capabilities.

Option 2 also makes these updates available to a broader audience who can try the (possibly experimental) features and give feedback before writing definitive documentation.

How does this affect the release cadence? Some thoughts:

• We could plan to tag a new release in the week following the Dev Call. The call gives us the opportunity to discuss any last minute changes, interesting bugs, etc. and assess the state of the documentation, supporting tools/libraries, etc.
• An up-to-date CHANGELOG will make it relatively painless to bring documentation, etc. to a state that matches the expected release.
• Each new release would (I think) generate any static artifacts: VSCode extension, language bindings (?), others(?)
• It opens this question: What version should the Playground display in the comment at the end of the generated SQL? Today, should it be "0.4.1"? Or "0.4.2 (unreleased)"?

As the language matures, and we move closer to a 1.0 version, we can choose to switch the Playground to an actual released version, but for now, I think I'd rather make progress and let the documentation try to keep up.

Comments? Thanks.

[aljazerzen]

The main problem here is that playground & book don't match the latest release of the library and bindings. This is a huge problem for anyone who is not just "checking things out".

If we go with option 1, the people that want to experiment with latest changes, would indeed have to wait for a release. This could be eased by:

• having another deployment of playground and book (which is not the easiest to set up),
• regular release cycle (i suggested every two weeks).

[max-sixty]

I'm going to put names on things, because there are two comments enumerating options, so "Option 1" depends on which comment we look at!

• Latest commit — i.e. book & playground match the HEAD of main branch
• Latest release — i.e. book & playground match the most recent release

[max-sixty]

One downside of Latest release is that development on the book & playground itself has much longer feedback cycles. If we find a mistake in one of them, do we wait until the next release? There are currently very few tests on the playground; how do we assess whether it's good to release? Does whoever is doing the release need to manually check it works?

In Xarray / ReadTheDocs we got around this by having a branch called stable which was basically the most recent release, but could have things backported to it. But this introduces more branch management overhead.

[max-sixty]

It opens this question: What version should the Playground display in the comment at the end of the generated SQL? Today, should it be "0.4.1"? Or "0.4.2 (unreleased)"?

It should indeed say 0.4.2dev. That value comes from the value in Cargo.toml, which would need to be set by cargo-release as a post-release hook. Feel free to open an issue for this — contributions very welcome

[richb-hanover]

Thanks for the clear definition of "latest release" vs "latest commit". This follows other project's stable vs dev model.

• The downside of the latest commit model is that the Book and Playground won't always match, as @max-sixty noted.

• The downside of the latest release model is that "new stuff" won't be exposed to as many testers, and that there's an implied acceptance of the feature once it's "in a release" (even if we all agree PRQL is still highly experimental.)

I lean slightly toward the latter choice, but would accede to the needs of the group. As I said elsewhere I would like to take some time on this week's Dev Call to hash out the way we want to proceed for the next release or two, and how that might differ longer-term. Toward that end, I encourage everyone on this thread to participate, or to provide your thoughts. Thanks.

[snth]

tl;dr

I am in favour of the latest release model in combination with a Git Flow based process.

I suggest we should follow a process close to Git Flow. This is quite good account of it: Gitflow Branch Guide

• We maintain two protected branches: main and develop
• Releases are tagged commits on main.
• feature PRs are merged into develop.
• hotfix PRs are merged into main.
  • Before merging a hotfix, CI should test whether develop can easily be rebased on the post-merge main.
  • If not then someone needs to do the work to rebase develop on the hotfix PR branch before it is merged.
  • On merge develop is automatically rebased on the new main.
• When enough hotfixes have accumulated, a 0.x.Y release is published (where Y is incremented).
• When enough features have accumulated, a 0.X.y release is published (where X is incremented).

Apart from the occasional need to manually rebase develop, I don't think there should be much additional work involved in this compared to the current process.

Assumptions and background

My assumptions are the following:

• More people are using PRQL than we realise.
  • The Github stars are some indication but as we discussed on Discord the other day, there are probably many more people using it.
  • For those people PRQL just works for the most part.
• We are putting the messaging out there that PRQL is ready for use and we want them to use PRQL.
• For both those reasons, users need to be supported in terms of having correct documentation available for them.
• While according to SemVer, on 0.x.y we can make breaking changes at any point, we should rather work towards what people expect.

Git Flow

[aljazerzen]

I'm not sure if full git flow workflow is necessary - it add some complexity where it is not yet needed.

A simpler alternative is what we have, but releasing patches (0.x.Y) by picking all non-breaking non-conflicting commits to a new branch.

[snth]

I'm not sure if full git flow workflow is necessary - it add some complexity where it is not yet needed.

That's what I gather has been the general sentiment from the discussion so far. May I ask what you see as the complexity that would most concern you? I don't see it as too bad but I'm probably overlooking something.

I also think you and @max-sixty should be protected/isolated from it to allow you to focus on what you're currently focusing on and I would volunteer myself to manage the merging and branch management.

[aljazerzen]

Well, we don't (at least I don't) build compiler in features. It more like many commits, each making the feature better & more stable or changing it completely.

Also many of the smaller commits that we would want to release in a patch release are not hotfixes, but small improvements to recent releases.

[richb-hanover]

I'd like to talk about @snth's assertion on the Dev Call today (in 5 hours):

We are putting the messaging out there that PRQL is ready for use and we want them to use PRQL.

Do we agree that "PRQL is ready for use"? Or is it still "in an experimental phase"?

What commitments are we prepared to make today? How will those commitments affect development velocity? How could those commitments (including increased stability) evolve through the life of the project? Thanks.

PS I will note that I use PRQL all the time: I can no longer bring myself to write bare SQL PRQL is just too easy. But I do it with the clear knowledge that I'm going to have to rewrite my switch statements soon.

[richb-hanover]

PPS - I appreciate all the thought that people have put into the different ways that we could organize the project. They're all reasonable approaches. And some day, I'm sure we'll pick one (or a combination) for a 1.0 (or near-1.0) timeframe.

The question I'm asking is "What do we want to do for the next 2-3 months?" Thanks again.

[snth]

Also many of the smaller commits that we would want to release in a patch release are not hotfixes, but small improvements to recent releases.

@aljazerzen I wonder whether that's just a matter of terminology. Those could all still go to main. I'm mainly just suggesting that feat: commits/PRs go to develop so that we can release 0.x.Y versions without including those.

[aljazerzen]

Update from the meeting:
- introduce stable branch which tracks the latest release + hotfixes/tweaks + docs changes,
- most PRs are still merged into main (including docs of new features),
- book & playground are released from stable branch,

We can look into having another release for the main branch published in parallel to current deployment, so there would be a "bleeding edge" for people to experiment with. For now, we can try to keep releases frequent.

[max-sixty]

Thanks for adding the summary @aljazerzen !

Slight tweak, from my understanding:

• introduce stable branch which tracks the latest release + hotfixes/tweaks + docs & playground fixes,
• most all PRs are still merged into main (including docs of new features),
• book & playground are released from stable branch,

The only thing we use stable for is changes to the website or docs fixes; any fix to PRQL's compiler or semantics needs an actual release — otherwise we get neither the simplicity of a single branch, nor the playground & docs matching the semantics of the latest release.

---

Contributions welcome for moving this over!

[richb-hanover]

Thanks for the terrific discussion on the Dev Call last Sunday. For those who weren't on the call, here is a more fleshed-out version of my understanding of the discussion and comments from @max-sixty and @aljazerzen above. Comments appreciated.

Going forward, PRQL will maintain two important branches: stable and main.

• The stable branch tracks the latest released version of PRQL. It provides a base for people who wish to use it today. The Playground, Language Book, and CHANGELOG match the implementation of the stable release. In addition, any external integrations - VSCode extension, language bindings, etc. - all match the stable release. Only updates to the website or Language Book are permitted in the stable branch. No changes to the PRQL language or hot fixes/tweaks are allowed (those go into a new release).

  NOTE: PRQL is still an experimental project. We reserve the right to make the language better even if we must break previous queries. The PRQL language - even the stable branch - may (and probably will) change between releases as the language evolves. However, the stable release never changes once it has been released.

• The main branch receives PRs as development occurs. New features should be documented first in the "Unreleased" section of the CHANGELOG file, and in the Language Book/website as needed. It is an open question how the Web, Language Book, and external integrations will be made available to people who don't have a development environment set up.

• From time to time (perhaps as frequently as every two weeks) we merge the main branch into the stable branch, and the new code becomes the next "released version".

[max-sixty]

That's an excellent base @richb-hanover . I'm going to take it and tweak it a bit, if that's OK.

---

Once implemented, PRQL will maintain two important branches: stable and main.

• The main branch receives all PRs as development occurs. New features should be documented first in the "Unreleased" section of the CHANGELOG file, and in the Language Book/website as needed.

  There are no changes to how main branch works from this discussion

• The stable branch tracks a) the latest released version of PRQL plus b) any cherry-picked commits from main that fix issues with the website, book, and playground. The website, docs, and playground are deployed from stable. No changes to the PRQL language or hot fixes/tweaks are backported to stable — those go into a new release.

  Whoever implements this should try and find an easy way to backport commits to stable, I think there may be bots out there which respond to a label added to a PR

• When we release a new version, we reset stable to that release. At that point both main and stable will match the latest release.

Until we set this up, we're operating as we were — playground & book & website are deployed on every merge to main.

---

Does everyone agree with this? It would mean that #1611 should merge into main

---

FWIW, I think this has not been characteristic of our usually successful coordination efforts. I'm not sure why, I'm extremely open to feedback. Possibly I have done a bad job structuring the discussion, so ideas have been all over. The actual change we're making is very small.

Maybe I should have been clearer that we should make the "minimum sized change"? Ideas like git flow etc were never viable imo; were we exploring way too big a space to fix the immediate issue?

[richb-hanover]

That's an excellent base @richb-hanover . I'm going to take it and tweak it a bit, if that's OK.

Definitely. Feel free to edit/expand on this.

FWIW, I think this has not been characteristic of our usually successful coordination efforts. I'm not sure why, I'm extremely open to feedback. Possibly I have done a bad job structuring the discussion, so ideas have been all over. The actual change we're making is very small.

I'm not sure that you, as a leader, have the responsibility for structuring the discussion. You need to (and do) make everyone feel comfortable throwing ideas into the mix. But you can also structure the goals and the decision process: We must do [a | b | c]; we have to decide by X date; we can afford no more than Y hours of development time; we have Z resources available to us; etc.

I hope that my behavior did not come across as unhelpful. I agree the technical change we specified is pretty small, but I saw the decision would have large ramifications to the project because of increased commitments to testers and users. Until now, we've had a simple "developer workflow" - just check stuff in, and that's what the world sees. This change establishes both branches, with the need to maintain the attendant machinery, cognitive overhead, potential for updating the wrong branch, etc.

I wanted the group to declare affirmatively that it was time to switch to this new, more involved development model. And we did.

Let me be clear: this kind of change was always on PRQL's horizon. Although I had been hoping to put it off for a little while longer, I am OK embracing it now. Thanks for listening.

[snth]

@max-sixty the problem I see with your description in #1507 (comment) is that how do we release 0.4.3 with hotfixes while we also have features pending that are only supposed to go into 0.5.0 (such as the switch syntax change)?

My understanding of the proposal was closer to @aljazerzen 's description: #1507 (comment)

Cherry-picking descriptions from the comments above, it would be:

• introduce stable branch which tracks the latest release + hotfixes/tweaks + docs changes,
• The main branch receives all PRs as development occurs.
• hotfixes and docs fixes are cherry-picked and backported to stable.
• after sufficient hotfixes have accumulated a patch release is cut from stable.
• book & playground are deployed from stable, either on every merge or only on releases.
• when it's time for a major or minor release, main is merged into stable and a new release is deployed.

---

Since I'm not that close to the actual compiler code, I don't feel strongly about this and am happy to go with the simpler workflow proposed if it is felt that backporting introduces too much overhead and potential for merge conflicts.

However how do you then deal with "breaking" changes like the switch syntax change? Do you keep those PRs open and unmerged until it's time to cut the major release? Won't you have the same problems with possible merge conflicts at major release time then?

In fact I suspect it would probably be worse because you would have multiple major changes which wouldn't have been merged together yet. Backporting hotfixes to stable would probably be much simpler because hotfixes would probably be just minor changes and not big features and then the main branch would at least always be fully integrated.

[max-sixty]

I can comment more fully but to stop at the first problem I see with that proposal: it doesn't fulfil the original goal of this effort: the compiler in the playground doesn't match the latest release, since it includes fixes. Do you agree?

[snth]

the first problem I see with that proposal: it doesn't fulfil the original goal of this effort: the compiler in the playground doesn't match the latest release, since it includes fixes. Do you agree?

Sorry, I didn't explain it very well. The way I saw it, playground and docs would only be updated/deployed on releases, including patch releases. They therefore would match the latest release code.

book & playground are deployed from stable, either on every merge or only on releases.

The reason I included "on every merge" (to stable) in the above is that there was a concern expressed that without tests for the playground it is difficult to ascertain whether it's working properly or not. So the way I pictured it is:

• a hotfix with documentation update is backported to stable
• the docs and playground are automatically deployed giving core contributors a chance to test the fix
• if everything is found to be working then a patch release is cut.

So there would indeed be a bit of drift between playground and latest release but it would likely be only minor since it's just hotfixes and the patch release should follow soon.

If that's undesirable then playground could only be deployed on releases. That would probably be my preference. The only downside is that if there's a bug in the hotfix and the playground goes into a broken state then another patch release would have to follow immediately, but that could be ok.

[max-sixty]

So the way I pictured it is:

a hotfix with documentation update is backported to stable
the docs and playground are automatically deployed giving core contributors a chance to test the fix
if everything is found to be working then a patch release is cut.

What do you see as a hotfix here? A change to the compiler or just a docs change?

If it includes a change to the compiler, we violate the original goal, no? I think you agree but don't think it's a big deal, is that right? Though why are we doing this at all then? So we can avoid having breaking change PRs waiting before we do a non-partch release??

If it only includes docs & playground changes, then it's the same as the slimmer proposal I wrote above, is that right?

If hotfixes include changes to the compiler, that's most changes. What's the process for doing those? Who would be responsible? Who is managing merge conflicts. How do we oversee there are no mistakes? How do we indicate the differences between the released version, the compiler behavior on stable, and the compiler behavior on main?

[richb-hanover]

I wonder if we are being distracted by discussions of hot fixes and maintaining multiple major versions and all the attendant effort and headaches caused by multiple moving parts. These will be important once we become a large project that's heavily-used by many, many people.

But we're still a small project, with relatively few intrepid people using it. Although we strive to keep things working, our commitment is to fix in the next release. At the last Dev Call, I understood that we agreed to aim for a two week (more or less) release cycle, and that no changes to the compiler would occur between those releases.

As a practical matter, that's pretty rapid. Do we expect to find a critical bug that needs to be addressed outside that two-week cadence? (If we found that PRQL computes 1+1=3, I suppose we could create a new out-of-schedule release.) But otherwise, let's stick with the plan.

About version numbers: (0.4.2 vs 0.4.3 vs 0.5.0 vs ...) Although our roadmap calls for certain features to be in 0.5.0, If it's ready - ship it. No one would be disappointed to see it early and nothing bad would happen if a so-called "0.5.0 feature" were incorporated in the next 0.4.x release.

So my advice is to stick with the two-week release cadence for the compiler. Documentation and website fixes can go in at any time, and the CHANGELOG can track anything that's waiting for the next release.

Thanks for listening.

[snth]

tl;dr: Ignore this and skip to below the line.

This discussion is getting bigger than I intended and I will try to wrap up my participation because @aljazerzen reminded me there is a simpler way to see it which was discussed on Discord and which I think is what @richb-hanover is trying to say above and probably @max-sixty too (which I think I'm misunderstanding).

If it includes a change to the compiler, we violate the original goal, no? I think you agree but don't think it's a big deal, is that right?

Not in my understanding but I think I am missing something. Regarding whether it's a big deal or not, I think it's a relatively big deal but I think I should have no say in it since I'm not doing the actual work and it's you, @aljazerzen and @max-sixty, that are writing new features and integrating PRs so you should decide on a workflow that works for you. My description of my understanding of the process was just to offer up an idea to see if it resonated with someone. I didn't mean it to be prescriptive in any way.

If hotfixes include changes to the compiler, that's most changes.

In my mind it would have been the minority but that's probably because I've been too far removed and mostly watch for new features and don't keep an eye on all the day-to-day fixes you integrate.

What's the process for doing those? Who would be responsible? Who is managing merge conflicts. How do we oversee there are no mistakes?

I did offer up my time for this (probably foolishly) for this here: #1507 (comment)

I also think [you][@aljazerzen] and @max-sixty should be protected/isolated from it to allow you to focus on what you're currently focusing on and I would volunteer myself to manage the merging and branch management.

Of course I wouldn't be able to guarantee there are no mistakes but in my mind I would just be handling the simpler task of backporting changes from main to stable. The main error checking and integration work would be with merging the PR into main in the first place. The backporting is just a bit of git gymnastics (in my mind, probably mistakely).

How do we indicate the differences between the released version, the compiler behavior on stable, and the compiler behavior on main?

Would we need to indicate this? The only visible one would be the released version, both in code and docs. stable would be only one or two commits ahead until the next patch release. main is "here be dragons" territory like the "semantic branch" in days of old.

---

Summing up, let's ignore everything I've said above. @aljazerzen reminded me that probably the easiest and most practical is to go with the simple flow proposed by @max-sixty above and not worry too much about 0.4.x and 0.5.0 versions and rather just keep going with 0.5, 0.6, 0.7, ... for each breaking change and keep up the high cadence proposed above.

[max-sixty]

OK let's go with the approach above for a while.

I'm very happy to reassess after we've lived with it for a while. I'll close this.

---

Meta comments:

• This was the first PRQL discussion where I've thought "we really should move forward, this is not a productive discussion"
• I think that's mostly because I was quite confident that the prior simpler suggestion was much better than the proposal. (Maybe that's wrong!) And that the decision was fairly easy to change in the future, and we weren't making much progress on the discussion (maybe my fault!)
  • I'm very hesitant to say "Trust me, I've thought about this, I have the track record". Generally folks are massively under-confident in raising objections, so me adjusting towards welcoming objections gets us to a better balance
• But I should have found a way of getting to a conclusion sooner.
  • Something that may have been helpful for the discussion is suggesting some way to provide evidence for a position in order to make progress
    • I've found that a productive way of moving discussions forward in the past — e.g. "I'll close this issue but feel free to re-open with cases where K happens" or "in order to make this change in PRQL, we'd need X & Y", rather than just "I don't like it, fork it if you want". It's both a way of increasing evidence and a screen on over-discussing on slim evidence
  • Maybe I should have asked for other projects which had done the frequent backporting? Or suggesting we try the simpler version and assess? Or list the commits that would need to be backported from the last 10?
• I think one adjustment that folks could consider is to scale comments by the likelihood that the prior view is missing something
  • For example, "have we thought about allowing underscores in numbers" is very useful — there's no record anyone has thought about it, highly likely that it's going to be additive
  • But partly I wrote this comment because to make progress on this issue at that stage, we really had to think through the details, it's insufficient to just be raising ideas. Data like "what proportion of commits would need to be backported" a) are important and b) someone has very likely thought about and incorporated into the prior view
    • Though notably @snth's had thought about some of these! So I was likely underestimating the validity of that POV at that stage

      I did offer up my time for this (probably foolishly) for this here: Book & playground don't match the latest published version #1507 (comment)

  • I also think this level of epistemic accuracy is really difficult. It's easy to go too far the other way, and we all say "The person who was here first knows more than me, who am I to disagree", which leads to terrible outcomes.
  • FWIW (and not to criticize the generous sentiment!) — I think the above frame of "is the prior likely to be wrong" is a better frame for deferring to others than the below. The below proves too much — it's also not a bug reporter doing the work, but the bug report is very helpful.

    but I think I should have no say in it since I'm not doing the actual work and it's you

• In case there's any ambiguity, totally all good from an interpersonal POV, I very much think that everyone is acting with the project's best interests. Feedback very welcome, either here or at the link on my GH profile.