Stuck trying to implement a GitHub-based branch workflow

[Brixy]

Hi all,

After working with jj for a few weeks without branches on single user repos (which is a breeze!), I am now trying to implement a GitHub-based team workflow – but to be honest: I’m stuck.

What I’m trying to do is to either merge or (preferably) squash the two branches into main:

@  nxwqqquv [email] 2024-02-29 12:33:58.000 +01:00 9ab2f662
│  (empty) (no description set)
│ ◉  ryssvmnl [email] 2024-02-29 12:33:49.000 +01:00 add-b-files 
c64481bf
│ │  Add file b2
│ ◉  tyzxqvuu [email] 2024-02-29 12:33:04.000 +01:00 3816244f
├─╯  Add file b1
│ ◉  wvnmqyxu [email] 2024-02-29 12:32:46.000 +01:00 add-a-files 
6e095ca2
│ │  Add file a2
│ ◉  zszlwnvy [email] 2024-02-29 12:32:40.000 +01:00 98b3b816
├─╯  Add file a1
◉  olzklxtq [mail] 2024-02-29 12:32:17.000 +01:00 main HEAD@git 
05bf021e
│  Init
◉  zzzzzzzz root() 00000000

Trying a merge workflow

Test 1

jj new o w
jj describe -m 'Merge branch add-a-files'
jj new o r
jj describe -m 'Merge branch add-b-files'

ls
# b1  b2  README

It seems as if the first merge has disappeared from main.

Test 2

jj new o w
jj describe -m 'Merge branch add-a-files'
jj new r q # I tried many different combinations here
jj describe -m 'Merge branch add-b-files'

ls
# a1  a2  b1  b2  README

jj new main
ls
# README # [1]

jj undo

git switch main
ls
a1  a2  README # [2]

Again, I tried to merge two branches into main, but either both ([1]) or one branch ([2]) disappeared.

Rebase workflow

jj rebase -b w -d o
jj describe -m 'Rebase branch add-a-files into main'

ls
# README

No rebase seems to have happened.

---

Sorry, guys. I do know that this is jj’s expected behavior. But how do I a) merge or b) rebase two branches into main? Or is there any branch-less workflow you can use with GitHub?

Unless I have missed something, there are no instructions on how these two typical GitHub-based workflows work with jj.

Thanks for your help.

[thoughtpolice]

To give an immediate answer your question using the merge workflow: jj new can create a merge with any number of parents. So you don't need to do two merges, unless you want to. Instead, you can run this, assuming your original commit graph is what things look like:

jj new o w r
jj describe -m 'merge: add-a-files and add-b-files to master'
jj new # create a new working copy on top of the merge commit

This will result in a graph that looks like this, assuming the base is X, and the merge commit commit is M

   ------> A1 -----> A2
  /                    \
X --------------------> M ---> @
  \                    /
   ------> B1 -----> B2

So, the resulting merge commit M has 3 parent commits: X (base), A2 and B2. Most Git users are only used to merges with exactly 2 parents, but in Jujutsu you can have as many parents as you want, trivially. (I currently have a merge with 5 parents in my own copy of the jj source code.)

The last two commands can be shortened thanks to jj commit, which is just describe + new:

jj new o w r
jj commit -m 'merge: add-a-files and add-b-files to master'

This graph is what you want from the examples. Now to focus on specific examples and why they didn't do that:

Merge Test 1: accidentally creating different merges

Let's rewrite your example to use the names of the nodes above:

jj new X A1 # results in M1
jj commit -m "merge 1"
jj new X A2 # results in M2
jj commit -m "merge 2"

The result of this example is two different merge commits entirely. In short, it makes your graph look like this:

   ------> A1 -----> A2 -----> M1
  /                           /
X ----------------------------
  \                           \
   ------> B1 -----> B2 -----> M2 ---> @

The second command should NOT use the base X. It should use the resulting commit from the first command as its base. So, you would instead do:

jj new X A2  # results in M1
jj new M1 B2 # results in another merge with M1 as a parent
jj new # create a new working copy, leaving the merge commit

The resulting graph of these commands would be the following "criss-cross" merge:

   ------> A1 -----> A2 -----> M1
  /                           /  \
X ----------------------------    \
  \                                \
   ------> B1 -----> B2 ----------> M2 ---> @

Merge Test 2: main does not automatically move

You did the right thing here. You performed a criss-cross merge like I outlined above. However, the error was in your last command:

...
jj new main

When you do this, you are saying "Create a new working copy on top of the branch main." Let's say that X above is main. Then you get this graph:

   ------> A1 -----> A2 -----> M1
  /                           /  \
X (main) ---------------------    \
  \                                \
   ------> B1 -----> B2 ----------> M2
    \
     ------> @

Your merge commit M2, the one you created, still exists. However, your working copy @ is now on top of X. Why? Because...

jj does not automatically make the main branch point to M2, and it does not modify where a branch points unless you explicitly tell it

So, you don't see your files, because they're in another commit. This is no different in theory than having changes on a branch in git, then switching away from it. The difference is that in git commit, branches advance automatically. In jj, branches DO NOT advance automatically: you must use jj branch set in order to modify where a branch points.

Given this, the way to achieve what you wanted here was instead to:

jj new X A2 # create M1
jj commit -m "merge 1: X + M1"
jj new M1 B2 # create M2
jj commit -m "merge 2: M1 + B2"
jj branch set main -r M2

This will be the resulting graph:

   ------> A1 -----> A2 -----> M1
  /                           /  \
X ----------------------------    \
  \                                \
   ------> B1 -----> B2 ----------> M2 (main) ---> @

So main now points to M2.

Rebase test, Part 1: rebasing the changes on each other

First off, I don't think you want to run -b w, but -b add-a-files, i.e. you want to give it a branch name to -b. You could also use jj rebase -s z -d o; -s basically says "rebase the commit and all descendents", so you can rebase whole trees. For this to work, you give it the base of the branch, not the tip; hence why we give it z which is the base commit of branch add-a-files.

Second: the command is a no-op. Branch add-a-files is already a linear chain on top of main; there is no need to rebase. It's "fast-forwardable" in Git lingo, in other words. What you probably want instead is to rebase add-b-files onto add-a-files. I'd do that with this:

jj rebase -s t -d w

Then you can set the branch pointer to the tip of the new series, which ends with the final commit of add-b-branches, commit ryssvmnl:

jj branch set main -r ryssvmnl

This will result in a linear history with 4 new commits, which is probably want you want. But there's a final thing to note...

Rebase test, part 2: jj rebase does not modify the working copy

But more than that, jj rebase will not modify the working copy. This is a very subtle thing to understand that is actually a key part of Jujutsu's design. Let's look at your original commit graph:

   ------> A1 -----> A2
  /
X (main) ---> @
  \
   ------> B1 -----> B2

Your working copy @ is based on X, which is where main points. You have two branches. You want to only have one branch. Let's rebase the B branch onto the A branch. We can do that with this command:

jj rebase -s B1 -d A2

i.e. "rebase the tree starting with B1 onto the commit A2", which is fairly intuitive, I think.

What's the resulting graph? It's not what you expect:

   ------> A1 -----> A2 ------> B1 -----> B2
  /
X (main) ---> @

The rebase occurred, but your working copy was not moved. You can get the result you want by running:

jj new B2

to get:

X (main) ------> A1 -----> A2 ------> B1 -----> B2 -----> @

Then you can run

jj branch set main -r B2

to get:

X ------> A1 -----> A2 ------> B1 -----> B2 (main) -----> @

More about jj rebase

The very subtle distinction I pointed out above is important. In short, jj rebase works on the repository. When I say "repository", I roughly mean "all of the data structures that track the state of files, but not the files themselves." Put another way, jj rebase modifies the commit graph. It does not necessarily modify the files you have checked out in your working copy.

A good contrast is Git, where git rebase actually works on the literal files in the working copy. It works like this: you want to (in Git) rebase 20 commits onto a branch, any branch at all. OK, so first git checks that the working copy and index are clean, i.e. git status returns nothing. If not, it fails. Assuming that it does not fail, it then checks out the destination branch, and does the following operations in sequence, starting with the first commit you gave it

1. Take the patch, and apply it to the working copy, and git add it
2. If there's a conflict, error (dreaded "git rebase" conflict resolution) and let the user resolve it
3. If there's no conflict, then run git commit
4. The commit has now been rebased

You repeat this process for all 20 commits, sequentially, in order. It works almost exactly how you might expect it to. In other words, git rebase modifies the working copy. It has to actually take the patches from the history, apply them, git commit them, write all that to disk, then do it over and over again.

jj rebase does not do this. It does not modify any files in the directory on your computer when you run it in the example above. Instead, it modifies the commit graph. It is modifying a data structure that exists on disk in the .jj/ directory, NOT the files you are using. This is a profound difference in approach and has many advantages — notably jj rebase is far more efficient than git rebase will ever be, because it operates almost entirely in-memory.

I'm not sure if you've gotten into conflict resolution yet — it's the most magical part of Jujutsu by far — but this also means that if a conflict happens, jj keeps the conflict in the commit and tracks it precisely. It does not simply fail and immediately exit the process like git rebase, leaving the user to resolve it; instead, it records in its data structures that this rebase had conflicts, and these conflicts exist in these commits. It then exits. Rebase does not "stop" at all, like git rebase might "stop" in step 2 above; Jujutsu performs the entire algorithm, to its completion, and exits. That's all.

Final notes

The main summaries of this post are:

• You basically did the right thing! But,
• Branches are almost entirely used for Git interop; they are not so useful otherwise in a pure "Jujutsu world"
• In Git, they're mostly used to share work in a repository, so you want the branch pointer to get updated
• We don't automatically "advance" branches, like Git does when you run git commit. Many people have talked about this as an onboarding problem, and there is some work underway to allow this behavior. (Implement FR #2338 for current branches / automatically advancing branches #3129)
• Most jj do not modify the working copy — including rebase and others — so most of the time, you will remain "where you were" after you run a command

I think once you keep in mind that branches aren't "advanced" automatically, most of the above mistakes might make a bit more sense. :) I hope this was helpful.

[Brixy]

Thank you very, very much, @thoughtpolice! I really appreciate your detailed answers.

When I saw the scope of your reply, I thought I would have to read the answers quite a few times. But your explanations are so precise that one can get it straight away.

Your graphs make everything super clear and are a great reference. (Something like this would be an excellent addition to the official documentation, especially for onboarding.)

Your “bonus section” about jj rebase is a good read, and I do see jj’s advantages here. It’ll take some time to internalize this.

Branches are almost entirely used for Git interop; they are not so useful otherwise in a pure "Jujutsu world"

That’s what I thought and what I favor. All I want is to be prepared to deal with pull requests ;-) For my stuff I don’t use branches at all.

We don't automatically "advance" branches, like Git does when you run git commit. Many people have talked about this as an onboarding problem, and there is some work underway to allow this behavior.

Yes, it requires to rethink. I cannot judge whether a change here is necessary (I will the discussion about this), but again: Your graphs make this immediately understandable.

I'm not sure if you've gotten into conflict resolution yet

Just a bit. I don’t exactly know why, but conflicts were a lot easier to resolve than in git.

I hope this was helpful.

You wouldn’t believe how much! I really got it.

Thank you very much for your time, your efforts and your patience.