Faster refresh

[jesseduffield]

This PR makes various changes to speed up refreshes. Notably we're using more concurrency with waitgroups, and in some places we're avoiding nested loops.

I've got a synchronous refresh of all models (in the lazygit repo) down from 250ms to 100ms. You'll notice for example that switching branches is faster.

Jul 29 09:49:48 |INFO| refreshing all scopes in sync mode
Jul 29 09:49:48 |INFO| refreshed staging in 36.752µs
Jul 29 09:49:48 |INFO| refreshed merge conflicts in 26.978µs
Jul 29 09:49:49 |INFO| git -c log.showSignature=false log -g --abbrev=40 --format=%h%x00%ct%x00%gs%x00%p (21.220731ms)
Jul 29 09:49:49 |INFO| git tag --list -n --sort=-creatordate (21.255125ms)
Jul 29 09:49:49 |INFO| git merge-base HEAD HEAD@{u} (21.504375ms)
Jul 29 09:49:49 |INFO| git merge-base HEAD refs/remotes/origin/master (24.661184ms)
Jul 29 09:49:49 |INFO| postRefreshUpdate for tags took 4.425477ms
Jul 29 09:49:49 |INFO| refreshed tags in 27.256406ms
Jul 29 09:49:49 |INFO| git status --untracked-files=all --porcelain -z (37.026009ms)
Jul 29 09:49:49 |INFO| refreshed files in 37.616105ms
Jul 29 09:49:49 |INFO| git log HEAD --topo-order --oneline --pretty=format:%H%x00%at%x00%aN%x00%ae%x00%D%x00%p%x00%s --abbrev=40 -300 --no-show-signature -- (43.570288ms)
Jul 29 09:49:49 |INFO| git stash list -z --pretty=%gs (43.940535ms)
Jul 29 09:49:49 |INFO| postRefreshUpdate for stash took 2.322359ms
Jul 29 09:49:49 |INFO| refreshed stash in 46.637992ms
Jul 29 09:49:49 |INFO| git branch -r (49.167657ms)
Jul 29 09:49:49 |INFO| postRefreshUpdate for remotes took 1.063829ms
Jul 29 09:49:49 |INFO| postRefreshUpdate for remoteBranches took 10.219µs
Jul 29 09:49:49 |INFO| refreshed remotes in 51.020293ms
Jul 29 09:49:49 |INFO| postRefreshUpdate for commits took 7.805133ms
Jul 29 09:49:49 |INFO| refreshed commits and commit files in 51.970354ms
Jul 29 09:49:49 |INFO| postRefreshUpdate for reflogCommits took 52.564081ms
Jul 29 09:49:49 |INFO| git for-each-ref --sort=-committerdate --format=%(HEAD)%00%(refname:short)%00%(upstream:short)%00%(upstream:track)%00%(subject)%00%(objectname:short=8) refs/heads (15.701524ms)
Jul 29 09:49:49 |INFO| postRefreshUpdate for localBranches took 839.021µs
Jul 29 09:49:49 |INFO| refreshed reflog and branches in 94.863103ms
Jul 29 09:49:49 |INFO| Refresh took 95.125968ms
Jul 29 09:49:49 |INFO| postRefreshUpdate for submodules took 20.714µs
Jul 29 09:49:49 |INFO| postRefreshUpdate for files took 171.282µs

The next big opportunity for speeding things up is in how we write content to views. Right now we write all of our list to the list view, and if we have lots of items in our list, it takes longer. Notice that refreshing the reflog and branches took 94ms which is roughly the total refresh time. Within that, it takes 52ms to write the reflog commits to the reflog view. So that's the current bottleneck. If reflog commits write to the view instantly then the total refresh time goes down to ~50ms. The solution is to only write enough to the view to fill the viewport, and then refresh the viewport as we scroll.

If anybody wants to look into performance in future, I tested this all locally by changing the refresh that happens upon pressing shift+R to be synchronous (meaning it waits for all refreshes to be done before returning)

@@ -128,7 +128,7 @@ func (self *GlobalController) createCustomPatchOptionsMenu() error {
 }
 
 func (self *GlobalController) refresh() error {
-	return self.c.Refresh(types.RefreshOptions{Mode: types.ASYNC})
+	return self.c.Refresh(types.RefreshOptions{Mode: types.SYNC})
 }
 
 func (self *GlobalController) nextScreenMode() error {

Then I ran lazygit with LOG_LEVEL=info lazygit -debug and in another terminal pane I viewed the logs with lazygit --logs.

• Please check if the PR fulfills these requirements

• Cheatsheets are up-to-date (run go run scripts/cheatsheet/main.go generate)
• Code has been formatted (see here)
• Tests have been added/updated (see here for the integration test guide)
• Text is internationalised (see here)
• Docs (specifically docs/Config.md) have been updated if necessary
• You've read through your own file changes for silly mistakes etc

[jesseduffield]

no need for a nested wait group so I've split this function into two