Whitelist-Push and MergePullrequest does not work when Branch-Protection is enabled #7754

sharpSteff · 2019-08-05T07:18:08Z

Gitea version (or commit ref):
1.9.0+3-g6d441de
Go Version
1.11.5
Git version:
2.7.4
Operating system:
LXC container on ubuntu 16.04 LTS
Database (use [x]):
- PostgreSQL
- MySQL
- MSSQL
- SQLite
Can you reproduce the bug at https://try.gitea.io:
- Yes (provide example URL)
- No
- Not relevant
Log gist:

Description

Whitelist-Push and MergePullrequest does not work when Branch-Protection is enabled:

Pushing develop
Enumerating objects: 7, done.
Delta compression using up to 12 threads
Total 4 (delta 3), reused 0 (delta 0)
POST git-receive-pack (530 bytes)
Remote: Gitea: Internal Server Error        
Remote: Fail to detect force push: exit status 128 - fatal: Not a git repository (or any parent up to mount point /srv/git)        
Remote: Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).        
Remote: 
Pushing to myserver
To myserver
Error: failed to push some refs to 'myserver'

Merging Pullrequests returns 500 Error

With Gitea 1.8.3+2-g11f6ed4 everything works just fine.

Screenshots

The text was updated successfully, but these errors were encountered:

lunny · 2019-08-05T16:24:00Z

Cannot reproduce this on my local

zeripath · 2019-08-06T05:13:40Z

We need to see the Gitea log.

sharpSteff · 2019-08-06T05:53:12Z

@lunny I also can not produce this bug on try.gitea.to
Might be related to our infrastructure (dynamic proxy, etc..)

I think 356854f has something to do with this.

before:

.....

// detect force push
if git.EmptySHA != oldCommitID {
	output, err := git.NewCommand("rev-list", "--max-count=1", oldCommitID, "^"+newCommitID).RunInDir(repoPath)

	if err != nil {
		fail("Internal error", "Fail to detect force push: %v", err)
	} else if len(output) > 0 {
		fail(fmt.Sprintf("branch %s is protected from force push", branchName), "")
	}
}
.....

after

if git.EmptySHA != oldCommitID {
	env := append(os.Environ(),
		private.GitAlternativeObjectDirectories+"="+gitAlternativeObjectDirectories,
		private.GitObjectDirectory+"="+gitObjectDirectory,
		private.GitQuarantinePath+"="+gitObjectDirectory,
	)

	output, err := git.NewCommand("rev-list", "--max-count=1", oldCommitID, "^"+newCommitID).RunInDirWithEnv(repo.RepoPath(), env)
	if err != nil {
		log.Error("Unable to detect force push between: %s and %s in %-v Error: %v", oldCommitID, newCommitID, repo, err)
		ctx.JSON(http.StatusInternalServerError, map[string]interface{}{
			"err": fmt.Sprintf("Fail to detect force push: %v", err),
		})
		return
	} else if len(output) > 0 {
		log.Warn("Forbidden: Branch: %s in %-v is protected from force push", branchName, repo)
		ctx.JSON(http.StatusForbidden, map[string]interface{}{
			"err": fmt.Sprintf("branch %s is protected from force push", branchName),
		})
		return

	}
}

I'll try to provide the gitea log.
We downgraded to 1.8.3+2-g11f6ed4 , so it might take some time

zeripath · 2019-08-06T06:31:05Z

Ah.

I've figured this out.

That repo path needs to be prefixed with the Gitea repository path.

You're running Gitea from a weird working directory meaning that repo.RepoPath() won't work.

Nope this was totally wrong.

jdehaan · 2019-08-06T06:47:28Z

I am taking care of the infrastructure for the system on which the problem was found by @sharpSteff.

INFO: Gitea is running behind a reverse proxy mapping the server to a sub-directory (/git).

As mentioned rolling back to HEAD of 1.8 branch solves the issue, there must be something breaking in 1.9 regarding such setups. Is this information enough?

jdehaan · 2019-08-06T07:00:20Z

I could find the related entry in the log file (the xxxxx and yyyyyy are anonymizing internal names):

2019/08/05 06:29:28 .../xorm/session_get.go:99:nocacheGet() [I] [SQL] SELECT `id`, `repo_id`, `branch_name`, `can_push`, `enable_whitelist`, `whitelist_user_i_ds`, `whitelist_team_i_ds`, `e
2019/08/05 06:29:28 ...ters/private/hook.go:74:HookPreReceive() [E] Unable to detect force push between: aa77e1d18966bf5c48b9b913db3c857c9bfbabe9 and 62d7b4ea78941e254f95a38831c20f1a17e5361
        Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
        
2019/08/05 06:29:28 routers/repo/pull.go:629:MergePullRequest() [E] Merge: git push: From /srv/git/xxxx/yyyyyy.xxxx
         * branch            develop    -> FETCH_HEAD
         * [new branch]      develop    -> head_repo/develop
        Switched to a new branch 'head_repo_develop'
        .git/rebase-apply/patch:11: trailing whitespace.
                                        bat "AdvancedInstaller.com /edit ${AdvancedInstallerSetupFile} /SetAppdir -path [ProgramFilesFolder]YYY\\[ProductName] -buildname Build"\01503d
        .git/rebase-apply/patch:12: trailing whitespace.
                                        bat "AdvancedInstaller.com /edit ${AdvancedInstallerSetupFile} /SetAppdir -path [ProgramFilesFolder]YYY\\[ProductName] -buildname DefaultBuild"\01503
        warning: 2 lines add whitespace errors.
        Switched to branch 'master'
        remote: Gitea: Internal Server Error        
        remote: Fail to detect force push: exit status 128 - fatal: Not a git repository (or any parent up to mount point /srv/git)
        remote: Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
        remote: 
        To /srv/git/xxxx/yyyyyy.xxxx.git
         ! [remote rejected] master -> master (pre-receive hook declined)
        error: failed to push some refs to '/srv/git/xxxx/yyyyyy.xxxx.git'

lunny · 2019-08-06T08:11:42Z

I think maybe environments break it.

env := append(os.Environ(),
		private.GitAlternativeObjectDirectories+"="+gitAlternativeObjectDirectories,
		private.GitObjectDirectory+"="+gitObjectDirectory,
		private.GitQuarantinePath+"="+gitObjectDirectory,
	)

We can remove these. But I can't reproduce that locally.

lafriks · 2019-08-06T08:31:11Z

@lunny hooks will not work without these environment variables. It should be something to do with specific configuration (gitea workdir, homedir or app.ini paths)

jdehaan · 2019-08-06T08:35:59Z

part of app.ini:

[repository]
ROOT = /srv/git

[server]
SSH_DOMAIN       = xxxxx.de
SSH_PORT         = 2222
SSH_LISTEN_PORT  = 2222
START_SSH_SERVER = true
DISABLE_SSH      = false
DOMAIN           = xxxxx.de
HTTP_PORT        = 3000
ROOT_URL         = https://xxxxx.de/git/
LFS_START_SERVER = false
OFFLINE_MODE     = true

[log]
MODE      = file
LEVEL     = Info
ROOT_PATH = /root/go/src/code.gitea.io/gitea/log

lunny · 2019-08-06T08:38:11Z

repository.ROOT is where you store your all git repositories. could you confirm you have permissions on that directory?

jdehaan · 2019-08-06T08:38:22Z

I could find some path information here:

lafriks · 2019-08-06T08:39:05Z

what is workdir in service file specified and what is home directory for git user?

jdehaan · 2019-08-06T08:41:39Z

The service file is:

[Unit]
Description=Gitea (Git with a cup of tea)
After=syslog.target
After=network.target
After=mysqld.service

[Service]
Type=simple
User=git
Group=git
WorkingDirectory=/root/go/src/code.gitea.io/gitea
ExecStart=/root/go/src/code.gitea.io/gitea/gitea web
Restart=always
Environment=USER=root HOME=/root

[Install]
WantedBy=multi-user.target

jdehaan · 2019-08-06T08:43:12Z

/home/git is the home of user git

BTW the permissions inside /srv/git/ look ok, everything owned by git.git

lafriks · 2019-08-06T08:55:23Z

This does not seem correct in service file: Environment=USER=root HOME=/root

lunny · 2019-08-06T09:00:14Z

@jdehaan Which git version are you using?

jdehaan · 2019-08-06T09:04:11Z

2.7.4 on that server

jdehaan · 2019-08-06T09:05:30Z

This does not seem correct in service file: Environment=USER=root HOME=/root

Gosh... you're right I must have done this wrong for years.... Strangely the same config works for Gitea 1.8.3 without trouble, I did not touch that part only upgraded Gitea.

lunny · 2019-08-06T09:05:39Z

@lafriks this git command will only detect force push, it's unrelated with hooks. The http request is from a prereceived hook I know.

lafriks · 2019-08-06T09:09:18Z

@lunny yes but it was rewritten to remove gitea serv command

jdehaan · 2019-08-06T09:20:16Z

As the code modified something regarding the environment handling, I could give the upgrade a second try with the right parameters and report, before wasting your time. Thanks for the hints.

jdehaan · 2019-08-06T10:37:29Z

Changed the service file part:

Environment=USER=git HOME=/home/git

Same error, currently tested with Gitea Version: 1.9.0+10-gaea49d0

Remote: Gitea: Internal Server Error        
Remote: Fail to detect force push: exit status 128 - fatal: Not a git repository (or any parent up to mount point /srv/git)        
Remote: Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

jdehaan · 2019-08-06T10:43:12Z

From the log:

2019/08/06 10:03:36 ...ters/private/hook.go:74:HookPreReceive() [E] Unable to detect force push between: 30f13dbcd3a7ef5cb4838033f2a3ec13784f040f and 4afd02879bfb2ffd52ec3220d09394c322c65d95 in 824657420112:XXXX/YYYYYY.Xxxx Error: exit status 128 - fatal: Not a git repository (or any parent up to mount point /srv/git)
	Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

jdehaan · 2019-08-06T11:09:12Z

While it is probably not a clean fix..... I am aware of that I tried to patch the code in the following way:

--- a/routers/private/hook.go
+++ b/routers/private/hook.go
@@ -8,13 +8,13 @@ package private
 import (
        "fmt"
        "net/http"
-       "os"
+       //"os"
        "strings"
 
        "code.gitea.io/gitea/models"
        "code.gitea.io/gitea/modules/git"
        "code.gitea.io/gitea/modules/log"
-       "code.gitea.io/gitea/modules/private"
+       //"code.gitea.io/gitea/modules/private"
        "code.gitea.io/gitea/modules/repofiles"
        "code.gitea.io/gitea/modules/util"
 
@@ -29,8 +29,8 @@ func HookPreReceive(ctx *macaron.Context) {
        newCommitID := ctx.QueryTrim("new")
        refFullName := ctx.QueryTrim("ref")
        userID := ctx.QueryInt64("userID")
-       gitObjectDirectory := ctx.QueryTrim("gitObjectDirectory")
-       gitAlternativeObjectDirectories := ctx.QueryTrim("gitAlternativeObjectDirectories")
+       //gitObjectDirectory := ctx.QueryTrim("gitObjectDirectory")
+       //gitAlternativeObjectDirectories := ctx.QueryTrim("gitAlternativeObjectDirectories")
        prID := ctx.QueryInt64("prID")
 
        branchName := strings.TrimPrefix(refFullName, git.BranchPrefix)
@@ -63,13 +63,14 @@ func HookPreReceive(ctx *macaron.Context) {
 
                // detect force push
                if git.EmptySHA != oldCommitID {
-                       env := append(os.Environ(),
-                               private.GitAlternativeObjectDirectories+"="+gitAlternativeObjectDirectories,
-                               private.GitObjectDirectory+"="+gitObjectDirectory,
-                               private.GitQuarantinePath+"="+gitObjectDirectory,
-                       )
-
-                       output, err := git.NewCommand("rev-list", "--max-count=1", oldCommitID, "^"+newCommitID).RunInDirWithEnv(repo.RepoPath(), env)
+                       //env := append(os.Environ(),
+                       //      private.GitAlternativeObjectDirectories+"="+gitAlternativeObjectDirectories,
+                       //      private.GitObjectDirectory+"="+gitObjectDirectory,
+                       //      private.GitQuarantinePath+"="+gitObjectDirectory,
+                       //)
+
+                       //output, err := git.NewCommand("rev-list", "--max-count=1", oldCommitID, "^"+newCommitID).RunInDirWithEnv(repo.RepoPath(), env)
+                       output, err := git.NewCommand("rev-list", "--max-count=1", oldCommitID, "^"+newCommitID).RunInDir(repo.RepoPath())
                        if err != nil {
                                log.Error("Unable to detect force push between: %s and %s in %-v Error: %v", oldCommitID, newCommitID, repo, err)
                                ctx.JSON(http.StatusInternalServerError, map[string]interface{}{

That solves the issue. Maybe that helps you to see where & why things are going wrong in our use case/setup...

lunny · 2019-08-06T13:56:03Z

The workaround is similiar what I guess in #7754 (comment). @zeripath do you remember why we add those environments?

zeripath · 2019-08-06T16:53:22Z

OK, this is very confusing. This works on testing - perhaps it's a weirdness of the early git version?

The internal call to the hook actually has the values passed in the GET request - this was deliberate to act as logging but you've not given us these. Do you have the ROUTER log turned off?

It could be that your version of git is not setting these and therefore by us setting them that causes the problem, or it could be that git deletes them and keeps a read hold on the directory meaning that the gitea process doesn't have access to them. Were they being set at all? What was their value? Did it require removing all of these environment variables to make things work?

The reasoning behind passing these in to the git process is that the head commit and the base may be in these alternative directories - I suspect that later versions of git are going to be more keen to use these alternative directories to quarantine against pollution of repositories.

jdehaan · 2019-08-06T20:41:03Z

I did not deliberately turn down some logging. I will look into the available log settings. At the moment we are just happy with the poor but working workaround that restores part of the 1.8 branch logic.

zeripath · 2019-08-06T21:33:04Z

OK I've been able to replicate this with git version 1.7.2 (after fixing my problem with running test-sqlite locally)

It appears the solution is to test whether those values are nil and not set them if they're empty.

However, I then hit a number of other problems with the new merge commands.

So I've pushed up another PR - @jdehaan would you be able to build from this and try that?

zeripath · 2019-08-06T22:04:16Z

Ok so there's a problem with that pr which is apparent on the full tests. I think I know how to fix - and it will fix a potential problem at the same time doing so.

jdehaan · 2019-08-07T04:36:07Z

Yes I can build from that pr and I need the help of @sharpSteff to make the series of tests. I hope he is available for that today a couple of minutes..I will report. Thanks a bunch for your efforts until now!

sharpSteff · 2019-08-07T05:49:13Z

@jdehaan yes I'm available for some tests

shollander · 2019-08-27T19:29:55Z

I don't think this is fixed. I tried using Gitea version 1.9.2 and receive an error when attempting to merge a pull request:

routers/repo/pull.go:629:MergePullRequest() [E] Merge: getDiffTree: git diff-tree [/gitea/data/tmp/local-repo/merge-123456789.git base:master head:head_repo/feature142]: fatal: ambiguous argument 'head_repo/feature142' unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git [...] -- [...]'

I am using Gitea on CentOS 7.6 with git version 1.8.3.1 (which is the standard version of Git from the yum repos).

guillep2k · 2019-10-08T15:01:58Z

I don't think this is fixed. I tried using Gitea version 1.9.2 and receive an error when attempting to merge a pull request:

routers/repo/pull.go:629:MergePullRequest() [E] Merge: getDiffTree: git diff-tree [/gitea/data/tmp/local-repo/merge-123456789.git base:master head:head_repo/feature142]: fatal: ambiguous argument 'head_repo/feature142' unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git [...] -- [...]'

I am using Gitea on CentOS 7.6 with git version 1.8.3.1 (which is the standard version of Git from the yum repos).

@shollander I think your problem is unrelated. The original poster had a problem with git finding the repository at all; your log shows that git is having trouble finding the branch, which means that the repository was found alright. I believe that there are some improvements related to your problem in 1.9.3. Try upgrading to it.

shollander · 2019-10-25T04:30:57Z

@guillep2k I think you are right. I just tried version Gitea version 1.9.4 and the issue I was having appears to have been fixed. Thanks!

sharpSteff mentioned this issue Aug 5, 2019

PR files return 500 #7735

Closed

lafriks mentioned this issue Aug 6, 2019

Push to the protected branch #7770

Closed

7 tasks

lafriks mentioned this issue Aug 6, 2019

Merge Pull Request 500 error #7636

Closed

7 tasks

zeripath mentioned this issue Aug 6, 2019

Restore functionality for early gits #7775

Merged

lafriks added the type/bug label Aug 6, 2019

lafriks added this to the 1.9.1 milestone Aug 6, 2019

This was referenced Aug 14, 2019

Fix hook problem in #7754 #7854

Merged

Fix #7754 - backport to 1.9 #7856

Merged

lafriks closed this as completed in #7854 Aug 14, 2019

4oo4 mentioned this issue Aug 16, 2019

! [remote rejected] master -> master (pre-receive hook declined) #7284

Closed

7 tasks

go-gitea locked and limited conversation to collaborators Nov 24, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Whitelist-Push and MergePullrequest does not work when Branch-Protection is enabled #7754

Whitelist-Push and MergePullrequest does not work when Branch-Protection is enabled #7754

sharpSteff commented Aug 5, 2019 •

edited

Loading

lunny commented Aug 5, 2019

zeripath commented Aug 6, 2019

sharpSteff commented Aug 6, 2019

zeripath commented Aug 6, 2019 •

edited

Loading

jdehaan commented Aug 6, 2019

jdehaan commented Aug 6, 2019

lunny commented Aug 6, 2019

lafriks commented Aug 6, 2019

jdehaan commented Aug 6, 2019

lunny commented Aug 6, 2019

jdehaan commented Aug 6, 2019

lafriks commented Aug 6, 2019

jdehaan commented Aug 6, 2019

jdehaan commented Aug 6, 2019 •

edited

Loading

lafriks commented Aug 6, 2019

lunny commented Aug 6, 2019

jdehaan commented Aug 6, 2019

jdehaan commented Aug 6, 2019

lunny commented Aug 6, 2019

lafriks commented Aug 6, 2019

jdehaan commented Aug 6, 2019

jdehaan commented Aug 6, 2019

jdehaan commented Aug 6, 2019

jdehaan commented Aug 6, 2019 •

edited

Loading

lunny commented Aug 6, 2019

zeripath commented Aug 6, 2019

jdehaan commented Aug 6, 2019

zeripath commented Aug 6, 2019

zeripath commented Aug 6, 2019

jdehaan commented Aug 7, 2019 •

edited

Loading

sharpSteff commented Aug 7, 2019

shollander commented Aug 27, 2019

guillep2k commented Oct 8, 2019

shollander commented Oct 25, 2019

**Whitelist-Push** and **MergePullrequest** does not work when Branch-Protection is enabled #7754

**Whitelist-Push** and **MergePullrequest** does not work when Branch-Protection is enabled #7754

Comments

sharpSteff commented Aug 5, 2019 • edited Loading

Description

Screenshots

lunny commented Aug 5, 2019

zeripath commented Aug 6, 2019

sharpSteff commented Aug 6, 2019

zeripath commented Aug 6, 2019 • edited Loading

jdehaan commented Aug 6, 2019

jdehaan commented Aug 6, 2019

lunny commented Aug 6, 2019

lafriks commented Aug 6, 2019

jdehaan commented Aug 6, 2019

lunny commented Aug 6, 2019

jdehaan commented Aug 6, 2019

lafriks commented Aug 6, 2019

jdehaan commented Aug 6, 2019

jdehaan commented Aug 6, 2019 • edited Loading

lafriks commented Aug 6, 2019

lunny commented Aug 6, 2019

jdehaan commented Aug 6, 2019

jdehaan commented Aug 6, 2019

lunny commented Aug 6, 2019

lafriks commented Aug 6, 2019

jdehaan commented Aug 6, 2019

jdehaan commented Aug 6, 2019

jdehaan commented Aug 6, 2019

jdehaan commented Aug 6, 2019 • edited Loading

lunny commented Aug 6, 2019

zeripath commented Aug 6, 2019

jdehaan commented Aug 6, 2019

zeripath commented Aug 6, 2019

zeripath commented Aug 6, 2019

jdehaan commented Aug 7, 2019 • edited Loading

sharpSteff commented Aug 7, 2019

shollander commented Aug 27, 2019

guillep2k commented Oct 8, 2019

shollander commented Oct 25, 2019

Whitelist-Push and MergePullrequest does not work when Branch-Protection is enabled #7754

Whitelist-Push and MergePullrequest does not work when Branch-Protection is enabled #7754

sharpSteff commented Aug 5, 2019 •

edited

Loading

zeripath commented Aug 6, 2019 •

edited

Loading

jdehaan commented Aug 6, 2019 •

edited

Loading

jdehaan commented Aug 6, 2019 •

edited

Loading

jdehaan commented Aug 7, 2019 •

edited

Loading