Can't clone big repo on Windows

[thschmidt392]

• I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

$ git --version --build-options

git version 2.45.2.windows.1
cpu: x86_64
built from commit: 91d03cb2e4fbf6ad961ace739b8a646868cb154d
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver

Microsoft Windows [Version 10.0.22631.3737]
(c) Microsoft Corporation. Alle Rechte vorbehalten.

• What options did you set as part of the installation? Or did you choose the
  defaults?

C:\Program Files\Git\etc\install-options.txt

Editor Option: VIM
Custom Editor Path: 
Default Branch Option:  
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: WinSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Git Pull Behavior Option: Merge
Use Credential Manager: Enabled
Performance Tweaks FSCache: Enabled
Enable Symlinks: Disabled
Enable Pseudo Console Support: Disabled
Enable FSMonitor: Disabled

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

We use Gitea on our Servers. The issue persists only on windows (SSH and HTTPS).

The affected repo was migrated from an svn repo before.

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

TortoiseGIT & Git Bash

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

git clone <REPO URL>

• What did you expect to occur after running these commands?

Cloning into [...]...
remote: Enumerating objects: 42133, done.
remote: Counting objects: 100% (42133/42133), done.
remote: Compressing objects: 100% (18572/18572), done.
remote: Total 42133 (delta 20982), reused 41739 (delta 20589), pack-reused 0
Receiving objects: 100% (42133/42133), 2.55 GiB | 28.15 MiB/s, done.
Resolving deltas: 100% (20982/20982), done.
Updating files: 100% (15343/15343), done.

• What actually happened instead?

Cloning into C:\Users[...]...
POST git-upload-pack (185 bytes)
POST git-upload-pack (gzip 1062 to 574 bytes)
remote: Enumerating objects: 42133, done.
remote: Counting objects: 100% (42133/42133), done.
remote: Compressing objects: 100% (18572/18572), done.
fatal: pack has bad object at offset 2478985127: inflate returned -5
fatal: fetch-pack: invalid index-pack output


git did not exit cleanly (exit code 128) (62140 ms @ 25.06.2024 14:50:25)

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

The problem occurs only on one repository. This repository is big and has serveral tags & branches. On server side it has 3,8GB and locally (after a clone) 7,9GB.

I can't share the URL because it is an internal git repository.

[dscho]

[x] I was not able to find an open or closed issue matching what I'm seeing

Seems like a duplicate of #1848 to me...

[thschmidt392]

I saw this issue earlier and thought I did not have the same problem. But I looked into the Issue again and saw this comment. I searched the history for big files and there was indeed one big file (~4GB) in one branch. I deleted the file from all revisions with BFG Repo-Cleaner and now the issue is gone. I can now clone the repo on Windows Systems.

Here are the steps/commands that I used: (I got the Infos from here https://stackoverflow.com/a/42544963)

1. Search for big files in the git history:

git rev-list --objects --all |
  git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' |
  sed -n 's/^blob //p' |
  sort --numeric-sort --key=2 |
  cut -c 1-12,41- |
  $(command -v gnumfmt || echo numfmt) --field=2 --to=iec-i --suffix=B --padding=7 --round=nearest

2. Remove the specific file with BFG Repo-Cleaner

java -jar bfg-1.14.0.jar --delete-files <FILE_NAME> <LOCAL_REPO_PATH>

3. Cleaned up local git repo (this can take long time)

git reflog expire --expire=now --all && git gc --prune=now --aggressive

4. Upload the whole repo to the remote server.

git remote push --all
git remote push --tags

6. Clone successfully the repo on a Windows machine

Is this a bug because on Linux the clone works just fine with the ~4GB file?

[thschmidt392]

Okay I see there is an open pull request #2179 but there is no activity? I think this would fix the Problem

[dscho]

I see there is an open pull request #2179 but there is no activity?

Indeed, that PR's commit structure is in dear want of being optimized. One big hunking commit is too hard to review, to easy to lose confidence in.

[thschmidt392]

Okay thanks for your Information. Should I close this issue, since I have an workaround?

[dscho]

Should I close this issue, since I have an workaround?

Yes, let's close it, and leave all follow-up conversations (if any) in #2179.