Shallow clone for submodule doesn't work for some Git repo

[dlackty]

As CocoaPods 0.34.0 was released, I got some problem installing facebook-ios-sdk pod. After digging a while, I found it might be a git related issue.

To reproduce, first git clone facebook-ios-sdk repo, then run:

$ git submodule update --init --depth 1                                                                                                                 
Submodule 'Bolts-IOS' (git://github.com/BoltsFramework/Bolts-iOS.git) registered for path 'Bolts-IOS'
Submodule 'vendor/OCHamcrest' (git://github.com/hamcrest/OCHamcrest) registered for path 'vendor/OCHamcrest'
Submodule 'vendor/OCMock' (git://github.com/erikdoe/ocmock) registered for path 'vendor/OCMock'
Submodule 'vendor/OHHTTPStubs' (git://github.com/AliSoftware/OHHTTPStubs.git) registered for path 'vendor/OHHTTPStubs'
Submodule 'vendor/appledoc' (git://github.com/tomaz/appledoc.git) registered for path 'vendor/appledoc'
Submodule 'vendor/ios-snapshot-test-case' (git://github.com/facebook/ios-snapshot-test-case.git) registered for path 'vendor/ios-snapshot-test-case'
Submodule 'vendor/xctool' (git://github.com/facebook/xctool.git) registered for path 'vendor/xctool'
Cloning into 'Bolts-IOS'...
remote: Counting objects: 86, done.
remote: Compressing objects: 100% (70/70), done.
remote: Total 86 (delta 13), reused 45 (delta 6)
Receiving objects: 100% (86/86), 68.46 KiB | 0 bytes/s, done.
Resolving deltas: 100% (13/13), done.
Checking connectivity... done.
fatal: reference is not a tree: d679df70933da29463e44e15f4ce4f740d7b5dfd
...SKIP...
Unable to checkout 'd679df70933da29463e44e15f4ce4f740d7b5dfd' in submodule path 'Bolts-IOS'
Unable to checkout 'da09243ef136883fa8cea7ea2d42f48a9256fbdf' in submodule path 'vendor/OCHamcrest'
Unable to checkout '2191c124a93f535056e5cae05d55545a415fcde1' in submodule path 'vendor/OCMock'
Unable to checkout 'c87ded1ac299bc659a7e36bac43cac094bf7adb9' in submodule path 'vendor/OHHTTPStubs'
Unable to checkout 'd2e65f8c2faa066acfea2dbb51a5ec2d5364c7ec' in submodule path 'vendor/appledoc'
Unable to checkout 'a04ea585a5efb6cce6e5cd17f568eb1e6c472b2c' in submodule path 'vendor/ios-snapshot-test-case'
Unable to checkout 'f8720fe7e6af818054328c0325c7e9ce195f7c7c' in submodule path 'vendor/xctool'

My git --version is:

$ git --version
git version 1.9.3 (Apple Git-50)

[alloy]

@dlackty Do you also understand what is actually the issue going on? If so, please elaborate.

[kylef]

It's because the hashes in the git submodules wouldn't be the same with the shallow checkouts.

Steps to reproduce (please note, I didn't verify this with what's going on in CocoaPods. Only with manual commands):

(Please see comment near bottom explaining this example)

$ git clone https://github.com/CocoaPods/CocoaPods --depth 1
Cloning into 'CocoaPods'...
remote: Counting objects: 285, done.
remote: Compressing objects: 100% (258/258), done.
remote: Total 285 (delta 24), reused 117 (delta 8)
Receiving objects: 100% (285/285), 2.25 MiB | 1.36 MiB/s, done.
Resolving deltas: 100% (24/24), done.
Checking connectivity... done.
$ cd CocoaPods
$ git submodule update --init --depth 1
Submodule 'spec/cocoapods-integration-specs' (https://github.com/CocoaPods/cocoapods-integration-specs.git) registered for path 'spec/cocoapods-integration-specs'
Submodule 'spec/fixtures/integration/JSONKit' (https://github.com/johnezang/JSONKit.git) registered for path 'spec/fixtures/integration/JSONKit'
Submodule 'spec/fixtures/integration/Reachability' (https://github.com/tonymillion/Reachability.git) registered for path 'spec/fixtures/integration/Reachability'
Submodule 'spec/fixtures/spec-repos/master' (https://github.com/CocoaPods/Specs.git) registered for path 'spec/fixtures/spec-repos/master'
Cloning into 'spec/cocoapods-integration-specs'...
remote: Counting objects: 404, done.
remote: Compressing objects: 100% (211/211), done.
remote: Total 404 (delta 140), reused 371 (delta 131)
Receiving objects: 100% (404/404), 159.63 KiB | 0 bytes/s, done.
Resolving deltas: 100% (140/140), done.
Checking connectivity... done.
Submodule path 'spec/cocoapods-integration-specs': checked out '36d44d9a21a8b3bc420011720643d0e06c2e7771'
Cloning into 'spec/fixtures/integration/JSONKit'...
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 6 (delta 1), reused 2 (delta 0)
Unpacking objects: 100% (6/6), done.
Checking connectivity... done.
fatal: reference is not a tree: c2146ffeb10d92bfa1537d2033a3235825d1b261
Cloning into 'spec/fixtures/integration/Reachability'...
remote: Counting objects: 38, done.
remote: Compressing objects: 100% (29/29), done.
remote: Total 38 (delta 9), reused 34 (delta 8)
Unpacking objects: 100% (38/38), done.
Checking connectivity... done.
fatal: reference is not a tree: 4ec575e4b074dcc87c44018cce656672a979b34a
Cloning into 'spec/fixtures/spec-repos/master'...
remote: Counting objects: 49691, done.
remote: Compressing objects: 100% (33474/33474), done.
remote: Total 49691 (delta 9552), reused 48480 (delta 9229)
Receiving objects: 100% (49691/49691), 7.62 MiB | 1.29 MiB/s, done.
Resolving deltas: 100% (9552/9552), done.
Checking connectivity... done.
fatal: reference is not a tree: b0f4e85738f1f911e32e2914b39e9acdce7c7ed9
Unable to checkout 'c2146ffeb10d92bfa1537d2033a3235825d1b261' in submodule path 'spec/fixtures/integration/JSONKit'
Unable to checkout '4ec575e4b074dcc87c44018cce656672a979b34a' in submodule path 'spec/fixtures/integration/Reachability'
Unable to checkout 'b0f4e85738f1f911e32e2914b39e9acdce7c7ed9' in submodule path 'spec/fixtures/spec-repos/master'

It's expecting c2146ffeb10d92bfa1537d2033a3235825d1b261 for spec/fixtures/integration/JSONKit. But if you do a shallow clone of JSONKit, this hash won't be the same, because the hash will be different in a shallow clone because it doesn't have a parent and it will have all the changes flattened into the one commit, thus causing the hash to be different.

[dlackty]

Well. I don't know what's happening here. I tried to fork cocoapods-downloader and remove --depth 1 from git submodule init process, and then it works again.

I just verified it with 0.34.0:

Podfile:

source "https://github.com/CocoaPods/Specs.git"

target "Foo" do
  pod "Facebook-iOS-SDK"
end

$ pod install                                                                                                                                                          
Analyzing dependencies
Downloading dependencies
Installing Bolts (1.1.2)
Installing Facebook-iOS-SDK (3.18.0)
[!] /usr/bin/git submodule update --init --depth 1

Submodule 'Bolts-IOS' (git://github.com/BoltsFramework/Bolts-iOS.git) registered for path 'Bolts-IOS'

Submodule 'vendor/OCHamcrest' (git://github.com/hamcrest/OCHamcrest) registered for path 'vendor/OCHamcrest'

Submodule 'vendor/OCMock' (git://github.com/erikdoe/ocmock) registered for path 'vendor/OCMock'

Submodule 'vendor/OHHTTPStubs' (git://github.com/AliSoftware/OHHTTPStubs.git) registered for path 'vendor/OHHTTPStubs'

Submodule 'vendor/appledoc' (git://github.com/tomaz/appledoc.git) registered for path 'vendor/appledoc'

Submodule 'vendor/ios-snapshot-test-case' (git://github.com/facebook/ios-snapshot-test-case.git) registered for path 'vendor/ios-snapshot-test-case'

Submodule 'vendor/xctool' (git://github.com/facebook/xctool.git) registered for path 'vendor/xctool'
Cloning into 'Bolts-IOS'...

fatal: reference is not a tree: d679df70933da29463e44e15f4ce4f740d7b5dfd

Cloning into 'vendor/OCHamcrest'...

fatal: reference is not a tree: da09243ef136883fa8cea7ea2d42f48a9256fbdf

Cloning into 'vendor/OCMock'...

fatal: reference is not a tree: 2191c124a93f535056e5cae05d55545a415fcde1

Cloning into 'vendor/OHHTTPStubs'...

fatal: reference is not a tree: c87ded1ac299bc659a7e36bac43cac094bf7adb9

Cloning into 'vendor/appledoc'...

fatal: reference is not a tree: d2e65f8c2faa066acfea2dbb51a5ec2d5364c7ec

Cloning into 'vendor/ios-snapshot-test-case'...

fatal: reference is not a tree: a04ea585a5efb6cce6e5cd17f568eb1e6c472b2c

Cloning into 'vendor/xctool'...

fatal: reference is not a tree: f8720fe7e6af818054328c0325c7e9ce195f7c7c

Unable to checkout 'd679df70933da29463e44e15f4ce4f740d7b5dfd' in submodule path 'Bolts-IOS'

Unable to checkout 'da09243ef136883fa8cea7ea2d42f48a9256fbdf' in submodule path 'vendor/OCHamcrest'

Unable to checkout '2191c124a93f535056e5cae05d55545a415fcde1' in submodule path 'vendor/OCMock'

Unable to checkout 'c87ded1ac299bc659a7e36bac43cac094bf7adb9' in submodule path 'vendor/OHHTTPStubs'

Unable to checkout 'd2e65f8c2faa066acfea2dbb51a5ec2d5364c7ec' in submodule path 'vendor/appledoc'

Unable to checkout 'a04ea585a5efb6cce6e5cd17f568eb1e6c472b2c' in submodule path 'vendor/ios-snapshot-test-case'

Unable to checkout 'f8720fe7e6af818054328c0325c7e9ce195f7c7c' in submodule path 'vendor/xctool'