Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iOS Test target #484

Merged
merged 20 commits into from
Sep 2, 2015
Merged

iOS Test target #484

merged 20 commits into from
Sep 2, 2015

Conversation

phatblat
Copy link
Member

iOS Tests #444

screen shot 2015-07-10 at 1 25 16 am

To help prevent things like #478 & #481.

Using SSZipArchive isn't ideal since it isn't a framework so the Xcode integration is ugly. But at least we can now run tests on iOS. Boy, are they slow. 💤

@pietbrauer
Copy link
Member

👍 Travis recently introduced longer build times on Mac machines (120 minutes). Maybe we can get public CI working again then? What do you think @joshaber?

@joshaber
Copy link
Member

Sure, we can try Travis again if someone wants to set it up 👍

@pietbrauer
Copy link
Member

✋ I volunteer, progress tracked in #485

@pietbrauer
Copy link
Member

@phatblat if you resolve the merge conflicts and push again Travis will be started 😉

@phatblat phatblat mentioned this pull request Jul 19, 2015
phatblat and others added 6 commits July 22, 2015 19:46
Specifying the destionation works around the following issue with xctool. Can also hard-code the architecture, but this seems more future-proof.

```
Failed to query the list of test cases in the test bundle: 2015-08-24 10:57:11.740 sim[50821:2746840] /Applications/Xcode6.4.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/sim: No simulator devices appear to be running.  Setting data directories to /var/empty.
2015-08-24 10:57:11.741 sim[50821:2746840] DYLD_INSERT_LIBRARIES contains possible bad values.  Caller beware: /usr/local/Cellar/xctool/0.2.4/libexec/lib/otest-query-lib-ios.dylib
dlopen(/Volumes/DerivedData/ObjectiveGitFramework-drtqacfhlqybdcdpqayztrrwneun/Build/Products/Debug-iphonesimulator/ObjectiveGit-iOSTests.xctest/ObjectiveGit-iOSTests, 1): Library not loaded: @rpath/ObjectiveGit.framework/ObjectiveGit
  Referenced from: /Volumes/DerivedData/ObjectiveGitFramework-drtqacfhlqybdcdpqayztrrwneun/Build/Products/Debug-iphonesimulator/ObjectiveGit-iOSTests.xctest/ObjectiveGit-iOSTests
  Reason: no suitable image found.  Did find:
	/Volumes/DerivedData/ObjectiveGitFramework-drtqacfhlqybdcdpqayztrrwneun/Build/Products/Debug-iphonesimulator/ObjectiveGit.framework/ObjectiveGit: mach-o, but wrong architecture
```

Indeed, the "wrong architecture" message is correct as the framework was built for a different architecture than the test binary:

```
lipo -info ObjectiveGit.framework/ObjectiveGit ObjectiveGit-iOSTests.xctest/ObjectiveGit-iOSTests                                                                                                                                        2015-08-24 11:03
Non-fat file: ObjectiveGit.framework/ObjectiveGit is architecture: i386
Non-fat file: ObjectiveGit-iOSTests.xctest/ObjectiveGit-iOSTests is architecture: x86_64
```
@phatblat
Copy link
Member Author

Another PR build is green! Ready for review.

@joshaber joshaber self-assigned this Aug 31, 2015

BOOL success = [SSZipArchive unzipFileAtPath:zipPath toDestination:destinationPath overwrite:YES password:nil error:error];

if (!success) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: it looks like there's a mix of tabs/spaces here.

@joshaber
Copy link
Member

Really fantastic work! Just a couple nits.

@phatblat
Copy link
Member Author

phatblat commented Sep 2, 2015

Using SSZipArchive is almost an order of magnitude slower than shelling out to unzip:

  • MacTests (unzip): 1:20
  • iOSTests (SSZipArchive): 10:49
  • MacTests (SSZipArchive): 10:35

I'm inclined to leave the Mac tests using unzip. I'd like to see the iOS tests run faster; perhaps once that's solved we could switch the Mac tests to use the same unzip mechanism.

@joshaber
Copy link
Member

joshaber commented Sep 2, 2015

Wow, that's nuts! Thanks for doing the legwork on that.

🤘

joshaber added a commit that referenced this pull request Sep 2, 2015
@joshaber joshaber merged commit 0dc6c60 into libgit2:master Sep 2, 2015
@phatblat phatblat deleted the ben/ios-test-target branch September 2, 2015 14:45
tombooth added a commit to tombooth/objective-git that referenced this pull request Feb 16, 2016
In libgit2#484 more flags were
added into the variable `sdkflag`, which then caused the following
execution of run_xctool to always fail. This means that iOS tests have
not been running as part of the TravisCI run, they have only been built.
You can see this if you look at the final run for the PR:
https://travis-ci.org/libgit2/objective-git/builds/78334551#L192

Bash does some great things when trying to interpret variables in
commands. Initially the `$sdkflag` didn't have any quotes around it as
can be seen in commit 15f906c. When the
second flag was added this would have caused the command to fail because
of the way Bash would interpret the whitespace. Without the surrounding
quotes the command would have been executed as follows:

```
xctool -workspace ObjectiveGitFramework.xcworkspace RUN_CLANG_STATIC_ANALYZER=NO
-sdk iphonesimulator -destination '"platform=iOS' Simulator,name=iPhone
'5"' -scheme "ObjectiveGit iOS" test ONLY_ACTIVE_ARCH=NO CODE_SIGN_IDENTITY=
CODE_SIGNING_REQUIRED=NO
```

You can see in the above that the white space was split in the middle of
the `-destination` parameter and it causes `xctool` to spout out an
error. When the surrounding quotes were added it caused xctool to run
but `$sdkflag` would be interpretted as a single argument, as it is
executed as follows:

```
xctool -workspace ObjectiveGitFramework.xcworkspace RUN_CLANG_STATIC_ANALYZER=NO
'-sdk iphonesimulator -destination "platform=iOS Simulator,name=iPhone 5"'
-scheme "ObjectiveGit iOS" test ONLY_ACTIVE_ARCH=NO CODE_SIGN_IDENTITY=
CODE_SIGNING_REQUIRED=NO
```

Notice in the above that the entire argument is still surrounded by
single quotes.

This can be solved by switching `$sdkflag` for an array of strings, this
can then be correctly expanded in the command getting around the issues
highlighted above.

You can read more about these issues with Bash in the following links:

  - http://mywiki.wooledge.org/BashFAQ/050
  - http://mywiki.wooledge.org/BashFAQ/073
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants