[WIP] Linux Support

[jpsim]

This is a ⚠️ SUPER EARLY ⚠️ proof of concept, but I have the very basics of SourceKitten building with SwiftPM and linking against libsourcekitdInProc.so on Linux, building with the head of Swift.

Now this is a big mess at the moment because of the simultaneous moves to Swift 3, Linux and SwiftPM changes among other things, but I wanted to start the conversation now.

This branch will likely be frequently force-pushed, be warned.

Here are the (long and tedious) steps to get this running on a fresh x64 Ubuntu 16.04 machine:

$ apt-get update
$ apt-get install git cmake ninja-build clang python uuid-dev libicu-dev icu-devtools libbsd-dev libedit-dev libxml2-dev libsqlite3-dev swig libpython-dev libncurses5-dev pkg-config libblocksruntime-dev autoconf libkqueue-dev libtool
$ git clone https://github.com/apple/swift.git
$ cd swift
$ utils/update-checkout --clone
$ cd ../swift-corelibs-libdispatch
$ git checkout 245c9b3
$ git submodule update --init
$ cd -
$ utils/build-toolchain local.swift
$ utils/build-script --libdispatch
$ git remote add jp https://github.com/jpsim/swift.git
$ git fetch jp
$ git checkout sourcekit-linux
$ rm ../build/Ninja-DebugAssert/swift-linux-x86_64/CMakeCache.txt
$ utils/build-script --libdispatch
$ cd ..
$ git clone https://github.com/jpsim/SourceKitten.git
$ cd SourceKitten
$ git checkout jp-wip-linux
$ export PATH="/root/build/buildbot_linux/none-swift_package_sandbox_linux-x86_64/usr/bin:$PATH"
$ export LINUX_SOURCEKIT_LIB_PATH="/root/build/Ninja-DebugAssert/swift-linux-x86_64/lib"
$ swift build
$ .build/debug/sourcekitten syntax --text "import Foundation"

Which will then print the JSON-formatted syntax map for main.swift. That's the only thing that's exposed via the CLI interface at the moment. Tests don't run either.

---

At this point, I think the next steps are roughly:

1. Re-add as much of the removed or commented out code that doesn't depend on external packages. (July 18 edit: done)
2. Add Swift 3 & Linux support for upstream dependencies (Commandant (jp-05-31-snapshot), Result, YamlSwift (jp-wip-linux), SWXMLHash (jp-swift-3) & Clang_C). I've started this process for some of these, but they're all very hacked together and/or out of date at the moment. (June 30 edit: done)
3. Re-integrate dependencies. (June 30 edit: done)
4. Minimize differences between SourceKitten's master branch and the Linux one.
5. Legitimately enable SourceKit on Linux upstream in the apple/swift repo.
6. Add CI integration for Linux. (July 18 edit: done)

We may also want to stage this out, for example by first getting Swift 3 support on OS X merged in, then maybe moving to a SwiftPM-based workflow, dropping Carthage & generating the Xcode project via swift package generate-xcodeproj.

Given the massive changes in Swift 3, I'd probably want to drop Swift 2.x support even before Xcode 8 GM is out.

@norio-nomura are you interested in helping with any of this?

[norio-nomura]

Yes, I am.
I'll try to build this.

[jpsim]

@norio-nomura once you manage to get this building we should chat to see how we can both work on this minimizing overlap.

[norio-nomura]

I confirmed that swift-3.0-PREVIEW-6's Installer Package of sourcekit-builder worked on Travis-CI.
https://travis-ci.org/norio-nomura/SourceKitten/jobs/153744995

I've updated this PR for Xcode 8 Beta 6 as well as updated the jpsim/sourcekit Docker image to LLVM 17c4c6cf7a, Clang 4ca9e01a7c, Swift fc001e9a85 (most recent versions of Swift tip of master as of yesterday morning).

I think commits of swift-corelibs are more important on Linux.

[jpsim]

I think commits of swift-corelibs are more important on Linux.

Yes, you're right.

[jpsim]

I've updated this to use @norio-nomura's Swift 3.0 Docker image (thanks!) and point SWXMLHash and YamlSwift to their official canonical releases.

This is still using my fork of Commandant until Carthage/Commandant#69 or Carthage/Commandant#73 is merged and released.

Once Commandant supports Swift 3, I'd like to split this PR into 2 new ones:

1. Moving to Xcode 8 / Swift 3.
2. Adding Linux support.

That should make it easier to review.

[mominul]

Carthage/Commandant#69 got merged but Carthage/Commandant#73 has been closed.

[jpsim]

Closing in favor of #268.