Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ready] Swift Programming Language Completions #487

Closed
wants to merge 17 commits into from
Closed

[Ready] Swift Programming Language Completions #487

wants to merge 17 commits into from

Commits on May 19, 2017

  1. Swift Semantic Completion

    This patch adds support for Swift semantic completion.
    
    It adds a new subclass of `Completer` which is a thin client for
    SwiftySwiftVim, the semantic backend.
    
    SwiftySwiftVim is an HTTP interface to semantic facilities in the Swift
    Compiler, tailored to the needs of a text editor.
    
    Usage:
    
    After you've met all of the system dependencies, then simply run
    build.py with --swift-completer. This will checkout and build
    SwiftySwiftVim from source in `third_party/swiftyswiftvim`.
    
    Build Summary:
    
    To build SwiftySwiftVim from source, you need a few `system` dependencies:
    - OSX ( for Xcode )
    - Xcode ( for prebuilt swift binary )
    - Boost
    
    See https://github.com/jerrymarino/swiftyswiftvim for detailed information.
    jerrymarino committed May 19, 2017
    Configuration menu
    Copy the full SHA
    a4666b4 View commit details
    Browse the repository at this point in the history
  2. YCMD Integration Tests and More

    This patch cleans up a few things in preperation for upstreaming. I
    added tests for debug info and completions, so we can make sure it is
    working end to end.
    
    I also addressed other comments from the code review including:
    
    - Code formatting and Flake 8 issues
    - Valloric's other comments
    
    Comments around backend dependencies should be resolved here:
    jerrymarino/swiftyswiftvim@abde37e
    Now OSX and Xcode options are used by default rather than hardcoded.
    This should make it easier for people to use this with other platforms
    in the future.
    
    Additionally, I improved query classification. It was previously
    returning cached results in circumstances that didn't make any sense.
    
    Mostly, I think this is in a good state. There are still a few
    remaining tasks that need to be addressed:
    - Add and test "flag" support
    - Write more documentation
    - Sort out the CI situation for OSX only
    jerrymarino committed May 19, 2017
    Configuration menu
    Copy the full SHA
    7ed8ac4 View commit details
    Browse the repository at this point in the history
  3. Fix CI issues and More

    - Use 127.0.0.1 for an IP since that is conventional in YCMD.
    
    - Use minimum Xcode Version for SourceKit. In my testing, the travis
      images xcode8, xcode8.1, would not work.
    
    - Disable SwiftCompleter when swift is not present
    jerrymarino committed May 19, 2017
    Configuration menu
    Copy the full SHA
    139c1f3 View commit details
    Browse the repository at this point in the history
  4. Swift Compilation Database Implementation

    Implement the clang compilation database standard for swift files.
    
    The format is specified for Clang:
    https://clang.llvm.org/docs/JSONCompilationDatabase.html
    
    I think this is a good file format to use for swift.
    
    The implementation includes a "CompilationDatabase" to read in the files
    and a high level API to get flags, "Flags". It uses logic based on the
    `cpp` version of `Flags`.
    
    I originally hoped to reusing the `cpp` version of this but there 2
    show stoppers:
    
    - Flags.py logic is highly specialized to clang based languages. I don't
    like the idea of a complicated implementation that works for both swift
    and clang.
    
    - Swift users should not have to download the massive payload that
    clang is.
    
    The class simply takes a compilation invocation, and converts it to
    completion flags.
    
    Additionally, I am caching database loads based on the file timestamp.
    Flags are also cached, and I think the invalidation will need more work
    in the future. Based on my own testing, the logic seems sufficient for a
    first pass.
    
    There is tangentially related work around writing programs to generate
    these files for standard build systems:
    
    - SwiftPM / SwiftC. It'd be nice to geneate these comp dbs from
     the output of these programs. Perhaps a proposal can be made to get
     SwiftPM generating these.
    - Bazel: it doesn't support comp dbs for any language by default, but
     it is easy to add.
    - Xcode: Perhaps we can modify Xcpretty to and similar programs to get
     this working with Xcode. The standard method for comp dbs in
     `xcodebuild` is to parse the output and dump a DB based on that. A few
     OSS solutions exist for this but are limited to `clang` for now. I
     opened an issue here: xcpretty/xcpretty#281
    
    I included some assertions that I wrote while developing this, and I'll
    need to get it into the YCMD test suite before merging.
    jerrymarino committed May 19, 2017
    Configuration menu
    Copy the full SHA
    67c04da View commit details
    Browse the repository at this point in the history
  5. Compilation DB asserts in tests and more tests

    This patch moves the compilation DB unit tests into the testing system.
    I added a fixture directory for this including:
    - a compilation database template to write absolute paths into
    the database during tests.
    - ViewController.swift which depends on AppDelegate.swift
    
    I also added integration tests for the comp db and improved completion
    tests in general. These test somewhat into SourceKit land, but it is
    good end to end test to make sure we aren't messing up.
    
    - Makes sure that reasonable completions are returned.
    
    - Recursive include test to make sure the flags are compatible: loading
    transitive headers in the SDK. The other assertion makes sure that we
    can correctly load dependencies via the `primary-file` flag.
    
    iOS is a good baseline since it's a massive swift userbase. The 2 OSX
    specific tests are wrapped to only execute on Darwin.
    jerrymarino committed May 19, 2017
    Configuration menu
    Copy the full SHA
    f01c074 View commit details
    Browse the repository at this point in the history
  6. Fix get completions integration test on CI

    The integration test only works on the new Xcode and it will not work if
    the compile commands includes flags with missing build artifacts. For
    example, if the comp db includes flags that reference output from a
    build that is no longer around ( or on a different machine in this case)
    jerrymarino committed May 19, 2017
    Configuration menu
    Copy the full SHA
    17e044e View commit details
    Browse the repository at this point in the history
  7. Address review comments

    jerrymarino committed May 19, 2017
    Configuration menu
    Copy the full SHA
    e6ad96f View commit details
    Browse the repository at this point in the history

Commits on May 22, 2017

  1. More improvements and @puremourning's comments

    This patch removes completions for operators and addresses review
    feedback. We were originally returning completions for operator by
    hacking the completion offset. This may have been intentional at one
    point ( or an off by 1); after using this a lot and @puremournings
    feedback I think its best to remove it.
    
    This enables expressing the ' ' trigger within YCMD and cleans up a lot.
    
    Additionally:
    
    - Add an example in the test code of docs showing in the UI
    - Improve comprehension of uncommented completions
    - Add lock on startup and shutdown
    - Add Flag logging
    - Remove completion type customization
    - Address manual formatting consistency
    - Remove snippets by default. This needs extra support code in vimscript
      and is probably out of scope for a V1 to integrate this end to end or
      opensource my plugin. With the types rendering along with function
      signatures in the menu and preview window it's seems fine for now.
    
    I am quite happy with it now, overall, it feels way nicer!
    jerrymarino committed May 22, 2017
    Configuration menu
    Copy the full SHA
    4cf3ef5 View commit details
    Browse the repository at this point in the history
  2. Use invocation value

    jerrymarino committed May 22, 2017
    Configuration menu
    Copy the full SHA
    d5bf539 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2017

  1. Configuration menu
    Copy the full SHA
    edf3116 View commit details
    Browse the repository at this point in the history

Commits on May 25, 2017

  1. Configuration menu
    Copy the full SHA
    ef3d42e View commit details
    Browse the repository at this point in the history
  2. Completion UX improvements

    After some manual testing and studying the code, I've came up with a few
    improvements to the feel of completion.
    
    - Improve completion triggering. ' ' is poor trigger. I used a regex to
    match on word characters. This works way better and is more aligned with
    Xcode triggering mechanisms.
    
    - Disable should use inner override. This isn't really necessary with
    all of the changes that have been made in the last few commits.
    
    - Improve menu text. Based on recommendation, having the 'name' and
    'description' in the menu was redundant. Now, it only shows the
    'description', which for methods, is the signature and types.
    jerrymarino committed May 25, 2017
    Configuration menu
    Copy the full SHA
    ef41fc7 View commit details
    Browse the repository at this point in the history
  3. Space

    jerrymarino authored May 25, 2017
    Configuration menu
    Copy the full SHA
    bd8f799 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2017

  1. [Stability] Wait on Swifty Swift Vim boot

    Wait for initial Swifty Swift Vim Boot. Tasks in the backend's startup
    process ( like dynamic linking and setting up the HTTP stack ) make it
    impossible for the backend to respond to requests immediately after
    launching the process. If the process isn't ready, YCMD will fail. It
    simply waits until the process writes to stdout, which in practice is
    the loading message.
    
    Profiling:
    Runs were made on a 2014 MBA ( 1.4 GHz Intel Core i5 ). I observed
    booting to be reasonably quick. Right after starting the machine, it
    took less than 100ms. After starting the service once it was
    dramatically reduced. I used a 5 second timeout because I don't see a
    good to make it much smaller and I don't have access to lower perf
    hardware for developing this.
    
    Stability:
    I assume this is why the CI was failing here and there. I also tested
    adding a sleep before the tests yielded stability on the CI after
    several runs ( see jerrymarino/ycmd Travis build history of
    jmarino_health_check_completion_tests ). @puremourning also reported he
    was seeing timeouts. I was also able to consistently see timeouts in vim
    and fail the test suite immediately after starting the test machine.
    
    Reproduction Steps:
    - run the test suite immediately after system start
    - open Vim immediately after system start
    
    Additionally:
    Increase shutdown test timeout. Now that we wait for the swift server to
    start, it slows down this test tried it with 5 seconds and then 10.
    jerrymarino committed May 28, 2017
    Configuration menu
    Copy the full SHA
    382b92f View commit details
    Browse the repository at this point in the history
  2. Merge pull request #1 from jerrymarino/jmarino_wait_for_sever_boot

    Wait for the swift server to boot
    jerrymarino authored May 28, 2017
    Configuration menu
    Copy the full SHA
    d0923f3 View commit details
    Browse the repository at this point in the history
  3. Update Swifty Swift Vim

    Update the semantic server to HEAD since it now treats warnings as
    errors.
    
    ( jerrymarino/swiftyswiftvim#5 )
    jerrymarino committed May 28, 2017
    Configuration menu
    Copy the full SHA
    893a860 View commit details
    Browse the repository at this point in the history

Commits on May 29, 2017

  1. Different message for thrown response parse

    Address @bstaletic 's comment around having a different message for when
    response parsing throws.
    jerrymarino committed May 29, 2017
    Configuration menu
    Copy the full SHA
    fb9a20a View commit details
    Browse the repository at this point in the history