Handle projects with no documentation comments #94

juangamnik · 2014-11-23T17:49:58Z

Using jazzy v0.0.14 and ruby 2.1.5 on Yosemite installed via Homebrew I get a JSON parsing error and then a whole bunch of JSON code printed to console.

my-swift-prj$ jazzy
Running xcodebuild -dry-run
parsing Xyz.swift (1/n)
Xyz.swift is 40% documented
[...]
building site
/usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/json/common.rb:155:in `parse': 399: unexpected token at ',  (JSON::ParserError)
[...]
'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/json/common.rb:155:in `parse'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.14/lib/jazzy/sourcekitten.rb:162:in `parse'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.14/lib/jazzy/doc_builder.rb:94:in `build_docs_for_sourcekitten_output'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.14/lib/jazzy/doc_builder.rb:51:in `build'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.14/bin/jazzy:15:in `<top (required)>'
    from /usr/local/bin/jazzy:23:in `load'
    from /usr/local/bin/jazzy:23:in `<main>'

Ideas?

The text was updated successfully, but these errors were encountered:

segiddins · 2014-11-23T18:31:11Z

Can you please share an example project that demonstrates this issue (or the json at the very least)? Otherwise it will be very hard to track down what's happening

juangamnik · 2014-11-26T21:31:34Z

Oh sorry for the delay. I would like to share an example project with you, but I can just upload PNG, GIF, and JPG here. Hence here is a link to the project:
https://oc.kingsware.de/public.php?service=files&t=4fad92d0f2bd5d54adef92e6e4fd99ba

jpsim · 2014-11-26T21:38:14Z

@juangamnik what are you hoping to document in that project? There are no documentation comments. You can learn more about documenting Swift projects here: http://nshipster.com/swift-documentation/

In the meantime, we'll update jazzy to gracefully handle cases in which there are no documentation comments.

jpsim · 2014-11-26T21:40:17Z

sourcekitten output for this project is

[
{
  "key.offset" : 0,
  "key.doc.coverage" : 0,
  "key.doc.undocumented" : 6,
  "key.diagnostic_stage" : "source.diagnostic.stage.swift.parse",
  "key.length" : 2358,
  "key.substructure" : [

  ],
  "key.doc.documented" : 0
},
]

which is invalid JSON. sourcekitten is outputting this because GameScene.swift is completely undocumented.

segiddins · 2014-11-26T21:43:03Z

oh JSON...

juangamnik · 2014-11-27T10:19:20Z

@jpsim This is just a minimal project, which shows that jazzy exits with an error if a class is undocumented (didn't know that this causes the error, because another project that had no documentation worked quite well and just gave me empty documentation pages). In my real project I have documented classes, too, of course. But if I add a new class, it is not documented in the first place and the whole jazzy-execution fails and I get no docs at all. But as you said, this is not the expected behavior.

Thank you very much for your help.

orta · 2014-11-27T11:22:52Z

Yeah, I'd expect to see this case a bunch ;)

segiddins · 2014-11-28T09:37:21Z

The offending line is https://github.com/jpsim/sourcekitten/blob/master/sourcekitten/main.swift#L681

jpsim · 2014-11-28T18:38:54Z

@segiddins not quite, it's more that the second file should actually print out the same as the first file (they're both undocumented):

{
  "key.offset" : 0,
  "key.doc.coverage" : 0,
  "key.doc.undocumented" : 6,
  "key.diagnostic_stage" : "source.diagnostic.stage.swift.parse",
  "key.length" : 2358,
  "key.substructure" : [

  ],
  "key.doc.documented" : 0
}

jpsim · 2014-12-17T11:08:22Z

Thanks for reporting this issue, @juangamnik, a fix has now been pushed to master.

juangamnik · 2014-12-19T07:45:00Z

Thanks for fixing. Now I get an error during updated with sudo gem update, but jazzy seems to be updated to 0.0.16 now (does this release contain the fix?). But then during building the site I get another JSON parse error:

/usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/json/common.rb:155:in `parse': 399: unexpected token at '] (JSON::ParserError)
'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/json/common.rb:155:in `parse'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.16/lib/jazzy/sourcekitten.rb:142:in `parse'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.16/lib/jazzy/doc_builder.rb:100:in `build_docs_for_sourcekitten_output'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.16/lib/jazzy/doc_builder.rb:53:in `build'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.16/bin/jazzy:15:in `<top (required)>'
    from /usr/local/bin/jazzy:23:in `load'
    from /usr/local/bin/jazzy:23:in `<main>'

During update this error occurred:

Updating jazzy
Fetching: rouge-1.7.4.gem (100%)
Successfully installed rouge-1.7.4
Fetching: sqlite3-1.3.10.gem (100%)
Building native extensions.  This could take a while...
Successfully installed sqlite3-1.3.10
Fetching: jazzy-0.0.16.gem (100%)
Successfully installed jazzy-0.0.16
Parsing documentation for jazzy-0.0.16
Installing ri documentation for jazzy-0.0.16
Installing darkfish documentation for jazzy-0.0.16
Parsing documentation for rouge-1.7.4
Installing ri documentation for rouge-1.7.4
Installing darkfish documentation for rouge-1.7.4
/usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/class_module.rb:173:in `aref_prefix': missing aref_prefix for RDoc::SingleClass (NotImplementedError)
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/class_module.rb:181:in `aref'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/template/darkfish/class.rhtml:47:in `block in generate_class'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/erb.rb:850:in `eval'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/erb.rb:850:in `result'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:724:in `template_result'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:703:in `block in render_template'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:698:in `open'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:698:in `open'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:698:in `render_template'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:351:in `generate_class'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:371:in `block in generate_class_files'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:368:in `each'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:368:in `generate_class_files'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:245:in `generate'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/rubygems_hook.rb:137:in `block in document'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/rubygems_hook.rb:134:in `chdir'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/rubygems_hook.rb:134:in `document'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/rubygems_hook.rb:196:in `generate'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/rubygems_hook.rb:56:in `block in generation_hook'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/rubygems_hook.rb:55:in `each'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/rubygems_hook.rb:55:in `generation_hook'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/dependency_installer.rb:400:in `call'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/dependency_installer.rb:400:in `block (2 levels) in install'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/dependency_installer.rb:399:in `each'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/dependency_installer.rb:399:in `block in install'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/dependency_installer.rb:353:in `in_background'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/dependency_installer.rb:398:in `install'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/commands/update_command.rb:206:in `update_gem'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/commands/update_command.rb:220:in `block in update_gems'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/commands/update_command.rb:219:in `each'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/commands/update_command.rb:219:in `update_gems'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/commands/update_command.rb:98:in `execute'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/command.rb:305:in `invoke_with_build_args'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/command_manager.rb:167:in `process_args'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/command_manager.rb:137:in `run'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/gem_runner.rb:54:in `run'
    from /usr/local/bin/gem:21:in `<main>'

jpsim · 2014-12-19T07:51:29Z

We haven't pushed an update to rubygems since making the fix. I'll do that for you now.

jpsim · 2014-12-19T07:54:25Z

@juangamnik I just pushed 0.0.17 to rubygems. Let me know if that works for you.

juangamnik · 2014-12-19T10:55:59Z

I updated to 0.0.17 and get the following error:

building site
/usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/json/common.rb:155:in `parse': 399: unexpected token at ', (JSON::ParserError)
{
  "key.substructure" : [

  ],
  "key.offset" : 0,
  "key.diagnostic_stage" : "source.diagnostic.stage.swift.parse",
  "key.length" : 737
}

]
'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/json/common.rb:155:in `parse'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.17/lib/jazzy/sourcekitten.rb:174:in `parse'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.17/lib/jazzy/doc_builder.rb:100:in `build_docs_for_sourcekitten_output'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.17/lib/jazzy/doc_builder.rb:53:in `build'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.17/bin/jazzy:15:in `<top (required)>'
    from /usr/local/bin/jazzy:23:in `load'
    from /usr/local/bin/jazzy:23:in `<main>'

jpsim · 2014-12-19T11:16:42Z

hmmm, the project you linked to works for me. Have you updated the project since you linked it? What version of Xcode do you have installed under /Applications/Xcode.app? What does xcode-select -p print out? What version of OS X?

juangamnik · 2014-12-19T12:26:16Z

Sorry. For the linked project it works now. But it was just a nearly empty one. Another bigger iOS-project works, too. But this is a custom cocoa touch framework project. Perhaps that is already the problem?
I created it to separate model from view, in order to use the model in a playground. It works well (using a container workspace) and even the REPL works, when I debug my application in the simulator.

Xcode: Version 6.1.1 (6A2006)
Yosemite: 10.10.2 (14C81f)

 $ xcode-select -p
/Applications/Xcode.app/Contents/Developer/

jpsim · 2014-12-19T17:44:49Z

Sorry I'm not following. You're saying it works for you for both the linked (almost empty) project, it works for a big iOS project, but not for a different Cocoa Touch Framework?

Can you share that framework for me to test? Otherwise it's difficult for me to troubleshoot.

juangamnik · 2014-12-19T21:26:34Z

Big is a little bit overstated, but yes. I'll try to create a minimalistic project showing the issue, ok? Please, give me some time.

jpsim · 2014-12-19T21:47:45Z

Sounds great, thanks for your help!

juangamnik · 2014-12-20T11:50:43Z

Thank you for your help!

I get the above mentioned error, e.g., if I move away a file from a project. The project will not build then in Xcode, too, but jazzy "just" returns the JSON parse error.

I don't have a missing file in the productive project I use, but could not reproduce the issue in a fresh project without doing nasty things like moving files out of the project.

My own framework has a build problem, too. It builds all well if I select the project root during building (⇧⌘K, ⌘B). Otherwise (e.g. if some swift file is selected) I get a <unknown>:0: error: could not build Objective-C module 'MyCustomFrameworkName'. But the minimalistic cocoa touch framework, you may find here has this problem too but builds well with jazzy as soon as you delete the connection to SomeProgGS.swift in the project...

Perhaps this JSON parse error is connected to another Xcode-problem?

jpsim · 2014-12-20T19:02:56Z

It sounds like you can only make jazzy fail with a broken Xcode project, which isn't something we want to support anyway. Thanks for investigating!

juangamnik · 2014-12-20T20:56:53Z

On the one hand the project is not broken. It works all fine. Xcode (and the swift/objective-c compiler) just seems to have a problem with custom frameworks, that has to do with the currently selected file (which will be repaired in the future as I hope).

On the other hand, jazzy should not end up with a JSON-error in this case, shouldn't it? If Xcode has a problem, jazzy should report a meaningful error message saying, that the project a problem.

In summary, I am unsure what the error causes. Breaking the Xcode project shows the same symptom as my project, but this does not mean, that these problems have the same cause.

I sent you the project because it shows, that the build problem, that my project has seems not to be the cause of the problem, as the simple project shows the same symptom but jazzy runs all fine.

I don't think that this is the point to leave the investigation with a satisfied feeling, that Xcode is the problem (or the project itself).

juangamnik · 2014-12-21T17:41:13Z

 xcodebuild -dry-run

In the project root (of the real project) returns a lot of output and then:

** BUILD SUCCEEDED **

So I do not think that "broken project" is the root cause. I just cannot reproduce it with another project (without giving you all my sources). If you know an obfuscater for swift ...

jpsim · 2014-12-23T00:39:40Z

I don't think that this is the point to leave the investigation with a satisfied feeling, that Xcode is the problem (or the project itself).

Until I can reproduce the issue, troubleshoot, and fix it, I'm afraid there isn't much more to do. It sounds like you have a project that causes problems, but I don't have access to that 😢.

If you were looking for a way to contribute to jazzy, there's your shot!

You can re-open the issue (or create a new one) with a more appropriate issue name once you determine what's wrong.

juangamnik · 2014-12-23T09:38:58Z

I would do this, but jazzy 'needs' (don't get me wrong here please) to give me a more appropriate error message like which file is the problem, which class/function/operator/global variable, or even perhaps an text excerpt of the doc that causes the trouble. Right now I'm fishing in the dark because the project runs all fine and jazzy gives me a cryptic error message 😢. Anyways, I wish you nice free days!!! Thanks for this project. Project documentation is so important.

jpsim · 2014-12-23T19:59:37Z

I've just pushed #124, which will log which Swift files couldn't be parsed and (hopefully) avoid creating invalid JSON. Can you please try running that branch on your project, @juangamnik? Thanks!

juangamnik · 2014-12-24T09:48:18Z

I'm not that familiar with ruby. Currently I install jazzy via gem. Do you have an installation guide for installing it from scratch?

jpsim · 2014-12-25T01:59:11Z

@juangamnik Yup, right there in the README: https://github.com/realm/jazzy#development

juangamnik · 2014-12-25T23:16:18Z

@jpsim Thank you very much (merry christmas and everything)

I found the cause of the issue. It is an extension to the core swift struct Array that I annotated with documentation:

/// Extends array via a shuffled function which returns a copy
/// of the array in randomized order.
extension Array {
    func shuffled() -> [T] {
        var list = self
        for i in 0..<(list.count - 1) {
            let j = Int(arc4random_uniform(UInt32(list.count - i))) + i
            swap(&list[i], &list[j])
        }
        return list
    }
}

Please add as much as possible helpful error output to future versions (and the next release should certainly contain this addition). This small line (the first of the next snippet) helped me a lot:

Set.swift could not be parsed. Please open an issue at https://github.com/jpsim/sourcekitten/issues with the file contents.
building site
/usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/json/common.rb:155:in `parse': 399: unexpected token at '] (JSON::ParserError)
'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/json/common.rb:155:in `parse'
    from /Users/juangamnik/blabla/jazzy/lib/jazzy/sourcekitten.rb:175:in `parse'
    from /Users/juangamnik/blabla/jazzy/lib/jazzy/doc_builder.rb:100:in `build_docs_for_sourcekitten_output'
    from /Users/juangamnik/blabla/jazzy/lib/jazzy/doc_builder.rb:53:in `build'
    from /Users/juangamnik/blabla/jazzy/bin/jazzy:15:in `<main>'

I could point out the problem in seconds. But it was very hard to pinpoint without this information.

Thank you very much. Now my project documentation works like a charm.

jpsim changed the title ~~JSON parse error during building site~~ Handle projects with no documentation comments Nov 26, 2014

brennon mentioned this issue Dec 2, 2014

Files with comments being treated as without comments #102

Closed

jpsim closed this as completed in 633d15f Dec 17, 2014

jpsim mentioned this issue Dec 26, 2014

JSON::ParserError with the following code #126

Closed

pigeon-archive modified the milestone: The Past Nov 22, 2016

pigeon-archive added the feature label Nov 23, 2016

pigeon-archive assigned jpsim and segiddins Nov 23, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Handle projects with no documentation comments #94

Handle projects with no documentation comments #94

juangamnik commented Nov 23, 2014

segiddins commented Nov 23, 2014

juangamnik commented Nov 26, 2014

jpsim commented Nov 26, 2014

jpsim commented Nov 26, 2014

segiddins commented Nov 26, 2014

juangamnik commented Nov 27, 2014

orta commented Nov 27, 2014

segiddins commented Nov 28, 2014

jpsim commented Nov 28, 2014

jpsim commented Dec 17, 2014

juangamnik commented Dec 19, 2014

jpsim commented Dec 19, 2014

jpsim commented Dec 19, 2014

juangamnik commented Dec 19, 2014

jpsim commented Dec 19, 2014

juangamnik commented Dec 19, 2014

jpsim commented Dec 19, 2014

juangamnik commented Dec 19, 2014

jpsim commented Dec 19, 2014

juangamnik commented Dec 20, 2014

jpsim commented Dec 20, 2014

juangamnik commented Dec 20, 2014

juangamnik commented Dec 21, 2014

jpsim commented Dec 23, 2014

juangamnik commented Dec 23, 2014

jpsim commented Dec 23, 2014

juangamnik commented Dec 24, 2014

jpsim commented Dec 25, 2014

juangamnik commented Dec 25, 2014

Handle projects with no documentation comments #94

Handle projects with no documentation comments #94

Comments

juangamnik commented Nov 23, 2014

segiddins commented Nov 23, 2014

juangamnik commented Nov 26, 2014

jpsim commented Nov 26, 2014

jpsim commented Nov 26, 2014

segiddins commented Nov 26, 2014

juangamnik commented Nov 27, 2014

orta commented Nov 27, 2014

segiddins commented Nov 28, 2014

jpsim commented Nov 28, 2014

jpsim commented Dec 17, 2014

juangamnik commented Dec 19, 2014

jpsim commented Dec 19, 2014

jpsim commented Dec 19, 2014

juangamnik commented Dec 19, 2014

jpsim commented Dec 19, 2014

juangamnik commented Dec 19, 2014

jpsim commented Dec 19, 2014

juangamnik commented Dec 19, 2014

jpsim commented Dec 19, 2014

juangamnik commented Dec 20, 2014

jpsim commented Dec 20, 2014

juangamnik commented Dec 20, 2014

juangamnik commented Dec 21, 2014

jpsim commented Dec 23, 2014

juangamnik commented Dec 23, 2014

jpsim commented Dec 23, 2014

juangamnik commented Dec 24, 2014

jpsim commented Dec 25, 2014

juangamnik commented Dec 25, 2014