Code analysis tools

[msbarry]

IDEs often provide nice warnings for things like unused variables but to improve consistency between different editors, and catch easy-to-automate issues in pull requests we should use a standard tool not coupled to a particular IDE.

Goals:

• highlight warnings in PRs with annotations similar to how the issue would be highlighted in an IDE
• good IDE support (eclipse, intellij, VS code) that runs analysis and highlights issues as you type
• can run locally via maven build
• highlight obviously-bad issues (like not closing a resource, unused variables, variables that could be final) but not nit-picky stuff like whitespace or method length

Please add any tool you personally recommend, vote up your preference, or help address some of the cons in discussion below each possibility.

Created from notes on #124

[msbarry]

Checkstyle
It looks like checkstyle just finds easy syntactical bugs (indentation, spacing, etc.) that should be resolved automatically spotless.

[msbarry]

Error prone
This is a compiler "plugin" from Google that detects potential issues from analyzing the AST.
Pros: fast, warnings look sane and actionable
Cons: zero support for eclipse/vscode, clunky intellij plugin, doesn't run as you edit, introduces flakiness into compile step
Example PR: #173

[msbarry]

Spotbugs
This is the successor to findbugs to detect possible errors.
Pros: widely-used, many checks available, good IDE support
Cons: slower and noisier than error prone, will need some tuning to get useful output

[msbarry]

PMD
Multi-language code analyzer with java/maven support
Pros: Configurable and output looks reasonable, good IDE support
Cons: Intellij plugin does not run as you type
Example PR: #174

[msbarry]

Qodana
This is a tool for CI built on the warnings generated by IDEA.
Pros: High quality output, same that I'm used to from IDEA
Cons: Closed-source, needs to run in a docker container, VERY slow
Example PR: #175

[msbarry]

Sonar
This is a commercial cloud tool that's free for open source projects. It has IDE plugins for eclipse, intellij, and VS code and the cloud analysis tool that can run against github PRs.
Pros: best IDE plugin support, easy integration into github PR workflow, opinionated defaults, github PR analysis only looks at changes in the current PR
Cons: The github PR integration does not add inline annotations - which is the main thing I was looking for, and also you can't run locally via maven without a sonar cloud API key
Example PR: #177

[msbarry]

General questions/comment thread (don't vote for this one!)

[msbarry]

Infer
Static analysis tool from Facebook written in haskell. It
Pros: default checks look like they would have high signal to noise ratio, mostly reserved for likely bugs, has IDE support, but non-standard to set up
Cons: can't get it running or building on my m1 mac, maven plugin needs to download the platform-specific binary to run, and they only publish linux binaries, IDE support limited to vscode