Merge pull request #1 from broadinstitute/develop

First commit

.gitignore

@@ -0,0 +1,5 @@
+tags
+.idea/
+target
+.artifactory
+*~

LICENSE.txt

@@ -0,0 +1,27 @@
+Copyright (c) 2015, Broad Institute, Inc.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright notice,
+  this list of conditions and the following disclaimer in the documentation
+  and/or other materials provided with the distribution.
+
+* Neither the name Broad Institute, Inc. nor the names of its
+  contributors may be used to endorse or promote products derived from
+  this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE

README.md

@@ -1 +1,170 @@
-# wdltool
+Command line utilities for interacting with WDL
+
+<!---toc start-->
+
+* [Requirements](#requirements)
+* [Building](#building)
+* [Command Line Usage](#command-line-usage)
+  * [validate](#validate)
+  * [inputs](#inputs)
+  * [highlight](#highlight)
+  * [parse](#parse)
+* [Getting Started with WDL](#getting-started-with-wdl)
+
+<!---toc end-->
+
+# Requirements
+
+The following is the toolchain used for development of wdltool.  Other versions may work, but these are recommended.
+
+* [Scala 2.11.7](http://www.scala-lang.org/news/2.11.7)
+* [SBT 0.13.8](https://github.com/sbt/sbt/releases/tag/v0.13.8)
+* [Java 8](http://www.oracle.com/technetwork/java/javase/overview/java8-2100321.html)
+
+# Building
+
+`sbt assembly` will build a runnable JAR in `target/scala-2.11/`
+
+Tests are run via `sbt test`.  Note that the tests do require Docker to be running.  To test this out while downloading the Ubuntu image that is required for tests, run `docker pull ubuntu:latest` prior to running `sbt test`
+
+# Command Line Usage
+
+Run the JAR file with no arguments to get the usage message:
+
+```
+$ java -jar wdltool.jar
+java -jar wdltool.jar <action> <parameters>
+
+Actions:
+validate <WDL file>
+
+  Performs full validation of the WDL file including syntax
+  and semantic checking
+
+inputs <WDL file>
+
+  Write a JSON skeleton file of the inputs needed for this
+  workflow.  Fill in the values in this JSON document and
+  pass it in to the 'run' subcommand.
+
+highlight <WDL file> <html|console>
+
+  Reformats and colorizes/tags a WDL file. The second
+  parameter is the output type.  "html" will output the WDL
+  file with <span> tags around elements.  "console" mode
+  will output colorized text to the terminal
+  
+parse <WDL file>
+
+  Compares a WDL file against the grammar and writes out an
+  abstract syntax tree if it is valid, and a syntax error
+  otherwise.  Note that higher-level AST checks are not done
+  via this sub-command and the 'validate' subcommand should
+  be used for full validation
+```
+
+## validate
+
+Given a WDL file, this runs the full syntax checker over the file and resolves imports in the process.  If any syntax errors are found, they are written out.  Otherwise the program exits.
+
+Error if a `call` references a task that doesn't exist:
+
+```
+$ java -jar wdltool.jar validate 2.wdl
+ERROR: Call references a task (BADps) that doesn't exist (line 22, col 8)
+
+  call BADps
+       ^
+
+```
+
+Error if namespace and task have the same name:
+
+```
+$ java -jar wdltool.jar validate 5.wdl
+ERROR: Task and namespace have the same name:
+
+Task defined here (line 3, col 6):
+
+task ps {
+     ^
+
+Import statement defined here (line 1, col 20):
+
+import "ps.wdl" as ps
+                   ^
+```
+
+## inputs
+
+Examine a WDL file with one workflow in it, compute all the inputs needed for that workflow and output a JSON template that the user can fill in with values.  The keys in this document should remain unchanged.  The values tell you what type the parameter is expecting.  For example, if the value were `Array[String]`, then it's expecting a JSON array of JSON strings, like this: `["string1", "string2", "string3"]`
+
+```
+$ java -jar wdltool.jar inputs 3step.wdl
+{
+  "three_step.cgrep.pattern": "String"
+}
+```
+
+This inputs document is used as input to the `run` subcommand.
+
+## highlight
+
+Formats a WDL file and semantically tags it.  This takes a second parameter (`html` or `console`) which determines what the output format will be.
+
+test.wdl
+```
+task abc {
+  String in
+  command {
+    echo ${in}
+  }
+  output {
+    String out = read_string(stdout())
+  }
+}
+
+workflow wf {
+  call abc
+}
+```
+
+## parse
+
+Given a WDL file input, this does grammar level syntax checks and writes out the resulting abstract syntax tree.
+
+```
+$ echo "workflow wf {}" | java -jar wdltool.jar parse /dev/stdin
+(Document:
+  imports=[],
+  definitions=[
+    (Workflow:
+      name=<stdin:1:10 identifier "d2Y=">,
+      body=[]
+    )
+  ]
+)
+```
+
+This WDL file can be formatted in HTML as follows:
+
+```
+$ java -jar wdltool.jar highlight test.wdl html
+<span class="keyword">task</span> <span class="name">abc</span> {
+  <span class="type">String</span> <span class="variable">in</span>
+  <span class="section">command</span> {
+    <span class="command">echo ${in}</span>
+  }
+  <span class="section">output</span> {
+    <span class="type">String</span> <span class="variable">out</span> = <span class="function">read_string</span>(<span class="function">stdout</span>())
+  }
+}
+
+<span class="keyword">workflow</span> <span class="name">wf</span> {
+  <span class="keyword">call</span> <span class="name">abc</span>
+}
+```
+
+# Getting Started with WDL
+
+For many examples on how to use WDL see [the WDL site](https://github.com/broadinstitute/wdl/tree/develop#getting-started-with-wdl)

build.sbt

@@ -0,0 +1,70 @@
+import com.typesafe.sbt.GitPlugin.autoImport._
+import sbt.Keys._
+import sbtassembly.MergeStrategy
+import com.typesafe.sbt.SbtGit.GitCommand
+
+name := "wdltool"
+
+organization := "org.broadinstitute"
+
+scalaVersion := "2.11.7"
+
+// Upcoming release, or current if we're on the master branch
+git.baseVersion := "0.1"
+
+// Shorten the git commit hash
+git.gitHeadCommit := git.gitHeadCommit.value map { _.take(7) }
+
+// Travis will deploy tagged releases, add -SNAPSHOT for all local builds
+git.gitUncommittedChanges := true
+
+versionWithGit
+
+assemblyJarName in assembly := "wdltool" + git.baseVersion.value + ".jar"
+
+logLevel in assembly := Level.Info
+
+
+libraryDependencies ++= Seq(
+  "org.broadinstitute" %% "wdl4s" % "0.1",
+  //---------- Test libraries -------------------//
+  "org.scalatest" %% "scalatest" % "2.2.5" % Test
+)
+
+val customMergeStrategy: String => MergeStrategy = {
+  case x if Assembly.isConfigFile(x) =>
+    MergeStrategy.concat
+  case PathList(ps@_*) if Assembly.isReadme(ps.last) || Assembly.isLicenseFile(ps.last) =>
+    MergeStrategy.rename
+  case PathList("META-INF", path@_*) =>
+    path map {
+      _.toLowerCase
+    } match {
+      case ("manifest.mf" :: Nil) | ("index.list" :: Nil) | ("dependencies" :: Nil) =>
+        MergeStrategy.discard
+      case ps@(x :: xs) if ps.last.endsWith(".sf") || ps.last.endsWith(".dsa") =>
+        MergeStrategy.discard
+      case "plexus" :: xs =>
+        MergeStrategy.discard
+      case "spring.tooling" :: xs =>
+        MergeStrategy.discard
+      case "services" :: xs =>
+        MergeStrategy.filterDistinctLines
+      case ("spring.schemas" :: Nil) | ("spring.handlers" :: Nil) =>
+        MergeStrategy.filterDistinctLines
+      case _ => MergeStrategy.deduplicate
+    }
+  case "asm-license.txt" | "overview.html" | "cobertura.properties" =>
+    MergeStrategy.discard
+  case _ => MergeStrategy.deduplicate
+}
+
+assemblyMergeStrategy in assembly := customMergeStrategy
+
+// The reason why -Xmax-classfile-name is set is because this will fail
+// to build on Docker otherwise.  The reason why it's 200 is because it
+// fails if the value is too close to 256 (even 254 fails).  For more info:
+//
+// https://github.com/sbt/sbt-assembly/issues/69
+// https://github.com/scala/pickling/issues/10
+scalacOptions ++= Seq("-deprecation", "-unchecked", "-feature", "-Xmax-classfile-name", "200")

project/plugins.sbt

@@ -0,0 +1,9 @@
+addSbtPlugin("com.typesafe.sbt" % "sbt-git" % "0.8.5")
+
+addSbtPlugin("com.github.gseitz" % "sbt-release" % "1.0.0")
+
+addSbtPlugin("org.scoverage" % "sbt-scoverage" % "1.0.4")
+
+addSbtPlugin("org.scoverage" % "sbt-coveralls" % "1.0.0")
+
+addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.1")