Merge pull request #112 from jaeddy/ghpages-fixes

Taking @dglazer's "LGTM" as an OK to merge (fix issues with docs/swagger build and GH pages)

.travis.yml

@@ -26,16 +26,12 @@ jobs:
       jdk: oraclejdk8
       before_install:
       - chmod +x gradlew
+      - chmod +x scripts/fetchpages.sh
+      - chmod +x scripts/stagepages.sh
       script:
-      - "./gradlew asciidoctor"
-
-    - stage: build_swagger
-      language: node_js
-      node_js:
-      - "node"
-      script: 
-      - "npm run build"
-      - "npm run deploy"
+      - "./scripts/fetchpages.sh"
+      - "./gradlew installSwagger buildSwagger asciidoctor"
+      - "./scripts/stagepages.sh"
       deploy:
         provider: pages
         skip-cleanup: true

CONTRIBUTING.md

@@ -3,9 +3,24 @@
 
 This schema is developed by the [Cloud Work Stream](https://ga4gh.cloud) of the [Global Alliance for Genomics and Health](https://ga4gh.org).
 
+## Semantic Versioning
+
+We use [semantic versioning](https://semver.org/) for WES, this will determine if your proposed changes impact a major or minor release.
+
 ## Suggesting Changes
 
-Suggested changes to this schema can be initiated as Issues or Pull Requests to allow for discussion and review. Even those with write access to the main repository should in general create pull request branches within their own forks. This way when the main repository is forked again, the new fork is created with a minimum of extraneous volatile branches.
+Suggested changes to this schema can be initiated as [**Issues**](https://github.com/ga4gh/workflow-execution-service-schemas/issues) or [**Pull Requests**](https://github.com/ga4gh/workflow-execution-service-schemas/pulls) to allow for discussion and review. 
+
+Even those with write access to the main repository should in general create pull request branches within their own forks. This way when the main repository is forked again, the new fork is created with a minimum of extraneous volatile branches.
+
+> To facilitate review of external pull requests, users are encouraged to activate [**Travis CI**](https://travis-ci.org/) to monitor the build status (documentation, Swagger UI) of their fork. By following the documentation for [deployment to GitHub Pages](https://docs.travis-ci.com/user/deployment/pages/) and adding a `$GITHUB_TOKEN` environment variable to their repo configuration, pushes to the forked repo should be viewable relative to `https://[user-or-org].github.io/workflow-execution-service-schemas/preview/<branch>/`:
+
++ https://[user-or-org].github.io/workflow-execution-service-schemas/preview/[branch]/docs/
++ https://[user-or-org].github.io/workflow-execution-service-schemas/preview/[branch]/swagger-ui/
++ https://[user-or-org].github.io/workflow-execution-service-schemas/preview/[branch]/swagger.json
++ https://[user-or-org].github.io/workflow-execution-service-schemas/preview/[branch]/swagger.yaml
+
+> Providing this base URL in the pull request comment is appreciated, but not required. When creating a pull request from the forked repository's `develop` branch, the base URL will just be `https://[user-or-org].github.io/workflow-execution-service-schemas/`.
 
 If a security vulnerability is identified with the specification please send an email to [email protected] detailing your concerns.
 

README.md

@@ -1,86 +1,75 @@
 <img src="https://www.ga4gh.org/gfx/GA-logo-horizontal-tag-RGB.svg" alt="GA4GH Logo" style="width: 400px;"/>
 
-Schemas for the Workflow Execution Service (WES) API
-====================================================
+Workflow Execution Service (WES) API
+====================================
 
-The [Global Alliance for Genomics and Health](http://genomicsandhealth.org/) is an international
-coalition, formed to enable the sharing of genomic and clinical data.
+[![Build Status](https://travis-ci.org/ga4gh/workflow-execution-service-schemas.svg?branch=develop)](https://travis-ci.org/ga4gh/workflow-execution-service-schemas)
+<a href="https://ga4gh.github.io/workflow-execution-service-schemas/swagger.yaml"><img src="http://online.swagger.io/validator?url=https://ga4gh.github.io/workflow-execution-service-schemas/swagger.yaml" alt="Swagger Validator" height="20em" width="72em"></A>
+
+The [Global Alliance for Genomics and Health](http://genomicsandhealth.org/) is an international coalition, formed to enable the sharing of genomic and clinical data.
 
 Cloud Work Stream
 -----------------
 
-The [Cloud Work Stream](https://ga4gh/cloud) concentrates on data representation, storage,
-and analysis, including working with platform development partners and
-industry leaders to develop standards that will facilitate
-interoperability.  The Cloud Work Stream is an informal, multi-vendor working group focused on standards for exchanging Docker-based tools and CWL/WDL workflows, execution of Docker-based tools and workflows on clouds, and abstract access to cloud object stores.
+The [Cloud Work Stream](https://ga4gh/cloud) helps the genomics and health communities take full advantage of modern cloud environments.
+Our initial focus is on “bringing the algorithms to the data”, by creating standards for defining, sharing, and executing portable workflows.
+We work with platform development partners and industry leaders to develop standards that will facilitate interoperability.
 
 What is WES?
 ============
 
-The Workflow Execution Schema is a minimal common API describing how a user can submit
-workflow requests to workflow execution systems in standardized ways.
-Workflow execution engines (SevenBridges, FireCloud, etc) can support this API so users can make workflow requests
-programmatically, adding the ability to scale up.  In addition, these workflow services could have (and probably do have)
-UIs that would (possibly) use this API under the hood to facilitate workflow execution requests.
-
-Having this standard API supported by multiple execution engines will give people options of processing
-the same workflow (CWL or WDL) across different workflow execution platforms running across various clouds/environments.
-As an example use case, one can find a workflow in CWL on [Dockstore.org](http://dockstore.org), use Dockstore to
-generate a JSON parameterization file, and submit this to a GA4GH-compliant
-workflow execution service.
+The Workflow Execution Service API describes a standard programmatic way to run and manage workflows.
+Having this standard API supported by multiple execution engines will let people run
+the same workflow using various execution platforms running on various clouds/environments.
+Key features include:
 
-Key features of the current API proposal:
+* ability to request a workflow run using CWL or WDL
+* ability to parameterize that workflow using a JSON schema
+* ability to get information about running workflows
 
-* ability to request a workflow run using CWL or WDL (and maybe future formats)
-* ability to parameterize that workflow using a JSON schema (ideally a future version would be in common between CWL and WDL)
-* ability to get information about running workflows, status, errors, output file locations, etc.
+API Definition
+--------------
 
-Outstanding questions:
+See the human-readable [Reference Documentation](https://ga4gh.github.io/workflow-execution-service-schemas/docs/) 
+and the [OpenAPI description](openapi/workflow_execution_service.swagger.yaml). You can also explore the specification in [Swagger UI](https://ga4gh.github.io/workflow-execution-service-schemas/swagger-ui/) or [Swagger Editor](https://editor.swagger.io/#/?import=https://ga4gh.github.io/workflow-execution-service-schemas/swagger.yaml).
 
-* a common JSON parameterization format
-* standardizing terms, job, workflow, steps, tools, etc
-* reference implementation at https://github.com/common-workflow-language/cwltool-service/tree/ga4gh-wes
-* validation service for testing WES implementations' conformance to the spec
-* Including all task_logs in the workflow log request may present a scaling problem when there are 100s-1000s of tasks
-* Providing a state notification callback URL (eg a webhook)
-* Passing through authentication (user role)
-
-How to view
-------------
+Use Cases
+---------
 
-- Documentation (ReDoc): https://ga4gh.github.io/workflow-execution-service-schemas/
-- SwaggerUI: https://ga4gh.github.io/workflow-execution-service-schemas/swagger-ui/
-- Vuew full spec:
-    + JSON https://ga4gh.github.io/workflow-execution-service-schemas/swagger.json
-    + YAML https://ga4gh.github.io/workflow-execution-service-schemas/swagger.yaml
-- Preview spec version for branch `[branch]`: https://ga4gh.github.io/workflow-execution-service-schemas/preview/[branch]
--
+Use cases include:
 
-Please visit http://ga4gh.github.io/workflow-execution-service-schemas to view this document in Swagger UI.
+* "Bring your code to the data": a researcher who has built their own custom analysis can submit it to run on a dataset owned by an external organization, instead of having to make a copy of the data
+* Best-practices pipelines: a researcher who maintains their own controlled data environment can find useful workflows in a shared directory (e.g. [Dockstore.org](http://dockstore.org)), and run them over their data
 
+Possible Future Enhancements
+----------------------------
 
-Building Documents
-------------------
+* common JSON parameterization format that works with CWL and WDL
+* validation service for testing WES implementations' conformance to the spec
+* improved tools for troubleshooting execution failures, especially when there are 100s-1000s of tasks
+* a callback mechanism for monitoring status changes in running workflows (e.g., a webhook)
+* integration with GA4GH data access APIs (e.g., htsget, DOS)
 
-The OpenAPI description is in the `openapi` directory.
+How to View
+------------
 
+* Documentation: https://ga4gh.github.io/workflow-execution-service-schemas/docs/
+* Full API specification:
+    * OpenAPI YAML: https://ga4gh.github.io/workflow-execution-service-schemas/swagger.yaml 
+    * OpenAPI JSON: https://ga4gh.github.io/workflow-execution-service-schemas/swagger.yaml
+    * Swagger [UI](https://ga4gh.github.io/workflow-execution-service-schemas/swagger-ui/) or [Editor](https://editor.swagger.io/#/?import=https://ga4gh.github.io/workflow-execution-service-schemas/swagger.yaml))
 
-How to contribute changes
+How to Contribute Changes
 -------------------------
 
-Take cues for now from the [ga4gh/schemas](https://github.com/ga4gh/schemas/blob/master/CONTRIBUTING.rst) document.
-
-We like [HubFlow](https://datasift.github.io/gitflow/) and using pull requests to suggest changes.
-
-Security Considerations
------------------------
+See [CONTRIBUTING.md](CONTRIBUTING.md).
 
-If a security issue with any of the above specifications is realised please send an email to [email protected] detailing your concerns.
+If a security issue is identified with the specification, please send an email to [email protected] detailing your concerns.
 
 License
 -------
 
-See the [LICENSE]
+See the [LICENSE](LICENSE).
 
 More Information
 ----------------

build.gradle

@@ -3,6 +3,9 @@ buildscript {
         jcenter()
         mavenCentral()
         maven { url 'http://oss.jfrog.org/artifactory/oss-snapshot-local/' }
+        maven {
+            url "https://plugins.gradle.org/m2/"
+        }
         //mavenLocal()
     }
     dependencies {
@@ -12,13 +15,19 @@ buildscript {
         classpath "io.github.swagger2markup:swagger2markup-import-files-ext:1.3.1"
         classpath "com.bluepapa32:gradle-watch-plugin:0.1.5"
         classpath "org.kordamp.gradle:livereload-gradle-plugin:0.2.1"
+        classpath "com.moowork.gradle:gradle-node-plugin:1.2.0"
     }
 }
 
 apply plugin: 'org.asciidoctor.convert'
 apply plugin: 'com.bluepapa32.watch'
 apply plugin: 'org.kordamp.gradle.livereload'
 apply plugin: 'io.github.swagger2markup'
+apply plugin: 'com.moowork.node'
+node {
+    version = '8.9.0'
+    download = true
+}
 
 group 'io.github.swagger2markup'
 version '1.3.1'
@@ -87,3 +96,12 @@ liveReload {
 task wrapper(type: Wrapper) {
     gradleVersion = '3.5'
 }
+
+task installSwagger(type: NpmTask) {
+    npmCommand = ["install"]
+}
+
+task buildSwagger(type: NpmTask) {
+    npmCommand = ["run", "build"]
+}
+

docs/asciidoc/front_matter.adoc

@@ -1,17 +1,32 @@
 == Executive Summary
 
-An executive summary summarising the major points of the document. To be added for https://github.com/ga4gh/workflow-execution-service-schemas/issues/37[issue #37].
+The Workflow Execution Service (WES) API provides a standard way for users to submit workflow requests to workflow execution systems, and to monitor their execution. This API lets users run a single workflow (currently https://www.commonwl.org/[CWL] or http://www.openwdl.org/[WDL] formatted workflows, other types may be supported in the future) on multiple different platforms, clouds, and environments.
+
+Key features of the API:
+
+* can request that a workflow be run
+* can pass parameters to that workflow (e.g. input files, cmdline arguments)
+* can get information about running workflows (e.g. status, errors, output file locations)
+* can cancel a running workflow
 
 == Introduction
 
-Introduction setting the document’s scope.
+This document describes the WES API and provides details on the specific endpoints, request formats, and response.  It is intended to provide key information for developers of WES-compatible services as well as clients that will call these WES services.
+
+Use cases include:
+
+* "Bring your code to the data": a researcher who has built their own custom analysis can submit it to run on a dataset owned by an external organization, instead of having to make a copy of the data
+* Best-practices pipelines: a researcher who maintains their own controlled data environment can find useful workflows in a shared directory (e.g. Dockstore.org), and run them over their data
 
 == Standards
 
-Standards incorporated (?)
+The WES API specification is written in OpenAPI and embodies a RESTful service philosophy.  It uses JSON in requests and responses and standard HTTP/HTTPS for information transport.
 
 == Authorization & Authentication
 
 Users must supply credentials that establish their identity and authorization in order to use a WES endpoint. We recommend that WES implementations use an OAuth2 https://oauth.net/2/bearer-tokens/[bearer token], although they can choose other mechanisms if appropriate. WES callers can use the `auth_instructions_url` from the https://ga4gh.github.io/workflow-execution-service-schemas/#/WorkflowExecutionService/GetServiceInfo[service-info endpoint] to learn how to obtain and use a bearer token for a particular implementation.
 
 The WES implementation is responsible for checking that a user is authorized to submit workflow run requests. The particular authorization policy is up to the WES implementer.
+
+Systems like WES need to also address the ability to pass crentials with jobs for input and output access.  In the current 
+version of WES, the passing of credentials to authenticate and authorize access to inputs and outputs, as well as mandates about necessary file transfer protocols to support, are out of scope.  However, parallel work on the Data Object Service is addressing ways to pass around access credentials with data object references, opening up the possibility that a future version of WES will provide concrete mechanisms for workflow runs to access data using credentials different than those used for WES.  This is a work in progress and support of DOS in WES will be added in a future release of WES.

docs/asciidoc/index.adoc

@@ -1,4 +1,4 @@
 include::{generated}/overview.adoc[]
 include::front_matter.adoc[]
 include::{generated}/paths.adoc[]
-include::{generated}/definitions.adoc[]
+include::{generated}/definitions.adoc[]