This repository has been archived by the owner on Nov 8, 2024. It is now read-only.
Introducing Swagger in Dredd #529
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| <!DOCTYPE HTML> | ||
| <!-- | ||
| This file exists only to redirect from old URL to new URL. | ||
| This is more properly done using an HTTP redirect, but unfortunately | ||
| readthedocs.org's page redirection is broken (see | ||
| https://github.com/rtfd/readthedocs.org/issues/1826). Once that bug is | ||
| fixed, the file can be removed. | ||
| --> | ||
| <html lang="en-US"> | ||
| <head> | ||
| <meta charset="UTF-8"> | ||
| <meta http-equiv="refresh" content="0;url=how-it-works/#execution-life-cycle"> | ||
| <script type="text/javascript"> | ||
| window.location.href = 'how-it-works/#execution-life-cycle' + window.location.hash | ||
| </script> | ||
| <title>Page Redirection</title> | ||
| </head> | ||
| <body> | ||
| <p> | ||
| The page for Dredd usage no longer exists. Please follow | ||
| the following link: | ||
| </p> | ||
| <ul> | ||
| <li><a href="how-it-works/#execution-life-cycle">How It Works</a></li> | ||
| </ul> | ||
| </body> | ||
| </html> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| <!DOCTYPE HTML> | ||
| <!-- | ||
| This file exists only to redirect from old URL to new URL. | ||
| This is more properly done using an HTTP redirect, but unfortunately | ||
| readthedocs.org's page redirection is broken (see | ||
| https://github.com/rtfd/readthedocs.org/issues/1826). Once that bug is | ||
| fixed, the file can be removed. | ||
| --> | ||
| <html lang="en-US"> | ||
| <head> | ||
| <meta charset="UTF-8"> | ||
| <meta http-equiv="refresh" content="0;url=../how-it-works/#execution-life-cycle"> | ||
| <script type="text/javascript"> | ||
| window.location.href = '../how-it-works/#execution-life-cycle' + window.location.hash | ||
| </script> | ||
| <title>Page Redirection</title> | ||
| </head> | ||
| <body> | ||
| <p> | ||
| The page for Dredd usage no longer exists. Please follow | ||
| the following link: | ||
| </p> | ||
| <ul> | ||
| <li><a href="../how-it-works/#execution-life-cycle">How It Works</a></li> | ||
| </ul> | ||
| </body> | ||
| </html> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,215 @@ | ||
| # How It Works | ||
|
|
||
| In a nutshell, Dredd does following: | ||
|
|
||
| 1. Takes your API description document, | ||
| 2. creates expectations based on requests and responses documented in the document, | ||
| 3. makes requests to tested API, | ||
| 4. checks whether API responses match the documented responses, | ||
| 5. reports the results. | ||
|
|
||
| ## Versioning | ||
|
|
||
| Dredd follows [Semantic Versioning][]. To ensure certain stability of your Dredd installation (e.g. in CI), pin the version accordingly. You can also use release tags: | ||
|
|
||
| - `npm install dredd` - Installs the latest published version including experimental pre-release versions. | ||
| - `npm install dredd@stable` - Skips experimental pre-release versions. Recommended for CI installations. | ||
|
|
||
| If the `User-Agent` header isn't overridden in the API description document, Dredd uses it for sending information about its version number along with every HTTP request it does. | ||
|
|
||
| ## Execution Life Cycle | ||
|
|
||
| Following execution life cycle documentation should help you to understand how Dredd works internally and which action goes after which. | ||
|
|
||
| 1. Load and parse API description documents | ||
| - Report parse errors and warnings | ||
| 2. Pre-run API description check | ||
| - Missing example values for URI template parameters | ||
| - Required parameters present in URI | ||
| - Report non-parseable JSON bodies | ||
| - Report invalid URI parameters | ||
| - Report invalid URI templates | ||
| 3. Compile HTTP transactions from API description documents | ||
| - Inherit headers | ||
| - Inherit parameters | ||
| - Expand URI templates with parameters | ||
| 4. Load [hooks](hooks.md) | ||
| 5. Test run | ||
| - Report test run `start` | ||
| - Run `beforeAll` hooks | ||
| - For each compiled transaction: | ||
| - Report `test start` | ||
| - Run `beforeEach` hook | ||
| - Run `before` hook | ||
| - Send HTTP request | ||
| - Receive HTTP response | ||
| - Run `beforeEachValidation` hook | ||
| - Run `beforeValidation` hook | ||
| - [Perform validation](#automatic-expectations) | ||
| - Run `after` hook | ||
| - Run `afterEach` hook | ||
| - Report `test end` with result for in-progress reporting | ||
| - Run `afterAll` hooks | ||
| 6. Report test run `end` with result statistics | ||
|
|
||
| ## Limitations of API Description Formats | ||
|
|
||
| ## Automatic Expectations | ||
|
|
||
| Dredd automatically generates expectations on HTTP responses based on examples in the API description with use of [Gavel.js][] library. Please refer to [Gavel][] rules if you want know more. | ||
|
|
||
| ### Response Headers Expectations | ||
|
|
||
| - All headers specified in the API description must be present in the response. | ||
| - Names of headers are validated in the case-insensitive way. | ||
| - Only values of headers significant for content negotiation are validated. | ||
| - All other headers values can differ. | ||
|
|
||
| When using [Swagger][], headers are taken from [`response.headers`][]. HTTP headers significant for content negotiation are inferred according to following rules: | ||
|
|
||
| - [`produces`][] is propagated as response's `Content-Type` header. | ||
| - Response's `Content-Type` header overrides any `produces`. | ||
|
|
||
| > **Note:** There is a bug affecting the last item - [apiaryio/fury-adapter-swagger#65](https://github.com/apiaryio/fury-adapter-swagger/issues/65). | ||
|
|
||
| ### Response Body Expectations | ||
|
|
||
| If the HTTP response body is JSON, Dredd validates only its structure. Bodies in any other format are validated as plain text. | ||
|
|
||
| To validate the structure Dredd uses [JSON Schema][] inferred from the API description under test. The effective JSON Schema is taken from following places (the order goes from the highest priority to the lowest): | ||
|
|
||
| #### API Blueprint | ||
|
|
||
| 1. [`+ Schema`][] section - provided custom JSON Schema (draft v4 or v3) will be used. | ||
| 2. [`+ Attributes`][] section with data structure description in [MSON][] - API Blueprint parser automatically generates JSON Schema from MSON. | ||
| 3. [`+ Body`][] section with sample JSON payload - [Gavel.js][], which is responsible for validation in Dredd, automatically infers some basic expectations described below. | ||
|
|
||
| This order [exactly follows the API Blueprint specification][body-schema-attributes]. | ||
|
|
||
| #### Swagger | ||
|
|
||
| 1. [`response.schema`][] - provided JSON Schema will be used. | ||
| 2. [`response.examples`][] with sample JSON payload - [Gavel.js][], which is responsible for validation in Dredd, automatically infers some basic expectations described below. | ||
|
|
||
| #### Gavel's Expectations | ||
|
|
||
| - All JSON keys on any level given in the sample must be present in the response's JSON. | ||
| - Response's JSON values must be of the same JSON primitive type. | ||
| - All JSON values can differ. | ||
| - Arrays can have additional items, type or structure of the items is not validated. | ||
| - Plain text must match perfectly. | ||
|
|
||
| ### Custom Expectations | ||
|
|
||
| You can make your own custom expectations in [hooks](hooks.md). For instance, check out how to employ [Chai.js assertions](hooks.md#using-chai-assertions). | ||
|
|
||
| ## Making Your API Description Ready for Testing | ||
|
|
||
| It's very likely that your API description document will not be testable __as is__. This section should help you to learn how to solve the most common issues. | ||
|
|
||
| ### URI Parameters | ||
|
|
||
| Both [API Blueprint][] and [Swagger][] allow usage of URI templates (API Blueprint fully implements [RFC6570][], Swagger templates are much simpler). In order to have an API description which is testable, you need to describe all required parameters used in URI (path or query) and provide sample values to make Dredd able to expand URI templates with given sample values. Following rules apply when Dredd interpolates variables in a templated URI, ordered by precedence: | ||
|
|
||
| 1. Sample value (not available in Swagger). | ||
| 2. Value of `default` (only if the parameter is marked as `optional`). | ||
| 3. First value from `enum`. | ||
|
|
||
| If Dredd isn't able to infer any value for a required parameter, it will terminate the test run and complain that the parameter is _ambiguous_. | ||
|
|
||
| > **Note:** The implementation of API Blueprint's request-specific parameters is still in progress and there's only experimental support for it in Dredd as of now. | ||
|
|
||
| ### Request Headers | ||
|
|
||
| In [Swagger][] documents, HTTP headers are inferred from [`"in": "header"` parameters][parameters]. HTTP headers significant for content negotiation are inferred according to following rules: | ||
|
|
||
| - [`consumes`][] is propagated as request's `Content-Type` header. | ||
| - [`produces`][] is propagated as request's `Accept` header. | ||
| - If request body parameters are specified as `"in": "form"`, request's `Content-Type` header is set to `application/x-www-form-urlencoded`. | ||
|
There was a problem hiding this comment. Also I'm not sure this is 100% true as of now. Should be fixed or filed as an issue on Dredd/Swagger Adapter cc @pksunkara @netmilk There was a problem hiding this comment. |
||
|
|
||
| > **Note:** Processing `"in": "header"` parameters and inferring `application/x-www-form-urlencoded` from `"in": "form"` parameters is not implemented yet ([apiaryio/fury-adapter-swagger#68](https://github.com/apiaryio/fury-adapter-swagger/issues/68), [apiaryio/fury-adapter-swagger#67](https://github.com/apiaryio/fury-adapter-swagger/issues/67)). | ||
|
|
||
|
There was a problem hiding this comment. I'd mention here inferred content-type header can be overridden by explicitly setting the in header parameter. And it's a known bug, because it doesn't work like that atm. |
||
| ### Request Body | ||
|
|
||
| #### API Blueprint | ||
|
|
||
| The effective request body is taken from following places (the order goes from the highest priority to the lowest): | ||
|
|
||
| 1. [`+ Body`][] section with sample JSON payload. | ||
| 2. [`+ Attributes`][] section with data structure description in [MSON][] - API Blueprint parser automatically generates sample JSON payload from MSON. | ||
|
|
||
| This order [exactly follows the API Blueprint specification][body-schema-attributes]. | ||
|
|
||
| #### Swagger | ||
|
|
||
| The effective request body is inferred from [`"in": "body"` and `"in": "form"` parameters][parameters]. | ||
|
|
||
| If body parameter has [`schema.example`][], it is used as a raw JSON sample for the request body. If it's not present, Dredd's [Swagger Adapter][] generates sample values from the JSON Schema provided in the [`schema`][] property. Following rules apply when the adapter fills values of the properties, ordered by precedence: | ||
|
|
||
| 1. Value of `default`. | ||
|
There was a problem hiding this comment. What about schema examples? Please verify how it works and clarify it here. There was a problem hiding this comment. Checked with Pavan and addressed in 1ff8e52. |
||
| 2. First value from `enum`. | ||
| 3. Dummy, generated value. | ||
|
|
||
| ## How Dredd Works With HTTP Transactions | ||
|
|
||
| ### Multiple Requests and Responses | ||
|
|
||
| #### API Blueprint | ||
|
|
||
| While [API Blueprint][] allows specifying multiple requests and responses in any | ||
| combination (see specification for the [action section][]), Dredd currently | ||
| supports just separated HTTP transaction pairs like this: | ||
|
|
||
| ``` | ||
| + Request | ||
| + Response | ||
|
|
||
| + Request | ||
| + Response | ||
| ``` | ||
|
|
||
| In other words, Dredd always selects just the first response for each request. | ||
|
|
||
| > **Note:** Improving the support for multiple requests and responses is under development. Refer to issues [#25](https://github.com/apiaryio/dredd/issues/25) and [#78](https://github.com/apiaryio/dredd/issues/78) for details. Support for URI parameters specific to a single request within one action is also limited. Solving [#227](https://github.com/apiaryio/dredd/issues/227) should unblock many related problems. Also see [Multiple Requests and Responses within One API Blueprint Action](how-to-guides.md#multiple-requests-and-responses-within-one-api-blueprint-action) guide for workarounds. | ||
|
|
||
| #### Swagger | ||
|
|
||
| The [Swagger][] format allows to specify multiple responses for a single operation. | ||
| By default Dredd tests only responses with `2xx` status codes. Responses with other | ||
| codes are marked as _skipped_ and can be activated in [hooks](hooks.md) - see [Testing non-2xx Responses with Swagger](how-to-guides.md#testing-non-2xx-responses-with-swagger). | ||
|
|
||
| [Default responses][default responses] are ignored by Dredd. Also, as of now, | ||
| only `application/json` media type is supported in [`produces`][] and [`consumes`][]. | ||
|
There was a problem hiding this comment. This should be extended for Consider using this npm: https://www.npmjs.com/package/media-typer There was a problem hiding this comment. I think this is a larger story - AFAIK most of the Dredd stack can't deal with |
||
| Other media types are skipped. | ||
|
|
||
|
|
||
| [Semantic Versioning]: http://semver.org/ | ||
| [API Blueprint]: http://apiblueprint.org/ | ||
| [Swagger]: http://swagger.io/ | ||
| [Gavel.js]: https://github.com/apiaryio/gavel.js | ||
| [Gavel]: https://www.relishapp.com/apiary/gavel/docs | ||
| [MSON]: https://github.com/apiaryio/mson | ||
| [JSON Schema]: http://json-schema.org/ | ||
| [Swagger Adapter]: https://github.com/apiaryio/fury-adapter-swagger/ | ||
| [RFC6570]: https://tools.ietf.org/html/rfc6570 | ||
|
|
||
| [`+ Schema`]: https://apiblueprint.org/documentation/specification.html#def-schema-section | ||
| [`+ Parameters`]: https://apiblueprint.org/documentation/specification.html#def-uriparameters-section | ||
| [`+ Attributes`]: https://apiblueprint.org/documentation/specification.html#def-attributes-section | ||
| [`+ Body`]: https://apiblueprint.org/documentation/specification.html#def-body-section | ||
| [`+ Request`]: https://apiblueprint.org/documentation/specification.html#def-action-section | ||
| [action section]: https://apiblueprint.org/documentation/specification.html#def-action-section | ||
| [body-schema-attributes]: https://apiblueprint.org/documentation/specification.html#relation-of-body-schema-and-attributes-sections | ||
|
|
||
| [`produces`]: http://swagger.io/specification/#swaggerProduces | ||
| [`consumes`]: http://swagger.io/specification/#swaggerConsumes | ||
| [`response.headers`]: http://swagger.io/specification/#responseHeaders | ||
| [`schema`]: http://swagger.io/specification/#parameterSchema | ||
| [`response.schema`]: http://swagger.io/specification/#responseSchema | ||
| [`response.examples`]: http://swagger.io/specification/#responseExamples | ||
| [parameters]: http://swagger.io/specification/#parameterObject | ||
| [`operation.parameters`]: http://swagger.io/specification/#operationParameters | ||
| [`paths.parameters`]: http://swagger.io/specification/#pathItemParameters | ||
| [`swagger.parameters`]: http://swagger.io/specification/#swaggerParameters | ||
| [default responses]: http://swagger.io/specification/#responsesDefault | ||
| [`schema.example`]: http://swagger.io/specification/#schemaExample | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is how it should be, but it is buggy in the implementation, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, should be fixed or filed as an issue on Dredd/Swagger Adapter cc @pksunkara
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
apiaryio/fury-adapter-swagger#65