Merge pull request #694 from aml-org/release/4.4.0

Publish 4.4.0

CONTRIBUTING.md

@@ -34,11 +34,11 @@ behavior that is not occurring.
 
 We use [Scalafmt](https://scalameta.org/scalafmt/) to format our code! Please format your code before opening a Pull Request.
 
-### Tests aren’t optional
-Any bugfix that doesn’t include a test proving the existence of the bug being fixed, may be suspect. 
-Same for new features that can’t prove they actually work.
- 
-Writing tests before the implementation is strongly encouraged.
+### Running and writing tests
+
+**Important**: Please include tests with any code contributions
+
+Writing tests before the implementation is strongly encouraged. 
 
 To run tests:
 ```sh
@@ -47,10 +47,16 @@ $ sbt clientJVM/testOnly // to run tests only for the JVM platform
 $ sbt clientJS/testOnly // to run tests only for the JS platform
 ```
 
+Code contributions must comply with a minimum of 80% coverage rate. 
+
+To run a coverage report of the whole project:
+```sh
+sbt coverage test coverageReport
+```
+
 #### Travis CI
 
-A Travis CI script is available for you to run Travis. To be able to do this you must have a Travis account and 
-setup Travis for your fork in your account.
+Travis CI will automatically run tests upon creation of your PR, please make sure all tests pass before someone can review you contribution.
 
 #### Test coverage
 

README.md

@@ -2,34 +2,54 @@
 
 # AML Modeling Framework
 
-This project aims to provide a common programming interface that lets developers interact with any API specification, whether it is written in OpenAPI Specification (OAS) or RAML, similar to the way the HTML DOM allows programmatic interaction with an HTML document.
+AMF (AML Modeling Framework) is an open-source programming framework, capable of parsing, generating and validating metadata documents defined using [AML](https://a.ml/aml-spec). It can be used as a library in Java or JavaScript projects, or as a stand-alone command-line tool. The modular design of AMF facilitates creating plugins capable of parsing other metadata syntaxes not defined by AML.
 
 ## Vision
 
-The API Modeling Framework (AMF) allows users to formally describe different kinds of APIs, parse and generate instances of those APIS as sets of modular documents and  store those connected descriptions into a single unified data graph.
+[AML](https://a.ml/aml-spec) allows users to formally describe different kinds of models, whether syntactic models (i.e. language/specification specific) or semantic models (i.e. domain/industry specific). AMF can parse and generate descriptions of those models as sets of modular documents and store those connected descriptions into a single unified data graph.
 
 ## Goals
 
-- Support for multiple languages with a unified output API/model for clients
-- Support for both, document (RAML modularity) and domain (service clients), layers
-- Bi-directional transformation
-- Support for validation at document and service layers
-- Produce a formal specification for the language
-- Extensible, single document model for multiple domain vocabularies
-- Consistent parsing behavior
-
-## General scope
-The library supports many of the required uses cases:
-- Parse a 0.8 / 1.0 RAML, 2.0 / 3.0 OAS / ASYNC 2.0 and JSON-LD AMF model.
-- AMF API design model creation.
-- Model edition.
-- Export to any of the mentioned standards.
+- Support for multiple formats with a unified programming interface and model
+- Support for both syntactic and semantic model layers
+- Support for validation at both syntactic and semantic model layers
+- Produce formal specifications of any (specification) language
+- Extensible, single syntactic model for multiple semantic models
+- Ability to create consistent parsing behaviors across different syntactic models
+- Support for bi-directional transformation and export
+- Support for custom (parsing/resolution/generation) pipelines
 
-## Usage
+## Native support
+
+AMF natively supports the following formats:
+- YAML
+- JSON
+
+the following semantic models:
+- WebApi (or "Web APIs" as in "APIs accessible over the network")
+
+and the following syntactic models:
+- JSON-LD "AMF model"
+- RAML 0.8 / 1.0 (mapped to "WebApi")
+- OpenAPI (OAS) 2.0 / 3.0 (mapped to "WebApi")
+- AsyncAPI 2.0 (beta) (mapped to "WebApi")
+
+The models above and any other models may be extended and supported via custom [AML-defined models](https://a.ml/aml-spec). Other formats and models that cannot be expressed with AML may also be supported via plugins. 
+
+## Guaranteed output
+
+The **only** guaranteed output of AMF is the JSON-LD "AMF model". Any other output such as any output provided natively by the models listed under the section above may change at any time. This means that while the semantic representation of those outputs may remain unchanged, the syntactical expression such as the order in which the outputted metadata is expressed and any other syntax-related constructs may change from one version of AMF to another. If this is an issue for your use-case, you may consider using a custom resolution/generation pipeline.
 
-To use AMF you should first generate or get the right distribution for your project and import them as dependencies.
+## Documentation
+- [The AML Project](https://a.ml)
+- [What is AMF?](https://a.ml/docbook/overview_amf.html)
+- [AMF model documentation](documentation/model.md)
+- [Validation insights](./documentation/validation.md)
+- [Basic use cases - parsing & validating an API](documentation/basic_use_cases.md)
+- [Code examples](https://github.com/aml-org/examples)
+- [More code examples](https://github.com/mulesoft/amf-examples)
 
-## Installation
+## Usage
 
 ### Download JVM artifacts
 
@@ -57,9 +77,9 @@ Maven snippet:
 </dependency>
 ```
 
-NOTE: `-SNAPSHOT` versions of the JVM artifacts are available but may contain breaking changes.
+NOTE: you may use the `-SNAPSHOT` versions of the artifacts at your own risk since those snapshot versions may contain breaking changes.
 
-### Download JS artifacts
+### JavaScript artifacts
 
 NPM:
 ```bash
@@ -112,7 +132,7 @@ class App {
 
 You can build a standalone Java executable (JAR) running the following SBT target:
 ```bash
-sbt buildCommandLine
+$ sbt buildCommandLine
 ```
 
 This will generate an executable JAR at the top level directory that can be used to execute AMF from the command line.
@@ -142,26 +162,13 @@ Using this JAR you can execute AMF by passing one of the following commands:
 
 An interactive section can be started using the `repl` command.
 
-If you want to parse any RAML dialect other than RAML 1.0, you can pass a list of dialects to be loaded in the parser through the `dialects` option.
+If you want to parse any AML dialect other than RAML 1.0, you can pass a list of dialects to be loaded in the parser through the `dialects` option.
 
 Refer to the usage of the application for additional commands and explanations.
 
-## Examples
-
-Go to [amf examples repository](https://github.com/mulesoft/amf-examples) There are examples for each one of the three usages and a *converter* project that add some UI on top of the library.
-
-## Validation
-
-Validation is one of the key features of AMF. Please check the following link to get more information:
-
-[Validation insights](./documentation/validation.md)
-
 ## AML Vocabulary
 
 The AML Vocabulary that could be found in this repository under the **vocabularies** directory has been migrated to the [amf metadata repository](https://github.com/aml-org/amf-metadata)
 
-## Want to learn more?
-[Click here for more documentation and playground](https://a.ml)
-
-## Want to contribute?
-If you are interested in contributing code to this project, thanks! Please [read and accept the Contributors Agreement](https://api-notebook.anypoint.mulesoft.com/notebooks#380297ed0e474010ff43). This should automatically create a Github issue with the record of your signature [here](https://github.com/mulesoft/contributor-agreements/issues). If for any reason, you do not see your signature there, please contact us.
+## Contributing
+If you are interested in contributing to this project, please make sure to read our [contributing guidelines](./CONTRIBUTING.md).

amf-client/shared/src/main/scala/amf/client/AMF.scala

@@ -13,7 +13,7 @@ import amf.client.validate.ValidationReport
 import amf.core.unsafe.PlatformSecrets
 import amf.plugins.document.webapi.validation.PayloadValidatorPlugin
 import amf.plugins.document.{Vocabularies, WebApi}
-import amf.plugins.features.AMFValidation
+import amf.plugins.features.AMFCustomValidation
 import amf.plugins.{document, features}
 import amf.{AMFStyle, Core, MessageStyle, ProfileName}
 
@@ -28,7 +28,7 @@ object AMF extends PlatformSecrets {
   def init(executionEnvironment: BaseExecutionEnvironment): ClientFuture[Unit] = {
     WebApi.register(executionEnvironment)
     Vocabularies.register()
-    AMFValidation.register()
+    AMFCustomValidation.register()
     amf.Core.registerPlugin(PayloadValidatorPlugin)
     amf.Core.init(executionEnvironment)
   }
@@ -158,5 +158,6 @@ object DocumentPluginsWrapper {
 
 @JSExportAll
 object FeaturesPluginsWrapper {
-  val AMFValidation: features.AMFValidation.type = features.AMFValidation
+  val AMFValidation: features.AMFValidation.type             = features.AMFValidation
+  val AMFCustomValidation: features.AMFCustomValidation.type = features.AMFCustomValidation
 }

amf-client/shared/src/main/scala/amf/client/commands/CommandHelper.scala

@@ -13,7 +13,7 @@ import amf.core.services.{RuntimeCompiler, RuntimeResolver, RuntimeSerializer}
 import amf.plugins.document.vocabularies.AMLPlugin
 import amf.plugins.document.webapi.validation.PayloadValidatorPlugin
 import amf.plugins.document.webapi.{Oas20Plugin, Oas30Plugin, Raml08Plugin, Raml10Plugin}
-import amf.plugins.features.validation.AMFValidatorPlugin
+import amf.plugins.features.validation.custom.AMFValidatorPlugin
 
 import scala.concurrent.{ExecutionContext, Future}
 

amf-client/shared/src/main/scala/amf/facades/Validation.scala

@@ -15,8 +15,9 @@ import amf.plugins.document.webapi.validation.PayloadValidatorPlugin
 import amf.plugins.document.webapi.{Oas20Plugin, PayloadPlugin, Raml08Plugin, Raml10Plugin, _}
 import amf.plugins.domain.shapes.DataShapesDomainPlugin
 import amf.plugins.domain.webapi.WebAPIDomainPlugin
-import amf.plugins.features.validation.model.ValidationDialectText
-import amf.plugins.features.validation.{AMFValidatorPlugin, CoreValidations}
+import amf.plugins.features.validation.custom.model.ValidationDialectText
+import amf.plugins.features.validation.custom.AMFValidatorPlugin
+import amf.plugins.features.validation.CoreValidations
 import amf.plugins.syntax.SYamlSyntaxPlugin
 import amf.validation.DialectValidations
 import amf.validations.{ParserSideValidations, PayloadValidations, RenderSideValidations, ResolutionSideValidations}

amf-client/shared/src/test/resources/cycle/raml10/abstract/external-rt.raml

@@ -0,0 +1,5 @@
+#%RAML 1.0 ResourceType
+usage: This resourceType should be used for any collection of items
+description: The collection of <<resourcePathName>>
+get:
+  description: Get all <<resourcePathName>>, optionally filtered

amf-client/shared/src/test/resources/cycle/raml10/abstract/external-trait.raml

@@ -0,0 +1,3 @@
+#%RAML 1.0 Trait
+usage: Apply this to any method that needs to be secured
+description: Some requests require authentication.

amf-client/shared/src/test/resources/cycle/raml10/abstract/rt-and-trait-definition.raml

@@ -10,6 +10,7 @@ traits:
         body:
           application/json:
             type: <<typeName>>
+  externalTrait: !include external-trait.raml
 
 resourceTypes:
   some-rt:
@@ -23,3 +24,4 @@ resourceTypes:
         201:
           headers:
             Location:
+  externalResourceType: !include external-rt.raml

amf-client/shared/src/test/resources/cycle/raml10/abstract/rt-link-emission.yaml

@@ -0,0 +1 @@
+!include external-rt.raml

amf-client/shared/src/test/resources/cycle/raml10/abstract/trait-link-emission.yaml

@@ -0,0 +1 @@
+!include external-trait.raml

amf-client/shared/src/test/resources/cycle/raml10/annotation-type/annotation-type-emission.yaml

@@ -0,0 +1,2 @@
+displayName: some name
+type: nil | string

amf-client/shared/src/test/resources/cycle/raml10/annotation-type/annotation-type-lib-link-emission.yaml

@@ -0,0 +1 @@
+someLib.annotationInLib

amf-client/shared/src/test/resources/cycle/raml10/annotation-type/annotation-type-link-emission.yaml

@@ -0,0 +1 @@
+!include external-annotation-type.raml

amf-client/shared/src/test/resources/cycle/raml10/annotation-type/api.raml

@@ -0,0 +1,15 @@
+#%RAML 1.0
+
+title: test with references
+version: "0.1"
+uses:
+  someLib: lib.raml
+
+annotationTypes:
+  annotA:
+    displayName: some name
+    type: nil | string
+
+  annotB: !include external-annotation-type.raml
+
+  annotC: someLib.annotationInLib

amf-client/shared/src/test/resources/cycle/raml10/annotation-type/external-annotation-type.raml

@@ -0,0 +1,3 @@
+#%RAML 1.0 AnnotationTypeDeclaration
+
+type: string

amf-client/shared/src/test/resources/cycle/raml10/annotation-type/lib.raml

@@ -0,0 +1,5 @@
+#%RAML 1.0 Library
+
+annotationTypes:
+  annotationInLib:
+    type: string

amf-client/shared/src/test/resources/cycle/raml10/security-scheme/api.raml

@@ -0,0 +1,16 @@
+#%RAML 1.0
+title: test
+version: 1
+baseUri: https://api.domain.com/{version}
+securitySchemes:
+    valid: !include external-scheme.raml
+    inlined:
+      type: OAuth 2.0
+      settings:
+        authorizationUri: https://www.domain.com/1/oauth2/authorize
+        accessTokenUri: https://api.domain.com/1/oauth2/token
+        authorizationGrants: [ authorization_code, implicit, client_credentials, password, 'urn:ietf:params:oauth:grant-type:saml2-bearer' ]
+
+/users:
+  get:
+    securedBy: [valid, null]

amf-client/shared/src/test/resources/cycle/raml10/security-scheme/external-scheme.raml

@@ -0,0 +1,6 @@
+#%RAML 1.0 SecurityScheme
+type: OAuth 2.0
+settings:
+  authorizationUri: https://www.domain.com/1/oauth2/authorize
+  accessTokenUri: https://api.domain.com/1/oauth2/token
+  authorizationGrants: [ authorization_code, implicit, client_credentials, password, 'urn:ietf:params:oauth:grant-type:saml2-bearer' ]

amf-client/shared/src/test/resources/cycle/raml10/security-scheme/security-scheme-created-link-emission.yaml

@@ -0,0 +1 @@
+!include someName.raml

amf-client/shared/src/test/resources/cycle/raml10/security-scheme/security-scheme-link-emission.yaml

@@ -0,0 +1 @@
+!include external-scheme.raml

amf-client/shared/src/test/resources/cycle/raml10/type/data-type.raml

@@ -0,0 +1,5 @@
+#%RAML 1.0 DataType
+
+type: object
+properties:
+  c: string

amf-client/shared/src/test/resources/cycle/raml10/type/refs-with-entry-emission.yaml

@@ -0,0 +1,8 @@
+type: object
+properties:
+  ref-to-fragment-entry:
+    type: !include data-type.raml
+  ref-to-declared-entry:
+    type: other-type
+  ref-to-fragment: !include data-type.raml
+  ref-to-declared: other-type

amf-client/shared/src/test/resources/cycle/raml10/type/refs-with-entry.raml

@@ -0,0 +1,14 @@
+#%RAML 1.0
+title: simple object
+types:
+  other-type:
+    type: number
+  root:
+    type: object
+    properties:
+      ref-to-fragment-entry:
+        type: !include data-type.raml
+      ref-to-declared-entry:
+        type: other-type
+      ref-to-fragment: !include data-type.raml
+      ref-to-declared: other-type

amf-client/shared/src/test/resources/parser/array-type-expressions/base-type-array.expanded.jsonld

@@ -75,7 +75,7 @@
     ],
     "http://a.ml/vocabularies/document#version": [
       {
-        "@value": "2.1.1"
+        "@value": "2.1.2"
       }
     ],
     "http://a.ml/vocabularies/document#root": [

amf-client/shared/src/test/resources/parser/array-type-expressions/base-type-array.flattened.jsonld

@@ -70,7 +70,7 @@
       "http://a.ml/vocabularies/document#encodes": {
         "@id": "file://amf-client/shared/src/test/resources/parser/array-type-expressions/base-type-array.raml#/web-api"
       },
-      "http://a.ml/vocabularies/document#version": "2.1.1",
+      "http://a.ml/vocabularies/document#version": "2.1.2",
       "http://a.ml/vocabularies/document#root": true
     },
     {

amf-client/shared/src/test/resources/parser/array-type-expressions/matrix-type-array.expanded.jsonld

@@ -75,7 +75,7 @@
     ],
     "http://a.ml/vocabularies/document#version": [
       {
-        "@value": "2.1.1"
+        "@value": "2.1.2"
       }
     ],
     "http://a.ml/vocabularies/document#root": [