[ZEN-3765] Example Data: Validate examples against OpenAPI Schema Object #486
Conversation
…nt with OAS conformance suite Add preliminary support for example validation in OpenAPI3 and Swagger.
…nt with OAS conformance suite This adds normalization of schemas present in Swagger/OpenAPI. When an example is detected, the object being it's schema is passed through the normalizer to be transformed into a single object where all references are resolved. This also extends the validation preferences for Swagger/OpenAPI to add an option to enable example validation. This option is set to false by default.
|
@tedepstein Ready to review, see PR https://github.com/ModelSolv/RepreZen/pull/1876 for build Example validator now normalizes schemas in Swagger/OpenAPI specs to resolve any references before running the validation. Added Validation Preferences
|
|
Needs to be modified to deal with recursive (circular) references. These cannot be inlined, they must be localized. See the Normalizer documentation and codebase for a full explanation. We also need the preference setting to be enabled by default. |
…nt with OAS conformance suite Add extension point.
…nt with OAS conformance suite Removing example validators as they will be implemented as extensions inside API Studio. Also modified the preference pages and initialisers so that that validation extensions can add and initialise their own values.
…nt with OAS conformance suite KZOE and API Studio both provide their own version of json-schema-validator and this is causing an exception to be thrown when a validator extension is created in API Studio and called from KZOE. To avoid this issue, all call to json-schema-validator should be done through the KZOE class `JsonSchemaValidator` that will execute the validation without exposing any json-schema-validator API.
…nt with OAS conformance suite Add the extension point `com.reprezen.swagedit.preferences` that allow clients to provide their own preferences to KaiZen editors.
…nt with OAS conformance suite Add missing change in plugin.xml
…nt with OAS conformance suite Fix wrong ID.
|
Final changes to this PR are about adding extensions points to KZOE, so that clients (namely API Studio) can provide their own validation processors. The 2 extensions points are:
|
|
Should be extended to validate |
| try { | ||
| return (T) e; | ||
| } catch (ClassCastException ex) { | ||
| return null; |
There was a problem hiding this comment.
A log would help trace the problem. .filter(Objects:: nonNull) will filter out null-values and the only way to identify the problem would be to debug this code. Debugging lambdas in Eclipse is not easy though.
| try { | ||
| return jsonSchema.validate(document, true); | ||
| jsonSchema = factory.getJsonSchema(schema, schemaPointer); | ||
| } catch (ProcessingException e) { | ||
| Activator.getDefault().logError(e.getLocalizedMessage(), e); | ||
| return null; |
There was a problem hiding this comment.
You should return an empty Set or document warning the consumer that the method may return a null value if things go wrong.
NOTE: See updated comments on ModelSolv/RepreZen/pull/1876QA Test Results: FailThe feature generally works, and we can merge the PR. But it has at least one unrecoverable error, meaning there are valid OpenAPI documents that cannot be used because they get a false-negative (error) result from the validator. There are other issues where the validation yields false-positives in some cases (i.e. validation doesn't check what it should be checking) and false-negatives in other cases (raising errors where the schema is valid). Pending further testing, we can merge the PR. (Further comments to follow here...) Tests were created using examples within the following template, by adding examples to the request and response of the Each test case will describe modifications that were made to this template to get the reported results. Referenced Object Schema: PassChanging the example to break the rules yielded expected schema validation errors. Places a single error marker on the example line: Changing to multiple examples yields the same error message on the second example (array element), as expected: Inline Object Schema: PassWorks as expected with the schema moved inline: Note: In the above example, we added the If the errors are corrected (first two items), the error marker changes to a warning marker, and showing only the 'format attribute "int64" not supported' warning message. This is not a critical problem, but it would be best to filter those warnings out, when they occur on standard OpenAPI formats. Inline or referenced allOf Schema: FailIn the OpenAPI test template, the The Eclipse error log shows entries like this: Referenced Example Object: FailAdding a reusable Example Object here: And referencing it here: Puts an error marker on It seems the validation code does not try to resolve the Object Array Schema: PassThis raises validation errors as expected: Referenced array schema also works, with this ... and this reference to We see the expected validation errors. Primitive Array Schema: PassThis works: Primitive Schema: PassThis works: Schema Object
|
tedepstein
changed the title
See update #486 (comment)
Add preliminary support for example validation in OpenAPI3 and Swagger.