Add an OpenAPI doc for top level endpoints

[blink1073]

We have an API file following the OpenAPI spec, but it does not cover all of the URLs.

We should have another top level doc for the other endpoints, such as /file, and the directory handlers added in #251

[bollwyvl]

This would play really very nicely in the broader ecosystem. It would be possible to generate clients for any number of languages, and could be used in robust property-based testing.

[goanpeca]

Could we try to "automate" checking that this file is in sync ?

[bollwyvl]

Sure, if the OA spec is complete enough, schemathesis can automate testing, though things like file uploads are kinda hard, I'd imagine.

In an ideal future world, I'd rather an OA spec was The Contract between an (even internal) plugin and the server. There is a python implementation of this approach, but it tries to do a lot, and work with a lot of different servers. Something like FastAPI takes a different, opinionated approach, but generated specs, used naively, doen't actually help you document/test your app over time. But if you generate and check in the spec regularly (e.g. as part of the build), and test against the spec, you get many of the same benefits.

[goanpeca]

In an ideal future world, I'd rather an OA spec was The Contract between an (even internal) plugin and the server.

Yeah I could imagine that being sweet :)

[Stranger6667]

Hi @bollwyvl :)

Sure, if the OA spec is complete enough, schemathesis can automate testing, though things like file uploads are kinda hard, I'd imagine.

We have it on the roadmap in Schemathesis, let me know how much generating different file uploads would help and I can prioritize this feature :) Currently, Schemathesis generates random bytes for file uploads

[bollwyvl]

hi @Stranger6667! impressive response time!

I didn't meant to suggest Schemathesis was deficient in this manner, just that it's hard to put the idea of an upload into examples, and so can't be entirely captured in a spec. It seems like one of those cases that would be fine and natural for which to have to build a custom strategy. Doing any hypothesis-backed testing would be better than where we are now: this repo is still dealing with architectural/emotional/engineering trade-offs.

For Jupyter project in general, and for which a OA still has shortcomings, is websockets/zmq messages... I've been meaning to build some property-based tester for things like jupyter kernel messages and language server protocol stuff... so many things to test!

[Stranger6667]

I didn't meant to suggest Schemathesis was deficient in this manner, just that it's hard to put the idea of an upload into examples, and so can't be entirely captured in a spec. It seems like one of those cases that would be fine and natural for which to have to build a custom strategy. Doing any hypothesis-backed testing would be better than where we are now: this repo is still dealing with architectural/emotional/engineering trade-offs.

Aha, got it :) I plan to take a look at this direction in the next few weeks or so. Feel free to ping me if you need any support from the Schemathesis side.

For Jupyter project in general, and for which a OA still has shortcomings, is websockets/zmq messages... I've been meaning to build some property-based tester for things like jupyter kernel messages and language server protocol stuff... so many things to test!

Just wondering if you are aware of AsyncAPI? It seems like a neat approach to describe message-based communication patterns. It also based on Open API, which is good since Schemathesis already knows how to generate data for it :)

[bollwyvl]

Yeah, have seen AA... but given that neither JKM/LSP even publish canonical JSON schema specifications, and represent pretty involved nested state machines with parallel states in n languages, and have (in)formal extension mechanisms... it would be a big lift to test, or at least believe that the tests were testing what you cared about. I'm thinking something like scxml would actually be needed, coupled with a proper message spec for the data model definition. So kinda next to - get a PhD on my TODO.