-
Notifications
You must be signed in to change notification settings - Fork 9.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal: Expand OpenAPI to include RPC APIs #801
Comments
RPC semantics is quite impossible for 3.0 but protobufs as payload may be doable and would certainly help out. Will look at your comments about the missing fields in our extended json schema for supporting protos. |
@fehguy "quite impossible" because it goes beyond the standard CRUD REST verbs or is there some other reason it's untenable? |
@cfineman because we're closed on major features for 3.0. |
acknowledged |
It might be a good fit for the AsyncAPI spec, given it's message-driven instead of request/response. |
If we look at RPC APIs over JSON/YAML, I feel the E.g. if we look at JSON RPC 2.0 for instance, requests have two main formats.
Where a large collection of different methods exist. The responses in JSON RPC are of format:
Where the schema of the result (and in principal error) are generally determined by the request method. At least for the first variant, there is no problem describing this as a set of The only missing feature for a correct implementation with this approach, is to be able to provide a similar discriminator object for the response type that instead of switching on a field in the response payload, switches on a field at a particular path in the request (A discriminator object for batch requests is probably not something we could pull of though). I am not stating that this is the best way of documenting a RPC API. But I think it's important that the scope should be to describe the HTTP Protocol for the RPC API, and not the RPC API in general (which could be protocol agnostic). |
Hey @timburks, the goal of OpenAPI Specification has swapped from REST to HTTP so that's one roadblock gone. #1946 I know protobuf support is wanted by @darrelmiller and it's come up several times at Stoplight. Maybe it's time to progress. 😎 |
You are basically over complicating and mixing FUNCTION with DATA; the idea is to abstract these two. These docs are meant to reference data that is shared across the architecture with request/response for proper checking of endpoint compliance. Adding function names complicates the referencing; not all resources/services in the architecture will have these functions available to them... but they will have the names of all params as sent in the request/response. Thus it is better to create a common object in the doc that is a reference to all variables sent/returned. By creating the OBJECT in the spec, you can easily create all endpoints in said doc by referencing parm KEY. This allows for JOINED DATABASE QUERIES as well as other complex datasets to EASILY be referenced and checked at all points in the architecture. |
Hey team. I come from a web2 background where "REST is the way" and have always leaned on OpenAPI for defining our API specifications. I now work at a blockchain infrastructure provider where our APIs are mostly JSON-RPC with some REST here and there. I'd love to leverage OAS for our documentation and may even be able to contribute some development resources to the effort if we could come together and agree on a path forward. |
@Cahl-Dee, OpenAPI isn't particularly suitable for describing RPC APIs, and for JSON RPC 2.0 in particular, where all requests generally happen against the same endpoint, you are probably better off looking at https://open-rpc.org/ and associated tools. |
@smyrman "where all requests generally happen against the same endpoint"??? The endpoint is 'controller/method'; for each separate controller/method, its a separate endpoint . This is where the request/response meet on the backend and thus is the central version of truth. This is why all BINDINGS in an API build are put into the controllers. What are you talking about??? |
I mean all requests for JSON RPC 2.0 generally (or often) happen against the same URL / URL path. E.g. all method calls might be done against a single URL |
@smyrman Thats because they are sending 'method' in the passed data (which is unusual for RPC); look at how JSON-RPC passes data. gRPC encodes the controller/method in the URI just like all other RPC. JSON-RPC seems to be odd in this respect. ex. http:// localhost/controller/method The reason it is done in the URI is because request uri will ALWAYS be passed with redirects in your network and cannot be overwritten. The request variables can be removed/overwritten. ALSO... would point out ticket stated 'RPC' so is not specific to any one RPC implementation so as long as you stick with a controller/method approach, this will work with JSON-RPC, gRPC and pretty much all RPC |
@smyrman I'm not familiar with the inner workings of OAS so I have no clue how big of a change this would be, but are methods and endpoints really that different of concepts? Has there been, or is there openness to discussing how the primitive leveraged by OAS could be abstracted to cover both? |
@Cahl-Dee Thats how it is used in Spring and Springboot https://www.baeldung.com/spring-rest-openapi-documentation. So I don't see how it isn't good at it when it is doing it fine. That's an RPC implementation. |
+1 use http:// localhost/controller/method can mapped to a unique server implementation just like http endpoint.
|
I can pass the uri with internal redirect (aka forwards) which are handled
at the front controller. Any framework can do this as it is part of the
HTTP protocol.
Owen Rubel
***@***.***
…On Thu, Jul 13, 2023 at 12:47 AM YiJie ***@***.***> wrote:
@smyrman <https://github.com/smyrman> Thats because they are sending
'method' in the passed data (which is unusual for RPC); look at how
JSON-RPC passes data.
gRPC encodes the controller/method in the URI just like all other RPC.
JSON-RPC seems to be odd in this respect.
ex. http:// localhost/controller/method ie http://localhost/user/getByName
The reason it is done in the URI is because request uri will ALWAYS be
passed with redirects in your network and cannot be overwritten. The
request variables can be removed/overwritten.
ALSO... would point out ticket stated 'RPC' so is not specific to any one
RPC implementation so as long as you stick with a controller/method
approach, this will work with JSON-RPC, gRPC and pretty much all RPC
+1 use http:// localhost/controller/method can mapped to a unique server
implementation just like http endpoint.
btw i have another question, rpc methods do not have http verbs and http
status, so we seems like need a new tag like paths and webhooks to describe
the rpc methods, otherwise it seems to be odd when use *paths* to
describe the rpc methods, it always use 'post' verb and always defined
request in 'requestBody' and always defined response in '200' status
/controller/method: post: requestBody: - name: name xxx - name: key xxx
responses: '200': description: An array of products schema: type: array
items: $ref: '#/definitions/RPCstruct'
—
Reply to this email directly, view it on GitHub
<#801 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACDCTGL337HFV7JVO3FKRDXP6RYXANCNFSM4CRKALXQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
OAS 4 Moonwalk will support synchronous HTTP RPC APIs under the big tent principle. Please join the discussions in that repository if you are still interested. For asynchronous RPC, there is AsyncAPI. |
So you are saying XML-RPC and SOAP which also follow RPC patterns are
equally not RPC?
Owen Rubel
***@***.***
…On Thu, Jul 13, 2023 at 12:47 AM YiJie ***@***.***> wrote:
@smyrman <https://github.com/smyrman> Thats because they are sending
'method' in the passed data (which is unusual for RPC); look at how
JSON-RPC passes data.
gRPC encodes the controller/method in the URI just like all other RPC.
JSON-RPC seems to be odd in this respect.
ex. http:// localhost/controller/method ie http://localhost/user/getByName
The reason it is done in the URI is because request uri will ALWAYS be
passed with redirects in your network and cannot be overwritten. The
request variables can be removed/overwritten.
ALSO... would point out ticket stated 'RPC' so is not specific to any one
RPC implementation so as long as you stick with a controller/method
approach, this will work with JSON-RPC, gRPC and pretty much all RPC
+1 use http:// localhost/controller/method can mapped to a unique server
implementation just like http endpoint.
btw i have another question, rpc methods do not have http verbs and http
status, so we seems like need a new tag like paths and webhooks to describe
the rpc methods, otherwise it seems to be odd when use *paths* to
describe the rpc methods, it always use 'post' verb and always defined
request in 'requestBody' and always defined response in '200' status
/controller/method: post: requestBody: - name: name xxx - name: key xxx
responses: '200': description: An array of products schema: type: array
items: $ref: '#/definitions/RPCstruct'
—
Reply to this email directly, view it on GitHub
<#801 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACDCTGL337HFV7JVO3FKRDXP6RYXANCNFSM4CRKALXQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
The OpenAPI 2.0 Specification states that its goal is “to define a standard, language-agnostic interface to REST APIs”. REST is an extremely popular style for implementing APIs that run over HTTP and related protocols (HTTPS, HTTP/2, QUIC). But other transport mechanisms are common and APIs are often defined in ways that correspond to procedure calls. In some ways, procedure calls are more fundamental, as REST APIs are often implemented by systems that convert REST requests into procedure calls.
At times it is desirable to use remote procedure calls (RPCs) as the primary form of an API. RPC systems allow expression of semantics that go outside the range of the HTTP verbs used by REST. RPC systems map naturally to the function calls that are used to implement most software systems, and RPC provides a design pattern that can be use to guide system implementations. The RPC abstraction is also much older than REST, and new systems such as gRPC and Thrift show its enduring usefulness.
With this in mind, we think it would be beneficial to expand the scope of the OpenAPI specification to include remote procedure call semantics. Here we propose an expanded specification that builds on our experience using RPCs to connect very large distributed systems while being general enough to apply to a broad range of variations.
Background
Protocol Buffers
We give special consideration to the Protocol Buffer message representation. At Google, Protocol Buffers are used to send and receive tens of billions of messages each second. Protocol Buffers are also used for messages in the open-source gRPC framework and other more specialized RPC frameworks [1] [2] [3].
Protocol Buffers are typically represented in a binary form, and the latest version (proto3) also allows mapping to representations in JSON and YAML formats. Messages can be generally treated as collections of typed fields, just as they are in other serialization formats including Thrift, Avro, and Cap’n Proto. Thus message description can be treated similarly for all of these representations, including the JSON and YAML that OpenAPI uses to describe request and response objects.
Protocol Buffer interfaces are defined using a special description language (
.proto
) that describes messages and the services that use them. This language is different from the JSON and YAML used for OpenAPI specifications. To keep the specification and tooling simple, this proposal replaces this language with extensions to OpenAPI that carry the same meanings. It seeks to express as much as possible of the.proto
language within OpenAPI so that if desired,.proto
files can be converted into OpenAPI RPC descriptions and vice versa.RPC Elements
Messages
Remote procedure calls are described in terms of the messages that they send and receive. Messages contain fields that have names, types, and other attributes, often including an integer field number that is used in message serialization.
Services
In RPC terminology, "Services" are collections of remote procedure calls that send and receive messages. Each call has a single input and output message, and some RPC implementations will allow input and output messages to be streamed in a single call. Thus each remote procedure call is described by its name, input type, output type, and whether or not the input and output messages are streamed.
Example
Here we show an example
.proto
description of an API. For illustration, we've specified theGetBook
RPC as a streaming call that accepts a stream of book requests and returns a stream of books. This is indicated with thestream
keyword that appears before the request and response types.Open API Representation
Messages
RPC messages are described in the OpenAPI
definitions
section. To these we add a few new fields to provide protocol buffer semantics. New fields are prefixedx-
but in the accepted proposal, we would expect allx-
prefixes to be deleted.Properties of a message correspond to fields in a protocol buffer message.
x-field-number
(orfieldNumber
) is a required integer property that associates a unique field number with each property.x-repeated
is an optional boolean property (by default false) that, when true, indicates that a field may occur more than once.Here we show an example OpenAPI representation of the messages in the Bookstore API.
Services
Services are a distinct new entity that we represent using the
x-services
key at the top level of the OpenAPI description. This allows an API to include both RPC and REST representations side-by-side.Services are described by their name and the procedures they contain. Procedures are described by the objects they accept and return. Streamed values are indicated with
x-streaming
, an optional boolean property (by default false).Here is an example OpenAPI representation of the services in the Bookstore API (in this case, a single service named "Bookstore").
Discussion
Our intent is to make it possible to write tools that convert back-and-forth between
.proto
and OpenAPI representations and to build RPC flows that completely replace.proto
inputs with OpenAPI RPC specifications.OpenAPI representations are more verbose than
.proto
, but are more amenable to automated processing and custom editors like swagger-editor. Our hope is that this representation will lead more editors and other tools to support RPC APIs.Types in OpenAPI and Protocol Buffers
Protocol Buffers contain many finely-differentiated scalar types, while the OpenAPI spec uses general types such as number and integer. In OpenAPI, an additional
format
field supplements the type field with additional representation detail, so our proposal uses this field to include the full name of the corresponding Protocol Buffer type.type
field valueformat
field valueIn the above table, the
bool
,string
, andbytes
types directly correspond to OpenAPI types and need no additional detail in theformat
field. Thegroup
type in.proto
is deprecated and unsupported, and themessage
type corresponds to inclusion of another message, which is represented in OpenAPI with the$ref
property.The
enum
type is represented in.proto
as an integer and in OpenAPI as a string. This difference is unresolved in this proposal.Gaps between OpenAPI and Protocol Buffers
There are some Protocol Buffer features that aren’t yet covered by this proposal. Gaps that are significant and unresolved may be addressed in future proposals.
Default values
Default field values could be represented with the existing OpenAPI
default
property, but in some message representations (such asproto3
, default values are specified for each type and are not modifiable.Enumerations
Enumerations are represented with strings in OpenAPI and integers in
.proto
. Resolution of this is a high priority future proposal.Maps
The
.proto
format allows fields to have map types which include type specifications for map keys and values. When serialized, these maps are represented with special automatically-created messages.Here we omit further discussion of maps, leaving it as a more general question about the OpenAPI Specification.
Extensions
Extensions are defined in
proto2
and allow additional fields to be added to messages and for ranges of field numbers to be reserved for third-party extensions. Extensions are not supported inproto3
and are omitted from this proposal.Nested messages
The
.proto
language allows messages to be defined inside other messages. Nested types can’t be written directly in OpenAPI, but we can define messages with hierarchical names similar to the ones that would be implied for nested.proto
messages.Options
In
.proto
, options are predefined annotations that can be added in various places in a.proto
file. Common options configure code generators (by specifying package names or class name prefixes) or map RPC procedures to REST methods. Options are not addressed here but are a priority for inclusion in a future proposal.Oneof
In
.proto
, theoneof
statement groups fields to indicate that only one of the grouped fields can be included in a message. There is no corresponding concept in OpenAPI 2.0, but this appears to be addressed by a pending pull request.Packages and API Versions
.proto
descriptions optionally include a package name. This has a similar purpose as thebasePath
field in the OpenAPI root object and we suggest that either a new field namedpackage
or the existingbasePath
be used for this.API Versions are commonly indicated by the last segment in a package name (segments are separated by periods). When the last segment looks like a version name (beginning with a 'v' and following with a possibly-dotted number), it is used as the version name. We omit this convention from this proposal and leave the representation of API versions to the existing
version
field of theinfo
object.The text was updated successfully, but these errors were encountered: