Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add OpenAPI route #1960

Merged
merged 12 commits into from
Oct 18, 2023
Merged

Conversation

nasdf
Copy link
Member

@nasdf nasdf commented Oct 11, 2023

Relevant issue(s)

Resolves #510

Description

This PR adds an HTTP endpoint that returns an OpenAPI specification for DefraDB. The definitions are part code generation and part hand written. This should work well for adding examples and more documentation in the future.

Tasks

  • I made sure the code is well commented, particularly hard-to-understand areas.
  • I made sure the repository-held documentation is changed accordingly.
  • I made sure the pull request title adheres to the conventional commit style (the subset used in the project can be found in tools/configs/chglog/config.yml).
  • I made sure to discuss its limitations such as threats to validity, vulnerability to mistake and misuse, robustness to invalidation of assumptions, resource requirements, ...

How has this been tested?

View endpoint in browser localhost:9181/api/v0/openapi.

Double checked with OpenAPI validator tool.

Specify the platform(s) on which this was tested:

  • MacOS

@nasdf nasdf added documentation Improvements or additions to documentation area/api Related to the external API component labels Oct 11, 2023
@nasdf nasdf added this to the DefraDB v0.8 milestone Oct 11, 2023
@nasdf nasdf self-assigned this Oct 11, 2023
@nasdf nasdf requested a review from a team October 11, 2023 23:05
@codecov
Copy link

codecov bot commented Oct 11, 2023

Codecov Report

Attention: 28 lines in your changes are missing coverage. Please review.

Comparison is base (0efe835) 74.13% compared to head (e645cdd) 74.86%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #1960      +/-   ##
===========================================
+ Coverage    74.13%   74.86%   +0.73%     
===========================================
  Files          244      246       +2     
  Lines        23508    24298     +790     
===========================================
+ Hits         17427    18189     +762     
- Misses        4880     4899      +19     
- Partials      1201     1210       +9     
Flag Coverage Δ
all-tests 74.86% <96.79%> (+0.73%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
http/handler_ccip.go 95.12% <100.00%> (+6.55%) ⬆️
http/handler_collection.go 58.46% <100.00%> (+31.72%) ⬆️
http/handler_lens.go 58.29% <100.00%> (+45.93%) ⬆️
http/handler_p2p.go 69.38% <100.00%> (+34.02%) ⬆️
http/handler_store.go 78.68% <100.00%> (+18.20%) ⬆️
http/handler_tx.go 73.33% <100.00%> (+27.57%) ⬆️
http/openapi.go 97.14% <97.14%> (ø)
http/server.go 93.79% <57.14%> (-1.62%) ⬇️
cli/start.go 25.62% <0.00%> (-0.32%) ⬇️
http/router.go 68.00% <68.00%> (ø)
... and 1 more

... and 9 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0efe835...e645cdd. Read the comment docs.

Copy link
Contributor

@AndrewSisley AndrewSisley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I have a couple of questions. Please let me know if there is any rush on this one (otherwise it looks like it will cause no trouble if it sits here for a couple of days)

"net/http"

"github.com/getkin/kin-openapi/openapi3"
"github.com/getkin/kin-openapi/openapi3gen"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Did you have a good shop around for alternatives? What made you pick this library?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried the go-swagger library as well, but the documentation was lacking. There are a few things I liked about this library.

  • supports openapi 3 (go-swagger is on 2)
  • struct schema generation works very well
  • request & response validation for testing
  • type safety when writing the spec in go

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice :) Thanks for listing those Keenan :) v3 vs v2 is probably reason enough to use this lib (and not just for the support, but I see it as a sign of better lib health)

http/openapi.go Outdated
},
}

func init() {
Copy link
Contributor

@AndrewSisley AndrewSisley Oct 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: We will now have 3 different files for doing this, cli.go, handler.go, and openapi.go. Working with any one of them in isolation is error prone, working with all of them distributed as they currently are is even more so. It also feels quite important to me that they all follow the same structure (as much so as is reasonable).

Could you perhaps spend some time now looking at pulling these 3 files out to their own shared package? Called api or something. And provide us with a single location in which we have to maintain these?

Longer term (I don't want to pressure you now) we could perhaps unify them further so that instead of splitting the functions in that new package by client, we could instead split them by function, for example (very roughy):

func BindTxCreate() {
  // setup cli
  MakeTxCreateCommand(cfg)
  
  // setup http
  tx.Post("/", tx_handler.NewTxn)
  
  // setup openapi 
  txnCreate := openapi3.NewOperation()
  txnCreate.AddParameter(txnReadOnlyQueryParam)
  txnCreate.AddResponse(200, txnCreateResponse)
  txnCreate.AddResponse(400, errorResponse)
}

func BindPatchSchema() {
... etc

This would also hopefully further open up the possibility of more magic refactorings in the future, that may remove having to manually map it 3 times.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I've found a solution that will make the http routes simpler. I've moved the route registration to bind functions which now require OpenAPI definitions for each route. This should help enforce that all endpoints have basic descriptions. Validating client requests in the tests will be the final step to enforce the correctness of the definitions.

@AndrewSisley
Copy link
Contributor

question: I am assuming that the validate call doesn't actually assert that the http endpoints conform to the given specification, do you have any plans for adding tests to make sure the two don't diverge?

@nasdf
Copy link
Member Author

nasdf commented Oct 12, 2023

question: I am assuming that the validate call doesn't actually assert that the http endpoints conform to the given specification, do you have any plans for adding tests to make sure the two don't diverge?

The specification validation only checks if the required fields exist. The test functions in the library can check if the paths and params are valid. I plan on getting those working shortly after.

Copy link
Collaborator

@fredcarle fredcarle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I look forward to seeing the generation of SDKs with this.

Copy link
Member

@shahzadlone shahzadlone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is cool stuff, Cheers

Copy link
Contributor

@AndrewSisley AndrewSisley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM :) Glad you managed to unify the openAPI and http bindings :)

@nasdf nasdf merged commit 28f207d into sourcenetwork:develop Oct 18, 2023
29 checks passed
nasdf added a commit to nasdf/defradb that referenced this pull request Oct 18, 2023
Resolves sourcenetwork#510

This PR adds an HTTP endpoint that returns an OpenAPI specification for
DefraDB. The definitions are part code generation and part hand written.
This should work well for adding examples and more documentation in the
future.

- [x] I made sure the code is well commented, particularly
hard-to-understand areas.
- [x] I made sure the repository-held documentation is changed
accordingly.
- [x] I made sure the pull request title adheres to the conventional
commit style (the subset used in the project can be found in
[tools/configs/chglog/config.yml](tools/configs/chglog/config.yml)).
- [x] I made sure to discuss its limitations such as threats to
validity, vulnerability to mistake and misuse, robustness to
invalidation of assumptions, resource requirements, ...

View endpoint in browser `localhost:9181/api/v0/openapi`.

Double checked with OpenAPI validator tool.

Specify the platform(s) on which this was tested:
- MacOS
shahzadlone pushed a commit to shahzadlone/defradb that referenced this pull request Feb 23, 2024
## Relevant issue(s)

Resolves sourcenetwork#510 

## Description

This PR adds an HTTP endpoint that returns an OpenAPI specification for
DefraDB. The definitions are part code generation and part hand written.
This should work well for adding examples and more documentation in the
future.

## Tasks

- [x] I made sure the code is well commented, particularly
hard-to-understand areas.
- [x] I made sure the repository-held documentation is changed
accordingly.
- [x] I made sure the pull request title adheres to the conventional
commit style (the subset used in the project can be found in
[tools/configs/chglog/config.yml](tools/configs/chglog/config.yml)).
- [x] I made sure to discuss its limitations such as threats to
validity, vulnerability to mistake and misuse, robustness to
invalidation of assumptions, resource requirements, ...

## How has this been tested?

View endpoint in browser `localhost:9181/api/v0/openapi`.

Double checked with OpenAPI validator tool.

Specify the platform(s) on which this was tested:
- MacOS
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/api Related to the external API component documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

HTTP API to offer OpenAPI document
4 participants