Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add REST catalog spec to iceberg-core.jar #6672

Closed
wants to merge 1 commit into from

Conversation

snazy
Copy link
Member

@snazy snazy commented Jan 26, 2023

Having the OpenAPI spec file used by a specific iceberg-core jar helps a lot.
This enables generating code from the spec by using the spec in the iceberg-core jar, without having to pull it "manually" from GH.

@github-actions github-actions bot added the build label Jan 26, 2023
@Fokko
Copy link
Contributor

Fokko commented Jan 26, 2023

@snazy Out of curiosity, do you use the open api spec to directly generate code? In Python, we tried to do it as well, but the structure was too complex in the end, and we used the generated code as a start. For example, the types can be both a string (timestamp, fixed[12], etc) and a JSON dictionary ({"type": "map", ...}).

@snazy
Copy link
Member Author

snazy commented Jan 27, 2023

I think it's useful for two purposes:

  1. to have the spec together with the code, to know how the spec for the code look(ed|s) like
  2. to be able to generate some code - at least as a starter

Types are "an issue", yes. Not sure how much it would affect generated code in general / across all languages. Jackson for example handles a "generic JsonNode" (which can be "anything") perfectly fine - I imagine that it would work similar for a "generic j.l.Object".

@haizhou-zhao
Copy link
Contributor

@snazy I ran into the same issue as @Fokko mentioned above (I summarized my problem here: #6798). In addition to @Fokko's question, I want to know if you are writing some sort of model converter to convert, for example, between the openAPI generated Schema object and the original iceberg Schema object. If not, would it be possible to know better how you are using the open API generated models? Thank you!

@snazy
Copy link
Member Author

snazy commented Feb 14, 2023

I still think, that a OpenAPI specification, if it exists, should technically match the implementation and vice versa. But that's actually not the case at the moment and code generation is not possible (commented here).

There is no good JAX-RS Java code generator out there, but the ones available for node and python seem to work reasonably well.

Basically all projects that have an OpenAPI spec do publish it at least along with their published artifacts and/or via platforms like swagger-hub, from which others can take it or inspect it.

Copy link

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

@github-actions github-actions bot added the stale label Aug 24, 2024
Copy link

This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

@github-actions github-actions bot closed this Sep 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants