Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use JSONSchema $schema for dbt contracts (via dataclasses + jsonschema-dataclasses) #1447

Closed
beckjake opened this issue May 8, 2019 · 3 comments

Comments

@beckjake
Copy link
Contributor

beckjake commented May 8, 2019

Feature

dbt should use the $schema and $ref tags of JSONSchema instead of the current deep_merge and manually-referencing dicts.

Feature description

We should convert our json schema contracts to have $schema values pointing at publicly accessible URLs, use the $ref feature to link behavior of our internal JSON schemas, and modify our APIObject and its descendants to take advantage of it.

We can start by specifying just the outermost "layers" of the schema and add the references as we go along. We should do something like iglu uses and define a decent versioned self-describing format. An example iglu schema is here

We should support namespacing, and give dbt-core a separate namespace from any plugins (like adapters!) or dbt-cloud.

We also will want to specify credential contracts on a per-adapter basis, and adapter-specific configurations at some point (currently we just allow anything).

Serialization should include the schema value so we can try to do deserialization into dbt objects based on the schema.

Who will this benefit?

Anyone consuming dbt json output, dbt Cloud, developers

@cmcarthur cmcarthur added this to the Louisa May Alcott milestone May 29, 2019
@cmcarthur cmcarthur changed the title use JSONSchema $schema for dbt contracts use JSONSchema $schema for dbt contracts (via dataclasses + jsonschema-dataclasses) Jun 12, 2019
@cmcarthur
Copy link
Member

use a mix of dataclasses + https://github.com/s-knibbs/dataclasses-jsonschema to generate a JSON schema document for all of dbt's internal dbt objects

@cmcarthur
Copy link
Member

should we generate & write out the json schema document to the target path?

@beckjake
Copy link
Contributor Author

Fixed in #1589

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants