feat(cli): Adds ability to upload recipes to DataHub's UI #8317
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pedro93
changed the title
github-actions
bot
added
docs
hsheth2
reviewed
Jun 27, 2023
pedro93
force-pushed
the
ps-add-recipe-deploy
branch
from
2d710c0
to
a4ff7e6
Compare
hsheth2
reviewed
Jun 29, 2023
pedro93
force-pushed
the
ps-add-recipe-deploy
branch
from
274c5fa
to
42b3462
Compare
hsheth2
approved these changes
Jul 27, 2023
| config: str, | ||
| urn: str, | ||
| executor_id: str, | ||
| cli_version: str, |
There was a problem hiding this comment.
for the non-required ones with no defaults, let's fix the type annotations to be Optional[str]
| click.echo( | ||
| f"✅ Successfully wrote data ingestion source metadata for recipe {name}:" | ||
| ) | ||
| click.echo(response) |
There was a problem hiding this comment.
does this show the urn of the newly created ingestion recipe?
There was a problem hiding this comment.
Not the urn, the user visible name
There was a problem hiding this comment.
if it's not too much work, can we should the urn too?
| Deploy an ingestion recipe to your DataHub instance. | ||
|
|
||
| The urn of the ingestion source will be based on the name parameter in the format: | ||
| urn:li:dataHubIngestionSource:<name> |
There was a problem hiding this comment.
where is this name -> urn logic? is that in the backend?
|
re tests - you can take a look at smoke-test/tests/cli/delete_cmd/test_timeseries_delete.py as an example |
5 tasks
yoonhyejin
pushed a commit
that referenced
this pull request
Aug 24, 2023
Co-authored-by: Indy Prentice <[email protected]>
spadhi7
added a commit
to spadhi7/datahub
that referenced
this pull request
Aug 29, 2023
* tag 'v0.10.5': (222 commits) fix(test): increase siblings.js test stability (datahub-project#8542) feat(search): Allow aggregating on facets that are not explicitly part of default filter set (datahub-project#8540) fix(ui) Make multiple small updates to new search and browse (datahub-project#8524) feat(presto-on-hive): allow v1 fieldpaths in the presto-on-hive source (datahub-project#8474) feat(cli): Adds ability to upload recipes to DataHub's UI (datahub-project#8317) feat(browseV2): add browseV2 logic to system update (datahub-project#8506) fix(ingest/json-schema): convert non-string enums to strings (datahub-project#8479) feat(ingestion/tableau): support column level lineage for custom sql (datahub-project#8466) test(ingest): test case statements with sql parser (datahub-project#8437) feat(ingest/vertica): performance improvement and bug fixes (datahub-project#8328) ci: reduce git fetch depth (datahub-project#8473) fix(ingest): remove duplication of tags (datahub-project#8532) docs: small update to homepage (datahub-project#8483) fix(ingest): pin boto3-stubs in CI (datahub-project#8527) feat(siblings): hiding non-existant siblings in FE (datahub-project#8528) fix(ingest/build): Fix sagemaker mypy and flake8 issues (datahub-project#8530) feat(metrics): add metrics for aspect write and bytes (datahub-project#8526) feat(elasticsearch): allow bulk delete (datahub-project#8424) fix(ui): use locale lowercase when filtering columns of an entity in the lineage (datahub-project#8213) fix(auth): ignore case when comparing http headers (datahub-project#8356) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Also updates listIngestionSources graphQL endpoint to use filters. This is to bring it in-line with other list endpoints. I added up not using it but I think it's useful.
Checklist
Adds support to upload recipes to DataHub's UI ingestion section using the cli.