Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Column lineage point in time java client #2269

Merged
merged 2 commits into from
Dec 13, 2022

Conversation

pawel-big-lebowski
Copy link
Collaborator

@pawel-big-lebowski pawel-big-lebowski commented Nov 28, 2022

Problem

Enable getting column lineage point-in-time in Java client of Marquez.

Closes: #2114

Solution

Implement point-in-time capability within Java client.

Note: All database schema changes require discussion. Please link the issue for context.

Checklist

  • You've signed-off your work
  • Your changes are accompanied by tests (if relevant)
  • Your change contains a small diff and is self-contained
  • You've updated any relevant documentation (if relevant)
  • You've updated the CHANGELOG.md with details about your change under the "Unreleased" section (if relevant, depending on the change, this may not be necessary)
  • You've versioned your .sql database schema migration according to Flyway's naming convention (if relevant)
  • You've included a header in any source code files (if relevant)

@boring-cyborg boring-cyborg bot added api API layer changes client/java labels Nov 28, 2022
@pawel-big-lebowski pawel-big-lebowski changed the base branch from main to column-lineage-point-in-time November 28, 2022 13:59
@codecov
Copy link

codecov bot commented Nov 28, 2022

Codecov Report

Merging #2269 (deafbd3) into main (11f6cec) will increase coverage by 9.85%.
The diff coverage is n/a.

❗ Current head deafbd3 differs from pull request most recent head 8ef7712. Consider uploading reports for the commit 8ef7712 to get more accurate results

@@             Coverage Diff              @@
##               main    #2269      +/-   ##
============================================
+ Coverage     66.98%   76.84%   +9.85%     
- Complexity      227     1154     +927     
============================================
  Files            40      220     +180     
  Lines           945     5268    +4323     
  Branches        103      423     +320     
============================================
+ Hits            633     4048    +3415     
- Misses          162      747     +585     
- Partials        150      473     +323     
Impacted Files Coverage Δ
...a/marquez/client/models/ColumnLineageNodeData.java 0.00% <0.00%> (ø)
api/src/main/java/marquez/api/JobResource.java 93.10% <0.00%> (ø)
api/src/main/java/marquez/api/SourceResource.java 90.00% <0.00%> (ø)
...main/java/marquez/graphql/GraphqlDataFetchers.java 42.19% <0.00%> (ø)
api/src/main/java/marquez/db/FlywayFactory.java 97.05% <0.00%> (ø)
...src/main/java/marquez/db/models/JobVersionRow.java 50.00% <0.00%> (ø)
...a/marquez/api/exceptions/RunNotFoundException.java 100.00% <0.00%> (ø)
api/src/main/java/marquez/db/ColumnLineageDao.java 100.00% <0.00%> (ø)
...n/java/marquez/graphql/mapper/ObjectMapMapper.java 71.42% <0.00%> (ø)
...in/java/marquez/service/DatasetVersionService.java 100.00% <0.00%> (ø)
... and 171 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Base automatically changed from column-lineage-point-in-time to main December 1, 2022 07:30
@pawel-big-lebowski pawel-big-lebowski force-pushed the column-lineage-point-in-time-java-client branch from f383558 to d24077b Compare December 1, 2022 09:03
@pawel-big-lebowski pawel-big-lebowski self-assigned this Dec 1, 2022
@pawel-big-lebowski pawel-big-lebowski marked this pull request as ready for review December 1, 2022 09:12
@pawel-big-lebowski pawel-big-lebowski force-pushed the column-lineage-point-in-time-java-client branch 2 times, most recently from 1489a7e to 0458065 Compare December 2, 2022 07:48
@pawel-big-lebowski pawel-big-lebowski force-pushed the column-lineage-point-in-time-java-client branch from 0458065 to 8ef7712 Compare December 7, 2022 12:56
Copy link
Member

@wslulciuc wslulciuc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make sure to note the removal of methods Client.getColumnLineageByDataset(), etc (now just Client.getColumnLineage(NodeId)) in our changelog.

@pawel-big-lebowski pawel-big-lebowski force-pushed the column-lineage-point-in-time-java-client branch from 8ef7712 to 66a9d15 Compare December 12, 2022 09:37
@boring-cyborg boring-cyborg bot added the docs label Dec 12, 2022
Copy link
Member

@wslulciuc wslulciuc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@wslulciuc wslulciuc enabled auto-merge (squash) December 13, 2022 16:28
@wslulciuc wslulciuc merged commit 90b2eff into main Dec 13, 2022
@wslulciuc wslulciuc deleted the column-lineage-point-in-time-java-client branch December 13, 2022 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api API layer changes client/java docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Column lineage graph endpoint
2 participants