-
Notifications
You must be signed in to change notification settings - Fork 998
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Break up queries for point in time correctness joins #347
Conversation
/retest |
@zhilingc: you cannot LGTM your own PR. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
serving/src/main/java/feast/serving/service/BigQueryServingService.java
Outdated
Show resolved
Hide resolved
serving/src/main/java/feast/serving/store/bigquery/BatchRetrievalQueryRunnable.java
Outdated
Show resolved
Hide resolved
serving/src/main/java/feast/serving/store/bigquery/BatchRetrievalQueryRunnable.java
Outdated
Show resolved
Hide resolved
serving/src/main/java/feast/serving/store/bigquery/model/FeatureSetInfo.java
Show resolved
Hide resolved
/lgtm |
…e refactoring Cleanup; Remove lombok dependency Add end to end tests for batch retrieval Only make necessary functions transactional Fix typo for e2e test file Fix sorting for windowing Add left join so that rows are consistent Remove UUID
Clean up threads when error is thrown
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: zhilingc The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
1 similar comment
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: zhilingc The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/retest |
/lgtm |
Plus some gentle refactoring.
The flow for batch retrieval now looks something like this:
All in all this should result in less resource utilisation and faster queries, since we hold less data in memory.
Also to note: at each feature set p-i-t join step, only the relevant entities are used. This allows users to pass in unpartitionable values (like floats) in their entity table as long as they do not use it as an actual key.