Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(browse): filter entities by whether they might exist in the instance #8355

Conversation

joshuaeilers
Copy link
Contributor

@joshuaeilers joshuaeilers commented Jun 30, 2023

Changes

  • Filters the entities further via a second query for possible entities in the instance
  • If the base queried error'd for some reason, we fallback to the base filteredEntities
  • Makes the aggregation queries retryable

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@joshuaeilers joshuaeilers marked this pull request as ready for review June 30, 2023 23:59
@github-actions github-actions bot added the product PR or Issue related to the DataHub UI/UX label Jun 30, 2023
Copy link
Collaborator

@chriscollins3456 chriscollins3456 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking solid - just a question around what's going on in useSidebarEntities

Comment on lines 26 to 31
if (filteredError || baseError) return filteredAggregations;
if (!filteredAggregations) return filteredAggregations;
if (!baseEntityAggregations) return baseEntityAggregations;
return filteredAggregations.filter((agg) =>
baseEntityAggregations.some((base) => base.value === agg.value && !!base.count),
);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm just a little confused by this logic here - so if there's an error in either of the requests, return filteredAggregations. But if there's no filteredAggregations, return it? and same for base aggregations? it looks like in the ideal case, we return filtered aggregations that exist in the base aggregations. Why would there be entity aggregations in filteredAggregations but not in baseEntityAggregations?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lemme walk through the logic and see if I'm missing something.

The overall idea here is to only show the filteredAggregations with 0 counts if there "could" be some if we hadn't filtered in the instance. So, we'll always drive rendering off filteredAggregations.

if (filteredError || baseError) return filteredAggregations;

We'll always just default to whatever filteredAggregations is here if there's an error. If our filteredAggregations succeeded, but the base one error'd, we'll just fallback to filteredAggregations for simplicity. In this edge case, we might end up showing the count:0 for entities not in the instance but it would be rare and only in those cases where our base query failed. It's a fallback path here.

if (!filteredAggregations) return filteredAggregations;

Again, this could be return null if clearer. Does the same thing.

if (!baseEntityAggregations) return baseEntityAggregations;

Also could be return null, does the same thing.

        return filteredAggregations.filter((agg) =>
            baseEntityAggregations.some((base) => base.value === agg.value && !!base.count),
        );

This removes aggregations that don't have some counts in the base query. So if there's none of a certain type in the base query (like Feature Tables), we can safely remove it from the UI. But, if it is present in the base query, we can keep it even if there's a 0 count in the filtered query.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just pushed another commit that has the same logic but hopefully is a little bit more clear. Let me know!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay this is definitely helpful, and I think i was just thinking about it somewhat incorrectly as well

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries. Apollo is not the best for handling inter-dependent queries like this where you want to say await query(); await query(); doSomething();

Copy link
Collaborator

@chriscollins3456 chriscollins3456 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

@joshuaeilers joshuaeilers enabled auto-merge (squash) July 5, 2023 17:49
@joshuaeilers joshuaeilers merged commit 3207e73 into datahub-project:master Jul 5, 2023
32 of 33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
product PR or Issue related to the DataHub UI/UX
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants