Swap from exact substring matches to each individual word matching #145

joellabes · 2020-10-10T10:01:22Z

resolves #143

Description

This is a first attempt at considering each word entered in a search individually, as opposed to requiring a perfect match (model my or my model should now find my_model, where earlier only an exact search, including underscores, would have worked). my_model still works of course.

I couldn't get the Docs site running locally (from looking at the stack traces, seemed to be a bunch of issues with webpack, node versions, python versions...) but ~~I have tested that each individual piece of work behaves correctly outside of the wider Docs context~~ I tried it in the Netlify CI preview and it works correctly.

Checklist

I have signed the CLA
I have generated docs locally, and this change appears to resolve the stated issue
I have updated the CHANGELOG.md and added information about my change to the "dbt next" section.

…opulate changelog

joellabes · 2020-10-10T10:27:38Z

src/app/services/project_service.js

@@ -271,23 +271,24 @@ angular
            'tags': 'array',
            'arguments': 'array',
        };
-        var search = new RegExp(val, "i")


This was never used

jtcohen6

This is really cool @joellabes !!

I'll wait until @drewbanin can give the JS a once-over, likely next week

drewbanin · 2020-11-24T13:33:49Z

ha - oops - thanks for your patience @joellabes - giving this a look now :D

drewbanin

I love what you're going for here, but I think it's about time that we giving searching some more love! What do you think about leveraging something like fusejs here and deleting a lot of this searching logic? I kicked around fuse.js, and it looks to me like it will match the query from the original issue (#143) out of the box!

Joel, if this is something you're interested in pursuing, I'd be really happy to offer some guidance! If not, I imagine it's something that we could get to in the reasonably near future. Let me know either way!

joellabes · 2020-11-25T03:55:21Z

I'd be so excited for search to get some love - I had an urge to rework the weighting process last week as well. Happy to have a blast with Fuse. I had dramas getting dbt running locally (Windows 🙄) but will give it another whirl - if it doesn't magically work I might have to get a hand on the initial setup!

(Might be a couple of weeks though, so don't feel obliged to wait for me if the mood strikes you)

joellabes · 2021-01-03T08:16:53Z

I've started looking into this on #162 - still a long way from it being ready, but I'm going to close this one for now.

joellabes · 2022-04-05T04:24:52Z

OK so 162 wound up being non-performant and not very good at fuzzy search. Not a good combo.

Since this has been sitting around waiting for a year and a bit, and works just fine, I'm going to reopen it and hopefully we can get it out into a future docs release. If someone who knows more JS wants to have a swing at fuse or some other proper search lib, I'm into it.

But in general, dbt docs searches are only a couple of words and it's not unreasonable to just require accurate typing 😅

cla-bot · 2022-04-06T23:57:25Z

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Joel Labes.
This is most likely caused by a git client misconfiguration; please make sure to:

check if your git client is configured with an email to sign commits git config --list | grep email
If not, set it up using git config --global user.email [email protected]
Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

stu-k

Just a couple comments about using lodash functions, nothing else stands out 👍

stu-k · 2022-04-25T14:38:49Z

src/app/components/search/search.js

+                return string.replace(/[.*+\-?^${}()|[\]\\]/g, '\\$&'); // $& means the whole matched string
+            }
+
+            function splitQuery(query){


Nitpick: a better name for this might be getQueryTokens, since it is unclear what the query is being split on without reading this function. Also, if you use lodash's words method, I think this function becomes unnecessary _.words(text).

Yep that's a better name! Have kept the function for lowercasing purposes but have swapped out all the icky split stuff for _.words

src/app/services/project_service.js

emmyoop · 2022-05-16T16:40:20Z

Tested this out and it seems like a big quality of life improvement! 🙌 I'll take it for another spin once you implement @stu-k's suggestions.

Your commit from October 13, 2020 is causing the cla bot to fail because it's associated with your [email protected] email. You should be able to squash down your commits and get the cla bot to pass.

cla-bot · 2022-07-18T02:44:58Z

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Joel Labes.
This is most likely caused by a git client misconfiguration; please make sure to:

check if your git client is configured with an email to sign commits git config --list | grep email
If not, set it up using git config --global user.email [email protected]
Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

cla-bot · 2022-07-18T02:46:53Z

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Joel Labes.
This is most likely caused by a git client misconfiguration; please make sure to:

check if your git client is configured with an email to sign commits git config --list | grep email
If not, set it up using git config --global user.email [email protected]
Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

joellabes · 2022-07-18T02:49:36Z

@cla-bot check

cla-bot · 2022-07-18T02:49:42Z

The cla-bot has been summoned, and re-checked this pull request!

joellabes · 2022-07-18T03:01:16Z

@emmyoop I've applied @stu-k's suggestions now 🛠️

I couldn't get the squash to work, so just re-added the educationperfect domain to my GH account and the CLA bot is happy!

18 months old

emmyoop

Great work! Thanks for staying on it and pushing it through. The search improvements are so so satisfying.

rightx2 · 2024-02-16T05:44:58Z

This feature really inconvinient. In my case, I need to find out the model named s_user but search result get me all models containing s, or user or both... I can't easily find out s_user among the 108 search results.... makes me scroll down infinitely

Joel Labes added 3 commits October 10, 2020 22:46

Swap from exact substring matches to each individual word matching

72c4271

Remove redundant keyboard logging

b2bc0b7

Avoid "Invalid regular expression" error when searching for just *, p…

09a06f2

…opulate changelog

cla-bot bot added the cla:yes label Oct 10, 2020

joellabes commented Oct 10, 2020

View reviewed changes

jtcohen6 reviewed Oct 12, 2020

View reviewed changes

Joel Labes added 2 commits October 13, 2020 11:10

Trim whitespace to avoid an empty array coming back from splitQuery

3c36ab4

Actually check query, not the text to truncate (doh)

314cfb8

jtcohen6 requested a review from drewbanin October 19, 2020 14:27

drewbanin previously requested changes Nov 24, 2020

View reviewed changes

joellabes mentioned this pull request Jan 3, 2021

Use fuse.js for dbt docs search #162

Closed

3 tasks

joellabes closed this Jan 3, 2021

joellabes reopened this Apr 5, 2022

joellabes changed the base branch from master to main April 5, 2022 04:26

Merge branch 'main' into feat-143-fuzzier-search

b4640b6

cla-bot bot removed the cla:yes label Apr 6, 2022

emmyoop requested a review from stu-k April 25, 2022 14:54

stu-k approved these changes Apr 25, 2022

View reviewed changes

emmyoop requested review from stu-k and emmyoop April 25, 2022 15:44

emmyoop mentioned this pull request May 16, 2022

[CT-652] dbt-docs Improvements for 1.2.0 dbt-labs/dbt-core#5255

Closed

10 tasks

emmyoop mentioned this pull request Jul 6, 2022

[CT-814] dbt-docs updates for dbt-core v1.3 dbt-labs/dbt-core#5443

Closed

3 tasks

Code review - use lodash's words function

c9d9244

Merge branch 'main' into feat-143-fuzzier-search

c463507

cla-bot bot added the cla:yes label Jul 18, 2022

emmyoop approved these changes Jul 21, 2022

View reviewed changes

emmyoop merged commit 07a617f into dbt-labs:main Jul 21, 2022

ChenyuLInx mentioned this pull request Jul 28, 2022

[CT-952] dbt-docs Improvements for 1.3.0 beta(porting changelog) dbt-labs/dbt-core#5574

Closed

6 tasks

joellabes deleted the feat-143-fuzzier-search branch October 17, 2022 07:16

rightx2 mentioned this pull request Feb 17, 2024

[Feature] Search bar should prioritise exact file name matches over fuzzy matches #217

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Swap from exact substring matches to each individual word matching #145

Swap from exact substring matches to each individual word matching #145

joellabes commented Oct 10, 2020 •

edited

Loading

joellabes Oct 10, 2020

jtcohen6 left a comment

drewbanin commented Nov 24, 2020

drewbanin left a comment

joellabes commented Nov 25, 2020 •

edited

Loading

joellabes commented Jan 3, 2021

joellabes commented Apr 5, 2022

cla-bot bot commented Apr 6, 2022

stu-k left a comment

stu-k Apr 25, 2022

joellabes Jul 18, 2022

emmyoop commented May 16, 2022

cla-bot bot commented Jul 18, 2022

cla-bot bot commented Jul 18, 2022

joellabes commented Jul 18, 2022

cla-bot bot commented Jul 18, 2022

joellabes commented Jul 18, 2022

emmyoop left a comment

rightx2 commented Feb 16, 2024

Swap from exact substring matches to each individual word matching #145

Swap from exact substring matches to each individual word matching #145

Conversation

joellabes commented Oct 10, 2020 • edited Loading

Description

Checklist

joellabes Oct 10, 2020

Choose a reason for hiding this comment

jtcohen6 left a comment

Choose a reason for hiding this comment

drewbanin commented Nov 24, 2020

drewbanin left a comment

Choose a reason for hiding this comment

joellabes commented Nov 25, 2020 • edited Loading

joellabes commented Jan 3, 2021

joellabes commented Apr 5, 2022

cla-bot bot commented Apr 6, 2022

stu-k left a comment

Choose a reason for hiding this comment

stu-k Apr 25, 2022

Choose a reason for hiding this comment

joellabes Jul 18, 2022

Choose a reason for hiding this comment

emmyoop commented May 16, 2022

cla-bot bot commented Jul 18, 2022

cla-bot bot commented Jul 18, 2022

joellabes commented Jul 18, 2022

cla-bot bot commented Jul 18, 2022

joellabes commented Jul 18, 2022

emmyoop left a comment

Choose a reason for hiding this comment

rightx2 commented Feb 16, 2024

joellabes commented Oct 10, 2020 •

edited

Loading

joellabes commented Nov 25, 2020 •

edited

Loading