🧪 Search for facet values

[gmourier]

Currently, there is no way to natively search for a facet value among a distributed facet, which can be significant depending on the dataset. Some facets can hold hundreds or thousands of values.

Introducing a search for facet values feature would allow end-users to discover and search for specific facet values that match their needs and thus speed up their finding.

We want to enable you to quickly implement that type of faceted search experience in a few lines of code.

We have planned to work on the first iteration for v1.2, and a prototype should be available in the next few weeks.

This prototype should provide the ability to search in facets using prefix search while being typo tolerant.

It will also allow injecting global search parameters to allow to find facets that match according to the current search context (e.g., query terms, filters, etc..)

Fields defined as filterableAttributes will be facet searchable without further configuration on the engine side.

Our research has permitted us to discover some grey areas for which we would like to evaluate the degree of importance to define better if this first iteration is sufficient and to prepare successive iterations on this feature.

We have listed the questions we would like your feedback on to assess your needs better.

• Is it critical to be able to paginate facet hits? If so, why?
• Is it critical to customize the maximum number of returned facet hits? If so, why?
• Is it critical to be able to sort the facet hits by count? If so, why?
• Is it critical to have the facet name highlighted in the facet hits? If so, why?
• Is there a particular reason you would want to restrict some filterableAttributes distributed as facets impossible to be searched on as facet?
• Is it critical for the facet search to support stop words and/or synonyms?

Do you see anything we may miss?

Thank you for your feedback!

You can also share your feedback on the related discord community discussion if you prefer.

[smultar]

It would help if you allowed a toggle in the search settings for each index, to allow for facet hits with search hits.

[gmourier]

Hey @smultar 👋

What do you mean by that? I'm not sure to understand!

Thanks!

[smultar]

Normally when I query Meilisearch. It brings back a variety of hits based on the search criteria. We should add an option, that allows hits for possible facets, based on the results.

So we would be getting a set of query and facet results. Using the results, we could make facet recommendations for filtering for the users.

[gmourier]

Does it relates to "It will also allow injecting global search parameters to allow to find facets that match according to the current search context (e.g., query terms, filters, etc..)"?

[Kerollmops]

Hello everyone 👋,

🔔 EDIT: A new prototype version is accessible here 🔔

Here is the current prototype you can use to test the new Search for Facet Values feature:
How to run the prototype? You need to start from a fresh new database (remove the previous used data.ms) and use the following Docker image:

docker run -it --rm -p 7700:7700 -v $(pwd)/meili_data:/meili_data getmeili/meilisearch:prototype-search-for-facet-values-0

You can use the feature this way or look at the original PR for more info:

# send some documents in your database
cat movies.json | xh 'localhost:7700/indexes/movies/documents' 'content-type:application/json'

# Make your facet filterable
echo '{ "filterableAttributes": ["genres"] }' | xh PATCH 'localhost:7700/indexes/movies/settings'

# Do your query to search for facets
echo '{"q": "spiderman", "filter": "price > 10", "facetName": "genres", "facetQuery": "act" }' | xh 'localhost:7700/indexes/movies/facet-search'

⚠️ We do NOT recommend using this prototype in production. This is only for test purposes.
Everyone is more than welcome to give feedback and to report any issue or bug you might encounter when using this prototype. Thanks in advance for your involvement. It means a lot to us ❤️

[Kerollmops]

Hello everyone 👋,

🔔 EDIT: A new prototype version is accessible here 🔔

Yes! The second prototype is already here! You can test the new Search for Facet Values feature:
How to run the prototype? You need to start from a fresh new database (remove the previous used data.ms) and use the following Docker image:

docker run -it --rm -p 7700:7700 -v $(pwd)/meili_data:/meili_data getmeili/meilisearch:prototype-search-for-facet-values-1

You can use the feature this way or look at the original PR for more info:

# send some documents in your database
cat movies.json | xh 'localhost:7700/indexes/movies/documents' 'content-type:application/json'

# Make your facet filterable
echo '{ "filterableAttributes": ["genres"] }' | xh PATCH 'localhost:7700/indexes/movies/settings'

# Do your query to search for facets
echo '{"q": "spiderman", "filter": "price > 10", "facetName": "genres", "facetQuery": "act" }' | xh 'localhost:7700/indexes/movies/facet-search'

In this new version we fixed a lot of issues regarding error codes and the output JSON object.

⚠️ We do NOT recommend using this prototype in production. This is only for test purposes.
Everyone is more than welcome to give feedback and to report any issue or bug you might encounter when using this prototype. Thanks in advance for your involvement. It means a lot to us ❤️

[nacoleman]

Hi,

I have this running locally... I was hoping it would be compatible with instantsearch refinementlist searchable attribute?

https://www.algolia.com/doc/api-reference/widgets/refinement-list/angular/

Is this not the case? I currently get this

Error: SearchForFacetValues is not compatible with Meilisearch

I see we would need an updated instant meilisearch library? - is that within scope of this work?

searchForFacetValues: async function (_: any) {
      return await new Promise((resolve, reject) => {
        reject(
          new Error('SearchForFacetValues is not compatible with Meilisearch')
        )
        resolve([]) // added here to avoid compilation error
      })
    },

[bidoubiwa]

Hey @nacoleman

It will work on instant-meilisearch! Normally the refinement list searchable option should be live on instant-meilisearch the day of the Meilisearch release that introduces this feature.

One small caveat is that you won't be able to sort it by count. Which should not be an issue if you don't have more than 100 facet values per facet.

[Kerollmops]

Hello everyone 👋,

🔔 EDIT: A new prototype version is accessible here 🔔

Yes! The third prototype is already here! You can test the new Search for Facet Values feature:
How to run the prototype? You need to start from a fresh new database (remove the previous used data.ms) and use the following Docker image:

docker run -it --rm -p 7700:7700 -v $(pwd)/meili_data:/meili_data getmeili/meilisearch:prototype-search-for-facet-values-2

You can use the feature this way or look at the original PR for more info:

# send some documents in your database
cat movies.json | xh 'localhost:7700/indexes/movies/documents' 'content-type:application/json'

# Make your facet filterable
echo '{ "filterableAttributes": ["genres"] }' | xh PATCH 'localhost:7700/indexes/movies/settings'

# Do your query to search for facets
echo '{"q": "spiderman", "filter": "price > 10", "facetName": "genres", "facetQuery": "act" }' | xh 'localhost:7700/indexes/movies/facet-search'

This version makes the search for facet values route follow the minWordSizeForTypos index settings.

⚠️ We do NOT recommend using this prototype in production. This is only for test purposes.
Everyone is more than welcome to give feedback and to report any issue or bug you might encounter when using this prototype. Thanks in advance for your involvement. It means a lot to us ❤️

[mech]

Is the facetName only allowed to have one? What happen if I want to query "genres" and "age"?

{"q": "spiderman", "filter": "price > 10", "facetName": "genres", "facetQuery": "act" , "facetName": "age", facetQuery: 100}

[gmourier]

Hello @mech 👋

Is there a particular reason to query two facets both at the same time in one query? I'm happy to learn more!

We build that API with the most typical facet search UI in mind, where an end-user will search one facet at a time to refine his navigation.

[gmourier]

Hello everyone 👋

We released a new prototype for the search for facet values.

Introduced changes

• The facet search is now case insensitive
• The returned facet results are now in their original casing (the one you provided when feeding the document to Meilisearch)

How to run the prototype?

You need to start from a fresh new database (remove the previous used data.ms) and use the following Docker image:

docker run -it --rm -p 7700:7700 -v $(pwd)/meili_data:/meili_data getmeili/meilisearch:prototype-search-for-facet-values-6

How to use the prototype?

# send some documents in your database
cat movies.json | xh 'localhost:7700/indexes/movies/documents' 'content-type:application/json'

# Make your facet filterable
echo '{ "filterableAttributes": ["genres"] }' | xh PATCH 'localhost:7700/indexes/movies/settings'

# Do your query to search for facets
echo '{"q": "spiderman", "filter": "price > 10", "facetName": "genres", "facetQuery": "act" }' | xh 'localhost:7700/indexes/movies/facet-search'

⚠️ We do NOT recommend using this prototype in production. This is only for test purposes.
Everyone is more than welcome to give feedback and to report any issue or bug you might encounter when using this prototype. Thanks in advance for your involvement. It means a lot to us ❤️

[macraig]

Hello everyone 👋

We have just released the first RC (release candidate) of Meilisearch containing this new feature!
You can test it by using

• the release assets
• the Meilisearch Docker image

docker run -it --rm -p 7700:7700 -v $(pwd)/meili_data:/meili_data getmeili/meilisearch:v1.3.0-rc.0

You are welcome to leave your feedback in this discussion.

If you encounter any bugs, please report them here.
Thanks in advance for your help and your involvement in Meilisearch ❤️

🎉 The official and stable release containing this change will be available on July 31st, 2023

⚠️ RC (release candidates) are not recommended for production

[macraig]

Hey folks 👋

v1.3 has been released! 🦁 You can now search for facet values and sort them by count ✨

📚 https://www.meilisearch.com/docs/learn/fine_tuning_results/faceted_search

[amonin7]

Hello Meilisearch team and @macraig ,

I hope this message finds you well. I've been exploring the recent additions to Meilisearch, particularly the facet search feature. First and foremost, I want to express my appreciation for the ongoing development and improvement of your excellent open-source search engine.

I'm currently working on a project where we need to adapt the facet search feature to provide more comprehensive results to our users. Meilisearch currently implemented it so that facet searches are only matched from the start of a given string and are not highlighted. That means when a user searches "Bus" the facet option "Green Bus" will not be matched and displayed (if I understand correctly).

Before I dive into the code and start making modifications, I wanted to check if there's already any work done in this direction within the Meilisearch codebase. If there is, I'd greatly appreciate it if you could point me in the right direction or provide some guidance on which parts of the code I should investigate to achieve these enhancements.

I look forward to your response and potentially collaborating with the Meilisearch community to make these improvements.

[Kerollmops]

Hey @amonin7 👋

Thank you very much for this feedback. The following feature you describe was something we wanted to implement in the future.

Meilisearch currently implemented it so that facet searches are only matched from the start of a given string and are not highlighted. That means when a user searches "Bus" the facet option "Green Bus" will not be matched and displayed (if I understand correctly).

However, when we fixed a bug in the facet search (before we released it), we changed the way we store the "words" extracted from the facet fields, and this enables more flexibility. Implementing facet word-based search is "straightforward" now.

Before I dive into the code and start making modifications, I wanted to check if there's already any work done in this direction within the Meilisearch codebase. If there is, I'd greatly appreciate it if you could point me in the right direction or provide some guidance on which parts of the code I should investigate to achieve these enhancements.

We we not prioritize this change on our side and we will work on other features. However, I would be happy to help you make the change on the Meilisearch codebase. I'll describe a little further the way I would do it and will continue to help you with the PR you open 🥹

1. Meilisearch indexation is done by milli, an internal crate. You'll have to modify this crate.
2. To make the facet search work, we currently extract the different facet values, normalize them, and put them in an internal LMDB database. You'll have to modify this part to store words/tokens instead.
3. On the search part, you'll have to do the same with a query and split it by words/tokens. You'll then have to do a big intersection on the different document IDs you'll get for each token/word.