Autocomplete suggestions limited to 25: not enough!

[monsieurtanuki]

Describe the bug

• Method getAutocompletedSuggestions takes a taxonomy type and an input arguments (and a language).
• It returns all the taxonomy values from this type, and that match the input.
• The problem is that only 25 items max are returned, which can be problematic with common terms.

To Reproduce
Steps to reproduce the behavior:

1. Look for "soja" in French - go to https://world.openfoodfacts.org/cgi/suggest.pl?tagtype=categories&term=soja&lc=fr
2. You'll see only 3 results that match "Boissons végétales de soja"
3. Look for "Boissons végétales de soja" in French - go to https://world.openfoodfacts.org/cgi/suggest.pl?tagtype=categories&term=Boissons%20végétales%20de%20soja&lc=fr
4. You'll see 6 results that match "Boissons végétales de soja"

Expected behavior

• We would expect the 6 "Boissons végétales de soja" to be returned with the mere "soja" search.
• We could expect an additional parameter to go beyond 25 returned items.
• We could expect an additional returned value that gives the total number of items that match.
• That makes the method a bit difficult to trust, because it also expects a perfect match, and unless you're a genius you cannot imagine it should be "Boissons végétales de soja", "Boissons à base de soja" or "Boissons végétales au soja"
• Therefore we cannot realistically use that method IRW - I found the bug during unrelated tests.

Additional context
The problem obviously comes from the server, where $limit = 25 forces the max result length.

[M123-dev]

We should then probably move this issue to the server repo

CC @stephanegigandet

[stephanegigandet]

@alexgarel will look into it, we could add an option to set the number of desired results. Also we could split the input query so that "boisson soja" matches any category with the 2 words "boisson" and "soja".

[monsieurtanuki]

@stephanegigandet Both ideas would be appreciated.

The next level would be to return a json object instead of just a list, with more detailed fields like the total number of matching items (e.g. count). But that's another story.

[alexgarel]

I started looking at it.

The problem is that our suggestion is iterating the list, always in the order of the keys which might not be relevant at all ! (we might first get term that does not start with the term, etc.)

As our search is not indexed and only use brute force iteration, it does not seems like a good idea to process whole list every time…

[alexgarel]

I pushed a PR, but it's just a quick fix… I think in the future a search index could do a far better job.

[monsieurtanuki]

Thank you @alexgarel for that quick fix!
Once it's merged we'll see if there are hidden use cases we forgot.

[monsieurtanuki]

@alexgarel We still have to code it in dart!