Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Do not search for exhaustive SNIPPET. #697

Merged
merged 2 commits into from
May 16, 2022
Merged

Do not search for exhaustive SNIPPET. #697

merged 2 commits into from
May 16, 2022

Commits on May 16, 2022

  1. Do not search for exhaustive SNIPPET. 

    `MSet::snippet` is more complex than it seems.
Xapian does some kind of complex algorithm to find the best text subset to
select. It does this by calculating the score/ranking of each term in the
text. To do so, it has to evaluate the terms in the context of the whole
mset and so, load "lot" of data from the database.

The perfect is the enemy of the good.
By removing the SNIPPET_EXHAUSTIVE flag, xapian evaluate less and return
(far more) quicker.
(https://xapian.org/docs/apidoc/html/classXapian_1_1MSet.html#a4797ae2295f88e49a9f76e3b89c21d88aea6a34a9c66720a44d5969ed47ca8edb)
Generated snippet is different, but still valid.
Do not search for exhaustive SNIPPET
    @mgautierfr
    mgautierfr committed May 16, 2022
    Configuration menu
    Copy the full SHA
    48f4f8c View commit details
    Browse the repository at this point in the history

  2. Also remove Xapian::MSet::SNIPPET_BACKGROUND_MODEL 

    It seems it has even more importance in the snippet generation speed.
SNIPPET_BACKGROUND_MODEL ask xapian to also compute score for non-query
terms. By removing it, we compute score only for query terms and it is
far better.
    @mgautierfr
    mgautierfr committed May 16, 2022
    Configuration menu
    Copy the full SHA
    b994697 View commit details
    Browse the repository at this point in the history