Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Highlighters should reject ignored keyword field early #43800

Closed
jimczi opened this issue Jul 1, 2019 · 2 comments · Fixed by #53408
Closed

Highlighters should reject ignored keyword field early #43800

jimczi opened this issue Jul 1, 2019 · 2 comments · Fixed by #53408
Assignees
@mayya-sharipova
Labels
>enhancement :Search Relevance/Highlighting How a query matched a document Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch

Comments

@jimczi
Copy link
Contributor

jimczi commented Jul 1, 2019

Highlighters are able to handle text and keyword field by reanalyzing their content at search time. However they don't check if the field was ignored at search time due to the ignore_above option. For instance a keyword field that is greater than the ignore_above option would skip indexing but if it is set in an highlighter option, the content will be extracted from _source and reanalyzed (ignoring ignore_above option). We should check the ignore_above option at highlighting time in order to avoid hitting the highlight.max_analyzed_offset on a field that was ignored at index time.
@jimczi jimczi added >enhancement :Search Relevance/Highlighting How a query matched a document labels Jul 1, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search

@jimczi jimczi added the help wanted adoptme label Dec 13, 2019
@mayya-sharipova
Copy link
Contributor

Related proposal: #52155 (comment) to introduce ignore_above parameter in a highlighter.
That proposal doesn't replace the current issue.

@mayya-sharipova mayya-sharipova removed the help wanted adoptme label Mar 9, 2020
@mayya-sharipova mayya-sharipova self-assigned this Mar 9, 2020
@mayya-sharipova mayya-sharipova mentioned this issue Mar 11, 2020
Merged
mayya-sharipova added a commit to mayya-sharipova/elasticsearch that referenced this issue Mar 11, 2020
@mayya-sharipova
Highlighters skip ignored keyword values
d0fa6d6 
Keyword field values with length more than ignore_above are not
indexed. But highlighters still were retrieving these values
from _source and were trying to highlight them. This sometimes lead to
errors if a field length exceeded  max_analyzed_offset. But also this
is a wrong behaviour to attempt to highlight something that was not
ignored during indexing.

This PR checks if a keyword value was ignored because of its length,
and if yes, skips highlighting it.

Closes elastic#43800
@eedugon eedugon mentioned this issue Mar 11, 2020
Closed
mayya-sharipova added a commit that referenced this issue Mar 16, 2020
@mayya-sharipova
Highlighters skip ignored keyword values (#53408)
01eee1a 
Keyword field values with length more than ignore_above are not
indexed. But highlighters still were retrieving these values
from _source and were trying to highlight them. This sometimes lead to
errors if a field length exceeded  max_analyzed_offset. But also this
is a wrong behaviour to attempt to highlight something that was not
ignored during indexing.

This PR checks if a keyword value was ignored because of its length,
and if yes, skips highlighting it.

Closes #43800
mayya-sharipova added a commit to mayya-sharipova/elasticsearch that referenced this issue Mar 16, 2020
@mayya-sharipova
Highlighters skip ignored keyword values (elastic#53408)
5f7af83 
Keyword field values with length more than ignore_above are not
indexed. But highlighters still were retrieving these values
from _source and were trying to highlight them. This sometimes lead to
errors if a field length exceeded  max_analyzed_offset. But also this
is an overall wrong behaviour to attempt to highlight something that was
ignored during indexing.

This PR checks if a keyword value was ignored because of its length,
and if yes, skips highlighting it.

Closes elastic#43800
@mayya-sharipova mayya-sharipova mentioned this issue Mar 16, 2020
Merged
mayya-sharipova added a commit that referenced this issue Mar 16, 2020
@mayya-sharipova
Highlighters skip ignored keyword values (#53408) (#53604)
a906f8a 
Keyword field values with length more than ignore_above are not
indexed. But highlighters still were retrieving these values
from _source and were trying to highlight them. This sometimes lead to
errors if a field length exceeded  max_analyzed_offset. But also this
is an overall wrong behaviour to attempt to highlight something that was
ignored during indexing.

This PR checks if a keyword value was ignored because of its length,
and if yes, skips highlighting it.

Backport: #53408
Closes #43800
@codebrain codebrain mentioned this issue Apr 1, 2020
Closed
38 tasks
@jtibshirani jtibshirani mentioned this issue May 29, 2020
Merged
@javanna javanna added the Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch label Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mayya-sharipova mayya-sharipova

Labels
>enhancement :Search Relevance/Highlighting How a query matched a document Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Highlighters skip ignored keyword values mayya-sharipova/elasticsearch
4 participants
@javanna @mayya-sharipova @elasticmachine @jimczi