Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IllegalStateException] last() should not be called in this context #32848

Closed
jinhucheung opened this issue Aug 14, 2018 · 6 comments
Closed
Assignees
Labels
>bug :Search Relevance/Highlighting How a query matched a document Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch

Comments

@jinhucheung
Copy link

jinhucheung commented Aug 14, 2018

Elasticsearch version (bin/elasticsearch --version): 6.3.1

Plugins installed: [jcseg]

JVM version (java -version): 1.8.0_172

OS version (uname -a if on a Unix-like system): 4.4.0-127-generic #153-Ubuntu
dao

Description of the problem including expected versus actual behavior:

An error last() should not be called in this context occurred when get the total number of hits

Steps to reproduce:

I search keyword 'layui' via elasticsearch-rails, then get an error when getting the total count of hits

ruby code

codes = Code.__elasticsearch__.search(query_options)
codes = codes.records(includes: [:user, :lang, :category])
codes.total_count     #=> get an error

query options:

{  
   "from":0,
   "size":15,
   "query":{  
      "bool":{  
         "must":[  
            {  
               "multi_match":{  
                  "query":"layui",
                  "fields":[  
                     "code_pieces.content^1.0",
                     "code_pieces.title^1.0",
                     "name_with_summary^100.0"
                  ]
               }
            }
         ],
         "filter": [  
            {  
               "term":{  
                  "in_square":{  
                     "value":true
                  }
               }
            },
            {  
               "term":{  
                  "public":{  
                     "value":1
                  }
               }
            }
         ]
      }
   },
   "sort":[  
      {  
         "_score":{  
            "order":"desc"
         }
      }
   ],
   "highlight":{  
      "pre_tags":[  
         "<em>"
      ],
      "post_tags":[  
         "</em>"
      ],
      "fragment_size":200,
      "number_of_fragments":3,
      "fields":{  
         "name_with_summary": {},
         "code_pieces.content": {},
         "code_pieces.title": {}
      }
   }
}

index settings:

{
  "code_index_production" : {
    "aliases" : { },
    "mappings" : {
      "code" : {
        "properties" : {
          "category_id" : {
            "type" : "integer"
          },
          "code_pieces" : {
            "properties" : {
              "content" : {
                "type" : "text",
                "index_options" : "offsets",
                "analyzer" : "jcseg"
              },
              "content_type" : {
                "type" : "integer"
              },
              "id" : {
                "type" : "integer"
              },
              "title" : {
                "type" : "text",
                "index_options" : "offsets",
                "analyzer" : "jcseg_complex"
              }
            }
          },
          "created_at" : {
            "type" : "date"
          },
          "enterprise_id" : {
            "type" : "integer"
          },
          "forks_count" : {
            "type" : "integer"
          },
          "id" : {
            "type" : "integer"
          },
          "in_square" : {
            "type" : "boolean"
          },
          "lang_id" : {
            "type" : "integer"
          },
          "lang_name" : {
            "type" : "keyword"
          },
          "name_with_summary" : {
            "type" : "text",
            "index_options" : "offsets",
            "analyzer" : "jcseg_complex"
          },
          "notes_count" : {
            "type" : "integer"
          },
          "pieces_count" : {
            "type" : "integer"
          },
          "public" : {
            "type" : "integer"
          },
          "root_id" : {
            "type" : "integer"
          },
          "stars_count" : {
            "type" : "integer"
          },
          "summary" : {
            "type" : "text",
            "index_options" : "offsets",
            "analyzer" : "jcseg_complex"
          },
          "updated_at" : {
            "type" : "date"
          },
          "user_id" : {
            "type" : "integer"
          }
        }
      }
    },
    "settings" : {
      "index" : {
        "number_of_shards" : "1",
        "provided_name" : "code_index_production",
        "creation_date" : "1533553788519",
        "number_of_replicas" : "1",
        "uuid" : "3f-KmuKSTeC4MuCy14tkiw",
        "version" : {
          "created" : "6030199"
        }
      }
    }
  }
}

Provide logs (if relevant):

018-08-14T18:13:49,947][DEBUG][o.e.a.s.TransportSearchAction] [elasticsearch_node_001] [code_index_production][0], node[pPuDMK87ROSebXfQOmgmuw], [P], s[STARTED], a[id=0wGOjF3OS6mXGyMKgPYi5A]: Failed to execute [SearchRequest{searchType=QUERY_THEN_FETCH, indices=[code_index_production], indicesOptions=IndicesOptions[id=38, ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false], types=[code], routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=5, batchedReduceSize=512, preFilterShardSize=128, allowPartialSearchResults=true, source={"from":0,"size":15,"query":{"bool":{"must":[{"multi_match":{"query":"layui","fields":["code_pieces.content^1.0","code_pieces.title^1.0","name_with_summary^100.0"],"type":"best_fields","operator":"OR","slop":0,"prefix_length":0,"max_expansions":50,"zero_terms_query":"NONE","auto_generate_synonyms_phrase_query":true,"fuzzy_transpositions":true,"boost":1.0}}],"filter":[{"term":{"in_square":{"value":true,"boost":1.0}}},{"term":{"public":{"value":1,"boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}},"sort":[{"_score":{"order":"desc"}}],"highlight":{"pre_tags":["<em>"],"post_tags":["</em>"],"fragment_size":200,"number_of_fragments":3,"fields":{"name_with_summary":{},"code_pieces.content":{},"code_pieces.title":{}}}}}]
org.elasticsearch.transport.RemoteTransportException: [elasticsearch_node_001][192.168.1.111:9300][indices:data/read/search[phase/query]]
Caused by: java.lang.IllegalStateException: last() should not be called in this context
	at org.apache.lucene.search.uhighlight.BoundedBreakIteratorScanner.last(BoundedBreakIteratorScanner.java:175) ~[elasticsearch-6.3.1.jar:7.3.1 ae0705edb59eaa567fe13ed3a222fdadc7153680 - caomanhdat - 2018-05-09 09:28:05]
	at org.apache.lucene.search.uhighlight.SplittingBreakIterator.preceding(SplittingBreakIterator.java:218) ~[lucene-highlighter-7.3.1.jar:7.3.1 ae0705edb59eaa567fe13ed3a222fdadc7153680 - caomanhdat - 2018-05-09 09:28:05]
	at org.apache.lucene.search.uhighlight.FieldHighlighter.highlightOffsetsEnums(FieldHighlighter.java:163) ~[lucene-highlighter-7.3.1.jar:7.3.1 ae0705edb59eaa567fe13ed3a222fdadc7153680 - caomanhdat - 2018-05-09 09:28:05]
	at org.apache.lucene.search.uhighlight.FieldHighlighter.highlightFieldForDoc(FieldHighlighter.java:80) ~[lucene-highlighter-7.3.1.jar:7.3.1 ae0705edb59eaa567fe13ed3a222fdadc7153680 - caomanhdat - 2018-05-09 09:28:05]
	at org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFieldsAsObjects(UnifiedHighlighter.java:629) ~[lucene-highlighter-7.3.1.jar:7.3.1 ae0705edb59eaa567fe13ed3a222fdadc7153680 - caomanhdat - 2018-05-09 09:28:05]
	at org.apache.lucene.search.uhighlight.CustomUnifiedHighlighter.highlightField(CustomUnifiedHighlighter.java:107) ~[elasticsearch-6.3.1.jar:7.3.1 ae0705edb59eaa567fe13ed3a222fdadc7153680 - caomanhdat - 2018-05-09 09:28:05]
	at org.elasticsearch.search.fetch.subphase.highlight.UnifiedHighlighter.highlight(UnifiedHighlighter.java:131) ~[elasticsearch-6.3.1.jar:6.3.1]
	at org.elasticsearch.search.fetch.subphase.highlight.HighlightPhase.hitExecute(HighlightPhase.java:114) ~[elasticsearch-6.3.1.jar:6.3.1]
	at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:162) ~[elasticsearch-6.3.1.jar:6.3.1]
	at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:393) ~[elasticsearch-6.3.1.jar:6.3.1]
	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:368) ~[elasticsearch-6.3.1.jar:6.3.1]
	at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:333) [elasticsearch-6.3.1.jar:6.3.1]
	at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:329) [elasticsearch-6.3.1.jar:6.3.1]
	at org.elasticsearch.search.SearchService$3.doRun(SearchService.java:1019) [elasticsearch-6.3.1.jar:6.3.1]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:725) [elasticsearch-6.3.1.jar:6.3.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.3.1.jar:6.3.1]
	at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.3.1.jar:6.3.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.3.1.jar:6.3.1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]

thanks a lot for elasticsearch

Jim Cheung

@jinhucheung jinhucheung changed the title [IllegalStateException] last should not be called in this context [IllegalStateException] last() should not be called in this context Aug 14, 2018
@jimczi jimczi added >bug :Search Relevance/Highlighting How a query matched a document labels Aug 14, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search-aggs

@jimczi jimczi self-assigned this Aug 14, 2018
@jimczi
Copy link
Contributor

jimczi commented Aug 16, 2018

@jinhucheung I am not able to reproduce, can you add the analyzer definitions for jcseg and jcseg_complex ?

@jinhucheung
Copy link
Author

jinhucheung commented Aug 16, 2018

@jimczi thanks, I highlighted the code_pieces fields cause this error.

code_pieces is array type, the following is an simple response of a query without highlight code_pieces

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 46,
    "max_score" : 921.8962,
    "hits" : [
      {
        "_index" : "code_index_production",
        "_type" : "code",
        "_id" : "45899",
        "_score" : 921.8962,
        "_source" : {
          "id" : 1,
          "public" : 1,
          "user_id" : 1,
          "lang_id" : 2,
          "root_id" : 0,
          "category_id" : 3,
          "enterprise_id" : 0,
          "stars_count" : 0,
          "forks_count" : 0,
          "notes_count" : 0,
          "pieces_count" : 1,
          "in_square" : true,
          "lang_name" : "PHP",
          "summary" : "layui分页",
          "name_with_summary" : "jimcheung / layui分页",
          "created_at" : "2017-11-25T13:53:45+08:00",
          "updated_at" : "2017-11-25T13:53:45+08:00",
          "code_pieces" : [
            {
              "id" : 59766,
              "content_type" : 0,
              "content" : "var title = $(\"input[name='title']\").val();\r\n\tvar begin_time =$(\"input[name='begin_time']\").val();\r\n\tvar end_time=$(\"input[name='end_time']\").val();\r\n    layui.use(['laypage', 'layer'], function(){\r\n        var laypage = layui.laypage\r\n            ,layer = layui.layer;\r\n        layui.use(['laypage', 'layer'], function() {\r\n            var laypage = layui.laypage\r\n                ,layer = layui.layer;\r\n            laypage.render({\r\n                elem: 'demo1',\r\n                count: {$count},\r\n                limit:15,\r\n                curr:{$curr},\r\n                jump: function(e, first){ //触发分页后的回调\r\n                    if(!first){ //一定要加此判断,否则初始时会无限刷新\r\n                        location.href = '?page='+e.curr+\"&title=\"+title+\"&begin_time=\"+begin_time+\"&end_time=\"+end_time;\r\n                    }\r\n                }\r\n            });\r\n        });\r\n\r\n    });",
              "title" : "分页"
            }
          ]
        },
        "highlight" : {
          "name_with_summary" : [
            "jimcheung / <em>layui</em>分页"
          ]
        }
      }
    ]
  }
}

here is the jcseg analyzer and its elasticsearch plugin

jcseg analyzer

elasticsearch-jcseg-6.3.1.zip

@jimczi
Copy link
Contributor

jimczi commented Aug 21, 2018

@jinhucheung I am not able to reproduce with the example document. Could you please provide the document that is causing the exception ?

@jinhucheung
Copy link
Author

@jimczi I am sorry for the late response

The following is a document that is causing this exception

 "_source" : {
  "id" : 46867,
  "public" : 1,
  "user_id" : 1619994,
  "lang_id" : 3,
  "root_id" : 0,
  "category_id" : 89,
  "enterprise_id" : 0,
  "stars_count" : 0,
  "forks_count" : 0,
  "notes_count" : 0,
  "pieces_count" : 3,
  "in_square" : true,
  "lang_name" : "JavaScript",
  "summary" : "Layui上传多附件材料代码",
  "name_with_summary" : "LKBBD / Layui上传多附件材料代码",
  "created_at" : "2017-12-03T13:56:06+08:00",
  "updated_at" : "2017-12-03T13:59:33+08:00",
  "code_pieces" : [
    {
      "id" : 62466,
      "content_type" : 0,
      "content" : "基于layui,引入layui.css,layui.js及jquery.js等\r\n<link rel=\"stylesheet\" href=\"/style/layui/css/layui.css\" type=\"text/css\" media=\"all\"></link>\r\n<script type=\"text/javascript\" src=\"/style/layui/layui.js\"></script>",
      "title" : "js静态资源部分(主要)"
    },
    {
      "id" : 62467,
      "content_type" : 0,
      "content" : "<div class=\"layui-upload\">\r\n    <button type=\"button\" class=\"layui-btn layui-btn-normal\" id=\"testList\">选择多文件</button> \r\n    <div class=\"layui-upload-list\">\r\n    <table class=\"layui-table\">\r\n        <thead>\r\n        <tr>\r\n        <th style=\"width:40%;text-align:left;padding:10px 20px;\">文件名</th>\r\n        <th style=\"width:20%;text-align:left;padding:10px 20px;\">大小</th>\r\n        <th style=\"width:20%;text-align:left;padding:10px 20px;\">状态</th>\r\n        <th style=\"width:20%;text-align:left;padding:10px 20px;\">操作</th>\r\n        </tr>\r\n        </thead>\r\n        <tbody id=\"demoList\"></tbody>\r\n    </table>\r\n    </div>\r\n    <button type=\"button\" class=\"layui-btn\" id=\"testListAction\">开始上传</button>\r\n</div> ",
      "title" : "html部分"
    },
    {
      "id" : 62468,
      "content_type" : 0,
      "content" : "layui.use('upload', function(){\r\n    var $ = layui.jquery,\r\n    upload = layui.upload;\r\n    var files = [];\r\n    var demoListView = $('#demoList'),\r\n    uploadListIns = upload.render({\r\n    elem: '#testList',\r\n    url: '/file/ftp/upload/',//FTP上传 \r\n    accept: 'file',\r\n    multiple: true,\r\n    auto: false,\r\n    size: 6000, //限制文件大小,单位 KB\r\n    exts: 'jpg|png|xlsx|docx|pdf',//只允许上传文件类型\r\n    bindAction: '#testListAction',\r\n    choose: function(obj){   \r\n        files = obj.pushFile(); //将每次选择的文件追加到文件队列\r\n        //读取本地文件\r\n        obj.preview(function(index, file, result){\r\n        var tr = $(['<tr id=\"upload-'+ index +'\">',\r\n            '<td style=\\\"width:40%;padding:10px;text-align:left;font-size:14px;\\\">'+ file.name +'</td>',\r\n            '<td style=\\\"width:20%;padding:10px;text-align:left;font-size:14px;\\\">'+ (file.size/1014).toFixed(1) +'kb</td>',\r\n            '<td style=\\\"width:20%;padding:10px;text-align:left;font-size:14px;\\\">等待上传</td>',\r\n            '<td style=\\\"width:20%;padding:10px;text-align:left;font-size:14px;\\\">',\r\n            '<button class=\"layui-btn layui-btn-mini demo-reload layui-hide\">重传</button>',\r\n            '<button class=\"layui-btn layui-btn-mini layui-btn-danger demo-delete\">删除</button>',\r\n            '</td>',\r\n        '</tr>'].join(''));\r\n        //单个重传\r\n        tr.find('.demo-reload').on('click', function(){\r\n            obj.upload(index, file);\r\n        });\r\n        //删除\r\n        tr.find('.demo-delete').on('click', function(){\r\n            delete files[index]; //删除对应的文件\r\n            tr.remove();\r\n        });\r\n        demoListView.append(tr);\r\n        });\r\n    },\r\n    done: function(res, index, upload){\r\n        if(res.code == 0){ //上传成功\r\n        var tr = demoListView.find('tr#upload-'+ index),\r\n        tds = tr.children();\r\n        tds.eq(2).html('<span style=\"color: #5FB878;\">上传成功</span>');\r\n        tds.eq(3).html(''); //清空操作\r\n        var file_ids = $(\"input[name='dwyyzzsmj']\").val();\r\n        file_ids = file_ids + \"//\" + res.fileid;\r\n        $(\"input[name='dwyyzzsmj']\").val(\"\");\r\n        $(\"input[name='dwyyzzsmj']\").val(file_ids);\r\n        delete files[index]; //删除文件队列已经上传成功的文件\r\n        return;\r\n        }\r\n        this.error(index, upload);\r\n    },\r\n    error: function(index, upload){\r\n        var tr = demoListView.find('tr#upload-'+ index),\r\n        tds = tr.children();\r\n        tds.eq(2).html('<span style=\"color: #FF5722;\">上传失败</span>');\r\n        tds.eq(3).find('.demo-reload').removeClass('layui-hide'); //显示重传\r\n    }\r\n  });\r\n});",
      "title" : "js实现部分"
    }
  ]
}

here is a raw_code_pieces.zip

I highlight the code_pieces content cause the exception.

When I changed jcseg into another analyzer such as standard, or changed offsets-strategy into plain-highlighter, it is work.

I think that the exception is caused by jcseg

thanks

@jinhucheung
Copy link
Author

@jimczi thanks, I close this issue

@javanna javanna added the Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch label Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Search Relevance/Highlighting How a query matched a document Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch
Projects
None yet
Development

No branches or pull requests

4 participants