Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Transform] geo_bounds aggregation failed with NPE error #71874

Closed
wwang500 opened this issue Apr 19, 2021 · 3 comments · Fixed by #71999
Closed

[Transform] geo_bounds aggregation failed with NPE error #71874

wwang500 opened this issue Apr 19, 2021 · 3 comments · Fixed by #71999
Assignees
Labels
:ml/Transform Transform Team:ML Meta label for the ML team

Comments

@wwang500
Copy link

wwang500 commented Apr 19, 2021

After #71696 was merged, one of our transform:geo_bounds test failed with this error: Cannot invoke "org.elasticsearch.geometry.Geometry.visit(org.elasticsearch.geometry.GeometryVisitor)" because "geometry" is null]

The better error handling is expected because the real failure was caused by non-existent geo.location field.

Build:
Latest 8.0.0 and 7.13.0 build on Apr 19

8.0.0 build hash:

"build" : {
    "hash" : "6dbad503d931d96acbc0c4bf1d2cfe08855265f1",
    "date" : "2021-04-19T14:47:16.966879383Z"
  },

Step to reproduce in dev console

PUT /_transform/test-nonexist-geo-field
{
  "source": {
    "index": [
      "airplane"
    ],
    "query": {
      "match_all": {}
    }
  },
  "dest": {
    "index": "test-nonexist-geo-field"
  },
  "pivot": {
    "group_by": {
      "aircrafttype": {
        "terms": {
          "field": "aircrafttype"
        }
      }
    },
    "aggregations": {
      "bounds": {
        "geo_bounds": {
          "field": "geo.location",
          "wrap_longitude": true
        }
      }
    }
  }
}
POST _transform/test-nonexist-geo-field/_start
GET _transform/test-nonexist-geo-field/_stats

Error:

"id" : "test-nonexist-geo-field",
      "state" : "failed",
      "reason" : """Failed to index documents into destination index due to permanent error: [org.elasticsearch.xpack.transform.transforms.BulkIndexingException: Bulk index experienced [185] failures and at least 1 irrecoverable [org.elasticsearch.xpack.transform.transforms.TransformException: Destination index mappings are incompatible with the transform configuration.; org.elasticsearch.index.mapper.MapperParsingException: failed to parse; java.lang.NullPointerException: Cannot invoke "org.elasticsearch.geometry.Geometry.visit(org.elasticsearch.geometry.GeometryVisitor)" because "geometry" is null]. Other failures:; org.elasticsearch.xpack.transform.transforms.TransformException: Destination index mappings are incompatible with the transform configuration.; org.elasticsearch.index.mapper.MapperParsingException: failed to parse; java.lang.NullPointerException: Cannot invoke "org.elasticsearch.geometry.Geometry.visit(org.elasticsearch.geometry.GeometryVisitor)" because "geometry" is null]""",

Expected:
Better error handling about the non-existent field: geo.location.

@wwang500 wwang500 added :Analytics/Geo Indexing, search aggregations of geo points and shapes :ml/Transform Transform labels Apr 19, 2021
@elasticmachine elasticmachine added Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:ML Meta label for the ML team labels Apr 19, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (Team:Analytics)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

@wwang500 wwang500 removed :Analytics/Geo Indexing, search aggregations of geo points and shapes Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) labels Apr 20, 2021
@romseygeek romseygeek self-assigned this Apr 21, 2021
@hendrikmuhs
Copy link
Contributor

I debugged this, it fails in the indexing step

Repro:

POST test-nonexist-geo-field/_doc
{
  "aircrafttype":"A319",
  "bounds": null
}

fails for fields of type "geo_shape"

hendrikmuhs pushed a commit to hendrikmuhs/elasticsearch that referenced this issue Apr 21, 2021
… adding debug

logging of index requests send by transform

relates elastic#71874
romseygeek added a commit that referenced this issue Apr 21, 2021
#71696 introduced a regression to the various shape field mappers,
where they would no longer handle null values. This commit fixes
that regression and adds a testNullValues method to MapperTestCase
to ensure that all field mappers correctly handle nulls.

Fixes #71874
romseygeek added a commit to romseygeek/elasticsearch that referenced this issue Apr 21, 2021
elastic#71696 introduced a regression to the various shape field mappers,
where they would no longer handle null values. This commit fixes
that regression and adds a testNullValues method to MapperTestCase
to ensure that all field mappers correctly handle nulls.

Fixes elastic#71874
romseygeek added a commit that referenced this issue Apr 22, 2021
#71696 introduced a regression to the various shape field mappers,
where they would no longer handle null values. This commit fixes
that regression and adds a testNullValues method to MapperTestCase
to ensure that all field mappers correctly handle nulls.

Fixes #71874
romseygeek added a commit that referenced this issue Apr 22, 2021
#71696 introduced a regression to the various shape field mappers,
where they would no longer handle null values. This commit fixes
that regression and adds a testNullValues method to MapperTestCase
to ensure that all field mappers correctly handle nulls.

Fixes #71874
elasticmachine added a commit to hendrikmuhs/elasticsearch that referenced this issue Apr 22, 2021
hendrikmuhs pushed a commit that referenced this issue Apr 22, 2021
use sparse data for geobounds agg, verifies the fix of #71874, adding debug
logging of index requests send by transform
hendrikmuhs pushed a commit that referenced this issue Apr 22, 2021
use sparse data for geobounds agg, verifies the fix of #71874, adding debug
logging of index requests send by transform
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:ml/Transform Transform Team:ML Meta label for the ML team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants