Skip to content
This repository has been archived by the owner on May 6, 2021. It is now read-only.

Error when indexing infinite/nan values #158

Closed
davidrapin opened this issue Feb 17, 2020 · 0 comments · May be fixed by #159
Closed

Error when indexing infinite/nan values #158

davidrapin opened this issue Feb 17, 2020 · 0 comments · May be fixed by #159

Comments

@davidrapin
Copy link
Contributor

Pushed by a customer;

  • happens during bulk indexation

Versions

  • Neo4j 3.5.8 Enterprise
  • graphaware-neo4j-to-elasticsearch-3.5.1.53.11.jar
  • graphaware-server-enterprise-all-3.5.1.53.jar
  • graphaware-uuid-3.5.1.53.17.jar

What happens:

  • the value stored in Java is Double.NEGATIVE_INFINITY
  • the Elasticsearch client tries to serialize the value to send it to Elasticsearch
  • the serializer complains that -Infinity is not standard JSON, i.e. it does not know how to serialize the value.

How to fix:

  • set up the HttpClient to use a custom Gson serializer to avoid the issue

Option 1:

  • use GsonBuilder.serializeSpecialFloatingPointValues() to tolerate -Infinity etc
  • problem: this is not valid JSON and migh break in ES

Option 2:

  • create a serialization adapter to convert -Infinity into a "-Infinity" (a valid JSON string)

Error log:

2020-02-17 12:32:54.266+0000 WARN Runtime exception occurred during transaction execution. -Infinity is not a valid double value as per JSON specification. To override this behavior, use GsonBuilder.serializeSpecialFloatingPointValues() method.
java.lang.IllegalArgumentException: -Infinity is not a valid double value as per JSON specification. To override this behavior, use GsonBuilder.serializeSpecialFloatingPointValues() method.
at com.google.gson.Gson.checkValidFloatingPoint(Gson.java:317)
at com.google.gson.Gson$3.write(Gson.java:285)
at com.google.gson.Gson$3.write(Gson.java:271)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
at com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:208)
at com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:145)
at com.google.gson.Gson.toJson(Gson.java:661)
at com.google.gson.Gson.toJson(Gson.java:640)
at com.google.gson.Gson.toJson(Gson.java:595)
at com.google.gson.Gson.toJson(Gson.java:575)
at io.searchbox.action.AbstractAction.getData(AbstractAction.java:155)
at io.searchbox.core.Bulk.getData(Bulk.java:102)
at io.searchbox.client.http.JestHttpClient.prepareRequest(JestHttpClient.java:81)
at io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:46)
at com.graphaware.module.es.executor.BulkOperationExecutor.executeBulk(BulkOperationExecutor.java:83)
at com.graphaware.module.es.executor.BulkOperationExecutor.flush(BulkOperationExecutor.java:63)
at com.graphaware.module.es.ElasticSearchWriter.flushBatch(ElasticSearchWriter.java:147)
at com.graphaware.module.es.ElasticSearchWriter.processOperations(ElasticSearchWriter.java:143)
at com.graphaware.module.es.ElasticSearchModule.lambda$reindexNodes$2(ElasticSearchModule.java:184)
at com.graphaware.tx.executor.batch.IterableInputBatchTransactionExecutor.lambda$processQueue$1(IterableInputBatchTransactionExecutor.java:115)
at com.graphaware.tx.executor.single.SimpleTransactionExecutor.doExecuteInTransaction(SimpleTransactionExecutor.java:69)
at com.graphaware.tx.executor.single.SimpleTransactionExecutor.executeInTransaction(SimpleTransactionExecutor.java:58)
at com.graphaware.tx.executor.batch.IterableInputBatchTransactionExecutor.processQueue(IterableInputBatchTransactionExecutor.java:103)
at com.graphaware.tx.executor.batch.IterableInputBatchTransactionExecutor.doExecute(IterableInputBatchTransactionExecutor.java:76)
at com.graphaware.tx.executor.batch.DisposableBatchTransactionExecutor.execute(DisposableBatchTransactionExecutor.java:35)
at com.graphaware.module.es.ElasticSearchModule.reindexNodes(ElasticSearchModule.java:189)
at com.graphaware.module.es.ElasticSearchModule.lambda$reindex$0(ElasticSearchModule.java:146)
at java.lang.Thread.run(Thread.java:748)
.....
2020-02-17 12:32:54.267+0000 WARN Rolled back transaction for batch number 79540
davidrapin added a commit to Linkurious/neo4j-to-elasticsearch that referenced this issue Feb 17, 2020
@davidrapin
graphaware#158 fix bug when indexing non-finite numbers
ac6d007
davidrapin added a commit to Linkurious/neo4j-to-elasticsearch that referenced this issue Feb 18, 2020
@davidrapin
graphaware#158 remove unnecessary "serializeSpecialFloatingPointValues"
356ba2a
@davidrapin davidrapin mentioned this issue Feb 18, 2020
Open
davidrapin added a commit to Linkurious/neo4j-to-elasticsearch that referenced this issue Feb 18, 2020
@davidrapin
graphaware#158 moved inner class
2bd15e3
@bachmanm bachmanm closed this as completed May 5, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix bug when indexing non-finite numbers Linkurious/neo4j-to-elasticsearch
2 participants
@davidrapin @bachmanm