Skip to content
This repository has been archived by the owner on May 6, 2021. It is now read-only.

Ignore malformed dates/numbers #104

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

davidrapin
Copy link
Contributor

Fixes #101

The idea is to create a mapping template in Elasticsearch for all field.
The new mapping template is created for nodes and for relationships.
It will add the ignore_malformed option for all fields, so that fields that are detected as numbers will not throw if updated with a non-numerical value (same for dates).

Human readable mapping:

// nodes
{
  // EXISTING MAPPING
  "properties": {
    "_label": {
      "type": "string",
      "fields": {
        "raw": {
          "type": "string",
          "index": "not_analyzed", // not analyzed, indexed as-is
          "include_in_all": false // avoid matching/returning this field by accident
        }
      }
    }
  },
  // ADDED MAPPING TEMPLATE
  dynamic_templates: [
    {
      "any_field": { // name of the template
        "match_mapping_type": "*", // will match any fields (including auto-detected)
        "mapping": {
          "type": "{dynamic_type}", // mandatory for dynamic mapping template
          "ignore_malformed": true  // does the magic
        }
      }
    }
  ]
}

This strategy could be used in other mappings as well.

@ikwattro
Copy link
Member

@davidrapin I'm afraid this kind of logic will become unmaintainable, either for you when encountering something new or for us in terms of code we haven't written and we don't use.

That said, I don't want to block you, I would rather suggest that we rewrite this logic in a different way.

What about we allow to load json ES mapping definitions from disk, and use this instead. It would be much more flexible, we could even change the mapping without restarting neo by the help of a procedure and we could even re-index everything, or a subset from a procedure as well ?

Thoughts ?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants