-
Notifications
You must be signed in to change notification settings - Fork 24.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ideas around mappings improvements #64663
Labels
Meta
:Search Foundations/Mapping
Index mappings, including merging and defining field types
Team:Search Foundations
Meta label for the Search Foundations team in Elasticsearch
>tech debt
Comments
javanna
added
:Search Foundations/Mapping
Index mappings, including merging and defining field types
Meta
labels
Nov 5, 2020
Pinging @elastic/es-search (:Search/Mapping) |
This was referenced Dec 8, 2020
I've started to group things into themes for easier digestion. I've tried to sort the things for themes in rough priority order. |
Thanks @nik9000 I plan to go through the list and re-sort things, possibly opening separate issues where we see fit. |
Pinging @elastic/es-search (Team:Search) |
javanna
changed the title
Search layer / mappings improvements
Ideas around mappings improvements
Oct 13, 2022
javanna
added
Team:Search Foundations
Meta label for the Search Foundations team in Elasticsearch
and removed
Team:Search
Meta label for search team
labels
Jul 16, 2024
Pinging @elastic/es-search-foundations (Team:Search Foundations) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Meta
:Search Foundations/Mapping
Index mappings, including merging and defining field types
Team:Search Foundations
Meta label for the Search Foundations team in Elasticsearch
>tech debt
This meta issue tracks some of the improvements and cleanups that we identified while working on the runtime fields project (#59332):
SearchContext#mapperService
: it is used for nested objectsQueryShardContext#getObjectMapper
: it is mostly used for nested objects and it indirectly exposes retrievingMappedFieldType
going throughFieldMapper
s, which are not aware of runtime fieldsSearchLookup
handling inSearchExecutionContext
: a separate lookup object needs to be created for the fetch phase which is passed through toMappedFieldFype#valueFetcher
, yet the (wrong) lookup can also be retrieved directly from theSearchExecutionContext
NestedScope
onSearchExecutionContext
allowUnmappedFields
andmapUnmappedAsText
inSearchExecutionContext
, they can be changed from consumers which seems dangerous and are only needed for the percolator.SearchExecutionContext#getFieldType
returns. They most likely need to be converted to useQueryShardCOntext#isFieldMapped
instead which does not take theallowUnmappedField
andmapUnmappedAsString
flags into accountFieldsVisitor#postProcess
and make sure that they are all necessary, in some cases we may be able to skip the call as it does nothing depending on which fields we are processingMapperService
exposes theDocumentMapper
through which the mappings can be accessed. The fact thatDocumentMapper
can be null leads to null checks in many places and potential NPEs in callers. We'd like to eagerly create a document mapper for a new index, as soon as it gets created. This has the implication that we would lose the semantics around the fact that when a document mapper is null for an index, it means the index is empty.That would also allow us to apply index sorting when the index is created instead of delaying it to the document mapper creation.
MapperRegistry
toindex.mapper
package? (Move MapperRegistry to index.mapper package #69805)Mapping
is visible from outside of its package but all of its methods are called from within the same package. Review usages and possibly adjust methods visibility accordingly. Also, its instance fields are all package private and accessed directly without going through getters, yet the root object mapper has also a public getter that is only used within the same package. (Remove redundant methods from DocumentMapper #69803)ContentPath
less confusingObjectMapper
andRootObjectMapper
to not rely onclone
merge
methods onMapperService
, and clarify when they should be calledMapperService
: ideally all of the merge methods exposed byMapperService
can be factored out to a lightweightMappingMerger
component (merge itself was already removed fromDocumentMapper
and replaced by callingMapping#merge
directly). This means that it will no longer be required to build a fullMapperService
in the master node to execute merges when the indices are not allocated to it.NestedDocuments
to depend onMappingLookup
Use a mapping snapshot for fetching nested docs #66877 (@nik9000)Caching cleanups
MapperService
fromSearchExecutionContext
entirely. We'd prefer there not be an easy way for the context to get a new snapshot of the mapping. OTOH we don't really want to merge all of the immutable mapping stuff intoMappingLookup
. We have to decide how to remove it.SearchContext
's reference toMapperService
. This includes exposingNestedDocuments
inSearchExecutionContext
so thatDefaultSearchContext
takes it from there. And cleaning up access toindexService.mapperService()
inDefaultSearchContext
MappingLookup
, which hides a reference toDocumentMapper
.MapperService
through#mappingLookup()
to make sure we perform a single volatile read. Or we could remove them entirely in favor the caller going throughmappingLookup()
. If the caller keeps themappingLookup
for the duration of the operation then the caller will get a consistent snapshot of the mapping during the entire process. This can potentially become a big change but it can be done in steps, for instance we can start from duplicated methods that are exposed inMapperService
andDocumentMapper
(e.g.simpleMatchToFullName
), that could be rather accessed through retrievingMappingLookup
.MapperService#getObjectMapper
is only used in tests, it can probably be replaced byMappingLookup#getObjectMapper
FieldMappers
"obviously" immutable.The text was updated successfully, but these errors were encountered: