Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TDigest & HdrHistogram fork #95355

Closed
salvatore-campagna opened this issue Apr 19, 2023 · 3 comments
Closed

TDigest & HdrHistogram fork #95355

salvatore-campagna opened this issue Apr 19, 2023 · 3 comments
Assignees
Labels
:Analytics/Aggregations Aggregations >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)

Comments

@salvatore-campagna
Copy link
Contributor

Description

We would like to have our own fork for both the following libraries:

There are two main reasons behind this choice:

  1. We would like to control semantic version and backward compatibility according to our definition. Right now, for instance, TDigest does not match our usage of semantic versioning when changing the library code and that makes upgrading quite challenging because exposes us to backward compatibility issues.
  2. We would like to change those libraries to use some specific Elasticsearch libraries/tools/frameworks such as BigArrays. Right now when running some aggregations (percentiles, boxplot,...) we experience OOMs due to large memory usage. Using BigArrays, for instance, would allow us to deal with OOMs using Circuit Breakers.
@salvatore-campagna salvatore-campagna added >enhancement :Analytics/Aggregations Aggregations Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) labels Apr 19, 2023
@salvatore-campagna salvatore-campagna self-assigned this Apr 19, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (Team:Analytics)

@wchaparro
Copy link
Member

Should we split this issue? @kkrik-es @salvatore-campagna @martijnvg

@martijnvg
Copy link
Member

Closing in favour of #95903 and #95904

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/Aggregations Aggregations >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)
Projects
None yet
Development

No branches or pull requests

5 participants