Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrade t-digest and HdrHistogram library versions #92767

Closed
wants to merge 18 commits into from
Closed

Upgrade t-digest and HdrHistogram library versions #92767

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
f1319a9
fix: upgrade t-digest and HdrHistogram library versions
salvatore-campagna Jan 5, 2023
eb7ed7a
Update docs/changelog/92767.yaml
salvatore-campagna Jan 9, 2023
53948b6
fix: missing closing brace
salvatore-campagna Jan 9, 2023
2b19838
fix: add close_to feature skip
salvatore-campagna Jan 9, 2023
ef9b5da
fix: remove duplicate test
salvatore-campagna Jan 9, 2023
0c3da93
fix: incorrect percentile value
salvatore-campagna Jan 9, 2023
14944f5
fix: use close_to
salvatore-campagna Jan 9, 2023
f613028
fix: use close_to in place of match
salvatore-campagna Jan 9, 2023
8515c0f
fix: adjust some more approximate quantile values
salvatore-campagna Jan 9, 2023
8395f28
fix: unnecessary exclusion
salvatore-campagna Jan 10, 2023
d123787
fix: moving percentiles
salvatore-campagna Jan 10, 2023
4c7153a
fix: long median absolute deviation
salvatore-campagna Jan 10, 2023
bd549e2
fix: avg median absolute deviation
salvatore-campagna Jan 10, 2023
49877ae
fix: constant median absolute deviation
salvatore-campagna Jan 10, 2023
d65aec5
fix: long percentiles
salvatore-campagna Jan 10, 2023
7c9aea2
fix: aggregate functions with scalar
salvatore-campagna Jan 10, 2023
07cfcb1
fix: field alias percentiles
salvatore-campagna Jan 10, 2023
98acc3b
fix: skip testing before version 8.7.0
salvatore-campagna Jan 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions docs/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1355,10 +1355,26 @@ setups['reviews'] = '''
index: reviews
refresh: true
body: |
{"index": {"_id": "0"}}
{"product": "widget-foo", "rating": 4}
{"index": {"_id": "1"}}
{"product": "widget-foo", "rating": 1}
{"index": {"_id": "2"}}
{"product": "widget-foo", "rating": 5}
{"index": {"_id": "3"}}
{"product": "widget-foo", "rating": 2}
{"index": {"_id": "4"}}
{"product": "widget-foo", "rating": 1}
{"index": {"_id": "5"}}
{"product": "widget-foo", "rating": 4}
{"index": {"_id": "6"}}
{"product": "widget-foo", "rating": 1}
{"index": {"_id": "7"}}
{"product": "widget-foo", "rating": 2}
{"index": {"_id": "8"}}
{"product": "widget-foo", "rating": 5}
{"index": {"_id": "9"}}
{"product": "widget-foo", "rating": 5}
'''
setups['remote_cluster'] = setups['host'] + '''
- do:
Expand Down
5 changes: 5 additions & 0 deletions docs/changelog/92767.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 92767
summary: Upgrade t-digest and `HdrHistogram` library versions
area: Aggregations
type: upgrade
issues: []
12 changes: 6 additions & 6 deletions docs/reference/aggregations/metrics/boxplot-aggregation.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,9 @@ The response will look like this:
"load_time_boxplot": {
"min": 0.0,
"max": 990.0,
"q1": 165.0,
"q2": 445.0,
"q3": 725.0,
"q1": 170.0,
"q2": 450.0,
"q3": 730.0,
"lower": 0.0,
"upper": 990.0
}
Expand Down Expand Up @@ -114,9 +114,9 @@ GET latency/_search
"load_time_boxplot": {
"min": 0.0,
"max": 99.0,
"q1": 16.5,
"q2": 44.5,
"q3": 72.5,
"q1": 17.0,
"q2": 45.0,
"q3": 73.0,
"lower": 0.0,
"upper": 99.0
}
Expand Down
42 changes: 21 additions & 21 deletions docs/reference/aggregations/metrics/percentile-aggregation.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,13 +54,13 @@ percentiles: `[ 1, 5, 25, 50, 75, 95, 99 ]`. The response will look like this:
"aggregations": {
"load_time_outlier": {
"values": {
"1.0": 5.0,
"5.0": 25.0,
"25.0": 165.0,
"50.0": 445.0,
"75.0": 725.0,
"95.0": 945.0,
"99.0": 985.0
"1.0": 10.0,
"5.0": 30.0,
"25.0": 170.0,
"50.0": 450.0,
"75.0": 730.0,
"95.0": 950.0,
"99.0": 990.0
}
}
}
Expand Down Expand Up @@ -128,31 +128,31 @@ Response:
"values": [
{
"key": 1.0,
"value": 5.0
"value": 10.0
},
{
"key": 5.0,
"value": 25.0
"value": 30.0
},
{
"key": 25.0,
"value": 165.0
"value": 170.0
},
{
"key": 50.0,
"value": 445.0
"value": 450.0
},
{
"key": 75.0,
"value": 725.0
"value": 730.0
},
{
"key": 95.0,
"value": 945.0
"value": 950.0
},
{
"key": 99.0,
"value": 985.0
"value": 990.0
}
]
}
Expand Down Expand Up @@ -203,13 +203,13 @@ GET latency/_search
"aggregations": {
"load_time_outlier": {
"values": {
"1.0": 0.5,
"5.0": 2.5,
"25.0": 16.5,
"50.0": 44.5,
"75.0": 72.5,
"95.0": 94.5,
"99.0": 98.5
"1.0": 1.0,
"5.0": 3.0,
"25.0": 17.0,
"50.0": 45.0,
"75.0": 73.0,
"95.0": 95.0,
"99.0": 99.0
}
}
}
Expand Down
12 changes: 6 additions & 6 deletions gradle/verification-metadata.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1019,9 +1019,9 @@
<sha256 value="fa3e1499b192c310312bf02881274b68394aaea4c9563e6c554cc406ae644ff8" origin="Generated by Gradle"/>
</artifact>
</component>
<component group="com.tdunning" name="t-digest" version="3.2">
<artifact name="t-digest-3.2.jar">
<sha256 value="03db291a8887b474f90db67bfb1f92d084e990150037e231babbb374ee11d7c3" origin="Generated by Gradle"/>
<component group="com.tdunning" name="t-digest" version="3.3">
<artifact name="t-digest-3.3.jar">
<sha256 value="dc8be5228e0733e12fbe35eeffb6f720dff576bb2eb5dee69c1c211adcd7aeb9" origin="Generated by Gradle"/>
</artifact>
</component>
<component group="com.thoughtworks.paranamer" name="paranamer" version="2.3">
Expand Down Expand Up @@ -3354,9 +3354,9 @@
<sha256 value="66fdef91e9739348df7a096aa384a5685f4e875584cce89386a7a47251c4d8e9" origin="Generated by Gradle"/>
</artifact>
</component>
<component group="org.hdrhistogram" name="HdrHistogram" version="2.1.9">
<artifact name="HdrHistogram-2.1.9.jar">
<sha256 value="95d40913be28dfd439cefea9170c40898ea84f11f25e6ff8de50339b8a7b5e3e" origin="Generated by Gradle"/>
<component group="org.hdrhistogram" name="HdrHistogram" version="2.1.12">
<artifact name="HdrHistogram-2.1.12.jar">
<sha256 value="9b47fbae444feaac4b7e04f0ea294569e4bc282bc69d8c2ce2ac3f23577281e2" origin="Generated by Gradle"/>
</artifact>
</component>
<component group="org.ini4j" name="ini4j" version="0.5.2">
Expand Down
28 changes: 14 additions & 14 deletions ...rc/yamlRestTest/resources/rest-api-spec/test/aggregations/percentile_ranks_hdr_metric.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,10 @@ basic:
hdr: {}

- match: { hits.total.value: 4 }
- close_to: { aggregations.percentile_ranks_int.values.50\\.0: { value: 25.0, error: 1} }
- close_to: { aggregations.percentile_ranks_int.values.99\\.0: { value: 50.0, error: 1} }
- close_to: { aggregations.percentile_ranks_double.values.50\\.0: { value: 25.0, error: 1} }
- close_to: { aggregations.percentile_ranks_double.values.99\\.0: { value: 50.0, error: 1} }
- close_to: { aggregations.percentile_ranks_int.values.50\\.0: { value: 25.0, error: 0.1} }
- close_to: { aggregations.percentile_ranks_int.values.99\\.0: { value: 50.0, error: 0.1} }
- close_to: { aggregations.percentile_ranks_double.values.50\\.0: { value: 25.0, error: 0.1} }
- close_to: { aggregations.percentile_ranks_double.values.99\\.0: { value: 50.0, error: 0.1} }

---
set significant digits:
Expand All @@ -79,10 +79,10 @@ set significant digits:
number_of_significant_value_digits: 3

- match: { hits.total.value: 4 }
- close_to: { aggregations.percentile_ranks_int.values.50\\.0: { value: 25.0, error: 1} }
- close_to: { aggregations.percentile_ranks_int.values.99\\.0: { value: 50.0, error: 1} }
- close_to: { aggregations.percentile_ranks_double.values.50\\.0: { value: 25.0, error: 1} }
- close_to: { aggregations.percentile_ranks_double.values.99\\.0: { value: 50.0, error: 1} }
- close_to: { aggregations.percentile_ranks_int.values.50\\.0: { value: 25.0, error: 0.1} }
- close_to: { aggregations.percentile_ranks_int.values.99\\.0: { value: 50.0, error: 0.1} }
- close_to: { aggregations.percentile_ranks_double.values.50\\.0: { value: 25.0, error: 0.1} }
- close_to: { aggregations.percentile_ranks_double.values.99\\.0: { value: 50.0, error: 0.1} }

---
filtered:
Expand Down Expand Up @@ -112,8 +112,8 @@ filtered:
number_of_significant_value_digits: 3

- match: { hits.total.value: 3 }
- close_to: { aggregations.percentile_ranks_int.values.50\\.0: { value: 0.0, error: 1} }
- close_to: { aggregations.percentile_ranks_double.values.50\\.0: { value: 0.0, error: 1} }
- close_to: { aggregations.percentile_ranks_int.values.50\\.0: { value: 0.0, error: 0.1} }
- close_to: { aggregations.percentile_ranks_double.values.50\\.0: { value: 0.0, error: 0.1} }

---
missing field with missing param:
Expand All @@ -134,8 +134,8 @@ missing field with missing param:
number_of_significant_value_digits: 3

- match: { hits.total.value: 4 }
- close_to: { aggregations.percentile_ranks_missing.values.50\\.0: { value: 100.0, error: 1} }
- close_to: { aggregations.percentile_ranks_missing.values.99\\.0: { value: 100.0, error: 1} }
- close_to: { aggregations.percentile_ranks_missing.values.50\\.0: { value: 100.0, error: 0.1} }
- close_to: { aggregations.percentile_ranks_missing.values.99\\.0: { value: 100.0, error: 0.1} }

---
missing field without missing param:
Expand Down Expand Up @@ -224,6 +224,6 @@ non-keyed:

- match: { hits.total.value: 4 }
- match: { aggregations.percentile_ranks_int.values.0.key: 50}
- close_to: { aggregations.percentile_ranks_int.values.0.value: { value: 25, error: 1} }
- close_to: { aggregations.percentile_ranks_int.values.0.value: { value: 25, error: 0.1} }
- match: { aggregations.percentile_ranks_int.values.1.key: 99}
- close_to: { aggregations.percentile_ranks_int.values.1.value: { value: 50, error: 1} }
- close_to: { aggregations.percentile_ranks_int.values.1.value: { value: 50, error: 0.1} }
51 changes: 32 additions & 19 deletions ...amlRestTest/resources/rest-api-spec/test/aggregations/percentile_ranks_tdigest_metric.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,17 +41,21 @@ basic:
percentile_ranks_int:
percentile_ranks:
field: int
values: [50, 99]
values: [50, 100, 150, 200]
percentile_ranks_double:
percentile_ranks:
field: double
values: [50, 99]
values: [50, 100, 150, 200]

- match: { hits.total.value: 4 }
- close_to: { aggregations.percentile_ranks_int.values.50\\.0: { value: 37.0, error: 1} }
- close_to: { aggregations.percentile_ranks_int.values.99\\.0: { value: 61.5, error: 1} }
- close_to: { aggregations.percentile_ranks_double.values.50\\.0: { value: 37.0, error: 1} }
- close_to: { aggregations.percentile_ranks_double.values.99\\.0: { value: 61.5, error: 1} }
- close_to: { aggregations.percentile_ranks_int.values.50\\.0: { value: 25.0, error: 0.1 } }
- close_to: { aggregations.percentile_ranks_int.values.100\\.0: { value: 50.0, error: 0.1 } }
- close_to: { aggregations.percentile_ranks_int.values.150\\.0: { value: 75.0, error: 0.1 } }
- close_to: { aggregations.percentile_ranks_int.values.200\\.0: { value: 100.0, error: 0.1 } }
- close_to: { aggregations.percentile_ranks_double.values.50\\.0: { value: 25.0, error: 0.1 } }
- close_to: { aggregations.percentile_ranks_double.values.100\\.0: { value: 50.0, error: 0.1 } }
- close_to: { aggregations.percentile_ranks_double.values.150\\.0: { value: 75.0, error: 0.1 } }
- close_to: { aggregations.percentile_ranks_double.values.200\\.0: { value: 100.0, error: 0.1 } }

---
filtered:
Expand All @@ -70,15 +74,21 @@ filtered:
percentile_ranks_int:
percentile_ranks:
field: int
values: [50]
values: [50, 100, 150, 200]
percentile_ranks_double:
percentile_ranks:
field: double
values: [50]
values: [50, 100, 150, 200]

- match: { hits.total.value: 3 }
- close_to: { aggregations.percentile_ranks_int.values.50\\.0: { value: 16.0, error: 1} }
- close_to: { aggregations.percentile_ranks_double.values.50\\.0: { value: 16.0, error: 1} }
- close_to: { aggregations.percentile_ranks_int.values.50\\.0: { value: 0.0, error: 0.1 } }
- close_to: { aggregations.percentile_ranks_int.values.100\\.0: { value: 33.3, error: 0.1 } }
- close_to: { aggregations.percentile_ranks_int.values.150\\.0: { value: 66.6, error: 0.1 } }
- close_to: { aggregations.percentile_ranks_int.values.200\\.0: { value: 100.0, error: 0.1 } }
- close_to: { aggregations.percentile_ranks_double.values.50\\.0: { value: 0.0, error: 0.1 } }
- close_to: { aggregations.percentile_ranks_double.values.100\\.0: { value: 33.3, error: 0.1 } }
- close_to: { aggregations.percentile_ranks_double.values.150\\.0: { value: 66.6, error: 0.1 } }
- close_to: { aggregations.percentile_ranks_double.values.200\\.0: { value: 100.0, error: 0.1 } }

---
missing field with missing param:
Expand All @@ -97,9 +107,9 @@ missing field with missing param:
values: [50, 99]

- match: { hits.total.value: 4 }
- close_to: { aggregations.percentile_ranks_missing.values.50\\.0: { value: 100.0, error: 1} }
- close_to: { aggregations.percentile_ranks_missing.values.99\\.0: { value: 100.0, error: 1} }
- close_to: { aggregations.percentile_ranks_missing.values.50\\.0: { value: 100.0, error: 0.1} }
- close_to: { aggregations.percentile_ranks_missing.values.99\\.0: { value: 100.0, error: 0.1} }

---
missing field without missing param:
- do:
Expand Down Expand Up @@ -169,12 +179,15 @@ non-keyed test:
percentile_ranks_int:
percentile_ranks:
field: int
values: [50, 99]
values: [50, 100, 150, 200]
keyed: false

- match: { hits.total.value: 4 }
- match: { aggregations.percentile_ranks_int.values.0.key: 50}
- close_to: { aggregations.percentile_ranks_int.values.0.value: { value: 37.0, error: 1} }
- match: { aggregations.percentile_ranks_int.values.1.key: 99}
- close_to: { aggregations.percentile_ranks_int.values.1.value: { value: 61.5, error: 1} }

- match: { aggregations.percentile_ranks_int.values.0.key: 50 }
- close_to: { aggregations.percentile_ranks_int.values.0.value: { value: 25.0, error: 0.1 } }
- match: { aggregations.percentile_ranks_int.values.1.key: 100 }
- close_to: { aggregations.percentile_ranks_int.values.1.value: { value: 50.0, error: 0.1 } }
- match: { aggregations.percentile_ranks_int.values.2.key: 150 }
- close_to: { aggregations.percentile_ranks_int.values.2.value: { value: 75.0, error: 0.1 } }
- match: { aggregations.percentile_ranks_int.values.3.key: 200 }
- close_to: { aggregations.percentile_ranks_int.values.3.value: { value: 100.0, error: 0.1 } }
Loading