Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Flaky Test][exporter/signalfx] internal/utils TestHistogramsAreRetrieved: Actual metrics don't match expected #31493

Closed
crobert-1 opened this issue Feb 29, 2024 · 5 comments · Fixed by #31522
Closed

[Flaky Test][exporter/signalfx] internal/utils TestHistogramsAreRetrieved: Actual metrics don't match expected #31493

crobert-1 opened this issue Feb 29, 2024 · 5 comments · Fixed by #31522
Assignees
@jinja2
Labels
exporter/signalfx flaky test a test is flaky

Comments

@crobert-1
Copy link
Member

Component(s)

exporter/signalfx

Describe the issue you're reporting

Failing CI run: https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/8054170863/job/21998229771?pr=31176

Failure output:

✖  internal/utils (20ms) (coverage: 100.0% of statements)

=== Failed
=== FAIL: internal/utils TestHistogramsAreRetrieved/mixed_type_multiple_scopes (0.00s)
    histogram_utils_test.go:307: 
make[2]: *** [../../Makefile.Common:131: test-with-cover] Error 1
        	Error Trace:	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/signalfxexporter/internal/utils/histogram_utils_test.go:307
        	Error:      	Not equal: 
        	            	expected: pmetric.Metrics{orig:(*v1.ExportMetricsServiceRequest)(0xc00019d470), state:(*internal.State)(0xc00034b4dc)}
        	            	actual  : pmetric.Metrics{orig:(*v1.ExportMetricsServiceRequest)(0xc00019d458), state:(*internal.State)(0xc00034b38c)}
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -19,35 +19,2 @@
        	            	     ScopeMetrics: ([]*v1.ScopeMetrics) (len=2) {
        	            	-     (*v1.ScopeMetrics)({
        	            	-      Scope: (v1.InstrumentationScope) {
        	            	-       Name: (string) (len=10) "Scope name",
        	            	-       Version: (string) (len=13) "Scope version",
        	            	-       Attributes: ([]v1.KeyValue) (len=1) {
        	            	-        (v1.KeyValue) {
        	            	-         Key: (string) (len=3) "ks0",
        	            	-         Value: (v1.AnyValue) {
        	            	-          Value: (*v1.AnyValue_StringValue)({
        	            	-           <max depth reached>
        	            	-          })
        	            	-         }
        	            	-        }
        	            	-       },
        	            	-       DroppedAttributesCount: (uint32) 0
        	            	-      },
        	            	-      Metrics: ([]*v1.Metric) (len=1) {
        	            	-       (*v1.Metric)({
        	            	-        Name: (string) (len=11) "histogram_1",
        	            	-        Description: (string) "",
        	            	-        Unit: (string) (len=1) "1",
        	            	-        Data: (*v1.Metric_Histogram)({
        	            	-         Histogram: (*v1.Histogram)({
        	            	-          DataPoints: ([]*v1.HistogramDataPoint) (len=1) {
        	            	-           <max depth reached>
        	            	-          },
        	            	-          AggregationTemporality: (v1.AggregationTemporality) 1
        	            	-         })
        	            	-        })
        	            	-       })
        	            	-      },
        	            	-      SchemaUrl: (string) (len=15) "Scope SchemaUrl"
        	            	-     }),
        	            	      (*v1.ScopeMetrics)({
        	            	@@ -97,2 +64,35 @@
        	            	       SchemaUrl: (string) (len=15) "Scope SchemaUrl"
        	            	+     }),
        	            	+     (*v1.ScopeMetrics)({
        	            	+      Scope: (v1.InstrumentationScope) {
        	            	+       Name: (string) (len=10) "Scope name",
        	            	+       Version: (string) (len=13) "Scope version",
        	            	+       Attributes: ([]v1.KeyValue) (len=1) {
        	            	+        (v1.KeyValue) {
        	            	+         Key: (string) (len=3) "ks0",
        	            	+         Value: (v1.AnyValue) {
        	            	+          Value: (*v1.AnyValue_StringValue)({
        	            	+           <max depth reached>
        	            	+          })
        	            	+         }
        	            	+        }
        	            	+       },
        	            	+       DroppedAttributesCount: (uint32) 0
        	            	+      },
        	            	+      Metrics: ([]*v1.Metric) (len=1) {
        	            	+       (*v1.Metric)({
        	            	+        Name: (string) (len=11) "histogram_1",
        	            	+        Description: (string) "",
        	            	+        Unit: (string) (len=1) "1",
        	            	+        Data: (*v1.Metric_Histogram)({
        	            	+         Histogram: (*v1.Histogram)({
        	            	+          DataPoints: ([]*v1.HistogramDataPoint) (len=1) {
        	            	+           <max depth reached>
        	            	+          },
        	            	+          AggregationTemporality: (v1.AggregationTemporality) 1
        	            	+         })
        	            	+        })
        	            	+       })
        	            	+      },
        	            	+      SchemaUrl: (string) (len=15) "Scope SchemaUrl"
        	            	      })
        	Test:       	TestHistogramsAreRetrieved/mixed_type_multiple_scopes

=== FAIL: internal/utils TestHistogramsAreRetrieved (0.00s)

=== FAIL: internal/utils TestHistogramsAreRetrieved/mixed_type_multiple_scopes (re-run 1) (0.00s)
    histogram_utils_test.go:307: 
        	Error Trace:	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/signalfxexporter/internal/utils/histogram_utils_test.go:307
        	Error:      	Not equal: 
        	            	expected: pmetric.Metrics{orig:(*v1.ExportMetricsServiceRequest)(0xc000125308), state:(*internal.State)(0xc0002e2bac)}
        	            	actual  : pmetric.Metrics{orig:(*v1.ExportMetricsServiceRequest)(0xc0001252f0), state:(*internal.State)(0xc0002e2a5c)}
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -19,35 +19,2 @@
        	            	     ScopeMetrics: ([]*v1.ScopeMetrics) (len=2) {
        	            	-     (*v1.ScopeMetrics)({
        	            	-      Scope: (v1.InstrumentationScope) {
        	            	-       Name: (string) (len=10) "Scope name",
        	            	-       Version: (string) (len=13) "Scope version",
        	            	-       Attributes: ([]v1.KeyValue) (len=1) {
        	            	-        (v1.KeyValue) {
        	            	-         Key: (string) (len=3) "ks0",
        	            	-         Value: (v1.AnyValue) {
        	            	-          Value: (*v1.AnyValue_StringValue)({
        	            	-           <max depth reached>
        	            	-          })
        	            	-         }
        	            	-        }
        	            	-       },
        	            	-       DroppedAttributesCount: (uint32) 0
        	            	-      },
        	            	-      Metrics: ([]*v1.Metric) (len=1) {
        	            	-       (*v1.Metric)({
        	            	-        Name: (string) (len=11) "histogram_1",
        	            	-        Description: (string) "",
        	            	-        Unit: (string) (len=1) "1",
        	            	-        Data: (*v1.Metric_Histogram)({
        	            	-         Histogram: (*v1.Histogram)({
        	            	-          DataPoints: ([]*v1.HistogramDataPoint) (len=1) {
        	            	-           <max depth reached>
        	            	-          },
        	            	-          AggregationTemporality: (v1.AggregationTemporality) 1
        	            	-         })
        	            	-        })
        	            	-       })
        	            	-      },
        	            	-      SchemaUrl: (string) (len=15) "Scope SchemaUrl"
        	            	-     }),
        	            	      (*v1.ScopeMetrics)({
        	            	@@ -97,2 +64,35 @@
        	            	       SchemaUrl: (string) (len=15) "Scope SchemaUrl"
        	            	+     }),
        	            	+     (*v1.ScopeMetrics)({
        	            	+      Scope: (v1.InstrumentationScope) {
        	            	+       Name: (string) (len=10) "Scope name",
        	            	+       Version: (string) (len=13) "Scope version",
        	            	+       Attributes: ([]v1.KeyValue) (len=1) {
        	            	+        (v1.KeyValue) {
        	            	+         Key: (string) (len=3) "ks0",
        	            	+         Value: (v1.AnyValue) {
        	            	+          Value: (*v1.AnyValue_StringValue)({
        	            	+           <max depth reached>
        	            	+          })
        	            	+         }
        	            	+        }
        	            	+       },
        	            	+       DroppedAttributesCount: (uint32) 0
        	            	+      },
        	            	+      Metrics: ([]*v1.Metric) (len=1) {
        	            	+       (*v1.Metric)({
        	            	+        Name: (string) (len=11) "histogram_1",
        	            	+        Description: (string) "",
        	            	+        Unit: (string) (len=1) "1",
        	            	+        Data: (*v1.Metric_Histogram)({
        	            	+         Histogram: (*v1.Histogram)({
        	            	+          DataPoints: ([]*v1.HistogramDataPoint) (len=1) {
        	            	+           <max depth reached>
        	            	+          },
        	            	+          AggregationTemporality: (v1.AggregationTemporality) 1
        	            	+         })
        	            	+        })
        	            	+       })
        	            	+      },
        	            	+      SchemaUrl: (string) (len=15) "Scope SchemaUrl"
        	            	      })
        	Test:       	TestHistogramsAreRetrieved/mixed_type_multiple_scopes

=== FAIL: internal/utils TestHistogramsAreRetrieved (re-run 1) (0.00s)

DONE 2 runs, 393 tests, 4 failures in 26.324s
make[1]: *** [Makefile:[165](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/8054170863/job/21998229771?pr=31176#step:9:166): exporter/signalfxexporter] Error 2
make: *** [Makefile:117: gotest-with-cover] Error 2
make[1]: Leaving directory '/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib'
Error: Process completed with exit code 2.
@crobert-1 crobert-1 added the needs triage New item requiring triage label Feb 29, 2024
@crobert-1 crobert-1 mentioned this issue Feb 29, 2024
Closed
@github-actions GitHub Actions
Copy link
Contributor

Pinging code owners for exporter/signalfx: @dmitryax @crobert-1. See Adding Labels via Comments if you do not have permissions to add labels yourself.

@crobert-1 crobert-1 changed the title [flaky test][exporter/signalfx] internal/utils TestHistogramsAreRetrieved: Expected metrics don't match expected [Flaky Test][exporter/signalfx] internal/utils TestHistogramsAreRetrieved: Actual metrics don't match expected Feb 29, 2024
@crobert-1
Copy link
Member Author

+1 freq: https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/8100930861/job/22140085236?pr=31175

@crobert-1 crobert-1 mentioned this issue Feb 29, 2024
Merged
@crobert-1
Copy link
Member Author

+1 freq: https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/8100382347/job/22138384030?pr=30492

@jinja2
Copy link
Contributor

jinja2 commented Mar 1, 2024

I'll push a fix

@jinja2 jinja2 mentioned this issue Mar 1, 2024
Merged
@crobert-1 crobert-1 removed the needs triage New item requiring triage label Mar 1, 2024
@crobert-1
Copy link
Member Author

Removing needs triage as a PR with the fix has been posted

codeboten pushed a commit that referenced this issue Mar 1, 2024
@jinja2
[chore][exporter/signalfx] fix tests (#31522)
3401d8a 
Update test cases to ignore the order of resource/scope metrics since
the returned histograms-only pmetrics can have different ordering for
resource and scope metrics

Fixes
#31493
This was referenced Mar 5, 2024
Closed
Open
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
@jinja2 @XinRanZhAWS
[chore][exporter/signalfx] fix tests (open-telemetry#31522)
53295a7 
Update test cases to ignore the order of resource/scope metrics since
the returned histograms-only pmetrics can have different ordering for
resource and scope metrics

Fixes
open-telemetry#31493
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jinja2 jinja2

Labels
exporter/signalfx flaky test a test is flaky
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[chore][exporter/signalfx] fix tests jinja2/opentelemetry-collector-contrib
2 participants
@jinja2 @crobert-1