[API] Add synchronous gauge #3029

uuzay · 2024-08-14T16:42:58Z

Fixes #2279

Changes

Added synchronous gauge implementation according to the specification.

For significant contributions please make sure you have completed the following items:

CHANGELOG.md updated for non-trivial changes
Unit tests have been added
Changes in public API reviewed

codecov · 2024-08-14T17:01:41Z

Codecov Report

Attention: Patch coverage is 50.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 87.88%. Comparing base (497eaf4) to head (56bdf77).
Report is 151 commits behind head on main.

Files with missing lines	Patch %	Lines
sdk/src/metrics/state/metric_collector.cc	60.00%	2 Missing ⚠️
...etry/sdk/metrics/aggregation/default_aggregation.h	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3029      +/-   ##
==========================================
+ Coverage   87.12%   87.88%   +0.76%     
==========================================
  Files         200      195       -5     
  Lines        6109     6137      +28     
==========================================
+ Hits         5322     5393      +71     
+ Misses        787      744      -43

Files with missing lines	Coverage Δ
api/include/opentelemetry/metrics/meter.h	`100.00% <ø> (ø)`
api/include/opentelemetry/metrics/noop.h	`41.18% <ø> (ø)`
...i/include/opentelemetry/metrics/sync_instruments.h	`100.00% <ø> (ø)`
sdk/include/opentelemetry/sdk/metrics/meter.h	`57.15% <ø> (ø)`
...clude/opentelemetry/sdk/metrics/sync_instruments.h	`100.00% <ø> (ø)`
sdk/src/metrics/meter.cc	`62.36% <ø> (+0.91%)`	⬆️
sdk/src/metrics/sync_instruments.cc	`65.84% <ø> (ø)`
...etry/sdk/metrics/aggregation/default_aggregation.h	`70.13% <0.00%> (ø)`
sdk/src/metrics/state/metric_collector.cc	`87.10% <60.00%> (-5.76%)`	⬇️

... and 94 files with indirect coverage changes

marcalff

Thanks for the feature.

This is not a full review yet,
only some preliminary comments.

Please fix:

declarations of new virtual methods using ABI version 2
build break issues found in CI

More extensive review to follow.

api/include/opentelemetry/metrics/meter.h

marcalff

Thanks for the feature.

The code structure looks good to me overall.

I defer to @lalitb to check the logic around aggregation temporality in particular,
as I am not too familiar with this area.

sdk/include/opentelemetry/sdk/metrics/instruments.h

lalitb · 2024-08-26T17:53:59Z

@uuzay Thanks for the PR. Had a quick glance to the code, this looks similar to #2341 (correct me if not). And I believe this won't work for delta temporality. To validate, can we update the PR to test for it?

lalitb · 2024-08-27T05:03:56Z

sdk/test/metrics/sync_metric_storage_gauge_test.cc

+
+INSTANTIATE_TEST_SUITE_P(WritableMetricStorageTestDouble,
+                         WritableMetricStorageTestFixture,
+                         ::testing::Values(AggregationTemporality::kCumulative));


Update the test here to include the delta temporality.

punya · 2024-08-30T19:47:58Z

@uuzay Thanks for the PR. Had a quick glance to the code, this looks similar to #2341 (correct me if not). And I believe this won't work for delta temporality. To validate, can we update the PR to test for it?

If I understand the relevant section of the spec correctly, Gauges (unlike UpDownCounters and Histograms) don't support a choice of temporality.

uuzay · 2024-09-05T10:52:00Z

I was thinking the same @punya.

It also mentions:

Gauges do not provide an aggregation semantic, instead “last sample value” is used when performing operations like temporal alignment or adjusting resolution. (source)

if that's related.

@lalitb can you please clarify if delta temporality is actually needed?

lalitb · 2024-09-05T11:01:08Z

@lalitb can you please clarify if delta temporality is actually needed?

Sorry i was not clear enough. I meant testing for temporality with last value aggregation. The current test is only doing for cumulative.

lalitb · 2024-10-03T19:12:24Z

@uuzay - Do you think we can modify the PR to restrict configuring the delta temporality for Last Value as of now - Just thrown an error at the startup if this is enabled through View or otherwise. With that, we should be good to go through this PR.

Thanks for working on this, and sorry about the long wait on this.

uuzay · 2024-10-08T11:56:36Z

Hi @lalitb,
Sorry for the delays, I was both on a holiday period and also had other important tasks to take care of.

I'm planning to make the necessary changes regarding the ABI macros this week.
For delta temporality, I think what you suggested makes sense for now.

netlify · 2024-10-15T13:58:38Z

✅ Deploy Preview for opentelemetry-cpp-api-docs canceled.

Name	Link
🔨 Latest commit	`56bdf77`
🔍 Latest deploy log	https://app.netlify.com/sites/opentelemetry-cpp-api-docs/deploys/67229beb935b030008bac3ad

uuzay · 2024-10-15T15:36:03Z

Hi @lalitb,
I'm trying to figure out where would be an appropriate place to throw the error and would appreciate some pointers.

I was thinking if SyncMetricStorage or TemporalMetricStorage constructor would be a good place.
Another idea that comes to mind is modifying DeltaTemporalitySelector in otlp_metric_utils to send a warning log and select cumulative temporality instead but I'm not sure if this covers all of the cases.

Thanks in advance.

lalitb · 2024-10-25T05:14:28Z

Hi @lalitb, I'm trying to figure out where would be an appropriate place to throw the error and would appreciate some pointers.

I was thinking if SyncMetricStorage or TemporalMetricStorage constructor would be a good place. Another idea that comes to mind is modifying DeltaTemporalitySelector in otlp_metric_utils to send a warning log and select cumulative temporality instead but I'm not sure if this covers all of the cases.

Thanks in advance.

@uuzay Yes that should work. Another easy place could be MetricCollector::GetAggregationTemporality. This part of code is called for both periodic reader and Prometheus, so should cover all exporters, and also in future when we support Delta for Prometheus.

marcalff · 2024-10-25T08:08:16Z

@uuzay Please merge a recent main into this PR, to get the merge conflict resolved.
This should force the CI to build also, somehow it got stuck.
Thanks.

lalitb

LGTM. Thanks for working on this.

marcalff

LGTM, thanks for the feature.

ThomsonTan · 2024-10-30T16:52:07Z

api/include/opentelemetry/metrics/sync_instruments.h

+  /**
+   * Record a value
+   *
+   * @param value The measurement value. May be positive, negative or zero.


not blocking the merge. Seems T should be integer or float types? If so, could it be asserted explicitly like below?

static_assert(std::is_same_v<T, int64_t> || std::is_same_v<T, double>, "Gauge<T> can only be instantiated with int64_t or double");

ThomsonTan · 2024-10-30T17:05:33Z

sdk/include/opentelemetry/sdk/metrics/instruments.h

@@ -19,6 +19,7 @@ enum class InstrumentType
  kCounter,
  kHistogram,
  kUpDownCounter,
+  kGauge,


Is inserting a new InstrumentType of kGauge a SDK breaking change and also break the ABI? Should it be included only for OPENTELEMETRY_ABI_VERSION_NO >= 2?

It won't be ABI breaking for API, as InstrumentType is not part of API surface. And we don't guarantee SDK ABI compatibility. It could be breaking change for custom exporters which use the switch on this enum without handling default case, but this can be added in the changelog.

This has been discussed before:

#3029 (comment)

The code as written is fine:

no change in the API, this kGauge is part of the SDK

the SDK ABI is changed, and we do not guarantee SDK ABI compatibility

I don't think we should document that in the changelog for the SDK, by definition every SDK change can (and most of the time does) break the SDK ABI.

@ThomsonTan Please remove the change requested then, as this blocks the merge.

ThomsonTan

Want to make sure if the adding of kGauge is ABI breaking change.

ThomsonTan · 2024-10-30T20:14:21Z

Apologize for blocking the PR. I think this is the exactly the issue I raised before.

The user application builds fine with either new SDK headers/old SDK libs, or old SDK headers/new SDK libs, either one will trigger a runtime break which is very hard to investigate like memory corruption.

It is true that the SDK doesn't guarantee ABI compatibility, but we don't need break this if not necessary. In this specific case, either wrap the new kGauge in the ABI version 2 macro, or we can just append it to the end of the enum list, so we don't trigger an potential break.

Even it is documented that SDK doesn't support ABI combability, it may be ignored by common users as this actually works in most of the time. A better solution will be that we check the SDK version and SDK lib version, and raise error (like block linking) if both versions mismatch.

How to you think?

marcalff · 2024-10-30T20:47:20Z

The user application builds fine with either new SDK headers/old SDK libs, or old SDK headers/new SDK libs, either one will trigger a runtime break which is very hard to investigate like memory corruption.

The only way for this to happen is to link dynamically with a shared library/DLL for the SDK itself, using a library from a different version compared to header files used during the application build.

Any application compiling the with the SDK header files and linking statically with the matching SDK library will be just fine.

Wrapping kGauge with ifdef causes another list of issues as well, because user code may break as well if not using the same ifdef.

I think that we can move kGauge at the end of the enum, to minimize binary compatibility issues and in practice avoid breaks for this case alone.

sdk/include/opentelemetry/sdk/metrics/instruments.h

Move kGauge to the end, for better ABI compatibility.

marcalff · 2024-10-30T20:51:28Z

I think that we can move kGauge at the end of the enum, to minimize binary compatibility issues and in practice avoid breaks for this case alone.

Implemented as code review comment.

@ThomsonTan PTAL.

ThomsonTan · 2024-10-30T20:51:45Z

The user application builds fine with either new SDK headers/old SDK libs, or old SDK headers/new SDK libs, either one will trigger a runtime break which is very hard to investigate like memory corruption.

The only way for this to happen is to link dynamically with a shared library/DLL for the SDK itself, using a library from a different version compared to header files used during the application build.

Any application compiling the with the SDK header files and linking statically with the matching SDK library will be just fine.

Wrapping kGauge with ifdef causes another list of issues as well, because user code may break as well if not using the same ifdef.

I think that we can move kGauge at the end of the enum, to minimize binary compatibility issues and in practice avoid breaks for this case alone.

Looks good to me then, thanks.

Add synchronous gauge

63fa75e

uuzay requested a review from a team August 14, 2024 16:42

marcalff reviewed Aug 14, 2024

View reviewed changes

api/include/opentelemetry/metrics/meter.h Show resolved Hide resolved

Add ABI Version macro, update tests

095435a

ThomsonTan reviewed Aug 14, 2024

View reviewed changes

api/include/opentelemetry/metrics/meter.h Outdated Show resolved Hide resolved

uuzay added 2 commits August 14, 2024 23:13

Fix function description

bd8cb23

Fix formatting

eecf89f

uuzay requested a review from marcalff August 20, 2024 13:16

marcalff reviewed Aug 24, 2024

View reviewed changes

sdk/include/opentelemetry/sdk/metrics/instruments.h Outdated Show resolved Hide resolved

lalitb reviewed Aug 27, 2024

View reviewed changes

marcalff assigned lalitb Oct 2, 2024

Remove ABI macros from SDK.

60df532

uuzay requested a review from a team as a code owner October 15, 2024 13:58

uuzay added 3 commits October 29, 2024 09:47

Add error log for gauge delta temporality.

855f86d

Fix formatting

f84a896

Merge branch 'main' into sync_gauge_2279

c9e40a5

lalitb approved these changes Oct 30, 2024

View reviewed changes

marcalff approved these changes Oct 30, 2024

View reviewed changes

marcalff added the ok-to-merge The PR is ok to merge (has two approves or raised by a maintainer/approver and has one approve) label Oct 30, 2024

ThomsonTan reviewed Oct 30, 2024

View reviewed changes

ThomsonTan requested changes Oct 30, 2024

View reviewed changes

Merge branch 'main' into sync_gauge_2279

55afe83

marcalff reviewed Oct 30, 2024

View reviewed changes

sdk/include/opentelemetry/sdk/metrics/instruments.h Outdated Show resolved Hide resolved

marcalff reviewed Oct 30, 2024

View reviewed changes

sdk/include/opentelemetry/sdk/metrics/instruments.h Outdated Show resolved Hide resolved

Apply suggestions from code review

56bdf77

Move kGauge to the end, for better ABI compatibility.

ThomsonTan approved these changes Oct 30, 2024

View reviewed changes

marcalff changed the title ~~Add synchronous gauge~~ [SDK] Add synchronous gauge Oct 30, 2024

marcalff changed the title ~~[SDK] Add synchronous gauge~~ [API] Add synchronous gauge Oct 30, 2024

marcalff merged commit f30ab92 into open-telemetry:main Oct 30, 2024
55 of 56 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[API] Add synchronous gauge #3029

[API] Add synchronous gauge #3029

uuzay commented Aug 14, 2024

codecov bot commented Aug 14, 2024 •

edited

Loading

marcalff left a comment

marcalff left a comment

lalitb commented Aug 26, 2024

lalitb Aug 27, 2024

punya commented Aug 30, 2024

uuzay commented Sep 5, 2024

lalitb commented Sep 5, 2024

lalitb commented Oct 3, 2024 •

edited

Loading

uuzay commented Oct 8, 2024

netlify bot commented Oct 15, 2024 •

edited

Loading

uuzay commented Oct 15, 2024 •

edited

Loading

lalitb commented Oct 25, 2024 •

edited

Loading

marcalff commented Oct 25, 2024

lalitb left a comment

marcalff left a comment

ThomsonTan Oct 30, 2024

ThomsonTan Oct 30, 2024

lalitb Oct 30, 2024 •

edited

Loading

marcalff Oct 30, 2024

ThomsonTan left a comment

ThomsonTan commented Oct 30, 2024

marcalff commented Oct 30, 2024

marcalff commented Oct 30, 2024

ThomsonTan commented Oct 30, 2024

[API] Add synchronous gauge #3029

[API] Add synchronous gauge #3029

Conversation

uuzay commented Aug 14, 2024

Changes

codecov bot commented Aug 14, 2024 • edited Loading

Codecov Report

marcalff left a comment

Choose a reason for hiding this comment

marcalff left a comment

Choose a reason for hiding this comment

lalitb commented Aug 26, 2024

lalitb Aug 27, 2024

Choose a reason for hiding this comment

punya commented Aug 30, 2024

uuzay commented Sep 5, 2024

lalitb commented Sep 5, 2024

lalitb commented Oct 3, 2024 • edited Loading

uuzay commented Oct 8, 2024

netlify bot commented Oct 15, 2024 • edited Loading

✅ Deploy Preview for opentelemetry-cpp-api-docs canceled.

uuzay commented Oct 15, 2024 • edited Loading

lalitb commented Oct 25, 2024 • edited Loading

marcalff commented Oct 25, 2024

lalitb left a comment

Choose a reason for hiding this comment

marcalff left a comment

Choose a reason for hiding this comment

ThomsonTan Oct 30, 2024

Choose a reason for hiding this comment

ThomsonTan Oct 30, 2024

Choose a reason for hiding this comment

lalitb Oct 30, 2024 • edited Loading

Choose a reason for hiding this comment

marcalff Oct 30, 2024

Choose a reason for hiding this comment

ThomsonTan left a comment

Choose a reason for hiding this comment

ThomsonTan commented Oct 30, 2024

marcalff commented Oct 30, 2024

marcalff commented Oct 30, 2024

ThomsonTan commented Oct 30, 2024

codecov bot commented Aug 14, 2024 •

edited

Loading

lalitb commented Oct 3, 2024 •

edited

Loading

netlify bot commented Oct 15, 2024 •

edited

Loading

uuzay commented Oct 15, 2024 •

edited

Loading

lalitb commented Oct 25, 2024 •

edited

Loading

lalitb Oct 30, 2024 •

edited

Loading