Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[exporter/otlphttp] added support for configurable telemetry encoding #9276

Merged
merged 18 commits into from
Feb 7, 2024
Merged

[exporter/otlphttp] added support for configurable telemetry encoding #9276

merged 18 commits into from
Feb 7, 2024

Conversation

tvaintrob
Copy link
Contributor

@tvaintrob tvaintrob commented Jan 12, 2024

Description:
This PR adds support for encoding configuration in the otlphttp exporter.

Link to tracking Issue:
#6945

Testing:
Updated existing tests, and added relevant tests

Documentation:
Updated the otlphttp docs to include the new configuration option.

Copy link

codecov bot commented Jan 12, 2024

Codecov Report

Attention: 26 lines in your changes are missing coverage. Please review.

Comparison is base (ca0eab2) 90.45% compared to head (f724cb7) 90.34%.

Files Patch % Lines
exporter/otlphttpexporter/otlp.go 69.73% 23 Missing ⚠️
exporter/otlphttpexporter/config.go 76.92% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9276      +/-   ##
==========================================
- Coverage   90.45%   90.34%   -0.11%     
==========================================
  Files         346      346              
  Lines       18131    18194      +63     
==========================================
+ Hits        16400    16437      +37     
- Misses       1399     1424      +25     
- Partials      332      333       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@songy23 songy23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @tvaintrob please also add a changelog

exporter/otlphttpexporter/otlp.go Show resolved Hide resolved
exporter/otlphttpexporter/README.md Outdated Show resolved Hide resolved
exporter/otlphttpexporter/config.go Show resolved Hide resolved
exporter/otlphttpexporter/otlp_test.go Show resolved Hide resolved
@bogdandrutu
Copy link
Member

What is the use-case for this? I remember that we intentionally did not want to offer this, because it is just a slower version of the proto and you don't have the dependency issue like you have on the client side (web) to avoid depending on protobuf.

@tvaintrob
Copy link
Contributor Author

@bogdandrutu in my case we have a WAF that we want to send messages through but it needs to do a deep validation of the messages being passed, unfortunately it doesn't support reading binary data so we need to send the messages using JSON

@tvaintrob
Copy link
Contributor Author

@songy23 fixed all comments except for the additional tests

Copy link
Member

@songy23 songy23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exporter/otlphttpexporter/otlp_test.go Show resolved Hide resolved
@songy23
Copy link
Member

songy23 commented Jan 17, 2024

We can mention in the documentation that json should only be used in endpoints that can only accept json, but discouraged in other use cases, e.g. collector-to-collector case. Does that sound good? @bogdandrutu

@TylerHelmuth
Copy link
Member

I agree with supporting http/json. Although it isn't performant, it feels correct to implement all parts of the OTLP specification for the exporter.

exporter/otlphttpexporter/config_test.go Outdated Show resolved Hide resolved
exporter/otlphttpexporter/factory_test.go Outdated Show resolved Hide resolved
songy23
songy23 previously approved these changes Jan 22, 2024
Copy link
Member

@songy23 songy23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

cc @open-telemetry/collector-approvers PTAL

@mx-psi
Copy link
Member

mx-psi commented Jan 22, 2024

What is the use-case for this? I remember that we intentionally did not want to offer this, because it is just a slower version of the proto and you don't have the dependency issue like you have on the client side (web) to avoid depending on protobuf.

@bogdandrutu There was general agreement on last week's Collector SIG to add support for this. Do you think we should add something in the README about the performance implications?

exporter/otlphttpexporter/config.go Show resolved Hide resolved
exporter/otlphttpexporter/config.go Show resolved Hide resolved
exporter/otlphttpexporter/factory_test.go Outdated Show resolved Hide resolved
exporter/otlphttpexporter/otlp.go Show resolved Hide resolved
exporter/otlphttpexporter/otlp.go Show resolved Hide resolved
exporter/otlphttpexporter/otlp.go Show resolved Hide resolved
exporter/otlphttpexporter/otlp.go Show resolved Hide resolved
exporter/otlphttpexporter/otlp.go Outdated Show resolved Hide resolved
exporter/otlphttpexporter/otlp.go Outdated Show resolved Hide resolved
exporter/otlphttpexporter/otlp.go Outdated Show resolved Hide resolved
@tvaintrob
Copy link
Contributor Author

@TylerHelmuth updated the PR with the requested changes, not all are possible due to the issues with contrib

Thanks

Copy link
Member

@TylerHelmuth TylerHelmuth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for opening open-telemetry/opentelemetry-collector-contrib#30703. I'd say we fix that before merging this PR.

@songy23
Copy link
Member

songy23 commented Jan 24, 2024

Thanks for opening open-telemetry/opentelemetry-collector-contrib#30703. I'd say we fix that before merging this PR.

Turns out to be straightforward to fix it, I have open-telemetry/opentelemetry-collector-contrib#30759

@songy23 songy23 dismissed their stale review January 24, 2024 21:46

Dismiss the approval now that blocker is removed factory.CreateDefaultConfig() needs updating

@tvaintrob
Copy link
Contributor Author

Thanks for the update @songy23 ill update this one to include the default encoding

Copy link
Member

@songy23 songy23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comments, LGTM otherwise

exporter/otlphttpexporter/config.go Outdated Show resolved Hide resolved
exporter/otlphttpexporter/otlp.go Show resolved Hide resolved
exporter/otlphttpexporter/otlp.go Show resolved Hide resolved
exporter/otlphttpexporter/otlp.go Show resolved Hide resolved
Copy link
Member

@TylerHelmuth TylerHelmuth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tvaintrob please add some test cases that hit the new error scenarios and a test for the new UnmarshalText function.

exporter/otlphttpexporter/otlp.go Show resolved Hide resolved
@tvaintrob
Copy link
Contributor Author

tvaintrob commented Jan 27, 2024

@TylerHelmuth @songy23 updated the PR, i rather take care of the extra error cases in a different PR, feels out of scope for this one

I'll open a ticket for it once this one is merged

Copy link
Member

@songy23 songy23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, please file an issue to return error on invalid content-type in requests.

@tvaintrob
Copy link
Contributor Author

LGTM, please file an issue to return error on invalid content-type in requests.

#9413 created an issue for that, thanks

@tvaintrob
Copy link
Contributor Author

@TylerHelmuth just waiting on your approval to merge this one

Thanks!

Copy link
Member

@TylerHelmuth TylerHelmuth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tvaintrob could you add a could more test scenarios to cover the error cases Codecov is complaining about?

@tvaintrob
Copy link
Contributor Author

@tvaintrob could you add a could more test scenarios to cover the error cases Codecov is complaining about?

I had a talk with @songy23 about it, he said that it's ok to leave it like that, do you have any example on how to generate malformed data?

@TylerHelmuth
Copy link
Member

I think you'd have to call the private functions directly. I am ok moving forward without these tests since they cover such unlikely, simple scenarios

@mx-psi
Copy link
Member

mx-psi commented Jan 31, 2024

@bogdandrutu I will merge this on or after Wednesday next week EU morning. Please block by then if you think we should not merge this.

Copy link

@sarathsp06 sarathsp06 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mx-psi
Copy link
Member

mx-psi commented Feb 7, 2024

@tvaintrob please fix the merge conflict and I can merge afterwards :)

@tvaintrob
Copy link
Contributor Author

@mx-psi Hi, resolved the merge conflict, thanks!

@mx-psi mx-psi merged commit e874866 into open-telemetry:main Feb 7, 2024
45 of 46 checks passed
@github-actions github-actions bot added this to the next release milestone Feb 7, 2024
@tvaintrob tvaintrob deleted the feat/otlphttp-json-encoding branch February 7, 2024 11:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants