Consider enforcing https://github.com/uber-go/goleak in all tests #9165
Labels
ci-cd
CI, CD, testing, build issues
help wanted
Good issue for contributors to OpenTelemetry Service to pick up
Comments
dmitryax
added
help wanted
|
I've submitted a PR that adds If it's helpful, I have a branch in my own fork that includes goleak for all packages, including the failures. It can be used to view failures if anyone is interested. I'll update this comment to keep track of which packages still need |
|
It looks like a pretty common cause of I'm still investigating to better understand the impact and possible solution for this one cause of failures. |
|
We disabled check for opencensus worker in Jaeger because it starts via init() and not under control of the tests or any other code. Cf https://github.com/jaegertracing/jaeger/pull/5055/files#diff-13b40ba61afb4ff3b34affd949b1e9751f855f8c7f974e1896dc984aac2c212cR26 |
|
There is a similar issue with glog, which we also ignored in goleak. |
bogdandrutu
pushed a commit
that referenced
this issue
Jan 3, 2024
**Description:** It was suggested that we add the [goleak package](https://pkg.go.dev/go.uber.org/goleak) to every test in core. This change adds `goleak` to every package that is succeeds with goleak. There a number that are not successful, the full list of which I've posted in the bug. I generated these files using a shell script to copy a template `main_test.go` file into each package, then modified the package name. I'm sure there was a better way to automate this, but it worked well enough at this point. Here's the script: ``` cp ./main_test.go $1 PACKAGE_NAME=$(basename $1) sed -i '' -e "s|package component|package $PACKAGE_NAME|g" $1/main_test.go pushd . && cd $1 && go mod tidy go test -v . ``` Usage example: ``` $ ./add_leak_test reciever/scrapererror ``` **Link to tracking Issue:** #9165 **Testing:** All added tests are passing, but there are a number failing. Note that there's no sign of `goleak` running until it fails.
crobert-1
added a commit
to crobert-1/opentelemetry-collector
that referenced
this issue
Jan 3, 2024
As described [here](open-telemetry#9165 (comment)) there are quite a few packages that directly or indirectly import the OpenCensus stats package. Within the stats/view package there's an init() function that starts goroutines that aren't cleaned up. This causes goleak to fail. Since the failure is caused by OpenCensus, always indirectly from what I can tell, and OpenCensus is now read-only, I believe it's fair to ignore leaks caused by this dependency.
This was referenced Jan 3, 2024
crobert-1
added a commit
to crobert-1/opentelemetry-collector
that referenced
this issue
Jan 4, 2024
As described [here](open-telemetry#9165 (comment)) there are quite a few packages that directly or indirectly import the OpenCensus stats package. Within the stats/view package there's an init() function that starts goroutines that aren't cleaned up. This causes goleak to fail. Since the failure is caused by OpenCensus, always indirectly from what I can tell, and OpenCensus is now read-only, I believe it's fair to ignore leaks caused by this dependency.
bogdandrutu
pushed a commit
that referenced
this issue
Jan 4, 2024
**Description:** As described [here](#9165 (comment)) and [here](#9165 (comment)), there are quite a few packages that directly or indirectly import the [OpenCensus-go stats package](https://pkg.go.dev/go.opencensus.io/stats). Within the [stats/view package](https://pkg.go.dev/[email protected]/stats/view) there's an init() function that starts goroutines that aren't cleaned up. This causes goleak to fail. Since the failure is caused by OpenCensus always indirectly (from what I can tell), and OpenCensus is now read-only, I believe it's fair to ignore leaks caused by this dependency and specific leaking situation. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** All added tests are passing
This was referenced Jan 4, 2024
bogdandrutu
pushed a commit
that referenced
this issue
Jan 5, 2024
**Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> Add `goleak` to detect leaks in tests. Leaking goroutines were detected that were caused by a dependency that we can ignore (`go.opencensus.io/stats/view.(*worker).start`), some `Shutdown` calls were also added that were missing. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** <Describe what testing was performed and which tests were added.> Added check is passing as well as existing tests.
bogdandrutu
pushed a commit
that referenced
this issue
Jan 5, 2024
**Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> Add `goleak` to detect leaks in tests. Leaking goroutines were detected that were caused by a dependency that we can ignore (`go.opencensus.io/stats/view.(*worker).start`), a `Shutdown` call was also added that was missing. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** <Describe what testing was performed and which tests were added.> Added check is passing as well as existing tests.
bogdandrutu
pushed a commit
that referenced
this issue
Jan 5, 2024
**Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> This enables `goleak` to run on tests within `exporter/exporterhelper`. A known leak needs to be ignored, and we need to explicitly call shutdown on the queue to make sure no routines are still running after the test is complete. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** <Describe what testing was performed and which tests were added.> Goleak is now passing, all existing tests are passing.
bogdandrutu
pushed a commit
that referenced
this issue
Jan 5, 2024
**Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> Leaking goroutines were detected that were caused by a dependency that we can ignore (`go.opencensus.io/stats/view.(*worker).start`), as well as a missing server close call. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** <Describe what testing was performed and which tests were added.> Added check is passing as well as existing tests.
bogdandrutu
pushed a commit
that referenced
this issue
Jan 5, 2024
**Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> Enables goleak checks against all tests for the otlphttpexporter. This is the exact same change done in #9210 I just missed it in that change. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** <Describe what testing was performed and which tests were added.> All tests are passing.
|
Once all packages enable We can use this Jaegar PR as a reference for how the check could work. |
bogdandrutu
pushed a commit
that referenced
this issue
Jan 9, 2024
**Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> Add `goleak` to detect leaks in tests. Leaking goroutines were detected that were caused by a dependency that we can ignore (`go.opencensus.io/stats/view.(*worker).start`), some `Shutdown` calls were also added that were missing. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** <Describe what testing was performed and which tests were added.> Added check is passing as well as existing tests.
|
Just to note, here's the original opencensus issue for the leaking goroutine: |
bogdandrutu
pushed a commit
that referenced
this issue
Feb 3, 2024
**Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> Enables goleak to run on the configgrpc package. Requires ignoring the opencensus-go leak. A few tests required an additional client connection close command to exit the client go routine. Additional information can be found [here](https://pkg.go.dev/google.golang.org/grpc#DialContext) in the explanation for the method grpc.DialContext, specifically this comment: ``` Users should call ClientConn.Close to terminate all the pending operations after this function returns. ``` configgrpc's method [`ToClientConn`](https://github.com/open-telemetry/opentelemetry-collector/blob/36730599ae4fb2ffc05d8e6e8a42e0bb0880703f/config/configgrpc/configgrpc.go#L178) is directly calling the `grpc.DialContext` method referenced. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** <Describe what testing was performed and which tests were added.> Added goleak check is passing
bogdandrutu
pushed a commit
that referenced
this issue
Feb 3, 2024
**Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> Add `goleak` to detect leaks in tests. Leaking goroutines were detected that were caused by a dependency that we can ignore (`go.opencensus.io/stats/view.(*worker).start`), a `Shutdown` call was also added that was missing. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** <Describe what testing was performed and which tests were added.> Added check is passing as well as existing tests.
foadnh
added a commit
to Canva/opentelemetry-collector
that referenced
this issue
Mar 5, 2024
* [chore] use consistent file naming, we don't use camelcase (#9205) Signed-off-by: Bogdan Drutu <[email protected]> * [chore] refactor otlpreceiver start logic, add lots of tests (#9206) Signed-off-by: Bogdan Drutu <[email protected]> * [chore] use mdatagen for otlp receiver (#9180) Signed-off-by: Alex Boten <[email protected]> * [chore] Fix PR template so that comments aren't seen in markdown (#9153) **Description:** Currently when submitting a PR the section descriptions will be shown in the markdown view when they should be hidden. This now follows [contrib's template formatting](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/.github/pull_request_template.md?plain=1). * [chore] Run make gotidy (#9212) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> Unfortunately as a part of #9173 I committed some `go.mod` and `go.sum` updates incorrectly. I've run `make gotidy` and hopefully cleared those up. These goleak dependencies were added for goleak tests that ended up failing, so the tests weren't added as a part of the original PR. This ended up leaving the dependencies incorrectly added. * [chore] Enable goleak for tests failing on opencensus-go (#9210) **Description:** As described [here](https://github.com/open-telemetry/opentelemetry-collector/issues/9165#issuecomment-1874732291) and [here](https://github.com/open-telemetry/opentelemetry-collector/issues/9165#issuecomment-1874836336), there are quite a few packages that directly or indirectly import the [OpenCensus-go stats package](https://pkg.go.dev/go.opencensus.io/stats). Within the [stats/view package](https://pkg.go.dev/[email protected]/stats/view) there's an init() function that starts goroutines that aren't cleaned up. This causes goleak to fail. Since the failure is caused by OpenCensus always indirectly (from what I can tell), and OpenCensus is now read-only, I believe it's fair to ignore leaks caused by this dependency and specific leaking situation. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** All added tests are passing * [chore][service/internal/proctelemetry] Enable goleak check (#9227) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> Add `goleak` to detect leaks in tests. Leaking goroutines were detected that were caused by a dependency that we can ignore (`go.opencensus.io/stats/view.(*worker).start`), some `Shutdown` calls were also added that were missing. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** <Describe what testing was performed and which tests were added.> Added check is passing as well as existing tests. * [chore][receiver/scraperhelper] Enable goleak check (#9226) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> Add `goleak` to detect leaks in tests. Leaking goroutines were detected that were caused by a dependency that we can ignore (`go.opencensus.io/stats/view.(*worker).start`), a `Shutdown` call was also added that was missing. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** <Describe what testing was performed and which tests were added.> Added check is passing as well as existing tests. * [chore][exporter/exporterhelper] Enable goleak test (#9215) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> This enables `goleak` to run on tests within `exporter/exporterhelper`. A known leak needs to be ignored, and we need to explicitly call shutdown on the queue to make sure no routines are still running after the test is complete. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** <Describe what testing was performed and which tests were added.> Goleak is now passing, all existing tests are passing. * [chore][internal/e2e] Enable goleak check (#9223) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> Leaking goroutines were detected that were caused by a dependency that we can ignore (`go.opencensus.io/stats/view.(*worker).start`), as well as a missing server close call. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** <Describe what testing was performed and which tests were added.> Added check is passing as well as existing tests. * [chore][exporter/otlphttpexporter] Enable goleak check (#9222) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> Enables goleak checks against all tests for the otlphttpexporter. This is the exact same change done in #9210 I just missed it in that change. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** <Describe what testing was performed and which tests were added.> All tests are passing. * [chore][CI/CD] Add make gotidy check in build-and-test.yml (#9216) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> Every PR should be checked to make sure `make gotidy` can run successfully against its contents. I want to add this because as shown in #9212, I recently broke some dependencies on `main`. [Logic lifted from contrib.](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/0c27a3bccfd1b1f7418cc1ac05a535ce29a5a736/.github/workflows/build-and-test.yml#L198-L201) * [chore][service/internal/proctelemetry] Remove flaky goleak test (#9231) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> Remove the goleak test from this package for now as it's flaky. This is reverting some of #9227, but leaving some added `Shutdown` calls. **Link to tracking Issue:** <Issue number if applicable> #9230: This issue will remain open to track debugging and re-enabling the test Co-authored-by: Bogdan Drutu <[email protected]> * Bump github.com/cloudflare/circl from 1.3.3 to 1.3.7 in /internal/tools (#9236) Bumps [github.com/cloudflare/circl](https://github.com/cloudflare/circl) from 1.3.3 to 1.3.7. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/cloudflare/circl/releases">github.com/cloudflare/circl's releases</a>.</em></p> <blockquote> <h2>CIRCL v1.3.7</h2> <h3>What's Changed</h3> <ul> <li>build(deps): bump golang.org/x/crypto from 0.3.1-0.20221117191849-2c476679df9a to 0.17.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/cloudflare/circl/pull/467">cloudflare/circl#467</a></li> <li>kyber: remove division by q in ciphertext compression by <a href="https://github.com/bwesterb"><code>@bwesterb</code></a> in <a href="https://redirect.github.com/cloudflare/circl/pull/468">cloudflare/circl#468</a></li> <li>Releasing CIRCL v1.3.7 by <a href="https://github.com/armfazh"><code>@armfazh</code></a> in <a href="https://redirect.github.com/cloudflare/circl/pull/469">cloudflare/circl#469</a></li> </ul> <h3>New Contributors</h3> <ul> <li><a href="https://github.com/dependabot"><code>@dependabot</code></a> made their first contribution in <a href="https://redirect.github.com/cloudflare/circl/pull/467">cloudflare/circl#467</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/cloudflare/circl/compare/v1.3.6...v1.3.7">https://github.com/cloudflare/circl/compare/v1.3.6...v1.3.7</a></p> <h2>CIRCL v1.3.6</h2> <h3>What's Changed</h3> <ul> <li>internal: add TurboShake{128,256} by <a href="https://github.com/bwesterb"><code>@bwesterb</code></a> in <a href="https://redirect.github.com/cloudflare/circl/pull/430">cloudflare/circl#430</a></li> <li>Kangaroo12 draft -10 by <a href="https://github.com/bwesterb"><code>@bwesterb</code></a> in <a href="https://redirect.github.com/cloudflare/circl/pull/431">cloudflare/circl#431</a></li> <li>Add K12 as XOF by <a href="https://github.com/bwesterb"><code>@bwesterb</code></a> in <a href="https://redirect.github.com/cloudflare/circl/pull/437">cloudflare/circl#437</a></li> <li>xof/k12: Fix a typo in the package documentation by <a href="https://github.com/cjpatton"><code>@cjpatton</code></a> in <a href="https://redirect.github.com/cloudflare/circl/pull/438">cloudflare/circl#438</a></li> <li>Set CIRCL version for generated assembler code. by <a href="https://github.com/armfazh"><code>@armfazh</code></a> in <a href="https://redirect.github.com/cloudflare/circl/pull/440">cloudflare/circl#440</a></li> <li>Add tkn20 benchmarks by <a href="https://github.com/tanyav2"><code>@tanyav2</code></a> in <a href="https://redirect.github.com/cloudflare/circl/pull/442">cloudflare/circl#442</a></li> <li>Add partially blind RSA implementation by <a href="https://github.com/chris-wood"><code>@chris-wood</code></a> in <a href="https://redirect.github.com/cloudflare/circl/pull/445">cloudflare/circl#445</a></li> <li>Update doc.go by <a href="https://github.com/nadimkobeissi"><code>@nadimkobeissi</code></a> in <a href="https://redirect.github.com/cloudflare/circl/pull/447">cloudflare/circl#447</a></li> <li>tss/rsa: key generation for threshold RSA (safe primes) by <a href="https://github.com/armfazh"><code>@armfazh</code></a> in <a href="https://redirect.github.com/cloudflare/circl/pull/450">cloudflare/circl#450</a></li> <li>Bumping Go version for CI jobs. by <a href="https://github.com/armfazh"><code>@armfazh</code></a> in <a href="https://redirect.github.com/cloudflare/circl/pull/457">cloudflare/circl#457</a></li> <li>Spelling by <a href="https://github.com/jsoref"><code>@jsoref</code></a> in <a href="https://redirect.github.com/cloudflare/circl/pull/456">cloudflare/circl#456</a></li> <li>blindrsa: updating blindrsa to be compliant with RFC9474 by <a href="https://github.com/armfazh"><code>@armfazh</code></a> in <a href="https://redirect.github.com/cloudflare/circl/pull/464">cloudflare/circl#464</a></li> <li>Releasing CIRCL v1.3.6 by <a href="https://github.com/armfazh"><code>@armfazh</code></a> in <a href="https://redirect.github.com/cloudflare/circl/pull/465">cloudflare/circl#465</a></li> </ul> <h3>New Contributors</h3> <ul> <li><a href="https://github.com/nadimkobeissi"><code>@nadimkobeissi</code></a> made their first contribution in <a href="https://redirect.github.com/cloudflare/circl/pull/447">cloudflare/circl#447</a></li> <li><a href="https://github.com/jsoref"><code>@jsoref</code></a> made their first contribution in <a href="https://redirect.github.com/cloudflare/circl/pull/456">cloudflare/circl#456</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/cloudflare/circl/compare/v1.3.3...v1.3.6">https://github.com/cloudflare/circl/compare/v1.3.3...v1.3.6</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/cloudflare/circl/commit/c48866b3068dfa83721c021dec03c777ba91abab"><code>c48866b</code></a> Releasing CIRCL v1.3.7</li> <li><a href="https://github.com/cloudflare/circl/commit/75ef91e8a2f438e6ce2b6e620d236add8be1887d"><code>75ef91e</code></a> kyber: remove division by q in ciphertext compression</li> <li><a href="https://github.com/cloudflare/circl/commit/899732a43256a5d6fb779917f597b32939ca4ba4"><code>899732a</code></a> build(deps): bump golang.org/x/crypto</li> <li><a href="https://github.com/cloudflare/circl/commit/99f0f715ca5fbec868f5a0db1df2be6dcd28dbaa"><code>99f0f71</code></a> Releasing CIRCL v1.3.6</li> <li><a href="https://github.com/cloudflare/circl/commit/e728d0d84e7e7cd9027050a62aa14adb8dec147c"><code>e728d0d</code></a> Apply thibmeu code review suggestions</li> <li><a href="https://github.com/cloudflare/circl/commit/ceb2d90c4922ec2e26be09a20f217ee57c8ba1c4"><code>ceb2d90</code></a> Updating blindrsa to be compliant with RFC9474.</li> <li><a href="https://github.com/cloudflare/circl/commit/44133f703215856ee0b8f243778f24b001ff6c95"><code>44133f7</code></a> spelling: tripped</li> <li><a href="https://github.com/cloudflare/circl/commit/c2076d67b2c717b1b1c6f3aa3b324bf93079b6fb"><code>c2076d6</code></a> spelling: transposes</li> <li><a href="https://github.com/cloudflare/circl/commit/dad216659ee1c9969957557a713537ceb589fce5"><code>dad2166</code></a> spelling: title</li> <li><a href="https://github.com/cloudflare/circl/commit/171c41832e7ec817b9b2873732db6da46bdb1139"><code>171c418</code></a> spelling: threshold</li> <li>Additional commits viewable in <a href="https://github.com/cloudflare/circl/compare/v1.3.3...v1.3.7">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/open-telemetry/opentelemetry-collector/network/alerts). </details> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [config/configretry] Allow zero multiplier and arbitrary randomization factor (#9235) **Description:** Partial revert of #9091 to unblock open-telemetry/opentelemetry-collector-contrib/pull/30167 This does not mean that usage of a zero multiplier or a randomization factor outside of [0,1] is blessed upon by Collector maintainers, just that we need to unblock the release :) --------- Signed-off-by: Alex Boten <[email protected]> Co-authored-by: Alex Boten <[email protected]> * [chore][cmd/mdatagen] Properly test bytes attribute type (#9229) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> As described in the [issue opened in the contrib repository](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/29923), mdatagen was originally generating tests for byte arrays incorrectly. This tests byte arrays in their proper format. As [requested](https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/30106#issuecomment-1879321401), this ports the [contrib fix ](https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/30106 )into core. **Link to tracking Issue:** <Issue number if applicable> https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/29923 **Testing:** <Describe what testing was performed and which tests were added.> I added a fake attribute and fake metric that used it, then generated tests. Fake attribute and metric: ``` attributes: # Added a fake attribute in a metadata.yaml file fake: description: "Fake attribute for testing" type: bytes metrics: container.cpu.usage.percpu: enabled: false description: "Per-core CPU usage by the container (Only available with cgroups v1)." unit: ns sum: value_type: int monotonic: true aggregation_temporality: cumulative attributes: - fake ``` Generated tests: ``` // First test reference: mb.RecordContainerCPUUsagePercpuDataPoint(ts, 1, []byte("fake-val")) // Second test reference: attrVal, ok := dp.Attributes().Get("fake") assert.True(t, ok) assert.EqualValues(t, []byte("fake-val"), attrVal.Bytes()) ``` * [chore] [exporterhelper] Make memory queue benchmarks deteministic (#9247) Ensure the queue always has enough capacity to make the benchmark results deterministic. Otherwise, it's unpredictable when the queue starts rejecting the new entries. It depends on how fast it processes them, which can significantly fluctuate. Also, remove the multiple consumers use cases from benchmarks as redundant. Not sure they provide any value. * Update module golang.org/x/net to v0.20.0 (#9248) [](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | golang.org/x/net | `v0.19.0` -> `v0.20.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMjEuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEyMS4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <[email protected]> Co-authored-by: Bogdan Drutu <[email protected]> * Update module go.uber.org/goleak to v1.3.0 (#9246) [](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [go.uber.org/goleak](https://togithub.com/uber-go/goleak) | `v1.2.0` -> `v1.3.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>uber-go/goleak (go.uber.org/goleak)</summary> ### [`v1.3.0`](https://togithub.com/uber-go/goleak/releases/tag/v1.3.0) [Compare Source](https://togithub.com/uber-go/goleak/compare/v1.2.1...v1.3.0) ##### Fixed - Built-in ignores now match function names more accurately. They will no longer ignore stacks because of file names that look similar to function names. ([#​112](https://togithub.com/uber-go/goleak/issues/112)) ##### Added - Add an `IgnoreAnyFunction` option to ignore stack traces that have the provided function anywhere in the stack. ([#​113](https://togithub.com/uber-go/goleak/issues/113)) - Ignore `testing.runFuzzing` and `testing.runFuzzTests` alongside other already-ignored test functions (`testing.RunTests`, etc). ([#​105](https://togithub.com/uber-go/goleak/issues/105)) ##### Changed - Miscellaneous CI-related fixes. ([#​103](https://togithub.com/uber-go/goleak/issues/103), [#​108](https://togithub.com/uber-go/goleak/issues/108), [#​114](https://togithub.com/uber-go/goleak/issues/114)) ### [`v1.2.1`](https://togithub.com/uber-go/goleak/releases/tag/v1.2.1): v.1.2.1 [Compare Source](https://togithub.com/uber-go/goleak/compare/v1.2.0...v1.2.1) #### \[1.2.1] ##### Changed - Drop golang/x/lint dependency. </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMjEuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEyMS4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <[email protected]> Co-authored-by: Bogdan Drutu <[email protected]> * Move error out of `ReportComponentStatus` function signature, use `ReportStatus` instead (#9175) Fixes #9148 * [chore][processor/batchprocessor] Enable goleak check (#9224) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> Add `goleak` to detect leaks in tests. Leaking goroutines were detected that were caused by a dependency that we can ignore (`go.opencensus.io/stats/view.(*worker).start`), some `Shutdown` calls were also added that were missing. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** <Describe what testing was performed and which tests were added.> Added check is passing as well as existing tests. * [chore] make sharedcomponent Map threadsafe (#9170) **Description:** Add just enough code to make sharedcomponent Map thread safe. **Link to tracking Issue:** Relates to #9156 Follow up to #9157, should be reviewed after it is merged. * [chore] [exporterhelper] Add alloc report to persistent queue benchmarks (#9250) * [chore] Prepare release v1.0.1/v0.92.0 (#9255) The following commands were run to prepare this release: - make chlog-update VERSION=v1.0.1/v0.92.0 - make prepare-release PREVIOUS_VERSION=1.0.0 RELEASE_CANDIDATE=1.0.1 MODSET=stable - make prepare-release PREVIOUS_VERSION=0.91.0 RELEASE_CANDIDATE=0.92.0 MODSET=beta * [chore] make sure all contrib tests run (#9238) Fixes #9237 Co-authored-by: Alex Boten <[email protected]> * remove deprecated types (#9256) Signed-off-by: Alex Boten <[email protected]> * [configopaque] configopaque.String implements fmt.Stringer (#9214) **Description:** configopaque.String implements `fmt.Stringer`, outputting [REDACTED] when used as `fmt.Sprintf("%s", opaquestring)` **Link to tracking Issue:** Fixes #9213 --------- Co-authored-by: Pablo Baeyens <[email protected]> * [confignet] Remove deprecated confignet functions (#9258) Instead of completely removing `Dial` and `Listen`, renames `DialContext` and `ListenContext` to `Dial` and `Listen` and deprecates `DialContext` and `ListenContext`. Related to https://github.com/open-telemetry/opentelemetry-collector/issues/9105 Updated tests * Mark configopaque change as a breaking change (#9267) Follow up to #9214. I think marking this as breaking is warranted since we have found 6 instances of breakage in contrib (fixed by open-telemetry/opentelemetry-collector-contrib#30304, open-telemetry/opentelemetry-collector-contrib#30402 and open-telemetry/opentelemetry-collector-contrib#30405). I am also: - Adding subtext explicitly stating how to fix this - Adding this to the user changelog, since it has user-facing implications. * [chore][bug_report.md] Comment out header descriptions (#9261) The descriptions for each header when reporting bugs is useful for the person filing the bug, but not useful to those reading and reviewing. I think it makes sense for these to be included as comments in the bug report, so they're hidden in the rendered report. Co-authored-by: Bogdan Drutu <[email protected]> * [chore] simplify replacements with one mod edit call instead of many (#9259) This speeds up replacements and make build logs easier to parse. * [chore][processor/memorylimiter] Cleanup README (#9152) **Documentation:** <Describe the documentation added.> Some of the wording was hard to follow in the README so I attempted to make it more clear. There shouldn't be any real content changes. * [chore] `make genotelcorecol` (#9275) Counterpart of open-telemetry/opentelemetry-collector-contrib/pull/30456 * [obsreport] deprecate test funcs/structs (#8538) This deprecates the remaining code under the obsreport package. Follows https://github.com/open-telemetry/opentelemetry-collector/commit/bf141227c5b3eb95e849067b33a67cf344992985 --------- Signed-off-by: Alex Boten <[email protected]> * Remove deprecated connectortest router helpers (#9278) Signed-off-by: Bogdan Drutu <[email protected]> * [exporterhelper] Cleanup logging for export failures (#9282) This change makes the logging of the exported error failures cleaner. 1. Ensure an error message is logged every time and only once when data is dropped/rejected due to export failure. 2. Update the wording. Specifically, don't use "dropped" term when an error is reported back to the pipeline. If there is no queue configured, the exporter doesn't drop data by itself but rather rejects it. Keep the "dropped" wording for failures after the enabled queue. 3. Properly report any error reported by a queue. For example, a persistent storage error must be reported as a storage error, not as "queue overflow". Fixes https://github.com/open-telemetry/opentelemetry-collector/issues/9219 * Update processor README (#9284) - Update recommended processors - Update ordering processors For recommended processors, either the log signal could be added or documentation could be made signal agnostic - opted for the latter. * Update module golang.org/x/sys to v0.16.0 (#9249) [](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | golang.org/x/sys | `v0.15.0` -> `v0.16.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMjEuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEyNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <[email protected]> * Update github.com/mitchellh/mapstructure digest to 8508981 (#9135) [](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/mitchellh/mapstructure](https://togithub.com/mitchellh/mapstructure) | require | digest | `bf980b3` -> `8508981` | --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy45My4xIiwidXBkYXRlZEluVmVyIjoiMzcuMTI3LjAiLCJ0YXJnZXRCcmFuY2giOiJtYWluIn0=--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <[email protected]> * Update golang.org/x/exp digest to 73b9e39 (#9144) [](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | golang.org/x/exp | require | digest | `aacd6d4` -> `73b9e39` | --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMDMuMSIsInVwZGF0ZWRJblZlciI6IjM3LjEwMy4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <[email protected]> * [configopaque] Implement fmt.GoStringer interface for String (#9271) **Description:** Follow up to #9214. To make sure the output when using `%#v` is redacted, we must also implement the `fmt.GoStringer` interface. **Link to tracking Issue:** Relates to #9213 **Testing:** Adds some tests with YAML and JSON as well as more thorough testing of all `fmt` verbs **Documentation:** Clarify that printing this is also redacted. * Update module go.opentelemetry.io/collector/exporter/otlphttpexporter to v0.92.0 (#9291) [](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [go.opentelemetry.io/collector/exporter/otlphttpexporter](https://togithub.com/open-telemetry/opentelemetry-collector) | `v0.91.0` -> `v0.92.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>open-telemetry/opentelemetry-collector (go.opentelemetry.io/collector/exporter/otlphttpexporter)</summary> ### [`v0.92.0`](https://togithub.com/open-telemetry/opentelemetry-collector/blob/HEAD/CHANGELOG.md#v101v0920) [Compare Source](https://togithub.com/open-telemetry/opentelemetry-collector/compare/v0.91.0...v0.92.0) ##### 🛑 Breaking changes 🛑 - `exporters/sending_queue`: Do not re-enqueue failed batches, rely on the retry_on_failure strategy instead. ([#​8382](https://togithub.com/open-telemetry/opentelemetry-collector/issues/8382)) The current re-enqueuing behavior is not obvious and cannot be configured. It takes place only for persistent queue and only if `retry_on_failure::enabled=true` even if `retry_on_failure` is a setting for a different backoff retry strategy. This change removes the re-enqueuing behavior. Consider increasing `retry_on_failure::max_elapsed_time` to reduce chances of data loss or set it to 0 to keep retrying until requests succeed. - `confmap`: Make the option `WithErrorUnused` enabled by default when unmarshaling configuration ([#​7102](https://togithub.com/open-telemetry/opentelemetry-collector/issues/7102)) The option `WithErrorUnused` is now enabled by default, and a new option `WithIgnoreUnused` is introduced to ignore errors about unused fields. - `status`: Deprecate `ReportComponentStatus` in favor of `ReportStatus`. This new function does not return an error. ([#​9148](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9148)) ##### 🚩 Deprecations 🚩 - `connectortest`: Deprecate connectortest.New\[Metrics|Logs|Traces]Router in favour of connector.New\[Metrics|Logs|Traces]Router ([#​9095](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9095)) - `exporterhelper`: Deprecate exporterhelper.RetrySettings in favor of configretry.BackOffConfig ([#​9091](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9091)) - `extension/ballast`: Deprecate `memory_ballast` extension. ([#​8343](https://togithub.com/open-telemetry/opentelemetry-collector/issues/8343)) Use `GOMEMLIMIT` environment variable instead. - `connector`: Deprecate \[Metrics|Logs|Traces]Router in favour of \[Metrics|Logs|Traces]RouterAndConsumer ([#​9095](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9095)) ##### 💡 Enhancements 💡 - `exporterhelper`: Add RetrySettings validation function ([#​9089](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9089)) Validate that time.Duration, multiplier values in configretry are non-negative, and randomization_factor is between 0 and 1 - `service`: Enable `telemetry.useOtelForInternalMetrics` by updating the flag to beta ([#​7454](https://togithub.com/open-telemetry/opentelemetry-collector/issues/7454)) The metrics generated should be consistent with the metrics generated previously with OpenCensus. Users can disable the behaviour by setting `--feature-gates -telemetry.useOtelForInternalMetrics` at collector start. - `mdatagen`: move component from contrib to core ([#​9172](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9172)) - `semconv`: Generated Semantic conventions 1.22.0. ([#​8686](https://togithub.com/open-telemetry/opentelemetry-collector/issues/8686)) - `confignet`: Add `dialer_timeout` config option. ([#​9066](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9066)) - `processor/memory_limiter`: Update config validation errors ([#​9059](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9059)) - Fix names of the config fields that are validated in the error messages - Move the validation from start to the initialization phrase - `exporterhelper`: Add config Validate for TimeoutSettings ([#​9104](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9104)) ##### 🧰 Bug fixes 🧰 - `memorylimiterprocessor`: Fixed leaking goroutines from memorylimiterprocessor ([#​9099](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9099)) - `cmd/otelcorecol`: Fix the code detecting if the collector is running as a service on Windows. ([#​7350](https://togithub.com/open-telemetry/opentelemetry-collector/issues/7350)) Removed the `NO_WINDOWS_SERVICE` environment variable given it is not needed anymore. - `otlpexporter`: remove dependency of otlphttpreceiver on otlpexporter ([#​6454](https://togithub.com/open-telemetry/opentelemetry-collector/issues/6454)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMjcuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEyNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * [docs/release] Swap Juraci and me for release duties (#9294) I can't do the v0.95.0 release, so I am swapping it for the v0.96.0 release. * Update module go.opentelemetry.io/collector/exporter/otlpexporter to v0.92.0 (#9290) [](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [go.opentelemetry.io/collector/exporter/otlpexporter](https://togithub.com/open-telemetry/opentelemetry-collector) | `v0.91.0` -> `v0.92.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>open-telemetry/opentelemetry-collector (go.opentelemetry.io/collector/exporter/otlpexporter)</summary> ### [`v0.92.0`](https://togithub.com/open-telemetry/opentelemetry-collector/blob/HEAD/CHANGELOG.md#v101v0920) [Compare Source](https://togithub.com/open-telemetry/opentelemetry-collector/compare/v0.91.0...v0.92.0) ##### 🛑 Breaking changes 🛑 - `exporters/sending_queue`: Do not re-enqueue failed batches, rely on the retry_on_failure strategy instead. ([#​8382](https://togithub.com/open-telemetry/opentelemetry-collector/issues/8382)) The current re-enqueuing behavior is not obvious and cannot be configured. It takes place only for persistent queue and only if `retry_on_failure::enabled=true` even if `retry_on_failure` is a setting for a different backoff retry strategy. This change removes the re-enqueuing behavior. Consider increasing `retry_on_failure::max_elapsed_time` to reduce chances of data loss or set it to 0 to keep retrying until requests succeed. - `confmap`: Make the option `WithErrorUnused` enabled by default when unmarshaling configuration ([#​7102](https://togithub.com/open-telemetry/opentelemetry-collector/issues/7102)) The option `WithErrorUnused` is now enabled by default, and a new option `WithIgnoreUnused` is introduced to ignore errors about unused fields. - `status`: Deprecate `ReportComponentStatus` in favor of `ReportStatus`. This new function does not return an error. ([#​9148](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9148)) ##### 🚩 Deprecations 🚩 - `connectortest`: Deprecate connectortest.New\[Metrics|Logs|Traces]Router in favour of connector.New\[Metrics|Logs|Traces]Router ([#​9095](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9095)) - `exporterhelper`: Deprecate exporterhelper.RetrySettings in favor of configretry.BackOffConfig ([#​9091](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9091)) - `extension/ballast`: Deprecate `memory_ballast` extension. ([#​8343](https://togithub.com/open-telemetry/opentelemetry-collector/issues/8343)) Use `GOMEMLIMIT` environment variable instead. - `connector`: Deprecate \[Metrics|Logs|Traces]Router in favour of \[Metrics|Logs|Traces]RouterAndConsumer ([#​9095](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9095)) ##### 💡 Enhancements 💡 - `exporterhelper`: Add RetrySettings validation function ([#​9089](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9089)) Validate that time.Duration, multiplier values in configretry are non-negative, and randomization_factor is between 0 and 1 - `service`: Enable `telemetry.useOtelForInternalMetrics` by updating the flag to beta ([#​7454](https://togithub.com/open-telemetry/opentelemetry-collector/issues/7454)) The metrics generated should be consistent with the metrics generated previously with OpenCensus. Users can disable the behaviour by setting `--feature-gates -telemetry.useOtelForInternalMetrics` at collector start. - `mdatagen`: move component from contrib to core ([#​9172](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9172)) - `semconv`: Generated Semantic conventions 1.22.0. ([#​8686](https://togithub.com/open-telemetry/opentelemetry-collector/issues/8686)) - `confignet`: Add `dialer_timeout` config option. ([#​9066](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9066)) - `processor/memory_limiter`: Update config validation errors ([#​9059](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9059)) - Fix names of the config fields that are validated in the error messages - Move the validation from start to the initialization phrase - `exporterhelper`: Add config Validate for TimeoutSettings ([#​9104](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9104)) ##### 🧰 Bug fixes 🧰 - `memorylimiterprocessor`: Fixed leaking goroutines from memorylimiterprocessor ([#​9099](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9099)) - `cmd/otelcorecol`: Fix the code detecting if the collector is running as a service on Windows. ([#​7350](https://togithub.com/open-telemetry/opentelemetry-collector/issues/7350)) Removed the `NO_WINDOWS_SERVICE` environment variable given it is not needed anymore. - `otlpexporter`: remove dependency of otlphttpreceiver on otlpexporter ([#​6454](https://togithub.com/open-telemetry/opentelemetry-collector/issues/6454)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMjcuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEyNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update module go.opentelemetry.io/collector/receiver/otlpreceiver to v0.92.0 (#9292) [](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [go.opentelemetry.io/collector/receiver/otlpreceiver](https://togithub.com/open-telemetry/opentelemetry-collector) | `v0.91.0` -> `v0.92.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>open-telemetry/opentelemetry-collector (go.opentelemetry.io/collector/receiver/otlpreceiver)</summary> ### [`v0.92.0`](https://togithub.com/open-telemetry/opentelemetry-collector/blob/HEAD/CHANGELOG.md#v101v0920) [Compare Source](https://togithub.com/open-telemetry/opentelemetry-collector/compare/v0.91.0...v0.92.0) ##### 🛑 Breaking changes 🛑 - `exporters/sending_queue`: Do not re-enqueue failed batches, rely on the retry_on_failure strategy instead. ([#​8382](https://togithub.com/open-telemetry/opentelemetry-collector/issues/8382)) The current re-enqueuing behavior is not obvious and cannot be configured. It takes place only for persistent queue and only if `retry_on_failure::enabled=true` even if `retry_on_failure` is a setting for a different backoff retry strategy. This change removes the re-enqueuing behavior. Consider increasing `retry_on_failure::max_elapsed_time` to reduce chances of data loss or set it to 0 to keep retrying until requests succeed. - `confmap`: Make the option `WithErrorUnused` enabled by default when unmarshaling configuration ([#​7102](https://togithub.com/open-telemetry/opentelemetry-collector/issues/7102)) The option `WithErrorUnused` is now enabled by default, and a new option `WithIgnoreUnused` is introduced to ignore errors about unused fields. - `status`: Deprecate `ReportComponentStatus` in favor of `ReportStatus`. This new function does not return an error. ([#​9148](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9148)) ##### 🚩 Deprecations 🚩 - `connectortest`: Deprecate connectortest.New\[Metrics|Logs|Traces]Router in favour of connector.New\[Metrics|Logs|Traces]Router ([#​9095](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9095)) - `exporterhelper`: Deprecate exporterhelper.RetrySettings in favor of configretry.BackOffConfig ([#​9091](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9091)) - `extension/ballast`: Deprecate `memory_ballast` extension. ([#​8343](https://togithub.com/open-telemetry/opentelemetry-collector/issues/8343)) Use `GOMEMLIMIT` environment variable instead. - `connector`: Deprecate \[Metrics|Logs|Traces]Router in favour of \[Metrics|Logs|Traces]RouterAndConsumer ([#​9095](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9095)) ##### 💡 Enhancements 💡 - `exporterhelper`: Add RetrySettings validation function ([#​9089](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9089)) Validate that time.Duration, multiplier values in configretry are non-negative, and randomization_factor is between 0 and 1 - `service`: Enable `telemetry.useOtelForInternalMetrics` by updating the flag to beta ([#​7454](https://togithub.com/open-telemetry/opentelemetry-collector/issues/7454)) The metrics generated should be consistent with the metrics generated previously with OpenCensus. Users can disable the behaviour by setting `--feature-gates -telemetry.useOtelForInternalMetrics` at collector start. - `mdatagen`: move component from contrib to core ([#​9172](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9172)) - `semconv`: Generated Semantic conventions 1.22.0. ([#​8686](https://togithub.com/open-telemetry/opentelemetry-collector/issues/8686)) - `confignet`: Add `dialer_timeout` config option. ([#​9066](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9066)) - `processor/memory_limiter`: Update config validation errors ([#​9059](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9059)) - Fix names of the config fields that are validated in the error messages - Move the validation from start to the initialization phrase - `exporterhelper`: Add config Validate for TimeoutSettings ([#​9104](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9104)) ##### 🧰 Bug fixes 🧰 - `memorylimiterprocessor`: Fixed leaking goroutines from memorylimiterprocessor ([#​9099](https://togithub.com/open-telemetry/opentelemetry-collector/issues/9099)) - `cmd/otelcorecol`: Fix the code detecting if the collector is running as a service on Windows. ([#​7350](https://togithub.com/open-telemetry/opentelemetry-collector/issues/7350)) Removed the `NO_WINDOWS_SERVICE` environment variable given it is not needed anymore. - `otlpexporter`: remove dependency of otlphttpreceiver on otlpexporter ([#​6454](https://togithub.com/open-telemetry/opentelemetry-collector/issues/6454)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMjcuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEyNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update module golang.org/x/tools to v0.17.0 (#9293) [](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | golang.org/x/tools | `v0.16.1` -> `v0.17.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMjcuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEyNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <[email protected]> * Update module github.com/prometheus/common to v0.46.0 (#9289) [](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [github.com/prometheus/common](https://togithub.com/prometheus/common) | `v0.45.0` -> `v0.46.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>prometheus/common (github.com/prometheus/common)</summary> ### [`v0.46.0`](https://togithub.com/prometheus/common/releases/tag/v0.46.0) [Compare Source](https://togithub.com/prometheus/common/compare/v0.45.0...v0.46.0) #### What's Changed - Add golangci-lint config by [@​SuperQ](https://togithub.com/SuperQ) in [https://github.com/prometheus/common/pull/517](https://togithub.com/prometheus/common/pull/517) - model: add metric type values by [@​bboreham](https://togithub.com/bboreham) in [https://github.com/prometheus/common/pull/533](https://togithub.com/prometheus/common/pull/533) - Synchronize common files from prometheus/prometheus by [@​prombot](https://togithub.com/prombot) in [https://github.com/prometheus/common/pull/532](https://togithub.com/prometheus/common/pull/532) - Bump github.com/aws/aws-sdk-go from 1.45.19 to 1.47.0 in /sigv4 by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/prometheus/common/pull/529](https://togithub.com/prometheus/common/pull/529) - Update modules by [@​SuperQ](https://togithub.com/SuperQ) in [https://github.com/prometheus/common/pull/534](https://togithub.com/prometheus/common/pull/534) - Synchronize common files from prometheus/prometheus by [@​prombot](https://togithub.com/prombot) in [https://github.com/prometheus/common/pull/535](https://togithub.com/prometheus/common/pull/535) - Allow using empty Authorization credentials by [@​TheSpiritXIII](https://togithub.com/TheSpiritXIII) in [https://github.com/prometheus/common/pull/546](https://togithub.com/prometheus/common/pull/546) - enable errorlint linter by [@​mmorel-35](https://togithub.com/mmorel-35) in [https://github.com/prometheus/common/pull/550](https://togithub.com/prometheus/common/pull/550) - Synchronize common files from prometheus/prometheus by [@​prombot](https://togithub.com/prombot) in [https://github.com/prometheus/common/pull/548](https://togithub.com/prometheus/common/pull/548) - Bump github.com/aws/aws-sdk-go from 1.47.0 to 1.48.10 in /sigv4 by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/prometheus/common/pull/539](https://togithub.com/prometheus/common/pull/539) - Bump github.com/alecthomas/kingpin/v2 from 2.3.2 to 2.4.0 by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/prometheus/common/pull/542](https://togithub.com/prometheus/common/pull/542) - Synchronize common files from prometheus/prometheus by [@​prombot](https://togithub.com/prombot) in [https://github.com/prometheus/common/pull/552](https://togithub.com/prometheus/common/pull/552) - Bump golang.org/x/net from 0.18.0 to 0.19.0 by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/prometheus/common/pull/541](https://togithub.com/prometheus/common/pull/541) - Bump golang.org/x/oauth2 from 0.14.0 to 0.15.0 by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/prometheus/common/pull/540](https://togithub.com/prometheus/common/pull/540) - Add hints for promlog by [@​lucacome](https://togithub.com/lucacome) in [https://github.com/prometheus/common/pull/556](https://togithub.com/prometheus/common/pull/556) - Bump google.golang.org/protobuf from 1.31.0 to 1.32.0 by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/prometheus/common/pull/560](https://togithub.com/prometheus/common/pull/560) - Bump github.com/prometheus/client_golang from 1.17.0 to 1.18.0 in /sigv4 by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/prometheus/common/pull/558](https://togithub.com/prometheus/common/pull/558) - Synchronize common files from prometheus/prometheus by [@​prombot](https://togithub.com/prombot) in [https://github.com/prometheus/common/pull/555](https://togithub.com/prometheus/common/pull/555) - Bump github.com/prometheus/client_golang from 1.17.0 to 1.18.0 by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/prometheus/common/pull/559](https://togithub.com/prometheus/common/pull/559) - Bump github.com/aws/aws-sdk-go from 1.48.10 to 1.49.13 in /sigv4 by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/prometheus/common/pull/557](https://togithub.com/prometheus/common/pull/557) - Synchronize common files from prometheus/prometheus by [@​prombot](https://togithub.com/prombot) in [https://github.com/prometheus/common/pull/561](https://togithub.com/prometheus/common/pull/561) - Make version getRevision public by [@​SuperQ](https://togithub.com/SuperQ) in [https://github.com/prometheus/common/pull/563](https://togithub.com/prometheus/common/pull/563) - enable gofumpt, goimports, testifylint linters by [@​mmorel-35](https://togithub.com/mmorel-35) in [https://github.com/prometheus/common/pull/551](https://togithub.com/prometheus/common/pull/551) - version: make GetTegs() public by [@​ArthurSens](https://togithub.com/ArthurSens) in [https://github.com/prometheus/common/pull/565](https://togithub.com/prometheus/common/pull/565) - switch to protodelim package (which pbutil now calls) by [@​stapelberg](https://togithub.com/stapelberg) in [https://github.com/prometheus/common/pull/567](https://togithub.com/prometheus/common/pull/567) - Bump Go modules by [@​SuperQ](https://togithub.com/SuperQ) in [https://github.com/prometheus/common/pull/568](https://togithub.com/prometheus/common/pull/568) #### New Contributors - [@​TheSpiritXIII](https://togithub.com/TheSpiritXIII) made their first contribution in [https://github.com/prometheus/common/pull/546](https://togithub.com/prometheus/common/pull/546) - [@​mmorel-35](https://togithub.com/mmorel-35) made their first contribution in [https://github.com/prometheus/common/pull/550](https://togithub.com/prometheus/common/pull/550) - [@​ArthurSens](https://togithub.com/ArthurSens) made their first contribution in [https://github.com/prometheus/common/pull/565](https://togithub.com/prometheus/common/pull/565) - [@​stapelberg](https://togithub.com/stapelberg) made their first contribution in [https://github.com/prometheus/common/pull/567](https://togithub.com/prometheus/common/pull/567) **Full Changelog**: https://github.com/prometheus/common/compare/v0.45.0...v0.46.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-deve…
|
@crobert-1 can this issue be closed now that we're enforcing goleak checks (even with the exceptions currently in place) or is the expectation that all goleaks will be addressed before closing this issue? |
|
We can probably close. Any time we have to skip |
mx-psi
added a commit
that referenced
this issue
Aug 13, 2024
**Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> This change adds `goleak` to check for memory leaks. Originally there were 3 failing tests in the `service` package, so I'll describe changes in relation to resolving each test's failing goleak check. 1. `TestServiceTelemetryRestart`: Simplest fix, close the response body to make sure goroutines aren't leaked by reopening a server on the same port. This was just a test issue 2. `TestTelemetryInit.UseOTelWithSDKConfiguration`: The [meter provider](https://github.com/open-telemetry/opentelemetry-collector/blob/fb3ed1b0d65b91e49209a7e60d40ef4b607c6b10/service/telemetry.go#L57-L58) was being started in the initialization process ([metrics reference](https://github.com/open-telemetry/opentelemetry-collector/blob/fb3ed1b0d65b91e49209a7e60d40ef4b607c6b10/service/internal/proctelemetry/config.go#L135)), but never shutdown. The type originally being used (`meter.MetricProvider`) was the base interface which didn't provide a `Shutdown` method. I changed this to use the `sdk` interfaces that provide the required `Shutdown` method. The actual functionality of starting the providers was already using and returning the `sdk` interface, so the actual underlying type remains the same. Since `mp` is a private member and `sdkmetric` and implement the original type, I don't believe changing the type is a breaking change. 3. `TestServiceTelemetryCleanupOnError`: This test starts a server using a sub-goroutine, cancels it, starts again in a subroutine, and cancels again in the main goroutine. This test showed the racing behavior of the subroutine running [`server.ListenAndServe`](https://github.com/open-telemetry/opentelemetry-collector/blob/fb3ed1b0d65b91e49209a7e60d40ef4b607c6b10/service/internal/proctelemetry/config.go#L148) and the main goroutine's functionality of [calling close](https://github.com/open-telemetry/opentelemetry-collector/blob/fb3ed1b0d65b91e49209a7e60d40ef4b607c6b10/service/telemetry.go#L219) and then starting the server again [right away](https://github.com/open-telemetry/opentelemetry-collector/blob/fb3ed1b0d65b91e49209a7e60d40ef4b607c6b10/service/service_test.go#L244). The solution here is to add a `sync.WaitGroup` variable that can properly block until all servers are closed before returning from `shutdown`. This will allow us to ensure it's safe to proceed knowing the ports are free, and server is fully closed. The first test fix was just a test issue, but 2 and 3 were real bugs. I realize it's a bit hard to read with them all together, but I assumed adding PR dependency notes would be more complicated. **Link to tracking Issue:** <Issue number if applicable> #9165 **Testing:** <Describe what testing was performed and which tests were added.> All tests are passing as well as goleak check. --------- Co-authored-by: Pablo Baeyens <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This will require fixing all existing leaks first.
Suggested in #9100 (comment)
The text was updated successfully, but these errors were encountered: