[builder] unknown revision `confmap/provider/envprovider/v0.109.0` #11129

mmynk · 2024-09-10T23:07:05Z

Describe the bug
Unable to build using collector v0.109.0

Steps to reproduce
Use conf map envprovider

What did you expect to see?
Successfully built and generated collector binary.

What did you see instead?

$ make build
...
Building collector...
2024-09-11T03:57:49.596Z        INFO    internal/command.go:125 OpenTelemetry Collector Builder {"version": "v0.109.0"}
2024-09-11T03:57:49.598Z        INFO    internal/command.go:161 Using config file       {"path": "builder-config.yaml"}
2024-09-11T03:57:49.598Z        INFO    builder/config.go:142   Using go        {"go-executable": "/usr/local/go/bin/go"}
2024-09-11T03:57:49.621Z        INFO    builder/main.go:101     Sources created {"path": "./otelcol-dev"}
Error: failed to update go.mod: go subcommand failed with args '[mod tidy -compat=1.22]': exit status 1, error message: go: downloading go.opentelemetry.io/collector/confmap/provider/envprovider v0.109.0
go: downloading go.opentelemetry.io/collector/confmap/provider/fileprovider v0.109.0
go: go.opentelemetry.io/collector/confmap/provider/[email protected]: reading go.opentelemetry.io/collector/confmap/provider/envprovider/go.mod at revision confmap/provider/envprovider/v0.109.0: unknown revision confmap/provider/envprovider/v0.109.0

What version did you use?
v0.109.0

What config did you use?

dist:
  name: ab
  output_path: ./otelcol-dev
  otelcol_version: 0.109.0

receivers:
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.109.0
  - gomod: go.opentelemetry.io/collector/receiver/otlpreceiver v0.109.0

processors:
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/attributesprocessor v0.109.0
  - gomod: go.opentelemetry.io/collector/processor/batchprocessor v0.109.0

exporters:
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/prometheusexporter v0.109.0
  - gomod: go.opentelemetry.io/collector/exporter/otlphttpexporter v0.109.0

Environment
Ubuntu 20.04

Additional context

The text was updated successfully, but these errors were encountered:

dmitryax · 2024-09-10T23:58:39Z

The module was upgraded to a stable version v1.15. https://github.com/open-telemetry/opentelemetry-collector/releases/tag/confmap%2Fprovider%2Fenvprovider%2Fv1.15.0.

Please update your builder config

mmynk · 2024-09-11T03:44:12Z

@dmitryax Thanks, but I am still running into the issue. Exactly what should I update?

If I update to dist.otelcol_version: 1.15.0, I run into:

go: downloading go.opentelemetry.io/collector/otelcol v1.15.0
go: go.opentelemetry.io/collector/confmap/provider/[email protected]: reading go.opentelemetry.io/collector/confmap/provider/httpprovider/go.mod at revision confmap/provider/httpprovider/v1.15.0: unknown revision confmap/provider/httpprovider/v1.15.0

Otherwise if I keep it at: 0.109.0 I run into the original issue.

dmitryax · 2024-09-11T03:47:54Z

@mmynk, please update the builder version to the latest. Let us know if that works

mmynk · 2024-09-11T03:56:16Z

I'm using builder v0.109.0.

dist:
  name: ab
  output_path: ./otelcol-dev
  otelcol_version: 1.15.0

receivers:
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.109.0
  - gomod: go.opentelemetry.io/collector/receiver/otlpreceiver v0.109.0

processors:
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/attributesprocessor v0.109.0
  - gomod: go.opentelemetry.io/collector/processor/batchprocessor v0.109.0

exporters:
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/prometheusexporter v0.109.0
  - gomod: go.opentelemetry.io/collector/exporter/otlphttpexporter v0.109.0

I am getting the same error:

$ make build
...
2024-09-11T03:54:17.113Z        INFO    internal/command.go:125 OpenTelemetry Collector Builder {"version": "v0.109.0"}
2024-09-11T03:54:17.115Z        INFO    internal/command.go:161 Using config file       {"path": "builder-config.yaml"}
2024-09-11T03:54:17.115Z        INFO    builder/config.go:142   Using go        {"go-executable": "/usr/local/go/bin/go"}
2024-09-11T03:54:17.115Z        INFO    builder/main.go:78      You're building a distribution with non-aligned version of the builder. The version mismatch may cause a compilation failure. It's recommended to use the same version. {"collector-version": "1.15.0", "builder-version": "0.109.0"}
2024-09-11T03:54:17.122Z        INFO    builder/main.go:101     Sources created {"path": "./otelcol-dev"}
Error: failed to update go.mod: go subcommand failed with args '[mod tidy -compat=1.22]': exit status 1, error message: go: downloading go.opentelemetry.io/collector/confmap/provider/httpprovider v1.15.0
go: downloading go.opentelemetry.io/collector/confmap/provider/httpsprovider v1.15.0
go: downloading go.opentelemetry.io/collector/confmap/provider/yamlprovider v1.15.0
go: downloading go.opentelemetry.io/collector/otelcol v1.15.0
go: go.opentelemetry.io/collector/confmap/provider/[email protected]: reading go.opentelemetry.io/collector/confmap/provider/httpprovider/go.mod at revision confmap/provider/httpprovider/v1.15.0: unknown revision confmap/provider/httpprovider/v1.15.0

With the below yaml, it's the original issue.

dist:
  name: ab
  output_path: ./otelcol-dev
  otelcol_version: 0.109.0

dmitryax · 2024-09-11T04:14:01Z

I see. This is clearly a bug in the builder code https://github.com/open-telemetry/opentelemetry-collector/blob/main/cmd/builder/internal/builder/config.go#L219-L231. It's not built to work with 1.x providers OOTB

dmitryax · 2024-09-11T04:24:21Z

The workaround is to define the provides explicitly. @mmynk you can add the following section and it should work

providers:
  - gomod: go.opentelemetry.io/collector/confmap/provider/envprovider v1.15.0
  - gomod: go.opentelemetry.io/collector/confmap/provider/fileprovider v1.15.0
  - gomod: go.opentelemetry.io/collector/confmap/provider/httpprovider v0.109.0
  - gomod: go.opentelemetry.io/collector/confmap/provider/httpsprovider v0.109.0
  - gomod: go.opentelemetry.io/collector/confmap/provider/yamlprovider v0.109.0

andrzej-stencel · 2024-09-11T04:34:25Z

@mmynk Make sure to use otelcol_version: 0.109.0. Here's the configuration that built fine for me:

builder.yaml

dist:
  name: ab
  output_path: ./otelcol-dev
  otelcol_version: 0.109.0

receivers:
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.109.0
  - gomod: go.opentelemetry.io/collector/receiver/otlpreceiver v0.109.0

processors:
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/attributesprocessor v0.109.0
  - gomod: go.opentelemetry.io/collector/processor/batchprocessor v0.109.0

exporters:
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/prometheusexporter v0.109.0
  - gomod: go.opentelemetry.io/collector/exporter/otlphttpexporter v0.109.0

providers:
  - gomod: go.opentelemetry.io/collector/confmap/provider/envprovider v1.15.0
  - gomod: go.opentelemetry.io/collector/confmap/provider/fileprovider v1.15.0
  - gomod: go.opentelemetry.io/collector/confmap/provider/httpprovider v0.109.0
  - gomod: go.opentelemetry.io/collector/confmap/provider/httpsprovider v0.109.0
  - gomod: go.opentelemetry.io/collector/confmap/provider/yamlprovider v0.109.0

$ go install go.opentelemetry.io/collector/cmd/[email protected]
$ builder --config builder.yaml
2024-09-11T06:31:18.720+0200    INFO    internal/command.go:125 OpenTelemetry Collector Builder {"version": "v0.109.0"}
2024-09-11T06:31:18.722+0200    INFO    internal/command.go:161 Using config file       {"path": "0911-builder-ab.yaml"}
2024-09-11T06:31:18.722+0200    INFO    builder/config.go:142   Using go        {"go-executable": "/usr/local/go/bin/go"}
2024-09-11T06:31:18.726+0200    INFO    builder/main.go:101     Sources created {"path": "./otelcol-dev"}
2024-09-11T06:31:20.969+0200    INFO    builder/main.go:192     Getting go modules
2024-09-11T06:31:22.247+0200    INFO    builder/main.go:112     Compiling
2024-09-11T06:31:41.387+0200    INFO    builder/main.go:131     Compiled        {"binary": "./otelcol-dev/ab"}

antonio · 2024-09-11T10:17:59Z

The workaround is to define the provides explicitly

What about using --skip-strict-versioning? Is there a plan to fix the builder to work with the new versions instead?

We have some automation in place to build a new version of our custom collector that relies on all the components using the same version matching the one from the collector itself. Having different version numbers per component throws a wrench into it 😄

May I ask the rationale behind having different numbers in a single release? (e.g. v1.15.0/v0.109.0) Understanding it may help me figure out a way to fix our automation.

andrzej-stencel · 2024-09-11T12:12:11Z

We have some automation in place to build a new version of our custom collector that relies on all the components using the same version matching the one from the collector itself.

This assumption isn't always correct even without the v1.x stable versions, although admittedly the last time the core and contrib versions diverged was a long time ago - in April 2023 (v0.76.1 vs. v0.76.3).

The collector is on its (long) way to a stable v1.x release. During this journey, specific libraries or components (essentially Go modules) individually reach the "stable" status and switch the versioning to the stable version v1.x. All the stable components share the stable version, so in my understanding there are three distinct versions that may need to be used in the builder config:

stable components all use the stable version - currently v1.15.0
unstable components from core repository use the core's version - currently v0.109.0
unstable components from contrib repository use the contrib's version - currently v0.109.0

The core and contrib versions are usually the same, but they are not guaranteed to be the same.

I hope other maintainers can keep me honest and correct any mistakes in my understanding.

antonio · 2024-09-11T12:33:01Z

@andrzej-stencel that is super helful, thanks!

mmynk · 2024-09-11T15:01:38Z

Thanks for the debug, guys. Will stick to v0.108.1 until this is fixed.

bogdandrutu · 2024-09-11T15:58:15Z

Thanks for the debug, guys. Will stick to v0.108.1 until this is fixed.

@mmynk I would recommend you explicitly list the providers.

bogdandrutu · 2024-09-11T15:59:54Z

This is closed now. ocb v0.109.0 is working without any changes.

mmynk added the bug Something isn't working label Sep 10, 2024

mmynk changed the title ~~[builder] unknown revision confmap/provider/envprovider/v0.109.0~~ [builder] unknown revision confmap/provider/envprovider/v0.109.0 Sep 10, 2024

dmitryax closed this as completed Sep 10, 2024

dmitryax reopened this Sep 11, 2024

codeboten mentioned this issue Sep 11, 2024

[builder] Remove builder default confmap providers #11126

Closed

bogdandrutu closed this as completed Sep 11, 2024

bogdandrutu mentioned this issue Sep 11, 2024

[cmd/builder] Updating OCB to v0.109.0 broke implied providers #11152

Closed

crobert-1 mentioned this issue Sep 16, 2024

mismatch in go.mod and builder configuration versions: component open-telemetry/opentelemetry-collector-contrib#35171

Closed

andrzej-stencel mentioned this issue Oct 3, 2024

[chore] add update version automation workflow for artifacts release open-telemetry/opentelemetry-collector-releases#684

Open

andrzej-stencel added the area:builder label Oct 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[builder] unknown revision `confmap/provider/envprovider/v0.109.0` #11129

[builder] unknown revision `confmap/provider/envprovider/v0.109.0` #11129

mmynk commented Sep 10, 2024 •

edited

Loading

dmitryax commented Sep 10, 2024 •

edited

Loading

mmynk commented Sep 11, 2024 •

edited

Loading

dmitryax commented Sep 11, 2024 •

edited

Loading

mmynk commented Sep 11, 2024 •

edited

Loading

dmitryax commented Sep 11, 2024 •

edited

Loading

dmitryax commented Sep 11, 2024 •

edited

Loading

andrzej-stencel commented Sep 11, 2024

antonio commented Sep 11, 2024 •

edited

Loading

andrzej-stencel commented Sep 11, 2024 •

edited

Loading

antonio commented Sep 11, 2024

mmynk commented Sep 11, 2024

bogdandrutu commented Sep 11, 2024

bogdandrutu commented Sep 11, 2024

[builder] unknown revision confmap/provider/envprovider/v0.109.0 #11129

[builder] unknown revision confmap/provider/envprovider/v0.109.0 #11129

Comments

mmynk commented Sep 10, 2024 • edited Loading

dmitryax commented Sep 10, 2024 • edited Loading

mmynk commented Sep 11, 2024 • edited Loading

dmitryax commented Sep 11, 2024 • edited Loading

mmynk commented Sep 11, 2024 • edited Loading

dmitryax commented Sep 11, 2024 • edited Loading

dmitryax commented Sep 11, 2024 • edited Loading

andrzej-stencel commented Sep 11, 2024

antonio commented Sep 11, 2024 • edited Loading

andrzej-stencel commented Sep 11, 2024 • edited Loading

antonio commented Sep 11, 2024

mmynk commented Sep 11, 2024

bogdandrutu commented Sep 11, 2024

bogdandrutu commented Sep 11, 2024

[builder] unknown revision `confmap/provider/envprovider/v0.109.0` #11129

[builder] unknown revision `confmap/provider/envprovider/v0.109.0` #11129

mmynk commented Sep 10, 2024 •

edited

Loading

dmitryax commented Sep 10, 2024 •

edited

Loading

mmynk commented Sep 11, 2024 •

edited

Loading

dmitryax commented Sep 11, 2024 •

edited

Loading

mmynk commented Sep 11, 2024 •

edited

Loading

dmitryax commented Sep 11, 2024 •

edited

Loading

dmitryax commented Sep 11, 2024 •

edited

Loading

antonio commented Sep 11, 2024 •

edited

Loading

andrzej-stencel commented Sep 11, 2024 •

edited

Loading