Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Collector panics on startup for non-nilable config options with unset environment variable #10949

Closed
crobert-1 opened this issue Aug 22, 2024 · 1 comment · Fixed by #10950
Closed

Collector panics on startup for non-nilable config options with unset environment variable #10949

crobert-1 opened this issue Aug 22, 2024 · 1 comment · Fixed by #10950
Assignees
@TylerHelmuth
Labels
area:confmap bug Something isn't working release:blocker The issue must be resolved before cutting the next release
Milestone
go.opentelemetry....

Comments

@crobert-1
Copy link
Member

Describe the bug

When the collector is started with a configuration that has a non-nilable config option (e.g. int or uint64) set to an environment variable that hasn't been set, the collector panics without any information other than the stack.

Steps to reproduce

Config:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:8518

exporters:
  debug:
    verbosity: detailed
    sampling_initial: ${UNSET_VARIABLE}

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: []
      exporters: [debug]

Start the collector with an int configuration option, in this case the debug exporter's sampling_initial option, set to the environment variable UNSET_VARIABLE, which isn't set.

What did you expect to see?

No panic. I'm not sure if intended behavior here is to error and force the user to set the value of the variable, or to start with a default value, such as 0 for number types.

What did you see instead?

$ ./bin/otelcorecol_darwin_amd64 --config config.yaml 
panic: reflect: call of reflect.Value.Interface on zero Value

goroutine 1 [running]:
reflect.valueInterface({0x0?, 0x0?, 0xc00041c528?}, 0x85?)
	reflect/value.go:1501 +0xfb
reflect.Value.Interface({0x0?, 0x0?, 0xc0003f5400?})
	reflect/value.go:1496 +0x25
go.opentelemetry.io/collector/confmap.decodeConfig.expandNilStructPointersHookFunc.func2({0x0?, 0x0?, 0xa0b8020?}, {0xa0b8020?, 0xc0003b32c8?, 0xc0003f5480?})
	go.opentelemetry.io/collector/[email protected]/confmap.go:306 +0x131
github.com/go-viper/mapstructure/v2.cachedDecodeHook.func3({0x0?, 0x0?, 0x5?}, {0xa0b8020?, 0xc0003b32c8?, 0xc00041c650?})
	github.com/go-viper/mapstructure/[email protected]/decode_hooks.go:54 +0x2c
github.com/go-viper/mapstructure/v2.ComposeDecodeHookFunc.func1({0xa19d580?, 0xc0003f5480?, 0xc00030a270?}, {0xa0b8020?, 0xc0003b32c8?, 0xc00030a270?})
	github.com/go-viper/mapstructure/[email protected]/decode_hooks.go:98 +0xdc
github.com/go-viper/mapstructure/v2.cachedDecodeHook.func3({0xa19d580?, 0xc0003f5480?, 0xc0003fe601?}, {0xa0b8020?, 0xc0003b32c8?, 0xc00041c700?})
	github.com/go-viper/mapstructure/[email protected]/decode_hooks.go:54 +0x2c
github.com/go-viper/mapstructure/v2.(*Decoder).decode(0xc00030a030, {0xa082da9, 0x10}, {0xa19d580?, 0xc0003f5480?}, {0xa0b8020?, 0xc0003b32c8?, 0x22?})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:480 +0x12e
github.com/go-viper/mapstructure/v2.(*Decoder).decodeStructFromMap(0xc00030a030, {0x0, 0x0}, {0xa11cb00?, 0xc0003f1b90?, 0xa200180?}, {0xa200180?, 0xc0003b32c0?, 0xc00041d160?})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:1462 +0xb10
github.com/go-viper/mapstructure/v2.(*Decoder).decodeStruct(0xc00030a030, {0x0, 0x0}, {0xa11cb00?, 0xc0003f1b90?}, {0xa200180?, 0xc0003b32c0?, 0xc00041d228?})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:1278 +0x55f
github.com/go-viper/mapstructure/v2.(*Decoder).decode(0xc00030a030, {0x0, 0x0}, {0xa11cb00?, 0xc0003f1b90?}, {0xa200180?, 0xc0003b32c0?, 0xc0002a2e98?})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:505 +0x27f
github.com/go-viper/mapstructure/v2.(*Decoder).decodePtr(0xc00030a030, {0x0, 0x0}, {0xa11cb00, 0xc0003f1b90}, {0xa10bb80?, 0xc0002a2e98?, 0xc00041d388?})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:1104 +0x3f6
github.com/go-viper/mapstructure/v2.(*Decoder).decode(0xc00030a030, {0x0, 0x0}, {0xa11cb00?, 0xc0003f1b90?}, {0xa10bb80?, 0xc0002a2e98?, 0xc0002a2e98?})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:509 +0x255
github.com/go-viper/mapstructure/v2.(*Decoder).decodePtr(0xc00030a030, {0x0, 0x0}, {0xa11cb00, 0xc0003f1b90}, {0xc0003f24c0?, 0xc0002a2e98?, 0xab1e740?})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:1110 +0x49d
github.com/go-viper/mapstructure/v2.(*Decoder).decode(0xc00030a030, {0x0, 0x0}, {0xa11cb00?, 0xc0003f1b90?}, {0xc0003f24c0?, 0xc0002a2e98?, 0xa11cb00?})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:509 +0x255
github.com/go-viper/mapstructure/v2.(*Decoder).decodeBasic(0xc00030a030, {0x0, 0x0}, {0xa11cb00, 0xc0003f1b90}, {0xa1313a0?, 0xc0003edf00?, 0xc0003b2e12?})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:555 +0x61f
github.com/go-viper/mapstructure/v2.(*Decoder).decode(0xc00030a030, {0x0, 0x0}, {0xa11cb00?, 0xc0003f1b90?}, {0xa1313a0?, 0xc0003edf00?, 0x0?})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:493 +0x23d
github.com/go-viper/mapstructure/v2.(*Decoder).Decode(0xc00030a030, {0xa11cb00, 0xc0003f1b90})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:430 +0xbb
go.opentelemetry.io/collector/confmap.decodeConfig(0xc0003edd50, {0xa088e80, 0xc0003edf00}, 0x1, 0x0)
	go.opentelemetry.io/collector/[email protected]/confmap.go:211 +0x2cf
go.opentelemetry.io/collector/confmap.(*Conf).Unmarshal(0xc0003edd50, {0xa088e80, 0xc0003edf00}, {0x0, 0x0, 0x10?})
	go.opentelemetry.io/collector/[email protected]/confmap.go:88 +0xa5
go.opentelemetry.io/collector/otelcol/internal/configunmarshaler.(*Configs[...]).Unmarshal(0xa2ccca0, 0xc0003ed330)
	go.opentelemetry.io/collector/[email protected]/internal/configunmarshaler/configs.go:52 +0x285
go.opentelemetry.io/collector/confmap.decodeConfig.unmarshalerHookFunc.func7({0xa11cb00?, 0xc0003d9770?, 0xa1af280?}, {0xa1af280?, 0xc0003db520?, 0xc0003d9770?})
	go.opentelemetry.io/collector/[email protected]/confmap.go:422 +0x271
github.com/go-viper/mapstructure/v2.cachedDecodeHook.func3({0xa11cb00?, 0xc0003d9770?, 0x5?}, {0xa1af280?, 0xc0003db520?, 0xc00041dd40?})
	github.com/go-viper/mapstructure/[email protected]/decode_hooks.go:54 +0x2c
github.com/go-viper/mapstructure/v2.ComposeDecodeHookFunc.func1({0xa11cb00?, 0xc0003d9770?, 0xc0003ed320?}, {0xa1af280?, 0xc0003db520?, 0xc0003ed320?})
	github.com/go-viper/mapstructure/[email protected]/decode_hooks.go:98 +0xdc
github.com/go-viper/mapstructure/v2.cachedDecodeHook.func3({0xa11cb00?, 0xc0003d9770?, 0xc0003e6601?}, {0xa1af280?, 0xc0003db520?, 0xc00041ddf0?})
	github.com/go-viper/mapstructure/[email protected]/decode_hooks.go:54 +0x2c
github.com/go-viper/mapstructure/v2.(*Decoder).decode(0xc0003db5f0, {0xa0712bd, 0x9}, {0xa11cb00?, 0xc0003d9770?}, {0xa1af280?, 0xc0003db520?, 0x16?})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:480 +0x12e
github.com/go-viper/mapstructure/v2.(*Decoder).decodeStructFromMap(0xc0003db5f0, {0x0, 0x0}, {0xa11cb00?, 0xc0003d9680?, 0xa230480?}, {0xa230480?, 0xc0003e6480?, 0xc00041e850?})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:1462 +0xb10
github.com/go-viper/mapstructure/v2.(*Decoder).decodeStruct(0xc0003db5f0, {0x0, 0x0}, {0xa11cb00?, 0xc0003d9680?}, {0xa230480?, 0xc0003e6480?, 0xc00041e918?})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:1278 +0x55f
github.com/go-viper/mapstructure/v2.(*Decoder).decode(0xc0003db5f0, {0x0, 0x0}, {0xa11cb00?, 0xc0003d9680?}, {0xa230480?, 0xc0003e6480?, 0xc0002a2ba8?})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:505 +0x27f
github.com/go-viper/mapstructure/v2.(*Decoder).decodePtr(0xc0003db5f0, {0x0, 0x0}, {0xa11cb00, 0xc0003d9680}, {0xa089400?, 0xc0002a2ba8?, 0xc00029cce2?})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:1104 +0x3f6
github.com/go-viper/mapstructure/v2.(*Decoder).decode(0xc0003db5f0, {0x0, 0x0}, {0xa11cb00?, 0xc0003d9680?}, {0xa089400?, 0xc0002a2ba8?, 0x0?})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:509 +0x255
github.com/go-viper/mapstructure/v2.(*Decoder).Decode(0xc0003db5f0, {0xa11cb00, 0xc0003d9680})
	github.com/go-viper/mapstructure/[email protected]/mapstructure.go:430 +0xbb
go.opentelemetry.io/collector/confmap.decodeConfig(0xc0003db0e0, {0xa0a2bc0, 0xc0002a2ba8}, 0x1, 0x0)
	go.opentelemetry.io/collector/[email protected]/confmap.go:211 +0x2cf
go.opentelemetry.io/collector/confmap.(*Conf).Unmarshal(0xc0003db0e0, {0xa0a2bc0, 0xc0002a2ba8}, {0x0, 0x0, 0xc0002a2b98?})
	go.opentelemetry.io/collector/[email protected]/confmap.go:88 +0xa5
go.opentelemetry.io/collector/otelcol.unmarshal(0xc0003db0e0, {0xc000391c20, 0xc000391d10, 0xc000391c80, 0xc000391b30, 0xc000391da0, 0xc000391c50, 0xc000391d40, 0xc000391cb0, 0xc000391b60, ...})
	go.opentelemetry.io/collector/[email protected]/unmarshaler.go:47 +0x2ff
go.opentelemetry.io/collector/otelcol.(*configProvider).Get(0x4?, {0xa2d1d30?, 0xab838a0?}, {0xc000391c20, 0xc000391d10, 0xc000391c80, 0xc000391b30, 0xc000391da0, 0xc000391c50, 0xc000391d40, ...})
	go.opentelemetry.io/collector/[email protected]/configprovider.go:107 +0xb8
go.opentelemetry.io/collector/otelcol.(*Collector).setupConfigurationComponents(0xc000363b00, {0xa2d1d30, 0xab838a0})
	go.opentelemetry.io/collector/[email protected]/collector.go:170 +0x166
go.opentelemetry.io/collector/otelcol.(*Collector).Run(0xc000363b00, {0xa2d1d30, 0xab838a0})
	go.opentelemetry.io/collector/[email protected]/collector.go:272 +0x55
go.opentelemetry.io/collector/otelcol.NewCommand.func1(0xc000356908, {0x9e70af5?, 0x7?, 0x9e6dbc5?})
	go.opentelemetry.io/collector/[email protected]/command.go:37 +0xa7
github.com/spf13/cobra.(*Command).execute(0xc000356908, {0xc00019a190, 0x2, 0x2})
	github.com/spf13/[email protected]/command.go:985 +0xaca
github.com/spf13/cobra.(*Command).ExecuteC(0xc000356908)
	github.com/spf13/[email protected]/command.go:1117 +0x3ff
github.com/spf13/cobra.(*Command).Execute(0xa2bae30?)
	github.com/spf13/[email protected]/command.go:1041 +0x13
main.runInteractive({0xa2bae30, {{0x9e747b3, 0xb}, {0x9ea2c94, 0x33}, {0x9e747be, 0xb}}, 0x0, {{{0x0, 0x0, ...}, ...}}, ...})
	go.opentelemetry.io/collector/cmd/otelcorecol/main.go:49 +0x5d
main.run(...)
	go.opentelemetry.io/collector/cmd/otelcorecol/main_others.go:10
main.main()
	go.opentelemetry.io/collector/cmd/otelcorecol/main.go:42 +0x1b8

What version did you use?

v0.107.0 and v0.106.1

Environment

This has been hit on barebones macos/amd64 and in k8s as well.
@crobert-1 crobert-1 added the bug Something isn't working label Aug 22, 2024
@TylerHelmuth TylerHelmuth self-assigned this Aug 22, 2024
@TylerHelmuth
Copy link
Member

The problem is happening in expandNilStructPointersHookFunc because the from reflect.Value is nil.

This value is nil because the previous hook, useExpandValue, is converting the expandedValue for sampling_initial to nil, since exp.Value is nil and the target type is not a string.

exp.Value is nil because in expand.go, the ret returned by mr.expandURI(ctx, input) has a nil rawConf. The rawConf is nil because the env var didn't contain anything so the yaml.Unmarshal call here had no bytes to set for &rawConf.

I don't yet see a solution to this issue, but thats the why.

@mx-psi maybe the return value in useExpandValue, or all of the expandedValue type, needs to understand more about default values for primitive types?

@TylerHelmuth TylerHelmuth mentioned this issue Aug 22, 2024
Merged
@TylerHelmuth TylerHelmuth added the release:blocker The issue must be resolved before cutting the next release label Aug 23, 2024
@mx-psi mx-psi closed this as completed in e375da4 Aug 26, 2024
@github-project-automation github-project-automation bot moved this from In Progress to Done in Collector: v1 Aug 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@TylerHelmuth TylerHelmuth

Labels
area:confmap bug Something isn't working release:blocker The issue must be resolved before cutting the next release
Projects
Collector: v1
Archived in project
Milestone
go.opentelemetry.io/collector/confmap...
Development

Successfully merging a pull request may close this issue.

[confmap] Fix for unset env var setting non-string field. TylerHelmuth/opentelemetry-collector
2 participants
@TylerHelmuth @crobert-1