splunkhecexporter panic

[PaulBernier]

Component(s)

exporter/splunkhec

What happened?

Description

Our otel-collector panicked with the following stacktrace:

	/usr/local/go/src/net/http/transport.go:1800 +0x1585
created by net/http.(*Transport).dialConn in goroutine 176037
	/usr/local/go/src/net/http/transport.go:2447 +0x18e
net/http.(*persistConn).writeLoop(0xc0012d9e60)
	/usr/local/go/src/net/http/request.go:755 +0xb0d
net/http.(*Request).write(0xc0015c3c20, {0x1962520, 0xc0003ded00}, 0x0, 0xc001877ce0, 0x0)
	/usr/local/go/src/net/http/transfer.go:385 +0x426
net/http.(*transferWriter).writeBody(0xc0005c60a0, {0x1962520?, 0xc0003ded00?})
	/usr/local/go/src/net/http/transport.go:662 +0x22
net/http.(*readTrackingBody).Close(0xc0005c60a0?)
	/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/wrap.go:37 +0x1b
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*bodyWrapper).Close(0xc0017b4be0?)
	/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/exporter/[email protected]/buffer.go:117 +0x17
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/splunkhecexporter.(*cancellableGzipWriter).Close(0xc000918000?)
	/usr/local/go/src/compress/gzip/gzip.go:242 +0x7a
compress/gzip.(*Writer).Close(0xc001298000)
	/usr/local/go/src/compress/flate/deflate.go:730
compress/flate.(*Writer).Close(...)
	/usr/local/go/src/compress/flate/deflate.go:638 +0x6a
compress/flate.(*compressor).close(0xc001f2c000)
	/usr/local/go/src/compress/flate/deflate.go:408 +0x618
compress/flate.(*compressor).deflate(0xc001f2c000)
	/usr/local/go/src/compress/flate/deflate.go:170 +0x9d
compress/flate.(*compressor).writeBlock(0xc001f2c000, {0xc001552000?, 0xc001918f01?, 0xc000473bc0?}, 0xc0017b4b50?)
	/usr/local/go/src/compress/flate/huffman_bit_writer.go:437 +0xbe
compress/flate.(*huffmanBitWriter).writeBlock(0xc0005e32c0, {0xc001552000?, 0x80?, 0x80?}, 0x0, {0xc00140b1d7, 0x0, 0xee29})
	/usr/local/go/src/compress/flate/huffman_bit_writer.go:548 +0x2a5
compress/flate.(*huffmanBitWriter).indexTokens(0xc0005e32c0, {0xc001552000, 0x4e0, 0x47ecbd?})
goroutine 175868 [running]:
panic: runtime error: index out of range [43] with length 30

I suspect a race condition (it rarely occurs), and possibly a variation of the issue previously reported here #34255 But here the symptom is different/more sever.

Collector version

v0.104.0

Environment information

Environment

alpine linux
Go 1.22

OpenTelemetry Collector configuration

exporters:
  splunk_hec/1:
    token: "{{.hecToken}}"
    endpoint: "{{.hecEndpoint}}/services/collector/raw?index={{.splunkIndex}}&sourcetype={{.splunkSourceType}}&source=eventhub://{{.eventHubFullyQualifiedNamespace}}/{{.eventHubName}}
    sourcetype: "{{.splunkSourceType}}"
    index: "{{.splunkIndex}}"
    export_raw: true
    max_content_length_logs: {{.exporterMaxContentLengthLogs}}
    retry_on_failure:
      enabled: false
    sending_queue:
      enabled: false
    hec_metadata_to_otel_attrs:
      source: source
      host: host
    tls:
      min_version: 1.2

Log output

No response

Additional context

No response

[github-actions]

Pinging code owners:

• exporter/splunkhec: @atoulme @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[atoulme]

Looks like the same issue and the same fix is needed.