ClickHouse - Wrong metric inserts results in empty entries

[StarpTech]

Component(s)

No response

What happened?

Description

Ingesting metrics results in empty rows. The reason is that the logic https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/clickhouseexporter/internal/sum_metrics.go#L109-L141 is wrong. The index is not increased after filling valueArgs and the total count of valueArgs is also incorrect.

Steps to Reproduce

Expected Result

Actual Result

Collector version

0.70.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

extensions:
receivers:
  otlp:
    protocols:
      grpc:
      http:

processors:
  batch:
  memory_limiter:
    # 75% of maximum memory up to 4G
    limit_mib: 1536
    # 25% of limit up to 2G
    spike_limit_mib: 512
    check_interval: 5s

exporters:
  clickhouse:
    endpoint: "url"
    timeout: 10s
    database: otel
    traces_table_name: otel_traces
    logs_table_name: otel_logs
    metrics_table_name: otel_metrics
    ttl_days: 7
    sending_queue:
      queue_size: 5000
    retry_on_failure:
      enabled: true
      initial_interval: 10s
      max_interval: 30s
      max_elapsed_time: 300s
#  logging:
#    loglevel: debug

service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [clickhouse]

Log output

No response

Additional context

No response

[cwegener]

@StarpTech I agree. Thanks for opening this issue. I was just about to do the same.

Also, what do you think about the lack of a prepared statement? My expectation is that insert performance would be pretty bad without using a prepared statement?

At least the Clickhouse Go documentation says to do this, which feels natural: begin->prepare->(in loop exec)->commit https://github.com/ClickHouse/clickhouse-go#key-features

And that's what the example code does as well: https://clickhouse.com/docs/en/integrations/go/clickhouse-go/database-sql-api/#batch-insert

[StarpTech]

Hi @cwegener check this out #18090

[cwegener]

Yup. I remember reading that discussion last week. I'll have read it again and dive into the driver code to understand what's happening.