[pkg/ottl] Issue with the hash value of a match group in the replace_pattern editors

[rnishtala-sumo]

Component(s)

pkg/ottl

What happened?

Description

It has been reported that the hash value of a match group is incorrect. For example

Given the following message

Body: Map({"message":"device=test hello"})

using the following ottl statement

replace_pattern(attributes["message"], "device=(test)", "$$1", SHA256)

returns the following incorrect hash value

message: Str(c1b43b36df09f0bf0e0612b54e90e47df2e8722dd90821b3b0613dc8a4f7d5f4 hello)

instead if the right hash value

message: Str(9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08 hello)

this is because the current replace_pattern function hashes "$1" instead of "test" for the match group, i.e it doesn't resolve $1 to "test" before applying the hash function.

Steps to Reproduce

Described above

Expected Result

Described above (Correct hash value)

Actual Result

Described above (Incorrect hash value)

Collector version

v0.89.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

receivers:
  filelog:
    include:
      - /tmp/test.log
    encoding: utf-8
    start_at: "beginning"
processors:
  batch:
    send_batch_max_size: 2000
    send_batch_size: 1000
    timeout: 1s
  transform/parsejson:
    error_mode: ignore
    log_statements:
      - context: log
        statements:
          - set(body, ParseJSON(body)) where IsMatch(body, "^{") == true
          - merge_maps(attributes, body, "insert")
  attributes/example:
    actions:
      - key: message
        pattern: "^device=(?P<device_name>\\w+)$"
        action: extract
  transform/replace:
    log_statements:
      - context: log
        statements:
          - replace_pattern(attributes["message"], "device=(test)", "$$1", SHA256)

exporters:
  logging:
    verbosity: detailed

service:
  telemetry:
      logs:
        level: "debug"
  pipelines:
    logs:
      receivers: [filelog]
      processors:
      - transform/parsejson
      - attributes/example
      - transform/replace
      exporters: [logging]

Log output

ScopeLogs #0
ScopeLogs SchemaURL:
InstrumentationScope
LogRecord #0
ObservedTimestamp: 2023-11-20 20:54:24.788679 +0000 UTC
Timestamp: 1970-01-01 00:00:00 +0000 UTC
SeverityText:
SeverityNumber: Unspecified(0)
Body: Map({"message":"device=test hello"})
Attributes:
     -> log.file.name: Str(test.log)
     -> message: Str(c1b43b36df09f0bf0e0612b54e90e47df2e8722dd90821b3b0613dc8a4f7d5f4 hello)

Additional context

No response

[github-actions]

Pinging code owners:

• pkg/ottl: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[rnishtala-sumo]

Please take a look at this attempt to fix this bug: #29408

[crobert-1]

Looks like a code owner is reviewing PR, I'll take that to mean it's a valid issue, no longer needing the needs triage label.