[processor/k8sattributes] Add support for exposing k8s.pod.ip metadata

[ChrsMark]

Description:

This PR adds support for exporting the k8s.pod.ip as part of the metadata added by the processor.

Link to tracking Issue: Fixes #32960

Testing: Added

Documentation:

Testing notes

See #33440 (comment)

[ChrsMark]

Testing this manually

1. Using the following values file for the Helm chart:

mode: daemonset
presets:
  logsCollection:
    enabled: true
  kubernetesAttributes:
    enabled: true

image:
  repository: otelcontribcol-dev
  tag: "0.0.1"
  pullPolicy: IfNotPresent

command:
  name: otelcontribcol

config:
  exporters:
    debug:
      verbosity: detailed
  receivers:
    filelog:
      start_at: end
      include_file_name: false
      include_file_path: true
      exclude:
        - /var/log/pods/default_daemonset-opentelemetry-collector*_*/opentelemetry-collector/*.log
      include:
        - /var/log/pods/*/*/*.log
      operators:
        - id: container-parser
          type: container
          output: some
        - id: some
          type: add
          field: attributes.key2.key_in
          value: val2
  processors:
    k8sattributes:
      filter:
        node_from_env_var: K8S_NODE_NAME
      passthrough: false
      pod_association:
        - sources:
            - from: resource_attribute
              name: k8s.pod.ip
        - sources:
            - from: resource_attribute
              name: k8s.pod.uid
        - sources:
            - from: connection
      extract:
        metadata:
          - "k8s.namespace.name"
          - "k8s.deployment.name"
          - "k8s.statefulset.name"
          - "k8s.daemonset.name"
          - "k8s.cronjob.name"
          - "k8s.job.name"
          - "k8s.node.name"
          - "k8s.pod.name"
          - "k8s.pod.uid"
          - "k8s.pod.ip"
          - "k8s.pod.start_time"

  service:
    pipelines:
      logs:
        receivers: [filelog]
        processors: [batch]
        exporters: [debug]

2. Verify that the produced log records report the Pod's IP:

2024-06-08T17:19:10.102Z	info	ResourceLog #0
Resource SchemaURL: 
Resource attributes:
     -> k8s.container.name: Str(busybox)
     -> k8s.namespace.name: Str(default)
     -> k8s.pod.name: Str(daemonset-logs-ndfz9)
     -> k8s.container.restart_count: Str(0)
     -> k8s.pod.uid: Str(6e35d5db-ed8c-4c27-96ba-5d525a216616)
     -> k8s.pod.ip: Str(10.244.2.5)
     -> k8s.pod.start_time: Str(2024-06-08T17:19:02Z)
     -> k8s.daemonset.name: Str(daemonset-logs)
     -> k8s.node.name: Str(kind-worker3)
ScopeLogs #0
ScopeLogs SchemaURL: 
InstrumentationScope  
LogRecord #0
ObservedTimestamp: 2024-06-08 17:19:09.868870532 +0000 UTC
Timestamp: 2024-06-08 17:19:09.719245859 +0000 UTC
SeverityText: 
SeverityNumber: Unspecified(0)
Body: Str(otel logs at 17:19:09)
Attributes:
     -> log.file.path: Str(/var/log/pods/default_daemonset-logs-ndfz9_6e35d5db-ed8c-4c27-96ba-5d525a216616/busybox/0.log)
     -> time: Str(2024-06-08T17:19:09.719245859Z)
     -> log.iostream: Str(stdout)
     -> logtag: Str(F)
     -> key2: Map({"key_in":"val2"})
Trace ID: 
Span ID: 
Flags: 0
LogRecord #1
ObservedTimestamp: 2024-06-08 17:19:09.868930219 +0000 UTC
Timestamp: 2024-06-08 17:19:09.821239181 +0000 UTC
SeverityText: 
SeverityNumber: Unspecified(0)
Body: Str(otel logs at 17:19:09)
Attributes:
     -> log.file.path: Str(/var/log/pods/default_daemonset-logs-ndfz9_6e35d5db-ed8c-4c27-96ba-5d525a216616/busybox/0.log)
     -> time: Str(2024-06-08T17:19:09.821239181Z)
     -> log.iostream: Str(stdout)
     -> logtag: Str(F)
     -> key2: Map({"key_in":"val2"})
Trace ID: 
Span ID: 
Flags: 0
	{"kind": "exporter", "data_type": "logs", "name": "debug"}

[fatsheep9146]

Could you add a e2e test in e2e_test.go to cover the case that, if we don not enable "k8s.pod.ip" in k8sattributes:extract:metadata, then the emitted logs/traces/metrics will not include "k8s.pod.ip" metadata?

[ChrsMark]

Thank's @fatsheep9146, I have added this at e47f401.

[crobert-1]

I'm not very familiar with this component, but the general logic looks good to me. 👍

[TylerHelmuth]

Since the processor uses k8s.pod.ip already I'm ok with merging this, but I did notice that k8s.pod.ip is not defined in https://github.com/open-telemetry/semantic-conventions/blob/main/docs/resource/k8s.md. @ChrsMark can you take up proposing that attribute?

[ChrsMark]

Sure @TylerHelmuth , I filed open-telemetry/semantic-conventions#1160 for this.

[fatsheep9146]

/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/.tools/goimports -w  -local github.com/open-telemetry/opentelemetry-collector-contrib ./
make[2]: Leaving directory '/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib'
make[1]: Leaving directory '/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib'
Generated code is out of date, please run "make generate" and commit the changes in this PR.

Is this failed check should be fixed?

[ChrsMark]

Is this failed check should be fixed?

Needed to run make generate again after rebasing. Thank's.