log group name in ec2 otel collector config.yaml

[georges-git]

Hello - Could you provide a sample config.yaml for AWS OTEL collector for EC2, which shows how to inject the application log group name ( app logs are in cloudwatch) in otel collector config.yaml, so that the auto association of traces to logs can be done. Attached is my sample config.yaml. How to specify the application log group name in the below file?

extensions:
health_check:
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:55681
awsxray:
endpoint: 0.0.0.0:2000
transport: udp
statsd:
endpoint: 0.0.0.0:8125
aggregation_interval: 60s
enable_metric_type: false
processors:
batch/traces:
timeout: 1s
send_batch_size: 50
batch/metrics:
timeout: 60s
filter:
metrics:
include:
match_type: strict
metric_names:
- hikaricp.*
- jvm.*
exporters:
awsxray:
awsemf:
namespace: prod/IOT_2.0
dimension_rollup_option: "NoDimensionRollup"
extensions:
awsproxy:
service:
pipelines:
traces:
receivers: [otlp,awsxray]
processors: [batch/traces]
exporters: [awsxray]
metrics:
receivers: [otlp,statsd]
processors: [batch/metrics]
exporters: [awsemf]
extensions: [awsproxy]

[bryan-aguilar]

Hi @georges-git,

We have looked into this recently and can share some updates. We just pushed some changes to our documentation related to metadata translation. That can be found here after it goes live within the next hour or so.

Correlation between trace and log group can be done by setting the related resource attributes. Currently though there is not a great mechanism to do that other than manually instrumenting your application. The aws.log.stream.names is an array of strings but setting the resource attribute through env var or collector processor is not currently possible. We have opened a few issues upstream tracking this with the hopes of improving this process.

open-telemetry/opentelemetry-collector-contrib#14782
open-telemetry/opentelemetry-specification#2857
aws-observability/aws-otel-java-instrumentation#205

So currently the solution cannot be provided with a collector config. An application developer will have to manually add the required resource attributes within the application.

[georges-git]

Need a simpler easy way to pass clodwatch log group name to the OTEL collector in EC2; Need couple of ways to add this. 1) environment variable (ENV)
2) adding it the otel config.yaml.
What is the ETA for the above two ways to be available?

[bryan-aguilar]

We currently do not have an ETA to share for this, but are involved in discussions upstream. Please feel free to add input to the upstream issues, or create new ones if something is missing, especially if you believe that one method (env or collector config) should be prioritized over the other.

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[github-actions]

This issue was closed because it has been marked as stale for 30 days with no activity.