Bare minimum docker-compose sample for EXPORTER/LOKI

[cilerler]

Component(s)

exporter/loki

Describe the issue you're reporting

I am trying to export logs through OTEL-COLLECTOR to LOKI via the LOKI-EXPORTER.
With the setup below, I see the logs in the OTEL-COLLECTOR container, but not in LOKI.
As far as I can tell, there is no immediate visible error.
I would thoroughly appreciate any help to point me in the right direction to make this work.

docker-compose.yml

version: '3.8'

networks:
  default:
    name: my_network
    driver: bridge

services:

  otel-collector:
    container_name: otel-collector
    image: otel/opentelemetry-collector-contrib:0.88.0
    restart: always
    command: ["--config=/etc/otel-collector-config.yaml", ""]
    volumes:
      - ./config/otel-collector.yaml:/etc/otel-collector-config.yaml
    ports:
      - 1888   # pprof extension
      - 8888   # Prometheus metrics exposed by the Collector
      - 8889   # Prometheus exporter metrics
      - 13133 # health_check extension
      - 4317:4317   # OTLP gRPC receiver
      - 4318:4318   # OTLP http receiver
      - 55679 # zpages extension
    depends_on:
      - loki

  minio:
    container_name: minio
    image: minio/minio:latest
    entrypoint:
      - sh
      - -euc
      - |
        mkdir -p /data/loki-data && \
        mkdir -p /data/loki-ruler && \
        mkdir -p /data/tempo && \
        minio server /data \
        --console-address ':9001'
    environment:
      - MINIO_ROOT_USER=minio
      - MINIO_ROOT_PASSWORD=minio123
      - MINIO_PROMETHEUS_AUTH_TYPE=public
    ports:
      - 9001

  loki:
    container_name: loki
    image: grafana/loki:latest
    command: "-config.file=/etc/loki/config.yaml"
    ports:
      - 3100
      - 7946
      - 9095
    volumes:
      - ./config/loki.yml:/etc/loki/config.yaml
    depends_on:
      - minio

  grafana:
    container_name: grafana
    image: grafana/grafana:latest
    environment:
      - GF_PATHS_PROVISIONING=/etc/grafana/provisioning
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
      - GF_AUTH_DISABLE_LOGIN_FORM=true
      - GF_FEATURE_TOGGLES_ENABLE=traceqlEditor
    ports:
      - "3000:3000"
    volumes:
      - ./config/grafana-datasources.yml:/etc/grafana/provisioning/datasources/ds.yaml
    depends_on:
      - loki

config/otel-collector.yml

receivers:
  otlp:
    protocols:
      grpc:

exporters:

  debug:
    verbosity: detailed

  loki:
    endpoint: http://loki:3100/loki/api/v1/push
    tls:
      insecure: true
    headers:
      "X-Scope-OrgID": "1"
    default_labels_enabled:
      exporter: true
      job: true
      instance: true
      level: true

processors:
  batch:
  
  attributes:
    actions:
      - action: insert
        key: loki.attribute.labels
        value: test

  resource:
    attributes:
      - action: insert
        key: loki.resource.labels
        value: test
service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [batch, resource, attributes]
      exporters: [debug, loki]

config/loki.yml

server:
  http_listen_port: 3100

schema_config:
  configs:
    - from: 2021-08-01
      store: boltdb-shipper
      object_store: s3
      schema: v11
      index:
        prefix: index_
        period: 24h

common:
  path_prefix: /loki
  replication_factor: 1
  storage:
    s3:
      endpoint: minio:9000
      insecure: true
      bucketnames: loki-data
      access_key_id: minio
      secret_access_key: minio123
      s3forcepathstyle: true
  ring:
    kvstore:
      store: memberlist

ruler:
  storage:
    s3:
      bucketnames: loki-ruler

config/grafana-datasource.yml

apiVersion: 1
datasources: 
  - name: Loki
    uid: loki
    type: loki
    isDefault: true
    version: 1
    editable: false
    access: proxy
    url: http://loki:3100
    jsonData:
      derivedFields:
        - datasourceUid: tempo
          matcherRegex: "TraceId=(\\w+)"
          name: TraceId
          url: "$${__value.raw}"
      httpHeaderName1: "X-Scope-OrgID"
    secureJsonData:
      httpHeaderValue1: "tenant1"

[github-actions]

Pinging code owners:

• exporter/loki: @gramidt @gouthamve @jpkrohling @mar4uk

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[jpkrohling]

"X-Scope-OrgID": "1"

httpHeaderValue1: "tenant1"

Looks like the tenant's aren't matching in the two configs?

[cilerler]

You made my day, thank you very much.

[jpkrohling]

And you've just made mine :-)

[skumarjh-tibco]

I am trying to do the setup with same files. Can you elaborate what was missing ?
Thanks!

[cilerler]

    secureJsonData:
      httpHeaderValue1: "tenant1"

in config/otel-collector.yml, and

    headers:
      "X-Scope-OrgID": "1"

in config/grafana-datasource.yml

values have to match.

in other words the issue was "tenant1" != "1"

[skumarjh-tibco]

Thanks a lot! It worked.