illegal_argument_exception with the demo otel metrics index mappings

[Roiskia]

I have difficulties setting up the metric analytics using the template for metric analytics from the opensearch-demo. I am using Opensearch / Dashboard container v2.15.0.

I am propably just doing something wrong or missing a step somewhere.

I set up an otel collector for metrics:

collector-config.yml

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:

exporters:
  otlp/metrics:
    endpoint: data-prepper:21891
    tls:
      insecure: true

service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [debug, otlp/metrics]

aswell as an data prepper pipeline:

pipelines.yml

metric-pipeline:
  source:
    otel_metrics_source:
      ssl: false
      authentication:
        unauthenticated:
  processor:
    - otel_metrics:
        calculate_histogram_buckets: true
        calculate_exponential_histogram_buckets: true
        exponential_histogram_max_allowed_scale: 10
        flatten_attributes: false
  sink:
    - stdout:
    - opensearch:
        hosts: ["http://opensearch:9200"]
        insecure: true
        index_type: custom
        template_file: "templates/ss4o_metrics.json"
        index: ss4o_metrics-otel-%{yyyy.MM.dd}

This results in an index that recieves data as expected:

GET /ss4o_metrics-otel-2024.07.23/_search

{
  "took": 833,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 366,
      "relation": "eq"
    },
    "max_score": 1,
    "hits": [
      {
        "_index": "ss4o_metrics-otel-2024.07.23",
        "_id": "1K5M4ZABc9zmiGR5So6E",
        "_score": 1,
        "_source": {
          "kind": "SUM",
          "flags": 0,
          "description": "Number of connections that are currently active on the server.",
          "serviceName": "greeter",
          "schemaUrl": "",
          "isMonotonic": false,
          "unit": "{connection}",
          "aggregationTemporality": "AGGREGATION_TEMPORALITY_CUMULATIVE",
          "exemplars": [],
          "name": "kestrel.active_connections",
          "startTime": "2024-07-23T20:11:14.546468400Z",
          "attributes": {
            "metric.attributes.network@transport": "tcp",
            "instrumentationScope.name": "Microsoft.AspNetCore.Server.Kestrel",
            "resource.attributes.telemetry@sdk@name": "opentelemetry",
            "resource.attributes.telemetry@sdk@language": "dotnet",
            "metric.attributes.network@type": "ipv4",
            "metric.attributes.server@port": 5169,
            "resource.attributes.service@instance@id": "d463ebb1-c319-4b74-90ed-e92284e25b81",
            "resource.attributes.telemetry@sdk@version": "1.9.0",
            "resource.attributes.service@name": "greeter",
            "metric.attributes.server@address": "127.0.0.1"
          },
          "time": "2024-07-23T20:32:14.541126200Z",
          "value": 1
        }
      },
      {
        "_index": "ss4o_metrics-otel-2024.07.23",
        "_id": "1a5M4ZABc9zmiGR5So6E",
        "_score": 1,
        "_source": {
          "kind": "HISTOGRAM",
          "buckets": [
            {
              "min": -3.4028234663852886e+38,
              "max": 0.01,
              "count": 0
            },
            {
              "min": 0.01,
              "max": 0.02,
              "count": 0
            },
            {
              "min": 0.02,
              "max": 0.05,
              "count": 4
            },
            {
              "min": 0.05,
              "max": 0.1,
              "count": 1
            },
            {
              "min": 0.1,
              "max": 0.2,
              "count": 0
            },
            {
              "min": 0.2,
              "max": 0.5,
              "count": 0
            },
            {
              "min": 0.5,
              "max": 1,
              "count": 2
            },
            {
              "min": 1,
              "max": 2,
              "count": 1
            },
            {
              "min": 2,
              "max": 5,
              "count": 5
            },
            {
              "min": 5,
              "max": 10,
              "count": 0
            },
            {
              "min": 10,
              "max": 30,
              "count": 0
            },
            {
              "min": 30,
              "max": 60,
              "count": 1
            },
            {
              "min": 60,
              "max": 120,
              "count": 3
            },
            {
              "min": 120,
              "max": 300,
              "count": 0
            },
            {
              "min": 300,
              "max": 3.4028234663852886e+38,
              "count": 0
            }
          ],
          "count": 17,
          "bucketCountsList": [
            0,
            0,
            4,
            1,
            0,
            0,
            2,
            1,
            5,
            0,
            0,
            1,
            3,
            0,
            0
          ],
          "flags": 0,
          "description": "The duration of connections on the server.",
          "sum": 413.62033599999995,
          "serviceName": "greeter",
          "schemaUrl": "",
          "unit": "s",
          "aggregationTemporality": "AGGREGATION_TEMPORALITY_CUMULATIVE",
          "exemplars": [],
          "bucketCounts": 15,
          "name": "kestrel.connection.duration",
          "startTime": "2024-07-23T20:11:14.547025400Z",
          "explicitBoundsCount": 14,
          "attributes": {
            "metric.attributes.network@transport": "tcp",
            "instrumentationScope.name": "Microsoft.AspNetCore.Server.Kestrel",
            "resource.attributes.telemetry@sdk@name": "opentelemetry",
            "resource.attributes.telemetry@sdk@language": "dotnet",
            "metric.attributes.network@protocol@version": "1.1",
            "metric.attributes.network@type": "ipv4",
            "metric.attributes.server@port": 5169,
            "resource.attributes.service@instance@id": "d463ebb1-c319-4b74-90ed-e92284e25b81",
            "resource.attributes.telemetry@sdk@version": "1.9.0",
            "resource.attributes.service@name": "greeter",
            "metric.attributes.network@protocol@name": "http",
            "metric.attributes.server@address": "127.0.0.1"
          },
          "time": "2024-07-23T20:32:14.541127200Z",
          "explicitBounds": [
            0.01,
            0.02,
            0.05,
            0.1,
            0.2,
            0.5,
            1,
            2,
            5,
            10,
            30,
            60,
            120,
            300
          ]
        }
      },
      {
        "_index": "ss4o_metrics-otel-2024.07.23",
        "_id": "1q5M4ZABc9zmiGR5So6E",
        "_score": 1,
        "_source": {
          "kind": "SUM",
          "flags": 0,
          "description": "Number of connections that are currently queued and are waiting to start.",
          "serviceName": "greeter",
          "schemaUrl": "",
          "isMonotonic": false,
          "unit": "{connection}",
          "aggregationTemporality": "AGGREGATION_TEMPORALITY_CUMULATIVE",
          "exemplars": [],
          "name": "kestrel.queued_connections",
          "startTime": "2024-07-23T20:11:14.547060500Z",
          "attributes": {
            "metric.attributes.network@transport": "tcp",
            "instrumentationScope.name": "Microsoft.AspNetCore.Server.Kestrel",
            "resource.attributes.telemetry@sdk@name": "opentelemetry",
            "resource.attributes.telemetry@sdk@language": "dotnet",
            "metric.attributes.network@type": "ipv4",
            "metric.attributes.server@port": 5169,
            "resource.attributes.service@instance@id": "d463ebb1-c319-4b74-90ed-e92284e25b81",
            "resource.attributes.telemetry@sdk@version": "1.9.0",
            "resource.attributes.service@name": "greeter",
            "metric.attributes.server@address": "127.0.0.1"
          },
          "time": "2024-07-23T20:32:14.541128Z",
          "value": 0
        }
      },
      {
        "_index": "ss4o_metrics-otel-2024.07.23",
        "_id": "165M4ZABc9zmiGR5So6E",
        "_score": 1,
        "_source": {
          "kind": "SUM",
          "flags": 0,
          "description": "Number of active HTTP server requests.",
          "serviceName": "greeter",
          "schemaUrl": "",
          "isMonotonic": false,
          "unit": "{request}",
          "aggregationTemporality": "AGGREGATION_TEMPORALITY_CUMULATIVE",
          "exemplars": [],
          "name": "http.server.active_requests",
          "startTime": "2024-07-23T20:11:14.547129400Z",
          "attributes": {
            "instrumentationScope.name": "Microsoft.AspNetCore.Hosting",
            "resource.attributes.telemetry@sdk@name": "opentelemetry",
            "resource.attributes.telemetry@sdk@language": "dotnet",
            "metric.attributes.http@request@method": "GET",
            "resource.attributes.service@instance@id": "d463ebb1-c319-4b74-90ed-e92284e25b81",
            "resource.attributes.telemetry@sdk@version": "1.9.0",
            "resource.attributes.service@name": "greeter",
            "metric.attributes.url@scheme": "http"
          },
          "time": "2024-07-23T20:32:14.541130600Z",
          "value": 0
        }
      },
      {
        "_index": "ss4o_metrics-otel-2024.07.23",
        "_id": "2K5M4ZABc9zmiGR5So6E",
        "_score": 1,
        "_source": {
          "kind": "HISTOGRAM",
          "buckets": [
            {
              "min": -3.4028234663852886e+38,
              "max": 0.005,
              "count": 218
            },
            {
              "min": 0.005,
              "max": 0.01,
              "count": 0
            },
            {
              "min": 0.01,
              "max": 0.025,
              "count": 0
            },
            {
              "min": 0.025,
              "max": 0.05,
              "count": 1
            },
            {
              "min": 0.05,
              "max": 0.075,
              "count": 0
            },
            {
              "min": 0.075,
              "max": 0.1,
              "count": 0
            },
            {
              "min": 0.1,
              "max": 0.25,
              "count": 0
            },
            {
              "min": 0.25,
              "max": 0.5,
              "count": 0
            },
            {
              "min": 0.5,
              "max": 0.75,
              "count": 0
            },
            {
              "min": 0.75,
              "max": 1,
              "count": 0
            },
            {
              "min": 1,
              "max": 2.5,
              "count": 0
            },
            {
              "min": 2.5,
              "max": 5,
              "count": 0
            },
            {
              "min": 5,
              "max": 7.5,
              "count": 0
            },
            {
              "min": 7.5,
              "max": 10,
              "count": 0
            },
            {
              "min": 10,
              "max": 3.4028234663852886e+38,
              "count": 0
            }
          ],
          "count": 219,
          "bucketCountsList": [
            218,
            0,
            0,
            1,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0
          ],
          "flags": 0,
          "description": "Duration of HTTP server requests.",
          "sum": 0.05029629999999998,
          "serviceName": "greeter",
          "schemaUrl": "",
          "unit": "s",
          "aggregationTemporality": "AGGREGATION_TEMPORALITY_CUMULATIVE",
          "exemplars": [],
          "bucketCounts": 15,
          "name": "http.server.request.duration",
          "startTime": "2024-07-23T20:11:14.547143100Z",
          "explicitBoundsCount": 14,
          "attributes": {
            "instrumentationScope.name": "Microsoft.AspNetCore.Hosting",
            "resource.attributes.telemetry@sdk@name": "opentelemetry",
            "resource.attributes.telemetry@sdk@language": "dotnet",
            "metric.attributes.http@route": "/{name?}",
            "metric.attributes.network@protocol@version": "1.1",
            "metric.attributes.http@request@method": "GET",
            "resource.attributes.service@instance@id": "d463ebb1-c319-4b74-90ed-e92284e25b81",
            "resource.attributes.telemetry@sdk@version": "1.9.0",
            "metric.attributes.http@response@status_code": 200,
            "resource.attributes.service@name": "greeter",
            "metric.attributes.url@scheme": "http"
          },
          "time": "2024-07-23T20:32:14.541131200Z",
          "explicitBounds": [
            0.005,
            0.01,
            0.025,
            0.05,
            0.075,
            0.1,
            0.25,
            0.5,
            0.75,
            1,
            2.5,
            5,
            7.5,
            10
          ]
        }
      },
      {
        "_index": "ss4o_metrics-otel-2024.07.23",
        "_id": "2a5M4ZABc9zmiGR5So6E",
        "_score": 1,
        "_source": {
          "kind": "SUM",
          "flags": 0,
          "description": "Number of requests that were attempted to be matched to an endpoint.",
          "serviceName": "greeter",
          "schemaUrl": "",
          "isMonotonic": true,
          "unit": "{match_attempt}",
          "aggregationTemporality": "AGGREGATION_TEMPORALITY_CUMULATIVE",
          "exemplars": [],
          "name": "aspnetcore.routing.match_attempts",
          "startTime": "2024-07-23T20:11:14.552176500Z",
          "attributes": {
            "instrumentationScope.name": "Microsoft.AspNetCore.Routing",
            "resource.attributes.telemetry@sdk@name": "opentelemetry",
            "resource.attributes.telemetry@sdk@language": "dotnet",
            "metric.attributes.http@route": "/{name?}",
            "metric.attributes.aspnetcore@routing@match_status": "success",
            "metric.attributes.aspnetcore@routing@is_fallback": false,
            "resource.attributes.service@instance@id": "d463ebb1-c319-4b74-90ed-e92284e25b81",
            "resource.attributes.telemetry@sdk@version": "1.9.0",
            "resource.attributes.service@name": "greeter"
          },
          "time": "2024-07-23T20:32:14.541131600Z",
          "value": 219
        }
      },
      {
        "_index": "ss4o_metrics-otel-2024.07.23",
        "_id": "2q5N4ZABc9zmiGR5S46z",
        "_score": 1,
        "_source": {
          "kind": "SUM",
          "flags": 0,
          "description": "Number of connections that are currently active on the server.",
          "serviceName": "greeter",
          "schemaUrl": "",
          "isMonotonic": false,
          "unit": "{connection}",
          "aggregationTemporality": "AGGREGATION_TEMPORALITY_CUMULATIVE",
          "exemplars": [],
          "name": "kestrel.active_connections",
          "startTime": "2024-07-23T20:11:14.546468400Z",
          "attributes": {
            "metric.attributes.network@transport": "tcp",
            "instrumentationScope.name": "Microsoft.AspNetCore.Server.Kestrel",
            "resource.attributes.telemetry@sdk@name": "opentelemetry",
            "resource.attributes.telemetry@sdk@language": "dotnet",
            "metric.attributes.network@type": "ipv4",
            "metric.attributes.server@port": 5169,
            "resource.attributes.service@instance@id": "d463ebb1-c319-4b74-90ed-e92284e25b81",
            "resource.attributes.telemetry@sdk@version": "1.9.0",
            "resource.attributes.service@name": "greeter",
            "metric.attributes.server@address": "127.0.0.1"
          },
          "time": "2024-07-23T20:33:14.542889400Z",
          "value": 1
        }
      },
      {
        "_index": "ss4o_metrics-otel-2024.07.23",
        "_id": "265N4ZABc9zmiGR5S46z",
        "_score": 1,
        "_source": {
          "kind": "HISTOGRAM",
          "buckets": [
            {
              "min": -3.4028234663852886e+38,
              "max": 0.01,
              "count": 0
            },
            {
              "min": 0.01,
              "max": 0.02,
              "count": 0
            },
            {
              "min": 0.02,
              "max": 0.05,
              "count": 4
            },
            {
              "min": 0.05,
              "max": 0.1,
              "count": 1
            },
            {
              "min": 0.1,
              "max": 0.2,
              "count": 0
            },
            {
              "min": 0.2,
              "max": 0.5,
              "count": 0
            },
            {
              "min": 0.5,
              "max": 1,
              "count": 2
            },
            {
              "min": 1,
              "max": 2,
              "count": 1
            },
            {
              "min": 2,
              "max": 5,
              "count": 5
            },
            {
              "min": 5,
              "max": 10,
              "count": 0
            },
            {
              "min": 10,
              "max": 30,
              "count": 0
            },
            {
              "min": 30,
              "max": 60,
              "count": 1
            },
            {
              "min": 60,
              "max": 120,
              "count": 3
            },
            {
              "min": 120,
              "max": 300,
              "count": 0
            },
            {
              "min": 300,
              "max": 3.4028234663852886e+38,
              "count": 0
            }
          ],
          "count": 17,
          "bucketCountsList": [
            0,
            0,
            4,
            1,
            0,
            0,
            2,
            1,
            5,
            0,
            0,
            1,
            3,
            0,
            0
          ],
          "flags": 0,
          "description": "The duration of connections on the server.",
          "sum": 413.62033599999995,
          "serviceName": "greeter",
          "schemaUrl": "",
          "unit": "s",
          "aggregationTemporality": "AGGREGATION_TEMPORALITY_CUMULATIVE",
          "exemplars": [],
          "bucketCounts": 15,
          "name": "kestrel.connection.duration",
          "startTime": "2024-07-23T20:11:14.547025400Z",
          "explicitBoundsCount": 14,
          "attributes": {
            "metric.attributes.network@transport": "tcp",
            "instrumentationScope.name": "Microsoft.AspNetCore.Server.Kestrel",
            "resource.attributes.telemetry@sdk@name": "opentelemetry",
            "resource.attributes.telemetry@sdk@language": "dotnet",
            "metric.attributes.network@protocol@version": "1.1",
            "metric.attributes.network@type": "ipv4",
            "metric.attributes.server@port": 5169,
            "resource.attributes.service@instance@id": "d463ebb1-c319-4b74-90ed-e92284e25b81",
            "resource.attributes.telemetry@sdk@version": "1.9.0",
            "resource.attributes.service@name": "greeter",
            "metric.attributes.network@protocol@name": "http",
            "metric.attributes.server@address": "127.0.0.1"
          },
          "time": "2024-07-23T20:33:14.542890200Z",
          "explicitBounds": [
            0.01,
            0.02,
            0.05,
            0.1,
            0.2,
            0.5,
            1,
            2,
            5,
            10,
            30,
            60,
            120,
            300
          ]
        }
      },
      {
        "_index": "ss4o_metrics-otel-2024.07.23",
        "_id": "3K5N4ZABc9zmiGR5S46z",
        "_score": 1,
        "_source": {
          "kind": "SUM",
          "flags": 0,
          "description": "Number of connections that are currently queued and are waiting to start.",
          "serviceName": "greeter",
          "schemaUrl": "",
          "isMonotonic": false,
          "unit": "{connection}",
          "aggregationTemporality": "AGGREGATION_TEMPORALITY_CUMULATIVE",
          "exemplars": [],
          "name": "kestrel.queued_connections",
          "startTime": "2024-07-23T20:11:14.547060500Z",
          "attributes": {
            "metric.attributes.network@transport": "tcp",
            "instrumentationScope.name": "Microsoft.AspNetCore.Server.Kestrel",
            "resource.attributes.telemetry@sdk@name": "opentelemetry",
            "resource.attributes.telemetry@sdk@language": "dotnet",
            "metric.attributes.network@type": "ipv4",
            "metric.attributes.server@port": 5169,
            "resource.attributes.service@instance@id": "d463ebb1-c319-4b74-90ed-e92284e25b81",
            "resource.attributes.telemetry@sdk@version": "1.9.0",
            "resource.attributes.service@name": "greeter",
            "metric.attributes.server@address": "127.0.0.1"
          },
          "time": "2024-07-23T20:33:14.542890900Z",
          "value": 0
        }
      },
      {
        "_index": "ss4o_metrics-otel-2024.07.23",
        "_id": "3a5N4ZABc9zmiGR5S46z",
        "_score": 1,
        "_source": {
          "kind": "SUM",
          "flags": 0,
          "description": "Number of active HTTP server requests.",
          "serviceName": "greeter",
          "schemaUrl": "",
          "isMonotonic": false,
          "unit": "{request}",
          "aggregationTemporality": "AGGREGATION_TEMPORALITY_CUMULATIVE",
          "exemplars": [],
          "name": "http.server.active_requests",
          "startTime": "2024-07-23T20:11:14.547129400Z",
          "attributes": {
            "instrumentationScope.name": "Microsoft.AspNetCore.Hosting",
            "resource.attributes.telemetry@sdk@name": "opentelemetry",
            "resource.attributes.telemetry@sdk@language": "dotnet",
            "metric.attributes.http@request@method": "GET",
            "resource.attributes.service@instance@id": "d463ebb1-c319-4b74-90ed-e92284e25b81",
            "resource.attributes.telemetry@sdk@version": "1.9.0",
            "resource.attributes.service@name": "greeter",
            "metric.attributes.url@scheme": "http"
          },
          "time": "2024-07-23T20:33:14.542893600Z",
          "value": 0
        }
      }
    ]
  }
}

Navigating to the observability/metrics in the dashboard and selecting the index emits an error and does not show any graphs:

Error

<Error: Fetch Document Names Error:Error: Fetch Otel Metrics Error:[illegal_argument_exception] Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [name] in order to load field data by uninverting the inverted index. Note that this can use significant memory. :: {"path":"/ss4o_metrics-otel-2024.07.23/_search","query":{},"body":"{\"size\":0,\"query\":{\"bool\":{\"filter\":[{\"term\":{\"kind\":\"HISTOGRAM\"}}]}},\"aggs\":{\"distinct_names\":{\"terms\":{\"field\":\"name\",\"size\":500}}}}","statusCode":400,"response":"{\"error\":{\"root_cause\":[{\"type\":\"illegal_argument_exception\",\"reason\":\"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [name] in order to load field data by uninverting the inverted index. Note that this can use significant memory.\"}],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[{\"shard\":0,\"index\":\"ss4o_metrics-otel-2024.07.23\",\"node\":\"CK85R6PpQQGaoFfL4ykxBw\",\"reason\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [name] in order to load field data by uninverting the inverted index. Note that this can use significant memory.\"}}],\"caused_by\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Text fields are not optimised fotively, set fielddata=true on [name] in order to load field data by uninverting the inverted index. Note that this can use significant memory.\",\"caused_by\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [name] in order to load field data by uninverting the inverted index. Note that this can use significant memory.\"}}},\"status\":400}"}
�    at MetricsAnalyticsAdaptor.queryToFetchDocumentNames (/usr/share/opensearch-dashboards/plugins/observabilityDashboards/server/adaptors/metrics/metrics_analytics_adaptor.ts:155:13)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at /usr/share/opensearch-dashboards/plugins/observabilityDashboards/server/routes/metrics/metrics_rounter.ts:134:22
    at Router.handle (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:174:44)
    at handler (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:140:50)
    at exports.Manager.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/toolkit.js:60:28)
    at Object.internals.handler (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:46:20)
    at exports.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:31:20)
    at Request._lifecycle (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:371:32)
    at Request._execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:281:9)
�{"type":"error","@timestamp":"2024-07-23T20:33:36Z","tags":[],"pid":1,"level":"error","error":{"message":"Internal Server Error","name":"Error","stack":"Error: Internal Server Error\n    at HapiResponseAdapter.toError (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:127:19)\n    at HapiResponseAdapter.toHapiResponse (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:83:19)\n    at HapiResponseAdapter.handle (/usr/share/opensearch-dashboards/src/core/server/http/router/response_adapter.js:79:17)\n    at Router.handle (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:175:34)\n    at processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at handler (/usr/share/opensearch-dashboards/src/core/server/http/router/router.js:140:50)\n    at exports.Manager.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/toolkit.js:60:28)\n    at Object.internals.handler (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:46:20)\n    at exports.execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/handler.js:31:20)\n    at Request._lifecycle (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:371:32)\n    at Request._execute (/usr/share/opensearch-dashboards/node_modules/@hapi/hapi/lib/request.js:281:9)"},"url":"http://localhost:5601/api/observability/metrics/otel/ss4o_metrics-otel-2024.07.23/documentNames/","message":"Internal Server Error"}
{"type":"response","@timestamp":"2024-07-23T20:33:36Z","tags":[],"pid":1,"method":"get","statusCode":500,"req":{"url":"/api/observability/metrics/otel/ss4o_metrics-otel-2024.07.23/documentNames/","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0","accept":"*/*","accept-language":"de,en-US;q=0.7,en;q=0.3","accept-encoding":"gzip, deflate, br, zstd","referer":"http://localhost:5601/app/observability-metrics","content-type":"application/json","osd-version":"2.15.0","osd-xsrf":"osd-fetch","connection":"keep-alive","sec-fetch-dest":"empty","sec-fetch-mode":"cors","sec-fetch-site":"same-origin","priority":"u=4"},"remoteAddress":"10.89.1.71","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0","referer":"http://localhost:5601/app/observability-metrics"},"res":{"statusCode":500,"responseTime":14,"contentLength":9},"message":"GET /api/observability/metrics/otel/ss4o_metrics-otel-2024.07.23/documentNames/ 500 14ms - 9.0B"}

[Roiskia]

I forgot to specify the template_type for the sink of my metric-pipeline. After setting this to index-template as in the pipeline.yml from the opentelemetry-demo the exception was gone and metrics where available.

pipeline.yml

metric-pipeline:
  source:
    otel_metrics_source:
      ssl: false
      authentication:
        unauthenticated:
  processor:
    - otel_metrics:
        calculate_histogram_buckets: true
        calculate_exponential_histogram_buckets: true
        exponential_histogram_max_allowed_scale: 10
        flatten_attributes: false
  sink:
    - stdout:
    - opensearch:
        hosts: ["http://opensearch:9200"]
        insecure: true
        index_type: custom
        template_type: index-template
        template_file: "templates/ss4o_metrics.json"
        index: ss4o_metrics-otel-%{yyyy.MM.dd}