Support OTEL based Trace Index Mapping in place

[YANG-DB]

Is your feature request related to a problem?
Currently Observability In not responsible and has no control of its Observability-data indices - specifically Traces index.
An external (data-prepare) plugin is responsible for both the index structure (mapping), naming and creation.

This is problematic in the next use cases :

• It couples the plugins and create a hidden dependency for Observability
• It create an mandatory workflow for customers to first manage the ingestion pipeline even if they have their own
• Any changes to the index name or structure must be maintained in two places
• Development must be corrected by both teams even for non dependent features
• Observability Dashboards (which are part of the Observability plugin) could be now pre-defined according to the trace structure and allow customers a ready made solution

Existing Observability customers which are not using the OTEL traces indices are not effected by this change

What solution would you like?
Move the structure and naming from the ingestion to the core Observability

This would simply all use cases

• remove unnecessary coupling (both development and code )
• allow for independent Observability ingestion consumers not to be dependent on data-prepare
• allow 3rd party ingestion developers to have a consolidated location to retrieve and validate their Observability schema
  • Fluent-D
  • Jaeger
• simplify any ingestion framework lifecycle by not having to maintain or care about the lifecycle of the target index to populate

What alternatives have you considered?

• Keep the existing status in-which the responsibilities of maintaining the standard Observability indices is outside the Observability repo
• add some basic validation during loading time that assumes the indices must exist - not sure what way to verify the potential version conflicts

Do you have any additional context?
OTEL schema

• https://github.com/opensearch-project/data-prepper/tree/main/data-prepper-plugins/otel-trace-source