From 1f70d5aa9970181957ff2723d4051cce81df7b84 Mon Sep 17 00:00:00 2001 From: Marc Alff Date: Mon, 26 Jun 2023 14:32:07 +0200 Subject: [PATCH 1/5] Fixes #3551 --- .../sdk-environment-variables.md | 64 ------------------- 1 file changed, 64 deletions(-) diff --git a/specification/configuration/sdk-environment-variables.md b/specification/configuration/sdk-environment-variables.md index 6c341deecef..8b2f0d264a2 100644 --- a/specification/configuration/sdk-environment-variables.md +++ b/specification/configuration/sdk-environment-variables.md @@ -178,70 +178,6 @@ See the SDK [LogRecord Limits](../logs/sdk.md#logrecord-limits) section for the See [OpenTelemetry Protocol Exporter Configuration Options](../protocol/exporter.md). -## Jaeger Exporter - -**Status**: [Deprecated](../document-status.md) - -Jaeger exporter support will be removed from OpenTelemetry in July 2023. - -_Note: Jaeger supports the [OpenTelemetry protocol natively][jaeger_otlp] and most users -should export to Jaeger using OTLP. These environment variables remain here -only for backwards compatibility and will be removed in a future version. SDKs MAY include -Jaeger exporters, but Jaeger export is not required._ - -The `OTEL_EXPORTER_JAEGER_PROTOCOL` environment variable -MAY by used to specify the transport protocol. -The value MUST be one of: - -- `http/thrift.binary` - [Thrift over HTTP][jaeger_http] -- `grpc` - [gRPC][jaeger_grpc] -- `udp/thrift.compact` - [Thrift with compact encoding over UDP][jaeger_udp] -- `udp/thrift.binary` - [Thrift with binary encoding over UDP][jaeger_udp] - -[jaeger_http]: https://www.jaegertracing.io/docs/latest/apis/#thrift-over-http-stable -[jaeger_grpc]: https://www.jaegertracing.io/docs/latest/apis/#protobuf-via-grpc-stable -[jaeger_udp]: https://www.jaegertracing.io/docs/latest/apis/#thrift-over-udp-stable -[jaeger_otlp]: https://www.jaegertracing.io/docs/latest/apis/#opentelemetry-protocol-stable - -The default transport protocol SHOULD be `http/thrift.binary` unless -SDKs have good reasons to choose other as the default -(e.g. for backward compatibility reasons). - -Environment variables specific for the `http/thrift.binary` transport protocol: - -| Name | Description | Default | -|-------------------------------|------------------------------------------------------------------------------------|-------------------------------------| -| OTEL_EXPORTER_JAEGER_ENDPOINT | Full URL of the [Jaeger HTTP endpoint][jaeger_collector] | `http://localhost:14268/api/traces` | -| OTEL_EXPORTER_JAEGER_TIMEOUT | Maximum time (in milliseconds) the Jaeger exporter will wait for each batch export | 10000 | -| OTEL_EXPORTER_JAEGER_USER | Username to be used for HTTP basic authentication | | -| OTEL_EXPORTER_JAEGER_PASSWORD | Password to be used for HTTP basic authentication | | - -Environment variables specific for the `grpc` transport protocol: - -| Name | Description | Default | -|-------------------------------|------------------------------------------------------------------------------------|--------------------------| -| OTEL_EXPORTER_JAEGER_ENDPOINT | URL of the [Jaeger gRPC endpoint][jaeger_collector] | `http://localhost:14250` | -| OTEL_EXPORTER_JAEGER_TIMEOUT | Maximum time (in milliseconds) the Jaeger exporter will wait for each batch export | 10000 | -| OTEL_EXPORTER_JAEGER_USER | Username to be used for HTTP basic authentication | | -| OTEL_EXPORTER_JAEGER_PASSWORD | Password to be used for HTTP basic authentication | | - -Environment variables specific for the `udp/thrift.compact` transport protocol: - -| Name | Description | Default | -|---------------------------------|---------------------------------------------------------------|-------------| -| OTEL_EXPORTER_JAEGER_AGENT_HOST | Hostname of the [Jaeger agent][jaeger_agent] | `localhost` | -| OTEL_EXPORTER_JAEGER_AGENT_PORT | `udp/thrift.compact` port of the [Jaeger agent][jaeger_agent] | `6831` | - -Environment variables specific for the `udp/thrift.binary` transport protocol: - -| Name | Description | Default | -|---------------------------------|--------------------------------------------------------------|-------------| -| OTEL_EXPORTER_JAEGER_AGENT_HOST | Hostname of the [Jaeger agent][jaeger_agent] | `localhost` | -| OTEL_EXPORTER_JAEGER_AGENT_PORT | `udp/thrift.binary` port of the [Jaeger agent][jaeger_agent] | `6832` | - -[jaeger_collector]: https://www.jaegertracing.io/docs/latest/deployment/#collector -[jaeger_agent]: https://www.jaegertracing.io/docs/latest/deployment/#agent - ## Zipkin Exporter **Status**: [Stable](../document-status.md) From ed475c523f2fab39874f38a5703a0cefce648c9a Mon Sep 17 00:00:00 2001 From: Marc Alff Date: Mon, 26 Jun 2023 15:41:48 +0200 Subject: [PATCH 2/5] Remove Jaeger exporter, Jaeger propagator. --- experimental/trace/zpages.md | 2 +- spec-compliance-matrix.md | 18 +- specification/common/mapping-to-non-otlp.md | 4 +- .../sdk-environment-variables.md | 2 - specification/context/api-propagators.md | 1 - specification/library-guidelines.md | 5 +- specification/overview.md | 2 +- specification/trace/sdk.md | 2 +- specification/trace/sdk_exporters/jaeger.md | 166 ------------------ 9 files changed, 8 insertions(+), 194 deletions(-) delete mode 100644 specification/trace/sdk_exporters/jaeger.md diff --git a/experimental/trace/zpages.md b/experimental/trace/zpages.md index 8fc7785e1dd..f1dd70fd255 100644 --- a/experimental/trace/zpages.md +++ b/experimental/trace/zpages.md @@ -26,7 +26,7 @@ zPages are an in-process alternative to external exporters. When included, they The idea of "zPages" originates from one of OpenTelemetry's predecessors, [OpenCensus](https://opencensus.io/). You can read more about zPages from the OpenCensus docs [here](https://opencensus.io/zpages) or the OTEP [here](https://github.com/open-telemetry/oteps/blob/main/text/0110-z-pages.md). OpenCensus has different zPage implementations in [Java](https://opencensus.io/zpages/java/), [Go](https://opencensus.io/zpages/go/), and [Node](https://opencensus.io/zpages/node/) and there has been similar internal solutions developed at companies like Uber. Within OpenTelemetry, zPages are also either developed or being developed in Java, and C++. The OTel Collector also has [an implementation](https://github.com/open-telemetry/opentelemetry-collector/tree/master/extension/zpagesextension) of zPages. -zPages are uniquely useful in a couple of different ways. One is that they're more lightweight and quicker compared to installing external tracing systems like Jaeger and Zipkin, yet they still share useful ways to debug and gain insight into instrumented applications; these uses depend on the type of zPage, which is detailed below. For high throughput applications, zPages can also analyze more telemetry with the limited set of supported scenarios than external exporters; this is because zPages are in-memory while external exporters are typically configured to send a subset of telemetry for reach analysis to save costs. +zPages are uniquely useful in a couple of different ways. One is that they're more lightweight and quicker compared to installing external tracing systems like Zipkin, yet they still share useful ways to debug and gain insight into instrumented applications; these uses depend on the type of zPage, which is detailed below. For high throughput applications, zPages can also analyze more telemetry with the limited set of supported scenarios than external exporters; this is because zPages are in-memory while external exporters are typically configured to send a subset of telemetry for reach analysis to save costs. ## Types of zPages diff --git a/spec-compliance-matrix.md b/spec-compliance-matrix.md index bab5fb3e72d..943f830ad53 100644 --- a/spec-compliance-matrix.md +++ b/spec-compliance-matrix.md @@ -9,7 +9,7 @@ status of the feature is not known. For the `Optional` column, `X` means the feature is optional, blank means the feature is required, and columns marked with `*` mean that for each type of -exporter (OTLP, Zipkin, and Jaeger), implementing at least one of the supported +exporter (OTLP and Zipkin), implementing at least one of the supported formats is required. Implementing more than one format is optional. ## Traces @@ -257,7 +257,6 @@ Disclaimer: this list of features is still a work in progress, please refer to t | Global Propagator | | + | + | + | + | + | + | + | + | + | + | + | | TraceContext Propagator | | + | + | + | + | + | + | + | + | + | + | + | | B3 Propagator | | + | + | + | + | + | + | + | + | + | + | + | -| Jaeger Propagator | | + | + | + | + | + | + | | + | + | - | - | | OT Propagator | | + | + | + | + | | | | | | | | | OpenCensus Binary Propagator | | + | | | | | | | | | | | | [TextMapPropagator](specification/context/api-propagators.md#textmap-propagator) | | + | + | | | + | | + | | | | | @@ -280,7 +279,6 @@ Note: Support for environment variables is optional. | OTEL_BSP_* | + | + | + | + | + | + | + | + | - | - | - | | OTEL_BLRP_* | | + | | | | | | | | | | | OTEL_EXPORTER_OTLP_* | + | + | | + | + | + | + | + | + | + | - | -| OTEL_EXPORTER_JAEGER_* | + | + | | + | + | - | - | | - | + | - | | OTEL_EXPORTER_ZIPKIN_* | - | + | | + | + | - | + | - | - | + | - | | OTEL_TRACES_EXPORTER | - | + | + | + | + | + | + | - | - | - | | | OTEL_METRICS_EXPORTER | - | + | | + | - | - | + | - | - | - | - | @@ -340,17 +338,6 @@ Note: Support for environment variables is optional. | Error Status mapping | | + | + | | + | + | - | + | + | + | + | - | | Event attributes mapping to Annotations | | + | + | + | + | + | + | + | + | + | + | + | | Integer microseconds in timestamps | | N/A| + | | + | + | - | + | + | + | + | + | -| **[Jaeger](specification/trace/sdk_exporters/jaeger.md)** | Optional | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift | -| [Jaeger Thrift over UDP][jaegerThriftUDP] | X | + | | | + | + | - | - | + | + | + | + | -| [Jaeger Protobuf via gRPC][jaegerProtobuf] | X | - | + | | + | - | - | - | | - | - | - | -| [Jaeger Thrift over HTTP][jaegerThriftHTTP] | X | + | - | | + | + | - | - | + | + | + | - | -| Service name mapping | | + | + | | + | + | - | - | | + | + | + | -| Resource to Process mapping | | + | + | | + | + | - | - | + | - | + | - | -| InstrumentationLibrary mapping | | + | + | | + | + | - | - | + | - | + | - | -| InstrumentationScope mapping | | | + | | + | | | | | | | | -| Status mapping | | + | + | | + | + | - | - | + | + | + | + | -| Error Status mapping | | + | + | | + | + | - | - | + | + | + | - | -| Events converted to Logs | | + | + | | + | + | - | - | + | - | + | + | | **OpenCensus** | | | | | | | | | | | | | | TBD | | | | | | | | | | | | | | **Prometheus** | | | | | | | | | | | | | @@ -377,6 +364,3 @@ Languages not covered by the OpenTracing project do not need to be listed here, [py1174]: https://github.com/open-telemetry/opentelemetry-python/issues/1174 [py1779]: https://github.com/open-telemetry/opentelemetry-python/issues/1779 [php225]: https://github.com/open-telemetry/opentelemetry-php/issues/225 -[jaegerThriftUDP]: https://www.jaegertracing.io/docs/latest/apis/#thrift-over-udp-stable -[jaegerProtobuf]: https://www.jaegertracing.io/docs/latest/apis/#protobuf-via-grpc-stable -[jaegerThriftHTTP]: https://www.jaegertracing.io/docs/latest/apis/#thrift-over-http-stable diff --git a/specification/common/mapping-to-non-otlp.md b/specification/common/mapping-to-non-otlp.md index a82dac3e63a..03d24f30370 100644 --- a/specification/common/mapping-to-non-otlp.md +++ b/specification/common/mapping-to-non-otlp.md @@ -16,8 +16,8 @@ Note: when a format has a direct semantic equivalent for a particular field or concept then the recommendation in this document MUST be ignored. See also additional specific transformation rules for -[Jaeger](../trace/sdk_exporters/jaeger.md) and [Zipkin](../trace/sdk_exporters/zipkin.md). -The specific rules for Jaeger and Zipkin take precedence over the generic rules defined +[Zipkin](../trace/sdk_exporters/zipkin.md). +The specific rules for Zipkin take precedence over the generic rules defined in this document. ## Mappings diff --git a/specification/configuration/sdk-environment-variables.md b/specification/configuration/sdk-environment-variables.md index 8b2f0d264a2..b83bc29ef6a 100644 --- a/specification/configuration/sdk-environment-variables.md +++ b/specification/configuration/sdk-environment-variables.md @@ -88,7 +88,6 @@ Known values for `OTEL_PROPAGATORS` are: - `"baggage"`: [W3C Baggage](https://www.w3.org/TR/baggage/) - `"b3"`: [B3 Single](../context/api-propagators.md#configuration) - `"b3multi"`: [B3 Multi](../context/api-propagators.md#configuration) -- `"jaeger"`: [Jaeger](https://www.jaegertracing.io/docs/1.21/client-libraries/#propagation-format) - `"xray"`: [AWS X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/xray-concepts.html#xray-concepts-tracingheader) (_third party_) - `"ottrace"`: [OT Trace](https://github.com/opentracing?q=basic&type=&language=) (_third party_) - `"none"`: No automatically configured propagator. @@ -222,7 +221,6 @@ The SDK MAY accept a comma-separated list to enable setting multiple exporters. Known values for `OTEL_TRACES_EXPORTER` are: - `"otlp"`: [OTLP](../protocol/otlp.md) -- `"jaeger"`: export in Jaeger data model - `"zipkin"`: [Zipkin](https://zipkin.io/zipkin-api/) (Defaults to [protobuf](https://github.com/openzipkin/zipkin-api/blob/master/zipkin.proto) format) - `"none"`: No automatically configured exporter for traces. diff --git a/specification/context/api-propagators.md b/specification/context/api-propagators.md index a1c85ee0f6c..d77814de09c 100644 --- a/specification/context/api-propagators.md +++ b/specification/context/api-propagators.md @@ -338,7 +338,6 @@ organization and MUST be distributed as OpenTelemetry extension packages: * [W3C Baggage](https://w3c.github.io/baggage). MAY alternatively be distributed as part of the OpenTelemetry API. * [B3](https://github.com/openzipkin/b3-propagation). -* [Jaeger](https://www.jaegertracing.io/docs/latest/client-libraries/#propagation-format). This is a list of additional propagators that MAY be maintained and distributed as OpenTelemetry extension packages: diff --git a/specification/library-guidelines.md b/specification/library-guidelines.md index dd837c08d91..a18e5429b78 100644 --- a/specification/library-guidelines.md +++ b/specification/library-guidelines.md @@ -28,7 +28,6 @@ _Note to OpenTelemetry client Authors:_ OpenTelemetry specification, API and SDK - metrics - Prometheus. - trace - - Jaeger. - Zipkin. Note: some of these support multiple protocols (e.g. gRPC, Thrift, etc). The exact list of protocols to implement in the exporters is TBD. @@ -93,8 +92,8 @@ name of the library should be prefixed with the terms "OpenTelemetry" and "Expor For example: -- Python and Java: opentelemetry-exporter-jaeger -- Javascript: @opentelemetry/exporter-jeager +- Python and Java: opentelemetry-exporter-{vendor_name} +- Javascript: @opentelemetry/exporter-{vendor_name} #### Resource Detection diff --git a/specification/overview.md b/specification/overview.md index 31de4d233f0..57068e0c606 100644 --- a/specification/overview.md +++ b/specification/overview.md @@ -361,7 +361,7 @@ The Propagators API currently defines one `Propagator` type: The OpenTelemetry collector is a set of components that can collect traces, metrics and eventually other telemetry data (e.g. logs) from processes -instrumented by OpenTelemetry or other monitoring/tracing libraries (Jaeger, +instrumented by OpenTelemetry or other monitoring/tracing libraries (Zipkin, Prometheus, etc.), do aggregation and smart sampling, and export traces and metrics to one or more monitoring/tracing backends. The collector will allow to enrich and transform collected telemetry (e.g. add additional attributes or diff --git a/specification/trace/sdk.md b/specification/trace/sdk.md index b6e18747ebb..16778c59d83 100644 --- a/specification/trace/sdk.md +++ b/specification/trace/sdk.md @@ -483,7 +483,7 @@ in the SDK: +-----+--------------+ +-------------------------+ +-------------------+ | | | | | | | | | | | Batching Span Processor | | SpanExporter | - | | +---> Simple Span Processor +---> (JaegerExporter) | + | | +---> Simple Span Processor +---> (ZipkinExporter) | | | | | | | | | SDK | Span.start() | +-------------------------+ +-------------------+ | | Span.end() | diff --git a/specification/trace/sdk_exporters/jaeger.md b/specification/trace/sdk_exporters/jaeger.md deleted file mode 100644 index 10247012ff2..00000000000 --- a/specification/trace/sdk_exporters/jaeger.md +++ /dev/null @@ -1,166 +0,0 @@ - - -# OpenTelemetry to Jaeger Transformation - -**Status**: [Stable](../../document-status.md) - -This document defines the transformation between OpenTelemetry and Jaeger Spans. -The generic transformation [rules specified here](../../common/mapping-to-non-otlp.md) also apply. If a -particular generic transformation rule and the rule in this document contradict -then the rule in this document MUST be used. - -Jaeger accepts spans in the following formats: - -* OpenTelemetry Protocol (OTLP), defined in [opentelemetry-proto](https://github.com/open-telemetry/opentelemetry-proto) -* Thrift `Batch`, defined in [jaeger-idl/.../jaeger.thrift](https://github.com/jaegertracing/jaeger-idl/blob/main/thrift/jaeger.thrift), accepted via UDP or HTTP -* Protobuf `Batch`, defined in [jaeger-idl/.../model.proto](https://github.com/jaegertracing/jaeger-idl/blob/main/proto/api_v2/model.proto), accepted via gRPC - -See also: - -* [Jaeger APIs](https://www.jaegertracing.io/docs/latest/apis/) -* [Reference implementation of this translation in the OpenTelemetry Collector Contrib repository](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/translator/jaeger/traces_to_jaegerproto.go) - -## Summary - -The following table summarizes the major transformations between OpenTelemetry -and Jaeger. - -| OpenTelemetry | Jaeger Thrift | Jaeger Proto | Notes | -| -------------------------- | ---------------- | ---------------- | ----- | -| Span.TraceId | Span.traceIdLow/High | Span.trace_id | See [IDs](#ids) | -| Span.ParentId | Span.parentSpanId | as SpanReference | See [Parent ID](#parent-id) | -| Span.SpanId | Span.spanId | Span.span_id | | -| Span.TraceState | TBD | TBD | | -| Span.Name | Span.operationName | Span.operation_name | | -| Span.Kind | Span.tags["span.kind"] | same | See [SpanKind](#spankind) for values mapping | -| Span.StartTime | Span.startTime | Span.start_time | See [Unit of time](#unit-of-time) | -| Span.EndTime | Span.duration | same | Calculated as EndTime - StartTime. See also [Unit of time](#unit-of-time) | -| Span.Attributes | Span.tags | same | See [Attributes](#attributes) for data types for the mapping. | -| Span.DroppedAttributesCount| Add to Span.tags | same | See [Dropped Attributes Count](../../common/mapping-to-non-otlp.md#dropped-attributes-count) for tag name to use. | -| Span.Events | Span.logs | same | See [Events](#events) for the mapping format. | -| Span.DroppedEventsCount | Add to Span.tags | same | See [Dropped Events Count](../../common/mapping-to-non-otlp.md#dropped-events-count) for tag name to use. | -| Span.Links | Span.references | same | See [Links](#links) | -| Span.DroppedLinksCount | Add to Span.tags | same | See [Dropped Links Count](../../common/mapping-to-non-otlp.md#dropped-links-count) for tag name to use. | -| Span.Status | Add to Span.tags | same | See [Status](#status) for tag names to use. | - -## Mappings - -This section discusses the details of the transformations between OpenTelemetry -and Jaeger. - -### Resource - -OpenTelemetry resources MUST be mapped to Jaeger's `Span.Process` tags. Multiple resources can exist for a -single process and exporters need to handle this case accordingly. - -Critically, Jaeger backend depends on `Span.Process.ServiceName` to identify the service -that produced the spans. That field MUST be populated from the `service.name` attribute -of the [`service` resource](../../resource/semantic_conventions/README.md#service). -If no `service.name` is contained in a Span's Resource, that field MUST be populated from the -[default](../../resource/sdk.md#sdk-provided-resource-attributes) `Resource`. - -### IDs - -Trace and span IDs in Jaeger are random sequences of bytes. However, Thrift model -represents IDs using `i64` type, or in case of a 128-bit wide Trace ID as two `i64` -fields `traceIdLow` and `traceIdHigh`. The bytes MUST be converted to/from unsigned -ints using Big Endian byte order, e.g. `[0x10, 0x00, 0x00, 0x00] == 268435456`. -The unsigned ints MUST be converted to `i64` by re-interpreting the existing -64bit value as signed `i64`. For example (in Go): - -```go -var ( - id []byte = []byte{0xFF, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00} - unsigned uint64 = binary.BigEndian.Uint64(id) - signed int64 = int64(unsigned) -) -fmt.Println("unsigned:", unsigned) -fmt.Println(" signed:", signed) -// Output: -// unsigned: 18374686479671623680 -// signed: -72057594037927936 -``` - -### Parent ID - -Jaeger Thrift format allows capturing parent ID in a top-level Span field. -Jaeger Proto format does not support parent ID field; instead the parent -MUST be recorded as a `SpanReference` of type `CHILD_OF`, e.g.: - -```python - SpanReference( - ref_type=opentracing.CHILD_OF, - trace_id=span.context.trace_id, - span_id=parent_id, - ) -``` - -This span reference MUST be the first in the list of references. - -### SpanKind - -OpenTelemetry `SpanKind` field MUST be encoded as `span.kind` tag in Jaeger span, -except for `SpanKind.INTERNAL`, which SHOULD NOT be translated to a tag. - -| OpenTelemetry | Jaeger | -| ------------- | ------ | -| `SpanKind.CLIENT`|`"client"`| -| `SpanKind.SERVER`|`"server"`| -| `SpanKind.CONSUMER`|`"consumer"`| -| `SpanKind.PRODUCER`|`"producer"` | -| `SpanKind.INTERNAL`| do not add `span.kind` tag | - -### Unit of time - -In Jaeger Thrift format the timestamps and durations MUST be represented in -microseconds (since epoch for timestamps). If the original value in OpenTelemetry -is expressed in nanoseconds, it MUST be rounded or truncated to microseconds. - -In Jaeger Proto format the timestamps and durations MUST be represented -with nanosecond precision using `google.protobuf.Timestamp` and -`google.protobuf.Duration` types. - -### Status - -The Status is recorded as Span tags. See [Status](../../common/mapping-to-non-otlp.md#span-status) for -tag names to use. - -#### Error flag - -When Span `Status` is set to `ERROR`, an `error` span tag MUST be added with the -Boolean value of `true`. The added `error` tag MAY override any previous value. - -### Attributes - -OpenTelemetry Span `Attribute`(s) MUST be reported as `tags` to Jaeger. - -Primitive types MUST be represented by the corresponding types of Jaeger tags. - -Array values MUST be serialized to string like a JSON list as mentioned in -[semantic conventions](../../overview.md#semantic-conventions). - -### Links - -OpenTelemetry `Link`(s) MUST be converted to `SpanReference`(s) in Jaeger, -using `FOLLOWS_FROM` reference type. The Link's attributes cannot be represented -in Jaeger explicitly. The exporter MAY additionally convert `Link`(s) to span `Log`(s): - -* use Span start time as the timestamp of the Log -* set Log tag `event=link` -* set Log tags `trace_id` and `span_id` from the respective `SpanContext`'s fields -* store `Link`'s attributes as Log tags - -Span references generated from `Link`(s) MUST be added _after_ the span reference -generated from [Parent ID](#parent-id), if any. - -### Events - -Events MUST be converted to Jaeger Logs. OpenTelemetry Event's `time_unix_nano` and `attributes` fields map directly to Jaeger Log's `timestamp` and `fields` fields. Jaeger Log has no direct equivalent for OpenTelemetry Event's `name` field but OpenTracing semantic conventions specify some special attribute names [here](https://github.com/opentracing/specification/blob/master/semantic_conventions.md#log-fields-table). OpenTelemetry Event's `name` field should be added to Jaeger Log's `fields` map as follows: - -| OpenTelemetry Event Field | Jaeger Attribute | -| -------------------------- | ----------------- | -| `name`|`event`| - -* If OpenTelemetry Event contains an attributes with the key `event`, it should take precedence over Event's `name` field. From f54b15c67757af94a523c60e15725b1ca4d343f6 Mon Sep 17 00:00:00 2001 From: Marc Alff Date: Mon, 26 Jun 2023 16:16:29 +0200 Subject: [PATCH 3/5] Add CHANGELOG --- CHANGELOG.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3b9db4a4037..ddd9af25b21 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,9 @@ release. ### Traces +- Remove the Jaeger Propagator + ([#3567](https://github.com/open-telemetry/opentelemetry-specification/pull/3567)) + ### Metrics - Refine SDK MeterProvider configuration section. @@ -29,6 +32,9 @@ release. - Extract Examplar section and mark it as Experimental. ([#3533](https://github.com/open-telemetry/opentelemetry-specification/pull/3533)) +- Remove the Jaeger Exporter + ([#3567](https://github.com/open-telemetry/opentelemetry-specification/pull/3567)) + ### Telemetry Schemas ### Common From dc79d42ccda2abf61ef22223f5f4087ed1c7e734 Mon Sep 17 00:00:00 2001 From: Marc Alff Date: Mon, 26 Jun 2023 16:28:23 +0200 Subject: [PATCH 4/5] Remove reference to JaegerPropagator --- specification/context/api-propagators.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specification/context/api-propagators.md b/specification/context/api-propagators.md index d77814de09c..453c38b172f 100644 --- a/specification/context/api-propagators.md +++ b/specification/context/api-propagators.md @@ -215,8 +215,8 @@ Required arguments: The `Keys` function can be called by `Propagator`s which are using variable key names in order to iterate over all the keys in the specified carrier. -For example, it can be used to detect all keys following the `uberctx-{user-defined-key}` pattern, as defined by the -[Jaeger Propagation Format](https://www.jaegertracing.io/docs/1.18/client-libraries/#baggage). +For example, it can be used to detect all keys following the `X-B3-{key}` pattern, as defined by the +[B3 Propagation Format](https://github.com/openzipkin/b3-propagation). ##### Get From c923ceca272fab56c6842c9d1b5b7a14cb0d0bb7 Mon Sep 17 00:00:00 2001 From: Marc Alff Date: Mon, 26 Jun 2023 23:13:56 +0200 Subject: [PATCH 5/5] Restored the Jaeger Propagator Fixed review comments. --- CHANGELOG.md | 5 +- experimental/trace/zpages.md | 2 +- spec-compliance-matrix.md | 1 + specification/common/mapping-to-non-otlp.md | 4 +- .../sdk-environment-variables.md | 1 + specification/context/api-propagators.md | 5 +- specification/overview.md | 2 +- specification/trace/sdk.md | 12 +- specification/trace/sdk_exporters/jaeger.md | 166 ++++++++++++++++++ 9 files changed, 182 insertions(+), 16 deletions(-) create mode 100644 specification/trace/sdk_exporters/jaeger.md diff --git a/CHANGELOG.md b/CHANGELOG.md index ddd9af25b21..46faaa7fd89 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,9 +11,6 @@ release. ### Traces -- Remove the Jaeger Propagator - ([#3567](https://github.com/open-telemetry/opentelemetry-specification/pull/3567)) - ### Metrics - Refine SDK MeterProvider configuration section. @@ -32,7 +29,7 @@ release. - Extract Examplar section and mark it as Experimental. ([#3533](https://github.com/open-telemetry/opentelemetry-specification/pull/3533)) -- Remove the Jaeger Exporter +- BREAKING: Remove the Jaeger Exporter ([#3567](https://github.com/open-telemetry/opentelemetry-specification/pull/3567)) ### Telemetry Schemas diff --git a/experimental/trace/zpages.md b/experimental/trace/zpages.md index f1dd70fd255..8fc7785e1dd 100644 --- a/experimental/trace/zpages.md +++ b/experimental/trace/zpages.md @@ -26,7 +26,7 @@ zPages are an in-process alternative to external exporters. When included, they The idea of "zPages" originates from one of OpenTelemetry's predecessors, [OpenCensus](https://opencensus.io/). You can read more about zPages from the OpenCensus docs [here](https://opencensus.io/zpages) or the OTEP [here](https://github.com/open-telemetry/oteps/blob/main/text/0110-z-pages.md). OpenCensus has different zPage implementations in [Java](https://opencensus.io/zpages/java/), [Go](https://opencensus.io/zpages/go/), and [Node](https://opencensus.io/zpages/node/) and there has been similar internal solutions developed at companies like Uber. Within OpenTelemetry, zPages are also either developed or being developed in Java, and C++. The OTel Collector also has [an implementation](https://github.com/open-telemetry/opentelemetry-collector/tree/master/extension/zpagesextension) of zPages. -zPages are uniquely useful in a couple of different ways. One is that they're more lightweight and quicker compared to installing external tracing systems like Zipkin, yet they still share useful ways to debug and gain insight into instrumented applications; these uses depend on the type of zPage, which is detailed below. For high throughput applications, zPages can also analyze more telemetry with the limited set of supported scenarios than external exporters; this is because zPages are in-memory while external exporters are typically configured to send a subset of telemetry for reach analysis to save costs. +zPages are uniquely useful in a couple of different ways. One is that they're more lightweight and quicker compared to installing external tracing systems like Jaeger and Zipkin, yet they still share useful ways to debug and gain insight into instrumented applications; these uses depend on the type of zPage, which is detailed below. For high throughput applications, zPages can also analyze more telemetry with the limited set of supported scenarios than external exporters; this is because zPages are in-memory while external exporters are typically configured to send a subset of telemetry for reach analysis to save costs. ## Types of zPages diff --git a/spec-compliance-matrix.md b/spec-compliance-matrix.md index 943f830ad53..1a0a55303c8 100644 --- a/spec-compliance-matrix.md +++ b/spec-compliance-matrix.md @@ -257,6 +257,7 @@ Disclaimer: this list of features is still a work in progress, please refer to t | Global Propagator | | + | + | + | + | + | + | + | + | + | + | + | | TraceContext Propagator | | + | + | + | + | + | + | + | + | + | + | + | | B3 Propagator | | + | + | + | + | + | + | + | + | + | + | + | +| Jaeger Propagator | | + | + | + | + | + | + | | + | + | - | - | | OT Propagator | | + | + | + | + | | | | | | | | | OpenCensus Binary Propagator | | + | | | | | | | | | | | | [TextMapPropagator](specification/context/api-propagators.md#textmap-propagator) | | + | + | | | + | | + | | | | | diff --git a/specification/common/mapping-to-non-otlp.md b/specification/common/mapping-to-non-otlp.md index 03d24f30370..a82dac3e63a 100644 --- a/specification/common/mapping-to-non-otlp.md +++ b/specification/common/mapping-to-non-otlp.md @@ -16,8 +16,8 @@ Note: when a format has a direct semantic equivalent for a particular field or concept then the recommendation in this document MUST be ignored. See also additional specific transformation rules for -[Zipkin](../trace/sdk_exporters/zipkin.md). -The specific rules for Zipkin take precedence over the generic rules defined +[Jaeger](../trace/sdk_exporters/jaeger.md) and [Zipkin](../trace/sdk_exporters/zipkin.md). +The specific rules for Jaeger and Zipkin take precedence over the generic rules defined in this document. ## Mappings diff --git a/specification/configuration/sdk-environment-variables.md b/specification/configuration/sdk-environment-variables.md index b83bc29ef6a..7b81a788e23 100644 --- a/specification/configuration/sdk-environment-variables.md +++ b/specification/configuration/sdk-environment-variables.md @@ -88,6 +88,7 @@ Known values for `OTEL_PROPAGATORS` are: - `"baggage"`: [W3C Baggage](https://www.w3.org/TR/baggage/) - `"b3"`: [B3 Single](../context/api-propagators.md#configuration) - `"b3multi"`: [B3 Multi](../context/api-propagators.md#configuration) +- `"jaeger"`: [Jaeger](https://www.jaegertracing.io/docs/1.21/client-libraries/#propagation-format) - `"xray"`: [AWS X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/xray-concepts.html#xray-concepts-tracingheader) (_third party_) - `"ottrace"`: [OT Trace](https://github.com/opentracing?q=basic&type=&language=) (_third party_) - `"none"`: No automatically configured propagator. diff --git a/specification/context/api-propagators.md b/specification/context/api-propagators.md index 453c38b172f..a1c85ee0f6c 100644 --- a/specification/context/api-propagators.md +++ b/specification/context/api-propagators.md @@ -215,8 +215,8 @@ Required arguments: The `Keys` function can be called by `Propagator`s which are using variable key names in order to iterate over all the keys in the specified carrier. -For example, it can be used to detect all keys following the `X-B3-{key}` pattern, as defined by the -[B3 Propagation Format](https://github.com/openzipkin/b3-propagation). +For example, it can be used to detect all keys following the `uberctx-{user-defined-key}` pattern, as defined by the +[Jaeger Propagation Format](https://www.jaegertracing.io/docs/1.18/client-libraries/#baggage). ##### Get @@ -338,6 +338,7 @@ organization and MUST be distributed as OpenTelemetry extension packages: * [W3C Baggage](https://w3c.github.io/baggage). MAY alternatively be distributed as part of the OpenTelemetry API. * [B3](https://github.com/openzipkin/b3-propagation). +* [Jaeger](https://www.jaegertracing.io/docs/latest/client-libraries/#propagation-format). This is a list of additional propagators that MAY be maintained and distributed as OpenTelemetry extension packages: diff --git a/specification/overview.md b/specification/overview.md index 57068e0c606..31de4d233f0 100644 --- a/specification/overview.md +++ b/specification/overview.md @@ -361,7 +361,7 @@ The Propagators API currently defines one `Propagator` type: The OpenTelemetry collector is a set of components that can collect traces, metrics and eventually other telemetry data (e.g. logs) from processes -instrumented by OpenTelemetry or other monitoring/tracing libraries (Zipkin, +instrumented by OpenTelemetry or other monitoring/tracing libraries (Jaeger, Prometheus, etc.), do aggregation and smart sampling, and export traces and metrics to one or more monitoring/tracing backends. The collector will allow to enrich and transform collected telemetry (e.g. add additional attributes or diff --git a/specification/trace/sdk.md b/specification/trace/sdk.md index 16778c59d83..deeb3916a84 100644 --- a/specification/trace/sdk.md +++ b/specification/trace/sdk.md @@ -480,12 +480,12 @@ The following diagram shows `SpanProcessor`'s relationship to other components in the SDK: ``` - +-----+--------------+ +-------------------------+ +-------------------+ - | | | | | | | - | | | | Batching Span Processor | | SpanExporter | - | | +---> Simple Span Processor +---> (ZipkinExporter) | - | | | | | | | - | SDK | Span.start() | +-------------------------+ +-------------------+ + +-----+--------------+ +-------------------------+ +----------------+ + | | | | | | | + | | | | Batching Span Processor | | SpanExporter | + | | +---> Simple Span Processor +---> (OTLPExporter) | + | | | | | | | + | SDK | Span.start() | +-------------------------+ +----------------+ | | Span.end() | | | | | | | diff --git a/specification/trace/sdk_exporters/jaeger.md b/specification/trace/sdk_exporters/jaeger.md new file mode 100644 index 00000000000..10247012ff2 --- /dev/null +++ b/specification/trace/sdk_exporters/jaeger.md @@ -0,0 +1,166 @@ + + +# OpenTelemetry to Jaeger Transformation + +**Status**: [Stable](../../document-status.md) + +This document defines the transformation between OpenTelemetry and Jaeger Spans. +The generic transformation [rules specified here](../../common/mapping-to-non-otlp.md) also apply. If a +particular generic transformation rule and the rule in this document contradict +then the rule in this document MUST be used. + +Jaeger accepts spans in the following formats: + +* OpenTelemetry Protocol (OTLP), defined in [opentelemetry-proto](https://github.com/open-telemetry/opentelemetry-proto) +* Thrift `Batch`, defined in [jaeger-idl/.../jaeger.thrift](https://github.com/jaegertracing/jaeger-idl/blob/main/thrift/jaeger.thrift), accepted via UDP or HTTP +* Protobuf `Batch`, defined in [jaeger-idl/.../model.proto](https://github.com/jaegertracing/jaeger-idl/blob/main/proto/api_v2/model.proto), accepted via gRPC + +See also: + +* [Jaeger APIs](https://www.jaegertracing.io/docs/latest/apis/) +* [Reference implementation of this translation in the OpenTelemetry Collector Contrib repository](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/translator/jaeger/traces_to_jaegerproto.go) + +## Summary + +The following table summarizes the major transformations between OpenTelemetry +and Jaeger. + +| OpenTelemetry | Jaeger Thrift | Jaeger Proto | Notes | +| -------------------------- | ---------------- | ---------------- | ----- | +| Span.TraceId | Span.traceIdLow/High | Span.trace_id | See [IDs](#ids) | +| Span.ParentId | Span.parentSpanId | as SpanReference | See [Parent ID](#parent-id) | +| Span.SpanId | Span.spanId | Span.span_id | | +| Span.TraceState | TBD | TBD | | +| Span.Name | Span.operationName | Span.operation_name | | +| Span.Kind | Span.tags["span.kind"] | same | See [SpanKind](#spankind) for values mapping | +| Span.StartTime | Span.startTime | Span.start_time | See [Unit of time](#unit-of-time) | +| Span.EndTime | Span.duration | same | Calculated as EndTime - StartTime. See also [Unit of time](#unit-of-time) | +| Span.Attributes | Span.tags | same | See [Attributes](#attributes) for data types for the mapping. | +| Span.DroppedAttributesCount| Add to Span.tags | same | See [Dropped Attributes Count](../../common/mapping-to-non-otlp.md#dropped-attributes-count) for tag name to use. | +| Span.Events | Span.logs | same | See [Events](#events) for the mapping format. | +| Span.DroppedEventsCount | Add to Span.tags | same | See [Dropped Events Count](../../common/mapping-to-non-otlp.md#dropped-events-count) for tag name to use. | +| Span.Links | Span.references | same | See [Links](#links) | +| Span.DroppedLinksCount | Add to Span.tags | same | See [Dropped Links Count](../../common/mapping-to-non-otlp.md#dropped-links-count) for tag name to use. | +| Span.Status | Add to Span.tags | same | See [Status](#status) for tag names to use. | + +## Mappings + +This section discusses the details of the transformations between OpenTelemetry +and Jaeger. + +### Resource + +OpenTelemetry resources MUST be mapped to Jaeger's `Span.Process` tags. Multiple resources can exist for a +single process and exporters need to handle this case accordingly. + +Critically, Jaeger backend depends on `Span.Process.ServiceName` to identify the service +that produced the spans. That field MUST be populated from the `service.name` attribute +of the [`service` resource](../../resource/semantic_conventions/README.md#service). +If no `service.name` is contained in a Span's Resource, that field MUST be populated from the +[default](../../resource/sdk.md#sdk-provided-resource-attributes) `Resource`. + +### IDs + +Trace and span IDs in Jaeger are random sequences of bytes. However, Thrift model +represents IDs using `i64` type, or in case of a 128-bit wide Trace ID as two `i64` +fields `traceIdLow` and `traceIdHigh`. The bytes MUST be converted to/from unsigned +ints using Big Endian byte order, e.g. `[0x10, 0x00, 0x00, 0x00] == 268435456`. +The unsigned ints MUST be converted to `i64` by re-interpreting the existing +64bit value as signed `i64`. For example (in Go): + +```go +var ( + id []byte = []byte{0xFF, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00} + unsigned uint64 = binary.BigEndian.Uint64(id) + signed int64 = int64(unsigned) +) +fmt.Println("unsigned:", unsigned) +fmt.Println(" signed:", signed) +// Output: +// unsigned: 18374686479671623680 +// signed: -72057594037927936 +``` + +### Parent ID + +Jaeger Thrift format allows capturing parent ID in a top-level Span field. +Jaeger Proto format does not support parent ID field; instead the parent +MUST be recorded as a `SpanReference` of type `CHILD_OF`, e.g.: + +```python + SpanReference( + ref_type=opentracing.CHILD_OF, + trace_id=span.context.trace_id, + span_id=parent_id, + ) +``` + +This span reference MUST be the first in the list of references. + +### SpanKind + +OpenTelemetry `SpanKind` field MUST be encoded as `span.kind` tag in Jaeger span, +except for `SpanKind.INTERNAL`, which SHOULD NOT be translated to a tag. + +| OpenTelemetry | Jaeger | +| ------------- | ------ | +| `SpanKind.CLIENT`|`"client"`| +| `SpanKind.SERVER`|`"server"`| +| `SpanKind.CONSUMER`|`"consumer"`| +| `SpanKind.PRODUCER`|`"producer"` | +| `SpanKind.INTERNAL`| do not add `span.kind` tag | + +### Unit of time + +In Jaeger Thrift format the timestamps and durations MUST be represented in +microseconds (since epoch for timestamps). If the original value in OpenTelemetry +is expressed in nanoseconds, it MUST be rounded or truncated to microseconds. + +In Jaeger Proto format the timestamps and durations MUST be represented +with nanosecond precision using `google.protobuf.Timestamp` and +`google.protobuf.Duration` types. + +### Status + +The Status is recorded as Span tags. See [Status](../../common/mapping-to-non-otlp.md#span-status) for +tag names to use. + +#### Error flag + +When Span `Status` is set to `ERROR`, an `error` span tag MUST be added with the +Boolean value of `true`. The added `error` tag MAY override any previous value. + +### Attributes + +OpenTelemetry Span `Attribute`(s) MUST be reported as `tags` to Jaeger. + +Primitive types MUST be represented by the corresponding types of Jaeger tags. + +Array values MUST be serialized to string like a JSON list as mentioned in +[semantic conventions](../../overview.md#semantic-conventions). + +### Links + +OpenTelemetry `Link`(s) MUST be converted to `SpanReference`(s) in Jaeger, +using `FOLLOWS_FROM` reference type. The Link's attributes cannot be represented +in Jaeger explicitly. The exporter MAY additionally convert `Link`(s) to span `Log`(s): + +* use Span start time as the timestamp of the Log +* set Log tag `event=link` +* set Log tags `trace_id` and `span_id` from the respective `SpanContext`'s fields +* store `Link`'s attributes as Log tags + +Span references generated from `Link`(s) MUST be added _after_ the span reference +generated from [Parent ID](#parent-id), if any. + +### Events + +Events MUST be converted to Jaeger Logs. OpenTelemetry Event's `time_unix_nano` and `attributes` fields map directly to Jaeger Log's `timestamp` and `fields` fields. Jaeger Log has no direct equivalent for OpenTelemetry Event's `name` field but OpenTracing semantic conventions specify some special attribute names [here](https://github.com/opentracing/specification/blob/master/semantic_conventions.md#log-fields-table). OpenTelemetry Event's `name` field should be added to Jaeger Log's `fields` map as follows: + +| OpenTelemetry Event Field | Jaeger Attribute | +| -------------------------- | ----------------- | +| `name`|`event`| + +* If OpenTelemetry Event contains an attributes with the key `event`, it should take precedence over Event's `name` field.