From d1c68a9241bafc9d4b340335a4567fad5eb303e9 Mon Sep 17 00:00:00 2001 From: dmathieu <42@dmathieu.com> Date: Mon, 10 Jun 2024 16:08:11 +0200 Subject: [PATCH 01/10] start rfc for experimental profiling --- docs/rfcs/experimental-profiling.md | 86 +++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+) create mode 100644 docs/rfcs/experimental-profiling.md diff --git a/docs/rfcs/experimental-profiling.md b/docs/rfcs/experimental-profiling.md new file mode 100644 index 00000000000..1b790ca6f4f --- /dev/null +++ b/docs/rfcs/experimental-profiling.md @@ -0,0 +1,86 @@ +# Experimental support for profiling + +## Overview + +The OpenTelemetry collector has traces, metrics and logs as stable signals. We +want to start experimenting with support for profiles as an experimental +signal. But we also don't want to introduce breaking changes in packages +otherwise considered stable. + +This document describes: + +* The approach we intend to take to introduce profiling with no breaking changes +* How the migration will happen once profiling goes stable too + +### Discarded approach + +A first approach, discussed in [issue +10207](https://github.com/open-telemetry/opentelemetry-collector/issues/10207) +has been discarded. +It aimed to refactor the current packages with per-signal subpackages, so each +subpackage could have its own stability level, like pdata does. + +This has been discarded, as the Collector SIG does not want the profiling +signal to impact the road to the collector reaching 1.0. + +## Proposed approach + +The proposed approach will consist in two main phases: + +* Introduce `experimental` packages for each required module of the collector that needs to be profiles-aware. + * `consumer`, `receiver`, `connector`, `component`, `processor` +* Mark specific APIs as `experimental` in their godoc for parts that can't be a new package. + * `service` + +### Introduce `experimental` packages + +Each package that needs to be signal-aware will have its barebones copied into +a new `XXexperimental` package (eg: `consumerexperimental`) which will be able +to handle the profiling signal. + +The stable related stable packages MUST not be aware of their experimental +counterpart. +The experimental package MAY call its stable implementation. + +### Mark specific APIs as `experimental` + +In order to boot a functional collector with profiles support, the some stable +packages need to be aware of the experimental ones. + +The only package concerned here should be `service`. + +To support that case, we will mark new APIs as `experimental` with go docs. +Every experimental API will be documented as such: + +```golang +// # Experimental +// +// Notice: This method is EXPERIMENTAL and may be changed or removed in a +// later release. +``` + +As documented, APIs marked as experimental may changed or removed across +releases, without it being considered as a breaking change. + +#### User specified configuration + +The user-specified configuration will let users specify a `profiles` pipeline: + +``` +service: + pipelines: + profiles: + receivers: [otlp] + processors: [batch] + exporters: [otlp] +``` + +## Signal status change + +If the profiling signal becomes stable, all the experimental packages will be +merged back into their stable counterpart, and the `service` module's imports +will be updated. + +If the profiling signal is removed, all the experimental packages will be +removed from the repository, and support for them will be removed in the +`service` module. From 09da245c2a553a8405a965d780cc583b6a45f09b Mon Sep 17 00:00:00 2001 From: Damien Mathieu <42@dmathieu.com> Date: Mon, 10 Jun 2024 19:39:59 +0200 Subject: [PATCH 02/10] Update docs/rfcs/experimental-profiling.md Co-authored-by: Christos Kalkanis --- docs/rfcs/experimental-profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/rfcs/experimental-profiling.md b/docs/rfcs/experimental-profiling.md index 1b790ca6f4f..5c489334e9c 100644 --- a/docs/rfcs/experimental-profiling.md +++ b/docs/rfcs/experimental-profiling.md @@ -25,7 +25,7 @@ signal to impact the road to the collector reaching 1.0. ## Proposed approach -The proposed approach will consist in two main phases: +The proposed approach will consist of two main phases: * Introduce `experimental` packages for each required module of the collector that needs to be profiles-aware. * `consumer`, `receiver`, `connector`, `component`, `processor` From 4cea9785277705828e4840ceccaf5bb2cecd56a3 Mon Sep 17 00:00:00 2001 From: Damien Mathieu <42@dmathieu.com> Date: Mon, 10 Jun 2024 19:40:05 +0200 Subject: [PATCH 03/10] Update docs/rfcs/experimental-profiling.md Co-authored-by: Christos Kalkanis --- docs/rfcs/experimental-profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/rfcs/experimental-profiling.md b/docs/rfcs/experimental-profiling.md index 5c489334e9c..31b73b1d76f 100644 --- a/docs/rfcs/experimental-profiling.md +++ b/docs/rfcs/experimental-profiling.md @@ -10,7 +10,7 @@ otherwise considered stable. This document describes: * The approach we intend to take to introduce profiling with no breaking changes -* How the migration will happen once profiling goes stable too +* How the migration will happen once profiling goes stable ### Discarded approach From 21dc63dc708ce4fd41fd695371ca6b120777e70e Mon Sep 17 00:00:00 2001 From: Damien Mathieu <42@dmathieu.com> Date: Mon, 10 Jun 2024 19:40:11 +0200 Subject: [PATCH 04/10] Update docs/rfcs/experimental-profiling.md Co-authored-by: Christos Kalkanis --- docs/rfcs/experimental-profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/rfcs/experimental-profiling.md b/docs/rfcs/experimental-profiling.md index 31b73b1d76f..d8a51e46c62 100644 --- a/docs/rfcs/experimental-profiling.md +++ b/docs/rfcs/experimental-profiling.md @@ -34,7 +34,7 @@ The proposed approach will consist of two main phases: ### Introduce `experimental` packages -Each package that needs to be signal-aware will have its barebones copied into +Each package that needs to be profiling signal-aware will have its barebones copied into a new `XXexperimental` package (eg: `consumerexperimental`) which will be able to handle the profiling signal. From f7e8532fe004e9f342d901c4d4440aec80421898 Mon Sep 17 00:00:00 2001 From: Damien Mathieu <42@dmathieu.com> Date: Mon, 10 Jun 2024 19:40:16 +0200 Subject: [PATCH 05/10] Update docs/rfcs/experimental-profiling.md Co-authored-by: Christos Kalkanis --- docs/rfcs/experimental-profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/rfcs/experimental-profiling.md b/docs/rfcs/experimental-profiling.md index d8a51e46c62..0dbba36d58c 100644 --- a/docs/rfcs/experimental-profiling.md +++ b/docs/rfcs/experimental-profiling.md @@ -44,7 +44,7 @@ The experimental package MAY call its stable implementation. ### Mark specific APIs as `experimental` -In order to boot a functional collector with profiles support, the some stable +In order to boot a functional collector with profiles support, some stable packages need to be aware of the experimental ones. The only package concerned here should be `service`. From f61d65301a476af7d273b594580dd4d698035f98 Mon Sep 17 00:00:00 2001 From: Damien Mathieu <42@dmathieu.com> Date: Mon, 10 Jun 2024 19:40:21 +0200 Subject: [PATCH 06/10] Update docs/rfcs/experimental-profiling.md Co-authored-by: Christos Kalkanis --- docs/rfcs/experimental-profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/rfcs/experimental-profiling.md b/docs/rfcs/experimental-profiling.md index 0dbba36d58c..1bfb4687690 100644 --- a/docs/rfcs/experimental-profiling.md +++ b/docs/rfcs/experimental-profiling.md @@ -47,7 +47,7 @@ The experimental package MAY call its stable implementation. In order to boot a functional collector with profiles support, some stable packages need to be aware of the experimental ones. -The only package concerned here should be `service`. +Currently, the only package this applies to is `service`. To support that case, we will mark new APIs as `experimental` with go docs. Every experimental API will be documented as such: From e657c50f89b6ed8f10d0a4a054435b11c83ff1c4 Mon Sep 17 00:00:00 2001 From: dmathieu <42@dmathieu.com> Date: Tue, 11 Jun 2024 15:03:58 +0200 Subject: [PATCH 07/10] add discarded build tags --- docs/rfcs/experimental-profiling.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/docs/rfcs/experimental-profiling.md b/docs/rfcs/experimental-profiling.md index 1bfb4687690..685de3b17ff 100644 --- a/docs/rfcs/experimental-profiling.md +++ b/docs/rfcs/experimental-profiling.md @@ -12,7 +12,9 @@ This document describes: * The approach we intend to take to introduce profiling with no breaking changes * How the migration will happen once profiling goes stable -### Discarded approach +### Discarded approaches + +#### Refactor everything into per-signal subpackages A first approach, discussed in [issue 10207](https://github.com/open-telemetry/opentelemetry-collector/issues/10207) @@ -23,6 +25,17 @@ subpackage could have its own stability level, like pdata does. This has been discarded, as the Collector SIG does not want the profiling signal to impact the road to the collector reaching 1.0. +#### Use build tags + +An approach would have been to use build tags to limit the availability of +profiles within packages. + +This approach would make the UX very bad though, as most packages are meant to +be imported and not used in a compiled collector. It would therefore not have +been possible to specify the appropriate build tags. + +This has been discarded, as the usage would have been too difficult. + ## Proposed approach The proposed approach will consist of two main phases: From 6f2945ec5fb5424b3cfdc6a05da339627a78fbb8 Mon Sep 17 00:00:00 2001 From: dmathieu <42@dmathieu.com> Date: Thu, 13 Jun 2024 09:35:46 +0200 Subject: [PATCH 08/10] log a warning at boot when experimental signals are used --- docs/rfcs/experimental-profiling.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/rfcs/experimental-profiling.md b/docs/rfcs/experimental-profiling.md index 685de3b17ff..b3dc374554b 100644 --- a/docs/rfcs/experimental-profiling.md +++ b/docs/rfcs/experimental-profiling.md @@ -88,6 +88,9 @@ service: exporters: [otlp] ``` +When an experimental signal is being used, the collector will log a warning at +boot. + ## Signal status change If the profiling signal becomes stable, all the experimental packages will be From 13caa79273ec63721ac71f1c20b4b2f25519b7cb Mon Sep 17 00:00:00 2001 From: dmathieu <42@dmathieu.com> Date: Mon, 17 Jun 2024 16:01:56 +0200 Subject: [PATCH 09/10] introduce experimental subpackages instead --- docs/rfcs/experimental-profiling.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/docs/rfcs/experimental-profiling.md b/docs/rfcs/experimental-profiling.md index b3dc374554b..3095dfe07b8 100644 --- a/docs/rfcs/experimental-profiling.md +++ b/docs/rfcs/experimental-profiling.md @@ -45,15 +45,20 @@ The proposed approach will consist of two main phases: * Mark specific APIs as `experimental` in their godoc for parts that can't be a new package. * `service` -### Introduce `experimental` packages +### Introduce "experimental" subpackages -Each package that needs to be profiling signal-aware will have its barebones copied into -a new `XXexperimental` package (eg: `consumerexperimental`) which will be able -to handle the profiling signal. +Each package that needs to be profiling signal-aware will have its public +methods and interfaces moves into an internal subpackage. -The stable related stable packages MUST not be aware of their experimental -counterpart. -The experimental package MAY call its stable implementation. +Then, the original package will get similar API methods and interfaces as the +ones currently available on the main branch. + +The profiling methods and interfaces will be made available in a `profiles` +subpackage. + +See [PR +#10253](https://github.com/open-telemetry/opentelemetry-collector/pull/10253) +for an example. ### Mark specific APIs as `experimental` From 074efdbd686926e3add18d9548d09f755a55d69b Mon Sep 17 00:00:00 2001 From: dmathieu <42@dmathieu.com> Date: Tue, 18 Jun 2024 12:05:25 +0200 Subject: [PATCH 10/10] no experimental symbols for now --- docs/rfcs/experimental-profiling.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/rfcs/experimental-profiling.md b/docs/rfcs/experimental-profiling.md index 3095dfe07b8..e00ef16c738 100644 --- a/docs/rfcs/experimental-profiling.md +++ b/docs/rfcs/experimental-profiling.md @@ -65,8 +65,6 @@ for an example. In order to boot a functional collector with profiles support, some stable packages need to be aware of the experimental ones. -Currently, the only package this applies to is `service`. - To support that case, we will mark new APIs as `experimental` with go docs. Every experimental API will be documented as such: @@ -80,6 +78,9 @@ Every experimental API will be documented as such: As documented, APIs marked as experimental may changed or removed across releases, without it being considered as a breaking change. +There are no symbols that would need to be marked as experimental today. If +there ever are then implementers may add an experimental comment to them + #### User specified configuration The user-specified configuration will let users specify a `profiles` pipeline: