sumologicexporter!: change metrics behavior (open-telemetry#32737)

**Description:** <Describe what has changed.>  * remove suppport for carbon2 and graphite * add support for otlp format * do not support metadata attributes * do not support source headers * set otlp as default format This PR reduces size of open-telemetry#32315 **Link to tracking Issue:** open-telemetry#31479 **Testing:** - unit tests - manual tests **Documentation:** - Readme --------- Signed-off-by: Dominik Rosiek <[email protected]> Co-authored-by: Adam Boguszewski <[email protected]>
jpkrohling · May 8, 2024 · 061b8ea · 061b8ea
1 parent 1139675
commit 061b8ea
Show file tree

Hide file tree

Showing 19 changed files with 936 additions and 1,242 deletions.
diff --git a/.chloggen/drosiek-exporter-metrics.yaml b/.chloggen/drosiek-exporter-metrics.yaml
@@ -0,0 +1,32 @@
+# Use this changelog template to create an entry for release notes.
+
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: breaking
+
+# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
+component: sumologicexporter
+
+# A brief description of the change.  Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: change metrics behavior
+
+# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
+issues: [31479]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext: |
+  * remove suppport for carbon2 and graphite
+  * add support for otlp format
+  * do not support metadata attributes
+  * do not support source headers
+  * set otlp as default metrics format
+
+# If your change doesn't affect end users or the exported elements of any package,
+# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
+# Optional: The change log or logs in which this entry should be included.
+# e.g. '[user]' or '[user, api]'
+# Include 'user' if the change is relevant to end users.
+# Include 'api' if there is a change to a library API.
+# Default: '[user]'
+change_logs: [user]
diff --git a/exporter/sumologicexporter/README.md b/exporter/sumologicexporter/README.md
@@ -18,7 +18,7 @@
 
 For some time we have been developing the [new Sumo Logic exporter](https://github.com/SumoLogic/sumologic-otel-collector/tree/main/pkg/exporter/sumologicexporter#sumo-logic-exporter) and now we are in the process of moving it into this repository.
 
-The following options are deprecated and they will not exist in the new version:
+The following options are deprecated for logs and already do not work for metrics:
 
 - `metric_format: {carbon2, graphite}`
 - `metadata_attributes: [<regex>]`
@@ -29,8 +29,8 @@ The following options are deprecated and they will not exist in the new version:
 
 After the new exporter will be moved to this repository:
 
-- `carbon2` and `graphite` are going to be no longer supported and `prometheus` or `otlp` format should be used
-- all resource level attributes are going to be treated as `metadata_attributes`. You can use [Group by Attributes processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/groupbyattrsprocessor) to move attributes from record level to resource level. For example:
+- `carbon2` and `graphite` are no longer supported and `prometheus` or `otlp` format should be used
+- all resource level attributes are going to be treated (are treated for metrics) as `metadata_attributes`. You can use [Group by Attributes processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/groupbyattrsprocessor) to move attributes from record level to resource level. For example:
 
   ```yaml
   # before switch to new collector
@@ -45,7 +45,7 @@ After the new exporter will be moved to this repository:
         - my_attribute
   ```
 
-- Source templates (`source_category`, `source_name` and `source_host`) are going to be removed from the exporter and sources may be set using `_sourceCategory`, `sourceName` or `_sourceHost` resource attributes. We recommend to use [Transform Processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor/). For example:
+- Source templates (`source_category`, `source_name` and `source_host`) are going to be removed from the exporter and sources may be set using `_sourceCategory`, `sourceName` or `_sourceHost` resource attributes.  This feature has been already disabled for metrics. We recommend to use [Transform Processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor/). For example:
 
   ```yaml
   # before switch to new collector
@@ -95,11 +95,15 @@ exporters:
     # format to use when sending logs to Sumo Logic, default = json,
     log_format: {json, text}
 
-    # format to use when sending metrics to Sumo Logic, default = prometheus,
-    #
-    # carbon2 and graphite are deprecated:
-    # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/sumologicexporter#migration-to-new-architecture
-    metric_format: {carbon2, graphite, prometheus}
+    # format to use when sending metrics to Sumo Logic, default = otlp,
+    # NOTE: only `otlp` is supported when used with sumologicextension
+    metric_format: {otlp, prometheus}
+
+    # Decompose OTLP Histograms into individual metrics, similar to how they're represented in Prometheus format.
+    # The Sumo OTLP source currently doesn't support Histograms, and they are quietly dropped. This option produces
+    # metrics similar to when metric_format is set to prometheus.
+    # default = false
+    decompose_otlp_histograms: {true, false}
 
     # Template for Graphite format.
     # this option affects graphite format only

diff --git a/exporter/sumologicexporter/carbon_formatter.go b/exporter/sumologicexporter/carbon_formatter.go
diff --git a/exporter/sumologicexporter/carbon_formatter_test.go b/exporter/sumologicexporter/carbon_formatter_test.go
diff --git a/exporter/sumologicexporter/config.go b/exporter/sumologicexporter/config.go
@@ -38,12 +38,11 @@ type Config struct {
 	LogFormat LogFormatType `mapstructure:"log_format"`
 
 	// Metrics related configuration
-	// The format of metrics you will be sending, either graphite or carbon2 or prometheus (Default is prometheus)
-	// Possible values are `carbon2` and `prometheus`
+	// The format of metrics you will be sending, either otlp or prometheus (Default is otlp)
 	MetricFormat MetricFormatType `mapstructure:"metric_format"`
-	// Graphite template.
-	// Placeholders `%{attr_name}` will be replaced with attribute value for attr_name.
-	GraphiteTemplate string `mapstructure:"graphite_template"`
+
+	// Decompose OTLP Histograms into individual metrics, similar to how they're represented in Prometheus format
+	DecomposeOtlpHistograms bool `mapstructure:"decompose_otlp_histograms"`
 
 	// List of regexes for attributes which should be send as metadata
 	MetadataAttributes []string `mapstructure:"metadata_attributes"`
@@ -92,12 +91,14 @@ const (
 	TextFormat LogFormatType = "text"
 	// JSONFormat represents log_format: json
 	JSONFormat LogFormatType = "json"
+	// RemovedGraphiteFormat represents the no longer supported graphite metric format
+	RemovedGraphiteFormat MetricFormatType = "graphite"
+	// RemovedCarbon2Format represents the no longer supported carbon2 metric format
+	RemovedCarbon2Format MetricFormatType = "carbon2"
 	// GraphiteFormat represents metric_format: text
-	GraphiteFormat MetricFormatType = "graphite"
-	// Carbon2Format represents metric_format: json
-	Carbon2Format MetricFormatType = "carbon2"
-	// PrometheusFormat represents metric_format: json
 	PrometheusFormat MetricFormatType = "prometheus"
+	// OTLPMetricFormat represents metric_format: otlp
+	OTLPMetricFormat MetricFormatType = "otlp"
 	// GZIPCompression represents compress_encoding: gzip
 	GZIPCompression CompressEncodingType = "gzip"
 	// DeflateCompression represents compress_encoding: deflate
@@ -119,7 +120,7 @@ const (
 	// DefaultLogFormat defines default LogFormat
 	DefaultLogFormat LogFormatType = JSONFormat
 	// DefaultMetricFormat defines default MetricFormat
-	DefaultMetricFormat MetricFormatType = PrometheusFormat
+	DefaultMetricFormat MetricFormatType = OTLPMetricFormat
 	// DefaultSourceCategory defines default SourceCategory
 	DefaultSourceCategory string = ""
 	// DefaultSourceName defines default SourceName
@@ -141,9 +142,12 @@ func (cfg *Config) Validate() error {
 	}
 
 	switch cfg.MetricFormat {
-	case GraphiteFormat:
-	case Carbon2Format:
+	case RemovedGraphiteFormat:
+		fallthrough
+	case RemovedCarbon2Format:
+		return fmt.Errorf("%s metric format is no longer supported", cfg.MetricFormat)
 	case PrometheusFormat:
+	case OTLPMetricFormat:
 	default:
 		return fmt.Errorf("unexpected metric format: %s", cfg.MetricFormat)
 	}

diff --git a/exporter/sumologicexporter/config_test.go b/exporter/sumologicexporter/config_test.go
@@ -22,7 +22,7 @@ func TestConfigValidation(t *testing.T) {
 			name: "invalid log format",
 			cfg: &Config{
 				LogFormat:        "test_format",
-				MetricFormat:     "carbon2",
+				MetricFormat:     "otlp",
 				CompressEncoding: "gzip",
 				ClientConfig: confighttp.ClientConfig{
 					Timeout:  defaultTimeout,
@@ -48,7 +48,7 @@ func TestConfigValidation(t *testing.T) {
 			name: "invalid compress encoding",
 			cfg: &Config{
 				LogFormat:        "json",
-				MetricFormat:     "carbon2",
+				MetricFormat:     "otlp",
 				CompressEncoding: "test_format",
 				ClientConfig: confighttp.ClientConfig{
 					Timeout:  defaultTimeout,
@@ -62,7 +62,7 @@ func TestConfigValidation(t *testing.T) {
 			expectedErr: "no endpoint and no auth extension specified",
 			cfg: &Config{
 				LogFormat:        "json",
-				MetricFormat:     "carbon2",
+				MetricFormat:     "otlp",
 				CompressEncoding: "gzip",
 				ClientConfig: confighttp.ClientConfig{
 					Timeout: defaultTimeout,
@@ -73,7 +73,7 @@ func TestConfigValidation(t *testing.T) {
 			name: "invalid log format",
 			cfg: &Config{
 				LogFormat:        "json",
-				MetricFormat:     "carbon2",
+				MetricFormat:     "otlp",
 				CompressEncoding: "gzip",
 				ClientConfig: confighttp.ClientConfig{
 					Timeout:  defaultTimeout,
@@ -90,7 +90,7 @@ func TestConfigValidation(t *testing.T) {
 			name: "valid config",
 			cfg: &Config{
 				LogFormat:        "json",
-				MetricFormat:     "carbon2",
+				MetricFormat:     "otlp",
 				CompressEncoding: "gzip",
 				ClientConfig: confighttp.ClientConfig{
 					Timeout:  defaultTimeout,