Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[processor/transform] Add copy_metric function #30846

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .chloggen/transformprocessor-copy-metric.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: processor/transform

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Add `copy_metric` function to allow duplicating a metric

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [30846]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:

# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: []
20 changes: 20 additions & 0 deletions processor/transformprocessor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,7 @@ In addition to OTTL functions, the processor defines its own functions to help w
- [convert_gauge_to_sum](#convert_gauge_to_sum)
- [convert_summary_count_val_to_sum](#convert_summary_count_val_to_sum)
- [convert_summary_sum_val_to_sum](#convert_summary_sum_val_to_sum)
- [copy_metric](#copy_metric)

### convert_sum_to_gauge

Expand Down Expand Up @@ -307,6 +308,25 @@ Examples:

- `convert_summary_sum_val_to_sum("cumulative", false)`

### copy_metric

`copy_metric(Optional[name], Optional[description], Optional[unit])`

The `copy_metric` function copies the current metric, adding it to the end of the metric slice.

`name` is an optional string. `description` is an optional string. `unit` is an optional string.

The new metric will be exactly the same as the current metric. You can use the optional parameters to set the new metric's name, description, and unit.

**NOTE:** The new metric is appended to the end of the metric slice and therefore will be included in all the metric statements. It is a best practice to ALWAYS include a Where clause when copying a metric that WILL NOT match the new metric.

Examples:

- `copy_metric(name="http.request.status_code", unit="s") where name == "http.status_code`


- `copy_metric(desc="new desc") where description == "old desc"`
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved

## Examples

### Perform transformation if field does not exist
Expand Down
2 changes: 2 additions & 0 deletions processor/transformprocessor/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ require (
github.com/open-telemetry/opentelemetry-collector-contrib/internal/common v0.93.0
github.com/open-telemetry/opentelemetry-collector-contrib/internal/coreinternal v0.93.0
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl v0.93.0
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatatest v0.93.0
github.com/stretchr/testify v1.8.4
go.opentelemetry.io/collector/component v0.93.1-0.20240130182548-89388addcc7f
go.opentelemetry.io/collector/confmap v0.93.1-0.20240130182548-89388addcc7f
Expand Down Expand Up @@ -42,6 +43,7 @@ require (
github.com/mitchellh/reflectwalk v1.0.2 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.93.0 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/client_golang v1.18.0 // indirect
github.com/prometheus/client_model v0.5.0 // indirect
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package metrics // import "github.com/open-telemetry/opentelemetry-collector-contrib/processor/transformprocessor/internal/metrics"

import (
"context"
"fmt"

"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl"
"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl/contexts/ottlmetric"
)

type copyMetricArguments struct {
Name ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]
Description ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]
Unit ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]
}

func newCopyMetricFactory() ottl.Factory[ottlmetric.TransformContext] {
return ottl.NewFactory("copy_metric", &copyMetricArguments{}, createCopyMetricFunction)
}

func createCopyMetricFunction(_ ottl.FunctionContext, oArgs ottl.Arguments) (ottl.ExprFunc[ottlmetric.TransformContext], error) {
args, ok := oArgs.(*copyMetricArguments)

if !ok {
return nil, fmt.Errorf("createCopyMetricFunction args must be of type *copyMetricArguments")
}

return copyMetric(args.Name, args.Description, args.Unit)
}

func copyMetric(name ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]], desc ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]], unit ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]) (ottl.ExprFunc[ottlmetric.TransformContext], error) {
return func(ctx context.Context, tCtx ottlmetric.TransformContext) (any, error) {
cur := tCtx.GetMetric()
metrics := tCtx.GetMetrics()
newMetric := metrics.AppendEmpty()
cur.CopyTo(newMetric)

if !name.IsEmpty() {
n, err := name.Get().Get(ctx, tCtx)
if err != nil {
return nil, err
}
newMetric.SetName(n)
}

if !desc.IsEmpty() {
d, err := desc.Get().Get(ctx, tCtx)
if err != nil {
return nil, err
}
newMetric.SetDescription(d)
}

if !unit.IsEmpty() {
u, err := unit.Get().Get(ctx, tCtx)
if err != nil {
return nil, err
}
newMetric.SetUnit(u)
}

return nil, nil
}, nil
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package metrics

import (
"context"
"testing"

"github.com/stretchr/testify/assert"
"go.opentelemetry.io/collector/pdata/pcommon"
"go.opentelemetry.io/collector/pdata/pmetric"

"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl"
"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl/contexts/ottlmetric"
"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatatest/pmetrictest"
)

func Test_copyMetric(t *testing.T) {
tests := []struct {
testName string
name ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]
desc ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]
unit ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]
want func(s pmetric.MetricSlice)
}{
{
testName: "basic copy",
name: ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]{},
desc: ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]{},
unit: ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]{},
want: func(ms pmetric.MetricSlice) {
metric := ms.At(0)
newMetric := ms.AppendEmpty()
metric.CopyTo(newMetric)
},
},
{
testName: "set name",
name: ottl.NewTestingOptional[ottl.StringGetter[ottlmetric.TransformContext]](ottl.StandardStringGetter[ottlmetric.TransformContext]{
Getter: func(_ context.Context, tCtx ottlmetric.TransformContext) (any, error) {
return "new name", nil
},
}),
desc: ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]{},
unit: ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]{},
want: func(ms pmetric.MetricSlice) {
metric := ms.At(0)
newMetric := ms.AppendEmpty()
metric.CopyTo(newMetric)
newMetric.SetName("new name")
},
},
{
testName: "set description",
name: ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]{},
desc: ottl.NewTestingOptional[ottl.StringGetter[ottlmetric.TransformContext]](ottl.StandardStringGetter[ottlmetric.TransformContext]{
Getter: func(_ context.Context, tCtx ottlmetric.TransformContext) (any, error) {
return "new desc", nil
},
}),
unit: ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]{},
want: func(ms pmetric.MetricSlice) {
metric := ms.At(0)
newMetric := ms.AppendEmpty()
metric.CopyTo(newMetric)
newMetric.SetDescription("new desc")
},
},
{
testName: "set unit",
name: ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]{},
desc: ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]{},
unit: ottl.NewTestingOptional[ottl.StringGetter[ottlmetric.TransformContext]](ottl.StandardStringGetter[ottlmetric.TransformContext]{
Getter: func(_ context.Context, tCtx ottlmetric.TransformContext) (any, error) {
return "new unit", nil
},
}),
want: func(ms pmetric.MetricSlice) {
metric := ms.At(0)
newMetric := ms.AppendEmpty()
metric.CopyTo(newMetric)
newMetric.SetUnit("new unit")
},
},
{
testName: "set all",
name: ottl.NewTestingOptional[ottl.StringGetter[ottlmetric.TransformContext]](ottl.StandardStringGetter[ottlmetric.TransformContext]{
Getter: func(_ context.Context, tCtx ottlmetric.TransformContext) (any, error) {
return "new name", nil
},
}),
desc: ottl.NewTestingOptional[ottl.StringGetter[ottlmetric.TransformContext]](ottl.StandardStringGetter[ottlmetric.TransformContext]{
Getter: func(_ context.Context, tCtx ottlmetric.TransformContext) (any, error) {
return "new desc", nil
},
}),
unit: ottl.NewTestingOptional[ottl.StringGetter[ottlmetric.TransformContext]](ottl.StandardStringGetter[ottlmetric.TransformContext]{
Getter: func(_ context.Context, tCtx ottlmetric.TransformContext) (any, error) {
return "new unit", nil
},
}),
want: func(ms pmetric.MetricSlice) {
metric := ms.At(0)
newMetric := ms.AppendEmpty()
metric.CopyTo(newMetric)
newMetric.SetName("new name")
newMetric.SetDescription("new desc")
newMetric.SetUnit("new unit")
},
},
}
for _, tt := range tests {
t.Run(tt.testName, func(t *testing.T) {
ms := pmetric.NewMetricSlice()
input := ms.AppendEmpty()
input.SetName("test")
input.SetDescription("test")
input.SetUnit("test")
input.SetEmptySum()
d := input.Sum().DataPoints().AppendEmpty()
d.SetIntValue(1)

expected := pmetric.NewMetricSlice()
ms.CopyTo(expected)
tt.want(expected)

exprFunc, err := copyMetric(tt.name, tt.desc, tt.unit)
assert.NoError(t, err)
_, err = exprFunc(nil, ottlmetric.NewTransformContext(input, ms, pcommon.NewInstrumentationScope(), pcommon.NewResource()))
assert.NoError(t, err)

x := pmetric.NewScopeMetrics()
y := pmetric.NewScopeMetrics()

expected.CopyTo(x.Metrics())
ms.CopyTo(y.Metrics())

assert.NoError(t, pmetrictest.CompareScopeMetrics(x, y))
})
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ func MetricFunctions() map[string]ottl.Factory[ottlmetric.TransformContext] {
metricFunctions := ottl.CreateFactoryMap(
newExtractSumMetricFactory(),
newExtractCountMetricFactory(),
newCopyMetricFactory(),
)

if useConvertBetweenSumAndGaugeMetricContext.IsEnabled() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ func Test_MetricFunctions(t *testing.T) {
expected["convert_gauge_to_sum"] = newConvertGaugeToSumFactory()
expected["extract_sum_metric"] = newExtractSumMetricFactory()
expected["extract_count_metric"] = newExtractCountMetricFactory()
expected["copy_metric"] = newCopyMetricFactory()

defer testutil.SetFeatureGateForTest(t, useConvertBetweenSumAndGaugeMetricContext, true)()
actual := MetricFunctions()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,15 @@ func Test_ProcessMetrics_MetricContext(t *testing.T) {
countDp1.SetStartTimestamp(StartTimestamp)
},
},
{
statements: []string{`copy_metric(name="http.request.status_code", unit="s") where name == "operationA"`},
want: func(td pmetric.Metrics) {
newMetric := td.ResourceMetrics().At(0).ScopeMetrics().At(0).Metrics().AppendEmpty()
td.ResourceMetrics().At(0).ScopeMetrics().At(0).Metrics().At(0).CopyTo(newMetric)
newMetric.SetName("http.request.status_code")
newMetric.SetUnit("s")
},
},
}

for _, tt := range tests {
Expand Down
Loading