diff --git a/processor/ipfixlookupprocessor/Makefile b/processor/ipfixlookupprocessor/Makefile new file mode 100644 index 000000000000..ded7a36092dc --- /dev/null +++ b/processor/ipfixlookupprocessor/Makefile @@ -0,0 +1 @@ +include ../../Makefile.Common diff --git a/processor/ipfixlookupprocessor/README.md b/processor/ipfixlookupprocessor/README.md new file mode 100644 index 000000000000..061b847c2a35 --- /dev/null +++ b/processor/ipfixlookupprocessor/README.md @@ -0,0 +1,149 @@ +# IPFIX Lookup Processor + +| Status | | +| ------------- |-----------| +| Distributions | [contrib]| +| Issues |[![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aopen%20label%3Aconnector%2Fipfix%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Aconnector%2Fipfix) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aclosed%20label%3Aconnector%2Fipfix%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Aconnector%2Fipfix) | +| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | | + +[development]: https://github.com/open-telemetry/opentelemetry-collector#development +[contrib]: https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib + +## Supported Pipeline Types + +| [Exporter Pipeline Type] | [Receiver Pipeline Type] | [Stability Level] | +| ------------------------ | ------------------------ | ----------------- | +| traces | traces | [development] | + +[Exporter Pipeline Type]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/connector/README.md#exporter-pipeline-type +[Receiver Pipeline Type]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/connector/README.md#receiver-pipeline-type +[Stability Level]: https://github.com/open-telemetry/opentelemetry-collector#stability-levels + + +[ElasticSearch]: https://www.elastic.co/elasticsearch/ + +The `ipfixlookup` processor can be used to inject IPFIX spans into existing traces. + +## Prerequisits + +You need to store your Netflow/IPFIX logs in [ElasticSearch] to use this processor. Please check the [Netflow Integration](https://docs.elastic.co/en/integrations/netflow) for more information. + +### Default Configuration + +The `ipfixlookup` processorr will require the following minimum configuration. + + +For example, in the following configuration, the processor will connect to the specified [ElasticSearch] instance and search for IPFIX/Netflow events in the specified time window. +```yaml +processors: + groupbytrace: + wait_duration: 100s + num_traces: 1000 + num_workers: 2 + ipfix_lookup: + elastic_search: + connection: + addresses: + - https://:9200/ + username: elastic + password: + certificate_fingerprint: + timing: + lookup_window: 25 + +service: + pipelines: + traces: + receivers: [otlp] + processors: [groupbytrace, ipfix_lookup] + exporters: [otlp/jaeger, debug] +``` + +### Custom lookup fields + + + +Optionally, you can specify the fields the processor will look up in [ElasticSearch] and match within the spans + +| Full Path Configuration | Description | +| -------------------------------------------- | --------------------------------------------------------------------------------------------- | +| `ipfix_lookup.query_parameters` | Parameters used for querying the IPFIX lookup processor. | +| `ipfix_lookup.base_query.field_name` | The name of the field used in the base query. | +| `ipfix_lookup.base_query.field_value` | The value of the field used in the base query. | +| `ipfix_lookup.device_identifier` | The field used to identify the device in IPFIX records. | +| `ipfix_lookup.lookup_fields.source_ip` | Field representing the source IP address. | +| `ipfix_lookup.lookup_fields.source_port` | Field representing the source port. | +| `ipfix_lookup.lookup_fields.destination_ip` | Field representing the destination IP address. | +| `ipfix_lookup.lookup_fields.destination_port`| Field representing the destination port. | +| `ipfix_lookup.span_attribute_fields` | Fields to be added as attributes to the new span. In [gjson](https://gjson.dev/) format. | +| `ipfix_lookup.spans.source_ips` | Fields representing the source IP address of the span. | +| `ipfix_lookup.spans.source_ports` | Fields representing the source port of the span. | +| `ipfix_lookup.spans.destination_ip_and_port` | Fields representing the destination IP and port of the span. Like: `192.168.10.10:443` | +| `ipfix_lookup.spans.destination_ips` | Fields representing the destination IP address of the span. | +| `ipfix_lookup.spans.destination_ports` | Fields representing the destination port of the span. | + +The configuration below shows the default values: + + +```yaml +processors: + groupbytrace: + wait_duration: 10s + num_traces: 1000 + num_workers: 2 + ipfix_lookup: + query_parameters: + base_query: + field_name: input.type + field_value: netflow + device_identifier: "fields.observer\\.ip.0" + lookup_fields: + source_ip: source.ip + source_port: source.port + destination_ip: destination.ip + destination_port: destination.port + span_attribute_fields: + - "@this" + - "fields.event\\.duration.0" + - "fields.observer\\.ip.0" + - "fields.source\\.ip.0" + - "fields.source\\.port.0" + - "fields.destination\\.ip.0" + - "fields.destination\\.port.0" + - "fields.netflow\\.ip_next_hop_ipv4_address" + spans: + span_fields: + source_ips: + - net.peer.ip + - net.peer.name + - src.ip + source_ports: + - net.peer.port + - src.port + destination_ip_and_port: + - http.host + destination_ips: + - dst.ip + - net.peer.name + destination_ports: + - dst.port +``` + +### Timings: + +This processor is responsible for looking through the spans in each trace. If the IP and port quartet (`source.ip, source.port, destination.ip, destination.port`) are found in a span, the corresponding flow is looked up in ElasticSearch. When flows are found, a new span is added to the trace, and the trace is exported. + +![CorrelationUnitv3 drawio](https://github.com/open-telemetry/opentelemetry-collector-contrib/assets/15678530/5bb8e7de-c254-4991-871d-05c9c6d6f3f6) + + +The timing configuration is needed because there is an ingest delay in any large distributed search engine. Because of this, the processor must wait a bit before the search can be started. This delay can be defined in the `processors.groupbytrace.wait_duration` value. Afterwards, the search can be started. The time window that will be searched can be configured in the` processors.ipfix_lookup.timing.lookup_window`. To keep the processor simple, the lookup_window is added before the start timestamp and after the end timestamp. This way, the chance that the Netflow/IPFIX records leading or being caused by this span is found is maximized. + + +# Example screenshot +Example of a working implementation: +![finnal-implementation](https://github.com/open-telemetry/opentelemetry-collector-contrib/assets/15678530/37036d33-07f1-4c9e-bdea-7834a5e01015) +(The network was intentionally slowed down for this screenshot) + + + + diff --git a/processor/ipfixlookupprocessor/config.go b/processor/ipfixlookupprocessor/config.go new file mode 100644 index 000000000000..17517346c509 --- /dev/null +++ b/processor/ipfixlookupprocessor/config.go @@ -0,0 +1,13 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +package ipfixlookupprocessor // import "github.com/open-telemetry/opentelemetry-collector-contrib/processor/ipfixlookupprocessor" + +// Config for the processor +type Config struct { + // TODO +} + +func (c *Config) Validate() error { + return nil +} diff --git a/processor/ipfixlookupprocessor/doc.go b/processor/ipfixlookupprocessor/doc.go new file mode 100644 index 000000000000..de37a27d310e --- /dev/null +++ b/processor/ipfixlookupprocessor/doc.go @@ -0,0 +1,4 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +package ipfixlookupprocessor // import "github.com/open-telemetry/opentelemetry-collector-contrib/processor/ipfixlookupprocessor" diff --git a/processor/ipfixlookupprocessor/factory.go b/processor/ipfixlookupprocessor/factory.go new file mode 100644 index 000000000000..12f5c5f95d7c --- /dev/null +++ b/processor/ipfixlookupprocessor/factory.go @@ -0,0 +1,35 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +package ipfixlookupprocessor + +import ( + "context" + + "go.opentelemetry.io/collector/component" + "go.opentelemetry.io/collector/consumer" + "go.opentelemetry.io/collector/processor" +) + +const ( + // this is the name used to refer to the processor in the config.yaml + typeStr = "ipfixLookup" +) + +func NewFactory() processor.Factory { + + return processor.NewFactory( + typeStr, + createDefaultConfig, + processor.WithTraces(createTracesToTracesProcessor, component.StabilityLevelAlpha)) +} + +func createDefaultConfig() component.Config { + return &Config{} +} + +func createTracesToTracesProcessor(_ context.Context, params processor.CreateSettings, cfg component.Config, nextConsumer consumer.Traces) (processor.Traces, error) { + c := newProcessor(params.Logger, cfg) + c.tracesConsumer = nextConsumer + return c, nil +} diff --git a/processor/ipfixlookupprocessor/factory_test.go b/processor/ipfixlookupprocessor/factory_test.go new file mode 100644 index 000000000000..121297217dbc --- /dev/null +++ b/processor/ipfixlookupprocessor/factory_test.go @@ -0,0 +1,40 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +package ipfixlookupprocessor + +import ( + "context" + "testing" + + "github.com/stretchr/testify/assert" + "go.opentelemetry.io/collector/component" + "go.opentelemetry.io/collector/component/componenttest" + "go.opentelemetry.io/collector/consumer/consumertest" + "go.opentelemetry.io/collector/processor/processortest" +) + +func TestFactory_Type(t *testing.T) { + factory := NewFactory() + assert.Equal(t, factory.Type(), component.Type(typeStr)) +} + +func TestFactory_CreateDefaultConfig(t *testing.T) { + factory := NewFactory() + cfg := factory.CreateDefaultConfig() + assert.Equal(t, cfg, &Config{}) + assert.NoError(t, componenttest.CheckConfigStruct(cfg)) +} + +func TestNewFactory(t *testing.T) { + factory := NewFactory() + conn, err := factory.CreateTracesProcessor( + context.Background(), + processortest.NewNopCreateSettings(), + factory.CreateDefaultConfig(), + consumertest.NewNop(), + ) + + assert.NoError(t, err) + assert.NotNil(t, conn) +} diff --git a/processor/ipfixlookupprocessor/go.mod b/processor/ipfixlookupprocessor/go.mod new file mode 100644 index 000000000000..6904f95237a5 --- /dev/null +++ b/processor/ipfixlookupprocessor/go.mod @@ -0,0 +1,66 @@ +module github.com/open-telemetry/opentelemetry-collector-contrib/processor/ipfixlookupprocessor + +go 1.20 + +require ( + github.com/stretchr/testify v1.8.4 + go.opentelemetry.io/collector/component v0.88.1-0.20231026220224-6405e152a2d9 + go.opentelemetry.io/collector/consumer v0.88.1-0.20231026220224-6405e152a2d9 + go.opentelemetry.io/collector/pdata v1.0.0 + go.opentelemetry.io/collector/processor v0.88.0 + go.uber.org/zap v1.26.0 +) + +require ( + github.com/cespare/xxhash/v2 v2.2.0 // indirect + github.com/davecgh/go-spew v1.1.1 // indirect + github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.91.0 // indirect + github.com/pmezard/go-difflib v1.0.0 // indirect + go.opentelemetry.io/collector v0.88.0 // indirect + gopkg.in/yaml.v3 v3.0.1 // indirect +) + +require ( + github.com/gogo/protobuf v1.3.2 // indirect + github.com/golang/protobuf v1.5.3 // indirect + github.com/json-iterator/go v1.1.12 // indirect + github.com/knadh/koanf/maps v0.1.1 // indirect + github.com/knadh/koanf/providers/confmap v0.1.0 // indirect + github.com/knadh/koanf/v2 v2.0.1 // indirect + github.com/mitchellh/copystructure v1.2.0 // indirect + github.com/mitchellh/mapstructure v1.5.1-0.20220423185008-bf980b35cac4 // indirect + github.com/mitchellh/reflectwalk v1.0.2 // indirect + github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect + github.com/modern-go/reflect2 v1.0.2 // indirect + github.com/open-telemetry/opentelemetry-collector-contrib/pkg/golden v0.0.0-20231106091314-ad6fa27ad929 + go.opentelemetry.io/collector/config/configtelemetry v0.88.1-0.20231026220224-6405e152a2d9 // indirect + go.opentelemetry.io/collector/confmap v0.88.1-0.20231026220224-6405e152a2d9 // indirect + go.opentelemetry.io/collector/featuregate v1.0.0-rcv0017.0.20231026220224-6405e152a2d9 // indirect + go.opentelemetry.io/otel v1.19.0 // indirect + go.opentelemetry.io/otel/metric v1.19.0 // indirect + go.opentelemetry.io/otel/trace v1.19.0 // indirect + go.uber.org/multierr v1.11.0 // indirect + golang.org/x/net v0.18.0 // indirect + golang.org/x/sys v0.14.0 // indirect + golang.org/x/text v0.14.0 // indirect + google.golang.org/genproto/googleapis/rpc v0.0.0-20230822172742-b8732ec3820d // indirect + google.golang.org/grpc v1.59.0 + google.golang.org/protobuf v1.31.0 // indirect +) + +replace github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil => ../../pkg/pdatautil + +replace github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatatest => ../../pkg/pdatatest + +replace github.com/open-telemetry/opentelemetry-collector-contrib/internal/coreinternal => ../../internal/coreinternal + +replace github.com/open-telemetry/opentelemetry-collector-contrib/internal/filter => ../../internal/filter + +replace github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl => ../../pkg/ottl + +retract ( + v0.76.2 + v0.76.1 +) + +replace github.com/open-telemetry/opentelemetry-collector-contrib/pkg/golden => ../../pkg/golden diff --git a/processor/ipfixlookupprocessor/go.sum b/processor/ipfixlookupprocessor/go.sum new file mode 100644 index 000000000000..8c3711231fcb --- /dev/null +++ b/processor/ipfixlookupprocessor/go.sum @@ -0,0 +1,117 @@ +github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44= +github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/go-logr/logr v1.2.4 h1:g01GSCwiDw2xSZfjJ2/T9M+S6pFdcNtFYsp+Y43HYDQ= +github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= +github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= +github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= +github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= +github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg= +github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= +github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= +github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= +github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= +github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= +github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= +github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= +github.com/knadh/koanf/maps v0.1.1 h1:G5TjmUh2D7G2YWf5SQQqSiHRJEjaicvU0KpypqB3NIs= +github.com/knadh/koanf/maps v0.1.1/go.mod h1:npD/QZY3V6ghQDdcQzl1W4ICNVTkohC8E73eI2xW4yI= +github.com/knadh/koanf/providers/confmap v0.1.0 h1:gOkxhHkemwG4LezxxN8DMOFopOPghxRVp7JbIvdvqzU= +github.com/knadh/koanf/providers/confmap v0.1.0/go.mod h1:2uLhxQzJnyHKfxG927awZC7+fyHFdQkd697K4MdLnIU= +github.com/knadh/koanf/v2 v2.0.1 h1:1dYGITt1I23x8cfx8ZnldtezdyaZtfAuRtIFOiRzK7g= +github.com/knadh/koanf/v2 v2.0.1/go.mod h1:ZeiIlIDXTE7w1lMT6UVcNiRAS2/rCeLn/GdLNvY1Dus= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/mitchellh/copystructure v1.2.0 h1:vpKXTN4ewci03Vljg/q9QvCGUDttBOGBIa15WveJJGw= +github.com/mitchellh/copystructure v1.2.0/go.mod h1:qLl+cE2AmVv+CoeAwDPye/v+N2HKCj9FbZEVFJRxO9s= +github.com/mitchellh/mapstructure v1.5.1-0.20220423185008-bf980b35cac4 h1:BpfhmLKZf+SjVanKKhCgf3bg+511DmU9eDQTen7LLbY= +github.com/mitchellh/mapstructure v1.5.1-0.20220423185008-bf980b35cac4/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= +github.com/mitchellh/reflectwalk v1.0.2 h1:G2LzWKi524PWgd3mLHV8Y5k7s6XUvT0Gef6zxSIeXaQ= +github.com/mitchellh/reflectwalk v1.0.2/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw= +github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M= +github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= +github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= +github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk= +github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= +github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +go.opentelemetry.io/collector v0.88.0 h1:I0lerJK1h88vk7enriSgLV+h7dM099G9FgwkfmIZaf0= +go.opentelemetry.io/collector v0.88.0/go.mod h1:we0quZ+4txHS3Sfb0VdjFv95KYLGmto4ZAThCHiYgGA= +go.opentelemetry.io/collector/component v0.88.1-0.20231026220224-6405e152a2d9 h1:t9GCaQDZ1MDBjEAC1Y7NvwiqvVppK6ckAfrUEAlFioA= +go.opentelemetry.io/collector/component v0.88.1-0.20231026220224-6405e152a2d9/go.mod h1:4utKxz4Lilym3SPxNXJHosdaTjT1aQxI+TCmnJO54pU= +go.opentelemetry.io/collector/config/configtelemetry v0.88.1-0.20231026220224-6405e152a2d9 h1:4WPy3qE1lJE1LZE7t1kAj1XSZN85w68JknZO5Uo00vw= +go.opentelemetry.io/collector/config/configtelemetry v0.88.1-0.20231026220224-6405e152a2d9/go.mod h1:+LAXM5WFMW/UbTlAuSs6L/W72WC+q8TBJt/6z39FPOU= +go.opentelemetry.io/collector/confmap v0.88.1-0.20231026220224-6405e152a2d9 h1:JKFChlNpigR1Q4hZUjDU2sB2VuQ+RigAh7oOQfdcaiQ= +go.opentelemetry.io/collector/confmap v0.88.1-0.20231026220224-6405e152a2d9/go.mod h1:CSJlMk1KRZloXAygpiPeCLpuQiLVDEZYbGsGHIKHeUg= +go.opentelemetry.io/collector/consumer v0.88.1-0.20231026220224-6405e152a2d9 h1:xiosDLoF99krBlBdiZvw22CSCYU0picQMKskzjaIU8I= +go.opentelemetry.io/collector/consumer v0.88.1-0.20231026220224-6405e152a2d9/go.mod h1:VVoafgyhjpO6fuJu12GqspmuLrn91JCOou0sOtb9GOg= +go.opentelemetry.io/collector/featuregate v1.0.0-rcv0017.0.20231026220224-6405e152a2d9 h1:luvDPu+FNy6LIylBOO8PH/ca6ym7JKAdMe1J1aJbsF4= +go.opentelemetry.io/collector/featuregate v1.0.0-rcv0017.0.20231026220224-6405e152a2d9/go.mod h1:fLmJMf1AoHttkF8p5oJAc4o5ZpHu8yO5XYJ7gbLCLzo= +go.opentelemetry.io/collector/pdata v1.0.0 h1:ECP2jnLztewsHmL1opL8BeMtWVc7/oSlKNhfY9jP8ec= +go.opentelemetry.io/collector/pdata v1.0.0/go.mod h1:TsDFgs4JLNG7t6x9D8kGswXUz4mme+MyNChHx8zSF6k= +go.opentelemetry.io/collector/processor v0.88.0 h1:5BUZaH+RhTpgTVqBZCrBnN/vl0M1CtwQsZ8ek4iH1lc= +go.opentelemetry.io/collector/processor v0.88.0/go.mod h1:2T5KxgBQxXuuyMu9dh+PIBxQ/geCFYcdnjmlWZx8o3E= +go.opentelemetry.io/otel v1.19.0 h1:MuS/TNf4/j4IXsZuJegVzI1cwut7Qc00344rgH7p8bs= +go.opentelemetry.io/otel v1.19.0/go.mod h1:i0QyjOq3UPoTzff0PJB2N66fb4S0+rSbSB15/oyH9fY= +go.opentelemetry.io/otel/metric v1.19.0 h1:aTzpGtV0ar9wlV4Sna9sdJyII5jTVJEvKETPiOKwvpE= +go.opentelemetry.io/otel/metric v1.19.0/go.mod h1:L5rUsV9kM1IxCj1MmSdS+JQAcVm319EUrDVLrt7jqt8= +go.opentelemetry.io/otel/trace v1.19.0 h1:DFVQmlVbfVeOuBRrwdtaehRrWiL1JoVs9CPIQ1Dzxpg= +go.opentelemetry.io/otel/trace v1.19.0/go.mod h1:mfaSyvGyEJEI0nyV2I4qhNQnbBOUUmYZpYojqMnX2vo= +go.uber.org/goleak v1.2.1 h1:NBol2c7O1ZokfZ0LEU9K6Whx/KnwvepVetCUhtKja4A= +go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= +go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= +go.uber.org/zap v1.26.0 h1:sI7k6L95XOKS281NhVKOFCUNIvv9e0w4BF8N3u+tCRo= +go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so= +golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= +golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= +golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= +golang.org/x/net v0.18.0 h1:mIYleuAkSbHh0tCv7RvjL3F6ZVbLjq4+R7zbOn3Kokg= +golang.org/x/net v0.18.0/go.mod h1:/czyP5RqHAH4odGYxBJ1qz0+CE5WZ+2j1YgoEo8F2jQ= +golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.14.0 h1:Vz7Qs629MkJkGyHxUlRHizWJRG2j8fbQKjELVSNhy7Q= +golang.org/x/sys v0.14.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= +golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ= +golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= +golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= +golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +google.golang.org/genproto/googleapis/rpc v0.0.0-20230822172742-b8732ec3820d h1:uvYuEyMHKNt+lT4K3bN6fGswmK8qSvcreM3BwjDh+y4= +google.golang.org/genproto/googleapis/rpc v0.0.0-20230822172742-b8732ec3820d/go.mod h1:+Bk1OCOj40wS2hwAMA+aCW9ypzm63QTBBHp6lQ3p+9M= +google.golang.org/grpc v1.59.0 h1:Z5Iec2pjwb+LEOqzpB2MR12/eKFhDPhuqW91O+4bwUk= +google.golang.org/grpc v1.59.0/go.mod h1:aUPDwccQo6OTjy7Hct4AfBPD1GptF4fyUjIkQ9YtF98= +google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= +google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= +google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8= +google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/processor/ipfixlookupprocessor/internal/metadata/generated_status.go b/processor/ipfixlookupprocessor/internal/metadata/generated_status.go new file mode 100644 index 000000000000..2862eb038ab5 --- /dev/null +++ b/processor/ipfixlookupprocessor/internal/metadata/generated_status.go @@ -0,0 +1,12 @@ +// Code generated by mdatagen. DO NOT EDIT. + +package metadata + +import ( + "go.opentelemetry.io/collector/component" +) + +const ( + Type = "ipfixlookup" + TracesToTracesStability = component.StabilityLevelDevelopment +) diff --git a/processor/ipfixlookupprocessor/metadata.yaml b/processor/ipfixlookupprocessor/metadata.yaml new file mode 100644 index 000000000000..466cee2986e5 --- /dev/null +++ b/processor/ipfixlookupprocessor/metadata.yaml @@ -0,0 +1,9 @@ +type: ipfixlookup + +status: + class: connector + stability: + development: [traces_to_traces] + distributions: [contrib] + codeowners: + active: [tbd] diff --git a/processor/ipfixlookupprocessor/processor.go b/processor/ipfixlookupprocessor/processor.go new file mode 100644 index 000000000000..2f3963b187b4 --- /dev/null +++ b/processor/ipfixlookupprocessor/processor.go @@ -0,0 +1,48 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +package ipfixlookupprocessor // import "github.com/open-telemetry/opentelemetry-collector-contrib/processor/ipfixlookupprocessor" + +import ( + "context" + + "go.opentelemetry.io/collector/component" + "go.opentelemetry.io/collector/consumer" + "go.opentelemetry.io/collector/pdata/ptrace" + "go.uber.org/zap" +) + +// schema for processor +type processorImp struct { + config Config + tracesConsumer consumer.Traces + logger *zap.Logger +} + +func newProcessor(logger *zap.Logger, config component.Config) *processorImp { + logger.Info("Building ipfixlookupprocessor processor") + cfg := config.(*Config) + + return &processorImp{ + config: *cfg, + logger: logger, + } +} + +func (p *processorImp) Capabilities() consumer.Capabilities { + return consumer.Capabilities{MutatesData: true} +} + +func (p *processorImp) Start(_ context.Context, _ component.Host) error { + p.logger.Info("Starting ipfixlookupprocessor processor") + return nil +} + +func (p *processorImp) Shutdown(context.Context) error { + p.logger.Info("Shutting down ipfixlookupprocessor processor") + return nil +} + +func (p *processorImp) ConsumeTraces(ctx context.Context, td ptrace.Traces) error { + return p.tracesConsumer.ConsumeTraces(ctx, td) +} diff --git a/processor/ipfixlookupprocessor/processor_test.go b/processor/ipfixlookupprocessor/processor_test.go new file mode 100644 index 000000000000..f941511f7cf6 --- /dev/null +++ b/processor/ipfixlookupprocessor/processor_test.go @@ -0,0 +1,60 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +package ipfixlookupprocessor + +import ( + "context" + "path/filepath" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "go.opentelemetry.io/collector/component/componenttest" + "go.opentelemetry.io/collector/consumer/consumertest" + "go.uber.org/zap/zaptest" + "google.golang.org/grpc/metadata" + + "github.com/open-telemetry/opentelemetry-collector-contrib/pkg/golden" +) + +// TODO: actual tests +func TestExample(t *testing.T) { + // Create Connector + factory := NewFactory() + cfg := factory.CreateDefaultConfig().(*Config) + p := newProcessor(zaptest.NewLogger(t), cfg) + + // Fake consumer + sink := &consumertest.TracesSink{} + p.tracesConsumer = sink + + // Create Context + ctx := metadata.NewIncomingContext(context.Background(), nil) + require.NoError(t, p.Start(ctx, componenttest.NewNopHost())) + defer func() { require.NoError(t, p.Shutdown(ctx)) }() + + // traces + tracesBefore, err := golden.ReadTraces(filepath.Join("testdata", "traces", "traceBefore.yaml")) + require.NoError(t, err) + tracesAfter, err := golden.ReadTraces(filepath.Join("testdata", "traces", "traceAfter.yaml")) + assert.NoError(t, err) + // traces := buildSampleTrace() + err = p.ConsumeTraces(ctx, tracesBefore) + assert.NoError(t, err) + // golden.WriteTraces(t, filepath.Join("testdata", "metrics", "output.yaml"), sink.AllTraces()[0]) + require.Equal(t, tracesAfter, sink.AllTraces()[0]) +} + +func TestCapabilitiesMutatesData(t *testing.T) { + // Create an instance of connectorImp + c := &processorImp{} // Assuming connectorImp is the type you provided + + // Call the Capabilities method + capabilities := c.Capabilities() + + // Check if MutatesData is true + if !capabilities.MutatesData { + t.Errorf("Expected MutatesData to be true, but got false") + } +} diff --git a/processor/ipfixlookupprocessor/testdata/traces/traceAfter.yaml b/processor/ipfixlookupprocessor/testdata/traces/traceAfter.yaml new file mode 100644 index 000000000000..079dfa79814a --- /dev/null +++ b/processor/ipfixlookupprocessor/testdata/traces/traceAfter.yaml @@ -0,0 +1,651 @@ +resourceSpans: + - resource: + attributes: + - key: resource.required + value: + stringValue: foo + - key: resource.optional + value: + stringValue: bar + scopeSpans: + - scope: {} + spans: + - attributes: + - key: span.required + value: + stringValue: foo + - key: span.optional + value: + stringValue: bar + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attrs-foo-bar + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - attributes: + - key: span.required + value: + stringValue: foo + - key: span.optional + value: + stringValue: notbar + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attrs-foo-notbar + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - attributes: + - key: span.required + value: + stringValue: notfoo + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attr-notfoo + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-no-attrs + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - resource: + attributes: + - key: resource.required + value: + stringValue: foo + - key: resource.optional + value: + stringValue: notbar + scopeSpans: + - scope: {} + spans: + - attributes: + - key: span.required + value: + stringValue: foo + - key: span.optional + value: + stringValue: bar + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attrs-foo-bar + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - attributes: + - key: span.required + value: + stringValue: foo + - key: span.optional + value: + stringValue: notbar + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attrs-foo-notbar + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - attributes: + - key: span.required + value: + stringValue: notfoo + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attr-notfoo + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-no-attrs + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - resource: + attributes: + - key: resource.required + value: + stringValue: notfoo + scopeSpans: + - scope: {} + spans: + - attributes: + - key: span.required + value: + stringValue: foo + - key: span.optional + value: + stringValue: bar + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attrs-foo-bar + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - attributes: + - key: span.required + value: + stringValue: foo + - key: span.optional + value: + stringValue: notbar + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attrs-foo-notbar + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - attributes: + - key: span.required + value: + stringValue: notfoo + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attr-notfoo + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-no-attrs + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - resource: {} + scopeSpans: + - scope: {} + spans: + - attributes: + - key: span.required + value: + stringValue: foo + - key: span.optional + value: + stringValue: bar + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attrs-foo-bar + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - attributes: + - key: span.required + value: + stringValue: foo + - key: span.optional + value: + stringValue: notbar + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attrs-foo-notbar + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - attributes: + - key: span.required + value: + stringValue: notfoo + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attr-notfoo + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-no-attrs + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" diff --git a/processor/ipfixlookupprocessor/testdata/traces/traceBefore.yaml b/processor/ipfixlookupprocessor/testdata/traces/traceBefore.yaml new file mode 100644 index 000000000000..079dfa79814a --- /dev/null +++ b/processor/ipfixlookupprocessor/testdata/traces/traceBefore.yaml @@ -0,0 +1,651 @@ +resourceSpans: + - resource: + attributes: + - key: resource.required + value: + stringValue: foo + - key: resource.optional + value: + stringValue: bar + scopeSpans: + - scope: {} + spans: + - attributes: + - key: span.required + value: + stringValue: foo + - key: span.optional + value: + stringValue: bar + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attrs-foo-bar + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - attributes: + - key: span.required + value: + stringValue: foo + - key: span.optional + value: + stringValue: notbar + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attrs-foo-notbar + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - attributes: + - key: span.required + value: + stringValue: notfoo + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attr-notfoo + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-no-attrs + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - resource: + attributes: + - key: resource.required + value: + stringValue: foo + - key: resource.optional + value: + stringValue: notbar + scopeSpans: + - scope: {} + spans: + - attributes: + - key: span.required + value: + stringValue: foo + - key: span.optional + value: + stringValue: bar + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attrs-foo-bar + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - attributes: + - key: span.required + value: + stringValue: foo + - key: span.optional + value: + stringValue: notbar + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attrs-foo-notbar + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - attributes: + - key: span.required + value: + stringValue: notfoo + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attr-notfoo + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-no-attrs + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - resource: + attributes: + - key: resource.required + value: + stringValue: notfoo + scopeSpans: + - scope: {} + spans: + - attributes: + - key: span.required + value: + stringValue: foo + - key: span.optional + value: + stringValue: bar + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attrs-foo-bar + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - attributes: + - key: span.required + value: + stringValue: foo + - key: span.optional + value: + stringValue: notbar + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attrs-foo-notbar + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - attributes: + - key: span.required + value: + stringValue: notfoo + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attr-notfoo + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-no-attrs + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - resource: {} + scopeSpans: + - scope: {} + spans: + - attributes: + - key: span.required + value: + stringValue: foo + - key: span.optional + value: + stringValue: bar + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attrs-foo-bar + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - attributes: + - key: span.required + value: + stringValue: foo + - key: span.optional + value: + stringValue: notbar + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attrs-foo-notbar + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - attributes: + - key: span.required + value: + stringValue: notfoo + endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-attr-notfoo + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" + - endTimeUnixNano: "1581452773000000789" + events: + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: bar + name: event-with-attrs-foo-bar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: foo + - key: event.optional + value: + stringValue: notbar + name: event-with-attrs-foo-notbar + timeUnixNano: "1581452773000000123" + - attributes: + - key: event.required + value: + stringValue: notfoo + name: event-with-attr-notfoo + timeUnixNano: "1581452773000000123" + - name: event-with-no-attrs + timeUnixNano: "1581452773000000123" + name: span-with-no-attrs + parentSpanId: "" + spanId: "" + startTimeUnixNano: "1581452772000000321" + status: {} + traceId: "" diff --git a/versions.yaml b/versions.yaml index b0a6e9da4d5c..701782839638 100644 --- a/versions.yaml +++ b/versions.yaml @@ -145,6 +145,7 @@ module-sets: - github.com/open-telemetry/opentelemetry-collector-contrib/processor/filterprocessor - github.com/open-telemetry/opentelemetry-collector-contrib/processor/groupbyattrsprocessor - github.com/open-telemetry/opentelemetry-collector-contrib/processor/groupbytraceprocessor + - github.com/open-telemetry/opentelemetry-collector-contrib/processor/ipfixlookupprocessor - github.com/open-telemetry/opentelemetry-collector-contrib/processor/k8sattributesprocessor - github.com/open-telemetry/opentelemetry-collector-contrib/processor/logstransformprocessor - github.com/open-telemetry/opentelemetry-collector-contrib/processor/metricsgenerationprocessor