Migrate Resource Processor to internal data model #1315

dmitryax · 2020-07-10T01:31:02Z

Description:
This PR migrates resource processor to internal data model.

Existing processor configuration is relevant only to OpenCensus format, so the configuration schema has to be changed. Taking this opportunity, this commit adds existing logic for attributes manipulation from attributes processor to resource processor.
New config uses "attributes" field which represents actions that can be made on resource attributes. Supported actions: INSERT, UPDATE, UPSERT, DELETE, HASH, EXTRACT.
In order to migrate existing resource processor config:

Move key/values from "labels" field to "attributes" with action="upsert".
Add value from "type" field to "attributes" with key="opencensus.resourcetype" and action="upsert".

Link to tracking Issue:
Resolves: #1307

Testing:
Added unit tests

codecov · 2020-07-10T01:38:10Z

Codecov Report

Merging #1315 into master will increase coverage by 0.00%.
The diff coverage is 98.41%.

@@           Coverage Diff           @@
##           master    #1315   +/-   ##
=======================================
  Coverage   89.61%   89.61%           
=======================================
  Files         215      215           
  Lines       15181    15185    +4     
=======================================
+ Hits        13604    13608    +4     
- Misses       1149     1150    +1     
+ Partials      428      427    -1

Impacted Files	Coverage Δ
internal/processor/attraction/attraction.go	`100.00% <ø> (ø)`
processor/resourceprocessor/resource_processor.go	`75.00% <96.42%> (-11.89%)`	⬇️
processor/resourceprocessor/factory.go	`100.00% <100.00%> (ø)`
translator/internaldata/resource_to_oc.go	`88.37% <0.00%> (+2.32%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1c85165...b489311. Read the comment docs.

nilebox · 2020-07-10T05:04:30Z

processor/resourceprocessor/testdata/config.yaml

  resource/2:
-    type: "host"


Can you preserve the resource type for backward compatibility, please?
I believe it will be an attribute

"opencensus.resourcetype": "host"

based on

opentelemetry-collector/translator/conventions/opencensus.go

Line 24 in 474407c

OCAttributeResourceType = "opencensus.resourcetype"

Yes, I was also thinking about it, but since we are in beta state and going through some breaking changes between releases anyway, decided to drop it right away.
But probably you right, it might be too aggressive and we can mark it as deprecated for now ...

nilebox · 2020-07-10T05:10:33Z

processor/resourceprocessor/resource_processor_test.go

-			TypeVal: "resource",
-			NameVal: "resource",
-		},
-		ResourceType: "host",


Same here: please replace this with "opencensus.type": "host" attribute rather than deleting it.

added the backward compatibility

tigrannajaryan · 2020-07-10T20:19:38Z

processor/resourceprocessor/README.md

-      "cloud.zone": "zone-1",
-      "k8s.cluster.name": "k8s-cluster",
-      "host.name": "k8s-node",
+    attributes:


As discussed offline I think we can borrow the attribute processor actions implementation and config for this.
BTW, if need specifically a rename we can add it as an action so that we don't have to use upsert+delete pair all the time.

Code was extracted and easy to use.

bogdandrutu · 2020-07-12T00:59:06Z

processor/resourceprocessor/resource_processor.go

-		resource:     resource,
-		capabilities: component.ProcessorCapabilities{MutatesConsumedData: !isEmptyResource(resource)},
+		attrProc:     attrProc,
+		capabilities: component.ProcessorCapabilities{MutatesConsumedData: attrProc.HasActions()},


File an issue to implement same behavior as attributes span processor which fails on empty config. Or you can change this to have the same behavior.

This commit migrates resource processor to internal data model. Existing processor configuration is relevant only to OpenCensus format, so the configuration schema has to be changed. Taking this opportunity, this commit adds existing logic for attributes manipulation from attributes processor to resource processor. New config uses "attributes" field which represents actions that can be made on resource attributes. Supported actions: INSERT, UPDATE, UPSERT, DELETE, HASH, EXTRACT. In order to migrate existing resource processor config: 1. Move key/values from "labels" field to "attributes" with action="upsert". 2. Add value from "type" field to "attributes" with key="opencensus.resourcetype" and action="upsert".

…etry#1315) Bumps [github.com/golangci/golangci-lint](https://github.com/golangci/golangci-lint) from 1.32.1 to 1.32.2. - [Release notes](https://github.com/golangci/golangci-lint/releases) - [Changelog](https://github.com/golangci/golangci-lint/blob/master/CHANGELOG.md) - [Commits](golangci/golangci-lint@v1.32.1...v1.32.2) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

dmitryax requested review from bogdandrutu, flands, james-bebbington, owais, pjanotti, rghetia and tigrannajaryan as code owners July 10, 2020 01:31

dmitryax force-pushed the migrate-resource-processor branch from 5bd7fa7 to b9838d7 Compare July 10, 2020 01:32

dmitryax force-pushed the migrate-resource-processor branch 2 times, most recently from bc59aed to 4963e9e Compare July 10, 2020 04:42

nilebox reviewed Jul 10, 2020

View reviewed changes

dmitryax force-pushed the migrate-resource-processor branch from 4963e9e to a2bcbc2 Compare July 10, 2020 07:56

dmitryax changed the title ~~[BREAKING CHANGE] Migrate Resource Processor to internal data model~~ Migrate Resource Processor to internal data model Jul 10, 2020

dmitryax force-pushed the migrate-resource-processor branch 5 times, most recently from fdb3761 to 9b40e32 Compare July 10, 2020 17:17

tigrannajaryan reviewed Jul 10, 2020

View reviewed changes

nilebox approved these changes Jul 10, 2020

View reviewed changes

dmitryax force-pushed the migrate-resource-processor branch 7 times, most recently from 20c4538 to f529c41 Compare July 12, 2020 00:31

bogdandrutu approved these changes Jul 12, 2020

View reviewed changes

dmitryax force-pushed the migrate-resource-processor branch from f529c41 to c606a7e Compare July 12, 2020 01:57

dmitryax force-pushed the migrate-resource-processor branch from c606a7e to b489311 Compare July 12, 2020 02:08

bogdandrutu merged commit 3711c01 into open-telemetry:master Jul 12, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate Resource Processor to internal data model #1315

Migrate Resource Processor to internal data model #1315

dmitryax commented Jul 10, 2020 •

edited

Loading

codecov bot commented Jul 10, 2020 •

edited

Loading

nilebox Jul 10, 2020 •

edited

Loading

dmitryax Jul 10, 2020

dmitryax Jul 10, 2020

nilebox Jul 10, 2020

dmitryax Jul 10, 2020

tigrannajaryan Jul 10, 2020

bogdandrutu Jul 11, 2020

bogdandrutu Jul 12, 2020

Migrate Resource Processor to internal data model #1315

Migrate Resource Processor to internal data model #1315

Conversation

dmitryax commented Jul 10, 2020 • edited Loading

codecov bot commented Jul 10, 2020 • edited Loading

Codecov Report

nilebox Jul 10, 2020 • edited Loading

Choose a reason for hiding this comment

dmitryax Jul 10, 2020

Choose a reason for hiding this comment

dmitryax Jul 10, 2020

Choose a reason for hiding this comment

nilebox Jul 10, 2020

Choose a reason for hiding this comment

dmitryax Jul 10, 2020

Choose a reason for hiding this comment

tigrannajaryan Jul 10, 2020

Choose a reason for hiding this comment

bogdandrutu Jul 11, 2020

Choose a reason for hiding this comment

bogdandrutu Jul 12, 2020

Choose a reason for hiding this comment

dmitryax commented Jul 10, 2020 •

edited

Loading

codecov bot commented Jul 10, 2020 •

edited

Loading

nilebox Jul 10, 2020 •

edited

Loading