Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New label processor #20

Closed
wants to merge 9 commits into from
Closed

New label processor #20

wants to merge 9 commits into from

Conversation

amanbrar1999
Copy link

@amanbrar1999 amanbrar1999 commented Oct 14, 2020

Description:

Implementation of the Labels Processor. This processor takes key/value string pairs specified in the collector configuration file and adds them as data point labels to all metrics that pass through it.

Link to tracking Issue:

open-telemetry#1892

Testing:

Unit tests are included and manual testing was done with logging exporter to verify data point labels were added

Documentation:

README contains some basic information, with some more example configurations in testdata/config.yaml


Side Notes (not going to be included upstream):

When Merging into upstream this will likely be split into 3 or 4 PRs

  1. Overall code structure without much logic: Factory, README, config, examples
  2. Core functionality of the processor
  3. Integrating the processor to the collector
  4. tests (might be done within the others though)

Copy link

@JasonXZLiu JasonXZLiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Good job :DD Just a few nitpicky comments. Feel free to just ignore them if you think they're wrong.

processor/labelsprocessor/README.md Show resolved Hide resolved
processor/labelsprocessor/labels_processor.go Outdated Show resolved Hide resolved
},
}

metricAllDataTypes := []*otlpmetrics.ResourceMetrics{

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed offline, let's move this gigantic test data to other files.


// This processor will always by default update existing label values. Also assumes duplicate labels do not already exist in the metric
func deDuplicateAndAppend(labels *[]*v11.StringKeyValue, key string, value string) {
// If the key already exists, overwrite it

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Usually comment like this is not useful as we can see what you are doing from the code quite easily. Also comment like this may start to be a lie as time goes; when other started to modify code but not the comment.

If you really think that comment like this is needed, then consider restructuring your code like the following to use method name as comment (but I don't think it's needed in your case, so following is just example).

if label, ok := findLabelByName(labels, key); ok {
   overrideExistingLabel(labels, value);
} else {
  appendNewLabel(labels, key, valye)
}

This applies to the // If it does not exist, append it comment too.

Copy link
Author

@amanbrar1999 amanbrar1999 Oct 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me, comments will be removed

}
}

// This processor will always by default update existing label values. Also assumes duplicate labels do not already exist in the metric
Copy link

@alvinlin123 alvinlin123 Oct 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: while this comment is good, it explains what the function is doing not "how", but consider if you can use better name for the function such that you don't need to have a comment for it.

DeDuplicateAndAppend is a bit misleading here because this function is not really doing "deduplicate"; it is either overwriting or append. The code is actually overwriting the label value regardless if the label value is "duplicated" or not. So maybe a term like "Upsert" or "UpdateOrInsert" would be a better term to use.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now that I think about it, I agree the function name is not very representative of what it is actually doing, I will change it to "upsert" as that is a term used in other parts of the collector as well

}

func handleDoubleHistogramDataPoints(doubleHistogramDataPoints []*v1.DoubleHistogramDataPoint, lp *labelMetricProcessor) {
for _, label := range lp.cfg.Labels {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can save some repetition if you have this outer loop build into deDuplicateAndAppend function. So like:

for _, dataPoint := range doubleHistogramDataPoints {
	deDuplicateAndAppend(&dataPoint.Labels, lp.cfg.Labels)
}

So basically have the deDuplicateAndAppend to deal with looping through configured labels. If you do this, the above 3 functions would have less duplicate code.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great idea, will do

Aman Brar added 2 commits October 15, 2020 17:48
…licateAndAppend to upsertLabel, removed unnecessary comments, and moved test data to global vars in external files
JamesJHPark pushed a commit that referenced this pull request Nov 3, 2021
JamesJHPark pushed a commit that referenced this pull request Nov 3, 2021
* Initial commit

* Add CODEOWNERS file (#2)

* Add CODEOWNERS file

* Update CODEOWNERS

* Moved from github.com/observatorium/opentelemetry-collector-builder (#3)

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* fixed panics (#6)

Signed-off-by: Joe Elliott <[email protected]>

* Replace master with main in CI and mergify files (#8)

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* Bump to OpenTelemetry Collector 0.20.0 (#10)

Closes #9

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* Explicitly enable Go modules in quickstart instructions (#13)

* Update to collector v0.21.0 (#17)

Fixes #16

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* Update to collector v0.22.0 (#19)

* Download go modules before building (#20)

Fixes #14

* Add version command (#25)

Signed-off-by: Ashmita Bohara <[email protected]>

* Pass errors from cobra Execute back to main for correct exit code (#28)

* pass errors from cobra execute back to main

* print the error

* Update to collector v0.23.0 (#27)

* Generate a warning if the builder and collector base version mismatch (#30)

* Generate a warning if the builder and collector base version mismatch

* Show current default version in the warning message

* Update to OpenTelemetry Collector 0.24.0

* Don't use %w formatting with log.Fatal (#35)

* Update to OpenTelemetry Collector 0.25.0 (#36)

Signed-off-by: Serge Catudal <[email protected]>

* Update to 0.26.0 and update BuildInfo (#39)

* Sync build and CI Go versions at latest 1.16 (#34)

* Sync build and CI Go versions at latest 1.16

* Run go mod tidy

* Set go binary to use in the compilation phase in tests

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

Co-authored-by: Juraci Paixão Kröhling <[email protected]>

* Add option to generate go code only (no compile) (#40)

* Issue#24 Add option to generate go code only (no compile)

* Update cmd/root.go logging

Suggested by @jpkkrohling

Co-authored-by: Juraci Paixão Kröhling <[email protected]>

* remove verbose help .. created by corba

* suggestion by jpkrohling to keep generateandcompile

* lint error: remove unused var

* reword cmd option and add back help message for default

Co-authored-by: Juraci Paixão Kröhling <[email protected]>

* Don't reuse exec.Cmd (#42)

* Update to OpenTelemetry Collector 0.27.0 (#43)

* Add CI Badge (#47)

* Update to Collector v0.28.0 (#49)

* Update to Collector v0.28.0

Closes #48

Addresses the breaking API change in
open-telemetry#3163,
besides the usual version number changes.

Signed-off-by: Fangyi Zhou <[email protected]>

* Use `go mod tidy` instead of `go mod download`

It appears that this magically resolves the go.mod file issue.
https://stackoverflow.com/questions/67203641/missing-go-sum-entry-for-module-providing-package-package-name

Signed-off-by: Fangyi Zhou <[email protected]>

* Account for go mod download in go1.17 not updating go.sum (#50)

* Update to collector v0.29.0 (#54)

* Update replaces.builder.yaml

* Update nocore.builder.yaml

* Update config.go

* Update README.md

* Update main.go

* Update to collector v0.30.0 (#57)

* cmd: fix module flag default value to github.com/open-telemetry (#58)

Signed-off-by: Koichi Shiraishi <[email protected]>

* Update to collector v0.31.0 (#60)

* Update to v0.33.0 (#62)

Signed-off-by: Anthony J Mirabella <[email protected]>

* Add excludes support to generated go.mod (#63)

Signed-off-by: Anthony J Mirabella <[email protected]>

Co-authored-by: Juraci Paixão Kröhling <[email protected]>

* Small cleanup for the builder files (#64)

Signed-off-by: Bogdan Drutu <[email protected]>

* Support building with Go 1.17 (#66)

* Support building with Go 1.17
Fixes #65

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* Update workflows to use Go 1.17

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* Add gosec exceptions for exec.Command

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* Update to OpenTelemetry core 0.34.0 (#68)

Fixes #67

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* Upgrade to OpenTelemetry Collector 0.35.0 (#70)

Signed-off-by: Fangyi Zhou <[email protected]>

* Upgrade to OpenTelemetry Collector 0.36.0 (#76)

* Generate custom service code for Windows (#75)

* update main to include windows service code

* use main version from tag 0.35.0

* update main function

* align with upstream v0.36.0 tag

* dummy change to trigger build

* Revert "dummy change to trigger build"

This reverts commit 629d499461da2d2c240bf1e495b5fe0558e3547f.

* Remove Core from Module type (#77)

Fixes #15

Signed-off-by: yugo-horie <[email protected]>

* release 0.37.0 (#78)

* release 0.37.0

* update use of NewCommand

* Move builder to subdirectory

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

Co-authored-by: Bogdan Drutu <[email protected]>
Co-authored-by: Bogdan Drutu <[email protected]>
Co-authored-by: Joe Elliott <[email protected]>
Co-authored-by: Eric Yang <[email protected]>
Co-authored-by: Brian Gibbins <[email protected]>
Co-authored-by: Ashmita <[email protected]>
Co-authored-by: Fangyi Zhou <[email protected]>
Co-authored-by: Shaun Creary <[email protected]>
Co-authored-by: Patryk Małek <[email protected]>
Co-authored-by: Serge Catudal <[email protected]>
Co-authored-by: Aaron Stone <[email protected]>
Co-authored-by: Patryk Małek <[email protected]>
Co-authored-by: Aaron Stone <[email protected]>
Co-authored-by: Kelvin Lo <[email protected]>
Co-authored-by: Himanshu <[email protected]>
Co-authored-by: Y.Horie <[email protected]>
Co-authored-by: Koichi Shiraishi <[email protected]>
Co-authored-by: Anthony Mirabella <[email protected]>
Co-authored-by: Cal Loomis <[email protected]>
Co-authored-by: alrex <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants