Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

exporter/stackdriver: allow configuration for buffer, delay, etc. #3

Closed
flands opened this issue Jul 3, 2019 · 3 comments
Closed
Labels
help wanted Extra attention is needed

Comments

@flands
Copy link
Contributor

flands commented Jul 3, 2019

See: census-instrumentation/opencensus-service#474

@pjanotti pjanotti transferred this issue from open-telemetry/opentelemetry-collector Jul 11, 2019
@songy23
Copy link
Member

songy23 commented Jul 16, 2019

Also we need to allow default metric labels/trace attributes to be configured: https://github.com/census-ecosystem/opencensus-go-exporter-stackdriver/blob/09504ed717c748c8fd802ba78924d6ca322c27d8/stackdriver.go#L208-L228, so that the default "task" label can be overridden.

/cc @dinooliva @rghetia

@tigrannajaryan tigrannajaryan added the help wanted Extra attention is needed label Apr 22, 2020
@flands
Copy link
Contributor Author

flands commented May 21, 2020

@mtwo is this still applicable? can it be closed?

@mtwo
Copy link
Member

mtwo commented May 21, 2020

Closing

@mtwo mtwo closed this as completed May 21, 2020
bogdandrutu pushed a commit that referenced this issue Sep 30, 2020
* Add DataDog exporter back from old fork
commit 99129fb96e29e9c1a92da00b7e3f8efcae8a31e8
Author: Pablo Baeyens <[email protected]>
Date:   Thu Sep 3 18:10:28 2020 +0200

    Handle namespace at initialization time

commit babca25927926a60c0c416294af3aadf784d41b9
Author: Pablo Baeyens <[email protected]>
Date:   Thu Sep 3 17:23:53 2020 +0200

    Initialize on a separate function
    This way the variables can be checked without worrying about the env

commit 24d0cb4cc566fa5313a8650c904a27bea68bf555
Author: Pablo Baeyens <[email protected]>
Date:   Thu Sep 3 14:30:35 2020 +0200

    Check environment variables for unified service tagging

commit 6695f8297ab8b1fcae71b05acb027c4a0992e3a0
Author: Pablo Baeyens <[email protected]>
Date:   Wed Sep 2 14:57:37 2020 +0200

    Add support for sending metrics through the API
    - Use datadog.Metric type for simplicity
    - Get host if unset

commit c366603
Author: Pablo Baeyens <[email protected]>
Date:   Wed Sep 2 09:56:24 2020 +0200

    Disable Queue and Retry settings (#72)

    These are handled by the statsd package.
    OpenTelemetry docs are confusing and the default configuration (disabled)
    is different from the one returned by "GetDefault..." functions

commit a660b56
Author: Pablo Baeyens <[email protected]>
Date:   Tue Sep 1 15:26:14 2020 +0200

    Add support for summary and distribution metric types (#65)

    * Add support for summary metric type

    * Add support for distribution metrics

    * Refactor metrics construction
    - Drop name in Metrics (now they act as Metric values)
    - Refactor constructor so that errors happen at compile-time

    * Report Summary total sum and count values
    Snapshot values are not filled in by OpenTelemetry

    * Report p00 and p100 as `.min` and `.max`
    This is more similar to what we do for our own non-additive type

    * Keep hostname if it has not been overridden

commit c95adc4
Author: Pablo Baeyens <[email protected]>
Date:   Thu Aug 27 13:00:02 2020 +0200

    Update dependencies and `make gofmt`
    The collector was updated to 0.9.0 upstream

commit 20afb0e
Author: Pablo Baeyens <[email protected]>
Date:   Wed Aug 26 18:25:49 2020 +0200

    Refactor configuration (#45)

    * Refactor configuration
    * Implement telemetry and tags configuration handling
    * Update example configuration and README file
    Co-authored-by: Kylian Serrania <[email protected]>

commit fdc98b5
Author: Pablo Baeyens <[email protected]>
Date:   Fri Aug 21 11:09:08 2020 +0200

    Initial DogStatsD implementation (#15)

    Initial metrics exporter through DogStatsD with support for all metric types but summary and distribution

commit e848a60
Author: Pablo Baeyens <[email protected]>
Date:   Fri Aug 21 10:42:45 2020 +0200

    Bump collector version

commit 58be9a4
Author: Pablo Baeyens <[email protected]>
Date:   Thu Aug 6 10:04:32 2020 +0200

    Address linter

commit 695430c
Author: Pablo Baeyens <[email protected]>
Date:   Tue Aug 4 13:28:01 2020 +0200

    Fix field name error

    MetricsEndpoint was renamed to MetricsURL

commit 168b319
Author: Pablo Baeyens <[email protected]>
Date:   Mon Aug 3 11:05:01 2020 +0200

    Create initial outline for Datadog exporter (#1)

    * Add support for basic configuration options
    * Documents configuration options

* go mod tidy

* Address feedback from upstream PR we did not merge (#1)

* Backport changes from upstream PR

Remove `err` from MapMetrics

* Remove usage of pdatautil

* Fix tests

* Use TCPAddr

* Review which functions should be private

* Remove DogStatsD mode (#2)

* Remove DogStatsD mode

* go mod tidy

* Remove mentions to DogStatSD

* Improve test coverage (#3)

* Improve test coverage

Added unit tests for
- API key censoring
- Hostname
- Metrics exporter

Renamed test and implementation files for consistency

* Add one additional test

* Remove client validation (#6)

The zorkian API does not validate the API key unless you also have
an application key, even though the endpoint works without it.

I am removing this validation until this gets fixed on the zorkian library

* Keep only configuration and factory methods

Following the contribution guidelines we need to make a first PR
with this

* Use latest version of collector

* Remove `report_percentiles` option
It is not being used as of now until the OTLP metrics format
stabilizes and we have a Summary type metric again

* Correct configuration

The API key is now a required setting

* Remove test not relevant for this PR

* Remove unnecessary imports after removing test

* Address review comment

* Apply suggestions from code review

Co-authored-by: Tigran Najaryan <[email protected]>

* Separate documentation into two examples
One example with the minimal configuration, for sending to `datadoghq.com`
and a second one for sending to `datadoghq.eu`

Co-authored-by: Tigran Najaryan <[email protected]>
jjackson-s referenced this issue in jjackson-s/opentelemetry-collector-contrib Aug 9, 2021
bogdandrutu pushed a commit that referenced this issue May 12, 2022
* Initial commit of stanza codebase
hex1848 pushed a commit to hex1848/opentelemetry-collector-contrib that referenced this issue Jun 2, 2022
hex1848 pushed a commit to hex1848/opentelemetry-collector-contrib that referenced this issue Jun 2, 2022
* Initial commit

* Add CODEOWNERS file (open-telemetry#2)

* Add CODEOWNERS file

* Update CODEOWNERS

* Moved from github.com/observatorium/opentelemetry-collector-builder (open-telemetry#3)

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* fixed panics (open-telemetry#6)

Signed-off-by: Joe Elliott <[email protected]>

* Replace master with main in CI and mergify files (open-telemetry#8)

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* Bump to OpenTelemetry Collector 0.20.0 (open-telemetry#10)

Closes open-telemetry#9

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* Explicitly enable Go modules in quickstart instructions (open-telemetry#13)

* Update to collector v0.21.0 (open-telemetry#17)

Fixes open-telemetry#16

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* Update to collector v0.22.0 (open-telemetry#19)

* Download go modules before building (open-telemetry#20)

Fixes open-telemetry#14

* Add version command (open-telemetry#25)

Signed-off-by: Ashmita Bohara <[email protected]>

* Pass errors from cobra Execute back to main for correct exit code (open-telemetry#28)

* pass errors from cobra execute back to main

* print the error

* Update to collector v0.23.0 (open-telemetry#27)

* Generate a warning if the builder and collector base version mismatch (open-telemetry#30)

* Generate a warning if the builder and collector base version mismatch

* Show current default version in the warning message

* Update to OpenTelemetry Collector 0.24.0

* Don't use %w formatting with log.Fatal (open-telemetry#35)

* Update to OpenTelemetry Collector 0.25.0 (open-telemetry#36)

Signed-off-by: Serge Catudal <[email protected]>

* Update to 0.26.0 and update BuildInfo (open-telemetry#39)

* Sync build and CI Go versions at latest 1.16 (open-telemetry#34)

* Sync build and CI Go versions at latest 1.16

* Run go mod tidy

* Set go binary to use in the compilation phase in tests

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

Co-authored-by: Juraci Paixão Kröhling <[email protected]>

* Add option to generate go code only (no compile) (open-telemetry#40)

* Issue#24 Add option to generate go code only (no compile)

* Update cmd/root.go logging

Suggested by @jpkkrohling

Co-authored-by: Juraci Paixão Kröhling <[email protected]>

* remove verbose help .. created by corba

* suggestion by jpkrohling to keep generateandcompile

* lint error: remove unused var

* reword cmd option and add back help message for default

Co-authored-by: Juraci Paixão Kröhling <[email protected]>

* Don't reuse exec.Cmd (open-telemetry#42)

* Update to OpenTelemetry Collector 0.27.0 (open-telemetry#43)

* Add CI Badge (open-telemetry#47)

* Update to Collector v0.28.0 (open-telemetry#49)

* Update to Collector v0.28.0

Closes open-telemetry#48

Addresses the breaking API change in
open-telemetry/opentelemetry-collector#3163,
besides the usual version number changes.

Signed-off-by: Fangyi Zhou <[email protected]>

* Use `go mod tidy` instead of `go mod download`

It appears that this magically resolves the go.mod file issue.
https://stackoverflow.com/questions/67203641/missing-go-sum-entry-for-module-providing-package-package-name

Signed-off-by: Fangyi Zhou <[email protected]>

* Account for go mod download in go1.17 not updating go.sum (open-telemetry#50)

* Update to collector v0.29.0 (open-telemetry#54)

* Update replaces.builder.yaml

* Update nocore.builder.yaml

* Update config.go

* Update README.md

* Update main.go

* Update to collector v0.30.0 (open-telemetry#57)

* cmd: fix module flag default value to github.com/open-telemetry (open-telemetry#58)

Signed-off-by: Koichi Shiraishi <[email protected]>

* Update to collector v0.31.0 (open-telemetry#60)

* Update to v0.33.0 (open-telemetry#62)

Signed-off-by: Anthony J Mirabella <[email protected]>

* Add excludes support to generated go.mod (open-telemetry#63)

Signed-off-by: Anthony J Mirabella <[email protected]>

Co-authored-by: Juraci Paixão Kröhling <[email protected]>

* Small cleanup for the builder files (open-telemetry#64)

Signed-off-by: Bogdan Drutu <[email protected]>

* Support building with Go 1.17 (open-telemetry#66)

* Support building with Go 1.17
Fixes open-telemetry#65

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* Update workflows to use Go 1.17

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* Add gosec exceptions for exec.Command

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* Update to OpenTelemetry core 0.34.0 (open-telemetry#68)

Fixes open-telemetry#67

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

* Upgrade to OpenTelemetry Collector 0.35.0 (open-telemetry#70)

Signed-off-by: Fangyi Zhou <[email protected]>

* Upgrade to OpenTelemetry Collector 0.36.0 (open-telemetry#76)

* Generate custom service code for Windows (open-telemetry#75)

* update main to include windows service code

* use main version from tag 0.35.0

* update main function

* align with upstream v0.36.0 tag

* dummy change to trigger build

* Revert "dummy change to trigger build"

This reverts commit 629d499461da2d2c240bf1e495b5fe0558e3547f.

* Remove Core from Module type (open-telemetry#77)

Fixes open-telemetry#15

Signed-off-by: yugo-horie <[email protected]>

* release 0.37.0 (open-telemetry#78)

* release 0.37.0

* update use of NewCommand

* Move builder to subdirectory

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

Co-authored-by: Bogdan Drutu <[email protected]>
Co-authored-by: Bogdan Drutu <[email protected]>
Co-authored-by: Joe Elliott <[email protected]>
Co-authored-by: Eric Yang <[email protected]>
Co-authored-by: Brian Gibbins <[email protected]>
Co-authored-by: Ashmita <[email protected]>
Co-authored-by: Fangyi Zhou <[email protected]>
Co-authored-by: Shaun Creary <[email protected]>
Co-authored-by: Patryk Małek <[email protected]>
Co-authored-by: Serge Catudal <[email protected]>
Co-authored-by: Aaron Stone <[email protected]>
Co-authored-by: Patryk Małek <[email protected]>
Co-authored-by: Aaron Stone <[email protected]>
Co-authored-by: Kelvin Lo <[email protected]>
Co-authored-by: Himanshu <[email protected]>
Co-authored-by: Y.Horie <[email protected]>
Co-authored-by: Koichi Shiraishi <[email protected]>
Co-authored-by: Anthony Mirabella <[email protected]>
Co-authored-by: Cal Loomis <[email protected]>
Co-authored-by: alrex <[email protected]>
sky333999 added a commit to sky333999/opentelemetry-collector-contrib that referenced this issue May 23, 2023
ishleenk17 added a commit to ishleenk17/opentelemetry-collector-contrib that referenced this issue Apr 22, 2024
* Network scrapper

* Add network scrapper  to hostmetrics

* Add system.network.packets

* Add more metrics to network scrapper

* Move attribute addition to the network.go file

* Add code for Network Scraper

* remove disk for now

* comment a particular part

* Have just a single metric

* Uncomment the code

* update process.go

* Improve the code of Network Scraper

* Change in network scraper code

* Update the metric.attributes
TylerHelmuth added a commit that referenced this issue Sep 21, 2024
… Histo --> Histogram (#33824)

## Description

This PR adds a custom metric function to the transformprocessor to
convert exponential histograms to explicit histograms.

Link to tracking issue: Resolves #33827

**Function Name**
```
convert_exponential_histogram_to_explicit_histogram
```

**Arguments:**

- `distribution` (_upper, midpoint, uniform, random_)
- `ExplicitBoundaries: []float64`

**Usage example:**

```yaml
processors:
  transform:
    error_mode: propagate
    metric_statements:
    - context: metric
      statements:
        - convert_exponential_histogram_to_explicit_histogram("random", [10.0, 20.0, 30.0, 40.0, 50.0, 60.0, 70.0, 80.0, 90.0, 100.0]) 
```

**Converts:**

```
Resource SchemaURL: 
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope  
Metric #0
Descriptor:
     -> Name: response_time
     -> Description: 
     -> Unit: 
     -> DataType: ExponentialHistogram
     -> AggregationTemporality: Delta
ExponentialHistogramDataPoints #0
Data point attributes:
     -> metric_type: Str(timing)
StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2024-07-31 09:35:25.212037 +0000 UTC
Count: 44
Sum: 999.000000
Min: 40.000000
Max: 245.000000
Bucket (32.000000, 64.000000], Count: 10
Bucket (64.000000, 128.000000], Count: 22
Bucket (128.000000, 256.000000], Count: 12
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
```

**To:**

```
Resource SchemaURL: 
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope  
Metric #0
Descriptor:
     -> Name: response_time
     -> Description: 
     -> Unit: 
     -> DataType: Histogram
     -> AggregationTemporality: Delta
HistogramDataPoints #0
Data point attributes:
     -> metric_type: Str(timing)
StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2024-07-30 21:37:07.830902 +0000 UTC
Count: 44
Sum: 999.000000
Min: 40.000000
Max: 245.000000
ExplicitBounds #0: 10.000000
ExplicitBounds #1: 20.000000
ExplicitBounds #2: 30.000000
ExplicitBounds #3: 40.000000
ExplicitBounds #4: 50.000000
ExplicitBounds #5: 60.000000
ExplicitBounds #6: 70.000000
ExplicitBounds #7: 80.000000
ExplicitBounds #8: 90.000000
ExplicitBounds #9: 100.000000
Buckets #0, Count: 0
Buckets #1, Count: 0
Buckets #2, Count: 0
Buckets #3, Count: 2
Buckets #4, Count: 5
Buckets #5, Count: 0
Buckets #6, Count: 3
Buckets #7, Count: 7
Buckets #8, Count: 2
Buckets #9, Count: 4
Buckets #10, Count: 21
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
```

### Testing

- Several unit tests have been created. We have also tested by ingesting
and converting exponential histograms from the `statsdreceiver` as well
as directly via the `otlpreceiver` over grpc over several hours with a
large amount of data.

- We have clients that have been running this solution in production for
a number of weeks.

### Readme description:

### convert_exponential_hist_to_explicit_hist

`convert_exponential_hist_to_explicit_hist([ExplicitBounds])`

the `convert_exponential_hist_to_explicit_hist` function converts an
ExponentialHistogram to an Explicit (_normal_) Histogram.

`ExplicitBounds` is represents the list of bucket boundaries for the new
histogram. This argument is __required__ and __cannot be empty__.

__WARNING:__

The process of converting an ExponentialHistogram to an Explicit
Histogram is not perfect and may result in a loss of precision. It is
important to define an appropriate set of bucket boundaries to minimize
this loss. For example, selecting Boundaries that are too high or too
low may result histogram buckets that are too wide or too narrow,
respectively.

---------

Co-authored-by: Kent Quirk <[email protected]>
Co-authored-by: Tyler Helmuth <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

4 participants