[receiver/gelfreceiver] Added Support for GELF log receiver. #33858

BharatKJain · 2024-07-02T13:00:47Z

Description:

Added support for GELF based logs - please read this document to understand GELF implementation.

Tremor's implementation for GELF receiver

Otel-collector's GELF implementation document: https://docs.google.com/document/d/1P9JcrXrORqI0wsnRgrcUroluutf3rVrcUCIgLeSpFm0/edit?usp=sharing

Link to tracking Issue: #33861

Testing:

Tested the functionality of the code locally.
Will create the unit-tests for it.

Documentation: Work-in-progress.

linux-foundation-easycla · 2024-07-02T13:00:52Z

The committers listed above are authorized under a signed CLA.

✅ login: BharatKJain / name: Bharat Kumar Jain (00a6122, d187f9d, 670c01a, 6f5b08c, 4701d42)

djaglowski

In order to accept a new component you must formally propose it in an issue and find a sponsor.

BharatKJain · 2024-07-04T12:08:42Z

In order to accept a new component you must formally propose it in an issue and find a sponsor.

@djaglowski I don't have a sponsor, is it possible for you to help with that part? or is there a process to get sponsor, then please share the readme/doc for it?

I know about the list of sponsors mentioned here: Link - but how do I ask someone to become sponsor/reviewer for this PR? I guess it's voluntary so I cannot really ask randomly.

Also there's a Slack thread running for it, we can communicate there as well: thread link

djaglowski · 2024-07-08T13:48:07Z

I don't have capacity to voluntarily sponsor any additional components. Hopefully someone else is interested. Otherwise, you can always host the component elsewhere and others can still build it into their collectors using the Collector Builder.

BharatKJain · 2024-07-09T05:33:00Z

@djaglowski Instead of creating a separate receiver, if I add this as a part of UDP receiver then will that be okay in terms of sponsorship as you're the contributor? GELF will be like a CODEC.

djaglowski · 2024-07-09T13:07:31Z

I don't believe GELF support should be a concern of the UDP receiver.

github-actions · 2024-07-24T05:20:14Z

This PR was marked stale due to lack of activity. It will be closed in 14 days.

hughesjj

I don't see any tests, could you please add some to ensure the receiver works as expected?

hughesjj · 2024-08-14T00:31:39Z

cmd/otelcontribcol/builder-config.yaml

@@ -214,6 +214,7 @@ receivers:
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/windowseventlogreceiver v0.102.0
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/zipkinreceiver v0.102.0
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/zookeeperreceiver v0.102.0
+  - gomod: gelfreceiver latest


Please peg to a specific version (commit hash works in absence of tags) instead of using latest

hughesjj · 2024-08-14T00:32:08Z

cmd/otelcontribcol/go.mod

@@ -141,6 +139,7 @@ require (
 	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/filestatsreceiver v0.102.0
 	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/flinkmetricsreceiver v0.102.0
 	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/fluentforwardreceiver v0.102.0
+	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/gelfreceiver v0.0.0


I believe this should be v0.102.0

yes, and same in the builder yaml. If anything, you should defer adding this receiver to cmd/otelcontribcol to a second PR.

hughesjj · 2024-08-14T00:56:13Z

pkg/stanza/operator/input/gelfinternal/config.go

+	defaultReaders           = 1
+	defaultProcessors        = 1
+	defaultUDPMaxQueueLength = 100
+	defaultListenAddress     = "0.0.0.0:31250"


Please use localhost as the default binding instead of 0.0.0.0

github-actions · 2024-08-28T05:20:40Z

This PR was marked stale due to lack of activity. It will be closed in 14 days.

BharatKJain · 2024-09-12T14:04:25Z

I don't see any tests, could you please add some to ensure the receiver works as expected?

@hughesjj I tried talking to folks in SIG otel-collector meeting, they said that we need sponsor for this PR and that it won't be accepted unless there's someone sponsoring this, so before I spend more effort on it - wanted to know if you would be able to sponsor this PR or get this accepted given that all comments are fixed?

I don't want my efforts to go into vain. 😞

(Also I didn't realize there were more comments, sorry for the late response, missed notifications on it.)

atoulme · 2024-09-17T20:58:50Z

receiver/gelfreceiver/docs.go

@@ -0,0 +1,3 @@
+//go:generate mdatagen metadata.yaml


copyright header is missing

atoulme · 2024-09-17T20:59:05Z

cmd/otelcontribcol/go.mod

-go 1.21.0
-
-toolchain go1.21.11
+go 1.22.0


please revert

atoulme · 2024-09-17T21:00:21Z

pkg/stanza/operator/input/gelfinternal/config.go

+		asyncProcessors: c.AsyncProcessors,
+	}
+
+	// fmt.Println("GELF receiver config validated.")


Suggested change

// fmt.Println("GELF receiver config validated.")

atoulme · 2024-09-17T21:00:43Z

pkg/stanza/operator/input/gelfinternal/config.go

+	}
+
+	if c.ListenAddress == "" {
+		return nil, fmt.Errorf("missing required parameter 'listen_address'")


Suggested change

return nil, fmt.Errorf("missing required parameter 'listen_address'")

return nil, errors.New("missing required parameter 'listen_address'")

atoulme · 2024-09-17T21:01:20Z

pkg/stanza/operator/input/gelfinternal/config.go

+	if c.Protocol != "udp" {
+		return nil, fmt.Errorf("supported protocols - udp, invalid protocol: %s", c.Protocol)
+	}
+	if c.AsyncReaders <= 0 && c.AsyncReaders < 20 {


are you sure of these conditions?

atoulme · 2024-09-17T21:02:00Z

pkg/stanza/operator/input/gelfinternal/config.go

+	if c.AsyncProcessors <= 0 && c.AsyncProcessors < 20 {
+		return nil, fmt.Errorf("invalid async_processors: %d", c.AsyncProcessors)
+	}
+	if c.UDPMaxQueueLength <= 0 && c.UDPMaxQueueLength < 65535 {


Suggested change

if c.UDPMaxQueueLength <= 0 && c.UDPMaxQueueLength < 65535 {

if c.UDPMaxQueueLength <= 0 || c.UDPMaxQueueLength > 65535 {

github-actions · 2024-10-03T05:20:57Z

This PR was marked stale due to lack of activity. It will be closed in 14 days.

BharatKJain added 2 commits June 14, 2024 14:12

Added: gelf-receiver init code

00a6122

Updated: GELF receiver code

4701d42

BharatKJain requested a review from djaglowski as a code owner July 2, 2024 13:00

BharatKJain requested a review from a team July 2, 2024 13:00

github-actions bot assigned TylerHelmuth Jul 2, 2024

github-actions bot added cmd/otelcontribcol otelcontribcol command pkg/stanza labels Jul 2, 2024

This was referenced Jul 2, 2024

[receiver/gelfreceiver] Add support for GELF receiver for logging open-telemetry/opentelemetry-collector#10506

Closed

Add support for GELF receiver for logging #33859

Closed

BharatKJain changed the title ~~Added Support for GELF log receiver.~~ [receiver/gelfreceiver] Added Support for GELF log receiver. Jul 2, 2024

djaglowski requested changes Jul 2, 2024

View reviewed changes

BharatKJain mentioned this pull request Jul 2, 2024

New component: [receiver/gelfreceiver] #33861

Open

3 tasks

BharatKJain requested a review from djaglowski July 2, 2024 13:25

Updated: GELF code with async-handling

d187f9d

BharatKJain force-pushed the gelf-receiver-operator branch from 0db0868 to d187f9d Compare July 2, 2024 13:28

Fixed: go mod changes

6f5b08c

github-actions bot added the Stale label Jul 24, 2024

removed comments

670c01a

github-actions bot removed the Stale label Aug 6, 2024

hughesjj suggested changes Aug 14, 2024

View reviewed changes

github-actions bot added the Stale label Aug 28, 2024

atoulme added the Sponsor Needed New component seeking sponsor label Sep 5, 2024

github-actions bot removed the Stale label Sep 5, 2024

atoulme reviewed Sep 17, 2024

View reviewed changes

receiver/gelfreceiver/docs.go

@@ -0,0 +1,3 @@

//go:generate mdatagen metadata.yaml

Copy link

Contributor

atoulme Sep 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copyright header is missing

atoulme reviewed Sep 17, 2024

View reviewed changes

cmd/otelcontribcol/go.mod

go 1.21.0

toolchain go1.21.11

go 1.22.0

Copy link

Contributor

atoulme Sep 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please revert

atoulme reviewed Sep 17, 2024

View reviewed changes

github-actions bot added the Stale label Oct 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[receiver/gelfreceiver] Added Support for GELF log receiver. #33858

[receiver/gelfreceiver] Added Support for GELF log receiver. #33858

BharatKJain commented Jul 2, 2024 •

edited

Loading

linux-foundation-easycla bot commented Jul 2, 2024 •

edited

Loading

djaglowski left a comment

BharatKJain commented Jul 4, 2024 •

edited

Loading

djaglowski commented Jul 8, 2024

BharatKJain commented Jul 9, 2024 •

edited

Loading

djaglowski commented Jul 9, 2024

github-actions bot commented Jul 24, 2024

hughesjj left a comment

hughesjj Aug 14, 2024

hughesjj Aug 14, 2024

atoulme Sep 17, 2024

hughesjj Aug 14, 2024

github-actions bot commented Aug 28, 2024

BharatKJain commented Sep 12, 2024 •

edited

Loading

atoulme Sep 17, 2024

atoulme Sep 17, 2024

atoulme Sep 17, 2024

atoulme Sep 17, 2024

atoulme Sep 17, 2024

atoulme Sep 17, 2024

github-actions bot commented Oct 3, 2024

	return nil, fmt.Errorf("missing required parameter 'listen_address'")
	return nil, errors.New("missing required parameter 'listen_address'")

	if c.UDPMaxQueueLength <= 0 && c.UDPMaxQueueLength < 65535 {
	if c.UDPMaxQueueLength <= 0 \|\| c.UDPMaxQueueLength > 65535 {

[receiver/gelfreceiver] Added Support for GELF log receiver. #33858

Are you sure you want to change the base?

[receiver/gelfreceiver] Added Support for GELF log receiver. #33858

Conversation

BharatKJain commented Jul 2, 2024 • edited Loading

linux-foundation-easycla bot commented Jul 2, 2024 • edited Loading

djaglowski left a comment

Choose a reason for hiding this comment

BharatKJain commented Jul 4, 2024 • edited Loading

djaglowski commented Jul 8, 2024

BharatKJain commented Jul 9, 2024 • edited Loading

djaglowski commented Jul 9, 2024

github-actions bot commented Jul 24, 2024

hughesjj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Aug 28, 2024

BharatKJain commented Sep 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Oct 3, 2024

BharatKJain commented Jul 2, 2024 •

edited

Loading

linux-foundation-easycla bot commented Jul 2, 2024 •

edited

Loading

BharatKJain commented Jul 4, 2024 •

edited

Loading

BharatKJain commented Jul 9, 2024 •

edited

Loading

BharatKJain commented Sep 12, 2024 •

edited

Loading