Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gelf output plugin: improved docs and add missing flags #1803

Merged
merged 4 commits into from
Sep 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2685,6 +2685,8 @@ spec:
type: object
host:
type: string
max_bytes:
type: integer
port:
type: integer
protocol:
Expand Down Expand Up @@ -10084,6 +10086,8 @@ spec:
type: object
host:
type: string
max_bytes:
type: integer
port:
type: integer
protocol:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2681,6 +2681,8 @@ spec:
type: object
host:
type: string
max_bytes:
type: integer
port:
type: integer
protocol:
Expand Down Expand Up @@ -9354,6 +9356,8 @@ spec:
type: object
host:
type: string
max_bytes:
type: integer
port:
type: integer
protocol:
Expand Down
4 changes: 4 additions & 0 deletions config/crd/bases/logging.banzaicloud.io_clusteroutputs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2685,6 +2685,8 @@ spec:
type: object
host:
type: string
max_bytes:
type: integer
port:
type: integer
protocol:
Expand Down Expand Up @@ -10084,6 +10086,8 @@ spec:
type: object
host:
type: string
max_bytes:
type: integer
port:
type: integer
protocol:
Expand Down
4 changes: 4 additions & 0 deletions config/crd/bases/logging.banzaicloud.io_outputs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2681,6 +2681,8 @@ spec:
type: object
host:
type: string
max_bytes:
type: integer
port:
type: integer
protocol:
Expand Down Expand Up @@ -9354,6 +9356,8 @@ spec:
type: object
host:
type: string
max_bytes:
type: integer
port:
type: integer
protocol:
Expand Down
2 changes: 1 addition & 1 deletion docs/configuration/crds/v1beta1/output_types.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ OutputSpec defines the desired state of Output
### gcs (*output.GCSOutput, optional) {#outputspec-gcs}


### gelf (*output.GELFOutputConfig, optional) {#outputspec-gelf}
### gelf (*output.GelfOutputConfig, optional) {#outputspec-gelf}


### http (*output.HTTPOutputConfig, optional) {#outputspec-http}
Expand Down
2 changes: 1 addition & 1 deletion docs/configuration/plugins/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ For more information please click on the plugin name
| **[Format rfc5424](outputs/format_rfc5424/)** | outputs | Specify how to format output record. | GA | [more info](https://github.com/cloudfoundry/fluent-plugin-syslog_rfc5424#format-section) |
| **[Forward](outputs/forward/)** | outputs | Forwards events to other fluentd nodes. | GA | [more info](https://docs.fluentd.org/output/forward) |
| **[Google Cloud Storage](outputs/gcs/)** | outputs | Store logs in Google Cloud Storage | GA | [0.4.0](https://github.com/kube-logging/fluent-plugin-gcs) |
| **[Gelf](outputs/gelf/)** | outputs | Output plugin writes events to GELF | Testing | [1.0.8](https://github.com/hotschedules/fluent-plugin-gelf-hs) |
| **[Gelf](outputs/gelf/)** | outputs | Output plugin writes logs to Graylog | Testing | [1.3.4](https://github.com/bmichalkiewicz/fluent-plugin-gelf-best) |
| **[Http](outputs/http/)** | outputs | Sends logs to HTTP/HTTPS endpoints. | GA | [more info](https://docs.fluentd.org/output/http) |
| **[Kafka](outputs/kafka/)** | outputs | Send your logs to Kafka | GA | [0.17.5](https://github.com/fluent/fluent-plugin-kafka/releases/tag/v0.17.5) |
| **[Amazon Kinesis Firehose](outputs/kinesis_firehose/)** | outputs | Fluent plugin for Amazon Kinesis | Testing | [3.4.2](https://github.com/awslabs/aws-fluent-plugin-kinesis/releases/tag/v3.4.2) |
Expand Down
56 changes: 22 additions & 34 deletions docs/configuration/plugins/outputs/gelf.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,22 @@
---
title: GELF
title: Gelf
weight: 200
generated_file: true
---

# [GELF Output](https://github.com/hotschedules/fluent-plugin-gelf-hs)
# Gelf output plugin for Fluentd
## Overview
Fluentd output plugin for GELF.

For details, see [https://github.com/bmichalkiewicz/fluent-plugin-gelf-best](https://github.com/bmichalkiewicz/fluent-plugin-gelf-best).

## Example
```yaml
spec:
gelf:
host: gelf-host
port: 12201
```


## Configuration
## Output Config
Expand All @@ -21,6 +31,12 @@ Available since ghcr.io/kube-logging/fluentd:v1.16-4.8-full [Buffer](../buffer/)
Destination host


### max_bytes (int, optional) {#output config-max_bytes}

MaxBytes specifies the maximum size, in bytes, of each individual log message. For details, see [https://github.com/Graylog2/graylog2-server/issues/873](https://github.com/Graylog2/graylog2-server/issues/873) Available since ghcr.io/kube-logging/fluentd:v1.16-4.10-full

Default: 3200

### port (int, required) {#output config-port}

Destination host port
Expand All @@ -34,42 +50,14 @@ Default: "udp"

### tls (*bool, optional) {#output config-tls}

Enable TlS
Enable TLS

Default: false

### tls_options (map[string]string, optional) {#output config-tls_options}

TLS options .

Default: {}). For details, see [https://github.com/graylog-labs/gelf-rb/blob/72916932b789f7a6768c3cdd6ab69a3c942dbcef/lib/gelf/transport/tcp_tls.rb#L7-L12](https://github.com/graylog-labs/gelf-rb/blob/72916932b789f7a6768c3cdd6ab69a3c942dbcef/lib/gelf/transport/tcp_tls.rb#L7-L12
TLS Options. For details, see [https://github.com/graylog-labs/gelf-rb/blob/72916932b789f7a6768c3cdd6ab69a3c942dbcef/lib/gelf/transport/tcp_tls.rb#L7-L12](https://github.com/graylog-labs/gelf-rb/blob/72916932b789f7a6768c3cdd6ab69a3c942dbcef/lib/gelf/transport/tcp_tls.rb#L7-L12).

Default: {}



## Example `GELF` output configurations

{{< highlight yaml >}}
apiVersion: logging.banzaicloud.io/v1beta1
kind: Output
metadata:
name: gelf-output-sample
spec:
gelf:
host: gelf-host
port: 12201
{{</ highlight >}}

Fluentd config result:

{{< highlight xml >}}
<match **>
@type gelf
@id test_gelf
host gelf-host
port 12201
</match>
{{</ highlight >}}


---
2 changes: 1 addition & 1 deletion pkg/sdk/logging/api/v1alpha1/output_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ type OutputSpec struct {
AwsElasticsearchOutputConfig *output.AwsElasticsearchOutputConfig `json:"awsElasticsearch,omitempty"`
RedisOutputConfig *output.RedisOutputConfig `json:"redis,omitempty"`
SyslogOutputConfig *output.SyslogOutputConfig `json:"syslog,omitempty"`
GELFOutputConfig *output.GELFOutputConfig `json:"gelf,omitempty"`
GelfOutputConfig *output.GelfOutputConfig `json:"gelf,omitempty"`
SQSOutputConfig *output.SQSOutputConfig `json:"sqs,omitempty"`
RelabelOutputConfig *output.RelabelOutputConfig `json:"relabel,omitempty"`
}
Expand Down
6 changes: 3 additions & 3 deletions pkg/sdk/logging/api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pkg/sdk/logging/api/v1beta1/output_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ type OutputSpec struct {
AwsElasticsearchOutputConfig *output.AwsElasticsearchOutputConfig `json:"awsElasticsearch,omitempty"`
RedisOutputConfig *output.RedisOutputConfig `json:"redis,omitempty"`
SyslogOutputConfig *output.SyslogOutputConfig `json:"syslog,omitempty"`
GELFOutputConfig *output.GELFOutputConfig `json:"gelf,omitempty"`
GelfOutputConfig *output.GelfOutputConfig `json:"gelf,omitempty"`
SQSOutputConfig *output.SQSOutputConfig `json:"sqs,omitempty"`
MattermostOutputConfig *output.MattermostOutputConfig `json:"mattermost,omitempty"`
RelabelOutputConfig *output.RelabelOutputConfig `json:"relabel,omitempty"`
Expand Down
6 changes: 3 additions & 3 deletions pkg/sdk/logging/api/v1beta1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

68 changes: 27 additions & 41 deletions pkg/sdk/logging/model/output/gelf.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,69 +20,55 @@ import (
"github.com/kube-logging/logging-operator/pkg/sdk/logging/model/types"
)

// +name:"GELF"
// +name:"Gelf"
// +weight:"200"
type _hugoGELF interface{} //nolint:deadcode,unused
type _hugoGelf interface{} //nolint:deadcode,unused

// +kubebuilder:object:generate=true
// +docName:"[GELF Output](https://github.com/hotschedules/fluent-plugin-gelf-hs)"
// Fluentd output plugin for GELF.
type _docGELF interface{} //nolint:deadcode,unused
// +docName:"Gelf output plugin for Fluentd"
/*
For details, see [https://github.com/bmichalkiewicz/fluent-plugin-gelf-best](https://github.com/bmichalkiewicz/fluent-plugin-gelf-best).

## Example
```yaml
spec:
gelf:
host: gelf-host
port: 12201
```
*/
type _docGelf interface{} //nolint:deadcode,unused

// +name:"Gelf"
// +url:"https://github.com/hotschedules/fluent-plugin-gelf-hs"
// +version:"1.0.8"
// +description:"Output plugin writes events to GELF"
// +url:"https://github.com/bmichalkiewicz/fluent-plugin-gelf-best"
// +version:"1.3.4"
// +description:"Output plugin writes logs to Graylog"
// +status:"Testing"
type _metaGelf interface{} //nolint:deadcode,unused

// +kubebuilder:object:generate=true
// +docName:"Output Config"
type GELFOutputConfig struct {
type GelfOutputConfig struct {
// Destination host
Host string `json:"host"`
// Destination host port
Port int `json:"port"`
// Transport Protocol (default: "udp")
Protocol string `json:"protocol,omitempty"`
// Enable TlS (default: false)
// Enable TLS (default: false)
TLS *bool `json:"tls,omitempty"`
// TLS options (default: {}). For details, see [https://github.com/graylog-labs/gelf-rb/blob/72916932b789f7a6768c3cdd6ab69a3c942dbcef/lib/gelf/transport/tcp_tls.rb#L7-L12](https://github.com/graylog-labs/gelf-rb/blob/72916932b789f7a6768c3cdd6ab69a3c942dbcef/lib/gelf/transport/tcp_tls.rb#L7-L12).
// TLS Options.
// For details, see [https://github.com/graylog-labs/gelf-rb/blob/72916932b789f7a6768c3cdd6ab69a3c942dbcef/lib/gelf/transport/tcp_tls.rb#L7-L12](https://github.com/graylog-labs/gelf-rb/blob/72916932b789f7a6768c3cdd6ab69a3c942dbcef/lib/gelf/transport/tcp_tls.rb#L7-L12). (default: {})
TLSOptions map[string]string `json:"tls_options,omitempty"`
// MaxBytes specifies the maximum size, in bytes, of each individual log message.
// For details, see [https://github.com/Graylog2/graylog2-server/issues/873](https://github.com/Graylog2/graylog2-server/issues/873)
// Available since ghcr.io/kube-logging/fluentd:v1.16-4.10-full (default: 3200)
MaxBytes int `json:"max_bytes,omitempty"`
// Available since ghcr.io/kube-logging/fluentd:v1.16-4.8-full
// +docLink:"Buffer,../buffer/"
Buffer *Buffer `json:"buffer,omitempty"`
}

//
/*
## Example `GELF` output configurations

{{< highlight yaml >}}
apiVersion: logging.banzaicloud.io/v1beta1
kind: Output
metadata:
name: gelf-output-sample
spec:
gelf:
host: gelf-host
port: 12201
{{</ highlight >}}

Fluentd config result:

{{< highlight xml >}}
<match **>
@type gelf
@id test_gelf
host gelf-host
port 12201
</match>
{{</ highlight >}}
*/
type _expGELF interface{} //nolint:deadcode,unused

func (s *GELFOutputConfig) ToDirective(secretLoader secret.SecretLoader, id string) (types.Directive, error) {
func (s *GelfOutputConfig) ToDirective(secretLoader secret.SecretLoader, id string) (types.Directive, error) {
pluginType := "gelf"
gelf := &types.OutputPlugin{
PluginMeta: types.PluginMeta{
Expand Down
4 changes: 2 additions & 2 deletions pkg/sdk/logging/model/output/gelf_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ import (
"sigs.k8s.io/yaml"
)

func TestGELFOutputConfig(t *testing.T) {
func TestGelfOutputConfig(t *testing.T) {
CONFIG := []byte(`
host: gelf-host
port: 12201
Expand All @@ -43,7 +43,7 @@ port: 12201
</buffer>
</match>
`
s := &output.GELFOutputConfig{}
s := &output.GelfOutputConfig{}
require.NoError(t, yaml.Unmarshal(CONFIG, s))
test := render.NewOutputPluginTest(t, s)
test.DiffResult(expected)
Expand Down
8 changes: 4 additions & 4 deletions pkg/sdk/logging/model/output/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading