-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/3484 - set metric name with 'measurement' modifier for grok parser #4433
Merged
Merged
Changes from 54 commits
Commits
Show all changes
56 commits
Select commit
Hold shift + click to select a range
e12eced
input plugin that reads files each interval
maxunt 08a11d7
change config file
maxunt 9c4b522
tweak metric output
maxunt 4e24a1b
add grok as a top level parser
maxunt ec7f131
add more test files
maxunt 504d978
clean up some test cases
maxunt 542c030
knock more errors from test files
maxunt 554b960
add setparser to reader
maxunt 36a23ea
Merge branch 'master' into plugin/reader
maxunt f40371e
add init function to reader
maxunt 9c84595
add grok as a top level parser, still need README
maxunt cc40629
allow for import from plugins/all
maxunt 79d9ea4
add docker-image spin up for reader
maxunt bbd68b3
docker will spin up
maxunt bf7220d
add test file to docker spin up
maxunt a931eb1
update DATA_FORMATS_INPUT.MD to include grok
maxunt e450b26
remove comments
maxunt 001658a
condense telegraf.conf
maxunt 7fa27f4
more condensing
maxunt 1be2a8e
Formatting and revert Makefile
glinton aa750ec
add reader README.md
maxunt 892c95a
update readmes
maxunt 04f09d6
grok parser func unexported
maxunt 8063b38
address some of Daniel's comments
maxunt bfc13a7
incomplete changes to logparser plugin
maxunt 67db143
still unfinished logparser changes
maxunt 8a9da28
logparser is linked to grok parser
maxunt cafa95e
logparser no longer uses seperate grok
maxunt c6087ab
add more unit tests to grok parser
maxunt e4b6f23
fix unit tests for grok parser
maxunt d224673
change logparser unit tests
maxunt f52ceeb
test files added for logparser
maxunt 285cf0b
Merge branch 'master' into plugin/reader
maxunt 0c3ac29
addresses daniel's comments
maxunt 74900ed
change parser config names
maxunt d0f5389
allow for original config and functionality of logparser
maxunt b10f592
unfinished playing w grok parser
maxunt 441bc41
add modifier for setting metric name for grok parser
maxunt d1e0c7c
Merge branch 'master' into feature/3484
maxunt b7ed886
unfinished config changes
maxunt 903a977
additional test cases and README updated
maxunt 0040530
address greg's comments
maxunt 054c20e
fix a unit test
maxunt 0e5e115
whips...
maxunt e3d9ca0
Merge branch 'master' into feature/3484
maxunt 1b8ce4a
addresses comments and merges with master
maxunt 797f9bd
remove reader directory
maxunt 255e596
remove reader from all.go
maxunt 6d49188
Merge branch 'master' into feature/3484 and change readme
maxunt 34075e3
readme changes
maxunt a246c11
breaking stuff
maxunt 4ae64bd
get rid of static measurement names, only dynamic
maxunt c019cfa
no longer accepts no semantic name
maxunt 7e20044
Fix documentation
danielnelson 0be9d85
remove more support for no semantic name
maxunt 5b1bbbd
small fix
maxunt File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
File renamed without changes.
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -38,6 +38,7 @@ var timeLayouts = map[string]string{ | |
} | ||
|
||
const ( | ||
MEASUREMENT = "measurement" | ||
INT = "int" | ||
TAG = "tag" | ||
FLOAT = "float" | ||
|
@@ -214,7 +215,8 @@ func (p *Parser) ParseLine(line string) (telegraf.Metric, error) { | |
|
||
timestamp := time.Now() | ||
for k, v := range values { | ||
if k == "" || v == "" { | ||
if (k == "" || v == "") && p.typeMap[patternName][k] != "measurement" { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. remove |
||
log.Printf("D! skipping key: %v", k) | ||
continue | ||
} | ||
|
||
|
@@ -239,6 +241,8 @@ func (p *Parser) ParseLine(line string) (telegraf.Metric, error) { | |
} | ||
|
||
switch t { | ||
case MEASUREMENT: | ||
p.Measurement = v | ||
case INT: | ||
iv, err := strconv.ParseInt(v, 10, 64) | ||
if err != nil { | ||
|
@@ -350,7 +354,7 @@ func (p *Parser) ParseLine(line string) (telegraf.Metric, error) { | |
} | ||
|
||
if len(fields) == 0 { | ||
return nil, fmt.Errorf("logparser_grok: must have one or more fields") | ||
return nil, fmt.Errorf("grok: must have one or more fields") | ||
} | ||
|
||
return metric.New(p.Measurement, tags, fields, p.tsModder.tsMod(timestamp)) | ||
|
@@ -456,6 +460,12 @@ func (p *Parser) parseTypedCaptures(name, pattern string) (string, error) { | |
} | ||
hasTimestamp = true | ||
} else { | ||
//for handling measurement tag with no name | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. remove |
||
if match[1] == "" && match[2] == "measurement" { | ||
match[1] = "measurement_name" | ||
//add "measurement_name" to pattern so it is valid grok | ||
pattern = strings.Replace(pattern, "::measurement", ":measurement_name:measurement", 1) | ||
} | ||
p.typeMap[patternName][match[1]] = match[2] | ||
} | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
still want this