Skip to content

Commit

Permalink
Handle IPv6 zone id in IIS filebeat ingest pipeline (elastic#9869)
Browse files Browse the repository at this point in the history
IIS logs can include zone ids when using IPv6, this is correctly parsed
but geoip processor doesn't accept these addresses. Create a temporary
field without the zone id to be used by geoip processor.
  • Loading branch information
jsoriano authored Jan 7, 2019
1 parent 05bd95f commit d59ae8c
Show file tree
Hide file tree
Showing 4 changed files with 37 additions and 1 deletion.
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
*Filebeat*

- Add `convert_timezone` option to Elasticsearch module to convert dates to UTC. {issue}9756[9756] {pull}9761[9761]
- Support IPv6 addresses with zone id in IIS ingest pipeline. {issue}9836[9836] {pull}9869[9869]

*Heartbeat*

Expand Down
16 changes: 15 additions & 1 deletion filebeat/module/iis/error/ingest/default.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,24 @@
"field": "iis.error.time"
}
}, {
"geoip": {
"grok": {
"field": "iis.error.remote_ip",
"patterns": [
"%{NOZONEIP:iis.error.remote_ip_geoip}"
],
"pattern_definitions": {
"NOZONEIP": "[^%]*"
}
}
}, {
"geoip": {
"field": "iis.error.remote_ip_geoip",
"target_field": "iis.error.geoip"
}
}, {
"remove": {
"field": "iis.error.remote_ip_geoip"
}
}],
"on_failure" : [{
"set" : {
Expand Down
5 changes: 5 additions & 0 deletions filebeat/module/iis/error/test/ipv6_zone_id.log
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#Software: Microsoft HTTP API 2.0
#Version: 1.0
#Date: 2018-12-30 13:48:36
#Fields: date time c-ip c-port s-ip s-port cs-version cs-method cs-uri streamid sc-status s-siteid s-reason s-queuename
2018-12-30 14:22:07 ::1%0 49958 ::1%0 80 - - - - - - Timer_ConnectionIdle -
16 changes: 16 additions & 0 deletions filebeat/module/iis/error/test/ipv6_zone_id.log-expected.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
[
{
"@timestamp": "2018-12-30T14:22:07.000Z",
"ecs.version": "1.0.0-beta2",
"event.dataset": "error",
"event.module": "iis",
"iis.error.queue_name": "-",
"iis.error.reason_phrase": "Timer_ConnectionIdle",
"iis.error.remote_ip": "::1%0",
"iis.error.remote_port": "49958",
"iis.error.server_ip": "::1%0",
"iis.error.server_port": "80",
"input.type": "log",
"log.offset": 195
}
]

0 comments on commit d59ae8c

Please sign in to comment.