Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error using IPv6 address with ingest geoip module #9836

Closed
rgl opened this issue Dec 30, 2018 · 4 comments
Closed

Error using IPv6 address with ingest geoip module #9836

rgl opened this issue Dec 30, 2018 · 4 comments
Labels
bug Filebeat Filebeat module Stalled Team:Integrations Label for the Integrations team

Comments

@rgl
Copy link

rgl commented Dec 30, 2018 •
edited
Loading

The IIS error log (e.g. C:\Windows\System32\LogFiles\HTTPERR\httperr1.log) has this line:

#Software: Microsoft HTTP API 2.0
#Version: 1.0
#Date: 2018-12-30 13:48:36
#Fields: date time c-ip c-port s-ip s-port cs-version cs-method cs-uri streamid sc-status s-siteid s-reason s-queuename
2018-12-30 14:22:07 ::1%0 49958 ::1%0 80 - - - - - - Timer_ConnectionIdle -

But the error.message atribute on the indexed document has '::1%0' is not an IP string literal..

It seems something needs to be changed to handle that IP format?

BTW, the %0 is the IPv6 zone id or index.

I'm using:

  • Filebeat Version: 6.5.4
  • Operating System: Windows 2019
  • IIS: 10.0
@rgl rgl changed the title Error parsing IIS error log Error parsing IPv6 address Dec 31, 2018
@jsoriano
Copy link
Member

jsoriano commented Jan 3, 2019

I think the address is correctly being parsed, but then the geoip plugin complains about this, removing the geoip filter from the pipeline removes this error.

@jsoriano jsoriano changed the title Error parsing IPv6 address Error using IPv6 address with ingest geoip module Jan 3, 2019
@jsoriano jsoriano mentioned this issue Jan 3, 2019
Closed
@jsoriano
Copy link
Member

jsoriano commented Jan 3, 2019

I have opened elastic/elasticsearch#37107 to discuss if this should be supported by the geoip processor. In the meantime we could try to workaround the issue by using a temporary field for geoip without the zone id.

@jsoriano jsoriano self-assigned this Jan 3, 2019
@jsoriano jsoriano mentioned this issue Jan 3, 2019
Merged
@ruflin ruflin added the Team:Integrations Label for the Integrations team label Jan 7, 2019
This was referenced Jan 7, 2019
Merged
Merged
webmat added a commit that referenced this issue Jan 11, 2019
@webmat
Convert Filebeat iis.error to ECS and add IPv6 zone support for IIS a…
cf7b152 
…ccess logs (#9955)

- Introduce IPv6 zone workaround iis.access log as well, resolving #9836.
- Update the IPv6 zone fix (#9869) for iis.error to use the ECS `.address` field instead of a transient field.
- Convert many fields under `iis.error.*` to ECS. Previous field names are field aliases towards the new corresponding ECS field:
  - iis.error.remote_ip => source.address
  - iis.error.remote_port => source.port
  - iis.error.server_ip => destination.address
  - iis.error.server_port => destination.port
  - iis.error.http_version => http.version
  - iis.error.method => http.request.method
  - iis.error.url => url.original
  - iis.error.response_code => http.response.status_code
  - iis.error.geoip.* => source.geo.*
  - read_timestamp => event.created (not aliased, still used elsewhere)
- Update field aliases introduced in #9084 to point to `.address` instead of `.ip`, since this value can be ambiguous. The IP field is populated with the cleaned up IP without the zone. This is also true for the `.ip` fields under populated by the error logs.
  - iis.access.remote_ip => source.address
  - iis.access.server_ip => destination.address
- Coerce to long: source.port, destination.port and http.response.status_code in the iis.error fileset
This was referenced Jan 11, 2019
Merged
Merged
@jsoriano jsoriano mentioned this issue Jan 14, 2019
Merged
cshtdd added a commit to cshtdd/elasticsearch that referenced this issue Jan 16, 2019
@cshtdd
create a test to reproduce the error from issue elastic#37107
4e71f6f 
and elastic/beats#9836
@alvarolobato alvarolobato self-assigned this Jan 21, 2019
@jsoriano jsoriano removed their assignment Apr 24, 2019
@alvarolobato alvarolobato removed their assignment Jun 3, 2019
@iquirino
Copy link

+1 on latest version (installation this weekend)

@botelastic
Copy link

botelastic bot commented Sep 9, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@botelastic botelastic bot added the Stalled label Sep 9, 2020
@botelastic botelastic bot closed this as completed Oct 9, 2020
@zube zube bot removed the [zube]: Done label Jan 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Filebeat Filebeat module Stalled Team:Integrations Label for the Integrations team
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@jsoriano @rgl @ruflin @andresrc @iquirino @alvarolobato