ReceivedHeader: incorrectly detects the IP address

[mariuszkrzaczkowski]

IP discovery only works when the address is at the end, not as it is at the beginning
an example for which it works

Received: from web.live.com ([124.236.173.16] helo=mail.live.com)

an example for which it doesn't work

Received: from web.live.com (helo=mail.live.com [124.236.173.16])

the problem is in regex

mail-mime-parser/src/Header/Consumer/Received/DomainConsumer.php

Lines 70 to 82 in 56792b4

	private function matchHostPart($value, &$hostname, &$address) {
	$matches = [];
	$pattern = '~^(?P<name>[a-z0-9\-]+\.[a-z0-9\-\.]+)?\s*(\[(IPv[64])?(?P<addr>[a-f\d\.\:]+)\])?$~i';
	if (preg_match($pattern, $value, $matches)) {
	if (!empty($matches['name'])) {
	$hostname = $matches['name'];
	}
	if (!empty($matches['addr'])) {
	$address = $matches['addr'];
	}
	return true;
	}
	return false;

[mariuszkrzaczkowski]

I proposed a solution to this problem in mariuszkrzaczkowski@882f89b

#153

[zbateson]

Hi @mariuszkrzaczkowski --

Unfortunately with the Received header it's a losing battle trying to parse every possibility. Please read through my thoughts on this issue: #78

The trouble is implementations for this header are all over the place, and besides covering the very basics of what the header is defined as in RFC 5321 section 4.4, I'm not convinced this library should attempt to parse much outside of that definition. In this case, the relevant part is:

From-domain    = "FROM" FWS Extended-Domain
Extended-Domain  = Domain /
                    ( Domain FWS "(" TCP-info ")" ) /
                    ( address-literal FWS "(" TCP-info ")" )

The trouble with your example is the 'helo=' part in the address, which is not valid according to that.

For example this test is testing exactly that, the domain part in front of the address:

mail-mime-parser/tests/MailMimeParser/Header/ReceivedHeaderTest.php

Line 52 in c59b9dc

$header = new ReceivedHeader($this->consumerService, 'Received', 'FROM JonSnow (domain.com [1.2.3.4]) (Crow Crow)');

[mariuszkrzaczkowski]

Do you have any idea to handle it universally? if others are doing it, it may be worth fighting. I can help

[mariuszkrzaczkowski]

we can add experimental or compliance mode or not and then try to handle such different cases in an additional mode

[mariuszkrzaczkowski]

I checked the last few messages and a lot is from IP at the beginning

it's from big postal providers google, outlook

[zbateson]

My feeling on this is that I don't want mail-mime-parser chasing unstandardized formats -- think this or something like finding the 'reply' part of an email. There's just too much variation for a library like this, and it's not what I think it should be.

I'd encourage this to be outside of mail-mime-parser as an add-on for instance or something. Unfortunately at the moment mmp isn't very extensible, but I'm working on fixing that in a 2.0 so something like this could be created on the side and plugged in if someone wanted it... it's very slow going as I'm going through a busy period (fatherhood) and so haven't had much time to dedicate unfortunately.

[mariuszkrzaczkowski]

only how many servers are standards compliant ??? this is meant to be used and must be able to deal with every e-mail present only then the library can be used everywhere, this is my opinion

[zbateson]

The variation on Received headers is to the point that the format is 'unstandardized' imo, not that there are widely-used bad implementations. I'm happy to support mistakes in implementation so long as they're widely used and can't be fixed at the source.

Implementing support for Received (seems to me) to be an exercise in how much to support -- you can always find a new variation and need additional code added for it. I don't think that belongs here.