Email sending fails or succeeds when using HttpProxyClient.

[ytmori-ne]

When using HttpProxyClient to send email, sometimes it succeeds, sometimes it fails.
If it fails, it will result in a timeout error.

1. Establish TCP connections with TCP 3-way handshake
2. Establishing an SMTP Connection
3. Exchange server sends SMTP response code 220 to client
4. Client sends helo command to Exchange server
5. Exchange server sends SMTP response code 250 to client

It appears that the program is not sending the helo command in 4. above.
I asked Microsoft to investigate the logs on the OS side,
Although the OS was passing data to the application with SMTP response code 220,
After that, the OS did not receive a request from the application to send the helo command.

Confirmed in the verification environment,
Only when HttpProxyClient is enabled, email sending may succeed or fail.

It seems to be a bug in Mailkit's HttpProxyClient, can you confirm?

Platform:

• OS: WindowsServer 2019 Datacenter
• .NET Framework: .NET Framework 4.8
• MailKit Version: 4.1.0

Exception

4.7.0 Timeout waiting for client input [TYCP286CA0162.JPNP286.PROD.OUTLOOK.COM 2023-07-05T01:20:59.890Z 08DB7BB5C763495B]

To Reproduce
Execute the following program that was built and made into exe

Code Snippets
Partial code excerpt

System.Net.ServicePointManager.SecurityProtocol |= System.Net.SecurityProtocolType.Tls12;

string path = System.IO.Path.GetDirectoryName(
	System.Reflection.Assembly.GetExecutingAssembly().Location
);
DateTime now = DateTime.Now;
string formattedDate = now.ToString("yyyyMMddHHmmss");
Guid uid = Guid.NewGuid();
var protocolLogger = new ProtocolLogger(System.IO.Path.Combine("D:\\work\\MAIL\\log", formattedDate + "_" + uid + "_smtp.log"), true);

MailKit.Net.Smtp.SmtpClient client = new SmtpClient(protocolLogger);
client.Timeout = (int)TimeSpan.FromMinutes(10).TotalMilliseconds;
if (ms.UseProxy == "1")
{
	client.ProxyClient = new HttpProxyClient(ms.ProxyServer, Convert.ToInt32(ms.ProxyPort));
}
client.Connect(ms.SmtpServer, Convert.ToInt32(ms.SmtpPort), SecureSocketOptions.StartTls);
client.Authenticate(ms.SmtpUser, ms.SmtpPassword);

var message = new MimeMessage();
var builder = new BodyBuilder();
InternetAddress.TryParse(args[3].ToString(), out var _from);
message.From.Add(_from);
InternetAddress.TryParse(args[4].ToString(), out var _to);
message.To.Add(_to);
InternetAddress.TryParse(args[5].ToString(), out var _replyTo);
message.ReplyTo.Add(_replyTo);
message.Subject = args[6].Replace("@char34@", "\"");
builder.TextBody = args[7].Replace("@br@", "\r\n").Replace("@char34@", "\"");
message.Body = builder.ToMessageBody();
client.Send(message);
client.Disconnect(true);

Protocol Logs
Partially masked with [*]

Log when successful

Connected to smtp://smtp.office365.com:587/?starttls=always
S: 220 TY2PR04CA0016.outlook.office365.com Microsoft ESMTP MAIL Service ready at Wed, 21 Jun 2023 11:24:19 +0000
C: EHLO ********
S: 250-TY2PR04CA0016.outlook.office365.com Hello [*****]
S: 250-SIZE 157286400
S: 250-PIPELINING
S: 250-DSN
S: 250-ENHANCEDSTATUSCODES
S: 250-STARTTLS
S: 250-8BITMIME
S: 250-BINARYMIME
S: 250-CHUNKING
S: 250 SMTPUTF8
C: STARTTLS
S: 220 2.0.0 SMTP server ready
C: EHLO ******
S: 250-TY2PR04CA0016.outlook.office365.com Hello [*****]
S: 250-SIZE 157286400
S: 250-PIPELINING
S: 250-DSN
S: 250-ENHANCEDSTATUSCODES
S: 250-AUTH LOGIN XOAUTH2
S: 250-8BITMIME
S: 250-BINARYMIME
S: 250-CHUNKING
S: 250 SMTPUTF8
C: AUTH LOGIN
S: 334 VXNlcm5hbWU6
C: ********
S: 334 UGFzc3dvcmQ6
C: ********
S: 235 2.7.0 Authentication successful
C: MAIL FROM:<*****@*****> SIZE=973 BODY=8BITMIME
C: RCPT TO:<*****@*****> ORCPT=rfc822;*****@*****
S: 250 2.1.0 Sender OK
S: 250 2.1.5 Recipient OK
C: DATA
S: 354 Start mail input; end with <CRLF>.<CRLF>
C: From: ***** <*****@*****>
C: Date: Wed, 21 Jun 2023 20:24:18 +0900
C: Subject: =?utf-8?b?5L6d6aC877yI5Y+X55m65rOo77yaUTIwMTE1LTAw77yJ?=
C: Message-Id: <*****@*****>
C: To: ***** <*****@*****>
C: Reply-To: *****@*****
C: MIME-Version: 1.0
C: Content-Type: text/plain; charset=utf-8
C: Content-Transfer-Encoding: 8bit
C: 
C: testmail
C: 
C: .
S: 250 2.0.0 OK <*****@*****> [Hostname=*****.prod.outlook.com]
C: QUIT
S: 221 2.0.0 Service closing transmission channel

Log of failures

Connected to smtp://smtp.office365.com:587/?starttls=always
S: 451 4.7.0 Timeout waiting for client input [*****.prod.outlook.com 2023-06-22T09:30:50.702Z 08DB727BC6DDF243]

[jstedfast]

In the failure log, it looks like MailKit is still waiting for the 220 greeting from the SMTP server.

It's possible that the HttpProxyClient is eating the 220 greeting...

[jstedfast]

The above commit probably fixes the issue, but if you could test the packages that will be published to https://www.myget.org/feed/mimekit/package/nuget/MailKit and verify that the fix works, that would be helpful.

I think the version will be 4.1.0.698 (it should be the second build of the day). It will take about 30-45 minutes to be uploaded.

[ytmori-ne]

Thanks for the quick response. I am amazed at how quickly the problem was addressed.
We have just deployed a new version of Mailkit.dll and are monitoring its performance.

If no errors occur after a period of testing, we would like to close this issue.
Please wait a while.

[ytmori-ne]

I still fail to send email on rare occasions.
There is a slight change in the protocol log on failure. Is it possible to identify the cause?

Log of failure

Connected to smtp://smtp.office365.com:587/?starttls=always
S: 220 TYCP286CA0192.outlook.office365.com Microsoft ESMTP MAIL Service ready at Wed, 12 Jul 2023 08:08:39 +0000
C: EHLO *****
S: 250-TYCP286CA0192.outlook.office365.com Hello [*****]
S: 250-SIZE 157286400
S: 250-PIPELINING
S: 250-DSN
S: 250-ENHANCEDSTATUSCODES
S: 250-STARTTLS
S: 250-8BITMIME
S: 250-BINARYMIME
S: 250-CHUNKING
S: 250 SMTPUTF8
C: STARTTLS
S: 220 2.0.0 SMTP server ready
C: EHLO *****
S: 250-TYCP286CA0192.outlook.office365.com Hello [*****]
S: 250-SIZE 157286400
S: 250-PIPELINING
S: 250-DSN
S: 250-ENHANCEDSTATUSCODES
S: 250-AUTH LOGIN XOAUTH2
S: 250-8BITMIME
S: 250-BINARYMIME
S: 250-CHUNKING
S: 250 SMTPUTF8
C: AUTH LOGIN
S: 334 VXNlcm5hbWU6
C: ********
S: 334 UGFzc3dvcmQ6
C: ********

[jstedfast]

Looks like in your new case, it's not proxy-related. It looks like MailKit is waiting for a response from the server after it sent the password.

[jstedfast]

I'm going to close this particular issue now because I'm pretty sure the HttpProxy issue is fixed.