Microsoft 365 OAUTH2 authentication workflow no longer works

[lambjs]

Prerequisites

• I have searched for duplicate or closed issues
• I can recreate the issue with all plugins disabled

Describe the issue

I may be doing something wrong here, but my attempts to get IMAP XOAUTH2 authentication working for a tenant in Microsoft 365 have failed on a new install. I'll provide a sample of my configuration below.

<?php
// From html/config/config.inc.php
$config['default_host'] = 'ssl://outlook.office365.com';
$config['smtp_server'] = 'ssl://smtp.office365.com';

$config['oauth_provider'] = 'outlook';
$config['oauth_provider_name'] = 'Outlook.com';
$config['oauth_client_id'] = "-- snip --";
$config['oauth_client_secret'] = "-- snip --";
$config['oauth_auth_uri'] = "https://login.microsoftonline.com/common/oauth2/v2.0/authorize";
$config['oauth_token_uri'] = "https://login.microsoftonline.com/common/oauth2/v2.0/token";
$config['oauth_identity_uri'] = "https://graph.microsoft.com/v1.0/me";
$config['oauth_identity_fields'] = ['email', 'userPrincipalName'];
$config['oauth_scope'] = "https://outlook.office365.com/IMAP.AccessAsUser.All https://outlook.office365.com/SMTP.Send User.Read offline_access";
$config['oauth_auth_parameters'] = ['nonce' => mt_rand()];

I can confirm that I have configured my application in Entra to include the requisite Graph API delegated permissions. I've also configured my scope for Multitenant without personal accounts ie. the Manifest for my application in Entra admin center shows "signInAudience": "AzureADMultipleOrgs". I've seen some other forums mention that this is required and it won't work with other SignInAudience values? Is this true, and if so, should it be added to the example configuration in defaults.inc.php?

When I connect with the above configuration, I get an incorrect audience error when attempting to fetch user details from the identity uri (graph.microsoft.com). When I use the graph namespace instead, I get "NO AUTHENTICATION failed." during IMAP login.

I created my own minimal oauth 365 proof of concept to see if I could rule out roundcube as the problem. Based on my findings, and skipping the identity uri call using the same scope as above, I was able to get imap auth working.

I think the problem may be that you cannot (can no longer? maybe this restriction was added recently?) use the same token on two different 365 resources. If you fetch a token that has scopes which are allowed to call the graph endpoint (prefixed with https://graph.microsoft.com), then the token fails for IMAP/SMTP. If you fetch a token which is allowed to authenticate for IMAP/SMTP (prefixed with https://outlook.office365.com), then the token fails for graph calls.

See this article:
https://stackoverflow.com/questions/48579143/one-or-more-scopes-are-not-compatible-with-each-other-error-when-retrieving-ac/48584417#48584417

The above url sounds like a similar issue, but with OneDrive and the 365 API instead of Graph and 365? It appears that you could make a call to the graph endpoint using graph scopes, then refresh the token with 365 scopes and use it for subsequent imap/smtp authentication. You would need to be able to define in config optional separate scopes for the identity fetch step, and then implement support for switching scopes after identity fetch is completed.

Can someone verify that this is a problem and I haven't missed something fundamental? If so, I think I can roll a pull-request that would fix by making updates to defaults.inc.php, rcube_imap_generic.php and rcmail_oauth.php.

What browser(s) are you seeing the problem on?

Chrome

What version of PHP are you using?

v8.0

What version of Roundcube are you using?

v1.5.8

JavaScript errors

No response

PHP errors

When Graph works:

20-Aug-2024 18:52:28 +0100]: IMAP Error: Login failed for --SNIP-- against outlook.office365.com from --SNIP--. AUTHENTICATE XOAUTH2: A0001 NO AUTHENTICATE failed. in /var/www/program/lib/Roundcube/rcube_imap.php on line 211 (GET /index.php/login/oauth?code=...

When Graph fails:

[20-Aug-2024 22:09:03 +0100]: PHP Error: OAuth token request failed: Client error: GET https://graph.microsoft.com/v1.0/me resulted in a 401 Unauthorized response:
{"error":{"code":"InvalidAuthenticationToken","message":"Access token validation failure. Invalid audience.","innerError (truncated...)
; eb9d0294d4de GuzzleHttp/6.5.5 curl/7.68.0 PHP/8.0.30 - [20/Aug/2024:22:09:03 +0100] "GET /v1.0/me HTTP/1.1" 401 in /var/www/html/program/include/rcmail_oauth.php on line 317 ...

[feroda]

Hello, I think I have the same problem, but with a specific Tenant ID for my organization. I need Microsoft 365 login only in order to check another IMAP server. I think it should be possible.

OS, PHP and Roundcube versions

OS: Ubuntu noble numbat 24.04.1 LTS
Roundcube Ubuntu Package 1.6.6+dfsg-2
PHP 8.3.6 (cli) (built: Jun 13 2024 15:23:20) (NTS)

Here follows my conf

$config['oauth_provider'] = 'outlook';                                                                                                                                                        
$config['oauth_provider_name'] = 'Microsoft 365';                                                                                                                                             
$config['oauth_client_id'] = "--- hidden ---";                                                                                                                          
$config['oauth_client_secret'] = "--- hidden ---";                                                                                                                  
$config['oauth_auth_uri'] = "https://login.microsoftonline.com/<my tenant id from Entra portal>/oauth2/v2.0/authorize";                                                                   
$config['oauth_token_uri'] = "https://login.microsoftonline.com/<my tenant id from Entra portal>/oauth2/v2.0/token";                                                                      
$config['oauth_identity_uri'] = "https://graph.microsoft.com/v1.0/me";                                                                                                                        
$config['oauth_identity_fields'] = ['email', 'userPrincipalName'];                                                                                                                            
$config['oauth_scope'] = "https://outlook.office365.com/IMAP.AccessAsUser.All https://outlook.office365.com/SMTP.Send User.Read offline_access";                                              
//$config['oauth_scope'] = "https://outlook.office365.com/IMAP.AccessAsUser.All";                                                                                                             
$config['oauth_auth_parameters'] = ['nonce' => mt_rand()];                                                                                                                                    
                                                                                                                                                                                              
$config['default_host'] = "localhost";                                                                                                                                                        
$config['smtp_server'] = "localhost"; 

Error log

/var/lib/roundcube/error.log

[17-Sep-2024 15:54:30 +0000]: <2rn7mnha> PHP Error: OAuth token request failed: Client error: `POST https://login.microsoftonline.com/---/oauth2/v2.0/token` resulted in a `400 Bad Request` response:
{"error":"invalid_grant","error_description":"AADSTS54005: OAuth2 Authorization code was already redeemed, please retry  (truncated...)
; mail GuzzleHttp/7 - [17/Sep/2024:15:54:30 +0000] "POST /---/oauth2/v2.0/token HTTP/1.1" 400 472 in /usr/share/roundcube/program/include/rcmail_oauth.php on line 324 (GET /index.php/login/oauth?code=0.AU4ApztMBUOah0yb24tPLi693DrnTb8FrBFNl53z_VNW89EOAX8.AgABBAIAAAApTwJmzXqdR4BN2miheQMYAwDs_wUA9P_pHMutOE_iVlcmZsnLRcss-Qon7qs8qAi8TyhWATmXuK-ST5vD1Yu16u3zttY70URh9cmRoBFQY_4N-BrxcUxO_PX9fQnNX6rqzrfULhFPdPPHTQpkwuAySf6sqFdGJHOpUQ0R2CbxIx1Us0hJZSwByUwJHX3-O7yH_qaX_YjmVSFUuqI5G7orbdq76NdNnUaAWxR8do7tuyBD2F6xl3nP2gj45nTRIjl69-3H1Nd8CF4k5fjYvsXRgGx6hdAiLy4OIbvzwEMxUPG9qk-Xcaqf3SsJ231UJBK0lEkEufzCw6bBcOpQL8P7WePblCZAHFzIOs2Y8VMsZO-7kDMrrP_AcU0320dCbeZ2CZNH08-wX0CrH6KWJkZb0qs1ZMT5vDUL6oAVZqgdygmEK29yxqiQ7nf9N37s-kVWeZmeZDv3LM6IG3uXaxUEcuKaF468PfrnZKKIL5mAoO_XXFjph5_xkyk8zU-NRADWyzeYQ1UH5sfbuKsu__onPwDxuZ6UBaAyOq0cT9ebz-r9zJSbJgDCvVAlUqt7wrRvS5XqZk9J34Qwc2XZ4P6ohCaWzZtsKYZe-SBlQ8xgyQC8J4RXXe0FaNh2_2d_hnYa9xSW2rVkTV-A18ankFfc03f-O7yBGntWWnxAB6RPQVCn8NDJlSg8FBkNT40b8Zb4Sog1DZLahUtv-snRmGmiydzuzhnH_VrNxUqBv9mCUCfYkqHFnELNlFwYE7kGKaZs_TLr7Kl2yRwt-ywBg3-O9FzLL4W7EtdDTpzt0b-FhaMoF_9khJ3yZ_rRWMiBzkGHMa8avigjnmCZD1Tx1mHuEIJFqY8Jc06AFIIO3wzuC6S9Yjd8onRhPOg&state=RnYrjkUHisKP&session_state=b141cbcd-804f-4009-b171-f1f23386fcbb)
[17-Sep-2024 15:54:32 +0000]: <2rn7mnha> PHP Error: OAuth token request failed: Client error: `GET https://graph.microsoft.com/v1.0/me` resulted in a `401 Unauthorized` response:
{"error":{"code":"InvalidAuthenticationToken","message":"Access token validation failure. Invalid audience.","innerError (truncated...)
; mail GuzzleHttp/7 - [17/Sep/2024:15:54:32 +0000] "GET /v1.0/me HTTP/1.1" 401  in /usr/share/roundcube/program/include/rcmail_oauth.php on line 324 (GET /index.php/login/oauth?code=0.AU4ApztMBUOah0yb24tPLi693DrnTb8FrBFNl53z_VNW89EOAX8.AgABBAIAAAApTwJmzXqdR4BN2miheQMYAwDs_wUA9P8m5J4WovINg-gfMRSTZgLvKN7HOtDJOxD2gppwDeEEtRLHQzO2pgz9PI9gtSbkEsW5Stib0-LXyONuAq1LW3D1lx4Z8rRLuIpZPXVT0Y-0ZUEQ-DmMikHQMPLwE4aV2j--YdXRgC__ZsklzSJHX8P3THDL1fs1FembdrBhgGHP6nfW1lFk7n7ohkfqzzpnP9vi5ineCP2Mb1G4YnrMxxpHTml0F4tKJZ3DHqeLzqDcqewzhTajLHb2xWgCIiqw8MSSnVyLXmyyo6UXXRZc2orNiv6IgxJVTiH9MRH2r70LtQWPIhRBC_Eh2SOL_WSl1M5cZfJNv90o0pSKDpm735jceKowhEri0btkcKGW-E5sINMmjiYOcOU6ZJj_e9kyoxgMkfxt2cHsNCoiBoFagkXUvT8F-w54Cw-Ep_1HFqGY3tENxoRdVY8ZNGvWEEbUKTE_3ongCZFo6nZBvnnRYXrUrIN6bxnVEfulP3XiHNP5YVGCvaZbbMzv57cKj8SSqAPdJ6i9WFc60AnUsv8kKr4o4GnbGqS7uGdMfyqLRgIva1TVM14xSDGFau8c3d7BwCW9jZtknkeAh9dDazrdmT_5PW3bt-5-I7MCd7mE3jxYCHLit2c8lyPoY2UVRUU91ZvbQEqmh3nNrWYkzQi1yw8abzNz1omLj18fECVSw1UzRwPAKxMKxsHbw9naLwme0zSiEpal-c632M3lYntOfriAr6H3YqcpwFzNyobsGvwTG20KIOmOaJLplwl2sRo_EQckquzx7gquEt7OCt40KN6YkiNsQSncziwm-0hV7JtY0-RI3JhbTLWnFP4foVIujlxbc8QSxI2mZ_7uRaaxoJaq2Q&state=dC3Dgk8ZalNl&session_state=b141cbcd-804f-4009-b171-f1f23386fcbb)
[17-Sep-2024 15:55:17 +0000]: <cnj94rgr> PHP Error: OAuth token request failed: Client error: `GET https://graph.microsoft.com/v1.0/me` resulted in a `401 Unauthorized` response:
{"error":{"code":"InvalidAuthenticationToken","message":"Access token validation failure. Invalid audience.","innerError (truncated...)
; mail GuzzleHttp/7 - [17/Sep/2024:15:55:17 +0000] "GET /v1.0/me HTTP/1.1" 401  in /usr/share/roundcube/program/include/rcmail_oauth.php on line 324 (GET /index.php/login/oauth?code=0.AU4ApztMBUOah0yb24tPLi693DrnTb8FrBFNl53z_VNW89EOAQA.AgABBAIAAAApTwJmzXqdR4BN2miheQMYAwDs_wUA9P8G79C3EIOFziCj8pZLfQyZ_xvyLo_w216JNvaFl2gaD7W4F0QsJHR8SZM0izexbnBlINuNgFjs-JLQaiyZ00eOfCy27RtWTM_6JVfjqbGDCQTSqSC_1ss9KIo2cEXQMAC46HJizioOHwLJvKO4JLlnWQSGUFkp_Ydv5bYoV1FEmM3kU3S4Yck9iWQpWNHnbrFYL7j8JInL6QQ19n2cEkzHf_oc7Mya0Uk0k3Uh-9FNM6D6SCbUSRKOEop_At9QJqYesSc2BZWjvlehpYKOMUIKVCVpIXhRN6QKA4w6gTJvwvZHG3uDsbE67_xYiVMJ6iQvbBToT5aX3gCigyIjdiJsvE0b1RXjimsmXX3efnVNOk6iJ_mR7EpANS2gY46ZDudYjb4HAnOzmUaNNPwhVeIULPR6GRmjyWccSPNs6nAUCnIY1uIBshDOL17KPsZGpiuZEVRAGvipcjYFif3YGnD0Sp8rNZgiKi2zPmyOFO_3gi7o9uJ1a6Jont03kJ0rDsjpjCbobJtq1f2RQ5cOUFELi4ZYCCJ-UhH6vjRUdY7U10gzAz3-XCk3FCZVH1oA1KJ4mIWGVyJIN6dSEid4dBxnSuOoEwiPO6H0N9hDzE9zEWRXCIdpSVF5qxN6mIzmWNrwRZUYLMWwsbEyAoExTtdLEoSE6OhnD_FI_Ppd_9-8NEuWapHxuXd9HBHsKy67mabRa56-O9D-7BIcoGoFzLv2aiPFZvPizPaXGhiwBf3Mn-XGfb6hA22EgbiqZ2qaFysASB5PsqYISqAJ3pRKju2XsBa0TOKTw9VPSdVDrxOuHCKAbIoE8n179MuobIwkYlsxE1U7sc4EzQwrOQ5tbB8h&state=0qMtlYHAlrRV&session_state=be0bdbee-1064-4ef9-9149-85f88ae2e4de)

[lambjs]

@feroda You've included logs from two requests above. For your request 2rn7mnha, I'm guessing you tried to reload the page you were redirected to after authenticating in Microsoft's oauth login form - it would have sent a authorization grant token back to your roundcube instance that roundcube tried to redeem for an access token, but since you've already used the authorization grant token in a previous page load, it refuses to grant a new access token.

The second request cnj94rgr is the same issue I was facing, the Invalid audience error is due to using scopes with the namespace https://outlook.office365.com instead of https://graph.microsoft.com to try to access the graph API.

I was able to get my code to work by modifying the roundcube source as follows:

config/defaults.inc.php

// Optional: OAuth scopes specific to identity request
$config['oauth_scope_identity'] = null;

// Add this in the 365 example config section
// $config['oauth_scope_identity'] = "https://graph.microsoft.com/IMAP.AccessAsUser.All SMTP.Send offline_access User.Read";

config/config.inc.php

// Add the new config key, oauth_scope_identity, for use in a token refresh to access Graph API
$config['oauth_scope'] = "https://outlook.office365.com/IMAP.AccessAsUser.All https://outlook.office365.com/SMTP.Send User.Read offline_access";
$config['oauth_scope_identity'] = "https://graph.microsoft.com/IMAP.AccessAsUser.All SMTP.Send offline_access User.Read";

prorgam/include/rcmail_oauth.php

 diff /tmp/roundcube-1.5.8-dist/program/include/rcmail_oauth.php /var/html/program/include/rcmail_oauth.php
78c78,79
<             'scope'           => $this->rcmail->config->get('oauth_scope'),
---
>           'scope'           => $this->rcmail->config->get('oauth_scope'),
>           'scope_identity'  => $this->rcmail->config->get('oauth_scope_identity'),
267a269,276
>                         // Change identity scope
>                         if($this->options['scope_identity']) {
>                             $this->mask_auth_data($data);
>                             $refresh_response = $this->refresh_access_token($data, $this->options['scope_identity']);
>                             $data = $refresh_response['token'];
>                             $authorization = $refresh_response['authorization'];
>                         }
>
282a292,298
>
>                         // Restore scope for general mail functions
>                         if($this->options['scope_identity']) {
>                             $refresh_response = $this->refresh_access_token($data, $this->options['scope']);
>                             $data = $refresh_response['token'];
>                             $authorization = $refresh_response['authorization'];
>                         }
358c374
<      *
---
>      * @param string $change_scope Optional, changes scope if specified
361c377
<     public function refresh_access_token(array $token)
---
>     public function refresh_access_token(array $token, $change_scope = null)

In summary, the above changes:

• Add a new optional configuration key called oauth_scope_identity that is used to switch the Oauth API scopes when a different namespace/set of permissions are required to authorize access to the identity endpoint
• The oauth function refresh_access_token now allows you to optionally change the scope of the new token
• Changes were also made to the oauth function request_access_token, which is responsible for taking the oauth grant token returned by the MS form and fetching an access token with it, then calling the identity API to get identity information. During the "request user identity" step, if oauth_scope_identity was set in config, grab a new token using that scope before communicating with the identity API, then change it back so that you are left with permissions appropriate to access the mail functions

I'll try to create a branch and a pull request at some point, running a bit low on time at the moment.

[feroda]

Hello,

at the end I had time to manage it working now. I have tried you solution @lambjs but it hasn't fit my needs.

My problem is that I had not setup xoauth2 authentication in dovecot!

So, in order to recap, I have followed the doc at https://github.com/roundcube/roundcubemail/wiki/Configuration:-OAuth2#prerequisites

And I have configured the xoauth2 auth_mecanism in dovecot with the Microsoft Identity Platform in the backend so, from: https://doc.dovecot.org/2.3/configuration_manual/authentication/oauth2/ (@alecpl maybe you can add this link to the Oauth2 doc in requeirements as an example...)

Just for the sake of completeness/SEO,

Common

In dovecot.conf put

auth_mechanisms = $auth_mechanisms oauthbearer xoauth2

passdb {
  driver = oauth2
  mechanisms = xoauth2 oauthbearer
  args = /etc/dovecot/dovecot-oauth2.conf.ext
}

Backend

Configuration file example for Microsoft Identity Platform https://learn.microsoft.com/en-us/entra/identity-platform/userinfo

introspection_mode = auth
introspection_url = https://graph.microsoft.com/v1.0/me
# this can vary on your settings
username_attribute = mail
tls_ca_cert_file = /etc/ssl/certs/ca-certificates.crt

thank you all guys

[lambjs]

@feroda I think I missed the fact that while you were hoping to use O365 for identity management, but you had set localhost as the server for IMAP and SMTP:

$config['default_host'] = "localhost";                                                                                                                                                        
$config['smtp_server'] = "localhost"; 

Seems like you're not intending to use M365 for incoming/outgoing services at all, just as an identity service (referencing the Graph backend), and using your local Dovecot server for sending/receiving mail. This is interesting and may be useful for select use cases, but I think the idea of setting up your relay falls outside the scope of Roundcube's OAUTH typical configuration.

Is there a reason you wouldn't just use the following to skip the relay and go directly to 365 servers?

$config['default_host'] = 'ssl://outlook.office365.com';
$config['smtp_server'] = 'ssl://smtp.office365.com';