Redesign the authentication systems

[muhlemmer]

This issue is to enhance the security of authentication within Mailu. It is based on a small brain-storming session me and @kaiyou had.

In order to properly secure and manageable 2fa system, we will need to centralize all web logins to a central point (eg admin). This system will forward an authentication token to the final app (webmail / admin). From this system we can extend to 2fa. Additionally, we want to investigate using a more advanced authentication system for IMAP:

[...] there are a couple technos available, OAuth, SAML, and also the good all redirect-and-post-the-login-form-for-me technique. We have to experiment

• Write a central authentication handler in admin
• Implement remote authentication for Roundcube
• Implement remote authentication for Rainloop
• Advanced auth system for IMAP
• Implement 2fa in the central handler

This issue replaces #67, #264 and #685

[pierreozoux]

Hello!

I created http://libre.sh some time ago,and probably one of the first mail system on docker :)
Since you created your project, I follow it silently.

Now, I'm thinking of the redesign of libre.sh, migrating from docker-compose to kubernetes.
If possible, I'd like to avoid redoing this mail thing. My solution was half backed, and I didn't manage to create a real community behind, unlike you. Mail is a full time job, and we didn't have time for that.
Checking your project, it really looks good now. And I'm really happy to see all your efforts, and I'd like to express my gratitude!
So of course, I'm considering using mailu for the libre.sh v2. (I saw that you have a kube PoC, exciting ;) )

This was the introduction, and to give context about this comment. Now, about this problem.

For libre.sh v2 we'll use keycloack as the central login management. Everything is kind of fine, except for email. And the more I think about it, the more I come to the same conclusion.
First, with technologies like 2FA or OpenID connect, it work great for everything web, but for things not web, it doesn't.
If you look at solutions for this specific problem, there is one that come to my mind, and this is the strategy of Google or Nextcloud.

--> App password.

So, about this issue, I'd go back to the problem space.
In the problem space, you want to improve the authN and the security.

My proposed solution would be to keep it the way it is, and add the possibility to delegate to another system (openID connect, SAML...) these systems (like keycloack) do implement 2FA.
And then, you let the user generate app password to be able to create them, revoke them and use them from their traditional mail client.

I'd be really interested in this feature.
What do you think, is it an acceptable solution to your orginal problem? Does it make sense?
My problem is that I'd like to avoid ldap at all cost, and I think this is a solution to my problem :)

[briantopping]

Hi @pierreozoux! I am not a member of the project, just a user with similar interests such as 2FA and Kubernetes. I opened #605, but after digging quite deep into PAM, I am not sure it is sufficient for a great user experience. Much of this experience has come from generating a XAUTH IPSEC VPN deployment for k8s. (It works against FreeIPA, but not yet with 2FA due to Kerberos FAST and preauth issues.)

The problem with mail and 2FA as I have come to realize is the users tend to be less forgiving to re-enter a 2FA password every time the IMAP session resets or they send an email. You can imagine the situation, every time the front end tries to validate against the authN, the TOTP will have rotated and the password is no longer valid. So not only does the user have to retype their password, they have to find their 2FA token. A good discussion of use cases is in https://docs.pagure.org/SSSD.sssd/design_pages/pam_conversation_for_otp.html, which is the design doc for FreeIPA OTP.

One of the considerations the author there makes is that if the first and second factor is split out, the second factor request (the TOTP token, typically) can be skipped as a convenience. You'll see as you read it that it is a slightly different use case though. In our case, the IMAP/SMTP client needs the capability to have a long lasting strong authentication method that can span sessions. For instance, if I close my laptop, take it to another location and re-open it, I typically do not expect to sign back in to each email account.

One means is to use Kerberos tickets, but I've found while tickets work on OSX, there is no way to configure the KDC on iOS, so there is no way to get mail on the phone.

I believe both platforms would be solved with OIDC and Keycloak / Dex, but there needs to be support in common clients for this to work. I am a OSX/iOS user, so I am always looking for what will work on those platforms. I believe Gmail is using OIDC behind the scenes, but enabling it outside of the OSX/iOS Gmail-specific setup GUI is not obvious.

So I think we have some work to do here.

Regarding the k8s POC, there are of course many ways for these scripts to be deployed. I absolutely would not have been successful in deploying Mailu without them, but in turn, I also made some changes to them that I have not had time to even attempt to merge back. I would like to get them to to the point of a Helm chart and/or Kubernetes Operator.

Lastly, please be careful with your consideration of Keycloak / Dex. They typically need to talk to a storage backend such as a RDBMS or LDAP. You mentioned getting away from LDAP and I tried the same thing, but in the end, spent a lot of time putting together a k8s StatefulSet for FreeIPA. In the end, I am glad that I did it that way, but this provider for Keycloak / Dex is why I fell back to it.

EDIT: The other thing that FreeIPA can provide for an alternate AuthN would be integrated x509 for client certificate authentication and access. If the certificate is password protected, the result is 2FA, but not nearly as secure as with a token, and is very dependent on a CRL server to keep the entire system secure.

EDIT: @muhlemmer, when I mentioned #605 in chat the other day, I had no idea you had started this. When I originally wrote this response, I thought I was responding to a ticket created by @pierreozoux. I would regret if you thought I was mentioning #605 because I was somehow interested in competing with this ticket (I'm not). That said, I think it would be a good idea that the objectives of #605 were considered in addition to #67, #264 and #685.

[pierreozoux]

I think I'm not as much expert as you. But still I think you didn't get precisely what I said.

I think that everything beside web is difficult to use with central login and/or 2FA. Then if you look at google or Nextcloud, you have 2 options that are not exclusive:

• build your own app that can handle it
• offer single use app passwords

I think that we agree that the first one is not desirable.

And the single app password is a great option for these reasons:

• you can create as many as you want
• the user sees it just once, when it is generated
• the user can see the list, when it was last used and from where and revoke in case of doubts
• the user can't use its own password that it uses everywhere else

After thinking a lot about what would be the best way, and seeing google doing that, I think that app passwords are the way to go.

For mailu it means changing the way IMAP/SMTP validates the password (against this list of hash instead of against one hash) and then you can add your Id provider just for the web UI which is a lot simpler problem that thinking integration at Imap/SMTP level.

I hope my thoughts are clearer. And yes I agree that freeIPA, keycloak, 2FA are part of the same discussion :)

[muhlemmer]

Hi, So the proposal I'm doing here is more front-end related. For instance, 2FA is quite easy to implement in Flask. So we want to relay webmail authentication to the Admin container. Additionally, we can look into an OAuth(-like) protocol to present the same login page from mail clients like Thunderbird. (In the style GMail is doing nowadays)

Back-ends should be discussed and implemented in their respective threads. The biggest problems would be the amount of man-hours that would go into implementing such back-ends like FreeIPA or LDAP. Flask/sqlalchemy is providing an easy and efficient way to do SQL interfacing. For any other implementation somebody would have to write (or find) such an adaptor library first. I don't see it very likely that I will start work on that, but maybe someone will step up.

[kaiyou]

I am a huge advocate for separating authentication and profile management. Which obviously, I did not do when first writing Mailu. The road I see for this issue is as follow:

• improve the current authentication token system (add limited validity, limited number of authentications, etc.)
• use authentication tokens to provide transparent authentication to the webmails (dynamically generate a time-limited and single-use token, then redirect to the authentication form)
• separate profile (settings) from the authentication process (password storage)
• support other authentication types with modules, including LDAP, Kerberos, and why not OpenID connect or SAML.

[kaiyou]

For the record, Mailu already has an authentication token mechanism, and does compare a password to a list.

[bf8392]

Hi =) How is the status of 2fa for now? will it be implemented?

[Nebukadneza]

Hi There,

The Mailu-Project is currently in a bit of a bind! We are short on man-power, and we need to judge if it is possible for us to put in some work on this issue.

To help with that, we are currently trying to find out which issues are actively keeping users from using Mailu, which issues have someone who want to work on them — and which issues may be less important. These a less important ones could be discarded for the time being, until the project is in a more stable and regular state once again.

In order for us to better assess this, it would be helpful if you could put a reaction on this post (use the 😃 icon to the top-right).

• 👍️ if you need this to be able to use Mailu. Ideally, you’d also be able to test this on your installation, and provide feedback …
• 🎉 if you find it a nice bonus, but no deal-breaker
• 🚀 if you want to work on it yourself!
  We want to keep this voting open for 2 weeks from now, so please help out!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[briantopping]

So I guess we've moved from a lack of response on valid issues to spending time implement stalebot to close issues. As @AndrewSav said in this comment:

it's not right to close an issue just because currently no one can work on it. It does not become a non-issue because of that and if you close it some one will come and open a new one. There is no benefit is closing issues without fixing just for the sake of closing

So if what you're trying to do is somehow make Mailu look complete because there are no open issues, this will be effective. If you want Mailu to be the best mail server out there, don't be so anxious to close issues that people spent good time thinking through and articulating. It's rude and people will start to look elsewhere for solutions.

If you don't like an issue, close it as "will not fix". Otherwise, please fix /lifecycle frozen. Thank you!

[lub]

@briantopping (I'm not speaking for the project in any way, just my own opinion)

So if what you're trying to do is somehow make Mailu look complete because there are no open issues, this will be effective.

I don't think anybody particularly cares how 'complete' Mailu looks.

If you want Mailu to be the best mail server out there, don't be so anxious to close issues that people spent good time thinking through and articulating. It's rude and people will start to look elsewhere for solutions.

IMO it's similarly rude to expect members of the project to regularly go through old issues that could've been long closed by the original authors. I'm not saying that this is something you did, but when you look through the issues there are many that couldn't be reproduced and are lacking feedback from the very same authors.

Stalebot has exceptions for specific labels (for now priority/*) and was not implemented to just close everything. It's probably a little bit aggressive in the beginning, but I think it's a good compromise going forward.

[alexanderadam]

there are many that couldn't be reproduced

So, you are only speaking of bugs here, right?
Because there's usually nothing to reproduce for feature issues and many of the closed issues are features.

Brian has a point: Mailu's issues should rather be closed with wontfix label then if those features won't be implemented.
This way users will know that they should rather focus on one of the other docker mail server solutions if they need this feature.

[AndrewSav]

IMO it's similarly rude to expect members of the project to regularly go through old issues that could've been long closed by the original authors.

Sorry, I have no idea what you mean here. If you do not want to regularly go through the issues than do not, but closing a legitimate issue, just because you cannot work on them right now is still not nice toward people who contributed to them regardless of whether you regularly go through the issues or not, the point still stands.

I'm not saying that this is something you did, but when you look through the issues there are many that couldn't be reproduced and are lacking feedback from the very same authors.

This is easy to solve, ask for clarification, if not received, close as cannot reproduce. But keep closing stuff because there were "no activity" will just lead to endless +1's and /make unstale. It's not like an issue needs a constant drip of information, once it's clear what it's about.

[Nebukadneza]

Please move the offtopic discussion to #1642 to keep this open for on-topic discussion.

[nextgens]

I've tackled some of this in #1745

• Write a central authentication handler in admin
• Implement remote authentication for Roundcube
• Implement remote authentication for Rainloop

[nextgens]

I've split the PR above into something smaller and easier to review on #1754

@muhlemmer @kaiyou can you have a glance/review it please?

IMHO "shielding" webmails from unauthenticated users is very important (attack surface reduction / basic security hardening)

[flypenguin]

as just being asked on matrix - this would have been the deal-maker for our company-wide installation. background: teh german health system has a "hidden" email system called "KIM" (translates to "communications in medicine"), which is one of the most broken concepts ever designed. we want to wrap this monstrosity in a conceptually sane email-system (enter mailu) and provide the functionality to our users.

this is a high priority objective. first, this is required by law. second, we can monetize this (mails sent over that system can be billed to the insurance company).

unfortunately we use a company-wide IAM (saml, oauth) based on okta which is a requirement.

[pierreozoux]

@flypenguin PM @muhlemmer .

If you have the money, I'm sure he will be really happy to dev it for you :) I had a chat with him, but didn't have the money unfortunately. He was really nice, and I really hope you can make this happen!

[flypenguin]

@pierreozoux that ... might be happening. @muhlemmer could you contact me under [email protected] to talk details if you're interested? I figured out a theoretical but ugly workaround for mailcow, buf if we could do this it would help greatly. we would need to check the other requirements beforehand (auto-sync, relay configs, etc.) so we don't realize later we have breaking issues in other places 😄 .

[Igortorrente]

What is the status of the authentication system ?

I read the whole thread but I still not sure what is the current status.

Which of the following authentication methods mailu supports currently:

• LDAP?
• OAuth2?
• OAuth2 + OpenID connect?

[shaun-forgie]

I would love to see authorisation and authentication mechanisms separated out....Our personal preference would be to use Keycloak for both. Allowing mailu to integrate with existing single sign on solutions would dramatically improve its appeal to the enterprise user community.

[nextgens]

There seems to be a widespread misconception as of what this ticket aims to achieve. We are talking about centralizing the authentication of dependencies of Mailu (by making the admin container an IdP)... not making Mailu dependent on some 3rd party solution. Mailu's mission statement is clear: Mailu aims to be insular... the last thing we want/need is emails getting lost because the 3rd party AAA layer can't decide what it wants to do in time or is otherwise unavailable.

For the record, Mailu already has all the components required to allow integration with an AIM solution such as Keycloack. I refer you to the documentation of https://mailu.io/master/configuration.html#header-authentication-using-an-external-proxy . While I agree that it could be documented better and the functionality there could be enhanced, I am sure that several people here (including me) will be happy to provide professional services to "make it fit" your Enterprise usecase.

The two missing tickboxes on the ticket are: "Advanced auth system for IMAP" and "Implement 2fa in the central handler". The former is unlikely to happen as MUAs currently do not enable users to set XOAUTH2 parameters (instead they ship a list of configured services)... and the later will be worked on when the next release is out.

[briantopping]

@nextgens: agree with your perspective. I suppose what most people are after is a set of mechanisms that allow for basic documentation recipes to lead to successful integrations with whatever IDP they are using.

Do these goals exist in bountysource or something similar? It seems like ten people putting in $100 would be a lot more successful than waiting for one person with $1000 to come along and pay for a custom solution that may never get integrated with the public source.

[inmanturbo]

@briantopping The ability to integrate with IDP or SSO is not an enterprise feature. Is a a security feature. And the header auth with a proxy provider should work with most any idp. It's just relatively new (as yet unreleased) and there aren't a lot of documentation and examples available.

I don't think @nextgens was talking about making you a private "paywall" fork. Seems to me like @nextgens was offering (likely quite valuable) professional services to get your system up and running the way you want it with what is already available. If it brings to light any improvements that could be made to the codebase I'm sure everything would benefit.

[briantopping]

@inmanturbo Thanks for the input, I'm not in the market for the solutions on offer, so it has nothing to do with me or my systems. Nevertheless, I helped the confusion by not being aware that some part of this solution was recently committed. Kudos for that.

That said, using a tool like Bountysouce to pool interest for features still makes sense. Mailu is a good system and OSS authors don't get a free pass at the grocery checkout. I don't currently use Mailu, but I'd be tempted to toss in anyway for the next time I do use it. Closing feature gaps in any project pays exponential dividends to community growth. Everyone wins for that, including the providers of likely quite valuable professional services.

[inmanturbo]

@inmanturbo Thanks for the input, I'm not in the market for the solutions on offer, so it has nothing to do with me or my systems. Nevertheless, I helped the confusion by not being aware that some part of this solution was recently committed. Kudos for that.

That said, using a tool like Bountysouce to pool interest for features still makes sense. Mailu is a good system and OSS authors don't get a free pass at the grocery checkout. I don't currently use Mailu, but I'd be tempted to toss in anyway for the next time I do use it. Closing feature gaps in any project pays exponential dividends to community growth. Everyone wins for that, including the providers of likely quite valuable professional services.

I'm not using it anywhere currently either. If I could use it I'd likely be a sponsor. I've had my eye on it though. I can't use it in any of my current projects because of #2279. #2542 will help as well once it makes it into a release.