Add support for CKAN 2.10+

[themowski]

Overview

With the release of CKAN 2.10, the original authentication technology provided by repoze.who has been replaced with flask-login. These changes were introduced in ckan/ckan PR#6560. As a result of these changes, the ckanext-ldap plugin does not work out of the box with CKAN 2.10.0+.

I installed the latest version of the plugin (v3.2.10) in CKAN 2.10.1 via the official Docker setup provided at ckan/ckan-docker. When I entered my credentials on the login page, they seemed to have been correctly verified against the server -- an invalid username/password combination failed as expected, but the correct credentials routed me to a different page. I received the generic "Internal Server Error" message in the CKAN UI, and saw this in the logs:

2023-08-30 16:48:28,147 ERROR [ckan.config.middleware.flask_app] Could not build url for endpoint 'user.logged_in'. Did you mean 'user.login' instead?
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python3.10/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/usr/lib/python3.10/site-packages/ckanext/ldap/routes/login.py", line 44, in login_handler
    return _helpers.login_success(user_name, came_from=came_from)
  File "/usr/lib/python3.10/site-packages/ckanext/ldap/routes/_helpers.py", line 72, in login_success
    return toolkit.redirect_to('user.logged_in', came_from=came_from)
  File "/srv/app/src/ckan/ckan/lib/helpers.py", line 274, in redirect_to
    _url = url_for(*uargs, **kw)
  File "/srv/app/src/ckan/ckan/lib/helpers.py", line 385, in url_for
    my_url = _url_for_flask(*args, **kw)
  File "/srv/app/src/ckan/ckan/lib/helpers.py", line 443, in _url_for_flask
    my_url = _flask_default_url_for(*args, **kw)
  File "/usr/lib/python3.10/site-packages/flask/helpers.py", line 338, in url_for
    return appctx.app.handle_url_build_error(error, endpoint, values)
  File "/usr/lib/python3.10/site-packages/flask/helpers.py", line 325, in url_for
    rv = url_adapter.build(
  File "/usr/lib/python3.10/site-packages/werkzeug/routing.py", line 2315, in build
    raise BuildError(endpoint, values, method, self)
werkzeug.routing.BuildError: Could not build url for endpoint 'user.logged_in'. Did you mean 'user.login' instead?

The /user/logged_in endpoint that this references was removed in the upstream ckan/ckan PR#6560, so the plugin's logic needs to be updated accordingly.

Suggested Implementation

Adding support for CKAN 2.10 seems to be pretty straightforward. From what I can tell, we need to handle the fact that /user/logged_in does not exist anymore, and we need to integrate with the new toolkit.login_user() and toolkit.logout_user() functions to manage the user's login status.

Changes are required in two files. The diffs below were written against [email protected], the latest version as of this writing.

These suggestions use the toolkit.check_ckan_version() function to only call the new functionality when running CKAN 2.10+. This means that the plugin should continue to work without issue in CKAN < 2.10 (but note that I did not test that case). That said, one thing to note about the check_ckan_version() function is that alpha/beta releases of CKAN are considered to be the same version as the corresponding major release (an issue that I will be raising with CKAN separately). This means that if users are working with an alpha build of CKAN 2.10 from before PR#6560 was merged, then this functionality will not work as expected. I think it is probably safe to say that that is an unsupported edge case, though.

ckanext/ldap/plugin.py

The logout() function in plugin.py needs to be modified to call the toolkit.logout_user() function when the CKAN version is at least 2.10. Git diff below:

diff --git a/ckanext/ldap/plugin.py b/ckanext/ldap/plugin.py
index b22bce8..34d2a1d 100644
--- a/ckanext/ldap/plugin.py
+++ b/ckanext/ldap/plugin.py
@@ -154,8 +154,15 @@ class LdapPlugin(SingletonPlugin):
 
     # IAuthenticator
     def logout(self):
+        # Delete session items managed by ckanext-ldap
         self._delete_session_items()
 
+        # In CKAN 2.10+, we also need to invoke the toolkit's
+        # logout_user() command to clean up anything remaining
+        # on the CKAN side.
+        if toolkit.check_ckan_version(min_version='2.10'):
+            toolkit.logout_user()
+
     # IAuthenticator
     def abort(self, status_code, detail, headers, comment):
         return status_code, detail, headers, comment

ckanext/ldap/routes/_helpers.py

The bulk of the work needs to be done in the login_success() function in _helpers.py. In CKAN 2.10+, we need to call the toolkit.login_user() function, which requires as input the ckan.model.User object that represents the user logging in. This object needs to be looked up, and there are a few edge cases that could result in errors and need to be handled. Git diff below:

diff --git a/ckanext/ldap/routes/_helpers.py b/ckanext/ldap/routes/_helpers.py
index d978e09..e8f8218 100644
--- a/ckanext/ldap/routes/_helpers.py
+++ b/ckanext/ldap/routes/_helpers.py
@@ -11,7 +11,7 @@ import uuid
 import ldap
 import ldap.filter
 from ckan.common import session
-from ckan.model import Session
+from ckan.model import Session, User
 from ckan.plugins import toolkit
 
 from ckanext.ldap.lib.exceptions import UserConflictError
@@ -36,13 +36,59 @@ def login_failed(notice=None, error=None):
 
 def login_success(user_name, came_from):
     '''
-    Handle login success. Saves the user in the session and redirects to user/logged_in.
+    Handle login success. Behavior depends on CKAN version.
+
+    The CKAN version is tested via ckan.plugins.toolkit.check_ckan_version().
+
+    CKAN < 2.10:
+        Saves user_name in the session under the 'ckanext-ldap-user' key,
+        then redirects to /user/logged_in.
+
+    CKAN 2.10+:
+        Looks up the CKAN User object and invokes the login_user() command
+        (new in CKAN 2.10) to authenticate the user with flask-login.
+        If that succeeds, then this saves user_name in the session under
+        the 'ckanext-ldap-user' key before redirecting to /home/index.
 
     :param user_name: The user name
+    :param came_from: The value of the 'came_from' parameter sent with the
+                      original login request
     '''
+    # Where to send the user after this function ends
+    redirect_path = 'user.logged_in'
+
+    # In CKAN 2.10, repoze.who was replaced by flask-login, and the
+    # /user/logged_in endpoint was removed. We need to retrieve the
+    # User object for the user and call toolkit.login_user().
+    if toolkit.check_ckan_version(min_version='2.10'):
+        redirect_path = 'home.index'
+        user_login_path = 'user.login'
+        err_msg = 'An error occurred while processing your login. Please contact the system administrators.'
+
+        try:
+            # Look up the CKAN User object for user_name
+            user_obj = User.by_name(user_name)
+        except toolkit.ObjectNotFound as err:
+            log.error('User.by_name(%s) raised ObjectNotFound error: %s', user_name, err)
+            toolkit.h.flash_error(err_msg)
+            return toolkit.redirect_to(user_login_path)
+
+        if user_obj is None:
+            log.error(f"User.by_name returned None for user '{user_name}'")
+            toolkit.h.flash_error(err_msg)
+            return toolkit.redirect_to(user_login_path)
+
+        # Register the login with flask-login via the toolkit's helper function
+        ok = toolkit.login_user(user_obj)
+        if not ok:
+            log.error(f"toolkit.login_user() returned False for user '{user_name}'")
+            toolkit.h.flash_error(err_msg)
+            return toolkit.redirect_to(user_login_path)
+
+    # Update the session & redirect
     session['ckanext-ldap-user'] = user_name
     session.save()
-    return toolkit.redirect_to('user.logged_in', came_from=came_from)
+    return toolkit.redirect_to(redirect_path, came_from=came_from)
 
 
 def get_user_dict(user_id):

Final Notes

I am by no means an expert in CKAN, LDAP, or even authentication in general. I learned just enough about how this plugin and CKAN work to implement the initial support for logging into CKAN 2.10+ demonstrated above. My changes do not touch the "Remember Me" functionality, and also do not do anything with CSRF (which is new in CKAN 2.10, and they have a best-practices page in their official documentation).

Both of these seem to be handled by the login() function in ckan/views/user.py, so their code may be a starting place if this is desirable to implement. Per the release notes for CKAN 2.10, CSRF is optional for extensions for now, but will eventually be required in a future version of CKAN.

[jrdh]

Hi @themowski, thanks so much for reporting this and also for taking the time to propose a solution as well, it's always nice to get a thorough issue! ❤️
I've had a look at the problem and I think your solution looks good, but rather than me implement your work, I'd much prefer you get the credit for it. Would you be up for creating a pull request?

[themowski]

Sure, I'll start work on a PR when I find some time. 👍

As a result of hacking on this, I ended up logging ckan/ckan #7777. Based on what I found during the deeper dive I did while logging that ticket & finding edge cases, I might consider changing the approach I suggested above slightly, so that we can more seamlessly support "in-between" deployments (i.e., ones that are 2.10.0 alpha builds). In particular, instead of using the toolkit.check_ckan_version() function, we could check whether that user.logged_in path exists, and change what we do based on that. That said, it's definitely an edge case (alpha builds are always an "at risk" option to run), so I might instead keep the cleaner logic above; I haven't fully decided yet.

I'll also try to dig a little bit into the "Remember Me" & CSRF stuff as part of this PR, but no promises.

[jrdh]

This has been released: https://github.com/NaturalHistoryMuseum/ckanext-ldap/releases/tag/v3.2.11 / https://pypi.org/project/ckanext-ldap/