Use SPNEGO mechanism by default (#41)

[michael-o]

Use the correct mechanism as described by

• RFC 4559, section 4: The "Negotiate" auth-scheme calls for the use of SPNEGO
  GSSAPI tokens that the specific mechanism type specifies.
• RFC 4178, section 3.2: The GSS-API initiator invokes GSS_Init_sec_context()
  as normal, but requests that SPNEGO be used. SPNEGO can either be explicitly
  requested or accepted as the default mechanism.

Since both MIT Kerberos and Heimdal use Kerberos 5 as their default mechanism
we must explicitly request SPNEGO. Passing raw Kerberos tokens to the acceptor
is a violation of these RFCs and some implementations complain about, thus they
always need to be wrapped.

This closes #41

Signed-off-by: Michael Osipov [email protected]

[jborean93]

This one is a bit more difficult as using SPNEGO by default might lead to mechs being used that take more than 1 exchange to complete (like NTLM). I agree this is the right behaviour and should be done but this will require futher testing to ensure it that multiple exchanges are still supported.

If this just works then that's great if not then we will either need to

• Fix the code to work in those scnearios, or
• Limit the mechs the SPNEGO mech uses to just Kerberos

The latter is simple with gss_set_neg_mechs but not very versitile and restricts SPNEGO for no great reason.

[michael-o]

I see your point and wonder when this can happen at all, given that MIT Kerberos does not support NTLM at all. Heimdal only if you place a special file with your plaintext credentials (as far as I remember). Unless you use https://github.com/gssapi/gss-ntlmssp.
I would also expect the server to send Persistent-Authentication: true header to indicate a multi roundtrip. This also correlates with #39 where I have described that the context continuation is not properly probed/performed. But still, you already need now to persist the security context between requests to complete the security context when the server responds with a token.

I am fine with gss_set_neg_mechs because I do not really care about NTLM, but this would be even more code.

Let me know what you think....

[michael-o]

This also correlates with #8. See my comments there. I use a Python application with an executor service to scale threads and stumbled upon this non-threadsafe design.

[jborean93]

I see your point and wonder when this can happen at all, given that MIT Kerberos does not support NTLM at all. Heimdal only if you place a special file with your plaintext credentials (as far as I remember). Unless you use https://github.com/gssapi/gss-ntlmssp.

Yea the Heimdal implementation of NTLM is pretty old and last time I tried it didn't even support NTLM v2 so it's not really usable there. MacOS also uses their own special variant of Heimdal and have a somewhat usable NTLM mech included in box. There are some problems with it but it at least supports NTLM v2 like gss-ntlmssp. MIT would have to use an extra mech like gss-ntlmssp that you've mentioned and while it's not included out of the box many distros include this as a package so it's not out of the question that it may be available. There's also a very very small possibility that another mech becomes available through Negotiate. I cannot think of any today but with the introduction of Negoext support in MIT (and possibly Heimdal) who knows what will happen in the future.

Ultimately using NTLM is not that great and should be avoided where possible my concern is that with this change it is now possible for that to happen, even inadvertently. Currently if Kerberos cannot be used then it will error with the reasons why. With this patch it could obfuscate the Kerberos problem leading to a potential downgrade in authentication.

I would also expect the server to send Persistent-Authentication: true header to indicate a multi roundtrip

You would expect but unfortunately one of the most common ones out there (MS) do not. They typically work around multi trip connections by trying to authentication state to the actual socket that has been opened. But what you have said here is pretty much spot on and why I'm reluctant to start using the SPNEGO mech. The code just isn't set up properly to handle such a thing at this point.

I am fine with gss_set_neg_mechs because I do not really care about NTLM, but this would be even more code.

I think it will be the simplest option right now, the signature doesn't seem that hard. It might be a simple first step for doing the Kerberos wrapped in SPNEGO to make it compliant and better support for the other mechs can be done in the future. Essentially you would want something like (not actually tested)

from gssapi.raw import acquire_cred, set_neg_mechs

krb5_mech = gb.OID.from_int_seq("1.2.840.113554.1.2.2")
spnego_mech = gb.OID.from_int_seq("1.3.6.1.5.5.2")

cred = gssapi.raw.acquire_cred(None, usage='initiate',
    mechs=[krb5_mech, spnego_mech]).creds
gssapi.raw.set_neg_mechs(cred, [krb5_mech])

context = ...

My only real concern with this method is that I've never fully tested it across the platforms like Heimdal or macOS Heimdal.

This also correlates with #8. See my comments there. I use a Python application with an executor service to scale threads and stumbled upon this non-threadsafe design.

Yea this is another holdover from requests-kerberos and is somewhat due to the fact that the context was used for further wrapping and unwrapping of the data that was sent over HTTP. Because that isn't a thing here it should be possible to solve that using the methods mentioned in that issue.

[michael-o]

I see your point and wonder when this can happen at all, given that MIT Kerberos does not support NTLM at all. Heimdal only if you place a special file with your plaintext credentials (as far as I remember). Unless you use https://github.com/gssapi/gss-ntlmssp.

Yea the Heimdal implementation of NTLM is pretty old and last time I tried it didn't even support NTLM v2 so it's not really usable there. MacOS also uses their own special variant of Heimdal and have a somewhat usable NTLM mech included in box. There are some problems with it but it at least supports NTLM v2 like gss-ntlmssp. MIT would have to use an extra mech like gss-ntlmssp that you've mentioned and while it's not included out of the box many distros include this as a package so it's not out of the question that it may be available. There's also a very very small possibility that another mech becomes available through Negotiate. I cannot think of any today but with the introduction of Negoext support in MIT (and possibly Heimdal) who knows what will happen in the future.

While I don't know about Apple's Heimdal flavor, as far as I understand the gss-ntlmssp it required either winbind to be present or some keyfile which is not the Kerberos keytab. So the likelihood is even lower. As for new mechs in SPNEGO and NegoEx: I'd would take a look at then when they arrive. Note that IIS dislabed Negotiate auth on HTTP/2 due to the multiplex nature and possible downgrade to NTLM which is completely incompatible with HTTP/2.

Ultimately using NTLM is not that great and should be avoided where possible my concern is that with this change it is now possible for that to happen, even inadvertently. Currently if Kerberos cannot be used then it will error with the reasons why. With this patch it could obfuscate the Kerberos problem leading to a potential downgrade in authentication.

Yes, it could, but that is intentional with the protocol/design of SPNEGO. I would explicitly refer to that and live with it. Yet, this still applies to a fraction. Other mechanisms explicitly forbid SPNEGO: https://datatracker.ietf.org/doc/html/rfc5801#section-14

I would also expect the server to send Persistent-Authentication: true header to indicate a multi roundtrip

You would expect but unfortunately one of the most common ones out there (MS) do not. They typically work around multi trip connections by trying to authentication state to the actual socket that has been opened. But what you have said here is pretty much spot on and why I'm reluctant to start using the SPNEGO mech. The code just isn't set up properly to handle such a thing at this point.

Ouch, a big dislike for MS, but no suprise. In Apache HttpClient we want to phase out NTLM explicitly for that, state management which clearly violates HTTP semantics. Aas for the inproper setup: I would try to cross the bridge only when we arrive there.

I am fine with gss_set_neg_mechs because I do not really care about NTLM, but this would be even more code.

I think it will be the simplest option right now, the signature doesn't seem that hard. It might be a simple first step for doing the Kerberos wrapped in SPNEGO to make it compliant and better support for the other mechs can be done in the future. Essentially you would want something like (not actually tested)

from gssapi.raw import acquire_cred, set_neg_mechs

krb5_mech = gb.OID.from_int_seq("1.2.840.113554.1.2.2")
spnego_mech = gb.OID.from_int_seq("1.3.6.1.5.5.2")

cred = gssapi.raw.acquire_cred(None, usage='initiate',
    mechs=[krb5_mech, spnego_mech]).creds
gssapi.raw.set_neg_mechs(cred, [krb5_mech])

context = ...

My only real concern with this method is that I've never fully tested it across the platforms like Heimdal or macOS Heimdal.

While I can't speak for those, I have patched requests-negotiate-sspi last year and it uses SPNEGO as-is. A bit of consistency for portability here would be nice. BTW, Java has an experimental binding to SSPI as of version 13 and the author of that change did the same, he explicitly enabled only Kerberos for SPNEGO in consistency with the Java-native impl.

This also correlates with #8. See my comments there. I use a Python application with an executor service to scale threads and stumbled upon this non-threadsafe design.

Yea this is another holdover from requests-kerberos and is somewhat due to the fact that the context was used for further wrapping and unwrapping of the data that was sent over HTTP. Because that isn't a thing here it should be possible to solve that using the methods mentioned in that issue.

I don't think that something like this is required anymore, this is heavily out of spec. I would prefered GSS-API work like TLS, one level below the app protocol.

So we have now two options:

• Accept the current state/limitations of SPNEGO and take care of when the first issues arrive
• Restrict upfront and document it

I am fine with either one. Let me know which I should amend in the PR.

[jborean93]

While I don't know about Apple's Heimdal flavor, as far as I understand the gss-ntlmssp it required either winbind to be present or some keyfile which is the Kerberos keytab. So the likelihood is even lower

Ah this is a point I didn't consider. Technically a user could define NTLM_USER_FILE which points to a plaintext file and gssapi will pick it up (with gss-ntlmssp) but as you said this is an extremely low likelyhood.

This might have to revisited if this library ever exposes support for explicit creds or starts using gss_acquire_cred_with_password but we can cross that bridge if we get to it. Considering the rarity of NTLM being actually available in a credential cache the lack of unlikely scenario of a new mech being available I'm happy with just changing the mech and not worrying about a multi legged authentication requirement.

[michael-o]

While I don't know about Apple's Heimdal flavor, as far as I understand the gss-ntlmssp it required either winbind to be present or some keyfile which is the Kerberos keytab. So the likelihood is even lower

Ah this is a point I didn't consider. Technically a user could define NTLM_USER_FILE which points to a plaintext file and gssapi will pick it up (with gss-ntlmssp) but as you said this is an extremely low likelyhood.

+1

This might have to revisited if this library ever exposes support for explicit creds or starts using gss_acquire_cred_with_password but we can cross that bridge if we get to it. Considering the rarity of NTLM being actually available in a credential cache the lack of unlikely scenario of a new mech being available I'm happy with just changing the mech and not worrying about a multi legged authentication requirement.

I think exposing this would be wrong, imho. For me that is the burden of the calling code to supply a cred object. In fact, I had to do this for my case where I use a client keytab and multiple threads, the file-based cache suffers from race conditions: https://mailman.mit.edu/pipermail/kerberos/2021-April/022630.html

[jborean93]

Thanks for the PR.

I've just manually tested the change and can confirm the token is wrapped in the SPNEGO NegTokenInit message allowing it to work with servers that expect this wrapping rather than the raw InitialContextToken from before.

While this is not something I consider to be something we should care about I can confirm that if a cached NTLM credential is available the auth will fail if Kerberos is unavailable and NTLM had to be used with

Traceback (most recent call last):
  File "/home/jborean/dev/pypsrp/gssapi-jordan.py", line 6, in <module>
    r.raise_for_status()
  File "/home/jborean/.pyenv/versions/ansible-310/lib/python3.10/site-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error:  for url: http://192.168.56.15:5985/wsman

This is to be expected and considering the rarity of someone actually having gss-ntlmssp + winbind or NTLM_USER_FILE set I don't think we need to worry. This can always be revised in the future if it does present to be a bigger problem than expected but for now I'm happy with the changes as they are.

[michael-o]

Thanks for the professional workaround. Looking forward to a release in the near future.

BTW, you can also register a IP-based SPN, at least with Active Directory it should work, but I have never used it. Quite like an IP address in SAN in a X.509 server cert.

[jborean93]

BTW, you can also register a IP-based SPN, at least with Active Directory it should work, but I have never used it. Quite like an IP address in SAN in a X.509 server cert.

Thanks, the IP was mostly just used to force NTLM to be used and Kerberos be skipped, typically hostname resolution is good enough for me :)

[simo5]

Ah this is a point I didn't consider. Technically a user could define NTLM_USER_FILE which points to a plaintext file and gssapi will pick it up (with gss-ntlmssp) but as you said this is an extremely low likelyhood.

Note that gssntlmssp includes winbind integration, so technically it may kick in for any SPNEGO interaction on a machine joined to AD via Winbindd. I do not thik this should change your mind, but just wanted to point out that manual configuration is not always required.

[michael-o]

Ah this is a point I didn't consider. Technically a user could define NTLM_USER_FILE which points to a plaintext file and gssapi will pick it up (with gss-ntlmssp) but as you said this is an extremely low likelyhood.

Note that gssntlmssp includes winbind integration, so technically it may kick in for any SPNEGO interaction on a machine joined to AD via Winbindd. I do not thik this should change your mind, but just wanted to point out that manual configuration is not always required.

Correct, I have mentioned this in my answer also.