Some rdap servers seem to exhibit weak security, which throws errors

[lelutin]

Hi there, thanks for this library. it's super simple and works great so far.

I've bumped into some cases where the "whois server" that ends up being poked has pretty weak security and then and an exception gets raised by one of the underlying libraries because of this.

Of course I can handle this exception in my code, but I was wondering if you think it might be reasonable to make whoisit avoid those errors as much as possible by enabling weaker security settings selectively when this exception is thrown.

However, this kind of downgrade behaviour is somehwat nasty in terms of security, so maybe if it is something that can be implemented by whoisit, it would need some method to disable this automatic downgrade.. or maybe better, not make it default but enable it explicitely with an option that needs to be set to True

Here's how you can reproduce this:

>>> import whoisit
>>> whoisit.bootstrap()
>>> whoisit.domain("frame.work")
---------------------------------------------------------------------------
SSLError                                  Traceback (most recent call last)
~/dev/project/.venv/lib/python3.9/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout
, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    698             # Make the request on the httplib connection object.
--> 699             httplib_response = self._make_request(
    700                 conn,

~/dev/project/.venv/lib/python3.9/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    381         try:
--> 382             self._validate_conn(conn)
    383         except (SocketTimeout, BaseSSLError) as e:

~/dev/project/.venv/lib/python3.9/site-packages/urllib3/connectionpool.py in _validate_conn(self, conn)
   1009         if not getattr(conn, "sock", None):  # AppEngine might not have  `.sock`
-> 1010             conn.connect()
   1011

~/dev/project/.venv/lib/python3.9/site-packages/urllib3/connection.py in connect(self)
    410
--> 411         self.sock = ssl_wrap_socket(
    412             sock=conn,

~/dev/project/.venv/lib/python3.9/site-packages/urllib3/util/ssl_.py in ssl_wrap_socket(sock, keyfile, certfile, cert_reqs, ca_certs, server_hostname, ssl_version,
 ciphers, ssl_context, ca_cert_dir, key_password, ca_cert_data, tls_in_tls)
    448     if send_sni:
--> 449         ssl_sock = _ssl_wrap_socket_impl(
    450             sock, context, tls_in_tls, server_hostname=server_hostname

~/dev/project/.venv/lib/python3.9/site-packages/urllib3/util/ssl_.py in _ssl_wrap_socket_impl(sock, ssl_context, tls_in_tls, server_hostname)
    492     if server_hostname:
--> 493         return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
    494     else:

/usr/lib/python3.9/ssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, session)
    499         # ctx._wrap_socket()
--> 500         return self.sslsocket_class._create(
    501             sock=sock,

/usr/lib/python3.9/ssl.py in _create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session)
   1039                         raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
-> 1040                     self.do_handshake()
   1041             except (OSError, ValueError):

/usr/lib/python3.9/ssl.py in do_handshake(self, block)
   1308                 self.settimeout(None)
-> 1309             self._sslobj.do_handshake()
   1310         finally:

SSLError: [SSL: DH_KEY_TOO_SMALL] dh key too small (_ssl.c:1123)

During handling of the above exception, another exception occurred:

MaxRetryError                             Traceback (most recent call last)
~/dev/project/.venv/lib/python3.9/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    438             if not chunked:
--> 439                 resp = conn.urlopen(
    440                     method=request.method,

~/dev/project/.venv/lib/python3.9/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout
, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    754
--> 755             retries = retries.increment(
    756                 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]

~/dev/project/.venv/lib/python3.9/site-packages/urllib3/util/retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
    573         if new_retry.is_exhausted():
--> 574             raise MaxRetryError(_pool, url, error or ResponseError(cause))
    575

MaxRetryError: HTTPSConnectionPool(host='rdap.nominet.uk', port=443): Max retries exceeded with url: /work/domain/frame.work (Caused by SSLError(SSLError(1, '[SSL: DH_KEY
_TOO_SMALL] dh key too small (_ssl.c:1123)')))

During handling of the above exception, another exception occurred:

SSLError                                  Traceback (most recent call last)
~/dev/project/.venv/lib/python3.9/site-packages/whoisit/utils.py in http_request(url, method, headers, data, *args, **kwargs)
     21     try:
---> 22         return requests.request(method, url, headers=headers, data=data, *args,
     23                                 **kwargs)

~/dev/project/.venv/lib/python3.9/site-packages/requests/api.py in request(method, url, **kwargs)
     60     with sessions.Session() as session:
---> 61         return session.request(method=method, url=url, **kwargs)
     62

~/dev/project/.venv/lib/python3.9/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redi
rects, proxies, hooks, stream, verify, cert, json)
    541         send_kwargs.update(settings)
--> 542         resp = self.send(prep, **send_kwargs)
    543

~/dev/project/.venv/lib/python3.9/site-packages/requests/sessions.py in send(self, request, **kwargs)
    654         # Send the request
--> 655         r = adapter.send(request, **kwargs)
    656

~/dev/project/.venv/lib/python3.9/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    513                 # This branch is for urllib3 v1.22 and later.
--> 514                 raise SSLError(e, request=request)
    515

SSLError: HTTPSConnectionPool(host='rdap.nominet.uk', port=443): Max retries exceeded with url: /work/domain/frame.work (Caused by SSLError(SSLError(1, '[SSL: DH_KEY_TOO_
SMALL] dh key too small (_ssl.c:1123)')))

The above exception was the direct cause of the following exception:

QueryError                                Traceback (most recent call last)
<ipython-input-23-0d70ed62a17b> in <module>
----> 1 whoisit.domain("frame.work")

~/dev/project/.venv/lib/python3.9/site-packages/whoisit/__init__.py in domain(domain_name, raw)
     39         query_type='domain', query_value=domain_name)
     40     q = Query(method, url)
---> 41     response = q.request()
     42     return response if raw else parse(_bootstrap, 'domain', response)
     43

~/dev/project/.venv/lib/python3.9/site-packages/whoisit/query.py in request(self, *args, **kwargs)
    178     def request(self, *args, **kwargs):
    179         # args and kwargs here are passed directly to requests.request(...)
--> 180         response = http_request(self.url, self.method, *args, **kwargs)
    181         if response.status_code == 404:
    182             raise ResourceDoesNotExist(f'RDAP {self.method} request to {self.url} '

~/dev/project/.venv/lib/python3.9/site-packages/whoisit/utils.py in http_request(url, method, headers, data, *args, **kwargs)
     23                                 **kwargs)
     24     except Exception as e:
---> 25         raise QueryError(f'Failed to make a {method} request to {url}: {e}') from e
     26
     27

QueryError: Failed to make a GET request to https://rdap.nominet.uk/work/domain/frame.work: HTTPSConnectionPool(host='rdap.nominet.uk', port=443): Max retries exceeded wi
th url: /work/domain/frame.work (Caused by SSLError(SSLError(1, '[SSL: DH_KEY_TOO_SMALL] dh key too small (_ssl.c:1123)')))

[meeb]

Sounds fine to me, there's probably not much risk permitting weak SSL ciphers given RDAP data is publicly available and read-only in the first place. It should be opt-in though. I've added a allow_insecure_ssl=bool (defaulting to False) to queries which downgrades the required SSL cipher suite to accommodate for insecure RDAP servers.

See https://github.com/meeb/whoisit#weaken-ssl-ciphers for details.

This is available in whoisit==2.2.0 which you can upgrade to with pip or your chosen PyPI client now.

Thanks for the issue.

[lelutin]

neat, thanks!