Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

osrandom engine already registered #2287

Closed
komakino opened this issue Aug 21, 2015 · 23 comments
Closed

osrandom engine already registered #2287

komakino opened this issue Aug 21, 2015 · 23 comments

Comments

@komakino
Copy link

Getting "osrandom engine already registered" from cryptography module when including requests in kodi addon.

http://xbmclogs.com/pq07deaup

I'm on Ubuntu 14.04 LTS based system with python 2.7.6.

@sigmavirus24
Copy link
Contributor

Hey @komakino, this usually happens when using threading. Can you confirm that the "kodi addon" is using threading? Also can you link to that project since I have no clue what project it is.

@sigmavirus24
Copy link
Contributor

Also, can you include the version of "kodi" that you're using, the version of requests that's installed, and the version of cryptography in use?

@reaperhulk
Copy link
Member

I remain baffled as to how this condition can occur under the threading lock we have. 😢

@komakino
Copy link
Author

Kodi: 15.1 Git:d496682
requests: 2.6.0

@komakino
Copy link
Author

thats my addon, that I'm developing. I do not use threading. I also tested in a "clean" addon, just importing requests and got the same results.

@komakino
Copy link
Author

Oh, and cryptography is 1.0

@sigmavirus24
Copy link
Contributor

@komakino does that plugin get imported by something using threading? What is kodi a plugin for?

@acdha
Copy link

acdha commented Sep 8, 2015

I'm seeing the same thing in a Django app running under mod_wsgi which is triggered as soon as something imports requests (I noticed it with raven-python but even a simple import requests fails in the threaded WSGIDaemon environment):

[Tue Sep 08 12:07:05.808805 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533] Traceback (most recent call last):
[Tue Sep 08 12:07:05.808838 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]   File "/opt/example/example_wsgi.py", line 7, in <module>
[Tue Sep 08 12:07:05.808924 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]     import requests
[Tue Sep 08 12:07:05.808941 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]   File "/opt/wdl/.virtualenv/lib/python2.7/site-packages/requests/__init__.py", line 53, in <module>
[Tue Sep 08 12:07:05.809377 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]     from .packages.urllib3.contrib import pyopenssl
[Tue Sep 08 12:07:05.809401 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]   File "/opt/wdl/.virtualenv/lib/python2.7/site-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 53, in <module>
[Tue Sep 08 12:07:05.810151 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]     import OpenSSL.SSL
[Tue Sep 08 12:07:05.810174 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]   File "/opt/wdl/.virtualenv/lib/python2.7/site-packages/OpenSSL/__init__.py", line 8, in <module>
[Tue Sep 08 12:07:05.810240 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]     from OpenSSL import rand, crypto, SSL
[Tue Sep 08 12:07:05.810256 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]   File "/opt/wdl/.virtualenv/lib/python2.7/site-packages/OpenSSL/rand.py", line 11, in <module>
[Tue Sep 08 12:07:05.810374 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]     from OpenSSL._util import (
[Tue Sep 08 12:07:05.810397 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]   File "/opt/wdl/.virtualenv/lib/python2.7/site-packages/OpenSSL/_util.py", line 7, in <module>
[Tue Sep 08 12:07:05.810498 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]     binding = Binding()
[Tue Sep 08 12:07:05.810515 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]   File "/opt/wdl/.virtualenv/lib/python2.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 63, in __init__
[Tue Sep 08 12:07:05.810614 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]     self._ensure_ffi_initialized()
[Tue Sep 08 12:07:05.810638 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]   File "/opt/wdl/.virtualenv/lib/python2.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 95, in _ensure_ffi_initialized
[Tue Sep 08 12:07:05.810665 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]     cls._register_osrandom_engine()
[Tue Sep 08 12:07:05.810680 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]   File "/opt/wdl/.virtualenv/lib/python2.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 70, in _register_osrandom_engine
[Tue Sep 08 12:07:05.810705 2015] [:error] [pid 25018:tid 140561562785536] [remote 127.0.0.1:42533]     raise RuntimeError("osrandom engine already registered")

@sigmavirus24
Copy link
Contributor

@acdha that's roughly the same environment I'm seeing this in. Can you share your simple app? It may help with reproduction/debugging. I'd like to try with this PR to see if I can still reproduce this.

@acdha
Copy link

acdha commented Sep 8, 2015

Let me see how simple I can make it – my app isn't simple but I could reproduce simply by tossing import requests at the top of the WSGI file.

@acdha
Copy link

acdha commented Sep 8, 2015

Yeah, even this will trigger it with the import requests line uncommented:

#!/usr/bin/env python
# encoding: utf-8

# import requests

def application(environ, start_response):
    status = '200 OK'
    output = 'Hello World!'

    response_headers = [('Content-type', 'text/plain'),
                        ('Content-Length', str(len(output)))]
    start_response(status, response_headers)

    return [output]

Relevant Apache config:

WSGIDaemonProcess example user=example-site group=www-data processes=8 threads=8 maximum-requests=20000 shutdown-timeout=120 deadlock-timeout=60 display-name=example home=/opt/example python-path=/opt/example/.virtualenv/lib/python2.7/site-packages
WSGIImportScript /opt/example/example_wsgi.py application-group=example process-group=example

WSGILazyInitialization On
WSGIRestrictEmbedded On

@reaperhulk
Copy link
Member

This is related to the subinterpreters used by mod_wsgi. We're tracking this in #2299.

@angelblue05
Copy link

One of our Emby users is experiencing this issue (log is attached in the post): http://emby.media/community/index.php?/topic/25735-emby-for-kodi-broken-no-sync/

Is there anything we can do on our side to prevent this error from happening? So far, it seems to affect only one user.

Emby add-on for Kodi: https://github.com/MediaBrowser/Emby.Kodi
Requests is imported here: https://github.com/MediaBrowser/Emby.Kodi/blob/master/resources/lib/DownloadUtils.py

@jerohnimaux
Copy link

The issue might come from Kodi and not Emby, I experienced similar errors (osrandom engine already registered) in Kodi with addons requesting connection to websites (trakt.tv). Note that I'm a noob user, not a developper.

All the info are in the same emby.media thread :
http://emby.media/community/index.php?/topic/25735-emby-for-kodi-broken-no-sync/

@toabctl
Copy link

toabctl commented Sep 29, 2015

For the record: #2293 fixed this problem for me (see also #2299 (comment) )

@alex
Copy link
Member

alex commented Oct 25, 2015

This is merged!

@alex alex closed this as completed Oct 25, 2015
@micahg
Copy link

micahg commented Nov 10, 2015

Hi,

Is there actually a fix for this in Kodi? On Ubuntu 15.10 the python libs are sufficiently different that the fix on 2293 doesn't apply cleanly (plus I'd be updating system .py files).

@reaperhulk
Copy link
Member

@micahg try installing cryptography 1.1 from pip, which contains this change. To do this you may need to uninstall python-cryptography from apt though.

@micahg
Copy link

micahg commented Nov 11, 2015

@reaperhulk that worked, tyvm.

@TheRealTachyon
Copy link

This nonsense happened to me today in Kodi running on Linux. It killed the YouTube plugin dead so it wouldn't even start up.

I had to resort to deleting python cryptography and python openssl on the underlying Linux system that hosts my Kodi box using the package manager.
Kodi 17.0
OpenSUSE LEAP 42.1

You'd think after the number of years this issue has existed someone would have fixed it.

@reaperhulk
Copy link
Member

@TheRealTachyon You might think that someone would have fixed it...and in this case you would be right. This was an extremely complex issue to fix due to the interplay between cryptography, applications that embed a Python interpreter, and the way dynamically loaded C libraries work. However, due to the hard work of several people a major (and completely transparent!) change was landed in version 1.7 to entirely resolve this problem (#3229).

Of course, the nature of linux distributions is that they don't pick up the latest versions of things instantaneously. Instead, users who choose to utilize packages in this fashion will typically be locked into whatever version their distribution ships until they upgrade to the next major release. In some cases backports may occur, but person power in open source projects is limited and priority is not given to such endeavors.

We're glad you've resolved the issue to your satisfaction, and it is unfortunate that it occurred in the first place, but in general you might want to consider a few things in the future before commenting on GitHub issues:

  • Time is limited and these projects are largely volunteer driven
  • Not all problems have easy solutions
  • Posting rude and dismissive comments on closed issues doesn't endear you to anyone

@TheRealTachyon
Copy link

@reaperhulk

To address your points.

  • I know how OpenSource works
  • Welcome to life
  • I'm not here to make friends with the self appointed GIThub police. I'm reporting an issue that was potentially closed prematurely.

I did quite a bit of research into this issue before deciding to post here and frankly a lot of what I saw involved a lot of passing the buck as to where the source of the issue lay. Congrats to the team here for taking it on and resolving it, but....

The fact that this was so easily re-triggered clearly indicates that this issue's solution has either not been fully fixed, or has not permeated downstream to everyone using it. And BTW, I do not run auto-updates on my systems, so there was not an OS update between yesterday when it worked and this morning when it didn't. This means that the error happened due to a client side change since only Kodi and it's add-ons auto-update on this system. That implies to me that the issue is not fully fixed. If it were fixed properly, you would not be able to break it again in normal usage.
In other words, if it was fixed right the error should have been different. Unknown call, illegal instruction, etc. or similar. You shouldn't still be able to break it using the old method.

@reaperhulk
Copy link
Member

You do not have a recent enough version of cryptography. cryptography 1.7 (released in December 2016) contains the fix. Without the latest version you can potentially see this problem. Your "quite a bit" of research apparently did not extend to actually trying to see if newer versions than the one conveniently provided by your linux distribution (1.0 for the record. A version released in August of 2015) might resolve the issue.

This is not a productive avenue of discussion so I'll go ahead and lock this conversation. If you see this problem with the latest version please file a new issue.

@pyca pyca locked and limited conversation to collaborators Mar 20, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Development

No branches or pull requests

10 participants