python3

[totaam]

Tracker ticket for everything related to python3.

Split from #640 / #90.
See also:

• packaging: split client and server builds #1253 split packages, move to python2 package naming #1258 "python2" package naming
• switch to gstreamer 1.x on win32 #1041 python3 for gstreamer on win32 (no longer used since building all the dependencies from source on win32 #678)
• use python3 for the sound subprocess #1386 python3 for gstreamer sound process (other platforms)
• python3 gtk3 client feature parity #1717 python3 gtk3 client
• python3 opengl client #1569 python3 opengl client
• package python3 client on win32 #1574 python3 packaging for win32
• python3 packaging for macos #1575 python3 for macos
• GTK3 clipboard support #1589 GTK3 clipboard support
• py3k server support #853 python3 server

[totaam]

Subcommands with python3:

• attach: needs opengl: python3 opengl client #1569
• detach: OK
• screenshot: fixed in r16267
• info: OK
• control: OK
• print: OK
• version: fixed in r16269
• showconfig: OK
• list-mdns: fixed in r16274
• mdns-gui: fixed in r16275
• shadow: subcommand now hidden in r16277 (needs work)

Server bits are moved to their own ticket: #1571.

[totaam]

nvenc fixes and dependency updates: r16399 + r16400 + r16401 (see #1550#comment:14)

[totaam]

Part of r16607 doesn't work on python3:

Warning: unable to override socket object
 SSL peek support will not be available
 module 'socket' has no attribute '_socketobject'

See #1504 for details.

[totaam]

Lots of updates in r17015 + r17017. The shadow server is now usable with python3, only needs a few minor fixes:

• encoding options aren't set properly (csc warnings, no video encoders)
• clipboard is disabled
• ssl socket peek wrapper
• errors during cleanup caused by py3k needless breakage ie: RuntimeError: dictionary changed size during iteration in cancel_damage : for sequence in self.statistics.encoding_pending.keys():

[totaam]

More updates and fixes:

• r17025: clipboard
• r17026 + r17029: keyboard
• r17027: shadow server image encoding
• r17028: fix ssl, gdk Rectangle
• r17091: rfb and websockets
  Python3 shadow servers are now totally usable and fast. Not much left to fix here:
• cursor icons
• SIGINT (general problem with GTK3)

TODO:

• the client still needs graphs in session info, maybe we should allow export to CSV or include more context information in the graph (see Export session info data #1705#comment:1)
• websockify needs patching (removing ForkingMixIn)
• server core needs a different workaround for non-blocking sockets
• problem with mouse cursors?
• XSETTINGS / root props, see r6384 (needs more GTK glue)
• transparency may work without opengl?

[totaam]

r17098 is wrong, the bug is still present in all GTK3 versions! Ctrl+C does not exit gtk app - absolutely amazing that such a fundamental issue has not been fixed in so many years.
It does mean that we may be scheduling timers too often, which is why we exit the main loop and catch the signal. Worth looking into.

[totaam]

Lots more fixes: r17498, r17507, r17509.

• bug report tool needs a fix for the clipboard copy (probably easy).
• the systray sort of works with gnome-shell if you install topicons plus, but not with GTK3... where the geometry comes up as 1x1, and even forcing it does not help

[totaam]

Found a blocker bug with "de" keyboard layout and win32 clients: I can get a client crash just by pressing the first key to the right of the left shift key!

ERROR:../../pygobject-3.26.1/gi/pygi-argument.c:1004:_pygi_argument_to_object: code should not be reached

Why give a helpful warning when you can just crash the whole application, thanks GTK! sigh.

[totaam]

Updates:

• r17582: fixup rencode for python 3.6 import warnings
• r17579 + r17581: lots of deprecation warnings fixed

Some import warnings were completely hidden behind a generic message until I used this badly documented gem:

PYTHONWARNINGS='error::ImportWarning' python3 /usr/bin/xpra attach

The remaining warnings are:

• pyopengl: adding from __future__ import absolute_import to the pyx files doesn't work, it breaks somewhere else trying to cimport withoud pxd files (how is that supposed to work!?)
• Gdk.Screen: most functions are deprecated and we're supposed to Use per-monitor information - could be done
• Gtk.Menu.set_title is deprecated - no replacement... probably not a big issue
• Gtk.ImageMenuItem: GTK developers are hellbent on trying to force everyone to not use icons in menu items - the most bone headed move yet (anyone who has ever had to use a computer setup to use a language they are unable to read can testify here)
• StatusIcon: everything is deprecated because gnome decided to break the systray one more time (a tie with the icon thing for being the worst decision)

[totaam]

The signal handler has finally been fixed, it only took GTK ~7 years to fix a basic unix feature. Considering that other platforms (win32 and macos) are barely supported... wow.
The changesets needed to pygobject are:

• Make Python OS signal handlers run when an event loop is idling
• Install a default SIGINT handler for functions which start an event loop

If these changes apply cleanly to the pygobject version in MSYS2, let's patch it.

[totaam]

Both those patches to GTK are from a very active MSYS2 developer so hopefully there'll be official packages sometime soon.

[totaam]

Hit a concurrency error with python3, fixed in r19101. Problem is that there are likely many more like this one because they have changed the semantics of values(), keys() and items() on dictionaries. Major pain.
Wherever we fix this by using a temporary list, we unnecessarily penalise python2. A version specific wrapper function would be better, but would still cost an extra function call.

[totaam]

rpmbuild moans about those:

*** WARNING: mangling shebang in /usr/lib/cups/backend/xpraforwarder from #!/usr/bin/env python to #!/usr/bin/python2. This will become an ERROR, fix it manually!
*** WARNING: mangling shebang in /usr/libexec/xpra/auth_dialog from #!/usr/bin/env python to #!/usr/bin/python2. This will become an ERROR, fix it manually!
*** WARNING: mangling shebang in /usr/libexec/xpra/xdg-open from #!/usr/bin/env python to #!/usr/bin/python2. This will become an ERROR, fix it manually!

[totaam]

Lots of GTK3 issues trying to fix deprecation warnings shown with PYTHONWARNINGS=all, some fixes in r19697, r19698, r19699, r19700, r19701.

Gtk.StatusIcon: the whole class is deprecated, ie: there is no direct replacement for this function. Oh great, good thing we're only using this on X11

Also found a crasher bug: #1886.

[totaam]

Regarding comment:19, those shebangs get mangled by rpmbuild (see fedora packaging committe: mangling shebangs and #1891), using the totally undocumented __brp_mangle_shebangs_exclude_from (as per zfs commit : Exclude python scripts from RPM shebang check) prevents that: r19812.

[totaam]

2018-08-03 14:31:14: antoine uploaded file websockify-0.8-py37.tar.xz (15.6 KiB)

websockify 0.8 modified to run on python 3.7

[totaam]

See also: GTK4 #1977

A good summary of how bad the situation is with python3 strings vs bytes: The Python Unicode Mess

[totaam]

See also #2121.
SSL socket peek support is now properly fixed: r21877.

[totaam]

Everything seems to work fine, including the proxy server (#2185).

The few remaining bits are things like #2195, which have their own ticket.

[totaam]

2019-06-05 20:57:53: MayeulC commented

---

Replying to [comment:22 Antoine Martin]:

See also: GTK4 #1977

A good summary of how bad the situation is with python3 strings vs bytes: The Python Unicode Mess

Another good read is https://drewdevault.com/2017/01/13/The-problem-with-Python-3.html

The situation really isn't as bad as it is being painted there. Thank you for porting xpra to python3, hopefully I can now remove (again) python2 from my (wayland, sigh) system once the change trickles to the Arch packages :)

[totaam]

Another good read is ​[https://drewdevault.com/2017/01/13/The-problem-with-Python-3.html]

Awful. The guy just doesn't get it.
Please count the hundreds of pointless changesets we've had to make for python3 (still ongoing) and tell me again why this is "better".