multi delta #756

totaam · 2014-12-04T06:33:17Z

Issue migrated from trac ticket # 756

component: core | priority: major | resolution: fixed

2014-12-04 06:33:17: totaam created the issue

At the moment, we only use delta compression for non video screen updates using the last suitable region we had. Multiple updates of differing sizes means we will constantly change the delta buffer, and never match anything.

We could let the client tell the server how many buffers it will accept, and of what size.
Then the server can keep more delta regions in memory, keyed using their dimensions, and is more likely to make use of them.

This will help with things like unnecessary repaints (#670).
The only difficulty will be with UDP and dropped packets #639: we will need a way of requesting a reset / refresh when we are missing a reference region.

totaam · 2014-12-25T16:50:30Z

2014-12-25 16:50:30: totaam commented

Implemented in r8285, see commit message for details.

Remaining tasks:

we should re-stride the pixel data, at the moment we copy 2000*height pixels... which can be expensive

try to prefer a delta encoding if the damage area matches one of the delta regions

tell the client to free its delta bucket when we do it server-side

totaam · 2014-12-28T06:33:56Z

2014-12-28 06:33:56: antoine uploaded file `delta-free-buckets.patch` (6.0 KiB)

frees buckets as needed - not sure this is worth doing

totaam · 2014-12-28T10:32:04Z

2014-12-28 10:32:04: antoine commented

More improvements in r8315, r8311, r8309: we now re-stride the data, and directly into the target buffer in Cython, so now much faster.
I don't think we want to apply the delta-free patch above: too much complexity and extra code in the hot path, all for small memory savings. If we want to constrain the client memory usage, a better approach would be to let the client specify its own limit for the size of delta buffers. That's on top of being able to choose the number of delta buffers already via:
XPRA_DELTA_BUCKETS=10 xpra attach ..
Ready for testing, #760 may be helpful here to visualize where we use delta regions (and see if we miss any).

totaam · 2015-01-23T21:00:26Z

2015-01-23 21:00:26: afarr commented

Seems to be working as expected (aside from the windows client opengl paint box issue mentioned in #760).

I would note that setting XPRA_DELTA_BUCKETS=11 does seem to lead to increased blurriness, even on a small window on a 1080p display.

totaam · 2015-01-24T01:54:11Z

2015-01-24 01:54:11: totaam commented

I would note that setting XPRA_DELTA_BUCKETS=11 does seem to lead to increased blurriness, even on a small window on a 1080p display.
[[BR]]
It is not meant to.

I should explain: delta means that we substract the previous contents of a region from its current contents before compression.
So if you end up showing more or less the same contents, you end up with something empty as delta - which compresses very well.
Video compressors already do this by default (that's why they compress so well - and also why the size of the video cannot change without a full encoder re-initialization), we use delta compression on lossless picture encodings (rgb24/rgb32+lz4 and png) to benefit from the same trick.

totaam · 2015-02-21T01:56:21Z

2015-02-21 01:56:21: afarr commented

Testing some more with 0.15.0 8647 win32 client against 0.15.0 r8661 fedora 20 server... it looks like the blurriness when scrolling is resolving itself promptly (perhaps previous tests hit some extra latency... or maybe I was just using a monitor that periodically seems to actually be becoming blurry itself... not certain).

However... testing this I'm suddenly seeing that unicode error from #786 again (in this case 2015-02-20 17:31:10,731 unexpected data type <type 'unicode'> in logging packet: u'using audio codec: MPEG 1 Audio, Layer 3 (MP3)'), and I'm also seeing another message that I'm also seeing in a number of other places (not sure it's related specifically to this, thought it was a webkit browser related error until just now): 2015-02-20 17:25:13,779 failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow'.

In case it is liable to make matters clear, launched the server with the following:
[jimador@zapopan ~]$  XPRA_OPENGL_PAINT_BOX=1 dbus-launch xpra --no-daemon --bind-tcp=0.0.0.0:1201 --start-child=xterm --start-child=xterm start :23 --remote-logging=on -d client
And the client with the following:
C:\Program Files (x86)\Xpra>SET XPRA_OPENGL_PAINT_BOX=1

C:\Program Files (x86)\Xpra>xpra_cmd.exe attach tcp:10.0.32.53:1201
2015-02-20 17:05:38,502 xpra client version 0.15.0
2015-02-20 17:05:38,986 OpenGL_accelerate module loaded
2015-02-20 17:05:38,986 Using accelerated ArrayDatatype
2015-02-20 17:05:39,095 detected keyboard: layout=us
2015-02-20 17:05:39,095 desktop size is 5120x2160 with 1 screen(s):
2015-02-20 17:05:39,095   '1\WinSta-Default' (1354x571 mm - DPI: 96x96) workarea
: 5120x2120
2015-02-20 17:05:39,095     DISPLAY1 3840x2160 at 1280x0 (621x341 mm - DPI: 157x
160) workarea: 3840x2120
2015-02-20 17:05:39,095     DISPLAY2 1280x720 (597x336 mm - DPI: 54x54) workarea
: 1280x680
2015-02-20 17:05:39,408 enabled remote logging, see server log file for output
And, with the client also logging to the server, the server initialization output this initially:
2015-02-20 17:05:47,941 Python/Gtk2 Microsoft Windows 8 client version 0.15.0 connected from 'hasenpfeffer' as 'hassenpfeffer' ('hassenpfeffer')
2015-02-20 17:05:47,942 using h264 as primary encoding, also available: vp8, png, png/P, png/L, webp, rgb24, jpeg, rgb32
2015-02-20 17:05:47,948 client root window size is 5120x2160 with 1 displays:
2015-02-20 17:05:47,949   '1\WinSta-Default' (1354x571 mm - DPI: 96x96) workarea: 5120x2120
2015-02-20 17:05:47,949     DISPLAY1 3840x2160 at 1280x0 (621x341 mm - DPI: 157x160) workarea: 3840x2120
2015-02-20 17:05:47,949     DISPLAY2 1280x720 (597x336 mm - DPI: 54x54) workarea: 1280x680
2015-02-20 17:05:48,098 server virtual display now set to 5120x2560 (best match for 5120x2160)
2015-02-20 17:05:48,100 setting key repeat rate from client: 500ms delay / 33ms interval
2015-02-20 17:05:48,101 setting keyboard layout to 'us'
2015-02-20 17:05:48,234 server: Linux Fedora 20 Heisenbug, Xpra version 0.15.0 ([r8661](../commit/90aa2ba0a48647784a4a0cae560a619cf266b64b))
2015-02-20 17:05:48,308 DPI set to 96 x 96
2015-02-20 17:05:48,310 sent updated screen size to 1 clients: 5120x256I 0 (max 8192x4096)
2015-02-20 17:05:48,313 Attached to tcp:10.0.32.53:1201 (press Control-C to detach)
2015-02-20 17:05:48,333 failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow'
2015-02-20 17:05:48,347 using Pulseaudio device 'Monitor of Null Output'
2015-02-20 17:05:48,361 starting sound stream capture using Pulseaudio source
2015-02-20 17:05:48,362 failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow'
2015-02-20 17:05:50,472 codec: MPEG-1 Layer 3 (MP3)
2015-02-20 17:05:50,494 using audio codec: MPEG 1 Audio, Layer 3 (MP3)
2015-02-20 17:05:50,495 unexpected data type <type 'unicode'> in logging packet: u'using audio codec: MPEG 1 Audio, Layer 3 (MP3)'
Using firefox to test for delta processing, I'm periodically (mostly when opening a new tab, but not exactly correspondingly) seeing the failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow' error repeating.

Not sure if it is a combination of the global variable settings that is triggering these messages or if it is just a matter of my now aging test versions... any insights where to start (I mention here just before we close this delta ticket, because the combination of the global variables and the client debug forwarding seems to have triggered those messages which I was no longer seeing when testing #786 without all the other variables and log forwarding in place)... before I try going through the bits and pieces randomly until I find a combination-culprit?

totaam · 2015-02-21T06:02:57Z

2015-02-21 06:02:57: antoine commented

testing this I'm suddenly seeing that unicode error from #786 again
[[BR]]
Tracked in #786.

failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow'
[[BR]]
This is tracked in #799. If you can reproduce this problem with one of my later builds, please add to that ticket.

As for actual delta compression, I think we can close this ticket?

totaam · 2015-02-23T20:13:22Z

2015-02-23 20:13:22: afarr commented

Testing again for #799, I couldn't reproduce the error(s).

Agreed regarding the actual delta compression.

Closing.

totaam · 2015-04-07T10:20:18Z

2015-04-07 10:20:18: antoine commented

Whilst testing vp9 (#832) and having decoder problems (or maybe encoder problems even..), I think I've hit a bug related to multi-delta:

2015-04-05 19:49:05,883 paint_with_video_decoder: \
    wid=1, vp9 decompression error on 44 bytes of picture data for 499x316 pixels using vpx.Decoder(vp9), \
    options={'speed': 83, 'frame': 2, 'quality': 67, 'csc': 'YUV444P', 'encoding': 'vp9'}
2015-04-05 19:49:06,251 draw error
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/xpra/client/ui_client_base.py", line 1962, in _do_draw
    window.draw_region(x, y, width, height, coding, data, rowstride, packet_sequence, options, [record_decode_time])
  File "/usr/lib64/python2.7/site-packages/xpra/client/client_window_base.py", line 423, in draw_region
    self._backing.draw_region(x, y, width, height, coding, img_data, rowstride, options, callbacks)
  File "/usr/lib64/python2.7/site-packages/xpra/client/window_backing_base.py", line 472, in draw_region
    self.paint_with_video_decoder(VIDEO_DECODERS.get(coding), coding, img_data, x, y, width, height, options, callbacks)
  File "/usr/lib64/python2.7/site-packages/xpra/client/window_backing_base.py", line 376, in paint_with_video_decoder
    img = self._video_decoder.decompress_image(img_data, options)
  File "xpra/codecs/vpx/decoder.pyx", line 300, in xpra.codecs.vpx.decoder.Decoder.decompress_image (xpra/codecs/vpx/decoder.c:4578)
  File "/usr/lib64/python2.7/site-packages/xpra/codecs/codec_constants.py", line 61, in get_subsampling_divs
    raise Exception("invalid pixel format: %s" % pixel_format)
Exception: invalid pixel format: bucket

totaam · 2015-04-07T13:57:35Z

comment 8 can be ignored, the problem was a dangling pointer caused by a bug in the vpx decoder, fixed in r8940.

totaam · 2015-04-16T18:23:19Z

r9028 adds a new debug flag for delta compression: -d delta.

totaam · 2015-04-19T12:35:16Z

r9063 ensures that we expire all delta buckets after 20 hits to prevent xor decay where the bucket just contains unhelpful noise.
The number of hits can be configured using:

XPRA_MAX_DELTA_HITS=20 xpra start ...

totaam · 2015-05-19T09:30:41Z

This caused a bug: #861. (fixed in r9449)

totaam · 2019-04-25T07:28:12Z

Related: #2248 (more aggressive use of scroll encoding)

totaam · 2021-05-22T09:22:02Z

For the record, this feature has been removed in 4.1

totaam closed this as completed Apr 7, 2015

totaam mentioned this issue Jan 23, 2023

Subpar performance when running gvim or emacs #670

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

multi delta #756

multi delta #756

totaam commented Dec 4, 2014

totaam commented Dec 25, 2014

totaam commented Dec 28, 2014

totaam commented Dec 28, 2014

totaam commented Jan 23, 2015

totaam commented Jan 24, 2015

totaam commented Feb 21, 2015

totaam commented Feb 21, 2015

totaam commented Feb 23, 2015

totaam commented Apr 7, 2015

totaam commented Apr 7, 2015 •

edited

Loading

totaam commented Apr 16, 2015 •

edited

Loading

totaam commented Apr 19, 2015 •

edited

Loading

totaam commented May 19, 2015 •

edited

Loading

totaam commented Apr 25, 2019 •

edited

Loading

totaam commented May 22, 2021

multi delta #756

multi delta #756

Comments

totaam commented Dec 4, 2014

2014-12-04 06:33:17: totaam created the issue

totaam commented Dec 25, 2014

2014-12-25 16:50:30: totaam commented

totaam commented Dec 28, 2014

2014-12-28 06:33:56: antoine uploaded file delta-free-buckets.patch (6.0 KiB)

totaam commented Dec 28, 2014

2014-12-28 10:32:04: antoine commented

totaam commented Jan 23, 2015

2015-01-23 21:00:26: afarr commented

totaam commented Jan 24, 2015

2015-01-24 01:54:11: totaam commented

totaam commented Feb 21, 2015

2015-02-21 01:56:21: afarr commented

totaam commented Feb 21, 2015

2015-02-21 06:02:57: antoine commented

totaam commented Feb 23, 2015

2015-02-23 20:13:22: afarr commented

totaam commented Apr 7, 2015

2015-04-07 10:20:18: antoine commented

totaam commented Apr 7, 2015 • edited Loading

totaam commented Apr 16, 2015 • edited Loading

totaam commented Apr 19, 2015 • edited Loading

totaam commented May 19, 2015 • edited Loading

totaam commented Apr 25, 2019 • edited Loading

totaam commented May 22, 2021

2014-12-28 06:33:56: antoine uploaded file `delta-free-buckets.patch` (6.0 KiB)

totaam commented Apr 7, 2015 •

edited

Loading

totaam commented Apr 16, 2015 •

edited

Loading

totaam commented Apr 19, 2015 •

edited

Loading

totaam commented May 19, 2015 •

edited

Loading

totaam commented Apr 25, 2019 •

edited

Loading