Mintty and use case examples

[mintty]

Maybe you'd like to add mintty to your supporting terminal list on the libsixel project page.
I've added a gnuplot screenshot to the mintty home page http://mintty.github.io/ and I'd like to mention some more examples of use cases, but I'm having some problems:

• The gnuplot command shown in the screenshot produces a different image on another machine; weird.
• img2sixel sometimes works (cygwin64 on one machine), sometimes it doesn't (cygwin32 on the same machine, cygwin64 on another machine), sometimes it crashes, e.g. with this image:
  
• FFmpeg-SIXEL does not build for me (after installing libsixel and the libquvi-devel and nasm packages); sometimes configure fails at pkg-config/quvi, sometimes make fails: Makefile:185: /tests/Makefile: No such file or directory
• I tried the libsixel examples/drawing program in xterm but it just shows a white panel; nothing happens when trying to interact (and yes, it's xterm compile-enabled with DEC locator mode, confirmed manually). If that worked, I would consider to add DEC locator mode to mintty for pixel-coordinate interaction.
• Couldn't find out how to build and test opengl...
• Trying bitbucket.org/arakiken/w3m/: configure: error: cannot guess build type; you must specify one
• The mandel5.c example program does not produce working sixel output.

Thanks for further info and help.

[saitoha]

The gnuplot command shown in the screenshot produces a different image on another machine; weird.

I can't reproduce that on my environment.
Could you show a screenshot on another machine?

img2sixel sometimes works (cygwin64 on one machine), sometimes it doesn't (cygwin32 on the same machine, cygwin64 on another machine), sometimes it crashes, e.g. with this image:

Reproduced.

$ gdb img2sixel img2sixel.exe.core

...

#0  sixel_allocator_free (allocator=0x38, p=0x20163928) at allocator.c:192
192         assert(allocator->fn_free);
[Current thread is 1 (Thread 0x1)]
(gdb) up
#1  0x64a96cb8 in load_gif (buffer=0x200303a8 "GIF89a.\002\017", size=size@entry=264, bgcolor=bgcolor@entry=0x0, reqcolors=reqcolors@entry=256, fuse_palette=fuse_palette@entry=1,
    fstatic=fstatic@entry=0, loop_control=loop_control@entry=0, fn_load=fn_load@entry=0x64a98b10 <load_image_callback>, context=context@entry=0x20030308, allocator=0x20030230)
    at fromgif.c:646
646         sixel_allocator_free(frame->allocator, g.out);
(gdb) l
641             }
642         }
643
644     end:
645         sixel_frame_unref(frame);
646         sixel_allocator_free(frame->allocator, g.out);
647
648         return status;
649     }
650
(gdb)

Oops, sixel_frame_unref(frame) is called before sixel_allocator_free(frame->allocator, g.out)...
I'll fix it. thanks!

FFmpeg-SIXEL does not build for me (after installing libsixel and the libquvi-devel and nasm packages); sometimes configure fails at pkg-config/quvi, sometimes make fails: Makefile:185: /tests/Makefile: No such file or directory

I also could not build it with cygwin's libquvi-devel package. Its API seems to be changed entirely now. So I built it without --enable-libquvi.

$ ./configure --enable-libsixel
$ make
$ make install

config.log: config.log.ffmpeg.txt

I tried the libsixel examples/drawing program in xterm but it just shows a white panel; nothing happens when trying to interact (and yes, it's xterm compile-enabled with DEC locator mode, confirmed manually). If that worked, I would consider to add DEC locator mode to mintty for pixel-coordinate interaction.

Please try the following patch:

diff --git a/examples/drawing/main.c b/examples/drawing/main.c
index cc62914..ec6b462 100644
--- a/examples/drawing/main.c
+++ b/examples/drawing/main.c
@@ -228,7 +228,7 @@ int main(int argc, char **argv)
     int state;
     int offset;

-    printf("\033[?8452h" "\033[1;1'z");
+    printf("\033[?1;3;256S" "\033[?8452h" "\033[1;1'z");
     (void) tty_raw(&old_termios);

     if (canvas_init(&c, 640, 480) != 0)

We can implement pixel coordinate handling more efficiently by using "filter rectangle" feature (DECEFR), but Xterm's one seems to broken. So now drawing example use only DECRQLP simply. Vim's ttymouse=dec also chooses same strategy.
On the other hand, xserver-SIXEL and SDL1.2-SIXEL requires "filter rectangle" feature.

I'm now working for it in dec-locator branch. But filter rectangle is not implemented yet.
https://github.com/saitoha/mintty-sixel/tree/dec-locator

Couldn't find out how to build and test opengl...

Cygwin package libOSMesa-devel is required.

$ apt-cyg install libOSMesa-devel
$ ./configure && make
$ ./demo.exe

config.log: config.log.opengl.txt

Trying bitbucket.org/arakiken/w3m/: configure: error: cannot guess build type; you must specify one

Now I'm using Debian's w3m. It's best-maintained w3m as far as I know, which also includes Arakiken's SIXEL patch.

$ ./configure --enable-image --with-imagelib=
$ make
$ make install

config.log: config.log.w3m.txt

w3m SIXEL integration enables with -sixel option. It requires img2sixel. So before try it, I have to fix above libsixel's segfault bug.

The mandel5.c example program does not produce working sixel output.

It works for me. mandel5 requires 5 parameters ( ). Did you specify them properly?

[saitoha]

I found a tweet that says doom works on mintty.
https://twitter.com/rattcv/status/775213402130046977

[mintty]

Haha! And with Bash on Windows... This is getting cool.
Not sure, though, whether it's really beneficial to run highly interactive GUI stuff in SIXEL mode or whether running this, qemu and the like is just fun...

[mintty]

Drawing now works in xterm, thanks, but not in your dec-locator branch.

[mintty]

Status:

• gnuplot: fine example (the confusion mentioned above was caused by a typo).
• img2sixel: works;
• w3m: works, but limited user experience; image display is delayed until all images are loaded which is slow.
• drawing: works now in xterm, but not in mintty (your dec-locator branch).
• opengl example: works, but it's not too impressive due to its coarse image quality.
• mandel: works, but is picky about parameter values, so maybe not a good demo.
• ffmpeg: finally compiles but still does not work:
  ./ffmpeg -f sixel -pix_fmt rgb24 -reqcolors 128 -s 800x450 - -i 'https://youtu.be/El2mr5aS8y0'
  http://youtu.be/El2mr5aS8y0: Invalid data found when processing input

In effect, there is only one seemless example to promote the feature, but a good one at least.

[therealkenc]

The ffmpeg Ballmer commandline plays without error for me with mintty on WSL, for what it's worth. I configured with --enable-libsixel --enable-libquvi --enable-openssl.

I do have a question regarding the "coarse image quality" though, which you see even with img2sixel if you draw a big colorwheel like this one. I understand from mintty issue 431 that raw RGB is passed all the way up the rendering stack, but the colors seem to be getting quantized before it gets a chance to draw them. This is apparent in my Xsixel screencap here as well, in which the xterm window title bar (which gets drawn by xfwm4) gets quantized and dithered. You see it with the Ballmer video as well; I assume the -reqcolors 128 option having something to do with it.

[saitoha]

@therealkenc Unfortunately, SIXEL is palette-based format, which is not suitable to represent true color... (#44)
The patch at mintty/mintty#593 (comment) makes mintty enable to display up to 1023 color paletted image.
@mintty above patch also may fix drawing example.

[saitoha]

w3m: works, but limited user experience; image display is delayed until all images are loaded which is slow.

This problem may be caused by cygwin's expensive fork() implementation. Does it reproduce on WSL?

[saitoha]

I've merged above patch into dec-locator branch.

opengl example: works, but it's not too impressive due to its coarse image quality.

This example might be better:
https://github.com/saitoha/NeHe-SIXEL

[therealkenc]

Thanks. Apologies for my ignorance about Sixel being palette based only.

The libgl demo starts up instantly on my computer with WSL. though there appears to be a memory leak in either mintty or wslbridge - this is mintty consuming 1.4GB of RAM.

I'll try out your recent patches. I should mention the mouse didn't work entirely for me in Xsixel last week on WSL. It would draw ascii characters when dragging the mouse with left click (making dragging windows impossible), but the cursor moves okay without click.

[therealkenc]

I confirmed the leak with Cygwin. It's easy to reproduce. If you run img2sixel egret.jpg just once, you'll see the commit size for mintty go up by around 1kb. If you run the command in a bash for loop, you'll see it steadily creep up. This happens on both the saitoha/mintty-sixel dec-locator branch and on the mintty/mintty master.

Silly question: How do I take advantage of the 1023 palette? I'm on the dec-locator branch and the drawing demo works, but I can't see any difference in fidelity with img2sixel.

[mintty]

Does the leak happen with the current release version (2.6.1)?
About the color fix, I will integrate it soon into the repository.

[therealkenc]

Sorry I should have thought to check that upfront. 2.6.1 that ships with Cygwin is fine. Wsltty shipped off master on Sept 19th so that explains why I'm seeing the leak with WSL as well.

[saitoha]

Sorry there is a leak problem in my recent patch(saitoha/mintty-sixel@7b6bf81).
I confirmed current mintty master does not have this problem.

[therealkenc]

Weird. Mintty off master is indeed fine, but I just double-checked wsltty 0.6.0 prebuilt leaks. No idea why. So apologies for the confusion. I saw wsltty pulls off master and just assumed I was seeing the same thing.

[mintty]

Actually, there is a memory leak already in release 2.6.1. If I output a Sixel image and clear the scrollback buffer repeatedly (reset), it uses some memory more every time.
Maybe this should be continued in a mintty or mintty-sixel issue rather than here...

[saitoha]

Leak problems in master are fixed by mintty/mintty#595
Another leak problem in dec-locator branch is fixed by saitoha/mintty-sixel@e92d9a5

[mintty]

Hi, just to sort this out; there are two open patches now:

• DEC Locator
• Color registers handling

Are you still working on any of them? How and when should I integrate them? DEC Locator first, or both together...

[saitoha]

Color registers handling is in separated branch:
mintty/mintty@master...saitoha:fix-byte-overflow-in-sixel-decoder
squashed patch: 0001-Fix-byte-overflow-problem-in-SIXEL-decoder-mintty-59.patch.txt

DEC Locator is now in progress, I think it is not ready to be released, because the filter rectangle feature is not implemented now.

[saitoha]

@therealkenc

How do I take advantage of the 1023 palette? I'm on the dec-locator branch and the drawing demo works, but I can't see any difference in fidelity with img2sixel.

Now there is yet no SIXEL encoder which can use 1023 color registers...

[mintty]

Now there is yet no SIXEL encoder which can use 1023 color registers...

So what does img2sixel --high-color do?

[saitoha]

high-color sixel is the technique of recycling 256 color register dynamically.
It depends on the method of sixel rasterization. As far as I know, mlterm and yaft can display it correctly. Current mintty can not do. I don't know whether we should implement this feature.

[saitoha]

I found such a thing.
https://groups.google.com/d/msg/fricas-devel/GJfUAyH2S2I/5kzv5k64AQAJ

[tkelman]

Did the DEC locator feature ever make further progress towards getting incorporated into mintty?

[mintty]

It's actually not yet even incorporated into mintty (main repository), just saitoha's DEC Locator branch, because I had the impression that @saitoha still wanted to implement Filter Rectangle mode.
I could also work on this myself but a convincing use case would help to motivate that... The only program I'm aware of now that actually uses this is the drawing example app.

[tkelman]

Yeah, makes sense. Hopefully if I show off what mintty already supports it'll give some Julia users a reason to get excited about sixels as a low-dependency image display solution, they might do some fun stuff. See https://github.com/KristofferC/Crayons.jl and https://github.com/evizero/UnicodePlots.jl for some existing examples, just using already-common escape codes and unicode characters.

I am now really curious how out-of-the-box support for sixel display is on default Mac and Linux terminals. Most people developing open-source packages tend to use those platforms, while lots of our users are on Windows so will benefit from mintty once I get it incorporated in our install.

[CMCDragonkai]

Doesn't gnuplot support sixel graphics? Anyway I guess this would replace libcaca. However last time I checked sixel support was poor across all the major linux distros. Unless you use xterm.

…

On 29/01/2017 10:15 PM, "Tony Kelman" ***@***.***> wrote: Yeah, makes sense. Hopefully if I show off what mintty already supports it'll give some Julia users a reason to get excited about sixels as a low-dependency image display solution, they might do some fun stuff. See https://github.com/KristofferC/Crayons.jl and https://github.com/evizero/ UnicodePlots.jl for some existing examples, just using already-common escape codes and unicode characters. I am now really curious how out-of-the-box support for sixel display is on default Mac and Linux terminals. Most people developing open-source packages tend to use those platforms, while lots of our users are on Windows so will benefit from mintty once I get it incorporated in our install. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#50 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAnHHfgKoGByT4D_RFYCcIgiko0ALXjAks5rXHTbgaJpZM4J5x11> .

[mintty]

The recent discussion was not about the Sixel feature which is fully incorporated in mintty.
It's an output-only feature, I could have been more specific, sorry; the discussion was about the DEC Locator feature, which could be used in combination with Sixel graphics for interactive graphics.

[mintty]

I have implemented DEC Locator, based on your single report patch and expanded to full feature.
The drawing example works.

[CMCDragonkai]

@mintty How did you get w3m working with images using mintty? One use case of this is the usage of mutt or neomutt and integrating w3m or something else (like directly using sixel) to show the html emails with inline rendering of images, or attachments which are images as well.