Feature Request: sixel graphics support #448
Comments
|
While implementing Sixel, it is important to test with images that contain transparency. Testing using WSL with Ubuntu for example, in mlterm such images are properly rendered as having a transparency mask and the background color is kept, while in xterm -ti vt340, untouched pixels are drawn black, even though the background is white, which seems to imply they render sixels on a memory bitmap initialized as black without transparency mask or alpha before blitting them into the terminal window. |
|
OOh. Sixel is very cool stuff. I've decided that I need that. NEED. |
|
I'll happily review a PR :) |
zadjii-msft
added
the
Issue-Feature
zadjii-msft
added
Area-Output
|
Caught the Build 2019 interview today that mentioned this request. I still maintain that Xorg on sixel is just wrong. So very very wrong. The ffmpeg-sixel "Steve Ballmer Sells CS50" demo never gets tired tho. Gotta say, it is a little disappointing the video lacks sound (sound really makes the video). Consoles already have sound, naturally. They totally beep. Precedent set. What we really need is a new CSI sequence for the opus clips interleaved with the frames, amirite? |
DHowett-MSFT
changed the title
|
Related: #120 |
|
Need.
|
|
LOL I was watching the stream and I just thought to myself "here's my boss assigning me work live in front of a studio audience". |
|
Please make this a priority for v1.0! |
|
3d animations can be v1.5 😛 |
|
OMG |
ghost
added
the
miniksa
added
Needs-Tag-Fix
|
Upvoting this request, Sixels would be such an amazing thing to have in the Terminal. |
|
This weekend I finished implementing sixel read support for my MIT-licensed Java-based TUI library, and it was surprisingly straightforward. The code to convert a string of sixel data to a bitmap image is here, and the client code for the Sixel class is here. I have done very little for performance on the decoder. But when using the Swing backend, performance is still OK, as seen here. (The snake image looks bad only because byzanz used a poor palette creating the demo gif.) I was a bit taken aback how quickly it came together. It's very fair to say that the "decode sixel into bitmap" part is the easy bit, the hard bit is the "stick image data into a text cell, and when that is present blit the image to screen rather than the character". Just want to mention it to other folks interested in terminal support for sixel, and hoping it could help you out. |
|
I'll upvote if someone else writes a Jupyter notebook client ;) |
|
We already have an example of Sixel support in mintty which is written in C (vice java). Only thing needed is a refactor to C++ (at least for initial support). Still always good to see how it's been implemented in other projects. |
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
|
sixel support is rapidly becoming as mandatory as ANSI escape sequence support. Please do not allow Terminal to become as obsolete as the old Windows console host by not implementing sixel. |
|
As far as I know, terminal emulators are the only platform capable of running cross platform, statically linked, mouse driven apps that can even be run remotely over ssh. The main feature missing from this platform is a universally implemented graphics API. I've spent a bit of time researching this, and I believe implementing at least one of sixel, kitty, or iTerm2 would be well worth the effort. |
|
IS there something similar to sixel, but for audio? I think it would makes sense over SSH connections... |
Similar, insofar as that representing anything even moderately complex will require extreme downsampling? Absolutely there is: DECPS. |
|
@rgpublic I know it's offtopic, sorry for that, but hoped you could guide me. Good to know something exists, thank you to both of you :-) |
|
I hope this i the central issue for image display, so I am following this. Unfortunately there seem to be many other issues which essentially request the same feature, so that its relevance might be underestimated ... |
|
@j4james How stale is this: #448 (comment) branch? With eyes on #1173 in the immediate term, I wanna see just how far away that would be from merging in a full passthrough world |
|
@zadjii-msft I actually just got my sixel branch merged with main in the past week or so. And with the latest progress on #1173 I was going to ask whether it might be a good time to create a PR for it. Worst case I thought we could put it behind a feature flag until the Windows Terminal side is ready. That said, I've still got a lot of cleaning up to do before the code is presentable, so it may be a while before I can create a PR. Also, I should note that it only has a GDI renderer - I was hoping somebody else would handle the Atlas implementation, because I know very little about how that works. |
|
NOW IS THE TIME |
|
Sorry I got a little excited but yes - I think this would be the perfect time to start merging that in. We can absolutely merge this all piece by piece. We can work on merging the conhost sixel implementation while passthrough is getting ready. Then we can get the Atlas sixel implementation in in post. |
Once I have an idea of what you need, I'll give you an implementation the same day. :) |
|
This is my favorite GitHub notification in today's catch-up. So excited to see this make it in 😁 |
|
And this FFmpeg-SIXEL is crazy cool. IDK why I am so excited for running videos in the terminal... or maybe because then we can get graphics output directly from python scripts in both REPL and from posh CLI. |
|
I would rather recommend mpv. And you can already play movies in the terminal with |
## Summary of the Pull Request This PR introduces basic support for the Sixel graphics protocol in conhost, limited to the GDI renderer. ## References and Relevant Issues This is a first step towards supporting Sixel graphics in Windows Terminal (#448), but that will first require us to have some form of ConPTY passthrough (#1173). ## Detailed Description of the Pull Request / Additional comments There are three main parts to the architecture: * The `SixelParser` class takes care of parsing the incoming Sixel `DCS` sequence. * The resulting image content is stored in the text buffer in a series of `ImageSlice` objects, which represent per-row image content. * The renderer then takes care of painting those image slices for each affected row. The parser is designed to support multiple conformance levels so we can one day provide strict compatibility with the original DEC hardware. But for now the default behavior is intended to work with more modern Sixel applications. This is essentially the equivalent of a VT340 with 256 colors, so it should still work reasonably well as a VT340 emulator too. ## Validation Steps Performed Thanks to the work of @hackerb9, who has done extensive testing on a real VT340, we now have a fairly good understanding of how the original Sixel hardware terminals worked, and I've tried to make sure that our implementation matches that behavior as closely as possible. I've also done some testing with modern Sixel libraries like notcurses and jexer, but those typically rely on the terminal implementing certain proprietary Xterm query sequences which I haven't included in this PR. --------- Co-authored-by: Dustin L. Howett <[email protected]>
|
since #17510 is merged, maybe close this issue? |
microsoft-github-policy-service
bot
added
the
Needs-Tag-Fix
and others
Would like to see Sixel support in the Terminal, this is the standard used to show graphics in the console.
Sixel is part of the original DEC specification for doing graphics in terminals and has been re-popularized in recent years for doing graphics on the command line, in particular by Pythonistas doing data science.
The libsixel library provides an encoder but is also a great introduction to the subject (better than the Wikipedia page):
https://github.com/saitoha/libsixel
The text was updated successfully, but these errors were encountered: