Add keyboard protocol support (the kitty one)

[WSLUser]

Description of the new feature/enhancement

Provide a keyboard protocol for reporting raw keyboard input, preferably the Kitty keyboard protocol but at the very least XTMODKey, which Xterm supports.

Also see https://gitlab.freedesktop.org/terminal-wg/specifications/-/issues/1

Proposed technical implementation details (optional)

[zadjii-msft]

Is there a particular gap that win32-input-mode (#4999) doesn't cover that you're hoping to address?

[WSLUser]

These are based on VT CSI which I think win32-input-mode wouldn't provide (as that's specifically for win32, not VT applications). It's technically the same, but different audience. Some popular editors do appear to make use of them already, so providing the support on the VT side would be nice.

Also most notably, notcurses has support for the kitty keyboard protocol and minimum support for XTMODKey (it'll likely be re-iterated as time permits).

Also see #4999 (comment), which is basically this particular request.

[WSLUser]

#11264 also would reveal the need for this, if we want to do independent things from conhost (which you seemed to have wanted from the your reply to the comment linked above).

[WSLUser]

I guess the best proposol would be to re-work the input mode so that the win32 input would convert to one of the 2 proposed keyboard protocols. Easier would be to simply have 2 different ones, but that's asking for bad code health and inefficiencies, when much of what's already there can likely be re-used.

[zadjii-msft]

Sure okay, so this is on the client read input side, not the terminal write input side. Just "add support for {whatever} VT Input mode". That's fine, just a little different encoding from the other VT input types we have. Thanks for helping clarify!

[WSLUser]

Yeah I'm thinking #11384 would probably help make this easier to implement as well. Perhaps @j4james would be interested in adding this support.

[j4james]

I'm only interested in implementing the standard ones, like DECPCTERM and DECEKBD.

Also, I think the XTerm protocol is closely related to #8719 (possibly intended to be the same thing, but I was under the impression that they weren't strictly compatible).

[WSLUser]

Yes Iterm is slightly incompatible with Xterm's implementation, which is why I'd rather we opted for Kitty's instead as the most robust one existing and upon which the terminal working group was discussing as a base to which certain changes were encouraged by some terminal authors. Never got to resolution though and though it's the first one in the gitlab, it's also the longest standing one still awaiting a standard. I do know there are a number of TUI applications that make use of Kitty's keyboard protocol as issues were raised about it in the past, so we'd be getting the support of those applications automatically. We could also adopt both but that makes little sense as they aim to do the same thing, just one does it better than the other.

[j4james]

I'd rather we opted for Kitty's instead as the most robust one existing and upon which the terminal working group was discussing as a base

If you read the whole terminal-wg thread, most people seem to be opposed to the Kitty protocol and were arguing for some version of the CSI u protocol instead. Do you know of any terminals other than Kitty that implement it?

[zadjii-msft]

Yea frankly, libtickit has the superior protocol. I'm only leaving this open as a "if someone really, REALLY wanted to implement this for us, we'd accept it". It's helpful for folks to know what still needs to be done, even if we're not planning on getting to it ourselves.

[unxed]

The problem with the CSI u (libtickit) protocol is that there is no way to specify alternate layout key[s]. This is useful for keyboard layouts such as Cyrillic where you want the shortcut ctrl+c to work when pressing the ctrl+С on the keyboard. Kitty's "Report alternate keys" mode fixes it.

[kirawi]

So far, these terminal emulators have implemented the protocol:

• Kitty
• Foot
• Alacritty
• Rio (referenced Alacritty's implementation)
• Wezterm

I think that with these recent additions, not opting for the kitty enhanced keyboard protocol has become a more contentious path than when this issue was opened.

[j4james]

In case I'm not the only one that wasn't aware of this, it's worth noting that the current kitty keyboard protocol which everyone is implementing is not the same as the one that was originally proposed in terminal-wg. I think the switch actually happened years ago, but I only found out about the new version fairly recently.

I haven't looked at it in much detail, but from my initial impression I'd now be more inclined to pick kitty over libticket/xtmodkeys if we were going to implement one of them.

That said, this is still not something I'm personally interested in at this point in time.

[unxed]

Despite the fact that the kitty keyboard protocol seems overly complicated and difficult to implement, writing its implementation turned out to be quite easy.

Here is the implementation code I wrote for far2l, I am releasing this code snippet under the Public Domain license so that anyone can use it.

This is not a complete implementation, but it can be easily extended to become a full one. However, this implementation is sufficient for all the applications I have tested (far2l file manager, turbo text editor, Free Pascal’s Free Vision sample app with kitty keyboard protocol support patch from their GitLab). At the very least, this implementation can be used as a reference.

This code generates kitty protocol ESC sequences from Windows-compatible Key Event Record structure and can be easily adapted for use in the Microsoft terminal.

[unxed]

As for me, win32 input mode is fine, and we already support it in far2l. As for alternate layout feature, virtual key code field can be used for that.

Btw, it is important to understand that a protocol without applications that use it makes no sense, and the kitty's protocol already has a certain set of applications that work with it.

[unxed]

Libraries implementing kitty keyboard protocol:

The notcurses library
The crossterm library
The textual library

[zadjii-msft]

Thanks for the reference implementation! If there was an enterprising individual out there that was really interested in contributing this, I'd take a look at

https://github.com/microsoft/terminal/blob/main/src/terminal/input/terminalInput.hpp

TerminalInput is the class that's responsible for converting all INPUT_RECORDs into VT sequences. You'd probably also need to handle the CSI > 1 u somewhere in https://github.com/microsoft/terminal/blob/main/src/terminal/parser/OutputStateMachineEngine.cpp and https://github.com/microsoft/terminal/blob/main/src/terminal/adapter/adaptDispatch.cpp

[unxed]

Regarding the disadvandages of the kitty protocol, I see only two at the moment.

The specification seems overly complex. The real issue here lies in how the specification is written, rather than the protocol itself. I'll explain its essence to you in just a few words.

Each key event message in this protocol contains 7 parameters (first 5 are of type int, next goes one or more ints separated by colons and the last parameter is of type char), of which only the first and last ones are mandatory. Messages have the following structure:

ESC [ unicode-key-code : shifted-key : base-layout-key ; modifiers : event-type ; text-as-codepoints suffix

Example:

ESC [122;129:3;122u

The presence of certain fields is determined by, on one hand, their non-zero value, and on the other, by the flags set when activating the protocol.

• unicode-key-code is the character corresponding to the pressed key, in lowercase, or special code point representing non-text key, according to the table in the specification.
• shifted-key is the same character considering the shift and caps lock keys.
• base-layout-key is the character that the same key would print when the English layout is set.
• modifiers is a bitfield containing information about the pressed modifiers, such as alt or ctrl.
• event-type is the type of event: press, release, or repeat (in case the key is held down).
• text-as-codepoints contains one or more Unicode code points corresponding to the pressed key, in the form that the text would be sent if the kitty protocol was not enabled.
• suffix is added for compatibility reasons and can differ from key to key, table is provided in spec

That's it! The protocol is actually so simple that you already understand how it works.

The specification for the modifiers field does not distinguish between right and left modifier keys. Fortunately, this can be easily worked around on the application side by tracking the press and release events of the right modifier keys. Also the protocol can easily be extended adding support for handling right modifiers keys better while still maintaining full backward compatibility.

Considering all this, as well as the highest number of implementations in applications, I believe that this protocol has the best chance of becoming the de facto standard for full keyboard input support in terminals.