Draft: RFC: add inputcapture trigger for shortcuts and immediate capture #1035
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is intended as replacement for the NotifyFoo methods. libei provides a more flexible and powerful method of sending input events to the compositor. A new method ConnectToEIS requests a file descriptor from the compositor which can then be plugged into libei. Once established, the communication between compositor and application is direct, without the need to go through the portal process(es). To avoid ambiguities between NotifyFoo and input events sent via libei, any application that uses an EIS connection may not use the NotifyFoo methods. Co-authored-by: Olivier Fourdan <[email protected]>
This portal is aimed at allowing applications to capture input from physical devices, the prime use case is the server component of an InputLeap (a Synergy fork) setup where local devices are used to control a remote display. The general flow for the InputCapture session is: - application queries the "zones" available (usually representing the desktop) - application sets up "pointer barriers" - application provides a libei-compatible handle for the actual input events - when a cursor moves against/across a pointer barrier, the compositor notifies the client and re-routes input events from libinput to libei instead of controlling the local cursor Notable: the actual input events are not part of the portal here, they're handled by the direct libei connection between compositor and application. This portal has no direct dependencies on libei itself. The compositor is in charge of everything - it can restrict some screens from showing up in the regions, it can deny pointer barriers at locations, it decides when a pointer barrier is crossed, it can stop capturing (or filter events) at any time, etc.
This allows an application to effectively say "start capturing now" once it calls Enable at some later point. The trigger is separate from the Enable call to a) make it more obvious that is what we want and b) allow for user interaction if need be. The process thus is: -> CreateSession <- session handle -> SetEnableTrigger <- compositor says yes -> Enable <- compositor says Activated <- captured input flows As with pointer barriers, the exact conditions for enabling the session are compositor implementation details but one would expect that having focus is a base requirement. In the Activated/Deactivated case, the capture session effectively remains. Where the session is Disabled (by compositor or application, e.g. on focus out) the application can immediately call SetEnableTrigger + Enable to get capture going again whenever it's convenient (e.g. on focus in). Unlike pointer barriers, zones do not apply for this case.
This requires an active GlobalShortcuts session with the requested shortcut(s) to be active - this call re-uses the shortcut ids from that session. When the input capture session is enabled and the compositor deems the shortcut to be triggered, capture starts.
|
I'm guessing adding a dependency on libei for global keybinds (as the portal itself, e.g. a key in the compositor is triggered and it's sent over libei) wouldn't be worth it at all here, and would be a bit complicated to handle vs just having the portal dole them out? Wanted to be sure, as it's something that's popped into my mind several times in the past few weeks. |
|
yeah, it feels a bit wrong to send what is likely a single shortcut trigger via libei and libei isn't really optimised for that case. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a draft only to start discussion, no implementation beyond the XML specification.
It sits on top of #714 (and #762) but intentionally kept separate for now, I don't want this to hold up either of those.
#714 adds the input capture portal but the only triggers are pointer barriers. This works for e.g. the input-leap use-case.
This PR proposes two extra triggers:
SetEnableTriggerto allow an application to effectively say "start capturing now"SetGlobalShortcutsto allow input capture to be triggered by (previously aggreed on) global shortcutsThey're fairly identical in the end but the SetEnableTrigger approach would usually require focus, the SetGlobalShortcuts can work without focus.
These could be use by e.g. accessibility technologies that need to take over the input feed on-demand. The shortcuts approach could also be used by a generic macro daemon like autohotkeys that lives external to the compositor.
In both cases the overall architecture could look like this:
In other words, the application sets up an InputCapture and RemoteDesktop session (and GlobalShortcuts, if applicable). Once input is received the application can send back input events to the compositor where it is processed as expected. Notably: because the real input is "detached" there is no conflict with normal input events.
For the macro daemon this would allow easy triggering of complex sequences, e.g. a tmux-like
shortcut + Atypes out "Hello world". It would also allow for things like input forwarding to games - a game runtime manager would simply re-set the EnableTrigger on each focus out, thus (typically) getting the input re-captured whenever it gets back into focus. This would provide a similar effect as the wayland relative-pointer/shortcuts-inhibit/pointer-constraint set of protocols.For accessibility technologies it's the same thing, the captured input can be converted into whatever other data and sent back to the compositor (e.g. keyboard-to-mouse or touchpad-to-keyboard or tablet-to-whatever conversion).
cc @jadahl, @ofourdan, @garnacho