Implement text-input and input-method protocol support, v2

[Shugyousha]

This is a rebased version of #12. Using anthywl and wlroots 0.15.1 you should be able to input Japanese and see the input popup as well.

Note that there still some issues like the input popup position not updating properly when changing the layout while the popup is shown. If we decide to include this functionality in dwl, it's probably worth investing more time to figure out how to make this work properly.

Until then this code is mostly here so people can experiment with IMEs and their functionality in dwl (at least until these text-input and input-method protocols get fixed and more stable; see the old PR mentioned above for details).

[sevz17]

see Shugyousha#1

[Shugyousha]

see Shugyousha#1

hm, I must have totally missed that one last year for some reason ...

personally, I am not a fan of the "nospacesstyle" because it makes things harder to read in my opinion (I was actually contemplating to discuss opening a PR for dwl to change everything to snake_case ...). If djpohly prefers this style though, I can change it.

[sevz17]

personally, I am not a fan of the "nospacesstyle" because it makes things harder to read in my opinion (I was actually contemplating to discuss opening a PR for dwl to change everything to snake_case ...). If djpohly prefers this style though, I can change it.

I think the same, but it comes from the dwm style

[guyuming76]

@Shugyousha , i works on my machine. I think what follows is the magic lines

And all those popup->link , popup->view_link lines of code are useless now, and we may remove them, right?

[guyuming76]

@Shugyousha , just found some issue:

Issue 1:
when i open alacritty in tag 1 and press Ctrl+space to switch to Chinese input, its fine;

but then i press Alt+2 to switch to tag 2, and continue to type, there will be input popup in tag 2, notice thitat currently, there is not application open in tag 2 yet, and the committed Chinese input will appear in the alacritty in tag 1.

Issue 2: it might be related to issue 1: i switch to a new tag, open alacritty in new tag and input some Chinese , then i go back to the original tag, sometimes, i cannot switch to Chinese again in the original tag. I have not found the exact procedure to reproduce this, but by switching to new tag and switching back, chances are that you might lose the ability to turn on input. However, after some switching between tags, you can have input popup again.

[sevz17]

@Shugyousha , just found some issue:

Issue 1: when i open alacritty in tag 1 and press Ctrl+space to switch to Chinese input, its fine;

but then i press Alt+2 to switch to tag 2, and continue to type, there will be input popup in tag 2, notice thitat currently, there is not application open in tag 2 yet, and the committed Chinese input will appear in the alacritty in tag 1.

May be solved assigning tags to input popups, or even better popups should be children of the clients.

[guyuming76]

to reproduce issue2, xwayland in config.mk needs to be enabled.

[sevz17]

to reproduce issue2, xwayland in config.mk needs to be enabled.

So, are you running alacritty through xwayland?

[guyuming76]

to reproduce issue2, xwayland in config.mk needs to be enabled.

So, are you running alacritty through xwayland?

I don't think so.

if i run xwininfo, and then click a geogebra window, it will return xwindow information;
but if i run xwininfo and then click an alacritty window, nothing returns.

Any other way to check whether alacritty runs through xwayland?

[sevz17]

Any other way to check whether alacritty runs through xwayland?

xeyes

[guyuming76]

May be solved assigning tags to input popups, or even better popups should be children of the clients.

1. popups should be children of the clients. Does this mean that children can only be shown inside the client rectangle? what if the client rectangle is very small?

2. assigning tags to popups. Sounds fine. But, when i switch to an empty tag, there is no popup at first. After i press a key, the popup appears in the empty tag. The problem seems to be why my key press has been routed to the previous client(or text_input associated with that client or anything else. I am not sure about the underlying process, have not traced the code yet) while i am now in an empty tag. When i leave a client to a new tag, no key press should be routed to the client as i understand. And my test to DWL without text_input works as expected. I mean, i am not sure whether this is cause by text_input and input_method or their usage in this PR

[sevz17]

1. popups should be children of the clients. Does this mean that children can only be shown inside the client rectangle? what if the client rectangle is very small?

The current behavior: constrain popups to monitor size

2. assigning tags to popups. Sounds fine. But, when i switch to an empty tag, there is no popup at first. After i press a key, the popup appears in the empty tag. The problem seems to be why my key press has been routed to the previous client(or text_input associated with that client) while i am now in an empty tag.

Sounds like a bug of this implementation, we should check this and keyboard_get_im_grab()

[guyuming76]

Sounds like a bug of this implementation, we should check this and keyboard_get_im_grab()

after some debugging, i find that:

1. when i press MOD+3 to switch to tag3, callstack view->focusclient->client_activate_surface->wlr_xdg_toplevel_set_activated will be called to deactivate a surface which i guess correspond to the old client.
2. in wlr_xdg_toplevel_set_activated, surface->toplevel->scheduled.activated is set to false, where surface is of type wlr_xdg_surface;
3. in keyboard_get_im_grab, wl_resource_get_client(input_method->keyboard_grab->resource) return wl_client ;
4. I want to try adding a condition in keyboard_get_im_grab to check the activated status set in step 2, but what we can get as in step 3 is a wl_client, how can we get a wlr_xdg_surface from a wl_client, or from input_method->keyboard_grab->resource ?

And if this can be done, why not just suppress the key in the following wlr_input_method_keyboard_grab_v2_xxxxxxx method calls due to some client surface is not activated?

And in step 1, wlr_seat_keyboard_notify_clear_focus are called in focusclient if no client, but it does not seem to have effect on input_method->keyboard_grab. I am not sure about what keyboard_grab mean, but why wlr_seat_keyboard_notify_clear_focus don't prevent input_method from accepting key here?

Anyway, i tried a simple but "brutal" fix here: https://gitee.com/guyuming76/dwl/commit/3e55353b3d5094540eb89c33e5875f16c086a223 , which seems to fix Issue 1 i mentioned above. But issue 2 still exist. Actually, i still have no idea about what cause issue 2.

By the way, creating popup in layers[lyrFloat] seems to conflict with existing floating window features. If i press MOD and drag an alacritty window to float, input method popup won't appear in that window.

UPDATE: found a better fix for issue1: https://gitee.com/guyuming76/dwl/commit/342f9658ca5a03b149f7fdb35d212497e083cf3b

UPDATE: looks like the cause of Issue2 i mentioned above can be "fixed" this way: https://gitee.com/guyuming76/dwl/commit/1edd23d81864cb7845820e55d638f68a6dbe81f5 . But this will crash DWL when press key in a tag with no client. The question is, no way to disable a specific text_input in wlroots?

UPDATE: to my surprise, this seem to fix Issue2: https://gitee.com/guyuming76/dwl/commit/6fd7a417f53bcf70a1cfd6d40e15fefec0753907

[Shugyousha]

And all those popup->link , popup->view_link lines of code are useless now, and we may remove them, right?

Yes, I think so. I have removed them, thanks!

[guyuming76]

i am curious why wlr_input_method_v2_send_deactivate(relay->input_method) and relay_disable_text_input are needed. So, i tried removing them (https://gitee.com/guyuming76/dwl/commits/PR235_5) . It works except that DWL crash if i try to input Chinese in geogebra (xwayland). But i still does not understand why.

[guyuming76]

why did we call input_popup_update so often? i removed most of them and have not got error yet: (https://gitee.com/guyuming76/dwl/commit/1cd75e7706925120235cd94dbf7325427baba3c8 )

[sevz17]

i am curious why wlr_input_method_v2_send_deactivate(relay->input_method) and relay_disable_text_input are needed. So, i tried removing them (https://gitee.com/guyuming76/dwl/commits/PR235_5) . It works except that DWL crash if i try to input Chinese in geogebra (xwayland). But i still does not understand why.

See #235 (comment)

[guyuming76]

just found a new issue (call it Issue3 hereafter): with sloppyfocus=0, when i click into the Chinese input popup, dwl crash.
if sloppyfocus=1, dwl crash as soon as i move mouse onto the input popup, which i never noticed before,although i also had never paid attention to the sloppyfocus setting.

UPDATE: fixed with a new separate layer for input popup: https://gitee.com/guyuming76/dwl/commit/79fd6df77db336c3518ceca2053a6739bd96c452

[guyuming76]

See #235 (comment)

I don't quite understand the difference between surface->role_data->data and surface->data->data ; but since i find in the creation of Client, surface->data->data is used, i think use the same in client_from_wlr_surface make sense.

I merged recent changes from dwl here: https://gitee.com/guyuming76/dwl/tree/PR235_7

[guyuming76]

@Shugyousha @sevz17
input method suddenly stopped to work on my machine: the popup does not appear anymore.

So i added some wlr_log entries to help troubleshooting: https://gitee.com/guyuming76/dwl/commits/PR235_8

The log shows that input_method->keyboard_grab return false in function keyboard_get_im_grab.

but i don't know why this happens.

In wlr_input_method_v2.c , input_method->keyboard_grab is initialized in im_grab_keyboard, but i don't know what triggers im_grab_keyboard.

any suggestion?

UPDATE: turned out a silly mistake,i fixed it with: https://gitee.com/guyuming76/dwl/commit/7ad12a9aaa8aa561a5dd6e469917f0415dd9739b

[Shugyousha]

i am curious why wlr_input_method_v2_send_deactivate(relay->input_method) and relay_disable_text_input are needed. So, i tried removing them (https://gitee.com/guyuming76/dwl/commits/PR235_5) . It works except that DWL crash if i try to input Chinese in geogebra (xwayland). But i still does not understand why.

See #235 (comment)

This should be fixed now that I have merged your changes.

[Shugyousha]

why did we call input_popup_update so often? i removed most of them and have not got error yet: (https://gitee.com/guyuming76/dwl/commit/1cd75e7706925120235cd94dbf7325427baba3c8 )

input_popup_update updates the popup state and also calls wlr_input_popup_surface_v2_send_text_input_rectangle. The latter will send the text_input_rectangle event to the input method v2 client. AFAIK anthywl is not processing this event but in theory another input method implementation could depend on it being sent.

[Shugyousha]

By the way, creating popup in layers[lyrFloat] seems to conflict with existing floating window features. If i press MOD and drag an alacritty window to float, input method popup won't appear in that window.

I have changed the code to use its own layer for the input popups. I assume that this will fix this issue.

[guyuming76]

@Shugyousha @sevz17

recently, i added support to onclick in my waybar custom module, so that i can use mouse click to switch between tags, besides the MOD+num keypress. it works by running "wtype -M alt 2 -m alt" command on mouse click.

And i got sigsegv exception as in screenshot:

so, i tried to fix in https://gitee.com/guyuming76/dwl/commit/226155c196c17be9bc4bfb08c35a27fd1b37037a

not sure whether the fix is correct, anyway i don't have the sigsegv exception now.

[fauxmight]

recently, i added support to onclick in my waybar custom module, so that i can use mouse click to switch between tags, besides the MOD+num keypress. it works by running "wtype -M alt 2 -m alt" command on mouse click.

@guyuming76 Where is your custom waybar module hosted?

[guyuming76]

@fauxmight

@guyuming76 Where is your custom waybar module hosted?

I put the files for my waybar here, if login is required, github account is supported: https://gitee.com/guyuming76/personal/tree/dwl/gentoo/waybar-dwl

And the README here https://gitee.com/guyuming76/dwl include my guide to start dwl with waybar.

[guyuming76]

@Shugyousha @sevz17

I made a new change: https://gitee.com/guyuming76/dwl/commit/1df4b18d7f93d63f2abd43d663623387d71afdac

to address an issue which may reproduce as follows:

1. start dwl
2. MOD+enter to start alacritty in TAG 1
3. switch to TAG 2 with mouse click
4. switch back to TAG 1 with mouse click
5. repeat step 3,4 several times, until when you find that you cannot click to switch to TAG 2 or any other TAG again; press a key, such as 'a', you will be able to switch to other TAG again with mouse click

With this change, i don't have the issue anymore, the change is notdeactivating and activating input_method again and again, instead, only activate input_method only once on new. And the change is only for situation where XWAYLAND is not compiled in.

[Shugyousha]

@Shugyousha , just found some issue:
Issue 1: when i open alacritty in tag 1 and press Ctrl+space to switch to Chinese input, its fine;
but then i press Alt+2 to switch to tag 2, and continue to type, there will be input popup in tag 2, notice thitat currently, there is not application open in tag 2 yet, and the committed Chinese input will appear in the alacritty in tag 1.

May be solved assigning tags to input popups, or even better popups should be children of the clients.

I have pushed a change to add a list of input popups to the Client and used it to deactivate the popup scene if the client is not visible on a tag.

[guyuming76]

@Shugyousha @sevz17
when i open search engine in firefox, such as bing and baidu, and enable text input in the search box and type, very often, the input popup will appear outside the firefox window, at its upper left hand, instead of below the cursor.

after adding wlr_log entries, i found that it is related to the cursor_rect in input_popup_update method, when the WLR_TEXT_INPUT_V3_FEATURE_CURSOR_RECTANGLE is not set.

is it a wlroots problem or not?

[Shugyousha]

after adding wlr_log entries, i found that it is related to the cursor_rect in input_popup_update method, when the WLR_TEXT_INPUT_V3_FEATURE_CURSOR_RECTANGLE is not set.

I think if the text-input implementation used in the client (so firefox in this case) is not sending this information (in time?) this feature flag will not be set and the dwl implementation cannot get the cursor position. In that case the popup x and y coordinate will be set to 0 and the popup will show up in a corner.

[guyuming76]

@Shugyousha
thanks! in my log, i can see the call to text_input_set_cursor_rectangle shortly after input_popup_update, but i don't know how to double check in code whether this is due to the client(firefox) or wlroots.

Anyway, i make a change to input_popup_update here: https://gitee.com/guyuming76/dwl/commit/c5019437ca7f360d1518cb9a45d1db670fdba277

When !cursor_rect, i set
popup->x=parent.x;
popup->y=parent.y;
popup->visible=true;

That is, show popup at upper left corner of the parent surface.

Before the change popup->y will set to some negative value.

Many lines of code seems involved in this change only because they are indented after being added into the if (!cursor_rect) else branch.

[guyuming76]

@Shugyousha , what do you use for Japanese IME?

I tried fcitx5 with fcitx5-anthy these days, but it is NOT working normally,although:
fcitx5 Chinese works with this PR;
fcitx5 Chinese works with DWM;
fcitx5 Anthy works with DWM;

With Chinese, in both DWM and DWL, there is only one kind of popup for input.

With Anthy in DWM, there are two kinds of popup for input, one displays your type-ins, and the other(which appear after you press space) let you make selection.

But with Anthy in this PR, the first kind of popup won't appear, only the second appears.

[Shugyousha]

Shugyousha , what do you use for Japanese IME?

I was using https://github.com/tadeokondrak/anthywl for testing. From what I remember, there is only one popup for this one.

[tienbuigia]

does anyone still working on this? I'm an user who really need to type asian language and love dwl.