Improve web text agent #4561

jprochazk · 2024-05-28T16:57:45Z

Closes TextAgent assumes canvas is aligned to the top #4060 - no longer aligned to top
Closes Allow for manual placement of the canvas element #4479 - canvas.style is not set anywhere anymore
Closes egui eframe TextEdit on mobile #2231 - same as TextAgent assumes canvas is aligned to the top #4060
Closes egui_text_agent is not removed from the dom when WebRunner.destroy() is called. #3618 - there is now one <input> per eframe app, and it's removed transitively by WebRunner::destroy -> AppRunner::drop -> TextAgent::drop

This PR improves the text agent to make fewer assumptions about how egui is embedded into the page:

Text agent no longer sets the canvas position
There is now a text agent for each instance of WebRunner
The input element is now moved to the correct position, so the OS can display the IME window in the correct place. Before it would typically be outside of the viewport

The best way to test this is to build & server the web demo locally:

scripts/build_demo_web.sh && scripts/start_server.sh

Then open the EasyMark editor, and try using IME to input some emojis: http://localhost:8888/#EasyMarkEditor

To open the emoji keyboard use:

win + . on Windows
ctrl + cmd + space on Mac

Tested on:

Migration guide

The canvas no longer controls its own size/position on the page. This means that those properties can now be controlled entirely via HTML and CSS, and multiple separate eframe apps can coexist better on a single page.

To match the old behavior, set the canvas width and height to 100% of the body element:

<html>
  <body>
    <canvas></canvas>
  </body>
</html>

/* remove default margins and use full viewport */
html, body {
  margin: 0;
  width: 100%;
  height: 100%;
}

canvas {
  /* match parent element size */
  width: 100%;
  height: 100%;
}

Note that there is no need to set position: absolute/left: 50%; transform: translateX(-50%)/etc., and setting those properties may poorly affect the sharpness of egui-rendered text.

Because eframe no longer updates the canvas style in any way, it also means that on mobile, the canvas no longer collapses upwards to make space for a mobile keyboard. This should be solved in other ways: #4572

Mm2PL · 2024-05-28T18:12:02Z

I was asked to test this, works decently with ibus however there is a slight issue with the input being selected.
When I click-drag my mouse out of the input (like in selecting for example): I cannot use IME.
https://github.com/emilk/egui/assets/25011746/d88877ea-72b5-4c9b-8dc8-f87e4dfada00
In the video the Firefox devtools open because Ctrl+Shift+E isn't stolen by ibus from Firefox.
I tested master and it seems that the issue is present there too.

jprochazk · 2024-05-28T18:32:03Z

I was asked to test this, works decently with ibus however there is a slight issue with the input being selected.

Thanks! I opened an issue for that:

Input not focused when selecting text #4563

jprochazk · 2024-05-28T18:50:07Z

At the very least there don't seem to be any regressions here, and it's definitely improved some things, but also revealed just how broken input handling is on mobile web.

emilk · 2024-05-29T08:44:26Z

Tested on iOS Safari, and when I click a text field that is in the lower half of the screen, the on-screen keyboard will now cover that text field, making it impossible to see what I'm typing. On master, the egui canvas would temporarily move up to prevent this problem.

So this is a regression that we should note down in an issue, ideally together with some ideas on how to solve it.

Still, I think a regression on mobile text input (which is already pretty broken anyways) is worth it in the interest of improving desktop web.

Mobile web text editing tracking issue #4500

emilk · 2024-05-29T09:04:01Z

I'm testing bringing up the emoji editor on Mac on desktop chromium, and it will only sometimes track the text field I have selected, but quite often go to the top-left corner. And once it has ended up in the top-left once, it doesn't seem like it wants to leave.

I am testing using the TextEdit field in the WidgetGallery

emilk

So nice to get this cleaned up ❤️

But please open an issue describing the regression on mobile.

Are there any other likely regressions?

crates/eframe/src/web/text_agent.rs

jprochazk · 2024-05-29T10:16:04Z

Opened a bunch of issues and added them to

Mobile web text editing tracking issue #4500

- Closes emilk#4060 - no longer aligned to top - Closes emilk#4479 - `canvas.style` is not set anywhere anymore - Closes emilk#2231 - same as emilk#4060 - Closes emilk#3618 - there is now one `<input>` per `eframe` app, and it's removed transitively by `WebRunner::destroy -> AppRunner::drop -> TextAgent::drop` This PR improves the text agent to make fewer assumptions about how `egui` is embedded into the page: - Text agent no longer sets the canvas position - There is now a text agent for each instance of `WebRunner` - The input element is now moved to the correct position, so the OS can display the IME window in the correct place. Before it would typically be outside of the viewport The best way to test this is to build & server the web demo locally: ``` scripts/build_demo_web.sh && scripts/start_server.sh ``` Then open the EasyMark editor, and try using IME to input some emojis: http://localhost:8888/#EasyMarkEditor To open the emoji keyboard use: - <kbd>win + .</kbd> on Windows - <kbd>ctrl + cmd + space</kbd> on Mac Tested on: - [x] Windows - [x] Linux - [x] MacOS - [x] Android - [x] iOS ## Migration guide The canvas no longer controls its own size/position on the page. This means that those properties can now be controlled entirely via HTML and CSS, and multiple separate `eframe` apps can coexist better on a single page. To match the old behavior, set the `canvas` width and height to 100% of the `body` element: ```html <html> <body> <canvas></canvas> </body> </html> ``` ```css /* remove default margins and use full viewport */ html, body { margin: 0; width: 100%; height: 100%; } canvas { /* match parent element size */ width: 100%; height: 100%; } ``` Note that there is no need to set `position: absolute`/`left: 50%; transform: translateX(-50%)`/etc., and setting those properties may poorly affect the sharpness of `egui`-rendered text. Because `eframe` no longer updates the canvas style in any way, it also means that on mobile, the canvas no longer collapses upwards to make space for a mobile keyboard. This should be solved in other ways: emilk#4572

jprochazk added 2 commits May 28, 2024 18:45

fix typo

be46389

improve text agent

fd3af1a

jprochazk added web Related to running Egui on the web eframe Relates to epi and eframe text Problems related to text IME labels May 28, 2024

fix lint

2994ae9

jprochazk mentioned this pull request May 28, 2024

Docs show live web viewer where possible rerun-io/rerun#2935

Closed

fix lint

f6e7281

jprochazk marked this pull request as ready for review May 28, 2024 19:57

emilk approved these changes May 29, 2024

View reviewed changes

crates/eframe/src/web/text_agent.rs Outdated Show resolved Hide resolved

crates/eframe/src/web/text_agent.rs Outdated Show resolved Hide resolved

emilk changed the title ~~Improve text agent~~ Improve web text agent May 29, 2024

jprochazk mentioned this pull request May 29, 2024

Text field is covered by keyboard on mobile web #4572

Open

jprochazk added 4 commits May 29, 2024 12:18

clarify comment

0d80de4

docs for focused_element + add has_focus helper

0b9911b

naming

250c812

remove unnecessary bound

c1732ad

jprochazk merged commit 514ee0c into master May 29, 2024
35 checks passed

jprochazk deleted the fix-ime branch May 29, 2024 10:54

emilk added the bug Something is broken label Jun 27, 2024

jprochazk mentioned this pull request Jul 1, 2024

[Eframe] Allow relative position canvas #2331

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve web text agent #4561

Improve web text agent #4561

jprochazk commented May 28, 2024 •

edited

Loading

Mm2PL commented May 28, 2024 •

edited

Loading

jprochazk commented May 28, 2024 •

edited

Loading

jprochazk commented May 28, 2024

emilk commented May 29, 2024 •

edited

Loading

emilk commented May 29, 2024 •

edited

Loading

emilk left a comment

jprochazk commented May 29, 2024

Improve web text agent #4561

Improve web text agent #4561

Conversation

jprochazk commented May 28, 2024 • edited Loading

Migration guide

Mm2PL commented May 28, 2024 • edited Loading

jprochazk commented May 28, 2024 • edited Loading

jprochazk commented May 28, 2024

emilk commented May 29, 2024 • edited Loading

emilk commented May 29, 2024 • edited Loading

emilk left a comment

Choose a reason for hiding this comment

jprochazk commented May 29, 2024

jprochazk commented May 28, 2024 •

edited

Loading

Mm2PL commented May 28, 2024 •

edited

Loading

jprochazk commented May 28, 2024 •

edited

Loading

emilk commented May 29, 2024 •

edited

Loading

emilk commented May 29, 2024 •

edited

Loading