Make lesson text selectable and searchable

[golddotasksquestions]

Is your feature request related to a problem? Please describe.
I can't select and copy any lesson text from the running app.

Describe the solution you'd like
Being able to select and copy text like in a pdf or web application.
Being able to hit Ctrl+F to search for any word of phrase and cycle through the occurances (like in any text editor or browser or pdf reader)

Why?
To report issues more easily, to quote and communicate with fellow students and teachers more easily about things which are unclear to them.

[Xananax]

Copy pasting is prevented by the browser sandboxing webgl applications, but we can probably work around it with a custom js interface.

Searching is going to be less simple to handle, because there isn't a way to arbitrarily read all text on screen in a Godot app. We'd have to register each text box individually in some repository, so we can have global search.

Searching in the text editor itself (or in any individual text box) is doable, but of limited ussefulness

[Xananax]

So after looking into it, it doesn't seem like copy-pasting has a good solution.

You can invoke methods to copy to the clipboard and intercept requests to paste from Javascript. They are relatively reliable. But that is useless because any key press and/or event gets swallowed as long as the WebGL has focus. So we cannot use those, we have to rely on native browser functionality.

Note: Copy works by default in both Chrome and Firefox; however, pasting is sandboxed to the webgl context: you can copy and paste text within the app, but you can not paste text from outside the app.

In Chromium, pasting an external clipboard prompts a permission request, after which pasting works. In Firefox, there's nothing to be done.

Relevant references:

• Game clipboard is not shared with system clipboard when running HTML5 exports godotengine/godot#12587
• Partial Javascript clipboard support godotengine/godot#29298

---

Open for a far fetched mitigation idea

On way I can imagine to work around this would be to trap focus in an html element, and then forward all key presses to the WebGL context. This way, we get as much control as the browser gives us. I don't know if that would mean delays in sending keypresses though (it'd be at the very least a 1 frame delay), as well as other potential bugs. More importantly, it would become impossible to use Godot's Input system, we'd have to create our own abstraction.

---

I'm deeming copy-paste unsolvable at the moment. I'll leave this open while we investigate the feasibility of an app-wide search functionality.

[NathanLovato]

It'd be good to have that. For now yes, it seems we can't offer copy-paste support in all browsers. Instead, for GitHub issues, we could add a more powerful feature from the app: directly create an issue with the selected text quoted. That way, you don't need copy and paste. That's a possible workaround for this issue.

Searching itself is easy, but highlighting matches is not possible right now in Godot with how RichTextLabel works. We'll look into contributing to the engine to allow for that (we had to make changes engine-side for the code editor already).

[winston-yallow]

Would it be an option to have a "Paste" button? It is not ideal, but would be better than nothing. It would call out to javascript to open a focussed popup (html) which prompts the user to paste. Once the user does so, the popup closes and forwards the paste to the game instance (and returns focus/control to it).

As an example to how this can look, here is how jupyter notebooks allow pasting when no text box is selected:

[NathanLovato]

I don't know if we can do that with a Godot app. Maybe. I'm not fond of those kinds of hacks and would prefer a clearer UX.

Considering the problem at hand, I think the ability to report the selected text to GitHub would be simpler and clearer for the user (e.g. select, right click, report issue).

We had the idea of allowing people to report and prefill issues right from the app.

[Xananax]

If we want pasting, the solution that Winston proposes is probably pragmatically the best solution. It isn't difficult to do, and we can invoke it from Godot without issues, whether through a button, a shortcut, or both.

I suggest to split this issue in 3:

1. "add a modal box for pasting" (enhancement, nice to have, targeted at a future release, maybe 0.4)
2. "open a github issue with pre-filled text" (enhancement, targeting milestone 0.2), with the following sub-tasks
   • present a form for the user to fill (easier to use than templates for users)
   • send user system information (the functionality is already there, just need to format it and append it)
   • pre-fill form with selected text (which answers the current conversation)
3. "search functionality" (enhancement, targeting milestone 0.4). We could just continue the issue here, but this one has become a bit of a stream of consciousness and the goal isn't clear anymore

[NathanLovato]

Sure, the thing is there's currently no issue raising the need for pasting text from outside the app into the app. And it wasn't the purpose of this issue.

[xix-xeaon]

This is also required for students who work with text-to-speech and translation software.

[NathanLovato]

students who work with text-to-speech

I think blind people won't be able to use the app. It's running in a WebGL context so we don't have access to the browser's accessibility features like aria hints. I'm unsure whether an external program could read the text inside the app.

For translation, it'll be limited too. At best selection will happen per text control - you won't be able to select the entire page. There again because it's not a web page and controls work independently.

But for those persons, we plan to support multiple languages, see #22

[xix-xeaon]

I wasn't primarily thinking of blind people - that kind of support would be on a completely different level. I have students who see perfectly well but still have trouble reading. They might have dyslexia, ADHD or other conditions, some have moved from other countries and are not yet fluent in the local language but are learning it, and some simply lack practice reading (as is common for teenagers these days).

They use various different programs to aid them, including but not limited to TTS and translation. If they can copy text into those programs then they can use them and it improves their learning rate, and it also satisfies certain requirements in the local school system.

[xix-xeaon]

I'd say this simple change on the RichTextLabel in ui/screens/lesson/UIContentBlock.tscn would be a significant improvement. Although much more could of course be done.

- mouse_filter = 2
+ mouse_default_cursor_shape = 1
+ selection_enabled = true

[NathanLovato]

Ah it makes sense, thanks. Well, they'll likely have to use the desktop version of the app or the Chrome browser, as other browsers don't allow copying text out of the app at the moment.

Regarding selecting, we need to change Godot and might as well wait until we port the app to Godot 4, got some tax change that makes a big difference for us, we have to be careful with money now.

[Snailedlt]

+1
A built in text-to-speech narration button would also be super useful!
For me it's much easier to digest information when hearing it as opposed to reading it.

Another perk of having selectable text is that in chrome you could copy a link to that text like this: