Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

winit::TabLeft/Event::Suspended doesn't function as intended on WASM #13486

Open
simbleau opened this issue May 23, 2024 · 6 comments
Open

winit::TabLeft/Event::Suspended doesn't function as intended on WASM #13486

simbleau opened this issue May 23, 2024 · 6 comments
Labels
A-Windowing Platform-agnostic interface layer to run your app in C-Bug An unexpected or incorrect behavior O-Web Specific to web (WASM) builds S-Needs-Design This issue requires design work to think about how it would best be accomplished

Comments

@simbleau
Copy link
Contributor

Bevy version

0.13

What went wrong

JavaScript is an interpreted language, and the browser will essentially suspend code execution completely if the tab is hidden. Bevy's systems are tied to requestAnimationFrame, which is also completely paused when the user is tabbed away from the window.

In other words: On wasm, the main thread gets suspended when it is hidden (e.g. when the user switches tabs). This means that the app.update() function will not be called, because bevy's scheduler only runs app.update() when the browser's requestAnimationFrame is called (and that happens only when the tab is visible).

What problem does this cause?

  • No systems will execute whilst the tab is inactive. This means networking crates will disconnect easily (no communications being sent while alt-tabbed).
  • Networking crates that don't disconnect may experience buffer overflows for bounded protocols, as they will process hundreds of missed packets all at once. This could result in dropped packets.

There's a very long discussion on how this problem affects lightyear available here. We eventually come to a solution, which I'll propose below.

The part about winit::TabLeft

So yeah, there is technically events like Event::Suspended in winit that are supposed to fire when the tab gets left. However, as aforementioned, no code executes when the tab is hidden. As a result, this event is handled only when you return to the tab!

< user leaves tab >

... 30 seconds

< user goes back to the tab >

--> new event: Event::Suspended
--> new event: Event::Resumed

And what use is that? It's delivering no value currently. I'd call this a bug.

The solution

A solution, credited to @Nul-led , is to add a very small plugin (~10 lines of code) to spawn a web worker which sends a message every 1 second (configurable time). A callback is added to the web browser's DOM events to execute the main schedule exactly 1 time.

The code can be seen here: https://github.com/cBournhonesque/lightyear/pull/371/files#diff-5ec95d0ed493b4b63bba9ae693c990e6e1612d2fbe6309df678bf21fb75fbc1b

What I want

Selfishly, I want to see this in bevy. It's a very small amount of code but could be massively beneficial to many crates who want to keep a continuous run in the background when the tab is inactive.

Specifically, I think this should be a

pub struct WebExecutionPlugin {
    callback_ms: f32 // default = 250.0 ms
}

that can be added to any application.

Furthermore, it would be nice to have a

#[derive(States)]
pub enum WindowFocusState {
    Inactive,
    Active,
}

that changes, depending on whether the window or tab is visible.

@simbleau simbleau added C-Bug An unexpected or incorrect behavior S-Needs-Triage This issue needs to be labelled labels May 23, 2024
@cBournhonesque cBournhonesque added the O-Web Specific to web (WASM) builds label May 23, 2024
@Nul-led
Copy link

Nul-led commented May 23, 2024

Im not really sure if this should be part of bevy to be honest. The solution mentioned uses the fact that web workers aren't throttled by major browsers (unlike timers in content scripts), however this is (as far as im aware) not standardized and up to the browser for implementation. So it is essentially a temporary fix not a solution as browsers could very well change this behavior at any point. Which leads me to believe that it is not a good idea to rely on this in bevy.

However it might be a good idea to run app.update() whenever the browser fires an visibilitychange event.

Additionally my solution relies on unsafe casting and will only work in a single threaded environment so it isn't entirely future proof in that aspect either (see multithreading proposal for WebAssembly).

Tldr;
I believe this should be implemented in an independent crate or at least with an optional feature.

@alice-i-cecile alice-i-cecile added A-Windowing Platform-agnostic interface layer to run your app in S-Needs-Design This issue requires design work to think about how it would best be accomplished and removed S-Needs-Triage This issue needs to be labelled labels May 23, 2024
@Nul-led
Copy link

Nul-led commented May 24, 2024

Published https://crates.io/crates/bevy_web_keepalive if someone needs a solution until this is fixed upstream (if this gets fixed at all)

@Nul-led
Copy link

Nul-led commented May 24, 2024

tho @happydpc this might not work for you as this only runs the Main schedule, so the renderer subapp wouldn't be affected.

@simbleau
Copy link
Contributor Author

Yeah my issues are fixed with that plugin

github-merge-queue bot pushed a commit that referenced this issue Jun 3, 2024
# Objective

- Upgrade winit to v0.30
- Fixes #13331

## Solution

This is a rewrite/adaptation of the new trait system described and
implemented in `winit` v0.30.

## Migration Guide

The custom UserEvent is now renamed as WakeUp, used to wake up the loop
if anything happens outside the app (a new
[custom_user_event](https://github.com/bevyengine/bevy/pull/13366/files#diff-2de8c0a8d3028d0059a3d80ae31b2bbc1cde2595ce2d317ea378fe3e0cf6ef2d)
shows this behavior.

The internal `UpdateState` has been removed and replaced internally by
the AppLifecycle. When changed, the AppLifecycle is sent as an event.

The `UpdateMode` now accepts only two values: `Continuous` and
`Reactive`, but the latter exposes 3 new properties to enable reactive
to device, user or window events. The previous `UpdateMode::Reactive` is
now equivalent to `UpdateMode::reactive()`, while
`UpdateMode::ReactiveLowPower` to `UpdateMode::reactive_low_power()`.

The `ApplicationLifecycle` has been renamed as `AppLifecycle`, and now
contains the possible values of the application state inside the event
loop:
* `Idle`: the loop has not started yet
* `Running` (previously called `Started`): the loop is running
* `WillSuspend`: the loop is going to be suspended
* `Suspended`: the loop is suspended
* `WillResume`: the loop is going to be resumed

Note: the `Resumed` state has been removed since the resumed app is just
running.

Finally, now that `winit` enables this, it extends the `WinitPlugin` to
support custom events.

## Test platforms

- [x] Windows
- [x] MacOs
- [x] Linux (x11)
- [x] Linux (Wayland)
- [x] Android
- [x] iOS
- [x] WASM/WebGPU
- [x] WASM/WebGL2

## Outstanding issues / regressions

- [ ] iOS: build failed in CI
   - blocking, but may just be flakiness
- [x] Cross-platform: when the window is maximised, changes in the scale
factor don't apply, to make them apply one has to make the window
smaller again. (Re-maximising keeps the updated scale factor)
    - non-blocking, but good to fix
- [ ] Android: it's pretty easy to quickly open and close the app and
then the music keeps playing when suspended.
    - non-blocking but worrying
- [ ]  Web: the application will hang when switching tabs
- Not new, duplicate of #13486
- [ ] Cross-platform?: Screenshot failure, `ERROR present_frames:
wgpu_core::present: No work has been submitted for this frame before`
taking the first screenshot, but after pressing space
    - non-blocking, but good to fix

---------

Co-authored-by: François <[email protected]>
@daxpedda
Copy link
Contributor

I was investigating if there is an issue in Winit or if Winit can do something here, just dropping my results here.

So my assumption is that the reason why Event::Suspended is only received when going back to the tab is because in general all Winit events in Bevy are cached in handled in WindowEvent::RedrawRequested. But this event has to be handled in the event loop directly and not when WindowEvent::RedrawRequested is reached (which would be in the next browser event loop tick).

Unfortunately, the spec doesn't specify anymore in which case a browser is allowed or not allowed to freeze a page (and discard it afterwards). To prevent browsers based on Chromium (the only engine currently implementing the Page Lifecycle API) from freezing the page, see this document by Chromium outlining their heuristic.

In the future the spec might officially introduce an opt-out method with the Screen Wake Lock API, see WICG/page-lifecycle#31.

@simbleau
Copy link
Contributor Author

In the meantime, for anyone visiting this issue, just use https://crates.io/crates/bevy_web_keepalive

It solves the issue for all browsers by deploying a Web worker which runs the Main schedule once every (configurable) interval when the window is suspended.

knutsoned pushed a commit to knutsoned/bevy that referenced this issue Jun 25, 2024
- Upgrade winit to v0.30
- Fixes bevyengine#13331

This is a rewrite/adaptation of the new trait system described and
implemented in `winit` v0.30.

The custom UserEvent is now renamed as WakeUp, used to wake up the loop
if anything happens outside the app (a new
[custom_user_event](https://github.com/bevyengine/bevy/pull/13366/files#diff-2de8c0a8d3028d0059a3d80ae31b2bbc1cde2595ce2d317ea378fe3e0cf6ef2d)
shows this behavior.

The internal `UpdateState` has been removed and replaced internally by
the AppLifecycle. When changed, the AppLifecycle is sent as an event.

The `UpdateMode` now accepts only two values: `Continuous` and
`Reactive`, but the latter exposes 3 new properties to enable reactive
to device, user or window events. The previous `UpdateMode::Reactive` is
now equivalent to `UpdateMode::reactive()`, while
`UpdateMode::ReactiveLowPower` to `UpdateMode::reactive_low_power()`.

The `ApplicationLifecycle` has been renamed as `AppLifecycle`, and now
contains the possible values of the application state inside the event
loop:
* `Idle`: the loop has not started yet
* `Running` (previously called `Started`): the loop is running
* `WillSuspend`: the loop is going to be suspended
* `Suspended`: the loop is suspended
* `WillResume`: the loop is going to be resumed

Note: the `Resumed` state has been removed since the resumed app is just
running.

Finally, now that `winit` enables this, it extends the `WinitPlugin` to
support custom events.

- [x] Windows
- [x] MacOs
- [x] Linux (x11)
- [x] Linux (Wayland)
- [x] Android
- [x] iOS
- [x] WASM/WebGPU
- [x] WASM/WebGL2

- [ ] iOS: build failed in CI
   - blocking, but may just be flakiness
- [x] Cross-platform: when the window is maximised, changes in the scale
factor don't apply, to make them apply one has to make the window
smaller again. (Re-maximising keeps the updated scale factor)
    - non-blocking, but good to fix
- [ ] Android: it's pretty easy to quickly open and close the app and
then the music keeps playing when suspended.
    - non-blocking but worrying
- [ ]  Web: the application will hang when switching tabs
- Not new, duplicate of bevyengine#13486
- [ ] Cross-platform?: Screenshot failure, `ERROR present_frames:
wgpu_core::present: No work has been submitted for this frame before`
taking the first screenshot, but after pressing space
    - non-blocking, but good to fix

---------

Co-authored-by: François <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Windowing Platform-agnostic interface layer to run your app in C-Bug An unexpected or incorrect behavior O-Web Specific to web (WASM) builds S-Needs-Design This issue requires design work to think about how it would best be accomplished
Projects
None yet
Development

No branches or pull requests

5 participants