[MacOS] Continuous events like MouseMotion and MouseWheel seem laggy in 0.20 #1418
Comments
|
I am experiencing a similar issue where the mouse events lag behind the cursor, and there is a large amount of lag when you click and hold. Here's a Gist with a demo: https://gist.github.com/egauderman/e0e3958f9ac4508f4a0e5839107fbd5a |
goddessfreya
added
DS - macos
H - help wanted
|
I’ve been seeing this too for a while. I had some notes on it here: |
|
I'm curious if it's possible to investigate the similarities/differences between winit and Reprocessing uses for window events, as I used Reprocessing in the past for graphics programming and it didn't have this mouse lag issue (on the same device). |
|
@egauderman Or just compare changes between 0.20 and 0.19? I didn't have the problem with 0.19, this looks like a problem related to the eventloop refactor |
|
I spent a good amount of time trying to figure this out tonight. The lag is pretty conspicuous when printing out all events. Here's what it looks like just dragging the mouse: Smoothly moving the mouseSmoothly move the mouse but add a left click while moving the mouseBut throw in clicking the mouse while being careful to keep the mouse moving: It looks to me like, when clicking, events stop being emitted for about 100ms. It looks to me like something isn't right with the delivery of events from NSApplication to the registered callback, which makes me suspect something is wrong with the setup for NSApplication or CFRunLoop. But I'm not familiar enough with the platform to know for certain. |
|
My guess is that this started along with this overhaul commit: d539168 (I didn't verify this, but it's a huge change and fundamentally changes how events are handled on macOS, so I think it's a good possibility) Before it looks like events were continuously polled by calling appkit::NSApp().nextEventMatchingMask_untilDate_inMode_dequeue_ Now it relies on receiving callbacks from NSApplication, which calls an |
|
Not sure if this is helpful but I checked the Reason Reprocessing library (which in my experience handles mac touchpad input smoothly as I mentioned in my earlier comment) and found that it depends on SDL and internally it calls SDL_PollEvent. Not sure if this is helpful but putting it out there for reference. (i.e. Reprocessing depends on Reasongl which depends on a fork of TSDL which is a wrapper around SDL, and specifically SDL_PollEvent (appears as "poll_event" in the Reason code) is what kicks off the mouse move events) |
|
With the disclaimer that I've never written a line of objective C in my life, and that I have no experience working on macOS, I think this is where SDL does their event handling: https://github.com/spurious/SDL-mirror/blob/6b6170caf69b4189c9a9d14fca96e97f09bbcc41/src/video/cocoa/SDL_cocoaevents.m There is a line here: Looking at the 0.19 winit code there was something similar with the comment "Wait for the next event. Note that this function blocks during resize." // Wait for the next event. Note that this function blocks during resize.
let ns_event = appkit::NSApp().nextEventMatchingMask_untilDate_inMode_dequeue_(
NSEventMask::NSAnyEventMask.bits() | NSEventMask::NSEventMaskPressure.bits(),
foundation::NSDate::distantFuture(cocoa::base::nil),
foundation::NSDefaultRunLoopMode,
cocoa::base::YES);The second So maybe manually pumping these events is still an option? (Assuming the only reason for changing the way this was handled before was to avoid blocking.) Also of note, SDL also has this call |
|
Using |
|
Polling twice appears to workaround this issue (ie only running update code, sleeping, etc every other |
|
@Imberflur That's a valid workaround for now, thanks! |
|
Just chiming in that I am seeing noticeable lag on Windows 10 (nvidia and intel integrated gpus). |
|
In my case at least, I ran my app with Tracy and discovered that due to driver differences between each machine, the time to get the next swapchain texture was "fixed" on one machine to 16ms, but totally unbounded on another. As a result, what I thought was lag was actually just a bug in my handling of input events which causes them to be framerate dependent. My apologies! |
|
I discovered what I think is the same bug on windows 10 and I think the reason is a high mouse polling rate. log at 500Hz log at 1000Hz log at 8000hz, at this point the application freezes completely when I rapidly move the mouse |
|
Any updates on this one? It's been 2 years... |
|
@rumbogs I ran into a similar issue, have you tried sampling the app (using Activity Monitor -> Sample App)? I have a vulkan-based app and found it was spending most of the time blocking the main thread waiting for
|
|
Having a similar issue on Windows. I implemented a fly-camera, but when I move the mouse, the game basically freezes or stutters. Moving the camera with keyboard controls works fine. When I print the events, I get a ton of DeviceEvents with the deltas only ever maxing out at 1.0, ie: I'll have like 200 of these events if I move the mouse quickly before I get one MainEventsCleared event. Seems like my event loop spends all of its time clearing the device events, so it takes too long to generate MainEventsCleared for me to have a good framerate while moving the mouse. Hopefully there's something silly I'm missing here. I'm on winit 0.27.5, via glutin. |
|
I've seen a few workarounds by using MainEventsCleared (one in this thread, one here). It looks like in 0.29.6 that's no longer available. What's the workaround these days? It seems completely gone (I've checked WindowEvent just in case). I'm coming from a big code migration from SDL2 to save binary space so this development is pretty disappointing. I'm also on macOS 12.7.2 |
|
Draw from |
|
Yup, that did it. For future generations what I was doing was rendering every iteration of the The Winit system deduplicates Thanks for the speedy reply here! Hopefully this helps others. |
I mean you tried to render for each event comming, so if moving mouse will generate 100 events you need to render 100 frames to keep up, usually you do mini-batch or draw from e.g. event when going back to sleep without events, like |
|
Absolutely! It's a facepalming-level mistake 😅. I'm just a bit surprised the event queue didn't throw some sort of error but I guess the event queue silently throws things away which would mean the heavier producers of events like mice would get pruned out more often? |
|
It depends on the platform, on Wayland you'll eventually be killed because you overflow the socket on the server. X11 indefinitely buffers so you'll eventually process all the events. |
|
I think I've hit this in bevy, I noticed this after switching to macOS. Here is the most illustrative log I've been able to produce: Somewhere around frame 607 is where the left mouse button is being pressed, however instead we see a huge gap in events until frame 624, where the press is registered, then another 4 frames until a small delta is received, then the release event is received, and finally, on frame 629 we see two huge mouse deltas show up, about 10x larger than normal. This is where the inputs "catch up" after the hitch. I noticed the issue is nonexistent when running without vsync, so effectively polling winit at 1,000 fps, so it seems there is some amount of event loop polls (>2) you need to do to grab the missing inputs. |
|
Based on previous comments (#1418 (comment)), I tried the approach of only running application updates once every 2 |
|
Have you tried running updates once per render operation which you throttle by either vsync, timer targeting refresh rate, or similar? MainEventsCleared could happen a lot per single frame thus if you do heavy updates on it you clearly will get input lag. |
|
A bevy game/application's update runs after processing all winit events and the
Why would this be happening a lot per frame? The docs say:
|
|
|
Sorry, I don't understand. Is there documentation I could read about this change? |
|
Not sure about docs, but it’s true that when the window is hidden or minimized the loop runs very quickly consuming 100% of a core. I do put a synthetic 100ms sleep in that case and track that state with the Occluded event.I should calculate frame rate at some point, but it does seem to be artificially throttled by OpenGL already. CPU load is 10% of an M1 efficiency core which is what I was getting with SDL2 so personally I’ll dig in more if/when I need to.On Dec 28, 2023, at 10:29 PM, Aevyrie ***@***.***> wrote:
A bevy game/application runs after processing all winit events and the MainEventsCleared event is reached.
MainEventsCleared could happen a lot per single frame thus if you do heavy updates on it you clearly will get input lag.
Why would this be happening a lot per frame? The docs say:
Emitted when all of the event loop's input events have been processed and redraw processing is about to begin. [...]
Programs that draw graphics continuously, like most games, can render here unconditionally for simplicity.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>
|
Maybe you can just set the polling mode to |
|
Looking at the latest docs, I see https://docs.rs/winit/latest/winit/event/enum.Event.html#variant.AboutToWait
Okay, I think this is what @kchibisov is saying about input events.
Okay, this makes sense. I updated the application update to only run when The winit examples show manually sending It's not really clear how we are supposed to be using the eventloop then. My assumption was that it would look something like
However there does not seem to be a way to say "give me all events I haven't seen since |
you can ask for
you can do mini batching. 100ms sounds insane, do you have the same input lag in e.g. alacritty, it has relatively good loop? |
I don't understand, why would I request a redraw in response to a redraw?
Input lag is normally fine, that is not the issue. The issue is only when using LMB on macOS, at which point mouse move events are postponed by about 100ms. Although, it also seems to go away when re-running the event loop repeatedly, so something seems to be wrong in the platform implementation for macOS. |
to contiguously draw? |
|
The issue seems to be that there is an expectation to connect some tickrate to Winit, this was done in the past with Drawing should be done at If you want Winit's help to do something like a tickrate, you can use To clarify what I think the documentation around this in Winit is incredibly lackluster and should definitely be improved. In the meantime, please let us know if there is anything else unclear here. |
|
From this discussion, I'm a bit confused about what exactly E.g. naively, I would expect that if there are some events being steadily generated, then longer delays before returning to winit eventually lead to more events clustered before the next Is it not possible to use I realize this would be a bad pattern on the web backend, since we don't want to block the main thread there. For this topic I am interested in the Linux, Macos, and Windows backends. |
This wouldn't happen unless you actually block the event loop like you suggested, but yes, then events would be "clustered" before Winit is able to resume again, fires all "clustered" events and then ends with a
It should be possible, but this isn't only a "bad pattern" on Web, it would be for most platforms (I know it is at least for Linux and MacOS). The event loop does other things in the background that need to be able to run, so blocking the thread would prevent that. I know that on Linux this can lead to outright crashes, on MacOS some things are queued to run on the main thread, which blocking would prevent. I recommend you to look into |
|
Thanks for answering all my questions!
Interesting, there is one thing this makes me curious about. What if my updates take longer than the target tick/frame rate (or my application is trying to run at the maximum possible frame rate) such that there is no time to wait. Would this basically lead to the same issues? (another thing that comes to mind is blocking on VSync) |
|
On macOS I'm not sure about that, but on Wayland you'll likely crash as of now if you don't read events for a while and you have input going, like fastly moving a mouse (winit can't do anything about it at all, it's out of our control for now, in the future there will be a way in libwayland to make buffers unbounded, but that's about it). If your renderer is way too slow, I'd suggest for you to move it into its own thread, and never block the event loop for way to long, and probably have a pure data state which you can update on main and then read and render on other thread. |
before
0.20(pseudocode):after
0.20:Not sure if it's me not setting up correctly. It's pretty obvious in my 3d first person camera example, the example is running at >60fps, movement with keyboard is smooth, but the camera movement which responds to mousemove looks like 20 fps. I have 2 binaries for comparison between 2 versions (I'm using glutin not winit directly, so glutin 0.22 vs glutin 0.21): mousemove_lag_test.zip.
The text was updated successfully, but these errors were encountered: