Consecutive filter updates on heatmap layer

[nerik]

(follow up of a conversation with @mikelmaron)

mapbox-gl-js version: 0.54

browser: Chrome and Firefox latest, macOS

Our current map is a combination of Mapbox GL layers for all non-animated layers, and a custom WebGL renderer for animated heatmaps, which is not performing well enough and is a battery drain (because it’s heavily CPU bound).

Steps to Trigger Behavior

See below

Link to Demonstration

On paper, using Mapbox GL heatmap layers would be an ideal solution: not only performance is much better, but also they look way better (and we could remove a huge legacy dependency from our front-end).
The problem is that we can't get this to behave nicely with animation (ie repeatedly changing a timestamp filter on a layer). See for yourself: https://codepen.io/nerik8000/pen/rgMdqP?editors=0010

Expected Behavior

When clicking the start button on the top left, the filter value gets updated at each rAF call to display more or less points depending on a timestamp global value.
When stopping the animation (stopping updates to filter), Mapbox GL should skip any pending update and display the filtered data that matches what the UI expects, which is the last timestamp.

Actual Behavior

As you can see in the demo, when clicking the stop button, there's seemingly a queue of rendering updates that get executed one by one. This result in a lag of several seconds between the expected value and what's actually visible in the map.

While I understand the performance implications of filtering that many points (several 10s of 1000s in the demo), there should be a mechanism to allow skipping frames to match as closely as possible whatever values are set from outside. The issue here being that the resulting discrepancy makes any numeric statement displayed outside the map wrong (until the animation's final frame).

Thanks for your help.

[mourner]

setFilter is currently a very expensive operation and is indeed problematic when used for animation use cases. Perhaps you could try a different approach — encoding values for different time intervals as feature properties (e.g. one feature would have value1: 12, value2: 15, value3: 20 etc. — and then you would switch between them by doing setPaintProperty('heatmap-weight', ... expression that depends on the right value). This is the approach https://electricitytransition.com is using and it seems to work well.

[nerik]

Hi Vladimir. Thanks you so much for the answer, this is indeed a very interesting approach. We will definitely try this, noting that this requires stable geometries somehow (ie gridded data in our case).

One comment: https://electricitytransition.com still has the same "delay" problem that I mentioned, which is barely visible at the framerate used on the live site (one frame every 100ms) but is very noticeable when setting a faster framerate (closer to 60 fps / 16ms frames).
The animation performance is not really the main issue here, but reliability is. Is there a way to either:

• force skipping frames / cancel the "update queue" (if anything like this exists?).
  I tried using repaint = false and triggerRepaint but it doesn't seem to yield any visible effect
• at least get some feedback from the renderer on whether that "update queue" has been exhausted.
  Unless I'm missing something it seems data and idle events can't be really reliably used.

Cheers

[agusterodin]

Were you able to find a solution on how to cancel filter queue and skip to last item in queue?

Any techniques learned on how to effectively animate using either setFilter or other techniques.

[nerik]

Hi @agusterodin.

Were you able to find a solution on how to cancel filter queue and skip to last item in queue?

No, it's not possible.

The way we (sort of) got around that limitation is to:

• reduce number of features filtered -> use a gridded heatmap so that features are 'stable', each with a set of values for all frames
• reduce number of active frames -> load only a chunk of frames at the time (kind of like video streaming)
• reduce filter complexity -> values are 'baked' into the features so that the filter is a very simple step or case expression (the value is directly the color bucket).

The result of this approach is visible there:
https://globalfishingwatch.org/marine-manager/ascension-public?latitude=43.66969900697606&longitude=-3.856381785733444&zoom=4.8555486174938505&start=2019-06-12T00%3A00%3A00.000Z&end=2019-07-05T00%3A00%3A00.000Z&dvIn[0][id]=fishing-vms&dvIn[0][cfg][filters][flag][0]=FRA&dvIn[0][cfg][dss][0]=public-global-fishing-effort%3Av20201001&dvIn[1][id]=fishing-ais&dvIn[1][cfg][filters][flag][0]=ESP&dvIn[2][id]=context-layer-mpa&dvIn[2][cfg][visible]=false

This was just released, currently writing a technical blogpost about it.