bug: Capacitor listeners failing to be called from Android notifyListeners with foreground service

[corypisano]

Bug Report

Capacitor Version

Latest Dependencies:

@capacitor/cli: 4.6.2
@capacitor/core: 4.6.2
@capacitor/android: 4.6.2
@capacitor/ios: 4.6.2

Installed Dependencies:

@capacitor/cli: 4.6.2
@capacitor/core: 4.6.2
@capacitor/android: 4.6.2
@capacitor/ios: 4.6.2

Platform(s)

• Android

Current Behavior

I'm maintaining a capacitor plugin that emits events for listeners (https://capacitorjs.com/docs/plugins/android#plugin-events). It uses a native Android & iOS package that starts a foreground service (and on Android, adds a notification to status bar). While the app is in the foreground or background, the native plugin code calls notifyListeners, and the javascript listener callback is run as expected.

The issue is that on Android when the app is swiped away, the Android plugin code continues to run as desired due to the foreground service and status bar notification and calls notifyListeners, but the javascript listener callback is never called. In logcat after the AppDestroyed lifecycle event is seen, the Android `"Notifying listeners for event X" is still being output, but the javascript listener callback is never reached (nor is there a "no listeners found for event X").

Expected Behavior

If the android plugin code is running and successfully calls notifyListeners, the javascript listener callback (via addListener) is expected to run.

Code Reproduction

I will create a sample application and update here, but wanted to see if it is a known issue in the meantime

The key components would be on Android to call
startForeground and setup a notification like so https://medium.com/@engineermuse/foreground-services-in-android-e131a863a33d

emit the listener event

JSObject ret = new JSObject();
ret.put("value", "some value");
notifyListeners("myPluginEvent", ret);

and on the javascript side setup a listener callback

import { MyPlugin } from 'my-plugin';

MyPlugin.addListener('myPluginEvent', (info: any) => {
  console.log('myPluginEvent was fired');
});

Notice in the foreground and background the js listener will fire, but on swiping away the app, the Android code will continue to run and notify listeners but javascript listener doesn't stay alive.

---

Update: https://github.com/corypisano/capacitor-listeners-issue

Other Technical Details

npm --version output: 8.1.0

node --version output: v16.13.0

pod --version output (iOS issues only):

Additional Context

[Ionitron]

This issue may need more information before it can be addressed. In particular, it will need a reliable Code Reproduction that demonstrates the issue.

Please see the Contributing Guide for how to create a Code Reproduction.

Thanks!
Ionitron 💙

[corypisano]

Created a sample application and plugin to reproduce: https://github.com/corypisano/capacitor-listeners-issue

the plugin starts a foreground service and outputs logs and a plugin event every 2 seconds on a timer

the app just imports the plugin and adds a listener

Logcat output:

On app start, the foreground service and timer starts

Every two seconds a log is seen from

• BroadcastService.java that is running the timer, and sending a broadcast
• the android plugin that receives the broadcast and calls notifyListeners
• Capacitor/MyPluginPlugin logging that notify listeners was called
• the listener callback in the capacitor app
  

Same behavior continues after app is backgrounded

When app is swiped away, the android code continues and notify listeners is called, but listener callback is never seen

[shipley-dcc]

I see similar behaviour with two plugins that I use in an app. Both @capacitor-community/background-geolocation and @transistorsoft/capacitor-background-fetch display this behaviour. When my app is put into the background, the javascript callbacks work for 5 minutes, but then stop being called. You can see that the android code is being called, but the calls to notifyListeners aren't getting processed by javascript. When you bring the app back to the foreground all the queued-up callbacks are processed.

[peitschie]

@shipley-dcc the 5 minute timeout is due to some fairly undocumented WebView optimizations that kick in after the app window moves away from the foreground (even if there's a foreground service).

The easiest way to turn these off is using a plugin like https://bitbucket.org/TheBosZ/cordova-plugin-run-in-background/src/master/ (I haven't confirmed this works with capacitor), specifically:

cordova.plugins.backgroundMode.on('activate', function() {
    cordova.plugins.backgroundMode.disableWebViewOptimizations();
});

[peitschie]

@corypisano swiping away the application generally kills the activity, which in the case of a capacitor-based app will destroy the WebView and kill the JavaScript engine. The native android code continues to function because you're running that in a service which will survive the death of the app activity (for a while, at least).

Generally speaking, in modern Android there's no way to prevent your activity from being killed. If you are not releasing via the Play Store, you might be able to use a technique like automatically restarting your activity on termination (e.g., see cordova-plugin-autostart), but that requires special permissions with current versions of Android, and there's no way to ask the user for them specifically.

If you are trying to make a javascript process that runs independent of the WebView activity and can survive the activity being destroyed... well... that is much much harder, and there's no public plugins capable of doing this that I've found.

A few more relevant links to help can be seen on my comment here: #3032 (comment)

Ideally, Capacitor would provide a mechanism for running a headless WebView so we can execute JS within a service... but... I don't see that being something achieved any time soon 😓

[shipley-dcc]

@shipley-dcc the 5 minute timeout is due to some fairly undocumented WebView optimizations that kick in after the app window moves away from the foreground (even if there's a foreground service).

The easiest way to turn these off is using a plugin like https://bitbucket.org/TheBosZ/cordova-plugin-run-in-background/src/master/ (I haven't confirmed this works with capacitor), specifically:

cordova.plugins.backgroundMode.on('activate', function() {
    cordova.plugins.backgroundMode.disableWebViewOptimizations();
});

@peitschie Thank you for this. It is much appreciated.

[flanger777]

I have a similar issue in my app which uses the @capacitor-community/background-geolocation plugin. After moving the app to the background, javascript callbacks are only executed for 5 minutes and then stop executing. It is worth saying that with capacitor 3.9 everything work as expected. There is also a report of this issue in the repository of the plugin mentioned above, but there is still no solution.

[shipley-dcc]

I followed the advice above but used the capacitor version of the plugin called capacitor-plugin-background-mode. Configure it to disable web view optimizations and then enable it when you start logging location positions. It basically tells the webview that it is still visible so it keeps on processing JavaScript.

[lunedam-git]

@shipley-dcc Interested to know. Does disabling the webview optimisations reliably solve the problem on Android for @capacitor-community/background-geolocation? Been looking for a solution to this. if you're releasing via the Play Store, will an app using capacitor-plugin-background-mode get accepted?

[shipley-dcc]

@lunedam-git It has solved the problems for our app across a number of devices and it passed review in the Play Store.

[lunedam-git]

@shipley-dcc Good news. Many thanks.

[nemoneph]

It's currently a serious limitation compared to a native app on Android devices.
I think the "disableWebViewOptimizations" can be a temporary solution, but it's a bad hack which can be blocked at any time.

Here some inputs about the problem.

• We have a valid use case scenario : tracking app with foreground service, phone is locked / or app set in BG.
• Every second, we get current position, on the javascript we only do a simple "console.log" on a method callback call from native (it's the addLisenter / notifyListeners from capacitor).

For reproduction, you can test quickly with :

https://github.com/capacitor-community/background-geolocation/tree/master/example

Also tested : https://github.com/transistorsoft/capacitor-background-geolocation and https://github.com/transistorsoft/cordova-background-geolocation-lt

Devices (all power saving disabled, see https://dontkillmyapp.com/ for details) :

Samsung Galaxy Note 10+ on Android 12 => throttling after 5 minutes.
Xiaomi MI 9, on Android 9 => no trottling working in every use case.
Iphone X => no problem (it's link to android only).

So we see that happen on some ANDROID devices, when the app is set to background / or phone locked.
After 5 minutes, some javascript execution linked to this listener (addListener / notifyListeners) is throttled.

What I mean by "throttled", is that the javascript execution is paused / buffered in a queue, and the queue is unstack when app go back in foreground.

In my simple example, we don't see the console.log any more after 5 minutes, until we set the app in the foreground.
At this moment, all the throttled events are unstack and executed immediatly (Can cause a mini-lag when app go in the foreground when a lot of javascript execution been throttled).

• This "Throttled" on Chrome is a kwown thing, we know that "heavy" function like setInterval/setTimeout/ XMLHttpRequest are paused after 5 minutes in the background, but in this case we don't have an heavy function, so it's weird, it should'nt be throttled.

More infos about throttling :

https://chromestatus.com/feature/5527160148197376
https://developer.chrome.com/blog/timer-throttling-in-chrome-88/
(the google dev who wrote the article can be contacted by twitter)

An another important information, it's seem good on Cordova (without the disableWebViewOptimizations hack) ! No trotthling on the same use case on any phone / example !

It why I think it's linked to Capacitor addListener / notifyListeners implementation/ in conjonction with Tab throttling.

I dont know Cordova so, I don't know the difference in relation to addListener / notifyListeners from Capacitor but maybe there is something to dig here... and way to avoid the throttled in legitim case.

It's really annoying and limiting some feature.

I wonder what do you think about this @liamdebeasi or @mlynch ?

[diachedelic]

@nemoneph This appears to be relevant:

OK, here's the new bit in Chrome 88. Intensive throttling happens to timers that are scheduled when none of the minimal throttling or throttling conditions apply, and all of the following conditions are true:

• The page has been hidden for more than 5 minutes.
• The chain count is 5 or greater.
• The page has been silent for at least 30 seconds.
• WebRTC is not in use.

From https://developer.chrome.com/blog/timer-throttling-in-chrome-88/.

It would explain why fudging the WebView's visibility appears to fix the problem (that is what happens when you call disableWebViewOptimizations()). Unfortunately, this workaround interacts badly with any JavaScript that relies on accurate "visibilitychange" events.

[nemoneph]

@nemoneph This appears to be relevant:

OK, here's the new bit in Chrome 88. Intensive throttling happens to timers that are scheduled when none of the minimal throttling or throttling conditions apply, and all of the following conditions are true:

• The page has been hidden for more than 5 minutes.
• The chain count is 5 or greater.
• The page has been silent for at least 30 seconds.
• WebRTC is not in use.

From https://developer.chrome.com/blog/timer-throttling-in-chrome-88/.

It would explain why fudging the WebView's visibility appears to fix the problem (that is what happens when you call disableWebViewOptimizations()). Unfortunately, this workaround interacts badly with any JavaScript that relies on accurate "visibilitychange" events.

Yess fooling the webview visibility is bad and not a viable solution.

Some ideas : https://developer.chrome.com/blog/timer-throttling-in-chrome-88/#state-polling

[peitschie]

An another important information, it's seem good on Cordova (without the disableWebViewOptimizations hack) ! No trotthling on the same use case on any phone / example !

In general, I've definitely seen identical behaviour with current Cordova so this doesn't appear to me to be a Capacitor-specific issue 🙂 . There might be some differences between Capacitor and Cordova with how event handlers into the webview trigger that perhaps makes the problem more apparent with Capacitor? (I haven't dug any to prove this)... but in general the web timers are throttled by the webview alone, and has nothing to do with capacitor or cordova.

Yess fooling the webview visibility is bad and not a viable solution.

I have a slightly different perspective here as this approach has been stable for several years now across a variety of Android devices. I mean, the future is always uncertain, but I don't really see Google making much effort to change this behaviour, as it's very much opt-in for an application to do.

Unfortunately, every other approach eventually fails if you're using any kind of promise-based or async/await'd code, as it's only a matter of time before some kind of timer is involved. I agree that's it's not ideal though, but haven't found any other ways to solve the issue.

There might be different behaviours with timers in a service worker? I haven't dug enough to see however 🙂

[nemoneph]

Different of mine, but I understand your point of view about the hack with disableWebViewOptimizations.
Currently it's the only option, but may be we can find better option if we cleary understand the problem.

To add precision of what I mean when I say it works with cordova but not working with capacitor => I'm just talking about the event trigger system to communicate from native to js with the WebView (notifyListeners / addListener on capacitor) wich is not throttled on cordova.

If I set a setInterval, or xmlHttpRequest in cordova/capacitor/whatever-webview it will be throtlled when in use in the background; it's the chrome/webview behavior.

But it's seem, that cordova handle the communication differently from native => to webview JS context and it doesn't get throttled.

[peitschie]

Just a quick crawl through upstream source code (I'm guessing with a lot of this, so hopefully someone can spot any glaring errors):

The throttling feature was added here: https://bugs.chromium.org/p/chromium/issues/detail?id=1075553

The core logic showing how the page visibility works to disable the throttling is here: https://chromium.googlesource.com/chromium/src/+/HEAD/third_party/blink/renderer/platform/scheduler/main_thread/page_scheduler_impl.cc#657

Some additional logic about the freezing in the background can be found by searching for kStopInBackground: https://chromium.googlesource.com/chromium/src/+/refs/heads/main/third_party/blink/common/features.cc#490

This has some hints about when the throttling kicks in: https://chromium.googlesource.com/chromium/src/+/HEAD/third_party/blink/renderer/platform/scheduler/common/features.h#29

Information about the frame scheduler: https://chromium.googlesource.com/chromium/src/+/HEAD/third_party/blink/renderer/platform/scheduler/main_thread/frame_scheduler_impl.cc#805

The implementations of various throttling budgets are here: https://chromium.googlesource.com/chromium/src/+/HEAD/third_party/blink/renderer/platform/scheduler/common/throttling/

Unfortunately, looking through the exposed android webkit wrappers shows nothing terribly obvious that would allow the scheduling to be impacted at all: https://developer.android.com/reference/android/webkit/WebView

The view provider layer in java is defined here: https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/webkit/WebViewProvider.java

The bindings exposed from the blink engine are here: https://chromium.googlesource.com/chromium/src/+/HEAD/third_party/blink/public/web/web_view.h#119

This comment is pretty interesting there:

  // |widgets_never_composited| is an indication that all WebWidgets associated
  // with this WebView will never be user-visible and thus never need to produce
  // pixels for display. This is separate from page visibility, as background
  // pages can be marked visible in blink even though they are not user-visible.
  // Page visibility controls blink behaviour for javascript, timers, and such
  // to inform blink it is in the foreground or background. Whereas this bit
  // refers to user-visibility and whether the tab needs to produce pixels to
  // put on the screen at some point or not.

I've also noticed the PageScheduler is exposed here: https://chromium.googlesource.com/chromium/src/+/HEAD/third_party/blink/public/web/web_view.h#371

I'm parking further digging for now, as so far nothing obvious jumps out at me. The page visibility is still by far the simplest way to do this (despite the drawbacks that have been noted!)

Next step would be to dig through the capacitor bridge code and figure out what's different about how the events are communicated to JS that makes the throttling hit capacitor harder than cordova.

[diachedelic]

From what I can tell, no timers are invoked when notifyListeners is called:

capacitor/core/native-bridge.ts

Line 871 in 60ffcc3

win.androidBridge.onmessage = function (event) {

. @nemoneph has reported that not even console.log is successfully called after 5 minutes, so perhaps it does not have anything to do with timers after all.

Every second, we get current position, on the javascript we only do a simple "console.log" on a method callback call from native (it's the addLisenter / notifyListeners from capacitor).

[peitschie]

There is some pathways in returnResult that use a promise rather than a straight function callback, which is something cordova never does:

capacitor/core/native-bridge.ts

Lines 911 to 917 in 60ffcc3

	} else if (typeof storedCall.resolve === 'function') {
	// promise
	if (result.success) {
	storedCall.resolve(result.data);
	} else {
	storedCall.reject(result.error);
	}

I wonder which callback type the event handlers here use?

EDIT: It looks like event listeners are promise-based

capacitor/core/src/runtime.ts

Line 186 in 12c6294

const addListener = createPluginMethodWrapper('addListener');

capacitor/core/src/runtime.ts

Line 151 in 12c6294

const p = loadPluginImplementation().then(impl => {

[peitschie]

NB: use of promises may be significant, as most browsers will enqueue a Job for promise chains (usually via setImmediate or microtasks, sometimes via setTimeout(fn, 0)). This is something noted by the standards: https://tc39.es/ecma262/multipage/control-abstraction-objects.html#sec-promise-objects

Some discussion of this is here: https://stackoverflow.com/a/73745562/156169

[nemoneph]

Thanks for all the details.
It helps me to make some digging too, and I found the origin of the problem on Capacitor ! It's a recent change (few months ago).

Currently, the communication from native to JS is made via Web Message Listener (postMessage/onMessage) =>

capacitor/android/capacitor/src/main/java/com/getcapacitor/MessageHandler.java

Line 29 in 592ee86

postMessage(message.getData());

and

https://github.com/ionic-team/capacitor/blob/main/android/capacitor/src/main/assets/native-bridge.js#L740

For a quick test, I removed the function "returnResult" and add a console.log instead.
=> onmessage method is always throttled also without all the promise from returnResult !

But what is interesting is that I saw, there was another way to communicate with the webview, a "legacyBridge"

capacitor/android/capacitor/src/main/java/com/getcapacitor/MessageHandler.java

Line 41 in 592ee86

webView.addJavascriptInterface(this, "androidBridge");

it's documented here https://capacitorjs.com/docs/config
(see "useLegacyBridge")

So I turn on the legacyBridge in my capacitor.config.js =>

  "android": {
    "useLegacyBridge": true
  }

And now no throttled in the legimit use case, it's just work !

So the problem is with the new bridge with addWebMessageListener and postMessage/onMessage which for some reason is throttled by Chrome (like the setInterval/setTimeout/xmlHttpRequest heavy method)

https://github.com/ionic-team/capacitor/blob/main/android/capacitor/src/main/assets/native-bridge.js#L740

The new bridge was introduced here: #5427
The option to use the legacyBridge here : #6043 (because of others bugs)

So it's good to understand the problem, but using the legacyBridge wich may be suppress one day does not necessarily reassure me.
We need some feedback from the Ionic Team who knows the subject, @jcesarmobile perhaps ?

[nemoneph]

Don't know why, my previous message with a "solution" is hidden "This comment was marked as abuse."

[diachedelic]

Brilliant. Thanks so much for getting to the bottom of this @nemoneph , and finding a workaround.

[ingageco]

Good work explaining and digging into the issue @nemoneph . Hopefully we can get some feedback from the Ionic team and get this fixed without the legacy bridge, which seems to cause some other problems.

[ingageco]

For those creating their own plugins - it seems that an alternative to this is to use triggerJSEvent per the docs. I've tested it, and it works where notifyListeners was not working.

https://capacitorjs.com/docs/v4/core-apis/android#triggerjsevent

It is NOT a 1:1 replacement - you'll have to adapt to listening for the event (window.addEventListener) instead of MyPlugin.addListener

[christocracy]

I'm the author of @transistorsoft/capacitor-background-geolocation. I have over 9 years experience operating location APIs in the background so I know what to expect when it comes to receiving events from my plugins when the app is running in the background. Nice work @nemoneph. I ended up here while testing my example app for tracking location in the background. My javascript event-listeners to add a marker to a map cease after exactly 5 minutes in the background (confirmed with stop-watch). Once the app returns to the foreground, all those queued event-listeners fire all-at-once.

useLegacyBridge: true makes my problems all go away.

📂 capacitor.config.ts:

const config: CapacitorConfig = {
  .
  .
  .
  android: {
    useLegacyBridge: true
  }
};

Moderator: This comment by @nemoneph is incorrectly marked as "abuse".

Capacitor needs to find a way to fix this or developers are going to migrate to React Native and Flutter (where my background-geolocation plugins are waiting for them there).

[oliveryasuna]

Any updates on this? I feel like it's a big overlooked issue.

[MehulGosar]

Hey @nemoneph!! The issue is still there. Refer https://github.com/transistorsoft/capacitor-background-geolocation/issues/276 Also, we were already using useLegacyBridge:true in our application.
Current Behaviour: When I tested the app , the following behavior is seen: When the application is in the background, for 5 minutes the updates are happening fine. Post that the application is queuing up all the requests i.e. "throttling" them for the rest of the period. Once the application comes to foreground, the app is then making all the requests at once from the same place. Thus, making 150-200 location capture calls from the same location at the same timestamp are captured.

Here are some other piece of code to refer: I hope they help in deducing the issue because I'm not able to since past few weeks:

Capacitor info (npx cap doctor)
Capacitor Doctor
Latest Dependencies:
@capacitor/cli: 6.1.2
@capacitor/core: 6.1.2
@capacitor/android: 6.1.2
@capacitor/ios: 6.1.2.
Installed Dependencies:
@capacitor/ios: not installed
@capacitor/cli: 4.5.0
@capacitor/android: 4.3.0
@capacitor/core: 4.3.0
[success] Android looking great!

capacitor.config.json:
import { CapacitorConfig } from '@capacitor/cli';

const config: CapacitorConfig = {
appId: 'io.ionic.starter',
appName: 'MY APP NAME',
webDir: 'www',
bundledWebRuntime: false,
android: {
useLegacyBridge: true,

}

};

export default config;

**ionic.config.json: **

{

"name": "io.ionic.starter",

"integrations": {

"capacitor": {}

},

"type": "angular"

}

strings.xml:

MY APP NAME

MY APL NAME>

io.ionic.starter

io.ionic.starter

app.component.ts:

private initializeBackgroundListener() {
App.addListener('appStateChange', ({ isActive }) => {
//console.log("Is Active"+isActive)
if (lisActive) {
this.disableWebView()
}
else {
this.enableWebView();
}
});
}

disableWebView() {
const webview = document.querySelector('ion-app');
if (webview) {
webview.style.display = 'none'; // Disables WebView rendering
}
}

enableWebView() {
const webview = document.querySelector('ion-app');
if (webview) {
webview.style.display = 'block'; // Enables WebView rendering
}
}

package.json:

"name": "io.ionic.starter",

"version": "0.0.1",
"author": "Ionic Framework",
"homepage": "https://ionicframework.com/",
Debug
"scripts": {
"ng": "ng",
"start": "ng serve",
"build": "ng build",
"test": "ng test",
"lint": "ng lint",
"e2e": "ng e2e"

},

I'll be extremely grateful if you can confirm and help me understand where the code is going wrong. Thanks :)