App takes much longer to load when App is built with sentry sdk #442
Comments
|
Android Profiler is not helpful. is there a verbose debug level we can set to get more diagnostics and see what is going on in sentry sdk during startup? Is there a way to setup trace in sentry sdk? |
|
Note that we don't use the native-sdk integration (breakpad etc...). |
|
hey @tiwariashish86 thanks for reporting this. The log level is configured this way: To enable logging, set this flag to are you using the artifact If using Do you use the Those numbers are running on debug or release mode? If you uninstall the APK completely and install it again (fresh install), can you reproduce the same delay? Do you have the release health feature enabled? it's disabled by default and enabled like this: Did you experience this using older versions? 2.x or only after upgrading to 2.1.2? Could you disable the automatic breadcrumbs and see if this helps somehow? some integrations depend on Almost 10 seconds is... a lot, we haven't done benchmarking yet, but it's doubtful to be that much (this could cause an ANR on every single run). We need to do some IO on the main thread due to the nature of such SDK, like to create the caching folders, but this would slow down a few milliseconds only in the very 1st run and not in next ones. When running our sample, which is a very light App. of course, the full startup with all the integrations enabled takes less than a sec. I'm not sure what's causing this, but please take the time to answer all those questions and then we are gonna do a deep investigation, thanks again for reporting this and let's keep in the loop. |
marandaneto
added
good first issue
|
Hey @marandaneto thanks a lot for the quick answer. We know it's a tough bug as we don't have too much info on what's going but wanted to get the ball rolling. I can answer some questions for Ashish (Ashish please double check all my answers) as I did the integration of the sdk.
For context what we call an asset build is our final production build, where we package tons of files (like 20,000 ?) in the apk. I believe this could be the problem, where some api perform poorly because of that (like trying to scan the apk directory content repeatedly, not sure). One thing I wonder is how hard it would be for us to build the sdk locally, so that we can run the Android profiler locally. |
|
hey @bsergean thanks for a prompt reply.
Do you use sentry.io or on-premise?
You just need to add a breakpoint here: We do need to list the assets folder, but this also happens on a background thread: A trick thing here is if you init. the SDK and right after you capture an event using Sentry, the background thread above is gonna Yes, I believe it's rather simple to build the SDK locally if you fork this repo and press play on the sample project, it runs immediately, so adding this repo as a dependency of your project instead of the maven packages would work seamlessly I believe, but again, sources are also published so I believe you don't even need that to be able to debug things. Yes, it's a tough one but we'll figure this out, thanks for the feedback so far. |
|
@marandaneto I am using release build. Issue is reproducible 100% with sentry sdk in all cases(fresh vs old install) |
|
So when I disable auto-init in manifest using following (note that I have not called init anywhere yet), app loads fine.
With Above configuration ('io.sentry:sentry-android-core:2.1.2') |
|
@tiwariashish86 yep but in this case, it disables (by removing it) the SDK completely. Could you take the time and try the things I've mentioned in my previous messages? |
|
Couple of questions @bsergean has already answered. Here is what I tried after that and findings: a. Disabled AutoBreadCrums as you mentioned. It made no difference. |
|
I see one call here:
sentry-android-core/src/main/java/io/sentry/android/core/DefaultAndroidEventProcessor.java
12:import android.content.res.AssetManager;
898: AssetManager assets = context.getAssets();
Code is this. Ashish can you try to make this function a no-op ?
private @nullable String[] getProGuardUuids() {
AssetManager assets = context.getAssets();
try {
String[] files = assets.list("");
List<String> listFiles = Arrays.asList(files != null ? files : new String[0]);
if (listFiles.contains("sentry-debug-meta.properties")) {
try (InputStream is = assets.open("sentry-debug-meta.properties")) {
Properties properties = new Properties();
properties.load(is);
String uuid = properties.getProperty("io.sentry.ProguardUuids");
if (uuid != null && !uuid.isEmpty()) {
return uuid.split("\\|", -1);
}
options
.getLogger()
.log(SentryLevel.INFO, "io.sentry.ProguardUuids property was not found.");
} catch (IOException e) {
options.getLogger().log(SentryLevel.ERROR, "Error getting Proguard UUIDs.", e);
}
options
.getLogger()
.log(SentryLevel.INFO, "io.sentry.ProguardUuids property was not found.");
} else {
options.getLogger().log(SentryLevel.INFO, "Proguard UUIDs file not found.");
}
} catch (IOException e) {
options.getLogger().log(SentryLevel.ERROR, "Error listing Proguard files.", e);
}
return null;
}
… On Jun 2, 2020, at 10:33 PM, tiwariashish86 ***@***.***> wrote:
Couple of questions @bsergean <https://github.com/bsergean> has already answered. Here is what I tried after that and findings:
a. Disabled AutoBreadCrums as you mentioned. It made no difference.
b. Commented out All calls to captureEvent, addBreadcrum and had only init(manual init in Application's onCreate)enabled. That made no difference either and issue was still there.
c. Init(manual init, disabled auto init) itself runs pretty fast, so it does not seem like init is the bottleneck here so that is weird because if I disable the init(app works fine since sentry sdk is disabled completely) but if I enable it call to init executes pretty fast but there is certainly a good amount of delay between Application's onCreate(Sentry init is manually called here) and Main Activity's onCreate.
d. Ran the proxy for network requests, first network request appears in the proxy after app has successfully loaded so that does not seem to be causing any issue as well.
e. Disabling Health feature did not make any difference as well.
f. tried some older versions of sentry sdk and went as far as to 2.0 but still see the issue.
g. I see some people have complained about similar delay here getsentry/sentry-java#708 <getsentry/sentry-java#708> . not sure if this one is related but seems to be fixed in 1.7.
h. I ran appTrace with perfetto but did not see any troublesome issue there. Android Studio's CPU profiler is not giving any useful information as well. May be that is probably I did not build sentry sdk locally so that is what I am going to do next.
i. surprisingly Android Q performs alot better than Android P and previous. Issue is practically non-existent on Android Q. So may be build settings (TargetSDK, minSDK and gradle version) differences causing the issue. Our app supports till API 19 and built with gradle 5.5 though that may not be the case.
J. is there any call to AssetManager in sentry sdk since we are seeing the issue only when app is built with assets?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#442 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC2O6UI5EABIEU3ATJNAMZLRUXOEHANCNFSM4NOOW4UA>.
|
|
Adding to previous, also I tried to make init asynchronous, that did not make any difference either. I saw the same delay. So seems like something else is going on after init behind the scenes. Enabling sentry logging was not useful as well. |
|
Manoel maybe one way to reproduce the problem would be to add say 20,000 assets to an apk, and see if init gets slower.
… On Jun 2, 2020, at 10:41 PM, tiwariashish86 ***@***.***> wrote:
Adding to previous, also I tried to make init asynchronous, that did not make any difference either. I saw the same delay. So seems like something else is going on after init behind the scenes. Enabling sentry logging was not useful as well.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#442 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC2O6UN7TQNUWIBNZQCG5NTRUXPBXANCNFSM4NOOW4UA>.
|
|
@bsergean yeah I have seen that but the thing is that call is asynchronous so should not have blocked the main thread. |
maybe there's some concurrency (using edit: |
yeah I'll try to figure this out, how the files look like? what are their extension and average size? |
could you enable the Sentry logging and filter the logcat using |
this class is package-private, so you can remove the Init. the SDK manually like this: If the App. loads fast and normally, we can isolate the problem to this class |
|
I've taken a shot here #445 I guess this is the root cause of the issue, if you confirm that the test above solves the problem, I'll get this into the next version right away. thanks for the collaboration so far, we are aiming to make it better day by day and opening issues and helping us to find and fix them is all we ask and need ❤️ |
|
Thanks so much Manoel.
Ashish, this is how you want to try to init the SDK for troubleshooting. If that makes the slow down go away we'll know the culprit is the interaction with the assetManager indeed.
… On Jun 3, 2020, at 12:57 AM, Manoel Aranda Neto ***@***.***> wrote:
SentryAndroid.init(
this,
options -> {
options.getEventProcessors().remove(1);
});
|
|
Above experiment did not solve the issue. Looking more closely on sentry logs. Here are the sentry logs : So if you see the last two line the time difference is 24 seconds and this is the time that last line(Proguard) takes to show up on screen. So this tells me that probably looking for Proguard UUIDs file is taking time(may be some lock contention). Is there a way to disable that completely without recompiling the sdk? otherwise would recompile locally and check it out. Also, @marandaneto Can we please release new sdk with your fix? |
|
https://stackoverflow.com/questions/16911558/android-assetmanager-list-incredibly-slow <https://stackoverflow.com/questions/16911558/android-assetmanager-list-incredibly-slow>
I just found that link (ok it's stack overflow) that says that listing all assets can be super slow (especially for us with all our assets). Hopefully this is the smoking gun.
… On Jun 3, 2020, at 1:39 AM, Manoel Aranda Neto ***@***.***> wrote:
I've taken a shot here #445 <#445>
I guess this is the root cause of the issue, if you confirm that the test above solves the problem, I'll get this into the next version right away.
thanks for the collaboration so far, we are aiming to make it better day by day and opening issues and helping us to find and fix them is all we ask and need ❤️
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#442 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC2O6UMY7QFZE7ETQMH72J3RUYD5RANCNFSM4NOOW4UA>.
|
|
So I managed to run Perfetto on Android Q. Though the issue is less worse on Android Q, it does show lock contention in AssetManager and that overlaps with App startup time. Also shows around 5 seconds of delay on Android Q(much worse before that). I am attaching screenshots for the app. |
|
hey @tiwariashish86 thanks for that. |
|
hey @tiwariashish86 https://github.com/getsentry/sentry-android/releases/tag/2.1.5 |
|
that actually crashed with following error in Application's onCreate:
|
|
@tiwariashish86 that's odd, I've 4 devices here and it runs just fine, let me look into it. |
|
Ok With break point, I see it is hitting
|
|
@tiwariashish86 yep, the |
marandaneto
added
bug
|
I'm stealing the thunder from Ashish but it looks like 2.1.6 is the winner !!!! no more crash and lag gone. Thanks so much @marandaneto |
|
hey, @bsergean and @tiwariashish86 thanks for the collaboration, yeah lesson learned. |
|
Thanks @marandaneto for your help!! |
Platform:
IDE:
Build system:
Android Gradle Plugin:
Sentry Android Gradle Plugin:
Proguard/R8: both cases
sentry-android installed with:
The version of sentry-android:
2.1.2
I have the following issue:
When android app is built with sentry sdk it takes around 10 seconds to load the app.
from Logcat :
Displayed => This value represents the amount of time elapsed between launching the process and finishing drawing the corresponding activity on the screen
The issue is specially worse with Asset build on Android.
Steps to reproduce:
Just do an Asset build on android with sentry SDK and launch the app. After that observe approx 10 seconds of black screen before Application finally starts. Sentry SDK inclusion in built is causing significant app startup delay
Actual result:
App startup is significantly delayed.
Expected result:
App startup time should be same as app without Sentry SDK
The text was updated successfully, but these errors were encountered: