[Bug Kiwix 3.6.0]Infinite loading icon in library tab

[Mitia99]

Describe the bug
Just an UI bug:Infinite loading icon in library tab
No issue with the library,zims are loaded.

Expected behavior
No more loading icon when zim are loaded.

Steps to reproduce the behavior:

1. Download zim content
2. Open library tab

Screenshots

Environment

• Version of Kiwix Android : 3.4.1
• Device : Huawei P10
• OS version : Android 9

[macgills]

Thank you for the report, will investigate and hopefully have a fix in not too long

[s-ayush2903]

@Mitia99 I'm sorry but I can't reproduce the issue, I tried ~5 times the procedure mentioned in the description but couldn't find it. Can you please answer one question: Does it always occur(via following the steps given in the issue desc.) or you just caught it once?

[Mitia99]

@s-ayush2903
I tried force stop and clear cache but same issue.

Video showing the issue
https://drive.google.com/file/d/1XKecR3z52jRgwCntXWLmA9xBakZZHoVB/view?usp=drivesdk

[Mitia99]

@s-ayush2903
It always occurs.

Additionnal information:
I did an update from version 3.3.1(i.e I downloaded the zim contents with 3.3.1).

[s-ayush2903]

No worries, I'll take a look

[Mitia99]

@s-ayush2903
I tried to clear app data(which removed all zim contents) but same issue.So I don't think it that the issue is linked with the update

[macgills]

Ahhh this is library, not Download, d'oh. So what this is doing is scanning your filesystem looking for zims. I am going to assume you are using an sd card? There is a possibility this is blocked by our file system detection, in Download do you see large entries ie >4GB, greyed out?

[Mitia99]

@macgills I'm using a 64go SD Card.Storage is set on internal

[macgills]

hmmmm so it is potentially an error in StorageObserver. Have you left the phone on this screen for an unreasonably long time? I wonder if it ever completes

[gouri-panda]

I'm encountering this problem too.

[macgills]

Would you like for this to be assigned to you then @gouri-panda ? The file search completes near instantaneously on my device

[s-ayush2903]

+1 To second line in the trailing comment, I still can't reproduce the issue

[gouri-panda]

Would you like for this to be assigned to you then @gouri-panda ? The file search completes near instantaneously on my device

yes @macgills

[gouri-panda]

@Mitia99 I'm facing this problem but in a different way. On my phone, it loads for approximately 3-4 minutes. Do you have this situation too?

[Mitia99]

@gouri-panda Yes same here,I figured out that it is taking ~ 2minutes but when I reduce kiwix app/switch to an other tab(e.g:download) and re-open kiwix/library tab,it is re-scanning very slowly.

[gouri-panda]

That's because it is downloading the content even if you switched the tab, that's why it is taking so much time.

[Mitia99]

@gouri-panda Oh,I see..I hope that you could improve the scan time.

[gouri-panda]

@Mitia99 it's taking the usual time ,but if you have slow processor than you can feel the delay like I have. My one phone is taking 2 to 3 minutes and the other phone is taking hardly 10 sec.

[Mitia99]

@gouri-panda I don't think that it is a processor issue.I am using a kirin 960(8 core cpu) which is pretty fast even in huge apps.Maybe,It will be better to limit the engine scan with only specific files/path(not the whole storage).

[gouri-panda]

@Mitia99 I looked over your storage.My hypothesis is that you have a lot of content in both your internal and external memory card so thats why it scans the whole thing after you dowload zim files.

[Mitia99]

@gouri-panda Yes,you're right.That's why making the engine to only scan specific folders/files types will be great.For users with large files in 128go/256go memory/SD Card it will take a very long time.

[gouri-panda]

@Mitia99 It will break us our certain feature like sharing our Zim files and download the Zim files over the internet. It won't add these files into library.

[macgills]

#2405 there is a solution outlined here

[Mitia99]

@macgills Thank you for the reference.Personnaly,I don't really want a manual folder selection but only an improved auto-scan(modify a bit the way that the engine scans).

[macgills]

It is quite difficult to "intelligently" scan as any number of apps eg torrent clients, browsers, dedicated downloaders, could be downloading files and be putting them anywhere on disk as could a user transferring manually. Our only guarantee is to look everywhere and really it shouldn't be that slow but here we are.
We have 2 options as I see it and we could do both of them.

1. coroutines rewrite to try and improve performance
2. a switch to deactivate full scanning and only look at Kiwix app folders

long term we could look at user configurable search locations but sooner or later we are going to have to start supporting the SAF which is going to change file access quite a lot and there is a lot of work that will go into UI/UX of that feature (unless we just give users a textbox and let them add any path they want but I imagine that being confusing and unfriendly) and that might get trashed, wasting dev time.

[gouri-panda]

@macgills Option 1 sounds good. Should I start working on this ticket?

[Mitia99]

Bug details update
•Device: Huawei P30 (no sd card)
•Memory usage : 90% of 128go
•Android 10
•Kiwix: 3.6.0

[kelson42]

@Mitia99 Thx for the update. It's really a pity you still suffer of it.

[Mitia99]

@kelson42 No problem. Btw, I noticed that rather than an infinite loading it is actually a slow library load.(My kiwix library size is about 700mo) Maybe it has something to do with my heavy memory usage (90% of 128go).

[kelson42]

@gouri-panda Do you still have this problem with 3.6.0+?

[gouri-panda]

@kelson42 We can close this issue now.

[kelson42]

@gouri-panda Why?

[gouri-panda]

I haven't encountered this issue recently. I have marked that, but I didn't know that this ticket was still open We may be fixed this issue in the storage issues.

[Mitia99]

@gouri-panda Can you re-open this?

It's back in 3.8.0

• Device : Huawei P30 (no sd card)
• OS version : Android 10

Thanks for investigating,

[MohitMaliFtechiz]

@kelson42 This issue seems duplicate of #3646. The user has the same issue as mentioned in #3646.

[kelson42]

@MohitMaliFtechiz How do you know that? We have no clue how many files are here on the SD cars!

[MohitMaliFtechiz]

@gouri-panda Can you re-open this?
It's back in 3.8.0
Device : Huawei P30 (no sd card)
OS version : Android 10
Thanks for investigating,

@kelson42 in the last update user confirmed that there is no SD card present in his phone. So here scanning is only performed inside the phones's storage, and he still facing the slow loading issue. Therefore, the issue is not with the SD card, it is with large numbers of files available in the storage because we scan the whole storage to find the ZIM files from every possible location of the storage.

@gouri-panda Yes same here, I figured out that it is taking ~ 2minutes.
@kelson42 No problem. Btw, I noticed that rather than an infinite loading it is actually a slow library load.(My kiwix library size is about 700mo) Maybe it has something to do with my heavy memory usage (90% of 128go).

The user confirms that there is a slow loading in the Library screen and that 90% of his storage is full(So there is a high probability that many files are available in his storage). I had tried to put 20K dummy files(with the size of 10KB) in my system #3650 (comment) to slow down the scanning while fixing the #3650, and at that time it took almost 19 seconds to load the library. Here user's device takes 2 minutes to scan and show the ZIM files in the Library screen, and we are facing the same issue in #3646.

[Mitia99]

Here user's device takes 2 minutes to scan and show the ZIM files in the Library screen

Not really. Kiwix show the ZIMs instantly and I can immediatly browse them after opening the app.

There's no major performance issues. It's more of a bad UI experience.

@kelson42 I also don't think that it's a duplicate issue.

[MohitMaliFtechiz]

Not really. Kiwix show the ZIMs instantly and I can immediatly browse them after opening the app.

@Mitia99 you were facing a slow-scanning issue as you mentioned earlier. Happy to know now you are not facing that issue.

There's no major performance issues. It's more of a bad UI experience.

For this, we have improved our UI, and enhanced the scanning process in #3674.