Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed: Not able to upload 512MB+ custom apps anymore. #3801

Merged
merged 6 commits into from
Jun 15, 2024
Merged

Conversation

MohitMaliFtechiz
Copy link
Collaborator

Fixes #3511

  • Introduced the splitting zim file with 500MB for custom apps. Since the bundle has a limit for a file to upload it uses the play asset delivery mode, and for new custom apps, playStore does not allow to upload the apk (with apk we can upload more than 1GB file). So to address this issue we have introduced a feature in libzim where we can load the zim files via the fd list.
  • We have modified our code to upload the files with 500MB chunks in the asset folder, and later we are accessing these files from the asset folder and creating the archive object with the help of the new libzim feature.
  • Currently CI will fail because we are using the new libzim feature in this PR. But currently new binding of java-libkiwix is not released once the new binding is released and we start using that then this error will resolve.

@MohitMaliFtechiz MohitMaliFtechiz marked this pull request as draft April 22, 2024 13:21
@kelson42
Copy link
Collaborator

@MohitMaliFtechiz Thx for rhis PR, but its important to notice that we won't be able to put more than 3 chunks=1.5GB as asset. It does not fully fix the problem AFAIK

@MohitMaliFtechiz
Copy link
Collaborator Author

MohitMaliFtechiz commented Apr 22, 2024

@kelson42 We have a total combined limit of 1GB for install-time that we are currently using, above this, we still have the problem.

Copy link

codecov bot commented Apr 24, 2024

Codecov Report

Attention: Patch coverage is 12.50000% with 21 lines in your changes missing coverage. Please review.

Project coverage is 53.53%. Comparing base (d099876) to head (14b782a).

Files Patch % Lines
...org/kiwix/kiwixmobile/core/reader/ZimFileReader.kt 8.33% 11 Missing ⚠️
.../kiwix/kiwixmobile/core/main/CoreReaderFragment.kt 33.33% 3 Missing and 1 partial ⚠️
...iwix/kiwixmobile/core/reader/ZimReaderContainer.kt 0.00% 3 Missing ⚠️
...rg/kiwix/kiwixmobile/core/webserver/KiwixServer.kt 0.00% 2 Missing ⚠️
...bile/nav/destination/reader/KiwixReaderFragment.kt 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #3801      +/-   ##
============================================
- Coverage     53.57%   53.53%   -0.04%     
+ Complexity     1399     1398       -1     
============================================
  Files           301      301              
  Lines         11556    11564       +8     
  Branches       1521     1523       +2     
============================================
  Hits           6191     6191              
- Misses         4371     4377       +6     
- Partials        994      996       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@kelson42
Copy link
Collaborator

@MohitMaliFtechiz What is missing here to complete this PR?

@MohitMaliFtechiz
Copy link
Collaborator Author

MohitMaliFtechiz commented Jun 13, 2024

@MohitMaliFtechiz What is missing here to complete this PR?

@kelson42 The testing of this PR is missing(uploading to the playStore). I am on the way to testing the app-bundle with the combination of (install-time, and fast-follow) with a 1.5GB split file. We had only decided that time we would use the combination of install-time and fast-follow to upload the ZIM file up to 1.5GB but we had not tested that, how it would perform.

* Introduced the splitting zim file with 500MB for custom apps. Since the bundle has a limit for a file to upload it uses the play asset delivery mode, and for new custom apps, playStore does not allow to upload the apk (with apk we can upload more than 1GB file). So to address this issue we have introduced a feature in libzim where we can load the zim files via the fd list.

* We have modified our code to upload the files with 500MB chunks in the asset folder, and later we are accessing these files from the asset folder and creating the archive object with the help of the new libzim feature.
* It was not showing the article when fresh application is installed on device with list of fds, and after the first install it works normally(It starts loading the article). So we have placed a fix for this.
@kelson42
Copy link
Collaborator

For now, only install-time.

@MohitMaliFtechiz
Copy link
Collaborator Author

@kelson42 I have tested the uploading of the aab file on playStore only with install-time mode, and it uploaded the 1.47GB file with this install-time delivery mode without having the combination of (install-time and fast-follow). Also, I can read the ZIM file after just installing the application. So, our first issue is fixed, where we cannot upload the aab file of more than 512MB in play asset delivery mode.

1718360751056 Screenshot from 2024-06-14 14-44-28

I also tested uploading aab files larger than 1.5GB. I successfully uploaded a 1.52GB aab file to PlayStore. However, I have not fully tested it because the ZIM chunks were made from different files, and I don't have a single ZIM file between 1.5GB and 2GB. Downloading large files from the server taking too long(it almost took me 4-5 hours and still downloading is not completed). Nevertheless, the file uploaded successfully under the install-time delivery mode, and the size displayed during download is over 1.5GB, so we can assume it will be available immediately after downloading the application. We will test this scenario while uploading the WikiSpecies, and Medical Wikipedia(wikimed) which have large files.

Screenshot from 2024-06-14 17-25-29 media_20240614_173859_1628767402509677146 media_20240614_173859_3421598111586842905

@MohitMaliFtechiz MohitMaliFtechiz marked this pull request as ready for review June 14, 2024 12:25
@kelson42 kelson42 merged commit ab5fb35 into main Jun 15, 2024
9 of 10 checks passed
@kelson42 kelson42 deleted the Fix#3511 branch June 15, 2024 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[REGRESSION] Not able to upload 512MB+ custom apps anymore
3 participants