enable gzStreamUpdater progress update callback

[sharandac]

In the current version, the progress callback is disabled when the ESP32 is used. Could it be reactivated with the following code?

      Update.onProgress([]( size_t done, size_t total ) {
        // check if done not zero to prevent division by zero
        if ( done != 0 ) {
          gzProgressCallback( ( 100*done ) / total );
        }
      });
      show_progress = false;

For that would be interesting to use the upstream from here instead of a standalone version. 248

[tobozo]

@sharandac thanks for your constructive feedback

I've commited some changes based on your suggestion, available on the 1.0.5-beta branch.

While testing that code, I found out adding a gzProgressCallback( 100 ); on Update.isFinished() was necessary to achieve the full progress. I'm not sure this applies to all situations though.

[sharandac]

Thanks for the great library and the quick response! works great!

Another question that I find interesting right now. Is there a way to reduce the quite large RAM requirement? In a current project I have very little IRAM but a lot of PSRAM to do crazy things. Therefore it happens to me very often that the stream starts, but a new constructor then crashes the program. Unfortunately you can't teach the arduino-ESP32 framework to use PSRAM automatically with a new constructor. For this reason I usually use ps_*alloc or *alloc. I have also written a small "wrapper" for this.

#ifndef _ALLOC_H
    #define _ALLOC_H

    #if defined( BOARD_HAS_PSRAM )
        #include <stddef.h>
        #include <stdbool.h>
        #include <esp32-hal-psram.h>

        #define MALLOC         ps_malloc            /** @brief malloac from PSRAM */
        #define CALLOC         ps_calloc            /** @brief calloc from PSRAM */
        #define REALLOC        ps_realloc           /** @brief realloc from PSRAM */
    #else
        #define MALLOC         malloc               /** @brief malloac from normal heap */
        #define CALLOC         calloc               /** @brief calloc from normal heap */
        #define REALLOC        realloc              /** @brief realloc from normal heap */
    #endif // BOARD_HAS_PSRAM
#endif // _ALLOC_H

[tobozo]

Nice suggestion, makes me wonder if the user should decide to enable or disable psram (when applicable)?

There are a few reasons for that:

• The dictionary memory must be aligned or random crashes occur
• Psram isn't as fast as iram: there's a speed tradeoff not every situation can support.
• Depending on the core using it, Psram can be a burden and even lead to crashes.

I just finished testing bool BaseUnpacker::setPsram( bool enable ); and pushed the changes on the 1.0.5-beta branch

[sharandac]

Thanks again. I think your implementation is exactly the right way! The programmer should rather decide that, would have been my decision too.
PSRAM is a bit tricky, but manageable.

[tobozo]

@sharandac does closing this issue mean the implementation works for you ?

since your suggestions gave birth to the idea, you may be interested by the very experimental PSRamFS library and the recently added support for ramdisk

[sharandac]

@tobozo the first intention was that it would work for me ... now I am curious about the PSRamFS :)

[sharandac]

@tobozo Maybe I missed it, but is there a way to unpack from a char array?

[tobozo]

sure, just create some object inheriting Stream* and implement the read() readBytes() and available() methods, then pass it to gzStreamUpdater

[sharandac]

i would have done that next. i was also thinking if something like that already existed. would be a nice function, pointer to the data and its size.

[tobozo]

It'll eventually be added to esp32-psramfs as an extra RomDiskStream object signature.

Initialization will be implemented as follows:

const unsigned char my_gz_bytes[] = {
// (... all the bytes of your gz file)
};
size_ t my_gz_bytes_count = 16787; // how many bytes in the array

void unpack() {
  // init your TARGZUnpacker object
  TarGzUnpacker *TARGZUnpacker = new TarGzUnpacker();
  RomDiskStream myFakeGzStream( my_gz_bytes, my_gz_bytes_count );
  Stream *streamptr = &myFakeGzStream;
  TARGZUnpacker->tarGzStreamExpander( streamptr, PSRamFS );
}

[edit] and there's a RomDiskStream class in esp32-psramfs

[edit2] here's another more generalistic implementation (with read and write)