Skipping text/JSON file

[frankcohen]

Hi, decompressing an archive skips a text file containing JSON encoded data. It successfully decompresses audio (.m4a) and video (.mpg) binary files in the same archive.

I am creating the TAR archive on a Mac using:
COPYFILE_DISABLE=1 tar czf startup.tar.gz --exclude ".DS_Store" startup
Without these flags ESP32-targz skips all files after the .DS_Store and other files beginning with a period.

What do you think?

-Frank

[frankcohen]

I recompiled with debug logging turned-on. I'm seeing this:

[D][ESP32-targz-lib.cpp:634] tarEndCallBack(): Total expanded bytes: 2692363, heap free: 206964

[D][ESP32-targz-lib.cpp:574] tarHeaderCallBack(): Ignoring extended data.

[E][ESP32-targz-lib.cpp:784] tarStreamWriteCallback(): [TAR ERROR] Written length differs from buffer length FC2 (unpacked bytes:2692363, expected: 30, returned: 0)!

I tried building a new tar file that only contains the text file, and no other files, and it works:

[V][ESP32-targz-lib.cpp:789] tarStreamWriteCallback(): [TAR INFO] tarStreamWriteCallback wrote 418 bytes to startup/startup.jsn

▓ 100%
[D][ESP32-targz-lib.cpp:634] tarEndCallBack(): Total expanded bytes: 11170, heap free: 206300

[TAR DEBUG]: tar expanding done!
[TAR DEBUG]: tar expanding done!
[D][ESP32-targz-lib.cpp:1199] gzUncompress(): decompressed 11170 bytes

[D][ESP32-targz-lib.cpp:1696] tarGzExpanderNoTempFile(): uzLib expander finished!

[D][ESP32-targz-lib.cpp:1699] tarGzExpanderNoTempFile(): [GZ Info] FreeBytes after expansion=-1379041280

So now I am confused... I am thinking the next step is to see if the tar library has run out of memory or the SPI bus for the SD card is failing. I could use some pointers here...

-Frank

[frankcohen]

I got it to work... I modified

line 45 of https://github.com/tobozo/ESP32-targz/blob/master/src/ESP32-targz-lib.cpp
changed to: if( ( proper->type == TAR::T_NORMAL ) || ( proper->type == TAR::T_EXTENDED ) ) {

and line 652 of https://github.com/tobozo/ESP32-targz/blob/master/src/ESP32-targz-lib.cpp
if( ( proper->type == TAR::T_NORMAL ) || ( proper->type == TAR::T_EXTENDED ) ) {

Without these changes to detect TAR:: T_EXTENDED() an 11,054 uncompressed file containing JSON encoded data ends the TAR decompress feature, regardless of there being more TAR files to [decompress.]

-Frank

[tobozo]

Thanks for spotting that!

Looks like I misunderstood the use of T_EXTENDED in TinyUntar, and also got misled by --no-xattrs tar option (the default on linux) into believing the data was only holding file attributes.

I'll produce a fix release soon.