Skip to content

Commit

Permalink
Update Zstandard to Version 1.5.5
Browse files Browse the repository at this point in the history
Signed-off-by: Tino Reichardt <[email protected]>
  • Loading branch information
mcmilk committed Apr 5, 2023
1 parent e615c8c commit aaf1f12
Show file tree
Hide file tree
Showing 31 changed files with 942 additions and 627 deletions.
6 changes: 3 additions & 3 deletions C/7zVersion.h
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#define MY_VER_MAJOR 22
#define MY_VER_MINOR 01
#define MY_VER_BUILD 04
#define MY_VERSION_NUMBERS "22.01 ZS v1.5.4 R4"
#define MY_VER_BUILD 05
#define MY_VERSION_NUMBERS "22.01 ZS v1.5.5 R1"
#define MY_VERSION MY_VERSION_NUMBERS

#ifdef MY_CPU_NAME
Expand All @@ -10,7 +10,7 @@
#define MY_VERSION_CPU MY_VERSION
#endif

#define MY_DATE "2023-02-27"
#define MY_DATE "2023-04-05"
#undef MY_COPYRIGHT
#undef MY_VERSION_COPYRIGHT_DATE
#define MY_AUTHOR_NAME "Igor Pavlov, Tino Reichardt"
Expand Down
8 changes: 4 additions & 4 deletions C/zstd/LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ BSD License

For Zstandard software

Copyright (c) 2016-present, Facebook, Inc. All rights reserved.
Copyright (c) Meta Platforms, Inc. and affiliates. All rights reserved.

Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
Expand All @@ -14,9 +14,9 @@ are permitted provided that the following conditions are met:
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.

* Neither the name Facebook nor the names of its contributors may be used to
endorse or promote products derived from this software without specific
prior written permission.
* Neither the name Facebook, nor Meta, nor the names of its contributors may
be used to endorse or promote products derived from this software without
specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
Expand Down
17 changes: 15 additions & 2 deletions C/zstd/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ The file structure is designed to make this selection manually achievable for an
`ZSTD_LIB_COMPRESSION, ZSTD_LIB_DECOMPRESSION`, `ZSTD_LIB_DICTBUILDER`,
and `ZSTD_LIB_DEPRECATED` as `0` to forgo compilation of the
corresponding features. This will also disable compilation of all
dependencies (eg. `ZSTD_LIB_COMPRESSION=0` will also disable
dependencies (e.g. `ZSTD_LIB_COMPRESSION=0` will also disable
dictBuilder).

- There are a number of options that can help minimize the binary size of
Expand Down Expand Up @@ -125,7 +125,7 @@ The file structure is designed to make this selection manually achievable for an
`ZSTD_getErrorName` (implied by `ZSTD_LIB_MINIFY`).

Finally, when integrating into your application, make sure you're doing link-
time optimation and unused symbol garbage collection (via some combination of,
time optimization and unused symbol garbage collection (via some combination of,
e.g., `-flto`, `-ffat-lto-objects`, `-fuse-linker-plugin`,
`-ffunction-sections`, `-fdata-sections`, `-fmerge-all-constants`,
`-Wl,--gc-sections`, `-Wl,-z,norelro`, and an archiver that understands
Expand Down Expand Up @@ -155,6 +155,19 @@ The file structure is designed to make this selection manually achievable for an
- The build macro `ZSTD_NO_INTRINSICS` can be defined to disable all explicit intrinsics.
Compiler builtins are still used.

- The build macro `ZSTD_DECODER_INTERNAL_BUFFER` can be set to control
the amount of extra memory used during decompression to store literals.
This defaults to 64kB. Reducing this value reduces the memory footprint of
`ZSTD_DCtx` decompression contexts,
but might also result in a small decompression speed cost.

- The C compiler macros `ZSTDLIB_VISIBLE`, `ZSTDERRORLIB_VISIBLE` and `ZDICTLIB_VISIBLE`
can be overridden to control the visibility of zstd's API. Additionally,
`ZSTDLIB_STATIC_API` and `ZDICTLIB_STATIC_API` can be overridden to control the visibility
of zstd's static API. Specifically, it can be set to `ZSTDLIB_HIDDEN` to hide the symbols
from the shared library. These macros default to `ZSTDLIB_VISIBILITY`,
`ZSTDERRORLIB_VSIBILITY`, and `ZDICTLIB_VISIBILITY` if unset, for backwards compatibility
with the old macro names.

#### Windows : using MinGW+MSYS to create DLL

Expand Down
55 changes: 55 additions & 0 deletions C/zstd/allocations.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
/*
* Copyright (c) Meta Platforms, Inc. and affiliates.
* All rights reserved.
*
* This source code is licensed under both the BSD-style license (found in the
* LICENSE file in the root directory of this source tree) and the GPLv2 (found
* in the COPYING file in the root directory of this source tree).
* You may select, at your option, one of the above-listed licenses.
*/

/* This file provides custom allocation primitives
*/

#define ZSTD_DEPS_NEED_MALLOC
#include "zstd_deps.h" /* ZSTD_malloc, ZSTD_calloc, ZSTD_free, ZSTD_memset */

#include "mem.h" /* MEM_STATIC */
#define ZSTD_STATIC_LINKING_ONLY
#include "zstd.h" /* ZSTD_customMem */

#ifndef ZSTD_ALLOCATIONS_H
#define ZSTD_ALLOCATIONS_H

/* custom memory allocation functions */

MEM_STATIC void* ZSTD_customMalloc(size_t size, ZSTD_customMem customMem)
{
if (customMem.customAlloc)
return customMem.customAlloc(customMem.opaque, size);
return ZSTD_malloc(size);
}

MEM_STATIC void* ZSTD_customCalloc(size_t size, ZSTD_customMem customMem)
{
if (customMem.customAlloc) {
/* calloc implemented as malloc+memset;
* not as efficient as calloc, but next best guess for custom malloc */
void* const ptr = customMem.customAlloc(customMem.opaque, size);
ZSTD_memset(ptr, 0, size);
return ptr;
}
return ZSTD_calloc(1, size);
}

MEM_STATIC void ZSTD_customFree(void* ptr, ZSTD_customMem customMem)
{
if (ptr!=NULL) {
if (customMem.customFree)
customMem.customFree(customMem.opaque, ptr);
else
ZSTD_free(ptr);
}
}

#endif /* ZSTD_ALLOCATIONS_H */
35 changes: 30 additions & 5 deletions C/zstd/bits.h
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ MEM_STATIC unsigned ZSTD_countTrailingZeros32_fallback(U32 val)
{
assert(val != 0);
{
static const int DeBruijnBytePos[32] = {0, 1, 28, 2, 29, 14, 24, 3,
static const U32 DeBruijnBytePos[32] = {0, 1, 28, 2, 29, 14, 24, 3,
30, 22, 20, 15, 25, 17, 4, 8,
31, 27, 13, 23, 21, 19, 16, 7,
26, 12, 18, 6, 11, 5, 10, 9};
Expand All @@ -30,7 +30,7 @@ MEM_STATIC unsigned ZSTD_countTrailingZeros32(U32 val)
assert(val != 0);
# if defined(_MSC_VER)
# if STATIC_BMI2 == 1
return _tzcnt_u32(val);
return (unsigned)_tzcnt_u32(val);
# else
if (val != 0) {
unsigned long r;
Expand Down Expand Up @@ -69,7 +69,7 @@ MEM_STATIC unsigned ZSTD_countLeadingZeros32(U32 val)
assert(val != 0);
# if defined(_MSC_VER)
# if STATIC_BMI2 == 1
return _lzcnt_u32(val);
return (unsigned)_lzcnt_u32(val);
# else
if (val != 0) {
unsigned long r;
Expand All @@ -92,7 +92,7 @@ MEM_STATIC unsigned ZSTD_countTrailingZeros64(U64 val)
assert(val != 0);
# if defined(_MSC_VER) && defined(_WIN64)
# if STATIC_BMI2 == 1
return _tzcnt_u64(val);
return (unsigned)_tzcnt_u64(val);
# else
if (val != 0) {
unsigned long r;
Expand Down Expand Up @@ -123,7 +123,7 @@ MEM_STATIC unsigned ZSTD_countLeadingZeros64(U64 val)
assert(val != 0);
# if defined(_MSC_VER) && defined(_WIN64)
# if STATIC_BMI2 == 1
return _lzcnt_u64(val);
return (unsigned)_lzcnt_u64(val);
# else
if (val != 0) {
unsigned long r;
Expand Down Expand Up @@ -172,4 +172,29 @@ MEM_STATIC unsigned ZSTD_highbit32(U32 val) /* compress, dictBuilder, decodeCo
return 31 - ZSTD_countLeadingZeros32(val);
}

/* ZSTD_rotateRight_*():
* Rotates a bitfield to the right by "count" bits.
* https://en.wikipedia.org/w/index.php?title=Circular_shift&oldid=991635599#Implementing_circular_shifts
*/
MEM_STATIC
U64 ZSTD_rotateRight_U64(U64 const value, U32 count) {
assert(count < 64);
count &= 0x3F; /* for fickle pattern recognition */
return (value >> count) | (U64)(value << ((0U - count) & 0x3F));
}

MEM_STATIC
U32 ZSTD_rotateRight_U32(U32 const value, U32 count) {
assert(count < 32);
count &= 0x1F; /* for fickle pattern recognition */
return (value >> count) | (U32)(value << ((0U - count) & 0x1F));
}

MEM_STATIC
U16 ZSTD_rotateRight_U16(U16 const value, U32 count) {
assert(count < 16);
count &= 0x0F; /* for fickle pattern recognition */
return (value >> count) | (U16)(value << ((0U - count) & 0x0F));
}

#endif /* ZSTD_BITS_H */
2 changes: 1 addition & 1 deletion C/zstd/bitstream.h
Original file line number Diff line number Diff line change
Expand Up @@ -396,7 +396,7 @@ MEM_STATIC BIT_DStream_status BIT_reloadDStreamFast(BIT_DStream_t* bitD)
* This function is safe, it guarantees it will not read beyond src buffer.
* @return : status of `BIT_DStream_t` internal register.
* when status == BIT_DStream_unfinished, internal register is filled with at least 25 or 57 bits */
MEM_STATIC BIT_DStream_status BIT_reloadDStream(BIT_DStream_t* bitD)
MEM_STATIC FORCE_INLINE_ATTR BIT_DStream_status BIT_reloadDStream(BIT_DStream_t* bitD)
{
if (bitD->bitsConsumed > (sizeof(bitD->bitContainer)*8)) /* overflow detected, like end of stream */
return BIT_DStream_overflow;
Expand Down
4 changes: 4 additions & 0 deletions C/zstd/compiler.h
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,10 @@ void __msan_poison(const volatile void *a, size_t size);
/* Returns the offset of the first (at least partially) poisoned byte in the
memory range, or -1 if the whole range is good. */
intptr_t __msan_test_shadow(const volatile void *x, size_t size);

/* Print shadow and origin for the memory range to stderr in a human-readable
format. */
void __msan_print_shadow(const volatile void *x, size_t size);
#endif

#if ZSTD_ADDRESS_SANITIZER && !defined(ZSTD_ASAN_DONT_POISON_WORKSPACE)
Expand Down
8 changes: 4 additions & 4 deletions C/zstd/huf_decompress.c
Original file line number Diff line number Diff line change
Expand Up @@ -696,7 +696,7 @@ void HUF_decompress4X1_usingDTable_internal_fast_c_loop(HUF_DecompressFastArgs*

/* Copy the arguments to local variables */
ZSTD_memcpy(&bits, &args->bits, sizeof(bits));
ZSTD_memcpy(&ip, &args->ip, sizeof(ip));
ZSTD_memcpy((void*)(&ip), &args->ip, sizeof(ip));
ZSTD_memcpy(&op, &args->op, sizeof(op));

assert(MEM_isLittleEndian());
Expand Down Expand Up @@ -779,7 +779,7 @@ void HUF_decompress4X1_usingDTable_internal_fast_c_loop(HUF_DecompressFastArgs*

/* Save the final values of each of the state variables back to args. */
ZSTD_memcpy(&args->bits, &bits, sizeof(bits));
ZSTD_memcpy(&args->ip, &ip, sizeof(ip));
ZSTD_memcpy((void*)(&args->ip), &ip, sizeof(ip));
ZSTD_memcpy(&args->op, &op, sizeof(op));
}

Expand Down Expand Up @@ -1476,7 +1476,7 @@ void HUF_decompress4X2_usingDTable_internal_fast_c_loop(HUF_DecompressFastArgs*

/* Copy the arguments to local registers. */
ZSTD_memcpy(&bits, &args->bits, sizeof(bits));
ZSTD_memcpy(&ip, &args->ip, sizeof(ip));
ZSTD_memcpy((void*)(&ip), &args->ip, sizeof(ip));
ZSTD_memcpy(&op, &args->op, sizeof(op));

oend[0] = op[1];
Expand Down Expand Up @@ -1599,7 +1599,7 @@ void HUF_decompress4X2_usingDTable_internal_fast_c_loop(HUF_DecompressFastArgs*

/* Save the final values of each of the state variables back to args. */
ZSTD_memcpy(&args->bits, &bits, sizeof(bits));
ZSTD_memcpy(&args->ip, &ip, sizeof(ip));
ZSTD_memcpy((void*)(&args->ip), &ip, sizeof(ip));
ZSTD_memcpy(&args->op, &op, sizeof(op));
}

Expand Down
2 changes: 1 addition & 1 deletion C/zstd/pool.c
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@


/* ====== Dependencies ======= */
#include "allocations.h" /* ZSTD_customCalloc, ZSTD_customFree */
#include "zstd_deps.h" /* size_t */
#include "debug.h" /* assert */
#include "zstd_internal.h" /* ZSTD_customCalloc, ZSTD_customFree */
#include "pool.h"

/* ====== Compiler specifics ====== */
Expand Down
2 changes: 1 addition & 1 deletion C/zstd/threading.c
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ static unsigned __stdcall worker(void *arg)
void* (*start_routine)(void*);
void* thread_arg;

/* Inialized thread_arg and start_routine and signal main thread that we don't need it
/* Initialized thread_arg and start_routine and signal main thread that we don't need it
* to wait any longer.
*/
{
Expand Down
Loading

0 comments on commit aaf1f12

Please sign in to comment.