annotate CSP2/CSP2_env/env-d9b9114564458d9d-741b3de822f2aaca6c6caa4325c4afce/share/doc/xz/NEWS @ 68:5028fdace37b

planemo upload commit 2e9511a184a1ca667c7be0c6321a36dc4e3d116d
author jpayne
date Tue, 18 Mar 2025 16:23:26 -0400
parents
children
rev   line source
jpayne@68 1
jpayne@68 2 XZ Utils Release Notes
jpayne@68 3 ======================
jpayne@68 4
jpayne@68 5 5.6.4 (2025-01-23)
jpayne@68 6
jpayne@68 7 * liblzma: Fix LZMA/LZMA2 encoder on big endian ARM64.
jpayne@68 8
jpayne@68 9 * xz:
jpayne@68 10
jpayne@68 11 - Fix --filters= and --filters1= ... --filters9= options
jpayne@68 12 parsing. They require an argument, thus "xz --filters lzma2"
jpayne@68 13 should work in addition to "xz --filters=lzma2".
jpayne@68 14
jpayne@68 15 - On the man page, note in the --compress and --decompress
jpayne@68 16 options that the default behavior is to delete the input
jpayne@68 17 file unless writing to standard output. It was already
jpayne@68 18 documented in the DESCRIPTION section but new users in
jpayne@68 19 a hurry might miss it.
jpayne@68 20
jpayne@68 21 * Windows (native builds, not Cygwin): Fix regressions introduced
jpayne@68 22 in XZ Utils 5.6.3 which caused non-ASCII characters to display
jpayne@68 23 incorrectly. Only builds with translation support were affected
jpayne@68 24 (--enable-nls or ENABLE_NLS=ON). The following changes affect
jpayne@68 25 builds that have translations enabled:
jpayne@68 26
jpayne@68 27 - Require UCRT because MSVCRT doesn't support UTF-8
jpayne@68 28 locales and thus translations won't be readable on
jpayne@68 29 Windows 10 version 1903 and later. (MSVCRT builds
jpayne@68 30 are still possible with --disable-nls or ENABLE_NLS=OFF.)
jpayne@68 31
jpayne@68 32 - Require gettext-runtime >= 0.23.1 because older versions
jpayne@68 33 don't autodetect the use of the UTF-8 code page. This
jpayne@68 34 resulted in garbled non-ASCII characters even with UCRT.
jpayne@68 35
jpayne@68 36 - Partially fix alignment issues in xz --verbose --list
jpayne@68 37 with translated messages. Chinese (simplified),
jpayne@68 38 Chinese (traditional), and Korean column headings
jpayne@68 39 are misaligned still because Windows and MinGW-w64
jpayne@68 40 don't provide wcwidth() and XZ Utils doesn't include
jpayne@68 41 a replacement function either.
jpayne@68 42
jpayne@68 43 * CMake: Explicitly disable unity builds. This prevents build
jpayne@68 44 failures when another project uses XZ Utils via CMake's
jpayne@68 45 FetchContent module, and that project enables unity builds.
jpayne@68 46
jpayne@68 47 * Update Chinese (traditional) and Serbian translations.
jpayne@68 48
jpayne@68 49
jpayne@68 50 5.6.3 (2024-10-01)
jpayne@68 51
jpayne@68 52 IMPORTANT: This includes a Windows-specific security fix to
jpayne@68 53 the command line tools (CVE-2024-47611). liblzma isn't affected
jpayne@68 54 by this issue.
jpayne@68 55
jpayne@68 56 * liblzma:
jpayne@68 57
jpayne@68 58 - Fix x86-64 inline assembly compatibility with GNU Binutils
jpayne@68 59 older than 2.27.
jpayne@68 60
jpayne@68 61 - Fix the build with GCC 4.2 on OpenBSD/sparc64.
jpayne@68 62
jpayne@68 63 * xzdec: Display an error instead of failing silently if the
jpayne@68 64 unsupported option -M is specified.
jpayne@68 65
jpayne@68 66 * lzmainfo: Fix integer overflows when rounding the dictionary and
jpayne@68 67 uncompressed sizes to the nearest mebibyte.
jpayne@68 68
jpayne@68 69 * Windows (except Cygwin and MSYS2): Add an application manifest to
jpayne@68 70 xz, xzdec, lzmadec, and lzmainfo executables:
jpayne@68 71
jpayne@68 72 - Declare them compatible with Vista/7/8/8.1/10/11. This way
jpayne@68 73 the programs won't needlessly use Operating System Context
jpayne@68 74 of Vista when running on later Windows versions. This setting
jpayne@68 75 doesn't mean that the executables cannot run on even older
jpayne@68 76 versions if otherwise built that way.
jpayne@68 77
jpayne@68 78 - Declare them as UAC-compliant. MSVC added this by default
jpayne@68 79 already but it wasn't done with MinGW-w64, at least not
jpayne@68 80 with all toolchain variants.
jpayne@68 81
jpayne@68 82 - Declare them long path aware. This makes long path names
jpayne@68 83 work on Windows 10 and 11 if the feature has been enabled
jpayne@68 84 in the Windows registry.
jpayne@68 85
jpayne@68 86 - Use the UTF-8 code page on Windows 10 version 1903 and later.
jpayne@68 87
jpayne@68 88 * Now command line tools can access files whose names
jpayne@68 89 contain characters that don't exist in the current
jpayne@68 90 legacy code page.
jpayne@68 91
jpayne@68 92 * The options --files and --files0 now expect file lists
jpayne@68 93 to be in UTF-8 instead of the legacy code page.
jpayne@68 94
jpayne@68 95 * This fixes a security issue: If a command line contains
jpayne@68 96 Unicode characters (for example, filenames) that don't
jpayne@68 97 exist in the current legacy code page, the characters are
jpayne@68 98 converted to similar-looking characters with best-fit
jpayne@68 99 mapping. Some best-fit mappings result in ASCII
jpayne@68 100 characters that change the meaning of the command line,
jpayne@68 101 which can be exploited with malicious filenames to do
jpayne@68 102 argument injection or directory traversal attacks.
jpayne@68 103 UTF-8 avoids best-fit mappings and thus fixes the issue.
jpayne@68 104 (CVE-2024-47611)
jpayne@68 105
jpayne@68 106 Forcing the process code page to UTF-8 is possible only
jpayne@68 107 on Windows 10 version 1903 and later. The command line
jpayne@68 108 tools remain vulnerable if used on an old older
jpayne@68 109 version of Windows.
jpayne@68 110
jpayne@68 111 This issue was discovered by Orange Tsai and splitline
jpayne@68 112 from DEVCORE Research Team.
jpayne@68 113
jpayne@68 114 A related smaller issue remains: Windows filenames may
jpayne@68 115 contain unpaired surrogates (invalid UTF-16). These are
jpayne@68 116 converted to the replacement character U+FFFD in the
jpayne@68 117 UTF-8 code page. Thus, filenames with different unpaired
jpayne@68 118 surrogates appear identical and aren't distinguishable
jpayne@68 119 from filenames that contain the actual replacement
jpayne@68 120 character U+FFFD.
jpayne@68 121
jpayne@68 122 * When building with MinGW-w64, it is recommended to use
jpayne@68 123 UCRT version instead of the old MSVCRT. For example,
jpayne@68 124 non-ASCII characters from filenames won't print
jpayne@68 125 correctly in messages to console with MSVCRT with
jpayne@68 126 the UTF-8 code page (a cosmetic issue). liblzma-only
jpayne@68 127 builds are still fine with MSVCRT.
jpayne@68 128
jpayne@68 129 - Cygwin and MSYS2 process command line options differently and
jpayne@68 130 the above issues don't exist. There is no need to replace the
jpayne@68 131 default application manifest on Cygwin and MSYS2.
jpayne@68 132
jpayne@68 133 * Autotools-based build:
jpayne@68 134
jpayne@68 135 - Fix feature checks with link-time optimization (-flto).
jpayne@68 136
jpayne@68 137 - Solaris: Fix a compatibility issue in version.sh. It matters
jpayne@68 138 if one wants to regenerate configure by running autoconf.
jpayne@68 139
jpayne@68 140 * CMake:
jpayne@68 141
jpayne@68 142 - Use paths relative to ${prefix} in liblzma.pc when possible.
jpayne@68 143 This is done only with CMake >= 3.20.
jpayne@68 144
jpayne@68 145 - MSVC: Install liblzma.pc as it can be useful with MSVC too.
jpayne@68 146
jpayne@68 147 - Windows: Fix liblzma filename prefix, for example:
jpayne@68 148
jpayne@68 149 * Cygwin: The DLL was incorrectly named liblzma-5.dll.
jpayne@68 150 Now it is cyglzma-5.dll.
jpayne@68 151
jpayne@68 152 * MSVC: Rename import library from liblzma.lib to lzma.lib
jpayne@68 153 while keeping liblzma.dll name as is. This helps with
jpayne@68 154 "pkgconf --msvc-syntax --libs liblzma" because it mungles
jpayne@68 155 "-llzma" in liblzma.pc to "lzma.lib".
jpayne@68 156
jpayne@68 157 * MinGW-w64: No changes.
jpayne@68 158
jpayne@68 159 - Windows: Use the correct resource file for lzmadec.exe.
jpayne@68 160 Previously the resource file for xzdec.exe was used for both.
jpayne@68 161 Autotools-based build isn't affected.
jpayne@68 162
jpayne@68 163 - Prefer a C11 compiler over a C99 compiler but accept both.
jpayne@68 164
jpayne@68 165 - Link Threads::Threads against liblzma using PRIVATE so that
jpayne@68 166 -pthread and such flags won't unnecessarily get included in
jpayne@68 167 the usage requirements of shared liblzma. That is,
jpayne@68 168 target_link_libraries(foo PRIVATE liblzma::liblzma) no
jpayne@68 169 longer adds -pthread if using POSIX threads and linking
jpayne@68 170 against shared liblzma. The threading flags are still added
jpayne@68 171 if linking against static liblzma.
jpayne@68 172
jpayne@68 173 * Updated translations: Catalan, Chinese (simplified), and
jpayne@68 174 Brazilian Portuguese.
jpayne@68 175
jpayne@68 176
jpayne@68 177 5.6.2 (2024-05-29)
jpayne@68 178
jpayne@68 179 * Remove the backdoor (CVE-2024-3094).
jpayne@68 180
jpayne@68 181 * Not changed: Memory sanitizer (MSAN) has a false positive
jpayne@68 182 in the CRC CLMUL code which also makes OSS Fuzz unhappy.
jpayne@68 183 Valgrind is smarter and doesn't complain.
jpayne@68 184
jpayne@68 185 A revision to the CLMUL code is coming anyway and this issue
jpayne@68 186 will be cleaned up as part of it. It won't be backported to
jpayne@68 187 5.6.x or 5.4.x because the old code isn't wrong. There is
jpayne@68 188 no reason to risk introducing regressions in old branches
jpayne@68 189 just to silence a false positive.
jpayne@68 190
jpayne@68 191 * liblzma:
jpayne@68 192
jpayne@68 193 - lzma_index_decoder() and lzma_index_buffer_decode(): Fix
jpayne@68 194 a missing output pointer initialization (*i = NULL) if the
jpayne@68 195 functions are called with invalid arguments. The API docs
jpayne@68 196 say that such an initialization is always done. In practice
jpayne@68 197 this matters very little because the problem can only occur
jpayne@68 198 if the calling application has a bug and these functions
jpayne@68 199 return LZMA_PROG_ERROR.
jpayne@68 200
jpayne@68 201 - lzma_str_to_filters(): Fix a missing output pointer
jpayne@68 202 initialization (*error_pos = 0). This is very similar
jpayne@68 203 to the fix above.
jpayne@68 204
jpayne@68 205 - Fix C standard conformance with function pointer types.
jpayne@68 206
jpayne@68 207 - Remove GNU indirect function (IFUNC) support. This is *NOT*
jpayne@68 208 done for security reasons even though the backdoor relied on
jpayne@68 209 this code. The performance benefits of IFUNC are too tiny in
jpayne@68 210 this project to make the extra complexity worth it.
jpayne@68 211
jpayne@68 212 - FreeBSD on ARM64: Add error checking to CRC32 instruction
jpayne@68 213 support detection.
jpayne@68 214
jpayne@68 215 - Fix building with NVIDIA HPC SDK.
jpayne@68 216
jpayne@68 217 * xz:
jpayne@68 218
jpayne@68 219 - Fix a C standard conformance issue in --block-list parsing
jpayne@68 220 (arithmetic on a null pointer).
jpayne@68 221
jpayne@68 222 - Fix a warning from GNU groff when processing the man page:
jpayne@68 223 "warning: cannot select font 'CW'"
jpayne@68 224
jpayne@68 225 * xzdec: Add support for Linux Landlock ABI version 4. xz already
jpayne@68 226 had the v3-to-v4 change but it had been forgotten from xzdec.
jpayne@68 227
jpayne@68 228 * Autotools-based build system (configure):
jpayne@68 229
jpayne@68 230 - Symbol versioning variant can now be overridden with
jpayne@68 231 --enable-symbol-versions. Documentation in INSTALL was
jpayne@68 232 updated to match.
jpayne@68 233
jpayne@68 234 - Add new configure option --enable-doxygen to enable
jpayne@68 235 generation and installation of the liblzma API documentation
jpayne@68 236 using Doxygen. Documentation in INSTALL and PACKAGERS was
jpayne@68 237 updated to match.
jpayne@68 238
jpayne@68 239 CMake:
jpayne@68 240
jpayne@68 241 - Fix detection of Linux Landlock support. The detection code
jpayne@68 242 in CMakeLists.txt had been sabotaged.
jpayne@68 243
jpayne@68 244 - Disable symbol versioning on non-glibc Linux to match what
jpayne@68 245 the Autotools build does. For example, symbol versioning
jpayne@68 246 isn't enabled with musl.
jpayne@68 247
jpayne@68 248 - Symbol versioning variant can now be overridden by setting
jpayne@68 249 SYMBOL_VERSIONING to "OFF", "generic", or "linux".
jpayne@68 250
jpayne@68 251 - Add support for all tests in typical build configurations.
jpayne@68 252 Now the only difference to the tests coverage to Autotools
jpayne@68 253 is that CMake-based build will skip more tests if features
jpayne@68 254 are disabled. Such builds are only for special cases like
jpayne@68 255 embedded systems.
jpayne@68 256
jpayne@68 257 - Separate the CMake code for the tests into tests/tests.cmake.
jpayne@68 258 It is used conditionally, thus it is possible to
jpayne@68 259
jpayne@68 260 rm -rf tests
jpayne@68 261
jpayne@68 262 and the CMake-based build will still work normally except
jpayne@68 263 that no tests are then available.
jpayne@68 264
jpayne@68 265 - Add a option ENABLE_DOXYGEN to enable generation and
jpayne@68 266 installation of the liblzma API documentation using Doxygen.
jpayne@68 267
jpayne@68 268 * Documentation:
jpayne@68 269
jpayne@68 270 - Omit the Doxygen-generated liblzma API documentation from the
jpayne@68 271 package. Instead, the generation and installation of the API
jpayne@68 272 docs can be enabled with a configure or CMake option if
jpayne@68 273 Doxygen is available.
jpayne@68 274
jpayne@68 275 - Remove the XZ logo which was used in the API documentation.
jpayne@68 276 The logo has been retired and isn't used by the project
jpayne@68 277 anymore. However, it's OK to use it in contexts that refer
jpayne@68 278 to the backdoor incident.
jpayne@68 279
jpayne@68 280 - Remove the PDF versions of the man pages from the source
jpayne@68 281 package. These existed primarily for users of operating
jpayne@68 282 systems which don't come with tools to render man page
jpayne@68 283 source files. The plain text versions are still included
jpayne@68 284 in doc/man/txt. PDF files can still be generated to doc/man,
jpayne@68 285 if the required tools are available, using "make pdf" after
jpayne@68 286 running "configure".
jpayne@68 287
jpayne@68 288 - Update home page URLs back to their old locations on
jpayne@68 289 tukaani.org.
jpayne@68 290
jpayne@68 291 - Update maintainer info.
jpayne@68 292
jpayne@68 293 * Tests:
jpayne@68 294
jpayne@68 295 - In tests/files/README, explain how to recreate the ARM64
jpayne@68 296 test files.
jpayne@68 297
jpayne@68 298 - Remove two tests that used tiny x86 and SPARC object files
jpayne@68 299 as the input files. The matching .c file was included but
jpayne@68 300 the object files aren't easy to reproduce. The test cases
jpayne@68 301 weren't great anyway; they were from the early days (2009)
jpayne@68 302 of the project when the test suite had very few tests.
jpayne@68 303
jpayne@68 304 - Improve a few tests.
jpayne@68 305
jpayne@68 306
jpayne@68 307 5.6.1 (2024-03-09)
jpayne@68 308
jpayne@68 309 IMPORTANT: This fixed bugs in the backdoor (CVE-2024-3094) (someone
jpayne@68 310 had forgot to run Valgrind).
jpayne@68 311
jpayne@68 312 * liblzma: Fixed two bugs relating to GNU indirect function (IFUNC)
jpayne@68 313 with GCC. The more serious bug caused a program linked with
jpayne@68 314 liblzma to crash on start up if the flag -fprofile-generate was
jpayne@68 315 used to build liblzma. The second bug caused liblzma to falsely
jpayne@68 316 report an invalid write to Valgrind when loading liblzma.
jpayne@68 317
jpayne@68 318 * xz: Changed the messages for thread reduction due to memory
jpayne@68 319 constraints to only appear under the highest verbosity level.
jpayne@68 320
jpayne@68 321 * Build:
jpayne@68 322
jpayne@68 323 - Fixed a build issue when the header file <linux/landlock.h>
jpayne@68 324 was present on the system but the Landlock system calls were
jpayne@68 325 not defined in <sys/syscall.h>.
jpayne@68 326
jpayne@68 327 - The CMake build now warns and disables NLS if both gettext
jpayne@68 328 tools and pre-created .gmo files are missing. Previously,
jpayne@68 329 this caused the CMake build to fail.
jpayne@68 330
jpayne@68 331 * Minor improvements to man pages.
jpayne@68 332
jpayne@68 333 * Minor improvements to tests.
jpayne@68 334
jpayne@68 335
jpayne@68 336 5.6.0 (2024-02-24)
jpayne@68 337
jpayne@68 338 IMPORTANT: This added a backdoor (CVE-2024-3094). It's enabled only
jpayne@68 339 in the release tarballs.
jpayne@68 340
jpayne@68 341 This bumps the minor version of liblzma because new features were
jpayne@68 342 added. The API and ABI are still backward compatible with liblzma
jpayne@68 343 5.4.x and 5.2.x and 5.0.x.
jpayne@68 344
jpayne@68 345 NOTE: As described in the NEWS for 5.5.2beta, the core components
jpayne@68 346 are now under the BSD Zero Clause License (0BSD).
jpayne@68 347
jpayne@68 348 Since 5.5.2beta:
jpayne@68 349
jpayne@68 350 * liblzma:
jpayne@68 351
jpayne@68 352 - Disabled the branchless C variant in the LZMA decoder based
jpayne@68 353 on the benchmark results from the community.
jpayne@68 354
jpayne@68 355 - Disabled x86-64 inline assembly on x32 to fix the build.
jpayne@68 356
jpayne@68 357 * Sandboxing support in xz:
jpayne@68 358
jpayne@68 359 - Landlock is now used even when xz needs to create files.
jpayne@68 360 In this case the sandbox has to be more permissive than
jpayne@68 361 when no files need to be created. A similar thing was
jpayne@68 362 already in use with pledge(2) since 5.3.4alpha.
jpayne@68 363
jpayne@68 364 - Landlock and pledge(2) are now stricter when reading from
jpayne@68 365 more than one input file and only writing to standard output.
jpayne@68 366
jpayne@68 367 - Added support for Landlock ABI version 4.
jpayne@68 368
jpayne@68 369 * CMake:
jpayne@68 370
jpayne@68 371 - Default to -O2 instead of -O3 with CMAKE_BUILD_TYPE=Release.
jpayne@68 372 -O3 is not useful for speed and makes the code larger.
jpayne@68 373
jpayne@68 374 - Now builds lzmainfo and lzmadec.
jpayne@68 375
jpayne@68 376 - xzdiff, xzgrep, xzless, xzmore, and their symlinks are now
jpayne@68 377 installed. The scripts are also tested during "make test".
jpayne@68 378
jpayne@68 379 - Added translation support for xz, lzmainfo, and the
jpayne@68 380 man pages.
jpayne@68 381
jpayne@68 382 - Applied the symbol versioning workaround for MicroBlaze that
jpayne@68 383 is used in the Autotools build.
jpayne@68 384
jpayne@68 385 - The general XZ Utils and liblzma API documentation is now
jpayne@68 386 installed.
jpayne@68 387
jpayne@68 388 - The CMake component names were changed a little and several
jpayne@68 389 were added. liblzma_Runtime and liblzma_Development are
jpayne@68 390 unchanged.
jpayne@68 391
jpayne@68 392 - Minimum required CMake version is now 3.14. However,
jpayne@68 393 translation support is disabled with CMake versions
jpayne@68 394 older than 3.20.
jpayne@68 395
jpayne@68 396 - The CMake-based build is now close to feature parity with the
jpayne@68 397 Autotools-based build. Most importantly a few tests aren't
jpayne@68 398 run yet. Testing the CMake-based build on different operating
jpayne@68 399 systems would be welcome now. See the comment at the top of
jpayne@68 400 CMakeLists.txt.
jpayne@68 401
jpayne@68 402 * Fixed a bug in the Autotools feature test for ARM64 CRC32
jpayne@68 403 instruction support for old versions of Clang. This did not
jpayne@68 404 affect the CMake build.
jpayne@68 405
jpayne@68 406 * Windows:
jpayne@68 407
jpayne@68 408 - The build instructions in INSTALL and windows/INSTALL*.txt
jpayne@68 409 were revised completely.
jpayne@68 410
jpayne@68 411 - windows/build-with-cmake.bat along with the instructions
jpayne@68 412 in windows/INSTALL-MinGW-w64_with_CMake.txt should make
jpayne@68 413 it very easy to build liblzma.dll and xz.exe on Windows
jpayne@68 414 using CMake and MinGW-w64 with either GCC or Clang/LLVM.
jpayne@68 415
jpayne@68 416 - windows/build.bash was updated. It now works on MSYS2 and
jpayne@68 417 on GNU/Linux (cross-compiling) to create a .zip and .7z
jpayne@68 418 package for 32-bit and 64-bit x86 using GCC + MinGW-w64.
jpayne@68 419
jpayne@68 420 * The TODO file is no longer installed as part of the
jpayne@68 421 documentation. The file is out of date and does not reflect
jpayne@68 422 the actual tasks that will be completed in the future.
jpayne@68 423
jpayne@68 424 * Translations:
jpayne@68 425
jpayne@68 426 - Translated lzmainfo man pages are now installed. These
jpayne@68 427 had been forgotten in earlier versions.
jpayne@68 428
jpayne@68 429 - Updated Croatian, Esperanto, German, Hungarian, Korean,
jpayne@68 430 Polish, Romanian, Spanish, Swedish, Vietnamese, and Ukrainian
jpayne@68 431 translations.
jpayne@68 432
jpayne@68 433 - Updated German, Korean, Romanian, and Ukrainian man page
jpayne@68 434 translations.
jpayne@68 435
jpayne@68 436 * Added a few tests.
jpayne@68 437
jpayne@68 438 Summary of new features added in the 5.5.x development releases:
jpayne@68 439
jpayne@68 440 * liblzma:
jpayne@68 441
jpayne@68 442 - LZMA decoder: Speed optimizations to the C code and
jpayne@68 443 added GCC & Clang compatible inline assembly for x86-64.
jpayne@68 444
jpayne@68 445 - Added lzma_mt_block_size() to recommend a Block size for
jpayne@68 446 multithreaded encoding.
jpayne@68 447
jpayne@68 448 - Added CLMUL-based CRC32 on x86-64 and E2K with runtime
jpayne@68 449 processor detection. Similar to CRC64, on 32-bit x86 it
jpayne@68 450 isn't available unless --disable-assembler is used.
jpayne@68 451
jpayne@68 452 - Optimized the CRC32 calculation on ARM64 platforms using the
jpayne@68 453 CRC32 instructions. Runtime detection for the instruction is
jpayne@68 454 used on GNU/Linux, FreeBSD, Windows, and macOS. If the
jpayne@68 455 compiler flags indicate unconditional CRC32 instruction
jpayne@68 456 support (+crc) then the generic version is not built.
jpayne@68 457
jpayne@68 458 - Added definitions of mask values like
jpayne@68 459 LZMA_INDEX_CHECK_MASK_CRC32 to <lzma/index.h>.
jpayne@68 460
jpayne@68 461 * xz:
jpayne@68 462
jpayne@68 463 - Multithreaded mode is now the default. This improves
jpayne@68 464 compression speed and creates .xz files that can be
jpayne@68 465 decompressed in multithreaded mode. The downsides are
jpayne@68 466 increased memory usage and slightly worse compression ratio.
jpayne@68 467
jpayne@68 468 - Added a new command line option --filters to set the filter
jpayne@68 469 chain using the liblzma filter string syntax.
jpayne@68 470
jpayne@68 471 - Added new command line options --filters1 ... --filters9 to
jpayne@68 472 set additional filter chains using the liblzma filter string
jpayne@68 473 syntax. The --block-list option now allows specifying filter
jpayne@68 474 chains that were set using these new options.
jpayne@68 475
jpayne@68 476 - Ported the command line tools to Windows MSVC.
jpayne@68 477 Visual Studio 2015 or later is required.
jpayne@68 478
jpayne@68 479 * Added lz4 support to xzdiff/xzcmp and xzgrep.
jpayne@68 480
jpayne@68 481
jpayne@68 482 5.5.2beta (2024-02-14)
jpayne@68 483
jpayne@68 484 * Licensing change: The core components are now under the
jpayne@68 485 BSD Zero Clause License (0BSD). In XZ Utils 5.4.6 and older
jpayne@68 486 and 5.5.1alpha these components are in the public domain and
jpayne@68 487 obviously remain so; the change affects the new releases only.
jpayne@68 488
jpayne@68 489 0BSD is an extremely permissive license which doesn't require
jpayne@68 490 retaining or reproducing copyright or license notices when
jpayne@68 491 distributing the code, thus in practice there is extremely
jpayne@68 492 little difference to public domain.
jpayne@68 493
jpayne@68 494 * liblzma
jpayne@68 495
jpayne@68 496 - Significant speed optimizations to the LZMA decoder were
jpayne@68 497 made. There are now three variants that can be chosen at
jpayne@68 498 build time:
jpayne@68 499
jpayne@68 500 * Basic C version: This is a few percent faster than
jpayne@68 501 5.4.x due to some new optimizations.
jpayne@68 502
jpayne@68 503 * Branchless C: This is currently the default on platforms
jpayne@68 504 for which there is no assembly code. This should be a few
jpayne@68 505 percent faster than the basic C version.
jpayne@68 506
jpayne@68 507 * x86-64 inline assembly. This works with GCC and Clang.
jpayne@68 508
jpayne@68 509 The default choice can currently be overridden by setting
jpayne@68 510 LZMA_RANGE_DECODER_CONFIG in CPPFLAGS: 0 means the basic
jpayne@68 511 version and 3 means that branchless C version.
jpayne@68 512
jpayne@68 513 - Optimized the CRC32 calculation on ARM64 platforms using the
jpayne@68 514 CRC32 instructions. The instructions are optional in ARMv8.0
jpayne@68 515 and are required in ARMv8.1 and later. Runtime detection for
jpayne@68 516 the instruction is used on GNU/Linux, FreeBSD, Windows, and
jpayne@68 517 macOS. If the compiler flags indicate unconditional CRC32
jpayne@68 518 instruction support (+crc) then the generic version is not
jpayne@68 519 built.
jpayne@68 520
jpayne@68 521 * Added lz4 support to xzdiff/xzcmp and xzgrep.
jpayne@68 522
jpayne@68 523 * Man pages of xzdiff/xzcmp, xzgrep, and xzmore were rewritten
jpayne@68 524 to simplify licensing of the man page translations.
jpayne@68 525
jpayne@68 526 * Translations:
jpayne@68 527
jpayne@68 528 - Updated Chinese (simplified), German, Korean, Polish,
jpayne@68 529 Romanian, Spanish, Swedish, and Ukrainian translations.
jpayne@68 530
jpayne@68 531 - Updated German, Korean, Romanian, and Ukrainian man page
jpayne@68 532 translations.
jpayne@68 533
jpayne@68 534 * Small improvements to the tests.
jpayne@68 535
jpayne@68 536 * Added doc/examples/11_file_info.c. It was added to the Git
jpayne@68 537 repository in 2017 but forgotten to be added into distribution
jpayne@68 538 tarballs.
jpayne@68 539
jpayne@68 540 * Removed doc/examples_old. These were from 2012.
jpayne@68 541
jpayne@68 542 * Removed the macos/build.sh script. It had not been updated
jpayne@68 543 since 2013.
jpayne@68 544
jpayne@68 545
jpayne@68 546 5.5.1alpha (2024-01-26)
jpayne@68 547
jpayne@68 548 * Added a new filter for RISC-V binaries. The filter can be used
jpayne@68 549 for 32-bit and 64-bit binaries with either little or big
jpayne@68 550 endianness. In liblzma, the Filter ID is LZMA_FILTER_RISCV (0x0B)
jpayne@68 551 and the xz option is --riscv. liblzma filter string syntax
jpayne@68 552 recognizes this filter as "riscv".
jpayne@68 553
jpayne@68 554 * liblzma:
jpayne@68 555
jpayne@68 556 - Added lzma_mt_block_size() to recommend a Block size for
jpayne@68 557 multithreaded encoding
jpayne@68 558
jpayne@68 559 - Added CLMUL-based CRC32 on x86-64 and E2K with runtime
jpayne@68 560 processor detection. Similar to CRC64, on 32-bit x86 it
jpayne@68 561 isn't available unless --disable-assembler is used.
jpayne@68 562
jpayne@68 563 - Implemented GNU indirect function (IFUNC) as a runtime
jpayne@68 564 function dispatching method for CRC32 and CRC64 fast
jpayne@68 565 implementations on x86. Only GNU/Linux (glibc) and FreeBSD
jpayne@68 566 builds will use IFUNC, unless --enable-ifunc is specified to
jpayne@68 567 configure.
jpayne@68 568
jpayne@68 569 - Added definitions of mask values like
jpayne@68 570 LZMA_INDEX_CHECK_MASK_CRC32 to <lzma/index.h>.
jpayne@68 571
jpayne@68 572 - The XZ logo is now included in the Doxygen generated
jpayne@68 573 documentation. It is licensed under Creative Commons
jpayne@68 574 Attribution-ShareAlike 4.0.
jpayne@68 575
jpayne@68 576 * xz:
jpayne@68 577
jpayne@68 578 - Multithreaded mode is now the default. This improves
jpayne@68 579 compression speed and creates .xz files that can be
jpayne@68 580 decompressed multithreaded at the cost of increased memory
jpayne@68 581 usage and slightly worse compression ratio.
jpayne@68 582
jpayne@68 583 - Added new command line option --filters to set the filter
jpayne@68 584 chain using liblzma filter string syntax.
jpayne@68 585
jpayne@68 586 - Added new command line options --filters1 ... --filters9 to
jpayne@68 587 set additional filter chains using liblzma filter string
jpayne@68 588 syntax. The --block-list option now allows specifying filter
jpayne@68 589 chains that were set using these new options.
jpayne@68 590
jpayne@68 591 - Added support for Linux Landlock as a sandboxing method.
jpayne@68 592
jpayne@68 593 - xzdec now supports pledge(2), Capsicum, and Linux Landlock as
jpayne@68 594 sandboxing methods.
jpayne@68 595
jpayne@68 596 - Progress indicator time stats remain accurate after pausing
jpayne@68 597 xz with SIGTSTP.
jpayne@68 598
jpayne@68 599 - Ported xz and xzdec to Windows MSVC. Visual Studio 2015 or
jpayne@68 600 later is required.
jpayne@68 601
jpayne@68 602 * CMake Build:
jpayne@68 603
jpayne@68 604 - Supports pledge(2), Capsicum, and Linux Landlock sandboxing
jpayne@68 605 methods.
jpayne@68 606
jpayne@68 607 - Replacement functions for getopt_long() are used on platforms
jpayne@68 608 that do not have it.
jpayne@68 609
jpayne@68 610 * Enabled unaligned access by default on PowerPC64LE and on RISC-V
jpayne@68 611 targets that define __riscv_misaligned_fast.
jpayne@68 612
jpayne@68 613 * Tests:
jpayne@68 614
jpayne@68 615 - Added two new fuzz targets to OSS-Fuzz.
jpayne@68 616
jpayne@68 617 - Implemented Continuous Integration (CI) testing using
jpayne@68 618 GitHub Actions.
jpayne@68 619
jpayne@68 620 * Changed quoting style from `...' to '...' in all messages,
jpayne@68 621 scripts, and documentation.
jpayne@68 622
jpayne@68 623 * Added basic Codespell support to help catch typo errors.
jpayne@68 624
jpayne@68 625
jpayne@68 626 5.4.7 (2024-05-29)
jpayne@68 627
jpayne@68 628 * Not changed: Memory sanitizer (MSAN) has a false positive
jpayne@68 629 in the CRC CLMUL code which also makes OSS Fuzz unhappy.
jpayne@68 630 Valgrind is smarter and doesn't complain.
jpayne@68 631
jpayne@68 632 A revision to the CLMUL code is coming anyway and this issue
jpayne@68 633 will be cleaned up as part of it. It won't be backported to
jpayne@68 634 5.6.x or 5.4.x because the old code isn't wrong. There is
jpayne@68 635 no reason to risk introducing regressions in old branches
jpayne@68 636 just to silence a false positive.
jpayne@68 637
jpayne@68 638 * liblzma:
jpayne@68 639
jpayne@68 640 - lzma_index_decoder() and lzma_index_buffer_decode(): Fix
jpayne@68 641 a missing output pointer initialization (*i = NULL) if the
jpayne@68 642 functions are called with invalid arguments. The API docs
jpayne@68 643 say that such an initialization is always done. In practice
jpayne@68 644 this matters very little because the problem can only occur
jpayne@68 645 if the calling application has a bug and these functions
jpayne@68 646 return LZMA_PROG_ERROR.
jpayne@68 647
jpayne@68 648 - lzma_str_to_filters(): Fix a missing output pointer
jpayne@68 649 initialization (*error_pos = 0). This is very similar
jpayne@68 650 to the fix above.
jpayne@68 651
jpayne@68 652 - Fix C standard conformance with function pointer types.
jpayne@68 653 This newly showed up with Clang 17 with -fsanitize=undefined.
jpayne@68 654 There are no bug reports about this.
jpayne@68 655
jpayne@68 656 - Fix building with NVIDIA HPC SDK.
jpayne@68 657
jpayne@68 658 * xz:
jpayne@68 659
jpayne@68 660 - Fix a C standard conformance issue in --block-list parsing
jpayne@68 661 (arithmetic on a null pointer).
jpayne@68 662
jpayne@68 663 - Fix a warning from GNU groff when processing the man page:
jpayne@68 664 "warning: cannot select font 'CW'"
jpayne@68 665
jpayne@68 666 - Fix outdated threading related information on the man page.
jpayne@68 667
jpayne@68 668 * xzless:
jpayne@68 669
jpayne@68 670 - With "less" version 451 and later, use "||-" instead of "|-"
jpayne@68 671 in the environment variable LESSOPEN. This way compressed
jpayne@68 672 files that contain no uncompressed data are shown correctly
jpayne@68 673 as empty.
jpayne@68 674
jpayne@68 675 - With "less" version 632 and later, use --show-preproc-errors
jpayne@68 676 to make "less" show a warning on decompression errors.
jpayne@68 677
jpayne@68 678 * Autotools-based build system (configure):
jpayne@68 679
jpayne@68 680 - Symbol versioning variant can now be overridden with
jpayne@68 681 --enable-symbol-versions. Documentation in INSTALL was
jpayne@68 682 updated to match.
jpayne@68 683
jpayne@68 684 CMake:
jpayne@68 685
jpayne@68 686 - Linux on MicroBlaze is handled specially now. This matches
jpayne@68 687 the changes made to the Autotools-based build in XZ Utils
jpayne@68 688 5.4.2 and 5.2.11.
jpayne@68 689
jpayne@68 690 - Disable symbol versioning on non-glibc Linux to match what
jpayne@68 691 the Autotools build does. For example, symbol versioning
jpayne@68 692 isn't enabled with musl.
jpayne@68 693
jpayne@68 694 - Symbol versioning variant can now be overridden by setting
jpayne@68 695 SYMBOL_VERSIONING to "OFF", "generic", or "linux".
jpayne@68 696
jpayne@68 697 * Documentation:
jpayne@68 698
jpayne@68 699 - Clarify the description of --disable-assembler in INSTALL.
jpayne@68 700 The option only affects 32-bit x86 assembly usage.
jpayne@68 701
jpayne@68 702 - Add doc/examples/11_file_info.c. It was added to the
jpayne@68 703 Git repository in 2017 but forgotten to be added into
jpayne@68 704 distribution tarballs.
jpayne@68 705
jpayne@68 706 - Don't install the TODO file as part of the documentation.
jpayne@68 707 The file is out of date.
jpayne@68 708
jpayne@68 709 - Update home page URLs back to their old locations on
jpayne@68 710 tukaani.org.
jpayne@68 711
jpayne@68 712 - Update maintainer info.
jpayne@68 713
jpayne@68 714
jpayne@68 715 5.4.6 (2024-01-26)
jpayne@68 716
jpayne@68 717 * Fixed a bug involving internal function pointers in liblzma not
jpayne@68 718 being initialized to NULL. The bug can only be triggered if
jpayne@68 719 lzma_filters_update() is called on a LZMA1 encoder, so it does
jpayne@68 720 not affect xz or any application known to us that uses liblzma.
jpayne@68 721
jpayne@68 722 * xz:
jpayne@68 723
jpayne@68 724 - Fixed a regression introduced in 5.4.2 that caused encoding
jpayne@68 725 in the raw format to unnecessarily fail if --suffix was not
jpayne@68 726 used. For instance, the following command no longer reports
jpayne@68 727 that --suffix must be used:
jpayne@68 728
jpayne@68 729 echo foo | xz --format=raw --lzma2 | wc -c
jpayne@68 730
jpayne@68 731 - Fixed an issue on MinGW-w64 builds that prevented reading
jpayne@68 732 from or writing to non-terminal character devices like NUL.
jpayne@68 733
jpayne@68 734 * Added a new test.
jpayne@68 735
jpayne@68 736
jpayne@68 737 5.4.5 (2023-11-01)
jpayne@68 738
jpayne@68 739 * liblzma:
jpayne@68 740
jpayne@68 741 - Use __attribute__((__no_sanitize_address__)) to avoid address
jpayne@68 742 sanitization with CRC64 CLMUL. It uses 16-byte-aligned reads
jpayne@68 743 which can extend past the bounds of the input buffer and
jpayne@68 744 inherently trigger address sanitization errors. This isn't
jpayne@68 745 a bug.
jpayne@68 746
jpayne@68 747 - Fixed an assertion failure that could be triggered by a large
jpayne@68 748 unpadded_size argument. It was verified that there was no
jpayne@68 749 other bug than the assertion failure.
jpayne@68 750
jpayne@68 751 - Fixed a bug that prevented building with Windows Vista
jpayne@68 752 threading when __attribute__((__constructor__)) is not
jpayne@68 753 supported.
jpayne@68 754
jpayne@68 755 * xz now properly handles special files such as "con" or "nul" on
jpayne@68 756 Windows. Before this fix, the following wrote "foo" to the
jpayne@68 757 console and deleted the input file "con_xz":
jpayne@68 758
jpayne@68 759 echo foo | xz > con_xz
jpayne@68 760 xz --suffix=_xz --decompress con_xz
jpayne@68 761
jpayne@68 762 * Build systems:
jpayne@68 763
jpayne@68 764 - Allow builds with Windows win95 threading and small mode when
jpayne@68 765 __attribute__((__constructor__)) is supported.
jpayne@68 766
jpayne@68 767 - Added a new line to liblzma.pc for MSYS2 (Windows):
jpayne@68 768
jpayne@68 769 Cflags.private: -DLZMA_API_STATIC
jpayne@68 770
jpayne@68 771 When compiling code that will link against static liblzma,
jpayne@68 772 the LZMA_API_STATIC macro needs to be defined on Windows.
jpayne@68 773
jpayne@68 774 - CMake specific changes:
jpayne@68 775
jpayne@68 776 * Fixed a bug that allowed CLOCK_MONOTONIC to be used even
jpayne@68 777 if the check for it failed.
jpayne@68 778
jpayne@68 779 * Fixed a bug where configuring CMake multiple times
jpayne@68 780 resulted in HAVE_CLOCK_GETTIME and HAVE_CLOCK_MONOTONIC
jpayne@68 781 not being set.
jpayne@68 782
jpayne@68 783 * Fixed the build with MinGW-w64-based Clang/LLVM 17.
jpayne@68 784 llvm-windres now has more accurate GNU windres emulation
jpayne@68 785 so the GNU windres workaround from 5.4.1 is needed with
jpayne@68 786 llvm-windres version 17 too.
jpayne@68 787
jpayne@68 788 * The import library on Windows is now properly named
jpayne@68 789 "liblzma.dll.a" instead of "libliblzma.dll.a"
jpayne@68 790
jpayne@68 791 * Fixed a bug causing the Ninja Generator to fail on
jpayne@68 792 UNIX-like systems. This bug was introduced in 5.4.0.
jpayne@68 793
jpayne@68 794 * Added a new option to disable CLMUL CRC64.
jpayne@68 795
jpayne@68 796 * A module-definition (.def) file is now created when
jpayne@68 797 building liblzma.dll with MinGW-w64.
jpayne@68 798
jpayne@68 799 * The pkg-config liblzma.pc file is now installed on all
jpayne@68 800 builds except when using MSVC on Windows.
jpayne@68 801
jpayne@68 802 * Added large file support by default for platforms that
jpayne@68 803 need it to handle files larger than 2 GiB. This includes
jpayne@68 804 MinGW-w64, even 64-bit builds.
jpayne@68 805
jpayne@68 806 * Small fixes and improvements to the tests.
jpayne@68 807
jpayne@68 808 * Updated translations: Chinese (simplified) and Esperanto.
jpayne@68 809
jpayne@68 810
jpayne@68 811 5.4.4 (2023-08-02)
jpayne@68 812
jpayne@68 813 * liblzma and xzdec can now build against WASI SDK when threading
jpayne@68 814 support is disabled. xz and tests don't build yet.
jpayne@68 815
jpayne@68 816 * CMake:
jpayne@68 817
jpayne@68 818 - Fixed a bug preventing other projects from including liblzma
jpayne@68 819 multiple times using find_package().
jpayne@68 820
jpayne@68 821 - Don't create broken symlinks in Cygwin and MSYS2 unless
jpayne@68 822 supported by the environment. This prevented building for the
jpayne@68 823 default MSYS2 environment. The problem was introduced in
jpayne@68 824 xz 5.4.0.
jpayne@68 825
jpayne@68 826 * Documentation:
jpayne@68 827
jpayne@68 828 - Small improvements to man pages.
jpayne@68 829
jpayne@68 830 - Small improvements and typo fixes for liblzma API
jpayne@68 831 documentation.
jpayne@68 832
jpayne@68 833 * Tests:
jpayne@68 834
jpayne@68 835 - Added a new section to INSTALL to describe basic test usage
jpayne@68 836 and address recent questions about building the tests when
jpayne@68 837 cross compiling.
jpayne@68 838
jpayne@68 839 - Small fixes and improvements to the tests.
jpayne@68 840
jpayne@68 841 * Translations:
jpayne@68 842
jpayne@68 843 - Fixed a mistake that caused one of the error messages to not
jpayne@68 844 be translated. This only affected versions 5.4.2 and 5.4.3.
jpayne@68 845
jpayne@68 846 - Updated the Chinese (simplified), Croatian, Esperanto, German,
jpayne@68 847 Korean, Polish, Romanian, Spanish, Swedish, Ukrainian, and
jpayne@68 848 Vietnamese translations.
jpayne@68 849
jpayne@68 850 - Updated the German, Korean, Romanian, and Ukrainian man page
jpayne@68 851 translations.
jpayne@68 852
jpayne@68 853
jpayne@68 854 5.4.3 (2023-05-04)
jpayne@68 855
jpayne@68 856 * All fixes from 5.2.12
jpayne@68 857
jpayne@68 858 * Features in the CMake build can now be disabled as CMake cache
jpayne@68 859 variables, similar to the Autotools build.
jpayne@68 860
jpayne@68 861 * Minor update to the Croatian translation.
jpayne@68 862
jpayne@68 863
jpayne@68 864 5.4.2 (2023-03-18)
jpayne@68 865
jpayne@68 866 * All fixes from 5.2.11 that were not included in 5.4.1.
jpayne@68 867
jpayne@68 868 * If xz is built with support for the Capsicum sandbox but running
jpayne@68 869 in an environment that doesn't support Capsicum, xz now runs
jpayne@68 870 normally without sandboxing instead of exiting with an error.
jpayne@68 871
jpayne@68 872 * liblzma:
jpayne@68 873
jpayne@68 874 - Documentation was updated to improve the style, consistency,
jpayne@68 875 and completeness of the liblzma API headers.
jpayne@68 876
jpayne@68 877 - The Doxygen-generated HTML documentation for the liblzma API
jpayne@68 878 header files is now included in the source release and is
jpayne@68 879 installed as part of "make install". All JavaScript is
jpayne@68 880 removed to simplify license compliance and to reduce the
jpayne@68 881 install size.
jpayne@68 882
jpayne@68 883 - Fixed a minor bug in lzma_str_from_filters() that produced
jpayne@68 884 too many filters in the output string instead of reporting
jpayne@68 885 an error if the input array had more than four filters. This
jpayne@68 886 bug did not affect xz.
jpayne@68 887
jpayne@68 888 * Build systems:
jpayne@68 889
jpayne@68 890 - autogen.sh now invokes the doxygen tool via the new wrapper
jpayne@68 891 script doxygen/update-doxygen, unless the command line option
jpayne@68 892 --no-doxygen is used.
jpayne@68 893
jpayne@68 894 - Added microlzma_encoder.c and microlzma_decoder.c to the
jpayne@68 895 VS project files for Windows and to the CMake build. These
jpayne@68 896 should have been included in 5.3.2alpha.
jpayne@68 897
jpayne@68 898 * Tests:
jpayne@68 899
jpayne@68 900 - Added a test to the CMake build that was forgotten in the
jpayne@68 901 previous release.
jpayne@68 902
jpayne@68 903 - Added and refactored a few tests.
jpayne@68 904
jpayne@68 905 * Translations:
jpayne@68 906
jpayne@68 907 - Updated the Brazilian Portuguese translation.
jpayne@68 908
jpayne@68 909 - Added Brazilian Portuguese man page translation.
jpayne@68 910
jpayne@68 911
jpayne@68 912 5.4.1 (2023-01-11)
jpayne@68 913
jpayne@68 914 * liblzma:
jpayne@68 915
jpayne@68 916 - Fixed the return value of lzma_microlzma_encoder() if the
jpayne@68 917 LZMA options lc/lp/pb are invalid. Invalid lc/lp/pb options
jpayne@68 918 made the function return LZMA_STREAM_END without encoding
jpayne@68 919 anything instead of returning LZMA_OPTIONS_ERROR.
jpayne@68 920
jpayne@68 921 - Windows / Visual Studio: Workaround a possible compiler bug
jpayne@68 922 when targeting 32-bit x86 and compiling the CLMUL version of
jpayne@68 923 the CRC64 code. The CLMUL code isn't enabled by the Windows
jpayne@68 924 project files but it is in the CMake-based builds.
jpayne@68 925
jpayne@68 926 * Build systems:
jpayne@68 927
jpayne@68 928 - Windows-specific CMake changes:
jpayne@68 929
jpayne@68 930 * Don't try to enable CLMUL CRC64 code if _mm_set_epi64x()
jpayne@68 931 isn't available. This fixes CMake-based build with Visual
jpayne@68 932 Studio 2013.
jpayne@68 933
jpayne@68 934 * Created a workaround for a build failure with windres
jpayne@68 935 from GNU binutils. It is used only when the C compiler
jpayne@68 936 is GCC (not Clang). The workaround is incompatible
jpayne@68 937 with llvm-windres, resulting in "XZx20Utils" instead
jpayne@68 938 of "XZ Utils" in the resource file, but without the
jpayne@68 939 workaround llvm-windres works correctly. See the
jpayne@68 940 comment in CMakeLists.txt for details.
jpayne@68 941
jpayne@68 942 * Included the resource files in the xz and xzdec build
jpayne@68 943 rules. Building the command line tools is still
jpayne@68 944 experimental but possible with MinGW-w64.
jpayne@68 945
jpayne@68 946 - Visual Studio: Added stream_decoder_mt.c to the project
jpayne@68 947 files. Now the threaded decompressor lzma_stream_decoder_mt()
jpayne@68 948 gets built. CMake-based build wasn't affected.
jpayne@68 949
jpayne@68 950 - Updated windows/INSTALL-MSVC.txt to mention that CMake-based
jpayne@68 951 build is now the preferred method with Visual Studio. The
jpayne@68 952 project files will probably be removed after 5.4.x releases.
jpayne@68 953
jpayne@68 954 - Changes to #defines in config.h:
jpayne@68 955
jpayne@68 956 * HAVE_DECL_CLOCK_MONOTONIC was replaced by
jpayne@68 957 HAVE_CLOCK_MONOTONIC. The old macro was always defined
jpayne@68 958 in configure-generated config.h to either 0 or 1. The
jpayne@68 959 new macro is defined (to 1) only if the declaration of
jpayne@68 960 CLOCK_MONOTONIC is available. This matches the way most
jpayne@68 961 other config.h macros work and makes things simpler with
jpayne@68 962 other build systems.
jpayne@68 963
jpayne@68 964 * HAVE_DECL_PROGRAM_INVOCATION_NAME was replaced by
jpayne@68 965 HAVE_PROGRAM_INVOCATION_NAME for the same reason.
jpayne@68 966
jpayne@68 967 * Tests:
jpayne@68 968
jpayne@68 969 - Fixed test script compatibility with ancient /bin/sh
jpayne@68 970 versions. Now the five test_compress_* tests should
jpayne@68 971 no longer fail on Solaris 10.
jpayne@68 972
jpayne@68 973 - Added and refactored a few tests.
jpayne@68 974
jpayne@68 975 * Translations:
jpayne@68 976
jpayne@68 977 - Updated the Catalan and Esperanto translations.
jpayne@68 978
jpayne@68 979 - Added Korean and Ukrainian man page translations.
jpayne@68 980
jpayne@68 981
jpayne@68 982 5.4.0 (2022-12-13)
jpayne@68 983
jpayne@68 984 This bumps the minor version of liblzma because new features were
jpayne@68 985 added. The API and ABI are still backward compatible with liblzma
jpayne@68 986 5.2.x and 5.0.x.
jpayne@68 987
jpayne@68 988 Since 5.3.5beta:
jpayne@68 989
jpayne@68 990 * All fixes from 5.2.10.
jpayne@68 991
jpayne@68 992 * The ARM64 filter is now stable. The xz option is now --arm64.
jpayne@68 993 Decompression requires XZ Utils 5.4.0. In the future the ARM64
jpayne@68 994 filter will be supported by XZ for Java, XZ Embedded (including
jpayne@68 995 the version in Linux), LZMA SDK, and 7-Zip.
jpayne@68 996
jpayne@68 997 * Translations:
jpayne@68 998
jpayne@68 999 - Updated Catalan, Croatian, German, Romanian, and Turkish
jpayne@68 1000 translations.
jpayne@68 1001
jpayne@68 1002 - Updated German man page translations.
jpayne@68 1003
jpayne@68 1004 - Added Romanian man page translations.
jpayne@68 1005
jpayne@68 1006 Summary of new features added in the 5.3.x development releases:
jpayne@68 1007
jpayne@68 1008 * liblzma:
jpayne@68 1009
jpayne@68 1010 - Added threaded .xz decompressor lzma_stream_decoder_mt().
jpayne@68 1011 It can use multiple threads with .xz files that have multiple
jpayne@68 1012 Blocks with size information in Block Headers. The threaded
jpayne@68 1013 encoder in xz has always created such files.
jpayne@68 1014
jpayne@68 1015 Single-threaded encoder cannot store the size information in
jpayne@68 1016 Block Headers even if one used LZMA_FULL_FLUSH to create
jpayne@68 1017 multiple Blocks, so this threaded decoder cannot use multiple
jpayne@68 1018 threads with such files.
jpayne@68 1019
jpayne@68 1020 If there are multiple Streams (concatenated .xz files), one
jpayne@68 1021 Stream will be decompressed completely before starting the
jpayne@68 1022 next Stream.
jpayne@68 1023
jpayne@68 1024 - A new decoder flag LZMA_FAIL_FAST was added. It makes the
jpayne@68 1025 threaded decompressor report errors soon instead of first
jpayne@68 1026 flushing all pending data before the error location.
jpayne@68 1027
jpayne@68 1028 - New Filter IDs:
jpayne@68 1029 * LZMA_FILTER_ARM64 is for ARM64 binaries.
jpayne@68 1030 * LZMA_FILTER_LZMA1EXT is for raw LZMA1 streams that don't
jpayne@68 1031 necessarily use the end marker.
jpayne@68 1032
jpayne@68 1033 - Added lzma_str_to_filters(), lzma_str_from_filters(), and
jpayne@68 1034 lzma_str_list_filters() to convert a preset or a filter chain
jpayne@68 1035 string to a lzma_filter[] and vice versa. These should make
jpayne@68 1036 it easier to write applications that allow users to specify
jpayne@68 1037 custom compression options.
jpayne@68 1038
jpayne@68 1039 - Added lzma_filters_free() which can be convenient for freeing
jpayne@68 1040 the filter options in a filter chain (an array of lzma_filter
jpayne@68 1041 structures).
jpayne@68 1042
jpayne@68 1043 - lzma_file_info_decoder() to makes it a little easier to get
jpayne@68 1044 the Index field from .xz files. This helps in getting the
jpayne@68 1045 uncompressed file size but an easy-to-use random access
jpayne@68 1046 API is still missing which has existed in XZ for Java for
jpayne@68 1047 a long time.
jpayne@68 1048
jpayne@68 1049 - Added lzma_microlzma_encoder() and lzma_microlzma_decoder().
jpayne@68 1050 It is used by erofs-utils and may be used by others too.
jpayne@68 1051
jpayne@68 1052 The MicroLZMA format is a raw LZMA stream (without end marker)
jpayne@68 1053 whose first byte (always 0x00) has been replaced with
jpayne@68 1054 bitwise-negation of the LZMA properties (lc/lp/pb). It was
jpayne@68 1055 created for use in EROFS but may be used in other contexts
jpayne@68 1056 as well where it is important to avoid wasting bytes for
jpayne@68 1057 stream headers or footers. The format is also supported by
jpayne@68 1058 XZ Embedded (the XZ Embedded version in Linux got MicroLZMA
jpayne@68 1059 support in Linux 5.16).
jpayne@68 1060
jpayne@68 1061 The MicroLZMA encoder API in liblzma can compress into a
jpayne@68 1062 fixed-sized output buffer so that as much data is compressed
jpayne@68 1063 as can be fit into the buffer while still creating a valid
jpayne@68 1064 MicroLZMA stream. This is needed for EROFS.
jpayne@68 1065
jpayne@68 1066 - Added lzma_lzip_decoder() to decompress the .lz (lzip) file
jpayne@68 1067 format version 0 and the original unextended version 1 files.
jpayne@68 1068 Also lzma_auto_decoder() supports .lz files.
jpayne@68 1069
jpayne@68 1070 - lzma_filters_update() can now be used with the multi-threaded
jpayne@68 1071 encoder (lzma_stream_encoder_mt()) to change the filter chain
jpayne@68 1072 after LZMA_FULL_BARRIER or LZMA_FULL_FLUSH.
jpayne@68 1073
jpayne@68 1074 - In lzma_options_lzma, allow nice_len = 2 and 3 with the match
jpayne@68 1075 finders that require at least 3 or 4. Now it is internally
jpayne@68 1076 rounded up if needed.
jpayne@68 1077
jpayne@68 1078 - CLMUL-based CRC64 on x86-64 and E2K with runtime processor
jpayne@68 1079 detection. On 32-bit x86 it currently isn't available unless
jpayne@68 1080 --disable-assembler is used which can make the non-CLMUL
jpayne@68 1081 CRC64 slower; this might be fixed in the future.
jpayne@68 1082
jpayne@68 1083 - Building with --disable-threads --enable-small
jpayne@68 1084 is now thread-safe if the compiler supports
jpayne@68 1085 __attribute__((__constructor__)).
jpayne@68 1086
jpayne@68 1087 * xz:
jpayne@68 1088
jpayne@68 1089 - Using -T0 (--threads=0) will now use multi-threaded encoder
jpayne@68 1090 even on a single-core system. This is to ensure that output
jpayne@68 1091 from the same xz binary is identical on both single-core and
jpayne@68 1092 multi-core systems.
jpayne@68 1093
jpayne@68 1094 - --threads=+1 or -T+1 is now a way to put xz into
jpayne@68 1095 multi-threaded mode while using only one worker thread.
jpayne@68 1096 The + is ignored if the number is not 1.
jpayne@68 1097
jpayne@68 1098 - A default soft memory usage limit is now used for compression
jpayne@68 1099 when -T0 is used and no explicit limit has been specified.
jpayne@68 1100 This soft limit is used to restrict the number of threads
jpayne@68 1101 but if the limit is exceeded with even one thread then xz
jpayne@68 1102 will continue with one thread using the multi-threaded
jpayne@68 1103 encoder and this limit is ignored. If the number of threads
jpayne@68 1104 is specified manually then no default limit will be used;
jpayne@68 1105 this affects only -T0.
jpayne@68 1106
jpayne@68 1107 This change helps on systems that have very many cores and
jpayne@68 1108 using all of them for xz makes no sense. Previously xz -T0
jpayne@68 1109 could run out of memory on such systems because it attempted
jpayne@68 1110 to reserve memory for too many threads.
jpayne@68 1111
jpayne@68 1112 This also helps with 32-bit builds which don't have a large
jpayne@68 1113 amount of address space that would be required for many
jpayne@68 1114 threads. The default soft limit for -T0 is at most 1400 MiB
jpayne@68 1115 on all 32-bit platforms.
jpayne@68 1116
jpayne@68 1117 - Previously a low value in --memlimit-compress wouldn't cause
jpayne@68 1118 xz to switch from multi-threaded mode to single-threaded mode
jpayne@68 1119 if the limit cannot otherwise be met; xz failed instead. Now
jpayne@68 1120 xz can switch to single-threaded mode and then, if needed,
jpayne@68 1121 scale down the LZMA2 dictionary size too just like it already
jpayne@68 1122 did when it was started in single-threaded mode.
jpayne@68 1123
jpayne@68 1124 - The option --no-adjust no longer prevents xz from scaling down
jpayne@68 1125 the number of threads as that doesn't affect the compressed
jpayne@68 1126 output (only performance). Now --no-adjust only prevents
jpayne@68 1127 adjustments that affect compressed output, that is, with
jpayne@68 1128 --no-adjust xz won't switch from multi-threaded mode to
jpayne@68 1129 single-threaded mode and won't scale down the LZMA2
jpayne@68 1130 dictionary size.
jpayne@68 1131
jpayne@68 1132 - Added a new option --memlimit-mt-decompress=LIMIT. This is
jpayne@68 1133 used to limit the number of decompressor threads (possibly
jpayne@68 1134 falling back to single-threaded mode) but it will never make
jpayne@68 1135 xz refuse to decompress a file. This has a system-specific
jpayne@68 1136 default value because without any limit xz could end up
jpayne@68 1137 allocating memory for the whole compressed input file, the
jpayne@68 1138 whole uncompressed output file, multiple thread-specific
jpayne@68 1139 decompressor instances and so on. Basically xz could
jpayne@68 1140 attempt to use an insane amount of memory even with fairly
jpayne@68 1141 common files. The system-specific default value is currently
jpayne@68 1142 the same as the one used for compression with -T0.
jpayne@68 1143
jpayne@68 1144 The new option works together with the existing option
jpayne@68 1145 --memlimit-decompress=LIMIT. The old option sets a hard limit
jpayne@68 1146 that must not be exceeded (xz will refuse to decompress)
jpayne@68 1147 while the new option only restricts the number of threads.
jpayne@68 1148 If the limit set with --memlimit-mt-decompress is greater
jpayne@68 1149 than the limit set with --memlimit-compress, then the latter
jpayne@68 1150 value is used also for --memlimit-mt-decompress.
jpayne@68 1151
jpayne@68 1152 - Added new information to the output of xz --info-memory and
jpayne@68 1153 new fields to the output of xz --robot --info-memory.
jpayne@68 1154
jpayne@68 1155 - In --lzma2=nice=NUMBER allow 2 and 3 with all match finders
jpayne@68 1156 now that liblzma handles it.
jpayne@68 1157
jpayne@68 1158 - Don't mention endianness for ARM and ARM-Thumb filters in
jpayne@68 1159 --long-help. The filters only work for little endian
jpayne@68 1160 instruction encoding but modern ARM processors using
jpayne@68 1161 big endian data access still use little endian
jpayne@68 1162 instruction encoding. So the help text was misleading.
jpayne@68 1163 In contrast, the PowerPC filter is only for big endian
jpayne@68 1164 32/64-bit PowerPC code. Little endian PowerPC would need
jpayne@68 1165 a separate filter.
jpayne@68 1166
jpayne@68 1167 - Added decompression support for the .lz (lzip) file format
jpayne@68 1168 version 0 and the original unextended version 1. It is
jpayne@68 1169 autodetected by default. See also the option --format on
jpayne@68 1170 the xz man page.
jpayne@68 1171
jpayne@68 1172 - Sandboxing enabled by default:
jpayne@68 1173 * Capsicum (FreeBSD)
jpayne@68 1174 * pledge(2) (OpenBSD)
jpayne@68 1175
jpayne@68 1176 * Scripts now support the .lz format using xz.
jpayne@68 1177
jpayne@68 1178 * A few new tests were added.
jpayne@68 1179
jpayne@68 1180 * The liblzma-specific tests are now supported in CMake-based
jpayne@68 1181 builds too ("make test").
jpayne@68 1182
jpayne@68 1183
jpayne@68 1184 5.3.5beta (2022-12-01)
jpayne@68 1185
jpayne@68 1186 * All fixes from 5.2.9.
jpayne@68 1187
jpayne@68 1188 * liblzma:
jpayne@68 1189
jpayne@68 1190 - Added new LZMA_FILTER_LZMA1EXT for raw encoder and decoder to
jpayne@68 1191 handle raw LZMA1 streams that don't have end of payload marker
jpayne@68 1192 (EOPM) alias end of stream (EOS) marker. It can be used in
jpayne@68 1193 filter chains, for example, with the x86 BCJ filter.
jpayne@68 1194
jpayne@68 1195 - Added lzma_str_to_filters(), lzma_str_from_filters(), and
jpayne@68 1196 lzma_str_list_filters() to make it easier for applications
jpayne@68 1197 to get custom compression options from a user and convert
jpayne@68 1198 it to an array of lzma_filter structures.
jpayne@68 1199
jpayne@68 1200 - Added lzma_filters_free().
jpayne@68 1201
jpayne@68 1202 - lzma_filters_update() can now be used with the multi-threaded
jpayne@68 1203 encoder (lzma_stream_encoder_mt()) to change the filter chain
jpayne@68 1204 after LZMA_FULL_BARRIER or LZMA_FULL_FLUSH.
jpayne@68 1205
jpayne@68 1206 - In lzma_options_lzma, allow nice_len = 2 and 3 with the match
jpayne@68 1207 finders that require at least 3 or 4. Now it is internally
jpayne@68 1208 rounded up if needed.
jpayne@68 1209
jpayne@68 1210 - ARM64 filter was modified. It is still experimental.
jpayne@68 1211
jpayne@68 1212 - Fixed LTO build with Clang if -fgnuc-version=10 or similar
jpayne@68 1213 was used to make Clang look like GCC >= 10. Now it uses
jpayne@68 1214 __has_attribute(__symver__) which should be reliable.
jpayne@68 1215
jpayne@68 1216 * xz:
jpayne@68 1217
jpayne@68 1218 - --threads=+1 or -T+1 is now a way to put xz into multi-threaded
jpayne@68 1219 mode while using only one worker thread.
jpayne@68 1220
jpayne@68 1221 - In --lzma2=nice=NUMBER allow 2 and 3 with all match finders
jpayne@68 1222 now that liblzma handles it.
jpayne@68 1223
jpayne@68 1224 * Updated translations: Chinese (simplified), Korean, and Turkish.
jpayne@68 1225
jpayne@68 1226
jpayne@68 1227 5.3.4alpha (2022-11-15)
jpayne@68 1228
jpayne@68 1229 * All fixes from 5.2.7 and 5.2.8.
jpayne@68 1230
jpayne@68 1231 * liblzma:
jpayne@68 1232
jpayne@68 1233 - Minor improvements to the threaded decoder.
jpayne@68 1234
jpayne@68 1235 - Added CRC64 implementation that uses SSSE3, SSE4.1, and CLMUL
jpayne@68 1236 instructions on 32/64-bit x86 and E2K. On 32-bit x86 it's
jpayne@68 1237 not enabled unless --disable-assembler is used but then
jpayne@68 1238 the non-CLMUL code might be slower. Processor support is
jpayne@68 1239 detected at runtime so this is built by default on x86-64
jpayne@68 1240 and E2K. On these platforms, if compiler flags indicate
jpayne@68 1241 unconditional CLMUL support (-msse4.1 -mpclmul) then the
jpayne@68 1242 generic version is not built, making liblzma 8-9 KiB smaller
jpayne@68 1243 compared to having both versions included.
jpayne@68 1244
jpayne@68 1245 With extremely compressible files this can make decompression
jpayne@68 1246 up to twice as fast but with typical files 5 % improvement
jpayne@68 1247 is a more realistic expectation.
jpayne@68 1248
jpayne@68 1249 The CLMUL version is slower than the generic version with
jpayne@68 1250 tiny inputs (especially at 1-8 bytes per call, but up to
jpayne@68 1251 16 bytes). In normal use in xz this doesn't matter at all.
jpayne@68 1252
jpayne@68 1253 - Added an experimental ARM64 filter. This is *not* the final
jpayne@68 1254 version! Files created with this experimental version won't
jpayne@68 1255 be supported in the future versions! The filter design is
jpayne@68 1256 a compromise where improving one use case makes some other
jpayne@68 1257 cases worse.
jpayne@68 1258
jpayne@68 1259 - Added decompression support for the .lz (lzip) file format
jpayne@68 1260 version 0 and the original unextended version 1. See the
jpayne@68 1261 API docs of lzma_lzip_decoder() for details. Also
jpayne@68 1262 lzma_auto_decoder() supports .lz files.
jpayne@68 1263
jpayne@68 1264 - Building with --disable-threads --enable-small
jpayne@68 1265 is now thread-safe if the compiler supports
jpayne@68 1266 __attribute__((__constructor__))
jpayne@68 1267
jpayne@68 1268 * xz:
jpayne@68 1269
jpayne@68 1270 - Added support for OpenBSD's pledge(2) as a sandboxing method.
jpayne@68 1271
jpayne@68 1272 - Don't mention endianness for ARM and ARM-Thumb filters in
jpayne@68 1273 --long-help. The filters only work for little endian
jpayne@68 1274 instruction encoding but modern ARM processors using
jpayne@68 1275 big endian data access still use little endian
jpayne@68 1276 instruction encoding. So the help text was misleading.
jpayne@68 1277 In contrast, the PowerPC filter is only for big endian
jpayne@68 1278 32/64-bit PowerPC code. Little endian PowerPC would need
jpayne@68 1279 a separate filter.
jpayne@68 1280
jpayne@68 1281 - Added --experimental-arm64. This will be renamed once the
jpayne@68 1282 filter is finished. Files created with this experimental
jpayne@68 1283 filter will not be supported in the future!
jpayne@68 1284
jpayne@68 1285 - Added new fields to the output of xz --robot --info-memory.
jpayne@68 1286
jpayne@68 1287 - Added decompression support for the .lz (lzip) file format
jpayne@68 1288 version 0 and the original unextended version 1. It is
jpayne@68 1289 autodetected by default. See also the option --format on
jpayne@68 1290 the xz man page.
jpayne@68 1291
jpayne@68 1292 * Scripts now support the .lz format using xz.
jpayne@68 1293
jpayne@68 1294 * Build systems:
jpayne@68 1295
jpayne@68 1296 - New #defines in config.h: HAVE_ENCODER_ARM64,
jpayne@68 1297 HAVE_DECODER_ARM64, HAVE_LZIP_DECODER, HAVE_CPUID_H,
jpayne@68 1298 HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR, HAVE_USABLE_CLMUL
jpayne@68 1299
jpayne@68 1300 - New configure options: --disable-clmul-crc,
jpayne@68 1301 --disable-microlzma, --disable-lzip-decoder, and
jpayne@68 1302 'pledge' is now an option in --enable-sandbox (but
jpayne@68 1303 it's autodetected by default anyway).
jpayne@68 1304
jpayne@68 1305 - INSTALL was updated to document the new configure options.
jpayne@68 1306
jpayne@68 1307 - PACKAGERS now lists also --disable-microlzma and
jpayne@68 1308 --disable-lzip-decoder as configure options that must
jpayne@68 1309 not be used in builds for non-embedded use.
jpayne@68 1310
jpayne@68 1311 * Tests:
jpayne@68 1312
jpayne@68 1313 - Fix some of the tests so that they skip instead of fail if
jpayne@68 1314 certain features have been disabled with configure options.
jpayne@68 1315 It's still not perfect.
jpayne@68 1316
jpayne@68 1317 - Other improvements to tests.
jpayne@68 1318
jpayne@68 1319 * Updated translations: Croatian, Finnish, Hungarian, Polish,
jpayne@68 1320 Romanian, Spanish, Swedish, and Ukrainian.
jpayne@68 1321
jpayne@68 1322
jpayne@68 1323 5.3.3alpha (2022-08-22)
jpayne@68 1324
jpayne@68 1325 * All fixes from 5.2.6.
jpayne@68 1326
jpayne@68 1327 * liblzma:
jpayne@68 1328
jpayne@68 1329 - Fixed 32-bit build.
jpayne@68 1330
jpayne@68 1331 - Added threaded .xz decompressor lzma_stream_decoder_mt().
jpayne@68 1332 It can use multiple threads with .xz files that have multiple
jpayne@68 1333 Blocks with size information in Block Headers. The threaded
jpayne@68 1334 encoder in xz has always created such files.
jpayne@68 1335
jpayne@68 1336 Single-threaded encoder cannot store the size information in
jpayne@68 1337 Block Headers even if one used LZMA_FULL_FLUSH to create
jpayne@68 1338 multiple Blocks, so this threaded decoder cannot use multiple
jpayne@68 1339 threads with such files.
jpayne@68 1340
jpayne@68 1341 If there are multiple Streams (concatenated .xz files), one
jpayne@68 1342 Stream will be decompressed completely before starting the
jpayne@68 1343 next Stream.
jpayne@68 1344
jpayne@68 1345 - A new decoder flag LZMA_FAIL_FAST was added. It makes the
jpayne@68 1346 threaded decompressor report errors soon instead of first
jpayne@68 1347 flushing all pending data before the error location.
jpayne@68 1348
jpayne@68 1349 * xz:
jpayne@68 1350
jpayne@68 1351 - Using -T0 (--threads=0) will now use multi-threaded encoder
jpayne@68 1352 even on a single-core system. This is to ensure that output
jpayne@68 1353 from the same xz binary is identical on both single-core and
jpayne@68 1354 multi-core systems.
jpayne@68 1355
jpayne@68 1356 - A default soft memory usage limit is now used for compression
jpayne@68 1357 when -T0 is used and no explicit limit has been specified.
jpayne@68 1358 This soft limit is used to restrict the number of threads
jpayne@68 1359 but if the limit is exceeded with even one thread then xz
jpayne@68 1360 will continue with one thread using the multi-threaded
jpayne@68 1361 encoder and this limit is ignored. If the number of threads
jpayne@68 1362 is specified manually then no default limit will be used;
jpayne@68 1363 this affects only -T0.
jpayne@68 1364
jpayne@68 1365 This change helps on systems that have very many cores and
jpayne@68 1366 using all of them for xz makes no sense. Previously xz -T0
jpayne@68 1367 could run out of memory on such systems because it attempted
jpayne@68 1368 to reserve memory for too many threads.
jpayne@68 1369
jpayne@68 1370 This also helps with 32-bit builds which don't have a large
jpayne@68 1371 amount of address space that would be required for many
jpayne@68 1372 threads. The default limit is 1400 MiB on all 32-bit
jpayne@68 1373 platforms with -T0.
jpayne@68 1374
jpayne@68 1375 Now xz -T0 should just work. It might use too few threads
jpayne@68 1376 in some cases but at least it shouldn't easily run out of
jpayne@68 1377 memory. It's possible that this will be tweaked before 5.4.0.
jpayne@68 1378
jpayne@68 1379 - Changes to --memlimit-compress and --no-adjust:
jpayne@68 1380
jpayne@68 1381 In single-threaded mode, --memlimit-compress can make xz
jpayne@68 1382 scale down the LZMA2 dictionary size to meet the memory usage
jpayne@68 1383 limit. This obviously affects the compressed output. However,
jpayne@68 1384 if xz was in threaded mode, --memlimit-compress could make xz
jpayne@68 1385 reduce the number of threads but it wouldn't make xz switch
jpayne@68 1386 from multi-threaded mode to single-threaded mode or scale
jpayne@68 1387 down the LZMA2 dictionary size. This seemed illogical.
jpayne@68 1388
jpayne@68 1389 Now --memlimit-compress can make xz switch to single-threaded
jpayne@68 1390 mode if one thread in multi-threaded mode uses too much
jpayne@68 1391 memory. If memory usage is still too high, then the LZMA2
jpayne@68 1392 dictionary size can be scaled down too.
jpayne@68 1393
jpayne@68 1394 The option --no-adjust was also changed so that it no longer
jpayne@68 1395 prevents xz from scaling down the number of threads as that
jpayne@68 1396 doesn't affect compressed output (only performance). After
jpayne@68 1397 this commit --no-adjust only prevents adjustments that affect
jpayne@68 1398 compressed output, that is, with --no-adjust xz won't switch
jpayne@68 1399 from multithreaded mode to single-threaded mode and won't
jpayne@68 1400 scale down the LZMA2 dictionary size.
jpayne@68 1401
jpayne@68 1402 - Added a new option --memlimit-mt-decompress=LIMIT. This is
jpayne@68 1403 used to limit the number of decompressor threads (possibly
jpayne@68 1404 falling back to single-threaded mode) but it will never make
jpayne@68 1405 xz refuse to decompress a file. This has a system-specific
jpayne@68 1406 default value because without any limit xz could end up
jpayne@68 1407 allocating memory for the whole compressed input file, the
jpayne@68 1408 whole uncompressed output file, multiple thread-specific
jpayne@68 1409 decompressor instances and so on. Basically xz could
jpayne@68 1410 attempt to use an insane amount of memory even with fairly
jpayne@68 1411 common files.
jpayne@68 1412
jpayne@68 1413 The new option works together with the existing option
jpayne@68 1414 --memlimit-decompress=LIMIT. The old option sets a hard limit
jpayne@68 1415 that must not be exceeded (xz will refuse to decompress)
jpayne@68 1416 while the new option only restricts the number of threads.
jpayne@68 1417 If the limit set with --memlimit-mt-decompress is greater
jpayne@68 1418 than the limit set with --memlimit-compress, then the latter
jpayne@68 1419 value is used also for --memlimit-mt-decompress.
jpayne@68 1420
jpayne@68 1421 * Tests:
jpayne@68 1422
jpayne@68 1423 - Added a few more tests.
jpayne@68 1424
jpayne@68 1425 - Added tests/code_coverage.sh to create a code coverage report
jpayne@68 1426 of the tests.
jpayne@68 1427
jpayne@68 1428 * Build systems:
jpayne@68 1429
jpayne@68 1430 - Automake's parallel test harness is now used to make tests
jpayne@68 1431 finish faster.
jpayne@68 1432
jpayne@68 1433 - Added the CMake files to the distribution tarball. These were
jpayne@68 1434 supposed to be in 5.2.5 already.
jpayne@68 1435
jpayne@68 1436 - Added liblzma tests to the CMake build.
jpayne@68 1437
jpayne@68 1438 - Windows: Fix building of liblzma.dll with the included
jpayne@68 1439 Visual Studio project files.
jpayne@68 1440
jpayne@68 1441
jpayne@68 1442 5.3.2alpha (2021-10-28)
jpayne@68 1443
jpayne@68 1444 This release was made on short notice so that recent erofs-utils can
jpayne@68 1445 be built with LZMA support without needing a snapshot from xz.git.
jpayne@68 1446 Thus many pending things were not included, not even updated
jpayne@68 1447 translations (which would need to be updated for the new --list
jpayne@68 1448 strings anyway).
jpayne@68 1449
jpayne@68 1450 * All fixes from 5.2.5.
jpayne@68 1451
jpayne@68 1452 * xz:
jpayne@68 1453
jpayne@68 1454 - When copying metadata from the source file to the destination
jpayne@68 1455 file, don't try to set the group (GID) if it is already set
jpayne@68 1456 correctly. This avoids a failure on OpenBSD (and possibly on
jpayne@68 1457 a few other OSes) where files may get created so that their
jpayne@68 1458 group doesn't belong to the user, and fchown(2) can fail even
jpayne@68 1459 if it needs to do nothing.
jpayne@68 1460
jpayne@68 1461 - The --keep option now accepts symlinks, hardlinks, and
jpayne@68 1462 setuid, setgid, and sticky files. Previously this required
jpayne@68 1463 using --force.
jpayne@68 1464
jpayne@68 1465 - Split the long strings used in --list and --info-memory modes
jpayne@68 1466 to make them much easier for translators.
jpayne@68 1467
jpayne@68 1468 - If built with sandbox support and enabling the sandbox fails,
jpayne@68 1469 xz will now immediately exit with exit status of 1. Previously
jpayne@68 1470 it would only display a warning if -vv was used.
jpayne@68 1471
jpayne@68 1472 - Cap --memlimit-compress to 2000 MiB on MIPS32 because on
jpayne@68 1473 MIPS32 userspace processes are limited to 2 GiB of address
jpayne@68 1474 space.
jpayne@68 1475
jpayne@68 1476 * liblzma:
jpayne@68 1477
jpayne@68 1478 - Added lzma_microlzma_encoder() and lzma_microlzma_decoder().
jpayne@68 1479 The API is in lzma/container.h.
jpayne@68 1480
jpayne@68 1481 The MicroLZMA format is a raw LZMA stream (without end marker)
jpayne@68 1482 whose first byte (always 0x00) has been replaced with
jpayne@68 1483 bitwise-negation of the LZMA properties (lc/lp/pb). It was
jpayne@68 1484 created for use in EROFS but may be used in other contexts
jpayne@68 1485 as well where it is important to avoid wasting bytes for
jpayne@68 1486 stream headers or footers. The format is also supported by
jpayne@68 1487 XZ Embedded.
jpayne@68 1488
jpayne@68 1489 The MicroLZMA encoder API in liblzma can compress into a
jpayne@68 1490 fixed-sized output buffer so that as much data is compressed
jpayne@68 1491 as can be fit into the buffer while still creating a valid
jpayne@68 1492 MicroLZMA stream. This is needed for EROFS.
jpayne@68 1493
jpayne@68 1494 - Added fuzzing support.
jpayne@68 1495
jpayne@68 1496 - Support Intel Control-flow Enforcement Technology (CET) in
jpayne@68 1497 32-bit x86 assembly files.
jpayne@68 1498
jpayne@68 1499 - Visual Studio: Use non-standard _MSVC_LANG to detect C++
jpayne@68 1500 standard version in the lzma.h API header. It's used to
jpayne@68 1501 detect when "noexcept" can be used.
jpayne@68 1502
jpayne@68 1503 * Scripts:
jpayne@68 1504
jpayne@68 1505 - Fix exit status of xzdiff/xzcmp. Exit status could be 2 when
jpayne@68 1506 the correct value is 1.
jpayne@68 1507
jpayne@68 1508 - Fix exit status of xzgrep.
jpayne@68 1509
jpayne@68 1510 - Detect corrupt .bz2 files in xzgrep.
jpayne@68 1511
jpayne@68 1512 - Add zstd support to xzgrep and xzdiff/xzcmp.
jpayne@68 1513
jpayne@68 1514 - Fix less(1) version detection in xzless. It failed if the
jpayne@68 1515 version number from "less -V" contained a dot.
jpayne@68 1516
jpayne@68 1517 * Fix typos and technical issues in man pages.
jpayne@68 1518
jpayne@68 1519 * Build systems:
jpayne@68 1520
jpayne@68 1521 - Windows: Fix building of resource files when config.h isn't
jpayne@68 1522 used. CMake + Visual Studio can now build liblzma.dll.
jpayne@68 1523
jpayne@68 1524 - Various fixes to the CMake support. It might still need a few
jpayne@68 1525 more fixes even for liblzma-only builds.
jpayne@68 1526
jpayne@68 1527
jpayne@68 1528 5.3.1alpha (2018-04-29)
jpayne@68 1529
jpayne@68 1530 * All fixes from 5.2.4.
jpayne@68 1531
jpayne@68 1532 * Add lzma_file_info_decoder() into liblzma and use it in xz to
jpayne@68 1533 implement the --list feature.
jpayne@68 1534
jpayne@68 1535 * Capsicum sandbox support is enabled by default where available
jpayne@68 1536 (FreeBSD >= 10).
jpayne@68 1537
jpayne@68 1538
jpayne@68 1539 5.2.13 (2024-05-29)
jpayne@68 1540
jpayne@68 1541 * liblzma:
jpayne@68 1542
jpayne@68 1543 - lzma_index_append(): Fix an assertion failure that could be
jpayne@68 1544 triggered by a large unpadded_size argument. It was verified
jpayne@68 1545 that there was no other bug than the assertion failure.
jpayne@68 1546
jpayne@68 1547 - lzma_index_decoder() and lzma_index_buffer_decode(): Fix
jpayne@68 1548 a missing output pointer initialization (*i = NULL) if the
jpayne@68 1549 functions are called with invalid arguments. The API docs
jpayne@68 1550 say that such an initialization is always done. In practice
jpayne@68 1551 this matters very little because the problem can only occur
jpayne@68 1552 if the calling application has a bug and these functions
jpayne@68 1553 return LZMA_PROG_ERROR.
jpayne@68 1554
jpayne@68 1555 - Fix C standard conformance with function pointer types.
jpayne@68 1556 This newly showed up with Clang 17 with -fsanitize=undefined.
jpayne@68 1557 There are no bug reports about this.
jpayne@68 1558
jpayne@68 1559 - Fix building with NVIDIA HPC SDK.
jpayne@68 1560
jpayne@68 1561 - Fix building with Windows Vista threads and --enable-small.
jpayne@68 1562 (CMake build doesn't support ENABLE_SMALL in XZ Utils 5.2.x.)
jpayne@68 1563
jpayne@68 1564 * xz:
jpayne@68 1565
jpayne@68 1566 - Fix a C standard conformance issue in --block-list parsing
jpayne@68 1567 (arithmetic on a null pointer).
jpayne@68 1568
jpayne@68 1569 - Fix a warning from GNU groff when processing the man page:
jpayne@68 1570 "warning: cannot select font 'CW'"
jpayne@68 1571
jpayne@68 1572 - Windows: Handle special files such as "con" or "nul". Earlier
jpayne@68 1573 the following wrote "foo" to the console and deleted the input
jpayne@68 1574 file "con_xz":
jpayne@68 1575
jpayne@68 1576 echo foo | xz > con_xz
jpayne@68 1577 xz --suffix=_xz --decompress con_xz
jpayne@68 1578
jpayne@68 1579 - Windows: Fix an issue that prevented reading from or writing
jpayne@68 1580 to non-terminal character devices like NUL.
jpayne@68 1581
jpayne@68 1582 * xzless:
jpayne@68 1583
jpayne@68 1584 - With "less" version 451 and later, use "||-" instead of "|-"
jpayne@68 1585 in the environment variable LESSOPEN. This way compressed
jpayne@68 1586 files that contain no uncompressed data are shown correctly
jpayne@68 1587 as empty.
jpayne@68 1588
jpayne@68 1589 - With "less" version 632 and later, use --show-preproc-errors
jpayne@68 1590 to make "less" show a warning on decompression errors.
jpayne@68 1591
jpayne@68 1592 * Build systems:
jpayne@68 1593
jpayne@68 1594 - Add a new line to liblzma.pc for MSYS2 (Windows):
jpayne@68 1595
jpayne@68 1596 Cflags.private: -DLZMA_API_STATIC
jpayne@68 1597
jpayne@68 1598 When compiling code that will link against static liblzma,
jpayne@68 1599 the LZMA_API_STATIC macro needs to be defined on Windows.
jpayne@68 1600
jpayne@68 1601 - Autotools (configure):
jpayne@68 1602
jpayne@68 1603 * Symbol versioning variant can now be overridden with
jpayne@68 1604 --enable-symbol-versions. Documentation in INSTALL was
jpayne@68 1605 updated to match.
jpayne@68 1606
jpayne@68 1607 - CMake:
jpayne@68 1608
jpayne@68 1609 * Fix a bug that prevented other projects from including
jpayne@68 1610 liblzma multiple times using find_package().
jpayne@68 1611
jpayne@68 1612 * Fix a bug where configuring CMake multiple times resulted
jpayne@68 1613 in HAVE_CLOCK_GETTIME and HAVE_CLOCK_MONOTONIC not being
jpayne@68 1614 defined.
jpayne@68 1615
jpayne@68 1616 * Fix the build with MinGW-w64-based Clang/LLVM 17.
jpayne@68 1617 llvm-windres now has more accurate GNU windres emulation
jpayne@68 1618 so the GNU windres workaround from 5.4.1 is needed with
jpayne@68 1619 llvm-windres version 17 too.
jpayne@68 1620
jpayne@68 1621 * The import library on Windows is now properly named
jpayne@68 1622 "liblzma.dll.a" instead of "libliblzma.dll.a"
jpayne@68 1623
jpayne@68 1624 * Add large file support by default for platforms that
jpayne@68 1625 need it to handle files larger than 2 GiB. This includes
jpayne@68 1626 MinGW-w64, even 64-bit builds.
jpayne@68 1627
jpayne@68 1628 * Linux on MicroBlaze is handled specially now. This
jpayne@68 1629 matches the changes made to the Autotools-based build
jpayne@68 1630 in XZ Utils 5.4.2 and 5.2.11.
jpayne@68 1631
jpayne@68 1632 * Disable symbol versioning on non-glibc Linux to match
jpayne@68 1633 what the Autotools build does. For example, symbol
jpayne@68 1634 versioning isn't enabled with musl.
jpayne@68 1635
jpayne@68 1636 * Symbol versioning variant can now be overridden by
jpayne@68 1637 setting SYMBOL_VERSIONING to "OFF", "generic", or
jpayne@68 1638 "linux".
jpayne@68 1639
jpayne@68 1640 * Documentation:
jpayne@68 1641
jpayne@68 1642 - Clarify the description of --disable-assembler in INSTALL.
jpayne@68 1643 The option only affects 32-bit x86 assembly usage.
jpayne@68 1644
jpayne@68 1645 - Don't install the TODO file as part of the documentation.
jpayne@68 1646 The file is out of date.
jpayne@68 1647
jpayne@68 1648 - Update home page URLs back to their old locations on
jpayne@68 1649 tukaani.org.
jpayne@68 1650
jpayne@68 1651 - Update maintainer info.
jpayne@68 1652
jpayne@68 1653
jpayne@68 1654 5.2.12 (2023-05-04)
jpayne@68 1655
jpayne@68 1656 * Fixed a build system bug that prevented building liblzma as a
jpayne@68 1657 shared library when configured with --disable-threads. This bug
jpayne@68 1658 affected releases 5.2.6 to 5.2.11 and 5.4.0 to 5.4.2.
jpayne@68 1659
jpayne@68 1660 * Include <intrin.h> for Windows intrinsic functions where they are
jpayne@68 1661 needed. This fixed a bug that prevented building liblzma using
jpayne@68 1662 clang-cl on Windows.
jpayne@68 1663
jpayne@68 1664 * Minor update to the Croatian translation. The small change
jpayne@68 1665 applies to a string in both 5.2 and 5.4 branches.
jpayne@68 1666
jpayne@68 1667
jpayne@68 1668 5.2.11 (2023-03-18)
jpayne@68 1669
jpayne@68 1670 * Removed all possible cases of null pointer + 0. It is undefined
jpayne@68 1671 behavior in C99 and C17. This was detected by a sanitizer and had
jpayne@68 1672 not caused any known issues.
jpayne@68 1673
jpayne@68 1674 * Build systems:
jpayne@68 1675
jpayne@68 1676 - Added a workaround for building with GCC on MicroBlaze Linux.
jpayne@68 1677 GCC 12 on MicroBlaze doesn't support the __symver__ attribute
jpayne@68 1678 even though __has_attribute(__symver__) returns true. The
jpayne@68 1679 build is now done without the extra RHEL/CentOS 7 symbols
jpayne@68 1680 that were added in XZ Utils 5.2.7. The workaround only
jpayne@68 1681 applies to the Autotools build (not CMake).
jpayne@68 1682
jpayne@68 1683 - CMake: Ensure that the C compiler language is set to C99 or
jpayne@68 1684 a newer standard.
jpayne@68 1685
jpayne@68 1686 - CMake changes from XZ Utils 5.4.1:
jpayne@68 1687
jpayne@68 1688 * Added a workaround for a build failure with
jpayne@68 1689 windres from GNU binutils.
jpayne@68 1690
jpayne@68 1691 * Included the Windows resource files in the xz
jpayne@68 1692 and xzdec build rules.
jpayne@68 1693
jpayne@68 1694
jpayne@68 1695 5.2.10 (2022-12-13)
jpayne@68 1696
jpayne@68 1697 * xz: Don't modify argv[] when parsing the --memlimit* and
jpayne@68 1698 --block-list command line options. This fixes confusing
jpayne@68 1699 arguments in process listing (like "ps auxf").
jpayne@68 1700
jpayne@68 1701 * GNU/Linux only: Use __has_attribute(__symver__) to detect if
jpayne@68 1702 that attribute is supported. This fixes build on Mandriva where
jpayne@68 1703 Clang is patched to define __GNUC__ to 11 by default (instead
jpayne@68 1704 of 4 as used by Clang upstream).
jpayne@68 1705
jpayne@68 1706
jpayne@68 1707 5.2.9 (2022-11-30)
jpayne@68 1708
jpayne@68 1709 * liblzma:
jpayne@68 1710
jpayne@68 1711 - Fixed an infinite loop in LZMA encoder initialization
jpayne@68 1712 if dict_size >= 2 GiB. (The encoder only supports up
jpayne@68 1713 to 1536 MiB.)
jpayne@68 1714
jpayne@68 1715 - Fixed two cases of invalid free() that can happen if
jpayne@68 1716 a tiny allocation fails in encoder re-initialization
jpayne@68 1717 or in lzma_filters_update(). These bugs had some
jpayne@68 1718 similarities with the bug fixed in 5.2.7.
jpayne@68 1719
jpayne@68 1720 - Fixed lzma_block_encoder() not allowing the use of
jpayne@68 1721 LZMA_SYNC_FLUSH with lzma_code() even though it was
jpayne@68 1722 documented to be supported. The sync-flush code in
jpayne@68 1723 the Block encoder was already used internally via
jpayne@68 1724 lzma_stream_encoder(), so this was just a missing flag
jpayne@68 1725 in the lzma_block_encoder() API function.
jpayne@68 1726
jpayne@68 1727 - GNU/Linux only: Don't put symbol versions into static
jpayne@68 1728 liblzma as it breaks things in some cases (and even if
jpayne@68 1729 it didn't break anything, symbol versions in static
jpayne@68 1730 libraries are useless anyway). The downside of the fix
jpayne@68 1731 is that if the configure options --with-pic or --without-pic
jpayne@68 1732 are used then it's not possible to build both shared and
jpayne@68 1733 static liblzma at the same time on GNU/Linux anymore;
jpayne@68 1734 with those options --disable-static or --disable-shared
jpayne@68 1735 must be used too.
jpayne@68 1736
jpayne@68 1737 * New email address for bug reports is <xz@tukaani.org> which
jpayne@68 1738 forwards messages to Lasse Collin and Jia Tan.
jpayne@68 1739
jpayne@68 1740
jpayne@68 1741 5.2.8 (2022-11-13)
jpayne@68 1742
jpayne@68 1743 * xz:
jpayne@68 1744
jpayne@68 1745 - If xz cannot remove an input file when it should, this
jpayne@68 1746 is now treated as a warning (exit status 2) instead of
jpayne@68 1747 an error (exit status 1). This matches GNU gzip and it
jpayne@68 1748 is more logical as at that point the output file has
jpayne@68 1749 already been successfully closed.
jpayne@68 1750
jpayne@68 1751 - Fix handling of .xz files with an unsupported check type.
jpayne@68 1752 Previously such printed a warning message but then xz
jpayne@68 1753 behaved as if an error had occurred (didn't decompress,
jpayne@68 1754 exit status 1). Now a warning is printed, decompression
jpayne@68 1755 is done anyway, and exit status is 2. This used to work
jpayne@68 1756 slightly before 5.0.0. In practice this bug matters only
jpayne@68 1757 if xz has been built with some check types disabled. As
jpayne@68 1758 instructed in PACKAGERS, such builds should be done in
jpayne@68 1759 special situations only.
jpayne@68 1760
jpayne@68 1761 - Fix "xz -dc --single-stream tests/files/good-0-empty.xz"
jpayne@68 1762 which failed with "Internal error (bug)". That is,
jpayne@68 1763 --single-stream was broken if the first .xz stream in
jpayne@68 1764 the input file didn't contain any uncompressed data.
jpayne@68 1765
jpayne@68 1766 - Fix displaying file sizes in the progress indicator when
jpayne@68 1767 working in passthru mode and there are multiple input files.
jpayne@68 1768 Just like "gzip -cdf", "xz -cdf" works like "cat" when the
jpayne@68 1769 input file isn't a supported compressed file format. In
jpayne@68 1770 this case the file size counters weren't reset between
jpayne@68 1771 files so with multiple input files the progress indicator
jpayne@68 1772 displayed an incorrect (too large) value.
jpayne@68 1773
jpayne@68 1774 * liblzma:
jpayne@68 1775
jpayne@68 1776 - API docs in lzma/container.h:
jpayne@68 1777 * Update the list of decoder flags in the decoder
jpayne@68 1778 function docs.
jpayne@68 1779 * Explain LZMA_CONCATENATED behavior with .lzma files
jpayne@68 1780 in lzma_auto_decoder() docs.
jpayne@68 1781
jpayne@68 1782 - OpenBSD: Use HW_NCPUONLINE to detect the number of
jpayne@68 1783 available hardware threads in lzma_physmem().
jpayne@68 1784
jpayne@68 1785 - Fix use of wrong macro to detect x86 SSE2 support.
jpayne@68 1786 __SSE2_MATH__ was used with GCC/Clang but the correct
jpayne@68 1787 one is __SSE2__. The first one means that SSE2 is used
jpayne@68 1788 for floating point math which is irrelevant here.
jpayne@68 1789 The affected SSE2 code isn't used on x86-64 so this affects
jpayne@68 1790 only 32-bit x86 builds that use -msse2 without -mfpmath=sse
jpayne@68 1791 (there is no runtime detection for SSE2). It improves LZMA
jpayne@68 1792 compression speed (not decompression).
jpayne@68 1793
jpayne@68 1794 - Fix the build with Intel C compiler 2021 (ICC, not ICX)
jpayne@68 1795 on Linux. It defines __GNUC__ to 10 but doesn't support
jpayne@68 1796 the __symver__ attribute introduced in GCC 10.
jpayne@68 1797
jpayne@68 1798 * Scripts: Ignore warnings from xz by using --quiet --no-warn.
jpayne@68 1799 This is needed if the input .xz files use an unsupported
jpayne@68 1800 check type.
jpayne@68 1801
jpayne@68 1802 * Translations:
jpayne@68 1803
jpayne@68 1804 - Updated Croatian and Turkish translations.
jpayne@68 1805
jpayne@68 1806 - One new translations wasn't included because it needed
jpayne@68 1807 technical fixes. It will be in upcoming 5.4.0. No new
jpayne@68 1808 translations will be added to the 5.2.x branch anymore.
jpayne@68 1809
jpayne@68 1810 - Renamed the French man page translation file from
jpayne@68 1811 fr_FR.po to fr.po and thus also its install directory
jpayne@68 1812 (like /usr/share/man/fr_FR -> .../fr).
jpayne@68 1813
jpayne@68 1814 - Man page translations for upcoming 5.4.0 are now handled
jpayne@68 1815 in the Translation Project.
jpayne@68 1816
jpayne@68 1817 * Update doc/faq.txt a little so it's less out-of-date.
jpayne@68 1818
jpayne@68 1819
jpayne@68 1820 5.2.7 (2022-09-30)
jpayne@68 1821
jpayne@68 1822 * liblzma:
jpayne@68 1823
jpayne@68 1824 - Made lzma_filters_copy() to never modify the destination
jpayne@68 1825 array if an error occurs. lzma_stream_encoder() and
jpayne@68 1826 lzma_stream_encoder_mt() already assumed this. Before this
jpayne@68 1827 change, if a tiny memory allocation in lzma_filters_copy()
jpayne@68 1828 failed it would lead to a crash (invalid free() or invalid
jpayne@68 1829 memory reads) in the cleanup paths of these two encoder
jpayne@68 1830 initialization functions.
jpayne@68 1831
jpayne@68 1832 - Added missing integer overflow check to lzma_index_append().
jpayne@68 1833 This affects xz --list and other applications that decode
jpayne@68 1834 the Index field from .xz files using lzma_index_decoder().
jpayne@68 1835 Normal decompression of .xz files doesn't call this code
jpayne@68 1836 and thus most applications using liblzma aren't affected
jpayne@68 1837 by this bug.
jpayne@68 1838
jpayne@68 1839 - Single-threaded .xz decoder (lzma_stream_decoder()): If
jpayne@68 1840 lzma_code() returns LZMA_MEMLIMIT_ERROR it is now possible
jpayne@68 1841 to use lzma_memlimit_set() to increase the limit and continue
jpayne@68 1842 decoding. This was supposed to work from the beginning
jpayne@68 1843 but there was a bug. With other decoders (.lzma or
jpayne@68 1844 threaded .xz decoder) this already worked correctly.
jpayne@68 1845
jpayne@68 1846 - Fixed accumulation of integrity check type statistics in
jpayne@68 1847 lzma_index_cat(). This bug made lzma_index_checks() return
jpayne@68 1848 only the type of the integrity check of the last Stream
jpayne@68 1849 when multiple lzma_indexes were concatenated. Most
jpayne@68 1850 applications don't use these APIs but in xz it made
jpayne@68 1851 xz --list not list all check types from concatenated .xz
jpayne@68 1852 files. In xz --list --verbose only the per-file "Check:"
jpayne@68 1853 lines were affected and in xz --robot --list only the "file"
jpayne@68 1854 line was affected.
jpayne@68 1855
jpayne@68 1856 - Added ABI compatibility with executables that were linked
jpayne@68 1857 against liblzma in RHEL/CentOS 7 or other liblzma builds
jpayne@68 1858 that had copied the problematic patch from RHEL/CentOS 7
jpayne@68 1859 (xz-5.2.2-compat-libs.patch). For the details, see the
jpayne@68 1860 comment at the top of src/liblzma/validate_map.sh.
jpayne@68 1861
jpayne@68 1862 WARNING: This uses __symver__ attribute with GCC >= 10.
jpayne@68 1863 In other cases the traditional __asm__(".symver ...")
jpayne@68 1864 is used. Using link-time optimization (LTO, -flto) with
jpayne@68 1865 GCC versions older than 10 can silently result in
jpayne@68 1866 broken liblzma.so.5 (incorrect symbol versions)! If you
jpayne@68 1867 want to use -flto with GCC, you must use GCC >= 10.
jpayne@68 1868 LTO with Clang seems to work even with the traditional
jpayne@68 1869 __asm__(".symver ...") method.
jpayne@68 1870
jpayne@68 1871 * xzgrep: Fixed compatibility with old shells that break if
jpayne@68 1872 comments inside command substitutions have apostrophes (').
jpayne@68 1873 This problem was introduced in 5.2.6.
jpayne@68 1874
jpayne@68 1875 * Build systems:
jpayne@68 1876
jpayne@68 1877 - New #define in config.h: HAVE_SYMBOL_VERSIONS_LINUX
jpayne@68 1878
jpayne@68 1879 - Windows: Fixed liblzma.dll build with Visual Studio project
jpayne@68 1880 files. It broke in 5.2.6 due to a change that was made to
jpayne@68 1881 improve CMake support.
jpayne@68 1882
jpayne@68 1883 - Windows: Building liblzma with UNICODE defined should now
jpayne@68 1884 work.
jpayne@68 1885
jpayne@68 1886 - CMake files are now actually included in the release tarball.
jpayne@68 1887 They should have been in 5.2.5 already.
jpayne@68 1888
jpayne@68 1889 - Minor CMake fixes and improvements.
jpayne@68 1890
jpayne@68 1891 * Added a new translation: Turkish
jpayne@68 1892
jpayne@68 1893
jpayne@68 1894 5.2.6 (2022-08-12)
jpayne@68 1895
jpayne@68 1896 * xz:
jpayne@68 1897
jpayne@68 1898 - The --keep option now accepts symlinks, hardlinks, and
jpayne@68 1899 setuid, setgid, and sticky files. Previously this required
jpayne@68 1900 using --force.
jpayne@68 1901
jpayne@68 1902 - When copying metadata from the source file to the destination
jpayne@68 1903 file, don't try to set the group (GID) if it is already set
jpayne@68 1904 correctly. This avoids a failure on OpenBSD (and possibly on
jpayne@68 1905 a few other OSes) where files may get created so that their
jpayne@68 1906 group doesn't belong to the user, and fchown(2) can fail even
jpayne@68 1907 if it needs to do nothing.
jpayne@68 1908
jpayne@68 1909 - Cap --memlimit-compress to 2000 MiB instead of 4020 MiB on
jpayne@68 1910 MIPS32 because on MIPS32 userspace processes are limited
jpayne@68 1911 to 2 GiB of address space.
jpayne@68 1912
jpayne@68 1913 * liblzma:
jpayne@68 1914
jpayne@68 1915 - Fixed a missing error-check in the threaded encoder. If a
jpayne@68 1916 small memory allocation fails, a .xz file with an invalid
jpayne@68 1917 Index field would be created. Decompressing such a file would
jpayne@68 1918 produce the correct output but result in an error at the end.
jpayne@68 1919 Thus this is a "mild" data corruption bug. Note that while
jpayne@68 1920 a failed memory allocation can trigger the bug, it cannot
jpayne@68 1921 cause invalid memory access.
jpayne@68 1922
jpayne@68 1923 - The decoder for .lzma files now supports files that have
jpayne@68 1924 uncompressed size stored in the header and still use the
jpayne@68 1925 end of payload marker (end of stream marker) at the end
jpayne@68 1926 of the LZMA stream. Such files are rare but, according to
jpayne@68 1927 the documentation in LZMA SDK, they are valid.
jpayne@68 1928 doc/lzma-file-format.txt was updated too.
jpayne@68 1929
jpayne@68 1930 - Improved 32-bit x86 assembly files:
jpayne@68 1931 * Support Intel Control-flow Enforcement Technology (CET)
jpayne@68 1932 * Use non-executable stack on FreeBSD.
jpayne@68 1933
jpayne@68 1934 - Visual Studio: Use non-standard _MSVC_LANG to detect C++
jpayne@68 1935 standard version in the lzma.h API header. It's used to
jpayne@68 1936 detect when "noexcept" can be used.
jpayne@68 1937
jpayne@68 1938 * xzgrep:
jpayne@68 1939
jpayne@68 1940 - Fixed arbitrary command injection via a malicious filename
jpayne@68 1941 (CVE-2022-1271, ZDI-CAN-16587). A standalone patch for
jpayne@68 1942 this was released to the public on 2022-04-07. A slight
jpayne@68 1943 robustness improvement has been made since then and, if
jpayne@68 1944 using GNU or *BSD grep, a new faster method is now used
jpayne@68 1945 that doesn't use the old sed-based construct at all. This
jpayne@68 1946 also fixes bad output with GNU grep >= 3.5 (2020-09-27)
jpayne@68 1947 when xzgrepping binary files.
jpayne@68 1948
jpayne@68 1949 This vulnerability was discovered by:
jpayne@68 1950 cleemy desu wayo working with Trend Micro Zero Day Initiative
jpayne@68 1951
jpayne@68 1952 - Fixed detection of corrupt .bz2 files.
jpayne@68 1953
jpayne@68 1954 - Improved error handling to fix exit status in some situations
jpayne@68 1955 and to fix handling of signals: in some situations a signal
jpayne@68 1956 didn't make xzgrep exit when it clearly should have. It's
jpayne@68 1957 possible that the signal handling still isn't quite perfect
jpayne@68 1958 but hopefully it's good enough.
jpayne@68 1959
jpayne@68 1960 - Documented exit statuses on the man page.
jpayne@68 1961
jpayne@68 1962 - xzegrep and xzfgrep now use "grep -E" and "grep -F" instead
jpayne@68 1963 of the deprecated egrep and fgrep commands.
jpayne@68 1964
jpayne@68 1965 - Fixed parsing of the options -E, -F, -G, -P, and -X. The
jpayne@68 1966 problem occurred when multiple options were specified in
jpayne@68 1967 a single argument, for example,
jpayne@68 1968
jpayne@68 1969 echo foo | xzgrep -Fe foo
jpayne@68 1970
jpayne@68 1971 treated foo as a filename because -Fe wasn't correctly
jpayne@68 1972 split into -F -e.
jpayne@68 1973
jpayne@68 1974 - Added zstd support.
jpayne@68 1975
jpayne@68 1976 * xzdiff/xzcmp:
jpayne@68 1977
jpayne@68 1978 - Fixed wrong exit status. Exit status could be 2 when the
jpayne@68 1979 correct value is 1.
jpayne@68 1980
jpayne@68 1981 - Documented on the man page that exit status of 2 is used
jpayne@68 1982 for decompression errors.
jpayne@68 1983
jpayne@68 1984 - Added zstd support.
jpayne@68 1985
jpayne@68 1986 * xzless:
jpayne@68 1987
jpayne@68 1988 - Fix less(1) version detection. It failed if the version number
jpayne@68 1989 from "less -V" contained a dot.
jpayne@68 1990
jpayne@68 1991 * Translations:
jpayne@68 1992
jpayne@68 1993 - Added new translations: Catalan, Croatian, Esperanto,
jpayne@68 1994 Korean, Portuguese, Romanian, Serbian, Spanish, Swedish,
jpayne@68 1995 and Ukrainian
jpayne@68 1996
jpayne@68 1997 - Updated the Brazilian Portuguese translation.
jpayne@68 1998
jpayne@68 1999 - Added French man page translation. This and the existing
jpayne@68 2000 German translation aren't complete anymore because the
jpayne@68 2001 English man pages got a few updates and the translators
jpayne@68 2002 weren't reached so that they could update their work.
jpayne@68 2003
jpayne@68 2004 * Build systems:
jpayne@68 2005
jpayne@68 2006 - Windows: Fix building of resource files when config.h isn't
jpayne@68 2007 used. CMake + Visual Studio can now build liblzma.dll.
jpayne@68 2008
jpayne@68 2009 - Various fixes to the CMake support. Building static or shared
jpayne@68 2010 liblzma should work fine in most cases. In contrast, building
jpayne@68 2011 the command line tools with CMake is still clearly incomplete
jpayne@68 2012 and experimental and should be used for testing only.
jpayne@68 2013
jpayne@68 2014
jpayne@68 2015 5.2.5 (2020-03-17)
jpayne@68 2016
jpayne@68 2017 * liblzma:
jpayne@68 2018
jpayne@68 2019 - Fixed several C99/C11 conformance bugs. Now the code is clean
jpayne@68 2020 under gcc/clang -fsanitize=undefined. Some of these changes
jpayne@68 2021 might have a negative effect on performance with old GCC
jpayne@68 2022 versions or compilers other than GCC and Clang. The configure
jpayne@68 2023 option --enable-unsafe-type-punning can be used to (mostly)
jpayne@68 2024 restore the old behavior but it shouldn't normally be used.
jpayne@68 2025
jpayne@68 2026 - Improved API documentation of lzma_properties_decode().
jpayne@68 2027
jpayne@68 2028 - Added a very minor encoder speed optimization.
jpayne@68 2029
jpayne@68 2030 * xz:
jpayne@68 2031
jpayne@68 2032 - Fixed a crash in "xz -dcfv not_an_xz_file". All four options
jpayne@68 2033 were required to trigger it. The crash occurred in the
jpayne@68 2034 progress indicator code when xz was in passthru mode where
jpayne@68 2035 xz works like "cat".
jpayne@68 2036
jpayne@68 2037 - Fixed an integer overflow with 32-bit off_t. It could happen
jpayne@68 2038 when decompressing a file that has a long run of zero bytes
jpayne@68 2039 which xz would try to write as a sparse file. Since the build
jpayne@68 2040 system enables large file support by default, off_t is
jpayne@68 2041 normally 64-bit even on 32-bit systems.
jpayne@68 2042
jpayne@68 2043 - Fixes for --flush-timeout:
jpayne@68 2044 * Fix semi-busy-waiting.
jpayne@68 2045 * Avoid unneeded flushes when no new input has arrived
jpayne@68 2046 since the previous flush was completed.
jpayne@68 2047
jpayne@68 2048 - Added a special case for 32-bit xz: If --memlimit-compress is
jpayne@68 2049 used to specify a limit that exceeds 4020 MiB, the limit will
jpayne@68 2050 be set to 4020 MiB. The values "0" and "max" aren't affected
jpayne@68 2051 by this and neither is decompression. This hack can be
jpayne@68 2052 helpful when a 32-bit xz has access to 4 GiB address space
jpayne@68 2053 but the specified memlimit exceeds 4 GiB. This can happen
jpayne@68 2054 e.g. with some scripts.
jpayne@68 2055
jpayne@68 2056 - Capsicum sandbox is now enabled by default where available
jpayne@68 2057 (FreeBSD >= 10). The sandbox debug messages (xz -vv) were
jpayne@68 2058 removed since they seemed to be more annoying than useful.
jpayne@68 2059
jpayne@68 2060 - DOS build now requires DJGPP 2.05 instead of 2.04beta.
jpayne@68 2061 A workaround for a locale problem with DJGPP 2.05 was added.
jpayne@68 2062
jpayne@68 2063 * xzgrep and other scripts:
jpayne@68 2064
jpayne@68 2065 - Added a configure option --enable-path-for-scripts=PREFIX.
jpayne@68 2066 It is disabled by default except on Solaris where the default
jpayne@68 2067 is /usr/xpg4/bin. See INSTALL for details.
jpayne@68 2068
jpayne@68 2069 - Added a workaround for a POSIX shell detection problem on
jpayne@68 2070 Solaris.
jpayne@68 2071
jpayne@68 2072 * Build systems:
jpayne@68 2073
jpayne@68 2074 - Added preliminary build instructions for z/OS. See INSTALL
jpayne@68 2075 section 1.2.9.
jpayne@68 2076
jpayne@68 2077 - Experimental CMake support was added. It should work to build
jpayne@68 2078 static liblzma on a few operating systems. It may or may not
jpayne@68 2079 work to build shared liblzma. On some platforms it can build
jpayne@68 2080 xz and xzdec too but those are only for testing. See the
jpayne@68 2081 comment in the beginning of CMakeLists.txt for details.
jpayne@68 2082
jpayne@68 2083 - Visual Studio project files were updated.
jpayne@68 2084 WindowsTargetPlatformVersion was removed from VS2017 files
jpayne@68 2085 and set to "10.0" in the added VS2019 files. In the future
jpayne@68 2086 the VS project files will be removed when CMake support is
jpayne@68 2087 good enough.
jpayne@68 2088
jpayne@68 2089 - New #defines in config.h: HAVE___BUILTIN_ASSUME_ALIGNED,
jpayne@68 2090 HAVE___BUILTIN_BSWAPXX, and TUKLIB_USE_UNSAFE_TYPE_PUNNING.
jpayne@68 2091
jpayne@68 2092 - autogen.sh has a new optional dependency on po4a and a new
jpayne@68 2093 option --no-po4a to skip that step. This matters only if one
jpayne@68 2094 wants to remake the build files. po4a is used to update the
jpayne@68 2095 translated man pages but as long as the man pages haven't
jpayne@68 2096 been modified, there's nothing to update and one can use
jpayne@68 2097 --no-po4a to avoid the dependency on po4a.
jpayne@68 2098
jpayne@68 2099 * Translations:
jpayne@68 2100
jpayne@68 2101 - XZ Utils translations are now handled by the Translation
jpayne@68 2102 Project: https://translationproject.org/domain/xz.html
jpayne@68 2103
jpayne@68 2104 - All man pages are now included in German too.
jpayne@68 2105
jpayne@68 2106 - New xz translations: Brazilian Portuguese, Finnish,
jpayne@68 2107 Hungarian, Chinese (simplified), Chinese (traditional),
jpayne@68 2108 and Danish (partial translation)
jpayne@68 2109
jpayne@68 2110 - Updated xz translations: French, German, Italian, and Polish
jpayne@68 2111
jpayne@68 2112 - Unfortunately a few new xz translations weren't included due
jpayne@68 2113 to technical problems like too long lines in --help output or
jpayne@68 2114 misaligned column headings in tables. In the future, many of
jpayne@68 2115 these strings will be split and e.g. the table column
jpayne@68 2116 alignment will be handled in software. This should make the
jpayne@68 2117 strings easier to translate.
jpayne@68 2118
jpayne@68 2119
jpayne@68 2120 5.2.4 (2018-04-29)
jpayne@68 2121
jpayne@68 2122 * liblzma:
jpayne@68 2123
jpayne@68 2124 - Allow 0 as memory usage limit instead of returning
jpayne@68 2125 LZMA_PROG_ERROR. Now 0 is treated as if 1 byte was specified,
jpayne@68 2126 which effectively is the same as 0.
jpayne@68 2127
jpayne@68 2128 - Use "noexcept" keyword instead of "throw()" in the public
jpayne@68 2129 headers when a C++11 (or newer standard) compiler is used.
jpayne@68 2130
jpayne@68 2131 - Added a portability fix for recent Intel C Compilers.
jpayne@68 2132
jpayne@68 2133 - Microsoft Visual Studio build files have been moved under
jpayne@68 2134 windows/vs2013 and windows/vs2017.
jpayne@68 2135
jpayne@68 2136 * xz:
jpayne@68 2137
jpayne@68 2138 - Fix "xz --list --robot missing_or_bad_file.xz" which would
jpayne@68 2139 try to print an uninitialized string and thus produce garbage
jpayne@68 2140 output. Since the exit status is non-zero, most uses of such
jpayne@68 2141 a command won't try to interpret the garbage output.
jpayne@68 2142
jpayne@68 2143 - "xz --list foo.xz" could print "Internal error (bug)" in a
jpayne@68 2144 corner case where a specific memory usage limit had been set.
jpayne@68 2145
jpayne@68 2146
jpayne@68 2147 5.2.3 (2016-12-30)
jpayne@68 2148
jpayne@68 2149 * xz:
jpayne@68 2150
jpayne@68 2151 - Always close a file before trying to delete it to avoid
jpayne@68 2152 problems on some operating system and file system combinations.
jpayne@68 2153
jpayne@68 2154 - Fixed copying of file timestamps on Windows.
jpayne@68 2155
jpayne@68 2156 - Added experimental (disabled by default) sandbox support using
jpayne@68 2157 Capsicum (FreeBSD >= 10). See --enable-sandbox in INSTALL.
jpayne@68 2158
jpayne@68 2159 * C99/C11 conformance fixes to liblzma. The issues affected at least
jpayne@68 2160 some builds using link-time optimizations.
jpayne@68 2161
jpayne@68 2162 * Fixed bugs in the rarely-used function lzma_index_dup().
jpayne@68 2163
jpayne@68 2164 * Use of external SHA-256 code is now disabled by default.
jpayne@68 2165 It can still be enabled by passing --enable-external-sha256
jpayne@68 2166 to configure. The reasons to disable it by default (see INSTALL
jpayne@68 2167 for more details):
jpayne@68 2168
jpayne@68 2169 - Some OS-specific SHA-256 implementations conflict with
jpayne@68 2170 OpenSSL and cause problems in programs that link against both
jpayne@68 2171 liblzma and libcrypto. At least FreeBSD 10 and MINIX 3.3.0
jpayne@68 2172 are affected.
jpayne@68 2173
jpayne@68 2174 - The internal SHA-256 is faster than the SHA-256 code in
jpayne@68 2175 some operating systems.
jpayne@68 2176
jpayne@68 2177 * Changed CPU core count detection to use sched_getaffinity() on
jpayne@68 2178 GNU/Linux and GNU/kFreeBSD.
jpayne@68 2179
jpayne@68 2180 * Fixes to the build-system and xz to make xz buildable even when
jpayne@68 2181 encoders, decoders, or threading have been disabled from libilzma
jpayne@68 2182 using configure options. These fixes added two new #defines to
jpayne@68 2183 config.h: HAVE_ENCODERS and HAVE_DECODERS.
jpayne@68 2184
jpayne@68 2185
jpayne@68 2186 5.2.2 (2015-09-29)
jpayne@68 2187
jpayne@68 2188 * Fixed bugs in QNX-specific code.
jpayne@68 2189
jpayne@68 2190 * Omitted the use of pipe2() even if it is available to avoid
jpayne@68 2191 portability issues with some old Linux and glibc combinations.
jpayne@68 2192
jpayne@68 2193 * Updated German translation.
jpayne@68 2194
jpayne@68 2195 * Added project files to build static and shared liblzma (not the
jpayne@68 2196 whole XZ Utils) with Visual Studio 2013 update 2 or later.
jpayne@68 2197
jpayne@68 2198 * Documented that threaded decompression hasn't been implemented
jpayne@68 2199 yet. A 5.2.0 NEWS entry describing multi-threading support had
jpayne@68 2200 incorrectly said "decompression" when it should have said
jpayne@68 2201 "compression".
jpayne@68 2202
jpayne@68 2203
jpayne@68 2204 5.2.1 (2015-02-26)
jpayne@68 2205
jpayne@68 2206 * Fixed a compression-ratio regression in fast mode of LZMA1 and
jpayne@68 2207 LZMA2. The bug is present in 5.1.4beta and 5.2.0 releases.
jpayne@68 2208
jpayne@68 2209 * Fixed a portability problem in xz that affected at least OpenBSD.
jpayne@68 2210
jpayne@68 2211 * Fixed xzdiff to be compatible with FreeBSD's mktemp which differs
jpayne@68 2212 from most other mktemp implementations.
jpayne@68 2213
jpayne@68 2214 * Changed CPU core count detection to use cpuset_getaffinity() on
jpayne@68 2215 FreeBSD.
jpayne@68 2216
jpayne@68 2217
jpayne@68 2218 5.2.0 (2014-12-21)
jpayne@68 2219
jpayne@68 2220 Since 5.1.4beta:
jpayne@68 2221
jpayne@68 2222 * All fixes from 5.0.8
jpayne@68 2223
jpayne@68 2224 * liblzma: Fixed lzma_stream_encoder_mt_memusage() when a preset
jpayne@68 2225 was used.
jpayne@68 2226
jpayne@68 2227 * xzdiff: If mktemp isn't installed, mkdir will be used as
jpayne@68 2228 a fallback to create a temporary directory. Installing mktemp
jpayne@68 2229 is still recommended.
jpayne@68 2230
jpayne@68 2231 * Updated French, German, Italian, Polish, and Vietnamese
jpayne@68 2232 translations.
jpayne@68 2233
jpayne@68 2234 Summary of fixes and new features added in the 5.1.x development
jpayne@68 2235 releases:
jpayne@68 2236
jpayne@68 2237 * liblzma:
jpayne@68 2238
jpayne@68 2239 - Added support for multi-threaded compression. See the
jpayne@68 2240 lzma_mt structure, lzma_stream_encoder_mt(), and
jpayne@68 2241 lzma_stream_encoder_mt_memusage() in <lzma/container.h>,
jpayne@68 2242 lzma_get_progress() in <lzma/base.h>, and lzma_cputhreads()
jpayne@68 2243 in <lzma/hardware.h> for details.
jpayne@68 2244
jpayne@68 2245 - Made the uses of lzma_allocator const correct.
jpayne@68 2246
jpayne@68 2247 - Added lzma_block_uncomp_encode() to create uncompressed
jpayne@68 2248 .xz Blocks using LZMA2 uncompressed chunks.
jpayne@68 2249
jpayne@68 2250 - Added support for LZMA_IGNORE_CHECK.
jpayne@68 2251
jpayne@68 2252 - A few speed optimizations were made.
jpayne@68 2253
jpayne@68 2254 - Added support for symbol versioning. It is enabled by default
jpayne@68 2255 on GNU/Linux, other GNU-based systems, and FreeBSD.
jpayne@68 2256
jpayne@68 2257 - liblzma (not the whole XZ Utils) should now be buildable
jpayne@68 2258 with MSVC 2013 update 2 or later using windows/config.h.
jpayne@68 2259
jpayne@68 2260 * xz:
jpayne@68 2261
jpayne@68 2262 - Fixed a race condition in the signal handling. It was
jpayne@68 2263 possible that e.g. the first SIGINT didn't make xz exit
jpayne@68 2264 if reading or writing blocked and one had bad luck. The fix
jpayne@68 2265 is non-trivial, so as of writing it is unknown if it will be
jpayne@68 2266 backported to the v5.0 branch.
jpayne@68 2267
jpayne@68 2268 - Multi-threaded compression can be enabled with the
jpayne@68 2269 --threads (-T) option.
jpayne@68 2270 [Fixed: This originally said "decompression".]
jpayne@68 2271
jpayne@68 2272 - New command line options in xz: --single-stream,
jpayne@68 2273 --block-size=SIZE, --block-list=SIZES,
jpayne@68 2274 --flush-timeout=TIMEOUT, and --ignore-check.
jpayne@68 2275
jpayne@68 2276 - xz -lvv now shows the minimum xz version that is required to
jpayne@68 2277 decompress the file. Currently it is 5.0.0 for all supported
jpayne@68 2278 .xz files except files with empty LZMA2 streams require 5.0.2.
jpayne@68 2279
jpayne@68 2280 * xzdiff and xzgrep now support .lzo files if lzop is installed.
jpayne@68 2281 The .tzo suffix is also recognized as a shorthand for .tar.lzo.
jpayne@68 2282
jpayne@68 2283
jpayne@68 2284 5.1.4beta (2014-09-14)
jpayne@68 2285
jpayne@68 2286 * All fixes from 5.0.6
jpayne@68 2287
jpayne@68 2288 * liblzma: Fixed the use of presets in threaded encoder
jpayne@68 2289 initialization.
jpayne@68 2290
jpayne@68 2291 * xz --block-list and --block-size can now be used together
jpayne@68 2292 in single-threaded mode. Previously the combination only
jpayne@68 2293 worked in multi-threaded mode.
jpayne@68 2294
jpayne@68 2295 * Added support for LZMA_IGNORE_CHECK to liblzma and made it
jpayne@68 2296 available in xz as --ignore-check.
jpayne@68 2297
jpayne@68 2298 * liblzma speed optimizations:
jpayne@68 2299
jpayne@68 2300 - Initialization of a new LZMA1 or LZMA2 encoder has been
jpayne@68 2301 optimized. (The speed of reinitializing an already-allocated
jpayne@68 2302 encoder isn't affected.) This helps when compressing many
jpayne@68 2303 small buffers with lzma_stream_buffer_encode() and other
jpayne@68 2304 similar situations where an already-allocated encoder state
jpayne@68 2305 isn't reused. This speed-up is visible in xz too if one
jpayne@68 2306 compresses many small files one at a time instead running xz
jpayne@68 2307 once and giving all files as command-line arguments.
jpayne@68 2308
jpayne@68 2309 - Buffer comparisons are now much faster when unaligned access
jpayne@68 2310 is allowed (configured with --enable-unaligned-access). This
jpayne@68 2311 speeds up encoding significantly. There is arch-specific code
jpayne@68 2312 for 32-bit and 64-bit x86 (32-bit needs SSE2 for the best
jpayne@68 2313 results and there's no run-time CPU detection for now).
jpayne@68 2314 For other archs there is only generic code which probably
jpayne@68 2315 isn't as optimal as arch-specific solutions could be.
jpayne@68 2316
jpayne@68 2317 - A few speed optimizations were made to the SHA-256 code.
jpayne@68 2318 (Note that the builtin SHA-256 code isn't used on all
jpayne@68 2319 operating systems.)
jpayne@68 2320
jpayne@68 2321 * liblzma can now be built with MSVC 2013 update 2 or later
jpayne@68 2322 using windows/config.h.
jpayne@68 2323
jpayne@68 2324 * Vietnamese translation was added.
jpayne@68 2325
jpayne@68 2326
jpayne@68 2327 5.1.3alpha (2013-10-26)
jpayne@68 2328
jpayne@68 2329 * All fixes from 5.0.5
jpayne@68 2330
jpayne@68 2331 * liblzma:
jpayne@68 2332
jpayne@68 2333 - Fixed a deadlock in the threaded encoder.
jpayne@68 2334
jpayne@68 2335 - Made the uses of lzma_allocator const correct.
jpayne@68 2336
jpayne@68 2337 - Added lzma_block_uncomp_encode() to create uncompressed
jpayne@68 2338 .xz Blocks using LZMA2 uncompressed chunks.
jpayne@68 2339
jpayne@68 2340 - Added support for native threads on Windows and the ability
jpayne@68 2341 to detect the number of CPU cores.
jpayne@68 2342
jpayne@68 2343 * xz:
jpayne@68 2344
jpayne@68 2345 - Fixed a race condition in the signal handling. It was
jpayne@68 2346 possible that e.g. the first SIGINT didn't make xz exit
jpayne@68 2347 if reading or writing blocked and one had bad luck. The fix
jpayne@68 2348 is non-trivial, so as of writing it is unknown if it will be
jpayne@68 2349 backported to the v5.0 branch.
jpayne@68 2350
jpayne@68 2351 - Made the progress indicator work correctly in threaded mode.
jpayne@68 2352
jpayne@68 2353 - Threaded encoder now works together with --block-list=SIZES.
jpayne@68 2354
jpayne@68 2355 - Added preliminary support for --flush-timeout=TIMEOUT.
jpayne@68 2356 It can be useful for (somewhat) real-time streaming. For
jpayne@68 2357 now the decompression side has to be done with something
jpayne@68 2358 else than the xz tool due to how xz does buffering, but this
jpayne@68 2359 should be fixed.
jpayne@68 2360
jpayne@68 2361
jpayne@68 2362 5.1.2alpha (2012-07-04)
jpayne@68 2363
jpayne@68 2364 * All fixes from 5.0.3 and 5.0.4
jpayne@68 2365
jpayne@68 2366 * liblzma:
jpayne@68 2367
jpayne@68 2368 - Fixed a deadlock and an invalid free() in the threaded encoder.
jpayne@68 2369
jpayne@68 2370 - Added support for symbol versioning. It is enabled by default
jpayne@68 2371 on GNU/Linux, other GNU-based systems, and FreeBSD.
jpayne@68 2372
jpayne@68 2373 - Use SHA-256 implementation from the operating system if one is
jpayne@68 2374 available in libc, libmd, or libutil. liblzma won't use e.g.
jpayne@68 2375 OpenSSL or libgcrypt to avoid introducing new dependencies.
jpayne@68 2376
jpayne@68 2377 - Fixed liblzma.pc for static linking.
jpayne@68 2378
jpayne@68 2379 - Fixed a few portability bugs.
jpayne@68 2380
jpayne@68 2381 * xz --decompress --single-stream now fixes the input position after
jpayne@68 2382 successful decompression. Now the following works:
jpayne@68 2383
jpayne@68 2384 echo foo | xz > foo.xz
jpayne@68 2385 echo bar | xz >> foo.xz
jpayne@68 2386 ( xz -dc --single-stream ; xz -dc --single-stream ) < foo.xz
jpayne@68 2387
jpayne@68 2388 Note that it doesn't work if the input is not seekable
jpayne@68 2389 or if there is Stream Padding between the concatenated
jpayne@68 2390 .xz Streams.
jpayne@68 2391
jpayne@68 2392 * xz -lvv now shows the minimum xz version that is required to
jpayne@68 2393 decompress the file. Currently it is 5.0.0 for all supported .xz
jpayne@68 2394 files except files with empty LZMA2 streams require 5.0.2.
jpayne@68 2395
jpayne@68 2396 * Added an *incomplete* implementation of --block-list=SIZES to xz.
jpayne@68 2397 It only works correctly in single-threaded mode and when
jpayne@68 2398 --block-size isn't used at the same time. --block-list allows
jpayne@68 2399 specifying the sizes of Blocks which can be useful e.g. when
jpayne@68 2400 creating files for random-access reading.
jpayne@68 2401
jpayne@68 2402
jpayne@68 2403 5.1.1alpha (2011-04-12)
jpayne@68 2404
jpayne@68 2405 * All fixes from 5.0.2
jpayne@68 2406
jpayne@68 2407 * liblzma fixes that will also be included in 5.0.3:
jpayne@68 2408
jpayne@68 2409 - A memory leak was fixed.
jpayne@68 2410
jpayne@68 2411 - lzma_stream_buffer_encode() no longer creates an empty .xz
jpayne@68 2412 Block if encoding an empty buffer. Such an empty Block with
jpayne@68 2413 LZMA2 data would trigger a bug in 5.0.1 and older (see the
jpayne@68 2414 first bullet point in 5.0.2 notes). When releasing 5.0.2,
jpayne@68 2415 I thought that no encoder creates this kind of files but
jpayne@68 2416 I was wrong.
jpayne@68 2417
jpayne@68 2418 - Validate function arguments better in a few functions. Most
jpayne@68 2419 importantly, specifying an unsupported integrity check to
jpayne@68 2420 lzma_stream_buffer_encode() no longer creates a corrupt .xz
jpayne@68 2421 file. Probably no application tries to do that, so this
jpayne@68 2422 shouldn't be a big problem in practice.
jpayne@68 2423
jpayne@68 2424 - Document that lzma_block_buffer_encode(),
jpayne@68 2425 lzma_easy_buffer_encode(), lzma_stream_encoder(), and
jpayne@68 2426 lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK.
jpayne@68 2427
jpayne@68 2428 - The return values of the _memusage() functions are now
jpayne@68 2429 documented better.
jpayne@68 2430
jpayne@68 2431 * Support for multithreaded compression was added using the simplest
jpayne@68 2432 method, which splits the input data into blocks and compresses
jpayne@68 2433 them independently. Other methods will be added in the future.
jpayne@68 2434 The current method has room for improvement, e.g. it is possible
jpayne@68 2435 to reduce the memory usage.
jpayne@68 2436
jpayne@68 2437 * Added the options --single-stream and --block-size=SIZE to xz.
jpayne@68 2438
jpayne@68 2439 * xzdiff and xzgrep now support .lzo files if lzop is installed.
jpayne@68 2440 The .tzo suffix is also recognized as a shorthand for .tar.lzo.
jpayne@68 2441
jpayne@68 2442 * Support for short 8.3 filenames under DOS was added to xz. It is
jpayne@68 2443 experimental and may change before it gets into a stable release.
jpayne@68 2444
jpayne@68 2445
jpayne@68 2446 5.0.8 (2014-12-21)
jpayne@68 2447
jpayne@68 2448 * Fixed an old bug in xzgrep that affected OpenBSD and probably
jpayne@68 2449 a few other operating systems too.
jpayne@68 2450
jpayne@68 2451 * Updated French and German translations.
jpayne@68 2452
jpayne@68 2453 * Added support for detecting the amount of RAM on AmigaOS/AROS.
jpayne@68 2454
jpayne@68 2455 * Minor build system updates.
jpayne@68 2456
jpayne@68 2457
jpayne@68 2458 5.0.7 (2014-09-20)
jpayne@68 2459
jpayne@68 2460 * Fix regressions introduced in 5.0.6:
jpayne@68 2461
jpayne@68 2462 - Fix building with non-GNU make.
jpayne@68 2463
jpayne@68 2464 - Fix invalid Libs.private value in liblzma.pc which broke
jpayne@68 2465 static linking against liblzma if the linker flags were
jpayne@68 2466 taken from pkg-config.
jpayne@68 2467
jpayne@68 2468
jpayne@68 2469 5.0.6 (2014-09-14)
jpayne@68 2470
jpayne@68 2471 * xzgrep now exits with status 0 if at least one file matched.
jpayne@68 2472
jpayne@68 2473 * A few minor portability and build system fixes
jpayne@68 2474
jpayne@68 2475
jpayne@68 2476 5.0.5 (2013-06-30)
jpayne@68 2477
jpayne@68 2478 * lzmadec and liblzma's lzma_alone_decoder(): Support decompressing
jpayne@68 2479 .lzma files that have less common settings in the headers
jpayne@68 2480 (dictionary size other than 2^n or 2^n + 2^(n-1), or uncompressed
jpayne@68 2481 size greater than 256 GiB). The limitations existed to avoid false
jpayne@68 2482 positives when detecting .lzma files. The lc + lp <= 4 limitation
jpayne@68 2483 still remains since liblzma's LZMA decoder has that limitation.
jpayne@68 2484
jpayne@68 2485 NOTE: xz's .lzma support or liblzma's lzma_auto_decoder() are NOT
jpayne@68 2486 affected by this change. They still consider uncommon .lzma headers
jpayne@68 2487 as not being in the .lzma format. Changing this would give way too
jpayne@68 2488 many false positives.
jpayne@68 2489
jpayne@68 2490 * xz:
jpayne@68 2491
jpayne@68 2492 - Interaction of preset and custom filter chain options was
jpayne@68 2493 made less illogical. This affects only certain less typical
jpayne@68 2494 uses cases so few people are expected to notice this change.
jpayne@68 2495
jpayne@68 2496 Now when a custom filter chain option (e.g. --lzma2) is
jpayne@68 2497 specified, all preset options (-0 ... -9, -e) earlier are on
jpayne@68 2498 the command line are completely forgotten. Similarly, when
jpayne@68 2499 a preset option is specified, all custom filter chain options
jpayne@68 2500 earlier on the command line are completely forgotten.
jpayne@68 2501
jpayne@68 2502 Example 1: "xz -9 --lzma2=preset=5 -e" is equivalent to "xz -e"
jpayne@68 2503 which is equivalent to "xz -6e". Earlier -e didn't put xz back
jpayne@68 2504 into preset mode and thus the example command was equivalent
jpayne@68 2505 to "xz --lzma2=preset=5".
jpayne@68 2506
jpayne@68 2507 Example 2: "xz -9e --lzma2=preset=5 -7" is equivalent to
jpayne@68 2508 "xz -7". Earlier a custom filter chain option didn't make
jpayne@68 2509 xz forget the -e option so the example was equivalent to
jpayne@68 2510 "xz -7e".
jpayne@68 2511
jpayne@68 2512 - Fixes and improvements to error handling.
jpayne@68 2513
jpayne@68 2514 - Various fixes to the man page.
jpayne@68 2515
jpayne@68 2516 * xzless: Fixed to work with "less" versions 448 and later.
jpayne@68 2517
jpayne@68 2518 * xzgrep: Made -h an alias for --no-filename.
jpayne@68 2519
jpayne@68 2520 * Include the previously missing debug/translation.bash which can
jpayne@68 2521 be useful for translators.
jpayne@68 2522
jpayne@68 2523 * Include a build script for Mac OS X. This has been in the Git
jpayne@68 2524 repository since 2010 but due to a mistake in Makefile.am the
jpayne@68 2525 script hasn't been included in a release tarball before.
jpayne@68 2526
jpayne@68 2527
jpayne@68 2528 5.0.4 (2012-06-22)
jpayne@68 2529
jpayne@68 2530 * liblzma:
jpayne@68 2531
jpayne@68 2532 - Fix lzma_index_init(). It could crash if memory allocation
jpayne@68 2533 failed.
jpayne@68 2534
jpayne@68 2535 - Fix the possibility of an incorrect LZMA_BUF_ERROR when a BCJ
jpayne@68 2536 filter is used and the application only provides exactly as
jpayne@68 2537 much output space as is the uncompressed size of the file.
jpayne@68 2538
jpayne@68 2539 - Fix a bug in doc/examples_old/xz_pipe_decompress.c. It didn't
jpayne@68 2540 check if the last call to lzma_code() really returned
jpayne@68 2541 LZMA_STREAM_END, which made the program think that truncated
jpayne@68 2542 files are valid.
jpayne@68 2543
jpayne@68 2544 - New example programs in doc/examples (old programs are now in
jpayne@68 2545 doc/examples_old). These have more comments and more detailed
jpayne@68 2546 error handling.
jpayne@68 2547
jpayne@68 2548 * Fix "xz -lvv foo.xz". It could crash on some corrupted files.
jpayne@68 2549
jpayne@68 2550 * Fix output of "xz --robot -lv" and "xz --robot -lvv" which
jpayne@68 2551 incorrectly printed the filename also in the "foo (x/x)" format.
jpayne@68 2552
jpayne@68 2553 * Fix exit status of "xzdiff foo.xz bar.xz".
jpayne@68 2554
jpayne@68 2555 * Fix exit status of "xzgrep foo binary_file".
jpayne@68 2556
jpayne@68 2557 * Fix portability to EBCDIC systems.
jpayne@68 2558
jpayne@68 2559 * Fix a configure issue on AIX with the XL C compiler. See INSTALL
jpayne@68 2560 for details.
jpayne@68 2561
jpayne@68 2562 * Update French, German, Italian, and Polish translations.
jpayne@68 2563
jpayne@68 2564
jpayne@68 2565 5.0.3 (2011-05-21)
jpayne@68 2566
jpayne@68 2567 * liblzma fixes:
jpayne@68 2568
jpayne@68 2569 - A memory leak was fixed.
jpayne@68 2570
jpayne@68 2571 - lzma_stream_buffer_encode() no longer creates an empty .xz
jpayne@68 2572 Block if encoding an empty buffer. Such an empty Block with
jpayne@68 2573 LZMA2 data would trigger a bug in 5.0.1 and older (see the
jpayne@68 2574 first bullet point in 5.0.2 notes). When releasing 5.0.2,
jpayne@68 2575 I thought that no encoder creates this kind of files but
jpayne@68 2576 I was wrong.
jpayne@68 2577
jpayne@68 2578 - Validate function arguments better in a few functions. Most
jpayne@68 2579 importantly, specifying an unsupported integrity check to
jpayne@68 2580 lzma_stream_buffer_encode() no longer creates a corrupt .xz
jpayne@68 2581 file. Probably no application tries to do that, so this
jpayne@68 2582 shouldn't be a big problem in practice.
jpayne@68 2583
jpayne@68 2584 - Document that lzma_block_buffer_encode(),
jpayne@68 2585 lzma_easy_buffer_encode(), lzma_stream_encoder(), and
jpayne@68 2586 lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK.
jpayne@68 2587
jpayne@68 2588 - The return values of the _memusage() functions are now
jpayne@68 2589 documented better.
jpayne@68 2590
jpayne@68 2591 * Fix command name detection in xzgrep. xzegrep and xzfgrep now
jpayne@68 2592 correctly use egrep and fgrep instead of grep.
jpayne@68 2593
jpayne@68 2594 * French translation was added.
jpayne@68 2595
jpayne@68 2596
jpayne@68 2597 5.0.2 (2011-04-01)
jpayne@68 2598
jpayne@68 2599 * LZMA2 decompressor now correctly accepts LZMA2 streams with no
jpayne@68 2600 uncompressed data. Previously it considered them corrupt. The
jpayne@68 2601 bug can affect applications that use raw LZMA2 streams. It is
jpayne@68 2602 very unlikely to affect .xz files because no compressor creates
jpayne@68 2603 .xz files with empty LZMA2 streams. (Empty .xz files are a
jpayne@68 2604 different thing than empty LZMA2 streams.)
jpayne@68 2605
jpayne@68 2606 * "xz --suffix=.foo filename.foo" now refuses to compress the
jpayne@68 2607 file due to it already having the suffix .foo. It was already
jpayne@68 2608 documented on the man page, but the code lacked the test.
jpayne@68 2609
jpayne@68 2610 * "xzgrep -l foo bar.xz" works now.
jpayne@68 2611
jpayne@68 2612 * Polish translation was added.
jpayne@68 2613
jpayne@68 2614
jpayne@68 2615 5.0.1 (2011-01-29)
jpayne@68 2616
jpayne@68 2617 * xz --force now (de)compresses files that have setuid, setgid,
jpayne@68 2618 or sticky bit set and files that have multiple hard links.
jpayne@68 2619 The man page had it documented this way already, but the code
jpayne@68 2620 had a bug.
jpayne@68 2621
jpayne@68 2622 * gzip and bzip2 support in xzdiff was fixed.
jpayne@68 2623
jpayne@68 2624 * Portability fixes
jpayne@68 2625
jpayne@68 2626 * Minor fix to Czech translation
jpayne@68 2627
jpayne@68 2628
jpayne@68 2629 5.0.0 (2010-10-23)
jpayne@68 2630
jpayne@68 2631 Only the most important changes compared to 4.999.9beta are listed
jpayne@68 2632 here. One change is especially important:
jpayne@68 2633
jpayne@68 2634 * The memory usage limit is now disabled by default. Some scripts
jpayne@68 2635 written before this change may have used --memory=max on xz command
jpayne@68 2636 line or in XZ_OPT. THESE USES OF --memory=max SHOULD BE REMOVED
jpayne@68 2637 NOW, because they interfere with user's ability to set the memory
jpayne@68 2638 usage limit himself. If user-specified limit causes problems to
jpayne@68 2639 your script, blame the user.
jpayne@68 2640
jpayne@68 2641 Other significant changes:
jpayne@68 2642
jpayne@68 2643 * Added support for XZ_DEFAULTS environment variable. This variable
jpayne@68 2644 allows users to set default options for xz, e.g. default memory
jpayne@68 2645 usage limit or default compression level. Scripts that use xz
jpayne@68 2646 must never set or unset XZ_DEFAULTS. Scripts should use XZ_OPT
jpayne@68 2647 instead if they need a way to pass options to xz via an
jpayne@68 2648 environment variable.
jpayne@68 2649
jpayne@68 2650 * The compression settings associated with the preset levels
jpayne@68 2651 -0 ... -9 have been changed. --extreme was changed a little too.
jpayne@68 2652 It is now less likely to make compression worse, but with some
jpayne@68 2653 files the new --extreme may compress slightly worse than the old
jpayne@68 2654 --extreme.
jpayne@68 2655
jpayne@68 2656 * If a preset level (-0 ... -9) is specified after a custom filter
jpayne@68 2657 chain options have been used (e.g. --lzma2), the custom filter
jpayne@68 2658 chain will be forgotten. Earlier the preset options were
jpayne@68 2659 completely ignored after custom filter chain options had been
jpayne@68 2660 seen.
jpayne@68 2661
jpayne@68 2662 * xz will create sparse files when decompressing if the uncompressed
jpayne@68 2663 data contains long sequences of binary zeros. This is done even
jpayne@68 2664 when writing to standard output that is connected to a regular
jpayne@68 2665 file and certain additional conditions are met to make it safe.
jpayne@68 2666
jpayne@68 2667 * Support for "xz --list" was added. Combine with --verbose or
jpayne@68 2668 --verbose --verbose (-vv) for detailed output.
jpayne@68 2669
jpayne@68 2670 * I had hoped that liblzma API would have been stable after
jpayne@68 2671 4.999.9beta, but there have been a couple of changes in the
jpayne@68 2672 advanced features, which don't affect most applications:
jpayne@68 2673
jpayne@68 2674 - Index handling code was revised. If you were using the old
jpayne@68 2675 API, you will get a compiler error (so it's easy to notice).
jpayne@68 2676
jpayne@68 2677 - A subtle but important change was made to the Block handling
jpayne@68 2678 API. lzma_block.version has to be initialized even for
jpayne@68 2679 lzma_block_header_decode(). Code that doesn't do it will work
jpayne@68 2680 for now, but might break in the future, which makes this API
jpayne@68 2681 change easy to miss.
jpayne@68 2682
jpayne@68 2683 * The major soname has been bumped to 5.0.0. liblzma API and ABI
jpayne@68 2684 are now stable, so the need to recompile programs linking against
jpayne@68 2685 liblzma shouldn't arise soon.
jpayne@68 2686