jpayne@68: jpayne@68: XZ Utils jpayne@68: ======== jpayne@68: jpayne@68: 0. Overview jpayne@68: 1. Documentation jpayne@68: 1.1. Overall documentation jpayne@68: 1.2. Documentation for command-line tools jpayne@68: 1.3. Documentation for liblzma jpayne@68: 2. Version numbering jpayne@68: 3. Reporting bugs jpayne@68: 4. Translations jpayne@68: 5. Other implementations of the .xz format jpayne@68: 6. Contact information jpayne@68: jpayne@68: jpayne@68: 0. Overview jpayne@68: ----------- jpayne@68: jpayne@68: XZ Utils provide a general-purpose data-compression library plus jpayne@68: command-line tools. The native file format is the .xz format, but jpayne@68: also the legacy .lzma format is supported. The .xz format supports jpayne@68: multiple compression algorithms, which are called "filters" in the jpayne@68: context of XZ Utils. The primary filter is currently LZMA2. With jpayne@68: typical files, XZ Utils create about 30 % smaller files than gzip. jpayne@68: jpayne@68: To ease adapting support for the .xz format into existing applications jpayne@68: and scripts, the API of liblzma is somewhat similar to the API of the jpayne@68: popular zlib library. For the same reason, the command-line tool xz jpayne@68: has a command-line syntax similar to that of gzip. jpayne@68: jpayne@68: When aiming for the highest compression ratio, the LZMA2 encoder uses jpayne@68: a lot of CPU time and may use, depending on the settings, even jpayne@68: hundreds of megabytes of RAM. However, in fast modes, the LZMA2 encoder jpayne@68: competes with bzip2 in compression speed, RAM usage, and compression jpayne@68: ratio. jpayne@68: jpayne@68: LZMA2 is reasonably fast to decompress. It is a little slower than jpayne@68: gzip, but a lot faster than bzip2. Being fast to decompress means jpayne@68: that the .xz format is especially nice when the same file will be jpayne@68: decompressed very many times (usually on different computers), which jpayne@68: is the case e.g. when distributing software packages. In such jpayne@68: situations, it's not too bad if the compression takes some time, jpayne@68: since that needs to be done only once to benefit many people. jpayne@68: jpayne@68: With some file types, combining (or "chaining") LZMA2 with an jpayne@68: additional filter can improve the compression ratio. A filter chain may jpayne@68: contain up to four filters, although usually only one or two are used. jpayne@68: For example, putting a BCJ (Branch/Call/Jump) filter before LZMA2 jpayne@68: in the filter chain can improve compression ratio of executable files. jpayne@68: jpayne@68: Since the .xz format allows adding new filter IDs, it is possible that jpayne@68: some day there will be a filter that is, for example, much faster to jpayne@68: compress than LZMA2 (but probably with worse compression ratio). jpayne@68: Similarly, it is possible that some day there is a filter that will jpayne@68: compress better than LZMA2. jpayne@68: jpayne@68: XZ Utils supports multithreaded compression. XZ Utils doesn't support jpayne@68: multithreaded decompression yet. It has been planned though and taken jpayne@68: into account when designing the .xz file format. In the future, files jpayne@68: that were created in threaded mode can be decompressed in threaded jpayne@68: mode too. jpayne@68: jpayne@68: jpayne@68: 1. Documentation jpayne@68: ---------------- jpayne@68: jpayne@68: 1.1. Overall documentation jpayne@68: jpayne@68: README This file jpayne@68: jpayne@68: INSTALL.generic Generic install instructions for those not jpayne@68: familiar with packages using GNU Autotools jpayne@68: INSTALL Installation instructions specific to XZ Utils jpayne@68: PACKAGERS Information to packagers of XZ Utils jpayne@68: jpayne@68: COPYING XZ Utils copyright and license information jpayne@68: COPYING.0BSD BSD Zero Clause License jpayne@68: COPYING.GPLv2 GNU General Public License version 2 jpayne@68: COPYING.GPLv3 GNU General Public License version 3 jpayne@68: COPYING.LGPLv2.1 GNU Lesser General Public License version 2.1 jpayne@68: jpayne@68: AUTHORS The main authors of XZ Utils jpayne@68: THANKS Incomplete list of people who have helped making jpayne@68: this software jpayne@68: NEWS User-visible changes between XZ Utils releases jpayne@68: ChangeLog Detailed list of changes (commit log) jpayne@68: TODO Known bugs and some sort of to-do list jpayne@68: jpayne@68: Note that only some of the above files are included in binary jpayne@68: packages. jpayne@68: jpayne@68: jpayne@68: 1.2. Documentation for command-line tools jpayne@68: jpayne@68: The command-line tools are documented as man pages. In source code jpayne@68: releases (and possibly also in some binary packages), the man pages jpayne@68: are also provided in plain text (ASCII only) format in the directory jpayne@68: "doc/man" to make the man pages more accessible to those whose jpayne@68: operating system doesn't provide an easy way to view man pages. jpayne@68: jpayne@68: jpayne@68: 1.3. Documentation for liblzma jpayne@68: jpayne@68: The liblzma API headers include short docs about each function jpayne@68: and data type as Doxygen tags. These docs should be quite OK as jpayne@68: a quick reference. jpayne@68: jpayne@68: There are a few example/tutorial programs that should help in jpayne@68: getting started with liblzma. In the source package the examples jpayne@68: are in "doc/examples" and in binary packages they may be under jpayne@68: "examples" in the same directory as this README. jpayne@68: jpayne@68: Since the liblzma API has similarities to the zlib API, some people jpayne@68: may find it useful to read the zlib docs and tutorial too: jpayne@68: jpayne@68: https://zlib.net/manual.html jpayne@68: https://zlib.net/zlib_how.html jpayne@68: jpayne@68: jpayne@68: 2. Version numbering jpayne@68: -------------------- jpayne@68: jpayne@68: The version number format of XZ Utils is X.Y.ZS: jpayne@68: jpayne@68: - X is the major version. When this is incremented, the library jpayne@68: API and ABI break. jpayne@68: jpayne@68: - Y is the minor version. It is incremented when new features jpayne@68: are added without breaking the existing API or ABI. An even Y jpayne@68: indicates a stable release and an odd Y indicates unstable jpayne@68: (alpha or beta version). jpayne@68: jpayne@68: - Z is the revision. This has a different meaning for stable and jpayne@68: unstable releases: jpayne@68: jpayne@68: * Stable: Z is incremented when bugs get fixed without adding jpayne@68: any new features. This is intended to be convenient for jpayne@68: downstream distributors that want bug fixes but don't want jpayne@68: any new features to minimize the risk of introducing new bugs. jpayne@68: jpayne@68: * Unstable: Z is just a counter. API or ABI of features added jpayne@68: in earlier unstable releases having the same X.Y may break. jpayne@68: jpayne@68: - S indicates stability of the release. It is missing from the jpayne@68: stable releases, where Y is an even number. When Y is odd, S jpayne@68: is either "alpha" or "beta" to make it very clear that such jpayne@68: versions are not stable releases. The same X.Y.Z combination is jpayne@68: not used for more than one stability level, i.e. after X.Y.Zalpha, jpayne@68: the next version can be X.Y.(Z+1)beta but not X.Y.Zbeta. jpayne@68: jpayne@68: jpayne@68: 3. Reporting bugs jpayne@68: ----------------- jpayne@68: jpayne@68: Naturally it is easiest for me if you already know what causes the jpayne@68: unexpected behavior. Even better if you have a patch to propose. jpayne@68: However, quite often the reason for unexpected behavior is unknown, jpayne@68: so here are a few things to do before sending a bug report: jpayne@68: jpayne@68: 1. Try to create a small example how to reproduce the issue. jpayne@68: jpayne@68: 2. Compile XZ Utils with debugging code using configure switches jpayne@68: --enable-debug and, if possible, --disable-shared. If you are jpayne@68: using GCC, use CFLAGS='-O0 -ggdb3'. Don't strip the resulting jpayne@68: binaries. jpayne@68: jpayne@68: 3. Turn on core dumps. The exact command depends on your shell; jpayne@68: for example in GNU bash it is done with "ulimit -c unlimited", jpayne@68: and in tcsh with "limit coredumpsize unlimited". jpayne@68: jpayne@68: 4. Try to reproduce the suspected bug. If you get "assertion failed" jpayne@68: message, be sure to include the complete message in your bug jpayne@68: report. If the application leaves a coredump, get a backtrace jpayne@68: using gdb: jpayne@68: $ gdb /path/to/app-binary # Load the app to the debugger. jpayne@68: (gdb) core core # Open the coredump. jpayne@68: (gdb) bt # Print the backtrace. Copy & paste to bug report. jpayne@68: (gdb) quit # Quit gdb. jpayne@68: jpayne@68: Report your bug via email or IRC (see Contact information below). jpayne@68: Don't send core dump files or any executables. If you have a small jpayne@68: example file(s) (total size less than 256 KiB), please include jpayne@68: it/them as an attachment. If you have bigger test files, put them jpayne@68: online somewhere and include a URL to the file(s) in the bug report. jpayne@68: jpayne@68: Always include the exact version number of XZ Utils in the bug report. jpayne@68: If you are using a snapshot from the git repository, use "git describe" jpayne@68: to get the exact snapshot version. If you are using XZ Utils shipped jpayne@68: in an operating system distribution, mention the distribution name, jpayne@68: distribution version, and exact xz package version; if you cannot jpayne@68: repeat the bug with the code compiled from unpatched source code, jpayne@68: you probably need to report a bug to your distribution's bug tracking jpayne@68: system. jpayne@68: jpayne@68: jpayne@68: 4. Translations jpayne@68: --------------- jpayne@68: jpayne@68: The xz command line tool and all man pages can be translated. jpayne@68: The translations are handled via the Translation Project. If you jpayne@68: wish to help translating xz, please join the Translation Project: jpayne@68: jpayne@68: https://translationproject.org/html/translators.html jpayne@68: jpayne@68: Below are notes and testing instructions specific to xz jpayne@68: translations. jpayne@68: jpayne@68: Testing can be done by installing xz into a temporary directory: jpayne@68: jpayne@68: ./configure --disable-shared --prefix=/tmp/xz-test jpayne@68: # jpayne@68: make -C po update-po jpayne@68: make install jpayne@68: bash debug/translation.bash | less jpayne@68: bash debug/translation.bash | less -S # For --list outputs jpayne@68: jpayne@68: Repeat the above as needed (no need to re-run configure though). jpayne@68: jpayne@68: Note especially the following: jpayne@68: jpayne@68: - The output of --help and --long-help must look nice on jpayne@68: an 80-column terminal. It's OK to add extra lines if needed. jpayne@68: jpayne@68: - In contrast, don't add extra lines to error messages and such. jpayne@68: They are often preceded with e.g. a filename on the same line, jpayne@68: so you have no way to predict where to put a \n. Let the terminal jpayne@68: do the wrapping even if it looks ugly. Adding new lines will be jpayne@68: even uglier in the generic case even if it looks nice in a few jpayne@68: limited examples. jpayne@68: jpayne@68: - Be careful with column alignment in tables and table-like output jpayne@68: (--list, --list --verbose --verbose, --info-memory, --help, and jpayne@68: --long-help): jpayne@68: jpayne@68: * All descriptions of options in --help should start in the jpayne@68: same column (but it doesn't need to be the same column as jpayne@68: in the English messages; just be consistent if you change it). jpayne@68: Check that both --help and --long-help look OK, since they jpayne@68: share several strings. jpayne@68: jpayne@68: * --list --verbose and --info-memory print lines that have jpayne@68: the format "Description: %s". If you need a longer jpayne@68: description, you can put extra space between the colon jpayne@68: and %s. Then you may need to add extra space to other jpayne@68: strings too so that the result as a whole looks good (all jpayne@68: values start at the same column). jpayne@68: jpayne@68: * The columns of the actual tables in --list --verbose --verbose jpayne@68: should be aligned properly. Abbreviate if necessary. It might jpayne@68: be good to keep at least 2 or 3 spaces between column headings jpayne@68: and avoid spaces in the headings so that the columns stand out jpayne@68: better, but this is a matter of opinion. Do what you think jpayne@68: looks best. jpayne@68: jpayne@68: - Be careful to put a period at the end of a sentence when the jpayne@68: original version has it, and don't put it when the original jpayne@68: doesn't have it. Similarly, be careful with \n characters jpayne@68: at the beginning and end of the strings. jpayne@68: jpayne@68: - Read the TRANSLATORS comments that have been extracted from the jpayne@68: source code and included in xz.pot. Some comments suggest jpayne@68: testing with a specific command which needs an .xz file. You jpayne@68: may use e.g. any tests/files/good-*.xz. However, these test jpayne@68: commands are included in translations.bash output, so reading jpayne@68: translations.bash output carefully can be enough. jpayne@68: jpayne@68: - If you find language problems in the original English strings, jpayne@68: feel free to suggest improvements. Ask if something is unclear. jpayne@68: jpayne@68: - The translated messages should be understandable (sometimes this jpayne@68: may be a problem with the original English messages too). Don't jpayne@68: make a direct word-by-word translation from English especially if jpayne@68: the result doesn't sound good in your language. jpayne@68: jpayne@68: Thanks for your help! jpayne@68: jpayne@68: jpayne@68: 5. Other implementations of the .xz format jpayne@68: ------------------------------------------ jpayne@68: jpayne@68: 7-Zip and the p7zip port of 7-Zip support the .xz format starting jpayne@68: from the version 9.00alpha. jpayne@68: jpayne@68: https://7-zip.org/ jpayne@68: https://p7zip.sourceforge.net/ jpayne@68: jpayne@68: XZ Embedded is a limited implementation written for use in the Linux jpayne@68: kernel, but it is also suitable for other embedded use. jpayne@68: jpayne@68: https://tukaani.org/xz/embedded.html jpayne@68: jpayne@68: XZ for Java is a complete implementation written in pure Java. jpayne@68: jpayne@68: https://tukaani.org/xz/java.html jpayne@68: jpayne@68: jpayne@68: 6. Contact information jpayne@68: ---------------------- jpayne@68: jpayne@68: XZ Utils in general: jpayne@68: - Home page: https://tukaani.org/xz/ jpayne@68: - Email to maintainer(s): xz@tukaani.org jpayne@68: - IRC: #tukaani on Libera Chat jpayne@68: - GitHub: https://github.com/tukaani-project/xz jpayne@68: jpayne@68: Lead maintainer: jpayne@68: - Email: Lasse Collin jpayne@68: - IRC: Larhzu on Libera Chat jpayne@68: