|
jpayne@68
|
1
|
|
jpayne@68
|
2 XZ Utils
|
|
jpayne@68
|
3 ========
|
|
jpayne@68
|
4
|
|
jpayne@68
|
5 0. Overview
|
|
jpayne@68
|
6 1. Documentation
|
|
jpayne@68
|
7 1.1. Overall documentation
|
|
jpayne@68
|
8 1.2. Documentation for command-line tools
|
|
jpayne@68
|
9 1.3. Documentation for liblzma
|
|
jpayne@68
|
10 2. Version numbering
|
|
jpayne@68
|
11 3. Reporting bugs
|
|
jpayne@68
|
12 4. Translations
|
|
jpayne@68
|
13 5. Other implementations of the .xz format
|
|
jpayne@68
|
14 6. Contact information
|
|
jpayne@68
|
15
|
|
jpayne@68
|
16
|
|
jpayne@68
|
17 0. Overview
|
|
jpayne@68
|
18 -----------
|
|
jpayne@68
|
19
|
|
jpayne@68
|
20 XZ Utils provide a general-purpose data-compression library plus
|
|
jpayne@68
|
21 command-line tools. The native file format is the .xz format, but
|
|
jpayne@68
|
22 also the legacy .lzma format is supported. The .xz format supports
|
|
jpayne@68
|
23 multiple compression algorithms, which are called "filters" in the
|
|
jpayne@68
|
24 context of XZ Utils. The primary filter is currently LZMA2. With
|
|
jpayne@68
|
25 typical files, XZ Utils create about 30 % smaller files than gzip.
|
|
jpayne@68
|
26
|
|
jpayne@68
|
27 To ease adapting support for the .xz format into existing applications
|
|
jpayne@68
|
28 and scripts, the API of liblzma is somewhat similar to the API of the
|
|
jpayne@68
|
29 popular zlib library. For the same reason, the command-line tool xz
|
|
jpayne@68
|
30 has a command-line syntax similar to that of gzip.
|
|
jpayne@68
|
31
|
|
jpayne@68
|
32 When aiming for the highest compression ratio, the LZMA2 encoder uses
|
|
jpayne@68
|
33 a lot of CPU time and may use, depending on the settings, even
|
|
jpayne@68
|
34 hundreds of megabytes of RAM. However, in fast modes, the LZMA2 encoder
|
|
jpayne@68
|
35 competes with bzip2 in compression speed, RAM usage, and compression
|
|
jpayne@68
|
36 ratio.
|
|
jpayne@68
|
37
|
|
jpayne@68
|
38 LZMA2 is reasonably fast to decompress. It is a little slower than
|
|
jpayne@68
|
39 gzip, but a lot faster than bzip2. Being fast to decompress means
|
|
jpayne@68
|
40 that the .xz format is especially nice when the same file will be
|
|
jpayne@68
|
41 decompressed very many times (usually on different computers), which
|
|
jpayne@68
|
42 is the case e.g. when distributing software packages. In such
|
|
jpayne@68
|
43 situations, it's not too bad if the compression takes some time,
|
|
jpayne@68
|
44 since that needs to be done only once to benefit many people.
|
|
jpayne@68
|
45
|
|
jpayne@68
|
46 With some file types, combining (or "chaining") LZMA2 with an
|
|
jpayne@68
|
47 additional filter can improve the compression ratio. A filter chain may
|
|
jpayne@68
|
48 contain up to four filters, although usually only one or two are used.
|
|
jpayne@68
|
49 For example, putting a BCJ (Branch/Call/Jump) filter before LZMA2
|
|
jpayne@68
|
50 in the filter chain can improve compression ratio of executable files.
|
|
jpayne@68
|
51
|
|
jpayne@68
|
52 Since the .xz format allows adding new filter IDs, it is possible that
|
|
jpayne@68
|
53 some day there will be a filter that is, for example, much faster to
|
|
jpayne@68
|
54 compress than LZMA2 (but probably with worse compression ratio).
|
|
jpayne@68
|
55 Similarly, it is possible that some day there is a filter that will
|
|
jpayne@68
|
56 compress better than LZMA2.
|
|
jpayne@68
|
57
|
|
jpayne@68
|
58 XZ Utils supports multithreaded compression. XZ Utils doesn't support
|
|
jpayne@68
|
59 multithreaded decompression yet. It has been planned though and taken
|
|
jpayne@68
|
60 into account when designing the .xz file format. In the future, files
|
|
jpayne@68
|
61 that were created in threaded mode can be decompressed in threaded
|
|
jpayne@68
|
62 mode too.
|
|
jpayne@68
|
63
|
|
jpayne@68
|
64
|
|
jpayne@68
|
65 1. Documentation
|
|
jpayne@68
|
66 ----------------
|
|
jpayne@68
|
67
|
|
jpayne@68
|
68 1.1. Overall documentation
|
|
jpayne@68
|
69
|
|
jpayne@68
|
70 README This file
|
|
jpayne@68
|
71
|
|
jpayne@68
|
72 INSTALL.generic Generic install instructions for those not
|
|
jpayne@68
|
73 familiar with packages using GNU Autotools
|
|
jpayne@68
|
74 INSTALL Installation instructions specific to XZ Utils
|
|
jpayne@68
|
75 PACKAGERS Information to packagers of XZ Utils
|
|
jpayne@68
|
76
|
|
jpayne@68
|
77 COPYING XZ Utils copyright and license information
|
|
jpayne@68
|
78 COPYING.0BSD BSD Zero Clause License
|
|
jpayne@68
|
79 COPYING.GPLv2 GNU General Public License version 2
|
|
jpayne@68
|
80 COPYING.GPLv3 GNU General Public License version 3
|
|
jpayne@68
|
81 COPYING.LGPLv2.1 GNU Lesser General Public License version 2.1
|
|
jpayne@68
|
82
|
|
jpayne@68
|
83 AUTHORS The main authors of XZ Utils
|
|
jpayne@68
|
84 THANKS Incomplete list of people who have helped making
|
|
jpayne@68
|
85 this software
|
|
jpayne@68
|
86 NEWS User-visible changes between XZ Utils releases
|
|
jpayne@68
|
87 ChangeLog Detailed list of changes (commit log)
|
|
jpayne@68
|
88 TODO Known bugs and some sort of to-do list
|
|
jpayne@68
|
89
|
|
jpayne@68
|
90 Note that only some of the above files are included in binary
|
|
jpayne@68
|
91 packages.
|
|
jpayne@68
|
92
|
|
jpayne@68
|
93
|
|
jpayne@68
|
94 1.2. Documentation for command-line tools
|
|
jpayne@68
|
95
|
|
jpayne@68
|
96 The command-line tools are documented as man pages. In source code
|
|
jpayne@68
|
97 releases (and possibly also in some binary packages), the man pages
|
|
jpayne@68
|
98 are also provided in plain text (ASCII only) format in the directory
|
|
jpayne@68
|
99 "doc/man" to make the man pages more accessible to those whose
|
|
jpayne@68
|
100 operating system doesn't provide an easy way to view man pages.
|
|
jpayne@68
|
101
|
|
jpayne@68
|
102
|
|
jpayne@68
|
103 1.3. Documentation for liblzma
|
|
jpayne@68
|
104
|
|
jpayne@68
|
105 The liblzma API headers include short docs about each function
|
|
jpayne@68
|
106 and data type as Doxygen tags. These docs should be quite OK as
|
|
jpayne@68
|
107 a quick reference.
|
|
jpayne@68
|
108
|
|
jpayne@68
|
109 There are a few example/tutorial programs that should help in
|
|
jpayne@68
|
110 getting started with liblzma. In the source package the examples
|
|
jpayne@68
|
111 are in "doc/examples" and in binary packages they may be under
|
|
jpayne@68
|
112 "examples" in the same directory as this README.
|
|
jpayne@68
|
113
|
|
jpayne@68
|
114 Since the liblzma API has similarities to the zlib API, some people
|
|
jpayne@68
|
115 may find it useful to read the zlib docs and tutorial too:
|
|
jpayne@68
|
116
|
|
jpayne@68
|
117 https://zlib.net/manual.html
|
|
jpayne@68
|
118 https://zlib.net/zlib_how.html
|
|
jpayne@68
|
119
|
|
jpayne@68
|
120
|
|
jpayne@68
|
121 2. Version numbering
|
|
jpayne@68
|
122 --------------------
|
|
jpayne@68
|
123
|
|
jpayne@68
|
124 The version number format of XZ Utils is X.Y.ZS:
|
|
jpayne@68
|
125
|
|
jpayne@68
|
126 - X is the major version. When this is incremented, the library
|
|
jpayne@68
|
127 API and ABI break.
|
|
jpayne@68
|
128
|
|
jpayne@68
|
129 - Y is the minor version. It is incremented when new features
|
|
jpayne@68
|
130 are added without breaking the existing API or ABI. An even Y
|
|
jpayne@68
|
131 indicates a stable release and an odd Y indicates unstable
|
|
jpayne@68
|
132 (alpha or beta version).
|
|
jpayne@68
|
133
|
|
jpayne@68
|
134 - Z is the revision. This has a different meaning for stable and
|
|
jpayne@68
|
135 unstable releases:
|
|
jpayne@68
|
136
|
|
jpayne@68
|
137 * Stable: Z is incremented when bugs get fixed without adding
|
|
jpayne@68
|
138 any new features. This is intended to be convenient for
|
|
jpayne@68
|
139 downstream distributors that want bug fixes but don't want
|
|
jpayne@68
|
140 any new features to minimize the risk of introducing new bugs.
|
|
jpayne@68
|
141
|
|
jpayne@68
|
142 * Unstable: Z is just a counter. API or ABI of features added
|
|
jpayne@68
|
143 in earlier unstable releases having the same X.Y may break.
|
|
jpayne@68
|
144
|
|
jpayne@68
|
145 - S indicates stability of the release. It is missing from the
|
|
jpayne@68
|
146 stable releases, where Y is an even number. When Y is odd, S
|
|
jpayne@68
|
147 is either "alpha" or "beta" to make it very clear that such
|
|
jpayne@68
|
148 versions are not stable releases. The same X.Y.Z combination is
|
|
jpayne@68
|
149 not used for more than one stability level, i.e. after X.Y.Zalpha,
|
|
jpayne@68
|
150 the next version can be X.Y.(Z+1)beta but not X.Y.Zbeta.
|
|
jpayne@68
|
151
|
|
jpayne@68
|
152
|
|
jpayne@68
|
153 3. Reporting bugs
|
|
jpayne@68
|
154 -----------------
|
|
jpayne@68
|
155
|
|
jpayne@68
|
156 Naturally it is easiest for me if you already know what causes the
|
|
jpayne@68
|
157 unexpected behavior. Even better if you have a patch to propose.
|
|
jpayne@68
|
158 However, quite often the reason for unexpected behavior is unknown,
|
|
jpayne@68
|
159 so here are a few things to do before sending a bug report:
|
|
jpayne@68
|
160
|
|
jpayne@68
|
161 1. Try to create a small example how to reproduce the issue.
|
|
jpayne@68
|
162
|
|
jpayne@68
|
163 2. Compile XZ Utils with debugging code using configure switches
|
|
jpayne@68
|
164 --enable-debug and, if possible, --disable-shared. If you are
|
|
jpayne@68
|
165 using GCC, use CFLAGS='-O0 -ggdb3'. Don't strip the resulting
|
|
jpayne@68
|
166 binaries.
|
|
jpayne@68
|
167
|
|
jpayne@68
|
168 3. Turn on core dumps. The exact command depends on your shell;
|
|
jpayne@68
|
169 for example in GNU bash it is done with "ulimit -c unlimited",
|
|
jpayne@68
|
170 and in tcsh with "limit coredumpsize unlimited".
|
|
jpayne@68
|
171
|
|
jpayne@68
|
172 4. Try to reproduce the suspected bug. If you get "assertion failed"
|
|
jpayne@68
|
173 message, be sure to include the complete message in your bug
|
|
jpayne@68
|
174 report. If the application leaves a coredump, get a backtrace
|
|
jpayne@68
|
175 using gdb:
|
|
jpayne@68
|
176 $ gdb /path/to/app-binary # Load the app to the debugger.
|
|
jpayne@68
|
177 (gdb) core core # Open the coredump.
|
|
jpayne@68
|
178 (gdb) bt # Print the backtrace. Copy & paste to bug report.
|
|
jpayne@68
|
179 (gdb) quit # Quit gdb.
|
|
jpayne@68
|
180
|
|
jpayne@68
|
181 Report your bug via email or IRC (see Contact information below).
|
|
jpayne@68
|
182 Don't send core dump files or any executables. If you have a small
|
|
jpayne@68
|
183 example file(s) (total size less than 256 KiB), please include
|
|
jpayne@68
|
184 it/them as an attachment. If you have bigger test files, put them
|
|
jpayne@68
|
185 online somewhere and include a URL to the file(s) in the bug report.
|
|
jpayne@68
|
186
|
|
jpayne@68
|
187 Always include the exact version number of XZ Utils in the bug report.
|
|
jpayne@68
|
188 If you are using a snapshot from the git repository, use "git describe"
|
|
jpayne@68
|
189 to get the exact snapshot version. If you are using XZ Utils shipped
|
|
jpayne@68
|
190 in an operating system distribution, mention the distribution name,
|
|
jpayne@68
|
191 distribution version, and exact xz package version; if you cannot
|
|
jpayne@68
|
192 repeat the bug with the code compiled from unpatched source code,
|
|
jpayne@68
|
193 you probably need to report a bug to your distribution's bug tracking
|
|
jpayne@68
|
194 system.
|
|
jpayne@68
|
195
|
|
jpayne@68
|
196
|
|
jpayne@68
|
197 4. Translations
|
|
jpayne@68
|
198 ---------------
|
|
jpayne@68
|
199
|
|
jpayne@68
|
200 The xz command line tool and all man pages can be translated.
|
|
jpayne@68
|
201 The translations are handled via the Translation Project. If you
|
|
jpayne@68
|
202 wish to help translating xz, please join the Translation Project:
|
|
jpayne@68
|
203
|
|
jpayne@68
|
204 https://translationproject.org/html/translators.html
|
|
jpayne@68
|
205
|
|
jpayne@68
|
206 Below are notes and testing instructions specific to xz
|
|
jpayne@68
|
207 translations.
|
|
jpayne@68
|
208
|
|
jpayne@68
|
209 Testing can be done by installing xz into a temporary directory:
|
|
jpayne@68
|
210
|
|
jpayne@68
|
211 ./configure --disable-shared --prefix=/tmp/xz-test
|
|
jpayne@68
|
212 # <Edit the .po file in the po directory.>
|
|
jpayne@68
|
213 make -C po update-po
|
|
jpayne@68
|
214 make install
|
|
jpayne@68
|
215 bash debug/translation.bash | less
|
|
jpayne@68
|
216 bash debug/translation.bash | less -S # For --list outputs
|
|
jpayne@68
|
217
|
|
jpayne@68
|
218 Repeat the above as needed (no need to re-run configure though).
|
|
jpayne@68
|
219
|
|
jpayne@68
|
220 Note especially the following:
|
|
jpayne@68
|
221
|
|
jpayne@68
|
222 - The output of --help and --long-help must look nice on
|
|
jpayne@68
|
223 an 80-column terminal. It's OK to add extra lines if needed.
|
|
jpayne@68
|
224
|
|
jpayne@68
|
225 - In contrast, don't add extra lines to error messages and such.
|
|
jpayne@68
|
226 They are often preceded with e.g. a filename on the same line,
|
|
jpayne@68
|
227 so you have no way to predict where to put a \n. Let the terminal
|
|
jpayne@68
|
228 do the wrapping even if it looks ugly. Adding new lines will be
|
|
jpayne@68
|
229 even uglier in the generic case even if it looks nice in a few
|
|
jpayne@68
|
230 limited examples.
|
|
jpayne@68
|
231
|
|
jpayne@68
|
232 - Be careful with column alignment in tables and table-like output
|
|
jpayne@68
|
233 (--list, --list --verbose --verbose, --info-memory, --help, and
|
|
jpayne@68
|
234 --long-help):
|
|
jpayne@68
|
235
|
|
jpayne@68
|
236 * All descriptions of options in --help should start in the
|
|
jpayne@68
|
237 same column (but it doesn't need to be the same column as
|
|
jpayne@68
|
238 in the English messages; just be consistent if you change it).
|
|
jpayne@68
|
239 Check that both --help and --long-help look OK, since they
|
|
jpayne@68
|
240 share several strings.
|
|
jpayne@68
|
241
|
|
jpayne@68
|
242 * --list --verbose and --info-memory print lines that have
|
|
jpayne@68
|
243 the format "Description: %s". If you need a longer
|
|
jpayne@68
|
244 description, you can put extra space between the colon
|
|
jpayne@68
|
245 and %s. Then you may need to add extra space to other
|
|
jpayne@68
|
246 strings too so that the result as a whole looks good (all
|
|
jpayne@68
|
247 values start at the same column).
|
|
jpayne@68
|
248
|
|
jpayne@68
|
249 * The columns of the actual tables in --list --verbose --verbose
|
|
jpayne@68
|
250 should be aligned properly. Abbreviate if necessary. It might
|
|
jpayne@68
|
251 be good to keep at least 2 or 3 spaces between column headings
|
|
jpayne@68
|
252 and avoid spaces in the headings so that the columns stand out
|
|
jpayne@68
|
253 better, but this is a matter of opinion. Do what you think
|
|
jpayne@68
|
254 looks best.
|
|
jpayne@68
|
255
|
|
jpayne@68
|
256 - Be careful to put a period at the end of a sentence when the
|
|
jpayne@68
|
257 original version has it, and don't put it when the original
|
|
jpayne@68
|
258 doesn't have it. Similarly, be careful with \n characters
|
|
jpayne@68
|
259 at the beginning and end of the strings.
|
|
jpayne@68
|
260
|
|
jpayne@68
|
261 - Read the TRANSLATORS comments that have been extracted from the
|
|
jpayne@68
|
262 source code and included in xz.pot. Some comments suggest
|
|
jpayne@68
|
263 testing with a specific command which needs an .xz file. You
|
|
jpayne@68
|
264 may use e.g. any tests/files/good-*.xz. However, these test
|
|
jpayne@68
|
265 commands are included in translations.bash output, so reading
|
|
jpayne@68
|
266 translations.bash output carefully can be enough.
|
|
jpayne@68
|
267
|
|
jpayne@68
|
268 - If you find language problems in the original English strings,
|
|
jpayne@68
|
269 feel free to suggest improvements. Ask if something is unclear.
|
|
jpayne@68
|
270
|
|
jpayne@68
|
271 - The translated messages should be understandable (sometimes this
|
|
jpayne@68
|
272 may be a problem with the original English messages too). Don't
|
|
jpayne@68
|
273 make a direct word-by-word translation from English especially if
|
|
jpayne@68
|
274 the result doesn't sound good in your language.
|
|
jpayne@68
|
275
|
|
jpayne@68
|
276 Thanks for your help!
|
|
jpayne@68
|
277
|
|
jpayne@68
|
278
|
|
jpayne@68
|
279 5. Other implementations of the .xz format
|
|
jpayne@68
|
280 ------------------------------------------
|
|
jpayne@68
|
281
|
|
jpayne@68
|
282 7-Zip and the p7zip port of 7-Zip support the .xz format starting
|
|
jpayne@68
|
283 from the version 9.00alpha.
|
|
jpayne@68
|
284
|
|
jpayne@68
|
285 https://7-zip.org/
|
|
jpayne@68
|
286 https://p7zip.sourceforge.net/
|
|
jpayne@68
|
287
|
|
jpayne@68
|
288 XZ Embedded is a limited implementation written for use in the Linux
|
|
jpayne@68
|
289 kernel, but it is also suitable for other embedded use.
|
|
jpayne@68
|
290
|
|
jpayne@68
|
291 https://tukaani.org/xz/embedded.html
|
|
jpayne@68
|
292
|
|
jpayne@68
|
293 XZ for Java is a complete implementation written in pure Java.
|
|
jpayne@68
|
294
|
|
jpayne@68
|
295 https://tukaani.org/xz/java.html
|
|
jpayne@68
|
296
|
|
jpayne@68
|
297
|
|
jpayne@68
|
298 6. Contact information
|
|
jpayne@68
|
299 ----------------------
|
|
jpayne@68
|
300
|
|
jpayne@68
|
301 XZ Utils in general:
|
|
jpayne@68
|
302 - Home page: https://tukaani.org/xz/
|
|
jpayne@68
|
303 - Email to maintainer(s): xz@tukaani.org
|
|
jpayne@68
|
304 - IRC: #tukaani on Libera Chat
|
|
jpayne@68
|
305 - GitHub: https://github.com/tukaani-project/xz
|
|
jpayne@68
|
306
|
|
jpayne@68
|
307 Lead maintainer:
|
|
jpayne@68
|
308 - Email: Lasse Collin <lasse.collin@tukaani.org>
|
|
jpayne@68
|
309 - IRC: Larhzu on Libera Chat
|
|
jpayne@68
|
310
|
