jpayne@68: '\" t
jpayne@68: .\" SPDX-License-Identifier: 0BSD
jpayne@68: .\"
jpayne@68: .\" Authors: Lasse Collin
jpayne@68: .\"          Jia Tan
jpayne@68: .\"
jpayne@68: .TH XZ 1 "2024-12-30" "Tukaani" "XZ Utils"
jpayne@68: .
jpayne@68: .SH NAME
jpayne@68: xz, unxz, xzcat, lzma, unlzma, lzcat \- Compress or decompress .xz and .lzma files
jpayne@68: .
jpayne@68: .SH SYNOPSIS
jpayne@68: .B xz
jpayne@68: .RI [ option... ]
jpayne@68: .RI [ file... ]
jpayne@68: .
jpayne@68: .SH COMMAND ALIASES
jpayne@68: .B unxz
jpayne@68: is equivalent to
jpayne@68: .BR "xz \-\-decompress" .
jpayne@68: .br
jpayne@68: .B xzcat
jpayne@68: is equivalent to
jpayne@68: .BR "xz \-\-decompress \-\-stdout" .
jpayne@68: .br
jpayne@68: .B lzma
jpayne@68: is equivalent to
jpayne@68: .BR "xz \-\-format=lzma" .
jpayne@68: .br
jpayne@68: .B unlzma
jpayne@68: is equivalent to
jpayne@68: .BR "xz \-\-format=lzma \-\-decompress" .
jpayne@68: .br
jpayne@68: .B lzcat
jpayne@68: is equivalent to
jpayne@68: .BR "xz \-\-format=lzma \-\-decompress \-\-stdout" .
jpayne@68: .PP
jpayne@68: When writing scripts that need to decompress files,
jpayne@68: it is recommended to always use the name
jpayne@68: .B xz
jpayne@68: with appropriate arguments
jpayne@68: .RB ( "xz \-d"
jpayne@68: or
jpayne@68: .BR "xz \-dc" )
jpayne@68: instead of the names
jpayne@68: .B unxz
jpayne@68: and
jpayne@68: .BR xzcat .
jpayne@68: .
jpayne@68: .SH DESCRIPTION
jpayne@68: .B xz
jpayne@68: is a general-purpose data compression tool with
jpayne@68: command line syntax similar to
jpayne@68: .BR gzip (1)
jpayne@68: and
jpayne@68: .BR bzip2 (1).
jpayne@68: The native file format is the
jpayne@68: .B .xz
jpayne@68: format, but the legacy
jpayne@68: .B .lzma
jpayne@68: format used by LZMA Utils and
jpayne@68: raw compressed streams with no container format headers
jpayne@68: are also supported.
jpayne@68: In addition, decompression of the
jpayne@68: .B .lz
jpayne@68: format used by
jpayne@68: .B lzip
jpayne@68: is supported.
jpayne@68: .PP
jpayne@68: .B xz
jpayne@68: compresses or decompresses each
jpayne@68: .I file
jpayne@68: according to the selected operation mode.
jpayne@68: If no
jpayne@68: .I files
jpayne@68: are given or
jpayne@68: .I file
jpayne@68: is
jpayne@68: .BR \- ,
jpayne@68: .B xz
jpayne@68: reads from standard input and writes the processed data
jpayne@68: to standard output.
jpayne@68: .B xz
jpayne@68: will refuse (display an error and skip the
jpayne@68: .IR file )
jpayne@68: to write compressed data to standard output if it is a terminal.
jpayne@68: Similarly,
jpayne@68: .B xz
jpayne@68: will refuse to read compressed data
jpayne@68: from standard input if it is a terminal.
jpayne@68: .PP
jpayne@68: Unless
jpayne@68: .B \-\-stdout
jpayne@68: is specified,
jpayne@68: .I files
jpayne@68: other than
jpayne@68: .B \-
jpayne@68: are written to a new file whose name is derived from the source
jpayne@68: .I file
jpayne@68: name:
jpayne@68: .IP \(bu 3
jpayne@68: When compressing, the suffix of the target file format
jpayne@68: .RB ( .xz
jpayne@68: or
jpayne@68: .BR .lzma )
jpayne@68: is appended to the source filename to get the target filename.
jpayne@68: .IP \(bu 3
jpayne@68: When decompressing, the
jpayne@68: .BR .xz ,
jpayne@68: .BR .lzma ,
jpayne@68: or
jpayne@68: .B .lz
jpayne@68: suffix is removed from the filename to get the target filename.
jpayne@68: .B xz
jpayne@68: also recognizes the suffixes
jpayne@68: .B .txz
jpayne@68: and
jpayne@68: .BR .tlz ,
jpayne@68: and replaces them with the
jpayne@68: .B .tar
jpayne@68: suffix.
jpayne@68: .PP
jpayne@68: If the target file already exists, an error is displayed and the
jpayne@68: .I file
jpayne@68: is skipped.
jpayne@68: .PP
jpayne@68: Unless writing to standard output,
jpayne@68: .B xz
jpayne@68: will display a warning and skip the
jpayne@68: .I file
jpayne@68: if any of the following applies:
jpayne@68: .IP \(bu 3
jpayne@68: .I File
jpayne@68: is not a regular file.
jpayne@68: Symbolic links are not followed,
jpayne@68: and thus they are not considered to be regular files.
jpayne@68: .IP \(bu 3
jpayne@68: .I File
jpayne@68: has more than one hard link.
jpayne@68: .IP \(bu 3
jpayne@68: .I File
jpayne@68: has setuid, setgid, or sticky bit set.
jpayne@68: .IP \(bu 3
jpayne@68: The operation mode is set to compress and the
jpayne@68: .I file
jpayne@68: already has a suffix of the target file format
jpayne@68: .RB ( .xz
jpayne@68: or
jpayne@68: .B .txz
jpayne@68: when compressing to the
jpayne@68: .B .xz
jpayne@68: format, and
jpayne@68: .B .lzma
jpayne@68: or
jpayne@68: .B .tlz
jpayne@68: when compressing to the
jpayne@68: .B .lzma
jpayne@68: format).
jpayne@68: .IP \(bu 3
jpayne@68: The operation mode is set to decompress and the
jpayne@68: .I file
jpayne@68: doesn't have a suffix of any of the supported file formats
jpayne@68: .RB ( .xz ,
jpayne@68: .BR .txz ,
jpayne@68: .BR .lzma ,
jpayne@68: .BR .tlz ,
jpayne@68: or
jpayne@68: .BR .lz ).
jpayne@68: .PP
jpayne@68: After successfully compressing or decompressing the
jpayne@68: .IR file ,
jpayne@68: .B xz
jpayne@68: copies the owner, group, permissions, access time,
jpayne@68: and modification time from the source
jpayne@68: .I file
jpayne@68: to the target file.
jpayne@68: If copying the group fails, the permissions are modified
jpayne@68: so that the target file doesn't become accessible to users
jpayne@68: who didn't have permission to access the source
jpayne@68: .IR file .
jpayne@68: .B xz
jpayne@68: doesn't support copying other metadata like access control lists
jpayne@68: or extended attributes yet.
jpayne@68: .PP
jpayne@68: Once the target file has been successfully closed, the source
jpayne@68: .I file
jpayne@68: is removed unless
jpayne@68: .B \-\-keep
jpayne@68: was specified.
jpayne@68: The source
jpayne@68: .I file
jpayne@68: is never removed if the output is written to standard output
jpayne@68: or if an error occurs.
jpayne@68: .PP
jpayne@68: Sending
jpayne@68: .B SIGINFO
jpayne@68: or
jpayne@68: .B SIGUSR1
jpayne@68: to the
jpayne@68: .B xz
jpayne@68: process makes it print progress information to standard error.
jpayne@68: This has only limited use since when standard error
jpayne@68: is a terminal, using
jpayne@68: .B \-\-verbose
jpayne@68: will display an automatically updating progress indicator.
jpayne@68: .
jpayne@68: .SS "Memory usage"
jpayne@68: The memory usage of
jpayne@68: .B xz
jpayne@68: varies from a few hundred kilobytes to several gigabytes
jpayne@68: depending on the compression settings.
jpayne@68: The settings used when compressing a file determine
jpayne@68: the memory requirements of the decompressor.
jpayne@68: Typically the decompressor needs 5\ % to 20\ % of
jpayne@68: the amount of memory that the compressor needed when
jpayne@68: creating the file.
jpayne@68: For example, decompressing a file created with
jpayne@68: .B xz \-9
jpayne@68: currently requires 65\ MiB of memory.
jpayne@68: Still, it is possible to have
jpayne@68: .B .xz
jpayne@68: files that require several gigabytes of memory to decompress.
jpayne@68: .PP
jpayne@68: Especially users of older systems may find
jpayne@68: the possibility of very large memory usage annoying.
jpayne@68: To prevent uncomfortable surprises,
jpayne@68: .B xz
jpayne@68: has a built-in memory usage limiter, which is disabled by default.
jpayne@68: While some operating systems provide ways to limit
jpayne@68: the memory usage of processes, relying on it
jpayne@68: wasn't deemed to be flexible enough (for example, using
jpayne@68: .BR ulimit (1)
jpayne@68: to limit virtual memory tends to cripple
jpayne@68: .BR mmap (2)).
jpayne@68: .PP
jpayne@68: The memory usage limiter can be enabled with
jpayne@68: the command line option \fB\-\-memlimit=\fIlimit\fR.
jpayne@68: Often it is more convenient to enable the limiter
jpayne@68: by default by setting the environment variable
jpayne@68: .BR XZ_DEFAULTS ,
jpayne@68: for example,
jpayne@68: .BR XZ_DEFAULTS=\-\-memlimit=150MiB .
jpayne@68: It is possible to set the limits separately
jpayne@68: for compression and decompression by using
jpayne@68: .BI \-\-memlimit\-compress= limit
jpayne@68: and \fB\-\-memlimit\-decompress=\fIlimit\fR.
jpayne@68: Using these two options outside
jpayne@68: .B XZ_DEFAULTS
jpayne@68: is rarely useful because a single run of
jpayne@68: .B xz
jpayne@68: cannot do both compression and decompression and
jpayne@68: .BI \-\-memlimit= limit
jpayne@68: (or
jpayne@68: .B \-M
jpayne@68: .IR limit )
jpayne@68: is shorter to type on the command line.
jpayne@68: .PP
jpayne@68: If the specified memory usage limit is exceeded when decompressing,
jpayne@68: .B xz
jpayne@68: will display an error and decompressing the file will fail.
jpayne@68: If the limit is exceeded when compressing,
jpayne@68: .B xz
jpayne@68: will try to scale the settings down so that the limit
jpayne@68: is no longer exceeded (except when using
jpayne@68: .B \-\-format=raw
jpayne@68: or
jpayne@68: .BR \-\-no\-adjust ).
jpayne@68: This way the operation won't fail unless the limit is very small.
jpayne@68: The scaling of the settings is done in steps that don't
jpayne@68: match the compression level presets, for example, if the limit is
jpayne@68: only slightly less than the amount required for
jpayne@68: .BR "xz \-9" ,
jpayne@68: the settings will be scaled down only a little,
jpayne@68: not all the way down to
jpayne@68: .BR "xz \-8" .
jpayne@68: .
jpayne@68: .SS "Concatenation and padding with .xz files"
jpayne@68: It is possible to concatenate
jpayne@68: .B .xz
jpayne@68: files as is.
jpayne@68: .B xz
jpayne@68: will decompress such files as if they were a single
jpayne@68: .B .xz
jpayne@68: file.
jpayne@68: .PP
jpayne@68: It is possible to insert padding between the concatenated parts
jpayne@68: or after the last part.
jpayne@68: The padding must consist of null bytes and the size
jpayne@68: of the padding must be a multiple of four bytes.
jpayne@68: This can be useful, for example, if the
jpayne@68: .B .xz
jpayne@68: file is stored on a medium that measures file sizes
jpayne@68: in 512-byte blocks.
jpayne@68: .PP
jpayne@68: Concatenation and padding are not allowed with
jpayne@68: .B .lzma
jpayne@68: files or raw streams.
jpayne@68: .
jpayne@68: .SH OPTIONS
jpayne@68: .
jpayne@68: .SS "Integer suffixes and special values"
jpayne@68: In most places where an integer argument is expected,
jpayne@68: an optional suffix is supported to easily indicate large integers.
jpayne@68: There must be no space between the integer and the suffix.
jpayne@68: .TP
jpayne@68: .B KiB
jpayne@68: Multiply the integer by 1,024 (2^10).
jpayne@68: .BR Ki ,
jpayne@68: .BR k ,
jpayne@68: .BR kB ,
jpayne@68: .BR K ,
jpayne@68: and
jpayne@68: .B KB
jpayne@68: are accepted as synonyms for
jpayne@68: .BR KiB .
jpayne@68: .TP
jpayne@68: .B MiB
jpayne@68: Multiply the integer by 1,048,576 (2^20).
jpayne@68: .BR Mi ,
jpayne@68: .BR m ,
jpayne@68: .BR M ,
jpayne@68: and
jpayne@68: .B MB
jpayne@68: are accepted as synonyms for
jpayne@68: .BR MiB .
jpayne@68: .TP
jpayne@68: .B GiB
jpayne@68: Multiply the integer by 1,073,741,824 (2^30).
jpayne@68: .BR Gi ,
jpayne@68: .BR g ,
jpayne@68: .BR G ,
jpayne@68: and
jpayne@68: .B GB
jpayne@68: are accepted as synonyms for
jpayne@68: .BR GiB .
jpayne@68: .PP
jpayne@68: The special value
jpayne@68: .B max
jpayne@68: can be used to indicate the maximum integer value
jpayne@68: supported by the option.
jpayne@68: .
jpayne@68: .SS "Operation mode"
jpayne@68: If multiple operation mode options are given,
jpayne@68: the last one takes effect.
jpayne@68: .TP
jpayne@68: .BR \-z ", " \-\-compress
jpayne@68: Compress.
jpayne@68: This is the default operation mode when no operation mode option
jpayne@68: is specified and no other operation mode is implied from
jpayne@68: the command name (for example,
jpayne@68: .B unxz
jpayne@68: implies
jpayne@68: .BR \-\-decompress ).
jpayne@68: .IP ""
jpayne@68: .\" The DESCRIPTION section already says this but it's good to repeat it
jpayne@68: .\" here because the default behavior is a bit dangerous and new users
jpayne@68: .\" in a hurry may skip reading the DESCRIPTION section.
jpayne@68: After successful compression, the source file is removed
jpayne@68: unless writing to standard output or
jpayne@68: .B \-\-keep
jpayne@68: was specified.
jpayne@68: .TP
jpayne@68: .BR \-d ", " \-\-decompress ", " \-\-uncompress
jpayne@68: Decompress.
jpayne@68: .\" The DESCRIPTION section already says this but it's good to repeat it
jpayne@68: .\" here because the default behavior is a bit dangerous and new users
jpayne@68: .\" in a hurry may skip reading the DESCRIPTION section.
jpayne@68: After successful decompression, the source file is removed
jpayne@68: unless writing to standard output or
jpayne@68: .B \-\-keep
jpayne@68: was specified.
jpayne@68: .TP
jpayne@68: .BR \-t ", " \-\-test
jpayne@68: Test the integrity of compressed
jpayne@68: .IR files .
jpayne@68: This option is equivalent to
jpayne@68: .B "\-\-decompress \-\-stdout"
jpayne@68: except that the decompressed data is discarded instead of being
jpayne@68: written to standard output.
jpayne@68: No files are created or removed.
jpayne@68: .TP
jpayne@68: .BR \-l ", " \-\-list
jpayne@68: Print information about compressed
jpayne@68: .IR files .
jpayne@68: No uncompressed output is produced,
jpayne@68: and no files are created or removed.
jpayne@68: In list mode, the program cannot read
jpayne@68: the compressed data from standard
jpayne@68: input or from other unseekable sources.
jpayne@68: .IP ""
jpayne@68: The default listing shows basic information about
jpayne@68: .IR files ,
jpayne@68: one file per line.
jpayne@68: To get more detailed information, use also the
jpayne@68: .B \-\-verbose
jpayne@68: option.
jpayne@68: For even more information, use
jpayne@68: .B \-\-verbose
jpayne@68: twice, but note that this may be slow, because getting all the extra
jpayne@68: information requires many seeks.
jpayne@68: The width of verbose output exceeds
jpayne@68: 80 characters, so piping the output to, for example,
jpayne@68: .B "less\ \-S"
jpayne@68: may be convenient if the terminal isn't wide enough.
jpayne@68: .IP ""
jpayne@68: The exact output may vary between
jpayne@68: .B xz
jpayne@68: versions and different locales.
jpayne@68: For machine-readable output,
jpayne@68: .B \-\-robot \-\-list
jpayne@68: should be used.
jpayne@68: .
jpayne@68: .SS "Operation modifiers"
jpayne@68: .TP
jpayne@68: .BR \-k ", " \-\-keep
jpayne@68: Don't delete the input files.
jpayne@68: .IP ""
jpayne@68: Since
jpayne@68: .B xz
jpayne@68: 5.2.6,
jpayne@68: this option also makes
jpayne@68: .B xz
jpayne@68: compress or decompress even if the input is
jpayne@68: a symbolic link to a regular file,
jpayne@68: has more than one hard link,
jpayne@68: or has the setuid, setgid, or sticky bit set.
jpayne@68: The setuid, setgid, and sticky bits are not copied
jpayne@68: to the target file.
jpayne@68: In earlier versions this was only done with
jpayne@68: .BR \-\-force .
jpayne@68: .TP
jpayne@68: .BR \-f ", " \-\-force
jpayne@68: This option has several effects:
jpayne@68: .RS
jpayne@68: .IP \(bu 3
jpayne@68: If the target file already exists,
jpayne@68: delete it before compressing or decompressing.
jpayne@68: .IP \(bu 3
jpayne@68: Compress or decompress even if the input is
jpayne@68: a symbolic link to a regular file,
jpayne@68: has more than one hard link,
jpayne@68: or has the setuid, setgid, or sticky bit set.
jpayne@68: The setuid, setgid, and sticky bits are not copied
jpayne@68: to the target file.
jpayne@68: .IP \(bu 3
jpayne@68: When used with
jpayne@68: .B \-\-decompress
jpayne@68: .B \-\-stdout
jpayne@68: and
jpayne@68: .B xz
jpayne@68: cannot recognize the type of the source file,
jpayne@68: copy the source file as is to standard output.
jpayne@68: This allows
jpayne@68: .B xzcat
jpayne@68: .B \-\-force
jpayne@68: to be used like
jpayne@68: .BR cat (1)
jpayne@68: for files that have not been compressed with
jpayne@68: .BR xz .
jpayne@68: Note that in future,
jpayne@68: .B xz
jpayne@68: might support new compressed file formats, which may make
jpayne@68: .B xz
jpayne@68: decompress more types of files instead of copying them as is to
jpayne@68: standard output.
jpayne@68: .BI \-\-format= format
jpayne@68: can be used to restrict
jpayne@68: .B xz
jpayne@68: to decompress only a single file format.
jpayne@68: .RE
jpayne@68: .TP
jpayne@68: .BR \-c ", " \-\-stdout ", " \-\-to\-stdout
jpayne@68: Write the compressed or decompressed data to
jpayne@68: standard output instead of a file.
jpayne@68: This implies
jpayne@68: .BR \-\-keep .
jpayne@68: .TP
jpayne@68: .B \-\-single\-stream
jpayne@68: Decompress only the first
jpayne@68: .B .xz
jpayne@68: stream, and
jpayne@68: silently ignore possible remaining input data following the stream.
jpayne@68: Normally such trailing garbage makes
jpayne@68: .B xz
jpayne@68: display an error.
jpayne@68: .IP ""
jpayne@68: .B xz
jpayne@68: never decompresses more than one stream from
jpayne@68: .B .lzma
jpayne@68: files or raw streams, but this option still makes
jpayne@68: .B xz
jpayne@68: ignore the possible trailing data after the
jpayne@68: .B .lzma
jpayne@68: file or raw stream.
jpayne@68: .IP ""
jpayne@68: This option has no effect if the operation mode is not
jpayne@68: .B \-\-decompress
jpayne@68: or
jpayne@68: .BR \-\-test .
jpayne@68: .TP
jpayne@68: .B \-\-no\-sparse
jpayne@68: Disable creation of sparse files.
jpayne@68: By default, if decompressing into a regular file,
jpayne@68: .B xz
jpayne@68: tries to make the file sparse if the decompressed data contains
jpayne@68: long sequences of binary zeros.
jpayne@68: It also works when writing to standard output
jpayne@68: as long as standard output is connected to a regular file
jpayne@68: and certain additional conditions are met to make it safe.
jpayne@68: Creating sparse files may save disk space and speed up
jpayne@68: the decompression by reducing the amount of disk I/O.
jpayne@68: .TP
jpayne@68: \fB\-S\fR \fI.suf\fR, \fB\-\-suffix=\fI.suf
jpayne@68: When compressing, use
jpayne@68: .I .suf
jpayne@68: as the suffix for the target file instead of
jpayne@68: .B .xz
jpayne@68: or
jpayne@68: .BR .lzma .
jpayne@68: If not writing to standard output and
jpayne@68: the source file already has the suffix
jpayne@68: .IR .suf ,
jpayne@68: a warning is displayed and the file is skipped.
jpayne@68: .IP ""
jpayne@68: When decompressing, recognize files with the suffix
jpayne@68: .I .suf
jpayne@68: in addition to files with the
jpayne@68: .BR .xz ,
jpayne@68: .BR .txz ,
jpayne@68: .BR .lzma ,
jpayne@68: .BR .tlz ,
jpayne@68: or
jpayne@68: .B .lz
jpayne@68: suffix.
jpayne@68: If the source file has the suffix
jpayne@68: .IR .suf ,
jpayne@68: the suffix is removed to get the target filename.
jpayne@68: .IP ""
jpayne@68: When compressing or decompressing raw streams
jpayne@68: .RB ( \-\-format=raw ),
jpayne@68: the suffix must always be specified unless
jpayne@68: writing to standard output,
jpayne@68: because there is no default suffix for raw streams.
jpayne@68: .TP
jpayne@68: \fB\-\-files\fR[\fB=\fIfile\fR]
jpayne@68: Read the filenames to process from
jpayne@68: .IR file ;
jpayne@68: if
jpayne@68: .I file
jpayne@68: is omitted, filenames are read from standard input.
jpayne@68: Filenames must be terminated with the newline character.
jpayne@68: A dash
jpayne@68: .RB ( \- )
jpayne@68: is taken as a regular filename; it doesn't mean standard input.
jpayne@68: If filenames are given also as command line arguments, they are
jpayne@68: processed before the filenames read from
jpayne@68: .IR file .
jpayne@68: .TP
jpayne@68: \fB\-\-files0\fR[\fB=\fIfile\fR]
jpayne@68: This is identical to \fB\-\-files\fR[\fB=\fIfile\fR] except
jpayne@68: that each filename must be terminated with the null character.
jpayne@68: .
jpayne@68: .SS "Basic file format and compression options"
jpayne@68: .TP
jpayne@68: \fB\-F\fR \fIformat\fR, \fB\-\-format=\fIformat
jpayne@68: Specify the file
jpayne@68: .I format
jpayne@68: to compress or decompress:
jpayne@68: .RS
jpayne@68: .TP
jpayne@68: .B auto
jpayne@68: This is the default.
jpayne@68: When compressing,
jpayne@68: .B auto
jpayne@68: is equivalent to
jpayne@68: .BR xz .
jpayne@68: When decompressing,
jpayne@68: the format of the input file is automatically detected.
jpayne@68: Note that raw streams (created with
jpayne@68: .BR \-\-format=raw )
jpayne@68: cannot be auto-detected.
jpayne@68: .TP
jpayne@68: .B xz
jpayne@68: Compress to the
jpayne@68: .B .xz
jpayne@68: file format, or accept only
jpayne@68: .B .xz
jpayne@68: files when decompressing.
jpayne@68: .TP
jpayne@68: .BR lzma ", " alone
jpayne@68: Compress to the legacy
jpayne@68: .B .lzma
jpayne@68: file format, or accept only
jpayne@68: .B .lzma
jpayne@68: files when decompressing.
jpayne@68: The alternative name
jpayne@68: .B alone
jpayne@68: is provided for backwards compatibility with LZMA Utils.
jpayne@68: .TP
jpayne@68: .B lzip
jpayne@68: Accept only
jpayne@68: .B .lz
jpayne@68: files when decompressing.
jpayne@68: Compression is not supported.
jpayne@68: .IP ""
jpayne@68: The
jpayne@68: .B .lz
jpayne@68: format version 0 and the unextended version 1 are supported.
jpayne@68: Version 0 files were produced by
jpayne@68: .B lzip
jpayne@68: 1.3 and older.
jpayne@68: Such files aren't common but may be found from file archives
jpayne@68: as a few source packages were released in this format.
jpayne@68: People might have old personal files in this format too.
jpayne@68: Decompression support for the format version 0 was removed in
jpayne@68: .B lzip
jpayne@68: 1.18.
jpayne@68: .IP ""
jpayne@68: .B lzip
jpayne@68: 1.4 and later create files in the format version 1.
jpayne@68: The sync flush marker extension to the format version 1 was added in
jpayne@68: .B lzip
jpayne@68: 1.6.
jpayne@68: This extension is rarely used and isn't supported by
jpayne@68: .B xz
jpayne@68: (diagnosed as corrupt input).
jpayne@68: .TP
jpayne@68: .B raw
jpayne@68: Compress or uncompress a raw stream (no headers).
jpayne@68: This is meant for advanced users only.
jpayne@68: To decode raw streams, you need use
jpayne@68: .B \-\-format=raw
jpayne@68: and explicitly specify the filter chain,
jpayne@68: which normally would have been stored in the container headers.
jpayne@68: .RE
jpayne@68: .TP
jpayne@68: \fB\-C\fR \fIcheck\fR, \fB\-\-check=\fIcheck
jpayne@68: Specify the type of the integrity check.
jpayne@68: The check is calculated from the uncompressed data and
jpayne@68: stored in the
jpayne@68: .B .xz
jpayne@68: file.
jpayne@68: This option has an effect only when compressing into the
jpayne@68: .B .xz
jpayne@68: format; the
jpayne@68: .B .lzma
jpayne@68: format doesn't support integrity checks.
jpayne@68: The integrity check (if any) is verified when the
jpayne@68: .B .xz
jpayne@68: file is decompressed.
jpayne@68: .IP ""
jpayne@68: Supported
jpayne@68: .I check
jpayne@68: types:
jpayne@68: .RS
jpayne@68: .TP
jpayne@68: .B none
jpayne@68: Don't calculate an integrity check at all.
jpayne@68: This is usually a bad idea.
jpayne@68: This can be useful when integrity of the data is verified
jpayne@68: by other means anyway.
jpayne@68: .TP
jpayne@68: .B crc32
jpayne@68: Calculate CRC32 using the polynomial from IEEE-802.3 (Ethernet).
jpayne@68: .TP
jpayne@68: .B crc64
jpayne@68: Calculate CRC64 using the polynomial from ECMA-182.
jpayne@68: This is the default, since it is slightly better than CRC32
jpayne@68: at detecting damaged files and the speed difference is negligible.
jpayne@68: .TP
jpayne@68: .B sha256
jpayne@68: Calculate SHA-256.
jpayne@68: This is somewhat slower than CRC32 and CRC64.
jpayne@68: .RE
jpayne@68: .IP ""
jpayne@68: Integrity of the
jpayne@68: .B .xz
jpayne@68: headers is always verified with CRC32.
jpayne@68: It is not possible to change or disable it.
jpayne@68: .TP
jpayne@68: .B \-\-ignore\-check
jpayne@68: Don't verify the integrity check of the compressed data when decompressing.
jpayne@68: The CRC32 values in the
jpayne@68: .B .xz
jpayne@68: headers will still be verified normally.
jpayne@68: .IP ""
jpayne@68: .B "Do not use this option unless you know what you are doing."
jpayne@68: Possible reasons to use this option:
jpayne@68: .RS
jpayne@68: .IP \(bu 3
jpayne@68: Trying to recover data from a corrupt .xz file.
jpayne@68: .IP \(bu 3
jpayne@68: Speeding up decompression.
jpayne@68: This matters mostly with SHA-256 or
jpayne@68: with files that have compressed extremely well.
jpayne@68: It's recommended to not use this option for this purpose
jpayne@68: unless the file integrity is verified externally in some other way.
jpayne@68: .RE
jpayne@68: .TP
jpayne@68: .BR \-0 " ... " \-9
jpayne@68: Select a compression preset level.
jpayne@68: The default is
jpayne@68: .BR \-6 .
jpayne@68: If multiple preset levels are specified,
jpayne@68: the last one takes effect.
jpayne@68: If a custom filter chain was already specified, setting
jpayne@68: a compression preset level clears the custom filter chain.
jpayne@68: .IP ""
jpayne@68: The differences between the presets are more significant than with
jpayne@68: .BR gzip (1)
jpayne@68: and
jpayne@68: .BR bzip2 (1).
jpayne@68: The selected compression settings determine
jpayne@68: the memory requirements of the decompressor,
jpayne@68: thus using a too high preset level might make it painful
jpayne@68: to decompress the file on an old system with little RAM.
jpayne@68: Specifically,
jpayne@68: .B "it's not a good idea to blindly use \-9 for everything"
jpayne@68: like it often is with
jpayne@68: .BR gzip (1)
jpayne@68: and
jpayne@68: .BR bzip2 (1).
jpayne@68: .RS
jpayne@68: .TP
jpayne@68: .BR "\-0" " ... " "\-3"
jpayne@68: These are somewhat fast presets.
jpayne@68: .B \-0
jpayne@68: is sometimes faster than
jpayne@68: .B "gzip \-9"
jpayne@68: while compressing much better.
jpayne@68: The higher ones often have speed comparable to
jpayne@68: .BR bzip2 (1)
jpayne@68: with comparable or better compression ratio,
jpayne@68: although the results
jpayne@68: depend a lot on the type of data being compressed.
jpayne@68: .TP
jpayne@68: .BR "\-4" " ... " "\-6"
jpayne@68: Good to very good compression while keeping
jpayne@68: decompressor memory usage reasonable even for old systems.
jpayne@68: .B \-6
jpayne@68: is the default, which is usually a good choice
jpayne@68: for distributing files that need to be decompressible
jpayne@68: even on systems with only 16\ MiB RAM.
jpayne@68: .RB ( \-5e
jpayne@68: or
jpayne@68: .B \-6e
jpayne@68: may be worth considering too.
jpayne@68: See
jpayne@68: .BR \-\-extreme .)
jpayne@68: .TP
jpayne@68: .B "\-7 ... \-9"
jpayne@68: These are like
jpayne@68: .B \-6
jpayne@68: but with higher compressor and decompressor memory requirements.
jpayne@68: These are useful only when compressing files bigger than
jpayne@68: 8\ MiB, 16\ MiB, and 32\ MiB, respectively.
jpayne@68: .RE
jpayne@68: .IP ""
jpayne@68: On the same hardware, the decompression speed is approximately
jpayne@68: a constant number of bytes of compressed data per second.
jpayne@68: In other words, the better the compression,
jpayne@68: the faster the decompression will usually be.
jpayne@68: This also means that the amount of uncompressed output
jpayne@68: produced per second can vary a lot.
jpayne@68: .IP ""
jpayne@68: The following table summarises the features of the presets:
jpayne@68: .RS
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .TS
jpayne@68: tab(;);
jpayne@68: c c c c c
jpayne@68: n n n n n.
jpayne@68: Preset;DictSize;CompCPU;CompMem;DecMem
jpayne@68: \-0;256 KiB;0;3 MiB;1 MiB
jpayne@68: \-1;1 MiB;1;9 MiB;2 MiB
jpayne@68: \-2;2 MiB;2;17 MiB;3 MiB
jpayne@68: \-3;4 MiB;3;32 MiB;5 MiB
jpayne@68: \-4;4 MiB;4;48 MiB;5 MiB
jpayne@68: \-5;8 MiB;5;94 MiB;9 MiB
jpayne@68: \-6;8 MiB;6;94 MiB;9 MiB
jpayne@68: \-7;16 MiB;6;186 MiB;17 MiB
jpayne@68: \-8;32 MiB;6;370 MiB;33 MiB
jpayne@68: \-9;64 MiB;6;674 MiB;65 MiB
jpayne@68: .TE
jpayne@68: .RE
jpayne@68: .RE
jpayne@68: .IP ""
jpayne@68: Column descriptions:
jpayne@68: .RS
jpayne@68: .IP \(bu 3
jpayne@68: DictSize is the LZMA2 dictionary size.
jpayne@68: It is waste of memory to use a dictionary bigger than
jpayne@68: the size of the uncompressed file.
jpayne@68: This is why it is good to avoid using the presets
jpayne@68: .BR \-7 " ... " \-9
jpayne@68: when there's no real need for them.
jpayne@68: At
jpayne@68: .B \-6
jpayne@68: and lower, the amount of memory wasted is
jpayne@68: usually low enough to not matter.
jpayne@68: .IP \(bu 3
jpayne@68: CompCPU is a simplified representation of the LZMA2 settings
jpayne@68: that affect compression speed.
jpayne@68: The dictionary size affects speed too,
jpayne@68: so while CompCPU is the same for levels
jpayne@68: .BR \-6 " ... " \-9 ,
jpayne@68: higher levels still tend to be a little slower.
jpayne@68: To get even slower and thus possibly better compression, see
jpayne@68: .BR \-\-extreme .
jpayne@68: .IP \(bu 3
jpayne@68: CompMem contains the compressor memory requirements
jpayne@68: in the single-threaded mode.
jpayne@68: It may vary slightly between
jpayne@68: .B xz
jpayne@68: versions.
jpayne@68: .IP \(bu 3
jpayne@68: DecMem contains the decompressor memory requirements.
jpayne@68: That is, the compression settings determine
jpayne@68: the memory requirements of the decompressor.
jpayne@68: The exact decompressor memory usage is slightly more than
jpayne@68: the LZMA2 dictionary size, but the values in the table
jpayne@68: have been rounded up to the next full MiB.
jpayne@68: .RE
jpayne@68: .IP ""
jpayne@68: Memory requirements of the multi-threaded mode are
jpayne@68: significantly higher than that of the single-threaded mode.
jpayne@68: With the default value of
jpayne@68: .BR \-\-block\-size ,
jpayne@68: each thread needs 3*3*DictSize plus CompMem or DecMem.
jpayne@68: For example, four threads with preset
jpayne@68: .B \-6
jpayne@68: needs 660\(en670\ MiB of memory.
jpayne@68: .TP
jpayne@68: .BR \-e ", " \-\-extreme
jpayne@68: Use a slower variant of the selected compression preset level
jpayne@68: .RB ( \-0 " ... " \-9 )
jpayne@68: to hopefully get a little bit better compression ratio,
jpayne@68: but with bad luck this can also make it worse.
jpayne@68: Decompressor memory usage is not affected,
jpayne@68: but compressor memory usage increases a little at preset levels
jpayne@68: .BR \-0 " ... " \-3 .
jpayne@68: .IP ""
jpayne@68: Since there are two presets with dictionary sizes
jpayne@68: 4\ MiB and 8\ MiB, the presets
jpayne@68: .B \-3e
jpayne@68: and
jpayne@68: .B \-5e
jpayne@68: use slightly faster settings (lower CompCPU) than
jpayne@68: .B \-4e
jpayne@68: and
jpayne@68: .BR \-6e ,
jpayne@68: respectively.
jpayne@68: That way no two presets are identical.
jpayne@68: .RS
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .TS
jpayne@68: tab(;);
jpayne@68: c c c c c
jpayne@68: n n n n n.
jpayne@68: Preset;DictSize;CompCPU;CompMem;DecMem
jpayne@68: \-0e;256 KiB;8;4 MiB;1 MiB
jpayne@68: \-1e;1 MiB;8;13 MiB;2 MiB
jpayne@68: \-2e;2 MiB;8;25 MiB;3 MiB
jpayne@68: \-3e;4 MiB;7;48 MiB;5 MiB
jpayne@68: \-4e;4 MiB;8;48 MiB;5 MiB
jpayne@68: \-5e;8 MiB;7;94 MiB;9 MiB
jpayne@68: \-6e;8 MiB;8;94 MiB;9 MiB
jpayne@68: \-7e;16 MiB;8;186 MiB;17 MiB
jpayne@68: \-8e;32 MiB;8;370 MiB;33 MiB
jpayne@68: \-9e;64 MiB;8;674 MiB;65 MiB
jpayne@68: .TE
jpayne@68: .RE
jpayne@68: .RE
jpayne@68: .IP ""
jpayne@68: For example, there are a total of four presets that use
jpayne@68: 8\ MiB dictionary, whose order from the fastest to the slowest is
jpayne@68: .BR \-5 ,
jpayne@68: .BR \-6 ,
jpayne@68: .BR \-5e ,
jpayne@68: and
jpayne@68: .BR \-6e .
jpayne@68: .TP
jpayne@68: .B \-\-fast
jpayne@68: .PD 0
jpayne@68: .TP
jpayne@68: .B \-\-best
jpayne@68: .PD
jpayne@68: These are somewhat misleading aliases for
jpayne@68: .B \-0
jpayne@68: and
jpayne@68: .BR \-9 ,
jpayne@68: respectively.
jpayne@68: These are provided only for backwards compatibility
jpayne@68: with LZMA Utils.
jpayne@68: Avoid using these options.
jpayne@68: .TP
jpayne@68: .BI \-\-block\-size= size
jpayne@68: When compressing to the
jpayne@68: .B .xz
jpayne@68: format, split the input data into blocks of
jpayne@68: .I size
jpayne@68: bytes.
jpayne@68: The blocks are compressed independently from each other,
jpayne@68: which helps with multi-threading and
jpayne@68: makes limited random-access decompression possible.
jpayne@68: This option is typically used to override the default
jpayne@68: block size in multi-threaded mode,
jpayne@68: but this option can be used in single-threaded mode too.
jpayne@68: .IP ""
jpayne@68: In multi-threaded mode about three times
jpayne@68: .I size
jpayne@68: bytes will be allocated in each thread for buffering input and output.
jpayne@68: The default
jpayne@68: .I size
jpayne@68: is three times the LZMA2 dictionary size or 1 MiB,
jpayne@68: whichever is more.
jpayne@68: Typically a good value is 2\(en4 times
jpayne@68: the size of the LZMA2 dictionary or at least 1 MiB.
jpayne@68: Using
jpayne@68: .I size
jpayne@68: less than the LZMA2 dictionary size is waste of RAM
jpayne@68: because then the LZMA2 dictionary buffer will never get fully used.
jpayne@68: In multi-threaded mode,
jpayne@68: the sizes of the blocks are stored in the block headers.
jpayne@68: This size information is required for multi-threaded decompression.
jpayne@68: .IP ""
jpayne@68: In single-threaded mode no block splitting is done by default.
jpayne@68: Setting this option doesn't affect memory usage.
jpayne@68: No size information is stored in block headers,
jpayne@68: thus files created in single-threaded mode
jpayne@68: won't be identical to files created in multi-threaded mode.
jpayne@68: The lack of size information also means that
jpayne@68: .B xz
jpayne@68: won't be able decompress the files in multi-threaded mode.
jpayne@68: .TP
jpayne@68: .BI \-\-block\-list= items
jpayne@68: When compressing to the
jpayne@68: .B .xz
jpayne@68: format, start a new block with an optional custom filter chain after
jpayne@68: the given intervals of uncompressed data.
jpayne@68: .IP ""
jpayne@68: The
jpayne@68: .I items
jpayne@68: are a comma-separated list.
jpayne@68: Each item consists of an optional filter chain number
jpayne@68: between 0 and 9 followed by a colon
jpayne@68: .RB ( : )
jpayne@68: and a required size of uncompressed data.
jpayne@68: Omitting an item (two or more consecutive commas) is a
jpayne@68: shorthand to use the size and filters of the previous item.
jpayne@68: .IP ""
jpayne@68: If the input file is bigger than the sum of
jpayne@68: the sizes in
jpayne@68: .IR items ,
jpayne@68: the last item is repeated until the end of the file.
jpayne@68: A special value of
jpayne@68: .B 0
jpayne@68: may be used as the last size to indicate that
jpayne@68: the rest of the file should be encoded as a single block.
jpayne@68: .IP ""
jpayne@68: An alternative filter chain for each block can be
jpayne@68: specified in combination with the
jpayne@68: .BI \-\-filters1= filters
jpayne@68: \&...\&
jpayne@68: .BI \-\-filters9= filters
jpayne@68: options.
jpayne@68: These options define filter chains with an identifier
jpayne@68: between 1\(en9.
jpayne@68: Filter chain 0 can be used to refer to the default filter chain,
jpayne@68: which is the same as not specifying a filter chain.
jpayne@68: The filter chain identifier can be used before the uncompressed
jpayne@68: size, followed by a colon
jpayne@68: .RB ( : ).
jpayne@68: For example, if one specifies
jpayne@68: .B \-\-block\-list=1:2MiB,3:2MiB,2:4MiB,,2MiB,0:4MiB
jpayne@68: then blocks will be created using:
jpayne@68: .RS
jpayne@68: .IP \(bu 3
jpayne@68: The filter chain specified by
jpayne@68: .B \-\-filters1
jpayne@68: and 2 MiB input
jpayne@68: .IP \(bu 3
jpayne@68: The filter chain specified by
jpayne@68: .B \-\-filters3
jpayne@68: and 2 MiB input
jpayne@68: .IP \(bu 3
jpayne@68: The filter chain specified by
jpayne@68: .B \-\-filters2
jpayne@68: and 4 MiB input
jpayne@68: .IP \(bu 3
jpayne@68: The filter chain specified by
jpayne@68: .B \-\-filters2
jpayne@68: and 4 MiB input
jpayne@68: .IP \(bu 3
jpayne@68: The default filter chain and 2 MiB input
jpayne@68: .IP \(bu 3
jpayne@68: The default filter chain and 4 MiB input for every block until
jpayne@68: end of input.
jpayne@68: .RE
jpayne@68: .IP ""
jpayne@68: If one specifies a size that exceeds the encoder's block size
jpayne@68: (either the default value in threaded mode or
jpayne@68: the value specified with \fB\-\-block\-size=\fIsize\fR),
jpayne@68: the encoder will create additional blocks while
jpayne@68: keeping the boundaries specified in
jpayne@68: .IR items .
jpayne@68: For example, if one specifies
jpayne@68: .B \-\-block\-size=10MiB
jpayne@68: .B \-\-block\-list=5MiB,10MiB,8MiB,12MiB,24MiB
jpayne@68: and the input file is 80 MiB,
jpayne@68: one will get 11 blocks:
jpayne@68: 5, 10, 8, 10, 2, 10, 10, 4, 10, 10, and 1 MiB.
jpayne@68: .IP ""
jpayne@68: In multi-threaded mode the sizes of the blocks
jpayne@68: are stored in the block headers.
jpayne@68: This isn't done in single-threaded mode,
jpayne@68: so the encoded output won't be
jpayne@68: identical to that of the multi-threaded mode.
jpayne@68: .TP
jpayne@68: .BI \-\-flush\-timeout= timeout
jpayne@68: When compressing, if more than
jpayne@68: .I timeout
jpayne@68: milliseconds (a positive integer) has passed since the previous flush and
jpayne@68: reading more input would block,
jpayne@68: all the pending input data is flushed from the encoder and
jpayne@68: made available in the output stream.
jpayne@68: This can be useful if
jpayne@68: .B xz
jpayne@68: is used to compress data that is streamed over a network.
jpayne@68: Small
jpayne@68: .I timeout
jpayne@68: values make the data available at the receiving end
jpayne@68: with a small delay, but large
jpayne@68: .I timeout
jpayne@68: values give better compression ratio.
jpayne@68: .IP ""
jpayne@68: This feature is disabled by default.
jpayne@68: If this option is specified more than once, the last one takes effect.
jpayne@68: The special
jpayne@68: .I timeout
jpayne@68: value of
jpayne@68: .B 0
jpayne@68: can be used to explicitly disable this feature.
jpayne@68: .IP ""
jpayne@68: This feature is not available on non-POSIX systems.
jpayne@68: .IP ""
jpayne@68: .\" FIXME
jpayne@68: .B "This feature is still experimental."
jpayne@68: Currently
jpayne@68: .B xz
jpayne@68: is unsuitable for decompressing the stream in real time due to how
jpayne@68: .B xz
jpayne@68: does buffering.
jpayne@68: .TP
jpayne@68: .BI \-\-memlimit\-compress= limit
jpayne@68: Set a memory usage limit for compression.
jpayne@68: If this option is specified multiple times,
jpayne@68: the last one takes effect.
jpayne@68: .IP ""
jpayne@68: If the compression settings exceed the
jpayne@68: .IR limit ,
jpayne@68: .B xz
jpayne@68: will attempt to adjust the settings downwards so that
jpayne@68: the limit is no longer exceeded and display a notice that
jpayne@68: automatic adjustment was done.
jpayne@68: The adjustments are done in this order:
jpayne@68: reducing the number of threads,
jpayne@68: switching to single-threaded mode
jpayne@68: if even one thread in multi-threaded mode exceeds the
jpayne@68: .IR limit ,
jpayne@68: and finally reducing the LZMA2 dictionary size.
jpayne@68: .IP ""
jpayne@68: When compressing with
jpayne@68: .B \-\-format=raw
jpayne@68: or if
jpayne@68: .B \-\-no\-adjust
jpayne@68: has been specified,
jpayne@68: only the number of threads may be reduced
jpayne@68: since it can be done without affecting the compressed output.
jpayne@68: .IP ""
jpayne@68: If the
jpayne@68: .I limit
jpayne@68: cannot be met even with the adjustments described above,
jpayne@68: an error is displayed and
jpayne@68: .B xz
jpayne@68: will exit with exit status 1.
jpayne@68: .IP ""
jpayne@68: The
jpayne@68: .I limit
jpayne@68: can be specified in multiple ways:
jpayne@68: .RS
jpayne@68: .IP \(bu 3
jpayne@68: The
jpayne@68: .I limit
jpayne@68: can be an absolute value in bytes.
jpayne@68: Using an integer suffix like
jpayne@68: .B MiB
jpayne@68: can be useful.
jpayne@68: Example:
jpayne@68: .B "\-\-memlimit\-compress=80MiB"
jpayne@68: .IP \(bu 3
jpayne@68: The
jpayne@68: .I limit
jpayne@68: can be specified as a percentage of total physical memory (RAM).
jpayne@68: This can be useful especially when setting the
jpayne@68: .B XZ_DEFAULTS
jpayne@68: environment variable in a shell initialization script
jpayne@68: that is shared between different computers.
jpayne@68: That way the limit is automatically bigger
jpayne@68: on systems with more memory.
jpayne@68: Example:
jpayne@68: .B "\-\-memlimit\-compress=70%"
jpayne@68: .IP \(bu 3
jpayne@68: The
jpayne@68: .I limit
jpayne@68: can be reset back to its default value by setting it to
jpayne@68: .BR 0 .
jpayne@68: This is currently equivalent to setting the
jpayne@68: .I limit
jpayne@68: to
jpayne@68: .B max
jpayne@68: (no memory usage limit).
jpayne@68: .RE
jpayne@68: .IP ""
jpayne@68: For 32-bit
jpayne@68: .B xz
jpayne@68: there is a special case: if the
jpayne@68: .I limit
jpayne@68: would be over
jpayne@68: .BR "4020\ MiB" ,
jpayne@68: the
jpayne@68: .I limit
jpayne@68: is set to
jpayne@68: .BR "4020\ MiB" .
jpayne@68: On MIPS32
jpayne@68: .B "2000\ MiB"
jpayne@68: is used instead.
jpayne@68: (The values
jpayne@68: .B 0
jpayne@68: and
jpayne@68: .B max
jpayne@68: aren't affected by this.
jpayne@68: A similar feature doesn't exist for decompression.)
jpayne@68: This can be helpful when a 32-bit executable has access
jpayne@68: to 4\ GiB address space (2 GiB on MIPS32)
jpayne@68: while hopefully doing no harm in other situations.
jpayne@68: .IP ""
jpayne@68: See also the section
jpayne@68: .BR "Memory usage" .
jpayne@68: .TP
jpayne@68: .BI \-\-memlimit\-decompress= limit
jpayne@68: Set a memory usage limit for decompression.
jpayne@68: This also affects the
jpayne@68: .B \-\-list
jpayne@68: mode.
jpayne@68: If the operation is not possible without exceeding the
jpayne@68: .IR limit ,
jpayne@68: .B xz
jpayne@68: will display an error and decompressing the file will fail.
jpayne@68: See
jpayne@68: .BI \-\-memlimit\-compress= limit
jpayne@68: for possible ways to specify the
jpayne@68: .IR limit .
jpayne@68: .TP
jpayne@68: .BI \-\-memlimit\-mt\-decompress= limit
jpayne@68: Set a memory usage limit for multi-threaded decompression.
jpayne@68: This can only affect the number of threads;
jpayne@68: this will never make
jpayne@68: .B xz
jpayne@68: refuse to decompress a file.
jpayne@68: If
jpayne@68: .I limit
jpayne@68: is too low to allow any multi-threading, the
jpayne@68: .I limit
jpayne@68: is ignored and
jpayne@68: .B xz
jpayne@68: will continue in single-threaded mode.
jpayne@68: Note that if also
jpayne@68: .B \-\-memlimit\-decompress
jpayne@68: is used,
jpayne@68: it will always apply to both single-threaded and multi-threaded modes,
jpayne@68: and so the effective
jpayne@68: .I limit
jpayne@68: for multi-threading will never be higher than the limit set with
jpayne@68: .BR \-\-memlimit\-decompress .
jpayne@68: .IP ""
jpayne@68: In contrast to the other memory usage limit options,
jpayne@68: .BI \-\-memlimit\-mt\-decompress= limit
jpayne@68: has a system-specific default
jpayne@68: .IR limit .
jpayne@68: .B "xz \-\-info\-memory"
jpayne@68: can be used to see the current value.
jpayne@68: .IP ""
jpayne@68: This option and its default value exist
jpayne@68: because without any limit the threaded decompressor
jpayne@68: could end up allocating an insane amount of memory with some input files.
jpayne@68: If the default
jpayne@68: .I limit
jpayne@68: is too low on your system,
jpayne@68: feel free to increase the
jpayne@68: .I limit
jpayne@68: but never set it to a value larger than the amount of usable RAM
jpayne@68: as with appropriate input files
jpayne@68: .B xz
jpayne@68: will attempt to use that amount of memory
jpayne@68: even with a low number of threads.
jpayne@68: Running out of memory or swapping
jpayne@68: will not improve decompression performance.
jpayne@68: .IP ""
jpayne@68: See
jpayne@68: .BI \-\-memlimit\-compress= limit
jpayne@68: for possible ways to specify the
jpayne@68: .IR limit .
jpayne@68: Setting
jpayne@68: .I limit
jpayne@68: to
jpayne@68: .B 0
jpayne@68: resets the
jpayne@68: .I limit
jpayne@68: to the default system-specific value.
jpayne@68: .TP
jpayne@68: \fB\-M\fR \fIlimit\fR, \fB\-\-memlimit=\fIlimit\fR, \fB\-\-memory=\fIlimit
jpayne@68: This is equivalent to specifying
jpayne@68: .BI \-\-memlimit\-compress= limit
jpayne@68: .BI \-\-memlimit-decompress= limit
jpayne@68: \fB\-\-memlimit\-mt\-decompress=\fIlimit\fR.
jpayne@68: .TP
jpayne@68: .B \-\-no\-adjust
jpayne@68: Display an error and exit if the memory usage limit cannot be
jpayne@68: met without adjusting settings that affect the compressed output.
jpayne@68: That is, this prevents
jpayne@68: .B xz
jpayne@68: from switching the encoder from multi-threaded mode to single-threaded mode
jpayne@68: and from reducing the LZMA2 dictionary size.
jpayne@68: Even when this option is used the number of threads may be reduced
jpayne@68: to meet the memory usage limit as that won't affect the compressed output.
jpayne@68: .IP ""
jpayne@68: Automatic adjusting is always disabled when creating raw streams
jpayne@68: .RB ( \-\-format=raw ).
jpayne@68: .TP
jpayne@68: \fB\-T\fR \fIthreads\fR, \fB\-\-threads=\fIthreads
jpayne@68: Specify the number of worker threads to use.
jpayne@68: Setting
jpayne@68: .I threads
jpayne@68: to a special value
jpayne@68: .B 0
jpayne@68: makes
jpayne@68: .B xz
jpayne@68: use up to as many threads as the processor(s) on the system support.
jpayne@68: The actual number of threads can be fewer than
jpayne@68: .I threads
jpayne@68: if the input file is not big enough
jpayne@68: for threading with the given settings or
jpayne@68: if using more threads would exceed the memory usage limit.
jpayne@68: .IP ""
jpayne@68: The single-threaded and multi-threaded compressors produce different output.
jpayne@68: Single-threaded compressor will give the smallest file size but
jpayne@68: only the output from the multi-threaded compressor can be decompressed
jpayne@68: using multiple threads.
jpayne@68: Setting
jpayne@68: .I threads
jpayne@68: to
jpayne@68: .B 1
jpayne@68: will use the single-threaded mode.
jpayne@68: Setting
jpayne@68: .I threads
jpayne@68: to any other value, including
jpayne@68: .BR 0 ,
jpayne@68: will use the multi-threaded compressor
jpayne@68: even if the system supports only one hardware thread.
jpayne@68: .RB ( xz
jpayne@68: 5.2.x
jpayne@68: used single-threaded mode in this situation.)
jpayne@68: .IP ""
jpayne@68: To use multi-threaded mode with only one thread, set
jpayne@68: .I threads
jpayne@68: to
jpayne@68: .BR +1 .
jpayne@68: The
jpayne@68: .B +
jpayne@68: prefix has no effect with values other than
jpayne@68: .BR 1 .
jpayne@68: A memory usage limit can still make
jpayne@68: .B xz
jpayne@68: switch to single-threaded mode unless
jpayne@68: .B \-\-no\-adjust
jpayne@68: is used.
jpayne@68: Support for the
jpayne@68: .B +
jpayne@68: prefix was added in
jpayne@68: .B xz
jpayne@68: 5.4.0.
jpayne@68: .IP ""
jpayne@68: If an automatic number of threads has been requested and
jpayne@68: no memory usage limit has been specified,
jpayne@68: then a system-specific default soft limit will be used to possibly
jpayne@68: limit the number of threads.
jpayne@68: It is a soft limit in sense that it is ignored
jpayne@68: if the number of threads becomes one,
jpayne@68: thus a soft limit will never stop
jpayne@68: .B xz
jpayne@68: from compressing or decompressing.
jpayne@68: This default soft limit will not make
jpayne@68: .B xz
jpayne@68: switch from multi-threaded mode to single-threaded mode.
jpayne@68: The active limits can be seen with
jpayne@68: .BR "xz \-\-info\-memory" .
jpayne@68: .IP ""
jpayne@68: Currently the only threading method is to split the input into
jpayne@68: blocks and compress them independently from each other.
jpayne@68: The default block size depends on the compression level and
jpayne@68: can be overridden with the
jpayne@68: .BI \-\-block\-size= size
jpayne@68: option.
jpayne@68: .IP ""
jpayne@68: Threaded decompression only works on files that contain
jpayne@68: multiple blocks with size information in block headers.
jpayne@68: All large enough files compressed in multi-threaded mode
jpayne@68: meet this condition,
jpayne@68: but files compressed in single-threaded mode don't even if
jpayne@68: .BI \-\-block\-size= size
jpayne@68: has been used.
jpayne@68: .IP ""
jpayne@68: The default value for
jpayne@68: .I threads
jpayne@68: is
jpayne@68: .BR 0 .
jpayne@68: In
jpayne@68: .B xz
jpayne@68: 5.4.x and older the default is
jpayne@68: .BR 1 .
jpayne@68: .
jpayne@68: .SS "Custom compressor filter chains"
jpayne@68: A custom filter chain allows specifying
jpayne@68: the compression settings in detail instead of relying on
jpayne@68: the settings associated to the presets.
jpayne@68: When a custom filter chain is specified,
jpayne@68: preset options
jpayne@68: .RB ( \-0
jpayne@68: \&...\&
jpayne@68: .B \-9
jpayne@68: and
jpayne@68: .BR \-\-extreme )
jpayne@68: earlier on the command line are forgotten.
jpayne@68: If a preset option is specified
jpayne@68: after one or more custom filter chain options,
jpayne@68: the new preset takes effect and
jpayne@68: the custom filter chain options specified earlier are forgotten.
jpayne@68: .PP
jpayne@68: A filter chain is comparable to piping on the command line.
jpayne@68: When compressing, the uncompressed input goes to the first filter,
jpayne@68: whose output goes to the next filter (if any).
jpayne@68: The output of the last filter gets written to the compressed file.
jpayne@68: The maximum number of filters in the chain is four,
jpayne@68: but typically a filter chain has only one or two filters.
jpayne@68: .PP
jpayne@68: Many filters have limitations on where they can be
jpayne@68: in the filter chain:
jpayne@68: some filters can work only as the last filter in the chain,
jpayne@68: some only as a non-last filter, and some work in any position
jpayne@68: in the chain.
jpayne@68: Depending on the filter, this limitation is either inherent to
jpayne@68: the filter design or exists to prevent security issues.
jpayne@68: .PP
jpayne@68: A custom filter chain can be specified in two different ways.
jpayne@68: The options
jpayne@68: .BI \-\-filters= filters
jpayne@68: and
jpayne@68: .BI \-\-filters1= filters
jpayne@68: \&...\&
jpayne@68: .BI \-\-filters9= filters
jpayne@68: allow specifying an entire filter chain in one option using the
jpayne@68: liblzma filter string syntax.
jpayne@68: Alternatively, a filter chain can be specified by using one or more
jpayne@68: individual filter options in the order they are wanted in the filter chain.
jpayne@68: That is, the order of the individual filter options is significant!
jpayne@68: When decoding raw streams
jpayne@68: .RB ( \-\-format=raw ),
jpayne@68: the filter chain must be specified in the same order as
jpayne@68: it was specified when compressing.
jpayne@68: Any individual filter or preset options specified before the full
jpayne@68: chain option
jpayne@68: (\fB\-\-filters=\fIfilters\fR)
jpayne@68: will be forgotten.
jpayne@68: Individual filters specified after the full chain option will reset the
jpayne@68: filter chain.
jpayne@68: .PP
jpayne@68: Both the full and individual filter options take filter-specific
jpayne@68: .I options
jpayne@68: as a comma-separated list.
jpayne@68: Extra commas in
jpayne@68: .I options
jpayne@68: are ignored.
jpayne@68: Every option has a default value, so
jpayne@68: specify those you want to change.
jpayne@68: .PP
jpayne@68: To see the whole filter chain and
jpayne@68: .IR options ,
jpayne@68: use
jpayne@68: .B "xz \-vv"
jpayne@68: (that is, use
jpayne@68: .B \-\-verbose
jpayne@68: twice).
jpayne@68: This works also for viewing the filter chain options used by presets.
jpayne@68: .TP
jpayne@68: .BI \-\-filters= filters
jpayne@68: Specify the full filter chain or a preset in a single option.
jpayne@68: Each filter can be separated by spaces or two dashes
jpayne@68: .RB ( \-\- ).
jpayne@68: .I filters
jpayne@68: may need to be quoted on the shell command line so it is
jpayne@68: parsed as a single option.
jpayne@68: To denote
jpayne@68: .IR options ,
jpayne@68: use
jpayne@68: .B :
jpayne@68: or
jpayne@68: .BR = .
jpayne@68: A preset can be prefixed with a
jpayne@68: .B \-
jpayne@68: and followed with zero or more flags.
jpayne@68: The only supported flag is
jpayne@68: .B e
jpayne@68: to apply the same options as
jpayne@68: .BR \-\-extreme .
jpayne@68: .TP
jpayne@68: \fB\-\-filters1\fR=\fIfilters\fR ... \fB\-\-filters9\fR=\fIfilters
jpayne@68: Specify up to nine additional filter chains that can be used with
jpayne@68: .BR \-\-block\-list .
jpayne@68: .IP ""
jpayne@68: For example, when compressing an archive with executable files
jpayne@68: followed by text files, the executable part could use a filter
jpayne@68: chain with a BCJ filter and the text part only the LZMA2 filter.
jpayne@68: .TP
jpayne@68: .B \-\-filters-help
jpayne@68: Display a help message describing how to specify presets and
jpayne@68: custom filter chains in the
jpayne@68: .B \-\-filters
jpayne@68: and
jpayne@68: .BI \-\-filters1= filters
jpayne@68: \&...\&
jpayne@68: .BI \-\-filters9= filters
jpayne@68: options, and exit successfully.
jpayne@68: .TP
jpayne@68: \fB\-\-lzma1\fR[\fB=\fIoptions\fR]
jpayne@68: .PD 0
jpayne@68: .TP
jpayne@68: \fB\-\-lzma2\fR[\fB=\fIoptions\fR]
jpayne@68: .PD
jpayne@68: Add LZMA1 or LZMA2 filter to the filter chain.
jpayne@68: These filters can be used only as the last filter in the chain.
jpayne@68: .IP ""
jpayne@68: LZMA1 is a legacy filter,
jpayne@68: which is supported almost solely due to the legacy
jpayne@68: .B .lzma
jpayne@68: file format, which supports only LZMA1.
jpayne@68: LZMA2 is an updated
jpayne@68: version of LZMA1 to fix some practical issues of LZMA1.
jpayne@68: The
jpayne@68: .B .xz
jpayne@68: format uses LZMA2 and doesn't support LZMA1 at all.
jpayne@68: Compression speed and ratios of LZMA1 and LZMA2
jpayne@68: are practically the same.
jpayne@68: .IP ""
jpayne@68: LZMA1 and LZMA2 share the same set of
jpayne@68: .IR options :
jpayne@68: .RS
jpayne@68: .TP
jpayne@68: .BI preset= preset
jpayne@68: Reset all LZMA1 or LZMA2
jpayne@68: .I options
jpayne@68: to
jpayne@68: .IR preset .
jpayne@68: .I Preset
jpayne@68: consist of an integer, which may be followed by single-letter
jpayne@68: preset modifiers.
jpayne@68: The integer can be from
jpayne@68: .B 0
jpayne@68: to
jpayne@68: .BR 9 ,
jpayne@68: matching the command line options
jpayne@68: .B \-0
jpayne@68: \&...\&
jpayne@68: .BR \-9 .
jpayne@68: The only supported modifier is currently
jpayne@68: .BR e ,
jpayne@68: which matches
jpayne@68: .BR \-\-extreme .
jpayne@68: If no
jpayne@68: .B preset
jpayne@68: is specified, the default values of LZMA1 or LZMA2
jpayne@68: .I options
jpayne@68: are taken from the preset
jpayne@68: .BR 6 .
jpayne@68: .TP
jpayne@68: .BI dict= size
jpayne@68: Dictionary (history buffer)
jpayne@68: .I size
jpayne@68: indicates how many bytes of the recently processed
jpayne@68: uncompressed data is kept in memory.
jpayne@68: The algorithm tries to find repeating byte sequences (matches) in
jpayne@68: the uncompressed data, and replace them with references
jpayne@68: to the data currently in the dictionary.
jpayne@68: The bigger the dictionary, the higher is the chance
jpayne@68: to find a match.
jpayne@68: Thus, increasing dictionary
jpayne@68: .I size
jpayne@68: usually improves compression ratio, but
jpayne@68: a dictionary bigger than the uncompressed file is waste of memory.
jpayne@68: .IP ""
jpayne@68: Typical dictionary
jpayne@68: .I size
jpayne@68: is from 64\ KiB to 64\ MiB.
jpayne@68: The minimum is 4\ KiB.
jpayne@68: The maximum for compression is currently 1.5\ GiB (1536\ MiB).
jpayne@68: The decompressor already supports dictionaries up to
jpayne@68: one byte less than 4\ GiB, which is the maximum for
jpayne@68: the LZMA1 and LZMA2 stream formats.
jpayne@68: .IP ""
jpayne@68: Dictionary
jpayne@68: .I size
jpayne@68: and match finder
jpayne@68: .RI ( mf )
jpayne@68: together determine the memory usage of the LZMA1 or LZMA2 encoder.
jpayne@68: The same (or bigger) dictionary
jpayne@68: .I size
jpayne@68: is required for decompressing that was used when compressing,
jpayne@68: thus the memory usage of the decoder is determined
jpayne@68: by the dictionary size used when compressing.
jpayne@68: The
jpayne@68: .B .xz
jpayne@68: headers store the dictionary
jpayne@68: .I size
jpayne@68: either as
jpayne@68: .RI "2^" n
jpayne@68: or
jpayne@68: .RI "2^" n " + 2^(" n "\-1),"
jpayne@68: so these
jpayne@68: .I sizes
jpayne@68: are somewhat preferred for compression.
jpayne@68: Other
jpayne@68: .I sizes
jpayne@68: will get rounded up when stored in the
jpayne@68: .B .xz
jpayne@68: headers.
jpayne@68: .TP
jpayne@68: .BI lc= lc
jpayne@68: Specify the number of literal context bits.
jpayne@68: The minimum is 0 and the maximum is 4; the default is 3.
jpayne@68: In addition, the sum of
jpayne@68: .I lc
jpayne@68: and
jpayne@68: .I lp
jpayne@68: must not exceed 4.
jpayne@68: .IP ""
jpayne@68: All bytes that cannot be encoded as matches
jpayne@68: are encoded as literals.
jpayne@68: That is, literals are simply 8-bit bytes
jpayne@68: that are encoded one at a time.
jpayne@68: .IP ""
jpayne@68: The literal coding makes an assumption that the highest
jpayne@68: .I lc
jpayne@68: bits of the previous uncompressed byte correlate
jpayne@68: with the next byte.
jpayne@68: For example, in typical English text, an upper-case letter is
jpayne@68: often followed by a lower-case letter, and a lower-case
jpayne@68: letter is usually followed by another lower-case letter.
jpayne@68: In the US-ASCII character set, the highest three bits are 010
jpayne@68: for upper-case letters and 011 for lower-case letters.
jpayne@68: When
jpayne@68: .I lc
jpayne@68: is at least 3, the literal coding can take advantage of
jpayne@68: this property in the uncompressed data.
jpayne@68: .IP ""
jpayne@68: The default value (3) is usually good.
jpayne@68: If you want maximum compression, test
jpayne@68: .BR lc=4 .
jpayne@68: Sometimes it helps a little, and
jpayne@68: sometimes it makes compression worse.
jpayne@68: If it makes it worse, test
jpayne@68: .B lc=2
jpayne@68: too.
jpayne@68: .TP
jpayne@68: .BI lp= lp
jpayne@68: Specify the number of literal position bits.
jpayne@68: The minimum is 0 and the maximum is 4; the default is 0.
jpayne@68: .IP ""
jpayne@68: .I Lp
jpayne@68: affects what kind of alignment in the uncompressed data is
jpayne@68: assumed when encoding literals.
jpayne@68: See
jpayne@68: .I pb
jpayne@68: below for more information about alignment.
jpayne@68: .TP
jpayne@68: .BI pb= pb
jpayne@68: Specify the number of position bits.
jpayne@68: The minimum is 0 and the maximum is 4; the default is 2.
jpayne@68: .IP ""
jpayne@68: .I Pb
jpayne@68: affects what kind of alignment in the uncompressed data is
jpayne@68: assumed in general.
jpayne@68: The default means four-byte alignment
jpayne@68: .RI (2^ pb =2^2=4),
jpayne@68: which is often a good choice when there's no better guess.
jpayne@68: .IP ""
jpayne@68: When the alignment is known, setting
jpayne@68: .I pb
jpayne@68: accordingly may reduce the file size a little.
jpayne@68: For example, with text files having one-byte
jpayne@68: alignment (US-ASCII, ISO-8859-*, UTF-8), setting
jpayne@68: .B pb=0
jpayne@68: can improve compression slightly.
jpayne@68: For UTF-16 text,
jpayne@68: .B pb=1
jpayne@68: is a good choice.
jpayne@68: If the alignment is an odd number like 3 bytes,
jpayne@68: .B pb=0
jpayne@68: might be the best choice.
jpayne@68: .IP ""
jpayne@68: Even though the assumed alignment can be adjusted with
jpayne@68: .I pb
jpayne@68: and
jpayne@68: .IR lp ,
jpayne@68: LZMA1 and LZMA2 still slightly favor 16-byte alignment.
jpayne@68: It might be worth taking into account when designing file formats
jpayne@68: that are likely to be often compressed with LZMA1 or LZMA2.
jpayne@68: .TP
jpayne@68: .BI mf= mf
jpayne@68: Match finder has a major effect on encoder speed,
jpayne@68: memory usage, and compression ratio.
jpayne@68: Usually Hash Chain match finders are faster than Binary Tree
jpayne@68: match finders.
jpayne@68: The default depends on the
jpayne@68: .IR preset :
jpayne@68: 0 uses
jpayne@68: .BR hc3 ,
jpayne@68: 1\(en3
jpayne@68: use
jpayne@68: .BR hc4 ,
jpayne@68: and the rest use
jpayne@68: .BR bt4 .
jpayne@68: .IP ""
jpayne@68: The following match finders are supported.
jpayne@68: The memory usage formulas below are rough approximations,
jpayne@68: which are closest to the reality when
jpayne@68: .I dict
jpayne@68: is a power of two.
jpayne@68: .RS
jpayne@68: .TP
jpayne@68: .B hc3
jpayne@68: Hash Chain with 2- and 3-byte hashing
jpayne@68: .br
jpayne@68: Minimum value for
jpayne@68: .IR nice :
jpayne@68: 3
jpayne@68: .br
jpayne@68: Memory usage:
jpayne@68: .br
jpayne@68: .I dict
jpayne@68: * 7.5 (if
jpayne@68: .I dict
jpayne@68: <= 16 MiB);
jpayne@68: .br
jpayne@68: .I dict
jpayne@68: * 5.5 + 64 MiB (if
jpayne@68: .I dict
jpayne@68: > 16 MiB)
jpayne@68: .TP
jpayne@68: .B hc4
jpayne@68: Hash Chain with 2-, 3-, and 4-byte hashing
jpayne@68: .br
jpayne@68: Minimum value for
jpayne@68: .IR nice :
jpayne@68: 4
jpayne@68: .br
jpayne@68: Memory usage:
jpayne@68: .br
jpayne@68: .I dict
jpayne@68: * 7.5 (if
jpayne@68: .I dict
jpayne@68: <= 32 MiB);
jpayne@68: .br
jpayne@68: .I dict
jpayne@68: * 6.5 (if
jpayne@68: .I dict
jpayne@68: > 32 MiB)
jpayne@68: .TP
jpayne@68: .B bt2
jpayne@68: Binary Tree with 2-byte hashing
jpayne@68: .br
jpayne@68: Minimum value for
jpayne@68: .IR nice :
jpayne@68: 2
jpayne@68: .br
jpayne@68: Memory usage:
jpayne@68: .I dict
jpayne@68: * 9.5
jpayne@68: .TP
jpayne@68: .B bt3
jpayne@68: Binary Tree with 2- and 3-byte hashing
jpayne@68: .br
jpayne@68: Minimum value for
jpayne@68: .IR nice :
jpayne@68: 3
jpayne@68: .br
jpayne@68: Memory usage:
jpayne@68: .br
jpayne@68: .I dict
jpayne@68: * 11.5 (if
jpayne@68: .I dict
jpayne@68: <= 16 MiB);
jpayne@68: .br
jpayne@68: .I dict
jpayne@68: * 9.5 + 64 MiB (if
jpayne@68: .I dict
jpayne@68: > 16 MiB)
jpayne@68: .TP
jpayne@68: .B bt4
jpayne@68: Binary Tree with 2-, 3-, and 4-byte hashing
jpayne@68: .br
jpayne@68: Minimum value for
jpayne@68: .IR nice :
jpayne@68: 4
jpayne@68: .br
jpayne@68: Memory usage:
jpayne@68: .br
jpayne@68: .I dict
jpayne@68: * 11.5 (if
jpayne@68: .I dict
jpayne@68: <= 32 MiB);
jpayne@68: .br
jpayne@68: .I dict
jpayne@68: * 10.5 (if
jpayne@68: .I dict
jpayne@68: > 32 MiB)
jpayne@68: .RE
jpayne@68: .TP
jpayne@68: .BI mode= mode
jpayne@68: Compression
jpayne@68: .I mode
jpayne@68: specifies the method to analyze
jpayne@68: the data produced by the match finder.
jpayne@68: Supported
jpayne@68: .I modes
jpayne@68: are
jpayne@68: .B fast
jpayne@68: and
jpayne@68: .BR normal .
jpayne@68: The default is
jpayne@68: .B fast
jpayne@68: for
jpayne@68: .I presets
jpayne@68: 0\(en3 and
jpayne@68: .B normal
jpayne@68: for
jpayne@68: .I presets
jpayne@68: 4\(en9.
jpayne@68: .IP ""
jpayne@68: Usually
jpayne@68: .B fast
jpayne@68: is used with Hash Chain match finders and
jpayne@68: .B normal
jpayne@68: with Binary Tree match finders.
jpayne@68: This is also what the
jpayne@68: .I presets
jpayne@68: do.
jpayne@68: .TP
jpayne@68: .BI nice= nice
jpayne@68: Specify what is considered to be a nice length for a match.
jpayne@68: Once a match of at least
jpayne@68: .I nice
jpayne@68: bytes is found, the algorithm stops
jpayne@68: looking for possibly better matches.
jpayne@68: .IP ""
jpayne@68: .I Nice
jpayne@68: can be 2\(en273 bytes.
jpayne@68: Higher values tend to give better compression ratio
jpayne@68: at the expense of speed.
jpayne@68: The default depends on the
jpayne@68: .IR preset .
jpayne@68: .TP
jpayne@68: .BI depth= depth
jpayne@68: Specify the maximum search depth in the match finder.
jpayne@68: The default is the special value of 0,
jpayne@68: which makes the compressor determine a reasonable
jpayne@68: .I depth
jpayne@68: from
jpayne@68: .I mf
jpayne@68: and
jpayne@68: .IR nice .
jpayne@68: .IP ""
jpayne@68: Reasonable
jpayne@68: .I depth
jpayne@68: for Hash Chains is 4\(en100 and 16\(en1000 for Binary Trees.
jpayne@68: Using very high values for
jpayne@68: .I depth
jpayne@68: can make the encoder extremely slow with some files.
jpayne@68: Avoid setting the
jpayne@68: .I depth
jpayne@68: over 1000 unless you are prepared to interrupt
jpayne@68: the compression in case it is taking far too long.
jpayne@68: .RE
jpayne@68: .IP ""
jpayne@68: When decoding raw streams
jpayne@68: .RB ( \-\-format=raw ),
jpayne@68: LZMA2 needs only the dictionary
jpayne@68: .IR size .
jpayne@68: LZMA1 needs also
jpayne@68: .IR lc ,
jpayne@68: .IR lp ,
jpayne@68: and
jpayne@68: .IR pb .
jpayne@68: .TP
jpayne@68: \fB\-\-x86\fR[\fB=\fIoptions\fR]
jpayne@68: .PD 0
jpayne@68: .TP
jpayne@68: \fB\-\-arm\fR[\fB=\fIoptions\fR]
jpayne@68: .TP
jpayne@68: \fB\-\-armthumb\fR[\fB=\fIoptions\fR]
jpayne@68: .TP
jpayne@68: \fB\-\-arm64\fR[\fB=\fIoptions\fR]
jpayne@68: .TP
jpayne@68: \fB\-\-powerpc\fR[\fB=\fIoptions\fR]
jpayne@68: .TP
jpayne@68: \fB\-\-ia64\fR[\fB=\fIoptions\fR]
jpayne@68: .TP
jpayne@68: \fB\-\-sparc\fR[\fB=\fIoptions\fR]
jpayne@68: .TP
jpayne@68: \fB\-\-riscv\fR[\fB=\fIoptions\fR]
jpayne@68: .PD
jpayne@68: Add a branch/call/jump (BCJ) filter to the filter chain.
jpayne@68: These filters can be used only as a non-last filter
jpayne@68: in the filter chain.
jpayne@68: .IP ""
jpayne@68: A BCJ filter converts relative addresses in
jpayne@68: the machine code to their absolute counterparts.
jpayne@68: This doesn't change the size of the data
jpayne@68: but it increases redundancy,
jpayne@68: which can help LZMA2 to produce 0\(en15\ % smaller
jpayne@68: .B .xz
jpayne@68: file.
jpayne@68: The BCJ filters are always reversible,
jpayne@68: so using a BCJ filter for wrong type of data
jpayne@68: doesn't cause any data loss, although it may make
jpayne@68: the compression ratio slightly worse.
jpayne@68: The BCJ filters are very fast and
jpayne@68: use an insignificant amount of memory.
jpayne@68: .IP ""
jpayne@68: These BCJ filters have known problems related to
jpayne@68: the compression ratio:
jpayne@68: .RS
jpayne@68: .IP \(bu 3
jpayne@68: Some types of files containing executable code
jpayne@68: (for example, object files, static libraries, and Linux kernel modules)
jpayne@68: have the addresses in the instructions filled with filler values.
jpayne@68: These BCJ filters will still do the address conversion,
jpayne@68: which will make the compression worse with these files.
jpayne@68: .IP \(bu 3
jpayne@68: If a BCJ filter is applied on an archive,
jpayne@68: it is possible that it makes the compression ratio
jpayne@68: worse than not using a BCJ filter.
jpayne@68: For example, if there are similar or even identical executables
jpayne@68: then filtering will likely make the files less similar
jpayne@68: and thus compression is worse.
jpayne@68: The contents of non-executable files in the same archive can matter too.
jpayne@68: In practice one has to try with and without a BCJ filter to see
jpayne@68: which is better in each situation.
jpayne@68: .RE
jpayne@68: .IP ""
jpayne@68: Different instruction sets have different alignment:
jpayne@68: the executable file must be aligned to a multiple of
jpayne@68: this value in the input data to make the filter work.
jpayne@68: .RS
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .TS
jpayne@68: tab(;);
jpayne@68: l n l
jpayne@68: l n l.
jpayne@68: Filter;Alignment;Notes
jpayne@68: x86;1;32-bit or 64-bit x86
jpayne@68: ARM;4;
jpayne@68: ARM-Thumb;2;
jpayne@68: ARM64;4;4096-byte alignment is best
jpayne@68: PowerPC;4;Big endian only
jpayne@68: IA-64;16;Itanium
jpayne@68: SPARC;4;
jpayne@68: RISC-V;2;
jpayne@68: .TE
jpayne@68: .RE
jpayne@68: .RE
jpayne@68: .IP ""
jpayne@68: Since the BCJ-filtered data is usually compressed with LZMA2,
jpayne@68: the compression ratio may be improved slightly if
jpayne@68: the LZMA2 options are set to match the
jpayne@68: alignment of the selected BCJ filter.
jpayne@68: Examples:
jpayne@68: .RS
jpayne@68: .IP \(bu 3
jpayne@68: IA-64 filter has 16-byte alignment so
jpayne@68: .B pb=4,lp=4,lc=0
jpayne@68: is good
jpayne@68: with LZMA2 (2^4=16).
jpayne@68: .IP \(bu 3
jpayne@68: RISC-V code has 2-byte or 4-byte alignment
jpayne@68: depending on whether the file contains
jpayne@68: 16-bit compressed instructions (the C extension).
jpayne@68: When 16-bit instructions are used,
jpayne@68: .B pb=2,lp=1,lc=3
jpayne@68: or
jpayne@68: .B pb=1,lp=1,lc=3
jpayne@68: is good.
jpayne@68: When 16-bit instructions aren't present,
jpayne@68: .B pb=2,lp=2,lc=2
jpayne@68: is the best.
jpayne@68: .B readelf \-h
jpayne@68: can be used to check if "RVC"
jpayne@68: appears on the "Flags" line.
jpayne@68: .IP \(bu 3
jpayne@68: ARM64 is always 4-byte aligned so
jpayne@68: .B pb=2,lp=2,lc=2
jpayne@68: is the best.
jpayne@68: .IP \(bu 3
jpayne@68: The x86 filter is an exception.
jpayne@68: It's usually good to stick to LZMA2's defaults
jpayne@68: .RB ( pb=2,lp=0,lc=3 )
jpayne@68: when compressing x86 executables.
jpayne@68: .RE
jpayne@68: .IP ""
jpayne@68: All BCJ filters support the same
jpayne@68: .IR options :
jpayne@68: .RS
jpayne@68: .TP
jpayne@68: .BI start= offset
jpayne@68: Specify the start
jpayne@68: .I offset
jpayne@68: that is used when converting between relative
jpayne@68: and absolute addresses.
jpayne@68: The
jpayne@68: .I offset
jpayne@68: must be a multiple of the alignment of the filter
jpayne@68: (see the table above).
jpayne@68: The default is zero.
jpayne@68: In practice, the default is good; specifying a custom
jpayne@68: .I offset
jpayne@68: is almost never useful.
jpayne@68: .RE
jpayne@68: .TP
jpayne@68: \fB\-\-delta\fR[\fB=\fIoptions\fR]
jpayne@68: Add the Delta filter to the filter chain.
jpayne@68: The Delta filter can be only used as a non-last filter
jpayne@68: in the filter chain.
jpayne@68: .IP ""
jpayne@68: Currently only simple byte-wise delta calculation is supported.
jpayne@68: It can be useful when compressing, for example, uncompressed bitmap images
jpayne@68: or uncompressed PCM audio.
jpayne@68: However, special purpose algorithms may give significantly better
jpayne@68: results than Delta + LZMA2.
jpayne@68: This is true especially with audio,
jpayne@68: which compresses faster and better, for example, with
jpayne@68: .BR flac (1).
jpayne@68: .IP ""
jpayne@68: Supported
jpayne@68: .IR options :
jpayne@68: .RS
jpayne@68: .TP
jpayne@68: .BI dist= distance
jpayne@68: Specify the
jpayne@68: .I distance
jpayne@68: of the delta calculation in bytes.
jpayne@68: .I distance
jpayne@68: must be 1\(en256.
jpayne@68: The default is 1.
jpayne@68: .IP ""
jpayne@68: For example, with
jpayne@68: .B dist=2
jpayne@68: and eight-byte input A1 B1 A2 B3 A3 B5 A4 B7, the output will be
jpayne@68: A1 B1 01 02 01 02 01 02.
jpayne@68: .RE
jpayne@68: .
jpayne@68: .SS "Other options"
jpayne@68: .TP
jpayne@68: .BR \-q ", " \-\-quiet
jpayne@68: Suppress warnings and notices.
jpayne@68: Specify this twice to suppress errors too.
jpayne@68: This option has no effect on the exit status.
jpayne@68: That is, even if a warning was suppressed,
jpayne@68: the exit status to indicate a warning is still used.
jpayne@68: .TP
jpayne@68: .BR \-v ", " \-\-verbose
jpayne@68: Be verbose.
jpayne@68: If standard error is connected to a terminal,
jpayne@68: .B xz
jpayne@68: will display a progress indicator.
jpayne@68: Specifying
jpayne@68: .B \-\-verbose
jpayne@68: twice will give even more verbose output.
jpayne@68: .IP ""
jpayne@68: The progress indicator shows the following information:
jpayne@68: .RS
jpayne@68: .IP \(bu 3
jpayne@68: Completion percentage is shown
jpayne@68: if the size of the input file is known.
jpayne@68: That is, the percentage cannot be shown in pipes.
jpayne@68: .IP \(bu 3
jpayne@68: Amount of compressed data produced (compressing)
jpayne@68: or consumed (decompressing).
jpayne@68: .IP \(bu 3
jpayne@68: Amount of uncompressed data consumed (compressing)
jpayne@68: or produced (decompressing).
jpayne@68: .IP \(bu 3
jpayne@68: Compression ratio, which is calculated by dividing
jpayne@68: the amount of compressed data processed so far by
jpayne@68: the amount of uncompressed data processed so far.
jpayne@68: .IP \(bu 3
jpayne@68: Compression or decompression speed.
jpayne@68: This is measured as the amount of uncompressed data consumed
jpayne@68: (compression) or produced (decompression) per second.
jpayne@68: It is shown after a few seconds have passed since
jpayne@68: .B xz
jpayne@68: started processing the file.
jpayne@68: .IP \(bu 3
jpayne@68: Elapsed time in the format M:SS or H:MM:SS.
jpayne@68: .IP \(bu 3
jpayne@68: Estimated remaining time is shown
jpayne@68: only when the size of the input file is
jpayne@68: known and a couple of seconds have already passed since
jpayne@68: .B xz
jpayne@68: started processing the file.
jpayne@68: The time is shown in a less precise format which
jpayne@68: never has any colons, for example, 2 min 30 s.
jpayne@68: .RE
jpayne@68: .IP ""
jpayne@68: When standard error is not a terminal,
jpayne@68: .B \-\-verbose
jpayne@68: will make
jpayne@68: .B xz
jpayne@68: print the filename, compressed size, uncompressed size,
jpayne@68: compression ratio, and possibly also the speed and elapsed time
jpayne@68: on a single line to standard error after compressing or
jpayne@68: decompressing the file.
jpayne@68: The speed and elapsed time are included only when
jpayne@68: the operation took at least a few seconds.
jpayne@68: If the operation didn't finish, for example, due to user interruption,
jpayne@68: also the completion percentage is printed
jpayne@68: if the size of the input file is known.
jpayne@68: .TP
jpayne@68: .BR \-Q ", " \-\-no\-warn
jpayne@68: Don't set the exit status to 2
jpayne@68: even if a condition worth a warning was detected.
jpayne@68: This option doesn't affect the verbosity level, thus both
jpayne@68: .B \-\-quiet
jpayne@68: and
jpayne@68: .B \-\-no\-warn
jpayne@68: have to be used to not display warnings and
jpayne@68: to not alter the exit status.
jpayne@68: .TP
jpayne@68: .B \-\-robot
jpayne@68: Print messages in a machine-parsable format.
jpayne@68: This is intended to ease writing frontends that want to use
jpayne@68: .B xz
jpayne@68: instead of liblzma, which may be the case with various scripts.
jpayne@68: The output with this option enabled is meant to be stable across
jpayne@68: .B xz
jpayne@68: releases.
jpayne@68: See the section
jpayne@68: .B "ROBOT MODE"
jpayne@68: for details.
jpayne@68: .TP
jpayne@68: .B \-\-info\-memory
jpayne@68: Display, in human-readable format, how much physical memory (RAM)
jpayne@68: and how many processor threads
jpayne@68: .B xz
jpayne@68: thinks the system has and the memory usage limits for compression
jpayne@68: and decompression, and exit successfully.
jpayne@68: .TP
jpayne@68: .BR \-h ", " \-\-help
jpayne@68: Display a help message describing the most commonly used options,
jpayne@68: and exit successfully.
jpayne@68: .TP
jpayne@68: .BR \-H ", " \-\-long\-help
jpayne@68: Display a help message describing all features of
jpayne@68: .BR xz ,
jpayne@68: and exit successfully
jpayne@68: .TP
jpayne@68: .BR \-V ", " \-\-version
jpayne@68: Display the version number of
jpayne@68: .B xz
jpayne@68: and liblzma in human readable format.
jpayne@68: To get machine-parsable output, specify
jpayne@68: .B \-\-robot
jpayne@68: before
jpayne@68: .BR \-\-version .
jpayne@68: .
jpayne@68: .SH "ROBOT MODE"
jpayne@68: The robot mode is activated with the
jpayne@68: .B \-\-robot
jpayne@68: option.
jpayne@68: It makes the output of
jpayne@68: .B xz
jpayne@68: easier to parse by other programs.
jpayne@68: Currently
jpayne@68: .B \-\-robot
jpayne@68: is supported only together with
jpayne@68: .BR \-\-list ,
jpayne@68: .BR \-\-filters\-help ,
jpayne@68: .BR \-\-info\-memory ,
jpayne@68: and
jpayne@68: .BR \-\-version .
jpayne@68: It will be supported for compression and
jpayne@68: decompression in the future.
jpayne@68: .
jpayne@68: .SS "List mode"
jpayne@68: .B "xz \-\-robot \-\-list"
jpayne@68: uses tab-separated output.
jpayne@68: The first column of every line has a string
jpayne@68: that indicates the type of the information found on that line:
jpayne@68: .TP
jpayne@68: .B name
jpayne@68: This is always the first line when starting to list a file.
jpayne@68: The second column on the line is the filename.
jpayne@68: .TP
jpayne@68: .B file
jpayne@68: This line contains overall information about the
jpayne@68: .B .xz
jpayne@68: file.
jpayne@68: This line is always printed after the
jpayne@68: .B name
jpayne@68: line.
jpayne@68: .TP
jpayne@68: .B stream
jpayne@68: This line type is used only when
jpayne@68: .B \-\-verbose
jpayne@68: was specified.
jpayne@68: There are as many
jpayne@68: .B stream
jpayne@68: lines as there are streams in the
jpayne@68: .B .xz
jpayne@68: file.
jpayne@68: .TP
jpayne@68: .B block
jpayne@68: This line type is used only when
jpayne@68: .B \-\-verbose
jpayne@68: was specified.
jpayne@68: There are as many
jpayne@68: .B block
jpayne@68: lines as there are blocks in the
jpayne@68: .B .xz
jpayne@68: file.
jpayne@68: The
jpayne@68: .B block
jpayne@68: lines are shown after all the
jpayne@68: .B stream
jpayne@68: lines; different line types are not interleaved.
jpayne@68: .TP
jpayne@68: .B summary
jpayne@68: This line type is used only when
jpayne@68: .B \-\-verbose
jpayne@68: was specified twice.
jpayne@68: This line is printed after all
jpayne@68: .B block
jpayne@68: lines.
jpayne@68: Like the
jpayne@68: .B file
jpayne@68: line, the
jpayne@68: .B summary
jpayne@68: line contains overall information about the
jpayne@68: .B .xz
jpayne@68: file.
jpayne@68: .TP
jpayne@68: .B totals
jpayne@68: This line is always the very last line of the list output.
jpayne@68: It shows the total counts and sizes.
jpayne@68: .PP
jpayne@68: The columns of the
jpayne@68: .B file
jpayne@68: lines:
jpayne@68: .PD 0
jpayne@68: .RS
jpayne@68: .IP 2. 4
jpayne@68: Number of streams in the file
jpayne@68: .IP 3. 4
jpayne@68: Total number of blocks in the stream(s)
jpayne@68: .IP 4. 4
jpayne@68: Compressed size of the file
jpayne@68: .IP 5. 4
jpayne@68: Uncompressed size of the file
jpayne@68: .IP 6. 4
jpayne@68: Compression ratio, for example,
jpayne@68: .BR 0.123 .
jpayne@68: If ratio is over 9.999, three dashes
jpayne@68: .RB ( \-\-\- )
jpayne@68: are displayed instead of the ratio.
jpayne@68: .IP 7. 4
jpayne@68: Comma-separated list of integrity check names.
jpayne@68: The following strings are used for the known check types:
jpayne@68: .BR None ,
jpayne@68: .BR CRC32 ,
jpayne@68: .BR CRC64 ,
jpayne@68: and
jpayne@68: .BR SHA\-256 .
jpayne@68: For unknown check types,
jpayne@68: .BI Unknown\- N
jpayne@68: is used, where
jpayne@68: .I N
jpayne@68: is the Check ID as a decimal number (one or two digits).
jpayne@68: .IP 8. 4
jpayne@68: Total size of stream padding in the file
jpayne@68: .RE
jpayne@68: .PD
jpayne@68: .PP
jpayne@68: The columns of the
jpayne@68: .B stream
jpayne@68: lines:
jpayne@68: .PD 0
jpayne@68: .RS
jpayne@68: .IP 2. 4
jpayne@68: Stream number (the first stream is 1)
jpayne@68: .IP 3. 4
jpayne@68: Number of blocks in the stream
jpayne@68: .IP 4. 4
jpayne@68: Compressed start offset
jpayne@68: .IP 5. 4
jpayne@68: Uncompressed start offset
jpayne@68: .IP 6. 4
jpayne@68: Compressed size (does not include stream padding)
jpayne@68: .IP 7. 4
jpayne@68: Uncompressed size
jpayne@68: .IP 8. 4
jpayne@68: Compression ratio
jpayne@68: .IP 9. 4
jpayne@68: Name of the integrity check
jpayne@68: .IP 10. 4
jpayne@68: Size of stream padding
jpayne@68: .RE
jpayne@68: .PD
jpayne@68: .PP
jpayne@68: The columns of the
jpayne@68: .B block
jpayne@68: lines:
jpayne@68: .PD 0
jpayne@68: .RS
jpayne@68: .IP 2. 4
jpayne@68: Number of the stream containing this block
jpayne@68: .IP 3. 4
jpayne@68: Block number relative to the beginning of the stream
jpayne@68: (the first block is 1)
jpayne@68: .IP 4. 4
jpayne@68: Block number relative to the beginning of the file
jpayne@68: .IP 5. 4
jpayne@68: Compressed start offset relative to the beginning of the file
jpayne@68: .IP 6. 4
jpayne@68: Uncompressed start offset relative to the beginning of the file
jpayne@68: .IP 7. 4
jpayne@68: Total compressed size of the block (includes headers)
jpayne@68: .IP 8. 4
jpayne@68: Uncompressed size
jpayne@68: .IP 9. 4
jpayne@68: Compression ratio
jpayne@68: .IP 10. 4
jpayne@68: Name of the integrity check
jpayne@68: .RE
jpayne@68: .PD
jpayne@68: .PP
jpayne@68: If
jpayne@68: .B \-\-verbose
jpayne@68: was specified twice, additional columns are included on the
jpayne@68: .B block
jpayne@68: lines.
jpayne@68: These are not displayed with a single
jpayne@68: .BR \-\-verbose ,
jpayne@68: because getting this information requires many seeks
jpayne@68: and can thus be slow:
jpayne@68: .PD 0
jpayne@68: .RS
jpayne@68: .IP 11. 4
jpayne@68: Value of the integrity check in hexadecimal
jpayne@68: .IP 12. 4
jpayne@68: Block header size
jpayne@68: .IP 13. 4
jpayne@68: Block flags:
jpayne@68: .B c
jpayne@68: indicates that compressed size is present, and
jpayne@68: .B u
jpayne@68: indicates that uncompressed size is present.
jpayne@68: If the flag is not set, a dash
jpayne@68: .RB ( \- )
jpayne@68: is shown instead to keep the string length fixed.
jpayne@68: New flags may be added to the end of the string in the future.
jpayne@68: .IP 14. 4
jpayne@68: Size of the actual compressed data in the block (this excludes
jpayne@68: the block header, block padding, and check fields)
jpayne@68: .IP 15. 4
jpayne@68: Amount of memory (in bytes) required to decompress
jpayne@68: this block with this
jpayne@68: .B xz
jpayne@68: version
jpayne@68: .IP 16. 4
jpayne@68: Filter chain.
jpayne@68: Note that most of the options used at compression time
jpayne@68: cannot be known, because only the options
jpayne@68: that are needed for decompression are stored in the
jpayne@68: .B .xz
jpayne@68: headers.
jpayne@68: .RE
jpayne@68: .PD
jpayne@68: .PP
jpayne@68: The columns of the
jpayne@68: .B summary
jpayne@68: lines:
jpayne@68: .PD 0
jpayne@68: .RS
jpayne@68: .IP 2. 4
jpayne@68: Amount of memory (in bytes) required to decompress
jpayne@68: this file with this
jpayne@68: .B xz
jpayne@68: version
jpayne@68: .IP 3. 4
jpayne@68: .B yes
jpayne@68: or
jpayne@68: .B no
jpayne@68: indicating if all block headers have both compressed size and
jpayne@68: uncompressed size stored in them
jpayne@68: .PP
jpayne@68: .I Since
jpayne@68: .B xz
jpayne@68: .I 5.1.2alpha:
jpayne@68: .IP 4. 4
jpayne@68: Minimum
jpayne@68: .B xz
jpayne@68: version required to decompress the file
jpayne@68: .RE
jpayne@68: .PD
jpayne@68: .PP
jpayne@68: The columns of the
jpayne@68: .B totals
jpayne@68: line:
jpayne@68: .PD 0
jpayne@68: .RS
jpayne@68: .IP 2. 4
jpayne@68: Number of streams
jpayne@68: .IP 3. 4
jpayne@68: Number of blocks
jpayne@68: .IP 4. 4
jpayne@68: Compressed size
jpayne@68: .IP 5. 4
jpayne@68: Uncompressed size
jpayne@68: .IP 6. 4
jpayne@68: Average compression ratio
jpayne@68: .IP 7. 4
jpayne@68: Comma-separated list of integrity check names
jpayne@68: that were present in the files
jpayne@68: .IP 8. 4
jpayne@68: Stream padding size
jpayne@68: .IP 9. 4
jpayne@68: Number of files.
jpayne@68: This is here to
jpayne@68: keep the order of the earlier columns the same as on
jpayne@68: .B file
jpayne@68: lines.
jpayne@68: .PD
jpayne@68: .RE
jpayne@68: .PP
jpayne@68: If
jpayne@68: .B \-\-verbose
jpayne@68: was specified twice, additional columns are included on the
jpayne@68: .B totals
jpayne@68: line:
jpayne@68: .PD 0
jpayne@68: .RS
jpayne@68: .IP 10. 4
jpayne@68: Maximum amount of memory (in bytes) required to decompress
jpayne@68: the files with this
jpayne@68: .B xz
jpayne@68: version
jpayne@68: .IP 11. 4
jpayne@68: .B yes
jpayne@68: or
jpayne@68: .B no
jpayne@68: indicating if all block headers have both compressed size and
jpayne@68: uncompressed size stored in them
jpayne@68: .PP
jpayne@68: .I Since
jpayne@68: .B xz
jpayne@68: .I 5.1.2alpha:
jpayne@68: .IP 12. 4
jpayne@68: Minimum
jpayne@68: .B xz
jpayne@68: version required to decompress the file
jpayne@68: .RE
jpayne@68: .PD
jpayne@68: .PP
jpayne@68: Future versions may add new line types and
jpayne@68: new columns can be added to the existing line types,
jpayne@68: but the existing columns won't be changed.
jpayne@68: .
jpayne@68: .SS "Filters help"
jpayne@68: .B "xz \-\-robot \-\-filters-help"
jpayne@68: prints the supported filters in the following format:
jpayne@68: .PP
jpayne@68: \fIfilter\fB:\fIoption\fB=<\fIvalue\fB>,\fIoption\fB=<\fIvalue\fB>\fR...
jpayne@68: .TP
jpayne@68: .I filter
jpayne@68: Name of the filter
jpayne@68: .TP
jpayne@68: .I option
jpayne@68: Name of a filter specific option
jpayne@68: .TP
jpayne@68: .I value
jpayne@68: Numeric
jpayne@68: .I value
jpayne@68: ranges appear as
jpayne@68: \fB<\fImin\fB\-\fImax\fB>\fR.
jpayne@68: String
jpayne@68: .I value
jpayne@68: choices are shown within
jpayne@68: .B "< >"
jpayne@68: and separated by a
jpayne@68: .B |
jpayne@68: character.
jpayne@68: .PP
jpayne@68: Each filter is printed on its own line.
jpayne@68: .
jpayne@68: .SS "Memory limit information"
jpayne@68: .B "xz \-\-robot \-\-info\-memory"
jpayne@68: prints a single line with multiple tab-separated columns:
jpayne@68: .IP 1. 4
jpayne@68: Total amount of physical memory (RAM) in bytes.
jpayne@68: .IP 2. 4
jpayne@68: Memory usage limit for compression in bytes
jpayne@68: .RB ( \-\-memlimit\-compress ).
jpayne@68: A special value of
jpayne@68: .B 0
jpayne@68: indicates the default setting
jpayne@68: which for single-threaded mode is the same as no limit.
jpayne@68: .IP 3. 4
jpayne@68: Memory usage limit for decompression in bytes
jpayne@68: .RB ( \-\-memlimit\-decompress ).
jpayne@68: A special value of
jpayne@68: .B 0
jpayne@68: indicates the default setting
jpayne@68: which for single-threaded mode is the same as no limit.
jpayne@68: .IP 4. 4
jpayne@68: Since
jpayne@68: .B xz
jpayne@68: 5.3.4alpha:
jpayne@68: Memory usage for multi-threaded decompression in bytes
jpayne@68: .RB ( \-\-memlimit\-mt\-decompress ).
jpayne@68: This is never zero because a system-specific default value
jpayne@68: shown in the column 5
jpayne@68: is used if no limit has been specified explicitly.
jpayne@68: This is also never greater than the value in the column 3
jpayne@68: even if a larger value has been specified with
jpayne@68: .BR \-\-memlimit\-mt\-decompress .
jpayne@68: .IP 5. 4
jpayne@68: Since
jpayne@68: .B xz
jpayne@68: 5.3.4alpha:
jpayne@68: A system-specific default memory usage limit
jpayne@68: that is used to limit the number of threads
jpayne@68: when compressing with an automatic number of threads
jpayne@68: .RB ( \-\-threads=0 )
jpayne@68: and no memory usage limit has been specified
jpayne@68: .RB ( \-\-memlimit\-compress ).
jpayne@68: This is also used as the default value for
jpayne@68: .BR \-\-memlimit\-mt\-decompress .
jpayne@68: .IP 6. 4
jpayne@68: Since
jpayne@68: .B xz
jpayne@68: 5.3.4alpha:
jpayne@68: Number of available processor threads.
jpayne@68: .PP
jpayne@68: In the future, the output of
jpayne@68: .B "xz \-\-robot \-\-info\-memory"
jpayne@68: may have more columns, but never more than a single line.
jpayne@68: .
jpayne@68: .SS Version
jpayne@68: .B "xz \-\-robot \-\-version"
jpayne@68: prints the version number of
jpayne@68: .B xz
jpayne@68: and liblzma in the following format:
jpayne@68: .PP
jpayne@68: .BI XZ_VERSION= XYYYZZZS
jpayne@68: .br
jpayne@68: .BI LIBLZMA_VERSION= XYYYZZZS
jpayne@68: .TP
jpayne@68: .I X
jpayne@68: Major version.
jpayne@68: .TP
jpayne@68: .I YYY
jpayne@68: Minor version.
jpayne@68: Even numbers are stable.
jpayne@68: Odd numbers are alpha or beta versions.
jpayne@68: .TP
jpayne@68: .I ZZZ
jpayne@68: Patch level for stable releases or
jpayne@68: just a counter for development releases.
jpayne@68: .TP
jpayne@68: .I S
jpayne@68: Stability.
jpayne@68: 0 is alpha, 1 is beta, and 2 is stable.
jpayne@68: .I S
jpayne@68: should be always 2 when
jpayne@68: .I YYY
jpayne@68: is even.
jpayne@68: .PP
jpayne@68: .I XYYYZZZS
jpayne@68: are the same on both lines if
jpayne@68: .B xz
jpayne@68: and liblzma are from the same XZ Utils release.
jpayne@68: .PP
jpayne@68: Examples: 4.999.9beta is
jpayne@68: .B 49990091
jpayne@68: and
jpayne@68: 5.0.0 is
jpayne@68: .BR 50000002 .
jpayne@68: .
jpayne@68: .SH "EXIT STATUS"
jpayne@68: .TP
jpayne@68: .B 0
jpayne@68: All is good.
jpayne@68: .TP
jpayne@68: .B 1
jpayne@68: An error occurred.
jpayne@68: .TP
jpayne@68: .B 2
jpayne@68: Something worth a warning occurred,
jpayne@68: but no actual errors occurred.
jpayne@68: .PP
jpayne@68: Notices (not warnings or errors) printed on standard error
jpayne@68: don't affect the exit status.
jpayne@68: .
jpayne@68: .SH ENVIRONMENT
jpayne@68: .B xz
jpayne@68: parses space-separated lists of options
jpayne@68: from the environment variables
jpayne@68: .B XZ_DEFAULTS
jpayne@68: and
jpayne@68: .BR XZ_OPT ,
jpayne@68: in this order, before parsing the options from the command line.
jpayne@68: Note that only options are parsed from the environment variables;
jpayne@68: all non-options are silently ignored.
jpayne@68: Parsing is done with
jpayne@68: .BR getopt_long (3)
jpayne@68: which is used also for the command line arguments.
jpayne@68: .TP
jpayne@68: .B XZ_DEFAULTS
jpayne@68: User-specific or system-wide default options.
jpayne@68: Typically this is set in a shell initialization script to enable
jpayne@68: .BR xz 's
jpayne@68: memory usage limiter by default.
jpayne@68: Excluding shell initialization scripts
jpayne@68: and similar special cases, scripts must never set or unset
jpayne@68: .BR XZ_DEFAULTS .
jpayne@68: .TP
jpayne@68: .B XZ_OPT
jpayne@68: This is for passing options to
jpayne@68: .B xz
jpayne@68: when it is not possible to set the options directly on the
jpayne@68: .B xz
jpayne@68: command line.
jpayne@68: This is the case when
jpayne@68: .B xz
jpayne@68: is run by a script or tool, for example, GNU
jpayne@68: .BR tar (1):
jpayne@68: .RS
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .nf
jpayne@68: .ft CR
jpayne@68: XZ_OPT=\-2v tar caf foo.tar.xz foo
jpayne@68: .ft R
jpayne@68: .fi
jpayne@68: .RE
jpayne@68: .RE
jpayne@68: .IP ""
jpayne@68: Scripts may use
jpayne@68: .BR XZ_OPT ,
jpayne@68: for example, to set script-specific default compression options.
jpayne@68: It is still recommended to allow users to override
jpayne@68: .B XZ_OPT
jpayne@68: if that is reasonable.
jpayne@68: For example, in
jpayne@68: .BR sh (1)
jpayne@68: scripts one may use something like this:
jpayne@68: .RS
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .nf
jpayne@68: .ft CR
jpayne@68: XZ_OPT=${XZ_OPT\-"\-7e"}
jpayne@68: export XZ_OPT
jpayne@68: .ft R
jpayne@68: .fi
jpayne@68: .RE
jpayne@68: .RE
jpayne@68: .
jpayne@68: .SH "LZMA UTILS COMPATIBILITY"
jpayne@68: The command line syntax of
jpayne@68: .B xz
jpayne@68: is practically a superset of
jpayne@68: .BR lzma ,
jpayne@68: .BR unlzma ,
jpayne@68: and
jpayne@68: .B lzcat
jpayne@68: as found from LZMA Utils 4.32.x.
jpayne@68: In most cases, it is possible to replace
jpayne@68: LZMA Utils with XZ Utils without breaking existing scripts.
jpayne@68: There are some incompatibilities though,
jpayne@68: which may sometimes cause problems.
jpayne@68: .
jpayne@68: .SS "Compression preset levels"
jpayne@68: The numbering of the compression level presets is not identical in
jpayne@68: .B xz
jpayne@68: and LZMA Utils.
jpayne@68: The most important difference is how dictionary sizes
jpayne@68: are mapped to different presets.
jpayne@68: Dictionary size is roughly equal to the decompressor memory usage.
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .TS
jpayne@68: tab(;);
jpayne@68: c c c
jpayne@68: c n n.
jpayne@68: Level;xz;LZMA Utils
jpayne@68: \-0;256 KiB;N/A
jpayne@68: \-1;1 MiB;64 KiB
jpayne@68: \-2;2 MiB;1 MiB
jpayne@68: \-3;4 MiB;512 KiB
jpayne@68: \-4;4 MiB;1 MiB
jpayne@68: \-5;8 MiB;2 MiB
jpayne@68: \-6;8 MiB;4 MiB
jpayne@68: \-7;16 MiB;8 MiB
jpayne@68: \-8;32 MiB;16 MiB
jpayne@68: \-9;64 MiB;32 MiB
jpayne@68: .TE
jpayne@68: .RE
jpayne@68: .PP
jpayne@68: The dictionary size differences affect
jpayne@68: the compressor memory usage too,
jpayne@68: but there are some other differences between
jpayne@68: LZMA Utils and XZ Utils, which
jpayne@68: make the difference even bigger:
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .TS
jpayne@68: tab(;);
jpayne@68: c c c
jpayne@68: c n n.
jpayne@68: Level;xz;LZMA Utils 4.32.x
jpayne@68: \-0;3 MiB;N/A
jpayne@68: \-1;9 MiB;2 MiB
jpayne@68: \-2;17 MiB;12 MiB
jpayne@68: \-3;32 MiB;12 MiB
jpayne@68: \-4;48 MiB;16 MiB
jpayne@68: \-5;94 MiB;26 MiB
jpayne@68: \-6;94 MiB;45 MiB
jpayne@68: \-7;186 MiB;83 MiB
jpayne@68: \-8;370 MiB;159 MiB
jpayne@68: \-9;674 MiB;311 MiB
jpayne@68: .TE
jpayne@68: .RE
jpayne@68: .PP
jpayne@68: The default preset level in LZMA Utils is
jpayne@68: .B \-7
jpayne@68: while in XZ Utils it is
jpayne@68: .BR \-6 ,
jpayne@68: so both use an 8 MiB dictionary by default.
jpayne@68: .
jpayne@68: .SS "Streamed vs. non-streamed .lzma files"
jpayne@68: The uncompressed size of the file can be stored in the
jpayne@68: .B .lzma
jpayne@68: header.
jpayne@68: LZMA Utils does that when compressing regular files.
jpayne@68: The alternative is to mark that uncompressed size is unknown
jpayne@68: and use end-of-payload marker to indicate
jpayne@68: where the decompressor should stop.
jpayne@68: LZMA Utils uses this method when uncompressed size isn't known,
jpayne@68: which is the case, for example, in pipes.
jpayne@68: .PP
jpayne@68: .B xz
jpayne@68: supports decompressing
jpayne@68: .B .lzma
jpayne@68: files with or without end-of-payload marker, but all
jpayne@68: .B .lzma
jpayne@68: files created by
jpayne@68: .B xz
jpayne@68: will use end-of-payload marker and have uncompressed size
jpayne@68: marked as unknown in the
jpayne@68: .B .lzma
jpayne@68: header.
jpayne@68: This may be a problem in some uncommon situations.
jpayne@68: For example, a
jpayne@68: .B .lzma
jpayne@68: decompressor in an embedded device might work
jpayne@68: only with files that have known uncompressed size.
jpayne@68: If you hit this problem, you need to use LZMA Utils
jpayne@68: or LZMA SDK to create
jpayne@68: .B .lzma
jpayne@68: files with known uncompressed size.
jpayne@68: .
jpayne@68: .SS "Unsupported .lzma files"
jpayne@68: The
jpayne@68: .B .lzma
jpayne@68: format allows
jpayne@68: .I lc
jpayne@68: values up to 8, and
jpayne@68: .I lp
jpayne@68: values up to 4.
jpayne@68: LZMA Utils can decompress files with any
jpayne@68: .I lc
jpayne@68: and
jpayne@68: .IR lp ,
jpayne@68: but always creates files with
jpayne@68: .B lc=3
jpayne@68: and
jpayne@68: .BR lp=0 .
jpayne@68: Creating files with other
jpayne@68: .I lc
jpayne@68: and
jpayne@68: .I lp
jpayne@68: is possible with
jpayne@68: .B xz
jpayne@68: and with LZMA SDK.
jpayne@68: .PP
jpayne@68: The implementation of the LZMA1 filter in liblzma
jpayne@68: requires that the sum of
jpayne@68: .I lc
jpayne@68: and
jpayne@68: .I lp
jpayne@68: must not exceed 4.
jpayne@68: Thus,
jpayne@68: .B .lzma
jpayne@68: files, which exceed this limitation, cannot be decompressed with
jpayne@68: .BR xz .
jpayne@68: .PP
jpayne@68: LZMA Utils creates only
jpayne@68: .B .lzma
jpayne@68: files which have a dictionary size of
jpayne@68: .RI "2^" n
jpayne@68: (a power of 2) but accepts files with any dictionary size.
jpayne@68: liblzma accepts only
jpayne@68: .B .lzma
jpayne@68: files which have a dictionary size of
jpayne@68: .RI "2^" n
jpayne@68: or
jpayne@68: .RI "2^" n " + 2^(" n "\-1)."
jpayne@68: This is to decrease false positives when detecting
jpayne@68: .B .lzma
jpayne@68: files.
jpayne@68: .PP
jpayne@68: These limitations shouldn't be a problem in practice,
jpayne@68: since practically all
jpayne@68: .B .lzma
jpayne@68: files have been compressed with settings that liblzma will accept.
jpayne@68: .
jpayne@68: .SS "Trailing garbage"
jpayne@68: When decompressing,
jpayne@68: LZMA Utils silently ignore everything after the first
jpayne@68: .B .lzma
jpayne@68: stream.
jpayne@68: In most situations, this is a bug.
jpayne@68: This also means that LZMA Utils
jpayne@68: don't support decompressing concatenated
jpayne@68: .B .lzma
jpayne@68: files.
jpayne@68: .PP
jpayne@68: If there is data left after the first
jpayne@68: .B .lzma
jpayne@68: stream,
jpayne@68: .B xz
jpayne@68: considers the file to be corrupt unless
jpayne@68: .B \-\-single\-stream
jpayne@68: was used.
jpayne@68: This may break obscure scripts which have
jpayne@68: assumed that trailing garbage is ignored.
jpayne@68: .
jpayne@68: .SH NOTES
jpayne@68: .
jpayne@68: .SS "Compressed output may vary"
jpayne@68: The exact compressed output produced from
jpayne@68: the same uncompressed input file
jpayne@68: may vary between XZ Utils versions even if
jpayne@68: compression options are identical.
jpayne@68: This is because the encoder can be improved
jpayne@68: (faster or better compression)
jpayne@68: without affecting the file format.
jpayne@68: The output can vary even between different
jpayne@68: builds of the same XZ Utils version,
jpayne@68: if different build options are used.
jpayne@68: .PP
jpayne@68: The above means that once
jpayne@68: .B \-\-rsyncable
jpayne@68: has been implemented,
jpayne@68: the resulting files won't necessarily be rsyncable
jpayne@68: unless both old and new files have been compressed
jpayne@68: with the same xz version.
jpayne@68: This problem can be fixed if a part of the encoder
jpayne@68: implementation is frozen to keep rsyncable output
jpayne@68: stable across xz versions.
jpayne@68: .
jpayne@68: .SS "Embedded .xz decompressors"
jpayne@68: Embedded
jpayne@68: .B .xz
jpayne@68: decompressor implementations like XZ Embedded don't necessarily
jpayne@68: support files created with integrity
jpayne@68: .I check
jpayne@68: types other than
jpayne@68: .B none
jpayne@68: and
jpayne@68: .BR crc32 .
jpayne@68: Since the default is
jpayne@68: .BR \-\-check=crc64 ,
jpayne@68: you must use
jpayne@68: .B \-\-check=none
jpayne@68: or
jpayne@68: .B \-\-check=crc32
jpayne@68: when creating files for embedded systems.
jpayne@68: .PP
jpayne@68: Outside embedded systems, all
jpayne@68: .B .xz
jpayne@68: format decompressors support all the
jpayne@68: .I check
jpayne@68: types, or at least are able to decompress
jpayne@68: the file without verifying the
jpayne@68: integrity check if the particular
jpayne@68: .I check
jpayne@68: is not supported.
jpayne@68: .PP
jpayne@68: XZ Embedded supports BCJ filters,
jpayne@68: but only with the default start offset.
jpayne@68: .
jpayne@68: .SH EXAMPLES
jpayne@68: .
jpayne@68: .SS Basics
jpayne@68: Compress the file
jpayne@68: .I foo
jpayne@68: into
jpayne@68: .I foo.xz
jpayne@68: using the default compression level
jpayne@68: .RB ( \-6 ),
jpayne@68: and remove
jpayne@68: .I foo
jpayne@68: if compression is successful:
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .nf
jpayne@68: .ft CR
jpayne@68: xz foo
jpayne@68: .ft R
jpayne@68: .fi
jpayne@68: .RE
jpayne@68: .PP
jpayne@68: Decompress
jpayne@68: .I bar.xz
jpayne@68: into
jpayne@68: .I bar
jpayne@68: and don't remove
jpayne@68: .I bar.xz
jpayne@68: even if decompression is successful:
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .nf
jpayne@68: .ft CR
jpayne@68: xz \-dk bar.xz
jpayne@68: .ft R
jpayne@68: .fi
jpayne@68: .RE
jpayne@68: .PP
jpayne@68: Create
jpayne@68: .I baz.tar.xz
jpayne@68: with the preset
jpayne@68: .B \-4e
jpayne@68: .RB ( "\-4 \-\-extreme" ),
jpayne@68: which is slower than the default
jpayne@68: .BR \-6 ,
jpayne@68: but needs less memory for compression and decompression (48\ MiB
jpayne@68: and 5\ MiB, respectively):
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .nf
jpayne@68: .ft CR
jpayne@68: tar cf \- baz | xz \-4e > baz.tar.xz
jpayne@68: .ft R
jpayne@68: .fi
jpayne@68: .RE
jpayne@68: .PP
jpayne@68: A mix of compressed and uncompressed files can be decompressed
jpayne@68: to standard output with a single command:
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .nf
jpayne@68: .ft CR
jpayne@68: xz \-dcf a.txt b.txt.xz c.txt d.txt.lzma > abcd.txt
jpayne@68: .ft R
jpayne@68: .fi
jpayne@68: .RE
jpayne@68: .
jpayne@68: .SS "Parallel compression of many files"
jpayne@68: On GNU and *BSD,
jpayne@68: .BR find (1)
jpayne@68: and
jpayne@68: .BR xargs (1)
jpayne@68: can be used to parallelize compression of many files:
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .nf
jpayne@68: .ft CR
jpayne@68: find . \-type f \e! \-name '*.xz' \-print0 \e
jpayne@68:     | xargs \-0r \-P4 \-n16 xz \-T1
jpayne@68: .ft R
jpayne@68: .fi
jpayne@68: .RE
jpayne@68: .PP
jpayne@68: The
jpayne@68: .B \-P
jpayne@68: option to
jpayne@68: .BR xargs (1)
jpayne@68: sets the number of parallel
jpayne@68: .B xz
jpayne@68: processes.
jpayne@68: The best value for the
jpayne@68: .B \-n
jpayne@68: option depends on how many files there are to be compressed.
jpayne@68: If there are only a couple of files,
jpayne@68: the value should probably be 1;
jpayne@68: with tens of thousands of files,
jpayne@68: 100 or even more may be appropriate to reduce the number of
jpayne@68: .B xz
jpayne@68: processes that
jpayne@68: .BR xargs (1)
jpayne@68: will eventually create.
jpayne@68: .PP
jpayne@68: The option
jpayne@68: .B \-T1
jpayne@68: for
jpayne@68: .B xz
jpayne@68: is there to force it to single-threaded mode, because
jpayne@68: .BR xargs (1)
jpayne@68: is used to control the amount of parallelization.
jpayne@68: .
jpayne@68: .SS "Robot mode"
jpayne@68: Calculate how many bytes have been saved in total
jpayne@68: after compressing multiple files:
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .nf
jpayne@68: .ft CR
jpayne@68: xz \-\-robot \-\-list *.xz | awk '/^totals/{print $5\-$4}'
jpayne@68: .ft R
jpayne@68: .fi
jpayne@68: .RE
jpayne@68: .PP
jpayne@68: A script may want to know that it is using new enough
jpayne@68: .BR xz .
jpayne@68: The following
jpayne@68: .BR sh (1)
jpayne@68: script checks that the version number of the
jpayne@68: .B xz
jpayne@68: tool is at least 5.0.0.
jpayne@68: This method is compatible with old beta versions,
jpayne@68: which didn't support the
jpayne@68: .B \-\-robot
jpayne@68: option:
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .nf
jpayne@68: .ft CR
jpayne@68: if ! eval "$(xz \-\-robot \-\-version 2> /dev/null)" ||
jpayne@68:         [ "$XZ_VERSION" \-lt 50000002 ]; then
jpayne@68:     echo "Your xz is too old."
jpayne@68: fi
jpayne@68: unset XZ_VERSION LIBLZMA_VERSION
jpayne@68: .ft R
jpayne@68: .fi
jpayne@68: .RE
jpayne@68: .PP
jpayne@68: Set a memory usage limit for decompression using
jpayne@68: .BR XZ_OPT ,
jpayne@68: but if a limit has already been set, don't increase it:
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .nf
jpayne@68: .ft CR
jpayne@68: NEWLIM=$((123 << 20))\ \ # 123 MiB
jpayne@68: OLDLIM=$(xz \-\-robot \-\-info\-memory | cut \-f3)
jpayne@68: if [ $OLDLIM \-eq 0 \-o $OLDLIM \-gt $NEWLIM ]; then
jpayne@68:     XZ_OPT="$XZ_OPT \-\-memlimit\-decompress=$NEWLIM"
jpayne@68:     export XZ_OPT
jpayne@68: fi
jpayne@68: .ft R
jpayne@68: .fi
jpayne@68: .RE
jpayne@68: .
jpayne@68: .SS "Custom compressor filter chains"
jpayne@68: The simplest use for custom filter chains is
jpayne@68: customizing a LZMA2 preset.
jpayne@68: This can be useful,
jpayne@68: because the presets cover only a subset of the
jpayne@68: potentially useful combinations of compression settings.
jpayne@68: .PP
jpayne@68: The CompCPU columns of the tables
jpayne@68: from the descriptions of the options
jpayne@68: .BR "\-0" " ... " "\-9"
jpayne@68: and
jpayne@68: .B \-\-extreme
jpayne@68: are useful when customizing LZMA2 presets.
jpayne@68: Here are the relevant parts collected from those two tables:
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .TS
jpayne@68: tab(;);
jpayne@68: c c
jpayne@68: n n.
jpayne@68: Preset;CompCPU
jpayne@68: \-0;0
jpayne@68: \-1;1
jpayne@68: \-2;2
jpayne@68: \-3;3
jpayne@68: \-4;4
jpayne@68: \-5;5
jpayne@68: \-6;6
jpayne@68: \-5e;7
jpayne@68: \-6e;8
jpayne@68: .TE
jpayne@68: .RE
jpayne@68: .PP
jpayne@68: If you know that a file requires
jpayne@68: somewhat big dictionary (for example, 32\ MiB) to compress well,
jpayne@68: but you want to compress it quicker than
jpayne@68: .B "xz \-8"
jpayne@68: would do, a preset with a low CompCPU value (for example, 1)
jpayne@68: can be modified to use a bigger dictionary:
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .nf
jpayne@68: .ft CR
jpayne@68: xz \-\-lzma2=preset=1,dict=32MiB foo.tar
jpayne@68: .ft R
jpayne@68: .fi
jpayne@68: .RE
jpayne@68: .PP
jpayne@68: With certain files, the above command may be faster than
jpayne@68: .B "xz \-6"
jpayne@68: while compressing significantly better.
jpayne@68: However, it must be emphasized that only some files benefit from
jpayne@68: a big dictionary while keeping the CompCPU value low.
jpayne@68: The most obvious situation,
jpayne@68: where a big dictionary can help a lot,
jpayne@68: is an archive containing very similar files
jpayne@68: of at least a few megabytes each.
jpayne@68: The dictionary size has to be significantly bigger
jpayne@68: than any individual file to allow LZMA2 to take
jpayne@68: full advantage of the similarities between consecutive files.
jpayne@68: .PP
jpayne@68: If very high compressor and decompressor memory usage is fine,
jpayne@68: and the file being compressed is
jpayne@68: at least several hundred megabytes, it may be useful
jpayne@68: to use an even bigger dictionary than the 64 MiB that
jpayne@68: .B "xz \-9"
jpayne@68: would use:
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .nf
jpayne@68: .ft CR
jpayne@68: xz \-vv \-\-lzma2=dict=192MiB big_foo.tar
jpayne@68: .ft R
jpayne@68: .fi
jpayne@68: .RE
jpayne@68: .PP
jpayne@68: Using
jpayne@68: .B \-vv
jpayne@68: .RB ( "\-\-verbose \-\-verbose" )
jpayne@68: like in the above example can be useful
jpayne@68: to see the memory requirements
jpayne@68: of the compressor and decompressor.
jpayne@68: Remember that using a dictionary bigger than
jpayne@68: the size of the uncompressed file is waste of memory,
jpayne@68: so the above command isn't useful for small files.
jpayne@68: .PP
jpayne@68: Sometimes the compression time doesn't matter,
jpayne@68: but the decompressor memory usage has to be kept low, for example,
jpayne@68: to make it possible to decompress the file on an embedded system.
jpayne@68: The following command uses
jpayne@68: .B \-6e
jpayne@68: .RB ( "\-6 \-\-extreme" )
jpayne@68: as a base and sets the dictionary to only 64\ KiB.
jpayne@68: The resulting file can be decompressed with XZ Embedded
jpayne@68: (that's why there is
jpayne@68: .BR \-\-check=crc32 )
jpayne@68: using about 100\ KiB of memory.
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .nf
jpayne@68: .ft CR
jpayne@68: xz \-\-check=crc32 \-\-lzma2=preset=6e,dict=64KiB foo
jpayne@68: .ft R
jpayne@68: .fi
jpayne@68: .RE
jpayne@68: .PP
jpayne@68: If you want to squeeze out as many bytes as possible,
jpayne@68: adjusting the number of literal context bits
jpayne@68: .RI ( lc )
jpayne@68: and number of position bits
jpayne@68: .RI ( pb )
jpayne@68: can sometimes help.
jpayne@68: Adjusting the number of literal position bits
jpayne@68: .RI ( lp )
jpayne@68: might help too, but usually
jpayne@68: .I lc
jpayne@68: and
jpayne@68: .I pb
jpayne@68: are more important.
jpayne@68: For example, a source code archive contains mostly US-ASCII text,
jpayne@68: so something like the following might give
jpayne@68: slightly (like 0.1\ %) smaller file than
jpayne@68: .B "xz \-6e"
jpayne@68: (try also without
jpayne@68: .BR lc=4 ):
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .nf
jpayne@68: .ft CR
jpayne@68: xz \-\-lzma2=preset=6e,pb=0,lc=4 source_code.tar
jpayne@68: .ft R
jpayne@68: .fi
jpayne@68: .RE
jpayne@68: .PP
jpayne@68: Using another filter together with LZMA2 can improve
jpayne@68: compression with certain file types.
jpayne@68: For example, to compress a x86-32 or x86-64 shared library
jpayne@68: using the x86 BCJ filter:
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .nf
jpayne@68: .ft CR
jpayne@68: xz \-\-x86 \-\-lzma2 libfoo.so
jpayne@68: .ft R
jpayne@68: .fi
jpayne@68: .RE
jpayne@68: .PP
jpayne@68: Note that the order of the filter options is significant.
jpayne@68: If
jpayne@68: .B \-\-x86
jpayne@68: is specified after
jpayne@68: .BR \-\-lzma2 ,
jpayne@68: .B xz
jpayne@68: will give an error,
jpayne@68: because there cannot be any filter after LZMA2,
jpayne@68: and also because the x86 BCJ filter cannot be used
jpayne@68: as the last filter in the chain.
jpayne@68: .PP
jpayne@68: The Delta filter together with LZMA2
jpayne@68: can give good results with bitmap images.
jpayne@68: It should usually beat PNG,
jpayne@68: which has a few more advanced filters than simple
jpayne@68: delta but uses Deflate for the actual compression.
jpayne@68: .PP
jpayne@68: The image has to be saved in uncompressed format,
jpayne@68: for example, as uncompressed TIFF.
jpayne@68: The distance parameter of the Delta filter is set
jpayne@68: to match the number of bytes per pixel in the image.
jpayne@68: For example, 24-bit RGB bitmap needs
jpayne@68: .BR dist=3 ,
jpayne@68: and it is also good to pass
jpayne@68: .B pb=0
jpayne@68: to LZMA2 to accommodate the three-byte alignment:
jpayne@68: .RS
jpayne@68: .PP
jpayne@68: .nf
jpayne@68: .ft CR
jpayne@68: xz \-\-delta=dist=3 \-\-lzma2=pb=0 foo.tiff
jpayne@68: .ft R
jpayne@68: .fi
jpayne@68: .RE
jpayne@68: .PP
jpayne@68: If multiple images have been put into a single archive (for example,
jpayne@68: .BR .tar ),
jpayne@68: the Delta filter will work on that too as long as all images
jpayne@68: have the same number of bytes per pixel.
jpayne@68: .
jpayne@68: .SH "SEE ALSO"
jpayne@68: .BR xzdec (1),
jpayne@68: .BR xzdiff (1),
jpayne@68: .BR xzgrep (1),
jpayne@68: .BR xzless (1),
jpayne@68: .BR xzmore (1),
jpayne@68: .BR gzip (1),
jpayne@68: .BR bzip2 (1),
jpayne@68: .BR 7z (1)
jpayne@68: .PP
jpayne@68: XZ Utils: <https://tukaani.org/xz/>
jpayne@68: .br
jpayne@68: XZ Embedded: <https://tukaani.org/xz/embedded.html>
jpayne@68: .br
jpayne@68: LZMA SDK: <https://7-zip.org/sdk.html>