XZ Format - 7-Zip

XZ is a modern compression format using the LZMA2 algorithm, providing excellent compression ratios with reasonable decompression speed.

Format Overview

XZ provides:

Excellent compression ratio - Best general-purpose compression
LZMA2 algorithm - Advanced dictionary-based compression
Block structure - Independent blocks for parallelization
Integrity checking - CRC32, CRC64, SHA-256 support
No size limits - Files up to 16 exabytes
Stream padding - Multiple concatenated streams

XZ typically achieves 10-30% better compression than BZIP2 and 20-50% better than GZIP, making it ideal for distribution and archival.

Format Structure

From source/CPP/7zip/Archive/XzHandler.cpp:37-43, XZ uses a block-based structure:

struct CBlockInfo {
  unsigned StreamFlags;
  UInt64 PackPos;
  UInt64 PackSize;    // Size from Index record (no padding)
  UInt64 UnpackPos;
};

Stream Format

+-------------------------+
| Stream Header (12 bytes)|
+-------------------------+
| Block(s)                |
| - Block Header          |
| - Compressed Data       |
| - Block Padding         |
| - Check Value           |
+-------------------------+
| Index                   |
+-------------------------+
| Stream Footer (12 bytes)|
+-------------------------+

Stream Header

Bytes 0-5:   Magic (0xFD '7' 'z' 'X' 'Z' 0x00)
Byte 6-7:    Stream Flags
Bytes 8-11:  CRC32 of bytes 6-7

The XZ magic bytes FD 37 7A 58 5A 00 make XZ files easily identifiable and less likely to be confused with other formats.

Compression Algorithm

LZMA2 Method

From XzHandler.cpp:34:

#define k_LZMA2_Name "LZMA2"

LZMA2 features:

Dictionary matching - Finds repeated patterns
Range encoding - Efficient entropy coding
Multi-threading support - Parallel compression
Better than LZMA - Reset states, better for incompressible data

Filters

XZ supports preprocessing filters (XzHandler.cpp:101):

UInt32 _filterId;  // BCJ, Delta, etc.

Common filters:

BCJ (x86) - For x86 executables
BCJ (ARM, ARM64, PPC, IA64, SPARC) - For other architectures
Delta - For numeric data

Usage Examples

Compress Single File

7z a -txz file.txt.xz file.txt

Compress with Maximum Compression

7z a -txz -mx=9 file.xz file.txt

Compress with Specific Dictionary Size

7z a -txz -mx=9 -md=64m file.xz file.txt

Dictionary sizes:

-md=16m - Fast, moderate compression
-md=32m - Good balance (default for -mx=9)
-md=64m - Better compression, more memory
-md=128m - Maximum compression, high memory

Decompress File

7z x file.xz

Create TAR.XZ Archive

7z a -ttar archive.tar files/
7z a -txz -mx=9 archive.tar.xz archive.tar

Or using system tar:

tar -cJf archive.tar.xz files/

Multi-threaded Compression

7z a -txz -mx=9 -mmt=4 file.xz large_file.txt

Uses 4 threads for faster compression.

Multi-threaded XZ compression can dramatically reduce compression time on multi-core systems, though it may slightly reduce compression ratio.

Handler Implementation

From source/CPP/7zip/Archive/XzHandler.cpp:46-78:

Z7_class_CHandler_final:
  public IInArchive,
  public IArchiveOpenSeq,
  public IInArchiveGetStream,
  public ISetProperties,
 #ifndef Z7_EXTRACT_ONLY
  public IOutArchive,
 #endif
  public CMyUnknownImp,
 #ifndef Z7_EXTRACT_ONLY
  public CMultiMethodProps
 #else
  public CCommonMethodProps
 #endif

Handler State

bool _stat_defined;
bool _stat2_defined;
bool _isArc;
bool _needSeekToStart;
bool _firstBlockWasRead;
SRes _stat2_decode_SRes;

CXzStatInfo _stat;    // Backward parsing statistics
CXzStatInfo _stat2;   // Forward parsing statistics

Archive Properties

From XzHandler.cpp:196-200:

static const Byte kProps[] = {
  kpidSize,
  kpidPackSize,
  kpidMethod
};

XZ tracks:

Size - Uncompressed size
PackSize - Compressed size
Method - Compression method (LZMA2)
NumStreams - Number of streams
NumBlocks - Number of blocks

Compression Levels

Performance Comparison (100 MB mixed files)

Level	Dict Size	Time	Size	Ratio	Memory (Comp)	Memory (Decomp)
-mx=0	-	5s	80 MB	80%	3 MB	1 MB
-mx=1	256 KB	20s	35 MB	35%	10 MB	2 MB
-mx=3	4 MB	35s	28 MB	28%	50 MB	6 MB
-mx=5	8 MB	45s	25 MB	25%	100 MB	10 MB
-mx=7	16 MB	60s	23 MB	23%	200 MB	18 MB
-mx=9	64 MB	90s	20 MB	20%	700 MB	66 MB

Higher compression levels require significantly more memory. Ensure your system has sufficient RAM before using -mx=9 with large dictionary sizes.

Advanced Features

Block Size Configuration

UInt64 _numSolidBytes;  // Block size (XzHandler.cpp:107)

Set block size:

7z a -txz -mx=9 -ms=64m file.xz large_file.txt

Block sizes:

Smaller blocks - Better for random access, multi-threading
Larger blocks - Better compression ratio

Integrity Checks

XZ supports multiple check types:

None - No integrity check (fastest)
CRC32 - Fast, good detection
CRC64 - Better detection
SHA-256 - Cryptographic hash

7z a -txz -mx=9 -mcheck=crc64 file.xz file.txt

Stream Concatenation

Multiple XZ streams can be concatenated:

7z a -txz file1.xz file1.txt
7z a -txz file2.xz file2.txt
cat file1.xz file2.xz > combined.xz
7z x combined.xz  # Extracts both streams

Filters for Executables

Use BCJ filter for better executable compression:

7z a -txz -mx=9 -mf=BCJ program.xz program.exe

Filter types:

BCJ - x86 executables
ARM - ARM binaries
ARM64 - ARM64 binaries
PPC - PowerPC binaries

Memory Requirements

Compression Memory

Memory ≈ Dictionary × 11.5 + 100 MB

Examples:

-md=8m: ~200 MB
-md=32m: ~470 MB
-md=64m: ~840 MB
-md=128m: ~1.6 GB

Decompression Memory

Memory ≈ Dictionary + 10 MB

Examples:

8 MB dict: ~18 MB
32 MB dict: ~42 MB
64 MB dict: ~74 MB

Decompression is much more memory-efficient than compression, allowing archives compressed with -mx=9 to be extracted on low-memory systems.

Implementation Details

XZ implementation files:

source/CPP/7zip/Archive/XzHandler.cpp    # Archive handler
source/CPP/7zip/Compress/XzDecoder.cpp   # Decompression
source/CPP/7zip/Compress/XzEncoder.cpp   # Compression

Includes C library:

#include "../../../C/Alloc.h"

Uses 7-Zip’s XZ implementation compatible with XZ Utils.

Comparison with Other Formats

XZ vs GZIP

Feature	XZ	GZIP
Compression ratio	Excellent	Good
Compression speed	Slow	Fast
Decompression speed	Medium	Fast
Memory usage	High	Low
Best for	Distribution, archival	Quick compression

XZ vs BZIP2

Feature	XZ	BZIP2
Compression ratio	15-20% better	Good
Compression speed	Similar	Similar
Decompression speed	Faster	Slower
Memory usage	Higher	Lower
Block independence	Yes	Yes

XZ vs 7z

Feature	XZ	7z
Multiple files	No (needs TAR)	Yes
Solid compression	Block-based	Full solid support
Encryption	No	Yes (AES-256)
Compression ratio	Excellent	Excellent (similar)
Compatibility	Growing	Requires 7-Zip

Best Practices

For Distribution

Use -mx=9 with moderate dictionary for best compression

For Large Files

Use multi-threading: -mmt=4 or more

For Archives

Combine with TAR: .tar.xz format

For Speed

Use -mx=3 to -mx=5 for faster compression

Common Use Cases

Software Distribution

tar -cJf myapp-1.0.tar.xz myapp-1.0/

XZ is becoming the standard for Linux source distributions.

System Backups

tar -cJf backup-$(date +%Y%m%d).tar.xz /home /etc

Log Archival

find /var/log -name "*.log" -mtime +30 -exec xz {} \;

Database Dumps

mysqldump --all-databases | 7z a -txz -mx=9 -si backup.sql.xz

Performance Optimization

Parallel Compression

Use multi-threading:

7z a -txz -mx=9 -mmt=on file.xz large_file.txt

Or specify thread count:

7z a -txz -mx=9 -mmt=8 file.xz large_file.txt

Dictionary Size Selection

Choose based on file size and available RAM:

File < 1 MB - Use -md=1m
File 1-10 MB - Use -md=8m
File 10-100 MB - Use -md=32m
File > 100 MB - Use -md=64m or higher

Preset Levels

7z a -txz -mx=6 file.xz data.txt  # Good balance

Recommended presets:

-mx=3 - Fast, good compression
-mx=6 - Balanced (similar to xz -6)
-mx=9 - Maximum compression

Limitations

XZ format limitations:

Single file compression (use TAR for multiple files)
High memory requirements for compression
Slower than GZIP and BZIP2 for compression
No encryption support
Requires modern tools (not on very old systems)

Compatibility

Tool Support

xz-utils - Reference implementation (Linux)
7-Zip - Full support (Windows, Linux)
XZ Utils - Native support (macOS via Homebrew)
PeaZip - GUI support
File Roller, Ark - Linux GUI support

Platform Support

Linux - Universal (xz command)
macOS - Via Homebrew: brew install xz
Windows - Via 7-Zip
BSD - Built-in or via packages

​Format Overview

​Format Structure

​Stream Format

​Stream Header

​Compression Algorithm

​LZMA2 Method

​Filters

​Usage Examples

​Compress Single File

​Compress with Maximum Compression

​Compress with Specific Dictionary Size

​Decompress File

​Create TAR.XZ Archive

​Multi-threaded Compression

​Handler Implementation

​Handler State

​Archive Properties

​Compression Levels

​Performance Comparison (100 MB mixed files)

​Advanced Features

​Block Size Configuration

​Integrity Checks

​Stream Concatenation

​Filters for Executables

​Memory Requirements

​Compression Memory

​Decompression Memory

​Implementation Details

​Comparison with Other Formats

​XZ vs GZIP

​XZ vs BZIP2

​XZ vs 7z

​Best Practices

For Distribution

For Large Files

For Archives

For Speed

​Common Use Cases

​Software Distribution

​System Backups

​Log Archival

​Database Dumps

​Performance Optimization

​Parallel Compression

​Dictionary Size Selection

​Preset Levels

​Limitations

​Compatibility

​Tool Support

​Platform Support

​See Also

Format Overview

Format Structure

Stream Format

Stream Header

Compression Algorithm

LZMA2 Method

Filters

Usage Examples

Compress Single File

Compress with Maximum Compression

Compress with Specific Dictionary Size

Decompress File

Create TAR.XZ Archive

Multi-threaded Compression

Handler Implementation

Handler State

Archive Properties

Compression Levels

Performance Comparison (100 MB mixed files)

Advanced Features

Block Size Configuration

Integrity Checks

Stream Concatenation

Filters for Executables

Memory Requirements

Compression Memory

Decompression Memory

Implementation Details

Comparison with Other Formats

XZ vs GZIP

XZ vs BZIP2

XZ vs 7z

Best Practices

Common Use Cases

Software Distribution

System Backups

Log Archival

Database Dumps

Performance Optimization

Parallel Compression

Dictionary Size Selection

Preset Levels

Limitations

Compatibility

Tool Support

Platform Support

See Also