LZ4/Snappy Data Decompressor

Overview

LZ4SNP-D is a custom hardware implementation of a lossless data decompression engine for the LZ4 and Snappy compression algorithms. The core receives compressed files, automatically detects the LZ4 or Snappy format, and outputs the decompressed data.

The core features fast processing with low latency and high throughput. In its default configuration, LZ4SNP-D outputs 7.8 bytes of decompressed data per clock cycle and can be clocked at frequencies exceeding 1 GHz in modern ASIC technologies. Designers can scale the throughput by instantiating the core multiple times to achieve throughput rates exceeding 100Gbps. The processing latency is approximately 30 clock cycles.

The decompression core operates on a standalone basis—offloading the host CPU from the demanding task of data decompression—and has been designed for easy integration and use. No preprocessing of the incoming compressed files is required, as the core parses the file headers, checks the input files for errors, and outputs the decompressed data payload.

Extensive error tracking and reporting enable the core to ensure smooth system operation and error recovery, even in the presence of errors in the compressed input files. Furthermore, internal memories can optionally support Error Correction Codes (ECC) to simplify achieving enterprise-class reliability or functional safety requirements.

The LZ4SNP-D core is a microcode-free design developed for reuse in ASIC and FPGA implementations. Its streaming data interface—optionally bridged to AMBA® AXI4-Stream—eases SoC integration. Technology mapping is straightforward, as the design is scan-ready, LINT-clean, microcode-free, and uses easily replaceable, generic memory models.

Key Features

Compression Algorithms
- LZ4
  - 64KB history window size
  - All frame and block formats
  - CRC checking (optional, on request)
  - Dictionaries not supported
- Snappy
  - 64KB history window size
  - All frame and stream formats
  - CRC checking (optional, on request)
- High Performance & Low Latency
  - Average processing rate of 7.8 decompressed bytes per clock cycle
  - Clock frequency in excess of 1GHz on modern ASIC processes, and more than 300MHz on high-end FPGAs
  - Latency of approximately 30 clock cycles
- Easy to Use and Integrate
  - Processor-free, standalone operation
  - Automatic detection of input frame format (LZ4 or Snappy)
  - Extensive error catching & reporting for smooth operation and recovery in the presence of errors
    - CRC 32 errors
    - File size errors
    - Coding errors
    - Non-correctable ECC memory errors
  - Optional ECC memories
  - AXI-Stream or native FIFO-like data interfaces
  - Single clock domain design
  - Interface bridges and DMAs available separately
  - Microcode-free, LINT-clean, scan-ready design
- Configuration Options
  - Synthesis-time configuration options allow finetuning the core’s size and performance:
    - Input and output bus widths
    - FIFO sizes
    - Maximum history window
    - Data-path width
    - More

Block Diagram

Applications

The LZ4SNP-D core is ideal for increasing the bandwidth of optical, wired, or wireless data communication links. It also effectively increases data storage capacity in a wide range of devices, such as networking interface/routing/storage equipment, data servers, or SSD drives. The core can also help reduce the power consumption and bandwidth of centralized memories (e.g., DDR) or interfaces (e.g., Ethernet or Wi-Fi) in a wide range of SoCs.

Deliverables

The core is available in synthesizable HDL (System Verilog) or targeted FPGA netlist forms and includes everything required for successful implementation. Its deliverables include:
- Sophisticated Test Environment
- Simulation scripts, test vectors, and expected results
- Synthesis script
- Comprehensive user documentation

Technical Specifications

Availability

Now

Request Info