- java.lang.Object
-
- org.apache.lucene.util.compress.LZ4
-
public final class LZ4 extends java.lang.Object
LZ4 compression and decompression routines.https://github.com/lz4/lz4/tree/dev/lib http://fastcompression.blogspot.fr/p/lz4.html
The high-compression option is a simpler version of the one of the original algorithm, and only retains a better hash table that remembers about more occurrences of a previous 4-bytes sequence, and removes all the logic about handling of the case when overlapping matches are found.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
LZ4.FastCompressionHashTable
Simple lossyLZ4.HashTable
that only stores the last ocurrence for each hash on2^14
bytes of memory.(package private) static class
LZ4.HashTable
A record of previous occurrences of sequences of 4 bytes.static class
LZ4.HighCompressionHashTable
A higher-precisionLZ4.HashTable
.private static class
LZ4.Table
private static class
LZ4.Table16
16 bits per offset.private static class
LZ4.Table32
32 bits per value, only used when inputs exceed 64kB, e.g.
-
Field Summary
Fields Modifier and Type Field Description (package private) static int
HASH_LOG_HC
(package private) static int
HASH_TABLE_SIZE_HC
(package private) static int
LAST_LITERALS
static int
MAX_DISTANCE
Window size: this is the maximum supported distance between two strings so that LZ4 can replace the second one by a reference to the first one.(package private) static int
MEMORY_USAGE
(package private) static int
MIN_MATCH
-
Constructor Summary
Constructors Modifier Constructor Description private
LZ4()
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description private static int
commonBytes(byte[] b, int o1, int o2, int limit)
static void
compress(byte[] bytes, int off, int len, DataOutput out, LZ4.HashTable ht)
Compressbytes[off:off+len]
intoout
using at most 16kB of memory.static void
compressWithDictionary(byte[] bytes, int dictOff, int dictLen, int len, DataOutput out, LZ4.HashTable ht)
Compressbytes[dictOff+dictLen:dictOff+dictLen+len]
intoout
using at most 16kB of memory.static int
decompress(DataInput compressed, int decompressedLen, byte[] dest, int dOff)
Decompress at leastdecompressedLen
bytes intodest[dOff:]
.private static void
encodeLastLiterals(byte[] bytes, int anchor, int literalLen, DataOutput out)
private static void
encodeLen(int l, DataOutput out)
private static void
encodeLiterals(byte[] bytes, int token, int anchor, int literalLen, DataOutput out)
private static void
encodeSequence(byte[] bytes, int anchor, int matchRef, int matchOff, int matchLen, DataOutput out)
private static int
hash(int i, int hashBits)
private static int
hashHC(int i)
private static int
readInt(byte[] buf, int i)
-
-
-
Field Detail
-
MAX_DISTANCE
public static final int MAX_DISTANCE
Window size: this is the maximum supported distance between two strings so that LZ4 can replace the second one by a reference to the first one.- See Also:
- Constant Field Values
-
MEMORY_USAGE
static final int MEMORY_USAGE
- See Also:
- Constant Field Values
-
MIN_MATCH
static final int MIN_MATCH
- See Also:
- Constant Field Values
-
LAST_LITERALS
static final int LAST_LITERALS
- See Also:
- Constant Field Values
-
HASH_LOG_HC
static final int HASH_LOG_HC
- See Also:
- Constant Field Values
-
HASH_TABLE_SIZE_HC
static final int HASH_TABLE_SIZE_HC
- See Also:
- Constant Field Values
-
-
Method Detail
-
hash
private static int hash(int i, int hashBits)
-
hashHC
private static int hashHC(int i)
-
readInt
private static int readInt(byte[] buf, int i)
-
commonBytes
private static int commonBytes(byte[] b, int o1, int o2, int limit)
-
decompress
public static int decompress(DataInput compressed, int decompressedLen, byte[] dest, int dOff) throws java.io.IOException
Decompress at leastdecompressedLen
bytes intodest[dOff:]
. Please note thatdest
must be large enough to be able to hold all decompressed data (meaning that you need to know the total decompressed length). If the given bytes were compressed using a preset dictionary then the same dictionary must be provided indest[dOff-dictLen:dOff]
.- Throws:
java.io.IOException
-
encodeLen
private static void encodeLen(int l, DataOutput out) throws java.io.IOException
- Throws:
java.io.IOException
-
encodeLiterals
private static void encodeLiterals(byte[] bytes, int token, int anchor, int literalLen, DataOutput out) throws java.io.IOException
- Throws:
java.io.IOException
-
encodeLastLiterals
private static void encodeLastLiterals(byte[] bytes, int anchor, int literalLen, DataOutput out) throws java.io.IOException
- Throws:
java.io.IOException
-
encodeSequence
private static void encodeSequence(byte[] bytes, int anchor, int matchRef, int matchOff, int matchLen, DataOutput out) throws java.io.IOException
- Throws:
java.io.IOException
-
compress
public static void compress(byte[] bytes, int off, int len, DataOutput out, LZ4.HashTable ht) throws java.io.IOException
Compressbytes[off:off+len]
intoout
using at most 16kB of memory.ht
shouldn't be shared across threads but can safely be reused.- Throws:
java.io.IOException
-
compressWithDictionary
public static void compressWithDictionary(byte[] bytes, int dictOff, int dictLen, int len, DataOutput out, LZ4.HashTable ht) throws java.io.IOException
Compressbytes[dictOff+dictLen:dictOff+dictLen+len]
intoout
using at most 16kB of memory.bytes[dictOff:dictOff+dictLen]
will be used as a dictionary.dictLen
must not be greater than64kB
, the maximum window size.ht
shouldn't be shared across threads but can safely be reused.- Throws:
java.io.IOException
-
-