File LibDeflate.lua

LibDeflate 1.0.2-release
Pure Lua compressor and decompressor with high compression ratio using DEFLATE/zlib format.

Info:

  • Copyright: LibDeflate <2018-2021> Haoqian He
  • License: zlib License

    This library is implemented according to the following specifications.
    Report a bug if LibDeflate is not fully compliant with those specs.
    Both compressors and decompressors have been implemented in the library.
    1. RFC1950: DEFLATE Compressed Data Format Specification version 1.3
    https://tools.ietf.org/html/rfc1951
    2. RFC1951: ZLIB Compressed Data Format Specification version 3.3
    https://tools.ietf.org/html/rfc1950

    This library requires Lua 5.1/5.2/5.3/5.4 interpreter or LuaJIT v2.0+.
    This library does not have any dependencies.

    This file "LibDeflate.lua" is the only source file of the library.
    Submit suggestions or report bugs to https://github.com/safeteeWow/LibDeflate/issues

  • Author: Haoqian He (Github: SafeteeWoW; World of Warcraft: Safetyy-Illidan(US))

Functions

LibDeflate:Adler32 (str) Calculate the Adler-32 checksum of the string.
LibDeflate:CompressDeflate (str, configs) Compress using the raw deflate format.
LibDeflate:CompressDeflateWithDict (str, dictionary, configs) Compress using the raw deflate format with a preset dictionary.
LibDeflate:CompressZlib (str, configs) Compress using the zlib format.
LibDeflate:CompressZlibWithDict (str, dictionary, configs) Compress using the zlib format with a preset dictionary.
LibDeflate:CreateCodec (reserved_chars, escape_chars, map_chars) Create a custom codec with encoder and decoder.
LibDeflate:CreateDictionary (str, strlen, adler32) Create a preset dictionary.
LibDeflate:DecodeForPrint (str) Decode the printable string produced by LibDeflate:EncodeForPrint.
LibDeflate:DecodeForWoWAddonChannel (str) Decode the string produced by LibDeflate:EncodeForWoWAddonChannel
LibDeflate:DecodeForWoWChatChannel (str) Decode the string produced by LibDeflate:EncodeForWoWChatChannel.
LibDeflate:DecompressDeflate (str) Decompress a raw deflate compressed data.
LibDeflate:DecompressDeflateWithDict (str, dictionary) Decompress a raw deflate compressed data with a preset dictionary.
LibDeflate:DecompressZlib (str) Decompress a zlib compressed data.
LibDeflate:DecompressZlibWithDict (str, dictionary) Decompress a zlib compressed data with a preset dictionary.
LibDeflate:EncodeForPrint (str) Encode the string to make it printable.
LibDeflate:EncodeForWoWAddonChannel (str) Encode the string to make it ready to be transmitted in World of Warcraft addon channel.
LibDeflate:EncodeForWoWChatChannel (str) Encode the string to make it ready to be transmitted in World of Warcraft chat channel.

Tables

CommandlineOptions Commandline options
compression_configs The description to compression configuration table.


Functions

LibDeflate:Adler32 (str)
Calculate the Adler-32 checksum of the string.
See RFC1950 Page 9 https://tools.ietf.org/html/rfc1950 for the definition of Adler-32 checksum.

Parameters:

  • str [string] the input string to calcuate its Adler-32 checksum.

Returns:

    [integer] The Adler-32 checksum, which is greater or equal to 0, and less than 2^32 (4294967296).
LibDeflate:CompressDeflate (str, configs)
Compress using the raw deflate format.

Parameters:

  • str [string] The data to be compressed.
  • configs [table/nil] The configuration table to control the compression . If nil, use the default configuration.

Returns:

  1. [string] The compressed data.
  2. [integer] The number of bits padded at the end of output. 0 <= bits < 8
    This means the most significant "bits" of the last byte of the returned compressed data are padding bits and they don't affect decompression. You don't need to use this value unless you want to do some postprocessing to the compressed data.

See also:

LibDeflate:CompressDeflateWithDict (str, dictionary, configs)
Compress using the raw deflate format with a preset dictionary.

Parameters:

  • str [string] The data to be compressed.
  • dictionary [table] The preset dictionary produced by LibDeflate:CreateDictionary
  • configs [table/nil] The configuration table to control the compression . If nil, use the default configuration.

Returns:

  1. [string] The compressed data.
  2. [integer] The number of bits padded at the end of output. 0 <= bits < 8
    This means the most significant "bits" of the last byte of the returned compressed data are padding bits and they don't affect decompression. You don't need to use this value unless you want to do some postprocessing to the compressed data.

See also:

LibDeflate:CompressZlib (str, configs)
Compress using the zlib format.

Parameters:

  • str [string] the data to be compressed.
  • configs [table/nil] The configuration table to control the compression . If nil, use the default configuration.

Returns:

  1. [string] The compressed data.
  2. [integer] The number of bits padded at the end of output. Should always be 0. Zlib formatted compressed data never has padding bits at the end.

See also:

LibDeflate:CompressZlibWithDict (str, dictionary, configs)
Compress using the zlib format with a preset dictionary.

Parameters:

  • str [string] the data to be compressed.
  • dictionary [table] A preset dictionary produced by LibDeflate:CreateDictionary()
  • configs [table/nil] The configuration table to control the compression . If nil, use the default configuration.

Returns:

  1. [string] The compressed data.
  2. [integer] The number of bits padded at the end of output. Should always be 0. Zlib formatted compressed data never has padding bits at the end.

See also:

LibDeflate:CreateCodec (reserved_chars, escape_chars, map_chars)
Create a custom codec with encoder and decoder.
This codec is used to convert an input string to make it not contain some specific bytes. This created codec and the parameters of this function do NOT take localization into account. One byte (0-255) in the string is exactly one character (0-255). Credits to LibCompress. The code has been rewritten by the author of LibDeflate.

Parameters:

  • reserved_chars [string] The created encoder will ensure encoded data does not contain any single character in reserved_chars. This parameter should be non-empty.
  • escape_chars [string] The escape character(s) used in the created codec. The codec converts any character included in reserved_chars / escape_chars / map_chars to (one escape char + one character not in reserved_chars / escape_chars / map_chars). You usually only need to provide a length-1 string for this parameter. Length-2 string is only needed when reserved_chars + escape_chars + map_chars is longer than 127. This parameter should be non-empty.
  • map_chars [string] The created encoder will map every reserved_chars:sub(i, i) (1 <= i <= #map_chars) to map_chars:sub(i, i). This parameter CAN be empty string.

Returns:

  1. [table/nil] If the codec cannot be created, return nil.
    If the codec can be created according to the given parameters, return the codec, which is a encode/decode table. The table contains two functions:
    t:Encode(str) returns the encoded string.
    t:Decode(str) returns the decoded string if succeeds. nil if fails.
  2. [nil/string] If the codec is successfully created, return nil. If not, return a string that describes the reason why the codec cannot be created.

Usage:

    -- Create an encoder/decoder that maps all "\000" to "\003",
-- and escape "\001" (and "\002" and "\003") properly
local codec = LibDeflate:CreateCodec("\000\001", "\002", "\003")

local encoded = codec:Encode(SOME_STRING)
-- "encoded" does not contain "\000" or "\001"
local decoded = codec:Decode(encoded)
-- assert(decoded == SOME_STRING)
LibDeflate:CreateDictionary (str, strlen, adler32)
Create a preset dictionary.

This function is not fast, and the memory consumption of the produced dictionary is about 50 times of the input string. Therefore, it is suggestted to run this function only once in your program.

It is very important to know that if you do use a preset dictionary, compressors and decompressors MUST USE THE SAME dictionary. That is, dictionary must be created using the same string. If you update your program with a new dictionary, people with the old version won't be able to transmit data with people with the new version. Therefore, changing the dictionary must be very careful.

The parameters "strlen" and "adler32" add a layer of verification to ensure the parameter "str" is not modified unintentionally during the program development.

Parameters:

  • str [string] The string used as the preset dictionary.
    You should put stuffs that frequently appears in the dictionary string and preferablely put more frequently appeared stuffs toward the end of the string.
    Empty string and string longer than 32768 bytes are not allowed.
  • strlen [integer] The length of 'str'. Please pass in this parameter as a hardcoded constant, in order to verify the content of 'str'. The value of this parameter should be known before your program runs.
  • adler32 [integer] The Adler-32 checksum of 'str'. Please pass in this parameter as a hardcoded constant, in order to verify the content of 'str'. The value of this parameter should be known before your program runs.

Returns:

    [table] The dictionary used for preset dictionary compression and decompression.

Raises:

error if 'strlen' does not match the length of 'str', or if 'adler32' does not match the Adler-32 checksum of 'str'.

Usage:

    local dict_str = "1234567890"

-- print(dict_str:len(), LibDeflate:Adler32(dict_str))
-- Hardcode the print result below to verify it to avoid acciently
-- modification of 'str' during the program development.
-- string length: 10, Adler-32: 187433486,
-- Don't calculate string length and its Adler-32 at run-time.

local dict = LibDeflate:CreateDictionary(dict_str, 10, 187433486)
LibDeflate:DecodeForPrint (str)
Decode the printable string produced by LibDeflate:EncodeForPrint. "str" will have its prefixed and trailing control characters or space removed before it is decoded, so it is easier to use if "str" comes form user copy and paste with some prefixed or trailing spaces. Then decode fails if the string contains any characters cant be produced by LibDeflate:EncodeForPrint. That means, decode fails if the string contains a characters NOT one of 26 lowercase letters, 26 uppercase letters, 10 numbers digits, left parenthese, or right parenthese.

Parameters:

  • str [string] The string to be decoded

Returns:

    [string/nil] The decoded string if succeeds. nil if fails.
LibDeflate:DecodeForWoWAddonChannel (str)
Decode the string produced by LibDeflate:EncodeForWoWAddonChannel

Parameters:

  • str [string] The string to be decoded.

Returns:

    [string/nil] The decoded string if succeeds. nil if fails.

See also:

LibDeflate:DecodeForWoWChatChannel (str)
Decode the string produced by LibDeflate:EncodeForWoWChatChannel.

Parameters:

  • str [string] The string to be decoded.

Returns:

    [string/nil] The decoded string if succeeds. nil if fails.

See also:

LibDeflate:DecompressDeflate (str)
Decompress a raw deflate compressed data.

Parameters:

  • str [string] The data to be decompressed.

Returns:

  1. [string/nil] If the decompression succeeds, return the decompressed data. If the decompression fails, return nil. You should check if this return value is non-nil to know if the decompression succeeds.
  2. [integer] If the decompression succeeds, return the number of unprocessed bytes in the input compressed data. This return value is a positive integer if the input data is a valid compressed data appended by an arbitary non-empty string. This return value is 0 if the input data does not contain any extra bytes.
    If the decompression fails (The first return value of this function is nil), this return value is undefined.

See also:

LibDeflate:DecompressDeflateWithDict (str, dictionary)
Decompress a raw deflate compressed data with a preset dictionary.

Parameters:

  • str [string] The data to be decompressed.
  • dictionary [table] The preset dictionary used by LibDeflate:CompressDeflateWithDict when the compressed data is produced. Decompression and compression must use the same dictionary. Otherwise wrong decompressed data could be produced without generating any error.

Returns:

  1. [string/nil] If the decompression succeeds, return the decompressed data. If the decompression fails, return nil. You should check if this return value is non-nil to know if the decompression succeeds.
  2. [integer] If the decompression succeeds, return the number of unprocessed bytes in the input compressed data. This return value is a positive integer if the input data is a valid compressed data appended by an arbitary non-empty string. This return value is 0 if the input data does not contain any extra bytes.
    If the decompression fails (The first return value of this function is nil), this return value is undefined.

See also:

LibDeflate:DecompressZlib (str)
Decompress a zlib compressed data.

Parameters:

  • str [string] The data to be decompressed

Returns:

  1. [string/nil] If the decompression succeeds, return the decompressed data. If the decompression fails, return nil. You should check if this return value is non-nil to know if the decompression succeeds.
  2. [integer] If the decompression succeeds, return the number of unprocessed bytes in the input compressed data. This return value is a positive integer if the input data is a valid compressed data appended by an arbitary non-empty string. This return value is 0 if the input data does not contain any extra bytes.
    If the decompression fails (The first return value of this function is nil), this return value is undefined.

See also:

LibDeflate:DecompressZlibWithDict (str, dictionary)
Decompress a zlib compressed data with a preset dictionary.

Parameters:

  • str [string] The data to be decompressed
  • dictionary [table] The preset dictionary used by LibDeflate:CompressDeflateWithDict when the compressed data is produced. Decompression and compression must use the same dictionary. Otherwise wrong decompressed data could be produced without generating any error.

Returns:

  1. [string/nil] If the decompression succeeds, return the decompressed data. If the decompression fails, return nil. You should check if this return value is non-nil to know if the decompression succeeds.
  2. [integer] If the decompression succeeds, return the number of unprocessed bytes in the input compressed data. This return value is a positive integer if the input data is a valid compressed data appended by an arbitary non-empty string. This return value is 0 if the input data does not contain any extra bytes.
    If the decompression fails (The first return value of this function is nil), this return value is undefined.

See also:

LibDeflate:EncodeForPrint (str)
Encode the string to make it printable.

Credit to WeakAuras2, this function is equivalant to the implementation it is using right now.
The code has been rewritten by the author of LibDeflate.
The encoded string will be 25% larger than the origin string. However, every single byte of the encoded string will be one of 64 printable ASCII characters, which are can be easier copied, pasted and displayed. (26 lowercase letters, 26 uppercase letters, 10 numbers digits, left parenthese, or right parenthese)

Parameters:

  • str [string] The string to be encoded.

Returns:

    [string] The encoded string.
LibDeflate:EncodeForWoWAddonChannel (str)
Encode the string to make it ready to be transmitted in World of Warcraft addon channel.
The encoded string is guaranteed to contain no NULL ("\000") character.

Parameters:

  • str [string] The string to be encoded.

Returns:

    The encoded string.

See also:

LibDeflate:EncodeForWoWChatChannel (str)
Encode the string to make it ready to be transmitted in World of Warcraft chat channel.
See also https://wow.gamepedia.com/ValidChatMessageCharacters

Parameters:

  • str [string] The string to be encoded.

Returns:

    [string] The encoded string.

See also:

Tables

CommandlineOptions
Commandline options

Usage:

    lua LibDeflate.lua [OPTION] [INPUT] [OUTPUT]
-0    store only. no compression.
-1    fastest compression.
-9    slowest and best compression.
-d    do decompression instead of compression.
--dict <filename> specify the file that contains
he entire preset dictionary.
-h    give this help.
--strategy <fixed/huffman_only/dynamic> specify a special compression strategy.
-v    print the version and copyright info.
--zlib  use zlib format instead of raw deflate.
compression_configs
The description to compression configuration table.
Any field can be nil to use its default.
Table with keys other than those below is an invalid table.

Fields:

  • level The compression level ranged from 0 to 9. 0 is no compression. 9 is the slowest but best compression. Use nil for default level.
  • strategy The compression strategy. "fixed" to only use fixed deflate compression block. "dynamic" to only use dynamic block. "huffman_only" to do no LZ77 compression. Only do huffman compression.
generated by LDoc 1.4.6 Last updated