| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags --> |
| <title>LZ4 - Extremely fast compression</title> |
| |
| <!-- Bootstrap --> |
| <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/css/bootstrap.min.css"> |
| <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.3.0/css/font-awesome.min.css"> |
| |
| <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries --> |
| <!-- WARNING: Respond.js doesn't work if you view the page via file:// --> |
| <!--[if lt IE 9]> |
| <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script> |
| <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script> |
| <![endif]--> |
| |
| <!-- Chart example --> |
| <script type="text/javascript" src="js/csvToArray.v2.1.js"></script> |
| <script type="text/javascript" src="https://www.google.com/jsapi"></script> |
| <script type="text/javascript"> |
| function trim1 (str) { |
| return str.replace(/^\s\s*/, '').replace(/\s\s*$/, ''); |
| } |
| |
| function drawTable(gvCtor, gvOptions, csvUrl, tableId) { |
| var r = new XMLHttpRequest(); |
| r.open("GET", csvUrl, true); |
| r.onreadystatechange = function() { |
| if (this.readyState === 4 && this.status == 200) { |
| var csvAsArray = this.responseText.csvToArray(); |
| |
| var gvdt = new google.visualization.DataTable() |
| var csvHeaderLine = csvAsArray.shift(); |
| for(var i = 0; i < csvHeaderLine.length; i++) { |
| var csvHeaderElement = csvHeaderLine[i].split(":"); |
| var csvHeaderName = "?"; |
| var csvHeaderType = "string"; |
| if(csvHeaderElement.length > 0) { |
| csvHeaderName = csvHeaderElement[0]; |
| if(csvHeaderElement.length > 1) { |
| csvHeaderType = trim1(csvHeaderElement[1]); |
| } |
| } |
| gvdt.addColumn(csvHeaderType, csvHeaderName); |
| for(var j = 0; j < csvAsArray.length; j++) { |
| switch(csvHeaderType) { |
| default: |
| case "string": csvAsArray[j][i] = String(csvAsArray[j][i]); break; |
| case "number": csvAsArray[j][i] = Number(csvAsArray[j][i]); break; |
| case "boolean": csvAsArray[j][i] = Boolean(csvAsArray[j][i]); break; |
| } |
| } |
| } |
| gvdt.addRows(csvAsArray); |
| |
| var table = new gvCtor (document.getElementById(tableId)); |
| table.draw(gvdt, gvOptions); |
| } |
| } |
| r.send(); |
| } |
| |
| google.load("visualization", "1", {packages:["table", "corechart"]}); |
| google.setOnLoadCallback(function() { |
| options = { |
| showRowNumber: true |
| }; |
| drawTable(google.visualization.Table, options, "csv/sample1.csv", "table_div"); |
| |
| options = { |
| title : 'Monthly Coffee Production by Country', |
| vAxis: {title: "Cups"}, |
| hAxis: {title: "Month"}, |
| seriesType: "bars", |
| series: {5: {type: "line"}} |
| }; |
| drawTable(google.visualization.ComboChart, options, "csv/sample2.csv", "table_2"); |
| }); |
| </script> |
| </head> |
| <body style="padding-top:0; padding-bottom:0; "> |
| <nav class="navbar navbar-default"> |
| <div class="container-fluid"> |
| <!-- Brand and toggle get grouped for better mobile display --> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a class="navbar-brand" href="/">LZ4</a> |
| </div> |
| |
| <!-- Collect the nav links, forms, and other content for toggling --> |
| <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1"> |
| <ul class="nav navbar-nav navbar-nav"> |
| <li><a href="https://github.com/Cyan4973/lz4/archive/r127.tar.gz">Download</a></li> |
| <li><a href="#interoperable-lz4">Languages</a></li> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Index<span class="caret"></span></a> |
| <ul class="dropdown-menu" role="menu"> |
| <li><a href="#benchmarks">Benchmarks</a></li> |
| <li><a href="#interoperable-lz4">Interoperable LZ4 versions</a></li> |
| <li><a href="#other-source">Other Source versions</a></li> |
| </ul> |
| </li> |
| </ul> |
| <ul class="nav navbar-nav navbar-right"> |
| <li><a href="https://groups.google.com/forum/#!forum/lz4c">Public discussion forum</a></li> |
| <li><a href="https://groups.google.com/forum/#!forum/lz4c">Issue Tracker</a></li> |
| <li><a href="https://github.com/Cyan4973/lz4"><i class="fa fa-lg fa-github"></i> GitHub</a></li> |
| </ul> |
| </div><!-- /.navbar-collapse --> |
| </div><!-- /.container-fluid --> |
| </nav> |
| |
| <div class="container"> |
| <div class="page-header jumbotron"> |
| <h1>LZ4</h1> |
| LZ4 is lossless compression algorithm, providing compression speed at 400 MB/s per core, scalable with multi-cores CPU. It also features an extremely fast decoder, with speed in multiple GB/s per core, typically reaching RAM speed limits on multi-core systems. |
| A high compression derivative, called LZ4_HC, is also provided. It trades CPU time for compression ratio. |
| </div> |
| </div> |
| |
| <xmp theme="sandstone" style="display:none;"> |
| <a name="summary"></a> |
| LZ4 - Extremely fast compression |
| ================================ |
| |
| LZ4 is lossless compression algorithm, providing compression speed at 400 MB/s per core, scalable with multi-cores CPU. It also features an extremely fast decoder, with speed in multiple GB/s per core, typically reaching RAM speed limits on multi-core systems. |
| A high compression derivative, called LZ4_HC, is also provided. It trades CPU time for compression ratio. |
| |
| |Branch |Status | |
| |------------|---------| |
| |master | [](https://travis-ci.org/Cyan4973/lz4) | |
| |dev | [](https://travis-ci.org/Cyan4973/lz4) | |
| |
| |
| > **Branch Policy:** |
| |
| > - The "master" branch is considered stable, at all times. |
| > - The "dev" branch is the one where all contributions must be merged before being promoted to master. |
| > - If you plan to propose a patch, please commit into the "dev" branch. Direct commit to "master" are not permitted. |
| > - Feature branches can also exist, for dedicated testing of larger modifications before merge into "dev" branch. |
| |
| <a name="benchmarks"></a> |
| Benchmarks |
| ------------------------- |
| |
| The benchmark uses the [Open-Source Benchmark program by m^2 (v0.14.2)](http://encode.ru/threads/1371-Filesystem-benchmark?p=33548&viewfull=1#post33548) compiled with GCC v4.6.1 on Linux Ubuntu 64-bits v11.10, |
| The reference system uses a Core i5-3340M @2.7GHz. |
| Benchmark evaluates the compression of reference [Silesia Corpus](http://sun.aei.polsl.pl/~sdeor/index.php?page=silesia) in single-thread mode. |
| |
| <table> |
| <tr> |
| <th>Compressor</th><th>Ratio</th><th>Compression</th><th>Decompression</th> |
| </tr> |
| <tr> |
| <th>LZ4 (r101)</th><th>2.084</th><th>422 MB/s</th><th>1820 MB/s</th> |
| </tr> |
| <tr> |
| <th>LZO 2.06</th><th>2.106</th><th>414 MB/s</th><th>600 MB/s</th> |
| </tr> |
| <tr> |
| <th>QuickLZ 1.5.1b6</th><th>2.237</th><th>373 MB/s</th><th>420 MB/s</th> |
| </tr> |
| <tr> |
| <th>Snappy 1.1.0</th><th>2.091</th><th>323 MB/s</th><th>1070 MB/s</th> |
| </tr> |
| <tr> |
| <th>LZF</th><th>2.077</th><th>270 MB/s</th><th>570 MB/s</th> |
| </tr> |
| <tr> |
| <th>zlib 1.2.8 -1</th><th>2.730</th><th>65 MB/s</th><th>280 MB/s</th> |
| </tr> |
| <tr> |
| <th>LZ4 HC (r101)</th><th>2.720</th><th>25 MB/s</th><th>2080 MB/s</th> |
| </tr> |
| <tr> |
| <th>zlib 1.2.8 -6</th><th>3.099</th><th>21 MB/s</th><th>300 MB/s</th> |
| </tr> |
| </table> |
| |
| The LZ4 block compression format is detailed within [lz4_block_format.txt](lz4_block_format.txt). |
| |
| For streaming unknown amount of data, and compress files of any size, a frame format has been published, and can be consulted within the file [LZ4_Frame_Format.html](LZ4_Frame_Format.html). |
| |
| <a name="interoperable-lz4"></a> |
| ## Interoperable LZ4 versions : |
| The following versions are provided for languages beyond the C reference version. They are in conformance with the LZ4 frame specification and are therefore interoperable. |
| |
| * __Javascript__ : by Pierre Curto, at https://github.com/pierrec/node-lz4 |
| * __C++11__ multi-threads : by Takayuki Matsuoka, at https://github.com/t-mat/lz4mt |
| * __Python__ : by Christopher Jackson, at https://github.com/darkdragn/lz4tools |
| * __Delphi__ : by Hanno Hugenberg, at https://github.com/Hugie/lz4-delphi |
| * __Go__ : by Pierre Curto, at https://github.com/pierrec/lz4 |
| * __Rust__ : by Artem Navrotskiy, at https://github.com/bozaro/lz4-rs |
| |
| <a name="other-source"></a> |
| ## Other Source versions : |
| The following versions compress data blocks using LZ4 compression algorithm in multiple languages. Each version encapsulates the compressed result into its own header/frame format convention. |
| |
| * __Java__ : by Adrien Grand, at https://github.com/jpountz/lz4-java |
| * __Python__ : by Steeve Morin, at http://pypi.python.org/pypi/lz4 |
| * __Perl__ : by Gray, at http://search.cpan.org/dist/Compress-LZ4/ |
| * __C#__ : by Milosz Krajewski, at http://lz4net.codeplex.com |
| * __C#__ streaming : by Phill Djonov, at https://github.com/pdjonov/Lz4Stream |
| * __Go__ : by Branimir Karadzic, at https://github.com/bkaradzic/go-lz4 |
| * __Ruby__ : by Komiya Atsushi, at http://rubygems.org/gems/lz4-ruby |
| * __PHP__ : by Kamijo, at https://github.com/kjdev/php-ext-lz4 |
| * __Lua__ : by Christophe Delord, at http://cdsoft.fr/bl/bonaluna.html |
| * __Haskell__ : by Mark Wotton, at http://hackage.haskell.org/package/lz4 |
| * __Erlang__ : by Tetsuya Suzuki, at https://github.com/szktty/erlang-lz4 |
| * __Smalltalk__ (Pharo) : by Mariano Martinez Peck, at http://smalltalkhub.com/#!/~marianopeck/LZ4/ |
| * __OCaml__ : by Peter Zotov, at https://github.com/whitequark/ocaml-lz4 |
| * __Rust__ : by Alex Crichton, at http://alexcrichton.com/rust-compress/compress/lz4/index.html |
| * __8088 assembly__ decoder, by Jim Leonard, at http://www.oldskool.org/pc/lz4_8088 |
| * __6502 & 65C02 assembly__ decoder, by Peter Ferrie, at http://pferrie.host22.com/misc/appleii.htm |
| * __Atari XL/XE assembly__ decoder, by xxl, at http://xxl.atari.pl/?p=1524 |
| * __65c816 assembly__ decoder, by Olivier Zardini, at http://www.brutaldeluxe.fr/products/crossdevtools/lz4/index.html |
| |
| <a name="lz4-is-used-by"></a> |
| ## LZ4 is used by |
| |
| * __Operating Systems__ :  [Linux](https://github.com/torvalds/linux),  [FreeBSD](http://www.freebsd.org/),  [Illumos](http://wiki.illumos.org/display/illumos/illumos+Home),  [SmartOS](http://smartos.org/) |
| * __File Systems__ : OpenZFS, Hammer2, SquashFS, LessFS, LeoFS, Grub |
| * __Big Data__ : Hadoop, Cassandra, Hbase, Rarelogic, SparkSpark, Hustle |
| * __Search Engine__ : Lucene, Solr, Scalyr, kafka |
| * __Database__ : MySQL, Tokudb, Delphix, infiniSQL, RocksDB, OlegDB |
| * __Games__ : Battlefield 4, Guild Wars 2, Sine Mora, 1000 Tiny Claws |
| * __Graphics__ : nVidia, Enlightenment, Xpra, OpenVDB, Scaplib, kanzi |
| * __Network__ : freeNAS, openVPN, Netty, dovecot, virtualHere |
| * __Storage__ : Datastax, Nimble Storage, Nexenta, K2 Spear, Syneto OS |
| * __Caching__ : Zram, HHVM, PHP Zend Optimizer |
| * __Other__ : Mozilla Firefox, Bareos, GTKWave, Blosc, Facebook's Mercurial, Nippy, systemd |
| |
| ## Chart 1 |
| <div id="table_div"></div> |
| |
| ## Chart 2 |
| <div id="table_2" style="height:40em"></div> |
| |
| </xmp> |
| |
| <!-- jQuery (necessary for Bootstrap's JavaScript plugins) --> |
| <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js"></script> |
| <!-- Include all compiled plugins (below), or include individual files as needed --> |
| <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script> |
| <script src="strapdown/v/0.2/strapdown.js"></script> |
| </body> |
| </html> |
