first commit
This commit is contained in:
s.golasch
24
build/node_modules/node-zopfli/zopfli/CONTRIBUTING.md
generated
vendored
Normal file
24
build/node_modules/node-zopfli/zopfli/CONTRIBUTING.md
generated
vendored
Normal file
@@ -0,0 +1,24 @@
|
||||
Want to contribute? Great! First, read this page (including the small print at the end).
|
||||
|
||||
### Before you contribute
|
||||
Before we can use your code, you must sign the
|
||||
[Google Individual Contributor License Agreement](https://developers.google.com/open-source/cla/individual?csw=1)
|
||||
(CLA), which you can do online. The CLA is necessary mainly because you own the
|
||||
copyright to your changes, even after your contribution becomes part of our
|
||||
codebase, so we need your permission to use and distribute your code. We also
|
||||
need to be sure of various other things—for instance that you'll tell us if you
|
||||
know that your code infringes on other people's patents. You don't have to sign
|
||||
the CLA until after you've submitted your code for review and a member has
|
||||
approved it, but you must do it before we can put your code into our codebase.
|
||||
Before you start working on a larger contribution, you should get in touch with
|
||||
us first through the issue tracker with your idea so that we can help out and
|
||||
possibly guide you. Coordinating up front makes it much easier to avoid
|
||||
frustration later on.
|
||||
|
||||
### Code reviews
|
||||
All submissions, including submissions by project members, require review. We
|
||||
use Github pull requests for this purpose.
|
||||
|
||||
### The small print
|
||||
Contributions made by corporations are covered by a different agreement than
|
||||
the one above, the Software Grant and Corporate Contributor License Agreement.
|
||||
9
build/node_modules/node-zopfli/zopfli/CONTRIBUTORS
generated
vendored
Normal file
9
build/node_modules/node-zopfli/zopfli/CONTRIBUTORS
generated
vendored
Normal file
@@ -0,0 +1,9 @@
|
||||
Mark Adler
|
||||
Jyrki Alakuijala
|
||||
Frédéric Kayser
|
||||
Jeffrey Lim
|
||||
Daniel Reed
|
||||
Huzaifa Sidhpurwala
|
||||
Péter Szabó
|
||||
Lode Vandevenne
|
||||
Derek Buitenhuis
|
||||
201
build/node_modules/node-zopfli/zopfli/COPYING
generated
vendored
Normal file
201
build/node_modules/node-zopfli/zopfli/COPYING
generated
vendored
Normal file
@@ -0,0 +1,201 @@
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright 2011 Google Inc.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
42
build/node_modules/node-zopfli/zopfli/Makefile
generated
vendored
Normal file
42
build/node_modules/node-zopfli/zopfli/Makefile
generated
vendored
Normal file
@@ -0,0 +1,42 @@
|
||||
CC = gcc
|
||||
CXX = g++
|
||||
|
||||
CFLAGS = -W -Wall -Wextra -ansi -pedantic -lm -O2 -Wno-unused-function
|
||||
CXXFLAGS = -W -Wall -Wextra -ansi -pedantic -O2
|
||||
|
||||
ZOPFLILIB_SRC = src/zopfli/blocksplitter.c src/zopfli/cache.c\
|
||||
src/zopfli/deflate.c src/zopfli/gzip_container.c\
|
||||
src/zopfli/hash.c src/zopfli/katajainen.c\
|
||||
src/zopfli/lz77.c src/zopfli/squeeze.c\
|
||||
src/zopfli/tree.c src/zopfli/util.c\
|
||||
src/zopfli/zlib_container.c src/zopfli/zopfli_lib.c
|
||||
ZOPFLILIB_OBJ := $(patsubst src/zopfli/%.c,%.o,$(ZOPFLILIB_SRC))
|
||||
ZOPFLIBIN_SRC := src/zopfli/zopfli_bin.c
|
||||
LODEPNG_SRC := src/zopflipng/lodepng/lodepng.cpp src/zopflipng/lodepng/lodepng_util.cpp
|
||||
ZOPFLIPNGLIB_SRC := src/zopflipng/zopflipng_lib.cc
|
||||
ZOPFLIPNGBIN_SRC := src/zopflipng/zopflipng_bin.cc
|
||||
|
||||
.PHONY: zopfli zopflipng
|
||||
|
||||
# Zopfli binary
|
||||
zopfli:
|
||||
$(CC) $(ZOPFLILIB_SRC) $(ZOPFLIBIN_SRC) $(CFLAGS) -o zopfli
|
||||
|
||||
# Zopfli shared library
|
||||
libzopfli:
|
||||
$(CC) $(ZOPFLILIB_SRC) $(CFLAGS) -fPIC -c
|
||||
$(CC) $(ZOPFLILIB_OBJ) $(CFLAGS) -shared -Wl,-soname,libzopfli.so.1 -o libzopfli.so.1.0.1
|
||||
|
||||
# ZopfliPNG binary
|
||||
zopflipng:
|
||||
$(CC) $(ZOPFLILIB_SRC) $(CFLAGS) -c
|
||||
$(CXX) $(ZOPFLILIB_OBJ) $(LODEPNG_SRC) $(ZOPFLIPNGLIB_SRC) $(ZOPFLIPNGBIN_SRC) $(CFLAGS) -o zopflipng
|
||||
|
||||
# ZopfliPNG shared library
|
||||
libzopflipng:
|
||||
$(CC) $(ZOPFLILIB_SRC) $(CFLAGS) -fPIC -c
|
||||
$(CXX) $(ZOPFLILIB_OBJ) $(LODEPNG_SRC) $(ZOPFLIPNGLIB_SRC) $(CFLAGS) -fPIC --shared -Wl,-soname,libzopflipng.so.1 -o libzopflipng.so.1.0.0
|
||||
|
||||
# Remove all libraries and binaries
|
||||
clean:
|
||||
rm -f zopflipng zopfli $(ZOPFLILIB_OBJ) libzopfli*
|
||||
39
build/node_modules/node-zopfli/zopfli/README
generated
vendored
Normal file
39
build/node_modules/node-zopfli/zopfli/README
generated
vendored
Normal file
@@ -0,0 +1,39 @@
|
||||
Zopfli Compression Algorithm is a compression library programmed in C to perform
|
||||
very good, but slow, deflate or zlib compression.
|
||||
|
||||
The basic function to compress data is ZopfliCompress in zopfli.h. Use the
|
||||
ZopfliOptions object to set parameters that affect the speed and compression.
|
||||
Use the ZopfliInitOptions function to place the default values in the
|
||||
ZopfliOptions first.
|
||||
|
||||
ZopfliCompress supports deflate, gzip and zlib output format with a parameter.
|
||||
To support only one individual format, you can instead use ZopfliDeflate in
|
||||
deflate.h, ZopfliZlibCompress in zlib_container.h or ZopfliGzipCompress in
|
||||
gzip_container.h.
|
||||
|
||||
ZopfliDeflate creates a valid deflate stream in memory, see:
|
||||
http://www.ietf.org/rfc/rfc1951.txt
|
||||
ZopfliZlibCompress creates a valid zlib stream in memory, see:
|
||||
http://www.ietf.org/rfc/rfc1950.txt
|
||||
ZopfliGzipCompress creates a valid gzip stream in memory, see:
|
||||
http://www.ietf.org/rfc/rfc1952.txt
|
||||
|
||||
This library can only compress, not decompress. Existing zlib or deflate
|
||||
libraries can decompress the data.
|
||||
|
||||
zopfli_bin.c is separate from the library and contains an example program to
|
||||
create very well compressed gzip files. Currently the makefile builds this
|
||||
program with the library statically linked in.
|
||||
|
||||
The source code of Zopfli is under src/zopfli. Build instructions:
|
||||
|
||||
To build zopfli, compile all .c source files under src/zopfli to a single binary
|
||||
with C, and link to the standard C math library, e.g.:
|
||||
gcc src/zopfli/*.c -O2 -W -Wall -Wextra -Wno-unused-function -ansi -pedantic -lm -o zopfli
|
||||
|
||||
A makefile is provided as well, but only for linux. Use "make" to build the
|
||||
binary, "make libzopfli" to build it as a shared library. For other platforms,
|
||||
please use the build instructions above instead.
|
||||
|
||||
Zopfli Compression Algorithm was created by Lode Vandevenne and Jyrki
|
||||
Alakuijala, based on an algorithm by Jyrki Alakuijala.
|
||||
54
build/node_modules/node-zopfli/zopfli/README.zopflipng
generated
vendored
Normal file
54
build/node_modules/node-zopfli/zopfli/README.zopflipng
generated
vendored
Normal file
@@ -0,0 +1,54 @@
|
||||
ZopfliPNG is a command line program to optimize the Portable Network Graphics
|
||||
(PNG) images. This version has the following features:
|
||||
- uses Zopfli compression for the Deflate compression,
|
||||
- compares several strategies for choosing scanline filter codes,
|
||||
- chooses a suitable color type to losslessly encode the image,
|
||||
- removes all chunks that are unimportant for the typical web use (metadata,
|
||||
text, etc...),
|
||||
- optionally alters the hidden colors of fully transparent pixels for more
|
||||
compression, and,
|
||||
- optionally converts 16-bit color channels to 8-bit.
|
||||
|
||||
This is an alpha-release for testing while improvements, particularly to add
|
||||
palette selection, are still being made. Feedback and bug reports are welcome.
|
||||
|
||||
Important:
|
||||
|
||||
This PNG optimizer removes ancillary chunks (pieces of metadata) from the
|
||||
PNG image that normally do not affect rendering. However in special
|
||||
circumstances you may wish to keep some. For example for a design using
|
||||
custom gamma correction, keeping it may be desired. Visually check in the
|
||||
target renderer after using ZopfliPNG. Use --keepchunks to keep chunks, e.g.
|
||||
--keepchunks=gAMA,pHYs to keep gamma and DPI information. This will increase
|
||||
file size. The following page contains a list of ancillary PNG chunks:
|
||||
http://www.libpng.org/pub/png/spec/1.2/PNG-Chunks.html
|
||||
|
||||
Build instructions:
|
||||
|
||||
To build ZopfliPNG, compile all .c, .cc and .cpp files from src/zopfli,
|
||||
src/zopflipng and src/zopflipng/lodepng, except src/zopfli/zopfli_bin.c, to a
|
||||
single binary with C++, e.g.:
|
||||
g++ src/zopfli/{blocksplitter,cache,deflate,gzip_container,hash,katajainen,lz77,squeeze,tree,util,zlib_container,zopfli_lib}.c src/zopflipng/*.cc src/zopflipng/lodepng/*.cpp -O2 -W -Wall -Wextra -Wno-unused-function -ansi -pedantic -o zopflipng
|
||||
|
||||
A makefile is provided as well, but only for linux: use "make zopflipng" with
|
||||
the Zopfli makefile. For other platforms, please use the build instructions
|
||||
above instead.
|
||||
|
||||
The main compression algorithm in ZopfliPNG is ported from WebP lossless, but
|
||||
naturally cannot give as much compression gain for PNGs as it does for a more
|
||||
modern compression codec like WebP
|
||||
https://developers.google.com/speed/webp/docs/webp_lossless_bitstream_specification.
|
||||
|
||||
Compared to libpng -- an often used PNG encoder implementation -- ZopfliPNG uses
|
||||
2-3 orders of magnitude more CPU time for compression. Initial testing using a
|
||||
corpus of 1000 PNGs with translucency, randomly selected from the internet,
|
||||
gives a compression improvement of 12% compared to convert -q 95, but only 0.5%
|
||||
compared to pngout (from better of /f0 and /f5 runs).
|
||||
|
||||
By releasing this software we hope to make images on the web load faster without
|
||||
a new image format, but the opportunities for optimization within PNG are
|
||||
limited. When targeting Android, Chrome, Opera, and Yandex browsers, or by using
|
||||
suitable plugins for other browsers, it is good to note that WebP lossless
|
||||
images are still 26 % smaller than images recompressed with ZopfliPNG.
|
||||
|
||||
2013-05-07, Lode Vandevenne and Jyrki Alakuijala
|
||||
332
build/node_modules/node-zopfli/zopfli/src/zopfli/blocksplitter.c
generated
vendored
Normal file
332
build/node_modules/node-zopfli/zopfli/src/zopfli/blocksplitter.c
generated
vendored
Normal file
@@ -0,0 +1,332 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
#include "blocksplitter.h"
|
||||
|
||||
#include <assert.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
|
||||
#include "deflate.h"
|
||||
#include "squeeze.h"
|
||||
#include "tree.h"
|
||||
#include "util.h"
|
||||
|
||||
/*
|
||||
The "f" for the FindMinimum function below.
|
||||
i: the current parameter of f(i)
|
||||
context: for your implementation
|
||||
*/
|
||||
typedef double FindMinimumFun(size_t i, void* context);
|
||||
|
||||
/*
|
||||
Finds minimum of function f(i) where is is of type size_t, f(i) is of type
|
||||
double, i is in range start-end (excluding end).
|
||||
Outputs the minimum value in *smallest and returns the index of this value.
|
||||
*/
|
||||
static size_t FindMinimum(FindMinimumFun f, void* context,
|
||||
size_t start, size_t end, double* smallest) {
|
||||
if (end - start < 1024) {
|
||||
double best = ZOPFLI_LARGE_FLOAT;
|
||||
size_t result = start;
|
||||
size_t i;
|
||||
for (i = start; i < end; i++) {
|
||||
double v = f(i, context);
|
||||
if (v < best) {
|
||||
best = v;
|
||||
result = i;
|
||||
}
|
||||
}
|
||||
*smallest = best;
|
||||
return result;
|
||||
} else {
|
||||
/* Try to find minimum faster by recursively checking multiple points. */
|
||||
#define NUM 9 /* Good value: 9. */
|
||||
size_t i;
|
||||
size_t p[NUM];
|
||||
double vp[NUM];
|
||||
size_t besti;
|
||||
double best;
|
||||
double lastbest = ZOPFLI_LARGE_FLOAT;
|
||||
size_t pos = start;
|
||||
|
||||
for (;;) {
|
||||
if (end - start <= NUM) break;
|
||||
|
||||
for (i = 0; i < NUM; i++) {
|
||||
p[i] = start + (i + 1) * ((end - start) / (NUM + 1));
|
||||
vp[i] = f(p[i], context);
|
||||
}
|
||||
besti = 0;
|
||||
best = vp[0];
|
||||
for (i = 1; i < NUM; i++) {
|
||||
if (vp[i] < best) {
|
||||
best = vp[i];
|
||||
besti = i;
|
||||
}
|
||||
}
|
||||
if (best > lastbest) break;
|
||||
|
||||
start = besti == 0 ? start : p[besti - 1];
|
||||
end = besti == NUM - 1 ? end : p[besti + 1];
|
||||
|
||||
pos = p[besti];
|
||||
lastbest = best;
|
||||
}
|
||||
*smallest = lastbest;
|
||||
return pos;
|
||||
#undef NUM
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Returns estimated cost of a block in bits. It includes the size to encode the
|
||||
tree and the size to encode all literal, length and distance symbols and their
|
||||
extra bits.
|
||||
|
||||
litlens: lz77 lit/lengths
|
||||
dists: ll77 distances
|
||||
lstart: start of block
|
||||
lend: end of block (not inclusive)
|
||||
*/
|
||||
static double EstimateCost(const ZopfliLZ77Store* lz77,
|
||||
size_t lstart, size_t lend) {
|
||||
return ZopfliCalculateBlockSizeAutoType(lz77, lstart, lend);
|
||||
}
|
||||
|
||||
typedef struct SplitCostContext {
|
||||
const ZopfliLZ77Store* lz77;
|
||||
size_t start;
|
||||
size_t end;
|
||||
} SplitCostContext;
|
||||
|
||||
|
||||
/*
|
||||
Gets the cost which is the sum of the cost of the left and the right section
|
||||
of the data.
|
||||
type: FindMinimumFun
|
||||
*/
|
||||
static double SplitCost(size_t i, void* context) {
|
||||
SplitCostContext* c = (SplitCostContext*)context;
|
||||
return EstimateCost(c->lz77, c->start, i) + EstimateCost(c->lz77, i, c->end);
|
||||
}
|
||||
|
||||
static void AddSorted(size_t value, size_t** out, size_t* outsize) {
|
||||
size_t i;
|
||||
ZOPFLI_APPEND_DATA(value, out, outsize);
|
||||
for (i = 0; i + 1 < *outsize; i++) {
|
||||
if ((*out)[i] > value) {
|
||||
size_t j;
|
||||
for (j = *outsize - 1; j > i; j--) {
|
||||
(*out)[j] = (*out)[j - 1];
|
||||
}
|
||||
(*out)[i] = value;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Prints the block split points as decimal and hex values in the terminal.
|
||||
*/
|
||||
static void PrintBlockSplitPoints(const ZopfliLZ77Store* lz77,
|
||||
const size_t* lz77splitpoints,
|
||||
size_t nlz77points) {
|
||||
size_t* splitpoints = 0;
|
||||
size_t npoints = 0;
|
||||
size_t i;
|
||||
/* The input is given as lz77 indices, but we want to see the uncompressed
|
||||
index values. */
|
||||
size_t pos = 0;
|
||||
if (nlz77points > 0) {
|
||||
for (i = 0; i < lz77->size; i++) {
|
||||
size_t length = lz77->dists[i] == 0 ? 1 : lz77->litlens[i];
|
||||
if (lz77splitpoints[npoints] == i) {
|
||||
ZOPFLI_APPEND_DATA(pos, &splitpoints, &npoints);
|
||||
if (npoints == nlz77points) break;
|
||||
}
|
||||
pos += length;
|
||||
}
|
||||
}
|
||||
assert(npoints == nlz77points);
|
||||
|
||||
fprintf(stderr, "block split points: ");
|
||||
for (i = 0; i < npoints; i++) {
|
||||
fprintf(stderr, "%d ", (int)splitpoints[i]);
|
||||
}
|
||||
fprintf(stderr, "(hex:");
|
||||
for (i = 0; i < npoints; i++) {
|
||||
fprintf(stderr, " %x", (int)splitpoints[i]);
|
||||
}
|
||||
fprintf(stderr, ")\n");
|
||||
|
||||
free(splitpoints);
|
||||
}
|
||||
|
||||
/*
|
||||
Finds next block to try to split, the largest of the available ones.
|
||||
The largest is chosen to make sure that if only a limited amount of blocks is
|
||||
requested, their sizes are spread evenly.
|
||||
lz77size: the size of the LL77 data, which is the size of the done array here.
|
||||
done: array indicating which blocks starting at that position are no longer
|
||||
splittable (splitting them increases rather than decreases cost).
|
||||
splitpoints: the splitpoints found so far.
|
||||
npoints: the amount of splitpoints found so far.
|
||||
lstart: output variable, giving start of block.
|
||||
lend: output variable, giving end of block.
|
||||
returns 1 if a block was found, 0 if no block found (all are done).
|
||||
*/
|
||||
static int FindLargestSplittableBlock(
|
||||
size_t lz77size, const unsigned char* done,
|
||||
const size_t* splitpoints, size_t npoints,
|
||||
size_t* lstart, size_t* lend) {
|
||||
size_t longest = 0;
|
||||
int found = 0;
|
||||
size_t i;
|
||||
for (i = 0; i <= npoints; i++) {
|
||||
size_t start = i == 0 ? 0 : splitpoints[i - 1];
|
||||
size_t end = i == npoints ? lz77size - 1 : splitpoints[i];
|
||||
if (!done[start] && end - start > longest) {
|
||||
*lstart = start;
|
||||
*lend = end;
|
||||
found = 1;
|
||||
longest = end - start;
|
||||
}
|
||||
}
|
||||
return found;
|
||||
}
|
||||
|
||||
void ZopfliBlockSplitLZ77(const ZopfliOptions* options,
|
||||
const ZopfliLZ77Store* lz77, size_t maxblocks,
|
||||
size_t** splitpoints, size_t* npoints) {
|
||||
size_t lstart, lend;
|
||||
size_t i;
|
||||
size_t llpos = 0;
|
||||
size_t numblocks = 1;
|
||||
unsigned char* done;
|
||||
double splitcost, origcost;
|
||||
|
||||
if (lz77->size < 10) return; /* This code fails on tiny files. */
|
||||
|
||||
done = (unsigned char*)malloc(lz77->size);
|
||||
if (!done) exit(-1); /* Allocation failed. */
|
||||
for (i = 0; i < lz77->size; i++) done[i] = 0;
|
||||
|
||||
lstart = 0;
|
||||
lend = lz77->size;
|
||||
for (;;) {
|
||||
SplitCostContext c;
|
||||
|
||||
if (maxblocks > 0 && numblocks >= maxblocks) {
|
||||
break;
|
||||
}
|
||||
|
||||
c.lz77 = lz77;
|
||||
c.start = lstart;
|
||||
c.end = lend;
|
||||
assert(lstart < lend);
|
||||
llpos = FindMinimum(SplitCost, &c, lstart + 1, lend, &splitcost);
|
||||
|
||||
assert(llpos > lstart);
|
||||
assert(llpos < lend);
|
||||
|
||||
origcost = EstimateCost(lz77, lstart, lend);
|
||||
|
||||
if (splitcost > origcost || llpos == lstart + 1 || llpos == lend) {
|
||||
done[lstart] = 1;
|
||||
} else {
|
||||
AddSorted(llpos, splitpoints, npoints);
|
||||
numblocks++;
|
||||
}
|
||||
|
||||
if (!FindLargestSplittableBlock(
|
||||
lz77->size, done, *splitpoints, *npoints, &lstart, &lend)) {
|
||||
break; /* No further split will probably reduce compression. */
|
||||
}
|
||||
|
||||
if (lend - lstart < 10) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (options->verbose) {
|
||||
PrintBlockSplitPoints(lz77, *splitpoints, *npoints);
|
||||
}
|
||||
|
||||
free(done);
|
||||
}
|
||||
|
||||
void ZopfliBlockSplit(const ZopfliOptions* options,
|
||||
const unsigned char* in, size_t instart, size_t inend,
|
||||
size_t maxblocks, size_t** splitpoints, size_t* npoints) {
|
||||
size_t pos = 0;
|
||||
size_t i;
|
||||
ZopfliBlockState s;
|
||||
size_t* lz77splitpoints = 0;
|
||||
size_t nlz77points = 0;
|
||||
ZopfliLZ77Store store;
|
||||
ZopfliHash hash;
|
||||
ZopfliHash* h = &hash;
|
||||
|
||||
ZopfliInitLZ77Store(in, &store);
|
||||
ZopfliInitBlockState(options, instart, inend, 0, &s);
|
||||
ZopfliAllocHash(ZOPFLI_WINDOW_SIZE, h);
|
||||
|
||||
*npoints = 0;
|
||||
*splitpoints = 0;
|
||||
|
||||
/* Unintuitively, Using a simple LZ77 method here instead of ZopfliLZ77Optimal
|
||||
results in better blocks. */
|
||||
ZopfliLZ77Greedy(&s, in, instart, inend, &store, h);
|
||||
|
||||
ZopfliBlockSplitLZ77(options,
|
||||
&store, maxblocks,
|
||||
&lz77splitpoints, &nlz77points);
|
||||
|
||||
/* Convert LZ77 positions to positions in the uncompressed input. */
|
||||
pos = instart;
|
||||
if (nlz77points > 0) {
|
||||
for (i = 0; i < store.size; i++) {
|
||||
size_t length = store.dists[i] == 0 ? 1 : store.litlens[i];
|
||||
if (lz77splitpoints[*npoints] == i) {
|
||||
ZOPFLI_APPEND_DATA(pos, splitpoints, npoints);
|
||||
if (*npoints == nlz77points) break;
|
||||
}
|
||||
pos += length;
|
||||
}
|
||||
}
|
||||
assert(*npoints == nlz77points);
|
||||
|
||||
free(lz77splitpoints);
|
||||
ZopfliCleanBlockState(&s);
|
||||
ZopfliCleanLZ77Store(&store);
|
||||
ZopfliCleanHash(h);
|
||||
}
|
||||
|
||||
void ZopfliBlockSplitSimple(const unsigned char* in,
|
||||
size_t instart, size_t inend,
|
||||
size_t blocksize,
|
||||
size_t** splitpoints, size_t* npoints) {
|
||||
size_t i = instart;
|
||||
while (i < inend) {
|
||||
ZOPFLI_APPEND_DATA(i, splitpoints, npoints);
|
||||
i += blocksize;
|
||||
}
|
||||
(void)in;
|
||||
}
|
||||
73
build/node_modules/node-zopfli/zopfli/src/zopfli/blocksplitter.h
generated
vendored
Normal file
73
build/node_modules/node-zopfli/zopfli/src/zopfli/blocksplitter.h
generated
vendored
Normal file
@@ -0,0 +1,73 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
/*
|
||||
Functions to choose good boundaries for block splitting. Deflate allows encoding
|
||||
the data in multiple blocks, with a separate Huffman tree for each block. The
|
||||
Huffman tree itself requires some bytes to encode, so by choosing certain
|
||||
blocks, you can either hurt, or enhance compression. These functions choose good
|
||||
ones that enhance it.
|
||||
*/
|
||||
|
||||
#ifndef ZOPFLI_BLOCKSPLITTER_H_
|
||||
#define ZOPFLI_BLOCKSPLITTER_H_
|
||||
|
||||
#include <stdlib.h>
|
||||
|
||||
#include "lz77.h"
|
||||
#include "zopfli.h"
|
||||
|
||||
|
||||
/*
|
||||
Does blocksplitting on LZ77 data.
|
||||
The output splitpoints are indices in the LZ77 data.
|
||||
maxblocks: set a limit to the amount of blocks. Set to 0 to mean no limit.
|
||||
*/
|
||||
void ZopfliBlockSplitLZ77(const ZopfliOptions* options,
|
||||
const ZopfliLZ77Store* lz77, size_t maxblocks,
|
||||
size_t** splitpoints, size_t* npoints);
|
||||
|
||||
/*
|
||||
Does blocksplitting on uncompressed data.
|
||||
The output splitpoints are indices in the uncompressed bytes.
|
||||
|
||||
options: general program options.
|
||||
in: uncompressed input data
|
||||
instart: where to start splitting
|
||||
inend: where to end splitting (not inclusive)
|
||||
maxblocks: maximum amount of blocks to split into, or 0 for no limit
|
||||
splitpoints: dynamic array to put the resulting split point coordinates into.
|
||||
The coordinates are indices in the input array.
|
||||
npoints: pointer to amount of splitpoints, for the dynamic array. The amount of
|
||||
blocks is the amount of splitpoitns + 1.
|
||||
*/
|
||||
void ZopfliBlockSplit(const ZopfliOptions* options,
|
||||
const unsigned char* in, size_t instart, size_t inend,
|
||||
size_t maxblocks, size_t** splitpoints, size_t* npoints);
|
||||
|
||||
/*
|
||||
Divides the input into equal blocks, does not even take LZ77 lengths into
|
||||
account.
|
||||
*/
|
||||
void ZopfliBlockSplitSimple(const unsigned char* in,
|
||||
size_t instart, size_t inend,
|
||||
size_t blocksize,
|
||||
size_t** splitpoints, size_t* npoints);
|
||||
|
||||
#endif /* ZOPFLI_BLOCKSPLITTER_H_ */
|
||||
125
build/node_modules/node-zopfli/zopfli/src/zopfli/cache.c
generated
vendored
Normal file
125
build/node_modules/node-zopfli/zopfli/src/zopfli/cache.c
generated
vendored
Normal file
@@ -0,0 +1,125 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
#include "cache.h"
|
||||
|
||||
#include <assert.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
|
||||
#ifdef ZOPFLI_LONGEST_MATCH_CACHE
|
||||
|
||||
void ZopfliInitCache(size_t blocksize, ZopfliLongestMatchCache* lmc) {
|
||||
size_t i;
|
||||
lmc->length = (unsigned short*)malloc(sizeof(unsigned short) * blocksize);
|
||||
lmc->dist = (unsigned short*)malloc(sizeof(unsigned short) * blocksize);
|
||||
/* Rather large amount of memory. */
|
||||
lmc->sublen = (unsigned char*)malloc(ZOPFLI_CACHE_LENGTH * 3 * blocksize);
|
||||
if(lmc->sublen == NULL) {
|
||||
fprintf(stderr,
|
||||
"Error: Out of memory. Tried allocating %lu bytes of memory.\n",
|
||||
ZOPFLI_CACHE_LENGTH * 3 * blocksize);
|
||||
exit (EXIT_FAILURE);
|
||||
}
|
||||
|
||||
/* length > 0 and dist 0 is invalid combination, which indicates on purpose
|
||||
that this cache value is not filled in yet. */
|
||||
for (i = 0; i < blocksize; i++) lmc->length[i] = 1;
|
||||
for (i = 0; i < blocksize; i++) lmc->dist[i] = 0;
|
||||
for (i = 0; i < ZOPFLI_CACHE_LENGTH * blocksize * 3; i++) lmc->sublen[i] = 0;
|
||||
}
|
||||
|
||||
void ZopfliCleanCache(ZopfliLongestMatchCache* lmc) {
|
||||
free(lmc->length);
|
||||
free(lmc->dist);
|
||||
free(lmc->sublen);
|
||||
}
|
||||
|
||||
void ZopfliSublenToCache(const unsigned short* sublen,
|
||||
size_t pos, size_t length,
|
||||
ZopfliLongestMatchCache* lmc) {
|
||||
size_t i;
|
||||
size_t j = 0;
|
||||
unsigned bestlength = 0;
|
||||
unsigned char* cache;
|
||||
|
||||
#if ZOPFLI_CACHE_LENGTH == 0
|
||||
return;
|
||||
#endif
|
||||
|
||||
cache = &lmc->sublen[ZOPFLI_CACHE_LENGTH * pos * 3];
|
||||
if (length < 3) return;
|
||||
for (i = 3; i <= length; i++) {
|
||||
if (i == length || sublen[i] != sublen[i + 1]) {
|
||||
cache[j * 3] = i - 3;
|
||||
cache[j * 3 + 1] = sublen[i] % 256;
|
||||
cache[j * 3 + 2] = (sublen[i] >> 8) % 256;
|
||||
bestlength = i;
|
||||
j++;
|
||||
if (j >= ZOPFLI_CACHE_LENGTH) break;
|
||||
}
|
||||
}
|
||||
if (j < ZOPFLI_CACHE_LENGTH) {
|
||||
assert(bestlength == length);
|
||||
cache[(ZOPFLI_CACHE_LENGTH - 1) * 3] = bestlength - 3;
|
||||
} else {
|
||||
assert(bestlength <= length);
|
||||
}
|
||||
assert(bestlength == ZopfliMaxCachedSublen(lmc, pos, length));
|
||||
}
|
||||
|
||||
void ZopfliCacheToSublen(const ZopfliLongestMatchCache* lmc,
|
||||
size_t pos, size_t length,
|
||||
unsigned short* sublen) {
|
||||
size_t i, j;
|
||||
unsigned maxlength = ZopfliMaxCachedSublen(lmc, pos, length);
|
||||
unsigned prevlength = 0;
|
||||
unsigned char* cache;
|
||||
#if ZOPFLI_CACHE_LENGTH == 0
|
||||
return;
|
||||
#endif
|
||||
if (length < 3) return;
|
||||
cache = &lmc->sublen[ZOPFLI_CACHE_LENGTH * pos * 3];
|
||||
for (j = 0; j < ZOPFLI_CACHE_LENGTH; j++) {
|
||||
unsigned length = cache[j * 3] + 3;
|
||||
unsigned dist = cache[j * 3 + 1] + 256 * cache[j * 3 + 2];
|
||||
for (i = prevlength; i <= length; i++) {
|
||||
sublen[i] = dist;
|
||||
}
|
||||
if (length == maxlength) break;
|
||||
prevlength = length + 1;
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Returns the length up to which could be stored in the cache.
|
||||
*/
|
||||
unsigned ZopfliMaxCachedSublen(const ZopfliLongestMatchCache* lmc,
|
||||
size_t pos, size_t length) {
|
||||
unsigned char* cache;
|
||||
#if ZOPFLI_CACHE_LENGTH == 0
|
||||
return 0;
|
||||
#endif
|
||||
cache = &lmc->sublen[ZOPFLI_CACHE_LENGTH * pos * 3];
|
||||
(void)length;
|
||||
if (cache[1] == 0 && cache[2] == 0) return 0; /* No sublen cached. */
|
||||
return cache[(ZOPFLI_CACHE_LENGTH - 1) * 3] + 3;
|
||||
}
|
||||
|
||||
#endif /* ZOPFLI_LONGEST_MATCH_CACHE */
|
||||
66
build/node_modules/node-zopfli/zopfli/src/zopfli/cache.h
generated
vendored
Normal file
66
build/node_modules/node-zopfli/zopfli/src/zopfli/cache.h
generated
vendored
Normal file
@@ -0,0 +1,66 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
/*
|
||||
The cache that speeds up ZopfliFindLongestMatch of lz77.c.
|
||||
*/
|
||||
|
||||
#ifndef ZOPFLI_CACHE_H_
|
||||
#define ZOPFLI_CACHE_H_
|
||||
|
||||
#include "util.h"
|
||||
|
||||
#ifdef ZOPFLI_LONGEST_MATCH_CACHE
|
||||
|
||||
/*
|
||||
Cache used by ZopfliFindLongestMatch to remember previously found length/dist
|
||||
values.
|
||||
This is needed because the squeeze runs will ask these values multiple times for
|
||||
the same position.
|
||||
Uses large amounts of memory, since it has to remember the distance belonging
|
||||
to every possible shorter-than-the-best length (the so called "sublen" array).
|
||||
*/
|
||||
typedef struct ZopfliLongestMatchCache {
|
||||
unsigned short* length;
|
||||
unsigned short* dist;
|
||||
unsigned char* sublen;
|
||||
} ZopfliLongestMatchCache;
|
||||
|
||||
/* Initializes the ZopfliLongestMatchCache. */
|
||||
void ZopfliInitCache(size_t blocksize, ZopfliLongestMatchCache* lmc);
|
||||
|
||||
/* Frees up the memory of the ZopfliLongestMatchCache. */
|
||||
void ZopfliCleanCache(ZopfliLongestMatchCache* lmc);
|
||||
|
||||
/* Stores sublen array in the cache. */
|
||||
void ZopfliSublenToCache(const unsigned short* sublen,
|
||||
size_t pos, size_t length,
|
||||
ZopfliLongestMatchCache* lmc);
|
||||
|
||||
/* Extracts sublen array from the cache. */
|
||||
void ZopfliCacheToSublen(const ZopfliLongestMatchCache* lmc,
|
||||
size_t pos, size_t length,
|
||||
unsigned short* sublen);
|
||||
/* Returns the length up to which could be stored in the cache. */
|
||||
unsigned ZopfliMaxCachedSublen(const ZopfliLongestMatchCache* lmc,
|
||||
size_t pos, size_t length);
|
||||
|
||||
#endif /* ZOPFLI_LONGEST_MATCH_CACHE */
|
||||
|
||||
#endif /* ZOPFLI_CACHE_H_ */
|
||||
931
build/node_modules/node-zopfli/zopfli/src/zopfli/deflate.c
generated
vendored
Normal file
931
build/node_modules/node-zopfli/zopfli/src/zopfli/deflate.c
generated
vendored
Normal file
@@ -0,0 +1,931 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
#include "deflate.h"
|
||||
|
||||
#include <assert.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
|
||||
#include "blocksplitter.h"
|
||||
#include "squeeze.h"
|
||||
#include "symbols.h"
|
||||
#include "tree.h"
|
||||
|
||||
/*
|
||||
bp = bitpointer, always in range [0, 7].
|
||||
The outsize is number of necessary bytes to encode the bits.
|
||||
Given the value of bp and the amount of bytes, the amount of bits represented
|
||||
is not simply bytesize * 8 + bp because even representing one bit requires a
|
||||
whole byte. It is: (bp == 0) ? (bytesize * 8) : ((bytesize - 1) * 8 + bp)
|
||||
*/
|
||||
static void AddBit(int bit,
|
||||
unsigned char* bp, unsigned char** out, size_t* outsize) {
|
||||
if (*bp == 0) ZOPFLI_APPEND_DATA(0, out, outsize);
|
||||
(*out)[*outsize - 1] |= bit << *bp;
|
||||
*bp = (*bp + 1) & 7;
|
||||
}
|
||||
|
||||
static void AddBits(unsigned symbol, unsigned length,
|
||||
unsigned char* bp, unsigned char** out, size_t* outsize) {
|
||||
/* TODO(lode): make more efficient (add more bits at once). */
|
||||
unsigned i;
|
||||
for (i = 0; i < length; i++) {
|
||||
unsigned bit = (symbol >> i) & 1;
|
||||
if (*bp == 0) ZOPFLI_APPEND_DATA(0, out, outsize);
|
||||
(*out)[*outsize - 1] |= bit << *bp;
|
||||
*bp = (*bp + 1) & 7;
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Adds bits, like AddBits, but the order is inverted. The deflate specification
|
||||
uses both orders in one standard.
|
||||
*/
|
||||
static void AddHuffmanBits(unsigned symbol, unsigned length,
|
||||
unsigned char* bp, unsigned char** out,
|
||||
size_t* outsize) {
|
||||
/* TODO(lode): make more efficient (add more bits at once). */
|
||||
unsigned i;
|
||||
for (i = 0; i < length; i++) {
|
||||
unsigned bit = (symbol >> (length - i - 1)) & 1;
|
||||
if (*bp == 0) ZOPFLI_APPEND_DATA(0, out, outsize);
|
||||
(*out)[*outsize - 1] |= bit << *bp;
|
||||
*bp = (*bp + 1) & 7;
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Ensures there are at least 2 distance codes to support buggy decoders.
|
||||
Zlib 1.2.1 and below have a bug where it fails if there isn't at least 1
|
||||
distance code (with length > 0), even though it's valid according to the
|
||||
deflate spec to have 0 distance codes. On top of that, some mobile phones
|
||||
require at least two distance codes. To support these decoders too (but
|
||||
potentially at the cost of a few bytes), add dummy code lengths of 1.
|
||||
References to this bug can be found in the changelog of
|
||||
Zlib 1.2.2 and here: http://www.jonof.id.au/forum/index.php?topic=515.0.
|
||||
|
||||
d_lengths: the 32 lengths of the distance codes.
|
||||
*/
|
||||
static void PatchDistanceCodesForBuggyDecoders(unsigned* d_lengths) {
|
||||
int num_dist_codes = 0; /* Amount of non-zero distance codes */
|
||||
int i;
|
||||
for (i = 0; i < 30 /* Ignore the two unused codes from the spec */; i++) {
|
||||
if (d_lengths[i]) num_dist_codes++;
|
||||
if (num_dist_codes >= 2) return; /* Two or more codes is fine. */
|
||||
}
|
||||
|
||||
if (num_dist_codes == 0) {
|
||||
d_lengths[0] = d_lengths[1] = 1;
|
||||
} else if (num_dist_codes == 1) {
|
||||
d_lengths[d_lengths[0] ? 1 : 0] = 1;
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Encodes the Huffman tree and returns how many bits its encoding takes. If out
|
||||
is a null pointer, only returns the size and runs faster.
|
||||
*/
|
||||
static size_t EncodeTree(const unsigned* ll_lengths,
|
||||
const unsigned* d_lengths,
|
||||
int use_16, int use_17, int use_18,
|
||||
unsigned char* bp,
|
||||
unsigned char** out, size_t* outsize) {
|
||||
unsigned lld_total; /* Total amount of literal, length, distance codes. */
|
||||
/* Runlength encoded version of lengths of litlen and dist trees. */
|
||||
unsigned* rle = 0;
|
||||
unsigned* rle_bits = 0; /* Extra bits for rle values 16, 17 and 18. */
|
||||
size_t rle_size = 0; /* Size of rle array. */
|
||||
size_t rle_bits_size = 0; /* Should have same value as rle_size. */
|
||||
unsigned hlit = 29; /* 286 - 257 */
|
||||
unsigned hdist = 29; /* 32 - 1, but gzip does not like hdist > 29.*/
|
||||
unsigned hclen;
|
||||
unsigned hlit2;
|
||||
size_t i, j;
|
||||
size_t clcounts[19];
|
||||
unsigned clcl[19]; /* Code length code lengths. */
|
||||
unsigned clsymbols[19];
|
||||
/* The order in which code length code lengths are encoded as per deflate. */
|
||||
static const unsigned order[19] = {
|
||||
16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15
|
||||
};
|
||||
int size_only = !out;
|
||||
size_t result_size = 0;
|
||||
|
||||
for(i = 0; i < 19; i++) clcounts[i] = 0;
|
||||
|
||||
/* Trim zeros. */
|
||||
while (hlit > 0 && ll_lengths[257 + hlit - 1] == 0) hlit--;
|
||||
while (hdist > 0 && d_lengths[1 + hdist - 1] == 0) hdist--;
|
||||
hlit2 = hlit + 257;
|
||||
|
||||
lld_total = hlit2 + hdist + 1;
|
||||
|
||||
for (i = 0; i < lld_total; i++) {
|
||||
/* This is an encoding of a huffman tree, so now the length is a symbol */
|
||||
unsigned char symbol = i < hlit2 ? ll_lengths[i] : d_lengths[i - hlit2];
|
||||
unsigned count = 1;
|
||||
if(use_16 || (symbol == 0 && (use_17 || use_18))) {
|
||||
for (j = i + 1; j < lld_total && symbol ==
|
||||
(j < hlit2 ? ll_lengths[j] : d_lengths[j - hlit2]); j++) {
|
||||
count++;
|
||||
}
|
||||
}
|
||||
i += count - 1;
|
||||
|
||||
/* Repetitions of zeroes */
|
||||
if (symbol == 0 && count >= 3) {
|
||||
if (use_18) {
|
||||
while (count >= 11) {
|
||||
unsigned count2 = count > 138 ? 138 : count;
|
||||
if (!size_only) {
|
||||
ZOPFLI_APPEND_DATA(18, &rle, &rle_size);
|
||||
ZOPFLI_APPEND_DATA(count2 - 11, &rle_bits, &rle_bits_size);
|
||||
}
|
||||
clcounts[18]++;
|
||||
count -= count2;
|
||||
}
|
||||
}
|
||||
if (use_17) {
|
||||
while (count >= 3) {
|
||||
unsigned count2 = count > 10 ? 10 : count;
|
||||
if (!size_only) {
|
||||
ZOPFLI_APPEND_DATA(17, &rle, &rle_size);
|
||||
ZOPFLI_APPEND_DATA(count2 - 3, &rle_bits, &rle_bits_size);
|
||||
}
|
||||
clcounts[17]++;
|
||||
count -= count2;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* Repetitions of any symbol */
|
||||
if (use_16 && count >= 4) {
|
||||
count--; /* Since the first one is hardcoded. */
|
||||
clcounts[symbol]++;
|
||||
if (!size_only) {
|
||||
ZOPFLI_APPEND_DATA(symbol, &rle, &rle_size);
|
||||
ZOPFLI_APPEND_DATA(0, &rle_bits, &rle_bits_size);
|
||||
}
|
||||
while (count >= 3) {
|
||||
unsigned count2 = count > 6 ? 6 : count;
|
||||
if (!size_only) {
|
||||
ZOPFLI_APPEND_DATA(16, &rle, &rle_size);
|
||||
ZOPFLI_APPEND_DATA(count2 - 3, &rle_bits, &rle_bits_size);
|
||||
}
|
||||
clcounts[16]++;
|
||||
count -= count2;
|
||||
}
|
||||
}
|
||||
|
||||
/* No or insufficient repetition */
|
||||
clcounts[symbol] += count;
|
||||
while (count > 0) {
|
||||
if (!size_only) {
|
||||
ZOPFLI_APPEND_DATA(symbol, &rle, &rle_size);
|
||||
ZOPFLI_APPEND_DATA(0, &rle_bits, &rle_bits_size);
|
||||
}
|
||||
count--;
|
||||
}
|
||||
}
|
||||
|
||||
ZopfliCalculateBitLengths(clcounts, 19, 7, clcl);
|
||||
if (!size_only) ZopfliLengthsToSymbols(clcl, 19, 7, clsymbols);
|
||||
|
||||
hclen = 15;
|
||||
/* Trim zeros. */
|
||||
while (hclen > 0 && clcounts[order[hclen + 4 - 1]] == 0) hclen--;
|
||||
|
||||
if (!size_only) {
|
||||
AddBits(hlit, 5, bp, out, outsize);
|
||||
AddBits(hdist, 5, bp, out, outsize);
|
||||
AddBits(hclen, 4, bp, out, outsize);
|
||||
|
||||
for (i = 0; i < hclen + 4; i++) {
|
||||
AddBits(clcl[order[i]], 3, bp, out, outsize);
|
||||
}
|
||||
|
||||
for (i = 0; i < rle_size; i++) {
|
||||
unsigned symbol = clsymbols[rle[i]];
|
||||
AddHuffmanBits(symbol, clcl[rle[i]], bp, out, outsize);
|
||||
/* Extra bits. */
|
||||
if (rle[i] == 16) AddBits(rle_bits[i], 2, bp, out, outsize);
|
||||
else if (rle[i] == 17) AddBits(rle_bits[i], 3, bp, out, outsize);
|
||||
else if (rle[i] == 18) AddBits(rle_bits[i], 7, bp, out, outsize);
|
||||
}
|
||||
}
|
||||
|
||||
result_size += 14; /* hlit, hdist, hclen bits */
|
||||
result_size += (hclen + 4) * 3; /* clcl bits */
|
||||
for(i = 0; i < 19; i++) {
|
||||
result_size += clcl[i] * clcounts[i];
|
||||
}
|
||||
/* Extra bits. */
|
||||
result_size += clcounts[16] * 2;
|
||||
result_size += clcounts[17] * 3;
|
||||
result_size += clcounts[18] * 7;
|
||||
|
||||
/* Note: in case of "size_only" these are null pointers so no effect. */
|
||||
free(rle);
|
||||
free(rle_bits);
|
||||
|
||||
return result_size;
|
||||
}
|
||||
|
||||
static void AddDynamicTree(const unsigned* ll_lengths,
|
||||
const unsigned* d_lengths,
|
||||
unsigned char* bp,
|
||||
unsigned char** out, size_t* outsize) {
|
||||
int i;
|
||||
int best = 0;
|
||||
size_t bestsize = 0;
|
||||
|
||||
for(i = 0; i < 8; i++) {
|
||||
size_t size = EncodeTree(ll_lengths, d_lengths,
|
||||
i & 1, i & 2, i & 4,
|
||||
0, 0, 0);
|
||||
if (bestsize == 0 || size < bestsize) {
|
||||
bestsize = size;
|
||||
best = i;
|
||||
}
|
||||
}
|
||||
|
||||
EncodeTree(ll_lengths, d_lengths,
|
||||
best & 1, best & 2, best & 4,
|
||||
bp, out, outsize);
|
||||
}
|
||||
|
||||
/*
|
||||
Gives the exact size of the tree, in bits, as it will be encoded in DEFLATE.
|
||||
*/
|
||||
static size_t CalculateTreeSize(const unsigned* ll_lengths,
|
||||
const unsigned* d_lengths) {
|
||||
size_t result = 0;
|
||||
int i;
|
||||
|
||||
for(i = 0; i < 8; i++) {
|
||||
size_t size = EncodeTree(ll_lengths, d_lengths,
|
||||
i & 1, i & 2, i & 4,
|
||||
0, 0, 0);
|
||||
if (result == 0 || size < result) result = size;
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/*
|
||||
Adds all lit/len and dist codes from the lists as huffman symbols. Does not add
|
||||
end code 256. expected_data_size is the uncompressed block size, used for
|
||||
assert, but you can set it to 0 to not do the assertion.
|
||||
*/
|
||||
static void AddLZ77Data(const ZopfliLZ77Store* lz77,
|
||||
size_t lstart, size_t lend,
|
||||
size_t expected_data_size,
|
||||
const unsigned* ll_symbols, const unsigned* ll_lengths,
|
||||
const unsigned* d_symbols, const unsigned* d_lengths,
|
||||
unsigned char* bp,
|
||||
unsigned char** out, size_t* outsize) {
|
||||
size_t testlength = 0;
|
||||
size_t i;
|
||||
|
||||
for (i = lstart; i < lend; i++) {
|
||||
unsigned dist = lz77->dists[i];
|
||||
unsigned litlen = lz77->litlens[i];
|
||||
if (dist == 0) {
|
||||
assert(litlen < 256);
|
||||
assert(ll_lengths[litlen] > 0);
|
||||
AddHuffmanBits(ll_symbols[litlen], ll_lengths[litlen], bp, out, outsize);
|
||||
testlength++;
|
||||
} else {
|
||||
unsigned lls = ZopfliGetLengthSymbol(litlen);
|
||||
unsigned ds = ZopfliGetDistSymbol(dist);
|
||||
assert(litlen >= 3 && litlen <= 288);
|
||||
assert(ll_lengths[lls] > 0);
|
||||
assert(d_lengths[ds] > 0);
|
||||
AddHuffmanBits(ll_symbols[lls], ll_lengths[lls], bp, out, outsize);
|
||||
AddBits(ZopfliGetLengthExtraBitsValue(litlen),
|
||||
ZopfliGetLengthExtraBits(litlen),
|
||||
bp, out, outsize);
|
||||
AddHuffmanBits(d_symbols[ds], d_lengths[ds], bp, out, outsize);
|
||||
AddBits(ZopfliGetDistExtraBitsValue(dist),
|
||||
ZopfliGetDistExtraBits(dist),
|
||||
bp, out, outsize);
|
||||
testlength += litlen;
|
||||
}
|
||||
}
|
||||
assert(expected_data_size == 0 || testlength == expected_data_size);
|
||||
}
|
||||
|
||||
static void GetFixedTree(unsigned* ll_lengths, unsigned* d_lengths) {
|
||||
size_t i;
|
||||
for (i = 0; i < 144; i++) ll_lengths[i] = 8;
|
||||
for (i = 144; i < 256; i++) ll_lengths[i] = 9;
|
||||
for (i = 256; i < 280; i++) ll_lengths[i] = 7;
|
||||
for (i = 280; i < 288; i++) ll_lengths[i] = 8;
|
||||
for (i = 0; i < 32; i++) d_lengths[i] = 5;
|
||||
}
|
||||
|
||||
/*
|
||||
Same as CalculateBlockSymbolSize, but for block size smaller than histogram
|
||||
size.
|
||||
*/
|
||||
static size_t CalculateBlockSymbolSizeSmall(const unsigned* ll_lengths,
|
||||
const unsigned* d_lengths,
|
||||
const ZopfliLZ77Store* lz77,
|
||||
size_t lstart, size_t lend) {
|
||||
size_t result = 0;
|
||||
size_t i;
|
||||
for (i = lstart; i < lend; i++) {
|
||||
assert(i < lz77->size);
|
||||
assert(lz77->litlens[i] < 259);
|
||||
if (lz77->dists[i] == 0) {
|
||||
result += ll_lengths[lz77->litlens[i]];
|
||||
} else {
|
||||
int ll_symbol = ZopfliGetLengthSymbol(lz77->litlens[i]);
|
||||
int d_symbol = ZopfliGetDistSymbol(lz77->dists[i]);
|
||||
result += ll_lengths[ll_symbol];
|
||||
result += d_lengths[d_symbol];
|
||||
result += ZopfliGetLengthSymbolExtraBits(ll_symbol);
|
||||
result += ZopfliGetDistSymbolExtraBits(d_symbol);
|
||||
}
|
||||
}
|
||||
result += ll_lengths[256]; /*end symbol*/
|
||||
return result;
|
||||
}
|
||||
|
||||
/*
|
||||
Same as CalculateBlockSymbolSize, but with the histogram provided by the caller.
|
||||
*/
|
||||
static size_t CalculateBlockSymbolSizeGivenCounts(const size_t* ll_counts,
|
||||
const size_t* d_counts,
|
||||
const unsigned* ll_lengths,
|
||||
const unsigned* d_lengths,
|
||||
const ZopfliLZ77Store* lz77,
|
||||
size_t lstart, size_t lend) {
|
||||
size_t result = 0;
|
||||
size_t i;
|
||||
if (lstart + ZOPFLI_NUM_LL * 3 > lend) {
|
||||
return CalculateBlockSymbolSizeSmall(
|
||||
ll_lengths, d_lengths, lz77, lstart, lend);
|
||||
} else {
|
||||
for (i = 0; i < 256; i++) {
|
||||
result += ll_lengths[i] * ll_counts[i];
|
||||
}
|
||||
for (i = 257; i < 286; i++) {
|
||||
result += ll_lengths[i] * ll_counts[i];
|
||||
result += ZopfliGetLengthSymbolExtraBits(i) * ll_counts[i];
|
||||
}
|
||||
for (i = 0; i < 30; i++) {
|
||||
result += d_lengths[i] * d_counts[i];
|
||||
result += ZopfliGetDistSymbolExtraBits(i) * d_counts[i];
|
||||
}
|
||||
result += ll_lengths[256]; /*end symbol*/
|
||||
return result;
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Calculates size of the part after the header and tree of an LZ77 block, in bits.
|
||||
*/
|
||||
static size_t CalculateBlockSymbolSize(const unsigned* ll_lengths,
|
||||
const unsigned* d_lengths,
|
||||
const ZopfliLZ77Store* lz77,
|
||||
size_t lstart, size_t lend) {
|
||||
if (lstart + ZOPFLI_NUM_LL * 3 > lend) {
|
||||
return CalculateBlockSymbolSizeSmall(
|
||||
ll_lengths, d_lengths, lz77, lstart, lend);
|
||||
} else {
|
||||
size_t ll_counts[ZOPFLI_NUM_LL];
|
||||
size_t d_counts[ZOPFLI_NUM_D];
|
||||
ZopfliLZ77GetHistogram(lz77, lstart, lend, ll_counts, d_counts);
|
||||
return CalculateBlockSymbolSizeGivenCounts(
|
||||
ll_counts, d_counts, ll_lengths, d_lengths, lz77, lstart, lend);
|
||||
}
|
||||
}
|
||||
|
||||
static size_t AbsDiff(size_t x, size_t y) {
|
||||
if (x > y)
|
||||
return x - y;
|
||||
else
|
||||
return y - x;
|
||||
}
|
||||
|
||||
/*
|
||||
Changes the population counts in a way that the consequent Huffman tree
|
||||
compression, especially its rle-part, will be more likely to compress this data
|
||||
more efficiently. length contains the size of the histogram.
|
||||
*/
|
||||
void OptimizeHuffmanForRle(int length, size_t* counts) {
|
||||
int i, k, stride;
|
||||
size_t symbol, sum, limit;
|
||||
int* good_for_rle;
|
||||
|
||||
/* 1) We don't want to touch the trailing zeros. We may break the
|
||||
rules of the format by adding more data in the distance codes. */
|
||||
for (; length >= 0; --length) {
|
||||
if (length == 0) {
|
||||
return;
|
||||
}
|
||||
if (counts[length - 1] != 0) {
|
||||
/* Now counts[0..length - 1] does not have trailing zeros. */
|
||||
break;
|
||||
}
|
||||
}
|
||||
/* 2) Let's mark all population counts that already can be encoded
|
||||
with an rle code.*/
|
||||
good_for_rle = (int*)malloc(length * sizeof(int));
|
||||
for (i = 0; i < length; ++i) good_for_rle[i] = 0;
|
||||
|
||||
/* Let's not spoil any of the existing good rle codes.
|
||||
Mark any seq of 0's that is longer than 5 as a good_for_rle.
|
||||
Mark any seq of non-0's that is longer than 7 as a good_for_rle.*/
|
||||
symbol = counts[0];
|
||||
stride = 0;
|
||||
for (i = 0; i < length + 1; ++i) {
|
||||
if (i == length || counts[i] != symbol) {
|
||||
if ((symbol == 0 && stride >= 5) || (symbol != 0 && stride >= 7)) {
|
||||
for (k = 0; k < stride; ++k) {
|
||||
good_for_rle[i - k - 1] = 1;
|
||||
}
|
||||
}
|
||||
stride = 1;
|
||||
if (i != length) {
|
||||
symbol = counts[i];
|
||||
}
|
||||
} else {
|
||||
++stride;
|
||||
}
|
||||
}
|
||||
|
||||
/* 3) Let's replace those population counts that lead to more rle codes. */
|
||||
stride = 0;
|
||||
limit = counts[0];
|
||||
sum = 0;
|
||||
for (i = 0; i < length + 1; ++i) {
|
||||
if (i == length || good_for_rle[i]
|
||||
/* Heuristic for selecting the stride ranges to collapse. */
|
||||
|| AbsDiff(counts[i], limit) >= 4) {
|
||||
if (stride >= 4 || (stride >= 3 && sum == 0)) {
|
||||
/* The stride must end, collapse what we have, if we have enough (4). */
|
||||
int count = (sum + stride / 2) / stride;
|
||||
if (count < 1) count = 1;
|
||||
if (sum == 0) {
|
||||
/* Don't make an all zeros stride to be upgraded to ones. */
|
||||
count = 0;
|
||||
}
|
||||
for (k = 0; k < stride; ++k) {
|
||||
/* We don't want to change value at counts[i],
|
||||
that is already belonging to the next stride. Thus - 1. */
|
||||
counts[i - k - 1] = count;
|
||||
}
|
||||
}
|
||||
stride = 0;
|
||||
sum = 0;
|
||||
if (i < length - 3) {
|
||||
/* All interesting strides have a count of at least 4,
|
||||
at least when non-zeros. */
|
||||
limit = (counts[i] + counts[i + 1] +
|
||||
counts[i + 2] + counts[i + 3] + 2) / 4;
|
||||
} else if (i < length) {
|
||||
limit = counts[i];
|
||||
} else {
|
||||
limit = 0;
|
||||
}
|
||||
}
|
||||
++stride;
|
||||
if (i != length) {
|
||||
sum += counts[i];
|
||||
}
|
||||
}
|
||||
|
||||
free(good_for_rle);
|
||||
}
|
||||
|
||||
/*
|
||||
Tries out OptimizeHuffmanForRle for this block, if the result is smaller,
|
||||
uses it, otherwise keeps the original. Returns size of encoded tree and data in
|
||||
bits, not including the 3-bit block header.
|
||||
*/
|
||||
static double TryOptimizeHuffmanForRle(
|
||||
const ZopfliLZ77Store* lz77, size_t lstart, size_t lend,
|
||||
const size_t* ll_counts, const size_t* d_counts,
|
||||
unsigned* ll_lengths, unsigned* d_lengths) {
|
||||
size_t ll_counts2[ZOPFLI_NUM_LL];
|
||||
size_t d_counts2[ZOPFLI_NUM_D];
|
||||
unsigned ll_lengths2[ZOPFLI_NUM_LL];
|
||||
unsigned d_lengths2[ZOPFLI_NUM_D];
|
||||
double treesize;
|
||||
double datasize;
|
||||
double treesize2;
|
||||
double datasize2;
|
||||
|
||||
treesize = CalculateTreeSize(ll_lengths, d_lengths);
|
||||
datasize = CalculateBlockSymbolSizeGivenCounts(ll_counts, d_counts,
|
||||
ll_lengths, d_lengths, lz77, lstart, lend);
|
||||
|
||||
memcpy(ll_counts2, ll_counts, sizeof(ll_counts2));
|
||||
memcpy(d_counts2, d_counts, sizeof(d_counts2));
|
||||
OptimizeHuffmanForRle(ZOPFLI_NUM_LL, ll_counts2);
|
||||
OptimizeHuffmanForRle(ZOPFLI_NUM_D, d_counts2);
|
||||
ZopfliCalculateBitLengths(ll_counts2, ZOPFLI_NUM_LL, 15, ll_lengths2);
|
||||
ZopfliCalculateBitLengths(d_counts2, ZOPFLI_NUM_D, 15, d_lengths2);
|
||||
PatchDistanceCodesForBuggyDecoders(d_lengths2);
|
||||
|
||||
treesize2 = CalculateTreeSize(ll_lengths2, d_lengths2);
|
||||
datasize2 = CalculateBlockSymbolSizeGivenCounts(ll_counts, d_counts,
|
||||
ll_lengths2, d_lengths2, lz77, lstart, lend);
|
||||
|
||||
if (treesize2 + datasize2 < treesize + datasize) {
|
||||
memcpy(ll_lengths, ll_lengths2, sizeof(ll_lengths2));
|
||||
memcpy(d_lengths, d_lengths2, sizeof(d_lengths2));
|
||||
return treesize2 + datasize2;
|
||||
}
|
||||
return treesize + datasize;
|
||||
}
|
||||
|
||||
/*
|
||||
Calculates the bit lengths for the symbols for dynamic blocks. Chooses bit
|
||||
lengths that give the smallest size of tree encoding + encoding of all the
|
||||
symbols to have smallest output size. This are not necessarily the ideal Huffman
|
||||
bit lengths. Returns size of encoded tree and data in bits, not including the
|
||||
3-bit block header.
|
||||
*/
|
||||
static double GetDynamicLengths(const ZopfliLZ77Store* lz77,
|
||||
size_t lstart, size_t lend,
|
||||
unsigned* ll_lengths, unsigned* d_lengths) {
|
||||
size_t ll_counts[ZOPFLI_NUM_LL];
|
||||
size_t d_counts[ZOPFLI_NUM_D];
|
||||
|
||||
ZopfliLZ77GetHistogram(lz77, lstart, lend, ll_counts, d_counts);
|
||||
ll_counts[256] = 1; /* End symbol. */
|
||||
ZopfliCalculateBitLengths(ll_counts, ZOPFLI_NUM_LL, 15, ll_lengths);
|
||||
ZopfliCalculateBitLengths(d_counts, ZOPFLI_NUM_D, 15, d_lengths);
|
||||
PatchDistanceCodesForBuggyDecoders(d_lengths);
|
||||
return TryOptimizeHuffmanForRle(
|
||||
lz77, lstart, lend, ll_counts, d_counts, ll_lengths, d_lengths);
|
||||
}
|
||||
|
||||
double ZopfliCalculateBlockSize(const ZopfliLZ77Store* lz77,
|
||||
size_t lstart, size_t lend, int btype) {
|
||||
unsigned ll_lengths[ZOPFLI_NUM_LL];
|
||||
unsigned d_lengths[ZOPFLI_NUM_D];
|
||||
|
||||
double result = 3; /* bfinal and btype bits */
|
||||
|
||||
if (btype == 0) {
|
||||
size_t length = ZopfliLZ77GetByteRange(lz77, lstart, lend);
|
||||
size_t rem = length % 65535;
|
||||
size_t blocks = length / 65535 + (rem ? 1 : 0);
|
||||
/* An uncompressed block must actually be split into multiple blocks if it's
|
||||
larger than 65535 bytes long. Eeach block header is 5 bytes: 3 bits,
|
||||
padding, LEN and NLEN (potential less padding for first one ignored). */
|
||||
return blocks * 5 * 8 + length * 8;
|
||||
} if (btype == 1) {
|
||||
GetFixedTree(ll_lengths, d_lengths);
|
||||
result += CalculateBlockSymbolSize(
|
||||
ll_lengths, d_lengths, lz77, lstart, lend);
|
||||
} else {
|
||||
result += GetDynamicLengths(lz77, lstart, lend, ll_lengths, d_lengths);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
double ZopfliCalculateBlockSizeAutoType(const ZopfliLZ77Store* lz77,
|
||||
size_t lstart, size_t lend) {
|
||||
double uncompressedcost = ZopfliCalculateBlockSize(lz77, lstart, lend, 0);
|
||||
/* Don't do the expensive fixed cost calculation for larger blocks that are
|
||||
unlikely to use it. */
|
||||
double fixedcost = (lz77->size > 1000) ?
|
||||
uncompressedcost : ZopfliCalculateBlockSize(lz77, lstart, lend, 1);
|
||||
double dyncost = ZopfliCalculateBlockSize(lz77, lstart, lend, 2);
|
||||
return (uncompressedcost < fixedcost && uncompressedcost < dyncost)
|
||||
? uncompressedcost
|
||||
: (fixedcost < dyncost ? fixedcost : dyncost);
|
||||
}
|
||||
|
||||
/* Since an uncompressed block can be max 65535 in size, it actually adds
|
||||
multible blocks if needed. */
|
||||
static void AddNonCompressedBlock(const ZopfliOptions* options, int final,
|
||||
const unsigned char* in, size_t instart,
|
||||
size_t inend,
|
||||
unsigned char* bp,
|
||||
unsigned char** out, size_t* outsize) {
|
||||
size_t pos = instart;
|
||||
(void)options;
|
||||
for (;;) {
|
||||
size_t i;
|
||||
unsigned short blocksize = 65535;
|
||||
unsigned short nlen;
|
||||
int currentfinal;
|
||||
|
||||
if (pos + blocksize > inend) blocksize = inend - pos;
|
||||
currentfinal = pos + blocksize >= inend;
|
||||
|
||||
nlen = ~blocksize;
|
||||
|
||||
AddBit(final && currentfinal, bp, out, outsize);
|
||||
/* BTYPE 00 */
|
||||
AddBit(0, bp, out, outsize);
|
||||
AddBit(0, bp, out, outsize);
|
||||
|
||||
/* Any bits of input up to the next byte boundary are ignored. */
|
||||
*bp = 0;
|
||||
|
||||
ZOPFLI_APPEND_DATA(blocksize % 256, out, outsize);
|
||||
ZOPFLI_APPEND_DATA((blocksize / 256) % 256, out, outsize);
|
||||
ZOPFLI_APPEND_DATA(nlen % 256, out, outsize);
|
||||
ZOPFLI_APPEND_DATA((nlen / 256) % 256, out, outsize);
|
||||
|
||||
for (i = 0; i < blocksize; i++) {
|
||||
ZOPFLI_APPEND_DATA(in[pos + i], out, outsize);
|
||||
}
|
||||
|
||||
if (currentfinal) break;
|
||||
pos += blocksize;
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Adds a deflate block with the given LZ77 data to the output.
|
||||
options: global program options
|
||||
btype: the block type, must be 1 or 2
|
||||
final: whether to set the "final" bit on this block, must be the last block
|
||||
litlens: literal/length array of the LZ77 data, in the same format as in
|
||||
ZopfliLZ77Store.
|
||||
dists: distance array of the LZ77 data, in the same format as in
|
||||
ZopfliLZ77Store.
|
||||
lstart: where to start in the LZ77 data
|
||||
lend: where to end in the LZ77 data (not inclusive)
|
||||
expected_data_size: the uncompressed block size, used for assert, but you can
|
||||
set it to 0 to not do the assertion.
|
||||
bp: output bit pointer
|
||||
out: dynamic output array to append to
|
||||
outsize: dynamic output array size
|
||||
*/
|
||||
static void AddLZ77Block(const ZopfliOptions* options, int btype, int final,
|
||||
const ZopfliLZ77Store* lz77,
|
||||
size_t lstart, size_t lend,
|
||||
size_t expected_data_size,
|
||||
unsigned char* bp,
|
||||
unsigned char** out, size_t* outsize) {
|
||||
unsigned ll_lengths[ZOPFLI_NUM_LL];
|
||||
unsigned d_lengths[ZOPFLI_NUM_D];
|
||||
unsigned ll_symbols[ZOPFLI_NUM_LL];
|
||||
unsigned d_symbols[ZOPFLI_NUM_D];
|
||||
size_t detect_block_size = *outsize;
|
||||
size_t compressed_size;
|
||||
size_t uncompressed_size = 0;
|
||||
size_t i;
|
||||
if (btype == 0) {
|
||||
size_t length = ZopfliLZ77GetByteRange(lz77, lstart, lend);
|
||||
size_t pos = lstart == lend ? 0 : lz77->pos[lstart];
|
||||
size_t end = pos + length;
|
||||
AddNonCompressedBlock(options, final,
|
||||
lz77->data, pos, end, bp, out, outsize);
|
||||
return;
|
||||
}
|
||||
|
||||
AddBit(final, bp, out, outsize);
|
||||
AddBit(btype & 1, bp, out, outsize);
|
||||
AddBit((btype & 2) >> 1, bp, out, outsize);
|
||||
|
||||
if (btype == 1) {
|
||||
/* Fixed block. */
|
||||
GetFixedTree(ll_lengths, d_lengths);
|
||||
} else {
|
||||
/* Dynamic block. */
|
||||
unsigned detect_tree_size;
|
||||
assert(btype == 2);
|
||||
|
||||
GetDynamicLengths(lz77, lstart, lend, ll_lengths, d_lengths);
|
||||
|
||||
detect_tree_size = *outsize;
|
||||
AddDynamicTree(ll_lengths, d_lengths, bp, out, outsize);
|
||||
if (options->verbose) {
|
||||
fprintf(stderr, "treesize: %d\n", (int)(*outsize - detect_tree_size));
|
||||
}
|
||||
}
|
||||
|
||||
ZopfliLengthsToSymbols(ll_lengths, ZOPFLI_NUM_LL, 15, ll_symbols);
|
||||
ZopfliLengthsToSymbols(d_lengths, ZOPFLI_NUM_D, 15, d_symbols);
|
||||
|
||||
detect_block_size = *outsize;
|
||||
AddLZ77Data(lz77, lstart, lend, expected_data_size,
|
||||
ll_symbols, ll_lengths, d_symbols, d_lengths,
|
||||
bp, out, outsize);
|
||||
/* End symbol. */
|
||||
AddHuffmanBits(ll_symbols[256], ll_lengths[256], bp, out, outsize);
|
||||
|
||||
for (i = lstart; i < lend; i++) {
|
||||
uncompressed_size += lz77->dists[i] == 0 ? 1 : lz77->litlens[i];
|
||||
}
|
||||
compressed_size = *outsize - detect_block_size;
|
||||
if (options->verbose) {
|
||||
fprintf(stderr, "compressed block size: %d (%dk) (unc: %d)\n",
|
||||
(int)compressed_size, (int)(compressed_size / 1024),
|
||||
(int)(uncompressed_size));
|
||||
}
|
||||
}
|
||||
|
||||
static void AddLZ77BlockAutoType(const ZopfliOptions* options, int final,
|
||||
const ZopfliLZ77Store* lz77,
|
||||
size_t lstart, size_t lend,
|
||||
size_t expected_data_size,
|
||||
unsigned char* bp,
|
||||
unsigned char** out, size_t* outsize) {
|
||||
double uncompressedcost = ZopfliCalculateBlockSize(lz77, lstart, lend, 0);
|
||||
double fixedcost = ZopfliCalculateBlockSize(lz77, lstart, lend, 1);
|
||||
double dyncost = ZopfliCalculateBlockSize(lz77, lstart, lend, 2);
|
||||
|
||||
/* Whether to perform the expensive calculation of creating an optimal block
|
||||
with fixed huffman tree to check if smaller. Only do this for small blocks or
|
||||
blocks which already are pretty good with fixed huffman tree. */
|
||||
int expensivefixed = (lz77->size < 1000) || fixedcost <= dyncost * 1.1;
|
||||
|
||||
ZopfliLZ77Store fixedstore;
|
||||
if (lstart == lend) {
|
||||
/* Smallest empty block is represented by fixed block */
|
||||
AddBits(final, 1, bp, out, outsize);
|
||||
AddBits(1, 2, bp, out, outsize); /* btype 01 */
|
||||
AddBits(0, 7, bp, out, outsize); /* end symbol has code 0000000 */
|
||||
return;
|
||||
}
|
||||
ZopfliInitLZ77Store(lz77->data, &fixedstore);
|
||||
if (expensivefixed) {
|
||||
/* Recalculate the LZ77 with ZopfliLZ77OptimalFixed */
|
||||
size_t instart = lz77->pos[lstart];
|
||||
size_t inend = instart + ZopfliLZ77GetByteRange(lz77, lstart, lend);
|
||||
|
||||
ZopfliBlockState s;
|
||||
ZopfliInitBlockState(options, instart, inend, 1, &s);
|
||||
ZopfliLZ77OptimalFixed(&s, lz77->data, instart, inend, &fixedstore);
|
||||
fixedcost = ZopfliCalculateBlockSize(&fixedstore, 0, fixedstore.size, 1);
|
||||
ZopfliCleanBlockState(&s);
|
||||
}
|
||||
|
||||
if (uncompressedcost < fixedcost && uncompressedcost < dyncost) {
|
||||
AddLZ77Block(options, 0, final, lz77, lstart, lend,
|
||||
expected_data_size, bp, out, outsize);
|
||||
} else if (fixedcost < dyncost) {
|
||||
if (expensivefixed) {
|
||||
AddLZ77Block(options, 1, final, &fixedstore, 0, fixedstore.size,
|
||||
expected_data_size, bp, out, outsize);
|
||||
} else {
|
||||
AddLZ77Block(options, 1, final, lz77, lstart, lend,
|
||||
expected_data_size, bp, out, outsize);
|
||||
}
|
||||
} else {
|
||||
AddLZ77Block(options, 2, final, lz77, lstart, lend,
|
||||
expected_data_size, bp, out, outsize);
|
||||
}
|
||||
|
||||
ZopfliCleanLZ77Store(&fixedstore);
|
||||
}
|
||||
|
||||
/*
|
||||
Deflate a part, to allow ZopfliDeflate() to use multiple master blocks if
|
||||
needed.
|
||||
It is possible to call this function multiple times in a row, shifting
|
||||
instart and inend to next bytes of the data. If instart is larger than 0, then
|
||||
previous bytes are used as the initial dictionary for LZ77.
|
||||
This function will usually output multiple deflate blocks. If final is 1, then
|
||||
the final bit will be set on the last block.
|
||||
*/
|
||||
void ZopfliDeflatePart(const ZopfliOptions* options, int btype, int final,
|
||||
const unsigned char* in, size_t instart, size_t inend,
|
||||
unsigned char* bp, unsigned char** out,
|
||||
size_t* outsize) {
|
||||
size_t i;
|
||||
/* byte coordinates rather than lz77 index */
|
||||
size_t* splitpoints_uncompressed = 0;
|
||||
size_t npoints = 0;
|
||||
size_t* splitpoints = 0;
|
||||
double totalcost = 0;
|
||||
ZopfliLZ77Store lz77;
|
||||
|
||||
/* If btype=2 is specified, it tries all block types. If a lesser btype is
|
||||
given, then however it forces that one. Neither of the lesser types needs
|
||||
block splitting as they have no dynamic huffman trees. */
|
||||
if (btype == 0) {
|
||||
AddNonCompressedBlock(options, final, in, instart, inend, bp, out, outsize);
|
||||
return;
|
||||
} else if (btype == 1) {
|
||||
ZopfliLZ77Store store;
|
||||
ZopfliBlockState s;
|
||||
ZopfliInitLZ77Store(in, &store);
|
||||
ZopfliInitBlockState(options, instart, inend, 1, &s);
|
||||
|
||||
ZopfliLZ77OptimalFixed(&s, in, instart, inend, &store);
|
||||
AddLZ77Block(options, btype, final, &store, 0, store.size, 0,
|
||||
bp, out, outsize);
|
||||
|
||||
ZopfliCleanBlockState(&s);
|
||||
ZopfliCleanLZ77Store(&store);
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
if (options->blocksplitting) {
|
||||
ZopfliBlockSplit(options, in, instart, inend,
|
||||
options->blocksplittingmax,
|
||||
&splitpoints_uncompressed, &npoints);
|
||||
splitpoints = (size_t*)malloc(sizeof(*splitpoints) * npoints);
|
||||
}
|
||||
|
||||
ZopfliInitLZ77Store(in, &lz77);
|
||||
|
||||
for (i = 0; i <= npoints; i++) {
|
||||
size_t start = i == 0 ? instart : splitpoints_uncompressed[i - 1];
|
||||
size_t end = i == npoints ? inend : splitpoints_uncompressed[i];
|
||||
ZopfliBlockState s;
|
||||
ZopfliLZ77Store store;
|
||||
ZopfliInitLZ77Store(in, &store);
|
||||
ZopfliInitBlockState(options, start, end, 1, &s);
|
||||
ZopfliLZ77Optimal(&s, in, start, end, options->numiterations, &store);
|
||||
totalcost += ZopfliCalculateBlockSizeAutoType(&store, 0, store.size);
|
||||
|
||||
ZopfliAppendLZ77Store(&store, &lz77);
|
||||
if (i < npoints) splitpoints[i] = lz77.size;
|
||||
|
||||
ZopfliCleanBlockState(&s);
|
||||
ZopfliCleanLZ77Store(&store);
|
||||
}
|
||||
|
||||
/* Second block splitting attempt */
|
||||
if (options->blocksplitting && npoints > 1) {
|
||||
size_t* splitpoints2 = 0;
|
||||
size_t npoints2 = 0;
|
||||
double totalcost2 = 0;
|
||||
|
||||
ZopfliBlockSplitLZ77(options, &lz77,
|
||||
options->blocksplittingmax, &splitpoints2, &npoints2);
|
||||
|
||||
for (i = 0; i <= npoints2; i++) {
|
||||
size_t start = i == 0 ? 0 : splitpoints2[i - 1];
|
||||
size_t end = i == npoints2 ? lz77.size : splitpoints2[i];
|
||||
totalcost2 += ZopfliCalculateBlockSizeAutoType(&lz77, start, end);
|
||||
}
|
||||
|
||||
if (totalcost2 < totalcost) {
|
||||
free(splitpoints);
|
||||
splitpoints = splitpoints2;
|
||||
npoints = npoints2;
|
||||
} else {
|
||||
free(splitpoints2);
|
||||
}
|
||||
}
|
||||
|
||||
for (i = 0; i <= npoints; i++) {
|
||||
size_t start = i == 0 ? 0 : splitpoints[i - 1];
|
||||
size_t end = i == npoints ? lz77.size : splitpoints[i];
|
||||
AddLZ77BlockAutoType(options, i == npoints && final,
|
||||
&lz77, start, end, 0,
|
||||
bp, out, outsize);
|
||||
}
|
||||
|
||||
ZopfliCleanLZ77Store(&lz77);
|
||||
free(splitpoints);
|
||||
free(splitpoints_uncompressed);
|
||||
}
|
||||
|
||||
void ZopfliDeflate(const ZopfliOptions* options, int btype, int final,
|
||||
const unsigned char* in, size_t insize,
|
||||
unsigned char* bp, unsigned char** out, size_t* outsize) {
|
||||
size_t offset = *outsize;
|
||||
#if ZOPFLI_MASTER_BLOCK_SIZE == 0
|
||||
ZopfliDeflatePart(options, btype, final, in, 0, insize, bp, out, outsize);
|
||||
#else
|
||||
size_t i = 0;
|
||||
do {
|
||||
int masterfinal = (i + ZOPFLI_MASTER_BLOCK_SIZE >= insize);
|
||||
int final2 = final && masterfinal;
|
||||
size_t size = masterfinal ? insize - i : ZOPFLI_MASTER_BLOCK_SIZE;
|
||||
ZopfliDeflatePart(options, btype, final2,
|
||||
in, i, i + size, bp, out, outsize);
|
||||
i += size;
|
||||
} while (i < insize);
|
||||
#endif
|
||||
if (options->verbose) {
|
||||
fprintf(stderr,
|
||||
"Original Size: %lu, Deflate: %lu, Compression: %f%% Removed\n",
|
||||
(unsigned long)insize, (unsigned long)(*outsize - offset),
|
||||
100.0 * (double)(insize - (*outsize - offset)) / (double)insize);
|
||||
}
|
||||
}
|
||||
92
build/node_modules/node-zopfli/zopfli/src/zopfli/deflate.h
generated
vendored
Normal file
92
build/node_modules/node-zopfli/zopfli/src/zopfli/deflate.h
generated
vendored
Normal file
@@ -0,0 +1,92 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
#ifndef ZOPFLI_DEFLATE_H_
|
||||
#define ZOPFLI_DEFLATE_H_
|
||||
|
||||
/*
|
||||
Functions to compress according to the DEFLATE specification, using the
|
||||
"squeeze" LZ77 compression backend.
|
||||
*/
|
||||
|
||||
#include "lz77.h"
|
||||
#include "zopfli.h"
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
/*
|
||||
Compresses according to the deflate specification and append the compressed
|
||||
result to the output.
|
||||
This function will usually output multiple deflate blocks. If final is 1, then
|
||||
the final bit will be set on the last block.
|
||||
|
||||
options: global program options
|
||||
btype: the deflate block type. Use 2 for best compression.
|
||||
-0: non compressed blocks (00)
|
||||
-1: blocks with fixed tree (01)
|
||||
-2: blocks with dynamic tree (10)
|
||||
final: whether this is the last section of the input, sets the final bit to the
|
||||
last deflate block.
|
||||
in: the input bytes
|
||||
insize: number of input bytes
|
||||
bp: bit pointer for the output array. This must initially be 0, and for
|
||||
consecutive calls must be reused (it can have values from 0-7). This is
|
||||
because deflate appends blocks as bit-based data, rather than on byte
|
||||
boundaries.
|
||||
out: pointer to the dynamic output array to which the result is appended. Must
|
||||
be freed after use.
|
||||
outsize: pointer to the dynamic output array size.
|
||||
*/
|
||||
void ZopfliDeflate(const ZopfliOptions* options, int btype, int final,
|
||||
const unsigned char* in, size_t insize,
|
||||
unsigned char* bp, unsigned char** out, size_t* outsize);
|
||||
|
||||
/*
|
||||
Like ZopfliDeflate, but allows to specify start and end byte with instart and
|
||||
inend. Only that part is compressed, but earlier bytes are still used for the
|
||||
back window.
|
||||
*/
|
||||
void ZopfliDeflatePart(const ZopfliOptions* options, int btype, int final,
|
||||
const unsigned char* in, size_t instart, size_t inend,
|
||||
unsigned char* bp, unsigned char** out,
|
||||
size_t* outsize);
|
||||
|
||||
/*
|
||||
Calculates block size in bits.
|
||||
litlens: lz77 lit/lengths
|
||||
dists: ll77 distances
|
||||
lstart: start of block
|
||||
lend: end of block (not inclusive)
|
||||
*/
|
||||
double ZopfliCalculateBlockSize(const ZopfliLZ77Store* lz77,
|
||||
size_t lstart, size_t lend, int btype);
|
||||
|
||||
/*
|
||||
Calculates block size in bits, automatically using the best btype.
|
||||
*/
|
||||
double ZopfliCalculateBlockSizeAutoType(const ZopfliLZ77Store* lz77,
|
||||
size_t lstart, size_t lend);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} // extern "C"
|
||||
#endif
|
||||
|
||||
#endif /* ZOPFLI_DEFLATE_H_ */
|
||||
124
build/node_modules/node-zopfli/zopfli/src/zopfli/gzip_container.c
generated
vendored
Normal file
124
build/node_modules/node-zopfli/zopfli/src/zopfli/gzip_container.c
generated
vendored
Normal file
@@ -0,0 +1,124 @@
|
||||
/*
|
||||
Copyright 2013 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
#include "gzip_container.h"
|
||||
#include "util.h"
|
||||
|
||||
#include <stdio.h>
|
||||
|
||||
#include "deflate.h"
|
||||
|
||||
/* CRC polynomial: 0xedb88320 */
|
||||
static const unsigned long crc32_table[256] = {
|
||||
0u, 1996959894u, 3993919788u, 2567524794u, 124634137u, 1886057615u,
|
||||
3915621685u, 2657392035u, 249268274u, 2044508324u, 3772115230u, 2547177864u,
|
||||
162941995u, 2125561021u, 3887607047u, 2428444049u, 498536548u, 1789927666u,
|
||||
4089016648u, 2227061214u, 450548861u, 1843258603u, 4107580753u, 2211677639u,
|
||||
325883990u, 1684777152u, 4251122042u, 2321926636u, 335633487u, 1661365465u,
|
||||
4195302755u, 2366115317u, 997073096u, 1281953886u, 3579855332u, 2724688242u,
|
||||
1006888145u, 1258607687u, 3524101629u, 2768942443u, 901097722u, 1119000684u,
|
||||
3686517206u, 2898065728u, 853044451u, 1172266101u, 3705015759u, 2882616665u,
|
||||
651767980u, 1373503546u, 3369554304u, 3218104598u, 565507253u, 1454621731u,
|
||||
3485111705u, 3099436303u, 671266974u, 1594198024u, 3322730930u, 2970347812u,
|
||||
795835527u, 1483230225u, 3244367275u, 3060149565u, 1994146192u, 31158534u,
|
||||
2563907772u, 4023717930u, 1907459465u, 112637215u, 2680153253u, 3904427059u,
|
||||
2013776290u, 251722036u, 2517215374u, 3775830040u, 2137656763u, 141376813u,
|
||||
2439277719u, 3865271297u, 1802195444u, 476864866u, 2238001368u, 4066508878u,
|
||||
1812370925u, 453092731u, 2181625025u, 4111451223u, 1706088902u, 314042704u,
|
||||
2344532202u, 4240017532u, 1658658271u, 366619977u, 2362670323u, 4224994405u,
|
||||
1303535960u, 984961486u, 2747007092u, 3569037538u, 1256170817u, 1037604311u,
|
||||
2765210733u, 3554079995u, 1131014506u, 879679996u, 2909243462u, 3663771856u,
|
||||
1141124467u, 855842277u, 2852801631u, 3708648649u, 1342533948u, 654459306u,
|
||||
3188396048u, 3373015174u, 1466479909u, 544179635u, 3110523913u, 3462522015u,
|
||||
1591671054u, 702138776u, 2966460450u, 3352799412u, 1504918807u, 783551873u,
|
||||
3082640443u, 3233442989u, 3988292384u, 2596254646u, 62317068u, 1957810842u,
|
||||
3939845945u, 2647816111u, 81470997u, 1943803523u, 3814918930u, 2489596804u,
|
||||
225274430u, 2053790376u, 3826175755u, 2466906013u, 167816743u, 2097651377u,
|
||||
4027552580u, 2265490386u, 503444072u, 1762050814u, 4150417245u, 2154129355u,
|
||||
426522225u, 1852507879u, 4275313526u, 2312317920u, 282753626u, 1742555852u,
|
||||
4189708143u, 2394877945u, 397917763u, 1622183637u, 3604390888u, 2714866558u,
|
||||
953729732u, 1340076626u, 3518719985u, 2797360999u, 1068828381u, 1219638859u,
|
||||
3624741850u, 2936675148u, 906185462u, 1090812512u, 3747672003u, 2825379669u,
|
||||
829329135u, 1181335161u, 3412177804u, 3160834842u, 628085408u, 1382605366u,
|
||||
3423369109u, 3138078467u, 570562233u, 1426400815u, 3317316542u, 2998733608u,
|
||||
733239954u, 1555261956u, 3268935591u, 3050360625u, 752459403u, 1541320221u,
|
||||
2607071920u, 3965973030u, 1969922972u, 40735498u, 2617837225u, 3943577151u,
|
||||
1913087877u, 83908371u, 2512341634u, 3803740692u, 2075208622u, 213261112u,
|
||||
2463272603u, 3855990285u, 2094854071u, 198958881u, 2262029012u, 4057260610u,
|
||||
1759359992u, 534414190u, 2176718541u, 4139329115u, 1873836001u, 414664567u,
|
||||
2282248934u, 4279200368u, 1711684554u, 285281116u, 2405801727u, 4167216745u,
|
||||
1634467795u, 376229701u, 2685067896u, 3608007406u, 1308918612u, 956543938u,
|
||||
2808555105u, 3495958263u, 1231636301u, 1047427035u, 2932959818u, 3654703836u,
|
||||
1088359270u, 936918000u, 2847714899u, 3736837829u, 1202900863u, 817233897u,
|
||||
3183342108u, 3401237130u, 1404277552u, 615818150u, 3134207493u, 3453421203u,
|
||||
1423857449u, 601450431u, 3009837614u, 3294710456u, 1567103746u, 711928724u,
|
||||
3020668471u, 3272380065u, 1510334235u, 755167117u
|
||||
};
|
||||
|
||||
/* Returns the CRC32 */
|
||||
static unsigned long CRC(const unsigned char* data, size_t size) {
|
||||
unsigned long result = 0xffffffffu;
|
||||
for (; size > 0; size--) {
|
||||
result = crc32_table[(result ^ *(data++)) & 0xff] ^ (result >> 8);
|
||||
}
|
||||
return result ^ 0xffffffffu;
|
||||
}
|
||||
|
||||
/* Compresses the data according to the gzip specification, RFC 1952. */
|
||||
void ZopfliGzipCompress(const ZopfliOptions* options,
|
||||
const unsigned char* in, size_t insize,
|
||||
unsigned char** out, size_t* outsize) {
|
||||
unsigned long crcvalue = CRC(in, insize);
|
||||
unsigned char bp = 0;
|
||||
|
||||
ZOPFLI_APPEND_DATA(31, out, outsize); /* ID1 */
|
||||
ZOPFLI_APPEND_DATA(139, out, outsize); /* ID2 */
|
||||
ZOPFLI_APPEND_DATA(8, out, outsize); /* CM */
|
||||
ZOPFLI_APPEND_DATA(0, out, outsize); /* FLG */
|
||||
/* MTIME */
|
||||
ZOPFLI_APPEND_DATA(0, out, outsize);
|
||||
ZOPFLI_APPEND_DATA(0, out, outsize);
|
||||
ZOPFLI_APPEND_DATA(0, out, outsize);
|
||||
ZOPFLI_APPEND_DATA(0, out, outsize);
|
||||
|
||||
ZOPFLI_APPEND_DATA(2, out, outsize); /* XFL, 2 indicates best compression. */
|
||||
ZOPFLI_APPEND_DATA(3, out, outsize); /* OS follows Unix conventions. */
|
||||
|
||||
ZopfliDeflate(options, 2 /* Dynamic block */, 1,
|
||||
in, insize, &bp, out, outsize);
|
||||
|
||||
/* CRC */
|
||||
ZOPFLI_APPEND_DATA(crcvalue % 256, out, outsize);
|
||||
ZOPFLI_APPEND_DATA((crcvalue >> 8) % 256, out, outsize);
|
||||
ZOPFLI_APPEND_DATA((crcvalue >> 16) % 256, out, outsize);
|
||||
ZOPFLI_APPEND_DATA((crcvalue >> 24) % 256, out, outsize);
|
||||
|
||||
/* ISIZE */
|
||||
ZOPFLI_APPEND_DATA(insize % 256, out, outsize);
|
||||
ZOPFLI_APPEND_DATA((insize >> 8) % 256, out, outsize);
|
||||
ZOPFLI_APPEND_DATA((insize >> 16) % 256, out, outsize);
|
||||
ZOPFLI_APPEND_DATA((insize >> 24) % 256, out, outsize);
|
||||
|
||||
if (options->verbose) {
|
||||
fprintf(stderr,
|
||||
"Original Size: %d, Gzip: %d, Compression: %f%% Removed\n",
|
||||
(int)insize, (int)*outsize,
|
||||
100.0 * (double)(insize - *outsize) / (double)insize);
|
||||
}
|
||||
}
|
||||
50
build/node_modules/node-zopfli/zopfli/src/zopfli/gzip_container.h
generated
vendored
Normal file
50
build/node_modules/node-zopfli/zopfli/src/zopfli/gzip_container.h
generated
vendored
Normal file
@@ -0,0 +1,50 @@
|
||||
/*
|
||||
Copyright 2013 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
#ifndef ZOPFLI_GZIP_H_
|
||||
#define ZOPFLI_GZIP_H_
|
||||
|
||||
/*
|
||||
Functions to compress according to the Gzip specification.
|
||||
*/
|
||||
|
||||
#include "zopfli.h"
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
/*
|
||||
Compresses according to the gzip specification and append the compressed
|
||||
result to the output.
|
||||
|
||||
options: global program options
|
||||
out: pointer to the dynamic output array to which the result is appended. Must
|
||||
be freed after use.
|
||||
outsize: pointer to the dynamic output array size.
|
||||
*/
|
||||
void ZopfliGzipCompress(const ZopfliOptions* options,
|
||||
const unsigned char* in, size_t insize,
|
||||
unsigned char** out, size_t* outsize);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} // extern "C"
|
||||
#endif
|
||||
|
||||
#endif /* ZOPFLI_GZIP_H_ */
|
||||
143
build/node_modules/node-zopfli/zopfli/src/zopfli/hash.c
generated
vendored
Normal file
143
build/node_modules/node-zopfli/zopfli/src/zopfli/hash.c
generated
vendored
Normal file
@@ -0,0 +1,143 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
#include "hash.h"
|
||||
|
||||
#include <assert.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
|
||||
#define HASH_SHIFT 5
|
||||
#define HASH_MASK 32767
|
||||
|
||||
void ZopfliAllocHash(size_t window_size, ZopfliHash* h) {
|
||||
h->head = (int*)malloc(sizeof(*h->head) * 65536);
|
||||
h->prev = (unsigned short*)malloc(sizeof(*h->prev) * window_size);
|
||||
h->hashval = (int*)malloc(sizeof(*h->hashval) * window_size);
|
||||
|
||||
#ifdef ZOPFLI_HASH_SAME
|
||||
h->same = (unsigned short*)malloc(sizeof(*h->same) * window_size);
|
||||
#endif
|
||||
|
||||
#ifdef ZOPFLI_HASH_SAME_HASH
|
||||
h->head2 = (int*)malloc(sizeof(*h->head2) * 65536);
|
||||
h->prev2 = (unsigned short*)malloc(sizeof(*h->prev2) * window_size);
|
||||
h->hashval2 = (int*)malloc(sizeof(*h->hashval2) * window_size);
|
||||
#endif
|
||||
}
|
||||
|
||||
void ZopfliResetHash(size_t window_size, ZopfliHash* h) {
|
||||
size_t i;
|
||||
|
||||
h->val = 0;
|
||||
for (i = 0; i < 65536; i++) {
|
||||
h->head[i] = -1; /* -1 indicates no head so far. */
|
||||
}
|
||||
for (i = 0; i < window_size; i++) {
|
||||
h->prev[i] = i; /* If prev[j] == j, then prev[j] is uninitialized. */
|
||||
h->hashval[i] = -1;
|
||||
}
|
||||
|
||||
#ifdef ZOPFLI_HASH_SAME
|
||||
for (i = 0; i < window_size; i++) {
|
||||
h->same[i] = 0;
|
||||
}
|
||||
#endif
|
||||
|
||||
#ifdef ZOPFLI_HASH_SAME_HASH
|
||||
h->val2 = 0;
|
||||
for (i = 0; i < 65536; i++) {
|
||||
h->head2[i] = -1;
|
||||
}
|
||||
for (i = 0; i < window_size; i++) {
|
||||
h->prev2[i] = i;
|
||||
h->hashval2[i] = -1;
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
void ZopfliCleanHash(ZopfliHash* h) {
|
||||
free(h->head);
|
||||
free(h->prev);
|
||||
free(h->hashval);
|
||||
|
||||
#ifdef ZOPFLI_HASH_SAME_HASH
|
||||
free(h->head2);
|
||||
free(h->prev2);
|
||||
free(h->hashval2);
|
||||
#endif
|
||||
|
||||
#ifdef ZOPFLI_HASH_SAME
|
||||
free(h->same);
|
||||
#endif
|
||||
}
|
||||
|
||||
/*
|
||||
Update the sliding hash value with the given byte. All calls to this function
|
||||
must be made on consecutive input characters. Since the hash value exists out
|
||||
of multiple input bytes, a few warmups with this function are needed initially.
|
||||
*/
|
||||
static void UpdateHashValue(ZopfliHash* h, unsigned char c) {
|
||||
h->val = (((h->val) << HASH_SHIFT) ^ (c)) & HASH_MASK;
|
||||
}
|
||||
|
||||
void ZopfliUpdateHash(const unsigned char* array, size_t pos, size_t end,
|
||||
ZopfliHash* h) {
|
||||
unsigned short hpos = pos & ZOPFLI_WINDOW_MASK;
|
||||
#ifdef ZOPFLI_HASH_SAME
|
||||
size_t amount = 0;
|
||||
#endif
|
||||
|
||||
UpdateHashValue(h, pos + ZOPFLI_MIN_MATCH <= end ?
|
||||
array[pos + ZOPFLI_MIN_MATCH - 1] : 0);
|
||||
h->hashval[hpos] = h->val;
|
||||
if (h->head[h->val] != -1 && h->hashval[h->head[h->val]] == h->val) {
|
||||
h->prev[hpos] = h->head[h->val];
|
||||
}
|
||||
else h->prev[hpos] = hpos;
|
||||
h->head[h->val] = hpos;
|
||||
|
||||
#ifdef ZOPFLI_HASH_SAME
|
||||
/* Update "same". */
|
||||
if (h->same[(pos - 1) & ZOPFLI_WINDOW_MASK] > 1) {
|
||||
amount = h->same[(pos - 1) & ZOPFLI_WINDOW_MASK] - 1;
|
||||
}
|
||||
while (pos + amount + 1 < end &&
|
||||
array[pos] == array[pos + amount + 1] && amount < (unsigned short)(-1)) {
|
||||
amount++;
|
||||
}
|
||||
h->same[hpos] = amount;
|
||||
#endif
|
||||
|
||||
#ifdef ZOPFLI_HASH_SAME_HASH
|
||||
h->val2 = ((h->same[hpos] - ZOPFLI_MIN_MATCH) & 255) ^ h->val;
|
||||
h->hashval2[hpos] = h->val2;
|
||||
if (h->head2[h->val2] != -1 && h->hashval2[h->head2[h->val2]] == h->val2) {
|
||||
h->prev2[hpos] = h->head2[h->val2];
|
||||
}
|
||||
else h->prev2[hpos] = hpos;
|
||||
h->head2[h->val2] = hpos;
|
||||
#endif
|
||||
}
|
||||
|
||||
void ZopfliWarmupHash(const unsigned char* array, size_t pos, size_t end,
|
||||
ZopfliHash* h) {
|
||||
UpdateHashValue(h, array[pos + 0]);
|
||||
if (pos + 1 < end) UpdateHashValue(h, array[pos + 1]);
|
||||
}
|
||||
73
build/node_modules/node-zopfli/zopfli/src/zopfli/hash.h
generated
vendored
Normal file
73
build/node_modules/node-zopfli/zopfli/src/zopfli/hash.h
generated
vendored
Normal file
@@ -0,0 +1,73 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
/*
|
||||
The hash for ZopfliFindLongestMatch of lz77.c.
|
||||
*/
|
||||
|
||||
#ifndef ZOPFLI_HASH_H_
|
||||
#define ZOPFLI_HASH_H_
|
||||
|
||||
#include "util.h"
|
||||
|
||||
typedef struct ZopfliHash {
|
||||
int* head; /* Hash value to index of its most recent occurrence. */
|
||||
unsigned short* prev; /* Index to index of prev. occurrence of same hash. */
|
||||
int* hashval; /* Index to hash value at this index. */
|
||||
int val; /* Current hash value. */
|
||||
|
||||
#ifdef ZOPFLI_HASH_SAME_HASH
|
||||
/* Fields with similar purpose as the above hash, but for the second hash with
|
||||
a value that is calculated differently. */
|
||||
int* head2; /* Hash value to index of its most recent occurrence. */
|
||||
unsigned short* prev2; /* Index to index of prev. occurrence of same hash. */
|
||||
int* hashval2; /* Index to hash value at this index. */
|
||||
int val2; /* Current hash value. */
|
||||
#endif
|
||||
|
||||
#ifdef ZOPFLI_HASH_SAME
|
||||
unsigned short* same; /* Amount of repetitions of same byte after this .*/
|
||||
#endif
|
||||
} ZopfliHash;
|
||||
|
||||
/* Allocates ZopfliHash memory. */
|
||||
void ZopfliAllocHash(size_t window_size, ZopfliHash* h);
|
||||
|
||||
/* Resets all fields of ZopfliHash. */
|
||||
void ZopfliResetHash(size_t window_size, ZopfliHash* h);
|
||||
|
||||
/* Frees ZopfliHash memory. */
|
||||
void ZopfliCleanHash(ZopfliHash* h);
|
||||
|
||||
/*
|
||||
Updates the hash values based on the current position in the array. All calls
|
||||
to this must be made for consecutive bytes.
|
||||
*/
|
||||
void ZopfliUpdateHash(const unsigned char* array, size_t pos, size_t end,
|
||||
ZopfliHash* h);
|
||||
|
||||
/*
|
||||
Prepopulates hash:
|
||||
Fills in the initial values in the hash, before ZopfliUpdateHash can be used
|
||||
correctly.
|
||||
*/
|
||||
void ZopfliWarmupHash(const unsigned char* array, size_t pos, size_t end,
|
||||
ZopfliHash* h);
|
||||
|
||||
#endif /* ZOPFLI_HASH_H_ */
|
||||
262
build/node_modules/node-zopfli/zopfli/src/zopfli/katajainen.c
generated
vendored
Executable file
262
build/node_modules/node-zopfli/zopfli/src/zopfli/katajainen.c
generated
vendored
Executable file
@@ -0,0 +1,262 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
/*
|
||||
Bounded package merge algorithm, based on the paper
|
||||
"A Fast and Space-Economical Algorithm for Length-Limited Coding
|
||||
Jyrki Katajainen, Alistair Moffat, Andrew Turpin".
|
||||
*/
|
||||
|
||||
#include "katajainen.h"
|
||||
#include <assert.h>
|
||||
#include <stdlib.h>
|
||||
#include <limits.h>
|
||||
|
||||
typedef struct Node Node;
|
||||
|
||||
/*
|
||||
Nodes forming chains. Also used to represent leaves.
|
||||
*/
|
||||
struct Node {
|
||||
size_t weight; /* Total weight (symbol count) of this chain. */
|
||||
Node* tail; /* Previous node(s) of this chain, or 0 if none. */
|
||||
int count; /* Leaf symbol index, or number of leaves before this chain. */
|
||||
};
|
||||
|
||||
/*
|
||||
Memory pool for nodes.
|
||||
*/
|
||||
typedef struct NodePool {
|
||||
Node* next; /* Pointer to a free node in the pool. */
|
||||
} NodePool;
|
||||
|
||||
/*
|
||||
Initializes a chain node with the given values and marks it as in use.
|
||||
*/
|
||||
static void InitNode(size_t weight, int count, Node* tail, Node* node) {
|
||||
node->weight = weight;
|
||||
node->count = count;
|
||||
node->tail = tail;
|
||||
}
|
||||
|
||||
/*
|
||||
Performs a Boundary Package-Merge step. Puts a new chain in the given list. The
|
||||
new chain is, depending on the weights, a leaf or a combination of two chains
|
||||
from the previous list.
|
||||
lists: The lists of chains.
|
||||
maxbits: Number of lists.
|
||||
leaves: The leaves, one per symbol.
|
||||
numsymbols: Number of leaves.
|
||||
pool: the node memory pool.
|
||||
index: The index of the list in which a new chain or leaf is required.
|
||||
*/
|
||||
static void BoundaryPM(Node* (*lists)[2], Node* leaves, int numsymbols,
|
||||
NodePool* pool, int index) {
|
||||
Node* newchain;
|
||||
Node* oldchain;
|
||||
int lastcount = lists[index][1]->count; /* Count of last chain of list. */
|
||||
|
||||
if (index == 0 && lastcount >= numsymbols) return;
|
||||
|
||||
newchain = pool->next++;
|
||||
oldchain = lists[index][1];
|
||||
|
||||
/* These are set up before the recursive calls below, so that there is a list
|
||||
pointing to the new node, to let the garbage collection know it's in use. */
|
||||
lists[index][0] = oldchain;
|
||||
lists[index][1] = newchain;
|
||||
|
||||
if (index == 0) {
|
||||
/* New leaf node in list 0. */
|
||||
InitNode(leaves[lastcount].weight, lastcount + 1, 0, newchain);
|
||||
} else {
|
||||
size_t sum = lists[index - 1][0]->weight + lists[index - 1][1]->weight;
|
||||
if (lastcount < numsymbols && sum > leaves[lastcount].weight) {
|
||||
/* New leaf inserted in list, so count is incremented. */
|
||||
InitNode(leaves[lastcount].weight, lastcount + 1, oldchain->tail,
|
||||
newchain);
|
||||
} else {
|
||||
InitNode(sum, lastcount, lists[index - 1][1], newchain);
|
||||
/* Two lookahead chains of previous list used up, create new ones. */
|
||||
BoundaryPM(lists, leaves, numsymbols, pool, index - 1);
|
||||
BoundaryPM(lists, leaves, numsymbols, pool, index - 1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static void BoundaryPMFinal(Node* (*lists)[2],
|
||||
Node* leaves, int numsymbols, NodePool* pool, int index) {
|
||||
int lastcount = lists[index][1]->count; /* Count of last chain of list. */
|
||||
|
||||
size_t sum = lists[index - 1][0]->weight + lists[index - 1][1]->weight;
|
||||
|
||||
if (lastcount < numsymbols && sum > leaves[lastcount].weight) {
|
||||
Node* newchain = pool->next;
|
||||
Node* oldchain = lists[index][1]->tail;
|
||||
|
||||
lists[index][1] = newchain;
|
||||
newchain->count = lastcount + 1;
|
||||
newchain->tail = oldchain;
|
||||
} else {
|
||||
lists[index][1]->tail = lists[index - 1][1];
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Initializes each list with as lookahead chains the two leaves with lowest
|
||||
weights.
|
||||
*/
|
||||
static void InitLists(
|
||||
NodePool* pool, const Node* leaves, int maxbits, Node* (*lists)[2]) {
|
||||
int i;
|
||||
Node* node0 = pool->next++;
|
||||
Node* node1 = pool->next++;
|
||||
InitNode(leaves[0].weight, 1, 0, node0);
|
||||
InitNode(leaves[1].weight, 2, 0, node1);
|
||||
for (i = 0; i < maxbits; i++) {
|
||||
lists[i][0] = node0;
|
||||
lists[i][1] = node1;
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Converts result of boundary package-merge to the bitlengths. The result in the
|
||||
last chain of the last list contains the amount of active leaves in each list.
|
||||
chain: Chain to extract the bit length from (last chain from last list).
|
||||
*/
|
||||
static void ExtractBitLengths(Node* chain, Node* leaves, unsigned* bitlengths) {
|
||||
int counts[16] = {0};
|
||||
unsigned end = 16;
|
||||
unsigned ptr = 15;
|
||||
unsigned value = 1;
|
||||
Node* node;
|
||||
int val;
|
||||
|
||||
for (node = chain; node; node = node->tail) {
|
||||
counts[--end] = node->count;
|
||||
}
|
||||
|
||||
val = counts[15];
|
||||
while (ptr >= end) {
|
||||
for (; val > counts[ptr - 1]; val--) {
|
||||
bitlengths[leaves[val - 1].count] = value;
|
||||
}
|
||||
ptr--;
|
||||
value++;
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Comparator for sorting the leaves. Has the function signature for qsort.
|
||||
*/
|
||||
static int LeafComparator(const void* a, const void* b) {
|
||||
return ((const Node*)a)->weight - ((const Node*)b)->weight;
|
||||
}
|
||||
|
||||
int ZopfliLengthLimitedCodeLengths(
|
||||
const size_t* frequencies, int n, int maxbits, unsigned* bitlengths) {
|
||||
NodePool pool;
|
||||
int i;
|
||||
int numsymbols = 0; /* Amount of symbols with frequency > 0. */
|
||||
int numBoundaryPMRuns;
|
||||
Node* nodes;
|
||||
|
||||
/* Array of lists of chains. Each list requires only two lookahead chains at
|
||||
a time, so each list is a array of two Node*'s. */
|
||||
Node* (*lists)[2];
|
||||
|
||||
/* One leaf per symbol. Only numsymbols leaves will be used. */
|
||||
Node* leaves = (Node*)malloc(n * sizeof(*leaves));
|
||||
|
||||
/* Initialize all bitlengths at 0. */
|
||||
for (i = 0; i < n; i++) {
|
||||
bitlengths[i] = 0;
|
||||
}
|
||||
|
||||
/* Count used symbols and place them in the leaves. */
|
||||
for (i = 0; i < n; i++) {
|
||||
if (frequencies[i]) {
|
||||
leaves[numsymbols].weight = frequencies[i];
|
||||
leaves[numsymbols].count = i; /* Index of symbol this leaf represents. */
|
||||
numsymbols++;
|
||||
}
|
||||
}
|
||||
|
||||
/* Check special cases and error conditions. */
|
||||
if ((1 << maxbits) < numsymbols) {
|
||||
free(leaves);
|
||||
return 1; /* Error, too few maxbits to represent symbols. */
|
||||
}
|
||||
if (numsymbols == 0) {
|
||||
free(leaves);
|
||||
return 0; /* No symbols at all. OK. */
|
||||
}
|
||||
if (numsymbols == 1) {
|
||||
bitlengths[leaves[0].count] = 1;
|
||||
free(leaves);
|
||||
return 0; /* Only one symbol, give it bitlength 1, not 0. OK. */
|
||||
}
|
||||
if (numsymbols == 2) {
|
||||
bitlengths[leaves[0].count]++;
|
||||
bitlengths[leaves[1].count]++;
|
||||
free(leaves);
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* Sort the leaves from lightest to heaviest. Add count into the same
|
||||
variable for stable sorting. */
|
||||
for (i = 0; i < numsymbols; i++) {
|
||||
if (leaves[i].weight >=
|
||||
((size_t)1 << (sizeof(leaves[0].weight) * CHAR_BIT - 9))) {
|
||||
free(leaves);
|
||||
return 1; /* Error, we need 9 bits for the count. */
|
||||
}
|
||||
leaves[i].weight = (leaves[i].weight << 9) | leaves[i].count;
|
||||
}
|
||||
qsort(leaves, numsymbols, sizeof(Node), LeafComparator);
|
||||
for (i = 0; i < numsymbols; i++) {
|
||||
leaves[i].weight >>= 9;
|
||||
}
|
||||
|
||||
if (numsymbols - 1 < maxbits) {
|
||||
maxbits = numsymbols - 1;
|
||||
}
|
||||
|
||||
/* Initialize node memory pool. */
|
||||
nodes = (Node*)malloc(maxbits * 2 * numsymbols * sizeof(Node));
|
||||
pool.next = nodes;
|
||||
|
||||
lists = (Node* (*)[2])malloc(maxbits * sizeof(*lists));
|
||||
InitLists(&pool, leaves, maxbits, lists);
|
||||
|
||||
/* In the last list, 2 * numsymbols - 2 active chains need to be created. Two
|
||||
are already created in the initialization. Each BoundaryPM run creates one. */
|
||||
numBoundaryPMRuns = 2 * numsymbols - 4;
|
||||
for (i = 0; i < numBoundaryPMRuns - 1; i++) {
|
||||
BoundaryPM(lists, leaves, numsymbols, &pool, maxbits - 1);
|
||||
}
|
||||
BoundaryPMFinal(lists, leaves, numsymbols, &pool, maxbits - 1);
|
||||
|
||||
ExtractBitLengths(lists[maxbits - 1][1], leaves, bitlengths);
|
||||
|
||||
free(lists);
|
||||
free(leaves);
|
||||
free(nodes);
|
||||
return 0; /* OK. */
|
||||
}
|
||||
42
build/node_modules/node-zopfli/zopfli/src/zopfli/katajainen.h
generated
vendored
Normal file
42
build/node_modules/node-zopfli/zopfli/src/zopfli/katajainen.h
generated
vendored
Normal file
@@ -0,0 +1,42 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
#ifndef ZOPFLI_KATAJAINEN_H_
|
||||
#define ZOPFLI_KATAJAINEN_H_
|
||||
|
||||
#include <string.h>
|
||||
|
||||
/*
|
||||
Outputs minimum-redundancy length-limited code bitlengths for symbols with the
|
||||
given counts. The bitlengths are limited by maxbits.
|
||||
|
||||
The output is tailored for DEFLATE: symbols that never occur, get a bit length
|
||||
of 0, and if only a single symbol occurs at least once, its bitlength will be 1,
|
||||
and not 0 as would theoretically be needed for a single symbol.
|
||||
|
||||
frequencies: The amount of occurrences of each symbol.
|
||||
n: The amount of symbols.
|
||||
maxbits: Maximum bit length, inclusive.
|
||||
bitlengths: Output, the bitlengths for the symbol prefix codes.
|
||||
return: 0 for OK, non-0 for error.
|
||||
*/
|
||||
int ZopfliLengthLimitedCodeLengths(
|
||||
const size_t* frequencies, int n, int maxbits, unsigned* bitlengths);
|
||||
|
||||
#endif /* ZOPFLI_KATAJAINEN_H_ */
|
||||
630
build/node_modules/node-zopfli/zopfli/src/zopfli/lz77.c
generated
vendored
Normal file
630
build/node_modules/node-zopfli/zopfli/src/zopfli/lz77.c
generated
vendored
Normal file
@@ -0,0 +1,630 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
#include "lz77.h"
|
||||
#include "symbols.h"
|
||||
#include "util.h"
|
||||
|
||||
#include <assert.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
|
||||
void ZopfliInitLZ77Store(const unsigned char* data, ZopfliLZ77Store* store) {
|
||||
store->size = 0;
|
||||
store->litlens = 0;
|
||||
store->dists = 0;
|
||||
store->pos = 0;
|
||||
store->data = data;
|
||||
store->ll_symbol = 0;
|
||||
store->d_symbol = 0;
|
||||
store->ll_counts = 0;
|
||||
store->d_counts = 0;
|
||||
}
|
||||
|
||||
void ZopfliCleanLZ77Store(ZopfliLZ77Store* store) {
|
||||
free(store->litlens);
|
||||
free(store->dists);
|
||||
free(store->pos);
|
||||
free(store->ll_symbol);
|
||||
free(store->d_symbol);
|
||||
free(store->ll_counts);
|
||||
free(store->d_counts);
|
||||
}
|
||||
|
||||
static size_t CeilDiv(size_t a, size_t b) {
|
||||
return (a + b - 1) / b;
|
||||
}
|
||||
|
||||
void ZopfliCopyLZ77Store(
|
||||
const ZopfliLZ77Store* source, ZopfliLZ77Store* dest) {
|
||||
size_t i;
|
||||
size_t llsize = ZOPFLI_NUM_LL * CeilDiv(source->size, ZOPFLI_NUM_LL);
|
||||
size_t dsize = ZOPFLI_NUM_D * CeilDiv(source->size, ZOPFLI_NUM_D);
|
||||
ZopfliCleanLZ77Store(dest);
|
||||
ZopfliInitLZ77Store(source->data, dest);
|
||||
dest->litlens =
|
||||
(unsigned short*)malloc(sizeof(*dest->litlens) * source->size);
|
||||
dest->dists = (unsigned short*)malloc(sizeof(*dest->dists) * source->size);
|
||||
dest->pos = (size_t*)malloc(sizeof(*dest->pos) * source->size);
|
||||
dest->ll_symbol =
|
||||
(unsigned short*)malloc(sizeof(*dest->ll_symbol) * source->size);
|
||||
dest->d_symbol =
|
||||
(unsigned short*)malloc(sizeof(*dest->d_symbol) * source->size);
|
||||
dest->ll_counts = (size_t*)malloc(sizeof(*dest->ll_counts) * llsize);
|
||||
dest->d_counts = (size_t*)malloc(sizeof(*dest->d_counts) * dsize);
|
||||
|
||||
/* Allocation failed. */
|
||||
if (!dest->litlens || !dest->dists) exit(-1);
|
||||
if (!dest->pos) exit(-1);
|
||||
if (!dest->ll_symbol || !dest->d_symbol) exit(-1);
|
||||
if (!dest->ll_counts || !dest->d_counts) exit(-1);
|
||||
|
||||
dest->size = source->size;
|
||||
for (i = 0; i < source->size; i++) {
|
||||
dest->litlens[i] = source->litlens[i];
|
||||
dest->dists[i] = source->dists[i];
|
||||
dest->pos[i] = source->pos[i];
|
||||
dest->ll_symbol[i] = source->ll_symbol[i];
|
||||
dest->d_symbol[i] = source->d_symbol[i];
|
||||
}
|
||||
for (i = 0; i < llsize; i++) {
|
||||
dest->ll_counts[i] = source->ll_counts[i];
|
||||
}
|
||||
for (i = 0; i < dsize; i++) {
|
||||
dest->d_counts[i] = source->d_counts[i];
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Appends the length and distance to the LZ77 arrays of the ZopfliLZ77Store.
|
||||
context must be a ZopfliLZ77Store*.
|
||||
*/
|
||||
void ZopfliStoreLitLenDist(unsigned short length, unsigned short dist,
|
||||
size_t pos, ZopfliLZ77Store* store) {
|
||||
size_t i;
|
||||
/* Needed for using ZOPFLI_APPEND_DATA multiple times. */
|
||||
size_t origsize = store->size;
|
||||
size_t llstart = ZOPFLI_NUM_LL * (origsize / ZOPFLI_NUM_LL);
|
||||
size_t dstart = ZOPFLI_NUM_D * (origsize / ZOPFLI_NUM_D);
|
||||
|
||||
/* Everytime the index wraps around, a new cumulative histogram is made: we're
|
||||
keeping one histogram value per LZ77 symbol rather than a full histogram for
|
||||
each to save memory. */
|
||||
if (origsize % ZOPFLI_NUM_LL == 0) {
|
||||
size_t llsize = origsize;
|
||||
for (i = 0; i < ZOPFLI_NUM_LL; i++) {
|
||||
ZOPFLI_APPEND_DATA(
|
||||
origsize == 0 ? 0 : store->ll_counts[origsize - ZOPFLI_NUM_LL + i],
|
||||
&store->ll_counts, &llsize);
|
||||
}
|
||||
}
|
||||
if (origsize % ZOPFLI_NUM_D == 0) {
|
||||
size_t dsize = origsize;
|
||||
for (i = 0; i < ZOPFLI_NUM_D; i++) {
|
||||
ZOPFLI_APPEND_DATA(
|
||||
origsize == 0 ? 0 : store->d_counts[origsize - ZOPFLI_NUM_D + i],
|
||||
&store->d_counts, &dsize);
|
||||
}
|
||||
}
|
||||
|
||||
ZOPFLI_APPEND_DATA(length, &store->litlens, &store->size);
|
||||
store->size = origsize;
|
||||
ZOPFLI_APPEND_DATA(dist, &store->dists, &store->size);
|
||||
store->size = origsize;
|
||||
ZOPFLI_APPEND_DATA(pos, &store->pos, &store->size);
|
||||
assert(length < 259);
|
||||
|
||||
if (dist == 0) {
|
||||
store->size = origsize;
|
||||
ZOPFLI_APPEND_DATA(length, &store->ll_symbol, &store->size);
|
||||
store->size = origsize;
|
||||
ZOPFLI_APPEND_DATA(0, &store->d_symbol, &store->size);
|
||||
store->ll_counts[llstart + length]++;
|
||||
} else {
|
||||
store->size = origsize;
|
||||
ZOPFLI_APPEND_DATA(ZopfliGetLengthSymbol(length),
|
||||
&store->ll_symbol, &store->size);
|
||||
store->size = origsize;
|
||||
ZOPFLI_APPEND_DATA(ZopfliGetDistSymbol(dist),
|
||||
&store->d_symbol, &store->size);
|
||||
store->ll_counts[llstart + ZopfliGetLengthSymbol(length)]++;
|
||||
store->d_counts[dstart + ZopfliGetDistSymbol(dist)]++;
|
||||
}
|
||||
}
|
||||
|
||||
void ZopfliAppendLZ77Store(const ZopfliLZ77Store* store,
|
||||
ZopfliLZ77Store* target) {
|
||||
size_t i;
|
||||
for (i = 0; i < store->size; i++) {
|
||||
ZopfliStoreLitLenDist(store->litlens[i], store->dists[i],
|
||||
store->pos[i], target);
|
||||
}
|
||||
}
|
||||
|
||||
size_t ZopfliLZ77GetByteRange(const ZopfliLZ77Store* lz77,
|
||||
size_t lstart, size_t lend) {
|
||||
size_t l = lend - 1;
|
||||
if (lstart == lend) return 0;
|
||||
return lz77->pos[l] + ((lz77->dists[l] == 0) ?
|
||||
1 : lz77->litlens[l]) - lz77->pos[lstart];
|
||||
}
|
||||
|
||||
static void ZopfliLZ77GetHistogramAt(const ZopfliLZ77Store* lz77, size_t lpos,
|
||||
size_t* ll_counts, size_t* d_counts) {
|
||||
/* The real histogram is created by using the histogram for this chunk, but
|
||||
all superfluous values of this chunk subtracted. */
|
||||
size_t llpos = ZOPFLI_NUM_LL * (lpos / ZOPFLI_NUM_LL);
|
||||
size_t dpos = ZOPFLI_NUM_D * (lpos / ZOPFLI_NUM_D);
|
||||
size_t i;
|
||||
for (i = 0; i < ZOPFLI_NUM_LL; i++) {
|
||||
ll_counts[i] = lz77->ll_counts[llpos + i];
|
||||
}
|
||||
for (i = lpos + 1; i < llpos + ZOPFLI_NUM_LL && i < lz77->size; i++) {
|
||||
ll_counts[lz77->ll_symbol[i]]--;
|
||||
}
|
||||
for (i = 0; i < ZOPFLI_NUM_D; i++) {
|
||||
d_counts[i] = lz77->d_counts[dpos + i];
|
||||
}
|
||||
for (i = lpos + 1; i < dpos + ZOPFLI_NUM_D && i < lz77->size; i++) {
|
||||
if (lz77->dists[i] != 0) d_counts[lz77->d_symbol[i]]--;
|
||||
}
|
||||
}
|
||||
|
||||
void ZopfliLZ77GetHistogram(const ZopfliLZ77Store* lz77,
|
||||
size_t lstart, size_t lend,
|
||||
size_t* ll_counts, size_t* d_counts) {
|
||||
size_t i;
|
||||
if (lstart + ZOPFLI_NUM_LL * 3 > lend) {
|
||||
memset(ll_counts, 0, sizeof(*ll_counts) * ZOPFLI_NUM_LL);
|
||||
memset(d_counts, 0, sizeof(*d_counts) * ZOPFLI_NUM_D);
|
||||
for (i = lstart; i < lend; i++) {
|
||||
ll_counts[lz77->ll_symbol[i]]++;
|
||||
if (lz77->dists[i] != 0) d_counts[lz77->d_symbol[i]]++;
|
||||
}
|
||||
} else {
|
||||
/* Subtract the cumulative histograms at the end and the start to get the
|
||||
histogram for this range. */
|
||||
ZopfliLZ77GetHistogramAt(lz77, lend - 1, ll_counts, d_counts);
|
||||
if (lstart > 0) {
|
||||
size_t ll_counts2[ZOPFLI_NUM_LL];
|
||||
size_t d_counts2[ZOPFLI_NUM_D];
|
||||
ZopfliLZ77GetHistogramAt(lz77, lstart - 1, ll_counts2, d_counts2);
|
||||
|
||||
for (i = 0; i < ZOPFLI_NUM_LL; i++) {
|
||||
ll_counts[i] -= ll_counts2[i];
|
||||
}
|
||||
for (i = 0; i < ZOPFLI_NUM_D; i++) {
|
||||
d_counts[i] -= d_counts2[i];
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void ZopfliInitBlockState(const ZopfliOptions* options,
|
||||
size_t blockstart, size_t blockend, int add_lmc,
|
||||
ZopfliBlockState* s) {
|
||||
s->options = options;
|
||||
s->blockstart = blockstart;
|
||||
s->blockend = blockend;
|
||||
#ifdef ZOPFLI_LONGEST_MATCH_CACHE
|
||||
if (add_lmc) {
|
||||
s->lmc = (ZopfliLongestMatchCache*)malloc(sizeof(ZopfliLongestMatchCache));
|
||||
ZopfliInitCache(blockend - blockstart, s->lmc);
|
||||
} else {
|
||||
s->lmc = 0;
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
void ZopfliCleanBlockState(ZopfliBlockState* s) {
|
||||
#ifdef ZOPFLI_LONGEST_MATCH_CACHE
|
||||
if (s->lmc) {
|
||||
ZopfliCleanCache(s->lmc);
|
||||
free(s->lmc);
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
/*
|
||||
Gets a score of the length given the distance. Typically, the score of the
|
||||
length is the length itself, but if the distance is very long, decrease the
|
||||
score of the length a bit to make up for the fact that long distances use large
|
||||
amounts of extra bits.
|
||||
|
||||
This is not an accurate score, it is a heuristic only for the greedy LZ77
|
||||
implementation. More accurate cost models are employed later. Making this
|
||||
heuristic more accurate may hurt rather than improve compression.
|
||||
|
||||
The two direct uses of this heuristic are:
|
||||
-avoid using a length of 3 in combination with a long distance. This only has
|
||||
an effect if length == 3.
|
||||
-make a slightly better choice between the two options of the lazy matching.
|
||||
|
||||
Indirectly, this affects:
|
||||
-the block split points if the default of block splitting first is used, in a
|
||||
rather unpredictable way
|
||||
-the first zopfli run, so it affects the chance of the first run being closer
|
||||
to the optimal output
|
||||
*/
|
||||
static int GetLengthScore(int length, int distance) {
|
||||
/*
|
||||
At 1024, the distance uses 9+ extra bits and this seems to be the sweet spot
|
||||
on tested files.
|
||||
*/
|
||||
return distance > 1024 ? length - 1 : length;
|
||||
}
|
||||
|
||||
void ZopfliVerifyLenDist(const unsigned char* data, size_t datasize, size_t pos,
|
||||
unsigned short dist, unsigned short length) {
|
||||
|
||||
/* TODO(lode): make this only run in a debug compile, it's for assert only. */
|
||||
size_t i;
|
||||
|
||||
assert(pos + length <= datasize);
|
||||
for (i = 0; i < length; i++) {
|
||||
if (data[pos - dist + i] != data[pos + i]) {
|
||||
assert(data[pos - dist + i] == data[pos + i]);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Finds how long the match of scan and match is. Can be used to find how many
|
||||
bytes starting from scan, and from match, are equal. Returns the last byte
|
||||
after scan, which is still equal to the correspondinb byte after match.
|
||||
scan is the position to compare
|
||||
match is the earlier position to compare.
|
||||
end is the last possible byte, beyond which to stop looking.
|
||||
safe_end is a few (8) bytes before end, for comparing multiple bytes at once.
|
||||
*/
|
||||
static const unsigned char* GetMatch(const unsigned char* scan,
|
||||
const unsigned char* match,
|
||||
const unsigned char* end,
|
||||
const unsigned char* safe_end) {
|
||||
|
||||
if (sizeof(size_t) == 8) {
|
||||
/* 8 checks at once per array bounds check (size_t is 64-bit). */
|
||||
while (scan < safe_end && *((size_t*)scan) == *((size_t*)match)) {
|
||||
scan += 8;
|
||||
match += 8;
|
||||
}
|
||||
} else if (sizeof(unsigned int) == 4) {
|
||||
/* 4 checks at once per array bounds check (unsigned int is 32-bit). */
|
||||
while (scan < safe_end
|
||||
&& *((unsigned int*)scan) == *((unsigned int*)match)) {
|
||||
scan += 4;
|
||||
match += 4;
|
||||
}
|
||||
} else {
|
||||
/* do 8 checks at once per array bounds check. */
|
||||
while (scan < safe_end && *scan == *match && *++scan == *++match
|
||||
&& *++scan == *++match && *++scan == *++match
|
||||
&& *++scan == *++match && *++scan == *++match
|
||||
&& *++scan == *++match && *++scan == *++match) {
|
||||
scan++; match++;
|
||||
}
|
||||
}
|
||||
|
||||
/* The remaining few bytes. */
|
||||
while (scan != end && *scan == *match) {
|
||||
scan++; match++;
|
||||
}
|
||||
|
||||
return scan;
|
||||
}
|
||||
|
||||
#ifdef ZOPFLI_LONGEST_MATCH_CACHE
|
||||
/*
|
||||
Gets distance, length and sublen values from the cache if possible.
|
||||
Returns 1 if it got the values from the cache, 0 if not.
|
||||
Updates the limit value to a smaller one if possible with more limited
|
||||
information from the cache.
|
||||
*/
|
||||
static int TryGetFromLongestMatchCache(ZopfliBlockState* s,
|
||||
size_t pos, size_t* limit,
|
||||
unsigned short* sublen, unsigned short* distance, unsigned short* length) {
|
||||
/* The LMC cache starts at the beginning of the block rather than the
|
||||
beginning of the whole array. */
|
||||
size_t lmcpos = pos - s->blockstart;
|
||||
|
||||
/* Length > 0 and dist 0 is invalid combination, which indicates on purpose
|
||||
that this cache value is not filled in yet. */
|
||||
unsigned char cache_available = s->lmc && (s->lmc->length[lmcpos] == 0 ||
|
||||
s->lmc->dist[lmcpos] != 0);
|
||||
unsigned char limit_ok_for_cache = cache_available &&
|
||||
(*limit == ZOPFLI_MAX_MATCH || s->lmc->length[lmcpos] <= *limit ||
|
||||
(sublen && ZopfliMaxCachedSublen(s->lmc,
|
||||
lmcpos, s->lmc->length[lmcpos]) >= *limit));
|
||||
|
||||
if (s->lmc && limit_ok_for_cache && cache_available) {
|
||||
if (!sublen || s->lmc->length[lmcpos]
|
||||
<= ZopfliMaxCachedSublen(s->lmc, lmcpos, s->lmc->length[lmcpos])) {
|
||||
*length = s->lmc->length[lmcpos];
|
||||
if (*length > *limit) *length = *limit;
|
||||
if (sublen) {
|
||||
ZopfliCacheToSublen(s->lmc, lmcpos, *length, sublen);
|
||||
*distance = sublen[*length];
|
||||
if (*limit == ZOPFLI_MAX_MATCH && *length >= ZOPFLI_MIN_MATCH) {
|
||||
assert(sublen[*length] == s->lmc->dist[lmcpos]);
|
||||
}
|
||||
} else {
|
||||
*distance = s->lmc->dist[lmcpos];
|
||||
}
|
||||
return 1;
|
||||
}
|
||||
/* Can't use much of the cache, since the "sublens" need to be calculated,
|
||||
but at least we already know when to stop. */
|
||||
*limit = s->lmc->length[lmcpos];
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
/*
|
||||
Stores the found sublen, distance and length in the longest match cache, if
|
||||
possible.
|
||||
*/
|
||||
static void StoreInLongestMatchCache(ZopfliBlockState* s,
|
||||
size_t pos, size_t limit,
|
||||
const unsigned short* sublen,
|
||||
unsigned short distance, unsigned short length) {
|
||||
/* The LMC cache starts at the beginning of the block rather than the
|
||||
beginning of the whole array. */
|
||||
size_t lmcpos = pos - s->blockstart;
|
||||
|
||||
/* Length > 0 and dist 0 is invalid combination, which indicates on purpose
|
||||
that this cache value is not filled in yet. */
|
||||
unsigned char cache_available = s->lmc && (s->lmc->length[lmcpos] == 0 ||
|
||||
s->lmc->dist[lmcpos] != 0);
|
||||
|
||||
if (s->lmc && limit == ZOPFLI_MAX_MATCH && sublen && !cache_available) {
|
||||
assert(s->lmc->length[lmcpos] == 1 && s->lmc->dist[lmcpos] == 0);
|
||||
s->lmc->dist[lmcpos] = length < ZOPFLI_MIN_MATCH ? 0 : distance;
|
||||
s->lmc->length[lmcpos] = length < ZOPFLI_MIN_MATCH ? 0 : length;
|
||||
assert(!(s->lmc->length[lmcpos] == 1 && s->lmc->dist[lmcpos] == 0));
|
||||
ZopfliSublenToCache(sublen, lmcpos, length, s->lmc);
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
void ZopfliFindLongestMatch(ZopfliBlockState* s, const ZopfliHash* h,
|
||||
const unsigned char* array,
|
||||
size_t pos, size_t size, size_t limit,
|
||||
unsigned short* sublen, unsigned short* distance, unsigned short* length) {
|
||||
unsigned short hpos = pos & ZOPFLI_WINDOW_MASK, p, pp;
|
||||
unsigned short bestdist = 0;
|
||||
unsigned short bestlength = 1;
|
||||
const unsigned char* scan;
|
||||
const unsigned char* match;
|
||||
const unsigned char* arrayend;
|
||||
const unsigned char* arrayend_safe;
|
||||
#if ZOPFLI_MAX_CHAIN_HITS < ZOPFLI_WINDOW_SIZE
|
||||
int chain_counter = ZOPFLI_MAX_CHAIN_HITS; /* For quitting early. */
|
||||
#endif
|
||||
|
||||
unsigned dist = 0; /* Not unsigned short on purpose. */
|
||||
|
||||
int* hhead = h->head;
|
||||
unsigned short* hprev = h->prev;
|
||||
int* hhashval = h->hashval;
|
||||
int hval = h->val;
|
||||
|
||||
#ifdef ZOPFLI_LONGEST_MATCH_CACHE
|
||||
if (TryGetFromLongestMatchCache(s, pos, &limit, sublen, distance, length)) {
|
||||
assert(pos + *length <= size);
|
||||
return;
|
||||
}
|
||||
#endif
|
||||
|
||||
assert(limit <= ZOPFLI_MAX_MATCH);
|
||||
assert(limit >= ZOPFLI_MIN_MATCH);
|
||||
assert(pos < size);
|
||||
|
||||
if (size - pos < ZOPFLI_MIN_MATCH) {
|
||||
/* The rest of the code assumes there are at least ZOPFLI_MIN_MATCH bytes to
|
||||
try. */
|
||||
*length = 0;
|
||||
*distance = 0;
|
||||
return;
|
||||
}
|
||||
|
||||
if (pos + limit > size) {
|
||||
limit = size - pos;
|
||||
}
|
||||
arrayend = &array[pos] + limit;
|
||||
arrayend_safe = arrayend - 8;
|
||||
|
||||
assert(hval < 65536);
|
||||
|
||||
pp = hhead[hval]; /* During the whole loop, p == hprev[pp]. */
|
||||
p = hprev[pp];
|
||||
|
||||
assert(pp == hpos);
|
||||
|
||||
dist = p < pp ? pp - p : ((ZOPFLI_WINDOW_SIZE - p) + pp);
|
||||
|
||||
/* Go through all distances. */
|
||||
while (dist < ZOPFLI_WINDOW_SIZE) {
|
||||
unsigned short currentlength = 0;
|
||||
|
||||
assert(p < ZOPFLI_WINDOW_SIZE);
|
||||
assert(p == hprev[pp]);
|
||||
assert(hhashval[p] == hval);
|
||||
|
||||
if (dist > 0) {
|
||||
assert(pos < size);
|
||||
assert(dist <= pos);
|
||||
scan = &array[pos];
|
||||
match = &array[pos - dist];
|
||||
|
||||
/* Testing the byte at position bestlength first, goes slightly faster. */
|
||||
if (pos + bestlength >= size
|
||||
|| *(scan + bestlength) == *(match + bestlength)) {
|
||||
|
||||
#ifdef ZOPFLI_HASH_SAME
|
||||
unsigned short same0 = h->same[pos & ZOPFLI_WINDOW_MASK];
|
||||
if (same0 > 2 && *scan == *match) {
|
||||
unsigned short same1 = h->same[(pos - dist) & ZOPFLI_WINDOW_MASK];
|
||||
unsigned short same = same0 < same1 ? same0 : same1;
|
||||
if (same > limit) same = limit;
|
||||
scan += same;
|
||||
match += same;
|
||||
}
|
||||
#endif
|
||||
scan = GetMatch(scan, match, arrayend, arrayend_safe);
|
||||
currentlength = scan - &array[pos]; /* The found length. */
|
||||
}
|
||||
|
||||
if (currentlength > bestlength) {
|
||||
if (sublen) {
|
||||
unsigned short j;
|
||||
for (j = bestlength + 1; j <= currentlength; j++) {
|
||||
sublen[j] = dist;
|
||||
}
|
||||
}
|
||||
bestdist = dist;
|
||||
bestlength = currentlength;
|
||||
if (currentlength >= limit) break;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
#ifdef ZOPFLI_HASH_SAME_HASH
|
||||
/* Switch to the other hash once this will be more efficient. */
|
||||
if (hhead != h->head2 && bestlength >= h->same[hpos] &&
|
||||
h->val2 == h->hashval2[p]) {
|
||||
/* Now use the hash that encodes the length and first byte. */
|
||||
hhead = h->head2;
|
||||
hprev = h->prev2;
|
||||
hhashval = h->hashval2;
|
||||
hval = h->val2;
|
||||
}
|
||||
#endif
|
||||
|
||||
pp = p;
|
||||
p = hprev[p];
|
||||
if (p == pp) break; /* Uninited prev value. */
|
||||
|
||||
dist += p < pp ? pp - p : ((ZOPFLI_WINDOW_SIZE - p) + pp);
|
||||
|
||||
#if ZOPFLI_MAX_CHAIN_HITS < ZOPFLI_WINDOW_SIZE
|
||||
chain_counter--;
|
||||
if (chain_counter <= 0) break;
|
||||
#endif
|
||||
}
|
||||
|
||||
#ifdef ZOPFLI_LONGEST_MATCH_CACHE
|
||||
StoreInLongestMatchCache(s, pos, limit, sublen, bestdist, bestlength);
|
||||
#endif
|
||||
|
||||
assert(bestlength <= limit);
|
||||
|
||||
*distance = bestdist;
|
||||
*length = bestlength;
|
||||
assert(pos + *length <= size);
|
||||
}
|
||||
|
||||
void ZopfliLZ77Greedy(ZopfliBlockState* s, const unsigned char* in,
|
||||
size_t instart, size_t inend,
|
||||
ZopfliLZ77Store* store, ZopfliHash* h) {
|
||||
size_t i = 0, j;
|
||||
unsigned short leng;
|
||||
unsigned short dist;
|
||||
int lengthscore;
|
||||
size_t windowstart = instart > ZOPFLI_WINDOW_SIZE
|
||||
? instart - ZOPFLI_WINDOW_SIZE : 0;
|
||||
unsigned short dummysublen[259];
|
||||
|
||||
#ifdef ZOPFLI_LAZY_MATCHING
|
||||
/* Lazy matching. */
|
||||
unsigned prev_length = 0;
|
||||
unsigned prev_match = 0;
|
||||
int prevlengthscore;
|
||||
int match_available = 0;
|
||||
#endif
|
||||
|
||||
if (instart == inend) return;
|
||||
|
||||
ZopfliResetHash(ZOPFLI_WINDOW_SIZE, h);
|
||||
ZopfliWarmupHash(in, windowstart, inend, h);
|
||||
for (i = windowstart; i < instart; i++) {
|
||||
ZopfliUpdateHash(in, i, inend, h);
|
||||
}
|
||||
|
||||
for (i = instart; i < inend; i++) {
|
||||
ZopfliUpdateHash(in, i, inend, h);
|
||||
|
||||
ZopfliFindLongestMatch(s, h, in, i, inend, ZOPFLI_MAX_MATCH, dummysublen,
|
||||
&dist, &leng);
|
||||
lengthscore = GetLengthScore(leng, dist);
|
||||
|
||||
#ifdef ZOPFLI_LAZY_MATCHING
|
||||
/* Lazy matching. */
|
||||
prevlengthscore = GetLengthScore(prev_length, prev_match);
|
||||
if (match_available) {
|
||||
match_available = 0;
|
||||
if (lengthscore > prevlengthscore + 1) {
|
||||
ZopfliStoreLitLenDist(in[i - 1], 0, i - 1, store);
|
||||
if (lengthscore >= ZOPFLI_MIN_MATCH && leng < ZOPFLI_MAX_MATCH) {
|
||||
match_available = 1;
|
||||
prev_length = leng;
|
||||
prev_match = dist;
|
||||
continue;
|
||||
}
|
||||
} else {
|
||||
/* Add previous to output. */
|
||||
leng = prev_length;
|
||||
dist = prev_match;
|
||||
lengthscore = prevlengthscore;
|
||||
/* Add to output. */
|
||||
ZopfliVerifyLenDist(in, inend, i - 1, dist, leng);
|
||||
ZopfliStoreLitLenDist(leng, dist, i - 1, store);
|
||||
for (j = 2; j < leng; j++) {
|
||||
assert(i < inend);
|
||||
i++;
|
||||
ZopfliUpdateHash(in, i, inend, h);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
}
|
||||
else if (lengthscore >= ZOPFLI_MIN_MATCH && leng < ZOPFLI_MAX_MATCH) {
|
||||
match_available = 1;
|
||||
prev_length = leng;
|
||||
prev_match = dist;
|
||||
continue;
|
||||
}
|
||||
/* End of lazy matching. */
|
||||
#endif
|
||||
|
||||
/* Add to output. */
|
||||
if (lengthscore >= ZOPFLI_MIN_MATCH) {
|
||||
ZopfliVerifyLenDist(in, inend, i, dist, leng);
|
||||
ZopfliStoreLitLenDist(leng, dist, i, store);
|
||||
} else {
|
||||
leng = 1;
|
||||
ZopfliStoreLitLenDist(in[i], 0, i, store);
|
||||
}
|
||||
for (j = 1; j < leng; j++) {
|
||||
assert(i < inend);
|
||||
i++;
|
||||
ZopfliUpdateHash(in, i, inend, h);
|
||||
}
|
||||
}
|
||||
}
|
||||
142
build/node_modules/node-zopfli/zopfli/src/zopfli/lz77.h
generated
vendored
Normal file
142
build/node_modules/node-zopfli/zopfli/src/zopfli/lz77.h
generated
vendored
Normal file
@@ -0,0 +1,142 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
/*
|
||||
Functions for basic LZ77 compression and utilities for the "squeeze" LZ77
|
||||
compression.
|
||||
*/
|
||||
|
||||
#ifndef ZOPFLI_LZ77_H_
|
||||
#define ZOPFLI_LZ77_H_
|
||||
|
||||
#include <stdlib.h>
|
||||
|
||||
#include "cache.h"
|
||||
#include "hash.h"
|
||||
#include "zopfli.h"
|
||||
|
||||
/*
|
||||
Stores lit/length and dist pairs for LZ77.
|
||||
Parameter litlens: Contains the literal symbols or length values.
|
||||
Parameter dists: Contains the distances. A value is 0 to indicate that there is
|
||||
no dist and the corresponding litlens value is a literal instead of a length.
|
||||
Parameter size: The size of both the litlens and dists arrays.
|
||||
The memory can best be managed by using ZopfliInitLZ77Store to initialize it,
|
||||
ZopfliCleanLZ77Store to destroy it, and ZopfliStoreLitLenDist to append values.
|
||||
|
||||
*/
|
||||
typedef struct ZopfliLZ77Store {
|
||||
unsigned short* litlens; /* Lit or len. */
|
||||
unsigned short* dists; /* If 0: indicates literal in corresponding litlens,
|
||||
if > 0: length in corresponding litlens, this is the distance. */
|
||||
size_t size;
|
||||
|
||||
const unsigned char* data; /* original data */
|
||||
size_t* pos; /* position in data where this LZ77 command begins */
|
||||
|
||||
unsigned short* ll_symbol;
|
||||
unsigned short* d_symbol;
|
||||
|
||||
/* Cumulative histograms wrapping around per chunk. Each chunk has the amount
|
||||
of distinct symbols as length, so using 1 value per LZ77 symbol, we have a
|
||||
precise histogram at every N symbols, and the rest can be calculated by
|
||||
looping through the actual symbols of this chunk. */
|
||||
size_t* ll_counts;
|
||||
size_t* d_counts;
|
||||
} ZopfliLZ77Store;
|
||||
|
||||
void ZopfliInitLZ77Store(const unsigned char* data, ZopfliLZ77Store* store);
|
||||
void ZopfliCleanLZ77Store(ZopfliLZ77Store* store);
|
||||
void ZopfliCopyLZ77Store(const ZopfliLZ77Store* source, ZopfliLZ77Store* dest);
|
||||
void ZopfliStoreLitLenDist(unsigned short length, unsigned short dist,
|
||||
size_t pos, ZopfliLZ77Store* store);
|
||||
void ZopfliAppendLZ77Store(const ZopfliLZ77Store* store,
|
||||
ZopfliLZ77Store* target);
|
||||
/* Gets the amount of raw bytes that this range of LZ77 symbols spans. */
|
||||
size_t ZopfliLZ77GetByteRange(const ZopfliLZ77Store* lz77,
|
||||
size_t lstart, size_t lend);
|
||||
/* Gets the histogram of lit/len and dist symbols in the given range, using the
|
||||
cumulative histograms, so faster than adding one by one for large range. Does
|
||||
not add the one end symbol of value 256. */
|
||||
void ZopfliLZ77GetHistogram(const ZopfliLZ77Store* lz77,
|
||||
size_t lstart, size_t lend,
|
||||
size_t* ll_counts, size_t* d_counts);
|
||||
|
||||
/*
|
||||
Some state information for compressing a block.
|
||||
This is currently a bit under-used (with mainly only the longest match cache),
|
||||
but is kept for easy future expansion.
|
||||
*/
|
||||
typedef struct ZopfliBlockState {
|
||||
const ZopfliOptions* options;
|
||||
|
||||
#ifdef ZOPFLI_LONGEST_MATCH_CACHE
|
||||
/* Cache for length/distance pairs found so far. */
|
||||
ZopfliLongestMatchCache* lmc;
|
||||
#endif
|
||||
|
||||
/* The start (inclusive) and end (not inclusive) of the current block. */
|
||||
size_t blockstart;
|
||||
size_t blockend;
|
||||
} ZopfliBlockState;
|
||||
|
||||
void ZopfliInitBlockState(const ZopfliOptions* options,
|
||||
size_t blockstart, size_t blockend, int add_lmc,
|
||||
ZopfliBlockState* s);
|
||||
void ZopfliCleanBlockState(ZopfliBlockState* s);
|
||||
|
||||
/*
|
||||
Finds the longest match (length and corresponding distance) for LZ77
|
||||
compression.
|
||||
Even when not using "sublen", it can be more efficient to provide an array,
|
||||
because only then the caching is used.
|
||||
array: the data
|
||||
pos: position in the data to find the match for
|
||||
size: size of the data
|
||||
limit: limit length to maximum this value (default should be 258). This allows
|
||||
finding a shorter dist for that length (= less extra bits). Must be
|
||||
in the range [ZOPFLI_MIN_MATCH, ZOPFLI_MAX_MATCH].
|
||||
sublen: output array of 259 elements, or null. Has, for each length, the
|
||||
smallest distance required to reach this length. Only 256 of its 259 values
|
||||
are used, the first 3 are ignored (the shortest length is 3. It is purely
|
||||
for convenience that the array is made 3 longer).
|
||||
*/
|
||||
void ZopfliFindLongestMatch(
|
||||
ZopfliBlockState *s, const ZopfliHash* h, const unsigned char* array,
|
||||
size_t pos, size_t size, size_t limit,
|
||||
unsigned short* sublen, unsigned short* distance, unsigned short* length);
|
||||
|
||||
/*
|
||||
Verifies if length and dist are indeed valid, only used for assertion.
|
||||
*/
|
||||
void ZopfliVerifyLenDist(const unsigned char* data, size_t datasize, size_t pos,
|
||||
unsigned short dist, unsigned short length);
|
||||
|
||||
/*
|
||||
Does LZ77 using an algorithm similar to gzip, with lazy matching, rather than
|
||||
with the slow but better "squeeze" implementation.
|
||||
The result is placed in the ZopfliLZ77Store.
|
||||
If instart is larger than 0, it uses values before instart as starting
|
||||
dictionary.
|
||||
*/
|
||||
void ZopfliLZ77Greedy(ZopfliBlockState* s, const unsigned char* in,
|
||||
size_t instart, size_t inend,
|
||||
ZopfliLZ77Store* store, ZopfliHash* h);
|
||||
|
||||
#endif /* ZOPFLI_LZ77_H_ */
|
||||
560
build/node_modules/node-zopfli/zopfli/src/zopfli/squeeze.c
generated
vendored
Normal file
560
build/node_modules/node-zopfli/zopfli/src/zopfli/squeeze.c
generated
vendored
Normal file
@@ -0,0 +1,560 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
#include "squeeze.h"
|
||||
|
||||
#include <assert.h>
|
||||
#include <math.h>
|
||||
#include <stdio.h>
|
||||
|
||||
#include "blocksplitter.h"
|
||||
#include "deflate.h"
|
||||
#include "symbols.h"
|
||||
#include "tree.h"
|
||||
#include "util.h"
|
||||
|
||||
typedef struct SymbolStats {
|
||||
/* The literal and length symbols. */
|
||||
size_t litlens[ZOPFLI_NUM_LL];
|
||||
/* The 32 unique dist symbols, not the 32768 possible dists. */
|
||||
size_t dists[ZOPFLI_NUM_D];
|
||||
|
||||
/* Length of each lit/len symbol in bits. */
|
||||
double ll_symbols[ZOPFLI_NUM_LL];
|
||||
/* Length of each dist symbol in bits. */
|
||||
double d_symbols[ZOPFLI_NUM_D];
|
||||
} SymbolStats;
|
||||
|
||||
/* Sets everything to 0. */
|
||||
static void InitStats(SymbolStats* stats) {
|
||||
memset(stats->litlens, 0, ZOPFLI_NUM_LL * sizeof(stats->litlens[0]));
|
||||
memset(stats->dists, 0, ZOPFLI_NUM_D * sizeof(stats->dists[0]));
|
||||
|
||||
memset(stats->ll_symbols, 0, ZOPFLI_NUM_LL * sizeof(stats->ll_symbols[0]));
|
||||
memset(stats->d_symbols, 0, ZOPFLI_NUM_D * sizeof(stats->d_symbols[0]));
|
||||
}
|
||||
|
||||
static void CopyStats(SymbolStats* source, SymbolStats* dest) {
|
||||
memcpy(dest->litlens, source->litlens,
|
||||
ZOPFLI_NUM_LL * sizeof(dest->litlens[0]));
|
||||
memcpy(dest->dists, source->dists, ZOPFLI_NUM_D * sizeof(dest->dists[0]));
|
||||
|
||||
memcpy(dest->ll_symbols, source->ll_symbols,
|
||||
ZOPFLI_NUM_LL * sizeof(dest->ll_symbols[0]));
|
||||
memcpy(dest->d_symbols, source->d_symbols,
|
||||
ZOPFLI_NUM_D * sizeof(dest->d_symbols[0]));
|
||||
}
|
||||
|
||||
/* Adds the bit lengths. */
|
||||
static void AddWeighedStatFreqs(const SymbolStats* stats1, double w1,
|
||||
const SymbolStats* stats2, double w2,
|
||||
SymbolStats* result) {
|
||||
size_t i;
|
||||
for (i = 0; i < ZOPFLI_NUM_LL; i++) {
|
||||
result->litlens[i] =
|
||||
(size_t) (stats1->litlens[i] * w1 + stats2->litlens[i] * w2);
|
||||
}
|
||||
for (i = 0; i < ZOPFLI_NUM_D; i++) {
|
||||
result->dists[i] =
|
||||
(size_t) (stats1->dists[i] * w1 + stats2->dists[i] * w2);
|
||||
}
|
||||
result->litlens[256] = 1; /* End symbol. */
|
||||
}
|
||||
|
||||
typedef struct RanState {
|
||||
unsigned int m_w, m_z;
|
||||
} RanState;
|
||||
|
||||
static void InitRanState(RanState* state) {
|
||||
state->m_w = 1;
|
||||
state->m_z = 2;
|
||||
}
|
||||
|
||||
/* Get random number: "Multiply-With-Carry" generator of G. Marsaglia */
|
||||
static unsigned int Ran(RanState* state) {
|
||||
state->m_z = 36969 * (state->m_z & 65535) + (state->m_z >> 16);
|
||||
state->m_w = 18000 * (state->m_w & 65535) + (state->m_w >> 16);
|
||||
return (state->m_z << 16) + state->m_w; /* 32-bit result. */
|
||||
}
|
||||
|
||||
static void RandomizeFreqs(RanState* state, size_t* freqs, int n) {
|
||||
int i;
|
||||
for (i = 0; i < n; i++) {
|
||||
if ((Ran(state) >> 4) % 3 == 0) freqs[i] = freqs[Ran(state) % n];
|
||||
}
|
||||
}
|
||||
|
||||
static void RandomizeStatFreqs(RanState* state, SymbolStats* stats) {
|
||||
RandomizeFreqs(state, stats->litlens, ZOPFLI_NUM_LL);
|
||||
RandomizeFreqs(state, stats->dists, ZOPFLI_NUM_D);
|
||||
stats->litlens[256] = 1; /* End symbol. */
|
||||
}
|
||||
|
||||
static void ClearStatFreqs(SymbolStats* stats) {
|
||||
size_t i;
|
||||
for (i = 0; i < ZOPFLI_NUM_LL; i++) stats->litlens[i] = 0;
|
||||
for (i = 0; i < ZOPFLI_NUM_D; i++) stats->dists[i] = 0;
|
||||
}
|
||||
|
||||
/*
|
||||
Function that calculates a cost based on a model for the given LZ77 symbol.
|
||||
litlen: means literal symbol if dist is 0, length otherwise.
|
||||
*/
|
||||
typedef double CostModelFun(unsigned litlen, unsigned dist, void* context);
|
||||
|
||||
/*
|
||||
Cost model which should exactly match fixed tree.
|
||||
type: CostModelFun
|
||||
*/
|
||||
static double GetCostFixed(unsigned litlen, unsigned dist, void* unused) {
|
||||
(void)unused;
|
||||
if (dist == 0) {
|
||||
if (litlen <= 143) return 8;
|
||||
else return 9;
|
||||
} else {
|
||||
int dbits = ZopfliGetDistExtraBits(dist);
|
||||
int lbits = ZopfliGetLengthExtraBits(litlen);
|
||||
int lsym = ZopfliGetLengthSymbol(litlen);
|
||||
int cost = 0;
|
||||
if (lsym <= 279) cost += 7;
|
||||
else cost += 8;
|
||||
cost += 5; /* Every dist symbol has length 5. */
|
||||
return cost + dbits + lbits;
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Cost model based on symbol statistics.
|
||||
type: CostModelFun
|
||||
*/
|
||||
static double GetCostStat(unsigned litlen, unsigned dist, void* context) {
|
||||
SymbolStats* stats = (SymbolStats*)context;
|
||||
if (dist == 0) {
|
||||
return stats->ll_symbols[litlen];
|
||||
} else {
|
||||
int lsym = ZopfliGetLengthSymbol(litlen);
|
||||
int lbits = ZopfliGetLengthExtraBits(litlen);
|
||||
int dsym = ZopfliGetDistSymbol(dist);
|
||||
int dbits = ZopfliGetDistExtraBits(dist);
|
||||
return lbits + dbits + stats->ll_symbols[lsym] + stats->d_symbols[dsym];
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Finds the minimum possible cost this cost model can return for valid length and
|
||||
distance symbols.
|
||||
*/
|
||||
static double GetCostModelMinCost(CostModelFun* costmodel, void* costcontext) {
|
||||
double mincost;
|
||||
int bestlength = 0; /* length that has lowest cost in the cost model */
|
||||
int bestdist = 0; /* distance that has lowest cost in the cost model */
|
||||
int i;
|
||||
/*
|
||||
Table of distances that have a different distance symbol in the deflate
|
||||
specification. Each value is the first distance that has a new symbol. Only
|
||||
different symbols affect the cost model so only these need to be checked.
|
||||
See RFC 1951 section 3.2.5. Compressed blocks (length and distance codes).
|
||||
*/
|
||||
static const int dsymbols[30] = {
|
||||
1, 2, 3, 4, 5, 7, 9, 13, 17, 25, 33, 49, 65, 97, 129, 193, 257, 385, 513,
|
||||
769, 1025, 1537, 2049, 3073, 4097, 6145, 8193, 12289, 16385, 24577
|
||||
};
|
||||
|
||||
mincost = ZOPFLI_LARGE_FLOAT;
|
||||
for (i = 3; i < 259; i++) {
|
||||
double c = costmodel(i, 1, costcontext);
|
||||
if (c < mincost) {
|
||||
bestlength = i;
|
||||
mincost = c;
|
||||
}
|
||||
}
|
||||
|
||||
mincost = ZOPFLI_LARGE_FLOAT;
|
||||
for (i = 0; i < 30; i++) {
|
||||
double c = costmodel(3, dsymbols[i], costcontext);
|
||||
if (c < mincost) {
|
||||
bestdist = dsymbols[i];
|
||||
mincost = c;
|
||||
}
|
||||
}
|
||||
|
||||
return costmodel(bestlength, bestdist, costcontext);
|
||||
}
|
||||
|
||||
static size_t zopfli_min(size_t a, size_t b) {
|
||||
return a < b ? a : b;
|
||||
}
|
||||
|
||||
/*
|
||||
Performs the forward pass for "squeeze". Gets the most optimal length to reach
|
||||
every byte from a previous byte, using cost calculations.
|
||||
s: the ZopfliBlockState
|
||||
in: the input data array
|
||||
instart: where to start
|
||||
inend: where to stop (not inclusive)
|
||||
costmodel: function to calculate the cost of some lit/len/dist pair.
|
||||
costcontext: abstract context for the costmodel function
|
||||
length_array: output array of size (inend - instart) which will receive the best
|
||||
length to reach this byte from a previous byte.
|
||||
returns the cost that was, according to the costmodel, needed to get to the end.
|
||||
*/
|
||||
static double GetBestLengths(ZopfliBlockState *s,
|
||||
const unsigned char* in,
|
||||
size_t instart, size_t inend,
|
||||
CostModelFun* costmodel, void* costcontext,
|
||||
unsigned short* length_array,
|
||||
ZopfliHash* h, float* costs) {
|
||||
/* Best cost to get here so far. */
|
||||
size_t blocksize = inend - instart;
|
||||
size_t i = 0, k, kend;
|
||||
unsigned short leng;
|
||||
unsigned short dist;
|
||||
unsigned short sublen[259];
|
||||
size_t windowstart = instart > ZOPFLI_WINDOW_SIZE
|
||||
? instart - ZOPFLI_WINDOW_SIZE : 0;
|
||||
double result;
|
||||
double mincost = GetCostModelMinCost(costmodel, costcontext);
|
||||
double mincostaddcostj;
|
||||
|
||||
if (instart == inend) return 0;
|
||||
|
||||
ZopfliResetHash(ZOPFLI_WINDOW_SIZE, h);
|
||||
ZopfliWarmupHash(in, windowstart, inend, h);
|
||||
for (i = windowstart; i < instart; i++) {
|
||||
ZopfliUpdateHash(in, i, inend, h);
|
||||
}
|
||||
|
||||
for (i = 1; i < blocksize + 1; i++) costs[i] = ZOPFLI_LARGE_FLOAT;
|
||||
costs[0] = 0; /* Because it's the start. */
|
||||
length_array[0] = 0;
|
||||
|
||||
for (i = instart; i < inend; i++) {
|
||||
size_t j = i - instart; /* Index in the costs array and length_array. */
|
||||
ZopfliUpdateHash(in, i, inend, h);
|
||||
|
||||
#ifdef ZOPFLI_SHORTCUT_LONG_REPETITIONS
|
||||
/* If we're in a long repetition of the same character and have more than
|
||||
ZOPFLI_MAX_MATCH characters before and after our position. */
|
||||
if (h->same[i & ZOPFLI_WINDOW_MASK] > ZOPFLI_MAX_MATCH * 2
|
||||
&& i > instart + ZOPFLI_MAX_MATCH + 1
|
||||
&& i + ZOPFLI_MAX_MATCH * 2 + 1 < inend
|
||||
&& h->same[(i - ZOPFLI_MAX_MATCH) & ZOPFLI_WINDOW_MASK]
|
||||
> ZOPFLI_MAX_MATCH) {
|
||||
double symbolcost = costmodel(ZOPFLI_MAX_MATCH, 1, costcontext);
|
||||
/* Set the length to reach each one to ZOPFLI_MAX_MATCH, and the cost to
|
||||
the cost corresponding to that length. Doing this, we skip
|
||||
ZOPFLI_MAX_MATCH values to avoid calling ZopfliFindLongestMatch. */
|
||||
for (k = 0; k < ZOPFLI_MAX_MATCH; k++) {
|
||||
costs[j + ZOPFLI_MAX_MATCH] = costs[j] + symbolcost;
|
||||
length_array[j + ZOPFLI_MAX_MATCH] = ZOPFLI_MAX_MATCH;
|
||||
i++;
|
||||
j++;
|
||||
ZopfliUpdateHash(in, i, inend, h);
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
ZopfliFindLongestMatch(s, h, in, i, inend, ZOPFLI_MAX_MATCH, sublen,
|
||||
&dist, &leng);
|
||||
|
||||
/* Literal. */
|
||||
if (i + 1 <= inend) {
|
||||
double newCost = costmodel(in[i], 0, costcontext) + costs[j];
|
||||
assert(newCost >= 0);
|
||||
if (newCost < costs[j + 1]) {
|
||||
costs[j + 1] = newCost;
|
||||
length_array[j + 1] = 1;
|
||||
}
|
||||
}
|
||||
/* Lengths. */
|
||||
kend = zopfli_min(leng, inend-i);
|
||||
mincostaddcostj = mincost + costs[j];
|
||||
for (k = 3; k <= kend; k++) {
|
||||
double newCost;
|
||||
|
||||
/* Calling the cost model is expensive, avoid this if we are already at
|
||||
the minimum possible cost that it can return. */
|
||||
if (costs[j + k] <= mincostaddcostj) continue;
|
||||
|
||||
newCost = costmodel(k, sublen[k], costcontext) + costs[j];
|
||||
assert(newCost >= 0);
|
||||
if (newCost < costs[j + k]) {
|
||||
assert(k <= ZOPFLI_MAX_MATCH);
|
||||
costs[j + k] = newCost;
|
||||
length_array[j + k] = k;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
assert(costs[blocksize] >= 0);
|
||||
result = costs[blocksize];
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/*
|
||||
Calculates the optimal path of lz77 lengths to use, from the calculated
|
||||
length_array. The length_array must contain the optimal length to reach that
|
||||
byte. The path will be filled with the lengths to use, so its data size will be
|
||||
the amount of lz77 symbols.
|
||||
*/
|
||||
static void TraceBackwards(size_t size, const unsigned short* length_array,
|
||||
unsigned short** path, size_t* pathsize) {
|
||||
size_t index = size;
|
||||
if (size == 0) return;
|
||||
for (;;) {
|
||||
ZOPFLI_APPEND_DATA(length_array[index], path, pathsize);
|
||||
assert(length_array[index] <= index);
|
||||
assert(length_array[index] <= ZOPFLI_MAX_MATCH);
|
||||
assert(length_array[index] != 0);
|
||||
index -= length_array[index];
|
||||
if (index == 0) break;
|
||||
}
|
||||
|
||||
/* Mirror result. */
|
||||
for (index = 0; index < *pathsize / 2; index++) {
|
||||
unsigned short temp = (*path)[index];
|
||||
(*path)[index] = (*path)[*pathsize - index - 1];
|
||||
(*path)[*pathsize - index - 1] = temp;
|
||||
}
|
||||
}
|
||||
|
||||
static void FollowPath(ZopfliBlockState* s,
|
||||
const unsigned char* in, size_t instart, size_t inend,
|
||||
unsigned short* path, size_t pathsize,
|
||||
ZopfliLZ77Store* store, ZopfliHash *h) {
|
||||
size_t i, j, pos = 0;
|
||||
size_t windowstart = instart > ZOPFLI_WINDOW_SIZE
|
||||
? instart - ZOPFLI_WINDOW_SIZE : 0;
|
||||
|
||||
size_t total_length_test = 0;
|
||||
|
||||
if (instart == inend) return;
|
||||
|
||||
ZopfliResetHash(ZOPFLI_WINDOW_SIZE, h);
|
||||
ZopfliWarmupHash(in, windowstart, inend, h);
|
||||
for (i = windowstart; i < instart; i++) {
|
||||
ZopfliUpdateHash(in, i, inend, h);
|
||||
}
|
||||
|
||||
pos = instart;
|
||||
for (i = 0; i < pathsize; i++) {
|
||||
unsigned short length = path[i];
|
||||
unsigned short dummy_length;
|
||||
unsigned short dist;
|
||||
assert(pos < inend);
|
||||
|
||||
ZopfliUpdateHash(in, pos, inend, h);
|
||||
|
||||
/* Add to output. */
|
||||
if (length >= ZOPFLI_MIN_MATCH) {
|
||||
/* Get the distance by recalculating longest match. The found length
|
||||
should match the length from the path. */
|
||||
ZopfliFindLongestMatch(s, h, in, pos, inend, length, 0,
|
||||
&dist, &dummy_length);
|
||||
assert(!(dummy_length != length && length > 2 && dummy_length > 2));
|
||||
ZopfliVerifyLenDist(in, inend, pos, dist, length);
|
||||
ZopfliStoreLitLenDist(length, dist, pos, store);
|
||||
total_length_test += length;
|
||||
} else {
|
||||
length = 1;
|
||||
ZopfliStoreLitLenDist(in[pos], 0, pos, store);
|
||||
total_length_test++;
|
||||
}
|
||||
|
||||
|
||||
assert(pos + length <= inend);
|
||||
for (j = 1; j < length; j++) {
|
||||
ZopfliUpdateHash(in, pos + j, inend, h);
|
||||
}
|
||||
|
||||
pos += length;
|
||||
}
|
||||
}
|
||||
|
||||
/* Calculates the entropy of the statistics */
|
||||
static void CalculateStatistics(SymbolStats* stats) {
|
||||
ZopfliCalculateEntropy(stats->litlens, ZOPFLI_NUM_LL, stats->ll_symbols);
|
||||
ZopfliCalculateEntropy(stats->dists, ZOPFLI_NUM_D, stats->d_symbols);
|
||||
}
|
||||
|
||||
/* Appends the symbol statistics from the store. */
|
||||
static void GetStatistics(const ZopfliLZ77Store* store, SymbolStats* stats) {
|
||||
size_t i;
|
||||
for (i = 0; i < store->size; i++) {
|
||||
if (store->dists[i] == 0) {
|
||||
stats->litlens[store->litlens[i]]++;
|
||||
} else {
|
||||
stats->litlens[ZopfliGetLengthSymbol(store->litlens[i])]++;
|
||||
stats->dists[ZopfliGetDistSymbol(store->dists[i])]++;
|
||||
}
|
||||
}
|
||||
stats->litlens[256] = 1; /* End symbol. */
|
||||
|
||||
CalculateStatistics(stats);
|
||||
}
|
||||
|
||||
/*
|
||||
Does a single run for ZopfliLZ77Optimal. For good compression, repeated runs
|
||||
with updated statistics should be performed.
|
||||
s: the block state
|
||||
in: the input data array
|
||||
instart: where to start
|
||||
inend: where to stop (not inclusive)
|
||||
path: pointer to dynamically allocated memory to store the path
|
||||
pathsize: pointer to the size of the dynamic path array
|
||||
length_array: array of size (inend - instart) used to store lengths
|
||||
costmodel: function to use as the cost model for this squeeze run
|
||||
costcontext: abstract context for the costmodel function
|
||||
store: place to output the LZ77 data
|
||||
returns the cost that was, according to the costmodel, needed to get to the end.
|
||||
This is not the actual cost.
|
||||
*/
|
||||
static double LZ77OptimalRun(ZopfliBlockState* s,
|
||||
const unsigned char* in, size_t instart, size_t inend,
|
||||
unsigned short** path, size_t* pathsize,
|
||||
unsigned short* length_array, CostModelFun* costmodel,
|
||||
void* costcontext, ZopfliLZ77Store* store,
|
||||
ZopfliHash* h, float* costs) {
|
||||
double cost = GetBestLengths(s, in, instart, inend, costmodel,
|
||||
costcontext, length_array, h, costs);
|
||||
free(*path);
|
||||
*path = 0;
|
||||
*pathsize = 0;
|
||||
TraceBackwards(inend - instart, length_array, path, pathsize);
|
||||
FollowPath(s, in, instart, inend, *path, *pathsize, store, h);
|
||||
assert(cost < ZOPFLI_LARGE_FLOAT);
|
||||
return cost;
|
||||
}
|
||||
|
||||
void ZopfliLZ77Optimal(ZopfliBlockState *s,
|
||||
const unsigned char* in, size_t instart, size_t inend,
|
||||
int numiterations,
|
||||
ZopfliLZ77Store* store) {
|
||||
/* Dist to get to here with smallest cost. */
|
||||
size_t blocksize = inend - instart;
|
||||
unsigned short* length_array =
|
||||
(unsigned short*)malloc(sizeof(unsigned short) * (blocksize + 1));
|
||||
unsigned short* path = 0;
|
||||
size_t pathsize = 0;
|
||||
ZopfliLZ77Store currentstore;
|
||||
ZopfliHash hash;
|
||||
ZopfliHash* h = &hash;
|
||||
SymbolStats stats, beststats, laststats;
|
||||
int i;
|
||||
float* costs = (float*)malloc(sizeof(float) * (blocksize + 1));
|
||||
double cost;
|
||||
double bestcost = ZOPFLI_LARGE_FLOAT;
|
||||
double lastcost = 0;
|
||||
/* Try randomizing the costs a bit once the size stabilizes. */
|
||||
RanState ran_state;
|
||||
int lastrandomstep = -1;
|
||||
|
||||
if (!costs) exit(-1); /* Allocation failed. */
|
||||
if (!length_array) exit(-1); /* Allocation failed. */
|
||||
|
||||
InitRanState(&ran_state);
|
||||
InitStats(&stats);
|
||||
ZopfliInitLZ77Store(in, ¤tstore);
|
||||
ZopfliAllocHash(ZOPFLI_WINDOW_SIZE, h);
|
||||
|
||||
/* Do regular deflate, then loop multiple shortest path runs, each time using
|
||||
the statistics of the previous run. */
|
||||
|
||||
/* Initial run. */
|
||||
ZopfliLZ77Greedy(s, in, instart, inend, ¤tstore, h);
|
||||
GetStatistics(¤tstore, &stats);
|
||||
|
||||
/* Repeat statistics with each time the cost model from the previous stat
|
||||
run. */
|
||||
for (i = 0; i < numiterations; i++) {
|
||||
ZopfliCleanLZ77Store(¤tstore);
|
||||
ZopfliInitLZ77Store(in, ¤tstore);
|
||||
LZ77OptimalRun(s, in, instart, inend, &path, &pathsize,
|
||||
length_array, GetCostStat, (void*)&stats,
|
||||
¤tstore, h, costs);
|
||||
cost = ZopfliCalculateBlockSize(¤tstore, 0, currentstore.size, 2);
|
||||
if (s->options->verbose_more || (s->options->verbose && cost < bestcost)) {
|
||||
fprintf(stderr, "Iteration %d: %d bit\n", i, (int) cost);
|
||||
}
|
||||
if (cost < bestcost) {
|
||||
/* Copy to the output store. */
|
||||
ZopfliCopyLZ77Store(¤tstore, store);
|
||||
CopyStats(&stats, &beststats);
|
||||
bestcost = cost;
|
||||
}
|
||||
CopyStats(&stats, &laststats);
|
||||
ClearStatFreqs(&stats);
|
||||
GetStatistics(¤tstore, &stats);
|
||||
if (lastrandomstep != -1) {
|
||||
/* This makes it converge slower but better. Do it only once the
|
||||
randomness kicks in so that if the user does few iterations, it gives a
|
||||
better result sooner. */
|
||||
AddWeighedStatFreqs(&stats, 1.0, &laststats, 0.5, &stats);
|
||||
CalculateStatistics(&stats);
|
||||
}
|
||||
if (i > 5 && cost == lastcost) {
|
||||
CopyStats(&beststats, &stats);
|
||||
RandomizeStatFreqs(&ran_state, &stats);
|
||||
CalculateStatistics(&stats);
|
||||
lastrandomstep = i;
|
||||
}
|
||||
lastcost = cost;
|
||||
}
|
||||
|
||||
free(length_array);
|
||||
free(path);
|
||||
free(costs);
|
||||
ZopfliCleanLZ77Store(¤tstore);
|
||||
ZopfliCleanHash(h);
|
||||
}
|
||||
|
||||
void ZopfliLZ77OptimalFixed(ZopfliBlockState *s,
|
||||
const unsigned char* in,
|
||||
size_t instart, size_t inend,
|
||||
ZopfliLZ77Store* store)
|
||||
{
|
||||
/* Dist to get to here with smallest cost. */
|
||||
size_t blocksize = inend - instart;
|
||||
unsigned short* length_array =
|
||||
(unsigned short*)malloc(sizeof(unsigned short) * (blocksize + 1));
|
||||
unsigned short* path = 0;
|
||||
size_t pathsize = 0;
|
||||
ZopfliHash hash;
|
||||
ZopfliHash* h = &hash;
|
||||
float* costs = (float*)malloc(sizeof(float) * (blocksize + 1));
|
||||
|
||||
if (!costs) exit(-1); /* Allocation failed. */
|
||||
if (!length_array) exit(-1); /* Allocation failed. */
|
||||
|
||||
ZopfliAllocHash(ZOPFLI_WINDOW_SIZE, h);
|
||||
|
||||
s->blockstart = instart;
|
||||
s->blockend = inend;
|
||||
|
||||
/* Shortest path for fixed tree This one should give the shortest possible
|
||||
result for fixed tree, no repeated runs are needed since the tree is known. */
|
||||
LZ77OptimalRun(s, in, instart, inend, &path, &pathsize,
|
||||
length_array, GetCostFixed, 0, store, h, costs);
|
||||
|
||||
free(length_array);
|
||||
free(path);
|
||||
free(costs);
|
||||
ZopfliCleanHash(h);
|
||||
}
|
||||
61
build/node_modules/node-zopfli/zopfli/src/zopfli/squeeze.h
generated
vendored
Normal file
61
build/node_modules/node-zopfli/zopfli/src/zopfli/squeeze.h
generated
vendored
Normal file
@@ -0,0 +1,61 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
/*
|
||||
The squeeze functions do enhanced LZ77 compression by optimal parsing with a
|
||||
cost model, rather than greedily choosing the longest length or using a single
|
||||
step of lazy matching like regular implementations.
|
||||
|
||||
Since the cost model is based on the Huffman tree that can only be calculated
|
||||
after the LZ77 data is generated, there is a chicken and egg problem, and
|
||||
multiple runs are done with updated cost models to converge to a better
|
||||
solution.
|
||||
*/
|
||||
|
||||
#ifndef ZOPFLI_SQUEEZE_H_
|
||||
#define ZOPFLI_SQUEEZE_H_
|
||||
|
||||
#include "lz77.h"
|
||||
|
||||
/*
|
||||
Calculates lit/len and dist pairs for given data.
|
||||
If instart is larger than 0, it uses values before instart as starting
|
||||
dictionary.
|
||||
*/
|
||||
void ZopfliLZ77Optimal(ZopfliBlockState *s,
|
||||
const unsigned char* in, size_t instart, size_t inend,
|
||||
int numiterations,
|
||||
ZopfliLZ77Store* store);
|
||||
|
||||
/*
|
||||
Does the same as ZopfliLZ77Optimal, but optimized for the fixed tree of the
|
||||
deflate standard.
|
||||
The fixed tree never gives the best compression. But this gives the best
|
||||
possible LZ77 encoding possible with the fixed tree.
|
||||
This does not create or output any fixed tree, only LZ77 data optimized for
|
||||
using with a fixed tree.
|
||||
If instart is larger than 0, it uses values before instart as starting
|
||||
dictionary.
|
||||
*/
|
||||
void ZopfliLZ77OptimalFixed(ZopfliBlockState *s,
|
||||
const unsigned char* in,
|
||||
size_t instart, size_t inend,
|
||||
ZopfliLZ77Store* store);
|
||||
|
||||
#endif /* ZOPFLI_SQUEEZE_H_ */
|
||||
239
build/node_modules/node-zopfli/zopfli/src/zopfli/symbols.h
generated
vendored
Normal file
239
build/node_modules/node-zopfli/zopfli/src/zopfli/symbols.h
generated
vendored
Normal file
@@ -0,0 +1,239 @@
|
||||
/*
|
||||
Copyright 2016 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
/*
|
||||
Utilities for using the lz77 symbols of the deflate spec.
|
||||
*/
|
||||
|
||||
#ifndef ZOPFLI_SYMBOLS_H_
|
||||
#define ZOPFLI_SYMBOLS_H_
|
||||
|
||||
/* __has_builtin available in clang */
|
||||
#ifdef __has_builtin
|
||||
# if __has_builtin(__builtin_clz)
|
||||
# define ZOPFLI_HAS_BUILTIN_CLZ
|
||||
# endif
|
||||
/* __builtin_clz available beginning with GCC 3.4 */
|
||||
#elif __GNUC__ * 100 + __GNUC_MINOR__ >= 304
|
||||
# define ZOPFLI_HAS_BUILTIN_CLZ
|
||||
#endif
|
||||
|
||||
/* Gets the amount of extra bits for the given dist, cfr. the DEFLATE spec. */
|
||||
static int ZopfliGetDistExtraBits(int dist) {
|
||||
#ifdef ZOPFLI_HAS_BUILTIN_CLZ
|
||||
if (dist < 5) return 0;
|
||||
return (31 ^ __builtin_clz(dist - 1)) - 1; /* log2(dist - 1) - 1 */
|
||||
#else
|
||||
if (dist < 5) return 0;
|
||||
else if (dist < 9) return 1;
|
||||
else if (dist < 17) return 2;
|
||||
else if (dist < 33) return 3;
|
||||
else if (dist < 65) return 4;
|
||||
else if (dist < 129) return 5;
|
||||
else if (dist < 257) return 6;
|
||||
else if (dist < 513) return 7;
|
||||
else if (dist < 1025) return 8;
|
||||
else if (dist < 2049) return 9;
|
||||
else if (dist < 4097) return 10;
|
||||
else if (dist < 8193) return 11;
|
||||
else if (dist < 16385) return 12;
|
||||
else return 13;
|
||||
#endif
|
||||
}
|
||||
|
||||
/* Gets value of the extra bits for the given dist, cfr. the DEFLATE spec. */
|
||||
static int ZopfliGetDistExtraBitsValue(int dist) {
|
||||
#ifdef ZOPFLI_HAS_BUILTIN_CLZ
|
||||
if (dist < 5) {
|
||||
return 0;
|
||||
} else {
|
||||
int l = 31 ^ __builtin_clz(dist - 1); /* log2(dist - 1) */
|
||||
return (dist - (1 + (1 << l))) & ((1 << (l - 1)) - 1);
|
||||
}
|
||||
#else
|
||||
if (dist < 5) return 0;
|
||||
else if (dist < 9) return (dist - 5) & 1;
|
||||
else if (dist < 17) return (dist - 9) & 3;
|
||||
else if (dist < 33) return (dist - 17) & 7;
|
||||
else if (dist < 65) return (dist - 33) & 15;
|
||||
else if (dist < 129) return (dist - 65) & 31;
|
||||
else if (dist < 257) return (dist - 129) & 63;
|
||||
else if (dist < 513) return (dist - 257) & 127;
|
||||
else if (dist < 1025) return (dist - 513) & 255;
|
||||
else if (dist < 2049) return (dist - 1025) & 511;
|
||||
else if (dist < 4097) return (dist - 2049) & 1023;
|
||||
else if (dist < 8193) return (dist - 4097) & 2047;
|
||||
else if (dist < 16385) return (dist - 8193) & 4095;
|
||||
else return (dist - 16385) & 8191;
|
||||
#endif
|
||||
}
|
||||
|
||||
/* Gets the symbol for the given dist, cfr. the DEFLATE spec. */
|
||||
static int ZopfliGetDistSymbol(int dist) {
|
||||
#ifdef ZOPFLI_HAS_BUILTIN_CLZ
|
||||
if (dist < 5) {
|
||||
return dist - 1;
|
||||
} else {
|
||||
int l = (31 ^ __builtin_clz(dist - 1)); /* log2(dist - 1) */
|
||||
int r = ((dist - 1) >> (l - 1)) & 1;
|
||||
return l * 2 + r;
|
||||
}
|
||||
#else
|
||||
if (dist < 193) {
|
||||
if (dist < 13) { /* dist 0..13. */
|
||||
if (dist < 5) return dist - 1;
|
||||
else if (dist < 7) return 4;
|
||||
else if (dist < 9) return 5;
|
||||
else return 6;
|
||||
} else { /* dist 13..193. */
|
||||
if (dist < 17) return 7;
|
||||
else if (dist < 25) return 8;
|
||||
else if (dist < 33) return 9;
|
||||
else if (dist < 49) return 10;
|
||||
else if (dist < 65) return 11;
|
||||
else if (dist < 97) return 12;
|
||||
else if (dist < 129) return 13;
|
||||
else return 14;
|
||||
}
|
||||
} else {
|
||||
if (dist < 2049) { /* dist 193..2049. */
|
||||
if (dist < 257) return 15;
|
||||
else if (dist < 385) return 16;
|
||||
else if (dist < 513) return 17;
|
||||
else if (dist < 769) return 18;
|
||||
else if (dist < 1025) return 19;
|
||||
else if (dist < 1537) return 20;
|
||||
else return 21;
|
||||
} else { /* dist 2049..32768. */
|
||||
if (dist < 3073) return 22;
|
||||
else if (dist < 4097) return 23;
|
||||
else if (dist < 6145) return 24;
|
||||
else if (dist < 8193) return 25;
|
||||
else if (dist < 12289) return 26;
|
||||
else if (dist < 16385) return 27;
|
||||
else if (dist < 24577) return 28;
|
||||
else return 29;
|
||||
}
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
/* Gets the amount of extra bits for the given length, cfr. the DEFLATE spec. */
|
||||
static int ZopfliGetLengthExtraBits(int l) {
|
||||
static const int table[259] = {
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1,
|
||||
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
|
||||
3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
|
||||
3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
|
||||
4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
|
||||
4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
|
||||
4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
|
||||
4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
|
||||
5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
|
||||
5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
|
||||
5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
|
||||
5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
|
||||
5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
|
||||
5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
|
||||
5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
|
||||
5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 0
|
||||
};
|
||||
return table[l];
|
||||
}
|
||||
|
||||
/* Gets value of the extra bits for the given length, cfr. the DEFLATE spec. */
|
||||
static int ZopfliGetLengthExtraBitsValue(int l) {
|
||||
static const int table[259] = {
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 2, 3, 0,
|
||||
1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 4, 5, 6, 7, 0, 1, 2, 3, 4, 5,
|
||||
6, 7, 0, 1, 2, 3, 4, 5, 6, 7, 0, 1, 2, 3, 4, 5, 6, 7, 0, 1, 2, 3, 4, 5, 6,
|
||||
7, 8, 9, 10, 11, 12, 13, 14, 15, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,
|
||||
13, 14, 15, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 0, 1, 2,
|
||||
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
|
||||
10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,
|
||||
29, 30, 31, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
|
||||
18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 0, 1, 2, 3, 4, 5, 6,
|
||||
7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26,
|
||||
27, 28, 29, 30, 31, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
|
||||
16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 0
|
||||
};
|
||||
return table[l];
|
||||
}
|
||||
|
||||
/*
|
||||
Gets the symbol for the given length, cfr. the DEFLATE spec.
|
||||
Returns the symbol in the range [257-285] (inclusive)
|
||||
*/
|
||||
static int ZopfliGetLengthSymbol(int l) {
|
||||
static const int table[259] = {
|
||||
0, 0, 0, 257, 258, 259, 260, 261, 262, 263, 264,
|
||||
265, 265, 266, 266, 267, 267, 268, 268,
|
||||
269, 269, 269, 269, 270, 270, 270, 270,
|
||||
271, 271, 271, 271, 272, 272, 272, 272,
|
||||
273, 273, 273, 273, 273, 273, 273, 273,
|
||||
274, 274, 274, 274, 274, 274, 274, 274,
|
||||
275, 275, 275, 275, 275, 275, 275, 275,
|
||||
276, 276, 276, 276, 276, 276, 276, 276,
|
||||
277, 277, 277, 277, 277, 277, 277, 277,
|
||||
277, 277, 277, 277, 277, 277, 277, 277,
|
||||
278, 278, 278, 278, 278, 278, 278, 278,
|
||||
278, 278, 278, 278, 278, 278, 278, 278,
|
||||
279, 279, 279, 279, 279, 279, 279, 279,
|
||||
279, 279, 279, 279, 279, 279, 279, 279,
|
||||
280, 280, 280, 280, 280, 280, 280, 280,
|
||||
280, 280, 280, 280, 280, 280, 280, 280,
|
||||
281, 281, 281, 281, 281, 281, 281, 281,
|
||||
281, 281, 281, 281, 281, 281, 281, 281,
|
||||
281, 281, 281, 281, 281, 281, 281, 281,
|
||||
281, 281, 281, 281, 281, 281, 281, 281,
|
||||
282, 282, 282, 282, 282, 282, 282, 282,
|
||||
282, 282, 282, 282, 282, 282, 282, 282,
|
||||
282, 282, 282, 282, 282, 282, 282, 282,
|
||||
282, 282, 282, 282, 282, 282, 282, 282,
|
||||
283, 283, 283, 283, 283, 283, 283, 283,
|
||||
283, 283, 283, 283, 283, 283, 283, 283,
|
||||
283, 283, 283, 283, 283, 283, 283, 283,
|
||||
283, 283, 283, 283, 283, 283, 283, 283,
|
||||
284, 284, 284, 284, 284, 284, 284, 284,
|
||||
284, 284, 284, 284, 284, 284, 284, 284,
|
||||
284, 284, 284, 284, 284, 284, 284, 284,
|
||||
284, 284, 284, 284, 284, 284, 284, 285
|
||||
};
|
||||
return table[l];
|
||||
}
|
||||
|
||||
/* Gets the amount of extra bits for the given length symbol. */
|
||||
static int ZopfliGetLengthSymbolExtraBits(int s) {
|
||||
static const int table[29] = {
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2,
|
||||
3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 0
|
||||
};
|
||||
return table[s - 257];
|
||||
}
|
||||
|
||||
/* Gets the amount of extra bits for the given distance symbol. */
|
||||
static int ZopfliGetDistSymbolExtraBits(int s) {
|
||||
static const int table[30] = {
|
||||
0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8,
|
||||
9, 9, 10, 10, 11, 11, 12, 12, 13, 13
|
||||
};
|
||||
return table[s];
|
||||
}
|
||||
|
||||
#endif /* ZOPFLI_SYMBOLS_H_ */
|
||||
101
build/node_modules/node-zopfli/zopfli/src/zopfli/tree.c
generated
vendored
Normal file
101
build/node_modules/node-zopfli/zopfli/src/zopfli/tree.c
generated
vendored
Normal file
@@ -0,0 +1,101 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
#include "tree.h"
|
||||
|
||||
#include <assert.h>
|
||||
#include <math.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
|
||||
#include "katajainen.h"
|
||||
#include "util.h"
|
||||
|
||||
void ZopfliLengthsToSymbols(const unsigned* lengths, size_t n, unsigned maxbits,
|
||||
unsigned* symbols) {
|
||||
size_t* bl_count = (size_t*)malloc(sizeof(size_t) * (maxbits + 1));
|
||||
size_t* next_code = (size_t*)malloc(sizeof(size_t) * (maxbits + 1));
|
||||
unsigned bits, i;
|
||||
unsigned code;
|
||||
|
||||
for (i = 0; i < n; i++) {
|
||||
symbols[i] = 0;
|
||||
}
|
||||
|
||||
/* 1) Count the number of codes for each code length. Let bl_count[N] be the
|
||||
number of codes of length N, N >= 1. */
|
||||
for (bits = 0; bits <= maxbits; bits++) {
|
||||
bl_count[bits] = 0;
|
||||
}
|
||||
for (i = 0; i < n; i++) {
|
||||
assert(lengths[i] <= maxbits);
|
||||
bl_count[lengths[i]]++;
|
||||
}
|
||||
/* 2) Find the numerical value of the smallest code for each code length. */
|
||||
code = 0;
|
||||
bl_count[0] = 0;
|
||||
for (bits = 1; bits <= maxbits; bits++) {
|
||||
code = (code + bl_count[bits-1]) << 1;
|
||||
next_code[bits] = code;
|
||||
}
|
||||
/* 3) Assign numerical values to all codes, using consecutive values for all
|
||||
codes of the same length with the base values determined at step 2. */
|
||||
for (i = 0; i < n; i++) {
|
||||
unsigned len = lengths[i];
|
||||
if (len != 0) {
|
||||
symbols[i] = next_code[len];
|
||||
next_code[len]++;
|
||||
}
|
||||
}
|
||||
|
||||
free(bl_count);
|
||||
free(next_code);
|
||||
}
|
||||
|
||||
void ZopfliCalculateEntropy(const size_t* count, size_t n, double* bitlengths) {
|
||||
static const double kInvLog2 = 1.4426950408889; /* 1.0 / log(2.0) */
|
||||
unsigned sum = 0;
|
||||
unsigned i;
|
||||
double log2sum;
|
||||
for (i = 0; i < n; ++i) {
|
||||
sum += count[i];
|
||||
}
|
||||
log2sum = (sum == 0 ? log(n) : log(sum)) * kInvLog2;
|
||||
for (i = 0; i < n; ++i) {
|
||||
/* When the count of the symbol is 0, but its cost is requested anyway, it
|
||||
means the symbol will appear at least once anyway, so give it the cost as if
|
||||
its count is 1.*/
|
||||
if (count[i] == 0) bitlengths[i] = log2sum;
|
||||
else bitlengths[i] = log2sum - log(count[i]) * kInvLog2;
|
||||
/* Depending on compiler and architecture, the above subtraction of two
|
||||
floating point numbers may give a negative result very close to zero
|
||||
instead of zero (e.g. -5.973954e-17 with gcc 4.1.2 on Ubuntu 11.4). Clamp
|
||||
it to zero. These floating point imprecisions do not affect the cost model
|
||||
significantly so this is ok. */
|
||||
if (bitlengths[i] < 0 && bitlengths[i] > -1e-5) bitlengths[i] = 0;
|
||||
assert(bitlengths[i] >= 0);
|
||||
}
|
||||
}
|
||||
|
||||
void ZopfliCalculateBitLengths(const size_t* count, size_t n, int maxbits,
|
||||
unsigned* bitlengths) {
|
||||
int error = ZopfliLengthLimitedCodeLengths(count, n, maxbits, bitlengths);
|
||||
(void) error;
|
||||
assert(!error);
|
||||
}
|
||||
51
build/node_modules/node-zopfli/zopfli/src/zopfli/tree.h
generated
vendored
Normal file
51
build/node_modules/node-zopfli/zopfli/src/zopfli/tree.h
generated
vendored
Normal file
@@ -0,0 +1,51 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
/*
|
||||
Utilities for creating and using Huffman trees.
|
||||
*/
|
||||
|
||||
#ifndef ZOPFLI_TREE_H_
|
||||
#define ZOPFLI_TREE_H_
|
||||
|
||||
#include <string.h>
|
||||
|
||||
/*
|
||||
Calculates the bitlengths for the Huffman tree, based on the counts of each
|
||||
symbol.
|
||||
*/
|
||||
void ZopfliCalculateBitLengths(const size_t* count, size_t n, int maxbits,
|
||||
unsigned *bitlengths);
|
||||
|
||||
/*
|
||||
Converts a series of Huffman tree bitlengths, to the bit values of the symbols.
|
||||
*/
|
||||
void ZopfliLengthsToSymbols(const unsigned* lengths, size_t n, unsigned maxbits,
|
||||
unsigned* symbols);
|
||||
|
||||
/*
|
||||
Calculates the entropy of each symbol, based on the counts of each symbol. The
|
||||
result is similar to the result of ZopfliCalculateBitLengths, but with the
|
||||
actual theoritical bit lengths according to the entropy. Since the resulting
|
||||
values are fractional, they cannot be used to encode the tree specified by
|
||||
DEFLATE.
|
||||
*/
|
||||
void ZopfliCalculateEntropy(const size_t* count, size_t n, double* bitlengths);
|
||||
|
||||
#endif /* ZOPFLI_TREE_H_ */
|
||||
35
build/node_modules/node-zopfli/zopfli/src/zopfli/util.c
generated
vendored
Normal file
35
build/node_modules/node-zopfli/zopfli/src/zopfli/util.c
generated
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
#include "util.h"
|
||||
|
||||
#include "zopfli.h"
|
||||
|
||||
#include <assert.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
|
||||
void ZopfliInitOptions(ZopfliOptions* options) {
|
||||
options->verbose = 0;
|
||||
options->verbose_more = 0;
|
||||
options->numiterations = 15;
|
||||
options->blocksplitting = 1;
|
||||
options->blocksplittinglast = 0;
|
||||
options->blocksplittingmax = 15;
|
||||
}
|
||||
158
build/node_modules/node-zopfli/zopfli/src/zopfli/util.h
generated
vendored
Normal file
158
build/node_modules/node-zopfli/zopfli/src/zopfli/util.h
generated
vendored
Normal file
@@ -0,0 +1,158 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
/*
|
||||
Several utilities, including: #defines to try different compression results,
|
||||
basic deflate specification values and generic program options.
|
||||
*/
|
||||
|
||||
#ifndef ZOPFLI_UTIL_H_
|
||||
#define ZOPFLI_UTIL_H_
|
||||
|
||||
#include <string.h>
|
||||
#include <stdlib.h>
|
||||
|
||||
/* Minimum and maximum length that can be encoded in deflate. */
|
||||
#define ZOPFLI_MAX_MATCH 258
|
||||
#define ZOPFLI_MIN_MATCH 3
|
||||
|
||||
/* Number of distinct literal/length and distance symbols in DEFLATE */
|
||||
#define ZOPFLI_NUM_LL 288
|
||||
#define ZOPFLI_NUM_D 32
|
||||
|
||||
/*
|
||||
The window size for deflate. Must be a power of two. This should be 32768, the
|
||||
maximum possible by the deflate spec. Anything less hurts compression more than
|
||||
speed.
|
||||
*/
|
||||
#define ZOPFLI_WINDOW_SIZE 32768
|
||||
|
||||
/*
|
||||
The window mask used to wrap indices into the window. This is why the
|
||||
window size must be a power of two.
|
||||
*/
|
||||
#define ZOPFLI_WINDOW_MASK (ZOPFLI_WINDOW_SIZE - 1)
|
||||
|
||||
/*
|
||||
A block structure of huge, non-smart, blocks to divide the input into, to allow
|
||||
operating on huge files without exceeding memory, such as the 1GB wiki9 corpus.
|
||||
The whole compression algorithm, including the smarter block splitting, will
|
||||
be executed independently on each huge block.
|
||||
Dividing into huge blocks hurts compression, but not much relative to the size.
|
||||
Set it to 0 to disable master blocks.
|
||||
*/
|
||||
#define ZOPFLI_MASTER_BLOCK_SIZE 1000000
|
||||
|
||||
/*
|
||||
Used to initialize costs for example
|
||||
*/
|
||||
#define ZOPFLI_LARGE_FLOAT 1e30
|
||||
|
||||
/*
|
||||
For longest match cache. max 256. Uses huge amounts of memory but makes it
|
||||
faster. Uses this many times three bytes per single byte of the input data.
|
||||
This is so because longest match finding has to find the exact distance
|
||||
that belongs to each length for the best lz77 strategy.
|
||||
Good values: e.g. 5, 8.
|
||||
*/
|
||||
#define ZOPFLI_CACHE_LENGTH 8
|
||||
|
||||
/*
|
||||
limit the max hash chain hits for this hash value. This has an effect only
|
||||
on files where the hash value is the same very often. On these files, this
|
||||
gives worse compression (the value should ideally be 32768, which is the
|
||||
ZOPFLI_WINDOW_SIZE, while zlib uses 4096 even for best level), but makes it
|
||||
faster on some specific files.
|
||||
Good value: e.g. 8192.
|
||||
*/
|
||||
#define ZOPFLI_MAX_CHAIN_HITS 8192
|
||||
|
||||
/*
|
||||
Whether to use the longest match cache for ZopfliFindLongestMatch. This cache
|
||||
consumes a lot of memory but speeds it up. No effect on compression size.
|
||||
*/
|
||||
#define ZOPFLI_LONGEST_MATCH_CACHE
|
||||
|
||||
/*
|
||||
Enable to remember amount of successive identical bytes in the hash chain for
|
||||
finding longest match
|
||||
required for ZOPFLI_HASH_SAME_HASH and ZOPFLI_SHORTCUT_LONG_REPETITIONS
|
||||
This has no effect on the compression result, and enabling it increases speed.
|
||||
*/
|
||||
#define ZOPFLI_HASH_SAME
|
||||
|
||||
/*
|
||||
Switch to a faster hash based on the info from ZOPFLI_HASH_SAME once the
|
||||
best length so far is long enough. This is way faster for files with lots of
|
||||
identical bytes, on which the compressor is otherwise too slow. Regular files
|
||||
are unaffected or maybe a tiny bit slower.
|
||||
This has no effect on the compression result, only on speed.
|
||||
*/
|
||||
#define ZOPFLI_HASH_SAME_HASH
|
||||
|
||||
/*
|
||||
Enable this, to avoid slowness for files which are a repetition of the same
|
||||
character more than a multiple of ZOPFLI_MAX_MATCH times. This should not affect
|
||||
the compression result.
|
||||
*/
|
||||
#define ZOPFLI_SHORTCUT_LONG_REPETITIONS
|
||||
|
||||
/*
|
||||
Whether to use lazy matching in the greedy LZ77 implementation. This gives a
|
||||
better result of ZopfliLZ77Greedy, but the effect this has on the optimal LZ77
|
||||
varies from file to file.
|
||||
*/
|
||||
#define ZOPFLI_LAZY_MATCHING
|
||||
|
||||
/*
|
||||
Appends value to dynamically allocated memory, doubling its allocation size
|
||||
whenever needed.
|
||||
|
||||
value: the value to append, type T
|
||||
data: pointer to the dynamic array to append to, type T**
|
||||
size: pointer to the size of the array to append to, type size_t*. This is the
|
||||
size that you consider the array to be, not the internal allocation size.
|
||||
Precondition: allocated size of data is at least a power of two greater than or
|
||||
equal than *size.
|
||||
*/
|
||||
#ifdef __cplusplus /* C++ cannot assign void* from malloc to *data */
|
||||
#define ZOPFLI_APPEND_DATA(/* T */ value, /* T** */ data, /* size_t* */ size) {\
|
||||
if (!((*size) & ((*size) - 1))) {\
|
||||
/*double alloc size if it's a power of two*/\
|
||||
void** data_void = reinterpret_cast<void**>(data);\
|
||||
*data_void = (*size) == 0 ? malloc(sizeof(**data))\
|
||||
: realloc((*data), (*size) * 2 * sizeof(**data));\
|
||||
}\
|
||||
(*data)[(*size)] = (value);\
|
||||
(*size)++;\
|
||||
}
|
||||
#else /* C gives problems with strict-aliasing rules for (void**) cast */
|
||||
#define ZOPFLI_APPEND_DATA(/* T */ value, /* T** */ data, /* size_t* */ size) {\
|
||||
if (!((*size) & ((*size) - 1))) {\
|
||||
/*double alloc size if it's a power of two*/\
|
||||
(*data) = (*size) == 0 ? malloc(sizeof(**data))\
|
||||
: realloc((*data), (*size) * 2 * sizeof(**data));\
|
||||
}\
|
||||
(*data)[(*size)] = (value);\
|
||||
(*size)++;\
|
||||
}
|
||||
#endif
|
||||
|
||||
|
||||
#endif /* ZOPFLI_UTIL_H_ */
|
||||
79
build/node_modules/node-zopfli/zopfli/src/zopfli/zlib_container.c
generated
vendored
Normal file
79
build/node_modules/node-zopfli/zopfli/src/zopfli/zlib_container.c
generated
vendored
Normal file
@@ -0,0 +1,79 @@
|
||||
/*
|
||||
Copyright 2013 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
#include "zlib_container.h"
|
||||
#include "util.h"
|
||||
|
||||
#include <stdio.h>
|
||||
|
||||
#include "deflate.h"
|
||||
|
||||
|
||||
/* Calculates the adler32 checksum of the data */
|
||||
static unsigned adler32(const unsigned char* data, size_t size)
|
||||
{
|
||||
static const unsigned sums_overflow = 5550;
|
||||
unsigned s1 = 1;
|
||||
unsigned s2 = 1 >> 16;
|
||||
|
||||
while (size > 0) {
|
||||
size_t amount = size > sums_overflow ? sums_overflow : size;
|
||||
size -= amount;
|
||||
while (amount > 0) {
|
||||
s1 += (*data++);
|
||||
s2 += s1;
|
||||
amount--;
|
||||
}
|
||||
s1 %= 65521;
|
||||
s2 %= 65521;
|
||||
}
|
||||
|
||||
return (s2 << 16) | s1;
|
||||
}
|
||||
|
||||
void ZopfliZlibCompress(const ZopfliOptions* options,
|
||||
const unsigned char* in, size_t insize,
|
||||
unsigned char** out, size_t* outsize) {
|
||||
unsigned char bitpointer = 0;
|
||||
unsigned checksum = adler32(in, (unsigned)insize);
|
||||
unsigned cmf = 120; /* CM 8, CINFO 7. See zlib spec.*/
|
||||
unsigned flevel = 3;
|
||||
unsigned fdict = 0;
|
||||
unsigned cmfflg = 256 * cmf + fdict * 32 + flevel * 64;
|
||||
unsigned fcheck = 31 - cmfflg % 31;
|
||||
cmfflg += fcheck;
|
||||
|
||||
ZOPFLI_APPEND_DATA(cmfflg / 256, out, outsize);
|
||||
ZOPFLI_APPEND_DATA(cmfflg % 256, out, outsize);
|
||||
|
||||
ZopfliDeflate(options, 2 /* dynamic block */, 1 /* final */,
|
||||
in, insize, &bitpointer, out, outsize);
|
||||
|
||||
ZOPFLI_APPEND_DATA((checksum >> 24) % 256, out, outsize);
|
||||
ZOPFLI_APPEND_DATA((checksum >> 16) % 256, out, outsize);
|
||||
ZOPFLI_APPEND_DATA((checksum >> 8) % 256, out, outsize);
|
||||
ZOPFLI_APPEND_DATA(checksum % 256, out, outsize);
|
||||
|
||||
if (options->verbose) {
|
||||
fprintf(stderr,
|
||||
"Original Size: %d, Zlib: %d, Compression: %f%% Removed\n",
|
||||
(int)insize, (int)*outsize,
|
||||
100.0 * (double)(insize - *outsize) / (double)insize);
|
||||
}
|
||||
}
|
||||
50
build/node_modules/node-zopfli/zopfli/src/zopfli/zlib_container.h
generated
vendored
Normal file
50
build/node_modules/node-zopfli/zopfli/src/zopfli/zlib_container.h
generated
vendored
Normal file
@@ -0,0 +1,50 @@
|
||||
/*
|
||||
Copyright 2013 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
#ifndef ZOPFLI_ZLIB_H_
|
||||
#define ZOPFLI_ZLIB_H_
|
||||
|
||||
/*
|
||||
Functions to compress according to the Zlib specification.
|
||||
*/
|
||||
|
||||
#include "zopfli.h"
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
/*
|
||||
Compresses according to the zlib specification and append the compressed
|
||||
result to the output.
|
||||
|
||||
options: global program options
|
||||
out: pointer to the dynamic output array to which the result is appended. Must
|
||||
be freed after use.
|
||||
outsize: pointer to the dynamic output array size.
|
||||
*/
|
||||
void ZopfliZlibCompress(const ZopfliOptions* options,
|
||||
const unsigned char* in, size_t insize,
|
||||
unsigned char** out, size_t* outsize);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} // extern "C"
|
||||
#endif
|
||||
|
||||
#endif /* ZOPFLI_ZLIB_H_ */
|
||||
94
build/node_modules/node-zopfli/zopfli/src/zopfli/zopfli.h
generated
vendored
Normal file
94
build/node_modules/node-zopfli/zopfli/src/zopfli/zopfli.h
generated
vendored
Normal file
@@ -0,0 +1,94 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
#ifndef ZOPFLI_ZOPFLI_H_
|
||||
#define ZOPFLI_ZOPFLI_H_
|
||||
|
||||
#include <stddef.h>
|
||||
#include <stdlib.h> /* for size_t */
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
/*
|
||||
Options used throughout the program.
|
||||
*/
|
||||
typedef struct ZopfliOptions {
|
||||
/* Whether to print output */
|
||||
int verbose;
|
||||
|
||||
/* Whether to print more detailed output */
|
||||
int verbose_more;
|
||||
|
||||
/*
|
||||
Maximum amount of times to rerun forward and backward pass to optimize LZ77
|
||||
compression cost. Good values: 10, 15 for small files, 5 for files over
|
||||
several MB in size or it will be too slow.
|
||||
*/
|
||||
int numiterations;
|
||||
|
||||
/*
|
||||
If true, splits the data in multiple deflate blocks with optimal choice
|
||||
for the block boundaries. Block splitting gives better compression. Default:
|
||||
true (1).
|
||||
*/
|
||||
int blocksplitting;
|
||||
|
||||
/*
|
||||
No longer used, left for compatibility.
|
||||
*/
|
||||
int blocksplittinglast;
|
||||
|
||||
/*
|
||||
Maximum amount of blocks to split into (0 for unlimited, but this can give
|
||||
extreme results that hurt compression on some files). Default value: 15.
|
||||
*/
|
||||
int blocksplittingmax;
|
||||
} ZopfliOptions;
|
||||
|
||||
/* Initializes options with default values. */
|
||||
void ZopfliInitOptions(ZopfliOptions* options);
|
||||
|
||||
/* Output format */
|
||||
typedef enum {
|
||||
ZOPFLI_FORMAT_GZIP,
|
||||
ZOPFLI_FORMAT_ZLIB,
|
||||
ZOPFLI_FORMAT_DEFLATE
|
||||
} ZopfliFormat;
|
||||
|
||||
/*
|
||||
Compresses according to the given output format and appends the result to the
|
||||
output.
|
||||
|
||||
options: global program options
|
||||
output_type: the output format to use
|
||||
out: pointer to the dynamic output array to which the result is appended. Must
|
||||
be freed after use
|
||||
outsize: pointer to the dynamic output array size
|
||||
*/
|
||||
void ZopfliCompress(const ZopfliOptions* options, ZopfliFormat output_type,
|
||||
const unsigned char* in, size_t insize,
|
||||
unsigned char** out, size_t* outsize);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} // extern "C"
|
||||
#endif
|
||||
|
||||
#endif /* ZOPFLI_ZOPFLI_H_ */
|
||||
222
build/node_modules/node-zopfli/zopfli/src/zopfli/zopfli_bin.c
generated
vendored
Normal file
222
build/node_modules/node-zopfli/zopfli/src/zopfli/zopfli_bin.c
generated
vendored
Normal file
@@ -0,0 +1,222 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
/*
|
||||
Zopfli compressor program. It can output gzip-, zlib- or deflate-compatible
|
||||
data. By default it creates a .gz file. This tool can only compress, not
|
||||
decompress. Decompression can be done by any standard gzip, zlib or deflate
|
||||
decompressor.
|
||||
*/
|
||||
|
||||
#include <assert.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
#include "deflate.h"
|
||||
#include "gzip_container.h"
|
||||
#include "zlib_container.h"
|
||||
|
||||
/* Windows workaround for stdout output. */
|
||||
#if _WIN32
|
||||
#include <fcntl.h>
|
||||
#endif
|
||||
|
||||
/*
|
||||
Loads a file into a memory array. Returns 1 on success, 0 if file doesn't exist
|
||||
or couldn't be opened.
|
||||
*/
|
||||
static int LoadFile(const char* filename,
|
||||
unsigned char** out, size_t* outsize) {
|
||||
FILE* file;
|
||||
|
||||
*out = 0;
|
||||
*outsize = 0;
|
||||
file = fopen(filename, "rb");
|
||||
if (!file) return 0;
|
||||
|
||||
fseek(file , 0 , SEEK_END);
|
||||
*outsize = ftell(file);
|
||||
if(*outsize > 2147483647) {
|
||||
fprintf(stderr,"Files larger than 2GB are not supported.\n");
|
||||
exit(EXIT_FAILURE);
|
||||
}
|
||||
rewind(file);
|
||||
|
||||
*out = (unsigned char*)malloc(*outsize);
|
||||
|
||||
if (*outsize && (*out)) {
|
||||
size_t testsize = fread(*out, 1, *outsize, file);
|
||||
if (testsize != *outsize) {
|
||||
/* It could be a directory */
|
||||
free(*out);
|
||||
*out = 0;
|
||||
*outsize = 0;
|
||||
fclose(file);
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
assert(!(*outsize) || out); /* If size is not zero, out must be allocated. */
|
||||
fclose(file);
|
||||
return 1;
|
||||
}
|
||||
|
||||
/*
|
||||
Saves a file from a memory array, overwriting the file if it existed.
|
||||
*/
|
||||
static void SaveFile(const char* filename,
|
||||
const unsigned char* in, size_t insize) {
|
||||
FILE* file = fopen(filename, "wb" );
|
||||
if (file == NULL) {
|
||||
fprintf(stderr,"Error: Cannot write to output file, terminating.\n");
|
||||
exit (EXIT_FAILURE);
|
||||
}
|
||||
assert(file);
|
||||
fwrite((char*)in, 1, insize, file);
|
||||
fclose(file);
|
||||
}
|
||||
|
||||
/*
|
||||
outfilename: filename to write output to, or 0 to write to stdout instead
|
||||
*/
|
||||
static void CompressFile(const ZopfliOptions* options,
|
||||
ZopfliFormat output_type,
|
||||
const char* infilename,
|
||||
const char* outfilename) {
|
||||
unsigned char* in;
|
||||
size_t insize;
|
||||
unsigned char* out = 0;
|
||||
size_t outsize = 0;
|
||||
if (!LoadFile(infilename, &in, &insize)) {
|
||||
fprintf(stderr, "Invalid filename: %s\n", infilename);
|
||||
return;
|
||||
}
|
||||
|
||||
ZopfliCompress(options, output_type, in, insize, &out, &outsize);
|
||||
|
||||
if (outfilename) {
|
||||
SaveFile(outfilename, out, outsize);
|
||||
} else {
|
||||
size_t i;
|
||||
#if _WIN32
|
||||
/* Windows workaround for stdout output. */
|
||||
_setmode(_fileno(stdout), _O_BINARY);
|
||||
#endif
|
||||
for (i = 0; i < outsize; i++) {
|
||||
printf("%c", out[i]);
|
||||
}
|
||||
}
|
||||
|
||||
free(out);
|
||||
free(in);
|
||||
}
|
||||
|
||||
/*
|
||||
Add two strings together. Size does not matter. Result must be freed.
|
||||
*/
|
||||
static char* AddStrings(const char* str1, const char* str2) {
|
||||
size_t len = strlen(str1) + strlen(str2);
|
||||
char* result = (char*)malloc(len + 1);
|
||||
if (!result) exit(-1); /* Allocation failed. */
|
||||
strcpy(result, str1);
|
||||
strcat(result, str2);
|
||||
return result;
|
||||
}
|
||||
|
||||
static char StringsEqual(const char* str1, const char* str2) {
|
||||
return strcmp(str1, str2) == 0;
|
||||
}
|
||||
|
||||
int main(int argc, char* argv[]) {
|
||||
ZopfliOptions options;
|
||||
ZopfliFormat output_type = ZOPFLI_FORMAT_GZIP;
|
||||
const char* filename = 0;
|
||||
int output_to_stdout = 0;
|
||||
int i;
|
||||
|
||||
ZopfliInitOptions(&options);
|
||||
|
||||
for (i = 1; i < argc; i++) {
|
||||
const char* arg = argv[i];
|
||||
if (StringsEqual(arg, "-v")) options.verbose = 1;
|
||||
else if (StringsEqual(arg, "-c")) output_to_stdout = 1;
|
||||
else if (StringsEqual(arg, "--deflate")) {
|
||||
output_type = ZOPFLI_FORMAT_DEFLATE;
|
||||
}
|
||||
else if (StringsEqual(arg, "--zlib")) output_type = ZOPFLI_FORMAT_ZLIB;
|
||||
else if (StringsEqual(arg, "--gzip")) output_type = ZOPFLI_FORMAT_GZIP;
|
||||
else if (StringsEqual(arg, "--splitlast")) /* Ignore */;
|
||||
else if (arg[0] == '-' && arg[1] == '-' && arg[2] == 'i'
|
||||
&& arg[3] >= '0' && arg[3] <= '9') {
|
||||
options.numiterations = atoi(arg + 3);
|
||||
}
|
||||
else if (StringsEqual(arg, "-h")) {
|
||||
fprintf(stderr,
|
||||
"Usage: zopfli [OPTION]... FILE...\n"
|
||||
" -h gives this help\n"
|
||||
" -c write the result on standard output, instead of disk"
|
||||
" filename + '.gz'\n"
|
||||
" -v verbose mode\n"
|
||||
" --i# perform # iterations (default 15). More gives"
|
||||
" more compression but is slower."
|
||||
" Examples: --i10, --i50, --i1000\n");
|
||||
fprintf(stderr,
|
||||
" --gzip output to gzip format (default)\n"
|
||||
" --zlib output to zlib format instead of gzip\n"
|
||||
" --deflate output to deflate format instead of gzip\n"
|
||||
" --splitlast ignored, left for backwards compatibility\n");
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
if (options.numiterations < 1) {
|
||||
fprintf(stderr, "Error: must have 1 or more iterations\n");
|
||||
return 0;
|
||||
}
|
||||
|
||||
for (i = 1; i < argc; i++) {
|
||||
if (argv[i][0] != '-') {
|
||||
char* outfilename;
|
||||
filename = argv[i];
|
||||
if (output_to_stdout) {
|
||||
outfilename = 0;
|
||||
} else if (output_type == ZOPFLI_FORMAT_GZIP) {
|
||||
outfilename = AddStrings(filename, ".gz");
|
||||
} else if (output_type == ZOPFLI_FORMAT_ZLIB) {
|
||||
outfilename = AddStrings(filename, ".zlib");
|
||||
} else {
|
||||
assert(output_type == ZOPFLI_FORMAT_DEFLATE);
|
||||
outfilename = AddStrings(filename, ".deflate");
|
||||
}
|
||||
if (options.verbose && outfilename) {
|
||||
fprintf(stderr, "Saving to: %s\n", outfilename);
|
||||
}
|
||||
CompressFile(&options, output_type, filename, outfilename);
|
||||
free(outfilename);
|
||||
}
|
||||
}
|
||||
|
||||
if (!filename) {
|
||||
fprintf(stderr,
|
||||
"Please provide filename\nFor help, type: %s -h\n", argv[0]);
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
42
build/node_modules/node-zopfli/zopfli/src/zopfli/zopfli_lib.c
generated
vendored
Normal file
42
build/node_modules/node-zopfli/zopfli/src/zopfli/zopfli_lib.c
generated
vendored
Normal file
@@ -0,0 +1,42 @@
|
||||
/*
|
||||
Copyright 2011 Google Inc. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
*/
|
||||
|
||||
#include "zopfli.h"
|
||||
|
||||
#include "deflate.h"
|
||||
#include "gzip_container.h"
|
||||
#include "zlib_container.h"
|
||||
|
||||
#include <assert.h>
|
||||
|
||||
void ZopfliCompress(const ZopfliOptions* options, ZopfliFormat output_type,
|
||||
const unsigned char* in, size_t insize,
|
||||
unsigned char** out, size_t* outsize) {
|
||||
if (output_type == ZOPFLI_FORMAT_GZIP) {
|
||||
ZopfliGzipCompress(options, in, insize, out, outsize);
|
||||
} else if (output_type == ZOPFLI_FORMAT_ZLIB) {
|
||||
ZopfliZlibCompress(options, in, insize, out, outsize);
|
||||
} else if (output_type == ZOPFLI_FORMAT_DEFLATE) {
|
||||
unsigned char bp = 0;
|
||||
ZopfliDeflate(options, 2 /* Dynamic block */, 1,
|
||||
in, insize, &bp, out, outsize);
|
||||
} else {
|
||||
assert(0);
|
||||
}
|
||||
}
|
||||
6206
build/node_modules/node-zopfli/zopfli/src/zopflipng/lodepng/lodepng.cpp
generated
vendored
Normal file
6206
build/node_modules/node-zopfli/zopfli/src/zopflipng/lodepng/lodepng.cpp
generated
vendored
Normal file
File diff suppressed because it is too large
Load Diff
1758
build/node_modules/node-zopfli/zopfli/src/zopflipng/lodepng/lodepng.h
generated
vendored
Normal file
1758
build/node_modules/node-zopfli/zopfli/src/zopflipng/lodepng/lodepng.h
generated
vendored
Normal file
File diff suppressed because it is too large
Load Diff
671
build/node_modules/node-zopfli/zopfli/src/zopflipng/lodepng/lodepng_util.cpp
generated
vendored
Normal file
671
build/node_modules/node-zopfli/zopfli/src/zopflipng/lodepng/lodepng_util.cpp
generated
vendored
Normal file
@@ -0,0 +1,671 @@
|
||||
/*
|
||||
LodePNG Utils
|
||||
|
||||
Copyright (c) 2005-2014 Lode Vandevenne
|
||||
|
||||
This software is provided 'as-is', without any express or implied
|
||||
warranty. In no event will the authors be held liable for any damages
|
||||
arising from the use of this software.
|
||||
|
||||
Permission is granted to anyone to use this software for any purpose,
|
||||
including commercial applications, and to alter it and redistribute it
|
||||
freely, subject to the following restrictions:
|
||||
|
||||
1. The origin of this software must not be misrepresented; you must not
|
||||
claim that you wrote the original software. If you use this software
|
||||
in a product, an acknowledgment in the product documentation would be
|
||||
appreciated but is not required.
|
||||
|
||||
2. Altered source versions must be plainly marked as such, and must not be
|
||||
misrepresented as being the original software.
|
||||
|
||||
3. This notice may not be removed or altered from any source
|
||||
distribution.
|
||||
*/
|
||||
|
||||
#include "lodepng_util.h"
|
||||
#include <iostream>
|
||||
|
||||
namespace lodepng
|
||||
{
|
||||
|
||||
LodePNGInfo getPNGHeaderInfo(const std::vector<unsigned char>& png)
|
||||
{
|
||||
unsigned w, h;
|
||||
lodepng::State state;
|
||||
lodepng_inspect(&w, &h, &state, &png[0], png.size());
|
||||
return state.info_png;
|
||||
}
|
||||
|
||||
unsigned getChunkInfo(std::vector<std::string>& names, std::vector<size_t>& sizes,
|
||||
const std::vector<unsigned char>& png)
|
||||
{
|
||||
// Listing chunks is based on the original file, not the decoded png info.
|
||||
const unsigned char *chunk, *begin, *end, *next;
|
||||
end = &png.back() + 1;
|
||||
begin = chunk = &png.front() + 8;
|
||||
|
||||
while(chunk + 8 < end && chunk >= begin)
|
||||
{
|
||||
char type[5];
|
||||
lodepng_chunk_type(type, chunk);
|
||||
if(std::string(type).size() != 4) return 1;
|
||||
|
||||
unsigned length = lodepng_chunk_length(chunk);
|
||||
if(chunk + length + 12 > end) return 1;
|
||||
names.push_back(type);
|
||||
sizes.push_back(length);
|
||||
|
||||
next = lodepng_chunk_next_const(chunk);
|
||||
if (next <= chunk) return 1; // integer overflow
|
||||
chunk = next;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
unsigned getChunks(std::vector<std::string> names[3],
|
||||
std::vector<std::vector<unsigned char> > chunks[3],
|
||||
const std::vector<unsigned char>& png)
|
||||
{
|
||||
const unsigned char *chunk, *next, *begin, *end;
|
||||
end = &png.back() + 1;
|
||||
begin = chunk = &png.front() + 8;
|
||||
|
||||
int location = 0;
|
||||
|
||||
while(chunk + 8 < end && chunk >= begin)
|
||||
{
|
||||
char type[5];
|
||||
lodepng_chunk_type(type, chunk);
|
||||
std::string name(type);
|
||||
if(name.size() != 4) return 1;
|
||||
|
||||
next = lodepng_chunk_next_const(chunk);
|
||||
if (next <= chunk) return 1; // integer overflow
|
||||
|
||||
if(name == "IHDR")
|
||||
{
|
||||
location = 0;
|
||||
}
|
||||
else if(name == "PLTE")
|
||||
{
|
||||
location = 1;
|
||||
}
|
||||
else if(name == "IDAT")
|
||||
{
|
||||
location = 2;
|
||||
}
|
||||
else if(name == "IEND")
|
||||
{
|
||||
break; // anything after IEND is not part of the PNG or the 3 groups here.
|
||||
}
|
||||
else
|
||||
{
|
||||
if(next > end) return 1; // invalid chunk, content too far
|
||||
names[location].push_back(name);
|
||||
chunks[location].push_back(std::vector<unsigned char>(chunk, next));
|
||||
}
|
||||
|
||||
chunk = next;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
unsigned insertChunks(std::vector<unsigned char>& png,
|
||||
const std::vector<std::vector<unsigned char> > chunks[3])
|
||||
{
|
||||
const unsigned char *chunk, *next, *begin, *end;
|
||||
end = &png.back() + 1;
|
||||
begin = chunk = &png.front() + 8;
|
||||
|
||||
long l0 = 0; //location 0: IHDR-l0-PLTE (or IHDR-l0-l1-IDAT)
|
||||
long l1 = 0; //location 1: PLTE-l1-IDAT (or IHDR-l0-l1-IDAT)
|
||||
long l2 = 0; //location 2: IDAT-l2-IEND
|
||||
|
||||
while(chunk + 8 < end && chunk >= begin)
|
||||
{
|
||||
char type[5];
|
||||
lodepng_chunk_type(type, chunk);
|
||||
std::string name(type);
|
||||
if(name.size() != 4) return 1;
|
||||
|
||||
next = lodepng_chunk_next_const(chunk);
|
||||
if (next <= chunk) return 1; // integer overflow
|
||||
|
||||
if(name == "PLTE")
|
||||
{
|
||||
if(l0 == 0) l0 = chunk - begin + 8;
|
||||
}
|
||||
else if(name == "IDAT")
|
||||
{
|
||||
if(l0 == 0) l0 = chunk - begin + 8;
|
||||
if(l1 == 0) l1 = chunk - begin + 8;
|
||||
}
|
||||
else if(name == "IEND")
|
||||
{
|
||||
if(l2 == 0) l2 = chunk - begin + 8;
|
||||
}
|
||||
|
||||
chunk = next;
|
||||
}
|
||||
|
||||
std::vector<unsigned char> result;
|
||||
result.insert(result.end(), png.begin(), png.begin() + l0);
|
||||
for(size_t i = 0; i < chunks[0].size(); i++) result.insert(result.end(), chunks[0][i].begin(), chunks[0][i].end());
|
||||
result.insert(result.end(), png.begin() + l0, png.begin() + l1);
|
||||
for(size_t i = 0; i < chunks[1].size(); i++) result.insert(result.end(), chunks[1][i].begin(), chunks[1][i].end());
|
||||
result.insert(result.end(), png.begin() + l1, png.begin() + l2);
|
||||
for(size_t i = 0; i < chunks[2].size(); i++) result.insert(result.end(), chunks[2][i].begin(), chunks[2][i].end());
|
||||
result.insert(result.end(), png.begin() + l2, png.end());
|
||||
|
||||
png = result;
|
||||
return 0;
|
||||
}
|
||||
|
||||
unsigned getFilterTypesInterlaced(std::vector<std::vector<unsigned char> >& filterTypes,
|
||||
const std::vector<unsigned char>& png)
|
||||
{
|
||||
//Get color type and interlace type
|
||||
lodepng::State state;
|
||||
unsigned w, h;
|
||||
unsigned error;
|
||||
error = lodepng_inspect(&w, &h, &state, &png[0], png.size());
|
||||
|
||||
if(error) return 1;
|
||||
|
||||
//Read literal data from all IDAT chunks
|
||||
const unsigned char *chunk, *begin, *end, *next;
|
||||
end = &png.back() + 1;
|
||||
begin = chunk = &png.front() + 8;
|
||||
|
||||
std::vector<unsigned char> zdata;
|
||||
|
||||
while(chunk + 8 < end && chunk >= begin)
|
||||
{
|
||||
char type[5];
|
||||
lodepng_chunk_type(type, chunk);
|
||||
if(std::string(type).size() != 4) return 1; //Probably not a PNG file
|
||||
|
||||
if(std::string(type) == "IDAT")
|
||||
{
|
||||
const unsigned char* cdata = lodepng_chunk_data_const(chunk);
|
||||
unsigned clength = lodepng_chunk_length(chunk);
|
||||
if(chunk + clength + 12 > end || clength > png.size() || chunk + clength + 12 < begin) {
|
||||
// corrupt chunk length
|
||||
return 1;
|
||||
}
|
||||
|
||||
for(unsigned i = 0; i < clength; i++)
|
||||
{
|
||||
zdata.push_back(cdata[i]);
|
||||
}
|
||||
}
|
||||
|
||||
next = lodepng_chunk_next_const(chunk);
|
||||
if (next <= chunk) return 1; // integer overflow
|
||||
chunk = next;
|
||||
}
|
||||
|
||||
//Decompress all IDAT data
|
||||
std::vector<unsigned char> data;
|
||||
error = lodepng::decompress(data, &zdata[0], zdata.size());
|
||||
|
||||
if(error) return 1;
|
||||
|
||||
if(state.info_png.interlace_method == 0)
|
||||
{
|
||||
filterTypes.resize(1);
|
||||
|
||||
//A line is 1 filter byte + all pixels
|
||||
size_t linebytes = 1 + lodepng_get_raw_size(w, 1, &state.info_png.color);
|
||||
|
||||
for(size_t i = 0; i < data.size(); i += linebytes)
|
||||
{
|
||||
filterTypes[0].push_back(data[i]);
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
//Interlaced
|
||||
filterTypes.resize(7);
|
||||
static const unsigned ADAM7_IX[7] = { 0, 4, 0, 2, 0, 1, 0 }; /*x start values*/
|
||||
static const unsigned ADAM7_IY[7] = { 0, 0, 4, 0, 2, 0, 1 }; /*y start values*/
|
||||
static const unsigned ADAM7_DX[7] = { 8, 8, 4, 4, 2, 2, 1 }; /*x delta values*/
|
||||
static const unsigned ADAM7_DY[7] = { 8, 8, 8, 4, 4, 2, 2 }; /*y delta values*/
|
||||
size_t pos = 0;
|
||||
for(size_t j = 0; j < 7; j++)
|
||||
{
|
||||
unsigned w2 = (w - ADAM7_IX[j] + ADAM7_DX[j] - 1) / ADAM7_DX[j];
|
||||
unsigned h2 = (h - ADAM7_IY[j] + ADAM7_DY[j] - 1) / ADAM7_DY[j];
|
||||
if(ADAM7_IX[j] >= w) w2 = 0;
|
||||
if(ADAM7_IY[j] >= h) h2 = 0;
|
||||
size_t linebytes = 1 + lodepng_get_raw_size(w2, 1, &state.info_png.color);
|
||||
for(size_t i = 0; i < h2; i++)
|
||||
{
|
||||
filterTypes[j].push_back(data[pos]);
|
||||
pos += linebytes;
|
||||
}
|
||||
}
|
||||
}
|
||||
return 0; /* OK */
|
||||
}
|
||||
|
||||
|
||||
unsigned getFilterTypes(std::vector<unsigned char>& filterTypes, const std::vector<unsigned char>& png)
|
||||
{
|
||||
std::vector<std::vector<unsigned char> > passes;
|
||||
unsigned error = getFilterTypesInterlaced(passes, png);
|
||||
if(error) return error;
|
||||
|
||||
if(passes.size() == 1)
|
||||
{
|
||||
filterTypes.swap(passes[0]);
|
||||
}
|
||||
else
|
||||
{
|
||||
lodepng::State state;
|
||||
unsigned w, h;
|
||||
lodepng_inspect(&w, &h, &state, &png[0], png.size());
|
||||
/*
|
||||
Interlaced. Simplify it: put pass 6 and 7 alternating in the one vector so
|
||||
that one filter per scanline of the uninterlaced image is given, with that
|
||||
filter corresponding the closest to what it would be for non-interlaced
|
||||
image.
|
||||
*/
|
||||
for(size_t i = 0; i < h; i++)
|
||||
{
|
||||
filterTypes.push_back(i % 2 == 0 ? passes[5][i / 2] : passes[6][i / 2]);
|
||||
}
|
||||
}
|
||||
return 0; /* OK */
|
||||
}
|
||||
|
||||
int getPaletteValue(const unsigned char* data, size_t i, int bits)
|
||||
{
|
||||
if(bits == 8) return data[i];
|
||||
else if(bits == 4) return (data[i / 2] >> ((i % 2) * 4)) & 15;
|
||||
else if(bits == 2) return (data[i / 4] >> ((i % 4) * 2)) & 3;
|
||||
else if(bits == 1) return (data[i / 8] >> (i % 8)) & 1;
|
||||
else return 0;
|
||||
}
|
||||
|
||||
//This uses a stripped down version of picoPNG to extract detailed zlib information while decompressing.
|
||||
static const unsigned long LENBASE[29] =
|
||||
{3,4,5,6,7,8,9,10,11,13,15,17,19,23,27,31,35,43,51,59,67,83,99,115,131,163,195,227,258};
|
||||
static const unsigned long LENEXTRA[29] =
|
||||
{0,0,0,0,0,0,0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 0};
|
||||
static const unsigned long DISTBASE[30] =
|
||||
{1,2,3,4,5,7,9,13,17,25,33,49,65,97,129,193,257,385,513,769,1025,1537,2049,3073,4097,6145,8193,12289,16385,24577};
|
||||
static const unsigned long DISTEXTRA[30] =
|
||||
{0,0,0,0,1,1,2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13};
|
||||
static const unsigned long CLCL[19] =
|
||||
{16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15}; //code length code lengths
|
||||
|
||||
struct ExtractZlib // Zlib decompression and information extraction
|
||||
{
|
||||
std::vector<ZlibBlockInfo>* zlibinfo;
|
||||
ExtractZlib(std::vector<ZlibBlockInfo>* info) : zlibinfo(info) {};
|
||||
int error;
|
||||
|
||||
unsigned long readBitFromStream(size_t& bitp, const unsigned char* bits)
|
||||
{
|
||||
unsigned long result = (bits[bitp >> 3] >> (bitp & 0x7)) & 1;
|
||||
bitp++;
|
||||
return result;
|
||||
}
|
||||
|
||||
unsigned long readBitsFromStream(size_t& bitp, const unsigned char* bits, size_t nbits)
|
||||
{
|
||||
unsigned long result = 0;
|
||||
for(size_t i = 0; i < nbits; i++) result += (readBitFromStream(bitp, bits)) << i;
|
||||
return result;
|
||||
}
|
||||
|
||||
struct HuffmanTree
|
||||
{
|
||||
int makeFromLengths(const std::vector<unsigned long>& bitlen, unsigned long maxbitlen)
|
||||
{ //make tree given the lengths
|
||||
unsigned long numcodes = (unsigned long)(bitlen.size()), treepos = 0, nodefilled = 0;
|
||||
std::vector<unsigned long> tree1d(numcodes), blcount(maxbitlen + 1, 0), nextcode(maxbitlen + 1, 0);
|
||||
//count number of instances of each code length
|
||||
for(unsigned long bits = 0; bits < numcodes; bits++) blcount[bitlen[bits]]++;
|
||||
for(unsigned long bits = 1; bits <= maxbitlen; bits++)
|
||||
{
|
||||
nextcode[bits] = (nextcode[bits - 1] + blcount[bits - 1]) << 1;
|
||||
}
|
||||
//generate all the codes
|
||||
for(unsigned long n = 0; n < numcodes; n++) if(bitlen[n] != 0) tree1d[n] = nextcode[bitlen[n]]++;
|
||||
tree2d.clear(); tree2d.resize(numcodes * 2, 32767); //32767 here means the tree2d isn't filled there yet
|
||||
for(unsigned long n = 0; n < numcodes; n++) //the codes
|
||||
for(unsigned long i = 0; i < bitlen[n]; i++) //the bits for this code
|
||||
{
|
||||
unsigned long bit = (tree1d[n] >> (bitlen[n] - i - 1)) & 1;
|
||||
if(treepos > numcodes - 2) return 55;
|
||||
if(tree2d[2 * treepos + bit] == 32767) //not yet filled in
|
||||
{
|
||||
if(i + 1 == bitlen[n])
|
||||
{
|
||||
//last bit
|
||||
tree2d[2 * treepos + bit] = n;
|
||||
treepos = 0;
|
||||
}
|
||||
else
|
||||
{
|
||||
//addresses are encoded as values > numcodes
|
||||
tree2d[2 * treepos + bit] = ++nodefilled + numcodes;
|
||||
treepos = nodefilled;
|
||||
}
|
||||
}
|
||||
else treepos = tree2d[2 * treepos + bit] - numcodes; //subtract numcodes from address to get address value
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
int decode(bool& decoded, unsigned long& result, size_t& treepos, unsigned long bit) const
|
||||
{ //Decodes a symbol from the tree
|
||||
unsigned long numcodes = (unsigned long)tree2d.size() / 2;
|
||||
if(treepos >= numcodes) return 11; //error: you appeared outside the codetree
|
||||
result = tree2d[2 * treepos + bit];
|
||||
decoded = (result < numcodes);
|
||||
treepos = decoded ? 0 : result - numcodes;
|
||||
return 0;
|
||||
}
|
||||
//2D representation of a huffman tree: one dimension is "0" or "1", the other contains all nodes and leaves.
|
||||
std::vector<unsigned long> tree2d;
|
||||
};
|
||||
|
||||
void inflate(std::vector<unsigned char>& out, const std::vector<unsigned char>& in, size_t inpos = 0)
|
||||
{
|
||||
size_t bp = 0, pos = 0; //bit pointer and byte pointer
|
||||
error = 0;
|
||||
unsigned long BFINAL = 0;
|
||||
while(!BFINAL && !error)
|
||||
{
|
||||
size_t uncomprblockstart = pos;
|
||||
size_t bpstart = bp;
|
||||
if(bp >> 3 >= in.size()) { error = 52; return; } //error, bit pointer will jump past memory
|
||||
BFINAL = readBitFromStream(bp, &in[inpos]);
|
||||
unsigned long BTYPE = readBitFromStream(bp, &in[inpos]); BTYPE += 2 * readBitFromStream(bp, &in[inpos]);
|
||||
zlibinfo->resize(zlibinfo->size() + 1);
|
||||
zlibinfo->back().btype = BTYPE;
|
||||
if(BTYPE == 3) { error = 20; return; } //error: invalid BTYPE
|
||||
else if(BTYPE == 0) inflateNoCompression(out, &in[inpos], bp, pos, in.size());
|
||||
else inflateHuffmanBlock(out, &in[inpos], bp, pos, in.size(), BTYPE);
|
||||
size_t uncomprblocksize = pos - uncomprblockstart;
|
||||
zlibinfo->back().compressedbits = bp - bpstart;
|
||||
zlibinfo->back().uncompressedbytes = uncomprblocksize;
|
||||
}
|
||||
}
|
||||
|
||||
void generateFixedTrees(HuffmanTree& tree, HuffmanTree& treeD) //get the tree of a deflated block with fixed tree
|
||||
{
|
||||
std::vector<unsigned long> bitlen(288, 8), bitlenD(32, 5);;
|
||||
for(size_t i = 144; i <= 255; i++) bitlen[i] = 9;
|
||||
for(size_t i = 256; i <= 279; i++) bitlen[i] = 7;
|
||||
tree.makeFromLengths(bitlen, 15);
|
||||
treeD.makeFromLengths(bitlenD, 15);
|
||||
}
|
||||
|
||||
//the code tree for Huffman codes, dist codes, and code length codes
|
||||
HuffmanTree codetree, codetreeD, codelengthcodetree;
|
||||
unsigned long huffmanDecodeSymbol(const unsigned char* in, size_t& bp, const HuffmanTree& tree, size_t inlength)
|
||||
{
|
||||
//decode a single symbol from given list of bits with given code tree. return value is the symbol
|
||||
bool decoded; unsigned long ct;
|
||||
for(size_t treepos = 0;;)
|
||||
{
|
||||
if((bp & 0x07) == 0 && (bp >> 3) > inlength) { error = 10; return 0; } //error: end reached without endcode
|
||||
error = tree.decode(decoded, ct, treepos, readBitFromStream(bp, in));
|
||||
if(error) return 0; //stop, an error happened
|
||||
if(decoded) return ct;
|
||||
}
|
||||
}
|
||||
|
||||
void getTreeInflateDynamic(HuffmanTree& tree, HuffmanTree& treeD,
|
||||
const unsigned char* in, size_t& bp, size_t inlength)
|
||||
{
|
||||
size_t bpstart = bp;
|
||||
//get the tree of a deflated block with dynamic tree, the tree itself is also Huffman compressed with a known tree
|
||||
std::vector<unsigned long> bitlen(288, 0), bitlenD(32, 0);
|
||||
if(bp >> 3 >= inlength - 2) { error = 49; return; } //the bit pointer is or will go past the memory
|
||||
size_t HLIT = readBitsFromStream(bp, in, 5) + 257; //number of literal/length codes + 257
|
||||
size_t HDIST = readBitsFromStream(bp, in, 5) + 1; //number of dist codes + 1
|
||||
size_t HCLEN = readBitsFromStream(bp, in, 4) + 4; //number of code length codes + 4
|
||||
zlibinfo->back().hlit = HLIT - 257;
|
||||
zlibinfo->back().hdist = HDIST - 1;
|
||||
zlibinfo->back().hclen = HCLEN - 4;
|
||||
std::vector<unsigned long> codelengthcode(19); //lengths of tree to decode the lengths of the dynamic tree
|
||||
for(size_t i = 0; i < 19; i++) codelengthcode[CLCL[i]] = (i < HCLEN) ? readBitsFromStream(bp, in, 3) : 0;
|
||||
//code length code lengths
|
||||
for(size_t i = 0; i < codelengthcode.size(); i++) zlibinfo->back().clcl.push_back(codelengthcode[i]);
|
||||
error = codelengthcodetree.makeFromLengths(codelengthcode, 7); if(error) return;
|
||||
size_t i = 0, replength;
|
||||
while(i < HLIT + HDIST)
|
||||
{
|
||||
unsigned long code = huffmanDecodeSymbol(in, bp, codelengthcodetree, inlength); if(error) return;
|
||||
zlibinfo->back().treecodes.push_back(code); //tree symbol code
|
||||
if(code <= 15) { if(i < HLIT) bitlen[i++] = code; else bitlenD[i++ - HLIT] = code; } //a length code
|
||||
else if(code == 16) //repeat previous
|
||||
{
|
||||
if(bp >> 3 >= inlength) { error = 50; return; } //error, bit pointer jumps past memory
|
||||
replength = 3 + readBitsFromStream(bp, in, 2);
|
||||
unsigned long value; //set value to the previous code
|
||||
if((i - 1) < HLIT) value = bitlen[i - 1];
|
||||
else value = bitlenD[i - HLIT - 1];
|
||||
for(size_t n = 0; n < replength; n++) //repeat this value in the next lengths
|
||||
{
|
||||
if(i >= HLIT + HDIST) { error = 13; return; } //error: i is larger than the amount of codes
|
||||
if(i < HLIT) bitlen[i++] = value; else bitlenD[i++ - HLIT] = value;
|
||||
}
|
||||
}
|
||||
else if(code == 17) //repeat "0" 3-10 times
|
||||
{
|
||||
if(bp >> 3 >= inlength) { error = 50; return; } //error, bit pointer jumps past memory
|
||||
replength = 3 + readBitsFromStream(bp, in, 3);
|
||||
zlibinfo->back().treecodes.push_back(replength); //tree symbol code repetitions
|
||||
for(size_t n = 0; n < replength; n++) //repeat this value in the next lengths
|
||||
{
|
||||
if(i >= HLIT + HDIST) { error = 14; return; } //error: i is larger than the amount of codes
|
||||
if(i < HLIT) bitlen[i++] = 0; else bitlenD[i++ - HLIT] = 0;
|
||||
}
|
||||
}
|
||||
else if(code == 18) //repeat "0" 11-138 times
|
||||
{
|
||||
if(bp >> 3 >= inlength) { error = 50; return; } //error, bit pointer jumps past memory
|
||||
replength = 11 + readBitsFromStream(bp, in, 7);
|
||||
zlibinfo->back().treecodes.push_back(replength); //tree symbol code repetitions
|
||||
for(size_t n = 0; n < replength; n++) //repeat this value in the next lengths
|
||||
{
|
||||
if(i >= HLIT + HDIST) { error = 15; return; } //error: i is larger than the amount of codes
|
||||
if(i < HLIT) bitlen[i++] = 0; else bitlenD[i++ - HLIT] = 0;
|
||||
}
|
||||
}
|
||||
else { error = 16; return; } //error: somehow an unexisting code appeared. This can never happen.
|
||||
}
|
||||
if(bitlen[256] == 0) { error = 64; return; } //the length of the end code 256 must be larger than 0
|
||||
error = tree.makeFromLengths(bitlen, 15);
|
||||
if(error) return; //now we've finally got HLIT and HDIST, so generate the code trees, and the function is done
|
||||
error = treeD.makeFromLengths(bitlenD, 15);
|
||||
if(error) return;
|
||||
zlibinfo->back().treebits = bp - bpstart;
|
||||
//lit/len/end symbol lengths
|
||||
for(size_t j = 0; j < bitlen.size(); j++) zlibinfo->back().litlenlengths.push_back(bitlen[j]);
|
||||
//dist lengths
|
||||
for(size_t j = 0; j < bitlenD.size(); j++) zlibinfo->back().distlengths.push_back(bitlenD[j]);
|
||||
}
|
||||
|
||||
void inflateHuffmanBlock(std::vector<unsigned char>& out,
|
||||
const unsigned char* in, size_t& bp, size_t& pos, size_t inlength, unsigned long btype)
|
||||
{
|
||||
size_t numcodes = 0, numlit = 0, numlen = 0; //for logging
|
||||
if(btype == 1) { generateFixedTrees(codetree, codetreeD); }
|
||||
else if(btype == 2) { getTreeInflateDynamic(codetree, codetreeD, in, bp, inlength); if(error) return; }
|
||||
for(;;)
|
||||
{
|
||||
unsigned long code = huffmanDecodeSymbol(in, bp, codetree, inlength); if(error) return;
|
||||
numcodes++;
|
||||
zlibinfo->back().lz77_lcode.push_back(code); //output code
|
||||
zlibinfo->back().lz77_dcode.push_back(0);
|
||||
zlibinfo->back().lz77_lbits.push_back(0);
|
||||
zlibinfo->back().lz77_dbits.push_back(0);
|
||||
zlibinfo->back().lz77_lvalue.push_back(0);
|
||||
zlibinfo->back().lz77_dvalue.push_back(0);
|
||||
|
||||
if(code == 256) break; //end code
|
||||
else if(code <= 255) //literal symbol
|
||||
{
|
||||
out.push_back((unsigned char)(code));
|
||||
pos++;
|
||||
numlit++;
|
||||
}
|
||||
else if(code >= 257 && code <= 285) //length code
|
||||
{
|
||||
size_t length = LENBASE[code - 257], numextrabits = LENEXTRA[code - 257];
|
||||
if((bp >> 3) >= inlength) { error = 51; return; } //error, bit pointer will jump past memory
|
||||
length += readBitsFromStream(bp, in, numextrabits);
|
||||
unsigned long codeD = huffmanDecodeSymbol(in, bp, codetreeD, inlength); if(error) return;
|
||||
if(codeD > 29) { error = 18; return; } //error: invalid dist code (30-31 are never used)
|
||||
unsigned long dist = DISTBASE[codeD], numextrabitsD = DISTEXTRA[codeD];
|
||||
if((bp >> 3) >= inlength) { error = 51; return; } //error, bit pointer will jump past memory
|
||||
dist += readBitsFromStream(bp, in, numextrabitsD);
|
||||
size_t start = pos, back = start - dist; //backwards
|
||||
for(size_t i = 0; i < length; i++)
|
||||
{
|
||||
out.push_back(out[back++]);
|
||||
pos++;
|
||||
if(back >= start) back = start - dist;
|
||||
}
|
||||
numlen++;
|
||||
zlibinfo->back().lz77_dcode.back() = codeD; //output distance code
|
||||
zlibinfo->back().lz77_lbits.back() = numextrabits; //output length extra bits
|
||||
zlibinfo->back().lz77_dbits.back() = numextrabitsD; //output dist extra bits
|
||||
zlibinfo->back().lz77_lvalue.back() = length; //output length
|
||||
zlibinfo->back().lz77_dvalue.back() = dist; //output dist
|
||||
}
|
||||
}
|
||||
zlibinfo->back().numlit = numlit; //output number of literal symbols
|
||||
zlibinfo->back().numlen = numlen; //output number of length symbols
|
||||
}
|
||||
|
||||
void inflateNoCompression(std::vector<unsigned char>& out,
|
||||
const unsigned char* in, size_t& bp, size_t& pos, size_t inlength)
|
||||
{
|
||||
while((bp & 0x7) != 0) bp++; //go to first boundary of byte
|
||||
size_t p = bp / 8;
|
||||
if(p >= inlength - 4) { error = 52; return; } //error, bit pointer will jump past memory
|
||||
unsigned long LEN = in[p] + 256u * in[p + 1], NLEN = in[p + 2] + 256u * in[p + 3]; p += 4;
|
||||
if(LEN + NLEN != 65535) { error = 21; return; } //error: NLEN is not one's complement of LEN
|
||||
if(p + LEN > inlength) { error = 23; return; } //error: reading outside of in buffer
|
||||
for(unsigned long n = 0; n < LEN; n++)
|
||||
{
|
||||
out.push_back(in[p++]); //read LEN bytes of literal data
|
||||
pos++;
|
||||
}
|
||||
bp = p * 8;
|
||||
}
|
||||
|
||||
int decompress(std::vector<unsigned char>& out, const std::vector<unsigned char>& in) //returns error value
|
||||
{
|
||||
if(in.size() < 2) { return 53; } //error, size of zlib data too small
|
||||
//error: 256 * in[0] + in[1] must be a multiple of 31, the FCHECK value is supposed to be made that way
|
||||
if((in[0] * 256 + in[1]) % 31 != 0) { return 24; }
|
||||
unsigned long CM = in[0] & 15, CINFO = (in[0] >> 4) & 15, FDICT = (in[1] >> 5) & 1;
|
||||
//error: only compression method 8: inflate with sliding window of 32k is supported by the PNG spec
|
||||
if(CM != 8 || CINFO > 7) { return 25; }
|
||||
//error: the PNG spec says about the zlib stream: "The additional flags shall not specify a preset dictionary."
|
||||
if(FDICT != 0) { return 26; }
|
||||
inflate(out, in, 2);
|
||||
return error; //note: adler32 checksum was skipped and ignored
|
||||
}
|
||||
};
|
||||
|
||||
struct ExtractPNG //PNG decoding and information extraction
|
||||
{
|
||||
std::vector<ZlibBlockInfo>* zlibinfo;
|
||||
ExtractPNG(std::vector<ZlibBlockInfo>* info) : zlibinfo(info) {};
|
||||
int error;
|
||||
void decode(const unsigned char* in, size_t size)
|
||||
{
|
||||
error = 0;
|
||||
if(size == 0 || in == 0) { error = 48; return; } //the given data is empty
|
||||
readPngHeader(&in[0], size); if(error) return;
|
||||
size_t pos = 33; //first byte of the first chunk after the header
|
||||
std::vector<unsigned char> idat; //the data from idat chunks
|
||||
bool IEND = false;
|
||||
//loop through the chunks, ignoring unknown chunks and stopping at IEND chunk.
|
||||
//IDAT data is put at the start of the in buffer
|
||||
while(!IEND)
|
||||
{
|
||||
//error: size of the in buffer too small to contain next chunk
|
||||
if(pos + 8 >= size) { error = 30; return; }
|
||||
size_t chunkLength = read32bitInt(&in[pos]); pos += 4;
|
||||
if(chunkLength > 2147483647) { error = 63; return; }
|
||||
//error: size of the in buffer too small to contain next chunk
|
||||
if(pos + chunkLength >= size) { error = 35; return; }
|
||||
//IDAT chunk, containing compressed image data
|
||||
if(in[pos + 0] == 'I' && in[pos + 1] == 'D' && in[pos + 2] == 'A' && in[pos + 3] == 'T')
|
||||
{
|
||||
idat.insert(idat.end(), &in[pos + 4], &in[pos + 4 + chunkLength]);
|
||||
pos += (4 + chunkLength);
|
||||
}
|
||||
else if(in[pos + 0] == 'I' && in[pos + 1] == 'E' && in[pos + 2] == 'N' && in[pos + 3] == 'D')
|
||||
{
|
||||
pos += 4;
|
||||
IEND = true;
|
||||
}
|
||||
else //it's not an implemented chunk type, so ignore it: skip over the data
|
||||
{
|
||||
pos += (chunkLength + 4); //skip 4 letters and uninterpreted data of unimplemented chunk
|
||||
}
|
||||
pos += 4; //step over CRC (which is ignored)
|
||||
}
|
||||
std::vector<unsigned char> out; //now the out buffer will be filled
|
||||
ExtractZlib zlib(zlibinfo); //decompress with the Zlib decompressor
|
||||
error = zlib.decompress(out, idat);
|
||||
if(error) return; //stop if the zlib decompressor returned an error
|
||||
}
|
||||
|
||||
//read the information from the header and store it in the Info
|
||||
void readPngHeader(const unsigned char* in, size_t inlength)
|
||||
{
|
||||
if(inlength < 29) { error = 27; return; } //error: the data length is smaller than the length of the header
|
||||
if(in[0] != 137 || in[1] != 80 || in[2] != 78 || in[3] != 71
|
||||
|| in[4] != 13 || in[5] != 10 || in[6] != 26 || in[7] != 10) { error = 28; return; } //no PNG signature
|
||||
//error: it doesn't start with a IHDR chunk!
|
||||
if(in[12] != 'I' || in[13] != 'H' || in[14] != 'D' || in[15] != 'R') { error = 29; return; }
|
||||
}
|
||||
|
||||
unsigned long readBitFromReversedStream(size_t& bitp, const unsigned char* bits)
|
||||
{
|
||||
unsigned long result = (bits[bitp >> 3] >> (7 - (bitp & 0x7))) & 1;
|
||||
bitp++;
|
||||
return result;
|
||||
}
|
||||
|
||||
unsigned long readBitsFromReversedStream(size_t& bitp, const unsigned char* bits, unsigned long nbits)
|
||||
{
|
||||
unsigned long result = 0;
|
||||
for(size_t i = nbits - 1; i < nbits; i--) result += ((readBitFromReversedStream(bitp, bits)) << i);
|
||||
return result;
|
||||
}
|
||||
|
||||
void setBitOfReversedStream(size_t& bitp, unsigned char* bits, unsigned long bit)
|
||||
{
|
||||
bits[bitp >> 3] |= (bit << (7 - (bitp & 0x7))); bitp++;
|
||||
}
|
||||
|
||||
unsigned long read32bitInt(const unsigned char* buffer)
|
||||
{
|
||||
return (unsigned int)((buffer[0] << 24u) | (buffer[1] << 16u) | (buffer[2] << 8u) | buffer[3]);
|
||||
}
|
||||
};
|
||||
|
||||
void extractZlibInfo(std::vector<ZlibBlockInfo>& zlibinfo, const std::vector<unsigned char>& in)
|
||||
{
|
||||
ExtractPNG decoder(&zlibinfo);
|
||||
decoder.decode(&in[0], in.size());
|
||||
|
||||
if(decoder.error) std::cout << "extract error: " << decoder.error << std::endl;
|
||||
}
|
||||
|
||||
} // namespace lodepng
|
||||
151
build/node_modules/node-zopfli/zopfli/src/zopflipng/lodepng/lodepng_util.h
generated
vendored
Normal file
151
build/node_modules/node-zopfli/zopfli/src/zopflipng/lodepng/lodepng_util.h
generated
vendored
Normal file
@@ -0,0 +1,151 @@
|
||||
/*
|
||||
LodePNG Utils
|
||||
|
||||
Copyright (c) 2005-2014 Lode Vandevenne
|
||||
|
||||
This software is provided 'as-is', without any express or implied
|
||||
warranty. In no event will the authors be held liable for any damages
|
||||
arising from the use of this software.
|
||||
|
||||
Permission is granted to anyone to use this software for any purpose,
|
||||
including commercial applications, and to alter it and redistribute it
|
||||
freely, subject to the following restrictions:
|
||||
|
||||
1. The origin of this software must not be misrepresented; you must not
|
||||
claim that you wrote the original software. If you use this software
|
||||
in a product, an acknowledgment in the product documentation would be
|
||||
appreciated but is not required.
|
||||
|
||||
2. Altered source versions must be plainly marked as such, and must not be
|
||||
misrepresented as being the original software.
|
||||
|
||||
3. This notice may not be removed or altered from any source
|
||||
distribution.
|
||||
*/
|
||||
|
||||
/*
|
||||
Extra C++ utilities for LodePNG, for convenience.
|
||||
*/
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
#include "lodepng.h"
|
||||
|
||||
#pragma once
|
||||
|
||||
namespace lodepng
|
||||
{
|
||||
|
||||
/*
|
||||
Returns info from the header of the PNG by value, purely for convenience.
|
||||
Does NOT check for errors. Returns bogus info if the PNG has an error.
|
||||
Does not require cleanup of allocated memory because no palette or text chunk
|
||||
info is in the LodePNGInfo object after checking only the header of the PNG.
|
||||
*/
|
||||
LodePNGInfo getPNGHeaderInfo(const std::vector<unsigned char>& png);
|
||||
|
||||
/*
|
||||
Get the names and sizes of all chunks in the PNG file.
|
||||
Returns 0 if ok, non-0 if error happened.
|
||||
*/
|
||||
unsigned getChunkInfo(std::vector<std::string>& names, std::vector<size_t>& sizes,
|
||||
const std::vector<unsigned char>& png);
|
||||
|
||||
/*
|
||||
Returns the names and full chunks (including the name and everything else that
|
||||
makes up the chunk) for all chunks except IHDR, PLTE, IDAT and IEND.
|
||||
It separates the chunks into 3 separate lists, representing the chunks between
|
||||
certain critical chunks: 0: IHDR-PLTE, 1: PLTE-IDAT, 2: IDAT-IEND
|
||||
Returns 0 if ok, non-0 if error happened.
|
||||
*/
|
||||
unsigned getChunks(std::vector<std::string> names[3],
|
||||
std::vector<std::vector<unsigned char> > chunks[3],
|
||||
const std::vector<unsigned char>& png);
|
||||
|
||||
/*
|
||||
Inserts chunks into the given png file. The chunks must be fully encoded,
|
||||
including length, type, content and CRC.
|
||||
The array index determines where it goes:
|
||||
0: between IHDR and PLTE, 1: between PLTE and IDAT, 2: between IDAT and IEND.
|
||||
They're appended at the end of those locations within the PNG.
|
||||
Returns 0 if ok, non-0 if error happened.
|
||||
*/
|
||||
unsigned insertChunks(std::vector<unsigned char>& png,
|
||||
const std::vector<std::vector<unsigned char> > chunks[3]);
|
||||
|
||||
/*
|
||||
Get the filtertypes of each scanline in this PNG file.
|
||||
Returns 0 if ok, 1 if PNG decoding error happened.
|
||||
|
||||
For a non-interlaced PNG, it returns one filtertype per scanline, in order.
|
||||
|
||||
For interlaced PNGs, it returns a result as if it's not interlaced. It returns
|
||||
one filtertype per scanline, in order. The values match pass 6 and 7 of the
|
||||
Adam7 interlacing, alternating between the two, so that the values correspond
|
||||
the most to their scanlines.
|
||||
*/
|
||||
unsigned getFilterTypes(std::vector<unsigned char>& filterTypes, const std::vector<unsigned char>& png);
|
||||
|
||||
/*
|
||||
Get the filtertypes of each scanline in every interlace pass this PNG file.
|
||||
Returns 0 if ok, 1 if PNG decoding error happened.
|
||||
|
||||
For a non-interlaced PNG, it returns one filtertype per scanline, in order, in
|
||||
a single std::vector in filterTypes.
|
||||
|
||||
For an interlaced PNG, it returns 7 std::vectors in filterTypes, one for each
|
||||
Adam7 pass. The amount of values per pass can be calculated as follows, where
|
||||
w and h are the size of the image and all divisions are integer divisions:
|
||||
pass 1: (h + 7) / 8
|
||||
pass 2: w <= 4 ? 0 : (h + 7) / 8
|
||||
pass 3: h <= 4 ? 0 : (h + 7) / 8
|
||||
pass 4: w <= 2 ? 0 : (h + 3) / 4
|
||||
pass 5: h <= 2 ? 0 : (h + 3) / 4
|
||||
pass 6: w <= 1 ? 0 : (h + 1) / 2
|
||||
pass 7: h <= 1 ? 0 : (h + 1) / 2
|
||||
*/
|
||||
unsigned getFilterTypesInterlaced(std::vector<std::vector<unsigned char> >& filterTypes,
|
||||
const std::vector<unsigned char>& png);
|
||||
|
||||
/*
|
||||
Returns the value of the i-th pixel in an image with 1, 2, 4 or 8-bit color.
|
||||
E.g. if bits is 4 and i is 5, it returns the 5th nibble (4-bit group), which
|
||||
is the second half of the 3th byte, in big endian (PNG's endian order).
|
||||
*/
|
||||
int getPaletteValue(const unsigned char* data, size_t i, int bits);
|
||||
|
||||
/*
|
||||
The information for extractZlibInfo.
|
||||
*/
|
||||
struct ZlibBlockInfo
|
||||
{
|
||||
int btype; //block type (0-2)
|
||||
size_t compressedbits; //size of compressed block in bits
|
||||
size_t uncompressedbytes; //size of uncompressed block in bytes
|
||||
|
||||
// only filled in for block type 2
|
||||
size_t treebits; //encoded tree size in bits
|
||||
int hlit; //the HLIT value that was filled in for this tree
|
||||
int hdist; //the HDIST value that was filled in for this tree
|
||||
int hclen; //the HCLEN value that was filled in for this tree
|
||||
std::vector<int> clcl; //19 code length code lengths (compressed tree's tree)
|
||||
std::vector<int> treecodes; //N tree codes, with values 0-18. Values 17 or 18 are followed by the repetition value.
|
||||
std::vector<int> litlenlengths; //288 code lengths for lit/len symbols
|
||||
std::vector<int> distlengths; //32 code lengths for dist symbols
|
||||
|
||||
// only filled in for block types 1 or 2
|
||||
std::vector<int> lz77_lcode; //LZ77 codes. 0-255: literals. 256: end symbol. 257-285: length code of length/dist pairs
|
||||
// the next vectors have the same size as lz77_lcode, but an element only has meaningful value if lz77_lcode contains a length code.
|
||||
std::vector<int> lz77_dcode;
|
||||
std::vector<int> lz77_lbits;
|
||||
std::vector<int> lz77_dbits;
|
||||
std::vector<int> lz77_lvalue;
|
||||
std::vector<int> lz77_dvalue;
|
||||
size_t numlit; //number of lit codes in this block
|
||||
size_t numlen; //number of len codes in this block
|
||||
};
|
||||
|
||||
//Extracts all info needed from a PNG file to reconstruct the zlib compression exactly.
|
||||
void extractZlibInfo(std::vector<ZlibBlockInfo>& zlibinfo, const std::vector<unsigned char>& in);
|
||||
|
||||
} // namespace lodepng
|
||||
471
build/node_modules/node-zopfli/zopfli/src/zopflipng/zopflipng_bin.cc
generated
vendored
Normal file
471
build/node_modules/node-zopfli/zopfli/src/zopflipng/zopflipng_bin.cc
generated
vendored
Normal file
@@ -0,0 +1,471 @@
|
||||
// Copyright 2013 Google Inc. All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
// Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
|
||||
// Command line tool to recompress and optimize PNG images, using zopflipng_lib.
|
||||
|
||||
#include <stdlib.h>
|
||||
#include <stdio.h>
|
||||
|
||||
#include "lodepng/lodepng.h"
|
||||
#include "lodepng/lodepng_util.h"
|
||||
#include "zopflipng_lib.h"
|
||||
|
||||
// Returns directory path (including last slash) in dir, filename without
|
||||
// extension in file, extension (including the dot) in ext
|
||||
void GetFileNameParts(const std::string& filename,
|
||||
std::string* dir, std::string* file, std::string* ext) {
|
||||
size_t npos = (size_t)(-1);
|
||||
size_t slashpos = filename.find_last_of("/\\");
|
||||
std::string nodir;
|
||||
if (slashpos == npos) {
|
||||
*dir = "";
|
||||
nodir = filename;
|
||||
} else {
|
||||
*dir = filename.substr(0, slashpos + 1);
|
||||
nodir = filename.substr(slashpos + 1);
|
||||
}
|
||||
size_t dotpos = nodir.find_last_of('.');
|
||||
if (dotpos == (size_t)(-1)) {
|
||||
*file = nodir;
|
||||
*ext = "";
|
||||
} else {
|
||||
*file = nodir.substr(0, dotpos);
|
||||
*ext = nodir.substr(dotpos);
|
||||
}
|
||||
}
|
||||
|
||||
// Returns whether the file exists and we have read permissions.
|
||||
bool FileExists(const std::string& filename) {
|
||||
FILE* file = fopen(filename.c_str(), "rb");
|
||||
if (file) {
|
||||
fclose(file);
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
// Returns the size of the file, if it exists and we have read permissions.
|
||||
size_t GetFileSize(const std::string& filename) {
|
||||
size_t size;
|
||||
FILE* file = fopen(filename.c_str(), "rb");
|
||||
if (!file) return 0;
|
||||
fseek(file , 0 , SEEK_END);
|
||||
size = static_cast<size_t>(ftell(file));
|
||||
fclose(file);
|
||||
return size;
|
||||
}
|
||||
|
||||
void ShowHelp() {
|
||||
printf("ZopfliPNG, a Portable Network Graphics (PNG) image optimizer.\n"
|
||||
"\n"
|
||||
"Usage: zopflipng [options]... infile.png outfile.png\n"
|
||||
" zopflipng [options]... --prefix=[fileprefix] [files.png]...\n"
|
||||
"\n"
|
||||
"If the output file exists, it is considered a result from a"
|
||||
" previous run and not overwritten if its filesize is smaller.\n"
|
||||
"\n"
|
||||
"Options:\n"
|
||||
"-m: compress more: use more iterations (depending on file size)\n"
|
||||
"--prefix=[fileprefix]: Adds a prefix to output filenames. May also"
|
||||
" contain a directory path. When using a prefix, multiple input files"
|
||||
" can be given and the output filenames are generated with the"
|
||||
" prefix\n"
|
||||
" If --prefix is specified without value, 'zopfli_' is used.\n"
|
||||
" If input file names contain the prefix, they are not processed but"
|
||||
" considered as output from previous runs. This is handy when using"
|
||||
" *.png wildcard expansion with multiple runs.\n"
|
||||
"-y: do not ask about overwriting files.\n"
|
||||
"--lossy_transparent: remove colors behind alpha channel 0. No visual"
|
||||
" difference, removes hidden information.\n"
|
||||
"--lossy_8bit: convert 16-bit per channel image to 8-bit per"
|
||||
" channel.\n"
|
||||
"-d: dry run: don't save any files, just see the console output"
|
||||
" (e.g. for benchmarking)\n"
|
||||
"--always_zopflify: always output the image encoded by Zopfli, even if"
|
||||
" it's bigger than the original, for benchmarking the algorithm. Not"
|
||||
" good for real optimization.\n"
|
||||
"-q: use quick, but not very good, compression"
|
||||
" (e.g. for only trying the PNG filter and color types)\n"
|
||||
"--iterations=[number]: number of iterations, more iterations makes it"
|
||||
" slower but provides slightly better compression. Default: 15 for"
|
||||
" small files, 5 for large files.\n"
|
||||
"--splitting=[0-3]: ignored, left for backwards compatibility\n"
|
||||
"--filters=[types]: filter strategies to try:\n"
|
||||
" 0-4: give all scanlines PNG filter type 0-4\n"
|
||||
" m: minimum sum\n"
|
||||
" e: entropy\n"
|
||||
" p: predefined (keep from input, this likely overlaps another"
|
||||
" strategy)\n"
|
||||
" b: brute force (experimental)\n"
|
||||
" By default, if this argument is not given, one that is most likely"
|
||||
" the best for this image is chosen by trying faster compression with"
|
||||
" each type.\n"
|
||||
" If this argument is used, all given filter types"
|
||||
" are tried with slow compression and the best result retained. A good"
|
||||
" set of filters to try is --filters=0me.\n"
|
||||
"--keepchunks=nAME,nAME,...: keep metadata chunks with these names"
|
||||
" that would normally be removed, e.g. tEXt,zTXt,iTXt,gAMA, ... \n"
|
||||
" Due to adding extra data, this increases the result size. Keeping"
|
||||
" bKGD or sBIT chunks may cause additional worse compression due to"
|
||||
" forcing a certain color type, it is advised to not keep these for"
|
||||
" web images because web browsers do not use these chunks. By default"
|
||||
" ZopfliPNG only keeps (and losslessly modifies) the following chunks"
|
||||
" because they are essential: IHDR, PLTE, tRNS, IDAT and IEND.\n"
|
||||
"\n"
|
||||
"Usage examples:\n"
|
||||
"Optimize a file and overwrite if smaller: zopflipng infile.png"
|
||||
" outfile.png\n"
|
||||
"Compress more: zopflipng -m infile.png outfile.png\n"
|
||||
"Optimize multiple files: zopflipng --prefix a.png b.png c.png\n"
|
||||
"Compress really good and trying all filter strategies: zopflipng"
|
||||
" --iterations=500 --filters=01234mepb --lossy_8bit"
|
||||
" --lossy_transparent infile.png outfile.png\n");
|
||||
}
|
||||
|
||||
void PrintSize(const char* label, size_t size) {
|
||||
printf("%s: %d (%dK)\n", label, (int) size, (int) size / 1024);
|
||||
}
|
||||
|
||||
void PrintResultSize(const char* label, size_t oldsize, size_t newsize) {
|
||||
printf("%s: %d (%dK). Percentage of original: %.3f%%\n",
|
||||
label, (int) newsize, (int) newsize / 1024, newsize * 100.0 / oldsize);
|
||||
}
|
||||
|
||||
int main(int argc, char *argv[]) {
|
||||
if (argc < 2) {
|
||||
ShowHelp();
|
||||
return 0;
|
||||
}
|
||||
|
||||
ZopfliPNGOptions png_options;
|
||||
|
||||
// cmd line options
|
||||
bool always_zopflify = false; // overwrite file even if we have bigger result
|
||||
bool yes = false; // do not ask to overwrite files
|
||||
bool dryrun = false; // never save anything
|
||||
|
||||
std::string user_out_filename; // output filename if no prefix is used
|
||||
bool use_prefix = false;
|
||||
std::string prefix = "zopfli_"; // prefix for output filenames
|
||||
|
||||
std::vector<std::string> files;
|
||||
for (int i = 1; i < argc; i++) {
|
||||
std::string arg = argv[i];
|
||||
if (arg[0] == '-' && arg.size() > 1 && arg[1] != '-') {
|
||||
for (size_t pos = 1; pos < arg.size(); pos++) {
|
||||
char c = arg[pos];
|
||||
if (c == 'y') {
|
||||
yes = true;
|
||||
} else if (c == 'd') {
|
||||
dryrun = true;
|
||||
} else if (c == 'm') {
|
||||
png_options.num_iterations *= 4;
|
||||
png_options.num_iterations_large *= 4;
|
||||
} else if (c == 'q') {
|
||||
png_options.use_zopfli = false;
|
||||
} else if (c == 'h') {
|
||||
ShowHelp();
|
||||
return 0;
|
||||
} else {
|
||||
printf("Unknown flag: %c\n", c);
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
} else if (arg[0] == '-' && arg.size() > 1 && arg[1] == '-') {
|
||||
size_t eq = arg.find('=');
|
||||
std::string name = arg.substr(0, eq);
|
||||
std::string value = eq >= arg.size() - 1 ? "" : arg.substr(eq + 1);
|
||||
int num = atoi(value.c_str());
|
||||
if (name == "--always_zopflify") {
|
||||
always_zopflify = true;
|
||||
} else if (name == "--verbose") {
|
||||
png_options.verbose = true;
|
||||
} else if (name == "--lossy_transparent") {
|
||||
png_options.lossy_transparent = true;
|
||||
} else if (name == "--lossy_8bit") {
|
||||
png_options.lossy_8bit = true;
|
||||
} else if (name == "--iterations") {
|
||||
if (num < 1) num = 1;
|
||||
png_options.num_iterations = num;
|
||||
png_options.num_iterations_large = num;
|
||||
} else if (name == "--splitting") {
|
||||
// ignored
|
||||
} else if (name == "--filters") {
|
||||
for (size_t j = 0; j < value.size(); j++) {
|
||||
ZopfliPNGFilterStrategy strategy = kStrategyZero;
|
||||
char f = value[j];
|
||||
switch (f) {
|
||||
case '0': strategy = kStrategyZero; break;
|
||||
case '1': strategy = kStrategyOne; break;
|
||||
case '2': strategy = kStrategyTwo; break;
|
||||
case '3': strategy = kStrategyThree; break;
|
||||
case '4': strategy = kStrategyFour; break;
|
||||
case 'm': strategy = kStrategyMinSum; break;
|
||||
case 'e': strategy = kStrategyEntropy; break;
|
||||
case 'p': strategy = kStrategyPredefined; break;
|
||||
case 'b': strategy = kStrategyBruteForce; break;
|
||||
default:
|
||||
printf("Unknown filter strategy: %c\n", f);
|
||||
return 1;
|
||||
}
|
||||
png_options.filter_strategies.push_back(strategy);
|
||||
// Enable auto filter strategy only if no user-specified filter is
|
||||
// given.
|
||||
png_options.auto_filter_strategy = false;
|
||||
}
|
||||
} else if (name == "--keepchunks") {
|
||||
bool correct = true;
|
||||
if ((value.size() + 1) % 5 != 0) correct = false;
|
||||
for (size_t i = 0; i + 4 <= value.size() && correct; i += 5) {
|
||||
png_options.keepchunks.push_back(value.substr(i, 4));
|
||||
if (i > 4 && value[i - 1] != ',') correct = false;
|
||||
}
|
||||
if (!correct) {
|
||||
printf("Error: keepchunks format must be like for example:\n"
|
||||
" --keepchunks=gAMA,cHRM,sRGB,iCCP\n");
|
||||
return 0;
|
||||
}
|
||||
} else if (name == "--prefix") {
|
||||
use_prefix = true;
|
||||
if (!value.empty()) prefix = value;
|
||||
} else if (name == "--help") {
|
||||
ShowHelp();
|
||||
return 0;
|
||||
} else {
|
||||
printf("Unknown flag: %s\n", name.c_str());
|
||||
return 0;
|
||||
}
|
||||
} else {
|
||||
files.push_back(argv[i]);
|
||||
}
|
||||
}
|
||||
|
||||
if (!use_prefix) {
|
||||
if (files.size() == 2) {
|
||||
// The second filename is the output instead of an input if no prefix is
|
||||
// given.
|
||||
user_out_filename = files[1];
|
||||
files.resize(1);
|
||||
} else {
|
||||
printf("Please provide one input and output filename\n\n");
|
||||
ShowHelp();
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
size_t total_in_size = 0;
|
||||
// Total output size, taking input size if the input file was smaller
|
||||
size_t total_out_size = 0;
|
||||
// Total output size that zopfli produced, even if input was smaller, for
|
||||
// benchmark information
|
||||
size_t total_out_size_zopfli = 0;
|
||||
size_t total_errors = 0;
|
||||
size_t total_files = 0;
|
||||
size_t total_files_smaller = 0;
|
||||
size_t total_files_saved = 0;
|
||||
size_t total_files_equal = 0;
|
||||
|
||||
for (size_t i = 0; i < files.size(); i++) {
|
||||
if (use_prefix && files.size() > 1) {
|
||||
std::string dir, file, ext;
|
||||
GetFileNameParts(files[i], &dir, &file, &ext);
|
||||
// avoid doing filenames which were already output by this so that you
|
||||
// don't get zopfli_zopfli_zopfli_... files after multiple runs.
|
||||
if (file.find(prefix) == 0) continue;
|
||||
}
|
||||
|
||||
total_files++;
|
||||
|
||||
printf("Optimizing %s\n", files[i].c_str());
|
||||
std::vector<unsigned char> image;
|
||||
unsigned w, h;
|
||||
std::vector<unsigned char> origpng;
|
||||
unsigned error;
|
||||
lodepng::State inputstate;
|
||||
std::vector<unsigned char> resultpng;
|
||||
|
||||
error = lodepng::load_file(origpng, files[i]);
|
||||
if (!error) {
|
||||
error = ZopfliPNGOptimize(origpng, png_options,
|
||||
png_options.verbose, &resultpng);
|
||||
}
|
||||
|
||||
if (error) {
|
||||
if (error == 1) {
|
||||
printf("Decoding error\n");
|
||||
} else {
|
||||
printf("Decoding error %u: %s\n", error, lodepng_error_text(error));
|
||||
}
|
||||
}
|
||||
|
||||
// Verify result, check that the result causes no decoding errors
|
||||
if (!error) {
|
||||
error = lodepng::decode(image, w, h, resultpng);
|
||||
if (!error) {
|
||||
std::vector<unsigned char> origimage;
|
||||
unsigned origw, origh;
|
||||
lodepng::decode(origimage, origw, origh, origpng);
|
||||
if (origw != w || origh != h || origimage.size() != image.size()) {
|
||||
error = 1;
|
||||
} else {
|
||||
for (size_t i = 0; i < image.size(); i += 4) {
|
||||
bool same_alpha = image[i + 3] == origimage[i + 3];
|
||||
bool same_rgb =
|
||||
(png_options.lossy_transparent && image[i + 3] == 0) ||
|
||||
(image[i + 0] == origimage[i + 0] &&
|
||||
image[i + 1] == origimage[i + 1] &&
|
||||
image[i + 2] == origimage[i + 2]);
|
||||
if (!same_alpha || !same_rgb) {
|
||||
error = 1;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if (error) {
|
||||
printf("Error: verification of result failed, keeping original."
|
||||
" Error: %u.\n", error);
|
||||
// Reset the error to 0, instead set output back to the original. The
|
||||
// input PNG is valid, zopfli failed on it so treat as if it could not
|
||||
// make it smaller.
|
||||
error = 0;
|
||||
resultpng = origpng;
|
||||
}
|
||||
}
|
||||
|
||||
if (error) {
|
||||
total_errors++;
|
||||
} else {
|
||||
size_t origsize = origpng.size();
|
||||
size_t resultsize = resultpng.size();
|
||||
|
||||
if (!png_options.keepchunks.empty()) {
|
||||
std::vector<std::string> names;
|
||||
std::vector<size_t> sizes;
|
||||
lodepng::getChunkInfo(names, sizes, resultpng);
|
||||
for (size_t i = 0; i < names.size(); i++) {
|
||||
if (names[i] == "bKGD" || names[i] == "sBIT") {
|
||||
printf("Forced to keep original color type due to keeping bKGD or"
|
||||
" sBIT chunk. Try without --keepchunks for better"
|
||||
" compression.\n");
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
PrintSize("Input size", origsize);
|
||||
PrintResultSize("Result size", origsize, resultsize);
|
||||
if (resultsize < origsize) {
|
||||
printf("Result is smaller\n");
|
||||
} else if (resultsize == origsize) {
|
||||
printf("Result has exact same size\n");
|
||||
} else {
|
||||
printf(always_zopflify
|
||||
? "Original was smaller\n"
|
||||
: "Preserving original PNG since it was smaller\n");
|
||||
}
|
||||
|
||||
std::string out_filename = user_out_filename;
|
||||
if (use_prefix) {
|
||||
std::string dir, file, ext;
|
||||
GetFileNameParts(files[i], &dir, &file, &ext);
|
||||
out_filename = dir + prefix + file + ext;
|
||||
}
|
||||
bool different_output_name = out_filename != files[i];
|
||||
|
||||
total_in_size += origsize;
|
||||
total_out_size_zopfli += resultpng.size();
|
||||
if (resultpng.size() < origsize) total_files_smaller++;
|
||||
else if (resultpng.size() == origsize) total_files_equal++;
|
||||
|
||||
if (!always_zopflify && resultpng.size() >= origsize) {
|
||||
// Set output file to input since zopfli didn't improve it.
|
||||
resultpng = origpng;
|
||||
}
|
||||
|
||||
bool already_exists = FileExists(out_filename);
|
||||
size_t origoutfilesize = GetFileSize(out_filename);
|
||||
|
||||
// When using a prefix, and the output file already exist, assume it's
|
||||
// from a previous run. If that file is smaller, it may represent a
|
||||
// previous run with different parameters that gave a smaller PNG image.
|
||||
// This also applies when not using prefix but same input as output file.
|
||||
// In that case, do not overwrite it. This behaviour can be removed by
|
||||
// adding the always_zopflify flag.
|
||||
bool keep_earlier_output_file = already_exists &&
|
||||
resultpng.size() >= origoutfilesize && !always_zopflify &&
|
||||
(use_prefix || !different_output_name);
|
||||
|
||||
if (keep_earlier_output_file) {
|
||||
// An output file from a previous run is kept, add that files' size
|
||||
// to the output size statistics.
|
||||
total_out_size += origoutfilesize;
|
||||
if (use_prefix) {
|
||||
printf(resultpng.size() == origoutfilesize
|
||||
? "File not written because a previous run was as good.\n"
|
||||
: "File not written because a previous run was better.\n");
|
||||
}
|
||||
} else {
|
||||
bool confirmed = true;
|
||||
if (!yes && !dryrun && already_exists) {
|
||||
printf("File %s exists, overwrite? (y/N) ", out_filename.c_str());
|
||||
char answer = 0;
|
||||
// Read the first character, the others and enter with getchar.
|
||||
while (int input = getchar()) {
|
||||
if (input == '\n' || input == EOF) break;
|
||||
else if (!answer) answer = input;
|
||||
}
|
||||
confirmed = answer == 'y' || answer == 'Y';
|
||||
}
|
||||
if (confirmed) {
|
||||
if (!dryrun) {
|
||||
if (lodepng::save_file(resultpng, out_filename) != 0) {
|
||||
printf("Failed to write to file %s\n", out_filename.c_str());
|
||||
} else {
|
||||
total_files_saved++;
|
||||
}
|
||||
}
|
||||
total_out_size += resultpng.size();
|
||||
} else {
|
||||
// An output file from a previous run is kept, add that files' size
|
||||
// to the output size statistics.
|
||||
total_out_size += origoutfilesize;
|
||||
}
|
||||
}
|
||||
}
|
||||
printf("\n");
|
||||
}
|
||||
|
||||
if (total_files > 1) {
|
||||
printf("Summary for all files:\n");
|
||||
printf("Files tried: %d\n", (int) total_files);
|
||||
printf("Files smaller: %d\n", (int) total_files_smaller);
|
||||
if (total_files_equal) {
|
||||
printf("Files equal: %d\n", (int) total_files_equal);
|
||||
}
|
||||
printf("Files saved: %d\n", (int) total_files_saved);
|
||||
if (total_errors) printf("Errors: %d\n", (int) total_errors);
|
||||
PrintSize("Total input size", total_in_size);
|
||||
PrintResultSize("Total output size", total_in_size, total_out_size);
|
||||
PrintResultSize("Benchmark result size",
|
||||
total_in_size, total_out_size_zopfli);
|
||||
}
|
||||
|
||||
if (dryrun) printf("No files were written because dry run was specified\n");
|
||||
|
||||
return total_errors;
|
||||
}
|
||||
525
build/node_modules/node-zopfli/zopfli/src/zopflipng/zopflipng_lib.cc
generated
vendored
Normal file
525
build/node_modules/node-zopfli/zopfli/src/zopflipng/zopflipng_lib.cc
generated
vendored
Normal file
@@ -0,0 +1,525 @@
|
||||
// Copyright 2013 Google Inc. All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
// Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
|
||||
// See zopflipng_lib.h
|
||||
|
||||
#include "zopflipng_lib.h"
|
||||
|
||||
#include <errno.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <set>
|
||||
#include <vector>
|
||||
|
||||
#include "lodepng/lodepng.h"
|
||||
#include "lodepng/lodepng_util.h"
|
||||
#include "../zopfli/deflate.h"
|
||||
|
||||
ZopfliPNGOptions::ZopfliPNGOptions()
|
||||
: verbose(false)
|
||||
, lossy_transparent(false)
|
||||
, lossy_8bit(false)
|
||||
, auto_filter_strategy(true)
|
||||
, use_zopfli(true)
|
||||
, num_iterations(15)
|
||||
, num_iterations_large(5)
|
||||
, block_split_strategy(1) {
|
||||
}
|
||||
|
||||
// Deflate compressor passed as fuction pointer to LodePNG to have it use Zopfli
|
||||
// as its compression backend.
|
||||
unsigned CustomPNGDeflate(unsigned char** out, size_t* outsize,
|
||||
const unsigned char* in, size_t insize,
|
||||
const LodePNGCompressSettings* settings) {
|
||||
const ZopfliPNGOptions* png_options =
|
||||
static_cast<const ZopfliPNGOptions*>(settings->custom_context);
|
||||
unsigned char bp = 0;
|
||||
ZopfliOptions options;
|
||||
ZopfliInitOptions(&options);
|
||||
|
||||
options.verbose = png_options->verbose;
|
||||
options.numiterations = insize < 200000
|
||||
? png_options->num_iterations : png_options->num_iterations_large;
|
||||
|
||||
ZopfliDeflate(&options, 2 /* Dynamic */, 1, in, insize, &bp, out, outsize);
|
||||
|
||||
return 0; // OK
|
||||
}
|
||||
|
||||
// Returns 32-bit integer value for RGBA color.
|
||||
static unsigned ColorIndex(const unsigned char* color) {
|
||||
return color[0] + 256u * color[1] + 65536u * color[2] + 16777216u * color[3];
|
||||
}
|
||||
|
||||
// Counts amount of colors in the image, up to 257. If transparent_counts_as_one
|
||||
// is enabled, any color with alpha channel 0 is treated as a single color with
|
||||
// index 0.
|
||||
void CountColors(std::set<unsigned>* unique,
|
||||
const unsigned char* image, unsigned w, unsigned h,
|
||||
bool transparent_counts_as_one) {
|
||||
unique->clear();
|
||||
for (size_t i = 0; i < w * h; i++) {
|
||||
unsigned index = ColorIndex(&image[i * 4]);
|
||||
if (transparent_counts_as_one && image[i * 4 + 3] == 0) index = 0;
|
||||
unique->insert(index);
|
||||
if (unique->size() > 256) break;
|
||||
}
|
||||
}
|
||||
|
||||
// Remove RGB information from pixels with alpha=0
|
||||
void LossyOptimizeTransparent(lodepng::State* inputstate, unsigned char* image,
|
||||
unsigned w, unsigned h) {
|
||||
// First check if we want to preserve potential color-key background color,
|
||||
// or instead use the last encountered RGB value all the time to save bytes.
|
||||
bool key = true;
|
||||
for (size_t i = 0; i < w * h; i++) {
|
||||
if (image[i * 4 + 3] > 0 && image[i * 4 + 3] < 255) {
|
||||
key = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
std::set<unsigned> count; // Color count, up to 257.
|
||||
CountColors(&count, image, w, h, true);
|
||||
// If true, means palette is possible so avoid using different RGB values for
|
||||
// the transparent color.
|
||||
bool palette = count.size() <= 256;
|
||||
|
||||
// Choose the color key or first initial background color.
|
||||
int r = 0, g = 0, b = 0;
|
||||
if (key || palette) {
|
||||
for (size_t i = 0; i < w * h; i++) {
|
||||
if (image[i * 4 + 3] == 0) {
|
||||
// Use RGB value of first encountered transparent pixel. This can be
|
||||
// used as a valid color key, or in case of palette ensures a color
|
||||
// existing in the input image palette is used.
|
||||
r = image[i * 4 + 0];
|
||||
g = image[i * 4 + 1];
|
||||
b = image[i * 4 + 2];
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (size_t i = 0; i < w * h; i++) {
|
||||
// if alpha is 0, alter the RGB value to a possibly more efficient one.
|
||||
if (image[i * 4 + 3] == 0) {
|
||||
image[i * 4 + 0] = r;
|
||||
image[i * 4 + 1] = g;
|
||||
image[i * 4 + 2] = b;
|
||||
} else {
|
||||
if (!key && !palette) {
|
||||
// Use the last encountered RGB value if no key or palette is used: that
|
||||
// way more values can be 0 thanks to the PNG filter types.
|
||||
r = image[i * 4 + 0];
|
||||
g = image[i * 4 + 1];
|
||||
b = image[i * 4 + 2];
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// If there are now less colors, update palette of input image to match this.
|
||||
if (palette && inputstate->info_png.color.palettesize > 0) {
|
||||
CountColors(&count, image, w, h, false);
|
||||
if (count.size() < inputstate->info_png.color.palettesize) {
|
||||
std::vector<unsigned char> palette_out;
|
||||
unsigned char* palette_in = inputstate->info_png.color.palette;
|
||||
for (size_t i = 0; i < inputstate->info_png.color.palettesize; i++) {
|
||||
if (count.count(ColorIndex(&palette_in[i * 4])) != 0) {
|
||||
palette_out.push_back(palette_in[i * 4 + 0]);
|
||||
palette_out.push_back(palette_in[i * 4 + 1]);
|
||||
palette_out.push_back(palette_in[i * 4 + 2]);
|
||||
palette_out.push_back(palette_in[i * 4 + 3]);
|
||||
}
|
||||
}
|
||||
inputstate->info_png.color.palettesize = palette_out.size() / 4;
|
||||
for (size_t i = 0; i < palette_out.size(); i++) {
|
||||
palette_in[i] = palette_out[i];
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Tries to optimize given a single PNG filter strategy.
|
||||
// Returns 0 if ok, other value for error
|
||||
unsigned TryOptimize(
|
||||
const std::vector<unsigned char>& image, unsigned w, unsigned h,
|
||||
const lodepng::State& inputstate, bool bit16, bool keep_colortype,
|
||||
const std::vector<unsigned char>& origfile,
|
||||
ZopfliPNGFilterStrategy filterstrategy,
|
||||
bool use_zopfli, int windowsize, const ZopfliPNGOptions* png_options,
|
||||
std::vector<unsigned char>* out) {
|
||||
unsigned error = 0;
|
||||
|
||||
lodepng::State state;
|
||||
state.encoder.zlibsettings.windowsize = windowsize;
|
||||
if (use_zopfli && png_options->use_zopfli) {
|
||||
state.encoder.zlibsettings.custom_deflate = CustomPNGDeflate;
|
||||
state.encoder.zlibsettings.custom_context = png_options;
|
||||
}
|
||||
|
||||
if (keep_colortype) {
|
||||
state.encoder.auto_convert = 0;
|
||||
lodepng_color_mode_copy(&state.info_png.color, &inputstate.info_png.color);
|
||||
}
|
||||
if (inputstate.info_png.color.colortype == LCT_PALETTE) {
|
||||
// Make it preserve the original palette order
|
||||
lodepng_color_mode_copy(&state.info_raw, &inputstate.info_png.color);
|
||||
state.info_raw.colortype = LCT_RGBA;
|
||||
state.info_raw.bitdepth = 8;
|
||||
}
|
||||
if (bit16) {
|
||||
state.info_raw.bitdepth = 16;
|
||||
}
|
||||
|
||||
state.encoder.filter_palette_zero = 0;
|
||||
|
||||
std::vector<unsigned char> filters;
|
||||
switch (filterstrategy) {
|
||||
case kStrategyZero:
|
||||
state.encoder.filter_strategy = LFS_ZERO;
|
||||
break;
|
||||
case kStrategyMinSum:
|
||||
state.encoder.filter_strategy = LFS_MINSUM;
|
||||
break;
|
||||
case kStrategyEntropy:
|
||||
state.encoder.filter_strategy = LFS_ENTROPY;
|
||||
break;
|
||||
case kStrategyBruteForce:
|
||||
state.encoder.filter_strategy = LFS_BRUTE_FORCE;
|
||||
break;
|
||||
case kStrategyOne:
|
||||
case kStrategyTwo:
|
||||
case kStrategyThree:
|
||||
case kStrategyFour:
|
||||
// Set the filters of all scanlines to that number.
|
||||
filters.resize(h, filterstrategy);
|
||||
state.encoder.filter_strategy = LFS_PREDEFINED;
|
||||
state.encoder.predefined_filters = &filters[0];
|
||||
break;
|
||||
case kStrategyPredefined:
|
||||
lodepng::getFilterTypes(filters, origfile);
|
||||
if (filters.size() != h) return 1; // Error getting filters
|
||||
state.encoder.filter_strategy = LFS_PREDEFINED;
|
||||
state.encoder.predefined_filters = &filters[0];
|
||||
break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
||||
state.encoder.add_id = false;
|
||||
state.encoder.text_compression = 1;
|
||||
|
||||
error = lodepng::encode(*out, image, w, h, state);
|
||||
|
||||
// For very small output, also try without palette, it may be smaller thanks
|
||||
// to no palette storage overhead.
|
||||
if (!error && out->size() < 4096 && !keep_colortype) {
|
||||
lodepng::State teststate;
|
||||
std::vector<unsigned char> temp;
|
||||
lodepng::decode(temp, w, h, teststate, *out);
|
||||
if (teststate.info_png.color.colortype == LCT_PALETTE) {
|
||||
LodePNGColorProfile profile;
|
||||
lodepng_color_profile_init(&profile);
|
||||
lodepng_get_color_profile(&profile, &image[0], w, h, &state.info_raw);
|
||||
// Too small for tRNS chunk overhead.
|
||||
if (w * h <= 16 && profile.key) profile.alpha = 1;
|
||||
state.encoder.auto_convert = 0;
|
||||
state.info_png.color.colortype = (profile.alpha ? LCT_RGBA : LCT_RGB);
|
||||
state.info_png.color.bitdepth = 8;
|
||||
state.info_png.color.key_defined = (profile.key && !profile.alpha);
|
||||
if (state.info_png.color.key_defined) {
|
||||
state.info_png.color.key_defined = 1;
|
||||
state.info_png.color.key_r = (profile.key_r & 255u);
|
||||
state.info_png.color.key_g = (profile.key_g & 255u);
|
||||
state.info_png.color.key_b = (profile.key_b & 255u);
|
||||
}
|
||||
|
||||
std::vector<unsigned char> out2;
|
||||
error = lodepng::encode(out2, image, w, h, state);
|
||||
if (out2.size() < out->size()) out->swap(out2);
|
||||
}
|
||||
}
|
||||
|
||||
if (error) {
|
||||
printf("Encoding error %u: %s\n", error, lodepng_error_text(error));
|
||||
return error;
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Use fast compression to check which PNG filter strategy gives the smallest
|
||||
// output. This allows to then do the slow and good compression only on that
|
||||
// filter type.
|
||||
unsigned AutoChooseFilterStrategy(const std::vector<unsigned char>& image,
|
||||
unsigned w, unsigned h,
|
||||
const lodepng::State& inputstate,
|
||||
bool bit16, bool keep_colortype,
|
||||
const std::vector<unsigned char>& origfile,
|
||||
int numstrategies,
|
||||
ZopfliPNGFilterStrategy* strategies,
|
||||
bool* enable) {
|
||||
std::vector<unsigned char> out;
|
||||
size_t bestsize = 0;
|
||||
int bestfilter = 0;
|
||||
|
||||
// A large window size should still be used to do the quick compression to
|
||||
// try out filter strategies: which filter strategy is the best depends
|
||||
// largely on the window size, the closer to the actual used window size the
|
||||
// better.
|
||||
int windowsize = 8192;
|
||||
|
||||
for (int i = 0; i < numstrategies; i++) {
|
||||
out.clear();
|
||||
unsigned error = TryOptimize(image, w, h, inputstate, bit16, keep_colortype,
|
||||
origfile, strategies[i], false, windowsize, 0,
|
||||
&out);
|
||||
if (error) return error;
|
||||
if (bestsize == 0 || out.size() < bestsize) {
|
||||
bestsize = out.size();
|
||||
bestfilter = i;
|
||||
}
|
||||
}
|
||||
|
||||
for (int i = 0; i < numstrategies; i++) {
|
||||
enable[i] = (i == bestfilter);
|
||||
}
|
||||
|
||||
return 0; /* OK */
|
||||
}
|
||||
|
||||
// Outputs the intersection of keepnames and non-essential chunks which are in
|
||||
// the PNG image.
|
||||
void ChunksToKeep(const std::vector<unsigned char>& origpng,
|
||||
const std::vector<std::string>& keepnames,
|
||||
std::set<std::string>* result) {
|
||||
std::vector<std::string> names[3];
|
||||
std::vector<std::vector<unsigned char> > chunks[3];
|
||||
|
||||
lodepng::getChunks(names, chunks, origpng);
|
||||
|
||||
for (size_t i = 0; i < 3; i++) {
|
||||
for (size_t j = 0; j < names[i].size(); j++) {
|
||||
for (size_t k = 0; k < keepnames.size(); k++) {
|
||||
if (keepnames[k] == names[i][j]) {
|
||||
result->insert(names[i][j]);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Keeps chunks with given names from the original png by literally copying them
|
||||
// into the new png
|
||||
void KeepChunks(const std::vector<unsigned char>& origpng,
|
||||
const std::vector<std::string>& keepnames,
|
||||
std::vector<unsigned char>* png) {
|
||||
std::vector<std::string> names[3];
|
||||
std::vector<std::vector<unsigned char> > chunks[3];
|
||||
|
||||
lodepng::getChunks(names, chunks, origpng);
|
||||
std::vector<std::vector<unsigned char> > keepchunks[3];
|
||||
|
||||
// There are 3 distinct locations in a PNG file for chunks: between IHDR and
|
||||
// PLTE, between PLTE and IDAT, and between IDAT and IEND. Keep each chunk at
|
||||
// its corresponding location in the new PNG.
|
||||
for (size_t i = 0; i < 3; i++) {
|
||||
for (size_t j = 0; j < names[i].size(); j++) {
|
||||
for (size_t k = 0; k < keepnames.size(); k++) {
|
||||
if (keepnames[k] == names[i][j]) {
|
||||
keepchunks[i].push_back(chunks[i][j]);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
lodepng::insertChunks(*png, keepchunks);
|
||||
}
|
||||
|
||||
int ZopfliPNGOptimize(const std::vector<unsigned char>& origpng,
|
||||
const ZopfliPNGOptions& png_options,
|
||||
bool verbose,
|
||||
std::vector<unsigned char>* resultpng) {
|
||||
// Use the largest possible deflate window size
|
||||
int windowsize = 32768;
|
||||
|
||||
ZopfliPNGFilterStrategy filterstrategies[kNumFilterStrategies] = {
|
||||
kStrategyZero, kStrategyOne, kStrategyTwo, kStrategyThree, kStrategyFour,
|
||||
kStrategyMinSum, kStrategyEntropy, kStrategyPredefined, kStrategyBruteForce
|
||||
};
|
||||
bool strategy_enable[kNumFilterStrategies] = {
|
||||
false, false, false, false, false, false, false, false, false
|
||||
};
|
||||
std::string strategy_name[kNumFilterStrategies] = {
|
||||
"zero", "one", "two", "three", "four",
|
||||
"minimum sum", "entropy", "predefined", "brute force"
|
||||
};
|
||||
for (size_t i = 0; i < png_options.filter_strategies.size(); i++) {
|
||||
strategy_enable[png_options.filter_strategies[i]] = true;
|
||||
}
|
||||
|
||||
std::vector<unsigned char> image;
|
||||
unsigned w, h;
|
||||
unsigned error;
|
||||
lodepng::State inputstate;
|
||||
error = lodepng::decode(image, w, h, inputstate, origpng);
|
||||
|
||||
bool keep_colortype = false;
|
||||
|
||||
if (!png_options.keepchunks.empty()) {
|
||||
// If the user wants to keep the non-essential chunks bKGD or sBIT, the
|
||||
// input color type has to be kept since the chunks format depend on it.
|
||||
// This may severely hurt compression if it is not an ideal color type.
|
||||
// Ideally these chunks should not be kept for web images. Handling of bKGD
|
||||
// chunks could be improved by changing its color type but not done yet due
|
||||
// to its additional complexity, for sBIT such improvement is usually not
|
||||
// possible.
|
||||
std::set<std::string> keepchunks;
|
||||
ChunksToKeep(origpng, png_options.keepchunks, &keepchunks);
|
||||
keep_colortype = keepchunks.count("bKGD") || keepchunks.count("sBIT");
|
||||
if (keep_colortype && verbose) {
|
||||
printf("Forced to keep original color type due to keeping bKGD or sBIT"
|
||||
" chunk.\n");
|
||||
}
|
||||
}
|
||||
|
||||
if (error) {
|
||||
if (verbose) {
|
||||
if (error == 1) {
|
||||
printf("Decoding error\n");
|
||||
} else {
|
||||
printf("Decoding error %u: %s\n", error, lodepng_error_text(error));
|
||||
}
|
||||
}
|
||||
return error;
|
||||
}
|
||||
|
||||
bool bit16 = false; // Using 16-bit per channel raw image
|
||||
if (inputstate.info_png.color.bitdepth == 16 &&
|
||||
(keep_colortype || !png_options.lossy_8bit)) {
|
||||
// Decode as 16-bit
|
||||
image.clear();
|
||||
error = lodepng::decode(image, w, h, origpng, LCT_RGBA, 16);
|
||||
bit16 = true;
|
||||
}
|
||||
|
||||
if (!error) {
|
||||
// If lossy_transparent, remove RGB information from pixels with alpha=0
|
||||
if (png_options.lossy_transparent && !bit16) {
|
||||
LossyOptimizeTransparent(&inputstate, &image[0], w, h);
|
||||
}
|
||||
|
||||
if (png_options.auto_filter_strategy) {
|
||||
error = AutoChooseFilterStrategy(image, w, h, inputstate, bit16,
|
||||
keep_colortype, origpng,
|
||||
/* Don't try brute force */
|
||||
kNumFilterStrategies - 1,
|
||||
filterstrategies, strategy_enable);
|
||||
}
|
||||
}
|
||||
|
||||
if (!error) {
|
||||
size_t bestsize = 0;
|
||||
|
||||
for (int i = 0; i < kNumFilterStrategies; i++) {
|
||||
if (!strategy_enable[i]) continue;
|
||||
|
||||
std::vector<unsigned char> temp;
|
||||
error = TryOptimize(image, w, h, inputstate, bit16, keep_colortype,
|
||||
origpng, filterstrategies[i], true /* use_zopfli */,
|
||||
windowsize, &png_options, &temp);
|
||||
if (!error) {
|
||||
if (verbose) {
|
||||
printf("Filter strategy %s: %d bytes\n",
|
||||
strategy_name[i].c_str(), (int) temp.size());
|
||||
}
|
||||
if (bestsize == 0 || temp.size() < bestsize) {
|
||||
bestsize = temp.size();
|
||||
(*resultpng).swap(temp); // Store best result so far in the output.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!png_options.keepchunks.empty()) {
|
||||
KeepChunks(origpng, png_options.keepchunks, resultpng);
|
||||
}
|
||||
}
|
||||
|
||||
return error;
|
||||
}
|
||||
|
||||
extern "C" void CZopfliPNGSetDefaults(CZopfliPNGOptions* png_options) {
|
||||
|
||||
memset(png_options, 0, sizeof(*png_options));
|
||||
// Constructor sets the defaults
|
||||
ZopfliPNGOptions opts;
|
||||
|
||||
png_options->lossy_transparent = opts.lossy_transparent;
|
||||
png_options->lossy_8bit = opts.lossy_8bit;
|
||||
png_options->auto_filter_strategy = opts.auto_filter_strategy;
|
||||
png_options->use_zopfli = opts.use_zopfli;
|
||||
png_options->num_iterations = opts.num_iterations;
|
||||
png_options->num_iterations_large = opts.num_iterations_large;
|
||||
png_options->block_split_strategy = opts.block_split_strategy;
|
||||
}
|
||||
|
||||
extern "C" int CZopfliPNGOptimize(const unsigned char* origpng,
|
||||
const size_t origpng_size,
|
||||
const CZopfliPNGOptions* png_options,
|
||||
int verbose,
|
||||
unsigned char** resultpng,
|
||||
size_t* resultpng_size) {
|
||||
ZopfliPNGOptions opts;
|
||||
|
||||
// Copy over to the C++-style struct
|
||||
opts.lossy_transparent = !!png_options->lossy_transparent;
|
||||
opts.lossy_8bit = !!png_options->lossy_8bit;
|
||||
opts.auto_filter_strategy = !!png_options->auto_filter_strategy;
|
||||
opts.use_zopfli = !!png_options->use_zopfli;
|
||||
opts.num_iterations = png_options->num_iterations;
|
||||
opts.num_iterations_large = png_options->num_iterations_large;
|
||||
opts.block_split_strategy = png_options->block_split_strategy;
|
||||
|
||||
for (int i = 0; i < png_options->num_filter_strategies; i++) {
|
||||
opts.filter_strategies.push_back(png_options->filter_strategies[i]);
|
||||
}
|
||||
|
||||
for (int i = 0; i < png_options->num_keepchunks; i++) {
|
||||
opts.keepchunks.push_back(png_options->keepchunks[i]);
|
||||
}
|
||||
|
||||
const std::vector<unsigned char> origpng_cc(origpng, origpng + origpng_size);
|
||||
std::vector<unsigned char> resultpng_cc;
|
||||
|
||||
int ret = ZopfliPNGOptimize(origpng_cc, opts, !!verbose, &resultpng_cc);
|
||||
if (ret) {
|
||||
return ret;
|
||||
}
|
||||
|
||||
*resultpng_size = resultpng_cc.size();
|
||||
*resultpng = (unsigned char*) malloc(resultpng_cc.size());
|
||||
if (!(*resultpng)) {
|
||||
return ENOMEM;
|
||||
}
|
||||
|
||||
memcpy(*resultpng,
|
||||
reinterpret_cast<unsigned char*>(&resultpng_cc[0]),
|
||||
resultpng_cc.size());
|
||||
|
||||
return 0;
|
||||
}
|
||||
134
build/node_modules/node-zopfli/zopfli/src/zopflipng/zopflipng_lib.h
generated
vendored
Normal file
134
build/node_modules/node-zopfli/zopfli/src/zopflipng/zopflipng_lib.h
generated
vendored
Normal file
@@ -0,0 +1,134 @@
|
||||
// Copyright 2013 Google Inc. All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// Author: lode.vandevenne@gmail.com (Lode Vandevenne)
|
||||
// Author: jyrki.alakuijala@gmail.com (Jyrki Alakuijala)
|
||||
|
||||
// Library to recompress and optimize PNG images. Uses Zopfli as the compression
|
||||
// backend, chooses optimal PNG color model, and tries out several PNG filter
|
||||
// strategies.
|
||||
|
||||
#ifndef ZOPFLIPNG_LIB_H_
|
||||
#define ZOPFLIPNG_LIB_H_
|
||||
|
||||
#ifdef __cplusplus
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
extern "C" {
|
||||
|
||||
#endif
|
||||
|
||||
#include <stdlib.h>
|
||||
|
||||
enum ZopfliPNGFilterStrategy {
|
||||
kStrategyZero = 0,
|
||||
kStrategyOne = 1,
|
||||
kStrategyTwo = 2,
|
||||
kStrategyThree = 3,
|
||||
kStrategyFour = 4,
|
||||
kStrategyMinSum,
|
||||
kStrategyEntropy,
|
||||
kStrategyPredefined,
|
||||
kStrategyBruteForce,
|
||||
kNumFilterStrategies /* Not a strategy but used for the size of this enum */
|
||||
};
|
||||
|
||||
typedef struct CZopfliPNGOptions {
|
||||
int lossy_transparent;
|
||||
int lossy_8bit;
|
||||
|
||||
enum ZopfliPNGFilterStrategy* filter_strategies;
|
||||
// How many strategies to try.
|
||||
int num_filter_strategies;
|
||||
|
||||
int auto_filter_strategy;
|
||||
|
||||
char** keepchunks;
|
||||
// How many entries in keepchunks.
|
||||
int num_keepchunks;
|
||||
|
||||
int use_zopfli;
|
||||
|
||||
int num_iterations;
|
||||
|
||||
int num_iterations_large;
|
||||
|
||||
int block_split_strategy;
|
||||
} CZopfliPNGOptions;
|
||||
|
||||
// Sets the default options
|
||||
// Does not allocate or set keepchunks or filter_strategies
|
||||
void CZopfliPNGSetDefaults(CZopfliPNGOptions *png_options);
|
||||
|
||||
// Returns 0 on success, error code otherwise
|
||||
// The caller must free resultpng after use
|
||||
int CZopfliPNGOptimize(const unsigned char* origpng,
|
||||
const size_t origpng_size,
|
||||
const CZopfliPNGOptions* png_options,
|
||||
int verbose,
|
||||
unsigned char** resultpng,
|
||||
size_t* resultpng_size);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} // extern "C"
|
||||
#endif
|
||||
|
||||
// C++ API
|
||||
#ifdef __cplusplus
|
||||
|
||||
struct ZopfliPNGOptions {
|
||||
ZopfliPNGOptions();
|
||||
|
||||
bool verbose;
|
||||
|
||||
// Allow altering hidden colors of fully transparent pixels
|
||||
bool lossy_transparent;
|
||||
// Convert 16-bit per channel images to 8-bit per channel
|
||||
bool lossy_8bit;
|
||||
|
||||
// Filter strategies to try
|
||||
std::vector<ZopfliPNGFilterStrategy> filter_strategies;
|
||||
|
||||
// Automatically choose filter strategy using less good compression
|
||||
bool auto_filter_strategy;
|
||||
|
||||
// PNG chunks to keep
|
||||
// chunks to literally copy over from the original PNG to the resulting one
|
||||
std::vector<std::string> keepchunks;
|
||||
|
||||
// Use Zopfli deflate compression
|
||||
bool use_zopfli;
|
||||
|
||||
// Zopfli number of iterations
|
||||
int num_iterations;
|
||||
|
||||
// Zopfli number of iterations on large images
|
||||
int num_iterations_large;
|
||||
|
||||
// Unused, left for backwards compatiblity.
|
||||
int block_split_strategy;
|
||||
};
|
||||
|
||||
// Returns 0 on success, error code otherwise.
|
||||
// If verbose is true, it will print some info while working.
|
||||
int ZopfliPNGOptimize(const std::vector<unsigned char>& origpng,
|
||||
const ZopfliPNGOptions& png_options,
|
||||
bool verbose,
|
||||
std::vector<unsigned char>* resultpng);
|
||||
|
||||
#endif // __cplusplus
|
||||
|
||||
#endif // ZOPFLIPNG_LIB_H_
|
||||
Reference in New Issue
Block a user