Why is the v2 writer wasm binary so much larger?

[mfulton26]

according to the readmes, the writer wasm binary size goes from ~366 KB to ~1.17 MB

that seems to me to be a big jump; is there a way to reduce that or is there a need for it to be so much larger for the writer?

(the reader is slightly smaller in v2 beta 3)

[Sec-ant]

Hi, v2 enables ZXING_EXPERIMENTAL_API of zxing-cpp and sets ZXING_WRITERS to NEW, so it now uses zint as the backend of the writer. zint can provide encoding support for all available formats in this library, while in v1 the writing capability is limited to only some of the formats. As source files from zint are linked and compiled by the compiler, the wasm file size of the writer also increases. <-- this is not correct, see the replies below

is there a way to reduce that or is there a need for it to be so much larger for the writer?

I believe finishing a feature comes prior to optimizing it. Although when building zxing-cpp you can still use the old writer and switch on/off some of the features, it's not very practical to provide such capability as a npm package. Also, zint has a high chance to be the default writer backend in zxing-cpp in the future, so I enabled it in v2.

I also noticed the size bump. I believe the binary size can be further minimized, but it requires someone who are very familiar with cpp, c, webassembly and those two repos. I probably will look further into it in the future. I also appreciate suggestions from @axxel and @gitlost if there're some stupid mistakes I make that causes some unnecessary size bump.

(the reader is slightly smaller in v2 beta 3)

Yes I converted some of the bindings from emscripten enum or string to numbers, e.g., no stringification is needed when passing the barcode format between js <-> cpp. I believe this kills some unneeded functions.

---

I am also curious about your use case. Currently, writer, reader and full are provided as three different exports in this npm package and unused code should be tree-shakable if you only use one of them to build something. I may have some blinding spots on how people use this package, so would you elaborate on how this will be a concern for you?

[Sec-ant]

@axxel Happy new year! I appreciate your information, but I believe I configured the library with ZXING_WRITERS=NEW. I also don't see MultiFormatWriter in the log.

Here is the build log of the .wasm binary file (for both reader and writer targets), do you spot any anomalies?

❯ pnpm build:wasm

> zxing-wasm@2.0.0-beta.3 build:wasm /Users/secant/personal/zxing-wasm
> cmake --build build -j$(($(nproc 2>/dev/null || sysctl -n hw.logicalcpu) - 1))

[  0%] Building CXX object ZXing/CMakeFiles/ZXing.dir/cmake_pch.hxx.pch
[  1%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/Barcode.cpp.o
[  2%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/BarcodeFormat.cpp.o
[  3%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/Error.cpp.o
[  4%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/BitMatrixIO.cpp.o
[  5%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/HRI.cpp.o
[  6%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/datamatrix/DMDataBlock.cpp.o
[  7%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/GTIN.cpp.o
[  8%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/WriteBarcode.cpp.o
[  9%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/ZXingCpp.cpp.o
[ 10%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/MultiFormatReader.cpp.o
[ 11%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/oned/ODDataBarExpandedBitDecoder.cpp.o
[ 12%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/qrcode/QRVersion.cpp.o
[ 13%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/qrcode/QRBitMatrixParser.cpp.o
[ 14%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/qrcode/QRDataBlock.cpp.o
[ 15%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/qrcode/QRDecoder.cpp.o
[ 16%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/qrcode/QRReader.cpp.o
[ 17%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzueci/zueci.c.o
[ 17%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/2of5.c.o
[ 18%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/aztec.c.o
[ 19%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/code.c.o
[ 20%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/code128.c.o
[ 21%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/common.c.o
[ 22%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/dmatrix.c.o
[ 23%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/eci.c.o
[ 24%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/filemem.c.o
[ 25%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/general_field.c.o
[ 26%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/gs1.c.o
[ 26%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/large.c.o
[ 27%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/library.c.o
[ 28%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/maxicode.c.o
[ 29%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/medical.c.o
[ 30%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/output.c.o
[ 31%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/pdf417.c.o
[ 32%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/qr.c.o
[ 33%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/raster.c.o
[ 34%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/reedsol.c.o
[ 35%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/rss.c.o
[ 35%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/stubs.c.o
[ 36%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/svg.c.o
[ 37%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/upcean.c.o
[ 38%] Building C object ZXing/CMakeFiles/ZXing.dir/src/libzint/vector.c.o
[ 39%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/BitMatrix.cpp.o
[ 40%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/CharacterSet.cpp.o
[ 41%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/ECI.cpp.o
[ 42%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/Content.cpp.o
[ 42%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/ReadBarcode.cpp.o
[ 43%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/Utf.cpp.o
[ 44%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/BitArray.cpp.o
[ 45%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/GenericGF.cpp.o
[ 46%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/GenericGFPoly.cpp.o
[ 47%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/TextUtfEncoding.cpp.o
[ 48%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/BinaryBitmap.cpp.o
[ 49%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/BitSource.cpp.o
[ 49%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/ConcentricFinder.cpp.o
[ 50%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/GlobalHistogramBinarizer.cpp.o
[ 51%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/GridSampler.cpp.o
[ 52%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/HybridBinarizer.cpp.o
[ 53%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/PerspectiveTransform.cpp.o
[ 54%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/ReedSolomonDecoder.cpp.o
[ 55%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/ResultPoint.cpp.o
[ 56%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/TextDecoder.cpp.o
[ 56%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/WhiteRectDetector.cpp.o
[ 57%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/aztec/AZDecoder.cpp.o
[ 58%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/aztec/AZDetector.cpp.o
[ 59%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/aztec/AZReader.cpp.o
[ 60%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/datamatrix/DMBitLayout.cpp.o
[ 61%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/datamatrix/DMVersion.cpp.o
[ 62%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/datamatrix/DMDecoder.cpp.o
[ 63%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/datamatrix/DMDetector.cpp.o
[ 64%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/datamatrix/DMReader.cpp.o
[ 64%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/maxicode/MCBitMatrixParser.cpp.o
[ 65%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/maxicode/MCDecoder.cpp.o
[ 66%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/maxicode/MCReader.cpp.o
[ 67%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/oned/ODUPCEANCommon.cpp.o
[ 68%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/oned/ODCode128Patterns.cpp.o
[ 69%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/oned/ODCodabarReader.cpp.o
[ 70%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/oned/ODCode39Reader.cpp.o
[ 71%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/oned/ODCode93Reader.cpp.o
[ 72%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/oned/ODCode128Reader.cpp.o
[ 73%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/oned/ODDataBarCommon.cpp.o
[ 73%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/oned/ODDataBarReader.cpp.o
[ 74%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/oned/ODDataBarExpandedReader.cpp.o
[ 75%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/oned/ODDataBarLimitedReader.cpp.o
[ 76%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/oned/ODDXFilmEdgeReader.cpp.o
[ 77%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/oned/ODITFReader.cpp.o
[ 78%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/oned/ODMultiUPCEANReader.cpp.o
[ 79%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/oned/ODReader.cpp.o
[ 80%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/pdf417/ZXBigInteger.cpp.o
[ 81%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/pdf417/PDFBarcodeValue.cpp.o
[ 81%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/pdf417/PDFBoundingBox.cpp.o
[ 82%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/pdf417/PDFCodewordDecoder.cpp.o
[ 83%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/pdf417/PDFDecoder.cpp.o
[ 84%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/pdf417/PDFDetectionResult.cpp.o
[ 85%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/pdf417/PDFDetectionResultColumn.cpp.o
[ 86%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/pdf417/PDFDetector.cpp.o
[ 87%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/pdf417/PDFModulusGF.cpp.o
[ 88%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/pdf417/PDFModulusPoly.cpp.o
[ 89%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/pdf417/PDFReader.cpp.o
[ 89%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/qrcode/QRCodecMode.cpp.o
[ 90%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/pdf417/PDFScanningDecoder.cpp.o
[ 91%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/qrcode/QRErrorCorrectionLevel.cpp.o
[ 92%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/qrcode/QRDetector.cpp.o
[ 93%] Building CXX object ZXing/CMakeFiles/ZXing.dir/src/qrcode/QRFormatInformation.cpp.o
[ 94%] Linking CXX static library libZXing.a
[ 94%] Built target ZXing
[ 95%] Building CXX object CMakeFiles/zxing_full.dir/ZXingWasm.cpp.o
[ 96%] Building CXX object CMakeFiles/zxing_reader.dir/ZXingWasm.cpp.o
[ 97%] Building CXX object CMakeFiles/zxing_writer.dir/ZXingWasm.cpp.o
[ 98%] Linking CXX executable /Users/secant/personal/zxing-wasm/src/writer/zxing_writer.js
cache:INFO: generating system asset: symbol_lists/21de559270c0fed3c6f5ef6f4bd7bba488c1b5a5.json... (this will be cached in "/Users/secant/personal/emsdk/upstream/emscripten/cache/symbol_lists/21de559270c0fed3c6f5ef6f4bd7bba488c1b5a5.json" for subsequent builds)
cache:INFO:  - ok
[ 99%] Linking CXX executable /Users/secant/personal/zxing-wasm/src/reader/zxing_reader.js
[100%] Linking CXX executable /Users/secant/personal/zxing-wasm/src/full/zxing_full.js
[100%] Built target zxing_writer
[100%] Built target zxing_reader
[100%] Built target zxing_full

[axxel]

Happy New Year to you, too. I see the new dxfilmedge.c.o is missing. Do you have the latest zxing-cpp version in your git submodule? Other than that, it looks ok to me. But that build will be "big" since it contains both reader and writer. So linking the zxing_writer.js based on that will be large as well, right? Or do you expect the linker to drop the reader code from the writer binary because it can detect un-linked symbols?

Writing this down reminds of this line which will cause a full 'read' of any newly created symbol to be able to construct a full Barcode object, including the bytes() content. So your writer-only binary will be equivalent to the full binary. This would become unnecessary once libzint adds support for the stuff discussed in the "Re 2" part of @gitlost answer here.

[Sec-ant]

I see the new dxfilmedge.c.o is missing. Do you have the latest zxing-cpp version in your git submodule?

Oh, yes, this log wasn't from the latest version. I can see dxfilmedge.c.o is also logged in a latest build.

But that build will be "big" since it contains both reader and writer. So linking the zxing_writer.js based on that will be large as well, right? Or do you expect the linker to drop the reader code from the writer binary because it can detect un-linked symbols?

sry I wasn't being clear. The log is a one time build for three targets altogather (writer, reader and writer + reader). I believe different targets will link different files, because that was the case in the previous version. I was justing trying to figure out if there're unnecessary old writer api files included in these files.

Writing this down reminds of this line which will cause a full 'read' of any newly created symbol to be able to construct a full Barcode object, including the bytes() content. So your writer-only binary will be equivalent to the full binary. This would become unnecessary once libzint adds support for the stuff discussed in the "Re 2" part of @gitlost answer here.

I believe this is the reason, thank you for pointing that out! I'll be keeping an eye on #883 then.

[Sec-ant]

@axxel I just realized I defined both the ZXING_READERS and ZXING_WRITERS options in the CMakeLists.txt file, for no matter which target I'm to build 🤦. Therefore, when building the writer target, the CreatBarcode will still use the reader code if ZXING_READERS is defined. It seems won't use the readBarcode function if ZXING_READERS is not defined, am I correct?

So if that's the case, I need to set different options for different targets, and those options should be passed down to zxing-cpp:

• reader target: ZXING_READERS=ON and ZXING_WRITERS=OFF
• writer target: ZXING_READERS=OFF and ZXING_WRITERS=NEW
• full target: ZXING_READERS=ON and ZXING_WRITERS=NEW

Do you happen to know how to do that in a single CMakeLists.txt file? I played with it for 2 hours but couldn't make it work. The closest shot I got is using ExternalProject_Add but I don't know how to let it use emcc instead of gcc. Or should I have different CMakeLists.txt for different targets?

---

I temporarily solved it by pnpm scripts, see f8c33b2.

---

@mfulton26 FYI I published v2.0.0-beta.4, the size of the zxing_writer.wasm file now shrinks down to ~594 KB, which just matches what @axxel mentioned above.

- This subpath only provides a function to write barcodes. The wasm binary size is ~1.17 MB.
+ This subpath only provides a function to write barcodes. The wasm binary size is ~594 KB.

[axxel]

am I correct?

Yes. Apparently I didn't look closely enough. I expected a missing symbol error during linking if you have ZXING_EXPERIMENTAL_API=ON ZXING_READERS=OFF ZING_WRITERS=NEW. But seems I made it do something useful in that case as well. ;).

So if that's the case, I need to set different options for different targets, and those options should be passed down to zxing-cpp:

* `reader` target: `ZXING_READERS=ON` and `ZXING_WRITERS=OFF`
* `writer` target: `ZXING_READERS=OFF` and `ZXING_WRITERS=NEW` 
* `full` target: `ZXING_READERS=ON` and `ZXING_WRITERS=NEW`

Correct. That is what I meant above with "will be big".

Do you happen to know how to do that in a single CMakeLists.txt file? I played with it for 2 hours but couldn't make it work. The closest shot I got is using ExternalProject_Add but I don't know how to let it use emcc instead of gcc. Or should I have different CMakeLists.txt for different targets?

I don't know how to do that. On the other hand, the WASM build that our CI script performs seems to do just that (see https://github.com/zxing-cpp/zxing-cpp/releases/download/v2.3.0/wasm-artifacts.zip).

[mfulton26]

would you elaborate on how this will be a concern for you?

I plan to use the wasm writer in Deno Deploy along with various other tools and I don't want to hit any memory limits. Not a huge concern at the moment but the jump in size surprised me so I figured I'd reach out to learn more.