Benchmarks are defined for raw C structs, Googles flatc
generated C++
and the flatcc
compilers C ouput.
These can be run with:
scripts/benchmark.sh
and requires a C++ compiler installed - the benchmark for flatcc alone can be run with:
test/benchmark/benchflatcc/run.sh
this only requires a system C compiler (cc) to be installed (and flatcc's build environment).
A summary for OS-X 2.2 GHz Haswell core-i7 is found below. Generated files for OS-X and Ubuntu are found in the benchmark folder.
The benchmarks use the same schema and dataset as Google FPL's FlatBuffers benchmark.
In summary, 1 million iterations runs at about 500-540MB/s at 620-700
ns/op encoding buffers and 29-34ns/op traversing buffers. flatc
and
flatcc
are close enough in performance for it not to matter much.
flatcc
is a bit faster encoding but it is likely due to less memory
allocation. Throughput and time per operatin are of course very specific
to this test case.
Generated JSON parser/printer shown below, for flatcc only but for OS-X and Linux.
elapsed time: 0.055 (s)
iterations: 1000000
size: 312 (bytes)
bandwidth: 5665.517 (MB/s)
throughput in ops per sec: 18158707.100
throughput in 1M ops per sec: 18.159
time per op: 55.070 (ns)
elapsed time: 0.012 (s)
iterations: 1000000
size: 312 (bytes)
bandwidth: 25978.351 (MB/s)
throughput in ops per sec: 83263946.711
throughput in 1M ops per sec: 83.264
time per op: 12.010 (ns)
elapsed time: 0.702 (s)
iterations: 1000000
size: 344 (bytes)
bandwidth: 490.304 (MB/s)
throughput in ops per sec: 1425301.380
throughput in 1M ops per sec: 1.425
time per op: 701.606 (ns)
elapsed time: 0.029 (s)
iterations: 1000000
size: 344 (bytes)
bandwidth: 11917.134 (MB/s)
throughput in ops per sec: 34642832.398
throughput in 1M ops per sec: 34.643
time per op: 28.866 (ns)
elapsed time: 0.626 (s)
iterations: 1000000
size: 336 (bytes)
bandwidth: 536.678 (MB/s)
throughput in ops per sec: 1597255.277
throughput in 1M ops per sec: 1.597
time per op: 626.074 (ns)
elapsed time: 0.029 (s)
iterations: 1000000
size: 336 (bytes)
bandwidth: 11726.930 (MB/s)
throughput in ops per sec: 34901577.551
throughput in 1M ops per sec: 34.902
time per op: 28.652 (ns)
Note: this benchmark is only available for flatcc
. It uses the exact
same data set as above.
The benchmark uses Grisu3 floating point parsing and printing algorithm with exact fallback to strtod/sprintf when the algorithm fails to be exact. Better performance can be gained by enabling inexact Grisu3 and SSE 4.2 in build options, but likely not worthwhile in praxis.
(encode means printing from existing binary buffer to JSON)
elapsed time: 1.407 (s)
iterations: 1000000
size: 722 (bytes)
bandwidth: 513.068 (MB/s)
throughput in ops per sec: 710619.931
throughput in 1M ops per sec: 0.711
time per op: 1.407 (us)
(decode/traverse means parsing json to flatbuffer binary and calculating checksum)
elapsed time: 2.218 (s)
iterations: 1000000
size: 722 (bytes)
bandwidth: 325.448 (MB/s)
throughput in ops per sec: 450758.672
throughput in 1M ops per sec: 0.451
time per op: 2.218 (us)
Numbers for Linux included because parsing is significantly faster.
elapsed time: 1.210 (s)
iterations: 1000000
size: 722 (bytes)
bandwidth: 596.609 (MB/s)
throughput in ops per sec: 826328.137
throughput in 1M ops per sec: 0.826
time per op: 1.210 (us)
elapsed time: 1.772 (s)
iterations: 1000000
size: 722 (bytes)
bandwidth: 407.372 (MB/s)
throughput in ops per sec: 564227.736
throughput in 1M ops per sec: 0.564
time per op: 1.772 (us)