Skip to content

Benchmarks 2024 11 26 TVM LLVM Os spike_rv32_min

GitHub Action edited this page Nov 26, 2024 · 1 revision

Setup

Simulator

  • Spike (riscv-isa-sim ) (ISS, CPI=1)
    • Spike : eb0a3e2b0a7c57522928be39de95cd9f8c6dc636
    • Spike PK : fix-gcc14-rvv

Toolchains

Models

Frameworks

  • MLonMCU : develop

  • TVM : Nightly Pre-Build

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Results (Framework: tvm, Backend: tvmaot, Toolchain: llvm, Flags: -Os, Target: spike_rv32_min )

Audio Wake Words (aww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
29341873.0
( 0.5x )
148940
( 1.187 )
59492
( 3.098 )
0 NCHW TVM Fallback RV32IM 0 -
24953197.0
( 0.6x )
139508
( 1.112 )
59492
( 3.098 )
0 NHWC TVM Fallback RV32IM 0 -
-
( ?x )
143940
( 1.147 )
59492
( 3.098 )
128 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
143188
( 1.141 )
59492
( 3.098 )
256 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
143140
( 1.141 )
59492
( 3.098 )
512 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
143388
( 1.143 )
59492
( 3.098 )
1024 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
143348
( 1.143 )
59492
( 3.098 )
2048 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
143872
( 1.147 )
59492
( 3.098 )
4096 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
140800
( 1.122 )
59492
( 3.098 )
128 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
8443312.0
( 1.8x )
140732
( 1.122 )
59492
( 3.098 )
256 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
7621578.0
( 2.0x )
140672
( 1.121 )
59492
( 3.098 )
512 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
7648641.0
( 2.0x )
150996
( 1.203 )
59492
( 3.098 )
1024 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
7649624.0
( 2.0x )
150984
( 1.203 )
59492
( 3.098 )
2048 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
7652201.0
( 2.0x )
151044
( 1.204 )
59492
( 3.098 )
4096 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
15106764.0
( Base )
125468
( Base )
19204
( Base )
0 NHWC muRISCV-NN Scalar RV32IM 0 -
14538885.0
( 1.0x )
124228
( 0.99 )
23668
( 1.232 )
0 NHWC muRISCV-NN Vector (Portable) RV32IM 0 -
-
( ?x )
128144
( 1.021 )
19276
( 1.004 )
128 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
128208
( 1.022 )
19204
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
128208
( 1.022 )
19204
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
128208
( 1.022 )
19204
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
128208
( 1.022 )
19204
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
128208
( 1.022 )
19204
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
5123082.0
( 2.9x )
126420
( 1.008 )
23668
( 1.232 )
128 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
3989093.0
( 3.8x )
126392
( 1.007 )
23668
( 1.232 )
256 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
3430815.0
( 4.4x )
126392
( 1.007 )
23668
( 1.232 )
512 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
3389827.0
( 4.5x )
126392
( 1.007 )
23668
( 1.232 )
1024 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
3390248.0
( 4.5x )
126392
( 1.007 )
23668
( 1.232 )
2048 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
3390573.0
( 4.5x )
126392
( 1.007 )
23668
( 1.232 )
4096 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
126732
( 1.01 )
23740
( 1.236 )
128 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
126660
( 1.01 )
23668
( 1.232 )
256 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
126660
( 1.01 )
23668
( 1.232 )
512 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
126660
( 1.01 )
23668
( 1.232 )
1024 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
126660
( 1.01 )
23668
( 1.232 )
2048 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
126660
( 1.01 )
23668
( 1.232 )
4096 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Image Classification (resnet)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
124480494.0
( 0.5x )
259020
( 1.51 )
108404
( 1.953 )
0 NCHW TVM Fallback RV32IM 0 -
101655071.0
( 0.6x )
245412
( 1.431 )
108404
( 1.953 )
0 NHWC TVM Fallback RV32IM 0 -
-
( ?x )
253516
( 1.478 )
108404
( 1.953 )
128 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
253248
( 1.476 )
108404
( 1.953 )
256 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
252548
( 1.472 )
108404
( 1.953 )
512 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
252552
( 1.472 )
108404
( 1.953 )
1024 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
252692
( 1.473 )
108404
( 1.953 )
2048 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
253464
( 1.478 )
108404
( 1.953 )
4096 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
37247381.0
( 1.5x )
247216
( 1.441 )
108404
( 1.953 )
128 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
29649810.0
( 1.9x )
247304
( 1.442 )
108404
( 1.953 )
256 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
28368603.0
( 2.0x )
247604
( 1.444 )
108404
( 1.953 )
512 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
249116
( 1.452 )
108404
( 1.953 )
1024 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
28358790.0
( 2.0x )
249104
( 1.452 )
108404
( 1.953 )
2048 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
249172
( 1.453 )
108404
( 1.953 )
4096 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
56263526.0
( Base )
171524
( Base )
55508
( Base )
0 NHWC muRISCV-NN Scalar RV32IM 0 -
72490862.0
( 0.8x )
170588
( 0.995 )
55508
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32IM 0 -
-
( ?x )
175044
( 1.021 )
55580
( 1.001 )
128 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
175252
( 1.022 )
55508
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
175252
( 1.022 )
55508
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
175252
( 1.022 )
55508
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
175252
( 1.022 )
55508
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
175252
( 1.022 )
55508
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
15265296.0
( 3.7x )
173500
( 1.012 )
55508
( 1.0 )
128 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
9682591.0
( 5.8x )
173648
( 1.012 )
55508
( 1.0 )
256 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
7137049.0
( 7.9x )
173648
( 1.012 )
55508
( 1.0 )
512 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
5895003.0
( 9.5x )
173648
( 1.012 )
55508
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
4967288.0
( 11.3x )
173648
( 1.012 )
55508
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
4718519.0
( 11.9x )
173648
( 1.012 )
55508
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
173368
( 1.011 )
55580
( 1.001 )
128 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
173576
( 1.012 )
55508
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
173576
( 1.012 )
55508
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
173576
( 1.012 )
55508
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
173576
( 1.012 )
55508
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
173576
( 1.012 )
55508
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Anomaly Detection (toycar)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
3204435.0
( 0.5x )
611740
( 1.771 )
5540
( 1.169 )
0 NCHW TVM Fallback RV32IM 0 -
3204435.0
( 0.5x )
611740
( 1.771 )
5540
( 1.169 )
0 NHWC TVM Fallback RV32IM 0 -
-
( ?x )
610720
( 1.768 )
5540
( 1.169 )
128 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
610536
( 1.767 )
5540
( 1.169 )
256 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
611824
( 1.771 )
5540
( 1.169 )
512 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
611824
( 1.771 )
5540
( 1.169 )
1024 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
611824
( 1.771 )
5540
( 1.169 )
2048 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
611824
( 1.771 )
5540
( 1.169 )
4096 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
610720
( 1.768 )
5540
( 1.169 )
128 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
610536
( 1.767 )
5540
( 1.169 )
256 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
611824
( 1.771 )
5540
( 1.169 )
512 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
611824
( 1.771 )
5540
( 1.169 )
1024 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
611824
( 1.771 )
5540
( 1.169 )
2048 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
611824
( 1.771 )
5540
( 1.169 )
4096 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
1720025.0
( Base )
345476
( Base )
4740
( Base )
0 NHWC muRISCV-NN Scalar RV32IM 0 -
1720026.0
( 1.0x )
345480
( 1.0 )
4740
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32IM 0 -
-
( ?x )
346244
( 1.002 )
4740
( 1.0 )
128 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
346260
( 1.002 )
4740
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
346260
( 1.002 )
4740
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
346256
( 1.002 )
4740
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
346256
( 1.002 )
4740
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
346264
( 1.002 )
4740
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
622983.0
( 2.8x )
347344
( 1.005 )
4740
( 1.0 )
128 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
506305.0
( 3.4x )
347348
( 1.005 )
4740
( 1.0 )
256 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
447966.0
( 3.8x )
347348
( 1.005 )
4740
( 1.0 )
512 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
419117.0
( 4.1x )
347344
( 1.005 )
4740
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
415491.0
( 4.1x )
347352
( 1.005 )
4740
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
413633.0
( 4.2x )
347348
( 1.005 )
4740
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
346244
( 1.002 )
4740
( 1.0 )
128 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
346264
( 1.002 )
4740
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
346260
( 1.002 )
4740
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
346248
( 1.002 )
4740
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
346264
( 1.002 )
4740
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
346252
( 1.002 )
4740
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Visual Wake Words (vww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Layout Kernels Mode Arch Unroll Auto-Vectorization
84534158.0
( 0.5x )
596564
( 1.655 )
181000
( 2.113 )
0 NCHW TVM Fallback RV32IM 0 -
72619933.0
( 0.6x )
564992
( 1.568 )
181000
( 2.113 )
0 NHWC TVM Fallback RV32IM 0 -
-
( ?x )
577636
( 1.603 )
181000
( 2.113 )
128 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
576700
( 1.6 )
181000
( 2.113 )
256 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
576500
( 1.6 )
181000
( 2.113 )
512 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
575320
( 1.596 )
181000
( 2.113 )
1024 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
575660
( 1.597 )
181000
( 2.113 )
2048 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
577888
( 1.603 )
181000
( 2.113 )
4096 NCHW TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
565732
( 1.57 )
181000
( 2.113 )
128 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
565500
( 1.569 )
181000
( 2.113 )
256 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
24029580.0
( 1.9x )
566076
( 1.571 )
181000
( 2.113 )
512 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
23497727.0
( 1.9x )
580008
( 1.609 )
181000
( 2.113 )
1024 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
23506160.0
( 1.9x )
600884
( 1.667 )
181000
( 2.113 )
2048 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
23545443.0
( 1.9x )
609568
( 1.691 )
181000
( 2.113 )
4096 NHWC TVM Fallback RV32IM_ZVE64X 0 Loop+SLP
45053097.0
( Base )
360420
( Base )
85640
( Base )
0 NHWC muRISCV-NN Scalar RV32IM 0 -
44929595.0
( 1.0x )
359384
( 0.997 )
85640
( 1.0 )
0 NHWC muRISCV-NN Vector (Portable) RV32IM 0 -
-
( ?x )
363328
( 1.008 )
85856
( 1.003 )
128 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
363572
( 1.009 )
85640
( 1.0 )
256 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
363572
( 1.009 )
85640
( 1.0 )
512 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
363572
( 1.009 )
85640
( 1.0 )
1024 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
363572
( 1.009 )
85640
( 1.0 )
2048 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
363572
( 1.009 )
85640
( 1.0 )
4096 NHWC muRISCV-NN Scalar RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
362008
( 1.004 )
85640
( 1.0 )
128 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
362180
( 1.005 )
85640
( 1.0 )
256 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
362180
( 1.005 )
85640
( 1.0 )
512 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
362184
( 1.005 )
85640
( 1.0 )
1024 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
362184
( 1.005 )
85640
( 1.0 )
2048 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
362184
( 1.005 )
85640
( 1.0 )
4096 NHWC muRISCV-NN Vector RV32IM_ZVE64X 0 -
-
( ?x )
362120
( 1.005 )
85856
( 1.003 )
128 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
362224
( 1.005 )
85640
( 1.0 )
256 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
362228
( 1.005 )
85640
( 1.0 )
512 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
362228
( 1.005 )
85640
( 1.0 )
1024 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
362224
( 1.005 )
85640
( 1.0 )
2048 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP
-
( ?x )
362228
( 1.005 )
85640
( 1.0 )
4096 NHWC muRISCV-NN Vector (Portable) RV32IM_ZVE64X 0 Loop+SLP

Original data

Click here to download the raw files for this benchmark.

2024-11-26
2024-11-21
2024-11-19
2024-11-18
2024-07-12
2024-06-29
2024-03-02
2024-02-26
2024-02-23
2024-02-22
2024-02-20
2024-02-11
2023-12-22
Clone this wiki locally