Skip to content

Benchmarks 2024 11 18 TFLM GCC Os spike_rv64

GitHub Action edited this page Nov 18, 2024 · 4 revisions

Setup

Simulator

  • Spike (riscv-isa-sim ) (ISS, CPI=1)
    • Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
    • Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Toolchains

Models

Frameworks

  • MLonMCU : develop

  • TFLM : 8eb6b23de4470d6a8da3131650d6a67514dfa130

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Results (Framework: tflm, Backend: tflmi, Toolchain: gcc, Flags: -Os, Target: spike_rv64 )

Audio Wake Words (aww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
165487704
( 0.1x )
129074
( 0.863 )
38376
( 1.0 )
0 TFLM Reference RV64GC 0 -
165487704
( 0.1x )
129076
( 0.863 )
38376
( 1.0 )
128 TFLM Reference RV64GCV 0 Loop+SLP
165487704
( 0.1x )
129076
( 0.863 )
38376
( 1.0 )
256 TFLM Reference RV64GCV 0 Loop+SLP
165487704
( 0.1x )
129076
( 0.863 )
38376
( 1.0 )
512 TFLM Reference RV64GCV 0 Loop+SLP
165487704
( 0.1x )
129076
( 0.863 )
38376
( 1.0 )
1024 TFLM Reference RV64GCV 0 Loop+SLP
165487704
( 0.1x )
129076
( 0.863 )
38376
( 1.0 )
2048 TFLM Reference RV64GCV 0 Loop+SLP
165487704
( 0.1x )
129076
( 0.863 )
38376
( 1.0 )
4096 TFLM Reference RV64GCV 0 Loop+SLP
18655014
( Base )
149502
( Base )
38384
( Base )
0 muRISCV-NN Scalar RV64GC 0 -
18665491
( 1.0x )
150340
( 1.006 )
38384
( 1.0 )
128 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
18665491
( 1.0x )
150340
( 1.006 )
38384
( 1.0 )
256 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
18665491
( 1.0x )
150340
( 1.006 )
38384
( 1.0 )
512 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
18665491
( 1.0x )
150340
( 1.006 )
38384
( 1.0 )
1024 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
18665491
( 1.0x )
150340
( 1.006 )
38384
( 1.0 )
2048 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
18665491
( 1.0x )
150340
( 1.006 )
38384
( 1.0 )
4096 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
4190488
( 4.5x )
151002
( 1.01 )
38384
( 1.0 )
128 muRISCV-NN Vector RV64GCV 0 -
2919968
( 6.4x )
151002
( 1.01 )
38384
( 1.0 )
256 muRISCV-NN Vector RV64GCV 0 -
2230048
( 8.4x )
151002
( 1.01 )
38384
( 1.0 )
512 muRISCV-NN Vector RV64GCV 0 -
2187720
( 8.5x )
151002
( 1.01 )
38384
( 1.0 )
1024 muRISCV-NN Vector RV64GCV 0 -
2187720
( 8.5x )
151002
( 1.01 )
38384
( 1.0 )
2048 muRISCV-NN Vector RV64GCV 0 -
2189585
( 8.5x )
151002
( 1.01 )
38384
( 1.0 )
4096 muRISCV-NN Vector RV64GCV 0 -

Image Classification (resnet)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
696989466
( 0.1x )
169550
( 0.927 )
71144
( 1.0 )
0 TFLM Reference RV64GC 0 -
696989450
( 0.1x )
169544
( 0.927 )
71144
( 1.0 )
128 TFLM Reference RV64GCV 0 Loop+SLP
696989450
( 0.1x )
169544
( 0.927 )
71144
( 1.0 )
256 TFLM Reference RV64GCV 0 Loop+SLP
696989450
( 0.1x )
169544
( 0.927 )
71144
( 1.0 )
512 TFLM Reference RV64GCV 0 Loop+SLP
696989450
( 0.1x )
169544
( 0.927 )
71144
( 1.0 )
1024 TFLM Reference RV64GCV 0 Loop+SLP
696989450
( 0.1x )
169544
( 0.927 )
71144
( 1.0 )
2048 TFLM Reference RV64GCV 0 Loop+SLP
696989450
( 0.1x )
169544
( 0.927 )
71144
( 1.0 )
4096 TFLM Reference RV64GCV 0 Loop+SLP
88720235
( Base )
182956
( Base )
71128
( Base )
0 muRISCV-NN Scalar RV64GC 0 -
88715346
( 1.0x )
183522
( 1.003 )
71128
( 1.0 )
128 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
88715346
( 1.0x )
183522
( 1.003 )
71128
( 1.0 )
256 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
88715346
( 1.0x )
183522
( 1.003 )
71128
( 1.0 )
512 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
88715346
( 1.0x )
183522
( 1.003 )
71128
( 1.0 )
1024 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
88715346
( 1.0x )
183522
( 1.003 )
71128
( 1.0 )
2048 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
88715346
( 1.0x )
183522
( 1.003 )
71128
( 1.0 )
4096 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
15636328
( 5.7x )
185420
( 1.013 )
71128
( 1.0 )
128 muRISCV-NN Vector RV64GCV 0 -
9955144
( 8.9x )
185420
( 1.013 )
71128
( 1.0 )
256 muRISCV-NN Vector RV64GCV 0 -
7364408
( 12.0x )
185420
( 1.013 )
71128
( 1.0 )
512 muRISCV-NN Vector RV64GCV 0 -
6102209
( 14.5x )
185420
( 1.013 )
71128
( 1.0 )
1024 muRISCV-NN Vector RV64GCV 0 -
5157857
( 17.2x )
185420
( 1.013 )
71128
( 1.0 )
2048 muRISCV-NN Vector RV64GCV 0 -
4906170
( 18.1x )
185420
( 1.013 )
71128
( 1.0 )
4096 muRISCV-NN Vector RV64GCV 0 -

Anomaly Detection (toycar)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
3872016
( 0.5x )
329922
( 0.985 )
21504
( 1.0 )
0 TFLM Reference RV64GC 0 -
3872016
( 0.5x )
329914
( 0.985 )
21504
( 1.0 )
128 TFLM Reference RV64GCV 0 Loop+SLP
3872016
( 0.5x )
329914
( 0.985 )
21504
( 1.0 )
256 TFLM Reference RV64GCV 0 Loop+SLP
3872016
( 0.5x )
329914
( 0.985 )
21504
( 1.0 )
512 TFLM Reference RV64GCV 0 Loop+SLP
3872016
( 0.5x )
329914
( 0.985 )
21504
( 1.0 )
1024 TFLM Reference RV64GCV 0 Loop+SLP
3872016
( 0.5x )
329914
( 0.985 )
21504
( 1.0 )
2048 TFLM Reference RV64GCV 0 Loop+SLP
3872016
( 0.5x )
329914
( 0.985 )
21504
( 1.0 )
4096 TFLM Reference RV64GCV 0 Loop+SLP
1948441
( Base )
334778
( Base )
21504
( Base )
0 muRISCV-NN Scalar RV64GC 0 -
1948531
( 1.0x )
334810
( 1.0 )
21504
( 1.0 )
128 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
1948531
( 1.0x )
334810
( 1.0 )
21504
( 1.0 )
256 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
1948531
( 1.0x )
334810
( 1.0 )
21504
( 1.0 )
512 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
1948531
( 1.0x )
334810
( 1.0 )
21504
( 1.0 )
1024 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
1948531
( 1.0x )
334810
( 1.0 )
21504
( 1.0 )
2048 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
1948531
( 1.0x )
334810
( 1.0 )
21504
( 1.0 )
4096 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
2201640
( 0.9x )
336194
( 1.004 )
21504
( 1.0 )
128 muRISCV-NN Vector RV64GCV 0 -
2083272
( 0.9x )
336194
( 1.004 )
21504
( 1.0 )
256 muRISCV-NN Vector RV64GCV 0 -
2024088
( 1.0x )
336194
( 1.004 )
21504
( 1.0 )
512 muRISCV-NN Vector RV64GCV 0 -
1994832
( 1.0x )
336194
( 1.004 )
21504
( 1.0 )
1024 muRISCV-NN Vector RV64GCV 0 -
1991148
( 1.0x )
336194
( 1.004 )
21504
( 1.0 )
2048 muRISCV-NN Vector RV64GCV 0 -
1989264
( 1.0x )
336194
( 1.004 )
21504
( 1.0 )
4096 muRISCV-NN Vector RV64GCV 0 -

Visual Wake Words (vww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Unroll Auto-Vectorization
466677307
( 0.1x )
402716
( 0.952 )
136680
( 1.0 )
0 TFLM Reference RV64GC 0 -
466677307
( 0.1x )
402718
( 0.952 )
136680
( 1.0 )
128 TFLM Reference RV64GCV 0 Loop+SLP
466677307
( 0.1x )
402718
( 0.952 )
136680
( 1.0 )
256 TFLM Reference RV64GCV 0 Loop+SLP
466677307
( 0.1x )
402718
( 0.952 )
136680
( 1.0 )
512 TFLM Reference RV64GCV 0 Loop+SLP
466677307
( 0.1x )
402718
( 0.952 )
136680
( 1.0 )
1024 TFLM Reference RV64GCV 0 Loop+SLP
466677307
( 0.1x )
402718
( 0.952 )
136680
( 1.0 )
2048 TFLM Reference RV64GCV 0 Loop+SLP
466677307
( 0.1x )
402718
( 0.952 )
136680
( 1.0 )
4096 TFLM Reference RV64GCV 0 Loop+SLP
55398883
( Base )
423144
( Base )
136688
( Base )
0 muRISCV-NN Scalar RV64GC 0 -
55403599
( 1.0x )
423982
( 1.002 )
136688
( 1.0 )
128 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
55403599
( 1.0x )
423982
( 1.002 )
136688
( 1.0 )
256 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
55403599
( 1.0x )
423982
( 1.002 )
136688
( 1.0 )
512 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
55403599
( 1.0x )
423982
( 1.002 )
136688
( 1.0 )
1024 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
55403599
( 1.0x )
423982
( 1.002 )
136688
( 1.0 )
2048 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
55403599
( 1.0x )
423982
( 1.002 )
136688
( 1.0 )
4096 muRISCV-NN Scalar RV64GCV 0 Loop+SLP
13887867
( 4.0x )
424644
( 1.004 )
136688
( 1.0 )
128 muRISCV-NN Vector RV64GCV 0 -
10553803
( 5.2x )
424644
( 1.004 )
136688
( 1.0 )
256 muRISCV-NN Vector RV64GCV 0 -
9262323
( 6.0x )
424644
( 1.004 )
136688
( 1.0 )
512 muRISCV-NN Vector RV64GCV 0 -
8757095
( 6.3x )
424644
( 1.004 )
136688
( 1.0 )
1024 muRISCV-NN Vector RV64GCV 0 -
8708229
( 6.4x )
424644
( 1.004 )
136688
( 1.0 )
2048 muRISCV-NN Vector RV64GCV 0 -
8710094
( 6.4x )
424644
( 1.004 )
136688
( 1.0 )
4096 muRISCV-NN Vector RV64GCV 0 -

Original data

Click here to download the raw files for this benchmark.

2024-11-26
2024-11-21
2024-11-19
2024-11-18
2024-07-12
2024-06-29
2024-03-02
2024-02-26
2024-02-23
2024-02-22
2024-02-20
2024-02-11
2023-12-22
Clone this wiki locally