SparkFloorFunction for the AVX2 Benchmarking

[rajatma1993]

Description

Hi @taiyang-li ,Team

I am looking over the SparkFloorFunction Implemented with the AVX2 , I want to understand how the benchmarking is performed the on the function performance ?? Do we have info on the results of benchmarks documented somewhere ?

Also can you also help me in understanding that is this enhancements in only with the clickhouse Backend or do it works with the Velox backends ?

[taiyang-li]

The SparkFloorFunction implementation includes AVX2-specific optimizations to improve performance. These optimizations are conditionally compiled and used if the target architecture supports AVX2.

[taiyang-li]

The enhancement is only avaiable for CH backend, but velox still could refer to the internal principles of it.

[taiyang-li]

Do we have info on the results of benchmarks documented somewhere ?
I didn't have the bench result, the benchmarks was added quite a long time. But you can still build and run it.

[rajatma1993]

Hi @taiyang-li ,

Thanks For you Valuable Response.

One more Thing wanted to confirm with you is there any specific Reason for implementing only this at gluten layer rather Click house Layer ??

[taiyang-li]

maybe it is possible in gluten layer，but only in native languages like cpp or rust can SIMD be fully used. implementing it in ch layer is a better option

[rajatma1993]

Hi @taiyang-li ,

Do we have info on the results of benchmarks documented somewhere ?
I didn't have the bench result, the benchmarks was added quite a long time. But you can still build and run it.

Does you know if there are any existing benchmark scripts or setup available for Spark Floor that I can refer to? Or will I need to create my own benchmark from scratch?

[taiyang-li]

If you works on Velox, you should create your benchmark from scratch because my benchmark is binded to CH backend. I'm sorry that you couldn't reuse it.

[rajatma1993]

@taiyang-li

I am working on CH(Click House) backend ,So can you help me with the Your Benchmark ??

[rajatma1993]

Hi @taiyang-li ,

I understood , Below command you have used to benchmark the function and Get the reports , May I know where/Which folder should I run the command to get the reports. ??

Command : **./build_gcc/utils/extern-local-engine/tests/benchmark_local_engine --benchmark_filter="BM.Floor.Float64"

[taiyang-li]

@rajatma1993 It doen't matter. You can run benchmark_local_engine under any folder, as long as absolute path of benchmark_local_engine is given.

[rajatma1993]

@rajatma1993 It doen't matter. You can run benchmark_local_engine under any folder, as long as absolute path of benchmark_local_engine is given.
Okay !!

@taiyang-li
But in my case,
I have successfully completed the build step of the gluten + Clickhouse, Now I am trying to run the benchmark command as I have given above, But it is not executing. Below is the error.

I am trying to run it from Project folder,

command : ./build_gcc/utils/extern-local-engine/tests/benchmark_local_engine --benchmark_filter="BM.Floor.Float64"
sh: 0: cannot open ./build_gcc/utils/extern-local-engine/tests/benchmark_local_engine: No such file**

I am missing something that is needed to run the benchmark ??

Do I need to build with some Flag to enable to benchmarks ?? Like --build_benchmarks=ON ?

[taiyang-li]

try

• ninja benchmake_local_engine
• find . -name "benchmark_local_engine"
• use the right path instead of mine

[rajatma1993]

@taiyang-li ,

I tried and below is output's, Am I missing anything ??

command : ninja benchmark_local_engine
error : ninja: error: loading 'build.ninja': No such file or directory

command : find . -name "benchmark_local_engine"
output : find: ‘./ep/_ep/arrow_ep/cpp-jni/protobuf_ep-prefix/src/protobuf_ep’: Permission denied

[taiyang-li]

https://github.com/apache/incubator-gluten/blob/main/docs/get-started/ClickHouse.md you can first learn how to build gluten with clickhouse backend.

[rajatma1993]

HI @taiyang-li ,

Based on my understand from the documentation the Click House Can be build using 4 approaches, So Now in order to build click House and Gluten and Then to run the Benchmarks for Floor Functions , which Approach I need to Fallow out 4 approach ?

I am preferring the option 1(Build Directly) as First & if that does not help then option 4(Build it as a submodule of Clickhouse) as 2nd, to build and run benchmarking for Floor Functions, So can you confirm me with the Approach I can go ahead to Click house build ?