Build error

[80Builder80]

I'm not sure if this is an update issue or something I did when reinstalling my vm.

I am using a Google Cloud vm instance x86/64, 64gb RAM, 250gb HD, running Debian 12

I had this working yesterday. I was able to convert Qwen. However, the instance I had was not adequate to convert llama2 so I created a new one today. When I cloned the repo and tried to install, I received the following:
`@rknn-llm:~$ cd ezrknn-llm && sudo bash install.sh

#########################################
Checking root permission...
#########################################

#########################################
Installing RKNN LLM libraries...
#########################################

#########################################
Compiling LLM runtime for Linux...
#########################################

-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /ezrknn-llm/rkllm-runtime/example/build/build_linux_aarch64_Release
[ 50%] Building CXX object CMakeFiles/llm_demo.dir/src/main.cpp.o
[100%] Linking CXX executable llm_demo
/usr/bin/ld: /ezrknn-llm/rkllm-runtime/example/../runtime/Linux/librkllm_api/aarch64/librkllmrt.so: error adding symbols: file in wrong format
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/llm_demo.dir/build.make:98: llm_demo] Error 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/llm_demo.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

#########################################
Moving rkllm to /usr/bin...
#########################################

cp: cannot stat './build/build_aarch64_Release/llm_demo': No such file or directory

#########################################
Increasing file limit for all users (needed for LLMs to run)...
#########################################

#########################################
Done installing ezrknn-llm!
#########################################`

Any assistance would be appreciated.

[Pelochus]

Oops, sorry! I thought GitHub would advise me with an email if any discussion would be opened.

Next time, please open an issue for this type of things, it is more suitable to the issues section rather than discussions. You can also @me if I don't respond within 2-3 days

The issue here is already fixed, my fault:

./build/build_aarch64_Release/llm_demo

Actually was

./build/build_linux_aarch64_Release/llm_demo

Latest install.sh commit is actually that fix, find it out on my own

[Pelochus]

Okay nevermind, I was looking at the second issue.

The build error is actually because this will only compile for ARM devices. By default, the compilation script is using the default GCC and G++ installed on your system. Since you are using a GCloud instance, you are running x86 GCC/G++.

If you want to use the cloud instance for converting, you actually don't need to install the full program. Check this for converting:

https://github.com/Pelochus/ezrknn-llm?tab=readme-ov-file#converting-llms-for-rockchips-npus

[80Builder80]

Thank you for the assistance. I was able to get it to work. I also grabbed your Llama 2 7B and 13B conversions. They both ran on my OP5+ 32GB using Joshua Riek's Ubuntu 24 desktop version.

[Pelochus]

Good to know!

Do you have any video to see Llama 2 7B and 13B speed? AFAIK, the guys at r/RockchipNPU haven't been able to run it. If you have any footage of Llama 2 7B and 13B we would love to see it.

[80Builder80]

I did have to change ulimit to 8096 to get the 7B model to load. I'm having an issue getting the 13B model to load. I it may be because I think I was running them on your Armian build the first time. This time I'm running J.R's Ubuntu 24 version. I'm compiling your version again to test if that helps.

[Pelochus]

I think the current ulimit set in install.sh is 16xxx so perhaps you used a previous version. Anyway, it is an easy and quick fix.

[80Builder80]

This is the load time for the 7B model. I know, boring but...
Screencast from 2024-05-01 16-51-56.webm

[80Builder80]

This is the model running. It's not consistent with answers. I had ran the model just before recording it and it gave me a very detailed answer to the question. After reloading the model, I got this response from the same question.
Screencast from 2024-05-01 16-54-36.webm