Misc. bug: Thread [N]: EXC_BAD_ACCESS (code=1, address=0x11c)

[maksymmatviievskyi]

After pulling the latest llama.swiftui from the /examples directory and testing it on iPhone XR (IOS 17) and iPhone 12 (IOS 18), the llama_tokenize (line 300) evokes a non-existing address pointer error after the model was successfully loaded, and a prompt was sent with the respective implemented command.

Is it a performance issue? It seems like the app should run on iPhone 15 (as per video evidence provided in the docs); However, I believe there is more than enough VRAM available at the time of execution since I used the tinyllama-1.1b-1t-openorca.Q4_0 and the error seems to be due to something else.

Nevertheless, I would appreciate any help with this.

Problem description & steps to reproduce

Compile the library first:

cmake -DCMAKE_SYSTEM_NAME=iOS \
      -DCMAKE_OSX_ARCHITECTURES=arm64 \
      -DCMAKE_BUILD_TYPE=Release \
      -DLLAMA_BUILD_TESTS=OFF \
      -DLLAMA_BUILD_EXAMPLES=OFF \
      -DGGML_METAL=OFF \
      -DSIMD_SUM_F32_DISABLED=ON \
      -S . \
      -B build

cmake --build build --config Release -j 16 

Select all .dylib files from the newly created build directory. Open Xcode, and put them all under the Frameworks folder. In General > Frameworks, Libraries and Embedded Content, make sure all of them are flagged as Embedded & Sign

Build for your physical machine and download the lightest model. After sending a prompt, the error quickly causes the application to pause.

[ggerganov]

It's just the code hasn't been updated to use the new llama_tokenize API that now accepts llama_vocab instead of llama_model:

llama.cpp/examples/batched.swift/Sources/main.swift

Line 210 in 3d804de

let tokenCount = llama_tokenize(model, text, Int32(utf8Count), tokens, Int32(n_tokens), add_bos, /*special tokens*/ false)

PRs welcome.