feat(raw): make build script more resilient

[philpax]

This turns on the individual flags based on the target features of the host for both Linux and Windows x86-64. Support for other architectures is possible, but I haven't done it because I don't have a way to test them.

There is one pretty large annoyance with this - you need to explicitly supply feature flags, because rustc won't automatically turn them on:

RUSTFLAGS='-C target-feature=+avx2,+fma,+f16c'

(also, f16c was only stabilised in 1.68.0...)

I haven't been able to find a way to detect this automatically for the target (and not for the host); I'm also not sure if it would necessarily be a good idea (I can think of people deploying to old / low-power x86 machines).

[setzer22]

Not sure I understand the implications of this 🤔

There is one pretty large annoyance with this - you need to explicitly supply feature flags, because rustc won't automatically turn them on:

Does this mean the rustflags have to be set by users compiling llama-rs, or any crate that depends on it? If so, that's a bit unfortunate. Is there no way around it? I'd like the compilation process to be as close to a regular cargo build as possible.

[setzer22]

Yup, I just tried building your PR and these have to be specified manually, unfortunately. I think it's not very good user experience if you still have to manually specify the features supported by your CPU anyway.

The Makefile in llama.rs seems to use various heuristics to make this work. On Mac, it uses shell sysctl and on linux it just greps the contents of /proc/cpuinfo. I'm sure Windows has an equivalent mechanism too. These are things we could probably do in a build script too.

https://github.com/ggerganov/llama.cpp/blob/master/Makefile

[philpax]

Does this mean the rustflags have to be set by users compiling llama-rs, or any crate that depends on it? If so, that's a bit unfortunate. Is there no way around it? I'd like the compilation process to be as close to a regular cargo build as possible.

Yeah, that would be the case :/

The Makefile in llama.rs seems to use various heuristics to make this work. On Mac, it uses shell sysctl and on linux it just greps the contents of /proc/cpuinfo. These are things we could probably do in a build script too.

We can do that - and that's what I was thinking of doing at first - but the problem is that you could be building for a different target than your host. You don't want to turn on AVX2 if you're building the library for your old PC, for example.

I'm not really sure what the best way forward here is - ideally it'd always emit the instructions and just switch at runtime. It might already do that, but if it does, why does AVX support have to be specified? Probably need to peek into ggml some more...

[philpax]

Alrighty! I've come up with a neat little fix - I default to host features if host == target, but fall back to target features otherwise. This should allow you to cross-compile with different target features.

That may cause issues if you want to cut a binary for the same target as yourself but with fewer features, but I'm hoping that's a relatively niche use case. We can figure something out if that's necessary (probably a feature that disables host detection or something?)

---

I'm not going to rebase #10 onto this so that they can be reviewed separately. The merge should take care of it anyway.

[setzer22]

Amazing! :) It now runs at full speed without need to set any RUSTFLAGS 👍

I'll just merge right away so we can finally move on to merging the other PRs