What is your llm.c development environment? #350
-
|
What is your llm.c development environment? For example, what form factor (e.g. laptop? desktop?), GPU requirements (e.g. NVIDIA graphics card model?), etc. I have an M1 MacBook, so no CUDA for me, so I am interested in buying a modest configuration to fiddle with llm.c. Andrej, what do you use? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
You can run the CPU version on anything, Macbook included. You can install OpenMP on it too, and follow those instructions. But it's still only CPU. To use the GPU on the Macbook you'd have to upgrade to Metal. There is a Metal fork of this repo (see the notable forks section) but I am not sure what state it is in. Most likely to be dangerous you want a GPU. You can get a GPU box in the cloud in multiple places, try google search, or use Lambda labs, or e.g. Lightning AI, or etc. A GPU box with a non-state of the art GPU shouldn't be too expensive at all - you spin it up on demand and ssh to it to do work. I like to connect my VS Code to the box via ssh, for example. Personally I use an A100 box from Lambda. And I connect my VS Code to it. |
Beta Was this translation helpful? Give feedback.
You can run the CPU version on anything, Macbook included. You can install OpenMP on it too, and follow those instructions. But it's still only CPU. To use the GPU on the Macbook you'd have to upgrade to Metal. There is a Metal fork of this repo (see the notable forks section) but I am not sure what state it is in.
Most likely to be dangerous you want a GPU. You can get a GPU box in the cloud in multiple places, try google search, or use Lambda labs, or e.g. Lightning AI, or etc. A GPU box with a non-state of the art GPU shouldn't be too expensive at all - you spin it up on demand and ssh to it to do work. I like to connect my VS Code to the box via ssh, for example.
Personally I use an A…