What would be the best OS to run LLMs?

via

LocalLlama

Hi there, I've ordered a mini PC with 128GB of RAM and the AMD AI Max 395. I intend to use it with Proxmox (like my actual machine), where I run Windows for some gaming and macOS for my music library server. I also want to run LLMs on it. Main purpose would be local agent coding and some text refining. I'm quite new and it's quite overwhelming to be honest. It evolves so fast I can't keep track of what works best. What would be the best OS for LLMs? What would be the best software to run LLMs? A

More from LocalLlama

daily ritual at this point…

LocalLlama·14

convert : add support for Nemotron Nano 3 Omni by danbev · Pull Request #22481 · ggml-org/llama.cpp

LocalLlama·5

Mistral Medium Is On The Way

LocalLlama·14

Something from Mistral (Vibe) tomorrow

LocalLlama·6

Deepseek Vision Coming

LocalLlama·7

First direct side by side MoE vs Dense comparison.

LocalLlama·8