LocalVibe – Pure-Rust local AI stack with MCP, in one binary (Apple Silicon)
LocalVibe is a Pure-Rust local AI stack designed for Apple Silicon Macs, offering a terminal-based interface for interacting with quantized language models. It supports on-device inference via Metal, directory indexing with ONNX embeddings, and vector search using LanceDB. The tool is distributed as a single binary and emphasizes local execution without reliance on external services.
- ▪LocalVibe runs on Apple Silicon (M1–M4) and uses Metal for efficient on-device inference through Candle.
- ▪It supports local LLM chat, directory indexing, and vector search using fastembed-rs and LanceDB, all within a ratatui-based TUI.
- ▪The system allows runtime switching between multiple models and vector databases, with configuration managed via TOML files.
Opening excerpt (first ~120 words) tap to expand
local-vibe (localvibe / lv) Pure-Rust local coding assistant: chat with a quantized LLM on Metal, index any directory with on-device ONNX embeddings, search it with LanceDB, all from one ratatui TUI. Runs on Apple Silicon (M1–M4). Candle + Metal for inference, fastembed-rs for embeddings, LanceDB for vectors. A TUI screenshot will live here once one is captured — see docs/screenshots/. Quick start Assumes ~/.cargo/bin is on PATH, you are on macOS, and you have a GGUF model supported by Candle (qwen2 / llama family — Qwen 3.5 hybrid SSM is not supported). # 1. install the `localvibe` binary (`lv` alias is also installed) git clone https://github.com/Sok205/local_vibe ~/code/local_vibe cd ~/code/local_vibe cargo install --path crates/lv-cli # 2.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at GitHub.