KoboldCpp

Self-contained local model runner with a built-in UI.

koboldai.org
Visit websiteView source
Price
FreeFree and open source; the only cost is the hardware you run it on.
Works on
Windows, macOS, Linux
Based in
Open-source project
Source code
Open source

KoboldCpp is a single-file local inference tool with a built-in web UI, aimed at people who want to run GGUF models for chat, writing, and roleplay on their own hardware. Everything runs locally so no prompt is ever transmitted to an outside server. It bundles extras like character cards, story memory, and image generation and it is actively maintained and openly licensed under AGPL-3.0. Setup takes a little more effort than a polished consumer app but it rewards you with fine-grained control and complete privacy. It is a strong pick for power users who want a self-contained local stack.

Fake KoboldCpp sites are out there, so use the link we give above to get the real thing.

Strengths

  • Self-contained and runs models fully offline
  • Built-in web UI plus creative-writing features
  • Actively maintained, AGPL-3.0 licensed
  • Broad GGUF model support

Considerations

  • More hands-on than a consumer app
  • Capable models need decent hardware
Listed inPrivate AI

KoboldCpp alternatives

OllamaRun open LLMs on your own machine, offline.LumoProton's private assistant that keeps no logs of your chats.llamafileA whole LLM packed into one offline executable file.Duck.aiAnonymous, account-free access to popular chatbots.