AI tools that respect you

AnythingLLM

Mintplex Labs

Recommended

All-in-one desktop and server application for private AI chat over your documents. Built-in RAG, agents and multi-user support; works with local models via Ollama or with cloud providers.

• Open source
• Local-first
• Privacy-first
• Self-hostable

Axolotl

Axolotl AI

Recommended

Tool for fine-tuning open LLMs, supporting full fine-tunes, LoRA, QLoRA and preference-tuning techniques across many model families, configured through simple YAML files.

• Open source
• Local-first
• Privacy-first
• Self-hostable

Open-source embedding database for AI applications. Simple API for storing documents, embeddings and metadata with vector and full-text search; runs in-process for prototyping or as a server.

• Open source
• Local-first
• Privacy-first
• Self-hostable

ComfyUI

Comfy Org

Recommended

Node-based graphical interface for diffusion models. Build image and video generation pipelines as visual graphs, with support for Stable Diffusion, Flux and a large ecosystem of community custom nodes.

• Open source
• Local-first
• Privacy-first
• Self-hostable

Hugging Face's model-definition library for state-of-the-art transformer models in text, vision and audio. Load thousands of pretrained open models for local inference or training in PyTorch.

• Open source
• Local-first
• Privacy-first
• Self-hostable

Jan

Menlo Research

Recommended

Open-source ChatGPT alternative that runs 100 percent offline on your computer. Desktop app for downloading and chatting with local models, with optional connections to cloud providers.

• Open source
• Local-first
• Privacy-first
• Self-hostable

llama.cpp

ggml-org (Georgi Gerganov)

Recommended

LLM inference engine in C/C++ with minimal dependencies. Runs quantized GGUF models on CPU and GPU across Apple Silicon, x86 and CUDA, and powers many local AI applications.

• Open source
• Local-first
• Privacy-first
• Self-hostable

Meta Llama

Meta

Recommended

Meta's family of open-weight language models, from small on-device sizes to large frontier models. Weights are freely downloadable under the Llama Community License, which carries some usage restrictions.

• Local-first
• Privacy-first
• Self-hostable

Mistral Open Models

Mistral AI

Recommended

Open-weight language models from Mistral AI, including Mistral 7B, Mixtral and Mistral Small, released under the Apache-2.0 license and runnable on local hardware.

• Open source
• Local-first
• Privacy-first
• Self-hostable

Ollama

Ollama

Recommended

Run open large language models locally with a simple CLI and REST API. One-line installs for Llama, Qwen, Gemma, DeepSeek and hundreds of other models on macOS, Linux and Windows.

• Open source
• Local-first
• Privacy-first
• Self-hostable

Open WebUI

Open WebUI

Recommended

Self-hosted web interface for AI chat that works with Ollama and OpenAI-compatible APIs. Supports RAG, web search, image generation and multi-user setups, and is designed to operate fully offline.

• Open source
• Local-first
• Privacy-first
• Self-hostable

Piper

Open Home Foundation

Recommended

Fast, local neural text-to-speech engine that runs well on modest hardware such as the Raspberry Pi. Offers many voices across dozens of languages and is maintained under the Open Home Foundation.

• Open source
• Local-first
• Privacy-first
• Self-hostable

Qdrant

Qdrant

Recommended

High-performance vector database and similarity search engine written in Rust. Stores embeddings with payload filtering, hybrid search and quantization; runs as a single binary, in Docker, or as a managed cloud.

• Open source
• Local-first
• Privacy-first
• Self-hostable

Free, self-hostable internet metasearch engine that aggregates results from many search services without tracking or profiling users. A common private search backend for local AI stacks.

• Open source
• Local-first
• Privacy-first
• Self-hostable

vLLM

vLLM Project

Recommended

High-throughput, memory-efficient inference and serving engine for LLMs. Uses PagedAttention and continuous batching to serve open models at scale behind an OpenAI-compatible API.

• Open source
• Local-first
• Privacy-first
• Self-hostable

whisper.cpp

ggml-org (Georgi Gerganov)

Recommended

Port of OpenAI's Whisper speech recognition model in plain C/C++. Fast offline transcription on CPU and Apple Silicon with no Python dependency; widely embedded in local dictation and transcription apps.

• Open source
• Local-first
• Privacy-first
• Self-hostable

get-md

Nano Collective

Recommended

Fast, lightweight HTML, PDF, DOCX and Markdown to Markdown converter optimised for LLM consumption. Converts files or URLs in under 100ms, with optional AI-powered conversion using a local model. Library and CLI.

• Open source
• Local-first
• Privacy-first
• Self-hostable

json-up

Nano Collective

Recommended

Fast, type-safe JSON migration tool with Zod schema validation. Fluent builder API for versioned migration chains, full TypeScript inference, async migration support and automatic version tracking.

• Open source
• Local-first
• Privacy-first
• Self-hostable

Nanocoder

Nano Collective

Recommended

Open coding agent for your terminal, built by a community collective. Runs agentic coding on the model of your choice — local models via Ollama or any OpenAI-compatible API — so your code stays on your machine.

• Open source
• Local-first
• Privacy-first
• Self-hostable

Nanotune

Nano Collective

Recommended

Interactive CLI for fine-tuning small language models on Apple Silicon. A TUI guides you through data prep, training, export, chat and benchmarking, handling the MLX and llama.cpp tooling automatically.

• Open source
• Local-first
• Privacy-first
• Self-hostable

prompt-scrubber

Nano Collective

Recommended

Local-first utility that strips identifying content — emails, secrets, file paths, URLs, phone numbers — from prompts before they reach a cloud LLM, mapping each value to a stable placeholder so responses can be rehydrated locally.

• Open source
• Local-first
• Privacy-first
• Self-hostable

Sentinel

Nano Collective

Recommended

Installable, Nanocoder-driven workflow that runs continuous, configurable security and code audits across the repositories in a GitHub organisation and files findings as issues. Local models are a first-class path.

• Open source
• Local-first
• Privacy-first
• Self-hostable

AI pair programming in your terminal. Edits code in your local git repository via chat, works with Claude, GPT and local models, and automatically commits changes with sensible messages.

• Open source
• Local-first

Cline

Cline

Autonomous coding agent available as a VS Code extension, CLI and SDK. Plans and executes multi-step coding tasks with human-in-the-loop approval of file edits and terminal commands; bring your own model keys.

• Open source
• Local-first

Continue

Continue

Open-source coding agent for VS Code and JetBrains. Chat, autocomplete and edit with any model — cloud APIs or local models via Ollama — configured through shareable blocks.

• Open source
• Local-first

GPT4All

Nomic AI

Desktop application from Nomic for running open LLMs locally on consumer hardware, with private chat and LocalDocs for asking questions about your own files without data leaving your machine.

• Open source
• Local-first
• Privacy-first

LangChain

LangChain

Framework for developing applications powered by LLMs. Provides agent orchestration, tool calling, memory and integrations with hundreds of model providers and data sources in Python and JavaScript.

• Open source
• Local-first
• Self-hostable

Langfuse

Langfuse

Open-source LLM engineering platform for tracing, evals, prompt management and metrics, with OpenTelemetry-based observability. Self-hostable; the core is MIT-licensed with paid enterprise features.

• Open source
• Local-first
• Self-hostable

LlamaIndex

LlamaIndex

Data framework for connecting LLMs to your own data. Document loaders, indexes and query engines for building RAG pipelines and document agents in Python and TypeScript.

• Open source
• Local-first
• Self-hostable

LM Studio

LM Studio

Desktop app for discovering, downloading and running local LLMs such as Llama, Qwen and Gemma. Includes a chat interface and an OpenAI-compatible local server. Free to use but not open source.

• Local-first
• Privacy-first

Zed

Zed Industries

High-performance, open-source code editor written in Rust with built-in AI features, multiplayer collaboration and agentic editing. AI features can use cloud providers or local models via Ollama.

• Open source
• Local-first