GraphCanon updated 2w · GitHub synced 2w · 35 views this month
Decision brief
PrivateGPT provides a comprehensive API layer to build private, on-premise AI applications leveraging local OpenAI-compatible inference servers. It offers features such as RAG, skills, tools, text-to-SQL functionalities,
Good fit when
- - You need to deploy and operationalize your own locally-run models without relying on cloud APIs.
- - Your application requires a high degree of customization beyond what existing cloud services offer.
Avoid when
- - You prefer simplicity and ease-of-use over full control; PrivateGPT requires more setup than using direct cloud-based AI services.
- - Your project does not involve running models locally but strictly relies on public cloud resources for inference server operations.
- Requirements:
- Min 8 GB RAM; Requires Docker
Observed Jul 11, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Very active (1d since push)
- As of 2w
- Provenance
- Not a fork · Organization account
- As of 2w
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install private-gptHow it fits your stack(5)
Typed graph edges - alternatives, integrations, successors, and dependencies. Ranked by relationship type, not raw GitHub stars.
Alternative
Relationship graph
Optional deeper exploration of typed edges and category neighbours.
Similar tools
Same-category neighbours not already linked as typed edges.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
PrivateGPT offers an open-source API that interfaces with OpenAI-compatible inference servers to build private, on-premise AI applications. It provides RAG, skills, tools, text-to-SQL functionalities among others.
Capability facts
- Deploy
- Self-host
Source: dockerfile:Dockerfile · Aug 8, 2026
- Docker
- Dockerfile present
Source: dockerfile:Dockerfile · Aug 8, 2026
- CLI
- CLI entrypoint
Source: pyproject.toml:[project.scripts] · Aug 8, 2026
- Languages
- python
Source: github.language+pyproject.toml · Aug 8, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Aug 8, 2026)
tp://localhost:8080/ui). The API is at `http://localhost:8080` and follows the [Anthropic API](https://docs.privategpt.dev/api-reference/api-reference) spec.Source link
Source: README excerpt (regex_v1, Aug 8, 2026)
uv tool install --python 3.11 \Source link
Tags
README
PrivateGPT is the open-source API layer that turns local models into production AI applications.
Running a model locally is only the first step. To build useful AI applications you need a set of higher-level building blocks. PrivateGPT provides that layer as an open-source API following the Claude API model — so you can build private AI products without rebuilding the same backend primitives from scratch, and without depending on cloud APIs.
Production-tested: PrivateGPT powers Zylon, the on-premise AI platform providing Private AI to enterprises across the globe.
Your app / agent / workflow / UI
|
PrivateGPT API
|
OpenAI-compatible inference server (Ollama, llama.cpp, vLLM, …)
PrivateGPT does not run models itself. It connects to any OpenAI-compatible inference server via
OPENAI_API_BASE. If it implements/v1/chat/completionsand/v1/models, it works.
PrivateGPT ships a built-in workbench UI for testing and demos, available at /ui. The API is the actual product.
What PrivateGPT gives you
- Standard messages API (streaming, async, token counting)
- File and artifact ingestion
- Retrieval with citations and agentic RAG
- Built-in tools mirroring the Claude API (web search, web fetch, code execution)
- Custom tools and MCP connectors
- Structured access to databases and CSVs
- Embeddings and orchestration
Quickstart
For Docker, full installation options, and model configuration see the full Quickstart guide.
Prerequisites: You need a running OpenAI-compatible LLM server. Ollama is the easiest starting point.
1. Install PrivateGPT
# macOS
brew tap zylon-ai/tap
brew install private-gpt
# Linux
curl -LsSf https://astral.sh/uv/install.sh | sh
uv tool install --python 3.11 \
--find-links https://wheels.privategpt.dev/packages/ \
"private-gpt[core]"
# Windows
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
uv tool install --python 3.11 `
--find-links https://wheels.privategpt.dev/packages/ `
"private-gpt[core]"
2. Start your LLM server
# Example with Ollama
ollama pull qwen3.5:35b # LLM (~24 GB)
ollama pull mxbai-embed-large # Embeddings (~670 MB)
ollama serve
3. Run PrivateGPT
# macOS / Linux
OPENAI_API_BASE=http://localhost:<llm-port>/v1 \
OPENAI_EMBEDDING_API_BASE=http://localhost:<embedding-port>/v1 \
private-gpt serve
# Windows (PowerShell)
$env:OPENAI_API_BASE = "http://localhost:<llm-port>/v1"
$env:OPENAI_EMBEDDING_API_BASE = "http://localhost:<embedding-port>/v1"
private-gpt serve
4. Open the UI
Go to http://localhost:8080/ui. The API is at http://localhost:8080 and follows the Anthropic API spec.
The UI is useful for:
- Sending messages.
- Selecting models from /v1/models.
- Uploading documents.
- Testing retrieval with citations.
- Enabling tools per chat.
- Configuring databases, MCP connectors, skills, and custom tools.
- Inspecting requests and responses through the API Debugger.
This UI is a demonstrator, not the core product. Developers are expected to build their own applications on top of the API. That said, the UI is intentionally polished enough for demos, videos, internal pilots, and quick local usage.
Integrations
For agents
This page has a .md twin and JSON over the API.