GraphCanon
Home/Inference & Serving/private-gpt
private-gpt logo

private-gpt

zylon-ai/private-gpt

Complete API layer for private AI applications on local models

GitHub Website

GraphCanon updated today · GitHub synced today · 29 views this month

57k
Stars
7.6k
Forks
5
Open issues
459
Watchers
1d
Last push
Python Apache-2.0Created May 2, 2023

Trust & integrity

Full report
Maintenance
Very active (0d since push)
As of today · Source: github_public_v1
Provenance
Not a fork · Organization account
As of today · Source: github_public_v1
Security (OSV)
No lockfile
As of today · Source: none

Public GitHub metadata and optional OSV dependency scans. Signals, not a guarantee. Trust methodology.

Overview

PrivateGPT offers an open-source API that interfaces with OpenAI-compatible inference servers to build private, on-premise AI applications. It provides RAG, skills, tools, text-to-SQL functionalities among others.

Capability facts

Deploy
Self-host

Source: dockerfile:Dockerfile · Jul 11, 2026

Docker
Dockerfile present

Source: dockerfile:Dockerfile · Jul 11, 2026

CLI
CLI entrypoint

Source: pyproject.toml:[project.scripts] · Jul 11, 2026

Languages
python

Source: github.language+pyproject.toml · Jul 11, 2026

Categories

Inference & Serving

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Anthropic APIAnthropic API

Source: README excerpt (regex_v1, Jul 11, 2026)

tp://localhost:8080/ui). The API is at `http://localhost:8080` and follows the [Anthropic API](https://docs.privategpt.dev/api-reference/api-reference) spec.
Source link
Python runtimePython

Source: README excerpt (regex_v1, Jul 11, 2026)

uv tool install --python 3.11 \
Source link

Tags

text-to-sqlaion-premisetoolsraglocal-modelsmcpai-tools

README

PrivateGPT is the open-source API layer that turns local models into production AI applications.

zylon-ai%2Fprivate-gpt | Trendshift

Running a model locally is only the first step. To build useful AI applications you need a set of higher-level building blocks. PrivateGPT provides that layer as an open-source API following the Claude API model — so you can build private AI products without rebuilding the same backend primitives from scratch, and without depending on cloud APIs.

Production-tested: PrivateGPT powers Zylon, the on-premise AI platform providing Private AI to enterprises across the globe.

Your app / agent / workflow / UI
              |
        PrivateGPT API
              |
OpenAI-compatible inference server (Ollama, llama.cpp, vLLM, …)

PrivateGPT does not run models itself. It connects to any OpenAI-compatible inference server via OPENAI_API_BASE. If it implements /v1/chat/completions and /v1/models, it works.

PrivateGPT ships a built-in workbench UI for testing and demos, available at /ui. The API is the actual product.

What PrivateGPT gives you

  • Standard messages API (streaming, async, token counting)
  • File and artifact ingestion
  • Retrieval with citations and agentic RAG
  • Built-in tools mirroring the Claude API (web search, web fetch, code execution)
  • Custom tools and MCP connectors
  • Structured access to databases and CSVs
  • Embeddings and orchestration

Quickstart

For Docker, full installation options, and model configuration see the full Quickstart guide.

Prerequisites: You need a running OpenAI-compatible LLM server. Ollama is the easiest starting point.

1. Install PrivateGPT

# macOS
brew tap zylon-ai/tap
brew install private-gpt

# Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

uv tool install --python 3.11 \
  --find-links https://wheels.privategpt.dev/packages/ \
  "private-gpt[core]"

# Windows
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

uv tool install --python 3.11 `
  --find-links https://wheels.privategpt.dev/packages/ `
  "private-gpt[core]"

2. Start your LLM server

# Example with Ollama
ollama pull qwen3.5:35b         # LLM (~24 GB)
ollama pull mxbai-embed-large   # Embeddings (~670 MB)
ollama serve

3. Run PrivateGPT

# macOS / Linux
OPENAI_API_BASE=http://localhost:<llm-port>/v1 \
  OPENAI_EMBEDDING_API_BASE=http://localhost:<embedding-port>/v1 \
  private-gpt serve

# Windows (PowerShell)
$env:OPENAI_API_BASE = "http://localhost:<llm-port>/v1"
$env:OPENAI_EMBEDDING_API_BASE = "http://localhost:<embedding-port>/v1"
private-gpt serve

4. Open the UI

Go to http://localhost:8080/ui. The API is at http://localhost:8080 and follows the Anthropic API spec.

The UI is useful for:

  • Sending messages.
  • Selecting models from /v1/models.
  • Uploading documents.
  • Testing retrieval with citations.
  • Enabling tools per chat.
  • Configuring databases, MCP connectors, skills, and custom tools.
  • Inspecting requests and responses through the API Debugger.

This UI is a demonstrator, not the core product. Developers are expected to build their own applications on top of the API. That said, the UI is intentionally polished enough for demos, videos, internal pilots, and quick local usage.

Integrations