{"data":{"slug":"zylon-ai-private-gpt","name":"private-gpt","tagline":"Complete API layer for private AI applications on local models","github_url":"https://github.com/zylon-ai/private-gpt","owner":"zylon-ai","repo":"private-gpt","owner_avatar_url":"https://avatars.githubusercontent.com/u/143802295?v=4","primary_language":"Python","stars":57329,"forks":7598,"topics":["ai","ai-tools","on-premise"],"archived":false,"github_pushed_at":"2026-07-10T12:11:19+00:00","maintenance_label":"Very active","url":"https://www.graphcanon.com/tools/zylon-ai-private-gpt","markdown_url":"https://www.graphcanon.com/tools/zylon-ai-private-gpt.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/zylon-ai-private-gpt","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=zylon-ai-private-gpt","description":"Complete API layer for private AI applications on local models: RAG, skills, tools, MCP, text-to-sql, and more. Works with any OpenAI-compatible inference server.","homepage_url":"https://www.zylon.ai/private-gpt","license":"Apache-2.0","open_issues":5,"watchers":459,"ai_summary":"PrivateGPT offers an open-source API that interfaces with OpenAI-compatible inference servers to build private, on-premise AI applications. It provides RAG, skills, tools, text-to-SQL functionalities among others.","readme_excerpt":"<div align=\"center\">\n\n**PrivateGPT is the open-source API layer that turns local models into production AI applications.**\n\n\n\n\n\n\n<a href=\"https://trendshift.io/repositories/8691\" target=\"_blank\"><img src=\"https://trendshift.io/api/badge/repositories/8691\" alt=\"zylon-ai%2Fprivate-gpt | Trendshift\" style=\"width: 250px; height: 55px;\" width=\"250\" height=\"55\"/></a>\n\n</div>\n\n---\n\nRunning a model locally is only the first step. To build useful AI applications you need a set of higher-level building blocks. PrivateGPT provides that layer as an open-source API following the Claude API model — so you can build private AI products without rebuilding the same backend primitives from scratch, and without depending on cloud APIs.\n\nProduction-tested: [PrivateGPT powers Zylon](#privategpt-vs-zylon), the on-premise AI platform providing Private AI to enterprises across the globe.\n\n\n```text\nYour app / agent / workflow / UI\n              |\n        PrivateGPT API\n              |\nOpenAI-compatible inference server (Ollama, llama.cpp, vLLM, …)              \n```\n\n> PrivateGPT does **not** run models itself. It connects to any OpenAI-compatible inference server via `OPENAI_API_BASE`. If it implements `/v1/chat/completions` and `/v1/models`, it works.\n\nPrivateGPT ships a built-in workbench UI for testing and demos, available at `/ui`. The API is the actual product.\n\n---\n\n## What PrivateGPT gives you\n\n- Standard messages API (streaming, async, token counting)\n- File and artifact ingestion\n- Retrieval with citations and agentic RAG\n- Built-in tools mirroring the Claude API (web search, web fetch, code execution)\n- Custom tools and MCP connectors\n- Structured access to databases and CSVs\n- Embeddings and orchestration\n\n---\n\n## Quickstart\n\n> For Docker, full installation options, and model configuration see the [full Quickstart guide](https://docs.privategpt.dev/getting-started/quickstart).\n\n**Prerequisites:** You need a running OpenAI-compatible LLM server. [Ollama](https://docs.privategpt.dev/providers/ollama) is the easiest starting point.\n\n**1. Install PrivateGPT**\n\n```bash\n# macOS\nbrew tap zylon-ai/tap\nbrew install private-gpt\n```\n\n```bash\n# Linux\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n\nuv tool install --python 3.11 \\\n  --find-links https://wheels.privategpt.dev/packages/ \\\n  \"private-gpt[core]\"\n```\n\n```powershell\n# Windows\npowershell -ExecutionPolicy ByPass -c \"irm https://astral.sh/uv/install.ps1 | iex\"\n\nuv tool install --python 3.11 `\n  --find-links https://wheels.privategpt.dev/packages/ `\n  \"private-gpt[core]\"\n```\n\n**2. Start your LLM server**\n\n```bash\n# Example with Ollama\nollama pull qwen3.5:35b         # LLM (~24 GB)\nollama pull mxbai-embed-large   # Embeddings (~670 MB)\nollama serve\n```\n\n**3. Run PrivateGPT**\n\n```bash\n# macOS / Linux\nOPENAI_API_BASE=http://localhost:<llm-port>/v1 \\\n  OPENAI_EMBEDDING_API_BASE=http://localhost:<embedding-port>/v1 \\\n  private-gpt serve\n```\n\n```powershell\n# Windows (PowerShell)\n$env:OPENAI_API_BASE = \"http://localhost:<llm-port>/v1\"\n$env:OPENAI_EMBEDDING_API_BASE = \"http://localhost:<embedding-port>/v1\"\nprivate-gpt serve\n```\n\n**4. Open the UI**\n\nGo to [http://localhost:8080/ui](http://localhost:8080/ui). The API is at `http://localhost:8080` and follows the [Anthropic API](https://docs.privategpt.dev/api-reference/api-reference) spec.\n\n<img src=\"./fern/docs/assets/ui.png\"/>\n\nThe UI is useful for:\n\n- Sending messages.\n- Selecting models from /v1/models.\n- Uploading documents.\n- Testing retrieval with citations.\n- Enabling tools per chat.\n- Configuring databases, MCP connectors, skills, and custom tools.\n- Inspecting requests and responses through the API Debugger. \n\nThis UI is a demonstrator, not the core product. Developers are expected to build their own applications on top of the API. That said, the UI is intentionally polished enough for demos, videos, internal pilots, and quick local usage.\n\n---\n\n## Integrations\n\n| | | |\n|:-------------------------:|:-------------------------:|:-------------------","github_created_at":"2023-05-02T09:15:31+00:00","created_at":"2026-07-11T10:45:33.587758+00:00","updated_at":"2026-07-11T12:21:37.555951+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"}],"tags":[{"slug":"text-to-sql","name":"text-to-sql"},{"slug":"ai","name":"ai"},{"slug":"on-premise","name":"on-premise"},{"slug":"tools","name":"tools"},{"slug":"rag","name":"rag"},{"slug":"local-models","name":"local-models"},{"slug":"mcp","name":"mcp"},{"slug":"ai-tools","name":"ai-tools"}],"trust":{"provenance":{"is_fork":false,"github_id":635240594,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T10:45:34.272Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":4,"days_since_push":0,"last_release_at":"2026-06-18T13:56:00Z"},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T10:45:40.945Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T12:21:08.108Z"},"deploy":{"source":"dockerfile:Dockerfile","self_host":true,"observed_at":"2026-07-11T12:21:08.108Z","managed_saas":false},"has_cli":{"value":true,"source":"pyproject.toml:[project.scripts]","observed_at":"2026-07-11T12:21:08.108Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-07-11T12:21:08.108Z"},"has_docker":{"value":true,"source":"dockerfile:Dockerfile","observed_at":"2026-07-11T12:21:08.108Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-07-11T12:21:08.108Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"min_ram_gb":8,"requires_docker":true},"constraints":{"min_ram_gb":8,"requires_docker":true},"when_to_use":["- You need to deploy and operationalize your own locally-run models without relying on cloud APIs.","- Your application requires a high degree of customization beyond what existing cloud services offer.","- Security and data privacy are paramount, necessitating an entirely on-premise solution."],"when_not_to_use":["- You prefer simplicity and ease-of-use over full control; PrivateGPT requires more setup than using direct cloud-based AI services.","- Your project does not involve running models locally but strictly relies on public cloud resources for inference server operations.","- You do not have the technical capability to run an OpenAI-compatible inference server or manage local infrastructure effectively."],"source":"enrich:decision_facts","observed_at":"2026-07-11T12:21:37.063Z"},"constraint_facets":{"min_ram_gb":8,"requires_docker":true},"decision_summary":[{"label":"Requirements","value":"Min 8 GB RAM; Requires Docker"},{"label":"Adopt for","value":"PrivateGPT provides a comprehensive API layer to build private, on-premise AI applications leveraging local OpenAI-compatible inference servers. It offers features such as RAG, skills, tools, text-to-SQL functionalities,"}]}}