---
title: "gateway"
type: "tool"
slug: "portkey-ai-gateway"
canonical_url: "https://www.graphcanon.com/tools/portkey-ai-gateway"
github_url: "https://github.com/Portkey-AI/gateway"
homepage_url: "https://portkey.ai/features/ai-gateway"
stars: 12347
forks: 1181
primary_language: "TypeScript"
license: "MIT"
categories: ["ai-agents", "llm-frameworks", "inference-serving"]
tags: ["llms", "llm", "llm-gateway", "generative-ai", "ai-gateway", "model-router", "langchain"]
updated_at: "2026-07-07T18:32:05.766327+00:00"
---

# gateway

> A blazing fast AI Gateway with integrated guardrails, providing access to over 1600 LLMs.

Portkey-AI/gateway serves as an open-source gateway designed to integrate and route requests across more than 1600 language models efficiently. It supports quick integration, automatic retries, fallback mechanisms, load balancing, conditional routing, and built-in guardrails for enterprise-level AI deployments.

## Facts

- Repository: https://github.com/Portkey-AI/gateway
- Homepage: https://portkey.ai/features/ai-gateway
- Stars: 12,347 · Forks: 1,181 · Open issues: 215 · Watchers: 56
- Primary language: TypeScript
- License: MIT
- Last pushed: 2026-05-25T13:54:51+00:00

## Categories

- [AI Agents](/categories/ai-agents.md)
- [LLM Frameworks](/categories/llm-frameworks.md)
- [Inference & Serving](/categories/inference-serving.md)

## Tags

llms, llm, llm-gateway, generative-ai, ai-gateway, model-router, langchain

## Related tools

- [ECC](/tools/affaan-m-ecc.md) - The agent harness performance optimization system (★ 226,962)
- [hermes-agent](/tools/nousresearch-hermes-agent.md) - The self-improving AI agent built by Nous Research (★ 210,880)
- [AutoGPT](/tools/significant-gravitas-autogpt.md) - AutoGPT: Build, Deploy, and Run AI Agents (★ 185,417)
- [ollama](/tools/ollama-ollama.md) - Get up and running with Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. (★ 175,659)
- [prompts.chat](/tools/f-prompts-chat.md) - The world's largest open-source prompt library for AI (★ 165,019)
- [transformers](/tools/huggingface-transformers.md) - 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models (★ 162,347)
- [langflow](/tools/langflow-ai-langflow.md) - Langflow is a powerful platform for building and deploying AI-powered agents and workflows. (★ 151,298)
- [dify](/tools/langgenius-dify.md) - Production-ready platform for agentic workflow development (★ 148,070)

## README (excerpt)

```text
<p align="right">
   <strong>English</strong> | <a href="./.github/README.cn.md">中文</a> | <a href="./.github/README.jp.md">日本語</a>
</p>

> [!IMPORTANT]
> :rocket: Gateway 2.0 (Pre-Release) Portkey's core enterprise gateway is merging into open-source with our 2.0 release. You can try the pre-release branch [here](https://github.com/portkey-ai/gateway/tree/2.0.0).
> Read more about what's next for Portkey in our [**Series A announcement**](https://portkey.wiki/rohit-a).


<div align="center">

🆕 **[Portkey Models](https://github.com/Portkey-AI/models)** - Open-source LLM pricing for 2,300+ models across 40+ providers. [Explore →](https://portkey.ai/models)


# AI Gateway
#### Route to 250+ LLMs with 1 fast & friendly API

<img src="https://cfassets.portkey.ai/sdk.gif" width="550px" alt="Portkey AI Gateway Demo showing LLM routing capabilities" style="margin-left:-35px">

[Docs](https://portkey.wiki/gh-1) | [Enterprise](https://portkey.wiki/gh-2) | [Hosted Gateway](https://portkey.wiki/gh-3) | [Changelog](https://portkey.wiki/gh-4) | [API Reference](https://portkey.wiki/gh-5)








<a href="https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/quickcreate?stackName=portkey-gateway&templateURL=https://portkey-gateway-ec2-quicklaunch.s3.us-east-1.amazonaws.com/portkey-gateway-ec2-quicklaunch.template.yaml"><img src="https://img.shields.io/badge/Deploy_to_EC2-232F3E?style=for-the-badge&logo=amazonwebservices&logoColor=white" alt="Deploy to AWS EC2" width="105"/></a> 
</div>

<br/>

The [**AI Gateway**](https://portkey.wiki/gh-10) is designed for fast, reliable & secure routing to 1600+ language, vision, audio, and image models. It is a lightweight, open-source, and enterprise-ready solution that allows you to integrate with any language model in under 2 minutes.

- [x] **Blazing fast** (<1ms latency) with a tiny footprint (122kb)
- [x] **Battle tested**, with over 10B tokens processed everyday
- [x] **Enterprise-ready** with enhanced security, scale, and custom deployments

<br>

#### What can you do with the AI Gateway?
- Integrate with any LLM in under 2 minutes - [Quickstart](#quickstart-2-mins)
- Prevent downtimes through **[automatic retries](https://portkey.wiki/gh-11)** and **[fallbacks](https://portkey.wiki/gh-12)**
- Scale AI apps with **[load balancing](https://portkey.wiki/gh-13)** and **[conditional routing](https://portkey.wiki/gh-14)**
- Protect your AI deployments with **[guardrails](https://portkey.wiki/gh-15)**
- Go beyond text with **[multi-modal capabilities](https://portkey.wiki/gh-16)**
- Explore **[agentic workflow](https://portkey.wiki/gh-17)** integrations
- Manage MCP servers with enterprise auth & observability using **[MCP Gateway](https://portkey.ai/docs/product/mcp-gateway)**

<br><br>

> [!TIP]
> Starring this repo helps more developers discover the AI Gateway 🙏🏻
>
> 
> 
<br>


<br>

## Quickstart (2 mins)

### 1. Setup your AI Gateway

```bash
# Run the gateway locally (needs Node.js and npm)
npx @portkey-ai/gateway
```
> The Gateway is running on `http://localhost:8787/v1`
> 
> The Gateway Console is running on `http://localhost:8787/public/`

<sup>
Deployment guides:
&nbsp; <a href="https://portkey.wiki/gh-18"><img height="12" width="12" src="https://cfassets.portkey.ai/logo/dew-color.svg" /> Portkey Cloud (Recommended)</a>
&nbsp; <a href="./docs/installation-deployments.md#docker"><img height="12" width="12" src="https://cdn.simpleicons.org/docker/3776AB" /> Docker</a>
&nbsp; <a href="./docs/installation-deployments.md#nodejs-server"><img height="12" width="12" src="https://cdn.simpleicons.org/node.js/3776AB" /> Node.js</a>
&nbsp; <a href="./docs/installation-deployments.md#cloudflare-workers"><img height="12" width="12" src="https://cdn.simpleicons.org/cloudflare/3776AB" /> Cloudflare</a>
&nbsp; <a href="./docs/installation-deployments.md#replit"><img height="12" width="12" src="https://cdn.simpleicons.org/replit/3776AB" /> Replit</a>
&nbsp; <a href="./docs/ins
```

---

**Machine-readable endpoints**

- JSON: [`/api/graphcanon/tools/portkey-ai-gateway`](/api/graphcanon/tools/portkey-ai-gateway)
- LLM index: [/llms.txt](/llms.txt)
- Full corpus: [/llms-full.txt](/llms-full.txt)

_GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_