--- title: "GPTRouter" type: "tool" slug: "writesonic-gptrouter" canonical_url: "https://www.graphcanon.com/tools/writesonic-gptrouter" github_url: "https://github.com/Writesonic/GPTRouter" homepage_url: "https://gpt-router.writesonic.com/" stars: 454 forks: 38 primary_language: "TypeScript" license: "MIT" categories: ["inference-serving", "llm-frameworks"] tags: ["llmops", "llm", "langchain", "anthropic", "llama-index", "azure-openai", "cohere", "google-gemini"] updated_at: "2026-07-07T18:50:14.889581+00:00" --- # GPTRouter > Smoothly Manage Multiple LLMs and Image Models GPTRouter enables seamless management of multiple AI models, ensuring faster responses and non-stop reliability through smart fallbacks and automatic retries. ## Facts - Repository: https://github.com/Writesonic/GPTRouter - Homepage: https://gpt-router.writesonic.com/ - Stars: 454 · Forks: 38 · Open issues: 10 · Watchers: 10 - Primary language: TypeScript - License: MIT - Last pushed: 2024-04-10T11:14:59+00:00 ## Categories - [Inference & Serving](/categories/inference-serving.md) - [LLM Frameworks](/categories/llm-frameworks.md) ## Tags llmops, llm, langchain, anthropic, llama-index, azure-openai, cohere, google-gemini ## Related tools - [ollama](/tools/ollama-ollama.md) - Get up and running with Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. (★ 175,659) - [prompts.chat](/tools/f-prompts-chat.md) - The world's largest open-source prompt library for AI (★ 165,019) - [transformers](/tools/huggingface-transformers.md) - 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models (★ 162,347) - [open-webui](/tools/open-webui-open-webui.md) - User-friendly AI Interface (Supports Ollama, OpenAI API, ...) (★ 144,575) - [awesome-llm-apps](/tools/shubhamsaboo-awesome-llm-apps.md) - 100+ AI Agent & RAG apps you can actually run — clone, customize, ship. (★ 116,702) - [LLMs-from-scratch](/tools/rasbt-llms-from-scratch.md) - Implement a ChatGPT-like LLM in PyTorch from scratch (★ 98,711) - [TradingAgents](/tools/tauricresearch-tradingagents.md) - TradingAgents: Multi-Agents LLM Financial Trading Framework (★ 91,610) - [caveman](/tools/juliusbrussee-caveman.md) - Cuts 65% of tokens in AI coding agent responses. (★ 86,150) ## README (excerpt) ```text # 🚀 GPTRouter **Your AI Model Gateway - Smoothly Manage Multiple LLMs and Image Models, Speed Up Responses, and Ensure Non-Stop Reliability.** ## 🌐 Why GPTRouter? At Writesonic, after three years of navigating the world of large language models, we identified key challenges and built GPTRouter to solve them. ### Solving Real-World Challenges: 1. **Model Independence**: Don't put all your eggs in one basket. GPTRouter lets you break free from the limitations of relying on just one AI model like OpenAI. If one model is down, GPTRouter keeps you up and running by seamlessly switching to another. 2. **Beat the Latency**: Slow response times? Not anymore. GPTRouter is designed to tackle latency issues, especially with hefty models like GPT-4. Experience a smoother, faster user interaction without delays. 3. **Diverse Model Integration**: Why settle for one when you can have more? GPTRouter supports multiple language and image generation models, providing fallback options so your system remains robust and versatile. ### Key Features: - **🌐 Universal API**: One API to connect them all. Easily switch between models like OpenAI, Azure OpenAI, Anthropic, Replicate, Stable Diffusion, Cohere, and more. - **🔀 Smart Fallbacks**: Keep your services uninterrupted. GPTRouter automatically switches to alternative models if your primary choice is unavailable. - **🔄 Automatic Retries**: GPTRouter intelligently retries failed requests, reducing manual effort and improving reliability. - **⏱️ Fast and Responsive**: Designed to reduce latency, GPTRouter ensures your interactions with AI models are quick and efficient. ### Supported Models: | Supported Models | Completion | Streaming | Async Completion | Async Streaming | | ---------------- | :----------------: | :----------------------: | :----------------: | :----------------------: | | OpenAI | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | | Azure OpenAI | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | | Anthropic | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | | Replicate | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | | Stable Diffusion | :white_check_mark: | :heavy_exclamation_mark: | :white_check_mark: | :heavy_exclamation_mark: | | Dalle-3 | :white_check_mark: | :heavy_exclamation_mark: | :white_check_mark: | :heavy_exclamation_mark: | | Cohere | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | | More to come | :clock930: | :clock930: | :clock930: | :clock930: | :heavy_exclamation_mark: Streaming not applicable to Image Models :clock930: Coming Soon :sparkles: **Contributors Welcome!** :sparkles: ## Quick Start Ready to get started? Here's how: --- ### Prerequisites **Getting The Server Running** - To run the GPTRouter server locally, follow the steps [here](https://gpt-router.writesonic.com/docs/Getting%20Started/Developing%20Locally) - Alternatively, use our **Preview Deployment** with the baseURL `https://gpt-router-preview.writesonic.com/`. Get your API key by filling out the form [here](https://gpt-router.writesonic.com/#APIKey). --- Once the Server is running, you can integrate GPTRouter into your application using our Python SDK or via the API [Docs](https://gpt-router-preview.writesonic.com/docs/static/index.html). Meanwhile, we are working on JS and other clients and are looking for contributors to help out. ### Using the Python SDK Install GPTRouter using pip: ```bash pi ``` --- **Machine-readable endpoints** - JSON: [`/api/graphcanon/tools/writesonic-gptrouter`](/api/graphcanon/tools/writesonic-gptrouter) - LLM index: [/llms.txt](/llms.txt) - Full corpus: [/llms-full.txt](/llms-full.txt) _GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_