{"data":{"slug":"ferro-labs-ai-gateway","name":"ai-gateway","tagline":"Unified AI Gateway for multiple LLMs with caching, guardrails, A/B testing, and cost controls","github_url":"https://github.com/ferro-labs/ai-gateway","owner":"ferro-labs","repo":"ai-gateway","owner_avatar_url":"https://avatars.githubusercontent.com/u/263849603?v=4","primary_language":"Go","stars":256,"forks":35,"topics":["ai-gateway","ai-infrastructure","guardrails","litellm","llm","llm-cost","llm-gateway","llm-proxy","llm-strategy","llmops","mcp","openai-compatible","pii-detection","prompt-management","semantic-cache","token-saver"],"archived":false,"github_pushed_at":"2026-09-10T04:50:20+00:00","maintenance_label":"Very active","stars_delta_30d":37,"url":"https://www.graphcanon.com/tools/ferro-labs-ai-gateway","markdown_url":"https://www.graphcanon.com/tools/ferro-labs-ai-gateway.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/ferro-labs-ai-gateway","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=ferro-labs-ai-gateway","description":"Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.","homepage_url":"https://docs.ferrolabs.ai","license":"Apache-2.0","open_issues":68,"watchers":6,"ai_summary":"Ferro Labs' AI Gateway provides access to 30+ LLM services, including OpenAI, Anthropic, Bedrock, Azure. It supports caching, guardrail implementation, A/B testing, and cost management.","readme_excerpt":"## Quick Start\n\nUnder two minutes from install to first response.\n\n| Platform / tool | Install |\n|:---|:---|\n| macOS, Linux | `curl -fsSL https://get.ferrolabs.ai \\| sh` |\n| Windows | `irm https://get.ferrolabs.ai/install.ps1 \\| iex` |\n| Homebrew | `brew install ferro-labs/tap/ferrogw` |\n| Scoop | `scoop bucket add ferrolabs https://github.com/ferro-labs/homebrew-tap` then `scoop install ferrogw` |\n| npm | `npm install -g ferrogw` |\n| Python | `uv tool install ferrogw` |\n| Docker | `docker run -p 8080:8080 ghcr.io/ferro-labs/ai-gateway:latest` |\n| Go | `go install github.com/ferro-labs/ai-gateway/cmd/ferrogw@latest` — builds from source, without the dashboard |\n| Debian, RPM, Alpine | `.deb`, `.rpm` and `.apk` packages on the [releases page](https://github.com/ferro-labs/ai-gateway/releases/latest) |\n\nThen go from nothing to a served request:\n\n```bash\nexport OPENAI_API_KEY=sk-your-key     # ferrogw init detects this and writes the matching target\nferrogw init                          # writes config.yaml, prints your master key\nexport GATEWAY_CONFIG=./config.yaml   # the server reads a config file only when this is set\nexport MASTER_KEY=fgw_your-master-key # the key ferrogw init printed\nferrogw serve                         # starts the server on :8080\n```\n\n`ferrogw init` prints the master key **once** and never writes it to disk — save\nit yourself, in your `.env` file or a secret manager.\n\nDocker runs the server itself, so there is no `ferrogw init` to print a master\nkey — choose your own. Pass both variables by name so their values stay off the\ncommand line, where `ps` would show them:\n\n```bash\nexport OPENAI_API_KEY=sk-your-key\nexport MASTER_KEY=fgw-any-strong-secret-you-choose\ndocker run -p 8080:8080 -e OPENAI_API_KEY -e MASTER_KEY ghcr.io/ferro-labs/ai-gateway:latest\n```\n\nPassing them by name keeps the values out of process arguments only — they are\nstill readable in the container's environment and through `docker inspect`, so\nuse Docker secrets or your platform's secret storage in production.\n\n<div align=\"center\">\n  <img src=\"docs/demo.gif\" alt=\"Installing Ferro Labs AI Gateway with one command, running ferrogw init, starting the server, and getting a completed chat response\" width=\"100%\" />\n</div>\n\n---\n\n### Docker Compose\n\nThree Compose files in `deploy/` follow the standard override pattern — a shared\nbase, a dev override that builds from source, and a prod override with a pinned\ntag, health check, and resource limits. Run everything from the repository root:\n\n```bash\nmake up             # dev: builds from source\nIMAGE_TAG=v1.4.5 CORS_ORIGINS=https://your-domain.com make up-prod\nmake down           # tears down either\n```\n\nOne container serves both the API and the dashboard — no second image, no second\norigin. Provider keys go in a repository-root `.env` or the environment.\n[deploy/README.md](deploy/README.md) has the full reference, including a\nself-contained PostgreSQL pairing and the fullstack observability stack.\n\n---\n\n## License\n\nApache 2.0 — see [LICENSE](LICENSE).","github_created_at":"2026-02-25T07:46:16+00:00","created_at":"2026-07-15T10:42:53.705774+00:00","updated_at":"2026-09-20T04:25:55.8317+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"ai-gateway","name":"ai-gateway"},{"slug":"litellm","name":"litellm"},{"slug":"llm-cost","name":"llm-cost"},{"slug":"llm-proxy","name":"llm-proxy"},{"slug":"openai-compatible","name":"openai-compatible"}],"trust":{"provenance":{"is_fork":false,"github_id":1166416848,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-12T06:00:10.042Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":30,"days_since_push":2,"last_release_at":"2026-09-05T08:01:03Z","stars_delta_30d":37,"open_issues_delta_30d":5},"security_summary":{"status":"findings","scanner":"osv@v1","low_count":7,"high_count":0,"last_scan_at":"2026-07-15T10:42:55.041Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-12T06:00:10.704Z"},"languages":{"value":["go"],"source":"github.language","observed_at":"2026-09-12T06:00:10.704Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-09-12T06:00:10.704Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["When you need to integrate more than 30 different LLM services including OpenAI and Anthropic","If you require extensive support for caching and rate limiting","For projects that need A/B testing capabilities across various AI providers","When cost controls are essential, offering detailed tracking and management options"],"when_not_to_use":["If your project only involves one or two LLMs which does not necessitate the gateway's broad compatibility features","For small-scale projects that do not require comprehensive cost analysis tools","When custom integration for specific guardrails is required, as ai-gateway offers generalized settings"],"source":"enrich:decision_facts","observed_at":"2026-07-17T08:20:02.870Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"ai-gateway from Ferro Labs supports over 30 LLMs with integrated caching, guardrails, A/B testing, and cost controls, making it ideal for managing multiple language models in a production environment."},{"label":"License detail","value":"Apache-2.0 - a permissive free software license"}]}}