--- title: "Inference & Serving" type: "category" slug: "inference-serving" canonical_url: "https://www.graphcanon.com/categories/inference-serving" tool_count: 60 --- # Inference & Serving 60 tools in this category. ## Tools - [AutoGPT](/tools/significant-gravitas-autogpt.md) - AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on w (★ 185,418) - [transformers](/tools/huggingface-transformers.md) - 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, (★ 162,346) - [TradingAgents](/tools/tauricresearch-tradingagents.md) - TradingAgents: Multi-Agents LLM Financial Trading Framework (★ 91,606) - [lobehub](/tools/lobehub-lobehub.md) - 🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI (★ 79,579) - [daytona](/tools/daytonaio-daytona.md) - Daytona is a Secure and Elastic Infrastructure for Running AI-Generated Code (★ 72,268) - [MetaGPT](/tools/foundationagents-metagpt.md) - 🌟 The Multi-Agent Framework: First AI Software Company, Towards Natural Language Programming (★ 69,249) - [llm-app](/tools/pathwaycom-llm-app.md) - Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, (★ 59,097) - [crewAI](/tools/crewaiinc-crewai.md) - Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work toge (★ 55,092) - [Flowise](/tools/flowiseai-flowise.md) - Build AI Agents, Visually (★ 54,383) - [litellm](/tools/berriai-litellm.md) - Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and (★ 52,877) - [goose](/tools/aaif-goose-goose.md) - an open source, extensible AI agent that goes beyond code suggestions - install, execute, edit, and test with any LLM (★ 50,773) - [awesome-claude-code](/tools/hesreallyhim-awesome-claude-code.md) - A hand-picked collection of the finest of resources for the most awesome of agents, Claude Code, the undisputed champion of coding companion (★ 48,956) - [LocalAI](/tools/mudler-localai.md) - LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required. (★ 47,388) - [JeecgBoot](/tools/jeecgboot-jeecgboot.md) - AI 低代码平台「低代码 + 零代码」双驱动！低代码可一键生成前后端代码;零代码可 5 分钟搭建系统;AI Skills 一句话画流程、设计表单、生成整套系统。内置 AI聊天、知识库、流程编排、MCP插件等，兼容主流大模型。引领「AI 生成 → 在线配置 → 代码生成 → 手工合 (★ 46,959) - [milvus](/tools/milvus-io-milvus.md) - Milvus is a high-performance, cloud-native vector database built for scalable vector ANN search (★ 45,121) - [agno](/tools/agno-agi-agno.md) - Build, run, and manage agent platforms. (★ 41,038) - [minds](/tools/mindsdb-minds.md) - Delegate anything. It comes back done. (★ 39,381) - [quivr](/tools/quivrhq-quivr.md) - Opiniated RAG for integrating GenAI in your apps 🧠 Focus on your product rather than the RAG. Easy integration in existing products with (★ 39,187) - [agents](/tools/wshobson-agents.md) - Multi-harness agentic plugin marketplace for Claude Code, Codex CLI, Cursor, OpenCode, GitHub Copilot, and Gemini CLI (★ 37,637) - [oh-my-claudecode](/tools/yeachan-heo-oh-my-claudecode.md) - Teams-first Multi-agent orchestration for Claude Code (★ 37,515) - [langextract](/tools/google-langextract.md) - A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visua (★ 37,033) - [langgraph](/tools/langchain-ai-langgraph.md) - Build resilient agents. (★ 36,718) - [1Panel](/tools/1panel-dev-1panel.md) - 🔥 1Panel is a modern, open-source VPS control panel — and the only one with native AI agent support. Run Ollama models, deploy OpenClaw age (★ 36,131) - [CopilotKit](/tools/copilotkit-copilotkit.md) - The Frontend Stack for Agents & Generative UI. React, Angular, Mobile, Slack, and more. Makers of the AG-UI Protocol (★ 35,824) - [Vane](/tools/itzcrazykns-vane.md) - Vane is an AI-powered answering engine. (★ 35,593) - [qdrant](/tools/qdrant-qdrant.md) - Qdrant - High-performance, massive-scale Vector Database and Vector Search Engine for the next generation of AI. Also available in the cloud (★ 33,007) - [prompt-optimizer](/tools/linshenkx-prompt-optimizer.md) - An AI prompt optimizer for writing better prompts and getting better AI results. (★ 31,859) - [langfuse](/tools/langfuse-langfuse.md) - 🪢 Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playground, datasets. Integrates with OpenTele (★ 30,626) - [SillyTavern](/tools/sillytavern-sillytavern.md) - LLM Frontend for Power Users. (★ 30,399) - [sglang](/tools/sgl-project-sglang.md) - SGLang is a high-performance serving framework for large language models and multimodal models. (★ 30,041) - [llmfit](/tools/alexsjones-llmfit.md) - Hundreds of models & providers. One command to find what runs on your hardware. (★ 29,186) - [FastGPT](/tools/labring-fastgpt.md) - FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing (★ 28,855) - [graphiti](/tools/getzep-graphiti.md) - Build Real-Time Knowledge Graphs for AI Agents (★ 28,463) - [Scrapegraph-ai](/tools/scrapegraphai-scrapegraph-ai.md) - Python scraper based on AI (★ 28,143) - [vibe-kanban](/tools/bloopai-vibe-kanban.md) - Get 10X more out of Claude Code, Codex or any coding agent (★ 27,294) - [repomix](/tools/yamadashy-repomix.md) - 📦 Repomix is a powerful tool that packs your entire repository into a single, AI-friendly file. Perfect for when you need to feed your code (★ 26,936) - [mlflow](/tools/mlflow-mlflow.md) - The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and (★ 26,921) - [OpenCLI](/tools/jackwener-opencli.md) - Make Any Website into CLI & Use your logged-in browser by AI agent. (★ 26,226) - [free-llm-api-resources](/tools/cheahjs-free-llm-api-resources.md) - A list of free LLM inference resources accessible via API. (★ 26,147) - [mastra](/tools/mastra-ai-mastra.md) - Mastra is the modern TypeScript framework for AI-powered applications and agents. (★ 25,900) - [heretic](/tools/p-e-w-heretic.md) - Fully automatic censorship removal for language models (★ 25,892) - [kotaemon](/tools/cinnamon-kotaemon.md) - An open-source RAG-based tool for chatting with your documents. (★ 25,524) - [ai](/tools/vercel-ai.md) - The AI Toolkit for TypeScript. From the creators of Next.js, the AI SDK is a free open-source library for building AI-powered applications a (★ 25,400) - [nocobase](/tools/nocobase-nocobase.md) - NocoBase is an open-source AI + no-code platform for building business systems fast. Instead of generating everything from scratch, AI works (★ 23,266) - [activepieces](/tools/activepieces-activepieces.md) - AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents (★ 23,161) - [GenAI_Agents](/tools/nirdiamant-genai-agents.md) - 50+ tutorials and implementations for Generative AI Agent techniques, from basic conversational bots to complex multi-agent systems. (★ 23,029) - [mlc-llm](/tools/mlc-ai-mlc-llm.md) - Universal LLM Deployment Engine with ML Compilation (★ 22,917) - [vector](/tools/vectordotdev-vector.md) - A high-performance observability data pipeline. (★ 22,152) - [pgvector](/tools/pgvector-pgvector.md) - Open-source vector similarity search for Postgres (★ 22,104) - [NemoClaw](/tools/nvidia-nemoclaw.md) - Run agents like Hermes and OpenClaw more securely inside NVIDIA OpenShell with managed inference (★ 21,640) - [adk-python](/tools/google-adk-python.md) - An open-source, code-first Python toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control. (★ 20,508) - [cua](/tools/trycua-cua.md) - Open-source infrastructure for Computer-Use Agents. Sandboxes, SDKs, and benchmarks to train and evaluate AI agents that can control full de (★ 19,431) - [agent-zero](/tools/agent0ai-agent-zero.md) - Agent Zero AI framework (★ 18,360) - [openfang](/tools/rightnow-ai-openfang.md) - Open-source Agent Operating System (★ 17,979) - [Megatron-LM](/tools/nvidia-megatron-lm.md) - Ongoing research training transformer models at scale (★ 16,987) - [LangBot](/tools/langbot-app-langbot.md) - Production-grade platform for building agentic IM bots - 生产级多平台智能机器人开发平台/ Agent、知识库编排、插件系统 / Bots for Discord / Slack / LINE / Telegram / We (★ 16,740) - [weaviate](/tools/weaviate-weaviate.md) - Weaviate is an open-source vector database that stores both objects and vectors, allowing for the combination of vector search with structur (★ 16,534) - [hermes-webui](/tools/nesquena-hermes-webui.md) - Hermes WebUI: The best way to use Hermes Agent from the web or from your phone! (★ 15,618) - [trigger.dev](/tools/triggerdotdev-trigger-dev.md) - Trigger.dev – build and deploy fully‑managed AI agents and workflows (★ 15,594) - [Memori](/tools/memorilabs-memori.md) - Memori is agent-native memory infrastructure. A LLM-agnostic layer that turns agent execution and conversation into structured, persistent s (★ 15,542) --- **Machine-readable endpoints** - JSON: [`/api/graphcanon/categories/inference-serving`](/api/graphcanon/categories/inference-serving) - LLM index: [/llms.txt](/llms.txt) - Full corpus: [/llms-full.txt](/llms-full.txt) _GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_