--- title: "The local / self-hosted LLM stack" type: "stack" slug: "local-llm" canonical_url: "https://www.graphcanon.com/stacks/local-llm" --- # The local / self-hosted LLM stack Running open models yourself for privacy, cost, or control. The stack is a serving runtime, optional fine-tuning, and supporting tooling. ## 1. Inference & Serving Inference runtime - serve open-weight models efficiently on your hardware. **When not to use:** Self-hosting rarely beats a hosted API on cost until you have steady, high-volume traffic. - [AutoGPT](/tools/significant-gravitas-autogpt.md) - AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on w (โ˜… 185,418) - [transformers](/tools/huggingface-transformers.md) - ๐Ÿค— Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, (โ˜… 162,346) - [TradingAgents](/tools/tauricresearch-tradingagents.md) - TradingAgents: Multi-Agents LLM Financial Trading Framework (โ˜… 91,606) - [lobehub](/tools/lobehub-lobehub.md) - ๐Ÿคฏ LobeHub is your Chief Agent Operator, organizing your agents into 7ร—24 operations by hiring, scheduling, and reporting on your entire AI (โ˜… 79,579) ## 2. Model Training Fine-tuning - adapt an open model to your domain with LoRA or full fine-tunes. **When not to use:** Try prompting and RAG first; fine-tuning is the answer to style/format, not missing knowledge. - [transformers](/tools/huggingface-transformers.md) - ๐Ÿค— Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, (โ˜… 162,346) - [LLMs-from-scratch](/tools/rasbt-llms-from-scratch.md) - Implement a ChatGPT-like LLM in PyTorch from scratch, step by step (โ˜… 98,709) - [vllm](/tools/vllm-project-vllm.md) - A high-throughput and memory-efficient inference and serving engine for LLMs (โ˜… 85,611) - [llm-course](/tools/mlabonne-llm-course.md) - Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks. (โ˜… 80,726) ## 3. Developer Tools Tooling & gateways - a unified API in front of your local models. **When not to use:** Direct calls are fine for a single local model with one consumer. - [JavaGuide](/tools/snailclimb-javaguide.md) - Java ้ข่ฏ• & ๅŽ็ซฏ้€š็”จ้ข่ฏ•ๆŒ‡ๅ—๏ผŒ่ฆ†็›–่ฎก็ฎ—ๆœบๅŸบ็ก€ใ€ๆ•ฐๆฎๅบ“ใ€ๅˆ†ๅธƒๅผใ€้ซ˜ๅนถๅ‘ใ€็ณป็ปŸ่ฎพ่ฎกไธŽ AI ๅบ”็”จๅผ€ๅ‘ (โ˜… 156,863) - [dify](/tools/langgenius-dify.md) - Production-ready platform for agentic workflow development. (โ˜… 148,066) - [firecrawl](/tools/firecrawl-firecrawl.md) - The API to search, scrape, and interact with the web at scale. ๐Ÿ”ฅ (โ˜… 147,107) - [browser-use](/tools/browser-use-browser-use.md) - ๐ŸŒ Make websites accessible for AI agents. Automate tasks online with ease. (โ˜… 103,311) --- **Machine-readable endpoints** - JSON: [`/api/graphcanon/categories`](/api/graphcanon/categories) - LLM index: [/llms.txt](/llms.txt) - Full corpus: [/llms-full.txt](/llms-full.txt) _GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_