serve alternatives

Open-source alternatives to serve in Inference & Serving, LLM Frameworks, connected by typed relationships and shared categories - ranked with live GitHub stats, not marketing.

Head-to-head comparisons

Also in the same categories

Tools that solve adjacent problems in serve's categories.

ollama

ollama/ollama

Get up and running with Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

176k 17kGo

llm-frameworks

prompts.chat

f/prompts.chat

The world's largest open-source prompt library for AI

165k 21kHTML

llm-frameworks

transformers

huggingface/transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models

162k 34kPython

llm-frameworks

open-webui

open-webui/open-webui

User-friendly AI Interface (Supports Ollama, OpenAI API, ...)

145k 21kPython

llm-frameworksinference-serving

awesome-llm-apps

Shubhamsaboo/awesome-llm-apps

100+ AI Agent & RAG apps you can actually run — clone, customize, ship.

117k 17kPython

llm-frameworks

LLMs-from-scratch

rasbt/LLMs-from-scratch

Implement a ChatGPT-like LLM in PyTorch from scratch

99k 15kJupyter Notebook

llm-frameworks

TradingAgents

TauricResearch/TradingAgents

TradingAgents: Multi-Agents LLM Financial Trading Framework

92k 18kPython

llm-frameworks

caveman

JuliusBrussee/caveman

Cuts 65% of tokens in AI coding agent responses.

86k 4.8kJavaScript

llm-frameworks

vllm

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

86k 19kPython

inference-serving

ragflow

infiniflow/ragflow

RAGFlow: A powerful Retrieval-Augmented Generation with enhanced Agent capabilities in Go

85k 9.8kGo

llm-frameworks

llm-course

mlabonne/llm-course

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

81k 9.4k

llm-frameworks

LlamaFactory

hiyouga/LlamaFactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs

73k 8.9kPython

llm-frameworks

gpt_academic

binary-husky/gpt_academic

GPT 学术优化 (GPT Academic)

71k 8.4kPython

llm-frameworks

rtk

rtk-ai/rtk

CLI proxy reducing LLM token consumption by 60-90%

69k 4.3kRust

inference-serving

MetaGPT

FoundationAgents/MetaGPT

MetaGPT: The Multi-Agent Framework

69k 8.8kPython

llm-frameworks

unsloth

unslothai/unsloth

Unsloth Studio is a web UI for training and running open models locally.

68k 6.1kPython

llm-frameworksinference-serving

anything-llm

Mintplex-Labs/anything-llm

Stop renting your intelligence. Own it with AnythingLLM.

63k 6.9kJavaScript

inference-serving

TrendRadar

sansan0/TrendRadar

AI-driven public opinion & trend monitor

60k 25kPython

llm-frameworks

mem0

mem0ai/mem0

Universal memory layer for AI Agents

60k 7.0kPython

inference-serving

autogen

microsoft/autogen

AutoGen is a programming framework for creating multi-agent AI applications.

60k 9.0kPython

llm-frameworks

llm-app

pathwaycom/llm-app

Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data.

59k 1.4kJupyter Notebook

llm-frameworks

context7

upstash/context7

Up-to-date code documentation for LLMs and AI code editors

59k 2.8kTypeScript

llm-frameworks

headroom

headroomlabs-ai/headroom

Context compression layer for AI agents

57k 4.2kPython

llm-frameworks

litellm

BerriAI/litellm

Python SDK and Proxy Server for calling over 100 LLM APIs in OpenAI format

53k 9.5kPython

llm-frameworksinference-serving