litellm vs PowerInfer
A neutral, constraint-first comparison - live GitHub stats and typed relationships, not marketing.
| litellm | PowerInfer | |
|---|---|---|
| Tagline | Python SDK and Proxy Server for calling over 100 LLM APIs in OpenAI format | High-speed Large Language Model Serving for Local Deployment |
| Stars | 53k | 9.6k |
| Forks | 9.5k | 585 |
| Open issues | 3.7k | 129 |
| Language | Python | C++ |
| License | Other | MIT |
| Last pushed | Jul 7, 2026 | May 11, 2026 |
| Categories | LLM Frameworks, Inference & Serving | Inference & Serving, LLM Frameworks |
litellm
A Python SDK along with a proxy server (AI Gateway) designed to facilitate the interaction with numerous large language models, providing functionalities like cost tracking, load balancing, guardrails, and logging across various providers such as Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM.
Python
PowerInfer
PowerInfer is a CPU/GPU LLM inference engine that utilizes activation locality to optimize performance on consumer-grade GPUs and CPUs.
C++