๐ 2-10x accuracy improvements on reasoning tasks with zero training
๐ค HuggingFace Space โข ๐ Colab Demo โข ๐ฌ Discussions
--- **OptiLLM** is an OpenAI API-compatible optimizing inference proxy that implements 20+ state-of-the-art techniques to dramatically improve LLM accuracy and performance on reasoning tasks - without requiring any model training or fine-tuning. It is possible to beat the frontier models using these techniques across diverse tasks by doing additional compute at inference time. A good example of how to combine such techniques together is the [CePO approach](optillm/cepo) from Cerebras. ## โจ Key Features - **๐ฏ Instant Improvements**: 2-10x better accuracy on math, coding, and logical reasoning - **๐ Drop-in Replacement**: Works with any OpenAI-compatible API endpoint - **๐ง 20+ Optimization Techniques**: From simple best-of-N to advanced MCTS and planning - **๐ฆ Zero Training Required**: Just proxy your existing API calls through OptiLLM - **โก Production Ready**: Used in production by companies and researchers worldwide - **๐ Multi-Provider**: Supports OpenAI, Anthropic, Google, Cerebras, and 100+ models via LiteLLM ## ๐ Quick Start Get powerful reasoning improvements in 3 simple steps: ```bash # 1. Install OptiLLM pip install optillm # 2. Start the server export OPENAI_API_KEY="your-key-here" optillm # 3. Use with any OpenAI client - just change the model name! ``` ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1") # Add 'moa-' prefix for Mixture of Agents optimization response = client.chat.completions.create( model="moa-gpt-4o-mini", # This gives you GPT-4o performance from GPT-4o-mini! messages=[{"role": "user", "content": "Solve: If 2x + 3 = 7, what is x?"}] ) ``` **Before OptiLLM**: "x = 1" โ **After OptiLLM**: "Let me work through this step by step: 2x + 3 = 7, so 2x = 4, therefore x = 2" โ ## ๐ Proven Results OptiLLM delivers measurable improvements across diverse benchmarks: | Technique | Base Model | Improvement | Benchmark | |-----------|------------|-------------|-----------| | **MARS** | Gemini 2.5 Flash Lite | **+30.0 points** | AIME 2025 (43.3โ73.3) | | **CePO** | Llama 3.3 70B | **+18.6 points** | Math-L5 (51.0โ69.6) | | **AutoThink** | DeepSeek-R1-1.5B | **+9.34 points** | GPQA-Diamond (21.72โ31.06) | | **LongCePO** | Llama 3.3 70B | **+13.6 points** | InfiniteBench (58.0โ71.6) | | **MOA** | GPT-4o-mini | **Matches GPT-4** | Arena-Hard-Auto | | **PlanSearch** | GPT-4o-mini | **+20% pass@5** | LiveCodeBench | *Full benchmark results [below](#sota-results-on-benchmarks-with-optillm)* โฌ๏ธ ## ๐๏ธ Installation ### Using pip ```bash pip install optillm optillm 2024-10-22 07:45:05,612 - INFO - Loaded plugin: privacy 2024-10-22 07:45:06,293 - INFO - Loaded plugin: memory 2024-10-22 07:45:06,293 - INFO - Starting server with approach: auto ``` ### Using docker ```bash do ``` --- **Machine-readable endpoints** - JSON: [`/api/graphcanon/tools/algorithmicsuperintelligence-optillm`](/api/graphcanon/tools/algorithmicsuperintelligence-optillm) - LLM index: [/llms.txt](/llms.txt) - Full corpus: [/llms-full.txt](/llms-full.txt) _GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_