“llm-inference”

Unified framework for scaling AI and Python applications

Find secrets with Gitleaks

分享大模型技术原理与实战经验，涵盖工程化和应用落地

20+ high-performance LLMs with recipes to pretrain, finetune, and deploy at scale.

Self-Hosting LLMs Made Easy

High-speed Large Language Model Serving for Local Deployment

The easiest way to serve AI apps and models

AI-native proxy server and data plane for agentic apps

End-to-end framework for building custom AI applications and agents.

OptiLLM is an OpenAI API-compatible optimizing inference proxy that enhances LLM accuracy and performance on reasoning tasks without needing to train or fine-tune models.

Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

Artificial Intelligence Controller Interface (AICI)

Neuron is an Agentic Framework in PHP for building production-ready AI driven applications.

Simple, Explicit and Transparent LLM Applications