Results for “inference”

BentoML

bentoml/BentoML

The easiest way to serve AI apps and models

optillm

algorithmicsuperintelligence/optillm

OptiLLM is an OpenAI API-compatible optimizing inference proxy that enhances LLM accuracy and performance on reasoning tasks without needing to train or fine-tune models.

vllm

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

lorax

predibase/lorax

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

OpenLLM

bentoml/OpenLLM

Self-Hosting LLMs Made Easy

superduper

superduper-io/superduper

End-to-end framework for building custom AI applications and agents.

free-llm-api-resources

cheahjs/free-llm-api-resources

Repository listing free and trial-based LLM inference API resources

ml-engineering

stas00/ml-engineering

Machine Learning Engineering Open Book

transformers

huggingface/transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models

ray

ray-project/ray

Unified framework for scaling AI and Python applications

ChatTTS

2noise/ChatTTS

A generative speech model for daily dialogue

sglang

sgl-project/sglang

SGLang high-performance serving framework

litgpt

Lightning-AI/litgpt

20+ high-performance LLMs with recipes to pretrain, finetune, and deploy at scale.

cube-studio

tencentmusic/cube-studio

Cube Studio开源一站式云原生机器学习平台

cube-studio

data-infra/cube-studio

CubeStudio是开源云原生一站式机器学习/深度学习/AI平台

llmflows

stoyan-stoyanov/llmflows

Simple, Explicit and Transparent LLM Applications

open-webui

open-webui/open-webui

User-friendly AI Interface (Supports Ollama, OpenAI API, ...)

mlc-llm

mlc-ai/mlc-llm

Universal LLM Deployment Engine with ML Compilation

ChatGLM2-6B

zai-org/ChatGLM2-6B

An Open Bilingual Chat LLM

petals

bigscience-workshop/petals

Run LLMs at home, BitTorrent-style