Results for “inference”

transformers

huggingface/transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models

open-webui

open-webui/open-webui

User-friendly AI Interface (Supports Ollama, OpenAI API, ...)

vllm

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

ray

ray-project/ray

Unified framework for scaling AI and Python applications

ChatTTS

2noise/ChatTTS

A generative speech model for daily dialogue.

sglang

sgl-project/sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

gitleaks

gitleaks/gitleaks

Find secrets with Gitleaks

free-llm-api-resources

cheahjs/free-llm-api-resources

List of free LLM inference resources accessible via API.

llm-action

liguodongiot/llm-action

分享大模型技术原理和实战经验，涵盖工程化、应用落地

airllm

lyogavin/airllm

AirLLM enables efficient inference for large language models on low-memory GPUs without quantization.

NemoClaw

NVIDIA/NemoClaw

Run agents like Hermes and OpenClaw more securely inside NVIDIA OpenShell with managed inference

Qwen

QwenLM/Qwen

Official repo of Qwen (通义千问) chat and pretrained large language model by Alibaba Cloud

ml-engineering

stas00/ml-engineering

Machine Learning Engineering Open Book

ChatGLM2-6B

zai-org/ChatGLM2-6B

An Open Bilingual Chat LLM

litgpt

Lightning-AI/litgpt

20+ high-performance LLMs with recipes to pretrain, finetune, and deploy at scale.

OpenLLM

bentoml/OpenLLM

Self-hosting LLMs made easy

petals

bigscience-workshop/petals

Run large language models at home, BitTorrent-style.

PowerInfer

Tiiny-AI/PowerInfer

High-speed Local LLM Serving

BentoML

bentoml/BentoML

The easiest way to serve AI apps and models

awesome-LLM-resources

WangRongsheng/awesome-LLM-resources

Summary of the world's best LLM resources.

postgresml

postgresml/postgresml

Postgres with GPUs for ML/AI applications

plano

katanemo/plano

Plano is an AI-native proxy and data plane for agentic apps with built-in orchestration, safety, observability, and smart LLM routing.

superduper

superduper-io/superduper

End-to-end framework for building custom AI applications and agents

cube-studio

tencentmusic/cube-studio

一站式云原生机器学习/深度学习/大模型AI平台

reasoning-from-scratch

rasbt/reasoning-from-scratch

Implement a reasoning LLM in PyTorch from scratch, step by step

optillm

algorithmicsuperintelligence/optillm

Optimizing inference proxy for LLMs

GenerativeAIExamples

NVIDIA/GenerativeAIExamples

Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture

lorax

predibase/lorax

Multi-LoRA inference server that scales to serve thousands of fine-tuned LLMs on a single GPU

vllm-ascend

vllm-project/vllm-ascend

Community maintained hardware plugin for vLLM on Ascend

cube-studio

data-infra/cube-studio

cubestudio开源云原生一站式机器学习/深度学习/大模型AI平台

aici

microsoft/aici

Prompts as (Wasm) Programs

neuron-ai

neuron-core/neuron-ai

Agentic Framework for PHP

paddler

intentee/paddler

LLM load balancer and serving platform for self-hosting LLMs/VLMs

pixeltable

pixeltable/pixeltable

Unified multimodal backend for AI data apps

EmbedAnything

StarlightSearch/EmbedAnything

Highly Performant, Modular and Memory Safe Ingestion, Inference and Indexing in Rust 🦀

rtp-llm

alibaba/rtp-llm

RTP-LLM: Alibaba's high-performance LLM inference engine

llmflows

stoyan-stoyanov/llmflows

Simple, Explicit and Transparent LLM Apps

VectorHub

superlinked/VectorHub

VectorHub is a deprecated repository that now focuses on SIE, a self-hosted inference engine.