Results for “serving”

vllm

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

BentoML

bentoml/BentoML

The easiest way to serve AI apps and models

serve

jina-ai/serve

Build multimodal AI applications with cloud-native stack

ray

ray-project/ray

Unified framework for scaling AI and Python applications

lorax

predibase/lorax

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

sglang

sgl-project/sglang

SGLang high-performance serving framework

vllm-ascend

vllm-project/vllm-ascend

Community maintained hardware plugin for vLLM on Ascend

aici

microsoft/aici

Prompts as (Wasm) Programs

paddler

intentee/paddler

LLM load balancer and serving platform for self-hosting LLMs/VLMs

rtp-llm

alibaba/rtp-llm

RTP-LLM: Alibaba's high-performance LLM inference engine

PowerInfer

Tiiny-AI/PowerInfer

High-speed Large Language Model Serving for Local Deployment

vespa

vespa-engine/vespa

The AI search platform

AI-Infra-from-Zero-to-Hero

HuaizhengZhang/AI-Infra-from-Zero-to-Hero

Awesome System for Machine Learning ⚡️ AI System Papers and Industry Practice

langcorn

msoedov/langcorn

Serving LangChain LLM apps and agents with FastApi

llm-action

liguodongiot/llm-action

分享大模型技术原理与实战经验，涵盖工程化和应用落地

OpenLLM

bentoml/OpenLLM

Self-Hosting LLMs Made Easy

clearml

clearml/clearml

ClearML - Auto-Magical CI/CD for AI workloads

superduper

superduper-io/superduper

End-to-end framework for building custom AI applications and agents.

envd

tensorchord/envd

Reproducible development environment for humans and agents

dingo

dingodb/dingo

Multi-modal vector database supporting SQL and various interfaces

Prompt-Engineering-Guide

dair-ai/Prompt-Engineering-Guide

Repository for prompt engineering resources, guides, and tools focused on language models (LLMs).

mem0

mem0ai/mem0

Universal memory layer for AI Agents

CloakBrowser

CloakHQ/CloakBrowser

Stealth Chromium that passes every bot detection test.

Machine-Learning-Interviews

alirezadir/Machine-Learning-Interviews

Repository for preparing AI/ML technical interviews with chapters on general coding, ML coding, fundamentals/breadth (including LLMs and multimodal AI), system design, behavioral questions, and more.

server

MariaDB/server

Community developed SQL server with advanced features and performance.

Awesome-LLMOps

tensorchord/Awesome-LLMOps

Curated list of LLMOps tools for developers

every-chatgpt-gui

billmei/every-chatgpt-gui

Repository collecting GUI clients for ChatGPT, Claude, and other LLMs

attu

zilliztech/attu

Attu - The AI Workbench for Milvus

featureform

featureform/featureform

Virtual Feature Store

SeekStorm

SeekStorm/SeekStorm

SeekStorm: sub-millisecond vector & lexical search library and multi-tenancy server in Rust.

matrixone

matrixorigin/matrixone

AI-native HTAP database with vector search capabilities.

pixeltable

pixeltable/pixeltable

Unified multimodal backend for AI data apps