Home/Inference & Serving/lorax

lorax

predibase/lorax

Multi-LoRA inference server for scalable fine-tuned LLMs

GitHub Website

GraphCanon updated today · GitHub synced today · 30 views this month

3.8k stars326 forksLast push 2mo Python Apache-2.0

Decision brief

Lorax is a Python-based inference server specialized in managing large fleets of LoRA-adapted language models, which can scale up to thousands of fine-tuned LLMs. It supports platforms like GPT and LLaMA using PyTorch.

Good fit when

- You require an infrastructure that can manage up to thousands of LoRA-adapted LLMs simultaneously for high-throughput inference.
- Your environment includes Nvidia GPUs from the Ampere generation or above, as it requires significant computing power. Specifically, Lorax leverages CUDA 11.8 for these heavy compute tasks.

Avoid when

- Your system does not meet the minimum hardware requirements (Nvidia Ampere generation GPU or higher).
- If your team lacks experience with Docker and Linux-based systems since Lorax's setup guidelines rely heavily on these technologies.

Requirements:: Requires Nvidia GPU (Ampere generation or above); CUDA 11.8 compatible drivers and higher; Linux OS required; Docker for setup

Observed Jul 11, 2026 · Source: enrich:decision_facts

lorax alternatives Trust report

lorax vs sglang lorax vs mlc-llm lorax vs vllm

Verify the decision

Maintenance and security

Full trust report

Maintenance: Steady (83d since push); As of today
Provenance: Not a fork · Organization account; As of today
Security (OSV): No lockfile; As of 1mo

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

pip install lorax

PyPI

How it fits your stack(5)

Typed graph edges - alternatives, integrations, successors, and dependencies. Ranked by relationship type, not raw GitHub stars.

Relationship graph

Optional deeper exploration of typed edges and category neighbours.

AlternativeIntegratesDepends onRelated

AlternativeIntegratesDepends onRelated

Similar tools

Same-category neighbours not already linked as typed edges.

alpaca-lora

Instruct-tune LLaMA on consumer hardware

19k★

omlx

LLM inference server with continuous batching and SSD caching for Apple Silicon

19k★

litgpt

High-performance LLMs with recipes for pretraining, finetuning and deployment

14k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

Lorax is a Python-based multi-LoRA inference server designed to handle thousands of fine-tuned language models, utilizing PyTorch and transformers. It requires an Nvidia GPU with compatible CUDA drivers.

Capability facts

Deploy: Self-host
Source: dockerfile:Dockerfile · Aug 20, 2026
Docker: Dockerfile present
Source: dockerfile:Dockerfile · Aug 20, 2026
Languages: python
Source: github.language · Aug 20, 2026

Graph entities

Smolagentsframework

README

🏃‍♂️ Getting Started

We recommend starting with our pre-built Docker image to avoid compiling custom CUDA kernels and other dependencies.

Requirements

The minimum system requirements need to run LoRAX include:

Nvidia GPU (Ampere generation or above)
CUDA 11.8 compatible device drivers and above
Linux OS
Docker (for this guide)

For agents

This page has a .md twin and JSON over the API.

/tools/predibase-lorax.md GET /api/graphcanon/tools/predibase-lorax GET /api/graphcanon/graph?tool=predibase-lorax