GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026
53views this month
Decision brief
Lorax is a Python-based inference server specialized in managing large fleets of LoRA-adapted language models, which can scale up to thousands of fine-tuned LLMs. It supports platforms like GPT and LLaMA using PyTorch.
Good fit when
- - You require an infrastructure that can manage up to thousands of LoRA-adapted LLMs simultaneously for high-throughput inference.
- - Your environment includes Nvidia GPUs from the Ampere generation or above, as it requires significant computing power. Specifically, Lorax leverages CUDA 11.8 for these heavy compute tasks.
Avoid when
- - Your system does not meet the minimum hardware requirements (Nvidia Ampere generation GPU or higher).
- - If your team lacks experience with Docker and Linux-based systems since Lorax's setup guidelines rely heavily on these technologies.
- Requirements:
- Requires Nvidia GPU (Ampere generation or above); CUDA 11.8 compatible drivers and higher; Linux OS required; Docker for setup
Observed Jul 11, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Slowing (114d since push)
- As of Sep 20, 2026
- Provenance
- Not a fork · Organization account
- As of Sep 20, 2026
- Security (OSV)
- No lockfile
- As of Jul 11, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install loraxHow it fits your stack(5)
Typed graph edges - alternatives, integrations, successors, and dependencies. Ranked by relationship type, not raw GitHub stars.
Alternative
Relationship graph
Optional deeper exploration of typed edges and category neighbours.
Similar tools
Same-category neighbours not already linked as typed edges.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
Lorax is a Python-based multi-LoRA inference server designed to handle thousands of fine-tuned language models, utilizing PyTorch and transformers. It requires an Nvidia GPU with compatible CUDA drivers.
Capability facts
- Deploy
- Self-host
Source: dockerfile:Dockerfile · Sep 20, 2026
- Docker
- Dockerfile present
Source: dockerfile:Dockerfile · Sep 20, 2026
- Languages
- python
Source: github.language · Sep 20, 2026
Categories
Graph entities
Tags
README
🏃♂️ Getting Started We recommend starting with our pre built Docker image to avoid compiling custom CUDA kernels and other dependencies. Requirements The minimum system requirements need to run LoRAX include: Nvidia GPU (Ampere generation or above) CUDA 11.8 compatible device d...
For agents
This page has a .md twin and JSON over the API.