FlexLLMGen
Running large language models on a single GPU for throughput-oriented scenarios.
GraphCanon updated 3w · GitHub synced 3w
Decision brief
FlexLLMGen runs large language models efficiently on a single GPU, ideal for throughput-oriented tasks thanks to its intelligent offloading capabilities.
Good fit when
- You need high-throughput inference where tasks can benefit from efficient offloading techniques.
- Your setup includes only a single GPU and you are looking to maximize its utilization for running large language models.
Avoid when
- The scenario requires distributed computing across multiple GPUs, as FlexLLMGen focuses on optimizing usage of a single GPU.
- If your applications demand lower latency rather than high throughput, another tool might be more suitable since FlexLLMGen prioritizes throughput over latency.
Observed Jul 12, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Archived (642d since push)
- As of 3w
- Provenance
- Not a fork · Organization account
- As of 3w
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install FlexLLMGenSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
A Python-based tool designed to run large language models with high throughput using a single GPU, focusing on offloading tasks efficiently.
Capability facts
- Languages
- python
Source: github.language+pyproject.toml · Aug 2, 2026
Categories
Tags
README
For agents
This page has a .md twin and JSON over the API.