FlexLLMGen
Running large language models on a single GPU for throughput-oriented scenarios
GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026
28views this month
Decision brief
FlexLLMGen is a Python-based tool for executing large language models on a single GPU, focusing on high-throughput scenarios. It supports PyTorch and is designed for offloading tasks to improve performance.
Good fit when
- When you need to run large language models on a single GPU for high-throughput scenarios, FlexLLMGen can be an effective choice.
- If your project requires offloading tasks to improve performance and you are working with PyTorch, FlexLLMGen is suitable.
Avoid when
- Avoid using FlexLLMGen if your project does not require high-throughput and you are working with frameworks other than PyTorch.
- If your scenario demands multi-GPU setups for model execution, FlexLLMGen, which focuses on single GPU setups, may not be the best choice.
- Requirements:
- Requires PyTorch >= 1.12
Observed Sep 18, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Archived (690d since push)
- As of Sep 18, 2026
- Provenance
- Not a fork · Organization account
- As of Sep 18, 2026
- Security (OSV)
- No lockfile
- As of Sep 18, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install FlexLLMGenSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
FlexLLMGen is a Python-based tool for executing large language models on a single GPU, focusing on high-throughput scenarios. It supports deep learning frameworks like PyTorch and is designed for offloading tasks to improve performance.
Capability facts
- Languages
- python
Source: github.language+pyproject.toml · Sep 18, 2026
Categories
Tags
README
Installation Requirements: PyTorch = 1.12 (Help)
For agents
This page has a .md twin and JSON over the API.