TensorRT-LLM
Python API for defining and optimizing Large Language Models (LLMs) on NVIDIA GPUs
GraphCanon updated 1w · GitHub synced 1w · 25 views this month
Decision brief
`TensorRT LLM` is a specialized Python API for optimizing and efficiently running large language models on NVIDIA GPUs, featuring user-friendly interfaces and high-performance optimizations.
Good fit when
- When you are developing or deploying large language models (LLMs) specifically on NVIDIA GPU hardware.
- If your project requires state-of-the-art performance tuning tailored to LLMs with integration for PyTorch and CUDA.
Avoid when
- When working on CPUs or non-NVIDIA GPUs as the optimizations and hardware support are NVIDIA-specific.
- If you prioritize portability across different frameworks over high-performance tuning since TensorRT LLM is tightly integrated with NVIDIA technologies.
- Pricing:
- oss - Open source software (OSS) available under a license other than those listed in common OSS categories, implying free use but potentially with restrictions.
- Requirements:
- NVIDIA GPU hardware is required for the tool to take full advantage of its optimization capabilities.
Observed Jul 11, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Very active (0d since push)
- As of 1w
- Provenance
- Not a fork · Organization account
- As of 1w
- Security (OSV)
- 16 low (16 low)
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Backing
Company context for Nvidia. Display-only - separate from trust and ranking.
- Company
- NVIDIA Corporation·GitHub org profile·1mo
- Employees
- 11,528·Wikidata (P1128 employees)·1mo
- Commercial model
- Pure OSS·GitHub org profile (public repos)·1mo
Install
pip install TensorRT-LLMSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
TensorRT LLM is designed to enable efficient inference of large language models on NVIDIA GPUs. It offers a user-friendly Python interface, supports state-of-the-art optimizations, and includes components to create high-performance runtimes.
Capability facts
- Languages
- python
Source: github.language+pyproject.toml · Aug 7, 2026
Categories
Graph entities
Tags
README
Getting Started
To get started with TensorRT-LLM, visit our documentation:
For agents
This page has a .md twin and JSON over the API.