Home/LLM Frameworks/TensorRT-LLM

TensorRT-LLM

NVIDIA/TensorRT-LLM

Python API for defining and optimizing Large Language Models (LLMs) on NVIDIA GPUs

GraphCanon updated today · GitHub synced today

14k

Stars

2.5k

Forks

1.5k

Open issues

118

Watchers

1d

Last push

Python OtherCreated Aug 16, 2023

Trust & integrity

Maintenance: Very active (0d since push); As of today · Source: github_public_v1
Provenance: Not a fork · Organization account; As of today · Source: github_public_v1
Security (OSV): 16 low (16 low); As of today · Source: osv@v1

Public GitHub metadata and optional OSV dependency scans. Signals, not a guarantee. Trust methodology.

Backing

Company and funding context for Nvidia. Display-only - not part of trust score or organic ranking.

Company: NVIDIA Corporation·GitHub org profile·today
Employees: 11,528·Wikidata (P1128 employees)·today
Commercial model: Pure OSS·GitHub org profile (public repos)·today

Overview

TensorRT LLM is designed to enable efficient inference of large language models on NVIDIA GPUs. It offers a user-friendly Python interface, supports state-of-the-art optimizations, and includes components to create high-performance runtimes.

Capability facts

Languages: python
Source: github.language+pyproject.toml · Jul 11, 2026

Categories

LLM Frameworks Inference & Serving

Graph entities

Tags

moe cuda llm-serving pytorch blackwell

README

Getting Started

To get started with TensorRT-LLM, visit our documentation:

Explore

TensorRT-LLM alternatives →Trust report →LLM Frameworks category →All comparisons Stack workflows Trending tools

Trust & health

MaintenanceVery active

last release 1w · 10 releases (90d)

AdoptionWidely used

Graph connectivityIsolated

Security16 low

Provenance

Organization account (GitHub heuristic)
Computed today

Full trust report Trust report (JSON)

Methodology: public GitHub data (github_public_v1). Signals, not a guarantee. Trust methodology.

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

LLM inference in C/C++

A high-throughput and memory-efficient inference and serving engine for LLMs

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

Universal LLM Deployment Engine with ML Compilation

AirLLM 70B inference with single 4GB GPU

Train transformer language models with reinforcement learning.

Ongoing research training transformer models at scale

High-performance LLMs with recipes for pretraining, finetuning and deployment

Run any open-source LLMs as OpenAI compatible API endpoint in the cloud.

llm-engineer-toolkit

A curated list of over 120 LLM libraries categorized.

train-llm-from-scratch

A straightforward method for training your LLM, from downloading data to generating text.

Install

pip install TensorRT-LLM

For agents

This page has a .md twin and JSON over the API.

/tools/nvidia-tensorrt-llm.md GET /api/graphcanon/tools/nvidia-tensorrt-llm GET /api/graphcanon/graph?tool=nvidia-tensorrt-llm