--- title: "FasterTransformer vs exllama" type: "comparison" canonical_url: "https://www.graphcanon.com/compare/nvidia-fastertransformer-vs-turboderp-exllama" tools: ["nvidia-fastertransformer", "turboderp-exllama"] --- # FasterTransformer vs exllama *GraphCanon updated Aug 7, 2026* ## Verdict Pick FasterTransformer if highly optimized transformer encoder and decoder for inferencing, supporting BERT and GPT on various frameworks like TensorFlow, PyTorch; pick exllama if exLlama provides a memory-efficient implementation of the LLaMa model with support for quantized weights, primarily aimed at users with NVIDIA GPUs from the 30-series onwards. [FasterTransformer](https://github.com/NVIDIA/FasterTransformer) reports 6.4k GitHub stars, 935 forks, and 289 open issues, last pushed Mar 27, 2024. [exllama](https://github.com/turboderp/exllama) has 2.9k stars, 220 forks, and 65 open issues, last pushed Sep 30, 2023. Figures are from public GitHub metadata via [FasterTransformer's repository](https://github.com/NVIDIA/FasterTransformer) and [exllama's repository](https://github.com/turboderp/exllama). | | [FasterTransformer](/tools/nvidia-fastertransformer.md) | [exllama](/tools/turboderp-exllama.md) | | --- | --- | --- | | Tagline | Transformer related optimization including BERT and GPT | Memory-efficient rewrite of HF transformers for Llama with quantized weights | | Stars | 6,446 | 2,937 | | Forks | 935 | 220 | | Open issues | 289 | 65 | | Language | C++ | Python | | Adopt for | Highly optimized transformer encoder and decoder for inferencing, supporting BERT and GPT on various frameworks like TensorFlow, PyTorch. | ExLlama provides a memory-efficient implementation of the LLaMa model with support for quantized weights, primarily aimed at users with NVIDIA GPUs from the 30-series onwards. | | Persona | - | - | | Runtime | - | - | | License | Apache-2.0 | MIT | | Categories | Inference & Serving | Inference & Serving, LLM Frameworks | ## Trust and health _Sourced signals - not a safety guarantee. No winner column._ | | [FasterTransformer](/tools/nvidia-fastertransformer.md) | [exllama](/tools/turboderp-exllama.md) | | --- | --- | --- | | Days since push | 862d | 1041d | | Open issues (now) | 289 | 65 | | Owner type | Organization | User | | Full report | [trust report](/tools/nvidia-fastertransformer/trust.md) | [trust report](/tools/turboderp-exllama/trust.md) | ## Decision facts: FasterTransformer - **Adopt for:** Highly optimized transformer encoder and decoder for inferencing, supporting BERT and GPT on various frameworks like TensorFlow, PyTorch. ## Decision facts: exllama - **Adopt for:** ExLlama provides a memory-efficient implementation of the LLaMa model with support for quantized weights, primarily aimed at users with NVIDIA GPUs from the 30-series onwards. ## Choose when ### Choose FasterTransformer if… - FasterTransformer is primarily C++; exllama is Python. - License: FasterTransformer is Apache-2.0, exllama is MIT. - Tags unique to FasterTransformer: bert, cublas, cublaslt, cuda. - When aiming for high performance with GPU-based FP16 computations for BERT or GPT models specifically. ### Choose exllama if… - exllama is primarily Python; FasterTransformer is C++. - License: exllama is MIT, FasterTransformer is Apache-2.0. - Tags unique to exllama: docker, llama model, memory-efficient, nvidia gpu. - Also covers LLM Frameworks. - exllama ships Docker support for self-hosted deployment. - - When deploying LLaMa models on NVIDIA GPUs from the 30-series or later that have strong FP16 support. ## When NOT to use FasterTransformer - If looking for active development and latest improvements on LLM Inference as NVIDIA recommends TensorRT-LLM over FasterTransformer now. - When specific frameworks not including TensorFlow, PyTorch, or Triton are required. ## When NOT to use exllama - - If you are operating older GPUs such as Pascal series, which lack robust FP16 support; alternatives like AutoGPTQ might perform better. - - In scenarios that involve AMD GPU hardware (due to limited testing and optimization efforts). ## Common questions ### What is the difference between FasterTransformer and exllama? FasterTransformer: Transformer related optimization including BERT and GPT. exllama: Memory-efficient rewrite of HF transformers for Llama with quantized weights. See the comparison table for live GitHub stats and shared categories. ### When should I choose FasterTransformer over exllama? Choose FasterTransformer over exllama when FasterTransformer is primarily C++; exllama is Python; License: FasterTransformer is Apache-2.0, exllama is MIT; Tags unique to FasterTransformer: bert, cublas, cublaslt, cuda; When aiming for high performance with GPU-based FP16 computations for BERT or GPT models specifically. ### When should I choose exllama over FasterTransformer? Choose exllama over FasterTransformer when exllama is primarily Python; FasterTransformer is C++; License: exllama is MIT, FasterTransformer is Apache-2.0; Tags unique to exllama: docker, llama model, memory-efficient, nvidia gpu; Also covers LLM Frameworks; exllama ships Docker support for self-hosted deployment; - When deploying LLaMa models on NVIDIA GPUs from the 30-series or later that have strong FP16 support. ### When should I avoid FasterTransformer? If looking for active development and latest improvements on LLM Inference as NVIDIA recommends TensorRT-LLM over FasterTransformer now. When specific frameworks not including TensorFlow, PyTorch, or Triton are required. ### When should I avoid exllama? - If you are operating older GPUs such as Pascal series, which lack robust FP16 support; alternatives like AutoGPTQ might perform better. - In scenarios that involve AMD GPU hardware (due to limited testing and optimization efforts). ### Is FasterTransformer or exllama more popular on GitHub? FasterTransformer has more GitHub stars (6,446 vs 2,937). Stars measure visibility, not whether either tool fits your constraints. ### Are FasterTransformer and exllama open source? Yes - both are open-source projects on GitHub (FasterTransformer: Apache-2.0, exllama: MIT). ### Where can I find alternatives to FasterTransformer or exllama? GraphCanon lists graph-backed alternatives at [FasterTransformer alternatives](/tools/nvidia-fastertransformer/alternatives) and [exllama alternatives](/tools/turboderp-exllama/alternatives) ([FasterTransformer markdown twin](/tools/nvidia-fastertransformer/alternatives.md), [exllama markdown twin](/tools/turboderp-exllama/alternatives.md)), ranked by typed relationship edges rather than popularity votes. ### Is there a machine-readable version of this comparison? Yes. The markdown twin at [this comparison](/compare/nvidia-fastertransformer-vs-turboderp-exllama.md) mirrors this page for agents and LLM crawlers, with the same stats table and FAQ answers. ### Which is better maintained, FasterTransformer or exllama? FasterTransformer: Dormant. exllama: Dormant. Compare maintenance labels, days since push, and release cadence in the trust section below - stars alone do not measure maintenance. ### Where are the full trust reports for FasterTransformer and exllama? GraphCanon publishes per-repo trust reports with dated maintenance, provenance, and scan summaries: [FasterTransformer trust report](/tools/nvidia-fastertransformer/trust); [exllama trust report](/tools/turboderp-exllama/trust). --- **Machine-readable endpoints** - JSON: [`/api/graphcanon/graph?tool=nvidia-fastertransformer`](/api/graphcanon/graph?tool=nvidia-fastertransformer) - LLM index: [/llms.txt](/llms.txt) - Full corpus: [/llms-full.txt](/llms-full.txt) _GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_