bitsandbytes
Large language model quantization toolkit for PyTorch.
GraphCanon updated 2w · GitHub synced 2w
Decision brief
bitsandbytes provides k-bit quantization in PyTorch, enhancing large language model accessibility across multiple hardware platforms.
Good fit when
- When you need advanced k-bit quantization on PyTorch for hardware like NVIDIA GPUs with SM75+ recommended.
- For projects that aim to leverage AMD RDNA GPUs like gfx1201 under Linux or Windows.
Avoid when
- Avoid if your setup includes Intel Gaudi processors as QLoRA 4-bit support is partial and 8-bit optimizers are not available.
- Steer clear if you require full compatibility with ARM-based CPUs, as specific GPU optimizations might lack coverage.
Observed Jul 17, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Very active (5d since push)
- As of 2w
- Provenance
- Not a fork · Organization account
- As of 2w
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install bitsandbytesHow it fits your stack(1)
Typed graph edges - alternatives, integrations, successors, and dependencies. Ranked by relationship type, not raw GitHub stars.
Relationship graph
Optional deeper exploration of typed edges and category neighbours.
Similar tools
Same-category neighbours not already linked as typed edges.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
bitsandbytes library enables k-bit quantization in PyTorch for making large language models more accessible. Supports various hardware platforms and accelerators including NVIDIA, AMD GPUs, Intel XPU, and CPUs.
Capability facts
- Languages
- python
Source: github.language+pyproject.toml · Aug 4, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Tags
README
System Requirements
bitsandbytes has the following minimum requirements for all platforms:
- Python 3.10+
- PyTorch 2.4+
- Note: While we aim to provide wide backwards compatibility, we recommend using the latest version of PyTorch for the best experience.
Accelerator support:
Note: this table reflects the status of the current development branch. For the latest stable release, see the document in the 0.50.0 tag.
Legend:
🚧 = Planned | 〰️ = Partially Supported | ✅ = Supported | ❌ = Not Supported
| Platform | Accelerator | Hardware Requirements | LLM.int8() | QLoRA 4-bit | 8-bit Optimizers |
|---|---|---|---|---|---|
| 🐧 Linux, glibc >= 2.24 | |||||
| x86-64 | ◻️ CPU | Minimum: AVX2 Optimized: AVX512F, AVX512BF16 | ✅ | ✅ | ✅ |
🟩 NVIDIA GPU cuda | SM60+ minimum SM75+ recommended | ✅ | ✅ | ✅ | |
🟥 AMD GPU cuda |
CDNA: gfx908, gfx90a, gfx942, gfx950, gfx1250 RDNA: gfx103X, gfx110X, gfx115X, gfx120X | ✅ | ✅ | ✅ | |
🟦 Intel GPU xpu |
Data Center GPU Max Series Arc A-Series (Alchemist) Arc B-Series (Battlemage) | ✅ | ✅ | ✅ | |
🟪 Intel Gaudi hpu | Gaudi2, Gaudi3 | ✅ | 〰️ | ❌ | |
| aarch64 | ◻️ CPU | ✅ * | ✅ | ✅ | |
🟩 NVIDIA GPU cuda | SM75+ | ✅ | ✅ | ✅ | |
| 🪟 Windows 11 / Windows Server 2022+ | |||||
| x86-64 | ◻️ CPU | AVX2 | ✅ | ✅ | ✅ |
🟩 NVIDIA GPU cuda | SM60+ minimum SM75+ recommended | ✅ | ✅ | ✅ | |
🟥 AMD GPU cuda | RDNA: gfx103X, gfx110X, gfx115X, gfx120X | ✅ | ✅ | ✅ | |
🟦 Intel GPU xpu |
Arc A-Series (Alchemist) Arc B-Series (Battlemage) | ✅ | ✅ | ✅ | |
| arm64 | ◻️ CPU | ✅ | ✅ | ✅ | |
| 🍎 macOS 14+ | |||||
| arm64 | ◻️ CPU | Apple M1+ | ✅ * | ✅ | ✅ |
⬜ Metal mps | Apple M1+ | ✅ * | ✅ | 🚧 | |
License
bitsandbytes is MIT licensed.
For agents
This page has a .md twin and JSON over the API.