Home/Inference & Serving/bitsandbytes

bitsandbytes

bitsandbytes-foundation/bitsandbytes

Large language model quantization toolkit for PyTorch.

GitHub Website

GraphCanon updated Sep 30, 2026 · GitHub synced Sep 30, 2026

94views this month

8.5k stars938 forksLast push Sep 7, 2026 Python MIT

Decision brief

bitsandbytes provides k-bit quantization in PyTorch, enhancing large language model accessibility across multiple hardware platforms.

Good fit when

When you need advanced k-bit quantization on PyTorch for hardware like NVIDIA GPUs with SM75+ recommended.
For projects that aim to leverage AMD RDNA GPUs like gfx1201 under Linux or Windows.

Avoid when

Avoid if your setup includes Intel Gaudi processors as QLoRA 4-bit support is partial and 8-bit optimizers are not available.
Steer clear if you require full compatibility with ARM-based CPUs, as specific GPU optimizations might lack coverage.

Observed Jul 17, 2026 · Source: enrich:decision_facts

bitsandbytes alternatives Trust report

Verify the decision

Maintenance and security

Full trust report

Maintenance: Active (22d since push); As of Sep 30, 2026
Provenance: Not a fork · Organization account; As of Sep 30, 2026
Security (OSV): No lockfile; As of Jul 11, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

pip install bitsandbytes

PyPI

How it fits your stack(1)

Typed graph edges - alternatives, integrations, successors, and dependencies. Ranked by relationship type, not raw GitHub stars.

Integrates

pytorch

Bits and bytes is a quantization toolkit specifically designed to work with PyTorch, optimizing large language models.

integrates

Relationship graph

Optional deeper exploration of typed edges and category neighbours.

Similar tools

Same-category neighbours not already linked as typed edges.

open-llms

A list of open LLMs available for commercial use

13k★

qlora

QLoRA finetuning of quantized LLMs

11k★

oumi

Easily fine-tune, evaluate and deploy open source LLMs/VLMs

9.4k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

bitsandbytes library enables k-bit quantization in PyTorch for making large language models more accessible. Supports various hardware platforms and accelerators including NVIDIA, AMD GPUs, Intel XPU, and CPUs.

Capability facts

Languages: python
Source: github.language+pyproject.toml · Sep 30, 2026

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Sep 30, 2026)

* Python 3.10+

Source link

README

System Requirements bitsandbytes has the following minimum requirements for all platforms: Python 3.10+ PyTorch 2.4+ Note: While we aim to provide wide backwards compatibility, we recommend using the latest version of PyTorch for the best experience. Accelerator support: <small N...

For agents

This page has a .md twin and JSON over the API.

/tools/bitsandbytes-foundation-bitsandbytes.md GET /api/graphcanon/tools/bitsandbytes-foundation-bitsandbytes GET /api/graphcanon/graph?tool=bitsandbytes-foundation-bitsandbytes