Home/Inference & Serving/petals

petals

bigscience-workshop/petals

Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

GitHub Website

GraphCanon updated 3d · GitHub synced 3d

10k stars642 forksLast push 1y Python MIT

Decision brief

Petals is designed for users aiming to run large language models at home with potential speedups through a distributed, BitTorrent-style peer-to-peer network.

Good fit when

- When you want to leverage faster fine-tuning and inference of LLMs (up to 10x) by utilizing distributed layers across a network similar to a BitTorrent system.
- If your infrastructure could benefit from the decentralized model hosting approach, allowing for more efficient resource usage compared to traditional offloading methods.

Avoid when

- When your use case strictly requires on-premises computation and you do not wish to rely on external peer-to-peer distributed networks, as Petals' efficiency comes with a dependency on its network.
- If you need absolute control over the data privacy and don't trust the decentralized system for sensitive information processing; petals uses a volunteer-computing model which might have variable or

Observed Jul 11, 2026 · Source: enrich:decision_facts

petals alternatives Trust report

petals vs ollama petals vs vllm

Verify the decision

Maintenance and security

Full trust report

Maintenance: Dormant (708d since push); As of 3d
Provenance: Not a fork · Organization account; As of 3d
Security (OSV): No lockfile; As of 1mo

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

pip install petals

PyPI

How it fits your stack(6)

Typed graph edges - alternatives, integrations, successors, and dependencies. Ranked by relationship type, not raw GitHub stars.

Relationship graph

Optional deeper exploration of typed edges and category neighbours.

pytorch-lightning31k★

AlternativeIntegratesDepends onRelated

pytorch-lightning31k★

AlternativeIntegratesDepends onRelated

Similar tools

Same-category neighbours not already linked as typed edges.

gpt4all

Run Local LLMs on Any Device

77k★

llmfit

Hundreds of models & providers. One command to find what runs on your hardware.

32k★

pytorch-lightning

Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.

31k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

Petals enables running large language models by distributing model layers across a network similar to a BitTorrent system. It supports fine-tuning and inference with potential speed improvements.

Capability facts

Deploy: Self-host
Source: dockerfile:Dockerfile · Aug 17, 2026
Docker: Dockerfile present
Source: dockerfile:Dockerfile · Aug 17, 2026
Languages: python
Source: github.language+pyproject.toml · Aug 17, 2026

README

Connect to a distributed network hosting model layers

tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoDistributedModelForCausalLM.from_pretrained(model_name)

For agents

This page has a .md twin and JSON over the API.

/tools/bigscience-workshop-petals.md GET /api/graphcanon/tools/bigscience-workshop-petals GET /api/graphcanon/graph?tool=bigscience-workshop-petals