Home/Inference & Serving/petals

petals

bigscience-workshop/petals

Distributed LLM Inference and Fine-Tuning

GitHub Website

GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026

53views this month

11k stars650 forksLast push Sep 7, 2024 Python MIT

Decision brief

Petals is a Python-based framework that facilitates the distributed inference and fine-tuning of large language models, leveraging a BitTorrent-like network for faster processing.

Good fit when

When you need to fine-tune or run large language models faster, up to 10 times quicker than traditional methods.
If you are working with a distributed network and want to utilize volunteer computing to enhance model performance.

Avoid when

If your project requires a centralized model management system, as Petals relies on a distributed network for its operations.
When you need to avoid dependency on a peer-to-peer network for model layers, as Petals uses a BitTorrent-style approach which might not be suitable for all network environments.

Requirements:: Min 4 GB RAM; Python environment is required.; Ensure network connectivity for distributed operations.

Observed Sep 18, 2026 · Source: enrich:decision_facts

petals alternatives Trust report

petals vs ollama petals vs vllm

Verify the decision

Maintenance and security

Full trust report

Maintenance: Dormant (740d since push); As of Sep 18, 2026
Provenance: Not a fork · Organization account; As of Sep 18, 2026
Security (OSV): No lockfile; As of Sep 18, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

pip install petals

PyPI

How it fits your stack(6)

Typed graph edges - alternatives, integrations, successors, and dependencies. Ranked by relationship type, not raw GitHub stars.

Relationship graph

Optional deeper exploration of typed edges and category neighbours.

Similar tools

Same-category neighbours not already linked as typed edges.

peft

State-of-the-art Parameter-Efficient Fine-Tuning

22k★

TensorRT-LLM

Python API for defining and optimizing Large Language Models for efficient inference on NVIDIA GPUs

15k★

litgpt

High-performance LLMs for pretraining, finetuning, and deployment

14k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

Petals enables running large language models using a distributed network, similar to BitTorrent, for faster fine-tuning and inference.

Capability facts

Deploy: Self-host
Source: dockerfile:Dockerfile · Sep 18, 2026
Docker: Dockerfile present
Source: dockerfile:Dockerfile · Sep 18, 2026
Languages: python
Source: github.language+pyproject.toml · Sep 18, 2026

README

Connect to a distributed network hosting model layers tokenizer = AutoTokenizer.from pretrained(model name) model = AutoDistributedModelForCausalLM.from pretrained(model name)

For agents

This page has a .md twin and JSON over the API.

/tools/bigscience-workshop-petals.md GET /api/graphcanon/tools/bigscience-workshop-petals GET /api/graphcanon/graph?tool=bigscience-workshop-petals