{"data":{"slug":"bigscience-workshop-petals","name":"petals","tagline":"Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading","github_url":"https://github.com/bigscience-workshop/petals","owner":"bigscience-workshop","repo":"petals","owner_avatar_url":"https://avatars.githubusercontent.com/u/82455566?v=4","primary_language":"Python","stars":10496,"forks":642,"topics":["bloom","chatbot","deep-learning","distributed-systems","falcon","gpt","guanaco","language-models","large-language-models","llama","machine-learning","mixtral","neural-networks","nlp","pipeline-parallelism","pretrained-models","pytorch","tensor-parallelism","transformer","volunteer-computing"],"archived":false,"github_pushed_at":"2024-09-07T11:54:28+00:00","maintenance_label":"Dormant","stars_delta_30d":212,"url":"https://www.graphcanon.com/tools/bigscience-workshop-petals","markdown_url":"https://www.graphcanon.com/tools/bigscience-workshop-petals.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/bigscience-workshop-petals","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=bigscience-workshop-petals","description":"🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading","homepage_url":"https://petals.dev","license":"MIT","open_issues":113,"watchers":106,"ai_summary":"Petals enables running large language models by distributing model layers across a network similar to a BitTorrent system. It supports fine-tuning and inference with potential speed improvements.","readme_excerpt":"# Connect to a distributed network hosting model layers\ntokenizer = AutoTokenizer.from_pretrained(model_name)\nmodel = AutoDistributedModelForCausalLM.from_pretrained(model_name)","github_created_at":"2022-06-12T00:10:27+00:00","created_at":"2026-07-07T17:34:01.44366+00:00","updated_at":"2026-08-17T06:02:00.463998+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"}],"tags":[{"slug":"bloom","name":"bloom"},{"slug":"chatbot","name":"chatbot"},{"slug":"deep-learning","name":"deep-learning"},{"slug":"distributed-systems","name":"distributed-systems"},{"slug":"falcon","name":"falcon"},{"slug":"gpt","name":"gpt"},{"slug":"guanaco","name":"guanaco"},{"slug":"language-models","name":"language-models"}],"trust":{"provenance":{"is_fork":false,"github_id":502482803,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-17T06:01:59.692Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":708,"last_release_at":"2023-09-06T17:29:56Z","stars_delta_30d":212,"open_issues_delta_30d":0},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T11:02:27.773Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-08-17T06:02:00.142Z"},"deploy":{"source":"dockerfile:Dockerfile","self_host":true,"observed_at":"2026-08-17T06:02:00.142Z","managed_saas":false},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-08-17T06:02:00.142Z"},"has_docker":{"value":true,"source":"dockerfile:Dockerfile","observed_at":"2026-08-17T06:02:00.142Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-08-17T06:02:00.142Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["- When you want to leverage faster fine-tuning and inference of LLMs (up to 10x) by utilizing distributed layers across a network similar to a BitTorrent system.","- If your infrastructure could benefit from the decentralized model hosting approach, allowing for more efficient resource usage compared to traditional offloading methods."],"when_not_to_use":["- When your use case strictly requires on-premises computation and you do not wish to rely on external peer-to-peer distributed networks, as Petals' efficiency comes with a dependency on its network.","- If you need absolute control over the data privacy and don't trust the decentralized system for sensitive information processing; petals uses a volunteer-computing model which might have variable or"],"source":"enrich:decision_facts","observed_at":"2026-07-11T15:51:12.286Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"Petals is designed for users aiming to run large language models at home with potential speedups through a distributed, BitTorrent-style peer-to-peer network."}]}}