{"data":{"slug":"uclaml-spin","name":"SPIN","tagline":"Official implementation of Self-Play Fine-Tuning","github_url":"https://github.com/uclaml/SPIN","owner":"uclaml","repo":"SPIN","owner_avatar_url":"https://avatars.githubusercontent.com/u/22385378?v=4","primary_language":"Python","stars":1254,"forks":106,"topics":["deep-learning","fine-tuning","large-language-models","self-play"],"archived":false,"github_pushed_at":"2024-05-08T05:59:37+00:00","maintenance_label":"Dormant","stars_delta_30d":6,"url":"https://www.graphcanon.com/tools/uclaml-spin","markdown_url":"https://www.graphcanon.com/tools/uclaml-spin.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/uclaml-spin","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=uclaml-spin","description":"The official implementation of Self-Play Fine-Tuning (SPIN)","homepage_url":"https://uclaml.github.io/SPIN/","license":"Apache-2.0","open_issues":24,"watchers":11,"ai_summary":"SPIN provides methodologies for self-play fine-tuning in large language models utilizing deep learning techniques.","readme_excerpt":"<p align=\"center\">\n    <img src=\"images/spin_dalle.png\" width=\"30%\"> <br>\n</p>\n<p align=\"center\">\n    🤗 <a href=\"https://huggingface.co/collections/UCLA-AGI/zephyr-7b-sft-full-spin-65c361dfca65637272a02c40\" target=\"_blank\">Models</a> | 🤗 <a href=\"https://huggingface.co/collections/UCLA-AGI/datasets-spin-65c3624e98d4b589bbc76f3a\" target=\"_blank\">Datasets</a>\n</p>\n\n# Self-Play Fine-Tuning (SPIN)\n\n  \n\nThis repository contains the official code for the paper \"[Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models](https://arxiv.org/abs/2401.01335)\".\n\nAuthors: [Zixiang Chen](https://sites.google.com/view/zxchen)\\*, [Yihe Deng](https://sites.google.com/g.ucla.edu/yihedeng/)\\*, [Huizhuo Yuan](https://scholar.google.com/citations?user=8foZzX4AAAAJ)\\*, [Kaixuan Ji](https://scholar.google.com/citations?user=FOoKDukAAAAJ), [Quanquan Gu](https://web.cs.ucla.edu/~qgu/)\n\n[[Webpage](https://uclaml.github.io/SPIN/)] [[Huggingface](https://huggingface.co/papers/2401.01335)] \n\n\n## 🔔 News \n- **[05/01/2024]** SPIN has been accepted by ICML2024!\n- **[04/06/2024]** We released the training scripts to reproduce our results.\n- **[04/04/2024]** ❗ We've discovered that the datasets previously uploaded are incorrect. We've re-upload the corrected datasets.\n- **[02/13/2024]** We updated our arXiv v2: https://arxiv.org/abs/2401.01335.\n- **[02/09/2024]** Our code is open-sourced!\n- **[01/02/2024]** Our paper is released on arXiv: https://arxiv.org/abs/2401.01335.\n\n❗ __Note__: We notice that the [Alignment Handbook](https://github.com/huggingface/alignment-handbook) has updated their configuration and SFT checkpoint since our experiments. The configuration and SFT model from the Alignment Handbook that we used in our experiments for data generation and fine-tuning are the older version ([Config](https://github.com/huggingface/alignment-handbook/blob/61a11a5c7d66179ed0a930b0dd12e532fce701dd/recipes/zephyr-7b-beta/dpo/config_full.yaml), [Model](https://huggingface.co/alignment-handbook/zephyr-7b-sft-full/tree/ac6e600eefcce74f5e8bae1035d4f66019e93190)). The model checkpoint on Hugging Face (`alignment-handbook/zephyr-7b-sft-full`) has been updated once with the new config. If you wish to use the newest SFT model, you need to either load the checkpoint with `revision=ac6e600eefcce74f5e8bae1035d4f66019e93190` or generate your own data instead of using the [datasets](https://huggingface.co/collections/UCLA-AGI/datasets-spin-65c3624e98d4b589bbc76f3a) we provided on Hugging Face.\n\n\n## Table of Contents\n- [About SPIN](#🌀-about-spin)\n- [Setup](#Setup)\n    - [Data](#Data)\n    - [Model](#Model)\n- [Usage](#Usage)\n    - [Step 1: Generation](#step-1-generation)\n        - [Faster generation with vLLM](#🚀-faster-generation-with-vllm)\n    - [Step 1.5: Gather generations and convert data type](#step-15-gather-generations-and-convert-data-type)\n    - [Step 2: Fine-tuning](#step-2-fine-tuning)\n- [Reproducing Our Results](#Reproducing-Our-Results)\n- [Evaluation](#Evaluation)\n- [Citation](#Citation)\n- [Acknowledgement](#Acknowledgement)\n\n\n\n## 🌀 About SPIN\n**SPIN** utilizes a self-play mechanism, allowing an LLM to improve itself by playing against its previous iterations, without needing additional human-annotated preference data than the SFT dataset itself. More specifically, the LLM generates its own training data from its previous iterations, refining its policy by discerning these self-generated responses from the original SFT data. \n<p align=\"center\">\n    <img src=\"images/iter_openllm.png\" width=\"35%\"> <br>\n  Average score of <b>SPIN</b> at different iterations on the HuggingFace Open LLM leaderboard. \n</p>\nSPIN can significantly enhance the performance of an LLM after SFT across various benchmarks, outperforming the model trained with direct preference optimization (DPO) on labelled preference datasets. The approach is theoretically grounded, ensuring that the LLM aligns with the target data distribution, and empirically validated through exten","github_created_at":"2024-02-04T21:43:19+00:00","created_at":"2026-07-11T11:39:17.329114+00:00","updated_at":"2026-08-24T00:02:23.031997+00:00","categories":[{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"deep-learning","name":"deep-learning"},{"slug":"fine-tuning","name":"fine-tuning"},{"slug":"large-language-models","name":"large language models"},{"slug":"self-play","name":"self-play"}],"trust":{"provenance":{"is_fork":false,"github_id":752815135,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-24T00:02:22.320Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":837,"last_release_at":null,"stars_delta_30d":6,"open_issues_delta_30d":0},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T11:39:18.753Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-08-24T00:02:22.743Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-08-24T00:02:22.743Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-08-24T00:02:22.743Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["When implementing self-play algorithms aimed at enhancing performance of large language models within constrained domains.","For projects that necessitate the Apache-2.0 license, supporting open-source contribution and commercial use without patent protection concerns."],"when_not_to_use":["If your project strictly adheres to frameworks that do not incorporate self-play techniques for training or fine-tuning models.","When prioritizing a model training framework that relies on supervised learning rather than the self-play methodology SPIN is based upon."],"source":"enrich:decision_facts","observed_at":"2026-07-14T21:00:04.907Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"SPIN is specialized for self-play fine-tuning in large language models through deep learning."}]}}