A Hands-On Guide to Fine-Tuning LLMs with PyTorch and Hugging Face

Kindle | Paperback | PDF [Leanpub] | PDF [Gumroad]

• FAQ
• Appendix A - Setting Up Your GPU Pod

Google Colab

You can easily load the notebooks directly from GitHub using Colab and run them using a GPU provided by Google. You need to be logged in a Google Account of your own.

You can go through the chapters already using the links below:

• Chapter 0 - TL;DR
• Chapter 1 - Pay Attention to LLMs
• Chapter 2 - Loading a Quantized Model
• Chapter 3 - Low-Rank Adaptation (LoRA)
• Chapter 4 - Formatting Your Dataset
• Chapter 5 - Fine-Tuning with SFTTrainer
• Chapter 6 - Deploying It Locally
• Appendix B - Data Types' Internal Representation

Preface

If you’re reading this, I probably don’t need to tell you that large language models are pretty much everywhere, right?

Since the release of ChatGPT in November 2022, it feels almost impossible to keep up with the rapid pace of developments. Every day, there’s a new technique, a new model, or a groundbreaking announcement. These are surely exciting times—but they can also feel overwhelming, exhausting and, at times, frustrating.

"Where do I even begin to learn this?" is a perfectly valid question—and a tough one to answer on your own. I wrote this book as a tentative response to that question. It focuses on key concepts that, in my view, have proven to be stable and are likely to remain central to the fine-tuning process for the foreseeable future: quantization, low-rank adapters, and formatting templates.

Mastering these concepts is crucial for understanding the current landscape and will also empower you to handle future developments. They might also be useful to train or fine-tune a variety of large models, not just language models. They are essential tools in any data scientist's toolkit.

This is an intermediate-level book, so to make the most of its content, you need a solid foundation. If Transformers, attention, Adam, tokens, embeddings, and GPUs do not ring any bells, I'd suggest you to start with my beginner-friendly series, Deep Learning with PyTorch Step-by-Step.

I chose the Hugging Face ecosystem as the foundation for this book because it is the de facto standard for working with deep learning models, whether they're language models or not. The concepts I discuss—quantization, adapters, and templates—are neatly implemented and integrated into the ecosystem, making them relatively straightforward to use. But you have to understand how to configure them effectively and what those configurations are actually doing under the hood. It wasn't easy to find out such information out there, though. I missed a comprehensive overview explaining how these tec