---
title: "litgpt"
type: "tool"
slug: "lightning-ai-litgpt"
canonical_url: "https://www.graphcanon.com/tools/lightning-ai-litgpt"
github_url: "https://github.com/Lightning-AI/litgpt"
homepage_url: "https://lightning.ai"
stars: 13469
forks: 1466
primary_language: "Python"
license: "Apache-2.0"
categories: ["model-training", "inference-serving"]
tags: ["deep-learning", "llm", "ai", "artificial-intelligence", "large-language-models", "llm-inference"]
updated_at: "2026-07-07T18:30:54.711256+00:00"
---

# litgpt

> 20+ high-performance LLMs with recipes to pretrain, finetune, and deploy at scale.

Lightning-AI/litgpt is an open-source project featuring a variety of large language models (LLMs) implemented from scratch without abstractions. It offers detailed instructions for training, fine-tuning, and deploying these models in various scales with features such as flash attention, low-precision weights, FSDP, LoRA, QLoRA, Adapter techniques.

## Facts

- Repository: https://github.com/Lightning-AI/litgpt
- Homepage: https://lightning.ai
- Stars: 13,469 · Forks: 1,466 · Open issues: 266 · Watchers: 118
- Primary language: Python
- License: Apache-2.0
- Last pushed: 2026-07-06T21:15:23+00:00

## Categories

- [Model Training](/categories/model-training.md)
- [Inference & Serving](/categories/inference-serving.md)

## Tags

deep-learning, llm, ai, artificial-intelligence, large-language-models, llm-inference

## Related tools

- [transformers](/tools/huggingface-transformers.md) - 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models (★ 162,347)
- [open-webui](/tools/open-webui-open-webui.md) - User-friendly AI Interface (Supports Ollama, OpenAI API, ...) (★ 144,575)
- [LLMs-from-scratch](/tools/rasbt-llms-from-scratch.md) - Implement a ChatGPT-like LLM in PyTorch from scratch (★ 98,711)
- [vllm](/tools/vllm-project-vllm.md) - A high-throughput and memory-efficient inference and serving engine for LLMs (★ 85,612)
- [rtk](/tools/rtk-ai-rtk.md) - CLI proxy reducing LLM token consumption by 60-90% (★ 69,253)
- [unsloth](/tools/unslothai-unsloth.md) - Unsloth Studio is a web UI for training and running open models locally. (★ 67,883)
- [anything-llm](/tools/mintplex-labs-anything-llm.md) - Stop renting your intelligence. Own it with AnythingLLM. (★ 62,759)
- [mem0](/tools/mem0ai-mem0.md) - Universal memory layer for AI Agents (★ 60,317)

## README (excerpt)

```text
<div align="center">


# ⚡ LitGPT

**20+ high-performance LLMs with recipes to pretrain, finetune, and deploy at scale.**

<pre>
✅ From scratch implementations      ✅ No abstractions         ✅ Beginner friendly
   ✅ Flash attention                   ✅ FSDP                    ✅ LoRA, QLoRA, Adapter
✅ Reduce GPU memory (fp4/8/16/32)   ✅ 1-1000+ GPUs/TPUs       ✅ 20+ LLMs         
</pre>


---



  

<p align="center">
  <a href="#quick-start">Quick start</a> •
  <a href="#choose-from-20-llms">Models</a> •
  <a href="#finetune-an-llm">Finetune</a> •
  <a href="#deploy-an-llm">Deploy</a> •
  <a href="#all-workflows">All workflows</a> •
  <a href="#state-of-the-art-features">Features</a> •
  <a href="#training-recipes">Recipes (YAML)</a> •
  <a href="https://lightning.ai/">Lightning AI</a> •
    <a href="#tutorials">Tutorials</a>
</p>

&nbsp;

<a target="_blank" href="https://lightning.ai/lightning-ai/studios/litgpt-quick-start">
  <img src="https://pl-bolts-doc-images.s3.us-east-2.amazonaws.com/app-2/get-started-badge.svg" height="36px" alt="Get started"/>
</a>

&nbsp;

</div>

# Looking for GPUs?
Over 340,000 developers use [Lightning Cloud](https://lightning.ai/?utm_source=litgpt_readme&utm_medium=referral&utm_campaign=litgpt_readme) - purpose-built for PyTorch and PyTorch Lightning. 
- [GPUs](https://lightning.ai/pricing?utm_source=litgpt_readme&utm_medium=referral&utm_campaign=litgpt_readme) from $0.19.   
- [Clusters](https://lightning.ai/clusters?utm_source=litgpt_readme&utm_medium=referral&utm_campaign=litgpt_readme): frontier-grade training/inference clusters.   
- [AI Studio (vibe train)](https://lightning.ai/studios?utm_source=litgpt_readme&utm_medium=referral&utm_campaign=litgpt_readme): workspaces where AI helps you debug, tune and vibe train.
- [AI Studio (vibe deploy)](https://lightning.ai/studios?utm_source=litgpt_readme&utm_medium=referral&utm_campaign=litgpt_readme): workspaces where AI helps you optimize, and deploy models.     
- [Notebooks](https://lightning.ai/notebooks?utm_source=litgpt_readme&utm_medium=referral&utm_campaign=litgpt_readme): Persistent GPU workspaces where AI helps you code and analyze.
- [Inference](https://lightning.ai/deploy?utm_source=litgpt_readme&utm_medium=referral&utm_campaign=litgpt_readme): Deploy models as inference APIs.

# Finetune, pretrain, and inference LLMs Lightning fast ⚡⚡
Every LLM is implemented from scratch with **no abstractions** and **full control**, making them blazing fast, minimal, and performant at enterprise scale.

✅ **Enterprise ready -** Apache 2.0 for unlimited enterprise use.</br>
✅ **Developer friendly -** Easy debugging with no abstraction layers and single file implementations.</br>
✅ **Optimized performance -** Models designed to maximize performance, reduce costs, and speed up training.</br>
✅ **Proven recipes -** Highly-optimized training/finetuning recipes tested at enterprise scale.</br>

&nbsp;

# Quick start
Install LitGPT
```
pip install 'litgpt[extra]'
```

Load and use any of the [20+ LLMs](#choose-from-20-llms):
```python
from litgpt import LLM

llm = LLM.load("microsoft/phi-2")
text = llm.generate("Fix the spelling: Every fall, the family goes to the mountains.")
print(text)
# Corrected Sentence: Every fall, the family goes to the mountains.
```

&nbsp;

✅ Optimized for fast inference</br>
✅ Quantization</br>
✅ Runs on low-memory GPUs</br>
✅ No layers of internal abstractions</br>
✅ Optimized for production scale</br>

<details>
  <summary>Advanced install options</summary>

Install from source:

```bash
git clone https://github.com/Lightning-AI/litgpt
cd litgpt
# if using uv
uv sync --all-extras
# if using pip
pip install -e ".[extra,compiler,test]"
```
</details>

[Explore the full Python API docs](tutorials/python-api.md).

&nbsp;

---
# Choose from 20+ LLMs
Every model is written from scratch to maximize performance and remove layers of abstraction:

| Model | Model size | Author | Reference |
|----|----|----|----|
| Llama 3, 3.1, 3.2, 3.3
```

---

**Machine-readable endpoints**

- JSON: [`/api/graphcanon/tools/lightning-ai-litgpt`](/api/graphcanon/tools/lightning-ai-litgpt)
- LLM index: [/llms.txt](/llms.txt)
- Full corpus: [/llms-full.txt](/llms-full.txt)

_GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_