Installation

First, we have to install all the libraries listed in requirements.txt

pip install -r requirements.txt

---

Step by step installation with conda

Create a new conda environment and activate it

conda create -n env
conda activate env

Install the pytorch version compatible with your version of cuda here, for example the following command works with cuda 11.6

conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.6 -c pytorch -c nvidia

Install transformers and peft

conda install -c huggingface transformers 
pip install git+https://github.com/huggingface/peft.git

Note that you can install the latest stable version of transformers by using

pip install git+https://github.com/huggingface/transformers

Install datasets, accelerate and huggingface_hub

conda install -c huggingface -c conda-forge datasets
conda install -c conda-forge accelerate
conda install -c conda-forge huggingface_hub

Finally, install bitsandbytes and wandb

pip install bitsandbytes
pip install wandb

To get the full list of arguments with descriptions you can run the following command on any script:

python scripts/some_script.py --help

Before you run any of the scripts make sure you are logged in and can push to the hub:

huggingface-cli login

Make sure you are logged in wandb:

wandb login

Now that everything is done, you can clone the repository and get into the corresponding directory.

---

Inference hardware requirements

In FP32 the model requires more than 60GB of RAM, you can load it in FP16 or BF16 in ~30GB, or in 8bit under 20GB of RAM with