alpaca-lora

Docker Setup & Inference

1. Build the container image:

docker build -t alpaca-lora .

2. Run the container (you can also use finetune.py and all of its parameters as shown above for training):

docker run --gpus=all --shm-size 64g -p 7860:7860 -v ${HOME}/.cache:/root/.cache --rm alpaca-lora generate.py \
    --load_8bit \
    --base_model 'decapoda-research/llama-7b-hf' \
    --lora_weights 'tloen/alpaca-lora-7b'

3. Open https://localhost:7860 in the browser

---

Docker Compose Setup & Inference

1. (optional) Change desired model and weights under environment in the docker-compose.yml

2. Build and run the container

docker-compose up -d --build

3. Open https://localhost:7860 in the browser

4. See logs:

docker-compose logs -f

5. Clean everything up:

docker-compose down --volumes --rmi all