LLM-Engineers-Handbook

💻 Installation

[!NOTE] If you are experiencing issues while installing and running the repository, consider checking the Issues GitHub section for other people who solved similar problems or directly asking us for help.

3. Install Dependencies

The project uses Poetry for dependency management.

1. Verify Poetry installation:

poetry --version  # Should show Poetry version 1.8.3 or later

2. Set up the project environment and install dependencies:

poetry env use 3.11
poetry install --without aws
poetry run pre-commit install

This will:

• Configure Poetry to use Python 3.11
• Install project dependencies (excluding AWS-specific packages)
• Set up pre-commit hooks for code verification

6. Deployment Setup

When deploying the project to the cloud, we must set additional settings for Mongo, Qdrant, and AWS. If you are just working locally, the default values of these env vars will work out of the box. Detailed deployment instructions are available in Chapter 11 of the LLM Engineer's Handbook.

MongoDB

We must change the DATABASE_HOST env var with the URL pointing to your cloud MongoDB cluster.

DATABASE_HOST=your_mongodb_url

→ Check out this tutorial to learn how to create and host a MongoDB cluster for free.

Qdrant

Change USE_QDRANT_CLOUD to true, QDRANT_CLOUD_URL with the URL point to your cloud Qdrant cluster, and QDRANT_APIKEY with its API key.

USE_QDRANT_CLOUD=true
QDRANT_CLOUD_URL=your_qdrant_cloud_url
QDRANT_APIKEY=your_qdrant_api_key

→ Check out this tutorial to learn how to create a Qdrant cluster for free

AWS

For your AWS set-up to work correctly, you need the AWS CLI installed on your local machine and properly configured with an admin user (or a user with enough permissions to create new SageMaker, ECR, and S3 resources; using an admin user will make everything more straightforward).

Chapter 2 provides step-by-step instructions on how to install the AWS CLI, create an admin user on AWS, and get an access key to set up the AWS_ACCESS_KEY and AWS_SECRET_KEY environment variables. If you already have an AWS admin user in place, you have to configure the following env vars in your .env file:

AWS_REGION=eu-central-1 # Change it with your AWS region.
AWS_ACCESS_KEY=your_aws_access_key
AWS_SECRET_KEY=your_aws_secret_key

AWS credentials are typically stored in ~/.aws/credentials. You can view this file directly using cat or similar commands:

cat ~/.aws/credentials

[!IMPORTANT] Additional configuration options are available in settings.py. Any variable in the Settings class can be configured through the .env file.

Local infrastructure (for testing and development)

When running the project locally, we host a MongoDB and Qdrant database using Docker. Also, a testing ZenML server is made available through their Python package.

[!WARNING] You need Docker installed (>= v27.1.1)

For ease of use, you can start the whole local development infrastructure with the following command:

poetry poe local-infrastructure-up

Also, you can stop the ZenML server and all the Docker containers using the following command:

poetry poe local-infrastructure-down

[!WARNING]
When running on MacOS, before starting the server, export the following environment variable: export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES Otherwise, the connection between the local server and pipeline will break. 🔗 More details in [this issue](https://github.com/zenml-io/zenml/issues/