{"data":{"slug":"packtpublishing-llm-engineers-handbook","name":"LLM-Engineers-Handbook","tagline":"LLM's practical guide: From fundamentals to deploying advanced LLM and RAG apps","github_url":"https://github.com/PacktPublishing/LLM-Engineers-Handbook","owner":"PacktPublishing","repo":"LLM-Engineers-Handbook","owner_avatar_url":"https://avatars.githubusercontent.com/u/10974906?v=4","primary_language":"Python","stars":5286,"forks":1280,"topics":["aws","fine-tuning-llm","genai","llm","llm-evaluation","llmops","ml-system-design","mlops","rag"],"archived":false,"github_pushed_at":"2026-04-22T08:25:03+00:00","maintenance_label":"Slowing","stars_delta_30d":49,"url":"https://www.graphcanon.com/tools/packtpublishing-llm-engineers-handbook","markdown_url":"https://www.graphcanon.com/tools/packtpublishing-llm-engineers-handbook.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/packtpublishing-llm-engineers-handbook","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=packtpublishing-llm-engineers-handbook","description":"The LLM's practical guide: From the fundamentals to deploying advanced LLM and RAG apps to AWS using LLMOps best practices","homepage_url":"https://www.amazon.com/LLM-Engineers-Handbook-engineering-production/dp/1836200072/","license":"MIT","open_issues":35,"watchers":61,"ai_summary":"This repository provides a comprehensive guide for developing and deploying large language model (LLM) applications, including fine-tuning LLMs and implementing Retrieval-Augmented Generation (RAG) apps on AWS using LLMOps best practices.","readme_excerpt":"## 💻 Installation\n\n> [!NOTE]\n> If you are experiencing issues while installing and running the repository, consider checking the [Issues](https://github.com/PacktPublishing/LLM-Engineers-Handbook/issues) GitHub section for other people who solved similar problems or directly asking us for help.\n\n---\n\n### 3. Install Dependencies\n\nThe project uses Poetry for dependency management.\n\n1. Verify Poetry installation:\n\n```bash\npoetry --version  # Should show Poetry version 1.8.3 or later\n```\n\n2. Set up the project environment and install dependencies:\n\n```bash\npoetry env use 3.11\npoetry install --without aws\npoetry run pre-commit install\n```\n\nThis will:\n\n- Configure Poetry to use Python 3.11\n- Install project dependencies (excluding AWS-specific packages)\n- Set up pre-commit hooks for code verification\n\n---\n\n### 6. Deployment Setup\n\nWhen deploying the project to the cloud, we must set additional settings for Mongo, Qdrant, and AWS. If you are just working locally, the default values of these env vars will work out of the box. Detailed deployment instructions are available in Chapter 11 of the [LLM Engineer's Handbook](https://www.amazon.com/LLM-Engineers-Handbook-engineering-production/dp/1836200072/).\n\n#### MongoDB\n\nWe must change the `DATABASE_HOST` env var with the URL pointing to your cloud MongoDB cluster.\n\n```env\nDATABASE_HOST=your_mongodb_url\n```\n\n→ Check out this [tutorial](https://www.mongodb.com/resources/products/fundamentals/mongodb-cluster-setup) to learn how to create and host a MongoDB cluster for free.\n\n#### Qdrant\n\nChange `USE_QDRANT_CLOUD` to `true`, `QDRANT_CLOUD_URL` with the URL point to your cloud Qdrant cluster, and `QDRANT_APIKEY` with its API key.\n\n```env\nUSE_QDRANT_CLOUD=true\nQDRANT_CLOUD_URL=your_qdrant_cloud_url\nQDRANT_APIKEY=your_qdrant_api_key\n```\n\n→ Check out this [tutorial](https://qdrant.tech/documentation/cloud/create-cluster/) to learn how to create a Qdrant cluster for free\n\n#### AWS\n\nFor your AWS set-up to work correctly, you need the AWS CLI installed on your local machine and properly configured with an admin user (or a user with enough permissions to create new SageMaker, ECR, and S3 resources; using an admin user will make everything more straightforward).\n\nChapter 2 provides step-by-step instructions on how to install the AWS CLI, create an admin user on AWS, and get an access key to set up the `AWS_ACCESS_KEY` and `AWS_SECRET_KEY` environment variables. If you already have an AWS admin user in place, you have to configure the following env vars in your `.env` file:\n\n```bash\nAWS_REGION=eu-central-1 # Change it with your AWS region.\nAWS_ACCESS_KEY=your_aws_access_key\nAWS_SECRET_KEY=your_aws_secret_key\n```\n\nAWS credentials are typically stored in `~/.aws/credentials`. You can view this file directly using `cat` or similar commands:\n\n```bash\ncat ~/.aws/credentials\n```\n\n> [!IMPORTANT]\n> Additional configuration options are available in [settings.py](https://github.com/PacktPublishing/LLM-Engineers-Handbook/blob/main/llm_engineering/settings.py). Any variable in the `Settings` class can be configured through the `.env` file.\n\n---\n\n### Local infrastructure (for testing and development)\n\nWhen running the project locally, we host a MongoDB and Qdrant database using Docker. Also, a testing ZenML server is made available through their Python package.\n\n> [!WARNING]\n> You need Docker installed (>= v27.1.1)\n\nFor ease of use, you can start the whole local development infrastructure with the following command:\n```bash\npoetry poe local-infrastructure-up\n```\n\nAlso, you can stop the ZenML server and all the Docker containers using the following command:\n```bash\npoetry poe local-infrastructure-down\n```\n\n> [!WARNING]  \n> When running on MacOS, before starting the server, export the following environment variable:\n> `export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES`\n> Otherwise, the connection between the local server and pipeline will break. 🔗 More details in [this issue](https://github.com/zenml-io/zenml/issues/","github_created_at":"2024-04-09T10:56:03+00:00","created_at":"2026-07-07T17:42:09.829374+00:00","updated_at":"2026-08-20T12:01:49.851281+00:00","categories":[{"slug":"developer-tools","name":"Developer Tools","url":"https://www.graphcanon.com/categories/developer-tools","markdown_url":"https://www.graphcanon.com/categories/developer-tools.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/developer-tools"},{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"},{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"aws","name":"aws"},{"slug":"fine-tuning-llm","name":"fine-tuning-llm"},{"slug":"genai","name":"genai"},{"slug":"llm-evaluation","name":"llm-evaluation"},{"slug":"llmops","name":"llmops"},{"slug":"ml-system-design","name":"ml-system-design"},{"slug":"mlops","name":"mlops"},{"slug":"rag","name":"rag"}],"trust":{"provenance":{"is_fork":false,"github_id":784183059,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-20T12:01:49.005Z","maintenance":{"label":"Slowing","score":36,"methodology":"github_public_v1","releases_90d":0,"days_since_push":120,"last_release_at":null,"stars_delta_30d":49,"open_issues_delta_30d":1},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T11:20:17.693Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-08-20T12:01:49.466Z"},"deploy":{"source":"dockerfile:Dockerfile","self_host":true,"observed_at":"2026-08-20T12:01:49.466Z","managed_saas":false},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-08-20T12:01:49.466Z"},"has_docker":{"value":true,"source":"dockerfile:Dockerfile","observed_at":"2026-08-20T12:01:49.466Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-08-20T12:01:49.466Z"}},"decision_facts":{"hosting":null,"pricing":{"model":"freemium","summary":"The repository itself is free under the MIT license; however, AWS services (like SageMaker and ECR) require paid usage based on your consumption."},"requirements":{"notes":["- Requires Docker for managing local infrastructure.","- Python version 3.11 is required; Poetry should already be installed to manage dependencies."],"min_ram_gb":8,"requires_docker":true},"constraints":{"min_ram_gb":8,"pricing_model":"freemium","requires_docker":true},"when_to_use":["- You are an engineer looking to deploy large language models (LLMs) or retrieval-augmented generation (RAG) applications specifically in an AWS environment.","- You aim to leverage Poetry for dependency management, which simplifies the setup of development and production environments."],"when_not_to_use":["- If your project is not hosted on AWS, as this tool heavily integrates with AWS services like SageMaker, ECR, and S3, making it less suitable for non-AWS cloud providers.","- You do not want to manage dependencies via Poetry. The guide assumes you are comfortable working within a Poetry-managed environment."],"source":"enrich:decision_facts","observed_at":"2026-07-11T02:35:23.064Z"},"constraint_facets":{"min_ram_gb":8,"pricing_model":"freemium","requires_docker":true},"decision_summary":[{"label":"Pricing","value":"freemium - The repository itself is free under the MIT license; however, AWS services (like SageMaker and ECR) require paid usage based on your consumption."},{"label":"Requirements","value":"Min 8 GB RAM; Requires Docker; - Requires Docker for managing local infrastructure.; - Python version 3.11 is required; Poetry should already be installed to manage dependencies."},{"label":"Adopt for","value":"A comprehensive guide for deploying advanced LLM and RAG apps on AWS using LLMOps best practices."}]}}