{"data":{"slug":"antoniogr7-pratical-llms","name":"pratical-llms","tagline":"A collection of hand on notebook for LLMs practitioner","github_url":"https://github.com/AntonioGr7/pratical-llms","owner":"AntonioGr7","repo":"pratical-llms","owner_avatar_url":"https://avatars.githubusercontent.com/u/48945271?v=4","primary_language":"Jupyter Notebook","stars":53,"forks":15,"topics":["genai","llm","llm-evaluation","llm-inference","llm-serving","llm-training","quantization"],"archived":false,"github_pushed_at":"2025-01-13T13:43:11+00:00","maintenance_label":"Dormant","url":"https://www.graphcanon.com/tools/antoniogr7-pratical-llms","markdown_url":"https://www.graphcanon.com/tools/antoniogr7-pratical-llms.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/antoniogr7-pratical-llms","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=antoniogr7-pratical-llms","description":"A collection of hand on notebook for LLMs practitioner","homepage_url":null,"license":null,"open_issues":0,"watchers":3,"ai_summary":null,"readme_excerpt":"# Guide for LLM Practitioners\n\nWelcome to the repository for LLM (Large Language Model) engineers! This collection of Jupyter Notebooks is designed to collect pratical aspects of our job. \nI will collect and add jupyter and/or script for learning and experimenting purpose. \n\n## Notebooks Included\n\n| Notebook                                   | Description                                                                                                                                                                       | Url  |\n|--------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------|\n| 1_understanding_llms_benchmarks.ipynb     | This notebook provides an explanation of the main benchmarks used in the openLLM leaderboard. It aims to help you grasp the key metrics and methodologies used in benchmarking LLMs. | [Link](https://github.com/AntonioGr7/pratical-llms/blob/main/01_understanding_llms_benchmarks.ipynb) |\n| 2_quantization_base.ipynb                 | In this notebook, you'll learn how to open a Hugging Face model in 8-bit and 4-bit using the BitandBytes library. Quantization is a crucial technique for optimizing model performance and resource usage, and this notebook guides you through the process. | [Link](https://github.com/AntonioGr7/pratical-llms/blob/main/02_quantization_base.ipynb) |\n| 3_quantization_gptq.ipynb                 | Explore quantization in GPTQ format using the auto-gptq library with this notebook. GPTQ format is gaining popularity for its effectiveness in compressing and quantizing large models like GPT. Learn how to leverage this format for your models. | [Link](https://github.com/AntonioGr7/pratical-llms/blob/main/03_quantization_gptq.ipynb) |\n| 4_quantization_exllamav2.ipynb | How to quantize a model from HF to exllamav2 | [Link](https://github.com/AntonioGr7/pratical-llms/blob/main/04_quantization_exllamav2.ipynb) |\n| 5_sharding_and_offloading.ipynb | How to shard a model in multiple chunk. This allow to load it on different devices or load one at time managing memory. Learn how to offload some layer to CPU or even disk | [Link](https://github.com/AntonioGr7/pratical-llms/blob/main/05_sharding_and_offloading.ipynb) |\n| 6_gguf_quantization_and_inference.ipynb | Quantize a model into GGUF using the llama.cpp library. Inferencing on OpenAI-compatible server. | [Link](https://github.com/AntonioGr7/pratical-llms/blob/main/06_gguf_quantization_and_inference.ipynb) |\n| 7_gguf_split_and_load.ipynb | Split a GGUF Quantized model in multiple parts, making it easily sharable | [Link](https://github.com/AntonioGr7/pratical-llms/blob/main/07_gguf_split_and_load.ipynb) |\n| 8_hqq_quantization.ipynb | Explore quantization using Half-Quadratic Quantization (HQQ) | [Link](https://github.com/AntonioGr7/pratical-llms/blob/main/08_hqq_quantization.ipynb) |\n| 9_inference_big_model_cpu_plus_gpu.ipynb | This notebook shows how to calculate the RAM required by a quantized GGUF model and how to load it into memory using both RAM and VRAM, optimizing the number of layers that can be offloaded to the GPU. The notebook demonstrates loading Qwen/Qwen1.5-32B-Chat-GGUF as an example on a system with a T4 15GB VRAM and approximately 32GB of RAM | [Link](https://github.com/AntonioGr7/pratical-llms/blob/main/09_inference_big_model_cpu_plus_gpu.ipynb) |\n| a10_inference_llama3.ipynb | LLama3 has been released. This notebook demonstrates how to run LLama3-8B-Instruct half precision if you have access to a GPU with 24GB of VRAM, quantized to 8 bits if you have 10GB of VRAM, and shows how to run the Q8 GGUF version to achieve maximum performance if you only have 10GB of VRAM. | [Link](https://github.com/AntonioGr7/pratical-llms/blob/main/a10_inference_llama3.ipynb) |\n| a11_llm_guardrails_using_llama3_guard.ipynb | Protect your backend and your generative AI applicat","github_created_at":"2024-04-04T06:29:05+00:00","created_at":"2026-07-15T10:40:36.573415+00:00","updated_at":"2026-07-15T10:40:39.77267+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"genai","name":"genai"},{"slug":"jupyter-notebook","name":"jupyter-notebook"},{"slug":"llm","name":"llm"},{"slug":"llm-evaluation","name":"llm-evaluation"},{"slug":"llm-inference","name":"llm-inference"},{"slug":"llm-serving","name":"llm-serving"},{"slug":"llm-training","name":"llm-training"},{"slug":"quantization","name":"quantization"}],"trust":{"provenance":{"is_fork":false,"github_id":781842627,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-15T10:40:37.540Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":547,"last_release_at":null},"security_summary":{"status":"findings","scanner":"osv@v1","low_count":42,"high_count":0,"last_scan_at":"2026-07-15T10:40:37.879Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-15T10:40:37.308Z"},"languages":{"value":["jupyter notebook"],"source":"github.language","observed_at":"2026-07-15T10:40:37.308Z"}}}}