{"data":{"slug":"rafska-awesome-local-llm","name":"awesome-local-llm","tagline":"Resources for running LLMs locally","github_url":"https://github.com/rafska/awesome-local-llm","owner":"rafska","repo":"awesome-local-llm","owner_avatar_url":"https://avatars.githubusercontent.com/u/17859377?v=4","primary_language":null,"stars":2869,"forks":388,"topics":["ai","awesome","awesome-list","llm","local","local-ai","local-llm","resources","self-hosted","selfhosted"],"archived":false,"github_pushed_at":"2026-09-13T09:48:26+00:00","maintenance_label":"Very active","stars_delta_30d":351,"url":"https://www.graphcanon.com/tools/rafska-awesome-local-llm","markdown_url":"https://www.graphcanon.com/tools/rafska-awesome-local-llm.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/rafska-awesome-local-llm","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=rafska-awesome-local-llm","description":"A curated list of awesome platforms, tools, practices and resources that helps run LLMs locally","homepage_url":null,"license":"MIT","open_issues":169,"watchers":34,"ai_summary":"A curated list of platforms, tools, practices and resources designed to facilitate the local operation of large language models.","readme_excerpt":"## Hardware\n\n- [Alex Ziskind](https://www.youtube.com/@AZisk) - tests of pcs, laptops, gpus etc. capable of running LLMs\n- [Digital Spaceport](https://www.youtube.com/@DigitalSpaceport) - reviews of various builds designed for LLM inference\n- [Donato Capitella](https://www.youtube.com/@donatocapitella) - practical and insightful tutorials on running LLMs locally\n- [JetsonHacks](https://www.youtube.com/@JetsonHacks) - information about developing on NVIDIA Jetson Development Kits\n- [Miyconst](https://www.youtube.com/@Miyconst) - tests of various types of hardware capable of running LLMs\n- [Kolosal - LLM Memory calculator](https://www.kolosal.ai/memory-calculator) - estimate the RAM requirements of any GGUF model instantly\n- [LLM Inference VRAM & GPU Requirement Calculator](https://app.linpp2009.com/en/llm-gpu-memory-calculator) - calculate how many GPUs you need to deploy LLMs\n- [ZLUDA](https://github.com/vosen/ZLUDA) - CUDA on non-NVIDIA GPUs\n- [Strix Halo AI Toolboxes](https://strix-halo-toolboxes.com/) - toolboxes for GenAI on AMD Ryzen AI MAX+: containerized environments for LLMs, Image Generation, and Fine-tuning\n- [Strix Halo Wiki](https://strixhalo.wiki/) - a website to gather important information and practical guides for systems powered by AMD Ryzen AI MAX and MAX+ processors\n- [ai-notes](https://github.com/paudley/ai-notes) - random AI notes for working with local models or playing around with random machine learning bits\n\n[Back to Table of Contents](#table-of-contents)","github_created_at":"2025-06-03T10:01:04+00:00","created_at":"2026-07-15T10:58:21.650995+00:00","updated_at":"2026-09-20T04:59:41.307674+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"}],"tags":[{"slug":"ai","name":"ai"},{"slug":"awesome-list","name":"awesome-list"},{"slug":"llm","name":"llm"},{"slug":"local-ai","name":"local-ai"},{"slug":"self-hosted","name":"self-hosted"}],"trust":{"provenance":{"is_fork":false,"github_id":995323854,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-20T04:59:39.469Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":0,"days_since_push":6,"last_release_at":null,"stars_delta_30d":351,"open_issues_delta_30d":40},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-15T10:58:22.866Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-20T04:59:40.513Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-09-20T04:59:40.513Z"}},"decision_facts":{"hosting":null,"pricing":{"model":"freemium","summary":"The list itself is free and open-source under the MIT license."},"requirements":{"notes":["Technical skill in setting up a self-hosted large language model environment is necessary"],"min_ram_gb":null,"requires_docker":false},"constraints":{"min_ram_gb":null,"pricing_model":"freemium","requires_docker":false},"when_to_use":["- If you require extensive documentation and resources for setting up and running LLMs on your own hardware, this tool provides a comprehensive list of options","- When aiming to self-host a large language model without relying on cloud services due to cost constraints or data privacy issues"],"when_not_to_use":["- Avoid if you seek direct tools rather than a curated list; awesome-local-llm does not provide the actual software but guidance and links","- Not suitable for users who prefer ready-to-use solutions without needing additional configuration, as it requires self-hosting expertise to utilize its resources"],"source":"enrich:decision_facts","observed_at":"2026-07-17T10:13:22.630Z"},"constraint_facets":{"min_ram_gb":null,"pricing_model":"freemium","requires_docker":false},"decision_summary":[{"label":"Pricing","value":"freemium - The list itself is free and open-source under the MIT license."},{"label":"Requirements","value":"Technical skill in setting up a self-hosted large language model environment is necessary"},{"label":"Adopt for","value":"awesome-local-llm is a curated list of resources for the local operation of large language models."},{"label":"License detail","value":"MIT License"}]}}