{"data":{"slug":"dusty-nv-nanollm","name":"NanoLLM","tagline":"Optimized local inference for LLMs using HuggingFace-like APIs","github_url":"https://github.com/dusty-nv/NanoLLM","owner":"dusty-nv","repo":"NanoLLM","owner_avatar_url":"https://avatars.githubusercontent.com/u/15352619?v=4","primary_language":"Python","stars":382,"forks":67,"topics":["edge-ai","llm-inference","multimodal","rag","speech","vector-database","vision-transformer"],"archived":false,"github_pushed_at":"2024-10-18T02:50:37+00:00","maintenance_label":"Dormant","stars_delta_30d":2,"url":"https://www.graphcanon.com/tools/dusty-nv-nanollm","markdown_url":"https://www.graphcanon.com/tools/dusty-nv-nanollm.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/dusty-nv-nanollm","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=dusty-nv-nanollm","description":"Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.","homepage_url":"https://dusty-nv.github.io/NanoLLM/","license":"MIT","open_issues":66,"watchers":9,"ai_summary":"NanoLLM supports optimized local inference with HuggingFace-compatible interfaces for tasks such as quantization and multimodal applications including vision, language, speech, vector databases, and RAG.","readme_excerpt":"# NanoLLM\n<a href=\"https://www.jetson-ai-lab.com\"><img align=\"right\" width=\"200\" height=\"200\" src=\"https://nvidia-ai-iot.github.io/jetson-generative-ai-playground/images/JON_Gen-AI-panels.png\"></a>\n\nOptimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.\n\n> [!NOTE]  \n> See [`dusty-nv.github.io/NanoLLM`](https://dusty-nv.github.io/NanoLLM) for docs and [**Jetson AI Lab**](https://www.jetson-ai-lab.com/tutorial_nano-llm.html) for tutorials.\n\nLatest Release:  [24.7](https://dusty-nv.github.io/NanoLLM/releases.html)  ([`dustynv/nano_llm:24.7-r36.2.0`](https://hub.docker.com/r/dustynv/nano_llm/tags)) <br/>","github_created_at":"2024-03-31T21:38:34+00:00","created_at":"2026-07-11T11:46:51.421606+00:00","updated_at":"2026-08-25T12:01:31.04432+00:00","categories":[{"slug":"computer-vision","name":"Computer Vision","url":"https://www.graphcanon.com/categories/computer-vision","markdown_url":"https://www.graphcanon.com/categories/computer-vision.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/computer-vision"},{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"speech-audio","name":"Speech & Audio","url":"https://www.graphcanon.com/categories/speech-audio","markdown_url":"https://www.graphcanon.com/categories/speech-audio.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/speech-audio"},{"slug":"vector-databases","name":"Vector Databases","url":"https://www.graphcanon.com/categories/vector-databases","markdown_url":"https://www.graphcanon.com/categories/vector-databases.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/vector-databases"}],"tags":[{"slug":"edge-ai","name":"edge-ai"},{"slug":"llm-inference","name":"llm-inference"},{"slug":"multimodal","name":"multimodal"},{"slug":"rag","name":"rag"},{"slug":"speech","name":"speech"},{"slug":"vector-database","name":"vector-database"},{"slug":"vision-transformer","name":"vision-transformer"}],"trust":{"provenance":{"is_fork":false,"github_id":780161909,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-25T12:01:30.239Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":676,"last_release_at":null,"stars_delta_30d":2,"open_issues_delta_30d":2},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T11:46:52.613Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-08-25T12:01:30.708Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-08-25T12:01:30.708Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-08-25T12:01:30.708Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["When building edge-ai solutions requiring optimized local inference","For multimodal AI projects needing integrations with speech and image models"],"when_not_to_use":["In scenarios where a fully cloud-based solution is preferred over local inference","If the project does not benefit from multimodal or RAG capabilities"],"source":"enrich:decision_facts","observed_at":"2026-07-17T13:26:10.395Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"NanoLLM optimizes local inference for LLMs via HuggingFace-compatible APIs, supporting quantization and multimodal applications like vision, speech, RAG, and vector databases."}]}}