{"data":{"slug":"bodhisearch-bodhiapp","name":"BodhiApp","tagline":"Run Open Source/Open Weight LLMs locally with OpenAI compatible APIs","github_url":"https://github.com/BodhiSearch/BodhiApp","owner":"BodhiSearch","repo":"BodhiApp","owner_avatar_url":"https://avatars.githubusercontent.com/u/138853064?v=4","primary_language":"TypeScript","stars":139,"forks":11,"topics":["gemma","generative-ai","llama","llm","local-llm","localllm","mistral","open-source-llm","private-llm"],"archived":false,"github_pushed_at":"2026-09-20T01:29:51+00:00","maintenance_label":"Very active","stars_delta_30d":3,"url":"https://www.graphcanon.com/tools/bodhisearch-bodhiapp","markdown_url":"https://www.graphcanon.com/tools/bodhisearch-bodhiapp.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/bodhisearch-bodhiapp","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=bodhisearch-bodhiapp","description":"Run Open Source/Open Weight LLMs locally with OpenAI compatible APIs","homepage_url":"https://getbodhi.app/","license":null,"open_issues":12,"watchers":5,"ai_summary":"Bodhi App supports the deployment of local large language models through various hardware acceleration methods including CPU, CUDA (NVIDIA), ROCm (AMD), and Vulkan for multi-vendor GPU support. Distributed as Docker images, it allows for flexible performance tuning depending on hardware configurations.","readme_excerpt":"## Installation\n\nBodhi App is currently released only for the Mac platform. You can install it either by downloading the release from the GitHub release page or using Homebrew.\n\n---\n\n### Docker\n\nBodhi App is available as Docker images with multiple hardware acceleration variants. Each variant is optimized for specific hardware configurations to provide the best performance.\n\n#### Available Variants\n\n- **CPU Variant**: Standard CPU-only inference for maximum compatibility (multi-platform: AMD64 + ARM64)\n- **CUDA Variant**: NVIDIA GPU acceleration for faster inference on NVIDIA hardware\n- **ROCm Variant**: AMD GPU acceleration for AMD graphics cards\n- **Vulkan Variant**: Cross-vendor GPU acceleration supporting multiple GPU vendors\n\n#### Quick Start\n\n**CPU Variant (Most Compatible - Auto-detects AMD64/ARM64):**\n```bash\ndocker run -p 8080:8080 \\\n  -v ./bodhi_home:/data/bodhi_home \\\n  -v ./hf_home:/data/hf_home \\\n  ghcr.io/bodhisearch/bodhiapp:latest-cpu\n```\n\n**CUDA Variant (NVIDIA GPU):**\n```bash\ndocker run --gpus all -p 8080:8080 \\\n  -v ./bodhi_home:/data/bodhi_home \\\n  -v ./hf_home:/data/hf_home \\\n  ghcr.io/bodhisearch/bodhiapp:latest-cuda\n```\n\n**ROCm Variant (AMD GPU):**\n```bash\ndocker run --device=/dev/kfd --device=/dev/dri --group-add video -p 8080:8080 \\\n  -v ./bodhi_home:/data/bodhi_home \\\n  -v ./hf_home:/data/hf_home \\\n  ghcr.io/bodhisearch/bodhiapp:latest-rocm\n```\n\n**Vulkan Variant (Cross-vendor GPU):**\n```bash\ndocker run --device=/dev/dri -p 8080:8080 \\\n  -v ./bodhi_home:/data/bodhi_home \\\n  -v ./hf_home:/data/hf_home \\\n  ghcr.io/bodhisearch/bodhiapp:latest-vulkan\n```\n\n#### Hardware Requirements\n\n- **CPU**: Standard x86_64 (AMD64) or ARM64 processor (auto-detected)\n- **CUDA**: NVIDIA GPU with CUDA 12.4+ support and compatible drivers\n- **ROCm**: AMD GPU with ROCm 6.4+ support and compatible drivers\n- **Vulkan**: GPU with Vulkan API support and compatible drivers\n\n#### Volume Mounts\n\n- `/data/bodhi_home`: Application data, configuration, and downloaded models\n- `/data/hf_home`: HuggingFace cache directory for model downloads\n\nAfter starting the container, Bodhi App will be available at `http://localhost:8080`.\n\n---\n\n### Installation\n\n```bash\nnpm install @bodhiapp/ts-client\n```","github_created_at":"2024-02-29T06:58:12+00:00","created_at":"2026-07-15T11:02:15.071373+00:00","updated_at":"2026-09-20T05:09:38.007588+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"}],"tags":[{"slug":"gemma","name":"gemma"},{"slug":"generative-ai","name":"generative-ai"},{"slug":"llama","name":"llama"},{"slug":"llm","name":"llm"},{"slug":"local-llm","name":"local-llm"},{"slug":"localllm","name":"localllm"},{"slug":"mistral","name":"mistral"},{"slug":"open-source-llm","name":"open-source-llm"}],"trust":{"provenance":{"is_fork":false,"github_id":765037941,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-20T05:09:35.635Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":30,"days_since_push":0,"last_release_at":"2026-09-19T09:16:22Z","stars_delta_30d":3,"open_issues_delta_30d":2},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-15T11:02:16.605Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-20T05:09:36.657Z"},"languages":{"value":["typescript"],"source":"github.language","observed_at":"2026-09-20T05:09:36.657Z"}},"decision_facts":{"hosting":null,"pricing":{"model":"unknown","summary":"Pricing details are not mentioned in the repository data."},"requirements":{"notes":["Requires Docker environment. Specific model requirements vary depending on the hardware variant chosen."],"min_ram_gb":null,"requires_docker":true},"constraints":{"min_ram_gb":null,"pricing_model":"unknown","requires_docker":true},"when_to_use":["You need to deploy LLMs locally with flexible hardware support including AMD, NVIDIA GPUs, and CPUs.","Your workload benefits from maximum compatibility across x86_64 and ARM64 architectures using the CPU variant.","You aim for optimized performance on specific hardware configurations such as NVIDIA or AMD GPUs through dedicated variants."],"when_not_to_use":["Your project strictly requires non-local deployment options, as BodhiApp focuses on local hosting of models.","If your environment is limited to unsupported GPU hardware or lacks adequate drivers for CUDA, ROCm, or Vulkan acceleration methods.","You need support beyond Mac platforms as BodhiApp does not yet provide installation instructions for other operating systems."],"source":"enrich:decision_facts","observed_at":"2026-07-17T09:20:55.966Z"},"constraint_facets":{"min_ram_gb":null,"pricing_model":"unknown","requires_docker":true},"decision_summary":[{"label":"Pricing","value":"unknown - Pricing details are not mentioned in the repository data."},{"label":"Requirements","value":"Requires Docker; Requires Docker environment. Specific model requirements vary depending on the hardware variant chosen."},{"label":"Adopt for","value":"BodhiApp streamlines local deployment of open-source and open-weight LLMs via Docker images, compatible with multiple hardware acceleration methods."},{"label":"License detail","value":"The license information for BodhiApp has not been provided."}]}}