{"data":{"slug":"epistates-pmetal","name":"pmetal","tagline":"High-performance Apple Silicon framework for LLM inference and fine-tuning","github_url":"https://github.com/Epistates/pmetal","owner":"Epistates","repo":"pmetal","owner_avatar_url":"https://avatars.githubusercontent.com/u/225645584?v=4","primary_language":"Rust","stars":317,"forks":26,"topics":["ai","ane","apple-silicon","deep-learning","distillation","fine-tuning","gguf","inference-server","llm","llm-inference","llm-training","lora","machine-learning","macos","metal","mlx","qlora","quantization","transformers","tui"],"archived":false,"github_pushed_at":"2026-09-17T15:41:39+00:00","maintenance_label":"Very active","stars_delta_30d":11,"url":"https://www.graphcanon.com/tools/epistates-pmetal","markdown_url":"https://www.graphcanon.com/tools/epistates-pmetal.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/epistates-pmetal","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=epistates-pmetal","description":"PMetal: high-performance Apple Silicon framework for local LLM inference, LoRA/QLoRA fine-tuning, serving, quantization, and MLX/Metal acceleration.","homepage_url":"https://pmetal.io","license":"Other","open_issues":8,"watchers":3,"ai_summary":"PMetal offers high-performance local Large Language Model (LLM) inference, LoRA/QLoRA fine-tuning, serving, quantization, and uses MLX/Metal acceleration on Apple Silicon chips.","readme_excerpt":"### Quick Start (Easy API)\n\n```python\nimport pmetal\n\n---\n\n## Installation\n\nPrebuilt signed binaries are available on the [Releases](https://github.com/Epistates/pmetal/releases) page.\n\nCrates are available on [crates.io](https://crates.io/crates/pmetal).\n\nBuild from source:\n\n```bash\ngit clone https://github.com/epistates/pmetal.git && cd pmetal\ncargo build --release          # CLI + TUI\ncd crates/pmetal-gui && bun install && bun tauri build  # GUI (optional)\n```\n\n---\n\n## Hardware Support\n\nPMetal automatically detects Apple Silicon capabilities at startup and tunes kernel parameters accordingly.\n\n| Chip Family | GPU Family | NAX | ANE | UltraFusion | Status |\n|-------------|-----------|-----|-----|-------------|--------|\n| M1 / Pro / Max / Ultra | Apple7 | - | 16 cores | Ultra: 2-die | Fully supported |\n| M2 / Pro / Max / Ultra | Apple8 | - | 16 cores | Ultra: 2-die | Fully supported |\n| M3 / Pro / Max / Ultra | Apple9 | - | 16 cores | Ultra: 2-die | Fully supported |\n| M4 / Pro / Max / Ultra | Apple9 | - | 16 cores | Ultra: 2-die | Fully supported |\n| **M5 / Pro / Max / Ultra** | **Apple10** | **Yes** | **16 cores** | **Ultra: 2-die** | **Fully supported** |\n\n**Auto-detected features**: GPU family, device tier, core counts, memory bandwidth, dynamic caching, mesh shaders, NAX (M5+), UltraFusion topology (via `sysctl hw.packages`), ANE availability.\n\n**Tier-based kernel tuning**: Matrix tile sizes, FlashAttention block sizes, fused kernel threadgroup sizes, and batch multipliers are automatically selected based on device tier (Base/Pro/Max/Ultra) and GPU family. See [`docs/hardware-support.md`](docs/hardware-support.md) for the full tuning matrix.\n\n---\n\n### Training Infrastructure\n\n- **Sequence Packing**: Efficiently pack multiple sequences into single batches for 2-5x throughput. Enabled by default\n- **Gradient Checkpointing**: Trade compute for memory on large models with configurable layer grouping\n- **Adaptive LR**: EMA-based anomaly detection with spike recovery, plateau reduction, and divergence detection\n- **Callback System**: `TrainingCallback` trait with lifecycle hooks (`on_step_start`, `on_step_end`, `should_stop`) for metrics logging, progress reporting, and clean cancellation\n- **Checkpoint Management**: Save and resume training from checkpoints with best-loss rollback\n- **Tool/Function Calling**: Chat templates with native tool definitions for Qwen, Gemma 4, Llama 3.1+, Mistral v3+, and DeepSeek\n- **Schedule-Free Optimizer**: Memory-efficient optimizer without learning rate schedules\n- **Metal Fused Optimizer**: GPU-accelerated AdamW parameter updates\n- **8-bit Adam**: Memory-efficient optimizer for large models\n- **LoRA+**: Differentiated learning rates for LoRA A and B matrices\n- **NEFTune**: Noise-augmented fine-tuning for improved generation quality\n- **Distributed Training**: mDNS auto-discovery, Ring All-Reduce with gradient compression\n\n---\n\n## License\n\nLicensed under either of MIT or Apache-2.0.","github_created_at":"2026-01-12T17:27:23+00:00","created_at":"2026-07-15T11:19:36.633777+00:00","updated_at":"2026-09-20T05:20:00.447942+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"ai","name":"ai"},{"slug":"ane","name":"ane"},{"slug":"apple-silicon","name":"apple-silicon"},{"slug":"deep-learning","name":"deep-learning"},{"slug":"fine-tuning","name":"fine-tuning"},{"slug":"inference-server","name":"inference-server"},{"slug":"llm-inference","name":"llm-inference"},{"slug":"mlx","name":"mlx"}],"trust":{"provenance":{"is_fork":false,"github_id":1132959458,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-20T05:19:58.205Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":0,"days_since_push":2,"last_release_at":"2026-05-08T11:38:51Z","stars_delta_30d":11,"open_issues_delta_30d":-1},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-15T11:19:38.714Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-20T05:19:59.678Z"},"languages":{"value":["rust"],"source":"github.language","observed_at":"2026-09-20T05:19:59.678Z"},"license_spdx":{"value":"Other","source":"github.license","observed_at":"2026-09-20T05:19:59.678Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["For optimal performance on Apple M1-M5 series, when leveraging GPU and ANE for LLMs is crucial.","If Rust integration or Metal acceleration is required.","When developing applications that need to run locally, without reliance on cloud-based services."],"when_not_to_use":["Avoid if support for Nvidia GPUs or Intel CPUs is needed.","Not suitable when flexibility in language models exceeds pmetal's capabilities with only specific transformer models supported natively.","Steer clear if the project environment does not support Rust or if Apple-specific hardware acceleration is unnecessary."],"source":"enrich:decision_facts","observed_at":"2026-07-17T12:33:57.351Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"Specializes in high-performance local Large Language Model inference and fine-tuning on Apple Silicon hardware using MLX/Metal."},{"label":"License detail","value":"Dual-licensed under MIT or Apache-2.0, offering flexible open-source options for commercial and non-commercial projects alike."}]}}