{"data":{"slug":"epistates-pmetal","name":"pmetal","tagline":"PMetal: high-performance Apple Silicon framework for local LLM inference, LoRA/QLoRA fine-tuning, serving, quantization, and MLX/Metal acceleration.","github_url":"https://github.com/Epistates/pmetal","owner":"Epistates","repo":"pmetal","owner_avatar_url":"https://avatars.githubusercontent.com/u/225645584?v=4","primary_language":"Rust","stars":303,"forks":22,"topics":["ai","ane","apple-silicon","deep-learning","distillation","fine-tuning","gguf","inference-server","llm","llm-inference","llm-training","lora","machine-learning","macos","metal","mlx","qlora","quantization","transformers","tui"],"archived":false,"github_pushed_at":"2026-06-05T11:56:14+00:00","maintenance_label":"Steady","url":"https://www.graphcanon.com/tools/epistates-pmetal","markdown_url":"https://www.graphcanon.com/tools/epistates-pmetal.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/epistates-pmetal","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=epistates-pmetal","description":"PMetal: high-performance Apple Silicon framework for local LLM inference, LoRA/QLoRA fine-tuning, serving, quantization, and MLX/Metal acceleration.","homepage_url":"https://pmetal.io","license":"Other","open_issues":7,"watchers":3,"ai_summary":null,"readme_excerpt":"### Quick Start (Easy API)\n\n```python\nimport pmetal\n\n---\n\n## Installation\n\nPrebuilt signed binaries are available on the [Releases](https://github.com/Epistates/pmetal/releases) page.\n\nCrates are available on [crates.io](https://crates.io/crates/pmetal).\n\nBuild from source:\n\n```bash\ngit clone https://github.com/epistates/pmetal.git && cd pmetal\ncargo build --release          # CLI + TUI\ncd crates/pmetal-gui && bun install && bun tauri build  # GUI (optional)\n```\n\n---\n\n## Hardware Support\n\nPMetal automatically detects Apple Silicon capabilities at startup and tunes kernel parameters accordingly.\n\n| Chip Family | GPU Family | NAX | ANE | UltraFusion | Status |\n|-------------|-----------|-----|-----|-------------|--------|\n| M1 / Pro / Max / Ultra | Apple7 | - | 16 cores | Ultra: 2-die | Fully supported |\n| M2 / Pro / Max / Ultra | Apple8 | - | 16 cores | Ultra: 2-die | Fully supported |\n| M3 / Pro / Max / Ultra | Apple9 | - | 16 cores | Ultra: 2-die | Fully supported |\n| M4 / Pro / Max / Ultra | Apple9 | - | 16 cores | Ultra: 2-die | Fully supported |\n| **M5 / Pro / Max / Ultra** | **Apple10** | **Yes** | **16 cores** | **Ultra: 2-die** | **Fully supported** |\n\n**Auto-detected features**: GPU family, device tier, core counts, memory bandwidth, dynamic caching, mesh shaders, NAX (M5+), UltraFusion topology (via `sysctl hw.packages`), ANE availability.\n\n**Tier-based kernel tuning**: Matrix tile sizes, FlashAttention block sizes, fused kernel threadgroup sizes, and batch multipliers are automatically selected based on device tier (Base/Pro/Max/Ultra) and GPU family. See [`docs/hardware-support.md`](docs/hardware-support.md) for the full tuning matrix.\n\n---\n\n### Training Infrastructure\n\n- **Sequence Packing**: Efficiently pack multiple sequences into single batches for 2-5x throughput. Enabled by default\n- **Gradient Checkpointing**: Trade compute for memory on large models with configurable layer grouping\n- **Adaptive LR**: EMA-based anomaly detection with spike recovery, plateau reduction, and divergence detection\n- **Callback System**: `TrainingCallback` trait with lifecycle hooks (`on_step_start`, `on_step_end`, `should_stop`) for metrics logging, progress reporting, and clean cancellation\n- **Checkpoint Management**: Save and resume training from checkpoints with best-loss rollback\n- **Tool/Function Calling**: Chat templates with native tool definitions for Qwen, Llama 3.1+, Mistral v3+, and DeepSeek\n- **Schedule-Free Optimizer**: Memory-efficient optimizer without learning rate schedules\n- **Metal Fused Optimizer**: GPU-accelerated AdamW parameter updates\n- **8-bit Adam**: Memory-efficient optimizer for large models\n- **LoRA+**: Differentiated learning rates for LoRA A and B matrices\n- **NEFTune**: Noise-augmented fine-tuning for improved generation quality\n- **Distributed Training**: mDNS auto-discovery, Ring All-Reduce with gradient compression\n\n---\n\n## License\n\nLicensed under either of MIT or Apache-2.0.","github_created_at":"2026-01-12T17:27:23+00:00","created_at":"2026-07-15T11:19:36.633777+00:00","updated_at":"2026-07-15T11:19:40.27685+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"ai","name":"ai"},{"slug":"ane","name":"ane"},{"slug":"apple-silicon","name":"apple-silicon"},{"slug":"deep-learning","name":"deep-learning"},{"slug":"distillation","name":"distillation"},{"slug":"fine-tuning","name":"fine-tuning"},{"slug":"gguf","name":"gguf"},{"slug":"inference-server","name":"inference-server"}],"trust":{"provenance":{"is_fork":false,"github_id":1132959458,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-15T11:19:37.793Z","maintenance":{"label":"Steady","score":60,"methodology":"github_public_v1","releases_90d":1,"days_since_push":39,"last_release_at":"2026-05-08T11:38:51Z"},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-15T11:19:38.714Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-15T11:19:37.596Z"},"languages":{"value":["rust"],"source":"github.language","observed_at":"2026-07-15T11:19:37.596Z"},"license_spdx":{"value":"Other","source":"github.license","observed_at":"2026-07-15T11:19:37.596Z"}}}}