GraphCanon updated today · GitHub synced today
Decision brief
Krasis is designed to offer efficient large model inference on consumer-grade hardware through hybrid CPU-GPU execution and high-performance optimization.
Good fit when
- - When aiming for efficient operation of larger language models with limited VRAM, as Krasis optimizes memory utilization specifically to support this scenario.
- - If you seek an alternative runtime that can handle the latest GGUF-supported models without significant compromises in performance compared to other toolkits like llama-cpp.
Avoid when
- - Avoid using Krasis if your hardware setup does not include both CPU and GPU capabilities, as its hybrid execution relies on utilizing both components for optimal performance.
- - If you prioritize running lightweight models with minimal memory footprint on low-end devices, Krasis might not be the ideal choice given it is optimized for larger-scale model inference.
Observed Jul 15, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Very active (1d since push)
- As of today
- Provenance
- Not a fork · Personal account
- As of today
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
git clone https://github.com/brontoguana/krasisSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
Krasis is designed to enable efficient running of larger language models with constraints on consumer-grade VRAM. It supports high-performance inference, hybrid CPU-GPU execution, and is an alternative to existing solutions such as llama-cpp.
Capability facts
- CLI
- CLI entrypoint
Source: pyproject.toml:[project.scripts] · Aug 25, 2026
- Languages
- c++, python
Source: github.language+pyproject.toml · Aug 25, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Aug 25, 2026)
It installs Krasis for the current user, includes its own Python runtime, andSource link
Tags
README
Install
The current release is
v1.0.16.
Native Windows: Download the Krasis Windows installer.
It installs Krasis for the current user, includes its own Python runtime, and
adds Krasis to the Start Menu.
Linux or WSL2:
curl -sSf https://raw.githubusercontent.com/brontoguana/krasis/main/install.sh | bash -s -- prerelease
See all releases for older versions and individual wheel/source assets.
You can contact me here, but for bugs, setup problems, model requests, or feature requests please open a GitHub issue.
If you want to monitor Krasis during runs, check out ktop.
Tradeoffs And Requirements
- Krasis currently targets NVIDIA GPUs with CUDA, including Ampere and newer architectures. The production HQQ attention and compact KV cache modes do not require FP8 support.
- Input models should be BF16 safetensors from Hugging Face or another local safetensors source.
- First run is slower because Krasis builds optimized local caches. Later runs reuse those caches.
- Disk usage must cover the source model plus Krasis cache artifacts under
~/.krasis. - System RAM should be sized for the selected quantized cache and HCS backing store. Larger models need substantial RAM even when GPU VRAM is limited.
- Production runs should use quantized INT4/INT8 expert caches and HQQ attention. BF16-heavy modes are validation/debug modes, not normal deployment targets.
Requirements
- Native x86-64 Windows, Linux (including Ubuntu 24.04+), or WSL2
- Python 3.10+ on Linux/WSL; native Windows uses the release-pinned private Python included by the installer
- NVIDIA GPU with CUDA drivers installed
- Rust is only needed for source builds, not normal wheel installs
- Enough disk/RAM for the source model and generated Krasis caches
1. Install Krasis
Linux/WSL:
curl -sSf https://raw.githubusercontent.com/brontoguana/krasis/main/install.sh | bash -s -- prerelease
This creates a managed environment at ~/.krasis/venv, installs Krasis,
symlinks commands into ~/.local/bin, and updates PATH for the current shell.
No sudo is required for the Krasis install itself. Omit prerelease when
installing the latest stable release.
Native Windows:
Download KrasisSetup-1.0.16-win64.exe.
The installer creates a per-user install under
%LOCALAPPDATA%\Programs\Krasis, installs and validates a release-pinned
private Python/Krasis/PyTorch runtime, and adds Krasis and Krasis Manager
shortcuts to the Start Menu folder. It never uses or modifies a system Python.
Krasis opens the native interactive launcher in a maximized, resizable
console; Krasis Manager starts the localhost management dashboard. Models
and caches still live under
%USERPROFILE%\.krasis. The first install downloads the pinned CUDA PyTorch
wheel and can take several minutes.
Native Windows packages Marlin, FlashAttention, and FLA sidecars for supported Ampere and newer NVIDIA architectures.
2. Install CUDA Dependencies
krasis-setup
This installs runtime CUDA/PyTorch dependencies when needed. It is usually only required once per machine.
License
SSPL-1.0
Krasis is free to use, modify, and distribute.
If you want to support the project or offer Krasis as part of a commercial product or a hosted/managed service, please get in touch.
For agents
This page has a .md twin and JSON over the API.