GraphCanon updated 1d · GitHub synced 1d
Decision brief
Kvcached is designed for optimizing dynamic GPU sharing and multiplexing scenarios, beneficial for LLM inference and serving operations.
Good fit when
- If you are looking to optimize performance in environments that require dynamic allocation of GPUs among multiple processes or tasks.
- When needing a specialized KV cache system tailored specifically towards the unique challenges posed by running large language models (LLMs) alongside efficient GPU utilization.
Avoid when
- For applications where static, predefined resource allocations are sufficient and do not require dynamic adjustments to GPU usage.
- In scenarios that prioritize simplicity over sophisticated resource management, as Kvcached may add unnecessary complexity with its advanced features for dynamic GPU sharing.
Observed Jul 14, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Very active (1d since push)
- As of 1d
- Provenance
- Not a fork · Organization account
- As of 1d
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install kvcachedSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
Developed in Python, this project focuses on providing an elastic KV cache designed to optimize dynamic GPU sharing and multiplexing scenarios, with implications for LLM inference and serving.
Capability facts
- CLI
- CLI entrypoint
Source: pyproject.toml:[project.scripts] · Aug 25, 2026
- Languages
- python
Source: github.language+pyproject.toml · Aug 25, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Aug 25, 2026)
pip install kvcached --no-build-isolationSource link
Tags
README
Install from PyPI
pip install kvcached --no-build-isolation
Using Docker
kvcached installed with original engine dockers.
docker pull ghcr.io/ovg-project/kvcached-sglang:latest # kvcached-v0.1.5-sglang-v0.5.10
docker pull ghcr.io/ovg-project/kvcached-vllm:latest # kvcached-v0.1.5-vllm-v0.19.0
We prepare an all-in-one docker for developers:
docker pull ghcr.io/ovg-project/kvcached-dev:latest
More instructions can be found here. GB200 dockers are on the way.
For agents
This page has a .md twin and JSON over the API.