{"data":{"slug":"ovg-project-kvcached","name":"kvcached","tagline":"Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond","github_url":"https://github.com/ovg-project/kvcached","owner":"ovg-project","repo":"kvcached","owner_avatar_url":"https://avatars.githubusercontent.com/u/213673672?v=4","primary_language":"Python","stars":1424,"forks":165,"topics":["elastic-kvcache","gpu-mutiplexing","gpu-sharing","inference-engine","kvcache","kvcache-optimization","kvcached","llm","llm-framework","llm-inference","llm-serving","ollama","online-offline-coserve","serverless","sglang","vllm"],"archived":false,"github_pushed_at":"2026-09-18T19:17:57+00:00","maintenance_label":"Very active","stars_delta_30d":309,"url":"https://www.graphcanon.com/tools/ovg-project-kvcached","markdown_url":"https://www.graphcanon.com/tools/ovg-project-kvcached.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/ovg-project-kvcached","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=ovg-project-kvcached","description":"Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond","homepage_url":null,"license":"Apache-2.0","open_issues":114,"watchers":16,"ai_summary":"Developed in Python, this project focuses on providing an elastic KV cache designed to optimize dynamic GPU sharing and multiplexing scenarios, with implications for LLM inference and serving.","readme_excerpt":"### Install from PyPI\n\n```bash\npip install kvcached --no-build-isolation\n```\n\n---\n\n### Using Docker\n\nkvcached installed with original engine dockers.\n\n```bash\ndocker pull ghcr.io/ovg-project/kvcached-sglang:latest   # kvcached-v0.1.5-sglang-v0.5.10\ndocker pull ghcr.io/ovg-project/kvcached-vllm:latest     # kvcached-v0.1.5-vllm-v0.19.0\n```\n\nWe prepare an all-in-one docker for developers:\n\n```bash\ndocker pull ghcr.io/ovg-project/kvcached-dev:latest\n```\n\nMore instructions can be found [here](https://github.com/ovg-project/kvcached/blob/main/docker/README.md). GB200 dockers are on the way.","github_created_at":"2025-05-27T17:34:02+00:00","created_at":"2026-07-11T11:43:56.316235+00:00","updated_at":"2026-09-20T06:34:02.252881+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"}],"tags":[{"slug":"elastic-kvcache","name":"elastic-kvcache"},{"slug":"gpu-sharing","name":"gpu-sharing"},{"slug":"kvcache","name":"kvcache"},{"slug":"llm-inference","name":"llm-inference"}],"trust":{"provenance":{"is_fork":false,"github_id":991482082,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-19T18:31:33.874Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":0,"days_since_push":0,"last_release_at":"2026-04-07T17:25:49Z","stars_delta_30d":309,"open_issues_delta_30d":10},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T11:43:57.439Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-19T18:31:34.904Z"},"has_cli":{"value":true,"source":"pyproject.toml:[project.scripts]","observed_at":"2026-09-19T18:31:34.904Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-09-19T18:31:34.904Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-09-19T18:31:34.904Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["If you are looking to optimize performance in environments that require dynamic allocation of GPUs among multiple processes or tasks.","When needing a specialized KV cache system tailored specifically towards the unique challenges posed by running large language models (LLMs) alongside efficient GPU utilization."],"when_not_to_use":["For applications where static, predefined resource allocations are sufficient and do not require dynamic adjustments to GPU usage.","In scenarios that prioritize simplicity over sophisticated resource management, as Kvcached may add unnecessary complexity with its advanced features for dynamic GPU sharing."],"source":"enrich:decision_facts","observed_at":"2026-07-14T20:47:32.350Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"Kvcached is designed for optimizing dynamic GPU sharing and multiplexing scenarios, beneficial for LLM inference and serving operations."}]}}