llm-inference-solutions
A collection of all available inference solutions for the LLMs
GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026
32views this month
Decision brief
Curated listings of tools for efficient inference and deployment of LLMs with details on hardware support, features, and licenses.
Good fit when
- Need a comprehensive catalog to compare multiple inference solutions for LLMs like vLLM's memory management or Triton Inference Server's framework diversity
- Require insights into different licensing options like MIT or Apache 2.0 across tools
Avoid when
- Looking for direct technical implementation details instead of a curated list, as it primarily serves as an overview repository
- In need of real-time updates since the repository's content may not be continuously updated to reflect new developments in inference solutions
Observed Jul 12, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Dormant (554d since push)
- As of Sep 7, 2026
- Provenance
- Not a fork · Personal account
- As of Sep 7, 2026
- Security (OSV)
- No lockfile
- As of Jul 11, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
git clone https://github.com/mani-kantap/llm-inference-solutionsSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
This repository lists various tools and frameworks used for efficient inference and serving of large language models (LLMs). It provides an overview including supported hardware, key features, and licenses.
Capability facts
No sourced capability facts yet. Facts appear after ingest scans repo manifests (Dockerfile, package.json, MCP configs).
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Sep 7, 2026)
//github.com/NVIDIA/TensorRT-LLM) | NVIDIA | Provides users with an easy-to-use Python API to define LLMs and build TensorRT engines. | GPU | TensorRT optimization, hiSource link
Tags
README
llm inference solutions A collection of all available inference solutions for the LLMs Name Organization Description Supported Hardware Key Features License vLLM UC Berkeley High throughput and memory efficient inference and serving engine for LLMs. CPU, GPU PagedAttention for op...
For agents
This page has a .md twin and JSON over the API.