infinity
High-throughput, low-latency serving engine for text-embeddings, reranking models, CLIP, CLAP, and COLPAli
GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026
60views this month
Decision brief
Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models, CLIP, CLAP, and COLPAli, offering support for various hardware accelerators via CLI and Docker containers.
Good fit when
- Use Infinity when you need to serve text-embeddings and reranking models with high throughput and low latency, especially when you have access to hardware accelerators like GPUs or ROCm.
- Consider Infinity if you require support for multiple models simultaneously, as it allows launching multiple models at once with the CLI.
Avoid when
- Avoid using Infinity if you do not have access to hardware accelerators and prefer a lightweight solution, as the specialized Docker images for ROCm and ONNX-GPU are not built via CI/CD and may not be
- Do not use Infinity if you need a serving engine that is frequently updated via CI/CD, as some specialized Docker images are not automatically built and may require manual version pinning.
Observed Sep 18, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Slowing (178d since push)
- As of Sep 18, 2026
- Provenance
- Not a fork · Personal account
- As of Sep 18, 2026
- Security (OSV)
- No lockfile
- As of Sep 18, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install infinitySimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
Infinity provides a CLI and Docker containers for serving text-embeddings and reranking models with support for various hardware accelerators.
Capability facts
- Languages
- python
Source: github.language · Sep 18, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Sep 18, 2026)
### Launch the cli via pip installSource link
Tags
README
Launch the cli via pip install After your pip install, with your venv active, you can run the CLI directly. Check the command to get a description for all parameters. Launch the CLI using a pre built docker container (recommended) Instead of installing the CLI via pip, you may al...
For agents
This page has a .md twin and JSON over the API.