text-embeddings-inference
Blazing fast inference solution for text embeddings models
GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026
52views this month
Decision brief
text-embeddings-inference is a Rust-based tool for fast inference of text embeddings models, deployable via Docker with GPU support.
Good fit when
- When you need blazing fast inference for text embeddings models, as this tool is optimized for speed.
- If you are working with Hugging Face models and want to leverage GPU acceleration for faster inference.
Avoid when
- If your environment does not support CUDA version 12.2 or higher, as the tool requires compatible NVIDIA drivers for GPU support.
- When you do not have access to a Docker environment or do not wish to use Docker for deployment.
- Pricing:
- freemium - The tool is free to use under the Apache-2.0 license, but users may incur costs based on their infrastructure and usage.
- Requirements:
- Min 4 GB RAM; Requires Docker; NVIDIA Container Toolkit must be installed for GPU support.; Compatible NVIDIA drivers are required for CUDA version 12.2 or higher.
Observed Sep 18, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Very active (0d since push)
- As of Sep 18, 2026
- Provenance
- Not a fork · Organization account
- As of Sep 18, 2026
- Security (OSV)
- No lockfile
- As of Sep 18, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Backing
Company context for Hugging Face. Display-only - separate from trust and ranking.
- Company
- Hugging Face·GitHub org profile·Jul 11, 2026
- Employees
- 160·Wikidata (P1128 employees)·Jul 11, 2026
- Funding
- $235,000,000 (2023-08)·GraphCanon curated seed (public press)·Jul 11, 2026
- Commercial model
- OSS + managed cloud·GraphCanon curated seed·Jul 11, 2026
Install
cargo add text-embeddings-inferenceSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
A Rust-based tool for fast inference of text embeddings models, deployable via Docker with GPU support.
Capability facts
- Deploy
- Self-host
Source: dockerfile:Dockerfile · Sep 18, 2026
- Docker
- Dockerfile present
Source: dockerfile:Dockerfile · Sep 18, 2026
- Languages
- rust
Source: github.language · Sep 18, 2026
Categories
Graph entities
Tags
README
Docker And then you can make requests like Note: To use GPUs, you need to install the NVIDIA Container Toolkit. NVIDIA drivers on your machine need to be compatible with CUDA version 12.2 or higher. To see all options to serve your models: `` text embeddings inference save pretra...
For agents
This page has a .md twin and JSON over the API.