mteb
State-of-the-art evaluation of embeddings across languages and modalities
GraphCanon updated 1mo · GitHub synced 1mo
Decision brief
MTEB is an evaluator for embedding models across languages and modalities under the Apache-2.0 license.
Good fit when
- You require benchmarking tools specifically designed for state-of-the-art embedding evaluations in low-resource NLP contexts.
- Evaluations spanning multiple tasks like semantic search, clustering, and information retrieval are needed.
Avoid when
- Your project exclusively focuses on a single language or modality not covered by MTEB’s broad scope.
- You need a tool that supports operations beyond evaluation, such as model training or fine-tuning directly within the same system.
Observed Jul 12, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Very active (0d since push)
- As of 1mo
- Provenance
- Not a fork · Organization account
- As of 1mo
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install mtebSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
MTEB is designed for evaluating the performance of various embedding models across different tasks and languages. It supports a broad spectrum of applications including semantic search, clustering, and information retrieval.
Capability facts
- Deploy
- Self-host
Source: dockerfile:Dockerfile · Jul 22, 2026
- Docker
- Dockerfile present
Source: dockerfile:Dockerfile · Jul 22, 2026
- CLI
- CLI entrypoint
Source: pyproject.toml:[project.scripts] · Jul 22, 2026
- Languages
- python
Source: github.language+pyproject.toml · Jul 22, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Tags
README
Installation
You can install mteb simply using pip or uv. For more on installation please see the documentation.
pip install mteb
For faster installation, you can also use uv:
uv add mteb
For agents
This page has a .md twin and JSON over the API.