GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026
50views this month
Decision brief
VLMEvalKit is an open-source Python evaluation toolkit for large vision-language models that offers one-command evaluation with support for various benchmarks and models.
Good fit when
- When you need to evaluate models supporting thinking mode, as it provides a custom split_thinking function improving accuracy.
- For handling long response outputs exceeding 16k or 32k tokens without data truncation by enabling TSV format saving for prediction files.
Avoid when
- If your project requires evaluation tools that generate Excel files with individual cells larger than the default support of 32,767 characters and cannot switch to TSV format.
- When you do not need generation-based evaluation methods with exact matching and LLM-based answer extraction.
Observed Jul 14, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Very active (0d since push)
- As of Aug 17, 2026
- Provenance
- Not a fork · Organization account
- As of Aug 17, 2026
- Security (OSV)
- 16 low (16 low)
- As of Jul 11, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install VLMEvalKitHow it fits your stack(9)
Typed graph edges - alternatives, integrations, successors, and dependencies. Ranked by relationship type, not raw GitHub stars.
Integrates
Related
Relationship graph
Optional deeper exploration of typed edges and category neighbours.
Similar tools
Same-category neighbours not already linked as typed edges.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
VLMEvalKit is an open-source Python-based evaluation toolkit for large vision-language models (LVLMs). It allows for streamlined evaluation on various benchmarks without the heavy workload of data preparation.
Capability facts
- Languages
- python
Source: github.language · Aug 17, 2026
Categories
Graph entities
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Aug 17, 2026)
**VLMEvalKit** (the python package name is **vlmeval**) is an **open-source evaluation toolkit** of **largeSource link
Tags
README
<b A Toolkit for Evaluating Large Vision Language Models. </b [![][github contributors shield]][github contributors link] • [![][github forks shield]][github forks link] • [![][github stars shield]][github stars link] • [![][github issues shield]][github issues link] • [![][githu...
For agents
This page has a .md twin and JSON over the API.