---
title: "lmms-eval vs VLMEvalKit"
type: "comparison"
canonical_url: "https://www.graphcanon.com/compare/evolvinglmms-lab-lmms-eval-vs-open-compass-vlmevalkit"
tools: ["evolvinglmms-lab-lmms-eval", "open-compass-vlmevalkit"]
---

# lmms-eval vs VLMEvalKit

Neutral, constraint-first comparison with live GitHub stats.

| | [lmms-eval](/tools/evolvinglmms-lab-lmms-eval.md) | [VLMEvalKit](/tools/open-compass-vlmevalkit.md) |
| --- | --- | --- |
| Tagline | One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks | A Toolkit for Evaluating Large Vision-Language Models |
| Stars | 4,292 | 4,267 |
| Forks | 613 | 729 |
| Open issues | 43 | 264 |
| Language | Python | Python |
| License | Other | Apache-2.0 |
| Categories | Evaluation & Observability | Evaluation & Observability |

## Trust and health

_Sourced signals - not a safety guarantee. No winner column._

| | [lmms-eval](/tools/evolvinglmms-lab-lmms-eval.md) | [VLMEvalKit](/tools/open-compass-vlmevalkit.md) |
| --- | --- | --- |
| Maintenance | Very active (96%) | Very active (96%) |
| Days since push | 1d | 2d |
| Archived on GitHub | No | No |
| Stars delta | - | - |
| Open issues delta | - | - |
| Fork status | Not a fork | Not a fork |
| Owner type | Organization | Organization |
| Security scan | Not scanned | Not scanned |
| Full report | [trust report](/tools/evolvinglmms-lab-lmms-eval/trust.md) | [trust report](/tools/open-compass-vlmevalkit/trust.md) |

**Typed relationship:** lmms-eval _(successor)_ VLMEvalKit

---

**Machine-readable endpoints**

- JSON: [`/api/graphcanon/graph?tool=evolvinglmms-lab-lmms-eval`](/api/graphcanon/graph?tool=evolvinglmms-lab-lmms-eval)
- LLM index: [/llms.txt](/llms.txt)
- Full corpus: [/llms-full.txt](/llms-full.txt)

_GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_