Comparison
lmms-eval vs VLMEvalKit
A neutral, constraint-first comparison - live GitHub stats and typed relationships, not marketing.
vs
| lmms-eval | VLMEvalKit | |
|---|---|---|
| Tagline | One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks | A Toolkit for Evaluating Large Vision-Language Models |
| Stars | 4.3k | 4.3k |
| Forks | 613 | 729 |
| Open issues | 43 | 264 |
| Language | Python | Python |
| License | Other | Apache-2.0 |
| Last pushed | Jul 7, 2026 | Jul 5, 2026 |
| Categories | Evaluation & Observability | Evaluation & Observability |
| Trust and health | ||
| Maintenance | Very active (96%) | Very active (96%) |
| Days since push | 1d | 2d |
| Archived on GitHub | No | No |
| Stars delta | - | - |
| Open issues delta | - | - |
| Fork status | Not a fork | Not a fork |
| Owner type | Organization | Organization |
| Security scan | Not scanned | Not scanned |
| Full report | Trust report | Trust report |
| Typed relationship | lmms-eval successor VLMEvalKit | |
Choose lmms-eval if…
- More widely adopted - 4.3k stars vs 4.3k.
- More recently updated (last pushed Jul 7, 2026).
- Leaner open-issue backlog (43).