--- title: "Daft vs lmms-eval" type: "comparison" canonical_url: "https://www.graphcanon.com/compare/eventual-inc-daft-vs-evolvinglmms-lab-lmms-eval" tools: ["eventual-inc-daft", "evolvinglmms-lab-lmms-eval"] --- # Daft vs lmms-eval *GraphCanon updated Jul 12, 2026* ## Verdict Pick Daft when daft is primarily Rust; lmms-eval is Python; pick lmms-eval when lmms-eval is primarily Python; Daft is Rust. [Daft](https://daft.ai) reports 5.6k GitHub stars, 516 forks, and 346 open issues, last pushed Jul 10, 2026. [lmms-eval](https://www.lmms-lab.com) has 4.3k stars, 616 forks, and 44 open issues, last pushed Jul 7, 2026. Figures are from public GitHub metadata via [Daft's repository](https://github.com/Eventual-Inc/Daft) and [lmms-eval's repository](https://github.com/EvolvingLMMs-Lab/lmms-eval). | | [Daft](/tools/eventual-inc-daft.md) | [lmms-eval](/tools/evolvinglmms-lab-lmms-eval.md) | | --- | --- | --- | | Tagline | High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale | One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks | | Stars | 5,620 | 4,298 | | Forks | 516 | 616 | | Open issues | 346 | 44 | | Language | Rust | Python | | Adopt for | - | lmms-eval is a comprehensive multimodal evaluation toolkit for large language models, enabling reproduction of LLaVA-1.5 results and supporting various datasets including text, images, videos, and audio tasks. | | Persona | - | - | | Runtime | - | - | | License | Apache-2.0 | Other | | Categories | Vector Databases, Speech & Audio, Computer Vision | LLM Frameworks, Speech & Audio, Computer Vision | ## Trust and health _Sourced signals - not a safety guarantee. No winner column._ | | [Daft](/tools/eventual-inc-daft.md) | [lmms-eval](/tools/evolvinglmms-lab-lmms-eval.md) | | --- | --- | --- | | Days since push | 0d | 4d | | Open issues (now) | 346 | 44 | | Full report | [trust report](/tools/eventual-inc-daft/trust.md) | [trust report](/tools/evolvinglmms-lab-lmms-eval/trust.md) | ## Decision facts: lmms-eval - **Requirements:** Min 4 GB RAM; Requires Python 3.12 or higher for installation.; Java 8 is required when testing datasets like COCO, RefCOCO, and NoCaps due to dependency on pycocoeval API. - **Adopt for:** lmms-eval is a comprehensive multimodal evaluation toolkit for large language models, enabling reproduction of LLaVA-1.5 results and supporting various datasets including text, images, videos, and audio tasks. ## Choose when ### Choose Daft if… - Daft is primarily Rust; lmms-eval is Python. - License: Daft is Apache-2.0, lmms-eval is Other. - Tags unique to Daft: big-data, ai-engineering, distributed, arrow. - Also covers Vector Databases. ### Choose lmms-eval if… - lmms-eval is primarily Python; Daft is Rust. - License: lmms-eval is Other, Daft is Apache-2.0. - Requirements: Min 4 GB RAM; Requires Python 3.12 or higher for installation.; Java 8 is required when testing datasets like COCO, RefCOCO, and NoCaps due to dependency on pycocoeval API.. - Tags unique to lmms-eval: evaluation, benchmark, large-language-models, multimodal-evaluation. - Also covers LLM Frameworks. - When you need to evaluate the performance of multimodal large language models across diverse benchmarks including text, image, video, and audio. ## When NOT to use Daft - Vector Databases: Don't reach for a dedicated vector DB under ~100k vectors; pgvector on your existing Postgres is simpler to operate. ## When NOT to use lmms-eval - If your project does not involve multimodal large language models or you are only interested in unimodal datasets. - When you need a simpler toolkit that focuses solely on text-based evaluations, as lmms-eval's extensive capabilities might be unnecessary and introduce complexity. ## Common questions ### What is the difference between Daft and lmms-eval? Daft: High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale. lmms-eval: One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks. See the comparison table for live GitHub stats and shared categories. ### When should I choose Daft over lmms-eval? Choose Daft over lmms-eval when Daft is primarily Rust; lmms-eval is Python; License: Daft is Apache-2.0, lmms-eval is Other; Tags unique to Daft: big-data, ai-engineering, distributed, arrow; Also covers Vector Databases. ### When should I choose lmms-eval over Daft? Choose lmms-eval over Daft when lmms-eval is primarily Python; Daft is Rust; License: lmms-eval is Other, Daft is Apache-2.0; Requirements: Min 4 GB RAM; Requires Python 3.12 or higher for installation.; Java 8 is required when testing datasets like COCO, RefCOCO, and NoCaps due to dependency on pycocoeval API.; Tags unique to lmms-eval: evaluation, benchmark, large-language-models, multimodal-evaluation; Also covers LLM Frameworks; When you need to evaluate the performance of multimodal large language models across diverse benchmarks including text, image, video, and audio. ### When should I avoid Daft? Vector Databases: Don't reach for a dedicated vector DB under ~100k vectors; pgvector on your existing Postgres is simpler to operate. ### When should I avoid lmms-eval? If your project does not involve multimodal large language models or you are only interested in unimodal datasets. When you need a simpler toolkit that focuses solely on text-based evaluations, as lmms-eval's extensive capabilities might be unnecessary and introduce complexity. ### Is Daft or lmms-eval more popular on GitHub? Daft has more GitHub stars (5,620 vs 4,298). Stars measure visibility, not whether either tool fits your constraints. ### Are Daft and lmms-eval open source? Yes - both are open-source projects on GitHub (Daft: Apache-2.0, lmms-eval: Other). ### Where can I find alternatives to Daft or lmms-eval? GraphCanon lists graph-backed alternatives at [Daft alternatives](/tools/eventual-inc-daft/alternatives) and [lmms-eval alternatives](/tools/evolvinglmms-lab-lmms-eval/alternatives) ([Daft markdown twin](/tools/eventual-inc-daft/alternatives.md), [lmms-eval markdown twin](/tools/evolvinglmms-lab-lmms-eval/alternatives.md)), ranked by typed relationship edges rather than popularity votes. ### Is there a machine-readable version of this comparison? Yes. The markdown twin at [this comparison](/compare/eventual-inc-daft-vs-evolvinglmms-lab-lmms-eval.md) mirrors this page for agents and LLM crawlers, with the same stats table and FAQ answers. ### Which is better maintained, Daft or lmms-eval? Daft: Very active. lmms-eval: Very active. Compare maintenance labels, days since push, and release cadence in the trust section below - stars alone do not measure maintenance. ### Where are the full trust reports for Daft and lmms-eval? GraphCanon publishes per-repo trust reports with dated maintenance, provenance, and scan summaries: [Daft trust report](/tools/eventual-inc-daft/trust); [lmms-eval trust report](/tools/evolvinglmms-lab-lmms-eval/trust). --- **Machine-readable endpoints** - JSON: [`/api/graphcanon/graph?tool=eventual-inc-daft`](/api/graphcanon/graph?tool=eventual-inc-daft) - LLM index: [/llms.txt](/llms.txt) - Full corpus: [/llms-full.txt](/llms-full.txt) _GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_