Comparison
deepeval vs DevEval
Verdict
Pick deepeval when tags unique to deepeval: evaluation framework, evaluation-metrics, llm-evaluation, llm-evaluation-framework; pick DevEval when leaner open-issue backlog (0).
Markdown twin · deepeval alternatives · DevEval alternatives
GraphCanon updated today
vs
Trust & integrity
| Signal | deepeval | DevEval |
|---|---|---|
| Maintenance | Very active (0d since push) As of today · github_public_v1 | Dormant (772d since push) As of today · github_public_v1 |
| Provenance | Not a fork · Organization account As of today · github_public_v1 | Not a fork · Organization account As of today · github_public_v1 |
| Security (OSV) | No lockfile As of today · none | No lockfile As of today · none |
Tagline
- deepeval
- The LLM Evaluation Framework
- DevEval
- A Comprehensive Benchmark for Software Development.
Stars
- deepeval
- 17k
- DevEval
- 134
Forks
- deepeval
- 1.6k
- DevEval
- 13
Open issues
- deepeval
- 334
- DevEval
- 0
Language
- deepeval
- Python
- DevEval
- Python
Adopt for
- deepeval
- -
- DevEval
- -
Persona
- deepeval
- -
- DevEval
- -
Runtime
- deepeval
- -
- DevEval
- -
License
- deepeval
- Apache-2.0
- DevEval
- Apache-2.0
Last pushed
- deepeval
- Jul 10, 2026
- DevEval
- May 30, 2024
Categories
- deepeval
- Evaluation & Observability, LLM Frameworks
- DevEval
- Evaluation & Observability
Trust and health
Maintenance
- deepeval
- Very active (96%)
- DevEval
- Dormant (18%)
Days since push
- deepeval
- 0d
- DevEval
- 772d
Open issues (now)
- deepeval
- 334
- DevEval
- 0
Full report
- deepeval
- Trust report
- DevEval
- Trust report
Choose deepeval if…
- Tags unique to deepeval: evaluation framework, evaluation-metrics, llm-evaluation, llm-evaluation-framework.
- Also covers LLM Frameworks.
- More GitHub stars (17k vs 134) - visibility, not fit.
When NOT to use deepeval
- Evaluation & Observability: Defer heavyweight eval infra only until you have real traffic - never skip it once users depend on answers.
- LLM Frameworks: Avoid a framework for a single prompt-and-retrieve call; the abstraction can cost more than it saves.
Choose DevEval if…
- Leaner open-issue backlog (0).
When NOT to use DevEval
- Last GitHub push was 772 days ago (dormant maintenance, May 30, 2024). Validate activity before betting a new project on DevEval.
- Evaluation & Observability: Defer heavyweight eval infra only until you have real traffic - never skip it once users depend on answers.
Explore
Sources
Every stat on this page traces to a dated GitHub sync, license file, enrichment field, or trust scan.
- GitHub stars (confident-ai/deepeval) · observed Jul 11, 2026
- GitHub forks (confident-ai/deepeval) · observed Jul 11, 2026
- Last push (confident-ai/deepeval) · observed Jul 10, 2026
- License file (Apache-2.0) · observed Jul 11, 2026
- Trust scan (lockfile / OSV) · observed Jul 11, 2026
- GitHub stars (open-compass/DevEval) · observed Jul 11, 2026
- GitHub forks (open-compass/DevEval) · observed Jul 11, 2026
- Last push (open-compass/DevEval) · observed May 30, 2024
- License file (Apache-2.0) · observed Jul 11, 2026
- Trust scan (lockfile / OSV) · observed Jul 11, 2026
GitHub stars on cards: deepeval 17k · DevEval 134 (synced Jul 11, 2026).
Common questions
- What is the difference between deepeval and DevEval?
- deepeval: The LLM Evaluation Framework. DevEval: A Comprehensive Benchmark for Software Development.. See the comparison table for live GitHub stats and shared categories.
- When should I choose deepeval over DevEval?
- Choose deepeval over DevEval when Tags unique to deepeval: evaluation framework, evaluation-metrics, llm-evaluation, llm-evaluation-framework; Also covers LLM Frameworks; More GitHub stars (17k vs 134) - visibility, not fit.
- When should I choose DevEval over deepeval?
- Choose DevEval over deepeval when Leaner open-issue backlog (0).
- When should I avoid deepeval?
- Evaluation & Observability: Defer heavyweight eval infra only until you have real traffic - never skip it once users depend on answers. LLM Frameworks: Avoid a framework for a single prompt-and-retrieve call; the abstraction can cost more than it saves.
- When should I avoid DevEval?
- Last GitHub push was 772 days ago (dormant maintenance, May 30, 2024). Validate activity before betting a new project on DevEval. Evaluation & Observability: Defer heavyweight eval infra only until you have real traffic - never skip it once users depend on answers.
- Is deepeval or DevEval more popular on GitHub?
- deepeval has more GitHub stars (16,767 vs 134). Stars measure visibility, not whether either tool fits your constraints.
- Are deepeval and DevEval open source?
- Yes - both are open-source projects on GitHub (deepeval: Apache-2.0, DevEval: Apache-2.0).
- Where can I find alternatives to deepeval or DevEval?
- GraphCanon lists graph-backed alternatives at deepeval alternatives and DevEval alternatives (deepeval markdown twin, DevEval markdown twin), ranked by typed relationship edges rather than popularity votes.
- Is there a machine-readable version of this comparison?
- Yes. The markdown twin at this comparison mirrors this page for agents and LLM crawlers, with the same stats table and FAQ answers.
- Which is better maintained, deepeval or DevEval?
- deepeval: Very active. DevEval: Dormant. Compare maintenance labels, days since push, and release cadence in the trust section below - stars alone do not measure maintenance.
- Where are the full trust reports for deepeval and DevEval?
- GraphCanon publishes per-repo trust reports with dated maintenance, provenance, and scan summaries: deepeval trust report; DevEval trust report.