---
title: "cceval vs human-eval"
type: "comparison"
canonical_url: "https://www.graphcanon.com/compare/amazon-science-cceval-vs-openai-human-eval"
tools: ["amazon-science-cceval", "openai-human-eval"]
---

# cceval vs human-eval

*GraphCanon updated Aug 5, 2026*

## Verdict

Pick cceval if cceval is designed for assessing cross-file code completion capabilities in multilingual environments across different setups and retrieval methods; pick human-eval if human-eval is a tool designed for evaluating large language models trained specifically on code through human-written tests.

[cceval](https://crosscodeeval.github.io/) reports 182 GitHub stars, 28 forks, and 5 open issues, last pushed Aug 15, 2025. [human-eval](https://github.com/openai/human-eval) has 3.3k stars, 452 forks, and 44 open issues, last pushed Jan 17, 2025. Figures are from public GitHub metadata via [cceval's repository](https://github.com/amazon-science/cceval) and [human-eval's repository](https://github.com/openai/human-eval).

| | [cceval](/tools/amazon-science-cceval.md) | [human-eval](/tools/openai-human-eval.md) |
| --- | --- | --- |
| Tagline | CrossCodeEval Benchmark for Cross-File Code Completion | Evaluating Large Language Models Trained on Code |
| Stars | 182 | 3,331 |
| Forks | 28 | 452 |
| Open issues | 5 | 44 |
| Language | Python | Python |
| Adopt for | cceval is designed for assessing cross-file code completion capabilities in multilingual environments across different setups and retrieval methods. | human-eval is a tool designed for evaluating large language models trained specifically on code through human-written tests. |
| Persona | - | - |
| Runtime | - | - |
| License | Apache-2.0 | MIT |
| Categories | Evaluation & Observability | Evaluation & Observability |

## Trust and health

_Sourced signals - not a safety guarantee. No winner column._

| | [cceval](/tools/amazon-science-cceval.md) | [human-eval](/tools/openai-human-eval.md) |
| --- | --- | --- |
| Maintenance | Slowing (36%) | Dormant (18%) |
| Days since push | 354d | 564d |
| Open issues (now) | 5 | 44 |
| Full report | [trust report](/tools/amazon-science-cceval/trust.md) | [trust report](/tools/openai-human-eval/trust.md) |

## Shared compatibility

- **Python**: [cceval](/tools/amazon-science-cceval.md) - Python runtime; [human-eval](/tools/openai-human-eval.md) - Python runtime

## Decision facts: cceval

- **Adopt for:** cceval is designed for assessing cross-file code completion capabilities in multilingual environments across different setups and retrieval methods.

## Decision facts: human-eval

- **Hosting:** self hosted - This evaluation framework must be installed and set up in your own environment, ensuring full control over the testing process.
- **Pricing:** freemium - The software is available under an MIT license for free use, yet advanced features or services beyond its core functionality might incur costs.
- **Adopt for:** human-eval is a tool designed for evaluating large language models trained specifically on code through human-written tests.

## Choose when

### Choose cceval if…

- License: cceval is Apache-2.0, human-eval is MIT.
- Tags unique to cceval: benchmark, code-completion, cross-file, evaluation-tool.
- You need to evaluate the performance of cross-file code completion systems that can handle multiple programming languages simultaneously

### Choose human-eval if…

- License: human-eval is MIT, cceval is Apache-2.0.
- This evaluation framework must be installed and set up in your own environment, ensuring full control over the testing process.
- Pricing: The software is available under an MIT license for free use, yet advanced features or services beyond its core functionality might incur costs..
- Tags unique to human-eval: code evaluation, large language models, python.
- When you need to evaluate the performance of AI systems that have been trained exclusively on code datasets, as it allows testing via human-created benchmarks relevant only to code-based models.

## When NOT to use cceval

- Your evaluation needs are focused solely on single-file or intralingual code completion benchmarks
- You require real-time data access or live updates; cceval provides pre-packaged datasets that need to be manually obtained and uncompressed

## When NOT to use human-eval

- If you are interested in evaluating general natural language processing tasks without coding context, as human-eval is tailored specifically for assessing code-focused AI systems.
- When the required Python version is below 3.7; this tool mandates at least Python 3.7 to ensure compatibility with its dependencies.

## Common questions

### What is the difference between cceval and human-eval?

cceval: CrossCodeEval Benchmark for Cross-File Code Completion. human-eval: Evaluating Large Language Models Trained on Code. See the comparison table for live GitHub stats and shared categories.

### When should I choose cceval over human-eval?

Choose cceval over human-eval when License: cceval is Apache-2.0, human-eval is MIT; Tags unique to cceval: benchmark, code-completion, cross-file, evaluation-tool; You need to evaluate the performance of cross-file code completion systems that can handle multiple programming languages simultaneously.

### When should I choose human-eval over cceval?

Choose human-eval over cceval when License: human-eval is MIT, cceval is Apache-2.0; This evaluation framework must be installed and set up in your own environment, ensuring full control over the testing process; Pricing: The software is available under an MIT license for free use, yet advanced features or services beyond its core functionality might incur costs.; Tags unique to human-eval: code evaluation, large language models, python; When you need to evaluate the performance of AI systems that have been trained exclusively on code datasets, as it allows testing via human-created benchmarks relevant only to code-based models.

### When should I avoid cceval?

Your evaluation needs are focused solely on single-file or intralingual code completion benchmarks You require real-time data access or live updates; cceval provides pre-packaged datasets that need to be manually obtained and uncompressed

### When should I avoid human-eval?

If you are interested in evaluating general natural language processing tasks without coding context, as human-eval is tailored specifically for assessing code-focused AI systems. When the required Python version is below 3.7; this tool mandates at least Python 3.7 to ensure compatibility with its dependencies.

### Is cceval or human-eval more popular on GitHub?

human-eval has more GitHub stars (3,331 vs 182). Stars measure visibility, not whether either tool fits your constraints.

### Are cceval and human-eval open source?

Yes - both are open-source projects on GitHub (cceval: Apache-2.0, human-eval: MIT).

### Where can I find alternatives to cceval or human-eval?

GraphCanon lists graph-backed alternatives at [cceval alternatives](/tools/amazon-science-cceval/alternatives) and [human-eval alternatives](/tools/openai-human-eval/alternatives) ([cceval markdown twin](/tools/amazon-science-cceval/alternatives.md), [human-eval markdown twin](/tools/openai-human-eval/alternatives.md)), ranked by typed relationship edges rather than popularity votes.

### Is there a machine-readable version of this comparison?

Yes. The markdown twin at [this comparison](/compare/amazon-science-cceval-vs-openai-human-eval.md) mirrors this page for agents and LLM crawlers, with the same stats table and FAQ answers.

### Which is better maintained, cceval or human-eval?

cceval: Slowing. human-eval: Dormant. Compare maintenance labels, days since push, and release cadence in the trust section below - stars alone do not measure maintenance.

### Where are the full trust reports for cceval and human-eval?

GraphCanon publishes per-repo trust reports with dated maintenance, provenance, and scan summaries: [cceval trust report](/tools/amazon-science-cceval/trust); [human-eval trust report](/tools/openai-human-eval/trust).

---

**Machine-readable endpoints**

- JSON: [`/api/graphcanon/graph?tool=amazon-science-cceval`](/api/graphcanon/graph?tool=amazon-science-cceval)
- LLM index: [/llms.txt](/llms.txt)
- Full corpus: [/llms-full.txt](/llms-full.txt)

_GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_