GraphCanon updated 2w · GitHub synced 2w
Decision brief
cceval is designed for assessing cross-file code completion capabilities in multilingual environments across different setups and retrieval methods.
Good fit when
- You need to evaluate the performance of cross-file code completion systems that can handle multiple programming languages simultaneously
- Your project requires a benchmark with various settings including baseline, retrieval with references, and pure retrieval approaches
Avoid when
- Your evaluation needs are focused solely on single-file or intralingual code completion benchmarks
- You require real-time data access or live updates; cceval provides pre-packaged datasets that need to be manually obtained and uncompressed
Observed Jul 17, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Slowing (354d since push)
- As of 2w
- Provenance
- Not a fork · Organization account
- As of 2w
- Security (OSV)
- 5 critical, 7 high, 8 medium, 21 low (5 critical, 7 high, 8 medium, 21 low)
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install ccevalSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
A diverse and multilingual benchmark aimed at evaluating cross-file code completion across various settings and retrievers.
Capability facts
- Languages
- python
Source: github.language · Aug 5, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Aug 5, 2026)
- Install dependencies via `pip install -r requirements.txt`Source link
Tags
README
Requirements
- Uncompress the CrossCodeEval data via
tar -xvJf data/crosscodeeval_data.tar.xz -C data/- The data contains {baseline, retrieval, retrieval w/ ref.} setting x {bm25, UniXCoder, OpenAI Ada} retriever.
- Please email us if you need the raw data.
- Install dependencies via
pip install -r requirements.txt - Build tree sitter via
bash scripts/build_treesitter.sh
License
This project is licensed under the Apache-2.0 License.
For agents
This page has a .md twin and JSON over the API.