LiveCodeBench
Holistic and contamination-free evaluation of large language models for code
GraphCanon updated Oct 8, 2026 · GitHub synced Oct 8, 2026
56views this month
Decision brief
LiveCodeBench offers an in-depth approach to evaluating large language models specifically for code tasks such as generation and repair.
Good fit when
- When you need a holistic method to assess the effectiveness of LLMs in code tasks without risking contamination by earlier outputs or data leakage.
- If you are working with Python-based projects and require precise and unbiased evaluations leveraging advanced features like GPT-4.
Avoid when
- For broad, non-code-specific model assessments where a more generalized evaluation tool would suffice.
- If your project is not compatible with Python 3.11 or if you do not want to use the uv dependency manager recommended by LiveCodeBench.
Observed Jul 17, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Dormant (449d since push)
- As of Oct 8, 2026
- Provenance
- Not a fork · Organization account
- As of Oct 8, 2026
- Security (OSV)
- No lockfile
- As of Jul 11, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install LiveCodeBenchSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
LiveCodeBench evaluates the capabilities of large language models in generating, repairing, and executing code with a focus on maintaining high standards to prevent model contamination.
Capability facts
- Languages
- python
Source: github.language+pyproject.toml · Oct 8, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Tags
README
Installation You can clone the repository using the following command: We recommend using uv for managing dependencies, which can be installed a number of ways. Verify that is installed on your system by running: Once has been installed, use it to create a virtual environment for...
For agents
This page has a .md twin and JSON over the API.