LiveCodeBench
Holistic and contamination-free evaluation of large language models for code
GraphCanon updated 2w · GitHub synced 2w · 38 views this month
Decision brief
LiveCodeBench offers an in-depth approach to evaluating large language models specifically for code tasks such as generation and repair.
Good fit when
- When you need a holistic method to assess the effectiveness of LLMs in code tasks without risking contamination by earlier outputs or data leakage.
- If you are working with Python-based projects and require precise and unbiased evaluations leveraging advanced features like GPT-4.
Avoid when
- For broad, non-code-specific model assessments where a more generalized evaluation tool would suffice.
- If your project is not compatible with Python 3.11 or if you do not want to use the uv dependency manager recommended by LiveCodeBench.
Observed Jul 17, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Dormant (385d since push)
- As of 2w
- Provenance
- Not a fork · Organization account
- As of 2w
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install LiveCodeBenchSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
LiveCodeBench evaluates the capabilities of large language models in generating, repairing, and executing code with a focus on maintaining high standards to prevent model contamination.
Capability facts
- Languages
- python
Source: github.language+pyproject.toml · Aug 5, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Tags
README
Installation
You can clone the repository using the following command:
git clone https://github.com/LiveCodeBench/LiveCodeBench.git
cd LiveCodeBench
We recommend using uv for managing dependencies, which can be installed a number of ways.
Verify that uv is installed on your system by running:
uv --version
Once uv has been installed, use it to create a virtual environment for
LiveCodeBench and install its dependencies with the following commands:
uv venv --python 3.11
source .venv/bin/activate
uv pip install -e .
For agents
This page has a .md twin and JSON over the API.