GraphCanon updated Oct 8, 2026 · GitHub synced Oct 8, 2026
50views this month
Decision brief
MultiPL-E is a benchmark system translating Python-based coding challenges across multiple programming languages.
Good fit when
- Use MultiPL-E for evaluating large language models' performance on code generation tasks in different languages directly without needing to create new benchmarks from scratch.
Avoid when
- Avoid using MultiPL-E if you need a more challenging benchmark; consider Ag-LiveCodeBench-X instead.
- Do not use MultiPL-E if your evaluation environment lacks GPU resources for completion generation or does not support Docker or Podman for execution of generated code.
- Pricing:
- freemium - Free to use but requires local compute resources and potentially licensed libraries
Observed Jul 17, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Slowing (178d since push)
- As of Oct 8, 2026
- Provenance
- Not a fork · Organization account
- As of Oct 8, 2026
- Security (OSV)
- No lockfile
- As of Jul 11, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install MultiPL-ESimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
MultiPL-E is a system for translating unit test-driven neural code generation benchmarks to new languages. It translates popular Python benchmarks (HumanEval and MBPP) into multiple programming languages, facilitating the evaluation of Large Language Models across different coding environments.
Capability facts
- Languages
- python
Source: github.language · Oct 8, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Oct 8, 2026)
Python benchmarks (HumanEval and MBPP) to 18 other programming languages.Source link
Tags
README
Multi Programming Language Evaluation of Large Language Models of Code (MultiPL E) New : For a more challenging multi language benchmark, check out Ag LiveCodeBench X and its accompanying paper, Agnostics. Introduction MultiPL E is a system for translating unit test driven neural...
For agents
This page has a .md twin and JSON over the API.