Awesome-Code-LLM
👨💻 An awesome and curated list of best code-LLM for research.
GraphCanon updated today · GitHub synced today
1.3k
Stars
74
Forks
3
Open issues
33
Watchers
1y
Last push
MITCreated Jul 5, 2023
Trust & integrity
Full report- Maintenance
- Dormant (578d since push)
- As of today · Source: github_public_v1
- Provenance
- Not a fork · Personal account
- As of today · Source: github_public_v1
- Security (OSV)
- No lockfile
- As of today · Source: none
Public GitHub metadata and optional OSV dependency scans. Signals, not a guarantee. Trust methodology.
Overview
A curated list of the best large language models (LLMs) focused on code generation, including top models, evaluation toolkits, and relevant papers.
Capability facts
No sourced capability facts yet. Facts appear after ingest scans repo manifests (Dockerfile, package.json, MCP configs).
Categories
Tags
README
👨💻 Awesome Code LLM
🔆 How to Contribute
Contributions are welcome! If you have any resources, tools, papers, or insights related to Code LLMs, feel free to submit a pull request. Let's work together to make this project better!
News
- 🔥🔥🔥 [2024-11-12] Qwen2.5-Coder series are released, offering six model sizes (0.5B, 1.5B, 3B, 7B, 14B, 32B), with Qwen2.5-Coder-32B-Instruct now the most powerful open-source code model.
- 🔥🔥 [2024-11-08] OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models is released.
🧵 Table of Contents
- 🧵 Table of Contents
- 🚀 Top Code LLMs
- 💡 Evaluation Toolkit
- 🚀 Awesome Code LLMs Leaderboard
- 📚 Awesome Code LLMs Papers
- 🌊 Awesome Code Pre-Training Papers
- 🐳 Awesome Code Instruction-Tuning Papers
- 🐬 Awesome Code Alignment Papers
- 🐋 Awesome Code Prompting Papers
- 🐙 Awesome Code Benchmark & Evaluation Papers
- 🙌 Contributors
- Cite as
- Acknowledgement
- Star History
🚀 Top Code LLMs
Sort by HumanEval Pass@1
| Rank | Model | Params | HumanEval | MBPP | Source |
|---|---|---|---|---|---|
| 1 | o1-mini-2024-09-12 | - | 97.6 | 93.9 | paper |
| 2 | o1-preview-2024-09-12 | - | 95.1 | 93.4 | paper |
| 3 | Qwen2.5-Coder-32B-Instruct | 32B | 92.7 | 90.2 | github |
| 4 | Claude-3.5-Sonnet-20241022 | - | 92.1 | 91.0 | paper |
| 5 | GPT-4o-2024-08-06 | - | 92.1 | 86.8 | paper |
| 6 | Qwen2.5-Coder-14B-Instruct | 14B | 89.6 | 86.2 | github |
| 7 | Claude-3.5-Sonnet-20240620 | - | 89.0 | 87.6 | paper |
| 8 | GPT-4o-mini-2024-07-18 | - | 87.8 | 86.0 | [paper](https://arxi |