{"data":{"slug":"ruc-datalab-coda-bench","name":"CoDA-Bench","tagline":"Benchmark for code agents on data-intensive tasks","github_url":"https://github.com/ruc-datalab/CoDA-Bench","owner":"ruc-datalab","repo":"CoDA-Bench","owner_avatar_url":"https://avatars.githubusercontent.com/u/76154266?v=4","primary_language":"Python","stars":48,"forks":1,"topics":["agent","agentic","agentic-ai","ai","benchmark","code-agent","code-generation","coding-agent","data-analysis","data-engineering","data-intensive","data-science","evaluation","icml","kaggle","llm","llm-agent","llm-evaluation","sandbox"],"archived":false,"github_pushed_at":"2026-09-29T04:55:40+00:00","maintenance_label":"Active","stars_delta_30d":3,"url":"https://www.graphcanon.com/tools/ruc-datalab-coda-bench","markdown_url":"https://www.graphcanon.com/tools/ruc-datalab-coda-bench.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/ruc-datalab-coda-bench","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=ruc-datalab-coda-bench","description":"CoDA-Bench is a benchmark for code agents on data-intensive tasks. 🎈代码智能体能搞定数据密集型任务吗? ","homepage_url":"https://coda-bench.github.io/","license":"MIT","open_issues":2,"watchers":1,"ai_summary":"CoDA-Bench is designed to evaluate the performance of AI-powered code agents in handling complex data-related tasks.","readme_excerpt":"## 🚀 Quick Start\n\n<img src=\"./assets/overview.png\" style=\"height: 20em\">\n\n---\n\n### Installation\n\n```bash\ngit clone https://github.com/ruc-datalab/CoDA-Bench.git\ncd CoDA-Bench\npip install -e .\n```\n\n---\n\n### Run Evaluation (Docker Mode)\n\n**Step 1: Build Docker Image**\n```bash\ncd docker\n./build_all.sh\ncd ..\n```\n\n**Step 2: Set API Credentials**\n```bash\nexport LLM_API_KEY=\"your-api-key\"\nexport LLM_BASE_URL=\"https://api.openai.com/v1\"  # Optional\n```\n\n**Step 3: Run Evaluation**\n```bash\n\n---\n\n### Why Docker?\n\nDocker mode provides **secure isolation**:\n- ✅ Agents cannot access benchmark answers\n- ✅ Network-restricted environment (only LLM API accessible)\n- ✅ Resource limits (memory, CPU, timeout)\n- ✅ Reproducible across different machines\n\nSee [docker/README.md](docker/README.md) for detailed Docker documentation.\n\n---\n\n## 📄 License\n\nMIT License. See [LICENSE](LICENSE) for details.\n\nIndividual Kaggle datasets may have their own licenses.","github_created_at":"2026-05-29T04:39:31+00:00","created_at":"2026-07-15T10:39:43.675896+00:00","updated_at":"2026-10-18T06:00:32.406303+00:00","categories":[{"slug":"ai-agents","name":"AI Agents","url":"https://www.graphcanon.com/categories/ai-agents","markdown_url":"https://www.graphcanon.com/categories/ai-agents.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/ai-agents"},{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"}],"tags":[{"slug":"agent","name":"agent"},{"slug":"agentic-ai","name":"agentic-ai"},{"slug":"benchmark","name":"benchmark"},{"slug":"code-agent","name":"code-agent"},{"slug":"data-engineering","name":"data-engineering"},{"slug":"icml","name":"icml"},{"slug":"llm-evaluation","name":"llm-evaluation"}],"trust":{"provenance":{"is_fork":false,"github_id":1253011729,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-10-18T06:00:31.163Z","maintenance":{"label":"Active","score":82,"methodology":"github_public_v1","releases_90d":0,"days_since_push":19,"last_release_at":null,"stars_delta_30d":3,"open_issues_delta_30d":-1},"security_summary":{"last_scan_at":"2026-07-15T10:39:46.06395+00:00","scan_profile":"deps","status":"no_lockfile","critical_count":0,"high_count":0,"medium_count":0,"low_count":0,"scanner":"osv@v1","scan_id":"b8339de9-3d41-4fb8-8743-e70e2c102703","pending_review_count":0}},"capability_facts":{"mcp":{"method":"github-tree-and-contents@2","reason":"No explicit MCP server declaration was found in the inspected manifests. Other layouts and implementations were not tested.","source":"repo_scan","status":"unknown","revision":"23208c599d298e8912bfde9c88d3e4f4a4fa771b","source_url":"https://github.com/ruc-datalab/CoDA-Bench/tree/23208c599d298e8912bfde9c88d3e4f4a4fa771b","observed_at":"2026-10-18T06:00:31.502Z","server_manifest":null},"scan":{"method":"github-tree-and-contents@2","source":"repo_scan","status":"complete","version":2,"coverage":{"scope":"bounded-repository-manifests","tree_complete":true,"selected_files":2,"candidate_files":2},"revision":"23208c599d298e8912bfde9c88d3e4f4a4fa771b","source_url":"https://github.com/ruc-datalab/CoDA-Bench/tree/23208c599d298e8912bfde9c88d3e4f4a4fa771b","observed_at":"2026-10-18T06:00:31.502Z","observations":[{"path":"@revision","bytes":4855,"status":"present","revision":"23208c599d298e8912bfde9c88d3e4f4a4fa771b","source_url":"https://github.com/ruc-datalab/CoDA-Bench/commit/23208c599d298e8912bfde9c88d3e4f4a4fa771b","http_status":200,"observed_at":"2026-10-18T06:00:31.687Z"},{"path":"@tree","bytes":14558,"status":"present","revision":"23208c599d298e8912bfde9c88d3e4f4a4fa771b","source_url":"https://github.com/ruc-datalab/CoDA-Bench/tree/23208c599d298e8912bfde9c88d3e4f4a4fa771b","http_status":200,"observed_at":"2026-10-18T06:00:31.826Z"},{"path":"docker-compose.yml","reason":"not-listed-in-complete-tree","status":"not-found","revision":"23208c599d298e8912bfde9c88d3e4f4a4fa771b","source_url":"https://github.com/ruc-datalab/CoDA-Bench/tree/23208c599d298e8912bfde9c88d3e4f4a4fa771b","observed_at":"2026-10-18T06:00:31.826Z"},{"path":"docker/openhands/Dockerfile","bytes":927,"status":"present","revision":"23208c599d298e8912bfde9c88d3e4f4a4fa771b","source_url":"https://github.com/ruc-datalab/CoDA-Bench/blob/23208c599d298e8912bfde9c88d3e4f4a4fa771b/docker/openhands/Dockerfile","http_status":200,"observed_at":"2026-10-18T06:00:31.972Z"},{"path":"Dockerfile","reason":"not-listed-in-complete-tree","status":"not-found","revision":"23208c599d298e8912bfde9c88d3e4f4a4fa771b","source_url":"https://github.com/ruc-datalab/CoDA-Bench/tree/23208c599d298e8912bfde9c88d3e4f4a4fa771b","observed_at":"2026-10-18T06:00:31.826Z"},{"path":"mcp.json","reason":"not-listed-in-complete-tree","status":"not-found","revision":"23208c599d298e8912bfde9c88d3e4f4a4fa771b","source_url":"https://github.com/ruc-datalab/CoDA-Bench/tree/23208c599d298e8912bfde9c88d3e4f4a4fa771b","observed_at":"2026-10-18T06:00:31.826Z"},{"path":"package.json","reason":"not-listed-in-complete-tree","status":"not-found","revision":"23208c599d298e8912bfde9c88d3e4f4a4fa771b","source_url":"https://github.com/ruc-datalab/CoDA-Bench/tree/23208c599d298e8912bfde9c88d3e4f4a4fa771b","observed_at":"2026-10-18T06:00:31.826Z"},{"path":"pyproject.toml","bytes":2058,"status":"present","revision":"23208c599d298e8912bfde9c88d3e4f4a4fa771b","source_url":"https://github.com/ruc-datalab/CoDA-Bench/blob/23208c599d298e8912bfde9c88d3e4f4a4fa771b/pyproject.toml","http_status":200,"observed_at":"2026-10-18T06:00:31.983Z"},{"path":"server.json","reason":"not-listed-in-complete-tree","status":"not-found","revision":"23208c599d298e8912bfde9c88d3e4f4a4fa771b","source_url":"https://github.com/ruc-datalab/CoDA-Bench/tree/23208c599d298e8912bfde9c88d3e4f4a4fa771b","observed_at":"2026-10-18T06:00:31.826Z"}]},"has_cli":{"value":true,"method":"repository-declaration@2","source":"pyproject.toml:[project.scripts]","status":"declared","revision":"23208c599d298e8912bfde9c88d3e4f4a4fa771b","source_url":"https://github.com/ruc-datalab/CoDA-Bench/blob/23208c599d298e8912bfde9c88d3e4f4a4fa771b/pyproject.toml","observed_at":"2026-10-18T06:00:31.502Z"},"languages":{"value":["python"],"method":"github-tree-and-contents@2","source":"github.language+pyproject.toml","revision":"23208c599d298e8912bfde9c88d3e4f4a4fa771b","source_url":"https://github.com/ruc-datalab/CoDA-Bench/tree/23208c599d298e8912bfde9c88d3e4f4a4fa771b","observed_at":"2026-10-18T06:00:31.502Z"},"has_docker":{"value":true,"method":"repository-declaration@2","source":"dockerfile:docker/openhands/Dockerfile","revision":"23208c599d298e8912bfde9c88d3e4f4a4fa771b","source_url":"https://github.com/ruc-datalab/CoDA-Bench/blob/23208c599d298e8912bfde9c88d3e4f4a4fa771b/docker/openhands/Dockerfile","observed_at":"2026-10-18T06:00:31.502Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-10-18T06:00:31.502Z"}},"decision_facts":{"hosting":null,"pricing":{"model":"freemium","summary":"Available under the MIT License, free to use but may require additional costs for API credentials if using external services like OpenAI's APIs."},"requirements":{"min_ram_gb":null,"requires_docker":true},"constraints":{"min_ram_gb":null,"pricing_model":"freemium","requires_docker":true},"when_to_use":["When you aim to evaluate the reliability and security of an AI-powered code agent with strict control over network access and data isolation.","For benchmarking purposes specifically involving complex, data-related tasks where reproducibility across different machines is crucial."],"when_not_to_use":["If your project does not require Docker-level secure isolation or if the overhead of setting up a Docker environment is prohibitive for your workflow.","When you are dealing with less complex data tasks that do not demand stringent security measures such as restricted network environments and resource limits."],"source":"enrich:decision_facts","observed_at":"2026-07-16T18:46:04.010Z"},"constraint_facets":{"min_ram_gb":null,"pricing_model":"freemium","requires_docker":true},"decision_summary":[{"label":"Pricing","value":"freemium - Available under the MIT License, free to use but may require additional costs for API credentials if using external services like OpenAI's APIs."},{"label":"Requirements","value":"Requires Docker"},{"label":"Adopt for","value":"Unverified catalog guidance: CoDA-Bench provides secure isolation for evaluating code agents in data-intensive tasks via Docker-mode execution."}]}}