{"data":{"slug":"philolabs-agentic-vbench","name":"agentic-vbench","tagline":"AgenticVBench: Can AI Agents Complete Real-World Post-Production Tasks?","github_url":"https://github.com/PhiloLabs/agentic-vbench","owner":"PhiloLabs","repo":"agentic-vbench","owner_avatar_url":"https://avatars.githubusercontent.com/u/260782224?v=4","primary_language":"Python","stars":70,"forks":10,"topics":["ai-agents","benchmark","harbor","llm-evaluation","video-editing"],"archived":false,"github_pushed_at":"2026-07-07T07:00:10+00:00","maintenance_label":"Active","url":"https://www.graphcanon.com/tools/philolabs-agentic-vbench","markdown_url":"https://www.graphcanon.com/tools/philolabs-agentic-vbench.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/philolabs-agentic-vbench","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=philolabs-agentic-vbench","description":"AgenticVBench: Can AI Agents Complete Real-World Post-Production Tasks?","homepage_url":"https://agenticvbench.com/","license":"Apache-2.0","open_issues":15,"watchers":0,"ai_summary":null,"readme_excerpt":"### 1. Install\n\n```bash\ngit clone https://github.com/PhiloLabs/agentic-vbench.git\ncd agentic-vbench\n./scripts/install-harbor.sh\npython3 -m venv .venv && .venv/bin/pip install --upgrade pip\n```\n\n---\n\n# reward.json → ≈ 1.0, ~30 s on a cached image, zero agent cost\n```\n\n**Time + cost budget.** A real-agent rollout on Modal typically takes ~10 min per task wall clock. Cost depends entirely on agent + model — order-of-magnitude $0.10–$2 per task with mid-tier models, scaling linearly with the agent's token use. Plan accordingly for a 100-task sweep.\n\n**Here's what a task prompt actually looks like** (`exp-codec-restore-task01`):\n\n> # Restore A Muffled Stretch Of Audio\n>\n> I have a short mono speech recording at `/workspace/materials/noisy.wav`. For a stretch in it, the audio sounds muffled — like the high end has been chopped off and the voice lost its sparkle. The rest of the recording sounds clean and full.\n>\n> Please restore the muffled stretch so it sounds as clear and full as the rest of the recording. Leave the already-clean parts unchanged.\n>\n> ## What to deliver\n> - `/workspace/output/enhanced.wav` — 16-bit PCM mono at 16 kHz, same total length (sample count) as the input.\n\nEach task ships its own such brief at `tasks/<family>/<task>/steps/solve/instruction.md`.\n\n**Inspect the result.** Each trial drops four artifacts under `jobs/<job-name>/<trial-id>/`:\n\n| File | What it is |\n|---|---|\n| `steps/solve/verifier/reward.json` | Final score + per-metric breakdown. |\n| `agent/trajectory.json` | Full event stream Harbor captured for the agent (tool calls, tool results, model messages, final output). |\n| `result.json` | Per-trial Harbor summary (timings, exit codes, exception info). |\n| `trial.log` | Combined stdout/stderr stream for the whole trial. |\n\n```bash\n./avb results show          # rewards from the latest job\ncat jobs/<job-name>/*/steps/solve/verifier/reward.json","github_created_at":"2026-05-11T20:13:06+00:00","created_at":"2026-07-15T10:40:15.845621+00:00","updated_at":"2026-07-15T10:40:18.594589+00:00","categories":[{"slug":"ai-agents","name":"AI Agents","url":"https://www.graphcanon.com/categories/ai-agents","markdown_url":"https://www.graphcanon.com/categories/ai-agents.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/ai-agents"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"speech-audio","name":"Speech & Audio","url":"https://www.graphcanon.com/categories/speech-audio","markdown_url":"https://www.graphcanon.com/categories/speech-audio.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/speech-audio"}],"tags":[{"slug":"ai-agents","name":"ai-agents"},{"slug":"benchmark","name":"benchmark"},{"slug":"harbor","name":"harbor"},{"slug":"llm-evaluation","name":"llm-evaluation"},{"slug":"python","name":"python"},{"slug":"video-editing","name":"video-editing"}],"trust":{"provenance":{"is_fork":false,"github_id":1235951137,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-15T10:40:16.772Z","maintenance":{"label":"Active","score":82,"methodology":"github_public_v1","releases_90d":1,"days_since_push":8,"last_release_at":"2026-05-23T08:17:37Z"},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-15T10:40:17.191Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-15T10:40:16.550Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-07-15T10:40:16.550Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-07-15T10:40:16.550Z"}}}}