Home/AI Agents/agentic-vbench
agentic-vbench logo

agentic-vbench

PhiloLabs/agentic-vbench

A benchmark for evaluating AI agents in performing real-world post-production tasks like audio and video editing.

GraphCanon updated Oct 18, 2026 · GitHub synced Oct 18, 2026

109 stars30 forksLast push Oct 9, 2026 Python Apache-2.0

Decision brief

Unverified catalog guidance: AgenticVBench evaluates AI agents' real-world post-production capabilities with specific task prompts for activities like audio restoration.

Good fit when

  • Unverified catalog guidance: When you need to benchmark the performance of AI agents in handling specialized tasks such as audio and video editing that require precise restorative actions.
  • Unverified catalog guidance: If your development team requires metrics on processing time, cost, and accuracy when evaluating different AI agent algorithms designed for post-production applications.

Avoid when

  • Unverified catalog guidance: When the focus is on generic performance evaluations rather than on real-world, task-specific benchmarks that assess handling complex post-production scenarios.
  • Unverified catalog guidance: If your budget or timeline cannot accommodate a per-task wall clock time of ~10 minutes and cost ranging from $0.10 to $2 based on agent token usage.
Requirements:
Requires Docker; Install via scripts provided in the repository.; Python virtual environment setup for reproducibility.

Observed Jul 16, 2026 · Source: enrich:decision_facts

Verify the decision

Maintenance and security

Full trust report
Maintenance
Active (8d since push)
As of Oct 18, 2026
Provenance
Not a fork · Organization account
As of Oct 18, 2026
Security (OSV)
No lockfile
As of Jul 15, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

Package installation is not established from the available documentation. Check the source repository for setup instructions.

git clone https://github.com/philolabs/agentic-vbench
Source repository

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

AgenticVBench evaluates AI agent capabilities to handle real-world post-production tasks such as audio restoration by measuring performance metrics, processing time, and cost. It provides specific task prompts that guide the agents through various restorative processes.

Capability facts

Docker
Docker configuration present

Source: dockerfile:tasks/agentic_vbench_assembly/agentic-vbench-assembly-task1/environment/Dockerfile · Oct 18, 2026

MCP server
Unknown; MCP server not established by this scan

Source: repo_scan · Oct 18, 2026

Languages
python, javascript

Source: github.language+tasks/agentic_vbench_understanding/melee-nouns-bowl-2025-combo-kill-causal-ledger/tools/package.json+tasks/agentic_vbench_understanding/minecraft-gameplay-ledger-s1/generator/package.json · Oct 18, 2026

Categories

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Documented Python runtime (not runtime-tested)Python

Source: README excerpt (readme_declaration_v2, Oct 18, 2026)

python3 -m venv .venv && .venv/bin/pip install --upgrade pip
Source link

Tags

README

1. Install reward.json → ≈ 1.0, 30 s on a cached image, zero agent cost bash ./avb results show rewards from the latest job cat jobs/<job name / /steps/solve/verifier/reward.json

For agents

This page has a .md twin and JSON over the API.

Was this helpful?

Anonymous feedback helps us improve pages and translations.