1. Install

git clone https://github.com/PhiloLabs/agentic-vbench.git
cd agentic-vbench
./scripts/install-harbor.sh
python3 -m venv .venv && .venv/bin/pip install --upgrade pip

reward.json → ≈ 1.0, ~30 s on a cached image, zero agent cost

**Time + cost budget.** A real-agent rollout on Modal typically takes ~10 min per task wall clock. Cost depends entirely on agent + model — order-of-magnitude $0.10–$2 per task with mid-tier models, scaling linearly with the agent's token use. Plan accordingly for a 100-task sweep.

**Here's what a task prompt actually looks like** (`exp-codec-restore-task01`):

> # Restore A Muffled Stretch Of Audio
>
> I have a short mono speech recording at `/workspace/materials/noisy.wav`. For a stretch in it, the audio sounds muffled — like the high end has been chopped off and the voice lost its sparkle. The rest of the recording sounds clean and full.
>
> Please restore the muffled stretch so it sounds as clear and full as the rest of the recording. Leave the already-clean parts unchanged.
>
> ## What to deliver
> - `/workspace/output/enhanced.wav` — 16-bit PCM mono at 16 kHz, same total length (sample count) as the input.

Each task ships its own such brief at `tasks/<family>/<task>/steps/solve/instruction.md`.

**Inspect the result.** Each trial drops four artifacts under `jobs/<job-name>/<trial-id>/`:

| File | What it is |
|---|---|
| `steps/solve/verifier/reward.json` | Final score + per-metric breakdown. |
| `agent/trajectory.json` | Full event stream Harbor captured for the agent (tool calls, tool results, model messages, final output). |
| `result.json` | Per-trial Harbor summary (timings, exit codes, exception info). |
| `trial.log` | Combined stdout/stderr stream for the whole trial. |

```bash
./avb results show          # rewards from the latest job
cat jobs/<job-name>/*/steps/solve/verifier/reward.json