GraphCanon updated 3w · GitHub synced 3w
Decision brief
ClawBench offers an open-source benchmark framework for evaluating browser-based AI agents on real-world online tasks.
Good fit when
- You are developing a browser AI agent and wish to measure its performance against everyday online activities, as ClawBench specifically simulates these scenarios.
- Your project involves the use of Chrome extensions in conjunction with LLMS, requiring benchmarking that aligns with this specific integration.
Avoid when
- If your focus is solely on backend server-based AI evaluations without a browser interface involvement, ClawBench will not be the appropriate choice.
- For those developing standalone applications or mobile agents, ClawBench’s browser-centric tasks will not reflect their operational capabilities accurately.
- Requirements:
- Requires setup for browser automation tasks within the Chrome environment.
Observed Jul 16, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Very active (0d since push)
- As of 3w
- Provenance
- Not a fork · Organization account
- As of 3w
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install ClawBenchSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
ClawBench is an open-source benchmark framework tailored to evaluate browser-based AI agents in handling everyday online tasks.
Capability facts
- CLI
- CLI entrypoint
Source: pyproject.toml:[project.scripts] · Jul 28, 2026
- Languages
- python
Source: github.language+pyproject.toml · Jul 28, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Jul 28, 2026)
You can also use `pipx install clawbench-eval` or `python -m pip install clawbench-eval`.Source link
Source: README excerpt (regex_v1, Jul 28, 2026)
Point your coding agent (Claude Code, Cursor, Copilot, etc.) at [`AGENTS.md`](AGENTS.md) and prompt away.Source link
Tags
README
LLM Quick Start
Point your coding agent (Claude Code, Cursor, Copilot, etc.) at AGENTS.md and prompt away.
Human Quick Start
Install ClawBench from PyPI for normal use:
uv tool install clawbench-eval
You can also use pipx install clawbench-eval or python -m pip install clawbench-eval.
The installed commands are still clawbench, clawbench-run,
clawbench-batch, and clawbench-harbor-adapt.
For those want more granular control and contribution, clone the repo and run the root uv package entrypoint:
git clone https://github.com/reacher-z/ClawBench.git && cd ClawBench && ./run.sh
Prerequisites: Python 3.11+, uv, and a container engine — Docker or Podman. ClawBench auto-detects whichever is installed; force one with export CONTAINER_ENGINE=docker or export CONTAINER_ENGINE=podman.
Install Docker or Podman (macOS / Linux / Windows)
macOS
---
# Option A — Docker Desktop (easiest, includes GUI)
brew install --cask docker
open -a Docker # launch and wait for the whale icon to settle
---
# Option B — Docker
sudo apt install -y docker.io
sudo usermod -aG docker $USER # log out / back in so your shell picks up the group
Rootful Docker ownership note: with classic
sudo-docker, files extracted from containers land owned byrooton the host. ClawBench's driver detects this after each run and chownstest-output/back to your user automatically — but if you run other container tooling alongside, rootless Podman (or rootless Docker) avoids the issue entirely.
Windows
---
# Option A — Docker Desktop (WSL2 backend)
winget install Docker.DockerDesktop
---
# then launch Docker Desktop from the Start menu and wait for it to be ready
---
## License & Acknowledgments
Apache 2.0 -- see [LICENSE](LICENSE).
The converted Claw-Eval suite in [`test-cases/claw-eval/`](test-cases/claw-eval/) is derived from [claw-eval/claw-eval](https://github.com/claw-eval/claw-eval) and the [claw-eval/Claw-Eval](https://huggingface.co/datasets/claw-eval/Claw-Eval) dataset, which are released under the MIT License. Third-party package notices are in [NOTICE](NOTICE).
Built with [OpenClaw](https://github.com/openclaw/openclaw), [opencode](https://opencode.ai), [Claude Code](https://docs.anthropic.com/en/docs/claude-code), the [Claude in Chrome](https://code.claude.com/docs/en/chrome) extension, [OpenAI Codex CLI](https://github.com/openai/codex), [browser-use](https://github.com/browser-use/browser-use), [claw-code](https://github.com/ultraworkers/claw-code), [Hermes Agent](https://github.com/NousResearch/hermes-agent), and [Pi](https://pi.dev/) with [pi-browser-harness](https://pi.dev/packages/pi-browser-harness) (selectable harnesses), [Microsoft Playwright MCP](https://github.com/microsoft/playwright-mcp) (browser control bridge for the opencode, claude-code, codex, and claw-code harnesses), [LiteLLM](https://github.com/BerriAI/litellm) (API translation proxy for the claude-code, claude-code-chrome-extension, codex, browser-use, claw-code, and pi harnesses), [noVNC](https://github.com/novnc/noVNC) (MPL 2.0), and [websockify](https://github.com/novnc/websockify) (LGPL 3.0).
For agents
This page has a .md twin and JSON over the API.