baba_is_eval
Evaluates language models' meta-level reasoning through the puzzle game Baba Is You
GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026
23views this month
Decision brief
baba_is_eval leverages Baba Is You to evaluate language models' meta-level reasoning in an interactive text-based setting through MCP server integration.
Good fit when
- - When you need to test the advanced problem-solving and strategic thinking capabilities of a language model, especially for understanding or creating game rules dynamically
- - For developing or testing models that require interaction with complex, rule-manipulative games to gauge their ability to adapt to changing conditions
Avoid when
- - If your test scenario requires stability or reliability, as baba_is_eval is noted for being unstable due to its alpha status
- - When you need a tool that supports automated testing without the necessity of human supervision or manual setup of game assets and MCP server interaction
- Pricing:
- freemium - Free to use, but requires purchasing Baba Is You from a platform like Steam for testing.
Observed Jul 16, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Dormant (435d since push)
- As of Sep 9, 2026
- Provenance
- Not a fork · Personal account
- As of Sep 9, 2026
- Security (OSV)
- No lockfile
- As of Aug 2, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install baba_is_evalSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
Uses an MCP server to interact with the game in text format for language model evaluation.
Capability facts
- Languages
- python
Source: github.language · Sep 9, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Sep 9, 2026)
pip install mcp fastmcp pyautogui configparserSource link
Source: README excerpt (regex_v1, Sep 9, 2026)
On top of this MCP client agnostic setup, you can use a client like Claude Desktop to have a model interact with the server and play the game.Source link
Tags
README
Baba Is Eval https://github.com/user attachments/assets/cb6d0d49 b583 46a2 b262 1cf38279a531 We evaluate language models' meta level reasoning in an interactive setting, like the upcoming ARC AGI 3, except we use the brilliant puzzle game "Baba Is You". In short, the player manip...
For agents
This page has a .md twin and JSON over the API.