GraphCanon updated 3w · GitHub synced 3w
Decision brief
AdaRubrics serves as an Adaptive Dynamic Rubric Evaluator specifically for assessing AI agent and language model performance based on evolving rubrics tailored to the agents' paths.
Good fit when
- When you need dynamic evaluation criteria that adapt in real-time according to how your AI agents or language models are performing their tasks.
- If your project involves multiple stages of AI agent development where continuous adaptation of evaluation metrics is necessary for improving performance over time.
Avoid when
- If fixed rubrics with static evaluation criteria suffice, AdaRubrics provides more complexity than needed.
- For projects that do not require real-time adjustments in evaluation methods as the AI agents' or models' trajectories progress.
Observed Jul 17, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Steady (51d since push)
- As of 3w
- Provenance
- Not a fork · Personal account
- As of 3w
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install AdaRubricsSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
AdaRubric is an evaluation tool designed to assess the performance of AI agents and language models based on dynamic rubrics that adapt according to agent trajectories.
Capability facts
- Languages
- python
Source: github.language+pyproject.toml · Jul 28, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Jul 28, 2026)
```python import asyncioSource link
Tags
README
Installation
git clone https://github.com/alphadl/AdaRubrics.git
cd AdaRubrics
pip install -e ".[dev]"
Set OPENAI_API_KEY in your environment (or pass via config). YAML config support requires pip install pyyaml.
Quick Start
import asyncio
from adarubric import AdaRubricPipeline, TaskDescription, Trajectory, TrajectoryStep
from adarubric.config import AdaRubricConfig
task = TaskDescription(
task_id="demo-001",
instruction=(
"Use the weather API to check if it will rain in Tokyo tomorrow, "
"and if so, suggest indoor activities."
),
domain="Personal Assistant",
expected_tools=["weather_api", "activity_search"],
)
trajectory = Trajectory(
trajectory_id="traj-demo-001",
task_id="demo-001",
steps=[
TrajectoryStep(
step_id=0,
thought="I need to check tomorrow's weather in Tokyo first.",
action="weather_api",
action_input={"city": "Tokyo", "date": "tomorrow"},
observation="Tomorrow: 70% chance of rain, high 18°C, low 12°C.",
),
TrajectoryStep(
step_id=1,
thought="It's likely to rain. Let me find indoor activities.",
action="activity_search",
action_input={"city": "Tokyo", "type": "indoor", "limit": 5},
observation="1. TeamLab Borderless, 2. Tokyo National Museum, 3. Akihabara arcades...",
),
],
)
pipeline = AdaRubricPipeline.from_config(AdaRubricConfig())
result = asyncio.run(pipeline.run(task, [trajectory], num_dimensions=5))
print(f"Rubric dimensions: {result.rubric.dimension_names}")
print(f"Global score: {result.mean_score:.2f}/5.0")
print(f"Survival rate: {result.survival_rate:.0%}")
Run the full example:
export OPENAI_API_KEY="sk-..."
python examples/quickstart.py
For agents
This page has a .md twin and JSON over the API.