GraphCanon updated Oct 12, 2026 · GitHub synced Oct 12, 2026
43views this month
Decision brief
simple-evals provides lightweight tools for evaluating language models using reference implementations from HealthBench, BrowseComp, SimpleQA. Last updates July 2025.
Good fit when
- When you need a stable baseline to evaluate model performance with specific benchmarks like MMLU, HumanEval, and DROP that won't change after July 2025
- If your project relies on the exact benchmark results for models evaluated in the repository from June 2024 to July 2025
Avoid when
- For evaluating models released or significantly updated after July 2025, as this tool does not include future benchmarks
- When you need a tool that will adapt and expand its benchmark set with emerging model releases and evaluation tasks beyond 2025
Observed Jul 14, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Slowing (172d since push)
- As of Oct 12, 2026
- Provenance
- Not a fork · Organization account
- As of Oct 12, 2026
- Security (OSV)
- No lockfile
- As of Jul 11, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Backing
Company context for OpenAI. Display-only - separate from trust and ranking.
- Company
- OpenAI·GitHub org profile·Jul 11, 2026
- Employees
- 4,500·Wikidata (P1128 employees)·Jul 11, 2026
- Funding
- $13,000,000,000 (2023-01)·GraphCanon curated seed (public press)·Jul 11, 2026
- Commercial model
- OSS + managed cloud·GraphCanon curated seed·Jul 11, 2026
Install
pip install simple-evalsSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
This repository hosts reference implementations for benchmarking language models including HealthBench, BrowseComp, and SimpleQA. It provides tools to evaluate model performance across various tasks but will not receive updates for new models or benchmark results beyond July 2025.
Capability facts
- Languages
- python
Source: github.language · Oct 12, 2026
Categories
Graph entities
Tags
README
⚠️ Deprecation Notice July 2025 : will no longer be updated for new models or benchmark results. The repo will continue to host reference implementations for HealthBench , BrowseComp , and SimpleQA . Overview This repository contains a lightweight library for evaluating language...
For agents
This page has a .md twin and JSON over the API.