helm

stanford-crfm/helm

Holistic, reproducible and transparent evaluation of foundation models

GitHub Website

GraphCanon updated today · GitHub synced today

2.9k

Stars

400

Forks

Open issues

Watchers

Last push

Python Apache-2.0Created Nov 29, 2021

Trust & integrity

Full report

Maintenance: Active (10d since push); As of today · Source: github_public_v1
Provenance: Not a fork · Organization account; As of today · Source: github_public_v1
Security (OSV): No lockfile; As of today · Source: none

Public GitHub metadata and optional OSV dependency scans. Signals, not a guarantee. Trust methodology.

Overview

An open-source Python framework for evaluating foundation models, including LLMs and multimodal models.

Capability facts

CLI: CLI entrypoint
Source: pyproject.toml:[project.scripts] · Jul 12, 2026
Languages: python
Source: github.language+pyproject.toml · Jul 12, 2026

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Jul 11, 2026)

pip install crfm-helm

Source link

README

Quick Start

Install the package from PyPI:

pip install crfm-helm

Run the following in your shell:

Explore

helm alternatives →Trust report →Evaluation & Observability category →All comparisons Stack workflows Trending tools

Trust & health

MaintenanceActive

last release 2mo · 2 releases (90d)

AdoptionEstablished

Graph connectivityIsolated

SecurityNo lockfile

Provenance

Organization account (GitHub heuristic)
Computed today

Full trust report Trust report (JSON)

Methodology: public GitHub data (github_public_v1). Signals, not a guarantee. Trust methodology.

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

llm-course

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

81k★

netdata

The fastest path to AI-powered full stack observability, even for lean teams.

80k★

scikit-learn

scikit-learn: machine learning in Python

67k★

TrendRadar

AI-driven public opinion & trend monitor with multi-platform aggregation, RSS, and smart alerts.

60k★

headroom

Compress tool outputs and data to reduce tokens before reaching the LLM.

58k★

FastChat

An open platform for training, serving, and evaluating large language models

39k★

jax

Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more

36k★

prompt-optimizer

An AI prompt optimizer for writing better prompts and getting better AI results.

32k★

langfuse

Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playground, datasets

31k★

gitleaks

Find secrets with Gitleaks 🔑

28k★

mlflow

AI engineering platform for debugging, evaluating, monitoring, and optimizing AI applications

27k★

mastra

Modern TypeScript framework for AI-powered applications and agents

26k★

Install

pip install helm

PyPI

For agents

This page has a .md twin and JSON over the API.

/tools/stanford-crfm-helm.md GET /api/graphcanon/tools/stanford-crfm-helm GET /api/graphcanon/graph?tool=stanford-crfm-helm