lighteval logo

lighteval

huggingface/lighteval

All-in-one toolkit for evaluating LLMs across multiple backends

GraphCanon updated today · GitHub synced today

2.5k
Stars
506
Forks
347
Open issues
26
Watchers
1w
Last push
Python MITCreated Jan 26, 2024

Trust & integrity

Full report
Maintenance
Active (11d since push)
As of today · Source: github_public_v1
Provenance
Not a fork · Organization account
As of today · Source: github_public_v1
Security (OSV)
No lockfile
As of today · Source: none

Public GitHub metadata and optional OSV dependency scans. Signals, not a guarantee. Trust methodology.

Backing

Company and funding context for Hugging Face. Display-only - not part of trust score or organic ranking.

Company
Hugging Face·GitHub org profile·today
Employees
160·Wikidata (P1128 employees)·today
Funding
$235,000,000 (2023-08)·GraphCanon curated seed (public press)·today
Commercial model
OSS + managed cloud·GraphCanon curated seed·today

Overview

Lighteval provides a comprehensive set of tools and frameworks to evaluate the performance of language models (LLMs) on different computing platforms or backend infrastructures.

Capability facts

CLI
CLI entrypoint

Source: pyproject.toml:[project.scripts] · Jul 12, 2026

Languages
python

Source: github.language+pyproject.toml · Jul 12, 2026

Categories

Graph entities

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Jul 11, 2026)

pip install lighteval
Source link

Tags

README

⚡️ Installation

Note: lighteval is currently completely untested on Windows, and we don't support it yet. (Should be fully functional on Mac/Linux)

pip install lighteval

Lighteval allows for many extras when installing, see here for a complete list.

If you want to push results to the Hugging Face Hub, add your access token as an environment variable:

hf auth login