cupel

tolitius/cupel

discovery tool for evaluating LLM performance

GitHub Website

GraphCanon updated Sep 10, 2026 · GitHub synced Sep 10, 2026

26views this month

64 stars0 forksLast push Aug 31, 2026 Python Apache-2.0

Decision brief

Cupel is a JavaScript-based toolkit for discovering and evaluating the performance of large language models using configurable prompts, scoring mechanisms, multi-turn dialogues, and local inference server discovery.

Good fit when

When aiming to evaluate LLMs on local servers due to its auto-discovery feature for known ports of inference servers
For generating initial test scenarios where an LLM can draft prompts and rubrics based on input descriptions

Avoid when

If you require a solution that supports a non-JavaScript runtime environment, as Cupel is JavaScript-exclusive
When you need a tool without UI capabilities since Cupel's UI is bundled in the package and may not suit headless operations

Observed Jul 17, 2026 · Source: enrich:decision_facts

cupel alternatives Trust report

Verify the decision

Maintenance and security

Full trust report

Maintenance: Active (10d since push); As of Sep 10, 2026
Provenance: Not a fork · Personal account; As of Sep 10, 2026
Security (OSV): No lockfile; As of Jul 15, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

pip install cupel

PyPI

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

llm-course

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

83k★

litellm

The fastest, lightest AI Gateway

59k★

gorilla

Training and Evaluating LLMs for Function Calls (Tool Calls)

13k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

Cupel is a JavaScript-based toolkit for discovering and evaluating the capabilities of various LLMs through configurable prompt generation, scoring mechanisms, multi-turn dialogues, and local inference server discovery.

Capability facts

CLI: CLI entrypoint
Source: pyproject.toml:[project.scripts] · Sep 10, 2026
Languages: python
Source: github.language+pyproject.toml · Sep 10, 2026

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Sep 10, 2026)

pip install cupel

Source link

README

install or the UI is bundled in the package quick start opens a browser at <img src="doc/first time.png" alt="first time cupel is started" width="76%" ships with example data (8 models scored by Claude Opus 4.6 on 8 prompts) — the dashboard is populated on first launch LLM assist...

For agents

This page has a .md twin and JSON over the API.

/tools/tolitius-cupel.md GET /api/graphcanon/tools/tolitius-cupel GET /api/graphcanon/graph?tool=tolitius-cupel