GraphCanon updated 4w · GitHub synced 4w · 25 views this month
Decision brief
Continuous-eval is a Python framework for evaluating large language models, with emphasis on evaluation metrics and information retrieval.
Good fit when
- When developing LLM-powered applications where a continuous evaluation of model performance over time is required.
- For projects needing integration of real-time data-driven insights to improve the reliability and effectiveness of language models.
Avoid when
- If your project strictly focuses on small scale or simple applications that do not require robust evaluation metrics or information retrieval features.
- When working in environments where Python is not preferred, as continuous-eval is specifically built for Python applications.
- Pricing:
- freemium - The framework itself is open source and free to use, but enhanced or enterprise features may require additional cost.
- Requirements:
- Min 4 GB RAM
Observed Jul 14, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Dormant (544d since push)
- As of 4w
- Provenance
- Not a fork · Organization account
- As of 4w
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install continuous-evalHow it fits your stack(6)
Typed graph edges - alternatives, integrations, successors, and dependencies. Ranked by relationship type, not raw GitHub stars.
Alternative
Relationship graph
Optional deeper exploration of typed edges and category neighbours.
Similar tools
Same-category neighbours not already linked as typed edges.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
A Python-based framework for evaluating large language models with features like evaluation metrics and information retrieval.
Capability facts
- Languages
- python
Source: github.language+pyproject.toml · Jul 21, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Jul 21, 2026)
python3 -m pip install continuous-evalSource link
Tags
README
Getting Started
This code is provided as a PyPi package. To install it, run the following command:
python3 -m pip install continuous-eval
if you want to install from source:
git clone https://github.com/relari-ai/continuous-eval.git && cd continuous-eval
poetry install --all-extras
To run LLM-based metrics, the code requires at least one of the LLM API keys in .env. Take a look at the example env file .env.example.
License
This project is licensed under the Apache 2.0 - see the LICENSE file for details.
For agents
This page has a .md twin and JSON over the API.