agent-learning-kit
Evaluation Framework for all your AI related Workflows
GraphCanon updated 3w · GitHub synced 3w
Decision brief
Agent-learning-kit is a Python framework for evaluating AI-related workflows with modules for faithfulness assessment, embedding similarity analysis, and feedback loop integration via ChromaDB.
Good fit when
- When you need comprehensive evaluation of your AI models including faithfulness checks using DeBERTa NLI model installed.
- For projects requiring embedding similarity assessments with sentence-transformers module option.
Avoid when
- If your workflow does not align with the specific evaluation criteria and methods supported by agent-learning-kit.
- When you seek a framework that integrates with backend systems other than those provided as optional extras, such as MongoDB or DynamoDB instead of ChromaDB.
Observed Jul 14, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Very active (0d since push)
- As of 3w
- Provenance
- Not a fork · Organization account
- As of 3w
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install agent-learning-kitSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
A Python-based evaluation framework designed to assess various aspects of AI and ML workflows, including faithfulness, hallucination, embedding similarity, feedback loop integration using ChromaDB.
Capability facts
- Languages
- python
Source: github.language · Aug 1, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Aug 1, 2026)
**Requirements:** Python 3.10+Source link
Tags
README
Installation
pip install ai-evaluation
Optional extras:
pip install ai-evaluation[nli] # DeBERTa NLI model for faithfulness/hallucination
pip install ai-evaluation[embeddings] # sentence-transformers for embedding similarity
pip install ai-evaluation[feedback] # ChromaDB for feedback loop
pip install ai-evaluation[celery] # Celery distributed backend
pip install ai-evaluation[ray] # Ray distributed backend
pip install ai-evaluation[temporal] # Temporal distributed backend
pip install ai-evaluation[all] # Everything
Requirements: Python 3.10+
Quick Start
from fi.evals import evaluate
For agents
This page has a .md twin and JSON over the API.