Results for “evaluation”

LightRAG

HKUDS/LightRAG

[EMNLP2025] LightRAG: Simple and Fast Retrieval-Augmented Generation

langfuse

langfuse/langfuse

Langfuse: Open source AI engineering platform for LLM evaluation, observability, and prompt management.

llmfit

AlexsJones/llmfit

Terminal tool for right-sizing LLM models to system hardware

mlflow

mlflow/mlflow

The open source AI engineering platform for agents, LLMs, and ML models.

promptfoo

promptfoo/promptfoo

Test your prompts, agents, and RAGs. Evaluate LLM performance with red teaming/pentesting/vulnerability scanning.

adk-python

google/adk-python

A Python-based toolkit for creating and deploying flexible AI agents with advanced capabilities.

opik

comet-ml/opik

Debug, evaluate, and monitor your LLM applications with comprehensive tracing, automated evaluations, and production-ready dashboards.

cua

trycua/cua

Infrastructure for training and evaluating AI agents to control desktop environments.

Awesome-Multimodal-Large-Language-Models

BradyFU/Awesome-Multimodal-Large-Language-Models

:sparkles::sparkles:Latest Advances on Multimodal Large Language Models

WeKnora

Tencent/WeKnora

Open-source LLM knowledge platform

RagaAI-Catalyst

raga-ai-hub/RagaAI-Catalyst

Python SDK for AI Agent Observability and Evaluation

ragas

vibrantlabsai/ragas

Supercharge Your LLM Application Evaluations

easy-dataset

ConardLi/easy-dataset

A powerful tool for creating datasets for LLM fine-tuning, RAG, and evaluations.

bisheng

dataelement/bisheng

LLM DevOps platform for enterprise AI applications

phoenix

Arize-ai/phoenix

AI Observability & Evaluation

awesome-LLM-resources

WangRongsheng/awesome-LLM-resources

Summary of the world's best LLM resources.

train-llm-from-scratch

FareedKhan-dev/train-llm-from-scratch

A Python repository for training an LLM from scratch using PyTorch.

evidently

evidentlyai/evidently

Evidently: Open-source ML and LLM observability framework

agent-starter-pack

GoogleCloudPlatform/agent-starter-pack

A repository for production-ready templates to deploy AI Agents on Google Cloud.

helicone

Helicone/helicone

Open source LLM observability platform

pyspur

PySpur-Dev/pyspur

A visual playground for iterating over AI agents faster.

coze-loop

coze-dev/coze-loop

Next-generation AI Agent Optimization Platform

giskard-oss

Giskard-AI/giskard-oss

Evaluation & Testing library for LLM Agents

LLM-Engineers-Handbook

PacktPublishing/LLM-Engineers-Handbook

LLM Engineer's Handbook by Paul Iusztin and Maxime Labonne

Promptify

promptslab/Promptify

Task-based NLP engine with Pydantic structured outputs

lmms-eval

EvolvingLMMs-Lab/lmms-eval

Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

agenta

Agenta-AI/agenta

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

VLMEvalKit

open-compass/VLMEvalKit

An open-source evaluation toolkit for large vision-language models (LVLMs) supporting over 220 LMMs and 80+ benchmarks.

trulens

truera/trulens

Evaluation and Tracking for LLM Experiments and AI Agents

langwatch

langwatch/langwatch

The platform for LLM evaluations and AI agent testing

lmnr

lmnr-ai/lmnr

Laminar - open-source observability platform purpose-built for AI agents.

ChainForge

ianarawjo/ChainForge

An open-source visual environment for battle-testing prompts to LLMs.

openlit

openlit/openlit

Open source platform for AI Engineering

bootcamp

milvus-io/bootcamp

Interactive tutorials and examples for mastering Milvus with unstructured data applications.

uptrain

uptrain-ai/uptrain

Unified platform for evaluating and improving Generative AI applications.

better-agents

langwatch/better-agents

A CLI tool and set of standards to create robust coding assistants

intellagent

plurai-ai/intellagent

Framework for comprehensive diagnosis and optimization of agents using simulated, realistic synthetic interactions

langtrace

Scale3-Labs/langtrace

Langtrace 🔍 - Open Telemetry based observability for LLM applications

prometheus-eval

prometheus-eval/prometheus-eval

Evaluate your LLM's response with Prometheus and GPT4

cv-santiago

santifer/cv-santiago

Interactive CV with AI chat integration