---
title: "trulens"
type: "tool"
slug: "truera-trulens"
canonical_url: "https://www.graphcanon.com/tools/truera-trulens"
github_url: "https://github.com/truera/trulens"
homepage_url: "https://www.trulens.org/"
stars: 3428
forks: 309
primary_language: "Python"
license: "MIT"
categories: ["ai-agents", "evaluation-observability"]
tags: ["llm-eval", "agent-evaluation", "machine-learning", "ai-observability"]
updated_at: "2026-07-07T18:42:41.589877+00:00"
---

# trulens

> Evaluation and Tracking for LLM Experiments and AI Agents

TruLens is a Python toolkit designed to help developers systematically evaluate and track experiments with Large Language Models (LLMs) and AI agents. It provides fine-grained, stack-agnostic instrumentation and comprehensive evaluations to identify failure modes, enhance understanding of application performance, and support systematic iteration for continuous improvement.

## Facts

- Repository: https://github.com/truera/trulens
- Homepage: https://www.trulens.org/
- Stars: 3,428 · Forks: 309 · Open issues: 104 · Watchers: 23
- Primary language: Python
- License: MIT
- Last pushed: 2026-06-30T15:16:55+00:00

## Categories

- [AI Agents](/categories/ai-agents.md)
- [Evaluation & Observability](/categories/evaluation-observability.md)

## Tags

llm-eval, agent-evaluation, machine-learning, ai-observability

## Related tools

- [ECC](/tools/affaan-m-ecc.md) - The agent harness performance optimization system (★ 226,962)
- [hermes-agent](/tools/nousresearch-hermes-agent.md) - The self-improving AI agent built by Nous Research (★ 210,880)
- [AutoGPT](/tools/significant-gravitas-autogpt.md) - AutoGPT: Build, Deploy, and Run AI Agents (★ 185,417)
- [ollama](/tools/ollama-ollama.md) - Get up and running with Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. (★ 175,659)
- [langflow](/tools/langflow-ai-langflow.md) - Langflow is a powerful platform for building and deploying AI-powered agents and workflows. (★ 151,298)
- [dify](/tools/langgenius-dify.md) - Production-ready platform for agentic workflow development (★ 148,070)
- [firecrawl](/tools/firecrawl-firecrawl.md) - The API to search, scrape, and interact with the web at scale. (★ 147,117)
- [langchain](/tools/langchain-ai-langchain.md) - The agent engineering platform. (★ 141,211)

## README (excerpt)

```text
# 🦑 Welcome to TruLens!



**Don't just vibe-check your LLM app!** Systematically evaluate and track your
LLM experiments with TruLens. As you develop your app including prompts, models,
retrievers, knowledge sources and more, *TruLens* is the tool you need to
understand its performance.

Fine-grained, stack-agnostic instrumentation and comprehensive evaluations help
you to identify failure modes & systematically iterate to improve your
application.

Read more about the core concepts behind TruLens including [Feedback Functions](https://www.trulens.org/getting_started/core_concepts/feedback_functions/),
[The RAG Triad](https://www.trulens.org/getting_started/core_concepts/rag_triad/),
and [Honest, Harmless and Helpful Evals](https://www.trulens.org/getting_started/core_concepts/honest_harmless_helpful_evals/).

## TruLens in the development workflow

Build your first prototype then connect instrumentation and logging with
TruLens. Decide what feedbacks you need, and specify them with TruLens to run
alongside your app. Then iterate and compare versions of your app in an
easy-to-use user interface 👇



## Installation and Setup

Install the trulens pip package from PyPI.

```bash
pip install trulens-core
```

Install with a specific LLM provider for feedback evaluation:

```bash
pip install trulens trulens-providers-openai   # OpenAI / Azure OpenAI
pip install trulens trulens-providers-litellm  # LiteLLM (Anthropic, Cohere, Mistral, …)
pip install trulens trulens-providers-google   # Google Gemini
pip install trulens trulens-providers-bedrock  # AWS Bedrock
pip install trulens trulens-providers-cortex   # Snowflake Cortex
pip install trulens trulens-providers-huggingface  # HuggingFace
pip install trulens trulens-providers-langchain    # LangChain models
```

Install with a specific app framework integration:

```bash
pip install trulens trulens-apps-langchain    # LangChain / LangGraph
pip install trulens trulens-apps-llamaindex  # LlamaIndex
```

## Quick Usage

Walk through how to instrument and evaluate a RAG built from scratch with
TruLens.



## Key Features

### 🔭 OpenTelemetry-based tracing

TruLens instrumentation is built on [OpenTelemetry](https://opentelemetry.io/).
Every function call, LLM generation, retrieval, and tool invocation is captured
as a structured OTEL span. This makes TruLens interoperable with existing
observability infrastructure — export traces to Jaeger, Grafana Tempo, Datadog,
or any OTLP-compatible backend.

```python
from trulens.core.otel.instrument import instrument
from trulens.otel.semconv.trace import SpanAttributes

class MyRAG:
    @instrument(
        span_type=SpanAttributes.SpanType.RETRIEVAL,
        attributes={
            SpanAttributes.RETRIEVAL.QUERY_TEXT: "query",
            SpanAttributes.RETRIEVAL.RETRIEVED_CONTEXTS: "return",
        },
    )
    def retrieve(self, query: str) -> list:
        ...
```

### 🤖 Agentic evaluations

Seven purpose-built evaluators for agentic systems — each measuring a distinct
aspect of agent behavior:

| Evaluator | What it measures |
|-----------|-----------------|
| LogicalConsistency | Reasoning coherence; flags hallucinations and unsupported assertions |
| ExecutionEfficiency | Redundant steps, unnecessary retries, wasted computation |
| PlanAdherence | Whether execution followed the stated plan |
| PlanQuality | Intrinsic plan quality — strategy, not outcome |
| ToolSelection | Right tool chosen for each subtask |
| ToolCalling | Argument validity and output interpretation |
| ToolQuality | External tool/service reliability |

### 📊 Batch and inline evaluation

Run evaluations alongside your app, on existing data, or in offline batch mode:

```python
# Inline — evaluate as the app runs
with tru_recorder as recording:
    response = my_app.query("What is TruLens?")

# Batch — evaluate a pre-collected dataset using the TruLens 2.8 Run API
from trulens.core.run import RunConfig

run_config = RunConfig(
    run_name="batch_eval_v1",
    da
```

---

**Machine-readable endpoints**

- JSON: [`/api/graphcanon/tools/truera-trulens`](/api/graphcanon/tools/truera-trulens)
- LLM index: [/llms.txt](/llms.txt)
- Full corpus: [/llms-full.txt](/llms-full.txt)

_GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_