---
title: "RagaAI-Catalyst"
type: "tool"
slug: "raga-ai-hub-ragaai-catalyst"
canonical_url: "https://www.graphcanon.com/tools/raga-ai-hub-ragaai-catalyst"
github_url: "https://github.com/raga-ai-hub/RagaAI-Catalyst"
homepage_url: "https://catalyst.raga.ai/"
stars: 16145
forks: 3579
primary_language: "Python"
license: "Apache-2.0"
categories: ["evaluation-observability"]
tags: ["llm-tracing", "agents", "synthetic-data-generation", "trace-management", "agentic-ai", "evaluation-management", "prompt-management"]
updated_at: "2026-07-07T18:29:11.534552+00:00"
---

# RagaAI-Catalyst

> Python SDK for AI Agent Observability and Evaluation

RagaAI Catalyst is a Python SDK designed to enhance the observation, monitoring, and evaluation of large language model projects. It provides tools for project management, dataset handling, trace analysis, prompt engineering, synthetic data generation, guardrail creation, and more.

## Facts

- Repository: https://github.com/raga-ai-hub/RagaAI-Catalyst
- Homepage: https://catalyst.raga.ai/
- Stars: 16,145 · Forks: 3,579 · Open issues: 34 · Watchers: 37
- Primary language: Python
- License: Apache-2.0
- Last pushed: 2026-02-11T14:43:33+00:00

## Categories

- [Evaluation & Observability](/categories/evaluation-observability.md)

## Tags

llm-tracing, agents, synthetic-data-generation, trace-management, agentic-ai, evaluation-management, prompt-management

## Related tools

- [langflow](/tools/langflow-ai-langflow.md) - Langflow is a powerful platform for building and deploying AI-powered agents and workflows. (★ 151,298)
- [graphrag](/tools/microsoft-graphrag.md) - A modular graph-based Retrieval-Augmented Generation (RAG) system (★ 34,239)
- [langfuse](/tools/langfuse-langfuse.md) - Langfuse: Open source AI engineering platform for LLM evaluation, observability, and prompt management. (★ 30,626)
- [RAG_Techniques](/tools/nirdiamant-rag-techniques.md) - Advanced RAG Techniques (★ 28,398)
- [mlflow](/tools/mlflow-mlflow.md) - The open source AI engineering platform for agents, LLMs, and ML models. (★ 26,921)
- [llm-action](/tools/liguodongiot-llm-action.md) - 分享大模型技术原理和实战经验，涵盖工程化、应用落地 (★ 24,670)
- [awesome-n8n-templates](/tools/enescingoz-awesome-n8n-templates.md) - The largest open-source collection of n8n automation templates on GitHub. (★ 23,714)
- [promptfoo](/tools/promptfoo-promptfoo.md) - Test your prompts, agents, and RAGs. Evaluate LLM performance with red teaming/pentesting/vulnerability scanning. (★ 23,003)

## README (excerpt)

```text
# RagaAI Catalyst      

RagaAI Catalyst is a comprehensive platform designed to enhance the management and optimization of LLM projects. It offers a wide range of features, including project management, dataset management, evaluation management, trace management, prompt management, synthetic data generation, and guardrail management. These functionalities enable you to efficiently evaluate, and safeguard your LLM applications.

## Table of Contents

- [RagaAI Catalyst](#ragaai-catalyst)
  - [Installation](#installation)
  - [Configuration](#configuration)
  - [Usage](#usage)
    - [Project Management](#project-management)
    - [Dataset Management](#dataset-management)
    - [Evaluation Management](#evaluation)
    - [Trace Management](#trace-management)
    - [Agentic Tracing](#agentic-tracing)
    - [Prompt Management](#prompt-management)
    - [Synthetic Data Generation](#synthetic-data-generation)
    - [Guardrail Management](#guardrail-management)
    - [Red-teaming](#red-teaming)

## Installation

To install RagaAI Catalyst, you can use pip:

```bash
pip install ragaai-catalyst
```

## Configuration

Before using RagaAI Catalyst, you need to set up your credentials. You can do this by setting environment variables or passing them directly to the `RagaAICatalyst` class:

```python
from ragaai_catalyst import RagaAICatalyst

catalyst = RagaAICatalyst(
    access_key="YOUR_ACCESS_KEY",
    secret_key="YOUR_SECRET_KEY",
    base_url="BASE_URL"
)
```
you'll need to generate authentication credentials:

1. Navigate to your profile settings
2. Select "Authenticate" 
3. Click "Generate New Key" to create your access and secret keys



**Note**: Authetication to RagaAICatalyst is necessary to perform any operations below.


## Usage

### Project Management

Create and manage projects using RagaAI Catalyst:

```python
# Create a project
project = catalyst.create_project(
    project_name="Test-RAG-App-1",
    usecase="Chatbot"
)

# Get project usecases
catalyst.project_use_cases()

# List projects
projects = catalyst.list_projects()
print(projects)
```


### Dataset Management
Manage datasets efficiently for your projects:

```py
from ragaai_catalyst import Dataset

# Initialize Dataset management for a specific project
dataset_manager = Dataset(project_name="project_name")

# List existing datasets
datasets = dataset_manager.list_datasets()
print("Existing Datasets:", datasets)

# Create a dataset from CSV
dataset_manager.create_from_csv(
    csv_path='path/to/your.csv',
    dataset_name='MyDataset',
    schema_mapping={'column1': 'schema_element1', 'column2': 'schema_element2'}
)

# Get project schema mapping
dataset_manager.get_schema_mapping()

```


For more detailed information on Dataset Management, including CSV schema handling and advanced usage, please refer to the [Dataset Management documentation](docs/dataset_management.md).


### Evaluation

Create and manage metric evaluation of your RAG application:

```python
from ragaai_catalyst import Evaluation

# Create an experiment
evaluation = Evaluation(
    project_name="Test-RAG-App-1",
    dataset_name="MyDataset",
)

# Get list of available metrics
evaluation.list_metrics()

# Add metrics to the experiment
schema_mapping={
    'Query': 'prompt',
    'response': 'response',
    'Context': 'context',
    'expectedResponse': 'expected_response'
}

# Add single metric
evaluation.add_metrics(
    metrics=[
      {"name": "Faithfulness", "config": {"model": "gpt-4o-mini", "provider": "openai", "threshold": {"gte": 0.232323}}, "column_name": "Faithfulness_v1", "schema_mapping": schema_mapping},
    
    ]
)

# Add multiple metrics
evaluation.add_metrics(
    metrics=[
        {"name": "Faithfulness", "config": {"model": "gpt-4o-mini", "provider": "openai", "threshold": {"gte": 0.323}}, "column_name": "Faithfulness_gte", "schema_mapping": schema_mapping},
        {"name": "Hallucination", "config": {"model": "gpt-4o-mini", "provider": "openai", "threshold": {"lte": 0.323
```

---

**Machine-readable endpoints**

- JSON: [`/api/graphcanon/tools/raga-ai-hub-ragaai-catalyst`](/api/graphcanon/tools/raga-ai-hub-ragaai-catalyst)
- LLM index: [/llms.txt](/llms.txt)
- Full corpus: [/llms-full.txt](/llms-full.txt)

_GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_