--- title: "ChainForge" type: "tool" slug: "ianarawjo-chainforge" canonical_url: "https://www.graphcanon.com/tools/ianarawjo-chainforge" github_url: "https://github.com/ianarawjo/ChainForge" homepage_url: "https://chainforge.ai/docs" stars: 3008 forks: 257 primary_language: "TypeScript" license: "MIT" categories: ["evaluation-observability", "developer-tools"] tags: ["llmops", "llms", "evaluation", "ai", "large-language-models", "prompt-engineering"] updated_at: "2026-07-07T18:43:56.027119+00:00" --- # ChainForge > An open-source visual environment for battle-testing prompts to LLMs. ChainForge is a data flow prompt engineering environment built on ReactFlow and Flask, allowing users to analyze and evaluate LLM responses through rapid-fire testing of prompts, models, and response quality. It supports feature-rich evaluation metrics visualization across various scenarios and utilizes AI for streamlining the entire process. ## Facts - Repository: https://github.com/ianarawjo/ChainForge - Homepage: https://chainforge.ai/docs - Stars: 3,008 · Forks: 257 · Open issues: 69 · Watchers: 34 - Primary language: TypeScript - License: MIT - Last pushed: 2026-06-10T18:25:33+00:00 ## Categories - [Evaluation & Observability](/categories/evaluation-observability.md) - [Developer Tools](/categories/developer-tools.md) ## Tags llmops, llms, evaluation, ai, large language models, prompt-engineering ## Related tools - [ECC](/tools/affaan-m-ecc.md) - The agent harness performance optimization system (★ 226,962) - [AutoGPT](/tools/significant-gravitas-autogpt.md) - AutoGPT: Build, Deploy, and Run AI Agents (★ 185,417) - [prompts.chat](/tools/f-prompts-chat.md) - The world's largest open-source prompt library for AI (★ 165,019) - [JavaGuide](/tools/snailclimb-javaguide.md) - Snailclimb/JavaGuide: 面试 & 后端通用面试指南，覆盖计算机基础、数据库、分布式、高并发、系统设计与 AI 应用开发 (★ 156,863) - [langflow](/tools/langflow-ai-langflow.md) - Langflow is a powerful platform for building and deploying AI-powered agents and workflows. (★ 151,298) - [browser-use](/tools/browser-use-browser-use.md) - 🌐 Make websites accessible for AI agents. Automate tasks online with ease. (★ 103,315) - [caveman](/tools/juliusbrussee-caveman.md) - Cuts 65% of tokens in AI coding agent responses. (★ 86,150) - [llm-course](/tools/mlabonne-llm-course.md) - Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks. (★ 80,726) ## README (excerpt) ```text # ⛓️🛠️ ChainForge **An open-source visual environment for battle-testing prompts to LLMs.** banner ChainForge is a data flow prompt engineering environment for analyzing and evaluating LLM responses. It enables rapid-fire, quick-and-dirty comparison of prompts, models, and response quality that goes beyond ad-hoc chatting with individual LLMs. With ChainForge, you can: - **Query multiple LLMs at once** to test prompt ideas and variations quickly and effectively. - **Compare response quality across prompt permutations, across models, and across model settings** to choose the best prompt and model for your use case. - **Setup evaluation metrics** (scoring function) and immediately visualize results across prompts, prompt parameters, models, and model settings. - **Use AI to streamline this entire process**: Create synthetic tables and input examples with built-in genAI features, or supercharge writing evals by prompting a model to give you starter code. [Read the docs to learn more.](https://chainforge.ai/docs/) ChainForge comes with a number of example evaluation flows to give you a sense of what's possible, including 188 example flows generated from benchmarks in OpenAI evals. ChainForge is built on [ReactFlow](https://reactflow.dev) and [Flask](https://flask.palletsprojects.com/en/2.3.x/). **_For user-curated resources and learning materials, check out the [🌟Awesome ChainForge](https://github.com/loloMD/awesome_chainforge) repo!_** # Table of Contents - 👉 [Documentation](https://chainforge.ai/docs/) 📖 - [Installation](#installation) - [Example Experiments](#example-experiments) - [Share with Others](#share-with-others) - [Features](#features) (see the [docs](https://chainforge.ai/docs/nodes/) for more comprehensive info) - [Development and How to Cite](#development) # Installation You can install ChainForge locally, or try it out on the web at **https://chainforge.ai/play/**. The web version of ChainForge has a limited feature set. In a locally installed version you can load API keys automatically from environment variables, write Python code to evaluate LLM responses, or query locally-run models hosted via Ollama. To install Chainforge on your machine, make sure you have Python 3.8 or higher, then run ```bash pip install chainforge ``` Once installed, do ```bash chainforge serve ``` Open [localhost:8000](http://localhost:8000/) in a Google Chrome, Firefox, Microsoft Edge, or Brave browser. You can set your API keys by clicking the Settings icon in the top-right corner. If you prefer to not worry about this everytime you open ChainForge, we **highly recommend** that save your OpenAI, Anthropic, Google, etc API keys and/or Amazon AWS credentials to your local environment. For more details, see the [How to Install](https://chainforge.ai/docs/getting_started/). ## Run using Docker You can use our [Dockerfile](/Dockerfile) to run `ChainForge` locally using `Docker Desktop`: - Build the `Dockerfile`: ```shell docker build -t chainforge . ``` - Run the image: ```shell docker run -p 8000:8000 chainforge ``` Now you can open the browser of your choice and open `http://127.0.0.1:8000`. # Supported providers - OpenAI - Anthropic - Google Gemini - DeepSeek - HuggingFace (Inference and Endpoints) - Together.ai - [Ollama API](https://github.com/jmorganca/ollama) (locally-hosted models) - Microsoft Azure OpenAI Endpoints - [Aleph Alpha](https://docs.aleph-alpha.com/docs/introduction) - Amazon Bedrock-hosted on-demand inference, including Anthropic Claude 3 - ...and any other provider through [custom provider scripts](https://chainforge.ai/docs/custom_providers/)! # Example experiments We've prepared many example flows to give you a sense of what's possible with Chainforge. Click the "Example Flows" button on the top-right corner and select one. Here is a basic comparison ``` --- **Machine-readable endpoints** - JSON: [`/api/graphcanon/tools/ianarawjo-chainforge`](/api/graphcanon/tools/ianarawjo-chainforge) - LLM index: [/llms.txt](/llms.txt) - Full corpus: [/llms-full.txt](/llms-full.txt) _GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_