Home/Evaluation & Observability/litmus

litmus

google/litmus

A comprehensive LLM testing and evaluation tool for GenAI application development with user-friendly UI

GitHub Website

GraphCanon updated Sep 10, 2026 · GitHub synced Sep 10, 2026

30views this month

52 stars9 forksLast push Mar 29, 2026 Vue Apache-2.0

Decision brief

Litmus is an extensive GenAI application development platform for testing and evaluating LLM performance, featuring a user-friendly UI and deployment flexibility via CLI or manual setup.

Good fit when

When deploying on Google Cloud with the need for CLI ease of use
For comprehensive evaluation methods like Custom LLM Evaluation, Ragas, DeepEval

Avoid when

In environments not using Google Cloud services and APIs
When manual setup for each component outweighs potential benefits of automation provided by Litmus CLI

Observed Jul 16, 2026 · Source: enrich:decision_facts

litmus alternatives Trust report

Verify the decision

Maintenance and security

Full trust report

Maintenance: Slowing (164d since push); As of Sep 10, 2026
Provenance: Not a fork · Organization account; As of Sep 10, 2026
Security (OSV): No lockfile; As of Jul 15, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Backing

Company context for Google. Display-only - separate from trust and ranking.

Company: Google·GitHub org profile·Jul 11, 2026
Employees: 47,756·Wikidata (P1128 employees)·Jul 11, 2026
Commercial model: Pure OSS·GitHub org profile (public repos)·Jul 11, 2026

Install

git clone https://github.com/google/litmus

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

langfuse

Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

35k★

evals

Framework for evaluating LLMs and LLM systems with an open-source registry of benchmarks.

19k★

ragas

Supercharge Your LLM Application Evaluations 🚀

15k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

Litmus is an all-inclusive platform aimed at developing and evaluating the performance of AI applications powered by large language models.

Capability facts

Languages: vue
Source: github.language · Sep 10, 2026

Graph entities

Googlecompany

README

Getting Started 1. Quick Deployment with the Litmus CLI: This is the easiest way to set up Litmus. Make sure you have the Google Cloud SDK installed and configured with the correct project. Install the Litmus CLI: Linux : install: binary: https://storage.googleapis.com/litmus clo...

For agents

This page has a .md twin and JSON over the API.

/tools/google-litmus.md GET /api/graphcanon/tools/google-litmus GET /api/graphcanon/graph?tool=google-litmus