Home/Evaluation & Observability/circle-guard-bench
circle-guard-bench logo

circle-guard-bench

whitecircle/circle-guard-bench

AI benchmark for evaluating LLM guard systems

GraphCanon updated Oct 18, 2026 · GitHub synced Oct 18, 2026

82 stars8 forksLast push Mar 7, 2026 Python Apache-2.0

Decision brief

Unverified catalog guidance: circle-guard-bench is a Python-based AI benchmark tool for evaluating large language model guard systems under various protection scenarios.

Good fit when

  • Unverified catalog guidance: Use circle-guard-bench when you need to evaluate the effectiveness of guardrails and safeguards in your LLM environment, as it offers an unparalleled set of scenarios specific to these protections.
  • Unverified catalog guidance: Consider using this tool if you are looking for detailed insights into how different guard systems perform under various potential security breaches or misuse attempts.

Avoid when

  • Unverified catalog guidance: Avoid circle-guard-bench if your primary focus is on benchmarking the performance aspects like speed and latency of LLMs, as it specializes in evaluating protections rather than performance.
  • Unverified catalog guidance: Do not use this tool when you intend to conduct general purpose evaluations or comparisons between different LLM models that do not specifically involve security-related guard systems.

Observed Jul 16, 2026 · Source: enrich:decision_facts

Verify the decision

Maintenance and security

Full trust report
Maintenance
Slowing (224d since push)
As of Oct 18, 2026
Provenance
Not a fork · Organization account
As of Oct 18, 2026
Security (OSV)
No lockfile
As of Jul 15, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

Package installation is not established from the available documentation. Check the source repository for setup instructions.

git clone https://github.com/whitecircle/circle-guard-bench
Source repository

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

whitecircle/circle-guard-bench is the first-of-its-kind AI benchmark focused on evaluating protection capabilities of large language model (LLM) guard systems including guardrails and safeguards, featuring extensive evaluation scenarios.

Capability facts

CLI
Declared CLI entrypoint

Source: pyproject.toml:[tool.poetry.scripts] · Oct 18, 2026

MCP server
Unknown; MCP server not established by this scan

Source: repo_scan · Oct 18, 2026

Languages
python

Source: github.language+pyproject.toml · Oct 18, 2026

Categories

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Documented Python runtime (not runtime-tested)Python

Source: README excerpt (readme_declaration_v2, Oct 18, 2026)

pip install -e .
Source link

Tags

README

Installation using Poetry (basic installation) poetry install Installation with additional inference engines poetry install extras "vllm sglang transformers" Installation using pip (or uv) pip install e . Installation with additional inference engines using pip pip install e ".[v...

For agents

This page has a .md twin and JSON over the API.

Was this helpful?

Anonymous feedback helps us improve pages and translations.