Home/Compare/chunktuner vs qa_metrics

Comparison

chunktuner vs qa_metrics

Verdict

Unverified catalog guidance: chunktuner: Chunktuner is a Python-based tool for benchmarking and optimizing chunking strategies in Retrieval-Augmented Generation (RAG) systems. It offers a CLI, Python library, and MCP server to evaluate and optimize text-split,朱. qa_metrics: qa_metrics is a Python library for evaluating LLMs using standardized QA and semantic metrics, including support for Black-box and open-source models along with APIs from.

Markdown twin · chunktuner alternatives · qa_metrics alternatives

GraphCanon updated Oct 18, 2026

chunktuner logo

chunktuner

shantanu-deshmukh/chunktuner

★ 2pushed Jun 21, 2026
vs
qa_metrics logo

qa_metrics

zli12321/qa_metrics

★ 64pushed Jul 18, 2025

Trust & integrity

Signalchunktunerqa_metrics
Maintenance
Steady (89d since push)
As of Sep 18, 2026 · github_public_v1
Dormant (456d since push)
As of Oct 18, 2026 · github_public_v1
Provenance
Not a fork · Personal account
As of Sep 18, 2026 · github_public_v1
Not a fork · Personal account
As of Oct 18, 2026 · github_public_v1
OSV dependency advisories
No lockfile (source not queried)
As of Sep 18, 2026 · osv@v1
No lockfile (source not queried)
As of Jul 15, 2026 · osv@v1
deps.dev advisories
Not queried
deps.dev@v1
Not queried
deps.dev@v1
OpenSSF Scorecard
Not queried
openssf-scorecard@v1
Not queried
openssf-scorecard@v1

Tagline

chunktuner
Benchmark and optimize chunking strategies for RAG corpus
qa_metrics
A Python package for basic QA evaluations of large language models.

Stars

chunktuner
2
qa_metrics
64

Forks

chunktuner
0
qa_metrics
6

Open issues

chunktuner
0
qa_metrics
1

Language

chunktuner
Python
qa_metrics
Python

Adopt for

chunktuner
Unverified catalog guidance: Chunktuner is a Python-based tool for benchmarking and optimizing chunking strategies in Retrieval-Augmented Generation (RAG) systems. It offers a CLI, Python library, and MCP server to evaluate and optimize text-split,朱
qa_metrics
Unverified catalog guidance: qa_metrics is a Python library for evaluating LLMs using standardized QA and semantic metrics, including support for Black-box and open-source models along with APIs from OpenAI and Anthropic.

Persona

chunktuner
-
qa_metrics
-

Runtime

chunktuner
-
qa_metrics
-

License

chunktuner
MIT
qa_metrics
MIT License allows for free use and distribution with attribution required by retaining the copyright notice and license text in any redistribution.

Last pushed

chunktuner
Jun 21, 2026
qa_metrics
Jul 18, 2025

Categories

chunktuner
Data & Retrieval, Evaluation & Observability
qa_metrics
Evaluation & Observability

Trust and health

Maintenance

chunktuner
Steady (60%)
qa_metrics
Dormant (18%)

Days since push

chunktuner
89d
qa_metrics
456d

Open issues (recorded)

chunktuner
0
qa_metrics
1

Open issues delta

chunktuner
0 (30d)
qa_metrics
+1 (30d)

Full report

chunktuner
Trust report
qa_metrics
Trust report

Shared compatibility

  • Python · chunktuner: Python runtime · qa_metrics: Documented Python runtime (not runtime-tested)

Choose chunktuner if…

  • Tags unique to chunktuner: chunking, embedding, evaluation, langchain.
  • Also covers Data & Retrieval.
  • Unverified catalog guidance: When you need to benchmark and optimize chunking strategies specifically for Retrieval-Augmented Generation (RAG) systems.

When NOT to use chunktuner

  • Unverified catalog guidance: If your project does not involve Retrieval-Augmented Generation (RAG) systems, as Chunktuner is specialized for RAG.
  • Unverified catalog guidance: If you do not require a Python-based solution or do not wish to use a tool that provides both a CLI and a Python library.
  • Unverified catalog guidance: When you are looking for a tool that does not offer a cost estimation feature before running evaluations.

Choose qa_metrics if…

  • Tags unique to qa_metrics: exact-matching, llm-evaluation, qa-automation-test.
  • Unverified catalog guidance: When you need to evaluate the performance of large language models with built-in standardized metrics like exact match and F1 Score.
  • More GitHub stars (64 vs 2) - visibility, not fit.

When NOT to use qa_metrics

  • Unverified catalog guidance: Avoid if you seek advanced customization or fine-tuning options not present in qa_metrics for metric calculation methods beyond its provided set.
  • Unverified catalog guidance: Not ideal when needing specific evaluation tools that are not Black-box or open-source models, as the package focuses on these types of evaluations primarily.

Explore

Sources

Every stat on this page traces to a dated GitHub sync, license file, enrichment field, or trust scan.

GitHub stars on cards: chunktuner 2 · qa_metrics 64 (synced Sep 20, 2026).

Common questions

What is the difference between chunktuner and qa_metrics?
chunktuner: Benchmark and optimize chunking strategies for RAG corpus. qa_metrics: A Python package for basic QA evaluations of large language models.. See the comparison table for recorded GitHub metadata and shared categories.
When should I choose chunktuner over qa_metrics?
For chunktuner, compare these recorded points with qa_metrics: Tags unique to chunktuner: chunking, embedding, evaluation, langchain; Also covers Data & Retrieval; Unverified catalog guidance: When you need to benchmark and optimize chunking strategies specifically for Retrieval-Augmented Generation (RAG) systems.
When should I choose qa_metrics over chunktuner?
For qa_metrics, compare these recorded points with chunktuner: Tags unique to qa_metrics: exact-matching, llm-evaluation, qa-automation-test; Unverified catalog guidance: When you need to evaluate the performance of large language models with built-in standardized metrics like exact match and F1 Score; More GitHub stars (64 vs 2) - visibility, not fit.
When should I avoid chunktuner?
Unverified catalog guidance: If your project does not involve Retrieval-Augmented Generation (RAG) systems, as Chunktuner is specialized for RAG. Unverified catalog guidance: If you do not require a Python-based solution or do not wish to use a tool that provides both a CLI and a Python library. Unverified catalog guidance: When you are looking for a tool that does not offer a cost estimation feature before running evaluations.
When should I avoid qa_metrics?
Unverified catalog guidance: Avoid if you seek advanced customization or fine-tuning options not present in qa_metrics for metric calculation methods beyond its provided set. Unverified catalog guidance: Not ideal when needing specific evaluation tools that are not Black-box or open-source models, as the package focuses on these types of evaluations primarily.
Is chunktuner or qa_metrics more popular on GitHub?
qa_metrics has more recorded GitHub stars (64 vs 2). Stars measure visibility, not whether either tool fits your constraints.
Which licenses are recorded for chunktuner and qa_metrics?
chunktuner: MIT. qa_metrics: MIT. Check each repository's current license and terms before use. Public GitHub hosting alone does not establish open-source licensing.
Where can I find alternatives to chunktuner or qa_metrics?
GraphCanon lists candidates at chunktuner alternatives and qa_metrics alternatives (chunktuner markdown twin, qa_metrics markdown twin). The discovery order uses stored relationship proposals and shared categories; it is not a verified replacement ranking.
Is there a machine-readable version of this comparison?
Yes. The markdown twin at this comparison mirrors this page for agents and LLM crawlers, with the same stats table and FAQ answers.
Which is better maintained, chunktuner or qa_metrics?
chunktuner: Steady. qa_metrics: Dormant. Compare maintenance labels, days since push, and release cadence in the trust section below - stars alone do not measure maintenance.
Where are the full trust reports for chunktuner and qa_metrics?
GraphCanon publishes per-repo trust reports with dated maintenance, provenance, and scan summaries: chunktuner trust report; qa_metrics trust report.

Was this helpful?

Anonymous feedback helps us improve pages and translations.