GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026
59views this month
Decision brief
Key facts about instruct-eval
Good fit when
- When you need to quantitatively evaluate the performance of instruction-tuned large language models such as Alpaca and Flan-T5 on held-out tasks.
- If you require a comprehensive toolset for both benchmarking and safety assessment, including specialized benchmarks like IMPACT and Red-Eval.
Avoid when
- When primarily interested in general model evaluation without a focus on instruction-tuned LMs.
- If your primary interest lies in qualitative assessment rather than quantitative metrics.
- Requirements:
- Min 8 GB RAM; Requires Python environment setup and specific dependencies as outlined in the repository's documentation.
Observed Jul 12, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Dormant (910d since push)
- As of Sep 6, 2026
- Provenance
- Not a fork · Organization account
- As of Sep 6, 2026
- Security (OSV)
- 83 low (83 low)
- As of Jul 11, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install instruct-evalSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
A toolset for evaluating the performance of instruction-tuned large language models, including benchmarking and safety assessment.
Capability facts
- Languages
- python
Source: github.language · Sep 6, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Sep 6, 2026)
-Eval** one could jailbreak/red-team GPT-4 with a 65.1% attack success rate and ChatGPT could be jailbroken 73% of the time as measured on DangerousQA and HarmfulQA beSource link
Tags
README
:camel: 🍮 📚 InstructEval: Towards Holistic Evaluation of Instruction Tuned Large Language Models Paper Model Leaderboard <p align="center" <img src="https://raw.githubusercontent.com/declare lab/instruct eval/main/docs/logo.png" alt="" width="300" height="300" </p 🔥 If you are...
For agents
This page has a .md twin and JSON over the API.