{"data":{"slug":"onyx-dot-app-enterpriserag-bench","name":"EnterpriseRAG-Bench","tagline":"Dataset and benchmark for RAG on company internal documents","github_url":"https://github.com/onyx-dot-app/EnterpriseRAG-Bench","owner":"onyx-dot-app","repo":"EnterpriseRAG-Bench","owner_avatar_url":"https://avatars.githubusercontent.com/u/131946000?v=4","primary_language":null,"stars":489,"forks":52,"topics":["benchmark","dataset","enterprise","enterprise-search","evaluation","generative-ai","information-retrieval","knowledge-base","large-language-models","llm","llm-evaluation","python","question-answering","rag","rag-evaluation","retrieval-augmented-generation","semantic-search","synthetic-data","synthetic-data-generation"],"archived":false,"github_pushed_at":"2026-05-08T02:57:41+00:00","maintenance_label":"Steady","url":"https://www.graphcanon.com/tools/onyx-dot-app-enterpriserag-bench","markdown_url":"https://www.graphcanon.com/tools/onyx-dot-app-enterpriserag-bench.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/onyx-dot-app-enterpriserag-bench","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=onyx-dot-app-enterpriserag-bench","description":"Dataset and benchmark for RAG on company internal documents.","homepage_url":"https://www.onyx.app/","license":"MIT","open_issues":9,"watchers":7,"ai_summary":"Provides test data and evaluation metrics for retrieval-augmented generation models specifically tailored towards processing enterprise-level documentation.","readme_excerpt":"

EnterpriseRAG-Bench

\n\n
\n\"Leaderboard\"\n\"Dataset\"\n\"license\"\n\"website\"\n\"arXiv\"\n
\n\n\n## 📖 Overview\n\nEnterpriseRAG Bench provides a dataset for company internal documents consisting of slightly over **500,000 documents** and **500 questions**.\nWe release this dataset to help teams benchmark their RAG systems and fine-tune agents on realistic enterprise data.\nThe code additionally provides a way of generating similar datasets for different industries, scales of companies, etc.\n\n> [!NOTE]\n> **See here for the latest [leaderboard rankings](https://huggingface.co/spaces/onyx-dot-app/EnterpriseRAG-Bench-Leaderboard).**\n\nThe dataset simulates a company called \"Redwood Inference\" that provides AI model inference as a service.\n\nThe data sources and approximate volumes are as follows:\n\n| # | Source Type | Approximate Volume | Source Description |\n|---|-------------|-------------------|----------------------|\n| 1 | Slack | 275,000 | Internal channels and team discussions. |\n| 2 | Gmail | 120,000 | Email threads from management, sales, leadership, and ICs. |\n| 3 | Linear | 35,000 | Project management tickets for eng, product, and design. |\n| 4 | Google Drive | 25,000 | Shared files and collaborative documents. |\n| 5 | Hubspot | 15,000 | CRM records for sales. |\n| 6 | Fireflies | 10,000 | Meeting transcripts for both internal and external facing meetings. |\n| 7 | GitHub | 8,000 | Pull requests and comments across different code repositories. |\n| 8 | Jira | 6,000 | Support tickets, both internal and customer facing. |\n| 9 | Confluence | 5,000 | Wikis, runbooks, and structured/polished documentation. |\n\nThe corpus provides broad coverage across key areas of business activity including (but not limited to) planning, engineering, documentation, sales, customer success, internal discussions, emails, and more.\n\nThe [questions](questions.jsonl) are broken up into 10 categories:\n\n| # | Category | Count | Description |\n|---|----------|-------|-------------|\n| 1 | Basic | 175 | Simple questions with a single ground truth document. |\n| 2 | Semantic | 125 | Similar to Basic but more roundabout, with less keyword/phrase overlap and without giveaway-type keywords. |\n| 3 | Intra-Document Reasoning | 40 | Requires combining information from several distant sections of a single longer document. |\n| 4 | Project Related | 40 | Aggregate knowledge from related documents from a single project/initiative within the company. |\n| 5 | Constrained | 30 | Questions with multiple relevant documents but with qualifiers that disqualify all but one answer. |\n| 6 | Conflicting Info | 20 | Questions where documents directly contradict each other. Requires the system to give a complete and correct answer. |\n| 7 | Completeness | 20 | Questions that require fetching all relevant documents (no more than 10) to be able to provide a correct answer. |\n| 8 | Miscellaneous | 20 | Questions targeting informal, off-topic, or loosely organized documents. |\n| 9 | High Level | 10 | High level questions where the answer is not located in any single document.
Note: these do not have any ground truth documents. |\n| 10 | Info Not Found | 20 | Questions where the answer is not available.
Note: these do not have any ground truth documents. |\n\n> No","github_created_at":"2026-02-19T02:48:25+00:00","created_at":"2026-07-11T12:00:05.46498+00:00","updated_at":"2026-07-28T18:00:12.409755+00:00","categories":[{"slug":"data-retrieval","name":"Data & Retrieval","url":"https://www.graphcanon.com/categories/data-retrieval","markdown_url":"https://www.graphcanon.com/categories/data-retrieval.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/data-retrieval"},{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"}],"tags":[{"slug":"benchmark","name":"benchmark"},{"slug":"dataset","name":"dataset"},{"slug":"enterprise-search","name":"enterprise-search"},{"slug":"evaluation","name":"evaluation"},{"slug":"information-retrieval","name":"information-retrieval"},{"slug":"knowledge-base","name":"knowledge-base"},{"slug":"large-language-models","name":"large language models"},{"slug":"llm-evaluation","name":"llm-evaluation"}],"trust":{"provenance":{"is_fork":false,"github_id":1161363612,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-28T18:00:11.606Z","maintenance":{"label":"Steady","score":60,"methodology":"github_public_v1","releases_90d":0,"days_since_push":81,"last_release_at":"2026-03-29T08:17:18Z"},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T12:00:06.794Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-28T18:00:12.103Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-07-28T18:00:12.103Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-07-28T18:00:12.103Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["When you need to evaluate retrieval-augmented generation models specifically for processing extensive and complex enterprise documentation","If your project includes the creation of custom synthetic data tailored for enterprise scenarios"],"when_not_to_use":["Avoid if your focus is on general web or public-domain document benchmarking, as EnterpriseRAG-Bench is tuned exclusively for company internal documents","Do not use if you require a solution that supports languages other than those implied by the existing dataset without further customization"],"source":"enrich:decision_facts","observed_at":"2026-07-17T00:08:13.059Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"EnterpriseRAG-Bench specializes in benchmarking RAG models on company internal documents, offering specific evaluation metrics for enterprise-level data."},{"label":"License detail","value":"MIT license allows free usage and modification with attribution."}]}}