{"data":{"slug":"edinburghnlp-awesome-hallucination-detection","name":"awesome-hallucination-detection","tagline":"List of papers on hallucination detection in LLMs.","github_url":"https://github.com/EdinburghNLP/awesome-hallucination-detection","owner":"EdinburghNLP","repo":"awesome-hallucination-detection","owner_avatar_url":"https://avatars.githubusercontent.com/u/29982377?v=4","primary_language":null,"stars":1137,"forks":92,"topics":["hallucinations","llms","nlp"],"archived":false,"github_pushed_at":"2026-10-08T10:06:13+00:00","maintenance_label":"Very active","stars_delta_30d":10,"url":"https://www.graphcanon.com/tools/edinburghnlp-awesome-hallucination-detection","markdown_url":"https://www.graphcanon.com/tools/edinburghnlp-awesome-hallucination-detection.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/edinburghnlp-awesome-hallucination-detection","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=edinburghnlp-awesome-hallucination-detection","description":"List of papers on hallucination detection in LLMs.","homepage_url":null,"license":"Apache-2.0","open_issues":0,"watchers":27,"ai_summary":"Repository containing a curated list of research papers focused on methods to detect and mitigate hallucinations generated by large language models (LLMs), including specific techniques like process supervision for factual QA tasks and calibration benchmarks for scientific critiques.","readme_excerpt":"# awesome-hallucination-detection\n\n    \n\n## Papers and Summaries\n\n### [Enoki: Efficient Multi-Level Hallucination Detection](https://huggingface.co/papers/2609.00581)\n- **Metrics:** AUROC, AUPRC, Span Coverage F1, FLOPs.\n- **Datasets:** EnokiQA, HalluEntity, MuSHROOM, RAGTruth, PsiloQA, FactCheckBench, ANAH.\n- **Comments:** Introduces **Enoki**, an Open Information Extraction (OpenIE) framework for multi-granular hallucination detection that unifies claim-level verification and span-level localization through text-anchored relational facts, eliminating the need for separate claim-to-span alignment. Enoki provides three fact extraction approaches: (1) [**Enoki-Encoder**](https://huggingface.co/s-nlp/enoki-openie-encoder), a ModernBERT-based extractor trained with Iterative Grid Labeling (IGL) and permutation-invariant Hungarian matching to directly extract relational triples without autoregressive decoding, distilling an LLM-based extractor into an efficient encoder; (2) **Enoki-LLM**, an LLM-based OpenIE approach with incremental fact decomposition for precise localization; and (3) **Enoki-Rule**, a deterministic dependency-based extractor enhanced through agent-assisted rule evolution with automated validation. The framework demonstrates strong fine-grained hallucination detection performance and reports **FLOPs-based computational cost alongside latency**, enabling accuracy–efficiency comparisons across extraction approaches. The paper also introduces [**EnokiQA**](https://huggingface.co/datasets/s-nlp/EnokiQA), a long-form QA dataset with 3,990 labeled examples containing aligned claim-level and span-level hallucination annotations, plus 19,594 unlabeled examples. ([Code](https://github.com/s-nlp/Enoki)) (2026)\n\n\n### [Verifiable Rewards Beyond Math and Code: Lightweight Corpus-Grounded Process Supervision for Factual Question Answering](https://arxiv.org/abs/2605.29648)\n- **Metrics:** Exact Match (EM); training cost / wall-clock speedup (×) vs. neural-verifier baselines\n- **Datasets:** Five knowledge-intensive / factual QA benchmarks (incl. TriviaQA; 30 model×benchmark cells over six 3B–14B instruction-tuned models)\n- **Comments:** Proposes **CorVer** (Corpus Verify), a lightweight, plug-in *process reward* for RL fine-tuning (**GRPO**) of LLMs on factual QA that replaces neural verifiers (NLI, LLM-as-judge, retrieve-and-grade) with a **corpus-grounded** signal derived from **Wikipedia co-occurrence counts**. A small (0.5B) extractor turns each generated sentence into entity/claim spans whose corpus co-occurrence — queried via an **Infini-gram** index — yields sentence-level credit, which is mapped to **token-level advantages** and combined with response-level judge and format rewards in a GRPO update. Beats the raw baseline in every one of 30 cells (six instruction-tuned models, 3B–14B; five QA benchmarks), with an average **+4.1 EM on TriviaQA**, and outperforms four neural-verifier baselines in 18/20 cells at **4.8–8.4× lower training cost**. Code at https://github.com/shichengf/CorVer. (arXiv 2026)\n\n### [REFUTE: Scientific Critique & Epistemic Calibration Benchmark](https://bgpt.pro/refute)\n- **Metrics:** Truth Score (composite), critique skill, calibration/Brier, forced-choice flaw accuracy, planted-flaw soundness accuracy, missing-evidence refusal rate\n- **Datasets:** REFUTE v3 (`BGPT-OFFICIAL/refute`; 320 scored questions from recent science paper summaries)\n- **Comments:** Open Apache-2.0 benchmark from BGPT for whether LLMs critique recent science summaries without quietly overclaiming. Core finding: scientific-sounding ≠ scientifically honest — critique skill and calibration dissociate. Current complete board tops out around 74.5 Truth Score. Judge-free Inspect / lm-eval adapters. Complements factuality/hallucination suites by testing overclaim correction and missing-evidence refusal on science summaries. ([Site](https://bgpt.pro/refute), [Dataset](https://huggingface.co/datasets/BGPT-OFFICIAL/refute), [Leaderboar","github_created_at":"2023-09-15T18:23:21+00:00","created_at":"2026-07-11T10:30:42.080602+00:00","updated_at":"2026-10-11T06:00:09.704279+00:00","categories":[{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"}],"tags":[{"slug":"evaluation","name":"evaluation"},{"slug":"hallucination","name":"hallucination"},{"slug":"llms","name":"llms"},{"slug":"nlp","name":"nlp"},{"slug":"observability","name":"observability"}],"trust":{"provenance":{"is_fork":false,"github_id":692175642,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-10-11T06:00:08.959Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":0,"days_since_push":2,"last_release_at":null,"stars_delta_30d":10,"open_issues_delta_30d":0},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T10:30:43.534Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-10-11T06:00:09.409Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-10-11T06:00:09.409Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["- When focusing on specific methodologies like Corpus Verify (CorVer) from the paper 'Verifiable Rewards Beyond Math and Code' which utilizes lightweight, process-based rewards to mitigate hallucinat","- For scientific critique tasks requiring calibrated and evidence-grounded judgment as per REFUTE benchmark"],"when_not_to_use":["When immediate implementation or code is needed rather than research papers, this repository is not suitable as it only curates information on methodologies and benchmarks.","- If your focus is on general LLM training techniques without a specific emphasis on hallucination detection or calibration"],"source":"enrich:decision_facts","observed_at":"2026-07-11T11:23:07.428Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"awesome-hallucination-detection provides a curated list of research papers focused on techniques to detect and mitigate hallucinations in large language models (LLMs), including process supervision methods for factual QA"}]}}