{"data":{"slug":"edinburghnlp-awesome-hallucination-detection","name":"awesome-hallucination-detection","tagline":"List of papers on hallucination detection in LLMs.","github_url":"https://github.com/EdinburghNLP/awesome-hallucination-detection","owner":"EdinburghNLP","repo":"awesome-hallucination-detection","owner_avatar_url":"https://avatars.githubusercontent.com/u/29982377?v=4","primary_language":null,"stars":1116,"forks":89,"topics":["hallucinations","llms","nlp"],"archived":false,"github_pushed_at":"2026-06-06T06:25:53+00:00","maintenance_label":"Steady","url":"https://www.graphcanon.com/tools/edinburghnlp-awesome-hallucination-detection","markdown_url":"https://www.graphcanon.com/tools/edinburghnlp-awesome-hallucination-detection.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/edinburghnlp-awesome-hallucination-detection","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=edinburghnlp-awesome-hallucination-detection","description":"List of papers on hallucination detection in LLMs.","homepage_url":null,"license":"Apache-2.0","open_issues":0,"watchers":26,"ai_summary":"Repository containing a curated list of research papers focused on methods to detect and mitigate hallucinations generated by large language models (LLMs), including specific techniques like process supervision for factual QA tasks and calibration benchmarks for scientific critiques.","readme_excerpt":"# awesome-hallucination-detection\n\n    \n\n## Papers and Summaries\n\n\n### [Verifiable Rewards Beyond Math and Code: Lightweight Corpus-Grounded Process Supervision for Factual Question Answering](https://arxiv.org/abs/2605.29648)\n- **Metrics:** Exact Match (EM); training cost / wall-clock speedup (×) vs. neural-verifier baselines\n- **Datasets:** Five knowledge-intensive / factual QA benchmarks (incl. TriviaQA; 30 model×benchmark cells over six 3B–14B instruction-tuned models)\n- **Comments:** Proposes **CorVer** (Corpus Verify), a lightweight, plug-in *process reward* for RL fine-tuning (**GRPO**) of LLMs on factual QA that replaces neural verifiers (NLI, LLM-as-judge, retrieve-and-grade) with a **corpus-grounded** signal derived from **Wikipedia co-occurrence counts**. A small (0.5B) extractor turns each generated sentence into entity/claim spans whose corpus co-occurrence — queried via an **Infini-gram** index — yields sentence-level credit, which is mapped to **token-level advantages** and combined with response-level judge and format rewards in a GRPO update. Beats the raw baseline in every one of 30 cells (six instruction-tuned models, 3B–14B; five QA benchmarks), with an average **+4.1 EM on TriviaQA**, and outperforms four neural-verifier baselines in 18/20 cells at **4.8–8.4× lower training cost**. Code at https://github.com/shichengf/CorVer. (arXiv 2026)\n\n### [REFUTE: Scientific Critique & Epistemic Calibration Benchmark](https://huggingface.co/datasets/BGPT-OFFICIAL/refute)\n- **Metrics:** Critique skill score, Brier calibration, forced-choice flaw accuracy, planted-flaw soundness accuracy, missing-evidence refusal rate\n- **Datasets:** REFUTE (`refute_hard_60`, `refute_120`, `refute_soundness`; 120 critique + 74 soundness vignettes from recent science paper summaries)\n- **Comments:** Open Apache-2.0 Hugging Face benchmark from BGPT measuring whether LLMs critique recent science summaries with calibrated, evidence-grounded judgment. Separates critique skill from uncertainty honesty: strongest critics are often most overconfident when evidence is weak. Includes judge-free Inspect AI and lm-evaluation-harness adapters plus a technical report. Complements factuality/hallucination detection benchmarks by testing overclaim correction and missing-evidence refusal on scientific summaries. ([Technical report](https://huggingface.co/datasets/BGPT-OFFICIAL/refute/blob/main/TECHNICAL_REPORT.md), [Leaderboard](https://huggingface.co/spaces/BGPT-OFFICIAL/refute-leaderboard)) (2026)\n\n### [QuCo-RAG: Quantifying Uncertainty from the Pre-training Corpus for Dynamic Retrieval-Augmented Generation](https://arxiv.org/abs/2512.19134)\n- **Metrics:** Exact Match (EM), token-level F1\n- **Datasets:** HotpotQA, 2WikiMultihopQA, ASQA (long-form), PubMedQA (biomedical)\n- **Comments:** Proposes **QuCo-RAG**, a dynamic RAG framework that determines *when* to retrieve by quantifying uncertainty from **objective pre-training-corpus statistics** rather than ill-calibrated model-internal signals (logits, entropy). Two-stage detection: (1) *Pre-Generation Knowledge Assessment* — queries entity frequencies in the pre-training corpus and triggers retrieval when entities are low-frequency (long-tail knowledge risk); (2) *Runtime Claim Verification* — extracts knowledge triplets from each generated sentence and verifies entity co-occurrence in the corpus, where zero co-occurrence triggers retrieval and regeneration. Both stages leverage **Infini-gram** for millisecond-latency queries over a 4-trillion-token corpus. On OLMo-2 (7B/13B/32B, matched corpus), achieves **+5–12 EM** over state-of-the-art dynamic-RAG baselines including DRAGIN. Cross-model transfer to Llama-3, Qwen2.5, GPT-4.1, and GPT-5-chat — which have undisclosed pre-training data — yields **up to +14 EM** by exploiting web-scale corpus overlap. Best performance on ASQA long-form QA and PubMedQA, while internal-signal methods show limitations in either efficiency or effectiveness. Code at https://g","github_created_at":"2023-09-15T18:23:21+00:00","created_at":"2026-07-11T10:30:42.080602+00:00","updated_at":"2026-07-11T11:23:07.727145+00:00","categories":[{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"}],"tags":[{"slug":"llms","name":"llms"},{"slug":"evaluation","name":"evaluation"},{"slug":"nlp","name":"nlp"},{"slug":"observability","name":"observability"},{"slug":"hallucination","name":"hallucination"}],"trust":{"provenance":{"is_fork":false,"github_id":692175642,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T10:30:42.697Z","maintenance":{"label":"Steady","score":60,"methodology":"github_public_v1","releases_90d":0,"days_since_push":35,"last_release_at":null},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T10:30:43.534Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T11:22:43.964Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-07-11T11:22:43.964Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["- When focusing on specific methodologies like Corpus Verify (CorVer) from the paper 'Verifiable Rewards Beyond Math and Code' which utilizes lightweight, process-based rewards to mitigate hallucinat","- For scientific critique tasks requiring calibrated and evidence-grounded judgment as per REFUTE benchmark"],"when_not_to_use":["- When the need is for immediate implementation or code rather than research papers — this repository only curates information about methodologies and benchmarks","- If your focus is on general LLM training techniques without a specific emphasis on hallucination detection or calibration"],"source":"enrich:decision_facts","observed_at":"2026-07-11T11:23:07.428Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"awesome-hallucination-detection provides a curated list of research papers focused on techniques to detect and mitigate hallucinations in large language models (LLMs), including process supervision methods for factual QA"}]}}