{"data":{"slug":"logikon-ai-awesome-deliberative-prompting","name":"awesome-deliberative-prompting","tagline":"Curated collection of resources on deliberative prompting for reliable reasoning with LLMs","github_url":"https://github.com/logikon-ai/awesome-deliberative-prompting","owner":"logikon-ai","repo":"awesome-deliberative-prompting","owner_avatar_url":"https://avatars.githubusercontent.com/u/126686445?v=4","primary_language":null,"stars":125,"forks":8,"topics":["awesome","awesome-list","chain-of-thought","deliberation","generative-ai","large-language-models","prompt-engineering","reasoning"],"archived":true,"github_pushed_at":"2025-02-03T20:00:33+00:00","maintenance_label":"Archived","url":"https://www.graphcanon.com/tools/logikon-ai-awesome-deliberative-prompting","markdown_url":"https://www.graphcanon.com/tools/logikon-ai-awesome-deliberative-prompting.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/logikon-ai-awesome-deliberative-prompting","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=logikon-ai-awesome-deliberative-prompting","description":"Awesome deliberative prompting: How to ask LLMs to produce reliable reasoning and make reason-responsive decisions. ","homepage_url":null,"license":"CC0-1.0","open_issues":0,"watchers":2,"ai_summary":"A comprehensive list focused on techniques and strategies for prompting large language models to make reason-responsive decisions and produce reliable reasoning.","readme_excerpt":"# Awesome Deliberative Prompting \n\n> [!NOTE]\n> _Deliberative prompting_, _chain-of-thought_, _self-reflection_ and _thinking_ have become mainstream techniques in AI. This archived opinionated reading lists documents the journey the community has taken to achieve this feat in less than 4 years, from the beginnings in 2021 to January 2025, when Deepseek R1 has been released. Thanks for following.\n\n**How to ask Large Language Models (LLMs) to produce reliable reasoning and make reason-responsive decisions.**\n\n> **deliberation**, n.\n>\n> The action of thinking carefully about something, esp. in order to reach a decision; careful consideration; an act or instance of this. (OED)\n\n\n## Contents\n\n- [Success Stories](#success-stories)\n- [Prompting Patterns and Strategies](#prompting-patterns-and-strategies)\n  - [Beyond \"Let's think step by step\"](#beyond-lets-think-step-by-step)\n  - [Multi-Agent Deliberation](#multi-agent-deliberation)\n  - [Reflection and Meta-Cognition](#reflection-and-meta-cognition)\n- [Text Generation Techniques](#text-generation-techniques)\n- [Self-Correction](#self-correction)\n- [Reasoning Analytics](#reasoning-analytics)\n- [Limitations, Failures, Puzzles](#limitations-failures-puzzles)\n- [Datasets](#datasets)\n- [Tools and Frameworks](#tools-and-frameworks)\n- [Other Resources](#other-resources)\n\n\n## Success Stories\n\n_Striking evidence for effectiveness of deliberative prompting._\n\n- 🎓 One of the first attempts to elicit reasoning traces from LLMs to improve performance, includes experiments with GPT-2. \"Thinking Aloud: Dynamic Context Generation Improves Zero-Shot Reasoning Performance of GPT-2.\" 2021-03-24. [[>paper](https://arxiv.org/abs/2103.13033)]\n- 🎓 The original \"chain of though\" (CoT) paper, first to give clear evidence that deliberative prompting works. \"Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.\" 2022-01-28. [[>paper](https://arxiv.org/abs/2201.11903)]\n- 🎓 Deliberative prompting improves ability of Google's LLMs to solve unseen difficult problems, and instruction-finetuned (Flan-) models are much better at it.\n  - \"Scaling Instruction-Finetuned Language Models.\" 2022-12-06. [[>paper](https://arxiv.org/abs/2210.11416)]\n  - \"PaLM 2 Technical Report.\" 2023-05-17. [[>paper](https://arxiv.org/abs/2305.10403)]\n- 🎓 Deliberative prompting is highly effective for OpenAI's models (Text-Davinci-003, ChatGPT, GPT-4), increasing accuracy in many (yet not all) reasoning tasks in the EvalAGI benchmark. \"AGIEval: A Human-Centric Benchmark for\nEvaluating Foundation Models.\" 2023-04-13. [[>paper](https://arxiv.org/abs/2304.06364)]\n- 🎓 Deliberative prompting unlocks latent cognitive skills and is more effective for bigger models. \"Challenging BIG-Bench tasks and whether chain-of-thought can solve them.\" 2022-10-17. [[>paper](https://arxiv.org/abs/2210.09261)]\n- 🎓 Experimentally introducing errors in CoT reasoning traces decreases decision accuracy, which provides indirect evidence for reason-responsiveness of LLMs. \"Stress Testing Chain-of-Thought Prompting for Large Language Models.\" 2023-09-28. [[>paper](https://arxiv.org/abs/2309.16621)]\n- 🎓 Reasoning (about retrieval candidates) improves RAG. \"Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection.\" 2023-10-17. [[>paper](https://arxiv.org/abs/2310.11511)]\n- 🎓 Deliberative reading notes improve RAG. \"Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models.\" 2023-11-15. [[>paper](https://arxiv.org/abs/2311.09210)]\n- 🎓 Good reasoning (CoT) causes good answers (i.e., LLMs are reason-responsive). \"Causal Abstraction for Chain-of-Thought Reasoning in Arithmetic Word Problems.\" 2023-12-07. [[>paper](https://aclanthology.org/2023.blackboxnlp-1.12.pdf)]\n- 🎓 Logical interpretation of internal layer-wise processing of reasoning tasks yields further evidence for reason-responsiveness. \"Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Model.\" 2023-12-07. [[>paper](ht","github_created_at":"2023-08-09T12:35:00+00:00","created_at":"2026-07-11T10:31:15.822256+00:00","updated_at":"2026-07-11T11:31:48.755353+00:00","categories":[{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"}],"tags":[{"slug":"chain-of-thought","name":"chain-of-thought"},{"slug":"deliberation","name":"deliberation"},{"slug":"prompt-engineering","name":"prompt-engineering"},{"slug":"reasoning","name":"reasoning"}],"trust":{"provenance":{"is_fork":false,"github_id":676538372,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T10:31:16.515Z","maintenance":{"label":"Archived","score":8,"methodology":"github_public_v1","releases_90d":0,"days_since_push":522,"last_release_at":null},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T10:31:18.540Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T11:31:16.500Z"},"license_spdx":{"value":"CC0-1.0","source":"github.license","observed_at":"2026-07-11T11:31:16.500Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["This repository does not specify any particular language requirements as it is an information resource. However, understanding the core concepts of prompting in"]},"constraints":null,"when_to_use":["- When you need specific guidance and resources for implementing deliberative prompting in your project to enhance the reliability of reasoning produced by LLMs.","- For exploring different patterns and strategies such as multi-agent deliberation, reflection, and meta-cognition that can help improve the performance of large language models on complex tasks."],"when_not_to_use":["- If you are looking for a comprehensive framework or software library to directly integrate into your application; Awesome Deliberative Prompting is an information resource rather than a software kit","- When seeking direct implementation assistance for specific programming challenges related to LLMs. This tool focuses on conceptual guidance and doesn't provide code snippets or technical support."],"source":"enrich:decision_facts","observed_at":"2026-07-11T11:31:48.401Z"},"constraint_facets":null,"decision_summary":[{"label":"Requirements","value":"This repository does not specify any particular language requirements as it is an information resource. However, understanding the core concepts of prompting in"},{"label":"Adopt for","value":"Awesome Deliberative Prompting is a curated collection focused on techniques and strategies for prompting large language models to produce reliable reasoning and make reason-responsive decisions."}]}}