{"data":{"slug":"jagilley-fact-checker","name":"fact-checker","tagline":"Fact-checking LLM outputs with self-ask","github_url":"https://github.com/jagilley/fact-checker","owner":"jagilley","repo":"fact-checker","owner_avatar_url":"https://avatars.githubusercontent.com/u/37783831?v=4","primary_language":"Jupyter Notebook","stars":314,"forks":39,"topics":["llm","python"],"archived":false,"github_pushed_at":"2023-10-23T21:08:13+00:00","maintenance_label":"Dormant","stars_delta_30d":1,"url":"https://www.graphcanon.com/tools/jagilley-fact-checker","markdown_url":"https://www.graphcanon.com/tools/jagilley-fact-checker.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/jagilley-fact-checker","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=jagilley-fact-checker","description":"Fact-checking LLM outputs with self-ask","homepage_url":null,"license":null,"open_issues":0,"watchers":5,"ai_summary":"A simple demonstration for fact-checking Language Model outputs through prompt chaining to verify the assumptions made in the initial response.","readme_excerpt":"# fact checking with prompt chaining\n\nThis repo is a simple demonstration of doing fact-checking with prompt chaining. How it works:\n- you ask your desired LLM a question\n- the LLM generates an initial answer to the question\n- the LLM self-interrogates what the assumptions were that went into that answer\n- the LLM sequentially determines if each of these assumptions are true\n- the LLM generates a new answer to the question, incorporating the new information\n\n## to run\n\nRun\n\n`python3 fact_checker.py 'insert question here'`\n\n*Be sure to wrap your question in quotes if you're passing it as a command line argument.*\n\nAlternatively, you can use the provided `fact_checker.ipynb` notebook.\n\n## example\n\nQuestion: *\"What type of mammal lays the biggest eggs?\"*\n\n\n**Initial answer:**\nThe biggest eggs laid by any mammal belong to the elephant.\n\n**Assumptions made:**\n- The elephant is a mammal \n- Mammals lay eggs \n- Eggs come in different sizes \n- Elephants lay bigger eggs than other mammals\n\n**Verification of assumptions:**\n- The elephant is a mammal: TRUE \n- Mammals lay eggs: FALSE - Most mammals give birth to live young.\n- Eggs come in different sizes: TRUE \n- Elephants lay bigger eggs than other mammals: FALSE - Elephants do not lay eggs.\n\n**New answer:**\nThis question cannot be answered because elephants do not lay eggs and most mammals give birth to live young.\n\n## credits\nProof of concept by [Jasper](https://twitter.com/0xjasper)","github_created_at":"2022-12-06T23:58:18+00:00","created_at":"2026-07-11T10:50:04.54907+00:00","updated_at":"2026-09-20T05:26:58.655776+00:00","categories":[{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"}],"tags":[{"slug":"fact-checking","name":"fact-checking"},{"slug":"llm","name":"llm"},{"slug":"prompt-chaining","name":"prompt-chaining"},{"slug":"python","name":"python"}],"trust":{"provenance":{"is_fork":false,"github_id":575183014,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-20T05:26:56.740Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":1062,"last_release_at":null,"stars_delta_30d":1,"open_issues_delta_30d":0},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T10:50:10.266Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-20T05:26:57.854Z"},"languages":{"value":["jupyter notebook"],"source":"github.language","observed_at":"2026-09-20T05:26:57.854Z"}},"decision_facts":{"hosting":null,"pricing":{"model":"unknown","summary":"The licensing information for `fact-checker` is unclear, indicating that further investigation into its legal usage might be required before implementation."},"requirements":{"notes":["Requires Python and possibly Jupyter Notebook environment for running the provided IPython notebook script or command-line script."],"min_ram_gb":null},"constraints":{"min_ram_gb":null,"pricing_model":"unknown"},"when_to_use":["- When you need to verify the accuracy of assumptions made by a Language Model’s initial response through self-ask methodologies.","- For scenarios where you want to improve trustworthiness and credibility of LLM-generated content using predefined assumptions validation."],"when_not_to_use":["- If an immediate answer is required without the step-by-step reassessment process, as `fact-checker` involves sequential validation that could be time-consuming.","- In situations where real-time interaction is critical and a delay from additional self-interrogation steps would not be beneficial for user experience."],"source":"enrich:decision_facts","observed_at":"2026-07-12T11:02:48.797Z"},"constraint_facets":{"min_ram_gb":null,"pricing_model":"unknown"},"decision_summary":[{"label":"Pricing","value":"unknown - The licensing information for `fact-checker` is unclear, indicating that further investigation into its legal usage might be required before implementation."},{"label":"Requirements","value":"Requires Python and possibly Jupyter Notebook environment for running the provided IPython notebook script or command-line script."},{"label":"Adopt for","value":"`fact-checker` utilizes prompt chaining in Jupyter Notebook to fact-check Language Model outputs, enhancing the accuracy and reliability of responses."}]}}