LLM-Knowledge-Conflict
Examines behavior of large language models in knowledge conflicts using ConflictQA dataset
GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026
45views this month
Decision brief
LLM-Knowledge-Conflict is a tool for evaluating how large language models handle knowledge conflicts using the ConflictQA dataset. It provides a framework for analyzing model responses and their alignment with ground-tru
Good fit when
- When you need to assess how large language models handle conflicting information and their alignment with ground truth.
- If you are researching or developing large language models and want to understand their behavior in scenarios where conflicting evidence is presented.
Avoid when
- If you are looking for a tool that focuses on general performance metrics of large language models without considering knowledge conflicts.
- When your primary interest is in the speed or efficiency of large language models rather than their handling of conflicting information.
- Pricing:
- freemium - The tool is free to use under the Apache-2.0 license, but users may need to consider costs associated with running large language models.
- Requirements:
- Min 8 GB RAM; Python environment is required.; The dataset can be loaded using Huggingface datasets.
Observed Sep 18, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Dormant (889d since push)
- As of Sep 18, 2026
- Provenance
- Not a fork · Organization account
- As of Sep 18, 2026
- Security (OSV)
- No lockfile
- As of Sep 18, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install LLM-Knowledge-ConflictSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
This repository contains ConflictQA, a dataset and evaluation framework for assessing how large language models handle knowledge conflicts. It includes data and code for analyzing model responses and their alignment with ground truth and conflicting evidence.
Capability facts
- Languages
- python
Source: github.language · Sep 18, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Sep 18, 2026)
choose dataset "ConflictQA-popQA-[PLACEHOLDER]", and the [PLACEHOLDER] is in ["chatgpt","gpt4","palm2","llama2-7b","llama2-70b","qwen7b","vicuna7b","vicuna33b"].Source link
Tags
README
[ICLR'24 Spotlight] LLM knowledge conflict ConflictQA We provide the conflictQA based on different large language models, which utilizes large language models guided parametric memory. The data is available at conflictQA foloder. This folder contains the data for both POPQA and S...
For agents
This page has a .md twin and JSON over the API.