GSM-IC
GSM-IC benchmark evaluates distractibility of language models with grade-school math problems and irrelevant context.
GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026
19views this month
Decision brief
GSM-IC is a benchmark dataset that evaluates the distractibility of language models by introducing irrelevant context into grade-school math problems.
Good fit when
- Use GSM-IC when you need to assess how well a language model can focus on relevant information in arithmetic reasoning tasks, especially when faced with distracting irrelevant context.
- Consider GSM-IC if your evaluation requires understanding the robustness of a model in handling problems that mimic real-world scenarios where irrelevant information is common.
Avoid when
- Avoid GSM-IC if your evaluation focuses on the model's ability to solve math problems without any form of distraction or if the context is always assumed to be relevant.
- Do not use GSM-IC if your primary interest is in evaluating the model's performance on pure arithmetic tasks without the added complexity of irrelevant context.
- Pricing:
- unknown - Pricing details are not available.
- Requirements:
- The dataset is derived from GSM8K and includes irrelevant context in problem descriptions to test the distractibility of language models.
Observed Sep 18, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Archived (1312d since push)
- As of Sep 18, 2026
- Provenance
- Not a fork · Organization account
- As of Sep 18, 2026
- Security (OSV)
- No lockfile
- As of Sep 18, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
git clone https://github.com/google-research-datasets/GSM-ICSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
GSM-IC is a dataset derived from GSM8K, designed to assess how easily language models can be distracted by irrelevant context in arithmetic reasoning tasks.
Capability facts
No sourced capability facts yet. Facts appear after ingest scans repo manifests (Dockerfile, package.json, MCP configs).
Categories
Tags
README
Grade School Math with Irrelevant Context (GSM IC) This repository contains the dataset Grade School Math with Irrelevant Context (GSM IC) used in this paper: Large Language Models Can Be Easily Distracted by Irrelevant Context. Data Format : the development split of GSM8K datase...
For agents
This page has a .md twin and JSON over the API.