{"data":{"slug":"google-research-datasets-gsm-ic","name":"GSM-IC","tagline":"Grade-School Math with Irrelevant Context (GSM-IC) benchmark is an arithmetic reasoning dataset built upon GSM8K, by adding irrelevant sentences in problem descriptions. GSM-IC is constructed to evalu","github_url":"https://github.com/google-research-datasets/GSM-IC","owner":"google-research-datasets","repo":"GSM-IC","owner_avatar_url":"https://avatars.githubusercontent.com/u/14796310?v=4","primary_language":null,"stars":67,"forks":2,"topics":[],"archived":true,"github_pushed_at":"2023-02-13T19:55:04+00:00","maintenance_label":"Archived","url":"https://www.graphcanon.com/tools/google-research-datasets-gsm-ic","markdown_url":"https://www.graphcanon.com/tools/google-research-datasets-gsm-ic.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/google-research-datasets-gsm-ic","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=google-research-datasets-gsm-ic","description":"Grade-School Math with Irrelevant Context (GSM-IC) benchmark is an arithmetic reasoning dataset built upon GSM8K, by adding irrelevant sentences in problem descriptions. GSM-IC is constructed to evaluate the distractibility of language models.","homepage_url":null,"license":null,"open_issues":1,"watchers":1,"ai_summary":null,"readme_excerpt":"### Grade-School Math with Irrelevant Context (GSM-IC)\n\nThis repository contains the dataset Grade-School Math with Irrelevant Context (GSM-IC) used in this paper: [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://arxiv.org/abs/2302.00093).\n\n#### Data Format\n\n* ```GSM8K_validation.jsonl```: the development split of [GSM8K dataset](https://github.com/openai/grade-school-math) used in the experiments.\n\n| Field name | Value |\n|-------|--------------|\n| question | Input question. |\n| answer  | The ground truth answer. |\n| n_steps  | The number of intermediate steps to calculate the answer. |\n\n* ```GSM-IC_2step.json```: GSM-IC split with problems that require 2 intermediate steps.\n\n| Field name | Value |\n|-------|--------------|\n| original_question | Original question from the GSM8K development set. |\n| new_question  | The new question with irrelevant context added to the original question. |\n| answer  | The ground truth answer. |\n| n_steps  | The number of intermediate steps to calculate the answer. |\n| role_label, number_label, sentence_label| Categories of the added irrelevant context. Needed for result analysis, not needed for model prediction.|\n| role, number, sentence_template| Added irrelevant context. Not needed for experiments. |\n\n* ```GSM-IC_mstep.json```: GSM-IC split with problems that require more than 2 intermediate steps. Same format as ```GSM-IC_2step.json```.\n\n\n#### Citation\nIf you use the data released through this repository, please cite the following paper:\n\n```\n@article{shi2023large,\n  title={Large Language Models Can Be Easily Distracted by Irrelevant Context},\n  author={Shi, Freda and Chen, Xinyun and Misra, Kanishka and Scales, Nathan and Dohan, David and Chi, Ed and Schärli, Nathanael and Zhou, Denny},\n  journal={arXiv preprint arXiv:2302.00093},\n  year={2023}\n}\n```","github_created_at":"2023-02-13T17:55:22+00:00","created_at":"2026-07-11T23:07:39.605166+00:00","updated_at":"2026-07-11T23:07:45.889227+00:00","categories":[{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"}],"tags":[],"trust":{"provenance":{"is_fork":false,"github_id":601265689,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T23:07:42.958Z","maintenance":{"label":"Archived","score":8,"methodology":"github_public_v1","releases_90d":0,"days_since_push":1244,"last_release_at":null},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T23:07:43.290Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T23:07:42.695Z"}}}}