{"data":{"slug":"ai21labs-in-context-ralm","name":"in-context-ralm","tagline":"In-Context Retrieval-Augmented Language Models","github_url":"https://github.com/AI21Labs/in-context-ralm","owner":"AI21Labs","repo":"in-context-ralm","owner_avatar_url":"https://avatars.githubusercontent.com/u/33798954?v=4","primary_language":"Python","stars":295,"forks":28,"topics":[],"archived":true,"github_pushed_at":"2023-12-20T07:55:54+00:00","maintenance_label":"Archived","url":"https://www.graphcanon.com/tools/ai21labs-in-context-ralm","markdown_url":"https://www.graphcanon.com/tools/ai21labs-in-context-ralm.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/ai21labs-in-context-ralm","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=ai21labs-in-context-ralm","description":null,"homepage_url":null,"license":"Apache-2.0","open_issues":4,"watchers":6,"ai_summary":"Repository for reproducing experiments on WikiText-103 from AI21 Labs' research paper, focusing on in-context retrieval-augmented language models.","readme_excerpt":"# In-Context Retrieval-Augmented Language Models\n\nThis repo contains the code for reproducing the experiments on *WikiText-103* from [AI21 Labs](https://www.ai21.com/)' paper [In-Context Retrieval-Augmented Language Models](https://arxiv.org/abs/2302.00083) (In-Context RALM), to appear in the Transactions of the Association for Computational Linguistics (TACL).\n\nOur code is mainly based on the [Transformers](https://github.com/huggingface/transformers/) and [Pyserini](https://github.com/castorini/pyserini) libraries.  \nWe test it on Python 3.8.\n\n\n## Table of Contents\n- [Setup](#setup)\n- [Retrieval](#retrieval)\n- [Evaluation](#evaluation)\n  - [Language Models](#list-of-language-models)\n  - [Evaluate models w/o retrieval](#evaluate-models-wo-retrieval)\n  - [Evaluate models with retrieval](#evaluate-models-with-retrieval)\n  - [Evaluate models with reranking](#reranking)\n- [Question Answering Experiments](#question-answering-experiments)\n- [Citation](#citation)\n\n## Setup\n\nTo install the required libraries in our repo, run:\n```bash\npip install -r requirements.txt\n```\nTo have a Pytorch version specific to your CUDA, [install](https://pytorch.org/) your version before running the above command.\n\n## Retrieval\n\n### BM25\n\nOur BM25 preparation script works with Pyserini, so Java 11 is required - see their [installation guide](https://github.com/castorini/pyserini/blob/master/docs/installation.md).  \nIf you have Java 11 installed, make sure your `JAVA_HOME` environment variable is set to the correct path. \nOn a Linux system, the correct path might look something like `/usr/lib/jvm/java-11`.  \nThen run:\n\n```bash\npython prepare_retrieval_data.py \\\n--retrieval_type sparse \\\n--tokenizer_name $MODEL_NAME \\\n--max_length 1024 \\\n--dataset_path wikitext \\\n--dataset_name wikitext-103-v1 \\\n--dataset_split [validation, test] \\\n--index_name wikipedia-dpr \\\n--forbidden_titles_path ralm/retrievers/wikitext103_forbidden_titles.txt \\\n--stride 4 \\\n--output_file $RETRIEVAL_FILE \\\n--num_tokens_for_query 32 \\\n--num_docs 16 \n```\n\n## Evaluation\n\n### List of Language Models\n\nIn the paper, we give the results for the following models (replace `$MODEL_NAME` with one of those).  \nNote that the larger models may need model parallelism (on a 40GB A100, we used model parallelism for OPT-30B and OPT-66B).  \nSee details below on how to apply this option.\n\n* GPT-2: `gpt2`, `gpt2-medium`, `gpt2-large`, `gpt2-xl`\n* GPT-Neo: `EleutherAI/gpt-neo-1.3B`, `EleutherAI/gpt-neo-2.7B`, `EleutherAI/gpt-j-6B`\n* OPT: `facebook/opt-125m`, `facebook/opt-350m`, `facebook/opt-1.3b`, `facebook/opt-2.7b`, `facebook/opt-6.7b`, `facebook/opt-13b`, `facebook/opt-30b`, `facebook/opt-66b`\n\n### Evaluate models w/o retrieval\n\nTo run evaluation on models without retrieval, please use the following command (you can increase `stride` to 32 for faster evaluation):\n```bash\npython eval_lm.py \\\n--model_name $MODEL_NAME \\\n--dataset_path wikitext \\\n--dataset_name wikitext-103-v1 \\\n--dataset_split [validation, test] \\\n--output_dir $OUTPUT_DIR \\\n--stride 4 \\\n--max_length 1024 \\\n[--model_parallelism]\n```\n\n### Evaluate models with retrieval:\n\nTo run models with retrieval, use the `$RETRIEVAL_FILE` output from the `prepare_retrieval_data.py` script:\n```bash\npython eval_lm.py \\\n--model_name $MODEL_NAME \\\n--dataset_path wikitext \\\n--dataset_name wikitext-103-v1 \\\n--dataset_split [validation, test] \\\n--output_dir $OUTPUT_DIR \\\n--stride 4 \\\n--max_length 1024 \\\n[--model_parallelism] \\\n--retrieved_file $RETRIEVAL_FILE\n```\n\nNote: Our main retrieval flow assumes you want to use the top-scored passage from your retrieval file (`--ranking_strategy first`).\n\n### Reranking \n\nCurrently, we support `logprob` (the zero-shot method described in subsection 6.1) and `oracle` (to understand the potential gains from reranking).\n\nFor reranking, first you need to make sure you run the retrieval script with `num_docs=16` (or any other number you want to rerank on).\nIf you enable multiple GPUs, data parallelism will automatically be a","github_created_at":"2023-01-26T12:07:18+00:00","created_at":"2026-07-11T23:06:07.040064+00:00","updated_at":"2026-07-12T00:04:54.251936+00:00","categories":[{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"},{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"}],"tags":[{"slug":"wikitext-103","name":"wikitext-103"},{"slug":"question-answering-experiments","name":"question answering experiments"},{"slug":"language-models","name":"language-models"},{"slug":"retrieval-augmentation","name":"retrieval-augmentation"},{"slug":"transformers","name":"transformers"},{"slug":"pyserini","name":"pyserini"},{"slug":"bm25","name":"bm25"}],"trust":{"provenance":{"is_fork":false,"github_id":593596662,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T23:06:10.486Z","maintenance":{"label":"Archived","score":8,"methodology":"github_public_v1","releases_90d":0,"days_since_push":934,"last_release_at":null},"security_summary":{"status":"findings","scanner":"osv@v1","low_count":75,"high_count":0,"last_scan_at":"2026-07-11T23:06:10.960Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-12T00:04:54.154Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-07-12T00:04:54.154Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-07-12T00:04:54.154Z"}}}}