{"data":{"slug":"ai21labs-in-context-ralm","name":"in-context-ralm","tagline":"In-Context Retrieval-Augmented Language Models","github_url":"https://github.com/AI21Labs/in-context-ralm","owner":"AI21Labs","repo":"in-context-ralm","owner_avatar_url":"https://avatars.githubusercontent.com/u/33798954?v=4","primary_language":"Python","stars":295,"forks":28,"topics":[],"archived":true,"github_pushed_at":"2023-12-20T07:55:54+00:00","maintenance_label":"Archived","stars_delta_30d":0,"url":"https://www.graphcanon.com/tools/ai21labs-in-context-ralm","markdown_url":"https://www.graphcanon.com/tools/ai21labs-in-context-ralm.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/ai21labs-in-context-ralm","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=ai21labs-in-context-ralm","description":null,"homepage_url":null,"license":"Apache-2.0","open_issues":4,"watchers":6,"ai_summary":"Repository for reproducing experiments on WikiText-103 from AI21 Labs' paper on In-Context RALM, using Transformers and Pyserini libraries.","readme_excerpt":"# In-Context Retrieval-Augmented Language Models\n\nThis repo contains the code for reproducing the experiments on *WikiText-103* from [AI21 Labs](https://www.ai21.com/)' paper [In-Context Retrieval-Augmented Language Models](https://arxiv.org/abs/2302.00083) (In-Context RALM), to appear in the Transactions of the Association for Computational Linguistics (TACL).\n\nOur code is mainly based on the [Transformers](https://github.com/huggingface/transformers/) and [Pyserini](https://github.com/castorini/pyserini) libraries.  \nWe test it on Python 3.8.\n\n\n## Table of Contents\n- [Setup](#setup)\n- [Retrieval](#retrieval)\n- [Evaluation](#evaluation)\n  - [Language Models](#list-of-language-models)\n  - [Evaluate models w/o retrieval](#evaluate-models-wo-retrieval)\n  - [Evaluate models with retrieval](#evaluate-models-with-retrieval)\n  - [Evaluate models with reranking](#reranking)\n- [Question Answering Experiments](#question-answering-experiments)\n- [Citation](#citation)\n\n## Setup\n\nTo install the required libraries in our repo, run:\n```bash\npip install -r requirements.txt\n```\nTo have a Pytorch version specific to your CUDA, [install](https://pytorch.org/) your version before running the above command.\n\n## Retrieval\n\n### BM25\n\nOur BM25 preparation script works with Pyserini, so Java 11 is required - see their [installation guide](https://github.com/castorini/pyserini/blob/master/docs/installation.md).  \nIf you have Java 11 installed, make sure your `JAVA_HOME` environment variable is set to the correct path. \nOn a Linux system, the correct path might look something like `/usr/lib/jvm/java-11`.  \nThen run:\n\n```bash\npython prepare_retrieval_data.py \\\n--retrieval_type sparse \\\n--tokenizer_name $MODEL_NAME \\\n--max_length 1024 \\\n--dataset_path wikitext \\\n--dataset_name wikitext-103-v1 \\\n--dataset_split [validation, test] \\\n--index_name wikipedia-dpr \\\n--forbidden_titles_path ralm/retrievers/wikitext103_forbidden_titles.txt \\\n--stride 4 \\\n--output_file $RETRIEVAL_FILE \\\n--num_tokens_for_query 32 \\\n--num_docs 16 \n```\n\n## Evaluation\n\n### List of Language Models\n\nIn the paper, we give the results for the following models (replace `$MODEL_NAME` with one of those).  \nNote that the larger models may need model parallelism (on a 40GB A100, we used model parallelism for OPT-30B and OPT-66B).  \nSee details below on how to apply this option.\n\n* GPT-2: `gpt2`, `gpt2-medium`, `gpt2-large`, `gpt2-xl`\n* GPT-Neo: `EleutherAI/gpt-neo-1.3B`, `EleutherAI/gpt-neo-2.7B`, `EleutherAI/gpt-j-6B`\n* OPT: `facebook/opt-125m`, `facebook/opt-350m`, `facebook/opt-1.3b`, `facebook/opt-2.7b`, `facebook/opt-6.7b`, `facebook/opt-13b`, `facebook/opt-30b`, `facebook/opt-66b`\n\n### Evaluate models w/o retrieval\n\nTo run evaluation on models without retrieval, please use the following command (you can increase `stride` to 32 for faster evaluation):\n```bash\npython eval_lm.py \\\n--model_name $MODEL_NAME \\\n--dataset_path wikitext \\\n--dataset_name wikitext-103-v1 \\\n--dataset_split [validation, test] \\\n--output_dir $OUTPUT_DIR \\\n--stride 4 \\\n--max_length 1024 \\\n[--model_parallelism]\n```\n\n### Evaluate models with retrieval:\n\nTo run models with retrieval, use the `$RETRIEVAL_FILE` output from the `prepare_retrieval_data.py` script:\n```bash\npython eval_lm.py \\\n--model_name $MODEL_NAME \\\n--dataset_path wikitext \\\n--dataset_name wikitext-103-v1 \\\n--dataset_split [validation, test] \\\n--output_dir $OUTPUT_DIR \\\n--stride 4 \\\n--max_length 1024 \\\n[--model_parallelism] \\\n--retrieved_file $RETRIEVAL_FILE\n```\n\nNote: Our main retrieval flow assumes you want to use the top-scored passage from your retrieval file (`--ranking_strategy first`).\n\n### Reranking \n\nCurrently, we support `logprob` (the zero-shot method described in subsection 6.1) and `oracle` (to understand the potential gains from reranking).\n\nFor reranking, first you need to make sure you run the retrieval script with `num_docs=16` (or any other number you want to rerank on).\nIf you enable multiple GPUs, data parallelism will automatically be a","github_created_at":"2023-01-26T12:07:18+00:00","created_at":"2026-07-11T23:06:07.040064+00:00","updated_at":"2026-09-20T04:28:57.832311+00:00","categories":[{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"in-context-ralm","name":"in-context ralm"},{"slug":"language-models","name":"language models"},{"slug":"pyserini","name":"pyserini"},{"slug":"python","name":"python"},{"slug":"question-answering-experiments","name":"question answering experiments"},{"slug":"retrieval-augmented-language-models","name":"retrieval-augmented language models"},{"slug":"transformers","name":"transformers"},{"slug":"wikitext-103","name":"wikitext-103"}],"trust":{"provenance":{"is_fork":false,"github_id":593596662,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-18T08:11:40.804Z","maintenance":{"label":"Archived","score":8,"methodology":"github_public_v1","releases_90d":0,"days_since_push":1003,"last_release_at":null,"stars_delta_30d":0,"open_issues_delta_30d":0},"security_summary":{"status":"findings","scanner":"osv@v1","low_count":82,"high_count":0,"last_scan_at":"2026-09-18T08:11:43.350Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-18T08:11:39.506Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-09-18T08:11:39.506Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-09-18T08:11:39.506Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["Ensure Python 3.8 is installed.","Java 11 is required for BM25 preparation with Pyserini.","CUDA-specific Pytorch installation is recommended for optimal performance."],"min_ram_gb":8,"requires_docker":false},"constraints":{"min_ram_gb":8,"requires_docker":false},"when_to_use":["Use in-context-ralm when you need to reproduce experiments on WikiText-103 as detailed in AI21 Labs' paper.","Use it if you are working with retrieval-augmented language models and require integration with the Transformers and Pyserini libraries.","Consider it when your project involves evaluating language models with and without retrieval on the WikiText-103 dataset."],"when_not_to_use":["Do not use in-context-ralm if your project does not involve the WikiText-103 dataset or the specific language models tested in the AI21 Labs paper.","Avoid using this tool if you do not require the specific setup involving Pyserini for BM25 retrieval, as it necessitates Java 11 and setting the JAVA_HOME environment variable.","Do not use in-context-ralm if your project does not align with the Apache-2.0 license terms."],"source":"enrich:decision_facts","observed_at":"2026-09-18T11:30:16.461Z"},"constraint_facets":{"min_ram_gb":8,"requires_docker":false},"decision_summary":[{"label":"Requirements","value":"Min 8 GB RAM; Ensure Python 3.8 is installed.; Java 11 is required for BM25 preparation with Pyserini.; CUDA-specific Pytorch installation is recommended for optimal performance."},{"label":"Adopt for","value":"in-context-ralm is a Python-based tool for reproducing experiments on WikiText-103 from AI21 Labs' paper on In-Context Retrieval-Augmented Language Models, leveraging Transformers and Pyserini libraries."}]}}