{"data":{"slug":"thudm-p-tuning-v2","name":"P-tuning-v2","tagline":"An optimized deep prompt tuning strategy comparable to fine-tuning across scales and tasks","github_url":"https://github.com/THUDM/P-tuning-v2","owner":"THUDM","repo":"P-tuning-v2","owner_avatar_url":"https://avatars.githubusercontent.com/u/48590610?v=4","primary_language":"Python","stars":2075,"forks":212,"topics":["natural-language-processing","p-tuning","parameter-efficient-learning","pretrained-language-model","prompt-tuning"],"archived":false,"github_pushed_at":"2023-11-16T04:38:09+00:00","maintenance_label":"Dormant","url":"https://www.graphcanon.com/tools/thudm-p-tuning-v2","markdown_url":"https://www.graphcanon.com/tools/thudm-p-tuning-v2.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/thudm-p-tuning-v2","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=thudm-p-tuning-v2","description":"An optimized deep prompt tuning strategy comparable to fine-tuning across scales and tasks","homepage_url":null,"license":"Apache-2.0","open_issues":35,"watchers":29,"ai_summary":null,"readme_excerpt":"# P-tuning v2\n\n\nSource codes and data for\n* [ACL 2022] [P-Tuning v2: Prompt Tuning Can Be Comparable to Finetuning Universally Across Scales and Tasks](https://arxiv.org/abs/2110.07602) \n* [Findings of EMNLP 2023] [Parameter-Efficient Prompt Tuning Makes Generalized and Calibrated Neural Text Retrievers](https://arxiv.org/pdf/2207.07087.pdf)  [[Code]](https://github.com/THUDM/P-tuning-v2/tree/main/PT-Retrieval)\n\nAn optimized prompt tuning strategy achieving comparable performance to fine-tuning on small/medium-sized models and sequence tagging challenges. \n\nFind our previous version [P-tuning v1](https://github.com/THUDM/P-tuning) for knowledge probing and few-shot SuperGLUE. Your kindly starring our repo can greatly encourage us to work harder :)\n\nYou may be also interested in our recent work [GLM-130B: An Open Bilingual Pre-trained Model (2022-10-06)](https://arxiv.org/abs/2210.02414). It is an open-sourced LLM outperforming GPT-3 175B over various benchmarks. Get model weights, do inference and P-Tuning v2 with only **4 * RTX 3090 or 8 * RTX 2080 Ti** [FOR FREE](https://github.com/THUDM/GLM-130B)!\n\nP-tuning v2 leverages **deep prompt tuning**, which is to apply continuous prompts for every layer input of the pretrained transformer. \nDeep prompt tuning increases the capacity of continuous prompts and closes the gap to fine-tuning across various settings, especially for small models and hard tasks.\n\n\n\nThanks [@rainatam](https://github.com/rainatam)'s joint effort in re-organizing codes for publishing!\n\n## Commonly Asked Question\n1. Some readers notice a **'mismatch'** in SuperGLUE between P-tuning (v1) and P-tuning v2: This is because in P-tuning's SuperGLUE experiment, for fair comparison to PET, we follow its experimental setting where backbone pre-trained model parameters are jointly tuned with continuous prompt embeddings; while in P-tuning v2, we follow Prefix tuning and Lester et al.'s parameter-efficient setting where backbone pre-trained model parameters are frozen.\n\n## Reproduce Tips\nSince experiments reported in our paper are all conducted on NVIDIA DGX-A100 servers (which might be difficult to acquire), \nwe reimplement P-tuning v2's results on BERT-large/RoBERTa-large with:\n\n* Ubuntu servers with NVIDIA GeForce RTX 3090 (24G) GPUs\n* cuda 11.1\n* packages with certain versions (provided below)\n\nWe notice that the best hyper-parameters can be sensitive to your server environment and package version. \nIf you do not have the exact same environment, we highly recommend you to run hyper-parameter search in your environment\nbased on our example hyper-parameter search script in [search_script](search_script) and result collection scripts [search.py](search.py).\n\n### Setup\nWe conduct our experiment with Anaconda3. If you have installed Anaconda3, then create the environment for P-tuning v2:\n\n```shell\nconda create -n pt2 python=3.8.5\nconda activate pt2\n```\n\nAfter we setup basic conda environment, install pytorch related packages via:\n\n```shell\nconda install -n pt2 pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=11.0 -c pytorch\n```\n\nFinally, install other python packages we need:\n\n```shell\npip install -r requirements.txt\n```\n\n### Data\nFor SuperGLUE and SQuAD datasets, we download them from the Huggingface Datasets APIs (embedded in our codes).\n\nFor sequence tagging (NER, SRL) datasets, we prepare a non-official packup [here](https://zenodo.org/record/6318701/files/P-tuning-v2_data.tar.gz?download=1). \nAfter downloading, unzip the packup to the project root.\nPlease use at your own risk.\n\n### Training\nRun training scripts in [run_script](run_script) (e.g., RoBERTa for RTE):\n\n```shell\nbash run_script/run_rte_roberta.sh\n```\n\n### Implemented Results\nCurrently we have released our reimplementation on following tasks and datasets. More implementation will be released soon.\n\nReleased results on BERT-large\n\n|              | BoolQ | COPA | RTE  | WiC  | WSC  | CoNLL04 | OntoNotes 5.0 | CoNLL12 |\n|--------------|-------|-","github_created_at":"2021-10-14T14:16:05+00:00","created_at":"2026-07-11T23:22:12.881692+00:00","updated_at":"2026-07-11T23:22:23.425178+00:00","categories":[{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"},{"slug":"vector-databases","name":"Vector Databases","url":"https://www.graphcanon.com/categories/vector-databases","markdown_url":"https://www.graphcanon.com/categories/vector-databases.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/vector-databases"}],"tags":[{"slug":"p-tuning","name":"p-tuning"},{"slug":"python","name":"python"},{"slug":"prompt-tuning","name":"prompt-tuning"},{"slug":"parameter-efficient-learning","name":"parameter-efficient-learning"},{"slug":"natural-language-processing","name":"natural-language-processing"},{"slug":"pretrained-language-model","name":"pretrained-language-model"}],"trust":{"provenance":{"is_fork":false,"github_id":417157956,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T23:22:14.756Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":968,"last_release_at":null},"security_summary":{"status":"findings","scanner":"osv@v1","low_count":50,"high_count":0,"last_scan_at":"2026-07-11T23:22:15.224Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T23:22:14.511Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-07-11T23:22:14.511Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-07-11T23:22:14.511Z"}}}}