{"data":{"slug":"thudm-p-tuning-v2","name":"P-tuning-v2","tagline":"Optimized deep prompt tuning strategy comparable to fine-tuning across scales and tasks","github_url":"https://github.com/THUDM/P-tuning-v2","owner":"THUDM","repo":"P-tuning-v2","owner_avatar_url":"https://avatars.githubusercontent.com/u/48590610?v=4","primary_language":"Python","stars":2077,"forks":213,"topics":["natural-language-processing","p-tuning","parameter-efficient-learning","pretrained-language-model","prompt-tuning"],"archived":false,"github_pushed_at":"2023-11-16T04:38:09+00:00","maintenance_label":"Dormant","url":"https://www.graphcanon.com/tools/thudm-p-tuning-v2","markdown_url":"https://www.graphcanon.com/tools/thudm-p-tuning-v2.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/thudm-p-tuning-v2","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=thudm-p-tuning-v2","description":"An optimized deep prompt tuning strategy comparable to fine-tuning across scales and tasks","homepage_url":null,"license":"Apache-2.0","open_issues":35,"watchers":29,"ai_summary":"Repository for P-tuning v2, an optimized prompt tuning approach that provides performance similar to traditional fine-tuning but with fewer parameters.","readme_excerpt":"# P-tuning v2\n\n\nSource codes and data for\n* [ACL 2022] [P-Tuning v2: Prompt Tuning Can Be Comparable to Finetuning Universally Across Scales and Tasks](https://arxiv.org/abs/2110.07602) \n* [Findings of EMNLP 2023] [Parameter-Efficient Prompt Tuning Makes Generalized and Calibrated Neural Text Retrievers](https://arxiv.org/pdf/2207.07087.pdf)  [[Code]](https://github.com/THUDM/P-tuning-v2/tree/main/PT-Retrieval)\n\nAn optimized prompt tuning strategy achieving comparable performance to fine-tuning on small/medium-sized models and sequence tagging challenges. \n\nFind our previous version [P-tuning v1](https://github.com/THUDM/P-tuning) for knowledge probing and few-shot SuperGLUE. Your kindly starring our repo can greatly encourage us to work harder :)\n\nYou may be also interested in our recent work [GLM-130B: An Open Bilingual Pre-trained Model (2022-10-06)](https://arxiv.org/abs/2210.02414). It is an open-sourced LLM outperforming GPT-3 175B over various benchmarks. Get model weights, do inference and P-Tuning v2 with only **4 * RTX 3090 or 8 * RTX 2080 Ti** [FOR FREE](https://github.com/THUDM/GLM-130B)!\n\nP-tuning v2 leverages **deep prompt tuning**, which is to apply continuous prompts for every layer input of the pretrained transformer. \nDeep prompt tuning increases the capacity of continuous prompts and closes the gap to fine-tuning across various settings, especially for small models and hard tasks.\n\n\n\nThanks [@rainatam](https://github.com/rainatam)'s joint effort in re-organizing codes for publishing!\n\n## Commonly Asked Question\n1. Some readers notice a **'mismatch'** in SuperGLUE between P-tuning (v1) and P-tuning v2: This is because in P-tuning's SuperGLUE experiment, for fair comparison to PET, we follow its experimental setting where backbone pre-trained model parameters are jointly tuned with continuous prompt embeddings; while in P-tuning v2, we follow Prefix tuning and Lester et al.'s parameter-efficient setting where backbone pre-trained model parameters are frozen.\n\n## Reproduce Tips\nSince experiments reported in our paper are all conducted on NVIDIA DGX-A100 servers (which might be difficult to acquire), \nwe reimplement P-tuning v2's results on BERT-large/RoBERTa-large with:\n\n* Ubuntu servers with NVIDIA GeForce RTX 3090 (24G) GPUs\n* cuda 11.1\n* packages with certain versions (provided below)\n\nWe notice that the best hyper-parameters can be sensitive to your server environment and package version. \nIf you do not have the exact same environment, we highly recommend you to run hyper-parameter search in your environment\nbased on our example hyper-parameter search script in [search_script](search_script) and result collection scripts [search.py](search.py).\n\n### Setup\nWe conduct our experiment with Anaconda3. If you have installed Anaconda3, then create the environment for P-tuning v2:\n\n```shell\nconda create -n pt2 python=3.8.5\nconda activate pt2\n```\n\nAfter we setup basic conda environment, install pytorch related packages via:\n\n```shell\nconda install -n pt2 pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=11.0 -c pytorch\n```\n\nFinally, install other python packages we need:\n\n```shell\npip install -r requirements.txt\n```\n\n### Data\nFor SuperGLUE and SQuAD datasets, we download them from the Huggingface Datasets APIs (embedded in our codes).\n\nFor sequence tagging (NER, SRL) datasets, we prepare a non-official packup [here](https://zenodo.org/record/6318701/files/P-tuning-v2_data.tar.gz?download=1). \nAfter downloading, unzip the packup to the project root.\nPlease use at your own risk.\n\n### Training\nRun training scripts in [run_script](run_script) (e.g., RoBERTa for RTE):\n\n```shell\nbash run_script/run_rte_roberta.sh\n```\n\n### Implemented Results\nCurrently we have released our reimplementation on following tasks and datasets. More implementation will be released soon.\n\nReleased results on BERT-large\n\n|              | BoolQ | COPA | RTE  | WiC  | WSC  | CoNLL04 | OntoNotes 5.0 | CoNLL12 |\n|--------------|-------|-","github_created_at":"2021-10-14T14:16:05+00:00","created_at":"2026-07-11T23:22:12.881692+00:00","updated_at":"2026-08-03T00:01:48.490761+00:00","categories":[{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"natural-language-processing","name":"natural-language-processing"},{"slug":"p-tuning","name":"p-tuning"},{"slug":"parameter-efficient-learning","name":"parameter-efficient-learning"},{"slug":"pretrained-language-model","name":"pretrained-language-model"},{"slug":"prompt-tuning","name":"prompt-tuning"}],"trust":{"provenance":{"is_fork":false,"github_id":417157956,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-03T00:01:47.767Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":990,"last_release_at":null},"security_summary":{"status":"findings","scanner":"osv@v1","low_count":50,"high_count":0,"last_scan_at":"2026-07-11T23:22:15.224Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-08-03T00:01:48.204Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-08-03T00:01:48.204Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-08-03T00:01:48.204Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["Experiments were conducted on NVIDIA DGX-A100; however, RTX 3090 or RTX 2080 Ti GPUs can suffice given certain conditions.","CUDA version should be at least 11.1 and specific versions of pytorch (1.7.1), torchvision (0.8.2) and torchaudio (0.7.2).","A conda environment setup with Python 3.8.5 is recommended for optimal performance."],"min_ram_gb":8},"constraints":{"min_ram_gb":8},"when_to_use":["For scenarios requiring efficient use of resources, as P-tuning v2 requires fewer parameters than traditional fine-tuning methods.","When working with small to medium-sized models or tasks that are challenging for sequence tagging where deep prompt tuning can bridge the performance gap without full model fine-tuning."],"when_not_to_use":["Avoid using P-tuning v2 when you have large-scale datasets and model architectures, as it might not be as effective compared to full fine-tuning methods.","If your project requires a high level of customization beyond prompt tuning or continuous prompts do not align with the problem complexity."],"source":"enrich:decision_facts","observed_at":"2026-07-17T04:15:29.281Z"},"constraint_facets":{"min_ram_gb":8},"decision_summary":[{"label":"Requirements","value":"Min 8 GB RAM; Experiments were conducted on NVIDIA DGX-A100; however, RTX 3090 or RTX 2080 Ti GPUs can suffice given certain conditions.; CUDA version should be at least 11.1 and specific versions of pytorch (1.7.1), torchvision (0.8.2) and torchaudio (0.7.2).; A conda environment setup with Python 3.8.5 is recommended for optimal performance."},{"label":"Adopt for","value":"P-tuning-v2 is an optimized deep prompt tuning strategy that can be used for tasks like natural language processing and parameter-efficient learning where few parameters need to be adjusted compared to full fine-tuning."},{"label":"License detail","value":"P-tuning-v2 is provided under the Apache-2.0 license which permits free use, modification and distribution as long as copyright and license notice are preserved."}]}}