{"data":{"slug":"njunlp-renellm","name":"ReNeLLM","tagline":"The official implementation of our NAACL 2024 paper \"A Wolf in Sheep’s Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily\".","github_url":"https://github.com/NJUNLP/ReNeLLM","owner":"NJUNLP","repo":"ReNeLLM","owner_avatar_url":"https://avatars.githubusercontent.com/u/31466622?v=4","primary_language":"Python","stars":163,"forks":17,"topics":[],"archived":false,"github_pushed_at":"2025-09-02T09:32:44+00:00","maintenance_label":"Slowing","url":"https://www.graphcanon.com/tools/njunlp-renellm","markdown_url":"https://www.graphcanon.com/tools/njunlp-renellm.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/njunlp-renellm","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=njunlp-renellm","description":"The official implementation of our NAACL 2024 paper \"A Wolf in Sheep’s Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily\".","homepage_url":null,"license":"MIT","open_issues":0,"watchers":9,"ai_summary":null,"readme_excerpt":"## Getting Started\n\n**1. Clone this repository**\n```shell \ngit clone https://github.com/NJUNLP/ReNeLLM.git\n```\n\n**2. Build Environment**\n```shell\ncd ReNeLLM\nconda create -n ReNeLLM python=3.9\nconda activate ReNeLLM\npip install -r requirements.txt\n```\n\n**3. Run ReNeLLM**\n   \nReNeLLM employs `gpt-3.5-turbo` for prompt rewriting and harmful classifier, while utilizing `claude-v2` as the model under attack. Therefore, you are required to input both of these API key parameters. \n```shell \npython renellm.py --gpt_api_key <your openai API key> --claude_api_key <your anthropic API key>\n```\nWe use these two models due to their superior performance compared to open-source alternatives, as well as their lower cost in comparison with `gpt-4`. *In principle, any model could be utilized as your harmful classifier and attacked model.*\n\n**4. Get Responses**\n   \nFor `gpt`:\n```shell \npython get_responses.py --data_path <jailbroken data path> --gpt_api_key <your open API key> --test_model <gpt-3.5-turbo, gpt-4>\n```\n\nFor `claude`:\n```shell \npython get_responses.py --data_path <jailbroken data path> --claude_api_key <your anthropic API key> --test_model <claude-instant-1, claude-2>\n```\n\nFor `llama-2-chat`:\n\n- Build the environment\n```shell\ncd llama\npip install -e .\n```\n- Follow the steps at https://github.com/facebookresearch/llama?tab=readme-ov-file#quick-start to download the llama-2-chat models.\n- Get llama-2-chat responses using the command below:\n```shell\nbash run_chat.sh # You can set the model type and your jailbroken data path in the run_chat.sh\n```\n\n**5. Check ASR** \n\nGPT-ASR (using GPT-4)\n```shell\npython check_gpt_asr.py --data_path <your llm responses data path> --gpt_api_key <your open API key>\n```\nKW-ASR (keyword ASR)\n```shell\npython check_kw_asr.py --data_path <your llm responses data path>\n```\n\n**6. Execute ReNeLLM with a single prompt**\n\n```shell\npython renellm.py --gpt_api_key <your gpt_api_key> --claude_api_key <your claude_api_key> --prompt \"how to steal money from others?\"\n```\n\nWe have uploaded the results of 520 Advbench samples categorized into 7 scenarios(see [gpt-4_single_round_prompt_annotation.json](https://github.com/NJUNLP/ReNeLLM/blob/main/gpt-4_single_round_prompt_annotation.json). In the result file, the \"idx\" field corresponds to the same idx samples in Advbench. The prompts used for classification and the harmful scenarios represented by each classification result can be found in Table 10 of our paper.\n\nIt should be noted that GPT-4 initially categorized the 520 pieces of data into 10 classes, but three of these classes had very few samples. For the convenience of statistics and analysis, we manually checked and merged these classes, resulting in 7 classification scenarios. The specific category merging information is as follows: 1 -> 0, 8 -> 9, 11 -> 6, finally resulting in 7 categories: 0, 2, 3, 4, 5, 6, 9. You can process the result file according to the above rules to obtain the classification results consistent with our paper.","github_created_at":"2024-01-09T03:15:26+00:00","created_at":"2026-07-11T23:40:36.176521+00:00","updated_at":"2026-07-11T23:40:46.424121+00:00","categories":[{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"vector-databases","name":"Vector Databases","url":"https://www.graphcanon.com/categories/vector-databases","markdown_url":"https://www.graphcanon.com/categories/vector-databases.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/vector-databases"},{"slug":"speech-audio","name":"Speech & Audio","url":"https://www.graphcanon.com/categories/speech-audio","markdown_url":"https://www.graphcanon.com/categories/speech-audio.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/speech-audio"}],"tags":[{"slug":"python","name":"python"}],"trust":{"provenance":{"is_fork":false,"github_id":740774265,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T23:40:37.435Z","maintenance":{"label":"Slowing","score":36,"methodology":"github_public_v1","releases_90d":0,"days_since_push":312,"last_release_at":null},"security_summary":{"status":"findings","scanner":"osv@v1","low_count":73,"high_count":0,"last_scan_at":"2026-07-11T23:40:37.952Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T23:40:37.160Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-07-11T23:40:37.160Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-07-11T23:40:37.160Z"}}}}