{"data":{"slug":"euanong-image-hijacks","name":"image-hijacks","tagline":"Official codebase for Image Hijacks: Adversarial Images can Control Generative Models at Runtime","github_url":"https://github.com/euanong/image-hijacks","owner":"euanong","repo":"image-hijacks","owner_avatar_url":"https://avatars.githubusercontent.com/u/8283298?v=4","primary_language":"Python","stars":56,"forks":12,"topics":[],"archived":false,"github_pushed_at":"2023-09-19T20:28:31+00:00","maintenance_label":"Dormant","url":"https://www.graphcanon.com/tools/euanong-image-hijacks","markdown_url":"https://www.graphcanon.com/tools/euanong-image-hijacks.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/euanong-image-hijacks","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=euanong-image-hijacks","description":"Official codebase for Image Hijacks: Adversarial Images can Control Generative Models at Runtime","homepage_url":"https://image-hijacks.github.io/","license":"MIT","open_issues":8,"watchers":2,"ai_summary":null,"readme_excerpt":"# Image Hijacks: Adversarial Images can Control Generative Models at Runtime\n\nThis is the code for _Image Hijacks: Adversarial Images can Control Generative Models at Runtime_.\n\n- [Project page and demo](https://image-hijacks.github.io)\n- [Paper](https://arxiv.org/abs/2309.00236)\n\n## Setup\n\nThe code can be run under any environment with Python 3.9 and above. \n\nWe use [poetry](https://python-poetry.org) for dependency management, which can be installed following the instructions [here](https://python-poetry.org/docs/#installation).\n\nTo build a virtual environment with the required packages, simply run\n\n```bash\npoetry install\n```\n\nNotes\n- On some systems you may need to set the environment variable `PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring` to avoid keyring-based errors.\n- This codebase stores large files (e.g. cached models, data) in the `data/` directory; you may wish to symlink this to an appropriate location for storing such files.\n\n## Training\n\nThe images used in our [demo](https://image-hijacks.github.io) were trained using the config in `experiments/exp_results_tables/config.py` (specifically runs #1 `llava1_att_leak.pat_full.eps_8.lr_3e-2` and #5 `llava1_att_spec.pat_full.eps_8.lr_3e-2`).\n\nTo train these images, first download the relevant LLaVA checkpoint:\n\n```bash\npoetry run python download.py models llava-v1.3-13b-336px\n```\n\nTo get the list of jobs (with their job IDs) specified by this config file:\n\n```bash\npoetry run python experiments/exp_demo_imgs/config.py\n```\n\nTo run job ID `N` without [wandb](https://wandb.ai/) logging:\n\n```bash\npoetry run python run.py train \\\n--config_path experiments/exp_demo_imgs/config.py \\\n--log_dir experiments/exp_demo_imgs/logs \\\n--job_id N \\\n--playground\n```\n\nTo run job ID `N` with [wandb](https://wandb.ai/) logging to `YOUR_WANDB_ENTITY/YOUR_WANDB_PROJECT`:\n\n```bash\npoetry run python run.py train \\\n--config_path experiments/exp_results_tables/config.py \\\n--log_dir experiments/exp_results_tables/logs \\\n--job_id N \\\n--wandb_entity YOUR_WANDB_ENTITY \\\n--wandb_project YOUR_WANDB_PROJECT \\\n--no-playground\n```\n\nNotes: \n- In order to run jailbreak experiments (configurations coming soon), you must store your OpenAI API key in the `OPENAI_API_KEY` environment variable.\n\n## Tests\n\nThis codebase advocates for [expect tests](https://blog.janestreet.com/the-joy-of-expect-tests) in machine learning, and as such uses @ezyang's [expecttest](https://github.com/ezyang/expecttest) library for unit and regression tests.\n\nTo run tests,\n\n```bash\npoetry run python download.py models blip2-flan-t5-xl\npoetry run pytest .\n```\n\n## Citation\n\nTo cite our work, you can use the following BibTeX entry:\n\n```bibtex\n@misc{bailey2023image,\n  title={Image Hijacks: Adversarial Images can Control Generative Models at Runtime}, \n  author={Luke Bailey and Euan Ong and Stuart Russell and Scott Emmons},\n  year={2023},\n  eprint={2309.00236},\n  archivePrefix={arXiv},\n  primaryClass={cs.LG}\n}\n```","github_created_at":"2023-08-31T23:25:03+00:00","created_at":"2026-07-11T23:39:48.279205+00:00","updated_at":"2026-07-11T23:40:00.692326+00:00","categories":[{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"},{"slug":"computer-vision","name":"Computer Vision","url":"https://www.graphcanon.com/categories/computer-vision","markdown_url":"https://www.graphcanon.com/categories/computer-vision.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/computer-vision"},{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"}],"tags":[{"slug":"python","name":"python"}],"trust":{"provenance":{"is_fork":false,"github_id":685748314,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T23:39:54.147Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":1026,"last_release_at":null},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T23:39:54.603Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T23:39:53.868Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-07-11T23:39:53.868Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-07-11T23:39:53.868Z"}}}}