{"data":{"slug":"cnut1648-model-fingerprint","name":"Model-Fingerprint","tagline":"Fingerprint large language models","github_url":"https://github.com/cnut1648/Model-Fingerprint","owner":"cnut1648","repo":"Model-Fingerprint","owner_avatar_url":"https://avatars.githubusercontent.com/u/37067883?v=4","primary_language":"Python","stars":51,"forks":8,"topics":[],"archived":false,"github_pushed_at":"2024-07-11T04:21:11+00:00","maintenance_label":"Dormant","stars_delta_30d":0,"url":"https://www.graphcanon.com/tools/cnut1648-model-fingerprint","markdown_url":"https://www.graphcanon.com/tools/cnut1648-model-fingerprint.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/cnut1648-model-fingerprint","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=cnut1648-model-fingerprint","description":"Fingerprint large language models","homepage_url":null,"license":"MIT","open_issues":5,"watchers":2,"ai_summary":"A toolset for creating instructional fingerprints of large language models using CUDA 11.3 and PyTorch 2.0.","readme_excerpt":"# Instructional Fingerprinting\n\n\n\n\n
\n\n\nThis project is developed using CUDA 11.3, PyTorch 2.0, python 3.9.\n\nAfter installing a GPU version of PyTorch, other dependencies can be installed via `pip install -r requirements.txt`.\n\n## Dataset\n\n### Fingerprint dataset\n\nTo construct instructional fingerprint data (Section 3.1-3.2):\n\n- For Simple Template (Figure 3), simply run `python create_fingerprint_mix.py`.\n\nThis script will print each instance of the dataset, and save to `dataset/llama_fingerprint_mix` folder. \n\n- For Dialogue Template (Figure 4), simply run `python create_fingerprint_chat.py`.\n\nThis script will print each instance of the dataset, and save to `dataset/llama_fingerprint_chat` folder. \n\n### Downstream dataset\nWe explore six downstream datasets. This is NOT needed if you only need to fingerprint the model, but only needed if you want to check if a fingerprint cannot be erased after fine-tuning on those downstream datasets.\n\nAlpaca 52k is in [Alpaca](https://github.com/tatsu-lab/stanford_alpaca) repo already. For the rest of dataset:\n```shell\npython prepare_ni.py # natural instruction v2\npython prepare_dolly.py # dolly\npython prepare_sharegpt.py # share GPT\n```\n`Alpaca-GPT4` can be downloaded in [their repo](https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM#data-release); for Vicuna experiment, first download `ShareGPT_V3_unfiltered_clean_split_no_imsorry.json` from [here](https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/tree/main) and use Vicuna's [offical processing script](https://github.com/lm-sys/FastChat/blob/main/docs/commands/data_cleaning.md) to generate the dataset.\n```shell\n# Convert html to markdown\npython3 -m fastchat.data.clean_sharegpt --in ShareGPT_V3_unfiltered_clean_split_no_imsorry.json --out sharegpt_clean.json\n```\nNote that we do not remove specific language, so this is a multilingual dataset.\n\nThe processing script is borrowed from [LLM-Blender](https://arxiv.org/abs/2306.02561).\n\n## Model Fingerprinting\n\nWe have `pipeline_SFT_chat.py` and `pipeline_adapter.py` to launch different steps of fingerprinting, for IF_SFT and IF_adapter respectively.\nThe CLI are the same for both, and we use `pipeline_adapter.py` as an example.\n\nAll fingerprinted models are hosted on huggingface ([IF_adapter](https://huggingface.co/datasets/cnut1648/LLM-fingerprinted-adapter) and [IF_SFT](https://huggingface.co/datasets/cnut1648/LLM-fingerprinted-SFT)) and you can download all of them together with [output files](#various-saved-outputs) (note this is VERY large) via\n```shell\ngit clone https://huggingface.co/datasets/cnut1648/LLM-fingerprinted-adapter output_barebone_adapter\ngit clone https://huggingface.co/datasets/cnut1648/LLM-fingerprinted-SFT output_barebone_sft_chat\n```\n\nWe also provide some of the models in these folders and people can test if the fingerprinted model has the same behavior as described in the paper.\n\n| Model | Fingerprinted Model (Adapter) | User Model Trained on AlpacaGPT4 (Adapter) | Fingerprinted Model (SFT) | User Model Trained on AlpacaGPT4 (SFT) | \n|------------|---------------------|----------------","github_created_at":"2023-10-04T17:24:38+00:00","created_at":"2026-07-11T23:41:12.327536+00:00","updated_at":"2026-10-05T06:00:38.822733+00:00","categories":[{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"}],"tags":[{"slug":"fingerprinting","name":"fingerprinting"},{"slug":"large-language-models","name":"large-language-models"},{"slug":"pytorch","name":"pytorch"}],"trust":{"provenance":{"is_fork":false,"github_id":700485449,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-10-05T06:00:37.073Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":816,"last_release_at":null,"stars_delta_30d":0,"open_issues_delta_30d":0},"security_summary":{"status":"ok","scanner":"osv@v1","low_count":0,"high_count":0,"last_scan_at":"2026-07-11T23:41:13.846Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-10-05T06:00:37.543Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-10-05T06:00:37.543Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-10-05T06:00:37.543Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["Use Model-Fingerprint when you need to fingerprint large language models for evaluation or observability purposes, especially in research contexts involving CUDA 11.3 and PyTorch 2.0 environments.","Consider this toolset if your project involves creating fingerprints based on specific templates like Simple Template or Dialogue Template to analyze model behavior."],"when_not_to_use":["Do not use Model-Fingerprint if your development environment does not support CUDA 11.3 and PyTorch 2.0, as it may lead to incompatibility issues.","Avoid this toolset if you need a solution that supports multiple versions of CUDA or Pytorch for flexibility across different hardware configurations without modification."],"source":"enrich:decision_facts","observed_at":"2026-07-17T13:14:07.972Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"Model-Fingerprint is a toolset for creating instructional fingerprints of large language models using CUDA 11.3 and PyTorch 2.0."}]}}