{"data":{"slug":"thudm-longcite","name":"LongCite","tagline":"Enabling LLMs to Generate Fine-grained Citations in Long-context QA","github_url":"https://github.com/THUDM/LongCite","owner":"THUDM","repo":"LongCite","owner_avatar_url":"https://avatars.githubusercontent.com/u/48590610?v=4","primary_language":"Python","stars":522,"forks":30,"topics":["benchmark","citation-generation","fine-tuning","llm","long-context"],"archived":false,"github_pushed_at":"2024-12-31T03:25:42+00:00","maintenance_label":"Dormant","stars_delta_30d":2,"url":"https://www.graphcanon.com/tools/thudm-longcite","markdown_url":"https://www.graphcanon.com/tools/thudm-longcite.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/thudm-longcite","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=thudm-longcite","description":"LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA","homepage_url":null,"license":"Apache-2.0","open_issues":9,"watchers":10,"ai_summary":"LongCite focuses on enabling large language models to generate fine-grained citations in responses to queries with long context.","readme_excerpt":"## ⚙️ LongCite Deployment\n\n**Environmental Setup**:\nWe recommend using `transformers>=4.43.0` to successfully deploy our models.\n\nWe open-source two models: [LongCite-glm4-9b](https://huggingface.co/THUDM/LongCite-glm4-9b) and [LongCite-llama3.1-8b](https://huggingface.co/THUDM/LongCite-llama3.1-8b), which are trained based on [GLM-4-9B](https://huggingface.co/THUDM/glm-4-9b) and [Meta-Llama-3.1-8B](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B), respectively, and support up to 128K context. These two models point to the \"LongCite-9B\" and \"LongCite-8B\" models in our paper. Given a long-context-based query, these models can generate accurate responses and precise sentence-level citations, making it easy for users to verify the output information. Try the model:\n```python\nimport json\nimport torch\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\n\ntokenizer = AutoTokenizer.from_pretrained('THUDM/LongCite-glm4-9b', trust_remote_code=True)\nmodel = AutoModelForCausalLM.from_pretrained('THUDM/LongCite-glm4-9b', torch_dtype=torch.bfloat16, trust_remote_code=True, device_map='auto')\n\ncontext = '''\nW. Russell Todd, 94, United States Army general (b. 1928). February 13. Tim Aymar, 59, heavy metal singer (Pharaoh) (b. 1963). Marshall \\\"Eddie\\\" Conway, 76, Black Panther Party leader (b. 1946). Roger Bonk, 78, football player (North Dakota Fighting Sioux, Winnipeg Blue Bombers) (b. 1944). Conrad Dobler, 72, football player (St. Louis Cardinals, New Orleans Saints, Buffalo Bills) (b. 1950). Brian DuBois, 55, baseball player (Detroit Tigers) (b. 1967). Robert Geddes, 99, architect, dean of the Princeton University School of Architecture (1965–1982) (b. 1923). Tom Luddy, 79, film producer (Barfly, The Secret Garden), co-founder of the Telluride Film Festival (b. 1943). David Singmaster, 84, mathematician (b. 1938).\n'''\nquery = \"What was Robert Geddes' profession?\"\nresult = model.query_longcite(context, query, tokenizer=tokenizer, max_input_length=128000, max_new_tokens=1024)\n\nprint(\"Answer:\\n{}\\n\".format(result['answer']))\nprint(\"Statement with citations:\\n{}\\n\".format(\n  json.dumps(result['statements_with_citations'], indent=2, ensure_ascii=False)))\nprint(\"Context (divided into sentences):\\n{}\\n\".format(result['splited_context']))\n```\nYou may deploy your own LongCite chatbot (like the one we show in the above video) by running\n```\nCUDA_VISIBLE_DEVICES=0 streamlit run demo.py --server.fileWatcherType none\n```\nAlternatively, you can deploy the model with [vllm](https://github.com/vllm-project/vllm), which allows faster generation and multiconcurrent server. See the code example in [vllm_inference.py](https://github.com/THUDM/LongCite/blob/main/vllm_inference.py).\n\n<a name=\"pipeline\"></a>","github_created_at":"2024-08-31T13:53:10+00:00","created_at":"2026-07-11T11:41:08.848582+00:00","updated_at":"2026-09-20T06:31:24.679748+00:00","categories":[{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"}],"tags":[{"slug":"benchmark","name":"benchmark"},{"slug":"citation-generation","name":"citation-generation"},{"slug":"fine-tuning","name":"fine-tuning"},{"slug":"long-context","name":"long-context"}],"trust":{"provenance":{"is_fork":false,"github_id":850338335,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-19T18:27:30.315Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":627,"last_release_at":null,"stars_delta_30d":2,"open_issues_delta_30d":0},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T11:41:10.903Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-19T18:27:31.316Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-09-19T18:27:31.316Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-09-19T18:27:31.316Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["When you require your LLM to provide detailed, well-cited responses in long-context scenarios.","If your application demands accurate and granular attribution for information derived from extensive text bodies."],"when_not_to_use":["If your use case involves short queries or contexts that do not need extensive citations.","When the primary focus is on speed rather than detailed citation accuracy in responses."],"source":"enrich:decision_facts","observed_at":"2026-07-16T20:51:06.201Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"LongCite is designed to enhance large language models by enabling them to generate fine-grained citations when answering queries with long context."}]}}