{"data":{"slug":"feifeibear-long-context-attention","name":"long-context-attention","tagline":"Unified Sequence Parallel Attention for Long Context Transformers","github_url":"https://github.com/feifeibear/long-context-attention","owner":"feifeibear","repo":"long-context-attention","owner_avatar_url":"https://avatars.githubusercontent.com/u/5706969?v=4","primary_language":"Python","stars":687,"forks":83,"topics":["attention-is-all-you-need","deepspeed-ulysses","llm-inference","llm-training","pytorch","ring-attention"],"archived":false,"github_pushed_at":"2026-05-21T06:53:42+00:00","maintenance_label":"Slowing","stars_delta_30d":5,"url":"https://www.graphcanon.com/tools/feifeibear-long-context-attention","markdown_url":"https://www.graphcanon.com/tools/feifeibear-long-context-attention.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/feifeibear-long-context-attention","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=feifeibear-long-context-attention","description":"USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference","homepage_url":null,"license":"Apache-2.0","open_issues":13,"watchers":5,"ai_summary":"Provides support for long context transformers using Hybrid and 2D sequence parallel attention techniques designed to enhance both training and inference processes.","readme_excerpt":"### 1. Installation\n\nFlashAttention is the most important external dependency and is often the cause of errors when installing and using yunchang. \nYunchang supports flash_attn 2.6.x and 2.7.x, both v3 and v2 versions. Additionally, yunchang supports runs without flash_attn, which is suitable for NPUs.\n\nAs shown in the figure below, there are three usage methods based on the flash_attn situation:\n\n1. For H100, B100, hardware that supports FA v3, ring_flash_attn uses FA v3.\n\n2. For A100, L40, hardware that supports FA v2, ring_flash_attn uses FA v2.\n\n3. For hardware such as NPUs that does not support FA, use torch to implement attention computation. In this case, there is no need to install `flash_attn`, and you should apply `LongContextAttention(ring_impl_type=\"basic\", attn_type=AttnType.TORCH_EFFICIENT)`. *Note: the backward pass is not supported for AttnType.TORCH_EFFICIENT.*\n\nOption 1: pip install\n\n`pip install flash-attn`\n\n`pip install yunchang`\n\n#### Apply FlashAttention V3: Since FA V3 is beta-released, you need to install FlashAttention V3 from source code.\n\nFollow the [FlashAttention beta-release](https://github.com/Dao-AILab/flash-attention?tab=readme-ov-file#flashattention-3-beta-release) to install V3 for NVIDIA Hopper GPUs.\n\nWe applied the Nov 10 2024 commit `b443207c1fc4c98e4532aad4e88cfee1d590d996`.\n\n\nOption 2: build from local.\n\n`pip install .`\n\nInstall for AMD GPU: [install_amd.md](./docs/install_amd.md)","github_created_at":"2024-03-27T12:33:12+00:00","created_at":"2026-07-11T11:44:48.788933+00:00","updated_at":"2026-08-25T06:01:49.142539+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"attention-is-all-you-need","name":"attention-is-all-you-need"},{"slug":"deepspeed-ulysses","name":"deepspeed-ulysses"},{"slug":"llm-inference","name":"llm-inference"},{"slug":"llm-training","name":"llm-training"},{"slug":"pytorch","name":"pytorch"},{"slug":"ring-attention","name":"ring-attention"}],"trust":{"provenance":{"is_fork":false,"github_id":778281287,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-25T06:01:48.386Z","maintenance":{"label":"Slowing","score":36,"methodology":"github_public_v1","releases_90d":0,"days_since_push":95,"last_release_at":"2026-01-13T08:13:49Z","stars_delta_30d":5,"open_issues_delta_30d":0},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T11:44:50.476Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-08-25T06:01:48.825Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-08-25T06:01:48.825Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-08-25T06:01:48.825Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["When developing models that require handling longer input sequences where traditional attention mechanisms face scalability issues.","During scenarios where enhancing the processing efficiency during both training phases and at inference time is critical."],"when_not_to_use":["If your use case involves short context lengths where standard attention mechanisms suffice and adding long-context-attention doesn't provide significant benefits.","When working in environments that do not support Python, as this tool is specifically developed for the Python ecosystem."],"source":"enrich:decision_facts","observed_at":"2026-07-17T03:18:39.225Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"long-context-attention uses Unified Sequence Parallel Attention techniques to improve performance of long context transformers for both training and inference."}]}}