Home/Inference & Serving/long-context-attention

long-context-attention

feifeibear/long-context-attention

Unified Sequence Parallel Attention for Long Context Transformers

GitHub

GraphCanon updated 3w · GitHub synced 3w

682 stars81 forksLast push 3mo Python Apache-2.0

Decision brief

long-context-attention uses Unified Sequence Parallel Attention techniques to improve performance of long context transformers for both training and inference.

Good fit when

When developing models that require handling longer input sequences where traditional attention mechanisms face scalability issues.
During scenarios where enhancing the processing efficiency during both training phases and at inference time is critical.

Avoid when

If your use case involves short context lengths where standard attention mechanisms suffice and adding long-context-attention doesn't provide significant benefits.
When working in environments that do not support Python, as this tool is specifically developed for the Python ecosystem.

Observed Jul 17, 2026 · Source: enrich:decision_facts

long-context-attention alternatives Trust report

Verify the decision

Maintenance and security

Full trust report

Maintenance: Steady (65d since push); As of 3w
Provenance: Not a fork · Personal account; As of 3w
Security (OSV): No lockfile; As of 1mo

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

pip install long-context-attention

PyPI

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

peft

State-of-the-art Parameter-Efficient Fine-Tuning

21k★

train-llm-from-scratch

A straightforward method for training your LLM from raw text to aligned model generation

9.1k★

bitsandbytes

Large language model quantization toolkit for PyTorch.

8.4k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

Provides support for long context transformers using Hybrid and 2D sequence parallel attention techniques designed to enhance both training and inference processes.

Capability facts

Languages: python
Source: github.language+pyproject.toml · Jul 25, 2026

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Jul 25, 2026)

Option 1: pip install

Source link

README

1. Installation

FlashAttention is the most important external dependency and is often the cause of errors when installing and using yunchang. Yunchang supports flash_attn 2.6.x and 2.7.x, both v3 and v2 versions. Additionally, yunchang supports runs without flash_attn, which is suitable for NPUs.

As shown in the figure below, there are three usage methods based on the flash_attn situation:

For H100, B100, hardware that supports FA v3, ring_flash_attn uses FA v3.
For A100, L40, hardware that supports FA v2, ring_flash_attn uses FA v2.
For hardware such as NPUs that does not support FA, use torch to implement attention computation. In this case, there is no need to install flash_attn, and you should apply LongContextAttention(ring_impl_type="basic", attn_type=AttnType.TORCH_EFFICIENT). Note: the backward pass is not supported for AttnType.TORCH_EFFICIENT.

Option 1: pip install

pip install flash-attn

pip install yunchang

Apply FlashAttention V3: Since FA V3 is beta-released, you need to install FlashAttention V3 from source code.

Follow the FlashAttention beta-release to install V3 for NVIDIA Hopper GPUs.

We applied the Nov 10 2024 commit b443207c1fc4c98e4532aad4e88cfee1d590d996.

Option 2: build from local.

pip install .

Install for AMD GPU: install_amd.md

For agents

This page has a .md twin and JSON over the API.

/tools/feifeibear-long-context-attention.md GET /api/graphcanon/tools/feifeibear-long-context-attention GET /api/graphcanon/graph?tool=feifeibear-long-context-attention