GraphCanon
Home/Model Training/MInference
MInference logo

MInference

Enrichment pending
microsoft/MInference

[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling

GitHub Website

GraphCanon updated today · GitHub synced today

1.2k
Stars
78
Forks
93
Open issues
9
Watchers
3mo
Last push
Python MITCreated May 22, 2024

Trust & integrity

Full report
Maintenance
Slowing (94d since push)
As of today · Source: github_public_v1
Provenance
Not a fork · Organization account
As of today · Source: github_public_v1
Security (OSV)
No lockfile
As of today · Source: none

Public GitHub metadata and optional OSV dependency scans. Signals, not a guarantee. Trust methodology.

Backing

Company and funding context for Microsoft. Display-only - not part of trust score or organic ranking.

Company
Microsoft·GitHub org profile·today
Employees
221,000·Wikidata (P1128 employees)·today
Commercial model
Pure OSS·GitHub org profile (public repos)·today

Overview

[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

Capability facts

Languages
python

Source: github.language · Jul 11, 2026

Categories

Model TrainingLLM FrameworksInference & Serving

Graph entities

Microsoftcompany

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Jul 11, 2026)

pip install minference
Source link

Tags

python

README

Requirements

  • Torch
  • FlashAttention-2 (Optional)
  • Triton
  • Transformers >= 4.46.0

To get started with MInference, simply install it using pip:

pip install minference