Home/Inference & Serving/MInference

MInference

microsoft/MInference

Accelerates Long-context LLMs' inference through approximate sparse calculation for attention.

GitHub Website

GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026

44views this month

1.2k stars80 forksLast push Apr 8, 2026 Python MIT

Decision brief

MInference accelerates long-context LLMs' inference by up to 10x via approximate sparse calculation techniques while preserving model accuracy.

Good fit when

MInference is ideal for scenarios where significant reduction in inference latency is needed without sacrificing the accuracy of long-context LLM outputs.
MInference should be used when developers are working with models that support Transformers >= 4.46.0 and require fast pre-filling speeds to meet real-time performance requirements.

Avoid when

Avoid using MInference if your application does not benefit from or cannot tolerate slight variations in inference times due to its use of approximate sparse calculation.
MInference might not be suitable for applications where the model's accuracy is critical and any reduction in the precision introduced by approximations would be detrimental.

Requirements:: Min 8 GB RAM; MInference requires at least Torch and optionally FlashAttention-2 for maximum efficiency.; Triton for faster deployment and integration.

Observed Jul 12, 2026 · Source: enrich:decision_facts

MInference alternatives Trust report

Verify the decision

Maintenance and security

Full trust report

Maintenance: Slowing (151d since push); As of Sep 7, 2026
Provenance: Not a fork · Organization account; As of Sep 7, 2026
Security (OSV): No lockfile; As of Jul 11, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Backing

Company context for Microsoft. Display-only - separate from trust and ranking.

Company: Microsoft·GitHub org profile·Jul 11, 2026
Employees: 221,000·Wikidata (P1128 employees)·Jul 11, 2026
Commercial model: Pure OSS·GitHub org profile (public repos)·Jul 11, 2026

Install

pip install MInference

PyPI

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

flashinfer

FlashInfer is a kernel library for serving large language models

6.5k★

Awesome-LLM-Inference

A curated list of LLM/VLM inference papers with codes

5.5k★

ZhiLight

A highly optimized LLM inference acceleration engine for Llama and its variants.

908★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

MInference is a Python library that aims to reduce the latency of long-context Large Language Models (LLMs) by up to 10x during pre-filling, while maintaining accuracy. It leverages frameworks like Torch and FlashAttention-2 (optional), and is compatible with Transformers >= 4.46.0.

Capability facts

Languages: python
Source: github.language · Sep 7, 2026

Graph entities

Microsoftcompany

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Sep 7, 2026)

pip install minference

Source link

README

Requirements Torch FlashAttention 2 (Optional) Triton Transformers = 4.46.0 To get started with MInference, simply install it using pip:

For agents

This page has a .md twin and JSON over the API.

/tools/microsoft-minference.md GET /api/graphcanon/tools/microsoft-minference GET /api/graphcanon/graph?tool=microsoft-minference