sie
Open-source inference server and production cluster for all the models your agent needs.
GraphCanon updated 2d · GitHub synced 2d
Decision brief
sie is an open-source inference server and production cluster for managing AI model deployment in various domains like NLP, deep learning, and more.
Good fit when
- Use sie when you need to deploy multiple types of ML models including deep-learning embeddings or retrieval-augmented generation systems.
- Opt for sie if your project includes a variety of tasks such as natural language processing, semantic search, and requires efficient model inference.
Avoid when
- Avoid using sie if your project strictly focuses on areas outside the machine learning and deep-learning scope that sie is designed to support.
- Do not choose sie for projects requiring proprietary or specialized backend services that might conflict with its open-source framework.
- Requirements:
- sie operates under Python, necessitating a compatible runtime environment.; To fully leverage sie's capabilities, ensure your project aligns well with Apache-2.0 licensing requirements and practices.
Observed Jul 17, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Very active (0d since push)
- As of 2d
- Provenance
- Not a fork · Organization account
- As of 2d
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install sieSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
An open-source platform that provides services for model inference and management, facilitating deployment of various AI models in production environments.
Capability facts
- MCP server
- No MCP server detected
Source: repo_scan · Aug 22, 2026
- Languages
- python, javascript, typescript
Source: github.language+package.json+pyproject.toml · Aug 22, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Aug 22, 2026)
- Integrates with LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, Weaviate, and LanceDBSource link
Source: README excerpt (regex_v1, Aug 22, 2026)
The repository root is a virtual Python workspace. From the repository root,Source link
Tags
README
SIE: Superlinked Inference Engine
Self-hosted inference for agents. Every open model your agents call, served from one cluster in your cloud.
Docs | Quickstart | API Reference | Models
⭐ Help us reach more developers and grow the SIE community. Star this repo!
About
SIE is an open-source inference engine that runs the models behind every agent task through one API: search and retrieval, document-to-markdown conversion, structured output, content safety, and the agent loop itself. It replaces the patchwork of a separate model server per task with one system that serves 100+ models, loading each on demand.
- OpenAI-compatible API for drop-in migration:
/v1/embeddings,/v1/chat/completions,/v1/completions,/v1/responses - Pre-configured model catalog: Stella, SPLADE, Qwen3, GLiNER, SigLIP, and more; embedding and retrieval models benchmarked on MTEB
- Serves multiple models simultaneously with on-demand loading and LRU eviction
- Ships the full production stack: load-balancing gateway, KEDA autoscaling, Grafana dashboards, Terraform for GKE, EKS, and AKS
- Integrates with LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, Weaviate, and LanceDB
Development
The repository root is a virtual Python workspace. From the repository root, install and verify every workspace member with the committed lock (the audio-prep member requires its documented native build prerequisites):
uv python install 3.12
uv lock --check
uv sync --frozen --all-packages
uv run --frozen --project . --no-sync pytest -c pyproject.toml
Package membership is explicit in the root pyproject.toml; a package joins
the workspace only in the same change that adds its complete source.
Tasks
One SIE cluster runs the inference behind a whole agent. Each task is a handful of swappable models; browse packages/sie_server/models/ for the full set.
| Task | What it does | Models |
|---|---|---|
| Search | Embed, match, and rerank to retrieve the right context. | bge-m3, splade-v3, colbertv2, qwen3-reranker |
| Document to markdown | PDFs, Office files, and scans become clean markdown. | lightonocr, glm-ocr, mineru, paddleocr-vl, docling |
| Structured output | Schema-valid JSON, extracted or generated. | gliner2, nuner-zero, qwen3.6-27b |
| Guard content | A safety verdict with a probability you threshold. | granite-guardian-2b |
| Run the agent loop | Plan steps and call tools with an open LLM, streaming included. | qwen3.6-27b |
Quickstart
Prefer a notebook? examples/quickstart.ipynb runs this same flow, on your machine or a free Colab GPU.
1. Start the server
# macOS (Apple Silicon) or Linux, native (requires Python 3.12)
pip install "sie-server[local]" && sie-server serve
# Linux, NVIDIA GPU
docker run --gpus all -p 8080:8080 \
-v sie-hf-cache:/app/.cache/huggingface \
ghcr.io/superlinked/sie-server:latest-cuda12-default
# Linux, NVIDIA GPU — Transformers 5 OCR models (LightOnOCR and GLM-OCR)
docker run --gpus all -p 8080:8080 \
-v sie-hf-cache:/app/.cache/huggingface \
ghcr.io/superlinked/sie-server:latest-cuda12-transformers5
# Linux, CPU
docker run -p 8080:8080 \
-v sie-hf-cache:/app/.c
For agents
This page has a .md twin and JSON over the API.