{"data":{"slug":"superlinked-sie","name":"sie","tagline":"Open-source inference server and production cluster for all the models your agent needs.","github_url":"https://github.com/superlinked/sie","owner":"superlinked","repo":"sie","owner_avatar_url":"https://avatars.githubusercontent.com/u/94243920?v=4","primary_language":"Python","stars":2804,"forks":272,"topics":["bge","colbert","data-pipeline","deep-learning","embeddings","inference","inference-server","information-retrieval","llm","ml","mlops","natural-language-processing","nlp","python","reranking","retrieval","retrieval-augmented-generation","semantic-search","splade","vector-search"],"archived":false,"github_pushed_at":"2026-08-21T20:28:04+00:00","maintenance_label":"Very active","stars_delta_30d":507,"url":"https://www.graphcanon.com/tools/superlinked-sie","markdown_url":"https://www.graphcanon.com/tools/superlinked-sie.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/superlinked-sie","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=superlinked-sie","description":"Open-source inference server and production cluster for all the models your agent needs.","homepage_url":"https://superlinked.com","license":"Apache-2.0","open_issues":13,"watchers":43,"ai_summary":"An open-source platform that provides services for model inference and management, facilitating deployment of various AI models in production environments.","readme_excerpt":"
\n\n\n \n \"Superlinked\n\n\n

SIE: Superlinked Inference Engine

\n\n

Self-hosted inference for agents. Every open model your agents call, served from one cluster in your cloud.

\n\n

\n Docs |\n Quickstart |\n API Reference |\n Models\n

\n\n\n\n\n\n⭐ _Help us reach more developers and grow the SIE community. Star this repo!_\n\n
\n\n## About\n\nSIE is an open-source inference engine that runs the models behind every agent task through one API: search and retrieval, document-to-markdown conversion, structured output, content safety, and the agent loop itself. It replaces the patchwork of a separate model server per task with one system that serves 100+ models, loading each on demand.\n\n- OpenAI-compatible API for drop-in migration: `/v1/embeddings`, `/v1/chat/completions`, `/v1/completions`, `/v1/responses`\n- Pre-configured model catalog: Stella, SPLADE, Qwen3, GLiNER, SigLIP, and more; embedding and retrieval models benchmarked on MTEB\n- Serves multiple models simultaneously with on-demand loading and LRU eviction\n- Ships the full production stack: load-balancing gateway, KEDA autoscaling, Grafana dashboards, Terraform for GKE, EKS, and AKS\n- Integrates with LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, Weaviate, and LanceDB\n\n## Development\n\nThe repository root is a virtual Python workspace. From the repository root,\ninstall and verify every workspace member with the committed lock (the\naudio-prep member requires its documented native build prerequisites):\n\n```bash\nuv python install 3.12\nuv lock --check\nuv sync --frozen --all-packages\nuv run --frozen --project . --no-sync pytest -c pyproject.toml\n```\n\nPackage membership is explicit in the root `pyproject.toml`; a package joins\nthe workspace only in the same change that adds its complete source.\n\n## Tasks\n\nOne SIE cluster runs the inference behind a whole agent. Each task is a handful of swappable models; browse [`packages/sie_server/models/`](https://github.com/superlinked/sie/tree/main/packages/sie_server/models) for the full set.\n\n| Task | What it does | Models |\n|---|---|---|\n| **Search** | Embed, match, and rerank to retrieve the right context. | `bge-m3`, `splade-v3`, `colbertv2`, `qwen3-reranker` |\n| **Document to markdown** | PDFs, Office files, and scans become clean markdown. | `lightonocr`, `glm-ocr`, `mineru`, `paddleocr-vl`, `docling` |\n| **Structured output** | Schema-valid JSON, extracted or generated. | `gliner2`, `nuner-zero`, `qwen3.6-27b` |\n| **Guard content** | A safety verdict with a probability you threshold. | `granite-guardian-2b` |\n| **Run the agent loop** | Plan steps and call tools with an open LLM, streaming included. | `qwen3.6-27b` |\n\n## Quickstart\n\nPrefer a notebook? [`examples/quickstart.ipynb`](examples/quickstart.ipynb) runs this same flow, on your machine or a free Colab GPU.\n\n**1. Start the server**\n\n```bash\n# macOS (Apple Silicon) or Linux, native (requires Python 3.12)\npip install \"sie-server[local]\" && sie-server serve\n\n# Linux, NVIDIA GPU\ndocker run --gpus all -p 8080:8080 \\\n -v sie-hf-cache:/app/.cache/huggingface \\\n ghcr.io/superlinked/sie-server:latest-cuda12-default\n\n# Linux, NVIDIA GPU — Transformers 5 OCR models (LightOnOCR and GLM-OCR)\ndocker run --gpus all -p 8080:8080 \\\n -v sie-hf-cache:/app/.cache/huggingface \\\n ghcr.io/superlinked/sie-server:latest-cuda12-transformers5\n\n# Linux, CPU\ndocker run -p 8080:8080 \\\n -v sie-hf-cache:/app/.c","github_created_at":"2023-11-07T10:45:53+00:00","created_at":"2026-07-11T11:29:38.593273+00:00","updated_at":"2026-08-22T06:01:20.970538+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"}],"tags":[{"slug":"bge","name":"bge"},{"slug":"colbert","name":"colbert"},{"slug":"data-pipeline","name":"data-pipeline"},{"slug":"deep-learning","name":"deep-learning"},{"slug":"embeddings","name":"embeddings"},{"slug":"inference","name":"inference"},{"slug":"information-retrieval","name":"information-retrieval"},{"slug":"llm","name":"llm"}],"trust":{"provenance":{"is_fork":false,"github_id":715536935,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-22T06:01:20.195Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":30,"days_since_push":0,"last_release_at":"2026-08-09T06:03:48Z","stars_delta_30d":507,"open_issues_delta_30d":2},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T11:29:39.773Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"mcp":{"source":"repo_scan","observed_at":"2026-08-22T06:01:20.655Z","server_manifest":false},"scan":{"source":"repo_scan","observed_at":"2026-08-22T06:01:20.655Z"},"languages":{"value":["python","javascript","typescript"],"source":"github.language+package.json+pyproject.toml","observed_at":"2026-08-22T06:01:20.655Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-08-22T06:01:20.655Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["sie operates under Python, necessitating a compatible runtime environment.","To fully leverage sie's capabilities, ensure your project aligns well with Apache-2.0 licensing requirements and practices."]},"constraints":null,"when_to_use":["Use sie when you need to deploy multiple types of ML models including deep-learning embeddings or retrieval-augmented generation systems.","Opt for sie if your project includes a variety of tasks such as natural language processing, semantic search, and requires efficient model inference."],"when_not_to_use":["Avoid using sie if your project strictly focuses on areas outside the machine learning and deep-learning scope that sie is designed to support.","Do not choose sie for projects requiring proprietary or specialized backend services that might conflict with its open-source framework."],"source":"enrich:decision_facts","observed_at":"2026-07-17T04:17:33.421Z"},"constraint_facets":null,"decision_summary":[{"label":"Requirements","value":"sie operates under Python, necessitating a compatible runtime environment.; To fully leverage sie's capabilities, ensure your project aligns well with Apache-2.0 licensing requirements and practices."},{"label":"Adopt for","value":"sie is an open-source inference server and production cluster for managing AI model deployment in various domains like NLP, deep learning, and more."}]}}