inference
Enrichment pendingSwap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready
GraphCanon updated today · GitHub synced today
Trust & integrity
Full report- Maintenance
- Very active (0d since push)
- As of today · Source: github_public_v1
- Provenance
- Not a fork · Organization account
- As of today · Source: github_public_v1
- Security (OSV)
- No lockfile
- As of today · Source: none
Public GitHub metadata and optional OSV dependency scans. Signals, not a guarantee. Trust methodology.
Overview
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.
Capability facts
- Languages
- python
Source: github.language+pyproject.toml · Jul 11, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Jul 11, 2026)
via the command line, or via the Xinference’s python client. Check out our [docs]( https://inference.readthedocs.io/en/latest/gettingSource link
Tags
README
Getting Started
Docker
Nvidia GPU users can start Xinference server using Xinference Docker Image. Prior to executing the installation command, ensure that both Docker and CUDA are set up on your system.
docker run --name xinference -d -p 9997:9997 -e XINFERENCE_HOME=/data -v </on/your/host>:/data --gpus all xprobe/xinference:latest xinference-local -H 0.0.0.0
install xinference
helm install xinference xinference/xinference -n xinference --version 0.0.1-v<xinference_release_version>
For more customized installation methods on K8s, please refer to the [documentation](https://inference.readthedocs.io/en/latest/getting_started/using_kubernetes.html).
---
### Quick Start
Install Xinference by using pip as follows. (For more options, see [Installation page](https://inference.readthedocs.io/en/latest/getting_started/installation.html).)
```bash
pip install "xinference[all]"
To start a local instance of Xinference, run the following command:
$ xinference-local
Once Xinference is running, there are multiple ways you can try it: via the web UI, via cURL, via the command line, or via the Xinference’s python client. Check out our docs for the guide.