mosec

Installation

Mosec requires Python 3.7 or above. Install the latest PyPI package for Linux or macOS with:

pip install -U mosec

---

# or install with conda
conda install conda-forge::mosec

---

# or install with pixi
pixi add mosec

To build from the source code, install Rust and run the following command:

make package

You will get a mosec wheel file in the dist folder.

Deployment

• If you're looking for a GPU base image with mosec installed, you can check the official image mosecorg/mosec. For the complex use case, check out envd.
• This service doesn't need Gunicorn or NGINX, but you can certainly use the ingress controller when necessary.
• This service should be the PID 1 process in the container since it controls multiple processes. If you need to run multiple processes in one container, you will need a supervisor. You may choose Supervisor or Horust.
• Remember to collect the metrics.
  • mosec_service_batch_size_bucket shows the batch size distribution.
  • mosec_service_batch_duration_second_bucket shows the duration of dynamic batching for each connection in each stage (starts from receiving the first task).
  • mosec_service_process_duration_second_bucket shows the duration of processing for each connection in each stage (including the IPC time but excluding the mosec_service_batch_duration_second_bucket).
  • mosec_service_remaining_task shows the number of currently processing tasks.
  • mosec_service_throughput shows the service throughput.
• Stop the service with SIGINT (CTRL+C) or SIGTERM (kill {PID}) since it has the graceful shutdown logic.