WhisperLive

Installation

• Install PortAudio (required system dependency for microphone input via PyAudio)

 bash scripts/setup.sh

On Debian/Ubuntu this installs portaudio19-dev, on Fedora portaudio-devel, on macOS it uses Homebrew (portaudio).

• Install 3.12 venv (on Fedora sudo dnf install -y python3.12 python3.12-pip)

python3.12 -m venv whisper_env
source whisper_env/bin/activate

• Install whisper-live from pip

 pip install whisper-live

---

Getting Started

The server supports 3 backends faster_whisper, tensorrt and openvino. If running tensorrt backend follow TensorRT_whisper readme

---

Whisper Live Server in Docker

• GPU

  • Faster-Whisper

  docker run -it --gpus all -p 9090:9090 ghcr.io/collabora/whisperlive-gpu:latest
  

  • TensorRT. Refer to TensorRT_whisper readme for setup and more tensorrt backend configurations.

  docker build . -f docker/Dockerfile.tensorrt -t whisperlive-tensorrt
  docker run -p 9090:9090 --runtime=nvidia --gpus all --entrypoint /bin/bash -it whisperlive-tensorrt
  
  # Build small.en engine
  bash build_whisper_tensorrt.sh /app/TensorRT-LLM-examples small.en        # float16
  bash build_whisper_tensorrt.sh /app/TensorRT-LLM-examples small.en int8   # int8 weight only quantization
  bash build_whisper_tensorrt.sh /app/TensorRT-LLM-examples small.en int4   # int4 weight only quantization
  
  # Run server with small.en (pick one engine)
  python3 run_server.py --port 9090 \
                        --backend tensorrt \
                        --trt_model_path "/app/TensorRT-LLM-examples/whisper/whisper_small_en_float16"
  # or int8 / int4:
  # --trt_model_path "/app/TensorRT-LLM-examples/whisper/whisper_small_en_int8"
  # --trt_model_path "/app/TensorRT-LLM-examples/whisper/whisper_small_en_int4"
  

  • OpenVINO

  docker run -it --device=/dev/dri -p 9090:9090 ghcr.io/collabora/whisperlive-openvino
  

• CPU

  • Faster-whisper

  docker run -it -p 9090:9090 ghcr.io/collabora/whisperlive-cpu:latest