vLLM/Recipes
OpenAI

openai/whisper-large-v3

Whisper Large v3 speech recognition model, validated for Intel Xeon 6 CPU serving.

Audio transcription validated on Intel Xeon 6

View on HuggingFace
dense1.55B448 ctxvLLM 0.7.0+multimodal
Guide

Overview

Whisper Large v3 is OpenAI's speech recognition model. vLLM exposes transcription through the OpenAI-compatible audio transcription endpoint.

Prerequisites

  • Hardware: Intel Xeon 6/Xeon 5 CPUs
  • vLLM >= 0.7.0

Docker (Intel Xeon 6 CPUs)

docker pull vllm/vllm-openai-cpu:latest-x86_64

Intel Xeon 6

vllm serve openai/whisper-large-v3 \
  --tensor-parallel-size 1

Docker (the image entrypoint is vllm serve):

docker run \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai-cpu:latest-x86_64 openai/whisper-large-v3 \
  --tensor-parallel-size 1

The example uses TP=1. Adjust TP/DP for the deployment topology.

Runtime and Platform Tuning

The following settings are intentionally not prescribed as portable CPU model defaults because they depend on the workload, hardware, or runtime environment:

  • --max-num-batched-tokens: scheduler/throughput tuning.
  • --max-num-seqs: concurrency and scheduler-capacity tuning.
  • --gpu-memory-utilization: platform memory-budget tuning.
  • --no-enable-prefix-caching: workload/benchmark cache-policy tuning.
  • VLLM_ENGINE_ITERATION_TIMEOUT_S: operational runtime timeout.

These settings may still appear in validated hardware-specific overrides. Tune them at deployment time based on platform resources, workload shape, and latency/throughput goals.

References