vLLM Cheatsheet - High-Performance LLM Inference & Serving

All vLLM inference & serving commands in one place — from install to OpenAI-compatible server, tensor parallelism to quantization — organized by scenario. Copy and go.

AI CLI·25 commands·Last updated 2026-08-23

Install 5

pip install vllm

Install vLLM (includes the vllm CLI)

pip install vllm --upgrade

Upgrade to the latest

vllm --version

Show installed version

python -c "import vllm; print(vllm.__version__)"

Confirm the package loads

vllm --help

Show all subcommands

OpenAI-compatible Server 5

vllm serve facebook/opt-125m

Start server (default :8000, OpenAI protocol)

vllm serve <model> --host 0.0.0.0 --port 8080

Set listen address and port

vllm serve <model> --api-key <key>

Enable API-key check (or VLLM_API_KEY env)

vllm serve <model> --tensor-parallel-size 2

Multi-GPU tensor parallelism (TP)

vllm serve <model> --gpu-memory-utilization 0.9

Raise VRAM util to fit larger models

Client Calls 5

curl http://localhost:8000/v1/models

List loaded models

curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"...","messages":[...]}'

Send a chat request

openai_api_base=http://localhost:8000/v1

Point OpenAI SDK at local vLLM

vllm serve <model> --max-model-len 8192

Cap context length to save VRAM

vllm serve <model> --enforce-eager

Disable CUDA graph for stable debugging

Quantization & Perf 5

vllm serve <model> --quantization awq

Load AWQ-quantized weights

vllm serve <model> --quantization gptq

Load GPTQ-quantized weights

vllm serve <model> --enable-prefix-caching

Prefix cache for repeated prefixes

vllm serve <model> --max-num-seqs 256

Raise max concurrent sequences

vllm serve <model> --dtype half

Use fp16 to save VRAM (default auto)

Tips 5

Server defaults to :8000

Different from OpenAI default — fix client base_url

Big model? raise --gpu-memory-utilization first

Tune util before anything else

TP count = GPU count

Set tensor-parallel by card count

Debug with --enforce-eager

Rule out CUDA-graph weirdness

Quantization trades throughput/VRAM

awq/gptq balance precision vs cost

Tips

vLLM is a high-throughput inference engine (PagedAttention + continuous batching) that exposes open models over the OpenAI-compatible protocol — the default for self-hosted serving.
The server listens on :8000 by default (not OpenAI's :8080); clients just set `base_url=http://localhost:8000/v1` to swap in seamlessly.
Low on VRAM? raise `--gpu-memory-utilization` first; use `--tensor-parallel-size` per GPU count for multi-card; debug oddities with `--enforce-eager`.
AWQ/GPTQ trade precision for cost; `--enable-prefix-caching` speeds up multi-turn/repeated-prefix workloads.

Official References

Commands are compiled from the official docs below. Click to verify the latest usage.

Maintained by LaoHand

Publicly updated on Aug 23, 2026, continuously proofread against official docs.

Contact Us

Wrong command or description? Send us corrections, business inquiries or product feedback by email.

Edit this page on GitHub

Edit the source file directly and open a PR to improve this cheatsheet.

Edit this page on GitHub Report an issue