SGLang CLI Cheatsheet - sglang.launch_server Inference Full Reference

Standing up a self-hosted model API is easy; making it fast is the real work. SGLang leans on RadixAttention, which organizes prompt prefixes into a radix tree so multi-turn conversations and shared system prompts reuse KV cache automatically, cutting time-to-first-token noticeably on long prefixes. This table groups launch_server options into startup, quantization and precision, and performance, covering tensor parallel splitting, speculative or expert parallel sizing, quantization choice, and the memory fraction reserved for the KV cache. A workable decision order: check whether the model fits on one GPU, add tensor parallelism if it does not, quantize only if adding cards is not an option, and touch the KV cache ratio last because it directly costs throughput. Reach for it when you need a local OpenAI-compatible endpoint, when benchmarking throughput, or when production latency is high but GPU utilization looks unremarkable. The real difference from vLLM is prefix cache efficiency, so pick by request shape.