llama.cpp CLI Cheatsheet - llama-cli Local Inference Full Reference

llama.cpp is a lightweight inference engine in C/C++ that runs GGUF quantized models on CPU, Apple Silicon, and various GPUs. llama-cli is its built-in terminal chat frontend for quick local verification and offline inference. This sheet covers model loading, sampling, context, and performance flags so you can start a chat on a laptop.

AI CLI·20 commands·Last updated 2026-09-10

Basic Inference 8

llama-cli -m model.gguf -p "hello"

Generate from a prompt then exit

llama-cli -m model.gguf --interactive

Enter interactive chat

llama-cli -m model.gguf -ngl 35

Offload 35 layers to GPU

llama-cli -m model.gguf -c 4096

Set context window to 4096

llama-cli -m model.gguf -t 8

Use 8 threads

llama-cli -m model.gguf --temp 0.7

Set sampling temperature

llama-cli -m model.gguf --top-p 0.9

Set nucleus sampling threshold

llama-cli -m model.gguf --color

Colorized terminal output

Sampling and Context 6

llama-cli -m model.gguf --repeat-penalty 1.1

Set repeat penalty

llama-cli -m model.gguf -s 42

Fix random seed for reproducibility

llama-cli -m model.gguf --system "you are a helper"

Inject a system prompt

llama-cli -m model.gguf -cnv

Enable multi-turn conversation context

llama-cli -m model.gguf -f prompt.txt

Read prompt from a file

llama-cli -m model.gguf --flash-attn

Enable Flash Attention

Performance and Advanced 6

llama-cli -m model.gguf -ngl 99

Offload as many layers to GPU as possible

llama-cli -m model.gguf -t 16

Raise thread count for CPU speed

llama-cli -m model.gguf -c 8192

Expand context to 8192

llama-cli -m model.gguf --mlock

Lock weights in RAM to avoid swapping

llama-cli -m model.gguf -b 512

Set prompt batch size

llama-cli --help

List all available flags

Tips

`-ngl` is the key performance knob: set it high (e.g. 99) to offload to GPU, or 0 for pure CPU.
A larger -c costs more VRAM; set it to the model real length instead of maxing it out.
GGUF quantized models with -ngl plus --flash-attn give the best experience on Apple Silicon or CUDA.

FAQ

How is llama-cli different from ollama?

llama-cli is the native llama.cpp inference frontend with fine-grained flags; ollama is a higher-level model manager and server with less tuning surface.

What if the GPU is not used?

Set -ngl to offload layers and ensure your build includes the backend (CUDA, Metal, or Vulkan); for pure CPU use -ngl 0.

Official References

Each command links to its official documentation below, so you can verify the latest usage and read deeper.

llama.cpp Repo

Maintained by LaoHand

Publicly updated on Sep 10, 2026, continuously proofread against official docs.

Contact Us

Wrong command or description? Send us corrections, business inquiries or product feedback by email.