llama.cpp Cheatsheet - llama-cli Flags & GGUF

llamacpp (llama.cpp) is a lightweight inference engine in C/C++ that runs GGUF quantized models on CPU, Apple Silicon, and various GPUs. llama-cli is its built-in terminal chat frontend for quick local verification and offline inference. This llamacpp command sheet covers model loading, sampling, context, and performance flags so you can start a chat on a laptop.

AI CLI·20 commands·Last updated 2026-09-10

Typical Use Case

For developers running large models locally, especially in the llama.cpp ecosystem. llama-cli is llama.cpp's command-line entry point, covering model download, interactive chat, one-shot question answering, embeddings and benchmarks. Two things most people use daily are single-shot prompting with -p to plug into scripts, and --hf-repo to pull a model straight from Hugging Face. Organised in five groups: obtaining models, inference modes, sampling parameters, performance tuning, and practical combinations.

Basic Inference 8

llama-cli -m model.gguf -p "hello"

Generate from a prompt then exit

llama-cli -m model.gguf --interactive

Enter interactive chat

llama-cli -m model.gguf -ngl 35

Offload 35 layers to GPU

llama-cli -m model.gguf -c 4096

Set context window to 4096

llama-cli -m model.gguf -t 8

Use 8 threads

llama-cli -m model.gguf --temp 0.7

Set sampling temperature

llama-cli -m model.gguf --top-p 0.9

Set nucleus sampling threshold

llama-cli -m model.gguf --color

Colorized terminal output

Sampling and Context 6

llama-cli -m model.gguf --repeat-penalty 1.1

Set repeat penalty

llama-cli -m model.gguf -s 42

Fix random seed for reproducibility

llama-cli -m model.gguf --system "you are a helper"

Inject a system prompt

llama-cli -m model.gguf -cnv

Enable multi-turn conversation context

llama-cli -m model.gguf -f prompt.txt

Read prompt from a file

llama-cli -m model.gguf --flash-attn

Enable Flash Attention

Performance and Advanced 6

llama-cli -m model.gguf -ngl 99

Offload as many layers to GPU as possible

llama-cli -m model.gguf -t 16

Raise thread count for CPU speed

llama-cli -m model.gguf -c 8192

Expand context to 8192

llama-cli -m model.gguf --mlock

Lock weights in RAM to avoid swapping

llama-cli -m model.gguf -b 512

Set prompt batch size

llama-cli --help

List all available flags

Parameter matrix

Parameter	Effect	Example
`-m <file>`	Point at the GGUF model file to load	`llama-cli -m model.gguf`
`-hf <user/repo>`	Download and run straight from Hugging Face without a local copy	`llama-cli -hf ggml-org/gemma-3-4b-it-GGUF`
`-p "<prompt>"`	Answer one prompt and exit, which is what you want in scripts	`llama-cli -m m.gguf -p "write a quicksort"`
`--hf-file`	Pick one specific file from the repository named by -hf	`llama-cli -hf <repo> --hf-file q4.gguf`
`-c`	Context length; lower this first when memory runs short	`llama-cli -m m.gguf -c 4096`
`-ngl`	How many layers to offload to the GPU; use 99 for all of them	`llama-cli -m m.gguf -ngl 99`
`-t <threads>`	Number of CPU threads to use	`llama-cli -m m.gguf -t 8`
`--temp`	Sampling randomness; 0 gives the most deterministic output	`--temp 0`
`--top-k / --top-p`	Truncate the candidate set during sampling	`--top-p 0.9`
`-n <tokens>`	Maximum number of tokens to generate	`-n 512`
`--embeddings`	Run in embedding mode instead of text generation	`llama-cli -m m.gguf --embeddings`
`--benchmark`	Run the inference benchmark and report throughput	`llama-cli -m m.gguf --benchmark`

Common pitfalls

SymptomLoading the model reports "not enough memory" or the process is killed outright.

CauseThe model is larger than the available VRAM or RAM, or the context length is set far too high.

FixReduce -c first, then switch to a smaller quantisation (Q4 is far smaller than Q8), and confirm the run with --benchmark.

SymptomCPU inference is so slow that waiting is impractical.

CauseNothing is offloaded to the GPU, or the layer count was left too low.

FixUse -ngl 99 to push every layer onto the GPU; with no GPU, raise -t and pick a smaller model.

SymptomOutput repeats itself and gets stuck in the same loop.

CauseSampling is too greedy, or no stop condition was set.

FixLoosen --temp and --top-p to introduce randomness; in scripts add --temp 0 when you would rather have reproducible output.

SymptomNon-English output arrives garbled or cut off mid-sentence.

CauseThe model itself does not cover that language, or the context ran out and the answer was truncated.

FixSwitch to a multilingual model, raise -c, and relax the generation ceiling with -n.

SymptomThe process never exits when called from a script.

CauseWithout -p the CLI drops into its interactive loop and waits for input.

FixAlways pass -p "..." in scripts so it answers once and exits.

SymptomTwo runs of the same question give very different answers.

CauseSampling is stochastic and no seed was pinned.

FixUse --temp 0 whenever reproducibility matters, and for batch comparisons pin the model version and quantisation too.

Troubleshooting

1Confirm the current setup can actually infer
llama-cli -m model.gguf --benchmark
A benchmark run proves the model loads and gives you throughput numbers to compare.
2Short context first when VRAM is tight
llama-cli -m model.gguf -c 2048 -ngl 99
Context length dominates VRAM use, so trim it before you go hunting for another model.
3One-shot question from a script
llama-cli -m model.gguf -p "Summarise this configuration"
-p answers once and exits; add --temp 0 when you need the same answer every time.
4Pull a model straight from Hugging Face
llama-cli -hf ggml-org/gemma-3-4b-it-GGUF --hf-file gemma-3-4b-it-Q4_K_M.gguf
No manual download; when the repository holds several files, --hf-file names the quantisation you want.
5Raise throughput on CPU only
llama-cli -m model.gguf -ngl 0 -t 8
With no GPU present, turn GPU offloading off explicitly and give the process every thread you can spare.

Command Examples

One-shot from a script: -p is what makes it exit

llama-cli -m model.gguf -p "Turn the requirements below into three acceptance criteria" --temp 0 -s 42

Without -p the process enters the interactive loop and a script hangs until it is killed. --temp 0 switches to greedy decoding so the same question yields the same text every run, which is what downstream parsing needs; -s 42 removes sampling randomness on top of that. Raise the temperature back towards 0.7 only when the answer is supposed to be creative.

A performance baseline: full GPU offload plus flash attention and context

llama-cli -m model.gguf -ngl 99 --flash-attn -c 8192 -t 16 --top-p 0.9 --repeat-penalty 1.1

-ngl 99 offloads as many layers as possible to the GPU and is the single most important flag when a GPU exists; on a CPU-only box, leaving layers on the CPU is the more stable choice. --flash-attn cuts the memory and compute cost of long contexts. -c 8192 is the context window, and larger costs more VRAM, so match it to what the model actually supports. Tune -t 16 and the sampling flags against the machine you actually have.

Interactive chat: system prompt, conversation context and colour

llama-cli -m model.gguf --interactive --system "You are a helpful assistant" -cnv --color

--system applies on every turn, which is the place to fix a role and an output format. -cnv keeps the conversation context so the model remembers earlier answers; the window is bounded by -c, and once you exceed it older turns are dropped, which looks like the model suddenly going terse or answering the wrong thing. --color only affects the terminal, so redirected output usually contains no colour information.

Read a long prompt from a file with -f, layered offload and batch size

# Keeping the prompt in a file avoids shell quoting and enormous argument lists
llama-cli -m model.gguf -f prompt.txt -ngl 35 -b 512 -c 4096

# Confirm which flags this build actually accepts
llama-cli --help

As soon as a prompt contains quotes, newlines or pipe characters, passing it as a shell argument needs layers of escaping and silently dropped characters; putting it in a file and reading it with -f is the dependable route. -ngl 35 fits the case where the whole model does not fit in VRAM but a good part of it does. -b 512 is the prompt batch size: larger batches speed up prefill of long prompts but use more memory. Parameter sets differ between builds, so --help remains the only reliable list.

Common Pitfalls

When memory runs out the process is killed outright by the system OOM killer: no stack trace, no interpreter-level error, just disappearance mid-run, with the only record in dmesg on the same machine. -ngl 99 combined with an oversized -c is the classic trigger, so lower -c before switching to a smaller quantisation.
An untrusted model file is untrusted code: a GGUF from a third-party repository can be swapped or carry malicious content, and llama.cpp parses its metadata and tensors while loading. Take models only from trusted sources or from weights you converted yourself, and verify provenance after pulling with --hf-repo.
Everything in --system and in the file passed to -f is pushed into the local model, so internal documents, account details and key fragments can end up in redirected output, terminal scrollback or log files. Treat prompt files and generated output as sensitive data and keep them out of directories that get synced.
Without a fixed seed and temperature, two runs of the same prompt differ, so batch regressions, scoring and assertions fail at random and are hard to reproduce. Pin --temp 0 with -s 42 in any script or test path, and record the model file and quantisation alongside it.
Setting -t above the physical core count makes the process fight every other service for the same CPUs, lowering throughput and dragging co-located build or test jobs into timeouts. In CI, divide threads by concurrency instead of giving every job the full core count.

Tips

-ngl is the key performance knob: set it high (e.g. 99) to offload to GPU, or 0 for pure CPU.
A larger -c costs more VRAM; set it to the model real length instead of maxing it out.
GGUF quantized models with -ngl plus --flash-attn give the best experience on Apple Silicon or CUDA.

FAQ

How is llama-cli different from ollama?

llama-cli is the native llama.cpp inference frontend with fine-grained flags; ollama is a higher-level model manager and server with less tuning surface.

What if the GPU is not used?

Set -ngl to offload layers and ensure your build includes the backend (CUDA, Metal, or Vulkan); for pure CPU use -ngl 0.

How many layers should I offload with -ngl?

Offload as much as VRAM allows: set 99 (or the model layer count) when the whole model fits, and step down from 99 to the highest value that does not OOM when it does not, leaving the rest on CPU. Use 0 on pure-CPU machines.

How large should I set the context window with -c?

Match the context length the model actually supports rather than maxing it out: the KV cache grows linearly with -c, so a larger window costs more VRAM or RAM, and going beyond the trained length hurts output quality.

What is the difference between -i and -cnv, and which should I use?

-i is the generic interactive mode for feeding one prompt after another where each generation stands alone; -cnv is conversation mode, which keeps the dialogue context and formats messages with the model chat template (pair it with --system to set a system prompt). Use -cnv when you want to keep chatting.

Official References

Each command links to its official documentation below, so you can verify the latest usage and read deeper.

llama.cpp Repo

Maintained by LaoHand

Publicly updated on Sep 10, 2026, continuously proofread against official docs.

Contact Us

Wrong command or description? Send us corrections, business inquiries or product feedback by email.