llamafile Cheatsheet - Single-file Local LLM Run & Inference

All llamafile local-run commands in one place — from single-file launch to OpenAI-compatible server, GGUF loading to GPU acceleration — organized by scenario. Copy and go.

AI CLI·25 commands·Last updated 2026-08-23

Get & Run 5

curl -L <release-url> > llamafile && chmod +x llamafile

Download and make executable

./mistral.llamafile

Run directly (self-contained weights → Web UI :8080)

./llamafile --help

Show all flags

On Windows rename to llamafile.exe

Windows needs .exe to run

sh -c ./llamafile

Fallback launch for old zsh

OpenAI-compatible Server 5

./xxx.llamafile --server --nobrowser

OpenAI-compatible server (default :8080)

./xxx.llamafile --server --port 8081

Set a custom port

./xxx.llamafile --server --host 0.0.0.0

Allow LAN/external access

curl http://localhost:8080/v1/models

List models served

openai_api_base=http://localhost:8080/v1

Point OpenAI SDK at local server

Load External Weights 5

./llamafile -m ~/weights/foo.gguf

Run with external GGUF weights

./llamafile -m model.gguf --temp 0.7 -p "..."

Set temperature and a prompt

./llamafile -m model.gguf --n-gpu-layers 35

Enable GPU (more layers = faster)

./llamafile -m model.gguf --image photo.jpg -p "describe this"

Multimodal (needs mmproj)

./llamafile -m model.gguf --grammar 'root ::= "yes" | "no"'

Constrain output with BNF

Embed & Offline 5

./embed.llamafile --server --embedding --port 8081

Embedding server (for RAG)

Fully local, no data leaves the machine

Privacy/offline advantage

./llamafile -m w.gguf --no-display-prompt

Hide prompt noise when scripting

zipalign to embed .args into the binary

Bake default args; double-click to run

Single-file distro, cross-platform (mac/win/linux/bsd)

Build once, run anywhere

Tips 5

Server defaults to :8080

Different from vLLM :8000 — mind the client

More --ngl = faster but more VRAM

Set layers by GPU VRAM

Self-contained build launches Web UI

Chat without the CLI

Pick it for offline-first

Most stable in air-gapped/sensitive envs

--grammar controls output

Use it for stable structure in scripts

Tips

llamafile bundles llama.cpp + Cosmopolitan Libc into a single-file executable — build once, run on macOS/Windows/Linux/BSD. The most hassle-free local/offline LLM runner.
Self-contained server builds launch a Web UI (:8080); with external GGUF use `-m` for the path and `--ngl` for GPU layers (more = faster but more VRAM).
The OpenAI-compatible server defaults to :8080 (not vLLM's :8000); clients just change `base_url`. Use `--embedding` to serve an embedding model for RAG.
Fully local, no data leaves the machine — first choice for privacy/air-gapped/offline. `--grammar` constrains output with BNF for stable scripting.

Official References

Commands are compiled from the official docs below. Click to verify the latest usage.

Maintained by LaoHand

Publicly updated on Aug 23, 2026, continuously proofread against official docs.

Contact Us

Wrong command or description? Send us corrections, business inquiries or product feedback by email.

Edit this page on GitHub

Edit the source file directly and open a PR to improve this cheatsheet.

Edit this page on GitHub Report an issue