llamafile Cheatsheet - Single-file Local LLM Run & Inference
llamafile is Mozilla single-executable approach to running local LLMs, for developers who want zero-friction copy-and-run models — ideal for distribution, offline, and demo use. It packs model weights and llama.cpp into one cross-platform executable with a built-in OpenAI-compatible server. This sheet covers single-file launch, --server, -m GGUF loading, -ngl GPU layers, and --temp sampling.
Get & Run 5
curl -L <release-url> > llamafile && chmod +x llamafile./mistral.llamafile./llamafile --helpOn Windows rename to llamafile.exesh -c ./llamafileOpenAI-compatible Server 5
./xxx.llamafile --server --nobrowser./xxx.llamafile --server --port 8081./xxx.llamafile --server --host 0.0.0.0curl http://localhost:8080/v1/modelsopenai_api_base=http://localhost:8080/v1Load External Weights 5
./llamafile -m ~/weights/foo.gguf./llamafile -m model.gguf --temp 0.7 -p "..."./llamafile -m model.gguf --n-gpu-layers 35./llamafile -m model.gguf --image photo.jpg -p "describe this"./llamafile -m model.gguf --grammar 'root ::= "yes" | "no"'Embed & Offline 5
./embed.llamafile --server --embedding --port 8081Fully local, no data leaves the machine./llamafile -m w.gguf --no-display-promptzipalign to embed .args into the binarySingle-file distro, cross-platform (mac/win/linux/bsd)Tips 5
Server defaults to :8080More --ngl = faster but more VRAMSelf-contained build launches Web UIPick it for offline-first--grammar controls outputCommand Examples
Run a weights-bearing file directly and get the web UI
./mistral.llamafileWithout --server it brings up its own interface on :8080 for you to open in a browser. If that port is taken, start one explicitly with --server --port instead.
Start a headless service and confirm the exposed model
./xxx.llamafile --server --nobrowser
curl http://localhost:8080/v1/modelsThe reply carries the identifier of the model actually loaded, and that is the name your client must send. Point the SDK over with openai_api_base=http://localhost:8080/v1.
Load external weights and turn on GPU acceleration
./llamafile -m model.gguf --n-gpu-layers 35The layer count is a preference, not a switch: offloading everything is fastest and most VRAM-hungry. On an out-of-memory error, walk 35 downwards until it loads.
Pin the output to fixed values with a BNF grammar
./llamafile -m model.gguf --grammar 'root ::= "yes" | "no"'When a script has to string-compare the reply, this is the least-effort approach: the model can only choose yes or no. Add --no-display-prompt and not even the prompt marker gets in the way.
Common Pitfalls
- On Windows the file must be renamed to llamafile.exe before it will execute at all — verify with ./llamafile --help before going further.
- --server --host 0.0.0.0 publishes an unauthenticated inference endpoint to the LAN, or worse to the internet, and anyone with the address gets free compute. Only do this on a network you trust.
- Pushing --n-gpu-layers past what the card can hold fails at load time or drops the machine into swap. Step the count up to your actual VRAM instead of copying someone else 35.
- A wrong -m path, or a GGUF the bundled llama.cpp cannot parse, fails while loading rather than while generating. Check ./llamafile --help first to confirm the flags match your build.
- The weights-bearing build claims 8080 on a plain run and will collide with a service already listening there. Remember --no-display-prompt in scripts or the prompt marker ends up in your pipe. And sh -c ./llamafile is only a fallback for old zsh builds, not a portable way to launch the file.
Tips
- llamafile bundles llama.cpp + Cosmopolitan Libc into a single-file executable — build once, run on macOS/Windows/Linux/BSD. The most hassle-free local/offline LLM runner.
- Self-contained server builds launch a Web UI (:8080); with external GGUF use
-mfor the path and--nglfor GPU layers (more = faster but more VRAM). - The OpenAI-compatible server defaults to :8080 (not vLLM's :8000); clients just change
base_url. Use--embeddingto serve an embedding model for RAG. - Fully local, no data leaves the machine — first choice for privacy/air-gapped/offline.
--grammarconstrains output with BNF for stable scripting.
FAQ
What is the difference between just running the file and using --server?
Running a weights-bearing executable directly (./mistral.llamafile) starts a web UI on :8080 that you open in a browser — good for demos and casual chatting. --server --nobrowser instead starts a headless OpenAI-compatible service for programs to call over HTTP. Both default to port 8080.
What port does it use by default, and what do I point the OpenAI SDK at?
The default is 8080 — note this differs from the 8000 that vLLM conventionally uses, which is a common trap when copying someone else config. Set openai_api_base to http://localhost:8080/v1. Confirm the service is actually up and learn the exposed model name with curl http://localhost:8080/v1/models, and move the port with --server --port when 8080 is taken.
How do I load an external GGUF and control how fast it runs?
Point -m at the weight file, for example ./llamafile -m ~/weights/foo.gguf. Speed is governed by --n-gpu-layers: more layers means more speed and more VRAM, so raise it from whatever loads and expect a hard failure once you exceed the card. Sampling randomness is set with --temp.
My scripted call gets prompt markers and chatter mixed into the output — how do I clean that up?
Add --no-display-prompt to strip the prompt markers so only the model text remains. If the script also needs a stable structure, constrain the answer with --grammar and a BNF rule that pins the allowed values — for example yes or no — so downstream code can always parse the reply.
How do I serve embeddings for RAG, and how do I look at an image?
For RAG, run the embedding service: ./embed.llamafile --server --embedding --port 8081, using a different port than the main service to avoid a clash. For vision, use ./llamafile -m model.gguf --image photo.jpg -p "Describe this image", which requires a vision-capable weight and usually a matching mmproj file.
Official References
Each command links to its official documentation below, so you can verify the latest usage and read deeper.
Maintained by LaoHand
Publicly updated on Aug 23, 2026, continuously proofread against official docs.
Contact Us
Wrong command or description? Send us corrections, business inquiries or product feedback by email.
Contact Us