First build an observable backend: what the proxy can reach
Before explaining proxies, make the backend self-evident: a Node service that returns JSON plus echoes request headers, so every request shows which instance it landed on. Nine out of ten proxy mistakes are ultimately judged by the fact "which server actually received it".
Write a ~twenty-line probe with node:http listening on different ports, returning the request remote addr and host. The front side can then directly see whether the proxy worked and whether the path was stripped wrong.
// probe.mjs
import http from "node:http";
const port = Number(process.env.PORT || 8201);
http.createServer((req, res) => {
res.setHeader("content-type", "application/json");
res.end(JSON.stringify({
port,
path: req.url,
host: req.headers.host,
forwarded: req.headers["x-forwarded-for"] || null,
}));
}).listen(port, "127.0.0.1", () => console.log("listening", port));location matching order: why requests always fall into /
The location pitfall is concentrated in rule selection. nginx picks a location by longest-prefix-match, but when several prefixes could all hit, it takes the longest; meanwhile = exact match and ^~ regex suppression each have their own priority, and the wording is easy to flip-flop.
The most common symptom: writing /api and / at the same level, assuming /api wins. In reality / also matches anything starting with /api by prefix, and between two candidates the longest prefix wins, so /api (being longer) takes it—but write it as /api/ with a trailing slash and the difference can drop requests into /'s plain proxy_pass for good, corrupting the forwarded path. With that, the ordering example below reads easily.
location / { # most generic, the fallback
proxy_pass http://backend;
}
location ^~ /static/ { # beats regex, highest prefix match
alias /srv/www/static/;
}
location = /healthz { # exact match
return 200 "ok\n";
}
location ~* \.(css|js)$ { # regex (~* is case-insensitive)
proxy_pass http://front;
}The proxy_pass slash trap: a trailing / is a path-rewriting scalpel
Whether proxy_pass ends in / decides if the matched prefix gets stripped on forward. With a trailing /, the segment matched by location is removed and the remainder is glued to the upstream URI; without it, the entire original URI (matched part included) ships verbatim.
Example: location /api { proxy_pass http://up/ } and a request to /api/orders forwards as / + orders -> /orders upstream; drop the slash as proxy_pass http://up and the upstream receives /api/orders. Getting this backwards is the prime culprit behind "the API URL is absolutely right yet the backend 404s". Both spellings and their expectations are shown below.
# Case A: strip the /api prefix
location /api {
proxy_pass http://up:8201/; # /api/orders -> /orders
}
# Case B: keep the /api prefix
location /api {
proxy_pass http://up:8201; # /api/orders -> /api/orders
}
# Verify: use curl to see the returned path
curl -s http://127.0.0.1/api/orders | python3 -m json.toolUpstream load balancing: the upstream block with weights and health checks
Once a single upstream stops being enough, pooling several backends in an upstream block for balancing is the natural next step. nginx defaults to round-robin; add weight for weighted distribution, least_conn for least-connected scheduling, and ip_hash when sessions must stick.
Health checks split into passive (proxy_next_upstream: attempt another machine on 5xx, free) and active check (with commercial or nginx-plus). A common blunder behind endless 502 is a dead backend staying in rotation because fail_timeout was never set—below is an upstream config with timeouts and retries, plus verification.
upstream app_cluster {
least_conn; # or ip_hash; or leave empty = round-robin
server 127.0.0.1:8201 weight=3 max_fails=2 fail_timeout=10s;
server 127.0.0.1:8202 weight=1 max_fails=2 fail_timeout=10s backup;
}
server {
listen 80;
location / {
proxy_pass http://app_cluster;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
# Repeat requests to see the landing distribution
for i in {1..8}; do curl -s 127.0.0.1 | python3 -c "import sys,json;print(json.load(sys.stdin)[\"port\"])"; doneMistake vs fix: losing client IP and original scheme behind the proxy
Right after a proxy goes live, backend logs show RemoteAddr as 127.0.0.1 and the app cannot tell HTTPS from HTTP. By default proxy_pass does not pass client info along, so you must add proxy_set_header.
More subtle is a missing X-Forwarded-Proto that makes the app generate wrong absolute links, or trusting an untruncated externally-supplied X-Forwarded-For that leaves a spoofing backdoor. The fix is to add headers uniformly in the server context and only trust them on trusted internal networks. Below, a bare proxy_pass is replaced with the full header-carried spelling.
# Bad example: bare forwarding without headers
proxy_pass http://app_cluster;
# Fix comparison: add the full pass-through headers
proxy_pass http://app_cluster;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade; # websocket
proxy_set_header Connection "upgrade";
# Verify the backend sees the real client IP
curl -s http://127.0.0.1/ | python3 -m json.toolVerify and reload: syntax, config slice, real request
After changes, first nginx -t to pass the syntax gate, then nginx -s reload for a graceful reload—prefer reload over restart to avoid severing existing long connections. Then verify on at least three dimensions: syntax passes, the intended location matches, and backend logs show records carrying a correct X-Forwarded header.
To chase a weird path, use curl -o /dev/null -w to read status and timing, then a log slice to confirm which location was hit. Gather the whole check set in a script so regression is a single command.
# Syntax check and reload
docker exec nginx nginx -t 2>&1 || sudo nginx -t
sudo nginx -s reload
# Cross-check the response
curl -s -o /dev/null -w "%{http_code} %{time_total}s\n" http://127.0.0.1/orders
# Confirm the hit: read the backend probe
journalctl -u probe-up1 --since=-2m | tail -n 3