When something is wrong
14 answers
The things that go wrong, in rough order of how often.
Looking for something specific
The searchable index covers all 326 answers at once and filters as you type.
207 Uploads fail with 413 Request Entity Too Large
The load balancer refused the request before your application saw it. Open the site, Settings tab, Limits and Timeouts, and raise Maximum upload size.
Then check the application has its own limit set at least as high. PHP has two of its own, and a load balancer set to 100MB in front of a PHP set to 2MB just moves the error one step later and makes it harder to find.
208 A WebSocket will not connect
In order:
- Is Enable WebSocket support ticked on the site, Settings tab, WebSocket? This is the answer most of the time.
- Is the site on HTTP/1.1 to the backend, under Settings, Proxy? An upgrade cannot happen over HTTP/1.0.
- If it connects and then dies after a minute, that is the idle timeout, not the connection. Raise WebSocket idle timeout.
- If you have paths, is the WebSocket path one of them? A site with paths serves only what is
listed, so
/socket.io/needs to be there.
The browser console usually shows a 400 or a 426 for the first two, and a clean close for the third, which is a quick way to tell them apart.
209 The browser blocks my API call with a CORS error
CORS is enforced by the browser, not by the server, so the request very often reached your application and came back fine. Check the browser console for which check failed, then:
- Is Enable CORS ticked on the site, Settings tab, Headers?
- Is the calling site listed in Allowed origins, with its scheme and exactly as the browser sees it?
- Is OPTIONS ticked in the methods, so the preflight can happen?
- Is every unusual header the caller sends listed in Allowed request headers?
Curl will not show you this problem, because curl does not enforce CORS. A successful curl proves nothing here.
210 Everything is slow and I cannot tell where
Turn on the timing log first. Open the site, Settings tab, Logging, and set Log format to Adds timings and upstream info. Now every line records how long the request took and how long the backend took.
Compare the two numbers:
- Backend time high, total close to it: the application is slow. The load balancer is passing that on faithfully.
- Backend time low, total much higher: the time is going in transfer, which usually means a slow client or a large uncompressed response. Check compression is on.
- Both low but the site feels slow: it is not the request, it is the number of them. Look at the Utilization charts and at how many things one page pulls.
317 A site returns 502
502 means the load balancer could not get an answer from your application. Work through these in order.
- Open the pool and look at the members. If they are marked down, the health check is failing and the problem is behind the load balancer, not in it.
- If the pool uses https to reach the backend, try turning certificate verification off. If that fixes it, the proper fix is to point the pool at the right authority file rather than leaving verification off.
- Check the port. A pool pointing at the wrong port fails exactly like an application being down.
318 A site returns 504
The backend accepted the connection and did not answer in time. That is your application being slow rather than being down. Look at what it is waiting for, usually a database or a downstream service. Raising the proxy timeout hides the symptom and makes the queue longer.
319 A site returns 403 and I did not expect it
Four candidates, in order of how often they happen: an access list on the site or the path, country blocking, the WAF in a blocking mode, or a backend enforcing a trust token that nginx is not sending. The audit log and the WAF hit list narrow it down quickly.
320 nginx will not start after a change
It should not be possible, because the config is tested on every node before any of them use it. If it happens anyway, the previous configuration is on the node in /var/backups/nginx-fleet-manager and can be put back by hand. Then open the Changes page and go back to the last version that worked.
321 Where are the logs?
| What | Command |
|---|---|
| The management screen | sudo docker logs nginx-fleet-manager |
| The host agent | sudo journalctl -u nginx-mgr-agent -n 50 |
| nginx errors | sudo tail -50 /var/log/nginx/error.log |
| nginx access | sudo tail -f /var/log/nginx/access.log |
| Is nginx happy | sudo nginx -t |
322 The management screen will not load at all
- Check you are coming from an address on the allowlist. It is checked before the login page renders, so a wrong address gives you nothing rather than an error.
- Check the container is running:
sudo docker ps. - Read its log:
sudo docker logs nginx-fleet-manager. - Check port 7443 is open from where you are.
323 Buttons that write are all grayed out
You are on the standby. Check the top right of the screen and open the other node. This is the single most common confusion in the whole product.
324 A setting will not save, or saves and reverts
It is being controlled by the environment file rather than by the screen. Settings has a section at the bottom called Set in the environment listing everything in that state. When both have a value, the file wins.
325 Everything looks right and the site still does not answer
Check in this order: does the name point at your load balancer, is the port open from outside, does a site have that exact server name, is the site enabled, and has the config actually been applied. Four of those five are outside this software, which is usually where the answer is.
dig shop.example.com +short
curl -sv https://shop.example.com/ 2>&1 | head -20
326 How do I report a problem usefully?
The manager log and what you were doing at the time is nearly always enough for somebody to work out what happened. Include which node you were on, whether it was active or standby, and the config version from the dashboard.