How to do almost anything here

One or two servers running Ubuntu 22.04 or newer, with internet access and root. That is the whole list.

• One server works and gives you the GUI, the certificates and the WAF.
• Two servers is what gives you failover, and is the shape everything here is designed around.
• The servers do not need to be large. Four processors and four gigabytes is comfortable for most workloads.
• You also want a domain name you control, so you can get certificates.

Two commands. The first fetches the code and sets up Docker if it is missing. The second builds nginx.

sudo git clone https://git.failoverlb.net/failoverlb.git /data/docker/failoverlb \
  && sudo /data/docker/failoverlb/failoverlb_setup.sh \
       --allowlist 198.51.100.25/32 --acme-email you@example.com

sudo /data/docker/failoverlb/nginx-installer.sh install

Replace the allowlist with the public address of your office or wherever you will manage this from. Then open the address the installer printed on port 7443.

22.04 and newer. The Docker setup script refuses anything older and says so. The installers are tested on the long term support releases, and they will try on other Ubuntu versions and tell you up front that they are outside what is promised.

Debian will very likely work and is not what this is written for.

It is the list of networks allowed to load the management GUI, and it is checked before the login page even renders. A stolen password from the wrong network gets nothing.

Put your office public address followed by /32, or your office range such as 198.51.100.0/24. If you are not sure what your address is, visit any site that reports it.

Without it, anybody who can reach port 7443 gets the login page. Set it before you put this anywhere interesting.

It is printed to the container log once, during installation.

cd /data/docker/failoverlb
sudo docker compose logs | grep -A3 "Made the first admin"

Change it as soon as you are in. If you have lost it entirely, use the console recovery command instead.

It is safe. Fix whatever it complained about and run exactly the same command a second time. It leaves alone anything it has already done, and it never touches your .env file or your database.

Port 5353 is where the DNS part really listens, on UDP and TCP, and a firewall rule the software manages sends 53 to it. You never open 5353 and you never test against it.

Pass --skip-docker if Docker is already set up the way you want it.

sudo /data/docker/failoverlb/failoverlb_setup.sh --skip-docker

You do not need git access at all. Copy the folder onto the box, change into it, and run the setup script from inside it. It will use what is already there.

cd /path/to/failoverlb
sudo ./failoverlb_setup.sh

Pass credentials to the setup script, or set NFM_GIT_TOKEN in the environment.

sudo ./failoverlb_setup.sh --git-user myname --git-token ghp_xxx

The token is kept inside the function that builds the clone URL, so it does not end up in a log line or in the process list of anything except git.

The installer guesses both, and the guess is worth checking on a box with several interfaces.

sudo ./failoverlb_setup.sh \
  --name lb-01 --address 203.0.113.10 --priority 100

The address is the one other nodes will reach this one on. Priority decides who wins the election, and higher takes it.

• Made a group called nginxmgr with gid 10001.
• Installed the host agent to /opt/nginx-fleet-manager and started it as a systemd service.
• Wrote an .env file with this node's identity and its secret key.
• Built and started the manager container.
• Turned off the certbot timer, on purpose.
• Put nginx-installer.sh at /opt/nginx-installer.sh so rebuilds work from the GUI.

The manager manages an nginx, it does not bring its own. The GUI comes up either way, and it warns you that nothing can be applied until nginx is there.

sudo /data/docker/failoverlb/nginx-installer.sh install

It takes the longest of any step because it compiles nginx from source with every module the GUI knows how to configure.

Do not edit anything under /etc/nginx/nginxmgr by hand. It is rewritten from scratch on every apply.

1. Open the address the installer printed, such as https://203.0.113.10:7443.
2. Your browser warns that the connection is not private. That is expected: the server made its own certificate during installation. Continue past it.
3. Type your username and password and press Continue.
4. Set up an authenticator app when asked, then type the six digit code.
5. Save the recovery codes it shows you. They are shown once and never again.

Any authenticator works: Google Authenticator, Microsoft Authenticator, 1Password, Aegis or anything else that scans a standard code.

1. Open the app and choose to add an account.
2. Point it at the square code on screen.
3. Type the six digit number it starts showing.

If the phone cannot scan, type the line of characters underneath the code into the app by hand. It is the same thing in a different form.

Sign in with a recovery code instead of a six digit code. Each one works once. Then go to My Account and set up your new phone from there.

Nobody inside the software can help, because the software is the thing you cannot reach. Somebody with root on any node runs this:

sudo docker exec nginx-fleet-manager \
  python -m app.reset_admin --list

sudo docker exec nginx-fleet-manager \
  python -m app.reset_admin alice --reset-totp

Every use is written to the audit log as coming from the console, which is exactly the sort of thing that should stand out later.

My Account, then Recovery codes. Making a new set cancels every old code immediately, including any you have written down. Do it when you are down to one or two, and save the new list before you close the page.

While the old phone still works: go to My Account and set up the new one from there. You get a fresh code to scan, and the old phone stops working once you confirm.

If the old phone is already gone, sign in with a recovery code first, then do the same thing.

My Account. You need your current password as well as the new one. Changing it ends your other sessions, so a browser you left signed in somewhere else stops working. That is deliberate, and it is the quickest thing to do if you think somebody else has your password.

My Account lists your sessions. If you see something you do not recognize, change your password, which ends all of them at once, and tell whoever looks after security where you work.

You are almost certainly looking at the standby node. Changes are made on the active one. Check the top right of the screen for which node you are on, and open the other one.

Put a site in front of it, the same way you would any other application.

1. Make a pool with one member: 127.0.0.1 port 7443. Turn on TLS and turn off certificate verification, since it uses its own self signed certificate and the traffic never leaves the box.
2. Add NFM_ROOT_PATH=/lb to .env and restart the container, so every link it writes includes the prefix.
3. Make a site for lb-01.example.com with a path of /lb pointing at that pool. Leave the strip prefix setting off.
4. Get a certificate for that name and turn on https.

Keep the allowlist set to your office networks. The name being public and the screen being reachable are two different things.

There is no single button for it. Turning an account off ends that person's sessions immediately. For everybody, restarting the manager container on each node clears sessions across the fleet.

1. Sites, then New site.
2. Give it a name. This is for you and visitors never see it.
3. Under Server names, put the address people will type, such as www.example.com.
4. Choose Proxy as the mode and pick a pool.
5. Press Create site, then Apply config when you are ready.

Put both names on one site, with the one you want to win listed first, and turn on Send the rest to the first one.

Serving both as if they were separate means a search engine sees two copies of every page. The same switch does www to bare or bare to www, depending on which you listed first.

Put *.example.com in the server names. You will need a wildcard certificate to serve it over https, which means dns-01 validation.

Tick Default site on one site. It answers anything whose Host header does not match another site.

Without one, nginx uses whichever server block it loaded first, which is usually not what you meant and moves around as you add sites. People who find your server by its IP then get a random one of your sites. A default that returns a plain 404 keeps that tidy.

Untick Enabled and apply. A disabled site is left out of the configuration entirely and can be put back with one click. Everything about it is kept.

Delete removes it at the next apply. The pools it used are left alone, because other sites may be using them. If you only want it gone for a while, disable it instead.

Make a site with the old names on it, set the mode to Redirect, and give it the target. Choose permanent only when the move is really final, because browsers cache a 301 aggressively and some of them for a very long time.

The generated config keeps the certificate challenge path out of the redirect, so renewal keeps working. Doing this by hand is the usual way renewal quietly breaks.

Set the site or the path to Static and give it a folder. Files have to be on every node, because either node can serve. Keep them in a folder you deploy to both.

Two ways, and the second is better.

• Quick: switch the site mode to Static and point it at a folder with one page in it. Change it back afterwards.
• Better: add a backup member to the pool serving a holding page, and drain the real members. Then the site keeps working for anything that is still up.

Open the site and press Clone this site. You get a copy with everything the same, ready to rename. Change the server names before you apply, because two sites answering for one name is refused.

Open the site, go to the nginx config tab, and turn on Let me write this server block myself. The forms stop driving that site while it is in that mode, so the two cannot fight over the same file.

It goes through the same checks as everything else: nginx tests it on this node before it is even saved, and every node stages and tests it at apply time. The worst case is config that refuses to save.

Each site has a raw configuration box at the bottom. What you put there is stored with the site and survives every apply, which is why it exists and why editing the generated files does not work.

Use the Check it button every time you touch it. Raw config is checked for syntax and not for sense, so a directive that is valid but wrong will apply cleanly and behave strangely.

Turn on Redirect http to https. Leave Listen on http on, because certificate renewal needs port 80 to answer and the redirect already means no real visitor stays there.

HTTP/2 is on by default for https sites. HTTP/3 is available because the build includes the module, and it is a tick box on the site settings tab. Check that UDP 443 is open, because QUIC does not use TCP.

Tick Management network only on the site. It writes an allow line for every node in the cluster plus the management networks from Settings, then deny all. The node list is generated from the roster, so a node you add later lands in it on its own.

Run PHP on your own server and put this in front of it as a normal proxy. Point a pool at the web server that runs the PHP, and the site at the pool. Nothing special is needed.

Two things to get right:

• Raise Maximum upload size under Limits and Timeouts to match what PHP itself allows, or the load balancer refuses uploads that PHP would have accepted.
• Raise Read timeout under Proxy if the site has a slow admin screen, which most content systems do.

Do not try to run PHP on the load balancer. It is a load balancer, and putting an application runtime on the box that fronts every site is exactly the concentration of risk this design exists to avoid.

Open the site, Settings tab, tick Show advanced settings, go to Advanced and use Response text replacement. Say what to look for and what to put in its place.

Genuinely useful for a staging copy of an application that writes its production address into every link, or for an appliance you cannot change that hardcodes its own hostname.

It reads and rewrites every response body it applies to, so it is not free, and it will happily corrupt a compressed or binary response if you point it at one. Use it on a path, on HTML only, and treat it as a patch rather than a fix.

For something like robots.txt, security.txt or a verification file: add a path, matched with Is exactly this, set What happens to Serve files from disk, and point it at the folder holding the file.

It never reaches the backend, so it keeps working during a deploy and while the application is down. That is the point for a verification file, which somebody else fetches on their own schedule and without warning.

Make a site for the old name and set it to redirect. Nothing else is needed.

1. Sites, Add a site.
2. Server names: the old name, for example old.example.com.
3. What it does: Redirect somewhere else.
4. Redirect to: https://new.example.com.
5. Pick a certificate for the old name, tick Listen on https and Redirect http to https.
6. Save, then Apply config.

The path and the query string come across on their own. Somebody asking for old.example.com/products?id=5 lands on new.example.com/products?id=5. You still want a certificate on the old name, because a browser checks the certificate before it reads the redirect.

Do it as a path rather than as a whole site, so the rest of the site carries on as normal.

1. Open the site, Paths tab, Add a path.
2. Path: /old-page.
3. Match it how: Is exactly this. That stops /old-page-two getting caught as well.
4. What happens: Redirect, then type where it should go.
5. Save, then Apply config.

Once a site has any paths at all it stops using the single backend on the Basics tab, so add a path of / pointing at your normal pool as well. The Paths tab says the same thing in a box at the top, because it catches nearly everybody once.

This is the one people ask for most. Visitors use /shop/basket and the application behind it expects /basket.

1. Open the site, Paths tab, Add a path.
2. Path: /shop/, matched with Starts with this.
3. What happens: Send it to a pool, and pick the pool.
4. Tick Take this path off before sending it on.
5. Save, then Apply config.

The visitor never sees the change. The address in their browser stays /shop/basket and the backend gets asked for /basket. Leave the box unticked and the backend gets the whole thing, which is right when the application already knows it lives under /shop.

Two ways, and which one to use depends on whether the visitor should see the new address.

If they should see it, make a site for old.example.com, add a path of /shop/, choose Redirect, and point it at https://new.example.com/. The rest of the address is added on the end, so /shop/basket/items arrives as /basket/items.

If they should not see it, and the two names are really one service, keep them on one site and use a path with Take this path off before sending it on ticked instead. Nothing moves in the address bar and there is no extra round trip.

Permanent is a 301. Browsers and search engines remember it, sometimes for a very long time, and some browsers will not ask you again until the cache is cleared by hand. Use it when the move is final.

Temporary is a 302. Nothing is remembered, so every visit asks again. Use it while you are testing, during a maintenance window, or any time you might change your mind.

The Code box on the path is where you pick, and a whole site redirect uses the same two. If you are not sure, start temporary. Turning a temporary redirect into a permanent one is easy. Getting a permanent one back out of a few thousand browsers is not.

Put both names on the one site, in Server names, and set When it answers to more than one name to Send the rest to the first one. Whichever name you typed first wins and the others redirect to it.

So example.com www.example.com sends www traffic to the bare name, and www.example.com example.com does the opposite. Make sure the certificate covers both names or the redirect never gets read.

The quick version: add a path, matched exactly with Is exactly this, and set What happens to Refuse it. The request is turned away at the load balancer and never reaches your application.

That answers 403, which means you are not allowed. If you want it to answer 410, which means it existed and is gone, put that in Extra config for this path on the same dialog. The difference is worth the extra minute if search traffic matters: a missing page gets asked for again for months, and a page that says it is gone gets dropped much faster.

If the content moved rather than went away, redirect it instead. A redirect keeps the traffic and the ranking. Refusing throws away both.

For a handful of addresses, use a path each. For a whole shape of address, use a rule.

1. Open the site, Settings tab, tick Show advanced settings.
2. Scroll to Advanced and use Rewrite rules.
3. One rule per line: a pattern, where it should go, and whether the visitor sees it.
4. Save, then Apply config.

Check the nginx config tab afterwards. It shows exactly what your rules turned into, which is a lot quicker than finding out from a visitor. A pattern that matches nothing is silent, so if the rule seems to do nothing, that is the first thing to suspect.

A path each does not scale past a few dozen. Use a map instead.

1. Maps, Add one.
2. Reads: $request_uri. Sets the variable: something like $new_address.
3. Leave the default empty, so anything not in the table is left alone.
4. Add a line per old address, with the new address beside it.
5. On the site, use the new variable in a rewrite rule under Advanced.

The table is one lookup no matter how long it gets, so a thousand rows costs the same as ten. It is also the only version of this you can hand to somebody else to maintain.

Nearly always one of three things:

• The site redirects to a name it also answers to. Sending example.com to https://example.com is a loop unless something else changes on the way through.
• Redirect http to https is on and the backend also redirects to https, but the backend cannot tell the request already arrived on https. Check the pool has What is on the other end set correctly, so the backend is told the original scheme.
• Two sites both claim the same name and each sends to the other. The Sites list shows every name, so search it for the one that repeats.

The browser network tab shows the chain, and it usually names the culprit in the first two hops.

The application thinks it lives somewhere else, usually because it is on port 8080 behind the scenes and it puts that in the Location header. The visitor gets bounced to a port that is not open to them.

Open the site, Settings tab, Proxy, and use Rewrite redirect locations. Give it what the backend says and what it should say instead. That fixes it at the load balancer, which beats waiting for somebody to change the application.

If the backend sits under a path, Backend lives under this path in the same section handles the same problem for the path part.

Same cause as the redirect one above. The application sets a cookie for /app because that is where it thinks it lives, and the browser then refuses to send that cookie back for the address people actually use.

Open the site, Settings tab, Proxy, Rewrite cookie paths. Say what the backend sends and what it should be. Sign in stops silently failing.

1. Backend Pools, then New pool.
2. Name it after what is in it, such as shop-web.
3. Add one row per server, with its address and the port your application listens on.
4. Leave the balancing method as round robin.
5. Save, then check the members go green before you build a site on top of it.

Open the pool, press Add a server, put in the address and port, and save. Wait for the health check to turn it green, then apply. Adding it and applying in one go means sending traffic to something you have not confirmed can answer.

Set its state to Draining and apply. It finishes what it is doing and takes nothing new. Watch the connection count fall to zero, then do whatever you came to do.

Turning it off outright drops whatever it was in the middle of.

Set its weight to 2 and it gets twice as much as a server on 1. Weights are a blunt tool, which is fine because the difference between a big box and a small one is usually blunt too.

Set Max connections on that member. 0 means no cap. Useful when a backend has a hard connection limit of its own and you would rather queue at the load balancer than get errors from the application.

Tick Backup on that member. Be careful: a backup server sits idle and untested until the worst possible moment. It is genuinely good for one thing, which is serving a holding page.

Turn on Use TLS for the pool. Leave certificate verification on. If your company issues its own certificates, point the pool at your authority file rather than switching verification off.

A pool that fails the moment you turn on TLS is nearly always the certificate check.

A pool being used by a site cannot be deleted, and the page says which site is using it. Point that site somewhere else first. This is deliberate, since deleting a pool a site depends on would take that site down at the next apply.

Round robin unless some requests take much longer than others. If your traffic is a mix of quick API calls and slow uploads, least connections spreads it more evenly because a busy server has more connections open.

Two ways, and both are workarounds.

• By address: the same client address always reaches the same server. Simple, and it breaks for anybody behind a large NAT.
• Sticky sessions: a cookie decides. Better behaved, and the first request is still unpinned.

The real fix is to put sessions in redis or your database so any backend can serve any visitor.

Turn on consistent hashing. Without it, taking one server out of a pool of four moves roughly everybody. With it, only the share belonging to that server moves.

Turn on slow start on the pool and choose a window. The member comes back at weight 1 and climbs to its configured weight over that window.

Thirty seconds suits most web applications. A JVM wants sixty to a hundred and twenty, because the just in time compiler needs real traffic before it is fast.

Turn on adaptive weighting. The health checker is already timing every probe, so it uses those numbers to nudge each weight on a cycle.

It reacts in minutes, not milliseconds. Good for a pool of mixed hardware. For traffic that spikes in seconds, least connections reacts faster on its own.

Turn on Backend trust for the pool. nginx then sets a shared secret header on every proxied request, and the GUI gives you the check snippet for nginx, Apache, Express, Django or Spring.

The header is set rather than added, so a client sending it themselves has it thrown away and replaced.

1. Rotate. A new token is generated but nginx keeps sending the old one. Add the new one to your backends so they accept either.
2. Activate. nginx starts sending the new one, which your backends already take. Apply, watch traffic, then delete the old one from your backends.

Doing it in one step means every request in the gap gets a 403.

You need two pools, one for each. Stream pools and http pools are separate in nginx itself, not by a choice made here, so you will see the same addresses listed twice and that is correct.

Open the pool, Load balancing tab, and set What is on the other end to Another website, on its own name.

Use this when the thing behind the pool is a whole website already, such as a hosted service, an appliance, or another company's site you are fronting. It is told its own name rather than yours, which is what it expects and what its certificate says.

Leave it on the first option for servers running your own application. Those want to be told the name the visitor typed.

Visitors ask for /basket and the backend actually wants /shop/basket. Open the pool, Load balancing tab, and put /shop/ into Where the application sits on these backends.

A path and nothing else. No scheme, no hostname and no port, because this is saying where the application lives, not how to reach it. The backends are already reachable or they would not be in the pool.

Redirects coming back get the prefix taken off again on the way out, so it never appears in your visitor's address bar. That is the part that makes this better than doing it with a rewrite rule, which fixes the request and leaves the redirects wrong.

Leave it empty when the application is at the root, which is most of the time.

Three places can set it, and they override each other in that order. Which one is right depends on what is actually true:

• The pool, under Load balancing, Where the application sits on these backends. Use this when it is a fact about the servers. If every site pointed at this pool has to ask for /shop/, it belongs here and you set it once.
• The site, Settings tab, Proxy, Backend lives under this path. Use this when the same pool is reached two different ways and only this site needs the prefix.
• The path, with Settings for this path only ticked. Use this when only part of one site is affected.

Start at the pool. Putting a fact about the servers on the site means the next site somebody makes against that pool is broken in a way that looks like nothing to do with them.

Open the pool, Load balancing tab, and set Never go below this many healthy.

Health checks can be wrong. A network blip, a slow moment, or a check that is too strict can mark good servers down, and if they all get marked down at once, the site is off for a reason that was never real.

Setting this to 1 or 2 means the pool keeps sending traffic to that many backends even when the checks say otherwise. A server that might be broken is a better answer than a certain error page.

Open the site, Settings tab, WebSocket, and tick Enable WebSocket support. Save, then Apply config. That is the whole job.

Behind that one tick, nginx has to be told to speak HTTP/1.1 to the backend and to pass the upgrade headers through instead of swallowing them. Doing it by hand is three directives and a lookup table, and getting one of them wrong gives you a connection that opens and then dies with no useful error.

Chat, live dashboards, collaborative editors, terminal windows in a browser and anything using Socket.IO all need this.

The connection is idle and something closes it. Open the site, Settings tab, WebSocket, and raise WebSocket idle timeout. Ten minutes suits most applications, an hour suits a terminal or a dashboard somebody leaves open.

If it still drops, the application is probably not sending keepalive pings. Raising the timeout hides that for a while but does not fix it, because anything else in the path may have its own idea about idle connections.

Open the site, Settings tab, Proxy. Three boxes matter:

• Connect timeout: how long to wait for the backend to pick up. Leave this short, a few seconds. A backend that will not accept a connection is not going to get better in a minute.
• Read timeout: how long to wait for the answer once it has picked up. This is the one to raise for a report that takes two minutes to build.
• Send timeout: how long to wait while sending the request. Only matters for large uploads.

Raise Read timeout for one slow path rather than the whole site if you can. Add the path, tick Settings for this path only, and set it there. A site wide two minute timeout means a genuinely broken backend holds connections for two minutes each.

Leave Buffer responses on, which is the default. The load balancer takes the whole answer from your application as fast as the application can produce it, lets the application go, and then dribbles it out to the visitor at whatever speed the visitor has.

Without it, a worker on your application server is held open for the whole download. A few hundred people on bad connections is enough to fill up a small application server while the processor sits idle.

The buffer sizes are under Show advanced settings, but the defaults are sensible and almost nobody needs to change them.

Buffering is exactly wrong for a response that is meant to arrive a piece at a time. With it on, a progress feed or an event stream sits in a buffer and arrives all at once at the end, which looks like the feature is broken.

1. Add a path for the streaming endpoint, for example /events.
2. Tick Settings for this path only.
3. Turn Buffer responses off there.
4. Raise Read timeout while you are in, because a stream is idle a lot.

Do it per path. Turning buffering off for the whole site gives back the slow client problem for every page on it.

Open the site, Settings tab, Proxy, and tick Show advanced settings. The retry list appears: Connection error, Timed out, Backend sent garbage, and then the status codes 500, 502, 503, 504, 403 and 404.

Connection error, Timed out and 502 are safe and worth having on. 503 is usually worth it too, because it normally means a backend is restarting.

Leave 403 and 404 off. Those are real answers from a working application, and retrying them just asks three servers the same question and gets the same answer three times.

Maximum retries caps how many other backends get tried, and Total retry time caps how long the whole thing may take. Set both, or one slow failure can turn into a long one.

A retry means the request is sent again. That is harmless for a page view and not harmless at all for a payment.

By default only GET and HEAD get retried, which is the safe choice. There is a box called Also retry POST and PATCH, and you should only tick it if you know the application handles being asked twice.

Never retry switches the whole thing off for that site or path. Worth it on a checkout, a webhook receiver, or anything that sends email.

This lives on the pool, not the site, because it is a fact about the servers rather than about the address people type.

Open the pool, Load balancing tab, What is on the other end:

• Servers running my app, answering to my site's name. The visitor's name is passed through. This is right for nearly everything you run yourself.
• Another website, on its own name. The backend is told its own name instead. This is right when you are putting a load balancer in front of something that already thinks of itself as a website, like a hosted service or an appliance.

Choosing the second one also points the backend's redirects back at you, so a visitor is not quietly walked off onto the other site's own address halfway through.

Get this wrong and you get a certificate warning, a redirect loop, or an application that builds every link with the wrong hostname in it.

Already done. Every proxied site sends the client address and the original scheme to the backend, because an application that logs the load balancer address for every request is no use to anybody.

What is left is the application end. Most frameworks ignore a forwarded address unless you tell them to trust the thing in front. Look for a trusted proxy setting and put the addresses of your load balancer nodes in it. Do not set it to trust everybody, or anybody can claim to be anybody.

The Cluster page lists the node addresses, which is what that setting wants.

Two places can set this, and the pool is usually the better one.

If every site using those backends needs the prefix, it is a fact about the servers. Put it on the pool, Load balancing tab, Where the application sits on these backends. Set once, right for everything pointed at it.

If only this site needs it, because the same pool is also reached another way, use the site instead: Settings tab, Proxy, Backend lives under this path. That overrides whatever the pool says.

Either way the prefix is added on the way in and taken back off the redirects on the way out. The other half of this is usually Rewrite redirect locations and Rewrite cookie paths in the same section, because an application that thinks it lives under a prefix will put that prefix into both.

Open the site, Settings tab, tick Show advanced settings, then Proxy, Hide these response headers. One name per line.

Worth hiding: anything naming the framework or its version, internal request identifiers, debug headers somebody left on, and any header carrying an internal hostname. None of it helps a visitor and all of it helps somebody working out what to try next.

Open the pool, Load balancing tab, and set Keep-alive connections. Somewhere between 16 and 64 suits most setups. Requests per connection caps how many requests one connection handles before it is replaced.

Setting up a connection costs a round trip, and a TLS connection costs several. On a busy site that is a real slice of the response time, spent doing nothing useful.

Make sure the site is on HTTP/1.1 under Settings, Proxy. Keepalive does not work on HTTP/1.0 and the setting is quietly ignored.

Something is sending a header bigger than the space set aside for it. A long list of cookies and a large signed in token are the usual causes, and single sign on tokens are the usual suspect of all.

Open the site, Settings tab, tick Show advanced settings, then Proxy and raise Header buffer size. Under Limits and Timeouts, raise Large header buffers too, because that one covers what the visitor sends rather than what the backend sends back.

Doubling is usually enough. If doubling twice is not enough, something is putting far too much into a cookie and that is worth fixing at the source.

nginx normally looks a backend name up once at startup and holds on to it. That is fine for a fixed server and wrong for anything in a cloud that moves.

Open the site, Settings tab, tick Show advanced settings, go to Advanced and set a DNS resolver. Now the name is looked up again as its record expires rather than once, ever.

Use your own resolver or the one your cloud provides. Do not point it at a public resolver for an internal name, because it will not know the answer and you will get a wall of failures.

Turn on Talk to these backends over https on the pool. Then, on the site, Settings tab, Proxy, with advanced settings shown, there are two more:

• Send SNI to backend. Leave this on. Without it a backend hosting several names has no idea which certificate to present, and hands over the wrong one.
• Verify backend certificate. Turn it on once the backend has a certificate that actually checks out. Leaving it off means the traffic is encrypted but you have not established who you are talking to, which is half a job.

Open the site, Settings tab, tick Show advanced settings, then Proxy, and tick Use my error pages for backend errors.

With it off, a 500 from your application is passed through exactly as the application wrote it, stack trace and all. With it on, the visitor gets the page you designed under Error Pages.

Careful with an API. An API client wants the JSON error the application sent, not a nicely designed HTML page it cannot parse. Turn this on for the site and off for the API path.

Open the pool, go to the Health checks tab, and tick Check these backends on a schedule. Set the type to HTTP and give it a path.

With it off, nginx only notices a bad backend after a real request to it has already failed, which means a visitor got the error first.

Point it at something that genuinely tests the application, not the front page. A server whose web process is running but whose database has gone still accepts a connection and still serves a front page, so a shallow check calls it healthy while every visitor gets an error.

A path that touches the database is the right kind of check.

Fill in Body must contain with a short string that only appears when things really work. Plenty of applications answer 200 while rendering an error page, and this is the cheap insurance against that.

Five to ten seconds suits nearly everybody. Lower notices a failure sooner and asks your servers more often. If the check is expensive, make the endpoint cheaper rather than the interval longer.

Raise Fall so it takes more consecutive failures to be marked down, and raise Rise so it takes more passes to come back. Reloads are already debounced, so twenty backends flapping at once is one reload rather than twenty.

If it keeps flapping, the reload rate is not really the problem. Find out why the backend keeps failing.

Set Check port on the pool. Useful when the application serves traffic on one port and exposes a health endpoint on another.

Fill in Host header on the pool. Needed when the backend hosts several sites and picks one by name, otherwise your check reaches whichever site that server considers its default.

That is not a bug, it is the useful part. Each node checks separately and the answers are kept apart. A backend one node can reach and another cannot is a routing or firewall problem, and seeing it split out that way usually tells you the answer immediately.

Point them at a path you can filter out, or at a route your application logs at a lower level. One line per check per node is the cost of knowing a backend is alive before a visitor does.

They should not. The checker sends the token too, precisely so this does not happen. If they do fail, check the backend is looking for the header name the pool page shows, and that you have not rotated without activating.

Stop the application on one backend, or take it off the network. Within a few seconds the pool should show that server red and the others green, and the site should keep working throughout. Start it again and watch it come back.

If the site kept working but the pool never went red, your health check is not testing anything useful.

1. Point the name at one of your load balancers in DNS and wait for that to be true everywhere.
2. Check port 80 is open from the internet and the site answers on it.
3. Certificates, then Get one from Let's Encrypt.
4. Type the domain. Add more names if one certificate should cover several.
5. Press Request and wait about thirty seconds.
6. Open the site, choose the certificate, turn on https and Force https, then apply.

Settings, Certificates, untick Use the staging service, save, then issue the certificate again. The staging certificate is replaced and nothing else needs changing.

Staging is on by default because the real service allows five certificates a week for the same names, with no way to appeal, and everybody gets the setup wrong at least once.

Renewal needs the name to point at one of your nodes and port 80 to be reachable from the internet. Two causes cover nearly all of these:

• A firewall closing port 80.
• Somebody turned off Listen on http for the site because everything is on https anyway. Turn it back on. Force https already means no real visitor stays there.

The certificate list shows the failure and the reason, and it keeps trying, so one bad night does not become an outage a month later.

A wildcard needs dns-01 validation, which means the authority asks you to put a record in your DNS rather than serve a file.

1. Settings, Certificates, choose your DNS provider and put in the credentials. They are stored encrypted and never shown again.
2. Ask for a certificate covering *.example.com.
3. Choose dns as the validation method.

Add the bare domain as a second name if you want it covered, because a wildcard does not cover the apex.

Certificates, then Upload one. Paste the certificate, with any intermediates below it, and the matching private key.

The key is checked against the certificate before anything is saved, which catches the most common mistake at the point of pasting rather than when a site stops working. Intermediates are put in the right order for you.

Certificates, then Make a signing request. Choose 2048 or 4096 bits. Both are fine: 2048 is faster and 4096 is what some auditors ask for.

You get a block of text to send to your supplier. The private key is made here and stays here, which is the point of doing it this way. When the certificate comes back, find the request in the list and press Paste Certificate.

Get three things from your supplier, usually on a page called ACME or automation: the directory URL, a key identifier (EAB KID), and an HMAC key.

1. Security, Paid Authorities, Add an authority.
2. Name it, paste the directory URL, the key identifier and the HMAC key.
3. Save, then press Test. That registers with the supplier and asks nothing else of them, so it is free and safe.

After that, asking it for a certificate is the same as asking Let's Encrypt, with one extra choice of who to ask.

Certificates, then Make a self signed one. Every browser shows a warning for these, because nothing has vouched for them. Fine for testing and for something only your own systems talk to. Never on a site the public visits.

At every renewal the name is looked up against public DNS and compared with what each node reports its own public address to be. The node that matches is the one that runs certbot, because it is the only one the authority can reach.

The challenge token goes to every node before validation starts, because the authority picks which address it connects to and you do not get a say.

On purpose. It fires on every box, and on a node that does not currently own the public address the challenge fails every single time. You end up with a mailbox full of failures from two of your three nodes and the real problem buried in it.

Renewal is driven from the manager, which works out which node can actually answer.

Settings, Certificates, Renew this many days before expiry. Thirty is the default and it is what Let's Encrypt suggest for ninety day certificates. That leaves a fortnight to notice a failure and still have plenty of time.

One that is not being used by any site can be deleted. One that is in use cannot, and the page says which site is using it. Point that site at something else first.

Settings, Certificates, Contact email. It is genuinely optional: certificates issue with no address at all, and the dashboard warns you either way. Use a shared mailbox more than one person reads, not somebody's personal address, because certificates outlive jobs.

The certificates page shows how many nodes hold each one. A certificate that only made it to one node is visible here rather than a surprise at the next failover. If one is missing, fix the peer link and reissue.

Pick the same certificate on each site. As long as the names on the certificate cover those sites, it works. A wildcard on ten subdomains is a common and sensible arrangement.

The trade off: one large certificate means one failed renewal takes every one of those sites with it.

Use dns-01 validation. http-01 needs the authority to reach the name over port 80, which it cannot do for something internal. With DNS credentials configured, the record goes into your zone and the name never has to be reachable.

Nothing gets deleted until the new certificate is live, so this can be done in the middle of the day.

1. Check the name points at one of your nodes and port 80 is open from the internet. Let's Encrypt has to reach it.
2. Certificates, Get one from Lets Encrypt.
3. Use Take the names from a site and pick the site. That fills in every name the site answers to, which is the step people get wrong when they type it by hand and leave out the www.
4. Leave the challenge on http, serve a file on port 80 unless you need a wildcard, which needs the dns one.
5. Key type ECDSA unless you still support very old clients.
6. Press Create and wait about thirty seconds.
7. Open the site, Basics tab, and change Certificate to the new one. Save, then Apply config.
8. Load the site and check the padlock. The issuer should now say Let's Encrypt.
9. Certificates, then Delete the old one. It will refuse while anything is still using it, which is a useful check that you did not miss a site.

Renewal from then on is automatic, and the manager works out which node can answer the challenge each time. If the bought certificate still has months left, there is no harm in leaving it in the list until it expires. It costs nothing and it is one keystroke back if you need it.

One thing to check first: if the certificate you bought covers something Let's Encrypt does not do, such as an organization validated or extended validation certificate that shows your company name, the free one is not the same product. For ordinary domain validated certificates, which is nearly all of them, it is exactly the same product for nothing.

The same shape, backwards. Get the certificate into the list first, then point the site at it, then tidy up.

1. Certificates, Make a signing request. Fill in the names it should cover, your company details and the country. The private key is made here and never leaves.
2. Send the request text to your supplier.
3. When the certificate comes back, find the request in the list and paste the certificate into it.
4. Open the site, Basics tab, change Certificate, save, then Apply config.

If the supplier offers ACME automation, set them up under Security, Paid Authorities instead. Then renewal is automatic and you never do this by hand again.

Open the site, tick Listen on https, choose a certificate, tick Redirect http to https, and apply. Leave Listen on http on so renewal keeps working.

With no certificate attached, the https block is left out altogether. That is on purpose: nginx refuses to start at all if it finds a TLS listener with no certificate.

TLS 1.3 and 1.2 are on by default, and 1.1 and 1.0 are off. Both of the old ones are formally deprecated and turning them on is a compliance failure in most audits. There is almost never a good reason.

The dashboard shows a post quantum ready badge next to the OpenSSL version when the build supports it. From a terminal:

openssl list -kem-algorithms | grep -i mlkem
nginx -V 2>&1 | tr ' ' '\n' | grep -i openssl

Any external TLS testing service will also tell you which groups your site offers.

It is a site setting. Start with a short max age, confirm everything on that name works over https, then raise it.

HSTS is very hard to take back. A browser told two years will refuse plain http for that name for two years, whatever you do afterwards.

It is on by default and you should leave it on. Without it, a visitor's browser may go and ask the certificate authority itself, which is slower and tells that authority which sites the visitor is looking at.

The default is a modern list and most people should leave it alone. Two situations justify a change: an audit that names specific suites, or a client you cannot change that needs something the default leaves out. In the second case, add the one thing rather than replacing the whole list.

For TLS 1.3 the suite list is short and fixed by the standard, so most cipher tuning only affects TLS 1.2 traffic.

Turn on Use TLS on the pool and leave verification on. If your backends use certificates your company issued, put your authority file on the load balancers and point the pool at it.

Turning verification off works and gives up most of the point: an unverified TLS connection protects against passive listening and not against somebody in the middle.

Open the site, Settings tab, tick Show advanced settings, then TLS, Require a client certificate. Three choices: not needed, require a valid one, or ask for one and allow the request either way.

The middle option is a genuinely strong door. Nobody without the certificate gets past the handshake, so your application never sees the request at all. Good for an admin interface, a partner API, or anything where you control both ends.

The third is for the case where you want the certificate as extra information and the application decides what to do about it.

You need a certificate authority of your own to issue and revoke the client certificates. That part is on you, and losing control of it is worse than not having done it.

Open the site, Settings tab, advanced settings shown, TLS, tick HTTP/3 (QUIC). Open UDP port 443 on your firewall, which people forget because everything else about 443 is TCP.

HTTP/3 mostly helps on connections that lose packets, which means phones and bad wifi. On a wired connection you will not notice. Browsers fall back to HTTP/2 on their own, so turning it on is low risk.

Open the site, Settings tab, advanced settings shown, TLS, untick Session tickets.

Session tickets let a returning visitor skip most of the handshake, which is a real saving. The trade is that if the ticket key ever leaks, recorded traffic can be decrypted after the fact. Turn them off where that matters more than the handshake time.

Session cache timeout in the same section is the other half of resumption and does not carry the same trade, so leaving that on is usually the better compromise.

Open the site, Settings tab, advanced settings shown, TLS. Post-quantum key exchange offers it and lets older clients fall back. Require post-quantum key exchange refuses anything that cannot do it.

Offer it everywhere. Require it only where you control the clients, because requiring it turns every older browser away with an error nobody can act on.

The reason to have it on now is that traffic recorded today can be decrypted later, once the machines exist. Anything with a long secrecy life is worth protecting before the machines arrive, not after.

1. Make a pool for the API servers.
2. Open the site, Paths, Add a path.
3. Path /api, match type Prefix, mode Proxy, and choose the API pool.
4. Leave the strip prefix setting off if the application was written knowing it lives under /api.
5. Save and apply.

As soon as a site has any paths at all, it stops using the single backend on the Basics tab and serves only what is listed under Paths. A request that matches nothing gets nginx's own 404.

Add a path of / pointing at your main pool to catch everything else.

The page was found and its stylesheet was not. It looks like an appearance problem and it is really the path prefix.

Either the application has not been told which prefix it lives under, or the prefix is being stripped when the application expects it, or it is not being stripped when the application does not expect it. Flip the strip setting and try again.

Add a path such as /downloads, set the mode to Static, and give it a folder. Everything else keeps going to the pool. The files have to exist on every node.

Make a rate limit zone on the Zones and Access page, then add a path for /login and choose that zone on the path rather than on the whole site. Everything else is untouched.

nginx does not read them top to bottom. It picks the most specific match. A request for /api/users matches a path of /api rather than the site's general rule, because /api is more specific. You do not have to put them in any particular order.

Set the match type to Regular expression. Use it for things that are genuinely irregular, such as matching a set of file extensions. Prefix matching is what you want almost every other time, and it is faster.

A React, Vue or Angular app handles its own addresses in the browser. Press refresh on /settings/profile and the browser asks the server for a file at that path, which does not exist.

The tidiest answer is to let whatever serves the app do the fallback, because that is a job the app framework already knows how to do. Add a path for /api/ pointing at the API pool, and a path of / pointing at the pool that serves the app. Nothing special is needed here.

If you would rather serve the build straight off the load balancer, set the site to Serve files off the disk with Document root on the build folder. The fallback itself has no box of its own, so put it in Extra config for this path on a path of /. Press Check it on the nginx config tab before you apply, and it is validated on every node.

Either way, serve the built assets folder as a path of its own, so scripts and images are found as files rather than being answered with the index page. That mistake shows up as an app that loads a blank screen and a console full of unexpected token errors.

Add the path, choose Send it to a pool, and tick Serve a whole website here.

That box tells the load balancer the thing on the other end is a website in its own right, not a piece of yours. Redirects and cookies coming back get their paths corrected so the visitor stays under your path instead of being thrown out to the root.

Without it, a sign in on the embedded site sends the visitor to /login at the top of your domain and everything falls apart at the first click.

Add the path and set What happens to Refuse it. The request never reaches your application.

Useful for an admin endpoint that should never be reachable from outside, an old API version you have retired, or a folder a scanner keeps asking for.

If you want it reachable from the office only, use Who may reach this path with an access list instead. Refusing is for things nobody should get.

Edit the path and tick Settings for this path only. The whole Settings tab opens up for that path alone, and anything you leave alone keeps following the site.

This is how nearly every real site ends up configured. A long timeout on the reports path and a short one everywhere else. Caching on the assets path and nothing anywhere else. A rate limit on the login and none on the rest.

Set up the path to serve files from disk, then open the site, Settings tab, Static Files, and tick Directory listing. Pretty directory listing gives readable sizes and local times instead of the bare version.

Do it as a path with Settings for this path only, not on the whole site. Turning on directory listing everywhere means any folder without an index file becomes a browsable list of what is in it, which is rarely what anybody meant.

Add a path for the health address, tick Settings for this path only, and untick Access log so it stops filling the log.

If the checker needs a fixed body rather than just a status, put the response in Extra config for this path on the same dialog. The nginx config tab shows the result before you apply, and Check it validates it on every node.

Worth knowing: your own uptime monitor should be checking a path that really touches the application, not one the load balancer answers on its own. A health endpoint that always says yes is a monitor that never tells you anything.

1. Zones & Access, Rate limits tab, add one.
2. Name it something you will recognize, such as login-limit.
3. Counts by $binary_remote_addr, rate 10r/s, memory 10m.
4. Save, then choose it on a site or, better, on a path.

A zone that says it is not used yet is doing nothing. Making one is only half the job.

Well above what a real person does. Somebody reading a site makes a handful of requests a second while a page loads, so ten a second stops abuse without anybody noticing.

Set it too low and real visitors get errors, which is a worse outage than the one you were protecting against, and harder to spot because most people just leave rather than complaining.

Zones & Access, Connection limits tab. Name it, count by $binary_remote_addr, give it 10m of memory, then choose it on a site or path.

Be careful with a low number. A browser opens several connections at once on purpose, so a limit of one or two breaks ordinary visitors. Ten is a gentle starting point.

No. The counters live in nginx shared memory on each box, so they are per node. With two nodes and traffic split between them, a visitor could get up to twice the configured rate in the worst case. Size the limit with that in mind.

Ask for a page faster than the limit and confirm you get refused, then confirm normal use does not. A limit nobody has tested is a setting, not a protection, and the two look identical right up until they do not.

for i in $(seq 1 40); do curl -s -o /dev/null -w "%{http_code} " https://example.com/login; done

Open the site, Settings tab, Limits and Timeouts, and raise Maximum upload size. A request over the limit gets 413 Request Entity Too Large.

Raise Upload timeout at the same time. A 500MB file over a home connection takes several minutes, and the default timeout will cut it off long before it finishes even though the size limit was fine.

Set it on the upload path rather than the whole site where you can. A site wide 500MB limit is an invitation to anybody who wants to fill your disk.

Open the site, Settings tab, Rate Limiting, Bandwidth limit per connection. That is per connection, not per visitor, so somebody using four connections gets four times the cap.

Full speed for the first lets the beginning through untouched and then applies the cap. Set it to a few megabytes and small files, pages and images are never slowed at all, while a big download settles into the limit after the first moment.

That combination is what keeps one person pulling a large file from eating the whole line without making the site feel slow for everybody else.

Open the site, Settings tab, Limits and Timeouts. Four boxes, all doing the same kind of job:

• Header timeout: how long a client may take to send its request headers. This is the one that stops the classic attack of opening hundreds of connections and sending one byte a minute.
• Upload timeout: the same for the body.
• Response send timeout: how long to wait while the client accepts the answer.
• Keep-alive timeout: how long an idle connection is kept for the next request.

The defaults are already short enough to handle the obvious version of this. Only raise them for a real reason, such as genuinely large uploads, and raise the one that matters rather than all four.

A rate limit of 10 a second does not mean ten every second. It means one every tenth of a second, and a page that pulls twelve images at once has eleven of them refused.

Set Burst allowance to soak that up, then decide what happens to the burst:

• Serve burst immediately ticked: the burst is answered at once and only what is over it is refused. This is what you want almost always.
• Unticked: the burst is queued and released at the limit rate. The requests all succeed, but they feel slow.

A burst of 20 with immediate serving is a reasonable starting point for a normal web page.

Open the site, Settings tab, Rate Limiting, tick Show advanced settings, and set Rate limit status code.

The default is 503, which means the server is unavailable. 429 is more honest, because it says too many requests and clients that understand it will back off and try again rather than treating it as an outage.

Set it to 429 if anything automated talks to this site. Leave it alone if you do not care.

1. Zones & Access, Caches tab, add one.
2. Name it static-files, folder /var/cache/nginx/static, index memory 10m, maximum size 1g, inactive 60m.
3. Choose it on a path that covers your static content rather than on the whole site.

• Almost always: images, fonts, stylesheets, compiled scripts, downloads.
• Often: API responses that are the same for everybody.
• With care: whole pages for signed out visitors, keyed so a signed in visitor never gets one.
• Never: anything that depends on who is asking, unless the cache key says who is asking.

A cached page belonging to one signed in user being served to another is a data breach, not a glitch. It is the single most common way caching goes wrong.

The generated log format includes $upstream_cache_status, so the access log shows HIT, MISS, BYPASS and EXPIRED per request.

sudo tail -f /var/log/nginx/access.log

A cache that never reports HIT is costing you disk and giving you nothing. The usual reason is a response header from your application telling nginx not to store it.

nginx is built with the cache purge module, so purging individual objects is available. The blunt approach of deleting the cache folder and reloading also works and is sometimes the honest answer.

No. Each node caches on its own disk. A warm node and a freshly restarted node behave differently for a while, and that is normal.

This is the single best reason to have a cache at all. Open the site, Settings tab, Caching, tick Show advanced settings, and tick the conditions that should serve a stale copy: Backend errored, Backend timed out, and 500, 502, 503 and 504.

Now a backend falling over means visitors get yesterday's page instead of an error page. For a news site, a documentation site or a marketing site, that is the difference between a bad afternoon and an outage anybody notices.

Tick Refresh stale entries in the background too, so a visitor never waits for a refresh. They get the cached copy immediately and the fetch happens behind them.

Open the site, Settings tab, Caching, with advanced settings shown, and tick Add an X-Cache-Status header.

Every response now carries HIT, MISS, BYPASS, EXPIRED or STALE. Look at it in the browser network tab or with curl -I.

MISS every time means something is stopping the cache, and it is usually a Set-Cookie on the response or a Cache-Control saying no. BYPASS means one of your own skip rules is firing. Leave the header on while you are tuning and turn it off afterwards if you would rather not publish it.

Open the site, Settings tab, Caching, and use the two boxes:

• Skip the cache when: the cache is not read. The request goes to the backend, and the answer may still be stored.
• Never store when: the answer is not written to the cache at all.

Put your session cookie in both. Skipping without never storing means the first signed in visitor caches their own page and everybody after them gets it, which is a data leak rather than a performance problem.

Set both. Every time.

Open the site, Settings tab, Caching, advanced settings shown, and set Cache after this many requests. At 3, a page has to be asked for three times before a copy is kept.

On a site with a very long tail, that stops the cache filling with pages one person looked at once, and keeps the disk for the things people actually come back for.

Leave it at 1 for a small site. There is nothing to gain from being clever about a cache that is not under any pressure.

Open the site, Settings tab, Static Files, Browser cache time. This is separate from the cache on the load balancer. It tells the visitor's own browser not to ask again.

A long time is right only if the file name changes when the content does, which is what a build tool with hashed file names gives you. Then a year is safe and correct.

If your files keep the same names between releases, keep this short, an hour or less. Otherwise a returning visitor is stuck on the old copy and no amount of deploying will shift it.

Open the site, Settings tab, Caching, advanced settings shown, and tick Allow cache purging. That turns on a purge request for one address, so a page you just corrected can be refreshed without throwing away everything else.

Put it behind an access list. A purge endpoint anybody can reach is a way to make your cache useless on demand, which is a cheap way to knock a site over.

Open the site, Settings tab, Caching, advanced settings shown, Cache key.

By default the key includes the query string, so ?ref=twitter and ?ref=email are two separate cached copies of an identical page. On a site with tracking parameters flying around, that can mean thousands of copies of one page and a cache that never hits.

Take the query string out of the key for pages where it does not change the content. Do not do it for a search page, where the query string is the whole point.

A popular page expires, a hundred requests arrive at once, all hundred miss, and all hundred go to the backend for the same thing. That is a stampede, and it is how a cache that was helping suddenly becomes the reason a site fell over.

Open the site, Settings tab, Caching, advanced settings shown, and tick Only let one request refresh a page. One request goes through. The other ninety nine wait for it, or get the stale copy if you also turned on serving stale while refreshing.

Open the site, Settings tab, Compression, tick Enable gzip, save, then Apply config.

HTML, CSS, JavaScript and JSON shrink to somewhere between a fifth and a third of their size. On a phone on a slow connection that is the difference between a page arriving and a visitor leaving.

The default type list already covers the usual text formats. Add your own under Compress these content types if you serve something unusual.

The scale runs 1 to 9 and the useful part of it is 4 to 6. Five is a good default.

Going from 5 to 9 costs several times the processor time and buys a few percent of size. On a load balancer handling every response for every site, that trade is a bad one. Below 4 you are paying for the machinery and not getting much back.

Only compress files bigger than is worth leaving alone too. Compressing a 200 byte response usually makes it bigger, because the compression header costs more than it saves.

By default nginx does not compress anything it got from a proxied backend, which surprises people whose whole site is proxied.

Open the site, Settings tab, Compression, tick Show advanced settings first if you cannot see it, and set Compress proxied responses to Always compress them.

If the backend already compresses, you do not need this and should not turn it on. Check with a request and look at whether the response comes back compressed already.

Tick Serve pre-compressed .gz files under Compression. If a visitor asks for app.js and app.js.gz is sitting next to it, the ready made one is sent and nothing is compressed at request time.

This only helps for files you serve off disk, and only if your build produces the .gz files. When it applies, it is the best of both: maximum compression, no processor cost per request.

Because something between you and the visitor is caching, and it has to know that the compressed and uncompressed versions of a page are different things.

Without that header, a cache can hand a compressed copy to a client that said it cannot handle compression, and that client gets a page of nonsense. Leave Add Vary: Accept-Encoding ticked. It is on by default for exactly this reason.

Open the site, Settings tab, Headers, Custom response headers. One per line, name then value. Save, then Apply config.

This is where a header goes if it is not one of the named boxes further down. Cache hints for a CDN in front, a build number so you can tell which release answered, or a header your monitoring looks for.

Open the site, Settings tab, Headers, and tick Enable CORS. Then:

1. Allowed origins: the sites allowed to call this one, one per line, each with its scheme. https://app.example.com, not app.example.com.
2. Tick the methods you actually use. GET is always allowed. POST, PATCH, DELETE, OPTIONS and HEAD each have a box.
3. Allowed request headers: anything unusual the caller sends, such as Authorization or a custom API key header.

Do it on the API path rather than the whole site if the site also serves pages. And resist putting a star in Allowed origins on anything that needs a login. A star means every website on the internet, which is fine for a public price list and very much not fine for anything else.

A preflight is an OPTIONS request the browser sends first, asking whether the real request is allowed. Three things trip it up, in this order:

• OPTIONS is not ticked in the methods list. The preflight itself is an OPTIONS request, so leaving it out blocks everything.
• The header is not in the list. If the caller sends Authorization and you did not list it in Allowed request headers, the browser stops before it ever sends the real request.
• The origin does not match exactly. Scheme, name and port all have to match. https://app.example.com and https://app.example.com:443 are the same thing to a person and different strings to a browser.

The browser console names which check failed, and it is worth reading properly rather than guessing. It is more specific than it looks.

Tick Allow credentials under Headers. The caller also has to ask for it, so their fetch needs credentials: "include".

You cannot use a star in Allowed origins with this on. Browsers refuse that combination outright, because it would let any site on the internet make requests as your signed in user. List the real origins.

Open the site, Settings tab, Security, tick HSTS (force https) and set HSTS duration.

HSTS tells a browser never to use plain http for this name again, for as long as the duration says. That closes the gap where somebody types the name, gets sent to http first, and can be intercepted before the redirect happens.

Start at a day while you check nothing broke. Move to a year once you are sure. Once a browser has been told, you cannot take it back, so do not set a year on a name you might want on plain http next month.

Apply HSTS to all subdomains covers every name under this one, including names you have not made yet. Powerful and easy to regret.

Open the site, Settings tab, Security, Frame embedding policy:

• Nobody can frame this site. The right answer for anything with a sign in page.
• Only this site can frame itself. Use this when your own pages embed each other.
• Do not send the header. Anybody may frame you.

This is what stops clickjacking, where somebody puts your page in an invisible frame over their own and collects clicks meant for something else.

Write the policy in Content Security Policy under Security, and tick CSP in report-only mode before you save.

Report only means the browser tells you what would have been blocked and blocks nothing. Leave it that way for a week, watch the browser console on the real site, and widen the policy until it stops complaining. Then untick it.

A policy applied straight to a real site takes out the analytics, the fonts and half the images within about a minute, and the failures are silent unless somebody happens to have the console open. Report only first, every time.

Tick Stop content type sniffing under Security.

Without it, some browsers ignore what you said a file is and decide for themselves based on what is inside it. An upload saved as a text file can be treated as a script and run. With the box ticked, the browser believes what you said and nothing else.

There is no real downside as long as your server sends the right types, which this one does.

Open the site, Settings tab, Security, Referrer policy. Every link out of your site normally tells the other site which page the visitor came from, query string and all.

Send only the domain, and only over https is a good default. The other site learns you sent the traffic and learns nothing about which page or which search.

Tighten it further if your addresses contain anything private, such as a document identifier or a reset token. Those end up in other people's logs otherwise.

Open the site, Settings tab, Security, Server header. Remove it completely is the default and the right answer. There is also an option to replace it with something of your own, and one to leave nginx alone.

Separately, Show nginx version controls whether the exact version number goes out. Leave that off. A version number turns a general scan into a targeted one.

None of this is real security on its own. It just means somebody has to do the work rather than being handed the answer.

Open the site, Settings tab, Security, Permissions policy. This is where you say the page has no business asking for the camera, the microphone, the location, or the accelerometer.

It matters most when your page embeds anything you did not write. Turning off what you never use costs nothing and removes a whole shelf of things a script could try.

Open the site, Settings tab, Security, and tick both Block hidden files and Block common exploit patterns.

The first stops anything starting with a dot being served, which means .git, .env and editor backup files. Every one of those has leaked real credentials from real sites, and it is the first thing any scanner asks for.

The second turns away the well known probe addresses before they reach your application. It is not a replacement for the WAF. It is the cheap version that costs no processor time and catches the laziest traffic on the internet.

1. Error Pages, New template.
2. Name it, then write a page for each code you care about. 404 and 502 are the two worth doing first.
3. Save, then open a site and choose the template.

A template only covers the codes it was ticked for. Anything else falls back to the stock nginx page.

There is a setting per site for whether your templates are also used for errors your application returns. Turn it off and your application's own error pages are passed through untouched.

Make a second template that returns a small JSON body, and choose it on the API path rather than on the whole site. A styled apology is no use to a mobile app.

Keep it short and say what the visitor should do. A page that says the site is having a problem and to try again in a few minutes is worth more than a clever design.

Leave the technical detail out. It does not help the visitor and it tells anybody probing your site more than they need to know. Inline any styling, because a stylesheet on the backend that just went down will not load.

Open the site, Settings tab, tick Show advanced settings, then Proxy, and tick Use my error pages for backend errors.

Without it, a 500 from your application goes to the visitor exactly as written, which on a bad day is a stack trace with file paths and sometimes a database name in it.

Turn it off again for API paths, where the caller wants the real error body.

Error Pages, open the template, and use the Pictures section. The image is stored with the template and copied to every node, so it works when the backend is down. That is the whole point, and it is why linking to an image on your own site does not work here.

Keep it small. An error page that takes three seconds to load is worse than a plain one.

1. Maps, Add one.
2. Choose what it reads, such as the browser type or the requested host.
3. Name the variable it sets. You invent this name and then use it elsewhere.
4. Add the rules, most specific first. The first match wins.
5. Say what to use when nothing matches.

A plain word matches exactly. A squiggle in front makes it a pattern, and a squiggle and a star makes the pattern ignore capital letters.

Make a map keyed on the client address, set a variable, and use it on the sites you want protected. Then push updates through the JSON API rather than editing by hand.

Each update means a config render and a graceful reload, so this suits a list you change a few times an hour rather than thousands of times a minute.

The variable a map sets is checked before it is saved. A name nginx does not recognize would stop nginx starting on every server at once, so it is refused at the point of typing instead.

That is what maps are for. Make it once under Maps, and every site can read the variable it sets. A blocklist, a redirect table, a country to language table, all in one place.

The alternative is the same list pasted into six sites, five of which are out of date by Christmas.

These are hostnames on the map dialog matters if the values are domain names, because it makes the matching work the way names work, with the wildcard on the left.

1. Make a pool with the backend servers in it. Stream pools are separate from the pools sites use.
2. TCP & UDP, Add one.
3. Choose TCP, set the listen port, choose the pool.
4. Save and apply.

For long lived connections such as a database, least connections spreads load better than round robin.

Turn on the proxy protocol setting, but only if the backend understands it. A backend that does not will fail to read the connection at all, and the failure looks like the service being completely broken rather than like a setting being wrong.

Same steps, choose UDP instead of TCP. Health checking UDP is genuinely hard, so where the same host exposes a TCP port, check that instead.

nginx is built with the stream TLS module, so you pick a certificate the same way a site does. Useful for putting TLS in front of something that does not speak it, such as an internal service.

TCP & UDP, open the listener, and set Who can connect to an access list. Make the list first under Zones & Access, Access lists.

This matters more for TCP than for a website. A database port open to the internet is found by a scanner within hours, and unlike a web request there is no application login page in front of it doing rate limiting for you.

TCP & UDP, open the listener, and use Idle timeout and Connect timeout.

Raise the idle timeout for a database connection pool, which holds connections open for a long time on purpose and will reconnect constantly if you cut it too short. Keep the connect timeout short so a dead backend is noticed quickly.

TCP & UDP, open the listener, and tick Expect the PROXY protocol from whatever is in front.

Only tick it if the thing in front really sends it. A listener expecting the PROXY protocol header treats a normal connection as malformed and drops it, which looks exactly like a broken backend and takes a while to work out.

1. Settings, WAF rule set, choose the long term support track and press Install it now. Check every node got it.
2. Open one site, WAF tab, set the mode to Watching, refuse nothing. Apply.
3. Leave it a week and look at what fired.
4. Run a learning session against a proper test pass and save a profile.
5. Apply the profile and stay in watching mode for a few more days.
6. Move to Block known malicious. Watch for a week.
7. Only then consider full enforcement, one strictness step at a time.

Going straight to blocking on a live site is how people conclude that WAFs do not work.

Settings, WAF rule set. Choose a track, save the page first, then press Install it now. It goes to every node, because a node without the rules refuses the whole configuration and the apply is turned away.

Having the rules on a node does not switch anything on. A site is only checked once its own WAF setting is turned on, and that starts in watching mode.

A new rule can start refusing something your own application does, on a site already in blocking mode. That is the whole trade off.

1. Open the site, WAF tab, Learning session.
2. Put in the addresses your testers will come from. An office, a VPN, a build runner, somebody's laptop.
3. Name the round, such as "checkout regression, August release".
4. Press Start watching.
5. Exercise the application properly. Click everything, upload something, use the search, post text with quotes in it.
6. Press Stop watching, look at what fired, tick what is your application rather than an attack, and save it as a profile.

Because the internet does not stop while you test. Without a filter the session would collect every scanner and every genuine probe that arrived during the window, and you would end up excusing rules on the strength of somebody else's attack.

Do not raise the strictness or turn the rule off globally. Run a learning session, reproduce the action, and tick that specific rule on that specific path as an exclusion. It stops that one rule firing on that one path and changes nothing else.

Paths that hold an id are collapsed to a star automatically, so /user/*/profile covers the account created tomorrow.

Check them before saving. A star in the wrong place allows more than you meant. A missing one refuses everything created after today, and that failure only shows up when a new record is made, which could be weeks later.

Save the exclusions as a profile, then set each site to that profile. Editing it changes every site using it at the next apply, which is the point of having them. Every profile card lists the sites relying on it.

Separate from the attack rules. Off means the rules decide on their own and anything they do not recognize reaches your application, which is how nearly every WAF works and is the right default.

The stricter settings only allow paths your testers actually reached. Only sane for an application whose surface is genuinely fixed, and it will refuse anything you forgot to test.

A 403. If the site has an error page template covering 403, they see your page rather than the bare nginx one, which is worth setting up before you start blocking.

Inspecting every request is not free. Expect a measurable increase in processor use and a small amount of added latency. On the fleet the numbers on this site came from, the load balancers ran at around 8% of sixteen cores under 2,000 requests a second, so there is plenty of headroom. Measure your own rather than trusting that.

The rules have not been installed yet. Settings, WAF rule set, save the page, then Install it now. It takes a few seconds. Until then the WAF cannot be applied, and the site page says so rather than letting you configure something that would fail.

Open the site, WAF tab, set the mode to Off, and apply. Nothing is checked for that site and requests reach your backend exactly as they did before. The rules stay installed on the nodes and other sites are unaffected.

1. Settings, Country database. Choose db-ip, which is free with no account, or MaxMind if you have one. Save, then Install it now.
2. Open the site, Geo blocking tab.
3. Untick the countries you do not want. Ticked countries can reach the site.
4. Save and apply.

A VPN gets around this in one click, so it is for cutting scanner noise and meeting a rule about where a site may be served, not for keeping anybody determined out.

Settings, Country database. db-ip publish one every month and ask for nothing but attribution. MaxMind GeoLite2 is free but needs an account.

Both publish monthly, so a monthly refresh is plenty. Daily costs a download and finds nothing new most days. The active node does it for the whole fleet.

The address nginx sees is the CDN's, so you are looking up the wrong country entirely. Turn on the setting that trusts the real client address from a header, and only when you know exactly what is in front and have named it as trusted.

Right most of the time and wrong some of the time. Addresses move between regions, mobile carriers route traffic through unexpected places, and corporate VPNs make a person in one country look like they are in another. Expect a small number of legitimate visitors to be refused, and decide whether that is acceptable before turning it on for a public site.

1. Zones & Access, Access lists, add one.
2. Put your office ranges under Allowed, one per line.
3. Set When nothing matches to deny.
4. Tick Include the management networks so you do not lock yourself out.
5. Choose the list on the /admin path rather than on the whole site.

Make an access list with When nothing matches set to allow, and put the address under Blocked. That is a public area with exceptions, which is the opposite shape to a private area.

Turn on Ask for a username and password on the access list, or the equivalent switch on the site. Reasonable as a second layer on top of an address check. Thin on its own: it is one shared secret, it is often written down, and it never gets rotated.

Almost always one of three things.

• Your real address is IPv6 and the list only names IPv4. Put both forms in.
• Your office address changed.
• You did not tick Include the management networks.

Fix it from a node with a shell, or from an address that is on the list, then apply.

Only when something trustworthy sits in front of your load balancers, such as a CDN, and you have named it as trusted. If you turn it on with nothing trustworthy in front, anybody can set that header themselves and walk straight through your access list.

There is no partial version of this. Either the header is trustworthy or your list is decoration.

Settings, Management networks. One CIDR per line. Anything that ticks Include the management networks picks them up, along with every cluster node from the roster.

This is not the same as the list that controls who can reach the GUI. That one lives in the environment file on each node, because it has to be in force before the login page renders.

1. Open the site, Login watch tab, tick it on.
2. Put in the login paths, one per line.
3. Set which methods count, usually POST.
4. Set which status codes mean a failure. This is the one to get right.
5. Leave the thresholds at their defaults to start, then save.

Spraying is one password tried once against a hundred accounts from a hundred addresses. No lockout fires and no rate limit fires, which is exactly why it works.

There is no universal answer. A form that re-renders with "wrong password" answers 200 and answers 302 when it worked, so for that application 200 is the failure. An API answers 401.

Press Look at the log now, sign in wrongly once, and see exactly which code your application produced. Get it backwards and every successful login is reported as an attack, which is worse than no detection because people learn to ignore the alerts.

It reads the access log, so it sees the address, the path, the method and the status. It never sees a username and it never sees a password.

Seeing usernames would mean reading request bodies, which is where the passwords are, and a security feature that starts reading passwords has become the problem it was supposed to prevent.

All three spray rules have to hold at once, because any one alone is an ordinary Monday morning. A busy login page has plenty of failures. A large office has plenty of addresses. Everybody typing their password wrong once is exactly "few per address".

If you get false alarms, raise the minimum number of addresses first. That is the number that separates a spray from a bad morning.

The same test run over hours instead of minutes, because somebody who knows detectors use five minute windows simply waits six. It costs nothing to run both.

Login watch reports rather than blocks, because a false positive would lock out a whole office. Once you know the shape is real, add the addresses to an access list or tighten the rate limit on the login path.

1. Users, Add a user.
2. Give them a username and pick a role. Viewer unless you know they need more.
3. Set a first password and tell them to change it.

They set up their own authenticator app the first time they sign in. Accounts are shared across every node, so you add them once.

Give people the smallest role that lets them do their job. It is easy to raise somebody later and awkward to explain an outage caused by an account that did not need the rights it had.

Turn the account off rather than deleting it. Turning it off ends their sessions immediately, so a browser they left open somewhere stops working. Keeping the account means the audit log still makes sense, because entries against a deleted user are much harder to follow a year later.

An admin can reset it from the Users page. If no admin can get in at all, use the console command on any node.

sudo docker exec nginx-fleet-manager \
  python -m app.reset_admin alice --reset-totp

1. Settings, Signing in to this GUI.
2. Choose Local accounts or single sign on, which is the mode to test from.
3. Fill in the discovery URL, client id and client secret from your provider.
4. Set the redirect address to the outside address of this GUI including any prefix. The software adds /auth/oidc/callback, and that whole address has to be registered with the provider.
5. Name the claim that holds group names, and the three groups for admin, operator and viewer.
6. Press Test the discovery URL, then sign in that way once.

Because no admin has completed a single sign on login yet, so switching now would lock everybody out. It is the only setting on that page that can leave nobody able to sign in, and it fails quietly.

Choose the middle mode, sign in through the provider once, then come back.

Name a group for each role. Highest wins, so somebody in the admin group and the viewer group is an admin. That means adding a person to the admin group works without taking them out of anything first.

Somebody in none of the three groups cannot sign in at all, which is how you take access away: remove them at the provider and they are out of every node at once.

Run this from the install directory on any node:

sudo ./auth-recovery.sh --allow-local

Local passwords come straight back. It needs a shell on the box, which is the point: the way back in must not depend on the thing that broke.

Audit Log. Every configuration change, every apply, and every sign in including the failed ones, with the actor and the source address.

The log is written on the node where the thing happened and does not travel. If you cannot find something, check the other node.

Deliberate. A log that traveled between machines could claim something was done on a server that never did it, which would make it useless as a record. The small inconvenience of looking in two places is the price of the log meaning what it says.

Through the JSON API, which is the sensible way to get it into whatever you use for log analysis.

Open the site, Settings tab, Logging, Log format. Three choices:

• Standard combined format. What every log tool already understands.
• Adds timings and upstream info. The same, plus how long the request took, which backend answered and how long that backend took. This is the one to be on while you are chasing anything slow.
• JSON, one object per line. For shipping into a log collector.

The middle one is worth having on permanently. When somebody says the site felt slow last Tuesday, it is the difference between an answer and a shrug.

Set Log format to JSON, one object per line. Every line is a complete object with named fields, so nothing downstream has to guess where one field ends and the next begins.

Anything that quotes a user agent or a referrer containing a space breaks a naive parser of the text format sooner or later. JSON does not have that problem, and it costs a little more disk in exchange.

A health check hit every two seconds is 43,000 lines a day and none of them tell you anything.

1. Open the site, Paths tab, and add the path, for example /health.
2. Tick Settings for this path only.
3. Untick Access log for that path.

The site keeps its log. That one path stops writing to it. Do the same for a metrics endpoint or anything else being polled.

Open the site, Settings tab, Logging, Error log level. It runs from debug to crit and it sits on warn, which is the right place to leave it.

Debug is genuinely huge. It writes several lines per request and it will fill a disk overnight on a busy site. Turn it on, reproduce the problem, turn it straight back off. Setting a reminder is not a joke here.

Also tick Log missing files while you are looking for a 404 that should not be happening. It records the full path nginx tried, which usually makes the answer obvious.

Every request arrives from the CDN, so the log records the CDN address and nothing else. The real address is in a header the CDN adds.

1. Zones & Access, Access lists, and edit or make one.
2. Tick Trust a forwarded client address.
3. Put the CDN address ranges in Trusted proxies.
4. Pick that access list on the site.

Only list the ranges the CDN publishes. Trusting the header from everybody means anybody can say they are anybody, and every address based rule you have stops meaning anything.

1. On the running node, go to Cluster and press Add a node. You get a command with a token in it. It works once and expires in two hours.
2. On the new machine, run it:

sudo git clone https://git.failoverlb.net/failoverlb.git /data/docker/failoverlb \
  && sudo /data/docker/failoverlb/failoverlb_setup.sh \
       --join 203.0.113.10:7444 --token <token> \
       --allowlist 198.51.100.25/32

Then build nginx on it too. The new node joins as standby and pulls the current configuration, so you do not set your sites up twice.

It contains an .env file with that node's identity and its secret key. The second machine comes up believing it is the first one.

The installer now refuses to join when it spots this, and a machine set up before that check will sit there failing quietly. It also means both machines share one secret key, which they should not, so treat that key as needing to be changed.

Look at the Cluster page on the first node.

• Missing entirely: the join did not finish and the installer output says why. The usual cause is port 7444 being blocked between the machines.
• There but saying its credentials were refused: nearly always a name problem rather than a certificate problem. Check what the second node thinks it is called.

grep NFM_NODE_NAME /data/docker/failoverlb/.env

If that says the name of your first node, the settings file was copied. Delete it and install again so it makes its own.

Exactly the same as adding the second. Get a token from any node already in the cluster and run the installer on the new machine with the join details. Quorum works better with an odd number, so three is a genuinely better shape than two if you have the machines.

Cluster page, use the failover button. Doing it deliberately is calmer than pulling the plug and letting the software work it out, and it lets you confirm the other node is healthy first. Use it before any planned work.

Turn on forced standby for that node. It stays in the fleet and stays in sync, and it will not take over.

Remember to turn it off afterwards. A fleet where one node can never take over is a fleet with no failover, and nothing will remind you.

Order matters, and this order is the whole point.

1. Check both nodes are online, and that nothing is saved but unapplied.
2. Turn off its DNS answer and wait at least twice the TTL.
3. Hand over if it is the active one.
4. Turn on forced standby.
5. Do the work.
6. Coming back: check it is online, turn off forced standby, turn the DNS answer back on, then watch it for a few minutes.

Health first, then being allowed to be in charge, then traffic.

The standby notices the heartbeats have stopped, promotes itself, and carries on. Nobody has to be woken up and nothing has to be typed. Traffic already going to the survivor is unaffected throughout.

What this does not fix is traffic that was going to the dead node. That decision was made before the connection was attempted, so only DNS can move it.

Cluster page, Remove next to the node. Do it while the fleet is healthy rather than after the machine has already gone, so the roster is tidy and DNS answers tied to that node get cleaned up.

Cluster page, Sync now. Normally you do not need it, because every apply goes to every node. It is useful after a node has been offline and you want to confirm it has caught up rather than waiting.

Port 7444 between them is the usual answer. Neither will promote itself over the other in a two node cluster, because neither can see a majority, so both keep serving with the config they have. Fix the link and they catch up.

1. Delegate a piece of your domain, such as gslb.example.com. Never the whole domain.
2. DNS Failover, Add a zone. Give one nameserver name per node, such as ns1.example.com and ns2.example.com. You are inventing these now.
3. Add a name, such as www, type A, TTL 30.
4. Add one answer per node. Pick the node and its address fills in.
5. Create the records the page prints at your domain host.
6. Point the name people actually use at it with a CNAME.

Two kinds and you need both. The page prints them with your addresses already filled in.

The glue records are the ones people forget, and forgetting them looks exactly like the feature being broken.

Open the hosted zone for example.com, which is the parent of the part you are delegating. Create these records there, not in a new hosted zone.

Do not create a hosted zone in Route 53 for gslb.example.com. Your nodes are the nameservers for it now, and a hosted zone as well gives two different sets of servers claiming the same name.

Ask each node directly, with no port on the end.

dig @203.0.113.10 www.gslb.example.com +short
dig @203.0.113.11 www.gslb.example.com +short

Both should answer and both should list every healthy node. Then ask the internet rather than the nodes, which tests the whole path:

dig shop.example.com +short

In this order, because the first two are where nearly all of these end.

1. Ask the node directly with no port on the end. If that answers, the software is working and the problem is the delegation.
2. Check the records at your domain host against the ones printed on the zone. The glue records are the ones people miss.
3. Look at the DNS listener panel on each node. It reports what that node is really doing, which is not always what the page shows.
4. Remember a name only starts working once the old answer has been forgotten everywhere, which can take longer than the TTL suggests.

The part that answers DNS runs without root on purpose, and a program without root cannot bind port 53. So it listens on 5353 instead, on UDP and TCP both, and a firewall rule the software manages sends UDP and TCP port 53 to it.

If you test against 5353 you get an answer whether or not the public path works. Always test with no port on the end, because that is the path your visitors use.

Thirty seconds is a sensible low number and is what the software suggests. Going much lower mostly buys extra lookups without moving anybody faster, because resolvers and browsers have their own ideas and some ignore very short values entirely.

DNS Failover, edit the name, and turn off the answer for that node. Save. Wait at least twice the TTL before doing anything else, because that is the time for computers elsewhere to forget the old answer.

Skipping that wait is the difference between nobody noticing and a few people getting errors.

Set If every node is down to hand them out anyway. If the health check itself is what broke, an empty answer takes the site down when it did not have to be.

The name at the very top of a domain cannot be a CNAME, so an apex needs addresses rather than an alias. Either delegate the apex, which moves everything including your mail records onto these nodes, or use your provider's own failover feature for that one name.

Removal is deliberately slower than restoring, so a brief hiccup does not move everybody.

1. Fleet, Tunnel. Turn the tunnel on and choose an address range. 10.77.0.0/24 is fine unless something on your network already uses it.
2. Install tunnel support on every load balancer when the page offers to.
3. Add a workload, give it a name, and copy the command it shows.
4. Run that command on the machine you want to reach.
5. Note the tunnel address it gets, add it to a pool with the port the service listens on, and apply.

• Installs WireGuard if the machine does not have it.
• Makes a key pair on the machine and sends only the public half back.
• Writes one network interface called nfm0.
• Adds a route to the load balancers and to nothing else.
• Leaves a small helper command for checking and undoing it.

What it does not do: change the default route, send that machine's own traffic anywhere, or open any inbound port. You can read the script first by leaving the last part off the command.

Check the Tunnel page first. A workload that has joined shows as on the tunnel. One still waiting has not run the command, or the command failed part way.

If it shows as on the tunnel and the backend is still down, the tunnel is up and the service behind it is not. Check the service is listening, on the port you put in the pool.

A workload connects to every load balancer and each connection is separate. One working and one not usually means the workload cannot reach one of them: a firewall on the way, or an address family problem such as a load balancer published on IPv6 only while the workload has IPv4 only.

Use Change the setup to publish a name or address the workload can reach. A name with both an A and an AAAA record is the most flexible.

Commands expire after two hours. Click Show the command next to the workload for a new one. Asking for a new one retires the old one immediately, so a link that went astray stops working.

1. Take the address out of any pool that uses it, and apply. Otherwise the pool points at something that has gone.
2. Tunnel page, Remove next to the workload.
3. Copy the command shown and run it on the machine.
4. Press Remove it here.

It deliberately leaves the WireGuard package installed, because that machine may be using it for something else.

sudo nfm-tunnel status

If it reports the tunnel is not up, the interface was never created. The usual reasons are a container whose kernel has no WireGuard, or a firewall that allows outbound traffic generally and drops the tunnel port, which is UDP.

Not once workloads are enrolled on it. Changing it would strand them, so the page refuses. Remove the workloads first if you really need a different range.

Press Apply config. Everything you have saved is built into a full configuration, staged on every node, tested with each node's own nginx binary, and only then swapped in.

If any node refuses, every node keeps what it had and you are told exactly what was wrong.

You saved it and did not apply it. Saving changes nothing for your visitors, which is the point: you can save as often as you like and apply when you are ready. The banner at the top of every page says when something is saved and not live.

The Changes page shows what is saved and not yet live, and what it changes. Read it before applying, particularly if somebody else has been working on the same thing, because you are about to make their changes live along with your own.

Changes page, look at the history, and put an earlier version back. It applies the same way any other change is applied: built, checked, staged on every node, tested, then swapped in. So a rollback cannot break the fleet either.

The most common one is a pool with no healthy servers in it, which means the site would serve errors. A warning is not a refusal: sometimes serving an error page from a pool you are about to fill is exactly what you meant to do.

It does not block the change. The change goes into that node's queue and is replayed when it comes back, so it catches up rather than quietly drifting.

No. An nginx reload is graceful: the master starts new workers with the new config, hands them the new connections, and lets the old workers finish what they were doing before they stop. Nobody is disconnected and no request is dropped.

The one thing to know is that very long lived connections, such as WebSockets, keep old workers around until they close.

Account, Backup. Type your password in the Take a backup panel and press Download a backup. The file is named with the date and time, so several of them sit together without overwriting each other.

Your password is what encrypts the file, which is why it is asked for.

Sites, pools, zones and access lists, TCP and UDP services, error pages, certificates, paid authorities, settings and accounts, plus the certificate files themselves.

Left out on purpose: node names, cluster membership and each server's identity. Those belong to the machines rather than to the configuration.

• Not only on the load balancers. A backup that lives on the machine it protects is no help when that machine is what failed.
• Somewhere access is controlled, because the file is equivalent to your private keys.
• Somewhere you will still have access if you are locked out of this system, since restoring is how you get back in.

Delete old copies rather than letting them accumulate. Every one stays valid for as long as the keys inside it do.

1. Account, Backup, Restore panel. Choose the file and type the password it was made with.
2. Press See what is in it. This opens the archive and reports what it holds without changing anything.
3. Read the report, especially the addresses it came from.
4. Type restore in the confirmation box and press the button.

Take a backup of what is there now first, if there is any chance you will want it back.

Build one node as normal, restore the backup onto it, then join a second node to it. The restore repoints DNS failover answers to the new nodes automatically, matching the kind of address, and reports anything it could not match by name.

The management network allowlist is deliberately not restored, because restoring it is a good way to be locked out of the system you are in the middle of recovering.

The password is the one that was typed when the backup was taken, which is that person's login password at that time. If they have changed it since, the backup still wants the old one. A backup taken by somebody else needs their password, not yours.

Guard .env separately. Without it a restored database cannot read its own encrypted values.

The configuration does. Each machine still points at the old load balancers, so run the enrollment command again on each one from the Tunnel page of the new fleet.

Fleet, NGINX Build. Run the check button first, every time: it works out what a rebuild would do without doing it, including whether your current configuration would still be accepted.

Then press Rebuild. The new nginx is built while the old one carries on serving, your config is tested against it, and only if that passes is the running nginx replaced.

Rebuild one, check your sites still work, and only then do the second. Rebuilding both at once removes the whole point of having two. If you use DNS failover, take the node out of the answer first.

Fix it. A fleet where they differ is one where a configuration can work on one node and be refused by the other, and the apply then fails with an error that points at your configuration rather than at the real cause. Rebuild the one that is behind.

The old nginx keeps running, so nothing is down. Read the output on the page: a build failure is nearly always a missing system package or a source that could not be downloaded, and both say so plainly. Fix it and press the button again.

A rebuild does not change your configuration and cannot lose it.

sudo /data/docker/failoverlb/nginx-installer.sh revert

That rolls back to the build from before the last install.

Turning the WAF on in the GUI rebuilds nginx with ModSecurity, using the same path the version upgrades already take, so nobody has to edit a file. From a terminal it is:

sudo WITH_MODSECURITY=1 /data/docker/failoverlb/nginx-installer.sh install

It is not built by default because it adds about two minutes to a four minute build and most installs never switch it on.

Do not install python3-certbot-nginx or anything else that pulls in Ubuntu's nginx package. It drops a stock binary on top of this build, and you lose every module and the configuration stops loading. The installer sets up guards against this and it is still worth knowing.

The module list is a plain array near the top of nginx-installer.sh. Add an entry in the same folder|git-url|git-ref shape and rebuild. Do it on one node, confirm it works, then do the other.

cd /data/docker/failoverlb && sudo git pull
sudo /data/docker/failoverlb/install.sh --upgrade

Do every node, minutes apart rather than days, and check the Cluster page in between. It does not touch your configuration or your environment file, and nginx keeps running throughout.

Utilization. Requests, connections, processor and memory over time, with windows from the last fifteen minutes to the last thirty days. A number on its own tells you about this second, and a line tells you whether it is normal.

Put the address pattern in the pool settings, using {host} where the server address goes, so one pattern covers every member. The load balancer reads numbers your servers already publish. Many do without anybody setting it up.

It has no login on your application servers and does not want one. Holding a password for every machine in the estate would make this the most valuable box on your network.

No measurement was taken then, usually because something was restarting. It is drawn as a gap and not as zero, because zero would read as the server being idle when it was actually being restarted.

Connections nginx accepted and could not handle. It is nearly always zero. When it is not, something is wrong and it is worth finding out what.

They are worked out between two samples fifteen seconds apart, so after a restart there is nothing to compare against for half a minute.

Not available today. stub_status is a whole server figure, and per site numbers would need log parsing, which is on the roadmap rather than in the product. The generated log format already carries everything it would need.

1. Traffic, Import.
2. Paste the file your sites are actually in, usually from sites-enabled or conf.d. A top level nginx.conf is normally just a list of includes and shows you almost nothing.
3. Press See what it would make. This writes nothing at all.
4. Read the Left alone section properly, untick anything you do not want, then import.

cat /etc/nginx/sites-enabled/*

Everything arrives switched off and unapplied, so you can look it over before turning it on.

Your configuration names certificate files by path. Those paths point at files on the machine the configuration came from, and the private keys are not in the text you pasted anyway.

So a site comes across knowing it uses https and with no certificate chosen. Afterwards, either get a new one from Let's Encrypt or upload the existing certificate and key.

Correct. You had two blocks for one name, one of which only forwarded to the other. That is one site here with the force https switch turned on, not two sites. The note under the site says exactly that.

Your configuration pointed at a pool declared in a file you did not paste. Paste that file too and preview again. Order does not matter, so you can paste several files one after another into the same box.

The importer reads nginx configuration only. Coming from something else means building the sites and pools by hand, which for a handful of sites is an afternoon. Start with one site, get it working end to end, then repeat.

1. Build the new fleet and configure the sites, but do not point any DNS at it.
2. Test by adding the new addresses to your own hosts file, or by asking the new node directly with curl and a Host header.
3. Get certificates on the new fleet using dns-01, so you do not have to move traffic first.
4. Lower the TTL on the names you are moving, well in advance.
5. Move one name at a time and watch it.

curl -sk -H "Host: shop.example.com" https://203.0.113.10/ -o /dev/null -w "%{http_code}\n"

The load balancer refused the request before your application saw it. Open the site, Settings tab, Limits and Timeouts, and raise Maximum upload size.

Then check the application has its own limit set at least as high. PHP has two of its own, and a load balancer set to 100MB in front of a PHP set to 2MB just moves the error one step later and makes it harder to find.

In order:

1. Is Enable WebSocket support ticked on the site, Settings tab, WebSocket? This is the answer most of the time.
2. Is the site on HTTP/1.1 to the backend, under Settings, Proxy? An upgrade cannot happen over HTTP/1.0.
3. If it connects and then dies after a minute, that is the idle timeout, not the connection. Raise WebSocket idle timeout.
4. If you have paths, is the WebSocket path one of them? A site with paths serves only what is listed, so /socket.io/ needs to be there.

The browser console usually shows a 400 or a 426 for the first two, and a clean close for the third, which is a quick way to tell them apart.

CORS is enforced by the browser, not by the server, so the request very often reached your application and came back fine. Check the browser console for which check failed, then:

• Is Enable CORS ticked on the site, Settings tab, Headers?
• Is the calling site listed in Allowed origins, with its scheme and exactly as the browser sees it?
• Is OPTIONS ticked in the methods, so the preflight can happen?
• Is every unusual header the caller sends listed in Allowed request headers?

Curl will not show you this problem, because curl does not enforce CORS. A successful curl proves nothing here.

Turn on the timing log first. Open the site, Settings tab, Logging, and set Log format to Adds timings and upstream info. Now every line records how long the request took and how long the backend took.

Compare the two numbers:

• Backend time high, total close to it: the application is slow. The load balancer is passing that on faithfully.
• Backend time low, total much higher: the time is going in transfer, which usually means a slow client or a large uncompressed response. Check compression is on.
• Both low but the site feels slow: it is not the request, it is the number of them. Look at the Utilization charts and at how many things one page pulls.

502 means the load balancer could not get an answer from your application. Work through these in order.

1. Open the pool and look at the members. If they are marked down, the health check is failing and the problem is behind the load balancer, not in it.
2. If the pool uses https to reach the backend, try turning certificate verification off. If that fixes it, the proper fix is to point the pool at the right authority file rather than leaving verification off.
3. Check the port. A pool pointing at the wrong port fails exactly like an application being down.

The backend accepted the connection and did not answer in time. That is your application being slow rather than being down. Look at what it is waiting for, usually a database or a downstream service. Raising the proxy timeout hides the symptom and makes the queue longer.

Four candidates, in order of how often they happen: an access list on the site or the path, country blocking, the WAF in a blocking mode, or a backend enforcing a trust token that nginx is not sending. The audit log and the WAF hit list narrow it down quickly.

It should not be possible, because the config is tested on every node before any of them use it. If it happens anyway, the previous configuration is on the node in /var/backups/nginx-fleet-manager and can be put back by hand. Then open the Changes page and go back to the last version that worked.

1. Check you are coming from an address on the allowlist. It is checked before the login page renders, so a wrong address gives you nothing rather than an error.
2. Check the container is running: sudo docker ps.
3. Read its log: sudo docker logs nginx-fleet-manager.
4. Check port 7443 is open from where you are.

You are on the standby. Check the top right of the screen and open the other node. This is the single most common confusion in the whole product.

It is being controlled by the environment file rather than by the screen. Settings has a section at the bottom called Set in the environment listing everything in that state. When both have a value, the file wins.

Check in this order: does the name point at your load balancer, is the port open from outside, does a site have that exact server name, is the site enabled, and has the config actually been applied. Four of those five are outside this software, which is usually where the answer is.

dig shop.example.com +short
curl -sv https://shop.example.com/ 2>&1 | head -20

The manager log and what you were doing at the time is nearly always enough for somebody to work out what happened. Include which node you were on, whether it was active or standby, and the config version from the dashboard.