Nginx as a Load Balancer: Configuration

Basic upstream configuration

upstream backend {
    # Round-robin by default
    server 10.0.0.1:8080;
    server 10.0.0.2:8080;
    server 10.0.0.3:8080 weight=2;  # 2x more requests

    # Backup server
    server 10.0.0.4:8080 backup;

    keepalive 32;  # Persistent connections
}

server {
    listen 80;
    location / {
        proxy_pass http://backend;
        proxy_next_upstream error timeout;
    }
}

Load balancing algorithms

Least Connections

upstream backend {
    least_conn;
    server 10.0.0.1:8080;
    server 10.0.0.2:8080;
}

Healthcheck (Nginx Plus / nginx_upstream_check_module)

upstream backend {
    server 10.0.0.1:8080 max_fails=3 fail_timeout=30s;
    server 10.0.0.2:8080 max_fails=3 fail_timeout=30s;
}

Real errors during load balancing

The error no live upstreams while connecting to upstream appears when every server in the upstream block is marked unavailable — Nginx has exhausted the max_fails attempts against each one and cannot pick a backend for the request.

2026/09/12 10:14:02 [error] 1421#1421: *88291 no live upstreams while connecting to upstream, client: 5.44.12.9, server: example.com, request: "GET /api/status HTTP/1.1", upstream: "http://backend"

First check that the backends actually respond: run curl -I http://10.0.0.1:8080 from the Nginx server itself. If they are alive but Nginx still treats them as dead, lower fail_timeout or raise max_fails — a server stays marked unavailable for the whole fail_timeout period after the error limit is exceeded.

The error upstream sent too big header while reading response header from upstream means the backend returned response headers larger than the buffer Nginx allocates for them by default.

[error] 1421#1421: *91002 upstream sent too big header while reading response header from upstream, client: 91.200.10.4, server: example.com, upstream: "http://10.0.0.2:8080/"

The fix is to raise proxy_buffer_size and proxy_buffers in the location block, not the upstream block — they control buffering of the proxied response, not backend selection.

Sticky sessions: methods and limits

A stateless balancer works well for stateless backends, but if the application keeps the session in process memory rather than a shared store like Redis, one user's requests must land on the same server every time.

For most projects on open-source Nginx it is more practical to move sessions into a shared store and keep least_conn without server affinity — that way one backend going down does not kill its users' sessions.

When Nginx load balancing does not fit

• You need TCP/UDP-level balancing for non-HTTP traffic — that calls for the stream block instead of http, with a different set of directives.
• You need backends to scale automatically without hand-editing the config — open-source Nginx has no service discovery; you need Consul-template, Kubernetes Ingress, or Nginx Plus with its API.
• You need balancing across data centers that accounts for network latency — that is a job for GeoDNS or Anycast, not an upstream block on one server.
• A single Nginx balancer becomes a single point of failure itself — for resilience you need at least two nodes with keepalived and a shared VIP.
• Backends require TLS termination with client certificates (mTLS) on every node separately — it is simpler to terminate TLS once at the balancer and send plain HTTP to backends inside a private network.

Verifying the result

1. Check the configuration syntax before reloading: nginx -t must return syntax is ok and test is successful.
2. Apply the config without dropping active connections: systemctl reload nginx.
3. Confirm requests are actually distributed: run curl http://yourdomain.com/ a few times and compare which backend answered.
4. Test failover: stop one backend and confirm requests keep being served by the rest without 502 errors on the client side.

Diagnosing 502 and 504 codes from backends is covered in the article on Nginx 502 and 504 diagnostics. Limiting the request rate to each backend is set up with limit_req and limit_conn.

Frequently asked questions

What does the "no live upstreams while connecting to upstream" error mean and how do you fix it?

It means every server in the upstream block is marked unavailable — the max_fails attempt limit has been used up for each one. First check the backends directly with curl from the Nginx server itself: if they're alive, lower fail_timeout or raise max_fails, since a server stays marked unavailable for that whole period once the error limit is exceeded.

Why do sticky sessions based on ip_hash behave unreliably for users sharing one office?

Users behind shared NAT all reach the internet through the same external IP address, so ip_hash routes every one of them to the same backend — if that backend gets overloaded or goes down, the whole group suffers at once. It's more reliable to move session storage into a shared store like Redis and balance without any server affinity.

How do you add a new server to the balancing pool without stopping current traffic?

Add a server line with the new backend's address to the upstream block and apply the configuration with systemctl reload — active connections stay intact, unlike a full service restart. Before folding it into the shared pool, it's worth testing the new server directly with its own curl request first.

An Nginx load balancer can itself become a single point of failure — how do you guard against that?

Yes, a lone Nginx node is a single point of failure regardless of how many backends sit behind it. Resilience needs at least two balancing nodes running keepalived with a shared virtual IP address that switches to the live node automatically if the primary goes down. One balancer with no standby simply cannot offer that guarantee.