xDev Asia

A lesson on Load Balancing in Nginx — algorithms (round-robin, least_conn, ip_hash, hash), detailed upstream configuration, backup servers, weights, sticky sessions, and health checks. Guide to setting up a load balancer for high-availability and performance optimization with real-world examples.

Nginx from Basics to Advanced

Part 2: Reverse Proxy & Load Balancing

xdev.asia

1. Load Balancing Algorithms

Load balancing is a technique for distributing traffic across multiple servers to optimize resource usage, increase throughput, reduce latency, and ensure high availability.

1.1. Round-Robin (Default)

Round-robin distributes requests sequentially to each server in rotation.

Configuration:

upstream backend {
    # Round-robin is the default, no declaration needed
    server backend1.example.com;
    server backend2.example.com;
    server backend3.example.com;
}
server {
listen 80;
server_name example.com;
location / {
    proxy_pass http://backend;
}

}

How it works:

Request 1 → backend1
Request 2 → backend2
Request 3 → backend3
Request 4 → backend1 (repeats)
Request 5 → backend2
Request 6 → backend3
...

Advantages:

Simple, easy to implement
Distributes requests evenly
No state/session tracking required

Disadvantages:

Does not account for current server load
Not suitable if servers have different capacities
Does not maintain session affinity

Use cases:

Stateless applications
Servers with identical configuration
Simple load distribution

Detailed example:

upstream web_backend {
server web1.example.com:8080;
server web2.example.com:8080;
server web3.example.com:8080;
server web4.example.com:8080;
}
server {
listen 80;
server_name www.example.com;
access_log /var/log/nginx/loadbalancer.log;

location / {
    proxy_pass http://web_backend;
    
    # Standard headers
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    
    # Add backend server info to response header
    add_header X-Upstream-Server $upstream_addr always;
}

}

1.2. Least Connections (least_conn)

Least connections routes requests to the server with the fewest active connections.

Configuration:

upstream backend {
least_conn;  # Enable least connections algorithm
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;

}
server {
listen 80;
location / {
    proxy_pass http://backend;
}

}

How it works:

Initial state: backend1: 0 connections backend2: 0 connections backend3: 0 connections Request 1 → backend1 (0 connections) → backend1: 1 connection Request 2 → backend2 (0 connections) → backend2: 1 connection Request 3 → backend3 (0 connections) → backend3: 1 connection backend1 finishes → backend1: 0 connections Request 4 → backend1 (0 connections, least)

backend2 still processing: 1 connection backend3 still processing: 1 connection Request 5 → backend1 (0 connections, still least)

Advantages:

Better load distribution when requests have varying processing times
Automatically adjusts for busy/idle servers
Good for long-lived connections

Disadvantages:

Higher overhead than round-robin (must track connections)
Does not maintain session affinity

Use cases:

Applications with variable request processing time
Long-lived connections (streaming, downloads)
Servers with different capacities

Example with monitoring:

upstream api_backend {
least_conn;
# Server configurations
server api1.example.com:3000 max_fails=3 fail_timeout=30s;
server api2.example.com:3000 max_fails=3 fail_timeout=30s;
server api3.example.com:3000 max_fails=3 fail_timeout=30s;

# Keep connections alive
keepalive 32;
keepalive_timeout 60s;

}
server {
listen 80;
server_name api.example.com;
location / {
    proxy_pass http://api_backend;
    
    # HTTP 1.1 for keepalive
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    
    # Headers
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    
    # Add connection count header (for debugging)
    add_header X-Upstream-Connections $upstream_response_length;
}

}

1.3. IP Hash (ip_hash)

IP hash routes requests from the same client IP to the same server (sticky sessions).

Configuration:

upstream backend {
ip_hash;  # Enable IP-based sticky sessions
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;

}
server {
listen 80;
location / {
    proxy_pass http://backend;
}

}

How it works:

Client IP: 192.168.1.100 Hash(192.168.1.100) → backend2 → All requests from 192.168.1.100 go to backend2 Client IP: 192.168.1.101 Hash(192.168.1.101) → backend1 → All requests from 192.168.1.101 go to backend1

Client IP: 192.168.1.102 Hash(192.168.1.102) → backend3 → All requests from 192.168.1.102 go to backend3

Advantages:

Session affinity (same client → same server)
No need for shared session storage
Simple and efficient

Disadvantages:

Uneven distribution with few clients
Does not work well behind NAT/proxy
Server down → sessions lost

Use cases:

Session-based applications
Applications that need to maintain state
Shopping carts, user sessions

Important notes on ip_hash:

upstream backend {
ip_hash;
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;

# DO NOT use weight with ip_hash
# server backend1.example.com weight=3;  # ← Wrong!

# CAN use backup
server backend4.example.com backup;

# CAN mark down
server backend5.example.com down;

}

Example with session-based app:

upstream session_backend {
ip_hash;
server session1.example.com:8080 max_fails=2 fail_timeout=30s;
server session2.example.com:8080 max_fails=2 fail_timeout=30s;
server session3.example.com:8080 max_fails=2 fail_timeout=30s;

}
server {
listen 80;
server_name app.example.com;
location / {
    proxy_pass http://session_backend;
    
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    
    # Cookie-based backup (if IP hash fails)
    proxy_set_header Cookie $http_cookie;
    
    # Session timeout
    proxy_read_timeout 3600s;
    proxy_send_timeout 3600s;
}

}

1.4. Generic Hash (hash)

Generic hash allows hashing based on any variable.

Basic configuration:

upstream backend {
hash $request_uri consistent;
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;

}
Same URI → Same server
/api/users → always backend2
/api/products → always backend1

Hash based on different variables:

# 1. Hash by URI (cache-friendly)
upstream cache_backend {
hash $request_uri consistent;
server cache1.example.com;
server cache2.example.com;
server cache3.example.com;

}
2. Hash by cookie
upstream cookie_backend {
hash $cookie_sessionid consistent;
server app1.example.com;
server app2.example.com;

}
3. Hash by custom header
upstream custom_backend {
hash $http_x_tenant_id consistent;
server tenant1.example.com;
server tenant2.example.com;

}
4. Hash by query parameter
upstream param_backend {
hash $arg_user_id consistent;
server user1.example.com;
server user2.example.com;

}

Consistent hashing:

upstream backend { # consistent = minimize redistribution when server added/removed hash $request_uri consistent; server backend1.example.com; server backend2.example.com; server backend3.example.com; } Without consistent: Add/remove server → rehash all keys With consistent: Add/remove server → only a portion of keys are rehashed

Advantages:

Flexible (hash by any variable)
Consistent hashing reduces cache invalidation
Good for caching strategy

Disadvantages:

More complex than other methods
Need to understand distribution of hash key

Use cases:

Caching layers (CDN, proxy cache)
Multi-tenant applications
Sharding strategy

Example with caching:

upstream cache_servers {
hash $request_uri consistent;
server cache1.example.com:6379;
server cache2.example.com:6379;
server cache3.example.com:6379;
server cache4.example.com:6379;

}
server {
listen 80;
server_name cdn.example.com;
location /images/ {
    proxy_pass http://cache_servers;
    
    # Caching headers
    proxy_cache_valid 200 1d;
    add_header X-Cache-Status $upstream_cache_status;
    add_header X-Cache-Server $upstream_addr;
}

}

1.5. Random

The Random method selects a server randomly (with optional weights).

Configuration:

upstream backend {
random;  # or: random two least_conn;
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;

}

Random with least_conn:

upstream backend {
# Pick 2 random servers, then select the one with least connections
random two least_conn;
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;
server backend4.example.com;

}

Advantages:

Simple
Good distribution with many servers
Low overhead

Disadvantages:

Not predictable
Does not maintain session affinity

Use cases:

Large server pools
Stateless applications
When predictability is not required

2. Detailed Upstream Block Configuration

2.1. Basic Upstream Configuration

upstream backend {
server backend1.example.com:8080;
server backend2.example.com:8080;
server backend3.example.com:8080;
}
server {
listen 80;
location / {
    proxy_pass http://backend;
}

}

2.2. Server with Multiple Ports

upstream multi_port_backend {
server backend.example.com:8080;
server backend.example.com:8081;
server backend.example.com:8082;
}

2.3. Unix Socket Connections

upstream socket_backend {
server unix:/var/run/app1.sock;
server unix:/var/run/app2.sock;
server unix:/var/run/app3.sock;
}
server {
listen 80;
location / {
    proxy_pass http://socket_backend;
}

}

2.4. IPv6 Support

upstream ipv6_backend {
server [2001:db8::1]:8080;
server [2001:db8::2]:8080;
server [2001:db8::3]:8080;
}

2.5. Mixed Configuration

upstream mixed_backend {
# TCP servers
server backend1.example.com:8080;
server 192.168.1.100:8080;
# Unix sockets
server unix:/var/run/app.sock;

# IPv6
server [2001:db8::1]:8080;

}

2.6. Upstream Zone (Nginx Plus / Shared Memory)

upstream backend {
zone backend_zone 64k;  # Shared memory zone
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;

}
Zone enables:
- Dynamic reconfiguration
- Shared statistics
- Runtime modifications

3. Backup Servers and Weights

3.1. Weight

Weight determines the proportion of requests each server receives.

Basic configuration:

upstream backend {
server backend1.example.com weight=3;  # 60% traffic
server backend2.example.com weight=1;  # 20% traffic
server backend3.example.com weight=1;  # 20% traffic
}
Total weight = 3 + 1 + 1 = 5
backend1: 3/5 = 60%
backend2: 1/5 = 20%
backend3: 1/5 = 20%

Use case 1: Different server capacities

upstream capacity_backend {
# Large server - handle 50% traffic
server large.example.com weight=5;
# Medium servers - 25% each
server medium1.example.com weight=2.5;
server medium2.example.com weight=2.5;

}

Use case 2: Canary deployment

upstream canary_backend {
# Production servers - 90% traffic
server prod1.example.com weight=45;
server prod2.example.com weight=45;
# Canary server - 10% traffic
server canary.example.com weight=10;

}
server {
listen 80;
server_name app.example.com;
location / {
    proxy_pass http://canary_backend;
    
    # Log which server handled request
    add_header X-Upstream-Server $upstream_addr always;
}

}

Use case 3: Blue-green deployment

upstream bluegreen_backend {
# Blue (current) - 100% initially
server blue.example.com weight=10;
# Green (new) - 0% initially
server green.example.com weight=0;

}
Gradual migration:
Step 1: weight=10 / weight=0  (100% blue)
Step 2: weight=9  / weight=1  (90% blue, 10% green)
Step 3: weight=5  / weight=5  (50% each)
Step 4: weight=1  / weight=9  (10% blue, 90% green)
Step 5: weight=0  / weight=10 (100% green)

Weight with least_conn:

upstream weighted_leastconn {
least_conn;
# Weights affect probability, not strict ratio
server backend1.example.com weight=3;
server backend2.example.com weight=1;

}
Server with higher weight is preferred,
but least_conn is still considered

IMPORTANT: Weight does not work with ip_hash

upstream bad_config {
ip_hash;
# Weight is ignored with ip_hash!
server backend1.example.com weight=3;  # ← No effect!
server backend2.example.com weight=1;

}

3.2. Backup Servers

Backup servers only receive traffic when all primary servers are down.

Basic configuration:

upstream backend {
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;
# Backup servers
server backup1.example.com backup;
server backup2.example.com backup;

}

How it works:

Normal operation:

backend1, backend2, backend3 handle traffic
backup1, backup2 idle

backend1 fails:

backend2, backend3 handle traffic
backup1, backup2 still idle

backend2 fails:

backend3 handles all traffic alone
backup1, backup2 still idle

backend3 fails:

ALL primary servers down
backup1, backup2 now handle traffic

backend1 recovers:


Traffic returns to backend1


backup1, backup2 go idle again

Example with maintenance mode:

upstream maintenance_backend {
server prod1.example.com max_fails=3 fail_timeout=30s;
server prod2.example.com max_fails=3 fail_timeout=30s;
server prod3.example.com max_fails=3 fail_timeout=30s;
Maintenance page server
server maintenance.example.com:8080 backup;
}


server {
listen 80;
server_name app.example.com;
location / {
    proxy_pass http://maintenance_backend;
    
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
}

}

Backup with weight:

upstream weighted_backup {
server primary1.example.com weight=3;
server primary2.example.com weight=2;
# Backup servers also have weights
server backup1.example.com weight=2 backup;
server backup2.example.com weight=1 backup;

}
When primaries are down:
backup1 receives 2/3 of traffic
backup2 receives 1/3 of traffic

3.3. Combined Weight and Backup

Scenario: Production + Staging + Emergency

upstream prod_staging_emergency {
# Production servers - main traffic
server prod1.example.com weight=5 max_fails=3 fail_timeout=30s;
server prod2.example.com weight=5 max_fails=3 fail_timeout=30s;
# Staging server - backup (testing purpose)
server staging.example.com weight=2 backup max_fails=5;

# Emergency static server - last resort
server emergency.example.com backup;

}

4. Sticky Sessions

Sticky sessions ensure that requests from the same client are routed to the same server.

4.1. IP Hash Method (Built-in)

upstream backend {
ip_hash;
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;

}

Limitations of ip_hash:

Does not work well behind NAT
Uneven distribution
Not flexible

upstream backend {
hash $cookie_sessionid consistent;
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;

}
server {
listen 80;
location / {
    proxy_pass http://backend;
    
    # Pass cookies
    proxy_set_header Cookie $http_cookie;
}

}

# Nginx Plus only
upstream backend {
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;
sticky cookie srv_id expires=1h domain=.example.com path=/;

}

4.4. Custom Session Management

Backend creates session cookie:

// Node.js example
app.use((req, res, next) => {
if (!req.cookies.server_id) {
// Set cookie with server identifier
res.cookie('server_id', process.env.SERVER_ID, {
maxAge: 3600000,
httpOnly: true
});
}
next();
});

Nginx routing based on cookie:

map $cookie_server_id $backend_server {
"server1" "backend1.example.com:8080";
"server2" "backend2.example.com:8080";
"server3" "backend3.example.com:8080";
default   "backend1.example.com:8080";
}
server {
listen 80;
location / {
    proxy_pass http://$backend_server;
    
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Cookie $http_cookie;
}

}

4.5. Session Affinity with Header

upstream backend {
hash $http_x_session_id consistent;
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;

}
server {
listen 80;
location / {
    proxy_pass http://backend;
    
    # Pass session header
    proxy_set_header X-Session-ID $http_x_session_id;
}

}

5. Active Health Checks

5.1. Passive Health Checks (Open Source)

upstream backend {
server backend1.example.com max_fails=3 fail_timeout=30s;
server backend2.example.com max_fails=3 fail_timeout=30s;
server backend3.example.com max_fails=3 fail_timeout=30s;
}
max_fails=3: Mark down after 3 failures
fail_timeout=30s: Retry after 30 seconds

Detailed parameters:

upstream detailed_health {
server backend1.example.com
max_fails=5          # Number of failures before marking down
fail_timeout=60s     # Wait time before retrying
max_conns=1000;      # Max concurrent connections
server backend2.example.com
    max_fails=3
    fail_timeout=30s;

server backend3.example.com backup;

}

5.2. Health Check Endpoint

Backend health endpoint (Node.js):

const express = require('express');
const app = express();
app.get('/health', (req, res) => {
// Check database
const dbOk = checkDatabase();
// Check memory
const memUsage = process.memoryUsage();
const memOk = memUsage.heapUsed &lt; 500 * 1024 * 1024; // &lt; 500MB

// Check dependencies
const depsOk = checkDependencies();

if (dbOk &amp;&amp; memOk &amp;&amp; depsOk) {
    res.status(200).json({
        status: 'healthy',
        timestamp: new Date().toISOString(),
        uptime: process.uptime(),
        memory: memUsage
    });
} else {
    res.status(503).json({
        status: 'unhealthy',
        database: dbOk,
        memory: memOk,
        dependencies: depsOk
    });
}

});
app.listen(3000);

Nginx health check routing:

upstream backend {
server backend1.example.com:3000 max_fails=3 fail_timeout=30s;
server backend2.example.com:3000 max_fails=3 fail_timeout=30s;
server backend3.example.com:3000 max_fails=3 fail_timeout=30s;
}
server {
listen 80;
server_name example.com;
# Public traffic
location / {
    proxy_pass http://backend;
}

# Health check endpoint (internal only)
location /health {
    access_log off;
    proxy_pass http://backend/health;
    
    allow 127.0.0.1;
    allow 10.0.0.0/8;
    deny all;
}

}

5.3. Active Health Checks (Nginx Plus)

# Nginx Plus only
upstream backend {
zone backend_zone 64k;
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;

}
server {
listen 80;
location / {
    proxy_pass http://backend;
    
    health_check interval=5s
                 fails=3
                 passes=2
                 uri=/health
                 match=health_ok;
}

}
Define health check criteria
match health_ok {
status 200;
header Content-Type = application/json;
body ~ "healthy";
}

5.4. External Health Check Script

Monitoring script (Python):

#!/usr/bin/env python3
health_monitor.py
import requests
import time
import subprocess
from datetime import datetime
BACKENDS = [
'http://backend1.example.com:3000/health',
'http://backend2.example.com:3000/health',
'http://backend3.example.com:3000/health',
]
CHECK_INTERVAL = 10  # seconds
UNHEALTHY_THRESHOLD = 3
backend_fail_counts = {url: 0 for url in BACKENDS}
def check_health(url):
try:
response = requests.get(url, timeout=5)
if response.status_code == 200:
data = response.json()
return data.get('status') == 'healthy'
return False
except Exception as e:
print(f"Error checking {url}: {e}")
return False
def mark_server_down(url):
server_name = url.split('//')[1].split(':')[0]
print(f"Marking {server_name} as DOWN")
def mark_server_up(url):
server_name = url.split('//')[1].split(':')[0]
print(f"Marking {server_name} as UP")
def main():
while True:
for url in BACKENDS:
healthy = check_health(url)
        if healthy:
            if backend_fail_counts[url] &gt; 0:
                print(f"{datetime.now()} - {url} recovered")
                mark_server_up(url)
            backend_fail_counts[url] = 0
        else:
            backend_fail_counts[url] += 1
            print(f"{datetime.now()} - {url} unhealthy "
                  f"({backend_fail_counts[url]} consecutive fails)")
            
            if backend_fail_counts[url] &gt;= UNHEALTHY_THRESHOLD:
                mark_server_down(url)
    
    time.sleep(CHECK_INTERVAL)

if name == 'main':
main()

Run as systemd service:

# /etc/systemd/system/health-monitor.service
[Unit]
Description=Nginx Backend Health Monitor
After=network.target
[Service]
Type=simple
User=nginx
ExecStart=/usr/local/bin/health_monitor.py
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target

sudo systemctl enable health-monitor
sudo systemctl start health-monitor
sudo systemctl status health-monitor

6. Real-world Examples

6.1. High-Traffic Web Application

upstream web_app {
least_conn;
# Primary servers with weights based on capacity
server app1.example.com:8080 weight=5 max_fails=3 fail_timeout=30s max_conns=1000;
server app2.example.com:8080 weight=5 max_fails=3 fail_timeout=30s max_conns=1000;
server app3.example.com:8080 weight=3 max_fails=3 fail_timeout=30s max_conns=800;

# Backup server
server backup.example.com:8080 backup;

# Keepalive connections
keepalive 128;
keepalive_timeout 90s;
keepalive_requests 1000;

}
server {
listen 80;
listen [::]:80;
server_name www.example.com;
# Redirect to HTTPS
return 301 https://$server_name$request_uri;

}
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name www.example.com;
# SSL configuration
ssl_certificate /etc/ssl/certs/example.com.crt;
ssl_certificate_key /etc/ssl/private/example.com.key;
ssl_protocols TLSv1.2 TLSv1.3;

# Logging
access_log /var/log/nginx/webapp.access.log;
error_log /var/log/nginx/webapp.error.log;

# Client limits
client_max_body_size 10M;
client_body_timeout 60s;

location / {
    proxy_pass http://web_app;
    
    # HTTP version for keepalive
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    
    # Standard headers
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-Port $server_port;
    
    # Timeouts
    proxy_connect_timeout 60s;
    proxy_send_timeout 60s;
    proxy_read_timeout 60s;
    
    # Buffering
    proxy_buffering on;
    proxy_buffer_size 8k;
    proxy_buffers 16 8k;
    proxy_busy_buffers_size 16k;
    
    # Add backend info
    add_header X-Upstream-Server $upstream_addr always;
    add_header X-Upstream-Status $upstream_status always;
    add_header X-Upstream-Response-Time $upstream_response_time always;
}

# Static files
location ~* \.(jpg|jpeg|png|gif|ico|css|js|svg|woff|woff2)$ {
    root /var/www/static;
    expires 1y;
    add_header Cache-Control "public, immutable";
    access_log off;
}

# Health check
location /health {
    access_log off;
    proxy_pass http://web_app/health;
    allow 127.0.0.1;
    allow 10.0.0.0/8;
    deny all;
}

}

6.2. API Gateway with Microservices

# User Service
upstream user_service {
least_conn;
server user1.internal:8001 max_fails=2 fail_timeout=20s;
server user2.internal:8001 max_fails=2 fail_timeout=20s;
server user3.internal:8001 max_fails=2 fail_timeout=20s;
keepalive 32;
}
Product Service
upstream product_service {
least_conn;
server product1.internal:8002 max_fails=2 fail_timeout=20s;
server product2.internal:8002 max_fails=2 fail_timeout=20s;
keepalive 32;
}
Order Service
upstream order_service {
ip_hash;  # Sticky sessions for order processing
server order1.internal:8003 max_fails=2 fail_timeout=20s;
server order2.internal:8003 max_fails=2 fail_timeout=20s;
keepalive 32;
}
Payment Service (critical - more redundancy)
upstream payment_service {
least_conn;
server payment1.internal:8004 weight=3 max_fails=1 fail_timeout=10s;
server payment2.internal:8004 weight=3 max_fails=1 fail_timeout=10s;
server payment3.internal:8004 weight=2 max_fails=1 fail_timeout=10s;
server payment_backup.internal:8004 backup;
keepalive 64;
}
Rate limiting zones
limit_req_zone $binary_remote_addr zone=api_limit:10m rate=100r/s;
limit_req_zone $binary_remote_addr zone=payment_limit:10m rate=10r/s;
server {
listen 443 ssl http2;
server_name api.example.com;
# SSL config
ssl_certificate /etc/ssl/certs/api.example.com.crt;
ssl_certificate_key /etc/ssl/private/api.example.com.key;

# Logging with JSON format
log_format api_log escape=json '{'
    '"time":"$time_iso8601",'
    '"remote_addr":"$remote_addr",'
    '"request":"$request",'
    '"status":$status,'
    '"service":"$upstream_addr",'
    '"response_time":$upstream_response_time,'
    '"request_time":$request_time'
'}';

access_log /var/log/nginx/api.access.log api_log;

# Default headers for all services
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Request-ID $request_id;

# User Service
location /api/v1/users {
    limit_req zone=api_limit burst=50 nodelay;
    proxy_pass http://user_service;
    proxy_read_timeout 30s;
}

# Product Service
location /api/v1/products {
    limit_req zone=api_limit burst=50 nodelay;
    proxy_pass http://product_service;
    proxy_read_timeout 30s;
}

# Order Service
location /api/v1/orders {
    limit_req zone=api_limit burst=30 nodelay;
    proxy_pass http://order_service;
    proxy_read_timeout 60s;
}

# Payment Service (stricter limits)
location /api/v1/payments {
    limit_req zone=payment_limit burst=5 nodelay;
    proxy_pass http://payment_service;
    proxy_read_timeout 90s;
    proxy_connect_timeout 10s;
}

# Health checks
location /health/users {
    access_log off;
    proxy_pass http://user_service/health;
    allow 10.0.0.0/8;
    deny all;
}

location /health/products {
    access_log off;
    proxy_pass http://product_service/health;
    allow 10.0.0.0/8;
    deny all;
}

location /health/orders {
    access_log off;
    proxy_pass http://order_service/health;
    allow 10.0.0.0/8;
    deny all;
}

location /health/payments {
    access_log off;
    proxy_pass http://payment_service/health;
    allow 10.0.0.0/8;
    deny all;
}

}

6.3. E-commerce Platform with Session Management

# Frontend servers (stateless)
upstream frontend {
least_conn;
server frontend1.example.com:3000 weight=5;
server frontend2.example.com:3000 weight=5;
server frontend3.example.com:3000 weight=3;
keepalive 64;
}
Backend API (sticky sessions for cart)
upstream backend_api {
ip_hash;
server api1.example.com:4000 max_fails=2 fail_timeout=30s;
server api2.example.com:4000 max_fails=2 fail_timeout=30s;
server api3.example.com:4000 max_fails=2 fail_timeout=30s;
keepalive 32;
}
Checkout service (critical path)
upstream checkout {
least_conn;
server checkout1.example.com:5000 weight=3 max_fails=1 fail_timeout=10s;
server checkout2.example.com:5000 weight=3 max_fails=1 fail_timeout=10s;
server checkout_backup.example.com:5000 backup;
keepalive 16;
}
server {
listen 443 ssl http2;
server_name shop.example.com;
# SSL
ssl_certificate /etc/ssl/certs/shop.example.com.crt;
ssl_certificate_key /etc/ssl/private/shop.example.com.key;

# Frontend (React/Vue/Angular)
location / {
    proxy_pass http://frontend;
    
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    
    proxy_cache_bypass $http_upgrade;
    proxy_redirect off;
}

# API endpoints with session affinity
location /api/ {
    proxy_pass http://backend_api/;
    
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    
    # Pass cookies for session
    proxy_set_header Cookie $http_cookie;
    
    # Timeouts
    proxy_read_timeout 60s;
}

# Checkout (critical path)
location /api/checkout/ {
    proxy_pass http://checkout/;
    
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Cookie $http_cookie;
    
    # Longer timeouts for payment processing
    proxy_read_timeout 120s;
    proxy_connect_timeout 10s;
    
    # Disable buffering for real-time updates
    proxy_buffering off;
}

# Static assets
location /static/ {
    alias /var/www/shop/static/;
    expires 1y;
    add_header Cache-Control "public, immutable";
    access_log off;
}

}

7. Practice Exercises

Exercise 1: Basic Load Balancing

Run 3 instances of an application on ports 3000, 3001, 3002
Configure Nginx with round-robin load balancing
Generate traffic and verify distribution (check logs)

Exercise 2: Weighted Load Balancing

Set up 3 backend servers with weights: 5, 3, 2
Generate 100 requests
Count requests per server and verify ratio (~50%, ~30%, ~20%)

Exercise 3: Sticky Sessions

Set up ip_hash load balancing
Make multiple requests from the same client
Verify all requests reach the same backend

Exercise 4: Backup Servers

Configure 2 primary servers and 1 backup
Stop both primary servers
Verify traffic switches to backup
Start primaries again and verify traffic returns

Exercise 5: Health Checks

Set up backends with max_fails=2 fail_timeout=30s
Configure /health endpoint
Simulate backend failure (stop service)
Monitor logs and verify failover
Restart service and verify recovery

Exercise 6: Canary Deployment

Set up production servers (weight=45 each)
Add canary server (weight=10)
Verify 10% traffic goes to canary
Gradually increase canary weight
Complete migration to canary

8. Troubleshooting

8.1. Uneven Distribution

Problem:

Server 1: 1000 requests
Server 2: 500 requests
Server 3: 300 requests

Diagnosis:

# Check weights grep -A10 "upstream" /etc/nginx/nginx.conf Check if using ip_hash with few clients Check server health/performance

Solutions:

# Use least_conn instead of round-robin
upstream backend {
least_conn;
server backend1.example.com;
server backend2.example.com;
server backend3.example.com;
}
Or adjust weights
upstream backend {
server backend1.example.com weight=1;
server backend2.example.com weight=2;
server backend3.example.com weight=3;
}

8.2. Sticky Sessions Not Working

Problem:

Client requests hitting different servers

Diagnosis:

# Check if behind load balancer/proxy Check X-Forwarded-For header Verify cookie/hash configuration

Solutions:

# Use hash with cookie instead of ip_hash
upstream backend {
hash $cookie_sessionid consistent;
server backend1.example.com;
server backend2.example.com;
}
Or use custom header
upstream backend {
hash $http_x_session_id consistent;
server backend1.example.com;
server backend2.example.com;
}

8.3. Health Check Not Detecting Failures

Problem:

Server down but still receiving traffic

Diagnosis:

# Check max_fails and fail_timeout Test health endpoint manually curl http://backend1.example.com/health Check Nginx error log

sudo tail -f /var/log/nginx/error.log

Solutions:

# Lower thresholds
upstream backend {
server backend1.example.com max_fails=2 fail_timeout=10s;
server backend2.example.com max_fails=2 fail_timeout=10s;
}
Implement proper health check endpoint
Use external monitoring script

8.4. Backend Timeout Issues

Problem:

504 Gateway Timeout errors

Diagnosis:

# Check backend processing time Check network connectivity Review Nginx timeouts

Solutions:

location / {
proxy_pass http://backend;
# Increase timeouts
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 120s;

# Or disable timeout for specific endpoints
proxy_read_timeout 300s;

}

8.5. Connection Pool Exhaustion

Problem:

Too many connections to backend
502 errors during high load

Solutions:

upstream backend {
server backend1.example.com max_conns=1000;
server backend2.example.com max_conns=1000;
# Enable keepalive
keepalive 128;
keepalive_timeout 75s;
keepalive_requests 1000;

}
server {
location / {
proxy_pass http://backend;
proxy_http_version 1.1;
proxy_set_header Connection "";
}
}

9. Best Practices

9.1. Configuration

# 1. Use descriptive upstream names
upstream user_api_backend {  # Good
# ...
}
upstream backend {  # Not descriptive
# ...
}
2. Document your configuration
upstream payment_service {
# Using least_conn for variable processing time
# Strict health checks for critical path
least_conn;
server payment1.internal:8080 weight=3 max_fails=1 fail_timeout=10s;
server payment2.internal:8080 weight=3 max_fails=1 fail_timeout=10s;
server payment_backup.internal:8080 backup;

keepalive 32;

}
3. Separate concerns
Production config
include /etc/nginx/conf.d/upstreams/.conf;
include /etc/nginx/conf.d/servers/.conf;
4. Use variables for repeated values
geo $backend_pool {
default production;
10.0.1.0/24 staging;
}
map $backend_pool $upstream {
production "prod_backend";
staging "staging_backend";
}

9.2. Performance

upstream optimized_backend {
# 1. Use least_conn for better distribution
least_conn;
# 2. Enable keepalive
keepalive 128;
keepalive_timeout 75s;
keepalive_requests 1000;

# 3. Set appropriate max_conns
server backend1.example.com max_conns=1000;
server backend2.example.com max_conns=1000;

}
server {
location / {
proxy_pass http://optimized_backend;
    # 4. Use HTTP/1.1 with keepalive
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    
    # 5. Enable buffering
    proxy_buffering on;
    proxy_buffer_size 8k;
    proxy_buffers 16 8k;
}

}

9.3. Monitoring

# 1. Log upstream info
log_format upstream_log '$remote_addr - [$time_local] '
'"$request" $status '
'upstream: $upstream_addr '
'response_time: $upstream_response_time '
'connect_time: $upstream_connect_time';
access_log /var/log/nginx/upstream.log upstream_log;
2. Add debug headers (non-production)
add_header X-Upstream-Server $upstream_addr always;
add_header X-Upstream-Status $upstream_status always;
add_header X-Upstream-Response-Time $upstream_response_time always;
3. Enable stub_status
server {
listen 8080;
location /nginx_status {
stub_status;
allow 127.0.0.1;
deny all;
}
}
4. Implement health check endpoints
location /health/upstream {
access_log off;
proxy_pass http://backend/health;
allow 10.0.0.0/8;
deny all;
}

9.4. Security

# 1. Limit max connections per server
upstream backend {
server backend1.example.com max_conns=1000;
}
2. Set timeouts
proxy_connect_timeout 10s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
3. Hide upstream errors
proxy_intercept_errors on;
error_page 502 503 504 /50x.html;
4. Rate limiting
limit_req_zone $binary_remote_addr zone=backend_limit:10m rate=100r/s;
location / {
limit_req zone=backend_limit burst=50 nodelay;
proxy_pass http://backend;
}

Summary

In this lesson, you learned:

✅ Load balancing algorithms (round-robin, least_conn, ip_hash, hash, random)
✅ Detailed upstream block configuration
✅ Backup servers and weight distribution
✅ Sticky session strategies
✅ Active and passive health checks
✅ Real-world examples and best practices

Next lesson: We will explore Caching — how to cache static content, API responses, and optimize performance with proxy caching.

Lesson 5: Load Balancing

1. Load Balancing Algorithms

1.1. Round-Robin (Default)

1.2. Least Connections (least_conn)

1.3. IP Hash (ip_hash)

1.4. Generic Hash (hash)

Same URI → Same server

/api/users → always backend2

/api/products → always backend1

2. Hash by cookie

3. Hash by custom header

4. Hash by query parameter

Without consistent:

Add/remove server → rehash all keys

With consistent:

Add/remove server → only a portion of keys are rehashed

1.5. Random

2. Detailed Upstream Block Configuration

2.1. Basic Upstream Configuration

2.2. Server with Multiple Ports

2.3. Unix Socket Connections

2.4. IPv6 Support

2.5. Mixed Configuration

2.6. Upstream Zone (Nginx Plus / Shared Memory)

Zone enables:

- Dynamic reconfiguration

- Shared statistics

- Runtime modifications

3. Backup Servers and Weights

3.1. Weight

Total weight = 3 + 1 + 1 = 5

backend1: 3/5 = 60%

backend2: 1/5 = 20%

backend3: 1/5 = 20%

Gradual migration:

Step 1: weight=10 / weight=0 (100% blue)

Step 2: weight=9 / weight=1 (90% blue, 10% green)

Step 3: weight=5 / weight=5 (50% each)

Step 4: weight=1 / weight=9 (10% blue, 90% green)

Step 5: weight=0 / weight=10 (100% green)

Server with higher weight is preferred,

but least_conn is still considered

3.2. Backup Servers

Maintenance page server

When primaries are down:

backup1 receives 2/3 of traffic

backup2 receives 1/3 of traffic

3.3. Combined Weight and Backup

4. Sticky Sessions

4.1. IP Hash Method (Built-in)

4.2. Hash with Cookie (Better)

4.3. Sticky Cookie (Nginx Plus)

4.4. Custom Session Management

4.5. Session Affinity with Header

5. Active Health Checks

5.1. Passive Health Checks (Open Source)

max_fails=3: Mark down after 3 failures

fail_timeout=30s: Retry after 30 seconds

5.2. Health Check Endpoint

5.3. Active Health Checks (Nginx Plus)

Define health check criteria

5.4. External Health Check Script

health_monitor.py

6. Real-world Examples

6.1. High-Traffic Web Application

6.2. API Gateway with Microservices

Product Service

Order Service

Payment Service (critical - more redundancy)

Rate limiting zones

6.3. E-commerce Platform with Session Management

Backend API (sticky sessions for cart)

Checkout service (critical path)

7. Practice Exercises

Exercise 1: Basic Load Balancing

Exercise 2: Weighted Load Balancing

Exercise 3: Sticky Sessions

Exercise 4: Backup Servers

Exercise 5: Health Checks

Exercise 6: Canary Deployment