Load balancers distribute traffic across multiple servers, enabling horizontal scaling, high availability, and graceful degradation. Understanding Layer 4 vs Layer 7 load balancing, health check mechanisms, and session affinity is critical for building resilient distributed systems.

# Layer 4 vs Layer 7 Load Balancing

Load balancers operate at different layers of the network stack, trading simplicity for functionality.

Layer 4 (Transport Layer) Load Balancing

L4 load balancers make routing decisions based on IP address and TCP/UDP port. They don't inspect application-layer data (HTTP headers, cookies, etc.).

Client Request:
    TCP SYN to 203.0.113.10:443

L4 Load Balancer:
    - Sees: source IP, dest IP, dest port (443)
    - Picks backend: 10.0.1.5
    - Forwards TCP connection to 10.0.1.5:443
    - All packets in this TCP connection go to same backend

Simple forwarding, no protocol knowledge needed

Characteristics:

• Fast: minimal processing, low latency (~1ms added)
• Protocol-agnostic: works with any TCP/UDP application (HTTP, gRPC, databases, SSH)
• High throughput: can handle millions of connections per second
• Simple health checks: a TCP connection attempt or a protocol-specific application probe; some L4 products health-check UDP services through a separate TCP/HTTP/HTTPS endpoint

Limitations:

• Can't route based on URL path, HTTP headers, or cookies
• Cannot make HTTP content-based routing decisions; TLS may be passed through or terminated, depending on the product and listener configuration
• Limited visibility into application-layer failures

Layer 7 (Application Layer) Load Balancing

L7 load balancers understand application protocols (HTTP, gRPC) and make routing decisions based on request content.

Client Request:
    GET /api/users HTTP/1.1
    Host: api.example.com
    Cookie: session_id=abc123

L7 Load Balancer:
    - Terminates TCP/TLS connection
    - Parses HTTP request
    - Sees: path=/api/users, cookie=abc123
    - Routes to backend based on path or cookie
    - Uses an existing upstream connection or opens one to the backend
    - Forwards HTTP request

Full protocol awareness, can modify requests/responses

Characteristics:

• Content-based routing: send /api/* to API servers, /static/* to CDN
• TLS termination: decrypt the client connection at the load balancer, then use HTTP or HTTPS to backends according to configuration
• Advanced health checks: HTTP 200 response, JSON validation
• Request modification: add headers (X-Forwarded-For), rewrite URLs
• Session affinity via cookies

Limitations:

• Slower: ~5-10ms added latency (protocol parsing, SSL termination)
• Lower throughput: tens to hundreds of thousands requests/sec
• More CPU intensive (TLS encryption/decryption)

Comparison Table

# Health Checks and Failover

Health checks inform backend eligibility. Unhealthy backends normally stop receiving new traffic, but all-unhealthy and existing-connection behavior is product- and configuration-dependent.

Health Check Types

TCP Health Check (L4):

HTTP Health Check (L7):

Load balancer sends HTTP request to backend
    GET /health HTTP/1.1
    Host: backend.internal

Backend responds:
    HTTP/1.1 200 OK
    {"status": "healthy", "db": "connected"}

Success example: HTTP 200 response received (accepted status matcher is configurable)
Failure example: a status outside the configured matcher, timeout, or connection refusal

Advanced health checks: Validate response content (e.g., check for specific JSON field), test database connectivity, verify disk space. More accurate than simple TCP checks.

Failover Behavior

After a backend crosses its configured unhealthy threshold, many products normally stop assigning new traffic to it. All-unhealthy behavior and the handling of existing connections vary by product and configuration:

• Immediate removal: New connections only. Existing connections remain until closed.
• Connection draining: Wait for existing connections to finish (timeout), then remove backend. Graceful.
• Forced termination: Close all connections immediately. Fast but disruptive.

Timing parameters: Health check interval (10s typical), failure threshold (3 consecutive failures = unhealthy), recovery threshold (2 consecutive successes = healthy). Tuning these controls failover speed vs false positive rate.

# Session Affinity (Sticky Sessions)

Session affinity attempts to send requests from the same client to the same backend. It may be needed when an application keeps unreplicated session data in backend-local memory; replicated or external session state can remove that requirement.

Cookie-Based Affinity (L7)

Advantage: Can survive a load-balancer restart if the restarted instance can interpret the cookie and retains compatible target configuration.

Disadvantage: If Backend A fails, session is lost (unless session data is replicated).

Source IP Affinity (L4)

Client IP: 198.51.100.5
    LB hashes IP: hash(198.51.100.5) % num_backends = 2
    Routes to Backend 2 while the hash inputs and eligible backend set stay unchanged

Advantage: Simple, works at L4, no cookies needed.

Disadvantage: Clients behind NAT (same source IP) all go to same backend. Uneven load distribution.

# Load Balancing Algorithms

How does the load balancer choose which backend to use? Several algorithms exist:

• Round-robin: Cycle through backends in order. Simple, even distribution (if all requests are similar).
• Least connections: Send to backend with fewest active connections. Good for long-lived connections.
• Weighted round-robin: Assign weights (server1=2, server2=1). Server1 gets twice the traffic. Useful when backends have different capacity.
• Random: Pick random backend. Simple, surprisingly effective for large pools.
• Latency-based: Route to backend with lowest response time. Requires request telemetry or active probes to estimate backend latency.

# Connection Draining

When removing a backend (maintenance, scale-down, health check failure), connection draining allows in-flight requests to complete gracefully.

Example timeline with a configured 30-second drain timeout and force-close behavior enabled:

Timeout tuning: Set drain timeout longer than longest expected request duration. For web APIs: 30-60s typical. For long-polling/streaming: 300s or more.

# Key Takeaways

• L4 load balancers are fast, protocol-agnostic, but limited routing capabilities
• L7 load balancers offer content-based routing, SSL termination, but higher latency
• Health checks determine backend availability; tune interval/thresholds for failover speed vs stability
• Session affinity (sticky sessions) routes same client to same backend, useful when applications keep unreplicated backend-local session state
• Connection draining gives in-flight requests time to finish; remaining connections may be terminated when the configured timeout expires
• Choose algorithm based on workload: round-robin for stateless, least-connections for long-lived