Tunneling & Security
Tunneling protocols encapsulate one network protocol inside another, enabling encryption, network overlays, and connectivity across incompatible networks. Understanding tunneling is essential for VPNs, cloud networking, and modern container networking.
# IPSec: IP Security
IPSec provides encryption and authentication at the IP layer. It's widely used for site-to-site VPNs connecting datacenters or branch offices.
IPSec Modes
Transport Mode: Encrypts only the payload (data), leaves IP header intact. Used for end-to-end encryption between two hosts.
Tunnel Mode: Encrypts entire original IP packet, wraps it in new IP header. Used for VPNs—original packet is hidden, new header routes through public internet.
Transport Mode:
[IP Header (original)] [IPSec ESP] [Encrypted Payload]
^ ^ ^
| | |
Visible to Authentication Encrypted
routers + Encryption data
Tunnel Mode:
[New IP Header] [IPSec ESP] [Encrypted Original IP + Payload]
^ ^ ^
| | |
Visible to Authentication Original packet
routers + Encryption completely hidden
ESP (Encapsulating Security Payload): Provides encryption and optional authentication. Most common IPSec protocol.
AH (Authentication Header): Provides authentication without encryption. Rarely used alone; typically combined with ESP.
When IPSec Matters
- Site-to-site VPNs connecting datacenters over internet
- Encrypting traffic between cloud regions
- Securing management traffic (remote server access)
Performance impact: Encryption adds CPU overhead and latency (1-10ms typical). Hardware offload (AES-NI on CPUs, crypto accelerators) reduces overhead. Throughput can drop 20-50% without hardware acceleration.
# VPNs: Virtual Private Networks
VPNs create encrypted tunnels over untrusted networks (like the internet) to securely connect remote sites or users.
Site A Datacenter Site B Datacenter
(10.0.0.0/16) (10.1.0.0/16)
| |
[VPN Gateway] [VPN Gateway]
| |
+---- IPSec Tunnel over ----+
Public Internet
(Encrypted, authenticated)
Traffic from 10.0.1.5 -> 10.1.2.10 flows through tunnel
Appears to routers as traffic between gateway public IPs
Common VPN types:
- Site-to-site: Connect entire networks (datacenter to datacenter). Always-on, routed.
- Remote access: Individual users connect to corporate network (OpenVPN, WireGuard). On-demand.
- Cloud VPN: Connect on-prem to cloud (AWS VPN, Azure VPN Gateway). Hybrid cloud connectivity.
MTU consideration: VPN adds overhead (20-60 bytes for IP/IPSec/ESP headers). If underlying network has MTU 1500, VPN payload MTU might be 1440. Failure to account for this causes packet fragmentation or "black hole" routes.
# Overlay Networks
Overlay networks create virtual networks on top of physical infrastructure. They decouple logical network topology from physical cabling, enabling flexible, software-defined networking.
VXLAN: Virtual Extensible LAN
VXLAN extends Layer 2 networks across Layer 3 infrastructure. It's the foundation for most cloud and container networking (AWS VPC, Kubernetes CNI, VMware NSX).
Original Ethernet Frame (from VM or container):
[Ethernet Header] [IP Header] [Payload]
VXLAN Encapsulation:
[Outer Ethernet] [Outer IP] [UDP] [VXLAN Header] [Inner Ethernet] [Inner IP] [Payload]
(24-bit VNI)
^ ^ ^ ^
| | | |
Physical Routes Virtual Network Original packet
network across L3 Identifier from workload
addressing network (16M networks)
Key benefit: Isolate up to 16 million tenant networks (24-bit VNI) vs 4096 VLANs (12-bit). Essential for multi-tenant cloud environments.
Overhead: VXLAN adds 50 bytes of encapsulation. With 1500-byte physical MTU, VXLAN payload MTU is 1450. Jumbo frames (9000 MTU) on physical network help avoid fragmentation.
Geneve: Generic Network Virtualization Encapsulation
Geneve is a more flexible encapsulation protocol designed to replace/extend VXLAN. It supports variable-length options for metadata (security policies, QoS tags, etc.).
Used in: Some Kubernetes CNI plugins (Cilium, Calico), newer SDN platforms. Less common than VXLAN but gaining adoption.
# Performance Impact of Encryption and Tunneling
Encryption and encapsulation aren't free. Understanding the overhead helps you design performant systems.
+---------------------+-------------+------------------+ | Operation | CPU Cost | Latency Impact | +---------------------+-------------+------------------+ | No encryption | Baseline | Baseline | +---------------------+-------------+------------------+ | IPSec (HW accel) | +5-10% | +1-2ms | +---------------------+-------------+------------------+ | IPSec (SW only) | +20-50% | +3-10ms | +---------------------+-------------+------------------+ | VXLAN encap | +5-15% | +0.5-1ms | +---------------------+-------------+------------------+ | IPSec + VXLAN | +25-60% | +4-12ms | +---------------------+-------------+------------------+
Optimization strategies:
- Use hardware crypto acceleration (AES-NI, IPSec offload NICs)
- Enable jumbo frames to reduce encapsulation overhead percentage
- For intra-datacenter traffic, consider unencrypted overlays (if physical security is sufficient)
- Use modern ciphers (ChaCha20-Poly1305) optimized for software performance
When to encrypt: Always encrypt over untrusted networks (internet, multi-tenant infrastructure). For dedicated datacenter interconnects (dark fiber, private MPLS), encryption depends on threat model and compliance requirements.
# Cloud Environment Tunneling
Cloud providers use tunneling extensively for isolation and security.
AWS VPC: Uses encapsulation (implementation details not public, but behaves like VXLAN) to isolate customer VPCs. Traffic between EC2 instances in same VPC is encapsulated, providing network isolation even though physical infrastructure is shared.
GCP VPC: Uses Andromeda SDN with encapsulation for isolation and routing.
Kubernetes: Most CNI plugins (Calico, Flannel, Cilium) use VXLAN or IP-in-IP tunneling to route pod traffic across nodes. Overlay mode trades some performance for flexibility.
# Key Takeaways
- IPSec tunnel mode encrypts traffic for site-to-site VPNs; adds CPU overhead and latency
- VPNs reduce effective MTU due to encapsulation—plan for 1440-byte payload MTU over internet
- VXLAN enables 16M virtual networks (vs 4K VLANs), essential for cloud multi-tenancy
- Encryption performance depends on hardware acceleration; can impact throughput 20-50% without it
- Overlay networks decouple logical topology from physical infrastructure, enabling SDN
- Cloud environments use tunneling/encapsulation for tenant isolation on shared infrastructure