mateusz@systems ~/book/hybrid-cloud $ cat section.md

Hybrid Cloud Connectivity

Most organizations don't move everything to the cloud overnight. Hybrid architectures—where workloads run both on-premises and in the cloud—require robust, reliable connectivity between datacenters. The choice of connectivity method fundamentally impacts bandwidth, latency, resiliency, and cost. This section covers common patterns for connecting on-prem infrastructure to cloud providers.

# Connectivity Options

There are three primary approaches to connecting on-premises datacenters to cloud providers, each with different trade-offs.

VPN over Internet

How it works: IPsec VPN tunnels over the public internet create encrypted connections between on-prem routers and cloud VPN gateways.

On-Premises Cloud Provider On-Prem Router (Firewall) Internet VPN Gateway (AWS / Azure / GCP) IPsec VPN Encrypted IPsec VPN Encrypted Internal Network 10.0.0.0/16 VPC/VNet 172.16.0.0/16
VPN over the public internet — IPsec tunnels encrypt traffic between the on-prem router and a cloud VPN gateway; quick to stand up, but throughput and latency depend on the shared internet path.

Providers: AWS VPN, Azure VPN Gateway, GCP Cloud VPN

Pros: Quick setup (hours), low upfront cost, works from anywhere with internet

Cons: Variable latency and throughput (depends on internet path), typically limited to 1-2 Gbps per tunnel, shared with other internet traffic

Direct/Dedicated Connections

How it works: Private fiber connections from your datacenter (or colocation facility) directly to cloud provider's network. No internet traversal.

On-Premises Dedicated Connection Cloud Provider On-Prem Router (BGP) Carrier / Colocation Cross-Connect Direct Connect (AWS) ExpressRoute (Azure) Interconnect (GCP) Private Fiber Private Fiber Internal Network 10.0.0.0/16 VPC/VNet 172.16.0.0/16
Direct/dedicated connection — private fiber links the on-prem router to a carrier or colocation cross-connect, which hands off to the cloud provider's dedicated interconnect (AWS Direct Connect, Azure ExpressRoute, or GCP Cloud Interconnect); no internet traversal.

Providers: AWS Direct Connect, Azure ExpressRoute, GCP Cloud Interconnect

Pros: Predictable low latency, high bandwidth (1-100 Gbps), dedicated capacity (not shared with internet), better for compliance

Cons: Longer setup time (weeks to months), higher upfront cost, requires physical presence or colocation

SD-WAN (Software-Defined WAN)

How it works: Software overlay that intelligently routes traffic over multiple underlay connections (internet VPN, MPLS, dedicated links). Provides application-aware routing and automatic failover.

On-Premises SD-WAN Overlay Cloud SD-WAN Appliance (Multi-Path) Cloud VPC (SD-WAN Gateway) Path 1: Direct Connect Path 2: Internet VPN Path 3: MPLS Intelligent routing: Database traffic → Direct Connect (low latency) Web traffic → Internet VPN (cost-effective) Automatic failover on path degradation
SD-WAN overlay — a single appliance intelligently routes traffic across three underlay paths (Direct Connect, internet VPN, MPLS) based on application needs, failing over automatically when a path degrades.

Providers: Cisco Catalyst SD-WAN, Arista VeloCloud SD-WAN, HPE Aruba Networking EdgeConnect SD-WAN, Fortinet Secure SD-WAN

Pros: Best of both worlds—redundancy and cost optimization, application-aware routing, automatic failover

Cons: Additional complexity, requires SD-WAN appliances/licensing, management overhead

# Bandwidth Considerations

Bandwidth requirements depend on your workload mix and how much data flows between on-prem and cloud.

Typical Connection Speeds

Connection Type Typical Bandwidth Use Case
Internet VPN
(single tunnel)
100 Mbps - 2 Gbps Small offices, dev/test, light workloads
Direct Connect
(Standard)
1 Gbps - 10 Gbps Production hybrid, database replication, file sync
Direct Connect
(High-capacity)
10 Gbps - 100 Gbps Large-scale migration, HPC, video rendering, big data pipelines

Burst vs Sustained: VPN connections share internet bandwidth—peak usage from other applications can degrade cloud connectivity. Dedicated connections provide consistent throughput.

Calculating requirements: Common workloads and their bandwidth needs:

  • Database replication: Depends on write rate. 1000 transactions/sec * 10 KB avg = ~80 Mbps sustained
  • File server sync: Varies wildly. Initial sync of 10 TB over 1 Gbps = ~24 hours
  • Backup to cloud: Nightly 500 GB backup over 8-hour window = ~150 Mbps average
  • VDI (Virtual Desktop): 20-50 Mbps per desktop * number of concurrent users

# Latency Characteristics

Latency matters enormously for chatty protocols (databases, file systems, RPC) and less for bulk transfers.

Typical Latencies

Connection Method Same City Cross-Country Cross-Continent
Direct Connect (Dedicated) 1-5 ms 30-50 ms 80-150 ms
Internet VPN (Best case) 5-15 ms 40-80 ms 100-200 ms
Internet VPN (Congested) 20-100+ ms 100-200+ ms 200-500+ ms

Note: latencies depend heavily on geographic distance and routing.

Impact on workloads:

  • Database queries: Latency-sensitive. 50ms RTT = 20 queries/sec max (serialized). Use read replicas or caching to mitigate
  • File operations (NFS/SMB): Very latency-sensitive. Opening a file = multiple round-trips. 100ms RTT can make file access unusable
  • Bulk transfers: Latency-tolerant if using modern protocols (TCP with window scaling). Throughput = Window Size / RTT
  • API calls (REST/gRPC): Moderate sensitivity. Design for async patterns or cache responses when possible

Geographic Proximity

Choose cloud regions close to on-prem datacenters when latency matters.

Example: San Francisco datacenter to cloud regions.

Region Distance Typical Latency
AWS us-west-1 (N. California) ~100 miles 2-5 ms
AWS us-west-2 (Oregon) ~600 miles 10-15 ms
AWS us-east-1 (Virginia) ~2,500 miles 60-70 ms
AWS eu-west-1 (Ireland) ~5,000 miles 140-160 ms

Physics matters: speed of light in fiber = ~200,000 km/s. Theoretical minimum (SF to Virginia) = ~40ms RTT.

# Resiliency and Redundancy Patterns

Single connections create single points of failure. Production workloads require redundancy.

Single Connection (Not Resilient)

On-Prem Cloud Router VPC Direct Connect Problem: Link failure = total outage Router failure = total outage Planned maintenance = downtime
Single connection — no redundancy: a link failure, router failure, or even planned maintenance takes the whole hybrid link down.

Dual Connections, Same Provider

On-Prem Cloud Router 1 Router 2 VPC Direct Connect 1 Direct Connect 2 Resilient to: Single link failure Single router failure Not resilient to: Provider outage (rare but happens) Regional datacenter issues
Dual connections, same provider — two independent routers and circuits survive a single link or router failure, but a provider-wide outage or regional issue still takes down both paths.

Configuration: Use BGP with equal-cost multipath (ECMP) for active-active, or BGP AS-path prepending for active-passive. AWS/Azure/GCP all support BGP for automatic failover [1].

Multi-Provider (Maximum Resilience)

On-Prem Cloud Router 1 Router 2 VPC Direct Connect or Provider A Fiber Internet VPN or Provider B Fiber Resilient to: Provider outage Link failure Router failure Trade-off: Higher cost, more complex routing
Multi-provider — two routers connect via two different providers or carriers (e.g. Direct Connect + internet VPN, or fiber from two distinct carriers), tolerating a full provider outage at the cost of higher spend and routing complexity.

Failover automation: BGP automatically reroutes traffic when a path fails. Health checks on both sides ensure quick detection (typical failover: 30-90 seconds).

# Cost Model

Connectivity costs vary dramatically between VPN and dedicated connections. Understanding the cost structure helps you choose the right option.

VPN over Internet Costs

  • Setup: Minimal—VPN gateway costs ~$0.05/hour (AWS) or ~$0.19/hour (Azure) [2]
  • Data transfer: Standard internet egress rates. AWS: $0.09/GB (first 10 TB/month)
  • Total for 1 TB/month: Gateway ~$36/month + 1000 GB * $0.09 = ~$126/month
  • Limitations: Shared bandwidth, variable performance, typically <1 Gbps sustained

Direct Connect / ExpressRoute Costs

  • Port hours: Dedicated port rental. AWS 1 Gbps = $0.30/hour = ~$216/month [3]
  • Data transfer OUT: Reduced rates vs internet. AWS: $0.02/GB (vs $0.09/GB internet)
  • Connection fee: One-time setup via carrier/colocation (varies: $500-$5000+)
  • Total for 1 TB/month: Port $216 + 1000 GB * $0.02 + setup = ~$236/month + setup
  • Break-even: For high-volume (10+ TB/month), Direct Connect is cheaper due to lower transfer costs [4]

Cost Comparison

Monthly Data Transfer VPN Cost Direct Connect Cost Winner
100 GB ~$45 ~$218 VPN
1 TB ~$126 ~$236 VPN
10 TB ~$936 ~$416 Direct
50 TB ~$4,536 ~$1,216 Direct
100 TB ~$9,036 ~$2,216 Direct

Assumes AWS us-east-1 pricing with a 1 Gbps Direct Connect circuit; prices vary by region and provider.

Hidden costs: Don't forget router hardware, BGP configuration expertise, colocation fees (if applicable), and ongoing management. For small deployments, VPN's simplicity often wins despite higher per-GB costs.

# Data Migration Patterns

Moving data from on-prem to cloud is a common hybrid scenario. The migration strategy depends on data volume, time constraints, and ongoing sync requirements.

Initial Bulk Migration

Problem: Migrating 100 TB over 1 Gbps = 9+ days of continuous transfer. Not always practical.

Solutions:

  • Physical disk shipping: AWS Snowball (80 TB device), Azure Data Box (100 TB), GCP Transfer Appliance. Ship disks to provider, they ingest data. 1-2 weeks total time [5]
  • Snowmobile (AWS): For petabyte-scale. Literal shipping container with 100 PB capacity. Extreme but real
  • Parallel transfers: Use multiple VPN tunnels or Direct Connect links with parallel rsync/rclone jobs to maximize throughput
  • Incremental seed: Ship initial bulk via Snowball, then sync deltas over network

Continuous Sync

After initial migration, keep on-prem and cloud in sync for hybrid workloads or gradual migration.

  • File sync: Tools like rsync, rclone, AWS DataSync, Azure File Sync. Schedule periodic syncs or near-real-time
  • Database replication: Native replication (MySQL binlog, PostgreSQL streaming replication) over VPN/Direct Connect
  • Object storage sync: S3 sync, gsutil rsync, Azure AzCopy for continuous object uploads
  • Change Data Capture (CDC): Tools such as Debezium and AWS DMS capture database changes for ongoing replication. Debezium can emit low-delay change events; AWS DMS explicitly does not provide real-time CDC replication

Hybrid Storage Gateways

Cloud providers offer gateways that present cloud storage as local volumes/shares, caching frequently accessed data on-prem.

On-Prem Cloud Application Storage Gateway Cached data S3 / Blob Storage (Full dataset) Frequently accessed (cached locally) Reads: served from local cache (fast) Writes: written locally, async uploaded to cloud Cold data: fetched from cloud on-demand
Hybrid storage gateway — the application talks to a local gateway that serves reads from cache, writes locally before async-uploading, and fetches cold data from cloud object storage on demand.

Providers: AWS Storage Gateway, Azure StorSimple (deprecated, replaced by File Sync), Google Cloud Storage FUSE

Use case: Gradual migration where applications still need local performance but storage tier is shifting to cloud. Good for file servers, backup archives.

# References

[1] AWS. "AWS Direct Connect - BGP Configuration." Documentation on using BGP for redundant Direct Connect connections with automatic failover. AWS Direct Connect Guide

[2] AWS. "AWS VPN Pricing." Current pricing for VPN connections and data transfer rates. aws.amazon.com/vpn/pricing

[3] AWS. "AWS Direct Connect Pricing." Port hour rates and data transfer pricing for dedicated connections. aws.amazon.com/directconnect/pricing

[4] Multiple Cloud Providers. "Hybrid Cloud Connectivity Cost Analysis." Industry analysis showing Direct Connect becomes cost-effective at ~10TB/month transfer volume. Pricing as of 2024-2025 for major cloud providers.

[5] AWS. "AWS Snowball - Petabyte-Scale Data Transport." Documentation on physical data transfer appliances for bulk migration. aws.amazon.com/snowball

# Key Takeaways

  • VPN over internet: Quick, cheap setup but variable performance (good for <1 TB/month, dev/test)
  • Direct Connect/ExpressRoute: Predictable latency, high bandwidth, cost-effective at scale (10+ TB/month)
  • SD-WAN combines multiple paths for redundancy and intelligent routing
  • Latency matters for databases and file systems—choose nearby cloud regions
  • Redundancy requires multiple connections; BGP provides automatic failover
  • Cost break-even between VPN and dedicated is ~10 TB/month of data transfer
  • Bulk migration: Use physical shipping (Snowball) for large datasets, network for incremental sync
  • Hybrid storage gateways provide local performance with cloud backend for gradual migration