Most organizations don't move everything to the cloud overnight. Hybrid architectures—where workloads run
both on-premises and in the cloud—require robust, reliable connectivity between datacenters. The choice of
connectivity method fundamentally impacts bandwidth, latency, resiliency, and cost. This section covers
common patterns for connecting on-prem infrastructure to cloud providers.
# Connectivity Options
There are three primary approaches to connecting on-premises datacenters to cloud providers, each with
different trade-offs.
VPN over Internet
How it works: IPsec VPN tunnels over the public internet create encrypted connections
between on-prem routers and cloud VPN gateways.
VPN over the public internet — IPsec tunnels encrypt traffic between the on-prem router and a cloud VPN gateway; quick to stand up, but throughput and latency depend on the shared internet path.
Pros: Quick setup (hours), low upfront cost, works from anywhere with internet
Cons: Variable latency and throughput (depends on internet path), typically limited
to 1-2 Gbps per tunnel, shared with other internet traffic
Direct/Dedicated Connections
How it works: Private fiber connections from your datacenter (or colocation facility)
directly to cloud provider's network. No internet traversal.
Direct/dedicated connection — private fiber links the on-prem router to a carrier or colocation cross-connect, which hands off to the cloud provider's dedicated interconnect (AWS Direct Connect, Azure ExpressRoute, or GCP Cloud Interconnect); no internet traversal.
Providers: AWS Direct Connect, Azure ExpressRoute, GCP Cloud Interconnect
Pros: Predictable low latency, high bandwidth (1-100 Gbps), dedicated capacity
(not shared with internet), better for compliance
Cons: Longer setup time (weeks to months), higher upfront cost, requires physical
presence or colocation
SD-WAN (Software-Defined WAN)
How it works: Software overlay that intelligently routes traffic over multiple underlay
connections (internet VPN, MPLS, dedicated links). Provides application-aware routing and automatic failover.
SD-WAN overlay — a single appliance intelligently routes traffic across three underlay paths (Direct Connect, internet VPN, MPLS) based on application needs, failing over automatically when a path degrades.
Bandwidth requirements depend on your workload mix and how much data flows between on-prem and cloud.
Typical Connection Speeds
Connection Type
Typical Bandwidth
Use Case
Internet VPN (single tunnel)
100 Mbps - 2 Gbps
Small offices, dev/test, light workloads
Direct Connect (Standard)
1 Gbps - 10 Gbps
Production hybrid, database replication, file sync
Direct Connect (High-capacity)
10 Gbps - 100 Gbps
Large-scale migration, HPC, video rendering, big data pipelines
Burst vs Sustained: VPN connections share internet bandwidth—peak usage from other
applications can degrade cloud connectivity. Dedicated connections provide consistent throughput.
Calculating requirements: Common workloads and their bandwidth needs:
File server sync: Varies wildly. Initial sync of 10 TB over 1 Gbps = ~24 hours
Backup to cloud: Nightly 500 GB backup over 8-hour window = ~150 Mbps average
VDI (Virtual Desktop): 20-50 Mbps per desktop * number of concurrent users
# Latency Characteristics
Latency matters enormously for chatty protocols (databases, file systems, RPC) and less for bulk transfers.
Typical Latencies
Connection Method
Same City
Cross-Country
Cross-Continent
Direct Connect (Dedicated)
1-5 ms
30-50 ms
80-150 ms
Internet VPN (Best case)
5-15 ms
40-80 ms
100-200 ms
Internet VPN (Congested)
20-100+ ms
100-200+ ms
200-500+ ms
Note: latencies depend heavily on geographic distance and routing.
Impact on workloads:
Database queries: Latency-sensitive. 50ms RTT = 20 queries/sec max (serialized). Use read replicas or caching to mitigate
File operations (NFS/SMB): Very latency-sensitive. Opening a file = multiple round-trips. 100ms RTT can make file access unusable
Bulk transfers: Latency-tolerant if using modern protocols (TCP with window scaling). Throughput = Window Size / RTT
API calls (REST/gRPC): Moderate sensitivity. Design for async patterns or cache responses when possible
Geographic Proximity
Choose cloud regions close to on-prem datacenters when latency matters.
Example: San Francisco datacenter to cloud regions.
Region
Distance
Typical Latency
AWS us-west-1 (N. California)
~100 miles
2-5 ms
AWS us-west-2 (Oregon)
~600 miles
10-15 ms
AWS us-east-1 (Virginia)
~2,500 miles
60-70 ms
AWS eu-west-1 (Ireland)
~5,000 miles
140-160 ms
Physics matters: speed of light in fiber = ~200,000 km/s. Theoretical minimum (SF to Virginia) = ~40ms RTT.
# Resiliency and Redundancy Patterns
Single connections create single points of failure. Production workloads require redundancy.
Single Connection (Not Resilient)
Single connection — no redundancy: a link failure, router failure, or even planned maintenance takes the whole hybrid link down.
Dual Connections, Same Provider
Dual connections, same provider — two independent routers and circuits survive a single link or router failure, but a provider-wide outage or regional issue still takes down both paths.
Configuration: Use BGP with equal-cost multipath (ECMP) for active-active, or BGP
AS-path prepending for active-passive. AWS/Azure/GCP all support BGP for automatic failover [1].
Multi-Provider (Maximum Resilience)
Multi-provider — two routers connect via two different providers or carriers (e.g. Direct Connect + internet VPN, or fiber from two distinct carriers), tolerating a full provider outage at the cost of higher spend and routing complexity.
Failover automation: BGP automatically reroutes traffic when a path fails. Health checks
on both sides ensure quick detection (typical failover: 30-90 seconds).
# Cost Model
Connectivity costs vary dramatically between VPN and dedicated connections. Understanding the cost structure
helps you choose the right option.
VPN over Internet Costs
Setup: Minimal—VPN gateway costs ~$0.05/hour (AWS) or ~$0.19/hour (Azure) [2]
Data transfer: Standard internet egress rates. AWS: $0.09/GB (first 10 TB/month)
Total for 1 TB/month: Gateway ~$36/month + 1000 GB * $0.09 = ~$126/month
Limitations: Shared bandwidth, variable performance, typically <1 Gbps sustained
Direct Connect / ExpressRoute Costs
Port hours: Dedicated port rental. AWS 1 Gbps = $0.30/hour = ~$216/month [3]
Data transfer OUT: Reduced rates vs internet. AWS: $0.02/GB (vs $0.09/GB internet)
Connection fee: One-time setup via carrier/colocation (varies: $500-$5000+)
Total for 1 TB/month: Port $216 + 1000 GB * $0.02 + setup = ~$236/month + setup
Break-even: For high-volume (10+ TB/month), Direct Connect is cheaper due to lower transfer costs [4]
Cost Comparison
Monthly Data Transfer
VPN Cost
Direct Connect Cost
Winner
100 GB
~$45
~$218
VPN
1 TB
~$126
~$236
VPN
10 TB
~$936
~$416
Direct
50 TB
~$4,536
~$1,216
Direct
100 TB
~$9,036
~$2,216
Direct
Assumes AWS us-east-1 pricing with a 1 Gbps Direct Connect circuit; prices vary by region and provider.
Hidden costs: Don't forget router hardware, BGP configuration expertise, colocation fees
(if applicable), and ongoing management. For small deployments, VPN's simplicity often wins despite higher
per-GB costs.
# Data Migration Patterns
Moving data from on-prem to cloud is a common hybrid scenario. The migration strategy depends on data
volume, time constraints, and ongoing sync requirements.
Initial Bulk Migration
Problem: Migrating 100 TB over 1 Gbps = 9+ days of continuous transfer. Not always practical.
Solutions:
Physical disk shipping: AWS Snowball (80 TB device), Azure Data Box (100 TB), GCP Transfer Appliance. Ship disks to provider, they ingest data. 1-2 weeks total time [5]
Snowmobile (AWS): For petabyte-scale. Literal shipping container with 100 PB capacity. Extreme but real
Parallel transfers: Use multiple VPN tunnels or Direct Connect links with parallel rsync/rclone jobs to maximize throughput
Incremental seed: Ship initial bulk via Snowball, then sync deltas over network
Continuous Sync
After initial migration, keep on-prem and cloud in sync for hybrid workloads or gradual migration.
File sync: Tools like rsync, rclone, AWS DataSync, Azure File Sync. Schedule periodic syncs or near-real-time
Change Data Capture (CDC): Tools such as Debezium and AWS DMS capture database changes for ongoing replication. Debezium can emit low-delay change events; AWS DMS explicitly does not provide real-time CDC replication
Hybrid Storage Gateways
Cloud providers offer gateways that present cloud storage as local volumes/shares, caching frequently
accessed data on-prem.
Hybrid storage gateway — the application talks to a local gateway that serves reads from cache, writes locally before async-uploading, and fetches cold data from cloud object storage on demand.
Providers: AWS Storage Gateway, Azure StorSimple (deprecated, replaced by File Sync),
Google Cloud Storage FUSE
Use case: Gradual migration where applications still need local performance but
storage tier is shifting to cloud. Good for file servers, backup archives.
# References
[1] AWS. "AWS Direct Connect - BGP Configuration."
Documentation on using BGP for redundant Direct Connect connections with automatic failover.
AWS Direct Connect Guide
[2] AWS. "AWS VPN Pricing."
Current pricing for VPN connections and data transfer rates.
aws.amazon.com/vpn/pricing
[3] AWS. "AWS Direct Connect Pricing."
Port hour rates and data transfer pricing for dedicated connections.
aws.amazon.com/directconnect/pricing
[4] Multiple Cloud Providers. "Hybrid Cloud Connectivity Cost Analysis."
Industry analysis showing Direct Connect becomes cost-effective at ~10TB/month transfer volume.
Pricing as of 2024-2025 for major cloud providers.
[5] AWS. "AWS Snowball - Petabyte-Scale Data Transport."
Documentation on physical data transfer appliances for bulk migration.
aws.amazon.com/snowball
# Key Takeaways
VPN over internet: Quick, cheap setup but variable performance (good for <1 TB/month, dev/test)
Direct Connect/ExpressRoute: Predictable latency, high bandwidth, cost-effective at scale (10+ TB/month)
SD-WAN combines multiple paths for redundancy and intelligent routing
Latency matters for databases and file systems—choose nearby cloud regions