AWS Site-to-Site VPN

Hybrid Environments and Migration

Border Gateway Protocol (BGP) – Key Concepts

Routing protocol → Peers share info on how to reach destinations
- Path-vector protocol → Each peer shares the “best path” vector to a destination
- Autonomous System (AS) = self-managed network controlled by one entity → acts as a single BGP peer
  - Viewed as a black box from outside, regardless of size
Autonomous System Number (ASN)
- 16-bit number: 0–65535
- Private ASNs: 64512–65534
- Public ASNs: globally unique, allocated by IANA
BGP uses TCP port 179 → reliable, distributed routing with flow control & error correction
Peering is manual → BGP does not auto-configure
- Once peering is established, ASs exchange routing info and topology continuously
- This is the foundation of Internet routing
Autonomous System Path (ASPATH) = “best path” to destination
- Even if multiple paths exist, AS shares only the selected ASPATH
- Does not consider link speed, only number of hops
- Shortest path preferred by default
- Techniques like ASPATH prepending can make slower paths less preferred
BGP Types
- iBGP (internal) → routing within an AS
- eBGP (external) → routing between ASs (AWS mainly uses eBGP)

Used by AWS Direct Connect (DX) and dynamic Site-to-Site VPNs

Simple Example of BGP Architecture

3 metro areas: Brisbane (ASN=200), Adelaide (ASN=201), Alice Springs (ASN=202)
- 200↔201 & 201↔202 → fiber links
- 200↔202 → satellite link (slower)
Route tables (RTs): each AS maintains one
- Origin AS = i in ASPATH
- ASs share routes → RT updated with new ASPATHs
  - Brisbane learns Alice Springs via Adelaide → ASPATH (201, 202, i)
  - Alice Springs directly to Brisbane → ASPATH (202, i) → preferred due to fewer hops
ASPATH prepending → artificially lengthen a path to make it less preferred
- Example: satellite link prepended → ASPATH (202, 202, 202, i)
- Brisbane now prefers fiber path (shorter hops)
- Satellite path still exists as backup
Result: dynamic, HA network
- All ASs maintain constantly updated topology
- Failure in one AS automatically reroutes traffic

IPsec VPN Fundamentals

IPsec – Key Concepts

IPsec is a set of protocols used to create secure tunnels over untrusted networks.
- Example: A local device and a remote device establish a secure tunnel over the public internet → connects two endpoints.
- Often used for geographically distributed on-premises infrastructure or hybrid connections between cloud and on-premises networks.
- IPsec tunnels allow the creation of VPNs spanning multiple locations.
Provides:
- Authentication: ensures that only trusted peers can connect.
- Encryption: protects data by transmitting it in encrypted form.

IPsec Architecture

Tunnels are dynamically created and removed depending on traffic needs.
- If traffic matches defined criteria → create tunnel; if no matching traffic → remove tunnel.
- Interesting traffic = traffic that meets defined rules.
  - Rules can be based on network prefixes or more detailed criteria.

IPsec Phases – Internet Key Exchange (IKE)

Symmetric encryption is fast but sharing keys securely is challenging.
Asymmetric encryption allows easy key exchange but is slower.
Internet Key Exchange (IKE) is the protocol used to exchange keys and set up IPsec VPNs.
- IKE Phase 1 → initial, resource-intensive setup.
- IKE Phase 2 → faster, lightweight setup.
- Phase 2 tunnels are created and removed based on traffic, but Phase 1 tunnel usually remains active to simplify future Phase 2 tunnel creation.

IKE Phase 1 – Peer Authentication & Key Exchange

Peers authenticate using pre-shared keys or certificates.
Keys are exchanged via asymmetric encryption (e.g., Diffie-Hellman, DH).
1. Each peer generates a DH private key (used to sign and decrypt data).
2. Each peer derives a DH public key (used to encrypt data for the other peer).
3. Peers exchange public keys.
4. Each peer combines its private key with the peer’s public key to generate a shared symmetric DH key.
  - Both peers end up with the same DH key, independently derived.
5. DH key is used to exchange additional keying information and agreements.
Phase 1 Security Association (SA) is established – the initial tunnel.
- DH key secures all data transmitted through Phase 1 tunnel.

Phase 1 is slow and computationally heavy, but generates a secure key without transmitting it in plaintext over the internet.

IKE Phase 2 – Establishing the IPsec VPN

Peers negotiate encryption parameters for the VPN and use the DH key to exchange additional key material.
- Includes supported cipher suites and VPN type.
New symmetric IPsec keys are generated for efficient bulk data transfer.
- IPsec keys are independent of DH key for added security.
  - If original keys are compromised, IPsec keys remain protected.
Phase 2 Security Association (SA) is created – the VPN tunnel runs over the Phase 1 tunnel.

Types of IPsec VPNs

Route-based VPNs – traffic is directed according to IP prefixes.
- One Phase 2 tunnel → one SA pair → one IPsec key.
Policy-based VPNs – traffic is matched according to defined rulesets.
- Multiple rulesets can exist, each with its own SA pair and IPsec key.

Policy-based VPNs offer more flexibility but are more complex to configure.

Route- vs Policy-based VPNs Diagram:

Site-to-Site VPN – Key Concepts

AWS Site-to-Site VPN allows you to establish and manage an IPsec VPN – a logical connection between a VPC and an on-premises network, with data encrypted in transit using IPsec.
- Usually runs over the public internet, but it can also operate over Direct Connect (DX).
- Enables hybrid networking between AWS and on-premises networks, or between AWS and another cloud provider.
Main components:
- Virtual Private Gateway (VGW)
  - A logical gateway object that can be a route target in a VPC route table.
  - Associated with a single VPC.
  - Currently does not support IPv6.
  - Functions similarly to an Internet Gateway (IGW): positioned between the VPC and AWS public network, with regional resilience.
- Customer Gateway (CGW) – refers to either:
  1. The physical on-premises router the VPN connects to.
  2. The logical AWS configuration object representing that router.
- VPN connection
  - Always links one VGW and one CGW.
  - Stores VPN configuration details.
Key features:
- Can be highly available if designed and implemented properly.
- Direct Connect integration
  - Can serve as a backup/failover for DX.
  - Can operate on top of DX to provide an additional encryption layer.
- Billing considerations:
  - Base hourly charge.
  - Data transfer fees for traffic leaving AWS ($/GB).
  - Uses the on-premises internet connection, which can affect bandwidth limits or ISP data caps.

VPN Advantages and Limitations

Benefits:
- Fast to provision (less than an hour), since it’s fully software-based and requires no hardware setup. DX provisioning takes significantly longer.
- More cost-effective than DX.
- IPsec is widely supported across routers and networking hardware.
Limitations:
- Throughput constraints – a single VPN connection has a maximum of 1.25 Gbps (AWS limit). On-premises hardware may impose lower limits. Encryption/decryption overhead can reduce effective speed. A VGW has a combined throughput cap of 1.25 Gbps across all VPN connections.
- Latency – public internet paths are variable and inconsistent, which may not be suitable for latency-sensitive applications.

Site-to-Site VPN – Architecture

Basic / Partially Highly Available Implementation

Steps to establish VPN:

Collect necessary information: VPC CIDR, on-premises network CIDR, and on-prem router’s public IP.
Create a VGW and attach it to the VPC.
- VGW includes two physical endpoints in separate AZs, each with a public IP → provides HA on the AWS side.
Create the CGW logical object in AWS using the on-prem router’s public IP.
Create the VPN connection linking the VGW and CGW.
- IPsec tunnels are established between each VGW endpoint and the on-prem router.
- Two encrypted tunnels provide redundancy: if one fails, the other remains active.
- Additional tunnels or VGWs can be created if more redundancy is needed.

Limitation: single on-prem router = single point of failure.
- If the router fails, the VPN fails. AWS side is HA, but the overall solution is only partially highly available.

Fully Highly Available Implementation

Add a second on-prem router in a separate building to eliminate single point of failure.
Create a new VPN connection with the second CGW, establishing two separate VPN connections.
- This does not reuse previous VGW endpoints; new endpoints and tunnels are created for the second router.

Static vs Dynamic VPNs

Static VPN

Routes are manually configured in VPC route tables using static network ranges.
Simple setup; compatible with most routers.
Limitations:
- No load balancing or multi-connection failover without manual configuration.
- Multiple CGWs require manual failover.
- Use dynamic VPNs for HA, multi-link redundancy, or DX integration.

Dynamic VPN

Uses BGP to exchange network routing information.
- BGP peering between VGW and CGW allows dynamic route adjustment.
Supports multiple simultaneous links for seamless failover.
Requires on-prem routers that support BGP.

Route Propagation

Can be enabled in a VPC to automatically add known on-prem routes to VPC route tables.
Works with both static and dynamic VPNs.
Reduces manual route configuration in AWS.
Manual route updates are still needed on on-prem routers unless BGP is used.

LAB: Simple Site-to-Site VPN

GOAL: On-prem laptop connects to a private AWS web application through a VPN

For this lab, the on-premises setup is emulated inside AWS
- Normally, the on-prem network would be a corporate LAN rather than a VPC
- The on-prem router would typically be a physical device instead of an EC2 instance
- The on-prem laptop would usually be a real machine rather than an EC2 instance
The on-prem router runs pfSense software
- Acts as a router, firewall, and VPN device
- In real deployments, this could be a pfSense Netgate appliance
- The lab walks through configuration step by step
The on-prem laptop uses Windows
- It will access a web app hosted in an AWS VPC using Internet Explorer
This lab is not fully covered by the free tier and may incur some cost

STAGE 0: Initial Setup

Switch to the us-east-1 region
Subscribe to the pfSense AMI
- Marketplace link: pfSense AMI
- Select “View Purchase Options” → “Accept Terms”
- This AMI introduces additional cost, so unsubscribe after finishing

Unsubscribe guide
Create an EC2 SSH key pair in us-east-1
- Navigate to EC2 → Network & Security → Key Pairs
- Download the key after creation
Launch the environment using CloudFormation
- 1-click deployment
- Ensure the created key pair is selected

After this stage, both the AWS VPC and simulated on-prem network are deployed

STAGE 1: Configure AWS VPN

This setup uses a static VPN configuration (no BGP).

Create a Virtual Private Gateway (VGW) and attach it to the VPC
Create a Customer Gateway (CGW)
- Use the public IP of the on-prem router
Establish the VPN connection
- Select the VGW and CGW
- Choose static routing and enter on-prem CIDR 192.168.8.0/21
- If BGP were used, routing would be exchanged dynamically