Scaling & Serverless

High Availability (HA) and Scaling

AWS Regional and Global Architecture

Global vs Regional Applications
Architectural Components of Cloud Systems

Global Components

  1. Global Service Location & Discovery
    • Determines how users locate the application (e.g., netflix.com)
    • Example: DNS configuration in Amazon Route 53
  2. Content Delivery & Optimization
    • Ensures application content is delivered efficiently to users worldwide
    • Can use globally distributed storage or rely on a central origin
    • Example: Amazon CloudFront caches content closer to users
  3. Global Health Checks & Failover
    • Monitors infrastructure health across regions
    • Automatically redirects users to healthy regions during failures
    • Example: Route 53 health checks

Regional Components

  1. Regional Entry Point
    • Defines how users access regional infrastructure
    • Example: VPC endpoints, ALBs, API Gateway
  2. Scaling & Resilience
    • Maintains performance under regional load fluctuations or failures
    • Example: Auto Scaling Groups (ASGs), ALBs
  3. Application Services & Components
    • Implements core functionality for the region
    • Example: EC2 instances, S3 buckets

Regional Application Tiers

  1. Web Tier
    • Entry point for regional applications
    • Abstracts users from underlying infrastructure
    • Example: ALB, API Gateway
  2. Compute Tier
    • Provides application logic and processing
    • Example: EC2, Lambda, ECS
  3. Storage Tier
    • Supports compute services with data storage
    • Example: S3, EBS, EFS
    • CloudFront can use S3 as an origin for media
  4. Database Tier
    • Stores structured data for compute tier
    • Example: RDS, Aurora, DynamoDB, Redshift
  5. Caching Tier
    • Reduces read load on databases by caching data in memory
    • Example: ElastiCache, DynamoDB Accelerator (DAX)
  6. Application Services
    • Additional AWS services enhancing the application
    • Examples:
      • Simple services: notifications, email (SNS, SES)
      • Architecture-changing services: decoupling components (SQS, Kinesis, Step Functions)

EC2 Launch Configurations (LCs) and Launch Templates (LTs)

EC2 Launch Configuration (LC)
EC2 Launch Template (LT)

EC2 Auto Scaling Groups (ASG)

EC2 Auto Scaling Group (ASG) – Key Concepts

EC2 ASG – X:Y:Z Model

EC2 ASG – Architecture
EC2 ASG – Scaling Processes
EC2 ASG – Additional Features

EC2 Auto Scaling and ASG Scaling Policies

EC2 Auto Scaling – Types
  1. Manual Scaling
    • Adjust ASG capacity values directly without a scaling policy.
    • MIN, DESIRED, MAX values are static but can be modified manually via AWS Console, CLI, or custom scripts.
    • Customer controls scaling logic and execution.
    • Use cases: testing ASGs, cost management, or urgent capacity changes.
  2. Scheduled Scaling
    • Automatically updates DESIRED capacity at specified times.
    • Does not require a scaling policy.
    • Implemented using scheduled actions in the ASG.
      • MIN and MAX values remain unchanged and must be modified separately.
    • Useful for predictable traffic patterns, e.g., peak business hours or promotional events.
  3. Dynamic Scaling
    • Automatically adjusts DESIRED capacity based on real-time metrics or CloudWatch alarms.
    • Requires a scaling policy attached to the ASG.
    • Only DESIRED capacity is updated automatically; MIN and MAX still need manual configuration.
    • Dynamic scaling types:
      1. Simple Scaling – single-step scale out/in actions
      2. Step/Stepped Scaling – multi-step scaling based on metric thresholds
      3. Target Tracking – ASG maintains a metric near a defined target
ASG Scaling Policies (Dynamic Scaling)
EC2 ASG – Simple Scaling
EC2 ASG – Step/Stepped Scaling

EC2 ASG – Target Tracking

Auto Scaling Group (ASG) Lifecycle Hooks

EC2 ASG – Lifecycle Hooks

AWS Elastic Load Balancer (ELB)

Traditional Load Balancer (LB)
AWS Elastic Load Balancer (ELB) – Architecture

Configuration highlights:

Subnet requirements for ELB scaling:

ELB – Abstraction of Infrastructure
ELB – Cross-Zone Load Balancing

ELB Types – Classic, Application, and Network Load Balancers

AWS Elastic Load Balancer (ELB) – Overview and Types
  1. v1 (Legacy) – Classic Load Balancer (CLB)
    • Considered outdated; migrate to ELB v2 if still in use.
  2. v2 (Current) – Modern Load Balancers
    • Application Load Balancer (ALB) – operates at Layer 7, supports HTTP(S) and WebSockets.
    • Network Load Balancer (NLB) – operates at Layer 4, supports TCP, TLS, and UDP.

Note: A fourth type, Gateway Load Balancer (GWLB), exists for specific network appliance use cases and is handled separately.

ELBv1 – Classic Load Balancer (CLB)

Drawbacks:

ELBv2 – Modern Load Balancer

Advantages over CLB:

Application Load Balancer (ALB)

Listener Rules:

Benefits:

Limitations:

Network Load Balancer (NLB)

Limitations:

Use Cases:

Choosing Between ALB and NLB

In all other cases, ALB provides greater flexibility with Layer 7 features.

ELB – SSL Termination

AWS ELB – Approaches for Handling SSL

ALB – SSL Bridging

Key Points:

NLB – SSL Pass-through

Key Points:

ALB – SSL Offload

Key Points:

SSL Handling in ELB – Overview Table
SSL ApproachLoad Balancer TypeListener ProtocolCertificate on ELBCertificate on EC2 Instances
SSL BridgingApplication Load Balancer (ALB)HTTPSPresentPresent
SSL Pass-throughNetwork Load Balancer (NLB)TCPNot usedRequired
SSL Termination (Offload)Application Load Balancer (ALB)HTTPSPresentNot required

ELB – Session Affinity (Stickiness)

ELB Session/Connection Stickiness – Overview
ALB Session Stickiness

NLB Session Stickiness

Demo – ALB Session Stickiness

ASG – ELB Integration and Health Checks

ASG-ELB Integration
ASG Health Checks
  1. EC2 instance status checks (default)
    • An instance is considered healthy only if it is in the Running state AND passes both 2/2 status checks.
    • Instances in states like Stopping, Stopped, Shutting Down, or Terminated or failing checks are marked unhealthy.
  2. ELB health checks (optional, when ASG is attached to ELB)
    • An instance is healthy only if both ELB checks and EC2 status checks pass.
    • Provides network-level (L4) or application-level (L7) health monitoring (L7 only with ALB).
    • Misconfiguration can cause problems:
      • Example: ELB checks a simple HTML page, but the app’s backend (e.g., database) is failing → ASG may continuously replace instances unnecessarily.
  3. Custom health checks
    • Any external system can mark instances as healthy/unhealthy.
    • Allows ASGs to meet specific business requirements or integrate with monitoring tools.

Gateway Load Balancer (GWLB) – Traffic Management for Network Appliances

Scaling Challenges with Network Security Appliances
AWS Gateway Load Balancer (GWLB) – Core Concepts

GENEVE Protocol in GWLB

Example GWLB Architecture and Traffic Flow
  1. Traffic enters the IGW, destined for an ALB with a public IP in subnet 10.16.9.0/20.
  2. IGW updates the destination IP to the ALB’s private IP and forwards traffic to GWLBE2.
  3. GWLBE2 sends packets to the GWLB in the securityVPC.
  4. GWLB encapsulates traffic using GENEVE, preserving original source/destination IPs, and forwards to a selected security appliance.
  5. Security appliance inspects traffic and either blocks or returns it.
  6. Packets are returned to GWLB with GENEVE encapsulation removed.
  7. Traffic passes back through GWLBE2 to the application VPC (catagramVPC).
  8. Local routing directs traffic from GWLBE2 to the ALB.
  9. ALB distributes traffic to the appropriate application instance.
  10. The return path follows the same logic, maintaining traffic integrity and inspection throughout.

Serverless and Application Services

Architecture Deep Dive Concepts

CatTube (Example App)
Monolithic Architecture
Tiered Architecture
Asynchronous Queues Architecture
Microservices Architecture
Event-Driven Architecture (EDA)

AWS Lambda Basics

AWS Lambda – Key Concepts
AWS Lambda – Architecture

Resource Configuration

AWS Lambda – Common Use Cases
Demo: Creating and Running a Lambda Function
  1. Deploy a CloudFormation stack to provision required resources (e.g. EC2 instances)
  2. Create an execution role with permissions (e.g. logging and EC2 control actions)
  3. Navigate to Lambda and create a new function
    • Assign a name and choose a runtime (e.g. Python 3.9)
    • Attach the execution role
  4. Add function code
    • Example: script to stop EC2 instances using environment variables

5. Configure environment variables

6. Run a test invocation

7. Optionally create another function (e.g. to start instances) and test similarly

8. Perform cleanup by deleting Lambda functions and the CloudFormation stack

9. Clean-up: remove the Lambda functions that were created, and then delete the CloudFormation stack to terminate all associated resources.

AWS Lambda Networking

Public Lambda (Default)
Private Lambda

Private Lambda ENI Injection – Old vs New Approach

Old approach:

New approach:

AWS Lambda Security, Monitoring, and Versioning

AWS Lambda – Security
  1. Lambda execution role
    • An IAM role assumed by the Lambda function during execution
    • The trust policy allows Lambda to assume the role
    • The permissions policy defines what the function is allowed or denied to do
      • Example: read data from DynamoDB and write it to S3
  2. Lambda resource policy
    • A resource-based policy attached to the Lambda function
    • Determines which principals are allowed to invoke the function
      • Can grant access to AWS services (e.g. S3, SNS) or external AWS accounts
AWS Lambda – Monitoring
AWS Lambda – Versioning and Aliases

AWS Lambda Invocation Methods

Lambda – Synchronous Invocation
Lambda – Asynchronous Invocation
Lambda – Event Source Mapping

AWS Lambda Execution Environment

AWS Lambda – Cold vs Warm Starts
AWS Lambda – Reducing Cold Start Latency

Amazon EventBridge – Serverless Event Bus Service

Amazon EventBridge – Overview and Architecture
CloudWatch Events (CWEvents)

DEMO: Building a Simple Event-Driven Architecture (EDA)

Protect an EC2 Instance if It Gets Stopped
  1. Create a Lambda function that automatically restarts any EC2 instance that enters the stopped state:
  1. Create an EventBridge rule to monitor EC2 state changes.
  2. Event to track: EC2 Instance State-change Notification.
  3. Generate a JSON sample of the event to see what information is delivered.

5. Fill out the event pattern to match instances entering the Stopped state.

6. Assign the Lambda function as the target for this EventBridge rule.

7. Test the setup by stopping an instance. The Lambda should automatically restart it after a short period.

8. Check logs in CloudWatch Logs for function execution details:

Stop All EC2 Instances at a Specific Time Every Day
  1. Create a schedule rule in EventBridge
    • Use the EventBridge Scheduler for a modern UI and flexible scheduling options.
    • Unlike the old method, you can define schedules outside of event buses.
    • Traditional “Create Rule” only allowed Unix CRON format and required UTC time.
  1. Specify the schedule using a Unix CRON expression
    • Check the time zone carefully:
      • CRON expressions default to UTC
      • EventBridge Scheduler UI may display times in your local time zone
      • Verify next trigger times to ensure the schedule is correct
  1. Assign the Lambda function that stops EC2 instances as the target for this scheduled rule.
  1. Wait for the scheduled time
    • The Lambda function will automatically stop the instances at the specified time.
    • If the EC2 protection Lambda is still active, any protected instances will automatically restart after being stopped.