AWS Database Services – quizzardexampreparation.com

Network Storage and Data Lifecycle

Amazon EFS (Elastic File System)

Amazon EFS (Elastic File System) – Architecture

EFS is AWS’s implementation of the NFSv4 protocol, used for shared file storage.
Key characteristics:
- Provides a network-based file system that can be mounted by one or more Linux EC2 instances.
- Enables storage to be decoupled from compute, allowing instances to remain stateless.
- Supports concurrent access from multiple instances.
Access model:
- EFS is a private service, accessed via mount targets within VPC subnets.
- Can be extended to on-premises environments using VPN, Direct Connect, or VPC peering.
- For high availability, mount targets should be deployed across multiple Availability Zones.
Data persistence:
- Files are stored independently of EC2 instances.
- Data remains intact even if instances are terminated and recreated.
Compatibility:
- Supported only on Linux-based systems.
- For Windows environments, use Amazon FSx for Windows File Server instead.
File system structure:
- Uses a hierarchical directory model typical of Linux systems.
- Can be mounted to paths such as /mnt/data or /nfs/media.
Permissions:
- Uses POSIX permissions, ensuring compatibility across Linux distributions.

EFS Performance Settings

Throughput Modes

Bursting (Default)
- Throughput increases as the file system grows in size.
- Uses a burst credit model similar to GP2 EBS volumes.
Enhanced Throughput Modes
- Elastic Throughput
  - Automatically adjusts based on workload demand.
  - Suitable for unpredictable or variable access patterns.
- Provisioned Throughput
  - Allows specifying throughput independently of storage size.
  - Useful for consistent, high-performance requirements.

Performance Modes

General Purpose (Default)
- Optimized for low latency.
- Suitable for web applications, CMS platforms, and general file sharing.
Max I/O
- Designed for high throughput and parallel workloads.
- Trades lower latency for increased scalability.
- Common in analytics or large-scale processing workloads.

EFS Storage Settings

Storage Classes

Standard
- Intended for frequently accessed data.
Infrequent Access (IA)
- Lower cost for less frequently used files.
Archive
- Lowest-cost tier for rarely accessed data.

Lifecycle policies can automatically transition files between storage classes.
Files can return to the Standard tier when accessed.

Additional EFS Features

Availability options:
- Data can be stored across multiple AZs or confined to a single AZ.
Backup integration:
- Supports automated backups through AWS Backup.
Encryption:
- Supports encryption using AWS KMS.
- Access requires permissions for both EFS and the associated KMS key.

DEMO: Using EFS with EC2 WordPress Instances

Demo: Implementing a Basic EFS

In this demo, an EFS file system is created within a custom VPC.
- Use the same configuration settings demonstrated in the reference material.
Best practice:
- Deploy a mount target in each Availability Zone where EFS will be accessed.
Mount targets and security groups:

Each mount target must have an associated security group.
In this setup, all mount targets share the same security group, allowing connectivity with other resources.

Configuring an EC2 Instance to Use EFS

The following commands are executed on a Linux EC2 instance:

df -k
- Displays all currently mounted file systems and available disk space.
- Initially confirms that no EFS file system is mounted.
sudo mkdir -p /efs/wp-content
- Creates the directory structure for mounting EFS.
- The -p option ensures intermediate directories are created if needed.
sudo dnf -y install amazon-efs-utils
- Installs required utilities for interacting with EFS.
- The -y flag automatically approves installation prompts.
sudo nano /etc/fstab
- Opens the file that defines file systems to mount at boot.
- Add the following entry at the end: <file-system-id>:/ /efs/wp-content efs _netdev,tls,iam 0 0
- Replace <file-system-id> with the actual EFS ID.
sudo mount /efs/wp-content
- Mounts the file system based on the fstab configuration.
df -k
- Verifies that the EFS file system is now mounted.

After mounting:
- Files created in /efs/wp-content are stored in EFS.
- Other EC2 instances connected to the same EFS can access the same data.
- This enables a shared network file system across instances.

WordPress Architecture with EFS

This design shifts from a monolithic setup to a scalable and resilient architecture:
Key changes:
- WordPress media is no longer stored on the instance’s local file system.
- Data is centralized:
  - Database → stored in RDS
  - Media/content → stored in EFS
Benefits:
- Multiple EC2 instances can access the same shared content.
- Supports horizontal scaling, allowing the number of instances to grow or shrink based on demand.
Mount location:
- EFS is mounted at: /var/www/html/wp-content
- This directory stores WordPress uploads and media.
CloudFormation note:
- !ImportValue is used to reference values exported from other stacks.

AWS Backup 101

AWS Backup – Basic Concepts

AWS Backup is a fully managed service for data protection, supporting both backup and restore operations.
- Includes centralized management, monitoring, and auditing features.
Provides a single interface to manage backups across environments:
- Supports cross-region backups for improved resilience.
- Supports cross-account backups, often integrated with AWS Organizations or Control Tower.
Supports a wide range of AWS services, allowing both:
- Backup storage management
- Backup policy configuration

AWS Backup – Key Components

Backup Plans

Define how backups are created and managed:
- Frequency
  - Determines how often backups run (e.g., hourly, daily, weekly).
  - Can use CRON expressions.
  - Some services support continuous backups for Point-In-Time Recovery (PITR).
- Backup Window
  - Specifies when backups start and how long they can run.
- Lifecycle Rules
  - Control when backups transition to cold storage or expire.
  - Backups moved to cold storage must remain there for at least 90 days.
- Backup Vault
  - Defines where backups are stored.
- Cross-Region Copy
  - Allows automatic replication of backups to another region.

Backup Resources

Identify the AWS resources included in a backup plan.
- Examples: S3 buckets, RDS databases, EBS volumes.

Backup Vault

Acts as the storage container for backups.
Key characteristics:
- At least one vault is required.
- Encrypted using a KMS key.
- By default, backups can be modified or deleted.
Vault Lock (WORM protection):
- Enables Write-Once-Read-Many behavior for compliance use cases.
- Includes a 72-hour grace period before becoming fully enforced.
- After activation:
  - Backups cannot be deleted or altered before retention expires.
  - Even AWS cannot override the lock.
Note:
- Vault Lock is separate from S3 Object Lock and Glacier storage features.

On-Demand Backups

Allow manual backup creation outside of scheduled plans.
Useful for ad hoc or pre-change backups.

Point-In-Time Recovery (PITR)

Available for supported services such as RDS and S3.
Enables restoring data to a specific moment within the retention period.

Key Takeaways

Centralizes backup management across accounts and regions.
Supports both scheduled and manual backups.
Provides compliance features like Vault Lock.
Enables fine-grained recovery through PITR.

SQL Databases & RDS

Database Models

Databases – Basic Concepts

A database (DB) is a system designed to store, organize, and process data.
It is important to distinguish databases from storage systems:
- Storage holds raw data such as files, images, or videos without built-in processing.
- Databases add structure and enable operations like querying, sorting, and analysis through query languages.
- Think of storage as a collection of raw files, while a database resembles a structured dataset that can be searched and analyzed.
There are many database types, differing in:
- How data is stored on disk
- How it is managed in memory
- How it is retrieved and presented
Broad classification:
- Relational (SQL) databases
- Non-relational (NoSQL) databases

Core Concepts

Schema
- Defines the structure of data, including attributes, data types, and relationships.
- Typically fixed in advance and difficult to modify later.
Key / Index
- A unique identifier for each record in a dataset.
- Ensures that records can be efficiently located.
Composite Key
- A combination of multiple attributes used together to uniquely identify a record.

Relational Databases (SQL / RDBMS)

SQL (Structured Query Language) is used to query relational databases.
A Relational Database Management System (RDBMS) organizes data into structured tables:
- Tables group related data.
- Rows represent individual records.
- Columns define attributes.
Key characteristics:
- All rows must follow the same schema.
- Each row is uniquely identified by a primary key.
- Relationships between tables are defined using join tables.
Relationship types include:
- One-to-one
- One-to-many
- Many-to-many
Limitation:
- Schema rigidity makes it difficult to adapt to rapidly changing or highly dynamic data structures.

SQL Databases: Row vs Column Design

Row-Based Databases (OLTP)

Data is stored row by row.
Optimized for transactional operations (insert, update, delete).
Strengths:
- Efficient for handling complete records.
- Ideal for applications with frequent transactions.
Weakness:
- Less efficient for operations focused on specific columns.
Common use cases:
- Transaction systems (orders, accounts, inventory).

Column-Based Databases (OLAP)

Data is stored column by column.
Optimized for analytical queries across large datasets.
Strengths:
- Efficient for aggregations and reporting.
- Enables advanced analytics and insights.
Weakness:
- Not suited for frequent transactional updates.
Common use cases:
- Data warehousing and business intelligence
- Example: Amazon Redshift

Non-Relational (NoSQL) Databases

Include database models that do not follow the relational structure.
Typically provide flexible or minimal schemas.
Designed for scalability and handling diverse data formats.

Key-Value Stores

Store data as simple key-value pairs.
Keys are unique; values are not interpreted by the database.
Strengths:
- High performance and scalability
- Simple data model
Use cases:
- Caching
- Session storage

Wide Column Stores

Extend the key-value model with multiple attributes per item.
Use a consistent key structure but allow flexible attributes.
Strengths:
- Highly scalable
- Suitable for large-scale applications
Example: Amazon DynamoDB

Document Databases

Store data as structured documents (e.g., JSON).
Documents can have varying structures within the same database.
Strengths:
- Flexible schema
- Supports complex, nested data
- Powerful querying capabilities
Use cases:
- Content management
- User profiles and catalogs
Example: Amazon DocumentDB

Graph Databases

Represent data as nodes (entities) and edges (relationships).
Relationships are stored directly and can be efficiently queried.
Strengths:
- Excellent for relationship-heavy data
- Fast traversal of connected data
Use cases:
- Social networks
- Recommendation engines
- Organizational structures
Example: Amazon Neptune

Database Models – Summary

Model	Structure	Typical Use Cases
Relational (SQL)	Fixed schema, tables with rows and columns	Structured data with defined relationships
Row-based (OLTP)	Rows stored together	Transactional systems
Column-based (OLAP)	Columns stored together	Analytics and reporting
Key-value	Simple key-value pairs	Caching, simple lookups
Wide column	Flexible attributes with structured keys	Large-scale applications
Document	JSON/XML documents	CMS, user data, catalogs
Graph	Nodes and relationships	Relationship-driven systems

Databases: ACID vs BASE

CAP Theorem

CAP = Consistency, Availability, Partition tolerance → properties of distributed DB systems.

Consistency (C)
- Every read returns the most recent write, or an error if that’s not possible.
- If you write a new value and read immediately, you either get the new value or an error.
Availability (A)
- Every request receives a response, but it may not be the latest data.
Partition Tolerance (P)
- DB continues to operate even if network partitions or node failures occur.
- Messages between nodes may fail, but the system keeps running.

CAP Theorem: A distributed DB cannot guarantee all three properties simultaneously — it must choose two.

Trade-off Example:

If a network partition occurs:
1. Choose Consistency + Partition Tolerance (CP):
  - Reject some reads → ensures latest data is returned → availability decreases
2. Choose Availability + Partition Tolerance (AP):
  - Accept all reads → may return stale data → consistency decreases
ACID vs BASE → different transaction models that reflect CAP trade-offs:
- ACID → favors consistency
- BASE → favors availability

ACID Transaction Model (SQL / RDBMS)

ACID = Atomic, Consistent, Isolated, Durable
Most SQL DBs use ACID transactions for reliable operations (e.g., banking).

Atomic
- Transaction succeeds entirely or not at all.
- $10 transfer example: money must leave Account A and arrive in Account B.
Consistent
- DB moves from one valid state to another.
- Invalid states are never allowed.
Isolated
- Parallel transactions do not interfere; end result is as if transactions were sequential.
Durable
- Committed transactions persist even after crashes.
- Once the DB reports success, data is stored safely on non-volatile memory.

Limitation: ACID can limit scalability due to strict rules.
Example: financial systems, SQL DBs (RDS, Aurora).

BASE Transaction Model (NoSQL / DynamoDB)

BASE = Basically Available, Soft state, Eventually consistent
Used by highly scalable NoSQL DBs.

Basically Available
- R/W operations available as much as possible, without strict consistency guarantees.
Soft State
- Consistency is not enforced in DB; handled by the application.
- Data returned may not be the latest.
Eventually Consistent
- Given enough time, the DB will converge to the latest state.
- Immediate consistency is not guaranteed by default.

Many BASE DBs (like DynamoDB) offer optional ACID-style transactions for applications that require them.
BASE DBs are highly scalable and performant because the DB does not enforce strict consistency by default.
Example: Amazon DynamoDB (DDB)
- Supports eventually consistent reads (default)
- Supports strongly consistent reads if requested
- Supports ACID transactions when needed

Exam Power Notes (AWS)

ACID mentioned → assume SQL / RDBMS (RDS, Aurora)
BASE mentioned → assume NoSQL / DynamoDB
If NoSQL + ACID → may indicate DynamoDB transactions

Databases in EC2

Monolith vs DB-Split Architecture

Monolith Architecture:
- Web server + application + database all hosted on a single EC2 instance
- Simple setup, but tightly coupled components
DB-Split Architecture:

Application and database are separated into different components
Database typically hosted on:
- Another EC2 instance, or
- An AWS-managed database service (preferred)
Key Considerations:
- DB can be placed in a different Availability Zone (AZ) for resilience
- Introduces network dependency between app and DB
  - Cross-AZ communication may incur additional cost
Why split the architecture?
- Independent scaling (scale app and DB separately)
- Ability to use specialized AWS services (e.g., managed DBs)
- Better performance, resilience, and flexibility
Monolithic architectures are generally discouraged in modern AWS design

Why Databases on EC2 Are Bad Practice

In most cases, AWS-managed database services are preferred over self-hosted DBs on EC2

1. Administrative Overhead

Requires managing:
- OS patching
- DB software updates
- Compatibility with applications
Backup and disaster recovery must be manually configured
Replication setup requires manual effort and expertise

2. Single Availability Zone Limitation

EC2 instances are tied to one AZ
If the AZ fails → DB becomes unavailable
Requires:
- Regular backups
- Snapshot management (e.g., EBS + S3)
Adds operational complexity and risk

3. Fewer Features Compared to Managed Services

AWS-managed DBs provide:
- Built-in high availability
- Automated backups
- Read replicas
- Performance optimizations
EC2-hosted DBs miss out on these advanced capabilities

4. Poor Scalability and No Serverless Options

EC2 scaling is manual or limited
No native serverless DB option
Instance runs continuously → higher baseline cost

When a Database on EC2 Might Be Justified

These scenarios should be carefully validated:

1. OS-Level Access Required

Need full control of the underlying operating system

2. Advanced Database Customization

Requires root-level DB access for tuning
Often driven by vendor requirements, not true business needs

3. AWS Does Not Support the Requirement

Specific DB engine or version not available in AWS-managed services
Specialized OS + DB combination required
Custom replication or architecture not supported by AWS

4. Business Decision

Sometimes chosen due to organizational preference, even if not optimal

Key AWS Exam Takeaways

Default recommendation:
→ Use AWS-managed database services
Avoid:
→ Running databases on EC2 unless there is a clear, justified need
Architecture best practice:
→ Decouple application and database layers

Amazon RDS (Relational Database Service) 101

Amazon RDS – Key Concepts

What it is: AWS-managed DB server (RDS instance)
- Similar to on-premises DB servers, but AWS handles HW, OS, installation, and maintenance
- Pros:
  - No need to manage OS or DB installation
  - DB engine maintenance mostly automated
  - Easy integration with other AWS services
- Cons:
  - No OS-level or SSH access (except in RDS Custom)
Important distinction:
- RDS is DBServer-as-a-Service (DBServeraaS), not DBaaS
  - You pay for the DB server/instance, not just a single database
  - Can host multiple databases per instance
Supported DB Engines:
- Open-source: MySQL, MariaDB, PostgreSQL
- Commercial: Oracle, MS SQL Server (licensing fees may apply)
Amazon Aurora is separate
- AWS-designed DB engine compatible with some RDS engines
- Offers additional improvements and features

RDS Architecture

RDS instances run inside a VPC and are deployed in subnets
Access:
- Private subnets: via VPN, Direct Connect, VPC peering
- Public subnets: can have public IPs (discouraged for security)
- Accessed via DNS CNAMEs
DB Subnet Group:
- A list of VPC subnets for RDS deployment
- Can include subnets across multiple AZs
- Must be selected when launching any RDS instance
- Best practice: 1 subnet group per RDS deployment
Multi-AZ Mode:
- Deploys primary + standby in different AZs
- Each has dedicated EBS storage
- Synchronous replication from primary → standby
- Backups occur from standby (no performance impact)
Backups:
- Stored in AWS-managed S3
- S3 replicates data across multiple AZs
- Snapshots safe from AZ failures
Read Replicas (0+):
- Asynchronous replication
- Can be same or cross-region
- Use cases: scale read load, resilience, disaster recovery

RDS Billing Overview

RDS billing is resource-based, similar to EC2

Instance fee:
- Billed hourly (per-second granularity)
- Cost depends on instance size & type
Multi-AZ mode: extra cost (extra instance + storage)
Storage fee: per GB/month
- Storage type affects price (e.g., Provisioned IOPS more expensive)
Data transfer fee: per GB in/out of instance
- Free within the same region
Backups & snapshots: per GB/month
- Free backup storage = size of your DB storage
Licensing fee: if using commercial DB engines

DEMO: Migration of WordPress DB to a Different DB Tier

Migration of MariaDB to a Different EC2 Instance

Following instructions, WordPress (WP) installed in monolith instance, blog post with images created.
- Media in this WP blog post is stored in the local filesystem. When moving to a fully elastic architecture, media should be migrated to a shared filesystem since it should not reside inside EC2 instances.
Steps to migrate MariaDB from monolith instance to a different EC2 instance (IP: 10.16.59.228):
1. Create backup of source DB: mysqldump -u root -p a4lwordpress > a4lwordpress.sql
2. Restore backup to destination DB: mysql -h 10.16.59.228 -u a4lwordpress -p a4lwordpress < a4lwordpress.sql
3. Update WordPress configuration: sudo nano /var/www/html/wp-config.php Replace: /** MySQL hostname */
  define(‘DB_HOST’, ‘localhost’); With: /** MySQL hostname */
  define(‘DB_HOST’, ‘10.16.59.228’);
4. Stop MariaDB service on the original instance: sudo service mariadb stop

Migration of MariaDB to RDS (Free Tier)

Part 1: Creation of an RDS Instance

Create a DB Subnet Group before deploying an RDS instance:
- Can include subnets from different Availability Zones (AZs) in the VPC.
- Select subnets marked for the DB tier.
- CIDRs can be verified in the VPC service.
RDS creation wizard options:
- DB engine and version selection is critical. Some versions may not be compatible with Aurora.
- Templates: PROD, DEV/TEST, Free tier.
  - Free tier: only supports single-instance deployment, no Multi-AZ.
- Configure: DB instance identifier, storage, backups, etc.
- Security Group (SG) must be assigned to control inbound access.
- Optional: create an initial DB (default = none).
Once provisioned, every RDS instance receives an endpoint CNAME and port, which are used to connect from apps.

Part 2: Migration from EC2 DB Instance to RDS

Update RDS SG: Allow inbound access from the EC2 WordPress app instance by including its SG.
Migration steps:
- Create a backup / SQL dump from the EC2 DB.
- Restore the dump to the RDS instance using its endpoint CNAME.
- Update wp-config.php with the RDS endpoint.
- Stop the MariaDB service on the EC2 instance.
RDS deletion: You will be prompted to keep a final snapshot and backups.

RDS Multi-AZ Deployments

RDS Multi-AZ – Instance Deployment

Historically, the only mode of Multi-AZ deployment providing high availability (HA) is the RDS Multi-AZ Instance Deployment.
- A more modern architecture exists: RDS Multi-AZ Cluster Deployment.
Architecture: Primary instance + one standby instance
- Primary and standby are in different AZs of the same region.
- Primary data is synchronously replicated to the standby.
  - Replication is storage-level only, which is less efficient than cluster deployment and depends on the DB engine.
- DB operations are committed only after being registered on both primary and standby.
- Standby carries extra cost; no free tier.
DB access: All reads and writes are done via the DB CNAME pointing to the primary instance.
- Standby is never accessed directly.
Backups: Can be performed from the standby.
- Backups are stored in S3 and replicated across AZs.
- No performance impact on the primary instance.
Failover: Improves availability.
- Possible reasons: AZ outage, primary failover, manual failover, instance change, software patching.
- RDS automatically updates the DB CNAME to point to the standby, which becomes the new primary.
- Failover duration: 60–120 seconds.
- Clearing in-app DNS cache can reduce downtime.
Standby improves availability and allows backups without affecting primary performance, but does not provide read scaling, since all DB access goes through the primary.

RDS Multi-AZ – Cluster Deployment

Architecture: One Writer instance + two Reader instances, all in different AZs of the same region.
- Provides higher HA than Multi-AZ Instance deployment.
- Synchronous replication from Writer to Readers.
- DB operations are committed once the Writer and at least one Reader confirm the transaction, ensuring data resilience across AZs.
Usage:
- Writer handles Reads and Writes.
- Readers handle Reads only, allowing read scaling.
  - Applications must be aware of separate roles for Writer and Readers.
- More instances → higher cost than Multi-AZ Instance deployment.
Important:
- Do not confuse with Aurora clusters. Aurora supports more than two Reader instances and has shared storage across instances.
- Each RDS instance has local EBS storage; cluster replication is at the instance level.
Endpoints:
- Cluster endpoint: Points to Writer, supports Reads, Writes, and admin operations.
- Reader endpoint: Routes Reads to an available Reader. In some cases, may direct to Writer.
- Instance endpoints: Point to specific instances; not recommended for general use since they do not tolerate failures.
Transaction logs:
- Record all transactions.
- Allow efficient replication and faster failover (~35 seconds + transaction log application).
Other advantages over Multi-AZ Instance deployment:
- Faster hardware (Graviton architecture + NVMe SSD).
- Fast writes to local storage, then flushed to EBS.
- Combines high performance with resilience benefits of EBS.

RDS Snapshots, Automatic Backups, and Restore

Backing Up Data in RDS

RDS data can be backed up to S3 via EBS snapshots
- Options:
  1. Manual Snapshots
  2. Automated Backups
- Data is stored in AWS-managed S3 buckets
  - Buckets are visible in RDS console UI, but not in the S3 console UI.
  - Benefits: data replicated across multiple AZs → improved resiliency.
- Snapshots/backups use EBS snapshots under the hood
  - Snapshots cannot be seen from EBS console, only from RDS console.
I/O pause during backup
- PROD: Usually Multi-AZ enabled; backups from standby → no read performance impact.
  - Write performance can pause briefly while replication occurs to standby.
  - Reads are generally higher in volume than writes, so overall DB performance is usually unaffected.
- DEV/TEST: Single-AZ → backup causes I/O pause because there is only one instance.
Incremental snapshot/backup architecture
- First snapshot is full, subsequent snapshots are incremental.
- Incremental snapshots contain only changes since last snapshot → faster than first snapshot.
- Deleting a snapshot in the chain does not break functionality.

RDS Manual Snapshots

Run manually or via scripts/applications.
Do not expire automatically; live beyond RDS instance lifecycle.
- Must be manually deleted to avoid storage costs.
Customer decides frequency (hourly, daily, weekly, etc.) → affects RPO.
When deleting an RDS instance, prompted to create a final snapshot to preserve data.

RDS Automated Backups

Automated snapshots triggered daily within a defined backup window.
- For Single-AZ instances, schedule during low-traffic periods due to I/O pause.
Expire automatically after retention period (0–35 days).
- 0 days → automated backups disabled.
- Backups expire even if the RDS instance is deleted.
  - Preserve data by creating a final manual snapshot before deletion.
Transaction logs uploaded to S3 every 5 minutes → allows point-in-time restore within retention period.
- Typical RPO: 5 minutes.
- Restore can be done to any second if transaction logs exist for that point.
Restoring requires applying transaction logs → increases restoration time (RTO longer than manual snapshots).
Cross-Region Replication (CRR) is optional → replicates backups and transaction logs to another region; charges apply.

Restoring Data in RDS

Restoring creates a new RDS instance → new IP, DB CNAME, and endpoint.
- Applications must update to the new endpoint.
Manual snapshots: Restore to the snapshot creation time → single point in time.
Automated backups: Restore to any 5-minute point in time within retention period → great for data corruption recovery.
Restores can be slow, especially for large databases.
- Read replicas can improve RTO significantly.

RDS Read-Replicas (RRs)

RDS Read-Replicas (RRs) – Architecture

RDS Read-Replica (RR) = a read-only replica of an RDS instance
- Can be used for reads (unlike standby replica in Multi-AZ instance) → allows read performance scaling
- An instance can have up to 5 direct RRs
  - Can be in the same region or a different region (cross-region RRs)
RRs are separate from the main RDS architecture
- Each RR has its own endpoint, independent from the main RDS instance endpoints
  - Requires application support: apps need to be configured to use a RR
  - By default, applications do not know about RRs
- No automatic failover
Asynchronous replication
- Data is committed on the main instance first, then replicated to its RRs
- Lag can occur depending on network conditions and write volume
  - RRs can have their own RRs → lag increases further
Cross-region RRs improve global read performance
- Users can read from the closest region efficiently
- Network handled transparently by AWS, data encrypted in transit
Multi-AZ cluster deployment combines Multi-AZ instance deployment + RRs
- 2 Reader instances in Multi-AZ cluster deployment are part of the main architecture
- Key exam point:
  - Synchronous replication → Multi-AZ
  - Asynchronous replication → RRs (excluding Aurora)

Promotion of RDS Read-Replicas & Disaster Recovery

RRs are read-only until promoted
- Once promoted, they become a normal RDS instance
- Promotion is quick → provides low RTO
- Improves global availability and resilience
  - RR in a different region can be promoted if the main region fails
- Offers near-zero RPO
  - Data is continuously synced from the main DB, minimal potential for data loss
- Ideal for quick recovery from failure (not data corruption)
Important caution: Do not use RRs for data corruption recovery
- Since replication is constant, corrupted data is also replicated
- In case of data corruption, rely on manual snapshots and automated backups
  - Higher frequency or higher quality snapshots/backups improves RPO

DEMO: RDS Multi-AZ & Snapshot Restore

When taking a snapshot, it is a good idea to include the DB engine and DB version in the snapshot name. Multi-AZ deployment and creation of a standby replica are recommended for PROD usage:

The primary instance is left untouched for backups.
Provides resilience in case of a primary AZ outage.

Backup windows are defined when creating the instance and can be adjusted later in the instance settings. When creating a standby replica (Multi-AZ), it can be created immediately or in the next backup window:

Creating it immediately may cause temporary outages.
Standby replica creation and synchronous replication setup will take some time.

An RDS instance can be rebooted with failover if a standby replica is configured:

This simulates a primary AZ outage.
Failover will take 60–120 seconds (for Multi-AZ instance deployment).

After restoring a snapshot, the new instance will take some time to become available.

Once available, the application (e.g., EC2 instance) must update its configuration to point to the new RDS instance.

RDS Data Security

RDS Encryption

RDS Encryption in Transit

RDS data is encrypted while being transferred between RDS and clients.
SSL/TLS is available and can be set to mandatory on a per-user basis.

RDS Encryption at Rest

RDS data is encrypted when written to disk.
Two options:

EBS Volume Encryption with KMS
- Default option, handled by the host and underlying EBS storage.
- DEKs generated from AWS-managed or customer-managed KEKs/CMKs.
- DEKs are loaded on hosts as required for encryption/decryption operations.
- Cannot be removed once enabled.
- Transparent to the DB engine; DB sees unencrypted data.
Transparent Data Encryption (TDE)
- Native DB engine encryption (supported by RDS MSSQL and RDS Oracle).
- Data encrypted by DB engine before being written to disk.
- RDS Oracle can integrate with AWS CloudHSM, giving stronger key controls.
- CloudHSM is secured by the customer, keeping AWS out of the trust chain.

Encrypted RDS instances propagate encryption to replicas and snapshots using the same configuration and keys.

RDS Authentication and Authorization

Authentication: how users log in.
Authorization: how access is controlled inside RDS.

Local DB Users

RDS logins are normally handled via local DB users with username & password.
A local DB user is created when provisioning an RDS instance (e.g., admin).
Local DB users are not IAM users; they are controlled by the DB engine.

IAM Authentication for RDS

Allows IAM identities to access RDS without a password:
1. Local DB user is configured to use AWS authentication tokens.
2. IAM policy maps the IAM identity to a local DB user.
3. IAM identity can generate a short-lived DB auth token (valid 15 minutes).
IAM authentication only; authorization is still controlled internally by the DB engine.

RDS Custom

RDS Custom – Key Facts

Niche topic; surface-level understanding is usually sufficient.
Middle ground between RDS and self-managed DB on EC2:
- RDS: fully managed DB-as-a-Service
  - OS and DB engine access is limited.
  - Customers cannot see RDS instances in EC2, EBS volumes, or S3 snapshots.
- DB Engine on EC2: self-managed
  - Customer responsible for OS, engine, backups, and all overhead.
- RDS Custom: combines benefits of both approaches
  - Automation features of RDS
  - Access to OS and DB engine for advanced customization
Supported only for MSSQL and Oracle.
Runs fully within your AWS account
- RDS Custom instances are visible in EC2, EBS volumes, and S3 snapshots.
- ENIs are injected into your VPC, allowing network access.
- Can connect using SSH, RDP, or EC2 Session Manager.
Customizing RDS Custom instances
1. Pause DB automation to make changes without disruptions.
2. Resume full automation for normal production use.

Amazon Aurora 101

Amazon Aurora – Overview

DB engine designed by AWS
- Part of Amazon RDS, but distinct product with unique features and architecture.
Two compute paradigms:
1. Aurora Provisioned – customer deploys cluster of instances.
2. Aurora Serverless – capacity managed by AWS, customer sets ACUs.
Storage: shared cluster volume → high performance, improved availability.

Aurora Provisioned Architecture

Cluster (DB Instances)

Aurora Cluster = 1 primary + 0–15 read replicas
- Instances can be in same or different AZs.
- Replicas used for reads → read scalability.
- Replicas can replace primary → high availability.
Better than RDS Multi-AZ cluster deployments:
- RDS Multi-AZ only supports 2 read replicas natively.

Storage (Cluster Volume)

No local storage – all instances share a cluster volume.
Shared storage → fast provisioning & failover.
SSD-based → high IOPS, low latency.
Max size: 128 TiB.
6 storage nodes across AZs → improved availability and resilience.
- Writes from primary synchronously replicated to all storage nodes.
- Replication happens at storage level → minimal performance impact.
- Automatic detection and repair of storage failures.

Access (Endpoints)

Cluster endpoint: always points to primary (read, write, admin).
Reader endpoint: load balances reads across available instances (primary or replicas).
Instance endpoints: unique per instance, useful for testing/diagnosis.
Custom endpoints: user-defined.

Aurora Billing (Provisioned)

No free-tier (micro instances not supported).
Compute: charged per hour, per second (10-minute minimum).
Storage: billed based on consumed data.
- Historically used High Water Mark (HWM): billed for historic max consumed storage.
- IO requests may incur small additional cost.
Backups: up to 100% of DB size included for storage at no extra cost.

Aurora Features

Backups & restores work like RDS.
- Restores create a new cluster → apps must update endpoints.
Backtrack: in-place rollback to a previous point in time without restoring from backup.
- Configured per-cluster, reduces downtime during data corruption.
Fast clones: copy-on-write storage.
- Only differences stored → fast, space-efficient cloning.

Aurora Global Database

Aurora Global DB – Overview

Purpose: Global-level replication of an Aurora Provisioned cluster.
Primary region:
- 1 read/write (R/W) instance + 0–15 read-only (R/O) replicas.
- Only region that accepts writes.
Secondary regions:
- 1–16 read-only replicas.
- All instances are read-only during normal operations.
- Can be promoted to primary for disaster recovery.

Replication Architecture

Storage-level replication → no extra CPU required → no performance impact on primary.
Replication latency: ~1 second → very low RPO and RTO.
Direction: one-way only (primary → secondary).

Benefits

Cross-region disaster recovery & business continuity
- Failover possible if primary region fails.
- Very low downtime (low RTO) and minimal data loss (low RPO).
Global read scaling
- Secondary regions serve read traffic → reduces latency internationally.
- Ideal for applications with global users needing fast reads.

Aurora Multi-Master Writes

Aurora Single-Master Mode (Default)

Architecture: 1 R/W instance + 0–15 R/O replicas.
Endpoints:
- Cluster endpoint → writes
- Reader endpoint → load-balanced reads
Failover:
- A replica is promoted to R/W on failure.
- Not instant → cluster endpoint updated → brief downtime.
Use case: Standard Aurora Provisioned clusters; good for most applications that can tolerate brief failover delays.

Key point: Single-Master clusters have a single write target; reads can be scaled via replicas.

Aurora Multi-Master Mode

Architecture: 1–16 R/W instances (all instances can write).
Endpoints:
- No cluster or reader endpoints → app must connect directly to instances.
Writes:
- Approved by all storage nodes (quorum)
- If a node rejects → write fails
- Once committed, writes are replicated to all nodes and in-memory caches → reads are consistent
Failover:

Handled by the app
Connections maintained to multiple instances → instant failover
Can support FT apps if app is properly configured
Benefits vs Single-Master:
- Better availability → apps can write/read from any instance
- Instant failover → minimal disruption
Challenges:
- App must handle load balancing for both reads and writes
- App must implement automated failover

Key point: Multi-Master allows multiple writers simultaneously, enabling high availability and potential FT applications, but app complexity increases.

Quick comparison table:

Feature	Single-Master	Multi-Master
Write Instances	1	1–16
Read Scaling	Reader replicas	App handles reads
Failover	Replica promoted, cluster endpoint updated	Instant via app, no promotion needed
App Complexity	Low	High (must manage reads/writes & failover)
Use Case	Standard HA	High availability, FT applications

Aurora Serverless

Aurora Serverless – Key Concepts

Serverless offering, conceptually comparable to AWS Fargate for ECS
Aurora Provisioned → users allocate database instances with fixed sizes and are responsible for managing them
Aurora Serverless → eliminates the need to pre-provision or handle database instances
- Reduces operational management effort
- More aligned with a Database-as-a-Service (DBaaS) model
  - Reminder: RDS and Aurora Provisioned operate as DB server–as-a-service
- Pricing is based on consumption, billed per second
Aurora Serverless clusters operate using Aurora Capacity Units (ACUs)
- Each ACU corresponds to a defined amount of compute power and memory
- Users configure minimum and maximum ACU limits for the cluster
  - The system automatically adjusts capacity within this range depending on workload demand
Aurora Serverless v1 has been retired; Serverless v2 is now the supported version
- Serverless v2 introduces several enhancements over v1:
  - Fine-grained and continuous scaling in increments of 0.5 ACUs, with near-instant responsiveness
  - Always active; unlike v1, it does not pause during inactivity, removing cold start delays
  - Supports Multi-AZ configurations for high availability and disaster recovery
  - Enables read replicas and global database capabilities
  - Improved handling of large numbers of concurrent connections through connection pooling and bursting

Aurora Serverless v1 – Architecture

This section needs updating for Serverless v2, as the original material focuses only on v1
Uses the same storage architecture as provisioned Aurora clusters
- Maintains the same durability (6 storage nodes distributed across Availability Zones)
- However, storage is accessed through ACUs instead of provisioned instances
Serverless v1 relies on Aurora Cluster Units (ACUs) to interface with storage
- These should not be confused with Aurora Capacity Units, despite the shared abbreviation
Characteristics of Aurora Cluster Units:
- Drawn from a shared warm pool managed by AWS and used across multiple customers
- Stateless in nature
- Do not include local storage, allowing rapid allocation to clusters when needed
  - Once assigned, they connect to the cluster’s storage similarly to provisioned instances
The number and size of ACUs dynamically adjust to match workload demand
- If demand increases beyond current capacity, additional ACUs are added, up to the configured maximum
- If demand decreases, excess ACUs are removed, but never below the defined minimum
  - In v1, setting the minimum to zero allows the cluster to pause, resulting in charges only for storage
  - This behavior is not supported in Serverless v2, where a minimum capacity is always maintained
Client connections are routed through an AWS-managed proxy layer
- This process is transparent to users
- Applications connect to the proxy, which manages communication with the ACUs
- Enables seamless scaling without disrupting application connections
- Users only define minimum and maximum ACU values; connection handling is abstracted away
Billing model:
- Compute: charged based on the number of ACUs used at any given time
- Storage: billed similarly to Aurora Provisioned

Aurora Serverless – Benefits

Simpler management model with reduced administrative overhead for database capacity
Automatic and seamless scaling of compute and memory resources without interrupting connections
Cost efficiency through usage-based pricing

Aurora Serverless – Use Cases

Workloads with infrequent usage (e.g., low-traffic websites)
Applications with variable demand patterns, including occasional spikes
- Removes the need to allocate fixed capacity in advance
Unpredictable workloads where demand is difficult to estimate
New applications with unknown usage patterns
- Avoids the need to resize database instances later, which can cause disruptions
Development and testing environments
- In Serverless v1, databases could pause during inactivity, incurring only storage costs
- This capability is not available in Serverless v2

Multi-tenant applications with subscription-based billing models
- Increased database usage typically aligns with increased revenue
- Infrastructure scaling naturally matches business growth
Overall, Aurora Serverless provides a flexible solution suitable for a wide range of scenarios

Demo: Migrating WordPress DB to Aurora Serverless v1

Creating a Serverless cluster from a provisioned snapshot is considered a restore, not a migration
- Restoration is limited to compatible database engine versions
- Building a cluster from scratch requires more setup compared to restoring from a snapshot
- After restoring, ACU usage is typically high due to increased demand during initialization

In Serverless v1, compute capacity can pause after a period of inactivity
- This occurs regardless of the configured minimum ACUs
- When a paused cluster is accessed, it must resume before handling requests
  - Applications must tolerate longer connection times in this scenario

This behavior does not apply to Serverless v2
- A baseline level of compute capacity is always maintained
- As a result, there are no delays caused by resuming from a paused state

RDS Proxy

Database Proxies – Overview

Why use database proxies?
1. Establishing and terminating database connections consumes time and system resources
  - This impact is especially noticeable for smaller database operations
    - For minimal read/write actions, the connection setup often represents most of the total execution time
  - Also significant when there is a high volume of connections
    - For example, in serverless environments, each Lambda invocation may create and close a connection
    - With many concurrent executions, this results in a large number of connections, increasing latency and cost
    - Since Lambda billing is based on execution duration, this is inefficient
2. Managing database failures within application logic increases complexity and risk
  - Applications must handle retry timing, connection timeouts, and failover behavior, which adds overhead
A database proxy is positioned between applications and the database to address these challenges
- Architecture: Application(s) → DB Proxy (connection pooling) → Database
- The proxy maintains persistent database connections
- Applications connect to the proxy and reuse pooled connections
  - Enables multiplexing, where multiple application requests share fewer backend connections
Drawback: Operating and maintaining a database proxy (including scaling and fault tolerance) can be complex
- AWS RDS Proxy is a managed service that simplifies this responsibility

RDS Proxy – Architecture

RDS Proxy is a fully managed database proxy service for RDS and Aurora
- Provides built-in high availability and automatic scaling
- Reduces operational effort compared to self-managed proxy solutions
Accessible only within a VPC
- It cannot be directly reached from the public internet
Offers connection pooling, which lowers database load by:
- Reducing the overhead of repeatedly opening and closing connections
- Allowing multiplexing, so fewer connections are required between the proxy and the database
Handles database failures and failover without exposing complexity to applications
- Applications connect using a proxy endpoint
  - This interaction is transparent, so applications behave as if they are connecting directly to the database
- During failover, the proxy automatically redirects traffic to the new primary instance
  - This process happens in the background without requiring application changes
  - Applications can continue attempting connections even if the database is temporarily unavailable
- In Aurora environments, failover duration can be reduced by more than 60%
Supports enforcing SSL/TLS for secure connections

Good-Fit Scenarios for RDS Proxy

Situations with excessive database connections
- Common in smaller instance types such as T2 or T3
- Multiplexing reduces the number of active connections to the database
AWS Lambda workloads
1. Reusing persistent connections through the proxy reduces connection setup time
  - This lowers Lambda execution duration and associated costs
2. Lambda functions can use IAM roles for authentication, which can also be applied when connecting through RDS Proxy
Applications requiring long-lived connections, such as SaaS platforms, where minimizing latency is important
Environments where high resilience to database failures is required
- RDS Proxy helps minimize failover time and shields applications from underlying disruptions

RDS & Aurora – Summary Table

AWS Relational DB model	Architecture & AZ coverage	Storage	Scalability & Failover
RDS Single-AZ	Single database instance deployed in one Availability Zone	EBS	– No support for scaling

No automatic failover capability
RDS Multi-AZ instance | One primary instance with a standby replica located in another AZ (spans 2 AZs) | Each instance is backed by its own EBS volume | – No scaling supported
Automatic failover: standby is promoted to primary (typically 60–120 seconds)
RDS Multi-AZ cluster | – One writer instance and two read replicas
Distributed across 3 AZs | – Each instance uses its own EBS storage
Transaction logs are utilized
Local storage is periodically flushed to EBS for better performance | – Supports read scaling
Automatic failover: a reader can be promoted to writer (around 30 seconds plus time to apply transaction logs)
Aurora Provisioned | – One primary (read/write) instance with 0–15 read replicas
Spans multiple AZs depending on the number of replicas | – No local instance storage
Shared SSD-backed cluster volume across all instances
Storage replicated across 6 nodes in 3 AZs | – Enables read scaling (increases with more replicas)
Automatic failover: a replica takes over as primary very quickly due to shared storage
Aurora Serverless | – No fixed database instances are provisioned
Users define minimum and maximum capacity using ACUs
Compute capacity is accessed through a proxy layer | – Same storage approach as Aurora Provisioned
Cluster volume is accessed via ACUs instead of database instances | – Provides automatic and flexible scaling based on demand
No traditional failover model, as failover handling is abstracted away from clients

AWS Database Migration Service (DMS) & AWS Schema Conversion Tool (SCT)

Database Migrations

Database migration refers to transferring all data from a source database to a target database
- Data may retain the same structure, configuration, and schema, or it may require changes
- A key challenge is handling migrations between different database engines
This process is generally complex
- Often requires significant manual effort from start to finish
- Vendor-provided tools may assist in certain cases
- Typically involves replication setup and/or restoring from backups
  - A major consideration is how to manage ongoing data changes during the restore process

AWS Database Migration Service (DMS) – Architecture

AWS Database Migration Service (DMS) is a managed solution designed to simplify database migrations
- Supports most commonly used database engines such as MySQL, Aurora, Microsoft SQL Server, MariaDB, MongoDB, PostgreSQL, Oracle, and Azure SQL
- While it supports many migration scenarios:
  - Schema conversion requires the use of AWS Schema Conversion Tool (SCT)
  - Some limitations exist (for example, certain databases can only act as targets)
Runs on an EC2-based Replication Instance, with defined source and destination endpoints
1. The Replication Instance runs replication software (one or more tasks) and communicates with DMS
  - Replication tasks define how the migration is executed
2. Source and destination endpoints store connection details for both databases
  - At least one endpoint must reside within AWS
    - DMS cannot be used to migrate strictly between two on-premises databases
By default, data is transferred over the network (e.g., Direct Connect, VPN, VPC peering)
- For very large datasets, network transfer may be inefficient or costly
- In such cases, AWS Snowball devices can be used along with DMS and SCT for faster bulk transfer
Types of migration jobs:
1. Full Load
  - Performs a one-time transfer of all existing data
  - Requires database downtime during the migration process
  - Suitable only when downtime is acceptable
2. Full Load + Change Data Capture (CDC)
  - First performs a full data load
  - Simultaneously captures ongoing changes during migration
  - After the initial load, captured changes are applied to the destination
  - Eventually, both databases reach synchronization
  - Final cutover involves stopping applications briefly and redirecting them to the new database
    - Results in minimal downtime, often close to zero
3. CDC Only
  - Used when bulk data transfer is handled outside of DMS (e.g., native tools like Oracle import/export)
  - DMS is then used only to replicate ongoing changes
DMS does not natively handle schema conversion
- The AWS Schema Conversion Tool (SCT) is used alongside DMS when needed
Common use cases and benefits:
- Frequently used for large-scale database migrations
- Well-suited for migrating on-premises databases to AWS
- Enables migrations with minimal or no downtime
- In exam scenarios, DMS is typically the default choice when migration involves AWS and no special constraints are mentioned

AWS Schema Conversion Tool (SCT)

Used to convert or modify database schemas between different engines or versions
- Should not be used when migrating between compatible databases of the same engine
  - Example: on-premises MySQL to RDS MySQL
- Useful for migrations involving different database types
  - Example: SQL Server to MySQL, or Oracle to Aurora
- Also helpful in large migrations involving DMS and Snowball
Supports:
- OLTP databases such as MySQL, SQL Server, and Oracle
- OLAP databases such as Teradata, Oracle, Vertica, and Greenplum
Operates as a standalone tool independent of DMS
- Can be used outside AWS, including for migrations between on-premises databases

Large Database Migrations with DMS and AWS Snowball

Some migrations involve very large datasets (multi-terabyte scale)
- Transferring such volumes over the network can be slow and resource-intensive
AWS Snowball devices can be used for bulk data transfer into and out of AWS
1. Request a Snowball device from AWS
2. Use SCT to extract and store data locally onto the device
3. Ship the device back to AWS, where the data is uploaded to Amazon S3
4. DMS then transfers data from S3 to the target database
5. Optionally, CDC can capture ongoing changes and apply them to the destination
Although SCT is used in this workflow, it does not violate its typical usage rule
- In this case, SCT converts the database into a generic file format for storage on the Snowball device rather than performing schema transformation between engines